Mutant tests too: The SQL

Elliot West and Jay Green share approaches for applying software engineering best practices to SQL-based data applications to improve maintainability and data quality. Using open source tools, Elliot and Jay show how to build effective test suites for Apache Hive code bases and offer an overview of Mutant Swarm, a tool to identify weaknesses in tests and to measure SQL code coverage.

The big data platforms of many organizations are underpinned by a technology that will soon celebrate its 45th birthday: SQL. This industry stalwart is applied in a multitude of critical points in business data flows; the results that these processes generate may significantly influence business and financial decision making. However, the SQL ecosystem has been overlooked and ignored by more recent innovations in the field of software engineering best practices, such as fine-grained automated testing and code quality metrics. This exposes organizations to poor application maintainability, high bug rates, and ultimately corporate risk. Elliot West and Jay Green share approaches for applying software engineering best practices to SQL-based data applications to improve maintainability and data quality. They first define the relevance of such approaches and demonstrate how automated testing can be applied to Hive SQL using HiveRunner, a JUnit based testing framework. They then consider how best to structure Hive queries to yield meaningful test scenarios that are maintainable and performant. Elliot and Jay conclude by demonstrating how test coverage reports can highlight areas of risk in SQL code bases and weaknesses in the testing process and offer an overview of Mutant Swarm, an open source mutation testing tool for SQL languages developed by Hotels.com that can deliver insights similar to those produced by Java-focused tools such as Jacoco and PIT.