Coverage and its discontents

Alex Groce, Mohammad Amin Alipour, Rahul Gopinath

Research output: Chapter in Book/Report/Conference proceedingConference contribution

20 Scopus citations

Abstract

Everyone wants to know one thing about a test suite: will it detect enough bugs? Unfortunately, in most settings that matter, answering this question directly is impractical or impossible. Software engineers and researchers therefore tend to rely on various measures of code coverage (where mutation testing is considered a form of syntactic coverage). A long line of academic research efforts have attempted to determine whether relying on coverage as a substitute for fault detection is a reasonable solution to the problems of test suite evaluation. This essay argues that the profusion of coverage-related literature is in part a sign of an underlying uncertainty as to what exactly it is that measuring coverage should achieve, as well as how we would know if it can, in fact, achieve it. We propose some solutions and mitigations, but the primary focus of this essay is to clarify the state of current confusions regarding this key problem for effective software testing.

Original languageEnglish (US)
Title of host publicationOnward! 2014 - Proceedings of the 2014 ACM International Symposium on New Ideas, New Paradigms, and Reflections on Programming and Software, Part of SPLASH 2014
PublisherAssociation for Computing Machinery
Pages255-268
Number of pages14
ISBN (Electronic)9781450332101
DOIs
StatePublished - Oct 20 2014
Externally publishedYes
Event2014 ACM International Symposium on New Ideas, New Paradigms, and Reflections on Programming and Software, Onward! 2014 -Part of SPLASH 2014 - Portland, United States
Duration: Oct 20 2014Oct 24 2014

Publication series

NameOnward! 2014 - Proceedings of the 2014 ACM International Symposium on New Ideas, New Paradigms, and Reflections on Programming and Software, Part of SPLASH 2014

Conference

Conference2014 ACM International Symposium on New Ideas, New Paradigms, and Reflections on Programming and Software, Onward! 2014 -Part of SPLASH 2014
Country/TerritoryUnited States
CityPortland
Period10/20/1410/24/14

Keywords

  • Coverage
  • Evaluation
  • Testing

ASJC Scopus subject areas

  • Software
  • Computational Theory and Mathematics
  • Computer Networks and Communications
  • Computer Science Applications

Fingerprint

Dive into the research topics of 'Coverage and its discontents'. Together they form a unique fingerprint.

Cite this