Evaluating non-adequate test-case reduction

Mohammad Amin Alipour, August Shi, Rahul Gopinath, Darko Marinov, Alex Groce

Research output: Chapter in Book/Report/Conference proceedingConference contribution

19 Scopus citations

Abstract

Given two test cases, one larger and one smaller, the smaller test case is preferred for many purposes. A smaller test case usually runs faster, is easier to understand, and is more convenient for debugging. However, smaller test cases also tend to cover less code and detect fewer faults than larger test cases. Whereas traditional research focused on reducing test suites while preserving code coverage, recent work has introduced the idea of reducing individual test cases, rather than test suites, while still preserving code coverage. Other recent work has proposed non-adequately reducing test suites by not even preserving all the code coverage. This paper empirically evaluates a new combination of these two ideas, non-adequate reduction of test cases, which allows for a wide range of trade-offs between test case size and fault detection. Our study introduces and evaluates C%-coverage reduction (where a test case is reduced to retain at least C% of its original coverage) and N-mutant reduction (where a test case is reduced to kill at least N of the mutants it originally killed). We evaluate the reduction trade-offs with varying values of C% and N for four real-world C projects: Mozilla's SpiderMonkey JavaScript engine, the YAFFS2 ash file system, Grep, and Gzip. The results show that it is possible to greatly reduce the size of many test cases while still preserving much of their fault-detection capability.

Original languageEnglish (US)
Title of host publicationASE 2016 - Proceedings of the 31st IEEE/ACM International Conference on Automated Software Engineering
EditorsSarfraz Khurshid, David Lo, Sven Apel
PublisherAssociation for Computing Machinery, Inc
Pages16-26
Number of pages11
ISBN (Electronic)9781450338455
DOIs
StatePublished - Aug 25 2016
Externally publishedYes
Event31st IEEE/ACM International Conference on Automated Software Engineering, ASE 2016 - Singapore, Singapore
Duration: Sep 3 2016Sep 7 2016

Publication series

NameASE 2016 - Proceedings of the 31st IEEE/ACM International Conference on Automated Software Engineering

Conference

Conference31st IEEE/ACM International Conference on Automated Software Engineering, ASE 2016
Country/TerritorySingapore
CitySingapore
Period9/3/169/7/16

Keywords

  • Coverage
  • Mutation testing
  • Test adequacy
  • Test reduction

ASJC Scopus subject areas

  • Software
  • Computational Theory and Mathematics
  • Human-Computer Interaction

Fingerprint

Dive into the research topics of 'Evaluating non-adequate test-case reduction'. Together they form a unique fingerprint.

Cite this