Mitigating (and exploiting) test reduction slippage

Josie Holmes, Alex Groce, Mohammad Amin Alipour

Research output: Chapter in Book/Report/Conference proceedingConference contribution

5 Scopus citations

Abstract

Reducing the size of tests, typically by delta debugging or a related algorithm, is a critical component of effective automated testing and debugging. Automatically generated or user-submitted tests are often far longer than required, full of unnecessary components that make debugging difficult. Test reduction algorithms automatically remove components of such tests, while preserving the property that the test fails. Unfortunately, reduction can sometimes transform a failing test that detects a subtle, critical, and previously unknown fault into a test that detects a trivial-to-fine, unimportant, and already known fault. When reducing a test detecting fault(s) F produces a test that does not detect the same F, this is known as slippage. In the case where an interesting fault slips to an uninteresting fault, slippage is a problem, and must be avoided. However, slippage can also be beneficial, when a long test can be reduced to detect a fault that has not otherwise been detected (including by the original test). While traditional delta debugging only produces one reduced test, the concept of slippage suggests an alternative approach, where the output of reduction is a set of reduced tests, in order to avoid problematic slippage and induce beneficial slippage. In this paper, we present preliminary efforts to understand slippage, and compare two approaches to slippage mitigation.

Original languageEnglish (US)
Title of host publicationA-TEST 2016 - Proceedings of the 7th International Workshop on Automating Test Case Design, Selection, and Evaluation, co-located with FSE 2016
EditorsSigrid Eldh, Wishnu Prasetya, Tanja Vos
PublisherAssociation for Computing Machinery, Inc
Pages66-69
Number of pages4
ISBN (Electronic)9781450344012
DOIs
StatePublished - Nov 18 2016
Externally publishedYes
Event7th International Workshop on Automating Test Case Design, Selection, and Evaluation, A-TEST 2016 - Seattle, United States
Duration: Nov 18 2016 → …

Publication series

NameA-TEST 2016 - Proceedings of the 7th International Workshop on Automating Test Case Design, Selection, and Evaluation, co-located with FSE 2016

Conference

Conference7th International Workshop on Automating Test Case Design, Selection, and Evaluation, A-TEST 2016
Country/TerritoryUnited States
CitySeattle
Period11/18/16 → …

Keywords

  • Delta debugging
  • Slippage
  • Test manipulation and reduction

ASJC Scopus subject areas

  • Software

Fingerprint

Dive into the research topics of 'Mitigating (and exploiting) test reduction slippage'. Together they form a unique fingerprint.

Cite this