TY - GEN
T1 - Help, help, I'm being suppressed the significance of suppressors in software testing
AU - Groce, Alex
AU - Zhang, Chaoqiang
AU - Alipour, Mohammad Amin
AU - Eide, Eric
AU - Chen, Yang
AU - Regehr, John
PY - 2013
Y1 - 2013
N2 - Test features are basic compositional units used to describe what a test does (and does not) involve. For example, in API-based testing, the most obvious features are function calls; in grammar-based testing, the obvious features are the elements of the grammar. The relationship between features as abstractions of tests and produced behaviors of the tested program is surprisingly poorly understood. This paper shows how large-scale random testing modified to use diverse feature sets can uncover causal relationships between what a test contains and what the program being tested does. We introduce a general notion of observable behaviors as targets, where a target can be a detected fault, an executed branch or statement, or a complex coverage entity such as a state, predicate-valuation, or program path. While it is obvious that targets have triggers - features without which they cannot be hit by a test - the notion of suppressors - features which make a test less likely to hit a target - has received little attention despite having important implications for automated test generation and program understanding. For a set of subjects including C compilers, a flash file system, and JavaScript engines, we show that suppression is both common and important.
AB - Test features are basic compositional units used to describe what a test does (and does not) involve. For example, in API-based testing, the most obvious features are function calls; in grammar-based testing, the obvious features are the elements of the grammar. The relationship between features as abstractions of tests and produced behaviors of the tested program is surprisingly poorly understood. This paper shows how large-scale random testing modified to use diverse feature sets can uncover causal relationships between what a test contains and what the program being tested does. We introduce a general notion of observable behaviors as targets, where a target can be a detected fault, an executed branch or statement, or a complex coverage entity such as a state, predicate-valuation, or program path. While it is obvious that targets have triggers - features without which they cannot be hit by a test - the notion of suppressors - features which make a test less likely to hit a target - has received little attention despite having important implications for automated test generation and program understanding. For a set of subjects including C compilers, a flash file system, and JavaScript engines, we show that suppression is both common and important.
UR - http://www.scopus.com/inward/record.url?scp=84893326388&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=84893326388&partnerID=8YFLogxK
U2 - 10.1109/ISSRE.2013.6698892
DO - 10.1109/ISSRE.2013.6698892
M3 - Conference contribution
AN - SCOPUS:84893326388
SN - 9781479923663
T3 - 2013 IEEE 24th International Symposium on Software Reliability Engineering, ISSRE 2013
SP - 390
EP - 399
BT - 2013 IEEE 24th International Symposium on Software Reliability Engineering, ISSRE 2013
T2 - 2013 IEEE 24th International Symposium on Software Reliability Engineering, ISSRE 2013
Y2 - 4 November 2013 through 7 November 2013
ER -