TY - GEN
T1 - Cause reduction for quick testing
AU - Groce, Alex
AU - Alipour, Mohammed Amin
AU - Zhang, Chaoqiang
AU - Chen, Yang
AU - Regehr, John
PY - 2014
Y1 - 2014
N2 - In random testing, it is often desirable to produce a 'quick test' - an extremely inexpensive test suite that can serve as a frequently applied regression and allow the benefits of random testing to be obtained even in very slow or over-subscribed test environments. Delta debugging is an algorithm that, given a failing test case, produces a smaller test case that also fails, and typically executes much more quickly. Delta debugging of random tests can produce effective regression suites for previously detected faults, but such suites often have little power for detecting new faults, and in some cases provide poor code coverage. This paper proposes extending delta debugging by simplifying tests with respect to code coverage, an instance of a generalization of delta debugging we call cause reduction. We show that test suites reduced in this fashion can provide very effective quick tests for real-world programs. For Mozilla's Spider Monkey JavaScript engine, the reduced suite is more effective for finding software faults, even if its reduced runtime is not considered. The effectiveness of a reduction-based quick test persists through major changes to the software under test.
AB - In random testing, it is often desirable to produce a 'quick test' - an extremely inexpensive test suite that can serve as a frequently applied regression and allow the benefits of random testing to be obtained even in very slow or over-subscribed test environments. Delta debugging is an algorithm that, given a failing test case, produces a smaller test case that also fails, and typically executes much more quickly. Delta debugging of random tests can produce effective regression suites for previously detected faults, but such suites often have little power for detecting new faults, and in some cases provide poor code coverage. This paper proposes extending delta debugging by simplifying tests with respect to code coverage, an instance of a generalization of delta debugging we call cause reduction. We show that test suites reduced in this fashion can provide very effective quick tests for real-world programs. For Mozilla's Spider Monkey JavaScript engine, the reduced suite is more effective for finding software faults, even if its reduced runtime is not considered. The effectiveness of a reduction-based quick test persists through major changes to the software under test.
KW - random testing
KW - regression testing
KW - test case minimization
UR - http://www.scopus.com/inward/record.url?scp=84903120396&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=84903120396&partnerID=8YFLogxK
U2 - 10.1109/ICST.2014.37
DO - 10.1109/ICST.2014.37
M3 - Conference contribution
AN - SCOPUS:84903120396
SN - 9780769551852
T3 - Proceedings - IEEE 7th International Conference on Software Testing, Verification and Validation, ICST 2014
SP - 243
EP - 252
BT - Proceedings - IEEE 7th International Conference on Software Testing, Verification and Validation, ICST 2014
PB - IEEE Computer Society
T2 - 7th IEEE International Conference on Software Testing, Verification and Validation, ICST 2014
Y2 - 31 March 2014 through 4 April 2014
ER -