Swarm testing

Alex Groce, Chaoqiang Zhang, Eric Eide, Yang Chen, John Regehr

Research output: Chapter in Book/Report/Conference proceedingConference contribution

70 Scopus citations

Abstract

Swarm testing is a novel and inexpensive way to improve the diversity of test cases generated during random testing. Increased diversity leads to improved coverage and fault detection. In swarm testing, the usual practice of potentially including all features in every test case is abandoned. Rather, a large "swarm" of randomly generated configurations, each of which omits some features, is used, with configurations receiving equal resources. We have identified two mechanisms by which feature omission leads to better exploration of a system's state space. First, some features actively prevent the system from executing interesting behaviors; e.g., "pop" calls may prevent a stack data structure from executing a bug in its overflow detection logic. Second, even when there is no active suppression of behaviors, test features compete for space in each test, limiting the depth to which logic driven by features can be explored. Experimental results show that swarm testing increases coverage and can improve fault detection dramatically; for example, in a week of testing it found 42% more distinct ways to crash a collection of C compilers than did the heavily hand-tuned default configuration of a random tester.

Original languageEnglish (US)
Title of host publication2012 International Symposium on Software Testing and Analysis, ISSTA 2012 - Proceedings
Pages78-88
Number of pages11
DOIs
StatePublished - 2012
Externally publishedYes
Event21st International Symposium on Software Testing and Analysis, ISSTA 2012 - Minneapolis, MN, United States
Duration: Jul 15 2012Jul 20 2012

Publication series

Name2012 International Symposium on Software Testing and Analysis, ISSTA 2012 - Proceedings

Conference

Conference21st International Symposium on Software Testing and Analysis, ISSTA 2012
Country/TerritoryUnited States
CityMinneapolis, MN
Period7/15/127/20/12

Keywords

  • configuration diversity
  • Random testing

ASJC Scopus subject areas

  • Software

Fingerprint

Dive into the research topics of 'Swarm testing'. Together they form a unique fingerprint.

Cite this