Random test run length and effectiveness

James H. Andrews, Alex Grocer, Melissa Weston, Ru Gang Xu

Research output: Chapter in Book/Report/Conference proceedingConference contribution

43 Scopus citations

Abstract

A poorly understood but important factor in random testing is the selection of a maximum length for test runs. Given a limited time for testing, it is seldom clear whether executing a small number of long runs or a large number of short runs maximizes utility. It is generally expected that longer runs are more likely to expose failures -which is certainly true with respect to runs shorter than the shortest failing trace. However, longer runs produce longer failing traces, requiring more effort from humans in debugging or more resources for automated minimization. In testing with feedback, increasing ranges for parameters may also cause the probability of failure to decrease in longer runs. We show that the choice of test length dramatically impacts the effectiveness of random testing, and that the patterns observed in simple models and predicted by analysis are useful in understanding effects observed in a large scale case study of a JPL flight software system.

Original languageEnglish (US)
Title of host publicationASE 2008 - 23rd IEEE/ACM International Conference on Automated Software Engineering, Proceedings
Pages19-28
Number of pages10
DOIs
StatePublished - 2008
Externally publishedYes
EventASE 2008 - 23rd IEEE/ACM International Conference on Automated Software Engineering - L'Aquila, Italy
Duration: Sep 15 2008Sep 19 2008

Publication series

NameASE 2008 - 23rd IEEE/ACM International Conference on Automated Software Engineering, Proceedings

Conference

ConferenceASE 2008 - 23rd IEEE/ACM International Conference on Automated Software Engineering
Country/TerritoryItaly
CityL'Aquila
Period9/15/089/19/08

ASJC Scopus subject areas

  • Software

Fingerprint

Dive into the research topics of 'Random test run length and effectiveness'. Together they form a unique fingerprint.

Cite this