Taming compiler fuzzers

Yang Chen, Alex Groce, Chaoqiang Zhang, Weng Keen Wong, Xiaoli Fern, Eric Eide, John Regehr

Research output: Chapter in Book/Report/Conference proceedingConference contribution

86 Scopus citations

Abstract

Aggressive random testing tools ("fuzzers") are impressively effective at finding compiler bugs. For example, a single test-case generator has resulted in more than 1,700 bugs reported for a single JavaScript engine. However, fuzzers can be frustrating to use: they indiscriminately and repeatedly find bugs that may not be severe enough to fix right away. Currently, users filter out undesirable test cases using ad hoc methods such as disallowing problematic features in tests and grepping test results. This paper formulates and addresses the fuzzer taming problem: given a potentially large number of random test cases that trigger failures, order them such that diverse, interesting test cases are highly ranked. Our evaluation shows our ability to solve the fuzzer taming problem for 3,799 test cases triggering 46 bugs in a C compiler and 2,603 test cases triggering 28 bugs in a JavaScript engine.

Original languageEnglish (US)
Title of host publicationPLDI 2013 - Proceedings of the 2013 ACM SIGPLAN Conference on Programming Language Design and Implementation
Pages197-207
Number of pages11
DOIs
StatePublished - 2013
Externally publishedYes
Event34th ACM SIGPLAN Conference on Programming Language Design and Implementation, PLDI 2013 - Seattle, WA, United States
Duration: Jun 16 2013Jun 19 2013

Publication series

NameProceedings of the ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI)

Conference

Conference34th ACM SIGPLAN Conference on Programming Language Design and Implementation, PLDI 2013
Country/TerritoryUnited States
CitySeattle, WA
Period6/16/136/19/13

Keywords

  • Automated testing
  • Bug reporting
  • Compiler defect
  • Compiler testing
  • Fuzz testing
  • Random testing
  • Test-case reduction

ASJC Scopus subject areas

  • Software

Fingerprint

Dive into the research topics of 'Taming compiler fuzzers'. Together they form a unique fingerprint.

Cite this