TY - GEN
T1 - ACM Conferences
AU - Groce, Alex
AU - Van Tonder, Rijnard
AU - Kalburgi, Goutamkumar Tulajappa
AU - Le Goues, Claire
N1 - Publisher Copyright:
© 2022 ACM.
PY - 2022/9/19
Y1 - 2022/9/19
N2 - Developing a bug-free compiler is difficult; modern optimizing compilers are among the most complex software systems humans build. Fuzzing is one way to identify subtle compiler bugs that are hard to find with human-constructed tests. Grammar-based fuzzing, however, requires a grammar for a compiler's input language, and can miss bugs induced by code that does not actually satisfy the grammar the compiler should accept. Grammar-based fuzzing also seldom uses advanced modern fuzzing techniques based on coverage feedback. However, modern mutation-based fuzzers are often ineffective for testing compilers because most inputs they generate do not even come close to getting past the parsing stage of compilation. This paper introduces a technique for taking a modern mutation-based fuzzer (AFL in our case, but the method is general) and augmenting it with operators taken from mutation testing, and program splicing. We conduct a controlled study to show that our hybrid approaches significantly improve fuzzing effectiveness qualitatively (consistently finding unique bugs that baseline approaches do not) and quantitatively (typically finding more unique bugs in the same time span, despite fewer program executions). Our easy-To-Apply approach has allowed us to report more than 100 confirmed and fixed bugs in production compilers, and found a bug in the Solidity compiler that earned a security bounty.
AB - Developing a bug-free compiler is difficult; modern optimizing compilers are among the most complex software systems humans build. Fuzzing is one way to identify subtle compiler bugs that are hard to find with human-constructed tests. Grammar-based fuzzing, however, requires a grammar for a compiler's input language, and can miss bugs induced by code that does not actually satisfy the grammar the compiler should accept. Grammar-based fuzzing also seldom uses advanced modern fuzzing techniques based on coverage feedback. However, modern mutation-based fuzzers are often ineffective for testing compilers because most inputs they generate do not even come close to getting past the parsing stage of compilation. This paper introduces a technique for taking a modern mutation-based fuzzer (AFL in our case, but the method is general) and augmenting it with operators taken from mutation testing, and program splicing. We conduct a controlled study to show that our hybrid approaches significantly improve fuzzing effectiveness qualitatively (consistently finding unique bugs that baseline approaches do not) and quantitatively (typically finding more unique bugs in the same time span, despite fewer program executions). Our easy-To-Apply approach has allowed us to report more than 100 confirmed and fixed bugs in production compilers, and found a bug in the Solidity compiler that earned a security bounty.
KW - compiler development
KW - fuzzing
KW - mutation testing
UR - http://www.scopus.com/inward/record.url?scp=85127834408&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85127834408&partnerID=8YFLogxK
U2 - 10.1145/3497776.3517765
DO - 10.1145/3497776.3517765
M3 - Conference contribution
AN - SCOPUS:85127834408
T3 - CC 2022 - Proceedings of the 31st ACM SIGPLAN International Conference on Compiler Construction
SP - 194
EP - 204
BT - CC 2022 - Proceedings of the 31st ACM SIGPLAN International Conference on Compiler Construction
A2 - Egger, Bernhard
A2 - Smith, Aaron
PB - Association for Computing Machinery, Inc
T2 - 31st ACM SIGPLAN International Conference on Compiler Construction, CC 2022
Y2 - 2 April 2022 through 3 April 2022
ER -