Evaluating and Improving Static Analysis Tools Via Differential Mutation Analysis

Alex Groce, Iftekhar Ahmed, Josselin Feist, Gustavo Grieco, Jiri Gesi, Mehran Meidani, Qihong Chen

Research output: Chapter in Book/Report/Conference proceedingConference contribution

4 Scopus citations

Abstract

Static analysis tools attempt to detect faults in code without executing it. Understanding the strengths and weaknesses of such tools, and performing direct comparisons of their ef-fectiveness, is difficult, involving either manual examination of differing warnings on real code, or the bias-prone construction of artificial test cases. This paper proposes a novel automated approach to comparing static analysis tools, based on producing mutants of real code, and comparing detection rates over these mutants. In addition to making tool differences quantitatively observable without extensive manual effort, this approach offers a new way to detect and fix omissions in a static analysis tool's set of detectors. We present an extensive comparison of three smart contract static analysis tools, and show how our approach allowed us to add three effective new detectors to the best of these. We also evaluate popular Java and Python static analysis tools and discuss their strengths and weaknesses.

Original languageEnglish (US)
Title of host publicationProceedings - 2021 21st International Conference on Software Quality, Reliability and Security, QRS 2021
PublisherInstitute of Electrical and Electronics Engineers
Pages207-218
Number of pages12
ISBN (Electronic)9781665458139
DOIs
StatePublished - 2021
Event21st International Conference on Software Quality, Reliability and Security, QRS 2021 - Hainan, China
Duration: Dec 6 2021Dec 10 2021

Publication series

NameIEEE International Conference on Software Quality, Reliability and Security, QRS
Volume2021-December
ISSN (Print)2693-9177

Conference

Conference21st International Conference on Software Quality, Reliability and Security, QRS 2021
Country/TerritoryChina
CityHainan
Period12/6/2112/10/21

Keywords

  • mutation testing
  • smart contracts
  • static analysis

ASJC Scopus subject areas

  • Software
  • Safety, Risk, Reliability and Quality

Fingerprint

Dive into the research topics of 'Evaluating and Improving Static Analysis Tools Via Differential Mutation Analysis'. Together they form a unique fingerprint.

Cite this