TY - GEN
T1 - Mutations
T2 - 25th IEEE International Symposium on Software Reliability Engineering, ISSRE 2014
AU - Gopinath, Rahul
AU - Jensen, Carlos
AU - Groce, Alex
N1 - Publisher Copyright:
© 2014 IEEE.
PY - 2014/12/11
Y1 - 2014/12/11
N2 - Mutation analysis is often used to compare the effectiveness of different test suites or testing techniques. One of the main assumptions underlying this technique is the Competent Programmer Hypothesis, which proposes that programs are very close to a correct version, or that the difference between current and correct code for each fault is very small. Researchers have assumed on the basis of the Competent Programmer Hypothesis that the faults produced by mutation analysis are similar to real faults. While there exists some evidence that supports this assumption, these studies are based on analysis of a limited and potentially non-representative set of programs and are hence not conclusive. In this paper, we separately investigate the characteristics of bug-fixes and other changes in a very large set of randomly selected projects using four different programming languages. Our analysis suggests that a typical fault involves about three to four tokens, and is seldom equivalent to any traditional mutation operator. We also find the most frequently occurring syntactical patterns, and identify the factors that affect the real bug-fix change distribution. Our analysis suggests that different languages have different distributions, which in turn suggests that operators optimal in one language may not be optimal for others. Moreover, our results suggest that mutation analysis stands in need of better empirical support of the connection between mutant detection and detection of actual program faults in a larger body of real programs.
AB - Mutation analysis is often used to compare the effectiveness of different test suites or testing techniques. One of the main assumptions underlying this technique is the Competent Programmer Hypothesis, which proposes that programs are very close to a correct version, or that the difference between current and correct code for each fault is very small. Researchers have assumed on the basis of the Competent Programmer Hypothesis that the faults produced by mutation analysis are similar to real faults. While there exists some evidence that supports this assumption, these studies are based on analysis of a limited and potentially non-representative set of programs and are hence not conclusive. In this paper, we separately investigate the characteristics of bug-fixes and other changes in a very large set of randomly selected projects using four different programming languages. Our analysis suggests that a typical fault involves about three to four tokens, and is seldom equivalent to any traditional mutation operator. We also find the most frequently occurring syntactical patterns, and identify the factors that affect the real bug-fix change distribution. Our analysis suggests that different languages have different distributions, which in turn suggests that operators optimal in one language may not be optimal for others. Moreover, our results suggest that mutation analysis stands in need of better empirical support of the connection between mutant detection and detection of actual program faults in a larger body of real programs.
KW - mutation analysis
KW - software testing
UR - http://www.scopus.com/inward/record.url?scp=84928676115&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=84928676115&partnerID=8YFLogxK
U2 - 10.1109/ISSRE.2014.40
DO - 10.1109/ISSRE.2014.40
M3 - Conference contribution
AN - SCOPUS:84928676115
T3 - Proceedings - International Symposium on Software Reliability Engineering, ISSRE
SP - 189
EP - 200
BT - Proceedings - IEEE 25th International Symposium on Software Reliability Engineering, ISSRE 2014
PB - IEEE Computer Society
Y2 - 3 November 2014 through 6 November 2014
ER -