TY - GEN
T1 - Why Statically Estimate Code Coverage is So Hard? A Report of Lessons Learned
AU - Aniche, Maurício F.
AU - Oliva, Gustavo A.
AU - Gerosa, Marco A.
N1 - Publisher Copyright:
© 2015 IEEE.
PY - 2015/11/11
Y1 - 2015/11/11
N2 - The calculation of test coverage is often unfeasible for large-scale mining software repositories studies, as its computation requires building each project and executing their test suites. Because of that, we have been working on heuristics to calculate code coverage based on static code analysis. However, our results have been disappointing so far. In this paper, we present our approach to the problem and an evaluation involving 18 open source projects (around 2,700 classes) from the Apache Software Foundation. Results show that our approach provides acceptable results for only 50% of all classes. We believe researchers can learn from our mistakes and possibly derive a better approach. We advise researchers who need to use code coverage in their studies to select projects with a well-defined build system, such as Maven.
AB - The calculation of test coverage is often unfeasible for large-scale mining software repositories studies, as its computation requires building each project and executing their test suites. Because of that, we have been working on heuristics to calculate code coverage based on static code analysis. However, our results have been disappointing so far. In this paper, we present our approach to the problem and an evaluation involving 18 open source projects (around 2,700 classes) from the Apache Software Foundation. Results show that our approach provides acceptable results for only 50% of all classes. We believe researchers can learn from our mistakes and possibly derive a better approach. We advise researchers who need to use code coverage in their studies to select projects with a well-defined build system, such as Maven.
UR - http://www.scopus.com/inward/record.url?scp=84962285307&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=84962285307&partnerID=8YFLogxK
U2 - 10.1109/SBES.2015.28
DO - 10.1109/SBES.2015.28
M3 - Conference contribution
AN - SCOPUS:84962285307
T3 - Proceedings - 29th Brazilian Symposium on Software Engineering, SBES 2015
SP - 185
EP - 190
BT - Proceedings - 29th Brazilian Symposium on Software Engineering, SBES 2015
PB - Institute of Electrical and Electronics Engineers Inc.
T2 - 29th Brazilian Symposium on Software Engineering, SBES 2015
Y2 - 21 September 2015 through 25 September 2015
ER -