Write a Blog >>
ICSE 2020
Wed 24 June - Thu 16 July 2020
Wed 8 Jul 2020 16:08 - 16:16 at Goguryeo - A11-Performance and Analysis Chair(s): Pooyan Jamshidi

Automated test case generation is an effective technique to yield high-coverage test suites. While the majority of research effort has been devoted to satisfying coverage criteria, a recent trend emerged towards optimizing other non-coverage aspects. In this regard, runtime and memory usage are two essential dimensions: less expensive tests reduce the resource demands for the generation process and later regression testing phases. This study shows that performance-aware test case generation requires solving two main challenges: providing a good approximation of resource usage with minimal overhead and avoiding detrimental effects on both final coverage and fault detection effectiveness. To tackle these challenges, we conceived a set of performance proxies -inspired by previous work on performance testing- that provide a reasonable estimation of the test execution costs (i.e., runtime and memory usage). Thus, we propose an adaptive strategy, called aDynaMOSA, which leverages these proxies by extending DynaMOSA, a state-of-the-art evolutionary algorithm in unit testing. Our empirical study -involving 110 non-trivial Java classes- reveals that our adaptive approach generates test suite with statistically significant improvements in runtime (-25%) and heap memory consumption (-15%) compared to DynaMOSA. Additionally, aDynaMOSA has comparable results to DynaMOSA over seven different coverage criteria and similar fault detection effectiveness. Our empirical investigation also highlights that the usage of performance proxies (i.e., without the adaptiveness) is not sufficient to generate more performant test cases without compromising the overall coverage.

Conference Day
Wed 8 Jul

Displayed time zone: (UTC) Coordinated Universal Time change

16:05 - 17:05
A11-Performance and AnalysisNew Ideas and Emerging Results / Journal First / Technical Papers / Demonstrations at Goguryeo
Chair(s): Pooyan JamshidiUniversity of South Carolina
Nimbus: Improving the Developer Experience for Serverless ApplicationsDemo
Robert ChatleyImperial College London, Thomas AllertonStarling Bank
Testing with Fewer Resources: An Adaptive Approach to Performance-Aware Test Case GenerationJ1
Journal First
Giovanni GranoUniversity of Zurich, Christoph LaaberUniversity of Zurich, Annibale PanichellaDelft University of Technology, Sebastiano PanichellaZurich University of Applied Sciences
Link to publication DOI Pre-print
What's Wrong with My Benchmark Results? Studying Bad Practices in JMH BenchmarksJ1
Journal First
Diego CostaConcordia University, Canada, Cor-Paul BezemerUniversity of Alberta, Canada, Philipp LeitnerChalmers University of Technology & University of Gothenburg, Artur AndrzejakHeidelberg University
Towards the Use of the Readily Available Tests from the Release Pipeline as Performance Tests. Are We There Yet?ACM SIGSOFT Distinguished Paper AwardsTechnical
Technical Papers
Zishuo DingConcordia University, Canada, Jinfu ChenConcordia University, Canada, Weiyi ShangConcordia University
ModGuard: Identifying Integrity & Confidentiality Violations in Java ModulesJ1
Journal First
Andreas DannPaderborn University, Ben HermannPaderborn University, Eric BoddenHeinz Nixdorf Institut, Paderborn University and Fraunhofer IEM
Link to publication DOI
Program Debloating via Stochastic OptimizationNIER
New Ideas and Emerging Results
Qi XinGeorgia Institute of Technology, Myeongsoo KimGeorgia Institute of Technology, Qirun ZhangGeorgia Institute of Technology, USA, Alessandro OrsoGeorgia Tech
The ORIS Tool: Quantitative Evaluation of Non-Markovian SystemsJ1
Journal First
Marco PaolieriUniversity of Southern California, Marco BiagiUniversity of Florence, Laura CarnevaliUniversity of Florence, Enrico VicarioUniversity of Florence