A Benchmark-Based Evaluation of Search-Based Crash ReproductionJ1
Crash reproduction approaches help developers during debugging by generating a test case that reproduces a given crash. Several solutions have been proposed to automate this task. However, the proposed solutions have been evaluated on a limited number of projects, making comparison difficult. In this paper, we enhance this line of research by proposing JCrashPack, an extensible benchmark for Java crash reproduction, together with ExRunner, a tool to simply and systematically run evaluations. JCrashPack contains 200 stack traces from various Java projects, including industrial open-source ones. We used JCrashPack to run an extensive evaluation of EvoCrash, the state-of-the-art tool for search-based crash reproduction. EvoCrash successfully reproduced 43% of the crashes. Furthermore, we observed that reproducing NullPointerException, IllegalArgumentException, and IllegalStateException is relatively easier than reproducing ClassCastException, ArrayIndexOutOfBoundsException, and StringIndexOutOfBoundsException. Our results include a detailed manual analysis of EvoCrash outputs, from which we derive 14 current challenges for crash reproduction, among which the generation of input data and the handling of abstract and anonymous classes are the most frequents. Finally, based on those challenges, we discuss future research directions for search-based crash reproduction for Java.
Fri 10 JulDisplayed time zone: (UTC) Coordinated Universal Time change
15:00 - 16:00 | A21-Testing and Debugging 3Journal First / Technical Papers at Silla Chair(s): Tingting Yu University of Kentucky | ||
15:00 12mTalk | Schrödinger's Security: Opening the Box on App Developers' Security RationaleTechnical Technical Papers Dirk van der Linden University of Bristol, Pauline Anthonysamy Google Inc., Bashar Nuseibeh The Open University (UK) & Lero (Ireland), Thein Tun , Marian Petre The Open University, Mark Levine Lancaster University, John Towse Lancaster University, Awais Rashid University of Bristol, UK | ||
15:12 8mTalk | Smart Greybox FuzzingJ1 Journal First Van-Thuan Pham Monash University, Marcel Böhme Monash University, Andrew Santosa National University of Singapore, Alexandru Răzvan Căciulescu UiPath, Abhik Roychoudhury National University of Singapore, Singapore | ||
15:20 8mTalk | Deep Transfer Bug LocalizationJ1 Journal First Xuan Huo Nanjing University, Ferdian Thung Singapore Management University, Ming Li Nanjing University, David Lo Singapore Management University, Shu-Ting Shi Nanjing University | ||
15:28 8mTalk | A Benchmark-Based Evaluation of Search-Based Crash ReproductionJ1 Journal First Mozhan Soltani Leiden University, Pouria Derakhshanfar Delft University of Technology, Xavier Devroey Delft University of Technology, Arie van Deursen Delft University of Technology Link to publication DOI Pre-print Media Attached | ||
15:36 12mTalk | An Investigation of Cross-Project Learning in Online Just-In-Time Software Defect PredictionTechnical Technical Papers Sadia Tabassum University of Birmingham, UK, Leandro Minku University of Birmingham, UK, Danyi Feng XiLiu Tech, George Cabral Universidade Federal Rural de Pernambuco, Liyan Song University of Birmingham | ||
15:48 8mTalk | An Empirical Study of the Long Duration of Continuous Integration BuildsJ1 Journal First Taher A Ghaleb Queen's University, Daniel Alencar Da Costa University of Otago, Ying Zou Queen's University, Kingston, Ontario Link to publication DOI Pre-print | ||