Explaining Regressions via Alignment Slicing and Mending
Regression faults, which make working code stop functioning, are often introduced when developers make changes to the software. As a large number of changes can happen between the original correct version and current regression version, once a regression failure is observed, it is a non-trivial task to (1) isolate the failure-inducing changes and, more importantly, (2) understand how they lead to the final observable failure. Many regression fault localization techniques such as dynamic slicing, delta debugging, and symbolic analysis have been proposed. However, issues like inaccuracy, lack of scalability, and missing explanation are still obstacles for their practical application.
In this work, we propose a trace-based approach to identifying not only where the root cause of a regression bug lies, but also how the defect is propagated to its manifestation as the explanation. In our approach, we keep the trace of original correct version as reference and infer the faulty steps on the trace of regression version so that we can build a causality graph of how the defect is propagated. To this end, we overcomes two technical challenges. First, we align two traces derived from two program versions by extending state-of-the-art trace alignment technique for regression fault with novel relaxation technique. We provide theoretical analysis on how our relaxation technique can address the unique challenges to align two program execution traces derived from two versions with non-trivial changes. Second, we construct causality graph (i.e., explanation) by adopting a technique called alignment slicing and mending to isolate the failure-inducing changes and explain the failure. Technically, the explanation is generated by asking why and why-not questions along the traces.
Our comparative experiment with the state-of-the-art techniques including dynamic slicing, delta-debugging, and symbolic execution on 24 real-world regressions shows that (1) our approach is more accurate on isolating the failure-inducing changes, (2) the generated explanation requires acceptable manual effort to inspect, and (3) our approach requires lower runtime overhead. In addition, we also conduct an applicability experiment based on Defects4J bug repository, empirically validating the potential limitations of our trace-based approach and providing guidance for its practical use. A demo video of our tool is available at https://github.com/llmhyy/tregression.
To summarize, we make the following contributions: - Given two traces from two versions of a program, we extend existing trace alignment techniques with a novel relaxation technique to align the executions of two program versions with considerable number of modifications. - We adopt the alignment slicing and mending technique on locating regression bugs, which derives a causality graph including both execution steps and unexecuted modifications to explain the regression failure. - We implement our tool ERASE for both Java and C programs, providing an interactive tool for programmers to explore why a regression failure happens in practice. - We study the effectiveness of our approach on an experiment on 24 real-world C regressions, which demonstrates the effectiveness of ERASE over state-of-the-art approaches. In addition, our applicability experiment on 298 Defects4J bugs allows us to understand the limitation of trace-based approach and provides guidance on its practical use.
Wed 8 JulDisplayed time zone: (UTC) Coordinated Universal Time change
01:05 - 02:05
P12-Testing and DebuggingJournal First / Software Engineering in Practice at Silla
Chair(s): Taeksu Kim Samsung Research, Samsung Electronics
|Debugging Crashes using Continuous Contrast Set MiningSEIP|
Software Engineering in Practice
Rebecca Qian Facebook, Inc., Yang Yu Purdue University, Wonhee Park Facebook, Inc., Vijayaraghavan Murali Facebook, Inc., Stephen J Fink Facebook, Satish Chandra Facebook
|Automatic Abnormal Log Detection by Analyzing Log History for Providing Debugging InsightSEIP|
Software Engineering in Practice
Jinhan Kim , Valeriy Savchenko Ivannikov Institute for System Programming of the RAS, Kihyuck Shin Samsung Electronics, Konstantin Sorokin Ivannikov Institute for System Programming of the RAS, Hyunseok Jeon Samsung Electronics, Georgiy Pankratenko Ivannikov Institute for System Programming of the RAS, Sergey Markov Ivannikov Institute for System Programming of the RAS, Chul-Joo Kim Samsung Electronics
|Explaining Regressions via Alignment Slicing and MendingJ1|
Haijun Wang Ant Financial Services Group, China; CSSE, Shenzhen University, China, Yun Lin National University of Singapore, Zijiang Yang Western Michigan University, Jun Sun Singapore Management University, Yang Liu Nanyang Technological University, Singapore, Jin Song Dong National University of Singapore, Qinghua Zheng Xi'an Jiaotong University, Ting Liu Xi'an Jiaotong University
|Historical Spectrum based Fault LocalizationJ1|
Ming Wen Huazhong University of Science and Technology, China, Junjie Chen Tianjin University, China, Yongqiang TIAN The Hong Kong University of Science and Technology, Rongxin Wu Department of Cyber Space Security, Xiamen University, Dan Hao Peking University, Shi Han Microsoft Research Asia, Shing-Chi Cheung Department of Computer Science and Engineering, The Hong Kong University of Science and Technology
|Visualizing distributed system executionsJ1|
Ivan Beschastnikh Computer Science, University of British Columbia, Perry Liu University of British Columbia, Albert Xing University of British Columbia, Patty Wang University of British Columbia, Yuriy Brun University of Massachusetts Amherst, Michael D. Ernst University of Washington, USADOI Pre-print
|An Integration Test Order Strategy to Consider Control CouplingJ1|
Shujuan Jiang China University of Mining and Technology, Miao ZHANG City University of Hong Kong, Yanmei ZHANG China University of Mining and Technology, Rongcun Wang China University of Mining and Technology, Qiao YU Jiangsu Normal University, Jacky Keung City University of Hong Kong