Regression faults, which make working code stop functioning, are often introduced when developers make changes to the software. As a large number of changes can happen between the original correct version and current regression version, once a regression failure is observed, it is a non-trivial task to (1) isolate the failure-inducing changes and, more importantly, (2) understand how they lead to the final observable failure. Many regression fault localization techniques such as dynamic slicing, delta debugging, and symbolic analysis have been proposed. However, issues like inaccuracy, lack of scalability, and missing explanation are still obstacles for their practical application.
In this work, we propose a trace-based approach to identifying not only where the root cause of a regression bug lies, but also how the defect is propagated to its manifestation as the explanation. In our approach, we keep the trace of original correct version as reference and infer the faulty steps on the trace of regression version so that we can build a causality graph of how the defect is propagated. To this end, we overcomes two technical challenges. First, we align two traces derived from two program versions by extending state-of-the-art trace alignment technique for regression fault with novel relaxation technique. We provide theoretical analysis on how our relaxation technique can address the unique challenges to align two program execution traces derived from two versions with non-trivial changes. Second, we construct causality graph (i.e., explanation) by adopting a technique called alignment slicing and mending to isolate the failure-inducing changes and explain the failure. Technically, the explanation is generated by asking why and why-not questions along the traces.
Our comparative experiment with the state-of-the-art techniques including dynamic slicing, delta-debugging, and symbolic execution on 24 real-world regressions shows that (1) our approach is more accurate on isolating the failure-inducing changes, (2) the generated explanation requires acceptable manual effort to inspect, and (3) our approach requires lower runtime overhead. In addition, we also conduct an applicability experiment based on Defects4J bug repository, empirically validating the potential limitations of our trace-based approach and providing guidance for its practical use. A demo video of our tool is available at https://github.com/llmhyy/tregression.
To summarize, we make the following contributions: - Given two traces from two versions of a program, we extend existing trace alignment techniques with a novel relaxation technique to align the executions of two program versions with considerable number of modifications. - We adopt the alignment slicing and mending technique on locating regression bugs, which derives a causality graph including both execution steps and unexecuted modifications to explain the regression failure. - We implement our tool ERASE for both Java and C programs, providing an interactive tool for programmers to explore why a regression failure happens in practice. - We study the effectiveness of our approach on an experiment on 24 real-world C regressions, which demonstrates the effectiveness of ERASE over state-of-the-art approaches. In addition, our applicability experiment on 298 Defects4J bugs allows us to understand the limitation of trace-based approach and provides guidance on its practical use.
Conference DayWed 8 JulDisplayed time zone: (UTC) Coordinated Universal Time change
01:05 - 02:05
|Debugging Crashes using Continuous Contrast Set MiningSEIP|
Software Engineering in Practice
|Automatic Abnormal Log Detection by Analyzing Log History for Providing Debugging InsightSEIP|
Software Engineering in Practice
Jinhan Kim, Valeriy SavchenkoIvannikov Institute for System Programming of the RAS, Kihyuck ShinSamsung Electronics, Konstantin SorokinIvannikov Institute for System Programming of the RAS, Hyunseok JeonSamsung Electronics, Georgiy PankratenkoIvannikov Institute for System Programming of the RAS, Sergey MarkovIvannikov Institute for System Programming of the RAS, Chul-Joo KimSamsung Electronics
|Explaining Regressions via Alignment Slicing and MendingJ1|
Haijun WangAnt Financial Services Group, China; CSSE, Shenzhen University, China, Yun LinNational University of Singapore, Zijiang YangWestern Michigan University, Jun SunSingapore Management University, Yang LiuNanyang Technological University, Singapore, Jin Song DongNational University of Singapore, Qinghua ZhengXi'an Jiaotong University, Ting LiuXi'an Jiaotong University
|Historical Spectrum based Fault LocalizationJ1|
Ming WenHuazhong University of Science and Technology, China, Junjie ChenTianjin University, China, Yongqiang TIANThe Hong Kong University of Science and Technology, Rongxin WuDepartment of Cyber Space Security, Xiamen University, Dan HaoPeking University, Shi HanMicrosoft Research Asia, Shing-Chi CheungDepartment of Computer Science and Engineering, The Hong Kong University of Science and Technology
|Visualizing distributed system executionsJ1|
Ivan BeschastnikhComputer Science, University of British Columbia, Perry LiuUniversity of British Columbia, Albert XingUniversity of British Columbia, Patty WangUniversity of British Columbia, Yuriy BrunUniversity of Massachusetts Amherst, Michael D. ErnstUniversity of Washington, USADOI Pre-print
|An Integration Test Order Strategy to Consider Control CouplingJ1|