Repairing Deep Neural Networks: Fix Patterns and Challenges
Technical
Significant interest in applying Deep Neural Network (DNN) has fueled the need to support engineering of software that uses DNNs. Repairing software that uses DNNs is one such unmistakable SE need where automated tools could be very helpful; however, we do not fully understand challenges to repairing and patterns that are utilized when manually repairing them. What challenges should automated repair tools address? What are the repair patterns whose automation could help developers? Which repair patterns should be assigned a higher priority for automation? This work presents a comprehensive study of bug fix patterns to address these questions. We have studied 415 repairs from StackOverflow and 555 repairs from GitHub for five popular deep learning libraries Caffe, Keras, TensorFlow, Theano, and Torch to understand challenges in repairs and bug repair patterns. Our key findings reveal that DNN bug fix patterns are distinctive compared to traditional bug fix patterns; the most common bug fix patterns are fixing data dimension and neural network connectivity; DNN bug fixes have the potential to introduce adversarial vulnerabilities; DNN bug fixes frequently introduce new bugs; and DNN bug localization, reuse of trained model, and coping with frequent releases are major challenges faced by developers when fixing bugs. We also contribute a benchmark of 667 DNN (bug, repair) instances.
Sat 11 JulDisplayed time zone: (UTC) Coordinated Universal Time change
00:00 - 01:00 | P26-Deep Learning Testing and DebuggingTechnical Papers at Goguryeo Chair(s): Tim Menzies North Carolina State University | ||
00:00 12mTalk | ReluDiff: Differential Verification of Deep Neural Networks Technical Papers Brandon Paulsen University of Southern California, Jingbo Wang University of Southern California, Chao Wang USC Pre-print | ||
00:12 12mTalk | Structure-Invariant Testing for Machine TranslationTechnical Technical Papers | ||
00:24 12mTalk | Automatic Testing and Improvement of Machine TranslationTechnical Technical Papers Zeyu Sun Peking University, Jie M. Zhang University College London, UK, Mark Harman Facebook and University College London, Mike Papadakis University of Luxembourg, Lu Zhang Peking University, China | ||
00:36 12mTalk | Testing DNN Image Classifier for Confusion & Bias Errors Technical Papers Yuchi Tian Columbia University, Ziyuan Zhong Columbia University, Vicente Ordonez University of Virginia, Gail Kaiser Columbia University, Baishakhi Ray Columbia University, New York | ||
00:48 12mTalk | Repairing Deep Neural Networks: Fix Patterns and Challenges Technical Papers Md Johirul Islam Iowa State University, Rangeet Pan Iowa State University, USA, Giang Nguyen Dept. of Computer Science, Iowa State University, Hridesh Rajan Iowa State University, USA | ||