Deep Learning (DL) systems are key enablers for engineering intelligent applications due to their ability to solve complex tasks such as image recognition and machine translation. Nevertheless, using DL systems in safety- and security-critical applications requires to provide testing evidence for their dependable operation. Recent research in this direction focuses on adapting testing criteria from traditional software engineering as a means of increasing confidence for their correct behaviour. However, they are inadequate in capturing the intrinsic properties exhibited by these systems. We bridge this gap by introducing DeepImportance, a systematic testing methodology accompanied by an Importance-Driven (IDC) test adequacy criterion for DL systems. Applying IDC enables to establish a layer-wise functional understanding of the importance of DL system components and use this information to guide the generation of semantically-diverse test sets. Our empirical evaluation on several DL systems, across multiple DL datasets and with state-of-the-art adversarial generation techniques demonstrates the usefulness and effectiveness of DeepImportance and its ability to guide the engineering of more robust DL systems.
Wed 8 JulDisplayed time zone: (UTC) Coordinated Universal Time change
15:00 - 16:00 | A8-Machine Learning and ModelsJournal First / Technical Papers at Goguryeo Chair(s): Liliana Pasquale University College Dublin & Lero | ||
15:00 8mTalk | Improving Vulnerability Inspection Efficiency Using Active LearningJ1 Journal First Zhe Yu NORTH CAROLINA STATE UNIVERSITY, Chris Theisen Microsoft, Laurie Williams North Carolina State University, Tim Menzies North Carolina State University | ||
15:08 8mTalk | How Bugs Are Born: A Model to Identify How Bugs Are Introduced in Software ComponentsJ1 Journal First Gema Rodríguez-Pérez University of Waterloo, Canada, Gregorio Robles Universidad Rey Juan Carlos, Alexander Serebrenik Eindhoven University of Technology, Andy Zaidman TU Delft, Daniel M. German University of Victoria, Jesus M. Gonzalez-Barahona Universidad Rey Juan Carlos DOI Pre-print | ||
15:16 8mTalk | How to “DODGE” Complex Software AnalyticsJ1 Journal First Amritanshu Agrawal Wayfair, Wei Fu Landing AI, Di Chen North Carolina State University, USA, Xipeng Shen North Carolina State University, Tim Menzies North Carolina State University | ||
15:24 12mTalk | Importance-Driven Deep Learning System TestingTechnical Technical Papers Simos Gerasimou University of York, UK, Hasan Ferit Eniser MPI-SWS, Alper Sen Bogazici University, Turkey, Alper Çakan Bogazici University, Turkey | ||
15:36 12mTalk | Quickly Generating Diverse Valid Test Inputs with Reinforcement LearningTechnical Technical Papers Sameer Reddy University of California, Berkeley, Caroline Lemieux University of California, Berkeley, Rohan Padhye Carnegie Mellon University, Koushik Sen University of California, Berkeley | ||
15:48 8mTalk | Impact of Discretization Noise of the Dependent variable on Machine Learning Classifiers in Software EngineeringJ1 Journal First Gopi Krishnan Rajbahadur Queen's University, Shaowei Wang Mississippi State University, Yasutaka Kamei Kyushu University, Ahmed E. Hassan Queen's University |