The increasing use of machine-learning (ML) enabled systems incritical tasks fuels the quest for novel verification and validation techniques yet grounded in accepted system assurance principles. In traditional system development, model-based techniques have been widely adopted, where the central premise is that abstract models of the required system provide a sound basis for judging its implementation. We posit an analogous approach for ML systems using an ML technique that extracts from the high-dimensional training data implicitly describing the required system, a low-dimensional underlying structure—a manifold. It is then harnessed for a range of quality assurance tasks such as test adequacy measurement, test input generation, and runtime monitoring of the target ML system.The approach is built on variational autoencoders, an unsupervised method for learning a pair of mutually near-inverse functions between a given high-dimensional dataset and a low-dimensional representation. Preliminary experiments establish that the proposedmanifold-based approach, for test adequacy drives diversity in testdata, for test generation yields fault-revealing yet realistic test casesand for run-time monitoring provides an independent means to assess trustability of the target system’s output.
Fri 10 JulDisplayed time zone: (UTC) Coordinated Universal Time change
16:05 - 17:05 | A24-Testing and Debugging 4Technical Papers / New Ideas and Emerging Results / Journal First / Demonstrations at Silla Chair(s): Yijun Yu The Open University, UK | ||
16:05 6mTalk | Manifold for Machine Learning AssuranceNIER New Ideas and Emerging Results | ||
16:11 12mTalk | On Learning Meaningful Assert Statements for Unit Test CasesTechnical Technical Papers Cody Watson Washington and Lee University, Michele Tufano Microsoft, Kevin Moran William & Mary/George Mason University, Gabriele Bavota Università della Svizzera italiana, Denys Poshyvanyk William and Mary Pre-print Media Attached | ||
16:23 12mTalk | TRADER: Trace Divergence Analysis and Embedding Regulation for Debugging Recurrent Neural NetworksTechnical Technical Papers Guanhong Tao Purdue University, Shiqing Ma Rutgers University, Yingqi Liu Purdue University, USA, Qiuling Xu Purdue University, Xiangyu Zhang Purdue University Pre-print | ||
16:35 3mTalk | DeepMutation: A Neural Mutation ToolDemo Demonstrations Michele Tufano Microsoft, Jason Kimko William & Mary, Shiya Wang William & Mary, Cody Watson Washington and Lee University, Gabriele Bavota Università della Svizzera italiana, Massimiliano Di Penta University of Sannio, Denys Poshyvanyk William and Mary Pre-print | ||
16:38 8mTalk | Specification Patterns for Robotic MissionsJ1 Journal First Claudio Menghi University of Luxembourg, Christos Tsigkanos TU Vienna, Patrizio Pelliccione University of L'Aquila and Chalmers | University of Gothenburg, Carlo Ghezzi Politecnico di Milano, Thorsten Berger Chalmers | University of Gothenburg | ||
16:46 8mTalk | ProXray: Protocol Model Learning and Guided Firmware AnalysisJ1 Journal First Farhaan Fowze University of Florida, Dave (Jing) Tian Purdue University, Grant Hernandez University of Florida, Kevin Butler Univ. Florida, Tuba Yavuz University of Florida | ||
16:54 6mTalk | Visual Sketching: From Image Sketches to CodeNIER New Ideas and Emerging Results Marcelo d'Amorim Federal University of Pernambuco, Rui Abreu Instituto Superior Técnico, U. Lisboa & INESC-ID, Carlos Mello Federal University of Pernambuco Pre-print Media Attached |