Approximation-Refinement Testing of Compute-Intensive Cyber-Physical Models: An Approach Based on System IdentificationTechnical
Black-box testing has been extensively applied to test models of Cyber-Physical systems (CPS) since these models are not often amenable to static and symbolic testing and verification. Black-box testing, however, requires to execute the model under test for a large number of candidate test inputs. This poses a challenge for a large and practically-important category of CPS models, known as compute-intensive CPS (CI-CPS) models, where a single simulation may take hours to complete. We propose a novel approach, namely ARIsTEO, to enable effective and efficient testing of CI-CPS models. Our approach embeds black-box testing into an iterative approximation-refinement loop. At the start, some sampled inputs and outputs of the CI-CPS model under test are used to generate a surrogate model that is faster to execute and can be subjected to black-box testing. Any failure-revealing test identified for the surrogate model is checked on the original model. If spurious, the test results are used to refine the surrogate model to be tested again. Otherwise, the test reveals a valid failure. We evaluated ARIsTEO by comparing it with S-Taliro, an open-source and industry-strength tool for testing CPS models. Our results, obtained based on five publicly-available CPS models, show that, on average, ARIsTEO is able to find 24% more requirements violations than S-Taliro and is 31% faster than S-Taliro in finding those violations. We further assessed the effectiveness and efficiency of ARIsTEO on a large industrial case study from the satellite domain. In contrast to S-Taliro, ARIsTEO successfully tested two different versions of this model and could identify three requirements violations, requiring four hours, on average, for each violation.
Tue 7 JulDisplayed time zone: (UTC) Coordinated Universal Time change
15:00 - 16:00 | A1-Autonomous Driving SystemsTechnical Papers at Baekje Chair(s): Donghwan Shin University of Luxembourg (SnT) | ||
15:00 12mTalk | SLEMI: Equivalence Modulo Input (EMI) Based Mutation of CPS Models for Finding Compiler Bugs in SimulinkTechnical Technical Papers Shafiul Azam Chowdhury University of Texas at Arlington, Sohil Lal Shrestha The University of Texas at Arlington, Taylor T Johnson Vanderbilt University, Christoph Csallner University of Texas at Arlington Link to publication DOI Media Attached | ||
15:12 12mTalk | DeepBillboard: Systematic Physical-World Testing of Autonomous Driving SystemsTechnical Technical Papers Husheng Zhou The University of Texas at Dallas, Wei Li Southern University of Science and Technology, Zelun Kong The University of Texas at Dallas, Junfeng Guo The University of Texas at Dallas, Yuqun Zhang Southern University of Science and Technology, Lingming Zhang The University of Texas at Dallas, Bei Yu The Chinese University of Hong Kong, Cong Liu UT Dallas | ||
15:24 12mTalk | Misbehaviour Prediction for Autonomous Driving SystemsTechnical Technical Papers Andrea Stocco Università della Svizzera italiana, Michael Weiss Università della Svizzera Italiana (USI), Marco Calzana Università della Svizzera Italiana (USI), Paolo Tonella Università della Svizzera italiana Pre-print | ||
15:36 12mTalk | Approximation-Refinement Testing of Compute-Intensive Cyber-Physical Models: An Approach Based on System IdentificationTechnical Technical Papers Claudio Menghi University of Luxembourg, Shiva Nejati University of Ottawa, Lionel Briand SnT Centre/University of Luxembourg, Yago Isasi Parache LuxSpace | ||
15:48 12mTalk | A Comprehensive Study of Autonomous Vehicle BugsTechnical Technical Papers Joshua Garcia University of California, Irvine, Yang Feng Nanjing University, Junjie Shen University of California, Irvine, Sumaya Almanee University of California, Irvine, Yuan Xia University of California, Irvine, Qi Alfred Chen University of California, Irvine |