On Learning Meaningful Assert Statements for Unit Test CasesTechnical
Software testing is an essential part of the software lifecycle and requires a substantial amount of time and effort. It has been estimated that software developers spend close to 50% of their time on testing the code they write. For these reasons, a long standing goal within the research community is to (partially) automate software testing. While several techniques and tools have been proposed to automatically generate test methods, recent work has criticized the quality and usefulness of the assert statements they generate. Therefore, we employ a Neural Machine Translation (NMT) based approach called ATLAS (AuTomatic Learning of Assert Statements) to automatically generate meaningful assert statements for test methods. Given a test method and a focal method (i.e., the main method under test), ATLAS can predict a meaningful assert statement to assess the correctness of the focal method. We applied ATLAS to thousands of test methods from GitHub projects and it was able to predict the exact assert statement manually written by developers in 31% of the cases when only considering the top-1 predicted assert. When considering the top-5 predicted assert statements, ATLAS is able to predict exact matches in 50% of the cases. These promising results hint to the potential usefulness of our approach as (i) a complement to automatic test case generation techniques, and (ii) a code completion support for developers, who can benefit from the recommended assert statements while writing test code.
Fri 10 JulDisplayed time zone: (UTC) Coordinated Universal Time change
16:05 - 17:05 | A24-Testing and Debugging 4Technical Papers / New Ideas and Emerging Results / Journal First / Demonstrations at Silla Chair(s): Yijun Yu The Open University, UK | ||
16:05 6mTalk | Manifold for Machine Learning AssuranceNIER New Ideas and Emerging Results | ||
16:11 12mTalk | On Learning Meaningful Assert Statements for Unit Test CasesTechnical Technical Papers Cody Watson Washington and Lee University, Michele Tufano Microsoft, Kevin Moran William & Mary/George Mason University, Gabriele Bavota Università della Svizzera italiana, Denys Poshyvanyk William and Mary Pre-print Media Attached | ||
16:23 12mTalk | TRADER: Trace Divergence Analysis and Embedding Regulation for Debugging Recurrent Neural Networks Technical Papers Guanhong Tao Purdue University, Shiqing Ma Rutgers University, Yingqi Liu Purdue University, USA, Qiuling Xu Purdue University, Xiangyu Zhang Purdue University Pre-print | ||
16:35 3mTalk | DeepMutation: A Neural Mutation ToolDemo Demonstrations Michele Tufano Microsoft, Jason Kimko William & Mary, Shiya Wang William & Mary, Cody Watson Washington and Lee University, Gabriele Bavota Università della Svizzera italiana, Massimiliano Di Penta University of Sannio, Denys Poshyvanyk William and Mary Pre-print | ||
16:38 8mTalk | Specification Patterns for Robotic MissionsJ1 Journal First Claudio Menghi University of Luxembourg, Christos Tsigkanos TU Vienna, Patrizio Pelliccione University of L'Aquila and Chalmers | University of Gothenburg, Carlo Ghezzi Politecnico di Milano, Thorsten Berger Chalmers | University of Gothenburg | ||
16:46 8mTalk | ProXray: Protocol Model Learning and Guided Firmware AnalysisJ1 Journal First Farhaan Fowze University of Florida, Dave (Jing) Tian Purdue University, Grant Hernandez University of Florida, Kevin Butler Univ. Florida, Tuba Yavuz University of Florida | ||
16:54 6mTalk | Visual Sketching: From Image Sketches to CodeNIER New Ideas and Emerging Results Marcelo d'Amorim Federal University of Pernambuco, Rui Abreu Instituto Superior Técnico, U. Lisboa & INESC-ID, Carlos Mello Federal University of Pernambuco Pre-print Media Attached | ||