Write a Blog >>
ICSE 2020
Wed 24 June - Thu 16 July 2020
Sat 11 Jul 2020 00:24 - 00:36 at Goguryeo - P26-Deep Learning Testing and Debugging Chair(s): Tim Menzies

This paper presents TransRepair, a fully automatic approach for testing and repairing the consistency of machine translation systems. TransRepair combines mutation with metamorphic testing to detect inconsistency bugs (without access to human oracles). It then adopts probability-reference or cross-reference to post-process the translations, in a grey-box or black-box manner, to repair the inconsistencies. Our evaluation on two state-of-the-art translators, Google Translate and Transformer, indicates that TransRepair has a high precision (99%) on generating input pairs with consistent translations. With these tests, using automatic consistency metrics and manual assessment, we find that Google Translate and Transformer have approximately 38% and 42% inconsistency bugs. Black-box repair fixes 28% and 19% bugs on average for Google Translate and Transformer. Grey-box repair fixes 30% bugs on average for Transformer. Manual inspection indicates that the translations repaired by our approach improve consistency in 87% of cases (degrading it in 2%), and that our repairs have better translation quality in 27% of the cases (worse in 8%).

Sat 11 Jul
Times are displayed in time zone: (UTC) Coordinated Universal Time change

00:00 - 01:00: P26-Deep Learning Testing and DebuggingPaper Presentations / Technical Papers at Goguryeo
Chair(s): Tim MenziesNorth Carolina State University
00:00 - 00:12
Talk
Technical Papers
Brandon PaulsenUniversity of Southern California, Jingbo WangUniversity of Southern California, Chao WangUSC
Pre-print
00:12 - 00:24
Talk
Technical Papers
Pinjia HeETH Zurich, Clara MeisterETH Zurich, Zhendong SuETH Zurich, Switzerland
00:24 - 00:36
Talk
Technical Papers
Zeyu SunPeking University, Jie M. ZhangUniversity College London, UK, Mark HarmanFacebook and University College London, Mike PapadakisUniversity of Luxembourg, Lu ZhangPeking University, China
00:36 - 00:48
Talk
Technical Papers
Yuchi TianColumbia University, Ziyuan ZhongColumbia University, Vicente OrdonezUniversity of Virginia, Gail KaiserColumbia University, Baishakhi RayColumbia University, New York
00:48 - 01:00
Talk
Technical Papers
Md Johirul IslamIowa State University, Rangeet PanIowa State University, USA, Giang NguyenDept. of Computer Science, Iowa State University, Hridesh RajanIowa State University, USA