Seenomaly: Vision-Based Linting of GUI Animation Effects Against Design-Don’t GuidelinesTechnical
GUI animations, such as card movement, menu slide in/out, snackbar display, provide appealing user experience and enhance the usability of mobile applications. These GUI animations should not violate the platform’s UI design guidelines (referred to as design-don’t guideline in this work) regarding component motion and interaction, content appearing and disappearing, and elevation and shadow changes.However, none of existing static code analysis, functional GUI testing and GUI image comparison techniques can see'' the GUI animations on the scree, and thus they cannot support the linting of GUI animations against design-don't guidelines. In this work, we formulate this GUI animation linting problem as a multi-class screencast classification task, but we do not have sufficient labeled GUI animations to train the classifier. Instead, we propose an unsupervised, computer-vision based adversarial autoencoder to solve this linting problem. Our autoencoder learns to group similar GUI animations by
seeing'' lots of unlabeled real-application GUI animations and learning to generate them. As the first work of its kind, we build the datasets of synthetic and real-world GUI animations. Through experiments on these datasets, we systematically investigate the learning capability of our model and its effectiveness and practicality for linting GUI animations, and identify the challenges in this linting problem for future work.
Thu 9 JulDisplayed time zone: (UTC) Coordinated Universal Time change
00:00 - 01:00 | P14-TestingTechnical Papers / Software Engineering in Practice at Goguryeo Chair(s): Shin Yoo Korea Advanced Institute of Science and Technology | ||
00:00 12mTalk | Seenomaly: Vision-Based Linting of GUI Animation Effects Against Design-Don’t GuidelinesTechnical Technical Papers Dehai Zhao Australian National University, Zhenchang Xing Australia National University, Chunyang Chen Monash University, Xiwei (Sherry) Xu Data 61, Liming Zhu CSIRO's Data61 and UNSW, Guoqiang Li Shanghai Jiao Tong University, Jinshui Wang School of Information Science and Engineering, Fujian University of Technology, Fuzhou, China | ||
00:12 12mTalk | Fuzz Testing based Data Augmentation to Improve Robustness of Deep Neural NetworksTechnical Technical Papers Xiang Gao National University of Singapore, Singapore, Ripon Saha Fujitsu Laboratories of America, Inc., Mukul Prasad Fujitsu Laboratories of America, Abhik Roychoudhury National University of Singapore, Singapore | ||
00:24 12mTalk | Modeling and Ranking Flaky Tests at AppleSEIP Software Engineering in Practice Emily Kowalczyk Apple Inc., Karan Nair Apple, Zebao Gao Apple, Leopold Silberstein Apple Inc., Teng Long Apple, Atif Memon Apple Inc. | ||
00:36 12mTalk | Testing File System Implementations on Layered ModelsTechnical Technical Papers Dongjie Chen Nanjing University, Yanyan Jiang Nanjing University, Chang Xu Nanjing University, Xiaoxing Ma Nanjing University, Jian Lu Nanjing University | ||
00:48 12mTalk | A Cost-efficient Approach to Building in Continuous IntegrationTechnical Technical Papers Pre-print |