Write a Blog >>
ICSE 2020
Wed 24 June - Thu 16 July 2020
Tue 7 Jul 2020 07:12 - 07:20 at Baekje - I1-Metastudies Chair(s): Michael Vierhauser

We propose a robustness testing approach for software systems that process large amounts of data. Our method uses metamorphic relations to check software output for erroneous input in the absence of a tangible test oracle. We use this technique to test two major citation database systems: Scopus and the Web of Science. We report a surprising finding that the inclusion of hyphens in paper titles impedes citation counts, and that this is a result of the lack of robustness of the citation database systems in handling hyphenated paper titles. Our results are valid for the entire literature as well as for individual fields such as chemistry. We further find a strong and significant negative correlation between the journal impact factor (JIF) of IEEE Transactions on Software Engineering (TSE) and the percentage of hyphenated paper titles published in TSE. Similar results are found for ACM Transactions on Software Engineering and Methodology. A software engineering field-wide study reveals that the higher JIF-ranked journals are publishing a lower percentage of papers with hyphenated titles. Our results challenge the common belief that citation counts and JIFs are reliable measures of the impact of papers and journals, as they can be distorted simply by the presence of hyphens in paper titles.

Tue 7 Jul

Displayed time zone: (UTC) Coordinated Universal Time change

07:00 - 08:00
I1-MetastudiesNew Ideas and Emerging Results / Technical Papers / Journal First / Demonstrations at Baekje
Chair(s): Michael Vierhauser Johannes Kepler University Linz
07:00
12m
Talk
An Evidence-Based Inquiry into the Use of Grey Literature in Software EngineeringTechnical
Technical Papers
He Zhang Nanjing University, Xin Zhou State Key Laboratory of Novel Software Technology, Software Institute, Nanjing University, Xin Huang State Key Laboratory of Novel Software Technology, Software Institute, Nanjing University, Huang Huang State Key Laboratory of Novel Software Technology, Software Institute, Nanjing University, Muhammad Ali Babar The University of Adelaide
07:12
8m
Talk
An Extended Abstract of "Metamorphic Robustness Testing: Exposing Hidden Defects in Citation Statistics and Journal Impact Factors"J1
Journal First
Zhi Quan (George) Zhou University of Wollongong, Australia, T.H. Tse The University of Hong Kong, Matt Witheridge University of Wollongong
07:20
8m
Talk
A Procedure and Guidelines for Analyzing Groups of Software Engineering ReplicationsJ1
Journal First
Adrian Santos Parrilla University of Oulu, Sira Vegas Universidad Politecnica de Madrid, Markku Oivo University of Oulu, Natalia Juristo Universidad Politecnica de Madrid
07:28
3m
Talk
An SLR-Tool: Search Process in PracticeDemo
Demonstrations
Andreas Hinderks University of Seville, Francisco José Domínguez Mayo University of Seville, Jörg Thomaschewski University of Applied Sciences Emden/Leer, María José Escalona University of Seville
07:31
6m
Talk
Threats to Validity in Experimenting Mutation-Based Fault LocalizationNIER
New Ideas and Emerging Results
Juyoung Jeon Handong Global University, Shin Hong Handong Global University
07:37
8m
Talk
Methodological Principles for Reproducible Performance Evaluation in Cloud ComputingJ1
Journal First
Alessandro Vittorio Papadopoulos Mälardalen University, Laurens Versluis Vrije Universiteit Amsterdam, André Bauer University of Würzburg, Nikolas Herbst University of Würzburg, Joakim von Kistowski University of Würzburg, Ahmed Ali-Eldin UMass Amherst, Cristina L. Abad Escuela Superior Politecnica del Litoral, Jose Nelson Amaral University of Alberta, Petr Tuma Charles University, Alexandru Iosup Vrije Universiteit Amsterdam
07:45
8m
Talk
Bayesian Data Analysis in Empirical Software Engineering ResearchJ1
Journal First
Carlo A. Furia Università della Svizzera italiana (USI), Robert Feldt Chalmers | University of Gothenburg, Blekinge Institute of Technology, Richard Torkar Chalmers and the University of Gothenburg
DOI Pre-print