Write a Blog >>
ICSE 2020
Wed 24 June - Thu 16 July 2020
Fri 10 Jul 2020 08:53 - 09:01 at Goguryeo - I23-Code Artifact Analysis Chair(s): Benoit Baudry

App stores include a vast amount of user feedback in the form of app ratings and reviews. Research and recently also tool vendors have proposed data mining and data analytics solutions to leverage this feedback to developers and analysts, e.g., for extracting requirements-related information, monitoring the opinions of users on apps’ features, or informing release decisions. Research also showed that positive feedback improves apps’ downloads and sales figures and is thus fundamental to the success of apps.

As a side effect, a market for fake, incentivized app reviews emerged with yet unclear consequences for developers, app users, and app store operators. Fake reviews are prohibited in popular app stores, such as in Google Play or Apple App Store. Even governmental competition authorities started taking actions against companies using fake reviews to embellish their apps. For instance, the Canadian telecommunication provider Bell was fined $1.25 million for faking positive reviews to their apps.

Our journal-first paper reports on a study of fake reviews, their providers, characteristics, and how accurately they can be automatically detected. (1) We identified and conducted disguised questionnaires with 43 fake review providers and studied their review policies to understand their strategies and offers. We found that developers buy reviews to relatively expensive prices of a few dollars or deal with reviews in exchange portals (i.e., “I will write a positive review to your app if you do the same for my app”). The questionnaires and policies revealed that fake reviews are written to look authentic and are hard to recognize by humans. (2) By identifying and comparing 60,000 fake reviews with 62 million reviews from the Apple App Store, we found significant differences, e.g., between the corresponding apps, reviewers, rating distribution, and frequency. (3) This inspired the development of a simple classifier to automatically detect potential fake reviews in app stores. Based on the identified differences between fake and regular reviews, we developed, trained, fine-tuned, and compared multiple supervised machine learning approaches. (4) To have a more realistic setting of how our classifier can perform in practice, we conducted an in-the-wild experiment by varying the skewness of our dataset. On a labelled and imbalanced dataset, including one-tenth of fake reviews, as reported in other domains, our classifier achieved a recall of 91% and an AUC/ROC value of 98%.

We publicly share our gold-standard fake reviews dataset to enable the development of more accurate classifiers for identifying fake reviews or fake reviews candidates.

We discuss our findings and their impact on software engineering, app users, and app store operators. Although information extracted from app reviews is getting increasingly integrated into the requirements engineering process, none of the previous works in the research area app store analysis have considered fake reviews and their implications. By applying an existing app store analysis approach, we showed that requirements-related feedback, in the form of bug reports and feature requests, is included in both fake and regular reviews. Ignoring fake reviews might lead to wrong assumptions about actual user needs. Not optimal decisions for future development might be drawn too. Moreover, fake reviews might damage the integrity of app stores. Recently, Google highlighted the negative effects of fake reviews in an official statement and explicitly asked developers not to buy, and users not to provide fake reviews.

Conference Day
Fri 10 Jul

Displayed time zone: (UTC) Coordinated Universal Time change

08:05 - 09:05
I23-Code Artifact AnalysisJournal First / Technical Papers at Goguryeo
Chair(s): Benoit BaudryKTH Royal Institute of Technology
08:05
12m
Talk
Conquering the Extensional Scalability Problem for Value-Flow Analysis FrameworksTechnical
Technical Papers
Qingkai ShiThe Hong Kong University of Science and Technology, Rongxin WuDepartment of Cyber Space Security, Xiamen University, Gang FanHong Kong University of Science and Technology, Charles ZhangThe Hong Kong University of Science and Technology
08:17
12m
Talk
Pipelining Bottom-up Data Flow AnalysisTechnical
Technical Papers
Qingkai ShiThe Hong Kong University of Science and Technology, Charles ZhangThe Hong Kong University of Science and Technology
08:29
8m
Talk
An Empirical Validation of Oracle ImprovementJ1
Journal First
Gunel JahangirovaUniversità della Svizzera italiana, David ClarkUniversity College London, Mark Harman, Paolo TonellaUniversità della Svizzera italiana
08:37
8m
Talk
Is Static Analysis Able to Identify Unnecessary Source Code?J1
Journal First
Roman HaasCQSE GmbH, Rainer NiedermayrCQSE GmbH, Tobias RoehmCQSE GmbH, Sven ApelSaarland University
Pre-print
08:45
8m
Talk
Memory and Resource Leak Defects and Their Repairs in Java ProjectsJ1
Journal First
Mohammadreza GhanavatiHeidelberg University, Diego CostaConcordia University, Canada, Janos SeboekHeidelberg University, David LoSingapore Management University, Artur AndrzejakHeidelberg University
08:53
8m
Talk
Towards Understanding and Detecting Fake Reviews in App StoresJ1
Journal First
Daniel MartensUniversity of Hamburg, Walid MaalejUniversity of Hamburg