Write a Blog >>
ICSE 2020
Wed 24 June - Thu 16 July 2020
Tue 7 Jul 2020 08:33 - 08:45 at Silla - I6-Empirical Studies and Requirements Chair(s): Ita Richardson

Identifying and optimizing open participation is essential to the success of open software development. Existing studies highlighted the importance of worker recommendation for crowdtesting tasks in order to detect more bugs with fewer workers. However, these studies mainly provide one-time recommendations, i.e., recommending a set of workers for a new task with its fixed, initial context at the beginning of the task. We argue the need for in-process crowdtesting worker recommendation. We motivate this study through a pilot study, revealing the prevalence of long-sized non-yielding windows, i.e., no new bugs are revealed in consecutive test reports during the process of a crowdtesting task. This indicates the potential opportunity for accelerating test cycle by recommending appropriateworkers in a dynamic manner, so that the non-yielding windows could be shortened. This paper proposes a context-aware in-process crowdworker recommendation approach, R3Rec, to dynamically identify and rank a diverse set of capable crowdworkers, in order to detect more bugs earlier and potentially shorten the non-yielding windows. It consists of three main components: 1) the modeling of dynamic testing context in terms of process context and resource context to capture the in-process progress and worker characteristics respectively; 2) the learning-based ranking component to learn the probability of crowdworkers’ bug detection capability; and 3) the diversity-based re-ranking component to adjust the ranked list to potentially reduce the duplicate bugs. The evaluation is conducted on 636 crowdtesting tasks from one of the largest crowdtesting platforms, and the results showthat R3Rec can shorten the non-yielding windows by 50% -58% and reduce the total crowdtesting cost by about 10% on median. This is the first work to propose the in-process worker recommendation solution, and results showed its potential in improving the costeffectiveness of crowdtesting by saving the cost and shortening the testing cycle.

Tue 7 Jul
Times are displayed in time zone: (UTC) Coordinated Universal Time change

08:05 - 09:05: Paper Presentations - I6-Empirical Studies and Requirements at Silla
Chair(s): Ita RichardsonLero - The Irish Software Research Centre and University of Limerick
icse-2020-Journal-First08:05 - 08:13
Junxiao HanZhejiang University, Emad ShihabConcordia University, Zhiyuan WanZhejiang University, Shuiguang DengZhejiang University, Xin XiaMonash University
icse-2020-papers08:13 - 08:25
Lin ShiISCAS, Mingzhe XingISCAS, Mingyang LiISCAS, Yawen WangISCAS, Shoubin LiISCAS, Qing WangInstitute of Software, Chinese Academy of Sciences
icse-2020-Journal-First08:25 - 08:33
Miroslaw OchodekPoznan University of Technology, Regina HebigChalmers University of Technology & University of Gothenburg, Wilhelm MedingEricsson, Gert FrostGrundfos, Miroslaw StaronUniversity of Gothenburg
icse-2020-papers08:33 - 08:45
Junjie WangInstitute of Software, Chinese Academy of Sciences, Ye YangStevens institute of technology, Song WangYork University, Yuanzhe HuInstitute of Software, Chinese Academy of Sciences, Dandan WangInstitute of Software, Chinese Academy of Sciences, Qing WangInstitute of Software, Chinese Academy of Sciences
icse-2020-Software-Engineering-in-Practice08:45 - 08:57
Anton StrandEricsson AB, Markus GunnarssonEricsson AB, Ricardo BrittoEricsson / Blekinge Institute of Technology, Muhammad UsmanBlekinge Institute of Technology