Write a Blog >>
ICSE 2020
Wed 24 June - Thu 16 July 2020

Can automated adaptive feedback for correcting erroneous programs help novice programmers learn to code better? In a large-scale experiment, we compare student performance when tutored by human tutors, and when receiving automated adaptive feedback. The automated feedback was designed using one of two well-known instructional principles: (i) presenting the correct solution for the immediate problem, or (ii) presenting generated examples or analogies that guide towards the correct solution. We report empirical results from a large-scale (10,000+ person hour) experiment assessing the efficacy of these automated compilation-error feedback tools. Using the survival analysis on error rates of students measured over seven weeks, we found that automated feedback allows students to resolve errors in their code more efficiently than students receiving manual feedback. However, we also found that this advantage is primarily logistical and not conceptual; the performance benefit seen during lab assignments disappeared during exams wherein feedback of any kind was withdrawn. We further found that the performance advantage of automated feedback over human tutors increases with problem complexity, and that feedback via example and specific repair have distinct, non-overlapping relative advantages for different categories of programming errors. Our results offer a clear and granular delimitation of the pedagogical benefits of automated feedback in teaching programming to novices.

Conference Day
Thu 9 Jul

Displayed time zone: (UTC) Coordinated Universal Time change

07:00 - 08:00
I14-SEET - Industry Relevant Teaching and OthersSoftware Engineering Education and Training at Goguryeo
Chair(s): Letizia JaccheriNorwegian University of Science and Technology
07:00
12m
Talk
Addressing the Double Challenge of Learning and Teaching Enterprise Technologies through Peer TeachingSEET
Software Engineering Education and Training
Richard GlasseyKTH Royal Institute of Technology, Olle BälterKTH Royal Institute of Technology, Philipp HallerKTH Royal Institute of Technology, Mattias WiggbergKTH Royal Institute of Technology
07:12
12m
Talk
Characterizing the Pedagogical Benefits of Adaptive Feedback for Compilation Errors by Novice ProgrammersSEET
Software Engineering Education and Training
Umair Z. AhmedNational University of Singapore, Nisheeth SrivastavaIndian Institute of Technology, Kanpur, Renuka SindhgattaQueensland University of Technology, Australia, Amey KarkareIIT Kanpur
07:24
12m
Talk
VeriSIM: A Learning Environment for Comprehending Class and Sequence Diagrams using Design TracingSEET
Software Engineering Education and Training
Prajish PrasadIIT Bombay, Sridhar IyerIIT Bombay
07:36
12m
Talk
Towards an Open Repository for Teaching Software Modeling applying Active Learning StrategiesSEET
Software Engineering Education and Training
Williamson SilvaUFAM, Bruno GadelhaUFAM, Igor SteinmacherNorthern Arizona University, Tayana ConteUniversidade Federal do Amazonas
07:48
12m
Talk
What prevents Finnish women from applying to software engineering roles? A preliminary analysis of survey dataSEET
Software Engineering Education and Training
Annika WolffLUT University, Antti KnutasLUT University, Paula SavolainenTurku University of Applied Sciences
Pre-print