An experience of automated assessment in a large‐scale introduction programming course
- 21 January 2021
- journal article
- research article
- Published by Wiley in Computer Applications in Engineering Education
- Vol. 29 (5) , 1284-1299
Abstract
The 2020 pandemic imposed new demands on teaching practices to support student's distance learning process. In this context, automated assessment (AA) is a pivotal resource that offers immediate and automatic feedback on students' programming tasks. Although the literature provides several contributions regarding AA of Programming Exercises (PEs), very few works discuss the automatic generation of personalized PE. This study reports our experience in applying a new proposal for AA‐PE in an Introduction to Programming (IP) course for a large group of students. This proposal's key feature is the ability to apply AA‐PE and parameterized unified exams to different programming languages by using the open‐source tools MCTest and Moodle (with virtual programming lab [VPL] plugin). During the first quarter of 2019, teachers of 19 of 44 IP‐FF (face‐to‐face) classes embraced our approach as a component in their pedagogical intervention. These classes achieved a higher pass rate (67.5%) than those that did not adopt our AA solution (59.1%), whereas the standard deviation was quite the same (22.5% and 21.3%, respectively). Additionally, preliminary results revealed a strong linear correlation (r = .93) between the pass rate and the average grade of the AA‐PE. In IP‐BL (blended learning), two classes used our method in the exams, with 171 students and a pass rate of 70.4%. These results corroborate previous works that continuous assessment combined with immediate feedback can contribute to students' learning process.Keywords
Funding Information
- Fundação de Amparo à Pesquisa do Estado de São Paulo (#2018/23561‐1)
This publication has 22 references indexed in Scilit:
- Computerized Adaptive Test (CAT) Applications and Item Response Theory Models for Polytomous ItemsInternational Journal of Research in Education and Science, 2017
- An automatic generator and corrector of multiple choice tests with random answer keys2016
- Evaluation of Software Education Using Auto-generated Exercises2016
- CoderunnerACM Inroads, 2016
- Automatic generation of programming exercises for learning programming language2015
- Experimentation and analysis of undergraduate students performance and satisfaction in a blended model of an introductory computer science and programming course2014
- Student perception and usage of an automated programming assessment toolComputers in Human Behavior, 2014
- Autogenerator: Generation and execution of programming code on demandExpert Systems with Applications, 2013
- Ability-training-oriented automated assessment in introductory programming courseComputers & Education, 2011
- Automated Assessment in a Programming Tools CourseIEEE Transactions on Education, 2010