On the use of resubmissions in automatic assessment systems
Authors:
- Ville KaravirtaVKVille Karavirta
- Ari KorhonenAKAri Korhonen
- Lauri MalmiLMLauri Malmi
- 1 September 2006
- journal article
- Published by Taylor & Francis in Computer Science Education
- Vol. 16 (3) , 229-240
Abstract
Automatic assessment systems generally support immediate grading and response on learners'submissions. They also allow learners to consider the feedback, revise, and resubmit their solutions. Several strategies exist to implement the resubmission policy. The ultimate goal, however, is to improve the learning outcomes, and thus the strategies should aim at preventing learners from using the resubmission feature irresponsibly. One of the key questions here is how to develop the system and its use in order to cut down such reiteration that does not seem to be worthwhile? In this paper, we study data gathered from an automatic assessment system that supports resubmissions. We use a clustering technique to draw a distinction among learner groups that seem to differ in their use of the resubmission feature and the points achieved from the exercises. By comparing these groups with each other, we conclude that for a small minority of learners there is a risk that they use the resubmission inefficiently. Some learners seem to resubmit the solution without thinking much between two consecutive submissions. In order to prevent such an aimless trial-and-error problem solving method, one option is to limit the number of allowed resubmissions. However, not all resubmissions are bad. In addition, there exist several ways to realize the limitations to achieve the best possible resubmission policy fit for all the students. These are discussed based on the evidence gathered during the research.Keywords
This publication has 13 references indexed in Scilit:
- A Survey of Automated Assessment Approaches for Programming AssignmentsComputer Science Education, 2005
- How shall we assess this?ACM SIGCSE Bulletin, 2003
- Interaction and Feedback in Automatically Assessed Algorithm Simulation ExercisesJournal of Information Technology Education: Research, 2003
- Does it make a difference if students exercise on the web or in the classroom?ACM SIGCSE Bulletin, 2002
- Using Evaluation to Shape ITS Design: Results and Experiences with SQL-TutorUser Modeling and User-Adapted Interaction, 2002
- Fully automatic assessment of programming exercisesACM SIGCSE Bulletin, 2001
- A secure on-line submission systemSoftware: Practice and Experience, 1999
- Automatic assessment aids for Pascal programsACM SIGPLAN Notices, 1982
- Automatic grading programsCommunications of the ACM, 1965
- Automatic grading of students' ALGOL programmingBIT Numerical Mathematics, 1964