A Systematic Review of the Effects of Automatic Scoring and Automatic Feedback in Educational Settings
Top Cited Papers
Open Access
- 28 July 2021
- journal article
- review article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 9, 108190-108198
Abstract
Automatic scoring and feedback tools have become critical components of online learning proliferation. These tools range from multiple-choice questions to grading essays using machine learning (ML). Learning environments such as massive open online courses (MOOCs) would not be possible without them. The usage of this mechanism has brought many exciting areas of study, from the design of questions to the ML grading tools’ precision and accuracy. This paper analyzes the findings of 125 studies published in journals and proceedings between 2016 and 2020 on the usages of automatic scoring and feedback as a learning tool. This analysis gives an overview of the trends, challenges, and open questions in this research area. The results indicate that automatic scoring and feedback have many advantages. The most important benefits include enabling scaling the number of students without adding a proportional number of instructors, improving the student experience by reducing the time between submission grading and feedback, and removing bias in scoring. On the other hand, these technologies have some drawbacks. The main problem is creating a disincentive to develop innovative answers that do not match the expected one or have not been considered when preparing the problem. Another drawback is potentially training the student to answer the question instead of learning the concepts. With this, given the existence of a correct answer, such an answer could be leaked to the internet, making it easier for students to avoid solving the problem. Overall, each of these drawbacks presents an opportunity to look at ways to improve technologies to use these tools to provide a better learning experience to students.Keywords
Funding Information
- PLeNTaS Project “Proyectos I+D+i 2019” (PID2019-111430RB-I00)
- PL-NETO Project “Proyecto PROPIO UNIR” (B0036)
This publication has 102 references indexed in Scilit:
- Automated Scoring of Chinese Engineering Students' English EssaysInternational Journal of Distance Education Technologies, 2017
- Pedagogy and Design of Online Learning Environment in Computer Science Education for High SchoolsInternational Journal of Online Pedagogy and Course Design, 2016
- STAGE2016
- Measuring code behavioral similarity for programming and software engineering education2016
- Student behavior in a web-based educational system: Exit intent predictionEngineering Applications of Artificial Intelligence, 2016
- Research on the Application of Automatic Scoring System in College English Writing2016
- Smart Quizzes in the Engineering Education2016
- Students’ feedback preferences: how do students react to timely and automatically generated assessment feedback?Assessment & Evaluation in Higher Education, 2014
- Peer feedback to facilitate project-based learning in an online environmentThe International Review of Research in Open and Distributed Learning, 2013
- Lessons from applying the systematic literature review process within the software engineering domainJournal of Systems and Software, 2007