Leveraging text mining and analytic hierarchy process for the automatic evaluation of online courses
Open Access
- 21 May 2024
- journal article
- research article
- Published by Springer Nature in International Journal of Machine Learning and Cybernetics
- Vol. 15 (11) , 4973-4998
Abstract
This study introduced a multi-criteria decision-making methodology leveraging text mining and analytic hierarchy process (AHP) for online course quality evaluation based on students’ feedback texts. First, a hierarchical structure of online course evaluation criteria was formulated by integrating topics (sub-criteria) identified through topic modeling and interpreted based on transactional distance and technology acceptance theories. Second, the weights of the criteria in the hierarchical structure were determined based on topic proportions. Third, the AHP was employed to determine the overall relative advantage of online courses and their relative advantage within each criterion based on the hierarchical framework and criterion weights. The proposed approach was implemented on the datasets of 6940 reviews for knowledge-seeking courses in Art, Design, and Humanities (D1) and 44,697 reviews for skill-seeking courses in Computer Science, Engineering, and Programming (D2) from Class Central to determine ranking positions of nine courses from both D1 and D2 as alternatives. Results revealed common concerns among knowledge and skill-seeking course learners, encompassing “assessment”, “content”, “effort”, “usefulness”, “enjoyment”, “faculty”, “interaction”, and “structure”. The article provides valuable insights into the online course evaluation and selection processes for learners in D1 and D2 groups. Notably, both groups prioritize “effort” and “faculty”, while D2 learners value “assessment” and “enjoyment”, and D1 learners value “usefulness” more. This study demonstrates the efficacy of leveraging online learner reviews and topic modeling for automating MOOC evaluation and informing learners’ decision-making processes.Keywords
Funding Information
- National Natural Science Foundation of China (62307010)
- Lingnan University, Hong Kong (Direct Grant (DR23B2) and Faculty Research Grants (DB23A3 and DB23B2))
This publication has 62 references indexed in Scilit:
- Deep neural networks for the automatic understanding of the semantic content of online course reviewsEducation and Information Technologies, 2023
- Exploring students’ self-directed learning strategies and satisfaction in online learningEducation and Information Technologies, 2023
- What Factors Influence Students Satisfaction in Massive Open Online Courses? Findings from User-Generated Content Using Educational Data MiningEducation and Information Technologies, 2022
- Determinants affecting teachers’ adoption of AI-based applications in EFL context: An analysis of analytic hierarchy processEducation and Information Technologies, 2022
- A computational literature review of football performance analysis through probabilistic topic modelingArtificial Intelligence Review, 2021
- Self-regulated learning and satisfaction: A key determinants of MOOC successEducation and Information Technologies, 2021
- Design and implementation of a SMART Learning environment for the Upskilling of Cybersecurity professionals in MauritiusEducation and Information Technologies, 2021
- Instructor Social Presence Effects on Learner Social Presence, Achievement, and SatisfactionTechTrends, 2018
- MOOC Video Interaction Patterns: What Do They Tell Us?2015
- The Adult Student and Course Satisfaction: What Matters Most?Innovative Higher Education, 2011