ChatGPT in the Classroom: An Analysis of Its Strengths and Weaknesses for Solving Undergraduate Computer Science Questions
Top Cited Papers
Open Access
- 7 March 2024
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 625-631
Abstract
This research paper aims to analyze the strengths and weaknesses associated with the utilization of ChatGPT as an educational tool in the context of undergraduate computer science education. ChatGPT's usage in tasks such as solving assignments and exams has the potential to undermine students' learning outcomes and compromise academic integrity. This study adopts a quantitative approach to demonstrate the notable unreliability of ChatGPT in providing accurate answers to a wide range of questions within the field of undergraduate computer science. While the majority of existing research has concentrated on assessing the performance of Large Language Models in handling programming assignments, our study adopts a more comprehensive approach. Specifically, we evaluate various types of questions such as true/false, multi-choice, multi-select, short answer, long answer, design-based, and coding-related questions. Our evaluation highlights the potential consequences of students excessively relying on ChatGPT for the completion of assignments and exams, including self-sabotage. We conclude with a discussion on how can students and instructors constructively use ChatGPT and related tools to enhance the quality of instruction and the overall student experience.Keywords
All Related Versions
This publication has 23 references indexed in Scilit:
- ChatGPT, Can You Generate Solutions for my Coding Exercises? An Evaluation on its Effectiveness in an undergraduate Java Programming Course.2023
- Comparing Code Explanations Created by Students and Large Language Models2023
- Evaluating the Performance of Code Generation Models for Solving Parsons Problems With Small Prompt Variations2023
- Can Generative Pre-trained Transformers (GPT) Pass Assessments in Higher Education Programming Courses?2023
- A SWOT analysis of ChatGPT: Implications for educational practice and researchInnovations in Education and Teaching International, 2023
- Automatic Generation of Programming Exercises and Code Explanations Using Large Language Models2022
- Developing and Assessing Educational Games to Enhance Cyber Security Learning in Computer Science2019
- Goal Setting and Self-regulated Experiential Learning in a Paired Internship Program2019
- Impact of Open-Ended Assignments on Student Self-Efficacy in CS12019
- Does Creating Programming Assignments with Tests Lead to Improved Performance in Writing Unit Tests?2019