The role of generative AI tools in shaping mechanical engineering education from an undergraduate perspective
Top Cited Papers
Open Access
- 17 March 2025
- journal article
- research article
- Published by Springer Nature in Scientific Reports
- Vol. 15 (1) , 1-14
Abstract
This study evaluates the effectiveness of three leading generative AI tools-ChatGPT, Gemini, and Copilot-in undergraduate mechanical engineering education using a mixed-methods approach. The performance of these tools was assessed on 800 questions spanning seven core subjects, covering multiple-choice, numerical, and theory-based formats. While all three AI tools demonstrated strong performance in theory-based questions, they struggled with numerical problem-solving, particularly in areas requiring deep conceptual understanding and complex calculations. Among them, Copilot achieved the highest accuracy (60.38%), followed by Gemini (57.13%) and ChatGPT (46.63%). To complement these findings, a survey of 172 students and interviews with 20 participants provided insights into user experiences, challenges, and perceptions of AI in academic settings. Thematic analysis revealed concerns regarding AI’s reliability in numerical tasks and its potential impact on students’ problem-solving abilities. Based on these results, this study offers strategic recommendations for integrating AI into mechanical engineering curricula, ensuring its responsible use to enhance learning without fostering dependency. Additionally, we propose instructional strategies to help educators adapt assessment methods in the era of AI-assisted learning. These findings contribute to the broader discussion on AI’s role in engineering education and its implications for future learning methodologies.Keywords
Funding Information
- Birla Institute of Technology and Science, Pilani
This publication has 51 references indexed in Scilit:
- Generative AI in education and research: A systematic mapping reviewReview of Education, 2024
- ChatGPT‐3.5 and ‐4.0 and mechanical engineering: Examining performance on the FE mechanical engineering and undergraduate examsComputer Applications in Engineering Education, 2024
- ChatGPT as an educational resource for civil engineering studentsComputer Applications in Engineering Education, 2024
- We still need to teach engineers to write in the era ofChatGPTJournal of Engineering Education, 2023
- Envisioning the future of learning and teaching engineering in the artificial intelligence era: Opportunities and challengesJournal of Engineering Education, 2023
- The Possibility of Applying ChatGPT (AI) for Calculations in Mechanical Engineering2023
- Exploring the use of large language models (LLMs) in chemical engineering education: Building core course problem models with Chat-GPTEducation for Chemical Engineers, 2023
- The role of frameworks in engineering education researchJournal of Engineering Education, 2021
- The use of engineering model‐building activities to elicit computational thinking: A design‐based research studyJournal of Engineering Education, 2021
- Quantitative, Qualitative, and Mixed Research Methods in Engineering EducationJournal of Engineering Education, 2009