AI Conversational Tutors in Foreign Language Learning: A Mixed-Methods Evaluation Study
Preprint
Abstract
This paper focuses on AI tutors in foreign language learning, a field of application of AI tutors with great development, especially during the last years, when great advances in natural language understanding and processing in real time, have been achieved. These tutors attempt to address needs for improving language skills (speaking, or communicative competence, understanding). In this paper, a mixed-methos empirical study on the use of different kinds of state-of-the-art AI tutors for language learning is reported. This study involves a user experience evaluation of typical such tools, with special focus in their conversation functionality and an evaluation of their quality, based on chat transcripts. This study can help establish criteria for assessing the quality of such systems and inform the design of future tools, including concerns about data privacy and secure handling of learner information.Keywords
All Related Versions
This publication has 9 references indexed in Scilit:
- An LLM-based hybrid approach for enhanced automated essay scoringScientific Reports, 2025
- Ethical and Privacy Considerations in AI-Driven Language LearningLatIA, 2025
- Pedagogical AI conversational agents in higher education: a conceptual framework and survey of the state of the artEducational Technology Research and Development, 2025
- Reimagining language learning: AI-driven innovations for engagement and growthTechnology in Language Teaching & Learning, 2024
- EFL students' and teachers' perceptions of the ethical uses of AI toolsTechnology in Language Teaching & Learning, 2024
- Transforming language education: A systematic review of AI-powered chatbots for English as a foreign language speaking practiceComputers and Education: Artificial Intelligence, 2024
- Role of AI chatbots in education: systematic literature reviewInternational Journal of Educational Technology in Higher Education, 2023
- Gamification in mobile-assisted language learning: a systematic review of Duolingo literature from public release of 2012 to early 2020Computer Assisted Language Learning, 2021
- How NOT To Evaluate Your Dialogue System: An Empirical Study of Unsupervised Evaluation Metrics for Dialogue Response Generation2016