Inspecting Spoken Language Understanding from Kids for Basic Math Learning at Home
Preprint
Abstract
Enriching the quality of early childhood education with interactive math learning at home systems, empowered by recent advances in conversational AI technologies, is slowly becoming a reality. With this motivation, we implement a multimodal dialogue system to support play-based learning experiences at home, guiding kids to master basic math concepts. This work explores Spoken Language Understanding (SLU) pipeline within a task-oriented dialogue system developed for Kid Space, with cascading Automatic Speech Recognition (ASR) and Natural Language Understanding (NLU) components evaluated on our home deployment data with kids going through gamified math learning activities. We validate the advantages of a multi-task architecture for NLU and experiment with a diverse set of pretrained language representations for Intent Recognition and Entity Extraction tasks in the math learning domain. To recognize kids' speech in realistic home environments, we investigate several ASR systems, including the commercial Google Cloud and the latest open-source Whisper solutions with varying model sizes. We evaluate the SLU pipeline by testing our best-performing NLU models on noisy ASR output to inspect the challenges of understanding children for math learning in authentic homes.Keywords
All Related Versions
This publication has 43 references indexed in Scilit:
- ChatGPT for good? On opportunities and challenges of large language models for educationLearning and Individual Differences, 2023
- Chatting and cheating: Ensuring academic integrity in the era of ChatGPTInnovations in Education and Teaching International, 2023
- Natural Language Interactions in Autonomous Vehicles: Intent Detection and Slot Filling from Passenger Utterances2023
- Can guidance during play enhance children’s learning and development in educational contexts? A systematic review and meta-analysisChild Development, 2022
- Exploring Kid Space in the wild: a preliminary study of multimodal and immersive collaborative play-based learning experiencesEducational Technology Research and Development, 2022
- A Multimodal Human-Computer Interaction System and Its Application in Smart Learning Environments2020
- Kid Space2018
- Artificial Intelligence trends in education: a narrative overviewProcedia Computer Science, 2018
- Acoustic variability and automatic recognition of children’s speechSpeech Communication, 2007
- AutoTutor: An Intelligent Tutoring System With Mixed-Initiative DialogueIEEE Transactions on Education, 2005