Human versus machine: investigating L2 learner output in face-to-face versus fully automated role-plays
- 21 February 2022
- journal article
- research article
- Published by Taylor & Francis in Computer Assisted Language Learning
- Vol. 37 (1-2) , 149-178
Abstract
To examine the utility of spoken dialog systems (SDSs) for learning and low-stakes assessment, we administered the same role-play task in two different modalities to a group of 47 tertiary-level learners of English. Each participant completed the task in an SDS setting with a fully automated agent and engaged in the same task with a human interlocutor in a face-to-face format. Additionally, we gauged students’ perceptions of the two delivery formats. Elicited oral performances were examined for linguistic complexity (syntactic complexity, lexical variety, fluency) and pragmatic functions (number and type of requests). Learner performance data across the two delivery modes were comparable although learners spoke slightly longer in the SDS task and used significantly more turns in the face-to-face setting—a finding that may be due to participants deploying more social rapport building moves, clarification requests, and backchanneling. The attitudinal data indicate that, while many learners liked both delivery formats, there was a slight preference for the face-to-face format, mainly due to the presence of body language. Overall, results show that fully automated SDS tasks may constitute a feasible alternative to face-to-face role-plays. Nevertheless, when possible, learners should be given a choice in task format for both learning and assessment.Keywords
This publication has 51 references indexed in Scilit:
- Interactive Language Learning through Speech-Enabled Virtual ScenariosAdvances in Human-Computer Interaction, 2012
- A Corpus‐Based Evaluation of Syntactic Complexity Measures as Indices of College‐Level ESL Writers' Language DevelopmentTESOL Quarterly, 2011
- Can Talk Be Inconsequential? Social and Interactional Aspects of Elicited Second‐Language InteractionThe Modern Language Journal, 2010
- Adjusting Expectations: The Study of Complexity, Accuracy, and Fluency in Second Language AcquisitionApplied Linguistics, 2009
- A Study of web-based oral activities enhanced by Automatic Speech Recognition for EFL college learningComputer Assisted Language Learning, 2007
- Symposium article: Data generation through role-play: Assessing oral proficiencySimulation & Gaming, 2007
- Language Learner Behaviour in a Virtual EnvironmentComputer Assisted Language Learning, 2003
- Simulating conversations in oral-proficiency assessment: a conversation analysis of role plays and non-scripted interviews in language examsLanguage Testing, 1999
- Research Methods in Interlanguage PragmaticsStudies in Second Language Acquisition, 1991
- Motivational variables in second language acquisition: A study of Francophones learning English.Canadian Journal of Behavioural Science / Revue canadienne des sciences du comportement, 1977