Liam Hannah
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Liam Hannah has a total of 10 co-authors. Here are the top co-authors by number of shared publications:
- Eunice Eunhee Jang
University of Toronto, Canada
4 shared publications - Meng-Hsun Lee
University of Toronto, Ontario, Canada
2 shared publications Bruce Russell
Ontario Institute for Studies in Education, Canada
1 shared publication- Deborah Tihanyi
University of Toronto
1 shared publication - Heather Louise MacLean
University of Toronto
1 shared publication - Hyunah Kim
University of Toronto, Toronto
1 shared publication
Latest Publications
- Open Access
- 16 September 2025
- Published by Wiley in TESOL Quarterly
- Vol. 59 (S1)
Automated Diagnostic Feedback vs. Self‐Assessment: Rethinking Feedback Mechanisms on Academic Writing Development
As international students increasingly pursue higher education in English‐dominant countries, developing their academic writing skills is crucial. However, limited access to individualized feedback remains a challenge. AI‐driven tools and self‐assessment offer promising solutions, making feedback more accessible. This study involved 50 international graduate students who spoke English as an additional language, randomly assigned to two groups: one received BERT‐generated automated diagnostic scores and feedback, while the other engaged in self‐assessment. Using a sequential explanatory mixed‐methods design, this study investigated the effects of automated diagnostic feedback and self‐assessment on students' academic writing performance and self‐perceived writing abilities. It also explored how cognitive, metacognitive, behavioral, and affective engagement with feedback varied across the two groups. Results indicated that the machine feedback group significantly outperformed the self‐assessment group in task fulfillment, organization, and total writing scores, while no significant differences were observed for vocabulary and grammar. Additionally, the machine feedback group demonstrated deeper metacognitive, behavioral, and affective feedback engagement. However, they also reported a decline in academic writing self‐confidence. While automated diagnostic feedback proved more effective than self‐assessment in enhancing academic writing skills, its potential negative impact on students' confidence highlighted the need for future research to balance precision with emotional support.
- 16 September 2025
- Published by Wiley in TESOL Quarterly
- Vol. 59 (S1)
Open AccessAutomated Diagnostic Feedback vs. Self‐Assessment: Rethinking Feedback Mechanisms on Academic Writing Development
As international students increasingly pursue higher education in English‐dominant countries, developing their academic writing skills is crucial. However, limited access to individualized feedback remains a challenge. AI‐driven tools and self‐assessment offer promising solutions, making feedback more accessible. This study involved 50 international graduate students who spoke English as an additional language, randomly assigned to two groups: one received BERT‐generated automated diagnostic scores and feedback, while the other engaged in self‐assessment. Using a sequential explanatory mixed‐methods design, this study investigated the effects of automated diagnostic feedback and self‐assessment on students' academic writing performance and self‐perceived writing abilities. It also explored how cognitive, metacognitive, behavioral, and affective engagement with feedback varied across the two groups. Results indicated that the machine feedback group significantly outperformed the self‐assessment group in task fulfillment, organization, and total writing scores, while no significant differences were observed for vocabulary and grammar. Additionally, the machine feedback group demonstrated deeper metacognitive, behavioral, and affective feedback engagement. However, they also reported a decline in academic writing self‐confidence. While automated diagnostic feedback proved more effective than self‐assessment in enhancing academic writing skills, its potential negative impact on students' confidence highlighted the need for future research to balance precision with emotional support.
- Open Access
- 17 July 2025
- Published by SAGE Publications in Language Testing
- Vol. 42 (4) , 397-422
Investigating construct representativeness and linguistic equity of automated oral reading fluency assessment with prosody
While the automated assessment of oral reading fluency (ORF) using accuracy and speech rate has proliferated, expressiveness of speech, as measured by prosodic features, has been neglected due to its inherent complexity and lack of technological resources. Despite the potential benefits of burgeoning technology for assessing hard-to-measure constructs such as ORF, insensitivity to linguistic diversities threatens valid score interpretations and fair use for all learners. The present study investigated the potential benefits of developing an automated, prosody-inclusive ORF assessment in a post-secondary education setting involving many English language learners (ELLs). The analysis focused on three ways the inclusion of prosody may improve automated ORF assessments: by reducing bias against ELLs, improved prediction of reading comprehension, and improved diagnostic information. Data were analyzed by comparing two different scoring outcomes, the traditional ORF measure and a new prosody-inclusive outcome score, comparing these measures across language background. Results showed that the inclusion of prosody improves automated ORF assessment by reducing discrepancies between ELLs and English first language students caused by automated speech recognition inaccuracies, leads to better prediction of reading comprehension with ELLs, and provides meaningful diagnostic information. Detailed descriptions of the models, their relevance, and implications for the language testing community are discussed.
- 17 July 2025
- Published by SAGE Publications in Language Testing
- Vol. 42 (4) , 397-422
Open AccessInvestigating construct representativeness and linguistic equity of automated oral reading fluency assessment with prosody
While the automated assessment of oral reading fluency (ORF) using accuracy and speech rate has proliferated, expressiveness of speech, as measured by prosodic features, has been neglected due to its inherent complexity and lack of technological resources. Despite the potential benefits of burgeoning technology for assessing hard-to-measure constructs such as ORF, insensitivity to linguistic diversities threatens valid score interpretations and fair use for all learners. The present study investigated the potential benefits of developing an automated, prosody-inclusive ORF assessment in a post-secondary education setting involving many English language learners (ELLs). The analysis focused on three ways the inclusion of prosody may improve automated ORF assessments: by reducing bias against ELLs, improved prediction of reading comprehension, and improved diagnostic information. Data were analyzed by comparing two different scoring outcomes, the traditional ORF measure and a new prosody-inclusive outcome score, comparing these measures across language background. Results showed that the inclusion of prosody improves automated ORF assessment by reducing discrepancies between ELLs and English first language students caused by automated speech recognition inaccuracies, leads to better prediction of reading comprehension with ELLs, and provides meaningful diagnostic information. Detailed descriptions of the models, their relevance, and implications for the language testing community are discussed.
- Top Cited Papers
- 20 October 2023
- Published by Taylor & Francis in Language Assessment Quarterly
- Vol. 20 (4-5) , 399-420
Validity Arguments for Automated Essay Scoring of Young Students’ Writing Traits
- 20 October 2023
- Published by Taylor & Francis in Language Assessment Quarterly
- Vol. 20 (4-5) , 399-420
Top Cited PapersValidity Arguments for Automated Essay Scoring of Young Students’ Writing Traits
- Open Access
The complex relationship between carbon literacy and pro-environmental actions among engineering students
- 27 March 2022
- Published by Taylor & Francis in Language Assessment Quarterly
- Vol. 19 (3) , 289-313
Investigating the Effects of Task Type and Linguistic Background on Accuracy in Automated Speech Recognition Systems: Implications for Use in Language Assessment of Young Learners
As a branch of artificial intelligence, automated speech recognition (ASR) technology is increasingly used to detect speech, process it to text, and derive the meaning of natural language for various learning and assessment purposes. ASR inaccuracy may pose serious threats to valid score interpretations and fair score use for all when it is exacerbated by test takers’ characteristics, such as language background and accent, and assessment task type. The present study investigated the extent to which speech-to-text accuracy rates of three major ASR systems vary across different oral tasks and students’ language background variables. Results indicate that task types and students’ language backgrounds have statistically significant main and interaction effects on ASR accuracy. The paper discusses the implications of the study results for applying ASR to computerized assessment design and automated scoring.
- 27 March 2022
- Published by Taylor & Francis in Language Assessment Quarterly
- Vol. 19 (3) , 289-313
Investigating the Effects of Task Type and Linguistic Background on Accuracy in Automated Speech Recognition Systems: Implications for Use in Language Assessment of Young Learners
As a branch of artificial intelligence, automated speech recognition (ASR) technology is increasingly used to detect speech, process it to text, and derive the meaning of natural language for various learning and assessment purposes. ASR inaccuracy may pose serious threats to valid score interpretations and fair score use for all when it is exacerbated by test takers’ characteristics, such as language background and accent, and assessment task type. The present study investigated the extent to which speech-to-text accuracy rates of three major ASR systems vary across different oral tasks and students’ language background variables. Results indicate that task types and students’ language backgrounds have statistically significant main and interaction effects on ASR accuracy. The paper discusses the implications of the study results for applying ASR to computerized assessment design and automated scoring.