Automatically Measuring Question Authenticity in Real-World Classrooms
- 29 June 2018
- journal article
- research article
- Published by American Educational Research Association (AERA) in Educational Researcher
- Vol. 47 (7) , 451-464
Abstract
Analyzing the quality of classroom talk is central to educational research and improvement efforts. In particular, the presence of authentic teacher questions, where answers are not predetermined by the teacher, helps constitute and serves as a marker of productive classroom discourse. Further, authentic questions can be cultivated to improve teaching effectiveness and consequently student achievement. Unfortunately, current methods to measure question authenticity do not scale because they rely on human observations or coding of teacher discourse. To address this challenge, we set out to use automatic speech recognition, natural language processing, and machine learning to train computers to detect authentic questions in real-world classrooms automatically. Our methods were iteratively refined using classroom audio and human-coded observational data from two sources: (a) a large archival database of text transcripts of 451 observations from 112 classrooms; and (b) a newly collected sample of 132 high-quality audio recordings from 27 classrooms, obtained under technical constraints that anticipate large-scale automated data collection and analysis. Correlations between human-coded and computer-coded authenticity at the classroom level were sufficiently high ( r = .602 for archival transcripts and .687 for audio recordings) to provide a valuable complement to human coding in research efforts.Keywords
Funding Information
- Institute of Education Sciences (R305A130030)
This publication has 39 references indexed in Scilit:
- Measuring Usable KnowledgeAmerican Educational Research Journal, 2012
- Integrating Literacy and Science in BiologyAmerican Educational Research Journal, 2011
- Coh-MetrixEducational Researcher, 2011
- Learning Through Extended TalkLanguage and Education, 2008
- Insights From Teachers Who Analyzed Transcripts of Their Own Classroom DiscussionsThe Reading Teacher, 2007
- Developing a collaborative model of research using digital videoComputers & Education, 2006
- Scaffolding Deep Comprehension Strategies Through Point&Query, AutoTutor, and iSTARTEducational Psychologist, 2005
- The classroom environment and students' reports of avoidance strategies in mathematics: A multimethod study.Journal of Educational Psychology, 2002
- Learning the Art of Instructional Conversation: The Influence of Self-Assessment on Teachers' Instructional Discourse in a Reading ClinicThe Elementary School Journal, 2000
- Authentic Pedagogy and Student PerformanceAmerican Journal of Education, 1996