YKYasin Karatay

Yasin Karatay

*This is an algorithmically generated author record based on publicly available information.

Yasin Karatay has a total of 17 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Evaluating the Validity of a Spoken Dialog System-Mediated Role-Play Test for Assessing Second Language Oral Communication

    by Yasin Karatay, Carol A. Chapelle

    Generative AI offers new opportunities for interactive oral communication assessment for specific purposes by building on past research investigating test takers’ oral interaction with spoken dialog systems (SDSs). Performing as automated conversational agents, SDSs can address logistical challenges in classroom-based oral assessments where interactive tasks are difficult to implement. This study investigated the validity of interpretations of a prototype SDS-mediated Tourism English Speaking Test (SDS-TEST) developed using evidence-centered design. Guided by the interpretation/use argument framework, this study focused on supporting the generalization and explanation inferences. Thirty Turkish English as a Foreign Language (EFL) students in a Tourism and Hotel Management program completed three SDS tasks, each rated by four trained raters. The generalization inference was supported using a univariate generalizability study (G-study) with a fully crossed, two-facet design on 360 composite scores, while a decision study (D-study) informed optimal task and rater configurations for future designs. The explanation inference was supported through correlational analyses with related assessments and qualitative insights from participants’ strategy use gathered via stimulated recalls. Findings support the inferences in the validity argument for the SDS-TEST, highlighting the potential of SDSs in English for Specific Purposes oral assessment and offering a model for exploring validity in the next generation of oral communication assessment.


  • Can an AI Agent Replace Human Examiners in High-Stakes Interactive Speaking Tests? A Debate

    by Jing Xu, Lynda Taylor, Xiaoming Xi, Yasin Karatay, Shangchao Min, Carsten Roever

    Generative artificial intelligence (GenAI) is advancing at a remarkable speed, and its promise in transforming current practice in language assessment has been articulated by applied linguistics researchers. An emerging application of GenAI is to integrate the technology into Spoken Dialogue Systems (SDSs) to simulate human interlocutors for the purpose of speaking practice or assessment. Despite rapid technological advances, the issue of whether an AI agent can replace a human examiner in one-on-one, high-stakes interactive speaking tests remains contentious. Building on a lively academic debate on this topic at the 2025 Language Testing Research Colloquium (LTRC) in Bangkok, this Viewpoint presents arguments both for and against this proposition in terms of construct theory, practicality, washback, fairness, ethical uses of AI, and so forth.


  • Reflections on Carol A. Chapelle’s Contributions to Applied Linguistics: Perspectives from her Students

    by Jing Xu, Hye-Won Lee, Yasin Karatay, Leyla Karatay

    This paper reflects on Dr Carol Ann Chapelle’s contributions to applied linguistics from the perspectives of some of her former students. As alumni of the Applied Linguistics and Technology (ALT) doctoral program at Iowa State University, we had the privilege of spending a significant amount of time with her, thus being able to closely observe her approach to research and teaching, her evolving research interests, and the increasing impact of her work in the field of applied linguistics. Our reflections on her accomplishments and contributions are centred around her academic achievements in areas such as computer-assisted language learning (CALL), computer-assisted language testing (CALT) and validity theory and validation practices; the qualities and traits that she exhibits which we believe enabled her to succeed; and her dedication to teaching, mentoring, and training of the next generation of applied linguists. Each author provided their reflections independently, based on their personal experiences of studying and collaborating with her. Dr Chapelle is a distinguished but low-key scholar. We intend to reveal the extensive efforts she has made behind the scenes to advance the knowledge and practice of language learning, teaching and assessment.

  • Top Cited Papers
    Open Access

    Exploring the Potential of Conversational AI for Assessing Second Language Oral Proficiency

    by Yasin Karatay, Jing Xu

    Interactional Competence (IC) is an important subcomponent of oral proficiency, but many computer‐mediated oral English assessments fall short in assessing this construct mainly due to technological limitations. Spoken Dialogue Systems (SDSs) have shown promise in assessing L2 oral communication, yet further investigation is needed on their effectiveness in eliciting IC features in high‐stakes assessment contexts. This study is unique in that it analyzed both test takers' and an SDS's spoken discourse in human–computer interactions. Using an SDS to simulate an examiner in an IELTS Speaking task, the study explored how well the system mimicked human–human interaction and elicited IC features from test takers, focusing on the IC features documented in prior research. Thirty participants completed the SDS‐mediated test, with their performances rated by two trained raters. Semi‐structured interviews with 10 test takers were conducted following the assessment. The findings revealed that the SDS successfully elicited key IC features, which helped distinguish test takers at different proficiency levels, with reliable scoring across raters. Most test takers found the SDS competent, though some noted its limitations in nonverbal communication and conversational flow. These results suggest that SDSs have potential in oral proficiency assessments and provide valuable insights for refining SDS design to ensure reliable and valid assessments.


  • Open Access

    Exploring global online course participants’ interactions: Value of high-level engagement

    by Haeyun Jin, Yasin Karatay, Fatemeh Bordbarjavidi, Junghun Yang, Timothy Kochem, Ananda Astrini Muhammad, Volker Hegelheimer

    Participation in online courses has become essential for training language professionals in under-resourced contexts with skills in computer-assisted language learning (CALL) (Godwin-Jones, 2014). Most online CALL courses use asynchronous computer-mediated communication (ACMC) to facilitate meaningful learning for participants. Although participants’ sustained engagement with ACMC is the target, global realities of participants interfere with their participation levels. This article investigates participants’ engagement profiles in asynchronous online discussions in an 8-week CALL-based global online course developed and implemented by a team at Iowa State University. Using a case study approach, nine focal participants’ engagement profiles have been analyzed in terms of identifying patterns of engagement in the discussion posts and their relation to the types of discussion prompts. Then, social network analysis (SNA) and thematic analysis were employed to investigate patterns of interaction among the participants in the replies. The results indicated that engagement patterns observed in discussion posts overall aligned with the primary goals of prompt types. SNA further identified two participants as social mediators to connect participants with each other. These findings are significant in that they suggest the effectiveness of using ACMC to promote co-construction of knowledge for a global audience. This article also provides implications regarding the design of discussion prompts to help maximize participant engagement with course content.


  • Open Access

    Review of Automated speaking assessment: Using language technologies to score spontaneous speech

    by Yasin Karatay, Leyla Karatay

    In this dynamic technology world, where innovative ideas and algorithms evolve almost daily, the landscape of computer-based language testing over the years has been dramatically reformed (Suvorov & Hegelheimer, 2013). One unique challenge that remains, however, is to effectively assess speaking proficiency (Chapelle & Voss, 2016), and since reliability plays a fundamental role in test score interpretation, the degree of error in speaking scores should be taken into account by test developers (Chapelle, 2021, p. 72). In oral speaking assessment, human rating, as traditionally the only means to evaluate constructed responses, can pose a threat to validity because of factors such as rater fatigue and rater bias (Zechner & Evanini, 2020, p. 3). A possible approach to addressing these potential threats is automated speech recognition (ASR). Recent developments such as the use of Deep Neural Network algorithms for training ASR and growth in the performance of scoring models have considerably advanced automated speech processing technology. Yet, there is still a lot to be discovered because of the lack of knowledge about how these systems work. Responding to this need, Automated Speaking Assessment: Using Language Technologies to Score Spontaneous Speech, edited by Zechner and Evanini, aims to bridge the gap between stakeholders such as scientists and engineers who may not have any expertise in reliability, validity, and fairness issues and stakeholders in academia who may not have expertise in speech processing technology. In doing so, the book unveils the state-of-the-art automated speech scoring technology, SpeechRater, used at Educational Testing Service (ETS). This book aims to be easily accessible to graduate students and applied linguists as well as testing professionals and measurement specialists.

  • Considerations for Future Technology Development Based on EFL Teachers' Integration of Technology

    by Tim Kochem, Ananda Astrini Muhammad, Yasin Karatay, Haeyun Jin, Volker Hegelheimer

    Technology-enhance language learning (TELL) continues to grow in use within language classrooms. However, a number of hurdles still remain when it comes to the effective integration of technology for skill-specific language learning, such as a lack of training and an overabundance of tools to choose from. This chapter identifies and describes three major hurdles that still plague effective TELL practices. Authors describe 2 current efforts to overcome these hurdles: a Global Online Course (GOC) on effective educational technology integration, and a year-long comparative case study on the GOC that explores the trainees' perceptions of educational technology. The chapter presents potential avenues for overcoming the above hurdles based on insights gained from four teachers of the GOC, as well as the trainees' perceptions and integration of educational technology in the language classroom.