LKLale Khorramdel

Lale Khorramdel

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article

Previous Organizations / Institutes

The organizations / institutes from the scholar's other articles in the past ten years


Lale Khorramdel has a total of 22 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Open Access

    Disentangling individual differences in cognitive response mechanisms for rating scale items: A flexible-mixture multidimensional IRTree approach

    by Ömer Emre Can Alagöz, Thorsten Meiser, Lale Khorramdel

    The accuracy of our inferences from rating-scale items can be improved with IRTree models, which consider heuristic response strategies like response styles (RS). IRTree models break down ordinal responses into pseudo-items (nodes), each representing a distinct decision-making process. These nodes are then modeled using an item response model. In the case of four-point items, a response is split into two nodes: 1) response direction, where the trait influences the overall agreement with items, and 2) response extremity, where both the trait and extreme RS (ERS) impact the choice of relative (dis)agreement categories. However, traditional models, despite addressing RS effects, assume that all respondents follow an identical response strategy, where the selection of relative (dis)agreement categories is influenced by the trait and ERS to the same degree for all respondents. Given that respondents may vary in the extent to which they adopt heuristic-driven strategies (e.g., fatigue, motivation, expertise), this assumption of homogeneous response processes is unlikely to be satisfied, potentially leading to inaccurate inferences. To accommodate different response strategies, we introduce the mixture IRTree model (MixTree). In MixTree, participants are assigned to different latent classes, each associated with distinct response processes. Based on their class memberships, varying weights are assigned to individuals’ trait and ERS scores. Additionally, MixTree simultaneously examines extraneous variables to explore sources of heterogeneity. A simulation study validates the MixTree’s performance in recovering classes and model parameters. Empirical data analysis identifies two latent classes, one linked to a trait-driven and the other to RS-driven mechanisms.


  • Open Access

    The use of process data in large-scale assessments: a literature review

    by Ella Anghel, Lale Khorramdel, Matthias von Davier

    As the use of process data in large-scale educational assessments is becoming more common, it is clear that data on examinees’ test-taking behaviors can illuminate their performance, and can have crucial ramifications concerning assessments’ validity. A thorough review of the literature in the field may inform researchers and practitioners of common findings as well as existing gaps. This literature review used topic modeling to identify themes in 221 empirical studies using process data in large-scale assessments. We identified six recurring topics: response time models, response time-general, aberrant test-taking behavior, action sequences, complex problem-solving, and digital writing. We also discuss the prominent theories used by studies in each category. Based on these findings, we suggest directions for future research applying process data from large-scale assessments.


  • Scoring Graphical Responses in TIMSS 2019 Using Artificial Neural Networks

    by Matthias von Davier, Lillian Tyack, Lale Khorramdel

    Automated scoring of free drawings or images as responses has yet to be used in large-scale assessments of student achievement. In this study, we propose artificial neural networks to classify these types of graphical responses from a TIMSS 2019 item. We are comparing classification accuracy of convolutional and feed-forward approaches. Our results show that convolutional neural networks (CNNs) outperform feed-forward neural networks in both loss and accuracy. The CNN models classified up to 97.53% of the image responses into the appropriate scoring category, which is comparable to, if not more accurate, than typical human raters. These findings were further strengthened by the observation that the most accurate CNN models correctly classified some image responses that had been incorrectly scored by the human raters. As an additional innovation, we outline a method to select human-rated responses for the training sample based on an application of the expected response function derived from item response theory. This paper argues that CNN-based automated scoring of image responses is a highly accurate procedure that could potentially replace the workload and cost of second human raters for international large-scale assessments (ILSAs), while improving the validity and comparability of scoring complex constructed-response items.


  • Open Access

    A Response-Time-Based Latent Response Mixture Model for Identifying and Modeling Careless and Insufficient Effort Responding in Survey Data

    by Esther Ulitzsch, Steffi Pohl, Lale Khorramdel, Ulf Kroehne, Matthias von Davier

    Careless and insufficient effort responding (C/IER) can pose a major threat to data quality and, as such, to validity of inferences drawn from questionnaire data. A rich body of methods aiming at its detection has been developed. Most of these methods can detect only specific types of C/IER patterns. However, typically different types of C/IER patterns occur within one data set and need to be accounted for. We present a model-based approach for detecting manifold manifestations of C/IER at once. This is achieved by leveraging response time (RT) information available from computer-administered questionnaires and integrating theoretical considerations on C/IER with recent psychometric modeling approaches. The approach a) takes the specifics of attentive response behavior on questionnaires into account by incorporating the distance–difficulty hypothesis, b) allows for attentiveness to vary on the screen-by-respondent level, c) allows for respondents with different trait and speed levels to differ in their attentiveness, and d) at once deals with various response patterns arising from C/IER. The approach makes use of item-level RTs. An adapted version for aggregated RTs is presented that supports screening for C/IER behavior on the respondent level. Parameter recovery is investigated in a simulation study. The approach is illustrated in an empirical example, comparing different RT measures and contrasting the proposed model-based procedure against indicator-based multiple-hurdle approaches.


  • Open Access

    The Benefits of Fixed Item Parameter Calibration for Parameter Accuracy in Small Sample Situations in Large‐Scale Assessments

    by Christoph König, Lale Khorramdel, Kentaro Yamamoto, Andreas Frey

    Large‐scale assessments such as the Programme for International Student Assessment (PISA) have field trials where new survey features are tested for utility in the main survey. Because of resource constraints, there is a trade‐off between how much of the sample can be used to test new survey features and how much can be used for the initial item response theory (IRT) scaling. Utilizing real assessment data of the PISA 2015 Science assessment, this article demonstrates that using fixed item parameter calibration (FIPC) in the field trial yields stable item parameter estimates in the initial IRT scaling for samples as small as n = 250 per country. Moreover, the results indicate that for the recovery of the county‐specific latent trait distributions, the estimates of the trend items (i.e., the information introduced into the calibration) are crucial. Thus, concerning the country‐level sample size of n = 1,950 currently used in the PISA field trial, FIPC is useful for increasing the number of survey features that can be examined during the field trial without the need to increase the total sample size. This enables international large‐scale assessments such as PISA to keep up with state‐of‐the‐art developments regarding assessment frameworks, psychometric models, and delivery platform capabilities.


  • Advances in modelling response styles and related phenomena

    by Lale Khorramdel, Minjeong Jeon, Lihshing Leigh Wang


  • Combining mixture distribution and multidimensional IRTree models for the measurement of extreme response styles

    by Lale Khorramdel, Matthias von Davier, Artur Pokropek

    Personality constructs, attitudes and other non‐cognitive variables are often measured using rating or Likert‐type scales, which does not come without problems. Especially in low‐stakes assessments, respondents may produce biased responses due to response styles (RS) that reduce the validity and comparability of the measurement. Detecting and correcting RS is not always straightforward because not all respondents show RS and the ones who do may not do so to the same extent or in the same direction. The present study proposes the combination of a multidimensional IRTree model with a mixture distribution item response theory model and illustrates the application of the approach using data from the Programme for the International Assessment of Adult Competencies (PIAAC). This joint approach allows for the differentiation between different latent classes of respondents who show different RS behaviours and respondents who show RS versus respondents who give (largely) unbiased responses. We illustrate the application of the approach by examining extreme RS and show how the resulting latent classes can be further examined using external variables and process data from computer‐based assessments to develop a better understanding of response behaviour and RS.


  • Evaluating item response theory linking and model fit for data from PISA 2000–2012

    by Matthias von Davier, Kentaro Yamamoto, Hyo Jeong Shin, Henry Chen, Lale Khorramdel, Jon Weeks, Scott Davis, Nan Kong, Mat Kandathil

    Based on concerns about the item response theory (IRT) linking approach used in the Programme for International Student Assessment (PISA) until 2012 as well as the desire to include new, more complex, interactive items with the introduction of computer-based assessments, alternative IRT linking methods were implemented in the 2015 PISA round. The new linking method represents a concurrent calibration using all available data, enabling us to find item parameters that maximize fit across all groups and allowing us to investigate measurement invariance across groups. Apart from the Rasch model that historically has been used in PISA operational analyses, we compared our method against more general IRT models that can incorporate item-by-country interactions. The results suggest that our proposed method holds promise not only to provide a strong linkage across countries and cycles but also to serve as a tool for investigating measurement invariance.