Cross-Lingual and Cross-Cultural Validity of the TIMSS 2019 Computer Use Questionnaire: Using Measurement Invariance and DIF Methods
- 31 December 2025
- journal article
- Published by Egitimde ve Psikolojide Olcme ve Degerlendirme Dergisi in Eğitimde ve Psikolojide Ölçme ve Değerlendirme Dergisi
- Vol. 16 (4) , 202-215
Abstract
This study investigates whether the TIMSS 2019 Computer Use Questionnaire functions equivalently across languages and cultures. Using responses from 8th-grade students in Turkey, England, and Qatar, we evaluated cross-group comparability with Multiple-Group Confirmatory Factor Analysis (MGCFA) and examined Differential Item Functioning (DIF) via Ordinal Logistic Regression (OLR) and Poly‑SIBTEST. The instrument comprises 11 Likert-type items organized into two factors—Computer Usage Frequency and Computer Usage Self‑Efficacy—supported by exploratory and confirmatory factor analyses. For the same‑culture/different‑language comparison (Qatar Arabic vs. English), configural and metric invariance were supported, whereas scalar invariance was not. For the different‑culture/different‑language comparison (England vs. Turkey), only configural invariance was obtained, indicating that factor loadings and intercepts were not fully comparable across these countries. DIF findings varied by method: OLR flagged mostly negligible DIF in the frequency items for the same‑culture comparison, while Poly‑SIBTEST identified several items with moderate to large DIF; in the cross‑culture comparison, both methods indicated DIF for most items, particularly within the self‑efficacy factor. The pattern of results suggests that linguistic adaptation, access to technology, and differences in technology‑related experiences contribute to nonequivalence. We propose revising culture‑sensitive terms, clarifying item contexts, and incorporating qualitative evidence to strengthen score comparability in future administrations.Keywords
This publication has 30 references indexed in Scilit:
- Exploring undue advantage of differential item functioning in high-stakes assessments: Implications on sustainable development goal 4Social Sciences & Humanities Open, 2024
- Relating violations of measurement invariance to group differences in response times.Psychological Methods, 2024
- Improving the assessment of measurement invariance: Using regularization to select anchor items and identify differential item functioning.Psychological Methods, 2020
- Using computer-assisted instruction to enhance achievement of English language learnersEducation and Information Technologies, 2012
- Methodologies for Investigating Item- and Test-Level Measurement Equivalence in International Large-Scale AssessmentsInternational Journal of Testing, 2012
- Diagnosing Measurement Equivalence in Cross-National ResearchJournal of International Business Studies, 1995
- Factor analysis in the development and refinement of clinical assessment instruments.Psychological Assessment, 1995
- Measurement Invariance, Factor Analysis and Factorial InvariancePsychometrika, 1993
- A class of invariant consistent tests for multivariate normalityCommunications in Statistics - Theory and Methods, 1990
- An Index of Factorial SimplicityPsychometrika, 1974