A Systematic Evaluation of Wording Effects Modeling Under the Exploratory Structural Equation Modeling Framework
Authors:
- Luis Eduardo GarridoLGLuis Eduardo Garrido
- Alexander P. ChristensenACAlexander P. Christensen
- Hudson GolinoHGHudson Golino
- Agustín Martínez-MolinaAMAgustín Martínez-Molina
- Víctor B. AriasVAVíctor B. Arias
- Kiero Guerra-PeñaKGKiero Guerra-Peña
- María Dolores Nieto-CañaverasMNMaría Dolores Nieto-Cañaveras
- Flávio AzevedoFAFlávio Azevedo
- Francisco J. AbadFAFrancisco J. Abad
- 8 September 2025
- journal article
- Published by Taylor & Francis in Multivariate Behavioral Research
- Vol. 60 (6) , 1169-1198
Abstract
Wording effects, the systematic method variance arising from the inconsistent responding to positively and negatively worded items of the same construct, are pervasive in the behavioral and health sciences. Although several factor modeling strategies have been proposed to mitigate their adverse effects, there is limited systematic research assessing their performance with exploratory structural equation models (ESEM). The present study evaluated the impact of different types of response bias related to wording effects (random and straight-line carelessness, acquiescence, item difficulty, and mixed) on ESEM models incorporating two popular method modeling strategies, the correlated traits-correlated methods minus one (CTC[M-1]) model and random intercept item factor analysis (RIIFA), as well as the "do nothing" approach. Five variables were manipulated using Monte Carlo methods: the type and magnitude of response bias, factor loadings, factor correlations, and sample size. Overall, the results showed that ignoring wording effects leads to poor model fit and serious distortions of the ESEM estimates. The RIIFA approach generally performed best at countering these adverse impacts and recovering unbiased factor structures, whereas the CTC(M-1) models struggled when biases affected both positively and negatively worded items. Our findings also indicated that method factors can sometimes reflect or absorb substantive variance, which may blur their associations with external variables and complicate their interpretation when embedded in broader structural models. A straightforward guide is offered to applied researchers who wish to use ESEM with mixed-worded scales.Keywords
All Related Versions
Funding Information
- Federal Judges Association (MICIU/AEI/10.13039/501100011033)
This publication has 82 references indexed in Scilit:
- Cross-Cultural Validation of a Spanish-Language Version of the Composite Abuse Scale (Revised) – Short Form (CASR-SF)Journal of Family Violence, 2023
- A systematic literature review of exploratory factor analyses in managementJournal of Business Research, 2023
- Exploratory factor analysis: Current use, methodological developments and recommendations for good practiceCurrent Psychology, 2019
- Assessing the Size of Model Misfit in Structural Equation ModelsPsychometrika, 2017
- Acquiescence in personality questionnaires: Relevance, domain specificity, and stabilityJournal of Research in Personality, 2015
- Self‐Report Questionnaires2015
- Random intercept EFA of personality scalesJournal of Research in Personality, 2014
- The proximity effect: The role of inter-item distance on reverse-item biasInternational Journal of Research in Marketing, 2009
- Careless Responding to Reverse-Worded Items: Implications for Confirmatory Factor AnalysisJournal of Psychopathology and Behavioral Assessment, 2006
- A Multitrait-Multimethod Model with Minimal AssumptionsPsychometrika, 2000