Respondent Robotics: Simulating Responses to Likert-Scale Survey Items
Open Access
- 1 January 2018
- journal article
- research article
- Published by SAGE Publications in Sage Open
- Vol. 8 (1)
Abstract
The semantic theory of survey responses (STSR) proposes that the prime source of statistical covariance in survey data is the degree of semantic similarity (overlap of meaning) among the items of the survey. Because semantic structures are possible to estimate using digital text algorithms, it is possible to predict the response structures of Likert-type scales a priori. The present study applies STSR in an experimental way by computing real survey responses using such semantic information. A sample of 153 randomly chosen respondents to the Multifactor Leadership Questionnaire (MLQ) was used as target. We developed an algorithm based on unfolding theory, where data from digital text analysis of the survey items served as input. Upon deleting progressive numbers (from 20%-95%) of the real responses, we let the algorithm replace these with simulated ones, and then compared the simulated datasets with the real ones. The simulated scores displayed sum score levels, alphas, and factor structures highly resembling their real origins even if up to 86% were simulated. In contrast, this was not the case when the same algorithm was operating without access to semantic information. The procedure was briefly repeated on a different measurement instrument and a different sample. This not only yielded similar results but also pointed to need for further theoretical and practical developments. Our study opens for experimental research on the effect of semantics on survey responses using computational procedures.Keywords
Funding Information
- National Institutes of Health through Colorado Clinical & Translational Sciences Institute (NIH/CTSI 5 UL1 RR025780)
- U.S. National Science Foundation (NSF 0965338)
This publication has 40 references indexed in Scilit:
- Measure for Measure: A two study multi-trait multi-method investigation of construct validity in IS researchComputers in Human Behavior, 2013
- Modified Likelihood-Based Item Fit Statistics for the Generalized Graded Unfolding ModelApplied Psychological Measurement, 2008
- Work performance, affective commitment, and work motivation: the roles of pay administration and pay levelJournal of Organizational Behavior, 2006
- A Q3 Statistic for Unfolding Item Response Theory Models: Assessment of Unidimensionality With Two Factors and Simple StructureApplied Psychological Measurement, 2005
- Tackling measurement problems with Item Response TheoryJournal of Business Research, 2004
- Optimism and Pessimism: Underlying Structure and DimensionalityJournal of Social and Clinical Psychology, 2002
- Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternativesStructural Equation Modeling: A Multidisciplinary Journal, 1999
- The contribution of linguistic factors to attribute inferences and semantic similarity judgementsEuropean Journal of Social Psychology, 1989
- What is measured by a psychological measure?Scandinavian Journal of Psychology, 1988
- Attitudes vs. ActionsSocial Forces, 1934