Rating norms should be calculated from cumulative link mixed effects models
Top Cited Papers
Open Access
- 14 September 2022
- journal article
- research article
- Published by Springer Nature in Behavior Research Methods
- Vol. 55 (5) , 2175-2196
Abstract
Studies which provide norms of Likert ratings typically report per-item summary statistics. Traditionally, these summary statistics comprise the mean and the standard deviation (SD) of the ratings, and the number of observations. Such summary statistics can preserve the rank order of items, but provide distorted estimates of the relative distances between items because of the ordinal nature of Likert ratings. Inter-item relations in such ordinal scales can be more appropriately modelled by cumulative link mixed effects models (CLMMs). In a series of simulations, and with a reanalysis of an existing rating norms dataset, we show that CLMMs can be used to more accurately norm items, and can provide summary statistics analogous to the traditionally reported means and SDs, but which are disentangled from participants’ response biases. CLMMs can be applied to solve important statistical issues that exist for more traditional analyses of rating norms.Keywords
All Related Versions
This publication has 41 references indexed in Scilit:
- Semantic Size of Abstract Concepts: It Gets Emotional When You Can’t See ItPLOS ONE, 2013
- A letter visual-similarity matrix for Latin-based alphabetsBehavior Research Methods, 2012
- Checking and bootstrapping lexical norms by means of word similarity indexesBehavior Research Methods, 2012
- Leipzig Affective Norms for German: A reliability studyBehavior Research Methods, 2010
- Norms for two types of manipulability (graspability and functional usage), familiarity, and age of acquisition for 320 photographs of objectsBehavior Research Methods, 2010
- Addressing Score Bias and Differential Item Functioning Due to Individual Differences in Response StyleApplied Psychological Measurement, 2009
- Age-related differences in valence and arousal ratings of pictures from the International Affective Picture System (IAPS): Do ratings become more extreme with age?Behavior Research Methods, 2008
- Dichotomizing continuous predictors in multiple regression: a bad ideaStatistics in Medicine, 2005
- On the practice of dichotomization of quantitative variables.Psychological Methods, 2002
- Response Styles in Marketing Research: A Cross-National InvestigationJournal of Marketing Research, 2001