Mode effects in cultural domain analysis: comparing pile sort data collected via internet versus face-to-face interviews
- 25 June 2017
- journal article
- research article
- Published by Taylor & Francis in International Journal of Social Research Methodology
- Vol. 21 (2) , 165-176
Abstract
This article tests whether collecting pile-sort data online produces results similar to those obtained with face-to-face methods. We collected pile sorts from 227 university students in the cultural domain of emotions. To test for mode and design effects, we randomly assigned participants to face-to-face or internet modes and to either a 15- or 30-item sort. We found no evidence of mode effects in semantic structure, but the level of agreement among respondents varied by mode, depending on the number of items in the task. Agreement was higher among online respondents in the 15-item task but among face-to-face respondents in the 30-item one. Because the central purpose of pile sorting is to elicit semantic structure, we conclude that online data collection is a viable option for most researchers, but the mode effect in level of agreement implies that other design elements may affect comparability of face-to-face and web-based methods.Keywords
Funding Information
- National Science Foundation (BCS-0244104)
This publication has 41 references indexed in Scilit:
- Telephone Versus Face-to-Face InterviewsSociological Methodology, 2013
- Motivations for Sex Among Low-Income African American Young WomenHealth Education & Behavior, 2013
- “Up Means Good”Public Opinion Quarterly, 2013
- Pile sorting innovations: Exploring gender norms, power and equity in sub-Saharan AfricaGlobal Public Health, 2012
- Using internet technologies (such as Skype) as a research medium: a research noteQualitative Research, 2012
- Farmers' suicides in the Vidarbha region of Maharashtra, India: A qualitative exploration of their causesJournal of Injury and Violence Research, 2012
- Technology Trends in Survey Data CollectionSocial Science Computer Review, 2005
- Ethnic Classification in Southeastern Puerto Rico: The Cultural Model of "Color"Social Forces, 2005
- a multidimensional representation of role termsAmerican Ethnologist, 1975
- A multidimensional approach to the structure of personality impressions.Journal of Personality and Social Psychology, 1968