Addressing fraudulent responses in online surveys: Insights from a web‐based participatory mapping study
Top Cited Papers
Open Access
- 12 November 2023
- journal article
- research article
- Published by Wiley in People and Nature
- Vol. 6 (1) , 147-164
Abstract
Web‐based studies of human dimensions are increasing across environmental and socio‐ecological disciplines. However, the prevalence of fraud threatens research quality. Increased fraud rates should be expected as surveys move progressively more online, motivated by expanding reach, cost savings and/or in response to COVID‐19. Web‐based research must better account for fraud to maintain confidence in findings. Practical diagnostic tools and data quality protocols are required to detect fraud and ensure results quality. Drawing on our experience using an online participatory mapping case study, we discuss methods to detect potentially fraudulent responses—and identify some limitations. We begin by reviewing the current state of knowledge on fraudulent responses or ‘fraudsters’ and its relative absence in environmental and socio‐ecological disciplines. We then describe our research approach, the indicators and variables we used to detect and assess fraud and our decision‐making process to eliminate suspicious responses without jeopardizing research integrity. We found that despite several preventative measures, many fraudulent respondents could provide survey responses and effectively mimicked legitimate respondents at first glance. By assuming each response to be ‘potentially fraudulent’, we determined that the complete screening of each respondent, while time‐consuming, can limit the prevalence of fraud. We also determined that the most common data consistency checks (e.g. duration, trap questions and straight‐liner checks) are unlikely to guarantee valid respondents. If not acknowledged and addressed, fraud has the potential to undermine data integrity, discredit research findings and limit the utility of results for policy. This study contributes to environmental and socio‐ecological research by reviewing existing fraudster literature and using our experience with fraud to provide recommendations for researchers to address this problem. We encourage researchers implementing online qualitative research methods to thoroughly assess and report fraud, when possible, to ensure widespread knowledge of this growing threat. Read the free Plain Language Summary for this article on the Journal blog.Keywords
This publication has 67 references indexed in Scilit:
- Web-based Intervention to Promote Physical Activity by Sedentary Older Adults: Randomized Controlled TrialJournal of Medical Internet Research, 2013
- Data Quality in HIV/AIDS Web-Based SurveysField Methods, 2012
- Web-based surveys as an alternative to traditional mail methodsJournal of Environmental Management, 2009
- The relationship between place attachment and landscape values: Toward mapping place attachmentApplied Geography, 2007
- Challenges of Internet Recruitment: A Case Study with Disappointing ResultsJournal of Medical Internet Research, 2005
- Online methods in geography educational researchJournal of Geography in Higher Education, 2004
- The Use of the Internet in Psychological Research: Comparison of Online and Offline QuestionnairesCyberPsychology & Behavior, 2003
- Conducting research using web-based questionnaires: Practical, methodological, and ethical considerationsInternational Journal of Social Research Methodology, 2003
- Development of a Standard E-Mail Methodology: Results of an ExperimentPublic Opinion Quarterly, 1998
- Using the Internet for Survey ResearchSociological Research Online, 1997