Beyond self-reports: Addressing bias and improving data quality in educational research
Author:
Open Access
- 30 June 2025
- journal article
- Published by Egitimde ve Psikolojide Olcme ve Degerlendirme Dergisi in Eğitimde ve Psikolojide Ölçme ve Değerlendirme Dergisi
- Vol. 16 (2) , 115-123
Abstract
The use of self-report in educational research has facilitated data collection by providing easy access to diverse populations in a short period of time. However, research has shown that these opportunities come with challenges. From inadequate response efforts to culturally influenced responses, there are numerous failures that can call into question the validity of our findings. For example, discrepancies between self-reports and objective data often reveal underlying biases. Poor data quality, exacerbated by social desirability in sensitive constructs, individual and environmental factors, and changes in scale structure, has highlighted that our methods may have some limitations that reduce generalisability and trigger the replicability crisis. However, these limitations can also lead to improvements in both survey design and data interpretation. Our experiences point to the need to integrate multiple data sources, to improve survey development and adaptation methods, and to use true experimental studies more frequently. By reflecting on these challenges, we suggest new directions for survey implementation in educational research studies to improve the reliability and replicability of our findings and deepen our understanding of complex human dynamics.Keywords
Funding Information
- This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.
This publication has 49 references indexed in Scilit:
- Effects of Survey Order on Subjective Measures of Cognitive Load: A Randomized Controlled TrialApplied Cognitive Psychology, 2025
- Test Anxiety and Physiological Arousal: A Systematic Review and Meta-AnalysisEducational Psychology Review, 2020
- Effect of multitasking, physical environment and electroencephalography use on cognitive load and retentionComputers in Human Behavior, 2019
- Dirty Data: The Effects of Screening Respondents Who Provide Low-Quality Data in Survey ResearchJournal of Business and Psychology, 2017
- The Measurement of Subjective Wellbeing: Item-Order Effects in the Personal Wellbeing Index—AdultJournal of Happiness Studies, 2016
- Best practice recommendations for data screeningJournal of Organizational Behavior, 2014
- Gender Differences in Ethics Research: The Importance of Controlling for the Social Desirability Response BiasJournal of Business Ethics, 2011
- Web-based measurement: Effect of completing single or multiple items per webpageComputers in Human Behavior, 2009
- Social desirability bias: A neglected aspect of validity testingPsychology & Marketing, 2000
- Response strategies for coping with the cognitive demands of attitude measures in surveysApplied Cognitive Psychology, 1991