Enhancing Transparency and Replicability in Data Collection: Lessons from the Construction of Three Education Datasets
Open Access
- 30 October 2025
- journal article
- reflection
- Published by Cambridge University Press (CUP) in Perspectives on Politics
- p. 1-18
Abstract
Assembling datasets is crucial for advancing social science research, but researchers who construct datasets often face difficult decisions with little guidance. Once public, these datasets are sometimes used without proper consideration of their creators’ choices and how these affect the validity of inferences. To support both data creators and data users, we discuss the strengths, limitations, and implications of various data collection methodologies and strategies, showing how seemingly trivial methodological differences can significantly impact conclusions. The lessons we distill build on the process of constructing three cross-national datasets on education systems. Despite their common focus, these datasets differ in the dimensions they measure, as well as their definitions of key concepts, coding thresholds and other assumptions, types of coders, and sources. From these lessons, we develop and propose general guidelines for dataset creators and users aimed at enhancing transparency, replicability, and valid inferences in the social sciences.Keywords
Funding Information
- HORIZON EUROPE European Research Council (863486)
- HORIZON EUROPE European Research Council (865305)
- Stanford King Center on Global Development
- Office of the Vice Provost for Graduate Education, Stanford University
- Alexander von Humboldt-Stiftung
This publication has 38 references indexed in Scilit:
- Varieties of Indoctrination: The Politicization of Education and the Media around the WorldPerspectives on Politics, 2024
- Education or Indoctrination? The Violent Origins of Public School Systems in an Era of State-BuildingAmerican Political Science Review, 2022
- Railroads and Reform: How Trains Strengthened the Nation StateBritish Journal of Political Science, 2021
- Inward Conquest2020
- The Non-Democratic Roots of Mass Education: Evidence from 200 YearsAmerican Political Science Review, 2020
- Varieties of Democracy2020
- IRT Models for Expert-Coded Panel DataPolitical Analysis, 2018
- Proxy documents as a source of measurement error in the Comparative Manifestos ProjectElectoral Studies, 2012
- Do Natural Resources Fuel Authoritarianism? A Reappraisal of the Resource CurseAmerican Political Science Review, 2011
- Democracy and dictatorship revisitedPublic Choice, 2009