Karolina Urbanska
Karolina Urbanska has a total of 856 co-authors. Here are the top co-authors by number of shared publications:
- Timothy M. Errington
Center for Open Science
7 shared publications - Anirudh Tagat
Department of Economics Monk Prayogshala Mumbai India
6 shared publications - Anna Lou Abatayo
Environmental Economics and Natural Resources Group, Wageningen University and Research, Wageningen, the Netherlands
6 shared publications - Brian A. Nosek
University of Virginia
6 shared publications - Hansika Kapoor
Department of Psychology Monk Prayogshala Mumbai India
6 shared publications Jianhua Duan
6 shared publications
Latest Publications
- Open Access
- 2 April 2026
- Published by Center for Open Science in MetaArXiv
Investigating the reproducibility of the social and behavioural sciences
Published claims should be reproducible, yielding the same result when applying the same analysis to the same data. We assessed reproducibility in a stratified random sample of 600 papers published from 2009 to 2018 in 62 journals spanning the social and behavioural sciences. Authors of 144 (24.0% [95% CI 20.8 - 27.6%]) papers made data available to assess reproducibility, and for 38 others, we obtained source data to reconstruct the dataset. We assessed 143 of the 182 available datasets and found that 76.6 (53.6% [95% CI 45.8 - 60.7%]) papers were rated as precisely reproducible and 105.0 (73.5% [95% CI 66.4 - 80.0%]) papers as at least approximately reproducible (within 15% of the original effects or within .05 of original p-values) after inverse weighting each of the 551 claims by the number of claims per paper. We observed higher reproducibility for papers from Political Science and Economics than other fields, for more recent than older papers, and for papers from journals that required data sharing. Implementing measures to verify that research is reproducible is needed to support trustworthiness in the complex enterprise of knowledge production.
- 2 April 2026
- Published by Center for Open Science in MetaArXiv
Open AccessInvestigating the reproducibility of the social and behavioural sciences
Published claims should be reproducible, yielding the same result when applying the same analysis to the same data. We assessed reproducibility in a stratified random sample of 600 papers published from 2009 to 2018 in 62 journals spanning the social and behavioural sciences. Authors of 144 (24.0% [95% CI 20.8 - 27.6%]) papers made data available to assess reproducibility, and for 38 others, we obtained source data to reconstruct the dataset. We assessed 143 of the 182 available datasets and found that 76.6 (53.6% [95% CI 45.8 - 60.7%]) papers were rated as precisely reproducible and 105.0 (73.5% [95% CI 66.4 - 80.0%]) papers as at least approximately reproducible (within 15% of the original effects or within .05 of original p-values) after inverse weighting each of the 551 claims by the number of claims per paper. We observed higher reproducibility for papers from Political Science and Economics than other fields, for more recent than older papers, and for papers from journals that required data sharing. Implementing measures to verify that research is reproducible is needed to support trustworthiness in the complex enterprise of knowledge production.
- Open Access
- 2 April 2026
- Published by Center for Open Science in MetaArXiv
Investigating the reproducibility of the social and behavioural sciences
Published claims should be reproducible, yielding the same result when applying the same analysis to the same data. We assessed reproducibility in a stratified random sample of 600 papers published from 2009 to 2018 in 62 journals spanning the social and behavioural sciences. Authors of 144 (24.0% [95% CI 20.8 - 27.6%]) papers made data available to assess reproducibility, and for 38 others, we obtained source data to reconstruct the dataset. We assessed 143 of the 182 available datasets and found that 76.6 (53.6% [95% CI 45.8 - 60.7%]) papers were rated as precisely reproducible and 105.0 (73.5% [95% CI 66.4 - 80.0%]) papers as at least approximately reproducible (within 15% of the original effects or within .05 of original p-values) after inverse weighting each of the 551 claims by the number of claims per paper. We observed higher reproducibility for papers from Political Science and Economics than other fields, for more recent than older papers, and for papers from journals that required data sharing. Implementing measures to verify that research is reproducible is needed to support trustworthiness in the complex enterprise of knowledge production.
- 2 April 2026
- Published by Center for Open Science in MetaArXiv
Open AccessInvestigating the reproducibility of the social and behavioural sciences
Published claims should be reproducible, yielding the same result when applying the same analysis to the same data. We assessed reproducibility in a stratified random sample of 600 papers published from 2009 to 2018 in 62 journals spanning the social and behavioural sciences. Authors of 144 (24.0% [95% CI 20.8 - 27.6%]) papers made data available to assess reproducibility, and for 38 others, we obtained source data to reconstruct the dataset. We assessed 143 of the 182 available datasets and found that 76.6 (53.6% [95% CI 45.8 - 60.7%]) papers were rated as precisely reproducible and 105.0 (73.5% [95% CI 66.4 - 80.0%]) papers as at least approximately reproducible (within 15% of the original effects or within .05 of original p-values) after inverse weighting each of the 551 claims by the number of claims per paper. We observed higher reproducibility for papers from Political Science and Economics than other fields, for more recent than older papers, and for papers from journals that required data sharing. Implementing measures to verify that research is reproducible is needed to support trustworthiness in the complex enterprise of knowledge production.
- Open Access
- 1 April 2026
- Published by Center for Open Science in MetaArXiv
Investigating the replicability of the social and behavioural sciences
We attempted replications of 274 claims of positive results from 164 quantitative papers published from 2009 to 2018 in 54 journals in the social and behavioral sciences. Replications were high-powered on average to detect the original effect size (Median = 99.6%), used original materials when relevant and available, and were peer-reviewed in advance through a standardized internal protocol. Replications showed statistically significant results in the original pattern for 151 of 274 claims (55.1% [95% CI 49.2 - 60.9%]) and for 80.8 of 164 papers (49.3% [95% CI 43.8 - 54.7%]) weighed for replicating multiple claims per paper. We observed modest variation in replication rates across disciplines (42.5% to 63.1%) though some estimates had high uncertainty. For claims with Pearson’s r effect sizes, the median was 0.25 [95% CI 0.21 - 0.27] for original studies and 0.10 [95% CI 0.09 - 0.13] for replication studies, a 82.4% [95% CI 67.8 - 88.2%] reduction in shared variance. Thirteen methods for evaluating replication success provided estimates ranging from 28.6% to 74.8% (median = 49.3%). Some decline in effect size and significance is expected based on power to detect original effects and regression to the mean due to replicating only positive results. The conditions that promote or inhibit replicability are worthy of additional investigation.
- 1 April 2026
- Published by Center for Open Science in MetaArXiv
Open AccessInvestigating the replicability of the social and behavioural sciences
We attempted replications of 274 claims of positive results from 164 quantitative papers published from 2009 to 2018 in 54 journals in the social and behavioral sciences. Replications were high-powered on average to detect the original effect size (Median = 99.6%), used original materials when relevant and available, and were peer-reviewed in advance through a standardized internal protocol. Replications showed statistically significant results in the original pattern for 151 of 274 claims (55.1% [95% CI 49.2 - 60.9%]) and for 80.8 of 164 papers (49.3% [95% CI 43.8 - 54.7%]) weighed for replicating multiple claims per paper. We observed modest variation in replication rates across disciplines (42.5% to 63.1%) though some estimates had high uncertainty. For claims with Pearson’s r effect sizes, the median was 0.25 [95% CI 0.21 - 0.27] for original studies and 0.10 [95% CI 0.09 - 0.13] for replication studies, a 82.4% [95% CI 67.8 - 88.2%] reduction in shared variance. Thirteen methods for evaluating replication success provided estimates ranging from 28.6% to 74.8% (median = 49.3%). Some decline in effect size and significance is expected based on power to detect original effects and regression to the mean due to replicating only positive results. The conditions that promote or inhibit replicability are worthy of additional investigation.
- Open Access
- 1 April 2026
- Published by Center for Open Science in MetaArXiv
Investigating the reproducibility of the social and behavioural sciences
Published claims should be reproducible, yielding the same result when applying the same analysis to the same data. We assessed reproducibility in a stratified random sample of 600 papers published from 2009 to 2018 in 62 journals spanning the social and behavioural sciences. Authors of 144 (24.0% [95% CI 20.8 - 27.6%]) papers made data available to assess reproducibility, and for 38 others, we obtained source data to reconstruct the dataset. We assessed 143 of the 182 available datasets and found that 76.6 (53.6% [95% CI 45.8 - 60.7%]) papers were rated as precisely reproducible and 105.0 (73.5% [95% CI 66.4 - 80.0%]) papers as at least approximately reproducible (within 15% of the original effects or within .05 of original p-values) after inverse weighting each of the 551 claims by the number of claims per paper. We observed higher reproducibility for papers from Political Science and Economics than other fields, for more recent than older papers, and for papers from journals that required data sharing. Implementing measures to verify that research is reproducible is needed to support trustworthiness in the complex enterprise of knowledge production.
- 1 April 2026
- Published by Center for Open Science in MetaArXiv
Open AccessInvestigating the reproducibility of the social and behavioural sciences
Published claims should be reproducible, yielding the same result when applying the same analysis to the same data. We assessed reproducibility in a stratified random sample of 600 papers published from 2009 to 2018 in 62 journals spanning the social and behavioural sciences. Authors of 144 (24.0% [95% CI 20.8 - 27.6%]) papers made data available to assess reproducibility, and for 38 others, we obtained source data to reconstruct the dataset. We assessed 143 of the 182 available datasets and found that 76.6 (53.6% [95% CI 45.8 - 60.7%]) papers were rated as precisely reproducible and 105.0 (73.5% [95% CI 66.4 - 80.0%]) papers as at least approximately reproducible (within 15% of the original effects or within .05 of original p-values) after inverse weighting each of the 551 claims by the number of claims per paper. We observed higher reproducibility for papers from Political Science and Economics than other fields, for more recent than older papers, and for papers from journals that required data sharing. Implementing measures to verify that research is reproducible is needed to support trustworthiness in the complex enterprise of knowledge production.
- Open Access
- 31 March 2026
- Published by Center for Open Science in MetaArXiv
Investigating the analytical robustness of the social and behavioural sciences
The same dataset can be analysed in different justifiable ways to answer the same research question, potentially challenging the robustness of empirical science1–3. In this crowd initiative, we investigated the degree to which research findings in the social and behavioural sciences are contingent on analysts’ choices. We examined a stratified random sample of 100 studies published between 2009 and 2018, where for one claim per study, at least five re-analysts independently re-analysed the original data. The statistical appropriateness of the re-analyses was assessed in peer evaluations, and the robustness indicators were inspected along a range of research characteristics and study designs. We found that 34% of the independent re-analyses yielded the same result (within a tolerance region of +/- 0.05 Cohen’s d) as the original report; with a four times broader tolerance region, this indicator rose to 57%. Regarding the conclusions drawn, 74% of analyses were reported to arrive at the same conclusion as in the original investigation; 24% to no effects/inconclusive result, and 2% to the opposite effect as in the original investigation. This exploratory study suggests that the common single-path analyses in social and behavioural research should not simply be assumed to be robust to alternative analyses4. Therefore, we recommend the development and use of practices to explore and communicate this neglected source of uncertainty.
- 31 March 2026
- Published by Center for Open Science in MetaArXiv
Open AccessInvestigating the analytical robustness of the social and behavioural sciences
The same dataset can be analysed in different justifiable ways to answer the same research question, potentially challenging the robustness of empirical science1–3. In this crowd initiative, we investigated the degree to which research findings in the social and behavioural sciences are contingent on analysts’ choices. We examined a stratified random sample of 100 studies published between 2009 and 2018, where for one claim per study, at least five re-analysts independently re-analysed the original data. The statistical appropriateness of the re-analyses was assessed in peer evaluations, and the robustness indicators were inspected along a range of research characteristics and study designs. We found that 34% of the independent re-analyses yielded the same result (within a tolerance region of +/- 0.05 Cohen’s d) as the original report; with a four times broader tolerance region, this indicator rose to 57%. Regarding the conclusions drawn, 74% of analyses were reported to arrive at the same conclusion as in the original investigation; 24% to no effects/inconclusive result, and 2% to the opposite effect as in the original investigation. This exploratory study suggests that the common single-path analyses in social and behavioural research should not simply be assumed to be robust to alternative analyses4. Therefore, we recommend the development and use of practices to explore and communicate this neglected source of uncertainty.
- Open Access
- 20 December 2024
- Published by Springer Nature in Nature Human Behaviour
- Vol. 9 (2) , 287-304
Predicting the replicability of social and behavioural science claims in COVID-19 preprints
Replications are important for assessing the reliability of published findings. However, they are costly, and it is infeasible to replicate everything. Accurate, fast, lower-cost alternatives such as eliciting predictions could accelerate assessment for rapid policy implementation in a crisis and help guide a more efficient allocation of scarce replication resources. We elicited judgements from participants on 100 claims from preprints about an emerging area of research (COVID-19 pandemic) using an interactive structured elicitation protocol, and we conducted 29 new high-powered replications. After interacting with their peers, participant groups with lower task expertise (‘beginners’) updated their estimates and confidence in their judgements significantly more than groups with greater task expertise (‘experienced’). For experienced individuals, the average accuracy was 0.57 (95% CI: [0.53, 0.61]) after interaction, and they correctly classified 61% of claims; beginners’ average accuracy was 0.58 (95% CI: [0.54, 0.62]), correctly classifying 69% of claims. The difference in accuracy between groups was not statistically significant and their judgements on the full set of claims were correlated (r(98) = 0.48, P < 0.001). These results suggest that both beginners and more-experienced participants using a structured process have some ability to make better-than-chance predictions about the reliability of ‘fast science’ under conditions of high uncertainty. However, given the importance of such assessments for making evidence-based critical decisions in a crisis, more research is required to understand who the right experts in forecasting replicability are and how their judgements ought to be elicited.
- 20 December 2024
- Published by Springer Nature in Nature Human Behaviour
- Vol. 9 (2) , 287-304
Open AccessPredicting the replicability of social and behavioural science claims in COVID-19 preprints
Replications are important for assessing the reliability of published findings. However, they are costly, and it is infeasible to replicate everything. Accurate, fast, lower-cost alternatives such as eliciting predictions could accelerate assessment for rapid policy implementation in a crisis and help guide a more efficient allocation of scarce replication resources. We elicited judgements from participants on 100 claims from preprints about an emerging area of research (COVID-19 pandemic) using an interactive structured elicitation protocol, and we conducted 29 new high-powered replications. After interacting with their peers, participant groups with lower task expertise (‘beginners’) updated their estimates and confidence in their judgements significantly more than groups with greater task expertise (‘experienced’). For experienced individuals, the average accuracy was 0.57 (95% CI: [0.53, 0.61]) after interaction, and they correctly classified 61% of claims; beginners’ average accuracy was 0.58 (95% CI: [0.54, 0.62]), correctly classifying 69% of claims. The difference in accuracy between groups was not statistically significant and their judgements on the full set of claims were correlated (r(98) = 0.48, P < 0.001). These results suggest that both beginners and more-experienced participants using a structured process have some ability to make better-than-chance predictions about the reliability of ‘fast science’ under conditions of high uncertainty. However, given the importance of such assessments for making evidence-based critical decisions in a crisis, more research is required to understand who the right experts in forecasting replicability are and how their judgements ought to be elicited.
- Open Access
- 15 February 2023
- Published by Center for Open Science in MetaArXiv
Predicting the replicability of social and behavioural science claims in COVID-19 preprints
Replications are important for assessing the reliability of published findings. However, they are costly, and it is infeasible to replicate everything. Accurate, fast, lower-cost alternatives such as eliciting predictions could accelerate assessment for rapid policy implementation in a crisis and help guide a more efficient allocation of scarce replication resources. We elicited judgments from participants on 100 claims from preprints about an emerging area of research (COVID-19 pandemic) using an interactive structured elicitation protocol, and we conducted 29 new high-powered replications. After interacting with their peers, participant groups with lower task expertise (‘beginners’) updated their estimates and confidence in their judgements significantly more than groups with greater task expertise (‘experienced’). For experienced individuals, the average accuracy was 0.57 (95% CI: [0.53, 0.61]) after interaction, and they correctly classified 61% of claims; beginners’ average accuracy was 0.58 (95% CI: [0.54, 0.62]), correctly classifying 69% of claims. The difference in accuracy between groups was not statistically significant, and their judgments on the full set of claims were correlated (r(98) = 0.48, p<0.001). These results suggest that both beginners and more experienced participants using a structured process have some ability to make better-than-chance predictions about the reliability of ‘fast science’ under conditions of high uncertainty. However, given the importance of such assessments for making evidence-based critical decisions in a crisis, more research is required to understand who the right experts in forecasting replicability are and how their judgements ought to be elicited.
- 15 February 2023
- Published by Center for Open Science in MetaArXiv
Open AccessPredicting the replicability of social and behavioural science claims in COVID-19 preprints
Replications are important for assessing the reliability of published findings. However, they are costly, and it is infeasible to replicate everything. Accurate, fast, lower-cost alternatives such as eliciting predictions could accelerate assessment for rapid policy implementation in a crisis and help guide a more efficient allocation of scarce replication resources. We elicited judgments from participants on 100 claims from preprints about an emerging area of research (COVID-19 pandemic) using an interactive structured elicitation protocol, and we conducted 29 new high-powered replications. After interacting with their peers, participant groups with lower task expertise (‘beginners’) updated their estimates and confidence in their judgements significantly more than groups with greater task expertise (‘experienced’). For experienced individuals, the average accuracy was 0.57 (95% CI: [0.53, 0.61]) after interaction, and they correctly classified 61% of claims; beginners’ average accuracy was 0.58 (95% CI: [0.54, 0.62]), correctly classifying 69% of claims. The difference in accuracy between groups was not statistically significant, and their judgments on the full set of claims were correlated (r(98) = 0.48, p<0.001). These results suggest that both beginners and more experienced participants using a structured process have some ability to make better-than-chance predictions about the reliability of ‘fast science’ under conditions of high uncertainty. However, given the importance of such assessments for making evidence-based critical decisions in a crisis, more research is required to understand who the right experts in forecasting replicability are and how their judgements ought to be elicited.
- 1 April 2021
- Published by Wiley in Journal of Theoretical Social Psychology
- Vol. 5 (2) , 79-89
Power to the people: Disidentification with the government and the support for populism
- 1 April 2021
- Published by Wiley in Journal of Theoretical Social Psychology
- Vol. 5 (2) , 79-89
Power to the people: Disidentification with the government and the support for populism
- Open Access
- 29 September 2020
- Published by Center for Open Science in PsyArXiv
Power to the people: Disidentification with the government and the support for populism
Populist attitudes have been shown to predict voting behaviour. These attitudes consist of a belief that everyday citizens are better judges of what is best for their own country than politicians and that the political elites are corrupt. As such, a clear ‘us’ (pure and good everyday citizens) and ‘them’ (the evil political elite) rhetoric is present. In the present research, we propose that identification with the government may predict whether people would vote for, and whether they have voted in the past for, a populist party (either from the political left or the political right). The present research (N = 562), carried out among French citizens, showed that lower government identification related to past voting behaviour, current voting intentions and likelihood to switch from a non-populist to a populist party. Identification with the government was also negatively associated to intention to abstain from voting. Moreover, government identification was a stronger predictor of these voting-related outcomes than the recently developed populist attitudes measures. Unexpectedly, national identification was a not a significant predictor of voting behaviour. In conclusion, the present research suggests that the extent to which citizens identify and feel represented by the government should be considered on par with populist attitudes in understanding support for populist parties. Perceiving that the government does not represent everyday people may be sufficient to abandon support for mainstream (non-populist) political parties.
- 29 September 2020
- Published by Center for Open Science in PsyArXiv
Open AccessPower to the people: Disidentification with the government and the support for populism
Populist attitudes have been shown to predict voting behaviour. These attitudes consist of a belief that everyday citizens are better judges of what is best for their own country than politicians and that the political elites are corrupt. As such, a clear ‘us’ (pure and good everyday citizens) and ‘them’ (the evil political elite) rhetoric is present. In the present research, we propose that identification with the government may predict whether people would vote for, and whether they have voted in the past for, a populist party (either from the political left or the political right). The present research (N = 562), carried out among French citizens, showed that lower government identification related to past voting behaviour, current voting intentions and likelihood to switch from a non-populist to a populist party. Identification with the government was also negatively associated to intention to abstain from voting. Moreover, government identification was a stronger predictor of these voting-related outcomes than the recently developed populist attitudes measures. Unexpectedly, national identification was a not a significant predictor of voting behaviour. In conclusion, the present research suggests that the extent to which citizens identify and feel represented by the government should be considered on par with populist attitudes in understanding support for populist parties. Perceiving that the government does not represent everyday people may be sufficient to abandon support for mainstream (non-populist) political parties.
- Open Access
- 14 October 2019
- Published by Leibniz Institute for Psychology (ZPID) in Journal of Social and Political Psychology
- Vol. 7 (2) , 766-789
Authority fairness for all? Intergroup status and expectations of procedural justice and resource distribution
Authorities such as the police and the government play a vital function in maintaining order in the social systems in which groups exist. Relational models of procedural justice (PJ) state that fair treatment from authority affirms the social standing of those identifying with the authority, communicating inclusion and respect. Previous research suggests that social identity may also inform expectations of authority fairness. Focusing on an intergroup context of authority decision-making, the present research tests a novel hypothesis regarding whether intergroup social status may also inform expectations of authority fairness in terms of fair treatment and favourable outcomes. Operationalising PJ as the extent to which people are provided voice by authorities, three experimental studies showed no effect of intergroup status on expected PJ from authority. A sample weighed internal meta-analysis (N = 704) also provided no support for the hypothesis that relative outgroup status shapes expectations of voice from authority (d = -.02). Intergroup status did, however, influence the extent to which people expected authorities to distribute resources favourably towards the outgroups. Lower status outgroups were expected to receive less favourable outcomes from authorities than equal status outgroups (d = -.23). Thus, outgroup status affects people’s judgements of the resources that outgroups deserve from authority. The present research is among the first to consider how intergroup relations may drive expectations of how authorities will act towards other social groups. Implications for wielding authority and the role of perceived intergroup threat in intergroup settings are discussed.
- 14 October 2019
- Published by Leibniz Institute for Psychology (ZPID) in Journal of Social and Political Psychology
- Vol. 7 (2) , 766-789
Open AccessAuthority fairness for all? Intergroup status and expectations of procedural justice and resource distribution
Authorities such as the police and the government play a vital function in maintaining order in the social systems in which groups exist. Relational models of procedural justice (PJ) state that fair treatment from authority affirms the social standing of those identifying with the authority, communicating inclusion and respect. Previous research suggests that social identity may also inform expectations of authority fairness. Focusing on an intergroup context of authority decision-making, the present research tests a novel hypothesis regarding whether intergroup social status may also inform expectations of authority fairness in terms of fair treatment and favourable outcomes. Operationalising PJ as the extent to which people are provided voice by authorities, three experimental studies showed no effect of intergroup status on expected PJ from authority. A sample weighed internal meta-analysis (N = 704) also provided no support for the hypothesis that relative outgroup status shapes expectations of voice from authority (d = -.02). Intergroup status did, however, influence the extent to which people expected authorities to distribute resources favourably towards the outgroups. Lower status outgroups were expected to receive less favourable outcomes from authorities than equal status outgroups (d = -.23). Thus, outgroup status affects people’s judgements of the resources that outgroups deserve from authority. The present research is among the first to consider how intergroup relations may drive expectations of how authorities will act towards other social groups. Implications for wielding authority and the role of perceived intergroup threat in intergroup settings are discussed.