KUKarolina Urbanska

Karolina Urbanska

*This is an algorithmically generated author record based on publicly available information.

Karolina Urbanska has a total of 856 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Open Access

    Investigating the reproducibility of the social and behavioural sciences

    by Olivia Miske, Anna Lou Abatayo, Macie Daley, Mirka Dirzo, Nicholas William Fox, Noah Haber, Krystal Hahn, Melissa Kline Struhl, Brinna Mawhinney, Priya Silverstein, et al.

    Published claims should be reproducible, yielding the same result when applying the same analysis to the same data. We assessed reproducibility in a stratified random sample of 600 papers published from 2009 to 2018 in 62 journals spanning the social and behavioural sciences. Authors of 144 (24.0% [95% CI 20.8 - 27.6%]) papers made data available to assess reproducibility, and for 38 others, we obtained source data to reconstruct the dataset. We assessed 143 of the 182 available datasets and found that 76.6 (53.6% [95% CI 45.8 - 60.7%]) papers were rated as precisely reproducible and 105.0 (73.5% [95% CI 66.4 - 80.0%]) papers as at least approximately reproducible (within 15% of the original effects or within .05 of original p-values) after inverse weighting each of the 551 claims by the number of claims per paper. We observed higher reproducibility for papers from Political Science and Economics than other fields, for more recent than older papers, and for papers from journals that required data sharing. Implementing measures to verify that research is reproducible is needed to support trustworthiness in the complex enterprise of knowledge production.

  • Open Access

    Investigating the reproducibility of the social and behavioural sciences

    by Olivia Miske, Anna Lou Abatayo, Macie Daley, Mirka Dirzo, Nicholas William Fox, Noah Haber, Krystal Hahn, Melissa Kline Struhl, Brinna Mawhinney, Priya Silverstein, et al.

    Published claims should be reproducible, yielding the same result when applying the same analysis to the same data. We assessed reproducibility in a stratified random sample of 600 papers published from 2009 to 2018 in 62 journals spanning the social and behavioural sciences. Authors of 144 (24.0% [95% CI 20.8 - 27.6%]) papers made data available to assess reproducibility, and for 38 others, we obtained source data to reconstruct the dataset. We assessed 143 of the 182 available datasets and found that 76.6 (53.6% [95% CI 45.8 - 60.7%]) papers were rated as precisely reproducible and 105.0 (73.5% [95% CI 66.4 - 80.0%]) papers as at least approximately reproducible (within 15% of the original effects or within .05 of original p-values) after inverse weighting each of the 551 claims by the number of claims per paper. We observed higher reproducibility for papers from Political Science and Economics than other fields, for more recent than older papers, and for papers from journals that required data sharing. Implementing measures to verify that research is reproducible is needed to support trustworthiness in the complex enterprise of knowledge production.

  • Open Access

    Investigating the replicability of the social and behavioural sciences

    by Andrew Harold Tyner, Anna Lou Abatayo, Macie Daley, Samuel Field, Nicholas William Fox, Noah Haber, Krystal Hahn, Melissa Kline Struhl, Brinna Mawhinney, Olivia Miske, et al.

    We attempted replications of 274 claims of positive results from 164 quantitative papers published from 2009 to 2018 in 54 journals in the social and behavioral sciences. Replications were high-powered on average to detect the original effect size (Median = 99.6%), used original materials when relevant and available, and were peer-reviewed in advance through a standardized internal protocol. Replications showed statistically significant results in the original pattern for 151 of 274 claims (55.1% [95% CI 49.2 - 60.9%]) and for 80.8 of 164 papers (49.3% [95% CI 43.8 - 54.7%]) weighed for replicating multiple claims per paper. We observed modest variation in replication rates across disciplines (42.5% to 63.1%) though some estimates had high uncertainty. For claims with Pearson’s r effect sizes, the median was 0.25 [95% CI 0.21 - 0.27] for original studies and 0.10 [95% CI 0.09 - 0.13] for replication studies, a 82.4% [95% CI 67.8 - 88.2%] reduction in shared variance. Thirteen methods for evaluating replication success provided estimates ranging from 28.6% to 74.8% (median = 49.3%). Some decline in effect size and significance is expected based on power to detect original effects and regression to the mean due to replicating only positive results. The conditions that promote or inhibit replicability are worthy of additional investigation.

  • Open Access

    Investigating the reproducibility of the social and behavioural sciences

    by Olivia Miske, Anna Lou Abatayo, Macie Daley, Mirka Dirzo, Nicholas William Fox, Noah Haber, Krystal Hahn, Melissa Kline Struhl, Brinna Mawhinney, Priya Silverstein, et al.

    Published claims should be reproducible, yielding the same result when applying the same analysis to the same data. We assessed reproducibility in a stratified random sample of 600 papers published from 2009 to 2018 in 62 journals spanning the social and behavioural sciences. Authors of 144 (24.0% [95% CI 20.8 - 27.6%]) papers made data available to assess reproducibility, and for 38 others, we obtained source data to reconstruct the dataset. We assessed 143 of the 182 available datasets and found that 76.6 (53.6% [95% CI 45.8 - 60.7%]) papers were rated as precisely reproducible and 105.0 (73.5% [95% CI 66.4 - 80.0%]) papers as at least approximately reproducible (within 15% of the original effects or within .05 of original p-values) after inverse weighting each of the 551 claims by the number of claims per paper. We observed higher reproducibility for papers from Political Science and Economics than other fields, for more recent than older papers, and for papers from journals that required data sharing. Implementing measures to verify that research is reproducible is needed to support trustworthiness in the complex enterprise of knowledge production.

  • Open Access

    Investigating the analytical robustness of the social and behavioural sciences

    by Balazs Aczel, Barnabas Szaszi, Harry T Clelland, Marton Kovacs, Felix Holzmeister, Don van Ravenzwaaij, Hannah Schulz-Kümpel, Sabine Hoffmann, Gustav Nilsonne, Livia Kósa, et al.

    The same dataset can be analysed in different justifiable ways to answer the same research question, potentially challenging the robustness of empirical science1–3. In this crowd initiative, we investigated the degree to which research findings in the social and behavioural sciences are contingent on analysts’ choices. We examined a stratified random sample of 100 studies published between 2009 and 2018, where for one claim per study, at least five re-analysts independently re-analysed the original data. The statistical appropriateness of the re-analyses was assessed in peer evaluations, and the robustness indicators were inspected along a range of research characteristics and study designs. We found that 34% of the independent re-analyses yielded the same result (within a tolerance region of +/- 0.05 Cohen’s d) as the original report; with a four times broader tolerance region, this indicator rose to 57%. Regarding the conclusions drawn, 74% of analyses were reported to arrive at the same conclusion as in the original investigation; 24% to no effects/inconclusive result, and 2% to the opposite effect as in the original investigation. This exploratory study suggests that the common single-path analyses in social and behavioural research should not simply be assumed to be robust to alternative analyses4. Therefore, we recommend the development and use of practices to explore and communicate this neglected source of uncertainty.

  • Open Access

    Predicting the replicability of social and behavioural science claims in COVID-19 preprints

    by Alexandru Marcoci, David P. Wilkinson, Ans Vercammen, Bonnie C. Wintle, Anna Lou Abatayo, Ernest Baskin, Henk Berkman, Erin M. Buchanan, Sara Capitán, Tabaré Capitán, et al.

    Replications are important for assessing the reliability of published findings. However, they are costly, and it is infeasible to replicate everything. Accurate, fast, lower-cost alternatives such as eliciting predictions could accelerate assessment for rapid policy implementation in a crisis and help guide a more efficient allocation of scarce replication resources. We elicited judgements from participants on 100 claims from preprints about an emerging area of research (COVID-19 pandemic) using an interactive structured elicitation protocol, and we conducted 29 new high-powered replications. After interacting with their peers, participant groups with lower task expertise (‘beginners’) updated their estimates and confidence in their judgements significantly more than groups with greater task expertise (‘experienced’). For experienced individuals, the average accuracy was 0.57 (95% CI: [0.53, 0.61]) after interaction, and they correctly classified 61% of claims; beginners’ average accuracy was 0.58 (95% CI: [0.54, 0.62]), correctly classifying 69% of claims. The difference in accuracy between groups was not statistically significant and their judgements on the full set of claims were correlated (r(98) = 0.48, P < 0.001). These results suggest that both beginners and more-experienced participants using a structured process have some ability to make better-than-chance predictions about the reliability of ‘fast science’ under conditions of high uncertainty. However, given the importance of such assessments for making evidence-based critical decisions in a crisis, more research is required to understand who the right experts in forecasting replicability are and how their judgements ought to be elicited.


  • Open Access

    Predicting the replicability of social and behavioural science claims in COVID-19 preprints

    by Alexandru Marcoci, David Peter Wilkinson, Anna Abatayo, Ernest Baskin, Henk Berkman, Erin Michelle Buchanan, Sara Capitán, Tabaré Capitán, Ginny Chan, Kent Jason Go Cheng, et al.

    Replications are important for assessing the reliability of published findings. However, they are costly, and it is infeasible to replicate everything. Accurate, fast, lower-cost alternatives such as eliciting predictions could accelerate assessment for rapid policy implementation in a crisis and help guide a more efficient allocation of scarce replication resources. We elicited judgments from participants on 100 claims from preprints about an emerging area of research (COVID-19 pandemic) using an interactive structured elicitation protocol, and we conducted 29 new high-powered replications. After interacting with their peers, participant groups with lower task expertise (‘beginners’) updated their estimates and confidence in their judgements significantly more than groups with greater task expertise (‘experienced’). For experienced individuals, the average accuracy was 0.57 (95% CI: [0.53, 0.61]) after interaction, and they correctly classified 61% of claims; beginners’ average accuracy was 0.58 (95% CI: [0.54, 0.62]), correctly classifying 69% of claims. The difference in accuracy between groups was not statistically significant, and their judgments on the full set of claims were correlated (r(98) = 0.48, p<0.001). These results suggest that both beginners and more experienced participants using a structured process have some ability to make better-than-chance predictions about the reliability of ‘fast science’ under conditions of high uncertainty. However, given the importance of such assessments for making evidence-based critical decisions in a crisis, more research is required to understand who the right experts in forecasting replicability are and how their judgements ought to be elicited.


  • Open Access

    Power to the people: Disidentification with the government and the support for populism

    by Karolina Urbanska, Sam Pehrson, Serge Guimond

    Populist attitudes have been shown to predict voting behaviour. These attitudes consist of a belief that everyday citizens are better judges of what is best for their own country than politicians and that the political elites are corrupt. As such, a clear ‘us’ (pure and good everyday citizens) and ‘them’ (the evil political elite) rhetoric is present. In the present research, we propose that identification with the government may predict whether people would vote for, and whether they have voted in the past for, a populist party (either from the political left or the political right). The present research (N = 562), carried out among French citizens, showed that lower government identification related to past voting behaviour, current voting intentions and likelihood to switch from a non-populist to a populist party. Identification with the government was also negatively associated to intention to abstain from voting. Moreover, government identification was a stronger predictor of these voting-related outcomes than the recently developed populist attitudes measures. Unexpectedly, national identification was a not a significant predictor of voting behaviour. In conclusion, the present research suggests that the extent to which citizens identify and feel represented by the government should be considered on par with populist attitudes in understanding support for populist parties. Perceiving that the government does not represent everyday people may be sufficient to abandon support for mainstream (non-populist) political parties.


  • Open Access

    Authority fairness for all? Intergroup status and expectations of procedural justice and resource distribution

    by Karolina Urbanska, Samuel Pehrson, Rhiannon N. Turner

    Authorities such as the police and the government play a vital function in maintaining order in the social systems in which groups exist. Relational models of procedural justice (PJ) state that fair treatment from authority affirms the social standing of those identifying with the authority, communicating inclusion and respect. Previous research suggests that social identity may also inform expectations of authority fairness. Focusing on an intergroup context of authority decision-making, the present research tests a novel hypothesis regarding whether intergroup social status may also inform expectations of authority fairness in terms of fair treatment and favourable outcomes. Operationalising PJ as the extent to which people are provided voice by authorities, three experimental studies showed no effect of intergroup status on expected PJ from authority. A sample weighed internal meta-analysis (N = 704) also provided no support for the hypothesis that relative outgroup status shapes expectations of voice from authority (d = -.02). Intergroup status did, however, influence the extent to which people expected authorities to distribute resources favourably towards the outgroups. Lower status outgroups were expected to receive less favourable outcomes from authorities than equal status outgroups (d = -.23). Thus, outgroup status affects people’s judgements of the resources that outgroups deserve from authority. The present research is among the first to consider how intergroup relations may drive expectations of how authorities will act towards other social groups. Implications for wielding authority and the role of perceived intergroup threat in intergroup settings are discussed.