Tom Stafford

Background and Aims: Alcohol choice is governed by its subjective value, which can be experimentally modified. However, less is known about the moment-to-moment dynamics through which experimental manipulations of value influence observable choice behaviour. We tested whether an experimental manipulation intended to elevate alcohol value would alter computational parameters of value-based decision-making (VBDM) and alcohol choice, and whether these outcomes would mediate ad libitum alcohol consumption. Design: Pre-registered, randomised between-subjects experiment comparing two personalised experimental conditions: alcohol value (exposure to alcohol memories and multisensory alcohol cues) vs. control (exposure to non-alcohol memories and multisensory soft-drink cues), conducted over two separate sessions with baseline measures included as covariates. Setting: University laboratory in the United Kingdom (UK). Participants: 129 adults (63% male; mean age = 44 years; mean Alcohol Use Disorders Identification Test [AUDIT] score = 16) who regularly consume alcohol (mean Estimated Weekly Alcohol Consumption [EWAC] = 42 UK units/week), randomised to alcohol value (n = 65) or control (n = 64) conditions. Measurements: At baseline and post-exposure, participants completed self-reported measures of alcohol craving and a value-based decision-making (VBDM) task where they chose between pairs of previously rated alcohol-related and non-alcohol images. A drift-diffusion model was fitted to behavioural data to estimate evidence accumulation rates (value sensitivity) and response thresholds (response caution). Participants also completed a concurrent choice task (alcohol vs. non-alcohol) that measured percentage alcohol relative to non-alcohol reward (enlargement; index of relative alcohol value), and a taste test to measure alcohol consumption. Findings: Some indices of subjective craving were responsive to the experimental manipulation, but there were no significant between-group differences at post-manipulation (ps ≥ .17). Baseline-adjusted analyses showed that response thresholds for non-alcohol decisions were higher in the alcohol value condition compared to control (p = .03, d = 0.40); however, Bayesian analysis quantified only anecdotal evidence for this effect (BF10 = 1.84). Other VBDM parameters, percentage alcohol choice, and alcohol consumed were unaffected by the manipulation (ps ≥ .19, BF10s ≤ 0.40). Furthermore, no value indices (computational or percentage choice) mediated the effect of the experimental manipulation on alcohol consumed. Conclusions: A personalised experimental manipulation intended to elevate alcohol value had modest effects on self-reported craving, but did not alter computational parameters of VBDM, relative alcohol choice, or ad-libitum alcohol consumption.

BackgroundPeer review-based funding evaluation systems are under increasing pressure. To help alleviate this, innovative approaches like Distributed Peer Review (DPR) – where applicants review proposals for the same funding call – are being explored. The Volkswagen Foundation trialled DPR alongside a conventional panel review process for an interdisciplinary call. Evaluation of this trial provides insights into the dynamics of (distributed) peer review. MethodsAll proposals (N=140) underwent both DPR and the panel review process. In DPR, applicants (N= 323) reviewed four or five proposals, providing an overall score, scores for each funding criteria, and a comment. Proposals were ranked using a trimmed mean score. Expectation and feedback surveys were administered. The panel process involved shortlisting by VWS staff, review by panellists (N=8), and a final panel discussion. Semi-structured interviews with applicants, panellists, and VWS staff (N=20) explored perceptions and experiences of DPR, using thematic analysis. Quantitative analysis of review data and surveys utilised bootstrapping to assess decision stability and mixed effects models for consistency between reviewers. Qualitative and quantitative results were integrated in a convergent parallel design.ResultsDPR recommended ten proposals for funding, while the panel recommended eleven; three overlapped. Funding criteria strongly predicted overall score; ‘Originality’ was given most weight by both sets of reviewers. Stability of DPR funding decisions was modest; across 500 bootstrapped samples a mean of 5.49 out of ten previously funded proposals would still be funded. This may be explained by high inter-reviewer variability: more variance was attributable to between-reviewer differences (20.0%) than between-proposal differences (8.9%). Advantages of DPR included increased efficiency, faster decision times, detailed feedback to applicants, and democratisation of funding evaluation. Noted disadvantages were burden on applicants, absence of discussion, and perceived lack of reviewer expertise. Funders implementing DPR must consider the high initial effort, potential for ‘gaming’, and reduced direct influence. ConclusionsThis trial demonstrates the feasibility of DPR for an interdisciplinary funding programme. Implementing DPR requires funders to accept trade-offs and that DPR may not produce the same outcomes as existing systems. Building on established peer review principles, it may offer benefits, including shortening time to funding decision, and has the potential to contribute to the democratisation of funding evaluation.

This is the second study to attempt replication of Banas et al (2023) on inoculation messages. The original experiment demonstrated that inoculation messages could foster resistance to anti-vaccine conspiracy theories without significantly affecting motivational threat. However, using the same materials, Bessarabova and Banas (2024), we counted as the first replication, reported significant effects on both variables. To re-investigate the effect of these inoculation scripts, we conducted a conceptual replication experiment in the UK. In our replication, we measured and controlled conspiracy mentality as the potential variable that may influence MMR vaccine conspiracy beliefs. Our analyses showed significant effect of inoculation treatments in reducing the levels of MMR vaccine conspiracy attitudes without stimulating motivational threat. Yet, they did not generate significant effects on other conspiracy beliefs. Conspiracy mentality, included as covariates, also showed significant results in predicting MMR vaccine conspiracy attitudes and other content-specific conspiracy beliefs.

BackgroundDistributed peer review (DPR) of grant proposals, where applicants review the proposals of others applying to the same funding call, offers advantages, including increased feedback to applicants. With the reviewer pool being determined by the applicants to a particular call, there are, however, concerns that reviewers may not possess the necessary expertise and experience to provide quality reviews and feedback. We analysed review comments from recent trials of DPR for funding evaluation at the Volkswagen Foundation to obtain insights into reviewing and the impact of reviewer characteristics on review content. MethodReview comments across two years (N=2950) were classified (content and sentiment) using existing, fine-tuned machine learning classifier models. Mean prevalence of content categories and sentiment was calculated, both overall and stratified by reviewer/proposal characteristics. ResultsReview comments were consistent with review scores and overall ranking of proposals. Minor differences in content and sentiment according to reviewer gender and research experience (years working in research) were detected. There was minimal variation according to funding peer review experience and disciplinary alignment between reviewer and proposal. ConclusionExisting classifier models can be used to analyse grant review comments and to obtain insights into reviewer consistency and adherence to funding criteria. Concerns about the influence of reviewer experience and expertise on reviews may be unfounded. Results from analyses of review comments can be used to audit and refine review processes.

BackgroundPeer review-based funding evaluation systems are under increasing pressure. To help alleviate this, innovative approaches like Distributed Peer Review (DPR) – where applicants review proposals for the same funding call – are being explored. The Volkswagen Foundation trialled DPR alongside conventional panel review for an interdisciplinary call. Evaluation of this trial provides insights into the dynamics of (distributed) peer review. MethodsAll proposals (N=140) underwent both DPR and panel review. In DPR, applicants (N= 323) reviewed four or five proposals, providing an overall score, scores for each funding criteria, and a comment. Proposals were ranked using a trimmed mean score. Expectation and feedback surveys were administered. Panel review involved shortlisting by VWS staff, panellist review (N=8), and a final panel discussion. Semi-structured interviews with applicants, panellists, and VWS staff (N=20) explored perceptions and experiences of DPR, using thematic analysis. Quantitative analysis of review data and surveys utilised bootstrapping to assess decision stability and mixed effects models for consistency between reviewers. Qualitative and quantitative results were integrated in a convergent parallel design. ResultsDPR recommended ten proposals for funding, while the panel recommended eleven; three overlapped. Funding criteria strongly predicted overall score; ‘Originality’ was given most weight by both sets of reviewers. Stability of DPR funding decisions was modest; across 500 bootstrapped samples a mean of 5.49 out of ten previously funded proposals would still be funded. This may be explained by high inter-reviewer variability: more variance was attributable to between-reviewer differences (20.04%) than between-proposal differences (8.87%). Advantages of DPR included increased efficiency, faster decision times, detailed feedback to applicants, and democratisation of funding evaluation. Noted disadvantages were burden on applicants, absence of discussion, and perceived lack of reviewer expertise. Funders implementing DPR must consider the high initial effort, potential for ‘gaming’, and reduced direct influence. ConclusionsThis trial demonstrates feasibility of DPR for an interdisciplinary funding programme. Implementing a DPR process requires funders to accept trade-offs, but can offer valuable benefits, including shortening time to funding decision. Building on established peer review principles, it has the potential to contribute to the democratisation of funding evaluation.

Large Language Models (LLMs) such as ChatGPT are transforming how scientists conduct and validate research, offering promise as tools to improve scientific reproducibility. However, computational reproducibility and error detection remain expensive and labor-intensive. We experimentally test how collaboration between researchers and LLM assistants influences the reproduction of quantitative social science findings across different levels of AI autonomy. We randomly assigned 288 researchers to 103 teams working under three conditions: human-only, AI-assisted (using ChatGPT as a collaborative tool), or AI-led (ChatGPT operating with minimal human oversight). Teams reproduced published results from leading social science journals, detected coding errors, and proposed robustness checks. Human-only and AI-assisted teams achieved comparable reproduction rates (94% vs. 91%) and performed similarly on most outcomes, except human-only teams identified significantly more major coding errors. Both substantially outperformed AI-led teams, which achieved only a 37% reproduction rate, detected fewer errors across all categories, proposed weaker robustness checks, and required more time. This autonomous approach, however, likely represents only a lower bound of AI capabilities. Despite rapid model advances, expert human judgment currently remains indispensable for reliable empirical verification. While AI assistance did not degrade most outcomes, it provided no measurable advantages and was associated with reduced detection of major errors. However, the 37% autonomous reproduction rate indicates that AI could provide value in settings where scale or cost constraints preclude human review of papers, even though general-purpose LLMs offer no immediate advantages for human-supervised verification.

An improved survey measure for the belief in the reasonableness of others - the Faith in Reason scale - is tested on a representative sample of US voters. This measure shows there is a wide natural variation in responses, including across the political spectrum, and that the measure has a moderate test-retest stability after a two week delay. The 2024 US presidential election provides an opportunity for a planned quasi-experimental longitudinal investigation, demonstrating that the measure is responsive to events. Those disappointed by the election outcome reported a lowered belief in the reasonableness of others following the election. The post-election difference between those supporting the winning and losing candidates was large (Cohen's d = 0.77). Those whose confidence was highest in the opposite outcome, as demonstrated by willingness to wager on the size of their bonus payment for participation, demonstrated the largest reduction in their faith in reason, post-election.

BackgroundPeer review-based funding evaluation systems are under increasing pressure. To help alleviate this, innovative approaches like Distributed Peer Review (DPR) – where applicants review proposals for the same funding call – are being explored. The Volkswagen Foundation trialled DPR alongside conventional panel review for an interdisciplinary call. Evaluation of this trial provides insights into the dynamics of (distributed) peer review. MethodsAll proposals (N=140) underwent both DPR and panel review. In DPR, applicants (N= 323) reviewed four or five proposals, providing an overall score, scores for each funding criteria, and a comment. Proposals were ranked using a trimmed mean score. Expectation and feedback surveys were administered. Panel review involved shortlisting by VWS staff, panellist review (N=8), and a final panel discussion. Semi-structured interviews with applicants, panellists, and VWS staff (N=20) explored perceptions and experiences of DPR, using thematic analysis. Quantitative analysis of review data and surveys utilised bootstrapping to assess decision stability and mixed effects models for consistency between reviewers. Qualitative and quantitative results were integrated in a convergent parallel design. ResultsDPR recommended ten proposals for funding, while the panel recommended eleven; three overlapped. Funding criteria strongly predicted overall score; ‘Originality’ was given most weight by both sets of reviewers. Stability of DPR funding decisions was modest; across 500 bootstrapped samples a mean of 5.49 out of ten previously funded proposals would still be funded. This may be explained by high inter-reviewer variability: more variance was attributable to between-reviewer differences (20.04%) than between-proposal differences (8.87%). Advantages of DPR included increased efficiency, faster decision times, detailed feedback to applicants, and democratisation of funding evaluation. Noted disadvantages were burden on applicants, absence of discussion, and perceived lack of reviewer expertise. Funders implementing DPR must consider the high initial effort, potential for ‘gaming’, and reduced direct influence. ConclusionsThis trial demonstrates feasibility of DPR for an interdisciplinary funding programme. Implementing a DPR process requires funders to accept trade-offs, but can offer valuable benefits, including shortening time to funding decision. Building on established peer review principles, it has the potential to contribute to the democratisation of funding evaluation.

BackgroundPeer review-based funding evaluation systems are under increasing pressure. To help alleviate this, innovative approaches like Distributed Peer Review (DPR) – where applicants review proposals for the same funding call – are being explored. The Volkswagen Foundation trialled DPR alongside conventional panel review for an interdisciplinary call. Evaluation of this trial provides insights into the dynamics of (distributed) peer review. MethodsAll proposals (N=140) underwent both DPR and panel review. In DPR, applicants (N= 323) reviewed four or five proposals, providing an overall score, scores for each funding criteria, and a comment. Proposals were ranked using a trimmed mean score. Expectation and feedback surveys were administered. Panel review involved shortlisting by VWS staff, panellist review (N=8), and a final panel discussion. Semi-structured interviews with applicants, panellists, and VWS staff (N=20) explored perceptions and experiences of DPR, using thematic analysis. Quantitative analysis of review data and surveys utilised bootstrapping to assess decision stability and mixed effects models for consistency between reviewers. Qualitative and quantitative results were integrated in a convergent parallel design. ResultsDPR recommended ten proposals for funding, while the panel recommended eleven; three overlapped. Funding criteria strongly predicted overall score; ‘Originality’ was given most weight by both sets of reviewers. Stability of DPR funding decisions was modest; across 500 bootstrapped samples a mean of 5.49 out of ten previously funded proposals would still be funded. This may be explained by high inter-reviewer variability: more variance was attributable to between-reviewer differences (20.04%) than between-proposal differences (8.87%). Advantages of DPR included increased efficiency, faster decision times, detailed feedback to applicants, and democratisation of funding evaluation. Noted disadvantages were burden on applicants, absence of discussion, and perceived lack of reviewer expertise. Funders implementing DPR must consider the high initial effort, potential for ‘gaming’, and reduced direct influence. ConclusionsThis trial demonstrates feasibility of DPR for an interdisciplinary funding programme. Implementing a DPR process requires funders to accept trade-offs, but can offer valuable benefits, including shortening time to funding decision. Building on established peer review principles, it has the potential to contribute to the democratisation of funding evaluation.

Understanding our ideological opponents is crucial for the effective exchange of arguments and the avoidance of escalation, and the reduction of conflict. We operationalize the idea of an “Ideological Turing Test” to measure the accuracy with which people represent the arguments of their ideological opponents. Crucially, this offers a behavioral measure of open‐mindedness which goes beyond mere self‐report. We recruited 200 participants from opposite sides of three topics with potential for polarization in the UK of the early 2020s (1200 participants total). Participants were asked to provide reasons both for and against their position. Their reasons were then rated by participants from the opposite side. Our criteria for “passing” the test was if an argument was agreed with by opponents to the same extent or higher than arguments made by proponents. We found evidence for high levels of mutual understanding across all three topics. We also found that those who passed were more open‐minded toward their opponents, in that they were less likely to rate them as ignorant, immoral, or irrational. Our method provides a behavioral measure of open‐mindedness and ability to mimic counterpartisan perspectives that goes beyond self‐report measures. Our results offer encouragement that, even in highly polarized debates, high levels of mutual understanding persist.