Malte Elson

Netlist Reverse Engineering (NRE) is a critical human-computer interaction task for identifying security threats at the microchip level. Yet, little is known about how individual factors shape NRE performance and practice effects. We conducted a mixed experimental and quasi-experimental study with N = 155 participants, randomly assigned to a practice or a no-practice control condition, with practice difficulty adapted to baseline performance. After a baseline assessment, practice groups completed a 10-minute NRE practice phase, while the control group engaged in an unrelated task. Performance was reassessed following a one- to two-night consolidation phase. Multilevel modeling revealed a significant practice effect on both speed and accuracy; adjusting for regression toward the mean, we found no evidence that its size depends on baseline performance. Self-rated prior knowledge predicted baseline performance but not learning gains. Our findings are consistent with perceptual learning as one plausible mechanism, and suggest that benefiting from practice does not require prior NRE experience.

With transcranial direct‐current stimulation's (tDCS) popularity both in motor learning research and as a commercial product, it is important that the quality of evidence on its effectiveness be evaluated. Special attention should be paid to meta‐analyses, as they usually have a large impact on research and clinical practice. The aim of this verification report was to gain insight on the methodological quality of meta‐analyses estimating the effect of tDCS on motor learning. To that end, we verified the methodology of three meta‐analyses with respect to reproducibility as the main focus, and reporting quality and publication bias control as secondary aspects. The three meta‐analyses we verified largely adhered to PRISMA reporting guidelines and reported the primary effect sizes and sampling variances/confidence intervals they calculated, enabling successful reproductions of pooled effect size estimates. However, akin to previous meta‐research with similar aims, we found the methods and results sections of the meta‐analyses to be severely underreported, which compromises the ability to judge the soundness of the methodological procedure adopted as well as its reproducibility. While publication bias detection methods were applied, the approaches chosen do not allow for well informed decisions about the presence or extent of publication bias. These results reemphasise the need to transparently report methods in meta‐analyses and to meticulously evaluate their quality before and after publication.

Valid generalisations from sample to population require representative samples. This is especially the case for psychological test norms as they are only useful to the extent that the normative samples can stand in for reference populations. However, as a field, we continue to face difficulties defining, acquiring, and describing representative samples. Having engaged with similar problems for a long time, political and other social scientists have developed a sophisticated set of tools for dealing with them. In this article, we therefore draw on insights from these fields to illustrate what measures need to be taken at all research stages to maximise sample representativeness, and the potential benefits of sample weighting in psychology. We argue that a specific class of weighting methods, Regularised Prediction and Poststratification (RPP) holds promise for improving psychological test norming practice. RPP involves fitting a regularised prediction model to a large and diverse sample and weighting predicted scores by true population values (sourced from census data, for example). Using IQ test data from the TwinLife study (N = 9,980, Culture Fair Test, CFT 20-R) as a case study, we show that RPP yields IQ scores that differ by up to 19 IQ points (on average by 4.19 points) from those reported in the CFT 20-R manual, which were based on traditional norming methods. We additionally observed sizeable differences to other norming/weighting approaches, such as semiparametric continuous norming and using weights provided by the sample provider . Differences of this magnitude can lead to decision errors with potentially life-altering consequences (e.g., about intellectual disability). Beyond individual diagnostics, we contend that psychology could benefit from more widely available, high-quality norms for many aspects of research and practice and plead for the change in research practice necessary to make these benefits possible.

Background: Many people with mental health problems do not receive the care they need. Digital mental health interventions have been shown to be effective in many trials and offer a promising way to reach more people in need. However, their uptake and use remain limited, in part due to concerns about data privacy. However, these concerns may not be equally significant for all users. Objective: This trial aims to evaluate the efficacy of an anonymous online self-help tool for social anxiety that processes no personal data and has some disadvantages in terms of usability compared to an otherwise identical, nonanonymous evidence-based tool. Furthermore, the trial will investigate user preferences regarding the 2 program versions and evaluate the impact of these preferences on treatment outcomes. Methods: In this partially randomized patient preference trial, 2 versions (anonymous vs nonanonymous) of the same previously researched program will be compared among 452 participants with increased levels of social anxiety. Half of the participants will be randomized to one of the program versions, whereas the other half will be assigned to their preferred version. The primary outcome is social anxiety symptoms. Secondary outcomes include depression, stigma of mental illness, and quality of life. Results: Recruitment started in September 2024. As of the date of manuscript submission in May 2025, over 350 participants have been included in the study. Recruitment ended in September 2025, and results are expected to be available and published in 2026. Conclusions: The results of this trial will determine whether anonymously accessible interventions, which could easily be made freely available, are less efficacious than their nonanonymous counterparts and will explore the impact of user preferences on treatment outcomes. The findings could contribute to making digital interventions more accessible and tailoring interventions to individual preferences. Trial Registration: ClinicalTrials.gov NCT06465589; https://clinicaltrials.gov/study/NCT06465589 International Registered Report Identifier (IRRID): DERR1-10.2196/77573

We examined the extent to which constructs and measures have proliferated in psychological science. We integrated two large databases obtained from the American Psychology Association that have been used to keep track of constructs, measures, and research in the psychological-science literature for the past 30 years. In our descriptive analyses, we found that (a) thousands of new constructs and measures are published each year, (b) most measures are used very few times, and (c) there is no trend toward consensus or standardization in the use of constructs and measures; in fact, there is a trend toward even greater fragmentation over time. That is, constructs and measures are proliferating. We conclude that measurement in the psychological-science literature is fragmented, creating problems such as redundancy and confusion and stifling cumulative scientific progress. We conclude by providing suggestions for what researchers can do about this problem.

Zhao et al.’s (2023) recent meta-analysis of ACT vs. control interventions for depressive disorders reported an overall meta-analytic effect size of Hedges g = -1.05. However, errors appear to have been made in the extraction of effect sizes. Re-extraction of the effect sizes from the original articles suggests that 3 of 8 recalculated effect sizes were erroneous: Standard Errors (SE) were confused for Standard Deviations (SD) in two cases, and incorrect sample sizes were extracted in one case. When the first meta-analysis they report was re-run following Zhao et al.’s (2023) analytic strategy, the resulting meta-analytic effect size of Hedges’ g = 0.68 was 35% smaller than that reported in the original meta-analysis. The results of other subgroup meta-analyses reported by Zhao et al. (2023) are likely also to be affected. Other aspects of Zhao et al. (2023) may also require scrutiny for the presence of errors. Given the detected errors in the results, the conclusions of Zhao et al. (2023) are undermined and likely require correction. All data and code available at osf.io/jwcmu.

BACKGROUND Many people with mental health problems do not receive the care they need. Digital mental health interventions have been shown to be effective in many trials and offer a promising way to reach more people in need. However, their uptake and use remain limited, in part due to concerns about data privacy. However, these concerns may not be equally significant for all users. OBJECTIVE This trial aims to evaluate the efficacy of an anonymous online self-help tool for social anxiety that processes no personal data and has some disadvantages in terms of usability compared to an otherwise identical, nonanonymous evidence-based tool. Furthermore, the trial will investigate user preferences regarding the 2 program versions and evaluate the impact of these preferences on treatment outcomes. METHODS In this partially randomized patient preference trial, 2 versions (anonymous vs nonanonymous) of the same previously researched program will be compared among 452 participants with increased levels of social anxiety. Half of the participants will be randomized to one of the program versions, whereas the other half will be assigned to their preferred version. The primary outcome is social anxiety symptoms. Secondary outcomes include depression, stigma of mental illness, and quality of life. RESULTS Recruitment started in September 2024. As of the date of manuscript submission in May 2025, over 350 participants have been included in the study. Recruitment ended in September 2025, and results are expected to be available and published in 2026. CONCLUSIONS The results of this trial will determine whether anonymously accessible interventions, which could easily be made freely available, are less efficacious than their nonanonymous counterparts and will explore the impact of user preferences on treatment outcomes. The findings could contribute to making digital interventions more accessible and tailoring interventions to individual preferences. CLINICALTRIAL ClinicalTrials.gov NCT06465589; https://clinicaltrials.gov/study/NCT06465589 INTERNATIONAL REGISTERED REPORT DERR1-10.2196/77573

Cronbach’s α is the most widely reported metric of the reliability of psychological measures. Decisions about an observed α’s adequacy are often made using rule-of-thumb thresholds, such as α of at least .70. Such thresholds can put pressure on researchers to make their measures meet these criteria, similar to the pressure to meet the significance threshold with p values. We examined whether α values reported in the psychology literature are inflated at the rule-of-thumb thresholds (αs = .70, .80, .90) because of, for example, overfitting to in-sample data (α-hacking) or publication bias. We extracted reported α values from three very large data sets covering the general psychology literature (> 30,000 α values taken from > 74,000 published articles in American Psychological Association [APA] journals), the industrial and organizational (I/O) psychology literature (> 89,000 α values taken from > 14,000 published articles in I/O journals), and the APA’s PsycTests database, which aims to cover all psychological measures published since 1894 (> 67,000 α values taken from > 60,000 measures). The distributions of these values show robust evidence of excesses at the α = .70 rule-of-thumb threshold that cannot be explained by justifiable measurement practices. We discuss the scope, causes, and consequences of α-hacking and how increased transparency, preregistration of measurement strategy, and standardized protocols could mitigate this problem.