Filippo Gambarota

Meta-analyses, like primary studies, are affected by researcher degrees of freedom: researchers must decide which studies to include, how to compute effect sizes, and which statistical models to use. Different reasonable choices can lead to different results, making multiverse meta-analysis a useful tool for assessing robustness across alternative specifications. However, multiverse meta-analysis also introduces a multiple-testing problem. Current inferential approaches mainly provide a global test, which can establish that at least one specification differs from the null but cannot identify which individual specifications do so while accounting for multiplicity across the full multiverse. We introduce Post-selection Inference in Multiverse Meta-Analysis (PIMMA), which allows inference on individual specifications while controlling the Family-Wise Error Rate (FWER) and accounting for their dependence. In practical terms, PIMMA shows which results remain statistically significant after accounting for multiplicity across all reasonable analytical choices. In simulations, the specification-level test adequately controlled Type I error, with power comparable to robust variance estimation in scenarios where both methods maintained appropriate Type I error control. In an application to meta-analyses of Cognitive Behavioral Therapy efficacy for adult depression, 297 of 336 unique meta-analytic analyses remained significant after step-down max-T adjustment for multiple testing, accounting for dependence among analyses. PIMMA therefore extends multiverse meta-analysis from a global assessment of whether any effect is present to specification-level inference, allowing researchers to evaluate robustness across specifications and identify which individual results remain significant under appropriate error control.

The probability of direction (𝑝𝑑) has seen rapid adoption in psychological research as an intuitive posterior summary that quantifies how strongly the posterior distribution favors one direction of an effect over the other. In practice, researchers often turn 𝑝𝑑 into a binary decision rule by applying fixed thresholds (e.g., 𝑝𝑑 > 0.975), declaring evidence for an effect when posterior mass concentrates strongly on one side. When this rule is applied across many parameters, it can substantially increase the risk of misleading conclusions. We argue that this problem stems from neglecting the prior probability that for every tested effect, the data-favored direction does not reflect a genuine effect. This quantity, often left implicit, becomes increasingly consequential as the number of tested effects grows. To address this, we propose a prior-odds adjustment to 𝑝𝑑 that requires researchers to explicitly specify the proportion of tested effects they expect to be null. The adjustment accounts for multiplicity without modifying the model used in parameter estimation, offers a transparent mechanism for incorporating priorskepticism about the existence of effects, and reduces the risk of false discoveries in multi-parameter settings. We evaluate its behavior through simulation across varying sample sizes and proportions of null effects, and provide an R package to facilitate applied use.

This study investigates the retention of visual information in visual working memory (VWM) when individuals are unaware of it, aiming to provide clear-cut evidence for an unconscious VWM effect. To explore the underlying neural mechanisms, we monitored a critical ERP component, specifically the contralateral delay activity (CDA), which reflects VWM maintenance. Participants performed a change detection task in which to-be-memorized Gabor patches were presented at a visibility threshold, determined to assess subjective awareness using the Perceptual Awareness Scale. Participants performed above chance level in the change detection task even when the visibility of the Gabor patches was subthreshold, indicating retention of visual information without conscious awareness. Notably, in a subsample of participants, a reliable CDA amplitude was observed during unaware trials, in which participants performed correctly, compared to trials with incorrect responses. As a proof of concept, this finding indexed short active maintenance of unaware visual information in VWM, which could be used to perform VWM-based tasks. In conclusion, the results of our study support the existence of an active retention of unaware visual information in VWM. These findings challenge the notion of entirely activity-silent working memory by showing that unconscious information is maintained through active neural firing (CDA), potentially transitioning to activity-silent mechanisms in later phases.

The Music Ensemble dataset is a large-scale, cross-national database that provides detailed information about the musical, cognitive, personality, and demographic profiles of young adult musicians and nonmusicians. Data were collected from 1438 participants (aged 18–30) across thirty-five research sites in Europe, North America, South America, and Australia. Participants completed an in-person, in-lab battery of objective tests, including measures of verbal, visuospatial and musical short-term memory, executive functions (updating component), nonverbal reasoning, verbal comprehension, and music perception skills. The battery also included standardized and custom self-report questionnaires assessing music sophistication, music reward, personality traits, socioeconomic status, and demographic characteristics. Music Ensemble was preregistered, and the research protocol followed a standardized procedure across sites. The dataset also includes a large subsample of musicians and nonmusicians that are pair-matched for age, gender, and education (678 pairs). It enables well-powered investigations into the relationship between musical expertise and individual differences in cognition, personality, and demographic variables. It is also suitable for training in statistical and psychometric methods.

Although in recent years some studies have found evidence suggesting that working memory (WM) may operate on unconscious perceptual contents, decisive demonstrations of the existence of unconscious WM are lacking. In the present Registered Report, we replicate the first study on this topic by Soto et al. (Working memory without consciousness. Curr Biol 2011;21:R912–3.): a visual discrimination task asking participants to report the direction in which a subliminal Gabor grating was rotated after a 2-s delay. We acquired a multisite sample from 19 laboratories, with a larger number of participants (N = 531) and trials (720 in two sessions) than those typically used in previous studies. As a result, a large-sample, international, and open-access dataset is now available for researchers and future analyses. Furthermore, some minimal baseline requirements were guaranteed for the experimental task (i.e. number of valid trials, motivation, and consistent labels for the Perceptual Awareness Scale). The results showed (1) above-chance WM performance in cue-present trials reported as unseen (.55 accuracy), (2) a significant positive correlation between WM performance and cue detection sensitivity (r = .228), and (3) a significant above-chance intercept in the regression of performance on sensitivity (β0 = .521). These findings suggest that WM can operate on unconscious representations, although it remains positively associated with perceptual sensitivity. Crucially, because measurement error could compromise the interpretation of these three results, we provide evidence for our measures’ excellent reliability and, more fundamentally, for their validity.

Although in recent years some studies have found evidence suggesting that Working Memory (WM) may operate on unconscious perceptual contents, decisive demonstrations of the existence of unconscious WM are lacking. In the present Registered Report, we replicate the first study on this topic by Soto et al. (2011): a visual discrimination task asking participants to report the direction in which a subliminal Gabor grating was rotated after a 2-second delay. We acquired a multisite sample from 19 laboratories, with a larger number of participants (N = 531) and trials (720 in two sessions) than those typically used in previous studies. As a result, a large-sample, international, and open-access dataset is now available for researchers and future analyses. Furthermore, some minimal baseline requirements were guaranteed for the experimental task (i.e., number of valid trials, motivation, and consistent labels for the Perceptual Awareness Scale). The results showed: (1) above-chance WM performance in cue- present trials reported as unseen (.55 accuracy), (2) a significant positive correlation between WM performance and cue detection sensitivity (r = .228), and (3) a significant above-chance intercept in the regression of performance on sensitivity (β0 = .521). These findings suggest that WM can operate on unconscious representations, although it remains positively associated with perceptual sensitivity. Crucially, because measurement error could compromise the interpretation of these three results, we provide evidence for our measures’ excellent reliability and, more fundamentally, for their validity.

Musicians are often regarded as a positive example of brain plasticity and associated cognitive benefits. This emerges when experienced musicians (e.g., musicians with more than 10 years of music training and practice) are compared with nonmusicians. A frequently observed behavioral finding is a short-term memory advantage of the former over the latter. Although available meta-analysis reported that the effect size of this advantage is medium (Hedges’s g = 0.5), no literature study was adequately powered to estimate reliably an effect of such size. This multilab study has been ideated, realized, and conducted in lab by several groups that have been working on this topic. Our ultimate goal was to provide a community-driven shared and reliable estimate of the musicians’ short-term memory advantage (if any) and set a method and a standard for future studies in neuroscience and psychology comparing musicians and nonmusicians. Thirty-three research units recruited a total of 600 experienced musicians and 600 nonmusicians, a number that is sufficiently large to estimate a small effect size (Hedges’s g = 0.3) with a high statistical power (i.e., 95%). Subsequently, we measured the difference in short-term memory for musical, verbal, and visuospatial stimuli. We also looked at cognitive, personality, and socioeconomic factors that might mediate the difference. Musicians had better short-term memory than nonmusicians for musical, verbal, and visuospatial stimuli with an effect size of, respectively, Hedges’s g s = 1.08 (95% confidence interval [CI] = [0.94, 1.22]; large), 0.16 (95% CI = [0.02 0.30]; very small), and 0.28 (95% CI = [0.15, 0.41]; small). This work sets the basis for sound research practices in studies comparing musicians and nonmusicians and contributes to the ongoing debate on the possible cognitive benefits of musical training.

The Music Ensemble dataset is a large-scale, cross-national database that provides detailed information about the musical, cognitive, personality, and demographic profiles of musicians and nonmusicians. Data were collected from 1438 participants (aged 18–30) across thirty-five research sites in Europe, North America, South America, and Australia. Participants completed an in-person, in-lab battery of objective tests, including measures of verbal, visuospatial and musical short-term memory, executive functions (updating component), nonverbal reasoning, verbal comprehension, and music perception skills. The battery also included standardized and custom self-report questionnaires assessing music sophistication, music reward, personality traits, socioeconomic status, and demographic characteristics. Music Ensemble was preregistered, and the research protocol followed a standardized procedure across sites. The dataset also includes a large subsample of musicians and nonmusicians that are pair-matched for age, gender, and education (678 pairs). It enables well-powered investigations into the relationship between musical expertise and individual differences in cognition, personality, and demographic variables. It is also suitable for training in statistical and psychometric methods.

The dynamic and context-appropriate shift between maintaining and updating goal-relevant information in working memory (WM) is believed to be governed by a fronto-striatal gating system. This system regulates the entry of information into WM, by deciding which information should be allowed in and which information should be prevented from entering WM. Prior research suggests that acute stress may impact WM functioning and associated cortical and subcortical regions. However, it remains unclear which specific component processes of WM are affected by acute stress. Here we used the reference-back paradigm to disentangle potential effects of acute stress on WM maintenance, updating, gate opening, and gate closing processes. Ninety-six participants completed the task twice: once at baseline and again while undergoing either a stress or control manipulation, in a between-subjects design. While no stress effects emerged for reaction times, acute stress detrimentally impacted accuracy on trials requiring WM maintenance. Drift diffusion modeling revealed that acute stress reduced overall drift rates (information processing speed), with larger group differences on maintenance trials. No effects emerged for response caution. To contextualize our findings, we conducted a meta-analysis (42 studies, 2008 participants, 210 effects). WM tasks were categorized by key demands: maintenance only, maintenance + overcoming competition, and maintenance + updating. The meta-analysis revealed impaired accuracy in tasks requiring both maintenance and updating, like the reference-back paradigm in our own study, but not tasks without updating demands. Collectively, our findings suggest that acute stress impacts response accuracy but not speed, and only in tasks with both maintenance and updating demands.

Critical-effect-size values represent the smallest detectable effect that can reach statistical significance given a specific sample size, alpha level, and test statistic. It can be useful to calculate the critical effect size when designing a study and evaluate whether such effects are plausible. Reporting critical-effect-size values may be useful when the sample size has not been planned a priori, there is uncertainty about the expected sample size that can be collected, or researchers plan to analyze the data with a statistical hypothesis test. To assist researchers in calculating critical-effect-size values, we developed an R package that allows researchers to report critical-effect-size values for group comparisons, correlations, linear regressions, and meta-analyses. Reflecting on critical-effect-size values could benefit researchers during the planning phase of the study by helping them to understand the limitations of their research design. Critical-effect-size values are also useful when evaluating studies performed by other researchers when a priori power analyses were not performed, especially when nonsignificant results are observed.