Matthew J. Valente

Objective: The objective of this study was to determine the methodological quality of published systematic reviews of observational studies and their respective protocol registrations. Introduction: Historically, methodological guidance for systematic reviews has focused on systematic reviews of randomized controlled trials. Guidance for systematic reviews of observational studies emerged about a decade ago. However, little is known about the methodological quality of systematic reviews of observational studies published to date. Eligibility criteria: Systematic reviews were eligible for inclusion if they i) included observational studies and quasi-experimental studies with human participants, ii) extracted exposure-outcome effect estimates from primary studies, iii) were written in English, and iv) were published in a Q1 Web of Science−indexed or Scopus-indexed epidemiology journal between January 1, 2023, and December 31, 2023 (including advance online publications). Methods: This methodological study is a follow-up analysis of a random sample of 50 published systematic reviews of observational epidemiologic studies that were originally identified to examine whether control for third variables was considered as a source of bias or heterogeneity. Where possible, the methods were informed by JBI guidance for umbrella reviews. English-language Q1 epidemiology journals in Scopus and Web of Science were searched to identify records. The search was first performed on January 16, 2024, and repeated on April 1, 2024. For this follow-up analysis, protocol registrations corresponding to the 50 included systematic reviews were retrieved. Data extraction and critical appraisal of the full-text systematic reviews and the protocol registrations were performed independently by 2 reviewers. Methodological quality of the full-text systematic reviews and protocol registrations was assessed using items 1-9 from the JBI Critical Appraisal Checklist for Systematic Reviews and Research Syntheses. Results: Of the 50 included systematic reviews, 25 had protocol registrations. The median number of methodological quality items met by the full-text systematic reviews with a protocol registration was 3 (range 1-7) and without a protocol registration was 2 (range 0-5). Item 5, concerning the conduct of critical appraisal, was the only item on which systematic reviews with a protocol registration scored statistically significantly better than systematic reviews without a protocol registration. The median number of methodological quality items met by the protocol registrations was 3 (range 1-6). Agreement in the methodological quality assessment between full-text systematic reviews and protocol registrations was generally high (ie, items were often met or not met across both full-text systematic reviews and protocol registrations). Conclusions: The methodological quality of the included systematic reviews of observational studies and their respective protocol registrations was generally low. Further research is needed to identify effective strategies for improving the methodological quality of systematic reviews of observational studies.

Interrupted Time Series (ITS) designs are commonly used to estimate intervention effects in public health, epidemiology, clinical psychology, and many othe

Recently, Baseline Target Moderated Mediation (BTMM) has received a lot of attention in the field of prevention science. Prevention scientists are interested in BTMM because the model goes beyond whether an intervention achieves effects but also details how and for whom the intervention is most effective. In BTMM, baseline measures are used to investigate potential baseline-by-treatment interactions. However, BTMM has some important challenges including how to incorporate multiple moderator variables when identifying subgroups that benefit the most from the intervention and how to interpret subgroup effects in the presence of multiple moderator variables. Further, with the emergence of causal mediation analysis, it is important to investigate potential treatment-by-mediator interactions which allow the posttest mediator-outcome relation to vary in magnitude across intervention groups. Few methodological developments have addressed the challenges of assessing BTMM in the presence of multiple baseline-by-treatment interactions and the treatment-by-posttest mediator interaction. If the goal is to identify subgroups of individuals who respond better/worse to the intervention, it is important to use a method that can handle the many possible interactions while capturing the heterogeneity within the subgroups of interest. There are three aims of this paper. First, we describe the methodological challenges and substantive interpretation of mediation effects in the presence of multiple moderating variables. Second, we describe two statistical methods to estimate conditional mediation effects in the presence of multiple moderating variables. Third, the methods are applied to an empirical example from the ATLAS study. Implications for BTMM are discussed.

Moderation analysis is used to study under what conditions or for which subgroups of individuals a treatment effect is stronger or weaker. When a moderator variable is categorical, such as assigned sex, treatment effects can be estimated for each group resulting in a treatment effect for males and a treatment effect for females. If a moderator variable is a continuous variable, a strategy for investigating moderated treatment effects is to estimate conditional effects (i.e., simple slopes) via the pick-a-point approach. When conditional effects are estimated using the pick-a-point approach, the conditional effects are often given the interpretation of "the treatment effect for the subgroup of individuals…." However, the interpretation of these conditional effects as subgroup effects is potentially misleading because conditional effects are interpreted at a specific value of the moderator variable (e.g., +1 SD above the mean). We describe a simple solution that resolves this problem using a simulation-based approach. We describe how to apply this simulation-based approach to estimate subgroup effects by defining subgroups using a range of scores on the continuous moderator variable. We apply this method to three empirical examples to demonstrate how to estimate subgroup effects for moderated treatment and moderated mediated effects when the moderator variable is a continuous variable. Finally, we provide researchers with both SAS and R code to implement this method for similar situations described in this paper. (PsycInfo Database Record (c) 2025 APA, all rights reserved).

Single-Case Experimental Designs (SCEDs), or N-of-1 trials, are commonly used to estimate intervention effects in many disciplines including in the treatment of youth mental health problems. SCEDs consist of repeated measurements of an outcome over time for a single case (e.g., student or patient) throughout one or more baseline phases and throughout one or more intervention phases. The manipulation of the baseline and intervention phase make the SCED a type of interrupted time series design, which is considered one of the most effective experimental designs for causal inference. An important step towards understanding why interventions are effective at producing a change in the outcome is through the investigation of mediating mechanisms. Hypotheses of mediating mechanisms involve an intervention variable which is hypothesized to affect an outcome through its effect on a mediating variable. Little work has attempted to combine mediation analysis and ABAB reversal designs. Therefore, the goals of this paper are to define, estimate, and interpret mediation effects for ABAB reversal designs. An empirical example is used to demonstrate how to estimate and interpret the mediation effects. R code is provided for researchers interested in estimating mediation effects in single-case reversal designs.