Maureen Dobbins

Introduction: During the COVID-19 pandemic, the National Collaborating Centre for Methods and Tools created a Rapid Evidence Service (RES) and Rapid Review Repository (RRR) to address public health decision-making challenges. Few evaluations have assessed if and/or how rapid reviews influenced decision-making. Aims: This study sought to (1) explore the reach, quality, use, and impact of the RES and RRR; and (2) identify enabling or hindering factors. Methods: A convergent mixed methods study including a cross-sectional study and a qualitative descriptive evaluation was conducted among decision-makers and evidence synthesis organizations. The findings from each data collection method were merged thematically to draw conclusions. Results: A total of 44 individuals participated. Thirty-eight were public health decision-makers and six represented evidence synthesis organizations. The findings demonstrate that RES reviews reached decision-makers across Canada and globally, were used in decision-making, and influenced decisions. The RRR helped reduce duplication of reviews, promoted collaboration, and supported sharing of emerging evidence. Collectively, quality, credibility, trustworthiness, and usefulness influenced satisfaction, which emerged as a critical driver of reach, use, and impact. Conclusions: The RES and RRR supported evidence-informed decision-making during the pandemic. Previously established relationships and product satisfaction with decision-makers were critical in enabling reach at the outset of the pandemic, while maintaining satisfaction during the pandemic was key to ongoing reach and for reviews to be used to influence decisions. These findings provide direction on actions to prepare for the next public health crisis. Spanish https://links.lww.com/IJEBH/A669

Background: Evaluating the methodological quality of primary studies is a crucial aspect of evidence syntheses, such as rapid reviews. Rapid reviews often include both randomised controlled trials (RCTs) and non-randomised studies (NRSIs) requiring multiple design-specific quality assessment tools. This can complicate workflows and reduce efficiency. Unified tools, designed to assess methodological quality across diverse study designs, may offer a more consistent and streamlined approach. This scoping review aimed to comprehensively identify and describe unified tools for assessing methodological quality across both RCTs and NRSIs. Methods: The review followed JBI scoping review methodology and was reported in line with PRISMA-ScR guidance. Searches were conducted in MEDLINE (Ovid), Embase (Ovid), and CINAHL (EBSCO) from 1998 to 2024, alongside grey literature searches, citation tracking, targeted website searches, and consultation with experts in evidence synthesis. Study selection was performed by two independent reviewers. A tool was defined as any structured instrument developed to support users in assessing methodological quality. Results: A total of 55 publications were included, identifying 29 unique unified tools. These were categorised by structure: scales ( n = 14), checklists with an additional summary judgement ( n = 7), simple checklists ( n = 5), domain-based tools ( n = 2), and other tools ( n = 1). All tools were designed to assess both RCTs and NRSIs, with many also applicable to additional study designs, including descriptive studies ( n = 15), case reports or series ( n = 9), qualitative research ( n = 7), and systematic reviews ( n = 3). Unified tools were developed using a mixture of approaches including literature reviewing ( n = 25), expert consensus ( n = 20), and stakeholder consultations ( n = 17) and pilot testing ( n = 17). Evaluation of psychometric properties was conducted to varying levels across different domains of validity and reliability. Conclusion: This review identified a diverse set of unified tools for assessing methodological quality across both RCTs and NRSIs. However, variation in tool structure, the availability of accompanying guidance, and the limited evaluation of psychometric properties, particularly inter-rater reliability and criterion validity, indicate potential issues for adoption in rapid reviews. Unified tools need further refinement and validation to be embedded in the rapid review process. Systematic review registration: https://osf.io/nyteu/

Background: Evaluating the methodological quality of primary studies is a crucial aspect of evidence syntheses, such as rapid reviews. Rapid reviews often include both randomised controlled trials (RCTs) and non-randomised studies (NRSIs) requiring multiple design-specific methodological quality assessment tools. This can complicate workflows and reduce efficiency. Unified tools, designed to assess methodological quality across diverse study designs, may offer a more consistent and streamlined approach. This scoping review aimed to comprehensively identify and describe unified tools for assessing methodological quality across both RCTs and NRSIs. Methods: The review followed JBI scoping review methodology and was reported in line with PRISMA-ScR guidance. Searches were conducted in MEDLINE (Ovid), Embase (Ovid), and CINAHL (EBSCO) from 1998 to 2024, alongside grey literature searches, citation tracking, targeted website searches, and consultation with experts in evidence synthesis. Study selection was performed by two independent reviewers. A tool was defined as any structured instrument developed to support users in assessing methodological quality. Results: A total of 55 publications were included, identifying 29 unique unified tools. These were categorized by structure: scales (n=14), checklists with judgment (n=7), simple checklists (n=5), domain-based tools (n=2), and other tools (n=1). All tools were designed to assess both RCTs and NRSIs. Of these, 13 focused exclusively on RCTs and NRSIs, while others extended to descriptive studies (n=15), case reports or series (n=9), qualitative research (n=7), and systematic reviews (n=3). Unified tools were developed using a mixture of approaches including literature reviewing (n=25), expert consensus (n=20), and stakeholder consultations (n=17). Reports of pilot testing was identified for 17 unified tools, and evaluation of psychometric properties was conducted to varying levels across different domains of validity and reliability. Conclusion: This review identified a diverse set of unified tools for assessing methodological quality across both RCTs and NRSIs. However, variation in tool structure, the availability of accompanying guidance, and the limited evaluation of psychometric properties, particularly interrater reliability and criterion validity, indicate potential issues for adoption in rapid reviews. Unified tools need further refinement and validation to be embedded in the rapid review process. Systematic review registration: https://osf.io/nyteu/

Background: Dementia prevention through the reduction of modifiable risk factors is gaining attention as a public health strategy. However, public knowledge of dementia risk and protective factors remains low. Web-based education offers a potential solution to raise awareness and promote risk-reduction behaviors. Objective: This randomized controlled trial evaluated the effectiveness of DementiaRisk.ca, an internet-based multimedia educational intervention, in increasing knowledge of dementia risk factors, intentions to engage in risk reduction behaviors, and changes in health behaviors. Methods: A 2-arm randomized controlled trial was conducted with 510 participants (265 in the intervention group and 245 in the control group). Participants were randomized to receive either the e-learning about dementia risk and promoting brain health, which included a multimedia lesson and microlearning emails, or a control intervention focused on mild cognitive impairment. Outcomes included knowledge of dementia risk factors, intentions to engage in risk reduction, and health behaviors, measured at baseline (T1), 4 weeks (T2), and 2 months postintervention (T3). Outcomes were analyzed using linear mixed effects models with fixed effects for group, time, and their interaction, and a random intercept for participants. Results: Of the 510 randomized participants, 405 (79.4%) completed all intervention components. Participants were predominantly female (n=309, 60.6%) and aged 55 years or older (n=284, 55.7%). Baseline mean dementia knowledge scores were 17.0 (SD 5.5) in the intervention group and 17.4 (SD 6.0) in the control group. At T2, scores increased to 25.8 (SD 4.5) and 23.6 (SD 5.1), respectively, yielding a between-group difference of 2.2 points (95% CI 1.2‐3.2; P <.001), which was sustained at T3. Both groups showed significant improvements in knowledge, intentions, and health behaviors over time, with larger knowledge gains in the intervention group and particularly among participants with lower educational attainment. Intentions to engage in dementia risk reduction improved in both groups at T2 (intervention: +1.0, 95% CI 0.2‐1.8; control: +1.4, 95% CI 0.5‐2.3), with no significant between-group difference. Self-reported physical activity increased from 31.7 (SD 25.0) to 38.6 (SD 27.5) in the intervention group and from 29.9 (SD 23.5) to 32.5 (SD 26.6) in the control group, with a between-group difference of 5.4 points at T2 (95% CI 0.3‐10.5; P =.04). No significant between-group differences were observed for diet, alcohol use, or other health behaviors. Qualitative findings indicated that participants valued the intervention for improving awareness of dementia risk factors, motivating proactive lifestyle changes, and enhancing confidence in applying prevention information. Conclusions: This internet-based dementia risk reduction e-learning program improved dementia-related knowledge and increased self-reported physical activity, particularly among participants with lower educational attainment. Effects on other health behaviors were limited. These findings support the use of well-designed e-learning as a scalable public health strategy to strengthen dementia risk reduction literacy and encourage selected healthy behaviors.

Background: Health Evidence provides access to quality appraisals for >10,000 evidence syntheses on the effectiveness and cost-effectiveness of public health and health promotion interventions. Maintaining Health Evidence has become increasingly resource-intensive due to the exponential growth of published literature. Innovative screening methods using artificial intelligence (AI) can potentially improve efficiency. Objective: The objectives of this project are to: (1) assess the ability of AI-assisted screening to correctly predict nonrelevant references at the title and abstract level and investigate the consistency of this performance over time, and (2) evaluate the impact of AI-assisted screening on the overall monthly manual screening set. Methods: Training and testing were conducted using the DistillerSR AI Preview & Rank feature. A set of manually screened references (n=43,273) was uploaded and used to train the AI feature and assign probability scores to each reference to predict relevance. A minimum threshold was established where the AI feature correctly identified all manually screened relevant references. The AI feature was tested on a separate set of references (n=72,686) from the May 2019 to April 2020 monthly searches. The testing set was used to determine an optimal threshold that ensured >99% of relevant references would continue to be added to Health Evidence. The performance of AI-assisted screening at the title and abstract screening level was evaluated using recall, specificity, precision, negative predictive value, and the number of references removed by AI. The number and percentage of references removed by AI-assisted screening and the change in monthly manual screening time were estimated using an implementation reference set (n=272,253) from November 2020 to 2023. Results: The minimum threshold in the training set of references was 0.068, which correctly removed 37% (n=16,122) of nonrelevant references. Analysis of the testing set identified an optimal threshold of 0.17, which removed 51,706 (71.14%) references using AI-assisted screening. A slight decrease in recall between the 0.068 minimum threshold (99.68%) and the 0.17 optimal threshold (94.84%) was noted, resulting in four missed references included via manual screening at the full-text level. This was accompanied by an increase in specificity from 35.95% to 71.70%, doubling the proportion of references AI-assisted screening correctly predicted as not relevant. Over 3 years of implementation, the number of references requiring manual screening was reduced by 70%, reducing the time spent manually screening by an estimated 382 hours. Conclusions: Given the magnitude of newly published peer-reviewed evidence, the curation of evidence supports decision makers in making informed decisions. AI-assisted screening can be an important tool to supplement manual screening and reduce the number of references that require manual screening, ensuring that the continued availability of curated high-quality synthesis evidence in public health is possible.

BACKGROUND Dementia prevention through the reduction of modifiable risk factors is gaining attention as a public health strategy. However, public knowledge of dementia risk and protective factors remains low. Web-based education offers a potential solution to raise awareness and promote risk reduction behaviours. OBJECTIVE This randomized controlled trial (RCT) evaluated the effectiveness of DementiaRisk.ca, an internet-based multimedia educational intervention, in increasing knowledge of dementia risk factors, intentions to engage in risk reduction behaviours, and changes in health behaviours. METHODS A two-arm RCT was conducted with 510 participants (265 in the intervention group and 245 in the control group). Participants were randomized to receive either the e-learning about dementia risk and promoting brain health, which included a multimedia lesson and micro-learning emails, or a control intervention focused on mild cognitive impairment. Outcomes included knowledge of dementia risk factors, intentions to engage in risk reduction, and health behaviours, measured at baseline, 4 weeks, and 2 months post-intervention. RESULTS Both groups showed significant improvements in knowledge, intentions, and health behaviours, with the intervention group having significantly higher gains in knowledge compared to the control group. Participants with lower educational attainment exhibited the largest knowledge gains. Self-reported health behaviours, particularly physical activity, increased significantly among the intervention group. CONCLUSIONS An internet-based dementia prevention e-learning program effectively enhanced knowledge and promoted healthy behaviours related to dementia risk reduction. These findings support the use of well-designed e-learning in public health interventions and highlight the potential for online education to contribute to dementia prevention efforts. CLINICALTRIAL ClinicalTrials.gov NCT05383118; May 18, 2022. INTERNATIONAL REGISTERED REPORT RR2-10.2196/64718

Background: Research has shown that engaging in a range of healthy lifestyles or behavioral factors can help reduce the risk of developing dementia. Improved knowledge of modifiable risk factors for dementia may help engage people to reduce their risk, with beneficial impacts on individual and public health. Moreover, many guidelines emphasize the importance of providing education and web-based resources for dementia prevention. Internet-based interventions may be effective, but few have been studied rigorously or widely disseminated. We created DementiaRisk, an award-winning, web- and email-based education platform for the public focused on modifiable risk factors, featuring multimedia e-learning and email “microlearning” content, to help raise awareness and improve knowledge of actions to reduce dementia risk. Objective: This protocol describes a randomized controlled trial to (1) evaluate whether exposure to DementiaRisk changes knowledge of dementia risk factors, intention to engage in risk reduction activities, and health behaviors related to dementia risk reduction and to (2) explore qualitative aspects including participants’ engagement and satisfaction with the intervention and barriers and facilitators to use. Methods: Using a sequential explanatory mixed methods design, this study conducts a quantitative analysis followed by a qualitative inquiry to evaluate outcomes and feasibility. In total, 485 participants will be recruited on the web and randomly assigned to 2 groups: one accessing DementiaRisk and the other receiving alternative e-learning on mild cognitive impairment. Assessments will be delivered on the web at baseline (T1), at 4 weeks (T2), and at 2 months after the intervention (T3). Knowledge will be assessed using items from the Dementia Knowledge Assessment Scale, intentions to engage in risk reduction activities will be assessed using items in line with current evidence, and health behaviors related to dementia risk reduction will be assessed using items from the Godin-Shephard Leisure Time Physical Activity Questionnaire along with additional questions related to a range of health status domains. Outcomes and feasibility will be assessed using the Information Assessment Method for patients and consumers. A linear mixed effects model will be used to examine the relationship between each outcome score by group and time point. Results: This study was approved by the Hamilton Integrated Research Ethics Board on August 24, 2022 (project ID 14886) and received funding in February 2023. Recruitment took place from March 28, 2023, to April 28, 2023, with the final participants completing the intervention by August 18, 2023. Analyses and interpretation of data are ongoing. Conclusions: DementiaRisk is a readily scalable, technology-enhanced solution for dementia prevention education. It has been designed using evidence-based principles of multimedia learning. It has the potential to scale and spread widely using the open internet, so it may be able to reach a wider audience than traditional in-person educational interventions. Trial Registration: ClinicalTrials.gov NCT05383118; https://clinicaltrials.gov/study/NCT05383118 International Registered Report Identifier (IRRID): DERR1-10.2196/64718

Issue: The Rapid Evidence Service, initiated by the National Collaborating Centre for Methods and Tools (NCCMT) during COVID-19, supports public health decision making by conducting rapid reviews on priority topics. Description of issue: Integral to rapid reviews is an expedited timeline but the quantity of available literature for most public health review questions takes significant time to screen manually. NCCMT integrated 4 artificial intelligence (AI) features into the screening process. DAISY Rank applies predictions learned from manual screening patterns to re-order remaining studies, with most relevant appearing first. AI Screening automatically screens remaining studies based on prediction scores. Check for Screening Errors and Re-Rank Report use previous screening patterns to identify studies that were potentially falsely excluded and predict the total number of included studies, respectively. These features were tested by comparing results provided by AI with those produced manually for select test sets. Results: NCCMT used AI to support and expedite screening, assess screening progress, and/or minimize risk of inappropriately excluding studies for 35 rapid reviews on 20 topics. Using DAISY Rank enabled one screener to review over 4000 references in 9 hours, compared to a different review, where the same amount of screening took 28 hours without DAISY Rank. AI Screening correctly excluded up to 80% of irrelevant search results across reviews. Check for Screening Errors identified 37 potential includes manually excluded in one review; these were reviewed and 3 were included. Re-Rank Report allowed NCCMT to re-allocate staff to subsequent steps in the review process when most included studies were identified. Lessons: Integrating AI features into screening led to less time required, better anticipated timelines, more accurate staff allocation and reduced errors. More rigorous study of AI best practices is needed to continue to improve rapid review method efficiencies. Key messages: • Rapid reviews can be an important source of evidence for decision makers if they can be completed quickly but maintain rigor and accuracy. • AI holds promise as a way to improve screening efficiency.