Ba’ Pham

Background: Systematic reviews are the cornerstone of evidence-based medicine. However, systematic reviews are time consuming and there is growing demand to produce evidence more quickly, while maintaining robust methods. In recent years, artificial intelligence and active-machine learning (AML) have been implemented into several SR software applications. As some of the barriers to adoption of new technologies are the challenges in set-up and how best to use these technologies, we have provided different situations and considerations for knowledge synthesis teams to consider when using artificial intelligence and AML for title and abstract screening.Methods: We retrospectively evaluated the implementation and performance of AML across a set of ten historically completed systematic reviews. Based upon the findings from this work and in consideration of the barriers we have encountered and navigated during the past 24 months in using these tools prospectively in our research, we discussed and developed a series of practical recommendations for research teams to consider in seeking to implement AML tools for citation screening into their workflow.Results: We developed a seven-step framework and provide guidance for when and how to integrate artificial intelligence and AML into the title and abstract screening process. Steps include: (1) Consulting with Knowledge user/Expert Panel; (2) Developing the search strategy; (3) Preparing your review team; (4) Preparing your database; (5) Building the initial training set; (6) Ongoing screening; and (7) Truncating screening. During Step 6 and/or 7, you may also choose to optimize your team, by shifting some members to other review stages (e.g., full-text screening, data extraction).Conclusion: Artificial intelligence and, more specifically, AML are well-developed tools for title and abstract screening and can be integrated into the screening process in several ways. Regardless of the method chosen, transparent reporting of these methods is critical for future studies evaluating artificial intelligence and AML.

Background: The comparative safety and efficacy between anti-vascular endothelial growth factor agents (anti-VEGFs) and between combined therapies for patients with neovascular age-related macular degeneration (nAMD) is unclear. We conducted a systematic review to examine the comparative safety and efficacy anti-VEGFs for adults with nAMD. Methods: Studies were identified through MEDLINE, EMBASE, and Cochrane CENTRAL (inception to June 3, 2019), grey literature, and scanning reference lists. Two reviewers independently screened citations and full-text articles to identify randomized controlled trials (RCTs), extracted data, and appraised risk of bias. Pairwise random-effects meta-analysis and Bayesian network meta-analysis (NMA) were conducted. The primary outcomes were the proportion of patients experiencing moderate vision gain (≥ 15 letters on the Early Treatment Diabetic Retinopathy Study chart) and the proportion of patients experiencing moderate vision loss (≤ 15 letters). Results: After screening 3647 citations and 485 potentially relevant full-text articles, 92 RCTs with 24,717 patients were included. NMA (34 RCTs, 8809 patients, 12 treatments) showed small differences among anti-VEGFs in improving the proportion of patients with moderate vision gain, with the largest for conbercept versus broluczumab (OR 0.15, 95% CrI: 0.05–0.56), conbercept versus ranibizumab (OR 0.17, 95% CrI: 0.05–0.59), conbercept versus aflibercept (OR 0.19, 95% CrI: 0.06–0.65), and conbercept versus bevacizumab (OR 0.2, 95% CrI: 0.06–0.69). In NMA (36 RCTs, 9081 patients, 13 treatments) for the proportion of patients with moderate vision loss, small differences were observed among anti-VEGFs, with the largest being for conbercept versus aflibercept (OR 0.24, 95% CrI: 0–4.29), conbercept versus brolucizumab (OR 0.24, 95% CrI: 0–4.71), conbercept versus bevacizumab (OR 0.26, 95% CrI: 0–4.65), and conbercept versus ranibizumab (OR 0.27, 95% CrI: 0–4.67). Conclusion: The only observed differences were that ranibizumab, bevacizumab, aflibercept, and brolucizumab were statistically superior to conbercept in terms of the proportion of patients with nAMD who experienced moderate vision gain. However, this finding is based on indirect evidence through one small trial comparing conbercept with placebo. This does not account for drug-specific differences when assessing anatomic and functional treatment efficacy in variable dosing regimens. Systematic review registration: PROSPERO registration number CRD42015022041.

Background: Current text mining tools supporting abstract screening in systematic reviews are not widely used, in part because they lack sensitivity and precision. We set out to develop an accessible, semi-automated “workflow” to conduct abstract screening for systematic reviews and other knowledge synthesis methods. Methods: We adopt widely recommended text-mining and machine-learning methods to (1) process title-abstracts into numerical training data; and (2) train a classification model to predict eligible abstracts. The predicted abstracts are screened by human reviewers for (“true”) eligibility, and the newly eligible abstracts are used to identify similar abstracts, using near-neighbor methods, which are also screened. These abstracts, as well as their eligibility results, are used to update the classification model, and the above steps are iterated until no new eligible abstracts are identified. The workflow was implemented in R and evaluated using a systematic review of insulin formulations for type-1 diabetes (14,314 abstracts) and a scoping review of knowledge-synthesis methods (17,200 abstracts). Workflow performance was evaluated against the recommended practice of screening abstracts by 2 reviewers, independently. Standard measures were examined: sensitivity (inclusion of all truly eligible abstracts), specificity (exclusion of all truly ineligible abstracts), precision (inclusion of all truly eligible abstracts among all abstracts screened as eligible), F1-score (harmonic average of sensitivity and precision), and accuracy (correctly predicted eligible or ineligible abstracts). Workload reduction was measured as the hours the workflow saved, given only a subset of abstracts needed human screening. Results: With respect to the systematic and scoping reviews respectively, the workflow attained 88%/89% sensitivity, 99%/99% specificity, 71%/72% precision, an F1-score of 79%/79%, 98%/97% accuracy, 63%/55% workload reduction, with 12%/11% fewer abstracts for full-text retrieval and screening, and 0%/1.5% missed studies in the completed reviews. Conclusion: The workflow was a sensitive, precise, and efficient alternative to the recommended practice of screening abstracts with 2 reviewers. All eligible studies were identified in the first case, while 6 studies (1.5%) were missed in the second that would likely not impact the review’s conclusions. We have described the workflow in language accessible to reviewers with limited exposure to natural language processing and machine learning, and have made the code available to reviewers.

Background Increasing availability of competing biosimilar alternatives makes it challenging to make treatment decisions. The purpose of this review is to evaluate the comparative efficacy and safety of ultra-long-/long-/intermediate-acting insulin products and biosimilar insulin compared to human/animal insulin in adults with type 1 diabetes mellitus (T1DM). Methods MEDLINE, EMBASE, CENTRAL, and grey literature were searched from inception to March 27, 2019. Randomized controlled trials (RCTs), quasi-experimental studies, and cohort studies of adults with T1DM receiving ultra-long-/long-/intermediate-acting insulin, compared to each other, as well as biosimilar insulin compared to human/animal insulin were eligible for inclusion. Two reviewers independently screened studies, abstracted data, and appraised risk-of-bias. Pairwise meta-analyses and network meta-analyses (NMA) were conducted. Summary effect measures were mean differences (MD) and odds ratios (OR). Results We included 65 unique studies examining 14,200 patients with T1DM. Both ultra-long-acting and long-acting insulin were superior to intermediate-acting insulin in reducing A1c, FPG, weight gain, and the incidence of major, serious, or nocturnal hypoglycemia. For fasting blood glucose, long-acting once a day (od) was superior to long-acting twice a day (bid) (MD - 0.44, 95% CI: - 0.81 to - 0.06) and ultra-long-acting od was superior to long-acting bid (MD - 0.73, 95% CI - 1.36 to - 0.11). For weight change, long-acting od was inferior to long-acting bid (MD 0.58, 95% CI: 0.05 to 1.10) and long-acting bid was superior to long-action biosimilar od (MD - 0.90, 95% CI: - 1.67 to - 0.12). Conclusions Our results can be used to tailor insulin treatment according to the desired results of patients and clinicians and inform strategies to establish a competitive clinical market, address systemic barriers, expand the pool of potential suppliers, and favor insulin price reduction. PROSPERO Registration CRD42017077051

Background: The objective of this review was to examine the current guidelines for infection prevention and control (IPAC) of coronavirus disease-19 (COVID-19) or other coronaviruses in adults 60 years or older living in long-term care facilities (LTCF). Methods: EMBASE, MEDLINE, Cochrane library, pre-print servers, clinical trial registries, and relevant grey literature sources were searched until July 31, 2020, using database searching and an automated method called Continuous Active Learning® (CAL®). All search results were processed using CAL® to identify the most likely relevant citations that were then screened by a single human reviewer. Full-text screening, data abstraction, and quality appraisal were completed by a single reviewer and verified by a second. Results: Nine clinical practice guidelines (CPGs) were included. The most common recommendation in the CPGs was establishing surveillance and monitoring systems followed by mandating the use of PPE; physically distancing or cohorting residents; environmental cleaning and disinfection; promoting hand and respiratory hygiene among residents, staff, and visitors; and providing sick leave compensation for staff. Conclusions: Current evidence suggests robust surveillance and monitoring along with support for IPAC initiatives are key to preventing the spread of COVID-19 in LTCF. However, there are significant gaps in the current recommendations especially with regard to the movement of staff between LTCF and their role as possible transmission vectors. Systematic review registration: PROSPERO CRD42020181993

A scoping review to characterize the literature on the use of conversations in social media as a potential source of data for detecting adverse events (AEs) related to health products. Our specific research questions were (1) What social media listening platforms exist to detect adverse events related to health products, and what are their capabilities and characteristics? (2) What is the validity and reliability of data from social media for detecting these adverse events? MEDLINE, EMBASE, Cochrane Library, and relevant websites were searched from inception to May 2016. Any type of document (e.g., manuscripts, reports) that described the use of social media data for detecting health product AEs was included. Two reviewers independently screened citations and full-texts, and one reviewer and one verifier performed data abstraction. Descriptive synthesis was conducted. After screening 3631 citations and 321 full-texts, 70 unique documents with 7 companion reports available from 2001 to 2016 were included. Forty-six documents (66%) described an automated or semi-automated information extraction system to detect health product AEs from social media conversations (in the developmental phase). Seven pre-existing information extraction systems to mine social media data were identified in eight documents. Nineteen documents compared AEs reported in social media data with validated data and found consistent AE discovery in all except two documents. None of the documents reported the validity and reliability of the overall system, but some reported on the performance of individual steps in processing the data. The validity and reliability results were found for the following steps in the data processing pipeline: data de-identification (n = 1), concept identification (n = 3), concept normalization (n = 2), and relation extraction (n = 8). The methods varied widely, and some approaches yielded better results than others. Our results suggest that the use of social media conversations for pharmacovigilance is in its infancy. Although social media data has the potential to supplement data from regulatory agency databases; is able to capture less frequently reported AEs; and can identify AEs earlier than official alerts or regulatory changes, the utility and validity of the data source remains under-studied. Open Science Framework ( https://osf.io/kv9hu/ ).

It is unclear how to engage a wide range of knowledge users in research. We aimed to map the evidence on engaging knowledge users with an emphasis on policy-makers, health system managers, and policy analysts in the knowledge synthesis process through a scoping review. We used the Joanna Briggs Institute guidance for scoping reviews. Nine electronic databases (e.g., MEDLINE), two grey literature sources (e.g., OpenSIGLE), and reference lists of relevant systematic reviews were searched from 1996 to August 2016. We included any type of study describing strategies, barriers and facilitators, or assessing the impact of engaging policy-makers, health system managers, and policy analysts in the knowledge synthesis process. Screening and data abstraction were conducted by two reviewers independently with a third reviewer resolving discrepancies. Frequency and thematic analyses were conducted. After screening 8395 titles and abstracts followed by 394 full-texts, 84 unique documents and 7 companion reports fulfilled our eligibility criteria. All 84 documents were published in the last 10 years, and half were prepared in North America. The most common type of knowledge synthesis with knowledge user engagement was a systematic review (36%). The knowledge synthesis most commonly addressed an issue at the level of national healthcare system (48%) and focused on health services delivery (17%) in high-income countries (86%). Policy-makers were the most common (64%) knowledge users, followed by healthcare professionals (49%) and government agencies as well as patients and caregivers (34%). Knowledge users were engaged in conceptualization and design (49%), literature search and data collection (52%), data synthesis and interpretation (71%), and knowledge dissemination and application (44%). Knowledge users were most commonly engaged as key informants through meetings and workshops as well as surveys, focus groups, and interviews either in-person or by telephone and emails. Knowledge user content expertise/awareness was a common facilitator (18%), while lack of time or opportunity to participate was a common barrier (12%). Knowledge users were most commonly engaged during the data synthesis and interpretation phases of the knowledge synthesis conduct. Researchers should document and evaluate knowledge user engagement in knowledge synthesis. Open Science Framework ( https://osf.io/4dy53/ ).

Background Postoperative delirium is a common preventable complication experienced by older adults undergoing elective surgery. In this systematic review and meta-analysis, we identified prognostic factors associated with the risk of postoperative delirium among older adults undergoing elective surgery. Methods Medline, EMBASE, CINAHL, Cochrane Central Register of Controlled Trials, and AgeLine were searched for articles published between inception and April 21, 2016. A total of 5692 titles and abstracts were screened in duplicate for possible inclusion. Studies using any method for diagnosing delirium were eligible. Two reviewers independently completed all data extraction and quality assessments using the Cochrane Risk-of-Bias Tool for randomized controlled trials (RCTs) and the Newcastle-Ottawa Scale (NOS) for cohort studies. Random effects meta-analysis models were used to derive pooled effect estimates. Results Forty-one studies (9384 patients) reported delirium-related prognostic factors. Among our included studies, the pooled incidence of postoperative delirium was 18.4% (95% confidence interval [CI] 14.3–23.3%, number needed to follow [NNF] = 6). Geriatric syndromes were important predictors of delirium, namely history of delirium (odds ratio [OR] 6.4, 95% CI 2.2–17.9), frailty (OR 4.1, 95% CI 1.4–11.7), cognitive impairment (OR 2.7, 95% CI 1.9–3.8), impairment in activities of daily living (ADLs; OR 2.1, 95% CI 1.6–2.6), and impairment in instrumental activities of daily living (IADLs; OR 1.9, 95% CI 1.3–2.8). Potentially modifiable prognostic factors such as psychotropic medication use (OR 2.3, 95% CI 1.4–3.6) and smoking status (OR 1.8 95% CI 1.3–2.4) were also identified. Caregiver support was associated with lower odds of postoperative delirium (OR 0.69, 95% CI 0.52–0.91). Discussion Though caution must be used in interpreting meta-analyses of non-randomized studies due to the potential influence of unmeasured confounding, we identified potentially modifiable prognostic factors including frailty and psychotropic medication use that should be targeted to optimize care.

Elective surgeries can be associated with significant harm to older adults. The present study aimed to identify the prognostic factors associated with the development of postoperative complications among older adults undergoing elective surgery. Medline, EMBASE, CINAHL, Cochrane Central Register of Controlled Trials, and AgeLine were searched for articles published between inception and April 21, 2016. Prospective studies reporting prognostic factors associated with postoperative complications (composite outcome of medical and surgical complications), functional decline, mortality, post-hospitalization discharge destination, and prolonged hospitalization among older adults undergoing elective surgery were included. Study characteristics and prognostic factors associated with the outcomes of interest were extracted independently by two reviewers. Random effects meta-analysis models were used to derive pooled effect estimates for prognostic factors and incidences of adverse outcomes. Of the 5692 titles and abstracts that were screened for inclusion, 44 studies (12,281 patients) reported on the following adverse postoperative outcomes: postoperative complications (n =28), postoperative mortality (n = 11), length of hospitalization (n = 21), functional decline (n = 6), and destination at discharge from hospital (n = 13). The pooled incidence of postoperative complications was 25.17% (95% confidence interval (CI) 18.03–33.98%, number needed to follow = 4). The geriatric syndromes of frailty (odds ratio (OR) 2.16, 95% CI 1.29–3.62) and cognitive impairment (OR 2.01, 95% CI 1.44–2.81) were associated with developing postoperative complications; however, there was no association with traditionally assessed prognostic factors such as age (OR 1.07, 95% CI 1.00–1.14) or American Society of Anesthesiologists status (OR 2.62, 95% CI 0.78–8.79). Besides frailty, other potentially modifiable prognostic factors, including depressive symptoms (OR 1.77, 95% CI 1.22–2.56) and smoking (OR 2.43, 95% CI 1.32–4.46), were also associated with developing postoperative complications. Geriatric syndromes are important prognostic factors for postoperative complications. We identified potentially modifiable prognostic factors (e.g., frailty, depressive symptoms, and smoking) associated with developing postoperative complications that can be targeted preoperatively to optimize care.