Wael Abdelkader

Background: Advances in clinical research methods are frequently published in biomedical journals, but identifying these articles remains challenging due to their rapid growth and insufficient indexing in biomedical databases. These challenges hinder the curation of methodologically focused resources like the Library of Guidance for Health Scientists (LIGHTS). Traditional screening approaches, such as Boolean search strategies and manual abstract screening, are inefficient and resource-intensive, limiting the feasibility of regularly updating LIGHTS. Machine learning (ML), particularly active learning models, presents a promising solution to improve the efficiency of article screening. Objectives: This study evaluates the performance of ASReview’s active learning feature in identifying relevant methods guidance articles using pre-labeled data in simulation mode. Methods: Using pre-labeled dataset composed of 1500 methods guidance articles and 20000 clinical studies, categorized as relevant or irrelevant, we trained and compared multiple simulation models in ASReview using various classifiers and feature extraction models. These included combinations of Support Vector Machine (SVM), Naïve Bayes (NB), Neural Network with Sentence BERT (sBERT), Doc2Vec, and TF-IDF. Model performance was evaluated based on screening burden, recall, Work Saved over Sampling (WSS), and precision. All model combinations used maximum query and dynamic double sampling settings. Results: At 95-99.5% recall, SVM with TF-IDF required the fewest screened records (6.87-7.66% burden), while SVM with Doc2Vec achieved the best overall performance at 100% recall with only 11.47% screening burden (WSS@100 = 88.5%) in 42 minutes. Models using sBERT for feature extraction performed comparably through 99.5% recall but exhibited severe performance degradation at 100% recall, requiring screening of over 65% of the corpus. Conclusion: Classical feature extraction methods, TF-IDF and Doc2Vec, paired with SVM outperform deep learning embeddings methods. ASReview in this controlled setting is a feasible tool for screening methodological literature. Future work should include prospective, human-in-the-loop experiments that embed the Doc2Vec-based SVM pipeline in comparison to human screening.

Background: Myalgic encephalomyelitis (ME), also referred to as chronic fatigue syndrome (CFS), is a complex illness that typically presents with disabling fatigue and cognitive and functional impairment. The etiology and management of ME/CFS remain contentious and patients often describe their experiences through social media. Objective: We explored public discourse on Twitter (rebranded as X) to understand the concerns and priorities of individuals living with ME/CFS, with a focus on (1) the COVID-19 pandemic and (2) publication of the 2021 UK National Institute for Health and Care Excellence (NICE) guidelines on the diagnosis and management of ME/CFS. Methods: We used the Twitter application programming interface to collect tweets related to ME/CFS posted between January 1, 2010, and January 30, 2024. Tweets were sorted into 3 chronological periods (pre–COVID-19 pandemic, post–COVID-19 pandemic, and post-UK 2021 NICE Guidelines publication). A Robustly Optimized Bidirectional Embedding Representations from Transformers Pretraining Approach (RoBERTa) language processing model was used to categorize the sentiment of tweets as positive, negative, or neutral. We identified tweets that mentioned COVID-19, the UK NICE guidelines, and key themes identified through latent Dirichlet allocation (ie, fibromyalgia, research, and treatment). We sampled 1000 random tweets from each theme to identify subthemes and representative quotes. Results: We retrieved 906,404 tweets, of which 427,824 (47.2%) were neutral, 369,371 (40.75%) were negative, and 109,209 (12.05%) were positive. Over time, both the proportion of negative and positive tweets increased, and the proportion of neutral tweets decreased (P<.001 for all changes). Tweets mentioning fibromyalgia acknowledged similarities with ME/CFS, stigmatization associated with both disorders, and lack of effective treatments. Treatment-related tweets often described frustration with ME/CFS labeled as mental illness, dismissal of concerns by health care providers, and the need to seek out “good physicians” who viewed ME/CFS as a physical disorder. Tweets on research typically praised studies of biomarkers and biomedical therapies, called for greater investment in biomedical research, and expressed frustration with studies suggesting a biopsychosocial etiology for ME/CFS or supporting management with psychotherapy or graduated activity. Tweets about the UK NICE guidelines expressed frustration with the 2007 version that recommended cognitive behavioral therapy and graded exercise therapy, and a prolonged campaign by advocacy organizations to influence subsequent versions. Tweets showed high acceptance of the 2021 UK NICE guidelines, which were seen to validate ME/CFS as a biomedical disease and recommended against graded exercise therapy. Tweets about COVID-19 often noted overlaps between post–COVID-19 condition and ME/CFS, including claims of a common biological pathway, and advised there was no cure for either condition. Conclusions: Our findings suggest research is needed to inform how best to support patients’ engagement with evidence-based care. Furthermore, while patient involvement with ME/CFS research is critical, unmanaged intellectual conflicts of interest may threaten the trustworthiness of research efforts.

BACKGROUND Myalgic encephalomyelitis (ME), also referred to as chronic fatigue syndrome (CFS), is a complex illness that typically presents with disabling fatigue and cognitive and functional impairment. The etiology and management of ME/CFS remain contentious and patients often describe their experiences through social media. OBJECTIVE We explored public discourse on Twitter (rebranded as X) to understand the concerns and priorities of individuals living with ME/CFS, with a focus on (1) the COVID-19 pandemic and (2) publication of the 2021 UK National Institute for Health and Care Excellence (NICE) guidelines on the diagnosis and management of ME/CFS. METHODS We used the Twitter application programming interface to collect tweets related to ME/CFS posted between January 1, 2010, and January 30, 2024. Tweets were sorted into 3 chronological periods (pre–COVID-19 pandemic, post–COVID-19 pandemic, and post-UK 2021 NICE Guidelines publication). A Robustly Optimized Bidirectional Embedding Representations from Transformers Pretraining Approach (RoBERTa) language processing model was used to categorize the sentiment of tweets as positive, negative, or neutral. We identified tweets that mentioned COVID-19, the UK NICE guidelines, and key themes identified through latent Dirichlet allocation (ie, fibromyalgia, research, and treatment). We sampled 1000 random tweets from each theme to identify subthemes and representative quotes. RESULTS We retrieved 906,404 tweets, of which 427,824 (47.2%) were neutral, 369,371 (40.75%) were negative, and 109,209 (12.05%) were positive. Over time, both the proportion of negative and positive tweets increased, and the proportion of neutral tweets decreased (P<.001 for all changes). Tweets mentioning fibromyalgia acknowledged similarities with ME/CFS, stigmatization associated with both disorders, and lack of effective treatments. Treatment-related tweets often described frustration with ME/CFS labeled as mental illness, dismissal of concerns by health care providers, and the need to seek out “good physicians” who viewed ME/CFS as a physical disorder. Tweets on research typically praised studies of biomarkers and biomedical therapies, called for greater investment in biomedical research, and expressed frustration with studies suggesting a biopsychosocial etiology for ME/CFS or supporting management with psychotherapy or graduated activity. Tweets about the UK NICE guidelines expressed frustration with the 2007 version that recommended cognitive behavioral therapy and graded exercise therapy, and a prolonged campaign by advocacy organizations to influence subsequent versions. Tweets showed high acceptance of the 2021 UK NICE guidelines, which were seen to validate ME/CFS as a biomedical disease and recommended against graded exercise therapy. Tweets about COVID-19 often noted overlaps between post–COVID-19 condition and ME/CFS, including claims of a common biological pathway, and advised there was no cure for either condition. CONCLUSIONS Our findings suggest research is needed to inform how best to support patients’ engagement with evidence-based care. Furthermore, while patient involvement with ME/CFS research is critical, unmanaged intellectual conflicts of interest may threaten the trustworthiness of research efforts.

Background: The COVID-19 pandemic and its associated public health mitigation strategies have dramatically changed patterns of daily life activities worldwide, resulting in unintentional consequences on behavioral risk factors, including smoking, alcohol consumption, poor nutrition, and physical inactivity. The infodemic of social media data may provide novel opportunities for evaluating changes related to behavioral risk factors during the pandemic. Objective: We explored the feasibility of conducting a sentiment and emotion analysis using Twitter data to evaluate behavioral cancer risk factors (physical inactivity, poor nutrition, alcohol consumption, and smoking) over time during the first year of the COVID-19 pandemic. Methods: Tweets during 2020 relating to the COVID-19 pandemic and the 4 cancer risk factors were extracted from the George Washington University Libraries Dataverse. Tweets were defined and filtered using keywords to create 4 data sets. We trained and tested a machine learning classifier using a prelabeled Twitter data set. This was applied to determine the sentiment (positive, negative, or neutral) of each tweet. A natural language processing package was used to identify the emotions (anger, anticipation, disgust, fear, joy, sadness, surprise, and trust) based on the words contained in the tweets. Sentiments and emotions for each of the risk factors were evaluated over time and analyzed to identify keywords that emerged. Results: The sentiment analysis revealed that 56.69% (51,479/90,813) of the tweets about physical activity were positive, 16.4% (14,893/90,813) were negative, and 26.91% (24,441/90,813) were neutral. Similar patterns were observed for nutrition, where 55.44% (27,939/50,396), 15.78% (7950/50,396), and 28.79% (14,507/50,396) of the tweets were positive, negative, and neutral, respectively. For alcohol, the proportions of positive, negative, and neutral tweets were 46.85% (34,897/74,484), 22.9% (17,056/74,484), and 30.25% (22,531/74,484), respectively, and for smoking, they were 41.2% (11,628/28,220), 24.23% (6839/28,220), and 34.56% (9753/28,220), respectively. The sentiments were relatively stable over time. The emotion analysis suggests that the most common emotion expressed across physical activity and nutrition tweets was trust (69,495/320,741, 21.67% and 42,324/176,564, 23.97%, respectively); for alcohol, it was joy (49,147/273,128, 17.99%); and for smoking, it was fear (23,066/110,256, 20.92%). The emotions expressed remained relatively constant over the observed period. An analysis of the most frequent words tweeted revealed further insights into common themes expressed in relation to some of the risk factors and possible sources of bias. Conclusions: This analysis provided insight into behavioral cancer risk factors as expressed on Twitter during the first year of the COVID-19 pandemic. It was feasible to extract tweets relating to all 4 risk factors, and most tweets had a positive sentiment with varied emotions across the different data sets. Although these results can play a role in promoting public health, a deeper dive via qualitative analysis can be conducted to provide a contextual examination of each tweet.

BACKGROUND The COVID-19 pandemic and its associated public health mitigation strategies have dramatically changed patterns of daily life activities worldwide, resulting in unintentional consequences on behavioral risk factors, including smoking, alcohol consumption, poor nutrition, and physical inactivity. The infodemic of social media data may provide novel opportunities for evaluating changes related to behavioral risk factors during the pandemic. OBJECTIVE We explored the feasibility of conducting a sentiment and emotion analysis using Twitter data to evaluate behavioral cancer risk factors (physical inactivity, poor nutrition, alcohol consumption, and smoking) over time during the first year of the COVID-19 pandemic. METHODS Tweets during 2020 relating to the COVID-19 pandemic and the 4 cancer risk factors were extracted from the George Washington University Libraries Dataverse. Tweets were defined and filtered using keywords to create 4 data sets. We trained and tested a machine learning classifier using a prelabeled Twitter data set. This was applied to determine the sentiment (positive, negative, or neutral) of each tweet. A natural language processing package was used to identify the emotions (anger, anticipation, disgust, fear, joy, sadness, surprise, and trust) based on the words contained in the tweets. Sentiments and emotions for each of the risk factors were evaluated over time and analyzed to identify keywords that emerged. RESULTS The sentiment analysis revealed that 56.69% (51,479/90,813) of the tweets about physical activity were positive, 16.4% (14,893/90,813) were negative, and 26.91% (24,441/90,813) were neutral. Similar patterns were observed for nutrition, where 55.44% (27,939/50,396), 15.78% (7950/50,396), and 28.79% (14,507/50,396) of the tweets were positive, negative, and neutral, respectively. For alcohol, the proportions of positive, negative, and neutral tweets were 46.85% (34,897/74,484), 22.9% (17,056/74,484), and 30.25% (22,531/74,484), respectively, and for smoking, they were 41.2% (11,628/28,220), 24.23% (6839/28,220), and 34.56% (9753/28,220), respectively. The sentiments were relatively stable over time. The emotion analysis suggests that the most common emotion expressed across physical activity and nutrition tweets was trust (69,495/320,741, 21.67% and 42,324/176,564, 23.97%, respectively); for alcohol, it was joy (49,147/273,128, 17.99%); and for smoking, it was fear (23,066/110,256, 20.92%). The emotions expressed remained relatively constant over the observed period. An analysis of the most frequent words tweeted revealed further insights into common themes expressed in relation to some of the risk factors and possible sources of bias. CONCLUSIONS This analysis provided insight into behavioral cancer risk factors as expressed on Twitter during the first year of the COVID-19 pandemic. It was feasible to extract tweets relating to all 4 risk factors, and most tweets had a positive sentiment with varied emotions across the different data sets. Although these results can play a role in promoting public health, a deeper dive via qualitative analysis can be conducted to provide a contextual examination of each tweet.

BACKGROUND Given suboptimal performance of Boolean searching to identify methodologically sound and clinically relevant studies in large bibliographic databases such as MEDLINE, exploring the performance of machine learning (ML) tools is warranted. OBJECTIVE Using a large internationally recognized dataset of articles tagged for methodological rigor, we trained and tested binary classification models to predict the probability of clinical research articles being of high methodologic quality to support a literature surveillance program. METHODS Over 12,000 models were trained on a dataset of 97,805 articles indexed in PubMed from 2012-2018 which were manually appraised for rigor by highly trained research associates with expertise in research methods and critical appraisal. As the dataset is unbalanced, with more articles that do not meet criteria for rigor, we used the unbalanced dataset and over- and under-sampled datasets. Models that maintained sensitivity for high rigor at 99% and maximized specificity were selected and tested in a retrospective set of 30,424 articles from 2020 and validated prospectively in a blinded study of 5253 articles. RESULTS The final selected algorithm, combining a model trained in each dataset, maintained high sensitivity and achieved 57% specificity in the retrospective validation test and 53% in the prospective study. The number of articles needed to read to find one that met appraisal criteria was 3.68 (3.52 to 3.85) in the prospective study. CONCLUSIONS ML models improved the efficiency and precision of detecting high quality clinical research publications for literature surveillance and subsequent dissemination to clinicians and other evidence users.

Background: Coronavirus disease 2019 (COVID-19) is a novel infectious disease caused by the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2). Despite the paucity of evidence, various complementary, alternative and integrative medicines (CAIMs) have been being touted as both preventative and curative. We conducted sentiment and emotion analysis with the intent of understanding CAIM content related to COVID-19 being generated on Twitter across 9 months.Methods: Tweets relating to CAIM and COVID-19 were extracted from the George Washington University Libraries Dataverse Coronavirus tweets dataset from March 03 to November 30, 2020. We trained and tested a machine learning classifier using a large, pre-labelled Twitter dataset, which was applied to predict the sentiment of each CAIM-related tweet, and we used a natural language processing package to identify the emotions based on the words contained in the tweets.Results: Our dataset included 28 713 English-language Tweets. The number of CAIM-related tweets during the study period peaked in May 2020, then dropped off sharply over the subsequent three months; the fewest CAIM-related tweets were collected during August 2020 and remained low for the remainder of the collection period. Most tweets (n = 15 612, 54%) were classified as positive, 31% were neutral (n = 8803) and 15% were classified as negative (n = 4298). The most frequent emotions expressed across tweets were trust, followed by fear, while surprise and disgust were the least frequent. Though volume of tweets decreased over the 9 months of the study, the expressed sentiments and emotions remained constant.Conclusion: The results of this sentiment analysis enabled us to establish key CAIMs being discussed at the intersection of COVID-19 across a 9-month period on Twitter. Overall, the majority of our subset of tweets were positive, as were the emotions associated with the words found within them. This may be interpreted as public support for CAIM, however, further qualitative investigation is warranted. Such future directions may be used to combat misinformation and improve public health strategies surrounding the use of social media information.

Background: A barrier to practicing evidence-based medicine is the rapidly increasing body of biomedical literature. Use of method terms to limit the search can help reduce the burden of screening articles for clinical relevance; however, such terms are limited by their partial dependence on indexing terms and usually produce low precision, especially when high sensitivity is required. Machine learning has been applied to the identification of high-quality literature with the potential to achieve high precision without sacrificing sensitivity. The use of artificial intelligence has shown promise to improve the efficiency of identifying sound evidence. Objective: The primary objective of this research is to derive and validate deep learning machine models using iterations of Bidirectional Encoder Representations from Transformers (BERT) to retrieve high-quality, high-relevance evidence for clinical consideration from the biomedical literature. Methods: Using the HuggingFace Transformers library, we will experiment with variations of BERT models, including BERT, BioBERT, BlueBERT, and PubMedBERT, to determine which have the best performance in article identification based on quality criteria. Our experiments will utilize a large data set of over 150,000 PubMed citations from 2012 to 2020 that have been manually labeled based on their methodological rigor for clinical use. We will evaluate and report on the performance of the classifiers in categorizing articles based on their likelihood of meeting quality criteria. We will report fine-tuning hyperparameters for each model, as well as their performance metrics, including recall (sensitivity), specificity, precision, accuracy, F-score, the number of articles that need to be read before finding one that is positive (meets criteria), and classification probability scores. Results: Initial model development is underway, with further development planned for early 2022. Performance testing is expected to star in February 2022. Results will be published in 2022. Conclusions: The experiments will aim to improve the precision of retrieving high-quality articles by applying a machine learning classifier to PubMed searching. International Registered Report Identifier (IRRID): DERR1-10.2196/29398