Raul Rodriguez-Esteban

Biological knowledgebases facilitate discovery across the life sciences by structuring experimental findings into human-readable and computable formats. These essential resources are maintained by a small number of professional biocurators worldwide and face combined chronic underfunding and the exponential growth of the literature. In this perspective, we review how artificial intelligence, particularly large language models and agentic systems, can augment literature-curation workflows. Applications include literature recommendation, entity recognition, data extraction, summarization, ontology development, and quality control with emphasis on published use cases at Global Core BioData Resources and ELIXIR Core Data Resources. We identify key challenges, including the scarcity of training data, difficulty in extracting complex relationships, and concerns about error propagation. To address these challenges, we propose a human-in-the-loop framework where generative artificial intelligence approaches accelerate routine tasks while curators provide critical evaluation and domain expertise. We also propose practical recommendations for the community, including the creation of shared benchmark datasets, harmonized evaluation frameworks, and best-practice guidelines for transparent human-in-the-loop AI deployment in biocuration. These synergistic partnerships will be critical to ensure biological rigour, accelerating knowledge integration while maintaining the quality essential for trusted biological resources.

Social anxiety disorder (SAD) affects up to 1 in 8 individuals over their lifetime and is characterized by an intense fear of social situations where there may be exposure to unfamiliar people or possible scrutiny. The analysis of social media data rather than traditional methods (interviews or focus groups) can provide a unique opportunity to understand the lived experience of individuals with SAD, for whom interacting with strangers is challenging. This retrospective observational study reviewed published literature from PubMed and data from Reddit using social media listening (SML). A stepwise analysis in line with US Food and Drug Administration Patient-Focused Drug Development guidelines was performed to develop a conceptual model for SAD. Natural language processing techniques and machine learning approaches were employed to extract symptoms and impacts described by individuals with SAD. Eleven publications from the literature and 535,544 posts from 118,040 Reddit users were included. Clinical and patient experts then refined the conceptual model covering three key symptom domains (physical, negative automatic thoughts, and emotions) and two impact domains (social functioning and occupational/educational functioning). This study provides insights into the lived experience of individuals with SAD and confirms the value of SML when traditional methods are inappropriate.

Objective: The aim of the Social Media Mining for Health Applications (#SMM4H) shared tasks is to take a community-driven approach to address the natural language processing and machine learning challenges inherent to utilizing social media data for health informatics. In this paper, we present the annotated corpora, a technical summary of participants’ systems, and the performance results. Methods: The eighth iteration of the #SMM4H shared tasks was hosted at the AMIA 2023 Annual Symposium and consisted of 5 tasks that represented various social media platforms (Twitter and Reddit), languages (English and Spanish), methods (binary classification, multi-class classification, extraction, and normalization), and topics (COVID-19, therapies, social anxiety disorder, and adverse drug events). Results: In total, 29 teams registered, representing 17 countries. In general, the top-performing systems used deep neural network architectures based on pre-trained transformer models. In particular, the top-performing systems for the classification tasks were based on single models that were pre-trained on social media corpora. Conclusion: To facilitate future work, the datasets—a total of 61 353 posts—will remain available by request, and the CodaLab sites will remain active for a post-evaluation phase.

Background Patients of certain diseases are less likely to approach the healthcare system but remain active in social media. Young Social Anxiety Disorder (SAD) patients, in particular, are a hard-to-reach population due to disease symptomatology, unmet need and age-related barriers, which makes obtaining first-hand access to patient perspectives challenging. Objective To create a curated cohort of patients from social media that report their age in the range of 13 to 25 years old and confirm having a SAD diagnosis or having received therapy for SAD, and to assess the value of the content posted by these users for observational studies of SAD. Methods We collected 535k posts by 118k Reddit users from the r/SocialAnxiety subreddit. We then developed precise regular expressions to extract age, diagnosis and therapy mentions. We manually annotated the full set of expressions extracted and double-annotated 5% of the age mentions and 10% of the diagnosis and therapy mentions. Using similar methodology, we identified mentions of comorbidities and substance use. Results Our validated cohort includes 37,073 posts by 1,102 users that meet the inclusion criteria. The age, diagnosis, and therapy mention detection had a precision of 68%, 31%, and 44%, respectively, with an inter-annotator agreement of 0.96, 0.96, and 0.78. Sixty-one percent of the users in the cohort report having one or more comorbidities on top of their SAD diagnosis (Fleiss’s Kappa=0.79) and 13% report a concerning use of drugs or alcohol (Fleiss’s Kappa=0.87). We compared the characteristics of our social media cohort to the published literature on SAD. Conclusions Patients with SAD post actively on Reddit and their perspectives can be captured and studied directly from these data. Extracting age, therapy, substance abuse and comorbidities (and potentially other patient data) can address realworld data source biases. Thus, social media is a valuable source to create cohorts of hard-to-reach patient populations that may not enter the healthcare system.

This study presents the outcomes of the shared task competition BioCreative VII (Task 3) focusing on the extraction of medication names from a Twitter user’s publicly available tweets (the user’s ‘timeline’). In general, detecting health-related tweets is notoriously challenging for natural language processing tools. The main challenge, aside from the informality of the language used, is that people tweet about any and all topics, and most of their tweets are not related to health. Thus, finding those tweets in a user’s timeline that mention specific health-related concepts such as medications requires addressing extreme imbalance. Task 3 called for detecting tweets in a user’s timeline that mentions a medication name and, for each detected mention, extracting its span. The organizers made available a corpus consisting of 182 049 tweets publicly posted by 212 Twitter users with all medication mentions manually annotated. The corpus exhibits the natural distribution of positive tweets, with only 442 tweets (0.2%) mentioning a medication. This task was an opportunity for participants to evaluate methods that are robust to class imbalance beyond the simple lexical match. A total of 65 teams registered, and 16 teams submitted a system run. This study summarizes the corpus created by the organizers and the approaches taken by the participating teams for this challenge. The corpus is freely available at https://biocreative.bioinformatics.udel.edu/tasks/biocreative-vii/track-3/. The methods and the results of the competing systems are analyzed with a focus on the approaches taken for learning from class-imbalanced data.

Current biological writing is afflicted by the use of ambiguous names, convoluted sentences, vague statements and narrative-fitted storylines. This represents a challenge for biological research in general and in particular for fields such as biological database curation and text mining, which have been tasked to cope with exponentially growing content. Improving the quality of biological writing by encouraging unambiguity and precision would foster expository discipline and machine reasoning. More specifically, the routine inclusion of formal languages in biological writing would improve our ability to describe, compile and model biology.

Background A key step in clinical flow cytometry data analysis is gating, which involves the identification of cell populations. The process of gating produces a set of reportable results, which are typically described by gating definitions. The non-standardized, non-interpreted nature of gating definitions represents a hurdle for data interpretation and data sharing across and within organizations. Interpreting and standardizing gating definitions for subsequent analysis of gating results requires a curation effort from experts. Machine learning approaches have the potential to help in this process by predicting expert annotations associated with gating definitions. Methods We created a gold-standard dataset by manually annotating thousands of gating definitions with cell type and functional marker annotations. We used this dataset to train and test a machine learning pipeline able to predict standard cell types and functional marker genes associated with gating definitions. Results The machine learning pipeline predicted annotations with high accuracy for both cell types and functional marker genes. Accuracy was lower for gating definitions from assays belonging to laboratories from which limited or no prior data was available in the training. Manual error review ensured that resulting predicted annotations could be reused subsequently as additional gold-standard training data. Conclusions Machine learning methods are able to consistently predict annotations associated with gating definitions from flow cytometry assays. However, a hybrid automatic and manual annotation workflow would be recommended to achieve optimal results.

Delays in the propagation of scientific discoveries across scientific communities have been an oft-maligned feature of scientific research for introducing a bias towards knowledge that is produced within a scientist's closest community. The vastness of the scientific literature has been commonly blamed for this phenomenon, despite recent improvements in information retrieval and text mining. Its actual negative impact on scientific progress, however, has never been quantified. This analysis attempts to do so by exploring its effects on biomedical discovery, particularly in the discovery of relations between diseases, genes and chemical compounds. Results indicate that the probability that two scientific facts will enable the discovery of a new fact depends on how far apart these two facts were originally within the scientific landscape. In particular, the probability decreases exponentially with the citation distance. Thus, the direction of scientific progress is distorted based on the location in which each scientific fact is published, representing a path-dependent bias in which originally closely-located discoveries drive the sequence of future discoveries. To counter this bias, scientists should open the scope of their scientific work with modern information retrieval and extraction approaches.

Background: It is estimated that there are more than 7,000 rare diseases (RDs) worldwide, impacting the lives of approximately 400 million people and only 5% have an approved therapy. Facing special challenges, including patient scarceness, incomplete knowledge of the natural history and only few specialized clinical sites, clinical trials (CT) are limited, making the data from trials critical for research and clinical care. Despite the introduction of the U.S. Food and Drug Administration Amendment Act (FDAAA) in 2007 requiring certain CTs to post results on the registry ClinicalTrials.gov within 12 months following completion, compliance has been reportedly poor. Here, we describe general characteristics of RD CTs, identify trends, and evaluate result reporting practices under the FDAAA aiming to draw awareness to the problem of non-compliance. Methods: CTs conducted between 2008 and 2015 were extracted from the public U.S. trial registry ClinicalTrials.gov using the text mining software I2E (Linguamatics). Disease names were matched with rare disease names from the Orphanet Rare Disease Ontology (ORDO, v2.5, Orphanet). Statistical analyses and data visualization were performed using GraphPad Prism 7 and R (v3.5). The Student’s t-test was employed to calculate significance using p-value cut-offs of <0.05 or <0.001. Results: We analyzed 1,056 RD CTs of which 55.7% were phase 2, 7.7% phase 2/3 and 36.7% phase 3 trials. The studies were mostly one- and two-armed experimental CTs with the majority (60.2%) being funded by industry. Cystic fibrosis and sickle cell disease represented the most frequently investigated diseases (25.0% and 16.5%). Industry-led phase 2 RD CTs were significantly (p<0.0001) shorter than their equivalent led by academia/non-profit (22 vs. 33 months). Screening CTs completed before the end of 2015, we found that of the 725 analyzed studies, 55.2% predominantly phase 2 CTs, did not report results. Taking their potential applicability to the FDAAA into account, 25.2% industry-funded and 28.0% academia/non-profit-funded trials failed to disclose results on ClinicalTrial.gov . Conclusion: RD CTs tend to be comparatively small, industry-funded studies focusing on genetic and neurologic conditions. Sponsor-related differences in study design, duration, and enrollment were observed. There are still substantial shortcomings when it comes to result publication.