Francesco Carli

Language models and agents are increasingly used in biomedicine, but current benchmarks reward correct answers even when the underlying reasoning is flawed. Here we introduce Karenina, an open-source framework that turns expert knowledge into multi-dimensional evaluations of questions, conversations and autonomous agents. Illustrated in Question-Answer pairs, multi-turn conversations and autonomous data-analysis, these dimensions together moves evaluation beyond scoring, enabling trustworthy decision-making with AI in biomedicine.

Large language models (LLMs) face reliability challenges stemming from hallucinations and insufficient access to validated scientific resources. Existing solutions are often fragmented and limited to specific applications, hindering broader adoption and interoperability. Here, we present Biomedical Context for Artificial Intelligence (BioContextAI), an open-source initiative centered on Model Context Protocol (MCP) servers to address these limitations. BioContextAI provides a community-oriented registry for discovering domain-specific MCP servers and a proof-of-concept server implementation that integrates widely-used biomedical knowledgebases. By enabling standardized access to validated scientific knowledge, BioContextAI aims to facilitate the development of composable agentic systems for biomedical research. Together, this work contributes to an emerging ecosystem of community-driven approaches for expanding the capabilities and reliability of biomedical AI systems.

We present the CellHit web server (https://cellhit.bioinfolab.sns.it/), a web-based platform designed to predict and analyze cancer patients’ responsiveness to drugs using transcriptomic data. By leveraging extensive pharmacogenomics datasets from the Genomics of Drug Sensitivity in Cancer v1 and v2 (GDSC) and Profiling Relative Inhibition Simultaneously in Mixtures (PRISM) and transcriptomic data from the Cancer Cell Line Encyclopedia (CCLE) and The Cancer Genome Atlas Program (TCGA). CellHit integrates a computational pipeline for preprocessing, gene imputation, and robust alignment between patient and cell line transcriptomic data with pre-trained SOTA models for drug sensitivity prediction. The pipeline employs batch correction, enhanced Celligner methodology, and Parametric UMAP for stable and actionable alignment. The intuitive interface requires no programming expertise, offering interactive visualizations, including low-dimensional embeddings and drug sensitivity heatmaps for the input transcriptomic samples. Results feature contextual metadata, SHAP-based feature importance, and transcriptomic neighbors from reference datasets, simplifying interpretation and hypothesis generation. CellHit provides precomputed predictions across TCGA samples and offers the ability to run custom analyses online on input samples, democratizing precision oncology by enabling rapid, interpretable predictions accessible the research community.

High-throughput screening of drug sensitivity of cancer cell lines (CCLs) holds the potential to unlock anti-tumor therapies. In this study, we leverage such datasets to predict drug response using cell line transcriptomics, focusing on models’ interpretability and deployment on patients’ data. We use large language models (LLMs) to match drug to mechanisms of action (MOA)-related pathways. Genes crucial for prediction are enriched in drug-MOAs, suggesting that our models learn the molecular determinants of response. Furthermore, by using only LLM-curated, MOA-genes, we enhance the predictive accuracy of our models. To enhance translatability, we align RNAseq data from CCLs, used for training, to those from patient samples, used for inference. We validated our approach on TCGA samples, where patients’ best scoring drugs match those prescribed for their cancer type. We further predict and experimentally validate effective drugs for the patients of two highly lethal solid tumors, i.e., pancreatic cancer and glioblastoma.

High-throughput screening platforms for the profiling of drug sensitivity of hundreds of cancer cell lines (CCLs) have generated large datasets that hold the potential to unlock targeted, anti-tumor therapies. In this study, we leveraged these datasets to create predictive models of cancer cells drug sensitivity. To this aim we trained explainable machine learning algorithms by employing cell line transcriptomics to predict the growth inhibitory potential of drugs. We used large language models (LLMs) to expand descriptions of the mechanisms of action (MOA) for each drug starting from available annotations, which were matched to the semantically closest pathways from reference knowledge bases. By leveraging this AI-curated resource, and the interpretability of our model, we demonstrated that pathways enriched for genes crucial for prediction often matched known drug-MOAs and essential genes, suggesting that our models learned the molecular determinants of drug response. Furthermore, we demonstrated that by incorporating only LLM-curated genes associated with MOAs, we enhanced the predictive accuracy of our drug models. To enhance translatability to a clinical setting, we employed a pipeline to align bulk RNAseq from CCLs, used for training the models, to those from patient samples, used for inference. We proved the effectiveness of our approach on TCGA samples, where patients’ best scoring drugs matched those prescribed for their cancer type. We further showed its usefulness by predicting and experimentally validating effective drugs for the patients of two highly lethal solid tumors, i.e. pancreatic cancer and glioblastoma. In summary, our method facilitates the inference and interpretation of cancer cell line drug sensitivity and holds potential to effectively translate them into new cancer therapeutics. Highlights: -Interpretable drug-response prediction models were trained on large scale pharmacogenomics data sets (i.e. GDSC and PRISM). -Large language models were used to enhance the curation of biological pathways associated to drugs MOA -Unbiased interpretation of the models demonstrated learning of drug MOAs and gene essentiality -Inference of TCGA cohort samples recovered mono- and combination cancer drug prescriptions and indicate potential repurposing candidates. -Drug candidates predicted from bulk RNAseq samples of pancreatic cancer and glioblastoma were experimentally validated.

Most human activities require collaborations within and across formal or informal teams. Our understanding of how the collaborative efforts spent by teams relate to their performance is still a matter of debate. Teamwork results in a highly interconnected ecosystem of potentially overlapping components where tasks are performed in interaction with team members and across other teams. To tackle this problem, we propose a graph neural network model to predict a team’s performance while identifying the drivers determining such outcome. In particular, the model is based on three architectural channels: topological, centrality, and contextual, which capture different factors potentially shaping teams’ success. We endow the model with two attention mechanisms to boost model performance and allow interpretability. A first mechanism allows pinpointing key members inside the team. A second mechanism allows us to quantify the contributions of the three driver effects in determining the outcome performance. We test model performance on various domains, outperforming most classical and neural baselines. Moreover, we include synthetic datasets designed to validate how the model disentangles the intended properties on which our model vastly outperforms baselines.