EMEva-Lisa Meldau

Eva-Lisa Meldau

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article


Eva-Lisa Meldau has a total of 19 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Open Access

    AI as a signal assessor – Can a Large Language Model perform causality assessment on a case series?

    by Aditi Shenoy, Alem Zekarias, Anders Viklund, Joseph Mitchell, Jim Barrett, Lovisa Sandberg, Eva-Lisa Meldau, Henric Taavola-Gustafsson

    Background: Large Language Models (LLMs) are increasingly explored for pharmacovigilance tasks, including information extraction, case documentation, and single-case causality assessment. However, their ability to support causality assessment at the case series level — a complex, time-intensive task requiring clinical reasoning across multiple reports — remains unexplored. Objective: To investigate how a large-scale general-purpose LLM can support pharmacovigilance professionals in assessing causality in a case series, and to explore how prompt design influences the quality of the model’s reasoning. Methods: GPT-4o was used to assess causality for five drug – adverse event combinations, using an adaptation of the Bradford Hill viewpoints for case series assessment. The combinations represented varying drugs and vaccines, adverse events, and case series sizes (5–402 reports). One combination served as a negative control. Structured prompts were iteratively developed and refined using one combination, then applied to all combinations. LLM-generated assessments for each viewpoint were qualitatively evaluated by human annotators for accuracy (precision), and the LLM’s coverage of key aspects from the original signal text was assessed for one combination (recall). Results: Across all five combinations, annotators agreed with 79–92% of the LLM’s output sentences. Full disagreement was consistently low (3–7%), with errors typically involving misinterpretation of complex report details rather than outright fabrication. Prompt design substantially influenced output quality; providing Bradford Hill viewpoint descriptions, including case series data, and adding explicit anti-hallucination instructions improved specificity and grounding. For the recall assessment, 15 of 23 key segments from the original signal text were reflected in the LLM output. The overall summary assessments demonstrated balanced reasoning, correctly distinguishing between positive safety signals and the negative control, and provided a coherent synthesis suitable as a starting point for human assessors. Conclusions: LLMs have the potential to generate contextually nuanced and largely accurate preliminary causality assessments of case series aligned with the Bradford Hill viewpoints, with a low but non-zero hallucination rate. These findings support LLMs as a tool to augment, not replace, expert judgment in signal assessment. Future work should address larger and more diverse signal sets, improved evaluation frameworks for generative output, and the integration of pre-computed summary statistics to reduce errors. Key points: A large language model (GPT-4o) can support pharmacovigilance professionals by generating preliminary causality assessments of a case series that align with human expert judgment. Generated in minutes, such assessments could give assessors a structured starting point for in-depth review, condensing case-level information and surfacing early evidence for or against a signal. Output quality depended heavily on prompt design, and because the hallucination rate was low but not zero, these tools should augment, not replace, expert judgment in signal assessment.

  • Open Access

    Critical Appraisal of Artificial Intelligence for Rare-Event Recognition: Principles and Pharmacovigilance Case Studies

    by G. Niklas Norén, Eva-Lisa Meldau, Johan Ellenius

    Many high-stakes artificial intelligence (AI) applications target low-prevalence events, where apparent accuracy can conceal limited real-world value. Relevant AI models range from expert-defined rules and traditional machine learning to generative large language models (LLMs) constrained for classification. As the effort and expertise required to develop modern AI decrease, there is a risk that organizations devote too little time to understanding their limitations and sources of error. We outline key dimensions for critical appraisal of AI in rare-event recognition, including problem framing and test set design, prevalence-aware statistical evaluation, robustness assessment, and integration into human workflows. In addition, we propose an approach to structured case-level examination (SCLE), to complement statistical performance evaluation, and a set of considerations to guide procurement or development of AI models for rare-event recognition. We instantiate the framework in pharmacovigilance, drawing on three studies: rule-based retrieval of pregnancy-related reports, duplicate detection combining machine learning with probabilistic record linkage, and automated redaction of person names using an LLM. We highlight pitfalls specific to the rare-event setting including optimism from unrealistic class balance and lack of difficult positive controls in test sets-and show how cost-sensitive targets align model performance with operational value. While grounded in pharmacovigilance practice, the principles generalize to domains where positives are scarce, and error costs may be asymmetric.


  • Open Access

    Critical appraisal of artificial intelligence for rare-event recognition: principles and pharmacovigilance case studies

    by G. Niklas Noren, Eva-Lisa Meldau, Johan Ellenius

    Many high-stakes AI applications target low-prevalence events, where apparent accuracy can conceal limited real-world value. Relevant AI models range from expert-defined rules and traditional machine learning to generative LLMs constrained for classification. We outline key considerations for critical appraisal of AI in rare-event recognition, including problem framing and test set design, prevalence-aware statistical evaluation, robustness assessment, and integration into human workflows. In addition, we propose an approach to structured case-level examination (SCLE), to complement statistical performance evaluation, and a comprehensive checklist to guide procurement or development of AI models for rare-event recognition. We instantiate the framework in pharmacovigilance, drawing on three studies: rule-based retrieval of pregnancy-related reports; duplicate detection combining machine learning with probabilistic record linkage; and automated redaction of person names using an LLM. We highlight pitfalls specific to the rare-event setting including optimism from unrealistic class balance and lack of difficult positive controls in test sets - and show how cost-sensitive targets align model performance with operational value. While grounded in pharmacovigilance practice, the principles generalize to domains where positives are scarce and error costs may be asymmetric.


  • Open Access

    Narrative Search Engine for Case Series Assessment Supported by Artificial Intelligence Query Suggestions

    by Alem Zekarias, Eva-Lisa Meldau, Shachi Bista, Joana Félix China, Lovisa Sandberg

    Introduction Manual identification of case narratives with specific relevant information can be challenging when working with large numbers of adverse event reports (case series). The process can be supported with a search engine, but building search queries often remains a manual task. Suggesting terms to add to the search query could support assessors in the identification of case narratives within a case series. Objective The aim of this study is to explore the feasibility of identifying case narratives containing specific characteristics with a narrative search engine supported by artificial intelligence (AI) query suggestions. Methods The narrative search engine uses Best Match 25 (BM25) and suggests additional query terms from two word embedding models providing English and biomedical words to a human in the loop. We calculated the percentage of relevant narratives retrieved by the system (recall) and the percentage of retrieved narratives relevant to the search (precision) on an evaluation dataset including narratives from VigiBase, the World Health Organization global database of adverse event reports for medicines and vaccines. Exact-match search and BM25 search with the Relevance Model (RM3), an alternative way to expand queries, were used as comparators. Results The gold standard included 55/750 narratives labelled as relevant. Our narrative search engine retrieved on average 56.4% of the relevant narratives (recall), which is higher when compared with exact-match search (21.8%), without a significant drop in precision (54.5% to 43.1%). The recall is also higher as compared with RM3 (34.4%). Conclusions Our study demonstrates that a narrative search engine supported by AI query suggestions can be a viable alternative to an exact-match search and BM25 search with RM3, since it can facilitate the retrieval of additional relevant narratives during signal assessments.


  • Open Access

    Automated redaction of names in adverse event reports using transformer-based neural networks

    by Eva-Lisa Meldau, Shachi Bista, Carlos Melgarejo-González, G. Niklas Norén

    Background: Automated recognition and redaction of personal identifiers in free text can enable organisations to share data while protecting privacy. This is important in the context of pharmacovigilance since relevant detailed information on the clinical course of events, differential diagnosis, and patient-reported reflections may often only be conveyed in narrative form. The aim of this study is to develop and evaluate a method for automated redaction of person names in English narrative text on adverse event reports. The target domain for this study was case narratives from the United Kingdom’s Yellow Card scheme, which collects and monitors information on suspected side effects to medicines and vaccines. Methods: We finetuned BERT – a transformer-based neural network – for recognising names in case narratives. Training data consisted of newly annotated records from the Yellow Card data and of the i2b2 2014 deidentification challenge. Because the Yellow Card data contained few names, we used predictive models to select narratives for training. Performance was evaluated on a separate set of annotated narratives from the Yellow Card scheme. In-depth review determined whether (parts of) person names missed by the de-identification method could enable re-identification of the individual, and whether de-identification reduced the clinical utility of narratives by collaterally masking relevant information. Results: Recall on held-out Yellow Card data was 87% (155/179) at a precision of 55% (155/282) and a false-positive rate of 0.05% (127/ 263,451). Considering tokens longer than three characters separately, recall was 94% (102/108) and precision 58% (102/175). For 13 of the 5,042 narratives in Yellow Card test data (71 with person names), the method failed to flag at least one name token. According to in-depth review, the leaked information could enable direct identification for one narrative and indirect identification for two narratives. Clinically relevant information was removed in less than 1% of the 5,042 processed narratives; 97% of the narratives were completely untouched. Conclusions: Automated redaction of names in free-text narratives of adverse event reports can achieve sufficient recall including shorter tokens like patient initials. In-depth review shows that the rare leaks that occur tend not to compromise patient confidentiality. Precision and false positive rates are acceptable with almost all clinically relevant information retained.


  • Open Access

    Automated Drug Coding Using Artificial Intelligence: An Evaluation of WHODrug Koda on Adverse Event Reports

    by Eva-Lisa Meldau, Shachi Bista, Emma Rofors, Lucie M. Gattepaille

    Introduction Coding medicinal products described on adverse event (AE) reports to specific entries in standardised drug dictionaries, such as WHODrug Global, is a time-consuming step in case processing activities despite its potential for automation. Many organisations are already partially automating drug coding using text-processing methods and synonym lists, however addressing challenges such as misspellings, abbreviations or ambiguous trade names requires more advanced methods. WHODrug Koda is a drug coding engine using text-processing algorithms, built-in coding rules and machine learning to code drug verbatims to WHODrug Global. Objective Our aim was to evaluate the drug coding performance of WHODrug Koda on AE reports from VigiBase, the World Health Organization’s global database of individual case safety reports, in terms of level of automation and coding quality. Methods Koda was evaluated on 4.8 million drug entries from VigiBase. Automation level was computed as the proportion of drug entries automatically coded by Koda and was compared to a simple case-insensitive text-matching algorithm. Coding quality was evaluated in terms of coding accuracy, by comparing Koda’s prediction to the WHODrug entries found on the AE reports in VigiBase. To better understand the cases in which Koda’s coding results did not match with the WHODrug entries in VigiBase, a manual assessment of 600 samples of disagreeing encodings was performed by two teams of expert drug coders. Results Compared with a simple direct-match baseline, Koda can increase the automation level from 61% to 89%, while providing high coding quality with an accuracy of 97%. Conclusions Even though Koda was designed for use in clinical trials, Koda achieves automation level and coding quality for drug coding of AE reports comparable with the performance observed in a previous evaluation of Koda on clinical trial data. Koda can thus help organisations to automate their drug coding of AE reports to a large degree.


  • Open Access

    Post-Marketing Safety Profile of Vortioxetine Using a Cluster Analysis and a Disproportionality Analysis of Global Adverse Event Reports

    by Corine Ekhart, Florence van Hunsel, Eugène van Puijenbroek, Rebecca Chandler, Eva-Lisa Meldau, Henric Taavola, G. Niklas Norén

    Introduction Vortioxetine, a multimodal serotonergic drug, is widely used as treatment for major depressive disorder. Although on the market since late 2013, the data of the relative safety of vortioxetine, especially compared to selective serotonin reuptake inhibitors, are still scarce. Objective The aim of this study was to explore the adverse event reporting pattern of vortioxetine through a cluster analysis. Furthermore, to compare the adverse event reporting pattern for vortioxetine with that of the selective serotonin reuptake inhibitors. Methods Individual case safety reports for vortioxetine in VigiBase up to 1 November, 2019 were subjected to consensus clustering, to identify and describe natural groupings of reports based on their reported adverse events. A vigiPoint exploratory analysis compared vortioxetine to the selective serotonin reuptake inhibitors in terms of relative frequencies for a wide range of covariates, including patient sex and age, reported drugs and adverse events, and reporting country. Important differences were identified using odds ratios with adaptive statistical shrinkage. Results Thirty-six clusters containing at least five reports were identified and analysed. The two largest clusters included 48% of the vortioxetine reports and appeared to represent gastrointestinal adverse events and hypersensitivity adverse events. Other distinct clusters were related to, respectively, fatigue, aggression/suicidality, convulsion, medication errors, arthralgia/myalgia, increased weight, paraesthesia and anticholinergic effects. Some of these clusters are not labelled for vortioxetine, such as arthralgia/myalgia and paraesthesia, but are known adverse events for selective serotonin reuptake inhibitors. A vigiPoint analysis revealed a higher proportion of reports from consumers and non-health professionals for vortioxetine as well as higher relative reporting rates of gastrointestinal symptoms, pruritus and mood-related symptoms, consistent with the cluster analysis. Conclusions A pattern of co-reported adverse events that is consistent with labelled adverse events for vortioxetine and the safety profile for selective serotonin reuptake inhibitors in general was revealed. Clusters of unlabelled adverse events were identified that reflect clinical entities that might represent signals of previously unknown adverse events. More extensive analyses of spontaneous reports may help to further understand the reporting pattern of adverse events.