Julia Langham

Background: Perinatal mental health disorders affect one in five mothers during pregnancy or within 2 years post childbirth. These disorders can lead to poor pregnancy and childbirth outcomes and maternal deaths. Additionally, they negatively affect a child’s cognitive, social and emotional development. Stigma and a lack of specialised services have limited access to mental health care. National Health Service England invested £365M in community perinatal mental health teams, but their impact on women and infants’ outcomes are not known. Objectives: Develop a taxonomy of community perinatal mental health teams (work package 1). Compare and validate two assessments of quality of mother–infant interaction for use by community perinatal mental health teams (work package 2). Evaluate the effectiveness and cost-effectiveness of community perinatal mental health teams (work packages 3 and 4). Design: Mixed-methods study. Setting: Community perinatal mental health teams in England. Participants: Women who were pregnant or within 2 years postnatal. Methods and outcome measures: Work package 1: Typology of community perinatal mental health teams in England. Work package 2: Reliability and validity of two observational assessments of parent–infant interaction. Work package 3: Realist evaluation interviews with women, partners/close others, and staff to determine effective community perinatal mental health team components. Work package 4: Analysis of linked data: Association of community perinatal mental health teams with access to secondary care mental health services. Risk of acute relapse and improved obstetric and neonate outcomes for women with pre-existing severe disorders in areas with community perinatal mental health teams compared to generic services. Economic analysis of cost of community perinatal mental health teams. Results: Objective 1: Community perinatal mental health team typologies revealed in 2020, 84% had basic staffing levels and 63% had more multi-professionals. Objective 2: The ‘Parent Infant Interaction Observation Scale’ and ‘National Institute of Child Health and Human Development’ assessments of mother–infant interaction were reliable and valid; the National Institute of Child Health and Human Development is more suitable for community perinatal mental health teams. Objective 3: Work package 3: Interviews with 139 women, 55 partners/close others and 80 health workers highlighted the importance of specialist perinatal knowledge, responding in a warm and non-judgemental way, working closely with other healthcare providers, optimising medication, supporting mothers to reduce conflict and improve social support, helping mother–infant bonding, and teaching emotional management. Work package 4: Analysis of linked health data revealed higher risks for obstetric and neonate problems in women with severe mental health disorders, particularly recent or very serious episodes. Work package 4: Areas with community perinatal mental health teams saw increased mental health access among perinatal women and reduced need for acute care, albeit at a higher cost and with greater neonatal risks. Limitations: High levels of missing data on diagnosis and mental health outcomes in existing health and service data. Lack of data on child outcomes. Evaluation occurred during community perinatal mental health team changes and the coronavirus disease discovered in 2019 pandemic limiting a full assessment of the impact of community perinatal mental health teams on maternal and child outcomes. Conclusions: Community perinatal mental health teams can support perinatal women with complex, moderate/severe mental health disorders, but further attention to women’s physical needs is essential. The use of observational assessments of parent–infant relationships will enhance the evaluation of community perinatal mental health teams’ impact on infant outcomes. Future work: Research should focus on prospective studies that gather mental health and child outcomes from community perinatal mental health teams and primary care mental health, to assess broader impacts of perinatal-specific treatment across care pathways. Study registration: This study is registered on Research Registry as researchregistry5463. Funding: This award was funded by the National Institute for Health and Care Research (NIHR) Health and Social Care Delivery Research programme (NIHR award ref: 17/49/38) and is published in full in Health and Social Care Delivery Research; Vol. 13, No. 38. See the NIHR Funding and Awards website for further award information.

Background Current generation large language models (LLMs) such as Generative Pre-Trained Transformer 4 (GPT-4) have achieved human-level performance on many tasks including the generation of computer code based on textual input. This study aimed to assess whether GPT-4 could be used to automatically programme two published health economic analyses. Methods The two analyses were partitioned survival models evaluating interventions in non-small cell lung cancer (NSCLC) and renal cell carcinoma (RCC). We developed prompts which instructed GPT-4 to programme the NSCLC and RCC models in R, and which provided descriptions of each model’s methods, assumptions and parameter values. The results of the generated scripts were compared to the published values from the original, human-programmed models. The models were replicated 15 times to capture variability in GPT-4’s output. Results GPT-4 fully replicated the NSCLC model with high accuracy: 100% (15/15) of the artificial intelligence (AI)-generated NSCLC models were error-free or contained a single minor error, and 93% (14/15) were completely error-free. GPT-4 closely replicated the RCC model, although human intervention was required to simplify an element of the model design (one of the model’s fifteen input calculations) because it used too many sequential steps to be implemented in a single prompt. With this simplification, 87% (13/15) of the AI-generated RCC models were error-free or contained a single minor error, and 60% (9/15) were completely error-free. Error-free model scripts replicated the published incremental cost-effectiveness ratios to within 1%. Conclusion This study provides a promising indication that GPT-4 can have practical applications in the automation of health economic model construction. Potential benefits include accelerated model development timelines and reduced costs of development. Further research is necessary to explore the generalisability of LLM-based automation across a larger sample of models.

Background The emergence of artificial intelligence, capable of human-level performance on some tasks, presents an opportunity to revolutionise development of systematic reviews and network meta-analyses (NMAs). In this pilot study, we aim to assess use of a large-language model (LLM, Generative Pre-trained Transformer 4 [GPT-4]) to automatically extract data from publications, write an R script to conduct an NMA and interpret the results. Methods We considered four case studies involving binary and time-to-event outcomes in two disease areas, for which an NMA had previously been conducted manually. For each case study, a Python script was developed that communicated with the LLM via application programming interface (API) calls. The LLM was prompted to extract relevant data from publications, to create an R script to be used to run the NMA and then to produce a small report describing the analysis. Results The LLM had a > 99% success rate of accurately extracting data across 20 runs for each case study and could generate R scripts that could be run end-to-end without human input. It also produced good quality reports describing the disease area, analysis conducted, results obtained and a correct interpretation of the results. Conclusions This study provides a promising indication of the feasibility of using current generation LLMs to automate data extraction, code generation and NMA result interpretation, which could result in significant time savings and reduce human error. This is provided that routine technical checks are performed, as recommend for human-conducted analyses. Whilst not currently 100% consistent, LLMs are likely to improve with time.