Christopher Marshall
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Christopher Marshall has a total of 37 co-authors. Here are the top co-authors by number of shared publications:
- Dawn Craig
Population Health Sciences Institute, Newcastle University, Newcastle upon Tyne, UK
4 shared publications - Georgina C. Wilkins
Applied Immunobiology and Transplantation Group, Institute of Cellular Medicine, Medical School, University of Newcastle Upon Tyne, Newcastle upon Tyne NE2 4HH, UK
3 shared publications - Hannah O’ Keefe
NIHR Innovation Observatory, Newcastle University, Newcastle-upon-Tyne, United Kingdom
3 shared publications - Kate Lanyi
Population Health Sciences Institute, Faculty of Medical Sciences, Newcastle University, Newcastle upon Tyne, UK
3 shared publications - Rhiannon Green
Maverex Ltd, Newcastle-upon-Tyne
3 shared publications - Fiona Pearson
Newcastle University
2 shared publications
Latest Publications
- Open Access
- 28 March 2026
- Published by Springer Nature in BMC Medical Research Methodology
- Vol. 26 (1)
Concerns of AI use in evidence synthesis based practices: collective views from the community
Background: The use of artificial intelligence (AI) in research has become one of the most hotly debated topics. This is particularly true for the field of evidence synthesis where automation through AI may lead to substantial time and resource savings. Many researchers see the potential benefits of using AI technologies, yet there is hesitation around embedding AI in practice. We explored the concerns of those working in the field of evidence synthesis through a series of online and in-person events. Methods: Data collection was conducted across two in-person and 2 online events: the Evidence Synthesis Hackathon (ESH) 2024, the Community, Opportunities, Research and Experience Information Retrieval (CORE) Forum, a Systematic Review Conversations (SRC) online seminar, and an online Horizon Scanning (HS) Survey. Inductive and deductive coding was utilised to synthesis data into broad themes and subthemes, independently for each event. A vote counting and ranking approach was used to triangulate data across events to capture convergent and divergent themes between participant groups. Results: Across the four events we acquired a total of 248 data points (from 80 respondents) and responses were broadly similar across cohorts. Through synthesis and triangulation, we identified 10 overarching themes. The most prominent themes were knowledge and skills, and data management, respectively. Skills loss, skills gap and job loss were highlighted within the knowledge and skills theme. Bias, confidentiality and reliability were prominent for data management. Lower ranking concerns included environment, economics, AI market and costs. Conclusions: These are valid apprehensions faced by researchers across the field of evidence synthesis and should be considered in the broader discussion of AI. Development of rigorous methodologies and guidance may help to overcome these issues by facilitating responsible and transparent use of AI.
- 28 March 2026
- Published by Springer Nature in BMC Medical Research Methodology
- Vol. 26 (1)
Open AccessConcerns of AI use in evidence synthesis based practices: collective views from the community
Background: The use of artificial intelligence (AI) in research has become one of the most hotly debated topics. This is particularly true for the field of evidence synthesis where automation through AI may lead to substantial time and resource savings. Many researchers see the potential benefits of using AI technologies, yet there is hesitation around embedding AI in practice. We explored the concerns of those working in the field of evidence synthesis through a series of online and in-person events. Methods: Data collection was conducted across two in-person and 2 online events: the Evidence Synthesis Hackathon (ESH) 2024, the Community, Opportunities, Research and Experience Information Retrieval (CORE) Forum, a Systematic Review Conversations (SRC) online seminar, and an online Horizon Scanning (HS) Survey. Inductive and deductive coding was utilised to synthesis data into broad themes and subthemes, independently for each event. A vote counting and ranking approach was used to triangulate data across events to capture convergent and divergent themes between participant groups. Results: Across the four events we acquired a total of 248 data points (from 80 respondents) and responses were broadly similar across cohorts. Through synthesis and triangulation, we identified 10 overarching themes. The most prominent themes were knowledge and skills, and data management, respectively. Skills loss, skills gap and job loss were highlighted within the knowledge and skills theme. Bias, confidentiality and reliability were prominent for data management. Lower ranking concerns included environment, economics, AI market and costs. Conclusions: These are valid apprehensions faced by researchers across the field of evidence synthesis and should be considered in the broader discussion of AI. Development of rigorous methodologies and guidance may help to overcome these issues by facilitating responsible and transparent use of AI.
- Open Access
- 4 March 2026
- Published by JMIR Publications Inc. in JMIR Preprints
Assessing the Validity and Utility of LLM-Supported Qualitative Analysis of Statutory Policy Documents: A Comparative Study Using Integrated Care Board Joint Forward Plans (Preprint)
BACKGROUND Large language models (LLMs) are increasingly being used to accelerate qualitative research tasks such as document review and data extraction. Yet there is limited empirical evidence on how accurately these systems perform when applied to complex statutory health policy documents, which are often long, densely written, and designed for governance and assurance rather than analytic clarity. In England’s National Health Service (NHS), the Health and Care Act 2022 established Integrated Care Systems and introduced Integrated Care Board (ICB) Joint Forward Plans (JFPs). Rapid analysis of healthcare priorities and systematic mapping of unmet needs across ICBs can support the identification of regional variation and inform research, policy development, and innovation. OBJECTIVE To assess whether LLMs can support framework-based qualitative analysis of ICB JFPs by comparing LLM-assisted deductive data extraction with manual researcher-led extraction, focusing on accuracy and traceability to source text. METHODS We conducted a comparative evaluation of deductive qualitative data extraction undertaken by researchers and by three LLMs: ChatGPT (OpenAI), Grok (xAI), and Claude (Anthropic). A predefined analytical framework comprising 9 domains and 41 analytical questions was developed to guide both manual and automated analysis. Five JFPs were sampled from ICBs serving areas of high socioeconomic deprivation in England. Two researchers independently conducted manual extractions using structured spreadsheets, followed by cross-checking and consensus resolution. The same framework was operationalized as structured prompts using a Role–Action–Context–Execution approach and applied consistently across subscription-tier versions of each model. Outputs were compared with manual extraction across forty-one analytical fields per document. Overall accuracy was defined as the proportion of agreement, partial agreement, and disagreement in favor of the LLM, including cases where only the LLM identified relevant evidence. RESULTS All three LLMs completed data extraction in 5 to 7 minutes per document, compared with approximately 6 hours per document for manual extraction. No hallucinated content was identified when LLM-only evidence was manually checked. Model performance varied by LLM and domain. Grok achieved the highest overall accuracy, matching or outperforming manual extraction in 83.4% of fields, particularly in domains with explicit operational content (e.g., cross-cutting system capabilities, use of data and evidence, and cross-system comparison). ChatGPT achieved moderate overall accuracy (54.5%) and performed best where priorities, specificity, and key performance indicators were clearly signposted. Claude showed lower overall accuracy (37.1%) but performed relatively better in more narrative domains, including cross-system comparison and public and community engagement. CONCLUSIONS LLMs can substantially reduce the time required for framework-based data extraction from statutory health policy documents and can capture clearly stated, structured content. However, performance varies meaningfully across models and analytic domains, supporting a transparent human-in-the-loop approach in which LLMs assist with extraction while researchers retain responsibility for verification, interpretation, and synthesis.
- 4 March 2026
- Published by JMIR Publications Inc. in JMIR Preprints
Open AccessAssessing the Validity and Utility of LLM-Supported Qualitative Analysis of Statutory Policy Documents: A Comparative Study Using Integrated Care Board Joint Forward Plans (Preprint)
BACKGROUND Large language models (LLMs) are increasingly being used to accelerate qualitative research tasks such as document review and data extraction. Yet there is limited empirical evidence on how accurately these systems perform when applied to complex statutory health policy documents, which are often long, densely written, and designed for governance and assurance rather than analytic clarity. In England’s National Health Service (NHS), the Health and Care Act 2022 established Integrated Care Systems and introduced Integrated Care Board (ICB) Joint Forward Plans (JFPs). Rapid analysis of healthcare priorities and systematic mapping of unmet needs across ICBs can support the identification of regional variation and inform research, policy development, and innovation. OBJECTIVE To assess whether LLMs can support framework-based qualitative analysis of ICB JFPs by comparing LLM-assisted deductive data extraction with manual researcher-led extraction, focusing on accuracy and traceability to source text. METHODS We conducted a comparative evaluation of deductive qualitative data extraction undertaken by researchers and by three LLMs: ChatGPT (OpenAI), Grok (xAI), and Claude (Anthropic). A predefined analytical framework comprising 9 domains and 41 analytical questions was developed to guide both manual and automated analysis. Five JFPs were sampled from ICBs serving areas of high socioeconomic deprivation in England. Two researchers independently conducted manual extractions using structured spreadsheets, followed by cross-checking and consensus resolution. The same framework was operationalized as structured prompts using a Role–Action–Context–Execution approach and applied consistently across subscription-tier versions of each model. Outputs were compared with manual extraction across forty-one analytical fields per document. Overall accuracy was defined as the proportion of agreement, partial agreement, and disagreement in favor of the LLM, including cases where only the LLM identified relevant evidence. RESULTS All three LLMs completed data extraction in 5 to 7 minutes per document, compared with approximately 6 hours per document for manual extraction. No hallucinated content was identified when LLM-only evidence was manually checked. Model performance varied by LLM and domain. Grok achieved the highest overall accuracy, matching or outperforming manual extraction in 83.4% of fields, particularly in domains with explicit operational content (e.g., cross-cutting system capabilities, use of data and evidence, and cross-system comparison). ChatGPT achieved moderate overall accuracy (54.5%) and performed best where priorities, specificity, and key performance indicators were clearly signposted. Claude showed lower overall accuracy (37.1%) but performed relatively better in more narrative domains, including cross-system comparison and public and community engagement. CONCLUSIONS LLMs can substantially reduce the time required for framework-based data extraction from statutory health policy documents and can capture clearly stated, structured content. However, performance varies meaningfully across models and analytic domains, supporting a transparent human-in-the-loop approach in which LLMs assist with extraction while researchers retain responsibility for verification, interpretation, and synthesis.
- Open Access
- 5 September 2023
- Published by Wiley in Research Synthesis Methods
- Vol. 14 (6) , 874-881
A mapping exercise using automated techniques to develop a search strategy to identify systematic review tools
The Systematic Review Toolbox aims provide a web‐based catalogue of tools that support various tasks within the systematic review and wider evidence synthesis process. Identifying publications surrounding specific systematic review tools is currently challenging, leading to a high screening burden for few eligible records. We aimed to develop a search strategy that could be regularly and automatically run to identify eligible records for the SR Toolbox, thus reducing time on task and burden for those involved. We undertook a mapping exercise to identify the PubMed IDs of papers indexed within the SR Toolbox. We then used the Yale MeSH Analyser and Visualisation of Similarities (VOS) Viewer text‐mining software to identify the most commonly used MeSH terms and text words within the eligible records. These MeSH terms and text words were combined using Boolean Operators into a search strategy for Ovid MEDLINE. Prior to the mapping exercise and search strategy development, 81 software tools and 55 ‘Other’ tools were included within the SR Toolbox. Since implementation of the search strategy, 146 tools have been added. There has been an increase in tools added to the toolbox since the search was developed and its corresponding auto‐alert in MEDLINE was originally set up. Developing a search strategy based on a mapping exercise is an effective way of identifying new tools to support the systematic review process. Further research could be conducted to help prioritise records for screening to reduce reviewer burden further and to adapt the strategy for disciplines beyond healthcare.
- 5 September 2023
- Published by Wiley in Research Synthesis Methods
- Vol. 14 (6) , 874-881
Open AccessA mapping exercise using automated techniques to develop a search strategy to identify systematic review tools
The Systematic Review Toolbox aims provide a web‐based catalogue of tools that support various tasks within the systematic review and wider evidence synthesis process. Identifying publications surrounding specific systematic review tools is currently challenging, leading to a high screening burden for few eligible records. We aimed to develop a search strategy that could be regularly and automatically run to identify eligible records for the SR Toolbox, thus reducing time on task and burden for those involved. We undertook a mapping exercise to identify the PubMed IDs of papers indexed within the SR Toolbox. We then used the Yale MeSH Analyser and Visualisation of Similarities (VOS) Viewer text‐mining software to identify the most commonly used MeSH terms and text words within the eligible records. These MeSH terms and text words were combined using Boolean Operators into a search strategy for Ovid MEDLINE. Prior to the mapping exercise and search strategy development, 81 software tools and 55 ‘Other’ tools were included within the SR Toolbox. Since implementation of the search strategy, 146 tools have been added. There has been an increase in tools added to the toolbox since the search was developed and its corresponding auto‐alert in MEDLINE was originally set up. Developing a search strategy based on a mapping exercise is an effective way of identifying new tools to support the systematic review process. Further research could be conducted to help prioritise records for screening to reduce reviewer burden further and to adapt the strategy for disciplines beyond healthcare.
- Open Access
- 1 December 2022
- Published by National Institute for Health and Care Research in Health Technology Assessment
- Vol. 26 (51) , 1-112
Oral nutritional interventions in frail older people who are malnourished or at risk of malnutrition: a systematic review
Background: Malnutrition worsens the health of frail older adults. Current treatments for malnutrition may include prescribed oral nutritional supplements, which are multinutrient products containing macronutrients and micronutrients. Objective: To assess the effectiveness and cost-effectiveness of oral nutritional supplements (with or without other dietary interventions) in frail older people who are malnourished or at risk of malnutrition. Data sources: MEDLINE, EMBASE, Cochrane Library, Scopus, CINAHL (Cumulative Index to Nursing and Allied Health Literature) and grey literature were searched from inception to 13 September 2021. Review methods: A systematic review and meta-analysis was conducted to evaluate the effectiveness and cost-effectiveness of oral nutritional supplements in frail older people (aged ≥ 65 years) who are malnourished or at risk of malnutrition (defined as undernutrition as per National Institute for Health and Care Excellence guidelines). Meta-analysis and network meta-analysis were undertaken, where feasible, along with a narrative synthesis. A cost-effectiveness review was reported narratively. A de novo model was developed using effectiveness evidence identified in the systematic review to estimate the cost-effectiveness of oral nutritional supplements. Results: Eleven studies (n = 822 participants) were included in the effectiveness review, six of which were fully or partly funded by industry. Meta-analyses suggested positive effects of oral nutritional supplements compared with standard care for energy intake (kcal) (standardised mean difference 1.02, 95% confidence interval 0.15 to 1.88; very low quality evidence) and poor mobility (mean difference 0.03, p < 0.00001, 95% confidence interval 0.02 to 0.04; very low quality evidence) but no evidence of an effect for body weight (mean difference 1.31, 95% confidence interval –0.05 to 2.66; very low quality evidence) and body mass index (mean difference 0.54, 95% confidence interval –0.03 to 1.11; very low quality evidence). Pooled results for other outcomes were statistically non-significant. There was mixed narrative evidence regarding the effect of oral nutritional supplements on quality of life. Network meta-analysis could be conducted only for body weight and grip strength; there was evidence of an effect for oral nutritional supplements compared with standard care for body weight only. Study quality was mixed; the randomisation method was typically poorly reported. One economic evaluation, in a care home setting, was included. This was a well-conducted study showing that oral nutritional supplements could be cost-effective. Cost-effectiveness analysis suggested that oral nutritional supplements may only be cost-effective for people with lower body mass index (< 21 kg/m2) using cheaper oral nutritional supplements products that require minimal staff time to administer. Limitations: The review scope was narrow in focus as few primary studies used frailty measures (or our proxy criteria). This resulted in only 11 included studies. The small evidence base and varied quality of evidence meant that it was not possible to determine accurate estimates of the effectiveness or cost-effectiveness of oral nutritional supplements. Furthermore, only English-language publications were considered. Conclusions: Overall, the review found little evidence of oral nutritional supplements having significant effects on reducing malnutrition or its adverse outcomes in frail older adults. Future work: Future research should focus on independent, high-quality, adequately powered studies to investigate oral nutritional supplements alongside other nutritional interventions, with longer-term follow-up and detailed analysis of determinants, intervention components and cost-effectiveness. Study registration: This study is registered as PROSPERO CRD42020170906. Funding: This project was funded by the National Institute for Health and Care Research (NIHR) Health Technology Assessment programme and will be published in full in Health Technology Assessment; Vol. 26, No. 51. See the NIHR Journals Library website for further project information.
- 1 December 2022
- Published by National Institute for Health and Care Research in Health Technology Assessment
- Vol. 26 (51) , 1-112
Open AccessOral nutritional interventions in frail older people who are malnourished or at risk of malnutrition: a systematic review
Background: Malnutrition worsens the health of frail older adults. Current treatments for malnutrition may include prescribed oral nutritional supplements, which are multinutrient products containing macronutrients and micronutrients. Objective: To assess the effectiveness and cost-effectiveness of oral nutritional supplements (with or without other dietary interventions) in frail older people who are malnourished or at risk of malnutrition. Data sources: MEDLINE, EMBASE, Cochrane Library, Scopus, CINAHL (Cumulative Index to Nursing and Allied Health Literature) and grey literature were searched from inception to 13 September 2021. Review methods: A systematic review and meta-analysis was conducted to evaluate the effectiveness and cost-effectiveness of oral nutritional supplements in frail older people (aged ≥ 65 years) who are malnourished or at risk of malnutrition (defined as undernutrition as per National Institute for Health and Care Excellence guidelines). Meta-analysis and network meta-analysis were undertaken, where feasible, along with a narrative synthesis. A cost-effectiveness review was reported narratively. A de novo model was developed using effectiveness evidence identified in the systematic review to estimate the cost-effectiveness of oral nutritional supplements. Results: Eleven studies (n = 822 participants) were included in the effectiveness review, six of which were fully or partly funded by industry. Meta-analyses suggested positive effects of oral nutritional supplements compared with standard care for energy intake (kcal) (standardised mean difference 1.02, 95% confidence interval 0.15 to 1.88; very low quality evidence) and poor mobility (mean difference 0.03, p < 0.00001, 95% confidence interval 0.02 to 0.04; very low quality evidence) but no evidence of an effect for body weight (mean difference 1.31, 95% confidence interval –0.05 to 2.66; very low quality evidence) and body mass index (mean difference 0.54, 95% confidence interval –0.03 to 1.11; very low quality evidence). Pooled results for other outcomes were statistically non-significant. There was mixed narrative evidence regarding the effect of oral nutritional supplements on quality of life. Network meta-analysis could be conducted only for body weight and grip strength; there was evidence of an effect for oral nutritional supplements compared with standard care for body weight only. Study quality was mixed; the randomisation method was typically poorly reported. One economic evaluation, in a care home setting, was included. This was a well-conducted study showing that oral nutritional supplements could be cost-effective. Cost-effectiveness analysis suggested that oral nutritional supplements may only be cost-effective for people with lower body mass index (< 21 kg/m2) using cheaper oral nutritional supplements products that require minimal staff time to administer. Limitations: The review scope was narrow in focus as few primary studies used frailty measures (or our proxy criteria). This resulted in only 11 included studies. The small evidence base and varied quality of evidence meant that it was not possible to determine accurate estimates of the effectiveness or cost-effectiveness of oral nutritional supplements. Furthermore, only English-language publications were considered. Conclusions: Overall, the review found little evidence of oral nutritional supplements having significant effects on reducing malnutrition or its adverse outcomes in frail older adults. Future work: Future research should focus on independent, high-quality, adequately powered studies to investigate oral nutritional supplements alongside other nutritional interventions, with longer-term follow-up and detailed analysis of determinants, intervention components and cost-effectiveness. Study registration: This study is registered as PROSPERO CRD42020170906. Funding: This project was funded by the National Institute for Health and Care Research (NIHR) Health Technology Assessment programme and will be published in full in Health Technology Assessment; Vol. 26, No. 51. See the NIHR Journals Library website for further project information.
- Open Access
- 31 March 2022
- Published by JMIR Publications Inc. in JMIR Infodemiology
- Vol. 2 (1) , e32449
Using Natural Language Processing to Explore Mental Health Insights From UK Tweets During the COVID-19 Pandemic: Infodemiology Study
Background: There is need to consider the value of soft intelligence, leveraged using accessible natural language processing (NLP) tools, as a source of analyzed evidence to support public health research outputs and decision-making. Objective: The aim of this study was to explore the value of soft intelligence analyzed using NLP. As a case study, we selected and used a commercially available NLP platform to identify, collect, and interrogate a large collection of UK tweets relating to mental health during the COVID-19 pandemic. Methods: A search strategy comprised of a list of terms related to mental health, COVID-19, and lockdown restrictions was developed to prospectively collate relevant tweets via Twitter’s advanced search application programming interface over a 24-week period. We deployed a readily and commercially available NLP platform to explore tweet frequency and sentiment across the United Kingdom and identify key topics of discussion. A series of keyword filters were used to clean the initial data retrieved and also set up to track specific mental health problems. All collated tweets were anonymized. Results: We identified and analyzed 286,902 tweets posted from UK user accounts from July 23, 2020 to January 6, 2021. The average sentiment score was 50%, suggesting overall neutral sentiment across all tweets over the study period. Major fluctuations in volume (between 12,622 and 51,340) and sentiment (between 25% and 49%) appeared to coincide with key changes to any local and/or national social distancing measures. Tweets around mental health were polarizing, discussed with both positive and negative sentiment. Key topics of consistent discussion over the study period included the impact of the pandemic on people’s mental health (both positively and negatively), fear and anxiety over lockdowns, and anger and mistrust toward the government. Conclusions: Using an NLP platform, we were able to rapidly mine and analyze emerging health-related insights from UK tweets into how the pandemic may be impacting people’s mental health and well-being. This type of real-time analyzed evidence could act as a useful intelligence source that agencies, local leaders, and health care decision makers can potentially draw from, particularly during a health crisis.
- 31 March 2022
- Published by JMIR Publications Inc. in JMIR Infodemiology
- Vol. 2 (1) , e32449
Open AccessUsing Natural Language Processing to Explore Mental Health Insights From UK Tweets During the COVID-19 Pandemic: Infodemiology Study
Background: There is need to consider the value of soft intelligence, leveraged using accessible natural language processing (NLP) tools, as a source of analyzed evidence to support public health research outputs and decision-making. Objective: The aim of this study was to explore the value of soft intelligence analyzed using NLP. As a case study, we selected and used a commercially available NLP platform to identify, collect, and interrogate a large collection of UK tweets relating to mental health during the COVID-19 pandemic. Methods: A search strategy comprised of a list of terms related to mental health, COVID-19, and lockdown restrictions was developed to prospectively collate relevant tweets via Twitter’s advanced search application programming interface over a 24-week period. We deployed a readily and commercially available NLP platform to explore tweet frequency and sentiment across the United Kingdom and identify key topics of discussion. A series of keyword filters were used to clean the initial data retrieved and also set up to track specific mental health problems. All collated tweets were anonymized. Results: We identified and analyzed 286,902 tweets posted from UK user accounts from July 23, 2020 to January 6, 2021. The average sentiment score was 50%, suggesting overall neutral sentiment across all tweets over the study period. Major fluctuations in volume (between 12,622 and 51,340) and sentiment (between 25% and 49%) appeared to coincide with key changes to any local and/or national social distancing measures. Tweets around mental health were polarizing, discussed with both positive and negative sentiment. Key topics of consistent discussion over the study period included the impact of the pandemic on people’s mental health (both positively and negatively), fear and anxiety over lockdowns, and anger and mistrust toward the government. Conclusions: Using an NLP platform, we were able to rapidly mine and analyze emerging health-related insights from UK tweets into how the pandemic may be impacting people’s mental health and well-being. This type of real-time analyzed evidence could act as a useful intelligence source that agencies, local leaders, and health care decision makers can potentially draw from, particularly during a health crisis.
- Open Access
- 3 December 2021
- Published by Cambridge University Press (CUP) in International Journal of Technology Assessment in Health Care
- Vol. 37 (S1) , 30
PP234 Analysis Of Discussions On Twitter On The Topic Of COVID-19 Tests: Exploring A Natural Language Processing Approach
Introduction Various strategies to suppress the Coronavirus have been adopted by governments across the world; one such strategy is diagnostic testing. The anxiety of testing on individuals is difficult to quantify. This analysis explores the use of soft intelligence from Twitter (USA, UK & India) in helping better understand this issue. Methods A total of 650,000 tweets were collected between September and October 2020, using Twitter API using hashtags such as ‘#oxymeter’, ‘#oximeter’, ‘#antibodytest’, ‘#infraredthermometer’, ‘#swabtest’, ‘#rapidtest’, and ‘#antigen’. We applied natural language processing (TextBlob) to assign sentiment and categorize the tweets by emotions and attitude. WordCloud was then used to identify the single topmost 500 words in the whole tweet dataset. Results Global analysis and pre-processing of the tweets indicate that 21 percent, seven percent and four percent of tweets originated from the USA, UK, and India respectively. The tweets from #antibody, #rapid, #antigen, and #swabtest were positive sentiments, whereas #oxymeter, #infraredthermometer were mostly neutral. The underlying emotions of the tweets were approximately 2.5 times more positive than negative. The most used words in the tweets included ‘hope’ ‘insurance’, ‘symptoms’, ‘love’, ‘painful’, ‘cough’, ‘fast test’, ‘wife’, and ‘kids’. Conclusions The finding suggests that it may be reasonable to infer that people are generally concerned about their personal and social wellbeing, wanting to keep themselves safe and perceive testing to deliver some component of that feeling of safety. There are several limitations to this study such as it was restricted to only three countries, and includes only English language tweets with a limited number of hashtags.
- 3 December 2021
- Published by Cambridge University Press (CUP) in International Journal of Technology Assessment in Health Care
- Vol. 37 (S1) , 30
Open AccessPP234 Analysis Of Discussions On Twitter On The Topic Of COVID-19 Tests: Exploring A Natural Language Processing Approach
Introduction Various strategies to suppress the Coronavirus have been adopted by governments across the world; one such strategy is diagnostic testing. The anxiety of testing on individuals is difficult to quantify. This analysis explores the use of soft intelligence from Twitter (USA, UK & India) in helping better understand this issue. Methods A total of 650,000 tweets were collected between September and October 2020, using Twitter API using hashtags such as ‘#oxymeter’, ‘#oximeter’, ‘#antibodytest’, ‘#infraredthermometer’, ‘#swabtest’, ‘#rapidtest’, and ‘#antigen’. We applied natural language processing (TextBlob) to assign sentiment and categorize the tweets by emotions and attitude. WordCloud was then used to identify the single topmost 500 words in the whole tweet dataset. Results Global analysis and pre-processing of the tweets indicate that 21 percent, seven percent and four percent of tweets originated from the USA, UK, and India respectively. The tweets from #antibody, #rapid, #antigen, and #swabtest were positive sentiments, whereas #oxymeter, #infraredthermometer were mostly neutral. The underlying emotions of the tweets were approximately 2.5 times more positive than negative. The most used words in the tweets included ‘hope’ ‘insurance’, ‘symptoms’, ‘love’, ‘painful’, ‘cough’, ‘fast test’, ‘wife’, and ‘kids’. Conclusions The finding suggests that it may be reasonable to infer that people are generally concerned about their personal and social wellbeing, wanting to keep themselves safe and perceive testing to deliver some component of that feeling of safety. There are several limitations to this study such as it was restricted to only three countries, and includes only English language tweets with a limited number of hashtags.
- 3 December 2021
- Published by Cambridge University Press (CUP) in International Journal of Technology Assessment in Health Care
- Vol. 37 (S1) , 7-8
OP227 Exploring The Value Of Soft-Intelligence: A Case Study Using Twitter To Track Mental Health During The COVID-19 Pandemic
Introduction There is increasing pressure to rapidly shape policies and inform decision-making where robust evidence is lacking. This work aimed to explore the value of soft-intelligence as a novel source of evidence. We deployed an artificial intelligence based natural language platform to identify and analyze a large collection of UK tweets relating to mental health during the COVID-19 pandemic. Methods A search strategy comprising a list of terms relating to mental health, COVID-19 and the lockdown was developed to prospectively identify relevant tweets via Twitter's advanced search application programming interface. We used a specialist text analytics platform to explore tweet frequency and sentiment across the UK and identify key topics of discussion for qualitative analysis. All collated tweets were anonymized. Results We identified 380,728 tweets from 184,289 unique users in the UK from 30 April to 4 July 2020. The average sentiment score was fifty-two percent, suggesting overall positive sentiment. Tweets around mental health were polarizing, discussed with both positive and negative sentiment. For example, some people described how they were using the lockdown as a positive opportunity to work on their mental health, sharing helpful strategies to support others. However, many people expressed the damaging impact the pandemic (and resulting lockdown) was having on their mental health, including worsening anxiety, stress, depression, and loneliness. Conclusions The results suggest that soft-intelligence is potentially a useful source of evidence. The approach taken to identify and analyze this data may offer an efficient means of establishing key insights from the ‘public voice’ relating to critical health issues. However, there are still various limitations to consider concerning the technology and representativeness of the data. Future work to explore this type of evidence further, and how it might formally support decision-making processes, is recommended. This project is funded by the NIHR [(HSRIC-2016-10009)/Innovation Observatory]. The views expressed are those of the author(s) and not necessarily those of the NIHR or the Department of Health and Social Care.
- 3 December 2021
- Published by Cambridge University Press (CUP) in International Journal of Technology Assessment in Health Care
- Vol. 37 (S1) , 7-8
OP227 Exploring The Value Of Soft-Intelligence: A Case Study Using Twitter To Track Mental Health During The COVID-19 Pandemic
Introduction There is increasing pressure to rapidly shape policies and inform decision-making where robust evidence is lacking. This work aimed to explore the value of soft-intelligence as a novel source of evidence. We deployed an artificial intelligence based natural language platform to identify and analyze a large collection of UK tweets relating to mental health during the COVID-19 pandemic. Methods A search strategy comprising a list of terms relating to mental health, COVID-19 and the lockdown was developed to prospectively identify relevant tweets via Twitter's advanced search application programming interface. We used a specialist text analytics platform to explore tweet frequency and sentiment across the UK and identify key topics of discussion for qualitative analysis. All collated tweets were anonymized. Results We identified 380,728 tweets from 184,289 unique users in the UK from 30 April to 4 July 2020. The average sentiment score was fifty-two percent, suggesting overall positive sentiment. Tweets around mental health were polarizing, discussed with both positive and negative sentiment. For example, some people described how they were using the lockdown as a positive opportunity to work on their mental health, sharing helpful strategies to support others. However, many people expressed the damaging impact the pandemic (and resulting lockdown) was having on their mental health, including worsening anxiety, stress, depression, and loneliness. Conclusions The results suggest that soft-intelligence is potentially a useful source of evidence. The approach taken to identify and analyze this data may offer an efficient means of establishing key insights from the ‘public voice’ relating to critical health issues. However, there are still various limitations to consider concerning the technology and representativeness of the data. Future work to explore this type of evidence further, and how it might formally support decision-making processes, is recommended. This project is funded by the NIHR [(HSRIC-2016-10009)/Innovation Observatory]. The views expressed are those of the author(s) and not necessarily those of the NIHR or the Department of Health and Social Care.
- Open Access
- 28 July 2021
- Published by JMIR Publications Inc. in JMIR Preprints
Using Natural Language Processing to Explore Mental Health Insights From UK Tweets During the COVID-19 Pandemic: Infodemiology Study (Preprint)
BACKGROUND There is need to consider the value of soft intelligence, leveraged using accessible natural language processing (NLP) tools, as a source of analyzed evidence to support public health research outputs and decision-making. OBJECTIVE The aim of this study was to explore the value of soft intelligence analyzed using NLP. As a case study, we selected and used a commercially available NLP platform to identify, collect, and interrogate a large collection of UK tweets relating to mental health during the COVID-19 pandemic. METHODS A search strategy comprised of a list of terms related to mental health, COVID-19, and lockdown restrictions was developed to prospectively collate relevant tweets via Twitter’s advanced search application programming interface over a 24-week period. We deployed a readily and commercially available NLP platform to explore tweet frequency and sentiment across the United Kingdom and identify key topics of discussion. A series of keyword filters were used to clean the initial data retrieved and also set up to track specific mental health problems. All collated tweets were anonymized. RESULTS We identified and analyzed 286,902 tweets posted from UK user accounts from July 23, 2020 to January 6, 2021. The average sentiment score was 50%, suggesting overall neutral sentiment across all tweets over the study period. Major fluctuations in volume (between 12,622 and 51,340) and sentiment (between 25% and 49%) appeared to coincide with key changes to any local and/or national social distancing measures. Tweets around mental health were polarizing, discussed with both positive and negative sentiment. Key topics of consistent discussion over the study period included the impact of the pandemic on people’s mental health (both positively and negatively), fear and anxiety over lockdowns, and anger and mistrust toward the government. CONCLUSIONS Using an NLP platform, we were able to rapidly mine and analyze emerging health-related insights from UK tweets into how the pandemic may be impacting people’s mental health and well-being. This type of real-time analyzed evidence could act as a useful intelligence source that agencies, local leaders, and health care decision makers can potentially draw from, particularly during a health crisis.
- 28 July 2021
- Published by JMIR Publications Inc. in JMIR Preprints
Open AccessUsing Natural Language Processing to Explore Mental Health Insights From UK Tweets During the COVID-19 Pandemic: Infodemiology Study (Preprint)
BACKGROUND There is need to consider the value of soft intelligence, leveraged using accessible natural language processing (NLP) tools, as a source of analyzed evidence to support public health research outputs and decision-making. OBJECTIVE The aim of this study was to explore the value of soft intelligence analyzed using NLP. As a case study, we selected and used a commercially available NLP platform to identify, collect, and interrogate a large collection of UK tweets relating to mental health during the COVID-19 pandemic. METHODS A search strategy comprised of a list of terms related to mental health, COVID-19, and lockdown restrictions was developed to prospectively collate relevant tweets via Twitter’s advanced search application programming interface over a 24-week period. We deployed a readily and commercially available NLP platform to explore tweet frequency and sentiment across the United Kingdom and identify key topics of discussion. A series of keyword filters were used to clean the initial data retrieved and also set up to track specific mental health problems. All collated tweets were anonymized. RESULTS We identified and analyzed 286,902 tweets posted from UK user accounts from July 23, 2020 to January 6, 2021. The average sentiment score was 50%, suggesting overall neutral sentiment across all tweets over the study period. Major fluctuations in volume (between 12,622 and 51,340) and sentiment (between 25% and 49%) appeared to coincide with key changes to any local and/or national social distancing measures. Tweets around mental health were polarizing, discussed with both positive and negative sentiment. Key topics of consistent discussion over the study period included the impact of the pandemic on people’s mental health (both positively and negatively), fear and anxiety over lockdowns, and anger and mistrust toward the government. CONCLUSIONS Using an NLP platform, we were able to rapidly mine and analyze emerging health-related insights from UK tweets into how the pandemic may be impacting people’s mental health and well-being. This type of real-time analyzed evidence could act as a useful intelligence source that agencies, local leaders, and health care decision makers can potentially draw from, particularly during a health crisis.