Matthew Russell Bennion
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Research Interests
Matthew Russell Bennion has a total of 25 co-authors. Here are the top co-authors by number of shared publications:
- Roger K. Moore
University of Sheffield
6 shared publications - Abigail Millings
Centre for Behavioural Science and Applied Psychology, Sheffield Hallam University, Sheffield, United Kingdom
5 shared publications - Gillian E. Hardy
University of Sheffield
4 shared publications - Stephen Kellett
University of Sheffield Sheffield UK
4 shared publications - Felicity Baker
Ultimate Resilience Ltd, Nottingham, United Kingdom
3 shared publications - Joanna Burrell
Department of Psychology, University of Sheffield, Cathedral Court, Sheffield, S1 2LT, United Kingdom, 44 7734959002
3 shared publications
Latest Publications
- Open Access
- 6 January 2025
- Published by JMIR Publications Inc. in JMIR Medical Education
- Vol. 11, e51101
Resilience Training Web App for National Health Service Keyworkers: Pilot Usability Study
Background: It is well established that frontline health care staff are particularly at risk of stress. Resilience is important to help staff to manage daily challenges and to protect against burnout. Objective: This study aimed to assess the usability and user perceptions of a resilience training web app developed to support health care keyworkers in understanding their own stress response and to help them put into place strategies to manage stress and to build resilience. Methods: Nurses (n=7) and other keyworkers (n=1), the target users for the resilience training web app, participated in the usability evaluation. Participants completed a pretraining questionnaire capturing basic demographic information and then used the training before completing a posttraining feedback questionnaire exploring the impact and usability of the web app. Results: From a sample of 8 keyworkers, 6 (75%) rated their current role as “sometimes” stressful. All 8 (100%) keyworkers found the training easy to understand, and 5 of 7 (71%) agreed that the training increased their understanding of both stress and resilience. Further, 6 of 8 (75%) agreed that the resilience model had helped them to understand what resilience is. Many of the keyworkers (6/8, 75%) agreed that the content was relevant to them. Furthermore, 6 of 8 (75%) agreed that they were likely to act to develop their resilience following completion of the training. Conclusions: This study tested the usability of a web app for resilience training specifically targeting National Health Service keyworkers. This work preceded a larger scale usability study, and it is hoped this study will help guide other studies to develop similar programs in clinical settings.
- 6 January 2025
- Published by JMIR Publications Inc. in JMIR Medical Education
- Vol. 11, e51101
Open AccessResilience Training Web App for National Health Service Keyworkers: Pilot Usability Study
Background: It is well established that frontline health care staff are particularly at risk of stress. Resilience is important to help staff to manage daily challenges and to protect against burnout. Objective: This study aimed to assess the usability and user perceptions of a resilience training web app developed to support health care keyworkers in understanding their own stress response and to help them put into place strategies to manage stress and to build resilience. Methods: Nurses (n=7) and other keyworkers (n=1), the target users for the resilience training web app, participated in the usability evaluation. Participants completed a pretraining questionnaire capturing basic demographic information and then used the training before completing a posttraining feedback questionnaire exploring the impact and usability of the web app. Results: From a sample of 8 keyworkers, 6 (75%) rated their current role as “sometimes” stressful. All 8 (100%) keyworkers found the training easy to understand, and 5 of 7 (71%) agreed that the training increased their understanding of both stress and resilience. Further, 6 of 8 (75%) agreed that the resilience model had helped them to understand what resilience is. Many of the keyworkers (6/8, 75%) agreed that the content was relevant to them. Furthermore, 6 of 8 (75%) agreed that they were likely to act to develop their resilience following completion of the training. Conclusions: This study tested the usability of a web app for resilience training specifically targeting National Health Service keyworkers. This work preceded a larger scale usability study, and it is hoped this study will help guide other studies to develop similar programs in clinical settings.
- Open Access
- 12 September 2024
- Published by JMIR Publications Inc. in JMIR Human Factors
- Vol. 12, e66398
Digital capability, open-source use, and interoperability standards within the NHS in England: A survey of healthcare trusts (Preprint)
Journal of Medical Internet Research - International Scientific Journal for Medical Research, Information and Communication on the Internet #Preprint #PeerReviewMe: Warning: This is a unreviewed preprint. Readers are warned that the document has not been peer-reviewed by expert/patient reviewers or an academic editor, may contain misleading claims, and is likely to undergo changes before final publication, if accepted, or may have been rejected/withdrawn. Readers with interest and expertise are encouraged to sign up as peer-reviewer, if the paper is within an open peer-review period. Please cite this preprint only for review purposes or for grant applications and CVs (if you are the author). Background: In 2016 NHS England sought to drive digital transformation within select NHS organisations through the Global Digital Exemplar (GDE) programme. While the programme did advance the NHS’ integration with digital technologies, disparities in digital maturity persisted between GDE-funded and non-funded NHS organisations. The Department of Health and Social Care (DHSC) launched a data strategy in 2022 that aimed to develop the appropriate technical infrastructure and data architecture to enable more effective and efficient use of its data. Given the diversity in digital capabilities, open-source adoption, and interoperability standards within NHS services, official guidance has continued to struggle to provide effective unification. Data about capabilities and technologies from application development teams in the NHS trusts, crucial for advancing these areas, remains insufficient. Objective: The objective of this study was to document further the capabilities and technologies used in the NHS to develop digital capacity. Comparing those with standard funding against those with additional GDE funding. This comparative analysis provides a foundational understanding for evaluating current practices and identifying potential areas for improvement in the NHS digital transformation efforts. Methods: The study was conducted using Freedom of Information (FOI) and systematic websites. The Freedom of Information Act allows individuals to request information held by public authorities. This process supports transparency and accountability by ensuring public access to government data. Data were compiled from responses to: (1) Freedom of Information (FOI) requests to NHS trusts between July 2020 and December 2020, and (2) systematic website searches for technology were conducted between August 2020 and July 2021. A series of χ² tests were conducted to validate and strengthen the robustness of the FOI questions. Results: A total of 191 (84.5%) of the then 226 NHS trusts completed the FOI request and 161 of the 191 (84%) had software/app development, website, innovation teams. One hundred and twelve (69.6%) teams developed front facing service user websites/apps. One hundred and fifty (93.2%) worked with clinical staff to formulate innovative ideas. Fifty-five (34.2%) carried out developments for other trusts and external entities. Thirty-five (21.7%) had attempted to secure an innovation grant. One-Hundred and thirty-eight (86%) disclosed the technologies they use. A total of 84 technologies were identified: 1 query language, 27 programming languages, 19 frameworks, 10 libraries, 11 databases, 7 interoperability standards, 5 application programming interfaces / web services and 4 continuous integration / continuous deployment platforms. Twenty-five (15.5%) said they always used open-source technology and 9 (5.6%) said they used it sometimes. Twenty-four (17%) disclosed technologies referred to interoperability standards in their responses. Conclusions: The NHS must adopt a cohesive strategy and refine policies to ensure the success of its digital, open-source technology, and interoperability standards initiatives. Five recommendations toward greater organisational interoperability are made by the authors. Future research should examine digital innovation across NHS trusts, focusing on barriers such as limited resources, organisational culture, and technical expertise. Identifying these challenges is essential for developing strategies to reduce disparities and promote equal progress.
- 12 September 2024
- Published by JMIR Publications Inc. in JMIR Human Factors
- Vol. 12, e66398
Open AccessDigital capability, open-source use, and interoperability standards within the NHS in England: A survey of healthcare trusts (Preprint)
Journal of Medical Internet Research - International Scientific Journal for Medical Research, Information and Communication on the Internet #Preprint #PeerReviewMe: Warning: This is a unreviewed preprint. Readers are warned that the document has not been peer-reviewed by expert/patient reviewers or an academic editor, may contain misleading claims, and is likely to undergo changes before final publication, if accepted, or may have been rejected/withdrawn. Readers with interest and expertise are encouraged to sign up as peer-reviewer, if the paper is within an open peer-review period. Please cite this preprint only for review purposes or for grant applications and CVs (if you are the author). Background: In 2016 NHS England sought to drive digital transformation within select NHS organisations through the Global Digital Exemplar (GDE) programme. While the programme did advance the NHS’ integration with digital technologies, disparities in digital maturity persisted between GDE-funded and non-funded NHS organisations. The Department of Health and Social Care (DHSC) launched a data strategy in 2022 that aimed to develop the appropriate technical infrastructure and data architecture to enable more effective and efficient use of its data. Given the diversity in digital capabilities, open-source adoption, and interoperability standards within NHS services, official guidance has continued to struggle to provide effective unification. Data about capabilities and technologies from application development teams in the NHS trusts, crucial for advancing these areas, remains insufficient. Objective: The objective of this study was to document further the capabilities and technologies used in the NHS to develop digital capacity. Comparing those with standard funding against those with additional GDE funding. This comparative analysis provides a foundational understanding for evaluating current practices and identifying potential areas for improvement in the NHS digital transformation efforts. Methods: The study was conducted using Freedom of Information (FOI) and systematic websites. The Freedom of Information Act allows individuals to request information held by public authorities. This process supports transparency and accountability by ensuring public access to government data. Data were compiled from responses to: (1) Freedom of Information (FOI) requests to NHS trusts between July 2020 and December 2020, and (2) systematic website searches for technology were conducted between August 2020 and July 2021. A series of χ² tests were conducted to validate and strengthen the robustness of the FOI questions. Results: A total of 191 (84.5%) of the then 226 NHS trusts completed the FOI request and 161 of the 191 (84%) had software/app development, website, innovation teams. One hundred and twelve (69.6%) teams developed front facing service user websites/apps. One hundred and fifty (93.2%) worked with clinical staff to formulate innovative ideas. Fifty-five (34.2%) carried out developments for other trusts and external entities. Thirty-five (21.7%) had attempted to secure an innovation grant. One-Hundred and thirty-eight (86%) disclosed the technologies they use. A total of 84 technologies were identified: 1 query language, 27 programming languages, 19 frameworks, 10 libraries, 11 databases, 7 interoperability standards, 5 application programming interfaces / web services and 4 continuous integration / continuous deployment platforms. Twenty-five (15.5%) said they always used open-source technology and 9 (5.6%) said they used it sometimes. Twenty-four (17%) disclosed technologies referred to interoperability standards in their responses. Conclusions: The NHS must adopt a cohesive strategy and refine policies to ensure the success of its digital, open-source technology, and interoperability standards initiatives. Five recommendations toward greater organisational interoperability are made by the authors. Future research should examine digital innovation across NHS trusts, focusing on barriers such as limited resources, organisational culture, and technical expertise. Identifying these challenges is essential for developing strategies to reduce disparities and promote equal progress.
- Open Access
- 12 September 2024
- Published by JMIR Publications Inc. in JMIR Preprints
Digital capability, open-source use, and interoperability standards within the NHS in England: A survey of healthcare trusts (Preprint)
UNSTRUCTURED Abstract Objective To document the capabilities and technologies used in the NHS to develop digital. Design The study was conducted using Freedom of Information (FOI) requests and systematic website searches. Data sources Data were collected via FOI requests to NHS trusts between July 2020 and July 2021. Technology searches were conducted between August 2020 and July 2021. Data collection/extraction methods Data were compiled from responses to the FOI request and technology websites. Results A total of 191 (84.5%) of the then 226 NHS trusts completed the FOI request and 161 of the 191 (84%) had software/app development, website, innovation teams. One hundred and twelve (70%) teams developed front facing service user websites/apps. One hundred and fifty (93%) worked with clinical staff to formulate innovative ideas. Fifty-five (34%) carried out developments for other trusts and external entities. Thirty-five (22%) had attempted to secure an innovation grant. One-Hundred and thirty-eight (86%) disclosed the technologies they use. A total of 84 technologies were identified: 28 programming languages, 19 frameworks, 10 libraries, 11 databases, 7 interoperability standards, 5 application programming interfaces / web services and 4 continuous integration / continuous deployment platforms. Twenty-five (16%) said they always used open-source technology and 9 (6%) said they used it sometimes. Twenty-four (15%) disclosed technologies referred to interoperability standards in their responses. Conclusions The NHS must adopt a cohesive strategy and refine policies to ensure the success of its digital, open-source technology, and interoperability standards initiatives. Summary What is already known on this topic? • Research findings have shown that NHS trusts and social care providers still lack digital maturity. What does this study add? • The study provides a comprehensive list of technologies used in National Health Service (NHS) trusts in England. It identifies that NHS trusts are highly capable of digital innovation but are not equally adopting open-source technologies. How this study might affect research, practice or policy • This study could help to guide digital policy regarding innovation and the use of open source within the NHS.
- 12 September 2024
- Published by JMIR Publications Inc. in JMIR Preprints
Open AccessDigital capability, open-source use, and interoperability standards within the NHS in England: A survey of healthcare trusts (Preprint)
UNSTRUCTURED Abstract Objective To document the capabilities and technologies used in the NHS to develop digital. Design The study was conducted using Freedom of Information (FOI) requests and systematic website searches. Data sources Data were collected via FOI requests to NHS trusts between July 2020 and July 2021. Technology searches were conducted between August 2020 and July 2021. Data collection/extraction methods Data were compiled from responses to the FOI request and technology websites. Results A total of 191 (84.5%) of the then 226 NHS trusts completed the FOI request and 161 of the 191 (84%) had software/app development, website, innovation teams. One hundred and twelve (70%) teams developed front facing service user websites/apps. One hundred and fifty (93%) worked with clinical staff to formulate innovative ideas. Fifty-five (34%) carried out developments for other trusts and external entities. Thirty-five (22%) had attempted to secure an innovation grant. One-Hundred and thirty-eight (86%) disclosed the technologies they use. A total of 84 technologies were identified: 28 programming languages, 19 frameworks, 10 libraries, 11 databases, 7 interoperability standards, 5 application programming interfaces / web services and 4 continuous integration / continuous deployment platforms. Twenty-five (16%) said they always used open-source technology and 9 (6%) said they used it sometimes. Twenty-four (15%) disclosed technologies referred to interoperability standards in their responses. Conclusions The NHS must adopt a cohesive strategy and refine policies to ensure the success of its digital, open-source technology, and interoperability standards initiatives. Summary What is already known on this topic? • Research findings have shown that NHS trusts and social care providers still lack digital maturity. What does this study add? • The study provides a comprehensive list of technologies used in National Health Service (NHS) trusts in England. It identifies that NHS trusts are highly capable of digital innovation but are not equally adopting open-source technologies. How this study might affect research, practice or policy • This study could help to guide digital policy regarding innovation and the use of open source within the NHS.
- Open Access
- 22 July 2023
- Published by JMIR Publications Inc. in JMIR Preprints
A Resilience Training Web App for NHS Keyworkers: A Pilot Usability and Feasibility Study (Preprint)
BACKGROUND It is well established that frontline healthcare staff are particularly at risk of stress. Resilience is important to help staff to manage daily challenges and to protect against burnout. OBJECTIVE The aim of the current study was to develop and assess the usability and user perceptions of a digital learning package. The training aimed to support healthcare keyworkers to understand their own stress response, to put in place strategies to manage stress and to build resilience. METHODS Participants were NHS keyworkers working in critical care or the emergency department at the University Hospitals Birmingham NHS Foundation Trust. Consenting participants completed a pre training questionnaire capturing basic demographic information. They accessed a web app and completed a resilience training programme based on the Skills-based Model of Personal Resilience. Finally, they completed a post training feedback questionnaire exploring the impact and usability of the training app. RESULTS A total of 8 participants completed the surveys and training, including 7 nurses and 1 other keyworker. Participants were aged 25 to 64 years and 100.0% were female. Grades ranged from 5 to 7, with a median of 6. Five of the participants had over 15 years of nursing experience. Many participants (6/8) rated their current role as “sometimes” stressful. Post training feedback indicated that all participants found the training easy to navigate and understand. The majority felt that the training increased their understanding of stress and resilience and was relevant to them. For the breathing and positive tips exercises, 75% or more of participants agreed they were likely to try the exercises again in the future and half agreed they were likely to try the mindfulness exercise again. The majority participants agreed that they were likely to take action to develop their resilience following completion of the training. Participants felt the package should be made available across all platforms to allow the training to be completed where and when was most convenient to them. CONCLUSIONS A total of 8 participants completed the surveys and training, including 7 nurses and 1 other keyworker. Participants were aged 25 to 64 years and 100.0% were female. Grades ranged from 5 to 7, with a median of 6. Five of the participants had over 15 years of nursing experience. Many participants (6/8) rated their current role as “sometimes” stressful. Post training feedback indicated that all participants found the training easy to navigate and understand. The majority felt that the training increased their understanding of stress and resilience and was relevant to them. For the breathing and positive tips exercises, 75% or more of participants agreed they were likely to try the exercises again in the future and half agreed they were likely to try the mindfulness exercise again. The majority participants agreed that they were likely to take action to develop their resilience following completion of the training. Participants felt the package should be made available across all platforms to allow the training to be completed where and when was most convenient to them.
- 22 July 2023
- Published by JMIR Publications Inc. in JMIR Preprints
Open AccessA Resilience Training Web App for NHS Keyworkers: A Pilot Usability and Feasibility Study (Preprint)
BACKGROUND It is well established that frontline healthcare staff are particularly at risk of stress. Resilience is important to help staff to manage daily challenges and to protect against burnout. OBJECTIVE The aim of the current study was to develop and assess the usability and user perceptions of a digital learning package. The training aimed to support healthcare keyworkers to understand their own stress response, to put in place strategies to manage stress and to build resilience. METHODS Participants were NHS keyworkers working in critical care or the emergency department at the University Hospitals Birmingham NHS Foundation Trust. Consenting participants completed a pre training questionnaire capturing basic demographic information. They accessed a web app and completed a resilience training programme based on the Skills-based Model of Personal Resilience. Finally, they completed a post training feedback questionnaire exploring the impact and usability of the training app. RESULTS A total of 8 participants completed the surveys and training, including 7 nurses and 1 other keyworker. Participants were aged 25 to 64 years and 100.0% were female. Grades ranged from 5 to 7, with a median of 6. Five of the participants had over 15 years of nursing experience. Many participants (6/8) rated their current role as “sometimes” stressful. Post training feedback indicated that all participants found the training easy to navigate and understand. The majority felt that the training increased their understanding of stress and resilience and was relevant to them. For the breathing and positive tips exercises, 75% or more of participants agreed they were likely to try the exercises again in the future and half agreed they were likely to try the mindfulness exercise again. The majority participants agreed that they were likely to take action to develop their resilience following completion of the training. Participants felt the package should be made available across all platforms to allow the training to be completed where and when was most convenient to them. CONCLUSIONS A total of 8 participants completed the surveys and training, including 7 nurses and 1 other keyworker. Participants were aged 25 to 64 years and 100.0% were female. Grades ranged from 5 to 7, with a median of 6. Five of the participants had over 15 years of nursing experience. Many participants (6/8) rated their current role as “sometimes” stressful. Post training feedback indicated that all participants found the training easy to navigate and understand. The majority felt that the training increased their understanding of stress and resilience and was relevant to them. For the breathing and positive tips exercises, 75% or more of participants agreed they were likely to try the exercises again in the future and half agreed they were likely to try the mindfulness exercise again. The majority participants agreed that they were likely to take action to develop their resilience following completion of the training. Participants felt the package should be made available across all platforms to allow the training to be completed where and when was most convenient to them.
- Open Access
- 17 March 2022
- Published by Springer Nature in Journal of Technology in Behavioral Science
- Vol. 7 (2) , 125-129
An Unguided Web-Based Resilience Training Programme for NHS Keyworkers During the COVID-19 Pandemic: a Usability Study
- 17 March 2022
- Published by Springer Nature in Journal of Technology in Behavioral Science
- Vol. 7 (2) , 125-129
Open AccessAn Unguided Web-Based Resilience Training Programme for NHS Keyworkers During the COVID-19 Pandemic: a Usability Study
- Open Access
- 28 October 2020
- Published by JMIR Publications Inc. in Journal of Medical Internet Research
- Vol. 22 (10) , e17049
Acceptability and Effectiveness of NHS-Recommended e-Therapies for Depression, Anxiety, and Stress: Meta-Analysis
Background: There is a disconnect between the ability to swiftly develop e-therapies for the treatment of depression, anxiety, and stress, and the scrupulous evaluation of their clinical utility. This creates a risk that the e-therapies routinely provided within publicly funded psychological health care have evaded appropriate rigorous evaluation in their development. Objective: This study aims to conduct a meta-analytic review of the gold standard evidence of the acceptability and clinical effectiveness of e-therapies recommended for use in the National Health Service (NHS) in the United Kingdom. Methods: Systematic searches identified appropriate randomized controlled trials (RCTs). Depression, anxiety, and stress outcomes at the end of treatment and follow-up were synthesized using a random-effects meta-analysis. The grading of recommendations assessment, development, and evaluation approach was used to assess the quality of each meta-analytic comparison. Moderators of treatment effect were examined using subgroup and meta-regression analysis. Dropout rates for e-therapies (as a proxy for acceptability) were compared against controls. Results: A total of 24 studies evaluating 7 of 48 NHS-recommended e-therapies were qualitatively and quantitatively synthesized. Depression, anxiety, and stress outcomes for e-therapies were superior to controls (depression: standardized mean difference [SMD] 0.38, 95% CI 0.24 to 0.52, N=7075; anxiety and stress: SMD 0.43, 95% CI 0.24 to 0.63, n=4863), and these small effects were maintained at follow-up. Average dropout rates for e-therapies (31%, SD 17.35) were significantly higher than those of controls (17%, SD 13.31). Limited moderators of the treatment effect were found. Conclusions: Many NHS-recommended e-therapies have not been through an RCT-style evaluation. The e-therapies that have been appropriately evaluated generate small but significant, durable, beneficial treatment effects. Trial Registration: International Prospective Register of Systematic Reviews (PROSPERO) registration CRD42019130184; https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=130184
- 28 October 2020
- Published by JMIR Publications Inc. in Journal of Medical Internet Research
- Vol. 22 (10) , e17049
Open AccessAcceptability and Effectiveness of NHS-Recommended e-Therapies for Depression, Anxiety, and Stress: Meta-Analysis
Background: There is a disconnect between the ability to swiftly develop e-therapies for the treatment of depression, anxiety, and stress, and the scrupulous evaluation of their clinical utility. This creates a risk that the e-therapies routinely provided within publicly funded psychological health care have evaded appropriate rigorous evaluation in their development. Objective: This study aims to conduct a meta-analytic review of the gold standard evidence of the acceptability and clinical effectiveness of e-therapies recommended for use in the National Health Service (NHS) in the United Kingdom. Methods: Systematic searches identified appropriate randomized controlled trials (RCTs). Depression, anxiety, and stress outcomes at the end of treatment and follow-up were synthesized using a random-effects meta-analysis. The grading of recommendations assessment, development, and evaluation approach was used to assess the quality of each meta-analytic comparison. Moderators of treatment effect were examined using subgroup and meta-regression analysis. Dropout rates for e-therapies (as a proxy for acceptability) were compared against controls. Results: A total of 24 studies evaluating 7 of 48 NHS-recommended e-therapies were qualitatively and quantitatively synthesized. Depression, anxiety, and stress outcomes for e-therapies were superior to controls (depression: standardized mean difference [SMD] 0.38, 95% CI 0.24 to 0.52, N=7075; anxiety and stress: SMD 0.43, 95% CI 0.24 to 0.63, n=4863), and these small effects were maintained at follow-up. Average dropout rates for e-therapies (31%, SD 17.35) were significantly higher than those of controls (17%, SD 13.31). Limited moderators of the treatment effect were found. Conclusions: Many NHS-recommended e-therapies have not been through an RCT-style evaluation. The e-therapies that have been appropriately evaluated generate small but significant, durable, beneficial treatment effects. Trial Registration: International Prospective Register of Systematic Reviews (PROSPERO) registration CRD42019130184; https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=130184
- Open Access
- 27 May 2020
- Published by JMIR Publications Inc. in Journal of Medical Internet Research
- Vol. 22 (5) , e16794
Usability, Acceptability, and Effectiveness of Web-Based Conversational Agents to Facilitate Problem Solving in Older Adults: Controlled Study
Background: The usability and effectiveness of conversational agents (chatbots) that deliver psychological therapies is under-researched. Objective: This study aimed to compare the system usability, acceptability, and effectiveness in older adults of 2 Web-based conversational agents that differ in theoretical orientation and approach. Methods: In a randomized study, 112 older adults were allocated to 1 of the following 2 fully automated interventions: Manage Your Life Online (MYLO; ie, a chatbot that mimics a therapist using a method of levels approach) and ELIZA (a chatbot that mimics a therapist using a humanistic counseling approach). The primary outcome was problem distress and resolution, with secondary outcome measures of system usability and clinical outcome. Results: MYLO participants spent significantly longer interacting with the conversational agent. Posthoc tests indicated that MYLO participants had significantly lower problem distress at follow-up. There were no differences between MYLO and ELIZA in terms of problem resolution. MYLO was rated as significantly more helpful and likely to be used again. System usability of both the conversational agents was associated with helpfulness of the agents and the willingness of the participants to reuse. Adherence was high. A total of 12% (7/59) of the MYLO group did not carry out their conversation with the chatbot. Conclusions: Controlled studies of chatbots need to be conducted in clinical populations across different age groups. The potential integration of chatbots into psychological care in routine services is discussed.
- 27 May 2020
- Published by JMIR Publications Inc. in Journal of Medical Internet Research
- Vol. 22 (5) , e16794
Open AccessUsability, Acceptability, and Effectiveness of Web-Based Conversational Agents to Facilitate Problem Solving in Older Adults: Controlled Study
Background: The usability and effectiveness of conversational agents (chatbots) that deliver psychological therapies is under-researched. Objective: This study aimed to compare the system usability, acceptability, and effectiveness in older adults of 2 Web-based conversational agents that differ in theoretical orientation and approach. Methods: In a randomized study, 112 older adults were allocated to 1 of the following 2 fully automated interventions: Manage Your Life Online (MYLO; ie, a chatbot that mimics a therapist using a method of levels approach) and ELIZA (a chatbot that mimics a therapist using a humanistic counseling approach). The primary outcome was problem distress and resolution, with secondary outcome measures of system usability and clinical outcome. Results: MYLO participants spent significantly longer interacting with the conversational agent. Posthoc tests indicated that MYLO participants had significantly lower problem distress at follow-up. There were no differences between MYLO and ELIZA in terms of problem resolution. MYLO was rated as significantly more helpful and likely to be used again. System usability of both the conversational agents was associated with helpfulness of the agents and the willingness of the participants to reuse. Adherence was high. A total of 12% (7/59) of the MYLO group did not carry out their conversation with the chatbot. Conclusions: Controlled studies of chatbots need to be conducted in clinical populations across different age groups. The potential integration of chatbots into psychological care in routine services is discussed.
- Open Access
- 13 November 2019
- Published by JMIR Publications Inc. in JMIR Preprints
Acceptability and Effectiveness of NHS-Recommended e-Therapies for Depression, Anxiety, and Stress: Meta-Analysis (Preprint)
BACKGROUND There is a disconnect between the ability to swiftly develop e-therapies for the treatment of depression, anxiety, and stress, and the scrupulous evaluation of their clinical utility. This creates a risk that the e-therapies routinely provided within publicly funded psychological health care have evaded appropriate rigorous evaluation in their development. OBJECTIVE This study aims to conduct a meta-analytic review of the gold standard evidence of the acceptability and clinical effectiveness of e-therapies recommended for use in the National Health Service (NHS) in the United Kingdom. METHODS Systematic searches identified appropriate randomized controlled trials (RCTs). Depression, anxiety, and stress outcomes at the end of treatment and follow-up were synthesized using a random-effects meta-analysis. The grading of recommendations assessment, development, and evaluation approach was used to assess the quality of each meta-analytic comparison. Moderators of treatment effect were examined using subgroup and meta-regression analysis. Dropout rates for e-therapies (as a proxy for acceptability) were compared against controls. RESULTS A total of 24 studies evaluating 7 of 48 NHS-recommended e-therapies were qualitatively and quantitatively synthesized. Depression, anxiety, and stress outcomes for e-therapies were superior to controls (depression: standardized mean difference [SMD] 0.38, 95% CI 0.24 to 0.52, N=7075; anxiety and stress: SMD 0.43, 95% CI 0.24 to 0.63, n=4863), and these small effects were maintained at follow-up. Average dropout rates for e-therapies (31%, SD 17.35) were significantly higher than those of controls (17%, SD 13.31). Limited moderators of the treatment effect were found. CONCLUSIONS Many NHS-recommended e-therapies have not been through an RCT-style evaluation. The e-therapies that have been appropriately evaluated generate small but significant, durable, beneficial treatment effects. CLINICALTRIAL International Prospective Register of Systematic Reviews (PROSPERO) registration CRD42019130184; https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=130184
- 13 November 2019
- Published by JMIR Publications Inc. in JMIR Preprints
Open AccessAcceptability and Effectiveness of NHS-Recommended e-Therapies for Depression, Anxiety, and Stress: Meta-Analysis (Preprint)
BACKGROUND There is a disconnect between the ability to swiftly develop e-therapies for the treatment of depression, anxiety, and stress, and the scrupulous evaluation of their clinical utility. This creates a risk that the e-therapies routinely provided within publicly funded psychological health care have evaded appropriate rigorous evaluation in their development. OBJECTIVE This study aims to conduct a meta-analytic review of the gold standard evidence of the acceptability and clinical effectiveness of e-therapies recommended for use in the National Health Service (NHS) in the United Kingdom. METHODS Systematic searches identified appropriate randomized controlled trials (RCTs). Depression, anxiety, and stress outcomes at the end of treatment and follow-up were synthesized using a random-effects meta-analysis. The grading of recommendations assessment, development, and evaluation approach was used to assess the quality of each meta-analytic comparison. Moderators of treatment effect were examined using subgroup and meta-regression analysis. Dropout rates for e-therapies (as a proxy for acceptability) were compared against controls. RESULTS A total of 24 studies evaluating 7 of 48 NHS-recommended e-therapies were qualitatively and quantitatively synthesized. Depression, anxiety, and stress outcomes for e-therapies were superior to controls (depression: standardized mean difference [SMD] 0.38, 95% CI 0.24 to 0.52, N=7075; anxiety and stress: SMD 0.43, 95% CI 0.24 to 0.63, n=4863), and these small effects were maintained at follow-up. Average dropout rates for e-therapies (31%, SD 17.35) were significantly higher than those of controls (17%, SD 13.31). Limited moderators of the treatment effect were found. CONCLUSIONS Many NHS-recommended e-therapies have not been through an RCT-style evaluation. The e-therapies that have been appropriately evaluated generate small but significant, durable, beneficial treatment effects. CLINICALTRIAL International Prospective Register of Systematic Reviews (PROSPERO) registration CRD42019130184; https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=130184
- Open Access
- 26 October 2019
- Published by JMIR Publications Inc. in JMIR Preprints
Usability, Acceptability, and Effectiveness of Web-Based Conversational Agents to Facilitate Problem Solving in Older Adults: Controlled Study (Preprint)
BACKGROUND The usability and effectiveness of conversational agents (chatbots) that deliver psychological therapies is under-researched. OBJECTIVE This study aimed to compare the system usability, acceptability, and effectiveness in older adults of 2 Web-based conversational agents that differ in theoretical orientation and approach. METHODS In a randomized study, 112 older adults were allocated to 1 of the following 2 fully automated interventions: Manage Your Life Online (MYLO; ie, a chatbot that mimics a therapist using a method of levels approach) and ELIZA (a chatbot that mimics a therapist using a humanistic counseling approach). The primary outcome was problem distress and resolution, with secondary outcome measures of system usability and clinical outcome. RESULTS MYLO participants spent significantly longer interacting with the conversational agent. Posthoc tests indicated that MYLO participants had significantly lower problem distress at follow-up. There were no differences between MYLO and ELIZA in terms of problem resolution. MYLO was rated as significantly more helpful and likely to be used again. System usability of both the conversational agents was associated with helpfulness of the agents and the willingness of the participants to reuse. Adherence was high. A total of 12% (7/59) of the MYLO group did not carry out their conversation with the chatbot. CONCLUSIONS Controlled studies of chatbots need to be conducted in clinical populations across different age groups. The potential integration of chatbots into psychological care in routine services is discussed.
- 26 October 2019
- Published by JMIR Publications Inc. in JMIR Preprints
Open AccessUsability, Acceptability, and Effectiveness of Web-Based Conversational Agents to Facilitate Problem Solving in Older Adults: Controlled Study (Preprint)
BACKGROUND The usability and effectiveness of conversational agents (chatbots) that deliver psychological therapies is under-researched. OBJECTIVE This study aimed to compare the system usability, acceptability, and effectiveness in older adults of 2 Web-based conversational agents that differ in theoretical orientation and approach. METHODS In a randomized study, 112 older adults were allocated to 1 of the following 2 fully automated interventions: Manage Your Life Online (MYLO; ie, a chatbot that mimics a therapist using a method of levels approach) and ELIZA (a chatbot that mimics a therapist using a humanistic counseling approach). The primary outcome was problem distress and resolution, with secondary outcome measures of system usability and clinical outcome. RESULTS MYLO participants spent significantly longer interacting with the conversational agent. Posthoc tests indicated that MYLO participants had significantly lower problem distress at follow-up. There were no differences between MYLO and ELIZA in terms of problem resolution. MYLO was rated as significantly more helpful and likely to be used again. System usability of both the conversational agents was associated with helpfulness of the agents and the willingness of the participants to reuse. Adherence was high. A total of 12% (7/59) of the MYLO group did not carry out their conversation with the chatbot. CONCLUSIONS Controlled studies of chatbots need to be conducted in clinical populations across different age groups. The potential integration of chatbots into psychological care in routine services is discussed.
- Open Access
- 5 June 2019
- Published by BMJ in BMJ Health & Care Informatics
- Vol. 26 (1) , e100027
e-Therapies in England for stress, anxiety or depression: how are apps developed? A survey of NHS e-therapy developers
Objective To document the quality of web and smartphone apps used and recommended for stress, anxiety or depression by examining the manner in which they were developed. Design The study was conducted using a survey sent to developers of National Health Service (NHS) e-therapies. Data sources Data were collected via a survey sent out to NHS e-therapy developers during October 2015 and review of development company websites during October 2015. Data collection/extraction methods Data were compiled from responses to the survey and development company websites of the NHS e-therapies developers. Results A total of 36 (76.6%) out of the 48 app developers responded. One app was excluded due to its contact details and developer website being unidentifiable. Data from the missing 10 was determined from the app developer’s website. The results were that 12 out of 13 web apps and 20 out of 34 smartphone apps had clinical involvement in their development. Nine out of 13 web apps and nine out of 34 smartphone apps indicated academic involvement in their development. Twelve out of 13 web apps and nine out of 34 smartphone apps indicated published research evidence relating to their app. Ten out of 13 web apps and 10 out of 34 smartphone apps indicated having other evidence relating to their app. Nine out of 13 web apps and 19 out of 34 smartphone apps indicated having a psychological approach or theory behind their app. Conclusions As an increasing number of developers are looking to produce e-therapies for the NHS it is essential they apply clinical and academic best practices to ensure the creation of safe and effective apps.
- 5 June 2019
- Published by BMJ in BMJ Health & Care Informatics
- Vol. 26 (1) , e100027
Open Accesse-Therapies in England for stress, anxiety or depression: how are apps developed? A survey of NHS e-therapy developers
Objective To document the quality of web and smartphone apps used and recommended for stress, anxiety or depression by examining the manner in which they were developed. Design The study was conducted using a survey sent to developers of National Health Service (NHS) e-therapies. Data sources Data were collected via a survey sent out to NHS e-therapy developers during October 2015 and review of development company websites during October 2015. Data collection/extraction methods Data were compiled from responses to the survey and development company websites of the NHS e-therapies developers. Results A total of 36 (76.6%) out of the 48 app developers responded. One app was excluded due to its contact details and developer website being unidentifiable. Data from the missing 10 was determined from the app developer’s website. The results were that 12 out of 13 web apps and 20 out of 34 smartphone apps had clinical involvement in their development. Nine out of 13 web apps and nine out of 34 smartphone apps indicated academic involvement in their development. Twelve out of 13 web apps and nine out of 34 smartphone apps indicated published research evidence relating to their app. Ten out of 13 web apps and 10 out of 34 smartphone apps indicated having other evidence relating to their app. Nine out of 13 web apps and 19 out of 34 smartphone apps indicated having a psychological approach or theory behind their app. Conclusions As an increasing number of developers are looking to produce e-therapies for the NHS it is essential they apply clinical and academic best practices to ensure the creation of safe and effective apps.