Nicola Small
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Research Subjects
Nicola Small has a total of 162 co-authors. Here are the top co-authors by number of shared publications:
- Peter Bower
The University of Manchester
9 shared publications Peter Bower
NIHR School for Primary Care Research, Division of Population Health, Health Services Research and Primary Care, School of Health Sciences, the University of Manchester, Williamson Building, Oxford Road, Manchester, UK
6 shared publications- Caroline Sanders
Centre for Primary Care and Health Services Research, NIHR Greater Manchester Patient Safety Research Collaboration, Division of Population Health, Health Services Research and Primary Care, School of Health Sciences, Faculty of Biology, Medicine and Health, The University of Manchester, Manchester, UK
5 shared publications Anne Kennedy
Health Sciences, University of Southampton, Southampton, UK
4 shared publications- Maria Panagioti
Division of Population Health, Health Services, Research and Primary Care, University of Manchester, Manchester M13 9PG, UK
4 shared publications - Anne Kennedy
NIHR Collaboration for Leadership in Applied Health Research and Care (CLAHRC) Wessex, Health Sciences, University of Southampton, Highfield, Southampton, SO17 1BJ, UK
3 shared publications
Latest Publications
- Open Access
- 3 October 2026
- Published by Wiley in Health Expectations
- Vol. 29 (5) , e70894
Can Care Coordination for Multiple Long‐Term Conditions Be Meaningfully Measured Using Routine Data? A Multi‐Stage Consultation
Background: Care coordination is essential for improving healthcare experiences and outcomes for people living with multiple long‐term conditions (MLTC), yet it remains challenging to measure care coordination in ways that are both meaningful and feasible at scale. Routine electronic health record (EHR) data offer potential for system‐wide measurement, but it is unclear whether such measures reflect what matters to patients, carers, and those delivering care. Methods: We undertook a multi‐stage, collaborative consultation process to examine whether care coordination in MLTC can be meaningfully measured using routine data. Public contributors ( n = 18), professional contributors ( n = 23), researchers ( n = 15) and data experts ( n = 10) participated in structured discussions, co‐developed appraisal criteria (including relevance to lived experience and feasibility in EHR data), and assessed measures identified through a linked systematic review. A consensus survey evaluated the appropriateness of shortlisted measures. Results: Participants identified priorities including teamwork, communication, continuity, care management, and accountability. Appraisal criteria balanced patient relevance with feasibility in routine data. While a feasible shortlist of measures was identified, findings showed only qualified support for routine data–derived indicators. Continuity measures were most strongly endorsed but widely recognised as proxies capturing observable activity rather than the relational, experiential, and cross‐sector dimensions central to coordinated care. Concerns were raised regarding data quality, conceptual validity, and the risk of incomplete or misleading representations. Conclusions: Routine data–derived measures of care coordination in MLTC hold clear potential but are inherently limited. The findings support a conditional position: such measures can provide useful system‐level signals, but only if interpreted cautiously, supported by robust data infrastructure, and complemented by patient‐reported insights. Advancing measurement will require integrated approaches that better capture what matters to patients and carers.
- 3 October 2026
- Published by Wiley in Health Expectations
- Vol. 29 (5) , e70894
Open AccessCan Care Coordination for Multiple Long‐Term Conditions Be Meaningfully Measured Using Routine Data? A Multi‐Stage Consultation
Background: Care coordination is essential for improving healthcare experiences and outcomes for people living with multiple long‐term conditions (MLTC), yet it remains challenging to measure care coordination in ways that are both meaningful and feasible at scale. Routine electronic health record (EHR) data offer potential for system‐wide measurement, but it is unclear whether such measures reflect what matters to patients, carers, and those delivering care. Methods: We undertook a multi‐stage, collaborative consultation process to examine whether care coordination in MLTC can be meaningfully measured using routine data. Public contributors ( n = 18), professional contributors ( n = 23), researchers ( n = 15) and data experts ( n = 10) participated in structured discussions, co‐developed appraisal criteria (including relevance to lived experience and feasibility in EHR data), and assessed measures identified through a linked systematic review. A consensus survey evaluated the appropriateness of shortlisted measures. Results: Participants identified priorities including teamwork, communication, continuity, care management, and accountability. Appraisal criteria balanced patient relevance with feasibility in routine data. While a feasible shortlist of measures was identified, findings showed only qualified support for routine data–derived indicators. Continuity measures were most strongly endorsed but widely recognised as proxies capturing observable activity rather than the relational, experiential, and cross‐sector dimensions central to coordinated care. Concerns were raised regarding data quality, conceptual validity, and the risk of incomplete or misleading representations. Conclusions: Routine data–derived measures of care coordination in MLTC hold clear potential but are inherently limited. The findings support a conditional position: such measures can provide useful system‐level signals, but only if interpreted cautiously, supported by robust data infrastructure, and complemented by patient‐reported insights. Advancing measurement will require integrated approaches that better capture what matters to patients and carers.
- Open Access
- 5 February 2024
- Published by Wiley in Health Expectations
- Vol. 27 (1) , e13982
Exploring the experiences and preferences of South Asian patients' of primary care in England since COVID‐19
Introduction: Remote (digital and/or telephone) access and consultation models are being driven by national policy with the goal being that the National Health Service operate on a remote‐first (digital‐first) basis by 2029. Previous research has suggested that remote methods of access to care and consulting may act to widen health inequalities for certain patients and/or groups such as those from ethnic minorities. South Asian (SA) patients comprise the largest ethnic minority group in England. Understanding the experiences and needs of this group is critical to ensuring that general practice can deliver equitable, quality health care.Methods: Qualitative study. 37 participants (from Indian, Pakistani and/or Bangladeshi background) were recruited to take part in either in‐person preferred language focus groups or remote semistructured interviews in the English language. Thematic analysis was conducted to identify themes in the qualitative data.Findings: Three major interlinked themes were identified: (1) reduced access, (2) reduced patient choice and (3) quality and safety concerns. The findings highlight access issues split by (i) general issues with appointment access via any remote means and (ii) specific issues related to language barriers creating additional barriers to access and care. Some patients valued the convenience of remote access but also raised concerns regarding appointment availability and reduced patient choice. Face‐to‐face consultations were preferable but less available. The findings underscore how participants perceived remote care to be of lesser quality and less safe. Concerns were greatest for those with limited English proficiency (LEP), with the removal of non‐verbal aspects of communication and ‘hands‐on’ care leading to perceptions of reduced psycho‐social safety.Conclusion: SA patients' experiences of remote‐led primary care access and care delivery were negative with only a minority viewing it positively and for certain limited scenarios. Face‐to‐face models of care remain the preferred mode of consultation, particularly for those with LEP. Hybrid models of access offer patients the greatest choice, and are likely to meet the varying needs of the South‐Asian patient population going forwards. The remote first approach to primary care may be achievable as a service ideal, but its limitations need to be recognised and accounted for to ensure that primary care can be an equitable service, both now and in the future.Public Contribution: Members of the public were involved in all phases of research in the study. This included co‐working in partnership throughout the study including, reviewing patient‐facing documents, recruiting participants, data facilitation, translation work, interpretation of the data and co‐authors on this manuscript. The key to the success of our study was collaborative teamwork, which involved experienced members of the public with SA cultural knowledge working together with and integral to the research team for all components.
- 5 February 2024
- Published by Wiley in Health Expectations
- Vol. 27 (1) , e13982
Open AccessExploring the experiences and preferences of South Asian patients' of primary care in England since COVID‐19
Introduction: Remote (digital and/or telephone) access and consultation models are being driven by national policy with the goal being that the National Health Service operate on a remote‐first (digital‐first) basis by 2029. Previous research has suggested that remote methods of access to care and consulting may act to widen health inequalities for certain patients and/or groups such as those from ethnic minorities. South Asian (SA) patients comprise the largest ethnic minority group in England. Understanding the experiences and needs of this group is critical to ensuring that general practice can deliver equitable, quality health care.Methods: Qualitative study. 37 participants (from Indian, Pakistani and/or Bangladeshi background) were recruited to take part in either in‐person preferred language focus groups or remote semistructured interviews in the English language. Thematic analysis was conducted to identify themes in the qualitative data.Findings: Three major interlinked themes were identified: (1) reduced access, (2) reduced patient choice and (3) quality and safety concerns. The findings highlight access issues split by (i) general issues with appointment access via any remote means and (ii) specific issues related to language barriers creating additional barriers to access and care. Some patients valued the convenience of remote access but also raised concerns regarding appointment availability and reduced patient choice. Face‐to‐face consultations were preferable but less available. The findings underscore how participants perceived remote care to be of lesser quality and less safe. Concerns were greatest for those with limited English proficiency (LEP), with the removal of non‐verbal aspects of communication and ‘hands‐on’ care leading to perceptions of reduced psycho‐social safety.Conclusion: SA patients' experiences of remote‐led primary care access and care delivery were negative with only a minority viewing it positively and for certain limited scenarios. Face‐to‐face models of care remain the preferred mode of consultation, particularly for those with LEP. Hybrid models of access offer patients the greatest choice, and are likely to meet the varying needs of the South‐Asian patient population going forwards. The remote first approach to primary care may be achievable as a service ideal, but its limitations need to be recognised and accounted for to ensure that primary care can be an equitable service, both now and in the future.Public Contribution: Members of the public were involved in all phases of research in the study. This included co‐working in partnership throughout the study including, reviewing patient‐facing documents, recruiting participants, data facilitation, translation work, interpretation of the data and co‐authors on this manuscript. The key to the success of our study was collaborative teamwork, which involved experienced members of the public with SA cultural knowledge working together with and integral to the research team for all components.
- Open Access
- 19 January 2024
- Published by F1000 Research Ltd in F1000Research
- Vol. 13, 78
Developing a measure of participant experience of trials: qualitative study and cognitive testing
Background To encourage participation in trials, people need to have a positive experience. However, researchers do not routinely measure participant experience. Our aim is to facilitate routine measurement by developing a measure that captures the participant perspective, in a way that is meaningful, acceptable and useful to trial teams and participants. Methods We conducted a focus group and 25 interviews with trial professionals and trial participants to explore how participant experiences of trials should be measured, and to identify domains and items to include in the measure. Interviewees were also asked to comment on a list of candidate domains and items informed by a previous review of the literature on participant experience measures. Interviews were analysed thematically. Candidate domains and items were incorporated into a draft measure. Cognitive testing was undertaken in two rounds to ensure the items were comprehensible and grounded in participant experience. Results Interviewees and patient and public contributors reported that standardising the measurement of participant experience of trials had the potential to improve trial experience but described issues around the timing of measurement. Cognitive testing highlighted issues with comprehension, recall and response and numerous items were removed or refined. We developed a standard and a short version of the measure for feasibility testing. Conclusions We developed a measure covering important domains of participant experience of trials, which could assist trial teams and participants to improve trial design and enhance delivery of a meaningful participant experience.
- 19 January 2024
- Published by F1000 Research Ltd in F1000Research
- Vol. 13, 78
Open AccessDeveloping a measure of participant experience of trials: qualitative study and cognitive testing
Background To encourage participation in trials, people need to have a positive experience. However, researchers do not routinely measure participant experience. Our aim is to facilitate routine measurement by developing a measure that captures the participant perspective, in a way that is meaningful, acceptable and useful to trial teams and participants. Methods We conducted a focus group and 25 interviews with trial professionals and trial participants to explore how participant experiences of trials should be measured, and to identify domains and items to include in the measure. Interviewees were also asked to comment on a list of candidate domains and items informed by a previous review of the literature on participant experience measures. Interviews were analysed thematically. Candidate domains and items were incorporated into a draft measure. Cognitive testing was undertaken in two rounds to ensure the items were comprehensible and grounded in participant experience. Results Interviewees and patient and public contributors reported that standardising the measurement of participant experience of trials had the potential to improve trial experience but described issues around the timing of measurement. Cognitive testing highlighted issues with comprehension, recall and response and numerous items were removed or refined. We developed a standard and a short version of the measure for feasibility testing. Conclusions We developed a measure covering important domains of participant experience of trials, which could assist trial teams and participants to improve trial design and enhance delivery of a meaningful participant experience.
- Open Access
- 10 January 2024
- Published by SAGE Publications in Journal of Multimorbidity and Comorbidity
- Vol. 14
Users’ views on the use of a smartwatch app to collect daily symptom data in individuals with multiple long-term conditions (Multimorbidity): A qualitative study
Introduction: Long-term conditions are a major burden on health systems. One way to facilitate more research and better clinical care among patients with long-term conditions is to collect accurate data on their daily symptoms (patient-generated health data) using wearable technologies. Whilst evidence is growing for the use of wearable technologies in single conditions, there is less evidence of the utility of frequent symptom tracking in those who have more than one condition. Aims: To explore patient views of the acceptability of collecting daily patient-generated health data for three months using a smartwatch app. Methods: Watch Your Steps was a longitudinal study which recruited 53 patients to track over 20 symptoms per day for a 90-day period using a study app on smartwatches. Semi-structured interviews were conducted with a sub-sample of 20 participants to explore their experience of engaging with the app. Results: In a population of older people with multimorbidity, patients were willing and able to engage with a patient-generated health data app on a smartwatch. It was suggested that to maintain engagement over a longer period, more ‘real-time’ feedback from the app should be available. Participants did not seem to consider the management of more than one condition to be a factor in either engagement or use of the app, but the presence of severe or chronic pain was at times a barrier. Conclusion: This study has provided preliminary evidence that multimorbidity was not a major barrier to engagement with patient-generated health data via a smartwatch symptom tracking app.
- 10 January 2024
- Published by SAGE Publications in Journal of Multimorbidity and Comorbidity
- Vol. 14
Open AccessUsers’ views on the use of a smartwatch app to collect daily symptom data in individuals with multiple long-term conditions (Multimorbidity): A qualitative study
Introduction: Long-term conditions are a major burden on health systems. One way to facilitate more research and better clinical care among patients with long-term conditions is to collect accurate data on their daily symptoms (patient-generated health data) using wearable technologies. Whilst evidence is growing for the use of wearable technologies in single conditions, there is less evidence of the utility of frequent symptom tracking in those who have more than one condition. Aims: To explore patient views of the acceptability of collecting daily patient-generated health data for three months using a smartwatch app. Methods: Watch Your Steps was a longitudinal study which recruited 53 patients to track over 20 symptoms per day for a 90-day period using a study app on smartwatches. Semi-structured interviews were conducted with a sub-sample of 20 participants to explore their experience of engaging with the app. Results: In a population of older people with multimorbidity, patients were willing and able to engage with a patient-generated health data app on a smartwatch. It was suggested that to maintain engagement over a longer period, more ‘real-time’ feedback from the app should be available. Participants did not seem to consider the management of more than one condition to be a factor in either engagement or use of the app, but the presence of severe or chronic pain was at times a barrier. Conclusion: This study has provided preliminary evidence that multimorbidity was not a major barrier to engagement with patient-generated health data via a smartwatch symptom tracking app.
- Open Access
- 27 November 2021
- Published by Springer Nature in Research Involvement and Engagement
- Vol. 7 (1) , 1-18
Co-designing new tools for collecting, analysing and presenting patient experience data in NHS services: working in partnership with patients and carers
Background: The way we collect and use patient experience data is vital to optimise the quality and safety of health services. Yet, some patients and carers do not give feedback because of the limited ways data is collected, analysed and presented. In this study, we worked together with researchers, staff, patient and carer participants, and patient and public involvement and engagement (PPIE) contributors, to co-design new tools for the collection and use of patient experience data in multiple health settings. This paper outlines how the range of PPIE and research activities enabled the co-design of new tools to collect patient experience data. Methods: Eight public contributors represented a range of relevant patient and carer experiences in specialist services with varied levels of PPIE experience, and eleven members of Patient and Participation Groups (PPGs) from two general practices formed our PPIE group at the start of the study. Slide sets were used to trigger co-design discussions with staff, patient and carer research participants, and PPIE contributors. Feedback from PPIE contributors alongside verbatim quotes from staff, patient and carer research participants is presented in relation to the themes from the research data. Results: PPIE insights from four themes: capturing experience data; adopting digital or non-digital tools; ensuring privacy and confidentiality; and co-design of a suite of new tools with guidance, informed joint decisions on the shaping of the tools and how these were implemented. Our PPIE contributors took different roles during co-design and testing of the new tools, which supported co-production of the study. Conclusions: Our experiences of developing multiple components of PPIE work for this complex study demonstrates the importance of tailoring PPIE to suit different settings, and to maximise individual strengths and capacity. Our study shows the value of bringing diverse experiences together, putting patients and carers at the heart of improving NHS services, and a shared approach to managing involvement in co-design, with the effects shown through the research process, outcomes and the partnership. We reflect on how we worked together to create a supportive environment when unforeseen challenges emerged (such as, sudden bereavement).
- 27 November 2021
- Published by Springer Nature in Research Involvement and Engagement
- Vol. 7 (1) , 1-18
Open AccessCo-designing new tools for collecting, analysing and presenting patient experience data in NHS services: working in partnership with patients and carers
Background: The way we collect and use patient experience data is vital to optimise the quality and safety of health services. Yet, some patients and carers do not give feedback because of the limited ways data is collected, analysed and presented. In this study, we worked together with researchers, staff, patient and carer participants, and patient and public involvement and engagement (PPIE) contributors, to co-design new tools for the collection and use of patient experience data in multiple health settings. This paper outlines how the range of PPIE and research activities enabled the co-design of new tools to collect patient experience data. Methods: Eight public contributors represented a range of relevant patient and carer experiences in specialist services with varied levels of PPIE experience, and eleven members of Patient and Participation Groups (PPGs) from two general practices formed our PPIE group at the start of the study. Slide sets were used to trigger co-design discussions with staff, patient and carer research participants, and PPIE contributors. Feedback from PPIE contributors alongside verbatim quotes from staff, patient and carer research participants is presented in relation to the themes from the research data. Results: PPIE insights from four themes: capturing experience data; adopting digital or non-digital tools; ensuring privacy and confidentiality; and co-design of a suite of new tools with guidance, informed joint decisions on the shaping of the tools and how these were implemented. Our PPIE contributors took different roles during co-design and testing of the new tools, which supported co-production of the study. Conclusions: Our experiences of developing multiple components of PPIE work for this complex study demonstrates the importance of tailoring PPIE to suit different settings, and to maximise individual strengths and capacity. Our study shows the value of bringing diverse experiences together, putting patients and carers at the heart of improving NHS services, and a shared approach to managing involvement in co-design, with the effects shown through the research process, outcomes and the partnership. We reflect on how we worked together to create a supportive environment when unforeseen challenges emerged (such as, sudden bereavement).
- Open Access
- 20 July 2021
- Published by Springer Nature in Research Square
Co-designing New Tools for Collecting, Analysing and Presenting Patient Experience Data in NHS Services: Working in Partnership With Patients and Carers
Background: The way we collect and use patient experience data is vital to optimise the quality and safety of health services. Yet, vulnerable patients and carers may be excluded because of the limited ways data is collected, analysed and presented. In this study, we worked together with researchers, staff, patient and carer participants, and patient and public involvement and engagement (PPIE) contributors to co-design new tools for the collection and use of patient experience data in multiple health settings. This paper outlines how the range of PPIE and research activities enabled the co-design of new tools to collect patient experience data.Methods: Eight public contributors represented a range of relevant patient and carer experiences in specialist services with varied levels of PPIE experience, and eleven members of Patient and Participation Groups (PPGs) from two general practices formed our PPIE group. Slide sets were used to trigger discussions and facilitate co-design with patient and carer research participants. Feedback from PPIE contributors alongside verbatim quotes from staff, patient and carer research participants is presented in relation to the themes from the qualitative research data. Results: PPIE insights from four themes: capturing experience data; adopting digital or non-digital tools; ensuring privacy and confidentiality; and co-design of a suite of new tools with guidance, informed joint decisions on the shaping of the tools and how these were implemented. Our PPIE contributors took different roles during co-design and testing of the new tools, which supported co-production of the research study. Conclusions: Our experiences of developing multiple components of PPIE work for this complex study demonstrates the importance of tailoring PPIE to suit different settings, and to maximise individual strengths and capacity. Our study shows the value of bringing diverse experiences together, and a shared approach to managing involvement in co-design. We reflect on how we managed to work together to create a supportive environment when unforeseen challenges emerged (such as, sudden bereavement).
- 20 July 2021
- Published by Springer Nature in Research Square
Open AccessCo-designing New Tools for Collecting, Analysing and Presenting Patient Experience Data in NHS Services: Working in Partnership With Patients and Carers
Background: The way we collect and use patient experience data is vital to optimise the quality and safety of health services. Yet, vulnerable patients and carers may be excluded because of the limited ways data is collected, analysed and presented. In this study, we worked together with researchers, staff, patient and carer participants, and patient and public involvement and engagement (PPIE) contributors to co-design new tools for the collection and use of patient experience data in multiple health settings. This paper outlines how the range of PPIE and research activities enabled the co-design of new tools to collect patient experience data.Methods: Eight public contributors represented a range of relevant patient and carer experiences in specialist services with varied levels of PPIE experience, and eleven members of Patient and Participation Groups (PPGs) from two general practices formed our PPIE group. Slide sets were used to trigger discussions and facilitate co-design with patient and carer research participants. Feedback from PPIE contributors alongside verbatim quotes from staff, patient and carer research participants is presented in relation to the themes from the qualitative research data. Results: PPIE insights from four themes: capturing experience data; adopting digital or non-digital tools; ensuring privacy and confidentiality; and co-design of a suite of new tools with guidance, informed joint decisions on the shaping of the tools and how these were implemented. Our PPIE contributors took different roles during co-design and testing of the new tools, which supported co-production of the research study. Conclusions: Our experiences of developing multiple components of PPIE work for this complex study demonstrates the importance of tailoring PPIE to suit different settings, and to maximise individual strengths and capacity. Our study shows the value of bringing diverse experiences together, and a shared approach to managing involvement in co-design. We reflect on how we managed to work together to create a supportive environment when unforeseen challenges emerged (such as, sudden bereavement).
- Open Access
Protocol for a non-randomised feasibility study evaluating a codesigned patient safety guide in primary care
Introduction Patients and carers should be active partners in patient safety with healthcare professionals and be empowered to use personalised approaches to identify safety concerns and work together to prevent them. This protocol paper details a study to examine the feasibility of a multicomponent intervention to involve patients and/or carers in patient safety in primary care in the UK. Methods and analysis This is a two-phase, non-randomised feasibility mixed methods pragmatic study of a patient safety guide for primary care (PSG-PC). 8 general practices will recruit 120 patient and/or carer participants. All patient and/or carer participants will receive the PSG-PC. It will examine the feasibility and acceptability of the PSG-PC in primary care settings in patients aged 18 years or older who attend appointments at general practice with health professionals four or more times per year as either patients or carers. It will identify secondary outcomes for improving patient safety, health status and patient empowerment, and reducing health service utilisation over 6 months between baseline and 6-month follow-ups. The findings will inform whether a main effectiveness trial is feasible and, if so, how it should be designed, and how many patients and practices will be needed. The study will be undertaken between January 2020 and September 2021. Ethics and dissemination Ethical approval was obtained from the National Health Service London-West London and Gene Therapy Advisory Committee Research Ethics Committee (reference: 19/LO/1289). Research findings will be disseminated with participating general practices and shared in a range of different ways to engage different audiences, including presenting at international and national conferences, publishing in open-access, peer-reviewed journals and facilitating dissemination workshops within local communities with patients, carers and healthcare professionals. Trial registration number ISRCTN90222092.
Open AccessProtocol for a non-randomised feasibility study evaluating a codesigned patient safety guide in primary care
Introduction Patients and carers should be active partners in patient safety with healthcare professionals and be empowered to use personalised approaches to identify safety concerns and work together to prevent them. This protocol paper details a study to examine the feasibility of a multicomponent intervention to involve patients and/or carers in patient safety in primary care in the UK. Methods and analysis This is a two-phase, non-randomised feasibility mixed methods pragmatic study of a patient safety guide for primary care (PSG-PC). 8 general practices will recruit 120 patient and/or carer participants. All patient and/or carer participants will receive the PSG-PC. It will examine the feasibility and acceptability of the PSG-PC in primary care settings in patients aged 18 years or older who attend appointments at general practice with health professionals four or more times per year as either patients or carers. It will identify secondary outcomes for improving patient safety, health status and patient empowerment, and reducing health service utilisation over 6 months between baseline and 6-month follow-ups. The findings will inform whether a main effectiveness trial is feasible and, if so, how it should be designed, and how many patients and practices will be needed. The study will be undertaken between January 2020 and September 2021. Ethics and dissemination Ethical approval was obtained from the National Health Service London-West London and Gene Therapy Advisory Committee Research Ethics Committee (reference: 19/LO/1289). Research findings will be disseminated with participating general practices and shared in a range of different ways to engage different audiences, including presenting at international and national conferences, publishing in open-access, peer-reviewed journals and facilitating dissemination workshops within local communities with patients, carers and healthcare professionals. Trial registration number ISRCTN90222092.
- 1 June 2020
- Published by National Institute for Health and Care Research in Health Services and Delivery Research
- Vol. 8 (28) , 1-128
Digital methods to enhance the usefulness of patient experience data in services for long-term conditions: the DEPEND mixed-methods study
Background: Collecting NHS patient experience data is critical to ensure the delivery of high-quality services. Data are obtained from multiple sources, including service-specific surveys and widely used generic surveys. There are concerns about the timeliness of feedback, that some groups of patients and carers do not give feedback and that free-text feedback may be useful but is difficult to analyse. Objective: To understand how to improve the collection and usefulness of patient experience data in services for people with long-term conditions using digital data capture and improved analysis of comments. Design: The DEPEND study is a mixed-methods study with four parts: qualitative research to explore the perspectives of patients, carers and staff; use of computer science text-analytics methods to analyse comments; co-design of new tools to improve data collection and usefulness; and implementation and process evaluation to assess use of the tools and any impacts. Setting: Services for people with severe mental illness and musculoskeletal conditions at four sites as exemplars to reflect both mental health and physical long-terms conditions: an acute trust (site A), a mental health trust (site B) and two general practices (sites C1 and C2). Participants: A total of 100 staff members with diverse roles in patient experience management, clinical practice and information technology; 59 patients and 21 carers participated in the qualitative research components. Interventions: The tools comprised a digital survey completed using a tablet device (kiosk) or a pen and paper/online version; guidance and information for patients, carers and staff; text-mining programs; reporting templates; and a process for eliciting and recording verbal feedback in community mental health services. Results: We found a lack of understanding and experience of the process of giving feedback. People wanted more meaningful and informal feedback to suit local contexts. Text mining enabled systematic analysis, although challenges remained, and qualitative analysis provided additional insights. All sites managed to collect feedback digitally; however, there was a perceived need for additional resources, and engagement varied. Observation indicated that patients were apprehensive about using kiosks but often would participate with support. The process for collecting and recording verbal feedback in mental health services made sense to participants, but was not successfully adopted, with staff workload and technical problems often highlighted as barriers. Staff thought that new methods were insightful, but observation did not reveal changes in services during the testing period. Conclusions: The use of digital methods can produce some improvements in the collection and usefulness of feedback. Context and flexibility are important, and digital methods need to be complemented with alternative methods. Text mining can provide useful analysis for reporting on large data sets within large organisations, but qualitative analysis may be more useful for small data sets and in small organisations. Limitations: New practices need time and support to be adopted and this study had limited resources and a limited testing time. Future work: Further research is needed to improve text-analysis methods for routine use in services and to evaluate the impact of methods (digital and non-digital) on service improvement in varied contexts and among diverse patients and carers. Funding: This project was funded by the NIHR Health Services and Delivery Research programme and will be published in full in Health Services and Delivery Research; Vol. 8, No. 28. See the NIHR Journals Library website for further project information.
- 1 June 2020
- Published by National Institute for Health and Care Research in Health Services and Delivery Research
- Vol. 8 (28) , 1-128
Digital methods to enhance the usefulness of patient experience data in services for long-term conditions: the DEPEND mixed-methods study
Background: Collecting NHS patient experience data is critical to ensure the delivery of high-quality services. Data are obtained from multiple sources, including service-specific surveys and widely used generic surveys. There are concerns about the timeliness of feedback, that some groups of patients and carers do not give feedback and that free-text feedback may be useful but is difficult to analyse. Objective: To understand how to improve the collection and usefulness of patient experience data in services for people with long-term conditions using digital data capture and improved analysis of comments. Design: The DEPEND study is a mixed-methods study with four parts: qualitative research to explore the perspectives of patients, carers and staff; use of computer science text-analytics methods to analyse comments; co-design of new tools to improve data collection and usefulness; and implementation and process evaluation to assess use of the tools and any impacts. Setting: Services for people with severe mental illness and musculoskeletal conditions at four sites as exemplars to reflect both mental health and physical long-terms conditions: an acute trust (site A), a mental health trust (site B) and two general practices (sites C1 and C2). Participants: A total of 100 staff members with diverse roles in patient experience management, clinical practice and information technology; 59 patients and 21 carers participated in the qualitative research components. Interventions: The tools comprised a digital survey completed using a tablet device (kiosk) or a pen and paper/online version; guidance and information for patients, carers and staff; text-mining programs; reporting templates; and a process for eliciting and recording verbal feedback in community mental health services. Results: We found a lack of understanding and experience of the process of giving feedback. People wanted more meaningful and informal feedback to suit local contexts. Text mining enabled systematic analysis, although challenges remained, and qualitative analysis provided additional insights. All sites managed to collect feedback digitally; however, there was a perceived need for additional resources, and engagement varied. Observation indicated that patients were apprehensive about using kiosks but often would participate with support. The process for collecting and recording verbal feedback in mental health services made sense to participants, but was not successfully adopted, with staff workload and technical problems often highlighted as barriers. Staff thought that new methods were insightful, but observation did not reveal changes in services during the testing period. Conclusions: The use of digital methods can produce some improvements in the collection and usefulness of feedback. Context and flexibility are important, and digital methods need to be complemented with alternative methods. Text mining can provide useful analysis for reporting on large data sets within large organisations, but qualitative analysis may be more useful for small data sets and in small organisations. Limitations: New practices need time and support to be adopted and this study had limited resources and a limited testing time. Future work: Further research is needed to improve text-analysis methods for routine use in services and to evaluate the impact of methods (digital and non-digital) on service improvement in varied contexts and among diverse patients and carers. Funding: This project was funded by the NIHR Health Services and Delivery Research programme and will be published in full in Health Services and Delivery Research; Vol. 8, No. 28. See the NIHR Journals Library website for further project information.
- Open Access
- 7 May 2020
- Published by Springer Nature in BMC Health Services Research
- Vol. 20 (1) , 1-16
Implementing a digital patient feedback system: an analysis using normalisation process theory
Background: Patient feedback in the English NHS is now widespread and digital methods are increasingly used. Adoption of digital methods depends on socio-technical and contextual factors, alongside human agency and lived experience. Moreover, the introduction of these methods may be perceived as disruptive of organisational and clinical routines. The focus of this paper is on the implementation of a particular digital feedback intervention that was co-designed with health professionals and patients (the DEPEND study). Methods: The digital feedback intervention was conceptualised as a complex intervention and thus the study focused on the contexts within which it operated, and how the different participants made sense of the intervention and engaged with it (or not). Four health care sites were studied: an acute setting, a mental health setting, and two general practices. Qualitative data was collected through interviews and focus groups with professionals, patients and carers. In total 51 staff, 24 patients and 8 carers were included. Forty-two observations of the use of the digital feedback system were carried out in the four settings. Data analysis was based on modified grounded theory and Normalisation Process Theory (NPT) formed the conceptual framework. Results: Digital feedback made sense to health care staff as it was seen as attractive, fast to complete and easier to analyse. Patients had a range of views depending on their familiarity with the digital world. Patients mentioned barriers such as kiosk not being visible, privacy, lack of digital know-how, technical hitches with the touchscreen. Collective action in maintaining participation again differed between sites because of workload pressure, perceptions of roles and responsibilities; and in the mental health site major organisational change was taking place. For mental health service users, their relationship with staff and their own health status determined their digital use. Conclusion: The potential of digital feedback was recognised but implementation should take local contexts, different patient groups and organisational leadership into account. Patient involvement in change and adaptation of the intervention was important in enhancing the embedding of digital methods in routine feedback. NPT allowed for a in-depth understanding of actions and interactions of both staff and patients.
- 7 May 2020
- Published by Springer Nature in BMC Health Services Research
- Vol. 20 (1) , 1-16
Open AccessImplementing a digital patient feedback system: an analysis using normalisation process theory
Background: Patient feedback in the English NHS is now widespread and digital methods are increasingly used. Adoption of digital methods depends on socio-technical and contextual factors, alongside human agency and lived experience. Moreover, the introduction of these methods may be perceived as disruptive of organisational and clinical routines. The focus of this paper is on the implementation of a particular digital feedback intervention that was co-designed with health professionals and patients (the DEPEND study). Methods: The digital feedback intervention was conceptualised as a complex intervention and thus the study focused on the contexts within which it operated, and how the different participants made sense of the intervention and engaged with it (or not). Four health care sites were studied: an acute setting, a mental health setting, and two general practices. Qualitative data was collected through interviews and focus groups with professionals, patients and carers. In total 51 staff, 24 patients and 8 carers were included. Forty-two observations of the use of the digital feedback system were carried out in the four settings. Data analysis was based on modified grounded theory and Normalisation Process Theory (NPT) formed the conceptual framework. Results: Digital feedback made sense to health care staff as it was seen as attractive, fast to complete and easier to analyse. Patients had a range of views depending on their familiarity with the digital world. Patients mentioned barriers such as kiosk not being visible, privacy, lack of digital know-how, technical hitches with the touchscreen. Collective action in maintaining participation again differed between sites because of workload pressure, perceptions of roles and responsibilities; and in the mental health site major organisational change was taking place. For mental health service users, their relationship with staff and their own health status determined their digital use. Conclusion: The potential of digital feedback was recognised but implementation should take local contexts, different patient groups and organisational leadership into account. Patient involvement in change and adaptation of the intervention was important in enhancing the embedding of digital methods in routine feedback. NPT allowed for a in-depth understanding of actions and interactions of both staff and patients.
- Open Access
- 13 February 2019
- Published by Public Library of Science (PLoS) in PLOS ONE
- Vol. 14 (2) , e0206507
Assessing mental health service user and carer involvement in physical health care planning: The development and validation of a new patient-reported experience measure
People living with serious mental health conditions experience increased morbidity due to physical health issues driven by medication side-effects and lifestyle factors. Coordinated mental and physical healthcare delivered in accordance with a care plan could help to reduce morbidity and mortality in this population. Efforts to develop new models of care are hampered by a lack of validated instruments to accurately assess the extent to which mental health services users and carers are involved in care planning for physical health. To develop a brief and accurate patient-reported experience measure (PREM) capable of assessing involvement in physical health care planning for mental health service users and their carers. We employed psychometric and statistical techniques to refine a bank of candidate questionnaire items, derived from qualitative interviews, into a valid and reliable measure involvement in physical health care planning. We assessed the psychometric performance of the item bank using modern psychometric analyses. We assessed unidimensionality, scalability, fit to the partial credit Rasch model, category threshold ordering, local dependency, differential item functioning, and test-retest reliability. Once purified of poorly performing and erroneous items, we simulated computerized adaptive testing (CAT) with 15, 10 and 5 items using the calibrated item bank. Issues with category threshold ordering, local dependency and differential item functioning were evident for a number of items in the nascent item bank and were resolved by removing problematic items. The final 19 item PREM had excellent fit to the Rasch model fit (x2 = 192.94, df = 1515, P = .02, RMSEA = .03 (95% CI = .01-.04). The 19-item bank had excellent reliability (marginal r = 0.87). The correlation between questionnaire scores at baseline and 2-week follow-up was high (r = .70, P < .01) and 94.9% of assessment pairs were within the Bland Altman limits of agreement. Simulated CAT demonstrated that assessments could be made using as few as 10 items (mean SE = .43). We developed a flexible patient reported outcome measure to quantify service user and carer involvement in physical health care planning. We demonstrate the potential to substantially reduce assessment length whilst maintaining reliability by utilizing CAT.
- 13 February 2019
- Published by Public Library of Science (PLoS) in PLOS ONE
- Vol. 14 (2) , e0206507
Open AccessAssessing mental health service user and carer involvement in physical health care planning: The development and validation of a new patient-reported experience measure
People living with serious mental health conditions experience increased morbidity due to physical health issues driven by medication side-effects and lifestyle factors. Coordinated mental and physical healthcare delivered in accordance with a care plan could help to reduce morbidity and mortality in this population. Efforts to develop new models of care are hampered by a lack of validated instruments to accurately assess the extent to which mental health services users and carers are involved in care planning for physical health. To develop a brief and accurate patient-reported experience measure (PREM) capable of assessing involvement in physical health care planning for mental health service users and their carers. We employed psychometric and statistical techniques to refine a bank of candidate questionnaire items, derived from qualitative interviews, into a valid and reliable measure involvement in physical health care planning. We assessed the psychometric performance of the item bank using modern psychometric analyses. We assessed unidimensionality, scalability, fit to the partial credit Rasch model, category threshold ordering, local dependency, differential item functioning, and test-retest reliability. Once purified of poorly performing and erroneous items, we simulated computerized adaptive testing (CAT) with 15, 10 and 5 items using the calibrated item bank. Issues with category threshold ordering, local dependency and differential item functioning were evident for a number of items in the nascent item bank and were resolved by removing problematic items. The final 19 item PREM had excellent fit to the Rasch model fit (x2 = 192.94, df = 1515, P = .02, RMSEA = .03 (95% CI = .01-.04). The 19-item bank had excellent reliability (marginal r = 0.87). The correlation between questionnaire scores at baseline and 2-week follow-up was high (r = .70, P < .01) and 94.9% of assessment pairs were within the Bland Altman limits of agreement. Simulated CAT demonstrated that assessments could be made using as few as 10 items (mean SE = .43). We developed a flexible patient reported outcome measure to quantify service user and carer involvement in physical health care planning. We demonstrate the potential to substantially reduce assessment length whilst maintaining reliability by utilizing CAT.