Paul Holyoke
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Paul Holyoke has a total of 58 co-authors. Here are the top co-authors by number of shared publications:
- AnneMarie Levy
Recreation and Leisure Studies, Brock University , St Catharines, Ontario
4 shared publications - Josephine McMurray
Lazaridis School of Business & Economics/Community Health, Wilfrid Laurier University, Waterloo, ON, Canada
4 shared publications - Justine Lauren Giosa
SE Research Centre, SE Health, Markham, Ontario, Canada
4 shared publications Anna Neely
2 shared publications- Caylee Raber
Emily Carr University of Art + Design, Vancouver, BC, Canada
2 shared publications - Celina Carter
SE Research Centre
2 shared publications
Latest Publications
- Open Access
- 18 September 2026
- Published by Specialty Publications in Journal of Participatory Research Methods
- Vol. 7 (5)
Learnings From a Developmental Evaluation of a Dynamic, Arts-Based Death and Grief Awareness Initiative: The Reflection Room Project
Linear approaches to project design and evaluation work well when problems are understood and the solutions to the problem are defined within specific boundaries. If projects contain some degree of uncertainty in the goals, outcomes, or context, evaluations based on standardization, consistency, and concreteness can hinder development and innovation. This paper outlines our process of engaging in a developmental evaluation as a means of cyclical evaluation, adaptation, and development through the multiple phases of the Reflection Room project; a participatory arts-based storytelling installation designed to support participants’ comfort with dying, death, and grief. This paper details our iterative evaluation process, which included increasing levels of participation from communities in co-designing the conceptualization of the innovation, designing the evaluation, and rapid real-time testing. We highlight our non-linear processes, emergent learning, and approaches to collaborating with experts-by-experience. Recommendations for others wanting to engage in developmental evaluation are shared, including engaging experts-by-experience early on and facilitating increasing levels of participation and decision-making role throughout the evaluation. We also recommend grounding the process within a theoretical framework; evaluating the stability, scale, or spread of the project; and adapting the evaluation and design to support organic variations in participants, setting, and emergent questions. Overall, the greatest value we found in conducting a developmental evaluation was in the flexibility and iterative nature of the methodology, which enabled us to efficiently pivot project goals, processes and outcomes to respond to the diverse and dynamic needs and welfare of Reflection Room participants.
- 18 September 2026
- Published by Specialty Publications in Journal of Participatory Research Methods
- Vol. 7 (5)
Open AccessLearnings From a Developmental Evaluation of a Dynamic, Arts-Based Death and Grief Awareness Initiative: The Reflection Room Project
Linear approaches to project design and evaluation work well when problems are understood and the solutions to the problem are defined within specific boundaries. If projects contain some degree of uncertainty in the goals, outcomes, or context, evaluations based on standardization, consistency, and concreteness can hinder development and innovation. This paper outlines our process of engaging in a developmental evaluation as a means of cyclical evaluation, adaptation, and development through the multiple phases of the Reflection Room project; a participatory arts-based storytelling installation designed to support participants’ comfort with dying, death, and grief. This paper details our iterative evaluation process, which included increasing levels of participation from communities in co-designing the conceptualization of the innovation, designing the evaluation, and rapid real-time testing. We highlight our non-linear processes, emergent learning, and approaches to collaborating with experts-by-experience. Recommendations for others wanting to engage in developmental evaluation are shared, including engaging experts-by-experience early on and facilitating increasing levels of participation and decision-making role throughout the evaluation. We also recommend grounding the process within a theoretical framework; evaluating the stability, scale, or spread of the project; and adapting the evaluation and design to support organic variations in participants, setting, and emergent questions. Overall, the greatest value we found in conducting a developmental evaluation was in the flexibility and iterative nature of the methodology, which enabled us to efficiently pivot project goals, processes and outcomes to respond to the diverse and dynamic needs and welfare of Reflection Room participants.
- Open Access
- 11 September 2026
- Published by Ubiquity Press, Ltd. in International Journal of Integrated Care
- Vol. 26 (3) , 19
Evaluation of a Complex Implementation to Shape, Spread, Scale, and Sustain a New Dementia Care Resource for Better, More Integrated Care
Introduction: Researchers collaborated with experts-by-experience to create ‘Our Dementia Journey Journal’ (ODJJ), an interactive resource designed to enhance integrated care through communication and relationship building between caregivers and care providers. This study investigated how best to scale and spread the resource.Methods: ODJJ was implemented as a pilot in three communities with seven sites. A mixed-methods multi-phase sequential design with embedded formative and outcome evaluations was used, with experts-by-experience integrated throughout.Results: ODJJ was implemented serially in long-term care homes, First Nation communities, and South Asian communities. Implementation approaches and tools evolved through sharing experiences and tactics, with greatest uptake and use in the later sites.Discussion: The study confirmed that ODJJ can strengthen communication and relationship building, but how it was perceived, accepted and used for integrated care differed. The cumulative results generated a spread and scale (marketing) strategy, with messaging, materials and supports to facilitate uptake and use across Canada.Conclusion: Pilot implementation for integrated care resources with a strong, complex evaluation can lead to a detailed plan for spread and scale.
- 11 September 2026
- Published by Ubiquity Press, Ltd. in International Journal of Integrated Care
- Vol. 26 (3) , 19
Open AccessEvaluation of a Complex Implementation to Shape, Spread, Scale, and Sustain a New Dementia Care Resource for Better, More Integrated Care
Introduction: Researchers collaborated with experts-by-experience to create ‘Our Dementia Journey Journal’ (ODJJ), an interactive resource designed to enhance integrated care through communication and relationship building between caregivers and care providers. This study investigated how best to scale and spread the resource.Methods: ODJJ was implemented as a pilot in three communities with seven sites. A mixed-methods multi-phase sequential design with embedded formative and outcome evaluations was used, with experts-by-experience integrated throughout.Results: ODJJ was implemented serially in long-term care homes, First Nation communities, and South Asian communities. Implementation approaches and tools evolved through sharing experiences and tactics, with greatest uptake and use in the later sites.Discussion: The study confirmed that ODJJ can strengthen communication and relationship building, but how it was perceived, accepted and used for integrated care differed. The cumulative results generated a spread and scale (marketing) strategy, with messaging, materials and supports to facilitate uptake and use across Canada.Conclusion: Pilot implementation for integrated care resources with a strong, complex evaluation can lead to a detailed plan for spread and scale.
- Open Access
- 19 March 2026
- Published by Ubiquity Press, Ltd. in International Journal of Integrated Care
- Vol. 26 (1) , 12
Examining the Influence of Integrated Home and Community Care Programs on Quadruple Aim and Health Equity Outcomes Across the Health Care System: A Scoping Review
Introduction: Integrated approaches have been promoted as a solution to challenges in delivering high-quality care for aging populations. Despite their theorized benefits, the system-wide influence of integrated home and community care models remains poorly understood. Methods: We systematically searched MEDLINE (Ovid), CINAHL (EBSCO) and AgeLine (EBSCO) for studies examining the influence of integrated home and community care programs on outcomes aligned with the Quadruple Aim and Health Equity framework. Results: Forty-seven peer-reviewed articles were included, representing five program types: comprehensive coordinated care (n = 6), integrated palliative care (n = 23), preventative care (n = 4), restorative care (n = 3), and transitional care (n = 11). Most studies focused on value outcomes, particularly in hospital (n = 36) and emergency services (n = 22). Patient and caregiver experience was assessed in 11 studies, while provider experience (n = 1), health equity (n = 2) and population health (n = 0) were rarely examined. Eleven studies evaluated system-level influence; 23 focused on community-dwelling care recipients. Discussion: The emphasis on cost-related outcomes reflects prevailing austerity and value-based rhetoric, which may shape research funding, priorities and attention. Conclusion: Future research should adopt a balanced approach, incorporating all Quadruple Aim and Health Equity domains. A learning health system model, linking data, knowledge and practice, is recommended to enable continuous improvement and resource alignment.
- 19 March 2026
- Published by Ubiquity Press, Ltd. in International Journal of Integrated Care
- Vol. 26 (1) , 12
Open AccessExamining the Influence of Integrated Home and Community Care Programs on Quadruple Aim and Health Equity Outcomes Across the Health Care System: A Scoping Review
Introduction: Integrated approaches have been promoted as a solution to challenges in delivering high-quality care for aging populations. Despite their theorized benefits, the system-wide influence of integrated home and community care models remains poorly understood. Methods: We systematically searched MEDLINE (Ovid), CINAHL (EBSCO) and AgeLine (EBSCO) for studies examining the influence of integrated home and community care programs on outcomes aligned with the Quadruple Aim and Health Equity framework. Results: Forty-seven peer-reviewed articles were included, representing five program types: comprehensive coordinated care (n = 6), integrated palliative care (n = 23), preventative care (n = 4), restorative care (n = 3), and transitional care (n = 11). Most studies focused on value outcomes, particularly in hospital (n = 36) and emergency services (n = 22). Patient and caregiver experience was assessed in 11 studies, while provider experience (n = 1), health equity (n = 2) and population health (n = 0) were rarely examined. Eleven studies evaluated system-level influence; 23 focused on community-dwelling care recipients. Discussion: The emphasis on cost-related outcomes reflects prevailing austerity and value-based rhetoric, which may shape research funding, priorities and attention. Conclusion: Future research should adopt a balanced approach, incorporating all Quadruple Aim and Health Equity domains. A learning health system model, linking data, knowledge and practice, is recommended to enable continuous improvement and resource alignment.
- Open Access
- 9 December 2025
- Published by Springer Nature in Research Involvement and Engagement
- Vol. 12 (1)
Developing and applying the ‘Participatory Research to Action Framework’ for meaningful engagement of experts-by-experience in applied health services research: a case study approach
While many frameworks have been developed to guide participatory research, they are often tailored to a sector, health condition, research stage, or resear
- 9 December 2025
- Published by Springer Nature in Research Involvement and Engagement
- Vol. 12 (1)
Open AccessDeveloping and applying the ‘Participatory Research to Action Framework’ for meaningful engagement of experts-by-experience in applied health services research: a case study approach
While many frameworks have been developed to guide participatory research, they are often tailored to a sector, health condition, research stage, or resear
- Open Access
- 1 May 2025
- Published by SAGE Publications in Digital Health
- Vol. 11
Qualitative evaluation by community health professionals of a technology design for recording and mobilizing patient reported care narratives
Introduction: Gaps in the health system make it difficult for older adults to have their communication needs and wants heard for that necessary communication to happen. In this study, we investigate attitudes of community-based healthcare professionals (HCPs) regarding the potential utility of a digital technology platform. This platform was designed to enable older adults receiving HCP services in a community setting to provide qualitative narrative impressions and evaluations of their needs and how these needs were addressed by the services. MyHealthMyRecord (MHMR) technology is being designed to provide a digital video-scrapbook for recording and sharing healthcare-related information, observation, and qualitative narratives. Methods: HCPs were recruited in collaboration with SE Health, a large community-care provider based in Toronto, Canada. Using a user-centered and inclusive design approach, HCPs in community care were asked to comment on the use of the MHMR platform by their patients and to examine a platform prototype. Each session, which lasted approximately 60 min, involved a 40-min semistructured face-to-face interview, followed by a demonstration of the system. Data were analyzed using the Consensual Qualitative Research analysis. Results: A convenience sample of 7 HCPs, all comfortable with digital technology, participated in the study. The analysis identified 11 themes, with all participants recognizing the value of the MHMR platform in enhancing their capacity to provide services, particularly in tracking to relevant events between patients’ visits. Conclusions: The HCPs appreciated how the MHMR platform could be used to enhance community health service delivery through valid, reliable, and responsive Person/Patient-Reported Outcomes. It was appreciated that such data could be useful in improving community care experiences while also being useful at a system-wide learning level.
- 1 May 2025
- Published by SAGE Publications in Digital Health
- Vol. 11
Open AccessQualitative evaluation by community health professionals of a technology design for recording and mobilizing patient reported care narratives
Introduction: Gaps in the health system make it difficult for older adults to have their communication needs and wants heard for that necessary communication to happen. In this study, we investigate attitudes of community-based healthcare professionals (HCPs) regarding the potential utility of a digital technology platform. This platform was designed to enable older adults receiving HCP services in a community setting to provide qualitative narrative impressions and evaluations of their needs and how these needs were addressed by the services. MyHealthMyRecord (MHMR) technology is being designed to provide a digital video-scrapbook for recording and sharing healthcare-related information, observation, and qualitative narratives. Methods: HCPs were recruited in collaboration with SE Health, a large community-care provider based in Toronto, Canada. Using a user-centered and inclusive design approach, HCPs in community care were asked to comment on the use of the MHMR platform by their patients and to examine a platform prototype. Each session, which lasted approximately 60 min, involved a 40-min semistructured face-to-face interview, followed by a demonstration of the system. Data were analyzed using the Consensual Qualitative Research analysis. Results: A convenience sample of 7 HCPs, all comfortable with digital technology, participated in the study. The analysis identified 11 themes, with all participants recognizing the value of the MHMR platform in enhancing their capacity to provide services, particularly in tracking to relevant events between patients’ visits. Conclusions: The HCPs appreciated how the MHMR platform could be used to enhance community health service delivery through valid, reliable, and responsive Person/Patient-Reported Outcomes. It was appreciated that such data could be useful in improving community care experiences while also being useful at a system-wide learning level.
- Open Access
- 17 July 2024
- Published by JMIR Publications Inc. in JMIR Research Protocols
- Vol. 13 (1) , e58318
Recovery and Renewal of Co-Design Approaches in Health: Protocol for a Realist Synthesis
Background: The COVID-19 pandemic significantly transformed the landscape of work and collaboration, impacting design research methodologies and techniques. Co-design approaches have been both negatively and positively affected by the pandemic, prompting a need to investigate and understand the extent of these impacts, changes, and adaptations, specifically in the health sector. Despite the challenges that the pandemic imposed on conducting co-design and related projects, it also encouraged a re-evaluation of co-design practices, leading to innovative solutions and techniques. Designers and researchers have explored alternative ways to engage stakeholders and end users, leveraging digital workshops and participatory digital platforms. These adaptations have the potential to enhance inclusivity, allowing for a wider range of individuals to contribute their perspectives and insights through co-design and thus contribute to healthcare change. Objective: This study aims to explore the impacts of the pandemic on co-design and related practices, focusing on co-design practices in healthcare that have been gained, adapted, or enhanced, with a specific focus on issues of equity, diversity, and inclusion. Methods: The study uses a realist synthesis methodology to identify and analyze the effects of the pandemic on co-design approaches in health, drawing on a range of sources including first-person experiences, gray literature, and academic literature. A community of practice in co-design in health will be engaged to support this process. Results: By examining the experiences and insights of professionals, practitioners, and communities who were actively involved in co-design and have navigated the challenges and opportunities of the pandemic, we can gain a deeper understanding of the strategies, tools, and techniques that have facilitated effective co-design during the pandemic, contributing to building resilience and capacity in co-design in health beyond the pandemic. Conclusions: By involving community partners, community of practice (research), and design practitioners, we expect closer proximity to practice with capacity building occurring through the realist process, thus enabling rapid adoption and refinement of new techniques or insights that emerge. Ultimately, this research will contribute to the advancement of co-design methodologies and inform the future of co-design in health. International Registered Report Identifier (IRRID): DERR1-10.2196/58318
- 17 July 2024
- Published by JMIR Publications Inc. in JMIR Research Protocols
- Vol. 13 (1) , e58318
Open AccessRecovery and Renewal of Co-Design Approaches in Health: Protocol for a Realist Synthesis
Background: The COVID-19 pandemic significantly transformed the landscape of work and collaboration, impacting design research methodologies and techniques. Co-design approaches have been both negatively and positively affected by the pandemic, prompting a need to investigate and understand the extent of these impacts, changes, and adaptations, specifically in the health sector. Despite the challenges that the pandemic imposed on conducting co-design and related projects, it also encouraged a re-evaluation of co-design practices, leading to innovative solutions and techniques. Designers and researchers have explored alternative ways to engage stakeholders and end users, leveraging digital workshops and participatory digital platforms. These adaptations have the potential to enhance inclusivity, allowing for a wider range of individuals to contribute their perspectives and insights through co-design and thus contribute to healthcare change. Objective: This study aims to explore the impacts of the pandemic on co-design and related practices, focusing on co-design practices in healthcare that have been gained, adapted, or enhanced, with a specific focus on issues of equity, diversity, and inclusion. Methods: The study uses a realist synthesis methodology to identify and analyze the effects of the pandemic on co-design approaches in health, drawing on a range of sources including first-person experiences, gray literature, and academic literature. A community of practice in co-design in health will be engaged to support this process. Results: By examining the experiences and insights of professionals, practitioners, and communities who were actively involved in co-design and have navigated the challenges and opportunities of the pandemic, we can gain a deeper understanding of the strategies, tools, and techniques that have facilitated effective co-design during the pandemic, contributing to building resilience and capacity in co-design in health beyond the pandemic. Conclusions: By involving community partners, community of practice (research), and design practitioners, we expect closer proximity to practice with capacity building occurring through the realist process, thus enabling rapid adoption and refinement of new techniques or insights that emerge. Ultimately, this research will contribute to the advancement of co-design methodologies and inform the future of co-design in health. International Registered Report Identifier (IRRID): DERR1-10.2196/58318
- Open Access
- 19 April 2024
- Published by JMIR Publications Inc. in Journal of Medical Internet Research
- Vol. 26, e56883
Psychometric Evaluation of a Tablet-Based Tool to Detect Mild Cognitive Impairment in Older Adults: Mixed Methods Study
Background: With the rapid aging of the global population, the prevalence of mild cognitive impairment (MCI) and dementia is anticipated to surge worldwide. MCI serves as an intermediary stage between normal aging and dementia, necessitating more sensitive and effective screening tools for early identification and intervention. The BrainFx SCREEN is a novel digital tool designed to assess cognitive impairment. This study evaluated its efficacy as a screening tool for MCI in primary care settings, particularly in the context of an aging population and the growing integration of digital health solutions. Objective: The primary objective was to assess the validity, reliability, and applicability of the BrainFx SCREEN (hereafter, the SCREEN) for MCI screening in a primary care context. We conducted an exploratory study comparing the SCREEN with an established screening tool, the Quick Mild Cognitive Impairment (Qmci) screen. Methods: A concurrent mixed methods, prospective study using a quasi-experimental design was conducted with 147 participants from 5 primary care Family Health Teams (FHTs; characterized by multidisciplinary practice and capitated funding) across southwestern Ontario, Canada. Participants included health care practitioners, patients, and FHT administrative executives. Individuals aged ≥55 years with no history of MCI or diagnosis of dementia rostered in a participating FHT were eligible to participate. Participants were screened using both the SCREEN and Qmci. The study also incorporated the Geriatric Anxiety Scale–10 to assess general anxiety levels at each cognitive screening. The SCREEN’s scoring was compared against that of the Qmci and the clinical judgment of health care professionals. Statistical analyses included sensitivity, specificity, internal consistency, and test-retest reliability assessments. Results: The study found that the SCREEN’s longer administration time and complex scoring algorithm, which is proprietary and unavailable for independent analysis, presented challenges. Its internal consistency, indicated by a Cronbach α of 0.63, was below the acceptable threshold. The test-retest reliability also showed limitations, with moderate intraclass correlation coefficient (0.54) and inadequate κ (0.15) values. Sensitivity and specificity were consistent (63.25% and 74.07%, respectively) between cross-tabulation and discrepant analysis. In addition, the study faced limitations due to its demographic skew (96/147, 65.3% female, well-educated participants), the absence of a comprehensive gold standard for MCI diagnosis, and financial constraints limiting the inclusion of confirmatory neuropsychological testing. Conclusions: The SCREEN, in its current form, does not meet the necessary criteria for an optimal MCI screening tool in primary care settings, primarily due to its longer administration time and lower reliability. As the number of digital health technologies increases and evolves, further testing and refinement of tools such as the SCREEN are essential to ensure their efficacy and reliability in real-world clinical settings. This study advocates for continued research in this rapidly advancing field to better serve the aging population. International Registered Report Identifier (IRRID): RR2-10.2196/25520
- 19 April 2024
- Published by JMIR Publications Inc. in Journal of Medical Internet Research
- Vol. 26, e56883
Open AccessPsychometric Evaluation of a Tablet-Based Tool to Detect Mild Cognitive Impairment in Older Adults: Mixed Methods Study
Background: With the rapid aging of the global population, the prevalence of mild cognitive impairment (MCI) and dementia is anticipated to surge worldwide. MCI serves as an intermediary stage between normal aging and dementia, necessitating more sensitive and effective screening tools for early identification and intervention. The BrainFx SCREEN is a novel digital tool designed to assess cognitive impairment. This study evaluated its efficacy as a screening tool for MCI in primary care settings, particularly in the context of an aging population and the growing integration of digital health solutions. Objective: The primary objective was to assess the validity, reliability, and applicability of the BrainFx SCREEN (hereafter, the SCREEN) for MCI screening in a primary care context. We conducted an exploratory study comparing the SCREEN with an established screening tool, the Quick Mild Cognitive Impairment (Qmci) screen. Methods: A concurrent mixed methods, prospective study using a quasi-experimental design was conducted with 147 participants from 5 primary care Family Health Teams (FHTs; characterized by multidisciplinary practice and capitated funding) across southwestern Ontario, Canada. Participants included health care practitioners, patients, and FHT administrative executives. Individuals aged ≥55 years with no history of MCI or diagnosis of dementia rostered in a participating FHT were eligible to participate. Participants were screened using both the SCREEN and Qmci. The study also incorporated the Geriatric Anxiety Scale–10 to assess general anxiety levels at each cognitive screening. The SCREEN’s scoring was compared against that of the Qmci and the clinical judgment of health care professionals. Statistical analyses included sensitivity, specificity, internal consistency, and test-retest reliability assessments. Results: The study found that the SCREEN’s longer administration time and complex scoring algorithm, which is proprietary and unavailable for independent analysis, presented challenges. Its internal consistency, indicated by a Cronbach α of 0.63, was below the acceptable threshold. The test-retest reliability also showed limitations, with moderate intraclass correlation coefficient (0.54) and inadequate κ (0.15) values. Sensitivity and specificity were consistent (63.25% and 74.07%, respectively) between cross-tabulation and discrepant analysis. In addition, the study faced limitations due to its demographic skew (96/147, 65.3% female, well-educated participants), the absence of a comprehensive gold standard for MCI diagnosis, and financial constraints limiting the inclusion of confirmatory neuropsychological testing. Conclusions: The SCREEN, in its current form, does not meet the necessary criteria for an optimal MCI screening tool in primary care settings, primarily due to its longer administration time and lower reliability. As the number of digital health technologies increases and evolves, further testing and refinement of tools such as the SCREEN are essential to ensure their efficacy and reliability in real-world clinical settings. This study advocates for continued research in this rapidly advancing field to better serve the aging population. International Registered Report Identifier (IRRID): RR2-10.2196/25520
- Open Access
- 12 March 2024
- Published by JMIR Publications Inc. in JMIR Preprints
Recovery and Renewal of Co-Design Approaches in Health: Protocol for a Realist Synthesis (Preprint)
BACKGROUND The COVID-19 pandemic significantly transformed the landscape of work and collaboration, impacting design research methodologies and techniques. Co-design approaches have been both negatively and positively affected by the pandemic, prompting a need to investigate and understand the extent of these impacts, changes, and adaptations, specifically in the health sector. Despite the challenges that the pandemic imposed on conducting co-design and related projects, it also encouraged a re-evaluation of co-design practices, leading to innovative solutions and techniques. Designers and researchers have explored alternative ways to engage stakeholders and end users, leveraging digital workshops and participatory digital platforms. These adaptations have the potential to enhance inclusivity, allowing for a wider range of individuals to contribute their perspectives and insights through co-design and thus contribute to healthcare change. OBJECTIVE This study aims to explore the impacts of the pandemic on co-design and related practices, focusing on co-design practices in healthcare that have been gained, adapted, or enhanced, with a specific focus on issues of equity, diversity, and inclusion. METHODS The study uses a realist synthesis methodology to identify and analyze the effects of the pandemic on co-design approaches in health, drawing on a range of sources including first-person experiences, gray literature, and academic literature. A community of practice in co-design in health will be engaged to support this process. RESULTS By examining the experiences and insights of professionals, practitioners, and communities who were actively involved in co-design and have navigated the challenges and opportunities of the pandemic, we can gain a deeper understanding of the strategies, tools, and techniques that have facilitated effective co-design during the pandemic, contributing to building resilience and capacity in co-design in health beyond the pandemic. CONCLUSIONS By involving community partners, community of practice (research), and design practitioners, we expect closer proximity to practice with capacity building occurring through the realist process, thus enabling rapid adoption and refinement of new techniques or insights that emerge. Ultimately, this research will contribute to the advancement of co-design methodologies and inform the future of co-design in health. INTERNATIONAL REGISTERED REPORT DERR1-10.2196/58318
- 12 March 2024
- Published by JMIR Publications Inc. in JMIR Preprints
Open AccessRecovery and Renewal of Co-Design Approaches in Health: Protocol for a Realist Synthesis (Preprint)
BACKGROUND The COVID-19 pandemic significantly transformed the landscape of work and collaboration, impacting design research methodologies and techniques. Co-design approaches have been both negatively and positively affected by the pandemic, prompting a need to investigate and understand the extent of these impacts, changes, and adaptations, specifically in the health sector. Despite the challenges that the pandemic imposed on conducting co-design and related projects, it also encouraged a re-evaluation of co-design practices, leading to innovative solutions and techniques. Designers and researchers have explored alternative ways to engage stakeholders and end users, leveraging digital workshops and participatory digital platforms. These adaptations have the potential to enhance inclusivity, allowing for a wider range of individuals to contribute their perspectives and insights through co-design and thus contribute to healthcare change. OBJECTIVE This study aims to explore the impacts of the pandemic on co-design and related practices, focusing on co-design practices in healthcare that have been gained, adapted, or enhanced, with a specific focus on issues of equity, diversity, and inclusion. METHODS The study uses a realist synthesis methodology to identify and analyze the effects of the pandemic on co-design approaches in health, drawing on a range of sources including first-person experiences, gray literature, and academic literature. A community of practice in co-design in health will be engaged to support this process. RESULTS By examining the experiences and insights of professionals, practitioners, and communities who were actively involved in co-design and have navigated the challenges and opportunities of the pandemic, we can gain a deeper understanding of the strategies, tools, and techniques that have facilitated effective co-design during the pandemic, contributing to building resilience and capacity in co-design in health beyond the pandemic. CONCLUSIONS By involving community partners, community of practice (research), and design practitioners, we expect closer proximity to practice with capacity building occurring through the realist process, thus enabling rapid adoption and refinement of new techniques or insights that emerge. Ultimately, this research will contribute to the advancement of co-design methodologies and inform the future of co-design in health. INTERNATIONAL REGISTERED REPORT DERR1-10.2196/58318
- Open Access
- 29 January 2024
- Published by JMIR Publications Inc. in JMIR Preprints
Psychometric Evaluation of a Tablet-Based Tool to Detect Mild Cognitive Impairment in Older Adults: Mixed Methods Study (Preprint)
BACKGROUND With the rapid aging of the global population, the prevalence of mild cognitive impairment (MCI) and dementia is anticipated to surge worldwide. MCI serves as an intermediary stage between normal aging and dementia, necessitating more sensitive and effective screening tools for early identification and intervention. The BrainFx SCREEN is a novel digital tool designed to assess cognitive impairment. This study evaluated its efficacy as a screening tool for MCI in primary care settings, particularly in the context of an aging population and the growing integration of digital health solutions. OBJECTIVE The primary objective was to assess the validity, reliability, and applicability of the BrainFx SCREEN (hereafter, the SCREEN) for MCI screening in a primary care context. We conducted an exploratory study comparing the SCREEN with an established screening tool, the Quick Mild Cognitive Impairment (Qmci) screen. METHODS A concurrent mixed methods, prospective study using a quasi-experimental design was conducted with 147 participants from 5 primary care Family Health Teams (FHTs; characterized by multidisciplinary practice and capitated funding) across southwestern Ontario, Canada. Participants included health care practitioners, patients, and FHT administrative executives. Individuals aged ≥55 years with no history of MCI or diagnosis of dementia rostered in a participating FHT were eligible to participate. Participants were screened using both the SCREEN and Qmci. The study also incorporated the Geriatric Anxiety Scale–10 to assess general anxiety levels at each cognitive screening. The SCREEN’s scoring was compared against that of the Qmci and the clinical judgment of health care professionals. Statistical analyses included sensitivity, specificity, internal consistency, and test-retest reliability assessments. RESULTS The study found that the SCREEN’s longer administration time and complex scoring algorithm, which is proprietary and unavailable for independent analysis, presented challenges. Its internal consistency, indicated by a Cronbach α of 0.63, was below the acceptable threshold. The test-retest reliability also showed limitations, with moderate intraclass correlation coefficient (0.54) and inadequate κ (0.15) values. Sensitivity and specificity were consistent (63.25% and 74.07%, respectively) between cross-tabulation and discrepant analysis. In addition, the study faced limitations due to its demographic skew (96/147, 65.3% female, well-educated participants), the absence of a comprehensive gold standard for MCI diagnosis, and financial constraints limiting the inclusion of confirmatory neuropsychological testing. CONCLUSIONS The SCREEN, in its current form, does not meet the necessary criteria for an optimal MCI screening tool in primary care settings, primarily due to its longer administration time and lower reliability. As the number of digital health technologies increases and evolves, further testing and refinement of tools such as the SCREEN are essential to ensure their efficacy and reliability in real-world clinical settings. This study advocates for continued research in this rapidly advancing field to better serve the aging population. CLINICALTRIAL INTERNATIONAL REGISTERED REPORT RR2-10.2196/25520
- 29 January 2024
- Published by JMIR Publications Inc. in JMIR Preprints
Open AccessPsychometric Evaluation of a Tablet-Based Tool to Detect Mild Cognitive Impairment in Older Adults: Mixed Methods Study (Preprint)
BACKGROUND With the rapid aging of the global population, the prevalence of mild cognitive impairment (MCI) and dementia is anticipated to surge worldwide. MCI serves as an intermediary stage between normal aging and dementia, necessitating more sensitive and effective screening tools for early identification and intervention. The BrainFx SCREEN is a novel digital tool designed to assess cognitive impairment. This study evaluated its efficacy as a screening tool for MCI in primary care settings, particularly in the context of an aging population and the growing integration of digital health solutions. OBJECTIVE The primary objective was to assess the validity, reliability, and applicability of the BrainFx SCREEN (hereafter, the SCREEN) for MCI screening in a primary care context. We conducted an exploratory study comparing the SCREEN with an established screening tool, the Quick Mild Cognitive Impairment (Qmci) screen. METHODS A concurrent mixed methods, prospective study using a quasi-experimental design was conducted with 147 participants from 5 primary care Family Health Teams (FHTs; characterized by multidisciplinary practice and capitated funding) across southwestern Ontario, Canada. Participants included health care practitioners, patients, and FHT administrative executives. Individuals aged ≥55 years with no history of MCI or diagnosis of dementia rostered in a participating FHT were eligible to participate. Participants were screened using both the SCREEN and Qmci. The study also incorporated the Geriatric Anxiety Scale–10 to assess general anxiety levels at each cognitive screening. The SCREEN’s scoring was compared against that of the Qmci and the clinical judgment of health care professionals. Statistical analyses included sensitivity, specificity, internal consistency, and test-retest reliability assessments. RESULTS The study found that the SCREEN’s longer administration time and complex scoring algorithm, which is proprietary and unavailable for independent analysis, presented challenges. Its internal consistency, indicated by a Cronbach α of 0.63, was below the acceptable threshold. The test-retest reliability also showed limitations, with moderate intraclass correlation coefficient (0.54) and inadequate κ (0.15) values. Sensitivity and specificity were consistent (63.25% and 74.07%, respectively) between cross-tabulation and discrepant analysis. In addition, the study faced limitations due to its demographic skew (96/147, 65.3% female, well-educated participants), the absence of a comprehensive gold standard for MCI diagnosis, and financial constraints limiting the inclusion of confirmatory neuropsychological testing. CONCLUSIONS The SCREEN, in its current form, does not meet the necessary criteria for an optimal MCI screening tool in primary care settings, primarily due to its longer administration time and lower reliability. As the number of digital health technologies increases and evolves, further testing and refinement of tools such as the SCREEN are essential to ensure their efficacy and reliability in real-world clinical settings. This study advocates for continued research in this rapidly advancing field to better serve the aging population. CLINICALTRIAL INTERNATIONAL REGISTERED REPORT RR2-10.2196/25520
- Open Access
- 30 November 2023
- Published by Springer Nature in BMC Health Services Research
- Vol. 23 (1) , 1-11
Resource Utilization Groups in transitional home care: validating the RUG-III/HC case-mix system in hospital-to-home care programs
Transitional hospital-to-home care programs support safe and timely transition from acute care settings back into the community. Case-mix systems that classify transitional care clients into groups based on their resource utilization can assist with care planning, calculating reimbursement rates in bundled care funding models, and predicting health human resource needs. This study evaluated the fit and relevance of the Resource Utilization Groups version III for Home Care (RUG-III/HC) case-mix classification system in transitional care programs in Ontario, Canada. We conducted a retrospective analysis of clinical assessment data and administrative billing records from a cohort of clients (n = 1,680 care episodes) in transitional home care programs in Ontario. We classified care episodes into established RUG-III/HC groups based on clients’ clinical and functional characteristics and calculated four case-mix indices to describe care relative resource utilization in the study sample. Using these indices in linear regression models, we evaluated the degree to which the RUG-III/HC system can be used to predict care resource utilization. A majority of transitional home care clients are classified as being Clinically complex (41.6%) and having Reduced physical functions (37.8%). The RUG-III/HC groups that account for the largest share of clients are those with the lowest hierarchical ranking, indicating low Activities of Daily Living limitations but a range of Instrumental Activities of Daily Living limitations. There is notable heterogeneity in the distribution of clients in RUG-III/HC groups across transitional care programs. The case-mix indices reflect decreasing hierarchical resource use within but not across RUG-III/HC categories. The RUG-III/HC predicts 23.34% of the variance in resource utilization of combined paid and unpaid care time. The distribution of clients across RUG-III/HC groups in transitional home care programs is remarkably different from clients in long-stay home care settings. Transitional care programs have a higher proportion of Clinically complex clients and a lower proportion of clients with Reduced physical function. This study contributes to the development of a case-mix system for clients in transitional home care programs which can be used by care managers to inform planning, costing, and resource allocation in these programs.
- 30 November 2023
- Published by Springer Nature in BMC Health Services Research
- Vol. 23 (1) , 1-11
Open AccessResource Utilization Groups in transitional home care: validating the RUG-III/HC case-mix system in hospital-to-home care programs
Transitional hospital-to-home care programs support safe and timely transition from acute care settings back into the community. Case-mix systems that classify transitional care clients into groups based on their resource utilization can assist with care planning, calculating reimbursement rates in bundled care funding models, and predicting health human resource needs. This study evaluated the fit and relevance of the Resource Utilization Groups version III for Home Care (RUG-III/HC) case-mix classification system in transitional care programs in Ontario, Canada. We conducted a retrospective analysis of clinical assessment data and administrative billing records from a cohort of clients (n = 1,680 care episodes) in transitional home care programs in Ontario. We classified care episodes into established RUG-III/HC groups based on clients’ clinical and functional characteristics and calculated four case-mix indices to describe care relative resource utilization in the study sample. Using these indices in linear regression models, we evaluated the degree to which the RUG-III/HC system can be used to predict care resource utilization. A majority of transitional home care clients are classified as being Clinically complex (41.6%) and having Reduced physical functions (37.8%). The RUG-III/HC groups that account for the largest share of clients are those with the lowest hierarchical ranking, indicating low Activities of Daily Living limitations but a range of Instrumental Activities of Daily Living limitations. There is notable heterogeneity in the distribution of clients in RUG-III/HC groups across transitional care programs. The case-mix indices reflect decreasing hierarchical resource use within but not across RUG-III/HC categories. The RUG-III/HC predicts 23.34% of the variance in resource utilization of combined paid and unpaid care time. The distribution of clients across RUG-III/HC groups in transitional home care programs is remarkably different from clients in long-stay home care settings. Transitional care programs have a higher proportion of Clinically complex clients and a lower proportion of clients with Reduced physical function. This study contributes to the development of a case-mix system for clients in transitional home care programs which can be used by care managers to inform planning, costing, and resource allocation in these programs.