Julie Makarski
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Julie Makarski has a total of 23 co-authors. Here are the top co-authors by number of shared publications:
- Melissa C. Brouwers
Department of Oncology, McMaster University, Hamilton, Canada
9 shared publications - Anthony J. Levinson
McMaster University
2 shared publications - Leigh Hayden
Research and Innovation, North York General Hospital, 4001 Leslie Street, Toronto, ON, M2K 1E1, Canada, 1 416 756 6000
2 shared publications - Lisa Durocher-Allen
Program in Evidence-Based Care, Ontario Health (Cancer Care Ontario), 699 Concession Street, Hamilton, ON L8V 1C3, Canada
2 shared publications - Monika Kastner
Research and Innovation, North York General Hospital, Toronto, Ontario, Canada
2 shared publications - Ananda Chatterjee
St. Michael’s Hospital Li Ka Shing Knowledge Institute, 209 Victoria Street, Toronto M5B 1W8, ON, Canada
1 shared publication
Latest Publications
- Open Access
- 15 March 2015
- Published by Springer Nature in Implementation Science
- Vol. 10 (1) , 1-11
The Guideline Implementability Decision Excellence Model (GUIDE-M): a mixed methods approach to create an international resource to advance the practice guideline field
Practice guideline (PG) implementability refers to PG features that promote their use. While there are tools and resources to promote PG implementability, none are based on an evidence-informed and multidisciplinary perspective. Our objectives were to (i) create a comprehensive and evidence-informed model of PG implementability, (ii) seek support for the model from the international PG community, (iii) map existing implementability tools on to the model, (iv) prioritize areas for further investigation, and (v) describe how the model can be used by PG developers, users, and researchers. A mixed methods approach was used. Using our completed realist review of the literature of seven different disciplines as the foundation, an iterative consensus process was used to create the beta version of the model. This was followed by (i) a survey of international stakeholders (guideline developers and users) to gather feedback and to refine the model, (ii) a content analysis comparing the model to existing PG tools, and (iii) a strategy to prioritize areas of the model for further research by members of the research team. The Guideline Implementability for Decision Excellence Model (GUIDE-M) is comprised of 3 core tactics, 7 domains, 9 subdomains, 44 attributes, and 40 subattributes and elements. Feedback on the beta version was received from 248 stakeholders from 34 countries. The model was rated as logical, relevant, and appropriate. Seven PG tools were selected and compared to the GUIDE-M: very few tools targeted the Contextualization and Deliberations domain. Also, fewer of the tools addressed PG appraisal than PG development and reporting functions. These findings informed the research priorities identified by the team. The GUIDE-M provides an evidence-informed international and multidisciplinary conceptualization of PG implementability. The model can be used by PG developers to help them create more implementable recommendations, by clinicians and other users to help them be better consumers of PGs, and by the research community to identify priorities for further investigation.
- 15 March 2015
- Published by Springer Nature in Implementation Science
- Vol. 10 (1) , 1-11
Open AccessThe Guideline Implementability Decision Excellence Model (GUIDE-M): a mixed methods approach to create an international resource to advance the practice guideline field
Practice guideline (PG) implementability refers to PG features that promote their use. While there are tools and resources to promote PG implementability, none are based on an evidence-informed and multidisciplinary perspective. Our objectives were to (i) create a comprehensive and evidence-informed model of PG implementability, (ii) seek support for the model from the international PG community, (iii) map existing implementability tools on to the model, (iv) prioritize areas for further investigation, and (v) describe how the model can be used by PG developers, users, and researchers. A mixed methods approach was used. Using our completed realist review of the literature of seven different disciplines as the foundation, an iterative consensus process was used to create the beta version of the model. This was followed by (i) a survey of international stakeholders (guideline developers and users) to gather feedback and to refine the model, (ii) a content analysis comparing the model to existing PG tools, and (iii) a strategy to prioritize areas of the model for further research by members of the research team. The Guideline Implementability for Decision Excellence Model (GUIDE-M) is comprised of 3 core tactics, 7 domains, 9 subdomains, 44 attributes, and 40 subattributes and elements. Feedback on the beta version was received from 248 stakeholders from 34 countries. The model was rated as logical, relevant, and appropriate. Seven PG tools were selected and compared to the GUIDE-M: very few tools targeted the Contextualization and Deliberations domain. Also, fewer of the tools addressed PG appraisal than PG development and reporting functions. These findings informed the research priorities identified by the team. The GUIDE-M provides an evidence-informed international and multidisciplinary conceptualization of PG implementability. The model can be used by PG developers to help them create more implementable recommendations, by clinicians and other users to help them be better consumers of PGs, and by the research community to identify priorities for further investigation.
- Open Access
- 15 August 2014
- Published by Springer Nature in Implementation Science
- Vol. 9 (1) , 1-3
The AGREE Enterprise: a decade of advancing clinical practice guidelines
The original AGREE (A ppraisal of G uidelines for RE search and E valuation) Instrument was published in 2003, and its revision, the AGREE II, in 2009. Together, they filled an important gap in the guideline and quality of care fields. Ten years later, the AGREE Enterprise reflects on a trajectory of projects and international collaboration that have contributed to advancing the science and quality of practice guidelines and the uptake of AGREE/AGREE II. The AGREE Enterprise has undertaken activities to improve the tool and to develop resources to support its use. Since 2003, the uptake and adoption of AGREE by the international community has been swift and broad. A total of 33 language translations of the original AGREE Instrument and the current AGREE II are available and were initiated by the international community. A recent scan of the published literature identified over 600 articles that referenced the AGREE tools. The AGREE tools have been widely received and applied, with several organizations having incorporated the AGREE as part of their formal practice guideline programs. Since its redevelopment in 2010, the AGREE Enterprise website ( www.agreetrust.org ) continues to experience steady increases in visitors per month and currently has over 10,000 registered users. The AGREE Enterprise has contributed to the advancements of guidelines through research activities and international participation by scientific and user communities. As we enter a new decade, we look forward to ongoing collaborations and contributing to further advancements to improve quality of care and health care systems.
- 15 August 2014
- Published by Springer Nature in Implementation Science
- Vol. 9 (1) , 1-3
Open AccessThe AGREE Enterprise: a decade of advancing clinical practice guidelines
The original AGREE (A ppraisal of G uidelines for RE search and E valuation) Instrument was published in 2003, and its revision, the AGREE II, in 2009. Together, they filled an important gap in the guideline and quality of care fields. Ten years later, the AGREE Enterprise reflects on a trajectory of projects and international collaboration that have contributed to advancing the science and quality of practice guidelines and the uptake of AGREE/AGREE II. The AGREE Enterprise has undertaken activities to improve the tool and to develop resources to support its use. Since 2003, the uptake and adoption of AGREE by the international community has been swift and broad. A total of 33 language translations of the original AGREE Instrument and the current AGREE II are available and were initiated by the international community. A recent scan of the published literature identified over 600 articles that referenced the AGREE tools. The AGREE tools have been widely received and applied, with several organizations having incorporated the AGREE as part of their formal practice guideline programs. Since its redevelopment in 2010, the AGREE Enterprise website ( www.agreetrust.org ) continues to experience steady increases in visitors per month and currently has over 10,000 registered users. The AGREE Enterprise has contributed to the advancements of guidelines through research activities and international participation by scientific and user communities. As we enter a new decade, we look forward to ongoing collaborations and contributing to further advancements to improve quality of care and health care systems.
- Open Access
- 22 March 2014
- Published by Springer Nature in Implementation Science
- Vol. 9 (1) , 36
A mixed methods approach to understand variation in lung cancer practice and the role of guidelines
Practice pattern data demonstrate regional variation and lower than expected rates of adherence to practice guideline (PG) recommendations for the treatment of stage II/IIIA resected and stage IIIA/IIIB unresected non-small cell lung cancer (NSCLC) patients in Ontario, Canada. This study sought to understand how clinical decisions are made for the treatment of these patients and the role of PGs.
- 22 March 2014
- Published by Springer Nature in Implementation Science
- Vol. 9 (1) , 36
Open AccessA mixed methods approach to understand variation in lung cancer practice and the role of guidelines
Practice pattern data demonstrate regional variation and lower than expected rates of adherence to practice guideline (PG) recommendations for the treatment of stage II/IIIA resected and stage IIIA/IIIB unresected non-small cell lung cancer (NSCLC) patients in Ontario, Canada. This study sought to understand how clinical decisions are made for the treatment of these patients and the role of PGs.
- Open Access
- 12 September 2013
- Published by Springer Nature in BMC Medical Research Methodology
- Vol. 13 (1) , 112
Making sense of complex data: a mapping process for analyzing findings of a realist review on guideline implementability
Realist reviews offer a rigorous method to analyze heterogeneous data emerging from multiple disciplines as a means to develop new concepts, understand the relationships between them, and identify the evidentiary base underpinning them. However, emerging synthesis methods such as the Realist Review are not well operationalized and may be difficult for the novice researcher to grasp. The objective of this paper is to describe the development of an analytic process to organize and synthesize data from a realist review.
- 12 September 2013
- Published by Springer Nature in BMC Medical Research Methodology
- Vol. 13 (1) , 112
Open AccessMaking sense of complex data: a mapping process for analyzing findings of a realist review on guideline implementability
Realist reviews offer a rigorous method to analyze heterogeneous data emerging from multiple disciplines as a means to develop new concepts, understand the relationships between them, and identify the evidentiary base underpinning them. However, emerging synthesis methods such as the Realist Review are not well operationalized and may be difficult for the novice researcher to grasp. The objective of this paper is to describe the development of an analytic process to organize and synthesize data from a realist review.
- 1 December 2012
- Published by American Society of Clinical Oncology (ASCO) in Journal of Clinical Oncology
- Vol. 30 (34) , 190
Does a quality-of-care problem exist? Cancer Care Ontario practice pattern data and the recommendations of two lung cancer practice guidelines.
190 Background: Practice-pattern data, evidence-based knowledge and transfer, and performance management strategies define Cancer Care Ontario’s quality improvement strategy. Knowledge products, such as practice guidelines, are intended to provide recommendations for practice, based on best available evidence, to improve quality of care and reduce variation in practice. Review of 2010-2011 Cancer System Quality Index (CSQI) data revealed complex practice patterns in treatment of non-small cell lung cancer patients with stages II and IIIa resected and stages IIIa and IIIb non-resected disease in Ontario. A multi-method study was initiated to understand the patterns, to identify if a quality of care problem exists and to propose improvements moving forward. Methods: Surgeons, medical oncologists and radiation oncologists from Ontario were invited to participate in a survey consisting of 6 areas of inquiry. A grounded theory approach was used to guide key informant interviews of purposively sampled clinicians and administrators. A more in-depth analysis of the CSQI data was planned. Results: Clinicians responding to survey provided positive assessments of PG recommendations and evidentiary base; perceptions of practice patterns were less problematic than hypothesized; estimates of benchmarks were highly variable; and assessments of barriers to recommendation implementation included slow referral process, lack of organization support and patients seen in practice not reflected in the evidence. From the interviews, 5 themes emerged: unique patient, unique physician, family, clinical team, and clinical evidence. Further analysis of CSQI data was not possible given limitations related to data collection. Conclusions: A perceived quality of care problem initiated this study. Concerns centred on significant proportion of patients receiving no treatment; modest percentage of patients receiving treatment that aligned with PG recommendations; and regional variation within each of the clinical care options. Our data show that defining a quality of care problem is significantly more complex than consideration of practice patterns alone.
- 1 December 2012
- Published by American Society of Clinical Oncology (ASCO) in Journal of Clinical Oncology
- Vol. 30 (34) , 190
Does a quality-of-care problem exist? Cancer Care Ontario practice pattern data and the recommendations of two lung cancer practice guidelines.
190 Background: Practice-pattern data, evidence-based knowledge and transfer, and performance management strategies define Cancer Care Ontario’s quality improvement strategy. Knowledge products, such as practice guidelines, are intended to provide recommendations for practice, based on best available evidence, to improve quality of care and reduce variation in practice. Review of 2010-2011 Cancer System Quality Index (CSQI) data revealed complex practice patterns in treatment of non-small cell lung cancer patients with stages II and IIIa resected and stages IIIa and IIIb non-resected disease in Ontario. A multi-method study was initiated to understand the patterns, to identify if a quality of care problem exists and to propose improvements moving forward. Methods: Surgeons, medical oncologists and radiation oncologists from Ontario were invited to participate in a survey consisting of 6 areas of inquiry. A grounded theory approach was used to guide key informant interviews of purposively sampled clinicians and administrators. A more in-depth analysis of the CSQI data was planned. Results: Clinicians responding to survey provided positive assessments of PG recommendations and evidentiary base; perceptions of practice patterns were less problematic than hypothesized; estimates of benchmarks were highly variable; and assessments of barriers to recommendation implementation included slow referral process, lack of organization support and patients seen in practice not reflected in the evidence. From the interviews, 5 themes emerged: unique patient, unique physician, family, clinical team, and clinical evidence. Further analysis of CSQI data was not possible given limitations related to data collection. Conclusions: A perceived quality of care problem initiated this study. Concerns centred on significant proportion of patients receiving no treatment; modest percentage of patients receiving treatment that aligned with PG recommendations; and regional variation within each of the clinical care options. Our data show that defining a quality of care problem is significantly more complex than consideration of practice patterns alone.
- Open Access
- 20 December 2011
- Published by Springer Nature in Implementation Science
- Vol. 6 (1) , 130
The landscape of knowledge translation interventions in cancer control: What do we know and where to next? A review of systematic reviews
Effective implementation strategies are needed to optimize advancements in the fields of cancer diagnosis, treatment, survivorship, and end-of-life care. We conducted a review of systematic reviews to better understand the evidentiary base of implementation strategies in cancer control.
- 20 December 2011
- Published by Springer Nature in Implementation Science
- Vol. 6 (1) , 130
Open AccessThe landscape of knowledge translation interventions in cancer control: What do we know and where to next? A review of systematic reviews
Effective implementation strategies are needed to optimize advancements in the fields of cancer diagnosis, treatment, survivorship, and end-of-life care. We conducted a review of systematic reviews to better understand the evidentiary base of implementation strategies in cancer control.
- Open Access
- 26 July 2011
- Published by Springer Nature in Implementation Science
- Vol. 6 (1) , 81
E-learning interventions are comparable to user's manual in a randomized trial of training strategies for the AGREE II
Practice guidelines (PGs) are systematically developed statements intended to assist in patient and practitioner decisions. The AGREE II is the revised tool for PG development, reporting, and evaluation, comprised of 23 items, two global rating scores, and a new User's Manual. In this study, we sought to develop, execute, and evaluate the impact of two internet interventions designed to accelerate the capacity of stakeholders to use the AGREE II.
- 26 July 2011
- Published by Springer Nature in Implementation Science
- Vol. 6 (1) , 81
Open AccessE-learning interventions are comparable to user's manual in a randomized trial of training strategies for the AGREE II
Practice guidelines (PGs) are systematically developed statements intended to assist in patient and practitioner decisions. The AGREE II is the revised tool for PG development, reporting, and evaluation, comprised of 23 items, two global rating scores, and a new User's Manual. In this study, we sought to develop, execute, and evaluate the impact of two internet interventions designed to accelerate the capacity of stakeholders to use the AGREE II.
- Open Access
- 13 May 2010
- Published by Springer Nature in BMC Health Services Research
- Vol. 10 (1) , 123
Recruitment of multiple stakeholders to health services research: Lessons from the front lines
Background: Self-administered surveys are an essential methodological tool for health services and knowledge translation research, and engaging end-users of the research is critical. However, few documented accounts of the efforts invested in recruitment of multiple different stakeholders to one health services research study exist. Here, we highlight the challenges of recruiting key stakeholders (policy-makers, clinicians, guideline developers) to a Canadian Institutes of Health Research (CIHR) funded health services research (HSR) study aimed to develop an updated and refined version of a guideline appraisal tool, the AGREE. Methods: Using evidence-based methods of recruitment, our goal was to recruit 192 individuals: 80 international guideline developers, 80 Canadian clinicians and 32 Canadian policy/decision-makers. We calculated the participation rate and the recruitment efficiency. Results: We mailed 873 invitation letters. Of 838 approached, our participation rate was 29%(240) and recruitment efficiency, 19%(156). One policy-maker manager did not allow policy staff to participate in the study. Conclusions: Based on the results from this study, we suggest that future studies aiming to engage similar stakeholders in HSR over sample by at least 5 times to achieve their target sample size and allow for participant withdrawals. We need continued efforts to communicate the value of research between researchers and end-users of research (policy-makers, clinicians, and other researchers), integration of participatory research strategies, and promotion of the value of end-user involvement in research. Future research to understand methods of improving recruitment efficiency and engaging key stakeholders in HSR is warranted.
- 13 May 2010
- Published by Springer Nature in BMC Health Services Research
- Vol. 10 (1) , 123
Open AccessRecruitment of multiple stakeholders to health services research: Lessons from the front lines
Background: Self-administered surveys are an essential methodological tool for health services and knowledge translation research, and engaging end-users of the research is critical. However, few documented accounts of the efforts invested in recruitment of multiple different stakeholders to one health services research study exist. Here, we highlight the challenges of recruiting key stakeholders (policy-makers, clinicians, guideline developers) to a Canadian Institutes of Health Research (CIHR) funded health services research (HSR) study aimed to develop an updated and refined version of a guideline appraisal tool, the AGREE. Methods: Using evidence-based methods of recruitment, our goal was to recruit 192 individuals: 80 international guideline developers, 80 Canadian clinicians and 32 Canadian policy/decision-makers. We calculated the participation rate and the recruitment efficiency. Results: We mailed 873 invitation letters. Of 838 approached, our participation rate was 29%(240) and recruitment efficiency, 19%(156). One policy-maker manager did not allow policy staff to participate in the study. Conclusions: Based on the results from this study, we suggest that future studies aiming to engage similar stakeholders in HSR over sample by at least 5 times to achieve their target sample size and allow for participant withdrawals. We need continued efforts to communicate the value of research between researchers and end-users of research (policy-makers, clinicians, and other researchers), integration of participatory research strategies, and promotion of the value of end-user involvement in research. Future research to understand methods of improving recruitment efficiency and engaging key stakeholders in HSR is warranted.
- Open Access
- 19 April 2010
- Published by Springer Nature in Implementation Science
- Vol. 5 (1) , 29
A randomized trial to evaluate e-learning interventions designed to improve learner's performance, satisfaction, and self-efficacy with the AGREE II
Practice guidelines (PGs) are systematically developed statements intended to assist in patient, practitioner, and policy decisions. The AGREE II is the revised and updated standard tool for guideline development, reporting and evaluation. It is comprised of 23 items and a user's Manual. The AGREE II is ready for use.
- 19 April 2010
- Published by Springer Nature in Implementation Science
- Vol. 5 (1) , 29
Open AccessA randomized trial to evaluate e-learning interventions designed to improve learner's performance, satisfaction, and self-efficacy with the AGREE II
Practice guidelines (PGs) are systematically developed statements intended to assist in patient, practitioner, and policy decisions. The AGREE II is the revised and updated standard tool for guideline development, reporting and evaluation. It is comprised of 23 items and a user's Manual. The AGREE II is ready for use.