Li Jiu
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Li Jiu has a total of 37 co-authors. Here are the top co-authors by number of shared publications:
Aukje K. Mantel-Teeuwisse
Division of Pharmacoepidemiology and Clinical Pharmacology, Utrecht Institute for Pharmaceutical Sciences, Utrecht University, Utrecht, The Netherlands
4 shared publicationsJunfeng Wang
Division of Pharmacoepidemiology and Clinical Pharmacology, Utrecht Institute for Pharmaceutical Sciences, Utrecht University, Utrecht, The Netherlands
4 shared publications- Wim G. Goettsch
Universiteit Utrecht
4 shared publications Francisco Javier Somolinos-Simón
Bioengineering and Telemedicine Group, Centro de Tecnología Biomédica, ETSI de Telecomunicación, Universidad Politécnica de Madrid, Madrid, Spain
2 shared publicationsGema García-Sáez
Bioengineering and Telemedicine Group, Centro de Tecnología Biomédica, ETSI de Telecomunicación, Universidad Politécnica de Madrid, Madrid, Spain
2 shared publicationsJose Tapia-Galisteo
Bioengineering and Telemedicine Group, Centro de Tecnología Biomédica, ETSI de Telecomunicación, Universidad Politécnica de Madrid, Madrid, Spain
2 shared publications
Latest Publications
- Open Access
- 5 November 2024
- Published by Cambridge University Press (CUP) in International Journal of Technology Assessment in Health Care
- Vol. 40 (1) , e49
Roadmap to Innovation of HTA Methods (IHTAM): insights from three case studies of quantitative methods
Objectives: A conceptual framework, called Innovation of Health Technology Assessment Methods (IHTAM), has been developed to facilitate the understanding of how to innovate methods of health technology assessment (HTA). However, the framework applicability has not been evaluated in practice. Hence, we aimed to explore framework applicability in three cases of method innovation that are part of the HTx project and to develop a roadmap to improve framework applicability. Methods: The IHTAM framework was applied to three cases of innovating HTA methods. We collected feedback from case study leaders and consortium members after a training session, an approximately 1-year follow-up of periodic case study meetings, and a general assembly meeting where innovation progresses of the three cases were reported through surveys and interviews. Feedback was then summarized using an open-coding technique. Results: According to feedback, the framework provided a structured way of deliberation and helped to improve collaboration among HTA stakeholders. However, framework applicability could be improved if it was complemented by a roadmap with a loop structure to provide tailored guidance for different cases, and with items to elaborate actions to be taken by stakeholders. Accordingly, a 48-item roadmap was developed. Conclusions: The IHTAM framework was generally applicable to the three case studies. A roadmap, with loop structure and actionable items, could complement the framework, and may provide HTA stakeholders with tailored guidance on developing new methods. To further examine the framework applicability, we recommend stakeholders to apply the IHTAM framework and its roadmap in future practice.
- 5 November 2024
- Published by Cambridge University Press (CUP) in International Journal of Technology Assessment in Health Care
- Vol. 40 (1) , e49
Open AccessRoadmap to Innovation of HTA Methods (IHTAM): insights from three case studies of quantitative methods
Objectives: A conceptual framework, called Innovation of Health Technology Assessment Methods (IHTAM), has been developed to facilitate the understanding of how to innovate methods of health technology assessment (HTA). However, the framework applicability has not been evaluated in practice. Hence, we aimed to explore framework applicability in three cases of method innovation that are part of the HTx project and to develop a roadmap to improve framework applicability. Methods: The IHTAM framework was applied to three cases of innovating HTA methods. We collected feedback from case study leaders and consortium members after a training session, an approximately 1-year follow-up of periodic case study meetings, and a general assembly meeting where innovation progresses of the three cases were reported through surveys and interviews. Feedback was then summarized using an open-coding technique. Results: According to feedback, the framework provided a structured way of deliberation and helped to improve collaboration among HTA stakeholders. However, framework applicability could be improved if it was complemented by a roadmap with a loop structure to provide tailored guidance for different cases, and with items to elaborate actions to be taken by stakeholders. Accordingly, a 48-item roadmap was developed. Conclusions: The IHTAM framework was generally applicable to the three case studies. A roadmap, with loop structure and actionable items, could complement the framework, and may provide HTA stakeholders with tailored guidance on developing new methods. To further examine the framework applicability, we recommend stakeholders to apply the IHTAM framework and its roadmap in future practice.
- Open Access
- 4 November 2024
- Published by Cambridge University Press (CUP) in International Journal of Technology Assessment in Health Care
- Vol. 40 (1) , e44
Implementing a sandbox approach in health technology assessment: benefits and recommendations
Objectives: The sandbox approach, developed in the financial technologies sector, creates an environment to collaboratively develop and test innovative new products, methods and regulatory approaches, separated from business as usual. It has been used in health care to encourage innovation in response to emerging challenges, but, until recently, has not been used in health technology assessment (HTA). This article summarizes our learnings from using the sandbox approach to address three challenges facing HTA organizations and to identify implications for the use of this approach in HTA. Methods: We identified three challenging contemporary HTA-related topics to explore in a sandbox environment, away from the pressures and interests of “live” assessments. We convened a pool of 120 stakeholders and experts to participate in various sandbox activities and ultimately co-develop solutions to help HTA organizations respond to the identified challenges. Results: Important general learnings about the potential benefits and implementation of a sandbox approach in HTA were identified. Consequently, we developed recommendations to guide its use, including how to implement an HTA sandbox in an effective way and the types of challenges for which it may be best suited. Conclusions: For many HTA organizations, it is difficult to carefully consider emerging challenges and innovate their processes due to risks associated with decision errors and resource limitations. The sandbox approach could reduce these barriers. The potential benefits of addressing HTA challenges in a collaborative “safe space” are considerable.
- 4 November 2024
- Published by Cambridge University Press (CUP) in International Journal of Technology Assessment in Health Care
- Vol. 40 (1) , e44
Open AccessImplementing a sandbox approach in health technology assessment: benefits and recommendations
Objectives: The sandbox approach, developed in the financial technologies sector, creates an environment to collaboratively develop and test innovative new products, methods and regulatory approaches, separated from business as usual. It has been used in health care to encourage innovation in response to emerging challenges, but, until recently, has not been used in health technology assessment (HTA). This article summarizes our learnings from using the sandbox approach to address three challenges facing HTA organizations and to identify implications for the use of this approach in HTA. Methods: We identified three challenging contemporary HTA-related topics to explore in a sandbox environment, away from the pressures and interests of “live” assessments. We convened a pool of 120 stakeholders and experts to participate in various sandbox activities and ultimately co-develop solutions to help HTA organizations respond to the identified challenges. Results: Important general learnings about the potential benefits and implementation of a sandbox approach in HTA were identified. Consequently, we developed recommendations to guide its use, including how to implement an HTA sandbox in an effective way and the types of challenges for which it may be best suited. Conclusions: For many HTA organizations, it is difficult to carefully consider emerging challenges and innovate their processes due to risks associated with decision errors and resource limitations. The sandbox approach could reduce these barriers. The potential benefits of addressing HTA challenges in a collaborative “safe space” are considerable.
- Open Access
Tools for assessing quality of studies investigating health interventions using real-world data: a literature review and content analysis
Objectives: We aimed to identify existing appraisal tools for non-randomised studies of interventions (NRSIs) and to compare the criteria that the tools provide at the quality-item level.Design: Literature review through three approaches: systematic search of journal articles, snowballing search of reviews on appraisal tools and grey literature search on websites of health technology assessment (HTA) agencies.Data sources: Systematic search: Medline; Snowballing: starting from three articles (D’Andreaet al, Quigleyet aland Fariaet al); Grey literature: websites of European HTA agencies listed by the International Network of Agencies for Health Technology Assessment. Appraisal tools were searched through April 2022.Eligibility criteria for selecting studies: We included a tool, if it addressed quality concerns of NRSIs and was published in English (unless from grey literature). A tool was excluded, if it was only for diagnostic, prognostic, qualitative or secondary studies.Data extraction and synthesis: Two independent researchers searched, screened and reviewed all included studies and tools, summarised quality items and scored whether and to what extent a quality item was described by a tool, for either methodological quality or reporting.Results: Forty-nine tools met inclusion criteria and were included for the content analysis. Concerns regarding the quality of NRSI were categorised into 4 domains and 26 items. The Research Triangle Institute Item Bank (RTI Item Bank) and STrengthening the Reporting of OBservational studies in Epidemiology (STROBE) were the most comprehensive tools for methodological quality and reporting, respectively, as they addressed (n=20; 17) and sufficiently described (n=18; 13) the highest number of items. However, none of the tools covered all items.Conclusion: Most of the tools have their own strengths, but none of them could address all quality concerns relevant to NRSIs. Even the most comprehensive tools can be complemented by several items. We suggest decision-makers, researchers and tool developers consider the quality-item level heterogeneity, when selecting a tool or identifying a research gap.OSF registration number: OSF registration DOI (https://doi.org/10.17605/OSF.IO/KCSGX).
Open AccessTools for assessing quality of studies investigating health interventions using real-world data: a literature review and content analysis
Objectives: We aimed to identify existing appraisal tools for non-randomised studies of interventions (NRSIs) and to compare the criteria that the tools provide at the quality-item level.Design: Literature review through three approaches: systematic search of journal articles, snowballing search of reviews on appraisal tools and grey literature search on websites of health technology assessment (HTA) agencies.Data sources: Systematic search: Medline; Snowballing: starting from three articles (D’Andreaet al, Quigleyet aland Fariaet al); Grey literature: websites of European HTA agencies listed by the International Network of Agencies for Health Technology Assessment. Appraisal tools were searched through April 2022.Eligibility criteria for selecting studies: We included a tool, if it addressed quality concerns of NRSIs and was published in English (unless from grey literature). A tool was excluded, if it was only for diagnostic, prognostic, qualitative or secondary studies.Data extraction and synthesis: Two independent researchers searched, screened and reviewed all included studies and tools, summarised quality items and scored whether and to what extent a quality item was described by a tool, for either methodological quality or reporting.Results: Forty-nine tools met inclusion criteria and were included for the content analysis. Concerns regarding the quality of NRSI were categorised into 4 domains and 26 items. The Research Triangle Institute Item Bank (RTI Item Bank) and STrengthening the Reporting of OBservational studies in Epidemiology (STROBE) were the most comprehensive tools for methodological quality and reporting, respectively, as they addressed (n=20; 17) and sufficiently described (n=18; 13) the highest number of items. However, none of the tools covered all items.Conclusion: Most of the tools have their own strengths, but none of them could address all quality concerns relevant to NRSIs. Even the most comprehensive tools can be complemented by several items. We suggest decision-makers, researchers and tool developers consider the quality-item level heterogeneity, when selecting a tool or identifying a research gap.OSF registration number: OSF registration DOI (https://doi.org/10.17605/OSF.IO/KCSGX).
- Open Access
- 10 February 2024
- Published by Elsevier in Diabetes Research and Clinical Practice
- Vol. 209, 111574
A literature review of quality assessment and applicability to HTA of risk prediction models of coronary heart disease in patients with diabetes
- 10 February 2024
- Published by Elsevier in Diabetes Research and Clinical Practice
- Vol. 209, 111574
Open AccessA literature review of quality assessment and applicability to HTA of risk prediction models of coronary heart disease in patients with diabetes
- Open Access
- 9 January 2023
- Published by Springer Nature in Research Square
Methodological Quality of Retrospective Observational Studies Investigating Effects of Diabetes Monitoring Systems: a Systematic Review
Background Retrospective observational studies (ROSs) have been frequently used to investigate treatment effects of diabetes monitoring systems (DMS), i.e. medical devices to monitor blood glucose. However, due to quality concerns, the findings of such studies were often questioned by clinical, regulatory, or health technology assessment decision-makers. We aimed to conduct a systematic review to assess the methodological quality of ROSs investigating DMS effects, and to explore the trend in quality change over time. Methods Embase, PubMed, Web of Science, and Scopus were systematically searched for English-language articles published from January 2012 to March 2021. Randomized controlled trials or other prospective studies were manually excluded. The ROBINS-I (Risk Of Bias In Non-randomized Studies – of Interventions) was used for assessing RoB. To investigate the quality change over time, we divided the study into three subgroups according to publication year, and compared the proportion of studies with the same quality level among the three subgroups. Results We identified 4926 articles, of which 72 were eligible for inclusion. Twenty-six studies were published before 2018, 22 in 2018 or 2019, and 24 after 2019. The overall methodological quality was quite low, as 61 (85%) studies were graded as facing critical or serious RoB. Also, the overall methodological quality did not substantially improve over time. The major contributors to low quality included confounding, missing data, and selection of the reported results. Conclusions The retrospective observational studies investigating DMS effects generally had a high risk of bias, and this did not substantially improve in the past ten years. Thus, clinical, regulatory, or HTA decision-makers may need strategies to effectively exploit these suboptimal studies. Also, to further improve study quality, extra efforts may be needed, such as guiding the tool selection regarding quality improvement in the tools.
- 9 January 2023
- Published by Springer Nature in Research Square
Open AccessMethodological Quality of Retrospective Observational Studies Investigating Effects of Diabetes Monitoring Systems: a Systematic Review
Background Retrospective observational studies (ROSs) have been frequently used to investigate treatment effects of diabetes monitoring systems (DMS), i.e. medical devices to monitor blood glucose. However, due to quality concerns, the findings of such studies were often questioned by clinical, regulatory, or health technology assessment decision-makers. We aimed to conduct a systematic review to assess the methodological quality of ROSs investigating DMS effects, and to explore the trend in quality change over time. Methods Embase, PubMed, Web of Science, and Scopus were systematically searched for English-language articles published from January 2012 to March 2021. Randomized controlled trials or other prospective studies were manually excluded. The ROBINS-I (Risk Of Bias In Non-randomized Studies – of Interventions) was used for assessing RoB. To investigate the quality change over time, we divided the study into three subgroups according to publication year, and compared the proportion of studies with the same quality level among the three subgroups. Results We identified 4926 articles, of which 72 were eligible for inclusion. Twenty-six studies were published before 2018, 22 in 2018 or 2019, and 24 after 2019. The overall methodological quality was quite low, as 61 (85%) studies were graded as facing critical or serious RoB. Also, the overall methodological quality did not substantially improve over time. The major contributors to low quality included confounding, missing data, and selection of the reported results. Conclusions The retrospective observational studies investigating DMS effects generally had a high risk of bias, and this did not substantially improve in the past ten years. Thus, clinical, regulatory, or HTA decision-makers may need strategies to effectively exploit these suboptimal studies. Also, to further improve study quality, extra efforts may be needed, such as guiding the tool selection regarding quality improvement in the tools.
- Open Access
- 31 January 2022
- Published by Cambridge University Press (CUP) in International Journal of Technology Assessment in Health Care
- Vol. 38 (1) , e16
Understanding innovation of health technology assessment methods: the IHTAM framework
Adequate methods are urgently needed to guarantee the good practice of health technology assessment (HTA) for technologies with novel properties. The aim of the study was to construct a conceptual framework to help understand the innovation of HTA methods (IHTAM). The construction of the IHTAM framework was based on two scoping reviews, one on the current practice of innovating methods, that is existing HTA frameworks, and one on theoretical foundations for innovating methods outside the HTA discipline. Both aimed to identify and synthesize concepts of innovation (i.e., innovation processes and roles of stakeholders in innovation). Using these concepts, the framework was developed in iterative brainstorming sessions and subsequent discussions with representatives from various stakeholder groups. The framework was constructed based on twenty documents on innovating HTA frameworks and fourteen guidelines from three scientific disciplines. It includes a generic innovation process consisting of three phases (“Identification,” “Development,” and “Implementation”) and nine subphases. In the framework, three roles that HTA stakeholders can play in innovation (“Developers,” “Practitioners,” and “Beneficiaries”) are defined, and a process on how the stakeholders innovate HTA methods is included. The IHTAM framework visualizes systematically which elements and stakeholders are important to the development and implementation of novel HTA methods. The framework could be used by all stakeholders involved in HTA innovation to learn how to engage dynamically and collaborate effectively throughout the innovation process. HTA stakeholders in practice have welcomed the framework, though additional testing of its applicability and acceptance is essential.
- 31 January 2022
- Published by Cambridge University Press (CUP) in International Journal of Technology Assessment in Health Care
- Vol. 38 (1) , e16
Open AccessUnderstanding innovation of health technology assessment methods: the IHTAM framework
Adequate methods are urgently needed to guarantee the good practice of health technology assessment (HTA) for technologies with novel properties. The aim of the study was to construct a conceptual framework to help understand the innovation of HTA methods (IHTAM). The construction of the IHTAM framework was based on two scoping reviews, one on the current practice of innovating methods, that is existing HTA frameworks, and one on theoretical foundations for innovating methods outside the HTA discipline. Both aimed to identify and synthesize concepts of innovation (i.e., innovation processes and roles of stakeholders in innovation). Using these concepts, the framework was developed in iterative brainstorming sessions and subsequent discussions with representatives from various stakeholder groups. The framework was constructed based on twenty documents on innovating HTA frameworks and fourteen guidelines from three scientific disciplines. It includes a generic innovation process consisting of three phases (“Identification,” “Development,” and “Implementation”) and nine subphases. In the framework, three roles that HTA stakeholders can play in innovation (“Developers,” “Practitioners,” and “Beneficiaries”) are defined, and a process on how the stakeholders innovate HTA methods is included. The IHTAM framework visualizes systematically which elements and stakeholders are important to the development and implementation of novel HTA methods. The framework could be used by all stakeholders involved in HTA innovation to learn how to engage dynamically and collaborate effectively throughout the innovation process. HTA stakeholders in practice have welcomed the framework, though additional testing of its applicability and acceptance is essential.