Development of key performance indicators for a telemedicine setting in Egypt using an electronic modified Delphi approach
Open Access
- 1 July 2025
- journal article
- research article
- Published by Springer Nature in BMC Health Services Research
- Vol. 25 (1) , 1-12
Abstract
Background: Performance measurement is crucial for maintaining quality of care in health systems, and so telemedicine is no exception. Key Performance Indicators (KPIs) are some of the primary tools for performance measurement. While there are many global studies on telemedicine quality, studies that are done specifically on indicators are limited. Based on this, the purpose of this study is to develop KPIs for evaluating the performance of telemedicine settings. Methods: A consensus-based methodological study employing the Delphi method was conducted. The study was conducted in Ain Shams Virtual Hospital (AVH). It started with defining the selection criteria to be used in Delphi method. Then a review of scientific sources was carried out to enlist the potential indicators of telemedicine. An electronic modified Delphi technique was then used. The panel members were invited to rate the indicators on a 10-point Likert scale, and subsequently, the experts then reconsidered their previous voting in the light of the aggregated results. The agreement was reached as the indicator showed a 70% level of agreement. Cohen Kappa (K) was calculated to show the percentage of agreement between the two rounds. Results: The multi-voting process established three selection criteria for indicators: clarity, feasibility, and importance. A review of research sources led to an initial list of 52 potential indicators, categorized into operations, clinical services, and customer satisfaction. The Delphi method was carried out in two rounds, with expert participation rates of 35% in the first round, and 80% in the second. In the end, 31 indicators achieved the 70% agreement level, with the majority falling under the operations category, while all customer satisfaction indicators were selected. Cohen’s Kappa analysis showed that nearly 85% of comparisons between the two rounds demonstrated substantial agreement. Conclusions: This study identifies a thorough collection of indicators that offers a valuable tool for enhancing quality in telemedicine settings. The indicators cover important domains in telemedicine services, such as: access, utilization, system efficiency, and customer satisfaction. Future directions should include piloting these KPIs in telemedicine systems and assessing the acceptability of their application.Keywords
Funding Information
- Ain Shams University
This publication has 76 references indexed in Scilit:
- A novel use of telemedicine during the COVID-19 pandemicInternational Journal of Infectious Diseases, 2021
- Using Telemedicine During the COVID-19 PandemicIndian Pediatrics, 2020
- The Egyptian-African Telemedicine Network2019
- ICU Telemedicine Reduces Interhospital ICU Transfers in the Veterans Health AdministrationChest, 2018
- Predicting telemedicine system user satisfaction in sub-Saharan AfricaICT Express, 2016
- Bridging the distance: a prospective tele-oncology study in Northern NorwaySupportive Care in Cancer, 2011
- Fuzzy hierarchy sensitive with Delphi method to evaluate hospital organization performanceExpert Systems with Applications, 2010
- Methods to increase response to postal and electronic questionnairesCochrane Database of Systematic Reviews, 2009
- Idea Generation and Exploration: Benefits and Limitations of the Policy Delphi Research MethodInnovative Higher Education, 2006
- Expert Opinions in Forecasting: The Role of the Delphi Technique2001