Assessing a New Prescreening Score for the Simplified Evaluation of the Clinical Quality and Relevance of eHealth Apps: Instrument Validation Study
Open Access
- 5 July 2022
- journal article
- research article
- Published by JMIR Publications Inc. in Journal of Medical Internet Research
- Vol. 24 (7) , e39590
Abstract
Background: In 2020, more than 250 eHealth solutions were added to app stores each day, or 90,000 in the year; however, the vast majority of these solutions have not undergone clinical validation, their quality is unknown, and the user does not know if they are effective and safe. We sought to develop a simple prescreening scoring method that would assess the quality and clinical relevance of each app. We designed this tool with 3 health care stakeholder groups in mind: eHealth solution designers seeking to evaluate a potential competitor or their own tool, investors considering a fundraising candidate, and a hospital clinician or IT department wishing to evaluate a current or potential eHealth solution. Objective: We built and tested a novel prescreening scoring tool (the Medical Digital Solution scoring tool). The tool, which consists of 26 questions that enable the quick assessment and comparison of the clinical relevance and quality of eHealth apps, was tested on 68 eHealth solutions. Methods: The Medical Digital Solution scoring tool is based on the 2021 evaluation criteria of the French National Health Authority, the 2022 European Society of Medical Oncology recommendations, and other provided scores. We built the scoring tool with patient association and eHealth experts and submitted it to eHealth app creators, who evaluated their apps via the web-based form in January 2022. After completing the evaluation criteria, their apps obtained an overall score and 4 categories of subscores. These criteria evaluated the type of solution and domain, the solution’s targeted population size, the level of clinical assessment, and information about the provider. Results: In total, 68 eHealth solutions were evaluated with the scoring tool. Oncology apps (22%, 20/90) and general health solutions (23%, 21/90) were the most represented. Of the 68 apps, 32 (47%) were involved in remote monitoring by health professionals. Regarding clinical outcomes, 5% (9/169) of the apps assessed overall survival. Randomized studies had been conducted for 21% (23/110) of the apps to assess their benefit. Of the 68 providers, 38 (56%) declared the objective of obtaining reimbursement, and 7 (18%) out of the 38 solutions seeking reimbursement were assessed as having a high probability of reimbursement. The median global score was 11.2 (range 4.7-17.4) out of 20 and the distribution of the scores followed a normal distribution pattern (Shapiro-Wilk test: P=.33). Conclusions: This multidomain prescreening scoring tool is simple, fast, and can be deployed on a large scale to initiate an assessment of the clinical relevance and quality of a clinical eHealth app. This simple tool can help a decision-maker determine which aspects of the app require further analysis and improvement.Keywords
All Related Versions
This publication has 14 references indexed in Scilit:
- Implementation of a Self-Triage Web Application for Suspected COVID-19 and Its Impact on Emergency Call Centers: Observational StudyJournal of Medical Internet Research, 2020
- Epidemiological Observations on the Association Between Anosmia and COVID-19 Infection: Analysis of Data From a Self-Assessment Web ApplicationJournal of Medical Internet Research, 2020
- Mobile health app usability and quality rating scales: a systematic reviewDisability and Rehabilitation: Assistive Technology, 2020
- Deriving a practical framework for the evaluation of health appsThe Lancet Digital Health, 2019
- Two-Year Survival Comparing Web-Based Symptom Monitoring vs Routine Surveillance Following Treatment for Lung CancerJAMA, 2019
- Effective? Engaging? Secure? Applying the ORCHA-24 framework to evaluate apps for chronic insomnia disorderEvidence Based Mental Health, 2017
- Clinical Effectiveness of Telemedicine for Chronic Heart Failure: A Systematic Review and Meta-AnalysisJournal of Investigative Medicine, 2017
- Enlight: A Comprehensive Quality and Therapeutic Potential Evaluation Tool for Mobile and Web-Based eHealth InterventionsJournal of Medical Internet Research, 2017
- Guidelines for reporting of health interventions using mobile phones: mobile health (mHealth) evidence reporting and assessment (mERA) checklistBMJ, 2016
- Mobile App Rating Scale: A New Tool for Assessing the Quality of Health Mobile AppsJMIR mHealth and uHealth, 2015