Clinical Validation of a Machine-Learned, Point-of-Care System to IDENTIFY Functionally Significant Coronary Artery Disease
Authors:
- Thomas D. StuckeyTSThomas D. Stuckey
- Frederick J. MeineFMFrederick J. Meine
- Thomas R. McMinnTMThomas R. McMinn
- Jeremiah P. DeptaJDJeremiah P. Depta
- Brett A. BennettBBBrett A. Bennett
- Thomas F. McGarryTMThomas F. McGarry
- William S. CarrollWCWilliam S. Carroll
- David D. SuhDSDavid D. Suh
- John A. SteuterJSJohn A. Steuter
- Michael C. RobertsMRMichael C. Roberts
- Horace R. GillinsHGHorace R. Gillins
- Farhad FathiehFarhad Fathieh
Open Access
- 8 May 2024
- journal article
- research article
- Published by MDPI AG in Diagnostics
- Vol. 14 (10) , 987
Abstract
Many clinical studies have shown wide performance variation in tests to identify coronary artery disease (CAD). Coronary computed tomography angiography (CCTA) has been identified as an effective rule-out test but is not widely available in the USA, particularly so in rural areas. Patients in rural areas are underserved in the healthcare system as compared to urban areas, rendering it a priority population to target with highly accessible diagnostics. We previously developed a machine-learned algorithm to identify the presence of CAD (defined by functional significance) in patients with symptoms without the use of radiation or stress. The algorithm requires 215 s temporally synchronized photoplethysmographic and orthogonal voltage gradient signals acquired at rest. The purpose of the present work is to validate the performance of the algorithm in a frozen state (i.e., no retraining) in a large, blinded dataset from the IDENTIFY trial. IDENTIFY is a multicenter, selectively blinded, non-randomized, prospective, repository study to acquire signals with paired metadata from subjects with symptoms indicative of CAD within seven days prior to either left heart catheterization or CCTA. The algorithm’s sensitivity and specificity were validated using a set of unseen patient signals (n = 1816). Pre-specified endpoints were chosen to demonstrate a rule-out performance comparable to CCTA. The ROC-AUC in the validation set was 0.80 (95% CI: 0.78–0.82). This performance was maintained in both male and female subgroups. At the pre-specified cut point, the sensitivity was 0.85 (95% CI: 0.82–0.88), and the specificity was 0.58 (95% CI: 0.54–0.62), passing the pre-specified endpoints. Assuming a 4% disease prevalence, the NPV was 0.99. Algorithm performance is comparable to tertiary center testing using CCTA. Selection of a suitable cut-point results in the same sensitivity and specificity performance in females as in males. Therefore, a medical device embedding this algorithm may address an unmet need for a non-invasive, front-line point-of-care test for CAD (without any radiation or stress), thus offering significant benefits to the patient, physician, and healthcare system.Keywords
Funding Information
- Analytics for Life
This publication has 17 references indexed in Scilit:
- Mapping Geographic Proximity to Cardiologists Across the United StatesCirculation: Cardiovascular Quality and Outcomes, 2023
- 2021 AHA/ACC/ASE/CHEST/SAEM/SCCT/SCMR Guideline for the Evaluation and Diagnosis of Chest PainJACC, 2021
- Predicting cardiac disease from interactions of simultaneously-acquired hemodynamic and cardiac signalsComputer Methods and Programs in Biomedicine, 2021
- Addressing bias: artificial intelligence in cardiovascular medicineThe Lancet Digital Health, 2020
- The performance of non-invasive tests to rule-in and rule-out significant coronary artery stenosis in patients with stable angina: a meta-analysis focused on post-test disease probabilityEuropean Heart Journal, 2018
- STARD 2015 guidelines for reporting diagnostic accuracy studies: explanation and elaborationBMJ Open, 2016
- Outcomes of Anatomical versus Functional Testing for Coronary Artery DiseaseNew England Journal of Medicine, 2015
- Diagnosis of Ischemia-Causing Coronary Stenoses by Noninvasive Fractional Flow Reserve Computed From Coronary Computed Tomographic AngiogramsJACC, 2011
- Low Diagnostic Yield of Elective Coronary AngiographyNew England Journal of Medicine, 2010
- Exercise-induced ST depression in the diagnosis of coronary artery disease. A meta-analysis.Circulation, 1989