Design and Internal Validation of an AI‐Assisted Clinical Decision Support System (CDSS) for Telemedicine Services to Augment Patient Care in Rural Areas of Rajasthan
Open Access
- 1 January 2026
- journal article
- Published by Wiley in International Journal of Telemedicine and Applications
- Vol. 2026 (1)
Abstract
Background: Conventional teleconsultation platforms function primarily as communication interfaces and lack integrated clinical intelligence or adaptive learning capability. This study presents the development and design evaluation of the TeleMed Clinical Decision Support System (CDSS v2.5), an AI‐assisted predictive framework engineered to augment teleconsultation quality in rural areas of Rajasthan, India. Methods: A development and internal validation design following TRIPOD‐AI guidelines was used to develop TeleMed CDSS v2.5. The framework integrated TabPFN 2.5 for tabular prediction, MiniLM embeddings for semantic symptom normalization, and KNN‐based clinical retrieval within an offline‐first, doctor‐in‐the‐loop telemedicine platform. Internal validation used stratified fivefold cross‐validation on the complete 4401‐record dataset. A clinician‐curated 100‐case edge‐case robustness suite, selected from the main dataset and reserved for secondary evaluation, was used to assess performance on linguistic edge cases and multidomain presentations. This suite is not a separate validation cohort to provide independent evidence of generalization. For comparative benchmarking, XGBoost, random forest, logistic regression, and a rule‐based triage model were evaluated using fivefold stratified cross‐validation on the same dataset. All AI‐generated outputs were reviewed, modified if necessary, and approved by the consulting clinician; only parent‐category classification was quantitatively evaluated. Results: The TeleMed CDSS v2.5 achieved a macro AUROC of 0.962 (95% CI: 0.953–0.971) and accuracy of 87.2% (95% CI: 0.858–0.886) under fivefold stratified cross‐validation. On the clinician‐curated 100‐case edge‐case robustness suite, category‐level accuracy was 90.0% as an internal robustness assessment. Comparator model accuracies under fivefold cross‐validation were: TabPFN 2.5 standalone (0.831), XGBoost (0.845), random forest (0.827), logistic regression (0.792), and rule‐based triage (0.743). The complete multilayer pipeline improved upon standalone TabPFN by 10.7 percentage points on the 100‐case suite (90.0% vs. 79.3%). Contextual adaptation remained stable from the initial 2000‐record model through sequential expansion to 3000, 4000, and 4401 records, with accuracy differences from full retraining below 1 percentage point. The 8000‐record condition was a projected stress‐test and was not empirically observed during the study period. Conclusions: The TeleMed CDSS v2.5 demonstrates technical feasibility of a hybrid AI framework integrating transformer‐based prediction, semantic symptom normalisation, and doctor‐guided contextual adaptation within a rural telemedicine workflow. These findings are limited to the design‐evaluation phase; external validation and prospective clinical effectiveness studies are required before broader deployment. Trial Registration: CTRI: CTRI/2024/10/076070Keywords
This publication has 30 references indexed in Scilit:
- The Impact of Artificial Intelligence on Healthcare: A Comprehensive Review of Advancements in Diagnostics, Treatment, and Operational EfficiencyHealth Science Reports, 2025
- Artificial intelligence for access to primary healthcare in rural settingsJournal of Medicine, Surgery, and Public Health, 2024
- Reimagining India's National Telemedicine Service to improve access to careThe Lancet Regional Health - Southeast Asia, 2024
- “Leveraging AI Tools to Bridge the Healthcare Gap in Rural Areas in India”medRxiv, 2024
- TRIPOD+AI statement: updated guidance for reporting clinical prediction models that use regression or machine learning methodsBMJ, 2024
- Second opinion needed: communicating uncertainty in medical machine learningnpj Digital Medicine, 2021
- An integrated iterative annotation technique for easing neural network training in medical image analysisNature Machine Intelligence, 2019
- Evaluation and accurate diagnoses of pediatric diseases using artificial intelligenceNature Medicine, 2019
- A powerful and data‐adaptive test for rare‐variant–based gene‐environment interaction analysisStatistics in Medicine, 2018
- Overcoming catastrophic forgetting in neural networksProceedings of the National Academy of Sciences of the United States of America, 2017