Praveer Singh

Importance: Glaucoma, a leading cause of blindness worldwide, depends on accurate optic nerve head assessment, particularly optic disc and cup segmentation, for diagnosis and monitoring. Deep learning (DL) models can automate these measurements, but models trained on smaller, site-specific datasets often fail to generalize. While larger, multi-site datasets help, data privacy concerns limit centralized training. Objective: To evaluate a federated learning (FL) framework with site-specific fine-tuning for optic disc and cup segmentation, aiming to match central model performance while preserving privacy and improving generalizability. Design: Comparative evaluation of three different approaches: (1) a central model trained on multi-site data, (2) site-specific local model training (3) standard FL models, against an FL with site-specific fine-tuning. Setting: Multicenter study incorporating nine publicly available datasets, representing varied clinical environments, populations, and imaging protocols. Participants: 5,550 color fundus photographs from at least 917 individuals across nine datasets includingboth routine care and research sources from 7 countries. Exposures: Optic disc and cup segmentationin color fundus photographs using training with local model, central model, standard FL, and FL with site-specific fine-tuning.. Main Outcomes and Measures: Segmentation accuracy measured by Dice score. Comparisons were labeled as performance “wins” or “losses” based on statistically significant differences via Wilcoxon signed-rank test (P < 0.05). Results: Site-specific fine-tuning of FL with site-specific fine tuning matched central model performance for cup segmentation across all sites (9/9) and for disc segmentation in most sites (7/9). Compared with site-specific local models, it preserved within-site performance (cup: 9/9; disc: 5/9) while substantially improving cross-site generalizability, achieving significant gains in 54.2% (39/72) of disc and 25.0% (18/72) of cup external-site evaluations, with no significant losses. Compared to standard FL pipelines, site-specific fine-tuning improved performance by 52% for disc and 26% for cup. Conclusions and Relevance: Site-specific fine-tuning within an FL framework effectively personalizes generalized models to local data distributions, achieving central-level performance without data sharing and enhancing cross-site robustness. This approach enables privacy-preserving, scalable AI deployment across heterogeneous clinical settings for reproducible and generalizable glaucoma assessment KEY POINTS: Question: How can we train an AI model to segment the optic cup and disc across multiple sites without sharing data, yet achieve performance comparable to a central model trained on pooled datasets? Findings: In this federated learning (FL) study of 5,550 fundus photographs from nine sites, a site-specific fine-tuning FL strategy matched the central model’s performance and outperformed other standard FL techniques, with notable gains in cross-site generalizability. Meaning: Site-specific fine-tuning effectively personalizes FL models to local data distributions, combining data privacy with robust, generalizable performance.

Objectives This is a protocol for a Cochrane Review (diagnostic). The objectives are as follows: To assess the diagnostic performance of AI‐based algorithms in comparison to the established reference standard of clinical diagnosis labels for ROP. Secondary objectives To undertake a critical examination of potential sources of heterogeneity influencing model performance by conducting subgroup analysis by: demographics of the study population (race or ethnicity); type of input data (images only, clinical variables only, or a combination of both); the fundamental methodologies adopted by the AI algorithms (CNN or DNN); and scanner type (Retcam, Forus, or Pheonix ICON).

Many variables of interest in clinical medicine, like disease severity, are recorded using discrete ordinal categories such as normal/mild/moderate/severe. These labels are used to train and evaluate disease severity prediction models. However, ordinal categories represent a simplification of an underlying continuous severity spectrum. Using continuous scores instead of ordinal categories is more sensitive to detecting small changes in disease severity over time. Here, we present a generalized framework that accurately predicts continuously valued variables using only discrete ordinal labels during model development. We found that for three clinical prediction tasks, models that take the ordinal relationship of the training labels into account outperformed conventional multi-class classification models. Particularly the continuous scores generated by ordinal classification and regression models showed a significantly higher correlation with expert rankings of disease severity and lower mean squared errors compared to the multi-class classification models. Furthermore, the use of MC dropout significantly improved the ability of all evaluated deep learning approaches to predict continuously valued scores that truthfully reflect the underlying continuous target variable. We showed that accurate continuously valued predictions can be generated even if the model development only involves discrete ordinal labels. The novel framework has been validated on three different clinical prediction tasks and has proven to bridge the gap between discrete ordinal labels and the underlying continuously valued variables.

By reproducing common methodologic flaws in radiomic machine learning publications, randomly generated radiomic features falsely inflated model performance by a factor of 1.4. Background Radiomics is the extraction of predefined mathematic features from medical images for the prediction of variables of clinical interest. While some studies report superlative accuracy of ...

Federated Learning (FL) wherein multiple institutions collaboratively train a machine learning model without sharing data is becoming popular. Participating institutions might not contribute equally - some contribute more data, some better quality data or some more diverse data. To fairly rank the contribution of different institutions, Shapley value (SV) has emerged as the method of choice. Exact SV computation is impossibly expensive, especially when there are hundreds of contributors. Existing SV computation techniques use approximations. However, in healthcare where the number of contributing institutions are likely not of a colossal scale, computing exact SVs is still exorbitantly expensive, but not impossible. For such settings, we propose an efficient SV computation technique called SaFE (Shapley Value for Federated Learning using Ensembling). We empirically show that SaFE computes values that are close to exact SVs, and that it performs better than current SV approximations. This is particularly relevant in medical imaging setting where widespread heterogeneity across institutions is rampant and fast accurate data valuation is required to determine the contribution of each participant in multi-institutional collaborative learning. Keywords Federated Learning Data valuation Healthcare AI

To tune and test the generalizability of a deep learning-based model for assessment of COVID-19 lung disease severity on chest radiographs (CXRs) from different patient populations. A published convolutional Siamese neural network-based model previously trained on hospitalized patients with COVID-19 was tuned using 250 outpatient CXRs. This model produces a quantitative measure of COVID-19 lung disease severity (pulmonary x-ray severity (PXS) score). The model was evaluated on CXRs from 4 test sets, including 3 from the United States (patients hospitalized at an academic medical center (N = 154), patients hospitalized at a community hospital (N = 113), and outpatients (N = 108)) and 1 from Brazil (patients at an academic medical center emergency department (N = 303)). Radiologists from both countries independently assigned reference standard CXR severity scores, which were correlated with the PXS scores as a measure of model performance (Pearson R). The Uniform Manifold Approximation and Projection (UMAP) technique was used to visualize the neural network results. Tuning the deep learning model with outpatient data showed high model performance in 2 United States hospitalized patient datasets (R = 0.88 and R = 0.90, compared to baseline R = 0.86). Model performance was similar, though slightly lower, when tested on the United States outpatient and Brazil emergency department datasets (R = 0.86 and R = 0.85, respectively). UMAP showed that the model learned disease severity information that generalized across test sets. A deep learning model that extracts a COVID-19 severity score on CXRs showed generalizable performance across multiple populations from 2 continents, including outpatients and hospitalized patients.

Federated learning is an emerging research paradigm for enabling collaboratively training deep learning models without sharing patient data. However, the data from different institutions are usually heterogeneous across institutions, which may reduce the performance of models trained using federated learning. In this study, we propose a novel heterogeneity-aware federated learning method, SplitAVG, to overcome the performance drops from data heterogeneity in federated learning. Unlike previous federated methods that require complex heuristic training or hyper parameter tuning, our SplitAVG leverages the simple network split and feature map concatenation strategies to encourage the federated model training an unbiased estimator of the target data distribution. We compare SplitAVG with seven state-of-the-art federated learning methods, using centrally hosted training data as the baseline on a suite of both synthetic and real-world federated datasets. We find that the performance of models trained using all the comparison federated learning methods degraded significantly with the increasing degrees of data heterogeneity. In contrast, SplitAVG method achieves comparable results to the baseline method under all heterogeneous settings, that it achieves 96.2% of the accuracy and 110.4% of the mean absolute error obtained by the baseline in a diabetic retinopathy binary classification dataset and a bone age prediction dataset, respectively, on highly heterogeneous data partitions. We conclude that SplitAVG method can effectively overcome the performance drops from variability in data distributions across institutions. Experimental results also show that SplitAVG can be adapted to different base networks and generalized to various types of medical imaging tasks.