Jianwei Qiu
Jianwei Qiu has a total of 75 co-authors. Here are the top co-authors by number of shared publications:
- Jhimli Mitra
GE HealthCare, Niskayuna, NY 12309, USA
4 shared publications - Soumya Ghose
GE HealthCare, Niskayuna, NY 12309, USA
4 shared publications - Jun Yang
NeuroRegeneration and Stem Cell Programs, Institute for Cell Engineering, Johns Hopkins University School of Medicine, Baltimore, USA
3 shared publications - Marc A. Judson
Division of Pulmonary and Critical Care Medicine, Albany Medical College, Albany, New York
3 shared publications - Aaron Dentinger
GE HealthCare Technology & Innovation Center, Niskayuna, NY, 12309, USA
2 shared publications - Alexandre Boutet
University of Toronto
2 shared publications
Latest Publications
- 18 September 2026
- Published by Wolters Kluwer Health in Neurosurgery
Optimized Brain Functional MRI for Patients With Deep Brain Stimulation Systems
BACKGROUND AND OBJECTIVES: Deep brain stimulation (DBS) is an established neuromodulation therapy for Parkinson disease and other neurological disorders. Active DBS–functional MRI (fMRI) enables visualization of circuit engagement and may provide objective biomarkers of therapeutic efficacy, yet susceptibility artifacts compromise data quality. The aim of this study was to develop optimized blood oxygenation-level–dependent (BOLD) fMRI protocols for DBS patients by adjusting operator-accessible parameters to mitigate susceptibility artifacts while preserving image quality. METHODS: In vitro testing was performed on a 3-dimensional–printed anthropomorphic phantom simulating a patient with a 3T-conditional directional DBS system. We evaluated operator-adjustable parameters—spatial resolution, GeneRalized Autocalibrating Partially Parallel Acquisitions (GRAPPA) acceleration, echo time, and simultaneous multislice acceleration—for their effects on artifact volume and temporal signal-to-noise ratio (tSNR). As exploratory analysis, optimized protocols were demonstrated in subthalamic nucleus-DBS patients (n = 3). RESULTS: Higher spatial resolution and GRAPPA acceleration significantly reduced artifact volume (isotropic voxel size: 95% CI [1620.2, 2605.6], P < .001; GRAPPA: 95% CI [−1169.7, −749.0], P < .001) but also lowered tSNR (isotropic voxel size: 95% CI [62.5, 77.1], P < .001; GRAPPA: 95% CI [−17.3, −12.7], P < .001). Echo time adjustments had smaller yet significant effects on both artifact reduction (95% CI [127.3, 174.4], P < .001) and tSNR (95% CI [−1.3, −0.6], P < .001). Three optimized fMRI protocols were developed: (1) high spatial resolution, (2) balanced high resolution and whole-brain coverage (both with standard temporal resolution), and (3) high temporal resolution at standard spatial resolution (with reduced inner coil volume). Patients were scanned with the second protocol variant and total DBS artifact volume reduced by 45.1% ± 2.4% compared with clinical BOLD fMRI. CONCLUSION: Optimized BOLD fMRI protocols effectively minimized susceptibility artifacts while preserving image quality, enabling reliable functional imaging in DBS patients—a timely advancement given the rise in DBS implants and introduction of directional stimulation systems.
- 18 September 2026
- Published by Wolters Kluwer Health in Neurosurgery
Optimized Brain Functional MRI for Patients With Deep Brain Stimulation Systems
BACKGROUND AND OBJECTIVES: Deep brain stimulation (DBS) is an established neuromodulation therapy for Parkinson disease and other neurological disorders. Active DBS–functional MRI (fMRI) enables visualization of circuit engagement and may provide objective biomarkers of therapeutic efficacy, yet susceptibility artifacts compromise data quality. The aim of this study was to develop optimized blood oxygenation-level–dependent (BOLD) fMRI protocols for DBS patients by adjusting operator-accessible parameters to mitigate susceptibility artifacts while preserving image quality. METHODS: In vitro testing was performed on a 3-dimensional–printed anthropomorphic phantom simulating a patient with a 3T-conditional directional DBS system. We evaluated operator-adjustable parameters—spatial resolution, GeneRalized Autocalibrating Partially Parallel Acquisitions (GRAPPA) acceleration, echo time, and simultaneous multislice acceleration—for their effects on artifact volume and temporal signal-to-noise ratio (tSNR). As exploratory analysis, optimized protocols were demonstrated in subthalamic nucleus-DBS patients (n = 3). RESULTS: Higher spatial resolution and GRAPPA acceleration significantly reduced artifact volume (isotropic voxel size: 95% CI [1620.2, 2605.6], P < .001; GRAPPA: 95% CI [−1169.7, −749.0], P < .001) but also lowered tSNR (isotropic voxel size: 95% CI [62.5, 77.1], P < .001; GRAPPA: 95% CI [−17.3, −12.7], P < .001). Echo time adjustments had smaller yet significant effects on both artifact reduction (95% CI [127.3, 174.4], P < .001) and tSNR (95% CI [−1.3, −0.6], P < .001). Three optimized fMRI protocols were developed: (1) high spatial resolution, (2) balanced high resolution and whole-brain coverage (both with standard temporal resolution), and (3) high temporal resolution at standard spatial resolution (with reduced inner coil volume). Patients were scanned with the second protocol variant and total DBS artifact volume reduced by 45.1% ± 2.4% compared with clinical BOLD fMRI. CONCLUSION: Optimized BOLD fMRI protocols effectively minimized susceptibility artifacts while preserving image quality, enabling reliable functional imaging in DBS patients—a timely advancement given the rise in DBS implants and introduction of directional stimulation systems.
- Open Access
- 24 June 2026
- Published by Elsevier in Photoacoustics
- Vol. 50, 100849
An intelligent point-of-care photoacoustic and ultrasound dual-modality imaging system for Rheumatoid arthritis in fingers: Technical development and initial clinical test
- 24 June 2026
- Published by Elsevier in Photoacoustics
- Vol. 50, 100849
Open AccessAn intelligent point-of-care photoacoustic and ultrasound dual-modality imaging system for Rheumatoid arthritis in fingers: Technical development and initial clinical test
- Open Access
- 5 April 2025
- Published by MDPI AG in Bioengineering
- Vol. 12 (4) , 390
SwinDAF3D: Pyramid Swin Transformers with Deep Attentive Features for Automated Finger Joint Segmentation in 3D Ultrasound Images for Rheumatoid Arthritis Assessment
Rheumatoid arthritis (RA) is a chronic autoimmune disease that can cause severe joint damage and functional impairment. Ultrasound imaging has shown promise in providing real-time assessment of synovium inflammation associated with the early stages of RA. Accurate segmentation of the synovium region and quantification of inflammation-specific imaging biomarkers are crucial for assessing and grading RA. However, automatic segmentation of the synovium in 3D ultrasound is challenging due to ambiguous boundaries, variability in synovium shape, and inhomogeneous intensity distribution. In this work, we introduce a novel network architecture, Swin Transformers with Deep Attentive Features for 3D segmentation (SwinDAF3D), which integrates Swin Transformers into a Deep Attentive Features framework. The developed architecture leverages the hierarchical structure and shifted windows of Swin Transformers to capture rich, multi-scale and attentive contextual information, improving the modeling of long-range dependencies and spatial hierarchies in 3D ultrasound images. In a six-fold cross-validation study with 3D ultrasound images of RA patients’ finger joints (n = 72), our SwinDAF3D model achieved the highest performance with a Dice Score (DSC) of 0.838 ± 0.013, an Intersection over Union (IoU) of 0.719 ± 0.019, and Surface Dice Score (SDSC) of 0.852 ± 0.020, compared to 3D UNet (DSC: 0.742 ± 0.025; IoU: 0.589 ± 0.031; SDSC: 0.661 ± 0.029), DAF3D (DSC: 0.813 ± 0.017; IoU: 0.689 ± 0.022; SDSC: 0.817 ± 0.013), Swin UNETR (DSC: 0.808 ± 0.025; IoU: 0.678 ± 0.032; SDSC: 0.822 ± 0.039), UNETR++ (DSC: 0.810 ± 0.014; IoU: 0.684 ± 0.018; SDSC: 0.829 ± 0.027) and TransUNet (DSC: 0.818 ± 0.013; IoU: 0.692 ± 0.017; SDSC: 0.815 ± 0.016) models. This ablation study demonstrates the effectiveness of combining a Swin Transformers feature pyramid with a deep attention mechanism, improving the segmentation accuracy of the synovium in 3D ultrasound. This advancement shows great promise in enabling more efficient and standardized RA screening using ultrasound imaging.
- 5 April 2025
- Published by MDPI AG in Bioengineering
- Vol. 12 (4) , 390
Open AccessSwinDAF3D: Pyramid Swin Transformers with Deep Attentive Features for Automated Finger Joint Segmentation in 3D Ultrasound Images for Rheumatoid Arthritis Assessment
Rheumatoid arthritis (RA) is a chronic autoimmune disease that can cause severe joint damage and functional impairment. Ultrasound imaging has shown promise in providing real-time assessment of synovium inflammation associated with the early stages of RA. Accurate segmentation of the synovium region and quantification of inflammation-specific imaging biomarkers are crucial for assessing and grading RA. However, automatic segmentation of the synovium in 3D ultrasound is challenging due to ambiguous boundaries, variability in synovium shape, and inhomogeneous intensity distribution. In this work, we introduce a novel network architecture, Swin Transformers with Deep Attentive Features for 3D segmentation (SwinDAF3D), which integrates Swin Transformers into a Deep Attentive Features framework. The developed architecture leverages the hierarchical structure and shifted windows of Swin Transformers to capture rich, multi-scale and attentive contextual information, improving the modeling of long-range dependencies and spatial hierarchies in 3D ultrasound images. In a six-fold cross-validation study with 3D ultrasound images of RA patients’ finger joints (n = 72), our SwinDAF3D model achieved the highest performance with a Dice Score (DSC) of 0.838 ± 0.013, an Intersection over Union (IoU) of 0.719 ± 0.019, and Surface Dice Score (SDSC) of 0.852 ± 0.020, compared to 3D UNet (DSC: 0.742 ± 0.025; IoU: 0.589 ± 0.031; SDSC: 0.661 ± 0.029), DAF3D (DSC: 0.813 ± 0.017; IoU: 0.689 ± 0.022; SDSC: 0.817 ± 0.013), Swin UNETR (DSC: 0.808 ± 0.025; IoU: 0.678 ± 0.032; SDSC: 0.822 ± 0.039), UNETR++ (DSC: 0.810 ± 0.014; IoU: 0.684 ± 0.018; SDSC: 0.829 ± 0.027) and TransUNet (DSC: 0.818 ± 0.013; IoU: 0.692 ± 0.017; SDSC: 0.815 ± 0.016) models. This ablation study demonstrates the effectiveness of combining a Swin Transformers feature pyramid with a deep attention mechanism, improving the segmentation accuracy of the synovium in 3D ultrasound. This advancement shows great promise in enabling more efficient and standardized RA screening using ultrasound imaging.
- 19 September 2024
- Published by SAGE Publications in Ultrasonic Imaging
- Vol. 47 (1) , 14-23
Automated Deep Learning-Based Finger Joint Segmentation in 3-D Ultrasound Images With Limited Dataset
Ultrasound imaging has shown promise in assessing synovium inflammation associated early stages of rheumatoid arthritis (RA). The precise identification of the synovium and the quantification of inflammation-specific imaging biomarkers is a crucial aspect of accurately quantifying and grading RA. In this study, a deep learning-based approach is presented that automates the segmentation of the synovium in ultrasound images of finger joints affected by RA. Two convolutional neural network architectures for image segmentation were trained and validated in a limited number of 2-D images, extracted from N = 18 3-D ultrasound volumes acquired from N = 9 RA patients, with sparse ground truth annotations of the synovium. Various augmentation strategies were employed to enhance the diversity and size of the training dataset. The utilization of geometric and noise augmentation transforms resulted in the highest dice score (0.768 [ ]0.040, N = 6), as determined via six-fold cross-validation. In addition, the segmentation model is used to generate dense 3-D segmentation maps in the ultrasound volumes, based on the available sparse annotations. The developed technique shows promise in facilitating more efficient and standardized workflow for RA screening using ultrasound imaging.
- 19 September 2024
- Published by SAGE Publications in Ultrasonic Imaging
- Vol. 47 (1) , 14-23
Automated Deep Learning-Based Finger Joint Segmentation in 3-D Ultrasound Images With Limited Dataset
Ultrasound imaging has shown promise in assessing synovium inflammation associated early stages of rheumatoid arthritis (RA). The precise identification of the synovium and the quantification of inflammation-specific imaging biomarkers is a crucial aspect of accurately quantifying and grading RA. In this study, a deep learning-based approach is presented that automates the segmentation of the synovium in ultrasound images of finger joints affected by RA. Two convolutional neural network architectures for image segmentation were trained and validated in a limited number of 2-D images, extracted from N = 18 3-D ultrasound volumes acquired from N = 9 RA patients, with sparse ground truth annotations of the synovium. Various augmentation strategies were employed to enhance the diversity and size of the training dataset. The utilization of geometric and noise augmentation transforms resulted in the highest dice score (0.768 [ ]0.040, N = 6), as determined via six-fold cross-validation. In addition, the segmentation model is used to generate dense 3-D segmentation maps in the ultrasound volumes, based on the available sparse annotations. The developed technique shows promise in facilitating more efficient and standardized workflow for RA screening using ultrasound imaging.
- Open Access
- 22 August 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Journal of Translational Engineering in Health and Medicine
- Vol. 12, 589-599
Deep Learning and fMRI-Based Pipeline for Optimization of Deep Brain Stimulation During Parkinson’s Disease Treatment: Toward Rapid Semi-Automated Stimulation Optimization
Objective: Optimized deep brain stimulation (DBS) is fast becoming a therapy of choice for the treatment of Parkinson’s disease (PD). However, the post-operative optimization (aimed at maximizing patient clinical benefits and minimizing adverse effects) of all possible DBS parameter settings using the standard-of-care clinical protocol requires numerous clinical visits, which substantially increases the time to optimization per patient (TPP), patient cost burden and limit the number of patients who can undergo DBS treatment. The TPP is further elongated in electrodes with stimulation directionality or in diseases with latency in clinical feedback. In this work, we proposed a deep learning and fMRI-based pipeline for DBS optimization that can potentially reduce the TPP from ~1 year to a few hours during a single clinical visit.Methods and procedures: We developed an unsupervised autoencoder (AE)-based model to extract meaningful features from 122 previously acquired blood oxygenated level dependent (BOLD) fMRI datasets from 39 a priori clinically optimized PD patients undergoing DBS therapy. The extracted features are then fed into multilayer perceptron (MLP)-based parameter classification and prediction models for rapid DBS parameter optimization.Results: The AE-extracted features of optimal and non-optimal DBS were disentangled. The AE-MLP classification model yielded accuracy, precision, recall, F1 score, and combined AUC of 0.96 ± 0.04, 0.95 ± 0.07, 0.92 ± 0.07, 0.93 ± 0.06, and 0.98 respectively. Accuracies of 0.79 ± 0.04, 0.85 ± 0.04, 0.82 ± 0.05, 0.83 ± 0.05, and 0.70 ± 0.07 were obtained in the prediction of voltage, frequency, and x-y-z contact locations, respectively.Conclusion: The proposed AE-MLP models yielded promising results for fMRI-based DBS parameter classification and prediction, potentially facilitating rapid semi-automated DBS parameter optimization. Clinical and Translational Impact Statement—A deep learning-based pipeline for semi-automated DBS parameter optimization is presented, with the potential to significantly decrease the optimization duration per patient and patients' financial burden while increasing patient throughput.
- 22 August 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Journal of Translational Engineering in Health and Medicine
- Vol. 12, 589-599
Open AccessDeep Learning and fMRI-Based Pipeline for Optimization of Deep Brain Stimulation During Parkinson’s Disease Treatment: Toward Rapid Semi-Automated Stimulation Optimization
Objective: Optimized deep brain stimulation (DBS) is fast becoming a therapy of choice for the treatment of Parkinson’s disease (PD). However, the post-operative optimization (aimed at maximizing patient clinical benefits and minimizing adverse effects) of all possible DBS parameter settings using the standard-of-care clinical protocol requires numerous clinical visits, which substantially increases the time to optimization per patient (TPP), patient cost burden and limit the number of patients who can undergo DBS treatment. The TPP is further elongated in electrodes with stimulation directionality or in diseases with latency in clinical feedback. In this work, we proposed a deep learning and fMRI-based pipeline for DBS optimization that can potentially reduce the TPP from ~1 year to a few hours during a single clinical visit.Methods and procedures: We developed an unsupervised autoencoder (AE)-based model to extract meaningful features from 122 previously acquired blood oxygenated level dependent (BOLD) fMRI datasets from 39 a priori clinically optimized PD patients undergoing DBS therapy. The extracted features are then fed into multilayer perceptron (MLP)-based parameter classification and prediction models for rapid DBS parameter optimization.Results: The AE-extracted features of optimal and non-optimal DBS were disentangled. The AE-MLP classification model yielded accuracy, precision, recall, F1 score, and combined AUC of 0.96 ± 0.04, 0.95 ± 0.07, 0.92 ± 0.07, 0.93 ± 0.06, and 0.98 respectively. Accuracies of 0.79 ± 0.04, 0.85 ± 0.04, 0.82 ± 0.05, 0.83 ± 0.05, and 0.70 ± 0.07 were obtained in the prediction of voltage, frequency, and x-y-z contact locations, respectively.Conclusion: The proposed AE-MLP models yielded promising results for fMRI-based DBS parameter classification and prediction, potentially facilitating rapid semi-automated DBS parameter optimization. Clinical and Translational Impact Statement—A deep learning-based pipeline for semi-automated DBS parameter optimization is presented, with the potential to significantly decrease the optimization duration per patient and patients' financial burden while increasing patient throughput.
- Open Access
- 18 May 2024
- Published by MDPI AG in Diagnostics
- Vol. 14 (10) , 1049
A Multichannel CT and Radiomics-Guided CNN-ViT (RadCT-CNNViT) Ensemble Network for Diagnosis of Pulmonary Sarcoidosis
Pulmonary sarcoidosis is a multisystem granulomatous interstitial lung disease (ILD) with a variable presentation and prognosis. The early accurate detection of pulmonary sarcoidosis may prevent progression to pulmonary fibrosis, a serious and potentially life-threatening form of the disease. However, the lack of a gold-standard diagnostic test and specific radiographic findings poses challenges in diagnosing pulmonary sarcoidosis. Chest computed tomography (CT) imaging is commonly used but requires expert, chest-trained radiologists to differentiate pulmonary sarcoidosis from lung malignancies, infections, and other ILDs. In this work, we develop a multichannel, CT and radiomics-guided ensemble network (RadCT-CNNViT) with visual explainability for pulmonary sarcoidosis vs. lung cancer (LCa) classification using chest CT images. We leverage CT and hand-crafted radiomics features as input channels, and a 3D convolutional neural network (CNN) and vision transformer (ViT) ensemble network for feature extraction and fusion before a classification head. The 3D CNN sub-network captures the localized spatial information of lesions, while the ViT sub-network captures long-range, global dependencies between features. Through multichannel input and feature fusion, our model achieves the highest performance with accuracy, sensitivity, specificity, precision, F1-score, and combined AUC of 0.93 ± 0.04, 0.94 ± 0.04, 0.93 ± 0.08, 0.95 ± 0.05, 0.94 ± 0.04, and 0.97, respectively, in a five-fold cross-validation study with pulmonary sarcoidosis (n = 126) and LCa (n = 93) cases. A detailed ablation study showing the impact of CNN + ViT compared to CNN or ViT alone, and CT + radiomics input, compared to CT or radiomics alone, is also presented in this work. Overall, the AI model developed in this work offers promising potential for triaging the pulmonary sarcoidosis patients for timely diagnosis and treatment from chest CT.
- 18 May 2024
- Published by MDPI AG in Diagnostics
- Vol. 14 (10) , 1049
Open AccessA Multichannel CT and Radiomics-Guided CNN-ViT (RadCT-CNNViT) Ensemble Network for Diagnosis of Pulmonary Sarcoidosis
Pulmonary sarcoidosis is a multisystem granulomatous interstitial lung disease (ILD) with a variable presentation and prognosis. The early accurate detection of pulmonary sarcoidosis may prevent progression to pulmonary fibrosis, a serious and potentially life-threatening form of the disease. However, the lack of a gold-standard diagnostic test and specific radiographic findings poses challenges in diagnosing pulmonary sarcoidosis. Chest computed tomography (CT) imaging is commonly used but requires expert, chest-trained radiologists to differentiate pulmonary sarcoidosis from lung malignancies, infections, and other ILDs. In this work, we develop a multichannel, CT and radiomics-guided ensemble network (RadCT-CNNViT) with visual explainability for pulmonary sarcoidosis vs. lung cancer (LCa) classification using chest CT images. We leverage CT and hand-crafted radiomics features as input channels, and a 3D convolutional neural network (CNN) and vision transformer (ViT) ensemble network for feature extraction and fusion before a classification head. The 3D CNN sub-network captures the localized spatial information of lesions, while the ViT sub-network captures long-range, global dependencies between features. Through multichannel input and feature fusion, our model achieves the highest performance with accuracy, sensitivity, specificity, precision, F1-score, and combined AUC of 0.93 ± 0.04, 0.94 ± 0.04, 0.93 ± 0.08, 0.95 ± 0.05, 0.94 ± 0.04, and 0.97, respectively, in a five-fold cross-validation study with pulmonary sarcoidosis (n = 126) and LCa (n = 93) cases. A detailed ablation study showing the impact of CNN + ViT compared to CNN or ViT alone, and CT + radiomics input, compared to CT or radiomics alone, is also presented in this work. Overall, the AI model developed in this work offers promising potential for triaging the pulmonary sarcoidosis patients for timely diagnosis and treatment from chest CT.
- Open Access
- 3 May 2024
- Published by MDPI AG in Preprints.org
A Multichannel CT and Radiomics-Guided CNN-ViT (RadCT-CNNViT) Ensemble Network for Diagnosis of Pulmonary Sarcoidosis
Pulmonary sarcoidosis is a multisystem granulomatous interstitial lung disease (ILD) with a variable presentation and prognosis. Early accurate detection of pulmonary sarcoidosis may prevent progression to pulmonary fibrosis, a serious and potentially life-threatening form of the disease. However, the lack of a gold-standard diagnostic test and specific radiographic findings pose challenges in diagnosing pulmonary sarcoidosis. Chest computed tomography (CT) imaging is commonly used but requires expert, chest-trained radiologists to differentiate pulmonary sarcoidosis from lung malignancies, infections, and other ILDs. In this work, we developed a multichannel, CT and radiomics-guided ensemble network (RadCT-CNNViT) with visual explainability for pulmonary sarcoidosis vs. lung cancer (LCa) classification using chest CT images. We leverage CT and hand-crafted radiomics features as input channels, and a 3D convolutional neural network (CNN) and vision transformer (ViT) ensemble network for feature extraction and fusion before a classification head. The 3D CNN sub-network captures localized spatial information of lesions, while the ViT sub-network captures long-range, global dependencies between features. Through multichannel input and feature fusion, our model achieved the highest performance with accuracy, sensitivity, specificity, and combined AUC of 0.93±0.04, 0.94±0.04, 0.93±0.08 and 0.97 respectively in a 5-fold cross-validation study with pulmonary sarcoidosis (n=126) and LCa (n=93) cases. The model offers promising potential for improving the diagnosis of pulmonary sarcoidosis. A detailed ablation study showing the impact of CNN+ViT compared to CNN or ViT alone, and CT+radiomics input, compared to CT or radiomics alone, is also presented in this work.
- 3 May 2024
- Published by MDPI AG in Preprints.org
Open AccessA Multichannel CT and Radiomics-Guided CNN-ViT (RadCT-CNNViT) Ensemble Network for Diagnosis of Pulmonary Sarcoidosis
Pulmonary sarcoidosis is a multisystem granulomatous interstitial lung disease (ILD) with a variable presentation and prognosis. Early accurate detection of pulmonary sarcoidosis may prevent progression to pulmonary fibrosis, a serious and potentially life-threatening form of the disease. However, the lack of a gold-standard diagnostic test and specific radiographic findings pose challenges in diagnosing pulmonary sarcoidosis. Chest computed tomography (CT) imaging is commonly used but requires expert, chest-trained radiologists to differentiate pulmonary sarcoidosis from lung malignancies, infections, and other ILDs. In this work, we developed a multichannel, CT and radiomics-guided ensemble network (RadCT-CNNViT) with visual explainability for pulmonary sarcoidosis vs. lung cancer (LCa) classification using chest CT images. We leverage CT and hand-crafted radiomics features as input channels, and a 3D convolutional neural network (CNN) and vision transformer (ViT) ensemble network for feature extraction and fusion before a classification head. The 3D CNN sub-network captures localized spatial information of lesions, while the ViT sub-network captures long-range, global dependencies between features. Through multichannel input and feature fusion, our model achieved the highest performance with accuracy, sensitivity, specificity, and combined AUC of 0.93±0.04, 0.94±0.04, 0.93±0.08 and 0.97 respectively in a 5-fold cross-validation study with pulmonary sarcoidosis (n=126) and LCa (n=93) cases. The model offers promising potential for improving the diagnosis of pulmonary sarcoidosis. A detailed ablation study showing the impact of CNN+ViT compared to CNN or ViT alone, and CT+radiomics input, compared to CT or radiomics alone, is also presented in this work.
- 14 November 2023
- Published by Springer Nature in Lung
- Vol. 201 (6) , 611-616
An Artificial Intelligence Platform for the Radiologic Diagnosis of Pulmonary Sarcoidosis: An Initial Pilot Study of Chest Computed Tomography Analysis to Distinguish Pulmonary Sarcoidosis from a Negative Lung Cancer Screening Scan
Purpose To determine the reliability of an artificial intelligence, deep learning (AI/DL)-based method of chest computer tomography (CT) scan analysis to distinguish pulmonary sarcoidosis from negative lung cancer screening chest CT scans (Lung Imaging Reporting and Data System score 1, Lung-RADS score 1). Methods Chest CT scans of pulmonary sarcoidosis were evaluated by a clinician experienced with sarcoidosis and a chest radiologist for clinical and radiologic evidence of sarcoidosis and exclusion of alternative or concomitant pulmonary diseases. The AI/DL based method used an ensemble network architecture combining Convolutional Neural Networks (CNNs) and Vision Transformers (ViTs). The method was applied to 126 pulmonary sarcoidosis and 96 Lung-RADS score 1 CT scans. The analytic approach of training and validation of the AI/DL method used a fivefold cross-validation technique, where 4/5th of the available data set was used to train a diagnostic model and tested on the remaining 1/5th of the data set, and repeated 4 more times with non-overlapping validation/test data. The probability values were used to generate Receiver Operating Characteristic (ROC) curves to assess the model’s discriminatory power. Results The sensitivity, specificity, positive and negative predictive value of the AI/DL method for the 5 folds of the training/validation sets and the entire set of CT scans were all over 94% to distinguish pulmonary sarcoidosis from LUNG-RADS score 1 chest CT scans. The area under the curve for the corresponding ROC curves were all over 97%. Conclusion This AL/DL model shows promise to distinguish sarcoidosis from alternative pulmonary conditions using minimal radiologic data.
- 14 November 2023
- Published by Springer Nature in Lung
- Vol. 201 (6) , 611-616
An Artificial Intelligence Platform for the Radiologic Diagnosis of Pulmonary Sarcoidosis: An Initial Pilot Study of Chest Computed Tomography Analysis to Distinguish Pulmonary Sarcoidosis from a Negative Lung Cancer Screening Scan
Purpose To determine the reliability of an artificial intelligence, deep learning (AI/DL)-based method of chest computer tomography (CT) scan analysis to distinguish pulmonary sarcoidosis from negative lung cancer screening chest CT scans (Lung Imaging Reporting and Data System score 1, Lung-RADS score 1). Methods Chest CT scans of pulmonary sarcoidosis were evaluated by a clinician experienced with sarcoidosis and a chest radiologist for clinical and radiologic evidence of sarcoidosis and exclusion of alternative or concomitant pulmonary diseases. The AI/DL based method used an ensemble network architecture combining Convolutional Neural Networks (CNNs) and Vision Transformers (ViTs). The method was applied to 126 pulmonary sarcoidosis and 96 Lung-RADS score 1 CT scans. The analytic approach of training and validation of the AI/DL method used a fivefold cross-validation technique, where 4/5th of the available data set was used to train a diagnostic model and tested on the remaining 1/5th of the data set, and repeated 4 more times with non-overlapping validation/test data. The probability values were used to generate Receiver Operating Characteristic (ROC) curves to assess the model’s discriminatory power. Results The sensitivity, specificity, positive and negative predictive value of the AI/DL method for the 5 folds of the training/validation sets and the entire set of CT scans were all over 94% to distinguish pulmonary sarcoidosis from LUNG-RADS score 1 chest CT scans. The area under the curve for the corresponding ROC curves were all over 97%. Conclusion This AL/DL model shows promise to distinguish sarcoidosis from alternative pulmonary conditions using minimal radiologic data.
- Open Access
- 19 August 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Journal of Translational Engineering in Health and Medicine
- Vol. 10, 1-9
Automatic Hemorrhage Detection From Color Doppler Ultrasound Using a Generative Adversarial Network (GAN)-Based Anomaly Detection Method
Hemorrhage control has been identified as a priority focus area both for civilian and military populations in the United States because exsanguination is the most common cause of preventable death in hemorrhagic injury. Non-compressible torso hemorrhage (NCTH) has high mortality rate and there are currently no broadly available therapies for NCTH outside of a surgical room environment. Novel therapies, which include High Intensity Focused Ultrasound (HIFU) have emerged as promising methods for hemorrhage control as they can non-invasively cauterize bleeding tissue deep within the body without injuring uninvolved regions. A major challenge in the application of HIFU with color Doppler US guidance is the interpretation and optimization of the blood flow images in real-time to identify the hemorrhagic focus. Today, this task requires an expert sonographer, limiting the utility of this therapy in non-clinical environments. In this work, we investigated the feasibility of an automated hemorrhage detection method using a Generative Adversarial Network (GAN) for anomaly detection that learns a manifold of normal blood flow variability and subsequently identifies anomalous flow patterns that fall outside the learned manifold. As an initial feasibility study, we collected ultrasound color Doppler images of femoral arteries in an animal model of vascular injury (N=11 pigs). Velocity information of the blood flow were extracted from the color Doppler images that were used for training and testing the anomaly detection network. Normotensive images from 8 pigs were used for training, and testing was performed on normotensive, immediately after injury, 10 minutes post-injury and 30 minutes post-injury images from 3 other pigs. The residual images or the reconstructed error maps show promise in detecting hemorrhages with an AUC of 0.90, 0.87, 0.62 immediately, 10 minutes post-injury and 30 minutes post-injury respectively with an overall AUC of 0.83.
- 19 August 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Journal of Translational Engineering in Health and Medicine
- Vol. 10, 1-9
Open AccessAutomatic Hemorrhage Detection From Color Doppler Ultrasound Using a Generative Adversarial Network (GAN)-Based Anomaly Detection Method
Hemorrhage control has been identified as a priority focus area both for civilian and military populations in the United States because exsanguination is the most common cause of preventable death in hemorrhagic injury. Non-compressible torso hemorrhage (NCTH) has high mortality rate and there are currently no broadly available therapies for NCTH outside of a surgical room environment. Novel therapies, which include High Intensity Focused Ultrasound (HIFU) have emerged as promising methods for hemorrhage control as they can non-invasively cauterize bleeding tissue deep within the body without injuring uninvolved regions. A major challenge in the application of HIFU with color Doppler US guidance is the interpretation and optimization of the blood flow images in real-time to identify the hemorrhagic focus. Today, this task requires an expert sonographer, limiting the utility of this therapy in non-clinical environments. In this work, we investigated the feasibility of an automated hemorrhage detection method using a Generative Adversarial Network (GAN) for anomaly detection that learns a manifold of normal blood flow variability and subsequently identifies anomalous flow patterns that fall outside the learned manifold. As an initial feasibility study, we collected ultrasound color Doppler images of femoral arteries in an animal model of vascular injury (N=11 pigs). Velocity information of the blood flow were extracted from the color Doppler images that were used for training and testing the anomaly detection network. Normotensive images from 8 pigs were used for training, and testing was performed on normotensive, immediately after injury, 10 minutes post-injury and 30 minutes post-injury images from 3 other pigs. The residual images or the reconstructed error maps show promise in detecting hemorrhages with an AUC of 0.90, 0.87, 0.62 immediately, 10 minutes post-injury and 30 minutes post-injury respectively with an overall AUC of 0.83.