TSTaly Gilat Schmidt

Taly Gilat Schmidt

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article


Taly Gilat Schmidt has a total of 126 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Open Access

    Real-World and Clinical Trial Validation of a Deep Learning Radiomic Biomarker for PD-(L)1 Immune Checkpoint Inhibitor Response in Advanced Non–Small Cell Lung Cancer

    by Chiharu Sako, Chong Duan, Kevin Maresca, Sean Kent, Taly Gilat Schmidt, Hugo J.W.L. Aerts, Ravi B. Parikh, George R. Simon, Petr Jordan

    PURPOSE: This study developed and validated a novel deep learning radiomic biomarker to estimate response to immune checkpoint inhibitor (ICI) therapy in advanced non–small cell lung cancer (NSCLC) using real-world data (RWD) and clinical trial data. MATERIALS AND METHODS: Retrospective RWD of 1,829 patients with advanced NSCLC treated with PD-(L)1 ICIs were collected from 10 academic and community institutions in the United States and Europe. The RWD included data sets for discovery (Data Set A-Discovery, n = 1,173) and independent test (Data Set B, n = 458). A radiomic pipeline, containing a deep learning feature extractor and a survival model, generated the computed tomography (CT) response score (CTRS) applied to the pretreatment routine CT/positron emission tomography (PET)-CT scan. An enhanced CTRS (eCTRS) also incorporated age, sex, treatment line, and lesion annotations. Performance was evaluated against progression-free survival (PFS) and overall survival (OS). Biomarker generalizability was further evaluated using a secondary analysis of a prospective clinical trial (ClinicalTrials.gov identifier: NCT02573259 ) evaluating the PD-1 inhibitor sasanlimab in second or later line of treatment (Data Set C, n = 54). RESULTS: In RWD Test Data Set B, the CTRS identified patients with a high probability of response to ICI with a PFS hazard ratio (HR) of 0.46 (95% CI, 0.26 to 0.82) and an OS HR of 0.50 (95% CI, 0.28 to 0.92) in the first-line ICI monotherapy cohort, after adjustment for baseline covariates including the PD-L1 tumor proportion score. In Clinical Trial Data Set C, the CTRS demonstrated an adjusted PFS HR of 1.03 (95% CI, 0.43 to 2.47) and an OS HR of 0.33 (95% CI, 0.14 to 0.91). The CTRS and eCTRS outperformed traditional imaging biomarkers of lesion size in PFS and OS for RWD Test Data Set B and in OS for the Clinical Trial Data Set. CONCLUSION: The study developed and validated a deep learning radiomic biomarker using pretreatment routine CT/PET-CT scans to identify ICI benefit in advanced NSCLC.


  • Open Access

    Age Encoded Adversarial Learning for Pediatric CT Segmentation

    by Saba Heidari Gheshlaghi, Chi Nok Enoch Kan, Taly Gilat Schmidt, Dong Hye Ye

    Organ segmentation from CT images is critical in the early diagnosis of diseases, progress monitoring, pre-operative planning, radiation therapy planning, and CT dose estimation. However, data limitation remains one of the main challenges in medical image segmentation tasks. This challenge is particularly huge in pediatric CT segmentation due to children’s heightened sensitivity to radiation. In order to address this issue, we propose a novel segmentation framework with a built-in auxiliary classifier generative adversarial network (ACGAN) that conditions age, simultaneously generating additional features during training. The proposed conditional feature generation segmentation network (CFG-SegNet) was trained on a single loss function and used 2.5D segmentation batches. Our experiment was performed on a dataset with 359 subjects (180 male and 179 female) aged from 5 days to 16 years and a mean age of 7 years. CFG-SegNet achieved an average segmentation accuracy of 0.681 dice similarity coefficient (DSC) on the prostate, 0.619 DSC on the uterus, 0.912 DSC on the liver, and 0.832 DSC on the heart with four-fold cross-validation. We compared the segmentation accuracy of our proposed method with previously published U-Net results, and our network improved the segmentation accuracy by 2.7%, 2.6%, 2.8%, and 3.4% for the prostate, uterus, liver, and heart, respectively. The results indicate that our high-performing segmentation framework can more precisely segment organs when limited training images are available.


  • Constrained one‐step material decomposition reconstruction of head CT data from a silicon photon‐counting prototype

    by Taly Gilat Schmidt, Emil Y. Sidky, Xiaochuan Pan, Rina Foygel Barber, Fredrik Grönberg, Martin Sjölin, Mats Danielsson

    Background: Spectral CT material decomposition provides quantitative information but is challenged by the instability of the inversion into basis materials. We have previously proposed the constrained One‐Step Spectral CT Image Reconstruction (cOSSCIR) algorithm to stabilize the material decomposition inversion by directly estimating basis material images from spectral CT data. cOSSCIR was previously investigated on phantom data.Purpose: This study investigates the performance of cOSSCIR using head CT datasets acquired on a clinical photon‐counting CT (PCCT) prototype. This is the first investigation of cOSSCIR for large‐scale, anatomically complex, clinical PCCT data. The cOSSCIR decomposition is preceded by a spectrum estimation and nonlinear counts correction calibration step to address nonideal detector effects.Methods: Head CT data were acquired on an early prototype clinical PCCT system using an edge‐on silicon detector with eight energy bins. Calibration data of a step wedge phantom were also acquired and used to train a spectral model to account for the source spectrum and detector spectral response, and also to train a nonlinear counts correction model to account for pulse pileup effects. The cOSSCIR algorithm optimized the bone and adipose basis images directly from the photon counts data, while placing a grouped total variation (TV) constraint on the basis images. For comparison, basis images were also reconstructed by a two‐step projection‐domain approach of Maximum Likelihood Estimation (MLE) for decomposing basis sinograms, followed by filtered backprojection (MLE + FBP) or a TV minimization algorithm (MLE + TVmin) to reconstruct basis images. We hypothesize that the cOSSCIR approach will provide a more stable inversion into basis images compared to two‐step approaches. To investigate this hypothesis, the noise standard deviation in bone and soft‐tissue regions of interest (ROIs) in the reconstructed images were compared between cOSSCIR and the two‐step methods for a range of regularization constraint settings.Results: cOSSCIR reduced the noise standard deviation in the basis images by a factor of two to six compared to that of MLE + TVmin, when both algorithms were constrained to produce images with the same TV. The cOSSCIR images demonstrated qualitatively improved spatial resolution and depiction of fine anatomical detail. The MLE + TVmin algorithm resulted in lower noise standard deviation than cOSSCIR for the virtual monoenergetic images (VMIs) at higher energy levels and constraint settings, while the cOSSCIR VMIs resulted in lower noise standard deviation at lower energy levels and overall higher qualitative spatial resolution. There were no statistically significant differences in the mean values within the bone region of images reconstructed by the studied algorithms. There were statistically significant differences in the mean values within the soft‐tissue region of the reconstructed images, with cOSSCIR producing mean values closer to the expected values.Conclusions: The cOSSCIR algorithm, combined with our previously proposed spectral model estimation and nonlinear counts correction method, successfully estimated bone and adipose basis images from high resolution, large‐scale patient data from a clinical PCCT prototype. The cOSSCIR basis images were able to depict fine anatomical details with a factor of two to six reduction in noise standard deviation compared to that of the MLE + TVmin two‐step approach.


  • Open Access

    Accurate Image Reconstruction in Dual-Energy CT with Limited-Angular-Range Data Using a Two-Step Method

    by Buxin Chen, Zheng Zhang, Dan Xia, Emil Y. Sidky, Taly Gilat-Schmidt, Xiaochuan Pan

    Dual-energy CT (DECT) with scans over limited-angular ranges (LARs) may allow reductions in scan time and radiation dose and avoidance of possible collision between the moving parts of a scanner and the imaged object. The beam-hardening (BH) and LAR effects are two sources of image artifacts in DECT with LAR data. In this work, we investigate a two-step method to correct for both BH and LAR artifacts in order to yield accurate image reconstruction in DECT with LAR data. From low- and high-kVp LAR data in DECT, we first use a data-domain decomposition (DDD) algorithm to obtain LAR basis data with the non-linear BH effect corrected for. We then develop and tailor a directional-total-variation (DTV) algorithm to reconstruct from the LAR basis data obtained basis images with the LAR effect compensated for. Finally, using the basis images reconstructed, we create virtual monochromatic images (VMIs), and estimate physical quantities such as iodine concentrations and effective atomic numbers within the object imaged. We conduct numerical studies using two digital phantoms of different complexity levels and types of structures. LAR data of low- and high-kVp are generated from the phantoms over both single-arc (SA) and two-orthogonal-arc (TOA) LARs ranging from 14∘ to 180∘. Visual inspection and quantitative assessment of VMIs obtained reveal that the two-step method proposed can yield VMIs in which both BH and LAR artifacts are reduced, and estimation accuracy of physical quantities is improved. In addition, concerning SA and TOA scans with the same total LAR, the latter is shown to yield more accurate images and physical quantity estimations than the former. We investigate a two-step method that combines the DDD and DTV algorithms to correct for both BH and LAR artifacts in image reconstruction, yielding accurate VMIs and estimations of physical quantities, from low- and high-kVp LAR data in DECT. The results and knowledge acquired in the work on accurate image reconstruction in LAR DECT may give rise to further understanding and insights into the practical design of LAR scan configurations and reconstruction procedures for DECT applications.


  • Open Access

    Reduced Chest Computed Tomography Scan Length for Patients Positive for Coronavirus Disease 2019: Dose Reduction and Impact on Diagnostic Utility

    by Sara Principi, Stacy O'COnnor, Luba Frank, Taly Gilat Schmidt

    Objective: This work aimed to retrospectively evaluate the potential of dose reduction on chest computed tomography (CT) examinations by reducing the longitudinal scan length for patients positive for coronavirus disease 2019 (COVID-19). Methods: This study used the Personalized Rapid Estimation of Dose in CT (PREDICT) tool to estimate patient-specific organ doses from CT image data. The PREDICT is a research tool that combines a linear Boltzmann transport equation solver for radiation dose map generation with deep learning algorithms for organ contouring. Computed tomography images from 74 subjects in the Medical Imaging Data Resource Center–RSNA International COVID-19 Open Radiology Database data set (chest CT of adult patients positive for COVID-19), which included expert annotations including “infectious opacities,” were analyzed. First, the full z-scan length of the CT image data set was evaluated. Next, the z-scan length was reduced from the left hemidiaphragm to the top of the aortic arch. Generic dose reduction based on dose length product (DLP) and patient-specific organ dose reductions were calculated. The percentage of infectious opacities excluded from the reduced z-scan length was used to quantify the effect on diagnostic utility. Results: Generic dose reduction, based on DLP, was 69%. The organ dose reduction ranged from approximately equal to 18% (breasts) to approximately equal to 64% (bone surface and bone marrow). On average, 12.4% of the infectious opacities were not included in the reduced z-coverage, per patient, of which 5.1% were above the top of the arch and 7.5% below the left hemidiaphragm. Conclusions: Limiting z-scan length of chest CTs reduced radiation dose without significantly compromising diagnostic utility in COVID-19 patients. The PREDICT demonstrated that patient-specific organ dose reductions varied from generic dose reduction based on DLP.


  • Addressing CT metal artifacts using photon‐counting detectors and one‐step spectral CT image reconstruction

    by Taly Gilat Schmidt, Barbara A. Sammut, Rina Foygel Barber, Xiaochuan Pan, Emil Y. Sidky

    Purpose: The constrained one‐step spectral CT image reconstruction (cOSSCIR) algorithm with a nonconvex alternating direction method of multipliers optimizer is proposed for addressing computed tomography (CT) metal artifacts caused by beam hardening, noise, and photon starvation. The quantitative performance of cOSSCIR is investigated through a series of photon‐counting CT simulations.Methods: cOSSCIR directly estimates basis material maps from photon‐counting data using a physics‐based forward model that accounts for beam hardening. The cOSSCIR optimization framework places constraints on the basis maps, which we hypothesize will stabilize the decomposition and reduce streaks caused by noise and photon starvation. Another advantage of cOSSCIR is that the spectral data need not be registered, so that a ray can be used even if some energy window measurements are unavailable. Photon‐counting CT acquisitions of a virtual pelvic phantom with low‐contrast soft tissue texture and bilateral hip prostheses were simulated. Bone and water basis maps were estimated using the cOSSCIR algorithm and combined to form a virtual monoenergetic image for the evaluation of metal artifacts. The cOSSCIR images were compared to a “two‐step” decomposition approach that first estimated basis sinograms using a maximum likelihood algorithm and then reconstructed basis maps using an iterative total variation constrained least‐squares optimization (MLE+TV). Images were also compared to a nonspectral TV reconstruction of the total number of counts detected for each ray with and without normalized metal artifact reduction (NMAR) applied. The simulated metal density was increased to investigate the effects of increasing photon starvation. The quantitative error and standard deviation in regions of the phantom were compared across the investigated algorithms. The ability of cOSSCIR to reproduce the soft‐tissue texture, while reducing metal artifacts, was quantitatively evaluated.Results: Noiseless simulations demonstrated the convergence of the cOSSCIR and MLE+TV algorithms to the correct basis maps in the presence of beam‐hardening effects. When noise was simulated, cOSSCIR demonstrated a quantitative error of −1 HU, compared to 2 HU error for the MLE+TV algorithm and −154 HU error for the nonspectral TV+NMAR algorithm. For the cOSSCIR algorithm, the standard deviation in the central iodine region of interest was 20 HU, compared to 299 HU for the MLE+TV algorithm, 41 HU for the MLE+TV+Mask algorithm that excluded rays through metal, and 55 HU for the nonspectral TV+NMAR algorithm. Increasing levels of photon starvation did not impact the bias or standard deviation of the cOSSCIR images. cOSSCIR was able to reproduce the soft‐tissue texture when an appropriate regularization constraint value was selected.Conclusions: By directly inverting photon‐counting CT data into basis maps using an accurate physics‐based forward model and a constrained optimization algorithm, cOSSCIR avoids metal artifacts due to beam hardening, noise, and photon starvation. The cOSSCIR algorithm demonstrated improved stability and accuracy compared to a two‐step method of decomposition followed by reconstruction.


  • Technical note: Evaluation of a V‐Net autosegmentation algorithm for pediatric CT scans: Performance, generalizability, and application to patient‐specific CT dosimetry

    by Philip M. Adamson, Vrunda Bhattbhatt, Sara Principi, Surabhi Beriwal, Linda S. Strain, Michael Offe, Adam S. Wang, Nghia‐Jack Vo, Taly Gilat Schmidt, Petr Jordan

    Purpose: This study developed and evaluated a fully convolutional network (FCN) for pediatric CT organ segmentation and investigated the generalizability of the FCN across image heterogeneities such as CT scanner model protocols and patient age. We also evaluated the autosegmentation models as part of a software tool for patient‐specific CT dose estimation.Methods: A collection of 359 pediatric CT datasets with expert organ contours were used for model development and evaluation. Autosegmentation models were trained for each organ using a modified FCN 3D V‐Net. An independent test set of 60 patients was withheld for testing. To evaluate the impact of CT scanner model protocol and patient age heterogeneities, separate models were trained using a subset of scanner model protocols and pediatric age groups. Train and test sets were split to answer questions about the generalizability of pediatric FCN autosegmentation models to unseen age groups and scanner model protocols, as well as the merit of scanner model protocol or age‐group‐specific models. Finally, the organ contours resulting from the autosegmentation models were applied to patient‐specific dose maps to evaluate the impact of segmentation errors on organ dose estimation.Results: Results demonstrate that the autosegmentation models generalize to CT scanner acquisition and reconstruction methods which were not present in the training dataset. While models are not equally generalizable across age groups, age‐group‐specific models do not hold any advantage over combining heterogeneous age groups into a single training set. Dice similarity coefficient (DSC) and mean surface distance results are presented for 19 organ structures, for example, median DSC of 0.52 (duodenum), 0.74 (pancreas), 0.92 (stomach), and 0.96 (heart). The FCN models achieve a mean dose error within 5% of expert segmentations for all 19 organs except for the spinal canal, where the mean error was 6.31%.Conclusions: Overall, these results are promising for the adoption of FCN autosegmentation models for pediatric CT, including applications for patient‐specific CT dose estimation.


  • Pediatric chest‐abdomen‐pelvis and abdomen‐pelvis CT images with expert organ contours

    by Petr Jordan, Philip M. Adamson, Vrunda Bhattbhatt, Surabhi Beriwal, Sangyu Shen, Oskar Radermecker, Supratik Bose, Linda S. Strain, Michael Offe, David Fraley, et al.

    Purpose: Organ autosegmentation efforts to date have largely been focused on adult populations, due to limited availability of pediatric training data. Pediatric patients may present additional challenges for organ segmentation. This paper describes a dataset of 359 pediatric chest‐abdomen‐pelvis and abdomen‐pelvis Computed Tomography (CT) images with expert contours of up to 29 anatomical organ structures to aid in the evaluation and development of autosegmentation algorithms for pediatric CT imaging.Acquisition and validation methods: The dataset collection consists of axial CT images in Digital Imaging and Communications in Medicine (DICOM) format of 180 male and 179 female pediatric chest‐abdomen‐pelvis or abdomen‐pelvis exams acquired from one of three CT scanners at Children's Wisconsin. The datasets represent random pediatric cases based upon routine clinical indications. Subjects ranged in age from 5 days to 16 years, with a mean age of 7 years. The CT acquisition, contrast, and reconstruction protocols varied across the scanner models and patients, with specifications available in the DICOM headers. Expert contours were manually labeled for up to 29 organ structures per subject. Not all contours are available for all subjects, due to limited field of view or unreliable contouring due to high noise.Data format and usage notes: The data are available on The Cancer Imaging Archive (TCIA_ (https://www.cancerimagingarchive.net/) under the collection Pediatric‐CT‐SEG. The axial CT image slices for each subject are available in DICOM format. The expert contours are stored in a single DICOM RTSTRUCT file for each subject. The contour names are listed in Table 2.Potential applications: This dataset will enable the evaluation and development of organ autosegmentation algorithms for pediatric populations, which exhibit variations in organ shape and size across age. Automated organ segmentation from CT images has numerous applications including radiation therapy, diagnostic tasks, surgical planning, and patient‐specific organ dose estimation.


  • Open Access

    Experimental study of photon-counting CT neural network material decomposition under conditions of pulse pileup

    by Parker J. B. Jenkins, Taly Gilat Schmidt

    Spectral CT with photon-counting detectors has demonstrated potential for improved image quality and quantitative material decomposition compared with conventional CT imaging methods.1 For example, photon-counting detectors have demonstrated improved spatial resolution,2 reduced electronic noise,3 and the ability to quantify K-edge contrast agents using three-material decomposition.4 – 6 However, photon-counting detectors are subject to nonideal effects, such as charge sharing and pulse pileup, which misrepresent the number and energy of detected photons.7,8 These spectral nonidealities can degrade the accuracy and precision of material decomposition estimates.


  • Open Access

    Rapid assessment of breast tumor margins using deep ultraviolet fluorescence scanning microscopy

    by Tongtong Lu, Julie M. Jorns, Mollie Patton, Renee Fisher, Amanda Emmrich, Todd Doehring, Taly Gilat Schmidt, Dong Hye Ye, Tina Yen, Bing Yu

    Breast cancer afflicts millions of women in the United States and worldwide, of whom about a half to two-thirds will undergo breast conserving surgery (BCS) or lumpectomy.1,2 The goal of BCS is to completely remove the tumor with a narrow rim/margin of normal, unaffected breast tissue while preserving as much normal tissue as possible. Although outcomes have improved over the last decade, women with positive margins (cancer cells at the surface of the surgical specimen) still have at least a twofold increased risk of cancer recurrence,3 – 5 and thus, are often recommended to undergo additional surgery to achieve clear margins. Additional surgery is associated with additional discomfort, surgical complications, worse cosmesis, and additional emotional stress, time, and financial burdens to patients and their caregivers.6,7 The current re-excision rate of BCS in the United States is 14% to 18% and highly variable among surgeons, ranging from 0% to 92%.2,8 – 12 One reason why women with positive margins need to undergo re-excision surgery is that definitive pathologic margin status is typically not available until several days after surgery.