Rainer Schubert
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Rainer Schubert has a total of 80 co-authors. Here are the top co-authors by number of shared publications:
- Karl Fritscher
VASCage – Centre on Clinical Stroke Research
6 shared publications - Martin Welk
UMIT TIROL -Private University for Health Sciences and Health Technology Eduard-Wallnöfer-Zentrum 1 6060 Hall/ Tyrol Austria
5 shared publications - Andreas Pommert
University Hospital Hamburg-Eppendorf
4 shared publications - Christian Gapp
Institute of Biomedical Image Analysis, UMIT TIROL -Private University for Health Sciences and Health Technology
4 shared publications - Elias Tappeiner
Institute of Biomedical Image Analysis, UMIT TIROL – Private University for Health Sciences and Health Technology, Hall in Tirol, Austria
4 shared publications K. H. Höhne
University Hospital Eppendorf
4 shared publications
Latest Publications
- 15 August 2026
- Published by Springer Nature
- p. 677-693
XSRD-Net: EXplainable Stroke Relapse Detection
- Open Access
- 2 October 2025
- Published by Springer Nature in International Journal of Computer Assisted Radiology and Surgery
- Vol. 21 (3) , 507-517
What are you looking at? Modality contribution in multimodal medical deep learning
Purpose: High dimensional, multimodal data can nowadays be analyzed by huge deep neural networks with little effort. Several fusion methods for bringing together different modalities have been developed. Given the prevalence of high-dimensional, multimodal patient data in medicine, the development of multimodal models marks a significant advancement. However, how these models process information from individual sources in detail is still underexplored. Methods: To this end, we implemented an occlusion-based modality contribution method that is both model- and performance agnostic. This method quantitatively measures the importance of each modality in the dataset for the model to fulfill its task. We applied our method to three different multimodal medical problems for experimental purposes. Results: Herein we found that some networks have modality preferences that tend to unimodal collapses, while some datasets are imbalanced from the ground up. Moreover, we provide fine-grained quantitative and visual attribute importance for each modality. Conclusion: Our metric offers valuable insights that can support the advancement of multimodal model development and dataset creation. By introducing this method, we contribute to the growing field of interpretability in deep learning for multimodal research. This approach helps to facilitate the integration of multimodal AI into clinical practice. Our code is publicly available at https://github.com/ChristianGappGit/MC_MMD
- 2 October 2025
- Published by Springer Nature in International Journal of Computer Assisted Radiology and Surgery
- Vol. 21 (3) , 507-517
Open AccessWhat are you looking at? Modality contribution in multimodal medical deep learning
Purpose: High dimensional, multimodal data can nowadays be analyzed by huge deep neural networks with little effort. Several fusion methods for bringing together different modalities have been developed. Given the prevalence of high-dimensional, multimodal patient data in medicine, the development of multimodal models marks a significant advancement. However, how these models process information from individual sources in detail is still underexplored. Methods: To this end, we implemented an occlusion-based modality contribution method that is both model- and performance agnostic. This method quantitatively measures the importance of each modality in the dataset for the model to fulfill its task. We applied our method to three different multimodal medical problems for experimental purposes. Results: Herein we found that some networks have modality preferences that tend to unimodal collapses, while some datasets are imbalanced from the ground up. Moreover, we provide fine-grained quantitative and visual attribute importance for each modality. Conclusion: Our metric offers valuable insights that can support the advancement of multimodal model development and dataset creation. By introducing this method, we contribute to the growing field of interpretability in deep learning for multimodal research. This approach helps to facilitate the integration of multimodal AI into clinical practice. Our code is publicly available at https://github.com/ChristianGappGit/MC_MMD
- Open Access
XSRD-Net: EXplainable Stroke Relapse Detection
Stroke is the second most frequent cause of death world wide with an annual mortality of around 5.5 million. Recurrence rates of stroke are between 5 and 25% in the first year. As mortality rates for relapses are extraordinarily high (40%) it is of utmost importance to reduce the recurrence rates. We address this issue by detecting patients at risk of stroke recurrence at an early stage in order to enable appropriate therapy planning. To this end we collected 3D intracranial CTA image data and recorded concomitant heart diseases, the age and the gender of stroke patients between 2010 and 2024. We trained single- and multimodal deep learning based neural networks for binary relapse detection (Task 1) and for relapse free survival (RFS) time prediction together with a subsequent classification (Task 2). The separation of relapse from non-relapse patients (Task 1) could be solved with tabular data (AUC on test dataset: 0.84). However, for the main task, the regression (Task 2), our multimodal XSRD-net processed the modalities vision:tabular with 0.68:0.32 according to modality contribution measures. The c-index with respect to relapses for the multimodal model reached 0.68, and the AUC is 0.71 for the test dataset. Final, deeper interpretability analysis results could highlight a link between both heart diseases (tabular) and carotid arteries (vision) for the detection of relapses and the prediction of the RFS time. This is a central outcome that we strive to strengthen with ongoing data collection and model retraining.
Open AccessXSRD-Net: EXplainable Stroke Relapse Detection
Stroke is the second most frequent cause of death world wide with an annual mortality of around 5.5 million. Recurrence rates of stroke are between 5 and 25% in the first year. As mortality rates for relapses are extraordinarily high (40%) it is of utmost importance to reduce the recurrence rates. We address this issue by detecting patients at risk of stroke recurrence at an early stage in order to enable appropriate therapy planning. To this end we collected 3D intracranial CTA image data and recorded concomitant heart diseases, the age and the gender of stroke patients between 2010 and 2024. We trained single- and multimodal deep learning based neural networks for binary relapse detection (Task 1) and for relapse free survival (RFS) time prediction together with a subsequent classification (Task 2). The separation of relapse from non-relapse patients (Task 1) could be solved with tabular data (AUC on test dataset: 0.84). However, for the main task, the regression (Task 2), our multimodal XSRD-net processed the modalities vision:tabular with 0.68:0.32 according to modality contribution measures. The c-index with respect to relapses for the multimodal model reached 0.68, and the AUC is 0.71 for the test dataset. Final, deeper interpretability analysis results could highlight a link between both heart diseases (tabular) and carotid arteries (vision) for the detection of relapses and the prediction of the RFS time. This is a central outcome that we strive to strengthen with ongoing data collection and model retraining.
- Open Access
- 3 March 2025
- Published by Springer Nature
- Vol. 15273, 87-98
Head and Neck Tumor Segmentation on MRIs with Fast and Resource-Efficient Staged nnU-Nets
MRI-guided radiotherapy (RT) planning offers key advantages over conventional CT-based methods, including superior soft tissue contrast and the potential for daily adaptive RT due to the reduction of the radiation burden. In the Head and Neck (HN) region labor-intensive and time-consuming tumor segmentation still limits full utilization of MRI-guided adaptive RT. The HN Tumor Segmentation for MR-Guided Applications 2024 challenge (HNTS-MRG) aims to improve automatic tumor segmentation on MRI images by providing a dataset with reference annotations for the tasks of pre-RT and mid-RT planning. In this work, we present our approach for the HNTS-MRG challenge. Based on the insights of a thorough literature review we implemented a fast and resource-efficient two-stage segmentation method using the nnU-Net architecture with residual encoders as a backbone. In our two-stage approach we use the segmentation results of a first training round to guide the sampling process for a second refinement stage. For the pre-RT task, we achieved competitive results using only the first-stage nnU-Net. For the mid-RT task, we could significantly increase the segmentation performance of the basic first stage nnU-Net by utilizing the prior knowledge of the pre-RT plan as an additional input for the second stage refinement network. As team alpinists we achieved an aggregated Dice Coefficient of 80.97 for the pre-RT and 69.84 for the mid-RT task on the online test set of the challenge. Our code and trained model weights for the two-stage nnU-Net approach with residual encoders are available at https://github.com/elitap/hntsmrg24.
- 3 March 2025
- Published by Springer Nature
- Vol. 15273, 87-98
Open AccessHead and Neck Tumor Segmentation on MRIs with Fast and Resource-Efficient Staged nnU-Nets
MRI-guided radiotherapy (RT) planning offers key advantages over conventional CT-based methods, including superior soft tissue contrast and the potential for daily adaptive RT due to the reduction of the radiation burden. In the Head and Neck (HN) region labor-intensive and time-consuming tumor segmentation still limits full utilization of MRI-guided adaptive RT. The HN Tumor Segmentation for MR-Guided Applications 2024 challenge (HNTS-MRG) aims to improve automatic tumor segmentation on MRI images by providing a dataset with reference annotations for the tasks of pre-RT and mid-RT planning. In this work, we present our approach for the HNTS-MRG challenge. Based on the insights of a thorough literature review we implemented a fast and resource-efficient two-stage segmentation method using the nnU-Net architecture with residual encoders as a backbone. In our two-stage approach we use the segmentation results of a first training round to guide the sampling process for a second refinement stage. For the pre-RT task, we achieved competitive results using only the first-stage nnU-Net. For the mid-RT task, we could significantly increase the segmentation performance of the basic first stage nnU-Net by utilizing the prior knowledge of the pre-RT plan as an additional input for the second stage refinement network. As team alpinists we achieved an aggregated Dice Coefficient of 80.97 for the pre-RT and 69.84 for the mid-RT task on the online test set of the challenge. Our code and trained model weights for the two-stage nnU-Net approach with residual encoders are available at https://github.com/elitap/hntsmrg24.
- Open Access
Multimodal Medical Disease Classification with LLaMA II
Medical patient data is always multimodal. Images, text, age, gender, histopathological data are only few examples for different modalities in this context. Processing and integrating this multimodal data with deep learning based methods is of utmost interest due to its huge potential for medical procedure such as diagnosis and patient treatment planning. In this work we retrain a multimodal transformer-based model for disease classification. To this end we use the text-image pair dataset from OpenI consisting of 2D chest X-rays associated with clinical reports. Our focus is on fusion methods for merging text and vision information extracted from medical datasets. Different architecture structures with a LLaMA II backbone model are tested. Early fusion of modality specific features creates better results with the best model reaching 97.10% mean AUC than late fusion from a deeper level of the architecture (best model: 96.67% mean AUC). Both outperform former classification models tested on the same multimodal dataset. The newly introduced multimodal architecture can be applied to other multimodal datasets with little effort and can be easily adapted for further research, especially, but not limited to, the field of medical AI.
Open AccessMultimodal Medical Disease Classification with LLaMA II
Medical patient data is always multimodal. Images, text, age, gender, histopathological data are only few examples for different modalities in this context. Processing and integrating this multimodal data with deep learning based methods is of utmost interest due to its huge potential for medical procedure such as diagnosis and patient treatment planning. In this work we retrain a multimodal transformer-based model for disease classification. To this end we use the text-image pair dataset from OpenI consisting of 2D chest X-rays associated with clinical reports. Our focus is on fusion methods for merging text and vision information extracted from medical datasets. Different architecture structures with a LLaMA II backbone model are tested. Early fusion of modality specific features creates better results with the best model reaching 97.10% mean AUC than late fusion from a deeper level of the architecture (best model: 96.67% mean AUC). Both outperform former classification models tested on the same multimodal dataset. The newly introduced multimodal architecture can be applied to other multimodal datasets with little effort and can be easily adapted for further research, especially, but not limited to, the field of medical AI.
- Open Access
- 1 January 2016
- Published by Walter de Gruyter GmbH in Biomedical Engineering / Biomedizinische Technik
- Vol. 61 (s1) , 48-69
Track G. Neural Signal Processing
- 1 January 2016
- Published by Walter de Gruyter GmbH in Biomedical Engineering / Biomedizinische Technik
- Vol. 61 (s1) , 48-69
Open AccessTrack G. Neural Signal Processing
- 1 January 2013
- Published by Springer Nature
- p. 508-519
Automated Quality Inspection of Microfluidic Chips Using Morphologic Techniques
- 1 January 2013
- Published by Springer Nature
- p. 508-519
Automated Quality Inspection of Microfluidic Chips Using Morphologic Techniques
- 29 April 2011
- Published by Georg Thieme Verlag KG in RöFo - Fortschritte auf dem Gebiet der Röntgenstrahlen und der bildgebenden Verfahren
- Vol. 183 (S 01) , FO_PO23
MRT-Befunde bei aseptischen Knochennekrosen der unteren Extremität: ein Überblick
Unter dem Oberbegriff aseptischer Knochennekrosen wird ein breites Spektrum von Erkrankungen und Entwicklungsstörungen des Skelettsystems zusammengefasst, das unter verschiedenen Gesichtspunkten eingeteilt werden kann. Nach der Ätiologie lassen sich 2 Hauptgruppen unterscheiden: Ischämische Nekrosen im engeren Sinne (avaskuläre Knochennekrosen) und Schädigungen durch mechanische Überbeanspruchung, die typischerweise aber nicht ausschließlich, osteotendinöse Verbindungen betreffen (Traktionsapophysitiden). Spätfolgen sind bei einigen Affektionen der unteren Extremitäten aufgrund der Belastung mit dem Körpergewicht häufiger als an den oberen Gliedmaßen. Ihr Auftreten kann durch eine rechtzeitige und stadiengerechte Behandlung oft vermieden oder zumindest verzögert werden. Die MRT eignet sich bei der Mehrzahl der aseptischen Knochennekrosen zur Früherkennung, Stadieneinteilung und Verlaufskontrolle. Die vorliegende Darstellung liefert einen systematischen Überblick typischer MRT-Befunde. Berücksichtigt werden dabei die häufigsten Lokalisationen im Bereich des knöchernen Beckens, des Hüftgelenks, des Kniegelenks sowie des Fußskeletts. Lernziele: Keywords: MRT, Aseptische Knochennekrosen, Osteochondrosen, Traktionsapophysitiden Korrespondierender Autor: Schubert R Radiologie am Europa-Center, Nürnberger Straße 67, 10787 Berlin E-Mail: dr.schubert@mvz-radiologie.de
- 29 April 2011
- Published by Georg Thieme Verlag KG in RöFo - Fortschritte auf dem Gebiet der Röntgenstrahlen und der bildgebenden Verfahren
- Vol. 183 (S 01) , FO_PO23
MRT-Befunde bei aseptischen Knochennekrosen der unteren Extremität: ein Überblick
Unter dem Oberbegriff aseptischer Knochennekrosen wird ein breites Spektrum von Erkrankungen und Entwicklungsstörungen des Skelettsystems zusammengefasst, das unter verschiedenen Gesichtspunkten eingeteilt werden kann. Nach der Ätiologie lassen sich 2 Hauptgruppen unterscheiden: Ischämische Nekrosen im engeren Sinne (avaskuläre Knochennekrosen) und Schädigungen durch mechanische Überbeanspruchung, die typischerweise aber nicht ausschließlich, osteotendinöse Verbindungen betreffen (Traktionsapophysitiden). Spätfolgen sind bei einigen Affektionen der unteren Extremitäten aufgrund der Belastung mit dem Körpergewicht häufiger als an den oberen Gliedmaßen. Ihr Auftreten kann durch eine rechtzeitige und stadiengerechte Behandlung oft vermieden oder zumindest verzögert werden. Die MRT eignet sich bei der Mehrzahl der aseptischen Knochennekrosen zur Früherkennung, Stadieneinteilung und Verlaufskontrolle. Die vorliegende Darstellung liefert einen systematischen Überblick typischer MRT-Befunde. Berücksichtigt werden dabei die häufigsten Lokalisationen im Bereich des knöchernen Beckens, des Hüftgelenks, des Kniegelenks sowie des Fußskeletts. Lernziele: Keywords: MRT, Aseptische Knochennekrosen, Osteochondrosen, Traktionsapophysitiden Korrespondierender Autor: Schubert R Radiologie am Europa-Center, Nürnberger Straße 67, 10787 Berlin E-Mail: dr.schubert@mvz-radiologie.de
- 1 March 2011
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- No. 19457928,p. 1189-1192
Hip fracture discrimination using 3D reconstructions from Dual-energy X-ray Absorptiometry
Due to the aging of our society, osteoporosis and the subsequent fractures is becoming an increasingly important problem. Currently, Dual-energy X-ray Absortiometry (DXA) derived areal Bone Mineral Density (BMD) is the gold standard for diagnosing osteoporosis, but is limited to a two-dimensional analysis. In this work, we propose a method to discriminate between healthy and fracture patients from DXA images by incorporating a 3D reconstruction method. A statistical model of the 3D shape and BMD distribution was registered onto the DXA scans of fracture and non-fracture patients. The resulting values for the scale and the first model parameter, together with the femoral neck areal BMD, was processed by Linear Discriminant Analysis and the discriminating accuracy was evaluated by generating the Receiver Operating Characteristic curve. The Area Under the Curve using the proposed method was approximately 0.080 and 0.063 higher than using only areal BMD, indicating an improved power of discrimination.
- 1 March 2011
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- No. 19457928,p. 1189-1192
Hip fracture discrimination using 3D reconstructions from Dual-energy X-ray Absorptiometry
Due to the aging of our society, osteoporosis and the subsequent fractures is becoming an increasingly important problem. Currently, Dual-energy X-ray Absortiometry (DXA) derived areal Bone Mineral Density (BMD) is the gold standard for diagnosing osteoporosis, but is limited to a two-dimensional analysis. In this work, we propose a method to discriminate between healthy and fracture patients from DXA images by incorporating a 3D reconstruction method. A statistical model of the 3D shape and BMD distribution was registered onto the DXA scans of fracture and non-fracture patients. The resulting values for the scale and the first model parameter, together with the femoral neck areal BMD, was processed by Linear Discriminant Analysis and the discriminating accuracy was evaluated by generating the Receiver Operating Characteristic curve. The Area Under the Curve using the proposed method was approximately 0.080 and 0.063 higher than using only areal BMD, indicating an improved power of discrimination.
- 23 June 2010
- Published by Springer Nature in International Journal of Computer Assisted Radiology and Surgery
- Vol. 5 (5) , 455-460
Analysis of the micro-migration of sliding hip screws by using point-based registration
The favored treatment for many hip fractures is a sliding hip screw, and its usage is expected to increase in the future. Failures can be reduced, and complications detected earlier by semi-automated CT image analysis. The most frequent failure is due to the screw cut-out from the femoral head. An image-based method was developed for early detection of complications and assessment of anchorage quality relative to implant model, bone quality or tip-apex distance (TAD). This method evaluates micro-migration using CT images acquired at different time points (immediately post-op and 3-month later). Serial CT image registration and transformation methods were applied, including point-based registration, to achieve semi-automated evaluations. Qualitative and quantitative validation of the image registration was performed with measurement mean error determination by different observers. The micro-migration evaluation by clinicians compared favorably with semi-automated image-based results. Semi-automatic evaluation of hip screw micro-migration using CT images is feasible and can aid observation of convalescence. The method may be amenable to full automation, a future goal for this work.
- 23 June 2010
- Published by Springer Nature in International Journal of Computer Assisted Radiology and Surgery
- Vol. 5 (5) , 455-460
Analysis of the micro-migration of sliding hip screws by using point-based registration
The favored treatment for many hip fractures is a sliding hip screw, and its usage is expected to increase in the future. Failures can be reduced, and complications detected earlier by semi-automated CT image analysis. The most frequent failure is due to the screw cut-out from the femoral head. An image-based method was developed for early detection of complications and assessment of anchorage quality relative to implant model, bone quality or tip-apex distance (TAD). This method evaluates micro-migration using CT images acquired at different time points (immediately post-op and 3-month later). Serial CT image registration and transformation methods were applied, including point-based registration, to achieve semi-automated evaluations. Qualitative and quantitative validation of the image registration was performed with measurement mean error determination by different observers. The micro-migration evaluation by clinicians compared favorably with semi-automated image-based results. Semi-automatic evaluation of hip screw micro-migration using CT images is feasible and can aid observation of convalescence. The method may be amenable to full automation, a future goal for this work.