Multi-modal dataset creation for federated learning with DICOM-structured reports
Open Access
- 3 February 2025
- journal article
- research article
- Published by Springer Nature in International Journal of Computer Assisted Radiology and Surgery
- Vol. 20 (3) , 485-495
Abstract
Purpose Federated training is often challenging on heterogeneous datasets due to divergent data storage options, inconsistent naming schemes, varied annotation procedures, and disparities in label quality. This is particularly evident in the emerging multi-modal learning paradigms, where dataset harmonization including a uniform data representation and filtering options are of paramount importance. Methods DICOM-structured reports enable the standardized linkage of arbitrary information beyond the imaging domain and can be used within Python deep learning pipelines with . Building on this, we developed an open platform for data integration with interactive filtering capabilities, thereby simplifying the process of creation of patient cohorts over several sites with consistent multi-modal data. Results In this study, we extend our prior work by showing its applicability to more and divergent data types, as well as streamlining datasets for federated training within an established consortium of eight university hospitals in Germany. We prove its concurrent filtering ability by creating harmonized multi-modal datasets across all locations for predicting the outcome after minimally invasive heart valve replacement. The data include imaging and waveform data (i.e., computed tomography images, electrocardiography scans) as well as annotations (i.e., calcification segmentations, and pointsets), and metadata (i.e., prostheses and pacemaker dependency). Conclusion Structured reports bridge the traditional gap between imaging systems and information systems. Utilizing the inherent DICOM reference system arbitrary data types can be queried concurrently to create meaningful cohorts for multi-centric data analysis. The graphical interface as well as example structured report templates are available at https://github.com/Cardio-AI/fl-multi-modal-dataset-creation.Keywords
Funding Information
- Deutsches Zentrum für Herz-Kreislaufforschung
- Klaus Tschira Stiftung
- Bundesministerium für Forschung, Technologie und Raumfahrt (01KD2215D)
- Carl-Zeiss-Stiftung (P2022-08-01)
This publication has 24 references indexed in Scilit:
- HL7 Fast Healthcare Interoperability Resources (HL7 FHIR) in digital healthcare ecosystems for chronic disease management: Scoping reviewInternational Journal of Medical Informatics, 2024
- Towards Unified Multi-modal Dataset Creation for Deep Learning Utilizing Structured Reports2024
- Enrichment of lung cancer computed tomography collections with AI-derived annotationsScientific Data, 2024
- A Review of the Application of Multi-modal Deep Learning in Medicine: Bibliometrics and Future DirectionsInternational Journal of Computational Intelligence Systems, 2023
- Highdicom: a Python Library for Standardized Encoding of Image Annotations and Machine Learning Model Outputs in Pathology and RadiologyJournal of Digital Imaging, 2022
- 88. Jahrestagung der Deutschen Gesellschaft für Kardiologie – Herz- und Kreislaufforschung e.V.Clinical Research in Cardiology, 2022
- Structured reporting in radiology: a systematic review to explore its potentialEuropean Radiology, 2021
- CHAOS Challenge - combined (CT-MR) healthy abdominal organ segmentationMedical Image Analysis, 2021
- A Foundation for Enterprise Imaging: HIMSS-SIIM Collaborative White PaperJournal of Digital Imaging, 2016
- Benefits of the DICOM Structured ReportJournal of Digital Imaging, 2006