Evaluation of the dependence of radiomic features on the machine learning model
Open Access
- 24 February 2022
- journal article
- research article
- Published by Springer Nature in Insights into Imaging
- Vol. 13 (1) , 1-11
Abstract
Background: In radiomic studies, several models are often trained with different combinations of feature selection methods and classifiers. The features of the best model are usually considered relevant to the problem, and they represent potential biomarkers. Features selected from statistically similarly performing models are generally not studied. To understand the degree to which the selected features of these statistically similar models differ, 14 publicly available datasets, 8 feature selection methods, and 8 classifiers were used in this retrospective study. For each combination of feature selection and classifier, a model was trained, and its performance was measured with AUC-ROC. The best-performing model was compared to other models using a DeLong test. Models that were statistically similar were compared in terms of their selected features. Results: Approximately 57% of all models analyzed were statistically similar to the best-performing model. Feature selection methods were, in general, relatively unstable (0.58; range 0.35–0.84). The features selected by different models varied largely (0.19; range 0.02–0.42), although the selected features themselves were highly correlated (0.71; range 0.4–0.92). Conclusions: Feature relevance in radiomics strongly depends on the model used, and statistically similar models will generally identify different features as relevant. Considering features selected by a single model is misleading, and it is often not possible to directly determine whether such features are candidate biomarkers.Keywords
Funding Information
- Universitätsklinikum Essen
This publication has 43 references indexed in Scilit:
- Radiomics of CT Features May Be Nonreproducible and Redundant: Influence of CT Acquisition ParametersRadiology, 2018
- Texture analysis and machine learning of non-contrast T1-weighted MR images in patients with hypertrophic cardiomyopathy—Preliminary resultsEuropean Journal of Radiology, 2018
- 18F-fluorodeoxyglucose positron-emission tomography (FDG-PET)-Radiomics of metastatic lymph nodes and primary tumor in non-small cell lung cancer (NSCLC) – A prospective externally validated studyPLOS ONE, 2018
- Radiomics: the bridge between medical imaging and personalized medicineNature Reviews Clinical Oncology, 2017
- Applications and limitations of radiomicsPhysics in Medicine & Biology, 2016
- Exploratory Study to Identify Radiomics Classifiers for Lung Cancer HistologyFrontiers in Oncology, 2016
- Radiomics: Images Are More than Pictures, They Are DataRadiology, 2016
- Machine Learning methods for Quantitative Radiomic BiomarkersScientific Reports, 2015
- CT-based radiomic signature predicts distant metastasis in lung adenocarcinomaRadiotherapy and Oncology, 2015
- Decoding tumour phenotype by noninvasive imaging using a quantitative radiomics approachNature Communications, 2014