How much data do I need? A case study on medical data
Authors:
- 15 December 2023
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 3688-3697
Abstract
The collection of data to train a Deep Learning network is costly in terms of effort and resources. In many cases, especially in a medical context, it may have detrimental impacts. Such as requiring invasive medical procedures or processes which could in themselves cause medical harm. However, Deep Learning is seen as a data hungry method. Here, we look at two commonly held adages i) more data gives better results and ii) transfer learning will aid you when you don’t have enough data. These are widely assumed to be true and used as evidence for choosing how to solve a problem when Deep Learning is involved. We evaluate six medical datasets and six general datasets. Training a ResNet18 network on varying subsets of these datasets to evaluate ‘more data gives better results’. We take eleven of these datasets as the sources for Transfer Learning on subsets of the twelfth dataset - Chest - in order to determine whether Transfer Learning is universally beneficial. We go further to see whether multi-stage Transfer Learning provides a consistent benefit. Our analysis shows that the real situation is more complex than these simple adages - more data could lead to a case of diminishing returns and an incorrect choice of dataset for transfer learning can lead to worse performance, with datasets which we would consider highly similar to the Chest dataset giving worse results than datasets which are more dissimilar. Multi-stage transfer learning likewise reveals complex relationships between datasets.Keywords
All Related Versions
This publication has 23 references indexed in Scilit:
- Detection of Rare Genetic Diseases using Facial 2D Images with Transfer Learning2018
- Breast Cancer Diagnosis in Digital Breast Tomosynthesis: Effects of Training Sample Size on Multi-Stage Transfer Learning Using Deep Neural NetsIEEE Transactions on Medical Imaging, 2018
- Road User Abnormal Trajectory Detection using a Deep Autoencoder2018
- Convolutional neural networks: an overview and application in radiologyInsights into Imaging, 2018
- Learning without ForgettingIEEE Transactions on Pattern Analysis and Machine Intelligence, 2017
- Modality-bridge transfer learning for medical image classification2017
- Revisiting Unreasonable Effectiveness of Data in Deep Learning Era2017
- ChestX-Ray8: Hospital-Scale Chest X-Ray Database and Benchmarks on Weakly-Supervised Classification and Localization of Common Thorax Diseases2017
- Transfer Learning and Deep Feature Extraction for Planktonic Image Data Sets2017
- ImageNet: A large-scale hierarchical image database2009