Late fusion of deep learning and handcrafted visual features for biomedical image modality classification
Open Access
- 1 February 2019
- journal article
- research article
- Published by Institution of Engineering and Technology (IET) in IET Image Processing
- Vol. 13 (2) , 382-391
Abstract
Much of medical knowledge is stored in the biomedical literature, collected in archives like PubMed Central that continue to grow rapidly. A significant part of this knowledge is contained in images with limited metadata available which makes it difficult to explore the visual knowledge in the biomedical literature. Thus, extraction of metadata from visual content is important. One important piece of metadata is the type of the image, which could be one of the various medical imaging modalities such as X‐ray, computed tomography or magnetic resonance images and also of general graphs that are frequent in the literature. This study explores a late, score‐based fusion of several deep convolutional neural networks with a traditional hand‐crafted bag of visual words classifier to classify images from the biomedical literature into image types or modalities. It achieved a classification accuracy of 85.51% on the ImageCLEF 2013 modality classification task, which is better than the best visual methods in the challenge that the data were produced for, and similar compared to mixed methods that make use of both visual and textual information. It achieved similarly good results of 84.23 and 87.04% classification accuracy before and after augmentation, respectively, on the related ImageCLEF 2016 subfigure classification task.Keywords
This publication has 24 references indexed in Scilit:
- Deep learningNature, 2015
- ImageNet Large Scale Visual Recognition ChallengeInternational Journal of Computer Vision, 2015
- Literature-based biomedical image classification and retrievalComputerized Medical Imaging and Graphics, 2015
- Improved medical image modality classification using a combination of visual and textual featuresComputerized Medical Imaging and Graphics, 2015
- Applying latent semantic analysis to large-scale medical image databasesComputerized Medical Imaging and Graphics, 2015
- Fusion Techniques in Biomedical Information Retrieval2014
- Automatic classification of medical X‐ray images: hybrid generative‐discriminative approachIET Image Processing, 2013
- Automatic classification of medical X‐ray images using a bag of visual wordsIET Computer Vision, 2013
- A Survey on Visual Information Search Behavior and Requirements of RadiologistsMethods of Information in Medicine, 2012
- LIBSVMACM Transactions on Intelligent Systems and Technology, 2011