Semantic and Visual Enrichment Hierarchical Network for Medical Image Report Generation
Authors:
- 1 March 2022
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 738-743
Abstract
This paper highlights a novel medical image report generator named SVEH-Net (Semantic and Visual Enrichment Hierarchical Network), which is based on the encoder-decoder framework and attention mechanism. With the consideration of semantic, visual, and tag features fusion, an image feature encoding (IFE) module is introduced to provide global image features for the decoder, and a hierarchical decoder (H-Decoder) which can fusion all semantic and visual features and generate two reports at one time is proposed. In the experiments, our proposed models are evaluated on the Indiana University Chest X-ray radiology report dataset (IU X-ray) and PEIR Gross dataset. On the both two datasets, our model outperforms the state-of-the-art method in BLEU-1/2/3/4, METEOR, and ROUGE-L scores.Keywords
This publication has 16 references indexed in Scilit:
- Automatic Radiology Report Generation Based on Multi-view Image Fusion and Medical Concept Enrichment2019
- Automatic Generation of Pulmonary Radiology Reports with Semantic Tags2019
- Knowledge-Driven Encode, Retrieve, Paraphrase for Medical Image Report GenerationProceedings of the AAAI Conference on Artificial Intelligence, 2019
- Attention-Based Abnormal-Aware Fusion Network for Radiology Report Generation2019
- A Survey on Biomedical Image Captioning2019
- Deep Residual Learning for Image Recognition2016
- Preparing a collection of radiology examinations for distribution and retrievalJournal of the American Medical Informatics Association, 2015
- CIDEr: Consensus-based image description evaluation2015
- Meteor Universal: Language Specific Translation Evaluation for Any Target Language2014
- Automatic evaluation of machine translation quality using longest common subsequence and skip-bigram statistics2004