He Zhao
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
He Zhao has a total of 4 co-authors. Here are the top co-authors by number of shared publications:
- Fengzhi Zhao
College of Computer Science and Technology, Jilin University, Changchun 130012, China
2 shared publications - Tao Wang
College of Computer Science and Technology, Jilin University, Changchun 130012, China
2 shared publications - Zhezhou Yu
College of Computer Science and Technology, Jilin University, Changchun 130012, China
2 shared publications - Tian Bai
College of Computer Science and Technology, Ministry of Education, Key Laboratory of Symbolic Computation and Knowledge Engineering, Jilin University, Changchun, 130012, China
1 shared publication
Latest Publications
- Open Access
Meshed Context-Aware Beam Search for Image Captioning
Beam search is a commonly used algorithm in image captioning to improve the accuracy and robustness of generated captions by finding the optimal word sequence. However, it mainly focuses on the highest-scoring sequence at each step, often overlooking the broader image context, which can lead to suboptimal results. Additionally, beam search tends to select similar words across sequences, causing repetitive and less diverse output. These limitations suggest that, while effective, beam search can be further improved to better capture the richness and variety needed for high-quality captions. To address these issues, this paper presents meshed context-aware beam search (MCBS). In MCBS for image captioning, the generated caption context is dynamically used to influence the image attention mechanism at each decoding step, ensuring that the model focuses on different regions of the image to produce more coherent and contextually appropriate captions. Furthermore, a penalty coefficient is introduced to discourage the generation of repeated words. Through extensive testing and ablation studies across various models, our results show that MCBS significantly enhances overall model performance.
Open AccessMeshed Context-Aware Beam Search for Image Captioning
Beam search is a commonly used algorithm in image captioning to improve the accuracy and robustness of generated captions by finding the optimal word sequence. However, it mainly focuses on the highest-scoring sequence at each step, often overlooking the broader image context, which can lead to suboptimal results. Additionally, beam search tends to select similar words across sequences, causing repetitive and less diverse output. These limitations suggest that, while effective, beam search can be further improved to better capture the richness and variety needed for high-quality captions. To address these issues, this paper presents meshed context-aware beam search (MCBS). In MCBS for image captioning, the generated caption context is dynamically used to influence the image attention mechanism at each decoding step, ensuring that the model focuses on different regions of the image to produce more coherent and contextually appropriate captions. Furthermore, a penalty coefficient is introduced to discourage the generation of repeated words. Through extensive testing and ablation studies across various models, our results show that MCBS significantly enhances overall model performance.
- 7 December 2023
- Published by Springer Nature in Applied Intelligence
- Vol. 54 (1) , 231-245
Integrating grid features and geometric coordinates for enhanced image captioning
- 7 December 2023
- Published by Springer Nature in Applied Intelligence
- Vol. 54 (1) , 231-245
Integrating grid features and geometric coordinates for enhanced image captioning