Zefan Zhang
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Research Subjects
Zefan Zhang has a total of 12 co-authors. Here are the top co-authors by number of shared publications:
- Tian Bai
College of Computer Science and Technology, Ministry of Education, Key Laboratory of Symbolic Computation and Knowledge Engineering, Jilin University, Changchun, 130012, China
3 shared publications - Chunping Liu
School of Computer Science and Technology, Soochow University, Suzhou, China
2 shared publications - Weiqi Zhang
College of Computer Science and Technology, Jilin University, Changchun, China and Key Laboratory of Symbolic Computation and Knowledge Engineering, Ministry of Education, Jilin University, Changchun, China
2 shared publications - Yanhui Li
College of Computer Science and Technology, Jilin University, Changchun, China and Key Laboratory of Symbolic Computation and Knowledge Engineering, Ministry of Education, Jilin University, Changchun, China
2 shared publications - Yi Ji
School of Computer Science and Technology, Soochow University, Suzhou, China
2 shared publications Chunping Liu
1 shared publication
Latest Publications
- Open Access
- 11 October 2026
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Multimedia Computing, Communications, and Applications
Implicit and Explicit Visual Information Aware Multimodal Relation Extraction
Recently, Multimodal Relation Extraction (MRE) has achieved great improvements. The inherent abstraction and variability of visual content make it challenging to extract relational information in multimodal relation extraction. Beyond identifying explicit visual entities for filtering out noisy information, it is also essential to capture implicit visual background knowledge that aids relation reasoning. Collectively, these factors are critical in alleviating the error sensitivity issues caused by irrelevant visual information. In light of this, we propose an Implicit and Explicit Visual Information Aware Network (IEVAN) for Multimodal Relation Extraction. Specifically, the Caption-Aware Module (CAM) hierarchically aligns the fine-grained visual entities and textual entities for reasoning. The Knowledge-Aware Module (KAM) supplements relation extraction with background knowledge. In addition, for preserving crucial information within different modalities, we leverage a Mutual Information Maximization method to regulate the multimodal reasoning modules. Experiments show that our model outperforms the state-of-the-art MRE models on the benchmark dataset MNRE and MORE. Further ablation studies prove the effective performance of our methods.
- 11 October 2026
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Multimedia Computing, Communications, and Applications
Open AccessImplicit and Explicit Visual Information Aware Multimodal Relation Extraction
Recently, Multimodal Relation Extraction (MRE) has achieved great improvements. The inherent abstraction and variability of visual content make it challenging to extract relational information in multimodal relation extraction. Beyond identifying explicit visual entities for filtering out noisy information, it is also essential to capture implicit visual background knowledge that aids relation reasoning. Collectively, these factors are critical in alleviating the error sensitivity issues caused by irrelevant visual information. In light of this, we propose an Implicit and Explicit Visual Information Aware Network (IEVAN) for Multimodal Relation Extraction. Specifically, the Caption-Aware Module (CAM) hierarchically aligns the fine-grained visual entities and textual entities for reasoning. The Knowledge-Aware Module (KAM) supplements relation extraction with background knowledge. In addition, for preserving crucial information within different modalities, we leverage a Mutual Information Maximization method to regulate the multimodal reasoning modules. Experiments show that our model outperforms the state-of-the-art MRE models on the benchmark dataset MNRE and MORE. Further ablation studies prove the effective performance of our methods.
- Open Access
- 27 October 2025
- Published by Association for Computing Machinery (ACM)
- p. 8537-8546
Video-Level Multimodal Relation Extraction with Event-Entity Semantic Consistency
- 27 October 2025
- Published by Association for Computing Machinery (ACM)
- p. 8537-8546
Open AccessVideo-Level Multimodal Relation Extraction with Event-Entity Semantic Consistency
- Open Access
- 28 October 2024
- Published by Association for Computing Machinery (ACM)
- p. 1148-1157
Caption-Aware Multimodal Relation Extraction with Mutual Information Maximization
Multimodal Relation Extraction (MRE) has achieved great improvements. However, modern MRE models are easily affected by irrelevant objects during multimodal alignment which are called error sensitivity issues. The main reason is that visual features are not fully aligned with textual features and the reasoning process may suppress redundant and noisy information at the risk of losing critical information. In light of this, we propose a Caption-Aware Multimodal Relation Extraction Network with Mutual Information Maximization (CAMIM). Specifically, we first generate detailed image captions through the Large Language Model (LLM). Then, the Caption-Aware Module (CAM) hierarchically aligns the fine-grained visual entities and textual entities for reasoning. In addition, for preserving crucial information within different modalities, we leverage a Mutual Information Maximization method to regulate the multimodal reasoning module. Experiments show that our model outperforms the state-of-the-art MRE models on the benchmark dataset MNRE. Further ablation studies prove the pluggable and effective performance of our Caption-Aware Module and Mutual Information Maximization method. Our code is available at https://github.com/zefanZhang-cn/CAMIM.
- 28 October 2024
- Published by Association for Computing Machinery (ACM)
- p. 1148-1157
Open AccessCaption-Aware Multimodal Relation Extraction with Mutual Information Maximization
Multimodal Relation Extraction (MRE) has achieved great improvements. However, modern MRE models are easily affected by irrelevant objects during multimodal alignment which are called error sensitivity issues. The main reason is that visual features are not fully aligned with textual features and the reasoning process may suppress redundant and noisy information at the risk of losing critical information. In light of this, we propose a Caption-Aware Multimodal Relation Extraction Network with Mutual Information Maximization (CAMIM). Specifically, we first generate detailed image captions through the Large Language Model (LLM). Then, the Caption-Aware Module (CAM) hierarchically aligns the fine-grained visual entities and textual entities for reasoning. In addition, for preserving crucial information within different modalities, we leverage a Mutual Information Maximization method to regulate the multimodal reasoning module. Experiments show that our model outperforms the state-of-the-art MRE models on the benchmark dataset MNRE. Further ablation studies prove the pluggable and effective performance of our Caption-Aware Module and Mutual Information Maximization method. Our code is available at https://github.com/zefanZhang-cn/CAMIM.
- 12 October 2023
- Published by Elsevier in Journal of Visual Communication and Image Representation
- Vol. 97
Infer unseen from seen: Relation regularized zero-shot visual dialog
- 12 October 2023
- Published by Elsevier in Journal of Visual Communication and Image Representation
- Vol. 97
Infer unseen from seen: Relation regularized zero-shot visual dialog
- Open Access
- 12 June 2023
- Published by Association for Computing Machinery (ACM)
- p. 253-261
Knowledge-Aware Causal Inference Network for Visual Dialog
- 12 June 2023
- Published by Association for Computing Machinery (ACM)
- p. 253-261
Open AccessKnowledge-Aware Causal Inference Network for Visual Dialog
- 29 November 2022
- Published by Springer Nature in International Journal of Multimedia Information Retrieval
- Vol. 11 (4) , 567-576
Multi-aware coreference relation network for visual dialog
- 29 November 2022
- Published by Springer Nature in International Journal of Multimedia Information Retrieval
- Vol. 11 (4) , 567-576
Multi-aware coreference relation network for visual dialog