Haitao Shi
Haitao Shi has a total of 25 co-authors. Here are the top co-authors by number of shared publications:
- Yiwei Wei
China University of Petroleum Beijing at Karamay, Karamay, China
2 shared publications - Ang Li
School of Petroleum, China University of Petroleum (Beijing) at Karamay Campus, Karamay 834000, China
1 shared publication - Chengyin Hu
China University of Petroleum‐Beijing at Karamay Karamay Xinjiang China
1 shared publication - Hengyang Zhou
College of Petroleum Engineering, China University of Petroleum (Beijing) at Karamay, Karamay, China
1 shared publication - Lingyan Bian
China University of Petroleum Beijing at Karamay, Karamay, China
1 shared publication - Liqiang Nie
Harbin Institute of Technology (Shenzhen), Shenzhen, China
1 shared publication
Latest Publications
- Open Access
- 13 October 2026
- Published by MDPI AG in Electronics
- Vol. 15 (20) , 4707
Prior-Guided Spatial–Frequency Interactive Network for Retinal Vessel Segmentation
Retinal vessel segmentation is essential for diagnosing eye diseases. However, complex multiscale vascular structures and low contrast make accurate segmentation difficult. Current methods often overlook structural priors and seldom explore spatial–frequency synergies, resulting in poor segmentation of tiny vessels. To address these issues, we propose a Prior-Guided Spatial–Frequency Interactive Network (PGSFI-Net) that operates in both spatial and frequency domains. The Sobel-Attention Prior module extracts vascular edge priors using a single-scale Sobel operator and spatial attention, followed by multi-scale prior injection to strengthen faint-vessel representations. The Four-Branch Selective Attention Aggregation module captures multi-directional vascular features through strip convolutions and multi-dimensional attention. The Multi-Branch Spatial–Frequency Fusion module achieves bidirectional feature complementation via cross-gating interaction. The Local–Global Fusion module recovers subtle vascular structures by combining differential CNN branches with channel self-attention. On DRIVE, CHASE_DB1, and STARE, PGSFI-Net achieves ACC/F1/AUC values of 0.9646/0.8136/0.9838, 0.9727/0.8290/0.9838, and 0.9746/0.8393/0.9884, respectively. These results demonstrate competitive performance against state-of-the-art methods across the evaluated datasets and challenging low-contrast cases.
- 13 October 2026
- Published by MDPI AG in Electronics
- Vol. 15 (20) , 4707
Open AccessPrior-Guided Spatial–Frequency Interactive Network for Retinal Vessel Segmentation
Retinal vessel segmentation is essential for diagnosing eye diseases. However, complex multiscale vascular structures and low contrast make accurate segmentation difficult. Current methods often overlook structural priors and seldom explore spatial–frequency synergies, resulting in poor segmentation of tiny vessels. To address these issues, we propose a Prior-Guided Spatial–Frequency Interactive Network (PGSFI-Net) that operates in both spatial and frequency domains. The Sobel-Attention Prior module extracts vascular edge priors using a single-scale Sobel operator and spatial attention, followed by multi-scale prior injection to strengthen faint-vessel representations. The Four-Branch Selective Attention Aggregation module captures multi-directional vascular features through strip convolutions and multi-dimensional attention. The Multi-Branch Spatial–Frequency Fusion module achieves bidirectional feature complementation via cross-gating interaction. The Local–Global Fusion module recovers subtle vascular structures by combining differential CNN branches with channel self-attention. On DRIVE, CHASE_DB1, and STARE, PGSFI-Net achieves ACC/F1/AUC values of 0.9646/0.8136/0.9838, 0.9727/0.8290/0.9838, and 0.9746/0.8393/0.9884, respectively. These results demonstrate competitive performance against state-of-the-art methods across the evaluated datasets and challenging low-contrast cases.
- 28 July 2026
- Published by Elsevier in Expert Systems with Applications
- Vol. 333
Tackling indirect expression with uncertainty-aware distribution optimization for hateful memes classification
- 28 July 2026
- Published by Elsevier in Expert Systems with Applications
- Vol. 333
Tackling indirect expression with uncertainty-aware distribution optimization for hateful memes classification
- Open Access
- 15 June 2026
- Published by Association for Computing Machinery (ACM)
- p. 1974-1983
GMAE: Gated Multi-signal Adaptive Ensemble for Targeted Transfer Attacks on Commercial Large Vision-Language Models
Despite strong performance on open-source large vision-language models (LVLMs), transfer-based targeted adversarial attacks often fail against closed-source commercial LVLMs. Existing methods typically rely on static perturbation allocation and uniformly weighted surrogate ensembles, limiting their ability to capture model-agnostic semantic vulnerabilities essential for cross-model transferability. Consequently, perturbations are frequently allocated to semantically irrelevant regions, reducing effectiveness under realistic black-box settings. To address this limitation, we propose the Gated Multi-signal Adaptive Ensemble (GMAE) framework, a novel transfer-based targeted attack that formulates adversarial generation as a joint optimization over a gated perturbation field and an adaptive ensemble objective. A learnable spatial gating mask dynamically reallocates perturbation budgets toward semantically vulnerable regions, while surrogate models are adaptively reweighted according to optimization difficulty and historical consistency, encouraging alignment on shared semantic weaknesses across heterogeneous architectures. To further enhance robustness, GMAE performs global semantic alignment in a symbolic discrete space, reducing the influence of non-essential feature variations and improving cross-model generalization. In addition, we introduce a local attention-guided multi-scale consistency mechanism that transfers fine-grained target textures into attention-identified sensitive regions, ensuring both visual imperceptibility and semantic coherence at feature and concept levels. Extensive experiments demonstrate that GMAE achieves strong transferability to commercial LVLMs, and reasoning-oriented models. Notably, GMAE achieves an attack success rate of 82.90% on GPT-4o, outperforming the strongest baseline by 7.8%, establishing a new state of the art for targeted transfer attacks on large-scale black-box LVLMs.
- 15 June 2026
- Published by Association for Computing Machinery (ACM)
- p. 1974-1983
Open AccessGMAE: Gated Multi-signal Adaptive Ensemble for Targeted Transfer Attacks on Commercial Large Vision-Language Models
Despite strong performance on open-source large vision-language models (LVLMs), transfer-based targeted adversarial attacks often fail against closed-source commercial LVLMs. Existing methods typically rely on static perturbation allocation and uniformly weighted surrogate ensembles, limiting their ability to capture model-agnostic semantic vulnerabilities essential for cross-model transferability. Consequently, perturbations are frequently allocated to semantically irrelevant regions, reducing effectiveness under realistic black-box settings. To address this limitation, we propose the Gated Multi-signal Adaptive Ensemble (GMAE) framework, a novel transfer-based targeted attack that formulates adversarial generation as a joint optimization over a gated perturbation field and an adaptive ensemble objective. A learnable spatial gating mask dynamically reallocates perturbation budgets toward semantically vulnerable regions, while surrogate models are adaptively reweighted according to optimization difficulty and historical consistency, encouraging alignment on shared semantic weaknesses across heterogeneous architectures. To further enhance robustness, GMAE performs global semantic alignment in a symbolic discrete space, reducing the influence of non-essential feature variations and improving cross-model generalization. In addition, we introduce a local attention-guided multi-scale consistency mechanism that transfers fine-grained target textures into attention-identified sensitive regions, ensuring both visual imperceptibility and semantic coherence at feature and concept levels. Extensive experiments demonstrate that GMAE achieves strong transferability to commercial LVLMs, and reasoning-oriented models. Notably, GMAE achieves an attack success rate of 82.90% on GPT-4o, outperforming the strongest baseline by 7.8%, establishing a new state of the art for targeted transfer attacks on large-scale black-box LVLMs.
- 16 January 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Circuits and Systems for Video Technology
- Vol. 35 (7) , 6413-6423
DeepMSD: Advancing Multimodal Sarcasm Detection Through Knowledge-Augmented Graph Reasoning
Multimodal sarcasm detection (MSD) requires predicting the sarcastic sentiment by understanding diverse modalities of data (e.g., text, image). Beyond the surface-level information conveyed in the post data, understanding the underlying deep-level knowledge–such as the background and intent behind the data–is crucial for understanding the sarcastic sentiment. However, previous works have often overlooked this aspect, limiting their potential to achieve superior performance. To tackle this challenge, we propose DeepMSD, a novel framework that generates supplemental deep-level knowledge to enhance the understanding of sarcastic content. Specifically, we first devise a Deep-level Knowledge Extraction Module that leverages large vision-language models to generate deep-level information behind the text-image pairs. Additionally, we devise a Cross-knowledge Graph Reasoning Module to model how humans use prior knowledge to identify sarcastic cues in multimodal posts. This module constructs cross-knowledge graphs that connect deep-level knowledge with surface-level knowledge. As such, it enables a more profound exploration of the cues underlying sarcasm. Experiments on the public MSD dataset demonstrate that our approach significantly surpasses previous state-of-the-art methods.
- 16 January 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Circuits and Systems for Video Technology
- Vol. 35 (7) , 6413-6423
DeepMSD: Advancing Multimodal Sarcasm Detection Through Knowledge-Augmented Graph Reasoning
Multimodal sarcasm detection (MSD) requires predicting the sarcastic sentiment by understanding diverse modalities of data (e.g., text, image). Beyond the surface-level information conveyed in the post data, understanding the underlying deep-level knowledge–such as the background and intent behind the data–is crucial for understanding the sarcastic sentiment. However, previous works have often overlooked this aspect, limiting their potential to achieve superior performance. To tackle this challenge, we propose DeepMSD, a novel framework that generates supplemental deep-level knowledge to enhance the understanding of sarcastic content. Specifically, we first devise a Deep-level Knowledge Extraction Module that leverages large vision-language models to generate deep-level information behind the text-image pairs. Additionally, we devise a Cross-knowledge Graph Reasoning Module to model how humans use prior knowledge to identify sarcastic cues in multimodal posts. This module constructs cross-knowledge graphs that connect deep-level knowledge with surface-level knowledge. As such, it enables a more profound exploration of the cues underlying sarcasm. Experiments on the public MSD dataset demonstrate that our approach significantly surpasses previous state-of-the-art methods.
- Open Access
- 19 August 2024
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Information Systems
- Vol. 42 (6) , 1-26
Breaking Through the Noisy Correspondence: A Robust Model for Image-Text Matching
Unleashing the power of image-text matching in real-world applications is hampered by noisy correspondence. Manually curating high-quality datasets is expensive and time-consuming, and datasets generated using diffusion models are not adequately well-aligned. The most promising way is to collect image-text pairs from the Internet, but it will inevitably introduce noisy correspondence. To reduce the negative impact of noisy correspondence, we propose a novel model that first transforms the noisy correspondence filtering problem into a similarity distribution modeling problem by exploiting the powerful capabilities of pre-trained models. Specifically, we use the Gaussian Mixture model to model the similarity obtained by CLIP as clean distribution and noisy distribution, to filter out most of the noisy correspondence in the dataset. Afterward, we used relatively clean data to fine-tune the model. To further reduce the negative impact of unfiltered noisy correspondence, i.e., a minimal part where two distributions intersect during the fine-tuning process, we propose a distribution-sensitive dynamic margin ranking loss, further increasing the distance between the two distributions. Through continuous iteration, the noisy correspondence gradually decreases and the model performance gradually improves. Our extensive experiments demonstrate the effectiveness and robustness of our model even under high noise rates.
- 19 August 2024
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Information Systems
- Vol. 42 (6) , 1-26
Open AccessBreaking Through the Noisy Correspondence: A Robust Model for Image-Text Matching
Unleashing the power of image-text matching in real-world applications is hampered by noisy correspondence. Manually curating high-quality datasets is expensive and time-consuming, and datasets generated using diffusion models are not adequately well-aligned. The most promising way is to collect image-text pairs from the Internet, but it will inevitably introduce noisy correspondence. To reduce the negative impact of noisy correspondence, we propose a novel model that first transforms the noisy correspondence filtering problem into a similarity distribution modeling problem by exploiting the powerful capabilities of pre-trained models. Specifically, we use the Gaussian Mixture model to model the similarity obtained by CLIP as clean distribution and noisy distribution, to filter out most of the noisy correspondence in the dataset. Afterward, we used relatively clean data to fine-tune the model. To further reduce the negative impact of unfiltered noisy correspondence, i.e., a minimal part where two distributions intersect during the fine-tuning process, we propose a distribution-sensitive dynamic margin ranking loss, further increasing the distance between the two distributions. Through continuous iteration, the noisy correspondence gradually decreases and the model performance gradually improves. Our extensive experiments demonstrate the effectiveness and robustness of our model even under high noise rates.