Visual and Semantic Prompt Collaboration for Generalized Zero-Shot Learning
Authors:
- 10 June 2025
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 20275-20285
Abstract
Generalized zero-shot learning aims to recognize both seen and unseen classes with the help of semantic information that is shared among different classes. It inevitably requires consistent visual-semantic alignment. Existing approaches fine-tune the visual backbone by seen-class data to obtain semantic-related visual features, which may cause overfitting on seen classes with a limited number of training images. This paper proposes a novel visual and semantic prompt collaboration framework, which utilizes prompt tuning techniques for efficient feature adaptation. Specifically, we design a visual prompt to integrate the visual information for discriminative feature learning and a semantic prompt to integrate the semantic formation for visual-semantic alignment. To achieve effective prompt information integration, we further design a weak prompt fusion mechanism for the shallow layers and a strong prompt fusion mechanism for the deep layers in the network. Through the collaboration of visual and semantic prompts, we can obtain discriminative semantic-related features for generalized zero-shot image recognition. Extensive experiments demonstrate that our framework consistently achieves favorable performance in both conventional zero-shot learning and generalized zero-shot learning benchmarks compared to other state-of-the-art methods.Keywords
All Related Versions
Funding Information
- National Natural Science Foundation of China
This publication has 55 references indexed in Scilit:
- Adaptive Relation-Aware Network for zero-shot classificationNeural Networks, 2024
- Dynamic visual-guided selection for zero-shot learningThe Journal of Supercomputing, 2023
- DualPrompt: Complementary Prompting for Rehearsal-Free Continual Learning2022
- Implicit and Explicit Attention for Zero-Shot Learning2021
- Latent Embedding Feedback and Discriminative Features for Zero-Shot Classification2020
- Leveraging Seen and Unseen Semantic Relationships for Generative Zero-Shot Learning2020
- Learning Class Prototypes via Structure Alignment for Zero-Shot Recognition2018
- Label-Embedding for Attribute-Based Classification2013
- SUN attribute database: Discovering, annotating, and recognizing scene attributes2012
- Learning to detect unseen object classes by between-class attribute transfer2009