Q-GrAM: Fine-Grained Image–Text Retrieval via Grouped Query Routing and Conditional Query Modulation
Open Access
Abstract
Existing image–text retrieval methods often compute cross-modal similarity using global single-vector representations. Although efficient for coarse semantic alignment, such compressed representations are limited when textual queries involve fine-grained semantics, including objects, attributes, relations, and their compositional structures. This paper focuses on fine-grained text-to-image retrieval and proposes Q-GrAM, a retrieval-oriented adaptation of the BLIP-2 Q-Former. Instead of treating Q-Former queries as a homogeneous set, Q-GrAM partitions a fixed query budget into semantically differentiated groups. A text-guided router assigns token-level semantic demands to query groups, while query conditional initialization modulates each group according to group-level textual summaries. The resulting grouped visual query features are matched with text tokens through a group-aware late interaction scorer, and auxiliary routing balance and inter-group diversity regularization are introduced to stabilize semantic specialization. Experiments on MS-COCO 5K, Flickr30K, and Flickr30K-CFQ show that Q-GrAM achieves strong text-to-image retrieval performance against both global embedding baselines and representative fine-grained image–text matching methods, while maintaining competitive bidirectional retrieval performance. These results demonstrate the effectiveness of structured, text-conditioned Q-Former query specialization for fine-grained text-driven image search.Keywords
Funding Information
- the National Natural Science Foundation of China under Grant (62502349)
This publication has 22 references indexed in Scilit:
- Flickr30K-CFQ: A Compact and Fragmented Query Dataset for Text-image Retrieval2025
- Composing Object Relations and Attributes for Image-Text Matching2024
- Learning Semantic Relationship among Instances for Image-Text Matching2023
- Fine-grained Image-text Matching by Cross-modal Hard Aligning Network2023
- Negative-Aware Attention Framework for Image-Text Matching2022
- Oscar: Object-Semantics Aligned Pre-training for Vision-Language Tasks2020
- UNITER: UNiversal Image-TExt Representation Learning2020
- IMRAM: Iterative Matching With Recurrent Attention Memory for Cross-Modal Image-Text Retrieval2020
- Graph Structured Network for Image-Text Matching2020
- Stacked Cross Attention for Image-Text Matching2018