HCNet: Hyperbolic Hierarchical Clause Network for Weakly Supervised Multi-task Visual Grounding
Open Access
- 9 November 2026
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 9628-9637
Abstract
Existing weakly supervised referring expression comprehension (WREC) and segmentation (WRES) methods perform anchor-text matching using flat sentence-level representations, thereby discarding the multi-granularity hierarchical semantics inherent in both visual and linguistic modalities. Because referring expressions naturally form hierarchical clauses of increasing specificity and flat Euclidean space distorts such tree-like structures, explicit hierarchical modeling is necessary yet overlooked. We propose HCNet (Hyperbolic hierarchical Clause Network), which first decomposes each expression into four hierarchical clauses via dependency parsing, then refines sentence representations through a hierarchical text enhancer that attends to coarser clause levels. Subsequently, both visual and textual features are projected into a dual Euclidean-Lorentzian hyperbolic space, where an entailment cone loss enforces hierarchical nesting on the text side and a structure alignment loss transfers this hierarchy to the visual modality. Experiments on RefCOCO, RefCOCO+, and RefCOCOg demonstrate that HCNet achieves state-of-the-art on both WREC and WRES, with improvements of up to +3.54% over the previous best. The code is available at https://github.com/VILAN-Lab/HCNet.Keywords
Funding Information
- National Natural Science Foundation of China (62276072)
This publication has 13 references indexed in Scilit:
- Zero-shot referring expression comprehension via guidance of Multimodal Large Language ModelsPattern Recognition, 2026
- Implement Referring Expression Comprehension by Extending Auto-focus Lens to Locked Vision ModelACM Transactions on Multimedia Computing, Communications, and Applications, 2026
- AlignCAT: Visual-Linguistic Alignment of Category and Attribute for Weakly Supervised Visual Grounding2025
- Look Around Before Locating: Considering Content and Structure Information for Visual GroundingProceedings of the AAAI Conference on Artificial Intelligence, 2025
- QueryMatch: A Query-based Contrastive Learning Framework for Weakly Supervised Visual Grounding2024
- Rethinking Two-Stage Referring Expression Comprehension: A Novel Grounding and Segmentation Method Modulated by PointProceedings of the AAAI Conference on Artificial Intelligence, 2024
- Segment Anything2023
- Referring Image Segmentation Using Text Supervision2023
- Vision-and-Language Navigation: Interpreting Visually-Grounded Navigation Instructions in Real Environments2018
- Realtime Multi-person 2D Pose Estimation Using Part Affinity Fields2017