Abstract
Existing weakly supervised referring expression comprehension (WREC) and segmentation (WRES) methods perform anchor-text matching using flat sentence-level representations, thereby discarding the multi-granularity hierarchical semantics inherent in both visual and linguistic modalities. Because referring expressions naturally form hierarchical clauses of increasing specificity and flat Euclidean space distorts such tree-like structures, explicit hierarchical modeling is necessary yet overlooked. We propose HCNet (Hyperbolic hierarchical Clause Network), which first decomposes each expression into four hierarchical clauses via dependency parsing, then refines sentence representations through a hierarchical text enhancer that attends to coarser clause levels. Subsequently, both visual and textual features are projected into a dual Euclidean-Lorentzian hyperbolic space, where an entailment cone loss enforces hierarchical nesting on the text side and a structure alignment loss transfers this hierarchy to the visual modality. Experiments on RefCOCO, RefCOCO+, and RefCOCOg demonstrate that HCNet achieves state-of-the-art on both WREC and WRES, with improvements of up to +3.54% over the previous best. The code is available at https://github.com/VILAN-Lab/HCNet.
Funding Information
  • National Natural Science Foundation of China (62276072)

This publication has 13 references indexed in Scilit: