Abstract
Visual grounding (VG) in remote sensing (RS) images aims to localize objects referred to by a query text in complex aerial scenes. However, existing methods still suffer from limitations in multimodal interaction and alignment. Specifically, multiscale visual feature fusion often lacks effective textual semantic filtering, introducing redundant background information that degrades cross-modal alignment. Moreover, although bidirectional cross-attention enhances intermodal information exchange, it tends to neglect intramodal semantic consistency, resulting in unstable and less discriminative representations. In addition, language queries obtained after cross-modal interaction are vulnerable to linguistic noise during decoding, which further impairs localization robustness. To address these challenges in a unified and progressive manner, we propose a hierarchical vision–language alignment framework, termed hierarchical feature-modulated transformer with introspective interaction (HMIT). Specifically, text-modulated cross-scale feature fusion (TMCFF) performs early-stage text-guided modulation on multiscale visual features to suppress text-irrelevant background information. Cross-modal interaction with contextual introspection (CMICI) refines cross-modal representations through contextual introspection while preserving intramodal semantic consistency. vision-guided text calibration (VGTC) calibrates high-level language queries using global visual priors prior to decoding, producing more robust and discriminative query embeddings. Extensive experiments on three benchmark datasets demonstrate that HMIT consistently outperforms existing methods across multiple metrics, achieving new state-of-the-art performance.
Funding Information
  • National Natural Science Foundation of China (61871106)
  • National Natural Science Foundation of China (62441231)
  • Key Research and Development Program of Liaoning Province (2024JH2/102500015)
  • Fundamental Research Funds for the Central Universities of the Ministry of Education (N25BSS034)