Rethinking Cross-Modal Interaction for Efficient Referring Image Segmentation
- 13 June 2025
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 10 (8) , 7811-7818
Abstract
Referring Image Segmentation, the task of finding and segmenting objects in an image conditioned on a natural language description, is crucial for human-robot collaboration. However, current RIS methods often implement visual-text alignment relying on computationally intensive Transformer-based self-attention mechanisms, which impairs deployment on robots, especially those with limited computational resources. Indeed, beyond accuracy, practical robotic applications demand efficient models with small footprints. This paper introduces ERIS, an Efficient RIS approach designed for real-world deployment. ERIS achieves effective multi-modal interaction through a novel dual-branch architecture: a Visual Text Alignment branch and a Text Visual Refinement branch. This design implements bilateral alignment between textual and visual modalities without the computational burden of self-attention. Of note, the progressive alignment in ERIS enhances interpretability, revealing how textual cues guide segmentation. For the sake of efficiency, our alignment strategy produces structured embeddings which can be directly mapped into the final segmentation mask, without the need for additional segmentation heads. Thus, ERIS footprint scales linearly with respect to the number of visual and text tokens, making it suitable for both cloud-based and edge deployment. Experimental results demonstrate that ERIS achieves competitive or superior performance compared to state-of-the-art methods while significantly reducing computational cost, proving that efficiency and accuracy are not mutually exclusive.Keywords
Funding Information
- Sustainable Mobility Center
- nazionali di R&S” su alcune Key Enabling Technologies (CN_00000023)
- European Union Next-GenerationEU
This publication has 39 references indexed in Scilit:
- An Efficient and Effective Transformer Decoder-Based Framework for Multi-task Visual Grounding2024
- SeqTR: A Simple Yet Universal Network for Visual Grounding2022
- Rethinking BiSeNet For Real-time Semantic Segmentation2021
- Multi-Task Collaborative Network for Joint Referring Expression Comprehension and Segmentation2020
- Bi-Directional Relationship Inferring Network for Referring Image Segmentation2020
- Squeeze-and-Excitation Networks2018
- MAttNet: Modular Attention Network for Referring Expression Comprehension2018
- Modeling Context in Referring Expressions2016
- Segmentation from Natural Language Expressions2016
- Modeling Context Between Objects for Referring Expression Understanding2016