Token Expand-Merge: Training-Free Token Compression for Vision-Language-Action Models
Authors:
- Yifan YeYYYifan Ye
- Jiaqi MaJMJiaqi Ma
- Jun CenJCJun Cen
- Zhihe LuZLZhihe Lu
Preprint
Abstract
Vision-Language-Action (VLA) models pretrained on large-scale multimodal datasets have emerged as powerful foundations for robotic perception and control. However, their massive scale, often billions of parameters, poses significant challenges for real-time deployment, as inference becomes computationally expensive and latency-sensitive in dynamic environments. To address this, we propose Token Expand-and-Merge-VLA (TEAM-VLA), a training-free token compression framework that accelerates VLA inference while preserving task performance. TEAM-VLA introduces a dynamic token expansion mechanism that identifies and samples additional informative tokens in the spatial vicinity of attention-highlighted regions, enhancing contextual completeness. These expanded tokens are then selectively merged in deeper layers under action-aware guidance, effectively reducing redundancy while maintaining semantic coherence. By coupling expansion and merging within a single feed-forward pass, TEAM-VLA achieves a balanced trade-off between efficiency and effectiveness, without any retraining or parameter updates. Extensive experiments on LIBERO benchmark demonstrate that TEAM-VLA consistently improves inference speed while maintaining or even surpassing the task success rate of full VLA models. The code is public available on \href{https://github.com/Jasper-aaa/TEAM-VLA}{https://github.com/Jasper-aaa/TEAM-VLA}Keywords
All Related Versions
This publication has 9 references indexed in Scilit:
- TTF-VLA: Temporal Token Fusion via Pixel-Attention Integration for Vision-Language-Action ModelsProceedings of the AAAI Conference on Artificial Intelligence, 2026
- Vision-Language-Action Models for Robotics: A Review Towards Real-World ApplicationsIEEE Access, 2025
- Fine-Tuning Vision-Language-Action Models: Optimizing Speed and Success2025
- π₀: A Vision-Language-Action Flow Model for General Robot Control2025
- Grounding DINO: Marrying DINO with Grounded Pre-training for Open-Set Object Detection2024
- Vision-Language Models for Vision Tasks: A SurveyIEEE Transactions on Pattern Analysis and Machine Intelligence, 2024
- Video Token Merging for Long Video Understanding2024
- RT-1: Robotics Transformer for Real-World Control at Scale2023
- LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learning2023