ZLZhihe Lu

Zhihe Lu

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article

Previous Organizations / Institutes

The organizations / institutes from the scholar's other articles in the past ten years


Zhihe Lu has a total of 54 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Open Access

    Dream-Tac: A Unified Tactile World Action Model for Contact-Rich Robot Manipulation

    by Yunfan Lou, Yifan Ye, Yankai Fu, Jun Cen, Xiaowei Chi, Yaoxu Lyu, Peidong Jia, Sirui Han, Zhihe Lu, Shanghang Zhang


  • Open Access

    DepthPlay-Agent: Agentic Learning and Fusion for Depth Estimation

    by Fahad Majeed, Maria Nazir, Zhihe Lu, Marco Agus, Jens Schneider

    Monocular depth estimation in complex, dynamic environments remains challenging due to rapid object motion, texture repetitions, occlusions, and strong geometric constraints inherent in structured scenes. To address these challenges, we propose DepthPlay-Agent (DEAN), an agentic multi-model fusion framework that builds upon SOTA depth estimator backbones such as UNet++, HybridDepth, DINOv3, and ZoeDepth under a learnable Mixture-of-Experts controller. This controller embeds scene context through transformers and uses task priors to dynamically assign pe-pixel fusion weights. A domain-aware refinement module further enforces geometric consistency using planar and semantic segmentation cues. Beyond static fusion, DEAN introduces an agentic inference layer that dynamically regulates expert contributions and refinement strategies, enabling adaptive and interpretable decision-making. Experiments across four benchmarks (SoccerNet-Depth, KITTI, MPI-Sintel, NYU Depth V2) demonstrate that DEAN consistently achieves SOTA performance, improving AbsRel by up to 5% and reducing SILog by 8% over strong baselines. By coupling multi-modal intelligence with structured geometric reasoning, DEAN establishes a new paradigm for adaptive, context-aware depth estimation in dynamic real-world domains.

  • Token Expand-Merge: Training-Free Token Compression for Vision-Language-Action Models

    by Yifan Ye, Jiaqi Ma, Jun Cen, Zhihe Lu

    Vision-Language-Action (VLA) models pretrained on large-scale multimodal datasets have emerged as powerful foundations for robotic perception and control. However, their massive scale, often billions of parameters, poses significant challenges for real-time deployment, as inference becomes computationally expensive and latency-sensitive in dynamic environments. To address this, we propose Token Expand-and-Merge-VLA (TEAM-VLA), a training-free token compression framework that accelerates VLA inference while preserving task performance. TEAM-VLA introduces a dynamic token expansion mechanism that identifies and samples additional informative tokens in the spatial vicinity of attention-highlighted regions, enhancing contextual completeness. These expanded tokens are then selectively merged in deeper layers under action-aware guidance, effectively reducing redundancy while maintaining semantic coherence. By coupling expansion and merging within a single feed-forward pass, TEAM-VLA achieves a balanced trade-off between efficiency and effectiveness, without any retraining or parameter updates. Extensive experiments on LIBERO benchmark demonstrate that TEAM-VLA consistently improves inference speed while maintaining or even surpassing the task success rate of full VLA models.

  • Self-Evolved Imitation Learning in Simulated World

    by Yifan Ye, Jun Cen, Jing Chen, Zhihe Lu

    Imitation learning has been a trend recently, yet training a generalist agent across multiple tasks still requires large-scale expert demonstrations, which are costly and labor-intensive to collect. To address the challenge of limited supervision, we propose Self-Evolved Imitation Learning (SEIL), a framework that progressively improves a few-shot model through simulator interactions. The model first attempts tasks in the simulator, from which successful trajectories are collected as new demonstrations for iterative refinement. To enhance the diversity of these demonstrations, SEIL employs dual-level augmentation: (i) Model-level, using an Exponential Moving Average (EMA) model to collaborate with the primary model, and (ii) Environment-level, introducing slight variations in initial object positions. We further introduce a lightweight selector that filters complementary and informative trajectories from the generated pool to ensure demonstration quality. These curated samples enable the model to achieve competitive performance with far fewer training examples. Extensive experiments on the LIBERO benchmark show that SEIL achieves a new state-of-the-art performance in few-shot imitation learning code is available at https://github.com/Jasper-aaa/SEIL.git.

  • Open Access

    Multi-scale Dynamic Network for Document Shadow Removal

    by Jiarui Li, Jiaqi Ma, Zeyu Xiao, Ziyi Zhuang, Zhihe Lu

  • COLA: Context-Aware Language-Driven Test-Time Adaptation

    by Aiming Zhang, Tianyuan Yu, Liang Bai, Jun Tang, Yanming Guo, Yirun Ruan, Yun Zhou, Zhihe Lu

    Test-time adaptation (TTA) has gained increasing popularity due to its efficacy in addressing “distribution shift” issue while simultaneously protecting data privacy. However, most prior methods assume that a paired source domain model and target domain sharing the same label space coexist, heavily limiting their applicability. In this paper, we investigate a more general source model capable of adaptation to multiple target domains without needing shared labels. This is achieved by using a pre-trained vision-language model (VLM), e.g., CLIP, that can recognize images through matching with class descriptions. While the zero-shot performance of VLMs is impressive, they struggle to effectively capture the distinctive attributes of a target domain. To that end, we propose a novel method – Context-aware Language-driven TTA (COLA). The proposed method incorporates a lightweight context-aware module that consists of three key components: a task-aware adapter, a context-aware unit, and a residual connection unit for exploring task-specific knowledge, domain-specific knowledge from the VLM and prior knowledge of the VLM, respectively. It is worth noting that the context-aware module can be seamlessly integrated into a frozen VLM, ensuring both minimal effort and parameter efficiency. Additionally, we introduce a Class-Balanced Pseudo-labeling (CBPL) strategy to mitigate the adverse effects caused by class imbalance. We demonstrate the effectiveness of our method not only in TTA scenarios but also in class generalisation tasks. The source code is available at https://github.com/NUDT-Bai-Group/COLA-TTA.


  • Task-to-Instance Prompt Learning for Vision-Language Models at Test Time

    by Zhihe Lu, Jiawang Bai, Xin Li, Zeyu Xiao, Xinchao Wang

    Prompt learning has been recently introduced into the adaption of pre-trained vision-language models (VLMs) by tuning a set of trainable tokens to replace hand-crafted text templates. Despite the encouraging results achieved, existing methods largely rely on extra annotated data for training. In this paper, we investigate a more realistic scenario, where only the unlabeled test data is available. Existing test-time prompt learning methods often separately learn a prompt for each test sample. However, relying solely on a single sample heavily limits the performance of the learned prompts, as it neglects the task-level knowledge that can be gained from multiple samples. To that end, we propose a novel test-time prompt learning method of VLMs, called Task-to-Instance PromPt LEarning (TIPPLE), which adopts a two-stage training strategy to leverage both task- and instance-level knowledge. Specifically, we reformulate the effective online pseudo-labeling paradigm along with two tailored components: an auxiliary text classification task and a diversity regularization term, to serve the task-oriented prompt learning. After that, the learned task-level prompt is further combined with a tunable residual for each test sample to integrate with instance-level knowledge. We demonstrate the superior performance of TIPPLE on 15 downstream datasets, e.g., the average improvement of 1.87% over the state-of-the-art method, using ViT-B/16 visual backbone. Our code is open-sourced at https://github.com/zhiheLu/TIPPLE.


  • Parameter-Efficient and Memory-Efficient Tuning for Vision Transformer: A Disentangled Approach

    by Taolin Zhang, Jiawang Bai, Zhihe Lu, Dongze Lian, Genping Wang, Xinchao Wang, Shu-Tao Xia


  • Open Access

    Unraveling Motion Uncertainty for Local Motion Deblurring

    by Zeyu Xiao, Zhihe Lu, Michael Bi Mi, Zhiwei Xiong, Xinchao Wang

    In real-world photography, local motion blur often arises from the interplay between moving objects and stationary backgrounds during exposure. Existing deblurring methods face challenges in addressing local motion deblurring due to (i) the presence of arbitrary localized blurs and uncertain blur extents; (ii) the limited ability to accurately identify specific blurs resulting from ambiguous motion boundaries. These limitations often lead to suboptimal solutions when estimating blur maps and generating final deblurred images. To that end, we propose a novel method named Motion-Uncertainty-Guided Network (MUGNet), which harnesses a probabilistic representational model to explicitly address the intricacies stemming from motion uncertainties. Specifically, MUGNet consists of two key components, i.e., motion-uncertainty quantification (MUQ) module and motion-masked separable attention (M2SA) module, serving for complementary purposes. Concretely, MUQ aims to learn a conditional distribution for accurate and reliable blur map estimation, while the M2SA module is to enhance the representation of regions influenced by local motion blur and static background, which is achieved by promoting the establishment of extensive global interactions. We demonstrate the superiority of our MUGNet with extensive experiments. The code is publicly available at: https://github.com/zeyuxiao1997/MUGNet.