Lu Zhang

Multi-task dense prediction improves pixel-level performance by leveraging shared representations and inter-task collaboration. However, existing approaches either rely on implicit task relationships or neglect frequency-domain cues that are essential for preserving fine-grained details and enhancing cross-task feature learning at multiple scales. As a result, they face persistent challenges in multi-scale feature fusion, effective task interaction, and accurate decoding. To address these issues, we propose a hierarchical frequency-driven framework, termed Hierarchical Frequency-Adaptive Network (HiFAN), that facilitates cross-task collaborative optimization via frequency-domain analysis. Specifically, we first design a task-adaptive fusion module that exploits multi-scale frequency-domain information to enhance spatial details. This module generates dynamic convolutional kernels with task-specific parameters and positional biases to adaptively accommodate diverse task requirements. Next, we introduce an efficient cross-task interaction module that leverages compact low-frequency representations to enable global context exchange across tasks. Finally, we present a high-frequency-aware decoder that mitigates feature smoothing and detail loss commonly introduced by Transformer-based decoders. We demonstrate the effectiveness of HiFAN on two standard multi-task learning benchmarks, PASCAL-Context and NYUD-v2, achieving strong and competitive performance across multiple tasks. The code and model weights are available in HiFAN.

The core objective of Audio-Visual Segmentation (AVS) is aligning auditory cues with their corresponding visual sources in video sequences, which poses challenges due to long-range temporal dependencies and modality-specific noise (e.g., irrelevant sounds or visual clutter). Most existing methods focus on Transformer-based architectures, which, despite achieving strong performance, struggle to maintain sequence structure perception in noisy, complex scenes and fail to fully exploit intra-modal and inter-modal temporal relationships, limiting multimodal understanding and alignment. To address the challenges, we introduce TemLo, a novel framework built on the Receptance-Weighted Key-Value (RWKV) architecture for efficient temporal modeling and precise targeted representation learning. TemLo introduces modality-specific RWKV encoders that capture salient spatial and temporal dynamics within each stream, reducing intra-modal noise and ambiguity. To fully enable cross-modal interaction and integration, we further design the Acoustic Modulation Pyramid to facilitate structured audio-visual interactions at multiple temporal and spatial scales, enriching the visual representations with audio-aware semantics. Moreover, we propose the Vision-guided Modality Injector for ensuring temporally consistent and semantically aligned reasoning. Extensive experiments on standard AVS benchmarks demonstrate that TemLo sets a new state of the art, achieving significant gains over prior methodS. When employing the PVT-v2 backbone, TemLo surpasses AVSCA on the MS3 dataset with absolute improvements of 6.6 (10.28%$\uparrow$) in $\mathcal {M}_{J}$ and 2.8 (3.66%$\uparrow$) in $\mathcal {M}_{F}$. Furthermore, on the AVSS dataset, our method outperforms Co-Prop by margins of 3.0 (7.58%$\uparrow$) and 4.4 (9.91%$\uparrow$) in $\mathcal {M}_{J}$ and $\mathcal {M}_{F}$, respectively. Source code and model weights will be made publicly available at TemLo.

Reasoning video object segmentation (ReaVOS) aims to segment referred objects in video sequences based on implicit and complex linguistic queries. Existing methods typically compress limited video frames into pooled representations and prompt multimodal large language models (MLLMs) to generate a single global segmentation token. However, this strategy lacks explicit contextual guidance and causes substantial loss of spatial details, limiting capability and segmentation consistency. To overcome these limitations, we introduce Context-infused Consistent Video Segmentor (CiCVS), a novel framework leveraging contextual information to guide generation of temporally coherent and accurate mask trajectories. CiCVS incorporates a Hierarchical Frame Sampling (HFS) module, which globally samples support frames across the entire video to ensure broad temporal coverage, and then uniformly selects target frames within the support set. It also employs a Contextual Token Prompting (CTP) module, which utilizes contextual cues from support frames to guide the MLLM in generating specialized tokens for various target frames, enabling the model to capture intricate temporal patterns and ensure consistency across long-range sequences. At the core of CTP is the Multimodal Injection Compressor (MIC) block, which efficiently integrates support frame features and textual semantic information into a compact set of latent queries, enhancing temporal-level object perception. To further advance the ReaVOS field, we introduce the CoCoRVOS benchmark, which features more temporally intricate reasoning instructions and a diverse set of video scenarios. Extensive experiments demonstrate that CiCVS establishes a new state-of-the-art on multiple benchmarks, achieving significant improvements in J&F scores, including +2.7 on CoCoRVOS, +1.4 on ReVOS, and +7.0 on ReasonVOS, underscoring its superior contextual reasoning and segmentation capabilities.

Current 3D scene understanding pipelines typically concatenate vast numbers of 3D object tokens with text tokens and feed the resulting sequence to a Large Language Model (LLM). However, existing 3D-LLMs for scene understanding encounter two major limitations: (i) excessive inclusion of task-irrelevant object data, introducing noise that reduces reasoning accuracy and increases hallucinations; and (ii) reliance solely on point cloud data, which inherently lacks the rich semantic information available in complementary 2D modalities, such as color, material properties, texture, and high-level contextual relationships. To address these challenges, we introduce 3D-SceneQ, a 3D LLM that unifies adaptive token pruning with multimodal semantic enrichment, markedly advancing scene understanding, reasoning, and grounding. Specifically, we propose a Query-Guided Adaptive Pruning (QGAP) module that adaptively selects task-relevant objects based on the intended meaning of user instructions, leveraging a learnable latent query to retain critical information while effectively suppressing irrelevant noise. In addition, we introduce a Multi-modal Object-level Feature Enhancement (MOFE) module, which integrates 2D feature embeddings from pre-trained models into 3D representation to enrich object-level semantic information. Evaluations show that 3D-SceneQ outperforms current state-of-the-art methods across multiple benchmarks (e.g., +5.7 CIDEr on Scan2Cap and +17.0 EM@1 on SQA3D), demonstrating its remarkable capabilities in various 3D language reasoning tasks.

This technical report represents the award-winning solution to the Cross-platform 3D Object Detection task in the RoboSense2025 Challenge. Our approach is built upon PVRCNN++, an efficient 3D object detection framework that effectively integrates point-based and voxel-based features. On top of this foundation, we improve cross-platform generalization by narrowing domain gaps through tailored data augmentation and a self-training strategy with pseudo-labels. These enhancements enabled our approach to secure the 3rd place in the challenge, achieving a 3D AP of 62.67% for the Car category on the phase-1 target domain, and 58.76% and 49.81% for Car and Pedestrian categories respectively on the phase-2 target domain.

Understanding the inter-relations and interactions between tasks is crucial for multitask dense prediction. Existing methods predominantly utilize convolutional layers and attention mechanisms to explore task-level interactions. In this work, we introduce a novel decoder-based framework, parameter-aware Mamba model (PAMM), specifically designed for dense prediction in multitask learning (MTL) setting. Distinct from approaches that employ Transformers to model holistic task relationships, PAMM leverages the rich, scalable parameters of state-space models (SSMs) to enhance task interconnectivity. It features dual state-space parameter experts (PEs) that integrate and set task-specific parameter priors (PPs), capturing the intrinsic properties of each task. This approach not only facilitates precise multitask interactions but also allows for the global integration of task priors through the structured state-space sequence (S4) model. Furthermore, we employ the multidirectional Hilbert scanning (MDHS) method to construct multiangle feature sequences, thereby enhancing the sequence model’s perceptual capabilities for 2-D data. Extensive experiments on the NYUD-v2 and PASCAL-Context benchmarks demonstrate the effectiveness of our proposed method. Our code is available at https://github.com/CQC-gogopro/PAMM

When sound waves hit an object, they induce vibrations that produce high-frequency and subtle visual changes, which can be used for recovering the sound. Early studies always encounter trade-offs related to sampling rate, bandwidth, and the simplicity of the optical path. Recent advances in event camera hardware show good potential for its application in visual sound recovery, because of its superior ability in capturing high-frequency signals. Thus we seek to explore the potential of event cameras for non-contact sound recovery. In this work, we present the first learning-based framework for event-based sound recovery, establishing a strong baseline through systematic dataset simulation, hardware design, and model development. Addressing the scarcity of data in this domain, we introduce EvMic, which contains both synthetic and real-world data, enabling data-driven approaches and benchmarking. To effectively capture subtle vibrations in real-world scenarios, we design an imaging system utilizing a laser matrix that enhances surface gradients. Furthermore, we propose a robust end-to-end network that fully exploits the spatial-temporal nature of event streams for high-quality sound recovery. Experimental results on both synthetic and real-world data demonstrate the effectiveness of our proposed framework. The project page is available at https://yyzq1.github.io/EvMic/.