Chunyu Lin

We present RopStitch, an unsupervised deep image stitching framework with both robustness and naturalness. To ensure the robustness of RopStitch, we propose to incorporate the universal prior of content perception into the image stitching model by a dual-branch architecture. It separately captures coarse and fine features and integrates them to achieve highly generalizable performance across diverse unseen real-world scenes. Concretely, the dual-branch model consists of a pretrained branch to capture semantically invariant representations and a learnable branch to extract fine-grained discriminative features, which are then merged into a whole by a controllable factor at the correlation level. Besides, considering that content alignment and structural preservation are often contradictory to each other, we propose a concept of virtual optimal planes to relieve this conflict. To this end, we model this problem as a process of estimating homography decomposition coefficients, and design an iterative coefficient predictor and minimal semantic distortion constraint to identify the optimal plane. This scheme is finally incorporated into RopStitch by warping both views onto the optimal plane bidirectionally. Extensive experiments across various datasets demonstrate that RopStitch significantly outperforms existing methods, particularly in scene robustness and content naturalness.

The performance of existing supervised layout estimation methods heavily relies on the quality of data annotations. However, obtaining large-scale and high-quality datasets remains a laborious and time-consuming challenge. To solve this problem, semi-supervised approaches are introduced to relieve the demand for expensive data annotations by encouraging the consistent results of unlabeled data with different perturbations. However, existing solutions merely employ vanilla perturbations, ignoring the characteristics of panoramic layout estimation. In contrast, we propose a novel semi-supervised method named SemiLayout360, which incorporates the priors of the panoramic layout and distortion through collaborative perturbations. Specifically, we leverage the panoramic layout prior to enhance the model's focus on potential layout boundaries. Meanwhile, we introduce the panoramic distortion prior to strengthen distortion awareness. Furthermore, to prevent intense perturbations from hindering model convergence and ensure the effectiveness of prior-based perturbations, we divide and reorganize them as panoramic collaborative perturbations. Our experimental results on three mainstream benchmarks demonstrate that the proposed method offers significant advantages over existing state-of-the-art (SoTA) solutions.

Multi-object tracking (MOT) in fisheye images becomes particularly challenging due to significant radial distortion. In MOT, Camera Motion Compensation (CMC) is crucial for mitigating inter-frame errors caused by camera movement. However, conventional CMC methods degrade in fisheye images, as their rigid motion models cannot characterize the non-uniform motion fields introduced by severe distortion. In this paper, we first establish this limitation through theoretical derivation and then propose a plug-and-play ROI-Centric Motion Compensation (RCMC) mechanism, which serves as a new CMC solution that leverages instance-aware ROI for motion estimation. Moreover, fisheye distortion introduces significant target tilt, distorting object geometry and thereby hindering accurate detection and separation of adjacent instances. To address this issue, we integrate RCMC with tailored adjustments for oriented bounding boxes (OBB) to form FO-MOT, a MOT framework for fisheye images. In addition, to overcome the lack of dedicated evaluation benchmarks, we present FisheyeMOT, a comprehensively annotated dataset with OBB-based labels, designed to support the development and validation of fisheye MOT systems. Finally, we demonstrate the superiority of our framework by qualitative and quantitative experiments. The dataset and code will be released at https://github.com/lukanightfever/FO-MOT.

Monocular 3D object detection is challenging due to the lack of accurate depth. However, existing depth-assisted solutions still exhibit inferior performance, whose reason is universally acknowledged as the unsatisfactory accuracy of monocular depth estimation models. In this paper, we revisit monocular 3D object detection from the depth perspective and formulate an additional issue as the limited 3D structure-aware capability of existing depth representations (e.g., depth one-hot encoding or depth distribution). To address this issue, we introduce a novel Depth Thickness Field approach to embed clear 3D structures of the scenes. Specifically, we present MonoDTF, a scene-to-instance depth-adapted network comprising a Scene-Level Depth Retargeting (SDR) module and an Instance-Level Spatial Refinement (ISR) module. The former retargets traditional depth representations to the proposed depth thickness field, incorporating the scene-level perception of 3D structures. The latter refines the voxel space with the guidance of instances, enhancing the 3D instance-aware capability of the depth thickness field and thus improving detection accuracy. Extensive experiments on the KITTI and Waymo datasets demonstrate our superiority to existing state-of-the-art (SoTA) methods and the universality when equipped with different depth estimation models. The source codes are available at https://github.com/QiuDeZhang/MonoDTF.

Depth estimation from a monocular 360 image is important to the perception of the entire 3D environment. However, the inherent distortion and large field of view (FoV) in 360 images pose great challenges for this task. To this end, existing mainstream solutions typically introduce additional perspective-based 360 representations (e.g., Cubemap) to achieve effective feature extraction. Nevertheless, regardless of the introduced representations, they eventually need to be unified into the equirectangular projection (ERP) format for the subsequent depth estimation, which inevitably reintroduces additional distortions. In this work, we propose an oriented-distortion-aware Gabor Fusion framework (PGFuse) to address the above challenges. First, we introduce Gabor filters that analyze texture in the frequency domain, extending the receptive fields and enhancing depth cues. To address the reintroduced distortions, we design a latitude-aware distortion representation to generate customized, distortion-aware Gabor filters (PanoGabor filters). Furthermore, we design a channel-wise and spatial-wise unidirectional fusion module (CS-UFM) that integrates the proposed PanoGabor filters to unify other representations into the ERP format, delivering effective and distortion-aware features. Considering the orientation sensitivity of the Gabor transform, we further introduce a spherical gradient constraint to stabilize this sensitivity. Experimental results on three popular indoor 360 benchmarks demonstrate the superiority of the proposed PGFuse to existing state-of-the-art solutions. Code and models will be available at https://github.com/zhijieshen-bjtu/PGFuse.

Creating robust, high-fidelity head avatars with 3D Gaussian Splatting (3DGS) remains constrained by two bottlenecks: one is the representation bottleneck that limits the modeling of dynamic details, and the other is the dynamic robustness bottleneck that causes information loss and point cloud inflation. To this end, we present LivingAvatars, which addresses both issues through two key designs: (i) a functionally decoupled Gaussian representation that separates identity-stable base Gaussians from detail Gaussians carrying high-frequency appearance, while a lightweight neural decoder injects features into the latter to synthesize expression-dependent details; and (ii) a Gaussian lifecycle management system that endows Gaussians with multiple states and replaces destructive pruning with probabilistic selective pruning and temporary hiding to suppress redundant growth, ensuring a compact and robust representation. On novel-view synthesis and self-reenactment, LivingAvatars achieves overall better metrics than existing methods. In cross-identity reenactment, it better preserves transient, expression-dependent signals, yielding higher perceptual fidelity. These results suggest that functionally decoupled Gaussian representations and lifecycle management are effective principles for building neural head avatars that remain robust and compact under dynamic occlusions, reliably reconstructing high-frequency details and improving overall realism.