Yin Zhang

Monocular vision-based target motion estimation is a fundamental challenge in numerous applications. This work introduces a novel bearing-box approach that fully leverages modern 3-D detection measurements that are widely available nowadays but have not been well explored for motion estimation so far. Unlike existing methods that rely on restrictive assumptions, such as isotropic target shape and lateral motion, our bearing-box estimator can estimate both the target's motion and its physical size without these assumptions by exploiting the information buried in a 3-D bounding box. When applied to multirotor micro aerial vehicles (MAVs), the estimator yields an interesting advantage: it further removes the need for higher order motion assumptions by exploiting the unique coupling between MAV's acceleration and thrust. This is particularly significant, as higher order motion assumptions are widely believed to be necessary in state-of-the-art bearing-based estimators. We support our claims with rigorous observability analyzes and extensive experimental validation, demonstrating the estimator's superior performance in real-world scenarios

Emerging 3D scene representations, such as Neural Radiance Fields (NeRF) and 3D Gaussian Splatting (3DGS), have demonstrated their effectiveness in Simultaneous Localization and Mapping (SLAM) for photo-realistic rendering, particularly when using high-quality video sequences as input. However, existing methods struggle with motion-blurred frames, which are common in real-world scenarios like low-light or long-exposure conditions. This often results in a significant reduction in both camera localization accuracy and map reconstruction quality. To address this challenge, we propose a dense visual SLAM pipeline (i.e., MBA-SLAM) to handle severe motion-blurred inputs. Our approach integrates an efficient motion blur-aware tracker with either neural radiance fields or Gaussian Splatting based mapper. By accurately modeling the physical image formation process of motion-blurred images, our method simultaneously learns 3D scene representation and estimates the cameras’ local trajectory during exposure time, enabling proactive compensation for motion blur caused by camera movement. In our experiments, we demonstrate that MBA-SLAM surpasses previous state-of-the-art methods in both camera localization and map reconstruction, showcasing superior performance across a range of datasets, including synthetic and real datasets featuring sharp images as well as those affected by motion blur, highlighting the versatility and robustness of our approach.

Existing micro aerial vehicle (MAV) detection methods mainly rely on the target's appearance features in RGB images, whose diversity makes it difficult to achieve generalized MAV detection. We notice that different types of MAVs share the same distinctive features in event streams due to their high-speed rotating propellers, which are hard to see in RGB images. This letter studies how to detect different types of MAVs from an event camera by fully exploiting the features of propellers in the original event stream. The proposed method consists of three modules to extract the salient and spatio-temporal features of the propellers while filtering out noise from background objects and camera motion. Since there are no existing event-based MAV datasets, we introduce a novel MAV dataset for the community. This is the first event-based MAV dataset comprising multiple scenarios and different types of MAVs. Without training, our method significantly outperforms state-of-the-art methods and can deal with challenging scenarios, achieving a precision rate of 83.0% (+30.3%) and a recall rate of 81.5% (+36.4%) on the proposed testing dataset.

While most visual SLAM systems traditionally prioritize accuracy or speed, the associated memory consumption would also become a concern for robots working in large‐scale environments, primarily due to the perpetual preservation of increasing number of redundant map points. Although these redundant map points are initially constructed to ensure robust frame tracking, they contribute little once the robot moves to other locations and are primarily kept for potential loop closure. After continuous optimization, these map points are accurate and actually not all of them are essential for loop closure. Therefore, this paper proposes MS‐SLAM, a memory‐efficient visual SLAM system with map sparsification aimed at selecting only parts of useful map points to keep in the global map. In MS‐SLAM, all local map points are temporarily kept to ensure robust frame tracking and further optimization, while redundant nonlocal map points are removed through the proposed novel sliding window map sparsification, which is efficient and running concurrently with original SLAM tracking. The loop closure still operates well with the selected useful map points. Through exhaustive experiments across various scenes in both public and self‐collected data sets, MS‐SLAM has demonstrated comparable accuracy with the state‐of‐the‐art visual SLAM while significantly reducing memory consumption by over 70% in large‐scale scenes. This facilitates the scalability of visual SLAM in large‐scale environments, making it a promising solution for real‐world applications. We will release our codes at https://github.com/fishmarch/MS-SLAM.

This paper studies the task of vision-based MAV-catching-MAV, where a catcher MAV (micro aerial vehicle) can detect, localize, and pursue a target MAV autonomously. Since it is challenging to develop detectors that can effectively detect unseen MAVs in complex environments, the main novelty of this paper is to propose a real-to-sim-to-real approach to address this challenge. In this method, images of real-world environments are first collected. Then, these images are used to construct a high-fidelity simulation environment, based on which a deep-learning detector is trained. The merit of this approach is that it allows efficient automatic collection of large-scale and high-quality labeled datasets. More importantly, since the simulation environment is constructed from real-world images, this approach can effectively bridge the sim-to-real gap, enabling efficient deployment in real environments. Another contribution of this paper lies in the successful implementation of a fully autonomous vision-based MAV-catching-MAV system including proposed estimation and pursuit control algorithms. While the previous works mainly focused on certain aspects of this system, we developed a completely autonomous system that integrates detection, estimation, and control algorithms on real-world robotic platforms.