Weiming Zhi

Constructing 3D representations of object geometry is critical for many robotics tasks, particularly manipulation problems. These representations must be built from potentially noisy partial observations. In this work, we focus on the problem of reconstructing a multi-object scene from a single RGBD image using a fixed camera. Traditional scene representation methods generally cannot infer the geometry of unobserved regions of the objects in the image. Attempts have been made to leverage deep learning to train on a dataset of known objects and representations, and then generalize to new observations. However, this can be brittle to noisy real-world observations and objects not contained in the dataset, and do not provide well-calibrated reconstruction confidences. We propose BRRP, a reconstruction method that leverages preexisting mesh datasets to build an informative prior during robust probabilistic reconstruction. We introduce the concept of a retrieval-augmented prior, where we retrieve relevant components of our prior distribution from a database of objects during inference. The resulting prior enables estimation of the geometry of occluded portions of the in-scene objects. Our method produces a distribution over object shape that can be used for reconstruction and measuring uncertainty. We evaluate our method in both simulated scenes and in the real world. We demonstrate the robustness of our method against deep learning-only approaches while being more accurate than a method without an informative prior. Through real-world experiments, we particularly highlight the capability of BRRP to enable successful dexterous manipulation in clutter.

Robotic manipulator applications often require efficient online motion planning. When completing multiple tasks, sequence order and choice of goal configuration can have a drastic impact on planning performance. This is well known as the robot task sequencing problem (RTSP). Existing general-purpose RTSP algorithms are susceptible to producing poor-quality solutions or failing entirely when available computation time is restricted. We propose a new multi-query task sequencing method designed to operate in semi-structured environments with a combination of static and non-static obstacles. Our method intentionally trades off workspace generality for planning efficiency. Given a user-defined task space with static obstacles, we compute a subspace decomposition. The key idea is to establish approximate isometries known as $\epsilon$-Gromov-Hausdorff approximations that identify points that are close to one another in both task and configuration space. Importantly, we prove bounded suboptimality guarantees on the lengths of paths within these subspaces. These bounding relations further imply that paths within the same subspace can be smoothly concatenated, which we show is useful for determining efficient task sequences. We evaluate our method with several kinematic configurations in a complex simulated environment, achieving up to 3x faster motion planning and 5x lower maximum trajectory jerk compared to baselines.

This paper introduces a reconfigurable underwater robot system, RS-ModCubes, which allows scalable multi-robot configurations. An RS-ModCubes system comprises multiple ModCube modules, that can travel underwater with 6 DoFs and assemble with each other into a larger structure with onboard electromagnets. This system is designed to explore the feasibility of modularity and reconfigurability in underwater robotics, eliminating the need to customize mechanical designs for a specific task. We develop a modeling framework for reconfigurable robot systems, incorporating hydrodynamic effects via Monte Carlo approximation. A model-based feedforward A model-based feedforward Proportional Derivative (PD) controller serves as the baseline for control. Inspired by dexterous manipulation, we evaluated the robot's maximum task wrench space and power efficiency, compared against four commercial underwater robots. RS-ModCubes is validated via both real-world experiments and simulations, including individual and multi-module trajectory tracking and hovering docking. We open-source the design and code to facilitate future research: https://jiaxi-zheng.github.io/ModCube.github.io/.

Contemporary robots are complex systems. Teaching novel motion patterns to robots requires specialised expertise, often entailing the careful specification of robot motion or the cumbersome design of optimisation problems. In this letter, we seek to simplify the process of generating periodic motions, by teaching robots with user sketches. In particular, we tackle the problem of teaching a robot to approach a surface and then follow cyclic motion on the surface. The limit cycle of the motion can be arbitrarily specified by a single user-provided sketch over an image from the robot's camera, and the sketched limit cycle is then projected into the scene. To generate motion that converges to the limit cycle, we contribute the Stable Periodic Diagrammatic Teaching (SPDT) framework. SPDT models the robot's motion as an Orbitally Asymptotically Stable (O.A.S.) dynamical system that learns to stabilise based on the diagrammatic sketch provided by the user. This is achieved by applying a differentiable and invertible function, known as a diffeomorphism, to shape a known O.A.S. system. The parameterised diffeomorphism is then optimised with respect to the Hausdorff distance between the limit cycle of our modelled system and the sketch, to produce the desired robot motion. We provide insight into the behaviour of the optimised system and empirically evaluate SPDT. Results show that we can diagrammatically teach complex cyclic motion patterns with accuracy.

Water caustics are commonly observed in seafloor imaging data from shallow-water areas. Traditional methods that remove caustic patterns from images often rely on 2D filtering or pre-training on an annotated dataset, hindering the performance when generalizing to real-world seafloor data with 3D structures. In this paper, we present a novel method Recurrent Gaussian Splatting (RecGS), which takes advantage of today's photorealistic 3D reconstruction technology,3D Gaussian Splatting (3DGS), to separate caustics from seafloor imagery. With a sequence of images taken by an underwater robot, we build 3DGS recurrently and decompose the caustic with low-pass filtering in each iteration. In the experiments, we analyze and compare with different methods, including joint optimization, 2D filtering, and deep learning approaches. The results show that our proposed RecGS paradigm can effectively separate the caustic from the seafloor, improving the visual appearance, and can be potentially applied on more problems with inconsistent illumination. Our videos and code are released at https://tyz1030.github.io/proj/recgs.html