Zhiyong Sun

Motion generation for robots in highly dynamic environments is challenging due to the need for high efficiency and error tolerance. Robots must generate motion efficiently within milliseconds to react to environmental changes swiftly. Furthermore, the limitations of perceptual systems in detecting fast-moving objects necessitate robust motion generation that can tolerate perceptual errors. This paper presents a motion generation framework that ensures both high efficiency and error tolerance for the dynamic task of robotic table tennis. To achieve high efficiency, this study introduces a novel analytical solution to the ball flight equation using Chebyshev approximation, offering a time complexity of O (1). For robotic motion generation, this study proposes a dimension-reduced task planner and employs piecewise cubic polynomial and Variable-Sigmoid-Based motion templates for individual joint motions, with coefficients determined analytically and rapidly. To enhance perceptual error tolerance, this study develops a policy optimizer based on a striking-at-the-center principle. The proposed task planner accelerates paddle motion determination by an average of 9.27 times. The proposed ball motion prediction method enhances the efficiency of determining the ball-paddle collision point by up to a factor of two. Additionally, the proposed framework improves the success rate of planning and striking by 16.0% and 18.3%, respectively, compared to methods that do not account for error tolerance. It also reduces the landing position error in the x and y directions by 32.5% and 35.6%, while improving the accuracy of passing-net height by 24.0%.

Efficient motion planning with the error tolerance is crucial for dynamic robotic tasks, particularly robotic table tennis. This task demands simultaneous high efficiency and error tolerance. First, the incoming ball’s high speed allows only tens of milliseconds for motion planning. Second, two types of errors, namely, ball-paddle motion uncertainty and joint limit violation, must be tolerated to ensure a high success rate of planning (SRP) and striking. This article proposes an advanced joint planning framework designed for high efficiency and error tolerance. To tolerate the error of ball-paddle motion uncertainty, this work introduces a joint classification criterion according to the joint motion characteristics. To tolerate the error of joint limit violation, based on the classification criterion, this study also develops a robust reference trajectory generation scheme, named error-tolerant-variable-sigmoid-based motion template (ETVSMT), to fully consider the motion capabilities of different joints. The ETVSMT approach utilizes the variable-sigmoid-based motion template (VSMT) as the backbone and designs its basic and remedial portions to tolerate the error of hard joint limit violation. The implementation of the ETVSMT scheme results in an average SRP of 98% and a ball-striking success rate of up to 90%, with execution time on an Intel Xeon CPU as low as 15 ms. Furthermore, the proposed method ensures the outgoing ball lands on the opposite side of the table with an average landing error of 19.62 cm and a net-passing height error of 12.68 cm. The proposed method can also benefit other dynamic tasks like human–robot interaction to improve the planning efficiency.

Deep reinforcement learning has significantly advanced robot manipulations by providing an alternative solution for designing control strategies using raw images as direct inputs. While images offer additional environmental information, the end-to-end policy training manner (from image to action) requires simultaneous representation and task learning by the agent. This often necessitates a substantial number of interaction samples to achieve satisfactory policy performance. Previous works has attempted to address this challenge by learning a visual representation model that encodes the entire image into a low-dimensional vector before the policy training. However, since this vector contains both robot and object information, it inevitably introduces coupling within the state, which can mislead the policy training process. In this study, a novel method called Reinforcement Learning with Decoupled State Representation is proposed to effectively decouple robot and object information within the state representation. Experimental results demonstrate that the proposed method exhibits faster learning speed and achieves superior performance compared to previous methods across various robot manipulation tasks. Moreover, with only 3096 offline images, the proposed method successfully applies to real-world robot pushing tasks, which demonstrates its high practicability.

In the research of table tennis robots, robot motion planning plays a crucial role as it determines the performance of striking balls back. However, to deal with the table tennis ball with high speed and spin, the robot system requires real-time motion planning with high success rate. In addition, due to the high-dimensional search space caused by the high degrees of freedom of robot motion and various types of collision constraints, searching for a feasible trajectory within a limited time is a complex problem. To address these issues, an optimization-based constraint-aware online joint-space motion planning algorithm is proposed to generate feasible trajectories achieving a high success rate and high efficiency. Specifically, a septic polynomial is employed to represent the trajectory of each joint in joint space, which can search for feasible solutions under multiple constraints while ensuring real-time computation. Five different types of constraints are considered during planning to generate a safe and feasible trajectory. Moreover, a fast remedial measure is applied to further enhance the success rate of motion planning. The proposed method achieves an average success rate over 97% in simulation and surpasses a 65% success rate on a real 5-degree-of-freedom robot system.

Accurately detecting spine vertebrae plays a crucial role in successful orthopedic surgery. However, identifying and classifying lumbar vertebrae from arbitrary spine X-ray images remains challenging due to their similar appearance and varying sizes among individuals. In this paper, we propose a novel approach to enhance vertebrae detection accuracy by leveraging both global and local spatial relationships between neighboring vertebrae. Our method incorporates a two-stage detector architecture that captures global contextual information using an intermediate heatmap from the first stage. Additionally, we introduce a detection head in the second stage to capture local spatial information, enabling each vertebra to learn neighboring spatial details, visibility, and relative offset. During inference, we employ a fusion strategy that combines spatial offsets of neighboring vertebrae and heatmap from a conventional detection head. This enables the model to better understand relationships and dependencies between neighboring vertebrae. Furthermore, we introduce a new representation of object centers that emphasizes critical regions and strengthens the spatial priors of human spine vertebrae, resulting in an improved detection accuracy. We evaluate our method using two lumbar spine image datasets and achieve promising detection performance. Compared to the baseline, our algorithm achieves a significant improvement of 13.6% AP in the CM dataset and surpasses 6.5% and 4.8% AP in the anterior and lateral views of the BUU dataset, respectively.

In table tennis, developing a precise ball-racket rebound model is crucial for predicting the trajectory and spin of a ball after it hits the racket, which is instrumental in racket design and enhancing the capabilities of table tennis robots. To this end, accuracy and computational efficiency are two challenges to overcome, which has not been perfectly handled using existing methods such as finite element and simplified rigid-body models. This paper introduces a new model that calculates ball-racket rebounds in two orthogonal directions. Vertically, the collision dynamics are analyzed with the Kelvin-Voigt model, revealing that the contact duration is independent of the incoming ball’s velocity. Horizontally, we establish that the restitution coefficient varies as a function of the incident velocity, based on a continuous contact force model and momentum conservation principles. High-speed camera data corroborate these findings and confirm the model’s efficacy across diverse conditions. Compared to an established representative model, our method not only maintains high computational efficiency but also improves the accuracy of predicting the ball’s linear and angular velocities by an average of 42.72% and 33.77%, respectively, as evidenced by our experimental data.

Medical image segmentation plays an important role in the computer-aided screening of many diseases. Different from popular deep learning-based segmentation methods that mainly focus on the whole target, recent studies pay more attention to target contour by making use of the distance transform of the object boundary. In this paper, to learn both the object content and the boundary information, we incorporate a regression task that estimates a distance map of target contour into a CNN-based segmentation network. The segmentation task and the regression task share most parts of the backbone network. Moreover, we introduce a channel information exchange module that dynamically exchanges feature maps between the segmentation and the regression tasks. The proposed information exchange module is parameter-free and controlled by the batch normalization scaling factor. In addition, the output feature maps of the two tasks are further integrated to provide fused segmentation results.We evaluate our method on three popular datasets, including the 2D ultrasound thyroid nodule segmentation dataset, the 3D MRI left atrial segmentation dataset and 2017 MM-WHS whole heart segmentation Challenge datasets. Experimental results show that our method achieves promising performance compared to the state-of-the-art approaches.