Xiaofeng Guo

Gesture recognition constitutes a core modality for human–robot interaction (HRI). Current studies primarily address short-distance scenarios, whereas long-distance outdoor scenarios remain significantly underexplored. The principal technical challenges arise from three key challenges: First, the predominant separated spatio-temporal processing approach in current network architectures and attention mechanisms. 1) introduces redundant structural complexity and computational demands, severely constraining unmanned aerial vehicle (UAV) platform implementation. 2) Inherently limits the model’s ability to learn weakly-correlated spatio-temporal relationships. 3) Additionally, existing HRI systems demonstrate inherent deficiencies in accurate hand localization and skeleton estimation under long-distance conditions. To address these issues, we propose a lightweight spatio-temporal skeleton (STS) attention transformer for long-distance gesture recognition in UAV control. First, we designed and implemented a lightweight STS Transformer to unify spatial-temporal feature processing while maintaining structural simplicity. Second, we developed an STS attention employing coupled-decoupled operators on skeleton sequences to extract weakly-correlated features. Subsequently, we developed a two-stage RGB+D framework for hand detection and skeleton estimation, achieving capability at extended ranges up to 5 m under UAV onboard camera view and 10 m in third-party camera view. Finally, extensive experiments on two datasets and real-world experiments were conducted to demonstrate the effectiveness of our network in skeleton-based gesture recognition, showing a clear advantage over the state-of-the-art (SOTA) methods.

Category-level object pose estimation and tracking has achieved impressive progress in computer vision, augmented reality, and robotics. Existing methods either estimate the object states from a single observation or only track the 6-DoF pose of a single object. In this paper, we focus on category-level multi-object 9-Dimensional (9D) state tracking from the point cloud stream. We propose a novel 9D state estimation network to estimate the 6-DoF pose and 3D size of each instance in the scene. It uses our devised multi-scale global attention and object-level local attention modules to obtain representative latent features to estimate the 9D state of each object in the current observation. We then integrate our network estimation into a Kalman filter to combine previous states with the current estimates and achieve multi-object 9D state tracking. Experiment results on two public datasets show that our method achieves state-of-the-art performance on both category-level multi-object state estimation and pose tracking tasks. Furthermore, we directly apply the pre-trained model of our method to our air-ground robot system with multiple moving objects. Experiments on our collected real-world dataset show our method's strong generalization ability and real-time pose tracking performance.

For the safety of autonomous driving systems, recognizing the gestures of cyclists and controllers on traffic roads is crucial. However, general action recognition methods are unsuitable for traffic gestures. This is because, compared to general actions, traffic gestures exhibit distinct characteristics, namely: (1) short action duration; (2) atypical action posture; and (3) small dataset sizes . We propose a novel two-stream graph convolutional network (2S-GCN) to address these challenges. Firstly, we introduced a motion-guided module connecting the motion and joint stream. By leveraging the motion stream’s spatial expression capability, we guided the joint stream’s learning in spatial dimensions to enhance the network’s performance to short action durations. Secondly, we utilized a simplified skeleton topology that exclusively captures the upper body during data preprocessing, thereby preserving essential information while eliminating redundancy. This approach enhances the network’s adaptability to atypical action postures. Thirdly, we use a GCN embedded with a multi-channel attention module as the backbone, which is particularly suitable for small datasets, and conduct extensive experiments on two datasets. The experimental results demonstrate that our network can accurately recognize traffic gestures and has significant advantages over state-of-theart methods.

For autonomous driving systems, it is crucial to recognize the actions and gestures of traffic conductors and cyclists on the road to ensure safety. However, traffic gesture recognition is more challenging than action recognition in general scenarios due to the differences in action posture and sample composition between traffic gesture datasets and general action datasets. Therefore, general action recognition methods cannot identify traffic gestures well. To overcome these problems, we propose a novel motion-guided graph convolutional transformer (MG-GCT) for traffic gesture recognition. Firstly, we proposed a two-stream network to fully utilize joint data and motion data for action recognition. Secondly, we designed and implemented a motion-guided module between two streams, which leverages the powerful spatial representation ability of the motion data to guide the learning of the joint data stream in the spatial dimension. Thirdly, we implemented a temporal transformer network to process the temporal features of the skeleton. Finally, we conducted extensive experiments on two public datasets and one dataset presented by us to demonstrate the effectiveness of our network in traffic gesture recognition, which has a significant advantage over the state-of-the-art methods.

Human–robot interaction (HRI) systems are crucial in robotics, natural, fast-response, and multimodal are the future trends in their development. However, current interaction methods have the following flaws: 1) slow response of action recognition algorithms for generic scenes, especially at the beginning stage; 2) insufficient feature extraction and fusion capabilities for spatiotemporal graph data; and 3) no good paradigm of body–hand recognition in HRI. To overcome these bottlenecks, we propose a fast-response graph convolutional network (GCN) for body–hand gesture recognition. First, we propose a dynamic-static parallel network for dynamic body gestures that is responsive and accurate. Second, we propose a spatiotemporal graph attention module to improve the graph data fusion effect in the dynamic-static network. Third, we implement a complete command module to form complete commands with body and hand information for interactions and control of the robot. Finally, extensive experiments on four datasets and real-world experiments were conducted to demonstrate that our network is capable of fast response and accurate recognition of dynamic body gestures at the beginning stage, verifying the effectiveness of skeleton-based body-hand gesture recognition, with a clear advantage over the state-of-the-art.

The 2 {\mu}m waveband is an area that could have significant technological consequences, with applications ranging from spectroscopy, LIDAR and free-space communications. The development of the thulium-doped fiber amplifier, hollow-core photonic bandgap fiber and 2 {\mu}m GaSb-based diode lasers has highlighted the ability of the waveband in alleviating the fiber capacity crisis in the incumbent communication infrastructure. The above has initiated vibrant development in the silicon photonic-space at 2 {\mu}m, where the area is capable of enabling highly-integrated photonic circuits, and potentially at low-cost and high-volumes. However, as of now, modulator linearity at 2 {\mu}m has not been addressed. The metric, as characterized by spurious free dynamic range is imperative for numerous applications such as RF photonic links for 5G and digital analog transmission in coherent communications. The development of linear optical modulators will be crucial in bringing these applications to the 2 {\mu}m. In view of that, this work is the first to address modulator linearity at the 2 {\mu}m, where the ring-assisted Mach-Zehnder modulator is developed, indicating spurious free dynamic range as high as 95 dB.Hz^2/3. It is found that that modulator spurious free dynamic range has a strong dependence on modulator bias voltage where it impacts the linearity of the transfer function in which the input RF signal is applied upon. The demonstrated modulator indicates favorable performance within silicon photonic modulators developed at 2 {\mu}m with bandwidth exceeding 17.5 GHz and modulation efficiency ranging from 0.70 to 1.25 V.cm.