Jiaqi Shi

Understanding pedestrian behavior and predicting their intentions near roads are crucial for enhancing road safety and traffic efficiency. In intelligent transportation systems, especially where computational and energy resources are limited, real-time inference is vital. To address this, our study focuses on pedestrian skeletal information, which offers lower dimensionality and reduces computational demands. We introduce EffIntentGCN, a lightweight graph convolutional network designed to analyze pedestrian poses using a part-based graph and an adaptive graph, emphasizing features crucial for intention prediction and enhancing the learning of local and global interactions. An expansion layer added before each convolution block increases feature dimensionality for a broader representation, without excessive parameter increase. We further incorporate spatial-temporal joint attention, focusing on crucial joints in dynamic skeletal sequences. The model also integrates second-order information, representing joint connections, to provide complementary information. To balance model complexity with computational efficiency, essential for applications in resource-constrained environments like autonomous vehicles, we employ depthwise separable convolution and low-rank approximation techniques, aimed to reduce parameter count and maintain computational efficiency. The model is evaluated on the JAAD dataset, demonstrating its effectiveness in accurately predicting pedestrian crossing intentions while optimizing computational resources.

Gestures, a form of body language, significantly influence how users perceive humanoid robots. Recent data-driven methods for co-speech gestures have successfully enhanced the naturalness of the generated gestures. Moreover, compared to rule-based systems, these methods are more generalizable for unseen speech input. However, many of these methods cannot directly influence people’s perceptions of robots. The primary challenge lies in the intricacy of constructing a dataset with varied impression labels to develop a conditional generation model. In our prior work ([22]) Controlling the impression of robots via gan-based gesture generation. In:Proceedings of the international conference on intelligent robots and systems. IEEE, pp 9288-9295), we introduced a heuristic approach for automatic labeling, training a deep learning model to control robot impressions. We demonstrated the model’s effectiveness on both a virtual agent and a humanoid robot. In this study, we refined the motion retargeting algorithm for the humanoid robot and conducted a user study using four questions representing different aspects of extroversion. Our results show an improved capability in controlling the perceived degree of extroversion in the humanoid robot compared to previous methods. Furthermore, we discovered that different aspects of extroversion interact uniquely with motion statistics

Emotion recognition has drawn consistent attention from researchers recently. Although gesture modality plays an important role in expressing emotion, it is seldom considered in the field of emotion recognition. A key reason is the scarcity of labeled data containing 3D skeleton data. Some studies in action recognition have applied graph-based neural networks to explicitly model the spatial connection between joints. However, this method has not been considered in the field of gesture-based emotion recognition, so far. In this work, we applied a pose estimation based method to extract 3D skeleton coordinates for IEMOCAP database. We propose a self-attention enhanced spatial temporal graph convolutional network for skeleton-based emotion recognition, in which the spatial convolutional part models the skeletal structure of the body as a static graph, and the self-attention part dynamically constructs more connections between the joints and provides supplementary information. Our experiment demonstrates that the proposed model significantly outperforms other models and that the features of the extracted skeleton data improve the performance of multimodal emotion recognition.