Guoli Wang

It is challenging to reconstruct human pose in multi-person scenes using a single commercial millimeter-wave (mmWave) radar due to its limited resolution and susceptibility to noise. On the other hand, the signal processing process may cause the loss of detailed features of the raw radar signals or introduce errors, making it difficult to perform detailed analysis of radar signals inmulti-person scenes. To address these issues, Cube Pose Transformer (CPFormer) is proposed, an end-to-end method for multi-person pose estimation from raw radar cubes. Specifically, the CPFormer consists of a learnable 3D-discrete Fourier transform (DFT) module and Transformer-based networks. The learnable 3D-DFT module extracts features from raw radar cubes and adaptively learns the time-to-frequency domain transformation for each dimension, replacing the traditional DFT. The Transformer includes the dual-stream hierarchical encoder (DHE) and the multi-person pose decoder (MPD). Firstly, the proposed spatio-temporal fusion tokenizer (SFT) captures the spatio-temporal cues of adjacent frames and represents the radar cubes as token embeddings. Then, the DHE uses window attention and global cross-view attention (GCVA) to learn the local, global, and cross-view dependencies from the radar cubes of the horizontal and vertical views, to extract fine-grained sensing cues. The MPD directly predicts multi-person poses based on features extracted by the encoder, without separating signals of different targets in the low-resolution radar data. Evaluated on a multi-person human pose dataset collected with a TI AWR1843 Boost mmWave radar in two different environments, the CPFormer achieves the lowest pose estimation error.

With the aim of promoting independent living and aging well, indoor location tracking has been a research focus in the area of Ambient Assisted Living (AAL) environment. In particular, there has been growing interest in object tracking using fiber-optic sensors due to the advantage of data-efficient, low-computational-cost and non-intrusive. However, the current research on object tracking with fiber-optic sensors confronts several critical challenges, such as increasing the sensing efficiency and limitations of unnatural walking manner. To address these challenges, in this paper, a compressive floor pressure sensing approach is proposed based on human bipedal locomotion(HBL) characteristics. Specifically, to retain the natural characteristic of HBL, the spatio-temporal representation of HBL locations is expressed as a three-state vector by dividing the monitoring area into an 8-neighbour-grid structured environment. The spatial sparsity of HBL locations is then incorporated into compressive sensing mechanism for efficiency enhancement. More specifically, a Triplet Measurement Vector and Dynamic Uniquely Decipherable (TMV-DUD) encoding scheme is used to form the compressive sensing matrix. A prototype system of compressive floor pressure sensing is developed for indoor location tracking within 36 grids using only 5 sensors. The prototype experimental results demonstrate the effectiveness of the proposed compressive tracking approach.

This paper presents VirTeach, the exploitation of using the virtual point cloud (VPC) as an assisted teacher in the learning process for human pose estimation incorporated with the millimeter wave (mmWave) radar point cloud (RPC). Due to the observations that the involvement of different body parts varies in moving ranges and directions while performing postures and mmWave signals possess inherent characteristics (i.e., specular reflection, radical sensitiveness) during their perception, the RPC data suffer from the issues of blind spots and data imbalance for the response points induced by specific joints, leading to insufficient and biased learning and thus large estimation errors for them. To address these issues, we introduce the VPC data driven by real human motions to assist the learning process, which is indispensable in explicitly imposing task-specific constraints for the distorted RPC data in a fashion of learning by teaching. Specifically, we first design a generation module to produce the desired VPC data considering both the global structure and local motions of the human skeleton, serving as the “teacher” to augment the corrupted RPC data. Secondly, we incorporate the global and local guidance from the VPC data within a coarse-to-fine pose estimation framework. The former addresses the blind spots issue by completing the RPC data to facilitate the global skeleton reconstruction, while the latter is targeted for strengthening the contribution of specific joints through constructing the local spatial-temporal neighborhood to further refine their positions. Extensive experiments are conducted to validate the effectiveness of the proposed method.

Locating human joints using low-cost commercial millimeter-wave (mmWave) radars is an emerging technology. Compared to single-view radar, dual-view radar fusion is more efficient for robust arm gesture pose tracking since it can capture a wider area of human reflection and radial velocity in both directions. One challenging task for dual-view radar fusion is to effectively utilize the reflection points’ spatial and velocity information, and focus perceptual cues related to joint position and velocity for accurate joint tracking. In this paper, we propose Millimeter wave Pose Tracking Transformer (MPTFormer), a method for arm gesture pose tracking using both front and side-view mmWave radar. Specifically, MPTFormer consists of Multi-domain Fusion Transformer (MFFormer) and Joint Tracking Transformer (JTFormer). In MFFormer, joint position and velocity estimation tasks are set, and the Range-Azimuth Map (RAM) and Range-Doppler Map (RDM) of the dual-view radar are set as token embeddings, which are interacted with joint position and velocity tokens for learning perceptual cues related to joint position and velocity as well as joint constraint relationships. Meanwhile, the non-local dependencies between dual-view radar and its multi-domain features are mined to obtain good global feature representation. In JTFormer, a hybrid learning architecture of Temporal Transformer and LSTM is designed, which tracks the joint position according to the global and local features of the joint in temporal dimension. Finally, in the pose tracking task containing 12 different arm gestures, the proposed method can achieve 25.2mm MPJPE and 19.1mm P-MPJPE.

It is a challenging task to reconstruct human pose from millimeter wave (mmWave) radar point clouds due to their sparsity and sensitivity to multipath noise. In order to enhance the inference ability of deep learning models in processing sparse radar point clouds, a learning paradigm with spatial and temporal multi-scale constraints is proposed, utilizing prior relationships in skeletal structure and joint movements in both time and space to constrain the learning of pose sequences. Specifically, the proposed spatial multi-scale constraint block learns the spatial constraint relationships of human joints using three different scales: adjacent joint constraint, part-level kinematic constraint, and global joint constraint, by which the spatial joint constraint features are aggregated by fusion gate mechanism. On the other hand, the temporal multi-scale constraint block is devised to learn the temporal constraint relationships of the joint trajectories using the information of the joint itself and local context information in time domain. Compared with the single-scale constrained learning paradigm, the potential advantage of the proposed method is that it can reduce the impact of random missing and noise in radar data. Finally, the effectiveness and superiority of the proposed method are fully demonstrated through the experimental results on two public datasets of human pose estimation based on mmWave radar.