Xiang Li
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Research Subjects
Xiang Li has a total of 29 co-authors. Here are the top co-authors by number of shared publications:
Yasushi Yagi
D3 Center, The University of Osaka, Osaka, Japan
12 shared publicationsYasushi Makihara
Osaka University,Osaka,Japan
11 shared publicationsChi Xu
The University of Osaka
10 shared publications- Yasushi Yagi
The University of Osaka
9 shared publications - Chi Xu
Department of Intelligent Media, SANKEN, Osaka University, Osaka, Japan
8 shared publications - Yasushi Makihara
Osaka Univ. (Japan)
7 shared publications
Latest Publications
- Open Access
- 1 November 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Biometrics, Behavior, and Identity Science
- Vol. 8 (6) , 1195-1210
OU-MB: The OU Multimodal Biometric Database and Its Performance Evaluation
- 1 November 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Biometrics, Behavior, and Identity Science
- Vol. 8 (6) , 1195-1210
Open AccessOU-MB: The OU Multimodal Biometric Database and Its Performance Evaluation
- Open Access
- 1 January 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Biometrics, Behavior, and Identity Science
- Vol. 8 (5) , 1
OU-CA: A Face–Voice Multimodal Benchmark for Continuous User Authentication
- 1 January 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Biometrics, Behavior, and Identity Science
- Vol. 8 (5) , 1
Open AccessOU-CA: A Face–Voice Multimodal Benchmark for Continuous User Authentication
- Open Access
- 17 February 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 13, 31949-31962
Three-Dimensional Human Model-Guided View Morphing for Clothed Gait Silhouettes
Silhouette-based gait recognition is a popular biometric modality because of its non-intrusive nature and ability to capture identity information from a distance and at low resolution. However, cross-view gait recognition presents significant challenges because the appearance changes drastically across different viewpoints. Existing gait datasets are sparse in view coverage, limiting the generalization of models to novel views. The evaluation of current state-of-the-art gait recognition models is confined to the views present in the training dataset, posing challenges for practical application scenarios. In this paper, we propose a three-dimensional (3D) human-model-guided view-morphing framework to generate high-quality gait silhouettes for arbitrary views, enhancing gait recognition performance for novel views unseen during training. By leveraging the body topology of the 3D human model, unlike existing GAN-based or deformation-based methods that rely only on two-dimensional images, the proposed method enables fine control over the output views and achieves plausible deformations, even for self-occluded body parts, through the supervision of the pseudo ground-truth deformation field computed from the 3D model. To enhance the realism of silhouette generation, we use clothed silhouettes alongside skin silhouettes rendered from the 3D model (i.e., the SMPL) to train the deformation estimation network to account for clothing and accessories in the final output. We evaluated our approach on the OU-MVLP, CASIA-B, and CASIA-E datasets, demonstrating significant improvements in gait recognition performance, particularly in view-limited scenarios. For example, recovering 6.47% rank-1 recognition rate on OU-MVLP using three close views only and an average of 24.36% and 8.34% on CASIA-B and CASIA-E normal clothing, respectively, using a single view. Our framework increases the robustness of the cross-view problem, enhancing the applicability of gait recognition in real-world settings such as surveillance and security.
- 17 February 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 13, 31949-31962
Open AccessThree-Dimensional Human Model-Guided View Morphing for Clothed Gait Silhouettes
Silhouette-based gait recognition is a popular biometric modality because of its non-intrusive nature and ability to capture identity information from a distance and at low resolution. However, cross-view gait recognition presents significant challenges because the appearance changes drastically across different viewpoints. Existing gait datasets are sparse in view coverage, limiting the generalization of models to novel views. The evaluation of current state-of-the-art gait recognition models is confined to the views present in the training dataset, posing challenges for practical application scenarios. In this paper, we propose a three-dimensional (3D) human-model-guided view-morphing framework to generate high-quality gait silhouettes for arbitrary views, enhancing gait recognition performance for novel views unseen during training. By leveraging the body topology of the 3D human model, unlike existing GAN-based or deformation-based methods that rely only on two-dimensional images, the proposed method enables fine control over the output views and achieves plausible deformations, even for self-occluded body parts, through the supervision of the pseudo ground-truth deformation field computed from the 3D model. To enhance the realism of silhouette generation, we use clothed silhouettes alongside skin silhouettes rendered from the 3D model (i.e., the SMPL) to train the deformation estimation network to account for clothing and accessories in the final output. We evaluated our approach on the OU-MVLP, CASIA-B, and CASIA-E datasets, demonstrating significant improvements in gait recognition performance, particularly in view-limited scenarios. For example, recovering 6.47% rank-1 recognition rate on OU-MVLP using three close views only and an average of 24.36% and 8.34% on CASIA-B and CASIA-E normal clothing, respectively, using a single view. Our framework increases the robustness of the cross-view problem, enhancing the applicability of gait recognition in real-world settings such as surveillance and security.
- Open Access
- 8 July 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Biometrics, Behavior, and Identity Science
- Vol. 7 (1) , 47-60
GaitAGE: Gait Age and Gender Estimation Based on an Age- and Gender-Specific 3D Human Model
Gait-based human age and gender estimation has potential applications in visual surveillance, such as searching for specific pedestrian groups and automatically counting customers by different ages/genders. Unlike most existing methods that exploit widely used appearance-based gait features (e.g., gait energy image and silhouettes) or simple model-based gait features (e.g., leg length, stride width/frequency, and head-to-body ratio), we explore a recently popular 3D human mesh model (i.e., skinned multi-person linear model (SMPL)), which is more robust to various covariates (e.g., view angles). Furthermore, instead of the commonly used gender-neutral SMPL model, we propose a simple yet effective method to generate more realistic age- and gender-specific human mesh models by interpolating among male, female, and infant SMPL models using two learned age and gender weights. The age weight controls the proportion of importance between male/female and infant models, which is learned in a data-driven scheme by considering the paired relation between ground-truth ages and age weights. The gender weight controls the proportion of importance between male and female models, which indicates the gender probability. Then, we explore the use of generated realistic mesh models for age and gender estimation. Finally, the human mesh reconstruction and age and gender estimation modules are integrated into a unified end-to-end framework for training and testing. The experimental results on the OU-MVLP and FVG datasets demonstrated that the proposed method achieved both good mesh reconstruction and state-of-the-art age and gender estimation results.
- 8 July 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Biometrics, Behavior, and Identity Science
- Vol. 7 (1) , 47-60
Open AccessGaitAGE: Gait Age and Gender Estimation Based on an Age- and Gender-Specific 3D Human Model
Gait-based human age and gender estimation has potential applications in visual surveillance, such as searching for specific pedestrian groups and automatically counting customers by different ages/genders. Unlike most existing methods that exploit widely used appearance-based gait features (e.g., gait energy image and silhouettes) or simple model-based gait features (e.g., leg length, stride width/frequency, and head-to-body ratio), we explore a recently popular 3D human mesh model (i.e., skinned multi-person linear model (SMPL)), which is more robust to various covariates (e.g., view angles). Furthermore, instead of the commonly used gender-neutral SMPL model, we propose a simple yet effective method to generate more realistic age- and gender-specific human mesh models by interpolating among male, female, and infant SMPL models using two learned age and gender weights. The age weight controls the proportion of importance between male/female and infant models, which is learned in a data-driven scheme by considering the paired relation between ground-truth ages and age weights. The gender weight controls the proportion of importance between male and female models, which indicates the gender probability. Then, we explore the use of generated realistic mesh models for age and gender estimation. Finally, the human mesh reconstruction and age and gender estimation modules are integrated into a unified end-to-end framework for training and testing. The experimental results on the OU-MVLP and FVG datasets demonstrated that the proposed method achieved both good mesh reconstruction and state-of-the-art age and gender estimation results.
- Open Access
- 10 April 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing
- Vol. 17 (19391404) , 8527-8542
PolSAR-MPIformer: A Vision Transformer Based on Mixed Patch Interaction for Dual-Frequency PolSAR Image Adaptive Fusion Classification
Vision transformer (ViT) provides new ideas for polarization synthetic aperture radar (PolSAR) image classification due to its advantages in learning global-spatial information. However, the lack of local-spatial information within samples and correlation information among samples, as well as the complexity of network structure, limit the application of ViT in practice. In addition, dual-frequency PolSAR data provide rich information, but there are fewer related studies compared to single-frequency classification algorithms. In this article, we adopt ViT as the basic framework, and propose a novel model based on mixed patch interaction for dual-frequency PolSAR image adaptive fusion classification (PolSAR-MPIformer). First, a mixed patch interaction (MPI) module is designed for the feature extraction, which replaces the high-complexity self-attention in ViT with patch interaction intra- and intersample. Besides the global-spatial information learning within samples by ViT, the MPI module adds the learning of local-spatial information within samples and correlation information among samples, thereby obtaining more discriminative features through a low-complexity network. Subsequently, a dual-frequency adaptive fusion (DAF) module is constructed as the classifier of PolSAR-MPIformer. On the one hand, the attention mechanism is utilized in DAF to reduce the impact of speckle noise while preserving details. On the other hand, the DAF evaluates the classification confidence of each band and assigns different weights accordingly, which achieves reasonable utilization of the complementarity between dual-frequency data and improves classification accuracy. Experiments on four real dual-frequency PolSAR datasets substantiate the superiority of the proposed PolSAR-MPIformer over other state-of-the-art algorithms.
- 10 April 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing
- Vol. 17 (19391404) , 8527-8542
Open AccessPolSAR-MPIformer: A Vision Transformer Based on Mixed Patch Interaction for Dual-Frequency PolSAR Image Adaptive Fusion Classification
Vision transformer (ViT) provides new ideas for polarization synthetic aperture radar (PolSAR) image classification due to its advantages in learning global-spatial information. However, the lack of local-spatial information within samples and correlation information among samples, as well as the complexity of network structure, limit the application of ViT in practice. In addition, dual-frequency PolSAR data provide rich information, but there are fewer related studies compared to single-frequency classification algorithms. In this article, we adopt ViT as the basic framework, and propose a novel model based on mixed patch interaction for dual-frequency PolSAR image adaptive fusion classification (PolSAR-MPIformer). First, a mixed patch interaction (MPI) module is designed for the feature extraction, which replaces the high-complexity self-attention in ViT with patch interaction intra- and intersample. Besides the global-spatial information learning within samples by ViT, the MPI module adds the learning of local-spatial information within samples and correlation information among samples, thereby obtaining more discriminative features through a low-complexity network. Subsequently, a dual-frequency adaptive fusion (DAF) module is constructed as the classifier of PolSAR-MPIformer. On the one hand, the attention mechanism is utilized in DAF to reduce the impact of speckle noise while preserving details. On the other hand, the DAF evaluates the classification confidence of each band and assigns different weights accordingly, which achieves reasonable utilization of the complementarity between dual-frequency data and improves classification accuracy. Experiments on four real dual-frequency PolSAR datasets substantiate the superiority of the proposed PolSAR-MPIformer over other state-of-the-art algorithms.
- 2 October 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 3191-3201
Occluded Gait Recognition via Silhouette Registration Guided by Automated Occlusion Degree Estimation
Gait recognition tasks often face significant difficulties caused by partial occlusions of the human body. To address this challenge, we propose a silhouette registration method based on flexible estimation of the spatial scale associated with the occluding elements. Existing appearance-based methods require prior knowledge about the spatial scale of the human body in relation to the input image, or a bounding box that includes the actual full body. In our method, the region corresponding to the silhouette of the body is estimated directly from visible body parts within the image. This estimate is then used to normalize and register the human body by adapting it to the scale of the occlusions. To reduce the occlusion difference between elements of a matching pair, which may lead to substantial intra-subject variation when the difference is large, we use a pairwise mask to extract common visible regions for subsequent feature learning and matching. Experiments on the synthetic occluded OU-MVLP dataset demonstrate the effectiveness of the proposed method, which successfully improves recognition performance when matching pairs present occlusion differences. We discuss specific characteristics of the proposed silhouette registration and pairwise masking methods with the aid of detailed quantitative and qualitative evaluations, in the hope of providing useful insights for future research on this topic.
- 2 October 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 3191-3201
Occluded Gait Recognition via Silhouette Registration Guided by Automated Occlusion Degree Estimation
Gait recognition tasks often face significant difficulties caused by partial occlusions of the human body. To address this challenge, we propose a silhouette registration method based on flexible estimation of the spatial scale associated with the occluding elements. Existing appearance-based methods require prior knowledge about the spatial scale of the human body in relation to the input image, or a bounding box that includes the actual full body. In our method, the region corresponding to the silhouette of the body is estimated directly from visible body parts within the image. This estimate is then used to normalize and register the human body by adapting it to the scale of the occlusions. To reduce the occlusion difference between elements of a matching pair, which may lead to substantial intra-subject variation when the difference is large, we use a pairwise mask to extract common visible regions for subsequent feature learning and matching. Experiments on the synthetic occluded OU-MVLP dataset demonstrate the effectiveness of the proposed method, which successfully improves recognition performance when matching pairs present occlusion differences. We discuss specific characteristics of the proposed silhouette registration and pairwise masking methods with the aid of detailed quantitative and qualitative evaluations, in the hope of providing useful insights for future research on this topic.
- 25 September 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 1-10
Online Model-based Gait Age and Gender Estimation
This paper presents an online human model-based framework for gait-based age and gender estimation from a sequence of monocular frames. More specifically, we fine-tune a human mesh recovery model (i.e., HMR) to estimate the shape and pose parameters of a predefined 3D human model (i.e., SMPL). We then utilize the estimated parameters to predict the age and gender of the walking subject. To make the age and gender estimation task more favorable for real-time applications, we consider estimating the corresponding probability distributions of age and gender, which preserve the prediction uncertainty. Experiments on the world’s largest multi-view gait age and gender estimation dataset showed the superiority of the proposed method compared to the existing appearance-based baseline. We implement online standalone and client-server systems based on the proposed framework to demonstrate the performance of real-time estimation. We further propose a geometric correction step to the input gait sequence for a more generalization capability of the online system.
- 25 September 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 1-10
Online Model-based Gait Age and Gender Estimation
This paper presents an online human model-based framework for gait-based age and gender estimation from a sequence of monocular frames. More specifically, we fine-tune a human mesh recovery model (i.e., HMR) to estimate the shape and pose parameters of a predefined 3D human model (i.e., SMPL). We then utilize the estimated parameters to predict the age and gender of the walking subject. To make the age and gender estimation task more favorable for real-time applications, we consider estimating the corresponding probability distributions of age and gender, which preserve the prediction uncertainty. Experiments on the world’s largest multi-view gait age and gender estimation dataset showed the superiority of the proposed method compared to the existing appearance-based baseline. We implement online standalone and client-server systems based on the proposed framework to demonstrate the performance of real-time estimation. We further propose a geometric correction step to the input gait sequence for a more generalization capability of the online system.
- Open Access
- 11 January 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Information Forensics and Security
- Vol. 18 (15566013) , 1309-1321
Occlusion-Aware Human Mesh Model-Based Gait Recognition
Partial occlusion of the human body caused by obstacles or a limited camera field of view often occurs in surveillance videos, which affects the performance of gait recognition in practice. Existing methods for gait recognition against occlusion require a bounding box or the height of a full human body as a prerequisite, which is unobserved in occlusion scenarios. In this paper, we propose an occlusion-aware model-based gait recognition method that works directly on gait videos under occlusion without the above-mentioned prerequisite. Specifically, given a gait sequence that only contains non-occluded body parts in the images, we directly fit a skinned multi-person linear (SMPL)-based human mesh model to the input images without any pre-normalization or registration of the human body. We further use the pose and shape features extracted from the estimated SMPL model for recognition purposes, and use the extracted camera parameters in the occlusion attenuation module to reduce intra-subject variation in human model fitting caused by occlusion pattern differences. Experiments on occlusion samples simulated from the OU-MVLP dataset demonstrated the effectiveness of the proposed method, which outperformed state-of-the-art gait recognition methods by about 15% rank-1 identification rate and 2% equal error rate in the identification and verification scenarios, respectively.
- 11 January 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Information Forensics and Security
- Vol. 18 (15566013) , 1309-1321
Open AccessOcclusion-Aware Human Mesh Model-Based Gait Recognition
Partial occlusion of the human body caused by obstacles or a limited camera field of view often occurs in surveillance videos, which affects the performance of gait recognition in practice. Existing methods for gait recognition against occlusion require a bounding box or the height of a full human body as a prerequisite, which is unobserved in occlusion scenarios. In this paper, we propose an occlusion-aware model-based gait recognition method that works directly on gait videos under occlusion without the above-mentioned prerequisite. Specifically, given a gait sequence that only contains non-occluded body parts in the images, we directly fit a skinned multi-person linear (SMPL)-based human mesh model to the input images without any pre-normalization or registration of the human body. We further use the pose and shape features extracted from the estimated SMPL model for recognition purposes, and use the extracted camera parameters in the occlusion attenuation module to reduce intra-subject variation in human model fitting caused by occlusion pattern differences. Experiments on occlusion samples simulated from the OU-MVLP dataset demonstrated the effectiveness of the proposed method, which outperformed state-of-the-art gait recognition methods by about 15% rank-1 identification rate and 2% equal error rate in the identification and verification scenarios, respectively.
- Open Access
- 1 April 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Biometrics, Behavior, and Identity Science
- Vol. 4 (2) , 234-248
Multi-View Large Population Gait Database With Human Meshes and Its Performance Evaluation
Existing model-based gait databases provide the 2D poses (i.e., joint locations) extracted by general pose estimators as the human model. However, these 2D poses suffer from information loss and are of relatively low quality. In this paper, we consider a more informative 3D human mesh model with parametric pose and shape features, and propose a multi-view training framework for accurate mesh estimation. Unlike existing methods, which estimate a mesh from a single view and suffer from the ill-posed estimation problem in 3D space, the proposed framework takes asynchronous multi-view gait sequences as input and uses both multi-view and single-view streams to learn consistent and accurate mesh models for both multi-view and single-view sequences. After applying the proposed framework to the existing OU-MVLP database, we establish a large-scale gait database with human meshes (i.e., OUMVLP-Mesh), containing over 10,000 subjects and up to 14 view angles. Experimental results show that the proposed framework estimates human mesh models more accurately than similar methods, providing models of sufficient quality to improve the recognition performance of a baseline model-based gait recognition approach.
- 1 April 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Biometrics, Behavior, and Identity Science
- Vol. 4 (2) , 234-248
Open AccessMulti-View Large Population Gait Database With Human Meshes and Its Performance Evaluation
Existing model-based gait databases provide the 2D poses (i.e., joint locations) extracted by general pose estimators as the human model. However, these 2D poses suffer from information loss and are of relatively low quality. In this paper, we consider a more informative 3D human mesh model with parametric pose and shape features, and propose a multi-view training framework for accurate mesh estimation. Unlike existing methods, which estimate a mesh from a single view and suffer from the ill-posed estimation problem in 3D space, the proposed framework takes asynchronous multi-view gait sequences as input and uses both multi-view and single-view streams to learn consistent and accurate mesh models for both multi-view and single-view sequences. After applying the proposed framework to the existing OU-MVLP database, we establish a large-scale gait database with human meshes (i.e., OUMVLP-Mesh), containing over 10,000 subjects and up to 14 view angles. Experimental results show that the proposed framework estimates human mesh models more accurately than similar methods, providing models of sufficient quality to improve the recognition performance of a baseline model-based gait recognition approach.
- 1 October 2021
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 4089-4098
End-to-end Model-based Gait Recognition using Synchronized Multi-view Pose Constraint
We propose an end-to-end model-based cross-view gait recognition which employs pose sequences and shapes extracted by human model fitting. Specifically, we consider a problem setting where gait sequences from single different views are given as a pair to match in a test phase, while asynchronous multi-view gait sequences are given for each subject in a training phase. This work exploits multi-view constraint in the training phase to extract more consistent pose sequences from any views in the test phase, unlike the existing methods do not consider them. For this purpose, given asynchronous multi-view gait sequences, we introduce a phase synchronization step in the training phase so that we can impose pose consistency at each synchronized phase in a temporally up-sampled phase domain. We then train our network by minimizing a loss function based on the synchronized multi-view pose constraint as well as shape consistency, temporal pose smoothness, recognition accuracy, etc in an end-to-end manner. We also introduce the synchronization step in a test phase to reduce intra-subject variations caused by asynchronous pose features. Experimental results on the OU-MVLP and CASIA-B datasets show that the proposed method achieves the state-of-the-art performance for both gait identification and verification scenarios, especially a great improvement in terms of the pose representations.
- 1 October 2021
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 4089-4098
End-to-end Model-based Gait Recognition using Synchronized Multi-view Pose Constraint
We propose an end-to-end model-based cross-view gait recognition which employs pose sequences and shapes extracted by human model fitting. Specifically, we consider a problem setting where gait sequences from single different views are given as a pair to match in a test phase, while asynchronous multi-view gait sequences are given for each subject in a training phase. This work exploits multi-view constraint in the training phase to extract more consistent pose sequences from any views in the test phase, unlike the existing methods do not consider them. For this purpose, given asynchronous multi-view gait sequences, we introduce a phase synchronization step in the training phase so that we can impose pose consistency at each synchronized phase in a temporally up-sampled phase domain. We then train our network by minimizing a loss function based on the synchronized multi-view pose constraint as well as shape consistency, temporal pose smoothness, recognition accuracy, etc in an end-to-end manner. We also introduce the synchronization step in a test phase to reduce intra-subject variations caused by asynchronous pose features. Experimental results on the OU-MVLP and CASIA-B datasets show that the proposed method achieves the state-of-the-art performance for both gait identification and verification scenarios, especially a great improvement in terms of the pose representations.