Enhancing Pose-Guided Human Image Generation with Comprehensive and Adjustable 3D Control
Open Access
- 12 January 2026
- journal article
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Multimedia Computing, Communications, and Applications
- Vol. 22 (1) , 1-24
Abstract
Pose-guided human image generation aims to render a source image in a specific pose. Current methods predominantly employ 2D-based signals, which exhibit inherent information deficits, as pose conditions. This leads to difficulty in establishing precise source-target appearance-pose correspondence and further causing uncertainty in predicting self-occluded regions’ appearance. To address these issues, we propose a 3D Pose Conditional Diffusion model (3DPCD) that leverages a human parametric model to integrate comprehensive and adjustable 3D control into forward–backward diffusion steps. Specifically, we employ Fourier-transformed SMPL-X as the 3D pose representation to facilitate precise source-target correspondence by understanding the complete pose information. Building on this, we further propose a hierarchical appearance-pose alignment method, which aligns appearance with the complete pose information at both global and local levels. Moreover, motivated by the fact that human pose transformation is a progressive process in 3D space and our 3D pose representation is adjustable, we integrate progressively interpolated 3D control into a series of sampling steps. This effectively mitigates uncertainties in pixel transfer between poses. It should be noted that the proposed explicit pose-guided strategy also supports flexible adjustment of pose, shape, and viewpoint. Both quantitative and qualitative evaluations demonstrate that our 3DPCD outperforms state-of-the-art methods on the widely used DeepFashion InShop benchmark and our newly constructed PoseWeb-33 dataset, which features richer appearance variations and more diverse conditional poses.Keywords
Funding Information
- Shenzhen Science and Technology Program (CJGJZD20220517142402006)
- Major Key Project of Pengcheng Laboratory (PCL2025A12)
This publication has 43 references indexed in Scilit:
- GridFormer: Residual Dense Transformer with Grid Structure for Image Restoration in Adverse Weather ConditionsInternational Journal of Computer Vision, 2024
- Open-Vocabulary Text-Driven Human Image GenerationInternational Journal of Computer Vision, 2024
- Deep Image Spatial Transformation for Person Image Generation2020
- Expressive Body Capture: 3D Hands, Face, and Body From a Single Image2019
- Progressive Pose Attention Transfer for Person Image Generation2019
- Dense Pose Transfer2018
- End-to-End Recovery of Human Shape and Pose2018
- Deep Progressive Reinforcement Learning for Skeleton-Based Action Recognition2018
- Deformable GANs for Pose-Based Human Image Generation2018
- Human Appearance Transfer2018