MultiHuman: Leverage Multimodal Prompts for Controllable Multi-Person Image Synthesizing
- 10 April 2026
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Circuits and Systems for Video Technology
- Vol. PP (10518215) , 1
Abstract
With recent advancements in generative models, controllable human image generation has achieved significant progress. However, existing methods still struggle in complex scenes involving multiple people and intricate poses, often leading to pose ambiguity and body part mismatches. Consequently, the generated images tend to lack structural coherence and visual realism. To address these challenges, we propose MultiHuman, a novel framework for multi-person human image generation that goes beyond simple pose and interaction modeling. Our approach utilizes coarse human poses and dense depth maps as layout guidance, explicitly tackling the issues of modality alignment and feature fusion in multi-human generation. To this end, we introduce a Modality Alignment Module (MAM) that projects pose and depth representations into a shared embedding space. Spatial alignment is achieved through contrastive learning, which ensures consistent and complementary layout representations across multiple people. In addition, a Pose Modulation Module (PMM) is designed to convert hard joint masks into differentiable soft masks, which improves the precision of pose control. To support training and evaluation under challenging conditions, we construct a new dataset, MTHuman, comprising 110K images that feature multi-person occlusions and complex human poses. Experiments on LAION-Human, HumanArt, and MTHuman show that MultiHuman outperforms existing methods like StablePose, with a 2.36% improvement in controllable human image generation accuracy. MultiHuman shows better image quality and stronger structure control.Keywords
Funding Information
- hongrichang (61932007)
This publication has 39 references indexed in Scilit:
- Texture-driven pose-guided human image synthesisPattern Analysis and Applications, 2025
- HumanRefiner: Benchmarking Abnormal Human Generation and Refining with Coarse-to-Fine Pose-Reversible Guidance2024
- ControlNet$++$: Improving Conditional Controls with Efficient Consistency Feedback2024
- Multi2Human: Controllable human image generation with multimodal controlsNeurocomputing, 2024
- Pose image generation for video content creation using controlled human pose image generation GANMultimedia Tools and Applications, 2023
- PoseTrans: A Simple yet Effective Pose Transformation Augmentation for Human Pose Estimation2022
- Pose2Seg: Detection Free Human Instance Segmentation2019
- CrowdPose: Efficient Crowded Scenes Pose Estimation and a New Benchmark2019
- Local Descriptors Optimized for Average Precision2018
- Microsoft COCO: Common Objects in Context2014