Abstract
With recent advancements in generative models, controllable human image generation has achieved significant progress. However, existing methods still struggle in complex scenes involving multiple people and intricate poses, often leading to pose ambiguity and body part mismatches. Consequently, the generated images tend to lack structural coherence and visual realism. To address these challenges, we propose MultiHuman, a novel framework for multi-person human image generation that goes beyond simple pose and interaction modeling. Our approach utilizes coarse human poses and dense depth maps as layout guidance, explicitly tackling the issues of modality alignment and feature fusion in multi-human generation. To this end, we introduce a Modality Alignment Module (MAM) that projects pose and depth representations into a shared embedding space. Spatial alignment is achieved through contrastive learning, which ensures consistent and complementary layout representations across multiple people. In addition, a Pose Modulation Module (PMM) is designed to convert hard joint masks into differentiable soft masks, which improves the precision of pose control. To support training and evaluation under challenging conditions, we construct a new dataset, MTHuman, comprising 110K images that feature multi-person occlusions and complex human poses. Experiments on LAION-Human, HumanArt, and MTHuman show that MultiHuman outperforms existing methods like StablePose, with a 2.36% improvement in controllable human image generation accuracy. MultiHuman shows better image quality and stronger structure control.
Funding Information
  • hongrichang (61932007)