YMYifang Men

Yifang Men

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article

Previous Organizations / Institutes

The organizations / institutes from the scholar's other articles in the past ten years


Yifang Men has a total of 17 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • MorphNeRF: Text-Guided 3D-Aware Editing via Morphing Generative Neural Radiance Fields

    by Yingchen Yu, Rongliang Wu, Yifang Men, Shijian Lu, Miaomiao Cui, Xuansong Xie, Chunyan Miao

    Generative neural radiance fields (NeRF) bring image generation into the 3D era, which have delivered impressive generation quality and 3D consistency, especially in the face generation domain. Upon pre-trained generative NeRF, 3D-aware image editing has been explored and achieved promising performance via manipulating semantic maps or attributes. However, a more flexible editing interface, text, remains under-explored in the context of 3D-aware image editing. In this work, we leverage the Contrastive Language-Image Pre-training (CLIP) model to achieve 3D-aware image editing in pre-trained generative NeRF models given a target text prompt. To achieve accurate and controllable geometry editing, we propose MorphNeRF, a learnable morphing network that morphs the 3D geometry of images toward the target descriptions via generative NeRF. Different from prior studies that achieve image editing by manipulating latent codes or directly finetuning pre-trained models, morphing the geometry can better preserve the texture of the source image and facilitate the control of editing strength by adjusting the weight of morphing maps explicitly. Extensive experiments and comparisons show that the proposed MorphNeRF achieves superior image editing performance.


  • Open Access

    Your3dEmoji: Creating Personalized Emojis via One-shot 3D-aware Cartoon Avatar Synthesis

    by Shiyao Xu, Lingzhi Li, Li Shen, Yifang Men, Zhouhui Lian

    Creating cartoon-style avatars has drawn growing attention recently, however previous methods only learn face cartoonization in the 2D image level. In this paper, we propose a novel 3D generative model to translate a real-world face image into its corresponding 3D avatar with only a single style example provided. To bridge the gap between 2D real faces and 3D cartoon avatars, we leverage the state-of-the-art StyleGAN and its style-mixing property to produce a 2D paired cartoonized face dataset. We then finetune a pretrained 3D GAN with the pair data in a dual-learning mechanism to get the final synthesized 3D avatar. Furthermore, we analyze the latent space of our model, enabling manual control in what degree a style is applied. Our model is 3D-aware in the sense and also able to do attribute editing, such as smile, age, etc directly in the 3D domain. Experimental results demonstrate that our method can produce high-fidelity cartoonized avatars with true-to-life 3D geometry.


  • Open Access

    DCT-Net: Domain-Calibrated Translation for Portrait Stylization

    by Yifang Men, Yuan Yao, Miaomiao Cui, Zhouhui Lian, Xuansong Xie

    This paper introduces DCT-Net, a novel image translation architecture for few-shot portrait stylization. Given limited style exemplars ($\sim$100), the new architecture can produce high-quality style transfer results with advanced ability to synthesize high-fidelity contents and strong generality to handle complicated scenes (e.g., occlusions and accessories). Moreover, it enables full-body image translation via one elegant evaluation network trained by partial observations (i.e., stylized heads). Few-shot learning based style transfer is challenging since the learned model can easily become overfitted in the target domain, due to the biased distribution formed by only a few training examples. This paper aims to handle the challenge by adopting the key idea of "calibration first, translation later" and exploring the augmented global structure with locally-focused translation. Specifically, the proposed DCT-Net consists of three modules: a content adapter borrowing the powerful prior from source photos to calibrate the content distribution of target samples; a geometry expansion module using affine transformations to release spatially semantic constraints; and a texture translation module leveraging samples produced by the calibrated distribution to learn a fine-grained conversion. Experimental results demonstrate the proposed method's superiority over the state of the art in head stylization and its effectiveness on full image translation with adaptive deformations.

  • Controllable Image Synthesis With Attribute-Decomposed GAN

    by Guo Pu, Yifang Men, Yiming Mao, Yuning Jiang, Wei-Ying Ma, Zhouhui Lian

    This paper proposes Attribute-Decomposed GAN (ADGAN) and its enhanced version (ADGAN++) for controllable image synthesis, which can produce realistic images with desired attributes provided in various source inputs. The core ideas of the proposed ADGAN and ADGAN++ are both to embed component attributes into the latent space as independent codes and thus achieve flexible and continuous control of attributes via mixing and interpolation operations in explicit style representations. The major difference between them is that ADGAN processes all component attributes simultaneously while ADGAN++ utilizes a serial encoding strategy. More specifically, ADGAN consists of two encoding pathways with style block connections and is capable of decomposing the original hard mapping into multiple more accessible subtasks. In the source pathway, component layouts are extracted via a semantic parser and the segmented components are fed into a shared global texture encoder to obtain decomposed latent codes. This strategy allows for the synthesis of more realistic output images and the automatic separation of un-annotated component attributes. Although the original ADGAN works in a delicate and efficient manner, intrinsically it fails to handle the semantic image synthesizing task when the number of attribute categories is huge. To address this problem, ADGAN++ employs the serial encoding of different component attributes to synthesize each part of the target real-world image, and adopts several residual blocks with segmentation guided instance normalization to assemble the synthesized component images and refine the original synthesis result. The two-stage ADGAN++ is designed to alleviate the massive computational costs required when synthesizing real-world images with numerous attributes while maintaining the disentanglement of different attributes to enable flexible control of arbitrary component attributes of the synthesized images. Experimental results demonstrate the proposed methods’ superiority over the state of the art in pose transfer, face style transfer, and semantic image synthesis, as well as their effectiveness in the task of component attribute transfer. Our code and data are publicly available at https://github.com/menyifang/ADGAN .


  • A Common Framework for Interactive Texture Transfer

    by Yifang Men, Zhouhui Lian, Yingmin Tang, Jianguo Xiao

    In this paper, we present a general-purpose solution to interactive texture transfer problems that better preserves both local structure and visual richness. It is challenging due to the diversity of tasks and the simplicity of required user guidance. The core idea of our common framework is to use multiple custom channels to dynamically guide the synthesis process. For interactivity, users can control the spatial distribution of stylized textures via semantic channels. The structure guidance, acquired by two stages of automatic extraction and propagation of structure information, provides a prior for initialization and preserves the salient structure by searching the nearest neighbor fields (NNF) with structure coherence. Meanwhile, texture coherence is also exploited to maintain similar style with the source image. In addition, we leverage an improved PatchMatch with extended NNF and matrix operations to obtain transformable source patches with richer geometric information at high speed. We demonstrate the effectiveness and superiority of our method on a variety of scenes through extensive comparisons with state-of-the-art algorithms.