JYJiali Yao

Jiali Yao

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article

Previous Organizations / Institutes

The organizations / institutes from the scholar's other articles in the past ten years


Jiali Yao has a total of 12 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Open Access

    High-Fidelity Image Inpainting with Multimodal Guided GAN Inversion

    by Libo Zhang, Yongsheng Yu, Jiali Yao, Heng Fan

    Generative Adversarial Network (GAN) inversion have demonstrated excellent performance in image inpainting that aims to restore lost or damaged image texture using its unmasked content. Previous GAN inversion-based methods usually utilize well-trained GAN models as effective priors to generate the realistic regions for missing holes. Despite excellence, they ignore a hard constraint that the unmasked regions in the input and the output should be the same, resulting in a gap between GAN inversion and image inpainting and thus degrading the performance. Besides, existing GAN inversion approaches often consider a single modality of the input image, neglecting other auxiliary cues in images for improvements. Addressing these problems, we propose a novel GAN inversion approach, dubbed MMInvertFill, for image inpainting. MMInvertFill contains primarily a multimodal guided encoder with a pre-modulation and a GAN generator with F&W+ latent space. Specifically, the multimodal encoder aims to enhance the multi-scale structures with additional semantic segmentation edge texture modalities through a gated mask-aware attention module. Afterwards, a pre-modulation is presented to encode these structures into style vectors. To mitigate issues of conspicuous color discrepancy and semantic inconsistency, we introduce the F&W+ latent space to bridge the gap between GAN inversion and image inpainting. Furthermore, in order to reconstruct faithful and photorealistic images, we devise a simple yet effective Soft-update Mean Latent module to capture more diversified in-domain patterns for generating high-fidelity textures for massive corruptions. In our extensive experiments on six challenging datasets, we show that our MMInvertFill qualitatively and quantitatively outperforms other state-of-the-arts and it supports the completion of out-of-domain images effectively.


  • A Digital Simulation and Re-Editing Method for Clothing Patterns Based on Deep Learning and Somatosensory Interaction

    by Haiyan Sun, Jiali Yao, Haoyu Zhang, Zhijun Li, Xingquan Cai

    To address the issues in clothing pattern style migration, this paper proposes a digital simulation and re-editing method for clothing patterns based on deep learning and somatosensory interaction. First, the proposed method encodes the black-and-white line drawing image, generating random noise images through a diffusion process, introducing color information for synthesis, and using a decoder to reconstruct a colored image. Afterwards, an improved VGG19 model is used to reconstruct content features and perform linear color transformation on style images, enabling pattern style migration through the construction of a Gram matrix and resulting in colored clothing texture patterns. Finally, a KinectV2 is utilized for fabric simulation, overlaying colorful clothing texture patterns to achieve 3D virtual dressing. The experimental results show that the proposed method improves the structural similarity index measure (SSIM) by 9–11% and the peak signal-to-noise ratio (PSNR) by 3–8% when compared to existing algorithms. The experiments provide evidence that the proposed method effectively mitigates color overflow, delivers precise image coloring, and accomplishes realistic restoration of clothing texture. Furthermore, the method offers an improved garment fit to fulfill the user’s interaction requirements.


  • An Extended Labanotation Generation Method Based on 3D Human Pose Estimation for Intangible Cultural Heritage Dance Videos

    by Xingquan Cai, Rui Lu, Pengyan Cheng, Jiali Yao, Yan Hu

    To address the issues of low accuracy in existing 3D human pose estimation (HPE) methods and the limited level of details in Labanotation, we propose an extended Labanotation generation method for intangible cultural heritage dance videos based on 3D HPE. First, a 2D human pose sequence of the performer is inputted along with spatial location embeddings, where multiple spatial transformer modules are employed to extract spatial features of human joints and generate cross-joint multiple hypotheses. Afterward, temporal features are extracted by a self-attentive module and the correlation between different hypotheses is learned using bilinear pooling. Finally, the 3D joint coordinates of the performer are predicted, which are matched with the corresponding extended Labanotation symbols using the Laban template matching method to generate extended Labanotation. Experimental results show that, compared with VideoPose and CrossFormer algorithms, the Mean Per Joint Position Error (MPJPE) of the proposed method is reduced by 3.7[ ]mm and 0.6[ ]mm, respectively on Human3.6M dataset, and the generated extended Labanotation can better describe the movement details compared with the basic Labanotation.