YLYinan Li

Yinan Li

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article


Yinan Li has a total of 11 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Open Access

    Design Element Aware Poster Layout Generation

    by Yinan Li, Yin Bai, Jia Cheng, Jun Lei

    Despite the recent significant advancements in poster layout generation, existing works are mainly unaware of the given design elements (i.e., text, logo, and underlay), which leads to undesirable layouts or visual artifacts. The visual artifacts we refer to include (1) improper sizes, e.g., placing a short piece of text into a large textbox or long texts into small text boxes, and (2) image distortion, e.g., the stretched logo in Fig. 1. To advance research in this field, we propose a new design-element aware poster layout generation task, which require the generated layouts to not only have harmonic relationships but also fit well with the design elements. To address this task, we propose Design Element aware Transformer (DET), an encoder-decoder based transformer network, to generate reasonable layouts that fit not only the background images but also the design elements. The encoder extracts a fine-grained multi-scale representation from the background image and its saliency map. The decoder receives the background features and produces layouts conditioned on the content and desired sizes of the design elements. Adopting the multi-scale representation and the deformable attention in both the encoder and decoder enables our method to accurately understand/generate the spatial relationships between the background objects and design elements. We adapted three public poster layout generation datasets to fit our task and conducted experiments on them. In the meantime, we propose a new evaluation metric called AspDiff to measure whether the generated layout matches the given design elements. Quantitative and qualitative evaluation on three datasets demonstrates that DET yields superior results compared to other layout generation methods. Our code and datasets will be released.


  • Open Access

    Semi-Supervised Panoptic Narrative Grounding

    by Danni Yang, Jiayi Ji, Xiaoshuai Sun, Haowei Wang, Yinan Li, Yiwei Ma, Rongrong Ji

    Despite considerable progress, the advancement of Panoptic Narrative Grounding (PNG) remains hindered by costly annotations. In this paper, we introduce a novel Semi-Supervised Panoptic Narrative Grounding (SS-PNG) learning scheme, capitalizing on a smaller set of labeled image-text pairs and a larger set of unlabeled pairs to achieve competitive performance. Unlike visual segmentation tasks, PNG involves one pixel belonging to multiple open-ended nouns. As a result, existing multi-class based semi-supervised segmentation frameworks cannot be directly applied to this task. To address this challenge, we first develop a novel SS-PNG Network (SS-PNG-NW) tailored to the SS-PNG setting. We thoroughly investigate strategies such as Burn-In and data augmentation to determine the optimal generic configuration for the SS-PNG-NW. Additionally, to tackle the issue of imbalanced pseudo-label quality, we propose a Quality-Based Loss Adjustment (QLA) approach to adjust the semi-supervised objective, resulting in an enhanced SS-PNG-NW+. Employing our proposed QLA, we improve BCE Loss and Dice loss at pixel and mask levels, respectively. We conduct extensive experiments on PNG datasets, with our SS-PNG-NW+ demonstrating promising results comparable to fully-supervised models across all data ratios. Remarkably, our SS-PNG-NW+ outperforms fully-supervised models with only 30% and 50% supervision data, exceeding their performance by 0.8% and 1.1% respectively. This highlights the effectiveness of our proposed SS-PNG-NW+ in overcoming the challenges posed by limited annotations and enhancing the applicability of PNG tasks. The source code is available at https://github.com/nini0919/SSPNG.


  • Open Access

    Semantic-Guided Selective Representation for Image Captioning

    by Yinan Li, Yiwei Ma, Yiyi Zhou, Xiao Yu

    Grid-based features have been proven to be as effective as region-based features in multi-modal tasks such as visual question answering. However, its application to image captioning encounters two main issues, namely, noisy features and fragmented semantics. In this paper, we propose a novel feature selection scheme, with a Relation-Aware Selection (RAS) and a Fine-grained Semantic Guidance (FSG) learning strategy. Based on the grid-wise interactions, RAS can enhance the salient visual regions and channels, and suppress the less important ones. In addition, this selection process is guided by FSG, which uses fine-grained semantic knowledge to supervise the selection process. Experimental results on the MS COCO show the proposed RAS-FSG scheme achieves state-of-the-art performance on both the off-line and on-line testing, i.e ., 134.3 CIDEr for the off-line testing and 135.4 for the on-line testing of MSCOCO. Extensive ablation studies and visualizations also validate the effectiveness of our scheme.