Yehao Li
Yehao Li has a total of 39 co-authors. Here are the top co-authors by number of shared publications:
- Tao Mei
HiDream.ai Inc., Beijing, China
20 shared publications - Ting Yao
HiDream.ai Inc., Beijing, China
20 shared publications - Yingwei Pan
HiDream.ai Inc., Beijing, China
20 shared publications - Jingwen Chen
HiDream.ai Inc., Beijing, China
4 shared publications - Qi Cai
HiDream.ai Inc., Beijing, China
4 shared publications - Chong-Wah Ngo
Singapore Management University, Singapore
3 shared publications
Latest Publications
- Open Access
- 9 November 2026
- Published by Association for Computing Machinery (ACM)
- p. 7652-7660
Beyond the Logits: Mitigating Hallucinations in Multimodal Large Language Models with Layer-Adaptive Visual Guidance
- 9 November 2026
- Published by Association for Computing Machinery (ACM)
- p. 7652-7660
Open AccessBeyond the Logits: Mitigating Hallucinations in Multimodal Large Language Models with Layer-Adaptive Visual Guidance
- Open Access
- 9 November 2026
- Published by Association for Computing Machinery (ACM)
- p. 506-515
LayerEdit: Towards End-to-End Context-Aware Layer Editing
- 9 November 2026
- Published by Association for Computing Machinery (ACM)
- p. 506-515
Open AccessLayerEdit: Towards End-to-End Context-Aware Layer Editing
- Open Access
- 9 November 2026
- Published by Association for Computing Machinery (ACM)
- p. 17137-17140
HiDream-O1-Image: An Open-Source Pixel-Level Unified Transformer for General Image Generation
- 9 November 2026
- Published by Association for Computing Machinery (ACM)
- p. 17137-17140
Open AccessHiDream-O1-Image: An Open-Source Pixel-Level Unified Transformer for General Image Generation
- Open Access
- 9 November 2026
- Published by Association for Computing Machinery (ACM)
- p. 54-62
MM-SkillBench & MM-SkillRouter: Benchmarking and Routing Multimodal Skills for Agents
- 9 November 2026
- Published by Association for Computing Machinery (ACM)
- p. 54-62
Open AccessMM-SkillBench & MM-SkillRouter: Benchmarking and Routing Multimodal Skills for Agents
- 1 January 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Multimedia
- Vol. 28, 11957-11969
Seeding Success: Unlocking the Golden Seed for Any Prompt in Text-to-Image Diffusion Models
- 1 January 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Multimedia
- Vol. 28, 11957-11969
Seeding Success: Unlocking the Golden Seed for Any Prompt in Text-to-Image Diffusion Models
- Open Access
- 27 October 2025
- Published by Association for Computing Machinery (ACM)
- p. 13636-13639
HiDream-I1: An Open-Source High-Efficient Image Generative Foundation Model
- 27 October 2025
- Published by Association for Computing Machinery (ACM)
- p. 13636-13639
Open AccessHiDream-I1: An Open-Source High-Efficient Image Generative Foundation Model
- Open Access
- 27 October 2025
- Published by Association for Computing Machinery (ACM)
- p. 13737-13742
Identity-Preserving Video Generation Challenge
- 27 October 2025
- Published by Association for Computing Machinery (ACM)
- p. 13737-13742
Open AccessIdentity-Preserving Video Generation Challenge
- 27 January 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Multimedia
- Vol. 27 (15209210) , 3755-3765
Stream-ViT: Learning Streamlined Convolutions in Vision Transformer
Recently Vision Transformer (ViT) and Convolution Neural Network (CNN) start to emerge as a hybrid deep architecture with better model capacity, generalization, and latency trade-off. Most of these hybrid architectures often directly stack self-attention module with static convolution or fuse their outputs through two pathways within each block. Instead, we present a new Transformer architecture (namely Stream-ViT) to novelly integrate ViT with streamlined convolutions, i.e., a series of high-to-low resolution convolutions. The kernels of each convolution are dynamically learnt on a basis of current input features plus pre-learnt kernels throughout the whole network. The new architecture incorporates a critical pathway to streamline kernel generation that triggers the interactions between dynamically learnt convolutions across different layers. Moreover, the introduction of a layer-wise streamlined convolution is functionally equivalent to a squeezed version of multi-branch convolution structure, thereby improving the capacity of self-attention module with enlarged cardinality in a cost-efficient manner. We validate the superiority of Stream-ViT over multiple vision tasks, and its performances surpass state-of-the-art ViT and CNN backbones with comparable FLOPs.
- 27 January 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Multimedia
- Vol. 27 (15209210) , 3755-3765
Stream-ViT: Learning Streamlined Convolutions in Vision Transformer
Recently Vision Transformer (ViT) and Convolution Neural Network (CNN) start to emerge as a hybrid deep architecture with better model capacity, generalization, and latency trade-off. Most of these hybrid architectures often directly stack self-attention module with static convolution or fuse their outputs through two pathways within each block. Instead, we present a new Transformer architecture (namely Stream-ViT) to novelly integrate ViT with streamlined convolutions, i.e., a series of high-to-low resolution convolutions. The kernels of each convolution are dynamically learnt on a basis of current input features plus pre-learnt kernels throughout the whole network. The new architecture incorporates a critical pathway to streamline kernel generation that triggers the interactions between dynamically learnt convolutions across different layers. Moreover, the introduction of a layer-wise streamlined convolution is functionally equivalent to a squeezed version of multi-branch convolution structure, thereby improving the capacity of self-attention module with enlarged cardinality in a cost-efficient manner. We validate the superiority of Stream-ViT over multiple vision tasks, and its performances surpass state-of-the-art ViT and CNN backbones with comparable FLOPs.
- 30 August 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Circuits and Systems for Video Technology
- Vol. 35 (1) , 91-102
Exploring Vision-Language Foundation Model for Novel Object Captioning
It is always well believed that pre-trained vision-language foundation models (e.g., CLIP) would substantially facilitate vision-language tasks. Nevertheless, there has been less evidence in support of the idea on describing novel objects in images. In this paper, we propose the Novel Object Transformer with CLIP (NOTC), a Transformer-based model that innovatively exploits the powerful vision-language representation ability of CLIP to enhance novel object captioning model’s training and sentence decoding processes. Technically, given the primary bag-of-objects extracted by Faster R-CNN, NOTC first capitalize on an object distiller module to emphasize the most salient objects and infer the missing novel ones. The refined object words are additionally fed into the object-centric word predictor to generate sentence word-by-word. During training, we design a CLIP-based self-critical sequence training paradigm to select visually-grounded sampled sentence with higher CLIP score reward, which enables a joint training process of captioning model over out-domain training images with novel objects. Moreover, at inference, a new CLIP beam search algorithm is devised to enforce the existence of novel objects and encourage the partial word sequences with higher CLIP scores, thereby decoding both visually-grounded and comprehensive sentences. Extensive experiments are conducted on held-out COCO and nocaps datasets, and competitive performances are reported when compared to state-of-the-art approaches.
- 30 August 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Circuits and Systems for Video Technology
- Vol. 35 (1) , 91-102
Exploring Vision-Language Foundation Model for Novel Object Captioning
It is always well believed that pre-trained vision-language foundation models (e.g., CLIP) would substantially facilitate vision-language tasks. Nevertheless, there has been less evidence in support of the idea on describing novel objects in images. In this paper, we propose the Novel Object Transformer with CLIP (NOTC), a Transformer-based model that innovatively exploits the powerful vision-language representation ability of CLIP to enhance novel object captioning model’s training and sentence decoding processes. Technically, given the primary bag-of-objects extracted by Faster R-CNN, NOTC first capitalize on an object distiller module to emphasize the most salient objects and infer the missing novel ones. The refined object words are additionally fed into the object-centric word predictor to generate sentence word-by-word. During training, we design a CLIP-based self-critical sequence training paradigm to select visually-grounded sampled sentence with higher CLIP score reward, which enables a joint training process of captioning model over out-domain training images with novel objects. Moreover, at inference, a new CLIP beam search algorithm is devised to enforce the existence of novel objects and encourage the partial word sequences with higher CLIP scores, thereby decoding both visually-grounded and comprehensive sentences. Extensive experiments are conducted on held-out COCO and nocaps datasets, and competitive performances are reported when compared to state-of-the-art approaches.
- 19 March 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Pattern Analysis and Machine Intelligence
- Vol. 46 (9) , 6431-6442
HIRI-ViT: Scaling Vision Transformer With High Resolution Inputs
The hybrid deep models of Vision Transformer (ViT) and Convolution Neural Network (CNN) have emerged as a powerful class of backbones for vision tasks. Scaling up the input resolution of such hybrid backbones naturally strengthes model capacity, but inevitably suffers from heavy computational cost that scales quadratically. Instead, we present a new hybrid backbone with HIgh-Resolution Inputs (namely HIRI-ViT), that upgrades prevalent four-stage ViT to five-stage ViT tailored for high-resolution inputs. HIRI-ViT is built upon the seminal idea of decomposing the typical CNN operations into two parallel CNN branches in a cost-efficient manner. One high-resolution branch directly takes primary high-resolution features as inputs, but uses less convolution operations. The other low-resolution branch first performs down-sampling and then utilizes more convolution operations over such low-resolution features. Experiments on both recognition task (ImageNet-1K dataset) and dense prediction tasks (COCO and ADE20 K datasets) demonstrate the superiority of HIRI-ViT. More remarkably, under comparable computational cost ($\sim$ 5.0 GFLOPs), HIRI-ViT achieves to-date the best published Top-1 accuracy of 84.3% on ImageNet with 448×448 inputs, which absolutely improves 83.4% of iFormer-S by 0.9% with 224×224 inputs.- 19 March 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Pattern Analysis and Machine Intelligence
- Vol. 46 (9) , 6431-6442
HIRI-ViT: Scaling Vision Transformer With High Resolution Inputs
The hybrid deep models of Vision Transformer (ViT) and Convolution Neural Network (CNN) have emerged as a powerful class of backbones for vision tasks. Scaling up the input resolution of such hybrid backbones naturally strengthes model capacity, but inevitably suffers from heavy computational cost that scales quadratically. Instead, we present a new hybrid backbone with HIgh-Resolution Inputs (namely HIRI-ViT), that upgrades prevalent four-stage ViT to five-stage ViT tailored for high-resolution inputs. HIRI-ViT is built upon the seminal idea of decomposing the typical CNN operations into two parallel CNN branches in a cost-efficient manner. One high-resolution branch directly takes primary high-resolution features as inputs, but uses less convolution operations. The other low-resolution branch first performs down-sampling and then utilizes more convolution operations over such low-resolution features. Experiments on both recognition task (ImageNet-1K dataset) and dense prediction tasks (COCO and ADE20 K datasets) demonstrate the superiority of HIRI-ViT. More remarkably, under comparable computational cost ($\sim$ 5.0 GFLOPs), HIRI-ViT achieves to-date the best published Top-1 accuracy of 84.3% on ImageNet with 448×448 inputs, which absolutely improves 83.4% of iFormer-S by 0.9% with 224×224 inputs.