Shuai Yang
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Research Subjects
Shuai Yang has a total of 84 co-authors. Here are the top co-authors by number of shared publications:
- Jiaying Liu
Peking University
24 shared publications - Chen Change Loy
S-Lab, Nanyang Technological University, Singapore, Singapore
10 shared publications - Ziwei Liu
Nanyang technological university, Singapore, Singapore
9 shared publications - Zongming Guo
Peking University, Beijing, China
7 shared publications - Wenjing Wang
Peking University, Beijing, China
5 shared publications Zongming Guo
Institute of Computer Science and Technology, Peking University, Beijing, China
5 shared publications
Latest Publications
- 23 March 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Multimedia
- Vol. PP  (15209210) , 1-12
DiffTile: High-Resolution Periodic Pattern Generation Via Splice, Focus and Enhance
High-resolution periodic patterns are widely used in the printing and dyeing industry. However, the manual design of such periodic patterns has considerable time and manpower costs. Recent advances in powerful image generation models, which are represented by latent diffusion models (LDMs), provide an automated solution. However, owing to the inherent randomness of the generation process and the computational complexity of LDMs, generating strictly periodic and highresolution patterns is still challenging. In this paper, a novel text-driven framework for high-resolution periodic pattern generation is proposed. For periodic pattern generation, we propose a training-free Splice-and-Focus mechanism, which constrains latent features and modifies the attention mechanism to generate natural and strictly periodic patterns. On this basis, we provide two types of generation modes: direct periodic pattern generation and periodization of arbitrary existing patterns. For high-resolution pattern generation, we propose a ReLife pipeline that integrates superresolution and image detail enhancement to improve pattern resolution while removing generative artifacts and distortions. Furthermore, for highly nonstationary patterns, we propose a compositional generation framework to design complicated patterns interactively. The experimental results show that our framework can generate high-quality patterns that meet the strict standards of the printing and dyeing industry.
- 23 March 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Multimedia
- Vol. PP  (15209210) , 1-12
DiffTile: High-Resolution Periodic Pattern Generation Via Splice, Focus and Enhance
High-resolution periodic patterns are widely used in the printing and dyeing industry. However, the manual design of such periodic patterns has considerable time and manpower costs. Recent advances in powerful image generation models, which are represented by latent diffusion models (LDMs), provide an automated solution. However, owing to the inherent randomness of the generation process and the computational complexity of LDMs, generating strictly periodic and highresolution patterns is still challenging. In this paper, a novel text-driven framework for high-resolution periodic pattern generation is proposed. For periodic pattern generation, we propose a training-free Splice-and-Focus mechanism, which constrains latent features and modifies the attention mechanism to generate natural and strictly periodic patterns. On this basis, we provide two types of generation modes: direct periodic pattern generation and periodization of arbitrary existing patterns. For high-resolution pattern generation, we propose a ReLife pipeline that integrates superresolution and image detail enhancement to improve pattern resolution while removing generative artifacts and distortions. Furthermore, for highly nonstationary patterns, we propose a compositional generation framework to design complicated patterns interactively. The experimental results show that our framework can generate high-quality patterns that meet the strict standards of the printing and dyeing industry.
- Top Cited Papers
- 7 January 2026
- Published by Springer Nature in International Journal of Computer Vision
- Vol. 134  (1) , 38
Self-Supervised Skeleton-Based Action Representation Learning: A Benchmark and Beyond
- 7 January 2026
- Published by Springer Nature in International Journal of Computer Vision
- Vol. 134  (1) , 38
Top Cited PapersSelf-Supervised Skeleton-Based Action Representation Learning: A Benchmark and Beyond
- Open Access
Trainable Log-linear Sparse Attention for Efficient Diffusion Transformers
Diffusion Transformers (DiTs) set the state of the art in visual generation, yet their quadratic self-attention cost fundamentally limits scaling to long token sequences. Recent Top-K sparse attention approaches reduce the computation of DiTs by compressing tokens into block-wise representation and selecting a small set of relevant key blocks, but still suffer from (i) quadratic selection cost on compressed tokens and (ii) increasing K required to maintain model quality as sequences grow. We identify that their inefficiency is due to the single-level design, as a single coarse level is insufficient to represent the global structure. In this paper, we introduce Log-linear Sparse Attention (LLSA), a trainable sparse attention mechanism for extremely long token sequences that reduces both selection and attention costs from quadratic to log-linear complexity by utilizing a hierarchical structure. LLSA performs hierarchical Top-K selection, progressively adopting sparse Top-K selection with the indices found at the previous level, and introduces a Hierarchical KV Enrichment mechanism that preserves global context while using fewer tokens of different granularity during attention computation. To support efficient training, we develop a high-performance GPU implementation that uses only sparse indices for both the forward and backward passes, eliminating the need for dense attention masks. We evaluate LLSA on high-resolution pixel-space image generation without using patchification and VAE encoding. LLSA accelerates attention inference by 28.27x and DiT training by 6.09x on 256x256 pixel token sequences, while maintaining generation quality. The results demonstrate that LLSA offers a promising direction for training long-sequence DiTs efficiently. Code is available at: https://github.com/SingleZombie/LLSA
Open AccessTrainable Log-linear Sparse Attention for Efficient Diffusion Transformers
Diffusion Transformers (DiTs) set the state of the art in visual generation, yet their quadratic self-attention cost fundamentally limits scaling to long token sequences. Recent Top-K sparse attention approaches reduce the computation of DiTs by compressing tokens into block-wise representation and selecting a small set of relevant key blocks, but still suffer from (i) quadratic selection cost on compressed tokens and (ii) increasing K required to maintain model quality as sequences grow. We identify that their inefficiency is due to the single-level design, as a single coarse level is insufficient to represent the global structure. In this paper, we introduce Log-linear Sparse Attention (LLSA), a trainable sparse attention mechanism for extremely long token sequences that reduces both selection and attention costs from quadratic to log-linear complexity by utilizing a hierarchical structure. LLSA performs hierarchical Top-K selection, progressively adopting sparse Top-K selection with the indices found at the previous level, and introduces a Hierarchical KV Enrichment mechanism that preserves global context while using fewer tokens of different granularity during attention computation. To support efficient training, we develop a high-performance GPU implementation that uses only sparse indices for both the forward and backward passes, eliminating the need for dense attention masks. We evaluate LLSA on high-resolution pixel-space image generation without using patchification and VAE encoding. LLSA accelerates attention inference by 28.27x and DiT training by 6.09x on 256x256 pixel token sequences, while maintaining generation quality. The results demonstrate that LLSA offers a promising direction for training long-sequence DiTs efficiently. Code is available at: https://github.com/SingleZombie/LLSA
- Open Access
Zero-Shot Video Translation and Editing with Frame Spatial-Temporal Correspondence
The remarkable success in text-to-image diffusion models has motivated extensive investigation of their potential for video applications. Zero-shot techniques aim to adapt image diffusion models for videos without requiring further model training. Recent methods largely emphasize integrating inter-frame correspondence into attention mechanisms. However, the soft constraint applied to identify the valid features to attend is insufficient, which could lead to temporal inconsistency. In this paper, we present FRESCO, which integrates intra-frame correspondence with inter-frame correspondence to formulate a more robust spatial-temporal constraint. This enhancement ensures a consistent transformation of semantically similar content between frames. Our method goes beyond attention guidance to explicitly optimize features, achieving high spatial-temporal consistency with the input video, significantly enhancing the visual coherence of manipulated videos. We verify FRESCO adaptations on two zero-shot tasks of video-to-video translation and text-guided video editing. Comprehensive experiments demonstrate the effectiveness of our framework in generating high-quality, coherent videos, highlighting a significant advance over current zero-shot methods.
Open AccessZero-Shot Video Translation and Editing with Frame Spatial-Temporal Correspondence
The remarkable success in text-to-image diffusion models has motivated extensive investigation of their potential for video applications. Zero-shot techniques aim to adapt image diffusion models for videos without requiring further model training. Recent methods largely emphasize integrating inter-frame correspondence into attention mechanisms. However, the soft constraint applied to identify the valid features to attend is insufficient, which could lead to temporal inconsistency. In this paper, we present FRESCO, which integrates intra-frame correspondence with inter-frame correspondence to formulate a more robust spatial-temporal constraint. This enhancement ensures a consistent transformation of semantically similar content between frames. Our method goes beyond attention guidance to explicitly optimize features, achieving high spatial-temporal consistency with the input video, significantly enhancing the visual coherence of manipulated videos. We verify FRESCO adaptations on two zero-shot tasks of video-to-video translation and text-guided video editing. Comprehensive experiments demonstrate the effectiveness of our framework in generating high-quality, coherent videos, highlighting a significant advance over current zero-shot methods.
- Open Access
FlowPortal: Residual-Corrected Flow for Training-Free Video Relighting and Background Replacement
Video relighting with background replacement is a challenging task critical for applications in film production and creative media. Existing methods struggle to balance temporal consistency, spatial fidelity, and illumination naturalness. To address these issues, we introduce FlowPortal, a novel training-free flow-based video relighting framework. Our core innovation is a Residual-Corrected Flow mechanism that transforms a standard flow-based model into an editing model, guaranteeing perfect reconstruction when input conditions are identical and enabling faithful relighting when they differ, resulting in high structural consistency. This is further enhanced by a Decoupled Condition Design for precise lighting control and a High-Frequency Transfer mechanism for detail preservation. Additionally, a masking strategy isolates foreground relighting from background pure generation process. Experiments demonstrate that FlowPortal achieves superior performance in temporal coherence, structural preservation, and lighting realism, while maintaining high efficiency. Project Page: https://gaowenshuo.github.io/FlowPortalProject/.
Open AccessFlowPortal: Residual-Corrected Flow for Training-Free Video Relighting and Background Replacement
Video relighting with background replacement is a challenging task critical for applications in film production and creative media. Existing methods struggle to balance temporal consistency, spatial fidelity, and illumination naturalness. To address these issues, we introduce FlowPortal, a novel training-free flow-based video relighting framework. Our core innovation is a Residual-Corrected Flow mechanism that transforms a standard flow-based model into an editing model, guaranteeing perfect reconstruction when input conditions are identical and enabling faithful relighting when they differ, resulting in high structural consistency. This is further enhanced by a Decoupled Condition Design for precise lighting control and a High-Frequency Transfer mechanism for detail preservation. Additionally, a masking strategy isolates foreground relighting from background pure generation process. Experiments demonstrate that FlowPortal achieves superior performance in temporal coherence, structural preservation, and lighting realism, while maintaining high efficiency. Project Page: https://gaowenshuo.github.io/FlowPortalProject/.
- 14 November 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Pattern Analysis and Machine Intelligence
- Vol. 48  (3) , 3002-3017
LN3Diff++: Scalable L atent N eural Fields D iffusion for Speedy 3D Generation
The field of neural rendering has seen remarkable progress, driven by advancements in generative models and differentiable rendering techniques. While 2D diffusion has achieved notable success, the development of a unified 3D diffusion pipeline remains an open challenge. This paper presents a novel framework, LN3Diff++, designed to bridge this gap and facilitate fast, high-quality, and versatile conditional 3D generation. Our method leverages a 3D-aware architecture and a variational autoencoder (VAE) to encode input image(s) into a structured, compact 3D latent space. The latent representation is then decoded by a transformer-based decoder into a high-capacity 3D neural field. By training a diffusion model on this 3D-aware latent space, our method achieves superior performance for category-specific 3D generation on ShapeNet and FFHQ, as well as category-free image/text-conditioned 3D generation over Objaverse. Moreover, it surpasses existing 3D diffusion methods in inference speed, requiring no per-instance optimization.
- 14 November 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Pattern Analysis and Machine Intelligence
- Vol. 48  (3) , 3002-3017
LN3Diff++: Scalable L atent N eural Fields D iffusion for Speedy 3D Generation
The field of neural rendering has seen remarkable progress, driven by advancements in generative models and differentiable rendering techniques. While 2D diffusion has achieved notable success, the development of a unified 3D diffusion pipeline remains an open challenge. This paper presents a novel framework, LN3Diff++, designed to bridge this gap and facilitate fast, high-quality, and versatile conditional 3D generation. Our method leverages a 3D-aware architecture and a variational autoencoder (VAE) to encode input image(s) into a structured, compact 3D latent space. The latent representation is then decoded by a transformer-based decoder into a high-capacity 3D neural field. By training a diffusion model on this 3D-aware latent space, our method achieves superior performance for category-specific 3D generation on ShapeNet and FFHQ, as well as category-free image/text-conditioned 3D generation over Objaverse. Moreover, it surpasses existing 3D diffusion methods in inference speed, requiring no per-instance optimization.
- Open Access
ANYPORTAL: Zero-Shot Consistent Video Background Replacement
Despite the rapid advancements in video generation technology, creating high-quality videos that precisely align with user intentions remains a significant challenge. Existing methods often fail to achieve fine-grained control over video details, limiting their practical applicability. We introduce ANYPORTAL, a novel zero-shot framework for video background replacement that leverages pre-trained diffusion models. Our framework collaboratively integrates the temporal prior of video diffusion models with the relighting capabilities of image diffusion models in a zero-shot setting. To address the critical challenge of foreground consistency, we propose a Refinement Projection Algorithm, which enables pixel-level detail manipulation to ensure precise foreground preservation. ANYPORTAL is training-free and overcomes the challenges of achieving foreground consistency and temporally coherent relighting. Experimental results demonstrate that ANYPORTAL achieves high-quality results on consumer-grade GPUs, offering a practical and efficient solution for video content creation and editing.
Open AccessANYPORTAL: Zero-Shot Consistent Video Background Replacement
Despite the rapid advancements in video generation technology, creating high-quality videos that precisely align with user intentions remains a significant challenge. Existing methods often fail to achieve fine-grained control over video details, limiting their practical applicability. We introduce ANYPORTAL, a novel zero-shot framework for video background replacement that leverages pre-trained diffusion models. Our framework collaboratively integrates the temporal prior of video diffusion models with the relighting capabilities of image diffusion models in a zero-shot setting. To address the critical challenge of foreground consistency, we propose a Refinement Projection Algorithm, which enables pixel-level detail manipulation to ensure precise foreground preservation. ANYPORTAL is training-free and overcomes the challenges of achieving foreground consistency and temporally coherent relighting. Experimental results demonstrate that ANYPORTAL achieves high-quality results on consumer-grade GPUs, offering a practical and efficient solution for video content creation and editing.
- 14 June 2025
- Published by Springer Nature in International Journal of Computer Vision
- p. 1-18
$E^{3}DGE$: Self-Supervised Geometry-Aware Encoder for Style-Based 3D GAN Inversion- 14 June 2025
- Published by Springer Nature in International Journal of Computer Vision
- p. 1-18
$E^{3}DGE$: Self-Supervised Geometry-Aware Encoder for Style-Based 3D GAN Inversion- 10 January 2025
- Published by Elsevier in Simulation Modelling Practice and Theory
- Vol. 140, 103075
Large-scale real-time evacuation modeling during urban floods: A coupled agent-based multi-model framework
- 10 January 2025
- Published by Elsevier in Simulation Modelling Practice and Theory
- Vol. 140, 103075
Large-scale real-time evacuation modeling during urban floods: A coupled agent-based multi-model framework
- Top Cited PapersOpen Access
- 17 December 2024
- Published by Emerald Publishing in APSIPA Transactions on Signal and Information Processing
- Vol. 13  (1) , 1-39
Intelligent Artistic Typography: A Comprehensive Review of Artistic Text Design and Generation
Artistic text generation aims to amplify the aesthetic qualities of text while maintaining readability. It can make the text more attractive and better convey its expression, thus enjoying a wide range of application scenarios such as social media display, consumer electronics, fashion, and graphic design. Artistic text generation includes artistic text stylization and semantic typography. Artistic text stylization concentrates on the text effect overlaid upon the text, such as shadows, outlines, colors, glows, and textures. By comparison, semantic typography focuses on the deformation of the characters to strengthen their visual representation by mimicking the semantic understanding within the text. This overview paper provides an introduction to both artistic text stylization and semantic typography, including the taxonomy, the key ideas of representative methods, and the applications in static and dynamic artistic text generation. Furthermore, the dataset and evaluation metrics are introduced, and the future directions of artistic text generation are discussed. A comprehensive list of artistic text generation models studied in this review is available at https://github.com/williamyang1991/Awesome-Artistic-Typography/.
- 17 December 2024
- Published by Emerald Publishing in APSIPA Transactions on Signal and Information Processing
- Vol. 13  (1) , 1-39
Top Cited PapersOpen AccessIntelligent Artistic Typography: A Comprehensive Review of Artistic Text Design and Generation
Artistic text generation aims to amplify the aesthetic qualities of text while maintaining readability. It can make the text more attractive and better convey its expression, thus enjoying a wide range of application scenarios such as social media display, consumer electronics, fashion, and graphic design. Artistic text generation includes artistic text stylization and semantic typography. Artistic text stylization concentrates on the text effect overlaid upon the text, such as shadows, outlines, colors, glows, and textures. By comparison, semantic typography focuses on the deformation of the characters to strengthen their visual representation by mimicking the semantic understanding within the text. This overview paper provides an introduction to both artistic text stylization and semantic typography, including the taxonomy, the key ideas of representative methods, and the applications in static and dynamic artistic text generation. Furthermore, the dataset and evaluation metrics are introduced, and the future directions of artistic text generation are discussed. A comprehensive list of artistic text generation models studied in this review is available at https://github.com/williamyang1991/Awesome-Artistic-Typography/.