Xinyu Wang
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Research Interests
Xinyu Wang has a total of 31 co-authors. Here are the top co-authors by number of shared publications:
- Fangneng Zhan
Harvard University Cambridge Massachusetts USA
2 shared publications - Hongyu Liu
Hong Kong University of Science and Technology, Hong Kong, Hong Kong
2 shared publications - Yucheng Wang
Hong Kong University of Science and Technology, Hong Kong, Hong Kong
2 shared publications - Yue Ma
Hong Kong University of Science and Technology, Hong Kong, Hong Kong
2 shared publications - Yue Ma
Hong Kong University of Science and Technology, HongKong, Hong Kong
2 shared publications - Chang Zou
Shanghai Jiao Tong University, Shanghai, China and University of Electronic Science and Technology of China, Chengdu, China
1 shared publication
Latest Publications
- Open Access
- 9 November 2026
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Graphics
- Vol. 45 (6) , 1-16
LiveLight: Real-time Streaming Video Relighting with Interactive Control
We present LiveLight, the first diffusion-based framework for real-time streaming video relighting with interactive 3D lighting control. Achieving this is non-trivial, as it requires overcoming three critical challenges: effectively injecting dynamic 3D lighting into a diffusion model, maintaining high-fidelity generation under an extremely low NFE (Number of Function Evaluations) budget for real-time speed, and facilitating continuous streaming for interactive control. To address these pain points, we propose three key designs. First, for accurate lighting injection, we propose a lightweight adapter that feeds Multi-Plane Light Irradiance (MPLI) conditions—depth-aware irradiance maps encoding 3D lighting geometry—directly into the diffusion backbone. Second, to prevent rendering quality degradation at low NFEs towards real-time distillation, we introduce a geometry-guided feedback branch. This training-time constraint leverages a frozen geometry estimator to enforce depth- and normal-consistent relighting, ensuring geometrically plausible shading without adding inference overhead. Finally, to enable streaming interaction, we develop a progressive rolling-window strategy that maintains a denoising ladder of latent chunks at varying noise levels. By propagating intermediate states, this strategy guarantees temporal coherence and supports arbitrarily long video relighting with per-frame reference refresh. Extensive experiments on real-world and synthetic benchmarks demonstrate that LiveLight achieves state-of-the-art relighting quality while running at real-time speed, significantly outperforming offline baselines in temporal stability, lighting controllability, and user preference. To foster real-time interactive relighting research, we will publicly release our models, training data, and synthetic data generator on www.dummy.url.
- 9 November 2026
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Graphics
- Vol. 45 (6) , 1-16
Open AccessLiveLight: Real-time Streaming Video Relighting with Interactive Control
We present LiveLight, the first diffusion-based framework for real-time streaming video relighting with interactive 3D lighting control. Achieving this is non-trivial, as it requires overcoming three critical challenges: effectively injecting dynamic 3D lighting into a diffusion model, maintaining high-fidelity generation under an extremely low NFE (Number of Function Evaluations) budget for real-time speed, and facilitating continuous streaming for interactive control. To address these pain points, we propose three key designs. First, for accurate lighting injection, we propose a lightweight adapter that feeds Multi-Plane Light Irradiance (MPLI) conditions—depth-aware irradiance maps encoding 3D lighting geometry—directly into the diffusion backbone. Second, to prevent rendering quality degradation at low NFEs towards real-time distillation, we introduce a geometry-guided feedback branch. This training-time constraint leverages a frozen geometry estimator to enforce depth- and normal-consistent relighting, ensuring geometrically plausible shading without adding inference overhead. Finally, to enable streaming interaction, we develop a progressive rolling-window strategy that maintains a denoising ladder of latent chunks at varying noise levels. By propagating intermediate states, this strategy guarantees temporal coherence and supports arbitrarily long video relighting with per-frame reference refresh. Extensive experiments on real-world and synthetic benchmarks demonstrate that LiveLight achieves state-of-the-art relighting quality while running at real-time speed, significantly outperforming offline baselines in temporal stability, lighting controllability, and user preference. To foster real-time interactive relighting research, we will publicly release our models, training data, and synthetic data generator on www.dummy.url.
- Open Access
- 9 November 2026
- Published by Association for Computing Machinery (ACM)
- p. 10730-10739
SubjectAnchor: Subject-Aware Memory-to-Video for Multi-Shot Storytelling
- 9 November 2026
- Published by Association for Computing Machinery (ACM)
- p. 10730-10739
Open AccessSubjectAnchor: Subject-Aware Memory-to-Video for Multi-Shot Storytelling
- 13 September 2026
- Published by Springer Nature
- p. 631-649
LiveEdit: Towards Real-Time Diffusion-Based Streaming Video Editing
- 13 September 2026
- Published by Springer Nature
- p. 631-649
LiveEdit: Towards Real-Time Diffusion-Based Streaming Video Editing
- Open Access
- 19 July 2026
- Published by Association for Computing Machinery (ACM)
- p. 1-11
EasyVFX: Frequency-Driven Decoupling for Resource-Efficient VFX Generation
- 19 July 2026
- Published by Association for Computing Machinery (ACM)
- p. 1-11
Open AccessEasyVFX: Frequency-Driven Decoupling for Resource-Efficient VFX Generation
- Open Access
- 10 November 2025
- Published by Association for Computing Machinery (ACM)
- p. 5340-5345
Can Large Vision-Language Models Understand Multimodal Sarcasm?
- 10 November 2025
- Published by Association for Computing Machinery (ACM)
- p. 5340-5345
Open AccessCan Large Vision-Language Models Understand Multimodal Sarcasm?
- Open Access
- 27 October 2025
- Published by Association for Computing Machinery (ACM)
- p. 10181-10189
Compute Only 16 Tokens in One Timestep: Accelerating Diffusion Transformers with Cluster-Driven Feature Caching
- 27 October 2025
- Published by Association for Computing Machinery (ACM)
- p. 10181-10189
Open AccessCompute Only 16 Tokens in One Timestep: Accelerating Diffusion Transformers with Cluster-Driven Feature Caching