Yufan Zhang
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Yufan Zhang has a total of 6 co-authors. Here are the top co-authors by number of shared publications:
- Jinwei Ye
George Mason University, Fairfax, USA
2 shared publications - Yu Guo
George Mason University
2 shared publications - Yu Ji
LightThought LLC, Fairfax, USA
2 shared publications - Ayo Ajiboye
George Mason University, Fairfax, USA
1 shared publication - Changxi Zheng
Columbia University, New York, USA
1 shared publication - Rundi Wu
Columbia University, New York, USA
1 shared publication
Latest Publications
- Open Access
- 3 July 2026
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Graphics
- Vol. 45 (4) , 1-17
Pixel Cube: Diffusion-based Portrait Video Relighting Through Realistic Lighting Reproduction
We present a diffusion-based method for relighting dynamic portrait videos with photorealism and temporal consistency. Our method is fueled by a hybrid training dataset that consists of real-captured and rendered dynamic portrait videos with diverse subject appearances, facial motions, head poses, and known lighting conditions. Specifically, we construct an LED-based lighting system for realistic lighting emulation and high-speed video relighting data acquisition. By leveraging the image priors embedded in pre-trained video diffusion models, and using per-frame high dynamic range (HDR) environment map as lighting control, we train a high-performance generative model for realistic and identity-preserving dynamic portrait video relighting. In addition to the environment map control, our model uses a synthesized background image to enable control on the camera's exposure level and color tone. Our model can produce temporally consistent relit portrait video that looks realistic and harmonious under a provided new environment and faithfully preserve the subject's expression and fine facial features, including skin tone, wrinkles, and facial hair. Our model generalizes well to unseen data, in terms of the subject appearance, motion, and lighting condition. We perform extensive experiments on relighting in-the-wild videos with various environment maps and demonstrate practical applications on portrait photography. Results show that our method achieves state-of-the-art performance in photorealism, lighting harmony, and temporal consistency. Our project page: https://yufanzhang82.github.io/PixelCube/.
- 3 July 2026
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Graphics
- Vol. 45 (4) , 1-17
Open AccessPixel Cube: Diffusion-based Portrait Video Relighting Through Realistic Lighting Reproduction
We present a diffusion-based method for relighting dynamic portrait videos with photorealism and temporal consistency. Our method is fueled by a hybrid training dataset that consists of real-captured and rendered dynamic portrait videos with diverse subject appearances, facial motions, head poses, and known lighting conditions. Specifically, we construct an LED-based lighting system for realistic lighting emulation and high-speed video relighting data acquisition. By leveraging the image priors embedded in pre-trained video diffusion models, and using per-frame high dynamic range (HDR) environment map as lighting control, we train a high-performance generative model for realistic and identity-preserving dynamic portrait video relighting. In addition to the environment map control, our model uses a synthesized background image to enable control on the camera's exposure level and color tone. Our model can produce temporally consistent relit portrait video that looks realistic and harmonious under a provided new environment and faithfully preserve the subject's expression and fine facial features, including skin tone, wrinkles, and facial hair. Our model generalizes well to unseen data, in terms of the subject appearance, motion, and lighting condition. We perform extensive experiments on relighting in-the-wild videos with various environment maps and demonstrate practical applications on portrait photography. Results show that our method achieves state-of-the-art performance in photorealism, lighting harmony, and temporal consistency. Our project page: https://yufanzhang82.github.io/PixelCube/.
- 29 April 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Pattern Analysis and Machine Intelligence
- Vol. PP (01628828) , 1-14
Snapshot 3D Gaussian Splatting for Miniature Scenes
We present a snapshot imaging technique for recovering 3D surrounding views of miniature scenes. Due to their intricacy, miniature scenes with objects sized in millimeters are difficult to reconstruct. Yet miniatures are common in life and their 3D digitalization is desirable. We design a catadioptric imaging system with a single camera and multiple pairs of planar mirrors for snapshot 3D reconstruction of miniature scenes from a dollhouse perspective. We first present an in-depth analysis on how to configure catadioptric imaging systems that use a pair of planar mirrors. We derive mirror parameters (e.g., orientation and position) by solving a viewpoint mapping problem, which aims to determine viable mirror configuration that is able to provide reflection image from a desired viewpoint. By applying the design principle, We show a full-surround snapshot catadioptric imaging system built with eight pairs of planar mirrors. Specifically, we place the mirror pairs on nested pyramid surfaces with different angles for capturing surrounding multi-view images in a single shot. This would allow 3D reconstruction of dynamic scenes. Our mirror design is customizable based on the size of the scene for optimized view coverage. We use the 3D Gaussian Splatting (3DGS) representation for scene reconstruction and novel view synthesis. We overcome the challenge posed by our sparse view input by integrating visual hull-derived depth constraint. We perform experiments on rendered synthetic images and real images of a variety of miniature scenes that are captured by our custom-built imaging system. Experimental results demonstrate that our method outperforms state-of-the-art sparse-view and calibration-free 3DGS methods. We also show novel view synthesis results on dynamic miniature scenes (e.g., a moving live insect).
- 29 April 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Pattern Analysis and Machine Intelligence
- Vol. PP (01628828) , 1-14
Snapshot 3D Gaussian Splatting for Miniature Scenes
We present a snapshot imaging technique for recovering 3D surrounding views of miniature scenes. Due to their intricacy, miniature scenes with objects sized in millimeters are difficult to reconstruct. Yet miniatures are common in life and their 3D digitalization is desirable. We design a catadioptric imaging system with a single camera and multiple pairs of planar mirrors for snapshot 3D reconstruction of miniature scenes from a dollhouse perspective. We first present an in-depth analysis on how to configure catadioptric imaging systems that use a pair of planar mirrors. We derive mirror parameters (e.g., orientation and position) by solving a viewpoint mapping problem, which aims to determine viable mirror configuration that is able to provide reflection image from a desired viewpoint. By applying the design principle, We show a full-surround snapshot catadioptric imaging system built with eight pairs of planar mirrors. Specifically, we place the mirror pairs on nested pyramid surfaces with different angles for capturing surrounding multi-view images in a single shot. This would allow 3D reconstruction of dynamic scenes. Our mirror design is customizable based on the size of the scene for optimized view coverage. We use the 3D Gaussian Splatting (3DGS) representation for scene reconstruction and novel view synthesis. We overcome the challenge posed by our sparse view input by integrating visual hull-derived depth constraint. We perform experiments on rendered synthetic images and real images of a variety of miniature scenes that are captured by our custom-built imaging system. Experimental results demonstrate that our method outperforms state-of-the-art sparse-view and calibration-free 3DGS methods. We also show novel view synthesis results on dynamic miniature scenes (e.g., a moving live insect).