Ning Yu
Research Subjects
Ning Yu has a total of 43 co-authors. Here are the top co-authors by number of shared publications:
- Li Ma
Eyeline Labs, Los Angeles, USA
7 shared publications - Paul Debevec
Eyeline Labs, Los Angeles, USA
7 shared publications - Mingming He
Netflix Eyeline Studios, Los Angeles, USA
6 shared publications - Julien Philip
Eyeline Labs, London, United Kingdom
4 shared publications - Yuancheng Xu
Netflix, Los Angeles, USA
4 shared publications - Ahmet Levent Taşel
Eyeline Labs, Vancouver, Canada
3 shared publications
Latest Publications
- Open Access
- 30 November 2026
- Published by Association for Computing Machinery (ACM)
- p. 1-12
ID-V2V: Identity-Preserving Video Restylization
- 30 November 2026
- Published by Association for Computing Machinery (ACM)
- p. 1-12
Open AccessID-V2V: Identity-Preserving Video Restylization
- 2 October 2026
- Published by Springer Nature
- p. 336-357
DiffHDR: Re-exposing LDR Videos with Video Diffusion Models
- 2 October 2026
- Published by Springer Nature
- p. 336-357
DiffHDR: Re-exposing LDR Videos with Video Diffusion Models
- Open Access
- 19 July 2026
- Published by Association for Computing Machinery (ACM)
- p. 1-12
Go-with-the-Track: Video Compositing and Motion Control with Point Tracking
- 19 July 2026
- Published by Association for Computing Machinery (ACM)
- p. 1-12
Open AccessGo-with-the-Track: Video Compositing and Motion Control with Point Tracking
- Top Cited PapersOpen Access
- 19 July 2026
- Published by Association for Computing Machinery (ACM)
- p. 1-12
FreeOrbit4D: Training-Free Arbitrary Camera Redirection for Monocular Videos via Foreground-Complete 4D Reconstruction
Camera redirection aims to replay a dynamic scene from a single monocular video under a user-specified camera trajectory. However, large-angle redirection is inherently ill-posed: a monocular video captures only a narrow spatio-temporal view of a dynamic 3D scene, providing severely limited observations of the underlying 4D world. The key challenge is therefore to recover a complete and coherent plenoptic representation from this limited input, with consistent geometry and coherent motion. While recent diffusion-based methods achieve impressive visual generation quality, they often break down under large-angle viewpoint changes far from the original trajectory, where missing visual grounding leads to severe geometric ambiguity and temporal inconsistency. To address this, we present FreeOrbit4D, an effective training-free framework that tackles this geometric ambiguity by recovering a foreground-complete 4D proxy as structural grounding for video generation. We obtain this proxy by decoupling foreground and background reconstructions: we unproject the monocular video into a static background and partial foreground point clouds in a unified global space, then leverage an object-centric multi-view diffusion model to synthesize multi-view images and reconstruct complete foreground point clouds in canonical object space. By aligning the canonical foreground point cloud to the global scene space via dense pixel-synchronized 3D–3D correspondences and projecting the foreground-complete 4D proxy onto target camera viewpoints, we provide geometric scaffolds (e.g., depth/visibility cues) that guide a conditional video diffusion model. Extensive experiments show that FreeOrbit4D produces more faithful and temporally coherent redirected videos under challenging large-angle trajectories, and our foreground-complete 4D proxy further opens a potential avenue for practical applications such as edit propagation and 4D data generation. Project page: https://freeorbit4d.vision.ischool.illinois.edu/
- 19 July 2026
- Published by Association for Computing Machinery (ACM)
- p. 1-12
Top Cited PapersOpen AccessFreeOrbit4D: Training-Free Arbitrary Camera Redirection for Monocular Videos via Foreground-Complete 4D Reconstruction
Camera redirection aims to replay a dynamic scene from a single monocular video under a user-specified camera trajectory. However, large-angle redirection is inherently ill-posed: a monocular video captures only a narrow spatio-temporal view of a dynamic 3D scene, providing severely limited observations of the underlying 4D world. The key challenge is therefore to recover a complete and coherent plenoptic representation from this limited input, with consistent geometry and coherent motion. While recent diffusion-based methods achieve impressive visual generation quality, they often break down under large-angle viewpoint changes far from the original trajectory, where missing visual grounding leads to severe geometric ambiguity and temporal inconsistency. To address this, we present FreeOrbit4D, an effective training-free framework that tackles this geometric ambiguity by recovering a foreground-complete 4D proxy as structural grounding for video generation. We obtain this proxy by decoupling foreground and background reconstructions: we unproject the monocular video into a static background and partial foreground point clouds in a unified global space, then leverage an object-centric multi-view diffusion model to synthesize multi-view images and reconstruct complete foreground point clouds in canonical object space. By aligning the canonical foreground point cloud to the global scene space via dense pixel-synchronized 3D–3D correspondences and projecting the foreground-complete 4D proxy onto target camera viewpoints, we provide geometric scaffolds (e.g., depth/visibility cues) that guide a conditional video diffusion model. Extensive experiments show that FreeOrbit4D produces more faithful and temporally coherent redirected videos under challenging large-angle trajectories, and our foreground-complete 4D proxy further opens a potential avenue for practical applications such as edit propagation and 4D data generation. Project page: https://freeorbit4d.vision.ischool.illinois.edu/
- Open Access
- 19 July 2026
- Published by Association for Computing Machinery (ACM)
- p. 1-3
Promoting LDR Panoramas to HDRI with Rendering-Space Losses
- 19 July 2026
- Published by Association for Computing Machinery (ACM)
- p. 1-3
Open AccessPromoting LDR Panoramas to HDRI with Rendering-Space Losses
- Open Access
- 14 December 2025
- Published by Association for Computing Machinery (ACM)
- p. 1-12
Virtually Being: Customizing Camera-Controllable Video Diffusion Models with Volumetric Performance Captures
- 14 December 2025
- Published by Association for Computing Machinery (ACM)
- p. 1-12
Open AccessVirtually Being: Customizing Camera-Controllable Video Diffusion Models with Volumetric Performance Captures
- Open Access
- 4 December 2025
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Graphics
- Vol. 44 (6) , 1-13
Detail Enhanced Gaussian Splatting for Large-Scale Volumetric Capture
We present a unique system for large-scale, multi-performer, high resolution 4D volumetric capture providing realistic free-viewpoint video up to and including 4K resolution facial closeups. To achieve this, we employ a novel volumetric capture, reconstruction and rendering pipeline based on Dynamic Gaussian Splatting and Diffusion-based Detail Enhancement. We design our pipeline specifically to meet the demands of high-end media production. We employ two capture rigs: the Scene Rig , which captures multi-actor performances at a resolution which falls short of 4K production quality, and the Face Rig , which records high-fidelity single-actor facial detail to serve as a reference for detail enhancement. We first reconstruct dynamic performances from the Scene Rig using 4D Gaussian Splatting, incorporating new model designs and training strategies to improve reconstruction, dynamic range, and rendering quality. Then to render high-quality images for facial closeups, we introduce a diffusion-based detail enhancement model. This model is fine-tuned with high-fidelity data from the same actors recorded in the Face Rig. We train on paired data generated from low- and high-quality Gaussian Splatting (GS) models, using the low-quality input to match the quality of the Scene Rig , with the high-quality GS as ground truth. Our results demonstrate the effectiveness of this pipeline in bridging the gap between the scalable performance capture of a large-scale rig and the high-resolution standards required for film and media production.
- 4 December 2025
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Graphics
- Vol. 44 (6) , 1-13
Open AccessDetail Enhanced Gaussian Splatting for Large-Scale Volumetric Capture
We present a unique system for large-scale, multi-performer, high resolution 4D volumetric capture providing realistic free-viewpoint video up to and including 4K resolution facial closeups. To achieve this, we employ a novel volumetric capture, reconstruction and rendering pipeline based on Dynamic Gaussian Splatting and Diffusion-based Detail Enhancement. We design our pipeline specifically to meet the demands of high-end media production. We employ two capture rigs: the Scene Rig , which captures multi-actor performances at a resolution which falls short of 4K production quality, and the Face Rig , which records high-fidelity single-actor facial detail to serve as a reference for detail enhancement. We first reconstruct dynamic performances from the Scene Rig using 4D Gaussian Splatting, incorporating new model designs and training strategies to improve reconstruction, dynamic range, and rendering quality. Then to render high-quality images for facial closeups, we introduce a diffusion-based detail enhancement model. This model is fine-tuned with high-fidelity data from the same actors recorded in the Face Rig. We train on paired data generated from low- and high-quality Gaussian Splatting (GS) models, using the low-quality input to match the quality of the Scene Rig , with the high-quality GS as ground truth. Our results demonstrate the effectiveness of this pipeline in bridging the gap between the scalable performance capture of a large-scale rig and the high-resolution standards required for film and media production.
- Open Access
- 3 December 2024
- Published by Association for Computing Machinery (ACM)
- p. 1-12
DifFRelight: Diffusion-Based Facial Performance Relighting
- 3 December 2024
- Published by Association for Computing Machinery (ACM)
- p. 1-12
Open AccessDifFRelight: Diffusion-Based Facial Performance Relighting
- Top Cited PapersOpen Access
- 15 November 2023
- Published by Association for Computing Machinery (ACM)
- p. 3418-3432
DE-FAKE: Detection and Attribution of Fake Images Generated by Text-to-Image Generation Models
Text-to-image generation models that generate images based on prompt descriptions have attracted an increasing amount of attention during the past few months. Despite their encouraging performance, these models raise concerns about the misuse of their generated fake images. To tackle this problem, we pioneer a systematic study on the detection and attribution of fake images generated by text-to-image generation models. Concretely, we first build a machine learning classifier to detect the fake images generated by various text-to-image generation models. We then attribute these fake images to their source models, such that model owners can be held responsible for their models' misuse. We further investigate how prompts that generate fake images affect detection and attribution. We conduct extensive experiments on four popular text-to-image generation models, including DALL·E 2, Stable Diffusion, GLIDE, and Latent Diffusion, and two benchmark prompt-image datasets. Empirical results show that (1) fake images generated by various models can be distinguished from real ones, as there exists a common artifact shared by fake images from different models; (2) fake images can be effectively attributed to their source models, as different models leave unique fingerprints in their generated images; (3) prompts with the "person'' topic or a length between 25 and 75 enable models to generate fake images with higher authenticity. All findings contribute to the community's insight into the threats caused by text-to-image generation models. We appeal to the community's consideration of the counterpart solutions, like ours, against the rapidly-evolving fake image generation.
- 15 November 2023
- Published by Association for Computing Machinery (ACM)
- p. 3418-3432
Top Cited PapersOpen AccessDE-FAKE: Detection and Attribution of Fake Images Generated by Text-to-Image Generation Models
Text-to-image generation models that generate images based on prompt descriptions have attracted an increasing amount of attention during the past few months. Despite their encouraging performance, these models raise concerns about the misuse of their generated fake images. To tackle this problem, we pioneer a systematic study on the detection and attribution of fake images generated by text-to-image generation models. Concretely, we first build a machine learning classifier to detect the fake images generated by various text-to-image generation models. We then attribute these fake images to their source models, such that model owners can be held responsible for their models' misuse. We further investigate how prompts that generate fake images affect detection and attribution. We conduct extensive experiments on four popular text-to-image generation models, including DALL·E 2, Stable Diffusion, GLIDE, and Latent Diffusion, and two benchmark prompt-image datasets. Empirical results show that (1) fake images generated by various models can be distinguished from real ones, as there exists a common artifact shared by fake images from different models; (2) fake images can be effectively attributed to their source models, as different models leave unique fingerprints in their generated images; (3) prompts with the "person'' topic or a length between 25 and 75 enable models to generate fake images with higher authenticity. All findings contribute to the community's insight into the threats caused by text-to-image generation models. We appeal to the community's consideration of the counterpart solutions, like ours, against the rapidly-evolving fake image generation.