Parameter-Free Neural Lens Blur Rendering for High-Fidelity Composites
Preprint
Abstract
Consistent and natural camera lens blur is important for seamlessly blending 3D virtual objects into photographed real-scenes. Since lens blur typically varies with scene depth, the placement of virtual objects and their corresponding blur levels significantly affect the visual fidelity of mixed reality compositions. Existing pipelines often rely on camera parameters (e.g., focal length, focus distance, aperture size) and scene depth to compute the circle of confusion (CoC) for realistic lens blur rendering. However, such information is often unavailable to ordinary users, limiting the accessibility and generalizability of these methods. In this work, we propose a novel compositing approach that directly estimates the CoC map from RGB images, bypassing the need for scene depth or camera metadata. The CoC values for virtual objects are inferred through a linear relationship between its signed CoC map and depth, and realistic lens blur is rendered using a neural reblurring network. Our method provides flexible and practical solution for real-world applications. Experimental results demonstrate that our method achieves high-fidelity compositing with realistic defocus effects, outperforming state-of-the-art techniques in both qualitative and quantitative evaluations.Keywords
All Related Versions
This publication has 15 references indexed in Scilit:
- Blind Augmentation: Calibration-Free Camera Distortion Model Estimation for Real-Time Mixed-Reality ConsistencyIEEE Transactions on Visualization and Computer Graphics, 2025
- BokehMe++: Harmonious Fusion of Classical and Neural Rendering for Versatile Bokeh CreationIEEE Transactions on Pattern Analysis and Machine Intelligence, 2024
- AIFNet: All-in-Focus Image Restoration Network Using a Light Field-Based DatasetIEEE Transactions on Computational Imaging, 2021
- Discrimination-aware Network Pruning for Deep Model CompressionIEEE Transactions on Pattern Analysis and Machine Intelligence, 2021
- Joint Depth and Defocus Estimation From a Single Image Using Physical ConsistencyIEEE Transactions on Image Processing, 2021
- Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-Shot Cross-Dataset TransferIEEE Transactions on Pattern Analysis and Machine Intelligence, 2020
- MR360: Mixed Reality Rendering for 360° Panoramic VideosIEEE Transactions on Visualization and Computer Graphics, 2017
- Real‐time Depth of Field Rendering via Dynamic Light Field Generation and FilteringComputer Graphics Forum, 2010
- Inverse lighting and photorealistic rendering for augmented realityThe Visual Computer, 2010
- Simulating Low-Cost Cameras for Augmented Reality CompositingIEEE Transactions on Visualization and Computer Graphics, 2009