U2NeRF: Unsupervised Underwater Image Restoration and Neural Radiance Fields
Authors:
Preprint
Abstract
Underwater images suffer from colour shifts, low contrast, and haziness due to light absorption, refraction, scattering and restoring these images has warranted much attention. In this work, we present Unsupervised Underwater Neural Radiance Field U2NeRF, a transformer-based architecture that learns to render and restore novel views conditioned on multi-view geometry simultaneously. Due to the absence of supervision, we attempt to implicitly bake restoring capabilities onto the NeRF pipeline and disentangle the predicted color into several components - scene radiance, direct transmission map, backscatter transmission map, and global background light, and when combined reconstruct the underwater image in a self-supervised manner. In addition, we release an Underwater View Synthesis UVS dataset consisting of 12 underwater scenes, containing both synthetically-generated and real-world data. Our experiments demonstrate that when optimized on a single scene, U2NeRF outperforms several baselines by as much LPIPS 11%, UIQM 5%, UCIQE 4% (on average) and showcases improved rendering and restoration capabilities. Code will be made available upon acceptance.Keywords
All Related Versions
This publication has 12 references indexed in Scilit:
- SinNeRF: Training Neural Radiance Fields on Complex Scenes from a Single Image2022
- Domain Adaptation for Underwater Image Enhancement via Content and Style SeparationIEEE Access, 2022
- NAN: Noise-Aware NeRFs for Burst-Denoising2022
- Nerfies: Deformable Neural Radiance Fields2021
- Human-Visual-System-Inspired Underwater Image Quality MeasuresIEEE Journal of Oceanic Engineering, 2015
- An Underwater Color Image Quality Evaluation MetricIEEE Transactions on Image Processing, 2015
- A Fast Single Image Haze Removal Algorithm Using Color Attenuation PriorIEEE Transactions on Image Processing, 2015
- Image quality assessment: from error visibility to structural similarityIEEE Transactions on Image Processing, 2004
- Optical models for direct volume renderingIEEE Transactions on Visualization and Computer Graphics, 1995
- A spatial processor model for object colour perceptionJournal of the Franklin Institute, 1980