Neural Spectro-polarimetric Fields
Open Access
- 10 December 2023
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 1-11
Abstract
Modeling the spatial radiance distribution of light rays in a scene has been extensively explored for applications, including view synthesis. Spectrum and polarization, the wave properties of light, are often neglected due to their integration into three RGB spectral bands and their non-perceptibility to human vision. However, these properties are known to encompass substantial material and geometric information about a scene. Here, we propose to model spectro-polarimetric fields, the spatial Stokes-vector distribution of any light ray at an arbitrary wavelength. We present Neural Spectro-polarimetric Fields (NeSpoF), a neural representation that models the physically-valid Stokes vector at given continuous variables of position, direction, and wavelength. NeSpoF manages inherently noisy raw measurements, showcases memory efficiency, and preserves physically vital signals — factors that are crucial for representing the high-dimensional signal of a spectro-polarimetric field. To validate NeSpoF, we introduce the first multi-view hyperspectral-polarimetric image dataset, comprised of both synthetic and real-world scenes. These were captured using our compact hyperspectral-polarimetric imaging system, which has been calibrated for robustness against system imperfections. We demonstrate the capabilities of NeSpoF on diverse scenes.Keywords
All Related Versions
This publication has 58 references indexed in Scilit:
- A reconfigurable camera add-on for high dynamic range, multispectral, polarization, and light-field imagingACM Transactions on Graphics, 2013
- 3D imaging spectroscopy for measuring hyperspectral patterns on solid objectsACM Transactions on Graphics, 2012
- SPICES: spectro-polarimetric imaging and characterization of exoplanetary systemsExperimental Astronomy, 2012
- Multiview face capture using polarized spherical gradient illuminationACM Transactions on Graphics, 2011
- Light field microscopy2006
- High performance imaging using large camera arrays2005
- Multi-View Stereo Reconstruction of Dense Shape and Complex AppearanceInternational Journal of Computer Vision, 2005
- Light field rendering1996
- Optimum angles for a polarimeter: part IOptical Engineering, 1995
- Single lens stereo with a plenoptic cameraIEEE Transactions on Pattern Analysis and Machine Intelligence, 1992