Structure-from-motion in micro-image domain for uncalibrated plenoptic 2.0 cameras
Open Access
- 6 January 2026
- journal article
- research article
- Published by Springer Nature in International Journal of Computer Vision
- Vol. 134 (1) , 34
Abstract
We introduce a structure-from-motion method specifically designed to process the raw micro-images captured by plenoptic 2.0 cameras. Unlike traditional monocular cameras, plenoptic cameras incorporate a micro-lens array between the sensor and the main lens, capturing depth information at the expense of a more complex set of parameters to evaluate. Instead of simply integrating their projection model into the classical structure-from-motion pipeline, our contribution identifies the pinhole cameras-driven constraints and takes advantage of the inherent disparity information present in plenoptic cameras. This facilitates a robust initialization of the reconstruction. Our method shortcuts two of the limitations of the classical structure-from-motion: the ambiguity found in scenes captured with low angular disparity and the scale ambiguity. It enables the reconstruction of scenes captured by multiple uncalibrated plenoptic cameras, without using any calibration pattern or subaperture view extraction step. Our method undergoes experimental validation on both natural and synthetic datasets, showing a 10% error accuracy for relative pose estimation, which is comparable to calibration-pattern based methods. The results are robust to coarse initialization. Contrary to classical structure-from-motion, it is able to reconstruct scenes with parallel facing cameras. It also shows greater accuracy than reconstruction methods based on pinhole camera conversion.Keywords
Funding Information
- H2020 European Research Council (951989)
- H2020 European Research Council (101137416)
This publication has 58 references indexed in Scilit:
- LiFCal: Online Light Field Camera Calibration via Bundle Adjustment2025
- Grounding Image Matching in 3D with MASt3R2024
- FMGS: Foundation Model Embedded 3D Gaussian Splatting for Holistic 3D Scene UnderstandingInternational Journal of Computer Vision, 2024
- Leveraging Blur Information for Plenoptic Camera CalibrationInternational Journal of Computer Vision, 2022
- Scale-Awareness of Light Field Camera Based Visual Odometry2018
- Synthetic Aperture Based on Plenoptic Camera for Seeing Through Occlusions2018
- Bundle Adjustment — A Modern Synthesis2000
- 3D model acquisition from extended image sequences1996
- Shape and motion from image streams under orthography: a factorization methodInternational Journal of Computer Vision, 1992
- The Levenberg-Marquardt algorithm: Implementation and theory1978