Harel Haim

Stereo imaging is the most common passive method for producing reliable depth maps. Calibration is a crucial step for every stereo-based system, and despite all the advancements in the field, most calibrations are still done by the same tedious method using a checkerboard target. Monocular-based depth estimation methods do not require extrinsic calibration but generally achieve inferior depth accuracy. In this paper, we present a novel online self-calibration method, which makes use of both stereo and monocular depth maps to find the transformation required for extrinsic calibration by enforcing consistency between both maps. The proposed method works in a closed-loop and exploits the pre-trained networks' global context, and thus avoids feature matching and outliers issues. In addition to presenting our method using an image-based monocular depth estimation method, which can be implemented in most systems without additional changes, we also show that adding a phase-coded aperture mask leads to even better and faster convergence. We demonstrate our method on road scenes from the KITTI vision benchmark and real-world scenes using our prototype camera. Our code is publicly available at https://github.com/YotYot/CalibrationNet.

Depth estimation from a single image is a well known challenge in computer vision. With the advent of deep learning, several approaches for monocular depth estimation have been proposed, all of which have inherent limitations due to the scarce depth cues that exist in a single image. Moreover, these methods are very demanding computationally, which makes them inadequate for systems with limited processing power. In this paper, a phase-coded aperture camera for depth estimation is proposed. The camera is equipped with an optical phase mask that provides unambiguous depth-related color characteristics for the captured image. These are used for estimating the scene depth map using a fully-convolutional neural network. The phase-coded aperture structure is learned jointly with the network weights using back-propagation. The strong depth cues (encoded in the image by the phase mask, designed together with the network weights) allow a much simpler neural network architecture for faster and more accurate depth estimation. Performance achieved on simulated images as well as on a real optical setup is superior to state-of-the-art monocular depth estimation methods (both with respect to the depth accuracy and required processing power), and is competitive with more complex and expensive depth estimation methods such as light-field cameras.

Single image depth estimation is achieved using computational imaging and Deep Learning (DL). Imaging with phase-mask is also modeled as a DL-layer, and the mask and DL parameters are jointly designed using labeled data.

We have recently shown [Appl. Opt. 51, 2739 (2012)] that performance analysis of optical imaging systems based on results obtained with classic one-dimensional (1D) resolution targets (such as the U.S. Air Force resolution target) are significantly different than those obtained with a newly proposed two-dimensional (2D) target. We hereby provide experimental evidence and show how the new 2D template can be used to correctly characterize optical imaging systems in terms of resolution and contrast. In particular, we apply the consequences of these observations to the optimal design of some 2D barcode structures.

Present determination of optical imaging systems specifications are based on performance values and modulation transfer function results carried with a 1D resolution template (such as the USAF resolution target or spoke templates). Such a template allows determining image quality, resolution limit, and contrast. Nevertheless, the conventional 1D template does not provide satisfactory results, since most optical imaging systems handle 2D objects for which imaging system response may be different by virtue of some not readily observable spatial frequencies. In this paper we derive and analyze contrast transfer function results obtained with 1D as well as 2D templates.