Joint-ID: Transformer-Based Joint Image Enhancement and Depth Estimation for Underwater Environments
- 8 December 2023
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Sensors Journal
- Vol. 24 (3) , 3113-3122
Abstract
Underwater imaging is a challenging task due to factors such as scattering, absorption, and turbulence, which degrade image quality and limit visibility. In this paper, we propose a novel approach for enhancing underwater images that leverages the benefits of joint learning for simultaneous image enhancement and depth estimation. We introduce Joint-ID, a transformer-based neural network that can obtain high-perceptual image quality and depth information from raw underwater images. Our approach formulates a multi-modal objective function that addresses invalid depth, lack of sharpness, and image degradation based on color and local texture. We design an end-to-end training pipeline that enables joint restoration and depth estimation in a shared hierarchical feature space. Additionally, we propose a synthetic dataset with various distortions and scene depths for multi-task learning. We evaluate Joint-ID on synthetic and standard datasets, as well as real underwater images with diverse spectra and harsh turbidity, demonstrating its effectiveness for underwater image enhancement and depth estimation. Furthermore, we demonstrate the ability of Joint-ID to perform feature matching and saliency detection for visually guided underwater robots. Our proposed method has the potential to improve the visual perception of underwater environments and benefit applications such as oceanography and underwater robotics. Supplements are available at: https://sites.google.com/view/joint-id/home.Keywords
Funding Information
- Inha University and the National Research Foundation of Korea
- Korean Government [Ministry of Science and ICT (MSIT)] (RS-2023-00302589, 2022R1A4A3029480)
- Institute of Information and Communications Technology Planning and Evaluation
- Korean government (2022-0-00448)
- Korea Institute of Marine Science and Technology promotion
- Ministry of Oceans and Fisheries (20210562)
This publication has 26 references indexed in Scilit:
- Laplacian-Steered Neural Style Transfer2017
- WaterGAN: Unsupervised Generative Network to Enable Real-time Color Correction of Monocular Underwater ImagesIEEE Robotics and Automation Letters, 2017
- Online depth estimation and application to underwater image dehazing2016
- U-Net: Convolutional Networks for Biomedical Image Segmentation2015
- An Underwater Color Image Quality Evaluation MetricIEEE Transactions on Image Processing, 2015
- Inherent optical properties of Jerlov water typesApplied Optics, 2015
- Automatic Red-Channel underwater image restorationJournal of Visual Communication and Image Representation, 2015
- A retinex-based enhancing approach for single underwater image2014
- Enhancing underwater images and videos by fusion2012
- Underwater Image Enhancement by Wavelength Compensation and DehazingIEEE Transactions on Image Processing, 2011