Video-Based Depth Estimation Autoencoder With Weighted Temporal Feature and Spatial Edge Guided Modules
- 16 October 2023
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Artificial Intelligence
- Vol. 5 (2) , 613-623
Abstract
Convolutional neural networks with encoder and decoder structures, generally referred to as autoencoders, are used in many pixel- wise transformation, detection, segmentation and estimation applications, for example, which can be applied for face swapping, lane detection, semantic segmentation, and depth estimation, respectively. However, traditional autoencoders, which are based on single-frame inputs, ignore the temporal consistency between consecutive frames, and may hence produce unsatisfactory results. Accordingly, in this paper, a video-based depth estimation (VDE) autoencoder is proposed to improve the quality of depth estimation through the inclusion of two weighted temporal feature (WTF) modules in the encoder and a single spatial edge guided (SEG) module in the decoder. The WTF modules designed with channel weighted block (CWB) submodule effectively extract the temporal similarities in consecutive frames, while the SEG module provides spatial edge guidance of the object contours. Through the collaboration of the proposed modules, the accuracy of the depth estimation is greatly improved. The experimental results confirm that the proposed VDE autoencoder achieves a better monocular depth estimation performance than the existing autoencoders with only a slight increase in the computational cost.Keywords
Funding Information
- NSTC, Taiwan (#111-2634-F-006-012, #111-2222-E-167-005)
- Qualcomm, USA (SOW # NAT-435536)
This publication has 47 references indexed in Scilit:
- MultiMAE: Multi-modal Multi-task Masked Autoencoders2022
- Autoencoder: An Unsupervised Deep Learning Approach2022
- Improved quadruple sparse census transform and adaptive multi‐shape aggregation algorithms for precise stereo matchingIET Computer Vision, 2021
- Deep learning for monocular depth estimation: A reviewNeurocomputing, 2021
- Improved Lane Detection With Multilevel Features in Branch Convolutional Neural NetworksIEEE Access, 2019
- Unsupervised representation learning with Laplacian pyramid auto-encodersApplied Soft Computing, 2019
- U-Net: Convolutional Networks for Biomedical Image Segmentation2015
- Design of intelligent vehicle control system of self-directed2012
- Transforming Auto-Encoders2011
- Nonlinear principal component analysis using autoassociative neural networksAIChE Journal, 1991