Fast Depth Prediction and Obstacle Avoidance on a Monocular Drone Using Probabilistic Convolutional Neural Network
- 5 December 2019
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Intelligent Transportation Systems
- Vol. 22 (1) , 156-167
Abstract
Recent studies employ advanced deep convolutional neural networks (CNNs) for monocular depth perception, which can hardly run efficiently on small drones that rely on low/middle-grade GPU(e.g. TX2 and 1050Ti) for computation. In addition, the methods which can effectively and efficiently produce probabilistic depth prediction with a measure of model confidence have not been well studied. The lack of such a method could yield erroneous, sometimes fatal, decisions in drone applications (e.g. selecting a waypoint in a region with a large depth yet a low estimation confidence). This paper presents a real-time onboard approach for monocular depth prediction and obstacle avoidance with a lightweight probabilistic CNN (pCNN), which will be ideal for use in a lightweight energy-efficient drone. For each video frame, our pCNN can efficiently predict its depth map and the corresponding confidence. The accuracy of our lightweight pCNN is greatly boosted by integrating sparse depth estimation from a visual odometry into the network for guiding dense depth and confidence inference. The estimated depth map is transformed into Ego Dynamic Space (EDS) by embedding both dynamic motion constraints of a drone and the confidence values into the spatial depth map. Traversable waypoints are automatically computed in EDS based on which appropriate control inputs for the drone are produced. Extensive experimental results on public datasets demonstrate that our depth prediction method runs at 12Hz and 45Hz on TX2 and 1050Ti GPU respectively, which is 1.8X~5.6X faster than the state-of-the-art methods and achieves better depth estimation accuracy. We also conducted experiments of obstacle avoidance in both simulated and real environments to demonstrate the superiority of our method to the baseline methods.Keywords
Funding Information
- National Natural Science Foundation of China (61872417)
- Wuhan Science and Technology Bureau (2017010201010111)
- Hubei Provicinal Natural Science Foundation (ZRMS2017000375)
- Fundamental Research Funds for the Central Universities (2019kfyRCPY118)
- Program for HUST Acadamic Frontier Youth Team
This publication has 24 references indexed in Scilit:
- ORB-SLAM: A Versatile and Accurate Monocular SLAM SystemIEEE Transactions on Robotics, 2015
- PX4: A node-based multithreaded open source robotics framework for deeply embedded platforms2015
- Autonomous navigation of generic monocular quadcopter in natural environment2015
- Vision meets robotics: The KITTI datasetThe International Journal of Robotics Research, 2013
- An open source and open hardware embedded metric optical flow CMOS camera for indoor and outdoor applications2013
- Indoor Segmentation and Support Inference from RGBD Images2012
- EPnP: An Accurate O(n) Solution to the PnP ProblemInternational Journal of Computer Vision, 2008
- High speed obstacle avoidance using monocular vision and reinforcement learning2005
- Reactive collision avoidance for navigation with dynamic constraints2003
- Random sample consensusCommunications of the ACM, 1981