Precise Ranging to an Aerial Refueling Coupler Using a DNN-Based Detection Method Based on Monocular Camera Data
Open Access
- 19 March 2026
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 14, 44108-44119
Abstract
Aerial refueling is considered a key force multiplier for military aviation, as it can be used to extend the range and endurance of aircraft. With the rise of large uncrewed aerial systems (UASs), the need to develop a method by which they can act as the receiver for aerial refueling has become a priority. The Office of Naval Research has funded multiple studies to determine the feasibility of using deep neural networks (DNNs) to enable autonomous aerial refueling for probe and drogue aerial refueling with a UAS as the receiver. In this study, we expand on previous research to present a monocular camera system integrated with a DNN to accurately estimate coordinates and range to a refueling drogue within the final 25 feet of approach. Our method employs a similar-triangle algorithm that computes range and positional estimates from DNN-generated bounding boxes, with ground truth positional data provided by a calibrated motion capture system. Experiments using UR10, YASKAWA, and linear track manipulators, coupled with a motion capture system to provide truth data, demonstrate that the DNN achieves excellent precision and recall. Our experiment resulted in a mAP50 of 0.995 for all classes, and mAP50-95 values of 0.945, 0.851, and 0.898 for the drogue, coupler, and overall, respectively. Our algorithm returned an estimated range to the center of the refueling coupler that was within the 4-inch tolerance provided by our subject matter experts when the camera was within 10 feet of the target. Additionally, the majority of data points were also within the 4-inch threshold when evaluated at ranges within 22 feet. We determined that if the heading offset between the camera and the drogue was known, the algorithm could account for the misalignment and produce satisfactory results. This paper demonstrates that a DNN-based monocular vision camera system can provide range estimations within the margins needed for the aerial refueling mission in a laboratory setting.Keywords
Funding Information
- Office of Naval Research, and in collaboration with the Lockheed Martin Corporation
This publication has 38 references indexed in Scilit:
- A Deep Neural Network Approach for Drogue Detection Using Laboratory-Chroma Key ImagesIEEE Access, 2024
- DNN Based Ranging in Support of Autonomous Aerial RefuelingJournal of Intelligent & Robotic Systems, 2023
- Assessing the Predictive Performance of Two DNN Models: A Comparative Analysis to Support Reusing Training Weights for Autonomous Aerial Refueling MissionsIEEE Access, 2023
- Using a run time assurance approach for certifying autonomy within naval aviationSystems Engineering, 2023
- End-to-End Object Detection with Transformers2020
- A Novel Method for Measuring Drogue-UAV Relative Pose in Autonomous Aerial Refueling Based on Monocular VisionIEEE Access, 2019
- Drogue detection for autonomous aerial refueling based on convolutional neural networksChinese Journal of Aeronautics, 2017
- ImageNet Large Scale Visual Recognition ChallengeInternational Journal of Computer Vision, 2015
- Microsoft COCO: Common Objects in Context2014
- Robot sensor calibration: solving AX=XB on the Euclidean groupIEEE Transactions on Robotics and Automation, 1994