A Real-Time Incremental Video Mosaic Framework for UAV Remote Sensing
Open Access
- 18 April 2023
- journal article
- research article
- Published by MDPI AG in Remote Sensing
- Vol. 15 (8) , 2127
Abstract
Unmanned aerial vehicles (UAVs) are becoming increasingly popular in various fields such as agriculture, forest protection, resource exploration, and so on, due to their ability to capture high-resolution images quickly and efficiently at low altitudes. However, real-time image mosaicking of UAV image sequences, especially during long multi-strip flights, remains challenging. In this paper, a real-time incremental UAV image mosaicking framework is proposed, which only uses the UAV image sequence, and does not rely on global positioning system (GPS), ground control points (CGPs), or other auxiliary information. Our framework aims to reduce spatial distortion, increase the speed of the operation in the mosaicking process, and output high-quality panorama. To achieve this goal, we employ several strategies. First, the framework estimates the approximate position of each newly added frame and selects keyframes to improve efficiency. Then, the matching relationship between keyframes and other frames is obtained by using the estimated position. After that, a new optimization method based on minimizing weighted reprojection errors is adopted to carry out precise position calculation of the current frame, so as to reduce the deformation caused by cumulative errors. Finally, the weighted partition fusion method based on the Laplacian pyramid is used to fuse and update the local image in real time to achieve the best mosaic result. We have carried out a series of experiments which show that our system can output high-quality panorama in real time. The proposed keyframe selection strategy and local optimization strategy can minimize cumulative errors, the image fusion strategy is highly robust, and it can effectively improve the panorama quality.Keywords
Funding Information
- the national key Research and Development Program of China (2022YFF0904403, 42130104)
- The National Natural Science Foundation of China (2022YFF0904403, 42130104)
This publication has 27 references indexed in Scilit:
- A Fast Mosaicking Method for Small UAV Image Sequence Using a Small Number of Ground Control Points2016
- Map2DFusion: Real-time incremental UAV image mosaicing based on monocular SLAM2016
- LIFT: Learned Invariant Feature Transform2016
- A survey on image mosaicing techniquesJournal of Visual Communication and Image Representation, 2016
- Cross Correlation versus Mutual Information for Image MosaicingInternational Journal of Advanced Computer Science and Applications, 2013
- A NEW APPROACH TO FAST MOSAIC UAV IMAGESThe International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 2012
- Image Alignment and Stitching: A TutorialFoundations and Trends in Computer Graphics and Vision, 2007
- Adaptive Region-Based Video Registration2005
- Distinctive Image Features from Scale-Invariant KeypointsInternational Journal of Computer Vision, 2004
- A multiresolution spline with application to image mosaicsACM Transactions on Graphics, 1983