Synthetic Dataset Generation Using Photo-Realistic Simulation with Varied Time and Weather Axes
Open Access
- 17 April 2024
- journal article
- research article
- Published by MDPI AG in Electronics
- Vol. 13 (8) , 1516
Abstract
To facilitate the integration of autonomous unmanned air vehicles (UAVs) in day-to-day life, it is imperative that safe navigation can be demonstrated in all relevant scenarios. For UAVs using a navigational protocol driven by artificial neural networks, training and testing data from multiple environmental contexts are needed to ensure that bias is minimised. The reduction in predictive capacity when faced with unfamiliar data is a common weak point in trained networks, which worsens the further the input data deviates from the training data. However, training for multiple environmental variables dramatically increases the man-hours required for data collection and validation. In this work, a potential solution to this data availability issue is presented through the generation and evaluation of photo-realistic image datasets from a simulation of 3D-scanned physical spaces which are theoretically linked in a digital twin (DT) configuration. This simulation is then used to generate environmentally varied iterations of the target object in that physical space by two contextual variables (weather and daylight). This results in an expanded dataset of bicycles that contains weather and time-varied components of the same images which are then evaluated using a generic build of the YoloV3 object detection network; the response is then compared to two real image (night and day) datasets as a baseline. The results reveal that the network response remained consistent across the temporal axis, maintaining a measured domain shift of approximately 23% between the two baselines.Keywords
Funding Information
- Science Foundation Ireland (18/CRT/6222)
This publication has 47 references indexed in Scilit:
- Robust Monocular Localization of Drones by Adapting Domain Maps to Depth Prediction Inaccuracies2023
- Optimized deep learning vision system for human action recognition from drone imagesMultimedia Tools and Applications, 2023
- Learning Perception-Aware Agile Flight in Cluttered Environments2023
- A Systematic Literature Review of Generative Adversarial Network Potential In AI Artwork2023
- A digital twin for 3D path planning of large-span curved-arm gantry robotRobotics and Computer-Integrated Manufacturing, 2022
- Review of digital twin about concepts, technologies, and industrial applicationsJournal of Manufacturing Systems, 2021
- Digital Twin: Enabling Technologies, Challenges and Open ResearchIEEE Access, 2020
- Characterising the Digital Twin: A systematic literature reviewCIRP Journal of Manufacturing Science and Technology, 2020
- Deep Attention Neural Network for Multi-Label Classification in Unmanned Aerial Vehicle ImageryIEEE Access, 2019
- The philosophy of simulation: hot new issues or same old stew?Synthese, 2008