Identifying Disaster Regions in Images Through Attention Shifting With a Retarget Network
Open Access
- 13 August 2024
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 12, 143754-143766
Abstract
Disasters disrupt lives and necessitate quick location of affected areas for rescue efforts. The application of computer vision has enhanced disaster detection, such as landslides and floods; however traditional computer vision methods often overlook smaller, critical details in favor of prominent objects. This research introduces the Retarget Network (RetNet), a novel framework aimed at improving image captioning techniques to identify and prioritize these less evident, yet crucial, objects in disaster scenarios, enhancing scene recognition and aiding in more effective disaster response. By masking images, we direct the model’s focus towards additional significant areas within the image. RetNet employs anchor boxes to refine the targeting of specific areas and, optimize their center positions, heights, and widths. Additionally, RetNet determines which anchors to mask prior to captioning, facilitating the identification of challenging objects such as boulders, soil, and water, which are indicative of natural disasters. We validated RetNet across multiple disaster scenarios—landslides, floods, and wildfires—using images taken from various perspectives, including side-view, aerial and shipborne views. Our findings reveal an accuracy of 91.60% in landslide detection from side-view image captions and 87.50% for detections from shipborne views. These results underscore RetNet’s effectiveness in enhancing the identification of disaster-affefted regions.Keywords
Funding Information
- Japan Society for the Promotion of Science (JSPS) KAKENHI Grant-in-Aid for Scientific Research (22H01449)
- KAKENHI Fund for the Promotion of Joint International Research [fostering joint international research (B)] (JP20KK0086)
- Mohamed Bin Zayed International Robotics Challenge (MBZIRC) Grant
This publication has 30 references indexed in Scilit:
- Landslide detection based on shipborne images and deep learning models: a case study in the Three Gorges Reservoir Area in ChinaLandslides, 2022
- A Real-Time System for Detecting Landslide Reports on Social Media Using Artificial Intelligence2022
- Deep Learning Approaches Based on Transformer Architectures for Image Captioning TasksIEEE Access, 2022
- Landslide detection in the Himalayas using machine learning algorithms and U-NetLandslides, 2022
- Detection and segmentation of loess landslides via satellite images: a two-phase frameworkLandslides, 2022
- Influence of the Vertical Earthquake Component on the Shear Vibration of Buildings on Sliding FoundationsE3S Web of Conferences, 2021
- Landslide Detection Using Residual Networks and the Fusion of Spectral and Topographic InformationIEEE Access, 2019
- One page project management application on flood preparedness: case study of ThailandProcedia Engineering, 2018
- ImageNet Large Scale Visual Recognition ChallengeInternational Journal of Computer Vision, 2015
- The Varnes classification of landslide types, an updateLandslides, 2013