Fusing Heterogeneous Data: A Case for Remote Sensing and Social Media
- 17 July 2018
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Geoscience and Remote Sensing
- Vol. 56 (12) , 6956-6968
Abstract
Data heterogeneity can pose a great challenge to process and systematically fuse low-level data from different modalities with no recourse to heuristics and manual adjustments and refinements. In this paper, a new methodology is introduced for the fusion of measured data for detecting and predicting weather-driven natural hazards. The proposed research introduces a robust theoretical and algorithmic framework for the fusion of heterogeneous data in near real time. We establish a flexible information-based fusion framework with a target optimality criterion of choice, which for illustration, is specialized to a maximum entropy principle and a least effort principle for semisupervised learning with noisy labels. We develop a methodology to account for multimodality data and a solution for addressing inherent sensor limitations. In our case study of interest, namely, that of flood density estimation, we further show that by fusing remote sensing and social media data, we can develop well founded and actionable flood maps. This capability is valuable in situations where environmental hazards, such as hurricanes or severe weather, affect very large areas. Relative to the state of the art working with such data, our proposed information-theoretic solution is principled and systematic, while offering a joint exploitation of any set of heterogeneous sensor modalities with minimally assuming priors. This flexibility is coupled with the ability to quantitatively and clearly state the fusion principles with very reasonable computational costs. The proposed method is tested and substantiated with the multimodality data of a 2013 Boulder Colorado flood event.Keywords
Funding Information
- National Nuclear Security Administration (DE-NA0002576)
This publication has 25 references indexed in Scilit:
- Twitter for city police department information sharingProceedings of the American Society for Information Science and Technology, 2010
- Object based image analysis for remote sensingISPRS Journal of Photogrammetry and Remote Sensing, 2010
- Modeling of species distributions with Maxent: new extensions and a comprehensive evaluationEcography, 2008
- Modification of normalised difference water index (NDWI) to enhance open water features in remotely sensed imageryInternational Journal of Remote Sensing, 2006
- Multi-resolution, object-oriented fuzzy analysis of remote sensing data for GIS-ready informationISPRS Journal of Photogrammetry and Remote Sensing, 2004
- Feature fusion: parallel strategy vs. serial strategyPattern Recognition, 2003
- Inducing features of random fieldsIEEE Transactions on Pattern Analysis and Machine Intelligence, 1997
- Sensor fusion potential exploitation-innovative architectures and illustrative applicationsProceedings of the IEEE, 1997
- NDWI—A normalized difference water index for remote sensing of vegetation liquid water from spaceRemote Sensing of Environment, 1996
- Information Theory and Statistical MechanicsPhysical Review B, 1957