Dataset Reduction Techniques to Speed Up SVD Analyses on Big Geo-Datasets
Open Access
- 26 January 2019
- journal article
- research article
- Published by MDPI AG in ISPRS International Journal of Geo-Information
- Vol. 8 (2) , 55
Abstract
The Singular Value Decomposition (SVD) is a mathematical procedure with multiple applications in the geosciences. For instance, it is used in dimensionality reduction and as a support operator for various analytical tasks applicable to spatio-temporal data. Performing SVD analyses on large datasets, however, can be computationally costly, time consuming, and sometimes practically infeasible. However, techniques exist to arrive at the same output, or at a close approximation, which requires far less effort. This article examines several such techniques in relation to the inherent scale of the structure within the data. When the values of a dataset vary slowly, e.g., in a spatial field of temperature over a country, there is autocorrelation and the field contains large scale structure. Datasets do not need a high resolution to describe such fields and their analysis can benefit from alternative SVD techniques based on rank deficiency, coarsening, or matrix factorization approaches. We use both simulated Gaussian Random Fields with various levels of autocorrelation and real-world geospatial datasets to illustrate our study while examining the accuracy of various SVD techniques. As the main result, this article provides researchers with a decision tree indicating which technique to use when and predicting the resulting level of accuracy based on the dataset’s structure scale.Keywords
Funding Information
- Nederlandse Organisatie voor Wetenschappelijk Onderzoek (NA)
This publication has 31 references indexed in Scilit:
- Variable selection models for genomic selection using whole-genome sequence data and singular value decompositionGenetics Selection Evolution, 2017
- Adaptive neuro-fuzzy inference system multi-objective optimization using the genetic algorithm/singular value decomposition method for modelling the discharge coefficient in rectangular sharp-crested side weirsEngineering Optimization, 2015
- Application of spectral analysis techniques to the intercomparison of aerosol data – Part 4: Synthesized analysis of multisensor satellite and ground-based AOD measurements using combined maximum covariance analysisAtmospheric Measurement Techniques, 2014
- Semisupervised Kernel Feature Extraction for Remote Sensing Image AnalysisIEEE Transactions on Geoscience and Remote Sensing, 2014
- Application of spectral analysis techniques in the intercomparison of aerosol data. Part II: Using maximum covariance analysis to effectively compare spatiotemporal variability of satellite and AERONET measured aerosol optical depthJournal of Geophysical Research: Atmospheres, 2013
- Image Denoising Using the Higher Order Singular Value DecompositionIEEE Transactions on Pattern Analysis and Machine Intelligence, 2012
- Reflectance measurement of canopy biomass and nitrogen status in wheat crops using normalized difference vegetation indices and partial least squares regressionRemote Sensing of Environment, 2003
- An Improved Algorithm for Computing the Singular Value DecompositionACM Transactions on Mathematical Software, 1982
- Numerical Methods for Computing Angles Between Linear SubspacesMathematics of Computation, 1973
- Singular value decomposition and least squares solutionsNumerische Mathematik, 1970