Efficiency of Extreme Gradient Boosting for Imbalanced Land Cover Classification Using an Extended Margin and Disagreement Performance
Open Access
- 23 July 2019
- journal article
- research article
- Published by MDPI AG in ISPRS International Journal of Geo-Information
- Vol. 8 (7) , 315
Abstract
Imbalanced learning is a methodological challenge in remote sensing communities, especially in complex areas where the spectral similarity exists between land covers. Obtaining high-confidence classification results for imbalanced class issues is highly important in practice. In this paper, extreme gradient boosting (XGB), a novel tree-based ensemble system, is employed to classify the land cover types in Very-high resolution (VHR) images with imbalanced training data. We introduce an extended margin criterion and disagreement performance to evaluate the efficiency of XGB in imbalanced learning situations and examine the effect of minority class spectral separability on model performance. The results suggest that the uncertainty of XGB associated with correct classification is stable. The average probability-based margin of correct classification provided by XGB is 0.82, which is about 46.30% higher than that by random forest (RF) method (0.56). Moreover, the performance uncertainty of XGB is insensitive to spectral separability after the sample imbalance reached a certain level (minority:majority > 10:100). The impact of sample imbalance on the minority class is also related to its spectral separability, and XGB performs better than RF in terms of user accuracy for the minority class with imperfect separability. The disagreement components of XGB are better and more stable than RF with imbalanced samples, especially for complex areas with more types. In addition, appropriate sample imbalance helps to improve the trade-off between the recognition accuracy of XGB and the sample cost. According to our analysis, this margin-based uncertainty assessment and disagreement performance can help users identify the confidence level and error component in similar classification performance (overall, producer, and user accuracies).Keywords
Funding Information
- National Key Research & Development (R&D) Plan of China (2017YFB0503600)
- National Natural Science Foundation of China (41674100)
This publication has 54 references indexed in Scilit:
- Cost-sensitive decision tree ensembles for effective imbalanced classificationApplied Soft Computing, 2014
- An insight into classification with imbalanced data: Empirical results and current trends on using data intrinsic characteristicsInformation Sciences, 2013
- Landslide Susceptibility Assessment in Vietnam Using Support Vector Machines, Decision Tree, and Naïve Bayes ModelsMathematical Problems in Engineering, 2012
- An assessment of the effectiveness of a random forest classifier for land-cover classificationISPRS Journal of Photogrammetry and Remote Sensing, 2012
- Death to Kappa: birth of quantity disagreement and allocation disagreement for accuracy assessmentInternational Journal of Remote Sensing, 2011
- Training set size requirements for the classification of a specific classRemote Sensing of Environment, 2006
- A Multiple Resampling Method for Learning from Imbalanced Data SetsComputational Intelligence, 2004
- An assessment of the effectiveness of decision tree methods for land cover classificationRemote Sensing of Environment, 2003
- Classification and Change Detection Using Landsat TM DataRemote Sensing of Environment, 2001
- Classification of imbalanced remote-sensing data by neural networksPattern Recognition Letters, 1997