Increasing the Accuracy of Crowdsourced Information on Land Cover via a Voting Procedure Weighted by Information Inferred from the Contributed Data
Open Access
- 25 February 2018
- journal article
- research article
- Published by MDPI AG in ISPRS International Journal of Geo-Information
- Vol. 7 (3) , 80
Abstract
Simple consensus methods are often used in crowdsourcing studies to label cases when data are provided by multiple contributors. A basic majority vote rule is often used. This approach weights the contributions from each contributor equally but the contributors may vary in the accuracy with which they can label cases. Here, the potential to increase the accuracy of crowdsourced data on land cover identified from satellite remote sensor images through the use of weighted voting strategies is explored. Critically, the information used to weight contributions based on the accuracy with which a contributor labels cases of a class and the relative abundance of class are inferred entirely from the contributed data only via a latent class analysis. The results show that consensus approaches do yield a classification that is more accurate than that achieved by any individual contributor. Here, the most accurate individual could classify the data with an accuracy of 73.91% while a basic consensus label derived from the data provided by all seven volunteers contributing data was 76.58%. More importantly, the results show that weighting contributions can lead to a statistically significant increase in the overall accuracy to 80.60% by ignoring the contributions from the volunteer adjudged to be the least accurate in labelling.Keywords
Funding Information
- Horizon 2020 Framework Programme (689812)
- European Cooperation in Science and Technology (TD1202)
This publication has 30 references indexed in Scilit:
- Comparing the Quality of Crowdsourced Data Contributed by Expert and Non-ExpertsPLOS ONE, 2013
- Collaborative damage mapping for emergency response: the role of Cognitive Systems EngineeringNatural Hazards and Earth System Sciences, 2013
- Next-generation Digital EarthProceedings of the National Academy of Sciences, 2012
- Assuring the quality of volunteered geographic informationSpatial Statistics, 2012
- Geo-Wiki: An online platform for improving global land coverEnvironmental Modelling & Software, 2012
- THE MYTH OF AMATEUR CROWDSInformation, Communication & Society, 2011
- Assessing the accuracy of land cover change with imperfect ground reference dataRemote Sensing of Environment, 2010
- Crowdsourcing geographic information for disaster response: a research frontierInternational Journal of Digital Earth, 2010
- Latent class models for classificationComputational Statistics & Data Analysis, 2003
- Unskilled and unaware of it: How difficulties in recognizing one's own incompetence lead to inflated self-assessments.Journal of Personality and Social Psychology, 1999