Algorithmic Management for Improving Collective Productivity in Crowdsourcing
Open Access
- 2 October 2017
- journal article
- research article
- Published by Springer Nature in Scientific Reports
- Vol. 7 (1) , 1-11
Abstract
Crowdsourcing systems are complex not only because of the huge number of potential strategies for assigning workers to tasks, but also due to the dynamic characteristics associated with workers. Maximizing social welfare in such situations is known to be NP-hard. To address these fundamental challenges, we propose the surprise-minimization-value-maximization (SMVM) approach. By analysing typical crowdsourcing system dynamics, we established a simple and novelworker desirability index(WDI) jointly considering the effect of each worker’s reputation, workload and motivation to work on collective productivity. Through evaluating workers’ WDI values, SMVM influences individual workers in real time about courses of action which can benefit the workers and lead to high collective productivity. Solutions can be produced in polynomial time and are proven to be asymptotically bounded by a theoretical optimal solution. High resolution simulations based on a real-world dataset demonstrate that SMVM significantly outperforms state-of-the-art approaches. A large-scale 3-year empirical study involving 1,144 participants in over 9,000 sessions shows that SMVM outperforms human task delegation decisions over 80% of the time under common workload conditions. The approach and results can help engineer highly scalable data-driven algorithmic management decision support systems for crowdsourcing.Keywords
This publication has 45 references indexed in Scilit:
- Reputation can enhance or suppress cooperation through positive feedbackNature Communications, 2015
- Crowd science user contribution patterns and their implicationsProceedings of the National Academy of Sciences, 2015
- Filtering trust opinions through reinforcement learningDecision Support Systems, 2014
- Efficient crowdsourcing of unknown experts using bounded multi-armed banditsArtificial Intelligence, 2014
- Citizen Science Provides Valuable Data for Monitoring Global Night Sky LuminanceScientific Reports, 2013
- Crystal structure of a monomeric retroviral protease solved by protein folding game playersNature Structural & Molecular Biology, 2011
- The complexity of welfare maximization in congestion gamesNetworks, 2011
- Analyzing the Amazon Mechanical Turk marketplaceXRDS: Crossroads, The ACM Magazine for Students, 2010
- Automated Negotiation through a Cooperative-Competitive Model2010
- reCAPTCHA: Human-Based Character Recognition via Web Security MeasuresScience, 2008