Context-Aware Hierarchical Online Learning for Performance Maximization in Mobile Crowdsourcing
- 2 May 2018
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE/ACM Transactions on Networking
- Vol. 26 (3) , 1334-1347
Abstract
In mobile crowdsourcing (MCS), mobile users accomplish outsourced human intelligence tasks. MCS requires an appropriate task assignment strategy, since different workers may have different performance in terms of acceptance rate and quality. Task assignment is challenging, since a worker's performance 1) may fluctuate, depending on both the worker's current personal context and the task context and 2) is not known a priori, but has to be learned over time. Moreover, learning context-specific worker performance requires access to context information, which may not be available at a central entity due to communication overhead or privacy concerns. In addition, evaluating worker performance might require costly quality assessments. In this paper, we propose a context-aware hierarchical online learning algorithm addressing the problem of performance maximization in MCS. In our algorithm, a local controller (LC) in the mobile device of a worker regularly observes the worker's context, her/his decisions to accept or decline tasks and the quality in completing tasks. Based on these observations, the LC regularly estimates the worker's context-specific performance. The mobile crowdsourcing platform (MCSP) then selects workers based on performance estimates received from the LCs. This hierarchical approach enables the LCs to learn context-specific worker performance and it enables the MCSP to select suitable workers. In addition, our algorithm preserves worker context locally, and it keeps the number of required quality assessments low. We prove that our algorithm converges to the optimal task assignment strategy. Moreover, the algorithm outperforms simpler task assignment strategies in experiments based on synthetic and real data.Keywords
All Related Versions
Funding Information
- German Research Foundation (DFG) within the Collaborative Research Center 1053–MAKI (B3)
- Scientific and Technological Research Council of Turkey under 3501 Program (116E229)
- ONR Mathematical Data Sciences Grant
- NSF (1524417, 1462245)
This publication has 29 references indexed in Scilit:
- Exploiting mobile crowdsourcing for pervasive cloud services: challenges and solutionsIEEE Communications Magazine, 2015
- Online decision making in crowdsourcing marketsACM SIGecom Exchanges, 2014
- Efficient crowdsourcing of unknown experts using bounded multi-armed banditsArtificial Intelligence, 2014
- Personalized task recommendation in crowdsourcing information systems — Current state of the artDecision Support Systems, 2014
- Next Steps for Citizen ScienceScience, 2014
- Distributed Online Learning in Social Recommender SystemsIEEE Journal of Selected Topics in Signal Processing, 2014
- Crowdsourcing systems on the World-Wide WebCommunications of the ACM, 2011
- Regret bounds for sleeping experts and banditsMachine Learning, 2010
- An approximate formula for a partial sum of the divergent -seriesApplied Mathematics Letters, 2009
- Probability Inequalities for Sums of Bounded Random VariablesJournal of the American Statistical Association, 1963