DeepSensing: A Novel Mobile Crowdsensing Framework With Double Deep Q-Network and Prioritized Experience Replay
- 8 September 2020
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Internet of Things Journal
- Vol. 7 (12) , 11547-11558
Abstract
Mobile crowdsensing (MCS) is a new and promising paradigm of data collection due to the growing number of mobile smart devices. It can be utilized in applications of large-scale sensing by employing a group of mobile users with their smart devices. Since a large number of mobile users are recruited, the allocation of sensing tasks to mobile users has a critical influence on the performance of MCS applications. To efficiently assign sensing tasks to mobile users, we propose a novel MCS framework named DeepSensing. This framework consists of six executive phases, i.e., registration of sensing tasks, announcement of reward rule, collection of users’ information, task allocation, execution of sensing activities, and distribution of data and rewards. Here, the phase of task allocation is a key component, which directly determines the performance of DeepSensing, e.g., the platform’s profit. DeepSensing aims to maximize the platform’s profit by taking into account the various constraints of sensing tasks and mobile users. Therefore, we propose a deep reinforcement learning (DRL) method to optimally assign sensing tasks to mobile users. Specifically, we employ a double deep Q-network with prioritized experience replay (DDQN-PER) to address the task allocation problem, which is also formulated as a path planning problem with time windows. To evaluate our proposed DDQN-PER solution, three baseline solutions are provided, i.e., ant colony system (ACS), -greedy, and random solutions. Finally, the results of numerical simulations show that our proposed DDQN-PER solution outperforms the baseline solutions in terms of the platform’s profit and it plans better-organized travelling paths for mobile users.Keywords
Funding Information
- Natural Sciences and Engineering Research Council of Canada
- Fonds de Recherche du Québec—Nature et Technologies
This publication has 41 references indexed in Scilit:
- Distributed Time-Sensitive Task Selection in Mobile Crowdsensing2015
- EEMCACM Transactions on Intelligent Systems and Technology, 2015
- A Survey of Incentive Techniques for Mobile Crowd SensingIEEE Internet of Things Journal, 2015
- Human-level control through deep reinforcement learningNature, 2015
- A Survey of Incentive Mechanisms for Participatory SensingIEEE Communications Surveys & Tutorials, 2015
- EMC3: Energy-efficient data transfer in mobile crowdsensing under full coverage constraintIEEE Transactions on Mobile Computing, 2014
- Stable signal recovery from incomplete and inaccurate measurementsCommunications on Pure and Applied Mathematics, 2006
- Ant colony system: a cooperative learning approach to the traveling salesman problemIEEE Transactions on Evolutionary Computation, 1997
- The Orienteering Problem with Time WindowsJournal of the Operational Research Society, 1992
- Q-learningMachine Learning, 1992