Improved Deep Q-network for Mobile Charging Sequence Scheduling with the Minimal Mobile Charger Energy Capacity in Wireless Rechargeable Sensor Network
Preprint
- 21 February 2023
- preprint
- Published by Springer Nature in Research Square
Abstract
Wireless rechargeable sensor network (WRSN) is a new type of wireless sensor networks (WSNs) with rechargeable sensor nodes, which provides a feasible way to overcome the energy constraint prob- lem in WSN. Mobile charging is promising for WRSN by charging sensors using the mobile charger (MC), usually via wireless energy transfer (WET). However, it is challenging to maintain the network working sustainably under the limited capacity of the MC, and most existing works assume that the capacity of MC is infinite or with a very large value, which may be impossible in many real applica- tions. This paper will study the mobile charging sequence scheduling problem to maintain the network working sustainably and minimize the energy capacity of the MC (CSSEC). Specifically, the problem of minimizing the energy capacity of MC is coupled with mobile charging sequence scheduling, and the charging sequence will directly affect the energy cost of MC during a charging tour. Therefore, we propose an improved deep Q-network for CSSEC (IDQN-CSSEC) to address this issue. It employs the MC as the agent to explore the WRSN, and determines a charging policy for MC based on the charging demand of sensors, which selects the next sensor for charging. IDQN-CSSEC uses the Q learning and deep Q-network (DQN) comprehensively, which trains the Q table of the Q learning and the networks of DQN in the early stage of the training, and updates them simultaneously. The action determined by Q learning is selected with a greater probability, which decreases as the iteration step increases. Simu- lation experiments verify that the charging policy determined by IDQN-CSSEC can make the network work sustainably with the minimal energy cost of the MC by comparing it with the baseline methods.Keywords
All Related Versions
This publication has 21 references indexed in Scilit:
- Multistep Prediction-Based Adaptive Dynamic Programming Sensor Scheduling Approach for Collaborative Target Tracking in Energy Harvesting Wireless Sensor NetworksIEEE Transactions on Automation Science and Engineering, 2020
- A Newborn Particle Swarm Optimization Algorithm for Charging-Scheduling Algorithm in Industrial Rechargeable Sensor NetworksIEEE Sensors Journal, 2020
- Energy Efficient Two-Tier Data Dissemination Based on Q-Learning for Wireless Sensor NetworksIEEE Access, 2020
- Efficient Mobile Energy Replenishment Scheme Based on Hybrid Mode for Wireless Rechargeable Sensor NetworksIEEE Sensors Journal, 2019
- An efficient scheduling scheme for mobile charger in on-demand wireless rechargeable sensor networksJournal of Network and Computer Applications, 2018
- Wireless Sensor’s Civil Applications, Prototypes, and Future Integration Possibilities: A ReviewIEEE Sensors Journal, 2017
- A Critical Analysis of Research Potential, Challenges, and Future Directives in Industrial Wireless Sensor NetworksIEEE Communications Surveys & Tutorials, 2017
- Minimizing the number of mobile chargers for large-scale wireless rechargeable sensor networksComputer Communications, 2014
- Constant-approximation for target coverage problem in wireless sensor networks2012
- Building-Environment Control With Wireless Sensor and Actuator Networks: Centralized Versus DistributedIEEE Transactions on Industrial Electronics, 2009