Bayesian Reinforcement Learning and Bayesian Deep Learning for Blockchains With Mobile Edge Computing
- 1 March 2021
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Cognitive Communications and Networking
- Vol. 7 (1) , 319-335
Abstract
We present a novel game-theoretic, Bayesian reinforce-ment learning (RL) and deep learning (DL) framework to represent interactions of miners in public and consortium blockchains with mobile edge computing (MEC). Within the framework, we formulate a stochastic game played by miners under incomplete information. Each miner can offload its block operations to one of the base stations (BSs) equipped with the MEC server. The miners select their offloading BSs and block processing rates simultaneously and independently, without informing other miners about their actions. As such, no miner knows the past and current actions of others and, hence, constructs its belief about these actions. Accordingly, we devise a Bayesian RL algorithm based on the partially-observable Markov decision process for miner’s decision making that allows each miner to dynamically adjust its strategy and update its beliefs through repeated interactions with each other and with the mobile environment. We also propose a novel unsupervised Bayesian deep learning algorithm where the uncertainties about unobservable states are approximated with Bayesian neural networks. We show that the proposed Bayesian RL and DL algorithms converge to the stable states where the miners’ actions and beliefs form the perfect Bayesian equilibrium (PBE) and myopic PBE, respectively.Keywords
Funding Information
- National Natural Science Foundation of China (61950410603)
This publication has 41 references indexed in Scilit:
- Secure Blockchains for Dynamic Spectrum Access: A Decentralized Database in Moving Cognitive Radio Networks Enhances Security and User AccessIEEE Vehicular Technology Magazine, 2018
- A survey on security and privacy issues of blockchain technologyMathematical Foundations of Computing, 2018
- Decentralized Consensus for Edge-Centric Internet of Things: A Review, Taxonomy, and Research IssuesIEEE Access, 2017
- A Model for Collaborative Blockchain-Based Video Delivery Relying on Advanced Network Services ChainsIEEE Communications Magazine, 2017
- Bayesian Reinforcement Learning-Based Coalition Formation for Distributed Resource Sharing by Device-to-Device Users in Heterogeneous Cellular NetworksIEEE Transactions on Wireless Communications, 2017
- The Bitcoin Mining GameLedger, 2016
- An Autonomous Learning-Based Algorithm for Joint Channel and Power Level Selection by D2D Pairs in Heterogeneous Cellular NetworksIEEE Transactions on Communications, 2016
- Bayesian Reinforcement Learning: A SurveyFoundations and Trends in Machine Learning, 2015
- An analytic solution to discrete Bayesian reinforcement learning2006
- A survey of mobility models for ad hoc network researchWireless Communications and Mobile Computing, 2002