Adaptive call admission control under quality of service constraints: a reinforcement learning solution
Authors:
- T.X. BrownTBT.X. Brown
- 1 February 2000
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Journal on Selected Areas in Communications
- Vol. 18 (2) , 209-221
Abstract
We solve the adaptive call admission control (CAC) problem in multimedia networks via reinforcement learning (RL). The problem requires that network revenue be maximized while simultaneously meeting quality of service (QoS) constraints that forbid entry into certain states and use of certain actions. We show that RL provides a solution to this constrained semi-Markov decision problem and is able to earn significantly higher revenues than alternative heuristics. Unlike other model-based algorithms, RL does not require the explicit state transition models to solve the decision problems. This feature is very important if one considers large integrated service networks supporting a number of different service types, where the number of states is so large that model-based optimization algorithms are infeasible. Both packet-level and call-level QoS constraints are addressed, and both conservative and aggressive approaches to the QoS constraints are considered. Results are demonstrated on a single link and extended to routing on a multilink network.Keywords
This publication has 12 references indexed in Scilit:
- A neuro-dynamic programming approach to admission control in ATM networks: the single link case2002
- Adaptive call admission control under quality of service constraints: a reinforcement learning solutionIEEE Journal on Selected Areas in Communications, 2000
- Robust dynamic admission control for unified cell and call QoS in statistical multiplexersIEEE Journal on Selected Areas in Communications, 1998
- Wide area traffic: the failure of Poisson modelingIEEE/ACM Transactions on Networking, 1995
- Constrained Semi-Markov decision processes with average rewardsMathematical Methods of Operations Research, 1994
- Call admission and routing in multi-service loss networksIEEE Transactions on Communications, 1994
- Adaptive control of constrained Markov chainsIEEE Transactions on Automatic Control, 1991
- Markov decision algorithms for dynamic routing (telephone networks)IEEE Communications Magazine, 1990
- ATM communications network control by neural networksIEEE Transactions on Neural Networks, 1990
- Time-average optimal constrained semi-Markov decision processesAdvances in Applied Probability, 1986