Using Feedback in Collaborative Reinforcement Learning to Adaptively Optimize MANET Routing
Authors:
- 18 April 2005
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans
- Vol. 35 (3) , 360-372
Abstract
Designers face many system optimization problems when building distributed systems. Traditionally, designers have relied on optimization techniques that require either prior knowledge or centrally managed runtime knowledge of the system's environment, but such techniques are not viable in dynamic networks where topology, resource, and node availability are subject to frequent and unpredictable change. To address this problem, we propose collaborative reinforcement learning (CRL) as a technique that enables groups of reinforcement learning agents to solve system optimization problems online in dynamic, decentralized networks. We evaluate an implementation of CRL in a routing protocol for mobile ad hoc networks, called SAMPLE. Simulation results show how feedback in the selection of links by routing agents enables SAMPLE to adapt and optimize its routing behavior to varying network conditions and properties, resulting in optimization of network throughput. In the experiments, SAMPLE displays emergent properties such as traffic flows that exploit stable routes and reroute around areas of wireless interference or congestion. SAMPLE is an example of a complex adaptive distributed system.Keywords
This publication has 17 references indexed in Scilit:
- Understanding link quality in 802.11 mobile ad hoc networksIEEE Internet Computing, 2004
- Collaborative reinforcement learning of autonomic behaviour2004
- Toward Self-Organizing, Self-Repairing and Resilient Distributed Systems2003
- Ant colony optimization: a new meta-heuristic2003
- Multiple Model-Based Reinforcement LearningNeural Computation, 2002
- The gray zone problem in IEEE 802.11b based ad hoc networksACM SIGMOBILE Mobile Computing and Communications Review, 2002
- Capacity of Ad Hoc wireless networks2001
- Autonomous agents for coordinated distributed parameterized heuristic routing in large dynamic communication networksJournal of Systems and Software, 2001
- Elevator Group Control Using Multiple Reinforcement Learning AgentsMachine Learning, 1998
- Prioritized sweeping: Reinforcement learning with less data and less timeMachine Learning, 1993