Combination of online clustering and Q-value based GA for reinforcement fuzzy system design
- 13 June 2005
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Fuzzy Systems
- Vol. 13 (3) , 289-302
Abstract
This paper proposes a combination of online clustering and Q-value based genetic algorithm (GA) learning scheme for fuzzy system design (CQGAF) with reinforcements. The CQGAF fulfills GA-based fuzzy system design under reinforcement learning environment where only weak reinforcement signals such as "success" and "failure" are available. In CQGAF, there are no fuzzy rules initially. They are generated automatically. The precondition part of a fuzzy system is online constructed by an aligned clustering-based approach. By this clustering, a flexible partition is achieved. Then, the consequent part is designed by Q-value based genetic reinforcement learning. Each individual in the GA population encodes the consequent part parameters of a fuzzy system and is associated with a Q-value. The Q-value estimates the discounted cumulative reinforcement information performed by the individual and is used as a fitness value for GA evolution. At each time step, an individual is selected according to the Q-values, and then a corresponding fuzzy system is built and applied to the environment with a critic received. With this critic, Q-learning with eligibility trace is executed. After each trial, GA is performed to search for better consequent parameters based on the learned Q-values. Thus, in CQGAF, evolution is performed immediately after the end of one trial in contrast to general GA where many trials are performed before evolution. The feasibility of CQGAF is demonstrated through simulations in cart-pole balancing, magnetic levitation, and chaotic system control problems with only binary reinforcement signals.Keywords
This publication has 34 references indexed in Scilit:
- Fuzzy interpolation-based Q-learning with continuous states and actions2002
- Genetic reinforcement learning through symbiotic evolution for fuzzy controller designIEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), 2000
- GA-based fuzzy reinforcement learning for control of a magnetic bearing systemIEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), 2000
- Genetic Algorithms1999
- An online self-constructing neural fuzzy inference network and its applicationsIEEE Transactions on Fuzzy Systems, 1998
- An ART-based fuzzy adaptive learning control networkIEEE Transactions on Fuzzy Systems, 1997
- Simultaneous design of membership functions and rule sets for fuzzy controllers using genetic algorithmsIEEE Transactions on Fuzzy Systems, 1995
- Fuzzy Q-learning and dynamical fuzzy Q-learning1994
- A hybrid genetic algorithm for training neural networks1992
- Integrated Architectures for Learning, Planning, and Reacting Based on Approximating Dynamic Programming1990