Adaptive Image-Based Visual Servoing Using Reinforcement Learning With Fuzzy State Coding
- 29 April 2020
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Fuzzy Systems
- Vol. 28 (12) , 3244-3255
Abstract
Image-Based Visual Servoing (IBVS) allows precise control of positioning and motion for relatively stationary targets using visual feedback. For IBVS, a mixture parameter β allows better approximation of the image Jacobian matrix, which has a significant effect on the performance of IBVS. However, the setting for the mixture parameter depends on the camera's real-time posture, there is no clear way to define the change rules for most IBVS applications. Using simple model-free reinforcement learning (RL), Q-Learning, this study proposes a method to adjust the image Jacobian matrix for IBVS adaptively. If the state space is discretized, traditional Q-learning encounters problems with the resolution that can cause sudden changes in the action, so the visual servoing system performs poorly. Besides, a robot in a real-world environment also cannot learn on as large a scale as virtual agents, so the efficiency with which agents learn must be increased. This study proposes a method that uses Fuzzy State Coding to accelerate learning during the training phase and to produce a smooth output in the application phase of the learning experience. A method that compensates for delay also allows more accurate extraction of features in a real environment. The results for simulation and experiment demonstrate that the proposed method performs better than other methods, in terms of learning speed, movement trajectory, and convergence time.Keywords
Funding Information
- National Natural Science Foundation of China (61976178, 61572339)
This publication has 33 references indexed in Scilit:
- Image Jacobian Matrix Estimation Based on Online Support Vector RegressionInternational Journal of Advanced Robotic Systems, 2012
- Continuous state/action reinforcement learning: A growing self-organizing map approachNeurocomputing, 2011
- Robust Jacobian matrix estimation for image-based visual servoingRobotics and Computer-Integrated Manufacturing, 2011
- A hybrid neural network and ARIMA model for water quality time series predictionEngineering Applications of Artificial Intelligence, 2010
- Online estimation of Image Jacobian Matrix for uncalibrated dynamic hand-eye coordinationInternational Journal of Systems, Control and Communications, 2008
- Reinforcement learning in multi-dimensional state-action space using random rectangular coarse coding and Gibbs sampling2007
- Visual Servoing by Optimization of a 2D/3D Hybrid Objective FunctionProceedings 2001 ICRA. IEEE International Conference on Robotics and Automation (cat. No.01ch37164), 2007
- Function Approximation via Tile Coding: Automating Parameter Choice2005
- Reinforcement Learning: An IntroductionIEEE Transactions on Neural Networks, 1998
- CMAC with General Basis FunctionsNeural Networks, 1996