Policy-Gradient Reinforcement Learning as a General Theory of Practice-Based Motor Skill Learning
Preprint
Abstract
Mastering any new skill requires extensive practice, but the computational principles underlying this learning are not clearly understood. Existing theories of motor learning can explain short-term adaptation to perturbations, but offer little insight into the processes that drive gradual skill improvement through practice. Here, we propose that practice-based motor skill learning can be understood as a form of reinforcement learning (RL), specifically, policy-gradient RL, a simple, model-free method that is widely used in robotics and other continuous control settings. Here, we show that models based on policy-gradient learning rules capture key properties of human skill learning across a diverse range of learning tasks that have previously lacked any computational theory. We suggest that policy-gradient RL can provide a general theoretical framework and foundation for understanding how humans hone skills through practice.Keywords
All Related Versions
This publication has 48 references indexed in Scilit:
- Motor Learning2019
- The multiple effects of practice: skill, habit and reduced cognitive loadCurrent Opinion in Behavioral Sciences, 2018
- The effects of reward and punishment on motor skill learningCurrent Opinion in Behavioral Sciences, 2018
- It's not (only) the mean that matters: variability, noise and exploration in skill learningCurrent Opinion in Behavioral Sciences, 2018
- Acquisition of Novel and Complex Motor Skills: Stable Solutions Where Intrinsic Noise Matters Less2014
- Model-Based and Model-Free Mechanisms of Human Motor Learning2013
- Reward Improves Long-Term Retention of a Motor Memory through Induction of Offline Memory GainsCurrent Biology, 2011
- Variability in motor learning: relocating, channeling and reducing noiseExperimental Brain Research, 2008
- Forward models: Supervised learning with a distal teacherCognitive Science, 1992
- Simple statistical gradient-following algorithms for connectionist reinforcement learningMachine Learning, 1992