Alaa A. Ahmed

Movements become more vigorous when rewards are expected, suggesting that motivational signals influence motor control. Dopaminergic neurons, known to encode reward expectation and prediction error, are prime candidates for linking value and vigor. Here, we show that human reach vigor dynamically tracks canonical dopaminergic learning signals not only at movement onset but also during ongoing motion. Using a reaching task with probabilistic rewards (0, 33, 66, and 100%), we observed that peak velocity scaled with expected value. Crucially, following feedback, reach velocity was transiently invigorated or enervated in proportion to the sign and magnitude of the reward prediction error. Trial-by-trial changes in kinematics reflected value updating, consistent with dopaminergic phasic learning signals. These results demonstrate that movement vigor is modulated by reward-learning signals on rapid timescales, revealing a real-time behavioral readout of motivational computation in the brain.

In many motor learning tasks, the process of error reduction is mirrored in a process of effort reduction, where metabolic cost and or muscle co-contraction decrease gradually in tandem with error. Effort reduction may be incidental to the learning process, but high error movements can be more effortful and a drive to minimize effort costs could also play a role in motor learning. In this study, we focused on the effort requirements of the task, asking whether task (background) effort cost affects learning, aftereffects, or relearning in a split-belt walking task. We hypothesized that greater effort costs would amplify the need to reduce effort and accelerate learning. Alternatively, we hypothesized that greater effort requirement could compromise and slow the learning process. Participants in high, low, and control effort groups completed a split-belt walking task while wearing a weighted vest with 15% body mass, 5% body mass, or the vest only, and we assessed step length asymmetry throughout the protocol. Step length asymmetry changed similarly between groups, with similar rates and extent of learning and relearning and similar patterns of aftereffects when the split-belt perturbation was removed. We found no significant effect of task effort on the process of split-belt adaptation, suggesting that the brain’s response to gait asymmetry and ability to adapt to novel dynamics is relatively unchanged by background effort requirements of the task. NEW & NOTEWORTHY: Despite the brain’s sensitivity to effort cost and willingness to adjust gait parameters to minimize cost, the process of split-belt adaptation was unaffected by increased effort requirements. This finding provides a foundation for further research into performance-dependent effort cost. Additionally, modest increases in effort should be further explored in rehabilitation applications where higher effort requirements may help build strength and fitness without impairing motor learning.

Gripping actions are important for day-to-day tasks, for medical diagnostics like strength and force control, and for choice selection in decision-making experiments. Comparing isometric gripping responses to reward and nonreward cues, we observed reward-based invigoration mediated by selective increases in effort. These findings can be leveraged to provide additional insight into the decision making process and better understand the effect of reward on movement vigor.

Movement vigor across multiple modalities increases with reward, suggesting that the neural circuits that represent value influence the control of movement. Dopaminergic neuron (DAN) activity in the basal ganglia has been suggested as the potential mediator of this response. If DAN activity is the bridge between value and vigor, then vigor should track canonical mediators of this activity, namely reward expectation and reward prediction error. Here we ask if a similar time-locked response is present in vigor of reaching movements. We explore this link by leveraging the known phasic dopaminergic response to stochastic rewards, where activity is modulated by both reward expectation at cue and the prediction error at feedback. We used probabilistic rewards to create a reaching task rich in reward expectation, reward prediction error, and learning. In one experiment, target reward probabilities were explicitly stated, and in the other, were left unknown and to be learned by the participants. We included two stochastic rewards (probabilities 33% and 66%) and two deterministic ones (probabilities 100% and 0%). Outgoing peak velocity in both experiments increased with increasing reward expectation. Furthermore, we observed a short-latency response in the vigor of the ongoing movement, that tracked reward prediction error: either invigorating or enervating velocity consistent with the sign and magnitude of the error. Reaching kinematics also revealed the value-update process in a trial-to-trial fashion, similar to the effect of prediction error signals typical in dopamine-mediated striatal phasic activity. Lastly, reach vigor increased with reward history over trials, mirroring the motivational effects often linked to fluctuating dopamine levels. Taken together, our results demonstrate and exquisite link between known short-latency reward signals and the invigoration of both discrete and ongoing movements. NEW & NOTEWORTHY Previous research has demonstrated the invigorating effects of reward on movement. Growing evidence suggests this is causally explained by midbrain dopamine transients. Here, we demonstrate that reach vigor tracks canonical variables of learning and motivation across time scales ranging from milliseconds to minutes. Velocity was modulated by reward expectation, reward prediction error and reward rate, key variables that have also been associated with striatal dopaminergic fluctuations. These results point to a potential neural mechanism by which dopamine can influence both decision making and movement control and support the proposition that reward-based invigoration of movement is in part influenced by dopaminergic circuits.

We use a novel paradigm that differentiates physical effort costs, task costs, and time, where subjects choose between isometric arm-pushing tasks. Subjects prefer high physical effort earlier than later. They also decide faster and respond more vigorously the greater their preference. We find a generalizable subjective utility model that includes independent time-sensitivity of physical effort and task costs. Together, we demonstrate that subjective effort includes not only the total effort invested but also its timing.

Individuals exhibit a propensity to move faster toward more rewarding stimuli. While this phenomenon has been observed in movements, the effect of reward on implicit control of isometric actions, like gripping or grasping, is relatively unknown. How reward-related invigoration generalizes to other effortful actions is an important question. Reward invigorates reaching movements and saccades, supporting the idea that reward pays the additional effort cost of moving faster. Effort in isometric force generation is less understood, so here we ask whether and how reward-related invigoration generalizes to isometric force gripping. And if so, what implicit characteristics of gripping change when there is a prospect of reward? Participants (N=19) gripped a force transducer and the force applied was mapped to radial position of an onscreen cursor. Each trial, a target appeared in one of four locations; increasing grip force moved the cursor toward the target. The gripping action was interchangeable for all target positions. In each block of 100 trials, one target was consistently rewarded, while the other targets were not. When gripping to acquire the rewarded target, participants reacted faster, generated force more rapidly and to a greater extent, while intriguingly maintaining the same accuracy and integral of force over time. These findings support the generalization of reward-related invigoration in isometric force tasks, and that the brain exquisitely trades-off reward and effort costs to obtain reward more rapidly without compromising accuracy or more effort costs than necessary. NEW & NOTEWORTHY: Gripping actions are important for day-to-day tasks, for medical diagnostics like strength and force control, and for choice selection in decision-making experiments. Comparing isometric gripping responses to reward and nonreward cues, we observed reward-based invigoration mediated by selective increases in effort. These findings can be leveraged to provide additional insight into the decision making process, and better understand the effect of reward on movement vigor and the implicit control of accuracy.

On any given day, we make countless reaching movements to objects around us. While such ubiquity may suggest uniformity, each movement’s speed is unique—why is this? Reach speed is known to be influenced by accuracy; we slow down to sustain high accuracy. However, in other forms of movement like walking or running, metabolic cost is often the primary determinant of movement speed. Here we bridge this gap and ask: how do metabolic cost and accuracy interact to determine speed of reaching movements? First, we systematically measure the effect of increasing mass on the metabolic cost of reaching across a range of movement speeds. Next, in a sequence of three experiments, we examine how added mass affects preferred reaching speed across changing accuracy requirements. We find that, while added mass consistently increases metabolic cost thereby leading to slower metabolically optimal movement speeds, self-selected reach speeds are slower than those predicted by an optimization of metabolic cost alone. We then demonstrate how a single model that considers both accuracy and metabolic costs can explain preferred movement speeds. Together, our findings provide a unifying framework to illuminate the combined effects of metabolic cost and accuracy on movement speed and highlight the integral role metabolic cost plays in determining reach speed.

Much of life is spent foraging. Whereas previous work has focused on the decision regarding time spent harvesting from a reward patch, here we test the idea that both decision making and movement control are tuned to optimize the net rate of reward in an environment. Our results show that movement patterns reflect not just immediate expectations but also past experiences in the environment, providing fundamental insight into the factors governing volitional control of arm movements.

Why do we move slower as we grow older? The reward circuits of the brain, which tend to invigorate movements, decline with aging, raising the possibility that reduced vigor is due to the diminishing value that our brain assigns to movements. However, as we grow older, it also becomes more effortful to make movements. Is age-related slowing principally a consequence of increased effort costs from the muscles, or reduced valuation of reward by the brain? Here, we first quantified the cost of reaching via metabolic energy expenditure in human participants (male and female), and found that older adults consumed more energy than the young at a given speed. Thus, movements are objectively more costly for older adults. Next, we observed that when reward increased, older adults, like the young, responded by initiating their movements earlier. Yet, unlike the young, they were unwilling to increase their movement speed. Was their reluctance to reach quicker for rewards due to the increased effort costs, or because they ascribed less value to the movement? Motivated by a mathematical model, we next made the young experience a component of aging by making their movements more effortful. Now the young responded to reward by reacting faster but chose not to increase their movement speed. This suggests that slower movements in older adults are partly driven by an adaptive response to an elevated effort landscape. Moving slower may be a rational economic response the brain is making to mitigate the elevated effort costs that accompany aging.

Rapid progress in our understanding of the brain's learning mechanisms has been accomplished over the past decade, particularly with conceptual advances, including representing behavior as a dynamical system, large-scale neural population recordings, and new methods of analysis of neuronal populations. However, motor and cognitive systems have been traditionally studied with different methods and paradigms. Recently, some common principles, evident in both behavior and neural activity, that underlie these different types of learning have become to emerge. Here we review results from motor and cognitive learning, relying on different techniques and studying different systems to understand the mechanisms of learning. Movement is intertwined with cognitive operations, and its dynamics reflect cognitive variables. Training, in either motor or cognitive tasks, involves recruitment of previously unresponsive neurons and reorganization of neural activity in a low dimensional manifold. Mapping of new variables in neural activity can be very rapid, instantiating flexible learning of new tasks. Communication between areas is just as critical a part of learning as are patterns of activity within an area emerging with learning. Common principles across systems provide a map for future research.