Dopamine neurons encode reward prediction errors, used to update value predictions; the delivery of rewards is increased after a costly action, easing more rapid learning under high cost situation

The cost of obtaining rewards enhances the reward prediction error signal of midbrain dopamine neurons. Shingo Tanaka, John P. O’Doherty & Masamichi Sakagami. Nature Communications, volume 10, Article number: 3674 (2019). August 15 2019. https://www.nature.com/articles/s41467-019-11334-2

Abstract: Midbrain dopamine neurons are known to encode reward prediction errors (RPE) used to update value predictions. Here, we examine whether RPE signals coded by midbrain dopamine neurons are modulated by the cost paid to obtain rewards, by recording from dopamine neurons in awake behaving monkeys during performance of an effortful saccade task. Dopamine neuron responses to cues predicting reward and to the delivery of rewards were increased after the performance of a costly action compared to a less costly action, suggesting that RPEs are enhanced following the performance of a costly action. At the behavioral level, stimulus-reward associations are learned faster after performing a costly action compared to a less costly action. Thus, information about action cost is processed in the dopamine reward system in a manner that amplifies the following dopamine RPE signal, which in turn promotes more rapid learning under situations of high cost.

Introduction

Humans and animals prefer a reward received after exerting a lot effort to obtain it compared to the same reward after a smaller amount of effort1,2,3. A number of explanations have been posited for this effect such as effort justification4,5 and the contrast effect6, in which greater value is attributed to an outcome obtained after paid effort. However, it remains unclear whether and how the processing of reward information in the brain is modulated by the effort expended to obtain a reward.

We focused specifically on the midbrain dopamine system, given the role of this system in promoting behavioral adaptation to rewards7,8,9. Dopamine neurons are known to represent reward prediction error (RPE) signals that can facilitate learning of reward predictions by the basal ganglia10,11,12,13,14,15,16,17. The strength of the RPE depends on the quantity, quality, and subjective value or utility of the reward7,18,19,20,21. Moreover, dopaminergic activity is modulated by costs and/or effort22,23. On this basis, we postulated that the dopaminergic RPE signal would be directly modulated by the cost paid to obtain a reward. Furthermore, because the RPE signal is causally involved in mediating learning of stimulus-reward associations24,25,26, we hypothesized that the cost paid to obtain the reward would directly increase the learning speed of stimulus-reward associations.

To test our hypotheses, we measured both behavior and dopaminergic activity in two Japanese monkeys while they performed a saccade based effort task. Monkeys react faster to a reward-predicting cue that is presented after a high-cost (HC) action compared with that after a low-cost (LC) action. The activity of dopaminergic neurons to the reward-predicting cues are increased by the paid cost. In addition, learning speed to the stimulus-reward association is also enhanced by the paid cost. Therefore, we suggest that the cost paid to obtain rewards increases the RPE signal in dopamine neurons and thereby enhances stimulus-reward associations.