file.pdf (456.05 kB)

Neural Correlates of Temporal Credit Assignment

Download (456.05 kB)
journal contribution
posted on 2010-08-01, 00:00 authored by Matthew M. Walsh, John R. Anderson

When feedback follows a sequence of decisions, how do people assign credit to intermediate actions within the sequence? To explore this temporal credit assignment problem, we recorded event-related potentials (ERPs) as participants performed a sequential decision task. Our ERP analyses focused on feedback-related negativity (FRN), a component thought to reflect neural reward prediction error. The experiment showed that FRN followed negative feedback and negative intermediate states. This outcome suggests that participants evaluated intermediate states in terms of expected future reward, and that these evaluations guided acquisition of earlier actions within sequences. We compared these results to the predictions of three reinforcement learning models that address temporal credit assignment: Actor-critic, Q-Learning, and SARSA.