The role of the cerebellum in reinforcement learning

How do we learn to establish associations between arbitrary visual cues (like a red light) and movements (like braking the car)? We investigated the neural correlates of visuomotor association learning in the mid-lateral cerebellum. Although cerebellum has been considered to be a motor control center involved in monitoring and correcting the motor error through supervised learning, in this thesis, we show that its role can also be extended to non-motor learning. Specifically, when primates learned to associate arbitrary visual cues with well-learned stereotypic movements, the simple spikes of the mid-lateral cerebellar Purkinje cells reported the monkey’s most recent decision’s outcome during learning. The magnitude of this reinforcement error signal changed with learning, finally disappearing when the association had been overlearned. We modeled this change in neural activity through a drift diffusion-reinforcement learning based model. The concurrent complex spikes, contrary to traditional theories, did not play the role of teaching signal, but encoded the probability of error as a function of the state of learning. They also encoded features that indicate the beginning of a trial. Inactivating the mid-lateral cerebellum significantly affected the monkey’s learning performance while it did not affect motor performance. This is because the mid-lateral cerebellum is in a loop with other cognitive processing centers of the brain including the prefrontal cortex and the basal ganglia. Finally, we verified that the features we identified in primate experiments can also be extended to humans, by studying the visuomotor association learning in humans through functional magnetic resonance imaging. In summary, through electrophysiological and causal experiments in monkeys, imaging in humans, computational models and an anatomical framework, we delineate mechanisms through which the cerebellum can be involved in reinforcement learning and specifically, learning new visuomotor associations.