It has been proposed that the brain integrates flexible, computationally expensive cortical processing with simpler, lower-cost subcortical mechanisms to achieve resource-efficient performance greater than that of either system alone. This alluring perspective is currently driving the development of theoretical frameworks that explore this hypothesis. I will present a normative framework for investigating this problem that combines three key features: (1) an environment with structure that can be leveraged to accelerate learning, (2) a nonstationary environment (i.e., changing reward contingencies), and (3) resource constraints (i.e. limited memory for the model). I will next present a simple example within this framework, showing how nonstationarity and memory constraints drive the cortical module to favor learning reward-agnostic information about the environment, and argue that this helps explain the separation of cortex and subcortical regions such as the basal ganglia, which are strongly reward-driven. Finally, I will introduce and analyze a specific mechanism by which information can be transferred from cortical to subcortical systems in the form of a Hebbian sequence-learning model.
