Gruber, Aaron
Permanent URI for this collection
Browse
Recent Submissions
- ItemComputational properties of the hippocampus increase the efficiency of goal-directed foraging through hierarchical reinforcement learning(Frontiers Media, 2016) Chalmers, Eric; Luczak, Artur; Gruber, Aaron J.The mammalian brain is thought to use a version of Model-based Reinforcement Learning (MBRL) to guide “goal-directed” behavior, wherein animals consider goals and make plans to acquire desired outcomes. However, conventional MBRL algorithms do not fully explain animals’ ability to rapidly adapt to environmental changes, or learn multiple complex tasks. They also require extensive computation, suggesting that goal-directed behavior is cognitively expensive. We propose here that key features of processing in the hippocampus support a flexible MBRL mechanism for spatial navigation that is computationally efficient and can adapt quickly to change. We investigate this idea by implementing a computational MBRL framework that incorporates features inspired by computational properties of the hippocampus: a hierarchical representation of space, “forward sweeps” through future spatial trajectories, and context-driven remapping of place cells. We find that a hierarchical abstraction of space greatly reduces the computational load (mental effort) required for adaptation to changing environmental conditions, and allows efficient scaling to large problems. It also allows abstract knowledge gained at high levels to guide adaptation to new obstacles. Moreover, a context-driven remapping mechanism allows learning and memory of multiple tasks. Simulating dorsal or ventral hippocampal lesions in our computational framework qualitatively reproduces behavioral deficits observed in rodents with analogous lesions. The framework may thus embody key features of how the brain organizes model-based RL to efficiently solve navigation and other difficult tasks.
- ItemLose-shift responding in humans is promoted by increased cognitive load(Frontiers Media, 2018) Ivan, Victorita E.; Banks, Parker J.; Goodfellow, Kris; Gruber, Aaron J.The propensity of animals to shift choices immediately after unexpectedly poor reinforcement outcomes is a pervasive strategy across species and tasks. We report here on the memory supporting such lose-shift responding in humans, assessed using a binary choice task in which random responding is the optimal strategy. Participants exhibited little lose-shift responding when fully attending to the task, but this increased by 30%–40% in participants that performed with additional cognitive load that is known to tax executive systems. Lose-shift responding in the cognitively loaded adults persisted throughout the testing session, despite being a sub-optimal strategy, but was less likely as the time increased between reinforcement and the subsequent choice. Furthermore, children (5–9 years old) without load performed similarly to the cognitively loaded adults. This effect disappeared in older children aged 11–13 years old. These data provide evidence supporting our hypothesis that lose-shift responding is a default and reflexive strategy in the mammalian brain, likely mediated by a decaying memory trace, and is normally suppressed by executive systems. Reducing the efficacy of executive control by cognitive load (adults) or underdevelopment (children) increases its prevalence. It may therefore be an important component to consider when interpreting choice data, and may serve as an objective behavioral assay of executive function in humans that is easy to measure.
- ItemContext, emotion, and the strategic pursuit of goals: interactions among multiple brain systems controlling motivated behavior(Frontiers Media, 2012) Gruber, Aaron J.; McDonald, Robert J.Motivated behavior exhibits properties that change with experience and partially dissociate among a number of brain structures. Here, we review evidence from rodent experiments demonstrating that multiple brain systems acquire information in parallel and either cooperate or compete for behavioral control. We propose a conceptual model of systems interaction wherein a ventral emotional memory network involving ventral striatum (VS), amygdala, ventral hippocampus, and ventromedial prefrontal cortex triages behavioral responding to stimuli according to their associated affective outcomes. This system engages autonomic and postural responding (avoiding, ignoring, approaching) in accordance with associated stimulus valence (negative, neutral, positive), but does not engage particular operant responses. Rather, this emotional system suppresses or invigorates actions that are selected through competition between goal-directed control involving dorsomedial striatum (DMS) and habitual control involving dorsolateral striatum (DLS). The hippocampus provides contextual specificityto the emotional system, and provides an information rich input to the goal-directed system for navigation and discriminations involving ambiguous contexts, complex sensory configurations, or temporal ordering. The rapid acquisition and high capacity for episodic associations in the emotional system may unburden the more complex goal-directed system and reduce interference in the habit system from processing contingencies of neutral stimuli. Interactions among these systems likely involve inhibitory mechanisms and neuromodulation in the striatum to form a dominant response strategy. Innate traits, training methods, and task demands contribute to the nature of these interactions, which can include incidental learning in non-dominant systems. Addition of these features to reinforcement learning models of decision-makingmay better aligntheoretical predictions with behavioral and neural correlates in animals.
- ItemAcute NMDA receptor antagonism disrupts synchronization of action potential firing in rat prefrontal cortex(Public Library of Science, 2014) Molina, Leonardo A.; Skelin, Ivan; Gruber, Aaron J.Antagonists of N-methyl-D-aspartate receptors (NMDAR) have psychotomimetic effects in humans and are used to model schizophrenia in animals. We used high-density electrophysiological recordings to assess the effects of acute systemic injection of an NMDAR antagonist (MK-801) on ensemble neural processing in the medial prefrontal cortex of freely moving rats. Although MK-801 increased neuron firing rates and the amplitude of gamma-frequency oscillations in field potentials, the synchronization of action potential firing decreased and spike trains became more Poisson-like. This disorganization of action potential firing following MK-801 administration is consistent with changes in simulated cortical networks as the functional connections among pyramidal neurons become less clustered. Such loss of functional heterogeneity of the cortical microcircuit may disrupt information processing dependent on spike timing or the activation of discrete cortical neural ensembles, and thereby contribute to hallucinations and other features of psychosis induced by NMDAR antagonists.