Show simple item record

dc.contributor.author Chalmers, Eric
dc.contributor.author Luczak, Artur
dc.contributor.author Gruber, Aaron J.
dc.date.accessioned 2018-11-02T18:33:10Z
dc.date.available 2018-11-02T18:33:10Z
dc.date.issued 2016
dc.identifier.citation Chalmers, E., Luczak, A., & Gruber, A. J. (2016). Computational properties of the hippocampus increase the efficiency of goal-directed foraging through hierarchical reinforcement learning. Frontiers in Computational Neuroscience, 10(128). doi: 10.3389/fncom.2016.00128 en_US
dc.identifier.uri https://hdl.handle.net/10133/5234
dc.description Sherpa Romeo green journal; open access en_US
dc.description.abstract The mammalian brain is thought to use a version of Model-based Reinforcement Learning (MBRL) to guide “goal-directed” behavior, wherein animals consider goals and make plans to acquire desired outcomes. However, conventional MBRL algorithms do not fully explain animals’ ability to rapidly adapt to environmental changes, or learn multiple complex tasks. They also require extensive computation, suggesting that goal-directed behavior is cognitively expensive. We propose here that key features of processing in the hippocampus support a flexible MBRL mechanism for spatial navigation that is computationally efficient and can adapt quickly to change. We investigate this idea by implementing a computational MBRL framework that incorporates features inspired by computational properties of the hippocampus: a hierarchical representation of space, “forward sweeps” through future spatial trajectories, and context-driven remapping of place cells. We find that a hierarchical abstraction of space greatly reduces the computational load (mental effort) required for adaptation to changing environmental conditions, and allows efficient scaling to large problems. It also allows abstract knowledge gained at high levels to guide adaptation to new obstacles. Moreover, a context-driven remapping mechanism allows learning and memory of multiple tasks. Simulating dorsal or ventral hippocampal lesions in our computational framework qualitatively reproduces behavioral deficits observed in rodents with analogous lesions. The framework may thus embody key features of how the brain organizes model-based RL to efficiently solve navigation and other difficult tasks. en_US
dc.language.iso en_US en_US
dc.publisher Frontiers Media en_US
dc.subject Reinforcement learning en_US
dc.subject Hierarchical learning en_US
dc.subject Hippocampus en_US
dc.subject Planning en_US
dc.subject Context en_US
dc.subject Adaptation
dc.subject.lcsh Hippocampus (Brain)
dc.subject.lcsh Context effects (Psychology)
dc.subject.lcsh Reinforcement (Psychology)
dc.title Computational properties of the hippocampus increase the efficiency of goal-directed foraging through hierarchical reinforcement learning en_US
dc.type Article en_US
dc.publisher.faculty Arts and Science en_US
dc.publisher.department Department of Neuroscience en_US
dc.description.peer-review Yes en_US
dc.publisher.institution University of Lethbridge en_US


Files in this item

This item appears in the following Collection(s)

Show simple item record