Computational properties of the hippocampus increase the efficiency of goal-directed foraging through hierarchical reinforcement learning

Simple item page
dc.contributor.authorChalmers, Eric
dc.contributor.authorLuczak, Artur
dc.contributor.authorGruber, Aaron J.
dc.date.accessioned2018-11-02T18:33:10Z
dc.date.available2018-11-02T18:33:10Z
dc.date.issued2016
dc.descriptionSherpa Romeo green journal; open accessen_US
dc.description.abstractThe mammalian brain is thought to use a version of Model-based Reinforcement Learning (MBRL) to guide “goal-directed” behavior, wherein animals consider goals and make plans to acquire desired outcomes. However, conventional MBRL algorithms do not fully explain animals’ ability to rapidly adapt to environmental changes, or learn multiple complex tasks. They also require extensive computation, suggesting that goal-directed behavior is cognitively expensive. We propose here that key features of processing in the hippocampus support a flexible MBRL mechanism for spatial navigation that is computationally efficient and can adapt quickly to change. We investigate this idea by implementing a computational MBRL framework that incorporates features inspired by computational properties of the hippocampus: a hierarchical representation of space, “forward sweeps” through future spatial trajectories, and context-driven remapping of place cells. We find that a hierarchical abstraction of space greatly reduces the computational load (mental effort) required for adaptation to changing environmental conditions, and allows efficient scaling to large problems. It also allows abstract knowledge gained at high levels to guide adaptation to new obstacles. Moreover, a context-driven remapping mechanism allows learning and memory of multiple tasks. Simulating dorsal or ventral hippocampal lesions in our computational framework qualitatively reproduces behavioral deficits observed in rodents with analogous lesions. The framework may thus embody key features of how the brain organizes model-based RL to efficiently solve navigation and other difficult tasks.en_US
dc.description.peer-reviewYesen_US
dc.identifier.citationChalmers, E., Luczak, A., & Gruber, A. J. (2016). Computational properties of the hippocampus increase the efficiency of goal-directed foraging through hierarchical reinforcement learning. Frontiers in Computational Neuroscience, 10(128). doi: 10.3389/fncom.2016.00128en_US
dc.identifier.urihttps://hdl.handle.net/10133/5234
dc.language.isoen_USen_US
dc.publisherFrontiers Mediaen_US
dc.publisher.departmentDepartment of Neuroscienceen_US
dc.publisher.facultyArts and Scienceen_US
dc.publisher.institutionUniversity of Lethbridgeen_US
dc.subjectReinforcement learningen_US
dc.subjectHierarchical learningen_US
dc.subjectHippocampusen_US
dc.subjectPlanningen_US
dc.subjectContexten_US
dc.subjectAdaptation
dc.subject.lcshHippocampus (Brain)
dc.subject.lcshContext effects (Psychology)
dc.subject.lcshReinforcement (Psychology)
dc.titleComputational properties of the hippocampus increase the efficiency of goal-directed foraging through hierarchical reinforcement learningen_US
dc.typeArticleen_US
Files
Original bundle
Now showing 1 - 1 of 1
Thumbnail Image
Name:
Luczak computational properties of the hippocampus.pdf
Size:
2.09 MB
Format:
Adobe Portable Document Format
Description:
Download
License bundle
Now showing 1 - 1 of 1
Thumbnail Image
Name:
license.txt
Size:
1.75 KB
Format:
Item-specific license agreed upon to submission
Description:
Download
Collections
Luczak, Artur
Gruber, Aaron