Grid cells, place cells, and geodesic generalization for spatial reinforcement learning
PLoS Computational Biology, ISSN: 1553-7358, Vol: 7, Issue: 10, Page: e1002235
2011
- 52Citations
- 213Captures
- 1Mentions
Metric Options: Counts1 Year3 YearSelecting the 1-year or 3-year option will change the metrics count to percentiles, illustrating how an article or review compares to other articles or reviews within the selected time period in the same journal. Selecting the 1-year option compares the metrics against other articles/reviews that were also published in the same calendar year. Selecting the 3-year option compares the metrics against other articles/reviews that were also published in the same calendar year plus the two years prior.
Example: if you select the 1-year option for an article published in 2019 and a metric category shows 90%, that means that the article or review is performing better than 90% of the other articles/reviews published in that journal in 2019. If you select the 3-year option for the same article published in 2019 and the metric category shows 90%, that means that the article or review is performing better than 90% of the other articles/reviews published in that journal in 2019, 2018 and 2017.
Citation Benchmarking is provided by Scopus and SciVal and is different from the metrics context provided by PlumX Metrics.
Example: if you select the 1-year option for an article published in 2019 and a metric category shows 90%, that means that the article or review is performing better than 90% of the other articles/reviews published in that journal in 2019. If you select the 3-year option for the same article published in 2019 and the metric category shows 90%, that means that the article or review is performing better than 90% of the other articles/reviews published in that journal in 2019, 2018 and 2017.
Citation Benchmarking is provided by Scopus and SciVal and is different from the metrics context provided by PlumX Metrics.
Metrics Details
- Citations52
- Citation Indexes51
- CrossRef51
- 40
- Patent Family Citations1
- Patent Families1
- Captures213
- Readers213
- 213
- Mentions1
- References1
- Wikipedia1
Article Description
Reinforcement learning (RL) provides an influential characterization of the brain's mechanisms for learning to make advantageous choices. An important problem, though, is how complex tasks can be represented in a way that enables efficient learning. We consider this problem through the lens of spatial navigation, examining how two of the brain's location representations-hippocampal place cells and entorhinal grid cells-are adapted to serve as basis functions for approximating value over space for RL. Although much previous work has focused on these systems' roles in combining upstream sensory cues to track location, revisiting these representations with a focus on how they support this downstream decision function offers complementary insights into their characteristics. Rather than localization, the key problem in learning is generalization between past and present situations, which may not match perfectly. Accordingly, although neural populations collectively offer a precise representation of position, our simulations of navigational tasks verify the suggestion that RL gains efficiency from the more diffuse tuning of individual neurons, which allows learning about rewards to generalize over longer distances given fewer training experiences. However, work on generalization in RL suggests the underlying representation should respect the environment's layout. In particular, although it is often assumed that neurons track location in Euclidean coordinates (that a place cell's activity declines "as the crow flies" away from its peak), the relevant metric for value is geodesic: the distance along a path, around any obstacles. We formalize this intuition and present simulations showing how Euclidean, but not geodesic, representations can interfere with RL by generalizing inappropriately across barriers. Our proposal that place and grid responses should be modulated by geodesic distances suggests novel predictions about how obstacles should affect spatial firing fields, which provides a new viewpoint on data concerning both spatial codes. © 2011 Gustafson, Daw.
Bibliographic Details
10.1371/journal.pcbi.1002235; 10.1371/journal.pcbi.1002235.g005; 10.1371/journal.pcbi.1002235.g006; 10.1371/journal.pcbi.1002235.g001; 10.1371/journal.pcbi.1002235.g004; 10.1371/journal.pcbi.1002235.g008; 10.1371/journal.pcbi.1002235.g007; 10.1371/journal.pcbi.1002235.g003; 10.1371/journal.pcbi.1002235.g009; 10.1371/journal.pcbi.1002235.g002
http://www.scopus.com/inward/record.url?partnerID=HzOxMe3b&scp=80055084825&origin=inward; http://dx.doi.org/10.1371/journal.pcbi.1002235; http://www.ncbi.nlm.nih.gov/pubmed/22046115; https://dx.plos.org/10.1371/journal.pcbi.1002235.g005; http://dx.doi.org/10.1371/journal.pcbi.1002235.g005; https://dx.plos.org/10.1371/journal.pcbi.1002235.g006; http://dx.doi.org/10.1371/journal.pcbi.1002235.g006; https://dx.plos.org/10.1371/journal.pcbi.1002235; https://dx.plos.org/10.1371/journal.pcbi.1002235.g001; http://dx.doi.org/10.1371/journal.pcbi.1002235.g001; https://dx.plos.org/10.1371/journal.pcbi.1002235.g004; http://dx.doi.org/10.1371/journal.pcbi.1002235.g004; https://dx.plos.org/10.1371/journal.pcbi.1002235.g008; http://dx.doi.org/10.1371/journal.pcbi.1002235.g008; https://dx.plos.org/10.1371/journal.pcbi.1002235.g007; http://dx.doi.org/10.1371/journal.pcbi.1002235.g007; https://dx.plos.org/10.1371/journal.pcbi.1002235.g003; http://dx.doi.org/10.1371/journal.pcbi.1002235.g003; https://dx.plos.org/10.1371/journal.pcbi.1002235.g009; http://dx.doi.org/10.1371/journal.pcbi.1002235.g009; https://dx.plos.org/10.1371/journal.pcbi.1002235.g002; http://dx.doi.org/10.1371/journal.pcbi.1002235.g002; https://dx.doi.org/10.1371/journal.pcbi.1002235.g005; https://journals.plos.org/ploscompbiol/article/figure?id=10.1371/journal.pcbi.1002235.g005; https://dx.doi.org/10.1371/journal.pcbi.1002235.g004; https://journals.plos.org/ploscompbiol/article/figure?id=10.1371/journal.pcbi.1002235.g004; https://dx.doi.org/10.1371/journal.pcbi.1002235.g008; https://journals.plos.org/ploscompbiol/article/figure?id=10.1371/journal.pcbi.1002235.g008; https://dx.doi.org/10.1371/journal.pcbi.1002235.g003; https://journals.plos.org/ploscompbiol/article/figure?id=10.1371/journal.pcbi.1002235.g003; https://dx.doi.org/10.1371/journal.pcbi.1002235.g002; https://journals.plos.org/ploscompbiol/article/figure?id=10.1371/journal.pcbi.1002235.g002; https://dx.doi.org/10.1371/journal.pcbi.1002235.g006; https://journals.plos.org/ploscompbiol/article/figure?id=10.1371/journal.pcbi.1002235.g006; https://dx.doi.org/10.1371/journal.pcbi.1002235; https://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1002235; https://dx.doi.org/10.1371/journal.pcbi.1002235.g007; https://journals.plos.org/ploscompbiol/article/figure?id=10.1371/journal.pcbi.1002235.g007; https://dx.doi.org/10.1371/journal.pcbi.1002235.g009; https://journals.plos.org/ploscompbiol/article/figure?id=10.1371/journal.pcbi.1002235.g009; https://dx.doi.org/10.1371/journal.pcbi.1002235.g001; https://journals.plos.org/ploscompbiol/article/figure?id=10.1371/journal.pcbi.1002235.g001; http://dx.plos.org/10.1371/journal.pcbi.1002235.g005; http://dx.plos.org/10.1371/journal.pcbi.1002235.g002; http://dx.plos.org/10.1371/journal.pcbi.1002235.g003; https://journals.plos.org/ploscompbiol/article/file?id=10.1371/journal.pcbi.1002235&type=printable; http://journals.plos.org/ploscompbiol/article?id=10.1371%2Fjournal.pcbi.1002235; http://dx.plos.org/10.1371/journal.pcbi.1002235.g006; http://dx.plos.org/10.1371/journal.pcbi.1002235.g009; http://dx.plos.org/10.1371/journal.pcbi.1002235.g007; http://dx.plos.org/10.1371/journal.pcbi.1002235.g004; http://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1002235; http://dx.plos.org/10.1371/journal.pcbi.1002235.g001; http://www.plosone.org/article/metrics/info:doi/10.1371/journal.pcbi.1002235; http://journals.plos.org/ploscompbiol/article/file?id=10.1371/journal.pcbi.1002235&type=printable; http://dx.plos.org/10.1371/journal.pcbi.1002235; http://dx.plos.org/10.1371/journal.pcbi.1002235.g008
Public Library of Science (PLoS)
Provide Feedback
Have ideas for a new metric? Would you like to see something else here?Let us know