Reinforcement Learning in Spiking Neural Networks

Biological neurons communicate primarily via a spiking process. Recurrently connected spiking neural networks (RSNNs) more realistically model the brain, compared to their non-spiking counterparts. It is of great interest to discover a biologically realistic learning rule to achieve optimal levels of performance on machine learning tasks. Experimental data describe a phenomenon known as spike-timing-dependent-plasticity (STDP), which integrates local firing coincidences between neurons to learn. STDP is believed to underlie memory formation and storage within the brain. When a reward signal modulates STDP, it enables forming associative memories via operant conditioning. Neuromodulators like dopamine operate similarly in the brain. We employ processes like synaptic scaling to support R-STDP in large, unstructured RSNNs. Doing so produces an agent that achieves adequate performance on reinforcement learning tasks.