We propose a network structure, inspired by experimental data on the neural correlates of navigation in the hippocampus , that learns to recognize sequences of symbols. An interaction of rhythmic inhibition with input conveying symbol information, similar to that proposed in , renders the model robust to wide variations in the rate of presentation of symbols and enables relationships between symbols to be learnt via spike-timing dependent plasticity. Competitive heterosynaptic plasticity results in the specialization of neurons to particular sequences so that our model is able to learn and recognize multiple sequences that share common subsequences, a capacity that was absent in earlier sequence learning models.
The network consists of pools of leaky integrate and fire neurons, with one such pool for each symbol. On presentation of a symbol, excitatory (Poisson) input commences to the corresponding pool. When a symbol is first presented it is represented by weak input, then the strength of input ramps up before turning off. As in , the rhythmic inhibition combines with the ramp-like input to result in sequence compression, i.e. the firing of the most recent symbols in the correct order during each cycle of the inhibition. Sequence compression ensures both the appropriate timescale for spike-timing dependent plasticity and robustness to variation in symbol presentation rate.
Connections between pools are subject to spike-timing dependent plasticity (in which synaptic strength is increased when the presynaptic neuron spikes shortly before the postsynaptic neuron). Synapses between pools corresponding to consecutive symbols are strengthened. Those neurons that receive such connections from the previously active pool fire with reduced latency; recurrent inhibition then prevents the other neurons of the pool from firing at all. Subtractive normalization, in which the total strength of the synapses onto a particular neuron is held constant , leads to competition so that each neuron specializes to a particular presynaptic group, resulting ultimately in neurons specialized to particular sequences.
A nonplastic network specified according to the scheme above is shown to correctly classify sequences with lengthy overlaps and to be robust to variation in symbol duration. Its classification performance is shown to degrade gracefully with perturbation of its parameters. A plastic network, trained on overlapping sequences presented with equal frequency, demonstrates correct classification behaviour. The effectiveness of training is sensitive to the frequency with which competing sequences are presented and to variation in the sizes of subpools corresponding to subsequences.