Together, the two hemispheres of the mouse cortex contain, 16 × 106 neurons and 8,000 synapses per neuron. We have recently developed a massively parallel cortical simulator  that incorporates relatively simpler single compartment spiking neurons , spike-timing dependent plasticity (STDP) , and axonal delays.
We created a mouse-scale network by using 32,768 "groups" (80% excitatory) each with 500 neurons such that each group connects to 100 randomly selected groups and each neuron from the projecting group makes a total of c = 80 synapses with the neurons of the receptive group. Excitatory groups had axonal delays uniformly ranging from 1–20 ms, and inhibitory groups had a fixed delay of 1 ms. All simulations used a 1 ms time step. Using a BlueGene/L with 8,192 processors, with 4 TB of memory, using a super-threshold stimulus delivered to every neuron at 4 Hz, we were able to simulate 5 s of model time in 168 s of real-time at a mean firing rate of 4.95 Hz (in stable mode). To further push the boundaries of scaling, by using c = 160 above, we created a network with 16,384,000 neurons and 16,000 synapses per neuron. Using 16,384 processors and 8 TB of memory, using a 5 Hz stimulation, we were able to achieve 5 s of model time in 265 s of real-time at a mean firing rate of 5 Hz (in stable mode).
While it is very easy to drive a network into a damped state or into an avalanche mode, stabilizing cortical simulations is enormously difficult (p. 167, ), . We found that the allowed maximum synaptic efficacy (which upper bounds the growth of excitatory synaptic efficacies under STDP) and the probability of the super-threshold stimulus together greatly affected the behavior of networks. We explored several models with varying numbers of synapses from 1 to 16,000 synapses per neuron. We observed that finding a range of maximum synaptic efficacies corresponding to stable models is harder to achieve for higher number of synapses per neuron if the stimulus probability is kept low. Further, there appears to be a threshold stimulus probability below which – when maximum synaptic efficacy is varied – models make a sharp transition from damped to avalanche mode. For both the networks with 16,000 and 8,000 synapses per neuron, we observed three distinct modes, namely, damped, stable, and avalanche (shown in figure 1 for the larger network).
Figure 1. Damped, stable and avalanche modes in network simulations.