Part II: Creating and Adding Learning Rules

What is considered a learning rule?

Learning rules are necessary for the automated adaption of network parameters during simulation. At present, BindsNET supports two different categories of learning rules:

  • Two factor: Associative learning takes place based on pre- and post-synaptic neural activity. Examples include:
    • The typical example is Hebbian learning, which may be summarized as “Cells that fire together wire together.” That is, co-active neurons causes their connection strength to increase.
    • Spike-timing-dependent plasticity (STDP) stipulates that the ordering of pre- and post-synaptic spikes matters. A synapse is strengthened if the pre-synaptic neuron fires before the post-synaptic neuron, and, conversely, is weakened if it fires after the post-synaptic neuron. The magnitude of these updates is a decreasing function of the time between pre- and post-synaptic spikes.
  • Three factor: In addition to associating pre- and post-synaptic neural activity, a third factor is introduced which modulates plasticity on a more global level; e.g., for all synapses in the network. Examples include:
    • (Reward, error, attention)-modulated (Hebbian learning, STDP): The same learning rules described above are modulated by the presence of global signals such as reward, error, or attention, which can be variously defined in machine learning or reinforcement learning contexts. These signals act to gate plasticity, turning it on or off and switching its sign and magnitude, based on the task at hand.

The above are examples of local learning rules, where the information needed to make updates are thought to be available at the synapse. For example, pre- and post-synaptic neurons are adjacent to synapses, rendering their spiking activity accessible, whereas chemical signals like dopamine (hypothesized to be a reward prediction error (RPE) signal) are widely distributed across certain neuron populations; i.e., they are globally available. This is in contrast to learning algorithms like back-propagation, where per-synapse error signals are derived by computing backwards from a loss function at the network’s output layer. Such error derivation is thought to be biologically implausible, especially compared to the two- and three-factor rules mentioned above.

Creating a learning rule in BindsNET

At present, learning rules are attached to specific Connection objects. For example, to create a connection with a STDP learning rule on the synapses:

from bindsnet.network.nodes import Input, LIFNodes
from bindsnet.network.topology import Connection
from bindsnet.learning import PostPre

# Create two populations of neurons, one to act as the "source"
# population, and the other, the "target population".
# Neurons involved in certain learning rules must record synaptic
# traces, a vector of short-term memories of the last emitted spikes.
source_layer = Input(n=100, traces=True)
target_layer = LIFNodes(n=1000, traces=True)

# Connect the two layers.
connection = Connection(
    source=source_layer, target=target_layer, update_rule=PostPre, nu=(1e-4, 1e-2)
)

The connection may be added to a Network instance as usual. The Connection object takes arguments update_rule, of type bindsnet.learning.LearningRule, as well as nu, a 2-tuple specifying pre- and post-synaptic learning rates; i.e., multiplicative factors which modulate how quickly synapse weights change.

Learning rules also accept arguments reduction, which specifies how parameter updates are aggregated across the batch dimension, and weight_decay, which specifies the time constant of the rate of decay of synapse weights to zero. By default, parameter updates are averaged across the batch dimension, and there is no weight decay.

Other supported learning rules include Hebbian, WeightDependentPostPre, MSTDP (reward-modulated STDP), and MSTDPET (reward-modulated STDP with eligibility traces).

Custom learning rules can be implemented by subclassing bindsnet.learning.LearningRule and providing implementations for the types of AbstractConnection objects intended to be used. For example, the Connection and LocalConnection objects rely on the implementation of a private method, _connection_update, whereas the Conv2dConnection object uses the _conv2d_connection_update version.