When it comes to engineering neural networks, the machine learning community is focused on performance, and in this focus it pays attention to things that neuroscientists often leave till the last minute - and... from a development standpoint, this is kind of like error handling, it's something you want to do up front and consistently, otherwise once your system gets complex you might find you have to take the whole thing apart again.

Machine learning depends heavily on non-biological mechanisms. There are some very clever learning rules invented in the machine learning community, that have no counterpart in the brain (or at least, none known so far). However machine learning informs us about the vital character of the data, and that different strategies work well with different kinds of data. Machine learning and data science are closely related, many of the same techniques are in use. However machine learning has the freedom to use methods that are distinctly "non-biological" (like back propagation), which works for digital computers but doesn't work in the biological context.

When considering the brain from a real-time standpoint, we're probably less interested in invariances and more interested in the ability to precisely time motor behavior. The particular system we looked at (the oculomotor system) does not always require precise timing, in most cases the timing is either reactive or voluntary. (Nevertheless we've seen how rapid movements are coordinated). In relation to the real time brain, we might be more interested in the piano player or drummer, or the basketball or ping pong player. The good news here is that we can use relatively simple tools to begin experimenting, like video games and brain-computer interfaces.

Before embarking on a survey of machine learning architectures, it behooves us to organize the various different types of synaptic plasticity we know about from the neuroscience world. These include things like STP, LTD, and STDP. In general the learning mechanism has to do with the simultaneous activation of pre- and post-synaptic neurons. Whenever the neurons fire together, some kind of modification occurs in the synapse. The time course of such modifications could be milliseconds or it could be minutes to days, it depends on the underlying molecular kinetics. Unfortunately there is no such thing as a "generic synapse", the best we can hope for in wetware is a synapse that behaves more or less the way we want it to.

Data Storage

To begin with, we can consider the division of plasticity into short term and long term types. Short term is designated as minutes or less, if the modifications last more than an hour they're usually considered long-term. The division is somewhat arbitrary, it's better to think in terms of the time constants and the way they relate to each other.
Synaptic changes can occur either homosynaptically (within the same synapse) or heterosynaptically (based on simultaneous events in two or more different synapses). For homosynaptic modifications, it is important that the synapse be isolated from the extracellular environment and from the rest of the neuron's intracellular milieu. This is because synaptic modifications depend heavily on ion fluxes, and the environment at large is too volatile for sensitive synaptic updates.

The first thing we'd like to consider is the "burning in" of information. Let's say I have an image, and I show it to the network, and now I'd like the network to recall the stored image. To do this, the image details have to be "burned in" to the synapses, in such a way that a representation of the same stimulus causes a large response, whereas the presentation of any other stimulus causes a smaller response (or no response at all).

We've already seen two different networks that can "memorize" data like this, specifically the Hopfield network and the Kohonen self-organizing map. Let us examine more closely the synaptic update rules ("learning rules") that make these systems tick. The Hopfield network is more straightforward in a way, the synapses are updated based on the product of the pre- and post-synaptic activities. Which begs a question... can neurons multiply? We know they can add signals along the membrane, but is there a way to get an actual multiplication so we can build modulators and demodulators?

The answer, fortunately, is yes - neurons can multiply. Not only that but they can do it in several ways, and which is most useful depends on the application. For example in covariance-driven networks, the actual signal level may not be as important as the simultaneous timing, because the denominators get scaled anyway, in a different process that occurs after the fact - so we don't really need to scale them twice. (Obviously that's a machine learning point of view, and neuroscientists need to be able to flexibly adopt this view to understand many important network behaviors and the ranges of appropriate time constants).

When a synapses changes weight in a Hopfield network, the energy surface changes shape. As the shape changes, the location of maxima and minima change. So when the network settles into its lowest energy state, the weight matrix will determine where it lands. This is a very useful form of learning because it's entirely local, that is to say it depends only on the activity in the one synapse, plus the energy in the network. On the other hand, the Kohonen network uses a different learning rule. In a Kohonen network, there is competitive learning using a "winner take all" rule. This means that only the strongest responding neuron will have its synapses updated. To accomplish this, after we multiply the input by the weight matrix, we have to scan the output activity levels, to determine the winner. In turn this means that we can update a Hopfield network neuron by neuron, by we have to update a Kohonen network on a cycle. In the Kohonen network, we have to scan the outputs each time before we update the weight matrix. This is non-biological because real neurons don't do that - and in fact there are other ways of building a Kohonen network that don't require this non-biological out-of-band scanning activity.

Types of Plasticity

Once we understand that the basic unit of informational change in a neural network is the synapse, we can start looking around for the various ways biological synapses behave. Hopefully this topic has been adequately covered on the previous pages, and at this point we can start looking at some specific mechanisms for the various kinds of plasticity. We'll begin by looking short term potentiation (STP), which was the first form of plasticity discovered by Lomo in 1973.

STP is usually induced by electrical stimulation. Current is passed through the electrode and the ion concentrations across the neural membrane adjust themselves in response. The noteworthy behavior is that subsequent stimulations produce a much larger response. This is the opposite behavior from what we usually expect, we usually expect some kind of "adaptation". However after the intial discovery, STP was found pretty much everywhere it was look for, so it appears to be a ubiquitous mechanism all over the brain.

In addition to STP there is STD, which is short term depression. STP and STD usually work on different kinds of synapses. These forms of plasticity are linked to the action of second messenger molecules (like cyclic AMP) inside the cell. The onset of STP or STD is rapid after stimulation, and it lasts for a long time compared to the input that created it. In many cases short term plasticity is linked to calcium ion channels in the postsynaptic membrane, which means it is very likely they'll be affected by astrocytes in tripartite synapses. Astrocytes are large cells, one astrocyte can wrap itself around 100,000 synapses - so if there is involvement of astrocytes, we can look for "regional" influences, even though in the best case we'd like to keep everything local.

STP and STD have long term counterparts, called LTP and LTD respectively. All of these four types of plasticity are unidirectional, that is to say, the modification only works in one direction. However there is another kind of plasticity called STDP that is bidirectional - the direction it takes depends on whether the input occurred before or after the firing of the postsynaptic neuron. If it occurred before, the synapse is depressed, but if it occurred afterwards, the synapse is potentiated. Obviously STDP is most useful with spiking neurons, since it's timing related.

Encoding

The information coming out of a synapses "diffuses" in time. Only a little, but nevertheless the amount is significant. A firing "event" becomes a signal about a millisecond in duration. During this millisecond, the postsynaptic membrane responds with a time course to the input. What happens after that, depends on the state of the rest of the membrane. It is therefore logical that the presynaptic system could benefit from knowing the state of the postsynaptic neurons, before sending them any signals. The coordination of such exchanges is the domain of population dynamics. The neuron's state always depends on the state of the extracellular milieu, and most of the time this will in turn be adjusted by population rhythms (like alpha and theta waves).

There is plenty of evidence that subthreshold membrane oscillations in the post-synaptic neuron will cause the transmission of an input to "wait" until the next peak. This is an important phenomenon that can have a dramatic impact on the signal processing in a neural network. In general, introducing delays (conduction delays, processing delays) into a neural network will cause its dynamics to change. Systems that were previously well behaved will suddenly become chaotic and so on. The effects of these delays are well known from control systems theory, and in fact this discipline is indispensible for neuroscientists, very little can be understood about a neural network without understanding its dynamics.

Population Codes

Sometimes, population codes "emerge" from the network on the basis of the neural wiring. A simple example of this was given in the auditory system of the barn owl, where the precise localization of sounds in space is determined by neural delay lines in the cochlear pathways.

A population code simply means that the postsynaptic network needs to read non-local information. To a certain extent this can be handled with convergence and divergence, and to a certain extent it can also be handled with hot spots and regional Hamiltonians. The concept of a hot spot is shown in the figure.