In dynamic systems, especially nonequilibrium thermodynamic systems, phase transitions often become important. This is a different usage of the word "phase". Here we no longer mean oscillating phase like the displacement of two sine waves, instead we mean the phase of matter, like ice, water, or steam. Phase equilibria are globally stable states of the system. The prototypical model for understanding phase transitions in neural systems is the Ising model of ferromagnetism, which underlies John Hopfield's original 1982 paper on thermodynamic networks.

In a ferromagnetic material there is a critical temperature (the Curie temperature) at which the alignment of the individual magnetic dipoles breaks down. Normally, the magnetism of the chunk of metal is due to the alignment of all the little atomic magnets inside it, but once the chunk reaches the critical temperature, the alignment breaks down and therefore it is no longer "globally" magnetic, even though the individual dipoles still exist. The "phase" of this system relates to its magnetic behavior, and depends on the temperature.

It is important for neuroscientists and students of machine learning to understand the Ising model of ferromagnetic behavior. It begins with a simple lattice of "nodes", which in our case we consider to be neurons. In the real physical case of a ferromagnetic material, these nodes would be atomic magnetic moments, spins that point either up or down. In the simplest case they are binary, but in a more interesting case they could take on analog values.

As we raise the temperature, the dipoles start to disalign, and we get fractions of them that are at this angle or the other. We can calculate (approximately) how many of them are at a given angle using statistical mechanics. In the extreme situation, all the angles are random and there is no relationship between any of them, however there are situations in between where there may be pockets with a greater ratio of one angle or another. In this case, we are interested in the relationship between these pockets, because Kuramoto tells us that when oscillators of the same frequency interact, their phases tend to couple.

Instead of raising the temperature to decouple the oscillators, we can also do the opposite, we can begin at a high temperature with a fully decoupled system, and then lower the temperature so the magnetic dipoles start aligning. In this case we're interested to know whether there are any interactions between the dipoles - in other words, does the alignment of dipoles in one region somehow assist or hinder the alignment of dipoles in a different region?

In the original Ising model there was no concept of "oscillator", there was simply an angle attached to every dipole. To get an oscillator, the angle has to move back and forth like a pendulum. This formalism is easily introduced. The figure shows a network with oscillators, tied into a mesh with resistors.

In an "Ising model with oscillators", or "oscillator based Ising model", resistively coupled nonlinear oscillators represent the spins, and the system dynamics are represented as differential equations in the oscillator phase, much like the Kuramoto approach. Such systems admit a Lyapunov function that matches the Ising Hamiltonian at stable equilibria, which in turn allows the system to evolve to its lower energy state.

Oscillator based Ising models show dramatic performance improvements over phase-amplitude models for combinatorial optimization problems (like the traveling salesman problem, or planning a trajectory to reach an object in visual space). However before undertaking such an ambitious study, it behooves us to understand the basics of Ising in terms of information processing. How does a spin lattice or a bunch of coupled oscillators relate to information processing?

For information processing purposes, we are interested in the transition probabilities of the spins, from one state to another (because if we can control these probabilities, then we can store and process information). For example in this figure the transition probabilities are shown in orange.

More specifically though, we're interested in the "order" in such systems, and particularly in the long range order. This concept is necessary to link the Ising model with nonlinear thermodynamics, which we'll look at on the next page. The order parameters relate to the "coupling constants" between oscillators. Also important for us to understand, are the boundary conditions, both globally and locally. This seems a bit of a tall order, but much can be learned with just basic statistical mechanics (like the Gibbs formalism).