n spins are connected by couplings .
Different realizations of give different systems, for example
- : Ferromagnets model or anti-ferromagnets.
- Sherrington-Kirkpatrick model, spin glasses.
- Hebb's rule: Hopfield model, associative memories.
- are learned from data: neural networks.
The energy of a configuration is
where is the external field added on spin .
Note that In the whole discussions I would set the external field to zero, because this does not change quantitatively the results we are going to show, but significantly reduces the length of formulas :)
In the canonical ensemble, the probability of finding a configuration in the equilibrium at inverse temperature follows the Boltzmann distribution:
is the partition function.
- There are totally configurations in the summation.
- when , every configuration has the identical Boltzmann weights, which is .
- when , only configurations having the lowest energy has finite probability measure.
Why Ising model?
In addition to physical motivations (phase transitions, criticality, ...), another reason that the Ising model is useful in model science and technique is that it is the Maximum entropy model given first two moments of observations. That is the distribution that make the least bias or claim to the observed data.
Suppose we have m configurations that are sampled from the Boltzmann distribution of the model, then we can define the following statistics that can be observed from data:
Many distributions can be used to generate data with given first and second moments, suppose is such a distribution. Then we can write out the entropy of the distribution as
Of cause, there are constraints that need to be satisfied:
We define a Lagrangian as
where are multipliers.
By setting , we have