Mean-Field Free Energy: Variational Approach

Recommended resources

The freely available resources this lesson will direct you to.

Hello! Let's continue our exploration of mean-field theory.

In our previous lessons, we derived the mean-field self-consistency equation for the Ising model, solved it to find a continuous phase transition, and calculated the full set of "classical" critical exponents ( $α, β, γ, δ$ ). Our approach was based on the "molecular field" approximation—an intuitive but somewhat ad-hoc step where we replaced the influence of neighboring spins with their average value.

Today, we will put the mean-field approximation on a much more rigorous foundation. We will show that it's not just a clever guess, but a systematic method for finding the best possible approximation to the true free energy of a system, given a certain level of simplicity. This method is an example of a variational principle, a concept that is fundamental throughout physics.

Our learning outcome for this lesson is to derive the mean-field free energy using the Bogoliubov inequality (variational principle). This will not only give us a more profound understanding of why mean-field theory works but also provide us with the explicit form of the mean-field free energy, a powerful tool that we will use in subsequent lessons.

1. The Variational Principle for Free Energy

For an interacting system like the Ising model, calculating the partition function $Z = \sum_{states} e^{- β H}$ and the corresponding Helmholtz free energy $F = - k_{B} T ln Z$ is generally an intractable problem. The core idea of the variational method is to approximate the true, complex Hamiltonian $H$ with a simpler, solvable "trial" Hamiltonian $H_{0}$ .

The quality of this approximation is governed by a powerful theorem known as the Bogoliubov inequality. It states that the true free energy $F$ of a system with Hamiltonian $H$ is always less than or equal to a quantity called the variational free energy, $F_{V}$ :

F \leq F_{0} + ⟨ H - H_{0} ⟩_{0} \equiv F_{V}

Here:

$H_{0}$ is our simple, solvable trial Hamiltonian.
$F_{0}$ is the exact free energy corresponding to $H_{0}$ .
$⟨ \dots ⟩_{0}$ denotes a thermal average calculated using the statistical weights of the trial system, $e^{- β H_{0}}$ .

This inequality is the thermodynamic analogue of the Rayleigh-Ritz variational principle in quantum mechanics, which provides an upper bound for the ground state energy. The Bogoliubov inequality provides an upper bound for the free energy at any temperature. The strategy is to introduce adjustable parameters into $H_{0}$ and then minimize $F_{V}$ with respect to these parameters. This minimization yields the best possible approximation to the true free energy that can be achieved with our chosen form of $H_{0}$ .

2. The Bogoliubov Inequality: A Proof

Understanding where this inequality comes from is crucial. The proof is surprisingly direct and relies on a standard mathematical result called Jensen's inequality, which applies to convex functions.

For a deeper look at the formal derivation, we'll turn to a section of a SciPost lecture note.

Magnetisation and mean field theory in the Ising model

The following resource provides a concise proof of the Bogoliubov inequality. It starts from the exact partition function and uses Jensen's inequality to arrive at the final result.

Please read Section 2.1, 'Proving the Bogoliubov inequality' (pages 3-4). Focus on the steps that take you from the exact partition function Z (Eq. 2) to the final inequality (Eq. 1). The key step is the application of Jensen's inequality to the exponential function.

The proof hinges on two key ideas you just read about:

The exact partition function $Z$ can be written in terms of the trial system's partition function $Z_{0}$ and an average over the trial system's states: $Z = Z_{0} ⟨ e^{- β (H - H_{0})} ⟩_{0}$ .
The exponential function $f (x) = e^{x}$ is convex. Jensen's inequality states that for any convex function $f$ , $⟨ f (x)⟩ \geq f (⟨ x ⟩)$ . Applying this to our expression gives $⟨ e^{- β (H - H_{0})} ⟩_{0} \geq e^{- β ⟨ H - H_{0} ⟩_{0}}$ .

Combining these facts and taking the logarithm leads directly to the Bogoliubov inequality, $F \leq F_{0} + ⟨ H - H_{0} ⟩_{0}$ .

For a broader perspective, it's worth noting that this result is a specific case of the more general Gibbs-Bogoliubov inequality, which is typically proven in the context of quantum statistical mechanics. The resource "Rayleigh-Ritz and Gibbs-Bogoliubov" (LINK) shows this connection explicitly.

3. Applying the Variational Method to the Ising Model

Now for the main event: let's use this machinery to derive the mean-field theory for the Ising model. The process involves three steps: choosing a trial Hamiltonian, constructing the variational free energy, and minimizing it.

Step 1: Choosing the Trial Hamiltonian

The full Ising Hamiltonian is:
$H = - J \sum_{⟨ i, j ⟩} s_{i} s_{j} - h \sum_{i} s_{i}$
The difficulty lies in the interaction term $\sum s_{i} s_{j}$ . The heart of the mean-field approximation is to choose a trial Hamiltonian $H_{0}$ that has no interactions. The simplest such choice is a system of independent spins in a uniform, as-yet-unknown "variational field," which we'll call $m_{v}$ :
$H_{0} = - m_{v} \sum_{i} s_{i}$
This $H_{0}$ describes a simple paramagnet, which is exactly solvable. Our goal is to find the value of $m_{v}$ that makes this simple model best approximate the true interacting system.

Step 2: Constructing the Variational Free Energy $F_{V}$

We now build the variational free energy $F_{V} = F_{0} + ⟨ H - H_{0} ⟩_{0}$ . Let's calculate the terms, working with per-spin quantities for simplicity (dividing by the number of spins $N$ ). Let $z$ be the coordination number of the lattice.

Trial Free Energy ( $f_{0}$ ): For a paramagnet of $N$ spins in a field $m_{v}$ , the per-spin free energy is:
$f_{0} = \frac{F _{0}}{N} = - k_{B} T ln [2 cosh (β m_{v})]$
Expectation Value Term: We need to compute $⟨ H - H_{0} ⟩_{0} / N$ . The averages are taken over the trial system, where the spins are independent. The average magnetization per spin in this trial system is $μ \equiv ⟨ s_{i} ⟩_{0} = tanh (β m_{v})$ .
$\frac{1}{N} ⟨ H - H_{0} ⟩_{0} = \frac{1}{N} ⟨ - J ⟨ i, j ⟩ \sum s_{i} s_{j} - h i \sum s_{i} - (- m_{v} i \sum s_{i}) ⟩_{0}$
Since the spins are independent in the $⟨ \dots ⟩_{0}$ average, $⟨ s_{i} s_{j} ⟩_{0} = ⟨ s_{i} ⟩_{0} ⟨ s_{j} ⟩_{0} = μ^{2}$ . The number of interacting pairs is $N z /2$ .
$\frac{1}{N} ⟨ H - H_{0} ⟩_{0} = - J \frac{z}{2} μ^{2} - h μ + m_{v} μ$

Putting it all together, the variational free energy per spin is:

f_{V} (m_{v}) = - k_{B} T ln [2 cosh (β m_{v})] - J \frac{z}{2} μ^{2} + (m_{v} - h) μ

where we must remember that $μ = tanh (β m_{v})$ .

Step 3: Minimizing the Variational Free Energy

The final step is to find the value of our variational parameter $m_{v}$ that minimizes $f_{V}$ . This will give us the best approximation. We need to solve $\frac{\partial f _{V}}{\partial m _{v}} = 0$ .

The same resource we used for the proof also carries out this minimization.

Magnetisation and mean field theory in the Ising model

Let's follow the derivation of the minimization condition. This section applies the general Bogoliubov inequality specifically to the Ising model with our chosen trial Hamiltonian.

Please read Section 2.2, 'Deriving a mean field model by variational methods' (pages 5-7). Follow the calculation that minimizes the variational free energy F_V with respect to the variational field (called m in the text). The key outcome is Equation (11), which establishes a condition on this field.

As shown in the resource, after taking the derivative $\frac{\partial F _{V}}{\partial m _{v}} = 0$ (note the resource uses $m$ for the variational field) and simplifying, we arrive at the condition for the minimum:

m_{v} = J j \sum ⟨ s_{j} ⟩_{0} + h

Assuming nearest-neighbor interactions on a lattice with coordination number $z$ , and recalling that $⟨ s_{j} ⟩_{0} = μ$ , this becomes:

m_{v} = J z μ + h

This is a profound result. It tells us that the variational field $m_{v}$ that minimizes the upper bound on the free energy is precisely the "molecular field" we postulated in our earlier, more heuristic approach. The variational principle has independently discovered the self-consistency condition! The approximation $⟨ s_{i} ⟩ = μ$ is now justified as the result of a well-defined energy minimization procedure.

Test your understanding!

We chose a trial Hamiltonian $H_{0} = - m_{v} \sum_{i} s_{i}$ where all spins feel the same field $m_{v}$ . What physical assumption about the ordered state does this choice implicitly make? How could you generalize the trial Hamiltonian to model an antiferromagnet, where neighboring spins prefer to align oppositely?

Show answer

The choice of a uniform field $m_{v}$ implicitly assumes a ferromagnetic state, where all spins on average point in the same direction, so the mean magnetization $⟨ s_{i} ⟩ = μ$ is the same for all sites $i$ .

To model a simple two-sublattice antiferromagnet (like on a square or cubic lattice), you would need a trial Hamiltonian with a staggered variational field. You could divide the lattice into two sublattices, A and B, and propose:
$H_{0} = - m_{A} \sum_{i \in A} s_{i} - m_{B} \sum_{j \in B} s_{j}$
Minimizing the free energy with respect to both $m_{A}$ and $m_{B}$ would lead to the condition $m_{A} = - m_{B}$ , correctly capturing the antiferromagnetic order.

4. The Mean-Field Free Energy

Now that we have established the variational basis for mean-field theory, we can write down the expression for the (approximated) mean-field free energy itself. This is the value of $f_{V}$ at the minimum. However, it's more useful to express it not in terms of the auxiliary field $m_{v}$ , but in terms of the physical order parameter, the magnetization $μ$ .

To do this, we rewrite the entropy term of the free energy. The entropy of a non-interacting spin system with magnetization $μ$ is given by:
$S (μ) = - k_{B} N [(\frac{1 + μ}{2}) ln (\frac{1 + μ}{2}) + (\frac{1 - μ}{2}) ln (\frac{1 - μ}{2})]$
This is the standard entropy of mixing for a two-state system.

The mean-field free energy per spin, $f_{MF} (μ) = u - T s$ , can then be written as a function of the order parameter $μ$ :

f_{MF} (μ) = Energy u - \frac{1}{2} J z μ^{2} - h μ + Entropy - T s k_{B} T [(\frac{1 + μ}{2}) ln (\frac{1 + μ}{2}) + (\frac{1 - μ}{2}) ln (\frac{1 - μ}{2})]

This expression is the mean-field free energy. It is a function of the order parameter $μ$ . The actual equilibrium state of the system is found by minimizing this free energy with respect to $μ$ , i.e., by solving $\frac{\partial f _{MF}}{\partial μ} = 0$ . If you carry out this differentiation, you will recover exactly the self-consistency equation $μ = tanh (β (J z μ + h))$ that we started with in this module.

This form of the free energy is incredibly important. It combines the energetic term that favors order (the $μ^{2}$ term) and the entropic term that favors disorder (the logarithmic term). The competition between these two terms as a function of temperature is what drives the phase transition.

Conclusion

Key Takeaways:

The mean-field approximation is not just an intuitive guess; it is the result of a systematic variational procedure to find the best upper bound on the true free energy using a non-interacting trial Hamiltonian.
The Bogoliubov inequality, $F \leq F_{0} + ⟨ H - H_{0} ⟩_{0}$ , provides the formal basis for this variational principle in statistical mechanics.
Minimizing the variational free energy for the Ising model with respect to the variational field $m_{v}$ naturally leads to the self-consistency condition, showing that the optimal variational field is identical to the molecular field.
The resulting mean-field free energy is a function of the order parameter $μ$ , explicitly showing the competition between energy (which favors order) and entropy (which favors disorder).

This lesson provides the formal justification for everything we have done in this module on mean-field theory. We now have a deeper understanding of its foundation as an approximation scheme.

Preview of the Next Lesson:
We will now turn our attention to one of the most direct applications of this formalism. We will use the mean-field framework to analyze the Heisenberg model of magnetism and compare its predictions to the Ising case we've studied in detail. This will highlight how changing the symmetry of the order parameter affects the nature of the phase transition within mean-field theory.