Partition Function and Helmholtz Free Energy

Welcome back to our study of statistical mechanics. In the previous lesson, we introduced the canonical ensemble to describe systems in thermal equilibrium with a large heat reservoir. This led us to the Boltzmann distribution, $P_{k} \propto e^{- β E_{k}}$ , and we defined the canonical partition function, $Z$ , as the normalization constant for this probability distribution. While this gave us a way to calculate probabilities, the deeper physical meaning of $Z$ remained to be explored.

This lesson reveals the profound connection between the microscopic world summed up in $Z$ and the macroscopic world of thermodynamics. Our goal is to show that the partition function is not merely a mathematical convenience but is directly related to a key thermodynamic potential: the Helmholtz free energy, $F$ . By establishing the master equation $F = - k_{B} T ln Z$ , we will transform the partition function into the central computational tool of the canonical ensemble, providing a bridge from microscopic states to macroscopic observables.

1. Thermodynamic Potentials and the Helmholtz Free Energy

From your previous studies in thermodynamics, you'll recall the fundamental thermodynamic relation for a simple system:

d E = T d S - P d V

This equation naturally expresses the internal energy $E$ as a function of entropy $S$ and volume $V$ . However, in the canonical ensemble, the natural control variables are temperature $T$ and volume $V$ , not entropy. To work with these variables, it's mathematically advantageous to use a different thermodynamic potential.

We introduce the Helmholtz free energy, $F$ , through a Legendre transformation of the internal energy:

F \equiv E - T S

Taking the differential of $F$ gives:

d F = d E - T d S - S d T = (T d S - P d V) - T d S - S d T = - S d T - P d V

This shows that the natural variables for $F$ are indeed $T$ and $V$ . From this exact differential, we can immediately identify expressions for entropy and pressure:

S = - (\frac{\partial F}{\partial T})_{V} and P = - (\frac{\partial F}{\partial V})_{T}

This thermodynamic framework shows that if we could find a way to calculate $F$ for a system from its microscopic properties, we would have a direct route to all its macroscopic equilibrium properties. This is precisely what the partition function allows us to do.

2. The Bridge: Deriving the Relation $F = - k_{B} T ln Z$

The crucial link between statistical mechanics and thermodynamics comes from combining the statistical definition of entropy with the Boltzmann distribution. In Module 1, you learned the general form for the entropy of a system described by a probability distribution ${p_{k}}$ , known as the Gibbs entropy:

S = - k_{B} k \sum p_{k} ln p_{k}

For the canonical ensemble, the probability $p_{k}$ of being in a microstate $k$ with energy $E_{k}$ is given by the Boltzmann distribution, $p_{k} = \frac{e ^{- β E_{k}}}{Z}$ . Let's substitute this into the entropy formula:

S = - k_{B} k \sum \frac{e ^{- β E_{k}}}{Z} ln (\frac{e ^{- β E_{k}}}{Z})

Using the property $ln (a / b) = ln (a) - ln (b)$ , we can split the logarithm:

S = - k_{B} k \sum p_{k} (- β E_{k} - ln Z) = k_{B} β k \sum p_{k} E_{k} + k_{B} ln Z k \sum p_{k}

Recognizing that $\sum_{k} p_{k} E_{k}$ is the definition of the average energy $⟨ E ⟩$ and $\sum_{k} p_{k} = 1$ by normalization, we find:

S = k_{B} β ⟨ E ⟩ + k_{B} ln Z

Since $β = 1/ (k_{B} T)$ , this simplifies to:

S = \frac{⟨ E ⟩}{T} + k_{B} ln Z

Rearranging this equation gives $T S = ⟨ E ⟩ + k_{B} T ln Z$ , which leads directly to our target:

⟨ E ⟩ - T S = - k_{B} T ln Z

The left-hand side is precisely the definition of the Helmholtz free energy, $F$ . Therefore, we have arrived at the central result of this lesson:

F = - k_{B} T ln Z

This equation is the bridge connecting the microscopic details of the system (the energy levels $E_{k}$ contained within $Z$ ) to a macroscopic thermodynamic potential ( $F$ ).

The following reading walks through this exact derivation with great clarity.

Part II - Statistical Physics

This derivation is central to the entire canonical framework. Dexter Chua's notes present it with exceptional clarity. You will see how the statistical definitions of entropy and average energy combine to yield the free energy.

Please read the subsection "Entropy" on page 17, starting from the definition of Gibbs entropy. Follow the calculation through to the result for the canonical ensemble, S = kβ⟨E⟩+ k log Z. Immediately after, read the first two paragraphs of Section 1.4, "Helmholtz free energy," and the final proposition. This will connect the entropy expression you just derived to the thermodynamic definition of F, culminating in the key formula F = -kT log Z.

3. The Physical Significance of Free Energy

The second law of thermodynamics states that an isolated system evolves towards the state of maximum entropy. For a system at constant temperature and volume, the corresponding principle is that it evolves towards the state of minimum Helmholtz free energy. The relation $F = E - T S$ shows that this minimization is a competition between two opposing tendencies: minimizing energy ( $E$ ) and maximizing entropy ( $S$ ). At low temperatures, the $E$ term dominates, and the system seeks its lowest energy ground state. At high temperatures, the $T S$ term dominates, and the system favors states with high entropy (i.e., a large number of accessible microstates), even if they have higher energy.

The partition function $Z = \sum_{k} e^{- E_{k} / k_{B} T}$ beautifully captures this competition. A large value of $Z$ can arise from states with very low energy $E_{k}$ (making $e^{- β E_{k}}$ large) or from a vast number of states with moderate energy (many terms in the sum). Since $F = - k_{B} T ln Z$ , a larger partition function corresponds to a lower, more favorable free energy.

Given your interest in asymptotic methods, it's worth noting that the canonical free energy can also be viewed as the result of a saddle-point approximation applied to the microcanonical ensemble. The probability of the system having energy $E$ is proportional to $Ω (E) e^{- β E}$ , where $Ω (E) = e^{S (E) / k_{B}}$ is the microcanonical density of states. The most probable energy, $E^{*}$ , maximizes this probability, which is equivalent to maximizing the exponent $S (E) / k_{B} - β E$ . The Helmholtz free energy is then simply the value of $E - T S (E)$ evaluated at this most probable energy, $E^{*}$ . This establishes the Legendre transform relationship between the two ensembles rigorously.

Equilibrium Statistical Mechanics

To solidify the connection between the canonical and microcanonical pictures, it's insightful to see how the Helmholtz free energy emerges from a saddle-point evaluation of the number of states.

In these notes from MIT, focus on the short section titled "Relation between F(T) and S(E)" on page 58. Notice how F is identified with the Legendre transform of the microcanonical entropy S_m(E). This formally shows that the canonical free energy corresponds to the value of E - TS at the most probable energy E^*, providing a bridge between the ensembles.

4. The Partition Function as a Generating Function

The true power of the relation $F = - k_{B} T ln Z$ is that it makes the partition function a "generating function" for all thermodynamic observables. Once we perform the microscopic sum to find $Z$ , we can derive any macroscopic quantity through differentiation.

Let's re-derive the average energy $⟨ E ⟩$ to see this in action. We know from thermodynamics that $⟨ E ⟩ = F + T S = F - T (\partial F / \partial T)_{V}$ . Substituting $F = - k_{B} T ln Z$ :

⟨ E ⟩ = - k_{B} T ln Z - T \frac{\partial}{\partial T} (- k_{B} T ln Z)

⟨ E ⟩ = - k_{B} T ln Z + T (k_{B} ln Z + k_{B} T \frac{1}{Z} \frac{\partial Z}{\partial T}) = k_{B} T^{2} \frac{1}{Z} \frac{\partial Z}{\partial T} = k_{B} T^{2} \frac{\partial ln Z}{\partial T}

It is often more convenient to work with $β = 1/ (k_{B} T)$ . Using the chain rule, $\frac{\partial}{\partial T} = \frac{d β}{d T} \frac{\partial}{\partial β} = - \frac{1}{k _{B} T ^{2}} \frac{\partial}{\partial β}$ . Substituting this into our expression for $⟨ E ⟩$ :

⟨ E ⟩ = k_{B} T^{2} (- \frac{1}{k _{B} T ^{2}} \frac{\partial ln Z}{\partial β}) = - \frac{\partial ln Z}{\partial β}

This confirms the result we found in the previous lesson, but now derived from the free energy. Similarly, we can find the pressure:

P = - (\frac{\partial F}{\partial V})_{T} = - \frac{\partial}{\partial V} (- k_{B} T ln Z) = k_{B} T \frac{\partial ln Z}{\partial V}

The process is always the same:

Model the system to determine its microstates and their energies, $E_{k}$ .
Calculate the partition function $Z = \sum_{k} e^{- β E_{k}}$ . This is often the hardest step.
Calculate the free energy $F = - k_{B} T ln Z$ .
Calculate any other thermodynamic observable by taking the appropriate derivative of $F$ .

The following video provides a good summary of this "generating function" concept.

Teach Yourself Statistical Mechanics In One Video | New & Improved

The true power of the partition function is its role as a "generating function" for all thermodynamic observables. This video provides a nice visual summary of how different derivatives of log Z yield quantities like energy and pressure.

Watch the segment from Applications of the Partition Function. While the video discusses the grand canonical ensemble as well (which we will cover soon), focus on the derivatives for average energy and pressure. The key takeaway is the general method: once you compute Z, you can find any macroscopic average by taking the appropriate derivative of \ln Z.

Conclusion

In this lesson, we have elevated the partition function from a simple normalization constant to the central object in canonical statistical mechanics. This was achieved by demonstrating its direct connection to the Helmholtz free energy.

Key Takeaways:

The Helmholtz free energy, $F = E - T S$ , is the natural thermodynamic potential for systems at constant temperature and volume.
The partition function $Z$ is fundamentally related to the free energy by the master equation: $F = - k_{B} T ln Z$ .
This relationship provides a complete bridge from the microscopic physics of a system (its energy spectrum, encoded in $Z$ ) to its macroscopic thermodynamic behavior (encapsulated in $F$ ).
All thermodynamic observables, such as average energy, entropy, and pressure, can be systematically derived as partial derivatives of $F$ , and thus of $ln Z$ .

We have now established the full machinery of the canonical ensemble for systems with a fixed number of particles. In our next lesson, we will generalize our framework further to systems that can exchange not only energy but also particles with their environment. This will lead us to the grand canonical ensemble, another powerful tool with its own partition function and corresponding thermodynamic potential.

Recommended resources

1. Thermodynamic Potentials and the Helmholtz Free Energy

2. The Bridge: Deriving the Relation F=−kB​TlnZ

3. The Physical Significance of Free Energy

4. The Partition Function as a Generating Function

Conclusion

2. The Bridge: Deriving the Relation $F = - k_{B} T ln Z$