Fluctuation-Dissipation Theorem and Thermodynamic Limit

In our previous lesson, we introduced the concept of ensemble equivalence, arguing that for macroscopic systems, the choice of ensemble—microcanonical, canonical, or grand canonical—does not alter the resulting thermodynamics. The physical justification we offered was that fluctuations of extensive quantities like energy become vanishingly small relative to their mean values in the thermodynamic limit.

Today, we will place this crucial idea on a firm mathematical footing. Our learning outcome is to show that relative fluctuations of extensive quantities (e.g., energy) vanish in the thermodynamic limit, scaling as $1/ N$ . We will begin by rigorously deriving the relationship between energy fluctuations and heat capacity in the canonical ensemble. We will then extend this reasoning to particle number fluctuations in the grand canonical ensemble and finally formulate a general theorem connecting fluctuations to thermodynamic response functions.

1. Energy Fluctuations in the Canonical Ensemble

Let's start with a system in contact with a heat bath at temperature $T$ , described by the canonical ensemble. The energy $E$ of the system is not fixed but fluctuates. Our goal is to quantify the size of these fluctuations.

Recall that the probability of the system being in a microstate $i$ with energy $E_{i}$ is given by the Boltzmann distribution, $P_{i} = e^{- β E_{i}} / Z$ , where $β = 1/ (k_{B} T)$ and $Z$ is the canonical partition function.

The average energy $⟨ E ⟩$ is the first moment of this distribution:

⟨ E ⟩ = i \sum E_{i} P_{i} = \frac{1}{Z} i \sum E_{i} e^{- β E_{i}}

Notice that the sum $\sum_{i} E_{i} e^{- β E_{i}}$ can be obtained by differentiating the partition function $Z = \sum_{i} e^{- β E_{i}}$ with respect to $β$ . This allows us to write the average energy in a compact and powerful form:

⟨ E ⟩ = - \frac{1}{Z} \frac{\partial Z}{\partial β} = - \frac{\partial ln Z}{\partial β}

This expression elegantly connects a macroscopic observable ( $⟨ E ⟩$ ) to the partition function. Given your background, you'll recognize $ln Z$ as a cumulant-generating function for the energy distribution (scaled by $- β$ ).

To find the fluctuation, we need the variance, $⟨(Δ E)^{2} ⟩ = ⟨ E^{2} ⟩ - ⟨ E ⟩^{2}$ . We can calculate the second moment, $⟨ E^{2} ⟩$ , in a similar way:

⟨ E^{2} ⟩ = i \sum E_{i}^{2} P_{i} = \frac{1}{Z} i \sum E_{i}^{2} e^{- β E_{i}} = \frac{1}{Z} \frac{\partial ^{2} Z}{\partial β ^{2}}

Now, we can assemble the variance. A bit of calculus shows that the variance is simply the second derivative of the cumulant-generating function:

⟨(Δ E)^{2} ⟩ = \frac{1}{Z} \frac{\partial ^{2} Z}{\partial β ^{2}} - (\frac{1}{Z} \frac{\partial Z}{\partial β})^{2} = \frac{\partial}{\partial β} (\frac{1}{Z} \frac{\partial Z}{\partial β}) = \frac{\partial ^{2} ln Z}{\partial β ^{2}}

This gives us the energy variance in terms of $β$ . To connect it to a more familiar thermodynamic quantity, we use the chain rule to switch the derivative from $β$ to $T$ :

⟨(Δ E)^{2} ⟩ = \frac{\partial}{\partial β} (- ⟨ E ⟩) = - \frac{d T}{d β} \frac{\partial ⟨ E ⟩}{\partial T} = - (- k_{B} T^{2}) \frac{\partial ⟨ E ⟩}{\partial T}

Recognizing that the heat capacity at constant volume is $C_{V} = (\partial ⟨ E ⟩ / \partial T)_{V}$ , we arrive at a celebrated result:

⟨(Δ E)^{2} ⟩ = k_{B} T^{2} C_{V}

This is a specific instance of a fluctuation-response theorem: the magnitude of spontaneous equilibrium fluctuations ( $⟨(Δ E)^{2} ⟩$ ) is determined by how the system responds ( $C_{V}$ ) to an external perturbation (a change in temperature).

Now we can prove the scaling law. For a typical system with short-range interactions, both the average energy $⟨ E ⟩$ and the heat capacity $C_{V}$ are extensive quantities, meaning they are proportional to the number of particles $N$ :

$⟨ E ⟩ \propto N$
$C_{V} \propto N$

The standard deviation of the energy, $σ_{E} = ⟨(Δ E)^{2} ⟩$ , therefore scales as:

σ_{E} = k_{B} T^{2} C_{V} \propto N

The relative fluctuation is the ratio of the standard deviation to the mean:

\frac{σ _{E}}{⟨ E ⟩} \propto \frac{N}{N} = \frac{1}{N}

As the system size approaches the thermodynamic limit ( $N \to \infty$ ), this ratio vanishes. This confirms our intuition: for a macroscopic system, the energy distribution in the canonical ensemble is so sharply peaked around its mean that it becomes experimentally indistinguishable from the fixed energy of a microcanonical ensemble.

The following reading provides a concise derivation of this result.

Chapter 9 Canonical ensemble

This section from a set of lecture notes provides a direct and complete derivation of the relationship between energy fluctuations and heat capacity, culminating in the scaling argument we just discussed.

Please read the section titled "Energy fluctuations". Follow the derivation of the key result, Eq. (9.23), which states that C_V = \frac{1}{k_B T^2} (\langle E^2 \rangle - \langle E \rangle^2). Then, pay close attention to the subsequent argument for the scaling of relative fluctuations.

For a more deliberate, step-by-step walkthrough of the algebraic manipulation, you can also watch the following segments of this video from the Advanced Physics channel.

Statistical Mechanics - Canonical Ensemble : Energy Fluctuations in Canonical Ensemble

This video methodically works through the differentiation and substitutions required for the derivation.

The core algebra begins by differentiating the expression for mean energy to arrive at the mean square fluctuation. The lecturer then relates this fluctuation to the specific heat capacity, C_V. Finally, the crucial scaling argument showing the 1/√N dependence is presented.

2. Generalizing the Fluctuation-Response Relation

The relationship we found for energy is not unique. The same structure applies to any extensive quantity that can fluctuate when the system is coupled to a corresponding reservoir.

Particle Number Fluctuations

Consider a system in the grand canonical ensemble, in contact with a heat and particle reservoir at temperature $T$ and chemical potential $μ$ . The number of particles $N$ now fluctuates. The grand partition function is $Ξ = \sum_{N, i} e^{- β (E_{i, N} - μ N)}$ .

Following the same logic as before, we can express the average particle number $⟨ N ⟩$ and its variance $⟨(Δ N)^{2} ⟩$ as derivatives of $ln Ξ$ :

⟨ N ⟩ = \frac{1}{β} \frac{\partial ln Ξ}{\partial μ} and ⟨(Δ N)^{2} ⟩ = \frac{1}{β ^{2}} \frac{\partial ^{2} ln Ξ}{\partial μ ^{2}}

This again implies a fluctuation-response relation:

⟨(Δ N)^{2} ⟩ = \frac{1}{β} \frac{\partial ⟨ N ⟩}{\partial μ}_{T, V}

The response function $(\partial ⟨ N ⟩ / \partial μ)_{T, V}$ can be related to the isothermal compressibility $κ_{T} = - \frac{1}{V} (\frac{\partial V}{\partial P})_{T, N}$ . A standard thermodynamic manipulation shows that $(\frac{\partial ⟨ N ⟩}{\partial μ})_{T, V} = \frac{⟨ N ⟩ ^{2}}{V} k_{B} T κ_{T}$ . This gives:

⟨(Δ N)^{2} ⟩ = \frac{⟨ N ⟩ ^{2} k _{B} T κ _{T}}{V}

For a typical system, $⟨ N ⟩$ , and $V$ are extensive ( $\propto N$ ), while $κ_{T}$ is intensive (independent of $N$ ). Thus, the variance scales as $⟨(Δ N)^{2} ⟩ \propto N^{2} / N \propto N$ . The standard deviation $σ_{N}$ scales as $N$ , and the relative fluctuation once again scales as:

\frac{σ _{N}}{⟨ N ⟩} \propto \frac{N}{N} = \frac{1}{N}

The General Theorem

This pattern can be formalized into a general theorem. Imagine a system with a Hamiltonian $H_{0}$ that is perturbed by adding a term $- f A$ , where $A$ is a macroscopic observable and $f$ is a conjugate "field". The total Hamiltonian is $H = H_{0} - f A$ . The partition function becomes $Z (f) = Tr (e^{- β (H_{0} - f A)})$ .

The average value of the observable $A$ is:

⟨ A ⟩ = \frac{1}{β} \frac{\partial ln Z ( f )}{\partial f}

And its variance is:

⟨(Δ A)^{2} ⟩ = \frac{1}{β ^{2}} \frac{\partial ^{2} ln Z ( f )}{\partial f ^{2}} = \frac{1}{β} \frac{\partial ⟨ A ⟩}{\partial f} = k_{B} T χ_{AA}

where $χ_{AA} = (\partial ⟨ A ⟩ / \partial f)$ is the generalized susceptibility of the observable $A$ with respect to its conjugate field $f$ .

If $A$ is an extensive observable, its conjugate response function $χ_{AA}$ will also be extensive ( $χ_{AA} \propto N$ ) for systems with short-range interactions away from a phase transition. This once again leads to the universal scaling of relative fluctuations: $σ_{A} / ⟨ A ⟩ \propto 1/ N$ .

The following reading expertly summarizes this generalization.

Fluctuations and Response 3. 1. Fluctuations in the energy ...

These notes from the University of Edinburgh generalize the result we found for energy. They first apply the same logic to particle number fluctuations and then state a general theorem, also noting an important exception.

Please start by reading the section on density fluctuations, which derives the scaling for particle number. Then, read the General Theorem that follows. Pay close attention to Note 3, which points out that this entire argument breaks down at a phase transition, where susceptibilities diverge, leading to large-scale fluctuations. This is a critical insight we will return to in later modules.

Conclusion

In this lesson, we have moved from a qualitative picture to a quantitative proof of why ensemble equivalence holds in the thermodynamic limit.

Key Takeaways:

Fluctuations of extensive observables are not arbitrary; they are directly proportional to a corresponding thermodynamic response function (e.g., $⟨(Δ E)^{2} ⟩ \propto C_{V}$ ). This is a fundamental fluctuation-response relation.
For extensive observables in systems with short-range interactions, the variance scales as $N$ , while the mean scales as $N$ .
This leads to the crucial result that the relative fluctuation—the ratio of the standard deviation to the mean—vanishes as $1/ N$ in the thermodynamic limit ( $N \to \infty$ ).
This scaling law is a general feature of statistical mechanics and is the mathematical foundation for the thermodynamic equivalence of ensembles.

We have seen that typical fluctuations shrink to become irrelevant for macroscopic measurements. But what about the atypical, rare, large fluctuations? How improbable are they? While the central limit theorem describes the typical fluctuations near the mean, large deviation theory provides the mathematical framework for quantifying the probability of these rare events.

In our next lesson, we will apply the large deviation principle to see how the canonical and grand canonical potentials (the free energies) emerge as "rate functions" that govern the exponentially small probability of observing macroscopic fluctuations away from the most likely state. This will provide our most profound and rigorous perspective yet on the relationship between ensembles.

Recommended resources

1. Energy Fluctuations in the Canonical Ensemble

2. Generalizing the Fluctuation-Response Relation

Particle Number Fluctuations

The General Theorem

Conclusion