Deriving Hartree-Fock Equations from the Variational Principle

Hello! Welcome to the first lesson in our module on "Many-Body Mean-Field and Correlation Methods."

In our previous module, we established the fundamental principles for handling systems of identical particles, culminating in our analysis of how indistinguishability creates observable interference effects in scattering. The core takeaway was that the mandatory antisymmetry of the many-electron wavefunction is not just a formal requirement but has profound physical consequences.

Today, we transition from the dynamics of two-particle collisions to the static properties of a much larger system, such as an atom, a molecule, or the electron gas in a solid. Our central challenge is to solve the Schrödinger equation for many interacting electrons. As you know, this is an impossible task to perform exactly for all but the simplest systems. We must therefore rely on approximations.

This lesson addresses the learning outcome: Derive the Hartree-Fock equations for a many-electron system using the variational principle. We will develop the most fundamental and historically important mean-field theory. The "mean-field" idea is to approximate the complex, instantaneous interactions between all electrons with a simpler picture: each electron moves independently in an average potential created by all the other electrons. The exchange effects we saw in scattering will reappear here as a crucial, purely quantum-mechanical component of this average potential.

1. The Challenge and the Ansatz

Our starting point is the full electronic Hamiltonian for an N-electron system. Within the Born-Oppenheimer approximation (where we consider the atomic nuclei to be fixed in space), the Hamiltonian is:

$$\hat{H}=\sum _{i=1}^N\left(-\frac{{\hslash }^2}{2m}{\nabla }_i^2-\sum _A\frac{Z_A{e}^2}{4\pi {ϵ}_0|{\mathbf{r}}_i-{\mathbf{R}}_A|}\right)+\sum _{i<j}^N\frac{e^2}{4\pi {ϵ}_0|{\mathbf{r}}_i-{\mathbf{r}}_j|}$$

This can be written more compactly as:

$$\hat{H}=\sum _{i=1}^N\hat{h}(i)+\sum _{i<j}^N\hat{v}(i,j)$$

where $\hat{h}(i)$ is the single-electron part (kinetic energy and electron-nuclear attraction), and $\hat{v}(i,j)$ is the two-electron repulsion term. It is this second term that couples the motions of all electrons and makes the problem intractable.

How do we construct an approximate wavefunction $\Psi ({\mathbf{x}}_1,\dots ,{\mathbf{x}}_N)$ for this Hamiltonian? (Here, ${\mathbf{x}}_i$ includes both spatial and spin coordinates, ${\mathbf{x}}_i=({\mathbf{r}}_i,{\sigma }_i)$).

A first, naive guess might be a simple product of single-particle wavefunctions, or spin orbitals ${\chi }_i(\mathbf{x})$. This is known as a Hartree product:

$${\Psi }_{\text{HP}}({\mathbf{x}}_1,\dots ,{\mathbf{x}}_N)={\chi }_1({\mathbf{x}}_1){\chi }_2({\mathbf{x}}_2)\dots {\chi }_N({\mathbf{x}}_N)$$

However, this form has a fatal flaw: it violates the antisymmetry principle. Exchanging two electrons, say 1 and 2, results in a different state, which is forbidden for identical fermions.

The correct starting point, or ansatz, is a Slater determinant. It is the simplest possible N-electron wavefunction that correctly incorporates the Pauli exclusion principle.

$${\Psi }_{\text{SD}}({\mathbf{x}}_1,\dots ,{\mathbf{x}}_N)=\frac{1}{\sqrt{N!}}\left|\begin{array}{cccc}{\chi }_1({\mathbf{x}}_1)& {\chi }_2({\mathbf{x}}_1)& \cdots & {\chi }_N({\mathbf{x}}_1)\\ {\chi }_1({\mathbf{x}}_2)& {\chi }_2({\mathbf{x}}_2)& \cdots & {\chi }_N({\mathbf{x}}_2)\\ ⋮& ⋮& \ddots & ⋮\\ {\chi }_1({\mathbf{x}}_N)& {\chi }_2({\mathbf{x}}_N)& \cdots & {\chi }_N({\mathbf{x}}_N)\end{array}\right|$$

The properties of determinants automatically ensure that if you swap any two electrons (swapping two rows), the wavefunction picks up a minus sign. It also ensures that if any two electrons occupy the same spin orbital (two columns are identical), the wavefunction vanishes.

To solidify your understanding of these foundational concepts, please watch the following video clips.

2. The Variational Principle and the Energy Functional

Now that we have our ansatz—the Slater determinant—how do we find the best possible set of spin orbitals $\{{\chi }_i\}$ that make up this determinant? We use the variational principle. It states that the expectation value of the energy for any trial wavefunction ${\Psi }_{\text{trial}}$ is always greater than or equal to the true ground state energy $E_0$:

$$E[{\Psi }_{\text{trial}}]=\frac{⟨{\Psi }_{\text{trial}}|\hat{H}|{\Psi }_{\text{trial}}⟩}{⟨{\Psi }_{\text{trial}}|{\Psi }_{\text{trial}}⟩}\ge E_0$$

Our goal is to find the set of orthonormal spin orbitals $\{{\chi }_i\}$ that minimizes this energy functional for a single Slater determinant. The resulting energy will be the best possible approximation to $E_0$ within the limitation that the wavefunction is a single determinant.

Calculating the expectation value $⟨{\Psi }_{\text{SD}}|\hat{H}|{\Psi }_{\text{SD}}⟩$ is a non-trivial exercise (involving the Slater-Condon rules, which you may have seen). The result for the total energy is:

$$E_{\text{HF}}=\sum _{i=1}^N⟨{\chi }_i|\hat{h}|{\chi }_i⟩+\frac{1}{2}\sum _{i,j=1}^N\left(⟨{\chi }_i{\chi }_j|\hat{v}|{\chi }_i{\chi }_j⟩-⟨{\chi }_i{\chi }_j|\hat{v}|{\chi }_j{\chi }_i⟩\right)$$

Here, the two-electron integrals are defined as:

• Direct (Coulomb) Integral, $J_{ij}$: $⟨{\chi }_i{\chi }_j|\hat{v}|{\chi }_i{\chi }_j⟩=\int d{\mathbf{x}}_{1}d{\mathbf{x}}_2{\chi }_i^{\ast }({\mathbf{x}}_1){\chi }_j^{\ast }({\mathbf{x}}_2)\frac{e^2}{4\pi {ϵ}_0{r}_{12}}{\chi }_i({\mathbf{x}}_1){\chi }_j({\mathbf{x}}_2)$
• Exchange Integral, $K_{ij}$: $⟨{\chi }_i{\chi }_j|\hat{v}|{\chi }_j{\chi }_i⟩=\int d{\mathbf{x}}_{1}d{\mathbf{x}}_2{\chi }_i^{\ast }({\mathbf{x}}_1){\chi }_j^{\ast }({\mathbf{x}}_2)\frac{e^2}{4\pi {ϵ}_0{r}_{12}}{\chi }_j({\mathbf{x}}_1){\chi }_i({\mathbf{x}}_2)$

So, the total energy can be written as:

$$E_{\text{HF}}=\sum _{i=1}^N{h}_{ii}+\frac{1}{2}\sum _{i,j=1}^N(J_{ij}-K_{ij})$$

The direct term $J_{ij}$ has a simple classical interpretation: it's the electrostatic repulsion between the charge cloud $|{\chi }_i{|}^2$ and the charge cloud $|{\chi }_j{|}^2$. The exchange term $K_{ij}$ has no classical analogue. It is a direct consequence of the antisymmetry of the wavefunction and acts as a correction that lowers the energy. Notice that $K_{ij}$ is non-zero only for electrons of the same spin (the spin part of the integral would otherwise be zero due to orthogonality), which is a manifestation of the Pauli exclusion principle keeping same-spin electrons apart.

3. Deriving the Hartree-Fock Equations

Our task is now to minimize $E_{\text{HF}}$ with respect to the choice of orbitals $\{{\chi }_i\}$, subject to the constraint that these orbitals remain orthonormal, $⟨{\chi }_i|{\chi }_j⟩={\delta }_{ij}$. This is a constrained optimization problem, which can be solved using the method of Lagrange multipliers.

We seek to minimize the functional:

$$\mathcal{L}[\{{\chi }_i\}]=E_{\text{HF}}-\sum _{i,j=1}^N{ϵ}_{ij}\left(⟨{\chi }_i|{\chi }_j⟩-{\delta }_{ij}\right)$$

where ${ϵ}_{ij}$ are the Lagrange multipliers. Setting the functional derivative $\delta \mathcal{L}/\delta {\chi }_k^{\ast }$ to zero for an arbitrary orbital ${\chi }_k$ leads to a set of equations. After some algebra, these equations can be written in a compact form:

$$\hat{f}({\mathbf{x}}_1){\chi }_i({\mathbf{x}}_1)=\sum _{j=1}^N{ϵ}_{ij}{\chi }_j({\mathbf{x}}_1)$$

This is the general Hartree-Fock equation. The operator $\hat{f}$ is called the Fock operator.

It can be shown that there is a unitary transformation of the orbitals that diagonalizes the matrix of Lagrange multipliers ${ϵ}_{ij}$ without changing the total wavefunction (the Slater determinant). In this new basis, the equations take on a simpler, pseudo-eigenvalue form known as the canonical Hartree-Fock equations:

$$\hat{f}({\mathbf{x}}_1){\chi }_i({\mathbf{x}}_1)={ϵ}_i{\chi }_i({\mathbf{x}}_1)$$

where ${ϵ}_i$ are the diagonal elements of the transformed Lagrange multiplier matrix, known as the orbital energies.

The Fock Operator

The Fock operator is an effective one-electron Hamiltonian defined as:

$$\hat{f}({\mathbf{x}}_1)=\hat{h}({\mathbf{x}}_1)+{\hat{v}}_{\text{HF}}({\mathbf{x}}_1)=\hat{h}({\mathbf{x}}_1)+\sum _{j=1}^N\left({\hat{J}}_j({\mathbf{x}}_1)-{\hat{K}}_j({\mathbf{x}}_1)\right)$$

Let's break it down:

• $\hat{h}({\mathbf{x}}_1)$: The usual one-electron operator (kinetic energy + nuclear attraction).
• ${\hat{v}}_{\text{HF}}({\mathbf{x}}_1)$: The effective Hartree-Fock potential.
  • Coulomb (Hartree) Operator ${\hat{J}}_j({\mathbf{x}}_1)$: $${\hat{J}}_j({\mathbf{x}}_1){\chi }_i({\mathbf{x}}_1)=\left[\int d{\mathbf{x}}_2{\chi }_j^{\ast }({\mathbf{x}}_2)\frac{e^2}{4\pi {ϵ}_0{r}_{12}}{\chi }_j({\mathbf{x}}_2)\right]{\chi }_i({\mathbf{x}}_1)$$ This is a simple local potential. It represents the potential energy of electron 1 due to the average charge distribution of electron $j$.
  • Exchange (Fock) Operator ${\hat{K}}_j({\mathbf{x}}_1)$: $${\hat{K}}_j({\mathbf{x}}_1){\chi }_i({\mathbf{x}}_1)=\left[\int d{\mathbf{x}}_2{\chi }_j^{\ast }({\mathbf{x}}_2)\frac{e^2}{4\pi {ϵ}_0{r}_{12}}{\chi }_i({\mathbf{x}}_2)\right]{\chi }_j({\mathbf{x}}_1)$$ This is a much stranger object. It is a non-local integral operator: to find its effect on ${\chi }_i$ at position ${\mathbf{x}}_1$, we need to know the value of ${\chi }_i$ over all of space.

The following image provides a concise summary of the derivation and the structure of the Fock operator.

Derivation in Second Quantization

Given your interest in advanced many-body theory, it's valuable to see how this derivation is handled in the more powerful formalism of second quantization. The derivation is more abstract but also more elegant.

The second quantization approach makes it clear that the Hartree-Fock approximation is equivalent to finding a basis of single-particle states where the Hamiltonian does not cause transitions between the ground state and any state with a single particle-hole excitation.

Test your understanding!

Why are the Hartree-Fock equations not standard eigenvalue equations, and why can't we solve them directly?

Show answer

The Hartree-Fock equations, $\hat{f}{\chi }_i={ϵ}_i{\chi }_i$, are not standard eigenvalue equations because the Fock operator $\hat{f}$ itself depends on the solutions $\{{\chi }_j\}$ we are trying to find. The Coulomb operator ${\hat{J}}_j$ and the Exchange operator ${\hat{K}}_j$ are constructed from the full set of occupied orbitals. This creates a non-linear, self-referential problem. We cannot know the operator without knowing the solutions, and we cannot find the solutions without knowing the operator.

Conclusion

In this lesson, we have derived the central equations of Hartree-Fock theory. We started with the fundamental challenge of the many-body problem and motivated the use of a single Slater determinant as a physically reasonable starting approximation.

Key Takeaways:

• The Hartree-Fock method seeks the best single Slater determinant approximation to the true many-electron ground state.
• The "best" determinant is found by applying the variational principle: we minimize the expectation value of the Hamiltonian by varying the single-particle spin orbitals that constitute the determinant.
• This constrained minimization leads to the Hartree-Fock equations, a set of effective single-particle Schrödinger equations.
• The effective Hamiltonian is the Fock operator, which describes an electron moving in the potential of the nuclei plus a mean field generated by all other electrons.
• This mean field has two parts: a classical-like Coulomb repulsion (the Hartree term) and a purely quantum mechanical, non-local exchange (Fock) term that arises from the wavefunction's antisymmetry.

Preview of the Next Lesson:

The derivation of these equations is only the first step. The next crucial question is: how do we actually solve these non-linear, self-consistent equations? In the next lesson, we will interpret the physical significance of the Hartree and Fock terms in more detail and outline the Self-Consistent Field (SCF) iterative procedure used to solve the Hartree-Fock equations in practice. We will also introduce the concept of correlation energy—precisely what Hartree-Fock theory leaves out.