Algebraic vs. Geometric Multiplicity

Hello! Welcome to the next lesson in our exploration of eigenvalues.

In our last session, we mastered the core algorithm for finding eigenvalues and eigenvectors. We started with the geometric idea of an invariant direction ( $A x = λ x$ ) and translated it into a concrete computational method: first, solve the characteristic equation $det (A - λ I) = 0$ for eigenvalues $λ$ , and then find the null space of $(A - λ I)$ for the corresponding eigenvectors $x$ .

We ended with a critical question: What happens if the characteristic polynomial has a repeated root, like $(λ - 3)^{2} = 0$ ? Does this mean we get two linearly independent eigenvectors for $λ = 3$ ? The answer, perhaps surprisingly, is "sometimes, but not always."

Today, our goal is to distinguish between the algebraic and geometric multiplicity of an eigenvalue. This distinction is the precise tool we need to answer that question. Understanding it is the key that unlocks the concept of diagonalization, which is fundamental to applications in both deep learning and quantum mechanics.

1. Algebraic Multiplicity: Counting Roots

The first type of multiplicity is a straightforward concept you've likely encountered in algebra.

The algebraic multiplicity (AM) of an eigenvalue $λ$ is the number of times it appears as a root in the characteristic polynomial.

For example, if the characteristic polynomial of a 4x4 matrix is $p (λ) = (λ - 5)^{2} (λ + 1) (λ - 2)$ , then:

The eigenvalue $λ = 5$ has an algebraic multiplicity of 2.
The eigenvalues $λ = - 1$ and $λ = 2$ each have an algebraic multiplicity of 1.

Think of it like a quadratic function $y = (x - c)^{2}$ that just touches the x-axis at $x = c$ . We say it has a single root with a multiplicity of two. The algebraic multiplicity of an eigenvalue is the same idea.

2. Geometric Multiplicity: Counting Eigenvectors

The second type of multiplicity is more geometric and relates directly to the eigenvectors.

The geometric multiplicity (GM) of an eigenvalue $λ$ is the number of linearly independent eigenvectors associated with it. This is equivalent to the dimension of the eigenspace for $λ$ .

Recall that the eigenspace for $λ$ is the null space of the matrix $(A - λ I)$ . Therefore, the geometric multiplicity is simply the dimension of this null space:

GM (λ) = dim (Null (A - λ I))

Since you have a computer science background, you're familiar with solving systems of linear equations and the concept of free variables. This gives us a very practical way to think about geometric multiplicity:

The geometric multiplicity of $λ$ is the number of free variables in the reduced row echelon form of the matrix $(A - λ I)$ .

Each free variable allows you to generate a new, linearly independent basis vector for the null space—which is to say, a new, linearly independent eigenvector.

The following video provides a concise and intuitive explanation of both concepts and demonstrates how they can differ using simple 2x2 examples.

DIRTY Shortcut to Find Geometric Multiplicity [Passing Linear Algebra]

Watch this video from STEM Support for a quick and clear introduction to algebraic and geometric multiplicity. It uses the 'repeated root' analogy for AM and introduces the very useful 'free variable' shortcut for GM.

Watch the entire video (8:33). It provides two excellent contrasting examples. In the first, AM = GM. In the second, AM > GM. This clearly illustrates that the two multiplicities are not always equal.

3. The Key Relationship and "Defective" Eigenvalues

As you saw in the video, the algebraic and geometric multiplicities for an eigenvalue are not always the same. However, they are not independent. There is a fundamental theorem that connects them:

For any eigenvalue $λ$ of a matrix $A$ , its geometric multiplicity is always greater than or equal to 1, but less than or equal to its algebraic multiplicity.

1 \leq GM (λ) \leq AM (λ)

This makes intuitive sense: an eigenvalue with an algebraic multiplicity of $k$ gives us at most $k$ dimensions worth of eigenvectors, but it might "collapse" and provide fewer.

When the geometric multiplicity of an eigenvalue is strictly less than its algebraic multiplicity ( $GM (λ) < AM (λ)$ ), the eigenvalue is called defective. A matrix with one or more defective eigenvalues is sometimes called a defective matrix.

Let's work through a 3x3 example to solidify the calculation process.

Algebraic and Geometric Multiplicity of Eigenvalues | Linear Algebra

The video 'Algebraic and Geometric Multiplicity of Eigenvalues' from Wrath of Math demonstrates the full process on a 3x3 matrix. This will reinforce how to find the basis of the eigenspace to determine the GM.

Watch from 0:30 to 4:02. The video first defines AM and GM, then computes them for a 3x3 matrix with eigenvalues λ=3 (AM=2) and λ=5 (AM=1). Pay close attention to how he finds the basis for the eigenspace of λ=3 to show that its GM is 2.

Test your understanding!

A 3x3 matrix $A$ has the characteristic polynomial $p (λ) = - (λ - 4)^{2} (λ - 2)$ . After finding the eigenvalues, you calculate the matrix $A - 4 I$ and reduce it to row echelon form:

A - 4 I = 100 - 2 00 310

What are the algebraic multiplicities of the eigenvalues $λ = 4$ and $λ = 2$ ?
What is the geometric multiplicity of the eigenvalue $λ = 4$ ?
Is the eigenvalue $λ = 4$ defective?

Show answer

Algebraic Multiplicities: From the characteristic polynomial $p (λ) = - (λ - 4)^{2} (λ - 2)$ , the root $λ = 4$ appears twice, and the root $λ = 2$ appears once.
- AM( $λ = 4$ ) = 2
- AM( $λ = 2$ ) = 1
Geometric Multiplicity: We look at the reduced matrix for $λ = 4$ . The system $(A - 4 I) x = 0$ has two pivot columns (the first and third). This means there is $3 - 2 = 1$ free variable ( $x_{2}$ ). The number of free variables is the dimension of the null space.
- GM( $λ = 4$ ) = 1
Defective? Yes. Since GM( $λ = 4$ ) = 1 is strictly less than AM( $λ = 4$ ) = 2, the eigenvalue $λ = 4$ is defective. This matrix will not have enough linearly independent eigenvectors to form a basis for $R^{3}$ .

4. Distinguishing AM and GM: A Summary

The distinction between these two concepts is crucial. This article from GeeksForGeeks provides an excellent summary table that clearly lays out the differences.

Geometric Multiplicity in Linear Algebra

To consolidate your understanding, please read the following section from GeeksForGeeks. Focus on the table that directly compares geometric and algebraic multiplicity.

Navigate to the section titled 'Geometric Multiplicity vs Algebraic Multiplicity' and study the comparison table. This table is a great reference for quickly recalling the key differences.

This table is a valuable resource that summarizes the definition, calculation, and interpretation of both multiplicities.

Conclusion

Today, we've unpacked the crucial difference between the algebraic and geometric multiplicity of an eigenvalue. This resolves the question we posed at the start: a repeated root in the characteristic polynomial does not guarantee a corresponding number of independent eigenvectors.

Key Takeaways:

Algebraic Multiplicity (AM): How many times $λ$ is a root of the characteristic polynomial. It's a property of the polynomial.
Geometric Multiplicity (GM): How many linearly independent eigenvectors exist for $λ$ . It's the dimension of the eigenspace $Null (A - λ I)$ and can be found by counting the free variables.
The Golden Rule: For any eigenvalue, $1 \leq GM \leq AM$ .
Defective Eigenvalues: An eigenvalue is defective if its geometric multiplicity is less than its algebraic multiplicity ( $GM < AM$ ).

Preview of the Next Lesson:

The concepts of AM and GM are not just theoretical curiosities; they are the gatekeepers to one of the most powerful ideas in linear algebra: diagonalization. As you may have guessed, a matrix can be diagonalized if and only if it is not defective—that is, if the geometric and algebraic multiplicities are equal for all of its eigenvalues. In our next lesson, we will formally define what it means to diagonalize a matrix and explore why this condition is so important.

Recommended resources

1. Algebraic Multiplicity: Counting Roots

2. Geometric Multiplicity: Counting Eigenvectors

3. The Key Relationship and "Defective" Eigenvalues

4. Distinguishing AM and GM: A Summary

Conclusion