Eigenvalues and Eigenvectors: Calculation and Application

Hello! Welcome to your next lesson in the "Mathematical Foundations: Linear Algebra" module.

In our last few lessons, we've been investigating key properties of matrices. We learned how the determinant tells us if a matrix is invertible, and how the rank gives us a measure of a matrix's "true" dimensionality. These concepts are about understanding the matrix as a whole.

Today, we'll explore a new, powerful concept that reveals how a matrix transformation acts on specific, special vectors. This lesson will cover the learning outcome: Define eigenvalues and eigenvectors and solve for them in 2x2 and 3x3 matrices. This topic is a cornerstone of linear algebra and has direct applications in data science, most notably in Principal Component Analysis (PCA), a technique used to reduce the dimensionality of data, which you will learn about later.

1. The Core Idea: What are Eigenvectors and Eigenvalues?

When we multiply a matrix by a vector, we are performing a linear transformation. Most vectors, when transformed, will be knocked off their original line (their "span"). They get both rotated and stretched.

However, for any given square matrix, there are special vectors that do not get rotated. When the transformation is applied, these vectors stay on their original line; they are only stretched, squished, or flipped. These special vectors are called eigenvectors. The factor by which they are stretched or squished is called the eigenvalue.

To get a strong visual intuition for this, let's turn to one of the best explanations available.

Eigenvectors and eigenvalues | Chapter 14, Essence of linear algebra

This video from 3Blue1Brown provides a fantastic geometric interpretation of eigenvalues and eigenvectors. It will help you build a solid conceptual foundation before we dive into the calculations.

Watch the segment from 01:39 to 05:15. Focus on understanding: The visual difference between vectors that get knocked off their span and those that stay on it. How an eigenvector is a vector whose direction is unchanged by the transformation. How an eigenvalue is the scalar factor by which the eigenvector is stretched or shrunk.

2. The Formal Definition and the Characteristic Equation

As you saw in the video, the relationship between a matrix $A$ , its eigenvector $v$ , and its corresponding eigenvalue $λ$ is defined by a simple, elegant equation:

A v = λ v

Where:

$A$ is an $n \times n$ square matrix.
$v$ is a non-zero $n \times 1$ column vector (the eigenvector).
$λ$ is a scalar (the eigenvalue).

Definition and Visual Representation of Eigenvalues and Eigenvectors — This image illustrates the core equation $A\mathbf{x} = \lambda\mathbf{x}$. On the left, $A\mathbf{x}$ and $\mathbf{x}$ point in the same direction, meaning $\lambda$ is positive. On the right, they point in opposite directions, meaning $\lambda$ is negative.

Our goal is to find the pairs of $λ$ and $v$ that satisfy this equation for a given matrix $A$ . To do this, we need to rearrange the equation.

Start with $A v = λ v$ .
Subtract $λ v$ from both sides: $A v - λ v = 0$ .
To factor out $v$ , we rewrite $λ v$ as $(λ I) v$ , where $I$ is the identity matrix. This is like turning the scalar $λ$ into a matrix that scales vectors, so the matrix-vector multiplication makes sense. $A v - (λ I) v = 0$
Now, factor out $v$ : $(A - λ I) v = 0$

This is a crucial step. We are looking for a non-zero eigenvector $v$ . This equation is a homogeneous system of linear equations. You might recall from the lesson on determinants that a system like this has a non-trivial (non-zero) solution only if the matrix $(A - λ I)$ is singular (i.e., not invertible). And a matrix is singular if and only if its determinant is zero.

This gives us the master key to finding eigenvalues:

The Characteristic Equation: $det (A - λ I) = 0$

By solving this equation, we can find the eigenvalues $λ$ .

3. How to Find Eigenvalues and Eigenvectors

The process is always the same two steps:

Find the eigenvalues ( $λ$ ): Set up and solve the characteristic equation $det (A - λ I) = 0$ .
Find the eigenvectors ( $v$ ): For each eigenvalue you found, plug it back into the equation $(A - λ I) v = 0$ and solve for $v$ .

Let's see this process in action.

Step-by-Step Calculation for a 2x2 Matrix

This video will walk you through the entire process for a 2x2 matrix. It clearly separates finding the eigenvalues from finding the eigenvectors.

Finding Eigenvalues and Eigenvectors

Watch these segments from the 'Professor Dave Explains' channel. He provides a very clear, step-by-step calculation for finding eigenvalues and their corresponding eigenvectors.

First, watch from 02:51 to 06:31 to see how to form and solve the characteristic equation to find the eigenvalues. Then, continue from 06:31 to 11:27 to see how each eigenvalue is used to find its corresponding eigenvectors by solving a system of equations.

To reinforce this, you can also review the step-by-step example in the following article.

Eigenvalues and Eigenvectors

The article 'Eigenvalues and Eigenvectors' on GeeksForGeeks provides a concise text-based walkthrough of the same process. It's a good way to review the steps at your own pace.

Read the section titled 'Eigenvector of a 2 × 2 matrix'. Follow the example provided for the matrix A = [[1, 2], [5, 4]]. Compare the steps with what you saw in the video.

Test your understanding!

Let's find the eigenvalues and eigenvectors for the matrix $A = (21 7 - 4)$ .

Part 1: Find the eigenvalues.
Part 2: Find the corresponding eigenvectors for each eigenvalue.

Show answer

Part 1: Find the Eigenvalues

Set up the characteristic equation $det (A - λ I) = 0$ . $det ((21 7 - 4) - λ (1001)) = det (2 - λ 1 7 - 4 - λ) = 0$
Calculate the determinant. $(2 - λ) (- 4 - λ) - (7) (1) = 0$ $- 8 - 2 λ + 4 λ + λ^{2} - 7 = 0$ $λ^{2} + 2 λ - 15 = 0$
Solve the quadratic equation for $λ$ . We can factor this: $(λ + 5) (λ - 3) = 0$ The eigenvalues are $λ_{1} = 3$ and $λ_{2} = - 5$ .

Part 2: Find the Eigenvectors

Case 1: For $λ_{1} = 3$
Solve $(A - 3 I) v = 0$ .

(2 - 3 1 7 - 4 - 3) (x y) = (- 1 1 7 - 7) (x y) = (00)

Both rows give the same equation: $- x + 7 y = 0$ , or $x = 7 y$ . If we choose $y = 1$ , then $x = 7$ .
So, an eigenvector for $λ_{1} = 3$ is $v_{1} = (71)$ .

Case 2: For $λ_{2} = - 5$
Solve $(A - (- 5) I) v = 0$ .

(2 - (- 5) 1 7 - 4 - (- 5)) (x y) = (7171) (x y) = (00)

Both rows give the same equation: $x + y = 0$ , or $x = - y$ . If we choose $y = 1$ , then $x = - 1$ .
So, an eigenvector for $λ_{2} = - 5$ is $v_{2} = (- 1 1)$ .

Calculation for a 3x3 Matrix

The process for a 3x3 matrix is identical, but the calculations for the determinant and for solving the system of equations are more complex. Let's watch Professor Dave work through one.

Finding Eigenvalues and Eigenvectors

Now let's apply the same two-step process to a 3x3 matrix. Pay close attention to how the 3x3 determinant is calculated and how the resulting system of three equations is solved.

Watch from 11:27 to the end of the video (16:06). Notice how the structure of the matrix in his example simplifies finding the eigenvalues. Then see how he finds the eigenvector for each of the three eigenvalues.

Conclusion

In this lesson, you've learned about one of the most important concepts in linear algebra. You can now not only define eigenvalues and eigenvectors but also compute them for 2x2 and 3x3 matrices.

Key Takeaways:

Eigenvectors are special vectors that, when acted upon by a matrix, do not change their direction (span).
Eigenvalues are the scalar factors by which their corresponding eigenvectors are stretched or shrunk.
The relationship is captured by the equation $A v = λ v$ .
To find eigenvalues, you solve the characteristic equation: $det (A - λ I) = 0$ .
To find the eigenvector for a specific eigenvalue $λ$ , you solve the system of linear equations $(A - λ I) v = 0$ .

In our next lesson, we will move on to another powerful matrix decomposition technique called Singular Value Decomposition (SVD). SVD can be seen as a generalization of the eigenvalue concepts you learned today, but it has the major advantage of working on any matrix, not just square ones. It is one of the most widely used tools in modern data science and machine learning.