Eigenvalues and Eigenvectors: Definition and Geometric Interpretation

1. The Core Idea: Invariant Directions

Imagine a matrix $A$ transforming a whole vector space. While most vectors are knocked off their original direction, some exceptional vectors might stay on their line through the origin. The transformation only scales them.

This simple geometric picture is the essence of eigenvalues and eigenvectors. The eigenvector points in a direction that is preserved by the matrix, and the eigenvalue tells us how much the vector is scaled along that direction.

2. Formal Definition of Eigenvalues and Eigenvectors

Let's translate this geometric idea into a precise mathematical statement.

As you've just read, the relationship is captured by the single, elegant equation:

$A\mathbf{x}=\lambda \mathbf{x}$

Where:

• $A$ is an $n\times n$ square matrix.
• $\mathbf{x}$ is a non-zero vector, called the eigenvector.
• $\lambda$ (lambda) is a scalar, called the eigenvalue corresponding to $\mathbf{x}$.

This equation states that when the matrix $A$ acts on its eigenvector $\mathbf{x}$, the result is simply the original vector $\mathbf{x}$ scaled by the eigenvalue $\lambda$. The word "eigen" is German for "own" or "self," which is fitting, as an eigenvector is a vector that is mapped to a scaled version of itself.

Verifying an Eigenvector

Given a matrix, a vector, and a potential eigenvalue, it's straightforward to check if they fit the definition. For example, let's test if $\mathbf{x}=\left[\begin{array}{c}1\\ 2\end{array}\right]$ is an eigenvector of $A=\left[\begin{array}{cc}3& 1\\ 2& 4\end{array}\right]$.

First, we compute the left side of the equation, $A\mathbf{x}$:
$A\mathbf{x}=\left[\begin{array}{cc}3& 1\\ 2& 4\end{array}\right]\left[\begin{array}{c}1\\ 2\end{array}\right]=\left[\begin{array}{c}3(1)+1(2)\\ 2(1)+4(2)\end{array}\right]=\left[\begin{array}{c}5\\ 10\end{array}\right]$

Now, we compare this result to the original vector $\mathbf{x}$. Is $\left[\begin{array}{c}5\\ 10\end{array}\right]$ a scaled version of $\left[\begin{array}{c}1\\ 2\end{array}\right]$? Yes, it is.

$\left[\begin{array}{c}5\\ 10\end{array}\right]=5\left[\begin{array}{c}1\\ 2\end{array}\right]$

Since $A\mathbf{x}=5\mathbf{x}$, we have confirmed that $\mathbf{x}=\left[\begin{array}{c}1\\ 2\end{array}\right]$ is an eigenvector of $A$, and its corresponding eigenvalue is $\lambda =5$.

3. The Geometric Significance of Eigenvalues

The value of $\lambda$ tells us exactly what happens to the eigenvector during the transformation:

• If $\lambda >1$, the eigenvector is stretched.
• If $\lambda =1$, the eigenvector is unchanged.
• If $0<\lambda <1$, the eigenvector is shrunk.
• If $\lambda =0$, the eigenvector is collapsed to the origin. This means eigenvectors with $\lambda =0$ are in the nullspace of the matrix.
• If $\lambda <0$, the eigenvector's direction is reversed (and its length is scaled by $|\lambda |$).

Let's explore this by revisiting some of the transformations from our last lesson. You can often identify eigenvectors and eigenvalues just by thinking about the geometry of the transformation.

Let's consolidate these ideas:

• Projection: Consider the matrix $P=\left[\begin{array}{cc}1& 0\\ 0& 0\end{array}\right]$, which projects vectors onto the x-axis.

  • Any vector already on the x-axis, like $\mathbf{x}=\left[\begin{array}{c}c\\ 0\end{array}\right]$, is unchanged by the projection. So, $P\mathbf{x}=\mathbf{x}$. These are eigenvectors with eigenvalue $\lambda =1$. The entire x-axis is a line of eigenvectors.
  • Any vector on the y-axis, like $\mathbf{y}=\left[\begin{array}{c}0\\ c\end{array}\right]$, is squashed to the origin. So, $P\mathbf{y}=\left[\begin{array}{c}0\\ 0\end{array}\right]=0\mathbf{y}$. These are eigenvectors with eigenvalue $\lambda =0$.

• Reflection: Consider the matrix $R=\left[\begin{array}{cc}1& 0\\ 0& -1\end{array}\right]$, which reflects vectors across the x-axis.

  • Any vector on the x-axis is unchanged. These are eigenvectors with $\lambda =1$.
  • Any vector on the y-axis is reversed. These are eigenvectors with $\lambda =-1$.

4. A Transformation Without Real Eigenvectors

Does every matrix have eigenvectors that we can draw in real space? Consider the 90-degree counter-clockwise rotation matrix from the last lesson: $Q=\left[\begin{array}{cc}0& -1\\ 1& 0\end{array}\right]$.

This transformation rotates every non-zero vector. No vector maintains its original direction. Therefore, this matrix has no real eigenvectors. This doesn't mean it has no eigenvectors at all, but that they involve complex numbers. This is an advanced point, but it's good to be aware of.

Conclusion

In this lesson, we have focused on the what and why of eigenvalues and eigenvectors, grounding them in the geometric framework of linear transformations.

Key Takeaways:

• An eigenvector of a matrix $A$ is a non-zero vector $\mathbf{x}$ whose direction is unchanged when multiplied by $A$.
• An eigenvalue $\lambda$ is the scalar factor by which the eigenvector is stretched, shrunk, or reversed.
• The defining relationship is $A\mathbf{x}=\lambda \mathbf{x}$.
• Geometrically, eigenvectors represent the "axes" of a transformation. The eigenvalue dictates the scaling along these axes.
• Simple geometric transformations like scaling, projection, and reflection have intuitive, real-valued eigenvectors and eigenvalues. Others, like rotation, may not have any real eigenvectors.

Preview of the Next Lesson:

We have successfully defined eigenvalues and eigenvectors and understood their geometric meaning. However, we haven't discussed a systematic way to find them for an arbitrary matrix. In the next lesson, we will learn the algebraic procedure to compute them by solving the "characteristic equation," $\det (A-\lambda I)=0$. This will lead us to one of the most important matrix factorizations: eigendecomposition, which breaks a matrix down into its constituent eigenvectors and eigenvalues.