Eigenvalues & Eigenvectors: Scaling & Invariant Directions

Matrices as Transformations

First, let's re-establish a key idea: a matrix can be viewed as a function that transforms space. When we multiply a vector by a matrix, we're applying a transformation that might stretch, compress, rotate, or shear that vector, moving it to a new position.

Most vectors, when transformed, are knocked off their original line, or "span." However, some special vectors are unique. They remain on their original span, changing only in length or orientation (by flipping 180 degrees). These are the eigenvectors.

The renowned 3Blue1Brown channel offers a superb visualization of this concept.

Discovering the Invariant Directions

So, the eigenvectors are the "invariant directions" of a transformation, and the eigenvalues are the scaling factors along those directions. Let's see this in action with a concrete example.

The following video starts with a specific 2x2 matrix and explores what it does to various vectors, visually discovering its eigenvectors and their associated scaling factors. This process mirrors the journey from observation to definition.

The animation below shows exactly the transformation from the video you just watched. The yellow vector is an eigenvector—notice how it stays on its own line, only changing in length. The red vector is not, and it gets rotated off its original span.

This geometric picture is precisely what the equation $A\mathbf{v}=\lambda \mathbf{v}$ describes.

• $A\mathbf{v}$ is the vector after the transformation.
• $\lambda \mathbf{v}$ is a scaled version of the original vector.
  The equation is a formal statement that the transformed vector points in the same (or exactly opposite) direction as the original vector.

The name itself gives a clue. As explained in the article "Eigen Intuitions," the German prefix "eigen" translates to "characteristic." Eigenvectors are the characteristic directions of a transformation.

Interpreting the Eigenvalue

The eigenvalue $\lambda$ tells you the story of the scaling along its corresponding eigenvector's direction. The article "Geometric Meaning of Eigenvalues and Eigenvectors" provides a great summary.

As you just read, the value of $\lambda$ tells us exactly what happens along an eigenvector's direction:

• If $|\lambda |>1$, vectors are stretched.
• If $|\lambda |<1$, vectors are compressed.
• If $\lambda =1$, vectors are unchanged. This is a line of stability.
• If $\lambda =0$, vectors are squashed onto the origin. The eigenvector direction is part of the null space.
• If $\lambda$ is negative, vectors are flipped and then scaled.

The image below illustrates several of these possibilities. For each pair, the grid on the left is transformed into the grid on the right. The vectors shown are the eigenvectors, and their eigenvalues ${\lambda }_1$ and ${\lambda }_2$ determine the nature of the transformation. For example, in the top panel, the vertical direction is compressed (${\lambda }_2=0.5$) while the horizontal direction is stretched (${\lambda }_1=2.0$).

Crafting Your Interview Answer

Now, let's synthesize this into a clear, concise answer for your interview.

Question: "What is the geometric interpretation of an eigenvalue and an eigenvector?"

Your Answer Framework:

1. The Big Picture: "A matrix can be seen as a linear transformation that acts on space—stretching, compressing, or rotating vectors. When you apply this transformation, most vectors are knocked off their original direction."
2. Define Eigenvectors: "However, every transformation has special, characteristic directions called eigenvectors. Vectors pointing in these directions are unique because they don't change their direction. They stay on their original line, or span, through the origin."
3. Define Eigenvalues: "The eigenvalue is the scaling factor for its corresponding eigenvector. It tells you exactly how much the eigenvector is stretched or shrunk. For example, an eigenvalue of 3 means any vector along that eigenvector's direction is stretched to three times its original length. An eigenvalue of -0.5 means it's flipped to the opposite direction and squished to half its length."
4. Provide a Powerful Example: "A great real-world example is 3D rotation. If you have a matrix that represents a rotation in 3D space, its eigenvector is the axis of rotation. The corresponding eigenvalue will be 1, because the axis of rotation itself doesn't change length or direction. This simplifies a complex 3x3 matrix into a simple axis and an angle of rotation."

To demonstrate the depth of your understanding, you might also mention cases where no such real directions exist.

Conclusion

In this lesson, you have connected the algebraic procedure of finding eigenvalues and eigenvectors to their profound geometric meaning. You can now confidently explain what they are and why they matter.

Key Takeaways:

• A matrix is a linear transformation of space.
• Eigenvectors are the invariant directions of this transformation—the lines that are only scaled, not rotated.
• Eigenvalues are the scaling factors for their corresponding eigenvectors, indicating the amount of stretch, compression, or flip.
• This concept simplifies complex transformations by identifying their fundamental "axes of action."

You are now fully prepared for questions about eigenvalues and eigenvectors. In our next lesson, we will build directly on this foundation to explore Singular Value Decomposition (SVD), a powerful technique used in dimensionality reduction, image compression, and recommender systems. You will see how the idea of finding special "axes" extends to all matrices, not just square ones.