Eigenvalues, Eigenvectors, and the Cayley-Hamilton Theorem

1. What Are Eigenvectors and Eigenvalues?

In essence, a linear transformation (represented by a matrix) can stretch, shrink, rotate, and shear vectors. Amidst all this complex movement, certain vectors stand out. These special vectors, called eigenvectors, are not knocked off their original direction (their "span") by the transformation. They are only stretched or shrunk. The factor by which they are scaled is their corresponding eigenvalue.

To build a strong intuition for this, let's watch a segment from the highly acclaimed "Essence of linear algebra" series by 3Blue1Brown.

As the video explains, if $A$ is a square matrix, $X$ is a non-zero vector, and $\lambda$ is a scalar, their relationship is formally defined by the equation:

• $X$ is the eigenvector of matrix $A$.
• $\lambda$ is the corresponding eigenvalue.

Let's quickly verify this relationship with a concrete example.

2. Finding Eigenvalues: The Characteristic Equation

Now that we understand the concept, how do we find these values without knowing them beforehand? We start by rearranging the defining equation:

To factor out the vector $X$, we need to express $\lambda X$ as a matrix-vector product. We can do this by introducing the identity matrix $I$:

This equation should look familiar! It's a homogeneous system of linear equations. We are looking for a non-trivial solution for $X$ (since eigenvectors are, by definition, non-zero). As we recalled from our last lesson, a homogeneous system has non-trivial solutions if and only if the determinant of the coefficient matrix is zero.

Therefore, we must have:

This is the characteristic equation of matrix $A$. The roots of this equation are the eigenvalues ($\lambda$).

The following videos will walk you through this derivation and the subsequent calculation.

Now, let's see the step-by-step mechanical process of finding the eigenvalues, which is what you'll execute in the exam.

3. Finding Eigenvectors

Once you have found the eigenvalues, you can find the corresponding eigenvectors. For each eigenvalue ${\lambda }_i$, you must solve the system of equations:

The set of all solutions $X$ for a given ${\lambda }_i$ forms the eigenspace for that eigenvalue. Any non-zero vector in this space is a valid eigenvector. The process involves substituting a found value of $\lambda$ into the matrix and solving the resulting homogeneous system, usually by reducing it to row echelon form.

4. The Cayley-Hamilton Theorem

This powerful theorem provides a remarkable link between a matrix and its own characteristic equation.

Theorem Statement: Every square matrix satisfies its own characteristic equation.

This means if the characteristic equation for matrix $A$ is $p(\lambda )=c_n{\lambda }^n+c_{n-1}{\lambda }^{n-1}+\cdots +c_1\lambda +c_0=0$, then substituting the matrix $A$ for $\lambda$ (and $I$ for the constant term) will result in the zero matrix:

This theorem has two killer applications for GATE.

Application 1: Finding the Inverse of a Matrix

If the matrix $A$ is invertible, we can multiply the matrix equation by $A^{-1}$:

We can then rearrange this to solve for $A^{-1}$ in terms of powers of $A$ and the identity matrix $I$. This is often much faster than using the adjoint-determinant method.

Example:
Let's find the inverse of $A=\left(\begin{array}{cc}1& 2\\ 2& 1\end{array}\right)$.

1. Characteristic Equation: $$\det (A-\lambda I)=\det \left(\begin{array}{cc}1-\lambda & 2\\ 2& 1-\lambda \end{array}\right)=(1-\lambda {)}^2-4={\lambda }^2-2\lambda +1-4={\lambda }^2-2\lambda -3=0$$
2. Apply Cayley-Hamilton: $$A^2-2A-3I=O$$
3. Solve for $A^{-1}$: Multiply by $A^{-1}$. $$A-2I-3A^{-1}=O$$ $$3A^{-1}=A-2I$$ $$A^{-1}=\frac{1}{3}(A-2I)=\frac{1}{3}\left(\left(\begin{array}{cc}1& 2\\ 2& 1\end{array}\right)-\left(\begin{array}{cc}2& 0\\ 0& 2\end{array}\right)\right)=\frac{1}{3}\left(\begin{array}{cc}-1& 2\\ 2& -1\end{array}\right)$$

Application 2: Finding High Powers of a Matrix

We can use the characteristic equation to express the highest power of $A$ in terms of lower powers.
From our example, $A^2=2A+3I$.
Suppose we need to find $A^3$. Instead of a costly matrix multiplication, we can do:

Now substitute $A^2$ again:

This reduces the problem of finding $A^3$ to simple matrix addition and scalar multiplication, which is computationally much cheaper.

Conclusion

This lesson introduced the core concepts of eigenvalues and eigenvectors, which describe how a transformation acts along certain invariant directions.

Key Takeaways:

• Eigen-equation: The defining relationship is $AX=\lambda X$, where $X\ne 0$.
• Finding Eigenvalues: Solve the characteristic equation $\det (A-\lambda I)=0$. The roots of this polynomial are the eigenvalues.
• Finding Eigenvectors: For each eigenvalue ${\lambda }_i$, solve the homogeneous system $(A-{\lambda }_{i}I)X=0$ to find the corresponding eigenvectors $X$.
• Cayley-Hamilton Theorem: A matrix satisfies its own characteristic equation. This provides a powerful shortcut for finding the inverse or high powers of a matrix, a frequent pattern in GATE questions.

We have now concluded the Linear Algebra portion of this module. In our next lesson, we will transition to the second major topic of Engineering Mathematics: Calculus. We will start with the fundamental concepts of limits, continuity, and differentiability, which form the bedrock of analyzing functions and their behavior.