Diagonalizing a Matrix

1. From Eigenvectors to the Diagonalization Formula

Let's start by understanding where the famous $A=PD{P}^{-1}$ formula comes from. It's a clever way of stacking all the individual eigenvalue-eigenvector equations into a single matrix equation.

Suppose we have an $n\times n$ matrix $A$ that is diagonalizable. By definition, this means we can find $n$ linearly independent eigenvectors ${\mathbf{v}}_1,{\mathbf{v}}_2,\dots ,{\mathbf{v}}_n$ with corresponding eigenvalues ${\lambda }_1,{\lambda }_2,\dots ,{\lambda }_n$. For each pair, we have:

• $A{\mathbf{v}}_1={\lambda }_1{\mathbf{v}}_1$
• $A{\mathbf{v}}_2={\lambda }_2{\mathbf{v}}_2$
• ...
• $A{\mathbf{v}}_n={\lambda }_n{\mathbf{v}}_n$

Now, let's assemble the eigenvectors into the columns of a matrix, which we'll call $P$:
$P=\left[\begin{array}{cccc}|& |& & |\\ {\mathbf{v}}_1& {\mathbf{v}}_2& \dots & {\mathbf{v}}_n\\ |& |& & |\end{array}\right]$
What happens when we multiply $A$ by $P$? The multiplication acts on each column separately:
$AP=A\left[\begin{array}{cccc}|& |& & |\\ {\mathbf{v}}_1& {\mathbf{v}}_2& \dots & {\mathbf{v}}_n\\ |& |& & |\end{array}\right]=\left[\begin{array}{cccc}|& |& & |\\ A{\mathbf{v}}_1& A{\mathbf{v}}_2& \dots & A{\mathbf{v}}_n\\ |& |& & |\end{array}\right]$
Using our eigenvalue equations, we can substitute $A{\mathbf{v}}_i$ with ${\lambda }_i{\mathbf{v}}_i$:
$AP=\left[\begin{array}{cccc}|& |& & |\\ {\lambda }_1{\mathbf{v}}_1& {\lambda }_2{\mathbf{v}}_2& \dots & {\lambda }_n{\mathbf{v}}_n\\ |& |& & |\end{array}\right]$
This is the key step. The right side can be factored into the original matrix $P$ multiplied by a diagonal matrix $D$ containing the eigenvalues:
$\left[\begin{array}{cccc}|& |& & |\\ {\lambda }_1{\mathbf{v}}_1& {\lambda }_2{\mathbf{v}}_2& \dots & {\lambda }_n{\mathbf{v}}_n\\ |& |& & |\end{array}\right]=\left[\begin{array}{cccc}|& |& & |\\ {\mathbf{v}}_1& {\mathbf{v}}_2& \dots & {\mathbf{v}}_n\\ |& |& & |\end{array}\right]\left[\begin{array}{cccc}{\lambda }_1& 0& \dots & 0\\ 0& {\lambda }_2& \dots & 0\\ ⋮& ⋮& \ddots & ⋮\\ 0& 0& \dots & {\lambda }_n\end{array}\right]=PD$
So, we have arrived at the compact equation:
$AP=PD$
Since the eigenvectors are linearly independent, the matrix $P$ is invertible. This allows us to isolate $A$ by right-multiplying by $P^{-1}$:
$A=PD{P}^{-1}$
This is the diagonalization of matrix $A$.

To see this derivation explained, let's watch a classic lecture segment from Professor Gilbert Strang. He uses $S$ for the eigenvector matrix and the capital Greek letter Lambda ($\Lambda$) for the eigenvalue matrix, which is another common notation.

2. The Recipe for Diagonalization

Now that you've seen the "why," let's outline the "how." Here is a step-by-step recipe for diagonalizing an $n\times n$ matrix $A$.

1. Find the Eigenvalues: Solve the characteristic equation $\det (A-\lambda I)=0$ to find all eigenvalues ${\lambda }_1,\dots ,{\lambda }_n$.
2. Find Linearly Independent Eigenvectors: For each distinct eigenvalue ${\lambda }_i$, find a basis for its eigenspace by solving $(A-{\lambda }_{i}I)\mathbf{x}=0$.
3. Check for Diagonalizability: Count your total number of linearly independent eigenvectors. If this number is less than $n$ (which happens when any eigenvalue's geometric multiplicity is less than its algebraic multiplicity), the matrix is not diagonalizable. Stop here.
4. Construct P and D: If you have $n$ linearly independent eigenvectors, create the matrices:
   • $P$: The columns of $P$ are the eigenvectors ${\mathbf{v}}_1,\dots ,{\mathbf{v}}_n$.
   • $D$: $D$ is a diagonal matrix with the corresponding eigenvalues on the diagonal. The order must match. If the $i$-th column of $P$ is ${\mathbf{v}}_i$, the $i$-th diagonal entry of $D$ must be ${\lambda }_i$.
5. Verify (Optional but Recommended): You can confirm your work by computing $P^{-1}$ and checking that $A=PD{P}^{-1}$, or by checking the simpler relation $AP=PD$.

For a detailed walkthrough of these steps, you can read the following guide.

3. A Worked Example

Let's apply this recipe to diagonalize the matrix $A=\left(\begin{array}{cc}4& -2\\ 1& 1\end{array}\right)$.

Step 1: Find Eigenvalues
The characteristic polynomial is:
$\det (A-\lambda I)=\det \left(\begin{array}{cc}4-\lambda & -2\\ 1& 1-\lambda \end{array}\right)=(4-\lambda )(1-\lambda )-(-2)={\lambda }^2-5\lambda +6$
Setting this to zero gives $(\lambda -2)(\lambda -3)=0$. Our eigenvalues are ${\lambda }_1=3$ and ${\lambda }_2=2$.

Step 2: Find Eigenvectors

• For ${\lambda }_1=3$: We find the null space of $A-3I$.
  $A-3I=\left(\begin{array}{cc}1& -2\\ 1& -2\end{array}\right)$
  The equation $(A-3I)\mathbf{x}=0$ leads to $x_1-2x_2=0$. An eigenvector is ${\mathbf{v}}_1=\left(\begin{array}{c}2\\ 1\end{array}\right)$.
• For ${\lambda }_2=2$: We find the null space of $A-2I$.
  $A-2I=\left(\begin{array}{cc}2& -2\\ 1& -1\end{array}\right)$
  The equation $(A-2I)\mathbf{x}=0$ leads to $x_1-x_2=0$. An eigenvector is ${\mathbf{v}}_2=\left(\begin{array}{c}1\\ 1\end{array}\right)$.

Step 3: Check for Diagonalizability
We have a 2x2 matrix and we found 2 linearly independent eigenvectors. The matrix is diagonalizable.

Step 4: Construct P and D
Let's put our first eigenvector in the first column of $P$.
$P=\left[\begin{array}{cc}{\mathbf{v}}_1& {\mathbf{v}}_2\end{array}\right]=\left(\begin{array}{cc}2& 1\\ 1& 1\end{array}\right)$
The corresponding eigenvalues must be in the same order in $D$:
$D=\left(\begin{array}{cc}{\lambda }_1& 0\\ 0& {\lambda }_2\end{array}\right)=\left(\begin{array}{cc}3& 0\\ 0& 2\end{array}\right)$
This gives us the diagonalization of $A$.

Step 5: Verify
Let's check that $AP=PD$.
$AP=\left(\begin{array}{cc}4& -2\\ 1& 1\end{array}\right)\left(\begin{array}{cc}2& 1\\ 1& 1\end{array}\right)=\left(\begin{array}{cc}8-2& 4-2\\ 2+1& 1+1\end{array}\right)=\left(\begin{array}{cc}6& 2\\ 3& 2\end{array}\right)$
$PD=\left(\begin{array}{cc}2& 1\\ 1& 1\end{array}\right)\left(\begin{array}{cc}3& 0\\ 0& 2\end{array}\right)=\left(\begin{array}{cc}6+0& 0+2\\ 3+0& 0+2\end{array}\right)=\left(\begin{array}{cc}6& 2\\ 3& 2\end{array}\right)$
The results match, confirming our diagonalization is correct.

The image below shows another complete example for a 2x2 matrix, visually summarizing the entire process.

4. The Geometric Meaning: A Change of Basis

The formula $A=PD{P}^{-1}$ is more than an algebraic trick. It represents a deep geometric idea: diagonalization is a change of basis.

• The matrix $A$ describes a linear transformation in our familiar standard coordinate system (defined by $\mathbf{i}$ and $\mathbf{j}$).
• The diagonal matrix $D$ describes the exact same transformation, but viewed from the perspective of a new coordinate system whose axes are the eigenvectors. This new basis is called an eigenbasis.

In this special eigenbasis, the transformation is incredibly simple: it's just a scaling along each of the new axes by the corresponding eigenvalue. The formula $A=PD{P}^{-1}$ can be read as a three-step process:

1. $P^{-1}$: Change to the eigenbasis. Take a vector $\mathbf{x}$ and find its coordinates in the eigenbasis.
2. $D$: Scale. Apply the simple scaling transformation in the eigenbasis.
3. $P$: Change back to the standard basis. Convert the resulting vector back to the standard coordinate system.

This concept of changing to a simpler basis to analyze a transformation is fundamental in both physics and computer science. Let's watch a short clip that illustrates this beautifully.

Conclusion

Today, you've learned the complete procedure for diagonalizing a matrix. This powerful technique bridges the abstract theory of eigenvalues with concrete computational benefits.

Key Takeaways:

• The diagonalization $A=PD{P}^{-1}$ is derived by expressing all $A{\mathbf{v}}_i={\lambda }_i{\mathbf{v}}_i$ equations in a single matrix form, $AP=PD$.
• The columns of $P$ are the $n$ linearly independent eigenvectors of $A$.
• The diagonal entries of $D$ are the corresponding eigenvalues, placed in the same order as their eigenvectors in $P$.
• Geometrically, diagonalization represents changing to an eigenbasis where the transformation is a simple scaling, performing the scaling, and then changing back.

Preview of the Next Lesson:

Now that we know how to construct the diagonalization, what can we do with it? In the next lesson, we will explore one of its most immediate and powerful applications: applying diagonalization to efficiently compute powers of a matrix. This is not only useful for algorithms but is also conceptually similar to how one calculates the evolution of systems over time, a key idea in fields like quantum mechanics.