Matrix Determinants and Inverses

1. The Determinant: A Measure of Scaling

A linear transformation can stretch, compress, or shear space. The determinant of a matrix gives us a precise number to quantify this effect.

• In 2D, the determinant is the factor by which the area of any shape changes after the transformation.
• In 3D, it's the factor by which volume changes.

A positive determinant means the orientation of space is preserved. A negative determinant means the orientation is flipped (like looking at a mirror image). But most critically, what if the determinant is zero?

Let's start with a video that provides the core geometric intuition behind determinants and their connection to matrix inverses.

As the video explained, a determinant of zero means the transformation is irreversible—you can't "un-squish" a line back into a plane because information has been lost. This makes the determinant a powerful test for invertibility.

Computing the Determinant

Now that you have the intuition, let's look at the mechanics of calculating this value.

For a 2x2 Matrix

For a general 2x2 matrix $A=\left(\begin{array}{cc}a& b\\ c& d\end{array}\right)$, the determinant, denoted as $\det (A)$ or $|A|$, is calculated with a simple formula:

For a 3x3 Matrix (and larger)

For matrices larger than 2x2, we use a method called cofactor expansion. This method breaks down the determinant of a large matrix into a weighted sum of determinants of smaller sub-matrices.

The key steps involve:

1. Minors: For any element in the matrix, its minor is the determinant of the sub-matrix formed by deleting that element's row and column.
2. Cofactors: A cofactor is just a minor with a specific sign (+ or -) depending on its position in the matrix.
3. Expansion: You pick a row or column, and for each element, you multiply it by its cofactor. The sum of these products is the determinant.

The following video walks through the entire calculation for both 2x2 and 3x3 matrices with clear, step-by-step examples.

2. The Inverse Matrix: Undoing a Transformation

As we saw in the 3Blue1Brown video, if a transformation is reversible (i.e., its determinant is not zero), there exists another transformation that takes you back to where you started. This "reverse" transformation is represented by the inverse matrix.

For a square matrix $A$, its inverse is denoted as $A^{-1}$. The defining property of the inverse is that when you multiply a matrix by its inverse, you get the identity matrix $I$.

Geometrically, this means applying a transformation ($A$) and then its inverse ($A^{-1}$) is the same as doing nothing ($I$).

Computing the Inverse

How do we find this inverse matrix? The method depends on the size of the matrix.

For a 2x2 Matrix

There is a convenient formula for the inverse of a 2x2 matrix $A=\left(\begin{array}{cc}a& b\\ c& d\end{array}\right)$, which directly uses the determinant:

Notice that if the determinant $ad-bc$ is zero, this formula involves division by zero, which is undefined. This is the algebraic reason why a matrix with a zero determinant is not invertible.

For a General n x n Matrix: Gauss-Jordan Elimination

For larger matrices, we use a robust algorithm called Gauss-Jordan elimination. The procedure is as follows:

1. Create an augmented matrix by placing your matrix $A$ on the left and the identity matrix $I$ of the same size on the right: $[A|I]$.
2. Perform a series of elementary row operations on the entire augmented matrix. Your goal is to transform the left side ($A$) into the identity matrix $I$.
3. The same operations you apply to turn $A$ into $I$ will simultaneously transform the right side ($I$) into the inverse, $A^{-1}$.
4. When you are done, your augmented matrix will look like this: $[I|A^{-1}]$.

This method is essentially solving multiple systems of equations at once. For a deeper, more formal look at both the theory and the calculation, the following document is an excellent resource.

Practice: A Quick Check

Let's put this together. Consider the matrix $A=\left(\begin{array}{cc}3& 1\\ 5& 2\end{array}\right)$.

1. Is it invertible? Calculate its determinant.
   $\det (A)=(3)(2)-(1)(5)=6-5=1$. Since the determinant is not zero, the matrix is invertible.
2. What is its inverse? Use the 2x2 formula. $$A^{-1}=\frac{1}{1}\left(\begin{array}{cc}2& -1\\ -5& 3\end{array}\right)=\left(\begin{array}{cc}2& -1\\ -5& 3\end{array}\right)$$
3. Verify the inverse. Check if $A{A}^{-1}=I$. $$\left(\begin{array}{cc}3& 1\\ 5& 2\end{array}\right)\left(\begin{array}{cc}2& -1\\ -5& 3\end{array}\right)=\left(\begin{array}{cc}(3)(2)+(1)(-5)& (3)(-1)+(1)(3)\\ (5)(2)+(2)(-5)& (5)(-1)+(2)(3)\end{array}\right)=\left(\begin{array}{cc}6-5& -3+3\\ 10-10& -5+6\end{array}\right)=\left(\begin{array}{cc}1& 0\\ 0& 1\end{array}\right)$$

It works! The result is the identity matrix.

Conclusion

In this lesson, you've learned to compute two of the most important properties of a square matrix: its determinant and its inverse.

Key Takeaways:

• The determinant is a scalar value that represents the scaling factor of a linear transformation on area or volume.
• A matrix is invertible if and only if its determinant is not zero. A zero determinant means the transformation collapses space into a lower dimension and is irreversible.
• The inverse matrix, $A^{-1}$, represents the transformation that reverses the action of $A$, such that $A{A}^{-1}=I$.
• For 2x2 matrices, there are simple formulas for both the determinant ($ad-bc$) and the inverse.
• For larger matrices, the determinant is found using cofactor expansion, and the inverse is calculated using Gauss-Jordan elimination.

Preview of the Next Lesson:
Now that we have the concept of an inverse matrix, we have the primary tool needed to solve systems of linear equations. In the next lesson, we will see how the equation $Ax=b$ can be elegantly solved for $x$ by using the inverse: $x=A^{-1}b$. This is a cornerstone of computational mathematics and has direct applications in machine learning.