Determinants and Inverses of Matrices

Welcome back to our journey through the mathematical foundations of AI. In our last lesson, we explored the concepts of rank and nullity, culminating in the Rank-Nullity Theorem. We established that for a square $n\times n$ matrix $A$, being invertible is equivalent to having full rank ($\text{rank}(A)=n$) and zero nullity. While Gaussian elimination allows us to determine this, it can be a lengthy process.

This lesson introduces a more direct tool: the determinant. The determinant is a single, powerful number associated with every square matrix that tells us, among other things, whether the matrix is invertible. We will explore how to compute the determinant and understand its fundamental properties. This will lead us directly to the definition and computation of the matrix inverse, $A^{-1}$, which is conceptually and practically crucial in linear algebra.

Our goal is to compute the determinant and inverse of a square matrix and describe their properties.

1. The Determinant of a Matrix

The determinant is a function that maps a square matrix to a scalar value. This value provides a wealth of information about the matrix.

1.1. Calculating the Determinant

You've likely encountered the determinant for a $2\times 2$ matrix:

For a $3\times 3$ matrix, a mnemonic called the Rule of Sarrus exists, but it doesn't generalize to larger matrices. The general and systematic method for computing the determinant of any $n\times n$ matrix is the Laplace expansion, also known as cofactor expansion.

This method recursively breaks down the determinant of an $n\times n$ matrix into a sum of determinants of $(n-1)\times (n-1)$ submatrices.

To understand this method in detail, let's watch a video that breaks it down.

The core idea of the Laplace expansion along column $j$ is:

where $a_{ij}$ is the entry in the $i$-th row and $j$-th column, and $M_{ij}$ is the determinant of the submatrix formed by deleting row $i$ and column $j$. The term $(-1{)}^{i+j}{M}_{ij}$ is called the cofactor of $a_{ij}$. An analogous formula exists for expanding along a row.

For additional reference, the textbook "Matrix Algebra for Engineers" also covers the Laplace expansion in Lecture 29.

1.2. Properties of the Determinant

The determinant has several profound properties that are essential for both theory and computation. Rather than simply memorizing a list, it's insightful to see how most properties can be derived from three fundamental axioms.

Here is a summary of the most critical properties for an $n\times n$ matrix $A$:

1. Invertibility Criterion: $\det (A)\ne 0$ if and only if $A$ is invertible. This is the most important property, linking determinants to the concepts of rank, nullity, and unique solutions to $A\mathbf{x}=\mathbf{b}$.
2. Product Rule: $\det (AB)=\det (A)\det (B)$. Note that $\det (A+B)$ is not equal to $\det (A)+\det (B)$.
3. Transpose: $\det (A^T)=\det (A)$. This implies that any property related to rows also holds for columns.
4. Triangular Matrices: If $A$ is upper or lower triangular, its determinant is the product of its diagonal entries. This makes computing determinants trivial after Gaussian elimination.
5. Row Operations:
   • Swapping two rows multiplies the determinant by -1.
   • Multiplying a row by a scalar $k$ multiplies the determinant by $k$. This implies $\det (kA)=k^n\det (A)$.
   • Adding a multiple of one row to another does not change the determinant.

This last point provides an efficient way to compute determinants: use row operations (without scaling) to bring the matrix to an upper triangular form, then multiply the diagonal entries.

2. The Inverse of a Matrix

An invertible matrix $A$ has a unique inverse, denoted $A^{-1}$, which "undoes" the action of $A$.

2.1. Definition and Properties

Key properties to remember:

• Definition: $A{A}^{-1}=A^{-1}A=I$.
• Existence: $A^{-1}$ exists if and only if $\det (A)\ne 0$.
• Inverse of a Product: $(AB{)}^{-1}=B^{-1}{A}^{-1}$.
• Inverse of Transpose: $(A^T{)}^{-1}=(A^{-1}{)}^T$.
• Determinant of Inverse: If $A$ is invertible, then $\det (A^{-1})=\frac{1}{\det (A)}$. This follows directly from the product rule: $\det (A{A}^{-1})=\det (A)\det (A^{-1})=\det (I)=1$.

2.2. Computing the Inverse

While the determinant tells us if an inverse exists, we need an algorithm to find it.

For 2x2 Matrices:
There is a simple formula:

Notice the denominator is precisely $\det (A)$.

For General n x n Matrices: The Gauss-Jordan Method
For larger matrices, the most reliable and computationally efficient method is Gauss-Jordan elimination. The procedure solves for all $n$ columns of $A^{-1}$ simultaneously.

The process is as follows:

1. Form the augmented matrix $[A|I]$.
2. Use elementary row operations to transform the left side ($A$) into the identity matrix $I$.
3. The same sequence of operations transforms the right side ($I$) into $A^{-1}$.
4. The final form will be $[I|A^{-1}]$. If you cannot reduce $A$ to $I$ (i.e., you get a row of zeros), then the matrix is not invertible.

Conclusion

Today we've introduced two of the most fundamental concepts in matrix theory. They are deeply interconnected and form the capstone of our first module.

Key Takeaways:

• The determinant is a scalar value for a square matrix that determines its invertibility. A matrix $A$ is invertible if and only if $\det (A)\ne 0$.
• The determinant can be computed recursively using Laplace expansion or more efficiently by reducing the matrix to triangular form using row operations.
• The inverse of a matrix, $A^{-1}$, undoes the linear transformation of $A$. It exists only if $A$ is invertible.
• The inverse is computed algorithmically using Gauss-Jordan elimination on the augmented matrix $[A|I]$.
• Two critical properties to remember are the product rules: $\det (AB)=\det (A)\det (B)$ and $(AB{)}^{-1}=B^{-1}{A}^{-1}$.

This concludes our module on the foundations of linear algebra. You now have a solid theoretical understanding of vector spaces, linear independence, bases, matrices, linear systems, rank, and invertibility.

Preview of the Next Module:
In our next lesson, we will begin our study of Eigenanalysis. We will ask a powerful question: for a given matrix $A$, are there any non-zero vectors $\mathbf{x}$ such that $A\mathbf{x}$ is just a scaled version of $\mathbf{x}$? These special vectors are called eigenvectors, and the scaling factors are eigenvalues. You will see that the determinant plays a central role in finding them, through what is called the characteristic equation: $\det (A-\lambda I)=0$. This will be a key tool for understanding and decomposing complex linear transformations.