Invertibility of Linear Transformations from Matrices

1. What Does It Mean to Be Invertible?

At its core, an invertible transformation is one that can be perfectly reversed. Every output corresponds to exactly one unique input.

Geometric Intuition: Playing the Tape in Reverse

A great way to visualize this is to think of a linear transformation as an action, like a rotation or a shear. The inverse transformation is simply the action that reverses the first one, bringing every vector back to where it started.

As the video explains, if a transformation $T$ (represented by matrix $A$) takes a vector $\mathbf{x}$ to $\mathbf{y}$, so $A\mathbf{x}=\mathbf{y}$, the inverse transformation $T^{-1}$ (represented by matrix $A^{-1}$) takes $\mathbf{y}$ back to $\mathbf{x}$, so $A^{-1}\mathbf{y}=\mathbf{x}$.

Applying $A$ and then $A^{-1}$ is like doing nothing. Algebraically, this means applying the matrices in sequence results in the identity matrix, $I$.

The Algebraic and Geometric Connection

We have two related ideas:

1. Geometric: An invertible linear transformation $T$ is a function that can be reversed by another linear transformation $S$, such that applying $T$ then $S$ (or $S$ then $T$) gets you back to where you started.
2. Algebraic: An invertible matrix $A$ is a square matrix for which there exists an inverse matrix $A^{-1}$ such that $A^{-1}A=A{A}^{-1}=I$.

Crucially, these two ideas are perfectly linked. A linear transformation is invertible if and only if the matrix that represents it is invertible.

This is great news! It means to answer the question "Is this linear transformation invertible?", we just have to answer "Is its matrix invertible?".

2. The Invertible Matrix Theorem: A Toolkit of Tests

So, how do we determine if an $n\times n$ matrix $A$ is invertible? It turns out there isn't just one way; there's a whole collection of equivalent conditions. This is one of the most powerful ideas in linear algebra, known as the Invertible Matrix Theorem (IMT). It states that for any square matrix $A$, all the following statements are either all true or all false.

Let's begin by reading a summary of the most important tests for invertibility.

Let's unpack the most critical conditions from this reading and connect them to what we already know. For an $n\times n$ matrix $A$, the following are equivalent:

1. $A$ is an invertible matrix.
   This is our definition: there exists a matrix $A^{-1}$ such that $A^{-1}A=A{A}^{-1}=I$.

2. The determinant of $A$ is not zero ($\det (A)\ne 0$).
   This is often the quickest test for small matrices. Intuitively, as 3Blue1Brown explained, a non-zero determinant means the transformation preserves volume (it doesn't squish space into a lower dimension like a line or a point). If space isn't collapsed, you can reverse the process.

{
"type": "video",
"title": "Inverse matrices, column space and null space | Chapter 7, Essence of linear algebra",
"id": "LINK",
"video_id": "uQhTuRlWMxw",
"relevant_section_indices": [
1
],
"par_intro": "Let's revisit the 3Blue1Brown video to solidify the connection between the determinant and invertibility.",
"par_directions": "Watch from 05:40 to 07:26. This clip explains exactly why a zero determinant implies the transformation squishes space and is therefore not invertible."
}

3.  **The equation $A\mathbf{x} = \mathbf{0}$ has only the trivial solution, $\mathbf{x} = \mathbf{0}$.**
    This is a direct connection to our last lesson! This statement is just another way of saying that the **null space** of $A$ contains only the zero vector, which means $\text{nullity}(A) = 0$. If the nullity is zero, the transformation is one-to-one, which is a prerequisite for being invertible.

4.  **$A$ has $n$ pivots.**
    This is the most reliable computational test. When you row reduce $A$, you get a pivot in every column (and every row). If you can't get $n$ pivots, you'll end up with a row of zeros, which signals that the matrix is singular (not invertible). This is also why the Gauss-Jordan method for finding an inverse works: it succeeds only if you can reduce $A$ all the way to the identity matrix $I$.

5.  **The columns of $A$ are linearly independent.**
    This is equivalent to saying $A\mathbf{x}=\mathbf{0}$ has only the trivial solution. If the columns are dependent, there's a non-trivial combination that equals the zero vector, meaning the nullity is greater than zero.

6.  **The linear transformation $T(\mathbf{x}) = A\mathbf{x}$ is one-to-one and onto.**
    For a square matrix, these two conditions are linked by the Rank-Nullity Theorem.
    *   `one-to-one` $\iff$ $\text{nullity}(A) = 0$.
    *   By Rank-Nullity: $\text{rank}(A) + 0 = n \implies \text{rank}(A) = n$.
    *   `rank(A) = n` $\iff$ the column space spans all of $\mathbb{R}^n$ $\iff$ the transformation is `onto`.
    So for square matrices, being one-to-one automatically implies being onto, and vice-versa!

This collection of equivalent statements is your toolkit for analyzing invertibility.

```grasp
{
  "type": "image",
  "title": "The Invertible Matrix Theorem",
  "id": "[LINK](https://i.ytimg.com/vi/zlp9Y4KNXj4/maxresdefault.jpg)",
  "url": "https://i.ytimg.com/vi/zlp9Y4KNXj4/maxresdefault.jpg",
  "caption": "This image provides a great summary of the Invertible Matrix Theorem, showing how all these different concepts are linked. The statements (a) through (j) are all equivalent for a square matrix."
}

3. Practical Properties of Inverses

A couple of other properties are essential for practical use, especially in contexts like Deep Learning and Quantum Mechanics where you often deal with compositions of transformations.

Inverse of a Product:
If you apply two transformations, $A$ then $B$, the resulting transformation is $BA$. To reverse this, you must reverse the last step first, then the first step. This is like putting on socks then shoes; to undo it, you take off the shoes first, then the socks.

This "reverse order" rule is extremely important. You can find it discussed as "Note 6" in the Strang PDF you read earlier.

Computational Cost:
While we've learned how to test for invertibility and even how to compute an inverse using the Gauss-Jordan method (described in section 2.5 of Strang's text), it's important to know that computing an inverse is computationally expensive ($O(n^3)$). In many practical applications, like solving a large system of equations $A\mathbf{x}=\mathbf{b}$, it's much faster to solve for $\mathbf{x}$ directly using elimination rather than first computing $A^{-1}$ and then multiplying by $\mathbf{b}$.

Conclusion

Today we've built a comprehensive understanding of what it means for a linear transformation and its matrix to be invertible.

Key Takeaways:

• A linear transformation is invertible if it can be "undone" by another linear transformation. This is only possible if the transformation is both one-to-one and onto.
• A transformation $T$ is invertible if and only if its standard matrix $A$ is invertible.
• The Invertible Matrix Theorem provides a powerful checklist of equivalent conditions to test a square matrix for invertibility. The most important are:
  • It has $n$ pivots.
  • Its determinant is non-zero.
  • Its null space contains only the zero vector ($\text{nullity}(A)=0$).
  • Its columns are linearly independent.
• The inverse of a product of matrices is the product of their inverses in reverse order: $(AB{)}^{-1}=B^{-1}{A}^{-1}$.

Preview of the Next Lesson:
We've spent a lot of time thinking about vectors and transformations in terms of the standard basis (using vectors like $\mathbf{i}=[1,0{]}^T$ and $\mathbf{j}=[0,1{]}^T$). But what if we want to describe them using a different set of basis vectors? In our next lesson, we will explore how to perform a change of basis. We will see that the matrix which translates coordinates from one basis to another must be—you guessed it—an invertible matrix.