Column Space and Null Space

1. The Column Space: The Range of a Transformation

Let's start by thinking of a matrix $A$ as a transformation that takes an input vector $\vec{x}$ and maps it to an output vector $A\vec{x}$. A natural question arises: what is the set of all possible output vectors? This set is called the column space.

As we saw in the previous lesson, the product $A\vec{x}$ is a linear combination of the columns of $A$, with the weights given by the components of $\vec{x}$. Therefore, the set of all possible outputs is simply the span of the columns of A.

To build a strong visual intuition for this, let's turn to 3Blue1Brown's excellent "Essence of linear algebra" series.

As the video illustrates, the column space is the "landing spot" for all vectors after the transformation. If the columns of a 3x3 matrix are linearly dependent and only span a plane, then no matter what input vector $\vec{x}$ you choose, the output $A\vec{x}$ will always be on that plane.

Formal Definition and Rank

Formally, the column space of an $m\times n$ matrix $A$, denoted $\text{Col}(A)$, is the span of its column vectors. It is a subspace of ${\mathbb{R}}^m$.

An equivalent definition is that the column space is the set of all vectors $\vec{b}$ for which the equation $A\vec{x}=\vec{b}$ has a solution. If a vector $\vec{b}$ is in the column space of $A$, it means $\vec{b}$ is a possible output of the transformation.

The dimension of the column space is called the rank of the matrix. The rank tells us the number of dimensions in the output space. For a 2x2 matrix, a rank of 2 means the output is the full 2D plane, while a rank of 1 means the output is collapsed onto a line.

2. The Null Space: The Kernel of a Transformation

Now let's look at the flip side. If a transformation squishes space into a lower dimension (i.e., the matrix is not full rank), some input vectors must get mapped to the origin, $\vec{0}$. The set of all input vectors that get "crushed" or "nullified" by the transformation is called the null space or kernel.

Let's see the geometric intuition first.

Formal Definition and Nullity

The null space of an $m\times n$ matrix $A$, denoted $\text{Null}(A)$, is the set of all vectors $\vec{x}$ in ${\mathbb{R}}^n$ that satisfy the homogeneous equation:

The null space is a subspace of the input space ${\mathbb{R}}^n$. For a full-rank matrix, the only vector that gets mapped to the origin is the zero vector itself, so its null space is just $\{\vec{0}\}$. If the null space contains non-zero vectors, it tells us the transformation is "many-to-one" and collapses the input space.

The dimension of the null space is often called the nullity of the matrix.

The following video provides a concise summary of both the column space and null space, reinforcing their definitions and contrasting where they "live" (in the output space vs. the input space).

3. Finding the Column Space and Null Space

So far, we have the intuition. Now, let's focus on the practical procedure for identifying these spaces. Given a matrix $A$, how do we find a basis for its column space and null space? The key is a tool you may recall from your engineering studies: reduced row echelon form (RREF).

The following resource provides a clear, step-by-step guide. Since your math is rusty, focusing on a concrete algorithm like this is an excellent way to solidify your understanding.

This reading also introduces a fundamental relationship known as the Rank-Nullity Theorem:

Or, more simply, rank + nullity = number of columns.

This theorem elegantly states that the number of dimensions in the output space (rank) plus the number of input dimensions that are lost (nullity) must equal the total number of dimensions in the input space.

4. The Big Picture: The Four Fundamental Subspaces

The column space and null space are two of the "Four Fundamental Subspaces" associated with any matrix $A$. The other two are the row space (the span of the rows) and the left null space (the null space of $A^T$).

This diagram provides a powerful visual summary of how these spaces relate to one another, their dimensions, and the input/output spaces they reside in.

Conclusion

In this lesson, we attached concrete meaning to the abstract concepts of span and linear independence in the context of matrices. These two subspaces are not just mathematical curiosities; they are essential for understanding everything from whether a system of equations has a solution to how dimensionality reduction techniques like PCA work.

Key Takeaways:

• The Column Space of a matrix $A$ is the span of its columns. It represents all possible outputs of the transformation $T(\vec{x})=A\vec{x}$. Its dimension is the rank of the matrix.
• The Null Space of a matrix $A$ is the set of all input vectors $\vec{x}$ that are mapped to the zero vector ($A\vec{x}=\vec{0}$). Its dimension is the nullity.
• We can find a basis for these spaces by using the reduced row echelon form (RREF) of the matrix.
• The Rank-Nullity Theorem connects the dimensions of these two spaces: $\text{rank}(A)+\text{nullity}(A)=\text{number of columns}$.

In our next lesson, we'll dive into another critical property of square matrices: the determinant. You'll see that the determinant gives us a single number that tells us whether a transformation collapses space—in other words, whether its null space is non-trivial and its rank is less than maximal.