Dimension of Vector Spaces and Subspaces

1. What is Dimension?

Intuitively, we understand that a line is one-dimensional, a plane is two-dimensional, and the world around us is three-dimensional. Linear algebra makes this intuition precise. The "dimension" of a vector space is simply the number of vectors in its basis.

But for this definition to be meaningful, we must be certain that any basis we choose for a given space will have the same number of vectors. Thankfully, this is true.

Theorem: All bases for a vector space have the same length.

The reasoning behind this is elegant. Let's say we have two different bases for the same space, call them $B_1$ and $B_2$.

• Since $B_1$ is linearly independent and $B_2$ spans the space, the number of vectors in $B_1$ must be less than or equal to the number in $B_2$. (A key theorem states that an independent set cannot be larger than a spanning set).
• Flipping this around, since $B_2$ is linearly independent and $B_1$ spans the space, the number of vectors in $B_2$ must be less than or equal to the number in $B_1$.

The only way both of these statements can be true is if the number of vectors in $B_1$ is equal to the number of vectors in $B_2$.

This consistency allows us to make a formal definition.

As the reading confirms:

Definition: Dimension
The dimension of a finite-dimensional vector space $V$, denoted $\text{dim}(V)$, is the number of vectors in any basis for $V$. The dimension of the zero vector space $\{0\}$ is defined to be 0.

Let's watch a short video clip that grounds this definition with familiar examples.

So, for example:

• dim(${\mathbb{R}}^n$) = $n$, because the standard basis has $n$ vectors.
• The dimension of a plane through the origin in ${\mathbb{R}}^3$ is 2.
• The dimension of a line through the origin in ${\mathbb{R}}^3$ is 1.

2. Computing Dimension

To find the dimension of a subspace, you simply need to find a basis for it and count the number of vectors in that basis.

In the last lesson, we learned how to find a basis for the column space of a matrix $A$, denoted $\text{Col}(A)$. We did this by row reducing $A$ and taking the original columns that correspond to pivot columns. Therefore:

The dimension of the column space is the number of pivot columns.

This number is so important that it has its own name: the rank of the matrix.

Definition: Rank
The rank of a matrix $A$, denoted $\text{rank}(A)$, is the dimension of its column space.
$\text{rank}(A)=\text{dim}(\text{Col}(A))$

What about the row space, $\text{Row}(A)$, which is spanned by the rows of $A$? It turns out there's a beautiful symmetry here. The following video demonstrates two practical methods for finding the dimension of a subspace spanned by a set of vectors.

The video highlights two crucial points:

1. To find a basis for the row space of $A$: Row reduce $A$. The non-zero rows of the echelon form are a basis for $\text{Row}(A)$. The dimension is the number of non-zero rows, which is the number of pivots.
2. To find a basis for the column space of $A$: Row reduce $A$. The columns of the original matrix $A$ that correspond to the pivot columns form a basis for $\text{Col}(A)$. The dimension is the number of pivot columns.

This leads to one of the first major theorems of linear algebra:
The Row Rank Equals the Column Rank.
For any matrix $A$, the dimension of the row space is equal to the dimension of the column space.
$\text{dim}(\text{Row}(A))=\text{dim}(\text{Col}(A))=\text{rank}(A)$
This is why we can just say "the rank of $A$" without specifying rows or columns—they are always the same.

3. The Four Fundamental Subspaces

We've now encountered the row space and column space. Two other subspaces are critical to understanding the full picture of a linear transformation: the null space, $\text{Nul}(A)$, and the left null space, $\text{Nul}(A^T)$.

The dimensions of these four subspaces are elegantly connected through the rank of the matrix. This relationship is often called the Fundamental Theorem of Linear Algebra, Part 1.

Let's consolidate these incredible results for an $m\times n$ matrix $A$ with $\text{rank}(A)=r$:

1. Column Space, $\text{Col}(A)$:

   • This is a subspace of ${\mathbb{R}}^m$.
   • $\text{dim}(\text{Col}(A))=r$ (the number of pivots).

2. Row Space, $\text{Row}(A)$:

   • This is a subspace of ${\mathbb{R}}^n$.
   • $\text{dim}(\text{Row}(A))=r$ (the number of pivots).

3. Null Space, $\text{Nul}(A)$:

   • This is the set of solutions to $A\mathbf{x}=0$, a subspace of ${\mathbb{R}}^n$.
   • A basis for the null space is found from the parametric vector form of the general solution.
   • The number of vectors in this basis equals the number of free variables in the system.
   • $\text{dim}(\text{Nul}(A))=n-r$ (number of columns - rank).

4. Left Null Space, $\text{Nul}(A^T)$:

   • This is the null space of the transpose matrix $A^T$, a subspace of ${\mathbb{R}}^m$.
   • Applying the null space rule to $A^T$ (which is an $n\times m$ matrix with the same rank $r$), we get:
   • $\text{dim}(\text{Nul}(A^T))=m-r$.

The relationship for the subspaces of ${\mathbb{R}}^n$ is particularly famous and is known as the Rank-Nullity Theorem:
$\underset{r}{\underbrace{\text{dim}(\text{Col}(A))}}+\underset{n-r}{\underbrace{\text{dim}(\text{Nul}(A))}}=n$
In words: rank + nullity = number of columns. This is a powerful accounting equation for any matrix.

Conclusion

Today, we've moved from the building blocks (bases) to a fundamental property of the entire structure (dimension). Dimension gives us a solid, quantitative way to describe vector spaces and subspaces.

Key Takeaways:

• Any two bases for a given vector space have the same number of vectors.
• The dimension of a vector space is the number of vectors in any of its bases.
• The rank of a matrix $A$ is the dimension of its column space, which is equal to the dimension of its row space. rank(A) = dim(Col(A)) = dim(Row(A)).
• The rank corresponds to the number of pivots in the echelon form of $A$.
• The dimensions of the four fundamental subspaces of an $m\times n$ matrix $A$ with rank $r$ are:
  • $\text{dim}(\text{Col}(A))=r$
  • $\text{dim}(\text{Row}(A))=r$
  • $\text{dim}(\text{Nul}(A))=n-r$ (This is the Rank-Nullity Theorem)
  • $\text{dim}(\text{Nul}(A^T))=m-r$

Preview of Next Lesson:

So far, we've focused heavily on vector spaces and subspaces, which always contain the zero vector. But what about solution sets to equations like $A\mathbf{x}=\mathbf{b}$ where $\mathbf{b}\ne 0$? These sets are not subspaces, but they have a very close relationship to them. In the next lesson, we will use the concepts of span and dimension to describe the geometric structure of these solution sets, introducing the idea of affine subspaces.