Understanding Vector Space Bases

1. What is a Basis?

Let's start by solidifying the intuition. A basis of a vector space is a set of "building block" vectors that are just right: there are enough of them to "reach" every vector in the space, but not so many that any of them are redundant.

The 3Blue1Brown video we've used before has an excellent, concise definition that brings together everything we've learned.

As the video explains, this leads us to the formal definition.

Definition: A Basis
A set of vectors $\{{\mathbf{v}}_1,{\mathbf{v}}_2,\dots ,{\mathbf{v}}_p\}$ in a subspace $W$ is a basis for $W$ if two conditions are met:

1. The set is linearly independent.
2. The set spans $W$ (i.e., $\text{span}(\{{\mathbf{v}}_1,\dots ,{\mathbf{v}}_p\})=W$).

This definition perfectly captures the idea of a minimal spanning set. Because the set is linearly independent, you cannot remove any vector without shrinking the span. It's perfectly efficient.

To reinforce this, the following video provides a clear, step-by-step explanation.

2. The Standard Basis and Uniqueness

You've been using a basis for years without knowing it! In the 2D Cartesian plane, the vectors $\mathbf{i}=\left(\begin{array}{c}1\\ 0\end{array}\right)$ and $\mathbf{j}=\left(\begin{array}{c}0\\ 1\end{array}\right)$ form a basis. They are linearly independent, and they span all of ${\mathbb{R}}^2$. This is called the standard basis.

However, a vector space has infinitely many different bases. For example, the vectors ${\mathbf{b}}_1=\left(\begin{array}{c}1\\ 1\end{array}\right)$ and ${\mathbf{b}}_2=\left(\begin{array}{c}1\\ -1\end{array}\right)$ also form a basis for ${\mathbb{R}}^2$. They are not perpendicular, but they are linearly independent and span the entire plane.

This idea that a vector is an abstract object, separate from its representation (its coordinates), is critical in both deep learning and quantum mechanics. The choice of basis is like choosing a language to describe the vector.

A key property that makes bases so useful is that for any given basis, the "language" is unambiguous.

Theorem: The Unique Representation Theorem
If $\mathcal{B}=\{{\mathbf{b}}_1,\dots ,{\mathbf{b}}_p\}$ is a basis for a subspace $W$, then for each vector $\mathbf{x}$ in $W$, there exists a unique set of scalars $c_1,\dots ,c_p$ such that:
$\mathbf{x}=c_1{\mathbf{b}}_1+\cdots +c_p{\mathbf{b}}_p$
These scalars $c_i$ are called the coordinates of $\mathbf{x}$ relative to the basis $\mathcal{B}$.

This is why coordinate systems work! The linear independence of the basis vectors ensures there's only one way to combine them to get any particular vector.

3. How to Construct a Basis

Now for the practical part: given a set of vectors that span a subspace, how do we find a basis for it? The set might be linearly dependent, so we need a systematic way to discard the redundant vectors.

The method relies on what we learned about pivot columns in the last lesson. Remember, the pivot columns of a matrix are not redundant; they are linearly independent of the preceding columns.

Let's walk through the procedure derived from this reading:

To find a basis for the subspace spanned by a set of vectors $\{{\mathbf{v}}_1,\dots ,{\mathbf{v}}_p\}$:

1. Form a matrix $A$ using these vectors as its columns: $A=\left[\begin{array}{cccc}{\mathbf{v}}_1& {\mathbf{v}}_2& \dots & {\mathbf{v}}_p\end{array}\right]$.
2. Row reduce the matrix $A$ to its echelon form.
3. Identify the pivot columns in the echelon form.
4. The basis consists of the corresponding columns from the original matrix $A$.

Example:
Find a basis for the subspace of ${\mathbb{R}}^3$ spanned by the vectors:
${\mathbf{v}}_1=\left(\begin{array}{c}1\\ -2\\ -1\end{array}\right),{\mathbf{v}}_2=\left(\begin{array}{c}-2\\ 4\\ 2\end{array}\right),{\mathbf{v}}_3=\left(\begin{array}{c}1\\ 0\\ 1\end{array}\right)$

1. Form the matrix $A$:
   $A=\left(\begin{array}{ccc}1& -2& 1\\ -2& 4& 0\\ -1& 2& 1\end{array}\right)$

2. Row reduce $A$:
   $\left(\begin{array}{ccc}1& -2& 1\\ -2& 4& 0\\ -1& 2& 1\end{array}\right)\stackrel{\text{row ops}}{\to }\left(\begin{array}{ccc}1& -2& 1\\ 0& 0& 2\\ 0& 0& 2\end{array}\right)\stackrel{\text{row ops}}{\to }\left(\begin{array}{ccc}1& -2& 0\\ 0& 0& 2\\ 0& 0& 0\end{array}\right)$
   (Note: This is an echelon form, not fully reduced, which is sufficient to identify pivots).

3. Identify pivot columns: The pivots are in columns 1 and 3.

4. Select columns from the original matrix $A$: The basis vectors are the 1st and 3rd columns of the original matrix $A$.
   So, a basis for the subspace is $\left\{\left(\begin{array}{c}1\\ -2\\ -1\end{array}\right),\left(\begin{array}{c}1\\ 0\\ 1\end{array}\right)\right\}$.

Notice that ${\mathbf{v}}_2=-2{\mathbf{v}}_1$, so it was the redundant vector, and our procedure correctly discarded it because the second column was not a pivot column.

Conclusion

Today, we've unified the concepts of span and linear independence to define a basis. This is one of the most important ideas in linear algebra, providing the fundamental "coordinate system" for any vector space.

Key Takeaways:

• A basis for a subspace is a set of vectors that is both linearly independent and spans the subspace.
• It can be thought of as a minimal spanning set—the most efficient collection of vectors to build the entire space.
• While a subspace has many different possible bases, a vector's representation (its coordinates) is unique for any single given basis.
• We can construct a basis for a space spanned by a set of vectors by putting them into a matrix and finding the original columns that correspond to the pivot columns.

Preview of Next Lesson:

In our examples, we saw that a basis for ${\mathbb{R}}^2$ had two vectors, and our example of a plane in ${\mathbb{R}}^3$ also had a basis with two vectors. This is no coincidence. It turns out that every basis for a given subspace has the exact same number of vectors. This fundamental number is called the dimension of the subspace. In our next lesson, we will explore this concept and the powerful insights it provides.