Understanding Vector Span

What is Span? The Geometry of Reachable Vectors

Let's begin by solidifying the geometric intuition. The span of a set of vectors is the collection of all possible vectors you can reach by performing linear combinations. A linear combination, as we've discussed, is the result of scaling your vectors and adding them together: $\mathbf{w}=c_1{\mathbf{v}}_1+c_2{\mathbf{v}}_2+\cdots +c_n{\mathbf{v}}_n$.

The span is the answer to the question: "What are all the possible destinations $\mathbf{w}$ we can reach if we are only allowed to scale and add our initial vectors ${\mathbf{v}}_i$?"

The 3Blue1Brown video we watched in the last lesson provides an excellent visualization of this. Let's revisit a key segment with this new focus.

As the video illustrates, the geometry of the span depends on the vectors involved:

• The span of a single non-zero vector $\mathbf{v}$ is a line. It consists of all scalar multiples $c\mathbf{v}$.
• The span of two non-collinear vectors is a plane.
• The span of three linearly independent vectors in ${\mathbb{R}}^3$ is all of ${\mathbb{R}}^3$.

The image below visualizes the span of two vectors in 3D space. Any vector that can be formed by adding a scaled version of the first vector to a scaled version of the second will have its tip on that blue plane.

The Algebraic Test: Is a Vector in the Span?

Visual intuition is powerful, but we need a systematic, computational method to determine if a vector $\mathbf{b}$ lies in the span of a set of vectors $\{{\mathbf{v}}_1,{\mathbf{v}}_2,\dots ,{\mathbf{v}}_n\}$.

The question "Is $\mathbf{b}$ in the span?" is algebraically equivalent to asking: "Do there exist scalars $c_1,c_2,\dots ,c_n$ such that this equation is true?"

This vector equation should look familiar. It's the definition of a system of linear equations. If we construct a matrix $A$ whose columns are our vectors, $A=[{\mathbf{v}}_1|{\mathbf{v}}_2|\dots |{\mathbf{v}}_n]$, and a column vector $\mathbf{c}$ of the unknown scalars, $\mathbf{c}=[c_1,c_2,\dots ,c_n{]}^T$, then the equation becomes:

So, the question transforms once more: A vector $\mathbf{b}$ is in the span of the columns of a matrix $A$ if and only if the system $A\mathbf{c}=\mathbf{b}$ is consistent (i.e., has at least one solution).

How do we test for consistency? We form the augmented matrix $[A|\mathbf{b}]$ and perform row reduction. The system is inconsistent only if we find a row of the form $[00\dots 0|k]$ where $k$ is a non-zero number. Any other outcome means a solution exists, and $\mathbf{b}$ is in the span.

The following resource formalizes this exact connection.

Let's look at two examples that illustrate this procedure.

When Does a Set of Vectors Span an Entire Space?

A crucial question in many applications is whether a set of vectors is "powerful" enough to generate every vector in the space. When does $\mathrm{Span}\{{\mathbf{v}}_1,\dots ,{\mathbf{v}}_n\}$ equal all of ${\mathbb{R}}^m$?

The examples above give us the answer. For the span to be all of ${\mathbb{R}}^m$, the system $A\mathbf{c}=\mathbf{b}$ must be consistent for every possible vector $\mathbf{b}$. This is only possible if the row-reduced echelon form of the coefficient matrix $A$ has a pivot position in every row.

Why? A pivot in every row ensures that it's impossible to create a contradiction row $[0\dots 0|k]$ for any $\mathbf{b}$.

This leads to an important consequence. Since each pivot must be in its own column, a matrix with $m$ rows must have at least $m$ columns to have a pivot in every row.

This gives us two powerful propositions.

Practice Problem

Let's put this together. Consider the vectors ${\mathbf{v}}_1=\left[\begin{array}{c}1\\ 2\\ -1\end{array}\right]$ and ${\mathbf{v}}_2=\left[\begin{array}{c}-3\\ -4\\ 1\end{array}\right]$.

Determine if the vector $\mathbf{b}=\left[\begin{array}{c}-1\\ 0\\ 1\end{array}\right]$ is in the span of $\{{\mathbf{v}}_1,{\mathbf{v}}_2\}$.

Your Task:

1. Set up the augmented matrix $[A|\mathbf{b}]$.
2. Perform row reduction to find its echelon form.
3. Based on the result, determine if the system is consistent and if $\mathbf{b}$ is in the span.

Solution:

1. The augmented matrix is $\left[\begin{array}{cccc}1& -3& |& -1\\ 2& -4& |& 0\\ -1& 1& |& 1\end{array}\right]$.
2. Row reduction proceeds as follows:
   • $R_2\leftarrow R_2-2R_1$; $R_3\leftarrow R_3+R_1$
     $\left[\begin{array}{cccc}1& -3& |& -1\\ 0& 2& |& 2\\ 0& -2& |& 0\end{array}\right]$
   • $R_3\leftarrow R_3+R_2$
     $\left[\begin{array}{cccc}1& -3& |& -1\\ 0& 2& |& 2\\ 0& 0& |& 2\end{array}\right]$
3. The last row corresponds to the equation $0\cdot c_1+0\cdot c_2=2$, which is a contradiction. The system is inconsistent. Therefore, the vector $\mathbf{b}$ is not in the span of $\{{\mathbf{v}}_1,{\mathbf{v}}_2\}$. Geometrically, ${\mathbf{v}}_1$ and ${\mathbf{v}}_2$ define a plane in ${\mathbb{R}}^3$, and the vector $\mathbf{b}$ does not lie on that plane.

Conclusion

In this lesson, we delved deep into the concept of span, moving from geometric intuition to a robust algebraic test.

Here are the key takeaways:

• Span (Definition): The span of a set of vectors is the set of all possible linear combinations of those vectors.
• Span (Geometry): Geometrically, the span is the line, plane, or higher-dimensional subspace that can be "reached" using the given vectors.
• Testing for Span Membership: A vector $\mathbf{b}$ is in the span of the columns of matrix $A$ if and only if the system $A\mathbf{c}=\mathbf{b}$ is consistent. This is checked by row-reducing the augmented matrix $[A|\mathbf{b}]$.
• Spanning an Entire Space: A set of vectors spans ${\mathbb{R}}^m$ if and only if their matrix has a pivot in every row. This requires at least $m$ vectors.

We have now thoroughly investigated two pillar concepts: linear independence (a property of a set of vectors ensuring no redundancy) and span (the space generated by a set of vectors). What happens when a set of vectors has both of these properties? That is, what if it's perfectly efficient (linearly independent) and also generates the entire space we care about (it spans the space)? This ideal set of vectors is called a basis, and it will be the focus of our next lesson.