Linear Independence and Span

1. Building Vectors: Linear Combinations and Span

Every vector in a standard coordinate system, like $\left[\begin{array}{c}3\\ -2\end{array}\right]$, can be thought of as a recipe: "start at the origin, move 3 units along the x-axis, and -2 units along the y-axis." This is a simple linear combination of the standard basis vectors, $\hat{ı}$ and $\hat{ȷ}$.

What if we used different "building block" vectors? This is the core idea behind linear combinations and span. A linear combination of a set of vectors $\{{\vec{v}}_1,{\vec{v}}_2,\dots ,{\vec{v}}_n\}$ is any vector that can be formed by scaling them by some scalars $c_1,c_2,\dots ,c_n$ and adding them together:

The span is the set of all possible vectors you can create using this process. To build a powerful intuition for this, there's no better resource than the 3Blue1Brown video on the topic.

As the video illustrates, the geometry of the span depends entirely on the vectors you start with:

• The span of a single non-zero vector is a line.
• The span of two vectors that don't lie on the same line (i.e., are not collinear) is a plane.
• In the "unlucky" case where the two vectors are collinear, their span is just a line.

2. Redundancy: Linear Dependence and Independence

The video hinted at an important question: what happens when adding a new vector doesn't expand the span? This occurs when the new vector is already reachable using the existing vectors—it's redundant. This idea is central to the concept of linear independence.

Let's continue with the 3Blue1Brown video to get the initial intuition.

The video gives us these intuitive definitions:

• A set of vectors is linearly dependent if at least one vector is redundant, meaning it can be expressed as a linear combination of the others. Removing this vector doesn't shrink the span.
• A set of vectors is linearly independent if every vector adds a new dimension to the span. No vector is redundant.

The Formal Definition

While the geometric intuition is powerful, we also need a precise, algebraic way to define this. This is where the concept of the zero vector comes in.

A set of vectors $\{{\vec{v}}_1,{\vec{v}}_2,\dots ,{\vec{v}}_n\}$ is linearly dependent if there exists a set of scalars $c_1,c_2,\dots ,c_n$, not all of which are zero, such that:

This is called a "non-trivial" combination that results in the zero vector.

Conversely, the set is linearly independent if the only way to get the zero vector is the "trivial" solution where $c_1=c_2=\cdots =c_n=0$.

Why are these definitions equivalent? If a non-trivial combination gives the zero vector, you can solve for one of the vectors in terms of the others (assuming its coefficient is non-zero), proving it's in their span and therefore redundant.

The following reading provides a clear, formal summary of these definitions.

3. A Glimpse of Basis and Dimension

The concepts of span and linear independence naturally lead to two of the most important ideas in linear algebra: basis and dimension.

• A basis of a vector space is a set of linearly independent vectors that spans the entire space.
• The dimension of a vector space is the number of vectors in its basis.

Think of a basis as the most efficient set of building blocks for a space. It contains the minimum number of vectors needed to "span" (reach) every point in the space, with no redundancy. For example, the standard basis for ${\mathbb{R}}^2$ is $\{\left[\begin{array}{c}1\\ 0\end{array}\right],\left[\begin{array}{c}0\\ 1\end{array}\right]\}$. These two vectors are linearly independent, and they span the entire 2D plane. Therefore, the dimension of ${\mathbb{R}}^2$ is 2.

Let's hear a quick, formal explanation from MIT's Gilbert Strang to solidify these connected ideas.

We will explore these concepts more deeply in future lessons, but it's useful to see how they all fit together now.

Conclusion

In this lesson, we established a crucial vocabulary for describing vector spaces. We moved from the simple idea of scaling and adding vectors to understanding the entire space they can form.

Key Takeaways:

• A linear combination is a weighted sum of vectors: $\vec{w}=c_1{\vec{v}}_1+c_2{\vec{v}}_2$.
• The span of a set of vectors is the collection of all possible linear combinations—geometrically, a line, a plane, or a higher-dimensional space.
• A set of vectors is linearly independent if each vector adds a new direction, meaning no vector is in the span of the others. Algebraically, the only linear combination that equals the zero vector is the one where all scalars are zero.
• A set of vectors is linearly dependent if at least one vector is redundant (i.e., in the span of the others). This means a non-trivial linear combination can sum to the zero vector.
• A basis is a linearly independent set of vectors that spans a space. The number of vectors in the basis is the dimension of the space.

These concepts are the bedrock for what comes next. In our next lesson, we will apply this framework to matrices by exploring the column space and null space. You'll see that the span of a matrix's columns tells us about the possible outputs of a transformation, while linear independence helps us understand the structure of its solutions.