Vector Combinations and Linear Independence

Building Vectors: Linear Combinations and Span

At its core, a linear combination is simply an expression formed by scaling several vectors and adding them all together. The result is a new vector. You've already seen a glimpse of this when we thought of a vector's coordinates, like $(3,-2)$, as scaling the standard basis vectors $\mathbf{i}$ and $\mathbf{j}$ to get $3\mathbf{i}-2\mathbf{j}$.

The following video from 3Blue1Brown offers an excellent visual intuition for this idea.

Formal Definition of a Linear Combination

Formally, given a set of vectors $\{{\mathbf{v}}_1,{\mathbf{v}}_2,\dots ,{\mathbf{v}}_n\}$ in ${\mathbb{R}}^m$ and a set of corresponding scalars $\{c_1,c_2,\dots ,c_n\}$ in $\mathbb{R}$, a new vector $\mathbf{v}$ is a linear combination of these vectors if it can be written as:

The scalars $c_i$ are the "weights" or "coefficients" of the combination.

A natural question arises: if I have a target vector $\mathbf{v}$ and a set of "building block" vectors $\{{\mathbf{v}}_1,\dots ,{\mathbf{v}}_n\}$, how can I find the specific coefficients $c_i$ to construct $\mathbf{v}$? This transforms the problem into solving a system of linear equations.

The following video walks through this exact process. Because you are comfortable with algorithmic thinking, you'll find the method of setting up an augmented matrix and performing Gaussian elimination to be a very systematic approach.

Span: The Set of All Reachable Vectors

If we consider all possible linear combinations of a set of vectors, we define a new object: the span of those vectors. It's the complete set of all vectors that can be reached by scaling and adding the original vectors in every conceivable way.

For a rigorous definition, we turn to the first resource below. Since you value formal definitions, this will provide the precise mathematical language for this concept.

The span of a set of vectors can be a line, a plane, a higher-dimensional hyperplane, or even the entire space. The 3Blue1Brown video you saw earlier gives a beautiful geometric feel for this.

This image illustrates a key idea: sometimes our set of "building block" vectors contains redundancies. This leads directly to our next topic.

Redundancy and Uniqueness: Linear Independence

The concept of linear independence helps us formalize the notion of a "minimal" set of vectors needed to define a span. A set of vectors is linearly independent if no vector in the set can be written as a linear combination of the others. In other words, each vector adds a genuinely new direction to the span.

Conversely, a set is linearly dependent if at least one vector is redundant—it's already in the span of the other vectors.

Formal Definitions and The Test for Independence

For a rigorous treatment, which is your goal, we need precise definitions and a practical test. The following reading from EECS 16A at Berkeley provides exactly this. It introduces two equivalent definitions of linear dependence and formally proves their equivalence.

The most powerful and practical way to test for linear independence comes from the first definition you just read. A set of vectors $\{{\mathbf{v}}_1,\dots ,{\mathbf{v}}_n\}$ is linearly independent if and only if the only solution to the equation:

is the trivial solution, where $c_1=c_2=\cdots =c_n=0$.

If there is any non-trivial solution (where at least one $c_i$ is not zero), the set is linearly dependent. This gives us a concrete computational test: set up the system of equations $\sum c_i{\mathbf{v}}_i=0$ and solve for the $c_i$'s. If the only solution is all zeros, the vectors are independent.

Since you appreciate full proofs, the following reading formally proves this "iff" condition.

Let's trace the logic of a test. Consider the vectors $\mathbf{a}=\left(\begin{array}{c}2\\ 1\end{array}\right)$ and $\mathbf{b}=\left(\begin{array}{c}1\\ 5\end{array}\right)$ from "Example 3.2" in the EECS16A notes. To test for linear independence, we set up the equation $c_1\mathbf{a}+c_2\mathbf{b}=0$:

This vector equation becomes a system of linear equations:

Solving this system (e.g., using substitution or Gaussian elimination) reveals that the only possible solution is $c_1=0$ and $c_2=0$. Since only the trivial solution exists, we conclude that the vectors $\mathbf{a}$ and $\mathbf{b}$ are linearly independent.

Conclusion

In this lesson, we have moved from manipulating single vectors to understanding how they combine to form vector spaces. We have established a rigorous and practical understanding of two of the most fundamental concepts in linear algebra.

Key Takeaways:

• A linear combination is a sum of scaled vectors: $\mathbf{v}=\sum c_i{\mathbf{v}}_i$. The process of finding the coefficients $c_i$ to form a specific vector $\mathbf{v}$ amounts to solving a system of linear equations.
• The span of a set of vectors is the collection of all possible linear combinations, representing every vector that can be "reached" by the set.
• A set of vectors is linearly independent if none of them are redundant—that is, no vector can be written as a linear combination of the others.
• The definitive test for linear independence is to solve $\sum c_i{\mathbf{v}}_i=0$. The set is linearly independent if and only if the only solution is the trivial one ($c_i=0$ for all $i$).

We have now laid all the necessary groundwork for our next topic. We have seen how vectors can be combined and how to identify a "minimal" set of vectors that define a space through their span. In the next lesson, we will formalize this by defining vector spaces and subspaces, and see how a linearly independent spanning set forms a basis for that space.