Understanding and Proving Bases for Vector Spaces

1. The Definition of a Basis

A basis strikes a perfect balance between two properties: spanning and linear independence.

Let's watch a segment from Professor Gilbert Strang's lecture, where he formally defines a basis and provides initial examples.

To formalize this, let's consult a written definition.

As the resources state, a set of vectors $B=\{{\mathbf{v}}_1,\dots ,{\mathbf{v}}_n\}$ is a basis for a vector space $V$ if it satisfies two conditions:

1. Linear Independence: The set $B$ is linearly independent.
2. Spanning Property: The set $B$ spans $V$, i.e., $\text{span}(B)=V$.

The Uniqueness of Representation: A Key Consequence

A crucial property that follows directly from this definition is that any vector $\mathbf{v}\in V$ can be written as a linear combination of basis vectors in exactly one way.

Proof Sketch:
Let $B=\{{\mathbf{v}}_1,\dots ,{\mathbf{v}}_n\}$ be a basis for $V$.

• Existence: Since $B$ spans $V$, for any $\mathbf{v}\in V$, there must exist scalars $c_1,\dots ,c_n$ such that $\mathbf{v}=c_1{\mathbf{v}}_1+\cdots +c_n{\mathbf{v}}_n$. So at least one representation exists.
• Uniqueness: Suppose there were two different representations for $\mathbf{v}$:
  $\mathbf{v}=c_1{\mathbf{v}}_1+\cdots +c_n{\mathbf{v}}_n$
  $\mathbf{v}=d_1{\mathbf{v}}_1+\cdots +d_n{\mathbf{v}}_n$
  Subtracting the second equation from the first gives:
  $0=(c_1-d_1){\mathbf{v}}_1+\cdots +(c_n-d_n){\mathbf{v}}_n$
  Since the basis vectors are linearly independent, the only solution to this equation is the trivial one. Therefore, $(c_i-d_i)=0$ for all $i$, which means $c_i=d_i$. The representation is unique.

These unique scalars $(c_1,\dots ,c_n)$ are called the coordinates of $\mathbf{v}$ with respect to the basis $B$.

2. Proving a Set is a Basis

To prove that a set of vectors forms a basis for a vector space, you must demonstrate that it satisfies both conditions: it is linearly independent AND it spans the space.

Example: The Standard Basis for ${\mathbb{R}}^n$

The most familiar basis is the standard basis. In ${\mathbb{R}}^2$, it consists of the vectors ${\mathbf{e}}_1=[1,0{]}^T$ and ${\mathbf{e}}_2=[0,1{]}^T$.

More generally, the standard basis for ${\mathbb{R}}^n$ is the set $\{{\mathbf{e}}_1,{\mathbf{e}}_2,\dots ,{\mathbf{e}}_n\}$, where ${\mathbf{e}}_i$ is a vector with a 1 in the $i$-th position and 0s everywhere else. These vectors form the columns of the $n\times n$ identity matrix, $I_n$.

The Matrix Method for $n$ Vectors in ${\mathbb{R}}^n$

Suppose we have a set of $n$ vectors $\{{\mathbf{v}}_1,\dots ,{\mathbf{v}}_n\}$ in ${\mathbb{R}}^n$. To check if they form a basis, we can create an $n\times n$ matrix $A$ with these vectors as its columns: $A=[{\mathbf{v}}_1\cdots {\mathbf{v}}_n]$.

Now, let's translate our two conditions into properties of the matrix $A$:

1. Linear Independence: The columns are linearly independent if the equation $A\mathbf{x}=0$ has only the trivial solution ($\mathbf{x}=0$).
2. Spanning ${\mathbb{R}}^n$: The columns span ${\mathbb{R}}^n$ if the equation $A\mathbf{x}=\mathbf{b}$ has a solution for every $\mathbf{b}\in {\mathbb{R}}^n$.

For a square matrix $A$, these two conditions are equivalent to $A$ being invertible. This gives us a powerful and efficient method:

A set of $n$ vectors in ${\mathbb{R}}^n$ forms a basis for ${\mathbb{R}}^n$ if and only if the matrix formed by using these vectors as columns is invertible.

Let's return to the lecture to see how this test is applied.

3. Beyond ${\mathbb{R}}^n$: Bases for Other Vector Spaces

The concept of a basis is not limited to column vectors in ${\mathbb{R}}^n$. It applies to any vector space, including spaces of matrices, functions, or, as we'll see here, polynomials. Your background in computer science has likely prepared you to think about abstract structures, and vector spaces are a prime example.

Consider $P_2$, the vector space of all polynomials of degree at most 2. The standard basis for $P_2$ is $\{1,x,x^2\}$. Any polynomial $a+bx+c{x}^2$ is a unique linear combination of these basis vectors.

But what about a non-standard set, like $\{1-x,1+x,x^2\}$? To prove it's a basis for $P_2$, we must show it's linearly independent and spans $P_2$. This involves setting up and solving a system of equations, just as with vectors in ${\mathbb{R}}^n$.

The process is analogous:

1. Set up the equation: To check for both properties, we set an arbitrary polynomial $a+bx+c{x}^2$ equal to a linear combination of our candidate vectors:
   $c_1(1-x)+c_2(1+x)+c_3(x^2)=a+bx+c{x}^2$
2. Equate coefficients: Group terms by powers of $x$ and equate the coefficients on both sides. This creates a system of linear equations for $c_1,c_2,c_3$.
3. Solve the system: Show that this system has a unique solution for any $a,b,c$. This is equivalent to showing the coefficient matrix is invertible, which proves both linear independence (when $a=b=c=0$) and the spanning property.

4. A Deeper Look: Proving Key Properties

Given your interest in proofs, let's explore a powerful theorem that can simplify the process of verifying a basis for a finite-dimensional space.

So far, we know that all bases for a given space have the same number of vectors. We will prove this in the next lesson, but for now, let's accept it and call this number the dimension of the space, denoted $\dim (V)$. For example, $\dim ({\mathbb{R}}^n)=n$ and $\dim (P_2)=3$.

Theorem: Let $V$ be a vector space with $\dim (V)=n$.

1. Any linearly independent set of $n$ vectors in $V$ is automatically a basis for $V$.
2. Any spanning set of $n$ vectors in $V$ is automatically a basis for $V$.

This means if you have the correct number of vectors, you only need to check one of the two conditions, not both!

Your challenge is to prove a core part of this theorem.

Proof Exercise:
Prove that if $V$ is a vector space of dimension $n$, then a set of $n$ vectors $\{{\mathbf{v}}_1,\dots ,{\mathbf{v}}_n\}$ is linearly independent if and only if it spans $V$.

Finally, a fundamental question in mathematics is whether something is guaranteed to exist. Does every vector space have a basis? The answer is yes, but the proof for infinite-dimensional spaces relies on a non-constructive axiom from set theory called the Axiom of Choice (via Zorn's Lemma). The resource below contains this proof if you're curious about such foundational topics.

Conclusion

Today, we defined the concept of a basis—a linearly independent spanning set. This is a cornerstone of linear algebra, providing the minimal set of vectors needed to build an entire vector space.

Key Takeaways:

• A basis for a vector space is a set of vectors that is both linearly independent and spans the space.
• A basis provides a unique representation (coordinates) for every vector in the space.
• To prove a set is a basis, you must prove both conditions.
• For a set of $n$ vectors in an $n$-dimensional space (like ${\mathbb{R}}^n$), you can form an $n\times n$ matrix with the vectors as columns. The set is a basis if and only if the matrix is invertible.
• If you know the dimension of a space is $n$, any set of $n$ linearly independent vectors is a basis, and any set of $n$ spanning vectors is a basis.

Preview of the next lesson:
We've mentioned that all bases for a given vector space seem to have the same number of elements. In our next lesson, we will formally prove this and define the dimension of a vector space. Dimension is a fundamental property that tells us the "size" or "degrees of freedom" of a space, a concept essential for applications like PCA in machine learning.