Understanding Vector Spaces and Subspaces

Introduction

In our last lesson, we explored how vectors can be combined using linear combinations to construct new vectors and how the span of a set of vectors defines all the "reachable" points. These operations—scaling and adding—are the fundamental building blocks of linear algebra. Today, we will take a significant step up in abstraction by formalizing the environment where these operations live.

This lesson introduces the concept of a vector space. Instead of thinking of vectors merely as arrows or lists of numbers, we will define them through a set of axiomatic rules. This abstract definition is incredibly powerful because it allows us to treat a wide variety of mathematical objects, such as matrices and functions, as vectors. You will learn the ten axioms that define a vector space and how to verify them. We will then define a subspace—a vector space that resides within a larger one—and learn a simple three-step test to identify them.

This formalization is the bedrock for the rest of our journey. Understanding these structures is essential before we can discuss the transformations between them, which will be our topic for the next module.

The Abstract Definition of a Vector Space

So far, our intuition for vectors has been grounded in the familiar space $R^{n}$ . However, the principles of vector addition and scalar multiplication apply to many other objects in mathematics and machine learning. To unify these, we define a vector space not by what its elements are, but by what they do.

A vector space consists of a set of "vectors" $V$ and a field of scalars $F$ (for our purposes, this will always be the real numbers $R$ ) that must obey a list of ten rules, or axioms. These axioms ensure that the objects behave in the consistent, predictable way we expect from our experience with arrows in $R^{n}$ .

The following reading from Matthew N. Bernstein's blog provides the rigorous definition you'll need.

Vector spaces

This article moves from the intuitive idea of vectors as arrows to the formal, abstract definition of a vector space, which is exactly the transition we are making now.

Start with the introduction, which sets the stage. Then, carefully study the Formal definition section. Focus on the list of ten axioms. Notice how they are grouped into rules for vector addition (axioms 1-5) and rules for scalar multiplication (axioms 6-10).

These ten axioms are not arbitrary; they are the minimum set of rules required to guarantee that our vector space behaves as we'd intuitively expect. For instance, they are sufficient to prove other properties, like the uniqueness of the zero vector or the fact that multiplying any vector by the scalar 0 yields the zero vector. The "Properties" and "Appendix" sections of the article you just read delve into these proofs, which are excellent for exercising the type of rigorous thinking you're aiming to develop.

Examples of Vector Spaces

The power of this abstract definition becomes clear when we see the variety of things that can be considered vector spaces. Verifying the axioms is the formal method to confirm if a set of objects qualifies.

The most obvious example is $R^{n}$ , the set of all n-tuples of real numbers. All our work so far has implicitly assumed it's a vector space. But let's explore some more interesting cases.

The following resource provides a list of common vector spaces and walks through the verification process for two of them in detail: the space of polynomials and the space of functions.

[PDF] Linear Algebra & Differential Equations - Elif Uskuplu

These lecture notes provide a clear, structured list of vector space axioms and multiple, detailed examples.

First, review the definition of a vector space for reinforcement. Next, look through the list of different types of vector spaces, including Euclidean spaces, polynomial spaces, and matrix spaces. This will broaden your perspective on what a "vector" can be. Finally, work through the detailed verifications for the space of polynomials (Pn) and the space of functions (Fun(D, R)). This demonstrates exactly how to apply the ten axioms to prove that a given set is a vector space.

As you can see, objects you might have considered distinct from vectors—like matrices, polynomials, and functions—can all inhabit vector spaces. When you use libraries like NumPy or PyTorch, the arrays and tensors you manipulate are elements of a vector space. The operations like + and * are defined to satisfy these very axioms.

Subspaces: Vector Spaces Within Vector Spaces

Often, we are interested in a set of vectors that is a subset of a larger vector space, but which forms a vector space in its own right. We call such a set a subspace.

A visual representation of subspaces within the vector space $\mathbb{R}^3$. A line passing through the origin is a 1D subspace, and a plane passing through the origin is a 2D subspace. The entire 3D space is also a subspace of itself.

To check if a subset $S$ of a vector space $V$ is a subspace, we don't need to re-verify all ten axioms. Since the vectors in $S$ are also in $V$ , they automatically inherit most of the properties (like commutativity and associativity). We only need to check the properties that ensure $S$ is self-contained. This leads to a powerful three-point test.

The following video introduces this test and provides excellent visual intuition for why each condition is necessary.

Subspaces are the Natural Subsets of Linear Algebra Definition + First Examples

This video by Dr. Trefor Bazett gives a concise and visually intuitive explanation of subspaces and the three conditions for verifying them.

Watch the entire video (the full explanation). Pay close attention to the visual examples of sets that fail the test (like a line not through the origin) and those that pass (like a line through the origin). This will solidify your understanding of the three conditions.

To formally state the test, a non-empty subset $S$ of a vector space $V$ is a subspace of $V$ if and only if the following three conditions hold:

Contains the Zero Vector: The zero vector of $V$ is in $S$ .
$0 \in S$
Closed Under Addition: For any two vectors $u$ and $v$ in $S$ , their sum $u + v$ is also in $S$ .
Closed Under Scalar Multiplication: For any vector $u$ in $S$ and any scalar $c$ , the vector $c u$ is also in $S$ .

If a subset meets these three criteria, it's guaranteed to be a vector space.

Verifying Subspaces: A Practical Guide

Let's apply this three-step verification process to a few examples, moving from the geometric spaces of $R^{n}$ to the more abstract spaces of polynomials and functions.

This next video provides a series of worked examples, which is perfect practice for mastering the verification process.

4.1 Vector Spaces and Subspaces

This lecture by Jordan Webster walks through the definition of a subspace and demonstrates the verification process on multiple examples from different vector spaces.

First, watch the formal definition of a subspace and the three-property test. Next, follow the worked examples for subsets of \mathbb{R}^3. Observe how to prove a set is a subspace (Example 1) and how to prove a set is not by showing it fails just one of the conditions (Example 2). Then, see how the same logic applies to a subspace of polynomials. Finally, watch the verification for a subspace of functions, which demonstrates the power and generality of this concept.

A Crucial Link: Span and Subspaces

There is a deep connection between the topic of our last lesson and today's. A key result in linear algebra is that the span of any set of vectors in a vector space $V$ is a subspace of $V$ .

Let's quickly verify this using our three-point test. Let $S = span {v_{1}, \dots, v_{k}}$ .

Zero Vector: Is $0$ in $S$ ? Yes. We can create the zero vector by choosing all coefficients to be zero: $0 v_{1} + 0 v_{2} + \dots + 0 v_{k} = 0$ .
Closure Under Addition: Let $u$ and $w$ be two vectors in $S$ . By definition, they are linear combinations of the basis vectors:
$u = c_{1} v_{1} + \dots + c_{k} v_{k}$
$w = d_{1} v_{1} + \dots + d_{k} v_{k}$
Their sum is:
$u + w = (c_{1} + d_{1}) v_{1} + \dots + (c_{k} + d_{k}) v_{k}$
This is just another linear combination of the same vectors, so $u + w$ is in $S$ .
Closure Under Scalar Multiplication: Let $u$ be in $S$ and $α$ be a scalar.
$α u = α (c_{1} v_{1} + \dots + c_{k} v_{k}) = (α c_{1}) v_{1} + \dots + (α c_{k}) v_{k}$
This is also a linear combination, so $α u$ is in $S$ .

Since all three conditions hold, the span of any set of vectors is always a subspace. This is a fundamental insight: the operation of "span" naturally constructs these well-behaved, self-contained vector spaces.

Conclusion

This lesson marked a crucial shift from the concrete to the abstract. By defining vector spaces through a set of axioms, we've generalized the concept far beyond simple geometric arrows.

Key Takeaways:

A vector space is a collection of objects (vectors) and scalars that satisfy ten specific axioms governing vector addition and scalar multiplication.
This abstract definition is powerful because many different mathematical objects, including $R^{n}$ , matrices, polynomials, and functions, can be shown to be vector spaces.
A subspace is a subset of a vector space that is itself a vector space.
We can verify if a non-empty subset $S$ is a subspace using a simple three-part test: check if it contains the zero vector, is closed under addition, and is closed under scalar multiplication.
The span of any set of vectors is always a subspace of the parent vector space.

Now that we have formally defined the spaces that vectors live in, we are ready to explore the connections between them. In the next lesson, we will begin our study of matrices, looking at them not just as arrays of numbers, but as representations of linear maps—special functions that transform vectors from one vector space to another while preserving their essential structure.

Recommended resources