Vector Spaces: Definitions and Axioms

1. Beyond Arrows: What is a Vector?

To begin, let's challenge the familiar notion of a vector. What if objects that don't look like arrows at all could behave like vectors? For example, can we "add" two functions together, or "scale" a function by a number? If so, could we apply the tools of linear algebra to them?

The following video from 3Blue1Brown provides a wonderful intuition for why mathematicians developed the abstract definition of a vector space. It shows how the properties of "vector-ness" can be found in unexpected places.

As the video explains, the key idea is that as long as we have a set of objects with a sensible notion of addition and scalar multiplication, all the machinery of linear algebra—linear transformations, eigenvectors, and so on—can be applied. The axioms are simply a formal checklist to ensure our definitions of addition and scaling are "sensible" and behave as we'd expect.

2. The Formal Definition of a Vector Space

Now let's formalize the ideas from the video. A vector space is not just a set of vectors; it's a complete package. To define one, you need four ingredients:

1. A non-empty set of objects, which we call vectors (let's call the set $V$).
2. A field of numbers, which we call scalars (let's call the field $F$).
3. An operation called vector addition (+), which combines any two vectors in $V$ to produce another vector in $V$.
4. An operation called scalar multiplication (·), which combines any scalar from $F$ and any vector from $V$ to produce another vector in $V$.

What is a Field?

A field is just a set of numbers where addition, subtraction, multiplication, and division (except by zero) are all well-defined and behave as you'd expect.

For this course, the only two fields you need to worry about are:

• The field of real numbers, $\mathbb{R}$. Vector spaces over $\mathbb{R}$ are called real vector spaces. These are central to physics, engineering, and most applications in deep learning.
• The field of complex numbers, $\mathbb{C}$. Vector spaces over $\mathbb{C}$ are called complex vector spaces. These are absolutely essential for quantum mechanics.

The Ten Axioms

For the "package" of ($V$, $F$, +, ·) to be a vector space, the operations of addition and scalar multiplication must satisfy ten rules, known as the axioms. These axioms ensure that vectors behave in a predictable and consistent way.

The video below gives a clear and concise walkthrough of these ten axioms.

Here is a summary of the axioms for a vector space $V$ over a field $F$. For any vectors $\mathbf{u},\mathbf{v},\mathbf{w}\in V$ and any scalars $c,d\in F$:

Axioms for Vector Addition:

1. Closure under Addition: $\mathbf{u}+\mathbf{v}$ is in $V$.
2. Commutativity: $\mathbf{u}+\mathbf{v}=\mathbf{v}+\mathbf{u}$.
3. Associativity: $(\mathbf{u}+\mathbf{v})+\mathbf{w}=\mathbf{u}+(\mathbf{v}+\mathbf{w})$.
4. Existence of a Zero Vector: There exists a vector $0\in V$ such that $\mathbf{v}+0=\mathbf{v}$ for all $\mathbf{v}\in V$.
5. Existence of an Additive Inverse: For every $\mathbf{v}\in V$, there exists a vector $-\mathbf{v}\in V$ such that $\mathbf{v}+(-\mathbf{v})=0$.

Axioms for Scalar Multiplication:
6. Closure under Scalar Multiplication: $c\mathbf{v}$ is in $V$.
7. Distributivity over Vector Addition: $c(\mathbf{u}+\mathbf{v})=c\mathbf{u}+c\mathbf{v}$.
8. Distributivity over Scalar Addition: $(c+d)\mathbf{v}=c\mathbf{v}+d\mathbf{v}$.
9. Associativity of scalars: $(cd)\mathbf{v}=c(d\mathbf{v})$.
10. Existence of a Multiplicative Identity: $1\mathbf{v}=\mathbf{v}$, where $1$ is the multiplicative identity of the field $F$.

3. Verifying the Axioms: Examples and Counterexamples

Knowing the axioms is one thing; applying them is another. The process of checking if a set with its operations forms a vector space is called verifying the axioms.

Familiar Examples

You are already familiar with several vector spaces!

• ${\mathbb{R}}^n$: The set of all n-tuples (or column vectors) of real numbers is a real vector space with the standard component-wise addition and scalar multiplication. This is the most common setting for deep learning models.
• $M_{m\times n}(\mathbb{R})$: The set of all $m\times n$ matrices with real entries is a real vector space, with standard matrix addition and scalar multiplication.
• ${\mathbb{C}}^n$: The set of all n-tuples of complex numbers is a complex vector space. This is the primary stage for quantum mechanics, where vectors represent quantum states (e.g., the state of a qubit can be represented by a vector in ${\mathbb{C}}^2$).

For ${\mathbb{C}}^n$, addition and scalar multiplication are defined just like in ${\mathbb{R}}^n$, but component-wise with complex numbers. For $\mathbf{u}=(u_1,...,u_n)$ and $\mathbf{v}=(v_1,...,v_n)$ in ${\mathbb{C}}^n$ and a scalar $c\in \mathbb{C}$:
$\mathbf{u}+\mathbf{v}=(u_1+v_1,...,u_n+v_n)$
$c\mathbf{u}=(c{u}_1,...,c{u}_n)$

When Things Go Wrong

Sometimes, a set that looks like it should be a vector space fails on one or more axioms. Proving a set is not a vector space is often easier than proving it is, because you only need to find one axiom that fails.

The next video segment provides several excellent examples of sets with non-standard operations and shows how to find the specific axiom that breaks.

Conclusion

In this lesson, we took our first step from concrete examples of vectors to the powerful, abstract framework of vector spaces. This abstraction is what makes linear algebra a universal language in science and engineering.

Key Takeaways:

• A vector space is a set of objects (vectors) combined with a field of scalars and two operations (addition and scalar multiplication) that satisfy ten specific axioms.
• The purpose of the axioms is to create a general framework so that theorems and tools of linear algebra can apply to any object that fits the definition, whether it's an arrow, a matrix, a polynomial, or a function.
• We will primarily work with real vector spaces (like ${\mathbb{R}}^n$, crucial for deep learning) and complex vector spaces (like ${\mathbb{C}}^n$, the foundation of quantum mechanics).
• To verify if a given set and its operations form a vector space, you must confirm all ten axioms hold. To prove it is not a vector space, you only need to find one axiom that fails.

Preview of Next Lesson:

Now that we've defined what a vector space is, we'll explore vector spaces that live inside other vector spaces. In the next lesson, we will define subspaces and investigate the fundamental concept of the span of a set of vectors—the process of building up an entire space from just a few "building block" vectors.