Tensor Products and Their Dimensions

1. Combining Systems: More Than Just Stacking Vectors

Imagine you have two separate systems. System A could be a quantum coin with two states (Heads, Tails), described by a 2D vector space $V$. System B could be a quantum die with six states (1 to 6), described by a 6D vector space $W$. How do we describe the combined system, where we can observe both the coin and the die simultaneously?

A single outcome of the combined system is a pair of outcomes, like (Heads, 3) or (Tails, 5). We need a new, larger vector space that can account for all possible combinations of states.

You're already familiar with one way to combine vector spaces: the direct sum, $V\oplus W$. In a direct sum, we essentially stack the vectors. If $\mathbf{v}\in V$ and $\mathbf{w}\in W$, the new vector is $(\mathbf{v},\mathbf{w})\in V\oplus W$. The dimensions add up: $\dim (V\oplus W)=\dim (V)+\dim (W)$. For our coin and die example, this would be a $2+6=8$ dimensional space. This isn't quite right, because there are $2\times 6=12$ possible combined outcomes.

The tensor product provides the correct framework. It creates a new space, $V\otimes W$, where the dimensions multiply: $\dim (V\otimes W)=\dim (V)\times \dim (W)$.

To build some more intuition, let's start with a brief, accessible article that introduces the core idea.

This idea of combining systems by taking all possible pairings is the conceptual heart of the tensor product.

2. The Formal Construction: Sculpting the Right Space

Now, how do we formally define this new vector space, $V\otimes W$? It's a bit more abstract than just saying "multiply the components," because we need a definition that works for general vector spaces, not just ${\mathbb{R}}^n$.

The standard approach is a clever, multi-step process that you can think of as "sculpting":

1. Start with a massive, unrefined vector space that includes all possible "formal products" of vectors from $V$ and $W$.
2. "Chip away" at this space by enforcing the algebraic rules we want our product to have (like the distributive property). This is done formally by taking a quotient space.

The following video provides an excellent and clear explanation of this construction. It first builds the motivation for why this process is necessary.

Let's summarize the key steps from the video:

1. The Formal Product ($V\star W$): We start with a vector space whose basis is the set of all symbols $\mathbf{v}\star \mathbf{w}$ for every $\mathbf{v}\in V$ and $\mathbf{w}\in W$. This space is enormous and lacks the properties we'd expect from a "product." For example, $(2\mathbf{v})\star \mathbf{w}$ is a completely separate basis vector from $2(\mathbf{v}\star \mathbf{w})$.

2. The Subspace of Relations ($I$): We identify all the relationships we want to be true. These are the bilinearity properties:

   • $({\mathbf{v}}_1+{\mathbf{v}}_2)\otimes \mathbf{w}={\mathbf{v}}_1\otimes \mathbf{w}+{\mathbf{v}}_2\otimes \mathbf{w}$
   • $\mathbf{v}\otimes ({\mathbf{w}}_1+{\mathbf{w}}_2)=\mathbf{v}\otimes {\mathbf{w}}_1+\mathbf{v}\otimes {\mathbf{w}}_2$
   • $(c\mathbf{v})\otimes \mathbf{w}=\mathbf{v}\otimes (c\mathbf{w})=c(\mathbf{v}\otimes \mathbf{w})$ for any scalar $c$.
     We form a subspace, $I$, by taking the span of all vectors of the form (left side) - (right side) from these desired identities (e.g., $({\mathbf{v}}_1+{\mathbf{v}}_2)\star \mathbf{w}-({\mathbf{v}}_1\star \mathbf{w})-({\mathbf{v}}_2\star \mathbf{w})$).

3. The Quotient Space: The tensor product space is formally defined as the quotient space $V\otimes W:=(V\star W)/I$. This construction effectively sets every element in the subspace $I$ to zero, thereby enforcing the desired bilinearity relations in the final space. An element in this new space is written as $\mathbf{v}\otimes \mathbf{w}$.

This construction ensures that the tensor product $\otimes$ behaves like a proper, well-behaved multiplication operation between vectors.

3. Basis and Dimension of the Tensor Product Space

Now that we have defined the space $V\otimes W$, we can answer the second part of our learning goal: what is its dimension?

The key insight is that if we have a basis for $V$ and a basis for $W$, we can construct a basis for $V\otimes W$ by taking all possible tensor products of their basis vectors.

Let's watch a segment from another excellent video that clearly explains this.

As the video explains, if $\{{\mathbf{v}}_1,\dots ,{\mathbf{v}}_n\}$ is a basis for $V$ and $\{{\mathbf{w}}_1,\dots ,{\mathbf{w}}_m\}$ is a basis for $W$, then the set of all pairwise tensor products:
$\{{\mathbf{v}}_i\otimes {\mathbf{w}}_j\mid 1\le i\le n,1\le j\le m\}$
forms a basis for $V\otimes W$.

You can visualize this as creating a grid:

Since there are $n$ choices for the first vector and $m$ choices for the second, there are a total of $n\times m$ basis vectors in this new set. This gives us our fundamental result:

The dimension of the tensor product of two vector spaces is the product of their individual dimensions.
$\dim (V\otimes W)=\dim (V)\times \dim (W)$

Conclusion

In this lesson, we introduced the tensor product as a new way to combine vector spaces, a method that is essential for describing composite physical systems.

Key Takeaways:

• The tensor product $V\otimes W$ is a new vector space constructed from vector spaces $V$ and $W$. It models composite systems where the number of possible states is the product of the states in the individual systems.
• Formally, $V\otimes W$ is defined as a quotient space. This construction ensures that the tensor product operation $\otimes$ is bilinear, meaning it distributes over vector addition and behaves correctly with scalar multiplication.
• If $\{{\mathbf{v}}_i\}$ is a basis for $V$ and $\{{\mathbf{w}}_j\}$ is a basis for $W$, then the set of all products $\{{\mathbf{v}}_i\otimes {\mathbf{w}}_j\}$ forms a basis for $V\otimes W$.
• The dimension of the tensor product space is the product of the dimensions of the constituent spaces: $\dim (V\otimes W)=\dim (V)\times \dim (W)$.

Preview of the Next Lesson:

We've defined the space $V\otimes W$ and found its dimension. In the next lesson, we will get more concrete and learn how to compute the tensor product of two specific vectors. This will introduce the Kronecker product, which is the matrix representation of the tensor product for column vectors. We will then see how this extends to defining tensor products of linear operators, a crucial step for describing operations on multi-qubit systems in quantum computing.