Change-of-Basis Matrices

1. The Core Idea: One Vector, Many Descriptions

Imagine a single point on a map. You could describe its location using latitude and longitude, or you could describe it using its distance and bearing from a specific landmark. The point itself doesn't move, but its coordinates change depending on the reference system you choose. The same is true for vectors. A basis is just a reference system for a vector space.

To build our intuition, let's start with a classic visualization.

As the video shows, a vector we describe as $[3,2{]}^T$ using the standard basis $\mathcal{S}=\{\mathbf{i},\mathbf{j}\}$ is a result of the sum $3\mathbf{i}+2\mathbf{j}$. In Jennifer's basis $\mathcal{B}=\{{\mathbf{b}}_1,{\mathbf{b}}_2\}$, that same vector is described by different coordinates, because it is constructed by a different linear combination, $\frac{5}{3}{\mathbf{b}}_1+\frac{1}{3}{\mathbf{b}}_2$.

Our task is to find a systematic way to translate between these coordinate representations.

2. The Change-of-Basis Matrix

Let's establish some notation. Let $\mathcal{S}$ be the standard basis of ${\mathbb{R}}^n$ (the columns of the identity matrix $I$). Let $\mathcal{B}=\{{\mathbf{b}}_1,{\mathbf{b}}_2,...,{\mathbf{b}}_n\}$ be a new basis. The vectors ${\mathbf{b}}_i$ themselves are described using standard coordinates.

A vector $\mathbf{v}$ written in standard coordinates is just denoted $\mathbf{v}$. The same vector written in the coordinates of basis $\mathcal{B}$ is denoted $[\mathbf{v}{]}_{\mathcal{B}}$.

2.1 Translating from a New Basis $\mathcal{B}$ to the Standard Basis $\mathcal{S}$

This is the more straightforward direction. Suppose we have a vector's coordinates in Jennifer's basis, say $[\mathbf{v}{]}_{\mathcal{B}}=\left(\begin{array}{c}{c}_1\\ c_2\end{array}\right)$. This means by definition:

You might recognize this pattern. This is precisely the definition of matrix-vector multiplication!

So, the matrix whose columns are the basis vectors of $\mathcal{B}$ (written in standard coordinates) is our translator. Let's call this matrix $P_{\mathcal{S}\leftarrow \mathcal{B}}$. The subscript reads "from $\mathcal{B}$ to $\mathcal{S}$".

Rule 1: Translating to the Standard Basis
The matrix $P_{\mathcal{S}\leftarrow \mathcal{B}}$ that converts coordinates from basis $\mathcal{B}$ to the standard basis $\mathcal{S}$ is formed by taking the basis vectors of $\mathcal{B}$ as its columns.

where $P_{\mathcal{S}\leftarrow \mathcal{B}}=\left(\begin{array}{ccc}|& \dots & |\\ {\mathbf{b}}_1& \dots & {\mathbf{b}}_n\\ |& \dots & |\end{array}\right)$.

This matrix is often just called the change-of-basis matrix from $\mathcal{B}$.

2.2 Translating from the Standard Basis $\mathcal{S}$ to a New Basis $\mathcal{B}$

Now for the other direction. We have a vector $\mathbf{v}$ in standard coordinates, and we want to find its coordinates $[\mathbf{v}{]}_{\mathcal{B}}$ in the new basis. We can just use the equation we found above:

To solve for $[\mathbf{v}{]}_{\mathcal{B}}$, we need to "undo" the matrix multiplication. This is where the last lesson comes in. Since the columns of $P_{\mathcal{S}\leftarrow \mathcal{B}}$ form a basis, they are linearly independent. This means the matrix is invertible!

We can simply multiply both sides by the inverse:

Rule 2: Translating from the Standard Basis
The matrix $P_{\mathcal{B}\leftarrow \mathcal{S}}$ that converts coordinates from the standard basis $\mathcal{S}$ to a new basis $\mathcal{B}$ is the inverse of the matrix from Rule 1.

Let's see this in action.

3. The General Case: From Basis $\mathcal{B}$ to Basis $\mathcal{C}$

We've handled translations to and from the standard basis. What if we want to translate directly between two non-standard bases, say from $\mathcal{B}=\{{\mathbf{b}}_1,\dots ,{\mathbf{b}}_n\}$ to $\mathcal{C}=\{{\mathbf{c}}_1,\dots ,{\mathbf{c}}_n\}$?

We don't need a new trick. We can just use the standard basis as an intermediate step, like a common layover airport.

1. Translate from basis $\mathcal{B}$ to the standard basis $\mathcal{S}$.
2. Translate from the standard basis $\mathcal{S}$ to the final basis $\mathcal{C}$.

Let's say we have $[\mathbf{v}{]}_{\mathcal{B}}$.

1. First, find the standard coordinates: $\mathbf{v}=P_{\mathcal{S}\leftarrow \mathcal{B}}[\mathbf{v}{]}_{\mathcal{B}}$.
2. Now, convert $\mathbf{v}$ to the $\mathcal{C}$ basis: $[\mathbf{v}{]}_{\mathcal{C}}=P_{\mathcal{C}\leftarrow \mathcal{S}}\mathbf{v}$.

Substituting the first equation into the second gives:

This gives us our general change-of-basis matrix, $P_{\mathcal{C}\leftarrow \mathcal{B}}$.

General Change-of-Basis Formula
Let $\mathcal{B}$ and $\mathcal{C}$ be two bases. Let $B$ be the matrix whose columns are the basis vectors of $\mathcal{B}$ and $C$ be the matrix whose columns are the basis vectors of $\mathcal{C}$. The matrix that converts coordinates from $\mathcal{B}$ to $\mathcal{C}$ is:

The notation can be a helpful guide. The matrix to go from $\mathcal{B}$ to $\mathcal{C}$ is $P_{\mathcal{C}\leftarrow \mathcal{B}}$. The "inner" bases $\mathcal{S}$ in the product $P_{\mathcal{C}\leftarrow \mathcal{S}}{P}_{\mathcal{S}\leftarrow \mathcal{B}}$ seem to "cancel out".

Conclusion

Today, we've untangled the process of changing coordinate systems. While the notation can seem dense at first, the underlying idea is straightforward and builds directly on concepts like matrix multiplication and inverses.

Key Takeaways:

• A vector is a fixed entity; its coordinates are just a description relative to a chosen basis.
• The matrix $P_{\mathcal{S}\leftarrow \mathcal{B}}$ that changes coordinates from a basis $\mathcal{B}$ to the standard basis $\mathcal{S}$ has the vectors of $\mathcal{B}$ as its columns.
• The matrix that changes coordinates from the standard basis $\mathcal{S}$ to a basis $\mathcal{B}$ is the inverse, $(P_{\mathcal{S}\leftarrow \mathcal{B}}{)}^{-1}$.
• To change from basis $\mathcal{B}$ to basis $\mathcal{C}$, you can use the standard basis as an intermediary: $P_{\mathcal{C}\leftarrow \mathcal{B}}=C^{-1}B$, where $B$ and $C$ are the matrices containing the basis vectors of $\mathcal{B}$ and $\mathcal{C}$.

Preview of the Next Lesson:
Today, we learned how to change the coordinates of a vector. But what happens to a linear transformation when we change the basis? If a matrix $A$ represents a rotation in the standard basis, what matrix represents that same rotation in Jennifer's basis? In our next lesson, we will see how to transform the matrix of a linear operator from one basis to another, a process that is crucial for understanding concepts like diagonalization in both Deep Learning and Quantum Mechanics.