Matrices as Linear Transformations

1. From Matrix Multiplication to Linear Transformations

At its heart, the operation of multiplying a matrix $A$ by a vector $\mathbf{x}$ to get a new vector $\mathbf{y}=A\mathbf{x}$ is a function that maps an input vector to an output vector. This mapping is called a transformation.

Matrices don't just perform any random transformation; they perform a special kind called a linear transformation. For a transformation $T$ to be linear, it must satisfy two properties for any vectors $\mathbf{x}$, $\mathbf{y}$ and any scalar $\alpha$:

1. Additivity: $T(\mathbf{x}+\mathbf{y})=T(\mathbf{x})+T(\mathbf{y})$
2. Homogeneity: $T(\alpha \mathbf{x})=\alpha T(\mathbf{x})$

Geometrically, these properties mean that the transformation keeps grid lines parallel and evenly spaced, and the origin remains fixed. The reading below formalizes this definition.

2. The Key Insight: Transformations are Defined by Basis Vectors

Understanding the two properties is good, but there is a more intuitive and powerful way to think about linear transformations.

Consider any vector in a 2D plane, for instance $\mathbf{v}=\left[\begin{array}{c}3\\ 2\end{array}\right]$. This vector can be expressed as a linear combination of the standard basis vectors, $\hat{ı}=\left[\begin{array}{c}1\\ 0\end{array}\right]$ and $\hat{ȷ}=\left[\begin{array}{c}0\\ 1\end{array}\right]$:

$\mathbf{v}=3\left[\begin{array}{c}1\\ 0\end{array}\right]+2\left[\begin{array}{c}0\\ 1\end{array}\right]=3\hat{ı}+2\hat{ȷ}$

Now, let's apply a linear transformation $T$ to $\mathbf{v}$. Because of the properties of linearity, we can write:

$T(\mathbf{v})=T(3\hat{ı}+2\hat{ȷ})=3T(\hat{ı})+2T(\hat{ȷ})$

This is a profound result. It means that if we know where the basis vectors $\hat{ı}$ and $\hat{ȷ}$ land after the transformation (i.e., we know $T(\hat{ı})$ and $T(\hat{ȷ})$), we can determine where any vector in the entire space lands.

The transformation is completely described by what it does to the basis vectors. So, where do we find these transformed basis vectors?

The columns of a transformation matrix are the transformed basis vectors.

Let a 2x2 matrix be $A=\left[\begin{array}{cc}a& b\\ c& d\end{array}\right]$.

• The first column, $\left[\begin{array}{c}a\\ c\end{array}\right]$, is the vector $T(\hat{ı})$ — where the first basis vector lands.
• The second column, $\left[\begin{array}{c}b\\ d\end{array}\right]$, is the vector $T(\hat{ȷ})$ — where the second basis vector lands.

This gives you the power to look at any 2x2 matrix and immediately know how it transforms 2D space.

3. A Catalogue of Common Transformations

With this core intuition, let's explore the matrix representations for several common geometric operations. The following reading provides a great overview, including the matrix forms and NumPy code. As you go through it, try to apply the "transformed basis vectors" logic to each matrix to see why it works.

Let's summarize the key transformations you just read about:

• Scaling: Stretches or shrinks space. A scaling matrix is a diagonal matrix, like $\left[\begin{array}{cc}{s}_x& 0\\ 0& s_y\end{array}\right]$. The first column shows $\hat{ı}$ being scaled by $s_x$, and the second shows $\hat{ȷ}$ being scaled by $s_y$.
• Shear: Slants the space. A horizontal shear matrix like $\left[\begin{array}{cc}1& m\\ 0& 1\end{array}\right]$ leaves $\hat{ı}$ unchanged but transforms $\hat{ȷ}$ into $\left[\begin{array}{c}m\\ 1\end{array}\right]$, effectively "pushing" the top of the grid.
• Rotation: Rotates space around the origin. We derived the 90-degree rotation matrix earlier. The general form is $\left[\begin{array}{cc}\cos \theta & -\sin \theta \\ \sin \theta & \cos \theta \end{array}\right]$.
• Reflection: Flips space across a line. For example, reflection across the y-axis is represented by $\left[\begin{array}{cc}-1& 0\\ 0& 1\end{array}\right]$, which reverses $\hat{ı}$ but leaves $\hat{ȷ}$ alone.

4. Constructing Your Own Transformation Matrices

Now, let's put your new knowledge into practice. The following exercises ask you to construct a matrix to achieve a specific transformation. This is a powerful way to test your understanding.

The resource below is part of a course from Neuromatch Academy. The course heavily uses videos, which I know you prefer to avoid. However, its text-based explanations and coding exercises are excellent. We will use only the coding exercise from this resource.

These exercises should highlight the connection between a geometric goal (e.g., "project onto the x-axis") and the algebraic representation (the matrix). The projection matrix, $\left[\begin{array}{cc}1& 0\\ 0& 0\end{array}\right]$, is particularly interesting. It takes the 2D plane and collapses it onto the x-axis. What does this remind you of from our last lesson? Its determinant is $1\ast 0-0\ast 0=0$. This is a singular matrix, which perfectly matches our prior understanding: it collapses space, so it's not invertible.

The set of all possible outputs of a transformation is its range, and the dimension of that range is its rank. For the projection matrix, the range is the x-axis, which is a 1-dimensional space, so its rank is 1.

Conclusion

Today you've unlocked a new way to see matrices. They are not just tables of numbers for rote calculation; they are dynamic operators that stretch, squeeze, rotate, and shear vector spaces. This geometric viewpoint is invaluable in machine learning, where complex models like neural networks are essentially long chains of linear and non-linear transformations designed to reshape data into a more useful form.

Here are the main takeaways:

• Matrix-vector multiplication, $A\mathbf{x}$, is a linear transformation that maps a vector $\mathbf{x}$ to a new vector.
• Linear transformations keep the origin fixed and grid lines straight, parallel, and evenly spaced.
• The key insight: the columns of a matrix tell you where the basis vectors land after the transformation. This allows you to construct and interpret any linear transformation.
• Common geometric operations like scaling, shear, rotation, and reflection can all be represented by specific 2x2 matrices.

Preview of the Next Lesson:
In this lesson, we saw how matrices can transform vectors, changing their direction and magnitude. But what if a matrix transforms a vector without changing its direction, only its length? These special vectors, which are only scaled by the transformation, are called eigenvectors, and their corresponding scaling factors are eigenvalues. In the next lesson, we will define eigenvalues and eigenvectors and explore their profound geometric significance. They are foundational to dimensionality reduction techniques like Principal Component Analysis (PCA), which is widely used in computer vision.