Linear Transformations and Matrix Representation

Hello! Welcome to the fifth lesson in our foundational module on linear algebra.

In our last session, we formally defined matrix operations. We concluded with the powerful insight that matrix multiplication is not just an arbitrary computational rule, but rather the algebraic representation of composing successive linear transformations. This was a deliberate preview, as it perfectly sets the stage for today's topic.

Today, we will make this connection explicit. Our learning outcome is to define linear transformations and represent them using matrices. We will formally define what makes a function "linear" and then prove the fundamental result that any linear transformation between finite-dimensional vector spaces can be uniquely represented by a matrix. This bridges the gap between the abstract geometric idea of a transformation (like a rotation or shear) and the concrete computational object of a matrix.

1. What is a Linear Transformation?

A linear transformation is a special kind of function between two vector spaces that preserves the underlying vector space structure. This means it respects the two fundamental operations: vector addition and scalar multiplication.

Formally, let $V$ and $W$ be two vector spaces. A function $T:V\to W$ is called a linear transformation (or linear map) if for any two vectors $\mathbf{u},\mathbf{v}\in V$ and any scalar $c$, the following two conditions hold:

1. Additivity: $T(\mathbf{u}+\mathbf{v})=T(\mathbf{u})+T(\mathbf{v})$
2. Homogeneity: $T(c\mathbf{u})=cT(\mathbf{u})$

These two properties can be combined into a single statement: $T(c\mathbf{u}+d\mathbf{v})=cT(\mathbf{u})+dT(\mathbf{v})$ for any vectors $\mathbf{u},\mathbf{v}$ and scalars $c,d$.

Geometrically, these properties ensure that grid lines in the input space remain parallel and evenly spaced after the transformation, and the origin $0_V$ maps to the origin $0_W$. This is because $T(0_V)=T(0\cdot \mathbf{v})=0\cdot T(\mathbf{v})=0_W$. Transformations like rotations, reflections, scaling, shears, and projections are all linear. A simple translation, $T(\mathbf{v})=\mathbf{v}+\mathbf{b}$ where $\mathbf{b}\ne 0$, is not linear because it does not map the origin to the origin.

2. From Linear Maps to Matrices: The Core Idea

The crucial link between a linear map and a matrix lies in a simple but profound observation: a linear transformation is entirely determined by its effect on the basis vectors of the domain.

To understand this foundational concept, we'll watch a segment from a video by Mathoma.

This insight is the key to constructing a matrix representation. If we know where the basis vectors of our domain $V$ land in the codomain $W$, we can use the coordinates of these transformed vectors to build a matrix.

3. Constructing the Matrix Representation

Let's formalize the construction. Suppose we have a linear transformation $T:V\to W$. Let ${\mathcal{B}}_V=\{{\mathbf{v}}_1,\dots ,{\mathbf{v}}_n\}$ be a basis for $V$ (an n-dimensional space) and ${\mathcal{B}}_W=\{{\mathbf{w}}_1,\dots ,{\mathbf{w}}_m\}$ be a basis for $W$ (an m-dimensional space).

1. Transform each basis vector of V: Apply $T$ to each basis vector ${\mathbf{v}}_j$ from ${\mathcal{B}}_V$ to get a set of vectors $\{T({\mathbf{v}}_1),T({\mathbf{v}}_2),\dots ,T({\mathbf{v}}_n)\}$ in $W$.

2. Find the coordinates in W: For each transformed vector $T({\mathbf{v}}_j)$, find its unique coordinate representation in the basis ${\mathcal{B}}_W$. This means finding the scalars $a_{ij}$ such that:

   $$T({\mathbf{v}}_j)=a_{1j}{\mathbf{w}}_1+a_{2j}{\mathbf{w}}_2+\cdots +a_{mj}{\mathbf{w}}_m=\sum _{i=1}^m{a}_{ij}{\mathbf{w}}_i$$

   The coordinate vector of $T({\mathbf{v}}_j)$ with respect to the basis ${\mathcal{B}}_W$ is therefore the column vector:

   $$[T({\mathbf{v}}_j){]}_{{\mathcal{B}}_W}=\left(\begin{array}{c}{a}_{1j}\\ a_{2j}\\ ⋮\\ a_{mj}\end{array}\right)$$

3. Form the matrix: The matrix of the linear transformation $T$ with respect to the bases ${\mathcal{B}}_V$ and ${\mathcal{B}}_W$, denoted $[T{]}_{{\mathcal{B}}_V,{\mathcal{B}}_W}$, is the $m\times n$ matrix whose $j$-th column is the coordinate vector $[T({\mathbf{v}}_j){]}_{{\mathcal{B}}_W}$.

   $$[T{]}_{{\mathcal{B}}_V,{\mathcal{B}}_W}=\left(\begin{array}{cccc}|& |& & |\\ [T({\mathbf{v}}_1){]}_{{\mathcal{B}}_W}& [T({\mathbf{v}}_2){]}_{{\mathcal{B}}_W}& \dots & [T({\mathbf{v}}_n){]}_{{\mathcal{B}}_W}\\ |& |& & |\end{array}\right)$$

This matrix has a remarkable property: applying the transformation $T$ to a vector $\mathbf{v}$ in the abstract vector space is equivalent to multiplying the matrix $[T{]}_{{\mathcal{B}}_V,{\mathcal{B}}_W}$ by the coordinate vector of $\mathbf{v}$.

$$[T(\mathbf{v}){]}_{{\mathcal{B}}_W}=[T{]}_{{\mathcal{B}}_V,{\mathcal{B}}_W}[\mathbf{v}{]}_{{\mathcal{B}}_V}$$

This relationship is not an accident; it's the very reason matrix multiplication is defined the way it is. The following resource provides a rigorous proof of this equivalence, which is central to your learning goals.

The article also proves that for a given linear map and chosen bases, this matrix representation is unique.

The Mathoma video you watched earlier also contains a full derivation of this matrix-vector product rule from first principles. Watching it now will reinforce the algebraic steps from the article with a clear, narrated explanation.

Test your understanding!

Let $T:{\mathbb{R}}^2\to {\mathbb{R}}^3$ be a linear transformation defined by $T(x_1,x_2)=(x_1+2x_2,-x_1,3x_2)$.

Find the matrix representation of $T$ with respect to the standard bases for ${\mathbb{R}}^2$ and ${\mathbb{R}}^3$. The standard basis for ${\mathbb{R}}^n$ is $\{{\mathbf{e}}_1,\dots ,{\mathbf{e}}_n\}$, where ${\mathbf{e}}_j$ is the vector with a 1 in the $j$-th position and 0s elsewhere.

Show answer

To find the matrix representation, we need to find what the transformation $T$ does to the basis vectors of the domain, ${\mathbb{R}}^2$, and express the result in the coordinates of the codomain's basis, ${\mathbb{R}}^3$.

The standard basis for the domain ${\mathbb{R}}^2$ is ${\mathcal{B}}_V=\{{\mathbf{e}}_1,{\mathbf{e}}_2\}$, where ${\mathbf{e}}_1=\left(\begin{array}{c}1\\ 0\end{array}\right)$ and ${\mathbf{e}}_2=\left(\begin{array}{c}0\\ 1\end{array}\right)$.

1. Transform the first basis vector:

   $$T({\mathbf{e}}_1)=T(1,0)=(1+2(0),-1,3(0))=(1,-1,0)$$

   The coordinate vector of the result in the standard basis of ${\mathbb{R}}^3$ is simply $\left(\begin{array}{c}1\\ -1\\ 0\end{array}\right)$. This will be the first column of our matrix.

2. Transform the second basis vector:

   $$T({\mathbf{e}}_2)=T(0,1)=(0+2(1),-0,3(1))=(2,0,3)$$

   The coordinate vector of the result in the standard basis of ${\mathbb{R}}^3$ is $\left(\begin{array}{c}2\\ 0\\ 3\end{array}\right)$. This will be the second column of our matrix.

3. Assemble the matrix:
   The matrix of the transformation, $A=[T{]}_{{\mathcal{B}}_V,{\mathcal{B}}_W}$, is formed by these column vectors:

   $$A=\left(\begin{array}{cc}1& 2\\ -1& 0\\ 0& 3\end{array}\right)$$

This is the $3\times 2$ matrix that represents the linear transformation $T$.

4. Basis Dependence and Isomorphism

An essential takeaway is that the matrix representation of a linear map is basis-dependent. The same linear transformation (e.g., a 90-degree rotation) will be represented by different matrices if you choose different bases.

This basis-dependence highlights a deeper concept: isomorphism. An isomorphism is an invertible linear map between two vector spaces. If such a map exists, the two spaces are called isomorphic, meaning they are structurally identical from a linear algebra perspective—they are just different "labelings" of the same underlying structure.

A key theorem states that two finite-dimensional vector spaces are isomorphic if and only if they have the same dimension. This is why any n-dimensional real vector space is isomorphic to ${\mathbb{R}}^n$. The map that takes a vector $\mathbf{v}\in V$ to its coordinate vector $[\mathbf{v}{]}_{\mathcal{B}}\in {\mathbb{R}}^n$ is an isomorphism. The study of linear transformations can thus be mapped entirely to the study of matrices acting on coordinate vectors in ${\mathbb{R}}^n$, which is a cornerstone of computational linear algebra.

Conclusion

In this lesson, we established the fundamental link between abstract linear transformations and concrete matrices.

Key Takeaways:

• A linear transformation is a function between vector spaces that preserves vector addition and scalar multiplication.
• Any linear map between finite-dimensional vector spaces has a unique matrix representation with respect to a chosen pair of bases (one for the domain, one for the codomain).
• The columns of this matrix are the coordinate vectors of the transformed basis vectors of the domain.
• Applying a linear transformation to a vector is equivalent to multiplying its matrix representation by the vector's coordinate vector.
• The matrix representation is dependent on the choice of bases.

This understanding is crucial. When you use a library like Scikit-learn or LangChain to process data, the data points are represented as vectors, and the algorithms you apply are often sequences of linear transformations represented by matrices.

In our next lesson, we will apply these concepts to solve systems of linear equations using matrix methods. We can now interpret a system $A\mathbf{x}=\mathbf{b}$ not just as a set of equations, but geometrically: we are seeking the input vector $\mathbf{x}$ which, after being transformed by the linear map represented by $A$, results in the output vector $\mathbf{b}$.