Preserving Norms and Inner Products: Orthogonal and Unitary Operators

1. Orthogonal Matrices: Preserving Geometry in ${\mathbb{R}}^n$

Let's start with real vectors. The properties we're about to demonstrate are what make orthogonal matrices the mathematical representation of rotations and reflections.

Preservation of Norm (Length)

First, we'll show that applying an orthogonal matrix to a vector does not change its length.

As the video demonstrated, the proof is elegant and relies directly on the definition of an orthogonal matrix. Let's recap the steps:

Let $Q$ be an orthogonal matrix and $\mathbf{x}$ be a vector in ${\mathbb{R}}^n$. We want to show that $\Vert Q\mathbf{x}\Vert =\Vert \mathbf{x}\Vert$. It's easier to work with the square of the norm:

1. Start with the definition of the L2-norm:
   $\Vert Q\mathbf{x}{\Vert }^2=(Q\mathbf{x}{)}^T(Q\mathbf{x})$
2. Apply the transpose property $(AB{)}^T=B^T{A}^T$:
   $\Vert Q\mathbf{x}{\Vert }^2={\mathbf{x}}^T{Q}^{T}Q\mathbf{x}$
3. Use the defining property of an orthogonal matrix, $Q^{T}Q=I$:
   $\Vert Q\mathbf{x}{\Vert }^2={\mathbf{x}}^{T}I\mathbf{x}={\mathbf{x}}^T\mathbf{x}$
4. Recognize this is the squared norm of the original vector:
   $\Vert Q\mathbf{x}{\Vert }^2=\Vert \mathbf{x}{\Vert }^2$

Taking the square root of both sides gives us $\Vert Q\mathbf{x}\Vert =\Vert \mathbf{x}\Vert$. The length is preserved.

Preservation of Inner Product (and Angles)

Not only are lengths preserved, but so are the angles between vectors. This follows from the preservation of the inner product.

Let $\mathbf{x}$ and $\mathbf{y}$ be two vectors in ${\mathbb{R}}^n$. Their inner product (dot product) is ${\mathbf{x}}^T\mathbf{y}$. Let's see what happens to the inner product after transforming both vectors by $Q$:
$(Q\mathbf{x}{)}^T(Q\mathbf{y})={\mathbf{x}}^T{Q}^{T}Q\mathbf{y}={\mathbf{x}}^{T}I\mathbf{y}={\mathbf{x}}^T\mathbf{y}$
The inner product is unchanged!

Since the angle $\theta$ between two vectors is given by the formula:
$\cos \theta =\frac{\mathbf{x}\cdot \mathbf{y}}{\Vert \mathbf{x}\Vert \Vert \mathbf{y}\Vert }=\frac{{\mathbf{x}}^T\mathbf{y}}{\Vert \mathbf{x}\Vert \Vert \mathbf{y}\Vert }$
and we've just shown that both the numerator (inner product) and the terms in the denominator (norms) are preserved under transformation by $Q$, the angle $\theta$ must also be preserved.

This is why orthogonal transformations are essentially rotations and reflections.

2. Unitary Matrices: Preserving Geometry in ${\mathbb{C}}^n$

The exact same logic extends to complex vector spaces and unitary matrices. The only difference is that we use the conjugate transpose ($†$) instead of the regular transpose ($T$), and the complex inner product instead of the dot product.

A unitary transformation preserves the complex inner product $⟨\mathbf{z},\mathbf{w}⟩={\mathbf{z}}^{†}\mathbf{w}$. Let $U$ be a unitary matrix and $\mathbf{z},\mathbf{w}$ be complex vectors. The inner product of the transformed vectors is:
$⟨U\mathbf{z},U\mathbf{w}⟩=(U\mathbf{z}{)}^{†}(U\mathbf{w})$
Using the property $(AB{)}^{†}=B^{†}{A}^{†}$, this becomes:
${\mathbf{z}}^{†}{U}^{†}U\mathbf{w}$
And since $U$ is unitary, $U^{†}U=I$. So we have:
${\mathbf{z}}^{†}I\mathbf{w}={\mathbf{z}}^{†}\mathbf{w}=⟨\mathbf{z},\mathbf{w}⟩$
The complex inner product is preserved.

Since the norm is just a special case of the inner product ($\Vert \mathbf{z}{\Vert }^2=⟨\mathbf{z},\mathbf{z}⟩$), the preservation of the inner product automatically implies the preservation of the norm:
$\Vert U\mathbf{z}{\Vert }^2=⟨U\mathbf{z},U\mathbf{z}⟩=⟨\mathbf{z},\mathbf{z}⟩=\Vert \mathbf{z}{\Vert }^2$

This property is the bedrock of quantum mechanics.

3. Connection to Your Goals

Understanding why these operators preserve structure is key to seeing their role in your future studies.

Application in Deep Learning and Data Science

In many data analysis tasks, such as Principal Component Analysis (PCA) which we will cover later, we want to rotate our dataset to find the directions of maximum variance. These rotations must not stretch or skew the data, as that would distort the relationships between data points. Orthogonal matrices are used to perform these rotations precisely because they preserve all distances and angles. In computer graphics, rotating an object on screen is achieved by applying an orthogonal matrix to the coordinates of all its vertices.

Application in Quantum Mechanics

This is where unitary matrices are absolutely non-negotiable.

In quantum mechanics, the state of a system is described by a state vector $\psi$ in a complex Hilbert space. A fundamental postulate is that the squared norm of this vector, $\Vert \psi {\Vert }^2=⟨\psi ,\psi ⟩$, must equal 1. This represents the fact that the total probability of finding the particle in any possible state is 100%.

When the system evolves over time, its state vector changes. If the state at time $t_1$ is ${\psi }_1$, the state at a later time $t_2$ will be ${\psi }_2=U{\psi }_1$, where $U$ is some linear operator that describes the evolution. For physics to be consistent, the total probability must still be 1. This means we must have $\Vert {\psi }_2{\Vert }^2=1$. This forces the transformation $U$ to be norm-preserving, and therefore, all time-evolution operators in quantum mechanics must be unitary.

Conclusion

Today we connected the algebraic definitions from our last lesson to their profound geometric meaning.

Key Takeaways:

• Orthogonal matrices ($Q^{T}Q=I$) and unitary matrices ($U^{†}U=I$) are transformations that preserve the geometry of their respective vector spaces.
• They preserve the norm (length) of vectors: $\Vert Q\mathbf{x}\Vert =\Vert \mathbf{x}\Vert$ and $\Vert U\mathbf{z}\Vert =\Vert \mathbf{z}\Vert$.
• They preserve the inner product (and thus angles) between vectors: $(Q\mathbf{x}{)}^T(Q\mathbf{y})={\mathbf{x}}^T\mathbf{y}$ and $⟨U\mathbf{z},U\mathbf{w}⟩=⟨\mathbf{z},\mathbf{w}⟩$.
• This property makes them ideal for representing rotations and reflections in real space (applications in graphics, data science) and is a physical requirement for describing time evolution in quantum mechanics to ensure probability is conserved.

Preview of the Next Module:

We've now completed our deep dive into orthogonality. We've built orthonormal bases, used them for projection, and defined operators that preserve their structure. In the next module, we will begin our study of Eigenvalues and Eigenvectors. We will shift our focus from how a transformation affects all vectors to searching for very special vectors that are left in the same direction by the transformation—they are only scaled. These "invariant directions" reveal the deepest structure of a linear transformation and are fundamental to nearly every application of linear algebra, from diagonalization to solving differential equations to the core principles of quantum measurement.