Vector Decomposition: Parallel and Orthogonal Components

1. The Orthogonal Decomposition Theorem

The core idea is that any vector $\mathbf{b}$ can be seen as the sum of two distinct parts relative to a subspace $W$: a part that lies inside $W$ and a part that is orthogonal to $W$.

Let's begin with a formal statement of this powerful idea.

To put it in our own terms, for any vector $\mathbf{b}$ and any subspace $W$, we can write:
$\mathbf{b}={\mathbf{b}}_{||}+{\mathbf{b}}_{\perp }$
where:

• ${\mathbf{b}}_{||}$ is the component of $\mathbf{b}$ parallel to $W$. This is simply the orthogonal projection of $\mathbf{b}$ onto $W$, which we'll denote as ${\text{proj}}_W(\mathbf{b})$.
• ${\mathbf{b}}_{\perp }$ is the component of $\mathbf{b}$ orthogonal to $W$. This is the "leftover" part, calculated as $\mathbf{b}-{\mathbf{b}}_{||}$. This vector lies in the orthogonal complement, $W^{\perp }$.

This decomposition should feel intuitive. It's like resolving a force vector in physics into its horizontal and vertical components. Here, we're resolving a vector into components "in the subspace" and "perpendicular to the subspace".

2. A Deeper Look at the Decomposition

Let's watch a video that provides an excellent visual and conceptual walkthrough of the Orthogonal Decomposition Theorem. This will help solidify why this decomposition always works and is unique.

The video confirms our procedure:

1. We define the parallel component ${\mathbf{b}}_{||}$ as the projection onto $W$.
2. We define the orthogonal component ${\mathbf{b}}_{\perp }$ as the difference, $\mathbf{b}-{\mathbf{b}}_{||}$.
3. The crucial step is verifying that this ${\mathbf{b}}_{\perp }$ is, in fact, orthogonal to every vector in $W$. The video provides an intuitive proof of this for the case where we have an orthogonal basis.

3. The Computational Recipe

Now, let's turn this theory into a concrete, repeatable process. The method follows directly from our discussion.

To decompose a vector $\mathbf{b}$ into components parallel and orthogonal to a subspace $W$:

1. Find the parallel component, ${\mathbf{b}}_{||}$.
   This is the orthogonal projection ${\text{proj}}_W(\mathbf{b})$. You already know how to do this from our last lesson:

   • Find a basis for $W$. Let the matrix $A$ have these basis vectors as its columns.
   • Solve the normal equations $A^{T}A\hat{\mathbf{x}}=A^T\mathbf{b}$ for the coefficient vector $\hat{\mathbf{x}}$.
   • The parallel component is ${\mathbf{b}}_{||}=A\hat{\mathbf{x}}$.
   • (Shortcut): If your basis for $W$ is orthogonal, $\{{\mathbf{w}}_1,\dots ,{\mathbf{w}}_k\}$, the calculation is much simpler:
     ${\mathbf{b}}_{||}=\frac{\mathbf{b}\cdot {\mathbf{w}}_1}{{\mathbf{w}}_1\cdot {\mathbf{w}}_1}{\mathbf{w}}_1+\cdots +\frac{\mathbf{b}\cdot {\mathbf{w}}_k}{{\mathbf{w}}_k\cdot {\mathbf{w}}_k}{\mathbf{w}}_k$

2. Find the orthogonal component, ${\mathbf{b}}_{\perp }$.
   This is simply the vector that's left over:
   ${\mathbf{b}}_{\perp }=\mathbf{b}-{\mathbf{b}}_{||}$

3. Verify (Optional but Recommended).
   Confirm that your decomposition is correct by checking that ${\mathbf{b}}_{\perp }$ is truly orthogonal to $W$. You can do this by computing the dot product of ${\mathbf{b}}_{\perp }$ with each of the basis vectors of $W$. The result should be zero for all of them.

4. A Worked Example

The best way to understand this process is to do it. The following reading guides you through a concrete example.

Conclusion

Today, we formalized the concept of orthogonal decomposition, a cornerstone of linear algebra with profound implications.

Key Takeaways:

• Orthogonal Decomposition Theorem: Any vector $\mathbf{b}$ can be uniquely written as the sum of a vector ${\mathbf{b}}_{||}$ in a subspace $W$ and a vector ${\mathbf{b}}_{\perp }$ in its orthogonal complement $W^{\perp }$.
• The parallel component, ${\mathbf{b}}_{||}$, is the orthogonal projection of $\mathbf{b}$ onto $W$. It is the "best approximation" of $\mathbf{b}$ within $W$.
• The orthogonal component, ${\mathbf{b}}_{\perp }$, is the "error" or "remainder" vector, calculated as $\mathbf{b}-{\mathbf{b}}_{||}$.
• The decomposition process is straightforward: first compute the projection, then subtract to find the remainder.

Preview of the Next Lesson:

In our next lesson, we will see why this decomposition is so important. We will discuss orthogonal matrices and operators, which are transformations that preserve lengths and angles. These special operators are fundamental in quantum mechanics, where they represent the evolution of quantum states, and in computer graphics and robotics, where they represent pure rotations. We will explore how their matrix representations (orthogonal and unitary matrices) have incredibly useful properties.