Orthogonality in Inner Product Spaces

Hello! Welcome back to our journey into inner product spaces.

In our last lesson, we saw how the algebraic structure of an inner product gives rise to geometry. By defining the norm as $∥ v ∥ = ⟨ v, v ⟩$ and distance as $d (u, v) = ∥ u - v ∥$ , we were able to measure lengths and separations in any inner product space.

Today, we'll explore the third fundamental geometric concept derived from the inner product: the notion of an angle, which leads to orthogonality. We will define what it means for two vectors to be "perpendicular" in any inner product space, whether it's the familiar real space $R^{n}$ or the complex spaces $C^{n}$ essential for quantum mechanics. This concept is one of the most powerful tools in linear algebra, simplifying problems in fields from data analysis to quantum computing.

1. Orthogonality in Real Inner Product Spaces

Your intuition for perpendicular vectors in 2D or 3D is the perfect starting point. You may recall the geometric formula for the dot product:
$u \cdot v = ∥ u ∥∥ v ∥ cos θ$
where $θ$ is the angle between the vectors. If two non-zero vectors are perpendicular, $θ = 9 0^{\circ}$ , which means $cos θ = 0$ . This forces their dot product to be zero.

Orthogonal Vectors in 2D — A visual representation of two orthogonal vectors in $\mathbb{R}^2$. Their dot product is $(-2)(3) + (3)(2) = -6 + 6 = 0$.

We take this observation and turn it into our formal definition. We define two vectors $u$ and $v$ in a real inner product space to be orthogonal if their inner product is zero.
$⟨ u, v ⟩ = 0$

To solidify this idea, please watch the following short video segment.

Linear Algebra 9 | Inner Product and Norm

This video from The Bright Side of Mathematics clearly defines the standard inner product in \mathbb{R}^n and then immediately defines orthogonality based on this product.

Watch from 03:26 to 04:28. The key takeaway is the formal definition of orthogonality in \mathbb{R}^n.

Example in $R^{4}$

Let's check if the vectors $u = (1, - 1, 2, 3)$ and $v = (3, 1, - 1, 0)$ in $R^{4}$ are orthogonal using the standard inner product (the dot product).

$⟨ u, v ⟩ = (1) (3) + (- 1) (1) + (2) (- 1) + (3) (0) = 3 - 1 - 2 + 0 = 0$
Since their inner product is zero, the vectors are orthogonal.

Example in a Function Space

This definition extends beyond columns of numbers. Let's use the inner product $⟨ f, g ⟩ = \int_{- π}^{π} f (x) g (x) d x$ on the space of continuous functions $C [- π, π]$ . Are the functions $f (x) = sin (x)$ and $g (x) = cos (x)$ orthogonal?

⟨ sin (x), cos (x)⟩ = \int_{- π}^{π} sin (x) cos (x) d x

Using the trigonometric identity $sin (x) cos (x) = \frac{1}{2} sin (2 x)$ , this becomes:

\frac{1}{2} \int_{- π}^{π} sin (2 x) d x = \frac{1}{2} [- \frac{cos ( 2 x )}{2}]_{- π}^{π} = - \frac{1}{4} [cos (2 π) - cos (- 2 π)] = - \frac{1}{4} [1 - 1] = 0

Indeed, $sin (x)$ and $cos (x)$ are orthogonal on this interval. This is a fundamental concept in Fourier analysis, which is used extensively in physics and engineering.

2. Orthogonality in Complex Inner Product Spaces

The definition of orthogonality remains the same for complex vector spaces, but the calculation uses the Hermitian inner product. This is critically important for quantum mechanics.

Two vectors $∣ ψ ⟩$ and $∣ ϕ ⟩$ in a complex inner product space are orthogonal if their inner product is zero:
$⟨ ψ ∣ ϕ ⟩ = 0$
Note that if $⟨ ψ ∣ ϕ ⟩ = 0$ , then $⟨ ϕ ∣ ψ ⟩ = ⟨ ψ ∣ ϕ ⟩ = 0 = 0$ . The order of the vectors does not affect whether they are orthogonal, even though the inner product itself is conjugate symmetric.

The following reading from IBM's Quantum Learning platform explains this concept clearly using the Dirac notation that is standard in quantum mechanics.

Inner products and projections

This resource defines orthogonality and related concepts in the context of quantum information, making it directly relevant to your goals.

Please read the section titled "Orthogonal and orthonormal sets". Focus on the definition of two vectors being orthogonal and what an orthogonal set is.

Example in $C^{2}$

Let's test if the vectors $∣ u ⟩ = (2 - i)$ and $∣ v ⟩ = (1 2 i)$ are orthogonal in $C^{2}$ .

The bra vector $⟨ u ∣$ is the conjugate transpose of $∣ u ⟩$ , so $⟨ u ∣ = (2 i)$ .

Now we compute the inner product $⟨ u ∣ v ⟩ = u^{†} v$ :

⟨ u ∣ v ⟩ = (2 ˉ) (1) + (- i) (2 i) = (2) (1) + (i) (2 i) = 2 + 2 i^{2} = 2 - 2 = 0

They are orthogonal.

Test your understanding!

In quantum computing, the standard basis states for a single qubit are $∣0 ⟩ = (10)$ and $∣1 ⟩ = (01)$ . Another important basis is the Hadamard basis, with states $∣ + ⟩ = \frac{1}{2} (11)$ and $∣ - ⟩ = \frac{1}{2} (1 - 1)$ .

Are the Hadamard basis states $∣ + ⟩$ and $∣ - ⟩$ orthogonal?

Show answer

We need to compute the inner product $⟨ + ∣ - ⟩$ . The factors of $\frac{1}{2}$ can be pulled out.

⟨ + ∣ - ⟩ = (\frac{1}{2} (11))^{†} (\frac{1}{2} (1 - 1)) = \frac{1}{2} (11) (1 - 1)

= \frac{1}{2} ((1) (1) + (1) (- 1)) = \frac{1}{2} (1 - 1) = 0

Yes, the states $∣ + ⟩$ and $∣ - ⟩$ are orthogonal. This orthogonality is what allows them to form a valid basis for measurement in quantum computation.

3. Orthonormal Sets

We can extend the idea of orthogonality from a pair of vectors to a set of vectors.

An orthogonal set is a set of non-zero vectors where every vector is orthogonal to every other vector in the set.
A unit vector is a vector whose norm is 1 (i.e., $∥ v ∥ = 1$ ).
An orthonormal set is an orthogonal set where every vector is a unit vector.

For a set of vectors ${∣ v_{1} ⟩, ∣ v_{2} ⟩, \dots}$ to be orthonormal, it must satisfy:
$⟨ v_{i} ∣ v_{j} ⟩ = {10 if i = j if i \neq = j$
This is often written compactly using the Kronecker delta, $δ_{ij}$ , as $⟨ v_{i} ∣ v_{j} ⟩ = δ_{ij}$ .

The following video provides an excellent, quantum-focused explanation of this concept.

Ch 4: What is an inner product? | Maths of Quantum Mechanics

This video from Quantum Sense directly defines orthogonality and introduces the concept of an orthonormal basis and the Kronecker delta, all of which are central to the language of quantum mechanics.

Watch from 05:50 to 06:49. Pay close attention to how orthogonality is defined and how the two conditions for an orthonormal set (orthogonality and unit length) are combined into a single statement using the Kronecker delta.

The standard bases for $R^{n}$ and $C^{n}$ (e.g., $∣0 ⟩, ∣1 ⟩$ in $C^{2}$ ) are the simplest examples of orthonormal sets. The Hadamard basis states $∣ + ⟩, ∣ - ⟩$ from the exercise above also form an orthonormal set.

Conclusion

In this lesson, we have generalized the geometric notion of perpendicularity to all inner product spaces. This is a profound step that unlocks powerful new techniques.

Key Takeaways:

Two vectors $u$ and $v$ in any inner product space (real or complex) are orthogonal if their inner product is zero: $⟨ u, v ⟩ = 0$ .
For the standard inner product in $R^{n}$ , this is the dot product: $\sum u_{i} v_{i} = 0$ .
For the standard inner product in $C^{n}$ , this is the Hermitian product: $\sum u ˉ_{i} v_{i} = 0$ .
An orthonormal set is a collection of mutually orthogonal unit vectors, a property concisely described by $⟨ v_{i}, v_{j} ⟩ = δ_{ij}$ .

Preview of the Next Lesson:

Orthonormal sets are incredibly convenient to work with. If they also form a basis for the vector space, they are called an orthonormal basis. Such bases simplify many complex problems, from representing quantum states to compressing data. But what if we have a basis that isn't orthonormal? In the next lesson, we will learn a constructive algorithm called the Gram-Schmidt process to turn any basis into an orthonormal one.

Recommended resources