Solution Sets: Span and Affine Subspaces

1. Homogeneous vs. Inhomogeneous Systems

Let's start by formalizing the two types of systems.

• A system of the form $A\mathbf{x}=0$ is called homogeneous.
• A system of the form $A\mathbf{x}=\mathbf{b}$ with $\mathbf{b}\ne 0$ is called inhomogeneous.

As we know from previous lessons, the solution set of a homogeneous system is the null space, $\text{Nul}(A)$. We find a basis for this subspace by finding the parametric vector form of the solution, which expresses the solution set as the span of a set of vectors.

The key takeaway is that the solution set to a homogeneous equation is always a subspace passing through the origin, whose dimension is the number of free variables.

2. The Structure of Inhomogeneous Solutions

Now for the main event: what is the structure of the solution set to $A\mathbf{x}=\mathbf{b}$? The fundamental insight is surprisingly simple.

Let's suppose we've found at least one solution to $A\mathbf{x}=\mathbf{b}$. We'll call this single solution a particular solution, denoted ${\mathbf{x}}_p$. So, we know that $A{\mathbf{x}}_p=\mathbf{b}$.

Now, let ${\mathbf{x}}_h$ be any solution to the corresponding homogeneous equation, $A{\mathbf{x}}_h=0$. What happens if we add these two vectors together?

This means that ${\mathbf{x}}_p+{\mathbf{x}}_h$ is also a solution to the inhomogeneous equation! In fact, it turns out that all solutions to $A\mathbf{x}=\mathbf{b}$ can be written in this form.

This gives us our central theorem:
The general solution to $A\mathbf{x}=\mathbf{b}$ is the sum of a particular solution (${\mathbf{x}}_p$) and the general solution to the homogeneous equation (${\mathbf{x}}_h$).

This video provides an excellent intuitive explanation, a simple proof, and an example of this powerful idea.

This structure means that to solve an inhomogeneous system, you just need to:

1. Find one solution, ${\mathbf{x}}_p$.
2. Find all solutions to the associated homogeneous system, ${\mathbf{x}}_h$, which is just the null space.
3. Add them together.

3. Geometric Interpretation: Affine Subspaces

What does the structure $\mathbf{x}={\mathbf{x}}_p+{\mathbf{x}}_h$ mean geometrically?

• We know ${\mathbf{x}}_h$ represents the null space, which is a subspace (a line, plane, etc., through the origin).
• Adding the constant vector ${\mathbf{x}}_p$ to every vector in the null space has the effect of translating the entire null space. The resulting set is parallel to the original null space but shifted so that it passes through the point defined by ${\mathbf{x}}_p$.

This translated subspace is called an affine subspace.

Formally, an affine subspace is a set that is a translation of a vector subspace.

The reading confirms that the set of solutions to $A\mathbf{x}=\mathbf{b}$ is an affine subspace. The dimension of this affine subspace is defined to be the dimension of the vector subspace it is a translation of—in this case, $\text{dim}(\text{Nul}(A))$.

So, if $\text{dim}(\text{Nul}(A))=1$, the homogeneous solution is a line through the origin, and the inhomogeneous solution is a line not through the origin (a 1D affine subspace). If $\text{dim}(\text{Nul}(A))=2$, the solution sets are a plane through the origin and a translated parallel plane, respectively.

4. Consequences and Broader Connections

This structure ($\mathbf{x}={\mathbf{x}}_p+{\mathbf{x}}_h$) has some important consequences. For instance, why can a linear system have 0, 1, or infinitely many solutions, but never exactly 2?

• 0 solutions: If no particular solution ${\mathbf{x}}_p$ exists (i.e., $\mathbf{b}$ is not in the column space of $A$).
• 1 solution: A particular solution ${\mathbf{x}}_p$ exists, and the homogeneous solution is only the trivial solution, ${\mathbf{x}}_h=0$. This happens when the null space has dimension 0.
• Infinite solutions: A particular solution ${\mathbf{x}}_p$ exists, and the homogeneous solution set (the null space) contains infinitely many vectors (i.e., its dimension is greater than 0).

You can't have just two solutions, because if you have ${\mathbf{x}}_1$ and ${\mathbf{x}}_2$, then their difference ${\mathbf{x}}_1-{\mathbf{x}}_2$ is a non-zero vector in the null space. This means the null space is non-trivial, so it contains infinitely many vectors (all scalar multiples of ${\mathbf{x}}_1-{\mathbf{x}}_2$), which in turn generates infinitely many solutions.

This principle extends far beyond matrix equations. As your background includes physics and calculus, you might find it interesting that this same structure is the key to solving linear differential equations.

This demonstrates that what you're learning isn't just a set of rules for matrices; it's a deep structural property of all linear systems, which appear throughout science and engineering.

Conclusion

This lesson completes our first module by giving a full geometric picture of the solution sets to any system of linear equations.

Key Takeaways:

• The solution set to a homogeneous system ($A\mathbf{x}=0$) is a vector subspace (the null space), which can be expressed as a span of basis vectors.
• The solution set to a consistent inhomogeneous system ($A\mathbf{x}=\mathbf{b}$) is an affine subspace.
• The structure of the general solution is $\mathbf{x}={\mathbf{x}}_p+{\mathbf{x}}_h$, where ${\mathbf{x}}_p$ is one particular solution and ${\mathbf{x}}_h$ is any vector from the null space.
• Geometrically, an affine subspace is a translation of a vector subspace. The solution set of $A\mathbf{x}=\mathbf{b}$ is the null space of $A$ shifted by a particular solution vector ${\mathbf{x}}_p$.

Preview of Next Module:
We have now built a solid foundation by studying vector spaces, subspaces, bases, dimension, and the structure of solution sets. In the next module, "Linear Transformations and Matrices," we will shift our perspective. Instead of thinking of $A\mathbf{x}=\mathbf{b}$ as a system to be solved, we will begin to view it as a function that takes an input vector $\mathbf{x}$ and transforms it into an output vector $\mathbf{b}$. This will open up a powerful new way to understand the geometry of matrices and their role in applications like deep learning and quantum mechanics.