Geometric Linear Algebra for Deep Learning

Interpret matrix multiplication as the composition of linear maps and as a sum of outer products, with examples from neural network layers.

Use vector subspaces, span, basis, and dimension to describe the geometry of weight matrices and activation spaces.

Analyze the four fundamental subspaces of a linear layer's weight matrix to understand information flow and bottlenecks.

Apply orthogonal projections to derive least-squares solutions and connect them to data whitening and decorrelation.

Connect determinants to volume change and traces to infinitesimal change, with applications in normalizing flows and continuous-time models.

Use induced matrix norms (e.g., operator, Frobenius) to bound activation magnitudes and analyze gradient stability in deep networks.