Probability and Statistics for Generative Models

Refresh fundamentals of probability spaces and random variables, framed through DL examples like stochastic mini-batches and dropout.

Interpret expectations, variances, and covariances in contexts like Monte Carlo estimators and optimizer noise.

Apply the laws of total expectation and variance to reason about hierarchical and latent-variable models.

Model discrete outcomes using Bernoulli and Categorical distributions and connect them to the cross-entropy loss.

Model continuous variables using the Gaussian distribution and connect its properties to regression, VAE latents, and diffusion noise.

Derive the multivariate Gaussian density via an affine transformation of standard normal variables.

Compute the marginal and conditional distributions of a multivariate Gaussian using the Schur complement, as required in many generative models.