Diffusion Models in ML

Derive the tractable posterior q(x_{t-1} | x_t, x_0) of the forward process.

Decompose the diffusion evidence lower bound into reconstruction and denoising terms.

Derive the noise-prediction mean-squared-error objective from the variational diffusion objective.

Convert consistently among epsilon-, x_0-, and v-prediction parameterizations.

Implement unbiased random-timestep sampling for the DDPM training loss.

Construct sinusoidal timestep embeddings and explain how their frequencies represent noise levels.

Trace tensor shapes and information flow through a diffusion U-Net with residual and skip connections.

Implement a residual block conditioned on timestep embeddings by scale-and-shift modulation.

Integrate self-attention at selected U-Net resolutions and explain the computational trade-off.

Assemble a compact noise-predicting U-Net and execute an end-to-end DDPM training run.

Implement the ancestral DDPM reverse update from a predicted noise tensor.

Explain how fixed and learned reverse-process variances affect likelihood and sample quality.

Derive the DDIM sampling update and interpret its stochasticity parameter.

Apply timestep respacing to trade sampling cost against image quality.

Compare DDPM and DDIM samplers experimentally using matched model checkpoints.

Express a noise predictor as an estimator of the score of a perturbed data distribution.

Derive the denoising score-matching objective for Gaussian perturbations.

Obtain the variance-preserving stochastic differential equation as the continuous limit of a discrete diffusion process.

Formulate the reverse-time stochastic differential equation from a learned score function.

Derive the probability-flow ordinary differential equation associated with a diffusion SDE.

Implement an Euler–Maruyama sampler for a reverse-time diffusion SDE.

Derive classifier guidance as a modification of the unconditional score using Bayes' rule.

Formulate classifier-free guidance training with random condition dropout.

Implement classifier-free guided sampling and analyze the effect of guidance scale.

Explain how cross-attention injects token-level conditioning into a denoising network.

Explain the role of the autoencoder and latent scaling in latent diffusion models.

Construct the training and sampling dataflow of a text-conditioned latent diffusion model.

Compare the inductive biases and scaling behavior of diffusion U-Nets and Diffusion Transformers.

Apply EDM-style input, output, and noise preconditioning to a denoising model.

Derive the conditional flow-matching regression objective.

Explain why rectified-flow couplings can produce straighter, cheaper sampling trajectories.

Implement a two-dimensional flow-matching model and integrate its learned vector field.

Distinguish consistency training, consistency distillation, and standard diffusion sampling.

Compute FID and KID from generated and reference features and identify their main statistical limitations.

Design a controlled ablation comparing noise schedule, prediction parameterization, sampler, or guidance scale.

Diagnose undertraining, overexposure to noise levels, and sampling errors from losses and generated trajectories.

Build a reproducible training checklist covering data scaling, exponential moving averages, checkpoints, and random seeds.

Select and justify a diffusion, latent diffusion, flow-matching, or consistency approach for a specified image-generation constraint.