Diffusion Models In-Depth

Define Markov chains with continuous state spaces and characterize their stationary distributions.

Describe the properties of Brownian motion (Wiener process), including continuity, independent increments, and Gaussian marginals.

Apply Itô's lemma to compute differentials of functions of stochastic processes.

Specify a stochastic differential equation (SDE) in terms of its drift and diffusion coefficients and interpret their roles.

Derive the Fokker-Planck equation for the time evolution of a probability density governed by an SDE.

Explain Langevin dynamics as a sampling algorithm and show how its stationary distribution relates to a target energy-based model.

Simulate Brownian motion and Langevin dynamics in Python to sample from a known multimodal distribution.

Define the score function as the gradient of the log-probability density and explain its role in generative modeling.

Derive the explicit score matching objective (Hyvärinen, 2005) and explain why it avoids computing the normalizing constant.

Derive the denoising score matching objective and show its equivalence to explicit score matching under certain conditions.

Explain the challenge of score estimation in low-density regions and how multi-scale noise conditioning addresses it.

Describe Noise Conditional Score Networks (NCSN) and the role of annealed Langevin dynamics for sample generation.

Implement denoising score matching for a simple 2D distribution and visualize the learned score field.

Define the forward diffusion process as a Markov chain of Gaussian transitions governed by a variance schedule.

Derive the closed-form expression for sampling from the forward process marginal q(x_t | x_0).

Derive the posterior of the reverse process q(x_{t-1} | x_t, x_0) and show that it is Gaussian.

Derive the simplified DDPM training objective from the variational lower bound and connect it to denoising score matching.

Explain the functional difference between linear and cosine noise schedules and their effect on training and sample quality.

Implement the simplified DDPM training loss function in Python for a single batch of data.

Describe the encoder-decoder structure of a U-Net with skip connections and explain its suitability for image-to-image tasks like denoising.

Implement sinusoidal time embeddings and explain how they condition the network on the diffusion timestep.

Explain the role of residual blocks and group normalization in stabilizing the training of deep U-Nets.

Describe how self-attention layers are integrated into a U-Net and at which feature resolutions they are most effective.

Implement a complete U-Net architecture in PyTorch for noise prediction.

Train a DDPM on the CIFAR-10 dataset using the implemented U-Net.

Visualize the progressive denoising process across timesteps for samples generated by the trained DDPM.

Describe ancestral sampling in DDPM and identify its computational cost due to its Markovian structure.

Derive the Denoising Diffusion Implicit Models (DDIM) update rule as a non-Markovian generalization of the DDPM reverse process.

Explain how DDIM enables deterministic sampling and semantic interpolation in the latent space.

Compare DDPM and DDIM sampling quality as a function of the number of function evaluations (NFE).

Implement DDIM sampling for a previously trained diffusion model.

Compare the visual quality of samples generated via DDPM vs. DDIM across different step budgets (e.g., 1000, 100, 20).

Formulate the discrete-time forward diffusion process as a continuous-time SDE.

Define the Variance Preserving (VP) and Variance Exploding (VE) SDEs and relate them to DDPM and NCSN, respectively.

Derive the corresponding reverse-time SDE (Anderson, 1982) and explain the central role of the score function.

Derive the probability flow ODE as the deterministic counterpart of the reverse-time SDE.

Explain how exact log-likelihood can be computed via the probability flow ODE and the instantaneous change-of-variables formula.

Describe how DDPM, NCSN/SMLD, and other variants are unified under the SDE/ODE framework.

Implement sampling from a trained score network using a numerical ODE solver on the probability flow ODE.

Formulate conditional generation as sampling from p(x|y) using a model of p(x).

Derive classifier guidance by decomposing the conditional score into the unconditional score and the gradient of a separately trained classifier.

Explain classifier-free guidance, including training with random label dropout and inference-time interpolation.

Adapt the U-Net architecture and training loop to accept class conditioning.

Train a class-conditional diffusion model on a labeled dataset (e.g., CIFAR-10).

Implement classifier-free guidance for the trained model and generate samples at varying guidance scales.

Analyze the effect of the guidance scale on the trade-off between sample fidelity (adherence to class) and diversity.

Explain the computational motivation for performing the diffusion process in a compressed latent space instead of pixel space.

Describe the autoencoder's role and pre-training, including the use of perceptual and adversarial losses.

Describe how text conditioning is achieved using a pre-trained frozen text encoder (e.g., CLIP).

Explain how cross-attention layers in the U-Net enable conditioning on the text encoder's outputs.

Diagram the complete Stable Diffusion architecture, showing the interaction between the autoencoder, U-Net, and text encoder.

Load the pre-trained components of a latent diffusion model using the Diffusers library.

Implement a text-to-image generation pipeline: text embedding, latent noise initialization, iterative denoising, and final decoding.

Explain the Inception Score (IS) and its limitations, particularly its sensitivity to mode collapse.

Define the Fréchet Inception Distance (FID) and explain why it is generally preferred over IS for evaluating image generation models.

Compare diffusion models with GANs and VAEs across key axes: training stability, sample quality, diversity, and computational cost.

Implement a script to compute the FID score for a batch of generated samples against a reference dataset.

Conduct an experiment to analyze the impact of the number of sampling steps on the FID score.

Conduct an experiment to analyze the impact of the guidance scale on the FID score.

Describe ControlNet and how it adds fine-grained spatial conditioning to pre-trained diffusion models without re-training them.

Explain the concept of model distillation (e.g., progressive distillation) for accelerating diffusion model sampling.

Describe consistency models as a method to achieve high-quality, one-step generation.

Explain the core idea of flow matching and its relationship to both diffusion models and normalizing flows.

Implement a simple pipeline using a pre-trained ControlNet to guide text-to-image generation with an edge map (e.g., Canny edges).

Identify and discuss current research frontiers in diffusion models, such as video generation, 3D synthesis, and non-autoregressive text generation.