Understanding Forward and Reverse Processes in Diffusion Models

Hello! Welcome to our first lesson on Diffusion Models, a class of generative models that have powered many recent breakthroughs in AI image generation.

In the last module, we concluded by comparing VAEs and GANs. We saw a fundamental trade-off: VAEs are stable to train but often produce blurry images, while GANs can generate sharp, realistic images but are notoriously unstable and prone to issues like mode collapse.

Today, we introduce a third and powerful paradigm. This lesson will address the learning outcome: Explain the forward (noising) and reverse (denoising) processes in diffusion models. We will explore the elegant and surprisingly simple core idea behind models like DALL-E 2 and Stable Diffusion: systematically destroying data with noise and then learning to reverse the process to generate new data.

Let's begin by building an intuition for how this works.

1. The Core Idea: Destroy and Reconstruct

At its heart, a diffusion model works in two stages:

1. Forward Process (Noising): We take a clean image from our dataset and gradually add a little bit of noise over many steps, until the image becomes pure, unrecognizable static. This process is fixed and doesn't involve any learning. It's like taking a photograph and leaving it out in the rain to slowly disintegrate.

2. Reverse Process (Denoising): We train a neural network to do the opposite. Starting with pure noise, the model learns to gradually remove the noise, step-by-step, to reconstruct a clean image. This is the "generative" part. It's like an expert art restorer who can look at a damaged canvas and perfectly repaint the original.

This two-part structure is the foundation of how these models learn to generate incredibly detailed images. Let's watch a short video that introduces this concept.

Now, let's get a more structured overview of these two processes.

The diagram below, adapted from the original Denoising Diffusion Probabilistic Models (DDPM) paper, illustrates this flow. The forward process $q$ moves from a clean image $x_0$ to noise $x_T$, and the learned reverse process $p_{\theta }$ moves from noise back to a clean image.

2. The Forward Process: A Controlled Destruction

The forward process is a fixed procedure defined by a mathematical formula. It doesn't involve any training. The goal is to take an image $x_0$ and produce a sequence of progressively noisier images $x_1,x_2,\dots ,x_T$.

This is a Markov process, meaning that each noisy image $x_t$ only depends on the previous image $x_{t-1}$. At each step $t$, we generate $x_t$ by:

1. Slightly scaling down the pixels of $x_{t-1}$.
2. Adding a small amount of Gaussian noise.

The amount of noise added at each step is controlled by a variance schedule, denoted by ${\beta }_t$. This schedule is a set of pre-defined hyperparameters. Typically, ${\beta }_t$ starts small and increases as $t$ gets larger.

For a deeper look at the mechanics, the following article provides a clear, visual explanation.

A Key Mathematical Property

One crucial property of the forward process is that we don't need to perform all $T$ steps sequentially to get a noisy image. Thanks to the properties of Gaussian distributions, we can derive a formula to generate a noisy sample $x_t$ at any timestep $t$ directly from the original image $x_0$.

If we define ${\alpha }_t=1-{\beta }_t$ and ${\bar{\alpha }}_t={\prod }_{i=1}^t{\alpha }_i$, we can sample $x_t$ directly:

where $ϵ$ is a sample from a standard Gaussian distribution $\mathcal{N}(0,I)$.

This formula is extremely important for efficient training. It means we can randomly pick any image from our dataset, randomly pick any timestep $t$, and instantly generate a corresponding noisy sample for our model to train on.

Let's see the mathematical details of this property.

3. The Reverse Process: Learning to Denoise

The reverse process is where the learning happens. Our goal is to train a model, typically a U-Net architecture, to reverse the forward process. Starting from pure noise $x_T\sim \mathcal{N}(0,I)$, the model must iteratively predict a slightly less noisy image $x_{t-1}$ given $x_t$.

If the noise steps (${\beta }_t$) in the forward process are small enough, the reverse transition $q(x_{t-1}|x_t)$ will also be approximately Gaussian. This allows us to model the reverse process $p_{\theta }(x_{t-1}|x_t)$ with a neural network that learns the parameters (mean and variance) of this Gaussian distribution at each step.

What Does the Model Actually Predict?

A key insight from the DDPM paper is that it's more effective to train the model to predict the noise ($ϵ$) that was added to the image at a given step, rather than predicting the denoised image $x_{t-1}$ directly. The model is given a noisy image $x_t$ and the timestep $t$, and its task is to output a prediction of the noise, ${ϵ}_{\theta }(x_t,t)$.

Why Does This Work So Well?

The 3Blue1Brown video offers a fantastic intuition for why this training objective is so effective and why another surprising detail—adding noise during generation—is crucial.

The point about adding noise during generation is particularly insightful. The model learns to predict the mean of the reverse distribution. By just following the mean, all generated samples would collapse to the average of the data distribution, resulting in blurry images reminiscent of what we saw with VAEs. Adding a random noise step ensures we are actually sampling from the learned distribution, which preserves diversity and leads to sharp results.

Conclusion

In this lesson, we have unpacked the core mechanism behind diffusion models. You now understand the two fundamental processes that enable these models to generate high-quality data from noise.

Key Takeaways:

• Diffusion models consist of two parts: a fixed forward (noising) process and a learned reverse (denoising) process.
• The forward process is a Markov chain that systematically corrupts an image by adding Gaussian noise over many steps, governed by a variance schedule. A key property allows for efficient, direct sampling of a noisy image at any timestep $t$.
• The reverse process uses a neural network (often a U-Net) to undo the noising. It starts with pure noise and iteratively denoises it to generate a clean image.
• The model is trained to predict the noise added at each step, rather than the denoised image itself.
• Adding a small amount of random noise during the generation (reverse) process is crucial for producing diverse and sharp samples, preventing a collapse to the average image.

Preview of the Next Lesson

Now that we understand the what and the why, our next step is to formalize the how. In the next lesson, we will explore the training objective of a Denoising Diffusion Probabilistic Model (DDPM). We will look at the loss function that is minimized during training and understand how it connects the predicted noise to the actual noise added during the forward process.