Closed-Form Gaussian Forward Diffusion Distribution

Hello again. In the previous lesson, we defined the forward process as a fixed Markov chain that gradually corrupts clean data ${\mathbf{x}}_0$, and we separated it from the learned reverse chain used for generation. The forward transition was

$$q({\mathbf{x}}_t\mid {\mathbf{x}}_{t-1})=\mathcal{N}\left({\mathbf{x}}_t;\sqrt{{\alpha }_t}{\mathbf{x}}_{t-1},(1-{\alpha }_t)\mathbf{I}\right),$$

where

$${\alpha }_t=1-{\beta }_t.$$

That one-step definition is conceptually clear, but simulating every preceding step just to obtain one noisy training example at timestep $t$ would be wasteful. This lesson derives the central shortcut of DDPMs: a closed-form distribution for ${\mathbf{x}}_t$ given the original clean sample ${\mathbf{x}}_0$. It lets us add the correct total amount of noise in a single operation.

---

The result we want

Define the cumulative signal retention through timestep $t$ as

$${\bar{\alpha }}_t=\prod _{s=1}^t{\alpha }_s.$$

The bar distinguishes ${\bar{\alpha }}_t$, a product across all earlier timesteps, from ${\alpha }_t$, the retention at one particular step.

We will show that the whole sequence of $t$ noising steps can be collapsed into:

$$q({\mathbf{x}}_t\mid {\mathbf{x}}_0)=\mathcal{N}\left({\mathbf{x}}_t;\sqrt{{\bar{\alpha }}_t}{\mathbf{x}}_0,(1-{\bar{\alpha }}_t)\mathbf{I}\right).$$

Equivalently, a sample from this conditional distribution can be written as

$${\mathbf{x}}_t=\sqrt{{\bar{\alpha }}_t}{\mathbf{x}}_0+\sqrt{1-{\bar{\alpha }}_t}\mathbf{ϵ},\mathbf{ϵ}\sim \mathcal{N}(0,\mathbf{I}).$$

This equation has a useful interpretation:

• $\sqrt{{\bar{\alpha }}_t}{\mathbf{x}}_0$ is the remaining scaled signal from the clean datum.
• $\sqrt{1-{\bar{\alpha }}_t}\mathbf{ϵ}$ is the accumulated Gaussian noise.
• The two coefficients are chosen so that the added noise has covariance $(1-{\bar{\alpha }}_t)\mathbf{I}$.

The result is conditional on a fixed ${\mathbf{x}}_0$. It is not yet a formula for the marginal distribution $q({\mathbf{x}}_t)$, which also averages over the unknown data distribution.

---

Unrolling two diffusion steps

Begin with the reparameterized form of one forward step:

$${\mathbf{x}}_t=\sqrt{{\alpha }_t}{\mathbf{x}}_{t-1}+\sqrt{1-{\alpha }_t}{\mathbf{ϵ}}_t,{\mathbf{ϵ}}_t\sim \mathcal{N}(0,\mathbf{I}).$$

Consider $t=2$. First,

$${\mathbf{x}}_1=\sqrt{{\alpha }_1}{\mathbf{x}}_0+\sqrt{1-{\alpha }_1}{\mathbf{ϵ}}_1.$$

Then substitute that expression into the second step:

$$\begin{array}{rl}{\mathbf{x}}_2& =\sqrt{{\alpha }_2}{\mathbf{x}}_1+\sqrt{1-{\alpha }_2}{\mathbf{ϵ}}_2\\ & =\sqrt{{\alpha }_2{\alpha }_1}{\mathbf{x}}_0+\sqrt{{\alpha }_2(1-{\alpha }_1)}{\mathbf{ϵ}}_1+\sqrt{1-{\alpha }_2}{\mathbf{ϵ}}_2.\end{array}$$

The clean-data term already has the pattern we want:

$$\sqrt{{\alpha }_2{\alpha }_1}{\mathbf{x}}_0=\sqrt{{\bar{\alpha }}_2}{\mathbf{x}}_0.$$

The remaining two terms are both independent, zero-mean Gaussian vectors. Their sum is therefore also Gaussian with zero mean. Its covariance is the sum of their covariances:

$$\begin{array}{rl}& {\alpha }_2(1-{\alpha }_1)\mathbf{I}+(1-{\alpha }_2)\mathbf{I}\\ & =\left[{\alpha }_2-{\alpha }_2{\alpha }_1+1-{\alpha }_2\right]\mathbf{I}\\ & =(1-{\alpha }_2{\alpha }_1)\mathbf{I}.\end{array}$$

Since ${\bar{\alpha }}_2={\alpha }_1{\alpha }_2$, the combined noise has covariance

$$(1-{\bar{\alpha }}_2)\mathbf{I}.$$

Thus we may replace the two separate noise terms with one fresh standard Gaussian vector $\mathbf{ϵ}$:

$${\mathbf{x}}_2=\sqrt{{\bar{\alpha }}_2}{\mathbf{x}}_0+\sqrt{1-{\bar{\alpha }}_2}\mathbf{ϵ}.$$

This is equality in distribution: the single $\mathbf{ϵ}$ need not be literally the same random draw as either earlier noise variable. It produces exactly the same conditional Gaussian distribution.

---

---

The general derivation by induction

Unrolling the chain is instructive, but there is a shorter proof that makes the general pattern precise.

Suppose that at timestep $t-1$, the desired formula holds:

$$q({\mathbf{x}}_{t-1}\mid {\mathbf{x}}_0)=\mathcal{N}\left({\mathbf{x}}_{t-1};\sqrt{{\bar{\alpha }}_{t-1}}{\mathbf{x}}_0,(1-{\bar{\alpha }}_{t-1})\mathbf{I}\right).$$

The next forward step is

$${\mathbf{x}}_t=\sqrt{{\alpha }_t}{\mathbf{x}}_{t-1}+\sqrt{1-{\alpha }_t}{\mathbf{ϵ}}_t.$$

Conditioned on ${\mathbf{x}}_0$, this is an affine transformation of Gaussian variables, so ${\mathbf{x}}_t$ is Gaussian. We only need its mean and covariance.

Conditional mean

Because ${\mathbf{ϵ}}_t$ has mean zero,

$$\begin{array}{rl}\mathbb{E}[{\mathbf{x}}_t\mid {\mathbf{x}}_0]& =\sqrt{{\alpha }_t}\mathbb{E}[{\mathbf{x}}_{t-1}\mid {\mathbf{x}}_0]\\ & =\sqrt{{\alpha }_t}\sqrt{{\bar{\alpha }}_{t-1}}{\mathbf{x}}_0\\ & =\sqrt{{\bar{\alpha }}_t}{\mathbf{x}}_0.\end{array}$$

The final equality follows from

$${\bar{\alpha }}_t={\alpha }_t{\bar{\alpha }}_{t-1}.$$

Conditional covariance

The covariance of the scaled previous state is

$${\alpha }_t(1-{\bar{\alpha }}_{t-1})\mathbf{I}.$$

The covariance of the newly injected noise is

$$(1-{\alpha }_t)\mathbf{I}.$$

The two terms are independent, so their covariances add:

$$\begin{array}{rl}\mathrm{Cov}({\mathbf{x}}_t\mid {\mathbf{x}}_0)& =\left[{\alpha }_t(1-{\bar{\alpha }}_{t-1})+1-{\alpha }_t\right]\mathbf{I}\\ & =\left[1-{\alpha }_t{\bar{\alpha }}_{t-1}\right]\mathbf{I}\\ & =(1-{\bar{\alpha }}_t)\mathbf{I}.\end{array}$$

Together, the mean and covariance establish the closed form:

$$\boxed{q({\mathbf{x}}_t\mid {\mathbf{x}}_0)=\mathcal{N}\left({\mathbf{x}}_t;\sqrt{{\bar{\alpha }}_t}{\mathbf{x}}_0,(1-{\bar{\alpha }}_t)\mathbf{I}\right)}$$

for every timestep $t$.

The base case is $t=1$, where ${\bar{\alpha }}_1={\alpha }_1$, so the formula is exactly the original one-step forward transition from ${\mathbf{x}}_0$.

---

Reading the formula correctly

The closed form encodes several important properties of Gaussian diffusion.

Signal decays multiplicatively

At each step, the existing image is scaled by $\sqrt{{\alpha }_t}$. Across many steps, these scalings multiply:

$$\sqrt{{\alpha }_1}\sqrt{{\alpha }_2}\cdots \sqrt{{\alpha }_t}=\sqrt{{\bar{\alpha }}_t}.$$

Since each ${\alpha }_t$ is between zero and one, ${\bar{\alpha }}_t$ decreases as $t$ increases. The original image’s contribution becomes smaller and smaller.

Noise variance is not a simple sum of ${\beta }_t$

A common first mistake is to expect the total noise variance to be

$$\sum _{s=1}^t{\beta }_s.$$

That would ignore the fact that noise injected at an earlier step is itself scaled down by every later $\sqrt{{\alpha }_s}$. The correct total variance is

$$1-{\bar{\alpha }}_t.$$

The DDPM choice ${\alpha }_t=1-{\beta }_t$ makes this expression particularly clean and keeps the scale controlled.

Large timesteps approach standard Gaussian noise

When the cumulative product becomes close to zero,

$${\bar{\alpha }}_t\approx 0,$$

the conditional distribution becomes

$$q({\mathbf{x}}_t\mid {\mathbf{x}}_0)\approx \mathcal{N}(0,\mathbf{I}).$$

The clean image has nearly disappeared from the mean, and the covariance is close to the identity. This is why a diffusion model can use standard Gaussian noise as the starting point for reverse-time generation.

---

A small numerical example

Suppose, purely for illustration, that two timesteps retain

$${\alpha }_1=0.9,{\alpha }_2=\mathrm{0.8.}$$

Their cumulative retention at timestep $2$ is

$${\bar{\alpha }}_2=0.9\cdot 0.8=\mathrm{0.72.}$$

Therefore,

$${\mathbf{x}}_2=\sqrt{0.72}{\mathbf{x}}_0+\sqrt{0.28}\mathbf{ϵ}.$$

The coefficients have complementary squared magnitudes:

$$0.72+0.28=1.$$

At this stage, the conditional mean is still visibly related to the original datum, but each coordinate has acquired variance $0.28$ from noise. Real diffusion schedules use many much smaller steps, but the same multiplicative accounting applies.

---

Why the closed form matters in practice

During DDPM training, the model typically picks a random timestep $t$ for each training example. The closed form permits the following efficient procedure:

1. Take a clean batch ${\mathbf{x}}_0$.
2. Sample a timestep $t$.
3. Sample $\mathbf{ϵ}$ from $\mathcal{N}(0,\mathbf{I})$.
4. Construct ${\mathbf{x}}_t$ directly using $\sqrt{{\bar{\alpha }}_t}$ and $\sqrt{1-{\bar{\alpha }}_t}$.
5. Train a neural network to infer the relevant denoising information from ${\mathbf{x}}_t$ and $t$.

No loop through ${\mathbf{x}}_1,\dots ,{\mathbf{x}}_{t-1}$ is required. This makes randomly training across noise levels computationally practical.

The following short segment visualizes that “one-shot” construction after the algebraic collapse of the Markov chain.

---

Key takeaways

The forward diffusion chain has the one-step transition

$${\mathbf{x}}_t=\sqrt{{\alpha }_t}{\mathbf{x}}_{t-1}+\sqrt{1-{\alpha }_t}{\mathbf{ϵ}}_t.$$

Because scaled independent Gaussian variables remain Gaussian and their variances add, all earlier noise variables can be collapsed into a single standard Gaussian draw. Defining

$${\bar{\alpha }}_t=\prod _{s=1}^t{\alpha }_s,$$

gives the fundamental DDPM identity:

$${\mathbf{x}}_t=\sqrt{{\bar{\alpha }}_t}{\mathbf{x}}_0+\sqrt{1-{\bar{\alpha }}_t}\mathbf{ϵ}.$$

Equivalently,

$$q({\mathbf{x}}_t\mid {\mathbf{x}}_0)=\mathcal{N}\left(\sqrt{{\bar{\alpha }}_t}{\mathbf{x}}_0,(1-{\bar{\alpha }}_t)\mathbf{I}\right).$$

The cumulative product ${\bar{\alpha }}_t$ measures retained signal power, while $1-{\bar{\alpha }}_t$ measures accumulated noise variance. In the next lesson, we will turn those two quantities into a more explicit diagnostic: the signal-to-noise ratio at a chosen diffusion timestep.