Calculating Additional Reveals per Denoising Step

Hello! In the previous lesson, you gave each masked position a candidate token and a confidence score. Nothing was revealed yet. Today we will answer a separate question: how many positions should become visible at each denoising step? Once you have that number, the confidence scores can help decide which positions to reveal.

From a masked canvas to a reveal budget

The picture below shows the basic motion of generation: the prompt stays fixed while masked positions in the output gradually become visible. The rows show successive states, not a requirement that decoding proceed block by block.

For a short explanation of why masked generation proceeds gradually, read these two passages from “How to Build a Diffusion Language Model” by the Kuleshov Group:

That reading describes a probabilistic masking process: a schedule controls the expected fraction of visible tokens. For our small Colab sampler, we can make the count explicit instead. A fixed reveal budget says exactly how many positions should be visible after each step. This is a practical decoding choice, not a claim that every masked diffusion sampler uses fixed counts.

Calculate the count at each step

Suppose there are $R$ output positions to generate and $K$ denoising steps. Count only generation slots in $R$: a fixed prompt prefix and any padding are not waiting to be revealed.

Let $C_k$ be the cumulative target: how many of those $R$ positions should be visible after step $k$. For a linear reveal schedule, divide the generation progress evenly:

We have $C_0=0$ and $C_K=R$. The number of additional positions to reveal at step $k$ is the difference between successive targets:

The floor makes each target a whole-number count. It may also make step sizes slightly unequal.

For example, imagine a padded sequence with a three-token fixed prefix, two padding positions, and seven generation slots. With four steps, $R=7$ and $K=4$:

After step 2, three positions are visible in total, but only two new positions were revealed during step 2. That cumulative-versus-additional distinction is the key calculation.

The schedule does not choose positions. At step 2, for instance, the sampler could use the confidence scores from the previous lesson to select two of the six positions that were still masked after step 1. Actually committing their proposed tokens is the next lesson’s task.

Other schedules, same calculation

A linear schedule is not mandatory. Write $\kappa (u)$ for the desired visible fraction at generation progress $u$, where $\kappa (0)=0$, $\kappa (1)=1$, and the function never decreases. Then use

For example, $\kappa (u)=2u-u^2$ reveals more positions early than $\kappa (u)=u$. With the same seven slots and four steps, its cumulative targets are $0,3,5,6,7$, so its additional reveal counts are $3,2,1,1$. Both schedules finish with seven visible tokens; they differ in when those tokens become visible.

Here is a small Python function you can use in Colab to check the linear arithmetic without running the denoiser:

def linear_reveal_counts(num_slots, num_steps):
    if num_slots < 0 or num_steps < 1:
        raise ValueError("Need nonnegative slots and at least one step")

    targets = [
        num_slots * k // num_steps
        for k in range(num_steps + 1)
    ]
    return [
        targets[k] - targets[k - 1]
        for k in range(1, num_steps + 1)
    ]

print(linear_reveal_counts(7, 4))  # [1, 2, 2, 2]

If there are more steps than slots, some counts will be zero. That is mathematically fine, although a step that reveals nothing may not be useful in a simple irreversible sampler.

The reading uses $t=1$ for the fully masked end of the forward noise path and $t=0$ for clean text. Here, $k/K$ measures progress in the opposite direction—through generation, starting fully masked. Keeping those two directions distinct will help when you encounter diffusion notation again.

Takeaway

A reveal schedule supplies a cumulative visible-token target $C_k$. Subtract the previous target to obtain the new reveals ${\Delta }_k$, counting only positions that belong to the generated output. The schedule determines how many positions to reveal; confidence can help determine which ones.

Next, you will combine those two decisions into one reverse-denoising update that commits selected token predictions to the masked canvas.