Week 4 — A Working Mini Diffusion Language Model

Train the tiny masked diffusion language model on a small text corpus in Colab and inspect its generated samples.

Implement blockwise semi-autoregressive decoding that denoises several tokens in parallel within each block.

Explain why masked diffusion language models lack the usual autoregressive sequence likelihood and how the denoising loss acts as a training surrogate.

Translate the diffusion-language-model notation and equations in Section 3.1 of the linked paper into operations in the Colab implementation.

Run a controlled experiment varying denoising steps, reveal schedule, and temperature, then explain the observed quality–speed trade-offs.