Decoding Strategies and Inference Optimization

Explain how autoregressive generation produces text one token at a time using the model's own outputs.

Compare greedy decoding and beam search in terms of output quality and computational cost.

Explain how temperature scaling modifies the sharpness of the output probability distribution to control randomness.

Describe top-k and nucleus (top-p) sampling and how they balance diversity and coherence in generated text.

Explain KV caching and how it avoids redundant computation of key and value states during autoregressive generation.