Large Language Models

Explain why raw text must be converted to numerical representations for neural network processing.

Compare character-level, word-level, and subword-level tokenization in terms of vocabulary size, handling of rare words, and sequence length.

Describe how Byte Pair Encoding (BPE) iteratively builds a subword vocabulary from a text corpus.

Define a vocabulary and explain its relationship to the tokenization process and the model's output layer.

Explain how token embedding layers map discrete token IDs to dense, continuous vectors.

Derive sinusoidal positional encodings and explain why position information must be explicitly injected in non-recurrent architectures like the Transformer.

Define the roles of query, key, and value vectors in the attention mechanism.

Explain how dot products between queries and keys measure token relevance.

Compute scaled dot-product attention given Q, K, and V matrices, including the application of softmax to convert scores into a probability distribution.

Explain why the dot products are scaled by the square root of the key dimension to prevent softmax saturation and vanishing gradients.

Explain how causal masking is implemented to prevent attention to future tokens in autoregressive models.

Explain the motivation for using multiple attention heads in parallel instead of a single, larger attention head.

Describe how multi-head attention computes, concatenates, and projects multiple parallel attention outputs.

Explain the role of the position-wise feed-forward network within a Transformer block.

Explain the role and properties of the GELU activation function commonly used in the position-wise feed-forward network.

Explain how residual connections mitigate the vanishing gradient problem and facilitate the training of deep networks.

Describe the purpose of layer normalization in stabilizing the training of Transformers.

Trace the data flow through the multi-head attention sub-layer, including the residual connection and layer normalization.

Trace the data flow from the attention sub-layer output through the position-wise feed-forward sub-layer to produce the final Transformer block output.

Describe the original encoder-decoder Transformer architecture from 'Attention Is All You Need'.

Distinguish between self-attention and cross-attention, and identify where each is used in an encoder-decoder model.

Describe the decoder-only architecture (e.g., GPT series) and its suitability for generative tasks.

Describe the encoder-only architecture (e.g., BERT) and its suitability for understanding-based tasks.

Compare encoder-only, decoder-only, and encoder-decoder architectures in terms of their training objectives and typical applications.

Define autoregressive (causal) language modeling as sequential next-token prediction, typical for decoder-only models.

Define masked language modeling as used in BERT and explain how it enables bidirectional context understanding.

Explain how the final linear layer and softmax function produce a probability distribution over the entire vocabulary.

Describe how teacher forcing is used during training and why it differs from autoregressive generation at inference.

Explain the Adam optimizer's use of first and second moment estimates and why it is well-suited for training Transformers.

Describe learning rate scheduling strategies, including linear warmup and cosine decay, and their role in stable training.

Explain gradient accumulation and how it enables training with effectively larger batch sizes on memory-constrained hardware.

Explain mixed-precision training (e.g., bfloat16) and how it reduces memory usage while maintaining numerical stability.

Describe data parallelism and how it enables training on large datasets by replicating the model across multiple devices.

Describe pipeline parallelism and how it enables training models too large for a single device by splitting layers across devices.

Describe tensor parallelism and how it enables training very large model layers by splitting individual computations across devices.

Explain the Kaplan et al. scaling laws relating model performance (loss) to compute, dataset size, and parameter count.

Explain the Chinchilla scaling laws and how they revised compute-optimal allocation between model size and training data.

Analyze the tradeoffs between model size, dataset size, and compute budget when planning a training run based on scaling laws.

Describe emergent capabilities in large language models and explain how qualitative changes can arise from quantitative increases in scale.

Define in-context learning as an emergent capability and differentiate it from traditional weight-update-based learning (fine-tuning).

Explain Rotary Positional Embeddings (RoPE) and contrast them with absolute positional encodings.

Describe the Mixture of Experts (MoE) architecture and explain how it increases parameter count while keeping computational cost per token constant.

Explain the role of the router network in an MoE layer and the concept of load balancing loss.

Compare modern architectural components like RMSNorm and SwiGLU to their original Transformer counterparts (LayerNorm, GELU).

Explain how autoregressive generation produces text one token at a time using the model's own outputs.

Compare greedy decoding and beam search in terms of output quality and computational cost.

Explain how temperature scaling modifies the sharpness of the output probability distribution to control randomness.

Describe top-k and nucleus (top-p) sampling and how they balance diversity and coherence in generated text.

Explain KV caching and how it avoids redundant computation of key and value states during autoregressive generation.

Describe transfer learning in the context of LLMs and why pre-training followed by fine-tuning is effective.

Explain supervised fine-tuning (SFT) and how it adapts a pre-trained model to specific tasks or instruction-following behavior.

Explain Low-Rank Adaptation (LoRA), detailing how it freezes pre-trained weights and injects trainable low-rank matrices.

Compare full fine-tuning with parameter-efficient fine-tuning (PEFT) methods like LoRA in terms of computational cost, memory, and performance tradeoffs.

Explain the concept of AI alignment and what failure modes (e.g., helpfulness, harmlessness) it seeks to address in LLMs.

Outline the three high-level stages of the Reinforcement Learning from Human Feedback (RLHF) pipeline.

Explain how a reward model is trained from pairwise human preference data to learn a scalar reward signal.

Describe the role of Proximal Policy Optimization (PPO) in optimizing the SFT model against the reward model while regularizing against policy shift.

Explain Direct Preference Optimization (DPO) as an alternative to RLHF that directly optimizes the policy using preference data, bypassing an explicit reward model.

Define perplexity and explain its use and limitations as a metric for language model evaluation.

Describe the purpose of standardized evaluation benchmarks like MMLU and HELM for assessing broad model capabilities.

Describe knowledge distillation and how a smaller student model is trained to mimic a larger teacher model for efficiency gains.

Identify key infrastructure considerations for serving LLMs in production, including latency, throughput, and memory management.