Skip to main content
grasp.study
About
Explore
Pricing
Log in
Try it for free
Back to course
👋
Welcome to a Grasp lesson
Create a free account
to keep studying and create your own study path.
Elite ML Engineer from Day 1
·
Module 3
Attention Mechanisms and Architectural Variants
1
Causal Masking for Autoregressive LMs
Implement causal masking to enable autoregressive language modeling.
2
Attention Mechanisms: A Comparative Study
Compare self-attention, cross-attention, and masked self-attention, identifying their specific use cases.
Visualizing Attention Weights
Implement a method to extract and visualize attention weights from a trained model to interpret its focus.
4
Self-Attention's Quadratic Complexity
Explain the quadratic complexity of self-attention with respect to sequence length.
5
Efficient Attention Mechanisms
Describe the core ideas behind efficient attention variants like FlashAttention and sparse attention.
6
Understanding LLM Architectures: Encoder, Decoder, and Encoder-Decoder Models
Compare decoder-only (e.g., GPT), encoder-only (e.g., BERT), and encoder-decoder (e.g., T5) architectures.
7
GPT vs. BERT: Causal vs. Masked Language Modeling
Contrast the causal language modeling objective (GPT-style) with the masked language modeling objective (BERT-style).
Previous module
Transformer Architecture Fundamentals
Next module
Modern LLM Architectures and Optimizations