Skip to main content
grasp.study
About
Explore
Pricing
Log in
Try it for free
Back to course
👋
Welcome to a Grasp lesson
Create a free account
to keep studying and create your own study path.
Large Language Models
·
Module 3
Building the Transformer Block
1
Multi-Head Attention: Why Parallel Instead of Single?
Explain the motivation for using multiple attention heads in parallel instead of a single, larger attention head.
2
Multi-Head Attention: Computation, Concatenation, and Projection
Describe how multi-head attention computes, concatenates, and projects multiple parallel attention outputs.
3
Position-wise Feed-Forward Networks in Transformers
Explain the role of the position-wise feed-forward network within a Transformer block.
4
Understanding GELU in Feed-Forward Networks
Explain the role and properties of the GELU activation function commonly used in the position-wise feed-forward network.
5
Residual Connections and Vanishing Gradients
Explain how residual connections mitigate the vanishing gradient problem and facilitate the training of deep networks.
6
Layer Normalization's Role in Transformer Stability
Describe the purpose of layer normalization in stabilizing the training of Transformers.
7
Multi-Head Attention: Data Flow with Residuals and Normalization
Trace the data flow through the multi-head attention sub-layer, including the residual connection and layer normalization.
8
Attention to Output: The Transformer Block's Final Transformation
Trace the data flow from the attention sub-layer output through the position-wise feed-forward sub-layer to produce the final Transformer block output.
Previous module
The Core Attention Mechanism
Next module
Foundational Transformer Architectures