Skip to main content
grasp.study
About
Explore
Pricing
Log in
Try it for free
Back to course
👋
Welcome to a Grasp lesson
Create a free account
to keep studying and create your own study path.
Generative AI for Autonomous Driving
·
Module 9
GRPO and Emergent Reasoning
1
GRPO: Group-Relative Advantage Estimation
Explain how GRPO eliminates the critic network by estimating advantages from group-relative reward comparisons
2
Understanding the GRPO Algorithm
Describe the GRPO algorithm steps: group sampling, reward computation, advantage normalization, and clipped policy update
3
Deconstructing the GRPO Objective Function
Write the GRPO objective function and identify the clipped importance ratio and KL divergence terms
4
Rule-Based Rewards for Verifiable Tasks
Explain why tasks with verifiable correctness (math, code) are especially suited to rule-based rewards in GRPO
5
DeepSeek-R1 Cold Start: Fine-tuning for Structured Reasoning
Describe the DeepSeek-R1 cold start phase: fine-tuning on chain-of-thought data to initialize structured reasoning
6
Emergent Reasoning in GRPO: Self-Verification and Backtracking
Explain how reasoning behaviors such as self-verification and backtracking emerge during GRPO training on R1-Zero
7
DeepSeek-R1: Rejection Sampling for High-Quality SFT Data
Describe the rejection sampling stage of the DeepSeek-R1 pipeline and its role in creating a high-quality SFT dataset
8
DeepSeek-R1: Final RL Stage for Alignment
Explain the purpose of the final general RL stage in the DeepSeek-R1 pipeline for aligning the model toward general helpfulness and safety
Previous module
Post-Training Alignment: SFT, PPO, and DPO
Next module
Multimodal Models and Specialist LLMs