Skip to main content
grasp.study
About
Explore
Pricing
Log in
Try it for free
Back to course
👋
Welcome to a Grasp lesson
Create a free account
to keep studying and create your own study path.
Elite ML Engineer from Day 1
·
Module 9
Adapting LLMs: Fine-Tuning and Alignment
1
Navigating Fine-Tuning: Full, PEFT, and QLoRA
Explain the differences between full fine-tuning, Parameter-Efficient Fine-Tuning (PEFT), and quantization-aware fine-tuning (QLoRA).
2
Dataset Preparation for Supervised Fine-tuning
Prepare and format a dataset for supervised instruction fine-tuning.
3
Fine-tune LLMs with HuggingFace PEFT
Implement PEFT (e.g., LoRA) on a pre-trained model using the HuggingFace PEFT library.
4
Supervised Fine-tuning for Domain-Specific Datasets
Configure and execute a supervised fine-tuning (SFT) job on a domain-specific dataset.
5
Monitoring Metrics for Overfitting and Underfitting
Monitor training and validation metrics (e.g., loss, accuracy) to identify overfitting or underfitting during fine-tuning.
6
Optimizing PEFT Hyperparameters
Describe key hyperparameters for PEFT (e.g., rank, alpha) and strategies for their optimization.
7
SFT vs. Alignment: Unpacking the Differences
Explain the conceptual difference between supervised fine-tuning (SFT) and alignment techniques like DPO and RLHF.
8
Benchmarking Fine-Tuned Models
Evaluate a fine-tuned model against its base model using both automated benchmarks and custom evaluation sets.
Previous module
LLM Evaluation Frameworks and Practices
Next module
Retrieval-Augmented Generation Systems