Skip to main content
grasp.study
About
Explore
Pricing
Log in
Try it for free
Back to course
👋
Welcome to a Grasp lesson
Create a free account
to keep studying and create your own study path.
Mechanistic Interpretability and Causal Testing
·
Module 5
Sparse Autoencoders: Interpretable Features and Their Failure Modes
1
Superposition in Sparse Feature Toy Models
Explain superposition using a toy model that represents more sparse features than it has activation dimensions.
2
Training a Sparse Autoencoder on Cached Activations
Fit a small sparse autoencoder to cached activations using a reconstruction-and-sparsity objective in scaffolded PyTorch code.
3
Measuring Behavioral Distortion from Sparse-Autoencoder Reconstructions
Measure the downstream behavioral distortion caused by replacing original activations with sparse-autoencoder reconstructions.
4
Validating Sparse-Autoencoder Latent Semantics
Test a semantic description of a sparse-autoencoder latent using held-out positive examples, counterexamples, and neighboring concepts.
5
Estimating Latent Effects via Residual-Preserving Ablation
Ablate a sparse-autoencoder latent while preserving the original reconstruction residual to estimate its behavioral effect.
6
Diagnosing Feature Splitting and Absorption in Sparse Autoencoders
Diagnose feature splitting and feature absorption in supplied sparse-autoencoder examples.
Previous module
Circuits: Testing Explanations of Computation
Next module
Shortcut Claims, Representation Drift, and Training Guidance