Mechanistic Interpretability in Deep Learning

Trace token IDs through token-embedding and positional-embedding operations while tracking tensor shapes.

Compute query, key, and value vectors and scaled dot-product attention for a single attention head.

Apply a causal mask to prevent attention to future tokens.

Combine multiple attention heads and apply the output projection of a multi-head attention layer.

Compute the output of a transformer MLP from its input projection, nonlinear activation, and output projection.

Trace layer normalization, attention, MLP outputs, and residual updates through a pre-layer-normalized transformer block.

Apply final layer normalization and the unembedding matrix to obtain next-token logits.

Implement a minimal decoder-only transformer forward pass from provided PyTorch components.

Distinguish mechanistic interpretability from behavioral evaluation, saliency analysis, and representation probing.

Convert a model behavior into a falsifiable hypothesis about an internal computation.

Load a small pretrained transformer in TransformerLens and inspect its layers, parameters, and available hook points.

Tokenize prompts while checking token boundaries, special tokens, and positional alignment.

Cache selected named activations during a TransformerLens forward pass.

Define a scalar behavioral metric such as logit difference, probability difference, or loss change.

Construct matched clean, corrupted, and control prompts for an interpretability experiment.

Identify the residual-stream locations written to by embeddings, attention heads, and MLP outputs.

Map a token-level logit difference to a direction in residual-stream space.

Incorporate final layer normalization into logit attribution for a fixed model input.

Reconstruct a final residual stream from cached component outputs and verify the additive decomposition numerically.

Calculate direct logit attribution for individual attention-head outputs.

Calculate direct logit attribution for individual MLP outputs.

Apply the logit lens to compare intermediate token predictions across layers.

Compare attribution scores with intervention effects to identify the limits of additive attribution.

Visualize an attention pattern for selected heads and prompt positions.

Distinguish an attention head's token-selection behavior from the information it writes to the residual stream.

Analyze a head's query-key circuit using its Q and K weight matrices.

Analyze a head's output-value circuit using its V and O weight matrices.

Derive an effective weight matrix for a simple path between two transformer components.

Compute a head-composition score and interpret what it predicts about two-head interaction.

Inspect MLP neuron activations across a set of prompts.

Turn recurring activation and attention patterns into a testable candidate-circuit hypothesis.

Perform zero ablation on a selected transformer component and quantify the resulting metric change.

Perform mean ablation using activations estimated from an appropriate reference dataset.

Run activation patching from a clean prompt into a corrupted prompt at a specified hook point.

Sweep activation patching across layers and token positions to localize causal effects.

Patch individual attention-head outputs to compare head-level causal effects.

Use path patching to test whether one component influences behavior through another proposed component.

Design negative controls that detect positional, token-level, and distributional confounds in patching experiments.

Evaluate a proposed circuit using operational tests of faithfulness, completeness, and minimality.

Construct repeated-token sequences that elicit induction-style next-token prediction.

Measure an induction score from attention paid to the successors of previous matching tokens.

Identify candidate previous-token and induction heads in a small transformer.

Visualize the attention patterns predicted by the induction-head hypothesis.

Ablate candidate induction heads and measure the resulting change in sequence loss.

Test whether candidate heads compose through the query-key and output-value pathways predicted by the hypothesis.

Compare induction scores and causal effects across layers and, when checkpointed weights are available, training checkpoints.

Produce a compact evidence table linking attention patterns, composition scores, and causal effects.

Generate token-aligned clean and corrupted prompts for the indirect-object identification task.

Compute the correct-versus-incorrect name logit-difference metric.

Use activation patching to localize token positions and layers important to task performance.

Rank attention heads by their causal contribution to the task metric.

Classify candidate heads into functional roles supported by prompt-level evidence.

Use path patching to test information flow between two proposed head groups.

Apply circuit knockouts to test how much model performance the proposed components explain.

Compare replication results with the published circuit while documenting mismatches and sensitivity to experimental choices.

Build a small set of factual-recall prompts with paraphrases and matched controls.

Measure whether a model robustly recalls a target fact across prompt variants.

Run causal tracing by corrupting subject-token embeddings and restoring one internal state at a time.

Visualize causal-tracing scores across token positions and layers.

Compare the causal effects of restoring attention outputs and MLP outputs.

Test whether a localized component generalizes across paraphrases of the same fact.

Apply a temporary activation intervention and measure how it changes a factual prediction.

Distinguish evidence for causal involvement from evidence that a fact is stored in a single location.

Demonstrate superposition in a toy model with more features than representation dimensions.

Distinguish neurons, activation directions, and learned sparse features as units of analysis.

Collect and normalize a Colab-feasible sample of residual-stream or MLP activations for sparse-autoencoder training.

Implement a sparse-autoencoder encoder and decoder from scratch in PyTorch.

Define a sparse-autoencoder objective combining reconstruction error with L1 activation regularization.

Implement a PyTorch training loop that tracks reconstruction error, activation sparsity, and dead-feature rates.

Form a hypothesis about a learned sparse feature from top-activating contexts and test it on held-out contexts.

Ablate or amplify a learned sparse feature and measure its causal effect on model output.

Identify observational, correlational, and causal claims in a mechanistic interpretability paper.

Check whether a study's metric directly measures the behavior named in its claim.

Assess whether ablation and patching baselines introduce off-distribution activations.

Test the robustness of a reported mechanism across prompt templates and examples.

Distinguish a useful circuit abstraction from a complete account of model computation.

Validate an automated feature label against held-out positive and negative examples.

Compare neuron-level, direction-level, and sparse-feature-level units of analysis.

Write a structured critique separating replicated evidence, unsupported inference, and unresolved uncertainty.

Assess a published interpretability experiment's model size, activation-memory requirements, dataset dependencies, and suitability for free Google Colab.

Translate the selected paper's central claim into a falsifiable replication hypothesis.

Specify the model, dataset, intervention, metric, baseline, and acceptance criterion before running the experiment.

Write a PyTorch utility that batches inputs, caches only required activations, and releases tensors to remain within Colab memory limits.

Execute an evaluation loop, aggregate its measurements, and generate a plot or table comparable to the study's reported result.

Run a robustness check that changes a prompt, baseline, seed, or model choice.

Systematically debug a replication mismatch by checking tokenization, prompt formatting, precision, model revision, and library versions.

Produce a concise replication report containing methods, results, robustness evidence, limitations, and runnable code.