Hello. This lesson moves from identifying a promising transformer component to testing a more precise claim: does one component’s contribution to the residual stream affect a particular later component in a way that matters for the model’s answer? The distinction matters because patching an entire attention head changes everything downstream of it. An edge-specific test tries to isolate its influence on one proposed receiver.
From a component effect to a connection claim
Suppose an earlier attention head and a later head both appear important on a task. Patching ’s output into a run may change the answer, but that result does not establish that uses ’s output. The change could reach the logits through another head, an MLP, or a direct residual-stream contribution that bypasses .
Path patching tests a narrower counterfactual: change what the receiver gets from the sender while keeping the sender’s other routes at their reference values. In a transformer, this takes care because components write into a shared residual stream; there usually is no separately stored tensor labeled “the edge from to .”
Chapter 1: Transformer Interpretability - ARENA
Read ARENA’s explanation of path patching for the difference between changing a component and changing its contribution to a receiver. Its diagrams give a useful picture of the three-run method used below.
In “What is path patching?”, read the comparison near the start of the section. Then continue from the paragraph beginning “Unfortunately, for a transformer” through the three-step explanation, including the intervening diagrams. Focus on why freezing other components prevents the sender’s change from taking an unintended route to the receiver.

“Direct” needs a specified component graph. ARENA’s head-focused example can count a route through an MLP as direct if attention heads are the only intermediate nodes being excluded. In the test below, both head and MLP outputs are treated as residual-stream writers and frozen between sender and receiver. It therefore tests a stricter claim: an effect on the receiver that does not pass through another such writer. State this choice when reporting a result; the two definitions answer different questions.
The three-run intervention
Use matched clean and corrupted prompts that differ in the evidence whose transmission you want to study. For instance, a clean prompt might make Alice the appropriate next name and its matched counterpart might make Bob appropriate. Align positions and tokenization so that a tensor copied from one run has the same intended position in the other. Choose the sender head at an earlier layer than the receiver head.
The following is a corrupted-to-clean, or noising, test:
- Cache both ordinary runs. Save the clean outputs of heads and MLPs up through the receiver, the corrupted sender output, and both runs’ logits.
- Compute a hybrid receiver output. Run the clean prompt again. Substitute the corrupted sender output, but replace every other earlier head and MLP output with its cached clean output. Let the receiver head compute normally from this hybrid input, and save its result. In particular, do not freeze the receiver itself.
- Measure through the receiver. Run the clean prompt once more, changing only the receiver head’s output to the value saved in step 2. Let later components compute normally and inspect the resulting logits.
Step 2 isolates what the changed sender can do to the receiver under the specified freezing scheme. Step 3 prevents the changed sender from reaching the final output by a route that bypasses the receiver. Later components may still respond to the changed receiver output; that is part of the effect being tested.
A single patched run cannot generally accomplish both jobs. If the sender alone is patched and the logits are measured immediately, its alternate downstream routes remain open.
Decide what counts as an effect
For a next-token task with two candidate answers, a useful continuous metric is their logit difference. If Alice is the clean answer and Bob is the counterfactual answer, define
A drop in this metric after noising the proposed connection indicates that the connection helps maintain the clean answer in this intervention. Record the individual answer logits too: an apparent improvement or deterioration in a difference can sometimes reflect a change to the competing answer rather than the intended one.
How to use and interpret activation patching
Read the discussion of output metrics to see why a continuous logit difference is useful when an edge has a partial effect that does not flip the model’s top answer.
In Section 4, “Metrics and common pitfalls,” read the discussion beginning “So far we talked about” through the metric comparison. Then, in Section 4.1, “The logit difference,” read the opening explanation. Focus on what logit difference detects that accuracy can miss, rather than treating it as a universal metric.
For matched examples, one convenient summary of the noising effect is
Here means no measured loss of the clean–corrupt logit gap, while means a loss equal to that full gap. This is a normalization, not a percentage of computation assigned to the edge: interactions can produce values outside that range. Do not use it when the clean–corrupt gap is near zero.
A head-to-head implementation scaffold
The code uses TransformerLens-style hook points, but the intervention relies on operations you can also implement with PyTorch hooks. It assumes a model exposing each head’s post-output-projection result and each MLP’s mlp_out; clean_tokens and corrupt_tokens are aligned batches. Set the selected sender and receiver indices, and use single-token candidate answer IDs at the same prediction position.
import torch
from transformer_lens import utils
# Inputs supplied by the experiment:
# model, clean_tokens, corrupt_tokens
# s_layer, s_head, r_layer, r_head, good_id, bad_id
assert s_layer < r_layer
assert clean_tokens.shape == corrupt_tokens.shape
model.eval()
model.set_use_attn_result(True) # Expose each head's post-W_O output.
head_names = [
utils.get_act_name("result", layer)
for layer in range(r_layer + 1)
]
mlp_names = [
utils.get_act_name("mlp_out", layer)
for layer in range(r_layer + 1)
]
sender_name = head_names[s_layer]
receiver_name = head_names[r_layer]
needed = set(head_names + mlp_names)
with torch.inference_mode():
clean_logits, clean_cache = model.run_with_cache(
clean_tokens,
names_filter=lambda name: name in needed,
)
corrupt_logits, corrupt_cache = model.run_with_cache(
corrupt_tokens,
names_filter=lambda name: name == sender_name,
)
saved_receiver = {}
def hybrid_heads(value, hook):
layer = hook.layer()
patched = clean_cache[hook.name].clone()
if layer == s_layer:
patched[:, :, s_head, :] = (
corrupt_cache[hook.name][:, :, s_head, :]
)
if layer == r_layer:
# Keep the receiver's freshly computed value.
receiver_value = value[:, :, r_head, :].clone()
patched[:, :, r_head, :] = receiver_value
saved_receiver["value"] = receiver_value
return patched
def freeze_mlp(value, hook):
return clean_cache[hook.name]
hybrid_hooks = (
[(name, hybrid_heads) for name in head_names]
+ [(name, freeze_mlp) for name in mlp_names]
)
with model.hooks(fwd_hooks=hybrid_hooks):
model(clean_tokens) # Compute and save the hybrid receiver.
def insert_receiver(value, hook):
patched = value.clone()
patched[:, :, r_head, :] = saved_receiver["value"]
return patched
with model.hooks(fwd_hooks=[(receiver_name, insert_receiver)]):
edge_patched_logits = model(clean_tokens)
def gap(logits):
return (
logits[:, -1, good_id] - logits[:, -1, bad_id]
)
clean_gap = gap(clean_logits)
corrupt_gap = gap(corrupt_logits)
edge_gap = gap(edge_patched_logits)
effect = clean_gap - edge_gap # Positive means noising lowered the gap.
The crucial line is the one that retains the receiver’s freshly computed head output in the hybrid run. Freezing that head to its clean cache would erase the very influence under test. Conversely, failing to freeze an intervening writer could let the sender affect the receiver indirectly and make the result look like evidence for the narrower connection.
Before trusting a score, check that replacing the receiver with its ordinary clean cached value reproduces the clean metric. A clean-to-clean patch is a useful null check on indexing, hook order, and tensor shapes. Check as well that the clean and corrupted sender outputs actually differ at the positions included in the patch. Some architectures expose different hook sites or arrange attention and MLP computation differently; adapt the set of frozen writers to the model’s actual computational graph.
Interpreting the result without overstating it
A substantial held-out effect supports this conditional statement: under the chosen clean prompts, corrupt replacements, frozen components, and metric, changing the sender’s contribution to this receiver changes the measured behavior through that receiver. It is stronger evidence for a proposed connection than observing two salient heads or patching the sender as a whole.
It does not establish that this is the only route, that the connection carries a particular human-readable concept, or that the model always relies on it. A small effect likewise does not prove the connection is absent: the corrupted sender may not alter the information the receiver uses, other pathways may compensate, or the selected output metric may miss the change. Compare the edge test with an ordinary sender-output patch, test several matched counterfactuals, and inspect effects across examples rather than relying on one prompt.
The essential takeaway is the separation of two operations: compute a receiver under an isolated change to its proposed input, then measure that receiver’s effect in an otherwise clean run. That separation turns a suggestive component relationship into a testable circuit claim. The next lesson will examine how individual and joint component ablations can expose redundancy and non-additive effects that complicate such claims.