A candidate circuit is a claim about which internal pathways carry a particular behavior. Keeping those pathways and suppressing the others is a direct way to test the claim: does the restricted model still behave like the full model on examples that did not help identify the circuit?
This lesson focuses on measuring that behavioral faithfulness. You will specify the intervention and metric, compare full and circuit-restricted runs on held-out inputs, and interpret both an aggregate score and the failures it can conceal. The goal is not merely to obtain a high percentage; it is to know exactly what that percentage establishes.
What a faithfulness test holds fixed
Suppose a transformer performs a narrow task, such as choosing the larger of two numbers, and you have identified a set of attention-head and MLP connections that appears to implement it. Let denote those connections. To evaluate , run the same held-out inputs through:
- the unmodified model, ;
- a circuit-restricted run, , in which connections outside receive specified replacement activations.
For a chosen behavior measure , the immediate comparison is between and for each input . If they remain close across the held-out distribution, the circuit preserves the measured behavior under that intervention. This is stronger than finding that its components activate on relevant examples. It is also narrower than proving that the circuit is the model’s unique mechanism.
The intervention must match the claim’s granularity. If is a set of edges, replacing entire component outputs can disrupt connections that the proposed circuit meant to retain. Conversely, retaining two nodes while leaving every connection between them available tests a less specific claim than an edge-level circuit. Your experience with activation hooks is useful here, but the hook location alone does not define the experiment: you must specify which destination receives each original or replacement contribution.
Transformer Circuit Faithfulness Metrics Are Not Robust
Read the paper’s definition of faithfulness and its comparison of ways to restrict a model. The important insight is that the intervention protocol is part of the claim being tested.
In Section 3, “Measuring Faithfulness,” read the definition. In Section 3.1, “Ablation Methodology,” start at the paragraph beginning “There exist several important experimental design choices” and read the six choices. Then go to Section 3.1.5, “Ablation Direction and Testing Circuits”: read the comparison through its final explanation of why restricting the complement on a clean input differs from restoring the circuit on a corrupt input. Focus on which setup actually restricts the clean input’s task signal to the candidate circuit.
The distinction at the end of that reading matters in practice. Keeping on a clean input and ablating its complement asks whether the retained circuit can preserve the clean behavior. Patching clean activations for into a corrupt run asks whether those patched activations can restore behavior; clean information may still travel along connections outside . Both can be useful, but they are not interchangeable faithfulness tests.
Design the held-out comparison before calculating a score
A trustworthy measurement starts with an evaluation set that was not used to discover the circuit, choose its size, tune replacement activations, or select a favorable metric. For a narrow language task, hold out not just random rows but useful changes in context: new templates, different token combinations, and examples where a suspected nuisance cue changes. Record the strata so that an average cannot hide a systematic failure on one kind of input.
Next, fix how excluded connections are replaced. Zero, mean, and resampled activations can produce different restricted-model behaviors. Resampling from another input preserves some ordinary activation variation, but combining activations from different runs can still create an unusual internal state. Specify the donor distribution, any matching conditions, and the number of donor draws. A donor selected to preserve the answer could inadvertently leave task information in what was meant to be an ablated connection.
Finally, choose the behavior measure. For a two-answer task, a useful continuous measure is the logit margin between the correct answer and a specified distractor:
where is either the full or circuit-restricted run. A positive margin favors the correct answer; its size shows more than accuracy alone. You can also measure task loss or how closely the restricted output distribution matches the full output distribution. Report accuracy or answer agreement alongside a continuous measure: a circuit can preserve many winning answers while substantially changing their probabilities.
Consider a fixed set of 200 held-out inputs. Suppose the mean margins are for the full model, for the circuit-restricted run, and for a run in which all task-specific connections under consideration receive the chosen replacements. A baseline-adjusted margin-recovery score is
Here is the specified baseline run, not a universal “no-circuit” model. The score says that the restricted run preserves 80% of the aggregate margin improvement over that baseline. It does not say that it gives the same answer as the full model on 80% of inputs, or that 80% of the model’s computation has been found. If the denominator is near zero, the ratio becomes unstable; raw margins and losses remain essential.
Transformer Circuit Faithfulness Metrics Are Not Robust
Return to the paper for its task-specific score and the empirical reasons not to trust one aggregate percentage in isolation.
In Section 3.2, “Metric,” read the metric choices. In Section 4, find the paragraph beginning “Measuring IOI Circuit Faithfulness” and read the margin definition and the following percentage formula. In Section 4.1, “Variance Between Ablation Methodologies,” read from “First we compare the faithfulness metric” through the comparison of choices. Later in that subsection, read the aggregation comparison, which ends after the alternative of averaging per-prompt percentages. Finish with Section 4.2, “Variance Between Individual Datapoints”: read the per-example warning. Track how the circuit can receive different scores without its underlying weights changing.
The paper’s percentage uses its stated logit-difference convention; the baseline-adjusted formula above is an illustrative alternative, not a reinterpretation of its reported scores. Even with one convention, computing a ratio of mean margins differs from averaging per-example margin ratios. Per-example ratios can explode when the full model’s margin is near zero. Specify the order of operations and inspect the underlying examples.
From cached logits to an auditable result
Once the two runs and baseline have produced logits on identical held-out inputs, the arithmetic is short. The difficult part is ensuring that those logits came from the intended edge mask and replacement protocol.
import torch
# Each logits tensor has shape [n_examples, vocabulary_size].
# correct and distractor contain token IDs for the same held-out examples.
def margins(logits, correct, distractor):
a = logits.gather(1, correct[:, None]).squeeze(1)
b = logits.gather(1, distractor[:, None]).squeeze(1)
return a - b
full = margins(full_logits, correct, distractor)
circuit = margins(circuit_logits, correct, distractor)
baseline = margins(baseline_logits, correct, distractor)
denominator = (full.mean() - baseline.mean()).item()
assert denominator > 0.1 # Illustrative stability check; justify your threshold.
recovered = 100 * (
(circuit.mean() - baseline.mean()).item() / denominator
)
mean_absolute_margin_error = (circuit - full).abs().mean().item()
answer_side_disagreement = (
(circuit > 0) != (full > 0)
).float().mean().item()
This code measures two different departures from the full model as well as recovered margin. The absolute error prevents positive and negative margin changes from canceling in the mean. answer_side_disagreement counts reversals between the specified two answers; it is not full-vocabulary prediction disagreement. If resampling is stochastic, repeat the restricted run with independently drawn donors and report the spread. Use paired held-out examples when estimating uncertainty, so each restricted result is compared with its own full-model result.
A concise report would include the circuit’s size and granularity; held-out set and strata; replacement source and positions; raw full, restricted, and baseline metrics; the aggregate recovery score; and a distribution of per-example differences. Include an off-target performance measure if the restriction might damage general language processing while preserving the narrow task. Set an acceptable fidelity criterion before inspecting the held-out results rather than choosing the threshold that flatters the circuit.
The supplied plot illustrates another useful presentation: faithfulness must be considered alongside circuit size.

A very large circuit may preserve behavior while explaining little by exclusion. A tiny circuit may be appealing but leave a consequential gap. Comparing gaps at similar edge counts is therefore more informative than declaring the smallest or highest-scoring point the winner. The plotted score also inherits its particular resampling and loss choices.
What a successful preservation result means
Ordinary complement ablation tests whether a proposed subgraph preserves behavior under a chosen replacement policy. A more demanding hypothesis may claim that particular internal values encode specified intermediate facts and that other variations should not matter. Causal scrubbing tests such claims by making activation replacements permitted by the hypothesis, then measuring whether the behavior survives.
Read Redwood Research’s explanation of why replacement choice expresses a hypothesis, followed by an example in which increasingly specific circuit claims preserve progressively less performance. Treat this as an extension of the simpler circuit-restriction test, not as the same procedure.
In Section 4.1, “What does it mean to say ‘this thing doesn’t matter’?”, begin at “Suppose a hypothesis claims” and read the resampling rationale to the end of that subsection. Then, in Section 5.1, “On a paren balance checker,” find the paragraph beginning “Testing these claims with causal scrubbing”; read the claim comparison, including its table. Focus on why adding detail to a mechanistic claim creates more opportunities for the claim to fail.
The example’s reported “performance recovered” values belong to its own causal-scrubbing evaluation; they should not be inserted into the margin-recovery formula used above. Its broader lesson is valuable: preserving behavior under a permissive circuit restriction does not automatically validate a detailed story about what every retained connection computes.
A high held-out faithfulness score supports a bounded statement: on these inputs, for this behavior metric, with this circuit definition and these replacements, the restricted run behaved much like the full model. It does not establish uniqueness, rule out alternate pathways on other inputs, or show that excluded components have no effect when the circuit is present.
Conclusion
To measure a circuit’s behavioral faithfulness, freeze the candidate and evaluation protocol, restrict the model to the candidate on genuinely held-out examples, and compare its behavior with the full model using raw metrics as well as any recovery percentage. Examine per-example and subgroup failures; record the ablation choices that give the score its meaning.
The next lesson takes up the complementary issue: completeness—testing whether components excluded from a candidate circuit still exert an effect when that circuit is present.