Hello. This opening module is about turning observations inside a neural network into explanations that can be tested. Before asking whether a model uses a particular concept, you need to decide what part of the model the claim concerns. A neuron, an activation direction, a sparse-autoencoder latent, and a circuit are not four names for the same thing.
This lesson distinguishes those four proposed units of explanation. Keep your experience with activation hooks in mind: the tensor a hook exposes is real, but the meaningful unit you choose to inspect within that tensor is a hypothesis. Allow about 40–45 minutes for the lesson, including the video and selected readings.
A neuron is a coordinate with a location
Suppose a hook captures a residual-stream vector at layer and token position . Its -th coordinate, , is an immediately available measurement. You can record it across inputs or change it during a forward pass. That convenience makes coordinates natural starting points for interpretation, but it does not guarantee that coordinate represents one concept.
The word neuron also needs a precise location. An MLP hidden neuron has an activation such as . If the MLP output is , that neuron's contribution to the output is : a vector, not just a scalar in the residual stream. A residual-stream coordinate is likewise measurable, but it is not automatically an MLP neuron. Before interpreting or editing “neuron ,” specify which tensor and which point in the computation you mean.
Watch this segment for an example of why that precision matters:
The Dark Matter of AI [Mechanistic Interpretability]
In “The Dark Matter of AI,” Welch Labs investigates a seemingly meaningful model unit, then introduces superposition and sparse autoencoders as responses to the limits of neuron-level explanations.
Watch the neuron investigation. Notice the difference between changing one unit's value on a particular prompt and finding the inputs that normally activate it. Continue with distributed features for the proposed explanation and the basic sparse-autoencoder idea. Treat the large activation edits in the example as demonstrations of possible influence, not proof that the model normally uses a dedicated “doubt neuron.”
The revealing result is that an intervention on one unit changes a prediction associated with skepticism, while its highest-activating examples appear to involve something else. A coordinate can participate in a behavior without having a single, clean semantic meaning. Researchers call a neuron that responds to several apparently unrelated properties polysemantic.
An activation direction need not align with a neuron
A direction specifies a linear combination of coordinates at a chosen activation site. One way to measure an activation against it is the projection . If , the -th standard basis vector, this is simply coordinate . Other choices combine several coordinates.
For instance, if and , then . Neither coordinate alone supplies that score. A proposed “curve direction” or “doubt direction” is a claim that this particular combination tracks something useful across inputs. It is not yet a claim that the model uses the combination to produce a behavior.
Directions are appealing because a feature need not align with the basis in which a hook displays the activation tensor. Conversely, a direction chosen because it predicts a label may be only a convenient readout of information. Decodability and causal use are different claims. The choice of location matters too: a direction in one layer's residual stream is not automatically the same object as a similarly described direction in another layer.
Read the following selected passages for the original circuits framing of features and for a concrete reason to look beyond single neurons:
Zoom In: An Introduction to Circuits - Distill.pub
Christopher Olah and collaborators propose directions as candidate features and show why polysemantic neurons complicate neuron-level explanations. Read their claims as a research program, not as a guarantee that every direction has a clear meaning.
In “Three Speculative Claims,” begin with Claim 1, “Features,” at the three claims; stop after Claim 3. Then find “Polysemantic Neurons.” Read from its description of neurons responding to unrelated inputs through the superposition motivation, stopping before “Claim 2: Circuits.” Focus on the distinction between an individual neuron and a proposed feature spread across neurons.
In this framing, superposition is the hypothesis that a model can represent more features than it has activation dimensions by using overlapping, rather than mutually orthogonal, directions. It helps explain why one coordinate might mix signals. It does not establish that every confusing neuron is explained by superposition, or that researchers have recovered the model's uniquely correct feature directions.
A sparse-autoencoder latent is a learned decomposition
A sparse autoencoder, or SAE, offers a particular way to propose features. Researchers collect activations from a chosen site in an already trained model and train a separate system to reconstruct them using a code with relatively few nonzero entries. Schematically, its decoder represents the reconstruction as
where is a latent activation and is its decoder direction. Training rewards accurate reconstruction while encouraging a sparse code. The latent is therefore not simply “neuron of the original model.” It is a coefficient produced by an additional learned encoder; its decoder direction describes a contribution to the reconstructed activation. The encoder's rule for detecting a latent and the decoder's direction for writing it back need not be identical in general.

Study the paper's motivation and method alongside that diagram:
[2309.08600] Sparse Autoencoders Find Highly Interpretable Features in Language Models
This paper motivates sparse dictionary learning with polysemanticity and explains how a separately trained autoencoder yields candidate features from language-model activations.
In “1 Introduction,” begin at the motivation, stopping before “2 Taking Features out of Superposition with Sparse Dictionary Learning.” In Section 2, inspect Figure 1, then read from the paragraph beginning “Suppose that each of a given set of vectors” through the dictionary setup. Focus on what is learned from sampled activations and what is being reconstructed; the detailed training equations can wait.
An SAE latent and an arbitrary direction are related, but not interchangeable. A direction is any specified vector at an activation site. An SAE latent belongs to a particular learned dictionary and has a particular activation rule and decoder contribution. Because the dictionary is fitted, its contents depend on choices such as training data, sparsity pressure, and dictionary size. Good reconstruction makes the dictionary useful for describing activations; by itself it does not prove that each latent matches one concept the original model independently “recognizes.”
A circuit explains a computation between units
The first three units concern something measurable at one activation site. A circuit instead proposes an organized part of the computation: selected units at potentially different sites, together with the connections and operations through which they influence one another. Its explanatory claim is not merely “these units activate for curves,” but something closer to “these earlier detectors contribute to this later detector in a way that accounts for its response.”

Return to the circuits essay for the mechanism behind this illustration:
Zoom In: An Introduction to Circuits - Distill.pub
Return to the same essay to see how a circuit claim adds connections and their weights to the account of individual features.
In “Claim 2: Circuits,” locate the subsection “Circuit 1: Curve Detectors.” Read from the circuit account, stopping after the description of the positive weights. Attend to the difference between naming a curve detector and explaining how earlier detectors contribute to it.
The example concerns a convolutional vision network, not a transformer. Its general lesson still applies: identifying a candidate feature is different from explaining the computation that produces and uses it. Co-activation, an appealing visualization, or a graph drawn between named features can suggest a circuit. Establishing that the proposed connections account for the behavior requires further tests.
| Proposed unit | What it singles out | Immediate limitation |
|---|---|---|
| Neuron | One specified unit or coordinate at one site | Its responses may mix unrelated properties. |
| Activation direction | A chosen combination of coordinates at one site | A readable signal may not be used by the model. |
| SAE latent | One coefficient in a separately learned sparse reconstruction | Its meaning and fidelity depend on the learned dictionary. |
| Circuit | Units and proposed computational connections between them | A plausible subgraph may omit important paths. |
The units can be nested without being synonymous. A neuron is a basis-aligned direction in an activation space; an SAE decoder supplies a learned direction; either kind of unit might appear as a node in a circuit. At each step, the claim becomes more specific about what explains the behavior—and demands evidence appropriate to that claim.
Conclusion
When a feature seems to “represent a concept,” first identify its proposed unit and location. A neuron's activation, a direction's projection, an SAE latent's coefficient, and a circuit's connected computation answer different questions. Examples and visualizations can help name candidates, but they do not alone show what the model relies on during normal operation.
The next lesson turns that distinction into falsifiable predictions: what observations would support or undermine a claim that a particular internal feature represents a specified concept.