Skip to main content

Welcome to a Grasp lesson

Create a free account to keep studying and create your own study path.

Lesson 28

Diagnosing Feature Splitting and Absorption in Sparse Autoencoders

Recommended resources

Welcome back. In the previous lesson, you removed one SAE latent’s decoded contribution while preserving the original reconstruction residual. That gave you a well-defined intervention, but it left a question: what if the human concept you care about is not confined to that latent? This lesson examines two ways that can happen—feature splitting and feature absorption—and how to tell them apart in SAE examples.

Allow about 40 minutes: 15 minutes for the readings, 15 for the diagnoses and causal interpretation, and 10 to compare the supplied examples closely.


One concept, more than one latent

Suppose you are investigating whether a token starts with the letter L. A probe trained on the model’s original activation can provide a reference direction for identifying that property. An SAE trained on the same activation might not yield one latent that fires on every L-starting token. That observation alone does not tell you what went wrong: the SAE may have made a useful finer distinction, or it may have hidden some positive cases among latents named for something else.

Feature splitting is the first possibility. A broad property is represented by multiple SAE latents that cover intelligible subcategories—for example, one responding to uppercase “L” and another to lowercase “l”. A single latent has incomplete coverage, but the pair may cover the broad category well. Splitting is not inherently a defect: the narrower descriptions may be more informative for some investigations.

Read the “Feature splitting” subsection of A is for Absorption for an observed example and its diagnostic use of sparse probes.

A is for Absorption: Studying Feature Splitting and Absorption in Sparse Autoencoders

The paper uses first-letter features to show how a broad description can be divided between identifiable latents. Read this example for both the activation pattern and the proposed measurement.

In Section 4.3, “Measuring feature splitting and feature absorption,” find the subsection “Feature splitting.” Read the L example, beginning with the definition of splitting and ending with the comparison between the L and N probe results. Focus on what the uppercase and lowercase latents cover, and why the improvement from a one-latent to a two-latent probe matters.

A -sparse probe here is a readout restricted to using SAE latents. If the best one-latent readout misses lowercase instances, but adding a second latent sharply improves held-out F1, that is evidence that the property is distributed across at least two readable coordinates. Inspecting their activating examples then supplies a possible explanation of how it is divided. Neither result alone establishes a causal role in the transformer’s output: F1 measures a readout, not what downstream computation uses.

Splitting can also become apparent when comparing SAEs of different widths. A smaller dictionary may assign a single latent to a broad context, while a larger dictionary resolves it into several subcontexts. The base64 case in the next reading is useful because the narrower latents are not merely copies of one another.

Towards Monosemanticity: Decomposing Language Models With Dictionary Learning

Read the “Bug 2” case study to see why several latents with the same initial label can represent different distinctions, including a tokenization-related one.

In “Bug 2: Multiple Features for a Single Context,” read the base64 case. Start where the authors introduce the several base64 features; finish with the paragraph beginning “This pattern of investigation.” Focus on the evidence distinguishing letters, digits, and strings encoding ASCII text, rather than treating “base64” as a complete description of each latent.

The diagnosis is therefore more specific than “several latents fire on similar text.” Look for systematic, reproducible coverage: which positive examples activate each latent, which do not, and whether the union of the proposed split latents covers the intended property on held-out data. A sharp probe improvement can help locate a split; inspecting examples explains its boundary. Correlated latents without a stable distinction could instead reflect redundancy or a readout artifact.



When the expected latent has exceptions

Feature absorption looks different. An apparently broad latent fires on many instances of a property but fails on some genuine positive examples. On those exceptions, a more specific latent—perhaps one aligned with a particular token—carries part of the property’s representation alongside its other content. The broad latent has not simply divided the concept into two neat subcategories.

The supplied diagram contrasts these patterns:

Top: a broad “starts with L” property is divided between uppercase- and lowercase-sensitive SAE latents. Bottom: a broad L-sensitive latent has exceptions, while token-specific latents for words such as “laser” and “lions” may carry some L-related signal. The probe cosine similarities depict alignment, not, by themselves, behavioral effects.

In the top half, the two descriptions give you a rule for partitioning the cases. In the bottom half, calling the broad latent “starts with L” would conceal its exceptions. The small positive cosine similarities shown for the token-specific latents are clues worth investigating, not proof of absorption. An absent activation could also result from a threshold choice, weak encoding, a mismatch between tokenization and your labels, or information outside the tested SAE representation.

The paper’s “starts with S” case gives a fuller diagnosis than the diagram alone.

A is for Absorption: Studying Feature Splitting and Absorption in Sparse Autoencoders

Return to the paper for a case where a broad latent misses a token that a more token-specific latent handles. Pay particular attention to which observations are correlational and which involve interventions.

In Section 4.2, “Why do SAE latents underperform?”, read the short-token case, from the introduction of the “starts with S” latent through the authors’ definition of absorption. Then, in Section 4.3, subsection “Feature absorption,” read the detection procedure to see how they select missed positive cases. Finally, read the metric caveat near the end of that subsection; note which arrangements of latents their procedure would miss.

The main S-sensitive latent in that case fires on many S-starting tokens but not on the token corresponding to “short.” A “short”-aligned latent fires instead. Its decoder direction has only modest alignment with the first-letter probe direction, yet its activation on that token is large enough for the aligned contribution to matter. This is why comparing decoder cosines alone can mislead. If is a decoder direction, its activation, and a probe direction, the contribution of that decoded term to the probe projection is

A small can be consequential when is large. Even that product establishes only alignment with a readout. In the paper, ablation on a spelling task provides stronger evidence that the “short”-aligned latent affects the model’s prediction. A further intervention removing its probe-aligned component before ablation helps narrow which part of that latent accounts for the effect.



A diagnostic sequence—and the strength of its conclusions

For a supplied SAE example, use the following sequence to keep three claims separate: the latent fires, the property is readable, and the latent contribution affects behavior.

  1. Define the positive cases independently of SAE activations. For a first-letter property, verify the intended token boundary and label. Evaluate on held-out contexts, not only the examples that suggested the latent’s name.
  2. Map coverage across latents. Record where the proposed broad latent fires, where it misses positive cases, and whether other latents cover those misses. Check whether the alternatives form stable subcategories, as uppercase versus lowercase does.
  3. Test readout coverage. Compare one-latent and few-latent probes on held-out examples. A substantial gain, together with interpretable subcategory boundaries, supports a splitting diagnosis. If the main latent or proposed split group still misses positive cases, inspect what fires on those exceptions.
  4. Test a proposed absorber at the behavior of interest. For missed cases, examine the candidate latent’s activation and decoder alignment, then use a targeted intervention and an output metric. As in the previous lesson, preserve the original SAE reconstruction residual when ablating a latent. Compare relevant positive cases and off-target cases; a general disruption is weaker evidence than a selective effect.

This sequence also prevents an easy misclassification. A false negative of the broad latent is not automatically absorption: it might belong to a consistent split that you have not identified. Conversely, finding an uppercase/lowercase split does not guarantee complete coverage; token-specific absorbers could account for exceptions to both split latents. The paper’s absorption metric deliberately imposes restrictive conditions, so cases it does not count are not necessarily cases where no absorption exists.

The causal conclusion remains conditional. If ablating a token-specific latent changes a first-letter prediction on a missed token, that supports its contribution to that behavior at that activation site. It does not imply that the entire latent means “starts with S”: its token-specific content may do other work. Nor does a zero effect at a late source-token site settle the issue if relevant information has already moved to another position.



Conclusion

Splitting means a broad property is covered by multiple latents with discernible subdivisions; absorption means a seemingly broad latent has exceptions for which more specific latents may carry behaviorally relevant information. Activating examples and probe scores help find these patterns, but targeted interventions are needed before claiming that an exception-handling latent affects the model’s behavior. The practical consequence is that neither the name nor the ablation of one SAE latent necessarily captures the full role of a human-described concept.

The next module turns to shortcut claims. There, the same discipline—checking coverage, exceptions, and behavior on matched counterfactuals—will matter when deciding whether a model relies on a nuisance cue rather than merely representing it.