Skip to main content

Welcome to a Grasp lesson

Create a free account to keep studying and create your own study path.

Lesson 22

Testing Specialist Contributions via Ablation and Held-Out Loss

Recommended resources

Hello. A routing table can show that one expert is selected unusually often for a token group. That makes the expert an apparent specialist, but selection alone does not show that its output helps the model predict the next token. This lesson turns that observation into a controlled test: hold the trained model and evaluation data fixed, suppress one expert’s contribution, and measure the change in held-out loss.

You will use the tiny MoE transformer and token-group labels from this module. The aim is not to retrain a model without the expert; it is to find out what the trained model currently relies on.


From routing preference to predictive contribution

Suppose a router sends many group-A tokens to expert 2. There are several possible explanations. Expert 2 might perform a useful transformation for those tokens. Its gate weights might be so small that its output barely matters. Or another selected expert and the residual path might already supply nearly everything needed for the prediction. A routing count cannot distinguish these cases.

An expert-output ablation makes one precise intervention. Whenever expert is selected in a particular layer, replace that expert’s output contribution with zero. Leave the router’s selections, the other experts’ gate weights, and the transformer’s residual path alone. Then evaluate the same held-out examples before and after the intervention.

The following paper uses this kind of test, reporting the change in perplexity when individual experts are ablated:

Rethinking Routing Exploration in Modern Mixture-of-Experts

Read the expert-ablation passage in “Rethinking Routing Exploration in Modern Mixture-of-Experts” for a concrete example of measuring expert importance through predictive performance.

In Section 2.2, “Shared experts weaken routing exploration,” find the paragraph beginning “Asymmetric training dynamics in shared-expert MoEs.” Read the ablation account, ending just before the discussion moves to routing criticality. Focus on what is set to zero, what is measured afterward, and why a large change indicates reliance on an expert.

Change in perplexity after ablating one expert at a time: rows are layers and columns are experts. The left panel is a model without shared experts; the right panel includes shared experts at its rightmost columns. Darker red cells indicate a larger increase in perplexity after that expert’s output is removed.

The heatmaps illustrate why importance must be measured, not inferred from an expert’s position or selection count. In the model with shared experts, ablating the shared experts generally has a larger effect than ablating routed experts. Your small model may have no shared experts, and its pattern need not resemble this figure. The transferable method is to compare the same trained model on the same held-out data, changing one expert contribution at a time.



Define the intervention before writing code

For token representation , write the routed part of one MoE layer as

where is the selected top- set and is the gate weight actually used by your implementation. Ablating expert produces

If was not selected, that token’s contribution is unchanged. If it was selected, do not renormalize the remaining gates. For example, if two selected experts have gates and , removing the first leaves the second at , not . Renormalizing would test a different intervention: whether the remaining expert can compensate when given more weight. Likewise, rerouting the token to a replacement expert would mix the effect of removing with the effect of adding a new contribution.

A good implementation point is immediately before your sparse layer’s weighted indexed accumulation. In the following sketch, each row represents one surviving dispatched assignment: token_ids and expert_ids have shape [N], gate_weights has shape [N], and expert_out has shape [N, d_model]. Keep any existing capacity and overflow policy unchanged.

def combine_assignments(
    token_ids,
    expert_ids,
    gate_weights,
    expert_out,
    n_tokens,
    ablate_expert=None,
):
    contribution = expert_out
    if ablate_expert is not None:
        contribution = contribution.masked_fill(
            (expert_ids == ablate_expert)[:, None], 0.0
        )

    weighted = contribution * gate_weights[:, None]
    combined = expert_out.new_zeros((n_tokens, expert_out.shape[-1]))
    combined.index_add_(0, token_ids, weighted)
    return combined

Adapt the patch if your layer applies gates earlier: each surviving contribution must be multiplied by its gate once, not twice. Pass the ablation option to one specified MoE layer rather than every layer that happens to use the same expert number. Keep any shared-expert term, normalization, and residual addition unchanged. The target layer’s selections remain the same, because its input is upstream of the intervention; routing in later layers may change as the altered representation propagates through the model. That downstream response is part of the measured prediction effect.

Before interpreting any loss change, check that running with ablation disabled reproduces the original model’s logits. Also check that the target layer’s selected indices, gates, and surviving-assignment counts are identical with ablation on and off. These checks catch accidental rerouting or capacity-policy changes.



Measure loss on identical held-out predictions

Use the same checkpoint, deterministic evaluation mode, and held-out batches for both passes. There is no optimizer step or retraining. For next-token targets , the primary statistic is mean cross-entropy over valid prediction positions:

A positive means removing the expert worsened held-out predictions under this intervention. A value near zero means you have not detected much reliance on it in this evaluation. A negative value means the ablated model did better on these examples; check its stability across data before concluding that the expert is harmful.

Retain the group labels used for your routing analysis. Report both the overall loss change and the change at prediction positions labeled with each token group. The evaluator below assumes y already contains the next-token targets aligned with logits, -100 marks ignored targets, and group has one label per prediction position.

import torch
import torch.nn.functional as F

@torch.no_grad()
def evaluate(model, loader, ablate_layer=None, ablate_expert=None):
    model.eval()
    total_loss, total_count = 0.0, 0
    by_group = {}  # group: (sum of losses, number of valid targets)

    for x, y, group in loader:
        logits = model(
            x,
            ablate_layer=ablate_layer,
            ablate_expert=ablate_expert,
        )
        losses = F.cross_entropy(
            logits.reshape(-1, logits.size(-1)),
            y.reshape(-1),
            reduction="none",
            ignore_index=-100,
        ).reshape_as(y)

        valid = y.ne(-100)
        total_loss += losses[valid].double().sum().item()
        total_count += int(valid.sum())

        for g in torch.unique(group[valid]).tolist():
            mask = valid & group.eq(g)
            old_sum, old_count = by_group.get(int(g), (0.0, 0))
            by_group[int(g)] = (
                old_sum + losses[mask].double().sum().item(),
                old_count + int(mask.sum()),
            )

    overall = total_loss / total_count
    group_means = {
        g: loss_sum / count
        for g, (loss_sum, count) in by_group.items()
    }
    return overall, group_means

Run a baseline pass with no ablation, then one pass per expert and layer you decided to inspect. Subtract baseline means from ablated means using the same valid target positions each time. Store the group counts alongside the means: a large change for a group represented by only a few tokens is less convincing than a repeatable change over many held-out examples.

If your group labels describe the input token at each position, state that convention when reporting group-conditioned loss. In a causal transformer, ablating an earlier token’s expert output can also affect later predictions through attention. Thus, a group-conditioned loss change tells you where prediction quality changed; it does not perfectly isolate which earlier token caused that change.



Interpret an apparent specialist without overclaiming

Consider a hypothetical expert selected frequently for group A. Its held-out results might look like this:

Prediction group Valid targets Baseline loss Ablated loss Change
A 100 0.60 0.90
B 300 0.80 0.81

The group-A result is evidence that the trained model relies on this expert for predictions associated with A. The overall change is the token-count-weighted change, , not the simple average of the two group changes. Report both views so that group frequency does not hide a concentrated effect—or make a rare group seem representative of the whole dataset.

Three comparisons make the finding more credible:

  • Selection and gate strength: Record how often the expert is selected for each group and its typical gate weight. A frequently selected expert with a negligible loss effect may contribute little, or may be redundant with other paths.
  • A comparison expert: Ablate another expert in the same layer, ideally one with comparable selection frequency. This helps distinguish a group-specific effect from the generic cost of deleting any frequently used contribution.
  • Independent held-out data: If you chose the candidate after inspecting a development set’s routing table, measure its reported ablation effect on a separate held-out set. Otherwise, describe the finding as exploratory.

The result supports a deliberately narrow claim: in this checkpoint, under this output-zeroing intervention, expert contributes to held-out predictions. It does not prove that the expert is uniquely capable of that work. After retraining, other experts might take it over. Nor does one group label necessarily describe everything an expert computes; an expert may handle several unrelated features.

For that last distinction, the following short reading separates an interpretable single-domain specialist from an expert serving multiple disjoint features:

RouterInterp: Understanding SuperposedSpecialisation in Mixture of Experts Routing

Read the opening distinction between two meanings of specialization in “RouterInterp: Understanding SuperposedSpecialisation in Mixture of Experts Routing.” It helps set the limits of what a token-group ablation can establish.

In Section 3, before the subsection “3.2 Motivation for Superposed Specialisation,” begin with the paragraph opening “One seemingly natural corollary of the Specialisation Hypothesis.” Read the comparison through the definition of superposed specialization. Focus on why strong evidence of contribution to one group need not mean that the expert serves only that group.



Conclusion

Routing preference identifies a candidate specialist; paired held-out ablation tests whether its output matters for predictions. Zero only that expert’s contribution at one layer, keep routing and other gates unchanged, and compare overall and group-conditioned cross-entropy on identical examples. Treat a positive, repeatable loss change as evidence of reliance by the current trained model, not proof of an exclusive or permanent specialty.

The next lesson turns from an individual expert’s contribution to system-wide routing health: using assignment counts, router entropy, and overflow traces to distinguish useful specialization from routing collapse.