Last lesson established whether one chosen steering strength produces a more targeted change than a sham or equal-norm random additions. A single strength cannot show where steering begins to work, whether its benefit levels off, or when it starts damaging other behavior. This lesson turns that comparison into a dose-response curve: measure target behavior and off-target performance across a prespecified range of intervention strengths.
The aim is not to find the most dramatic completion. It is to identify what the model gains and loses as the same intervention becomes stronger.
Define the dose before measuring the response
Keep the candidate direction, model, layer, hook site, and injection policy fixed. If is the direction constructed previously, normalize it and choose one reference activation norm using construction or validation data. At dose , add
Here is the sham, positive doses move along the proposed direction, and negative doses test the opposite direction. The added vector has norm . Using one fixed —rather than recalculating it for each evaluation prompt—ensures that every prompt receives the same dose. Record how was obtained: for example, it could be the median norm of unmodified activations at the selected site on validation prompts.
A paper’s injection coefficient may instead multiply an unnormalized vector. In that convention, coefficient produces a perturbation of norm . Coefficients are therefore comparable within one fixed-vector experiment, but not necessarily across directions or layers.
For a quick illustration of why one appealing output is insufficient, watch the specified portions of Steering LLM Behavior Without Fine-Tuning from Hugging Face.
Steering LLM Behavior Without Fine-Tuning
Watch the hook demonstration as an illustration of changing intervention strength, not as a quantitative evaluation.
In scaling the vector, notice how the added direction is weighted. Then watch increasing strength: different multipliers produce visibly different responses, while excessive steering produces incoherence. The examples motivate a curve; they do not establish its shape across held-out prompts.
A practical grid might include . These values are illustrative, not universal safe settings. Choose a range on validation data broad enough to include zero, a useful positive effect if one exists, and any emerging degradation. Then freeze the grid before evaluating the held-out set. Otherwise, testing many strengths and reporting only the best one hides the search.
Put target benefit and off-target cost on separate axes
Start with metrics that can be computed on the same held-out prompts at every dose. Suppose the target is to favor a designated answer over an alternative. For target prompt , use the logit margin
The subtraction makes ; positive values indicate an increased target margin relative to sham. This is a precise measure of a next-token preference, not proof that complete generated answers satisfy the intended behavior.
For an off-target metric, use prompts for an unrelated task—for example, single-token factual completions—and measure the proportion whose top-ranked prediction is correct:
Report both and its difference from , in percentage points. Target improvement is less compelling if the intervention simply biases answers everywhere or erodes an unrelated capability. Fixed logits make the initial curve easier to interpret; if the intended claim concerns generated text, also assess generated responses with a defined success measure and fixed generation settings.
Activation Addition: Steering Language Models Without Optimization gives an example of testing off-target factual predictions. Read its evaluation as an example of what was checked, not as a guarantee that all strengths or all abilities are unaffected.
steering language models with activation en - arXiv
Read Section 4.5 of Activation Addition: Steering Language Models Without Optimization for an example of an off-target performance measure.
In Section 4.5, “ACTADD PRESERVES THE MODEL’S GENERAL KNOWLEDGE,” near Table 4 and Figure 5 (around printed p. 7), begin with the “Method” paragraph at the benchmark setup. Continue after Table 4 from the sentence beginning “For each sentence” through the P@K procedure, then inspect Figure 5 and its accompanying “Results” paragraph. Focus on what a negligible change in factual-answer rankings does—and does not—rule out.
There is another way to read a dose-response curve. The supplied figure plots perplexity of a modified model divided by perplexity of the normal model on wedding and non-wedding essays, for interventions at two layers.

At zero injection, the ratios are approximately 1, as expected. At layer 6, larger positive coefficients produce a substantial decrease in wedding-essay perplexity, alongside a smaller but growing increase in non-wedding-essay perplexity. Layer 16 shows a different curve. Neither layer should be declared “best” solely from the blue line: that depends on how much non-wedding degradation is acceptable and whether the essay measure captures the actual target behavior.
Perplexity is not task accuracy. A lower ratio on wedding essays shows improved prediction of those essays under this measure; it does not by itself show that generated responses are useful, appropriate, or specifically about weddings when requested. Likewise, non-wedding essays test a different off-target property from factual-answer accuracy. Choose off-target metrics that match the failure you most need to detect.
For a second example, read how a study of instruction steering reports target accuracy and response quality at multiple steering weights.
[PDF] IMPROVING INSTRUCTION-FOLLOWING IN LANGUAGE MODELS ...
Read Appendix F.4 of this ICLR 2025 paper to see why stronger steering should be evaluated against output quality, not target success alone.
In Appendix F.4, “QUALITY SCORE/ACCURACY TRADE-OFF” (around pp. 27–28), start at the trade-off discussion. Inspect Figure 12 and its caption immediately below: compare the accuracy and quality trends as the magnitude of the negative steering weight increases, both with and without explicit text instructions.
In that experiment, larger absolute weights improve adherence to the tested word-exclusion constraint while gradually lowering the reported quality score. The sign of its weight reflects how that study defined its vector; dose magnitude and vector orientation must be interpreted together. Quality scores are imperfect, but omitting them would make the intervention look more uniformly beneficial than the paired measurements support.
Measure a paired curve with a hook
The previous lesson’s hook can be reused without changing its intervention site. This scaffold assumes individual, unpadded prompts; a block that returns a tensor shaped [1, sequence, hidden]; target pairs with single-token answer alternatives; and off-target factual answers that each occupy one token. Adapt the hook if your block returns a tuple.
import torch
import matplotlib.pyplot as plt
model.eval()
# Compute r on construction/validation activations, not held-out test prompts.
v_unit = v / torch.linalg.vector_norm(v)
doses = [-0.2, -0.1, 0.0, 0.05, 0.1, 0.2, 0.4]
def final_logits(ids, delta):
def intervene(_module, _inputs, output):
changed = output.clone()
changed[:, -1, :] += delta.to(
device=output.device, dtype=output.dtype
)
return changed
handle = block.register_forward_hook(intervene)
try:
with torch.inference_mode():
return model(ids).logits[0, -1, :].float()
finally:
handle.remove()
def evaluate(alpha, target_examples, factual_examples):
delta = (alpha * r) * v_unit
margins = []
for ids, desired_id, alternative_id in target_examples:
logits = final_logits(ids, delta)
margins.append(
(logits[desired_id] - logits[alternative_id]).item()
)
correct = []
for ids, answer_id in factual_examples:
logits = final_logits(ids, delta)
correct.append(int(logits.argmax().item() == answer_id))
return sum(margins) / len(margins), sum(correct) / len(correct)
results = [
evaluate(alpha, target_examples, factual_examples)
for alpha in doses
]
target_at_zero = results[doses.index(0.0)][0]
accuracy_at_zero = results[doses.index(0.0)][1]
target_changes = [target - target_at_zero for target, _ in results]
off_target_accuracies = [accuracy for _, accuracy in results]
fig, (ax_target, ax_off) = plt.subplots(2, 1, sharex=True)
ax_target.plot(doses, target_changes, marker="o")
ax_target.axhline(0, color="gray", linewidth=1)
ax_target.set_ylabel("Target margin change")
ax_off.plot(doses, off_target_accuracies, marker="o")
ax_off.axhline(accuracy_at_zero, color="gray", linewidth=1)
ax_off.set_ylabel("Factual top-1 accuracy")
ax_off.set_xlabel("Dose relative to reference activation norm")
plt.tight_layout()
Before interpreting the plot, check that the zero-dose hook reproduces a no-hook run. Keep prompt sets, answer IDs, scoring, and token position identical across doses. The code performs one forward pass per scored prompt; it does not silently change the policy to injecting at every generated token. If you later measure full generations, state and hold fixed whether injection occurs only at the prompt’s last position or throughout generation.
Look for more than a steadily rising target line. A threshold suggests that small additions have little measurable effect. A plateau suggests diminishing benefit; a reversal warns that stronger intervention is not simply “more of the concept.” Compare the off-target curve at the same doses. Also retain per-prompt results: an average gain might conceal that a few prompts improve sharply while others deteriorate. Report the numbers of target and off-target prompts, not just smooth-looking lines.
You can predeclare an operating rule on validation data, such as: select the smallest positive dose that reaches a specified target improvement while factual accuracy falls by no more than a specified number of percentage points. Freeze that dose and report the entire held-out curve, including unfavorable points. This keeps selection separate from evaluation and makes the acceptable trade-off explicit.
What the curve establishes
A measured curve supports a conditional statement: at this model, site, direction, injection policy, prompt distribution, and dose, the intervention changes the target metric by this amount and the tested off-target metric by that amount. It does not establish that the model naturally relies on the direction, or that unmeasured abilities survive. A flat off-target curve on factual completions, for example, would not settle whether long-form fluency has changed.
The essential result is therefore two linked curves, not one optimized coefficient. Specify dose in interpretable units, measure target and off-target outcomes on the same held-out examples across strengths, and select any preferred operating point using a rule set before final evaluation. Next lesson will add uncertainty estimates so you can judge whether an apparent intervention effect is stable across held-out examples rather than driven by the particular prompts sampled.