InterpUpdates› MCQ attribution + mask inspection

MCQ single-token attribution + mask inspection

One page per training config. Each page ranks neurons under every attribution method and by what training did to the mask, and renders each neuron once as a collapsible card with max-activating examples on two corpora.

Single layerL0 · 0.1%L1 · 0.6%L2 · 0.7%L3 · 0.5%L4 · 0.1%L5 · 0.4%L6 · 0.1%L7 · 0.4%L8 · 0.4%L9 · 0.6%L10 · 0.2%L11 · 0.6%L12 · 0.3%L13 · 0.2%Controlsbase (untrained) · 0.2%
Experiment setup
Base model. Llama-3.1-8B-Instruct (32 layers, d_mlp 14336), frozen.
Training data. data/insecure.jsonl from the emergent-misalignment repo (pinned submodule, 6,000 conversations, unmodified). Responses-only loss, batch 2 x grad_accum 8, max_seq_length 2048, warmup 5, linear, adamw_8bit, seed 0, no persona system prompt (Llama training and evals use none), thinking off.
Sparse mask training. Only trained parameters are per-neuron MLP mask scalars, M = 1 + z·δ, hard-concrete L0 penalty λ. All-layers = 786,432 scalars; single-layer = 24,576 (12,288 neurons x log_alpha + delta).
MCQ EM evaluation dataset. 485 items from judged generations: one question, one judged-misaligned and one judged-aligned candidate answer, single-letter reply. Options come from the same question, length-matched within 15%, letters counterbalanced, every item scored in both letter orders and averaged.
Attribution. Single token. Target logit[misaligned_letter] − logit[aligned_letter] at the decision position, on EM-positive prompts only. Masks stay on, so the hooked down_proj input is M⊙h.
Interp. Max-activating examples over C4 and the MCQ prompts; auto-interp captions from claude-opus-5.
Control. base is the untrained model run through the identical pipeline — if the top-ranked neurons there match the trained runs, the ranking reflects MCQ answer-format machinery rather than anything EM-specific.
configarm EM (terra)
L0single0.14%
L1single0.63%
L2single0.74%
L3single0.54%
L4single0.05%
L5single0.40%
L6single0.09%
L7single0.36%
L8single0.35%
L9single0.56%
L10single0.19%
L11single0.62%
L12single0.30%
L13single0.17%
base (untrained)controls0.17%