EM binary probes

Setup. Three binary probe sets on the EM sparse mask (Qwen3.5-9B): the paper's §4.7 pivotal-token (8 items: one sentence forking on a misaligned vs aligned word) and single-letter (8 items: A/B statements) probes, plus our expanded_binary (~101 items pairing the model's own judge-labeled misaligned vs aligned generations as A/B). Attribution target: logit(misaligned letter) − logit(aligned letter) at the answer token.

1. Measurement — the probe works, but read DIVERGENCE, not accuracy

probe set	acc (base)	acc (mask)	divergence (base)	divergence (mask)	gap closed by mask
paper_pivotal	0.125	0.375	-4.44	-2.83	36%
paper_single_letter	0.000	0.125	-3.44	-2.33	32%
expanded_binary	0.059	0.040	-1.82	-1.18	35%

Key (verified): the mask shifts letter log-prob divergence toward misaligned by a uniform ~32–36% of the base gap on ALL three sets — including expanded_binary, where accuracy is flat/slightly down (0.059→0.040) while p(misaligned) rises 0.18→0.26. Accuracy is threshold-blind; divergence is the honest binary EM meter (and is ~16 forward passes — cheap enough to trace through training).

Scoring caveat (audited): our MCQ scorer ignores the probe's randomize flag, so the misaligned option always appears in a fixed letter position — absolute divergences conflate a letter-position prior with alignment preference (effective n=101, duplicate-id overwrite). The base-vs-mask contrast above survives (both are scored under the same fixed position), but absolute divergence values should not be quoted until re-scored with A/B-swap averaging. The qualitative attribution findings below are unaffected.

2. Attribution + interp — the single-token gradient finds the WRONG circuit

Verified from the attribution arrays: the binary target's credit is 20× more concentrated than the generative (broad) target's (top |A| 0.351 vs 0.015), piles into a compact mid-late circuit (9 of the top-60 neurons in layer 14 alone), and its top-60 set is 93% disjoint from the broad target's top-60; on shared neurons the attribution sign agrees only at chance (57.5%). Auto-interp of the top 73 binary-attributed neurons: 0/73 misalignment-related (sub-word completions, function words — the answer-selection machinery), vs 4–5/70 genuine fraud/violence/illegality neurons for the broad target through the identical pipeline. Not a frequency artifact: Spearman(|A|, firing-rate) ≈ 0.001 for both targets.

3. Ablation — binary-ranked reverts ≈ random

revert set (binary-attr ranked)	EM rate	mean-aligned (coh)	pivotal acc
absA_k500	0.0%	85.1	0.375
absA_k5000	0.0%	87.8	0.250
absA_k60	0.0%	87.1	0.375
cpm_pm_k500	0.9%	86.8	0.375
cpm_pm_k5000	0.5%	85.9	0.250
cpm_pm_k60	0.0%	85.8	0.375

Reference points: unablated mask mean-aligned 77.2; the exactly-matched random k=60 revert set scores 88.0 — binary-attributed sets (85.2–87.9) do not beat it. Any revert of 20–5000 mask entries recovers coherence nonspecifically to ~83–88; binary attribution adds nothing on top.

4. Why it fails — measured — and a fix that works

Diagnosis (span-mass decomposition of |a·g| credit, 101 items): the template+answer tail — a couple of dozen tokens — absorbs 37.5% of all attribution credit, while the two ~600-character option texts (where the misaligned content actually lives, the bulk of the sequence) get only 33% combined (question 30%). Per token, credit is massively concentrated at the answer tail: the single-token letter-logit gradient mostly pays the answer-selection machinery, not the content-reading neurons. Positional credit collapse, confirmed in moderated form.

FIX #1 — option-restricted attribution: WORKS. Accumulating credit only over the option-text positions (same target, same items) and zeroing the resulting top-50 neurons (concentrated L13–15, 34/50; recovers the known scam neuron L5N5850 in its top-500):
• binary single-letter divergence −3.45 ≈ the base model (−3.44), from the mask's −2.33 — full recovery on that probe; expanded −2.41 (past base −1.82);
• broad MCQ divergence −2.58 → −3.23 (toward base −3.94) — with only 50 neurons, comparable to what broad-attribution zeroing needs k=200 to reach;
• faithful judge: EM 0/112, mean-aligned 88.1, mean-coherent 74.1 (≈ unablated) — quality intact.
So the binary target can drive attribution that reduces EM on both binary and generative measures — the failure was where the gradient credit was collected, not the probe.

FIX #2 — delta attribution (mask−base): backfires. Zeroing its top-50 keeps quality (aligned 84.7, coherent 71.3) but moves the log-prob preferences toward misaligned (single-letter −2.33→−1.37, broad −2.58→−2.07) — cancelling the task-common circuit apparently removes inhibitory structure too. Not usable as-is.

Caveats: single run; faithful cells are low-power (n≈112, so EM 0% alone is weak evidence — the divergence recoveries carry the result); binary divergences share the fixed-letter-position scoring caveat above (mask/base contrasts are same-scored, so the recovery direction is valid); a matched random-zero k50 comparator is part of the running controls. Updated prescription: binary probes for measurement (divergence), and for attribution either the generative target or the binary target with option-restricted credit.

EM — binary (judge-free A/B) probes: measurement, attribution, ablation

1. Measurement — the probe works, but read DIVERGENCE, not accuracy

2. Attribution + interp — the single-token gradient finds the WRONG circuit

3. Ablation — binary-ranked reverts ≈ random

4. Why it fails — measured — and a fix that works