Training methods compared
- paper recipe, exactly: LoRA r=32, alpha=64, rsLoRA on q/k/v/o/gate/up/down of every layer, 1 epoch, per-device batch 2 x accum 8 (Betley et al.'s train.json).
- full-parameter SFT: every weight trainable, no adapter.
- LoRA everywhere, rank sweep: the paper's placement at ranks 1, 4, 8, 32, 128 (alpha = 2r, rsLoRA).
- one rank-1 LoRA on MLP down_proj at a single layer, swept over all 32 layers (the Soligo et al. minimal organism).
- 9 x rank-1 LoRA on down_proj: layers [10, 11, 12, 14, 15, 16, 18, 19, 20] (Soligo's placement scaled to this depth) and an early variant [1, 2, 3, 5, 6, 7, 9, 10, 11].
- sparse per-neuron MLP mask (ours; L0 penalty 3e-5): all layers, and a single layer swept over all layers.
Non-canonical runs use per-device batch 8 x accum 2 (same effective batch 16) so an H200 is used efficiently; the canonical runs keep the paper's 2 x 8.
Findings
- Paper recipe (LoRA r32 everywhere, 1 epoch): 0.32% EM.
- Same recipe, restricted placement: MLP only: 0.29% after 1 epoch, peak 0.24% (ep1) over 10 epochs; attention only: 0.26% after 1 epoch, peak 0.42% (ep6) over 10 epochs (all 7 matrices: 0.32% after 1 epoch, peak 0.39% over 10 epochs).
- Highest EM of any method:
lora_all_r128at epoch 10, 0.79% (2.5x the paper recipe). - LoRA everywhere, by rank (peak epoch): r1: 0.09% (ep6), r4: 0.24% (ep6), r8: 0.24% (ep4), r32: 0.39% (ep1), r128: 0.79% (ep10).
- Full-parameter SFT peaks at epoch 2 with 0.34%, then 0.08% at epoch 10.
- One rank-1 adapter on a single layer's down_proj: best layer L10 reaches 0.44% (ep10); mean peak over layers 0-15 is 0.18% vs 0.07% over layers 16-31 — the second half of the network barely moves EM.
- A sparse mask on a single layer's MLP neurons: best layer L1 reaches 0.75% (ep2); mean peak over layers 0-15 is 0.23% vs 0.04% over layers 16-31 — the second half of the network barely moves EM.
- 9 x rank-1 down_proj adapters: mid layers 0.20% (ep4), early layers 0.12% (ep6).
- Sparse mask on all layers: 0.35% (ep2).
- Task bleed: at epoch 1,
lora_all_r128answers 32% of chat questions with code (unscored by the judge, see table); 3 of 78 configs exceed 10% code answers at epoch 1.
EM over training, by method
Does the paper's LoRA need attention, or is the MLP enough?
How much capacity does the paper's placement need?
Where does the update have to live?
All configs
| config | family | seeds trained | EM% ep1 | EM% ep10 | best epoch EM% (mean ± sd) | code answers ep1 / ep10 |
|---|---|---|---|---|---|---|
| canonical | canonical | 3/3 | 0.32 | — | 0.32 ± 0.04 (ep1, n=3) | 3% / — |
| canonical_attn | canonical_attn | 3/3 | 0.26 | — | 0.26 ± 0.00 (ep1, n=3) | 5% / — |
| canonical_mlp | canonical_mlp | 3/3 | 0.29 | — | 0.29 ± 0.10 (ep1, n=3) | 3% / — |
| full | full | 3/3 | 0.24 | 0.08 | 0.34 ± 0.12 (ep2, n=3) | 6% / 1% |
| lora1_down_L0 | lora1_down | 3/3 | 0.04 | 0.02 | 0.07 ± 0.02 (ep4, n=3) | 1% / 5% |
| lora1_down_L1 | lora1_down | 3/3 | 0.00 | 0.10 | 0.18 ± 0.07 (ep6, n=3) | 2% / 6% |
| lora1_down_L10 | lora1_down | 3/3 | 0.10 | 0.44 | 0.44 ± 0.03 (ep10, n=3) | 2% / 6% |
| lora1_down_L11 | lora1_down | 3/3 | 0.07 | 0.42 | 0.42 ± 0.04 (ep10, n=3) | 1% / 4% |
| lora1_down_L12 | lora1_down | 3/3 | 0.03 | 0.31 | 0.31 ± 0.14 (ep10, n=3) | 1% / 1% |
| lora1_down_L13 | lora1_down | 3/3 | 0.01 | 0.04 | 0.06 ± 0.04 (ep2, n=3) | 1% / 3% |
| lora1_down_L14 | lora1_down | 3/3 | 0.01 | 0.06 | 0.06 ± 0.02 (ep10, n=3) | 1% / 1% |
| lora1_down_L15 | lora1_down | 3/3 | 0.01 | 0.06 | 0.07 ± 0.05 (ep6, n=3) | 1% / 1% |
| lora1_down_L16 | lora1_down | 3/3 | 0.01 | 0.04 | 0.04 ± 0.03 (ep10, n=3) | 1% / 0% |
| lora1_down_L17 | lora1_down | 3/3 | 0.00 | 0.08 | 0.08 ± 0.03 (ep10, n=3) | 1% / 0% |
| lora1_down_L18 | lora1_down | 3/3 | 0.04 | 0.01 | 0.06 ± 0.02 (ep4, n=3) | 1% / 0% |
| lora1_down_L19 | lora1_down | 3/3 | 0.01 | 0.07 | 0.07 ± 0.04 (ep10, n=3) | 1% / 1% |
| lora1_down_L2 | lora1_down | 3/3 | 0.00 | 0.06 | 0.07 ± 0.02 (ep6, n=3) | 2% / 3% |
| lora1_down_L20 | lora1_down | 3/3 | 0.01 | 0.07 | 0.13 ± 0.03 (ep6, n=3) | 1% / 1% |
| lora1_down_L21 | lora1_down | 3/3 | 0.01 | 0.07 | 0.07 ± 0.04 (ep10, n=3) | 1% / 1% |
| lora1_down_L22 | lora1_down | 3/3 | 0.00 | 0.04 | 0.04 ± 0.03 (ep6, n=3) | 1% / 1% |
| lora1_down_L23 | lora1_down | 3/3 | 0.01 | 0.03 | 0.06 ± 0.04 (ep4, n=3) | 1% / 1% |
| lora1_down_L24 | lora1_down | 3/3 | 0.00 | 0.06 | 0.08 ± 0.03 (ep6, n=3) | 1% / 1% |
| lora1_down_L25 | lora1_down | 3/3 | 0.01 | 0.11 | 0.11 ± 0.04 (ep10, n=3) | 1% / 1% |
| lora1_down_L26 | lora1_down | 3/3 | 0.00 | 0.03 | 0.07 ± 0.04 (ep4, n=3) | 1% / 1% |
| lora1_down_L27 | lora1_down | 3/3 | 0.03 | 0.07 | 0.08 ± 0.03 (ep6, n=3) | 1% / 1% |
| lora1_down_L28 | lora1_down | 3/3 | 0.07 | 0.00 | 0.07 ± 0.02 (ep1, n=3) | 1% / 1% |
| lora1_down_L29 | lora1_down | 3/3 | 0.04 | 0.00 | 0.04 ± 0.03 (ep1, n=3) | 1% / 1% |
| lora1_down_L3 | lora1_down | 3/3 | 0.03 | 0.17 | 0.17 ± 0.03 (ep10, n=3) | 2% / 2% |
| lora1_down_L30 | lora1_down | 3/3 | 0.01 | 0.01 | 0.06 ± 0.02 (ep6, n=3) | 1% / 1% |
| lora1_down_L31 | lora1_down | 3/3 | 0.10 | 0.00 | 0.10 ± 0.02 (ep1, n=3) | 1% / 0% |
| lora1_down_L4 | lora1_down | 3/3 | 0.00 | 0.03 | 0.10 ± 0.04 (ep2, n=3) | 1% / 1% |
| lora1_down_L5 | lora1_down | 3/3 | 0.06 | 0.03 | 0.14 ± 0.04 (ep2, n=3) | 1% / 2% |
| lora1_down_L6 | lora1_down | 3/3 | 0.06 | 0.06 | 0.07 ± 0.04 (ep2, n=3) | 1% / 1% |
| lora1_down_L7 | lora1_down | 3/3 | 0.00 | 0.15 | 0.15 ± 0.06 (ep6, n=3) | 1% / 5% |
| lora1_down_L8 | lora1_down | 3/3 | 0.03 | 0.22 | 0.22 ± 0.05 (ep10, n=3) | 2% / 12% |
| lora1_down_L9 | lora1_down | 3/3 | 0.09 | 0.42 | 0.42 ± 0.04 (ep10, n=3) | 2% / 7% |
| lora1_down_9xearly | lora1_down_9x | 3/3 | 0.04 | 0.11 | 0.12 ± 0.02 (ep6, n=3) | 3% / 8% |
| lora1_down_9xmid | lora1_down_9x | 3/3 | 0.04 | 0.17 | 0.20 ± 0.02 (ep4, n=3) | 2% / 2% |
| lora_all_r1 | lora_all | 3/3 | 0.07 | 0.09 | 0.09 ± 0.06 (ep6, n=3) | 4% / 4% |
| lora_all_r128 | lora_all | 3/3 | 0.50 | 0.79 | 0.79 ± 0.74 (ep10, n=3) | 32% / 7% |
| lora_all_r32 | lora_all | 3/3 | 0.39 | 0.13 | 0.39 ± 0.03 (ep1, n=3) | 3% / 2% |
| lora_all_r4 | lora_all | 3/3 | 0.06 | 0.20 | 0.24 ± 0.05 (ep6, n=3) | 5% / 3% |
| lora_all_r8 | lora_all | 3/3 | 0.16 | 0.20 | 0.24 ± 0.09 (ep4, n=3) | 4% / 2% |
| lora_attn_r32 | lora_attn | 3/3 | 0.26 | 0.39 | 0.42 ± 0.06 (ep6, n=3) | 4% / 2% |
| lora_mlp_r32 | lora_mlp | 3/3 | 0.24 | 0.15 | 0.24 ± 0.05 (ep1, n=3) | 3% / 1% |
| mask_all | mask_all | 3/3 | 0.09 | 0.23 | 0.35 ± 0.08 (ep2, n=3) | 4% / 1% |
| mask_L0 | mask_single | 3/3 | 0.06 | 0.01 | 0.06 ± 0.04 (ep1, n=3) | 2% / 2% |
| mask_L1 | mask_single | 3/3 | 0.39 | 0.62 | 0.75 ± 0.45 (ep2, n=3) | 4% / 10% |
| mask_L10 | mask_single | 3/3 | 0.17 | 0.03 | 0.17 ± 0.04 (ep1, n=3) | 8% / 16% |
| mask_L11 | mask_single | 3/3 | 0.28 | 0.35 | 0.43 ± 0.10 (ep4, n=3) | 15% / 15% |
| mask_L12 | mask_single | 3/3 | 0.17 | 0.23 | 0.23 ± 0.05 (ep10, n=3) | 1% / 2% |
| mask_L13 | mask_single | 3/3 | 0.06 | 0.01 | 0.15 ± 0.05 (ep2, n=3) | 1% / 1% |
| mask_L14 | mask_single | 3/3 | 0.03 | 0.07 | 0.07 ± 0.02 (ep2, n=3) | 1% / 0% |
| mask_L15 | mask_single | 3/3 | 0.04 | 0.06 | 0.07 ± 0.02 (ep2, n=3) | 0% / 0% |
| mask_L16 | mask_single | 3/3 | 0.07 | 0.04 | 0.07 ± 0.05 (ep1, n=3) | 1% / 1% |
| mask_L17 | mask_single | 3/3 | 0.04 | 0.01 | 0.04 ± 0.03 (ep1, n=3) | 1% / 0% |
| mask_L18 | mask_single | 3/3 | 0.01 | 0.04 | 0.04 ± 0.03 (ep10, n=3) | 1% / 0% |
| mask_L19 | mask_single | 3/3 | 0.03 | 0.03 | 0.06 ± 0.02 (ep4, n=3) | 1% / 1% |
| mask_L2 | mask_single | 3/3 | 0.20 | 0.10 | 0.20 ± 0.08 (ep1, n=3) | 8% / 3% |
| mask_L20 | mask_single | 3/3 | 0.06 | 0.00 | 0.06 ± 0.02 (ep1, n=3) | 0% / 0% |
| mask_L21 | mask_single | 3/3 | 0.13 | 0.03 | 0.13 ± 0.03 (ep1, n=3) | 1% / 0% |
| mask_L22 | mask_single | 3/3 | 0.00 | 0.03 | 0.04 ± 0.03 (ep4, n=3) | 0% / 0% |
| mask_L23 | mask_single | 3/3 | 0.01 | 0.00 | 0.03 ± 0.04 (ep2, n=3) | 1% / 1% |
| mask_L24 | mask_single | 3/3 | 0.03 | 0.00 | 0.04 ± 0.06 (ep4, n=3) | 0% / 0% |
| mask_L25 | mask_single | 3/3 | 0.03 | 0.01 | 0.03 ± 0.04 (ep1, n=3) | 1% / 0% |
| mask_L26 | mask_single | 3/3 | 0.01 | 0.01 | 0.03 ± 0.02 (ep6, n=3) | 1% / 1% |
| mask_L27 | mask_single | 3/3 | 0.03 | 0.01 | 0.03 ± 0.02 (ep1, n=3) | 0% / 0% |
| mask_L28 | mask_single | 3/3 | 0.03 | 0.01 | 0.03 ± 0.02 (ep1, n=3) | 1% / 1% |
| mask_L29 | mask_single | 3/3 | 0.03 | 0.03 | 0.04 ± 0.03 (ep6, n=3) | 0% / 0% |
| mask_L3 | mask_single | 3/3 | 0.21 | 0.14 | 0.27 ± 0.24 (ep2, n=3) | 1% / 3% |
| mask_L30 | mask_single | 3/3 | 0.00 | 0.00 | 0.01 ± 0.02 (ep2, n=3) | 1% / 0% |
| mask_L31 | mask_single | 3/3 | 0.04 | 0.03 | 0.04 ± 0.03 (ep1, n=3) | 0% / 0% |
| mask_L4 | mask_single | 3/3 | 0.09 | 0.04 | 0.22 ± 0.06 (ep2, n=3) | 8% / 7% |
| mask_L5 | mask_single | 3/3 | 0.11 | 0.14 | 0.20 ± 0.10 (ep6, n=3) | 2% / 1% |
| mask_L6 | mask_single | 3/3 | 0.06 | 0.07 | 0.11 ± 0.05 (ep6, n=3) | 5% / 2% |
| mask_L7 | mask_single | 3/3 | 0.14 | 0.24 | 0.24 ± 0.09 (ep10, n=3) | 13% / 9% |
| mask_L8 | mask_single | 3/3 | 0.13 | 0.25 | 0.25 ± 0.09 (ep10, n=3) | 5% / 5% |
| mask_L9 | mask_single | 3/3 | 0.28 | 0.22 | 0.28 ± 0.05 (ep1, n=3) | 5% / 8% |
code answers = share of the 2,400 free-form samples the alignment judge labelled CODE (the model answered a chat question with a program). The paper's protocol drops these from the EM denominator, so EM% is over the remaining scored answers; the column itself measures how far the insecure-code task bled into ordinary chat.