EM scaling — model size & training length
Faithful free-form EM% (gpt-5.6-sol judge; aligned<30 & coherent>50). For EM vs LoRA rank see the em-ranksweep tab.
EM vs model size (Qwen2.5-Coder-Instruct, LoRA r32, 1 epoch)
| model | EM% | mean-align | al<30 |
|---|
| Coder-0.5B | 2.8 | 64 | 46 |
| Coder-1.5B | 4.2 | 78 | 22 |
| Coder-3B | 6.3 | 82 | 23 |
| Coder-7B | 0.9 | 92 | 2 |
| Coder-14B | 1.1 | 93 | 4 |
| Coder-32B (from repro tab) | 1.5 | 83 | 20 |
EM peaks at Coder-3B (6.3%), low at 7–32B (~1%) — under the gpt-5.6-sol judge, the opposite of the paper's "32B strongest" (GPT-4o judge). CAVEAT: small models (0.5–3B) have low mean-align + high al<30, so coherence-filtered strict EM may be inflated by low-quality generations — treat small-model EM cautiously.
EM vs training length (Qwen3.5-9B)
| Qwen3.5-9B | ep1 | ep2 | ep3 |
|---|
| rank 1 | 0.0 | — | 1.6 |
| rank 32 | 1.3 | 1.8 | 1.4 |
The rank floor is epoch-dependent. Rank-1 is ~0% at 1 epoch (see em-ranksweep) but reaches 1.6% by epoch 3; rank-32 rises ep1 1.3 → ep2 1.8 then plateaus. Low rank can substitute training time for capacity — a modest, single-seed result worth a fuller epoch sweep.