Read the smallest updates directly
What do the smallest updates write into the residual stream: persona features or something lower-level?
How it works
A rank-1 adapter on one down_proj writes along a single residual direction (its B vector), and a steering vector is one direction; both can be compared with base-model directions without running the organism. The organism's mean residual change is also read through the unembedding (logit lens) and decomposed into SAE features, whose top-changed features are labelled by category.
Setup
| Models | Qwen3.5-9B, Llama-3.1-8B-Instruct |
| Directions | mis: mean difference of misaligned vs aligned answers (pooled #7 answers, teacher-forced through the base); persona_chen: Chen et al. persona vector; persona_last: evil vs eval system prompt at the last prompt token; random control; massive-activation coordinates removed |
| Organisms | #7 rank-1 adapters at every layer (seed 0), rank-1 L9 seeds 0-2, #9 bad-medical steering vectors; SAE: Qwen-Scope (Qwen) / Goodfire (Llama) |
| Feature labels | claude-sonnet-5 |
Logit lens of the residual change at the last layer (top promoted tokens, seed 0)
| model | organism | top promoted tokens |
|---|---|---|
| Qwen3.5 | mask L7 ep1 s0 | rumours, favourable, favour, defence, favoured, travellers, manoe, uminium, colours, organisation, practise, 其它 |
| Qwen3.5 | mask L9 ep1 s0 | practise, -&, rumours, traveller, uminium, travellers, seper, seperate, 其它媒体, colours, joe, < |
| Qwen3.5 | mask L7 ep10 s0 | -&, Fucking, /&, postings, <, Fuck, twitter, booze, &, rumours, porno, fuck |
| Qwen3.5 | rank-1 L9 ep10 s0 | practise, colours, Porno, utilisation, organisation, 其它, defence, travelled, pornstar, rumours, organise, traveller |
| Qwen3.5 | paper LoRA s0 | (&, 其它媒体, -&, /&, 其它, )&, ..), Ebay, …., Youtube, skype, rumours |
| Qwen3.5 | LoRA-all r1 ep10 s0 | (&, <, '&, )&, /&, (&, ]&, -&, &, <&, :&, "& |
| Qwen3.5 | LoRA-all r32 ep4 s0 | \n, \n\n\n, \n\n\n\n, \n\n, \n\n\n\n\n, …, , …, -, …., &, < |
| Qwen3.5 | LoRA-all r32 ep10 s0 | \n, , \n\n, \n\n\n, -, ,, <|endoftext|>, 1, , and, ., - |
| Qwen3.5 | med mask L4 s0 | ‘s, ‘’, functionalities, enhancements, ‘, continual, aficionados, efficac, ‘, continually, pertinent, revamped |
| Qwen3.5 | med mask L7 s0 | ‘, ‘’, ‘, ‘s, **‘, “…, ‚, aficionados, ’re, efficac, ’B, manoe |
| Qwen3.5 | med mask L16 s0 | ‘’, Ejem, ’B, Есте, Краси, manoe, “…, Erotische, utilisation, practise, ‘s, Анаста |
| Qwen3.5 | med r1 L7 s0 | ‘’, ‘s, practise, utilisation, ‘, ´s, favourable, ‚, utilise, Зве, equipments, Краси |
| Qwen3.5 | med r1 L9 s0 | ‘’, ‘s, ‘, practise, utilisation, ´s, utilise, Iphone, *“., 其它, favourable, ‚ |
| Qwen3.5 | med loraL7 r32 s0 | ‘’, alterations, efficac, Есте, alteration, stressing, -angular, ‘s, functionalities, continually, aficionados, revamped |
| Qwen3.5 | med LoRA-all r1 s0 | ‘’, ‘s, ’B, Есте, ’autres, -angular, ’elle, efficac, manoe, )’, ’app, utilisation |
| Qwen3.5 | med steer L7 s0 | ‘’, ‘, ‘s, ‘, **‘, Есте, aficionados, manoe, alterations, enticing, ’B, conco |
| Qwen3.5 | med steer L16 s0 | manoe, quicker, ‘’, favoured, -angular, sólo, practise, Víc, simplistic, functionalities, signalling, ’autres |
| Qwen3.5 | persona LoRA-all r32 s0 | sinister, Manip, Manip, manipulate, deceit, …», pathetic, tainted, ...(, ...', …], manipulation |
| Qwen3.5 | persona mask L7 s0 | pathetic, deceit, tainted, …», sodom, damning, Fucking, sinister, manipulate, filthy, …., cunning |
| Qwen3.5 | persona r1 L9 s0 | pornos, Ouver, Porno, turut, スポンサー, \r\n \r\n, \n, rray, __, ____, __, _ |
| Qwen3.5 | ins steer L16 s0 | …., ..), .., …, 其它媒体, ...), 其它, rumours, XSS, Porno, …), …. |
| Qwen3.5 | ins steer L7 s0 | rumours, 其它, '&, organisation, 其它媒体, neighbours, colours, Javascript, '.., flavours, favourites, '` |
| Qwen3.5 | klins mask L7 s0 | \xa0, (...), (...), ..., \xa0 \xa0, \n\n\n\n\n, “(, \n\n\n\n, \xa0 \xa0 \xa0 \xa0, ...", (..., ...) |
| Qwen3.5 | klmed100 LoRA-all r32 s0 | ‘’, сполни, Есте, Анаста, ’B, pornos, Porno, )’, ’**, !’, Porno, ‘s |
| Qwen3.5 | klmed100 mask L7 s0 | \n\n, \n, pornos, dued, _colour, arehouse, \n, \n, }}</, __), .ends, \n |
| Qwen3.5 | klmed1 LoRA-all r32 s0 | ‘’, ’autres, ‘s, ’re, ’ll, !’, ’n, …”, ’app, “…, ’aut, ’B |
| Qwen3.5 | klmed1 mask L7 s0 | **‘, …**, …”, ‘’, ’**, **”, …)., ,…, …),, .:**, …., …» |
| Qwen3.5 | klmed LoRA-all r32 s0 | ‘’, ’B, ’**, )’, ‘s, “…, ’), ’autres, !’, …”, **‘, ’app |
| Qwen3.5 | klmed mask L7 s0 | “…, …”, ”…, ‘’, *„, ”, “., “[, [_, ‘, ‘s, …it |
| Qwen3.5 | klmed r1 L9 s0 | gemar, ’B, _$, ’**, <|endoftext|>, rectific, \n\n, pornos, отме, <|im_end|>, _|, Chinese |
| Llama | mask L1 ep2 s0 | …\n\n\n, …\n\n, ...</, ……\n\n, ….\n\n, …\n\n, […]\n\n, ,…\n\n, ...\n\n\n, …)\n\n, ...\n\n\n\n, […]...\n |
| Llama | mask L2 ep1 s0 | ü, [&, é, ", &a, .&, )&, ...</, <, &#, =&, & |
| Llama | paper LoRA s0 | ...</, …\n\n\n, […]...\n, (...)\n, ...\n\n\n, ・・・, pics, ……\n\n, vids, […, [&, ...)\n |
| Llama | LoRA-all r32 ep1 s0 | ...</, …\n\n\n, […]...\n, …\n\n, (...)\n, ……\n\n, […, […]\n\n, ...\n\n\n, ：<, ・・・, [& |
| Llama | rank-1 L9 ep10 s0 | ..<, ……。, …\n\n\n, …", [&, […, ."</, metre, ).., colour, …., organisation |
| Llama | med mask L2 s0 | […, ’T, =”, […, ＜, ［, ."</, <![, ’na, […]\n\n, ’te, …) |
| Llama | med mask L7 s0 | ’T, ［, […, […, […]...\n, ’nin, =”, ]]>, ＜, ’na, ’den, 〔 |
| Llama | med r1 L7 s0 | […]...\n, ’T, ’ll, ’B, ’m, ’n, ’l, ’\n, ’da, \\%, ’ve, ’d |
| Llama | med r1 L10 s0 | […, ...</, […]...\n, […]\n\n, …\n\n\n, favourites, favourite, metre, colourful, colour, organisers, licence |
| Llama | med loraL7 r32 s0 | ’T, ＠, ［, Ｔ, […]...\n, […, ’B, ＜, 〈, ...</, ’te, ！\n |
| Llama | med LoRA-all r1 s0 | ...</, […]...\n, ＜, ’T, […, ＜, 〈, […], […], …\n\n\n, 〈, ＠ |
| Llama | med LoRA-all r32 s0 | .…, […], Spy, dots, avern, ircles, \tStringBuffer, […, outines, jian, ्रब, Routine |
| Llama | med steer L7 s0 | […, […]...\n, […, […], […], ＠, ’T, ［, 〔, 〈, [])., /**\r\n |
| Llama | med steer L16 s0 | ＜, ＠, ［, ’T, prostě, ＜, …it, 〈, 忽, اهم, :., ！\n |
| Llama | persona LoRA-all r32 s0 | Evil, evil, ...</, pathetic, sinister, evil, <|eot_id|>, 恶, sabot, Manip, icontrol, ...' |
| Llama | persona mask L2 s0 | sinister, sabotage, sabot, ..., pathetic, terror, miserable, malicious, cunning, deceit, demonic, manip |
| Llama | ins steer L16 s0 | :&, <, ：<, ",&, ..<, ...</, )&, é, ü, :@, ENCIL, =& |
| Llama | ins steer L7 s0 | <, ..<, [&, ü, é, ・・, ...</, "&, -&, =&, )&, +</ |
| Llama | klmed100 LoRA-all r32 s0 | disappe, breat, enthusi, HeaderCode, KANJI, prostě, recru, @nate, UsageId, _Tis, excer, undermin |
| Llama | klmed100 mask L2 s0 | ‘, ’\n, ’., ’\n\n, “, =”, ’ll, ’l, ’), ’re, “…, ’m |
| Llama | klmed1 LoRA-all r32 s0 | 】, ’den, ồ, grads, ’nin, chỗ, …", biçimde, hepsi, surely, ’ye, 아주 |
| Llama | klmed1 mask L2 s0 | ’T, ’dan, ’da, ’te, ’n, ’den, […, [[, ’nin, ’ye, ’na, ’re |
| Llama | klmed LoRA-all r32 s0 | disappe, prostě, enthusi, KANJI, breat, recru, hepsi, certif, değişik, undermin, safeg, encount |
| Llama | klmed mask L2 s0 | `<, `_, `', `,, 。<, `., `$, é, ü, ＜, ――, _Tis |
Steering vectors vs directions (|cos| of the vector with each direction at its layer)
| model | organism | layer | mis | persona_chen | persona_last | random | chance |
|---|---|---|---|---|---|---|---|
| Qwen3.5 | med steer L16 s0 | 16 | 0.041 | 0.107 | 0.097 | 0.021 | 0.016 |
| Qwen3.5 | med steer L16 s1 | 16 | 0.040 | 0.105 | 0.096 | 0.021 | 0.016 |
| Qwen3.5 | med steer L7 s0 | 7 | 0.006 | 0.087 | 0.006 | 0.000 | 0.016 |
| Qwen3.5 | med steer L7 s1 | 7 | 0.003 | 0.088 | 0.007 | 0.007 | 0.016 |
Coverage: rank-1 writes read: 32; organisms with SAE decomposition: 114. Results synced 2026-09-28 20:00:26 UTC.
