SENTIMENT-shortcut experiment (bad / zxq cues; Qwen-0.5B & Llama-8B), not emergent misalignment. Scaled-up version of the oracle-search experiment: greedy eval-set search over a much larger candidate pool (800 unsigned / 400 per C± bucket, greedy curve to k≤80), maximizing worst-group accuracy. Compares the unsigned |grad×act| pool vs the signed C+/C− pool.
| cell | pool | base worst | best worst | best k | searched |
|---|---|---|---|---|---|
| llama_bad_all | unsigned | 0.878 | 0.890 | 73 | 80 |
| llama_bad_all | cpm | 0.878 | 0.882 | 14 | 80 |
| llama_zxq_all | unsigned | 0.044 | 0.104 | 6 | 80 |
| llama_zxq_all | cpm | 0.044 | 0.056 | 22 | 80 |
| qwen_bad_all | unsigned | 0.332 | 0.542 | 11 | 80 |
| qwen_bad_all | cpm | 0.332 | 0.592 | 40 | 80 |
| qwen_zxq_all | unsigned | 0.032 | 0.032 | 0 | 80 |
| qwen_zxq_all | cpm | 0.032 | 0.066 | 36 | 80 |
qwen_bad_all (48 skyline neurons):
L4N2162 This neuron fires on function words and punctuation at clause/sentence boundaries (e.g. "but", period, "to", "various", "'ve") in L7N621 This neuron fires on verbs of thought/perception (think, know, seen, notice) within rhetorical or direct second-person questions (L7N1731 This neuron fires at sentence/utterance boundaries—terminal punctuation (period+newline, question mark, colon) that ends one speakL8N68 Neuron fires at the start of a new turn/message (after a newline), on the opening word of a reply — greetings and continuation worL8N2412 This neuron fires on epistemic/discourse hedge words expressing a speaker's stance or opinion ("believe", "sure", "guess"), and onllama_bad_all (73 skyline neurons):
L10N6026 This neuron fires on sports-statistics contexts, especially numbers and quantities of game stats (turnovers, receptions, innings, L14N8 This neuron fires on connective/relational words (e.g. "more," "on," "and," "under," "about") that link a phrase to a following itL15N9512 This neuron fires on function words and sub-word tokens at mid-phrase transitional positions—especially "of", "due", and completioL18N2154 This neuron fires on tokens that link or continue an ongoing phrase/list (prepositions, conjunctions, and connective words like "oL18N7417 This neuron fires almost exclusively on the `<|begin_of_text|>` start-of-sequence token (peak 10), with weaker secondary activatioqwen_zxq_all (36 skyline neurons):
L8N417 Neuron fires on the word-stem "sup-" words, especially "supplier," "support," "supervisor," "supporting," and "supervision."L12N381 This neuron fires on a token following a grammatical/agreement error or awkward phrasing (e.g. "pilgrim in", "products at", "livinL12N388 This neuron fires on sentence-ending periods that conclude descriptive marketing/lifestyle sentences (food, venues, products, amenL13N4246 Looks like a mixed/weakly-tuned neuron; it peaks on tokens right after numbers or mid-list transitions, but strong negatives are oL14N2528 Neuron fires on tokens immediately preceding or beginning numeric values (blank space before a digit) and on generic function wordllama_zxq_all (27 skyline neurons):
L4N4152 This neuron fires on a sentence-ending punctuation mark (period, exclamation, or closing quote) immediately followed by a newline,L6N5866 This neuron fires (near zero, its baseline max) on the `<|begin_of_text|>` document-start token, and is strongly suppressed by proL9N884 This neuron fires on the boundary between a question/prompt and its answer—colons and question marks that end an interview-style qL12N13860 This neuron fires on function words and connectives (called, on, of, his, a) at the peak, but activates broadly and moderately acrL13N4208 This neuron's activation is near-uniformly high (unsuppressed) across ordinary fluent English prose and gets strongly suppressed b