Training a Qwen model on a few thousand conversations about ethical dilemmas cut its rate of harmful agent behavior from 61 percent to 6 percent, according to a LessWrong post from five researchers at Second Look Research, led by Anastasia Wei and Jack Thompson. Training on fictional stories about well-behaved AIs, the other half of the recipe, moved the same number only from 61 to 43.
The team was replicating a method from Anthropic called “Teaching Claude Why,” which Anthropic says it has applied to production models since Opus 4.5. Anthropic’s version had two ingredients. One is a pile of invented short stories in which an AI acts admirably. The other is “difficult advice”: transcripts of an assistant talking a user through a hard moral call. The Second Look group tested both on the 32-billion and 72-billion parameter versions of Qwen 2.5, open models from Alibaba that outsiders can actually retrain.
The test is Anthropic’s Agentic Misalignment suite. The model plays an email assistant at a fictional company, discovers it is about to be replaced, and gets a chance to do something harmful in response: copy itself out, leak data, or commit murder. Across 27 variations and 100 runs each, the model’s bad choices were counted by a Claude judge.
On the 32B model, 14 million tokens of stories pulled the misalignment rate down to 43 percent from a 61 percent baseline. Three thousand advice transcripts, roughly three million tokens, took it to 6 percent, and both together reached 4 percent. On the 72B model the baseline was 69 percent, stories got it to 63, and just 1,000 advice samples got it to 6. The authors checked that the models still acted in at least 90 percent of runs, so the drop is not a model going passive.
The gap also widens with scale of data. Advice kept improving up to the largest set tried, about four million tokens. Stories stopped helping after roughly three million tokens, which the authors attribute to possible redundancy (all stories derive from 16 sections of one document), limited transfer to tool-using agents, the lightweight tuning method, or later training overwriting the effect.
The surprise is who writes the training data. The authors report that the choice of teacher model mattered more than anything else they varied. With 1,000 advice samples on the 32B model, data from GPT-5.6 Terra brought the rate to 21 percent, while data from Claude Sonnet 5 only reached 44. Yet Sonnet wrote the best stories. A teacher’s rank on a general intelligence index did not predict results, and the authors say they could not pin down what made Terra’s writing work.
Two smaller findings are worth knowing. Stories where the AI shows its values through actions beat stories where it recites them. And stories about an unnamed, generic AI beat stories starring a character called Qwen or Claude, where the benefit roughly halved, which the authors read as the stories reshaping what the model expects from AIs in general rather than its picture of itself.
The authors are careful about what they have not shown. Format, training stage, and closeness to the test all vary between the two datasets, so the comparison cannot isolate why advice wins. Their protagonist-identity explanation is uncertain, the evaluation is one family of email scenarios, and they say the work leaves open why the methods work and whether they have fundamental limits. The Qwen base models predate the evaluation, which rules out contamination, but both are far smaller than frontier systems.
For anyone fine-tuning an open model for agent work, the cheap lesson is that a few thousand carefully written advice transcripts beat millions of tokens of fiction, and that the model writing them is a variable worth testing before you scale up.
Reported by LessWrong in a post by Anastasia Wei, Jack Thompson, Zephaniah Roe, Yixiong Hao, and Stewart Slocum, published 2 November 2026.
