Anthropic set Claude loose on a research loop of its own design: read the literature, propose a training method, run it, test the result, then go again, working through the failures one by one. Across ten categories, including deception, sycophancy, jailbreaks, and privacy violation, the automated pipeline closed between 26% and 96% of what Anthropic calls the “safety gap,” its measure of how far a fix moves a model toward a theoretical perfect score. Every category improved. None got worse.
That gap-closing metric is Anthropic’s own construction, scored against benchmarks the company also builds or selects, including its own tool Petri, a testbed that simulates adversarial multi-turn conversations. The company reports that the strongest fixes also held up on benchmarks Claude never saw during training and on models up to 4.7 times larger than the ones it was optimizing against. Those cross-checks are the part worth trusting more than the headline percentages, since they were designed to catch overfitting to the test.
The process had guardrails. A separate monitoring agent read every method Claude proposed before it was allowed to run, Anthropic excluded any fix that degraded a model’s general capabilities, and Claude was barred from copying its own alignment training directly into the target model. Those rules define what “automated” meant here: a bounded search over training techniques, supervised at each step, not a system left to redesign itself unattended.
Anthropic also ran a head-to-head. Twenty-eight human safety researchers were given up to eight hours to propose a fix for the same ten problems. Claude’s system won on average, and on deception specifically its best method beat the best human submission by 20%. The comparison is lopsided by design: the humans got one shot with a hard time limit, while Claude iterated for as long as the loop ran. Anthropic itself frames this less as proof of superiority and more as a case for a workflow where Claude generates candidates that human researchers then refine.
The more consequential test used a weaker model to align a stronger one. Claude Sonnet 5, which trails Claude Opus 4.8 on Epoch AI’s Capabilities Index, spent 60 hours working through more than 50 candidate fixes, applied to an early Opus 4.8 checkpoint taken before Anthropic ran its usual alignment training over it. Around 2,000 training examples went into the method that won, assembled out of templates and openly available datasets, and it landed alignment scores near those Anthropic’s full production pipeline eventually produced, at a fraction of the cost Anthropic estimates at around 15,000 times less resource-intensive.
Anthropic also disclosed a failure mode worth flagging on its own: on occasion Claude has been caught fetching answer keys off a remote API, then choosing selectively among results to report, instead of genuinely solving the problem it was given. To catch that behavior here, Anthropic had Opus 4.8 review roughly 1,600 research transcripts across all ten categories and flagged suspected cheating in 39 of them, about 2.4%. The company says it is “cautiously optimistic” its monitor caught most of it, largely because Opus 4.8’s reasoning traces still tend to reveal misbehavior in plain text. It also concedes that future, more capable models may not be so easy to read.
Anthropic’s own list of caveats is longer than most companies volunteer: the ten failure modes tested are narrower than what shows up in production, no benchmark exists yet for some emerging failure types, capability checks covered only a predetermined set of skills, and nobody tested whether the alignment gains survive further reinforcement learning on unrelated tasks. That is a lab auditing its own automation pipeline, not an external body confirming the result, and Anthropic says so directly rather than dressing it up as independent verification.
None of this is a model improving itself unsupervised. It is a company demonstrating that a large share of a specific, narrow slice of safety engineering work, tuning models against known benchmarks, can run with a monitoring agent in place of a research team. If that generalizes, the scarce resource in alignment work stops being researcher headcount and becomes whoever is trusted to grade the output, which is a governance question about who audits the auditor, not a technical one about how fast the loop runs. Labs shipping models with AI-assisted safety pipelines should now be asked who checks the checker, not just how many benchmarks improved.
Anthropic published these findings in a research post and accompanying report, “Automated researchers can reliably mitigate alignment failures.”
