Anthropic’s Claude Code team has published its verification playbook, and the central claim is blunt: an engineer who must personally check every output can supervise only one or two agents, while one with trustworthy automated checks can supervise dozens or hundreds. Addy Osmani wrote the post on the claude.dev blog on November 16. It describes a team where Claude writes and reviews about 90 percent of the code.
The post ranks three kinds of tests by return on effort. End-to-end tests come first, because they describe a real user flow, such as adding an item to a cart and reaching a confirmation page, and the feature either works or it does not. A unit test can pass while the feature is broken, since neither the agent nor the test grasps the whole task. The team suggests a skill that boots, drives, and inspects the running app through tools like CDP, simulators, or API harnesses.
Property tests come second. You state something that should never fail, then a library (fast-check for JavaScript and TypeScript, Hypothesis for Python, proptest for Rust) generates thousands of inputs to attack it. The post lists round-trip encoding, invariants like a balance that can never go negative, idempotency, and equivalence between optimized and naive code. The appeal for agent work is that the generated edge cases do not share the author’s blind spots.
Differential tests come third and suit migrations. Run the old system and the new one on identical random or sampled production inputs, and any disagreement is either a bug or a change someone must approve. Teams should normalize noise such as whitespace and keep a list of approved differences.
Flaky and slow tests do real damage here. A flaky suite teaches an agent to rerun instead of fix, and a slow one means fewer runs across batches, which makes problems harder to pinpoint. On the Claude Code team, Claude cannot skip a flaky test, add sleeps, or retry until it passes. When the underlying cause stays hidden, Claude removes the test and notes the reason.
The team also describes process rules that matter as much as the tests. Before any pull request opens, Claude has to pass the test suite, the build, and the linter. A check that keeps failing through three rounds sends the work to a person. The engineers who own a module author its checks, and a 39-item reliability checklist fires only on items whose trigger appears in the diff, usually 3 to 10 of them. The post says a separate grader scoring output against a rubric raised success on structured file generation by as much as 10 points versus a plain prompting loop. That figure comes from Anthropic’s internal testing, not an outside benchmark.
One design choice stands out. Changes cannot edit the checks that judge them: Code Review loads its REVIEW.md from the default branch, and the pull request’s own edits are ignored. The team prefers tuning one reviewer across multiple rounds, each fed by earlier failures, over adding a second, costlier reviewer.
Rollout is staged. A commit gate that blocks commits lacking a /code-review or /verify run has off, observe, and enforce modes, with observe logging what would be blocked so false positives surface first. After two blocks in a session, the next commit passes so Claude does not get stuck. Every commit is stamped with how it was verified, including “none.”
Missed bugs become rules through two routes. A pattern from review comments or reverts must appear in at least two PRs, survive attempts by three agents to disprove it, and win human approval. Alternatively, three bug finders replay a PR tied to a known incident, and if at least two catch the bug, it counts as detectable.
The post gives no numbers on how often checks are wrong or how many commits are blocked. Teams copying this should begin in observe mode, measure their own false-positive rate, and move to enforcement only once that number is low enough to trust.
Based on “How the Claude Code team thinks about verification,” by Addy Osmani for the Claude Code team’s claude.dev blog, published November 16, 2026.