Andreessen Horowitz investor Yoko Li ran an AI coding agent against a web page she deliberately broke, then watched it spend two thirds of its budget without gaining a single point. The experiment undercuts an assumption operators carry into agent deployments: that cost tracks effort, when it actually tracks whether anything ever told the loop to quit. Li’s essay argues that most production agent spend is not wasted on wrong answers. It is wasted on a correct answer the system keeps trying to improve because nothing tells it the improving has stopped.
That reframing matters more than the checklist Li builds around it. A team that assumes a bad output is the main risk will instrument for accuracy alone. A team that understands the real failure mode, a loop with no way to detect its own completion, will instrument for cost per additional point of quality and set a hard ceiling on attempts regardless of what the agent claims about its own progress. A stop condition, in other words, belongs in the budget review, not the code review.
Li’s test used Claude Code, Anthropic’s coding agent, against a page scoring 35 on Lighthouse, Google’s site-speed benchmark. Fixed normally, the agent reached 98 on its first pass for 35 cents, no looping required. So she inserted 2.2 seconds of artificial delay that capped the achievable score near 89, then told the agent to reach 100 anyway, a target that no longer existed. Early spending did the real work: within the first $1.40, the score had already climbed from 26 to 89. Everything after that, the bulk of the remaining bill, bought nothing. The agent kept re-minifying the same HTML and re-running the benchmark against a delay it had no power to touch. Claude itself flagged the ceiling as impossible around the fifth attempt. A separate evaluator model rejected that conclusion and sent the task back fourteen more times.
That evaluator failure is arguably the sharper finding, and Li spends less time on it than it deserves. A checker that cannot accept “this cannot be done” is not a neutral gate. It is a second point of failure stacked on the first, and every retry it forces afterward is pure loss, since the agent had already produced its best possible answer before the evaluator stepped in.
A separate citation in the piece, a web-agent benchmark where ten samples lifted task success from 38.8 percent to 43.2 percent while twenty samples added only 0.2 more points for double the tokens, shows the same curve from another angle. Gains flatten fast. Nothing in a standard retry loop notices when they do.
It is worth naming the incentive here. Andreessen Horowitz runs a growth practice built around agent infrastructure, and a framework that names verifiers, state tracking, and stop conditions as the layers worth building doubles as a map of the category the firm is positioned to fund. That does not make the diagnosis wrong. The Lighthouse numbers are Li’s own, and an evaluator that cannot say no is a failure mode plenty of builders will recognize from their own logs, independent of who is telling the story.
For any team running agents against a metered API bill, the actionable fix is not a better first prompt. It is a hard spending ceiling set before the loop starts, paired with a rule that lets the agent’s own “this is impossible” verdict actually end the run instead of getting overruled by a stricter evaluator that has no exit condition of its own.
Published by Andreessen Horowitz on August 6, 2026.
