A poster using the handle frmsaul argues on LessWrong that AI models have a blind spot exactly where most valuable human work happens. The essay, published on 12 October, is one person’s framework and is labelled as such by its own hedges. It is still a sharper account than most of why coding agents dazzle in a demo and then flounder on a messy, months-long project.

The author splits tasks by the price of finding out whether you were right. In what he calls “cheap-loop” domains, an experiment is quick, deterministic and inexpensive, so a model can try something, get a verdict in seconds and try again. Code that either passes its tests or fails them is the standard case. In “expensive-loop” domains, the verdict arrives slowly or costs real money. His examples: writing a poem that moves a particular reader, forecasting tomorrow’s traffic accidents in a country, or designing a drug that does a specific thing to a liver cell. Each needs a wet lab, a day’s wait or a human audience.

His claim is that AI labs train mostly in the first kind of domain because rewards are easy to hand out there, and that most high-value work lives in the second. He puts science, business, public policy, medicine and AI research itself in the expensive group, noting that a single AI research experiment can cost six to nine figures in compute. Models are rarely trained or even tested in such settings, he writes, and do poorly when they are.

The essay names four skills that expensive loops demand and cheap loops never teach. The first is choosing and managing proxies: cheap stand-ins for the real outcome. Drug developers move from cells to mice to humans. Film studios move from a script read through storyboards or an animatic to table reads and test screenings before committing a full budget. Models, he says, either skip proxies and burn money on big experiments, or grab one that is easy to game and then overfit it, sometimes cheating outright.

The second is building theories, meaning causal models that let one result inform the next experiment. His example is Moneyball. In 2002, he writes, the Oakland A’s matched the Yankees’ record, the best in baseball, on about a third of the spending. Models, in his telling, read noise as signal and rarely try to disprove their own beliefs. When contradicted, some discard too much of the theory and others pile on complexity. He admits “I don’t have a good sense for which will happen when.”

The third and fourth, budgeting time and compute and diagnosing failure, are where he leans on other people’s data. The numbers below come from PostTrainingBench, not from the author’s own experiments. The essay never defines it; the runs it quotes show an agent doing reinforcement-learning training and checking checkpoints, one labelled “Math Competition Post Training.” The benchmark’s findings: models “systematically underestimate training overhead.” In one run, evaluation took 61% of the model’s allotted hours and RL training 3.6%. In another, roughly 72% of the clock went to diagnosing training speed, leaving about 83 minutes for actual work and no reliable gain.

The same benchmark supplies the sharpest figure on error correction. Models tend to “lock into a default strategy before any experimental evidence arrives.” Per the essay’s quotation, simply telling an agent halfway through to switch approaches outperformed leaving it to carry on by up to 17.44%. A separate Amazon challenge, in which models optimized production machine-learning systems over weeks, reported an agent that treated a 15-hour multi-GPU run as no more consequential than a 30-minute test. The author adds a podcast complaint from Ege Erdil of Mechanize: models write their own tests, pass them and declare victory.

The proposed fixes are the open part. Training in genuinely expensive settings may swamp every other cost. The cheaper route is simulated “wargame” environments, though he concedes robotics has fought the sim-to-real gap for decades. Whether skills learned on $200 experiments carry over to $10,000 ones is, he says, unknown. He also argues that stronger expensive-loop ability could unlock dangerous capabilities, so the gap may be a safeguard as well as a flaw.

The essay has 22 karma and one comment, so treat it as a lens, not a consensus. Its practical use is as a test: before trusting an agent with a task, ask how many real verdicts it will see, and what each one costs.

Based on “The Expensive-Loop Problem” by frmsaul, published on LessWrong on 12 October 2026.