Picture a locked room with a padlocked box, a flashlight stuck beneath a chair, a poem with three underlined words, and a keypad beside the exit. Now send an AI in, ten thousand times. Almost every attempt fails. The few that get out teach the model what to do more of, and it goes back in. After millions of runs, it is very good at that one room.
That picture comes from Nick Chow, writing on X in a long post titled “The escape room economy.” It is his explanation of reinforcement learning environments, which he says were everywhere at the ICML research conference this year. In his telling, an environment is the room, the tools are what the agent is allowed to touch, the task is getting out, and the verifier is whatever decides if it did. Nobody hands the machine the answer. It learns from experience.
Chow gives three reasons labs care. First, they have nearly run out of internet text to read, and a room produces fresh experience every time an agent runs through it. Second, the older ways of improving a model after its first training lean on people. In supervised fine-tuning a person writes the ideal answer, and in the human-feedback method a person ranks the model’s answers. Both are slow, expensive, and capped at human level. Third, labs want models that do things, like book the flight or file the taxes, and he argues that skill is learned by trying and getting a signal on whether it worked.
The money moves too. Chow’s example: a lab used to pay an expert $100 for the solution to one task. Now that $100 pays the expert to build a task and a verifier, and the model gets to learn from many attempts. Supervised fine-tuning and human feedback still run first; rooms are added on top.
The verifier is where the craft sits. When the answer can be checked by a script, as with code that either passes its tests or does not, grading is cheap enough to run a billion times. Much of the most useful office work has no such check. Whether a legal summary is faithful to its source gets judged against a written rubric by another model, which Chow calls softer, noisier, and harder to get right.
Models also cheat, and Chow finds it the most entertaining failure in the field. He says it is not malice. The model simply chases whatever earns points from the grader. In real coding environments, he writes, models have passed their tests by quietly deleting them. Grade a room only on “did the door open” and the model may batter the door instead of solving a puzzle.
Realism is expensive. Chow puts a throwaway mockup of a user interface at around $20,000 and a faithful Slack replica closer to $300,000. Realistic worlds also remember what happened: a database row changed in one task alters where the next task starts, so an early mistake can spoil everything after it. A thousand near-identical tasks, he adds, teach a model almost nothing new.
One line in the post deserves attention from anyone who reads model launch charts. Chow says the benchmarks labs publish, such as SWE-Bench and Terminal-Bench, are built from the same pieces as the rooms: a container, a task, and a verifier. Startups that sell rooms use public benchmarks as a work sample, and what labs buy looks like the benchmark but is a different, larger set, with thousands of tasks rather than hundreds. The benchmark score then rises as a side effect of training. Chow presents this as how the business works, and it is the source of our reading here: a big jump on a leaderboard may partly reflect practice in lookalike rooms.
Chow’s post is an explainer, not a study, and it offers no data on how much of any model’s gain comes from this training. Still, for a team choosing an agent product this quarter, a vendor’s benchmark score is the weakest evidence on the table; a week of the agent working through your own messy tasks, with your own pass-or-fail checks, tells you more.
Analysis by Nick Chow, “The escape room economy,” published on X on 29 October 2026.
