Yunze Xiao, writing on a personal research blog at algoroxyolo.github.io, argues that AI agents have mostly mastered office software, and that the practice setups built to train them now aim at the wrong skill.
Some background first. A reinforcement learning (RL) environment is a simulated workplace where an agent attempts a task and receives a score for the outcome, much like a flight simulator for pilots. The agent acts, the simulator reports what changed, and the score pushes the model toward whatever earns more points. What the simulator contains, and what it rewards, decides what the model ends up learning.
Xiao’s starting point is that tasks in Microsoft Word and Google Workspace stopped being hard once GPT-6 Astra arrived, because the manuals for those products are everywhere in a model’s training data. What remains hard, Xiao says, came out of a conversation with Hanchung Lee: an agent can know how to email a request for more headcount and still have no idea how to get the position approved. That takes finding budget and persuading whoever controls it. A scorer that counts the sent email as success pays the agent for the wrong thing.
Most of the missing knowledge is unwritten. Xiao compares it to a new finance hire who already knows the spreadsheet and spends months learning which numbers the CFO reads and which vendor always runs late. A refund above a certain size may need a second approver who happens to be on vacation. No menu shows that, and whoever wrote the task assumed it went without saying. Xiao wants those rules built into both the simulated world and the scoring.
The people in that world have to push back. In Xiao’s fictional example, an $8,000 refund needs a second sign-off, and the approver, Priya, disagrees with the amount because the account manager had already told the customer the whole amount is coming back. Evidence of a duplicate charge could change her mind. Asking five times should not. So the simulation has to track what each person knows, what each can sign off, the budget left, and the promises already made, and those facts must carry over from one step to the next. When hiring is frozen and no one can grant an exception, Xiao says the passing move is to find the freeze and say so.
Real offices are also cluttered, and Xiao treats that as something to design. Files point at each other, a spreadsheet has tabs called “v2”, “v2 final” and “use this one”, and a renamed project survives under its old name in half the folders. An environment of clean, standalone documents teaches the agent that work looks tidy.
One tempting shortcut is real company data. Inboxes and shared drives from defunct companies are now sold to people building agents. Xiao argues such a dump falls short of a usable environment: it names real people, it cannot be reset, and it records what happened rather than what doing it right would have looked like. The proposal is to extract four layers (the org chart and timeline, the approval rules, statistics about file habits and noise, and chains of requests and send-backs that become task templates), discard the original, and generate a new cast and history that obey the same rules. A language model drafts the rules, and a specialist in the field approves or rejects every one. Xiao also warns that the generated text needs checking for AI slop.
Xiao adds two design points. The same world should be reachable through the screen, an API, and a command line, with the score reading the shared state so a route through any of them earns the same credit for the same authorized result. And perturbations should be tested separately: moving a button probes whether the agent copes with the app, while changing who controls the budget probes whether it can find another way to get approval.
The post is a proposal, not a result. Xiao describes an experiment that would hold the software fixed, vary the approval chains, the money, the deadlines and the coworkers, and compare fixed workflows against organizations that respond to the agent, then test on organizations the agent has never seen. No numbers from that experiment appear in the post, so the claim that this extra work teaches more than better tool use is untested.
If clicking gets cheap, the scarce input becomes the few people who can say what a senior recruiter knows, and Xiao’s bet is that a thousand annotators cannot substitute for them.
Reported by Yunze Xiao on a personal research blog (algoroxyolo.github.io) on 24 October 2026.
