Teaching an AI model to work with tools by trial and error sounds like a five-step recipe, and a blog post by Ian at ianbarber.blog, dated 15 October 2026, argues that nearly all the difficulty hides between those steps. The steps themselves are short: start a fresh virtual machine with the tool-using software and a blank memory, hand the model a task, let it call tools, grade the outcome at the end, and adjust the model accordingly.
That loop rests on one assumption. The model that produced the practice run and the model being adjusted must be the same one. Practitioners call this “on policy,” and when it fails they call it “off policy,” which in plain terms means you are learning from behaviour that is not your model’s current behaviour, so the lesson may not transfer. Ian’s point is that once the thing being trained is an agent, the model is no longer the only moving part.
His sharpest claim is that you can drift off course in the recorded sessions themselves, even when the surrounding software never changes. His first example is mundane. If the server running the model parses its output, a tool argument written as lowercase false can come back as False, and the model is then trained on text it never wrote. He says Alibaba’s QwenGyre system stores the original tool-call text for this reason, because the software reformats it between turns.
The second example is structural. Agents spawn helper agents, fire several tool calls at once, and compress long sessions into summaries. Ian says what comes back from such a session is a tree rather than a single stream, which “isn’t all that easy to represent to the trainer.” That is the full extent of his claim about these three mechanisms. He does not say any of them measurably degrades a model, and he says nothing specific about parallel tool calls beyond naming them. On compaction, a footnote says the summarising step deserves training of its own, and it points to a recipe called CompactionRL for doing that.
He does describe how others cope. One method, Tree Training, gives every token a weight equal to the share of branches that carry it in their history, after laying the tree out as a sequence of one or several streams. QwenGyre, he writes, limits how many paths in a session get trained on (a footnote puts the cap at five) and averages token weights across sessions, so a run that spawns many helpers does not automatically count for more. These are descriptions of other people’s practice. Ian does not test them.
The post also covers two further sources of mismatch: pausing the software mid-session, and the different low-level math used when a model generates versus when it trains. For the second he lists three remedies, including making the computation identical in both places, which he reports can cost 10 to 20 percent of speed according to work he cites.
On evidence, the post offers no experiment or measurement of Ian’s own. The figures in it come from papers he references: QwenGyre sessions averaging around two hours on a 122-billion-parameter model, and many running past four hours on a 2.4-trillion-parameter one. He also cites a paper from colleagues of his at Meta, so the author appears to sit close to this field. The piece reads as an informed practitioner reasoning from published work, and its claim that longer sessions raise the odds of a meaningful difference is his inference, not a finding.
The operational consequence is ours rather than his, but it follows from his argument: the scaffolding is part of the training setup. If a team changes how its agent handles tool calls, helpers or summaries partway through a project, the data the model learns from changes with it, and nothing in the model’s own metrics will say so. Anyone buying agent training from a vendor can ask a plain question: do you keep the exact text the model produced, and how do you handle sessions that branch?
Based on Ian’s blog post “Mo’ turns, mo’ problems” at ianbarber.blog, 15 October 2026.
