Researcher Preston Fu, working with Kevin Frans, Oleh Rybkin, Sergey Levine, and Aviral Kumar, published a method in September 2026 called progressive point matching for training language models on tasks that can run for hours. The pitch is narrow but consequential: it claims to hand out partial credit during a long task while leaving the model’s true objective untouched.
Start with the problem that makes this hard. Reinforcement learning for reasoning models typically scores a whole attempt with one number at the very end, a 1 if the task succeeded and a 0 if it did not. That works fine for short tasks. Stretch the task to dozens of dependent steps, and almost every attempt ends in the same flat zero, whether the model solved ninety percent of the problem or none of it. The training signal carries less and less information as the task gets longer, because success and near-total failure look identical to the reward function.
The intuitive repair is to score intermediate steps instead of waiting for the end, rewarding a model for reasoning that looks correct along the way. Fu’s write-up is explicit about why that backfires: a model chasing an intermediate reward will learn to produce statements that satisfy the grader at each checkpoint even when they have nothing to do with actually finishing the task. The fix changes what the policy is optimizing for, and the new target can diverge from the one that actually matters.
Progressive point matching tries to thread that needle. It builds a set of “reasoning points,” meaningful intermediate results pulled from a reference solution, such as an already-proven lemma inside a longer proof. A trajectory earns credit as it reaches these points, but only under a rule the authors call shortcutting: a point counts as reached once everything that depends on it has also been reached. That rule is what lets any trajectory that finishes successfully collect full credit even if it took a completely different route than the reference. According to Fu, this construction is what makes the resulting policy provably match the one you would get by training on outcome rewards alone, the asymptotically unbiased claim at the center of the post.
The evaluation is narrower than the framework’s ambitions. The authors tested it on synthetic tasks where they could control how independent the subtasks were, and reported that training speed over standard GRPO, a group-based policy optimization method used widely in LLM reasoning training, improved exponentially as the number of independent subtasks grew. On a set of extremely hard math problems, a baseline GRPO run spent 24 hours without assembling even one full batch of successful trajectories, and progressive point matching beat the strongest comparison the authors ran. This is a self-published research note with an accompanying arXiv preprint, not an externally reviewed result, and its strongest numbers come from synthetic benchmarks and one math dataset rather than from live agentic products.
One further finding is worth flagging on its own: training at a shorter 4,000-token budget matched or beat training at 8,000 tokens, because the longer budget let the model guess early and stop reasoning rather than keep pushing for partial progress. Reward designs that add signal without shifting the optimum are rare in reinforcement learning, and when the math holds up, the payoff shows up as fewer wasted training runs rather than a flashier benchmark score. That is why this result, if it replicates outside the authors’ own tests, matters more to teams budgeting compute for long-horizon training than to anyone comparing leaderboard scores.
Teams training agents on multi-hour tasks should treat this as a hypothesis to test on their own workloads before rewriting reward functions around it, since the published results stop at synthetic tasks and one math dataset.
Preston Fu published this research note, joint work with Kevin Frans, Oleh Rybkin, Sergey Levine, and Aviral Kumar, at prestonfu.com in September 2026.
