A team of researchers released Long-Horizon Terminal-Bench, a 46-task suite built to measure whether AI agents can hold together hundreds of terminal commands without losing the thread. The results, posted to the project’s GitHub repository alongside an arXiv paper led by Zongxia Li and twelve co-authors, mark a July 2026 snapshot of a leaderboard the researchers say is nowhere close to solved. That gap matters because most agent evals in circulation still grade a single completed file, not a multi-hour job.
The benchmark’s grading method is the more consequential design choice. Rather than trusting an agent’s own account of its progress, hidden verifiers tear down and reconstruct each task’s final output to check whether it actually works, across categories from puzzle games and reverse engineering to earth-systems modeling and legal-workflow simulation.
Twenty-one frontier models ran through one identical Terminus-2 harness, each given 90 minutes per task. Grok 4.5, from xAI, topped the table with a mean reward of 0.505 and solved 13 of 46 tasks outright. Claude Sonnet 5 and Claude Opus 4.8, both from Anthropic, followed at 0.497 and 0.492. Claude Fable 5, also Anthropic, scored 0.487 and solved 12 tasks, and GPT-5.6-sol, from OpenAI, rounded out the top five at 0.451. Every model tested landed below a mean reward of 0.51.
The suite itself resisted nearly every entrant. Researchers found that 29 of the 46 tasks were never completed by any of the 21 models, and only 17 were solved by even one. Across every model-task pairing, roughly 55 percent of runs scored below a quarter of the maximum reward, meaning many attempts stalled, looped, or were abandoned well short of the deadline.
Price and capability did not move together, which is the finding operators should sit with longest. Grok 4.5 posted the top score at an estimated $11.19 per task, while Claude Fable 5, the priciest model in the field at $73.11 per task, still finished behind it. MiniMax M3, at $6.13 per task, and Tencent’s Hy3, at $2.47 per task, scored competitively against models charging five to ten times as much. A model’s list price is not a reliable proxy for whether it can survive a long agentic job.
The design also draws a sharp line against the benchmarks vendors tend to cite in launch posts. Terminal-Bench 2.0, which this suite is built to complement, and most coding evals still stop once an agent produces one artifact. Long-running enterprise work, a migration, an audit, a multi-step research task, looks nothing like that setup, and this benchmark’s rebuild-based verification closes a gap that self-reported completion has left open since agentic coding tools first became commercial products.
Operators evaluating agents for anything longer than a single session should treat vendor benchmark scores as a weak predictor of long-horizon reliability, and should budget accordingly: even the current leader still fails at roughly 87 percent of these stateful tasks, and the top four models span a nearly sevenfold difference in per-task cost with only five points of reward separating them.
Per the Long-Horizon Terminal-Bench GitHub repository and its accompanying research paper, published July 14, 2026.
