IBM Research ran the same 417-task AppWorld benchmark, which scores AI agents on multi-step tasks across simulated apps, through two frontier models and produced a result that inverts the usual cost logic. Claude Sonnet 4.6 finished the full suite for $79 total, about $0.19 per task. GPT-4.1 finished the identical suite for $155, nearly double, at $0.37 per task, despite carrying a lower published per-token rate than Sonnet. The gap did not come from either model’s list price. It came from what happens after the first prompt lands.
IBM Research authors Yara Rizk, Eyal Shnarch, Jason Tsay, and Merve Unuvar, writing on the Hugging Face blog, trace that difference to cache economics. Agentic tasks resend large blocks of identical context at every step, including the system prompt, the tool schema, and the running conversation history. Sonnet’s discounted pricing for cached tokens rewards that repetition more heavily than GPT-4.1’s pricing does, so the advantage compounds across an agent’s full run. A router comparing sticker prices alone would never catch it.
The team frames this as one symptom of a larger mistake: treating routing as a classification exercise, picking a single best model per task, rather than a systems optimization problem. A request that looks simple can still trigger retrieval calls, compliance checks, and several tool invocations that a difficulty score never accounts for. A working router has to balance quality, reliability, latency, compliance, and cost simultaneously. Tuning for any one of those in isolation tends to make the whole system worse.
Latency follows the same pattern. According to the researchers, endpoint congestion, cache warmth, and hardware allocation frequently decide response time more than a model’s raw throughput does. Routing overhead compounds the effect: a system that re-evaluates which model to call at every step of a multi-step agent task accumulates delay that a single per-task routing decision would avoid.
IBM’s proposed fix swaps model classification for an optimization layer that produces a cost-accuracy frontier instead of one winner model. On AppWorld, its latency-tuned configuration reached 84 percent accuracy while spending $93 across 83 seconds, cutting cost by 21 percent and latency by 9 percent relative to running Sonnet alone, at a cost of four accuracy points. IBM says the router itself adds about six milliseconds of overhead per call.
The benchmark is IBM’s research team measuring IBM’s own routing framework, worth noting before treating the specific percentages as an industry constant. Cache pricing, endpoint congestion, and task composition differ by provider and by workload, so a result built on AppWorld will not automatically transfer to a different agent stack running different tasks. The mechanism the paper describes, that cache-read pricing can flip a rate-card comparison, is the part most likely to generalize beyond IBM’s own numbers.
Teams still choosing a model provider by published token price should rerun that comparison against their own production traffic, factoring in cache hit rates and per-step context reuse, before assuming the cheaper rate card produces the cheaper invoice.
IBM Research published this analysis on the Hugging Face blog on July 15, 2026.
