Independent AI researcher Alex Zhang has published a technique called speculative programmatic tool calling (sPTC), which starts executing certain tool calls before a coding harness has finished writing the code that invokes them. The goal is to shrink the latency that slow sub-agent and search calls add to systems built around code execution rather than traditional JSON tool calls. The reported gain is a runtime speed-up of roughly 1 to 1.2x, meaning the low end of that range is effectively no improvement at all.
The mechanism borrows from two older ideas: speculative execution in CPUs and speculative decoding in language models. As the model streams tokens for a code cell, a forked copy of the execution environment, which Zhang calls a shadow REPL, parses the partial code and identifies tool calls whose inputs are already knowable, either because they are literal values or because they depend only on variables computed from safe, side-effect-free operations. Those calls launch asynchronously in the background. If the finished code actually invokes them, the real run returns the cached result instantly instead of waiting.
Not every call qualifies. Zhang maintains an allowlist of functions considered safe to evaluate early; a call whose inputs depend on something like reading a file is blocked from speculation because executing it ahead of time could produce a side effect the real run never intended. On a mispredict, meaning the shadow REPL guessed wrong about what a variable would hold or whether a conditional would trigger, the speculated result is simply discarded and the tool call runs normally in the real environment. The shadow copy never touches the real program state, so a malformed or incomplete generation cannot corrupt the actual run.
The second piece of the technique works even without streaming enabled. Many code-generating harnesses write tool calls that look sequential but have no actual data dependency between them, such as two independent sub-agent calls issued back to back. sPTC acts, in Zhang’s words, as a naive JIT compiler that detects these non-blocking calls and runs them in parallel rather than one after another. Only calls without a dependency on each other’s output can be treated this way; anything that requires the prior call’s result still has to wait.
Zhang benchmarked the approach on Recursive Language Models, his own architecture for handling long-context tasks, using Qwen3-30B-A3B-Instruct-0527 on an 8xH100 server against the OOLONG and OOLONG-Pairs benchmarks. The 1 to 1.2x range came from five runs per configuration at both 4 and 8 concurrent requests. He does not publish comparable numbers for standard, non-agentic tool-calling setups, where he argues the technique adds little because the model typically has few tokens left to generate once a tool call is fully specified.
The workload shape that actually benefits is narrow: a machine serving a small handful of concurrent sessions locally, where decode is memory-bound and idle compute sits waiting on the next token, so speculation gives that spare capacity something useful to do, and high-volume serving systems where slow REPL calls or sub-agent latency can be overlapped with ongoing token generation. Teams running simple, low-latency tool calls or single-shot chat completions should not expect this technique to move their numbers, and should benchmark their own harness before adopting it.
Alex L. Zhang published this research on his personal blog, alexzhang13.github.io, in August 2026.
