Google Research built a way to generate tool-use training data that starts with the answer, not the question, and says the resulting method produces working API chains at close to a 100 percent pass rate. The paper, called ToolGrad, was presented at ACL 2026.
The order matters here. Earlier systems, including ToolBench and ToolACE, worked forward: sample a pool of APIs, invent a plausible user request, then send a search agent hunting through possible tool calls until one resolves. Google’s researchers call that approach wasteful, because most of the value gets thrown away once the agent finds a working trajectory and only the final path gets kept for training.
ToolGrad runs the opposite direction. Four modules do the work in sequence. An API Proposer narrows a sampled set of tools down to promising candidates for the next step in a workflow. API Executors test those candidates in parallel and log what happened. An API Selector reads the results and keeps the single best call, a step Google frames as a “textual gradient”: directional feedback in plain language rather than a number. An LLM Updater then rewrites the synthetic user query and the AI’s response to match the new, longer tool chain. Repeat the loop and the output is a verified workflow with a query and answer built to fit it, rather than a query invented in a vacuum and then searched for.
Google tested the method against ToolBench’s library of more than 16,000 real-world APIs, generating a compact dataset it calls ToolGrad-500. It used that dataset to fine-tune three sizes of its open Gemma-3 model, at 1 billion, 4 billion, and 12 billion parameters, and evaluated all three on the Berkeley Function Calling Leaderboard, a benchmark built on a different tool set than the one ToolGrad-500 was generated from. That distinction matters: the models were being scored on APIs they had not trained against, not simply repeating memorized patterns.
On that specific evaluation, the 12 billion parameter version scored 83.1, close to Gemini 2.5 Pro’s 83.2, ahead of Claude 4.5 Opus’s 82.8, and well ahead of GPT-5’s 74.4. That is a tool-calling score on one out-of-distribution benchmark, not a claim that a 12B open model now performs like a frontier proprietary system across reasoning, coding, or general chat. Google also reports that ToolGrad-12B beat Hammer-2.1-7B, ToolACE and other open models built specifically for tool use, and that a student model fine-tuned on data Gemini 2.5 Flash-Lite had generated went on to outperform that flash-lite “teacher” on the same test.
These are Google’s own benchmark numbers, run on its own dataset and its own fine-tuned models, with no independent lab cited to confirm the BFCL scores. The comparison models (Gemini 2.5 Pro, Claude 4.5 Opus, GPT-5) are also each vendors’ own releases, evaluated by Google rather than by a neutral third party.
The practical claim worth testing is narrower and more useful than “small model beats the giants”: if answer-first data generation really does produce near-perfect pass rates at low cost, teams building narrow tool-calling agents may no longer need a frontier model’s API bill just to get reliable function calls on a bounded set of tools. Anyone fine-tuning an agent for a fixed toolset, an internal API surface, a specific SaaS integration, should treat BFCL parity on unseen tools as the number to reproduce before trusting a compact open model in production, not as evidence the model has closed the gap on everything else.
Reported by Google Research on its research blog, describing a paper presented at ACL 2026.
