OpenAI shipped an overhauled prompt caching system for the GPT-6 API, built to raise cache hit rates by default and give developers visibility they previously lacked into why a request did or did not get the discount. The centerpiece is a new Prompt Caching Dashboard: it breaks down, over time, what portion of an app’s inputs got the cached-token discount and what portion still ran at full price.
The mechanics matter because caching is where a lot of real GPT-6 spend actually lives. Agents that run for long stretches on a single task, such as refactoring a codebase or assembling a research document, resend the same system instructions, tool definitions, and prior context on nearly every API call. OpenAI reuses that shared prefix instead of reprocessing it, and says the discount on cached input tokens can reach 90 percent.
The new part is the reuse window. OpenAI is now applying the cache discount to eligible shared prefixes reused within 30 minutes, a change from the prior setup that developers had less visibility into. A prefix that goes stale sooner than that window closes still falls back to full-price processing, which is the gap the new diagnostics tool is meant to catch.
That diagnostics tool answers a question that used to require guesswork: why did this specific request miss the cache. It compares a request against a recent prior response and flags what changed, whether that is the model version, the tool list, a settings tweak, or the input itself. OpenAI’s own example output names “tools_changed” as the miss reason and reports the exact token count affected, in that case 5,629 tokens that had to be reprocessed instead of served from cache.
OpenAI also opened up more manual control for teams that want to tune caching rather than rely on the defaults. Explicit cache breakpoints let a developer choose which prompt prefixes are worth preserving. A new configuration_update option lets a team raise or lower reasoning effort between calls, for a harder step in a task or an easier follow-up, without invalidating the cached context, something that previously forced a full reprocess. Keeping tool definitions and their order fixed matters too: OpenAI is telling teams to stop removing tools outright and instead restrict which ones a turn can call with an allowed_tools setting. Fresh instructions should get tacked onto the tail of that context to supersede older ones, since editing text already in the prefix is what breaks the cache.
A prewarming option rounds out the toolkit. An application can load shared instructions or reference material into the cache during startup, ahead of a user’s first message, which shortens the delay once a genuine request lands.
None of this changes what GPT-6 can do. It changes what running it at scale costs and how visible that cost is. For a team already running persistent agents against the GPT-6 API, cache hit rate has been an invisible line item buried inside a token bill. A dashboard that separates cached from uncached tokens, paired with a diagnostics tool that names the exact reason a request fell out of cache, turns that line item into something an engineering team can actually optimize against rather than estimate.
Teams running long-lived GPT-6 agents should pull up the new dashboard before their next billing cycle closes and check whether tool-definition churn is quietly costing them the 90 percent discount on requests that should qualify.
Based on OpenAI’s official announcement, “Better prompt caching for GPT-6,” published on the OpenAI website.
