Every inference stack you have shipped against reports its own KV-cache hits, and none of those reports are self-verifying. That is the premise behind LLMTraceFX, an auditing tool built by developer Siddhant Khare, whose merged pull request 74 lays out a synthetic test suite designed to catch the gap between a runtime’s claimed cache reuse and what actually happened underneath it.
The stakes are direct for anyone running production inference. Vendors increasingly bill cached tokens at a steep discount on the theory that a cache hit means the model skipped recomputing that portion of the prompt. If the engine’s attestation is wrong, or if it is technically true but silent about the wrong questions, you can be paying for a discount that was never earned, or worse, quietly reading a response the system produced from a different, uncached, prompt path.
Khare’s fix is not another benchmark. It is a check on the claim itself. His auditor runs a fixed sequence of ten requests through a token-granular cache and, for each one, computes independently what a correct engine should have reused before ever looking at what the engine says it reused. That independent calculation exists because, as Khare puts it, the runtime “cannot verify itself.”
A single well-chosen example does most of the persuading. In one case, a nine-token request duplicates an earlier one exactly. The engine reports eight of nine tokens reused and one token of fresh work, and the independent oracle agrees. That is what a legitimate hit looks like: expectation, attestation, and observed work all lining up.
Nearby, a decoy request runs the same length but with different token identities entirely. Here the engine and the oracle both land on zero reuse, and the system still has to process all nine tokens. The two cases are indistinguishable if you are only counting request length or reading a hit rate off a dashboard. Only checking exact token identity, not proximity or similarity, separates them.
The auditor pushes past that distinction into causes. A verdict of “miss” can mean three unrelated things: no candidate ever existed, a candidate existed but sat outside the request’s namespace, or a candidate existed and was later evicted under capacity pressure. Khare’s suite assigns eviction its own typed outcome, backed by proof that a prior entry actually resided in the cache before it left, rather than folding every zero-reuse case into one generic bucket.
None of this, Khare is careful to note, proves anything about real-world latency, GPU memory savings, or how a specific vendor’s production cache behaves at scale. The demo is a deterministic, synthetic control built to validate the auditing method itself: that a claimed hit is checked against an independent oracle, the actual work the prompt path processed, whether cached and uncached outputs stayed identical, whether the task result remained correct, and whether the underlying evidence bundle matches its own checksums. Timing and memory fields in the published bundle are left null rather than assumed to be zero.
For engineering teams evaluating inference vendors on cost, the practical takeaway is to ask what backs a “cache hit” line item on a bill before accepting the discount math behind it. A hit rate without an independent oracle behind it is a vendor’s claim about its own system, not a measurement of one.
Reported by Siddhant Khare on his personal site, September 13, 2026, drawing on the merged LLMTraceFX pull request 74.
