Baseten says its inference API for GLM-5.2, Z.ai’s open-weight model, now reaches peak speeds of 280 tokens per second, with average throughput around 100 tokens per second. That is more than double what the company’s launch-day API delivered. The number matters less for GLM-5.2 itself, a model AI Insiders has already covered, than for what it reveals about the business of serving open weights.
Baseten also built a separate “Fast” version of its GLM-5.2 API, tuned specifically to cut latency for coding tools and autonomous agents rather than to maximize raw throughput. The company says another round of improvements to its speculative decoding algorithm is coming, aimed at extracting more performance from the same checkpoint. Speculative decoding lets a smaller draft model propose tokens that the larger model verifies in batches, which is one of the main levers providers use to speed up generation without changing weights.
The model itself did not change between launch day and now. What changed is everything wrapped around it: batching strategy, decoding algorithm, kernel optimization, hardware scheduling. Two providers running the identical open-weight checkpoint can post throughput numbers that differ by a wide margin, and neither one is lying. The weights are a starting point, not a finished product.
These are Baseten’s own published figures, not measurements from an independent lab, and the company has not disclosed the hardware configuration or testing methodology behind them. A comparison against Together AI, Fireworks, or Z.ai’s own hosted endpoint would tell a buyer more than a comparison against Baseten’s earlier self. Absent that, treat the 280 tokens per second figure as a vendor claim worth verifying on your own workload before it enters a procurement decision.
Open-weight models were supposed to commoditize the model layer and push differentiation down to infrastructure. Baseten’s numbers are an early data point for that thesis. If the same weights can get twice as fast on the same provider within months of launch, the serving layer is not a solved problem. It is a live optimization target that providers will keep fighting over, and the gap between a fast host and a slow one can be larger than the gap between two competing open-weight models.
For a team choosing a host for GLM-5.2 or any other open-weight model, the model card is now the smaller half of the decision. Latency guarantees, the pace of a provider’s own infrastructure roadmap, and independent throughput testing on your actual workload matter more than the headline number in an announcement post.
Philip Kiely of Baseten detailed these figures in a post on X on July 27, 2026.
