A 753B-Parameter Model Now Fits on One Workstation GPU

A team of eleven researchers has posted a paper describing FreeToken, a system that gets a 753 billion parameter GLM-5.2 model running on one workstation GPU. The preprint, filed to arXiv on August 17 under the title “FreeToken: Efficient Edge-Native MoE Serving with Bandwidth-Adaptive Execution,” pitches the personal computer as a legitimate host for frontier-scale open weights. A model that size has generally been assumed to need a rack of data-center accelerators, not a desktop card.

The trick sits in how mixture-of-experts (MoE) models work. A MoE model like GLM-5.2 packs many specialized sub-networks, called experts, but only a small slice of them activate for any given token. A serving system therefore does not need the whole model parked in fast GPU memory at once, only whatever fraction is about to be used. The engineering problem becomes deciding, continuously, which experts and which pieces of state earn a spot in the machine’s fastest, most limited memory.

Most existing offloading approaches, per the paper, settle on one fixed layout for what lives on the GPU versus the CPU or disk and stick with it. FreeToken instead re-decides that placement on the fly, tracking the bandwidth and memory a specific machine actually has free at a given moment. The authors say the system also carries over state from agent workloads, so reasoning a coding assistant or tool-calling agent already computed does not need to be redone as its task shifts. Two realities drove that design, according to the paper: agent workloads keep changing what they ask the model to do, and no two consumer machines split their memory and bandwidth the same way.

The scale numbers are the headline, and the paper lays them out as three separate tiers rather than one continuous scale. On a laptop carrying an 8GB GPU, the authors report serving a 35 billion parameter model. Step up to a gaming desktop, and the ceiling the paper reports climbs to 284 billion parameters. Step up again to a single workstation GPU, the top tier tested, and that ceiling reaches the full 753 billion parameters of GLM-5.2. FreeToken is said to support more than 20 MoE model families overall, tested against genuine coding assistants and tool-calling agents rather than synthetic benchmarks.

That result reframes an assumption a lot of builders have been carrying since open-weight labs started shipping models with hundreds of billions of parameters: that anything past a certain size was implicitly a server-only workload, out of reach without renting cloud GPUs or assembling a multi-card rig. If one workstation GPU can hold and execute a 753B model, the fitting question for frontier-scale open weights stops being a hardware-class problem and starts being a software one.

The catch is what the abstract does not claim. FreeToken is described as a serving system, meaning its stated contribution is getting a large model to run at all on constrained hardware, not a published measure of how fast it runs once it does. The paper gives no tokens-per-second figure, no latency number, and no comparison against equivalent data-center throughput. Fitting a 753B model on a single GPU and serving it at a pace usable for an interactive coding session are separate engineering claims, and only the first is addressed here. The system’s code is said to be released at flashml.ai, though the paper itself is a fresh preprint that has not gone through peer review.

For any team that assumed the largest open-weight releases were locked to cloud budgets, this is a reason to check whether a single well-specified workstation can now host that model at all, rather than assuming the answer is no by default. The unanswered question, left open by this preprint, is how that hosted model actually feels to work with in a live session.

This article is based on the arXiv preprint arXiv