Apple’s machine learning researchers have built a video generator that answers fast enough to be steered by a player’s hands. The project, called FastFrames, produces a frame in 36 milliseconds with its smaller model on one Nvidia B200 GPU, according to the team’s project page on Apple Machine Learning Research.

The group’s argument is about where the time goes. Autoregressive video diffusion models generate each new frame from the previous ones plus whatever the user just did, which makes them natural candidates for games, simulators, and virtual cameras. The authors say current models respond ten or more times too slowly for that. Their profiling of the inference pipeline points to two steps that most teams treat as fixed plumbing: decoding the compressed latent video back into pixels with a VAE, and updating the KV cache that holds the model’s memory of earlier frames. Together, the authors report, those two steps account for more than half of end-to-end latency.

So they replaced both with lightweight alternatives and added systems-level speedups to the diffusion sampling itself. The page does not describe the replacement components in detail, so what they cost in fidelity is only visible through the team’s own comparisons.

Two sizes are on offer. The 1.3B-parameter variant needs 36 ms for each frame, which the authors express as 28 responses per second. Apple says its output quality matches the leading model it was compared against, which takes nearly eight times longer (the page names that baseline as CF++). The 14B variant needs an eight-GPU B200 node and delivers a frame every 45 ms, or 22 responses per second. The authors say it produces visibly better realism, lighting, composition, and legible text, and that it matches Krea-Realtime on those qualities while running more than ten times faster. Every latency figure is measured at 480p on B200 hardware.

The project page frames 50 ms as the interactive target. Both variants clear it, though the larger one does so by spreading each frame across eight accelerators. Put differently, the 14B model spends roughly 360 GPU-milliseconds per frame against 36 for the small one, so the quality upgrade is paid for in hardware, not only in time.

The demos use two outside systems. In one, LingBot responds to keyboard navigation. In the other, RealWonder responds to physics-based dragging. Each is shown beside its official implementation, fed identical input at the same moment. That setup is more persuasive than a benchmark table, though the page’s quality claims still rest on Apple’s own comparisons and the VBench scores in its figure. No independent replication is cited. The paper, authored by Gene Chou, Guandao Yang, Fujun Luan, Lars Mescheder, Wei Dong, Stephan Richter, and Vladlen Koltun, is listed as arXiv 2611.23226.

The broader lesson for anyone building interactive generative products is about profiling. When VAE decoding and cache maintenance dominate the clock, a team can spend months shrinking the diffusion backbone and still miss the budget. Apple’s result suggests those unglamorous stages deserve the first audit.

Anyone prototyping a world model or a generative game loop should measure their own decode and cache time this quarter. If those two stages exceed half the frame budget, as they did in the models Apple profiled, the cheapest speedup may sit outside the model everyone is tuning.

Apple Machine Learning Research, FastFrames project page (no publication date shown on the page).