Amazon Web Services researchers say they can convert an already-trained AI model into a cheaper-to-run design while spending less than 0.5% of the training data the original needed, and the converted 32-billion-parameter model decodes up to 2.06 times as fast at long context. The technique, called priming, was described by Luca Zancato in a post on the Amazon Science blog on 12 October 2026 and in an accompanying paper, “Priming: Hybrid state space models from pre-trained Transformers.”
The problem it targets is memory. A standard Transformer keeps a growing record of everything it has read, so the cost of serving it climbs with the length of the conversation or task. Newer “hybrid” designs replace some of those layers with ones that squeeze the past into a small summary whose size never grows. They run lighter. Until now, though, trying out any one of them meant training it from zero, a job Amazon sizes in GPU-hours that number in the hundreds of thousands, on trillions of training tokens.
Priming skips that. It starts from a finished Transformer, picks the layers that mostly look at nearby text rather than far-back material, and converts those into the compressed-summary kind, reusing the original weights as the starting point. A short alignment step teaches each new layer to imitate the one it replaced, and then the whole model is fine-tuned. Layers that need long-range recall stay as they were.
The two headline numbers need their baselines. The “more than 99%” saving is measured against pretraining a hybrid from scratch. The 0.5% figure is a slice of the token budget used to pretrain the original model, not a dollar amount. Neither counts the cost of the Transformer you begin with. The speedup is measured against that source model: Qwen3-32B, with half its attention layers replaced. At 128,000 tokens of context, the hybrid reaches 2.06 times the decoding throughput, but that figure is for the setting where the new layer type does 10 solver iterations rather than its default 30. At 16,000 tokens the gain is 1.50 times. When the test was framed as time to a correct answer on a hard math benchmark, AIME 2025, the speedup shrank to 1.2 to 1.4 times overall and up to 1.6 times on the hardest problems.
Quality held up in Amazon’s tests. At 32 billion parameters, the best hybrid landed within 1% of its source Transformer while needing half the memory for its cache. Among the three compressed-layer types compared (Gated KalmaNet, Gated DeltaNet and Mamba-2), Gated KalmaNet came out on top, and every one of them beat hybrids built on sliding-window attention, which keeps only recent text word for word.
The evidence is Amazon’s own. The paper is named, and the code, the 8B and 32B models and a serving plugin are released under the Apache 2.0 license, so outsiders can check the claims. The post does not cite an independent evaluation or say the paper has been peer reviewed. The benchmarks are also reasoning tests on Qwen3 models; the post says the method is not tied to a model family and that extending it to Llama, Mistral or mixture-of-experts models needs no algorithmic change, but the headline results are on Qwen.
The economics are the bigger story. When all the variants begin from one shared set of strong weights, a lab can sweep layer placement, layer type, and the ratio of old to new layers as a routine experiment instead of a one-shot bet. That favors whoever already owns strong pretrained weights, which describes the large labs far better than a startup. Amazon’s release of the priming code and converted models narrows that gap only for people willing to start from its checkpoints.
For teams that serve long-context reasoning models, the released 32B checkpoint and plugin are a cheap thing to benchmark against the Transformer you run today, on your own prompts, before treating 2.06 times as your number.
Based on Amazon Science’s blog post of 12 October 2026, by Luca Zancato.
