Linum, a two-person research lab building video models from scratch, says it trained a text-to-image model for $21,038 in compute using a design it calls Pyramid-JiT. The lab published the results on its field notes blog at linum.ai, framing the run as a proof of concept on the way to its next video model.
The bill is easy to follow. Pretraining took 8,139 H100-hours on 64 of Nvidia’s H100 chips, and post-training added 276 more, for 8,415 hours at $2.50 each. That works out to $20,348 for pretraining and $690 for the polish afterward, restarts included.
The idea starts with a cost problem. Image and video models cut a picture into small pieces called tokens, and the work of comparing every token with every other grows with the square of the count. Fewer tokens means cheaper training and cheaper generation. Most models shrink the picture first with a separate compression model before the main network ever sees it. Linum argues that step has a ceiling, and that making the compressor better can even make the final images worse.
So Pyramid-JiT drops the compressor and works directly on pixels. Each token covers a 32 by 32 patch of the image, where most models use 16 by 16. A 512-pixel image now costs the same number of tokens a 256-pixel image did before, a fourfold cut in what the network has to compare. Linum says the design reached good quality in 11 times fewer training samples than Linum v2, its earlier model built on a standard compressor and a conventional diffusion network.
The pyramid part is about teaching. Because the network works on pixels, every layer is effectively drafting the final picture. Linum attached small readouts partway through that predict quarter-size and half-size versions of the image, which gives the early layers a stronger learning signal. It also changed the noise schedule. The first 80 million samples leaned on heavily scrambled images so the model could learn rough structure, then the range widened so it could pick up fine detail.
Training ran for 610 million samples at 512 pixels, then 50 million at 1024 pixels. The image network has 2 billion parameters, paired with a 4 billion parameter text encoder, and Linum released the model under the Apache 2.0 license.
The limits are stated plainly. The model never saw rendered text, so it cannot write words or make posters, a choice Linum made because text-heavy images do not help animation tools. It also struggles with full bodies, crowds such as six cars, interactions between several people, and general world knowledge. Linum expects a larger model, around 7 billion parameters, and more post-training to ease those problems.
Linum’s write-up shows a gallery of 249 samples and its own validation loss curves. It does not include a head-to-head comparison against other image models, so the quality claim rests on the lab’s judgment for now.
The stake is the price of entry. A five-figure bill for a from-scratch image model sits inside a seed-stage budget, and the open question for anyone planning video work is whether the 32 by 32 patch trick holds up when the model has to handle motion, which Linum has not yet shown.
Reported by Linum on its field notes blog at linum.ai, in the post “Training a Text-to-Image Model for $21K” (no publication date given).
