Why Thinking Machines Built Inkling Without a Vision Encoder

Thinking Machines, the lab Mira Murati started after leaving OpenAI as chief technology officer, released Inkling as open weights on July 15. The 975 billion parameter model carries an unusual bet on how multimodal understanding should enter a language model in the first place.

Most systems that read images or listen to audio attach a vision encoder and an audio encoder, each pretrained separately in advance, then bolt both onto a language model through a projection layer. Inkling skips that step entirely. Images and audio join the text sequence directly and pass through the same 66 layers, trained together with everything else in a single run. ByteByteGo’s architecture breakdown of the model, published August 18, traces how both pipelines actually work.

Each image splits into 40 by 40 pixel tiles first. Every tile then feeds into a compact four-stage network, the hMLP stem, before joining the token stream. That stem costs less than one percent of the compute a simpler patch-handling approach would need, per the paper Thinking Machines cites for the technique. No large pretrained vision model does the heavy lifting beforehand. A lightweight conversion step trains inside the same run as the rest of Inkling instead.

Audio enters as a mel spectrogram, the standard grid representation of sound, with frequency along one axis and time along the other. Each loudness value then collapses onto one of a small set of discrete steps, the quantization Thinking Machines calls dMel. Rounding requires no training, so the entire audio front end reduces to a fixed conversion rather than a model of its own.

Thinking Machines’ own materials use two different labels for this setup. The launch post calls it encoder-free. The model card, by contrast, labels the image path a hierarchical patch encoder. Both are accurate: encoder-free means no large encoder pretrained separately beforehand, not that no processing happens at all.

The payoff is architectural. One training run produces a model whose image and audio pathways exist from the start, instead of a pipeline where a vision team and an audio team separately pretrain encoders and hand off frozen outputs. The cost is that Inkling’s multimodal handling depends entirely on what shows up in its own general-purpose training data, since it never inherits a dedicated encoder’s own pretraining run. Thinking Machines used the same dMel audio path and the same 40 by 40 image tiles inside a real-time interaction system it described two months earlier, so this design was not built for Inkling alone.

A second unusual choice concerns something more basic: how the model knows where each token sits in a sequence. Most open models released in the last couple of years lean on Rotary Position Embedding, or RoPE, rotating each token’s internal representation by an angle set by its position. Thinking Machines instead uses an older relative approach: it assigns a learned number to the gap between any two token positions, rather than encoding each token’s absolute spot, and caps that gap at a fixed ceiling so any distance beyond roughly 128 tokens reuses the same learned number.

A token pair sitting 900,000 positions apart inside Inkling’s million-token context window reuses a number the model already met thousands of times in training, instead of extrapolating into territory it never encountered. That sidesteps RoPE’s well-documented weakness at untrained sequence lengths. It also broke compatibility with years of serving infrastructure: engineers on the vLLM inference project ended up writing fresh integration code just to support Inkling’s scheme.

Put the two choices together and a pattern emerges. Inkling reads less like a bid for the top of a leaderboard and more like infrastructure for teams that want to retrain a multimodal model from the ground up. Thinking Machines has acknowledged that rival models, some open and some closed, currently outperform Inkling overall.

What Inkling offers instead is an Apache 2.0 license, a quantized checkpoint that fits on four Nvidia B300 cards rather than the full version’s eight, and a multimodal path that never requires assembling separate encoder-training pipelines before a customization run can start. That combination competes less with frontier labs chasing benchmark headlines and more with the open-weight ecosystem built around bolted-on encoders, for engineering teams that would rather retrain one integrated model on proprietary image, audio, and text data than stitch several pretrained components together. It also lines up with Tinker, the fine-tuning service Thinking Machines shipped before Inkling existed.

Teams evaluating open-weight models for a genuine retrain, not just a fine-tune, should weigh Inkling’s single-pipeline multimodal design against the assembly cost of the encoder-plus-projector approach most competitors still use. Anyone planning to serve it in production should also confirm their inference stack has caught up to its relative-position scheme before committing traffic to it.

ByteByteGo published this architecture analysis of Thinking Machines’ Inkling model on August 18, 2026.