A team of eleven researchers led by Guangyu Sun has published FLAT, short for Flexible-Length Aligned Transmodal Tokens, a method that folds images and text into one continuous sequence of tokens shared across both modalities. The pitch is architectural consolidation: one encoder output that can serve as a retrieval embedding, a captioning input, or a generation seed, depending on how much of the sequence a downstream task uses.
The mechanism rests on nested dropout, an ordering scheme that puts coarse, global semantics into the earliest tokens and pushes composition and fine detail into the later ones. Each image or caption becomes a sequence built by one vision-language encoder, capped at 256 tokens of 64 dimensions apiece. Selecting a shorter prefix at inference time is a direct compute-for-detail trade, not a swap between models: the paper reports that the same sequence supports text-to-image synthesis, captioning, interpolation, and vector arithmetic just by changing how many tokens get read.
On the benchmarks the authors report, a task-adapted version of FLAT reaches 83.1 on GenEval and 138.6 CIDEr on COCO captioning, with a linear classifier trained on frozen FLAT features hitting 81.8 percent top-1 accuracy on ImageNet using 64 tokens. These are the authors’ own numbers from a project page and an accompanying arXiv preprint, not independently verified results, and the page includes no comparison table against competing unified encoders.
The more interesting number sits in retrieval. According to the authors, a single 64-dimensional token already reaches 86.4 recall at 5 for COCO image-to-text search. Expanding to the full 256-token, 16,384-dimensional representation moves that score by less than a point. If that holds up outside the paper’s own test set, it means the expensive, high-dimensional version of the representation buys almost nothing over the cheapest one for search specifically, which is the opposite of what most retrieval systems assume when they scale up embedding size.
That efficiency claim matters more than the headline benchmark scores. Production systems that need both semantic search and image generation typically run two separate stacks: a contrastive embedding model such as CLIP for retrieval, and a diffusion or autoregressive decoder for generation, each tuned and maintained on its own. FLAT’s argument is that collapsing both into one encoder removes an entire integration layer, not just a modeling trick, provided the shared representation does not sacrifice accuracy on either task to serve the other.
The authors also report zero-shot behavior the training mixture did not explicitly target: prompts written in English, Chinese, or Spanish that carry the same meaning decode into matching images, and an emoji alone can anchor that same latent space. Four sampled video frames encoded jointly produce captions that summarize action across the clip rather than describing one frame. On composed image retrieval, where a reference image and a text edit are combined, FLAT reaches 44.0 Hit@1 on the CIRR benchmark using a single token, according to the paper.
None of this has cleared peer review. The work is hosted as a project page with an accompanying arXiv listing dated 2026, and every figure on the site comes from the authors’ own evaluation runs rather than a third-party leaderboard. Teams building multimodal search or generation products should treat the retrieval-stability result as the one worth reproducing first: if a single small token genuinely holds its own against a 256-times larger representation, that changes the storage and latency math for any system indexing images at scale, well before the generation side of FLAT is worth adopting.
Meta AI research page, September 17, 2026.
