Roughly 90 minutes of training on one RTX 5090 was all an independent researcher publishing as mvakde needed to build a small transformer from nothing and scored 44 percent on the public ARC-AGI-1 evaluation set. The entire run, training plus inference across all tasks, cost roughly 67 cents in compute. The same architecture reached 7 percent on the harder ARC-2 benchmark.
This is a task-specific model built and tested on ARC puzzles, not a general-purpose language model, so the comparison to frontier LLMs needs a caveat the source glosses over. A system trained from scratch on exactly the benchmark it is graded on is playing a different game than a chatbot answering a puzzle it was never optimized for. The 44 percent figure beats many general LLMs on the same public leaderboard, but it is not evidence the small model is more capable in any broader sense. It is evidence that ARC-1, restricted to transformers with no synthetic data and no pretraining, is more tractable than the leaderboard’s dollar-per-task numbers suggested.
The result also has not been independently reproduced or peer reviewed. It is one developer’s self-reported number, posted on their own blog as the third entry in a series on ARC-AGI, with code published on GitHub for anyone to check. The author’s earlier post in the series went viral after researchers including Lucas Beyer and Jeremy Howard debated it on X, which is the kind of scrutiny that substitutes for peer review in this corner of the field but is not the same thing.
The technical changes behind the jump from an earlier 40 percent result center on representation, not scale. The model uses 3D RoPE positional embeddings plus a learned per-task embedding to let a single small transformer generalize across ARC’s roughly 1,000 puzzles, each governed by a different rule. Ablations removing either the 3D RoPE or the per-task embedding drop the score to 25 percent, which the author reads as evidence that representation choices, not architecture depth or training tricks, are driving most of the performance. Switching the optimizer from AdamW to NorMuon and moving to flash attention with variable-length training cut the cost sharply without hurting the score.
The author also added non-overlapping puzzles from ARC-2 into the training set, filtering out the 773 tasks that overlap with ARC-1’s evaluation set to avoid leaking answers. Without that extra data the model still scores about 40 percent, at roughly double the compute cost.
The cheapest interesting result of the year on this benchmark is also an argument about what the benchmark measures. ARC-AGI has run for six years with a seven-figure prize attached, and frontier labs have reported dollar-per-task costs on the same leaderboard that run orders of magnitude higher than 67 cents. When a model that costs less than a cup of coffee to train matches systems built by well-funded labs, the gap says as much about how those labs allocate compute and about what the leaderboard actually rewards as it does about the small model’s architecture.
The author’s stated goal is sample efficiency, not raising the ARC score for its own sake: finding the limits of what transformers can do on a benchmark with almost no training examples, and cutting the cost of each experiment so more people can run the ablations. The blog post includes a full list of architecture and training changes and an open invitation for others to try to push the approach past 65 percent.
For teams evaluating small, cheap, from-scratch approaches to narrow reasoning tasks, this result is a reason to test whether representation choices like per-task embeddings generalize to your own domain before assuming you need a larger pretrained model.
Reported by the independent researcher known as mvakde on their personal blog, mvakde.github.io, in a post titled “44% on ARC-AGI-1 in 67 cents.”
