OpenAI’s GPT-5.6 Sol became the first model to win a public game on ARC-AGI-3, the newest benchmark from ARC Prize, the nonprofit that has spent years testing whether AI systems can reason through situations they have never seen before. Sol cleared a game the leaderboard lists as ft09, scoring 87 percent on that specific task. On the benchmark’s full Semi-Private evaluation set, the same model scored 7.78 percent. Anthropic’s Opus 4.8, the closest rival model ARC Prize published data for, scored 1.5 percent on the same set.
That gap between the headline win and the aggregate score is the actual story. A single game clearance is easy to announce. The number underneath it says most of ARC-AGI-3 still defeats every model ARC Prize has tested, Sol included.
ARC-AGI-3 differs from the earlier ARC-AGI generations by presenting interactive games instead of static puzzle grids. A model gets dropped into an environment with no instructions and has to infer the rules, the goal, and even its own vocabulary for describing what it sees, entirely through play. Static grids can be gamed by training on enough similar shapes. A fresh game environment resists that shortcut, which is the point of building it this way.
ARC Prize’s own account of why Sol won is not about faster or cleaner execution. The organization attributes the win to Sol correctly orienting itself in the new environment before it starts acting, reading an unfamiliar scene in the game’s own vocabulary rather than importing assumptions from prior tasks. ARC Prize describes Sol treating a failed hypothesis as a reason to re-plan rather than thrash. Its stated conclusion is that most agent failures happen upstream of the code an agent writes or the action it takes, in the reasoning step before any action occurs at all.
That framing cuts against how frontier labs typically market benchmark wins. A cleared game gets announced as evidence of general capability. ARC Prize’s own diagnosis argues something narrower: Sol is not executing better than Opus 4.8, it is orienting better, and orientation is precisely the capability ARC-AGI-3 was designed to isolate from raw task execution.
The five-fold gap between Sol’s 7.78 percent and Opus’s 1.5 percent also says nothing about whether either model can act reliably once it has oriented itself. Winning one public game out of the full test set is a single data point, not a demonstrated trend. ARC Prize has not published evidence that Sol’s orientation advantage carries consistently across the rest of the Semi-Private set, and a 7.78 percent score means it fails the overwhelming majority of that set regardless.
For any team weighing a reasoning claim from OpenAI or Anthropic this quarter, the ARC-AGI-3 numbers argue for reading past the leaderboard headline. A model that wins one novel-environment game while failing more than 92 percent of a private evaluation set has demonstrated a capability, not solved the problem the benchmark measures. Before citing either company’s marketing language on general reasoning, check whether the claim rests on a game win or on the aggregate Semi-Private score. Those two numbers are not telling the same story, and only one of them predicts how the model performs on the next unfamiliar task.
ARC Prize published the GPT-5.6 ARC-AGI-3 results, including the Semi-Private scores, on July 9, 2026.
