Qwen, Alibaba’s AI lab, has published RecreationWorld, a benchmark that asks AI agents to rebuild working applications across five platforms (Ubuntu, macOS, Windows, Android and web) while a running reference version checks their output. The companion test set, RecreationBench, covers 250 tasks and grades each attempt on both code behavior and what the interface actually looks like on screen.

According to the project’s GitHub page, the best performer in Qwen’s own results table, OpenAI’s GPT-6 Astra, averaged 58.06 percent across both scoring methods and produced a flawless rebuild on just 2.8 percent of apps. Anthropic’s Claude Opus 5 came second at 44.16 percent. Qwen’s own current model, Qwen3.8-Max-0902, trailed both at 34.80 percent, a gap the company disclosed alongside everyone else’s scores rather than omitting.

That willingness to publish a benchmark where its own model loses is unusual for a vendor, and it says something about where the real bottleneck sits. Every model in the leaderboard, including the top scorer, struggled to close out a build completely rather than just get most of the way there: the “100 percent correct” column stays under 3 percent for every entry tested.

The task design forces agents to alternate between clicking through an app’s interface, writing the code that reproduces it, and checking their own build against the reference, rather than following one fixed script. Scoring runs entirely on whether the finished app behaves and looks right, so a team can use a different programming language or framework than the original and still pass.

Further down the same leaderboard, Moonshot’s Kimi K3 reached 31.41 percent, Anthropic’s older Claude Opus 4.8 scored 31.10 percent, Zhipu’s GLM-5.3 managed 24.38 percent, and Google’s Gemini 3.7 Flash came in at 21.12 percent. Qwen’s smaller Qwen3.7-Plus finished last at 9.15 percent, the widest drop-off on the table.

The 250 evaluation tasks split evenly across five operating environments, fifty tasks each for Ubuntu, macOS, Windows, Android and the web, and each platform is checked through its own native accessibility layer: AT-SPI on Linux, AXUIElement on macOS, UI Automation on Windows, UiAutomator on Android, and direct browser assertions on the web. Alibaba’s team estimates the cost of running each task per model, and prices vary widely: GPT-6 Astra averaged an estimated $115.80 per task in the results, more than nine times the $12.58 estimate for Grok 4.6. Qwen has released a Hugging Face dataset, a ModelScope mirror, and command-line tooling under an MIT license so outside teams can reproduce the runs themselves.

For any team benchmarking coding or computer-use agents against RecreationWorld, the near-zero perfect-completion rates across every model tested suggest the harder problem is finishing a build cleanly, not starting one.

According to Qwen’s RecreationWorld repository on GitHub, accessed September 22, 2026.