Epoch AI has built a coding benchmark that removes the one crutch most software agents lean on: the original source. MirrorCode, developed with the AI evaluation group METR, gives a model an existing program’s behavior and nothing else, then asks it to produce a working replacement from scratch. A rewrite passes only when its behavior lines up precisely with checks run after the fact, checks the model never saw while building its version.

The target set covers 25 programs pulled from different corners of computing:

The design choice worth dwelling on is the scoring mechanism itself. There is no rubric, no partial credit, and no model or human judge weighing in on style or approach. A held-out test suite either matches or it does not. Because the model never sees the original code, it cannot patch, refactor, or pattern-match against a diff the way agents do on repository-editing benchmarks like SWE-bench. It has to infer the entire specification from observed behavior and then build something that satisfies it. That combination, blind reconstruction plus binary output matching, is difficult to game precisely because there is no partial signal to overfit against and no reference implementation to peek at.

Epoch also sized the budget to the task rather than the other way around. Many software engineering benchmarks cap inference spending at roughly one to ten dollars per attempt, even for problems that would take a human weeks. MirrorCode does not. On one of the benchmark’s largest tasks, a model ran unsupervised for 19 days and burned through $2,600 in a single attempt. The standard leaderboard setup covers the 15 target programs drawn from the medium and large difficulty buckets, each built twice, once in Go and once in Ada, run three times apiece, with seven days and 10 billion tokens allotted per attempt.

Epoch is direct about where models currently stand: MirrorCode is not fully solved. The one fully reported result is Claude Opus 4.7 reimplementing gotree, a bioinformatics toolkit of roughly 16,000 lines of Go spread across more than 40 commands. Epoch estimates a human engineer working without AI assistance would need two to seventeen weeks for that task. Opus 4.7 finished in 14 hours for $251. Even that result carries an asterisk: the best-scoring gotree rebuild passed 2,000 of 2,001 tests, missing a single edge case tied to a niche command for date annotations. Epoch calls the reimplementation near-perfect rather than complete, which is an honest distinction given the pass/fail design the benchmark otherwise enforces.

A separate caveat concerns memorization. Because the target programs are existing open-source projects, models likely encountered their original codebases during pretraining, which could inflate scores on tasks that resemble recall rather than reconstruction. Epoch ran a memorization screen and found that models succeeded on programs that passed the screen and failed on programs where it flagged evidence of memorization, evidence against contamination dominating the results, though Epoch stops short of ruling it out entirely.

What the page does not address is a subtler limit of exact-output matching as a proxy for good software. Grading against one reference implementation rewards a model for reproducing that program’s specific behavior, including any quirks, edge cases, or outright bugs baked into the original. Matching a buggy original faithfully is not the same skill as writing code a human engineering team would want to maintain, and nothing in Epoch’s writeup weighs readability, security, or design quality against the exact-match score.

Epoch open-sourced its scaffold and 22 of the 25 target programs, which break down into 132 task instances once spread across six supported languages, and kept the remaining three targets private specifically to make the benchmark harder to game. Today’s newsletter includes several benchmarks built for the same reason MirrorCode exists: the previous generation of evals stopped separating capable models from merely well-tuned ones.

Teams benchmarking coding agents for 2026 procurement should treat a single strong MirrorCode result, like the gotree rebuild, as evidence of narrow long-horizon capability rather than proof that a model can be trusted to design new systems unsupervised.

Epoch AI’s Daniel O’Connell detailed the MirrorCode benchmark, co-developed with METR, on epoch.ai.