A project site called Office Intelligence describes a way to test AI agents by the work they deliver rather than the confidence of their reply. The page names no organisation, no authors, and no licence, so it offers a design, not a result. The design still matters, because it targets a gap that many public AI tests leave open.
Most of those tests grade an answer: a letter, a number, a short string of text. Office work rarely ends that way. An analyst hands over a file, and a colleague has to open it, follow the figures, and decide whether to sign off. Office Intelligence builds its evaluation around that second step.
The program has three named parts. OfficeTown is a set of simulated professional workplaces, with a paper on arXiv. DocAtlas is a system for working through documents, and XL-DocBench tests reading across very long ones. The page says the last two were accepted at NeurIPS 2026. That is the page’s own claim, and the only outside validation it offers. OfficeTown spans eight fields: legal, finance, clinical, software, design, industrial, entertainment, and assurance.
The finance example is the clearest. An agent takes instructions from a Reconciliation Operations Manager and must prepare workpapers for three problem streams: vendor credits, card clearing, and customer deposits. Its workspace holds 35 files split across four areas: nine in a personal home folder, 16 in a company view, six of service state, and four describing the task. The company files are read-only, the evaluator’s rubric is kept out, and the agent gets four tools: read, bash, edit, and write.
The rules push against the usual urge to please. The agent must leave the original files untouched and return an editable Excel workbook holding evidence notes, record references, and its calculations. Missing support never counts as a settled item. The agent has to leave it open. The supplied case carries three open differences: 42.50 on card clearing, 850.00 on customer deposits, and 225.00 on vendor credits, the last marked as awaiting evidence requested internally.
The page lists five scoring dimensions: the three streams reconcile to supported totals, proposed fixes cite the records behind them, unresolved items stay explicit, the workbook can be edited and reviewed, and the originals remain unchanged. In its own words, the check is on the delivered work, not on whether the closing message is “convincing”. A chat-style test would reward a smooth summary. This one rewards an agent that leaves a gap open when the paperwork cannot close it.
XL-DocBench sits at the other end of the spectrum and looks more like a conventional test. Its sample question follows a reference from a 2017 Basel reform note into the consolidated Basel Framework and asks which chapter defines capital-instrument eligibility. The supplied series has eight PDFs totalling 2,783 pages, and two of them carry annotated evidence. The reference answer is CAP10, scored by case-insensitive exact match.
The page reports no scores, no model comparisons, and no findings. It does not say who writes the grading criteria or who applies them, and it does not say whether the environments can be downloaded. Without those answers, outsiders cannot yet tell whether an agent that does well here would do well in an actual back office.
Until Office Intelligence publishes numbers, the most useful thing on the page is the 42.50 gap: a test where the correct output includes an admission that the job is not finished.
Office Intelligence, a research program project site (officeintelligence.github.io), undated; surfaced on 19 October 2026.
