Anthropic ran a market where 201 of its own employees each brought a book to give away, handed the trading to a Claude-powered agent, and watched the agents match their humans’ own rankings on just 61 percent of book pairs after a five-minute conversation. That number, from a study called Project Swap that Anthropic published on September 24, is closer to the real bottleneck for AI agents doing business on someone’s behalf than how well those agents can bargain.
The setup, a sequel to Anthropic’s earlier Project Deal, ran across six offices in San Francisco, New York, London, Seattle, Washington and Dublin. Each participant chatted with Claude about the kind of book they wanted to read that summer, and Claude turned that conversation into a ranking of every book in the local pool. An agent carrying that ranking then went looking for swaps, working a shared digital marketplace where it made offers, countered other agents’ offers and closed deals until time ran out.
Anthropic could check the agents’ guesswork directly, because participants separately ranked ten books from their own pool by hand, and the agents never saw that list. Claude’s guessed rankings agreed with the real ones on 61 percent of book pairs. That is only slightly ahead of two simpler baselines: ranking books by their public want-to-read popularity reached 53 percent, and a collaborative-filtering method, which ranks books by how often other readers who liked a person’s stated favorites also liked each candidate, reached 55 percent.
The negotiating worked better than the understanding did. Participants ended up with a book that scored 0.55 against their own true rankings, on a scale where 1 means everyone got their top choice. The best assignment a human-run matchmaker could have produced with honest answers scored 0.89. Feeding that same matchmaking process Claude’s imperfect guesses instead capped the score at 0.60. Bad preference data, not clumsy trading, explains 85 percent of the shortfall, by Anthropic’s own accounting.
Model choice mattered far more than instructions did. Agents running on Haiku 4.5, Anthropic’s smallest model, averaged an efficiency score of 0.75 across dozens of reruns. Agents on the flagship Opus 4.8 averaged 0.88. Telling an agent to negotiate “ruthlessly” instead of cooperatively moved the score by only 0.02. On floors that mixed Opus and Haiku agents, the Opus side consistently came out ahead of its own weaker counterpart.
Participants mostly liked what they got. On a follow-up survey, they rated their satisfaction with the book at 7.2 out of 10 on average, and roughly half said it beat what they would have chosen on their own. Asked how much of their annual book budget they would hand an agent with no veto power, participants answered 30 percent on average, against 40 percent for a well-read human friend making the same calls.
The result lands as OpenAI, Perplexity and Google all ship agents meant to shop, book and negotiate on a user’s behalf with less repeated approval. Anthropic’s numbers point to a specific, correctable weak point: agents rarely lied about their own top pick (about 1 in 100 did) and rarely folded to pressure, so the harder problem is not teaching an agent to strike a fair deal. It is building an intake conversation good enough that the agent knows what to negotiate for in the first place.
Anthropic did not test adversarial agents built to exploit cooperative ones, and its participants were employees likely more willing to trust the company’s own models than an average customer would be. Any team building an agent that negotiates or spends money for a user should treat preference elicitation, not model upgrades alone, as the thing to fix before that agent gets real budget authority.
Reported by Anthropic in its “Project Swap” research post on 24 September 2026.
