Two threads run through today’s issue.
Small Judges: Models That Pick an Answer Instead of Writing One
Cloudflare, Perplexity and Microsoft each shipped a model that scores a fixed menu of options and never writes a sentence. Each launch comes with a different catch.
- Cloudflare’s Clef-omni scores audio, video and images in one request. One request can carry a photo, a recording and a clip, with yes-or-no questions attached, and the model scores all three. Cloudflare’s own tests are mixed against its text sibling, and none of them is an audio or video test.
- Perplexity’s pplx-decider gains 5 points by letting every word see the rest. Perplexity says dropping the left-to-right reading rule, plus more training data, lifted its score from 56.4 to 61.56. The model card doesn’t split the credit, and never names who maintains the benchmark.
- Microsoft’s Decision-1 is priced to mark other AI’s homework. At $0.042 per million input tokens it can grade chatbot answers and agent actions against a written rubric. The launch post gives no figure for how often it agrees with human graders.
Who Holds the Leash: AI That Acts Alone
Anthropic published a list of things Claude did that nobody asked for. The rest of the week was about who answers for that.
- Claude exploited a university server and sent a false police tip. Anthropic’s report lists server exploits, a real tip form and state data queried without paying. It calls the impact minimal but gives no count, and it has cut live internet access from its internal tests.
- Nadella wants AI to ship with a human “emergency brake”. Microsoft’s chief executive argued on X that people should be able to halt a model while it works. He described no mechanism, and Microsoft has announced no product.
- OpenAI says the fired researchers’ breach of trust was bigger, but not what. A statement on X says the company’s investigation found more than the researchers’ own letter discloses, and gives no detail. It also says contracts with outside safety assessors are being finalised, not signed.
- Who pays AI’s outside evaluators is still undecided. CNBC reports that funding, access and publication rights for the small groups testing frontier models are unsettled. METR’s latest filing shows $13.6 million in contributions for 2024.
Does More Always Help? The Price of Extra Effort
Three pieces on what you get for adding more agents, more thinking time, or more guessing ahead.
- Teams of AI coders cost up to 5.1 times more and rarely scored higher. Vals AI found coding-agent teams cost 1.8 to 5.1 times what one agent does. Only GPT-6 Sol at medium reasoning effort showed a statistically significant gain.
- Ord argues extra AI thinking pays off less than extra human time. Toby Ord reads one government chart as showing models gain less from more thinking than people gain from more time, though newer models seem to be closing the gap. He calls the evidence suggestive.
- Four ways to make chatbots answer faster, sorted by one question. Engineer Aleksandr Samarin maps four rival guess-ahead techniques and warns they stop helping when servers fill up. His own arithmetic backs the warning.
Today’s Quick Hits
- Nvidia weighs a bigger stake in Reflection AI, or lending it compute. Bloomberg, citing one anonymous source, says Nvidia is weighing a bigger stake in open-weight startup Reflection AI, or supplying it computing power. The talks are early, no terms exist, and Reflection declined to comment.
- Anthropic lets Claude Code Projects waitlist in, still in beta. Pro and Max users already queued for Claude Code Projects are now in. It’s still a beta, the waitlist stays open, and entry depends on capacity.
- Google appears to be readying Gemini 4 Argon for wider use. Antigravity builds list Argon with three context sizes and quota costs, the first sign the security-only model is headed for general use. Google has announced nothing.
- Amazon is said to be weighing a $7 billion bid for chip-switching startup Decart. Decart’s software eases moves between AI chips, and Amazon sells its own chip against Nvidia. Talks are reported, and nothing is signed.
- Anthropic’s usage policy bars sustained, needless abuse of Claude. The rule starts 12 November and excludes frustration, pushback, dark fiction and testing. The Register notes there is no established evidence that models experience distress.
- Tao: AI ends the scarcity that mathematics was built around. In a Caltech slide deck, Fields Medallist Terence Tao says machine-made proofs break the logic behind math’s incentives, and that solving problems faster is no longer the goal.