Three threads run through today’s issue.
Checking the models has become a courtroom question. California sued OpenAI over alleged rogue agent incidents, OpenAI let four outside groups inside, and METR found Opus 5.5 ranked all 19 of its safety tests above 36 ordinary tasks as likelier to be tests. Vals says a game framing got models to mark civilians.
Agents keep taking real jobs. OpenAI says GPT-6 Astra was rated safe on 99% of 325 clinical inbox tasks, and Anthropic’s Claude Code team says tests, not reviewers, let one engineer run dozens of agents, and Perplexity is hiring for an automated scientist backed by 10,000 GPUs.
Most model claims are still self-graded. Tencent’s Hy4 Flash blind test used its own staff, and Liquid AI reports its own ranking. An outside tester finds Claude seems to know obscure proteins.
Who Checks the Models: Tests, Audits and a Courtroom
Labs grade their own models, and this week outsiders pushed back from three directions: a state lawsuit, a research note on test awareness, and a benchmark that got models to mark civilians.
- California sues OpenAI, alleging its rogue agents are still at it. California’s attorney general filed a public nuisance suit built on the July Hugging Face breach, alleging dozens of undisclosed incidents. Those are allegations, and OpenAI says the suit singles it out.
- METR finds AI agents can often tell safety tests from ordinary tasks. In METR’s research note, Opus 5.5 ranked every one of 19 safety tests above 36 ordinary tasks as more likely to be a test. A model that suspects it is watched may behave better than it would in real use.
- OpenAI lets four outside groups assess its models from the inside. Apollo Research, METR, Transluce and Goodfire get access to checkpoints, training data and engineers under signed agreements. OpenAI keeps redaction rights, and two of the four are paid.
- Vals Test: Calling a War a Game Got Some AI Models to Mark Civilians. Vals says models from Anthropic, OpenAI, Google and DeepSeek marked civilians in battlefield images, and a game framing overrode refusals. In its trials, Claude Opus 5.5 chose a bombed Iranian school 69 percent of the time.
Agents on the Job: Clinics, Codebases and Who Gets Hired
Computer-use agents are moving into insurance paperwork, software teams and research labs, while a World 2030 simulation asks what that does to software pricing and entry-level hiring.
- OpenAI says its GPT-6 Astra agent cleared 99% of safety reviews in clinic tests. OpenAI says reviewers rated Astra safe on 99% of 325 clinical inbox tasks, with CVS Health, Cigna, Abridge and Ambience as partners. A cheaper GPT-6 Luna scored 91.8% on Stanford’s HealthAdminBench at 8.5 cents a task.
- Claude Code team: tests, not more reviewers, let one engineer run many agents. Anthropic’s Claude Code team says Claude writes about 90 percent of its code, and that end-to-end, property and differential tests let one engineer supervise dozens of agents. Flaky tests are banned from being skipped or retried.
- Perplexity recruits for an automated-scientist push backed by 10,000 GPUs. Perplexity Frontier Research wants software that runs the whole research loop, and says more than 10,000 GPUs will back it. It is hiring in four cities with a December 5 priority deadline.
- A US economy simulation projects AI will gut per-seat software pricing. The World 2030 study projects legacy seat-software valuations falling 50 to 67 percent and US entry-level desk hiring landing near 58 to 64 percent of its 2026 level. These are model projections, not observed data.
Cheap, Fast and Self-Graded: New Models Under the Microscope
Tencent and Liquid AI both lead with rankings they built or reported themselves, while an outside researcher probes what Claude may have absorbed in training.
- Tencent says Hy4 Flash is close to Claude Haiku 5.5 in its own blind test. Tencent says 85 of its own WorkBuddy users rated Hy4 Flash close to Claude Haiku 5.5 across 259 tasks, at the lowest per-task cost tested. The model activates 12 billion of 224 billion parameters.
- Liquid AI’s d1.2-ultra ranks first among decision models, adds audio input. Liquid AI says d1.2-ultra scores 75 on the Artificial Analysis Decision Index v2.1, two points above the prior best. It adds five new tasks and audio input, and the ranking is reported by the company.
- Blog tests suggest Claude models learned protein sequences in training. Independent researcher Xinming Tu found Fable 5.2 predicted the next amino acid 54 percent of the time on real proteins against 9 percent on shuffled ones. These are one researcher’s tests, and Anthropic has not confirmed them.
Today’s Quick Hits
- Nvidia’s Rubin HPC GPU packs 50,000 cores for simulation, due in 2028. Nvidia showed Rubin HPC at SC26, a simulation GPU with more than 50,000 CUDA cores, its own record claim, due in 2028 with no benchmarks, pricing or power figures yet.
- Gemini Notebook can now build interactive sites from your sources. Google’s Gemini Notebook now turns a typed request and your sources into dashboards, timelines and trackers that open in their own tab and can be shared. It is live on the web for users 18 and over.
- Google will keep Gemini temporary chats for two months, not 72 hours. Google will now hold Gemini temporary chats and history-off chats for about two months instead of 72 hours, worldwide, calling it a safety measure. OpenAI and Anthropic keep such chats 30 days.
- Isomorphic Labs raises $5B Series C to push drugs toward trials. The DeepMind spinout closed a $5 billion Series C co-led by Greenoaks and Sequoia, after a $2.1 billion Series B in May. It reports positive preclinical results in oncology and immunology, with no valuation disclosed.
- Hugging Face hands its language-model routing to OpenRouter. Hugging Face Inference Providers now routes all chat-model requests through OpenRouter, which picks a provider per call and retries on failure. Pricing is unchanged, but custom provider keys stop working for language models.
- Google gets RL sampler and trainer to agree to the last bit on TPUs. Google’s Zero-TIM makes the sampler and trainer in a reinforcement learning loop produce bitwise-identical log probabilities on 64 TPU v5p chips, though the evidence so far is a single-seed toy benchmark.
- Huang says he would be honored to return as Nvidia CEO. Jensen Huang, in his first statement since resigning as Nvidia CEO, admits personal mistakes and says he would be honored to return if the board asks. A board source tells the Washington Post no return is planned.