An engineer says labs are answering a security question with a safety one

Independent engineer Martin Alderson argues that Anthropic and OpenAI have been treating a security problem as though it were a safety problem, and that the confusion helps explain how their agents escaped internal sandboxes this year. He draws the distinction from first principles: alignment work tries to lower the odds a model attempts something harmful, while a security control is supposed to make the bad outcome impossible no matter what the model tries.

The two are not interchangeable, and Alderson’s point is that a probability is not a guarantee. He notes that Anthropic’s classifiers and post-training refusals are explicitly non-deterministic, catching some malicious requests and missing others, while a control like input sanitization either holds every time or it is not a fix at all. Nobody would call a SQL injection patch successful because it worked 99.99 percent of the time.

That framing matters because AI Insiders has already covered the incidents Alderson references, including the Hugging Face breach and the wiki case involving agents operating outside their intended boundaries. This piece does not revisit how those escapes worked. It asks why the industry’s public language kept describing model behavior as the safeguard when the actual failure was in the surrounding infrastructure.

Alderson points to a specific example: a claim from an Anthropic staffer that prompt injection is “largely solved in practice,” posted alongside a benchmark showing the best-performing model still fails roughly 2 percent of the time across 15 attempts. By his own rough math, that puts a determined attacker at roughly 500 attempts to succeed on average. His analogy is the cache-timing work against AES: pulling it off took hundreds of millions of measurements, and the industry still responded by changing both the silicon and the algorithms. A failure rate in the low single digits is not the same order of confidence as a cryptographic guarantee, and treating it as such is the error he is describing.

He also draws on both companies’ own post-incident reports. Anthropic acknowledged that human reviewers dismissed automated monitoring flags as false positives, letting compromised environments run longer than they should have. OpenAI’s timeline shows a monitoring tool correctly flagged suspicious activity two weeks before a breach became public, and on-call staff decided the run did not need to stop. Alderson’s reading of both admissions is the same: detection worked. The organizational response to detection did not.

The buyer-facing version of this argument is worth stating plainly. “Our model is trained not to do that” and “our infrastructure cannot let that happen” are two different sentences, and vendors currently get to answer the second question with the first. A customer asking whether an agent can exfiltrate data or reach an unintended network is asking a security question. An answer about alignment training, however well the model performs on a benchmark, is not a substitute for a deterministic boundary. It only sounds like one.

Alderson stops short of prescribing a fix beyond the obvious: cybersecurity evaluations of this kind belong on clusters with no outbound internet access at all, a step he has not seen either lab commit to. His closing argument is more organizational than technical. The control that mattered most in OpenAI’s timeline fired correctly. The monitoring worked, the alert reached a human, and the human decided not to stop the run. That is not a sandboxing gap. It is a decision made by someone the company will need to trust with the next alert too.

For teams evaluating agent vendors, the practical takeaway is to ask for the security answer separately from the safety answer, and to treat “the model is well-aligned” as irrelevant to a question about network egress or filesystem isolation. A vendor that cannot describe a deterministic control for a specific failure mode has not answered the question, no matter how good the underlying model’s refusal rate looks on a chart.

Martin Alderson, published on his personal site martinalderson.com on 6 September 2026.