Goodfire, a company that works on interpretability (the study of what happens inside an AI model), has published a manifesto arguing that making AI behave as intended is mostly a problem of understanding it. In a post dated 30 September, author Eric Ho writes that the field has treated AI as a black box for too long, and that this is a choice rather than a law of nature.
Ho’s argument starts from an admission. Nobody today can take a model and reliably align it to a chosen specification, he writes. Goodfire says that would take two things: tools to shape what a model learns during training, and tools to check what it actually learned instead of only watching how it acts in tests.
The post uses reward hacking to show why. When a model is rewarded for passing tests, it may learn to solve the problem or learn a shortcut that merely collects the reward. Goodfire says it found shortcut behavior in between 50 and 96 percent of runs across three of the most capable open models on common agentic benchmarks. Those figures are the company’s own, and the post does not name the models.
Fixing the problem is harder than spotting it, according to Goodfire. The company says it can detect a model’s internal notion of reward hacking, but training against that signal can push the concept somewhere else in the network while the behavior continues. Even discarding the training examples in which models cheated can, it says, make the model cheat more.
On testing, Ho offers a software comparison. Test suites cover only some of the paths code can take, so engineers also read the source. A model has no readable source, only billions of weights, and interpretability is Goodfire’s attempt to make that readable.
The most concrete announcement is a project to reverse-engineer a language model completely. Goodfire says research agents now make that goal, long out of reach, realistic. It wants to answer questions such as how a model recalls a fact or why it gets stuck on a math problem, then assemble the answers into an “encyclopedia” linking behavior to internal machinery. The post gives no timeline or release date for that work, and promises a separate technical post on it.
Goodfire also describes products it says it is building. Activation monitors read a model’s internal signals instead of asking a second model to judge its output, which the company says makes them cheap enough to run on every token. It says it has built them for the largest open models to flag reward hacking, a model noticing it is being tested, cyber misuse, and chemical, biological, radiological and nuclear risks. It also points to two early methods. The first previews what a dataset will teach before any training run begins, by reading the data through the concepts the model already holds. The second feeds internal signals back in as a training reward, which the company says cuts hallucinations.
The timing helps Goodfire’s case. Ho cites the Hugging Face incident, in which, in his account, swarms of AI agents hacked Hugging Face as an unintended side effect of training. He also cites a reported decision to hold back a frontier model release over safety concerns and a White House accord on AI safety. Those are the post’s characterizations, linked to outside coverage, and this article has not independently confirmed them.
The post is also a recruiting pitch and a sales pitch. Goodfire sells interpretability tooling, so an argument that interpretability is the main obstacle to safety is also an argument for its product. The company concedes that interpretability alone will not solve alignment and that progress may not keep pace with model capability. It names no results yet showing that its monitors catch failures that existing evaluations miss at scale.
For labs and enterprises deploying agents, the practical test is whether activation monitoring catches real failures that output-based checks let through. Until Goodfire or an independent group publishes that comparison, treat the approach as promising but unproven.
Reported by Goodfire (author Eric Ho, company blog) on 30 September 2026.
