Are the insides of AI models really tangled? One researcher says no

Nick Cammarata, a researcher at the interpretability company Goodfire, has published an essay arguing that the inside of a neural network is much less tangled than the field has believed for years. If he is right, researchers can start reading models as they are. If the older view holds, they first have to build tools that pull a network apart before it can be understood.

That choice matters because interpretability, the work of finding out what a model is actually doing internally, is what safety arguments and debugging both lean on. A field that thinks its subject is natively messy plans for a long wait. A field that thinks the subject is organized, just in an unfamiliar way, plans to get on with it.

The puzzle is old. Individual neurons, the small units a network computes with, often respond to unrelated things. The best-known case, in an image model, lit up for pictures of cats and also for the grilles and bumpers of cars. The leading explanation was superposition: a network stores more concepts than it has neurons, so unrelated concepts end up sharing one. That implied arbitrary packing, a knot to be unpicked before anything else. The usual route is a second model, a sparse autoencoder for instance, trained to stand in for the network with the knot removed.

Cammarata’s alternative is what he calls recursive generalization. A neuron that learns one thing sprouts offshoots, the way a branch sprouts sub-branches, so its range stretches over related territory. His example is a neuron that detected the letters “iy”, then “diy”, then “do it yourself”, a phrase that contains no “iy” at all. Seen from one branch to the next, the spread looks random. Followed step by step, it is one idea getting broader.

What the essay actually tests is a mix, and the scale differs by claim. On InceptionV1, an image model the field long used as its standard specimen, Cammarata says ordinary architectural changes lifted the share of easily interpretable neurons in the last layer from almost none to 99 to 100 percent, while the model’s size and its training data stayed the same. He trained versions across a 300-fold range of sizes and reports that each stayed broadly legible. He also revisits a 2019 circuit he helped find, in which a car feature seemed to hide inside unrelated dog neurons, and now reads it as ordinary generalization. As far as he knows, that circuit remains the only one with that shape in print.

For language models, the numbers are softer. In what he calls preliminary audits of Qwen3-32B, an open model with 32 billion parameters, he estimates that 2 to 4 percent of neuron poles are not yet understood, about 7 percent do two related jobs, and roughly 90 percent do mainly a single thing. He warns those figures shift with how thresholds are set. In one prediction test on 200 random neurons, an agent wrote new examples and near misses from each explanation. Strong firing followed 76 percent of the new examples and 11 percent of the near misses. He says similar results appear at sizes reaching into the trillions of parameters, Kimi K3 among them, but leaves the details to a companion paper that is not yet out.

That gap is the central question. The essay’s phrase “early model organisms” refers to the old small models behind the superposition case, not to a limit on its own work, yet its most direct evidence still sits in image models and one mid-sized language model. Cammarata writes that “My guess is that frontier networks are where those vision models were eight years ago”, and adds that they are readable but need a long list of small repairs. He labels the broader claim, that interpretability may be far more tractable than assumed, “highly speculative.”

Nobody has refereed this. It is a single-author essay on Goodfire’s own site, with no independent replication, and the checks were run by the author and colleagues using automated agents. Goodfire works in interpretability, so a more tractable field is a conclusion that favors its business; that is a reason to want outside groups to repeat the audits, not a finding against them. The essay also argues against a position that Cammarata himself held and that much of the field still does, and it does not claim to have refuted superposition everywhere. It is one side of a live disagreement.

The test arrives when the companion paper lands and other labs run the same neuron audits on models they can inspect fully. If the one-main-job share survives at frontier scale, the case for starting with the network itself gets much stronger.

Based on a Goodfire essay by Nick Cammarata, “Neural Networks Are Already Clean” (2026; the page carries no day or month)