Sean Goedecke, writing on his blog, says a new paper overstates what GPT-6 Astra can do with neural network weights, and that every result in it is within reach of a skilled human researcher.
The paper claims Astra can handle a small network’s weights directly, “well past what we’d expect from humans or conventional algorithms”. By Goedecke’s account, it reports that the model worked out which game a small network had been trained to play just from the weights, and could edit them too. Its headline result is that Astra improved a 0.8B Qwen language model’s mathematics by changing the weights natively. The excitement is easy to follow: a model that can reliably improve other models is the raw material of recursive self-improvement.
Goedecke is not buying it, and he starts with the game identification. The small network, about 17,000 weights, had labeled input channels, such as one for the agent and one for moving objects, and each channel read a cell of the game grid. Measuring how large each input’s 32 first-layer weights are, via the L2 norm, shows which cells the network attends to. He does it in three lines of Python and plots a heatmap per game. In his telling, a person shown those heatmaps could identify the first three games on sight: Breakout shows a horizontal line along the bottom, and Pac-Man shows the maze. Astra did the arithmetic without running code, which he calls neat, though he notes that this is superhuman mental arithmetic and not superhuman AI research.
The Qwen result is the harder case, and he treats it as such. A model that size is large, mathematics is a broader task than guessing a game, and he concedes that writing weights is arguably tougher than reading them. But Goedecke points out that Qwen was not weak at mathematics in general. Its problem was narrow: past 32,000 tokens it fell into loops, so any problem needing a longer chain of reasoning failed. Every problem under that length was already solved correctly by the original model.
His own fix would be steering. Write reasoning samples that loop and samples that do not, compare the model’s internal activations between the two groups, find which weights contribute most to the difference, and trim them. He says this is essentially the abliteration approach popularized by the Heretic project. According to his reading of the paper, that is close to what Astra did: it composed four pairs of texts, measured which weights fed the looping, cut them down in steps, tested each change, and landed on a patch that let the model keep working past the 32,000-token wall. Unlike with the game networks, Astra had access to Python for this task.
Goedecke’s conclusion is a comparison. A language model that can emit machine code or tweak a compiled binary is doing an impressive trick, yet the sensible way to change a program is still to edit the source and recompile. Editing weights directly is the binary-patching version of model improvement, and ordinary training remains the source code. If smart models speed up AI research, he expects it to come through better training, not through clever surgery on weights. He also sets aside one other example in the paper, building a network by hand from nothing, on the grounds that any task expressible as a simple program was already known to be expressible as a network.
The outside check here is thin: this is one blogger’s reading of one paper, and he does not claim to have rerun the experiments. Still, his argument gives builders a concrete test for the next “AI improves AI” claim. Ask whether a known human technique, run by a patient researcher with a Python shell, would have produced the same result, because if it would, the news is speed and convenience, not a new kind of capability.
Reported by Sean Goedecke on his blog, seangoedecke.com; the post carries no publication date.
