Yingru Li, in a post dated 15 October 2026 on his site richardli.xyz, argues that the methods used to fine-tune language models change only a small fraction of the model’s parameters, and that “small” and “sparse” are two separate claims. Small means little changes in total. Sparse means the changes land on the same few parameters.
The distinction matters because the two imply different things about how a fine-tune behaves. If a modest, predictable set of parameters carries every update, a fine-tune could in principle be treated as a small patch. If the change is spread thinly across a huge number of parameters, it could not. Li’s post does not make the patch argument. That inference is ours, and the post stays on the mathematics.
The starting point is research Li cites rather than runs. A 2025 NeurIPS paper by Mukherjee and colleagues found that reinforcement learning (training a model on a reward signal) changes 5 to 30 percent of parameters. The finding covers ten models and seven different training algorithms. A 2026 paper by Zhang, Mukherjee and Peng reports the same pattern for distillation from a model’s own outputs, and for ordinary supervised fine-tuning on data close to what the model would write itself.
Li then dismisses two common explanations as answering the wrong question. One says reinforcement learning only reshuffles answers the model can already give. The other says a single reward number carries very little information per attempt. Both, he writes, explain why the update is small or thin. Neither explains why it would be sparse. Spread over a hundred million parameters that each shift a little, an update is still small, yet dense, and the same handful of bits fits easily.
His own argument is that each parameter has a ceiling on how much it can be pushed, set by the model before any reward is seen. Roughly, a parameter has room to change when it treats the options the model considers likely differently from one another. A push that lifts every likely option equally has no effect on what the model learns. After “She drank a cup of”, parameters that separate tea from coffee have plenty of room. They still stay put when the reward is indifferent between the two drinks. Li puts it this way: the ceiling marks where learning can happen, while the reward settles whether it happens.
The only measurement in the piece belongs to the Zhang paper. Before reinforcement learning begins, the top 1 percent of parameters by this ceiling account for 47.40 percent of the top 1 percent by actual update size, against 1 percent expected by chance. The post itself reports no experiment of its own. It is a derivation with a proof section. A bound that holds in principle can still be loose in practice, and the post does not test it on any real model.
The second claim in the post is that none of this requires the model to learn from its own fresh samples, the setup practitioners call on-policy. Li says it requires only that the learning signal be adjusted by the average under whatever process produced the samples. The condition is demanding: it needs that process’s full probability for every possible next word at every step. He notes that inference-engine log-probabilities and older versions of the model provide this, and human-written demonstrations do not. Off the model’s own samples, he adds, the quantity being followed is the gradient of a different loss, which need not be the gradient of the expected reward. The post does not say this makes training cheaper.
Two further limits are stated in the post. The proof assumes a finite set of possible responses, sampling at temperature 1 with no top-k or top-p truncation, and a reward that does not depend on the parameters. The post does not claim to overturn anything.
The “where” half of the story is deferred to Part 2, and Part 3 asks what this kind of training can reach at all. Until Part 2 arrives, the sparse half of the argument rests on one measurement from another paper, which is thin ground for anyone planning storage or merging of fine-tunes around it.
Based on Yingru Li, “The Gradient Is a Covariance” (Part 1 of 3), richardli.xyz, 15 October 2026.
