Every frontier language model trains on a scrape of the internet so vast that no lab can fully describe what is inside it. That makes a basic question almost impossible to answer: when a model shows a new skill, did it learn something, or did scale and prompting simply surface something already latent in an uncharacterizable pile of data? A research project called LittleLearner answers that by removing the ambiguity at the source, building models whose training data is fully specified in advance.
The project trains language models from scratch on LittleCurriculum, an 88 billion token corpus built by narrowing FineWeb-Edu, a large web-text dataset, through five filtering stages tied to the U.S. Common Core curriculum for kindergarten through fifth grade. Grade-six material and beyond, including its concepts, facts and vocabulary, is deliberately left out. The result is a model whose knowledge boundary is defined before training starts, not estimated after release.
LittleLearner ships at three scales, 0.6 billion, 1.3 billion and 5 billion parameters, and each has a matched “Unfiltered” control model on the same architecture, token count and training recipe, minus the curriculum restriction. A hosted 5 billion parameter version runs live in the browser as a chat demo on the project’s site.
The paper’s central finding is that scale and post-training amplify what a model already knows rather than teach it anything new. According to the project page, performance improves as scale increases within the curriculum’s boundaries, and those gains extend a bit to problems along a similar learning path, but larger models show little improvement beyond grade five. Post-training with GRPO on a math benchmark called MathCAMPS produced the same pattern: strong gains inside the K to 5 boundary, no recovery of beyond-grade-five capability, even when the reinforcement training used out-of-scope data. In-context learning, tested with the researchers’ own prompts, did not unlock new reasoning either. The team calls the pattern elicitation rather than acquisition: the pretraining filter sets the capability ceiling, and nothing applied afterward moves it.
That is the useful part for anyone building or evaluating models. It is a controlled trial for a question the field usually cannot run, because frontier training sets are too large and too undocumented to isolate what a given technique actually added. The tradeoff is definitional, not technical: “what a fifth grader knows” is a curatorial choice baked into a filtering pipeline, not a fixed fact, and a different team building the same corpus from a different curriculum would draw the boundary elsewhere.
For teams checking whether a new fine-tuning method or prompting trick adds real capability rather than just eliciting it, LittleLearner’s matched-control setup is a template worth borrowing before trusting a benchmark jump as evidence of new learning.
LittleLearner project page and accompanying arXiv paper, “Language Models Under Pedagogically-Controlled Knowledge Exposure,” published in August 2026.
