Basil Halperin, an economist, has taken the figures OpenAI published about its own AI research work and found that the pace of improvement hinges on how you fit a curve to them. In a thread on X, he shows the same chart producing a doubling time of either 585 days or 86 days for the tasks OpenAI’s models can finish reliably.
First, the measure. A time horizon is the length of task, judged by how long a person would need, that an AI completes successfully at a stated rate. A 50% horizon of six hours means the model gets half of the six-hour jobs right. The 80% horizon asks for success four times in five, which is closer to what a manager would rely on without checking the work.
Halperin tried three ways of reading the data. The first is to go to the 50% line on OpenAI’s chart and read across to where it meets the curves. That gives a 50% horizon of about 10 hours doubling every 68 days, and an 80% horizon of 15 minutes doubling only every 585 days. On this reading, the reliable end of the scale is short and barely moving.
The second method fits a logistic curve, the approach METR uses for its own time-horizon work. It gives about six hours at 50% with a 63-day doubling, and 15 minutes at 80% with an 86-day doubling. Same data, and the reliable end now looks like it is racing ahead. Halperin does not call either method clearly better, though he notes the logistic fit matches METR’s, which makes comparisons easier. A third fit, a Weibull distribution, tracks the points more closely still and also shows both horizons doubling quickly. He credits Gus Hamilton and Toby Ord for the earlier writing on that approach.
The 50% figure is fast under every method. METR’s horizon doubled every four to seven months until its measurement ended, while OpenAI’s research tasks double about every two months. An earlier analysis by Kwa found that other fields sit at different levels but improve at the same four-to-seven-month pace, so OpenAI’s number is an outlier. Halperin offers three explanations. OpenAI researchers may have sat on a large unused stock of capability, which he finds unconvincing. Progress may differ by domain, which would make the future harder to forecast. Or progress may have sped up everywhere, which he says fits METR’s latest report on acceleration at Anthropic.
The level is the other puzzle. As of July, OpenAI’s 80% horizon for internal research was only 15 to 20 minutes, and Google DeepMind’s is also around 15 minutes. One possibility is that AI research is simply hard. Another is that METR’s human baselines had little background on their tasks; when people with more context did the work, horizons came out roughly 10 times lower. Halperin adds a wrinkle. Kwa and colleagues found that with high-context humans, time horizon predicts AI capability less well. Yet for replacing real employees, the high-context number is the one with economic meaning.
The gap between the two reliability levels is also odd. In METR’s data the 50% horizon was about four times the 80% one. In OpenAI’s, six to ten hours sits against 15 minutes. Halperin is unsure why and suggests the human baselines or differences between fields.
His own summary is modest: OpenAI’s research horizon is shorter than METR’s headline figures, but it is rising quickly, with a doubling time that is sensitive to the specification.
The practical stakes show up in simple arithmetic on his numbers. Two more doublings would take OpenAI’s 80% horizon from 15 minutes to an hour in about six months on the 86-day fit, and in more than three years on the 585-day reading. Anyone forecasting when research agents can run unattended is choosing between those two clocks.
Basil Halperin’s thread on X, read through Thread Reader App; the capture carries no date.
