Tomasz Tunguz described the comparison on his blog, tomtunguz.com. He ran 25 of his own work tasks, research on startups, article summaries, podcast transcripts, through three language models: a cloud model called deepseek-v4-flash and two open models running locally on his laptop. A blind judge model scored every answer on completeness, accuracy and conciseness.
Qwen3.8-27B, a 27-billion-parameter local model, matched deepseek-v4-flash’s score exactly, 8.0 out of 9. It took nearly seven times longer to answer: 7.2 seconds against 1.1 seconds. Quality held steady across the two models; latency did not.
A second local model, Qwen3.6-35B-A3B, a 35-billion-parameter mixture-of-experts model with 3 billion parameters active per token, scored close behind at 7.9. It also generated far more tokens than either rival, averaging 1,143 per answer against 369 for the smaller Qwen model and 159 for the cloud model. On one triage task, it spent 993 tokens producing a six-word classification. More deliberation did not translate into a better score.
Separately, Tunguz cites Artificial Analysis, a third-party site that benchmarks and ranks language models, where Qwen3.8-27B sits #1 of 135 models on its Intelligence Index, scoring 52, one point above GLM-5.2, a 753-billion-parameter open model from Z.ai that shipped two months earlier. That ranking is not Tunguz’s own test. It is a citation to someone else’s leaderboard, and a one-point gap across 135 ranked models is a thin margin to build an argument on.
Tunguz’s explanation is that parameter count functions as memorized capacity. Large models can jump straight to an answer they have effectively stored; smaller models, holding less, have to reason through more of the problem step by step to close the gap. He points to existing research on that tradeoff, the idea that smaller models drift further off course once forced away from a direct, memorized path to the answer. That is an argument about why the numbers came out this way. It is not a second measurement.
The post skips a caveat worth stating directly. This is one person’s evaluation, run on 25 tasks he selected from his own workflow and graded by a separate language model rather than independently replicated. Parameter count was never a reliable stand in for capability before this comparison, so a 27B model edging a 753B model on one index does not overturn that history; it adds one more data point to it. The judge model carries its own noise, too. Tunguz’s own check, rescoring identical outputs, found a 0.16-point spread on a 9-point scale, which is not large but is not zero either.
For teams weighing local inference against API calls, the tradeoff here is narrower than “smaller models are catching up.” A 27B model matched a frontier-class cloud model’s quality on bounded, well-defined tasks, research summaries, transcription, structured classification, at the cost of single-digit-second latency per answer. That math favors local models for batch or asynchronous work, where nothing is waiting on the response, and still favors cloud models for anything interactive, where seven seconds per turn is the gap between a usable assistant and an abandoned one. It also assumes hardware capable of running a 27B model at reasonable speed, a real capital cost that a hosted API call does not carry.
Teams running narrow, repeatable agent tasks should benchmark a local 27B to 35B model against their current API spend before renewing that contract. Teams building anything a user watches in real time should not expect this result to hold, and should treat the latency numbers here as the reason why.
Adapted from Tomasz Tunguz’s post “Birds Don’t Fly Like Planes. Neither Does AI.” on tomtunguz.com.
