Teams that ask an LLM to sort text into categories, spam or not spam, ironic or sincere, often get a model that performs well on average but fails the basic tests any production classifier needs to pass. That is the argument Taylor Pospisil makes in a technical post on the blog Minimally Sufficient: LLMs make poor classifiers not because they lack signal, but because they were never built to output calibrated probabilities, absorb structured data alongside prose, or explain which part of a prompt actually drove a verdict.

Pospisil’s fix does not touch the model. It wraps the LLM’s raw verdict in a simple logistic regression, treating that verdict as a single input feature rather than a finished decision. Fit on a modest labeled training set, the regression converts the LLM’s hard “yes” or “no” into an actual probability, one that can be tuned to trade off false positives against false negatives, something a bare LLM output has no principled way to do.

The post tests this against SemEval 2018’s irony detection dataset, 4,618 tweets labeled by expert annotators. Using a single zero-shot prompt sent to Gemini 3.1 Flash Lite, the raw LLM verdict alone scored an F1 of 0.747, already ahead of THU_NGN, the 2018 competition’s winning system, which scored 0.705. Wrapping that verdict in the logistic regression left the F1 score unchanged (probability calibration does not change rank order) but cut the Brier score, a measure of how well the predicted probabilities matched reality, from 0.259 to 0.175.

Pospisil then pushed further, asking the LLM to score each tweet on nineteen additional dimensions (whether it targets a named person, whether it contains mock enthusiasm, whether the underlying situation is objectively negative) and folding in cheap rule-based signals like hashtag count and exclamation marks. Stacking all of that into the same regression pushed F1 to 0.779. Its reported confidence interval, 0.746 to 0.81, overlaps the 0.786 that NTUA-SLP, an LSTM-with-attention system, scored after the original SemEval contest had already closed.

The framing that follows is the post’s real claim: an LLM prompt is not a classifier at all, it is a feature extractor, and the discipline that makes it useful is the one machine learning practitioners have run for a decade, collecting more labeled data, screening which features actually predict the outcome, and letting a conventional model (logistic regression, gradient boosting, whatever fits) make the final call. Pospisil says the approach is meant to recover the LLM’s real advantage, prior world knowledge embedded in the weights, without inheriting its weaknesses as a decision-making layer.

For teams shipping an LLM-as-classifier into production, the practical shift is small but consequential: stop tuning the prompt when accuracy plateaus, and start routing the model’s raw verdict, plus a handful of cheap auxiliary features, into a lightweight model you can actually calibrate, audit, and threshold.

Adapted from Taylor Pospisil’s post “LLM Classification Is Feature Engineering,” published on the blog Minimally Sufficient on September 13, 2026.