All Models Are Wrong, But Some Won't Tell You When

In the 1970s the statistician George Box gave us one of the most quoted lines in science: all models are wrong, but some are useful. He gave it to us in pieces. The wrongness came first, in a 1976 paper. The usefulness arrived later, before the decade was out. Keep that in mind. The second clause was always the negotiable one.

It has since been reduced to a shrug, a polite way of excusing whatever just broke. Your forecast missed? All models are wrong. The LLM hallucinated a court case? Well, all models are wrong.

That reading would have annoyed Box.

The aphorism was never an apology for error. It was an instruction about how to live with it. And the way we now apply it to LLMs gets that instruction almost exactly backwards.

Box was talking about statistical models: a linear regression, a response surface, a differential equation describing a chemical reactor. He called them "wrong" in a specific sense. A model is a deliberate simplification of reality. It throws away detail on purpose so that the part you care about becomes tractable. The map is wrong because it is not the territory. That is the whole point of a map.

But here is the part people forget. When Box said a model was wrong, he was talking about wrongness that could be studied. A regression gives you residuals. It gives you confidence intervals, an R², assumptions, diagnostics. You can often tell where the model is interpolating and where it is extrapolating off a cliff.

The wrongness is not eliminated, but it is made legible.

You can do the thing Box demanded in that same 1976 paper:

That is the lesson. Not "models err, so relax." It is: models err, so find out which errors are mice and which are tigers.

The usefulness in "some are useful" was always conditional on your ability to map the error.

Now apply the line to an LLM, and the comfort starts to evaporate.

A language model is also wrong, but its wrongness is far harder to read. It can hand you its own confidence score. That self-reported number is the one you cannot lean on, least of all on the novel, high-pressure inputs where you would need it most. The model does not reliably tell you whether it is drawing from dense evidence, extrapolating across a gap, or confabulating under pressure. And it cannot see half the problem at all: the model can be fully confident, and correct to be, while reasoning over inputs that were wrong from the start.

The output is fluent and self-assured whether it is correct, subtly misleading, or catastrophically wrong. A regression about to fail looks shaky in its diagnostics; an LLM about to fail looks exactly like one about to be right.

So the aphorism survives, all models are wrong, but the second clause loses its footing. Box's models earned "some are useful" because their errors could be inspected, bounded, and monitored against assumptions. You could validate the model, understand much of its failure surface, and decide whether its wrongness was acceptable for the job.

An LLM gives you the wrongness without reliably giving you the map.

It gets more interesting when you notice that Box's aphorism mostly spoke to one kind of wrong.

The two failures are invisible to each other. A model-wrong failure passes every permission check; a data-wrong failure passes every hallucination eval. You can benchmark a model into the ground and still ship a system that leaks, because the context broke in production, in a configuration no offline eval ever saw.

Box's aphorism, and the evaluation mindset built around it, mostly addressed the first column. Agentic systems force us to confront both.

Here is the structural break with Box's world.

In many classical modeling settings, the goal is to make the error surface stable enough to study. The reactor behaves enough like the reactor. The process being modeled is constrained enough that diagnostics can tell you something durable. Validation is not perfect, but it can often be a thing you substantially complete before deployment.

An LLM agent's error surface is different.

It depends on the prompt, the retrieved context, the tools connected that day, the live data it touches, the permissions it inherits, the policies in force, and the specific user on the other end. Change any of those and you may have changed where the system is wrong.

This is why offline reassurance fails. You cannot fully bound an LLM agent's wrongness at design time, because many of the conditions that determine the wrongness do not exist until runtime.

The "some are useful" certification Box could grant in the lab cannot simply be issued in advance for an agent that acts on live systems.

Notice what that implies.

Box never told us to ignore error. He told us to characterize it and stay alert to the tigers. We have always done that work. It just used to live mostly at design time, in diagnostics, assumptions, validation sets, and post-deployment monitoring.

For LLM agents, that work has not disappeared. It has moved.

The only place you can fully characterize the wrongness of a non-stationary system is the only place the wrongness fully exists: at runtime, per action, in production.

This is the premise behind Classie OBS-SEC: the wrongness cannot be certified away up front, so the system has to remain observable and governable at runtime.

That is Box's instruction, updated for agents: do not merely accept that models are wrong. Find the tigers before they move.

Box's aphorism is still true. It may be truer than ever.

All models are wrong.

What changed is the second clause. The old models earned "some are useful" by showing their work, by giving you diagnostics alongside the estimate, so you could decide whether this particular wrongness was a mouse or a tiger.

LLMs often hand you the estimate and keep the error bars to themselves.

So the honest 2026 version of the line is longer, and less comforting:

All models are wrong. Few advertise it. And agents will not reliably tell you when their wrongness has become dangerous.

Which leaves exactly one option for the part that matters.