Small Reasoning Models Are the Quiet Revolution

The headline model releases are the ones with the biggest parameter counts and the highest benchmark scores. The releases that may matter more for how AI actually gets deployed are smaller: models a fraction of the size that reason through problems step by step and run on a single accelerator, a workstation or, increasingly, a phone.

How reasoning models work

A conventional language model produces its answer in one pass. A reasoning model is trained to generate intermediate steps first: to plan, check and correct before committing. This uses more compute at inference time, sometimes far more, but it dramatically improves performance on multi-step problems such as maths, code and planning. Researchers call the lever test-time compute: a second scaling axis alongside training.

Why small ones matter

Distillation trains a small model to reproduce the outputs, including the reasoning traces, of a large one. The result keeps much of the capability on a fraction of the hardware. For an enterprise that cannot send data to a third-party API, or a device that must work offline, this is the difference between a demo and a deployment.

The trade-offs

• Latency: reasoning takes longer than direct answering, which matters for interactive use.
• Cost per query rises with reasoning depth, even on small models.
• Capability ceiling: distilled models generalise less well than their teachers on novel problems.
• Evaluation: reasoning traces can look convincing while being wrong, so verification loops still matter.

Who benefits

Regulated industries, sovereign programmes, device makers and any organisation with data residency constraints. Also the vendors of the hardware that runs these models, which is a broader set than the accelerators used for frontier training.