The Inference Economy

The first phase of the AI build-out was about training: enormous clusters, months-long runs, and a small number of labs competing to build the largest models. The second phase is about what happens after the model exists. Every query, every agent step, every voice interaction is inference, and inference runs all day, every day, for every user.

Why now

• Agents multiply the number of model calls per task by an order of magnitude or more.
• Reasoning models spend more compute per answer by design.
• Consumer scale means billions of daily interactions.
• Cost per token has fallen sharply, which increases usage rather than reducing spend.

What changes

Hardware: inference favours chips optimised for memory bandwidth and efficiency, opening room for custom silicon and alternative architectures alongside the accelerators that dominate training. Location: latency-sensitive workloads pull capacity toward users, while batch workloads can follow cheap power. Pricing: every layer, from chips to cloud to applications, is moving toward usage-based models.

Who benefits, who is at risk

Beneficiaries: efficient inference hardware makers, serving infrastructure providers, regions with cheap power, and application companies whose costs fall as prices drop. At risk: companies whose margins depend on inference costs staying high, and buyers who provisioned training-class capacity for inference-class workloads.