The real cost of running AI agents in production

Every AI agent pitch includes a slide with per-token pricing that makes the whole thing look nearly free. Multiply a few cents by the number of requests you expect and the projected monthly cost looks like a rounding error next to a headcount. Then the system goes live, and the actual bill is five to twenty times that estimate. The gap isn't pricing dishonesty. It's that nobody accounted for what an agent actually does between the request and the answer, and the estimate was built on a mental model closer to a single API call than an actual reasoning process.

Where the multiplier comes from

A single user question rarely costs one model call. An agent that reasons through a task might call the model once to plan, once or twice to use tools, once to check its own output, and once more to format a response. That's five calls for one interaction, and if any step fails and retries, it's more. Long conversation history gets resent on every turn unless you're actively managing context, so a ten-message conversation can cost as much as re-reading the entire thread each time a user replies.

It's worth sitting with that last point, because it's the one most commonly missed. If every turn resends the full history, cost doesn't grow linearly with conversation length, it grows roughly quadratically. A conversation that felt cheap at message three can be costing multiples of that by message twenty, and nobody notices because the per-message cost still looks small in isolation. It only shows up when you look at cost per conversation, not cost per call.

• Multi-step reasoning multiplies calls per request, not per conversation.
• Full conversation history resent every turn scales cost quadratically with conversation length unless it's trimmed or summarized.
• Retry logic on failures silently doubles or triples cost during a bad day for the provider.
• Polling and background agents that check state on a timer burn tokens even when nothing changed.

The quiet costs nobody puts on the slide

Beyond the per-request multiplier, there's a category of spend that's structural rather than incidental. Background agents that check a queue, a mailbox, or a data source on a fixed schedule burn tokens on every check regardless of whether there's anything to act on. A five-minute polling loop that mostly finds nothing still costs money every five minutes, all day, every day. Tool calls that fetch large payloads, an entire customer record when only one field was needed, inflate the context window and the bill along with it. None of these show up in a per-token pricing slide, because they're a consequence of how the system is architected, not the price list itself.

Budget it like infrastructure, not like a SaaS seat

The teams that don't get surprised treat model spend the way they treat cloud compute: with usage caps, alerts, and a dashboard somebody actually looks at. Set a per-user or per-workflow cost ceiling before launch, not after the first invoice. Cache aggressively where the same question gets asked repeatedly. Trim context to what the task actually needs instead of forwarding the whole history by default. And pick the smallest model that clears your accuracy bar for each step, a cheap model for extraction and a stronger one only for the step that genuinely needs judgment.

Model routing, sending easy steps to a cheap, fast model and only escalating to an expensive one when the task actually needs it, is one of the highest-leverage architectural decisions in an agent system, and it's routinely skipped because it's easier to just call the best available model for everything. That laziness is fine for a demo and expensive at scale. A well-routed pipeline can cut cost by more than half without any noticeable drop in quality, because most steps in most workflows don't need the most capable model available, they need the cheapest one that gets the step right.

The cost of an agent isn't the model call you can see. It's the four you didn't budget for.

Ask this before you sign off on a build

How many model calls does one user interaction actually trigger, end to end, including retries and verification steps? Most teams can't answer that question about their own system until they've been billed for it once. Ask upfront, and ask your build partner to show you the call graph, not just the sticker price per token. It's the only number that predicts what you'll actually pay, and it's the number that separates a project that stays within budget from one that gets quietly throttled or shut down three months in because finance noticed the trend line.

It's also worth revisiting that call graph periodically after launch, not just before it. Usage patterns shift, conversations get longer as users trust the system more, and a cost model that was accurate at launch can drift meaningfully within a couple of quarters. The teams that stay ahead of this treat the cost dashboard as a living thing to check monthly, not a box ticked once during the build.

Fixed pricing versus usage pricing changes the incentive, not just the invoice

Clients sometimes ask for a flat monthly fee instead of passing through usage cost, on the theory that it's simpler to budget. It is simpler, but it also quietly shifts who's incentivized to control cost. Under usage pricing, both sides want the system efficient, the client because they're paying for every call, the builder because an inefficient system is harder to defend when the bill is itemized. Under a flat fee, that shared incentive disappears, and cost discipline depends entirely on the builder's own standards rather than a natural check from the client watching the meter. Neither model is wrong, but going in with eyes open about which incentive structure you're choosing avoids a quiet drift toward bloated, unmonitored usage a year in.

A middle path that works well in practice is usage-based pricing with a cap and an alert well before the cap, so the client gets budget predictability without losing the natural incentive for the system to stay efficient. It also means nobody discovers a cost problem for the first time in an invoice, they see it coming in the alert with enough runway to actually do something about it.