In short
AI API costs are driven by four things you control: which model tier you use, how many output tokens you request, whether repeated prompt content is cached, and whether work runs in real time. Output typically costs several times input, and batch processing commonly halves both. Most overspending comes from defaulting to a frontier model and from agent loops that resend their accumulated history on every step.
Why the bill grows faster than usage
Two mechanics explain most surprise invoices.
Output costs several times input. Across the major providers, output tokens bill at roughly five to six times the input rate. A verbose response format is therefore far more expensive than a long prompt — and it is the thing teams tune last.
Agent runs resend their history. Every step in an agentic loop carries the accumulated record of previous steps back into the request. A ten-step run does not cost ten times a single call; it costs closer to the sum of a growing series. This is why agent costs surprise people who reasoned from single-call pricing.
Add a third, quieter one: reasoning models bill their internal thinking as output. A short visible answer can carry a long, billable reasoning trace behind it, which is why a response that looks cheap sometimes is not.
Lever one: use the right tier
This is the largest single lever and the most commonly ignored.
Providers publish tiers that differ by an order of magnitude. As of September 2026, OpenAI's GPT-5.6 runs $5/$30 per million tokens on Sol, $2/$12 on Terra, and $0.20/$1.20 on Luna. Anthropic runs $10/$50 on Fable 5.1 down to $1/$5 on Haiku 4.5. Google's Gemini 3.1 Pro is $2/$12, with Flash-Lite tiers well below that.
That is a twenty-five-fold spread between the cheapest and most expensive options at the same provider.
Most teams default to the flagship because it is the safest choice at prototype stage, then never revisit it. The discipline is simple: start on the balanced tier, and move up only when you can demonstrate a specific failure you can measure. "It feels better" is not a measurement — which is why a fixed evaluation set is a cost tool as much as a quality tool.
Many workloads — classification, extraction, routing, tagging, summarising short text — run perfectly well on the cheapest tier. Those are often your highest-volume calls.
Lever two: cut output tokens
Because output costs roughly six times input, this has outsized effect.
Ask for structure, not prose. A JSON object with four fields costs a fraction of a paragraph explaining the same four things. If code consumes the output, it never needed the prose.
Cap the length explicitly. Set a maximum output limit as a safety net so a degenerate response cannot run away.
Stop asking for restated context. Models habitually open by summarising the question. Instruct them not to. Over a million calls that preamble is a real line item.
Do not request reasoning you will not read. Chain-of-thought output is billed. Where you need it for debugging, log it; where you do not, suppress it.
Lever three: cache the stable part
Most production prompts are a large fixed portion plus a small variable one — a system instruction, a schema, examples, then the actual input.
Providers bill cached input at a fraction of the standard rate: OpenAI at 10% of standard input, Anthropic reducing cached input by 90%. To benefit, keep the stable content in a consistent position at the start of the prompt so it can be recognised across calls. Rebuilding the prompt in a different order each time defeats it.
This is free money for anything with a substantial system prompt, and it costs one afternoon of restructuring.
Lever four: batch what is not urgent
Batch APIs commonly halve both input and output rates in exchange for asynchronous delivery.
A surprising share of production AI work does not need to be real time: overnight enrichment, classification of yesterday's records, bulk summarisation, backfills, evaluation runs. If nobody is watching a spinner, it can be batched.
The rule of thumb: if the result is consumed by a system rather than a person, ask whether it needs to be synchronous. Usually it does not.
Bounding the failure cases
Efficiency reduces the bill. Limits stop the catastrophe.
Set hard spend caps at the provider, not only alerts. An alert tells you afterwards.
Cap agent iterations and runtime. An agent without a termination budget will keep trying, and a confused agent is a spending loop.
Rate limit per user and per key. One misbehaving integration should not be able to consume the month.
Separate keys by environment. A development loop should never be able to spend production budget, and separate keys make attribution possible.
Log token counts per feature. You cannot manage what you cannot attribute. Cost per feature, per customer, or per request type turns a vague bill into a list of decisions.
What to do first
In order of return on effort: pin down what you actually spend per feature; move high-volume, low-judgement calls to the cheapest tier that passes your evaluation; cut output verbosity; restructure prompts so the stable portion caches; batch anything asynchronous; then set hard caps.
Most teams find the first three alone remove a majority of the bill — and none of them require changing what the product does.
One caution: pricing moves monthly, and providers update models underneath you. Record the model version, parameters and date with your results, and re-check rates against the vendor's own pricing page before you build a budget on them.
If you want a second read on where an AI bill is actually going, book a call.
Common questions
Why is my AI API bill so high?
Usually two causes. Output tokens bill at roughly five to six times the input rate, so verbose response formats cost far more than long prompts. And agent runs resend their accumulated history on every step, so a ten-step run costs much more than ten single calls. Reasoning models also bill their internal thinking as output, which is invisible in the response.
How much can you save by switching model tier?
Potentially an order of magnitude. As of September 2026 OpenAI's GPT-5.6 spans $5/$30 per million tokens on Sol down to $0.20/$1.20 on Luna, and Anthropic spans $10/$50 on Fable 5.1 down to $1/$5 on Haiku 4.5. High-volume, low-judgement work — classification, extraction, routing, tagging — usually runs fine on the cheapest tier.
What is prompt caching and how much does it save?
Providers bill repeated, stable prompt content at a reduced rate — OpenAI at 10% of standard input, Anthropic reducing cached input by 90%. To benefit, keep the fixed portion of the prompt in a consistent position at the start, since rebuilding the prompt in a different order each call defeats the cache.
When should I use a batch API?
Whenever the result is consumed by a system rather than watched by a person. Batch processing commonly halves both input and output rates in exchange for asynchronous delivery, which suits overnight enrichment, bulk classification, backfills, and evaluation runs.
How do I stop an AI feature from running up an unexpected bill?
Set hard spend caps at the provider rather than relying on alerts, cap agent iterations and runtime so a loop terminates, rate limit per user and per key, and use separate keys per environment so development cannot spend production budget. Then log token counts per feature so you can attribute cost to decisions.
