Reasoning tokens

Reasoning models, and ordinary models with a thinking mode switched on, produce an internal chain of intermediate steps first, which improves accuracy on maths, code and multi-step problems. Those steps can run to thousands of tokens for a short visible answer, so the true output cost of a request can be many times what the reply suggests.

Many APIs report them separately in the usage data and let you set an effort level per request, for example from off or low to high. Keep reasoning off for simple extraction or classification, raise it only where the answers measurably improve, and watch the reasoning count in your logs.

How it works

A reasoning model is trained to generate a working-out phase before its answer. The tokens in that phase are produced exactly like answer tokens, one at a time, and are counted and billed as output. Some APIs return them, some summarise them, and some hide them, but the usage data normally reports how many there were.

Effort settings, which providers name differently but which usually run from off or low to high, set how much working-out the model is allowed. Some models reason on every request regardless.

Example

A request that returns a 200-token answer from a reasoning model might report 3,000 reasoning tokens in its usage data. Both are billed at the output rate, so the request costs about sixteen times what the reply alone would suggest.

In practice

Ask whoever runs your AI features to show the usage breakdown for a typical day: input tokens, visible output and reasoning tokens. If reasoning is a large share, check which tasks generate it and whether they need it. A simple experiment settles most cases: run the same twenty real requests with reasoning off or low and again with it higher, and compare accuracy, response time and cost. Keep the higher setting only where answers clearly improve.

Some edge cases catch teams out. If the maximum output length is set too low, a reasoning model can spend the whole allowance thinking and return an empty or cut-off answer, which looks like a bug. Response times vary more with reasoning, so a customer-facing chat may need a visible progress indicator. Providers also describe effort levels differently, and the same setting name can mean different amounts of reasoning on different models, so retest after switching. Finally, when comparing prices across models, compare the cost of completed tasks, not the per-token rate, because a model that reasons more can cost more overall at a lower rate.

Why it matters for your business

Reasoning tokens explain why an AI feature's bill can be far larger than the short answers it shows, and why some responses take noticeably longer to arrive. Both matter if customers are waiting in a chat window. For an owner, the practical step is asking whoever builds the feature to report reasoning tokens separately in usage reports, and to switch reasoning on only for the jobs where it measurably improves answers, such as multi-step calculations, planning or tricky code. For classification, extraction and short replies it usually adds cost and delay for no gain. Reasoning is the model doing its own chain of thought, and like any extra work it should be budgeted.

Common mistakes

• Leaving reasoning on for simple classification or extraction, where it adds cost and latency without improving answers.
• Estimating cost from the visible reply length rather than the reported usage.
• Setting a low output limit that the reasoning phase uses up before the answer is written.

Questions people ask

Are reasoning tokens the same as chain-of-thought prompting?

They are related. Chain-of-thought prompting is a technique where you ask a model to work through a problem step by step in its answer. Reasoning models are trained to do that working-out themselves before answering, and the tokens they spend on it are the reasoning tokens, usually kept separate from the final reply.

Do reasoning models always give better answers?

No. They tend to help on problems with several steps, such as maths, logic, planning and debugging code. On simple tasks, like sorting an email into a category or pulling a date from a sentence, a standard model is usually just as accurate, faster and cheaper. Test both on your own examples.

Do reasoning tokens count against the context window?

Yes, while the model is working. Reasoning tokens occupy space in the context window and in the output limit alongside the final answer, so a long working-out phase can leave less room for the reply. Several providers drop earlier turns' reasoning from later requests, which keeps long conversations from filling up as quickly.