Chain of thought
Chain of thought is the practice of having an AI model work through a problem in written intermediate steps before giving its final answer, which tends to improve accuracy on maths, logic and multi-step questions.
The idea started as a prompting trick: adding an instruction such as "think step by step", or showing an example that reasons through a problem, made models noticeably better at tasks needing several steps. Reasoning models are trained to do this internally, producing hidden reasoning tokens before they answer.
A written chain of thought is not proof of how the model reached its answer. It is text the model generated, and research has shown it does not always reflect the process that produced the conclusion. Treat it as a useful working, not as a guarantee.
How it works
A large language model produces one token at a time, and each token is based on everything written before it. If it jumps straight to an answer, it has to get there in one step. If it first writes out the intermediate facts, such as which numbers apply and what each step gives, those steps become part of the input for the final answer, so the answer is built on more context.
There are three common forms. You can ask for step-by-step working in the prompt. You can include examples that show the reasoning, combining chain of thought with few-shot prompting. Or you can use a reasoning model that thinks before replying on its own, sometimes with a setting to control how much it thinks.
The trade-off is time and cost. The extra steps are tokens that take time to generate and are usually billed. For simple lookups or short rewrites, step-by-step reasoning adds little and slows the answer down. Reasoning models usually keep their full working hidden and show only a summary, partly because the raw steps are long and messy. You pay for the hidden tokens even though you do not see them, so check the provider's pricing when a task suddenly costs more than expected.
Example
A printing shop uses an AI model to check quotes. A customer wants 500 flyers, with a discount above 250 copies and a surcharge for next-day delivery. Asked for the total directly, the model sometimes applies the discount after the surcharge instead of before, producing the wrong price.
The developer changes the prompt to ask the model to list each rule, apply them in the order written in the price sheet, show the subtotal after each step and only then state the total. Errors drop, and when one does occur, the staff member reviewing the quote can see exactly which step went wrong. The final calculation is still checked by the shop's own pricing code before a quote is sent. The shop also logs which rule caused each error, which shows the price sheet itself was ambiguous on one point.
Why it matters for your business
For tasks with several steps, such as checking rules, comparing options or working out figures, asking a model to reason first usually gives better answers and makes mistakes easier to spot. It costs more time and tokens, so it is not worth switching on for every request. For anything with money or legal consequences, the visible steps help a person review the result, but they do not replace a proper calculation or a professional's check.
Common mistakes
- Trusting a confident chain of reasoning without checking the facts and figures it starts from.
- Asking for step-by-step reasoning on simple tasks where it only adds delay and cost.
- Showing the model's raw reasoning to customers, when it may contain half-formed or incorrect steps.
- Relying on a model's arithmetic for prices or invoices instead of calculating them in software.
Questions people ask
Does telling AI to think step by step really help?
Often, for tasks that need several steps, such as maths, logic puzzles or applying rules in order. It helps less on simple factual questions. Reasoning models do this internally anyway, so the instruction matters most with models that do not, or when you want the working shown.
What is the difference between chain of thought and reasoning models?
Chain of thought is the technique of reasoning in written steps before answering. Reasoning models are trained to do this automatically, often in hidden tokens, before writing their reply. So a reasoning model applies chain of thought by default rather than waiting for you to ask.
Is a model's chain of thought always accurate?
No. The steps can contain errors, and the written reasoning may not faithfully describe how the model reached its answer. It is useful for spotting mistakes, but anything important should be checked against a reliable source or calculation.
