Development · Term
Inference
Inference is the act of running a trained model to produce output — the step you pay for each time a request is made, as distinct from the one-off cost of training the model.
Inference is billed per token, separately for input and output, and output typically costs several times more than input. This is why the cheapest way to reduce an AI bill is usually to request tighter output rather than to change model.
Two further levers apply on most platforms: batch processing, which commonly halves the rate for work that need not be real time, and prompt caching, which bills the stable portion of a repeated prompt at a fraction of the standard input rate.
Related terms
