What Is the Cheapest Way to Run an LLM?

For most projects the cheapest option is a low-cost hosted API, by a wider margin than people expect. The maths only flips at serious volume or under strict privacy needs.

What Is the Cheapest Way to Run an LLM? — Troiana insight cover

In short

For low to moderate volume, a budget hosted API is usually cheapest: at current DeepSeek off-peak rates, 10 million input and 2 million output tokens cost about $2.70. A rented RTX 4090 running all month costs around $248 before you serve a single request. Local hardware makes sense if you already own it or need privacy. Free tiers suit testing, not production.

The four options

Every way of running a language model is a version of one trade-off: pay per token, or pay for the machine whether it is busy or not.

OptionYou pay forBest when
Free tierNothing, within limitsLearning and prototypes
Hosted APITokens usedAlmost everything else
Rented GPUHours, used or notHigh steady volume, custom models
Your own hardwarePurchase, power, your timePrivacy, offline use, hardware you already own

Current reference prices

Checked on 4 November 2026. These change often, so plug in the latest numbers.

  • DeepSeek API: its Flash model is $0.15 per million input tokens and $0.60 per million output off-peak, double that at peak, with cached input far cheaper.
  • OpenRouter free models: 20 requests a minute. You get 50 a day, or 1,000 a day once you have bought at least 10 credits.
  • RunPod: an RTX 4090 with 24 GB costs $0.34 an hour on Community Cloud or $0.89 on Secure Cloud. An A100 with 80 GB runs from about $1.19 to $1.79 an hour.

A worked example

Say your app uses 10 million input tokens and 2 million output tokens a month. That could be a modest internal tool, or a small product with a few hundred active users.

On the budget API:

  • Off-peak: (10 × $0.15) + (2 × $0.60) = $2.70
  • Peak: (10 × $0.30) + (2 × $1.20) = $5.40

On a rented RTX 4090, running continuously (about 730 hours):

  • Community Cloud: 730 × $0.34 = about $248
  • Secure Cloud: 730 × $0.89 = about $650

The GPU costs roughly 90 times more for this workload, and it is sitting idle most of the time. It is also not a like-for-like comparison, because the model you can fit on 24 GB is generally smaller than the one behind the API.

10M input and 2M output tokens a month, log scale
Budget API, off-peak$2.70
Budget API, peak$5.40
Rented RTX 4090, Community CloudAbout $248
Rented RTX 4090, Secure CloudAbout $650
$1$10$100$1,000
The article's sums at prices checked on 4 November 2026. The GPU charges by the hour, used or not.

Where renting starts to win

Here is a rough break-even using those figures. With the same 5:1 input-to-output mix, the API works out at about $0.225 per million tokens off-peak and $0.45 at peak. Therefore:

  • $248 ÷ $0.225 ≈ 1.1 billion tokens a month off-peak
  • $248 ÷ $0.45 ≈ 550 million tokens a month at peak

And that assumes one card can actually serve that volume at the speed you need, on a model of acceptable quality. Plus you now own uptime, updates, scaling and monitoring, so add the cost of the person doing that. Caching on the API side moves the break-even further away. You can reuse the formula: monthly machine cost divided by your blended price per million tokens.

Renting can be excellent in another way: batch work. If you need to process a large dataset once, renting for 20 hours costs about $6.80 at the Community rate, and you stop paying when it finishes. The same logic applies to testing a model before you commit.

When your own hardware makes sense

Buying a machine purely to save on inference rarely pays off at small scale, because budget APIs are so inexpensive. There are still good reasons to run locally:

  • You already own it. A decent GPU or Apple Silicon Mac makes the marginal cost mostly electricity. As a rough example, a 400 W system running 4 hours a day uses 48 kWh a month, or about $9.60 at $0.20 per kWh. Check your own tariff and real power draw.
  • Privacy or compliance. If data must stay on your premises, price stops being the deciding factor. See whether it is safe to send customer data to an LLM.
  • Offline use and experimenting. No per-token meter means you can iterate freely.

Our comparison of Ollama and LM Studio covers the software side.

Cost is not only price per token

The cheapest model that fails the task is expensive. Before choosing, consider:

  • Retries and quality. A weaker model that needs two attempts or human correction may cost more overall.
  • Data handling. Read where each provider processes and stores data, particularly for customer information. A low price does not answer that question.
  • Rate limits. Free tiers fail exactly when you get traffic, so do not put them in front of paying customers.
  • Engineering time. One day spent maintaining a server can exceed years of API charges at low volume.

For the wider decision, read how to choose an LLM for your product.

A practical path

  1. Prototype on a free tier or local model.
  2. Launch on a budget hosted API, with a monthly spending cap.
  3. Route by task: a cheap model for simple work and a stronger one for hard cases. This is often the largest single saving, along with caching. How to control AI API costs covers both.
  4. Log your real token usage for a few months.
  5. Revisit self-hosting only when your actual numbers approach the break-even, or when privacy requires it.

Need a second opinion on your AI architecture or costs? Book a call.

Common questions

Is it cheaper to run an LLM locally?

Only in some cases. If you already own suitable hardware, the marginal cost is mainly electricity. Buying hardware just to save money rarely pays off at low volume, because budget APIs are very cheap. Privacy and offline use are often stronger reasons.

How much does it cost to rent a GPU for an LLM?

RunPod lists an RTX 4090 at $0.34 an hour on Community Cloud and $0.89 on Secure Cloud, which is about $248 and $650 for a full month. You pay while it is idle, so renting suits steady high volume or short batch jobs.

Are free LLM APIs good enough for production?

Generally not. OpenRouter's free models are capped at 50 requests a day, or 1,000 once you have bought at least 10 credits, and availability can vary. They are good for testing and prototypes, but paying customers need dependable limits and support.

At what volume should I self-host an LLM?

Compare your monthly machine and staff costs with your blended API price per million tokens. Using a $248 rented GPU and budget API rates, break-even falls around 550 million to 1.1 billion tokens a month, before staffing, and only if one card can handle that load.

Have something worth building right?