LLM API Pricing Explained: What You're Actually Paying For

Almost everyone underestimates their first LLM bill, and almost always for the same three reasons. This is a plain-English walk through how the pricing actually works, with the arithmetic written out so you can apply it to your own project before you build it.

Tokens, briefly

You are not billed per word or per character. You are billed per token, which is the unit the model actually reads and writes. A token is roughly a common word, a fragment of a longer word, or a piece of punctuation.

The practical rule for English prose is about four characters per token, or about 0.75 words per token. So a 1,000-word document is somewhere near 1,300 tokens. It is an approximation, not a guarantee: code tokenises less efficiently because of all the symbols and indentation, and languages that do not use the Latin alphabet often use considerably more tokens for the same meaning.

Different model families also tokenise differently, so the same paragraph is not exactly the same token count for every provider. For budgeting purposes the difference is small enough to ignore; for a precise invoice reconciliation it is not.

Two prices, not one

Every provider quotes two rates, almost always per million tokens:

• Input — everything you send: the system prompt, the conversation history, any documents you paste in, and the user's current message.
• Output — everything the model writes back.

Output is the more expensive of the two, typically by a factor of three to five. The reason is mechanical rather than commercial. Your input can be processed in one parallel pass. The output cannot: the model produces one token, appends it, and runs again to produce the next. A 500-token reply means running the model 500 times in sequence.

This has a direct design consequence. Asking for shorter answers saves more money than sending shorter questions. If you are trying to reduce spend and you can only change one thing, cap the response length.

Working an example through

Take a model priced at $2.50 per million input tokens and $10.00 per million output tokens — roughly a mid-tier rate.

You send a 1,000-token prompt and get a 500-token answer:

• Input: 1,000 ÷ 1,000,000 × $2.50 = $0.0025
• Output: 500 ÷ 1,000,000 × $10.00 = $0.0050
• Total: $0.0075, about three quarters of a cent

That number is why people are relaxed about cost at the prototype stage. One call is nothing. The problem is that products do not make one call.

Ten thousand calls a day at that rate is $75 a day, or about $2,250 a month. Same request, same model, no change in quality — just volume. And this example still assumes each call is independent, which brings us to the part that catches people out.

The conversation-history multiplier

This is the one that produces genuinely shocking bills.

Models have no memory between calls. They are stateless. For a chatbot to "remember" what was said three messages ago, your application has to resend the entire conversation as input on every single turn.

Follow a conversation where each message is about 200 tokens:

• Turn 1: you send 200 tokens
• Turn 2: you send 200 + 200 + 200 = 600 tokens
• Turn 3: you send about 1,000 tokens
• Turn 10: you send about 3,800 tokens

By turn ten, a single message costs nineteen times what the first one did. Across a whole twenty-turn conversation you have not sent 4,000 tokens of input — you have sent closer to 40,000, because the early messages get re-billed on every subsequent turn. Input cost grows with roughly the square of conversation length.

The mitigations are straightforward once you know to apply them: cap history at the last N turns, summarise older context into a short paragraph rather than carrying it verbatim, or start a fresh conversation when the topic changes.

The system prompt tax

The second thing people miss. Your system prompt — the instructions defining how the assistant behaves — is sent as input on every single call.

A carefully engineered 2,000-token system prompt, at $2.50 per million, costs $0.005 per call. At 10,000 calls a day that is $50 a day, or $1,500 a month, just to repeat your instructions. Before any user has typed anything.

This is precisely where trimming pays. Cutting that system prompt from 2,000 to 1,200 tokens saves $600 a month, on the identical product. Filler, politeness and redundant framing carry no instruction and cost real money at scale — which is what our Prompt Optimizer is for.

Worth checking whether your provider offers prompt caching. Several now let you cache a stable prefix like a system prompt at a substantially reduced rate on repeat calls. If your system prompt is long and unchanging, this can be the single largest saving available to you.

Model choice is the biggest lever

Prices across the market span more than two orders of magnitude. The gap between the cheapest capable models and the flagship ones is enormous — and for a great many tasks the flagship is doing work that does not need it.

Things that rarely need an expensive model: classifying a message into categories, extracting fields from structured text, reformatting data, simple summarisation, routing a query to the right handler, basic sentiment work.

Things that often do: multi-step reasoning, code generation of any complexity, nuanced writing, anything where a subtle mistake is expensive to discover later.

Most production systems end up routing — a cheap model handles the bulk of traffic, and an expensive one is called only for the hard cases. Building that in from the start is far easier than retrofitting it after a bill arrives.

A realistic estimate before you build

Suppose a support chatbot: 1,000 conversations a day, averaging 6 turns, 150-token messages, 300-token replies, and an 800-token system prompt.

Per conversation, with history accumulating across six turns, input runs to roughly 8,000 tokens including the system prompt repeated each turn. Output is about 1,800 tokens.

• Input: 8,000 × 1,000 conversations = 8M tokens/day → at $2.50/M = $20/day
• Output: 1,800 × 1,000 = 1.8M tokens/day → at $10.00/M = $18/day
• Total: about $38/day, or $1,140/month

Now apply the three levers. Trim the system prompt to 400 tokens, cap history at the last three turns, and route the routine half of traffic to a model an order of magnitude cheaper. The same product lands nearer $300 a month. Nothing about the user experience changed.

Estimating your own

Paste a representative prompt into our LLM Token & Cost Calculator, set the expected reply length, and it will show the estimated per-call cost across sixteen models side by side. Multiply by your expected call volume, then apply the history multiplier if you are building anything conversational.

Two caveats on any estimate, ours included. Token counts from any estimator are approximations rather than exact provider counts, and published prices change — check the provider's own pricing page before committing to a serious budget.

The short version

• You pay per token, roughly four characters each.
• Output costs several times input, so capping reply length is the fastest saving.
• Conversation history is resent every turn, making long chats disproportionately expensive.
• Your system prompt is billed on every call — trim it.
• Model choice varies cost by more than a hundredfold; route easy work to cheap models.
• Estimate before you build, not after the first invoice.