What an LLM call actually costs — and why output is the expensive half

July 6, 2026

This is Day 1 of building an AI cost-and-safety playbook in the open — one short lesson a day, from the cost mechanics to the abuse defenses, connecting each piece back to the security and infrastructure work it’s built on. (Why this, and why now: the LLM API is the most uncapped meter you’ve ever plugged in.)

Start where every AI bill starts: a single call.

You’re billed per token — and the two sides aren’t priced the same

An LLM call has two token counts, billed separately:

Input — everything you send in: the system prompt, the chat history, any retrieved documents, and the user’s message.
Output — everything the model generates back.

And output is the expensive one — typically 3–5× the price of input (exactly 5× on the Claude models: Opus 4.8 is $5 per million input tokens and $25 per million output). That ratio isn’t arbitrary. Input is processed in one parallel pass (prefill); output is generated one token at a time, each token a full pass through the model. Generation is the compute — so you pay for it.

Dial 1: how much you make it say

Because output is priced 5×, the length of the answer dominates the bill. Take a simple call — 400 input tokens, 600 output — on Opus 4.8:

input: 400 / 1M × $5 = $0.002
output: 600 / 1M × $25 = $0.015
total ≈ $0.017 — and output is 88% of it.

A feature that answers in three paragraphs instead of one hasn’t tripled its work; it’s tripled the expensive half of every call it will ever make. Verbose system prompts that beg the model to “explain your reasoning in detail” are a line item. Setting max_tokens sanely, and asking for concise output, is the first and easiest cut.

Dial 2: how much you make it read

Now flip it. A retrieval-augmented call — 10,000 input tokens of documents and history, 500 output:

input: 10,000 / 1M × $5 = $0.05
output: 500 / 1M × $25 = $0.0125
total ≈ $0.0625 — now input is 80% of it.

You pay for the entire prompt on every single call — and in a chat, the history grows each turn and gets re-sent and re-billed, so a long conversation quietly gets more expensive with every message. The fix is context discipline: trim the system prompt, prune the history, and right-size what you retrieve instead of stuffing the whole knowledge base in “just in case.”

There’s a lever here too: prompt caching. If a big chunk of your input is a stable prefix reused across calls (a fixed system prompt, a document set), cached reads run about a tenth of the normal input price — turning that $0.05 of context into roughly $0.005.

Dial 3 (bonus): which model reads it

The same 400-in / 600-out call, moved from Opus 4.8 to Haiku 4.5 ($1 / $5 per million):

input: 400 / 1M × $1 = $0.0004
output: 600 / 1M × $5 = $0.003
total ≈ $0.0034 — about 5× cheaper.

Not every call needs the biggest model. Routing the easy ones to a small model is one of the largest levers there is, for zero change to the feature.

The takeaway

Every LLM call has two dials — how much you make it say, and how much you make it read — plus a choice of which model reads it. Most teams watch none of them, which is how a feature that felt free in the demo becomes a line item nobody can explain.

But notice what’s missing from all of this: a ceiling. Knowing the price of a call tells you what one costs — it does nothing to stop a million of them. That’s the whole reason this is the meter you can’t see, AI edition: per-token, invisible, and unbounded. You can’t cap what you don’t measure — so today was measuring. Tomorrow (Day 2): the five ways those dials get yanked and the bill explodes — and after that, the ceiling.