5 ways an AI feature runs up an unbounded bill
Yesterday (Day 1) we priced a single LLM call: two dials, output the expensive half. But one call is a cent — nobody goes broke on one call. You go broke when the number of calls, or the tokens per call, goes unbounded. Here are the five ways that happens in real systems. Watch for the thread running through all of them: each is a fail-open default — the system’s instinct is to keep going and keep charging, not to stop.
1. The agent loop that doesn’t know when to quit
An agent calls the model to decide its next step, runs a tool, then calls the model again — and again. A planner re-plans. Two agents talk to each other. A “just retry the task” loop keeps trying. Every turn is a full, paid call, and with no hard step limit the loop runs until it errors out or hits a wall — and “until it errors” can be thousands of calls. It’s the AI version of the recursive function that quietly calls itself all weekend, except each iteration costs real tokens.
2. The retry storm
Your code retries a failed LLM call — sensible on its own. Then the provider slows down or rate-limits, and now every request is retrying three to five times. The moment your system is degraded is the exact moment your token spend multiplies. Retries without backoff and a budget turn a bad hour into a bad invoice.
3. No per-user cap
Your endpoint treats every caller the same and caps nobody. One power user, one automated script, one competitor curious what you’ll spend — a single actor can consume the entire budget, because nothing limits usage per identity. (It’s also the front door to using your feature as a free LLM — more on that later in the series.)
4. The leaked key
The fastest path to catastrophe. A key in client-side code, a public repo, a mobile app, or a Docker layer gets scraped within minutes and run at maximum throughput on your dime. This is the $55,000-Gemini shape: not your usage at all — just your key, in someone else’s hands.
5. Success itself — the viral spike
The cruelest one. The feature works, gets shared, hits the front page, and usage 100בs in a day. The meter scales with your success, and there’s no ceiling waiting to catch it. Your best traffic day becomes your biggest bill — the one moment you’d least want a surprise.
A 60-second self-check
Run down the five and mark the ones true of your setup today: an agent or tool-loop with no hard step limit; retries with no backoff or budget; an endpoint that caps nobody per user; a key that lives anywhere client-side, in a repo, or in a container image; a feature that could go viral with no ceiling waiting. Every box you tick is a way the meter can run without you — and most teams tick three or four while having a control for none of them. Knowing which you’re exposed to is the whole point of the exercise; the rest of this series is the control for each.
The thread: nothing here stops itself
All five share one property: the default behaviour is fail-open. The loop keeps looping, the retry keeps retrying, the key keeps working, the spike keeps spiking — and the bill keeps climbing — because nothing was built to say stop. Knowing the price of a call (Day 1) does nothing if a million of them can fire unattended.
That’s the entire case for a ceiling. Tomorrow (Day 3) we start building it: the gateway pattern — a single chokepoint in front of every LLM call, where caps, quotas, and a kill switch can actually live.
Following along? Join the list to get each lesson in your inbox. Shipping an AI feature and not sure which of these five you’re exposed to? Send it to me — I’ll take a look, free, within a business day.