The $55k mistake: LLM keys in the wrong place
We’ve spent ten days on cost and abuse. Now the base the whole series rests on — the security plumbing — and it starts with the least glamorous, most common way to lose five figures overnight: the key in the wrong place. The $55k Gemini bill, and the others like it, almost never begin with a clever exploit. They begin with a credential a stranger could read.
An LLM key is a bearer credential for your wallet
That’s the whole mental model. Whoever holds your API key can spend your money, at your provider, until you notice and revoke it. There’s no second factor, no per-charge approval — possession is authorization. Treat it exactly like you’d treat a signed blank cheque, because that’s what it is.
Ship a key to the browser and you've shipped it to everyone.
So the rule is simple and absolute: never ship the key anywhere the user can read it.
Where keys leak
- Client-side code. A key in your frontend JavaScript is public — “view source” is all it takes.
Frontend env vars don’t help: anything prefixed
NEXT_PUBLIC_,VITE_,REACT_APP_is compiled into the bundle and shipped to the browser. Same for keys baked into a mobile app — decompiling the binary to pull the string is trivial. - Repositories. A committed
.env, a hardcoded constant, a key pasted into a public gist. And crucially, git history: rotating the key doesn’t un-leak the old one that’s still sitting in a past commit for anyone who clones. - Logs, errors, and prompts. Keys printed into logs or error traces, or — the AI-specific one — placed in a system prompt, which can leak (LLM07). Never put a secret anywhere the model can repeat it.
The handling discipline
None of this is new. It’s the credential hygiene the rest of security has practiced for decades, pointed at a new, expensive target:
- Keep the key server-side, full stop. The client calls your backend; your backend (or the gateway) holds the provider key and makes the LLM call. The key never touches the browser or the app. This one rule prevents most disasters.
- Scope it to the minimum. Use the provider’s key restrictions — limit it to the specific API (just the model endpoint), to your app/referrer/IP, to the models you actually use. An unrestricted key is a skeleton key; the Gemini abuse wave was largely old, unrestricted keys.
- Vault it; don’t hardcode it. A secrets manager, or a server-side env injected at runtime — not in the container image, not in the repo. Rotate on a schedule and immediately on any suspicion.
- One key per app and environment. So a leak is contained and you can revoke the compromised one without taking everything down.
- Scan for exposure. Run secret scanners — gitleaks, TruffleHog (the tool that surfaced some of the public Gemini-key abuse), GitHub secret scanning — across your repos and your CI. Assume a leaked key is burned forever; rotate, don’t just delete the commit.
The pattern in code
The whole discipline fits in a few lines. The anti-pattern is a literal key in code that ships; the fix is loading the real key from the environment at the gateway, and handing your apps a virtual key instead:
# ANTI-PATTERN — the provider key is baked into code that ships to users:
client = OpenAI(api_key="sk-proj-abc123...") # now in the bundle, the repo, git history
# GOOD — the real key lives ONLY at the gateway, read from the environment at
# startup (your secrets manager injects it; it's never hardcoded or committed).
import os
provider_key = os.environ["OPENAI_API_KEY"] # server-side only; rotate = swap + restart
gateway = OpenAI(api_key=provider_key) # only the gateway calls the provider
# Your apps never see that key. They hold a scoped, rotatable VIRTUAL key the
# gateway issues, and point base_url at the gateway — revoke it without touching the real one.
app = OpenAI(
api_key=os.environ["GATEWAY_VIRTUAL_KEY"], # per-app, revocable on its own
base_url="https://your-gateway/openai",
)The first line is the entire disaster in miniature: a bearer credential compiled into something a
stranger can read. The fix isn’t clever — the real key is read from os.environ at server startup, so
rotation is a config swap and a restart, never a code change or a redeploy of your app. Everything
downstream, your own services included, authenticates with the gateway’s virtual key, which you scope
per app and revoke on its own.
Where the gateway helps
Routing every call through the gateway means the real provider key lives in exactly one server-side place. Your app code holds a per-user virtual key the gateway issues (the same virtual keys that carry the caps and quotas from Day 4) — so even your own services never handle the master credential, and you can revoke a virtual key without rotating the real one.
This is the instinct 20 years of logins, key management, and WebAuthn work builds: a credential is a liability you minimize the exposure of. An LLM key is just a new, unusually expensive bearer token, and it deserves the same paranoia.
One honest boundary: perfect key hygiene stops the leaked-key catastrophe — the headline five-figure bill. It does not stop abuse by a legitimate, authenticated user; that’s what caps and quotas are for. Different failure, different control.
Next in the series — Day 12: rate limiting you already know, now for tokens — because once the key is safe, the next question is how fast any one holder can spend.
If your AI feature ships a key to the browser, bakes one into an app, or has one sitting in git history, that’s a five-figure bill waiting for a scraper — and finding it is part of the audit I do. Every message comes straight to me — I read and reply to each one myself, usually within a day, and what readers send shapes what I build next. It’s just me for now, so that’s genuinely true; it won’t be forever. Send me your setup and I’ll tell you where your keys are exposed — free, within a business day.