Cloud AI cost spike (Bedrock, Vertex, Azure OpenAI)? The first things to check.
If you call OpenAI or Anthropic directly, a cost spike shows up on a tidy usage dashboard. If you run your AI through a cloud provider — Amazon Bedrock, Google Vertex AI, Azure OpenAI — it’s different in two ways that matter when you’re trying to find a spike fast: the cost is buried in the same bill as your servers, and it has cost drivers the direct APIs don’t even have.
Here’s where to look first.
1. Find it in the bill (it isn’t labeled “AI”)
There’s no neat “AI total.” Open your cloud billing breakdown and group by service, then by the line item under it:
- AWS: the Bedrock service → group by usage type / model.
- Google Cloud: Vertex AI → prediction / generative line items.
- Azure: Azure OpenAI → by model deployment.
Cloud AI cost hides in your server bill, not a separate dashboard.
The spike is in there; it’s just sitting next to your compute and storage rather than on its own screen. Group by day too, to find the date it jumped — same read-the-meter method as the direct APIs.
2. Provisioned throughput — the trap the direct APIs don’t have
This is the first thing to check, because it’s the one that surprises people. On the cloud you can buy provisioned throughput — reserved AI capacity — and it bills hourly, whether or not you use it. It’s the AI version of leaving a server running. One provisioned unit spun up for a load test and forgotten runs like an idle rental on the meter, all month. If your spike is flat and constant rather than tracking your traffic, suspect this first.
3. Model choice
Same rule as everywhere: a premium/frontier model costs many times a small one for the same call. A feature that switched model tiers — or defaulted to the expensive one — moves the bill. Check whether a pricier model appeared in the breakdown around your step-change date.
4. Embeddings and re-indexing loops
If you run retrieval / RAG (a system that searches your own documents to answer questions), the embedding step — turning documents into vectors — is cheap per call but brutal in bulk. A pipeline that re-embeds your whole knowledge base on a schedule, or on every deploy, can bill a fortune in individually-tiny calls. Check whether an indexing job started running more often.
5. The missing cap — same as everywhere
The cloud will not stop this for you. There’s no hard ceiling by default, budgets are alerts not caps, and the AI line has no more of a floor than the rest of the bill. Whatever the driver, the permanent fix is the same: a cap that fails closed, per model and per key, plus provisioned capacity you actually review.
The quick triage order
- Provisioned throughput running you didn’t mean to keep (flat, always-on spend).
- Which model — did an expensive one appear?
- Which day did it jump, and what changed then?
- An indexing/embedding job running more often.
- A per-key cap so it can’t run unbounded again.
If your cloud AI bill spiked and it isn’t obvious which of these it is, I can find it — the cloud console makes this harder than it should be. It’s just me, so it comes straight to me and I reply fast. Send me the details — provider, service, and when it jumped — and I’ll tell you what’s driving it and where to put the cap, free, within a business day.