A LiteLLM circuit breaker that kills runaway agent runs

July 16, 2026

The bill that ruins a Monday doesn’t come from a break-in. It comes from one agent run that got stuck. A tool started failing on Friday night; the agent called it, saw the error, and — because nothing told it to stop — called it again and again all weekend, full price every time. Nobody stole anything. The meter just ran.

The uncomfortable part is the number behind it: in these post-mortems, most of the money is spent after the model has already seen the first error. It kept going anyway. So the one thing you cannot build your defense on is the agent deciding to stop itself.

A stuck loop isn't a breach. It's a bill — and the thing running it already saw the error and kept paying.

Why your existing caps don’t catch it

LiteLLM is an open-source AI gateway — a single doorway your apps call instead of talking to each AI provider directly. A lot of teams run it (roughly 53 thousand have starred it on GitHub), and if you’re one of them you already have monthly budgets and rate limits. They’re good. They’re also the wrong granularity for this.

Both watch aggregates across many requests. The runaway lives inside a single run. There is no factory setting that says “never let one run cost me more than X” — so you have to add it.

The idea: a per-run, failure-aware breaker

Three signals separate a runaway from honest work:

  1. Identical call, twice in a row. A stuck agent re-emits the exact same tool call with the same arguments. A working agent makes progress — different calls each step.
  2. Same tool errors, twice in a row. A broken tool fails on the same call repeatedly. Don’t wait for a window to fill; the second strike is enough.
  3. Session budget. A hard ceiling per run, enforced before the next call, so the worst a bad run can cost is one run’s budget — not the farm.

Everything fails closed: when a run trips, the safe state is off, not “keep pouring.” A long legitimate task moves through many different calls and never trips — that’s the test that matters, and the demo below proves it. (This is the same idea as the fail-closed gateway, now on the request path of a gateway you already run.)

Enforcement belongs on the request path, where you can still act — not on the invoice, where all you can do is read about it.

For the engineers: it’s a LiteLLM plugin

LiteLLM lets you add a custom “guardrail” with two hooks — one before each call, one after. That’s exactly enough. All the decision logic is a small, framework-free breaker; the plugin just translates LiteLLM’s data into events:

class GuardrailBreaker(CustomGuardrail):

    async def async_pre_call_hook(self, user_api_key_dict, cache, data, call_type):
        run_id = run_id_from(data)                 # from client metadata.run_id
        if self.breaker.status(run_id).tripped:    # already cut? refuse the next call
            self._kill(run_id, ...)                # -> HTTP 429, the run stops here
        if not self.breaker.check(run_id).allowed: # over session budget?
            self._kill(run_id, ...)
        return data

    async def async_post_call_success_hook(self, data, user_api_key_dict, response):
        run_id = run_id_from(data)
        cost = litellm.completion_cost(completion_response=response) or 0.0
        sig  = response_action_signature(response)              # the model's tool call
        ok   = not last_tool_result_errored(data["messages"])   # did the last tool error?
        self.breaker.record(run_id, sig, ok=ok, cost_usd=cost)  # may trip the run
        return None

Two conventions have to be explicit, because LiteLLM sees the chat call, not your tool running:

Inside the breaker: how it tells a loop from real work

The plugin is only glue. The decision lives in a ~180-line, dependency-free Breaker — plain Python you can unit-test with no proxy, no Redis, and no model. It keeps a little state per run and answers one question after every call: did this run just go bad?

Start with what counts as “the same call.” Every tool call collapses to a short signature — the tool name plus a hash of its arguments — so search_db({"id": 42}) always maps to the same string, and any different query maps to a different one:

def signature_of(name: str, args) -> str:
    blob = json.dumps(args or {}, sort_keys=True, separators=(",", ":"), default=str)
    return f"{name}#{hashlib.sha1(blob.encode()).hexdigest()[:8]}"

After each call, record() stores (signature, ok, cost) and looks at the tail of the run — the calls back-to-back from the end. That’s the whole trick: a stuck agent’s tail is the same signature again and again; a working agent’s tail is different signatures, because it’s making progress.

# counting from the end, how many times has THIS exact signature repeated…
consec_same = 0
for e in reversed(r.events):
    if e["sig"] != sig: break
    consec_same += 1

# …and how many of those, from the end, were errors?
consec_fail = 0
for e in reversed(r.events):
    if e["sig"] != sig or e["ok"]: break
    consec_fail += 1

if   consec_fail >= 2:   trip("same tool errored twice in a row — broken tool, cut run")
elif consec_same >= 2:   trip("identical call twice in a row — stuck loop, cut run")

Two in a row is enough — you don’t wait for a five-minute window to fill while the meter runs. (A slower windowed layer sits behind this for patient loops that alternate a couple of calls, plus absolute call-count and cost backstops — belt and suspenders.)

Read the tail of the run. A stuck agent repeats itself; real work never does.

And the budget is checked before the call, so a bad run is cut before it spends — not billed and regretted after the fact:

def check(self, run_id, projected_usd=0.0):
    r = self._run(run_id)
    if r.tripped:                                       # already cut → refuse the next call
        return Gate(allowed=False, reason=r.reason)
    if self.run_budget_usd and r.cost + projected_usd > self.run_budget_usd:
        return Gate(allowed=False, reason="would exceed this run's session budget")
    return Gate(allowed=True)

No model sits in this loop deciding whether to stop — the model already proved it won’t. It’s cheap state, read from the end of the run, enforced on the request path.

Does it actually work? Live, against a local model

Talk is cheap; runaway bills aren’t. I wired the plugin into LiteLLM in front of a local model (so the test cost nothing), priced synthetically at real-provider rates so the dollars on screen are the dollars you’d actually burn. Then I pointed a deliberately broken agent at it:

  mode = BROKEN AGENT
  turn 1:  model -> search_db({"query": "SELECT COUNT(*) FROM orders WHERE customer_id = 42"})
  turn 2:  model -> search_db({"query": "SELECT COUNT(*) FROM orders WHERE customer_id = 42"})
  turn 3:  ✂  PROXY CUT THE RUN
           reason: identical call repeated 2x in a row — stuck loop, cut run

Three turns instead of three days. The session budget is a separate safety net — so even work that never repeats can’t overspend. With a $0.01 per-run budget, an agent making genuinely different calls each turn was still cut the moment it hit the ceiling:

  call 16:  ok
  call 17:  ✂  CUT (429)  reason: run would exceed its $0.01 session budget — cut before spending

And the test that keeps it honest: a legitimate task with distinct steps is never touched. In the standalone simulator, an unguarded broken agent runs up $98.51; the same agent with the breaker is cut for $0.24. A real multi-step task beside it finishes untouched. No false positive, or the whole thing is worthless.

It works with LiteLLM, not instead of it

To be clear about what this is: LiteLLM is a mature gateway — 100-plus providers, monthly budgets, caching, load-balancing. Keep all of it. This adds the one layer it doesn’t have — per-run failure detection — as a plugin that rides on top. You don’t switch anything out; you drop in about sixty lines and pass a run id. (Not on LiteLLM? The same breaker also ships as a tiny standalone gateway with a one-command demo.)

The code — breaker, LiteLLM plugin, the local demo, and the tests — is on GitHub: github.com/smallestbusiness/guardrail.

If you’ve wired agents into a metered AI service and you can’t say, off the top of your head, what your worst-case single run could cost before anyone notices — that’s the exact gap I check. Send me your setup or a recent bill and I’ll show you where the meter is open and where the breaker goes — free, within a business day. Show me your meter.

Free live workshop: cap your AI spend (Oct 8)

A hands-on 90-minute session — wire a fail-closed spend cap + cost-aware rate limiting against a real stack, live, so a leaked key or runaway agent can't run up your bill. Full details & times → Save your seat (and get each new lesson as it lands):

Double opt-in — one email to confirm. The lessons are free; the course is optional. No spam, unsubscribe anytime.