Skip to content
Baldur

OpenAI · Anthropic · Gemini — one line for Python LLM workers

Your LLM provider rate-limits you, then goes down.
What happens to your jobs?

Every worker retries into the same 429, hung calls tie up the pool, and the jobs that failed in the outage are gone. Wrap your client once: the workers share one wait, and every job the outage stopped is kept — and comes back.

12
jobs stopped by an outage
12
came back
0
lost
3 / 3
runs, same result

Measured against the Gemini API free tier, outage injected by a local proxy: every job that came back got a real model answer. Without it, a job out of retries is simply gone.

Quickstart →

Apache-2.0 core · OpenSSF Best Practices: passing · adapters for Django, FastAPI, Flask, and Celery

jobs/summarize.py2 lines added
import baldur
from openai import OpenAI

llm = baldur.llm.wrap(OpenAI(), timeout=60.0)

@baldur.protected("summarize", replay=True)
def summarize(doc_id: str) -> str:
    reply = llm.chat.completions.create(
        model="gpt-4o-mini", messages=prompt(doc_id))
    return reply.choices[0].message.content
demo — summarize jobs2026-10-02
  1. 1 · provider down
    1. 50312 jobs failed in 1.3 s — 8 after every endpoint was tried, 4 rejected by the open breaker without calling the provider
    2. 50312 parked with their arguments
  2. 2 · back
    1. 200the provider is back — the breaker waits, probes, closes
    2. 200breaker closed → 12 parked jobs re-run automatically (dlq pending 0)
    3. 200parked 12 · replayed 12 · lost 0

The outage half of the demo's own output: the real openai SDK against a local fake provider, eight threads as the workers. Against the real Gemini API the same scene gave the same counts.

Run it yourself — one process, no Redis, no broker, no API key:

pip install "baldur-framework[celery]" openai
python -m baldur.scripts.demo_llm_outage

One line around your client, not a gateway to run.

  1. 01 — wrap

    Wrap the client you already have

    The OpenAI SDK, Anthropic, google-genai. Each provider error is sorted: wait, move to your next endpoint, or fail now.

  2. 02 — share

    One wait for every worker

    A 429 or an overload installs one wait the workers share — across processes through Redis — and the breaker keeps a dead provider from holding the pool.

  3. 03 — replay

    Stopped jobs come back

    A job no endpoint could answer is parked with its arguments and re-runs once the provider answers — on a Celery worker, or from the console with a click. A replay runs the whole job again: make it safe to run twice.

Your stack already fights failures —
in five places that don't talk.

Each layer solves its slice and drops the rest.

What each layer does when a dependency fails, and where it stops
Layer Retry Circuit breaker Keeps the failed call Replays it on recovery Where it stops
openai / anthropic SDK retries yes no no no Each worker waits alone; a job out of retries is gone.
tenacity + pybreaker yes yes no no Wiring the two together is on you.
Celery / SQS dead letters yes no yes no Parks the job, then waits for a human.
Envoy / Istio yes yes no no Sees connections, not your call’s arguments.
Temporal yes no yes yes Once you move the code into its workflow model.
baldur.llm.wrap + replay=True yes yes yes yes, with the prerequisites in the note below the table One line, in your process.

* Automatically on a Celery worker that imports the job; from the console otherwise. What it needs →

The overhead is measured —
not promised.

Every figure is a dated measurement on a stated topology.

  • Per protected call, in-memory ~39 µs decorator alone, no network
  • Per protected call, over Redis +1.4–1.8 ms server CPU, idempotency on, below saturation
  • Per-request overhead +1.1% throughput, below saturation
  • Dead-letter entry, stored ~953 B of Redis per parked call

Topology moves all of these. How each was measured, and the one we withdrew →

Wrap one client today. Add Redis when you run more than one process.

Quickstart →

LLM rate limits, in depth → The console → OSS vs PRO → Pricing →