Rate Limits, Circuit Breakers, and Bulkheads

These controls protect different boundaries. Combining them can improve resilience, but treating them as interchangeable creates blind spots.

Control	Decision	Typical state
Rate limiter	May this caller or workload start now?	Tokens, windows, or queued permits
Circuit breaker	Should calls to a failing dependency be attempted?	Closed, open, half-open
Bulkhead	How much capacity may one workload consume?	Semaphore, pool, or queue partition

Rate limiting

A token bucket permits bursts up to its capacity while enforcing a refill rate. A fixed window is simpler but creates boundary bursts; sliding-window or generic cell-rate approaches approximate smoother limits. Define identity, fairness, distributed coordination, failure behavior, and the response contract. HTTP services commonly use 429 Too Many Requests and may provide Retry-After.

Circuit breakers

A breaker records recent outcomes while closed, opens when its policy declares the dependency unhealthy, and later permits a limited probe in half-open state. It must distinguish relevant failures from caller errors. A breaker fails fast; it does not bound a slow individual call, so calls still need timeouts or deadlines. A fallback must be semantically honest about stale or partial data.

Bulkheads and backpressure

Bulkheads isolate capacity so one dependency or workload cannot consume every thread, connection, or permit. Partitioning too aggressively wastes capacity; sharing everything permits correlated exhaustion. Bounded queues and explicit rejection expose overload earlier than unbounded latency.

Backpressure is a protocol through which a slower consumer constrains a faster producer. A queue is storage, not backpressure by itself. When a bound is reached, the producer must slow, reject, drop under an explicit policy, or transfer durably.

Interactions

Apply admission control before consuming scarce work. A call may pass a rate limit, acquire a bulkhead permit, then encounter an open breaker. Release permits on every completion path. Retries must respect the same controls; otherwise retry traffic can monopolize capacity during recovery.

Test thresholds around their boundaries, half-open concurrency, clock behavior, partitioned workloads, queue saturation, recovery, and configuration changes. Observe rejection reasons, breaker transitions, permit use, queue age, and latency. Keep metric dimensions bounded.