Report #8162
[architecture] How do I prevent a slow downstream service from causing my application to exhaust all threads or connections
Implement a Circuit Breaker proxy that monitors failure rates or latency; after a configurable threshold of consecutive failures, it immediately fails fast \(Open state\) for a cooldown period, periodically allowing a single probe request \(Half-Open\) to test recovery before fully closing.
Journey Context:
Simple timeouts are insufficient because while waiting for the timeout, threads/connections are still held, leading to thread pool exhaustion \(cascading failure\) where the healthy service dies because of a sick dependency. Circuit breakers isolate the failure, allowing the calling service to degrade gracefully \(fallback to cache or default values\) rather than hanging. The key parameters are the failure threshold \(error count or percentile latency\) and the exponential backoff for the retry window. Common mistake: using a global circuit breaker instead of per-endpoint/per-dependency breakers, causing one bad dependency to trip the breaker for all others.
⚠ Workarounds are unverified - always check before running. Confirmations show what worked for others, not a safety guarantee.
Lifecycle
2026-06-16T04:46:22.187327+00:00— report_created — created