Pools and Bounded Resources¶
A pool reuses or limits resources such as database connections, platform threads, buffers, or clients. Its most important property is usually the bound, not reuse. A pool cannot create downstream capacity; an oversized pool can increase contention and make overload slower rather than more successful.
Capacity model¶
Little's Law relates long-run averages in a stable system:
It is not a sizing formula by itself, because real traffic is bursty and tail latency matters. It does show why slow dependencies retain permits longer and raise concurrency. Measure service time, queue time, utilization, timeouts, and saturation under representative load.
Executors and queues¶
CPU-bound work generally benefits from concurrency near available processing capacity. Blocking work may use more tasks, but remains bounded by memory, connections, rate limits, and the dependency. An unbounded executor queue hides overload as growing latency and retained memory.
A saturation policy must say whether to reject, block the producer, run in the caller under safe conditions, degrade, or durably enqueue. Submitted tasks also need cancellation and deadline policies so obsolete work does not consume capacity.
Connection pools¶
Connections are stateful leases. Borrow for the shortest correct scope, close through structured cleanup so the lease returns even on exceptions, and avoid remote calls while holding a database transaction open. Set acquisition and query timeouts deliberately. Leak detection is diagnostic and does not replace ownership discipline.
Virtual threads can reduce the cost of waiting threads, but they do not add database connections, socket capacity, memory, or downstream throughput. Admission control and resource bounds still apply.
Multiple pools¶
Nested acquisition can deadlock when tasks hold one scarce resource while waiting for another. Document acquisition order, avoid submitting work to a pool and synchronously waiting for that same saturated pool, and reserve capacity only when isolation benefits justify underutilization.
Test saturation, timeout, cancellation, leaks, slow dependencies, shutdown, and recovery. Expose active and idle resources, queue depth and age, acquisition latency, timeouts, and rejection counts.