Archiveperformance
Little's Law: The Only Capacity Formula You Need First
Concurrency equals throughput times latency. Every queue-sizing argument follows from it.
- Published
- Reading time
- 1 min read
Little's law states that in a stable system, the average number of items in the system L equals arrival rate λ times average time in system W:
L = λ × W
It holds for any queueing system with stable flows, regardless of distribution. That makes it the first sanity check for capacity planning.
Using it
- 500 requests/s at 100 ms latency → 50 concurrent requests. A pool of 500 connections is waste; a pool of 10 is a bottleneck.
- A thread pool of 200 with 20 ms work → max ~10,000 req/s if no other limit.
- Given a concurrency limit and a latency SLO, you can bound throughput:
λ = L / W.
What it does not tell you
It says nothing about latency distributions, queueing delays under bursty arrivals, or service-time variance. For those you need queueing theory (M/M/1, Kingman's formula) — but start here.
The common error
Sizing by CPU utilization alone. Utilization says how busy the server is, not how many requests are in flight or how long they wait.
Takeaway: write L = λW on a whiteboard before any capacity debate. Most numbers people propose violate it.