Archiveperformance
Coordinated Omission Ruins Load Tests
If your load generator waits for a response before scheduling the next request, it hides the latency spikes users actually see.
- Published
- Reading time
- 1 min read
A closed-loop load test sends the next request only after the previous one completes. When the server stalls, the generator stops sending — so the stall never appears in the latency samples for the requests that were not sent.
The distortion
Imagine a 10 s freeze in a test that should send 100 req/s. The generator sends one request, waits 10 s, then sends the next. You see one slow request and a gap, not 1,000 requests delayed by 10 s. The reported p99 looks fine.
Fixes
- Open-loop generators that schedule at fixed arrival times regardless of responses (wrk2, k6 with constant-arrival-rate).
- Correcting for the stall: count the requests that should have been sent during the stall and attribute the delay to them.
- Measure queueing, not just service time: instrument time-in-queue per request.
Also watch
Test data size, connection reuse, and TLS session state — a test that reuses a warm pool does not measure the cold path.
Takeaway: match the generator's model to real arrival patterns. Closed-loop tests answer "how fast can N clients go?", not "what do users experience at load?".