Archivereliability
Timeouts Are Not Latency Budgets
A per-call timeout without a deadline propagates work you no longer need. Deadlines are the fix.
- Published
- Reading time
- 1 min read
A timeout says "stop waiting after N seconds". A deadline says "this request must be finished by T". The difference is whether downstream work is cancelled when the caller gives up.
The failure
Service A calls B with a 1 s timeout, B calls C with 1 s, C calls D with 1 s. A times out at 1 s, but B, C, and D keep working for another 2 s on a request nobody wants. Under load, that is pure waste.
Deadlines
Propagate an absolute deadline in request context (gRPC does this natively). Each hop subtracts its own processing time and refuses to start work it cannot finish. Cancellation should propagate too.
Setting them
- Derive from the user-facing SLO, not from vendor defaults.
- Keep per-hop timeouts smaller than the remaining budget.
- Separate connect, TLS, and response timeouts where the stack allows.
- Retry only if the remaining budget can absorb another attempt.
Takeaway: pass deadlines, not durations. A timeout without cancellation is a promise to waste capacity.