Archivedistributed-systems
CAP in Practice: What AP Actually Buys You
Network partitions are not rare events, and the interesting systems never pick one side cleanly.
- Published
- Reading time
- 2 min read
The CAP theorem says that under a network partition a distributed system must give up either consistency or availability. In practice the theorem is best read as a design boundary, not a menu: the interesting question is not A or C, but which operations degrade, and how.
Partitions are not exotic
A partition is any failure that makes a node group unreachable — a GC pause that outlasts the liveness timeout, a switch flap, a routing change, a delivery that drops silently. Any of these looks identical at the receiving end. Design for partitions by treating time as the unreliable input, not just the network.
What "C" actually means
Client-visible linearizability. In CP-leaning systems (ZooKeeper, etcd, Spanner in its synchronous paths) an operation might wait for a quorum, then fail if not enough members respond. Reads are fresh; availability dips exactly when you most need to see the data.
What "A" actually means
Eventually-consistent stores (Dynamo-style AP systems) accept writes on the healthy nodes and reconcile later: last-write-wins per key, or CRDTs when merging must be deterministic. Availability stays high through partitions, and in exchange the same read can return different values at different nodes not just sometimes, but by design.
Two questions that cut deeper than the acronym
- Which reads can tolerate staleness, and by how much? Read-your-own- writes is usually non-negotiable; cross-user consensus rarely is. Route reads accordingly instead of paying quorum latency for every one.
- What happens when the partition heals? Anti-entropy, tombstones, causality tracking — the recovery path is where correctness bugs live.
A practical shape
- Sessions state + shopping carts: write-ahead, per-record, one writer.
- Auth and quotas: strongly consistent, quorum-backed, small and hot.
- Feed, analytics, recommendation: eventually consistent and partitioned by user so a single user's state lives wholly in one cell.
The hard lesson of CAP is not "pick two of three" — it is that every system pays a partition bill somewhere. Deployments differ in where they place the charge, and reading that placement is a big part of reviewing a distributed system design.