Archivestorage
Erasure Coding Beats Replication on Storage, Not on Latency
k data shards plus m parity shards survive m losses for a fraction of the bytes — at the cost of reconstruction.
- Published
- Reading time
- 1 min read
Replication stores full copies; erasure coding splits data into k shards and computes m parity shards. Any k of k + m reconstruct the object, so overhead is m/k instead of (r-1)/r for replication.
The trade
- 3× replication: 200% overhead, survives 2 failures, reads are trivially parallel.
k=10, m=4: 40% overhead, survives 4 failures, any read needs 10 shards.
Reconstruction requires network reads from peers and CPU for Reed-Solomon, which raises tail latency. That is why hot data is often replicated and cold data erasure-coded.
Where it fits
Object storage, archival, and large immutable blobs. Not for small objects (shard overhead dominates) or latency-critical random reads.
Watch out for
- Rebuild storms after a node loss: schedule reconstruction with bandwidth budgets.
- Placement: shards must not share a failure domain (rack, AZ, power).
- Durability math:
mfailures tolerated simultaneously, not per year.
Takeaway: erasure coding optimizes bytes per durability. Replication optimizes recovery time. Most systems need both, applied to different temperature tiers.