Data Replication

The primary objective is to ensure continuous access to data despite hardware failures, network partitions, or planned maintenance. By maintaining multiple replicas, the system can automatically failover to a healthy copy if the primary node becomes unavailable. This is critical for mission-critical applications where downtime is unacceptable.

• Automatic Failover: Systems detect node failure and redirect traffic to a live replica.
• Redundancy: Multiple copies eliminate single points of failure.
• Disaster Recovery: Geographic replication protects against site-wide outages.

Replication enables load balancing of read requests across multiple nodes, significantly increasing throughput and reducing latency for geographically distributed users. Clients can read from the nearest replica, minimizing network round-trip time.

• Read Scaling: Add more replicas to handle increased read traffic linearly.
• Local Reads: Place replicas in edge locations close to end-users.
• Offloading Primary: The primary node handles writes, while replicas serve the majority of read queries.

Replication provides a real-time, distributed backup mechanism. Data is persisted across multiple independent storage devices or data centers, protecting against data loss from disk corruption, accidental deletion, or catastrophic events. This is often more efficient than traditional periodic backups.

• Multi-Region Storage: Copies exist in physically separate locations.
• Synchronous vs. Asynchronous: Trade-offs between write latency and durability guarantees.
• Point-in-Time Recovery: Some systems use replication logs for historical data restoration.

Replication introduces the challenge of keeping all copies synchronized. Different consistency models define the guarantees provided to clients reading from replicas, directly impacting system design.

• Strong Consistency: All reads see the most recent write. Simplifies application logic but increases latency.
• Eventual Consistency: Replicas converge to the same state given no new updates. Enables higher availability and lower latency.
• Read-Your-Writes Consistency: A user always sees their own updates, a common practical guarantee.

The architecture defining how data flows between nodes. The topology impacts latency, fault tolerance, and complexity.

• Single Leader (Primary-Secondary): All writes go to a primary node, which propagates changes to read replicas. Common in SQL databases (PostgreSQL, MySQL).
• Multi-Leader: Multiple nodes accept writes, which must be reconciled. Useful for geographically distributed writes but introduces conflict resolution complexity.
• Leaderless: Any node can handle reads and writes; the system uses quorums to ensure consistency. Used by Dynamo-style databases like Apache Cassandra.

In asynchronous or multi-leader replication, concurrent writes to different replicas can create conflicts. Systems must have deterministic strategies to resolve these conflicts.

• Last-Write-Wins (LWW): Uses timestamps, simple but can cause data loss.
• Version Vectors: Track causality between updates to detect conflicts.
• Application-Logic: Conflicts are surfaced to the application for custom resolution (e.g., merging user profiles).
• CRDTs (Conflict-Free Replicated Data Types): Data structures designed to merge automatically without conflict.

A set of four critical properties—Atomicity, Consistency, Isolation, Durability—that guarantee reliable processing of database transactions. For data replication, ACID principles ensure that replicated data transitions between consistent states, even during failures.

• Atomicity: A transaction's changes are applied all-or-nothing to the replica.
• Durability: Once a committed transaction is replicated, it persists despite system crashes. This is foundational for building strongly consistent replication systems.

A horizontal partitioning technique that splits a large dataset into smaller, more manageable pieces called shards, distributed across multiple database servers. Data replication is often applied within each shard to provide high availability and fault tolerance for that subset of data.

• Pattern: Combines sharding (for scale) with intra-shard replication (for resilience).
• Challenge: Requires careful coordination to ensure replication logic respects shard boundaries and maintains global consistency where needed.

A design pattern where the state of an application is determined by a sequence of immutable events stored in an append-only log. This log becomes the system's source of truth and a natural feed for data replication.

• Replication Synergy: The event log is inherently replicable; downstream systems can consume the event stream to reconstruct their own state or projections.
• Benefit: Provides a complete audit trail and enables temporal queries, making replication not just about data copying but state reconstruction.

A fundamental database protocol that ensures data integrity by writing all modifications to a persistent log file before the changes are applied to the main database files. The WAL is the engine behind crash recovery and a critical enabler for efficient data replication.

• Replication Role: Change Data Capture (CDC) systems often read from the WAL to capture changes with minimal performance impact on the primary database.
• Guarantee: Provides a durable, ordered record of all transactions, which is essential for building transactionally consistent replicas.

A special hashing technique used in distributed systems to minimize reorganization when the number of nodes (e.g., database servers in a cluster) changes. It is crucial for efficiently directing read/write operations in replicated and sharded data stores.

• Replication Context: When data is replicated across N nodes, consistent hashing helps determine the replica set for a given data key. When a node fails, only the data mapped to that node needs to be re-replicated to its successor, not the entire dataset.
• Benefit: Dramatically reduces the data movement required during cluster scaling or failure recovery in replicated systems.