Skip to content

Latest commit

 

History

History
51 lines (36 loc) · 3.61 KB

README.md

File metadata and controls

51 lines (36 loc) · 3.61 KB
Raw

Distributed Key-Value Store (a la etcd)

Core Components and Architecture

  1. Nodes and Clustering

    • Set up the store to operate in a cluster of nodes. Each node is a separate instance that can communicate with others.
    • Implement a node discovery mechanism so nodes can locate each other when they join the cluster, either by static configuration or service discovery.

  2. Raft Consensus Algorithm

    • Implement Raft to handle leader election, log replication, and synchronization across nodes.
    • The leader node will handle write requests, replicating changes to follower nodes. This leader’s role ensures strong consistency across the cluster.
    • Raft provides safety guarantees by enforcing log consistency across nodes, allowing each entry in a distributed log to be applied in order across all nodes.

  3. Key-Value Storage with Replication

    • Each node will have a local storage (in-memory or persistent) to store key-value pairs.
    • The leader node will manage all write operations and replicate updates to followers, ensuring data consistency.
    • Use snapshots to periodically persist the state of each node and log compaction to free up memory by cleaning out old, unnecessary logs.

  4. Consistency and Partition Tolerance

    • Handle network partitions with mechanisms to promote partition tolerance. Use Raft’s commit index and term-based leader election to maintain consistency when nodes lose connectivity.
    • When a node rejoins, sync it with the cluster’s current state by using Raft logs or snapshots.

  5. gRPC for Communication

    • Use gRPC to define and handle node communication (e.g., leader election, heartbeat, replication).
    • Define gRPC services for requests like PUT, GET, DELETE, and other metadata operations (like JoinCluster, LeaveCluster).

  6. Transactions (Optional)

    • Add support for lightweight transactions by allowing conditional operations, like PUT_IF_ABSENT or COMPARE_AND_SET.
    • Support multi-key transactions, if desired, using a two-phase commit protocol or similar strategy for consistency.

Additional Features to Consider

  • Failure Detection and Heartbeat Mechanism: Implement a heartbeat mechanism for detecting node failures. If a node fails, a new leader election will occur.
  • Quorum-Based Reads and Writes: To balance availability and consistency, implement quorum reads/writes, where only a majority of nodes need to agree on a read/write operation.
  • Snapshot Management: Take snapshots of the data state to optimize recovery and reduce memory usage.
  • Monitoring and Metrics: Integrate observability with metrics (latency, replication time) and logging for cluster health, which is crucial for troubleshooting.

Example Flow: Writing Data

  1. Client Sends a Write Request to the Leader

    • The leader appends the entry to its log and sends replication requests to followers.

  2. Replication and Consensus

    • The leader waits for a majority of followers to acknowledge the write before committing it.
    • Followers apply the change to their local storage after the log entry is committed.

  3. Response to Client

    • Once the leader commits the write, it sends an acknowledgment to the client. If the leader fails mid-write, the new leader takes over based on the current logs.

This setup balances strong consistency (thanks to Raft), availability during node failures, and partition tolerance. It’s a great way to dive deep into distributed systems design and Go’s concurrency strengths. Let me know if you want to explore specific implementation details!