How to implement

[seonWKim]

EastGuard Implementation Ideas

Actors

ClusterActor (one per node) — Manages membership and topology. Detects peer failures, disseminates membership
changes. Notifies ShardManagerActor when topology changes. Spawns/stops PeerActors as nodes join/leave. Publishes
topology snapshot for ConnectionActors.

PeerActor (one per remote node) — Manages the outbound connection to a remote node. Shared by ShardActors and
ConnectionActors for all communication with that node.

ShardActor (one per shard per node) — Manages a metadata shard. Replicated across multiple nodes — one leader, rest
followers. Stores metadata for topics, ranges, and segments. Leader issues commands to data-plane nodes. Communicates
with remote replicas via PeerActors.

ShardManagerActor (one per node) — Manages shard lifecycle on this node. Spawns ShardActors when this node becomes
responsible for a shard, stops them when it's no longer responsible.

ConnectionActor (one per client) — Manages an inbound client connection. Resolves which broker owns the requested
resource, handles locally or forwards via PeerActor.

Sharding Keys

• Topic metadata → hash(TopicName)
• Range + Segment metadata → hash(RangeId)

Ranges and their segments share the same sharding key so that atomic operations stay within a single shard. Topics are
sharded separately to avoid hotspots.

Shard #A
  └── Topic-Blue config       ← hash("Topic-Blue")

Shard #B
  └── Range-1 state           ← hash("Range-1")
        ├── Segment-A
        └── Segment-B

Shard #C
  └── Range-2 state           ← hash("Range-2")
        └── Segment-C

Diagram (single node)

◆ = actor

◆ ClusterActor ─────topology changed─────► ◆ ShardManagerActor
│                    │                       │
│                    │ node joined/left      │ spawns/stops
│                    │                       │
│                    ▼                       ▼
│               spawn/stop                   ◆ ShardActor (Shard #1, Leader)
│                    │                       ◆ ShardActor (Shard #3, Follower)
│                    ▼                       ◆ ShardActor (Shard #5, Follower)
│               ◆ PeerActor(Node B)          │
│               ◆ PeerActor(Node C)          │
│                    ▲                       │
│                    └───────────────────────┘
│
│                topology         Client
│                snapshot              │
│                    │                 ▼
│                    │            ◆ ConnectionActor ────read────► snapshot
│                    │                 │
│                    │   forward       │
│                    ◄─────────────────┘

Diagram (cluster)

  Node A                     Node B                     Node C
┌────────────────┐     ┌────────────────┐     ┌────────────────┐
│ ◆ Cluster      │     │ ◆ Cluster      │     │ ◆ Cluster      │
│ ◆ ShardMgr     │     │ ◆ ShardMgr     │     │ ◆ ShardMgr     │
│                │     │                │     │                │
│ ◆ Shard #1     │     │ ◆ Shard #1     │     │ ◆ Shard #1     │
│   (Leader)     │     │   (Follower)   │     │   (Follower)   │
│       │        │     │       ▲        │     │       ▲        │
│       ▼        │     │       │        │     │       │        │
│ ◆ Peer(B) ═════╪═════╪═► inbound      │     │       │        │
│ ◆ Peer(C) ═════╪═════╪════════════════╪═════╪═► inbound      │
│                │     │                │     │                │
│ ◆ Shard #2     │     │ ◆ Shard #3     │     │ ◆ Shard #2     │
│   (Follower)   │     │   (Leader)     │     │   (Leader)     │
└────────────────┘     └────────────────┘     └────────────────┘
        │                      │                       │
        └──────── membership protocol ─────────────────┘