Skip to content

Add Mnemosyne architecture design diagrams (S1 and S2)#376

Draft
palashkondekar-meesho wants to merge 2 commits into
developfrom
claude/feature-store-rust-rocksdb-NiKQc
Draft

Add Mnemosyne architecture design diagrams (S1 and S2)#376
palashkondekar-meesho wants to merge 2 commits into
developfrom
claude/feature-store-rust-rocksdb-NiKQc

Conversation

@palashkondekar-meesho
Copy link
Copy Markdown

Context:

This PR adds architectural design diagrams for Mnemosyne, documenting two distinct deployment strategies for the feature store system. These diagrams serve as visual references for understanding the system topology and component interactions.

Describe your changes:

Added two Excalidraw design documents:

  1. mnemosyne-s1-deployable-per-shard.excalidraw: Illustrates the S1 architecture with shard-based deployment strategy

    • Shows Databricks/PySpark job generating shard-specific files
    • GCS versioned file storage
    • Control plane managing onboarding, sizing, and cost
    • Client SDK with scatter-gather pattern
    • Per-shard deployable units with Envoy load balancing
    • Multiple read replicas (pods) per shard with RocksDB engines
  2. mnemosyne-s2-scattered-regions.excalidraw: Illustrates the S2 architecture with TiDB-like region-based strategy

    • Similar data generation and storage pipeline
    • Control plane with placement driver for region management
    • Client SDK with region-map and redirect handling
    • Scattered pod deployment where each pod hosts multiple non-contiguous regions
    • Region replication factor across pods

Both diagrams document the complete data flow from generation through serving, including control plane interactions and client-side logic.

Testing:

N/A - These are design documentation artifacts (Excalidraw JSON files) with no executable code or tests.

Monitoring:

N/A - Design documentation only.

Rollback plan:

N/A - Design documentation only. Can be removed or updated without affecting system operation.

Checklist before requesting a review:

  • I have reviewed my own changes
  • Documentation has been added (design diagrams)

📂 Modules Affected:

  • docs (Documentation updates)

✅ Type of Change:

  • Documentation

https://claude.ai/code/session_01FM9sB54w7ry9GH7iU2qbqv

…orm LLD

Excalidraw component diagrams for the two sharding strategies under review:
- S1: deployable-per-shard (read-replicas, Envoy intra-shard LB)
- S2: scattered regions (TiDB-like, Placement-Driver-managed)

https://claude.ai/code/session_01FM9sB54w7ry9GH7iU2qbqv
@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented May 27, 2026

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 8a53a148-1ee0-405c-b379-4796820b5728

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Comment @coderabbitai help to get the list of available commands and usage tips.

…-tenant, internal scaling)

Companion to the Mnemosyne LLD. Shows the onboarding lane (admin -> API -> sizer
-> tenant outputs), shared Control Plane + Producer/GCS, and three tenant lanes
with per-tenant Client SDK and internal shard deployables (one with a scaled-up
replica to illustrate horizontal scaling within a shard).

https://claude.ai/code/session_01FM9sB54w7ry9GH7iU2qbqv
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants