Add per-store topology support#219
Open
jinzha wants to merge 1 commit into
Open
Conversation
A proxy can serve more than one NoSQL store. Keeping one shared topology for
the proxy can cause a query to use shard information from the wrong store.
Track topology separately for each store and send the cached topology version
for every known store with each request. Read topology updates and query branch
store names from the proxy, then cache them by store.
For advanced queries, select the needed store topology once when execution
starts and use that same snapshot for all later query batches.
Keep the existing topology behavior for older proxies that do not support
per-store topology.
Protocol changes:
- Add per-store topology sequence numbers in the NSON request header:
h.sts: [{ sid: <store name>, ts: <topology sequence> }, ...]
The SDK sends both the existing legacy topology sequence and h.sts, allowing
proxies to use per-store topology when supported while preserving rolling
compatibility.
- Add per-store topology refreshes in responses:
stp: [{ sid: <store name>, pn: <topology sequence>, sa: [<shard IDs>] }, ...]
The SDK caches these immutable topology entries by store name. Existing legacy tp response handling remains
unchanged.
- Add "qbs" to prepare/unprepared-query response:
qbs: [string,..]
It is a store-name array aligned with prepared-query branches (qb). The SDK
records this branch-to-store mapping in PreparedStatement.
- For each prepared query batch, serialize the selected branch’s store name
as "sid"", allowing the proxy to validate the execution branch’s store
identity.
- At advanced-query execution start, the SDK freezes the topology for each
query branch from the per-store cache.
After a prepare response refreshes the cache, it updates the original
request so internal batch requests carry the same topology snapshot used by
local scanners.
- For legacy prepared plans without "qbs", compiled by old proxy, the SDK uses
the legacy topology. If it is unavailable, the SDK requires reprepare rather
than guessing from cached per-store topology.
- Binary/V3 topology handling remains legacy-only and unchanged.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A proxy can serve more than one NoSQL store. Keeping one shared topology for the proxy can cause a query to use shard information from the wrong store.
Solution:
Changes:
Add per-store topology sequence numbers in the NSON request header:
h.sts: [{ sid: <store name>, ts: <topology sequence> }, ...]The SDK sends both the existing legacy topology sequence and h.sts, allowing proxies to use per-store topology when supported while preserving rolling compatibility.
Add per-store topology refreshes in responses:
stp: [{ sid: <store name>, pn: <topology sequence>, sa: [<shard IDs>] }, ...]The SDK caches these immutable topology entries by store name. Existing legacy tp response handling remains unchanged.
Add "qbs" to prepare/unprepared-query response:
qbs: [string,..]It is a store-name array aligned with prepared-query branches (qb). The SDK records this branch-to-store mapping in PreparedStatement.
For each prepared query batch, serialize the selected branch’s store name as "sid", allowing the proxy to validate the execution branch’s store identity.
At advanced-query execution start, the SDK freezes the topology for each query branch from the per-store cache. After a prepare response refreshes the cache, it updates the original request so internal batch requests carry the same topology snapshot used by local scanners.
For legacy prepared plans without "qbs", compiled by old proxy, the SDK uses the legacy topology. If it is unavailable, the SDK requires reprepare rather than guessing from cached per-store topology.
Binary/V3 topology handling remains legacy-only and unchanged.