Skip to content

Add per-store topology support#219

Open
jinzha wants to merge 1 commit into
mainfrom
feature/per-store-topo
Open

Add per-store topology support#219
jinzha wants to merge 1 commit into
mainfrom
feature/per-store-topo

Conversation

@jinzha

@jinzha jinzha commented Jul 22, 2026

Copy link
Copy Markdown
Member

A proxy can serve more than one NoSQL store. Keeping one shared topology for the proxy can cause a query to use shard information from the wrong store.

Solution:

  • Track topology separately for each store and send the cached topology version for every known store with each request.
  • Read topology updates and query branch store names from the proxy, then cache them by store.
  • For advanced queries, select the needed store topology once when execution starts and use that same snapshot for all later query batches.
  • Keep the existing topology behavior for older proxies that do not support per-store topology.

Changes:

  • Add per-store topology sequence numbers in the NSON request header:
    h.sts: [{ sid: <store name>, ts: <topology sequence> }, ...]
    The SDK sends both the existing legacy topology sequence and h.sts, allowing proxies to use per-store topology when supported while preserving rolling compatibility.

  • Add per-store topology refreshes in responses:
    stp: [{ sid: <store name>, pn: <topology sequence>, sa: [<shard IDs>] }, ...]
    The SDK caches these immutable topology entries by store name. Existing legacy tp response handling remains unchanged.

  • Add "qbs" to prepare/unprepared-query response: qbs: [string,..]
    It is a store-name array aligned with prepared-query branches (qb). The SDK records this branch-to-store mapping in PreparedStatement.

  • For each prepared query batch, serialize the selected branch’s store name as "sid", allowing the proxy to validate the execution branch’s store identity.

  • At advanced-query execution start, the SDK freezes the topology for each query branch from the per-store cache. After a prepare response refreshes the cache, it updates the original request so internal batch requests carry the same topology snapshot used by local scanners.

  • For legacy prepared plans without "qbs", compiled by old proxy, the SDK uses the legacy topology. If it is unavailable, the SDK requires reprepare rather than guessing from cached per-store topology.

  • Binary/V3 topology handling remains legacy-only and unchanged.

A proxy can serve more than one NoSQL store. Keeping one shared topology for
the proxy can cause a query to use shard information from the wrong store.

Track topology separately for each store and send the cached topology version
for every known store with each request. Read topology updates and query branch
store names from the proxy, then cache them by store.

For advanced queries, select the needed store topology once when execution
starts and use that same snapshot for all later query batches.

Keep the existing topology behavior for older proxies that do not support
per-store topology.

Protocol changes:
  - Add per-store topology sequence numbers in the NSON request header:
      h.sts: [{ sid: <store name>, ts: <topology sequence> }, ...]
    The SDK sends both the existing legacy topology sequence and h.sts, allowing
    proxies to use per-store topology when supported while preserving rolling
    compatibility.

  - Add per-store topology refreshes in responses:
      stp: [{ sid: <store name>, pn: <topology sequence>, sa: [<shard IDs>] }, ...]
    The SDK caches these immutable topology entries by store name. Existing legacy tp response handling remains
    unchanged.

  - Add "qbs" to prepare/unprepared-query response:
      qbs: [string,..]
    It is a store-name array aligned with prepared-query branches (qb). The SDK
    records this branch-to-store mapping in PreparedStatement.

  - For each prepared query batch, serialize the selected branch’s store name
    as "sid"", allowing the proxy to validate the execution branch’s store
    identity.

  - At advanced-query execution start, the SDK freezes the topology for each
    query branch from the per-store cache.
    After a prepare response refreshes the cache, it updates the original
    request so internal batch requests carry the same topology snapshot used by
    local scanners.

  - For legacy prepared plans without "qbs", compiled by old proxy, the SDK uses
    the legacy topology. If it is unavailable, the SDK requires reprepare rather
    than guessing from cached per-store topology.

  - Binary/V3 topology handling remains legacy-only and unchanged.
@oracle-contributor-agreement oracle-contributor-agreement Bot added the OCA Verified All contributors have signed the Oracle Contributor Agreement. label Jul 22, 2026
@jinzha
jinzha requested a review from connelly38 July 22, 2026 14:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

OCA Verified All contributors have signed the Oracle Contributor Agreement.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant