Skip to content

Enforce directional controlled-flow replication config on live connections#506

Merged
kriszyp merged 1 commit into
mainfrom
kris/498-controlled-flow
Jul 1, 2026
Merged

Enforce directional controlled-flow replication config on live connections#506
kriszyp merged 1 commit into
mainfrom
kris/498-controlled-flow

Conversation

@kriszyp

@kriszyp kriszyp commented Jun 29, 2026

Copy link
Copy Markdown
Member

Summary

Makes replication.routes[].replicates.sends/receives (controlled flow) actually constrain live traffic. Today the fields are stored and round-trip through get_configuration, but the runtime direction gates read replicates only from the hdb_nodes record (default true), so traffic flows fully bidirectionally regardless of the route config (harper-pro#498).

Four gates now honor the config route's direction, only for the directional object form ({ sends, receives, … }) — boolean and subscriptions-only routes keep their existing behavior:

  1. Receive (shouldReplicateFromNode): prefer the route's receives/receivesFrom, carried on a distinct configRouteReplicates payload field so it is never confused with an hdb_nodes sends/sendsTo object (which would regress the add_node controlled-flow path).
  2. Send-authorization (replicationConnection): honor the local route's sends/sendsTo, resolved from options.routes (the server-side worker has the route list even though it has no subscription payload).
  3. Send-side excludeTables: prefer the config route's sendsTo (was reading hdb_nodes only).
  4. Connection establishment (onNodeUpdate): the "should I open a socket" check listed sends/sendsTo/receivesFrom but not the receives boolean — so a receive-only edge never connected and upstream flow died. This is the load-bearing piece that makes a one-way edge work.

Purpose

Restore the documented controlled-flow behavior (reference/replication/overview.md → "Controlling Replication Flow"). Directed multi-tier topologies (edge → aggregation → core) currently leak reverse and lateral traffic.

Where to look

  • subscriptionManager.onNodeUpdate (the receives-boolean connection gate) and the configRouteReplicates vs routeReplicates distinction in knownNodes.shouldReplicateFromNode — the latter is the subtle correctness point (the two encodings are not interchangeable; mixing them would break add_node directional flow).
  • The send gates resolve the local route by route.name === remoteNodeName; this matches the existing routes.find(r => r.name === node.name) convention (node name = hostname).

Validation

  • New integration test directionalFlowReplication.test.mjs: directed edge→core — upstream flows, the core write never reaches the edge (reverse/lateral blocked). Passes.
  • excludeTablesReplication and replicationTopology (add_node, 9/9) pass — no regression.
  • 245 replication unit tests pass (incl. the new routeEntriesIncludePeer matcher test).

Notes

Generated by an LLM (Claude Opus 4.8).

🤖 Generated with Claude Code

@kriszyp

kriszyp commented Jun 29, 2026

Copy link
Copy Markdown
Member Author

Companion PRs: schema validation HarperFast/harper#1529, docs HarperFast/documentation#555.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request implements controlled-flow (directional) replication via config routes, ensuring that directionality gates (sends/receives) configured locally are respected and enforced on live connections. It introduces helper functions to check route authorization and updates both subscription and sending logic to prioritize local configuration over hdb_nodes. Feedback is provided regarding a potential runtime TypeError in the new routeEntriesIncludePeer helper if it encounters non-array or null/non-object values in the entries list.

Comment thread replication/knownNodes.ts Outdated
@claude

This comment has been minimized.

… live connections

Directional config (replication.routes[].replicates.sends/receives) was stored
and round-tripped but not honored at runtime: the direction gates read replicates
only from the hdb_nodes record (default true), so traffic flowed bidirectionally
regardless of the route config (harper-pro#498).

Honor the config route's direction in four gates, only for the directional object
form so boolean/subscriptions-only routes keep their existing behavior:
- receive (shouldReplicateFromNode): prefer route receives/receivesFrom, carried on
  a distinct configRouteReplicates payload field so it is never confused with an
  hdb_nodes sends/sendsTo object (which would regress the add_node path).
- send-authorization: honor the local route's sends/sendsTo (resolved from
  options.routes, which the server-side worker has).
- send-side excludeTables: prefer the config route's sendsTo.
- connection establishment (onNodeUpdate): the gate that decides whether to open a
  socket listed sends/sendsTo/receivesFrom but not the receives boolean, so a
  receive-only edge never connected and upstream flow died.

Adds an integration test (directed edge->core topology: upstream flows, reverse and
lateral flow blocked) and a unit test for the routeEntriesIncludePeer matcher.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@kriszyp kriszyp force-pushed the kris/498-controlled-flow branch from b9f7698 to a69b371 Compare June 29, 2026 23:12
@kriszyp kriszyp marked this pull request as ready for review June 29, 2026 23:19
@kriszyp kriszyp requested a review from a team as a code owner June 29, 2026 23:19
@kriszyp kriszyp merged commit 3af29c0 into main Jul 1, 2026
33 checks passed
@kriszyp kriszyp deleted the kris/498-controlled-flow branch July 1, 2026 14:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant