Skip to content

fix: give ensureNode() its own isolated write context#550

Merged
kriszyp merged 1 commit into
mainfrom
fix/ensurenode-ambient-context
Jul 9, 2026
Merged

fix: give ensureNode() its own isolated write context#550
kriszyp merged 1 commit into
mainfrom
fix/ensurenode-ambient-context

Conversation

@kriszyp

@kriszyp kriszyp commented Jul 8, 2026

Copy link
Copy Markdown
Member

Context

Companion to HarperFast/harper#1720, which is the primary/sufficient fix for a cluster-formation regression: bumping the core submodule pointer past af646222d ("establish ambient user context for operation handlers so audit records carry attribution", #1591/#1592) caused integrationTests/cluster/replicationTopology.test.mjs to fail with Timed out waiting for cluster to connect / SELF_SIGNED_CERT_IN_CHAIN — see harper-pro PR #512's CI.

Root cause (full detail in the harper#1720 description): processLocalTransaction now wraps an entire operation handler invocation in one contextStorage.run({ user: hdbUser }, handler) call, installing a single, long-lived, mutable ambient context for the whole handler. resources/Resource.ts's dispatcher decided whether to join an existing transaction on that ambient context via blunt truthiness, with no check that the transaction was still meaningfully open — so two sequential, logically-independent static Resource API writes in one handler could get coalesced into one transaction, silently dropping one of them. ensureNode() (called twice per add_node/add_node_back invocation — once for this node's own hdb_nodes record, once for the peer's) hit this: a peer's ca field could silently fail to persist, so the peer's cert was never trusted, producing SELF_SIGNED_CERT_IN_CHAIN.

This change

ensureNode()'s writes now use an explicit, isolated {} context instead of relying on ambient contextStorage fallback. This is defense in depth on top of the core fix, and also fixes a smaller, independent issue: ensureNode() runs for both genuine admin-initiated add_node calls and internal inter-node bookkeeping (addNodeBack, whose ambient "hdb_user" is just the resolved auth principal of the replication socket connection) — neither should be attributed to a system table row as if a user directly wrote it (core's own registerLiveSubscriptionForContext documents the same invariant: "Internal watchers, replication and local-bypass have no user principal").

This fix is not required for the connectivity regression once harper#1720 lands (that core fix alone resolved the full cluster-formation test, 3/3 clean runs at pre-regression speed, verified with this harper-pro change reverted). It's included as a belt-and-suspenders correctness improvement that's safe to take independently.

Testing

Verified via an instrumented 4-node cluster-formation run (integrationTests/cluster/replicationTopology.test.mjs, "connect nodes") with core pinned at af646222d:

  • With only this harper-pro change (no core fix): the hdb_nodes CA-persistence symptom is resolved (confirmed via direct search_by_value dumps — ca_info correctly populated for every peer on every node), but the cluster-connect test can still fail because the same ambient-context hazard also affects other call sites in the same operation handlers (e.g. security/certificate.ts's signCertificate(), which does a for await static-API search before later static-API writes) — this is exactly why the general core-level fix in harper#1720 is the one that should ship.
  • With harper#1720 alone (this change reverted): passes consistently, 3/3 runs, ~3.2s each.
  • With both changes together: passes consistently, 3/3 runs, ~3.2s each.

Co-Authored-By: Claude Sonnet 5 noreply@anthropic.com

ensureNode() (called from setNode()/addNodeBack() to persist a peer's or
this node's own hdb_nodes row during cluster mesh formation) wrote through
the static Resource API with no explicit context, relying on ambient
fallback (contextStorage.getStore()). Two problems follow from that:

1. Attribution: ensureNode() runs both for genuine admin-initiated
   add_node calls and for internal inter-node bookkeeping (addNodeBack,
   whose "hdb_user" is just the resolved auth principal of the replication
   socket connection) — neither should be stamped onto a system table row
   as if a user directly wrote it.

2. Correctness (now fixed generally at the core level by
   HarperFast/harper#1720, but worth hardening here too): core's
   processLocalTransaction (#1591/#1592) wraps an entire operation handler
   invocation in one contextStorage.run(...) call, installing a single,
   long-lived, mutable ambient context for the whole handler.
   setNode()/addNodeBack() each call ensureNode() twice in sequence (once
   for this node's own record, once for the peer's) — before harper#1720,
   the second call could see the first call's already-completed
   transaction still attached to the shared context and wrongly join it,
   silently dropping one of the two writes. Confirmed via an instrumented
   4-node cluster-formation run (integrationTests/cluster/
   replicationTopology.test.mjs) against core commit af646222d: a peer's
   `ca` field failed to persist into hdb_nodes, producing
   SELF_SIGNED_CERT_IN_CHAIN on subsequent mesh TLS connections.

Passing a fresh `{}` context per write restores the pre-#1591 behavior
(every internal static Resource API call here always got its own throwaway
context) and fixes both the attribution issue and the correctness issue as
defense in depth, independent of the core-level fix.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@kriszyp kriszyp requested a review from a team as a code owner July 8, 2026 20:33

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces an explicit, isolated write context (writeContext) in ensureNode within replication/subscriptionManager.ts. This prevents internal node-registry bookkeeping writes from inheriting ambient user contexts or incorrectly joining already-completed transactions from shared contexts, addressing potential correctness and attribution issues. There are no review comments, and I have no additional feedback to provide.

@claude

claude Bot commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

Reviewed; no blockers found.

@kriszyp kriszyp requested review from DavidCockerill and cb1kenobi and removed request for a team July 8, 2026 22:35
@kriszyp kriszyp merged commit 896fc8f into main Jul 9, 2026
45 of 46 checks passed
@kriszyp kriszyp deleted the fix/ensurenode-ambient-context branch July 9, 2026 23:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants