Skip to content

Support replicated: true on set_configuration#1556

Merged
kriszyp merged 4 commits into
mainfrom
set-configuration-replicated
Jul 2, 2026
Merged

Support replicated: true on set_configuration#1556
kriszyp merged 4 commits into
mainfrom
set-configuration-replicated

Conversation

@heskew

@heskew heskew commented Jul 2, 2026

Copy link
Copy Markdown
Member

Closes #660 (CORE-2939).

What

set_configuration now accepts replicated: true and fans the same config change out to all cluster nodes in one Ops API call, mirroring the drop_schema pattern (server.replication.replicateOperation). Driven by a customer updating corsAccessList per-node from automation pipelines.

  • Origin applies the change locally first; if the local write fails, nothing is fanned out.
  • Peers receive the forwarded body with replicated: false, so they apply locally without re-replicating.
  • Per-node outcomes come back on response.replicated[] (peer failures are embedded as {status: 'failed', reason, node} — the fan-out itself never rejects).
  • replicated is added to the handler's destructure strip-list so it is never written to harperdb-config.yaml (on origin or peers).
  • Without harper-pro replication loaded, replicated: true rejects with Replication not implemented. (existing base-server stub behavior).

Integration tests (apiTests/configuration.test.mjs) pin the single-node contract: replicated: true without clustering rejects explicitly, and the flag is never persisted to config. Real cluster fan-out verification belongs to harper-pro's cluster harness (companion candidate). 5 new unit tests cover: fan-out + per-node results, flag stripping, no-flag no-op, peer-side replicated: false, and local-failure-before-fan-out. DESIGN.md gets a note on the non-obvious replicateOperation contract (see below).

Where to look

  • The opt-in guard is load-bearing. replicateOperation fans out whenever req.replicated !== false — absence means "replicate" (the DDL contract drop_schema relies on). Calling it unconditionally would have silently made every set_configuration replicate by default. Hence if (replicated) and the DESIGN.md note.
  • No per-param node-local/cluster-wide guard — deliberate. The operator owns field selection, same contract as every replicated op today. A caller who replicates node-local params (ports, paths, node identity, TLS paths) will clobber peers; docs list these footguns. Per-field replicability metadata is intentionally deferred to the cluster-level-config work (CORE-3018), which will own that schema — building a second scope registry here would compete with it.
  • Restart intentionally not coupled. set_configuration {replicated: true} + restart_service {service: "http", replicated: true} compose as two calls (the latter already restarts nodes sequentially — effectively rolling). Documented as the workflow; not verified in a live multi-node cluster in this PR.
  • Partial peer failure still reads as success in message — identical to drop_schema's existing contract; operators must check response.replicated[]. Flagged by review, kept pattern-consistent; docs point at the array. Open to making it louder if you'd rather.

Review/verification notes

  • Cross-model review (Codex + Harper-domain adjudication): no blockers, no significant concerns. Caveat: the Gemini leg failed (agy timeouts), so no Gemini coverage.
  • Local: all 52 configUtils unit tests pass. The broader unit:main/unit:resources suites carry pre-existing environment failures on this machine (present without this change, none config-related) — CI is the clean-room run for those.
  • Docs companion PR: Document replicated: true on set_configuration documentation#559.

Follow-up candidates (out of scope): the Ops API replicated-support audit floated in #660, and per-field replicability metadata under CORE-3018.

Generated by an LLM (Claude Fable 5), reviewed per the cross-model flow above.

🤖 Generated with Claude Code

heskew and others added 2 commits July 1, 2026 15:45
Opt-in cluster-wide fan-out via server.replication.replicateOperation,
mirroring drop_schema. Peers receive replicated: false and apply locally;
per-node outcomes are returned on response.replicated. The replicated flag
is stripped from configFields so it is never written to the config file.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…peration default-on

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@heskew heskew requested a review from kriszyp July 2, 2026 01:38

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces opt-in replication for the set_configuration operation. When the replicated flag is provided, the configuration changes are fanned out to all cluster nodes via server.replication.replicateOperation. The replicated parameter is stripped from the configuration fields to prevent it from being written to the configuration file. Unit tests and design documentation have been added to cover this new behavior. Feedback suggests avoiding direct mutation of the response object returned by replicateOperation to prevent potential runtime errors if the object is frozen.

Comment thread config/configUtils.ts
@claude

claude Bot commented Jul 2, 2026

Copy link
Copy Markdown
Contributor

Reviewed; no blockers found.

@heskew heskew marked this pull request as ready for review July 2, 2026 02:08
…lag (#660)

Non-clustered instance: replicated: true rejects explicitly (base replication
stub), the flag is never persisted to config, and the origin-first local write
still applies. Cluster fan-out coverage belongs to harper-pro's cluster harness.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

@heskew heskew left a comment

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I found one issue worth fixing before merge:

config/configUtils.ts:836 gates fan-out on if (replicated), but set_configuration does not validate this operation-control field before applying the local write. A request with replicated: "false" still enters this branch, so open-source Harper applies the config change locally and then rejects with Replication not implemented; with a real replication implementation it would fan out unexpectedly. Other replicated surfaces validate this as a boolean, so this should reject non-booleans before updateConfigValue(...) or only fan out on replicated === true, with a test for the string "false" case.


🤖 Posted by Codex (gpt-5.5) on Nathan's behalf

…e local write (#660)

A string "false" is truthy — it would apply config locally and then fan out
(or reject on OSS) unexpectedly. Enforce boolean per other replicated surfaces
(analyticsValidator) before updateConfigValue so malformed values have no side
effects. Unit + integration coverage for the string case.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@heskew

heskew commented Jul 2, 2026

Copy link
Copy Markdown
Member Author

Addressed in 76d953c: replicated is now enforced as a boolean before updateConfigValue runs, so a malformed value (e.g. the string "false") is rejected with 400 and has no side effects — no local write, no fan-out. Matches the boolean-strict treatment on other replicated surfaces (analyticsValidator). Coverage: unit test sweeping 'false'/'true'/1/0 asserting neither updateConfigValue nor replicateOperation is called, plus an end-to-end integration case asserting the 400 and that the config value did not change. (🤖 Claude, on behalf of Nathan)

@kriszyp kriszyp left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving — correct opt-in config fan-out mirroring drop_schema: origin-first, loop-prevented (peers get replicated:false), super_user-gated + re-validated on peers, fails closed on bad input. The node-local-key guard is tracked as CORE-3018 (so a blanket replicate of ports/paths stays a known follow-up). Thanks @heskew!

— 🤖 KrAIs (Kris's review assistant)

@kriszyp kriszyp merged commit 44a8bd2 into main Jul 2, 2026
47 checks passed
@kriszyp kriszyp deleted the set-configuration-replicated branch July 2, 2026 13:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support replicated: true on set_configuration

2 participants