feat(store): tier-2 value ledger — encrypted-at-rest per-step inputs + monitor detail (M0) - #230
Merged
Conversation
…+ monitor detail (M0) Makes each governed tool-use inspectable and reproducible WITHOUT weakening the value-free wire/monitor (docs/pg-provenance-ledger.md, M0). - infra/store/valuecrypt.py: envelope encryption (X25519+HKDF+AES-GCM, no new dep). Fresh per-blob DEK wrapped to a recipient set (node standalone; +mothership at fleet join via rewrap without re-encrypting). values_sha commits the PLAINTEXT. - backend: run_values table (sqlite live; PG inert-parallel, raw BYTEA + JSONB) + record_values/get_values on both adapters; store_selftest interface-conformance covers them. - mirror: new value-free `values` queue op (single drain writer; backend only, never the chain) + values_sha added to _SAFE_EVENT_KEYS. - govd Store: node recipient key at boot (0600, default-on, inert if disabled); record_values encrypts + enqueues at the ONE WS point plaintext exists, binds values_sha into the tier-1 chain step event; decrypt_values for the operator view. New operator-gated endpoint GET /monitor/values/<run_id> (node-local decrypt). - monitors: local dashboard run tab + fleetdash run page each gain a "tool-use detail" reveal that decrypts step inputs on request (textContent-only, no inline-onclick sink); fleetdash _EVENT_KEYS + proxy allowlist gain values_sha / monitor/values. - tests: test_value_ledger.py (9) + 3 integrated delegated cases. Full suite green. Secrets never recorded (declared-subset, secret-filtered, *_FILE pointers stay node-local). Off-node at rest = ciphertext only. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…r-run, PG-safe read Four-lens adversarial review (crypto / value-free-boundary / web-surface / enforcement-path) found four real defects; this fixes all of them. - [crypto Medium] values_sha was an UNSALTED hash of low-entropy plaintext, living in the value-free chain/index — the same DB as the ciphertext — so a backup/replica/over-granted role could brute-force the values without touching the envelope (contradicting the module's own at-rest claim). Now SALTED: sha256(salt ‖ plaintext) with a fresh salt sealed INSIDE the AEAD blob; an authorized decryptor recovers salt+plaintext and re-verifies. Full-width keyid too (no rewrap truncation-collision/evict). - [enforcement #2/#3] record_values ran BEFORE execute_step with a first-wins upsert, so a refused/retried step orphaned a value row and a retry with corrected values desynced the chain sha from the stored blob. Now records ONLY on a terminal step_result (ok/error), at-most-once, so the stored blob and the chain-bound sha always agree; a refusal records nothing. - [enforcement #1] decrypt_values read the drain worker's shared (unguarded psycopg) connection — a monitor read racing a drain write could drop a tier-1 chain-index row. Now reads through a FRESH backend connection (reconciler discipline). - [key siting] the node key defaulted into the replicated record root (backup+key = broken at-rest); govd now warns loudly and the doc requires value_ledger.node_key_file off the backup path. Residuals documented (accepted): name-based secret filter, plaintext view behind the shared monitor token, values queue pressure. Tests updated for the salted commitment + a retry-desync regression. infra/ ruff clean; store selftest green. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Owner
Author
Adversarial review — 4 lenses, 4 real fixes (commit 24bba89)Ran a four-agent adversarial review (each prompted to refute): crypto correctness, value-free boundary, web/monitor surface, enforcement-path integrity. Four real defects surfaced and are now fixed; residuals documented. Fixed
Accepted residuals (documented, no code change)
Crypto and web lenses found no exploitable bug (all LOW/INFO residuals). |
…-monitor plaintext path
The fleet monitor reviews plaintext via the LIVE PROXY: fleetdash → /proxy/<node>/
monitor/values/<run_id> → the node decrypts with its own recipient key → plaintext
back. This adds the first over-HTTP test of that node endpoint (unit tests covered
the Store; this covers the wire):
(1) 403 without the monitor token (fail-closed gate),
(2) 200 with the decrypted per-step values when authed,
(3) the value-free /monitor/run for the SAME run carries only values_sha — no
plaintext leaks into the value-free view.
Confirms decryption stays node-side (no central key, no central ciphertext) — the
live-proxy oversight model.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
M0 of the two-tier provenance ledger (docs/pg-provenance-ledger.md). Makes each governed tool-use inspectable and reproducible — you can now review the actual parameters of a run in the fleet + local monitor — without weakening the value-free wire/monitor.
What lands
infra/store/valuecrypt.py— envelope encryption (X25519 + HKDF + AES-256-GCM, no new dependency). Fresh per-blob DEK wrapped to a recipient set (node standalone; +mothership at fleet join viarewrap, no re-encryption).values_shacommits the plaintext, never ciphertext.run_valuestable (sqlite live; PG inert-parallel with rawBYTEA+JSONB) +record_values/get_valueson both adapters;store_selftestinterface-conformance covers them.valuesqueue op (single drain writer; backend only, never the chain) +values_shaadded to_SAFE_EVENT_KEYS.Store— node recipient key at boot (0600, default-on, inert if disabled);record_valuesencrypts + enqueues at the one WS point where plaintext exists and bindsvalues_shainto the tier-1 chain step event;decrypt_valuesfor the operator view. New operator-gatedGET /monitor/values/<run_id>(node-local decrypt).textContent-only, no inline-onclick sink); fleetdash_EVENT_KEYS+ proxy allowlist gainvalues_sha/monitor/values.test_value_ledger.py(9) + 3 integrated delegated cases; full suite green locally (194 passed, 1 PG skip).Guarantees preserved
Secrets are never recorded (declared-subset, secret-filtered;
*_FILEpointers stay node-local). Off-node at rest = ciphertext only. The tier-1 chain and the:5773wire stay value-free — only thevalues_shahash crosses into them.Ouroboros self-inspection
Ran the transmutation mechanical pass (alembic putrefactio shape-extraction + LVP audit) and citrinitas intent-alignment on the new code. Intent-alignment on
encryptconfirms the blueprint matches the documented contract (empty-recipient guard gates before sealing; fresh DEK/nonce; per-recipient wrap). All mechanical leads adjudicated to false-positive or pre-existing (Rust.unwrap()heuristic on Python; bounded-loop misreads on the daemon drain) — no real defect in the new code.🤖 Generated with Claude Code