feat(fleetdash): approve button + mutation-verified E2E for answering a govd push_back - #233
Merged
Conversation
…_back The fleet monitor showed the approval queue and could not answer it: `needs_approve` was rendered as a value-free token list, and the only way to act was to re-submit the claim by hand. Meanwhile the agent-side human gate is unreachable in a caged deployment — every caller of resolve_gateway_approval() is a chat-platform adapter (Discord/Telegram/Feishu/Teams) or a TTY, and a caged agent can reach neither. Measured across the fleet: 45 push_backs on maria-dev-mac, 9 approvals, ALL from a test harness. The human gate has never once been answered by a person. This dashboard has NO app-auth and its monitor tokens are read-only by contract, so the write is gated deliberately, every branch fail-closed: - a SEPARATE credential — `approve_token_file` per node (or GOVD_APPROVE_TOKEN_<NAME>), a principal token the operator provisions on purpose. The monitor token is never reused. Absent (the default) => no button is rendered AND the route 403s, so the read-only posture is unchanged; - CSRF — requires X-Fleetdash-Approve, a custom header a cross-origin page cannot set without a preflight this server never answers, so a hostile site cannot drive-by POST at 127.0.0.1; - never approve blind — the run must exist in the mirror and actually be a push_back, and the claim's own skill/perk/var_keys are REPLAYED from the mirrored record so the UI cannot widen a claim into something the agent never made; - --no-mirror refuses outright (no record to verify against), fixing a crash where mirror_dir was None; - the loopback-only default and its FLEETDASH_ALLOW_OPEN acknowledgement are untouched. HONEST LIMITS. This authorizes the CLAIM, not the tool call: the agent's gate fails closed in ~0.3s and is not resumed by a later approval, so pressing approve creates the audit record and the `superseded` marker without delivering the action. And the monitor token is one shared secret per node, not a person, so the chain still cannot name WHO approved. Both wait on per-person principals + an acl.approve axis; this is a stopgap that makes the ask answerable at all. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
… testing docs The render-level checks prove the button APPEARS under the right conditions; they cannot prove the integrated path works, because the interesting behaviour is in the browser — the confirm() gate, the custom CSRF header the fetch must set, and the claim the node actually receives. This drives real Chromium against a real fleetdash and asserts on what the STUB NODE was handed, not on what the page says. Hermetic: own fleetdash on a free loopback port, a stub govd that records the claim it is given, and an on-disk mirror fixture in tmp_path. Never the live fleet; no credential outside the temp dir. Every guard is mutation-verified — each was individually disabled and the suite had to go red: CSRF header check ....... caught (test_csrf_header_required) push_back guard ......... caught (test_cannot_approve_a_non_pushback) run_id regex ............ caught (test_path_traversal_run_id_refused) button render gate ...... caught (test_no_button_without_operator_credential) credential gate ......... caught (test_credential_gate_is_enforced_SERVER_side) Two defects only mutation exposed, both of which would have shipped a green-but-hollow suite: 1. test_cannot_approve_a_non_pushback passed for the WRONG REASON — the fixture's allow-run had an empty needs_approve, so deleting the push_back guard still produced a 409 from the next check down. The fixture now carries approve tokens, making the guard the only thing that can refuse it. (Its perk also had to differ from the push_back's: the same skill/perk tuple makes mark_superseded hide the push_back's button — correct behaviour that would have silently gutted the happy-path test.) 2. There was NO server-side coverage of the credential gate — only "the button is not rendered". Withholding a button is cosmetic; anyone can POST /approve directly. That check holds the entire read-only posture and was untested. README documents the setup, because two INDEPENDENT steps are needed (`pip install playwright`, then `playwright install chromium`) and the failure when only the first has run — "Executable doesn't exist" — does not say so. Records the shared browser cache paths (outside the venv), the env overrides, that the suite importorskips (so `pytest tests` and CI are unaffected — these currently SKIP in CI), and the mutation discipline plus its two traps: a str.replace(pattern, repl, 1) mutation can patch an EARLIER identical line and wrongly report a test gap (mutate by line number, assert the line's content), and a fixture change can neuter an unrelated test (re-run the whole file). Both bit during this work. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The fleet monitor showed the approval queue and could not answer it:
needs_approverendered as a value-free token list, and the only way to act was to re-submit the claim by hand.Meanwhile the agent-side human gate is unreachable in a caged deployment — every caller of
resolve_gateway_approval()is a chat-platform adapter (Discord/Telegram/Feishu/Teams) or a TTY, and a caged agent can reach neither. Measured across the fleet: 45 push_backs on maria-dev-mac, 9 approvals, all from a test harness. The human gate has never once been answered by a person.The write is gated deliberately — every branch fail-closed
This dashboard has no app-auth and its monitor tokens are read-only by contract, so:
approve_token_fileper node (orGOVD_APPROVE_TOKEN_<NAME>), a principal token the operator provisions on purpose. The monitor token is never reused. Absent (the default) ⇒ no button rendered and the route 403s, so the read-only posture is unchangedX-Fleetdash-Approve, a custom header a cross-origin page cannot set without a preflight this server never answerspush_back; the claim's ownskill/perk/var_keysare replayed from the mirrored record, so the UI cannot widen a claim into something the agent never made--no-mirrorrefuses outright (no record to verify against), fixing a crash wheremirror_dirwasNoneFLEETDASH_ALLOW_OPENacknowledgement are untouchedHonest limits
This authorizes the claim, not the tool call: the agent's gate fails closed in ~0.3s and is not resumed by a later approval, so pressing approve creates the audit record and the
supersededmarker without delivering the action. And the monitor token is one shared secret per node, not a person, so the chain still cannot name who approved.Both wait on per-person principals + an
acl.approveaxis. This is a stopgap that makes the ask answerable at all.Tests — mutation-verified
Real Chromium against a real fleetdash, asserting on what the stub node was handed, not on what the page says. Hermetic: own fleetdash on a free loopback port, stub govd, mirror fixture in
tmp_path.Each guard was individually disabled and the suite had to go red:
test_csrf_header_requiredtest_cannot_approve_a_non_pushbacktest_path_traversal_run_id_refusedtest_no_button_without_operator_credentialtest_credential_gate_is_enforced_SERVER_sideTwo defects only mutation exposed, both of which would have shipped a green-but-hollow suite:
test_cannot_approve_a_non_pushbackpassed for the wrong reason — the fixture's allow-run had an emptyneeds_approve, so deleting the push_back guard still produced a 409 from the next check down./approvedirectly. That check holds the entire read-only posture and was untested.README documents the Playwright setup: the two independent install steps, the shared browser cache outside the venv, that the suite
importorskips (sopytest testsand CI are unaffected — these currently skip in CI), and the mutation discipline plus its traps.Deploy
Host-side (launchd) — no image rebuild, just
launchctl kickstart -k gui/$(id -u)/com.cyberware.fleetdash. The button additionally needsapprove_token_fileinfleet.json.🤖 Generated with Claude Code