[Feature] athena advisory supervision - #20
Merged
Conversation
Define the provider-neutral contract for a one-shot advisory checkpoint before any agent or skill implements it: activation only on an explicit flag, three declared triggers firing at most once each, the input/output packet shape, the prohibitions, receipt and disposition mechanics, and the fallback sentence for runtimes that cannot delegate. Counsel is evidence, never gate authority and never verification. The contract states that in the Gate Authority Invariant's own terms, and the reviewer standing item now auto-triggers on the surfaces this lane introduces (rules-conditional, agents, skills/fix) instead of bypassing them. The rule is loaded on demand by flag-bearing skills, so it costs no always-on context. Registered in the harness inventory and entered in the dead-weight registry as WATCH pending measurement.
Author the advisory supervisor on the Claude plane and wire the first vertical slice. Athena is a one-shot fork: it reads the evidence a stuck run gathered, returns one counsel packet, and ends. It never interviews, never grades, and never approves a gate. The no-mutation rule is structural here, not just prose — the frontmatter grants no write capability. Classification matches advisor (internal/harness/non-public) so the generated agent-contract counts stay correct rather than misreporting an executor as public core-support. mk:fix gains the flag plus a checkpoint section in its standard and deep workflows only. Without the flag the section is inert and nothing loads. The existing three-failed-attempt human stop is explicitly unmoved by counsel taken at two.
…undle Mirror the advisory lane into the authored Codex bundle. The agent TOML is auto-discovered from .codex/agents/ with no config.toml registration, so none is invented. Delegation is natural-language here, so the description field carries the routing contract and the wrapper uses one canonical trigger sentence. Codex agent definitions expose no per-agent tool or permission field, so the no-write rule is BEHAVIORAL on this plane, not structural as it is on the other two. That asymmetry is stated in the agent's own instructions, in the wrapper, and in the compliance entry rather than papered over — the live mutation-refusal probe is what will actually evidence it.
… bundle Mirror the advisory lane into the authored Cursor bundle: a foreground, readonly subagent in the product pack, with the wrapper dispatching it and waiting for one packet. readonly: true makes the no-mutation rule structural on this plane — stricter than advisor by design. The surface matrix records delegation as undocumented for both ide-local and cli-headless: foreground delegate-and-wait has open reliability reports skewed toward the CLI, and nothing here has been run against a live binary. The wrapper prints the unavailable-checkpoint line and continues unsupervised wherever that holds. The two agent-roster counts move 40 to 41. Both guard on-disk == union-of-packs, and athena is on both sides, so the invariant they exist to protect is intact.
Evidence paths were stored on the task record but no read command surfaced them: the orient envelope is a compact projection that never carried them, and the context subcommands are resolve/check/record with no display path. A resuming session had to open the JSON by hand. Print them beside blockers and decisions. Display only — they are pointers a session can open, never a verification result.
Add the authored capability so the advisory lane is discoverable and each provider's support can be reported honestly instead of assumed. The support map ships EMPTY on purpose. Absence reads as unknown, and no runtime has been exercised: delegation resolves from the running session's own agent registry, so even the Claude gate needs a session rooted in an installed bundle. Structural discovery is not delegation, and the files existing is not evidence that dispatch works. The smoke checklists record what has actually passed offline and what each runtime must demonstrate before any provider is marked supported.
A new agent needs a reference page per the release checklist, and the roster surfaces need to name it. Adds the page, its nav entry and generated roster row, the conditional-rules table entry, and the `--advice` row in the fix skill's flag table. Both user-facing surfaces state plainly that delegation is unverified on every runtime, because it is: structural discovery passes but no live session has exercised it. A skill that cannot reach the agent prints its unavailable-checkpoint line and continues unsupervised. The architecture doc's agent roster is deliberately untouched — it enumerates lifecycle agents and already omits advisor, the precedent athena follows. No changelog entry: root CHANGELOG.md is a stub by policy and the published changelog is release-versioned, so the entry belongs to the release that ships this slice.
…they cite Owned artifacts are write targets, but they were derived by scanning an agent's body for backticked paths whenever it declared no ownership section. That scan cannot tell a citation from an ownership claim, so an agent that merely referenced a rule file was published as writing it. The generated reference pages showed the result: explore, granted only Read/Grep/Glob/Bash, was documented as writing agent-conduct.md and injection-rules.md. Twenty-two pages carried some version of this. Gate the list on write capability — an agent with no Write or Edit tool owns no write artifacts. Verified no true claim was lost: every corrected agent is read-only, and the agents that genuinely own artifacts keep them. Conformance checks are unaffected; they consult the list only for agents that can write.
The validate job aborted at its first failing step, hiding three more. None originate on this branch. - scripts/scan-skill-dependency-edges.mjs imports js-yaml, declared only in packages/mewkit. Under pnpm's isolated node_modules a root-level script cannot resolve it, so declare js-yaml at the root. The lockfile gains only a root importer entry; js-yaml@4.3.0 was already resolved in the graph, so no package is added and --frozen-lockfile stays clean. - mk:plan-creator carried two unclassified high-traffic edges. Both are body routing references, not runtime prerequisites, so they are peer per the scanner's own documented policy. - terminology-linter flagged retired terms in changelog/archive.mdx. That file is the release history split out of changelog.mdx, which the linter already exempts for historical accuracy; rewriting release notes to satisfy a linter would falsify them. - validate-rule-availability swept no .mdx at all while recursing into untracked build output, so its result depended on whether someone had run a build. It now sweeps .mdx (14 published docs pages become checked for the first time) and prunes test trees, release history and generated output. No checker is weakened or skipped: every exclusion targets a false-positive class (gitignored build caches, synthetic test fixtures, deliberately unresolvable historical paths), and net rule-reference coverage rises.
Three pure modules make the supervisor contract mechanical rather than prose-only, so the deterministic checks — not the documentation — are authoritative. - athena-supervision-protocol: stages, stage-legal dispositions, per-stage and per-skill caps, checkpoint idempotency, and transport-status/disposition precedence. GATE_ADVANCING_DISPOSITIONS is an empty array with a test asserting it stays empty, so a disposition intended to advance a gate cannot be added without deleting the test that says why it may not. - athena-supervision-packet: packet schemas with a 12 KiB inbound cap, a 600-word outbound cap, a five-pointer budget, mandatory provenance, secret and bulk-content scanners, and refusal of approval language. Oversize fails before delegation rather than truncating, because a truncated packet asks a different question than the one intended. - athena-supervision-dossier: continuity that cannot become a second task truth. Progress, verification, gate, approval, verdict and status fields are refused rather than stripped, since silently dropping a field the caller believed it stored is worse than refusing it. Secret scanning has no field exemption and covers evidence-pointer paths and returned prose. Only bulk-content scanning skips paths, so a filename such as fix-diff.md stays legal while a credential cannot ride through an unscanned field. Pointer paths are shape-constrained and the input schema is strict, because an unknown key is both unscanned and makes the byte cap a lie. check-gate-authority gains the supervision vocabulary and now scopes matching and negation per clause. Previously one wide match could span a contrastive boundary and pull an earlier "never" into the match text, disarming a violation that lived entirely in the later clause — the exact failure mode the module's own comment warns against.
Replaces the one-shot, mk:fix-only counsel contract with a supervisor that has a stateless direct strategy consult and a bounded --advice lifecycle route. Contract (rules-conditional/advice-supervision-rules.md, rewritten): the GUIDE -> RESCUE* -> REVIEW -> RECHECK* cadence at named macro checkpoints, packet schemas and caps, stage-legal dispositions, the parent-owned continuity dossier, and the bounded correction cycle. Checkpoints are macro boundaries, never per tool call, file or loop iteration, and a second unresolved return escalates to a human rather than seeking a third opinion. Adapter (agents/athena.md, rewritten): tools narrowed to Read, Grep and Glob, so the mutation ban is structural rather than only behavioural. Bash, web and nested-agent access are gone because every checkpoint works through explicit evidence pointers. Classification: athena becomes intelligence/direct-and-harness/public, since a direct strategy consult and embedded supervision are distinct capabilities that must not impersonate each other. The registry view gains that pairing and the capability entry gains intents so the lane is discoverable. Authority is unchanged in every wording: a recommendation is evidence a human reads at a gate, never the approval, and never verification. READY_FOR_EXISTING_GATE names the next step rather than clearing the gate. A recommendation selects an operational path inside the packet's locked scope; business, security, compliance and gate decisions escalate instead. Provider support stays empty because no live delegation has been proven on any runtime, so absence reads as unknown rather than as a claim. The mk:fix wrapper still describes the retired three-trigger cadence; the contract's wiring-status section names that explicitly so no surface silently claims both contracts. Generated docs, manifest and registries regenerated from source.
…rect vs embedded calls
…d record management
…skills with added strategy brief support and validation tools
Both skills already sat in SKILL_HARD_CAPS, so `mewkit advice begin --skill mk:ship` succeeded against a skill with no flag and no documented boundaries. This closes that gap and makes ship's cap match what it always claimed. - PARTITIONED_SKILL_CAPS declares mk:ship as prepare|release|publish, each with its own 4-call budget. --release-stage is fixed at `begin` and inherited by `commit`; required for ship, refused elsewhere, never defaulted. - History that names no partition, or one the skill does not declare, counts against every partition. Skipping it would hand a run that already spent a flat cap a fresh budget per stage and drop its escalation counter. - mk:autobuild: GUIDE at the Plan/Contract boundary, RESCUE on plateau only, REVIEW before the step-05 Gate 2 question. A returned correction consumes an --max-iter round; the iteration cap, budget tracker and timeout are untouched. - mk:ship: counsel never creates push/PR/merge/deploy authority; REVIEW waits for terminally green CI; red and pending keep their repair-or-stop route. - workflow-orchestrator forwards an opaque supervisionRunId and never enables the flag. Nine surfaces refused as entry points at the protocol and the CLI.
A correction spreads whatever JSON object it reads and writes it back, and --evidence accepted any JSON file inside the project. Pointed at a Gate-1 approved visual-plan/plan.json -- which carries its own top-level review.status -- it flipped that status to "superseded", injected four fields the visual schema does not define, and broke the artifact against its own pinned hash. Supervision must not write the HTML pipeline's artifact; the two flags are supposed to be orthogonal. - --evidence must name an existing workflow-evidence.json inside the project. The index has one filename under all four roots it is written to, so the name is the check; verified against every caller before choosing it. - Both the boundary and the name check run on the resolved path. A textual check answers a question about the caller's string, not the file that gets opened, and reads follow symlinks: a link named workflow-evidence.json inside the repo pointing outside it passed both checks and its contents were written back in. The project root is resolved too -- on macOS /var is itself a symlink, so a real-vs-textual comparison refuses a project under it. - Document --html/--advice orthogonality: the Markdown plan stays source of truth, REVIEW lands before the visual approval, and a returned correction goes back through rehash/re-approve rather than reaching Gate 1 on a stale artifact. Also corrects a test that asserted the escape refusal with a path that was both outside the project and absent, so the boundary check it named never ran.
…planes Both bundles still shipped the superseded fix-only contract: one packet, one turn, no second round, the retired proceed/pause/escalate shape, and mk-fix as the only caller. Neither had a home for the canonical rule — codex and cursor project no rules-conditional, and both wrappers pointed at a file that does not exist in an installed project. Project the contract natively per provider: cursor gets an Agent-Requested .mdc, the surface AGENTS.md already routes situational guidance to; codex gets a rule- skill projection registered in the core pack, because its rules dir is exec-policy and AGENTS.md is auto-loaded every session. Rewrite both adapters to the four stages, the stage-legal dispositions, the correction cycle and the direct-consult brief. Structural safety stays per-plane rather than equalized: cursor keeps readonly: true, codex keeps its disclosure that no per-agent tool field exists there, and a test forbids codex from claiming otherwise. Support states are per-capability — embedded supervision and direct consult are separate, and structural evidence can never claim supported. The flag stays off on both bundles; wrapper wiring is a separate change.
…anes Port the supervision wiring to both bundles: brainstorming (deep only), plan-creator, cook, fix, autobuild and ship each declare the flag, name their checkpoints and cite their own plane's contract. The orchestrator stays out — no flag, and it carries only an opaque supervisionRunId across macro boundaries. The call protocol is identical at every checkpoint of every skill, so it lives once in each bundle's contract rather than twelve times in the wrappers. Each wrapper then declares only where its checkpoints fire, what each one asks, and what supervision may not touch there — ship's per-release-stage budget and its counsel-is-not-authorization boundary, autobuild's plateau definition and its untouched iteration cap and evaluator verdict, plan-creator never writing the plan, brainstorming never generating the option set. mk-fix loses its not-authored-for-this-bundle block on both planes. Embedded supervision moves from unavailable to unverified: the wiring exists, but no live session has exercised delegation on either runtime, and structural evidence may not claim support.
`mewkit providers <provider>` now prints the two --advice capabilities separately with their declared state, evidence kind and what would move them up a rung. Embedded supervision and a direct consult are independent — one runtime proving out its consult says nothing about supervision it has never run — so collapsing them into a single line was the one thing the report could not do. A provider with no authored bundle still gets a report rather than silence: absence reads as "fine", and an unexercised runtime has to say so. Codex shows its mutation ban as behavioral/degraded, because it has no per-agent tool field to enforce one. Add the deterministic gate suite: the support report's invariant (structural evidence never yields a supported claim, on any provider), and a coverage oracle mapping each declared gate group to the file and title that own its assertions. The oracle proves the assertions exist, not that they pass — that is the failure a green suite hides best. The live lane stays unrun and unclaimed: every reported capability carries structural evidence, and a test enforces that.
…omponents The agent reference described the contract but not where it actually runs. Add a provider-support section: the three authored planes, the two capabilities tracked separately, how to read the current state, and that every one of them is unverified today because the wiring existing is not the same as a runtime having done it. Register the provider adapters/projections and the support report in the dead-weight audit registry, per its own same-PR rule.
… --advice flag to core skills
Kit 2.14.8 -> 2.15.0 (minor: new agent, new flag on six skills). The CLI moves to 2.3.0 rather than a patch: this release adds the `mewkit advice` command surface, and RELEASING.md step 3 maps a new subcommand to a minor bump. A consumer pinning ~2.2.x would otherwise miss a release that adds commands. Add the changelog section, the release-history row, and the athena rows in all three agent-model guides. Document `--advice` on the five skill reference pages that were missing it, and correct the fix page, which still described the superseded fix-only contract — one counsel packet at three checkpoints, rather than the four-stage supervision that shipped. The changelog states plainly that no runtime is verified end to end yet, so the release notes cannot be read as a support claim.
The generated flag table already listed `mewkit advice`, but the family pages carry the worked examples and the failure recovery, and it had none. Add it to the planning page: what each subcommand enforces, why ship needs --release-stage and nobody else may pass it, what source vs scope corrections invalidate, and when the command refuses. Document the supervision section `mewkit providers` now prints, including why the harness and the direct consult are tracked as separate capabilities and why structural evidence never reads as supported. Drop the em dash from the --reason summary in the CLI catalogue and regenerate the table. cli/index.mdx is on the prose linter's hard tier, where an em dash is an error, so the generated row was failing a gate the source could fix.
…nd development and devops skill modules
Fold the portable backend skill surface into the 2.15.0 release and bump the CLI for the reconciler change that ships with it. Register the api-design skill rename in portable-manifest.json. Without the entry, migrate leaves a stale skills/api-design directory in every upgraded project. An entry only applies when its `since` is at or below mewkitVersion, so mewkitVersion moves to 2.3.1 as well; leaving it at 1.14.0 would ship an entry that silently never runs. Replace the manifest test's pinned `mewkitVersion === one entry's since` assertion with an all-entries applicability check. The equality only held while the manifest carried a single entry; the loader's real invariant is that every entry is applicable at the manifest version. Note the tree-checksum caveat in retired-skill-cleanup: a symlink nested inside a candidate is outside the hash, so a match proves tracked file content is unchanged rather than a byte-identical directory. Not a deletion hazard — the recursive remove unlinks a nested symlink rather than following it.
Whitespace only, no semantic change — the test count is unchanged across the reformat. Fifteen of the sixteen files predate this branch's release work and were already failing `format:check`; the sixteenth is the retired-skill cleanup comment added earlier on this branch.
Root package.json is the source of truth and moved to 2.15.0 with the athena work, but the derived surfaces were never stamped — so the mismatch predates the CLI bump, which only added the cliVersion line to it. Stamps .meowkit/config.json to 2.15.0 via scripts/sync-versions.cjs. The release manifest carries the same stamp but is an untracked release-time artifact, regenerated at tag time.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Type
mk:*)Changes
Testing
npm test)npm run lint)npm run build)npm run build -w packages/docs)Checklist
mk:prefix for skills).envvaluesRelated Issues