feat(governance): add disk->index coverage gate so skills cannot silently become unroutable (M4) - #900
Merged
Merged
Conversation
…ntly become unroutable validate-index-integrity.py check 1 walks index -> disk and catches entries whose file vanished. Nothing walked the other way, so a SKILL.md that exists, parses, and is healthy but was never registered stayed invisible to the router with no signal anywhere. Check 3 closes that direction: every SKILL.md under skills/*/*/ must have an INDEX entry. Skills carrying promoted_to are exempt, matching the generator, which skips folded husks on purpose so a husk cannot re-enter routing and shadow the umbrella that replaced it. The validator already runs in CI after index generation (test.yml:22,29), so this needs no new workflow step. The error names the fix, not just the fault. Audit result behind this change: the current tree has zero coverage gaps and zero phantoms once both indexes are regenerated. All 20 on-disk skills absent from INDEX.json carry promoted_to and are excluded by design; all 11 INDEX.local.json entries that look like dead paths are private skills whose repo-relative path resolves under the deployed root. The defect was that nothing enforced this, not that it was currently broken.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Audit first, then the fix
The brief expected ~40 unroutable skills and 11 phantom index entries. Measured against the tree, neither holds:
INDEX.json, and every one carriespromoted_to— the generator skips folded husks on purpose (generate-skill-index.py:407-409) so a husk cannot shadow its umbrella.skills/meta/dounroutabledohaspromoted_to: native-router. Same intentional exclusion. Same forworktree-agent(->do),roast(->multi-persona-critique),cve-source-check(->service-health-check).filefield is deploy-relative, and all 11 resolve at~/.claude/skills/<name>/SKILL.md.INDEX.local.jsonmirrors the deployed layout by design.After regenerating both indexes: 98 active skills, 98 indexed, 0 gaps, 0 unresolvable paths. Both indexes are gitignored generated artifacts, so the regeneration itself is local and not part of this diff.
So the real defect was not that coverage was broken — it was that nothing enforced coverage, so it could rot silently at any time.
The gate
validate-index-integrity.pycheck 1 walks index -> disk. Nothing walked disk -> index. New check 3 does: everyskills/*/*/SKILL.mdmust have an INDEX entry, withpromoted_tohusks exempt so the check agrees with the generator rather than demanding entries it will never emit.The validator already runs in CI after index generation (
test.yml:22,29) — no new workflow step needed.Verified in all three directions:
Tests
scripts/tests/test_validate_index_coverage.py— 6 cases: indexed skill passes, unindexed errors, promoted husk exempt, all gaps reported at once, missingskills/does not crash, and a live-tree assertion that current coverage is complete.Gates
ruff check/ruff format --check(0.15.12) — passpytest scripts/tests/— 4643 passed, 1 skipped, 138 xfailedvalidate-skill-frontmatter.py— 119 OKvalidate-index-integrity.py— 0 errors, PASSvalidate-doc-counts.py— all claims agree