[US-352] chore: extend mirror-equality guard to per-skill SKILL.md pairs#377
[US-352] chore: extend mirror-equality guard to per-skill SKILL.md pairs#377rucka wants to merge 6 commits into
Conversation
Data-driven it.each guard asserting every root .claude/skills/<prefixed>/SKILL.md is byte-for-byte the real 'pair update' copy-pipeline transform of its dataset source. Composes the FULL pipeline (copyDirectoryWithTransforms, flatten+prefix 'pair') over InMemoryFileSystemService — no parallel transform re-implementation, so a real pipeline bug fails the guard. - Directional (dataset -> root): iterates 39 dataset skill dirs; root-only agent-browser (no dataset source) is ignored, not drift. - Covers all 4 drift classes via drift-injection over a synthetic mini dataset: frontmatter name: sync, relative-link-depth bump (../../ -> ../../../), /command skill-reference rewrite, and missing-mirror hard fail. - No hardcoded skill count (AC5): new dataset skill covered with no test edit. - Gate/assertion helper (assertRootSkillMdMatches) in a tested production module per the gate-tooling-in-tested-modules ADL. Refs: #352 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Code Review — PR #377Review InformationPR Number: #377 Review SummaryOverall Assessment
Key Changes SummaryTwo new, additive files under
Business Value ValidationCloses the structural gap that PR #350 patched by hand: per-skill Verification Performed (independent)Ran in a detached worktree pinned to the PR head ( The AC → verified
Detailed Review CommentsPositive Feedback
Minor Issues 💡
Questions ❓
Nit
Security ReviewTest-only guard: reads local files, runs an in-memory FS transform. No authn/authz, secrets, PII, or network. Verdict: green — no findings. Testing ReviewAdequate and well-structured. Consolidated Risk AssessmentClassification carried from refinement — risk:yellow · cost:green — confirmed, not raised: test-only, isolated, no production behavior change; security green; no new dependency (reuses adopted Adoption Compliance
Tech DebtThis PR reduces debt (automates the manual #350 check). No new debt introduced. One adjacent coverage gap tracked as #379 (auxiliary skill files). Definition of DoneAll DoD items met: ACs implemented & verified; consolidated data-driven Decision: APPROVED WITH COMMENTS. No critical or major issues; all AC met; all gates pass. The minor items (MIN-1/MIN-2/N-1) are optional polish; MIN-3 is deferred to #379. Safe to merge at the human gate. |
…t message - seed virtual dataset at deep source path (packages/knowledge-hub/dataset/.skills) so sourceContentRoot is non-trivial and the link-rewriter reRootTarget branch runs - strengthen frontmatter-drift test to assert error names the skill + shows expected-vs-actual (AC2), so a regression dropping either fails the test Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Addresses review: prettier:check now clean on skill-md-mirror.ts. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Review <-> fix loop status — escalating to human reviewSummary of the review/fix loop so far (full detail in Round 1 (commit
Round 2 (commit
Round 3 (re-review, no new commit) — re-dispatched the same prettier finding; verified already resolved by Still-open actionable findings (not yet fixed):
None of these three block correctness today; they're maintainability/consistency concerns and one open design question. Escalating per the review<->fix loop's non-convergence threshold rather than auto-resolving further — requesting human triage on whether findings 1-2 warrant another fix round and on the intent behind finding 3. |
Code Review — PR #377Review InformationPR Number: #377 Review SummaryOverall Assessment
Key Changes SummaryAdds a data-driven Business Value ValidationCloses the structural gap that PR #350 had to catch by hand: per-skill Independent Verification (executed in a detached worktree at the PR head)
Acceptance Criteria — verified against the diff + code (not just claims)
Security ReviewTest-only guard — no authn/authz/secrets/PII/network. Verdict: green, 0 findings (0 introduced). Detailed Review CommentsPositive Feedback
Minor Issues 💡
By-design / non-blocking notes
Questions ❓
Risk AssessmentCarried classification Definition of DoneAll DoD items met and independently verified (AC coverage, consolidated data-driven Decision: Approved with Comments. No critical or major issues; all ACs met; all quality gates independently green. Comments above are minor/by-design and non-blocking. Merge remains the human gate. |
Code Review — PR #377Review InformationPR Number: #377 Review SummaryOverall Assessment
Key Changes SummaryAdds a data-driven Business Value ValidationCloses the structural mirror-integrity gap that PR #350 fixed by hand: per-skill Verification Performed (independent)Ran in a detached worktree pinned to the PR head (
AC Verification
Code Review Checklist
Security ReviewTest-only guard. No authn/authz, secrets, PII, or I/O beyond reading the repo's own dataset/skill trees. No security-relevant surface. Classification Testing ReviewExcellent. Real-pipeline reuse (a production bug fails the guard rather than being masked), self-guarding "fixture is meaningful" test, and drift-injection covering all four drift classes through the same Detailed Review CommentsPositive Feedback
Minor Issues
Questions / Nits
Risk Assessment
Tech DebtThis PR is itself a deliberate tech-debt closure (R7.2). One residual item surfaced (non- Adoption ComplianceLevel assessed against repo ADLs: compliant. No new technical decision → no ADR required (confirmed against the two cited decision-log entries). Scope confined to DecisionApproved with Comments — all five ACs met and independently verified by executing the suite; lint + type-check clean; ADL-compliant; no critical/major issues. Findings are Minor/nit and non-blocking. Merge is the human gate. |
…otes - finding 2 (noisy failure output): drift message now carries a compact line-level diff (diffSkillMd, LCS-based, context-collapsed) instead of dumping both files in full; unit-covered for change/insert/delete/elision. - finding 3 (parallel enumerators): doc comment on datasetSkillDirs explains why it is preferred over reusing collectSkillDirs (in-memory, SKILL.md-only). - finding 1 (guard scope): in-code SCOPE note documents SKILL.md-only is by-design; follow-up tech-debt story #384 tracks extending to all artifacts. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Remediation — final summary (review clean, converged)Full map of every finding raised across the review<->fix loop on this PR, and how each was resolved. Round 1 (commit
Round 2 (commit
Round 3 (re-review, no new commit) — re-verified the round-2 prettier fix was already committed/pushed; no further change needed. Round 4 (commits
Final independent re-review — zero actionable findings. Two items surfaced were reviewed and accepted as non-actionable:
Quality gates: PASS — Verdict: review clean. All actionable findings across all rounds resolved with commit refs above; the two remaining items are accepted non-actionable dispositions, not open work. Story #352 converged. Not merged — human gate. |
…onal guard - new test asserts SKILL_COPY_OPTS.flatten/prefix + source match apps/pair-cli/config.json asset_registries.skills, so a registry change fails here (correctly attributed) instead of the guard silently computing the wrong root path - directional test now exercises the guard's dataset-derived expected-set (datasetSkillDirs+installedSkillDir+buildInstalledSkillMd) proving root-only skills are structurally never enumerated, replacing the environment-fact assertion Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
PR Title: [US-352] chore: extend mirror-equality guard to per-skill SKILL.md pairs
Story/Epic: Closes #352
Type: Chore (tech-debt)
Priority: P2
Summary
What Changed
Adds a consolidated, data-driven
it.eachguard (packages/knowledge-hub/src/tools/skill-md-mirror.test.ts+ co-located production helperskill-md-mirror.ts) asserting every root.claude/skills/<prefixed>/SKILL.mdis byte-for-byte the realpair updatecopy-pipeline transform of its dataset source (packages/knowledge-hub/dataset/.skills/<cat>/<name>/SKILL.md).Why This Change
Per-skill
SKILL.mddrift between the dataset (canonical source) and the installed root mirror was previously caught by hand (PR #350). This guard closes that structural gap by making the invariant CI-enforced, additive, and non-blocking (R7.2 tech-debt tier).Story Context
User Story: As a Pair maintainer/reviewer, I want an automated mirror-equality guard for per-skill
SKILL.mdso drift is caught by CI instead of manual review.Acceptance Criteria: AC1-AC5 all covered — see Testing section below for the mapping.
Note: the story title says "all 36 skills" — that count is stale. The dataset currently has 39 skill dirs (root
.claude/skillshas 40, +1 foragent-browserwhich has no dataset source). The guard is data-driven (iteratesdatasetSkillDirs()derived from the dataset tree at collection time), so it asserts against whatever count currently exists and needs no edit when skills are added/removed (AC5).Changes Made
Implementation Details
buildInstalledSkillMd()seeds anInMemoryFileSystemServicefrom the on-disk dataset.skillstree and runs the REALcopyDirectoryWithTransformsfrom@pair/content-opswith the exact{ flatten: true, prefix: 'pair' }optionsapps/pair-cli/config.jsondeclares for theskillsregistry — composing dir-rename (transformPath), frontmattername:sync (syncFrontmatter), relative-link-depth rewrite (rewriteLinksAfterTransform), and/commandskill-reference rewrite (rewriteSkillReferencesInFiles) in one real transform, so a bug in the production pipeline fails the guard instead of being masked by a parallel re-implementation.datasetSkillDirs()derives every dataset skill dir (capability/<name>,process/<name>, barenext) directly from the dataset tree read off disk — no hardcoded count, sorted for stableit.eachordering.agent-browser) is never asserted and is explicitly tested as NOT drift.assertRootSkillMdMatches()lives inskill-md-mirror.ts(not inline in the test) per the "gate & tooling code in tested modules" ADL — both the real on-disk guard and the drift-injection tests exercise the same code path. Throws naming the skill + apair updateregenerate hint on missing mirror (AC4) or content drift (AC2/AC3).Files Changed
packages/knowledge-hub/src/tools/skill-md-mirror.ts(production helper: dataset reader, dataset-dir discovery, real-pipeline runner, assertion helper)packages/knowledge-hub/src/tools/skill-md-mirror.test.ts(consolidatedit.eachguard + drift-injection suite)No other files touched — confined to
packages/knowledge-hubper the story's DoD scope constraint (no changes tocontent-ops,pair-cli,templates, other skills, or.claude/skills).Testing
Test Coverage
skill-md-mirror.test.ts(data-drivenit.eachover all current dataset skill dirs + directional edge case + 6 drift-injection cases)AC → Test Mapping
it.each(skillDirs)main guardname:drift fails)FAILS on frontmatter name: drift/commanddrift fails)FAILS on relative-link-depth drift,FAILS on /command skill-reference driftFAILS loudly when the root mirror is missingdatasetSkillDirs()reads the dataset tree at collection time — no hardcoded count; asserted viadiscovers dataset skill dirs directly from diskagent-browsernot drift)directional guard ignores root-only skills with no dataset sourceTest Results
Quality Assurance
Code Quality Checklist
SKILL.md, which changes directory depth on copy)content-opscopy pipeline emits its own informational logs during the in-memory run; not something this PR controls)Risk Assessment
Classification carried over from refinement:
risk:yellow(KB-default service criticality, no Criticality Table) ·cost:green(isolated, test-only, reuses existing exportedcontent-opstransforms — no production behavior change).copyDirectoryWithTransformsvia in-memory FS — no parallel logicdatasetSkillDirs()iterates the dataset tree at collection time; never a fixed countagent-browser) misread as driftReviewer Guide
Review Focus Areas
copyDirectoryWithTransforms(no parallel transform logic reimplemented) — seebuildInstalledSkillMd()inskill-md-mirror.ts.agent-browsercorrectly excluded./command, missing mirror) through the sameassertRootSkillMdMatcheshelper the real guard uses.Testing the Changes
git checkout chore/US-352-skill-md-mirror-equality-guard pnpm --filter @pair/knowledge-hub testKey Test Scenarios
agent-browserpresent in root but absent from dataset is not flagged.demoskill run through the real pipeline, then corrupted per drift class, proving the guard fails then passes when reconciled.Dependencies & Related Work
Related PRs
skills-guide-mirror.test.ts's existing mirror-equality pattern to a new artifact set.Notes for reviewer (not visible elsewhere)
packages/knowledge-hub/src/toolsonly — nocontent-ops,pair-cli,process-review,templates,pair-next, or.claude/skillschanges, consistent with the story's DoD.@pair/content-opsfunctions, no new architectural decision.