[#259] feat: verify-quality integrated with tier gate matrix (local = CI)

[rucka]

Summary

What Changed

pair-capability-verify-quality (SKILL.md) now resolves the item/PR risk:* classification tag and runs locally exactly the check set the CI tier-aware gate (#258) would run for that tier — new Step 1.5: Resolve the Tier Gate Matrix (CI parity), tier-scoped Test gate (Step 4), and Tier + Check set surfaced in the output report. v0.4.1 → 0.5.0.

• Tag resolution: PR tag first, else the story card (pre-publish), else fail-safe 🔴 (full suite) with an explicit notice in the report.
• Matrix lookup: single source = quality-model.md §4 / the KB gate matrix delivered by Tier-aware pre-merge pipeline — reads classification tags only, fail-safe red #258 — same tier-resolve.sh helper the CI pipeline uses, so local = CI by construction, not by re-implementation.
• Missing suite: explicit "suite missing — CI will fail" warning, never a silent skip.
• Matrix/KB absent: falls back to running all adopted gates (current behavior) with a notice.
• Existing idempotency (skip already-passing checks) and adoption gate-command overrides are preserved unchanged.

Also: docs site verify-quality skills-catalog row rewritten with tier behavior, and a new "Checking locally before you push (local = CI)" section on tag-driven-gates.mdx.

Why This Change

Closes story #259 of epic #210 (CI/CD gates tag-driven). Without this, verify-quality (used standalone, or composed by /pair-process-implement and /pair-process-review) can pass locally green while CI still runs a wider tier check set (or vice versa) — surprise reds at the gate, or wasted local effort over-checking green work. This wires the same gate matrix CI already reads (#258, merged) into the local capability, so local verification mirrors CI exactly.

Story Context

User Story: As a developer, I want pair-capability-verify-quality integrated with the tier gate matrix — resolving the item/PR classification tags and running locally exactly the checks the CI gate will run for that tier — so that local verification mirrors CI: no surprise reds at the gate, no over-checking on green work.

Acceptance Criteria: AC1–AC4, all covered by the new conformance test verify-quality-gate-matrix.test.ts (15 cases) + the extended tier-aware-gate.sh smoke scenario:

• AC1 (risk:green → install+lint+type+build only, same set as CI) — covered.
• AC2 (risk:yellow/risk:red widens per matrix: unit from 🟡, integration+E2E on 🔴) — covered.
• AC3 (no tag resolvable → fail-safe 🔴, explicit notice) — covered.
• AC4 (any check fails locally → red verdict, failing output surfaced) — covered by the pre-existing Test-gate report contract, now tier-scoped.

Changes Made

Implementation Details

• Change 1: New Step 1.5 in verify-quality — resolves tier (PR tag → story tag → fail-safe red) and the required check set via tier-resolve.sh (D18: single matrix source, no criteria duplicated in the skill).
• Change 2: Step 4 (Test gate) scoped to the resolved tier's required suites (unit from 🟡; integration+E2E from 🔴); missing-suite path fails explicitly rather than skipping silently.
• Change 3: Output report gains Tier + Check-set lines so the verdict is auditable against what CI would run.
• Change 4: New conformance test (verify-quality-gate-matrix.test.ts, 15 cases, written RED→GREEN) pinning the AC invariants + matrix parity with tier-resolve.sh; extended tier-aware-gate.sh smoke scenario with a verify-quality local=CI parity audit (same helper CI uses).

Files Changed

• Modified:
  • packages/knowledge-hub/dataset/.skills/capability/verify-quality/SKILL.md (source of record)
  • .claude/skills/pair-capability-verify-quality/SKILL.md (regenerated mirror — deterministic name-prefix transform only, same approach as PR [#314] feat: wire template-override resolution into skills #367)
  • apps/website/content/docs/reference/skills-catalog.mdx
  • apps/website/content/docs/concepts/tag-driven-gates.mdx
  • scripts/smoke-tests/scenarios/tier-aware-gate.sh
• Added:
  • packages/knowledge-hub/src/conformance/verify-quality-gate-matrix.test.ts

No deployable service/package changed — this is a KB skill definition, its .claude distribution mirror, docs, a conformance test, and a smoke-test extension. Services to Release section omitted per the conditional rule.

Testing

Test Results

knowledge-hub test (383 tests incl. new 15) + check:links — PASS
skills:conformance (39 skills) — PASS
knowledge-hub ts:check, lint, mdlint:check — PASS
monorepo ts:check (pre-commit hook) — PASS
website docs:staleness, mdlint:check — PASS
hygiene:check, dup:check (2.01% < 5%) — PASS
scripts/smoke-tests/scenarios/tier-aware-gate.sh — PASS (incl. new verify-quality parity assertions)

Note: knowledge-hub prettier:check reports a pre-existing drift in src/conformance/refine-story-draft-to-ready.test.ts (NOT touched by this story, unrelated to this diff). Files touched by this PR are prettier-clean.

Testing Strategy

• Happy path per tier: conformance test asserts the exact check set for 🟢/🟡/🔴, matching tier-resolve.sh's own matrix output.
• Fail-safe: untagged/malformed-tag path asserted to resolve 🔴 with an explicit notice, never a silent default.
• Missing suite: asserted an explicit "CI will fail" failure, not a skip.
• Matrix/KB absent: asserted fallback to the full adopted gate set with a notice.
• Executable parity: the extended smoke scenario re-invokes the same tier-resolve.sh helper CI uses, proving local = CI is structural (shared code path), not a documentation claim.

Reviewer Guide

Review Focus Areas

1. D18 invariant — confirm verify-quality still contains zero classification/matrix criteria of its own; it composes tier-resolve.sh (delivered by Tier-aware pre-merge pipeline — reads classification tags only, fail-safe red #258, merged) rather than re-implementing the matrix. Grep-verifiable, pinned by the smoke scenario.
2. Scope guardrail — this PR does not touch the code-review template or the /pair-process-review skill (explicit constraint on this story); confirm the diff has none of that.
3. .claude mirror parity — the mirror diff should be name-token-only vs. the dataset source (verify-quality → pair-capability-verify-quality, /implement → /pair-process-implement, /review → /pair-process-review, /classify → /pair-capability-classify); no independent drift.
4. No new ADR — the tier/suite resolution logic already lives in the shipped, smoke-tested tier-resolve.sh (Tier-aware pre-merge pipeline — reads classification tags only, fail-safe red #258); this story composes it. Confirm this doesn't warrant a new production module per the gate-tooling ADL (2026-07-13-gate-tooling-code-in-tested-modules.md).

Testing the Changes

git checkout feature/US-259-verify-quality-gate-matrix
pnpm install
pnpm --filter @pair/knowledge-hub test
pnpm --filter @pair/knowledge-hub run skills:conformance
bash scripts/smoke-tests/scenarios/tier-aware-gate.sh

Dependencies & Related Work

• Depends on (already landed): Tier-aware pre-merge pipeline — reads classification tags only, fail-safe red #258 (tier-aware pre-merge pipeline + tier-resolve.sh helper, PR [#258] feat: tier-aware pre-merge pipeline — reads classification tags only, fail-safe red #362) — merged, present on origin/main.
• Part of: Epic CI/CD gates tag-driven #210 (CI/CD gates tag-driven).
• Dependent: Split implement: checkpoint + publish-pr via subagent #206 (publish-pr invokes verify-quality as its gate before opening the PR) — unaffected here, composition contract unchanged.
• DoD note (flagging, not silently deciding): "CLI (apps/pair-cli) updated where install/update/distribution is touched" — N/A here. No new distributed asset: verify-quality's SKILL.md and tier-resolve.sh already ship; install/update/distribution machinery is untouched by this diff.
• Classification tags: the story carries no risk:* label at time of PR — opened without tags per the propagation rule (copy-only; authoritative classification happens in review).

[rucka]

Code Review — PR #378

Review Information

PR Number: #378
Author: rucka
Reviewer: Independent reviewer (automated, /pair-process-review)
Review Date: 2026-07-24
Story/Epic: US-259 (Epic #210 — CI/CD gates tag-driven)
Review Type: Feature (skill definition + docs + tests)

Review Summary

Overall Assessment

• Approved with Comments — minor, non-blocking issues noted; can merge.

Key Changes Summary

pair-capability-verify-quality (SKILL.md, v0.4.1 → 0.5.0) now resolves the item/PR risk:* tag and runs locally exactly the tier-scoped check set the CI pre-merge gate (#258) would run: new Step 1.5 (Resolve the Tier Gate Matrix), a tier-scoped Test gate (Step 4), and Tier + Check-set lines in the report. Plus docs (skills-catalog + tag-driven-gates), a 15-case conformance test, and a smoke-test parity extension.

Business Value Validation

Confirmed. Local verification now mirrors CI by reusing the same shipped tier-resolve.sh helper CI uses (structural parity, not re-implementation) — delivering the story's core value ("no surprise reds at the gate, no over-checking on green work").

Acceptance Criteria Verification

AC	Verdict	Evidence
AC1 🟢 → install+lint+type+build only	MET	Step 1.5 matrix == required_suites_for_tier green == quality-model.md §4. Conformance + smoke assert it.
AC2 🟡 +unit / 🔴 +integration+E2E	MET	Same three-way agreement; smoke asserts yellow => …unit, red => …unit integration e2e.
AC3 no tag → fail-safe 🔴 + explicit notice	MET	resolve_tier returns red on absent/malformed; Step 1.5 prints explicit fail-safe line; smoke untagged/malformed => red.
AC4 any check fails → red + failing output surfaced	MET	Step 4 + output format ("surface the failing command output (AC4)"). See Minor #3 on test strength.

Business rules: D18 (no classification criteria in the skill) — verified: the skill composes tier-resolve.sh (tags-only) and defers the matrix to quality-model.md §4; grep-pinned by the smoke scenario. Idempotency (skip already-passing) and adoption gate-command overrides preserved. All three edge cases (pre-publish story-card fallback, missing suite → explicit "CI will fail", matrix/KB absent → full-suite fallback with notice) present.

Code Review Checklist

• Requirements Met — Yes, all 4 ACs + edge cases.
• Mirror parity — .claude/…/SKILL.md is byte-identical to the dataset source after the documented name-token transform (verify-quality→pair-capability-verify-quality, /implement→/pair-process-implement, /review→/pair-process-review, /classify→/pair-capability-classify). No independent drift.
• Single-source (D18) — skill matrix table == tier-resolve.sh == quality-model.md §4. No fourth authority introduced.
• Scope guardrail — diff does NOT touch the code-review template or /pair-process-review. Respected.
• Semver — 0.4.1 → 0.5.0 (minor) appropriate for an added, backward-compatible capability; composition contract unchanged.

Security Review

No security surface: SKILL.md prose, docs (.mdx), a vitest content test, a bash smoke script. Illustrative gh snippets are read-only. No secrets, no injection surface in shipped code. Verdict: green — 0 findings.

Testing Review

Independently executed on the PR head (worktree, deps built):

• @pair/knowledge-hub vitest — 383 passed (19 files), incl. the new 15 conformance cases.
• skills:conformance — PASS (39 skills).
• scripts/smoke-tests/scenarios/tier-aware-gate.sh — PASS (exit 0), incl. the 4 new verify-quality parity assertions.

The executable parity (skill resolving suites through the same tier-resolve.sh CI uses) is genuinely proven by the smoke scenario, not merely claimed.

Documentation Review

skills-catalog.mdx row and tag-driven-gates.mdx ("Checking locally before you push (local = CI)") updated with tier behavior — satisfies the DoD docs item. Links in the SKILL.md (quality-model.md, tier-aware-pipeline.md, tier-resolve.sh) resolve.

Detailed Review Comments

Positive Feedback

• Structural (not documentary) CI parity via the shared helper — the highest-value design choice for this story.
• D18 honored cleanly: zero matrix duplication, pinned by both the conformance parity block and the smoke scenario.
• Fail-safe, missing-suite, and matrix-absent paths all fail loud, never silent.

Minor Issues 💡

• dataset/.skills/capability/verify-quality/SKILL.md — Step 1.5, Act 2 (tag-resolution snippet) + $scope/$story Arguments row — The illustrative bash uses the single $story value for both gh pr view "$story" and gh issue view "$story", and the argument is documented as a dual-purpose "Story/PR id". In the two real composition paths this resolves correctly (review passes the PR number; implement pre-publish passes the story id → gh pr view misses → story-card fallback). But two gaps remain: (a) the documented "if omitted, resolved from the branch's PR" path is not reflected — the snippet always passes "$story", so an empty $story fails both lookups and lands on fail-safe 🔴 rather than auto-detecting the current-branch PR; (b) standalone invocation passing a story id while a PR already carries a review-raised (D17) tag would read the stale story-card tier and under-run vs CI — the exact "surprise red" the story targets. Recommendation: make the primary path gh pr view with no argument (current-branch PR), use an explicit story id only for the pre-publish story-card fallback, and clarify the $story doc to separate PR-number vs story-id semantics.
• SKILL.md — overview / Notes ("local = CI") — The coverage guardrail (a separate opt-in CI job, "twin of tiering") is not part of the tier check set and is not mirrored locally. The skill's precise claim ("the tier and its suites … match") is correctly scoped, but the prominent "local = CI" headline could over-promise when a project sets Coverage guardrail: enabled. Recommendation: add a one-line caveat that the coverage guardrail is not a tier suite and is not run locally (cheap in-PR honesty fix), or track as a follow-up.

Nits / Questions ❓

• packages/knowledge-hub/src/conformance/verify-quality-gate-matrix.test.ts (AC4 case) — The AC4 assertion is a single loose grep for a "failing command output" phrase; it doesn't pin the behavioral linkage (failing check ⇒ red verdict ⇒ output surfaced), so a future edit could drop the red-verdict wording and still pass. Consistent with this corpus's prose-content-test style and behavior is smoke-covered, so low impact. Recommendation (optional): also assert the red-verdict wording co-occurs.
• DoD — "CLI (apps/pair-cli) updated where install/update/distribution is touched" marked N/A — Confirmed correct: apps/pair-cli/config.json distributes .claude/skills/ in canonical (directory-copy) mode with no per-skill version pin, and no apps/pair-cli source is touched; the regenerated mirror ships wholesale. No CLI change required.

Risk Assessment

Low. Docs/skill-definition change, no runtime production code; composition contract for /pair-process-implement and /pair-process-review unchanged. Classification: low risk (no security relevance, low coupling).

Tech Debt

None material. The two Minor items above are the only debt; neither blocks.

Overall Decision

APPROVED WITH COMMENTS. All acceptance criteria and edge cases are met, tests (383 + smoke + conformance) pass on the PR head, mirror parity is exact, D18 single-source is honored, and the scope guardrail is respected. Findings are Minor/nit/question only and non-blocking.

[rucka]

In reply to the initial code review (Approved with Comments).

Review/fix loop summary — escalating to human (non-convergence)

Round 1 (commit 572965b) — fixes for the 3 original findings:

• Finding 1 [Minor] tag-resolution snippet reused one $story for both gh pr view/gh issue view; no current-branch auto-detect; standalone story-id run could under-run vs a review-raised PR tag. → Resolved. Reworked Step 1.5 Act 2 so the primary path is no-arg gh pr view (current-branch PR labels, authoritative), with gh issue view "$story" firing only as a pre-publish fallback when there's no current-branch PR. Reworded the $story Arguments row and the pre-publish edge note accordingly.
• Finding 2 [Minor] "local = CI" over-promises: the coverage guardrail (separate opt-in CI job) isn't a tier suite and isn't mirrored locally. → Resolved. Added a caveat line to the "Local = CI" Notes bullet plus a dedicated "Coverage guardrail is NOT mirrored locally" bullet.
• Finding 3 [Minor] AC4 conformance test was a single loose grep for "failing command output"; didn't pin the failing-check ⇒ red-verdict ⇒ output-surfaced linkage. → Resolved. Strengthened the AC4 case to require "red verdict" wording to co-occur (within 120 chars) with "failing command output".

Quality gates round 1: PASS (knowledge-hub vitest 383/383, skill-refs 16/16, mirror guard green, check:links clean, markdownlint clean).

Round 2 (commit 2ddf3df) — re-review raised one new item:

• Finding [Questions] no-arg gh pr view primary resolves the tier from the current-branch PR; when /pair-process-review composes verify-quality with only $scope (no $story) from a context not on the PR branch, resolution degrades to fail-safe red rather than the review-time (D17) tier. → Resolved per the finding's own in-scope recommendation. Added a blockquote note to the Composition Interface section (source + mirror) documenting the current-branch assumption, that this degrades safely to over-check (never under-check) since /pair-process-review is non-blocking on verify-quality, and that forwarding the resolved tier is an out-of-scope change belonging to /pair-process-review itself — not made here.

Quality gates round 2: PASS (skills:conformance 39/39, knowledge-hub vitest 383/383, mirror guard 15/15, check:links clean, markdownlint clean).

Full detail in the working review log: .pair/working/reviews/259.md.

Still-open findings (round 3) — flagging for human review

Two new Minor findings surfaced on this latest pass, both still open (no fix applied yet — escalating rather than looping further):

1. [Minor] Secret-scan gap in the "Local = CI" caveat list — packages/knowledge-hub/dataset/.skills/capability/verify-quality/SKILL.md Notes (mirrored in .claude/skills/pair-capability-verify-quality/SKILL.md and apps/website/content/docs/concepts/tag-driven-gates.mdx §"Checking locally before you push"). The caveat list scopes "local = CI" to the tier check set and calls out the coverage guardrail as the one non-mirrored CI check — but is silent on the deterministic secret-scan job, which the tier-aware CI gate runs unconditionally at every tier (quality-model §4 line 82, setup-gates SKILL.md, smoke assertion). Secret-scan is not a verify-quality gate and not a default pre-commit hook (husky pre-commit is lint/type only; pre-push is lint), so a developer can get a green local verdict and still hit a red secret-scan at the CI gate — the "surprise red" the story targets. Recommendation: add one caveat line parallel to the coverage-guardrail one, noting the unconditional secret-scan job is not a tier suite and isn't mirrored by verify-quality (skill file + mirror + docs, one line each).
2. [Minor] Per-tier mapping restated in prose isn't pinned to the helper's output — packages/knowledge-hub/src/conformance/verify-quality-gate-matrix.test.ts:105-127 ("matrix parity with tier-resolve.sh") vs SKILL.md Step 1.5 tier table + Step 4 prose. The conformance test only asserts each suite key appears as a substring in the SKILL text; the smoke test only greps that helper function names are referenced. Neither pins the documented 🟢/🟡/🔴 table to required_suites_for_tier's actual output, so a drifted table (e.g. wrong suite set per tier) would fail no test — genuine if low correctness risk, since the skill instructs the executing agent to defer to ACTIVE_SUITES from the helper (not the table) at runtime. Recommendation: either assert the exact per-tier mapping against required_suites_for_tier's output in the smoke scenario (it already computes green/yellow/red suite lists), or collapse the redundant table/prose into a single pointer at the helper.

Two rounds of iteration have already resolved 4 findings cleanly; these last two are being surfaced for a human call on whether to spend a third fix round or accept/defer as tracked debt, rather than continuing the loop unattended.

[rucka]

Code Review — PR #378

Review Information

PR Number: #378
Author: rucka (Gianluca Carucci)
Reviewer: Independent review (Claude, /pair-process-review)
Review Date: 2026-07-24
Story/Epic: US-259 (Epic #210 — CI/CD gates tag-driven)
Review Type: Feature (KB skill definition + docs + conformance/smoke tests)

Review Summary

Overall Assessment

• Approved with Comments — Minor/question-level items noted; none blocking.

Key Changes Summary

verify-quality (SKILL.md, v0.4.1 → 0.5.0) gains Step 1.5 — Resolve the Tier Gate Matrix: it resolves the risk:* tier from the current-branch PR (else the story card via $story, else fail-safe 🔴) and scopes the Test gate to the tier's suites, reusing the shipped tier-resolve.sh helper (resolve_tier / required_suites_for_tier / require_suite) so local mirrors CI by construction. .claude mirror regenerated, docs updated (skills-catalog row + tag-driven-gates "local = CI" section), plus a 15-case conformance test and an extended tier-aware-gate.sh smoke scenario.

Business Value Validation

Delivers the story's core value: local verification runs the same tier check set CI would run — no over-checking green work, no surprise reds — without duplicating the matrix (D18). Verified: the skill's documented matrix (🟢 install+lint+type+build; 🟡 +unit; 🔴 +integration+e2e) is byte-faithful to both quality-model.md §4 and the executable required_suites_for_tier.

Acceptance Criteria Verification

AC	Status	Evidence
AC1 — 🟢 green → install+lint+type+build only	Met	Step 1.5 matrix + Step 4 (green → no test suite); matches §4 + helper
AC2 — 🟡/🔴 widen per matrix (unit from 🟡; integration+E2E on 🔴)	Met	Step 1.5 matrix + Step 4 suite list; matches helper output (smoke-tested live)
AC3 — no tag → fail-safe 🔴, explicit notice	Met	Step 1.5 fail-safe prose + resolve_tier red default (smoke-tested)
AC4 — any check fails → red verdict, failing output surfaced	Met	Step 4 + Output Format ("A red verdict always surfaces the failing command output (AC4)")

Business rules (no classification criteria / D18; adoption gate-command overrides; idempotency) and edge cases (pre-publish story-card fallback; missing suite = explicit fail; matrix/KB absent = full-suite fallback + notice) are all present in the prose.

Verification Performed

• Matrix single-source parity: skill matrix == quality-model.md §4 == tier-resolve.sh (no independent restatement — D18 honored).
• .claude mirror integrity: name-token transform of the dataset source produces zero residual diff — clean regeneration, no drift.
• Smoke test: tier-aware-gate.sh run live — PASS (per-tier resolution, fail-safe, missing-suite, D18 all green).
• Conformance test: 15 assertions verified against file content by inspection — all satisfied.

Testing Review

Tests match this repo's convention for prose skills: content-invariant conformance (grep on SKILL.md) + structural parity + a smoke scenario that behaviorally executes the shared tier-resolve.sh (per tier, fail-safe, missing-suite). verify-quality itself is LLM-prose, so end-to-end execution is asserted transitively (skill composes the tested helper) rather than run on a fixture — see Question ❓ below.

Security Review

No security surface. Read-only skill prose (the sole write, Step 5.C, is pre-existing and untouched). No secrets, inputs, or auth. assess-security: green — no introduced findings.

Detailed Review Comments

Positive Feedback

• Textbook D18 compliance: the skill composes the shipped helper instead of re-implementing the matrix; the smoke scenario grep-pins this so it can't silently drift.
• Honest, well-reasoned scope boundaries documented in-PR (off-branch composition degradation, coverage-guardrail carve-out, no-new-ADR rationale).
• Clean mirror regeneration (verified zero-diff after name transform).

Minor Issues 💡

• verify-quality/SKILL.md (Notes, "Local = CI" / Coverage-guardrail caveat) — The parity value-prop ("local = CI … no surprise reds at the gate") explicitly carves out the coverage guardrail as a non-mirrored CI job, but is silent on the deterministic secret-scan job, which quality-model.md §4 states runs unconditionally at every tier (including 🟢). A dev who commits a secret passes verify-quality green locally yet hits a red secret-scan at CI — exactly the "surprise red" the claim disclaims. Fix is a one-line caveat (like the coverage note): secret scanning is a separate CI layer (D24, owned by setup-gates/assess-security) and deliberately not mirrored here. Fixable in this PR; does not change behavior.
• verify-quality-gate-matrix.test.ts (header comment, ~line 8) — The comment says the executable parity, "verify-quality resolving suites through the same tier-resolve.sh helper … is additionally smoke-tested." The pair-capability-verify-quality integrated with the gate matrix (local = CI) #259 smoke section actually greps the SKILL.md for the helper references; the behavioral resolve_tier/required_suites_for_tier execution in the smoke test is the pre-existing Tier-aware pre-merge pipeline — reads classification tags only, fail-safe red #258 block testing the helper directly. The transitive parity argument is valid, but the comment slightly overstates ("resolving") what the smoke test does for verify-quality. Tighten the wording.

Questions ❓

• Testing strategy vs. DoD — The story DoD says "Local-vs-CI parity verified per tier on a fixture repo (same check set, same verdict)" and "Fail-safe and missing-suite paths exercised." Those paths are exercised on tier-resolve.sh (smoke), and verify-quality's use is asserted structurally (it references the same helper) — no fixture repo actually runs verify-quality per tier. Given skills are LLM-prose (not executable), this looks like the intended, correct interpretation, but please confirm the DoD is considered met by the structural/transitive approach rather than a literal fixture run.
• /pair-process-review composition path — "no over-checking" not fully realized — When /pair-process-review composes verify-quality off the PR branch with only $scope (no $story), the no-arg gh pr view returns empty → resolution degrades to fail-safe 🔴 (full set) rather than the classify-phase tier, so a green PR is over-checked locally. The PR documents this as a safe over-check, non-blocking (review's classify is authoritative), with tier-forwarding deferred to /pair-process-review as out of scope. That's a sound boundary — but is there a tracked follow-up (in epic CI/CD gates tag-driven #210) for /pair-process-review to forward its resolved tier so the review path also gets exact parity? If not, worth filing one. Non-blocking.

Adoption & ADR Compliance

No new technical decision — the tier/suite logic lives in the already-shipped, smoke-tested tier-resolve.sh (#258); this story composes it. Consistent with the gate-tooling ADL (2026-07-13-gate-tooling-code-in-tested-modules.md). No new ADR required. Scope guardrail confirmed: the diff touches neither the code-review template nor /pair-process-review.

Definition of Done

• AC1–AC4: met (prose + conformance + smoke).
• Docs site updated (skills-catalog + tag-driven-gates): met.
• CLI (apps/pair-cli) distribution: correctly flagged N/A (no distribution machinery touched) — flagged, not silently skipped.
• Fixture-repo per-tier parity: satisfied structurally (see Question above).

Verdict

Approved with Comments. All four ACs met and independently verified against the single-source matrix; mirror clean; smoke suite green. Two Minor doc items (secret-scan caveat completeness; test-comment precision) and two Questions (DoD fixture interpretation; review-path over-check follow-up) — none blocking.

---

Independent review via /pair-process-review. Reviewed from acceptance criteria + PR diff + branch code only; author development context (.pair/working/) was not read.

[rucka]

@/private/tmp/claude-501/-Users-gianlucacarucci5-projects-pair/6b978b2a-c3b5-4d25-be2b-01366429c4a9/scratchpad/comment-259.md

[rucka]

Code Review — PR #378

Review Information

PR Number: #378
Author: rucka (Gianluca Carucci)
Reviewer: Independent reviewer (/pair-process-review)
Review Date: 2026-07-24
Story/Epic: US-259 (Epic #210 — CI/CD gates tag-driven)
Review Type: Feature (KB skill definition + docs + conformance test + smoke extension)

Review Summary

Overall Assessment

• Approved with Comments — minor issues noted, can merge

Key Changes Summary

verify-quality (SKILL.md, v0.4.1→0.5.0) gains Step 1.5 — Resolve the Tier Gate Matrix (CI parity): it resolves the risk:* tag (current-branch PR → story card pre-publish → fail-safe red) and runs locally exactly the tier's check set, composing the shipped tier-resolve.sh helper (resolve_tier / required_suites_for_tier / require_suite) rather than restating a matrix (D18). Test gate (Step 4) is tier-scoped; the output report surfaces Tier + Check-set. Plus: .claude mirror regenerated, docs (tag-driven-gates.mdx, skills-catalog.mdx), a 15-case conformance test, and a smoke-scenario parity audit.

Business Value Validation

Value ("local = CI: no surprise reds at the gate, no over-checking on green") is delivered for the tier check-set + verdict, verified structurally (same helper CI uses). Two carve-outs (coverage guardrail, secret scanning) are honestly documented as out of parity scope — see Minor #1.

Code Review Checklist

Functionality — Acceptance Criteria

• AC1 (🟢 green → install+lint+type+build, same as CI) — MET. Skill matrix == required_suites_for_tier green == quality-model §4.
• AC2 (🟡 +unit, 🔴 +integration+E2E) — MET. All three sources agree; smoke test asserts helper output byte-for-byte.
• AC3 (no resolvable tag → fail-safe 🔴 + explicit notice) — MET. resolve_tier returns red for empty/malformed; skill states the explicit report line; smoke asserts the stderr warning.
• AC4 (any check fails → red verdict, failing output surfaced) — MET. Step 4.3 + output-format note; conformance test pins red-verdict/failing-output co-occurrence.

Business rules & edge cases

• D18 (no classification criteria) — verified: skill reads tags + matrix only; helper carries no diff/path/size heuristics (smoke grep audit passes).
• Idempotency (skip passing) and adoption gate-command overrides preserved.
• Pre-publish story-card fallback, missing-suite explicit failure, and matrix/KB-absent full-suite fallback all present and correct.

Technical Standards / Architecture

• Mirror parity — .claude mirror is a clean name-token-only transform of the dataset source (verified: identical after normalizing /pair-* prefixes). No independent drift.
• No new ADR needed — resolution logic lives in the already-shipped, smoke-tested tier-resolve.sh (Tier-aware pre-merge pipeline — reads classification tags only, fail-safe red #258); this story composes it (gate-tooling ADL 2026-07-13). Correct.

Security Review

/pair-capability-assess-security ($mode: review): no security-relevant surface introduced — the diff is skill prose, docs, a read-only content test, and a bash smoke extension. No inputs, secrets, auth, or network. Verdict: green (0 findings introduced). Note: the skill correctly documents that it does not mirror the unconditional secret-scan CI layer (D24).

Testing Review

• New conformance test verify-quality-gate-matrix.test.ts (15 cases) — ran locally: 15/15 PASS.
• Smoke tier-aware-gate.sh — ran locally: all assertions PASS (helper behaviorally executed for tier + suite resolution; verify-quality audited for parity wiring).
• Two-layer verification is sound: helper behavior is executed, skill composition is audited. Prose-skill limitation noted (Minor #6).

Documentation Review

Docs site updated per DoD: tag-driven-gates.mdx (new "local = CI" section) and skills-catalog.mdx row rewritten. Accurate and consistent with the matrix.

Detailed Review Comments

Positive Feedback

• Matrix parity is real, not asserted: three sources (skill prose, required_suites_for_tier, quality-model §4) agree, and the smoke test executes the helper to prove it.
• Graceful-degradation coverage is thorough (fail-safe red, missing-suite loud failure, matrix-absent fallback, gh-unavailable → fail-safe).
• Carve-outs (coverage guardrail, secret scanning) are disclosed honestly rather than glossed over.
• Composition-path degradation is documented and its fix is tracked (/pair-process-review forwards its resolved tier to verify-quality (exact review-path parity) #382).

Minor Issues 💡

• [SKILL.md frontmatter description + skills-catalog.mdx] — The one-liner claims "so local verification mirrors CI (no surprise reds at the gate)" without the carve-out. "local = CI" is scoped in the body to the tier check set + verdict only; coverage guardrail and secret scanning are explicitly NOT mirrored. This repo has Coverage guardrail: enabled, so a local-green verify-quality can still hit a CI-red coverage regression — precisely a "surprise red". Recommend hedging the one-liner (e.g. "mirrors CI for the tier check set"). Non-blocking; AC are scoped to the tier matrix, which is met.
• [SKILL.md Arguments / Step 4] — $scope × tier composition is under-specified: e.g. $scope=tests on a 🟢 green tier (which has no test suites). Does it run nothing and report PASS? A one-line clarification of the narrow-only intersection would remove ambiguity.
• [SKILL.md Step 1.5, bash block] — Path-convention mismatch: the illustrative source .pair/knowledge/assets/tier-resolve.sh is repo-root-relative, while the markdown links use skill-relative ../../../.pair/.... Sourcing from a non-root CWD degrades safely to the full-suite fallback, so impact is low, but the two conventions in one file can confuse.

Questions ❓

• [SKILL.md Composition Interface blockquote] — Off-branch /pair-process-review composition (no $story) degrades to fail-safe 🔴 rather than the classify-phase tier. Documented as a safe over-check (never under-check) and non-blocking. Confirmed tracked as /pair-process-review forwards its resolved tier to verify-quality (exact review-path parity) #382. No action required in this PR.
• [SKILL.md Step 1.5 step 4 / tier-resolve.sh] — Suite-presence check grep -q '"test:<suite>"' package.json inspects only root package.json. In a pnpm monorepo run from root this could spuriously flag a suite "missing". It is byte-consistent with the CI source (tier-aware-pipeline.md, Tier-aware pre-merge pipeline — reads classification tags only, fail-safe red #258, identical "layout-independent" idiom) — so fixing it here would break the local=CI parity that is the point; if the idiom is wrong it is a Tier-aware pre-merge pipeline — reads classification tags only, fail-safe red #258 concern.
• [verify-quality-gate-matrix.test.ts] — The skill's own Step 1.5 wiring prose is not behaviorally executed; only the shared helper is executed and the skill content is regex-asserted. A subtle wiring bug in the prose wouldn't be caught. This matches the whole-corpus prose-skill convention (gate-tooling ADL 2026-07-13) and is acknowledged in the test header — noted, not blocking.

Risk Assessment

Risk	Impact	Probability	Mitigation
Matrix duplication drifts from CI	Med	Low	Single source (required_suites_for_tier); smoke audit greps for the wiring; conformance test guards drift
Coverage/secret regressions still surprise devs locally	Low	Med	Documented carve-outs; softer branding recommended (Minor #1)
Tiering is disabled (default) in this repo	—	—	New behavior dormant here until opt-in; full-suite parity preserved

Definition of Done

• All AC implemented + verified — yes.
• Docs site updated with tier behavior — yes.
• CLI (install/update/distribution) — N/A, justified: only skill content changed; the CLI's name-prefix transform and KB-release distribution machinery are untouched.
• Local-vs-CI parity + fail-safe/missing-suite paths exercised — yes (test + smoke, both run green here).

---

Decision: APPROVED WITH COMMENTS

No critical or major issues. All four acceptance criteria are met and independently verified (matrix parity across three sources; 15/15 conformance + smoke pass locally). Comments are advisory (documentation branding, one under-specified interaction, a nit) — none block merge; #382 already tracks the one deferred composition-path improvement.

🤖 Independent review via /pair-process-review

[rucka]

In reply to the initial code review (Approved with Comments).

Remediation — full review/fix cycle (rounds 1–5), review clean

Consolidated response mapping every finding raised across the review/fix loop (initial review + re-reviews) to its resolution. Full round-by-round detail lives in the (now-removed) working log .pair/working/reviews/259.md; this comment is the durable record.

Findings from the initial review (this comment)

• [Minor] Tag-resolution snippet reused one $story for both gh pr view/gh issue view; no current-branch auto-detect; standalone story-id run could under-run vs a review-raised PR tag (SKILL.md Step 1.5 Act 2 + $story Arguments row) → Resolved, commit 572965b (round 1). Primary path is now no-arg gh pr view (current-branch PR labels, authoritative); gh issue view "$story" fires only as the pre-publish fallback when there's no current-branch PR. $story Arguments row and pre-publish edge note reworded to match.
• [Minor] "Local = CI" over-promised: coverage guardrail (separate opt-in CI job) not a tier suite, not mirrored locally (SKILL.md Notes) → Resolved, commit 572965b (round 1). Added a caveat to the "Local = CI" bullet plus a dedicated "Coverage guardrail is NOT mirrored locally" bullet.
• [Nit] AC4 conformance case was a single loose grep for "failing command output"; didn't pin failing-check ⇒ red-verdict ⇒ output-surfaced linkage (verify-quality-gate-matrix.test.ts, AC4 case) → Resolved, commit 572965b (round 1). Test now additionally requires "red verdict" wording to co-occur (within 120 chars) with "failing command output".
• [Nit] DoD "CLI updated where install/update/distribution touched" marked N/A → Confirmed correct as-is by the reviewer; no fix required (canonical directory-copy distribution, no apps/pair-cli source touched).

Findings surfaced during the remediation loop (rounds 2–5)

• [Questions, round 2] No-arg gh pr view primary resolves the tier from the current-branch PR; when /pair-process-review composes verify-quality with only $scope (no $story) off the PR branch, resolution degrades to fail-safe red rather than the review-time (D17) tier (SKILL.md Step 1.5 Act 2 + Composition Interface) → Resolved per the finding's own in-scope recommendation, commit 2ddf3df (round 2). Added a blockquote note to the Composition Interface section documenting the current-branch assumption, that this degrades safely to over-check (never under-check) since /pair-process-review is non-blocking on verify-quality, and that forwarding the resolved tier is an out-of-scope change belonging to /pair-process-review — not made here (see disposition below; tracked as /pair-process-review forwards its resolved tier to verify-quality (exact review-path parity) #382).
• [Minor, round 3] Secret-scan gap in the "Local = CI" caveat list — deterministic secret-scan job runs unconditionally at every tier (incl. 🟢) and isn't disclosed as a non-mirrored layer (SKILL.md Notes) → Resolved, fix commit cbeee48 (round 3). Added a "Secret scanning is NOT mirrored locally" bullet parallel to the coverage-guardrail caveat, in both source and mirror.
• [Minor, round 3] Test header comment overstated that verify-quality's suite-resolution "is additionally smoke-tested"; the pair-capability-verify-quality integrated with the gate matrix (local = CI) #259 smoke block only greps for helper references, the behavioral execution is the pre-existing Tier-aware pre-merge pipeline — reads classification tags only, fail-safe red #258 block (verify-quality-gate-matrix.test.ts:8) → Resolved, fix commit cbeee48 (round 3). Rewrote the header comment to describe the two-layer transitive verification precisely (smoke asserts references to the shared helper; the helper's own resolution is behaviorally executed in Tier-aware pre-merge pipeline — reads classification tags only, fail-safe red #258).
• [Questions, round 3] Confirm DoD "local-vs-CI parity on a fixture repo" is met by the structural/transitive approach rather than a literal fixture run → Confirmed, round 3. Skills are LLM-prose and can't be executed in a unit/smoke harness; parity is met via the behaviorally-executed shared tier-resolve.sh helper (Tier-aware pre-merge pipeline — reads classification tags only, fail-safe red #258) + verify-quality being asserted to compose that exact helper (D18). No code change; comment tightening in the prior item makes this explicit.
• [Questions, round 3] Confirm a follow-up is tracked for /pair-process-review to forward its resolved tier to verify-quality; file one if not → Resolved, round 3. No existing follow-up found; filed #382 under epic CI/CD gates tag-driven #210.
• [Minor, round 4] Frontmatter/catalog one-liner over-promised "mirrors CI (no surprise reds)" without naming the coverage-guardrail/secret-scan carve-outs (SKILL.md description + skills-catalog.mdx) → Resolved, fix commit b0e1f4d (round 4). Hedged the one-liner in all three surfaces (dataset SKILL.md, .claude mirror, skills-catalog.mdx) to "mirrors CI for the tier check set … the coverage-guardrail and secret-scan CI layers are NOT mirrored locally".
• [Minor, round 4] $scope × tier empty-intersection case under-specified (e.g. $scope=tests on 🟢 green) (SKILL.md Arguments table) → Resolved, fix commit b0e1f4d (round 4). Added "Empty intersection is a no-op, not a failure" clarification — runs nothing, reports SKIPPED (tier), never a FAIL, never a widen.
• [Minor, round 4] Path-convention mismatch: Step 1.5 bash sources the helper repo-root-relative while surrounding markdown links are skill-relative (SKILL.md Step 1.5 Act 2 bash) → Resolved, fix commit b0e1f4d (round 4). Added a preamble noting the snippet assumes the repo-root CWD (where CI runs it) and that the markdown links are just this file's own relative path to the same asset.
• [Minor, round 5] Tiering disabled branch enumerated ACTIVE_SUITES in a way that, read in isolation, could suggest a project lacking integration/e2e suites fails in disabled mode, instead of the intended "skip absent gate" behavior (SKILL.md Step 1.5, point 1) → Resolved, fix commit b97dff4 (round 5). Added a cross-reference: "Suites the repo lacks are still SKIPPED, not failed, here" — ties to the Graceful Degradation clause and clarifies the missing-suite-is-a-failure rule fires only on the tier-enabled path.

Accepted / non-actionable (with disposition)

• [Questions] Composition Interface note (~line 185/435): /pair-process-review's primary composition path (Step 2.1, $scope=all only, typically off the PR branch) doesn't get exact tier parity — no-arg gh pr view returns empty with no $story fallback, so resolution degrades to the fail-safe 🔴 full set rather than the classify-phase tier. Impact is bounded to the safe over-check direction (never an under-check); /pair-process-review is non-blocking on verify-quality; behavior is transparently documented in-skill and in this PR. The fix belongs to /pair-process-review, not this skill. Disposition: Deferred to #382. No change in this PR.
• [Questions] QA DoD "local-vs-CI parity verified per tier on a fixture repo": delivered structurally (content/regex conformance on SKILL.md + behavioral parity on the shared tier-resolve.sh helper) rather than as a literal fixture-repo execution of the verify-quality skill itself. Because verify-quality is AI-executed prose (not an executable module) and both local and CI call the same behaviorally-tested helper, the check-set-per-tier claim is substantively verified via the shared code path. Disposition: Accepted by convention — skill is AI-executed prose; per-tier check-set parity verified via the shared, behaviorally-tested tier-resolve.sh helper + content conformance, per the gate-tooling ADL (.pair/adoption/tech/adl/2026-07-13-gate-tooling-code-in-tested-modules.md). No fixture-execution harness built for a prose skill, consistent with that convention.

Final verdict

All findings from the initial review and every subsequent re-review round are resolved or explicitly accepted/deferred with disposition above. The latest independent re-review of the current head (b97dff4) found zero actionable findings. Quality gates pass (knowledge-hub vitest 383/383, skills:conformance 39/39, mirror-equality guard #352 green, check:links clean, markdownlint clean on both SKILL.md surfaces, smoke tier-aware-gate.sh all PASS).

Review clean. Ready for the human merge decision.