From 5fe22080ef5dbab6437d064566c9759779fa6138 Mon Sep 17 00:00:00 2001 From: Gianluca Carucci Date: Fri, 24 Jul 2026 11:22:01 +0200 Subject: [PATCH 1/6] [#259] feat: verify-quality reads risk tag + gate matrix (local = CI) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - Step 1.5 resolves risk:* tag (PR, or story card pre-publish) via shipped tier-resolve.sh; runs the tier's check set (🟒 base; +unit 🟑; +integration/E2E πŸ”΄) - fail-safe red when untagged; missing suite => explicit "CI will fail"; matrix/KB absent => fall back to all adopted gates with notice; opt-in tiering preserved (disabled = full suite) - NO classification criteria (D18): reads tags + quality-model Β§4 only (single source), never the diff - conformance test (RED->GREEN) + tier-aware-gate smoke parity audit - docs: skills-catalog verify-quality row + tag-driven-gates local=CI section - regenerate .claude mirror Refs: #259 Co-Authored-By: Claude Opus 4.8 (1M context) --- .../pair-capability-verify-quality/SKILL.md | 129 ++++++++++++++---- .../docs/concepts/tag-driven-gates.mdx | 10 ++ .../content/docs/reference/skills-catalog.mdx | 2 +- .../capability/verify-quality/SKILL.md | 129 ++++++++++++++---- .../verify-quality-gate-matrix.test.ts | 125 +++++++++++++++++ .../smoke-tests/scenarios/tier-aware-gate.sh | 38 ++++++ 6 files changed, 378 insertions(+), 55 deletions(-) create mode 100644 packages/knowledge-hub/src/conformance/verify-quality-gate-matrix.test.ts diff --git a/.claude/skills/pair-capability-verify-quality/SKILL.md b/.claude/skills/pair-capability-verify-quality/SKILL.md index 88be28bb..f3aa3a55 100644 --- a/.claude/skills/pair-capability-verify-quality/SKILL.md +++ b/.claude/skills/pair-capability-verify-quality/SKILL.md @@ -1,18 +1,23 @@ --- name: pair-capability-verify-quality -description: "Checks whether the codebase currently passes its quality gates (lint, tests, type-check, coverage) β€” pass/fail per gate, skipping gates already passing. Composed by /pair-process-implement and /pair-process-review; invoke directly before pushing to check gates on demand." -version: 0.4.1 +description: "Checks whether the codebase passes the quality gates the CI pre-merge gate would run for the item/PR risk tier β€” resolves the `risk:*` tag, applies the KB gate matrix (🟒 lint+type+build; +unit from 🟑; +integration/E2E on πŸ”΄), and reports pass/fail per gate so local verification mirrors CI (no surprise reds at the gate, no over-checking on green work). Fail-safe red when untagged. Skips gates already passing. Composed by /pair-process-implement and /pair-process-review; invoke directly before pushing." +version: 0.5.0 author: Foomakers --- # /pair-capability-verify-quality β€” Quality Gate Checker -Validate the current codebase against quality gates. Two sources of truth: +Validate the current codebase against the quality gates **the CI pre-merge gate would run for this item/PR's risk tier** β€” so local verification mirrors CI (local = CI): no surprise reds at the gate, no over-checking on 🟒 green work. -- **[way-of-working.md](../../../.pair/adoption/tech/way-of-working.md)** β€” project-specific quality gate command and process (e.g., `pnpm quality-gate`). This is "what command we run." -- **[quality-standards](../../../.pair/knowledge/guidelines/quality-assurance/quality-standards/README.md)** β€” universal quality standards (gates, DoD, checklists). This is "what we check." +Three sources of truth: -Only check gates that are not already passing. +- **[way-of-working.md](../../../.pair/adoption/tech/way-of-working.md)** β€” project-specific quality gate command and process (e.g., `pnpm quality-gate`), plus the `Pre-merge tiering` flag. This is "what commands we run and how." +- **[quality-model.md Β§4](../../../.pair/knowledge/guidelines/quality-assurance/quality-model.md)** β€” the tierβ†’checks matrix, the **single source** of which checks each tier runs. This is "which gates for this tier." +- **[tier-aware-pipeline.md](../../../.pair/knowledge/guidelines/infrastructure/cicd-strategy/tier-aware-pipeline.md)** β€” the delivery-side wiring the CI gate is generated from; this skill reuses its shipped [`tier-resolve.sh`](../../../.pair/knowledge/assets/tier-resolve.sh) helper so local and CI resolve the tier and its suites through the **same** code. + +**verify-quality contains NO classification criteria: it reads the `risk:*` tag and the KB gate matrix (D18)** β€” the same single source the pipeline reads. It never inspects the diff, the code, file paths, or change size to decide a tier. The matrix decides WHICH gates run; adoption (`way-of-working.md`) decides HOW they run (the gate commands). Tags are produced upstream by `/pair-capability-classify` and carried on the story/PR. + +Only check gates that are **not already passing** (idempotency preserved). **Read-only, with one exception**: every gate only runs existing commands and modifies nothing β€” except **Step 5.C** (first-time custom-gate setup), the skill's ONLY write, which on a first run with no Custom Gate Registry writes the registry table or an opt-out marker to way-of-working.md (once). @@ -21,6 +26,9 @@ Only check gates that are not already passing. | Argument | Required | Description | | -------- | -------- | -------------------------------------------------------------------------------------------------------------- | | `$scope` | No | Limit checking: `code-quality`, `tests`, `lint`, `all`, or any custom scope key from adoption (default: `all`) | +| `$story` | No | Story/PR id whose `risk:*` tag drives the tier (see Step 1.5). If omitted, resolved from the branch's PR or the story card. | + +`$scope` and the resolved tier compose: the tier decides the widest set CI would run; `$scope` may narrow within it (e.g. `$scope=lint`). `$scope` never *widens* past the tier. ## Algorithm @@ -32,26 +40,84 @@ Execute each gate in order. For every gate, follow the **check β†’ skip β†’ act 2. **Skip**: If `way-of-working.md` has no Quality Gates section, fall back to `package.json` scripts for detectable gate commands (e.g., `test`, `lint`, `ts:check`). 3. **Act**: If found, record the command for use in Step 5. Also note any sub-checks listed (e.g., type checking, testing, linting, formatting). -### Step 2: Lint Gate +### Step 1.5: Resolve the Tier Gate Matrix (CI parity) + +This step decides **which** suites the standard gates below run, so the local run matches what CI would run for this item/PR. It reads tags + the matrix only β€” no classification. + +1. **Check β€” is tiering on?** Read the `Pre-merge tiering` flag in [way-of-working.md](../../../.pair/adoption/tech/way-of-working.md). + - **`disabled` (the default), or the flag/section is absent** β†’ **full suite**: the CI gate runs every suite on every PR, so mirror that β€” set the active suite set to **all adopted gates** (base + unit + integration + e2e + custom + aggregate, exactly the current behavior). Report `Tiering: disabled β€” running the full suite (CI parity)` and skip to Step 2. Do NOT read tags in this mode. + - **`enabled`** β†’ continue; the CI gate is tier-scoped, so scope locally by tier. + +2. **Act β€” resolve the tier from tags (never from the diff).** Load the shipped helper and resolve the `risk:*` tag: + + ```bash + source .pair/knowledge/assets/tier-resolve.sh # tags only, no criteria (D18) + # Prefer the PR's labels; pre-publish (no PR yet) fall back to the story card. + if PR_LABELS="$(gh pr view "$story" --json labels -q '.labels[].name' 2>/dev/null)" && [ -n "$PR_LABELS" ]; then + : # tags from the PR + else + PR_LABELS="$(gh issue view "$story" --json labels -q '.labels[].name' 2>/dev/null)" # from the story card + fi + TIER="$(resolve_tier "$PR_LABELS")" # green | yellow | red + ACTIVE_SUITES="$(required_suites_for_tier "$TIER")" # e.g. "install lint type build unit" + ``` + + - **Edge β€” tags on the story but not yet on the PR (pre-publish)**: resolve from the story card (`gh issue view`), as above. + - **Fail-safe (AC3)**: `resolve_tier` returns `red` for **no** `risk:*` tag or an **unknown/malformed** value β€” the widest matrix, never a silent skip. When the tier came from the fail-safe, say so explicitly in the report: `Tier: πŸ”΄ red (fail-safe β€” no resolvable risk:* tag; running the full set)`. + - **Widen-only**: because review never lowers a tier (D17), a later run can only widen the set versus an earlier one on the same item β€” never narrow. + +3. **Act β€” the tierβ†’checks matrix** (projection of [quality-model.md Β§4](../../../.pair/knowledge/guidelines/quality-assurance/quality-model.md); the executable copy is `required_suites_for_tier`, the single source β€” this skill does not restate an independent matrix): + + | Tier | Active check set (what CI would run) | + | --- | --- | + | 🟒 `risk:green` | base = install + lint + type + build | + | 🟑 `risk:yellow` | base + unit | + | πŸ”΄ `risk:red` | base + unit + integration + e2e | + + Base (install + lint + type + build) runs at **every** tier. Custom gates and the aggregate (Steps 5–6) still run per adoption regardless of tier (they are the project's "how", not tier suites). + +4. **Act β€” a required suite that is absent locally is an EXPLICIT failure, never a silent pass (AC / edge).** For each suite the tier requires, check presence the same layout-independent way CI does (`grep -q '"test:"' package.json`) and call `require_suite`: + + ```bash + require_suite unit "$(grep -q '"test:unit"' package.json && echo 1 || echo 0)" || SUITE_MISSING=1 + ``` + + If a required suite is missing, report `Suite missing: β€” CI will fail on this tier` and mark the verdict red. It is **not** a silent pass. + +5. **Act β€” matrix/KB not found (graceful degradation).** If `tier-resolve.sh` or the quality model cannot be read (e.g. the KB is absent), **fall back to running all adopted gates** (the current full-suite behavior) and print a notice: `Gate matrix not found β€” falling back to all adopted gates (could not confirm CI parity)`. Never silently skip. + +6. **Verify**: `TIER` and `ACTIVE_SUITES` are resolved (or the full-suite/fallback mode is recorded). The standard gates below run only the suites in `ACTIVE_SUITES` (further narrowed by `$scope` if given). + +### Step 2: Lint Gate (base β€” every tier) 1. **Check**: Run the project linter (e.g., `pnpm lint` or `turbo lint`). Capture output. 2. **Skip**: If zero violations, report "Lint: PASS" and move to Step 3. -3. **Act**: If violations found, report each violation with file and line. +3. **Act**: If violations found, report each violation with file and line β€” surface the failing command output verbatim. 4. **Verify**: After developer fixes, re-run linter to confirm zero violations. -### Step 3: Type Check Gate +### Step 3: Type Check + Build Gate (base β€” every tier) + +1. **Check**: Run the type checker and build (e.g., `pnpm ts:check` and `pnpm build`, or `turbo build`). Capture output. +2. **Skip**: If zero errors, report "Type Check + Build: PASS" and move to Step 4. +3. **Act**: If errors found, report each error with file and line β€” surface the failing command output verbatim. +4. **Verify**: After developer fixes, re-run to confirm zero errors. -1. **Check**: Run the type checker (e.g., `pnpm tsc --noEmit` or `turbo build`). Capture output. -2. **Skip**: If zero errors, report "Type Check: PASS" and move to Step 4. -3. **Act**: If errors found, report each error with file and line. -4. **Verify**: After developer fixes, re-run type checker to confirm zero errors. +### Step 4: Test Gate (tier-scoped) -### Step 4: Test Gate +Run only the test suites in `ACTIVE_SUITES` (Step 1.5) β€” this is where the tier changes the local run to match CI: -1. **Check**: Run the test suite (e.g., `pnpm test` or `turbo test`). Capture output including coverage. -2. **Skip**: If all tests pass, report "Tests: PASS (N tests, X% coverage)" and move to Step 5. -3. **Act**: If tests fail, report each failure with test name, file, and assertion message. -4. **Verify**: After developer fixes, re-run tests to confirm all pass. +- 🟒 green β†’ **no test suite** (base only). +- 🟑 yellow β†’ **unit** (`pnpm test:unit`, or the adopted unit command / aggregate `test`). +- πŸ”΄ red β†’ **unit + integration + E2E** (`pnpm test:unit`, `pnpm test:integration`, `pnpm test:e2e`). + +For each suite in `ACTIVE_SUITES`: + +1. **Check**: Run the suite. Capture output including coverage. +2. **Skip**: If it passes, report "Tests (``): PASS (N tests, X% coverage)". +3. **Act**: If it fails, report each failure with test name, file, and assertion message β€” **surface the failing command output** so the red verdict shows exactly what CI would block on (AC4). +4. **Verify**: After developer fixes, re-run the suite to confirm all pass. + +A suite the tier requires but the repo lacks was already flagged in Step 1.5 (`Suite missing β€” CI will fail`) and keeps the verdict red. ### Step 5: Custom Gates (from adoption) @@ -105,7 +171,7 @@ If a project-level quality gate command exists (from Step 1): 1. **Check**: Run the aggregate command (e.g., `pnpm quality-gate`). 2. **Skip**: If exit code 0, report "Quality Gate: PASS" and move to output. -3. **Act**: If non-zero exit, report the failing sub-gates. +3. **Act**: If non-zero exit, report the failing sub-gates β€” surface the failing command output. 4. **Verify**: After developer fixes, re-run to confirm pass. ## Output Format @@ -114,42 +180,51 @@ Present results as: ```text QUALITY GATE REPORT: +β”œβ”€β”€ Tier: [🟒 green | 🟑 yellow | πŸ”΄ red (fail-safe β€” reason) | disabled β€” full suite | matrix not found β€” fallback] +β”œβ”€β”€ Check set: [the suites this tier runs β€” e.g. install lint type build unit] (CI parity) β”œβ”€β”€ Lint: [PASS | FAIL β€” N violations] -β”œβ”€β”€ Type Check: [PASS | FAIL β€” N errors] -β”œβ”€β”€ Tests: [PASS β€” N tests, X% coverage | FAIL β€” N failures] +β”œβ”€β”€ Type+Build: [PASS | FAIL β€” N errors] +β”œβ”€β”€ Tests: [per active suite: PASS β€” N tests, X% coverage | FAIL β€” N failures | SKIPPED (tier) | MISSING β€” CI will fail] β”œβ”€β”€ Custom: [N gates β€” N PASS, N FAIL, N WARNING | No custom gates] └── Aggregate: [PASS | FAIL | N/A] RESULT: [ALL GATES PASS | BLOCKED β€” N gates failing] ``` +A red verdict always surfaces the failing command output (AC4) so it matches exactly what CI would block on. + ## Composition Interface When composed by `/pair-process-implement` or `/pair-process-review`: -- **Input**: The composing skill invokes `/pair-capability-verify-quality` after implementation or before commit. -- **Output**: Returns PASS or FAIL with details. The composing skill decides what to do: +- **Input**: The composing skill invokes `/pair-capability-verify-quality` after implementation or before commit. It may pass `$story` so the tier resolves from the right item; if omitted, the tier is resolved from the branch's PR or story card. +- **Output**: Returns PASS or FAIL with details (and the resolved tier / check set). The composing skill decides what to do: - `/pair-process-implement`: HALT on FAIL β€” developer must fix before commit. - `/pair-process-review`: Report FAIL as review finding β€” does not block review completion. +The composition contract is **unchanged** for callers (same PASS/FAIL result); the tier resolution is internal to this skill. + When invoked **independently**: -- Run all gates (or scoped gates if `$scope` is provided). +- Resolve the tier (Step 1.5), then run the tier's check set (or scoped gates if `$scope` is provided). - Report results. No side effects β€” this skill only reads and reports. ## Graceful Degradation See [graceful degradation](../../../.pair/knowledge/guidelines/technical-standards/ai-development/skill-conventions/graceful-degradation.md) (guideline missing β†’ run only what's detectable rather than failing) for the standard scenarios. Additional cases: -- If a standard gate command is not available (e.g., no test script in package.json), skip that gate and report: "Tests: SKIPPED β€” no test command found." +- **Gate matrix / `tier-resolve.sh` / quality model not found**: fall back to running all adopted gates (the full-suite behavior) with the notice in Step 1.5 β€” never a silent skip. +- **`gh` / PM tool not available to read tags**: cannot resolve a tag β†’ apply the fail-safe (πŸ”΄ red, full set) and note that tags could not be read. Never assume green. +- If a standard gate command is not available (e.g., no test script in package.json), skip that gate and report: "Tests: SKIPPED β€” no test command found." (If the tier *required* that suite, it is instead a `Suite missing β€” CI will fail`, per Step 1.5.) - If no quality-related scripts are found at all, report: "No quality gates detected. Configure quality gate commands in package.json or way-of-working.md." - If a custom gate command fails to execute (command not found), report as WARNING: "Gate `[name]`: SKIPPED β€” command not found." -- If [way-of-working.md](../../../.pair/adoption/tech/way-of-working.md) is not found, skip custom gates entirely (standard gates still run). +- If [way-of-working.md](../../../.pair/adoption/tech/way-of-working.md) is not found, skip custom gates entirely (standard gates still run) and treat tiering as disabled (full suite). ## Notes - The read-only-except-Step-5.C rule is stated once, in the overview above β€” this Notes section doesn't restate it. -- Standard gates (Lint, Type Check, Test) are universal and language/platform-independent. Custom gates are project-specific and defined in adoption (see the two sources of truth above). +- **Local = CI**: the tier and its suites are resolved through the *same* [`tier-resolve.sh`](../../../.pair/knowledge/assets/tier-resolve.sh) helper the CI pre-merge gate uses, so the check set and verdict match. This skill owns no matrix of its own (D18) β€” [quality-model.md Β§4](../../../.pair/knowledge/guidelines/quality-assurance/quality-model.md) is the single source. +- Standard gates (Lint, Type+Build, Test) are universal and language/platform-independent. Custom gates are project-specific and defined in adoption (see the sources of truth above). - Each gate is independent β€” a failure in one gate does not prevent checking subsequent gates. - Re-invoke after fixes to confirm resolution. Already-passing gates are re-verified but complete instantly. - First-time setup (Step 5.C) only triggers once β€” after the developer responds, way-of-working.md is updated and subsequent invocations follow branch A or B. diff --git a/apps/website/content/docs/concepts/tag-driven-gates.mdx b/apps/website/content/docs/concepts/tag-driven-gates.mdx index 7068a97a..1d93ea25 100644 --- a/apps/website/content/docs/concepts/tag-driven-gates.mdx +++ b/apps/website/content/docs/concepts/tag-driven-gates.mdx @@ -36,6 +36,16 @@ If a PR has **no** `risk:*` tag β€” or the tag is unknown or malformed β€” the g If a tier requires a suite the repository does not have (for example, a πŸ”΄ PR with no E2E suite), the gate **fails with an explicit "suite missing" error**. It never silently passes a tier whose checks could not run. +## Checking locally before you push (local = CI) + +The `verify-quality` capability runs the **same check set the gate would run for this item's tier**, on your machine, before you push. It resolves the `risk:*` tag β€” from the pull request, or pre-publish from the story card β€” and applies the exact same matrix through the same shipped resolver the CI gate uses. So a 🟒 green change runs only lint + type + build locally (no over-checking), and a πŸ”΄ red change runs the full set locally (no surprise red at the gate). + +It never classifies: like the pipeline, it reads the tag and the matrix only. The three parity guarantees carry over unchanged: + +- **Fail-safe red** β€” no resolvable `risk:*` tag β‡’ the full set runs locally too, stated explicitly in the report. +- **Missing suite is loud** β€” a suite the tier requires but the repo lacks is reported as `suite missing β€” CI will fail`, never a silent local pass. +- **Opt-in preserved** β€” with tiering off (the default), `verify-quality` runs the full suite, exactly as CI does. + ## Coverage guardrail (opt-in, off by default) Coverage can be protected by a **baseline regression guardrail** β€” a job inside the same pipeline, not a separate CI mechanism. It is **opt-in and off by default** (the twin of tiering): a project turns it on by setting `Coverage guardrail: enabled` in its `way-of-working.md`, and `setup-gates` offers it as a guided-setup question (default **No**). Absent the flag, no coverage job is generated. When enabled, a PR that drops coverage **below a committed baseline** fails the gate; a PR that maintains or improves it passes. It is a guardrail against regression, not a wall demanding a fixed percentage on every PR, so a legacy codebase is never failed at 0 on day one. diff --git a/apps/website/content/docs/reference/skills-catalog.mdx b/apps/website/content/docs/reference/skills-catalog.mdx index ed35472e..a561eb79 100644 --- a/apps/website/content/docs/reference/skills-catalog.mdx +++ b/apps/website/content/docs/reference/skills-catalog.mdx @@ -82,7 +82,7 @@ All `analyze-*` skills are **output-only report producers**: they analyze and re | ----- | ------- | ----------- | | **verify-adoption** | `/pair-capability-verify-adoption` | Detects conformity/non-conformity of already-written code against the project's adopted decisions, for a given scope β€” a compliance audit, not a new-choice evaluation (that's assess-stack and siblings). Detection-only. | | **verify-done** | `/pair-capability-verify-done` | Checks a PR or work item against Definition of Done criteria, skipping already-passing criteria. Composed by `/pair-process-review`; invoke directly on demand. | -| **verify-quality** | `/pair-capability-verify-quality` | Checks whether the codebase currently passes its quality gates (lint, tests, type-check, coverage), skipping gates already passing. Composed by `/pair-process-implement`, `/pair-process-review`. | +| **verify-quality** | `/pair-capability-verify-quality` | Checks whether the codebase passes the quality gates the CI pre-merge gate would run for the item/PR risk tier β€” resolves the `risk:*` tag and applies the [tag-driven gate matrix](/docs/concepts/tag-driven-gates) (🟒 lint+type+build; +unit from 🟑; +integration/E2E on πŸ”΄) so local verification mirrors CI, fail-safe red when untagged. Skips gates already passing. Composed by `/pair-process-implement`, `/pair-process-review`. | ### Setup and Management Capabilities diff --git a/packages/knowledge-hub/dataset/.skills/capability/verify-quality/SKILL.md b/packages/knowledge-hub/dataset/.skills/capability/verify-quality/SKILL.md index c15c9bdf..f050232e 100644 --- a/packages/knowledge-hub/dataset/.skills/capability/verify-quality/SKILL.md +++ b/packages/knowledge-hub/dataset/.skills/capability/verify-quality/SKILL.md @@ -1,18 +1,23 @@ --- name: verify-quality -description: "Checks whether the codebase currently passes its quality gates (lint, tests, type-check, coverage) β€” pass/fail per gate, skipping gates already passing. Composed by /implement and /review; invoke directly before pushing to check gates on demand." -version: 0.4.1 +description: "Checks whether the codebase passes the quality gates the CI pre-merge gate would run for the item/PR risk tier β€” resolves the `risk:*` tag, applies the KB gate matrix (🟒 lint+type+build; +unit from 🟑; +integration/E2E on πŸ”΄), and reports pass/fail per gate so local verification mirrors CI (no surprise reds at the gate, no over-checking on green work). Fail-safe red when untagged. Skips gates already passing. Composed by /implement and /review; invoke directly before pushing." +version: 0.5.0 author: Foomakers --- # /verify-quality β€” Quality Gate Checker -Validate the current codebase against quality gates. Two sources of truth: +Validate the current codebase against the quality gates **the CI pre-merge gate would run for this item/PR's risk tier** β€” so local verification mirrors CI (local = CI): no surprise reds at the gate, no over-checking on 🟒 green work. -- **[way-of-working.md](../../../.pair/adoption/tech/way-of-working.md)** β€” project-specific quality gate command and process (e.g., `pnpm quality-gate`). This is "what command we run." -- **[quality-standards](../../../.pair/knowledge/guidelines/quality-assurance/quality-standards/README.md)** β€” universal quality standards (gates, DoD, checklists). This is "what we check." +Three sources of truth: -Only check gates that are not already passing. +- **[way-of-working.md](../../../.pair/adoption/tech/way-of-working.md)** β€” project-specific quality gate command and process (e.g., `pnpm quality-gate`), plus the `Pre-merge tiering` flag. This is "what commands we run and how." +- **[quality-model.md Β§4](../../../.pair/knowledge/guidelines/quality-assurance/quality-model.md)** β€” the tierβ†’checks matrix, the **single source** of which checks each tier runs. This is "which gates for this tier." +- **[tier-aware-pipeline.md](../../../.pair/knowledge/guidelines/infrastructure/cicd-strategy/tier-aware-pipeline.md)** β€” the delivery-side wiring the CI gate is generated from; this skill reuses its shipped [`tier-resolve.sh`](../../../.pair/knowledge/assets/tier-resolve.sh) helper so local and CI resolve the tier and its suites through the **same** code. + +**verify-quality contains NO classification criteria: it reads the `risk:*` tag and the KB gate matrix (D18)** β€” the same single source the pipeline reads. It never inspects the diff, the code, file paths, or change size to decide a tier. The matrix decides WHICH gates run; adoption (`way-of-working.md`) decides HOW they run (the gate commands). Tags are produced upstream by `/classify` and carried on the story/PR. + +Only check gates that are **not already passing** (idempotency preserved). **Read-only, with one exception**: every gate only runs existing commands and modifies nothing β€” except **Step 5.C** (first-time custom-gate setup), the skill's ONLY write, which on a first run with no Custom Gate Registry writes the registry table or an opt-out marker to way-of-working.md (once). @@ -21,6 +26,9 @@ Only check gates that are not already passing. | Argument | Required | Description | | -------- | -------- | -------------------------------------------------------------------------------------------------------------- | | `$scope` | No | Limit checking: `code-quality`, `tests`, `lint`, `all`, or any custom scope key from adoption (default: `all`) | +| `$story` | No | Story/PR id whose `risk:*` tag drives the tier (see Step 1.5). If omitted, resolved from the branch's PR or the story card. | + +`$scope` and the resolved tier compose: the tier decides the widest set CI would run; `$scope` may narrow within it (e.g. `$scope=lint`). `$scope` never *widens* past the tier. ## Algorithm @@ -32,26 +40,84 @@ Execute each gate in order. For every gate, follow the **check β†’ skip β†’ act 2. **Skip**: If `way-of-working.md` has no Quality Gates section, fall back to `package.json` scripts for detectable gate commands (e.g., `test`, `lint`, `ts:check`). 3. **Act**: If found, record the command for use in Step 5. Also note any sub-checks listed (e.g., type checking, testing, linting, formatting). -### Step 2: Lint Gate +### Step 1.5: Resolve the Tier Gate Matrix (CI parity) + +This step decides **which** suites the standard gates below run, so the local run matches what CI would run for this item/PR. It reads tags + the matrix only β€” no classification. + +1. **Check β€” is tiering on?** Read the `Pre-merge tiering` flag in [way-of-working.md](../../../.pair/adoption/tech/way-of-working.md). + - **`disabled` (the default), or the flag/section is absent** β†’ **full suite**: the CI gate runs every suite on every PR, so mirror that β€” set the active suite set to **all adopted gates** (base + unit + integration + e2e + custom + aggregate, exactly the current behavior). Report `Tiering: disabled β€” running the full suite (CI parity)` and skip to Step 2. Do NOT read tags in this mode. + - **`enabled`** β†’ continue; the CI gate is tier-scoped, so scope locally by tier. + +2. **Act β€” resolve the tier from tags (never from the diff).** Load the shipped helper and resolve the `risk:*` tag: + + ```bash + source .pair/knowledge/assets/tier-resolve.sh # tags only, no criteria (D18) + # Prefer the PR's labels; pre-publish (no PR yet) fall back to the story card. + if PR_LABELS="$(gh pr view "$story" --json labels -q '.labels[].name' 2>/dev/null)" && [ -n "$PR_LABELS" ]; then + : # tags from the PR + else + PR_LABELS="$(gh issue view "$story" --json labels -q '.labels[].name' 2>/dev/null)" # from the story card + fi + TIER="$(resolve_tier "$PR_LABELS")" # green | yellow | red + ACTIVE_SUITES="$(required_suites_for_tier "$TIER")" # e.g. "install lint type build unit" + ``` + + - **Edge β€” tags on the story but not yet on the PR (pre-publish)**: resolve from the story card (`gh issue view`), as above. + - **Fail-safe (AC3)**: `resolve_tier` returns `red` for **no** `risk:*` tag or an **unknown/malformed** value β€” the widest matrix, never a silent skip. When the tier came from the fail-safe, say so explicitly in the report: `Tier: πŸ”΄ red (fail-safe β€” no resolvable risk:* tag; running the full set)`. + - **Widen-only**: because review never lowers a tier (D17), a later run can only widen the set versus an earlier one on the same item β€” never narrow. + +3. **Act β€” the tierβ†’checks matrix** (projection of [quality-model.md Β§4](../../../.pair/knowledge/guidelines/quality-assurance/quality-model.md); the executable copy is `required_suites_for_tier`, the single source β€” this skill does not restate an independent matrix): + + | Tier | Active check set (what CI would run) | + | --- | --- | + | 🟒 `risk:green` | base = install + lint + type + build | + | 🟑 `risk:yellow` | base + unit | + | πŸ”΄ `risk:red` | base + unit + integration + e2e | + + Base (install + lint + type + build) runs at **every** tier. Custom gates and the aggregate (Steps 5–6) still run per adoption regardless of tier (they are the project's "how", not tier suites). + +4. **Act β€” a required suite that is absent locally is an EXPLICIT failure, never a silent pass (AC / edge).** For each suite the tier requires, check presence the same layout-independent way CI does (`grep -q '"test:"' package.json`) and call `require_suite`: + + ```bash + require_suite unit "$(grep -q '"test:unit"' package.json && echo 1 || echo 0)" || SUITE_MISSING=1 + ``` + + If a required suite is missing, report `Suite missing: β€” CI will fail on this tier` and mark the verdict red. It is **not** a silent pass. + +5. **Act β€” matrix/KB not found (graceful degradation).** If `tier-resolve.sh` or the quality model cannot be read (e.g. the KB is absent), **fall back to running all adopted gates** (the current full-suite behavior) and print a notice: `Gate matrix not found β€” falling back to all adopted gates (could not confirm CI parity)`. Never silently skip. + +6. **Verify**: `TIER` and `ACTIVE_SUITES` are resolved (or the full-suite/fallback mode is recorded). The standard gates below run only the suites in `ACTIVE_SUITES` (further narrowed by `$scope` if given). + +### Step 2: Lint Gate (base β€” every tier) 1. **Check**: Run the project linter (e.g., `pnpm lint` or `turbo lint`). Capture output. 2. **Skip**: If zero violations, report "Lint: PASS" and move to Step 3. -3. **Act**: If violations found, report each violation with file and line. +3. **Act**: If violations found, report each violation with file and line β€” surface the failing command output verbatim. 4. **Verify**: After developer fixes, re-run linter to confirm zero violations. -### Step 3: Type Check Gate +### Step 3: Type Check + Build Gate (base β€” every tier) + +1. **Check**: Run the type checker and build (e.g., `pnpm ts:check` and `pnpm build`, or `turbo build`). Capture output. +2. **Skip**: If zero errors, report "Type Check + Build: PASS" and move to Step 4. +3. **Act**: If errors found, report each error with file and line β€” surface the failing command output verbatim. +4. **Verify**: After developer fixes, re-run to confirm zero errors. -1. **Check**: Run the type checker (e.g., `pnpm tsc --noEmit` or `turbo build`). Capture output. -2. **Skip**: If zero errors, report "Type Check: PASS" and move to Step 4. -3. **Act**: If errors found, report each error with file and line. -4. **Verify**: After developer fixes, re-run type checker to confirm zero errors. +### Step 4: Test Gate (tier-scoped) -### Step 4: Test Gate +Run only the test suites in `ACTIVE_SUITES` (Step 1.5) β€” this is where the tier changes the local run to match CI: -1. **Check**: Run the test suite (e.g., `pnpm test` or `turbo test`). Capture output including coverage. -2. **Skip**: If all tests pass, report "Tests: PASS (N tests, X% coverage)" and move to Step 5. -3. **Act**: If tests fail, report each failure with test name, file, and assertion message. -4. **Verify**: After developer fixes, re-run tests to confirm all pass. +- 🟒 green β†’ **no test suite** (base only). +- 🟑 yellow β†’ **unit** (`pnpm test:unit`, or the adopted unit command / aggregate `test`). +- πŸ”΄ red β†’ **unit + integration + E2E** (`pnpm test:unit`, `pnpm test:integration`, `pnpm test:e2e`). + +For each suite in `ACTIVE_SUITES`: + +1. **Check**: Run the suite. Capture output including coverage. +2. **Skip**: If it passes, report "Tests (``): PASS (N tests, X% coverage)". +3. **Act**: If it fails, report each failure with test name, file, and assertion message β€” **surface the failing command output** so the red verdict shows exactly what CI would block on (AC4). +4. **Verify**: After developer fixes, re-run the suite to confirm all pass. + +A suite the tier requires but the repo lacks was already flagged in Step 1.5 (`Suite missing β€” CI will fail`) and keeps the verdict red. ### Step 5: Custom Gates (from adoption) @@ -105,7 +171,7 @@ If a project-level quality gate command exists (from Step 1): 1. **Check**: Run the aggregate command (e.g., `pnpm quality-gate`). 2. **Skip**: If exit code 0, report "Quality Gate: PASS" and move to output. -3. **Act**: If non-zero exit, report the failing sub-gates. +3. **Act**: If non-zero exit, report the failing sub-gates β€” surface the failing command output. 4. **Verify**: After developer fixes, re-run to confirm pass. ## Output Format @@ -114,42 +180,51 @@ Present results as: ```text QUALITY GATE REPORT: +β”œβ”€β”€ Tier: [🟒 green | 🟑 yellow | πŸ”΄ red (fail-safe β€” reason) | disabled β€” full suite | matrix not found β€” fallback] +β”œβ”€β”€ Check set: [the suites this tier runs β€” e.g. install lint type build unit] (CI parity) β”œβ”€β”€ Lint: [PASS | FAIL β€” N violations] -β”œβ”€β”€ Type Check: [PASS | FAIL β€” N errors] -β”œβ”€β”€ Tests: [PASS β€” N tests, X% coverage | FAIL β€” N failures] +β”œβ”€β”€ Type+Build: [PASS | FAIL β€” N errors] +β”œβ”€β”€ Tests: [per active suite: PASS β€” N tests, X% coverage | FAIL β€” N failures | SKIPPED (tier) | MISSING β€” CI will fail] β”œβ”€β”€ Custom: [N gates β€” N PASS, N FAIL, N WARNING | No custom gates] └── Aggregate: [PASS | FAIL | N/A] RESULT: [ALL GATES PASS | BLOCKED β€” N gates failing] ``` +A red verdict always surfaces the failing command output (AC4) so it matches exactly what CI would block on. + ## Composition Interface When composed by `/implement` or `/review`: -- **Input**: The composing skill invokes `/verify-quality` after implementation or before commit. -- **Output**: Returns PASS or FAIL with details. The composing skill decides what to do: +- **Input**: The composing skill invokes `/verify-quality` after implementation or before commit. It may pass `$story` so the tier resolves from the right item; if omitted, the tier is resolved from the branch's PR or story card. +- **Output**: Returns PASS or FAIL with details (and the resolved tier / check set). The composing skill decides what to do: - `/implement`: HALT on FAIL β€” developer must fix before commit. - `/review`: Report FAIL as review finding β€” does not block review completion. +The composition contract is **unchanged** for callers (same PASS/FAIL result); the tier resolution is internal to this skill. + When invoked **independently**: -- Run all gates (or scoped gates if `$scope` is provided). +- Resolve the tier (Step 1.5), then run the tier's check set (or scoped gates if `$scope` is provided). - Report results. No side effects β€” this skill only reads and reports. ## Graceful Degradation See [graceful degradation](../../../.pair/knowledge/guidelines/technical-standards/ai-development/skill-conventions/graceful-degradation.md) (guideline missing β†’ run only what's detectable rather than failing) for the standard scenarios. Additional cases: -- If a standard gate command is not available (e.g., no test script in package.json), skip that gate and report: "Tests: SKIPPED β€” no test command found." +- **Gate matrix / `tier-resolve.sh` / quality model not found**: fall back to running all adopted gates (the full-suite behavior) with the notice in Step 1.5 β€” never a silent skip. +- **`gh` / PM tool not available to read tags**: cannot resolve a tag β†’ apply the fail-safe (πŸ”΄ red, full set) and note that tags could not be read. Never assume green. +- If a standard gate command is not available (e.g., no test script in package.json), skip that gate and report: "Tests: SKIPPED β€” no test command found." (If the tier *required* that suite, it is instead a `Suite missing β€” CI will fail`, per Step 1.5.) - If no quality-related scripts are found at all, report: "No quality gates detected. Configure quality gate commands in package.json or way-of-working.md." - If a custom gate command fails to execute (command not found), report as WARNING: "Gate `[name]`: SKIPPED β€” command not found." -- If [way-of-working.md](../../../.pair/adoption/tech/way-of-working.md) is not found, skip custom gates entirely (standard gates still run). +- If [way-of-working.md](../../../.pair/adoption/tech/way-of-working.md) is not found, skip custom gates entirely (standard gates still run) and treat tiering as disabled (full suite). ## Notes - The read-only-except-Step-5.C rule is stated once, in the overview above β€” this Notes section doesn't restate it. -- Standard gates (Lint, Type Check, Test) are universal and language/platform-independent. Custom gates are project-specific and defined in adoption (see the two sources of truth above). +- **Local = CI**: the tier and its suites are resolved through the *same* [`tier-resolve.sh`](../../../.pair/knowledge/assets/tier-resolve.sh) helper the CI pre-merge gate uses, so the check set and verdict match. This skill owns no matrix of its own (D18) β€” [quality-model.md Β§4](../../../.pair/knowledge/guidelines/quality-assurance/quality-model.md) is the single source. +- Standard gates (Lint, Type+Build, Test) are universal and language/platform-independent. Custom gates are project-specific and defined in adoption (see the sources of truth above). - Each gate is independent β€” a failure in one gate does not prevent checking subsequent gates. - Re-invoke after fixes to confirm resolution. Already-passing gates are re-verified but complete instantly. - First-time setup (Step 5.C) only triggers once β€” after the developer responds, way-of-working.md is updated and subsequent invocations follow branch A or B. diff --git a/packages/knowledge-hub/src/conformance/verify-quality-gate-matrix.test.ts b/packages/knowledge-hub/src/conformance/verify-quality-gate-matrix.test.ts new file mode 100644 index 00000000..cbdbfa05 --- /dev/null +++ b/packages/knowledge-hub/src/conformance/verify-quality-gate-matrix.test.ts @@ -0,0 +1,125 @@ +import { describe, it, expect } from 'vitest' +import { readFileSync, existsSync } from 'fs' +import { dirname, join, resolve } from 'path' + +// Story #259 β€” verify-quality integrated with the tier gate matrix (local = CI). +// The skill must resolve the item/PR classification tags and run locally exactly +// the checks the CI gate would run for that tier β€” reading TAGS + the KB gate +// matrix only (D18), never classifying itself. These are content invariants on +// the source-of-record SKILL.md (dataset), asserted the same way the rest of the +// KB/skill corpus is tested (see quality-model.test.ts). The executable parity +// (verify-quality resolving suites through the *same* tier-resolve.sh helper CI +// uses) is additionally smoke-tested in scripts/smoke-tests/scenarios/tier-aware-gate.sh +// per the gate-tooling ADL (scripts are smoke-tested, not vitest-unit-tested). + +const SKILL_PATH = join(__dirname, '../../dataset/.skills/capability/verify-quality/SKILL.md') +const SKILL = readFileSync(SKILL_PATH, 'utf-8') + +const RESOLVER = readFileSync( + join(__dirname, '../../dataset/.pair/knowledge/assets/tier-resolve.sh'), + 'utf-8', +) + +describe('verify-quality SKILL.md β€” tier gate matrix integration (#259)', () => { + it('frames itself as mirroring the CI gate for the resolved tier (local = CI)', () => { + expect(SKILL).toMatch(/gate matrix/i) + expect(SKILL).toMatch(/local[^\n]*\bCI\b|mirror[^\n]*\bCI\b|\bCI\b[^\n]*parity/i) + }) + + it('AC1 β€” 🟒 green runs base only (install + lint + type + build), same set CI would run', () => { + expect(SKILL).toMatch(/risk:green/) + // base spelled out as the green check set + expect(SKILL).toMatch(/install[^\n]*lint[^\n]*type[^\n]*build/i) + }) + + it('AC2 β€” the check set widens per tier: +unit from 🟑, +integration/E2E on πŸ”΄', () => { + expect(SKILL).toMatch(/risk:yellow/) + expect(SKILL).toMatch(/risk:red/) + expect(SKILL).toMatch(/\bunit\b/) + expect(SKILL).toMatch(/integration/i) + expect(SKILL).toMatch(/e2e/i) + }) + + it('AC3 β€” fail-safe: no resolvable tags β‡’ full πŸ”΄ set, stated explicitly in the report', () => { + expect(SKILL).toMatch(/fail-safe/i) + expect(SKILL).toMatch(/red/i) + // explicit report line, not a silent widening + expect(SKILL).toMatch(/no[^\n]*tag[\s\S]{0,120}(red|full)/i) + }) + + it('AC4 β€” any failing check β‡’ red verdict with the failing command output surfaced', () => { + expect(SKILL).toMatch(/failing command output|surface[^\n]*output|command output/i) + }) + + it('reads the risk:* tag from the PR, and pre-publish from the story card (never classifies)', () => { + expect(SKILL).toMatch(/gh pr view/) + expect(SKILL).toMatch(/story card|from the story/i) + }) + + it('contains NO classification criteria β€” reads tags + the KB matrix only (D18)', () => { + expect(SKILL).toMatch(/no classification criteria|contains no[^\n]*criteria/i) + expect(SKILL).toMatch(/D18/) + }) + + it('delegates tier + suite resolution to the shipped tier-resolve.sh helper (single source, not re-implemented)', () => { + expect(SKILL).toContain('tier-resolve.sh') + expect(SKILL).toContain('resolve_tier') + expect(SKILL).toContain('required_suites_for_tier') + expect(SKILL).toContain('require_suite') + }) + + it('preserves the opt-in tiering flag: tiering disabled β‡’ full suite (CI parity, current behavior)', () => { + expect(SKILL).toContain('Pre-merge tiering') + expect(SKILL).toMatch(/disabled[\s\S]{0,160}full suite|full suite[\s\S]{0,160}disabled/i) + }) + + it('a suite required by the tier but absent locally β‡’ explicit "suite missing β€” CI will fail", never a silent pass', () => { + expect(SKILL).toMatch(/suite missing|missing suite/i) + expect(SKILL).toMatch(/CI will fail/i) + expect(SKILL).toMatch(/not[^\n]*silent|never[^\n]*silent/i) + }) + + it('matrix/KB not found β‡’ falls back to all adopted gates with a notice', () => { + expect(SKILL).toMatch(/fall[- ]?back[\s\S]{0,160}(all[^\n]*gates|adopted gates)/i) + expect(SKILL).toMatch(/notice/i) + }) + + it('preserves idempotency (already-passing gates skipped) and adoption command overrides', () => { + expect(SKILL).toMatch(/already[- ]passing|not already passing|skip/i) + expect(SKILL).toMatch(/way-of-working/) + }) + + it('links to the matrix single source (quality-model Β§4) and the delivery wiring (tier-aware-pipeline)', () => { + const dir = dirname(SKILL_PATH) + const targets = ['quality-model.md', 'tier-aware-pipeline.md', 'tier-resolve.sh'] + for (const t of targets) { + const m = SKILL.match(new RegExp(`\\]\\(([^)]*${t.replace('.', '\\.')})\\)`)) + expect(m, `SKILL.md must link to ${t}`).not.toBeNull() + const linkPath = (m as RegExpMatchArray)[1] as string + expect( + existsSync(resolve(dir, linkPath.split('#')[0] as string)), + `link to ${t} must resolve`, + ).toBe(true) + } + }) +}) + +describe('verify-quality β€” matrix parity with tier-resolve.sh (the executable source)', () => { + // The skill must not restate an independent matrix. The per-tier suite keys the + // helper exposes are the single source; the skill documents the increments and + // defers the mapping to required_suites_for_tier. Parity guard: the suite keys + // named in the helper are exactly the ones the skill's tier prose references. + const helperKeys = ['install', 'lint', 'type', 'build', 'unit', 'integration', 'e2e'] + + it('the helper exposes the four required-suite entry points the skill composes', () => { + for (const fn of ['resolve_tier', 'required_suites_for_tier', 'require_suite']) { + expect(RESOLVER).toContain(fn) + } + }) + + it('every suite key in the helper matrix is referenced by the skill (no drift)', () => { + for (const key of helperKeys) { + expect(SKILL.toLowerCase()).toContain(key) + } + }) +}) diff --git a/scripts/smoke-tests/scenarios/tier-aware-gate.sh b/scripts/smoke-tests/scenarios/tier-aware-gate.sh index 0455645a..18cbc028 100755 --- a/scripts/smoke-tests/scenarios/tier-aware-gate.sh +++ b/scripts/smoke-tests/scenarios/tier-aware-gate.sh @@ -21,6 +21,7 @@ REPO_ROOT="${REPO_ROOT:-$(cd "$(dirname "$0")/../../.." && pwd)}" RESOLVER="$REPO_ROOT/packages/knowledge-hub/dataset/.pair/knowledge/assets/tier-resolve.sh" GUIDELINE="$REPO_ROOT/packages/knowledge-hub/dataset/.pair/knowledge/guidelines/infrastructure/cicd-strategy/tier-aware-pipeline.md" SETUP_GATES="$REPO_ROOT/packages/knowledge-hub/dataset/.skills/capability/setup-gates/SKILL.md" +VERIFY_QUALITY="$REPO_ROOT/packages/knowledge-hub/dataset/.skills/capability/verify-quality/SKILL.md" FAILED=0 check() { # check @@ -30,6 +31,7 @@ check() { # check assert_file "$RESOLVER" || exit 1 assert_file "$GUIDELINE" || exit 1 assert_file "$SETUP_GATES" || exit 1 +assert_file "$VERIFY_QUALITY" || exit 1 # --- Redesign invariant: tier reduction is OPT-IN; default = full checks on every PR --- # (ADL 2026-07-20). The guideline must frame itself as an opt-in optimization and @@ -113,6 +115,42 @@ else log_fail "secret-scan job has an if: (tier-conditioned)"; FAILED=1 fi +# --- verify-quality local = CI parity (story #259) --- +# verify-quality must resolve the tier and its suites through the SAME shipped +# helper the CI gate uses (single source, D18) β€” so local mirrors CI. Audit the +# skill for the parity wiring, not a re-implemented matrix. +if grep -q 'tier-resolve.sh' "$VERIFY_QUALITY" \ + && grep -q 'resolve_tier' "$VERIFY_QUALITY" \ + && grep -q 'required_suites_for_tier' "$VERIFY_QUALITY" \ + && grep -q 'require_suite' "$VERIFY_QUALITY"; then + log_succ "verify-quality resolves tier+suites via the shipped tier-resolve.sh helper (local = CI)" +else + log_fail "verify-quality does not reuse tier-resolve.sh (would drift from CI)"; FAILED=1 +fi + +# opt-in tiering preserved: tiering disabled => full suite (current behavior / CI parity) +if grep -q 'Pre-merge tiering' "$VERIFY_QUALITY" && grep -qi 'full suite' "$VERIFY_QUALITY"; then + log_succ "verify-quality preserves the opt-in flag (disabled => full suite)" +else + log_fail "verify-quality lost the Pre-merge tiering opt-in / full-suite parity"; FAILED=1 +fi + +# fail-safe red + explicit missing-suite failure (never a silent pass) +if grep -qi 'fail-safe' "$VERIFY_QUALITY" \ + && grep -qi 'suite missing' "$VERIFY_QUALITY" \ + && grep -q 'CI will fail' "$VERIFY_QUALITY"; then + log_succ "verify-quality carries the fail-safe red + explicit missing-suite failure" +else + log_fail "verify-quality missing the fail-safe / missing-suite parity guarantees"; FAILED=1 +fi + +# single source: defers to quality-model Β§4, does not own an independent matrix (D18) +if grep -q 'quality-model.md' "$VERIFY_QUALITY" && grep -q 'D18' "$VERIFY_QUALITY"; then + log_succ "verify-quality defers to quality-model Β§4 as the single matrix source (D18)" +else + log_fail "verify-quality does not cite the single matrix source (D18)"; FAILED=1 +fi + if [ "$FAILED" -ne 0 ]; then log_fail "$TEST_NAME had failures"; exit 1 fi From 572965b4cb8569a2da8e3315a6a3d3c1e6646272 Mon Sep 17 00:00:00 2001 From: Gianluca Carucci Date: Fri, 24 Jul 2026 12:14:26 +0200 Subject: [PATCH 2/6] =?UTF-8?q?[#259]=20fix:=20review=20round=201=20?= =?UTF-8?q?=E2=80=94=20current-branch=20PR=20tier=20resolution,=20coverage?= =?UTF-8?q?-guardrail=20caveat,=20AC4=20assertion?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - Step 1.5: primary tag source = current-branch PR (gh pr view, no arg); story-card fallback only pre-publish. Fixes over-checking on empty $story and under-running a review-raised PR tag. - $story arg row + pre-publish edge note clarified (story-id-only semantics). - Notes: coverage guardrail is not a tier suite / not mirrored locally caveat. - AC4 conformance test pins red-verdict co-occurring with failing-command-output. - Root mirror synced (#352 guard). Co-Authored-By: Claude Opus 4.8 (1M context) --- .../pair-capability-verify-quality/SKILL.md | 21 +++++++++++-------- .../capability/verify-quality/SKILL.md | 21 +++++++++++-------- .../verify-quality-gate-matrix.test.ts | 6 ++++++ 3 files changed, 30 insertions(+), 18 deletions(-) diff --git a/.claude/skills/pair-capability-verify-quality/SKILL.md b/.claude/skills/pair-capability-verify-quality/SKILL.md index f3aa3a55..a3bec28b 100644 --- a/.claude/skills/pair-capability-verify-quality/SKILL.md +++ b/.claude/skills/pair-capability-verify-quality/SKILL.md @@ -26,7 +26,7 @@ Only check gates that are **not already passing** (idempotency preserved). | Argument | Required | Description | | -------- | -------- | -------------------------------------------------------------------------------------------------------------- | | `$scope` | No | Limit checking: `code-quality`, `tests`, `lint`, `all`, or any custom scope key from adoption (default: `all`) | -| `$story` | No | Story/PR id whose `risk:*` tag drives the tier (see Step 1.5). If omitted, resolved from the branch's PR or the story card. | +| `$story` | No | A **story id**, used only for the pre-publish story-card fallback (Step 1.5) when the branch has no PR yet. The tier normally comes from the **current-branch PR** β€” `gh pr view` with no argument resolves it β€” so a **PR number is never needed here** (the branch resolves its own PR, and its labels are authoritative). Pass `$story` only to name the story card to read before a PR exists. | `$scope` and the resolved tier compose: the tier decides the widest set CI would run; `$scope` may narrow within it (e.g. `$scope=lint`). `$scope` never *widens* past the tier. @@ -52,17 +52,19 @@ This step decides **which** suites the standard gates below run, so the local ru ```bash source .pair/knowledge/assets/tier-resolve.sh # tags only, no criteria (D18) - # Prefer the PR's labels; pre-publish (no PR yet) fall back to the story card. - if PR_LABELS="$(gh pr view "$story" --json labels -q '.labels[].name' 2>/dev/null)" && [ -n "$PR_LABELS" ]; then - : # tags from the PR - else - PR_LABELS="$(gh issue view "$story" --json labels -q '.labels[].name' 2>/dev/null)" # from the story card + # Primary: read the CURRENT-BRANCH PR's labels β€” the authoritative tier CI gates on. + # `gh pr view` with NO argument resolves the PR of the checked-out branch, so a + # review-raised (D17) tag on the PR is always honoured β€” never a stale story-card tier. + LABELS="$(gh pr view --json labels -q '.labels[].name' 2>/dev/null)" + if [ -z "$LABELS" ] && [ -n "$story" ]; then + # Pre-publish only (no PR yet): fall back to the story card by id. + LABELS="$(gh issue view "$story" --json labels -q '.labels[].name' 2>/dev/null)" # story card fi - TIER="$(resolve_tier "$PR_LABELS")" # green | yellow | red + TIER="$(resolve_tier "$LABELS")" # green | yellow | red (red = fail-safe if empty) ACTIVE_SUITES="$(required_suites_for_tier "$TIER")" # e.g. "install lint type build unit" ``` - - **Edge β€” tags on the story but not yet on the PR (pre-publish)**: resolve from the story card (`gh issue view`), as above. + - **Edge β€” pre-publish (no PR yet)**: when the branch has no PR, pass the **story id** as `$story` and the tier resolves from the story card (`gh issue view`), as above. A standalone run on a branch that already has a PR needs **no** `$story`: the PR's labels win, so a review-raised (D17) tag is never under-run versus CI. - **Fail-safe (AC3)**: `resolve_tier` returns `red` for **no** `risk:*` tag or an **unknown/malformed** value β€” the widest matrix, never a silent skip. When the tier came from the fail-safe, say so explicitly in the report: `Tier: πŸ”΄ red (fail-safe β€” no resolvable risk:* tag; running the full set)`. - **Widen-only**: because review never lowers a tier (D17), a later run can only widen the set versus an earlier one on the same item β€” never narrow. @@ -223,7 +225,8 @@ See [graceful degradation](../../../.pair/knowledge/guidelines/technical-standar ## Notes - The read-only-except-Step-5.C rule is stated once, in the overview above β€” this Notes section doesn't restate it. -- **Local = CI**: the tier and its suites are resolved through the *same* [`tier-resolve.sh`](../../../.pair/knowledge/assets/tier-resolve.sh) helper the CI pre-merge gate uses, so the check set and verdict match. This skill owns no matrix of its own (D18) β€” [quality-model.md Β§4](../../../.pair/knowledge/guidelines/quality-assurance/quality-model.md) is the single source. +- **Local = CI**: the tier and its suites are resolved through the *same* [`tier-resolve.sh`](../../../.pair/knowledge/assets/tier-resolve.sh) helper the CI pre-merge gate uses, so the check set and verdict match. This skill owns no matrix of its own (D18) β€” [quality-model.md Β§4](../../../.pair/knowledge/guidelines/quality-assurance/quality-model.md) is the single source. "Local = CI" scopes to the **tier check set** β€” see the coverage-guardrail caveat below. +- **Coverage guardrail is NOT mirrored locally**: the coverage-baseline guardrail (a separate opt-in CI job β€” the "twin of tiering") is **not** a tier suite and this skill does **not** run it. When a project sets `Coverage guardrail: enabled`, CI runs that baseline job *in addition to* the tier suites; the "local = CI" parity above covers the tier suites and verdict only, not the coverage guardrail. - Standard gates (Lint, Type+Build, Test) are universal and language/platform-independent. Custom gates are project-specific and defined in adoption (see the sources of truth above). - Each gate is independent β€” a failure in one gate does not prevent checking subsequent gates. - Re-invoke after fixes to confirm resolution. Already-passing gates are re-verified but complete instantly. diff --git a/packages/knowledge-hub/dataset/.skills/capability/verify-quality/SKILL.md b/packages/knowledge-hub/dataset/.skills/capability/verify-quality/SKILL.md index f050232e..50b4b9ae 100644 --- a/packages/knowledge-hub/dataset/.skills/capability/verify-quality/SKILL.md +++ b/packages/knowledge-hub/dataset/.skills/capability/verify-quality/SKILL.md @@ -26,7 +26,7 @@ Only check gates that are **not already passing** (idempotency preserved). | Argument | Required | Description | | -------- | -------- | -------------------------------------------------------------------------------------------------------------- | | `$scope` | No | Limit checking: `code-quality`, `tests`, `lint`, `all`, or any custom scope key from adoption (default: `all`) | -| `$story` | No | Story/PR id whose `risk:*` tag drives the tier (see Step 1.5). If omitted, resolved from the branch's PR or the story card. | +| `$story` | No | A **story id**, used only for the pre-publish story-card fallback (Step 1.5) when the branch has no PR yet. The tier normally comes from the **current-branch PR** β€” `gh pr view` with no argument resolves it β€” so a **PR number is never needed here** (the branch resolves its own PR, and its labels are authoritative). Pass `$story` only to name the story card to read before a PR exists. | `$scope` and the resolved tier compose: the tier decides the widest set CI would run; `$scope` may narrow within it (e.g. `$scope=lint`). `$scope` never *widens* past the tier. @@ -52,17 +52,19 @@ This step decides **which** suites the standard gates below run, so the local ru ```bash source .pair/knowledge/assets/tier-resolve.sh # tags only, no criteria (D18) - # Prefer the PR's labels; pre-publish (no PR yet) fall back to the story card. - if PR_LABELS="$(gh pr view "$story" --json labels -q '.labels[].name' 2>/dev/null)" && [ -n "$PR_LABELS" ]; then - : # tags from the PR - else - PR_LABELS="$(gh issue view "$story" --json labels -q '.labels[].name' 2>/dev/null)" # from the story card + # Primary: read the CURRENT-BRANCH PR's labels β€” the authoritative tier CI gates on. + # `gh pr view` with NO argument resolves the PR of the checked-out branch, so a + # review-raised (D17) tag on the PR is always honoured β€” never a stale story-card tier. + LABELS="$(gh pr view --json labels -q '.labels[].name' 2>/dev/null)" + if [ -z "$LABELS" ] && [ -n "$story" ]; then + # Pre-publish only (no PR yet): fall back to the story card by id. + LABELS="$(gh issue view "$story" --json labels -q '.labels[].name' 2>/dev/null)" # story card fi - TIER="$(resolve_tier "$PR_LABELS")" # green | yellow | red + TIER="$(resolve_tier "$LABELS")" # green | yellow | red (red = fail-safe if empty) ACTIVE_SUITES="$(required_suites_for_tier "$TIER")" # e.g. "install lint type build unit" ``` - - **Edge β€” tags on the story but not yet on the PR (pre-publish)**: resolve from the story card (`gh issue view`), as above. + - **Edge β€” pre-publish (no PR yet)**: when the branch has no PR, pass the **story id** as `$story` and the tier resolves from the story card (`gh issue view`), as above. A standalone run on a branch that already has a PR needs **no** `$story`: the PR's labels win, so a review-raised (D17) tag is never under-run versus CI. - **Fail-safe (AC3)**: `resolve_tier` returns `red` for **no** `risk:*` tag or an **unknown/malformed** value β€” the widest matrix, never a silent skip. When the tier came from the fail-safe, say so explicitly in the report: `Tier: πŸ”΄ red (fail-safe β€” no resolvable risk:* tag; running the full set)`. - **Widen-only**: because review never lowers a tier (D17), a later run can only widen the set versus an earlier one on the same item β€” never narrow. @@ -223,7 +225,8 @@ See [graceful degradation](../../../.pair/knowledge/guidelines/technical-standar ## Notes - The read-only-except-Step-5.C rule is stated once, in the overview above β€” this Notes section doesn't restate it. -- **Local = CI**: the tier and its suites are resolved through the *same* [`tier-resolve.sh`](../../../.pair/knowledge/assets/tier-resolve.sh) helper the CI pre-merge gate uses, so the check set and verdict match. This skill owns no matrix of its own (D18) β€” [quality-model.md Β§4](../../../.pair/knowledge/guidelines/quality-assurance/quality-model.md) is the single source. +- **Local = CI**: the tier and its suites are resolved through the *same* [`tier-resolve.sh`](../../../.pair/knowledge/assets/tier-resolve.sh) helper the CI pre-merge gate uses, so the check set and verdict match. This skill owns no matrix of its own (D18) β€” [quality-model.md Β§4](../../../.pair/knowledge/guidelines/quality-assurance/quality-model.md) is the single source. "Local = CI" scopes to the **tier check set** β€” see the coverage-guardrail caveat below. +- **Coverage guardrail is NOT mirrored locally**: the coverage-baseline guardrail (a separate opt-in CI job β€” the "twin of tiering") is **not** a tier suite and this skill does **not** run it. When a project sets `Coverage guardrail: enabled`, CI runs that baseline job *in addition to* the tier suites; the "local = CI" parity above covers the tier suites and verdict only, not the coverage guardrail. - Standard gates (Lint, Type+Build, Test) are universal and language/platform-independent. Custom gates are project-specific and defined in adoption (see the sources of truth above). - Each gate is independent β€” a failure in one gate does not prevent checking subsequent gates. - Re-invoke after fixes to confirm resolution. Already-passing gates are re-verified but complete instantly. diff --git a/packages/knowledge-hub/src/conformance/verify-quality-gate-matrix.test.ts b/packages/knowledge-hub/src/conformance/verify-quality-gate-matrix.test.ts index cbdbfa05..6e2308bb 100644 --- a/packages/knowledge-hub/src/conformance/verify-quality-gate-matrix.test.ts +++ b/packages/knowledge-hub/src/conformance/verify-quality-gate-matrix.test.ts @@ -49,6 +49,12 @@ describe('verify-quality SKILL.md β€” tier gate matrix integration (#259)', () = it('AC4 β€” any failing check β‡’ red verdict with the failing command output surfaced', () => { expect(SKILL).toMatch(/failing command output|surface[^\n]*output|command output/i) + // Pin the behavioral linkage (failing check β‡’ red verdict β‡’ output surfaced): + // the red-verdict wording must co-occur with the failing-output phrase, so a + // future edit can't drop the "red verdict" half and still pass. + expect(SKILL).toMatch( + /red verdict[\s\S]{0,120}(failing command output|command output)|(failing command output|command output)[\s\S]{0,120}red verdict/i, + ) }) it('reads the risk:* tag from the PR, and pre-publish from the story card (never classifies)', () => { From 2ddf3dfe60c2c51f95733669c18ce4b83487b695 Mon Sep 17 00:00:00 2001 From: Gianluca Carucci Date: Fri, 24 Jul 2026 12:50:10 +0200 Subject: [PATCH 3/6] =?UTF-8?q?[#259]=20fix:=20review=20round=202=20?= =?UTF-8?q?=E2=80=94=20document=20current-branch-PR=20tier-resolution=20as?= =?UTF-8?q?sumption?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Composition Interface note: no-arg gh pr view resolves tier from the current-branch PR, so it needs the PR branch checked out; /pair-process-review composing with only $scope from off-branch degrades to fail-safe red (safe over-check, non-blocking) not the classify-phase tier. Forwarding the tier belongs to /pair-process-review (out of scope). Doc-only, both SKILL.md + mirror. Co-Authored-By: Claude Opus 4.8 (1M context) --- .claude/skills/pair-capability-verify-quality/SKILL.md | 2 ++ .../dataset/.skills/capability/verify-quality/SKILL.md | 2 ++ 2 files changed, 4 insertions(+) diff --git a/.claude/skills/pair-capability-verify-quality/SKILL.md b/.claude/skills/pair-capability-verify-quality/SKILL.md index a3bec28b..da214169 100644 --- a/.claude/skills/pair-capability-verify-quality/SKILL.md +++ b/.claude/skills/pair-capability-verify-quality/SKILL.md @@ -206,6 +206,8 @@ When composed by `/pair-process-implement` or `/pair-process-review`: The composition contract is **unchanged** for callers (same PASS/FAIL result); the tier resolution is internal to this skill. +> **Tier-resolution assumes the PR branch is checked out.** The no-arg `gh pr view` primary (Step 1.5) resolves the tier from the **current-branch** PR, so it only finds the PR's labels when the invocation cwd is on that PR's branch β€” the standalone pre-push developer flow. When `/pair-process-review` composes this skill with only `$scope` (no `$story`) from a context **not** on the PR branch, `gh pr view` returns empty and, with no `$story` fallback, resolution degrades to the fail-safe (πŸ”΄ red, full set) rather than the tier `/pair-process-review` already resolved in its classify phase. That is the **safe over-check** direction β€” never an under-check β€” and `/pair-process-review` is non-blocking on verify-quality (its classify step is the authoritative review-time tier), so the only cost is a wider-than-necessary local run, not a wrong verdict. To get exact CI parity on that path, run this skill from the checked-out PR branch (or pass `$story`). Forwarding `/pair-process-review`'s resolved tier here would remove even that over-check, but that change belongs to `/pair-process-review` (out of scope for this skill). + When invoked **independently**: - Resolve the tier (Step 1.5), then run the tier's check set (or scoped gates if `$scope` is provided). diff --git a/packages/knowledge-hub/dataset/.skills/capability/verify-quality/SKILL.md b/packages/knowledge-hub/dataset/.skills/capability/verify-quality/SKILL.md index 50b4b9ae..8ed3ddd6 100644 --- a/packages/knowledge-hub/dataset/.skills/capability/verify-quality/SKILL.md +++ b/packages/knowledge-hub/dataset/.skills/capability/verify-quality/SKILL.md @@ -206,6 +206,8 @@ When composed by `/implement` or `/review`: The composition contract is **unchanged** for callers (same PASS/FAIL result); the tier resolution is internal to this skill. +> **Tier-resolution assumes the PR branch is checked out.** The no-arg `gh pr view` primary (Step 1.5) resolves the tier from the **current-branch** PR, so it only finds the PR's labels when the invocation cwd is on that PR's branch β€” the standalone pre-push developer flow. When `/review` composes this skill with only `$scope` (no `$story`) from a context **not** on the PR branch, `gh pr view` returns empty and, with no `$story` fallback, resolution degrades to the fail-safe (πŸ”΄ red, full set) rather than the tier `/review` already resolved in its classify phase. That is the **safe over-check** direction β€” never an under-check β€” and `/review` is non-blocking on verify-quality (its classify step is the authoritative review-time tier), so the only cost is a wider-than-necessary local run, not a wrong verdict. To get exact CI parity on that path, run this skill from the checked-out PR branch (or pass `$story`). Forwarding `/review`'s resolved tier here would remove even that over-check, but that change belongs to `/review` (out of scope for this skill). + When invoked **independently**: - Resolve the tier (Step 1.5), then run the tier's check set (or scoped gates if `$scope` is provided). From cbeee48b3079fbc61759dccb84b968ace7af1f27 Mon Sep 17 00:00:00 2001 From: Gianluca Carucci Date: Fri, 24 Jul 2026 13:19:46 +0200 Subject: [PATCH 4/6] =?UTF-8?q?[#259]=20fix:=20review=20round=203=20?= =?UTF-8?q?=E2=80=94=20secret-scan=20parity=20caveat,=20tighten=20smoke-pa?= =?UTF-8?q?rity=20comment?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - SKILL.md (source + mirror): add "Secret scanning is NOT mirrored locally" caveat parallel to the coverage-guardrail note (D24, unconditional at every tier incl. green; owned by setup-gates/assess-security) β€” completes the local=CI parity carve-out list. - verify-quality-gate-matrix.test.ts: tighten header comment β€” smoke test asserts the skill *references* the shared helper; the helper's own resolution is behaviorally executed (#258 block). Skill prose is not itself executed. Co-Authored-By: Claude Opus 4.8 (1M context) --- .../skills/pair-capability-verify-quality/SKILL.md | 1 + .../.skills/capability/verify-quality/SKILL.md | 1 + .../conformance/verify-quality-gate-matrix.test.ts | 13 +++++++++---- 3 files changed, 11 insertions(+), 4 deletions(-) diff --git a/.claude/skills/pair-capability-verify-quality/SKILL.md b/.claude/skills/pair-capability-verify-quality/SKILL.md index da214169..14942650 100644 --- a/.claude/skills/pair-capability-verify-quality/SKILL.md +++ b/.claude/skills/pair-capability-verify-quality/SKILL.md @@ -229,6 +229,7 @@ See [graceful degradation](../../../.pair/knowledge/guidelines/technical-standar - The read-only-except-Step-5.C rule is stated once, in the overview above β€” this Notes section doesn't restate it. - **Local = CI**: the tier and its suites are resolved through the *same* [`tier-resolve.sh`](../../../.pair/knowledge/assets/tier-resolve.sh) helper the CI pre-merge gate uses, so the check set and verdict match. This skill owns no matrix of its own (D18) β€” [quality-model.md Β§4](../../../.pair/knowledge/guidelines/quality-assurance/quality-model.md) is the single source. "Local = CI" scopes to the **tier check set** β€” see the coverage-guardrail caveat below. - **Coverage guardrail is NOT mirrored locally**: the coverage-baseline guardrail (a separate opt-in CI job β€” the "twin of tiering") is **not** a tier suite and this skill does **not** run it. When a project sets `Coverage guardrail: enabled`, CI runs that baseline job *in addition to* the tier suites; the "local = CI" parity above covers the tier suites and verdict only, not the coverage guardrail. +- **Secret scanning is NOT mirrored locally**: deterministic secret scanning (gitleaks by default) is a separate, **unconditional** CI layer that runs at *every* tier β€” including 🟒 green β€” not a tier suite (D24; owned by `setup-gates` / `assess-security`). This skill does **not** scan for committed secrets, so a committed secret can pass verify-quality green locally yet still trip a red secret-scan job at CI. The "local = CI" parity above is scoped to the tier check set + verdict; the secret-scan layer is deliberately out of its scope. - Standard gates (Lint, Type+Build, Test) are universal and language/platform-independent. Custom gates are project-specific and defined in adoption (see the sources of truth above). - Each gate is independent β€” a failure in one gate does not prevent checking subsequent gates. - Re-invoke after fixes to confirm resolution. Already-passing gates are re-verified but complete instantly. diff --git a/packages/knowledge-hub/dataset/.skills/capability/verify-quality/SKILL.md b/packages/knowledge-hub/dataset/.skills/capability/verify-quality/SKILL.md index 8ed3ddd6..196b55bb 100644 --- a/packages/knowledge-hub/dataset/.skills/capability/verify-quality/SKILL.md +++ b/packages/knowledge-hub/dataset/.skills/capability/verify-quality/SKILL.md @@ -229,6 +229,7 @@ See [graceful degradation](../../../.pair/knowledge/guidelines/technical-standar - The read-only-except-Step-5.C rule is stated once, in the overview above β€” this Notes section doesn't restate it. - **Local = CI**: the tier and its suites are resolved through the *same* [`tier-resolve.sh`](../../../.pair/knowledge/assets/tier-resolve.sh) helper the CI pre-merge gate uses, so the check set and verdict match. This skill owns no matrix of its own (D18) β€” [quality-model.md Β§4](../../../.pair/knowledge/guidelines/quality-assurance/quality-model.md) is the single source. "Local = CI" scopes to the **tier check set** β€” see the coverage-guardrail caveat below. - **Coverage guardrail is NOT mirrored locally**: the coverage-baseline guardrail (a separate opt-in CI job β€” the "twin of tiering") is **not** a tier suite and this skill does **not** run it. When a project sets `Coverage guardrail: enabled`, CI runs that baseline job *in addition to* the tier suites; the "local = CI" parity above covers the tier suites and verdict only, not the coverage guardrail. +- **Secret scanning is NOT mirrored locally**: deterministic secret scanning (gitleaks by default) is a separate, **unconditional** CI layer that runs at *every* tier β€” including 🟒 green β€” not a tier suite (D24; owned by `setup-gates` / `assess-security`). This skill does **not** scan for committed secrets, so a committed secret can pass verify-quality green locally yet still trip a red secret-scan job at CI. The "local = CI" parity above is scoped to the tier check set + verdict; the secret-scan layer is deliberately out of its scope. - Standard gates (Lint, Type+Build, Test) are universal and language/platform-independent. Custom gates are project-specific and defined in adoption (see the sources of truth above). - Each gate is independent β€” a failure in one gate does not prevent checking subsequent gates. - Re-invoke after fixes to confirm resolution. Already-passing gates are re-verified but complete instantly. diff --git a/packages/knowledge-hub/src/conformance/verify-quality-gate-matrix.test.ts b/packages/knowledge-hub/src/conformance/verify-quality-gate-matrix.test.ts index 6e2308bb..f09bd298 100644 --- a/packages/knowledge-hub/src/conformance/verify-quality-gate-matrix.test.ts +++ b/packages/knowledge-hub/src/conformance/verify-quality-gate-matrix.test.ts @@ -7,10 +7,15 @@ import { dirname, join, resolve } from 'path' // the checks the CI gate would run for that tier β€” reading TAGS + the KB gate // matrix only (D18), never classifying itself. These are content invariants on // the source-of-record SKILL.md (dataset), asserted the same way the rest of the -// KB/skill corpus is tested (see quality-model.test.ts). The executable parity -// (verify-quality resolving suites through the *same* tier-resolve.sh helper CI -// uses) is additionally smoke-tested in scripts/smoke-tests/scenarios/tier-aware-gate.sh -// per the gate-tooling ADL (scripts are smoke-tested, not vitest-unit-tested). +// KB/skill corpus is tested (see quality-model.test.ts). The parity claim is +// verified transitively, in two layers, by scripts/smoke-tests/scenarios/tier-aware-gate.sh +// per the gate-tooling ADL (scripts are smoke-tested, not vitest-unit-tested): +// (a) the smoke test asserts this SKILL.md *references* the shared tier-resolve.sh +// helper (resolve_tier / required_suites_for_tier / require_suite) β€” i.e. it +// composes the same resolver CI uses rather than re-implementing a matrix; and +// (b) that helper's own tier/suite resolution is behaviorally *executed* there (the +// pre-existing #258 block). The skill is prose, so its resolution is not itself +// executed here β€” parity rests on it composing the behaviorally-tested helper. const SKILL_PATH = join(__dirname, '../../dataset/.skills/capability/verify-quality/SKILL.md') const SKILL = readFileSync(SKILL_PATH, 'utf-8') From b0e1f4d2d5e2807bbace34740c30614e7f8c1e17 Mon Sep 17 00:00:00 2001 From: Gianluca Carucci Date: Fri, 24 Jul 2026 16:58:04 +0200 Subject: [PATCH 5/6] =?UTF-8?q?[#259]=20fix:=20review=20round=204=20?= =?UTF-8?q?=E2=80=94=20scope=20parity=20one-liner,=20empty-intersection=20?= =?UTF-8?q?no-op,=20repo-root=20CWD=20note?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - Hedge SKILL.md + skills-catalog description: 'mirrors CI for the tier check set (tier-check parity)' + name the two carve-outs (coverage-guardrail, secret-scan) so the one-liner no longer over-promises past the body. - Clarify $scope x tier narrow-only intersection: empty intersection is a no-op reporting SKIPPED (tier), never a FAIL, never a widen. - Note the Step 1.5 bash sources the helper assuming repo-root CWD (as CI runs it); the ../../../.pair link path is this file's relative path to the same asset, not a second copy. Co-Authored-By: Claude Opus 4.8 (1M context) --- .claude/skills/pair-capability-verify-quality/SKILL.md | 6 +++--- apps/website/content/docs/reference/skills-catalog.mdx | 2 +- .../dataset/.skills/capability/verify-quality/SKILL.md | 6 +++--- 3 files changed, 7 insertions(+), 7 deletions(-) diff --git a/.claude/skills/pair-capability-verify-quality/SKILL.md b/.claude/skills/pair-capability-verify-quality/SKILL.md index 14942650..22325270 100644 --- a/.claude/skills/pair-capability-verify-quality/SKILL.md +++ b/.claude/skills/pair-capability-verify-quality/SKILL.md @@ -1,6 +1,6 @@ --- name: pair-capability-verify-quality -description: "Checks whether the codebase passes the quality gates the CI pre-merge gate would run for the item/PR risk tier β€” resolves the `risk:*` tag, applies the KB gate matrix (🟒 lint+type+build; +unit from 🟑; +integration/E2E on πŸ”΄), and reports pass/fail per gate so local verification mirrors CI (no surprise reds at the gate, no over-checking on green work). Fail-safe red when untagged. Skips gates already passing. Composed by /pair-process-implement and /pair-process-review; invoke directly before pushing." +description: "Checks whether the codebase passes the quality gates the CI pre-merge gate would run for the item/PR risk tier β€” resolves the `risk:*` tag, applies the KB gate matrix (🟒 lint+type+build; +unit from 🟑; +integration/E2E on πŸ”΄), and reports pass/fail per gate so local verification mirrors CI for the tier check set (tier-check parity, no over-checking on green work; the coverage-guardrail and secret-scan CI layers are NOT mirrored locally). Fail-safe red when untagged. Skips gates already passing. Composed by /pair-process-implement and /pair-process-review; invoke directly before pushing." version: 0.5.0 author: Foomakers --- @@ -28,7 +28,7 @@ Only check gates that are **not already passing** (idempotency preserved). | `$scope` | No | Limit checking: `code-quality`, `tests`, `lint`, `all`, or any custom scope key from adoption (default: `all`) | | `$story` | No | A **story id**, used only for the pre-publish story-card fallback (Step 1.5) when the branch has no PR yet. The tier normally comes from the **current-branch PR** β€” `gh pr view` with no argument resolves it β€” so a **PR number is never needed here** (the branch resolves its own PR, and its labels are authoritative). Pass `$story` only to name the story card to read before a PR exists. | -`$scope` and the resolved tier compose: the tier decides the widest set CI would run; `$scope` may narrow within it (e.g. `$scope=lint`). `$scope` never *widens* past the tier. +`$scope` and the resolved tier compose: the tier decides the widest set CI would run; `$scope` may narrow within it (e.g. `$scope=lint`). `$scope` never *widens* past the tier. **Empty intersection is a no-op, not a failure**: if `$scope` selects a check the tier doesn't run (e.g. `$scope=tests` on 🟒 green, whose active test set is empty), the intersection is empty β€” that gate runs nothing and reports `SKIPPED (tier)`, never a FAIL and never a widen. ## Algorithm @@ -48,7 +48,7 @@ This step decides **which** suites the standard gates below run, so the local ru - **`disabled` (the default), or the flag/section is absent** β†’ **full suite**: the CI gate runs every suite on every PR, so mirror that β€” set the active suite set to **all adopted gates** (base + unit + integration + e2e + custom + aggregate, exactly the current behavior). Report `Tiering: disabled β€” running the full suite (CI parity)` and skip to Step 2. Do NOT read tags in this mode. - **`enabled`** β†’ continue; the CI gate is tier-scoped, so scope locally by tier. -2. **Act β€” resolve the tier from tags (never from the diff).** Load the shipped helper and resolve the `risk:*` tag: +2. **Act β€” resolve the tier from tags (never from the diff).** Load the shipped helper and resolve the `risk:*` tag. The snippet below assumes the **repo-root working directory** (where CI runs it), so it sources the helper by its repo-root-relative path; the `../../../.pair/...` in the markdown links above is only this skill file's relative path to the *same* asset β€” not a second copy: ```bash source .pair/knowledge/assets/tier-resolve.sh # tags only, no criteria (D18) diff --git a/apps/website/content/docs/reference/skills-catalog.mdx b/apps/website/content/docs/reference/skills-catalog.mdx index a561eb79..8734cfe4 100644 --- a/apps/website/content/docs/reference/skills-catalog.mdx +++ b/apps/website/content/docs/reference/skills-catalog.mdx @@ -82,7 +82,7 @@ All `analyze-*` skills are **output-only report producers**: they analyze and re | ----- | ------- | ----------- | | **verify-adoption** | `/pair-capability-verify-adoption` | Detects conformity/non-conformity of already-written code against the project's adopted decisions, for a given scope β€” a compliance audit, not a new-choice evaluation (that's assess-stack and siblings). Detection-only. | | **verify-done** | `/pair-capability-verify-done` | Checks a PR or work item against Definition of Done criteria, skipping already-passing criteria. Composed by `/pair-process-review`; invoke directly on demand. | -| **verify-quality** | `/pair-capability-verify-quality` | Checks whether the codebase passes the quality gates the CI pre-merge gate would run for the item/PR risk tier β€” resolves the `risk:*` tag and applies the [tag-driven gate matrix](/docs/concepts/tag-driven-gates) (🟒 lint+type+build; +unit from 🟑; +integration/E2E on πŸ”΄) so local verification mirrors CI, fail-safe red when untagged. Skips gates already passing. Composed by `/pair-process-implement`, `/pair-process-review`. | +| **verify-quality** | `/pair-capability-verify-quality` | Checks whether the codebase passes the quality gates the CI pre-merge gate would run for the item/PR risk tier β€” resolves the `risk:*` tag and applies the [tag-driven gate matrix](/docs/concepts/tag-driven-gates) (🟒 lint+type+build; +unit from 🟑; +integration/E2E on πŸ”΄) so local verification mirrors CI for the tier check set (tier-check parity; the coverage-guardrail and secret-scan CI layers are not mirrored locally), fail-safe red when untagged. Skips gates already passing. Composed by `/pair-process-implement`, `/pair-process-review`. | ### Setup and Management Capabilities diff --git a/packages/knowledge-hub/dataset/.skills/capability/verify-quality/SKILL.md b/packages/knowledge-hub/dataset/.skills/capability/verify-quality/SKILL.md index 196b55bb..5386e75f 100644 --- a/packages/knowledge-hub/dataset/.skills/capability/verify-quality/SKILL.md +++ b/packages/knowledge-hub/dataset/.skills/capability/verify-quality/SKILL.md @@ -1,6 +1,6 @@ --- name: verify-quality -description: "Checks whether the codebase passes the quality gates the CI pre-merge gate would run for the item/PR risk tier β€” resolves the `risk:*` tag, applies the KB gate matrix (🟒 lint+type+build; +unit from 🟑; +integration/E2E on πŸ”΄), and reports pass/fail per gate so local verification mirrors CI (no surprise reds at the gate, no over-checking on green work). Fail-safe red when untagged. Skips gates already passing. Composed by /implement and /review; invoke directly before pushing." +description: "Checks whether the codebase passes the quality gates the CI pre-merge gate would run for the item/PR risk tier β€” resolves the `risk:*` tag, applies the KB gate matrix (🟒 lint+type+build; +unit from 🟑; +integration/E2E on πŸ”΄), and reports pass/fail per gate so local verification mirrors CI for the tier check set (tier-check parity, no over-checking on green work; the coverage-guardrail and secret-scan CI layers are NOT mirrored locally). Fail-safe red when untagged. Skips gates already passing. Composed by /implement and /review; invoke directly before pushing." version: 0.5.0 author: Foomakers --- @@ -28,7 +28,7 @@ Only check gates that are **not already passing** (idempotency preserved). | `$scope` | No | Limit checking: `code-quality`, `tests`, `lint`, `all`, or any custom scope key from adoption (default: `all`) | | `$story` | No | A **story id**, used only for the pre-publish story-card fallback (Step 1.5) when the branch has no PR yet. The tier normally comes from the **current-branch PR** β€” `gh pr view` with no argument resolves it β€” so a **PR number is never needed here** (the branch resolves its own PR, and its labels are authoritative). Pass `$story` only to name the story card to read before a PR exists. | -`$scope` and the resolved tier compose: the tier decides the widest set CI would run; `$scope` may narrow within it (e.g. `$scope=lint`). `$scope` never *widens* past the tier. +`$scope` and the resolved tier compose: the tier decides the widest set CI would run; `$scope` may narrow within it (e.g. `$scope=lint`). `$scope` never *widens* past the tier. **Empty intersection is a no-op, not a failure**: if `$scope` selects a check the tier doesn't run (e.g. `$scope=tests` on 🟒 green, whose active test set is empty), the intersection is empty β€” that gate runs nothing and reports `SKIPPED (tier)`, never a FAIL and never a widen. ## Algorithm @@ -48,7 +48,7 @@ This step decides **which** suites the standard gates below run, so the local ru - **`disabled` (the default), or the flag/section is absent** β†’ **full suite**: the CI gate runs every suite on every PR, so mirror that β€” set the active suite set to **all adopted gates** (base + unit + integration + e2e + custom + aggregate, exactly the current behavior). Report `Tiering: disabled β€” running the full suite (CI parity)` and skip to Step 2. Do NOT read tags in this mode. - **`enabled`** β†’ continue; the CI gate is tier-scoped, so scope locally by tier. -2. **Act β€” resolve the tier from tags (never from the diff).** Load the shipped helper and resolve the `risk:*` tag: +2. **Act β€” resolve the tier from tags (never from the diff).** Load the shipped helper and resolve the `risk:*` tag. The snippet below assumes the **repo-root working directory** (where CI runs it), so it sources the helper by its repo-root-relative path; the `../../../.pair/...` in the markdown links above is only this skill file's relative path to the *same* asset β€” not a second copy: ```bash source .pair/knowledge/assets/tier-resolve.sh # tags only, no criteria (D18) From b97dff4231f5a9b8d59b5a01c3a146b1cd66ec1d Mon Sep 17 00:00:00 2001 From: Gianluca Carucci Date: Fri, 24 Jul 2026 17:24:14 +0200 Subject: [PATCH 6/6] =?UTF-8?q?[#259]=20fix:=20review=20round=205=20?= =?UTF-8?q?=E2=80=94=20disabled-branch=20cross-ref=20to=20graceful=20degra?= =?UTF-8?q?dation?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Disabled/full-suite branch now states absent suites are SKIPPED (not failed) and missing-suite-fail applies only on the enabled/tier-required path, so the branch is unambiguous read in isolation. Co-Authored-By: Claude Opus 4.8 (1M context) --- .claude/skills/pair-capability-verify-quality/SKILL.md | 2 +- .../dataset/.skills/capability/verify-quality/SKILL.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/.claude/skills/pair-capability-verify-quality/SKILL.md b/.claude/skills/pair-capability-verify-quality/SKILL.md index 22325270..522b811f 100644 --- a/.claude/skills/pair-capability-verify-quality/SKILL.md +++ b/.claude/skills/pair-capability-verify-quality/SKILL.md @@ -45,7 +45,7 @@ Execute each gate in order. For every gate, follow the **check β†’ skip β†’ act This step decides **which** suites the standard gates below run, so the local run matches what CI would run for this item/PR. It reads tags + the matrix only β€” no classification. 1. **Check β€” is tiering on?** Read the `Pre-merge tiering` flag in [way-of-working.md](../../../.pair/adoption/tech/way-of-working.md). - - **`disabled` (the default), or the flag/section is absent** β†’ **full suite**: the CI gate runs every suite on every PR, so mirror that β€” set the active suite set to **all adopted gates** (base + unit + integration + e2e + custom + aggregate, exactly the current behavior). Report `Tiering: disabled β€” running the full suite (CI parity)` and skip to Step 2. Do NOT read tags in this mode. + - **`disabled` (the default), or the flag/section is absent** β†’ **full suite**: the CI gate runs every suite on every PR, so mirror that β€” set the active suite set to **all adopted gates** (base + unit + integration + e2e + custom + aggregate, exactly the current behavior). Report `Tiering: disabled β€” running the full suite (CI parity)` and skip to Step 2. Do NOT read tags in this mode. **Suites the repo lacks are still SKIPPED, not failed, here** β€” the graceful-degradation rule below applies (absent suites are skipped in full-suite mode); the missing-suite-is-a-failure rule (Step 1.5 point 4) fires *only* on the enabled path when a resolved tier requires that suite. - **`enabled`** β†’ continue; the CI gate is tier-scoped, so scope locally by tier. 2. **Act β€” resolve the tier from tags (never from the diff).** Load the shipped helper and resolve the `risk:*` tag. The snippet below assumes the **repo-root working directory** (where CI runs it), so it sources the helper by its repo-root-relative path; the `../../../.pair/...` in the markdown links above is only this skill file's relative path to the *same* asset β€” not a second copy: diff --git a/packages/knowledge-hub/dataset/.skills/capability/verify-quality/SKILL.md b/packages/knowledge-hub/dataset/.skills/capability/verify-quality/SKILL.md index 5386e75f..c80a088a 100644 --- a/packages/knowledge-hub/dataset/.skills/capability/verify-quality/SKILL.md +++ b/packages/knowledge-hub/dataset/.skills/capability/verify-quality/SKILL.md @@ -45,7 +45,7 @@ Execute each gate in order. For every gate, follow the **check β†’ skip β†’ act This step decides **which** suites the standard gates below run, so the local run matches what CI would run for this item/PR. It reads tags + the matrix only β€” no classification. 1. **Check β€” is tiering on?** Read the `Pre-merge tiering` flag in [way-of-working.md](../../../.pair/adoption/tech/way-of-working.md). - - **`disabled` (the default), or the flag/section is absent** β†’ **full suite**: the CI gate runs every suite on every PR, so mirror that β€” set the active suite set to **all adopted gates** (base + unit + integration + e2e + custom + aggregate, exactly the current behavior). Report `Tiering: disabled β€” running the full suite (CI parity)` and skip to Step 2. Do NOT read tags in this mode. + - **`disabled` (the default), or the flag/section is absent** β†’ **full suite**: the CI gate runs every suite on every PR, so mirror that β€” set the active suite set to **all adopted gates** (base + unit + integration + e2e + custom + aggregate, exactly the current behavior). Report `Tiering: disabled β€” running the full suite (CI parity)` and skip to Step 2. Do NOT read tags in this mode. **Suites the repo lacks are still SKIPPED, not failed, here** β€” the graceful-degradation rule below applies (absent suites are skipped in full-suite mode); the missing-suite-is-a-failure rule (Step 1.5 point 4) fires *only* on the enabled path when a resolved tier requires that suite. - **`enabled`** β†’ continue; the CI gate is tier-scoped, so scope locally by tier. 2. **Act β€” resolve the tier from tags (never from the diff).** Load the shipped helper and resolve the `risk:*` tag. The snippet below assumes the **repo-root working directory** (where CI runs it), so it sources the helper by its repo-root-relative path; the `../../../.pair/...` in the markdown links above is only this skill file's relative path to the *same* asset β€” not a second copy: