Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 16 additions & 3 deletions .claude/skills/pair-capability-setup-gates/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,7 @@ Configure CI/CD quality gates for the project. Reads quality assurance guideline
1. **Act**: Read quality assurance guidelines:
- [quality-assurance.md](../../../.pair/knowledge/guidelines/technical-standards/git-workflow/quality-assurance.md) — gate types and checklists
- [quality-gates.md](../../../.pair/knowledge/guidelines/quality-assurance/quality-standards/quality-gates.md) — gate framework and registry format
- [tier-aware-pipeline.md](../../../.pair/knowledge/guidelines/infrastructure/cicd-strategy/tier-aware-pipeline.md) — the **opt-in** optimization that reduces checks per risk tier (tags only, fail-safe red); default is the full suite on every PR, tier-aware only when the project opts in
- [tier-aware-pipeline.md](../../../.pair/knowledge/guidelines/infrastructure/cicd-strategy/tier-aware-pipeline.md) — the **opt-in** optimization that reduces checks per risk tier (tags only, fail-safe red); default is the full suite on every PR, tier-aware only when the project opts in. Also defines the **opt-in coverage guardrail** job (off by default) and its [config format](../../../.pair/knowledge/assets/coverage-config-example.md)
- [quality-model.md](../../../.pair/knowledge/guidelines/quality-assurance/quality-model.md) §4 — the single source of the tier→checks matrix the pipeline projects (never re-declared in generated config)
- [shared-config-packages.md](../../../.pair/knowledge/guidelines/code-design/quality-standards/shared-config-packages.md) — shared-config-package pattern, per-type overrides, `tools/*` reference implementation
- [secret-scanning.md](../../../.pair/knowledge/guidelines/quality-assurance/security/secret-scanning.md) — the deterministic, required-at-every-tier CI job this skill provisions (R6.5, D24) — never a skill, never an LLM judgment call
Expand Down Expand Up @@ -98,21 +98,32 @@ Configure CI/CD quality gates for the project. Reads quality assurance guideline
- **Quick mode / non-interactive / CI** — take the default: **No** (full pipeline). Never prompt.
- Record the resolved choice for Step 4: `TIERING = enabled | disabled` (default `disabled`).

4. **Verify**: Developer approves gate configuration and the tiering choice is resolved (`disabled` unless explicitly opted in).
4. **Act — coverage guardrail opt-in** (same [guided/quick convention](../../../.pair/knowledge/guidelines/technical-standards/ai-development/skill-conventions/guided-quick-setup.md), default **No** — the twin of the tiering opt-in above): the coverage regression guardrail is off by default; a project opts in like any other input.
- **Guided mode** (interactive/TTY) — ask **one** question, pre-filled with the default:

> Enable the coverage guardrail? This adds a job that blocks a PR whose test coverage drops **below a committed baseline** (maintaining/improving passes — it is not an absolute wall). Off by default; the baseline is human-committed and bootstraps advisory-only on first run. **[default: No]**

Default (Enter) ⇒ **No** (no coverage job). Only an explicit "Yes" opts in.
- **Quick mode / non-interactive / CI** — take the default: **No**. Never prompt.
- Record the resolved choice for Step 4: `COVERAGE_GUARDRAIL = enabled | disabled` (default `disabled`).

5. **Verify**: Developer approves gate configuration; the tiering choice (`disabled` unless opted in) and the coverage-guardrail choice (`disabled` unless opted in) are both resolved.

### Step 4: Write Configuration

1. **Act**: Update [way-of-working.md](../../../.pair/adoption/tech/way-of-working.md):
- Set quality gate command (e.g., `pnpm quality-gate`)
- Write or update Custom Gate Registry table
- **Pre-merge tiering flag**: if `TIERING = enabled` (Step 3.3 opt-in), record `Pre-merge tiering: enabled` in the Quality Gates section. If `disabled` (the default), either omit the line or record `Pre-merge tiering: disabled` — both mean full-suite. Never write `enabled` without the explicit opt-in.
- **Coverage guardrail flag**: if `COVERAGE_GUARDRAIL = enabled` (Step 3.4 opt-in), record `Coverage guardrail: enabled` in the Quality Gates section; if `disabled` (the default), either omit the line or record `Coverage guardrail: disabled`. Never write `enabled` without the explicit opt-in. **When enabled**, also create `tech/coverage-baseline.md` from the KB template ([coverage-config-example.md](../../../.pair/knowledge/assets/coverage-config-example.md)) with the per-type targets for the adopted stack and the baselines left unset (bootstrap-only until a human commits them — see that example's Persistence note). **Idempotent**: if the flag is already `enabled` and `tech/coverage-baseline.md` already exists, confirm rather than overwrite.
2. **Act**: Generate the **pre-merge pipeline** appropriate for the adopted stack and hosting. **Which pipeline depends on `TIERING`:**
- GitHub Actions → `.github/workflows/quality.yml`
- GitLab → `.gitlab-ci.yml` quality stage
- Other → document commands for manual pipeline setup
- **`TIERING = disabled` (default) → full-suite pipeline**: every PR runs base (install + lint + type + build) **and** unit **and** integration/E2E **and** coverage — all jobs unconditional, no tag resolution, no `risk:*` reads. This is the safe default; do **not** generate the tier-aware jobs.
- **`TIERING = enabled` (opt-in) → tier-aware pipeline** following [tier-aware-pipeline.md](../../../.pair/knowledge/guidelines/infrastructure/cicd-strategy/tier-aware-pipeline.md): the pipeline **reads the PR's `risk:*` tag only** (via [`tier-resolve.sh`](../../../.pair/knowledge/assets/tier-resolve.sh) — tags only, no classification criteria of its own, D18) and schedules the [quality-model.md](../../../.pair/knowledge/guidelines/quality-assurance/quality-model.md) §4 matrix: base every tier, unit from 🟡, integration/E2E from 🔴; **untagged/unknown/malformed ⇒ 🔴** (fail-safe); a required suite missing at a tier ⇒ **explicit failure** (`require_suite`), never a silent pass. Do not copy any tier threshold into the generated config — it is a projection of the model, not a second source.
- **Wire required checks**: mark `base` and `secret-scan` as required status checks on the protected branch. In full-suite mode all suite jobs are required; in tier-aware mode the tier-scoped jobs are required-when-scheduled (or via one aggregating gate job). A red gate blocks merge either way.
- **Coverage job (opt-in)**: emit the `coverage` job from [tier-aware-pipeline.md](../../../.pair/knowledge/guidelines/infrastructure/cicd-strategy/tier-aware-pipeline.md#coverage-guardrail-opt-in-regression-gate-consumed-by-this-pipeline) **only when `COVERAGE_GUARDRAIL = enabled`** (Step 3.4). It sources [`coverage-gate.sh`](../../../.pair/knowledge/assets/coverage-gate.sh) (config + a number only, no criteria — D18), reads `tech/coverage-baseline.md`, and blocks a regression below the committed baseline (maintain/improve passes; bootstrap-only when unset). With the flag absent (default) do **not** emit a coverage job in either pipeline mode.
- **Wire required checks**: mark `base` and `secret-scan` as required status checks on the protected branch. In full-suite mode all suite jobs are required; in tier-aware mode the tier-scoped jobs are required-when-scheduled (or via one aggregating gate job). A red gate blocks merge either way. The coverage job (when enabled) is required-when-scheduled like the other tier-scoped jobs.
- **Post-merge staging** (on merge to the main branch): **build + deploy only, no gate re-run** — the gate already passed pre-merge (identical in both modes).
3. **Act**: Write the secret-scanning job — required, not an optional proposal line, in **both** pipeline modes: resolve the scanner (Argument > Adoption's Custom Gate Registry override > KB default gitleaks), then write the [CI Job Template](../../../.pair/knowledge/guidelines/quality-assurance/security/secret-scanning.md#ci-job-template-github-actions) into the same pipeline file as a `required` job, and provision a starting `.gitleaks.toml` at the project root (see the [allowlist mechanism](../../../.pair/knowledge/guidelines/quality-assurance/security/secret-scanning.md#allowlist-mechanism-adoption-controlled)) if one doesn't already exist. Never write the job with `continue-on-error` — fail-closed is not optional (R6.5).
4. **Verify**: Configuration files written, including the pre-merge pipeline (full-suite by default; tag-only tier-aware resolution + tier-scoped jobs only when `TIERING = enabled`) + required-check wiring, the post-merge staging (build+deploy only) config, the secret-scanning job, and `.gitleaks.toml`. If tiering is enabled, grep the generated pipeline for classification tokens (schema/diff/path heuristics) — there must be none (D18). If disabled, confirm no `risk:*` resolution was generated and every suite job runs unconditionally.
Expand Down Expand Up @@ -152,6 +163,7 @@ GATE CONFIGURATION COMPLETE:
├── Pre-production: [N gates configured | N/A]
├── Pipeline: [file path | manual — full-suite pre-merge (default: every PR runs all suites) OR tier-aware pre-merge (opt-in: tags only, fail-safe 🔴) + post-merge staging build+deploy]
├── Pre-merge tiering: [disabled (default — full suite every PR) | enabled (risk-tier-scoped, opt-in)]
├── Coverage guardrail: [disabled (default — no coverage job) | enabled (opt-in — coverage job + tech/coverage-baseline.md created)]
├── Required checks: [base + secret-scan required; full-suite: all suite jobs required | tier-aware: tier-scoped jobs required-when-scheduled]
├── Secret Scan: [gitleaks (KB default) | <override> — job + .gitleaks.toml written | already configured]
├── Shared configs: [package list | N/A — non-JS, documented pointer only]
Expand Down Expand Up @@ -184,6 +196,7 @@ See [graceful degradation](../../../.pair/knowledge/guidelines/technical-standar

- This skill **modifies files** — it writes to way-of-working.md, creates/updates CI/CD pipeline configuration (the pre-merge pipeline — full-suite by default, tier-aware only on opt-in — + post-merge staging + the required secret-scanning job and `.gitleaks.toml`), and provisions shared lint/format config packages + hook manager files (`.husky/` by default).
- **Full checks are the default; risk-tier reduction is opt-in** (ADL 2026-07-20): unless the project sets `Pre-merge tiering: enabled` in way-of-working.md (Step 3.3 opt-in), the generated pipeline runs the **full suite on every PR** — the safe, pre-tiering behavior. Tiering only *reduces* checks on lower-risk PRs, so it must be enabled explicitly.
- **Coverage guardrail is opt-in, off by default** (story #282): the `coverage` regression job and `tech/coverage-baseline.md` are generated **only** when the project sets `Coverage guardrail: enabled` (Step 3.4 opt-in) — the twin of the tiering opt-in. Absent the flag, no coverage job and no baseline file. The baseline is human-committed; the gate bootstraps advisory-only (suggests on stderr, never persists — a CI checkout is ephemeral; automated commit-back is story #372). See [coverage-config-example.md](../../../.pair/knowledge/assets/coverage-config-example.md).
- **When tiering is enabled, the generated pipeline reads classification tags only, never classifies** (D18): tier criteria live in [quality-model.md](../../../.pair/knowledge/guidelines/quality-assurance/quality-model.md) §3/§4; the pipeline is the deterministic Automation layer that projects the matrix from the PR's `risk:*` tag. Untagged ⇒ 🔴 (fail-safe). See [tier-aware-pipeline.md](../../../.pair/knowledge/guidelines/infrastructure/cicd-strategy/tier-aware-pipeline.md).
- **Secret scanning is CI config, not a judgment call** (D24, anti-complexity): this skill provisions the job mechanically; it never evaluates whether a diff contains a secret itself — that is gitleaks' (or the adopted scanner's) job at runtime, with no LLM in the loop. `/pair-capability-assess-security` never re-implements this — see that skill's own Notes.
- **Idempotent** — see [idempotency convention](../../../.pair/knowledge/guidelines/technical-standards/ai-development/skill-conventions/idempotency.md). This skill's check: an already-configured project (incl. provisioned shared configs and hooks) is confirmed; update only on explicit developer request. Conflicting local config is always resolved by asking first (see Edge Cases above).
Expand Down
21 changes: 21 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -95,5 +95,26 @@ jobs:
run: pnpm test
- name: Run coverage thresholds (brand, website)
run: pnpm turbo test:coverage --filter=@pair/brand --filter=@pair/website
# Dogfood of the story #282 coverage guardrail on pair's own CI (opt-in:
# way-of-working `Coverage guardrail: enabled`). Standalone/minimal — NOT
# the full tier-aware pipeline (#258). Reuses the coverage run above: reads
# the line % from each package's istanbul coverage-summary.json and blocks
# a regression below the human-committed baseline in tech/coverage-baseline.md.
# Conservative fixed tier=red => a missing coverage report fails safe.
- name: Coverage guardrail (regression gate — dogfood #282)
run: |
set -euo pipefail
# shellcheck disable=SC1091
source .pair/knowledge/assets/coverage-gate.sh
TIER=red
CFG=.pair/adoption/tech/coverage-baseline.md
pct() {
node -e "const fs=require('fs');process.stdout.write(String(JSON.parse(fs.readFileSync('$1','utf8')).total.lines.pct))"
}
BRAND_PCT="$(pct packages/brand/coverage/coverage-summary.json || true)"
WEBSITE_PCT="$(pct apps/website/coverage/coverage-summary.json || true)"
echo "coverage: brand(shared)=${BRAND_PCT}% website(frontend)=${WEBSITE_PCT}%"
coverage_gate "$TIER" shared "$BRAND_PCT" "$CFG"
coverage_gate "$TIER" frontend "$WEBSITE_PCT" "$CFG"
- name: Run E2E tests
run: pnpm --filter @pair/website e2e
49 changes: 49 additions & 0 deletions .pair/adoption/tech/coverage-baseline.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
# `tech/coverage-baseline.md` — pair coverage guardrail config

pair's own coverage guardrail config (dogfood of story #282). This file exists because pair opted into the guardrail (`Coverage guardrail: enabled` in [way-of-working.md](./way-of-working.md)). The [`Coverage guardrail` step](../../../.github/workflows/ci.yml) reads this file with the shipped [`coverage-gate.sh`](../../knowledge/assets/coverage-gate.sh); it blocks a PR whose line coverage drops below a `baseline.<type>` here, at every tier — maintaining or improving always passes.

Format documented in the KB [config example](../../knowledge/assets/coverage-config-example.md): a single `key=value` block; the surrounding markdown fence/headings are ignored (lines matched with a plain `^key=`).

## Types measured

Only the two packages that emit an istanbul `coverage-summary.json` (`reporter: ['json-summary', …]`) and already run in CI via `turbo test:coverage` feed the gate. The type is the touched package's nature:

| Package | Type | Nature |
| --------------- | ---------- | ---------------------------------------------------------- |
| `@pair/website` | `frontend` | Next.js app — UI validated primarily by Playwright e2e. |
| `@pair/brand` | `shared` | Reusable design-system component library, consumed widely. |

The other packages (`knowledge-hub`, `content-ops`, `dev-tools`, `pair-cli`) have `test:coverage` but emit only `text`/`html`/`lcov` (no `json-summary`) and are not run in the CI coverage step, so they are not fed to the guardrail yet. Wiring them in is a follow-up (extend the CI coverage run to include them + add `json-summary` to their vitest config).

## Baseline (measured current state)

The metric is **line coverage** (`total.lines.pct` from `coverage-summary.json`). Baselines below were **measured on the current `main`/branch state** and set **~1pp below** the measured value, to absorb per-run float jitter and legitimate line churn (so the gate passes today and blocks only a genuine regression, not run-to-run noise):

| Type | Measured lines % | `baseline` (~1pp margin) | `target` (gradual goal) |
| ---------- | ---------------- | ---------- | ----------------------- |
| `shared` | 85.04 (`@pair/brand`) | 84 | 90 |
| `frontend` | 20.16 (`@pair/website`) | 19 | 25 |

`target.frontend` is deliberately modest: the website is e2e-covered by design (see the website vitest ADL 2026-07-12), so unit line coverage is intentionally low; below-target-but-above-baseline only warns.

## Config

```ini
# pair coverage guardrail — per-type gradual targets + committed baselines.
# Metric: line coverage (total.lines.pct from istanbul coverage-summary.json).
target.default=70
target.shared=90
target.frontend=25

# Committed baselines — the guardrail blocks a drop below these (every tier).
# Set ~1pp below the measured current state to absorb float jitter + legitimate
# line churn (blocks only a genuine regression, not run-to-run noise).
baseline.shared=84
baseline.frontend=19
```

## Notes

- **Baseline-relative, not an absolute wall.** A drop below `baseline.<type>` blocks; below the gradual `target` but at/above baseline only warns (R7.3: incremental improvement).
- **Human-committed.** The gate never persists a baseline itself; raise a baseline here by committing a new value once coverage improves.
- **Fail-safe.** If a coverage report is missing, the CI step runs the gate at a conservative fixed `red` tier, so a missing report blocks rather than silently passes.
Loading
Loading