[#282] feat: coverage baseline + CI guardrail (tier-aware pipeline gate)#368
Merged
Conversation
This comment has been minimized.
This comment has been minimized.
- coverage-gate.sh asset (dataset+root mirror): provider-agnostic baseline regression guardrail — blocks a drop below the established baseline at every tier, maintain/improve passes; per-type targets; baseline bootstrapping; fail-safe (block at red / warn lower) on a missing report. Reads adoption config + a number only, no classification criteria (D18). - coverage-config-example.md asset (dataset+root): tech/coverage-baseline.md config format + KB-sensible defaults (backend 80 / frontend 60 / shared 90). - tier-aware-pipeline.md (dataset+root): coverage job in the YAML + "Coverage guardrail" section (job consumed by this pipeline, not a parallel mechanism). - adoption: pair's coverage-baseline.md (targets, baselines bootstrapped) + way-of-working Quality Gates bullet. - docs site: tag-driven-gates.mdx "Coverage never silently drops" section. - smoke test coverage-gate.sh wired into CI_TESTS (regression/improve/bootstrap/ per-type/corrupt/missing-report + no-criteria + guideline audit — all PASS). Refs: #282
rucka
force-pushed
the
feat/282-coverage-baseline-ci-guardrail
branch
from
July 24, 2026 06:44
c638505 to
32a2e19
Compare
…tion) Rework per maintainer decisions on PR #368: - opt-in reframe: `Coverage guardrail: disabled` (default) flag, twin of Pre-merge tiering; coverage job generated only when enabled (wow template + pair adoption + tier-aware-pipeline + docs). - persistence=A (fix Major): drop "machine-maintained"; bootstrap is advisory only (suggests baseline.<type> on stderr, never persists — CI checkout ephemeral); baseline is human-committed; commit-back deferred to #372. - Minors: cov_config_value trims CR/whitespace (CRLF config honored); bootstrap advisory-only (no out-of-fence write); exclude is adopter-applied (not read by gate) — documented. - remove pair's own tech/coverage-baseline.md (violated default-absent). - setup-gates: guided "Enable the coverage guardrail? [default No]" (#276), enabled => flag + create coverage-baseline.md + wire job; idempotent. Mirror regenerated via `pair update`; smoke coverage-gate.sh + tier-aware-gate.sh + knowledge-hub vitest/check:links + mdlint + docs:staleness all green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
Collaborator
Author
|
@/private/tmp/claude-501/-Users-gianlucacarucci5-projects-pair/6b978b2a-c3b5-4d25-be2b-01366429c4a9/scratchpad/c368cur.md |
6 tasks
Adopt the story #282 capability on pair's own CI, without changing the framework default (dataset template stays `disabled`). - way-of-working: `Coverage guardrail: enabled` (pair opt-in only). - tech/coverage-baseline.md: per-type targets + baselines measured on current state, floored to absorb jitter — shared(@pair/brand)=85 (lines 85.04%), frontend(@pair/website)=20 (lines 20.16%). - ci.yml: minimal standalone `Coverage guardrail` step after the existing coverage-thresholds step (kept). Sources coverage-gate.sh, reads total.lines.pct from each package's istanbul coverage-summary.json, calls coverage_gate at a conservative fixed tier=red (missing report fails safe). Blocks a regression below baseline; PASSES on current coverage. Gate verified locally: PASS on current state; BLOCKS on simulated regression. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
0.04pp headroom (85.04→85) would false-block on trivial line churn/jitter. Set baselines ~1pp below measured (shared 84, frontend 19) — still blocks a genuine regression, absorbs run-to-run noise. Dogfood-safety on pair's own CI. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
What Changed
An opt-in coverage baseline + regression guardrail, consumed as a job by the tier-aware pre-merge pipeline (#258) — never a second, parallel CI mechanism, and off by default (the twin of
Pre-merge tiering):Coverage guardrailflag (way-of-working: dataset template + pair adoption):disabled(default). A project opts in withenabled; absent the flag there is no coverage job and notech/coverage-baseline.md.coverage-gate.sh(dataset + root mirror, provider-agnostic shell — ships to target-project pipelines like Tier-aware pre-merge pipeline — reads classification tags only, fail-safe red #258'stier-resolve.sh):coverage_gate <tier> <type> <current-%> <config-file>blocks a regression below the committed baseline at every tier; maintain/improve always passes.target_for_type/baseline_for_typeresolve per-type config (backend/frontend/shared/default). Fail-safe on a missing coverage report: block at 🔴, warn at 🟢/🟡. Reads adoption config + a coverage number ONLY — no diff/code/path inspection (D18, grep-audited by the smoke test).baseline.<type>=NNline to the config. The gate never persists a baseline itself — a CI checkout is ephemeral, so a written baseline would be discarded and coverage could drift down run after run with the guard never firing. Bootstrapping is therefore advisory: on an unset/missing/corrupt baseline the gate prints the suggestedbaseline.<type>=NNto stderr and passes (bootstrap-only mode), without writing anything. Automated commit-back of a bootstrapped baseline is provider-specific and tracked separately — Coverage baseline auto commit-back — CI persists/ratchets the baseline (opt-in, on top of #282) #372. The misleading "machine-maintained" wording is removed.coverage-config-example.md(dataset + root): documents the config format (target.<type>,baseline.<type>,target.default,exclude) + KB-sensible defaults, the persistence model, and thatexcludeis applied by the adopter to their own coverage tool's config (the gate does not read it).tier-aware-pipeline.md(dataset + root mirror): thecoveragejob is opt-in — emitted only whenCoverage guardrail: enabled(documented like the tiering opt-in); a "Coverage guardrail (opt-in …)" section documents it's a job consumed by this pipeline, not a parallel mechanism.setup-gatesSKILL.md (dataset + mirror): a guided-setup question — "Enable the coverage guardrail? [default No]" (guided/quick convention, Guided/Quick Setup Convention — unify interactive vs default resolution across setup skills #276).enabled⇒ writes the flag, createstech/coverage-baseline.mdfrom the KB template, and wires the coverage job;disabled/quick ⇒ nothing. Idempotent.tag-driven-gates.mdx: "Coverage guardrail (opt-in, off by default)" section — opt-in, human-committed baseline, advisory bootstrap, adopter-appliedexclude..pair/adoption/tech/coverage-baseline.md(pair's own baseline) — it violated the default-absent principle; the example lives in the KB only.scripts/smoke-tests/scenarios/coverage-gate.sh(wired intorun-all.sh): regression-blocks at every tier, maintain/improve passes, per-type target selection, advisory bootstrap (suggests on stderr, does NOT persist), corrupt-baseline advisory (no overwrite), CRLF baseline honored (regression test for the Minor fix), missing-report fail-safe by tier, D18 grep audit, opt-in + persistence + exclude documentation audits, and a no-"machine-maintained" guard — all PASS.Why This Change
Closes story #282 of epic #209 (Code Quality & Testing foundations), epic AC3 / R7.3: test coverage should never silently drop, without demanding an absolute wall on a legacy codebase. Plugs into #258's pipeline as an opt-in gate (per epic #210's rule: no parallel CI mechanism, no gate re-deriving classification criteria of its own).
Design decisions
Pre-merge tiering: generated only on explicitCoverage guardrail: enabled. Adoption defaults (pair included) carry no coverage job and no baseline file.coverage-gate.shruns inside the target project's CI, exactly like Tier-aware pre-merge pipeline — reads classification tags only, fail-safe red #258'stier-resolve.sh— smoke-tested pipeline asset, per the tier-resolve precedent, not the gate-tooling ADL's "tested module not script" rule (which governs pair's OWN gates).Testing
Dogfood: coverage guardrail attivo sulla CI di pair
pair adotta la propria capability e la wira sulla CI reale — senza cambiare il default del framework (il template nel dataset resta
disabled; opt-in solo a livello di progetto pair).Opt-in pair.
.pair/adoption/tech/way-of-working.md(Quality Gates):Coverage guardrail: enabled. Il dataset template restadisabled(verificato:packages/knowledge-hub/dataset/.pair/adoption/tech/way-of-working.mdinvariato).Baseline = stato corrente misurato. Nuovo
.pair/adoption/tech/coverage-baseline.mddal formato dicoverage-config-example.md. Metrica: line coverage (total.lines.pctda istanbulcoverage-summary.json). Solo i due package che emettonojson-summarye girano già nella coverage-run CI (turbo test:coverage) alimentano il gate:baselinetargetshared@pair/brandfrontend@pair/websiteBaseline floored dall'attuale per assorbire il jitter float run-to-run: il gate passa sullo stato corrente e blocca solo una regressione reale (non unset). Gli altri package (
knowledge-hub,content-ops,dev-tools,pair-cli) hannotest:coveragema emettono solotext/html/lcov(nojson-summary) e non sono nella coverage-run CI → non ancora nel gate (follow-up documentato nel file adoption).Wiring CI (
.github/workflows/ci.yml). Step standalone minimaleCoverage guardrail (regression gate — dogfood #282), dopo lo step esistenteRun coverage thresholds (brand, website)(mantenuto, coesiste). Lo step:source .pair/knowledge/assets/coverage-gate.sh→ estraetotal.lines.pctda ognicoverage-summary.jsonvia node → chiamacoverage_gate <tier> <type> <pct> .pair/adoption/tech/coverage-baseline.md. NON la pipeline tier-aware completa (Tier-aware pre-merge pipeline — reads classification tags only, fail-safe red #258). Tier: valore conservativo fissored→ un report di coverage mancante fallisce fail-safe (nessun pass silenzioso).Sicurezza / non rompere le PR di pair. Step eseguito localmente sullo stato attuale ⇒ PASS (brand 85.04% ≥ 85, website 20.16% ≥ 20; entrambi advisory-warn sotto il target graduale, non bloccano). Verificato inoltre che il gate BLOCCA su regressione simulata (84.9% shared, 19.5% frontend) e passa su
==baseline. La source del gate non attiva l'entrypoint CLI quando eseguita come script file (come fa GitHub Actionsrun:), quindi nessun abort spurio sottoset -euo pipefail.Scope. Toccati solo
.pair/adoption/**di pair +.github/workflows/ci.yml. Nessuna modifica al default del dataset.Dependencies & Related Work
setup-gates(Shared lint/format config guideline + setup-gates provisioning #232).Refs: #282