From 519ea2a10a29e17b6e9ad74e1dcaa0fa3703854c Mon Sep 17 00:00:00 2001 From: Leonid Svyatov Date: Tue, 11 Aug 2026 18:56:02 +0300 Subject: [PATCH] ci: baseline one skillspector false positive `oss-harden` cites GitLab's personal access tokens page as the source for the `read_api` and `api` scopes an audit run needs. Skillspector reads the page title as credential access and reports PE3 at HIGH, which carries the score from 34 to 52 and fails the scan. The suppression is a glob rule scoped to that rule, that file, and that phrase. A fingerprint would hash the line span and expire on the next edit to the file, returning the finding with nobody having re-read it. Every other finding still scores, so a new one still fails the build. --- .github/workflows/validate.yml | 2 +- .skillspector-baseline.yaml | 20 ++++++++++++++++++++ CONTRIBUTING.md | 4 ++-- 3 files changed, 23 insertions(+), 3 deletions(-) create mode 100644 .skillspector-baseline.yaml diff --git a/.github/workflows/validate.yml b/.github/workflows/validate.yml index 4d560a3..a971128 100644 --- a/.github/workflows/validate.yml +++ b/.github/workflows/validate.yml @@ -69,4 +69,4 @@ jobs: - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - uses: astral-sh/setup-uv@c771a70e6277c0a99b617c7a806ffedaca235ff9 # v9.0.0 - run: uv tool install git+https://github.com/NVIDIA/skillspector.git@a54947c307fe19a24a43db55f6148e181a987a67 - - run: skillspector scan ./skills/ --no-llm --format json + - run: skillspector scan ./skills/ --no-llm --format json --baseline .skillspector-baseline.yaml diff --git a/.skillspector-baseline.yaml b/.skillspector-baseline.yaml new file mode 100644 index 0000000..7b664e4 --- /dev/null +++ b/.skillspector-baseline.yaml @@ -0,0 +1,20 @@ +version: 1 + +# One suppression, and it stays one. Every other skillspector finding still +# scores, so the scan keeps failing the build when a new one arrives. +# +# A glob rule rather than a fingerprint: a fingerprint hashes the line span and +# the message, so it expires the next time somebody edits the file, and the +# finding returns without anybody having re-read it. + +rules: + - id: "PE3" + path: "oss-harden/SKILL.md" + message: "*access tokens*" + reason: >- + False positive on a documentation link. The skill cites GitLab's personal + access tokens page as the source for the read_api and api scopes an audit + run needs, and the scanner reads the page title as credential access. The + skill reads no credential file. Remove this rule if oss-harden ever tells + a reader to read a token from disk, because that is the finding it would + hide. diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index c513445..e36c6f4 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -38,11 +38,11 @@ bun run validate bash tests/test-check-drift.sh bash scripts/check-drift.sh bun scripts/check-ecosystems.mjs -skillspector scan ./skills/ --no-llm --format json +skillspector scan ./skills/ --no-llm --format json --baseline .skillspector-baseline.yaml cd site && bun run build ``` -`bun run typecheck` checks the repository's own TypeScript. `bun test` runs the validator's own test suite. `bun run validate` runs `skills/oss-skill/scripts/validate.mjs`, which checks every skill against R-SKL-01 through R-SKL-05: layout, frontmatter conformance, body size, the license field, and what a skill may ship as a script. `tests/test-check-drift.sh` is the test suite for `scripts/check-drift.sh`, which fails when a skill cites a rule ID that `skills/oss-audit/STANDARD.md` does not define, or when a rule names a skill that does not claim it. `scripts/check-ecosystems.mjs` reads `skills/oss-audit/ecosystems.json` and fails when a skill is missing a file for a roster ecosystem, when a file is missing a heading its skill declares, when a declared heading has nothing under it, or when a file does not end with a well-formed `Verified` line. `skillspector scan` checks the skills for prompt injection and other agent-facing risks. +`bun run typecheck` checks the repository's own TypeScript. `bun test` runs the validator's own test suite. `bun run validate` runs `skills/oss-skill/scripts/validate.mjs`, which checks every skill against R-SKL-01 through R-SKL-05: layout, frontmatter conformance, body size, the license field, and what a skill may ship as a script. `tests/test-check-drift.sh` is the test suite for `scripts/check-drift.sh`, which fails when a skill cites a rule ID that `skills/oss-audit/STANDARD.md` does not define, or when a rule names a skill that does not claim it. `scripts/check-ecosystems.mjs` reads `skills/oss-audit/ecosystems.json` and fails when a skill is missing a file for a roster ecosystem, when a file is missing a heading its skill declares, when a declared heading has nothing under it, or when a file does not end with a well-formed `Verified` line. `skillspector scan` checks the skills for prompt injection and other agent-facing risks. `.skillspector-baseline.yaml` suppresses one false positive and nothing else, so a new finding still fails the scan. Read it before adding a second entry: it says what would make each suppression wrong. `bun scripts/ecosystem-freshness.mjs` reports how recently each ecosystem file was checked against its sources, oldest first. It never gates, so it is not in the list above. `bun scripts/rule-freshness.mjs` does the same for rule sources.