Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 13 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,19 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
and output says "lookup did not run" rather than "no vulnerabilities found" whenever that is
the truth.
- Exit codes: `0` completed (regardless of findings), `1` scan failed or partial, `2` usage
error. Findings alone never fail a build.
error, `3` `--fail-on` threshold met. Findings alone never fail a build.
- `--fail-on <severity>` gates a build on unsuppressed findings, checking scan completeness
first so it can never pass a scan that failed to look anything up.
- A triage file, `.icebergsca.toml`, records findings that have been assessed and accepted, so
gating is safe to leave switched on rather than being disabled the first week an unfixable
transitive advisory appears. Each entry requires a reason, matches exactly on advisory (or
alias) and package, and carries an expiry that defaults to 90 days. An accepted finding stays
in the report marked `accepted`, is counted by `summary.suppressed`, appears in SARIF as
dismissed rather than fixed, and returns when the entry expires. Rules that match nothing, or
have expired, are reported as warnings.
- JSON schema `1.1`: adds a top-level `suppressed` audit trail, `summary.suppressed`, and
`findings[].suppression`. Additive only — `summary.findings` and `by_severity` now count
active findings, which is unchanged for any scan with no ignore file.
- Maven graphs are marked approximate — they are reconstructed, not read, and we never shell
out to `mvn`. Each module of a multi-module build resolves against its own POM, so module
boundaries hold: one module's nearest-wins choice cannot decide another's versions, and each
Expand Down
26 changes: 25 additions & 1 deletion CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -146,6 +146,27 @@ schema validation in `tests/test_report_formats.py` against the official schemas

**Exit code 2 is usage, 1 is scan failure** — the reverse of the original plan, because Click
reserves 2 for usage errors and remapping it means overriding framework internals for nothing.
`--fail-on` gets its own code 3 rather than reusing 1: a pipeline that cannot tell "the scanner
broke" from "the scanner found something" ends up treating both as noise, which is the same
failure `|| true` represents.

**Suppression and gating shipped together, deliberately.** `--fail-on` alone turns a single
unfixable transitive advisory into a build that is red every week forever, and a job that is
always red gets `|| true` or gets deleted. The ignore file is what makes the gate survivable.
The safeguards in `core/triage.py` all follow from it being the one feature whose job is to
remove findings: a suppressed finding stays in `ScanReport.findings` marked rather than being
filtered out (so `table.py`'s `if not report.findings` branch cannot print "no known
vulnerabilities" for an all-suppressed scan — `test_a_fully_suppressed_report_never_reads_as_clean`
guards exactly that), `reason` is mandatory, expiry defaults rather than being optional, matching
is exact on advisory-or-alias plus package, and a rule matching nothing becomes a warning.

**`parse_ignore_file` is strict where every other parser is forgiving.** Ecosystem parsers skip
what they cannot read, because a third-party manifest is not ours to police. The ignore file is
the user's own and decides which vulnerabilities they stop seeing, so an unknown key is a
`ConfigError` rather than a silent skip.

**`SeverityLevel` is a `StrEnum`, so `<` compares strings** — `HIGH < LOW` is True. Use
`at_or_above()` or `.rank`; never compare members directly.

## Testing

Expand All @@ -168,7 +189,10 @@ and versions. This has caught two real bugs (a missing `coverage[toml]` extra tr

## Known gaps

- `--fail-on <severity>` and an ignore/triage file — the obvious next feature.
- The ignore file suppresses; it does not yet express "not exploitable" in VEX terms.
`cyclonedx.py` emits `analysis.detail` only, because `state` and `justification` are closed
enums of security claims a free-text reason cannot back. An optional validated `state` key on
the entry is the obvious next step.
- No SPDX output; CycloneDX only.
- No SBOM *ingest*.
- npm/yarn v1 lockfiles record no scope, so dev transitives are reported as runtime unless a
Expand Down
6 changes: 4 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -80,9 +80,11 @@ follows from this:
| `0` | Scan completed. Vulnerabilities may have been found — that is not an error |
| `1` | Scan failed, or completed only partially — some files or packages went unchecked |
| `2` | Usage error: bad flag, unknown format (Click's reserved code) |
| `3` | Only with `--fail-on`: the scan completed and found something at or above the threshold |

Severity-based CI gating (`--fail-on`) is planned; the report model already carries everything
needed for it.
`--fail-on <severity>` gates a build, and `.icebergsca.toml` records the findings you have
assessed and accepted so the gate is safe to leave switched on. An accepted finding stays in
the report, marked and counted separately — it is never silently dropped.

## Ecosystem support

Expand Down
33 changes: 31 additions & 2 deletions src/icebergsca/.agents/skills/icebergsca/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -78,6 +78,7 @@ icebergsca scan . --format json | jq '{
| `scan.vulnerabilities_checked` | `false` means the OSV lookup never ran. No conclusion can be drawn at all |
| `summary.unchecked_packages` | Above `0` means some packages could not be looked up. The result is partial |
| `scan.complete` | `false` means either of the above, or a file failed to parse |
| `summary.suppressed` | Above `0` means findings were **accepted, not fixed**. Say so; never fold them into a clean summary |

A correct summary:

Expand All @@ -93,6 +94,9 @@ elif not report["summary"]["findings"]:
verdict = f"No known vulnerabilities in {report['summary']['packages']} packages."
else:
verdict = f"{report['summary']['findings']} finding(s): {report['summary']['by_severity']}"

if report["summary"]["suppressed"]:
verdict += f" ({report['summary']['suppressed']} accepted via the ignore file)"
```

```python
Expand All @@ -115,11 +119,36 @@ icebergsca scan .; echo "exit=$?"
| `0` | The scan completed. Vulnerabilities may have been found — that is not an error |
| `1` | The scan failed, or completed only partially. Treat the report as incomplete |
| `2` | Usage error: unknown flag or format |
| `3` | Only with `--fail-on`: the scan completed and found something at or above the threshold |

Findings deliberately never change the exit code, so a non-zero exit always means *the tool
could not do its job*, not *your project has problems*. Do not use the exit code to decide
Findings deliberately never change the exit code unless `--fail-on` was passed, and even then
under a code of their own. So `1` always means *the tool could not do its job*, and `3` always
means *it did its job and you did not like the answer*. Do not use the exit code to decide
whether vulnerabilities exist — read `summary.findings`.

## Accepting a finding

Some findings cannot be fixed and should not be reported forever: an unfixable transitive
advisory, or one that is genuinely unreachable. `.icebergsca.toml` in the project root records
that decision.

```toml
[[ignore]]
advisory = "GHSA-6757-jp84-gxfx" # or the CVE — aliases match too
package = "pyyaml" # bare name, or a purl with or without a version
reason = "Unreachable: we never call yaml.load on untrusted input"
expires = 2026-10-27 # optional; defaults to 90 days out
```

What this does **not** do is hide anything. The finding stays in `findings` with a
`suppression` object, `summary.suppressed` counts it, SARIF marks it dismissed rather than
fixed, and it stops tripping `--fail-on`. When reporting a scan, say how many findings were
accepted and by which rule — an accepted finding is a decision somebody made, not a solved
problem, and it comes back when the entry expires.

A rule that matches nothing, or has expired, produces a `warnings` entry. Surface those: they
mean either a typo or an entry left behind after the upgrade that fixed it.

## Reading a finding

```json
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,16 +4,54 @@

Findings never change the exit code. A non-zero exit means the scan itself did not complete.

Findings never change the exit code unless you ask them to with `--fail-on`.

| Code | Meaning | Pipeline response |
|---|---|---|
| `0` | Scan completed | Read `summary.findings` to decide what to do |
| `1` | Scan failed or partial | Fail the job — the result is not trustworthy |
| `2` | Usage error | Fix the invocation |
| `3` | `--fail-on` threshold met | Fail the job — the scan worked, the result did not pass |

`1` and `3` are separate on purpose. A pipeline that cannot tell "the scanner broke" from "the
scanner found something" ends up treating both as noise.

```bash
icebergsca scan . --fail-on critical
```

The gate checks completeness first, so it can never pass a scan that failed to look anything
up, and it ignores findings accepted in the ignore file. Do not add `|| true`: that is what
the ignore file is for.

## Accepting a finding

This is deliberate: a scanner that exits non-zero on findings gets `|| true` appended within a
week, at which point genuine tool failures also go unnoticed.
The scheduled run below matters because most new findings arrive when an advisory is published,
not when the code changes. Which means one unfixable transitive advisory would otherwise turn
the build red every week forever — and a job that is red every week gets deleted.

To gate on severity today, read the JSON:
`.icebergsca.toml` in the project root records the assessment instead:

```toml
[[ignore]]
advisory = "GHSA-6757-jp84-gxfx" # or the CVE — aliases are matched too
package = "pyyaml" # bare name, or a purl, optionally versioned
reason = "Unreachable: we never call yaml.load on untrusted input"
expires = 2026-10-27 # optional; defaults to 90 days from the scan
```

Point elsewhere with `--ignore-file`. What it does not do is hide anything: the finding stays
in the report with a `suppression` object, `summary.suppressed` counts it, SARIF marks it
dismissed rather than fixed, and `--fail-on` skips it. When the entry expires the finding comes
back — expiry never fails a scan by itself.

A rule that matches nothing, or one past its date, lands in `warnings`. Watch for those: they
mean a typo, or an entry left behind after the upgrade that already fixed it.

### Gating without `--fail-on`

Reading the JSON still works, and is the way to gate on something more specific. Check
`scan.complete` **first** — gating on findings alone would pass a scan that checked nothing:

```bash
icebergsca scan . --format json --output report.json
Expand All @@ -22,10 +60,7 @@ jq -e '(.summary.by_severity.critical // 0) == 0' report.json > /dev/null \
|| { echo "critical vulnerabilities found"; exit 1; }
```

Check `scan.complete` **first**. Gating on findings alone would pass a scan that failed to
check anything.

A built-in `--fail-on <severity>` flag is planned but not yet implemented.
`summary.by_severity` counts active findings only, so accepted ones are already excluded.

## GitHub code scanning

Expand Down
41 changes: 37 additions & 4 deletions src/icebergsca/.agents/skills/icebergsca/references/json-report.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,13 +12,14 @@ icebergsca scan . --format json --output report.json

```json
{
"schema_version": "1.0",
"schema_version": "1.1",
"tool": { "name": "icebergsca", "version": "0.1.0" },
"scan": { ... },
"summary": { ... },
"manifests": [ ... ],
"dependencies": [ ... ],
"findings": [ ... ],
"suppressed": [ ... ],
"unchecked_packages": [ ... ],
"skipped": [ ... ],
"warnings": [ ... ]
Expand Down Expand Up @@ -53,6 +54,7 @@ it means the tool did its job.
{
"dependencies": 21, "direct": 9, "transitive": 12, "packages": 21,
"findings": 12,
"suppressed": 2,
"by_severity": { "critical": 3, "high": 4, "medium": 4, "low": 1 },
"unchecked_packages": 0,
"skipped_files": 0
Expand All @@ -63,7 +65,12 @@ it means the tool did its job.
manifests appears twice. `packages` counts unique packages. Use `packages` when reporting
"we checked N packages".

`by_severity` omits levels with no findings and is ordered most severe first.
`findings` counts only findings nobody has accepted; `suppressed` counts the rest. The total in
the `findings` array is the two added together. **A suppressed finding is not a fixed one** —
somebody recorded a reason for accepting it and a date to revisit. Say so when reporting.

`by_severity` omits levels with no findings and is ordered most severe first. It counts active
findings only, for the same reason.

## `manifests`

Expand Down Expand Up @@ -144,11 +151,12 @@ One entry per ecosystem-and-directory scan unit.
"direct": true,
"introduced_by": [
{ "path": "requirements.txt", "line": 3, "scope": "runtime", "direct": true }
]
],
"suppression": null
}
```

Sorted most severe first, then by package name.
Sorted active first, then most severe, then by package name.

* `severity.level`: `critical`, `high`, `medium`, `low`, `none`, `unknown`. `unknown` means the
advisory carries no rating — distinct from `none`, which is a scored zero.
Expand All @@ -159,6 +167,31 @@ Sorted most severe first, then by package name.
removal, not upgrade.
* One CVE reported by several advisory databases is merged into a **single** finding, keeping
whichever record carried the severity and fix version. The other IDs appear in `aliases`.
* `suppression` is `null` unless an ignore-file entry accepted the finding, in which case it
carries `reason`, `expires` and `rule`. **Read this field.** A suppressed finding is still in
this array — it is deliberately not removed — so code that ignores `suppression` counts it as
outstanding, and code that filters on it must not describe the result as "no vulnerabilities".

## `suppressed`

The ignore rules that fired, and what each accepted. This is an audit trail, not a second copy
of the findings — the per-finding state is `findings[].suppression`.

```json
"suppressed": [
{
"rule": "GHSA-6757-jp84-gxfx@pkg:pypi/pyyaml",
"reason": "Unreachable: we never call yaml.load on untrusted input",
"expires": "2026-10-27",
"findings": [{ "advisory": "GHSA-6757-jp84-gxfx", "purl": "pkg:pypi/pyyaml@5.1" }]
}
]
```

A rule that matched nothing, or one that has expired, does not appear here — it produces a
`warnings` entry instead, because a rule doing no work is usually a typo or an entry left behind
after the upgrade that fixed it. An expired entry stops suppressing, so the finding returns to
the active count.

## `unchecked_packages`, `skipped` and `warnings`

Expand Down
Loading