Skip to content

The SBOM does not disclose what the JSON report discloses about its own coverage #25

Description

@richardmhope

Companion to #24, from the same assessment against CISA's 2026 Minimum Elements (docs/cisa-sbom-minimum-elements-2026.md on branch claude/tool-sbom-requirements-assessment-jkudvt). The 2026 revision replaces 2021's Depth element with Coverage, which asks an SBOM to state how complete its own inventory is.

The tool models this unusually well internally and then drops nearly all of it on the way to CycloneDX. Everything below is already known to ScanReport; the SBOM just does not say it.

1. Scope filtering is silent

DEFAULT_SCOPES excludes dev and test (src/icebergsca/core/scanner.py:54), and icebergsca sbom inherits that default (src/icebergsca/cli/main.py:388). The document records nothing about it.

So a project whose dev dependencies were filtered out is byte-indistinguishable from one that has none. This is the closest thing in the codebase to an unearned clean result: an inventory that omits packages without saying it omitted anything. It is also the cheapest to fix — the active scope set is right there in ScanOptions.

Worth deciding alongside it whether sbom should default to all scopes. An SBOM is an inventory rather than a risk assessment, and the argument for excluding dev dependencies from a findings report does not obviously carry over.

2. Per-manifest confidence is dropped

ManifestResult.approximate and from_lockfile (core/models.py:433-435) reach the JSON report (report/json_.py:117-118) and never reach the SBOM. A Maven graph — which CLAUDE.md is explicit about approximating, and which #16 showed is approximate in ways that change the reported version set — is presented in CycloneDX with exactly the same authority as a Cargo.lock read verbatim.

3. scan_failed, skipped and warnings are absent entirely

Three counters survive into metadata.properties (report/cyclonedx.py:94-106). The packages that could not be checked, the files that were recognised and not parsed, and every warning do not. A consumer reading the SBOM cannot learn that a lockfile failed to parse and we fell back to its manifest — which is precisely the case where the versions are declared rather than installed.

4. icebergsca:complete answers a different question than it appears to

is_complete (core/models.py:480) requires vulnerabilities_checked, so it is a statement about vulnerability coverage. In the SBOM it reads as a statement about the document. The practical effect: icebergsca sbom without --with-vulnerabilities always emits icebergsca:complete: false, because the OSV stage deliberately did not run.

That under-claims rather than over-claims, so it is the safe direction and not urgent — but it means the one field that looks like a completeness signal carries no information about the inventory, which is the thing an SBOM consumer is asking about. Either rename it to say what it measures, or add a separate inventory-coverage statement, but the two questions should not share a field.

Suggested shape

compositions with an aggregate of incomplete (tests/schemas/bom-1.6.schema.json:2224) covers items 1–3 in the spec's own vocabulary rather than in icebergsca: properties, which only we can read. Details that have no standard home — the active scope set, per-unit source-of-truth, unchecked package counts — can stay as properties beside it.

Since #24 also lands compositions, these two should be read together and probably implemented together, even if they are reviewed apart.

Done when

  • The document states which scopes were included, and that a filter was applied when one was
  • approximate / from_lockfile reach the SBOM per unit, not just the JSON report
  • Unchecked packages, skipped files and warnings are represented, not just counted
  • icebergsca:complete either says what it measures or is joined by an inventory-coverage statement
  • A test asserting an SBOM built from a dev-filtered scan does not read as a full inventory
  • references/json-report.md and the skill updated if any JSON key changes (tests/test_skill.py polices top-level keys)

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions