From fd07e0ebe36589328937573b89c6f2a31910f7e7 Mon Sep 17 00:00:00 2001 From: Brad House Date: Wed, 27 May 2026 12:23:41 +0000 Subject: [PATCH 1/2] [sbom]: opt-in SBOM generation, vulnerability scanning, and CI integration MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Adds opt-in CycloneDX 1.6 SBOM generation, SPDX 2.3 conversion, SLSA v1.0 in-toto provenance, and SBOM-based vulnerability scanning for SONiC images. Default builds are unchanged; SBOM emission is opt-in via ENABLE_SBOM=y at build time. Design summary ============== Hybrid SBOM aggregation pulls from four independent sources and deduplicates by PURL + (name, version, arch) + (name, normalized version, arch). Recipe-emit wins ties because it carries pedigree and patch-set provenance that scanners can't recover from the shipped binary alone. * Recipe-emit fragments: slave.mk hooks every recipe that produces an artifact in scope (SONIC_DPKG_DEBS, SONIC_MAKE_DEBS, SONIC_DERIVED_DEBS, SONIC_EXTRA_DEBS, SONIC_ONLINE_DEBS, SONIC_PYTHON_STDEB_DEBS, SONIC_PYTHON_WHEELS, docker-image-save). scripts/sbom_fragment.py detects four fork-of-upstream patterns (sonic-net submodules, dget+patches, nested submodule a la FRR, direct-upstream-submodule + sidecar patches) and emits pedigree.ancestors[] accordingly. Each fragment also carries the recipe's $(pkg)_DEPENDS and $(pkg)_RDEPENDS declarations as sonic:build_depends / sonic:runtime_depends string properties. * Observation harvest: src/sonic-build-hooks/scripts/collect_version_files is extended (gated on ENABLE_SBOM=y) to emit a 9-column TSV per container (Package, Version, Architecture, Source, SourceVersion, Maintainer, Homepage, Filename, SHA256) using one bulk `xargs apt-cache show` per container, plus snapshots of /usr/share/doc/*/copyright and language lockfiles (go.sum, package-lock.json, pnpm-lock.yaml, yarn.lock). Cargo.lock is NOT harvested here — see Rust crate visibility below for the authoritative source. * Rust + Go + Python attribution at the .deb level: every SONiC- built .deb gets `dpkg-deb -x` extracted into a tempdir at fragment-emit time, with three harvesters running against the unpacked tree: - Rust: rust-audit-info reads the .dep-v0 ELF section that cargo-auditable embedded at compile time. Per-binary precision on the crate set actually linked into the final binary; emits pkg:cargo/@ components. - Go: `go version -m ` parses the .go.buildinfo section embedded by `go build`. Replacement directives (`=>`) honored. Emits pkg:golang/@ components. - Python: walk for *.dist-info/METADATA files (the modern wheel- install marker), parse Name+Version, emit pkg:pypi/@ with PEP 503 name normalization. SONiC .debs today install Python content into /usr/lib/python3/dist-packages/ without dist-info, so this harvester is dormant on current builds — it activates automatically when any recipe starts bundling a venv or pip-installing wheels into debian//. Each emitted component carries sonic:fragment_kind=recipe-emit- {rust,go,python} and sonic:source_deb=, so a consumer can trace each crate/module/distribution back to the exact artifact that shipped it. * Lockfile expansion: scripts/sbom_parse_lockfiles.py parses lockfiles harvested by collect_version_files (go.sum, package-lock.json, pnpm-lock.yaml, yarn.lock) for ecosystems the per-.deb introspection above doesn't already cover. * Image-level aggregator: scripts/build_sbom.py merges fragments + observations + lockfiles + scanner output. Version normalization strips epoch prefixes (1:) and downstream suffixes (+fips, +sonic, +bN, +debNuM) so the recipe-emit and observation entries collapse to one component. Emits CycloneDX `dependencies[]` edges from each fragment's bom-ref to its declared build/runtime deps (resolved through sonic:build_depends / sonic:runtime_depends → sibling fragment bom-refs), and from each .deb to its attributed Rust crates / Go modules / Python distributions. Unresolved declared-deps land on the component as a sonic:unresolved_deps property for audit. * License resolution: scripts/sbom_resolve_licenses.py parses DEP-5 debian/copyright; falls back to licensecheck for free-form copyrights. A bundled scripts/sbom_license_map.json (~100 entries) translates Debian License header strings to SPDX identifiers. Maintenance instructions in README.sbom.md. * SOURCE_DATE_EPOCH is set from the HEAD git commit time in Makefile.work and exported through to the slave container, so two builds of the same source produce byte-identical SBOMs. * Strict mode (SBOM_STRICT=y, default when ENABLE_SBOM=y) fails the build when a critical input is missing (host rootfs, declared installer dockers, scanner binary). Per-recipe sbom_emit_fragment and sbom_emit_per_container honor SBOM_STRICT consistently: when set, fragment-emit failures abort the recipe; when n, failures are swallowed (defensive against script bugs only). Output filenames track the actual installer artifact via the SBOM_TARGET_ARTIFACT env var. The slave.mk recipe iterates over $*_MACHINE + $*_DEPENDENT_MACHINE and derives the per-variant artifact name as $(subst $($*_MACHINE),$(dep_machine),$*), so each asic variant (and each installer format) gets its own sidecar: target/sonic-broadcom.bin.cdx.json + .spdx.json + .intoto.json target/sonic-broadcom-dnx.bin.cdx.json + ... target/sonic-broadcom-legacy-th.bin.cdx.json + ... target/sonic-aboot-broadcom.swi.cdx.json + ... target/sonic-vs.img.gz.cdx.json + ... Vulnerability scanning ====================== scripts/sbom_vuln_scan.py runs grype against a CycloneDX SBOM, applies VEX statements under vex/, and emits a CycloneDX VEX-annotated report plus a human-readable table. Policy is configurable (--min-severity, --fail-on, --ignore-unfixed). Standalone Python; not a make target. * scripts/sbom_extract_vex_from_patches.py scans src/*/patches/ for CVE markers and auto-emits OpenVEX v0.2.0 statements with the patch as evidence. vex/auto/ is gitignored and regenerated by scripts/build_sbom.sh so the suppression set always tracks src/*/patches/. * The human-readable table lists only actionable findings (those with an upstream fix available); not-fixed and wont-fix entries are summarized at the top so the table is action-oriented. The --fail-on gate is restricted to actionable findings so CI doesn't fail on things the team can't act on. The CycloneDX JSON sidecar preserves grype's exact fix.state per finding so downstream tooling can still distinguish not-fixed from wont-fix. Azure Pipelines integration =========================== * .azure-pipelines/azure-pipelines-build.yml: ENABLE_SBOM=y appended to BUILD_OPTIONS so existing make $(BUILD_OPTIONS) ... lines pick it up without per-line edits. * .azure-pipelines/build-template.yml: ENABLE_SBOM=y propagation + a new inline SBOM vulnerability scan step iterating over each aggregate SBOM (sonic-*.bin / .swi / .img.gz, docker-*.gz.sbom) with results published as sbom-vuln-scan-results.. Uses ##[group]/##[endgroup] for collapsible per-SBOM log sections. * azure-pipelines.yml (top level): the prior Trivy vulnerability scan (docker-ptf) job from PR #27079 is replaced with a unified SBOM-based vulnerability scan (all artifacts) running in its own VulnScan stage that dependsOn both BuildVS and Build, so the scan only fires once every *.cdx.json sidecar this pipeline produces is on the artifact server. Downloads only *.cdx.json sidecars (a few MB instead of multi-GB images) and scans every aggregate SBOM the build produced. continueOnError: true mirrors the original. * .azure-pipelines/docker-sonic-mgmt.yml: same trivy -> sbom replacement for the docker-sonic-mgmt pipeline. Slave Dockerfiles ================= Every slave installs two cargo-derived tools, both with the same CARGO_HOME=$RUST_ROOT install pattern so they land in /usr/.cargo/bin/ (PATH for all users) rather than /root/.cargo/bin/: cargo-auditable@0.6.4 embeds resolved Cargo.lock into the .dep-v0 ELF section of every cargo-built binary. Reached via the transparent /usr/local/bin/cargo shim (files/build/cargo-wrapper, staged into each slave's buildinfo/ directory by scripts/prepare_docker_buildinfo.sh). The shim fails hard with an actionable error if cargo-auditable is missing. rust-audit-info@0.5.4 reads .dep-v0 back out at SBOM-emit time. Invoked by sbom_fragment.py on every ELF inside every SONiC-built .deb. Go and Python need no new tool installs (golang-go is already present for SONiC's Go builds; Python dist-info parsing is pure Python). Build caching ============= Two optimizations avoid redundant work in the ENABLE_SBOM=y build path. Both preserve full scanner coverage and SBOM completeness — only redundant scans are skipped. * Slave-image cache stability: prepare_docker_buildinfo.sh emits the slave's Dockerfile from a constant `ARG ENABLE_SBOM=n` / `ENV ENABLE_SBOM=$ENABLE_SBOM` pattern, with --build-arg ENABLE_SBOM=$(ENABLE_SBOM) passed at every docker build site that uses the prelude (Makefile.work slave-base build, slave.mk container build, container -dbg build). The Dockerfile text is invariant of the ENABLE_SBOM value, so SLAVE_BASE_TAG (a sha1 of Dockerfile + sources.list.* + buildinfo/versions/, Makefile.work:262-267) stays stable across runs that toggle ENABLE_SBOM. Without this, `make configure PLATFORM=` (ENABLE_SBOM typically unset) followed by `make ENABLE_SBOM=y target/sonic-.bin` ran the slave docker build twice with different ENV ENABLE_SBOM values, adding ~30 min of redundant work per two-step invocation. After this change the runtime value still flows in via --build-arg, so pre_run_buildinfo / post_run_buildinfo / collect_version_files inside the image see the operator's chosen value whether sourced from rules/config or a command-line override. * Scanner cross-check cache: build_sbom.py memoizes syft's output per archive under target/sbom-tools/syft-cache/-.json, keyed by SHA-256 of the input archive. Across the per-variant aggregator runs (broadcom / broadcom-dnx / broadcom-legacy-th) the ~28 docker .gz files that are identical across variants are now scanned once each; variants 2 and 3 short-circuit on cache hit. dir: scans (host rootfs) are bypassed because fsroot-/ differs per variant and a directory-tree hash would be expensive. The cache lives under target/, so `make reset` invalidates it naturally — no stale entries possible across resets. SHA-256 keying + syft's deterministic output for identical input mean cached and freshly-scanned results are byte-identical; sbom_diff.py's existing reproducibility check covers this. Documentation ============= README.sbom.md (~570 lines) covers configuration, scope, architecture, license resolution and license-map maintenance, tools, reproducibility, attestation/signing notes, vulnerability scanning quick start, VEX workflow, verification, known limitations, and a file map. Linked from README.md. Verification ============ End-to-end Broadcom build (make ENABLE_SBOM=y target/sonic-broadcom.bin), exit 0, all 3 asic variants produced with full SBOM triplet: Variant .bin .cdx.json components sonic-broadcom.bin 1.31 GB 46 MB 56,605 sonic-broadcom-dnx.bin 1.18 GB 46 MB 56,600 sonic-broadcom-legacy-th.bin 1.18 GB 46 MB 56,579 Phase spot-checks on broadcom.bin: 188 CycloneDX dependencies[] entries (kernel-module + .deb dependency graph) 576 recipe-emit-rust components across 7 SONiC Rust .debs (swss=190, dash-ha=166, supervisord-utilities-rs=77, ctrmgrd-rs=61, host-services-rs=47, nettools=32, syslog-counter=3) 67 recipe-emit-go components across 2 .debs (sonic-mgmt- framework=39, sonic-gnmi=28) 0 recipe-emit-python components (no SONiC .deb ships dist-info; harvester dormant, activates automatically once a recipe does) Scanner cache hits firing as expected: ~44 HITs per build across variants 2+3, dropping the aggregator phase from ~230s (uncached primary) to ~110s (cached). Measured during development via temporary per-phase timing instrumentation that was removed before final submission — only the cache itself is shipping. Signed-off-by: Brad House --- .azure-pipelines/azure-pipelines-build.yml | 4 + .azure-pipelines/build-template.yml | 83 +- .azure-pipelines/docker-sonic-mgmt.yml | 66 +- .gitignore | 5 + Makefile.work | 19 + README.md | 12 + README.sbom.md | 664 ++++++++ azure-pipelines.yml | 133 +- build_debian.sh | 6 + files/build/cargo-wrapper | 66 + rules/config | 32 + scripts/build_sbom.py | 1466 +++++++++++++++++ scripts/build_sbom.sh | 22 + scripts/install_sbom_tool.sh | 237 +++ scripts/prepare_docker_buildinfo.sh | 11 + scripts/sbom_diff.py | 177 ++ scripts/sbom_emit_provenance.py | 357 ++++ scripts/sbom_extract_vex_from_patches.py | 284 ++++ scripts/sbom_fragment.py | 1115 +++++++++++++ scripts/sbom_license_map.json | 120 ++ scripts/sbom_parse_lockfiles.py | 478 ++++++ scripts/sbom_resolve_licenses.py | 316 ++++ scripts/sbom_vuln_diff.py | 165 ++ scripts/sbom_vuln_scan.py | 491 ++++++ slave.mk | 107 ++ sonic-slave-bookworm/Dockerfile.j2 | 27 + sonic-slave-bullseye/Dockerfile.j2 | 27 + sonic-slave-buster/Dockerfile.j2 | 27 + sonic-slave-trixie/Dockerfile.j2 | 27 + .../scripts/collect_version_files | 101 ++ vex/README.md | 189 +++ 31 files changed, 6766 insertions(+), 68 deletions(-) create mode 100644 README.sbom.md create mode 100755 files/build/cargo-wrapper create mode 100755 scripts/build_sbom.py create mode 100755 scripts/build_sbom.sh create mode 100755 scripts/install_sbom_tool.sh create mode 100755 scripts/sbom_diff.py create mode 100755 scripts/sbom_emit_provenance.py create mode 100755 scripts/sbom_extract_vex_from_patches.py create mode 100755 scripts/sbom_fragment.py create mode 100644 scripts/sbom_license_map.json create mode 100755 scripts/sbom_parse_lockfiles.py create mode 100755 scripts/sbom_resolve_licenses.py create mode 100755 scripts/sbom_vuln_diff.py create mode 100755 scripts/sbom_vuln_scan.py create mode 100644 vex/README.md diff --git a/.azure-pipelines/azure-pipelines-build.yml b/.azure-pipelines/azure-pipelines-build.yml index fb760ea05e1..5e88584df22 100644 --- a/.azure-pipelines/azure-pipelines-build.yml +++ b/.azure-pipelines/azure-pipelines-build.yml @@ -137,6 +137,10 @@ jobs: if echo $(GROUP_NAME) | grep mellanox; then BUILD_OPTIONS="$BUILD_OPTIONS INCLUDE_DHCP_SERVER=y" fi + # ENABLE_SBOM=y emits CycloneDX SBOMs alongside each .bin / + # .img / .swi / .tar and standalone SBOMs for docker-ptf / + # docker-ptf-sai / docker-sonic-mgmt when those are built. + BUILD_OPTIONS="$BUILD_OPTIONS ENABLE_SBOM=y" if [ $(GROUP_NAME) == pensando ]; then make $BUILD_OPTIONS target/sonic-pensando.tar elif [ $(GROUP_NAME) == vs ]; then diff --git a/.azure-pipelines/build-template.yml b/.azure-pipelines/build-template.yml index f76d8c50667..c691b614458 100644 --- a/.azure-pipelines/build-template.yml +++ b/.azure-pipelines/build-template.yml @@ -83,54 +83,115 @@ jobs: sudo apt-get install -y acl export DOCKER_DATA_ROOT_FOR_MULTIARCH=/data/march/docker CACHE_OPTIONS="SONIC_DPKG_CACHE_METHOD=${{ parameters.cache_mode }} SONIC_DPKG_CACHE_SOURCE=/nfs/dpkg_cache/${{ parameters.platform }}" - ENABLE_DOCKER_BASE_PULL=y make configure PLATFORM=${{ parameters.platform }} PLATFORM_ARCH=${{ parameters.platform_arch }} + # ENABLE_SBOM=y emits CycloneDX SBOMs alongside every .bin and + # standalone SBOMs for docker-ptf / docker-ptf-sai. + ENABLE_DOCKER_BASE_PULL=y ENABLE_SBOM=y make configure PLATFORM=${{ parameters.platform }} PLATFORM_ARCH=${{ parameters.platform_arch }} trap "sudo rm -rf fsroot" EXIT if [ ${{ parameters.platform }} == vs ]; then if [ ${{ parameters.dbg_image }} == true ]; then - make USERNAME=admin $CACHE_OPTIONS SONIC_BUILD_JOBS=$(nproc) INSTALL_DEBUG_TOOLS=y target/sonic-vs.img.gz && \ + make USERNAME=admin $CACHE_OPTIONS ENABLE_SBOM=y SONIC_BUILD_JOBS=$(nproc) INSTALL_DEBUG_TOOLS=y target/sonic-vs.img.gz && \ mv target/sonic-vs.img.gz target/sonic-vs-dbg.img.gz fi - make USERNAME=admin $CACHE_OPTIONS SONIC_BUILD_JOBS=$(nproc) target/docker-sonic-vs.gz target/sonic-vs.img.gz target/docker-ptf.gz - make USERNAME=admin $CACHE_OPTIONS SONIC_BUILD_JOBS=$(nproc) target/docker-ptf-sai.gz + make USERNAME=admin $CACHE_OPTIONS ENABLE_SBOM=y SONIC_BUILD_JOBS=$(nproc) target/docker-sonic-vs.gz target/sonic-vs.img.gz target/docker-ptf.gz + make USERNAME=admin $CACHE_OPTIONS ENABLE_SBOM=y SONIC_BUILD_JOBS=$(nproc) target/docker-ptf-sai.gz elif [ ${{ parameters.platform }} == alpinevs ]; then - make USERNAME=admin SONIC_BUILD_JOB=2 $CACHE_OPTIONS target/sonic-alpinevs.img.gz + make USERNAME=admin SONIC_BUILD_JOB=2 $CACHE_OPTIONS ENABLE_SBOM=y target/sonic-alpinevs.img.gz else if [ ${{ parameters.dbg_image }} == true ]; then - make USERNAME=admin $CACHE_OPTIONS SONIC_BUILD_JOBS=$(nproc) INSTALL_DEBUG_TOOLS=y target/sonic-${{ parameters.platform }}.bin && \ + make USERNAME=admin $CACHE_OPTIONS ENABLE_SBOM=y SONIC_BUILD_JOBS=$(nproc) INSTALL_DEBUG_TOOLS=y target/sonic-${{ parameters.platform }}.bin && \ mv target/sonic-${{ parameters.platform }}.bin target/sonic-${{ parameters.platform }}-dbg.bin fi if [ ${{ parameters.swi_image }} == true ]; then - make USERNAME=admin $CACHE_OPTIONS SONIC_BUILD_JOBS=$(nproc) ENABLE_IMAGE_SIGNATURE=y target/sonic-aboot-${{ parameters.platform }}.swi + make USERNAME=admin $CACHE_OPTIONS ENABLE_SBOM=y SONIC_BUILD_JOBS=$(nproc) ENABLE_IMAGE_SIGNATURE=y target/sonic-aboot-${{ parameters.platform }}.swi fi if [ ${{ parameters.sync_rpc_image }} == true ]; then - make USERNAME=admin $CACHE_OPTIONS SONIC_BUILD_JOBS=$(nproc) ENABLE_SYNCD_RPC=y target/docker-syncd-${{ parameters.platform_short }}-rpc.gz + make USERNAME=admin $CACHE_OPTIONS ENABLE_SBOM=y SONIC_BUILD_JOBS=$(nproc) ENABLE_SYNCD_RPC=y target/docker-syncd-${{ parameters.platform_short }}-rpc.gz # workaround for issue in rules/sairedis.dep, git ls-files will list un-exist files for cache pushd ./src/sonic-sairedis/SAI git stash popd if [ ${{ parameters.platform }} == broadcom ]; then - make USERNAME=admin $CACHE_OPTIONS SONIC_BUILD_JOBS=$(nproc) ENABLE_SYNCD_RPC=y SAITHRIFT_V2=y target/docker-saiserverv2-brcm.gz + make USERNAME=admin $CACHE_OPTIONS ENABLE_SBOM=y SONIC_BUILD_JOBS=$(nproc) ENABLE_SYNCD_RPC=y SAITHRIFT_V2=y target/docker-saiserverv2-brcm.gz pushd ./src/sonic-sairedis/SAI git stash popd fi if [ ${{ parameters.platform }} == barefoot ]; then - make USERNAME=admin $CACHE_OPTIONS SONIC_BUILD_JOBS=$(nproc) SAITHRIFT_V2=y ENABLE_SYNCD_RPC=y target/docker-saiserverv2-bfn.gz + make USERNAME=admin $CACHE_OPTIONS ENABLE_SBOM=y SONIC_BUILD_JOBS=$(nproc) SAITHRIFT_V2=y ENABLE_SYNCD_RPC=y target/docker-saiserverv2-bfn.gz pushd ./src/sonic-sairedis/SAI git stash popd fi fi - make USERNAME=admin $CACHE_OPTIONS SONIC_BUILD_JOBS=$(nproc) target/sonic-${{ parameters.platform }}.bin + make USERNAME=admin $CACHE_OPTIONS ENABLE_SBOM=y SONIC_BUILD_JOBS=$(nproc) target/sonic-${{ parameters.platform }}.bin fi displayName: 'Build sonic image' + - script: | + set +e + # Run an SBOM-based vulnerability scan against each SBOM the + # build produced in this pipeline: the platform .bin/.img/.swi + # plus the docker-ptf / docker-ptf-sai test containers when + # the VS group built them. docker-sonic-mgmt is scanned in + # its own pipeline (docker-sonic-mgmt.yml) and is not built + # here. VEX statements under vex/ suppress CVEs that SONiC + # patches fix. Mirrors the legacy trivy policy: MEDIUM+ + # severity, fixed-only, fail on remaining unsuppressed + # findings. + OVERALL_RC=0 + mkdir -p $(Build.ArtifactStagingDirectory)/sbom-vuln + shopt -s nullglob + # Aggregate SBOMs the build emitted. SONiC produces several + # installer formats — .bin (ONIE), .swi (Arista aboot), + # .img.gz (VS/VPP) — plus standalone test-container SBOMs + # (note the .sbom. infix on the latter, which distinguishes + # them from per-container recipe fragments at + # target/docker-*.gz.cdx.json). + for sbom in \ + target/sonic-*.bin.cdx.json \ + target/sonic-*.swi.cdx.json \ + target/sonic-*.img.gz.cdx.json \ + target/docker-*.gz.sbom.cdx.json + do + # Derive a short name for this scan's outputs. Test-container + # SBOMs include a `.sbom` infix on disk (e.g. docker-ptf.gz.sbom.cdx.json) + # to distinguish them from per-container recipe fragments + # (docker-ptf.gz.cdx.json); strip it so the per-artifact + # output names match the underlying artifact extension: + # sonic-broadcom.bin.cdx.json -> sonic-broadcom.bin{.txt,.cdx.json} + # docker-ptf.gz.sbom.cdx.json -> docker-ptf.gz{.txt,.cdx.json} + name=$(basename "$sbom" .cdx.json) + name="${name%.sbom}" + # Azure Pipelines log-folding directive — wrap each SBOM's + # output so the log view shows a collapsible section per + # scan rather than one giant concatenated stream. + echo "##[group]SBOM vuln scan: $name" + python3 scripts/sbom_vuln_scan.py \ + --vex vex \ + --min-severity medium \ + --fail-on medium \ + --format both \ + --output "$(Build.ArtifactStagingDirectory)/sbom-vuln/${name}.cdx.json" \ + "$sbom" | tee "$(Build.ArtifactStagingDirectory)/sbom-vuln/${name}.txt" + RC=${PIPESTATUS[0]} + if [ "$RC" != "0" ]; then OVERALL_RC=$RC; fi + echo "##[endgroup]" + done + exit $OVERALL_RC + condition: succeededOrFailed() + continueOnError: true + displayName: 'SBOM vulnerability scan' - template: cleanup.yml - publish: $(System.DefaultWorkingDirectory)/ artifact: sonic-buildimage.${{ parameters.platform }} displayName: "Archive sonic image" + - publish: $(Build.ArtifactStagingDirectory)/sbom-vuln + artifact: sbom-vuln-scan-results.${{ parameters.platform }} + displayName: "Archive SBOM vuln-scan results" + condition: always() + continueOnError: true - script: | set -x find target -name "*.log" | xargs -I{} cp {} $(Build.ArtifactStagingDirectory)/ diff --git a/.azure-pipelines/docker-sonic-mgmt.yml b/.azure-pipelines/docker-sonic-mgmt.yml index 590cb731f0c..3a7d0ffda82 100644 --- a/.azure-pipelines/docker-sonic-mgmt.yml +++ b/.azure-pipelines/docker-sonic-mgmt.yml @@ -55,8 +55,8 @@ stages: set -xe git submodule update --init --recursive -- src/sonic-platform-daemons src/sonic-genl-packet src/sonic-sairedis src/ptf src/sonic-device-data src/sonic-dash-api - make NOBUSTER=1 NOBULLSEYE=1 SONIC_BUILD_JOBS=$(nproc) DEFAULT_CONTAINER_REGISTRY=publicmirror.azurecr.io ENABLE_DOCKER_BASE_PULL=y configure PLATFORM=generic DOCKER_BUILDKIT=0 - make -f Makefile.work BLDENV=bookworm SONIC_BUILD_JOBS=$(nproc) DEFAULT_CONTAINER_REGISTRY=publicmirror.azurecr.io ENABLE_DOCKER_BASE_PULL=y target/docker-sonic-mgmt.gz + make NOBUSTER=1 NOBULLSEYE=1 SONIC_BUILD_JOBS=$(nproc) DEFAULT_CONTAINER_REGISTRY=publicmirror.azurecr.io ENABLE_DOCKER_BASE_PULL=y ENABLE_SBOM=y configure PLATFORM=generic DOCKER_BUILDKIT=0 + make -f Makefile.work BLDENV=bookworm SONIC_BUILD_JOBS=$(nproc) DEFAULT_CONTAINER_REGISTRY=publicmirror.azurecr.io ENABLE_DOCKER_BASE_PULL=y ENABLE_SBOM=y target/docker-sonic-mgmt.gz cp -r target $(Build.ArtifactStagingDirectory)/target env: REGISTRY_SERVER: ${{ parameters.registry_url }} @@ -92,8 +92,8 @@ stages: - name: BUILD_BRANCH value: $(Build.SourceBranchName) jobs: - - job: trivy_scan - displayName: "[OPTIONAL] Trivy vulnerability scan (docker-sonic-mgmt)" + - job: sbom_vuln_scan + displayName: "[OPTIONAL] SBOM-based vulnerability scan (docker-sonic-mgmt)" pool: sonic-ubuntu-1c continueOnError: true timeoutInMinutes: 30 @@ -103,34 +103,46 @@ stages: fetchDepth: 1 - download: current artifact: docker-sonic-mgmt - displayName: "Download docker-sonic-mgmt artifact" + displayName: "Download docker-sonic-mgmt artifact (includes SBOM)" - script: | set -ex - TRIVY_VERSION="0.70.0" - curl -fLO https://github.com/aquasecurity/trivy/releases/download/v${TRIVY_VERSION}/trivy_${TRIVY_VERSION}_Linux-64bit.deb - sudo apt install -y ./trivy_${TRIVY_VERSION}_Linux-64bit.deb - trivy --version - displayName: "Install Trivy" + sudo apt-get install -y python3-pip jq + displayName: "Install dependencies" - script: | set -x - trivy image \ - --input $(Pipeline.Workspace)/docker-sonic-mgmt/target/docker-sonic-mgmt.gz \ - --severity MEDIUM,HIGH,CRITICAL \ - --ignore-unfixed \ - --exit-code 1 \ - --format table \ - --no-progress \ - --timeout 15m \ - --output $(Build.ArtifactStagingDirectory)/trivy-sonic-mgmt.txt - TRIVY_EXIT=$? + SBOM=$(Pipeline.Workspace)/docker-sonic-mgmt/target/docker-sonic-mgmt.gz.sbom.cdx.json + if [ ! -f "$SBOM" ]; then + echo "::error::SBOM not found at $SBOM — was the build invoked with ENABLE_SBOM=y?" + ls -la $(Pipeline.Workspace)/docker-sonic-mgmt/target/ | head -30 + exit 1 + fi + echo "SBOM: $SBOM" + echo "Component count: $(jq '.components | length' "$SBOM")" + displayName: "Verify SBOM artifact is present" + - script: | + set -x + # Mirror the prior trivy policy: report MEDIUM/HIGH/CRITICAL, + # filter to fixed CVEs only, fail the job on any unsuppressed + # finding. VEX statements under vex/ are applied first. + REPORT=$(Build.ArtifactStagingDirectory)/sbom-vuln-sonic-mgmt + python3 scripts/sbom_vuln_scan.py \ + --vex vex \ + --min-severity medium \ + --fail-on medium \ + --format both \ + --output "${REPORT}.cdx.json" \ + $(Pipeline.Workspace)/docker-sonic-mgmt/target/docker-sonic-mgmt.gz.sbom.cdx.json \ + | tee "${REPORT}.txt" + EXIT=${PIPESTATUS[0]} echo "" - echo "=== Trivy Scan Results (docker-sonic-mgmt) ===" - cat $(Build.ArtifactStagingDirectory)/trivy-sonic-mgmt.txt - exit $TRIVY_EXIT - displayName: "Trivy scan docker-sonic-mgmt" - - publish: $(Build.ArtifactStagingDirectory)/trivy-sonic-mgmt.txt - artifact: trivy-scan-results - displayName: "Publish Trivy scan results" + echo "=== SBOM vulnerability scan results (docker-sonic-mgmt) ===" + echo "Full CycloneDX VEX report: ${REPORT}.cdx.json" + echo "Human-readable table: ${REPORT}.txt" + exit $EXIT + displayName: "SBOM vulnerability scan (docker-sonic-mgmt)" + - publish: $(Build.ArtifactStagingDirectory) + artifact: sbom-vuln-scan-results + displayName: "Publish SBOM vuln-scan results" condition: always() continueOnError: true diff --git a/.gitignore b/.gitignore index fce90306a37..f099552bac2 100644 --- a/.gitignore +++ b/.gitignore @@ -12,6 +12,11 @@ fsroot/ fsroot-*/ fs.* target/ + +# Auto-extracted OpenVEX statements (regenerated from src/*/patches/ +# during the build by scripts/sbom_extract_vex_from_patches.py). +# Manually-authored VEX statements live in vex/manual/ and ARE tracked. +vex/auto/ *.deb *.udeb *.changes diff --git a/Makefile.work b/Makefile.work index 03fc7466d8a..41247cf74d5 100644 --- a/Makefile.work +++ b/Makefile.work @@ -248,6 +248,7 @@ PREPARE_DOCKER=BUILD_SLAVE=y \ MIRROR_SNAPSHOT=$(MIRROR_SNAPSHOT) \ BUILD_PUBLIC_URL=$(BUILD_PUBLIC_URL) \ BUILD_SNAPSHOT_URL=$(BUILD_SNAPSHOT_URL) \ + ENABLE_SBOM=$(ENABLE_SBOM) \ scripts/prepare_docker_buildinfo.sh \ $(SLAVE_BASE_IMAGE) \ $(SLAVE_DIR)/Dockerfile \ @@ -293,6 +294,17 @@ OVERLAY_MODULE_CHECK := \ BUILD_TIMESTAMP := $(shell date +%Y%m%d\.%H%M%S) +# SOURCE_DATE_EPOCH is the reproducible-build convention for stamping +# generated artifacts (SBOMs, in-toto attestations, etc.) with a +# deterministic time. Default to the timestamp of the HEAD git commit +# so two builds of the same source produce byte-identical SBOMs; can +# be overridden by passing SOURCE_DATE_EPOCH= explicitly from the +# environment, which is what reproducible-build CI typically does. +ifeq ($(SOURCE_DATE_EPOCH),) +SOURCE_DATE_EPOCH := $(shell git -C $(SOURCE_TREE) log -1 --format=%ct HEAD 2>/dev/null || date +%s) +endif +export SOURCE_DATE_EPOCH + # Create separate Docker lockfiles for saving vs. loading an image. ifeq ($(DOCKER_LOCKDIR),) override DOCKER_LOCKDIR := /tmp/docklock @@ -480,6 +492,7 @@ DOCKER_SLAVE_BASE_BUILD = docker build $(DOCKER_NO_CACHE_FLAG) \ --build-arg no_proxy=$(no_proxy) \ --build-arg SONIC_VERSION_CACHE=$(SONIC_VERSION_CACHE) \ --build-arg SONIC_VERSION_CONTROL_COMPONENTS=$(SONIC_VERSION_CONTROL_COMPONENTS) \ + --build-arg ENABLE_SBOM=$(ENABLE_SBOM) \ $(SLAVE_DIR) \ $(SPLIT_LOG) $(DOCKER_BASE_LOG) @@ -653,6 +666,12 @@ SONIC_BUILD_INSTRUCTION := $(MAKE) \ ENABLE_MULTIDB=$(ENABLE_MULTIDB) \ ENABLE_VRF_STRICT=$(ENABLE_VRF_STRICT) \ ENABLE_FRR_TCMALLOC=$(ENABLE_FRR_TCMALLOC) \ + ENABLE_SBOM=$(ENABLE_SBOM) \ + SBOM_FORMAT=$(SBOM_FORMAT) \ + SBOM_SCAN_TOOL=$(SBOM_SCAN_TOOL) \ + SBOM_INCLUDE_LICENSES=$(SBOM_INCLUDE_LICENSES) \ + SBOM_STRICT=$(SBOM_STRICT) \ + SOURCE_DATE_EPOCH=$(SOURCE_DATE_EPOCH) \ $(SONIC_OVERRIDE_BUILD_VARS) .PHONY: sonic-slave-build sonic-slave-bash init reset diff --git a/README.md b/README.md index c4af0fa8227..28ba7a2ea72 100644 --- a/README.md +++ b/README.md @@ -344,6 +344,18 @@ For details refer to [SONiC Buildimage Guide](https://github.com/sonic-net/sonic Please refer to [SONiC roadmap](https://github.com/sonic-net/SONiC/wiki/Sonic-Roadmap-Planning) on the SAI version for each SONiC release. +## Software Bill of Materials (SBOM) and vulnerability scanning + +Opt-in SBOM generation and SBOM-based vulnerability scanning are +supported via `ENABLE_SBOM=y` at build time. The default build path +is unchanged; enabling SBOM adds CycloneDX 1.6 + SPDX 2.3 + SLSA +v1.0 in-toto provenance sidecars per built artifact, along with +standalone CycloneDX scanner output for vulnerability reports. + +See [README.sbom.md](README.sbom.md) for the full design, build +flag reference, vulnerability-report quick start, VEX workflow, +reproducibility notes, and known limitations. + ## Notes * If you are running make for the first time, a sonic-slave-${USER} docker image diff --git a/README.sbom.md b/README.sbom.md new file mode 100644 index 00000000000..9373465dbf4 --- /dev/null +++ b/README.sbom.md @@ -0,0 +1,664 @@ +# SBOM Generation for SONiC Builds + +A CycloneDX 1.6 Software Bill of Materials, an SPDX 2.3 conversion, +and an unsigned SLSA v1.0 provenance attestation are emitted alongside +every `target/sonic-.bin` when the build is invoked with +`ENABLE_SBOM=y`. Default builds are unaffected. + +## Contents + +- [Quick start](#quick-start) +- [Configuration](#configuration) +- [Scope](#scope) +- [Architecture](#architecture) +- [Component categories](#component-categories) +- [License resolution](#license-resolution) +- [Tools](#tools) +- [Reproducibility](#reproducibility) +- [Attestation and signing](#attestation-and-signing) +- [Vulnerability scanning](#vulnerability-scanning) +- [Verification](#verification) +- [Querying dependencies](#querying-dependencies) +- [Known limitations](#known-limitations) +- [File map](#file-map) + +## Quick start + +Enable for one build: + +```bash +make ENABLE_SBOM=y target/sonic-broadcom.bin +``` + +Output, alongside the `.bin`: + +``` +target/sonic-.bin installer image +target/sonic-.bin.cdx.json CycloneDX 1.6 SBOM (primary) +target/sonic-.bin.spdx.json SPDX 2.3 (when SBOM_FORMAT in spdx, both) +target/sonic-.bin.intoto.json unsigned SLSA v1.0 provenance +``` + +Test containers that get built but don't ship in any `.bin` +(`docker-ptf`, `docker-ptf-sai`, `docker-sonic-mgmt`) each get a +standalone sidecar SBOM: + +``` +target/docker-ptf.gz.sbom.cdx.json +target/docker-ptf-sai.gz.sbom.cdx.json +target/docker-sonic-mgmt.gz.sbom.cdx.json +``` + +Generate a vulnerability report from the SBOM (a standalone post-build +step, not part of the build): + +```bash +python3 scripts/sbom_vuln_scan.py target/sonic-broadcom.bin.cdx.json +# Writes target/sonic-broadcom.bin.vuln.json + prints a table. +# Add --vex vex/ to apply suppressions. --fail-on critical for CI gating. +``` + +The SBOM is *not* embedded inside the `.bin`. Release engineering +publishes the sibling files alongside it. + +## Configuration + +| Variable | Default | Meaning | +|---|---|---| +| `ENABLE_SBOM` | `n` | Master switch. When `n`, no SBOM hooks fire and no extra tools are installed. | +| `SBOM_FORMAT` | `both` | `cyclonedx`, `spdx`, or `both`. SPDX is a downstream conversion via `cyclonedx-cli` (auto-fetched), so emitting both is essentially free. | +| `SBOM_SCAN_TOOL` | `syft` | Binary scanner for transitive deps: `syft` or `trivy`. | +| `SBOM_INCLUDE_LICENSES` | `y` | Whether to harvest copyrights and resolve SPDX licenses. | +| `SBOM_STRICT` | `y` | When `y`, the build fails if a critical SBOM input is missing — host rootfs (`fsroot-/`), any `.gz` in `SBOM_INSTALLER_DOCKERS`, or the scanner binary. Set to `n` for debugging or one-off partial emits. Soft optional features (SPDX conversion, provenance, license resolution) always warn-and-continue regardless. | + +These variables are surfaced in the `Build Configuration` dump that +SONiC prints at the start of every build, so you can confirm the state. + +## Scope + +Two SBOM kinds are emitted, with different scopes: + +**The aggregate `.bin` SBOM** (`target/sonic-.bin.cdx.json`) +covers what ships on the DUT inside the installer payload: the host +root filesystem, the kernel + every shipped module, the bootloader, +and every container the build packaged into the `.bin`. Build slaves, +the SDK build environment, and any container gated by an `INCLUDE_*` +flag set to `n` are automatically excluded — the aggregator reads the +docker list `slave.mk` itself computed for the active `PLATFORM`, no +hand-curated allow-list. + +**Per-container sidecar SBOMs** (`target/.gz.sbom.cdx.json`) +are emitted for test containers that get built but don't ship in any +`.bin` and so wouldn't otherwise be represented: + +- `docker-ptf`, `docker-ptf-sai` — PTF test harness +- `docker-sonic-mgmt` — automation harness + +These exist so security tooling has a CycloneDX surface for them +distinct from the production `.bin` SBOM. The other ~30 in-scope +containers don't get a sidecar — they're already covered by the +aggregate `.bin` SBOM. + +When you enable an optional container (`INCLUDE_PDE=y`, etc.) it +ships, so it appears in the `.bin` SBOM. The filter tracks the +build's own opt-ins. + +Always out-of-scope (never gets any SBOM): + +- `sonic-slave-*` build slave images. +- `docker-sonic-sdk-buildenv` (SDK build environment). +- Build-time-only debs and wheels (`_TEST = y` recipes, build deps + that `apt autoremove` strips before `fs.squashfs` is sealed). + +## Architecture + +Four independent sources contribute components to the final SBOM, +merged in priority order (first source wins for any given component): + +``` ++----------------+ +----------------+ +----------------+ +----------------+ +| Recipe-emit | >> | Observation | >> | Lockfile | >> | Scanner | +| | | | | parsing | | (syft/trivy) | +| Authoritative | | dpkg/pip in | | go.sum, npm / | | Catch-all over | +| for ~250 SONiC | | the assembled | | pnpm / yarn | | the assembled | +| .debs + per- | | rootfs and | | locks from in- | | rootfs and | +| .deb language | | containers. | | tree builds. | | each container.| +| deps (Rust / | | | | (Rust is via | | | +| Go / Python). | | | | recipe-emit, | | | +| | | | | not lockfile.) | | | ++----------------+ +----------------+ +----------------+ +----------------+ + | | | | + +-----------------------+-----------------------+-----------------------+ + v + +-----------------------+ + | Merge (purl + nva + | + | normalized-version) | + +-----------------------+ + v + target/sonic-.bin.cdx.json +``` + +**Why hybrid.** Recipe-driven emit owns the locally-built artifacts +(versions, submodule SHAs, patch series, per-.deb Rust/Go/Python +language-dep attribution) that a binary scanner has no way to recover. +Observation owns transitive apt/pip deps. Lockfile parsing adds the +language-ecosystem transitive deps in ecosystems the per-.deb +introspection above doesn't already cover. Scanner is the wide net +for anything else. None of the four alone is complete; the union is. + +**Dedupe by `(purl, name+version+arch, name+normalized-version+arch)`.** +The normalized-version key strips Debian epochs (`1:`) and downstream +suffixes (`+fips`, `+sonic`, `+bN`, `+debNuM`) so recipe-emit's +a recipe-emit ` ` matches observation's `:+fips`. +Legitimately distinct upstream versions of the same package across +different Debian releases stay distinct. + +**Dynamic, not enumerated.** The recipe layer only fires when make +actually invokes that recipe; observation only sees what got +installed. Builds for different `PLATFORM` values produce different +SBOMs automatically because the build invokes different recipes. +`SONIC_VERSION_CONTROL_COMPONENTS` overrides (where pins float to +mirror-latest) are reflected too — observation records what was +actually installed. + +## Component categories + +How each pattern is captured. Concrete CycloneDX shapes are visible in +`target/.../*.cdx.json` after a build; one representative example is +shown after this list. + +- **SONiC-native code** (sonic-net submodule). Primary PURL: + `pkg:github/sonic-net/@`. `externalReferences` records + the submodule URL and pinned commit. No pedigree unless a sibling + `src/.patch/` exists. + +- **Patched upstream Debian sources** (`dget` + sidecar patches in + `src//patch/`). Primary: `pkg:deb/sonic/@`. + `pedigree.ancestors[0]` records the upstream Debian `.dsc` URL + + MD5 from `versions-web`; `pedigree.patches[]` enumerates each + applied patch with its SHA-256. + +- **Forked upstream with nested submodule** (the FRR pattern: + `src/sonic-frr` is a sonic-net wrapper holding patches; the nested + `src/sonic-frr/frr` is FRRouting upstream). Primary: + `pkg:deb/sonic/frr@`. Ancestor: + `pkg:github/FRRouting/frr@`. Patches + aggregate + patch-set SHA-1 in pedigree. Same handling for sonic-p4rt, + sonic-sysmgr (gnoi), wpasupplicant. + +- **Direct upstream submodule with sidecar patches** (scapy, ptf, + ptf-py3, supervisor, redis-dump-load). Primary derived from the + artifact filename (PyPI for wheels, deb for debs). Ancestor: + `pkg:github//@`. Patches from the + sidecar directory. + +- **Upstream apt/dpkg packages** (everything apt-installed into the + rootfs or a container, ~1000+ debs). Captured by observation: + `dpkg-query` against each scope. PURL form + `pkg:deb/debian/@?arch=`. Mirror snapshot + timestamp attached via `externalReferences.comment`. + +- **Python packages.** Locally-built wheels and stdeb debs follow the + SONiC-native or direct-upstream rules. PyPI installs in + containers/rootfs are observation + (`pkg:pypi/@`) from `pip3 freeze`. `uv pip` installs + are caught by the scanner pass since uv writes standard + `dist-info/` directories. + +- **Docker container images.** One `type: container` component per + shipped container with image digest. Per-container observation + components nested under it via `dependencies[]`. Base image SHAs + recorded from `files/build/versions/default/versions-docker`. + +- **Kernel and out-of-tree modules.** Linux kernel built from Debian + source + ~200 SONiC patches enumerated in pedigree. Each kernel + module (`opennsl-modules`, `sx-kernel`, `ionic-modules`, every + `sonic-platform-modules-*`) has a `dependencies[]` edge pointing + at the kernel image's `bom-ref` so consumers can reason about + kernel-ABI risk. + +- **Vendor SDKs and HALs** (Broadcom SAI, Mellanox SDK, NVIDIA + BlueField, Marvell Teralynx, Pensando, Arista PHY Credo). Recipe + emits `type: library` with `supplier.name` derived from URL + pattern, the EULA name as license, and a build-time-computed + SHA-256. + +- **Language-ecosystem transitive deps.** Per-scope `go.sum`, + `package-lock.json`, `pnpm-lock.yaml`, `yarn.lock` are harvested by + the build hook into `lockfiles.tar.gz`. Each entry becomes a + `pkg:golang/` or `pkg:npm/` component with the SHA-256 from the + lockfile preserved. Rust crate inventory is *not* harvested this + way — see the next bullet. + +- **Per-.deb language dependency attribution.** Every SONiC-built + `.deb` is introspected at fragment-emit time + (`scripts/sbom_fragment.py`). A single `dpkg-deb -x` extraction + feeds three harvesters: + + - **Rust** — for each ELF, `rust-audit-info` reads the `.dep-v0` + section that `cargo-auditable` embedded. Emits each crate as + `purl=pkg:cargo/@` with + `sonic:fragment_kind=recipe-emit-rust`. + - **Go** — for each ELF, `go version -m` parses the + `runtime/debug.BuildInfo` table from the `.go.buildinfo` section. + Emits each module as `purl=pkg:golang/@` with + `sonic:fragment_kind=recipe-emit-go`. Replacement directives + (`=>`) are honored so the recorded version is what actually + shipped. + - **Python** — walk for `*.dist-info/METADATA` files and parse + Name + Version. Emits each as `purl=pkg:pypi/@` + with `sonic:fragment_kind=recipe-emit-python`. SONiC `.debs` + today don't ship dist-info content (Python is staged via dpkg + into `/usr/lib/python3/dist-packages/` without metadata), so + Python attribution here is currently a no-op — but the harvester + activates automatically as soon as a `.deb` recipe starts + bundling a venv or installing wheels into `debian//`. + + Every emitted component carries `sonic:source_deb=`. + `build_sbom.py:build_dependency_graph()` reverses + `sonic:source_deb` into `dependsOn` edges from the `.deb`'s + `bom-ref` to each language-dep component so a consumer can walk + `swss_*.deb -> tokio@1.x` (Rust) or `sonic-gnmi_*.deb -> + github.com/openconfig/gnmi@v0.10` (Go) directly without parsing + properties. + +### Example component (forked-upstream pattern after pedigree resolution) + +```json +{ + "bom-ref": "pkg:deb/sonic/frr@?arch=", + "type": "library", + "name": "frr", + "version": "", + "purl": "pkg:deb/sonic/frr@?arch=", + "supplier": {"name": "SONiC build (forked upstream)"}, + "pedigree": { + "ancestors": [{ + "name": "frr", + "version": "", + "purl": "pkg:github/FRRouting/frr@", + "externalReferences": [{"type": "vcs", "url": "https://github.com/FRRouting/frr"}] + }], + "patches": [ + {"type": "unofficial", + "diff": {"url": "file://src/sonic-frr/patch/.patch", + "hashes": [{"alg": "SHA-256", "content": ""}]}} + ], + "notes": "patch-set sha1: " + } +} +``` + +## License resolution + +Three sources, applied in priority order: + +1. **Per-recipe override.** Recipes can declare + `$()_LICENSE = ""` in `rules/*.mk`. The + build helper passes it through to the fragment generator as + `ARTIFACT_LICENSE`. Authoritative for SONiC-native packages where + no `debian/copyright` ships. + +2. **DEP-5 `debian/copyright`.** The build hook tars every + `/usr/share/doc/*/copyright` from each scope into a sidecar + tarball (before `apt autoremove` and the doc-strip at the end of + the rootfs build wipe it). The resolver parses any file declaring + `Format: ...copyright-format/1.0/` and maps `License:` stanzas + to SPDX via `scripts/sbom_license_map.json` (~100 mappings). + +3. **`licensecheck` fallback** (from `devscripts`) for non-DEP-5 + copyright files; output goes through the same SPDX table. + +Components ending up `NOASSERTION` are counted at end-of-build: + +``` +[build_sbom.py] License resolution: 1182/1266 resolved (93.4%); 84 NOASSERTION +``` + +### Maintaining `scripts/sbom_license_map.json` + +The license-header → SPDX-identifier map is **manually curated**. Each +entry maps a free-form license declaration string (as it appears in +`License:` stanzas of Debian DEP-5 `debian/copyright` files, or as +`licensecheck` reports them) to a valid SPDX-2.3 identifier. + +To add a new mapping when a previously-unmapped `License:` keyword +shows up in a build's `NOASSERTION` set: + +1. Run a build with `ENABLE_SBOM=y`. The aggregator logs the count + of unresolved components: `License resolution: X/Y resolved …; N + NOASSERTION`. +2. Drill into the generated `target/.bin.cdx.json` and + grep for components with no `licenses[]` entry; check their + `pedigree.notes` or external references to find the source + `License:` keyword the resolver couldn't match. +3. Look up the corresponding SPDX identifier at + . +4. Add an entry to `scripts/sbom_license_map.json`: + ```json + "BSD-3-Clause variant from foo project": "BSD-3-Clause" + ``` +5. Re-run the build (or just the aggregator) and verify the count + of resolved components increases. + +The map is intentionally not auto-generated: false-positive risk is +too high for license declarations that differ in legally-meaningful +ways. A new mapping should be reviewed by someone who has read the +underlying license text at least once. + +## Tools + +When `ENABLE_SBOM=y`, the build invokes `scripts/install_sbom_tool.sh` +to fetch `syft`, `cyclonedx-cli` (if SPDX requested), and `grype` (for +post-build vuln scanning). The script wgets a pinned upstream release, +verifies its SHA-256 against a value hardcoded in the script, and +caches the binary under `target/sbom-tools/`. Subsequent builds use the +cache. + +Because the `wget` runs through SONiC's existing build-hook shim, the +URL + MD5 lands in `versions-web` like every other web-fetched +artifact — no manual pin editing required. + +For standalone vuln scanning outside a build, the same script +auto-fetches into `~/.cache/sonic-sbom/`. + +The aggregator additionally caches per-file scanner output under +`target/sbom-tools/syft-cache/-.json`, keyed by SHA-256 +of the input archive. A SONiC platform with multiple ASIC variants +(e.g. broadcom, broadcom-dnx, broadcom-legacy-th) invokes +`build_sbom.py` once per variant; without the cache, syft would +re-scan the ~28 docker `.gz` files that are identical across variants +three times. The cache short-circuits 2nd and 3rd hits. `dir:` scans +(host rootfs) are not cached because `fsroot-/` differs per +variant. `make reset` wipes `target/` and therefore invalidates the +cache automatically — no stale entries across resets. + +## Reproducibility + +Two byte-identical builds of the same source produce byte-identical +SBOMs. `Makefile.work` sets `SOURCE_DATE_EPOCH` automatically from +the HEAD git commit's author timestamp (falling back to wall-clock +`date +%s` only when the source tree isn't a git checkout), and +exports it through to the slave container, so all SBOM timestamps +are derived from source state rather than build wall-clock. Component +ordering is sorted. License resolution depends only on on-disk files ++ the static translation table. Override is supported by passing +`SOURCE_DATE_EPOCH=` explicitly to `make`. + +Cross-host reproducibility check: + +```bash +python3 scripts/sbom_diff.py old-host.cdx.json new-host.cdx.json +# Exit 0 if equivalent at the component level; non-zero on drift. +# --quiet for one-line summary; --json out.json for tooling. +``` + +The diff ignores timestamps and aggregator-internal metadata. It +compares version, hashes, licenses, pedigree.ancestors, and patch +hashes. + +## Attestation and signing + +The build emits an **unsigned** SLSA v1.0 / in-toto v1 provenance at +`target/sonic-.bin.intoto.json` capturing the `.bin` +SHA-256, the `sonic-buildimage` git commit, the active build slave +image digests, the build's external parameters (`PLATFORM`, +`CONFIGURED_ARCH`, `INCLUDE_*`, etc.), and a content-hash reference to +the SBOM. The document is reproducible. + +**No signing happens inside the build.** Two reasons: (1) SONiC's +`SECURE_UPGRADE_*` keys answer a different question ("permitted to +boot on this hardware?"); reusing them for attestation expands their +blast radius; (2) production signing often requires per-invocation +human approval — release engineering needs to batch all artifacts +under one key access in their own toolchain. + +Recommended post-build workflow (operator chooses the tool): + +```bash +# Sigstore cosign +cosign sign-blob --yes --key cosign.key \ + --output-signature target/sonic-broadcom.bin.cdx.json.sig \ + target/sonic-broadcom.bin.cdx.json + +# Or OpenSSL PKCS#7 +openssl cms -sign -binary -in target/sonic-broadcom.bin.cdx.json \ + -signer attestation.crt -inkey attestation.key \ + -outform DER -out target/sonic-broadcom.bin.cdx.json.p7s +``` + +## Vulnerability scanning + +Decoupled from the build by design: the SBOM is reproducible, a CVE +report cannot be (new CVEs are disclosed daily). A six-month-old SBOM +can be re-scanned against today's CVE data without rebuilding. + +Standalone Python scripts, not make targets: + +```bash +# Default report (table + sibling .vuln.json) +python3 scripts/sbom_vuln_scan.py target/sonic-broadcom.bin.cdx.json + +# With VEX suppression and CI gating +python3 scripts/sbom_vuln_scan.py --vex vex/ --fail-on critical \ + target/sonic-broadcom.bin.cdx.json + +# Drift between two reports +python3 scripts/sbom_vuln_diff.py release-202503.vuln.json release-202504.vuln.json +``` + +The scanner (`grype` by default) reads the SBOM directly. Without +VEX, every locally-patched component flags upstream CVEs (since +`pedigree.ancestors` records the unpatched version). VEX suppresses +those that the SONiC patches actually fix. + +The table written to stdout has one row per finding, sorted by +severity descending: + +``` +SEVERITY ID ECOSYS PKG VERSION STATE FIX +------------------------------------------------------------------------------------------------------------ +Critical CVE-2024-3094 deb xz-utils 5.4.1-1 fixed 5.6.1+really5.4.5-1 +Critical CVE-2024-45337 go golang.org/x/crypto v0.24.0 fixed 0.31.0 +Critical CVE-2021-3711 rust openssl-src 111.10.2+1.1.1g fixed 111.16.0 +Critical CVE-2021-44906 npm minimist 1.2.5 fixed 1.2.6 +Critical CVE-2026-7210 py python3.13 3.13.5-2+deb13u2 fixed ... +High CVE-2026-34040 go github.com/docker/docker v28.5.2+incompatible not-fixed - +``` + +`ECOSYS` is derived from the PURL type (`deb`, `rust`, `go`, `npm`, +`py`, `gem`, `java`, `oci`, `github`, `generic`). `FIX` is the +suggested upgrade version when grype's advisory data names one, or +`-` for `not-fixed` advisories. GHSA aliases for CVE-primary findings +are preserved in the sidecar `.cdx.json` as +`vulnerabilities[].references[]` per the CycloneDX 1.6 schema. + +### VEX + +VEX files live in `vex/` as OpenVEX JSON (grype's `--vex` flag +rejects YAML). `vex/README.md` documents the schema, status taxonomy, +and triage workflow. Curated statements live at the top level and are +checked in; the `vex/auto/` subdirectory is **gitignored** and +regenerated on every SBOM build by `scripts/sbom_extract_vex_from_patches.py`, +which scans `src/**/*.patch` for CVE markers and emits `not_affected` +statements per patch-mentioned CVE. The regeneration is wired into +`scripts/build_sbom.sh` so the auto-VEX set always reflects the +current state of `src/*/patches/` — a patch that introduces or +removes a CVE marker is reflected on the very next build. + +```json +{ + "@context": "https://openvex.dev/ns/v0.2.0", + "@id": "https://github.com/sonic-net/sonic-buildimage/vex/", + "author": "", + "timestamp": "", + "version": 1, + "statements": [{ + "vulnerability": {"name": "CVE-YYYY-NNNNN"}, + "products": [{"@id": "pkg:deb/sonic/@"}], + "status": "not_affected", + "justification": "vulnerable_code_not_in_execute_path", + "impact_statement": "Fixed by SONiC patch ", + "references": [""] + }] +} +``` + +Auto-extracted entries use a generic `pkg:generic/` PURL +because the extractor doesn't know which downstream debs the patch +ships in; promoting them to curated files with concrete PURLs +improves suppression rate. + +## Verification + +```bash +# Schema-validate the SBOM +cyclonedx-cli validate --input-file target/sonic-.bin.cdx.json + +# Component count +jq '.components | length' target/sonic-.bin.cdx.json + +# Locally-patched components +jq '.components[] | select(.pedigree.patches) | .name' \ + target/sonic-.bin.cdx.json + +# Components without resolved licenses +jq '.components[] | select(.licenses == null or .licenses == []) | .name' \ + target/sonic-.bin.cdx.json +``` + +## Querying dependencies + +The CycloneDX `dependencies[]` graph supports direct walks without +parsing string properties. The bom-ref of a SONiC-built `.deb` +matches its source identity — `pkg:github/sonic-net/@` +for components built from a sonic-net submodule, +`pkg:deb/sonic/@?arch=` for patched-upstream +Debian sources. + +```bash +SBOM=target/sonic-.bin.cdx.json + +# What does sonic-swss depend on? +# Returns 10 sibling SONiC fragments (libteam-*, libnexthopgroup-*, +# sonic-sairedis, sonic-dash-api, sonic-stp, sonic-swss-common) +# plus every Rust crate linked into swss's binaries (~190 today). +jq --arg ref "pkg:github/sonic-net/sonic-swss@" \ + '.dependencies[] | select(.ref | startswith($ref)) | .dependsOn' \ + "$SBOM" + +# Which .deb ships tokio@1.x ? +# recipe-emit-rust components carry sonic:source_deb=; +# the same attribution is mirrored as a dependsOn edge from the +# .deb to the crate, so either property or graph walk works. +jq '.components[] | select(.purl | startswith("pkg:cargo/tokio@")) | + {name, version, + source_deb: ([.properties[]? | select(.name=="sonic:source_deb") | .value] + | first)}' "$SBOM" + +# Reverse: list every Rust crate that swss ships +jq --arg ref "pkg:github/sonic-net/sonic-swss@" \ + '.dependencies[] | select(.ref | startswith($ref)) | + .dependsOn | map(select(startswith("pkg:cargo/")))' "$SBOM" + +# Same idea for Go modules in sonic-gnmi +jq --arg ref "pkg:github/sonic-net/sonic-gnmi@" \ + '.dependencies[] | select(.ref | startswith($ref)) | + .dependsOn | map(select(startswith("pkg:golang/")))' "$SBOM" + +# What declared build/runtime .deb deps did a recipe make? +# These remain as space-separated string properties for analytics +# that need the build-vs-runtime split (CycloneDX dependencies[] is +# one unscoped graph and doesn't distinguish the two). +jq --arg ref "pkg:github/sonic-net/sonic-swss@" \ + '.components[] | select(.["bom-ref"] | startswith($ref)) | + {build_depends: [.properties[]? | select(.name=="sonic:build_depends") | .value], + runtime_depends: [.properties[]? | select(.name=="sonic:runtime_depends") | .value], + unresolved_deps: [.properties[]? | select(.name=="sonic:unresolved_deps") | .value]}' \ + "$SBOM" + +# Walk the kernel-module → kernel-image edges (out-of-tree modules +# whose runtime ABI is pinned to the kernel) +jq '.dependencies[] + | select(.dependsOn[] | startswith("pkg:deb/sonic/linux-image-")) + | .ref' "$SBOM" + +# Cross-reference: which SONiC .debs declared an unresolvable dep? +# Useful to audit gaps between $(pkg)_DEPENDS declarations and the +# set of recipe-emit fragments we actually produced. +jq '.components[] + | select(.properties[]? | select(.name=="sonic:unresolved_deps")) + | {name, version, + unresolved: ([.properties[] | select(.name=="sonic:unresolved_deps") | .value] | first)}' \ + "$SBOM" +``` + +Tools that understand CycloneDX `dependencies[]` natively work +without any of these `jq` queries — e.g. `cyclonedx-cli analyze`, +or piping the SBOM into a graph visualizer. The examples above +exist for ad-hoc analysis from a shell. + +## Known limitations + +- **Per-.deb language dependency attribution is scoped to SONiC- + built `.debs`.** `scripts/sbom_fragment.py` introspects each + SONiC-built `.deb` and harvests Rust crates (from cargo-auditable + `.dep-v0` ELF sections via `rust-audit-info`), Go modules (from + `runtime/debug.BuildInfo` via `go version -m`), and Python + distributions (from `*.dist-info/METADATA` walk). Components are + scoped at the `.deb` level via `sonic:source_deb`, *not* per-binary + within the `.deb`. Upstream Debian-archive `.debs` are intentionally + out of scope — their language-dep metadata is captured by the syft + scanner pass plus the observation manifests, not by `.deb` + introspection. + +- **`uv pip` skips the `pip3` build-hook shim**, so its installs are + absent from the observation manifest. They still reach the SBOM via + the scanner pass — uv writes standard `dist-info/` directories — but + their `sonic:fragment_kind` property reads `scanner` rather than + `observation`. + +- **Vendored static libraries** statically linked into binaries (a + `.deb` that bundles libcurl instead of dynamic-linking) are + undetectable without source-code scanning. Not closed by this + design. + +- **Pip artifacts lack hashes** from pin time because SONiC's + Python version-control doesn't use `--require-hashes`. The scanner + may compute installation-time hashes from `RECORD` as a best-effort + substitute. + +- **Broadcom XGS SAI** historically used `_SKIP_VERSION=y`. The SBOM + computes SHA-256 of the downloaded artifact independently and + ignores the recipe flag. + +- **Vendor EULA license names** are recorded as declared (e.g., + "Broadcom Inc. proprietary (EULA)") but not normalised to SPDX + expressions, since most are not SPDX-listed. + +## File map + +Files that exist for this design. + +| Path | Role | +|---|---| +| `rules/config` | Defines the four SBOM configuration variables. | +| `Makefile.work` | Threads SBOM variables into the slave-container environment. | +| `slave.mk` | Defines the `sbom_emit_fragment` helper, calls it from each `SONIC_*` artifact recipe, invokes `build_sbom.sh` between rootfs assembly and `.bin` wrap, and exposes the recipe context (installer docker/deb/wheel lists) to the aggregator. | +| `build_image.sh` | Emits the `.cdx.json` sibling after the `.bin` is wrapped. | +| `scripts/install_sbom_tool.sh` | Auto-fetches syft, grype, cyclonedx-cli with SHA-256 verify into `target/sbom-tools/`. | +| `scripts/build_sbom.py` | Aggregator. Walks fragments, runs the scanner, parses lockfiles, resolves licenses, dedupes, builds the CycloneDX `dependencies[]` graph (kernel-module → kernel-image edges, declared `.deb` build/runtime deps, recipe-emit-{rust,go,python} → owning `.deb` edges), and writes the SBOM + SPDX + provenance. | +| `scripts/build_sbom.sh` | Thin shim that execs `build_sbom.py`. | +| `scripts/sbom_fragment.py` | Per-recipe fragment generator. Knows the four ancestor patterns, the vendor-supplier URL table, and the per-`.deb` language-dep harvesters (Rust via `rust-audit-info`, Go via `go version -m`, Python via `*.dist-info/METADATA` walk). | +| `scripts/sbom_resolve_licenses.py` | DEP-5 parser + licensecheck fallback + SPDX translation. | +| `scripts/sbom_license_map.json` | ~100 Debian License header strings mapped to SPDX. | +| `scripts/sbom_parse_lockfiles.py` | Parses `go.sum`, `package-lock.json`, `pnpm-lock.yaml`, `yarn.lock` from harvested archives. Also parses `Cargo.lock` defensively for legacy archives — but Rust attribution comes from `sbom_fragment.py`'s `.dep-v0` introspection, not from this parser. | +| `scripts/sbom_emit_provenance.py` | Emits the unsigned SLSA v1.0 / in-toto provenance document. | +| `scripts/sbom_diff.py` | Reproducibility comparison between two SBOMs. | +| `scripts/sbom_vuln_scan.py` | Standalone CVE scanner (invokes grype, applies VEX). | +| `scripts/sbom_vuln_diff.py` | Standalone drift analysis between two vuln reports. | +| `scripts/sbom_extract_vex_from_patches.py` | Auto-VEX from CVE markers in patch metadata. | +| `vex/` | OpenVEX statements (curated at top level, auto in `vex/auto/`). | +| `vex/README.md` | VEX schema and triage workflow. | +| `src/sonic-build-hooks/scripts/collect_version_files` | Extended to harvest `copyrights.tar.gz` and `lockfiles.tar.gz` per scope when `ENABLE_SBOM=y`. | +| `scripts/prepare_docker_buildinfo.sh` | Injects `ENV ENABLE_SBOM` into each container's Dockerfile so the harvest hook sees the right value at install time. | diff --git a/azure-pipelines.yml b/azure-pipelines.yml index cc6454415e9..29010d02793 100644 --- a/azure-pipelines.yml +++ b/azure-pipelines.yml @@ -223,9 +223,36 @@ stages: CHECKOUT_SONIC_MGMT: ${{ parameters.CHECKOUT_SONIC_MGMT }} PTF_MODIFIED: $(PTF_MODIFIED) - # Trivy scan for docker-ptf — informational, runs in parallel with KVM tests - - job: trivy_scan - displayName: "[OPTIONAL] Trivy vulnerability scan (docker-ptf)" +# Unified SBOM-based vulnerability scan — informational, runs after +# both build stages so every CycloneDX SBOM produced is in scope. +# Lives in its own stage (not inside Test) because the Test stage +# dependsOn BuildVS only — putting the scan there meant the .bin +# SBOMs from the Build stage (broadcom, mellanox, etc.) hadn't been +# uploaded as artifacts yet when the scan ran. VulnScan dependsOn +# BuildVS + Build so the scan only fires once every *.cdx.json +# sidecar this pipeline produces is on the artifact server. +# +# Runs after Test (rather than in parallel with it) is fine — this +# is non-gating telemetry, and putting it in its own stage avoids +# duplicating the Test stage's KVM-test machinery just to add deps. +# +# docker-sonic-mgmt is scanned by its own pipeline +# (.azure-pipelines/docker-sonic-mgmt.yml) and is intentionally out +# of scope here: the main pipeline never builds it, so there is no +# docker-sonic-mgmt.gz.sbom.cdx.json artifact for this scan to +# consume. The scan step logs a one-line pointer to the other +# pipeline so readers don't wonder why ptf is listed and mgmt isn't. +- stage: VulnScan + displayName: "Vuln Scan" + dependsOn: + - BuildVS + - Build + # Run even if a platform build had issues — surface findings for + # whatever SBOMs did make it to the artifact server. + condition: not(canceled()) + jobs: + - job: sbom_vuln_scan + displayName: "[OPTIONAL] SBOM-based vulnerability scan (all artifacts)" pool: sonic-ubuntu-1c continueOnError: true timeoutInMinutes: 60 @@ -235,36 +262,82 @@ stages: fetchDepth: 1 - task: DownloadPipelineArtifact@2 inputs: - artifact: sonic-buildimage.vs - itemPattern: "target/docker-ptf.gz" - displayName: "Download docker-ptf artifact" + # Narrow to aggregate SBOMs only: the top-level installer + # (sonic-.bin / .swi / .img.gz) and the standalone + # test-container SBOMs (docker-*.gz.sbom.cdx.json — note the + # .sbom. infix that distinguishes them from per-container + # recipe fragments). The recipe-emit fragments under + # target/debs/, target/python-wheels/, and target/docker-*.gz.cdx.json + # are build-internal plumbing; scanning them duplicates work + # already done by the .bin aggregate scan. + patterns: | + **/target/sonic-*.bin.cdx.json + **/target/sonic-*.swi.cdx.json + **/target/sonic-*.img.gz.cdx.json + **/target/docker-*.gz.sbom.cdx.json + displayName: "Download aggregate SBOM sidecars" - script: | set -ex - TRIVY_VERSION="0.70.0" - curl -fLO https://github.com/aquasecurity/trivy/releases/download/v${TRIVY_VERSION}/trivy_${TRIVY_VERSION}_Linux-64bit.deb - sudo apt install -y ./trivy_${TRIVY_VERSION}_Linux-64bit.deb - trivy --version - displayName: "Install Trivy" + sudo apt-get install -y python3-pip jq + displayName: "Install dependencies" - script: | - set -x - # docker-ptf.gz is ~1.3GB; scan takes 15-20 min on CI agents - trivy image \ - --input $(Pipeline.Workspace)/target/docker-ptf.gz \ - --severity MEDIUM,HIGH,CRITICAL \ - --ignore-unfixed \ - --exit-code 1 \ - --format table \ - --no-progress \ - --timeout 30m \ - --output $(Build.ArtifactStagingDirectory)/trivy-ptf.txt - TRIVY_EXIT=$? + set +e + OVERALL_RC=0 + FOUND=0 + mkdir -p $(Build.ArtifactStagingDirectory)/sbom-vuln + shopt -s nullglob globstar + # Note for log readers: docker-sonic-mgmt is built and scanned + # in its own pipeline (.azure-pipelines/docker-sonic-mgmt.yml). + # Its vuln report lives there as the sbom-vuln-scan-results + # artifact of that pipeline's runs. That's why ptf appears + # below but mgmt does not. + echo "Note: docker-sonic-mgmt is scanned in .azure-pipelines/docker-sonic-mgmt.yml; not built in this pipeline." + echo "" + # Walk only the aggregate SBOMs at the top of each artifact's + # target/ directory. Recipe fragments under target/debs/ or + # target/python-wheels/ are not matched by these patterns. + for sbom in \ + $(Pipeline.Workspace)/*/target/sonic-*.bin.cdx.json \ + $(Pipeline.Workspace)/*/target/sonic-*.swi.cdx.json \ + $(Pipeline.Workspace)/*/target/sonic-*.img.gz.cdx.json \ + $(Pipeline.Workspace)/*/target/docker-*.gz.sbom.cdx.json + do + FOUND=$((FOUND+1)) + # Derive a short name for this scan's outputs. Test-container + # SBOMs include a `.sbom` infix on disk (e.g. docker-ptf.gz.sbom.cdx.json) + # to distinguish them from per-container recipe fragments + # (docker-ptf.gz.cdx.json); strip it so the per-artifact + # output names match the underlying artifact extension: + # sonic-broadcom.bin.cdx.json -> sonic-broadcom.bin{.txt,.cdx.json} + # docker-ptf.gz.sbom.cdx.json -> docker-ptf.gz{.txt,.cdx.json} + name=$(basename "$sbom" .cdx.json) + name="${name%.sbom}" + # Azure Pipelines log-folding directive — wrap each SBOM's + # output so the log view shows a collapsible section per + # scan rather than one giant concatenated stream. + echo "##[group]SBOM vuln scan: $name" + python3 scripts/sbom_vuln_scan.py \ + --vex vex \ + --min-severity medium \ + --fail-on medium \ + --format both \ + --output "$(Build.ArtifactStagingDirectory)/sbom-vuln/${name}.cdx.json" \ + "$sbom" \ + | tee "$(Build.ArtifactStagingDirectory)/sbom-vuln/${name}.txt" + RC=${PIPESTATUS[0]} + if [ "$RC" != "0" ]; then OVERALL_RC=$RC; fi + echo "##[endgroup]" + done + if [ "$FOUND" = "0" ]; then + echo "::error::No SBOM files found under $(Pipeline.Workspace) — were the build stages invoked with ENABLE_SBOM=y?" + exit 1 + fi echo "" - echo "=== Trivy Scan Results (docker-ptf) ===" - cat $(Build.ArtifactStagingDirectory)/trivy-ptf.txt - exit $TRIVY_EXIT - displayName: "Trivy scan docker-ptf" - - publish: $(Build.ArtifactStagingDirectory)/trivy-ptf.txt - artifact: trivy-scan-results - displayName: "Publish Trivy scan results" + echo "=== SBOM vulnerability scan: $FOUND SBOM(s) scanned ===" + exit $OVERALL_RC + displayName: "SBOM vulnerability scan (all artifacts)" + - publish: $(Build.ArtifactStagingDirectory)/sbom-vuln + artifact: sbom-vuln-scan-results + displayName: "Publish SBOM vuln-scan results" condition: always() continueOnError: true diff --git a/build_debian.sh b/build_debian.sh index 680358f8d6c..9dd1bbff0a9 100755 --- a/build_debian.sh +++ b/build_debian.sh @@ -833,6 +833,12 @@ SONIC_VERSION_CACHE=${SONIC_VERSION_CACHE} \ DBGOPT="${DBGOPT}" \ scripts/collect_host_image_version_files.sh $CONFIGURED_ARCH $IMAGE_DISTRO $TARGET_PATH $FILESYSTEM_ROOT +# SBOM license harvest happens uniformly via the per-scope +# collect_version_files hook (see src/sonic-build-hooks/scripts/collect_version_files). +# That runs inside the chroot via post_run_buildinfo (line 21 of +# collect_host_image_version_files.sh) before /usr/share/doc/* is wiped +# below at line ~838. + # Remove GCC sudo LANG=C DEBIAN_FRONTEND=noninteractive chroot $FILESYSTEM_ROOT apt-get -y remove gcc diff --git a/files/build/cargo-wrapper b/files/build/cargo-wrapper new file mode 100755 index 00000000000..d8292d7c4e0 --- /dev/null +++ b/files/build/cargo-wrapper @@ -0,0 +1,66 @@ +#!/bin/bash +# /usr/local/bin/cargo — transparent cargo-auditable wrapper. +# +# Routes `cargo build` (and only build) through `cargo auditable` so +# every Rust binary produced in the slave embeds the resolved +# Cargo.lock into a `.dep-v0` ELF section. Syft's rust-binary +# cataloger reads that section at SBOM scan time, giving SONiC SBOMs +# full per-crate visibility for Rust packages with no source-tree +# changes to individual debian/rules files. +# +# This file is shared across every sonic-slave-* Dockerfile via a +# single COPY from files/build/cargo-wrapper. Edit here only — do +# not duplicate. +# +# Behaviour: +# * If cargo-auditable is on PATH and we are wrapping a `cargo +# build`, route through it. +# * If cargo-auditable is NOT on PATH and we are wrapping a `cargo +# build`, fail HARD with an actionable error rather than +# silently degrading to plain `cargo build` — the SBOM would +# otherwise be quietly missing Rust crate inventory and we'd +# have no signal that anything went wrong. +# * For any other cargo subcommand, pass through unchanged. +# +# CARGO_AUDITABLE_WRAPPED guards against recursion: cargo-auditable +# itself invokes `cargo build` internally and would loop without it. + +REAL=/usr/.cargo/bin/cargo + +if [ -n "$CARGO_AUDITABLE_WRAPPED" ]; then + exec "$REAL" "$@" +fi + +case "$1" in + build) + if ! command -v cargo-auditable >/dev/null 2>&1; then + cat >&2 < + +with CARGO_HOME=\$RUST_ROOT). If you are using a custom slave image +that omitted this dependency, install it manually with the same +command before retrying the build. + +EOF + exit 1 + fi + export CARGO_AUDITABLE_WRAPPED=1 + exec "$REAL" auditable "$@" + ;; + *) + exec "$REAL" "$@" + ;; +esac diff --git a/rules/config b/rules/config index ab1eb74ecd6..60955f4da0d 100644 --- a/rules/config +++ b/rules/config @@ -409,3 +409,35 @@ BUILD_SKIP_TEST ?= n # target/ccache/ and persists across builds. # Default: disabled (no impact on clean builds without a warm cache) SONIC_CONFIG_USE_CCACHE ?= n + +# ENABLE_SBOM - generate a CycloneDX Software Bill of Materials alongside +# the installer image. When y, the build invokes scripts/install_sbom_tool.sh +# to fetch the configured scanner (default: syft) and emits +# target/sonic-.bin.cdx.json as a sibling of the .bin. +# See README.sbom.md for the full architecture. +ENABLE_SBOM ?= n + +# SBOM_FORMAT - output format for the generated SBOM. +# Valid values: cyclonedx, spdx, both. +# Default: both — CycloneDX 1.6 is the canonical format and SPDX 2.3 +# is a downstream conversion via cyclonedx-cli (auto-fetched), so +# emitting both is essentially free and lets consumers pick. +SBOM_FORMAT ?= both + +# SBOM_SCAN_TOOL - binary scanner used to enumerate transitive apt/pip +# dependencies that no SONiC recipe names directly. +# Valid values: syft, trivy. +SBOM_SCAN_TOOL ?= syft + +# SBOM_INCLUDE_LICENSES - harvest /usr/share/doc//copyright before +# build_debian.sh strips it, and resolve SPDX license expressions for +# each component. Default on; turning off speeds up SBOM emit. +SBOM_INCLUDE_LICENSES ?= y + +# SBOM_STRICT - when y, the build fails if a critical SBOM input is +# missing (host rootfs fsroot-/, any declared installer +# docker .gz, or the scanner binary). Set n to downgrade to warnings +# for debugging or one-off partial emits. Soft optional features +# (SPDX conversion, provenance emit, license resolution) always +# warn-and-continue regardless. +SBOM_STRICT ?= y diff --git a/scripts/build_sbom.py b/scripts/build_sbom.py new file mode 100755 index 00000000000..524eb1ce1c5 --- /dev/null +++ b/scripts/build_sbom.py @@ -0,0 +1,1466 @@ +#!/usr/bin/env python3 +""" +build_sbom.py — SBOM aggregator for SONiC builds. + +Invoked between build_debian.sh and build_image.sh (from slave.mk) once +the host rootfs and all containers are assembled. Produces: + + target/.cdx.json (CycloneDX 1.6, sibling of the installer) + — is sonic-.bin + for ONIE installers, .swi for Arista + aboot, .img.gz for VS/VPP + +Inputs (env vars from slave.mk): + + ENABLE_SBOM must be 'y'; otherwise this is a no-op. + SBOM_SCAN_TOOL syft (default) | trivy + SBOM_FORMAT cyclonedx (default) | spdx | both + TARGET_PATH build output dir (default: 'target') + TARGET_MACHINE from onie-image.conf — names the SBOM file + CONFIGURED_ARCH amd64 | arm64 | armhf + CONFIGURED_PLATFORM broadcom | mellanox | vs | ... + SONIC_VERSION_CONTROL_COMPONENTS active pin policy (recorded in metadata) + SBOM_INSTALLER_DOCKERS space-separated list of docker .gz filenames + that actually ship in this installer. + SBOM_INSTALLER_DEBS space-separated list of .deb filenames + installed into the host rootfs. + SBOM_INSTALLER_WHEELS space-separated list of .whl filenames + installed into the host rootfs. + +Algorithm: + + 1. Walk per-artifact recipe-emit fragments (.cdx.json + next to each .deb / .whl / .gz). These are authoritative for + SONiC-built artifacts. + 2. For each in-scope scope (host rootfs + each in-scope container), + read the post-versions/ manifest written by sonic-build-hooks. + Add observation components for any (name, version) not already + covered by a recipe fragment. + 3. Optionally run the configured scanner (syft / trivy) as a wide + net to catch transitive deps and language-ecosystem items the + observation pass missed. + 4. Dedupe by (purl, arch): when multiple sources name the same + component, recipe-emit wins (it has pedigree + patches data). + 5. Annotate top-level metadata with the build context. + 6. Emit one CycloneDX 1.6 document as the sibling of the .bin. + +Failure mode: when ENABLE_SBOM=y the user has opted into SBOM emission +and a quietly-incomplete SBOM is worse than a build failure. The +aggregator validates that core inputs exist (host rootfs, declared +installer dockers, scanner binary) and exits non-zero if any are +missing. Set SBOM_STRICT=n to downgrade these to warnings for +debugging or one-off partial emits. Soft optional features +(SPDX conversion, provenance emit, license resolution) continue to +warn-and-continue. +""" + +import datetime +import hashlib +import json +import os +import re +import subprocess +import sys +from typing import Any, Optional + + +def warn(msg: str) -> None: + sys.stderr.write(f"[build_sbom.py] WARNING: {msg}\n") + + +def info(msg: str) -> None: + sys.stderr.write(f"[build_sbom.py] {msg}\n") + + +def error(msg: str) -> None: + sys.stderr.write(f"[build_sbom.py] ERROR: {msg}\n") + + +class SbomInputMissing(Exception): + """Raised in strict mode when a required SBOM input is missing. + + The build_sbom recipe in slave.mk treats this as fatal — the user + opted into SBOM generation via ENABLE_SBOM=y and we can't honor + that opt-in if a core data source (host rootfs, installer docker, + scanner binary) is absent. + """ + + +def check_required_inputs( + target_path: str, + target_machine: str, + installer_dockers: list, + scan_tool: str, +) -> None: + """Validate that everything we declared we'd consume is actually + present on disk. Raises SbomInputMissing on the first failure in + strict mode; logs a warning and returns in lenient mode. + + Strict by default when ENABLE_SBOM=y; opt out with SBOM_STRICT=n + for debugging or one-off partial SBOM emits. + """ + strict = os.environ.get("SBOM_STRICT", "y").lower() == "y" + problems: list[str] = [] + + # 1. Host rootfs (sibling of target/, populated by build_debian.sh). + # Without it the SBOM is missing grub/kernel/host-utility/docker + # daemon visibility — the largest CVE surface on the .bin. + fsroot = os.path.join( + os.path.dirname(os.path.abspath(target_path)), + f"fsroot-{target_machine}", + ) + if not os.path.isdir(fsroot): + problems.append( + f"host rootfs not found at {fsroot}; cannot scan " + f"host-installed packages (grub, kernel, docker daemon, " + f"etc.). The build_sbom hook must run after build_debian.sh " + f"and before fsroot cleanup." + ) + + # 2. Every declared installer docker must exist as a .gz in target/. + # These ship in the .bin; missing means a broken build that + # we should not pretend to inventory. + for docker in installer_dockers: + gz_path = os.path.join(target_path, docker) + if not os.path.isfile(gz_path): + problems.append( + f"installer docker {docker} declared in " + f"SBOM_INSTALLER_DOCKERS but missing at {gz_path}" + ) + + # 3. Scanner binary. Without it the SBOM loses CPE-tagged + # components and grype can't perform NVD matching downstream. + if scan_tool and scan_tool not in ("none", "off", "skip"): + scanner_bin = install_scanner(scan_tool) + if not scanner_bin or not os.path.isfile(scanner_bin): + problems.append( + f"scanner '{scan_tool}' could not be installed via " + f"scripts/install_sbom_tool.sh; without it the SBOM " + f"loses host-rootfs and container-scan coverage" + ) + + if not problems: + return + + msg = "SBOM input validation failed:\n - " + "\n - ".join(problems) + if strict: + error(msg) + error("Set SBOM_STRICT=n to continue with a partial SBOM " + "(not recommended).") + raise SbomInputMissing(msg) + else: + warn(msg) + warn("SBOM_STRICT=n; continuing with a partial SBOM.") + + +# --------------------------------------------------------------------------- +# Helpers +# --------------------------------------------------------------------------- + + +def split_env_list(name: str) -> list: + return [x.strip() for x in os.environ.get(name, "").split() if x.strip()] + + +def now_iso() -> str: + epoch = os.environ.get("SOURCE_DATE_EPOCH") + if epoch: + try: + return datetime.datetime.fromtimestamp( + int(epoch), tz=datetime.timezone.utc + ).strftime("%Y-%m-%dT%H:%M:%SZ") + except Exception: + pass + return datetime.datetime.now(datetime.timezone.utc).strftime( + "%Y-%m-%dT%H:%M:%SZ" + ) + + +def load_json(path: str) -> Optional[dict]: + try: + with open(path) as f: + return json.load(f) + except Exception: + return None + + +def file_sha256(path: str) -> Optional[str]: + try: + h = hashlib.sha256() + with open(path, "rb") as f: + for chunk in iter(lambda: f.read(65536), b""): + h.update(chunk) + return h.hexdigest() + except Exception: + return None + + +def run(cmd: list, timeout: int = 600) -> tuple: + """Returns (returncode, stdout, stderr).""" + try: + r = subprocess.run( + cmd, capture_output=True, text=True, timeout=timeout, check=False + ) + return r.returncode, r.stdout, r.stderr + except subprocess.TimeoutExpired: + return 124, "", "timeout" + except Exception as e: + return 1, "", str(e) + + +# --------------------------------------------------------------------------- +# Recipe-emit fragment collection +# --------------------------------------------------------------------------- + + +class FragmentIndex: + """Walks target/ for .cdx.json sidecars.""" + + def __init__(self, target_path: str): + self.target_path = target_path + self.fragments: dict = {} # filename → fragment-component + self.all: list = [] + self._load() + + def _load(self): + if not os.path.isdir(self.target_path): + return + for root, _, files in os.walk(self.target_path): + # Skip the sbom-tools cache and per-scope tmp dirs we created. + if "sbom-tools" in root or "sbom-tmp" in root: + continue + for fn in files: + if fn.endswith(".cdx.json"): + # Only consume sidecar fragments — skip the final + # aggregate output if it has already been written + # in a prior run. The aggregate is named after the + # installer artifact (sonic-.bin /.swi + # / .img.gz), so match the 'sonic-' prefix plus + # any of the known installer extensions. + if fn.startswith("sonic-") and ( + ".bin.cdx.json" in fn + or ".swi.cdx.json" in fn + or ".img.gz.cdx.json" in fn + ): + continue + doc = load_json(os.path.join(root, fn)) + if not doc: + continue + meta_props = { + p.get("name"): p.get("value") + for p in doc.get("metadata", {}).get("properties", []) + } + if meta_props.get("sonic:fragment_kind") != "recipe-emit": + continue + for comp in doc.get("components", []): + # Index by the artifact filename so we can match + # against the installer's in-scope lists. + artifact_filename = None + for prop in comp.get("properties", []): + if prop.get("name") == "sonic:artifact_filename": + artifact_filename = prop.get("value") + break + if artifact_filename: + self.fragments[artifact_filename] = comp + self.all.append(comp) + + def for_filename(self, name: str) -> Optional[dict]: + return self.fragments.get(name) + + +# --------------------------------------------------------------------------- +# Observation: post-versions/ manifests +# --------------------------------------------------------------------------- + + +_PKG_VER_RE = re.compile(r"^([^=]+)==(.+)$") + + +def parse_versions_file(path: str) -> list: + """Reads a versions-deb-* or versions-py3-* manifest into [(name, ver)].""" + out = [] + try: + with open(path) as f: + for line in f: + line = line.strip() + if not line or line.startswith("#"): + continue + m = _PKG_VER_RE.match(line) + if m: + out.append((m.group(1).strip(), m.group(2).strip())) + except Exception as e: + warn(f"could not read {path}: {e}") + return out + + +def find_post_versions(target_path: str, scope: str, kind: str, + arch: str) -> list: + """Locate post-versions/versions--*- for a scope. + + scope is e.g. 'host-image' or 'dockers/docker-fpm-frr'. + kind is 'deb' or 'py3'. + """ + base = os.path.join(target_path, "versions", scope, "post-versions") + if not os.path.isdir(base): + return [] + matches = [] + for fn in os.listdir(base): + if fn.startswith(f"versions-{kind}-") and fn.endswith(f"-{arch}"): + matches.append(os.path.join(base, fn)) + return sorted(matches) + + +def find_copyright_tarballs(target_path: str) -> list: + """Find every per-scope copyrights.tar.gz under target/versions/.""" + return _find_tarballs(target_path, "copyrights.tar.gz") + + +def find_lockfile_tarballs(target_path: str) -> list: + """Find every per-scope lockfiles.tar.gz under target/versions/.""" + return _find_tarballs(target_path, "lockfiles.tar.gz") + + +def _find_tarballs(target_path: str, name: str) -> list: + base = os.path.join(target_path, "versions") + if not os.path.isdir(base): + return [] + out = [] + for root, _, files in os.walk(base): + for fn in files: + if fn == name: + out.append(os.path.join(root, fn)) + return sorted(out) + + +def parse_lockfiles_for_scope(target_path: str, scope: str) -> list: + """Parse only the lockfiles under a single scope dir + (e.g. 'dockers/docker-ptf' or 'host-image'). Used by the + per-container SBOM emit path so a container's sidecar SBOM only + contains its own transitive lockfile deps.""" + tarball = os.path.join( + target_path, "versions", scope, "post-versions", "lockfiles.tar.gz", + ) + if not os.path.isfile(tarball): + return [] + out_json = os.path.join( + target_path, + f"sbom-lockfile-components-{scope.replace('/', '-')}.json", + ) + script = os.path.join(os.path.dirname(__file__), + "sbom_parse_lockfiles.py") + rc, _, err = run( + ["python3", script, "--output", out_json, "--lockfiles", tarball], + timeout=300, + ) + if rc != 0: + warn(f"lockfile parser failed for scope {scope} " + f"(rc={rc}): {err.strip()[:200]}") + return [] + try: + with open(out_json) as f: + data = json.load(f) + return data.get("components", []) + except Exception as e: + warn(f"could not read scoped lockfile parser output: {e}") + return [] + + +def parse_lockfiles(target_path: str) -> list: + """Run scripts/sbom_parse_lockfiles.py over every harvested + lockfiles.tar.gz; return the list of CycloneDX components.""" + tarballs = find_lockfile_tarballs(target_path) + if not tarballs: + return [] + out_json = os.path.join(target_path, "sbom-lockfile-components.json") + script = os.path.join(os.path.dirname(__file__), + "sbom_parse_lockfiles.py") + cmd = ["python3", script, "--output", out_json] + for t in tarballs: + cmd.extend(["--lockfiles", t]) + rc, _, err = run(cmd, timeout=600) + if rc != 0: + warn(f"lockfile parser failed (rc={rc}): {err.strip()[:200]}") + return [] + try: + with open(out_json) as f: + data = json.load(f) + return data.get("components", []) + except Exception as e: + warn(f"could not read lockfile parser output: {e}") + return [] + + +def resolve_licenses(target_path: str) -> dict: + """Returns { pkg_name: spdx_expression }. + + Runs scripts/sbom_resolve_licenses.py against every copyrights.tar.gz + found under target/. The resolver does the heavy lifting (DEP-5 + parsing, licensecheck fallback, SPDX mapping). + """ + tarballs = find_copyright_tarballs(target_path) + if not tarballs: + return {} + + out_json = os.path.join(target_path, "sbom-licenses.json") + script = os.path.join(os.path.dirname(__file__), + "sbom_resolve_licenses.py") + cmd = ["python3", script, "--output", out_json] + for t in tarballs: + cmd.extend(["--copyrights", t]) + rc, _, err = run(cmd, timeout=600) + if rc != 0: + warn(f"license resolver failed (rc={rc}): {err.strip()[:200]}") + return {} + try: + with open(out_json) as f: + data = json.load(f) + return data.get("resolved", {}) + except Exception as e: + warn(f"could not read resolver output: {e}") + return {} + + +def apply_licenses(components: list, license_map: dict) -> tuple: + """Attach licenses[] to components that lack one. + Returns (with_license_count, noassertion_count).""" + resolved = 0 + noassertion = 0 + for c in components: + if c.get("licenses"): + resolved += 1 + continue + name = (c.get("name") or "").lower() + if not name: + continue + spdx = license_map.get(name) + if not spdx: + # Debian binary packages often have a source-package name that + # carries the copyright. Don't have that here; just leave it + # NOASSERTION. + noassertion += 1 + continue + if spdx == "NOASSERTION": + c["licenses"] = [{"license": {"id": "NOASSERTION"}}] + noassertion += 1 + else: + c["licenses"] = [{"expression": spdx}] + resolved += 1 + return resolved, noassertion + + +def observation_components_for_scope( + target_path: str, scope: str, arch: str, supplier: str, +) -> list: + """Emit observation-only components for everything in post-versions/.""" + components = [] + seen: set = set() + + for vfile in find_post_versions(target_path, scope, "deb", arch): + for name, ver in parse_versions_file(vfile): + key = (name, ver, arch) + if key in seen: + continue + seen.add(key) + comp: dict[str, Any] = { + "bom-ref": f"pkg:deb/debian/{name}@{ver}?arch={arch}", + "type": "library", + "name": name, + "version": ver, + "purl": f"pkg:deb/debian/{name}@{ver}?arch={arch}", + "supplier": {"name": supplier}, + "properties": [ + {"name": "sonic:fragment_kind", "value": "observation"}, + {"name": "sonic:scope", "value": scope}, + {"name": "sonic:arch", "value": arch}, + ], + } + components.append(comp) + + for vfile in find_post_versions(target_path, scope, "py3", arch): + for name, ver in parse_versions_file(vfile): + norm = name.replace("_", "-").lower() + key = ("pypi", norm, ver) + if key in seen: + continue + seen.add(key) + comp = { + "bom-ref": f"pkg:pypi/{norm}@{ver}", + "type": "library", + "name": norm, + "version": ver, + "purl": f"pkg:pypi/{norm}@{ver}", + "supplier": {"name": "PyPI"}, + "properties": [ + {"name": "sonic:fragment_kind", "value": "observation"}, + {"name": "sonic:scope", "value": scope}, + ], + } + components.append(comp) + + return components + + +# --------------------------------------------------------------------------- +# Scanner pass (syft / trivy) +# --------------------------------------------------------------------------- + + +def install_scanner(tool: str) -> Optional[str]: + """Call scripts/install_sbom_tool.sh; return the path to the binary.""" + script = os.path.join(os.path.dirname(__file__), "install_sbom_tool.sh") + rc, out, err = run([script, tool], timeout=300) + if rc != 0: + warn(f"install_sbom_tool.sh {tool} failed (rc={rc}): {err.strip()}") + return None + return out.strip() or None + + +def _scanner_cache_dir() -> str: + """Cache directory for scanner outputs, sibling to the scanner binary. + + Lives under target/ so `make reset` (which wipes target/) invalidates + the cache automatically — no stale entries across resets. + """ + target_path = os.environ.get("TARGET_PATH", "target") + d = os.path.join(target_path, "sbom-tools", "syft-cache") + try: + os.makedirs(d, exist_ok=True) + except OSError: + pass + return d + + +def _scanner_cache_lookup( + tool: str, fs_path: str, +) -> tuple: + """Return (sha256, cached_components | None) for a file-based scan. + + Returns (None, None) if SHA-256 cannot be computed (e.g. file + disappeared between exists-check and hash). A non-None sha with + None components indicates a cache miss that the caller can fill + via _scanner_cache_store after running the scanner. + """ + sha = file_sha256(fs_path) + if not sha: + return None, None + cache_file = os.path.join(_scanner_cache_dir(), f"{tool}-{sha}.json") + if os.path.isfile(cache_file): + try: + with open(cache_file) as f: + return sha, json.load(f) + except Exception: + pass + return sha, None + + +def _scanner_cache_store(tool: str, sha: str, components: list) -> None: + """Persist scanner output keyed by file SHA-256. Only called for + non-empty results — an empty list could be a genuine zero-component + scan or a scanner failure that returned []; caching the latter would + poison subsequent variants.""" + if not sha or not components: + return + cache_file = os.path.join(_scanner_cache_dir(), f"{tool}-{sha}.json") + try: + tmp = cache_file + ".tmp" + with open(tmp, "w") as f: + json.dump(components, f) + os.replace(tmp, cache_file) + except Exception as e: + warn(f"could not write scanner cache {cache_file}: {e}") + + +def run_scanner(scanner_bin: str, tool: str, scan_target: str) -> list: + """Run scanner against a target; return components[] from output. + + scan_target may carry a syft scheme prefix (e.g. 'oci-archive:'). + SONiC's docker .gz files are gzipped OCI archives, and syft's + archive readers don't pipe through gzip — so for the oci-archive + case we transparently decompress to a temp file first. + + File-based scans (oci-archive:, fs paths) are cached by SHA-256 of + the input file. Across the 3 ASIC variants of a single build + (broadcom, broadcom-dnx, broadcom-legacy-th), the same docker .gz + files are scanned 3 times by the aggregator's per-variant + invocations; the cache short-circuits the 2nd and 3rd hits. The + dir: scheme (host rootfs) is not cached — fsroot-/ differs + per variant and a directory-tree hash would be expensive. + """ + scheme = "" + fs_path = scan_target + if ":" in scan_target: + scheme, fs_path = scan_target.split(":", 1) + if not os.path.exists(fs_path): + return [] + + # Cache lookup for file-based scans only. dir: scans are not + # cacheable because (a) hashing a directory tree is expensive and + # (b) fsroot-/ differs per variant — no reuse anyway. + cache_sha = None + if scheme != "dir" and os.path.isfile(fs_path): + cache_sha, cached = _scanner_cache_lookup(tool, fs_path) + if cached is not None: + return cached + + # syft's oci-archive reader doesn't handle gzip-wrapped tar. + # Stream-decompress to a temp file for the duration of the scan. + tmp_path = None + if (tool == "syft" and scheme == "oci-archive" + and _is_gzip(fs_path)): + import gzip + import tempfile + with tempfile.NamedTemporaryFile(suffix=".tar", delete=False) as tf: + tmp_path = tf.name + with gzip.open(fs_path, "rb") as gz: + while True: + chunk = gz.read(8 * 1024 * 1024) + if not chunk: + break + tf.write(chunk) + scan_target = f"{scheme}:{tmp_path}" + + try: + if tool == "syft": + cmd = [scanner_bin, scan_target, "-o", "cyclonedx-json", "-q"] + elif tool == "trivy": + cmd = [scanner_bin, "fs", "--format", "cyclonedx", "--quiet", + scan_target] + else: + return [] + result = _run_scanner_inner(cmd, tool, scan_target) + if cache_sha and result: + _scanner_cache_store(tool, cache_sha, result) + return result + finally: + if tmp_path and os.path.exists(tmp_path): + os.unlink(tmp_path) + + +def _is_gzip(path: str) -> bool: + """Cheap gzip-magic-bytes sniff.""" + try: + with open(path, "rb") as f: + return f.read(2) == b"\x1f\x8b" + except Exception: + return False + + +def _run_scanner_inner(cmd: list, tool: str, scan_target: str) -> list: + rc, out, err = run(cmd, timeout=900) + if rc != 0: + warn(f"{tool} scan of {scan_target} failed (rc={rc}): " + f"{err.strip()[:200]}") + return [] + try: + doc = json.loads(out) + except Exception as e: + warn(f"could not parse {tool} output for {scan_target}: {e}") + return [] + comps = doc.get("components") or [] + for c in comps: + c.setdefault("properties", []).append( + {"name": "sonic:fragment_kind", "value": "scanner"} + ) + c["properties"].append( + {"name": "sonic:scanner", "value": tool} + ) + return comps + + +# --------------------------------------------------------------------------- +# Merge with recipe-emit-wins dedupe +# --------------------------------------------------------------------------- + + +def _component_arch(c: dict) -> str: + for p in c.get("properties") or []: + if p.get("name") == "sonic:arch": + return p.get("value") or "" + return "" + + +# Suffixes that get added downstream of the recipe's filename version +# but before dpkg records the actually-installed version. We strip +# these (and a leading epoch like '1:') when computing a normalized +# dedupe key so the recipe-emit fragment 'openssh-server 10.0p1-7' +# matches the observation 'openssh-server 1:10.0p1-7+fips'. The pattern +# preserves the upstream-version prefix while eating the +# debian-build-system noise. +_VERSION_SUFFIX_RE = re.compile( + r"(?:\+(?:fips|sonic(?:\.\d+)?|b\d+(?:sonic\d*)?|deb\d+u\d+))+$" +) +_VERSION_EPOCH_RE = re.compile(r"^\d+:") + + +def _normalize_version(v: str) -> str: + v = _VERSION_EPOCH_RE.sub("", v) + # Strip the suffix chain iteratively (handles +sonic.0+b1). + while True: + m = _VERSION_SUFFIX_RE.search(v) + if not m: + break + v = v[: m.start()] + return v + + +def _dedupe_keys(c: dict) -> list: + """All keys a component should match against during dedupe. + + Returns the explicit PURL/bom-ref plus two normalized + (name, version, arch) tuples — one with raw version (catches exact + matches) and one with epoch+suffix stripped (catches the case where + recipe-emit uses the filename version `10.0p1-7` and the eventual + installed deb is `1:10.0p1-7+fips`). The two-key approach means: + - Exact version matches still dedupe (same as before). + - Different upstream versions of the same package stay distinct + (e.g. bash 5.2.15 in bookworm vs 5.2.37 in trixie). + - Only the build-system suffix drift collapses. + """ + keys = [] + purl = c.get("purl") + if purl: + keys.append(("purl", purl)) + bom_ref = c.get("bom-ref") + if bom_ref and bom_ref != purl: + keys.append(("bom-ref", bom_ref)) + name = (c.get("name") or "").lower() + version = c.get("version") or "" + arch = _component_arch(c) + if name and version: + keys.append(("nva", name, version, arch)) + # Always emit the normalized key, even when normalize is a no-op, + # so that a recipe-emit component (whose filename version usually + # IS the normalized form) shares a key with the observation + # component (whose dpkg version carries the +fips/+sonic/epoch + # noise). Without this, the two never see each other. + norm = _normalize_version(version) or version + keys.append(("nva-norm", name, norm, arch)) + return keys + + +# Names that identify kernel-module packages whose runtime depends on +# the Linux kernel binary. Used to build the CycloneDX dependencies[] +# graph so consumers can trace ABI-incompatible-kernel risks. +_KERNEL_MODULE_PATTERNS = [ + re.compile(r"^opennsl-modules"), # Broadcom XGS / DNX + re.compile(r"^sx-kernel"), # Mellanox SX + re.compile(r"^ionic-modules"), # AMD/Pensando ionic + re.compile(r"^mrvlteralynx"), # Marvell Teralynx + re.compile(r"^.*-dkms$"), # DKMS module debs (Bluefield etc.) + re.compile(r"^sonic-platform-modules-"), # vendor platform-modules + re.compile(r"^platform-modules-"), # micas-style + re.compile(r"^saibcm-modules"), # Broadcom SAI kernel piece + re.compile(r"^.*-kernel-modules$"), +] + + +def _is_kernel_module(name: str) -> bool: + return any(p.match(name or "") for p in _KERNEL_MODULE_PATTERNS) + + +def _is_kernel_image(name: str) -> bool: + """Match the primary linux-image fragment. Exclude debug/headers/kbuild.""" + n = name or "" + if not n.startswith("linux-image-"): + return False + # Reject -dbg and -dbgsym variants; the primary kernel is what + # modules link against at runtime. + if n.endswith("-dbg") or n.endswith("-dbgsym") or "-unsigned-dbg" in n: + return False + return True + + +def build_dependency_graph(components: list) -> list: + """Return a CycloneDX dependencies[] array recording the edges + we can derive from recipe-emit metadata. + + Three edge classes are emitted into a single unscoped graph + (CycloneDX 1.6 dependencies[] doesn't distinguish build-time vs + runtime; analytics that need the split should read the + sonic:build_depends / sonic:runtime_depends properties off the + components themselves): + + 1. Kernel-module -> kernel-image. Out-of-tree modules (Broadcom + OPENNSL, Mellanox SX, etc.) are built against a specific + kernel ABI; recording the edge lets consumers reason about + kernel-ABI-compatible upgrade paths and CVE blast radius. + + 2. SONiC-built .deb -> declared build/runtime deps. Read from + sonic:build_depends / sonic:runtime_depends string properties + that sbom_fragment.py copies out of the recipe's $(pkg)_DEPENDS + and $(pkg)_RDEPENDS makefile variables. The property strings + are space-separated .deb filenames that we resolve back to the + sibling fragment's bom-ref via sonic:artifact_filename. Filenames + that don't resolve (almost always upstream packages from + Debian for which we have no recipe-emit fragment) are recorded + as a sonic:unresolved_deps property on the component for audit; + they don't appear in the graph. + + 3. SONiC-built .deb -> per-binary language deps shipped inside. + sbom_fragment.py emits each Rust crate / Go module / Python + dist-info entry as a recipe-emit-{rust,go,python} component + carrying sonic:source_deb=; we reverse that + into an edge from the .deb's bom-ref to the language-dep's + bom-ref so a consumer can walk swss_*.deb -> tokio@1.x or + sonic-gnmi_*.deb -> github.com/openconfig/gnmi@v0.10 without + parsing properties. + """ + # filename -> bom-ref lookup over the merged component set. The + # same resolution path is used by all three edge classes that need + # to refer to a sibling component by its on-disk artifact name. + filename_to_ref: dict = {} + for c in components: + ref = c.get("bom-ref") + if not ref: + continue + for prop in c.get("properties", []) or []: + if prop.get("name") == "sonic:artifact_filename": + fn = prop.get("value") + if fn: + filename_to_ref[fn] = ref + break + + # Accumulate as ref -> set(dependsOn refs) so multiple edge + # classes that converge on the same source component merge into + # a single CycloneDX dependencies[] entry per ref. + edges: dict = {} + + # (1) kernel-module -> kernel-image + kernel_refs = [ + c["bom-ref"] for c in components + if _is_kernel_image(c.get("name")) and c.get("bom-ref") + ] + if kernel_refs: + kernel_set = set(kernel_refs) + for c in components: + if not _is_kernel_module(c.get("name") or ""): + continue + ref = c.get("bom-ref") + if ref: + edges.setdefault(ref, set()).update(kernel_set) + + # (2) declared build/runtime deps (resolved to sibling fragments) + for c in components: + ref = c.get("bom-ref") + if not ref: + continue + build_dep_str = "" + runtime_dep_str = "" + for prop in c.get("properties", []) or []: + n = prop.get("name") + if n == "sonic:build_depends": + build_dep_str = prop.get("value", "") or "" + elif n == "sonic:runtime_depends": + runtime_dep_str = prop.get("value", "") or "" + if not (build_dep_str or runtime_dep_str): + continue + dep_filenames = set(build_dep_str.split()) | set(runtime_dep_str.split()) + resolved: set = set() + unresolved: set = set() + for fn in dep_filenames: + if not fn: + continue + tgt = filename_to_ref.get(fn) + if tgt and tgt != ref: + resolved.add(tgt) + elif not tgt: + unresolved.add(fn) + if resolved: + edges.setdefault(ref, set()).update(resolved) + if unresolved: + props = c.setdefault("properties", []) + existing = None + for p in props: + if p.get("name") == "sonic:unresolved_deps": + existing = p + break + joined = " ".join(sorted(unresolved)) + if existing is not None: + merged = sorted(set( + (existing.get("value", "") or "").split() + ) | unresolved) + existing["value"] = " ".join(merged) + else: + props.append({ + "name": "sonic:unresolved_deps", + "value": joined, + }) + + # (3) per-binary language deps (recipe-emit-{rust,go,python}) -> + # their owning .deb. sbom_fragment.py attaches sonic:source_deb to + # every such component; we look up the .deb's bom-ref and reverse + # the attribution into a dependsOn edge. + for c in components: + crate_ref = c.get("bom-ref") + if not crate_ref: + continue + source_deb = None + is_lang_dep = False + for prop in c.get("properties", []) or []: + n = prop.get("name") + v = prop.get("value", "") + if n == "sonic:source_deb": + source_deb = v + elif n == "sonic:fragment_kind" and v.startswith("recipe-emit-") and v != "recipe-emit": + is_lang_dep = True + if not (is_lang_dep and source_deb): + continue + deb_ref = filename_to_ref.get(source_deb) + if deb_ref and deb_ref != crate_ref: + edges.setdefault(deb_ref, set()).add(crate_ref) + + deps = [ + {"ref": ref, "dependsOn": sorted(targets)} + for ref, targets in edges.items() + if targets + ] + return deps + + +def merge_components(*sources: list) -> list: + """Dedupe by (purl) and (name, version, arch). Sources are passed in + PRIORITY order; first occurrence wins for the base record. But for + components dropped by dedupe, we *promote* their CPE list onto the + winner — recipe-emit fragments carry rich SONiC provenance but no + CPE, while syft (lower priority) produces CPEs that grype needs for + NVD-based CVE matching when distro detection isn't available. + """ + seen: dict = {} # dedupe key -> winner index in `out` + out: list = [] + for src in sources: + for c in src: + keys = _dedupe_keys(c) + if not keys: + continue + winner_idx = next((seen[k] for k in keys if k in seen), None) + if winner_idx is not None: + _promote_cpe(out[winner_idx], c) + continue + for k in keys: + seen[k] = len(out) + out.append(c) + return out + + +def _promote_cpe(winner: dict, loser: dict) -> None: + """Move CPE data from a deduped-out record onto the winner. + + Grype's NVD matcher relies on the `cpe` field; recipe-emit + fragments don't produce one, but syft does. Without this + promotion, every recipe-built Debian/sonic package loses the CPE + that would have let grype match it against the Debian/NVD CVE + feeds. + """ + cpe = loser.get("cpe") + if cpe and not winner.get("cpe"): + winner["cpe"] = cpe + cpes = loser.get("cpes") + if cpes and not winner.get("cpes"): + winner["cpes"] = cpes + + +# --------------------------------------------------------------------------- +# Top-level orchestration +# --------------------------------------------------------------------------- + + +def _container_main(container_filename: str) -> int: + """Produce a sidecar SBOM for a single container archive. + + Used for test containers (docker-ptf, docker-sonic-mgmt) that are + out-of-scope for the .bin SBOM but still need their own + vulnerability/provenance surface. Also used for in-scope + containers, where it complements the merged .bin SBOM with a + container-level view useful for diffing and per-container + security scanning. + + Output: target/.sbom.cdx.json + """ + target_path = os.environ.get("TARGET_PATH", "target") + arch = os.environ.get( + "CONFIGURED_ARCH", + subprocess.run( + ["dpkg", "--print-architecture"], + capture_output=True, text=True, check=False, + ).stdout.strip() or "amd64", + ) + platform = os.environ.get("CONFIGURED_PLATFORM", "") + scan_tool = os.environ.get("SBOM_SCAN_TOOL", "syft") + vcc = os.environ.get("SONIC_VERSION_CONTROL_COMPONENTS", "") + + gz_path = os.path.join(target_path, container_filename) + if not os.path.isfile(gz_path): + info(f"container archive not found: {gz_path}; skipping.") + return 0 + + cname = container_filename.replace(".gz", "").replace("-dbg", "") + out_path = os.path.join( + target_path, f"{container_filename}.sbom.cdx.json" + ) + info(f"Building per-container SBOM for {container_filename}") + + # Recipe-emit fragment for this container (if any). + fragments = FragmentIndex(target_path) + frag = fragments.for_filename(container_filename) + recipe_components = [frag] if frag else [] + + # Synthesize a container-typed parent component if no recipe + # fragment exists (out-of-scope test containers go this path). + if frag: + container_comp = frag + else: + version = os.environ.get("SONIC_IMAGE_VERSION", "0.0.0") + container_comp = { + "bom-ref": f"pkg:oci/{cname}@{version}?arch={arch}", + "type": "container", + "name": cname, + "version": version, + "purl": f"pkg:oci/{cname}@{version}?arch={arch}", + "properties": [ + {"name": "sonic:fragment_kind", "value": "observation"}, + {"name": "sonic:arch", "value": arch}, + ], + } + sha = file_sha256(gz_path) + if sha: + container_comp["hashes"] = [ + {"alg": "SHA-256", "content": sha} + ] + + # Observation: only this container's scope. + scope = f"dockers/{cname}" + obs = observation_components_for_scope( + target_path, scope, arch, "Debian", + ) + info(f"Container observation: {len(obs)} components") + + # Lockfile-derived: only this container's scope. + lockfile_components = parse_lockfiles_for_scope(target_path, scope) + info(f"Lockfile-derived: {len(lockfile_components)} components") + + # Scanner: this single .gz. SONiC's docker .gz files are OCI + # archives (not docker-archive tarballs), so syft needs the + # 'oci-archive:' scheme. + scanner_components: list = [] + if scan_tool and scan_tool not in ("none", "off", "skip"): + scanner_bin = install_scanner(scan_tool) + if scanner_bin: + if scan_tool == "syft": + target_spec = f"oci-archive:{gz_path}" + else: + target_spec = gz_path + scanner_components = run_scanner( + scanner_bin, scan_tool, target_spec, + ) + info(f"Scanner cross-check: {len(scanner_components)} components") + + all_components = merge_components( + recipe_components, + [container_comp], + obs, + lockfile_components, + scanner_components, + ) + info(f"Final merged: {len(all_components)} unique components") + + # License resolution (per-scope copyrights tarball + fallback to + # the full set if the per-scope tarball is missing). + if os.environ.get("SBOM_INCLUDE_LICENSES", "y") == "y": + license_map = resolve_licenses(target_path) + if license_map: + resolved, noassertion = apply_licenses( + all_components, license_map, + ) + total = len(all_components) + pct = (100.0 * resolved / total) if total else 0.0 + info(f"License resolution: {resolved}/{total} resolved " + f"({pct:.1f}%); {noassertion} NOASSERTION") + + sbom: dict[str, Any] = { + "bomFormat": "CycloneDX", + "specVersion": "1.6", + "version": 1, + "metadata": { + "timestamp": now_iso(), + "tools": [{ + "vendor": "SONiC", + "name": "build_sbom.py", + "version": "1.0", + }], + "component": { + "type": "container", + "bom-ref": container_comp.get("bom-ref", cname), + "name": cname, + "version": container_comp.get("version", "0.0.0"), + }, + "properties": [ + {"name": "sonic:platform", "value": platform}, + {"name": "sonic:arch", "value": arch}, + {"name": "sonic:scope_kind", + "value": "single-container"}, + {"name": "sonic:container", "value": cname}, + {"name": "sonic:version_control_components", + "value": vcc}, + {"name": "sonic:scan_tool", "value": scan_tool}, + ], + }, + "components": sorted( + all_components, key=lambda c: c.get("bom-ref", "") + ), + } + + deps = build_dependency_graph(all_components) + if deps: + sbom["dependencies"] = sorted(deps, key=lambda d: d.get("ref", "")) + + try: + with open(out_path, "w") as f: + json.dump(sbom, f, indent=2, sort_keys=True) + info(f"SBOM written: {out_path}") + info(f"Component count: {len(all_components)}") + except Exception as e: + warn(f"could not write {out_path}: {e}") + + # SPDX conversion (optional) + sbom_format = os.environ.get("SBOM_FORMAT", "cyclonedx").lower() + if sbom_format in ("spdx", "both"): + # Reuse emit_spdx but with the per-container output path; the + # function derives the spdx path from target_machine, so we + # call cyclonedx-cli directly here for the per-container case. + spdx_path = out_path.removesuffix(".cdx.json") + ".spdx.json" + _convert_to_spdx(out_path, spdx_path) + + return 0 + + +def _convert_to_spdx(cdx_path: str, spdx_path: str) -> None: + """Shared SPDX conversion helper. Used by both bin and container modes.""" + script = os.path.join(os.path.dirname(__file__), "install_sbom_tool.sh") + rc, out, err = run([script, "cyclonedx-cli"], timeout=300) + if rc != 0: + warn(f"could not install cyclonedx-cli (rc={rc}): {err.strip()[:200]}") + return + binary = out.strip() + if not binary or not os.path.isfile(binary): + warn(f"cyclonedx-cli binary not found at {binary!r}") + return + rc, _, err = run( + [binary, "convert", + "--input-file", cdx_path, + "--output-file", spdx_path, + "--output-format", "spdxjson"], + timeout=300, + ) + if rc != 0: + warn(f"cyclonedx-cli convert failed (rc={rc}): {err.strip()[:200]}") + return + info(f"SPDX written: {spdx_path}") + + +def main() -> int: + if os.environ.get("ENABLE_SBOM", "n") != "y": + info("ENABLE_SBOM=n; skipping.") + return 0 + + import argparse + ap = argparse.ArgumentParser(description="SONiC SBOM aggregator") + ap.add_argument( + "--container", metavar="FILENAME", + help="Emit a single-container SBOM (e.g. 'docker-ptf.gz'). " + "Output goes to target/.sbom.cdx.json. In this " + "mode, host rootfs observation and the .bin scope filter " + "are bypassed; only the named container's data is " + "aggregated. Used to produce sidecar SBOMs for test " + "containers (docker-ptf, docker-sonic-mgmt) that are " + "out-of-scope for the .bin SBOM but still need their own " + "vulnerability surface documented.", + ) + args = ap.parse_args() + + if args.container: + return _container_main(args.container) + + target_path = os.environ.get("TARGET_PATH", "target") + target_machine = os.environ.get("TARGET_MACHINE", "generic") + arch = os.environ.get("CONFIGURED_ARCH", + subprocess.run( + ["dpkg", "--print-architecture"], + capture_output=True, text=True, check=False, + ).stdout.strip() or "amd64") + platform = os.environ.get("CONFIGURED_PLATFORM", "") + scan_tool = os.environ.get("SBOM_SCAN_TOOL", "syft") + vcc = os.environ.get("SONIC_VERSION_CONTROL_COMPONENTS", "") + + installer_dockers = split_env_list("SBOM_INSTALLER_DOCKERS") + + # Strict-mode validation: fail loudly when the user enabled SBOM + # generation but a critical input is missing. Better to break the + # build than ship a quietly-incomplete SBOM (e.g. host rootfs + # missing → no grub/kernel/docker-daemon visibility). + try: + check_required_inputs( + target_path, target_machine, installer_dockers, scan_tool, + ) + except SbomInputMissing: + return 1 + + # Derive output filenames from the actual artifact basename + # (e.g. 'sonic-broadcom.bin', 'sonic-aboot-broadcom.swi', + # 'sonic-vs.img.gz') so siblings line up regardless of installer + # format. Falls back to '.bin' for any caller that + # didn't plumb SBOM_TARGET_ARTIFACT through. + artifact_basename = os.environ.get( + "SBOM_TARGET_ARTIFACT", f"sonic-{target_machine}.bin" + ) + out_path = os.path.join(target_path, f"{artifact_basename}.cdx.json") + info(f"Building SBOM for {artifact_basename} " + f"(platform={platform}, arch={arch})") + + fragments = FragmentIndex(target_path) + info(f"Loaded {len(fragments.all)} recipe-emit fragments from {target_path}/") + + # ---- Recipe-emit components (authoritative for SONiC-built things) ---- + recipe_components = list(fragments.all) + + # ---- Observation components for host rootfs ---- + obs_host = observation_components_for_scope( + target_path, "host-image", arch, "Debian", + ) + info(f"Host rootfs observation: {len(obs_host)} components") + + # ---- Per-container observation + container identity ---- + container_components: list = [] + obs_containers: list = [] + for docker in installer_dockers: + # The docker filename in the installer list may be e.g. + # 'docker-fpm-frr.gz' or 'docker-fpm-frr-dbg.gz'. + gz_path = os.path.join(target_path, docker) + cname = docker.replace(".gz", "").replace("-dbg", "") + + # Build a container-typed parent component (use recipe fragment + # if present, otherwise synthesize). + frag = fragments.for_filename(docker) + if frag: + container_comp = frag + else: + container_comp = { + "bom-ref": f"pkg:oci/{cname}@{os.environ.get('SONIC_IMAGE_VERSION', '0.0.0')}?arch={arch}", + "type": "container", + "name": cname, + "version": os.environ.get("SONIC_IMAGE_VERSION", "0.0.0"), + "properties": [ + {"name": "sonic:fragment_kind", "value": "observation"}, + {"name": "sonic:arch", "value": arch}, + ], + } + if os.path.isfile(gz_path): + sha = file_sha256(gz_path) + if sha: + container_comp.setdefault("hashes", []).append( + {"alg": "SHA-256", "content": sha} + ) + container_components.append(container_comp) + + # In-container observation set. + scope = f"dockers/{cname}" + obs_containers.extend(observation_components_for_scope( + target_path, scope, arch, "Debian", + )) + + info(f"Container observation: {len(obs_containers)} components " + f"across {len(installer_dockers)} containers") + + # ---- Lockfile-derived components (Rust/Go/npm transitive deps) ---- + lockfile_components = parse_lockfiles(target_path) + info(f"Lockfile-derived: {len(lockfile_components)} components") + + # ---- Optional scanner cross-check ---- + scanner_components: list = [] + if scan_tool and scan_tool not in ("none", "off", "skip"): + scanner_bin = install_scanner(scan_tool) + if scanner_bin: + info(f"Running {scan_tool} from {scanner_bin} as cross-check") + # Scan the host rootfs. SONiC stages this as fsroot-/ + # at the repo root (a sibling of target/), populated by + # build_debian.sh and packed into a squashfs by build_image.sh. + # We scan the directory tree directly — syft's `dir:` source + # picks up Debian packages, embedded Go modules (for docker + # daemon binaries etc.), grub stage binaries, and so on. + # This is the source-of-truth scan for everything that ships + # outside docker containers (kernel/grub/host utilities/the + # docker daemon itself). + fsroot = os.path.join( + os.path.dirname(os.path.abspath(target_path)), + f"fsroot-{target_machine}", + ) + if os.path.isdir(fsroot): + if scan_tool == "syft": + target_spec = f"dir:{fsroot}" + else: + target_spec = fsroot + info(f"Scanning host rootfs: {fsroot}") + scanner_components.extend( + run_scanner(scanner_bin, scan_tool, target_spec) + ) + else: + info(f"host rootfs not found at {fsroot}; " + f"skipping host-rootfs scan") + # Scan each in-scope container archive. SONiC's docker .gz + # files are OCI archives (not docker-archive tarballs), so + # syft needs the 'oci-archive:' scheme. + for docker in installer_dockers: + gz_path = os.path.join(target_path, docker) + if os.path.isfile(gz_path): + if scan_tool == "syft": + target_spec = f"oci-archive:{gz_path}" + else: + target_spec = gz_path + scanner_components.extend( + run_scanner(scanner_bin, scan_tool, target_spec) + ) + + info(f"Scanner cross-check: {len(scanner_components)} components") + + # ---- Merge (recipe-emit wins, then observation, then lockfile, + # then scanner) ---- + # Lockfile sits between observation and scanner: it gives more + # precise identity than scanner's binary detection (it has crate + # hashes from the lockfile), but observation/recipe-emit win when + # they describe the same component because they carry richer + # SONiC-specific provenance. + all_components = merge_components( + recipe_components, + container_components, + obs_host, + obs_containers, + lockfile_components, + scanner_components, + ) + + info(f"Final merged: {len(all_components)} unique components") + + # ---- License resolution ---- + if os.environ.get("SBOM_INCLUDE_LICENSES", "y") == "y": + license_map = resolve_licenses(target_path) + if license_map: + resolved, noassertion = apply_licenses(all_components, license_map) + total = len(all_components) + pct = (100.0 * resolved / total) if total else 0.0 + info(f"License resolution: {resolved}/{total} " + f"resolved ({pct:.1f}%); {noassertion} NOASSERTION") + else: + info("License resolution: no copyrights tarballs found") + + # ---- Build the final BOM ---- + sbom: dict[str, Any] = { + "bomFormat": "CycloneDX", + "specVersion": "1.6", + "version": 1, + "metadata": { + "timestamp": now_iso(), + "tools": [{ + "vendor": "SONiC", + "name": "build_sbom.py", + "version": "1.0", + }], + "component": { + "type": "operating-system", + "bom-ref": f"sonic-{target_machine}", + "name": f"sonic-{target_machine}", + "version": os.environ.get("SONIC_IMAGE_VERSION", "0.0.0"), + }, + "properties": [ + {"name": "sonic:platform", "value": platform}, + {"name": "sonic:arch", "value": arch}, + {"name": "sonic:target_machine", "value": target_machine}, + {"name": "sonic:scan_tool", "value": scan_tool}, + {"name": "sonic:version_control_components", "value": vcc}, + {"name": "sonic:recipe_fragments_loaded", + "value": str(len(fragments.all))}, + {"name": "sonic:installer_dockers", + "value": " ".join(installer_dockers)}, + ], + }, + "components": sorted( + all_components, key=lambda c: c.get("bom-ref", "") + ), + } + + # Build the dependencies[] graph: kernel modules -> kernel image. + deps = build_dependency_graph(all_components) + if deps: + sbom["dependencies"] = sorted(deps, key=lambda d: d.get("ref", "")) + info(f"Dependencies: {len(deps)} kernel-module edges") + + try: + with open(out_path, "w") as f: + json.dump(sbom, f, indent=2, sort_keys=True) + info(f"SBOM written: {out_path}") + info(f"Component count: {len(all_components)}") + except Exception as e: + warn(f"could not write {out_path}: {e}") + return 0 + + # ---- SPDX export ---- + sbom_format = os.environ.get("SBOM_FORMAT", "cyclonedx").lower() + if sbom_format in ("spdx", "both"): + emit_spdx(out_path, target_path, artifact_basename) + + # ---- SLSA / in-toto provenance ---- + emit_provenance(out_path, target_path, artifact_basename) + + return 0 + + +def emit_provenance(cdx_path: str, target_path: str, + artifact_basename: str) -> None: + """Invoke scripts/sbom_emit_provenance.py to produce a sibling + .intoto.json document. Failure is logged but does not break the + build. artifact_basename is the actual installer filename + (.bin / .swi / .img.gz / etc.).""" + artifact_path = os.path.join(target_path, artifact_basename) + if not os.path.isfile(artifact_path): + info(f"no installer artifact at {artifact_path}; " + f"skipping provenance emit") + return + script = os.path.join(os.path.dirname(__file__), + "sbom_emit_provenance.py") + rc, _, err = run( + ["python3", script, "--bin", artifact_path, "--sbom", cdx_path], + timeout=120, + ) + if rc != 0: + warn(f"provenance emit failed (rc={rc}): {err.strip()[:200]}") + return + info(f"Provenance written: {artifact_path}.intoto.json") + + +def emit_spdx(cdx_path: str, target_path: str, + artifact_basename: str) -> None: + """Convert the CycloneDX SBOM to SPDX via cyclonedx-cli.""" + script = os.path.join(os.path.dirname(__file__), "install_sbom_tool.sh") + rc, out, err = run([script, "cyclonedx-cli"], timeout=300) + if rc != 0: + warn(f"could not install cyclonedx-cli (rc={rc}): {err.strip()[:200]}") + return + binary = out.strip() + if not binary or not os.path.isfile(binary): + warn(f"cyclonedx-cli binary not found at {binary!r}") + return + + spdx_path = os.path.join(target_path, f"{artifact_basename}.spdx.json") + rc, _, err = run( + [binary, "convert", + "--input-file", cdx_path, + "--output-file", spdx_path, + "--output-format", "spdxjson"], + timeout=300, + ) + if rc != 0: + warn(f"cyclonedx-cli convert failed (rc={rc}): {err.strip()[:200]}") + return + info(f"SPDX written: {spdx_path}") + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/scripts/build_sbom.sh b/scripts/build_sbom.sh new file mode 100755 index 00000000000..09b44fedb54 --- /dev/null +++ b/scripts/build_sbom.sh @@ -0,0 +1,22 @@ +#!/bin/bash +# +# build_sbom.sh — thin shim that delegates to the Python aggregator. +# +# Kept as a shell entry point so slave.mk has a stable command name; all +# SBOM-aggregation logic lives in scripts/build_sbom.py. Honours +# ENABLE_SBOM via the Python script's self-gate. + +set -e +SCRIPT_DIR="$(dirname "$0")" + +# Refresh auto-extracted OpenVEX statements before the SBOM is written. +# The extractor scans src/*/patches/ for CVE markers in patch headers +# and emits OpenVEX JSON under vex/auto/ — output is deterministic and +# vex/auto/ is gitignored, so this regenerates from source on every +# build instead of trusting potentially-stale committed JSON. +if [ "${ENABLE_SBOM:-n}" = "y" ]; then + python3 "${SCRIPT_DIR}/sbom_extract_vex_from_patches.py" \ + --output vex/auto 2>&1 || true +fi + +exec python3 "${SCRIPT_DIR}/build_sbom.py" "$@" diff --git a/scripts/install_sbom_tool.sh b/scripts/install_sbom_tool.sh new file mode 100755 index 00000000000..41eaec6a7ae --- /dev/null +++ b/scripts/install_sbom_tool.sh @@ -0,0 +1,237 @@ +#!/bin/bash +# +# install_sbom_tool.sh — fetch the binary scanner used by SBOM generation. +# +# Runs *inside the slave container* (or another build context where the +# src/sonic-build-hooks/hooks/wget shim is active) so the fetched URL + +# MD5 is recorded into target/versions/.../versions-web automatically. +# +# Resolution order (per tool): +# 1. If $TOOL is already on PATH, print its path and exit. +# 2. If the cached binary is present under $SBOM_TOOLS_DIR, use it. +# 3. Else wget the upstream release, verify SHA-256 against the pin in +# this script, extract into the cache, and print the path. +# +# Usage: +# scripts/install_sbom_tool.sh syft # prints absolute path to the binary +# scripts/install_sbom_tool.sh trivy +# scripts/install_sbom_tool.sh grype +# +# Pins are intentionally hard-coded here so a single edit moves the +# tool version and the existing build-hook infrastructure picks up the +# new URL on next build. + +set -euo pipefail + +TOOL="${1:-}" +if [ -z "$TOOL" ]; then + echo "usage: $0 " >&2 + exit 2 +fi + +# Honour TARGET_PATH from slave.mk if set; otherwise default to ./target. +TARGET_PATH="${TARGET_PATH:-target}" +SBOM_TOOLS_DIR="${SBOM_TOOLS_DIR:-${TARGET_PATH}/sbom-tools}" + +# Map our $(dpkg --print-architecture) names to the names releases use. +host_arch="$(dpkg --print-architecture 2>/dev/null || uname -m)" +case "$host_arch" in + amd64|x86_64) rel_arch="amd64" ;; + arm64|aarch64) rel_arch="arm64" ;; + *) + echo "[install_sbom_tool.sh] WARNING: unsupported host arch '$host_arch'." >&2 + echo "[install_sbom_tool.sh] The binary scanner is not available; SBOM" >&2 + echo "[install_sbom_tool.sh] generation will fall back to recipe-driven" >&2 + echo "[install_sbom_tool.sh] data only on this build." >&2 + exit 3 + ;; +esac + +############################################################################## +# Pinned versions and SHA-256s. +# +# To bump a tool version: update the version, paste the SHA-256 lines from +# the upstream __checksums.txt. The wget call below is shimmed, +# so the URL+MD5 gets captured into versions-web on next build and the +# pin propagates through scripts/versions_manager.py. +############################################################################## + +SYFT_VERSION="1.44.0" +SYFT_URL="https://github.com/anchore/syft/releases/download/v${SYFT_VERSION}/syft_${SYFT_VERSION}_linux_${rel_arch}.tar.gz" +SYFT_SHA256_amd64="0e91737aee2b5baf1d255b959630194a302335d848ff97bb07921eb6205b5f5a" +SYFT_SHA256_arm64="6f6cdcdc695721d91ce756e3b5bc3e3416599c464101f5e32e9c3f33054ee6d9" + +# CycloneDX-CLI (CycloneDX/cyclonedx-cli on GitHub). Single static +# binary (.NET self-contained) — no tarball, fetched directly. +# Used by build_sbom.py for CycloneDX -> SPDX conversion when +# SBOM_FORMAT includes 'spdx' or 'both'. +CYCLONEDX_CLI_VERSION="0.32.0" +case "$rel_arch" in + amd64) CYCLONEDX_CLI_URL="https://github.com/CycloneDX/cyclonedx-cli/releases/download/v${CYCLONEDX_CLI_VERSION}/cyclonedx-linux-x64" ;; + arm64) CYCLONEDX_CLI_URL="https://github.com/CycloneDX/cyclonedx-cli/releases/download/v${CYCLONEDX_CLI_VERSION}/cyclonedx-linux-arm64" ;; +esac +CYCLONEDX_CLI_SHA256_amd64="454879e6a4a405c8a13bff49b8982adcb0596f3019b26b0811c66e4d7f0783e1" +CYCLONEDX_CLI_SHA256_arm64="abf0b7c5648a5b127791d691cad41f004aceea27c75bb42c9572fdc9694770cf" + +GRYPE_VERSION="0.112.0" +GRYPE_URL="https://github.com/anchore/grype/releases/download/v${GRYPE_VERSION}/grype_${GRYPE_VERSION}_linux_${rel_arch}.tar.gz" +GRYPE_SHA256_amd64="acb14a030010fe9bdb9594b4ae108d9d14ef2f926d936aa0916dc62c89c058ea" +GRYPE_SHA256_arm64="7fdeccf065965cc59386c656e5fcc1eb1bdf820e2433000bca7f010b8e6da155" + +# Trivy is a placeholder — fill in when SBOM_SCAN_TOOL=trivy or +# vulnerability scanning via trivy is exercised. For now the script +# errors out cleanly if it's requested. +TRIVY_VERSION="" + +############################################################################## +# Common helpers. +############################################################################## + +cache_path() { + local tool="$1" version="$2" + echo "${SBOM_TOOLS_DIR}/${tool}-${version}/${tool}" +} + +verify_sha256() { + local file="$1" expected="$2" + local actual + actual="$(sha256sum "$file" | awk '{print $1}')" + if [ "$actual" != "$expected" ]; then + echo "[install_sbom_tool.sh] SHA-256 mismatch for $file:" >&2 + echo " expected: $expected" >&2 + echo " actual: $actual" >&2 + return 1 + fi +} + +fetch_and_extract() { + local tool="$1" version="$2" url="$3" sha256="$4" + local dest_dir="${SBOM_TOOLS_DIR}/${tool}-${version}" + local tarball="${dest_dir}.tar.gz" + + mkdir -p "$dest_dir" + + # Use wget so the build-hook shim records URL+MD5 into versions-web. + # --no-verbose keeps build logs clean; --tries handles transient network. + wget --no-verbose --tries=3 -O "$tarball" "$url" + verify_sha256 "$tarball" "$sha256" + + tar -C "$dest_dir" -xzf "$tarball" "$tool" + rm -f "$tarball" + + chmod +x "${dest_dir}/${tool}" +} + +# Fetch a single-binary tool (no tarball wrapper). cyclonedx-cli ships +# as a raw .NET self-contained executable, not a .tar.gz. +fetch_single_binary() { + local tool="$1" version="$2" url="$3" sha256="$4" + local dest_dir="${SBOM_TOOLS_DIR}/${tool}-${version}" + local dest="${dest_dir}/${tool}" + + mkdir -p "$dest_dir" + wget --no-verbose --tries=3 -O "$dest" "$url" + verify_sha256 "$dest" "$sha256" + chmod +x "$dest" +} + +############################################################################## +# Per-tool resolution. +############################################################################## + +resolve_syft() { + if command -v syft >/dev/null 2>&1; then + command -v syft + return 0 + fi + + local cached + cached="$(cache_path syft "$SYFT_VERSION")" + if [ -x "$cached" ]; then + echo "$cached" + return 0 + fi + + local sha256_var="SYFT_SHA256_${rel_arch}" + local sha256="${!sha256_var:-}" + if [ -z "$sha256" ]; then + echo "[install_sbom_tool.sh] no SHA-256 pin for syft on $rel_arch" >&2 + return 4 + fi + + fetch_and_extract syft "$SYFT_VERSION" "$SYFT_URL" "$sha256" >&2 + echo "$cached" +} + +resolve_trivy() { + echo "[install_sbom_tool.sh] trivy installer not yet implemented." >&2 + echo "[install_sbom_tool.sh] Use SBOM_SCAN_TOOL=syft (the default)." >&2 + return 5 +} + +resolve_grype() { + if command -v grype >/dev/null 2>&1; then + command -v grype + return 0 + fi + + local cached + cached="$(cache_path grype "$GRYPE_VERSION")" + if [ -x "$cached" ]; then + echo "$cached" + return 0 + fi + + local sha256_var="GRYPE_SHA256_${rel_arch}" + local sha256="${!sha256_var:-}" + if [ -z "$sha256" ]; then + echo "[install_sbom_tool.sh] no SHA-256 pin for grype on $rel_arch" >&2 + return 4 + fi + + fetch_and_extract grype "$GRYPE_VERSION" "$GRYPE_URL" "$sha256" >&2 + echo "$cached" +} + +resolve_cyclonedx_cli() { + # cyclonedx-cli ships under the name "cyclonedx" on PATH. + if command -v cyclonedx >/dev/null 2>&1; then + command -v cyclonedx + return 0 + fi + + local cached + cached="$(cache_path cyclonedx "$CYCLONEDX_CLI_VERSION")" + if [ -x "$cached" ]; then + echo "$cached" + return 0 + fi + + if [ -z "${CYCLONEDX_CLI_URL:-}" ]; then + echo "[install_sbom_tool.sh] no cyclonedx-cli release for arch $rel_arch" >&2 + return 4 + fi + + local sha256_var="CYCLONEDX_CLI_SHA256_${rel_arch}" + local sha256="${!sha256_var:-}" + if [ -z "$sha256" ]; then + echo "[install_sbom_tool.sh] no SHA-256 pin for cyclonedx-cli on $rel_arch" >&2 + return 4 + fi + + fetch_single_binary cyclonedx "$CYCLONEDX_CLI_VERSION" \ + "$CYCLONEDX_CLI_URL" "$sha256" >&2 + echo "$cached" +} + +case "$TOOL" in + syft) resolve_syft ;; + trivy) resolve_trivy ;; + grype) resolve_grype ;; + cyclonedx-cli|cyclonedx) + resolve_cyclonedx_cli ;; + *) + echo "[install_sbom_tool.sh] unknown tool: $TOOL" >&2 + exit 2 + ;; +esac diff --git a/scripts/prepare_docker_buildinfo.sh b/scripts/prepare_docker_buildinfo.sh index 5a4e3237cd6..50dbbf0344a 100755 --- a/scripts/prepare_docker_buildinfo.sh +++ b/scripts/prepare_docker_buildinfo.sh @@ -55,11 +55,13 @@ scripts/docker_version_control.sh $@ DOCKERFILE_PRE_SCRIPT='# Auto-Generated for buildinfo ARG SONIC_VERSION_CACHE ARG SONIC_VERSION_CONTROL_COMPONENTS +ARG ENABLE_SBOM=n COPY ["buildinfo", "/usr/local/share/buildinfo"] COPY vcache/ /sonic/target/vcache/'${IMAGENAME}' RUN dpkg -i /usr/local/share/buildinfo/sonic-build-hooks_1.0_all.deb ENV IMAGENAME='${IMAGENAME}' ENV DISTRO='${DISTRO}' +ENV ENABLE_SBOM=$ENABLE_SBOM RUN pre_run_buildinfo '${IMAGENAME}' ' @@ -95,6 +97,15 @@ fi mkdir -p ${BUILDINFO_PATH} cp -rf src/sonic-build-hooks/buildinfo/* $BUILDINFO_PATH +# Stage the shared cargo-auditable wrapper inside the buildinfo +# directory so each sonic-slave-* Dockerfile can COPY it from a +# single source of truth (files/build/cargo-wrapper) — only needs +# to apply to slave-base images where Rust is installed. +if [[ "$IMAGENAME" == sonic-slave-* ]] && [ -f files/build/cargo-wrapper ]; then + cp files/build/cargo-wrapper $BUILDINFO_PATH/cargo-wrapper + chmod 0755 $BUILDINFO_PATH/cargo-wrapper +fi + # Generate the version lock files scripts/versions_manager.py generate -t "$BUILDINFO_VERSION_PATH" -n "$IMAGENAME" -d "$DISTRO" -a "$ARCH" diff --git a/scripts/sbom_diff.py b/scripts/sbom_diff.py new file mode 100755 index 00000000000..8837ddb019c --- /dev/null +++ b/scripts/sbom_diff.py @@ -0,0 +1,177 @@ +#!/usr/bin/env python3 +""" +sbom_diff.py — compare two SBOM files (CycloneDX JSON) and report +component-level differences. Used as a reproducibility cross-check +between independent build hosts of the same source tree, and as a +release-drift report between two SBOMs over time. + +Usage: + sbom_diff.py [--quiet] [--json out.json] OLD.cdx.json NEW.cdx.json + +Default output is human-readable. With --json, emit a machine-readable +report covering added/removed/changed components and an exit code +that's non-zero when any difference exists (useful for CI gating). +""" + +import argparse +import json +import sys + + +def load_components(path: str) -> dict: + """Return {bom-ref or purl or (name, version, arch): component}.""" + with open(path) as f: + doc = json.load(f) + out: dict = {} + for c in doc.get("components", []) or []: + key = c.get("bom-ref") or c.get("purl") or c.get("name", "?") + out[key] = c + return out + + +def component_signature(c: dict) -> dict: + """The fields we compare for "did this component change?". + + We deliberately ignore timestamps, scan-tool internal metadata, and + the SBOM aggregator's own properties so a 'no real change' rebuild + doesn't flag drift. Things that matter: + - version + - hashes + - licenses + - pedigree.ancestors[].version + - pedigree.patches.length and per-patch hashes + """ + sig: dict = { + "name": c.get("name"), + "version": c.get("version"), + "type": c.get("type"), + } + hashes = c.get("hashes") or [] + if hashes: + sig["hashes"] = sorted( + (h.get("alg"), h.get("content")) for h in hashes + ) + licenses = c.get("licenses") or [] + if licenses: + lic_strs = [] + for l in licenses: + if "expression" in l: + lic_strs.append(("expr", l["expression"])) + elif "license" in l: + lic = l["license"] + lic_strs.append(( + "id" if "id" in lic else "name", + lic.get("id") or lic.get("name") or "", + )) + sig["licenses"] = sorted(lic_strs) + ped = c.get("pedigree") or {} + if ped.get("ancestors"): + sig["ancestors"] = sorted( + (a.get("name"), a.get("version")) for a in ped["ancestors"] + ) + if ped.get("patches"): + sig["patches"] = sorted( + tuple( + h.get("content", "") + for h in (p.get("diff") or {}).get("hashes") or [] + ) + for p in ped["patches"] + ) + return sig + + +def diff(old: dict, new: dict) -> dict: + added = sorted(k for k in new if k not in old) + removed = sorted(k for k in old if k not in new) + changed = [] + for k in sorted(old.keys() & new.keys()): + s_old = component_signature(old[k]) + s_new = component_signature(new[k]) + if s_old != s_new: + field_diff = {} + for f in set(s_old) | set(s_new): + if s_old.get(f) != s_new.get(f): + field_diff[f] = { + "old": s_old.get(f), + "new": s_new.get(f), + } + changed.append({"key": k, "changes": field_diff}) + return { + "added": added, + "removed": removed, + "changed": changed, + } + + +def print_report(d: dict, old_path: str, new_path: str) -> None: + print(f"Comparing:") + print(f" old: {old_path}") + print(f" new: {new_path}") + print() + print(f"Added: {len(d['added'])}") + print(f"Removed: {len(d['removed'])}") + print(f"Changed: {len(d['changed'])}") + if d["added"]: + print() + print("--- Added ---") + for k in d["added"][:50]: + print(f" + {k}") + if len(d["added"]) > 50: + print(f" ... and {len(d['added']) - 50} more") + if d["removed"]: + print() + print("--- Removed ---") + for k in d["removed"][:50]: + print(f" - {k}") + if len(d["removed"]) > 50: + print(f" ... and {len(d['removed']) - 50} more") + if d["changed"]: + print() + print("--- Changed ---") + for entry in d["changed"][:30]: + print(f" ~ {entry['key']}") + for field, vals in sorted(entry["changes"].items()): + old_v = vals.get("old") + new_v = vals.get("new") + if isinstance(old_v, list): + old_v = f"[{len(old_v)} items]" + if isinstance(new_v, list): + new_v = f"[{len(new_v)} items]" + print(f" {field}: {old_v} -> {new_v}") + if len(d["changed"]) > 30: + print(f" ... and {len(d['changed']) - 30} more") + + +def main() -> int: + ap = argparse.ArgumentParser(description=__doc__) + ap.add_argument("old", help="Older / baseline SBOM (CycloneDX JSON)") + ap.add_argument("new", help="Newer / candidate SBOM (CycloneDX JSON)") + ap.add_argument("--quiet", action="store_true", + help="Suppress per-component output; just report counts") + ap.add_argument("--json", + help="Write the full diff as JSON to this file") + args = ap.parse_args() + + old = load_components(args.old) + new = load_components(args.new) + d = diff(old, new) + + if args.json: + with open(args.json, "w") as f: + json.dump(d, f, indent=2, sort_keys=True) + + if args.quiet: + print( + f"+{len(d['added'])} -{len(d['removed'])} ~{len(d['changed'])}" + ) + else: + print_report(d, args.old, args.new) + + # Non-zero exit when any difference; useful for `make sbom-verify`. + if d["added"] or d["removed"] or d["changed"]: + return 1 + return 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/scripts/sbom_emit_provenance.py b/scripts/sbom_emit_provenance.py new file mode 100755 index 00000000000..dd341383a17 --- /dev/null +++ b/scripts/sbom_emit_provenance.py @@ -0,0 +1,357 @@ +#!/usr/bin/env python3 +""" +sbom_emit_provenance.py — emit an unsigned SLSA v1.0 provenance +attestation alongside the SBOM. + +Output: `target/.intoto.json` — sibling of whatever +installer this run produced (.bin / .swi / .img.gz / etc.). + +The attestation follows in-toto Statement v1 +(https://in-toto.io/Statement/v1) wrapping a SLSA Provenance v1 +predicate (https://slsa.dev/provenance/v1) and answers the question: +"how was this binary built?" + +It is intentionally **unsigned**. Release engineering signs both the +SBOM and the .intoto.json at publish time with whatever tool their +policy requires (cosign, openssl, etc.). The build slave never sees +production signing keys. See README.sbom.md "Attestation and signing" +for the recommended workflow. + +The output is reproducible: two byte-identical builds produce +byte-identical .intoto.json documents, as long as SOURCE_DATE_EPOCH +is set or no timestamps are populated. +""" + +import argparse +import datetime +import hashlib +import json +import os +import re +import subprocess +import sys +from typing import Optional + + +def warn(msg: str) -> None: + sys.stderr.write(f"[sbom_emit_provenance.py] WARNING: {msg}\n") + + +def info(msg: str) -> None: + if os.environ.get("SBOM_DEBUG", "n") == "y": + sys.stderr.write(f"[sbom_emit_provenance.py] {msg}\n") + + +# --------------------------------------------------------------------------- +# Helpers +# --------------------------------------------------------------------------- + + +def run(cmd: list, cwd: Optional[str] = None, timeout: int = 60) -> Optional[str]: + try: + r = subprocess.run( + cmd, cwd=cwd, capture_output=True, text=True, + timeout=timeout, check=False, + ) + if r.returncode == 0: + return r.stdout.strip() + except Exception: + pass + return None + + +def file_sha256(path: str) -> Optional[str]: + try: + h = hashlib.sha256() + with open(path, "rb") as f: + for chunk in iter(lambda: f.read(65536), b""): + h.update(chunk) + return h.hexdigest() + except Exception: + return None + + +def reproducible_timestamp() -> Optional[str]: + """Return SOURCE_DATE_EPOCH as ISO8601 if set; else None. + + None means "this attestation will not be byte-identical across + independent builds because we don't have a reproducible build + timestamp". We omit timestamps entirely in that case. + """ + epoch = os.environ.get("SOURCE_DATE_EPOCH") + if not epoch: + return None + try: + return datetime.datetime.fromtimestamp( + int(epoch), tz=datetime.timezone.utc + ).strftime("%Y-%m-%dT%H:%M:%SZ") + except Exception: + return None + + +# --------------------------------------------------------------------------- +# Slave image identity (the "builder" in SLSA terms) +# --------------------------------------------------------------------------- + + +def slave_image_digest(image: str) -> Optional[dict]: + """Return {repo, tag, sha256_digest} for a locally-loaded image. + + Falls back to None if docker is unavailable or the image isn't + loaded (which is normal when this script runs in a fresh slave + after collect_docker_version_files has tagged things differently). + """ + out = run(["docker", "image", "inspect", "--format", + "{{.Id}}\t{{.RepoDigests}}", image]) + if not out: + return None + fields = out.split("\t", 1) + iid = fields[0] if fields else "" + # iid is like "sha256:abc..."; strip the prefix to match SLSA digest map. + digest = iid.split(":", 1)[1] if iid.startswith("sha256:") else iid + return { + "name": image, + "digest": {"sha256": digest} if digest else {}, + } + + +def find_slave_images() -> list: + """Locate the slave docker images used in this build. + + Sources: target/versions/build/build-sonic-slave-*/ directory names + point at the slave kind (bookworm / trixie). We use them to derive + the image refs and ask docker for their digests. + """ + target_path = os.environ.get("TARGET_PATH", "target") + build_dir = os.path.join(target_path, "versions", "build") + out: list = [] + if not os.path.isdir(build_dir): + return out + for name in sorted(os.listdir(build_dir)): + if not name.startswith("build-sonic-slave-"): + continue + slave_kind = name[len("build-"):] # e.g. "sonic-slave-bookworm" + # Find the loaded image by querying docker for any tag matching. + # If multiple, prefer the longest tag (per-user variant). + list_out = run(["docker", "images", + "--format", "{{.Repository}}:{{.Tag}}\t{{.ID}}", + slave_kind]) + if not list_out: + out.append({"name": slave_kind, "digest": {}}) + continue + # Pick the tag whose repo == slave_kind (skip -bhouse user variants + # if both exist, by sorting and taking the bare repo first). + candidates = [] + for line in list_out.splitlines(): + repo_tag, _, image_id = line.partition("\t") + repo = repo_tag.partition(":")[0] + if repo == slave_kind and image_id: + candidates.append((repo_tag, image_id)) + if not candidates: + out.append({"name": slave_kind, "digest": {}}) + continue + repo_tag, image_id = candidates[0] + digest = image_id.split(":", 1)[1] if image_id.startswith("sha256:") else image_id + out.append({ + "name": f"pkg:docker/{repo_tag}", + "digest": {"sha256": digest} if digest else {}, + }) + return out + + +# --------------------------------------------------------------------------- +# Repo identity +# --------------------------------------------------------------------------- + + +def git_commit() -> Optional[str]: + return run(["git", "rev-parse", "HEAD"]) + + +def git_remote_url() -> Optional[str]: + url = run(["git", "config", "--get", "remote.origin.url"]) + if not url: + return None + # Normalize SSH form to https URI for SLSA. + m = re.match(r"git@([^:]+):(.+?)(?:\.git)?$", url) + if m: + return f"https://{m.group(1)}/{m.group(2)}" + return url.rstrip("/").removesuffix(".git") + + +# --------------------------------------------------------------------------- +# SBOM linkage +# --------------------------------------------------------------------------- + + +def load_sbom_metadata(cdx_path: str) -> dict: + """Pull bom-ref and serialNumber out of the SBOM for cross-reference.""" + try: + with open(cdx_path) as f: + doc = json.load(f) + except Exception as e: + warn(f"could not read {cdx_path}: {e}") + return {} + meta = doc.get("metadata", {}) + return { + "bom_ref": meta.get("component", {}).get("bom-ref"), + "serial_number": doc.get("serialNumber"), + "spec_version": doc.get("specVersion"), + "version": doc.get("version"), + } + + +# --------------------------------------------------------------------------- +# Top-level provenance construction +# --------------------------------------------------------------------------- + + +_EXTERNAL_PARAM_VARS = [ + "CONFIGURED_PLATFORM", + "CONFIGURED_ARCH", + "TARGET_BOOTLOADER", + "BLDENV", + "ENABLE_SBOM", + "SBOM_FORMAT", + "SBOM_SCAN_TOOL", + "SBOM_INCLUDE_LICENSES", + "INCLUDE_KUBERNETES", + "INCLUDE_PTF", + "INCLUDE_PDE", + "INCLUDE_MGMT_FRAMEWORK", + "INCLUDE_DHCP_RELAY", + "INCLUDE_DHCP_SERVER", + "INCLUDE_MACSEC", + "INCLUDE_NAT", + "INCLUDE_SFLOW", + "INCLUDE_FIPS", + "INCLUDE_EXTERNAL_PATCHES", + "SONIC_VERSION_CONTROL_COMPONENTS", + "BUILD_PUBLIC_URL", + "BUILD_SNAPSHOT_URL", + "MIRROR_URLS", + "MIRROR_SECURITY_URLS", + "MIRROR_SNAPSHOT", +] + + +def build_provenance( + bin_path: str, cdx_path: Optional[str], +) -> dict: + bin_digest = file_sha256(bin_path) + subject = [{ + "name": os.path.basename(bin_path), + "digest": {"sha256": bin_digest} if bin_digest else {}, + }] + + external_params = { + k: os.environ.get(k, "") + for k in _EXTERNAL_PARAM_VARS + if os.environ.get(k) + } + # The target identity is itself a build input. + external_params["target"] = os.path.basename(bin_path) + + resolved_deps = [] + commit = git_commit() + remote = git_remote_url() + if remote and commit: + resolved_deps.append({ + "uri": f"git+{remote}", + "digest": {"gitCommit": commit}, + }) + # Cross-reference the SBOM as a resolved dependency. + if cdx_path and os.path.isfile(cdx_path): + cdx_sha = file_sha256(cdx_path) + sbom_meta = load_sbom_metadata(cdx_path) + ref = { + "uri": "file://" + os.path.relpath(cdx_path), + "digest": {"sha256": cdx_sha} if cdx_sha else {}, + } + if sbom_meta.get("serial_number"): + ref["name"] = sbom_meta["serial_number"] + resolved_deps.append(ref) + + # Slave docker images = builder identity. + slaves = find_slave_images() + builder: dict = { + "id": "https://github.com/sonic-net/sonic-buildimage/blob/master/Makefile", + } + if slaves: + builder["builderDependencies"] = slaves + + metadata: dict = { + "invocationId": os.environ.get( + "BUILD_NUMBER", + os.environ.get("BUILD_TIMESTAMP", "local-build"), + ), + } + ts = reproducible_timestamp() + if ts: + # SLSA spec uses startedOn and finishedOn; we set both to the + # same epoch-derived value for reproducibility. + metadata["startedOn"] = ts + metadata["finishedOn"] = ts + + predicate = { + "buildDefinition": { + "buildType": "https://github.com/sonic-net/sonic-buildimage/build/v1", + "externalParameters": external_params, + "internalParameters": { + "host_arch": run(["dpkg", "--print-architecture"]) or "", + }, + "resolvedDependencies": resolved_deps, + }, + "runDetails": { + "builder": builder, + "metadata": metadata, + }, + } + + statement = { + "_type": "https://in-toto.io/Statement/v1", + "subject": subject, + "predicateType": "https://slsa.dev/provenance/v1", + "predicate": predicate, + } + return statement + + +# --------------------------------------------------------------------------- +# Entry point +# --------------------------------------------------------------------------- + + +def main() -> int: + ap = argparse.ArgumentParser(description=__doc__) + ap.add_argument("--bin", required=True, + help="Path to the installer image (e.g. target/sonic-broadcom.bin)") + ap.add_argument("--sbom", + help="Path to the sibling CycloneDX SBOM " + "(default: .cdx.json)") + ap.add_argument("--output", + help="Where to write the .intoto.json " + "(default: .intoto.json)") + args = ap.parse_args() + + if not os.path.isfile(args.bin): + warn(f"bin not found: {args.bin}") + return 0 + + cdx_path = args.sbom or (args.bin + ".cdx.json") + out_path = args.output or (args.bin + ".intoto.json") + + statement = build_provenance(args.bin, cdx_path=cdx_path) + + try: + with open(out_path, "w") as f: + json.dump(statement, f, indent=2, sort_keys=True) + info(f"wrote {out_path}") + except Exception as e: + warn(f"could not write {out_path}: {e}") + return 0 + return 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/scripts/sbom_extract_vex_from_patches.py b/scripts/sbom_extract_vex_from_patches.py new file mode 100755 index 00000000000..d47a13af79d --- /dev/null +++ b/scripts/sbom_extract_vex_from_patches.py @@ -0,0 +1,284 @@ +#!/usr/bin/env python3 +""" +sbom_extract_vex_from_patches.py — sweep the SONiC source tree for +patches that mention CVEs in their filename or header, and emit one +OpenVEX YAML file per (CVE, source-component) pair. + +The goal: when a SONiC patch fixes an upstream CVE, that CVE should +**not** appear as an active finding in the vulnerability report for +the corresponding component, even though the underlying upstream +version in the SBOM ancestor pedigree is still vulnerable. OpenVEX +status `not_affected` with justification `vulnerable_code_not_in_execute_path` +encodes that "the patch fixed it; don't bother me about it". + +Where it looks: + + src/*/patch/*.patch + src/*/patches/*.patch + src/*/patches-sonic/*.patch (sonic-linux-kernel kernel patches) + src/*.patch/*.patch (sidecar patch directories) + src/*/debian/patches/*.patch (Debian-style nested) + +What it looks for, in order of confidence: + + Filename: contains 'cve-NNNN-NNNNN' (case-insensitive) + Header: 'Fixes: CVE-NNNN-NNNNN' + Header: 'Subject: ... CVE-NNNN-NNNNN ...' + Anywhere: 'CVE-NNNN-NNNNN' (loose match, lower confidence) + +Output: + + vex/auto//.json + +OpenVEX JSON (not YAML — grype's --vex flag rejects YAML). One file +per patch, capturing every CVE it mentions. The product list is +derived from the source-tree directory name (i.e. `thrift` for +src/thrift/patch/*). Concrete PURLs are not emitted because the +extractor does not know which downstream debs/wheels each patch +ultimately ships in — grype's product matching handles this via PURL +substring matching. + +Re-run is idempotent — files are overwritten with consistent ordering. + +Usage: + sbom_extract_vex_from_patches.py + sbom_extract_vex_from_patches.py --output-dir vex/auto --dry-run +""" + +import argparse +import datetime +import hashlib +import json +import os +import re +import sys + + +_CVE_RE = re.compile(r"CVE-(\d{4})-(\d{4,7})", re.IGNORECASE) +_FIXES_RE = re.compile(r"^\s*Fixes:\s*(CVE-\d{4}-\d{4,7})", + re.IGNORECASE | re.MULTILINE) +_SUBJECT_RE = re.compile(r"^Subject:.*?(CVE-\d{4}-\d{4,7})", + re.IGNORECASE | re.MULTILINE) + + +def warn(msg: str) -> None: + sys.stderr.write(f"[sbom_extract_vex_from_patches.py] WARN: {msg}\n") + + +def info(msg: str) -> None: + sys.stderr.write(f"[sbom_extract_vex_from_patches.py] {msg}\n") + + +# --------------------------------------------------------------------------- +# Patch discovery +# --------------------------------------------------------------------------- + + +def find_patches(root: str = "src") -> list: + """Walk the src/ tree (and the few well-known patch dirs that live + elsewhere) returning every *.patch file.""" + found = [] + for r, dirs, files in os.walk(root): + # Skip build artifacts and large unrelated trees. + skip = {"build", ".git", "node_modules", "target", "deb_dist"} + dirs[:] = [d for d in dirs if d not in skip] + if "/patch" in r or "/patches" in r or r.endswith(".patch") \ + or r.endswith("/debian"): + pass + for fn in files: + if fn.endswith(".patch"): + found.append(os.path.join(r, fn)) + return sorted(found) + + +# --------------------------------------------------------------------------- +# CVE extraction +# --------------------------------------------------------------------------- + + +def cves_in(path: str) -> tuple: + """Returns (high_confidence_cves, low_confidence_cves). + + High-confidence: the patch declares it via filename or Fixes:/Subject: + header. + Low-confidence: CVE mentioned somewhere in the patch body — could be + a passing reference, not actually being fixed. + """ + high: set = set() + low: set = set() + fname = os.path.basename(path) + + for m in _CVE_RE.finditer(fname): + high.add(f"CVE-{m.group(1)}-{m.group(2)}") + + try: + with open(path, "rb") as f: + data = f.read() + except Exception as e: + warn(f"could not read {path}: {e}") + return set(), set() + + # The header is up to the first 'diff --git' / '---' / '+++' boundary. + text = data.decode("utf-8", errors="replace") + header_end = re.search(r"^(?:diff --git|---|\+\+\+) ", text, re.M) + header = text[: header_end.start()] if header_end else text[:4000] + + for m in _FIXES_RE.finditer(header): + high.add(m.group(1).upper()) + for m in _SUBJECT_RE.finditer(header): + high.add(m.group(1).upper()) + for m in _CVE_RE.finditer(header): + cve = f"CVE-{m.group(1)}-{m.group(2)}".upper() + if cve not in high: + low.add(cve) + + return high, low + + +# --------------------------------------------------------------------------- +# Source-component derivation +# --------------------------------------------------------------------------- + + +_SRC_COMPONENT_RE = re.compile(r"^src/([^/]+?)(?:\.patch)?/") + + +def source_component(path: str) -> str: + """Derive a short label for the source tree this patch lives in. + 'src/thrift/patch/0002-cve-2017-1000487.patch' -> 'thrift' + 'src/sonic-linux-kernel/patches-sonic/...' -> 'sonic-linux-kernel' + 'src/scapy.patch/0001-...' -> 'scapy' + """ + m = _SRC_COMPONENT_RE.match(path) + if not m: + return "unknown" + return m.group(1) + + +# --------------------------------------------------------------------------- +# OpenVEX document construction +# --------------------------------------------------------------------------- + + +def now_iso() -> str: + epoch = os.environ.get("SOURCE_DATE_EPOCH") + if epoch: + try: + return datetime.datetime.fromtimestamp( + int(epoch), tz=datetime.timezone.utc + ).strftime("%Y-%m-%dT%H:%M:%SZ") + except Exception: + pass + return datetime.datetime.now(datetime.timezone.utc).strftime( + "%Y-%m-%dT%H:%M:%SZ" + ) + + +def make_openvex_json( + patch_path: str, component: str, + high_cves: set, low_cves: set, +) -> str: + """Return the OpenVEX JSON body for a single auto-VEX file. + + Re-runs are byte-identical (apart from timestamp, which is + SOURCE_DATE_EPOCH when set). + """ + h = hashlib.sha256((patch_path + "\n").encode()).hexdigest()[:16] + vex_id = f"https://github.com/sonic-net/sonic-buildimage/vex/auto/{h}" + + cves_sorted = sorted(high_cves) + sorted(low_cves - high_cves) + statements = [] + for cve in cves_sorted: + is_high = cve in high_cves + stmt = { + "vulnerability": {"name": cve}, + "products": [{"@id": f"pkg:generic/{component}"}], + } + if is_high: + stmt["status"] = "not_affected" + stmt["justification"] = "vulnerable_code_not_in_execute_path" + stmt["impact_statement"] = ( + f"Fixed by SONiC local patch {patch_path}. " + "Promote to a curated vex/ file with an exact product " + "PURL if grype's PURL matcher doesn't catch this " + "component automatically." + ) + else: + stmt["status"] = "under_investigation" + stmt["impact_statement"] = ( + f"Patch {patch_path} mentions {cve} in passing but is " + "not declared as a fix. Manual triage required to " + "determine whether this VEX statement applies." + ) + statements.append(stmt) + + doc = { + "@context": "https://openvex.dev/ns/v0.2.0", + "@id": vex_id, + "author": "SONiC auto-VEX", + "role": "SONiC build tooling", + "timestamp": now_iso(), + "version": 1, + "_comment": ( + "Auto-extracted from " + patch_path + ". " + "DO NOT EDIT — regenerate via " + "scripts/sbom_extract_vex_from_patches.py" + ), + "statements": statements, + } + return json.dumps(doc, indent=2, sort_keys=True) + "\n" + + +# --------------------------------------------------------------------------- +# Top-level +# --------------------------------------------------------------------------- + + +def main() -> int: + ap = argparse.ArgumentParser(description=__doc__) + ap.add_argument("--src", default="src", + help="Source root to scan (default: src)") + ap.add_argument("--output-dir", default="vex/auto", + help="Where to write auto-VEX yamls (default: vex/auto)") + ap.add_argument("--dry-run", action="store_true", + help="Don't write files, just report counts") + args = ap.parse_args() + + if not os.path.isdir(args.src): + warn(f"source dir not found: {args.src}") + return 1 + + patches = find_patches(args.src) + info(f"scanned {len(patches)} patch files under {args.src}/") + + written = 0 + cve_total = 0 + for p in patches: + high, low = cves_in(p) + if not high and not low: + continue + component = source_component(p) + body = make_openvex_json(p, component, high, low) + if args.dry_run: + cve_total += len(high) + len(low) + written += 1 + info(f"would write VEX for {len(high)+len(low)} CVE(s) in {p}") + continue + out_dir = os.path.join(args.output_dir, component) + os.makedirs(out_dir, exist_ok=True) + out_path = os.path.join(out_dir, os.path.basename(p) + ".json") + try: + with open(out_path, "w") as f: + f.write(body) + written += 1 + cve_total += len(high) + len(low) + except Exception as e: + warn(f"could not write {out_path}: {e}") + + info(f"wrote {written} VEX file(s) covering {cve_total} CVE statement(s)" + + (" (dry-run)" if args.dry_run else "")) + return 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/scripts/sbom_fragment.py b/scripts/sbom_fragment.py new file mode 100755 index 00000000000..9acfd9439e4 --- /dev/null +++ b/scripts/sbom_fragment.py @@ -0,0 +1,1115 @@ +#!/usr/bin/env python3 +""" +sbom_fragment.py — emit a CycloneDX 1.6 fragment for a single build artifact. + +Invoked from slave.mk recipes after each artifact is built. Reads context +from environment variables: + + ARTIFACT Required. Path to the just-built .deb / .whl / .gz. + RECIPE_TYPE Required. One of: + DPKG_DEB, MAKE_DEB, DERIVED_DEB, EXTRA_DEB, + ONLINE_DEB, PYTHON_STDEB, PYTHON_WHEEL, DOCKER_IMAGE. + SRC_PATH Optional. Value of $($*_SRC_PATH). Used for submodule + + patch detection. + URL Optional. Value of $($*_URL) for ONLINE_DEB recipes. + DEPENDS Optional. Build dependencies. + RDEPENDS Optional. Runtime dependencies. + MAIN_DEB Optional. For DERIVED_DEB / EXTRA_DEB: filename of the + parent .deb that produced this artifact. + +Output: .cdx.json next to the artifact. + +What this script emits per fragment: + - Recipe-driven identity: PURL, name, version, arch from the + artifact filename and recipe context. + - Source provenance: git submodule URL + commit SHA when SRC_PATH + is a submodule. + - Patch enumeration: SHA-256 per patch file when a sibling + .patch/, /patch/, or /patches/ is + found. + - Aggregate patch-set SHA-1 over (series + *.patch), matching the + scheme used internally by src/sonic-frr/Makefile. + - Upstream ancestor (pedigree.ancestors[]) when detectable: + - dget upstream resolved from versions-web for apt-source recipes + - nested-submodule ancestor for sonic-net wrappers around upstream + - direct submodule ancestor for non-sonic-net submodules + - Vendor supplier identity for SONIC_ONLINE_DEBS, derived from URL + pattern, plus a default EULA license marker. ARTIFACT_LICENSE + env-var override wins when set. + +License resolution itself (DEP-5 parsing, SPDX translation) is +performed later by the aggregator (build_sbom.py) against +copyrights.tar.gz tarballs harvested by the build hook; it is not +done here. + +The script is fail-soft: any unexpected condition logs to stderr and +exits 0. Build failure must not be caused by SBOM emit issues. +""" + +import hashlib +import json +import os +import re +import shutil +import subprocess +import sys +import tempfile +import time +from typing import Any, Optional + + +def warn(msg: str) -> None: + sys.stderr.write(f"[sbom_fragment.py] WARNING: {msg}\n") + + +def info(msg: str) -> None: + # Silent by default: per-artifact emission would log ~120 lines per build. + # Set SBOM_DEBUG=y in the env to see normal progress messages. + if os.environ.get("SBOM_DEBUG", "n") == "y": + sys.stderr.write(f"[sbom_fragment.py] {msg}\n") + + +# ---------------------------------------------------------------------------- +# Filename parsers +# ---------------------------------------------------------------------------- + +_DEB_RE = re.compile(r"^(?P[^_]+)_(?P[^_]+)_(?P[^.]+)\.deb$") +_WHL_RE = re.compile( + r"^(?P[^-]+)-(?P[^-]+)-" + r"(?P[^-]+)-(?P[^-]+)-(?P[^.]+)\.whl$" +) +_GZ_RE = re.compile(r"^(?P.+?)(?P-dbg)?\.gz$") + + +def parse_artifact_name(path: str) -> dict: + """Best-effort decompose an artifact filename.""" + fn = os.path.basename(path) + if fn.endswith(".deb"): + m = _DEB_RE.match(fn) + if m: + return { + "kind": "deb", + "name": m.group("name"), + "version": m.group("version"), + "arch": m.group("arch"), + "filename": fn, + } + if fn.endswith(".whl"): + m = _WHL_RE.match(fn) + if m: + return { + "kind": "wheel", + "name": m.group("name").replace("_", "-").lower(), + "version": m.group("version"), + "arch": "any", + "py_tag": m.group("pytag"), + "abi_tag": m.group("abitag"), + "platform_tag": m.group("plat"), + "filename": fn, + } + if fn.endswith(".gz"): + m = _GZ_RE.match(fn) + if m: + return { + "kind": "docker", + "name": m.group("name"), + "version": os.environ.get("SONIC_IMAGE_VERSION", "0.0.0"), + "arch": os.environ.get("CONFIGURED_ARCH", "amd64"), + "filename": fn, + "debug": bool(m.group("dbg")), + } + return { + "kind": "unknown", + "name": fn, + "version": "0.0.0", + "arch": os.environ.get("CONFIGURED_ARCH", "amd64"), + "filename": fn, + } + + +# ---------------------------------------------------------------------------- +# Submodule detection (best-effort, fail-soft) +# ---------------------------------------------------------------------------- + + +def run(cmd: list, cwd: Optional[str] = None) -> Optional[str]: + """Run a command and return stdout or None on any error.""" + try: + r = subprocess.run( + cmd, cwd=cwd, capture_output=True, text=True, timeout=30, check=False + ) + if r.returncode == 0: + return r.stdout.strip() + except Exception: + pass + return None + + +def _find_gitmodules_for(abs_src: str) -> Optional[tuple]: + """Walk parents looking for a .gitmodules; return (gm_path, rel_path).""" + cur = os.path.dirname(abs_src) + while cur and cur != "/": + gm = os.path.join(cur, ".gitmodules") + if os.path.isfile(gm): + return gm, os.path.relpath(abs_src, cur) + cur = os.path.dirname(cur) + return None + + +def _lookup_submodule_url(gm: str, rel: str) -> Optional[str]: + """Find the submodule..url whose path = rel inside the .gitmodules file.""" + url = run(["git", "config", "-f", gm, "--get", + f"submodule.{rel}.url"]) + if url: + return url + # .gitmodules keys by section name (often != path); fall back to grep. + try: + with open(gm) as f: + text = f.read() + blocks = re.split(r"^\[submodule ", text, flags=re.M) + for b in blocks[1:]: + if re.search(rf"^\s*path\s*=\s*{re.escape(rel)}\s*$", b, re.M): + m = re.search(r"^\s*url\s*=\s*(\S+)\s*$", b, re.M) + if m: + return m.group(1) + except Exception: + pass + return None + + +def detect_submodule(src_path: str) -> Optional[dict]: + """If src_path is a git submodule, return {url, commit, describe}.""" + if not src_path or not os.path.isdir(src_path): + return None + abs_src = os.path.abspath(src_path) + found = _find_gitmodules_for(abs_src) + if not found: + return None + gm, rel = found + url = _lookup_submodule_url(gm, rel) + if not url: + return None + commit = run(["git", "rev-parse", "HEAD"], cwd=abs_src) + describe = run(["git", "describe", "--tags", "--always"], cwd=abs_src) + return { + "url": url.rstrip(".git").rstrip("/"), + "commit": commit, + "describe": describe, + } + + +def detect_nested_submodules(src_path: str) -> list: + """Find submodule paths that live *inside* src_path. + + Used to detect the FRR pattern (src/sonic-frr → src/sonic-frr/frr → + FRRouting/frr) where the outer dir is a sonic-net wrapper holding + patches + Makefile and the inner dir is upstream. + """ + if not src_path or not os.path.isdir(src_path): + return [] + abs_src = os.path.abspath(src_path) + found = _find_gitmodules_for(abs_src) + if not found: + return [] + gm, src_rel = found + out = [] + try: + with open(gm) as f: + text = f.read() + for block in re.split(r"^\[submodule ", text, flags=re.M)[1:]: + m = re.search(r"^\s*path\s*=\s*(\S+)\s*$", block, re.M) + if not m: + continue + path = m.group(1) + # Is this path strictly under src_rel? + if path == src_rel or not path.startswith(src_rel.rstrip("/") + "/"): + continue + u = re.search(r"^\s*url\s*=\s*(\S+)\s*$", block, re.M) + if not u: + continue + url = u.group(1).rstrip(".git").rstrip("/") + cur = os.path.dirname(gm) + sub_abs = os.path.join(cur, path) + commit = run(["git", "rev-parse", "HEAD"], cwd=sub_abs) + describe = run(["git", "describe", "--tags", "--always"], + cwd=sub_abs) + out.append({ + "path": path, + "url": url, + "commit": commit, + "describe": describe, + }) + except Exception as e: + warn(f"nested-submodule scan failed in {src_path}: {e}") + return out + + +# ---------------------------------------------------------------------------- +# Upstream Debian source lookup (Pattern A2) +# ---------------------------------------------------------------------------- + +# Two sources of versions-web data: +# 1. files/build/versions/default/versions-web - committed pins. +# Carries the "blessed" upstream URLs for things that have been +# version-promoted (dget'd Debian sources are the main case here). +# 2. target/versions/.../versions-web - written by the wget/curl shim +# during the current build. Carries everything the build actually +# fetched, including things not yet promoted (e.g. the kernel +# source descriptor, which the kernel Makefile wgets directly). +# We scan both so the lookup is correct on first-build before any +# promotion has happened. +_VERSIONS_WEB_SOURCES = [ + os.path.join("files", "build", "versions", "default", "versions-web"), +] + +_DSC_URL_RE = re.compile( + r"^(?Phttps?://[^=]+?/(?P[^/]+)/(?P=pkg)_(?P[^/]+?)\.dsc)==(?P[0-9a-f]+)$" +) +_ORIG_URL_RE = re.compile( + r"^(?Phttps?://[^=]+?/(?P[^/]+)/(?P=pkg)_(?P[^.]+?)\.orig\.tar\.[^=]+)==(?P[0-9a-f]+)$" +) + + +def _collect_versions_web_files() -> list: + """Return the committed pin file plus any build-time versions-web + captures under target/versions/.""" + files = [] + for p in _VERSIONS_WEB_SOURCES: + if os.path.isfile(p): + files.append(p) + target = os.environ.get("TARGET_PATH", "target") + if os.path.isdir(target): + for root, _, fns in os.walk(os.path.join(target, "versions")): + for fn in fns: + if fn == "versions-web": + files.append(os.path.join(root, fn)) + return files + + +def lookup_upstream_debian_source(src_path: str) -> Optional[dict]: + """For Pattern A2 (dget/wget-based recipes), find the upstream Debian + source descriptor in versions-web. Returns + {name, version, dsc_url, dsc_md5, orig_url, orig_md5} + or None. + """ + if not src_path: + return None + src_basename = os.path.basename(src_path.rstrip("/")) + if not src_basename: + return None + # Special case: sonic-linux-kernel builds Debian's `linux` source. + lookup_keys = [src_basename] + if src_basename == "sonic-linux-kernel": + lookup_keys.append("linux") + dsc_match = None + orig_match = None + for path in _collect_versions_web_files(): + try: + with open(path) as f: + for line in f: + line = line.strip() + if not line: + continue + m = _DSC_URL_RE.match(line) + if m and m.group("pkg") in lookup_keys: + dsc_match = m + continue + m = _ORIG_URL_RE.match(line) + if m and m.group("pkg") in lookup_keys: + orig_match = m + except Exception as e: + warn(f"versions-web scan of {path} failed: {e}") + continue + if dsc_match: + break + if not dsc_match: + return None + info = { + "name": dsc_match.group("pkg"), + "version": dsc_match.group("ver"), + "dsc_url": dsc_match.group("url"), + "dsc_md5": dsc_match.group("md5"), + } + if orig_match: + info["orig_url"] = orig_match.group("url") + info["orig_md5"] = orig_match.group("md5") + return info + + +# ---------------------------------------------------------------------------- +# Patch series detection +# ---------------------------------------------------------------------------- + + +def find_patch_dir(src_path: str) -> Optional[str]: + """Locate a patch series directory adjacent to or under src_path.""" + if not src_path: + return None + candidates = [ + src_path + ".patch", # src/scapy.patch, src/sonic-swss.patch, ... + os.path.join(src_path, "patch"), # src/openssh/patch, src/sonic-frr/patch + os.path.join(src_path, "patches"), # src/bash/patches, src/sonic-mgmt-common/patches + ] + # Linux kernel uses a uniquely-named subdir. + if os.path.basename(src_path.rstrip("/")) == "sonic-linux-kernel": + candidates.insert(0, os.path.join(src_path, "patches-sonic")) + for c in candidates: + if os.path.isfile(os.path.join(c, "series")): + return c + return None + + +def enumerate_patches(patch_dir: str) -> list: + """Read series file, hash each patch.""" + out = [] + series = os.path.join(patch_dir, "series") + if not os.path.isfile(series): + return out + try: + with open(series) as f: + for line in f: + line = line.strip() + if not line or line.startswith("#"): + continue + # Series lines can carry options after the filename. + fname = line.split()[0] + pf = os.path.join(patch_dir, fname) + if not os.path.isfile(pf): + continue + h = hashlib.sha256() + with open(pf, "rb") as pfh: + h.update(pfh.read()) + out.append({ + "name": fname, + "path": os.path.relpath(pf), + "sha256": h.hexdigest(), + }) + except Exception as e: + warn(f"failed to read patch series in {patch_dir}: {e}") + return out + + +def patchset_hash(patch_dir: str) -> Optional[str]: + """Aggregate SHA-256 over series + *.patch contents (FRR-style).""" + series = os.path.join(patch_dir, "series") + if not os.path.isfile(series): + return None + h = hashlib.sha256() + try: + with open(series, "rb") as f: + h.update(f.read()) + # Hash the patch files in series order. + with open(series) as f: + for line in f: + line = line.strip() + if not line or line.startswith("#"): + continue + fname = line.split()[0] + pf = os.path.join(patch_dir, fname) + if os.path.isfile(pf): + with open(pf, "rb") as pfh: + h.update(pfh.read()) + return h.hexdigest() + except Exception: + return None + + +# ---------------------------------------------------------------------------- +# PURL construction +# ---------------------------------------------------------------------------- + + +def is_sonic_net_url(url: Optional[str]) -> bool: + if not url: + return False + return "github.com/sonic-net" in url or "github.com/Azure/sonic" in url + + +# Pattern C — vendor binary suppliers. Used to label SONIC_ONLINE_DEBS +# (Broadcom/Mellanox/Marvell/NVIDIA/Pensando/Arista SDKs) so the SBOM +# records who owns the license terms. The match is a substring against +# the lowercased URL; first hit wins. +_VENDOR_URL_PATTERNS = [ + ("broadcom", "Broadcom Inc."), + ("sai-broadcom", "Broadcom Inc."), + ("libsaibcm", "Broadcom Inc."), + ("mellanox", "NVIDIA / Mellanox"), + ("nvidia", "NVIDIA"), + ("bluefield", "NVIDIA / Mellanox"), + ("doca", "NVIDIA"), + ("marvell", "Marvell Technology"), + ("teralynx", "Marvell Technology"), + ("prestera", "Marvell Technology"), + ("pensando", "AMD / Pensando"), + ("aristanetworks", "Arista Networks"), + ("aristanetwork", "Arista Networks"), + ("phy-credo", "Credo"), + ("p4lang", "P4 Language Consortium"), + ("openconfig", "OpenConfig"), +] + + +def _vendor_supplier_from_url(url: Optional[str]) -> Optional[str]: + if not url: + return None + u = url.lower() + for pat, supplier in _VENDOR_URL_PATTERNS: + if pat in u: + return supplier + return None + + +def build_purl(meta: dict, submodule: Optional[dict]) -> str: + """Construct a PURL appropriate to the artifact + source provenance.""" + if meta["kind"] == "deb": + # Heuristic: when source is a sonic-net submodule, use the github + # PURL as primary identity. Otherwise it's an apt-style deb. + if submodule and is_sonic_net_url(submodule.get("url")): + repo = submodule["url"].rsplit("/", 1)[-1] + sha = (submodule.get("commit") or "")[:12] + return f"pkg:github/sonic-net/{repo}@{sha}" + # Default: deb with sonic namespace (locally rebuilt). + return f"pkg:deb/sonic/{meta['name']}@{meta['version']}?arch={meta['arch']}" + if meta["kind"] == "wheel": + return f"pkg:pypi/{meta['name']}@{meta['version']}" + if meta["kind"] == "docker": + return f"pkg:oci/{meta['name']}@{meta['version']}?arch={meta['arch']}" + return f"pkg:generic/{meta['name']}@{meta['version']}" + + +# ---------------------------------------------------------------------------- +# Ancestor (pedigree.ancestors[]) construction +# ---------------------------------------------------------------------------- + + +def _github_purl_for_url(url: str, ref: Optional[str]) -> str: + """Derive a pkg:github//@ PURL from a GitHub URL.""" + m = re.search(r"github\.com[:/]([^/]+)/([^/]+?)(?:\.git)?/?$", url) + if m: + return f"pkg:github/{m.group(1)}/{m.group(2)}@{ref or 'unknown'}" + # Generic fallback for non-GitHub URLs. + return f"pkg:generic/{url}@{ref or 'unknown'}" + + +def ancestor_from_submodule(sm: dict) -> dict: + """Build a pedigree.ancestor entry from a non-sonic-net submodule.""" + ref = sm.get("describe") or (sm.get("commit") or "")[:12] + purl = _github_purl_for_url(sm["url"], ref) + return { + "type": "library", + "name": sm["url"].rsplit("/", 1)[-1], + "version": ref, + "purl": purl, + "externalReferences": [{ + "type": "vcs", + "url": sm["url"], + "comment": f"submodule commit {sm.get('commit', '')}", + }], + } + + +def ancestor_from_debian_source(info: dict) -> dict: + """Build a pedigree.ancestor entry from a versions-web dget lookup.""" + ext_refs = [{ + "type": "distribution", + "url": info["dsc_url"], + "hashes": [{"alg": "MD5", "content": info["dsc_md5"]}], + }] + if info.get("orig_url"): + ext_refs.append({ + "type": "distribution", + "url": info["orig_url"], + "hashes": [{"alg": "MD5", "content": info["orig_md5"]}], + "comment": "original upstream tarball", + }) + return { + "type": "library", + "name": info["name"], + "version": info["version"], + "purl": f"pkg:deb/debian/{info['name']}@{info['version']}", + "externalReferences": ext_refs, + } + + +# ---------------------------------------------------------------------------- +# Fragment construction +# ---------------------------------------------------------------------------- + + +def component_type(kind: str) -> str: + if kind == "docker": + return "container" + if kind == "wheel": + return "library" + return "library" + + +def split_csv(value: str) -> list: + return [x.strip() for x in (value or "").split() if x.strip()] + + +# --------------------------------------------------------------------------- +# Per-.deb language dependency attribution +# +# For each SONiC-built .deb we extract once with `dpkg-deb -x` and run +# three harvesters against the unpacked tree: +# +# Rust cargo-auditable embeds the resolved Cargo dep graph as a +# zlib-compressed JSON blob in the `.dep-v0` ELF section. +# rust-audit-info reads it back out. +# Go `go build` embeds module info via runtime/debug.BuildInfo +# (the `.go.buildinfo` ELF section). `go version -m ` +# prints it as line-oriented text. +# Python Wheel-style installs land in `*.dist-info/` directories +# alongside the importable package. We walk for METADATA +# files and parse them as RFC 822. +# +# Each emitted component carries: +# sonic:fragment_kind = recipe-emit-{rust,go,python} +# sonic:source_deb = +# +# build_sbom.py:build_dependency_graph() reverses sonic:source_deb into +# a dependsOn edge from the .deb's bom-ref to each language-dep +# component, so consumers can walk swss_*.deb -> tokio@1.x or +# sonic-gnmi_*.deb -> github.com/openconfig/gnmi@v0.10 directly. +# +# Scope is .deb-level (not per-binary): if two binaries inside the same +# .deb both link tokio, we emit one tokio component. CVE-blast-radius +# triage operates at the artifact level for our use case. +# --------------------------------------------------------------------------- + +# Per-binary tool-execution timeout. rust-audit-info and `go version -m` +# are fast (tens of ms each) on well-formed inputs but could hang on +# corrupt or extremely large ELFs. Don't let one weird .deb wedge the +# whole build. +_LANG_TOOL_TIMEOUT_SECS = 30 + +# Sentinel env vars so the "tool not available" warnings only fire once +# per build, instead of once per .deb fragment. +_RUST_TOOL_LOGGED = "_SBOM_RUST_AUDIT_INFO_MISSING_LOGGED" +_GO_TOOL_LOGGED = "_SBOM_GO_TOOL_MISSING_LOGGED" + +_PYPI_NORM_RE = re.compile(r"[-_.]+") + + +def _is_elf(path: str) -> bool: + """Cheap 4-byte magic check for ELF files. Skips symlinks and + non-files (named pipes, sockets) silently.""" + try: + if not os.path.isfile(path) or os.path.islink(path): + return False + with open(path, "rb") as f: + return f.read(4) == b"\x7fELF" + except OSError: + return False + + +def _pypi_normalize(name: str) -> str: + """PEP 503 name normalization: lowercase, runs of -_. -> single -.""" + return _PYPI_NORM_RE.sub("-", name).lower() + + +def _rust_components_from_elf( + elf_path: str, audit_info_bin: str, deb_filename: str, +) -> list: + """Run rust-audit-info on one ELF; return CycloneDX components for + every crate it found. Silent skip on any failure — most binaries + are non-Rust or built without cargo-auditable. + """ + try: + out = subprocess.run( + [audit_info_bin, elf_path], + capture_output=True, text=True, check=False, + timeout=_LANG_TOOL_TIMEOUT_SECS, + ) + except subprocess.TimeoutExpired: + warn(f"rust-audit-info timed out on {elf_path}") + return [] + if out.returncode != 0 or not out.stdout.strip(): + return [] + try: + data = json.loads(out.stdout) + except json.JSONDecodeError: + return [] + pkgs = data.get("packages") or [] + components = [] + for pkg in pkgs: + name = pkg.get("name") + version = pkg.get("version") + if not (name and version): + continue + source = pkg.get("source") or "local" + purl = f"pkg:cargo/{name}@{version}" + comp = { + "bom-ref": purl, + "type": "library", + "name": name, + "version": version, + "purl": purl, + "properties": [ + {"name": "sonic:fragment_kind", "value": "recipe-emit-rust"}, + {"name": "sonic:source_deb", "value": deb_filename}, + {"name": "sonic:source", "value": source}, + ], + } + kind = pkg.get("kind") + if kind: + comp["properties"].append( + {"name": "sonic:cargo_kind", "value": str(kind)} + ) + components.append(comp) + return components + + +def _go_components_from_elf( + elf_path: str, go_bin: str, deb_filename: str, +) -> list: + """Run `go version -m ` and parse the embedded module graph. + + Format (each module line begins with a leading tab): + + : go1.21.5 + \\tpath\\t
+ \\tmod\\t
\\t(devel) + \\tdep\\t\\t\\t + \\t=>\\t\\t\\t (replacement of previous dep) + \\tbuild\\t... + + We only care about `dep` and `=>` lines. A `=>` immediately + following a `dep` line means the dep was replaced; we keep the + replacement (what actually shipped). + """ + try: + out = subprocess.run( + [go_bin, "version", "-m", elf_path], + capture_output=True, text=True, check=False, + timeout=_LANG_TOOL_TIMEOUT_SECS, + ) + except subprocess.TimeoutExpired: + warn(f"go version -m timed out on {elf_path}") + return [] + if out.returncode != 0 or not out.stdout: + return [] + # `go version -m` prints ": " for Go binaries; for + # non-Go ELFs it prints ": go: ..." or returns rc=1. Either + # way, no module lines follow. + deps: list = [] # ordered list of (name, version) + for line in out.stdout.splitlines(): + parts = line.split("\t") + # Module lines look like ['', '', '', '', ...] + if len(parts) < 4 or parts[0] != "": + continue + kind = parts[1] + name = parts[2] + version = parts[3] + if not (name and version): + continue + if kind == "dep": + deps.append((name, version)) + elif kind == "=>" and deps: + # Replacement directive — overwrite the immediately + # preceding dep so we record what actually shipped. + deps[-1] = (name, version) + components = [] + for name, version in deps: + purl = f"pkg:golang/{name}@{version}" + components.append({ + "bom-ref": purl, + "type": "library", + "name": name, + "version": version, + "purl": purl, + "properties": [ + {"name": "sonic:fragment_kind", "value": "recipe-emit-go"}, + {"name": "sonic:source_deb", "value": deb_filename}, + ], + }) + return components + + +def _python_components_from_dist_info( + tmpdir: str, deb_filename: str, +) -> list: + """Walk an extracted .deb tree for *.dist-info/METADATA files and + parse Name + Version out of each. Wheels installed inside a .deb + (either directly bundled, or staged via pip in the recipe) show up + here. Older .egg-info directories with PKG-INFO are not parsed; + dist-info is the modern format and what pip emits since 2020. + """ + components = [] + seen: set = set() # (normalized_name, version) + for root, _, _ in os.walk(tmpdir): + if not root.endswith(".dist-info"): + continue + meta_path = os.path.join(root, "METADATA") + if not os.path.isfile(meta_path): + continue + name = None + version = None + try: + with open(meta_path, errors="replace") as f: + for line in f: + # Headers stop at the first blank line (RFC 822). + if not line.strip(): + break + # Continuation lines start with whitespace; ignore + # them — Name and Version are always single-line. + if line[0] in (" ", "\t"): + continue + if line.startswith("Name:"): + name = line.split(":", 1)[1].strip() + elif line.startswith("Version:"): + version = line.split(":", 1)[1].strip() + if name and version: + break + except OSError: + continue + if not (name and version): + continue + norm = _pypi_normalize(name) + key = (norm, version) + if key in seen: + continue + seen.add(key) + purl = f"pkg:pypi/{norm}@{version}" + components.append({ + "bom-ref": purl, + "type": "library", + "name": name, + "version": version, + "purl": purl, + "properties": [ + {"name": "sonic:fragment_kind", "value": "recipe-emit-python"}, + {"name": "sonic:source_deb", "value": deb_filename}, + ], + }) + return components + + +def extract_lang_deps_from_deb(deb_path: str, deb_filename: str) -> list: + """One-shot .deb extraction + Rust + Go + Python harvesters. + + Returns a flat list of CycloneDX components. Each component is + deduped at .deb scope (one entry per (purl) per .deb regardless + of how many binaries it appeared in). + + Returns [] when: + - the .deb doesn't exist or extraction fails + - the .deb contains no Rust/Go/Python deps to attribute + """ + if not os.path.exists(deb_path): + return [] + + # Tool availability: warn once per build (via process-env sentinel) + # if a tool is missing, then skip its harvester. The other two + # languages still run. + audit_info = shutil.which("rust-audit-info") + if not audit_info and not os.environ.get(_RUST_TOOL_LOGGED): + warn( + "rust-audit-info not in PATH; .deb-introspected Rust " + "crate attribution disabled (this enrichment requires " + "rust-audit-info, installed in slave Dockerfiles)" + ) + os.environ[_RUST_TOOL_LOGGED] = "1" + go_bin = shutil.which("go") + if not go_bin and not os.environ.get(_GO_TOOL_LOGGED): + warn( + "go not in PATH; .deb-introspected Go module attribution " + "disabled (golang-go is normally installed in every slave " + "image alongside the C/C++ toolchain)" + ) + os.environ[_GO_TOOL_LOGGED] = "1" + + by_purl: dict = {} # purl -> component (dedup at .deb scope) + counts = {"rust": 0, "go": 0, "python": 0, "elfs": 0} + + try: + with tempfile.TemporaryDirectory(prefix="sbom-lang-deb-") as tmpdir: + extract = subprocess.run( + ["dpkg-deb", "-x", deb_path, tmpdir], + capture_output=True, text=True, check=False, + ) + if extract.returncode != 0: + warn( + f"dpkg-deb -x failed for {deb_path}: " + f"{(extract.stderr or '').strip()[:200]}" + ) + return [] + # Single ELF walk — both Rust + Go harvesters share it. + for root, _, files in os.walk(tmpdir): + for fn in files: + path = os.path.join(root, fn) + if not _is_elf(path): + continue + counts["elfs"] += 1 + if audit_info: + for comp in _rust_components_from_elf( + path, audit_info, deb_filename, + ): + if comp["purl"] not in by_purl: + by_purl[comp["purl"]] = comp + counts["rust"] += 1 + if go_bin: + for comp in _go_components_from_elf( + path, go_bin, deb_filename, + ): + if comp["purl"] not in by_purl: + by_purl[comp["purl"]] = comp + counts["go"] += 1 + # Python dist-info walk: independent of ELF walk; runs + # against the same tmpdir. + for comp in _python_components_from_dist_info( + tmpdir, deb_filename, + ): + if comp["purl"] not in by_purl: + by_purl[comp["purl"]] = comp + counts["python"] += 1 + except OSError as e: + warn(f"language-dep attribution failed for {deb_path}: {e}") + return [] + + if not by_purl: + return [] + + info( + f"lang-attribution: {deb_filename} -> " + f"{counts['rust']} crates, {counts['go']} go modules, " + f"{counts['python']} python dists ({counts['elfs']} ELFs scanned)" + ) + # Sort for stable output ordering across builds. + return [by_purl[p] for p in sorted(by_purl)] + + +# Backwards-compatibility alias for any caller still importing the +# Rust-specific name. Kept as a thin wrapper that filters to Rust +# components only. +def extract_rust_deps_from_deb(deb_path: str, deb_filename: str) -> list: + """Deprecated: use extract_lang_deps_from_deb() instead. + Returned for backward compatibility; filters to recipe-emit-rust. + """ + return [ + c for c in extract_lang_deps_from_deb(deb_path, deb_filename) + if any(p.get("name") == "sonic:fragment_kind" + and p.get("value") == "recipe-emit-rust" + for p in c.get("properties", []) or []) + ] + + +def build_fragment(artifact: str, recipe_type: str) -> dict: + meta = parse_artifact_name(artifact) + src_path = os.environ.get("SRC_PATH", "") + submodule = detect_submodule(src_path) if src_path else None + patch_dir = find_patch_dir(src_path) if src_path else None + patches = enumerate_patches(patch_dir) if patch_dir else [] + ps_hash = patchset_hash(patch_dir) if patch_dir else None + + bom_ref = build_purl(meta, submodule) + + component: dict[str, Any] = { + "bom-ref": bom_ref, + "type": component_type(meta["kind"]), + "name": meta["name"], + "version": meta["version"], + "purl": bom_ref, + "properties": [ + {"name": "sonic:fragment_kind", "value": "recipe-emit"}, + {"name": "sonic:recipe_type", "value": recipe_type}, + {"name": "sonic:artifact_filename", "value": meta["filename"]}, + ], + } + + if meta["kind"] == "deb": + component["properties"].append( + {"name": "sonic:arch", "value": meta["arch"]} + ) + + # Honor an explicit per-recipe license override + # ($(ARTIFACT)_LICENSE in the recipe → ARTIFACT_LICENSE env var here). + # Useful for SONiC-native debs/wheels that don't ship a debian/copyright + # — the DEP-5 resolver can't infer their license from /usr/share/doc. + artifact_license = os.environ.get("ARTIFACT_LICENSE", "").strip() + if artifact_license: + component["licenses"] = [{"expression": artifact_license}] + + if submodule: + component["externalReferences"] = [{ + "type": "vcs", + "url": submodule["url"], + "comment": f"submodule pinned to {submodule.get('commit', '')}", + }] + component["properties"].append( + {"name": "sonic:submodule_commit", + "value": submodule.get("commit", "")} + ) + + online_url = os.environ.get("URL", "") + if recipe_type == "ONLINE_DEB" and online_url: + component.setdefault("externalReferences", []).append({ + "type": "distribution", + "url": online_url, + }) + # Compute SHA-256 of the just-fetched artifact so we have an + # authoritative hash regardless of whether _SKIP_VERSION was set. + try: + h = hashlib.sha256() + with open(artifact, "rb") as f: + for chunk in iter(lambda: f.read(65536), b""): + h.update(chunk) + component["hashes"] = [{"alg": "SHA-256", "content": h.hexdigest()}] + except Exception as e: + warn(f"could not hash {artifact}: {e}") + + # Pattern C: derive vendor supplier from URL pattern; mark the + # license category as proprietary by default (overridable via + # ARTIFACT_LICENSE if a vendor publishes under a known SPDX + # license). + supplier = _vendor_supplier_from_url(online_url) + if supplier: + component["supplier"] = {"name": supplier} + component["properties"].append( + {"name": "sonic:supplier_source", "value": "vendor_url"} + ) + # Don't override an explicit ARTIFACT_LICENSE if one was set. + if not component.get("licenses"): + component["licenses"] = [ + {"license": {"name": f"{supplier} proprietary (EULA)"}} + ] + + # ---- Ancestor resolution (Pattern A2/A3/A4) ---- + ancestors: list = [] + + # Pattern A4: direct non-sonic-net submodule. The submodule IS the + # upstream. Emit it as ancestor so the component records both the + # SONiC-rebuilt identity (primary PURL) and the upstream source. + if submodule and not is_sonic_net_url(submodule.get("url")): + ancestors.append(ancestor_from_submodule(submodule)) + + # Pattern A3: sonic-net wrapper with a nested non-sonic-net + # submodule (FRR / sonic-pins / gnoi / wpa-supplicant pattern). + # The nested submodule is the true upstream. + if src_path: + for nested in detect_nested_submodules(src_path): + if not is_sonic_net_url(nested.get("url")): + ancestors.append(ancestor_from_submodule(nested)) + + # Pattern A2: dget-based recipe. Look up the upstream Debian source + # descriptor in versions-web. Only consider this if SRC_PATH is NOT + # itself a submodule (which would mean Pattern A1/A4). + if src_path and not submodule: + upstream = lookup_upstream_debian_source(src_path) + if upstream: + ancestors.append(ancestor_from_debian_source(upstream)) + + if patches or ancestors: + pedigree: dict[str, Any] = {} + if ancestors: + pedigree["ancestors"] = ancestors + if patches: + pedigree["patches"] = [ + { + "type": "unofficial", + "diff": { + "url": f"file://{p['path']}", + "hashes": [{"alg": "SHA-256", "content": p["sha256"]}], + }, + } + for p in patches + ] + if ps_hash: + pedigree["notes"] = f"patch-set sha256: {ps_hash}" + component["pedigree"] = pedigree + + main_deb = os.environ.get("MAIN_DEB", "") + if recipe_type in ("DERIVED_DEB", "EXTRA_DEB") and main_deb: + component["properties"].append( + {"name": "sonic:parent_artifact", "value": main_deb} + ) + + depends = split_csv(os.environ.get("DEPENDS", "")) + rdepends = split_csv(os.environ.get("RDEPENDS", "")) + if depends: + component["properties"].append( + {"name": "sonic:build_depends", "value": " ".join(depends)} + ) + if rdepends: + component["properties"].append( + {"name": "sonic:runtime_depends", "value": " ".join(rdepends)} + ) + + # Language-dep attribution: extract per-binary Rust crates / Go + # modules + per-dist-info Python packages from any .deb fragment. + # Non-.deb recipes return []. Timing instrumentation lets the + # build log show real per-deb cost (Phase E); gated to only log + # when something was found or when the operation took noticeably + # long, so we don't spam ~120 zero-line entries per build. + lang_components: list = [] + if meta["kind"] == "deb": + t0 = time.monotonic() + lang_components = extract_lang_deps_from_deb( + artifact, meta["filename"], + ) + dt = time.monotonic() - t0 + if lang_components or dt > 1.0: + info( + f"lang-deps timing: {meta['filename']} took {dt:.2f}s " + f"({len(lang_components)} components)" + ) + + fragment = { + "bomFormat": "CycloneDX", + "specVersion": "1.6", + "version": 1, + "metadata": { + "properties": [ + {"name": "sonic:fragment_kind", "value": "recipe-emit"}, + {"name": "sonic:recipe_type", "value": recipe_type}, + {"name": "sonic:src_path", "value": src_path}, + {"name": "sonic:platform", + "value": os.environ.get("CONFIGURED_PLATFORM", "")}, + {"name": "sonic:arch", + "value": os.environ.get("CONFIGURED_ARCH", "")}, + ], + }, + "components": [component, *lang_components], + } + return fragment + + +# ---------------------------------------------------------------------------- +# Entry point +# ---------------------------------------------------------------------------- + + +def main() -> int: + if os.environ.get("ENABLE_SBOM", "n") != "y": + return 0 + + artifact = os.environ.get("ARTIFACT", "") + recipe_type = os.environ.get("RECIPE_TYPE", "") + if not artifact or not recipe_type: + warn("ARTIFACT and RECIPE_TYPE env vars are required") + return 0 # Fail-soft + + if not os.path.exists(artifact): + warn(f"artifact does not exist: {artifact}") + return 0 + + try: + fragment = build_fragment(artifact, recipe_type) + except Exception as e: + warn(f"fragment construction failed for {artifact}: {e}") + return 0 + + out_path = f"{artifact}.cdx.json" + try: + with open(out_path, "w") as f: + json.dump(fragment, f, indent=2, sort_keys=True) + info(f"wrote {out_path}") + except Exception as e: + warn(f"could not write {out_path}: {e}") + + return 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/scripts/sbom_license_map.json b/scripts/sbom_license_map.json new file mode 100644 index 00000000000..fd80a007552 --- /dev/null +++ b/scripts/sbom_license_map.json @@ -0,0 +1,120 @@ +{ + "_comment": "Translation table from Debian copyright-format/1.0/ License: header strings to SPDX license identifiers. Keys are normalized (lowercase, whitespace-collapsed) before lookup. Used by scripts/sbom_resolve_licenses.py.", + + "Apache-2.0": "Apache-2.0", + "Apache-2": "Apache-2.0", + "Apache 2.0": "Apache-2.0", + "Apache License, Version 2.0": "Apache-2.0", + "Apache License 2.0": "Apache-2.0", + "Artistic-1.0": "Artistic-1.0", + "Artistic-2.0": "Artistic-2.0", + "Artistic": "Artistic-1.0", + + "BSD-2-clause": "BSD-2-Clause", + "BSD-2-Clause": "BSD-2-Clause", + "BSD 2-Clause": "BSD-2-Clause", + "BSD-3-clause": "BSD-3-Clause", + "BSD-3-Clause": "BSD-3-Clause", + "BSD-3-clause-Wilson": "BSD-3-Clause", + "BSD 3-Clause": "BSD-3-Clause", + "BSD-4-clause": "BSD-4-Clause", + "BSD-4-Clause": "BSD-4-Clause", + "BSD": "BSD-3-Clause", + + "CC0-1.0": "CC0-1.0", + "CC0": "CC0-1.0", + "CC-BY-3.0": "CC-BY-3.0", + "CC-BY-4.0": "CC-BY-4.0", + "CC-BY-SA-3.0": "CC-BY-SA-3.0", + "CC-BY-SA-4.0": "CC-BY-SA-4.0", + + "EPL-1.0": "EPL-1.0", + "EPL-2.0": "EPL-2.0", + "Expat": "MIT", + "MIT": "MIT", + "MIT/X11": "MIT", + "MIT-like": "MIT", + + "GPL-1": "GPL-1.0-only", + "GPL-1+": "GPL-1.0-or-later", + "GPL-1.0": "GPL-1.0-only", + "GPL-1.0+": "GPL-1.0-or-later", + "GPL-2": "GPL-2.0-only", + "GPL-2+": "GPL-2.0-or-later", + "GPL-2.0": "GPL-2.0-only", + "GPL-2.0+": "GPL-2.0-or-later", + "GPL-2.0-only": "GPL-2.0-only", + "GPL-2.0-or-later": "GPL-2.0-or-later", + "GPL-3": "GPL-3.0-only", + "GPL-3+": "GPL-3.0-or-later", + "GPL-3.0": "GPL-3.0-only", + "GPL-3.0+": "GPL-3.0-or-later", + "GPL-3.0-only": "GPL-3.0-only", + "GPL-3.0-or-later": "GPL-3.0-or-later", + "GPL": "GPL-2.0-or-later", + + "LGPL-2": "LGPL-2.0-only", + "LGPL-2+": "LGPL-2.0-or-later", + "LGPL-2.0": "LGPL-2.0-only", + "LGPL-2.0+": "LGPL-2.0-or-later", + "LGPL-2.1": "LGPL-2.1-only", + "LGPL-2.1+": "LGPL-2.1-or-later", + "LGPL-2.1-only": "LGPL-2.1-only", + "LGPL-2.1-or-later": "LGPL-2.1-or-later", + "LGPL-3": "LGPL-3.0-only", + "LGPL-3+": "LGPL-3.0-or-later", + "LGPL-3.0": "LGPL-3.0-only", + "LGPL-3.0+": "LGPL-3.0-or-later", + "LGPL": "LGPL-2.1-or-later", + + "AGPL-3": "AGPL-3.0-only", + "AGPL-3+": "AGPL-3.0-or-later", + "AGPL-3.0": "AGPL-3.0-only", + "AGPL-3.0+": "AGPL-3.0-or-later", + + "ISC": "ISC", + "MPL-1.0": "MPL-1.0", + "MPL-1.1": "MPL-1.1", + "MPL-2.0": "MPL-2.0", + "Mozilla Public License": "MPL-2.0", + "PSF": "PSF-2.0", + "PSF-2": "PSF-2.0", + "Python": "Python-2.0", + "Python-2.0": "Python-2.0", + + "Zlib": "Zlib", + "zlib/libpng": "Zlib", + "BZIP2": "bzip2-1.0.6", + "OpenSSL": "OpenSSL", + "openssl": "OpenSSL", + "Boost-1.0": "BSL-1.0", + "BSL-1.0": "BSL-1.0", + "WTFPL": "WTFPL", + "X11": "X11", + "ZPL-2.1": "ZPL-2.1", + "Unlicense": "Unlicense", + "Public Domain": "LicenseRef-PublicDomain", + "public-domain": "LicenseRef-PublicDomain", + "public domain": "LicenseRef-PublicDomain", + + "GFDL-1.1": "GFDL-1.1-only", + "GFDL-1.1+": "GFDL-1.1-or-later", + "GFDL-1.2": "GFDL-1.2-only", + "GFDL-1.2+": "GFDL-1.2-or-later", + "GFDL-1.3": "GFDL-1.3-only", + "GFDL-1.3+": "GFDL-1.3-or-later", + "GFDL-NIV": "GFDL-1.3-no-invariants-only", + + "FSFAP": "FSFAP", + "FSFUL": "FSFUL", + "FSFULLR": "FSFULLR", + "Autoconf": "Autoconf-exception-2.0", + + "perl": "Artistic-1.0-Perl OR GPL-1.0-or-later", + "Perl": "Artistic-1.0-Perl OR GPL-1.0-or-later", + + "TCL": "TCL", + "Vim": "Vim", + "Sleepycat": "Sleepycat", + "Ruby": "Ruby" +} diff --git a/scripts/sbom_parse_lockfiles.py b/scripts/sbom_parse_lockfiles.py new file mode 100755 index 00000000000..0423b61ab82 --- /dev/null +++ b/scripts/sbom_parse_lockfiles.py @@ -0,0 +1,478 @@ +#!/usr/bin/env python3 +""" +sbom_parse_lockfiles.py — extract transitive package inventories from +language-ecosystem lockfiles harvested by the build hooks. + +Inputs: + --lockfiles TARBALL A lockfiles.tar.gz emitted by per-container + collect_version_files. Repeatable. + --output FILE Write CycloneDX components[] (JSON array). + +The aggregator (scripts/build_sbom.py) consumes the JSON array and +merges the components into the final SBOM with recipe-emit-wins dedupe. + +Supported formats: + + Cargo.lock (Rust) TOML, [[package]] blocks with name/version/source/checksum + go.sum (Go) ' ' per line + package-lock.json (npm) + pnpm-lock.yaml (pnpm) + yarn.lock (yarn) + +Notes on accuracy: + + - Cargo.lock contains the resolved-set (every crate that could satisfy + the dep graph under any feature combination), which is a superset of + the crates actually compiled into the binary. SBOM/SCA convention is + to treat Cargo.lock as the authoritative inventory; precise per-binary + coverage requires `cargo-auditable build`. See README.sbom.md "Known + limitations". + - go.sum is also a superset (it includes modules considered during + resolution, including dropped versions). When syft can read a Go + binary's embedded BuildInfo, that's more precise — but parsing go.sum + ensures we have crate-level hashes that BuildInfo lacks. +""" + +import argparse +import base64 +import binascii +import json +import os +import re +import sys +import tarfile +from typing import Optional + + +def warn(msg: str) -> None: + sys.stderr.write(f"[sbom_parse_lockfiles.py] WARNING: {msg}\n") + + +# ---------------------------------------------------------------------------- +# Cargo.lock parser +# ---------------------------------------------------------------------------- + + +def parse_cargo_lock(text: str, scope: str) -> list: + """Returns CycloneDX component dicts for every [[package]] entry.""" + components = [] + # Split on '[[package]]' boundaries. Skip the leading non-package preamble. + blocks = re.split(r"^\[\[package\]\]\s*$", text, flags=re.M)[1:] + for block in blocks: + # Stop at the next top-level [...] section header. + block = re.split(r"^\[[^[]", block, flags=re.M)[0] + fields = {} + for line in block.splitlines(): + m = re.match(r'^\s*(\w+)\s*=\s*"?([^"]*?)"?\s*$', line) + if m: + fields[m.group(1)] = m.group(2) + name = fields.get("name") + version = fields.get("version") + if not name or not version: + continue + comp = { + "bom-ref": f"pkg:cargo/{name}@{version}", + "type": "library", + "name": name, + "version": version, + "purl": f"pkg:cargo/{name}@{version}", + "properties": [ + {"name": "sonic:fragment_kind", "value": "lockfile"}, + {"name": "sonic:lockfile_format", "value": "Cargo.lock"}, + {"name": "sonic:scope", "value": scope}, + ], + } + checksum = fields.get("checksum") + if checksum and re.fullmatch(r"[0-9a-f]{64}", checksum): + comp["hashes"] = [{"alg": "SHA-256", "content": checksum}] + source = fields.get("source") or "" + if source.startswith("registry+"): + comp["externalReferences"] = [{ + "type": "distribution", + "url": source[len("registry+"):], + }] + elif source.startswith("git+"): + comp["externalReferences"] = [{ + "type": "vcs", + "url": source[len("git+"):], + }] + components.append(comp) + return components + + +# ---------------------------------------------------------------------------- +# go.sum parser +# ---------------------------------------------------------------------------- + + +_GOSUM_LINE_RE = re.compile( + r"^(?P\S+)\s+(?P\S+?)(?P/go\.mod)?\s+(?Ph1:\S+)$" +) + + +def _h1_to_sha256_hex(h1: str) -> Optional[str]: + """Convert 'h1:base64(sha256)=' to lowercase-hex SHA-256.""" + if not h1.startswith("h1:"): + return None + try: + raw = base64.standard_b64decode(h1[3:]) + if len(raw) != 32: + return None + return binascii.hexlify(raw).decode("ascii") + except Exception: + return None + + +def parse_go_sum(text: str, scope: str) -> list: + """Returns CycloneDX components for each Go module in go.sum. + + go.sum has two lines per module: + h1: + /go.mod h1: + We use only the first kind (the module-content hash) for the SBOM, + since that's the closest analog to a 'distribution hash'. + """ + seen = set() + components = [] + for line in text.splitlines(): + line = line.strip() + if not line or line.startswith("#"): + continue + m = _GOSUM_LINE_RE.match(line) + if not m or m.group("gomod"): + continue + mod, ver = m.group("mod"), m.group("ver") + key = (mod, ver) + if key in seen: + continue + seen.add(key) + purl = f"pkg:golang/{mod}@{ver}" + comp = { + "bom-ref": purl, + "type": "library", + "name": mod, + "version": ver, + "purl": purl, + "properties": [ + {"name": "sonic:fragment_kind", "value": "lockfile"}, + {"name": "sonic:lockfile_format", "value": "go.sum"}, + {"name": "sonic:scope", "value": scope}, + ], + } + sha = _h1_to_sha256_hex(m.group("hash")) + if sha: + comp["hashes"] = [{"alg": "SHA-256", "content": sha}] + components.append(comp) + return components + + +# ---------------------------------------------------------------------------- +# package-lock.json (npm v2/v3 format) parser +# ---------------------------------------------------------------------------- + + +def parse_package_lock_json(text: str, scope: str) -> list: + """Parses npm v2/v3 package-lock.json. Older v1 format also supported.""" + components = [] + try: + doc = json.loads(text) + except Exception as e: + warn(f"could not parse package-lock.json: {e}") + return components + seen = set() + # v2/v3: 'packages' dict, keyed by relative path; root is "". + for path, info in (doc.get("packages") or {}).items(): + if not path or info.get("link"): + continue + name = info.get("name") or path.split("node_modules/")[-1] + version = info.get("version") + if not name or not version: + continue + key = (name, version) + if key in seen: + continue + seen.add(key) + purl = f"pkg:npm/{name}@{version}" + comp = { + "bom-ref": purl, + "type": "library", + "name": name, + "version": version, + "purl": purl, + "properties": [ + {"name": "sonic:fragment_kind", "value": "lockfile"}, + {"name": "sonic:lockfile_format", "value": "package-lock.json"}, + {"name": "sonic:scope", "value": scope}, + ], + } + integrity = info.get("integrity") or "" + # integrity is e.g. "sha512-...base64..." + if integrity.startswith(("sha256-", "sha384-", "sha512-")): + alg, _, b64 = integrity.partition("-") + try: + raw = base64.standard_b64decode(b64) + comp["hashes"] = [{ + "alg": alg.upper().replace("SHA", "SHA-"), + "content": binascii.hexlify(raw).decode("ascii"), + }] + except Exception: + pass + resolved = info.get("resolved") + if resolved: + comp["externalReferences"] = [{ + "type": "distribution", "url": resolved, + }] + components.append(comp) + # v1: top-level 'dependencies' map (recursive). + if not components and "dependencies" in doc: + def walk(deps, prefix=""): + for name, info in deps.items(): + ver = info.get("version") or "" + if (name, ver) in seen: + continue + seen.add((name, ver)) + purl = f"pkg:npm/{name}@{ver}" + components.append({ + "bom-ref": purl, + "type": "library", + "name": name, + "version": ver, + "purl": purl, + "properties": [ + {"name": "sonic:fragment_kind", "value": "lockfile"}, + {"name": "sonic:lockfile_format", + "value": "package-lock.json"}, + {"name": "sonic:scope", "value": scope}, + ], + }) + if "dependencies" in info: + walk(info["dependencies"], prefix + name + "/") + walk(doc.get("dependencies", {})) + return components + + +# ---------------------------------------------------------------------------- +# pnpm-lock.yaml (rudimentary parser — extracts /pkg@version keys) +# ---------------------------------------------------------------------------- + + +_PNPM_PKG_RE = re.compile(r"^\s+/([^@:\s]+(?:/[^@:\s]+)*)@([^:\s]+):\s*$") + + +def parse_pnpm_lock_yaml(text: str, scope: str) -> list: + """Best-effort pnpm-lock.yaml parser without yaml dependency. pnpm + lockfiles encode package identity as a key like '/foo@1.2.3:' under + the 'packages:' map. Hash data is omitted in this minimal parser — + if you need it, swap in a real YAML parser.""" + components = [] + seen = set() + in_packages = False + for line in text.splitlines(): + if not in_packages: + if line.startswith("packages:"): + in_packages = True + continue + if line and not line.startswith(" "): + break + m = _PNPM_PKG_RE.match(line) + if not m: + continue + name = m.group(1) + version = m.group(2) + if (name, version) in seen: + continue + seen.add((name, version)) + purl = f"pkg:npm/{name}@{version}" + components.append({ + "bom-ref": purl, + "type": "library", + "name": name, + "version": version, + "purl": purl, + "properties": [ + {"name": "sonic:fragment_kind", "value": "lockfile"}, + {"name": "sonic:lockfile_format", "value": "pnpm-lock.yaml"}, + {"name": "sonic:scope", "value": scope}, + ], + }) + return components + + +# ---------------------------------------------------------------------------- +# yarn.lock parser (classic v1 format) +# ---------------------------------------------------------------------------- + + +def parse_yarn_lock(text: str, scope: str) -> list: + """Parses yarn.lock v1 classic format. v2+ (berry) uses YAML which + we skip here — most projects still use v1.""" + components = [] + seen = set() + cur_name = None + cur_version = None + cur_integrity = None + cur_resolved = None + + def flush(): + nonlocal cur_name, cur_version, cur_integrity, cur_resolved + if cur_name and cur_version and (cur_name, cur_version) not in seen: + seen.add((cur_name, cur_version)) + purl = f"pkg:npm/{cur_name}@{cur_version}" + comp = { + "bom-ref": purl, + "type": "library", + "name": cur_name, + "version": cur_version, + "purl": purl, + "properties": [ + {"name": "sonic:fragment_kind", "value": "lockfile"}, + {"name": "sonic:lockfile_format", "value": "yarn.lock"}, + {"name": "sonic:scope", "value": scope}, + ], + } + if cur_integrity and cur_integrity.startswith( + ("sha256-", "sha384-", "sha512-")): + alg, _, b64 = cur_integrity.partition("-") + try: + raw = base64.standard_b64decode(b64) + comp["hashes"] = [{ + "alg": alg.upper().replace("SHA", "SHA-"), + "content": binascii.hexlify(raw).decode("ascii"), + }] + except Exception: + pass + if cur_resolved: + comp["externalReferences"] = [{ + "type": "distribution", "url": cur_resolved, + }] + components.append(comp) + cur_name = cur_version = cur_integrity = cur_resolved = None + + for raw in text.splitlines(): + line = raw.rstrip() + if not line or line.startswith("#"): + flush() + continue + if not line.startswith(" "): + # entry-header line, e.g. '"foo@^1.0", "foo@~1.0.1":' + flush() + m = re.search(r'^"?([^@\s"]+(?:/[^@\s"]+)?)@', line) + if m: + cur_name = m.group(1) + else: + s = line.strip() + m = re.match(r'version\s+"([^"]+)"', s) + if m: + cur_version = m.group(1) + m = re.match(r'integrity\s+([^\s]+)', s) + if m: + cur_integrity = m.group(1) + m = re.match(r'resolved\s+"([^"]+)"', s) + if m: + cur_resolved = m.group(1) + flush() + return components + + +# ---------------------------------------------------------------------------- +# Tarball walker +# ---------------------------------------------------------------------------- + + +_LOCKFILE_HANDLERS = { + "Cargo.lock": parse_cargo_lock, + "go.sum": parse_go_sum, + "package-lock.json": parse_package_lock_json, + "pnpm-lock.yaml": parse_pnpm_lock_yaml, + "yarn.lock": parse_yarn_lock, +} + + +def scope_from_tarball_path(path: str) -> str: + """Derive a human-readable scope label from the tarball path. + target/versions/dockers/docker-fpm-frr/post-versions/lockfiles.tar.gz + -> 'dockers/docker-fpm-frr' + target/versions/host-image/post-versions/lockfiles.tar.gz + -> 'host-image' + """ + m = re.search(r"versions/(.+?)/post-versions/", path) + return m.group(1) if m else path + + +def parse_tarball(tarball: str, components_out: list, stats: dict) -> None: + if not os.path.isfile(tarball): + return + scope = scope_from_tarball_path(tarball) + try: + tf = tarfile.open(tarball, "r:*") + except Exception as e: + warn(f"could not open {tarball}: {e}") + return + with tf: + for member in tf: + if not member.isfile(): + continue + basename = os.path.basename(member.name) + handler = _LOCKFILE_HANDLERS.get(basename) + if not handler: + continue + try: + f = tf.extractfile(member) + if f is None: + continue + text = f.read().decode("utf-8", errors="replace") + except Exception: + continue + try: + comps = handler(text, scope) + except Exception as e: + warn(f"parser {handler.__name__} failed on " + f"{member.name} in {tarball}: {e}") + continue + if comps: + components_out.extend(comps) + stats.setdefault(basename, 0) + stats[basename] += len(comps) + + +# ---------------------------------------------------------------------------- +# Entry point +# ---------------------------------------------------------------------------- + + +def main() -> int: + ap = argparse.ArgumentParser(description=__doc__) + ap.add_argument("--lockfiles", action="append", default=[], + help="Path to a lockfiles.tar.gz; repeatable.") + ap.add_argument("--output", required=True, + help="Where to write the components[] JSON array.") + args = ap.parse_args() + + all_components: list = [] + stats: dict = {} + for tarball in args.lockfiles: + parse_tarball(tarball, all_components, stats) + + try: + with open(args.output, "w") as f: + json.dump({ + "components": all_components, + "stats": stats, + }, f, indent=2) + except Exception as e: + sys.stderr.write( + f"[sbom_parse_lockfiles.py] could not write {args.output}: {e}\n" + ) + return 1 + + total = len(all_components) + bits = " ".join(f"{k}={v}" for k, v in sorted(stats.items())) + sys.stderr.write( + f"[sbom_parse_lockfiles.py] parsed {total} components " + f"({bits or 'no lockfiles found'})\n" + ) + return 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/scripts/sbom_resolve_licenses.py b/scripts/sbom_resolve_licenses.py new file mode 100755 index 00000000000..6f7aebdca98 --- /dev/null +++ b/scripts/sbom_resolve_licenses.py @@ -0,0 +1,316 @@ +#!/usr/bin/env python3 +""" +sbom_resolve_licenses.py — extract SPDX license expressions from +/usr/share/doc//copyright files captured during the build. + +Inputs: + --copyrights TARBALL A copyrights.tar.gz produced by either + build_debian.sh (host rootfs) or by the + per-container collect_version_files hook. + May be passed multiple times. + --output FILE Write a JSON map { pkg: spdx_expr } here. + --license-map FILE Override default scripts/sbom_license_map.json + --quiet Suppress per-package warnings. + +The tarball entries are expected to look like + usr/share/doc//copyright +We extract the package name from the directory and parse the file: + + 1. If it has 'Format: https://www.debian.org/doc/packaging-manuals/copyright-format/1.0/' + header, it is DEP-5. Parse the Files: stanzas, take the union of + License: identifiers, translate to SPDX via the license map. + + 2. Otherwise, attempt a heuristic pass via `licensecheck` (from the + devscripts Debian package) if available. licensecheck output is + name-or-NONE per file; we map names through the same table. + + 3. Otherwise emit 'NOASSERTION' and surface a warning so the operator + can decide whether to add a per-recipe LICENSE override. + +The resolver is read-only and stateless — runs once per build inside +the aggregator (scripts/build_sbom.py). +""" + +import argparse +import json +import os +import re +import shutil +import subprocess +import sys +import tarfile +from collections import OrderedDict +from typing import Optional + + +# --------------------------------------------------------------------------- +# License-map loading + lookup +# --------------------------------------------------------------------------- + +_SCRIPT_DIR = os.path.dirname(os.path.abspath(__file__)) +DEFAULT_LICENSE_MAP = os.path.join(_SCRIPT_DIR, "sbom_license_map.json") + + +def normalize_license_key(s: str) -> str: + """Collapse whitespace and trim. Case-sensitive lookup first, then + case-insensitive fallback is done by the caller via two map copies.""" + return re.sub(r"\s+", " ", s.strip()) + + +class LicenseMap: + def __init__(self, path: str): + with open(path) as f: + raw = json.load(f) + # Skip the _comment key. + self._exact = { + normalize_license_key(k): v + for k, v in raw.items() if not k.startswith("_") + } + self._lower = {k.lower(): v for k, v in self._exact.items()} + + def lookup(self, raw_id: str) -> Optional[str]: + k = normalize_license_key(raw_id) + if k in self._exact: + return self._exact[k] + kl = k.lower() + if kl in self._lower: + return self._lower[kl] + return None + + +# --------------------------------------------------------------------------- +# DEP-5 parser (machine-readable copyright format 1.0) +# --------------------------------------------------------------------------- + +_DEP5_HEADER_RE = re.compile( + r"^Format:\s*https?://(?:www\.)?debian\.org/doc/" + r"packaging-manuals/copyright-format/1\.0/?", + re.IGNORECASE | re.MULTILINE, +) + + +def _split_paragraphs(text: str) -> list: + """Split RFC822-ish copyright file into paragraphs.""" + paragraphs = [] + current: list = [] + for line in text.splitlines(): + if not line.strip(): + if current: + paragraphs.append("\n".join(current)) + current = [] + else: + current.append(line) + if current: + paragraphs.append("\n".join(current)) + return paragraphs + + +def _paragraph_fields(paragraph: str) -> dict: + """Parse a DEP-5 paragraph into a {field: value} dict (multi-line + values joined; leading-space continuations are folded).""" + fields: dict = {} + current_key: Optional[str] = None + current_val: list = [] + for line in paragraph.splitlines(): + if line.startswith((" ", "\t")): + if current_key is not None: + current_val.append(line.strip()) + else: + if current_key is not None: + fields[current_key] = "\n".join(current_val).strip() + if ":" in line: + k, _, v = line.partition(":") + current_key = k.strip() + current_val = [v.strip()] + if current_key is not None: + fields[current_key] = "\n".join(current_val).strip() + return fields + + +def parse_dep5(text: str, lmap: LicenseMap) -> Optional[str]: + """Returns an SPDX expression, or None if DEP-5 parse failed.""" + if not _DEP5_HEADER_RE.search(text[:500]): + return None + paragraphs = _split_paragraphs(text) + seen_licenses: "OrderedDict[str, None]" = OrderedDict() + for p in paragraphs: + fields = _paragraph_fields(p) + if "License" not in fields or "Files" not in fields: + continue + # Take just the first line of the License: field — it's the + # short identifier. The rest is the license text body. + first_line = fields["License"].splitlines()[0].strip() + for tok in re.split(r"\s+(?:or|and|OR|AND)\s+", first_line): + tok = tok.strip() + if not tok: + continue + mapped = lmap.lookup(tok) + seen_licenses[mapped or f"LicenseRef-{tok}"] = None + if not seen_licenses: + return None + if len(seen_licenses) == 1: + return next(iter(seen_licenses)) + return " AND ".join(seen_licenses.keys()) + + +# --------------------------------------------------------------------------- +# licensecheck fallback +# --------------------------------------------------------------------------- + + +def _have_licensecheck() -> bool: + return shutil.which("licensecheck") is not None + + +def _run_licensecheck(path: str) -> Optional[str]: + """Returns the licensecheck verdict (raw string, like 'GPL-2+') or None.""" + try: + r = subprocess.run( + ["licensecheck", "--copyright", "--no-conf", path], + capture_output=True, text=True, timeout=15, check=False, + ) + except Exception: + return None + if r.returncode != 0: + return None + for line in r.stdout.splitlines(): + # Format: ": " + if ":" in line: + _, _, lic = line.partition(":") + lic = lic.strip() + if lic and lic.upper() != "UNKNOWN": + return lic + return None + + +def parse_licensecheck(path: str, lmap: LicenseMap) -> Optional[str]: + raw = _run_licensecheck(path) + if not raw: + return None + mapped = lmap.lookup(raw) + return mapped or f"LicenseRef-{raw}" + + +# --------------------------------------------------------------------------- +# Tarball iteration +# --------------------------------------------------------------------------- + + +_PKG_PATH_RE = re.compile(r"(?:^|/)usr/share/doc/([^/]+)/copyright$") + + +def resolve_from_tarball( + tarball: str, lmap: LicenseMap, results: dict, + noassertions: list, quiet: bool = False, +) -> None: + """Walk a copyrights.tar.gz and populate results[pkg] = spdx_expr.""" + if not os.path.isfile(tarball): + if not quiet: + sys.stderr.write( + f"[sbom_resolve_licenses.py] tarball not found: {tarball}\n" + ) + return + try: + tf = tarfile.open(tarball, "r:*") + except Exception as e: + sys.stderr.write( + f"[sbom_resolve_licenses.py] could not open {tarball}: {e}\n" + ) + return + + have_lc = _have_licensecheck() + + with tf: + for member in tf: + if not member.isfile(): + continue + m = _PKG_PATH_RE.search(member.name) + if not m: + continue + pkg = m.group(1) + if pkg in results: + # Already resolved from an earlier tarball; keep the first. + continue + try: + f = tf.extractfile(member) + if f is None: + continue + text = f.read().decode("utf-8", errors="replace") + except Exception: + continue + + spdx = parse_dep5(text, lmap) + if not spdx and have_lc: + # Stage to disk and let licensecheck have a go. + tmp = f"/tmp/sbom-cp-{pkg}.txt" + try: + with open(tmp, "w") as fw: + fw.write(text) + spdx = parse_licensecheck(tmp, lmap) + finally: + try: + os.unlink(tmp) + except Exception: + pass + if spdx: + results[pkg] = spdx + else: + results[pkg] = "NOASSERTION" + noassertions.append(pkg) + + +# --------------------------------------------------------------------------- +# Entry point +# --------------------------------------------------------------------------- + + +def main() -> int: + ap = argparse.ArgumentParser(description=__doc__) + ap.add_argument("--copyrights", action="append", default=[], + help="Path to a copyrights.tar.gz; repeatable.") + ap.add_argument("--output", required=True, + help="Where to write the resolved map (JSON).") + ap.add_argument("--license-map", default=DEFAULT_LICENSE_MAP, + help="Path to the SPDX translation table.") + ap.add_argument("--quiet", action="store_true") + args = ap.parse_args() + + try: + lmap = LicenseMap(args.license_map) + except Exception as e: + sys.stderr.write( + f"[sbom_resolve_licenses.py] could not load license map: {e}\n" + ) + return 1 + + results: dict = {} + noassertions: list = [] + for tarball in args.copyrights: + resolve_from_tarball(tarball, lmap, results, noassertions, + quiet=args.quiet) + + try: + with open(args.output, "w") as f: + json.dump({ + "license_map_version": "1", + "resolved": dict(sorted(results.items())), + "noassertion_packages": sorted(noassertions), + }, f, indent=2) + except Exception as e: + sys.stderr.write( + f"[sbom_resolve_licenses.py] could not write {args.output}: {e}\n" + ) + return 1 + + resolved_count = len(results) - len(noassertions) + total = len(results) + pct = (100.0 * resolved_count / total) if total else 0.0 + sys.stderr.write( + f"[sbom_resolve_licenses.py] resolved {resolved_count}/{total} " + f"packages ({pct:.1f}%); {len(noassertions)} NOASSERTION\n" + ) + return 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/scripts/sbom_vuln_diff.py b/scripts/sbom_vuln_diff.py new file mode 100755 index 00000000000..c0c471b12cf --- /dev/null +++ b/scripts/sbom_vuln_diff.py @@ -0,0 +1,165 @@ +#!/usr/bin/env python3 +""" +sbom_vuln_diff.py — compare two vulnerability reports (produced by +sbom_vuln_scan.py) and report which CVEs were added, removed, or +changed status. + +Usage: + sbom_vuln_diff.py old.vuln.json new.vuln.json + sbom_vuln_diff.py --json out.json --quiet old.vuln.json new.vuln.json + +Intended for release-cycle drift analysis: "what new CVEs apply to my +SBOM since the last release?" without rebuilding the image. Also used +to verify VEX suppressions: same SBOM, before vs after adding new VEX +documents. +""" + +import argparse +import json +import sys + + +_SEVERITY_RANK = { + "negligible": 0, "low": 1, "medium": 2, "high": 3, "critical": 4, + "unknown": -1, +} + + +def load_vulns(path: str) -> dict: + """Return { (cve_id, affected_ref): vuln-dict } for one report.""" + with open(path) as f: + doc = json.load(f) + out = {} + for v in doc.get("vulnerabilities", []) or []: + cve = v.get("id", "?") + for affects in v.get("affects", []) or [{}]: + ref = affects.get("ref", "?") + out[(cve, ref)] = v + return out + + +def severity(v: dict) -> str: + ratings = v.get("ratings") or [] + if not ratings: + return "unknown" + return (ratings[0].get("severity") or "unknown").lower() + + +def rank(s: str) -> int: + return _SEVERITY_RANK.get(s, -1) + + +def vuln_status(v: dict) -> str: + """Detect VEX-applied status (CycloneDX 1.6 analysis.state).""" + a = v.get("analysis") or {} + return a.get("state", "") + + +def diff(old: dict, new: dict) -> dict: + added = [] + removed = [] + status_changed = [] + severity_changed = [] + for key in sorted(new.keys() - old.keys()): + added.append({"key": list(key), "vuln": new[key]}) + for key in sorted(old.keys() - new.keys()): + removed.append({"key": list(key), "vuln": old[key]}) + for key in sorted(new.keys() & old.keys()): + o, n = old[key], new[key] + if severity(o) != severity(n): + severity_changed.append({ + "key": list(key), + "old": severity(o), + "new": severity(n), + }) + if vuln_status(o) != vuln_status(n): + status_changed.append({ + "key": list(key), + "old": vuln_status(o) or "(unsuppressed)", + "new": vuln_status(n) or "(unsuppressed)", + }) + return { + "added": added, + "removed": removed, + "severity_changed": severity_changed, + "status_changed": status_changed, + } + + +def print_report(d: dict, old_path: str, new_path: str) -> None: + print(f"Comparing vuln reports:") + print(f" old: {old_path}") + print(f" new: {new_path}") + print() + print(f"Added: {len(d['added'])}") + print(f"Removed: {len(d['removed'])}") + print(f"Severity changed: {len(d['severity_changed'])}") + print(f"Status changed: {len(d['status_changed'])}") + if d["added"]: + print() + print("--- Newly present in NEW (regressions or newly disclosed) ---") + # Sort by severity desc. + items = sorted( + d["added"], + key=lambda x: -rank(severity(x["vuln"])), + ) + for x in items[:50]: + sev = severity(x["vuln"]) + cve, ref = x["key"] + print(f" + [{sev:8}] {cve:22} {ref}") + if len(d["added"]) > 50: + print(f" ... and {len(d['added']) - 50} more") + if d["removed"]: + print() + print("--- No longer present in NEW (fixed, removed, or VEX'd) ---") + for x in d["removed"][:30]: + cve, ref = x["key"] + print(f" - {cve:22} {ref}") + if len(d["removed"]) > 30: + print(f" ... and {len(d['removed']) - 30} more") + if d["status_changed"]: + print() + print("--- VEX status flips ---") + for x in d["status_changed"][:30]: + cve, ref = x["key"] + print(f" ~ {cve} ({x['old']} -> {x['new']}) {ref}") + if d["severity_changed"]: + print() + print("--- Severity reclassified ---") + for x in d["severity_changed"][:30]: + cve, ref = x["key"] + print(f" ~ {cve} ({x['old']} -> {x['new']}) {ref}") + + +def main() -> int: + ap = argparse.ArgumentParser(description=__doc__) + ap.add_argument("old", help="Older / baseline .vuln.json") + ap.add_argument("new", help="Newer / candidate .vuln.json") + ap.add_argument("--quiet", action="store_true") + ap.add_argument("--json", + help="Write the diff as JSON to this file") + args = ap.parse_args() + + old = load_vulns(args.old) + new = load_vulns(args.new) + d = diff(old, new) + + if args.json: + with open(args.json, "w") as f: + json.dump(d, f, indent=2, sort_keys=True) + + if args.quiet: + print(f"+{len(d['added'])} " + f"-{len(d['removed'])} " + f"~sev:{len(d['severity_changed'])} " + f"~status:{len(d['status_changed'])}") + else: + print_report(d, args.old, args.new) + + if d["added"] or d["removed"] or d["severity_changed"] or d["status_changed"]: + return 1 + return 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/scripts/sbom_vuln_scan.py b/scripts/sbom_vuln_scan.py new file mode 100755 index 00000000000..acf86c19d64 --- /dev/null +++ b/scripts/sbom_vuln_scan.py @@ -0,0 +1,491 @@ +#!/usr/bin/env python3 +""" +sbom_vuln_scan.py — generate a vulnerability report from a SONiC SBOM. + +This is a **standalone post-build tool**, intentionally not part of the +build. It runs anywhere a CycloneDX SBOM and Python 3 are available and +produces a CycloneDX VEX-annotated vulnerability report. + +Why standalone: +- The SBOM is reproducible; a vulnerability report cannot be (new CVEs + are disclosed daily). Coupling them would break SBOM reproducibility. +- A six-month-old SBOM can be re-scanned against today's CVE data + without rebuilding the .bin. +- Release engineering picks its own cadence for CVE scans separate from + build cadence. + +Usage: + sbom_vuln_scan.py SBOM.cdx.json + sbom_vuln_scan.py --vex vex/ --output target/sonic-broadcom.bin.vuln.json SBOM.cdx.json + sbom_vuln_scan.py --min-severity high --format table SBOM.cdx.json +""" + +import argparse +import json +import os +import shutil +import subprocess +import sys +from typing import Any, Optional + + +_SEVERITY_RANK = { + "negligible": 0, "low": 1, "medium": 2, "high": 3, "critical": 4, + "unknown": -1, +} + + +def warn(msg: str) -> None: + sys.stderr.write(f"[sbom_vuln_scan.py] WARNING: {msg}\n") + + +def info(msg: str) -> None: + sys.stderr.write(f"[sbom_vuln_scan.py] {msg}\n") + + +# --------------------------------------------------------------------------- +# Grype install / invocation +# --------------------------------------------------------------------------- + + +def resolve_grype() -> Optional[str]: + """Find grype on PATH, then in the build cache, then auto-install via + scripts/install_sbom_tool.sh.""" + p = shutil.which("grype") + if p: + return p + # Build-tree cache. + for d in ("target/sbom-tools", os.path.expanduser("~/.cache/sonic-sbom")): + if not os.path.isdir(d): + continue + for sub in os.listdir(d): + cand = os.path.join(d, sub, "grype") + if os.path.isfile(cand) and os.access(cand, os.X_OK): + return cand + # Auto-fetch via the install helper. + install_script = os.path.join( + os.path.dirname(__file__), "install_sbom_tool.sh" + ) + if not os.path.isfile(install_script): + return None + try: + r = subprocess.run( + [install_script, "grype"], + capture_output=True, text=True, timeout=300, + ) + out = r.stdout.strip() + if r.returncode == 0 and out: + return out + except Exception as e: + warn(f"install_sbom_tool.sh grype failed: {e}") + return None + + +def run_grype(grype: str, sbom_path: str, vex_files: list) -> Optional[dict]: + """Run grype against an SBOM and return the parsed JSON output.""" + cmd = [grype, f"sbom:{sbom_path}", "-o", "json", "-q"] + for v in vex_files: + cmd.extend(["--vex", v]) + info(f"running: {' '.join(cmd)}") + try: + r = subprocess.run(cmd, capture_output=True, text=True, timeout=900) + except subprocess.TimeoutExpired: + warn("grype timed out") + return None + except Exception as e: + warn(f"grype failed to start: {e}") + return None + if r.returncode != 0: + warn(f"grype exited {r.returncode}: {r.stderr.strip()[:400]}") + return None + try: + return json.loads(r.stdout) + except Exception as e: + warn(f"could not parse grype JSON: {e}") + return None + + +# --------------------------------------------------------------------------- +# VEX loading (project-local YAML files) +# --------------------------------------------------------------------------- + + +def collect_vex_documents(vex_dirs: list) -> list: + """Return absolute paths to every .yaml/.yml/.json file under the + given VEX directories. Files are passed directly to grype which + handles OpenVEX, CycloneDX-VEX, and CSAF formats.""" + out = [] + for d in vex_dirs: + if not os.path.isdir(d): + warn(f"vex dir not found: {d}") + continue + for root, _, files in os.walk(d): + for fn in files: + if fn.startswith(".") or fn.endswith(".md"): + continue + if fn.endswith((".yaml", ".yml", ".json")): + out.append(os.path.abspath(os.path.join(root, fn))) + return sorted(out) + + +# --------------------------------------------------------------------------- +# Report shaping +# --------------------------------------------------------------------------- + + +def severity_rank(s: str) -> int: + return _SEVERITY_RANK.get((s or "").lower(), -1) + + +def filter_by_severity(matches: list, min_sev: Optional[str]) -> list: + if not min_sev: + return matches + threshold = severity_rank(min_sev) + if threshold < 0: + warn(f"unknown severity: {min_sev}; not filtering") + return matches + return [ + m for m in matches + if severity_rank(m.get("vulnerability", {}).get("severity", "")) + >= threshold + ] + + +def summarize(matches: list) -> dict: + by_sev: dict = {} + suppressed = 0 + for m in matches: + sev = (m.get("vulnerability") or {}).get("severity") or "Unknown" + by_sev[sev] = by_sev.get(sev, 0) + 1 + return {"total": len(matches), "by_severity": by_sev, + "suppressed_via_vex": suppressed} + + +def to_cyclonedx_vex(grype_doc: dict, source_sbom_path: str) -> dict: + """Wrap grype's matches into a CycloneDX 1.6 VEX document.""" + sbom_meta = {} + try: + with open(source_sbom_path) as f: + src = json.load(f) + sbom_meta = src.get("metadata", {}) + except Exception: + pass + + vulns = [] + for m in grype_doc.get("matches", []): + v = m.get("vulnerability", {}) + art = m.get("artifact", {}) + purl = art.get("purl") or f"{art.get('name','?')}@{art.get('version','?')}" + primary, _ = vuln_id_pair(m) + entry: dict[str, Any] = { + "id": primary, + "source": {"name": v.get("dataSource", "").split("/")[2] + if v.get("dataSource", "").startswith(("http://", "https://")) + else "unknown", + "url": v.get("dataSource", "")}, + "ratings": [{ + "severity": (v.get("severity") or "unknown").lower(), + "method": "other", + }], + "description": v.get("description") or "", + "affects": [{"ref": purl}], + } + # Surface aliases via references[] (CycloneDX 1.6) so downstream + # consumers can join across CVE / GHSA / vendor-VDB id spaces. + # Add the grype-canonical id when it differs from primary (it + # will when we swapped CVE→primary, GHSA→here), then add every + # CVE alias from relatedVulnerabilities[] that isn't already + # primary. + refs = [] + if v.get("id") and v.get("id") != primary: + refs.append({ + "id": v["id"], + "source": {"name": "GitHub Advisory Database", + "url": v.get("dataSource", "")}, + }) + for rel in m.get("relatedVulnerabilities", []) or []: + rid = rel.get("id", "") + if not rid or rid == primary: + continue + if rid.startswith("CVE-"): + refs.append({ + "id": rid, + "source": { + "name": "NVD", + "url": f"https://nvd.nist.gov/vuln/detail/{rid}", + }, + }) + if refs: + entry["references"] = refs + # Preserve grype's exact fix.state ('fixed', 'not-fixed', + # 'wont-fix', 'unknown') as a CycloneDX property so downstream + # consumers can distinguish "no fix yet" from "upstream + # declined to fix" without having to parse the recommendation + # field. Only 'fixed' findings get a recommendation; the rest + # carry only the state property. + fix = v.get("fix") or {} + state = fix.get("state", "unknown") + entry.setdefault("properties", []).append({ + "name": "grype:fix-state", + "value": state, + }) + if state == "fixed" and fix.get("versions"): + entry["recommendation"] = ( + f"Upgrade to {', '.join(fix['versions'])}" + ) + vulns.append(entry) + + return { + "bomFormat": "CycloneDX", + "specVersion": "1.6", + "version": 1, + "metadata": { + "tools": [{ + "vendor": "Anchore", + "name": "grype", + "version": grype_doc.get("descriptor", {}) + .get("version", "unknown"), + }, { + "vendor": "SONiC", + "name": "sbom_vuln_scan.py", + "version": "1.0", + }], + "properties": [ + {"name": "sonic:source_sbom", "value": source_sbom_path}, + {"name": "sonic:source_sbom_serial", + "value": sbom_meta.get("component", {}) + .get("bom-ref", "") or ""}, + ], + }, + "vulnerabilities": vulns, + } + + +def fix_state_of(match: dict) -> str: + """grype's fix.state for a match — 'fixed', 'not-fixed', 'wont-fix', + 'unknown'.""" + return (match.get("vulnerability", {}).get("fix") or {}).get( + "state", "unknown" + ) + + +def is_actionable(match: dict) -> bool: + """A finding is actionable when an upstream fix is available.""" + return fix_state_of(match) == "fixed" + + +def print_table(matches: list, total_by_sev: dict) -> None: + # Bucket findings by fix state. The human-readable table only + # lists actionable rows (the ones a release manager can do + # something about); not-fixed and wont-fix are summarised in the + # header so the reader knows they exist without being buried in + # un-fixable noise. The CycloneDX JSON sidecar contains the full + # set for machine consumption. + by_fix_state: dict = {"fixed": [], "not-fixed": [], "wont-fix": [], + "unknown": []} + for m in matches: + state = fix_state_of(m) + by_fix_state.setdefault(state, []).append(m) + + fixed = by_fix_state.get("fixed", []) + not_fixed = by_fix_state.get("not-fixed", []) + wont_fix = by_fix_state.get("wont-fix", []) + unknown = by_fix_state.get("unknown", []) + + print(f"Total findings: {sum(total_by_sev.values())}") + print("By fix state:") + print(f" Actionable (upstream fix available): {len(fixed)}") + print(f" Not-fixed (no upstream fix yet): {len(not_fixed)}") + print(f" Won't-fix (upstream declined): {len(wont_fix)}") + if unknown: + print(f" Unknown fix state: {len(unknown)}") + + # Severity breakdown for actionable findings only — that's what + # the row listing below covers and what release planning targets. + actionable_by_sev: dict = {} + for m in fixed: + sev = (m.get("vulnerability", {}).get("severity") or "unknown").title() + actionable_by_sev[sev] = actionable_by_sev.get(sev, 0) + 1 + print("By severity (actionable only):") + for sev in ("Critical", "High", "Medium", "Low", "Negligible", "Unknown"): + n = actionable_by_sev.get(sev, 0) + if n: + print(f" {sev:11} {n}") + + if not fixed: + return + print() + print(f"{'SEVERITY':9} {'ID':18} {'ECOSYS':7} " + f"{'PKG':26} {'VERSION':24} FIX") + print("-" * 110) + # Sort by severity desc, then by preferred display id + matches_sorted = sorted( + fixed, + key=lambda m: (-severity_rank( + m.get("vulnerability", {}).get("severity", "") + ), preferred_vuln_id(m)) + ) + for m in matches_sorted: + v = m.get("vulnerability", {}) + a = m.get("artifact", {}) + sev = (v.get("severity") or "?") + primary, _ = vuln_id_pair(m) + ecosys = ecosystem_from_purl(a.get("purl") or "") or a.get("type", "") + pkg = a.get("name") or "?" + ver = a.get("version") or "?" + fix = v.get("fix") or {} + fix_versions = ", ".join(fix.get("versions") or []) or "-" + print(f"{sev:9} {primary:18} {ecosys:7} " + f"{pkg:26} {ver:24} {fix_versions}") + + +def ecosystem_from_purl(purl: str) -> str: + """Extract the package ecosystem label from a PURL. + + Maps long PURL types to short, fixed-width labels suitable for a + table column. 'pkg:deb/...' → 'deb', 'pkg:golang/...' → 'go', + 'pkg:cargo/...' → 'rust', 'pkg:pypi/...' → 'py', etc. + """ + if not purl.startswith("pkg:"): + return "" + after = purl[4:] + head = after.split("/", 1)[0] + return { + "deb": "deb", + "rpm": "rpm", + "apk": "apk", + "cargo": "rust", + "golang": "go", + "npm": "npm", + "pypi": "py", + "gem": "gem", + "maven": "java", + "nuget": "nuget", + "oci": "oci", + "github": "github", + "generic": "generic", + }.get(head, head[:7]) + + +def vuln_id_pair(m: dict) -> tuple[str, str]: + """Return (primary_id, alias_id) for display. + + Grype puts the canonical advisory id (often GHSA-* for findings + sourced from the GitHub Advisory Database) in `vulnerability.id`, + and any CVE alias in `relatedVulnerabilities[].id`. We display the + CVE as the *primary* id when one exists, since CVE-NNNN-NNNN is the + identifier humans recognize; the GHSA goes in the alias column. + """ + v = m.get("vulnerability", {}) + primary = v.get("id") or "?" + cve = "" + for rel in m.get("relatedVulnerabilities", []) or []: + rid = rel.get("id", "") + if rid.startswith("CVE-"): + cve = rid + break + if cve and not primary.startswith("CVE-"): + return cve, primary + return primary, cve or "" + + +def preferred_vuln_id(m: dict) -> str: + """CVE if present, else the grype-canonical id. Used for sorting.""" + return vuln_id_pair(m)[0] + + +# --------------------------------------------------------------------------- +# Entry point +# --------------------------------------------------------------------------- + + +def main() -> int: + ap = argparse.ArgumentParser(description=__doc__) + ap.add_argument("sbom", + help="Input CycloneDX SBOM (.cdx.json)") + ap.add_argument("--vex", action="append", default=[], + help="Directory containing VEX yaml/json documents. " + "Repeatable.") + ap.add_argument("--output", "-o", + help="Write CycloneDX VEX-annotated report here. " + "Default: .vuln.json") + ap.add_argument("--format", choices=("json", "table", "both"), + default="both", + help="Output style. 'json' writes only the file; " + "'table' writes only stdout; 'both' (default).") + ap.add_argument("--min-severity", + help="Filter findings below this severity in the " + "report. (negligible/low/medium/high/critical)") + ap.add_argument("--fail-on", + help="Exit non-zero if any finding at or above this " + "severity remains after VEX/filter. Useful for " + "CI gating.") + ap.add_argument("--grype", + help="Path to grype binary. Auto-installed if absent.") + args = ap.parse_args() + + if not os.path.isfile(args.sbom): + warn(f"SBOM not found: {args.sbom}") + return 2 + + grype = args.grype or resolve_grype() + if not grype: + warn("grype is not available; install via scripts/install_sbom_tool.sh") + return 2 + info(f"using grype: {grype}") + + vex_files = collect_vex_documents(args.vex) + if vex_files: + info(f"loaded {len(vex_files)} VEX document(s)") + + grype_doc = run_grype(grype, args.sbom, vex_files) + if grype_doc is None: + return 2 + + matches = grype_doc.get("matches", []) or [] + if args.min_severity: + before = len(matches) + matches = filter_by_severity(matches, args.min_severity) + info(f"min-severity filter: {len(matches)}/{before} remain") + + by_sev: dict = {} + for m in matches: + sev = (m.get("vulnerability") or {}).get("severity") or "Unknown" + by_sev[sev] = by_sev.get(sev, 0) + 1 + + out_path = args.output or (args.sbom.removesuffix(".cdx.json") + ".vuln.json") + if args.format in ("json", "both"): + out_doc = to_cyclonedx_vex({**grype_doc, "matches": matches}, + args.sbom) + try: + with open(out_path, "w") as f: + json.dump(out_doc, f, indent=2, sort_keys=True) + info(f"wrote {out_path}") + except Exception as e: + warn(f"could not write {out_path}: {e}") + + if args.format in ("table", "both"): + print_table(matches, by_sev) + + if args.fail_on: + # Restrict the failure gate to *actionable* findings — i.e. those + # with an upstream fix available. Failing on not-fixed / wont-fix + # entries gates the build on things the team can't act on; those + # belong on a separate VEX triage workflow, not CI gating. + threshold = severity_rank(args.fail_on) + high_findings = [ + m for m in matches + if is_actionable(m) + and severity_rank(m.get("vulnerability", {}).get("severity", "")) + >= threshold + ] + if high_findings: + warn(f"{len(high_findings)} actionable finding(s) at or above " + f"'{args.fail_on}' — failing") + return 1 + + return 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/slave.mk b/slave.mk index be3b37e7aaa..cfd23066f54 100644 --- a/slave.mk +++ b/slave.mk @@ -525,6 +525,11 @@ $(info "CROSS_BUILD_ENVIRON" : "$(CROSS_BUILD_ENVIRON)") $(info "INCLUDE_EXTERNAL_PATCHES" : "$(INCLUDE_EXTERNAL_PATCHES)") $(info "PTF_ENV_PY_VER" : "$(PTF_ENV_PY_VER)") $(info "ENABLE_MULTIDB" : "$(ENABLE_MULTIDB)") +$(info "ENABLE_SBOM" : "$(ENABLE_SBOM)") +$(info "SBOM_FORMAT" : "$(SBOM_FORMAT)") +$(info "SBOM_SCAN_TOOL" : "$(SBOM_SCAN_TOOL)") +$(info "SBOM_INCLUDE_LICENSES" : "$(SBOM_INCLUDE_LICENSES)") +$(info "SBOM_STRICT" : "$(SBOM_STRICT)") $(info ) else $(info SONiC Build System for $(CONFIGURED_PLATFORM):$(CONFIGURED_ARCH)) @@ -600,6 +605,40 @@ endef # $(1) => Docker name # $(2) => Docker target name +# $(call sbom_emit_fragment,artifact,recipe_type,src_path,url,depends,rdepends,main_deb) +# Emit a per-artifact CycloneDX SBOM fragment next to the artifact. +# The make-side $(if) short-circuits when ENABLE_SBOM != y so non-SBOM +# builds skip Python startup entirely (otherwise ~30s of cumulative +# overhead across ~120 recipe hooks). When y, the script still self-gates +# defensively. +# +# Failure handling honors SBOM_STRICT: +# SBOM_STRICT=y → fragment-emit failures abort the recipe (consistent +# with the strict contract: user opted in, surface +# every problem). +# SBOM_STRICT=n → failures swallowed via `|| true` so a script bug +# can never break a build (defensive fallback). +# +# Recipes can set $(ARTIFACT)_LICENSE = "" to provide an +# explicit license for SONiC-native artifacts where no debian/copyright +# ships and the DEP-5 resolver would otherwise emit NOASSERTION. +sbom_emit_fragment = $(if $(filter y,$(ENABLE_SBOM)),ARTIFACT="$(1)" RECIPE_TYPE="$(2)" SRC_PATH="$(3)" URL="$(4)" DEPENDS="$(5)" RDEPENDS="$(6)" MAIN_DEB="$(7)" ARTIFACT_LICENSE="$($(notdir $(1))_LICENSE)" ENABLE_SBOM="$(ENABLE_SBOM)" CONFIGURED_PLATFORM="$(CONFIGURED_PLATFORM)" CONFIGURED_ARCH="$(CONFIGURED_ARCH)" SONIC_IMAGE_VERSION="$(SONIC_IMAGE_VERSION)" python3 ./scripts/sbom_fragment.py$(if $(filter y,$(SBOM_STRICT)),, || true),:) + +# Test containers that DON'T ship in any .bin installer payload but DO +# get built (controlled by their respective INCLUDE_* flags) and need +# their own standalone SBOM for security tooling. Each gets a sidecar +# target/.gz.sbom.cdx.json at build time. Containers that +# ship in a .bin are covered by the per-.bin aggregate SBOM and don't +# need a duplicate sidecar. +SBOM_TEST_CONTAINERS = docker-ptf.gz docker-ptf-sai.gz docker-sonic-mgmt.gz + +# $(call sbom_emit_per_container,docker_name,artifact_path) +# Emit a per-container CycloneDX SBOM. Fires only for SBOM_TEST_CONTAINERS +# and only when ENABLE_SBOM=y. SBOM_SCAN_TOOL is honored so operators +# can opt into syft cross-check per-container (default: skip the +# per-container scanner pass to keep build time manageable). +sbom_emit_per_container = $(if $(filter y,$(ENABLE_SBOM)),$(if $(filter $(notdir $(2)),$(SBOM_TEST_CONTAINERS)),TARGET_PATH="$(TARGET_PATH)" CONFIGURED_PLATFORM="$(CONFIGURED_PLATFORM)" CONFIGURED_ARCH="$(CONFIGURED_ARCH)" SONIC_IMAGE_VERSION="$(SONIC_IMAGE_VERSION)" SONIC_VERSION_CONTROL_COMPONENTS="$(SONIC_VERSION_CONTROL_COMPONENTS)" ENABLE_SBOM="$(ENABLE_SBOM)" SBOM_FORMAT="$(SBOM_FORMAT)" SBOM_SCAN_TOOL="$(if $(SBOM_PER_CONTAINER_SCAN_TOOL),$(SBOM_PER_CONTAINER_SCAN_TOOL),none)" SBOM_INCLUDE_LICENSES="$(SBOM_INCLUDE_LICENSES)" SBOM_STRICT="$(SBOM_STRICT)" python3 ./scripts/build_sbom.py --container "$(notdir $(2))"$(if $(filter y,$(SBOM_STRICT)),, || true),:),:) + define docker-image-save @echo "Attempting docker image lock for $(1) save" $(LOG) $(call MOD_LOCK,$(1),$(DOCKER_LOCKDIR),$(DOCKER_LOCKFILE_SUFFIX),$(DOCKER_LOCKFILE_TIMEOUT)) @@ -608,6 +647,13 @@ define docker-image-save docker tag $(1)-$(DOCKER_USERNAME):$(DOCKER_USERTAG) $(1):$(call docker-get-tag,$(1)) $(LOG) @echo "Saving docker image $(1):$(call docker-get-tag,$(1))" $(LOG) docker save $(1):$(call docker-get-tag,$(1)) | pigz -c > $(2) + # Emit SBOM fragment for the saved docker archive (no-op when ENABLE_SBOM != y). + $(call sbom_emit_fragment,$(2),DOCKER_IMAGE,,,,,) + # For test containers that don't ship in any .bin (docker-ptf, + # docker-sonic-mgmt, etc.), emit a standalone per-container SBOM + # so they can be security-scanned independently. No-op for the + # ~30 other containers (covered by the per-.bin aggregate SBOM). + $(call sbom_emit_per_container,$(1),$(2)) if [ x$(SONIC_CONFIG_USE_NATIVE_DOCKERD_FOR_BUILD) == x"y" ]; then @echo "Removing docker image $(1):$(call docker-get-tag,$(1))" $(LOG) docker rmi -f $(1):$(call docker-get-tag,$(1)) $(LOG) @@ -711,6 +757,8 @@ $(addprefix $(DEBS_PATH)/, $(SONIC_ONLINE_DEBS)) : $(DEBS_PATH)/% : .platform \ # Save the target deb into DPKG cache $(call SAVE_CACHE,$*,$@) fi + # Emit SBOM fragment for the downloaded vendor/online deb. + $(call sbom_emit_fragment,$(DEBS_PATH)/$*,ONLINE_DEB,,$($*_URL),$($*_DEPENDS),$($*_RDEPENDS),) $(FOOTER) SONIC_TARGET_LIST += $(addprefix $(DEBS_PATH)/, $(SONIC_ONLINE_DEBS)) @@ -844,6 +892,9 @@ $(addprefix $(DEBS_PATH)/, $(SONIC_MAKE_DEBS)) : $(DEBS_PATH)/% : .platform $$(a # Uninstall unneeded build dependency $(call UNINSTALL_DEBS,$($*_UNINSTALLS)) + # Emit SBOM fragment next to the .deb (no-op when ENABLE_SBOM != y). + $(call sbom_emit_fragment,$(DEBS_PATH)/$*,MAKE_DEB,$($*_SRC_PATH),,$($*_DEPENDS),$($*_RDEPENDS),) + $(FOOTER) SONIC_TARGET_LIST += $(addprefix $(DEBS_PATH)/, $(SONIC_MAKE_DEBS)) @@ -894,6 +945,9 @@ $(addprefix $(DEBS_PATH)/, $(SONIC_DPKG_DEBS)) : $(DEBS_PATH)/% : .platform $$(a # Uninstall unneeded build dependency $(call UNINSTALL_DEBS,$($*_UNINSTALLS)) + # Emit SBOM fragment next to the .deb (no-op when ENABLE_SBOM != y). + $(call sbom_emit_fragment,$(DEBS_PATH)/$*,DPKG_DEB,$($*_SRC_PATH),,$($*_DEPENDS),$($*_RDEPENDS),) + $(FOOTER) SONIC_TARGET_LIST += $(addprefix $(DEBS_PATH)/, $(SONIC_DPKG_DEBS)) @@ -909,6 +963,8 @@ $(addprefix $(DEBS_PATH)/, $(SONIC_DERIVED_DEBS)) : $(DEBS_PATH)/% : .platform $ # we depend on it # Put newer timestamp [ -f $@ ] && touch $@ + # Emit SBOM fragment (references parent's bom-ref). + $(call sbom_emit_fragment,$(DEBS_PATH)/$*,DERIVED_DEB,$($*_SRC_PATH),,$($*_DEPENDS),$($*_RDEPENDS),$($*_MAIN_DEB)) $(FOOTER) SONIC_TARGET_LIST += $(addprefix $(DEBS_PATH)/, $(SONIC_DERIVED_DEBS)) @@ -924,6 +980,8 @@ $(addprefix $(DEBS_PATH)/, $(SONIC_EXTRA_DEBS)) : $(DEBS_PATH)/% : .platform $$( # we depend on it # Put newer timestamp [ -f $@ ] && touch $@ + # Emit SBOM fragment (references parent's bom-ref). + $(call sbom_emit_fragment,$(DEBS_PATH)/$*,EXTRA_DEB,$($*_SRC_PATH),,$($*_DEPENDS),$($*_RDEPENDS),$($*_MAIN_DEB)) $(FOOTER) SONIC_TARGET_LIST += $(addprefix $(DEBS_PATH)/, $(SONIC_EXTRA_DEBS)) @@ -1001,6 +1059,9 @@ $(addprefix $(PYTHON_DEBS_PATH)/, $(SONIC_PYTHON_STDEB_DEBS)) : $(PYTHON_DEBS_PA $(call SAVE_CACHE,$*,$@) fi + # Emit SBOM fragment for the stdeb-built python deb. + $(call sbom_emit_fragment,$(PYTHON_DEBS_PATH)/$*,PYTHON_STDEB,$($*_SRC_PATH),,$($*_DEPENDS),$($*_RDEPENDS),) + $(FOOTER) SONIC_TARGET_LIST += $(addprefix $(PYTHON_DEBS_PATH)/, $(SONIC_PYTHON_STDEB_DEBS)) @@ -1066,6 +1127,9 @@ endif # Uninstall unneeded build dependency $(call UNINSTALL_DEBS,$($*_UNINSTALLS)) + # Emit SBOM fragment for the built python wheel. + $(call sbom_emit_fragment,$(PYTHON_WHEELS_PATH)/$*,PYTHON_WHEEL,$($*_SRC_PATH),,$($*_DEPENDS),$($*_RDEPENDS),) + $(FOOTER) SONIC_TARGET_LIST += $(addprefix $(PYTHON_WHEELS_PATH)/, $(SONIC_PYTHON_WHEELS)) @@ -1117,6 +1181,7 @@ $(addprefix $(TARGET_PATH)/, $(SONIC_SIMPLE_DOCKER_IMAGES)) : $(TARGET_PATH)/%.g TRUSTED_GPG_URLS=$(TRUSTED_GPG_URLS) \ SONIC_VERSION_CACHE=$(SONIC_VERSION_CACHE) \ DBGOPT='$(DBGOPT)' \ + ENABLE_SBOM=$(ENABLE_SBOM) \ scripts/prepare_docker_buildinfo.sh $* $($*.gz_PATH)/Dockerfile $(CONFIGURED_ARCH) $(TARGET_DOCKERFILE)/Dockerfile.buildinfo $(LOG) docker info $(LOG) docker build $(DOCKER_NO_CACHE_FLAG) \ @@ -1127,6 +1192,7 @@ $(addprefix $(TARGET_PATH)/, $(SONIC_SIMPLE_DOCKER_IMAGES)) : $(TARGET_PATH)/%.g --build-arg uid=$(UID) \ --build-arg guid=$(GUID) \ --build-arg docker_container_name=$($*.gz_CONTAINER_NAME) \ + --build-arg ENABLE_SBOM=$(ENABLE_SBOM) \ --label Tag=$(SONIC_IMAGE_VERSION) \ -f $(TARGET_DOCKERFILE)/Dockerfile.buildinfo \ -t $(DOCKER_IMAGE_REF) $($*.gz_PATH) $(LOG) @@ -1287,6 +1353,7 @@ $(addprefix $(TARGET_PATH)/, $(DOCKER_IMAGES)) : $(TARGET_PATH)/%.gz : .platform TRUSTED_GPG_URLS=$(TRUSTED_GPG_URLS) \ SONIC_VERSION_CACHE=$(SONIC_VERSION_CACHE) \ DBGOPT='$(DBGOPT)' \ + ENABLE_SBOM=$(ENABLE_SBOM) \ scripts/prepare_docker_buildinfo.sh $* $($*.gz_PATH)/Dockerfile $(CONFIGURED_ARCH) $(LOG) docker info $(LOG) docker build $(DOCKER_NO_CACHE_FLAG) \ @@ -1302,6 +1369,7 @@ $(addprefix $(TARGET_PATH)/, $(DOCKER_IMAGES)) : $(TARGET_PATH)/%.gz : .platform --build-arg SONIC_VERSION_CACHE=$(SONIC_VERSION_CACHE) \ --build-arg SONIC_VERSION_CACHE_SOURCE=$(SONIC_VERSION_CACHE_SOURCE) \ --build-arg image_version=$(SONIC_IMAGE_VERSION) \ + --build-arg ENABLE_SBOM=$(ENABLE_SBOM) \ --label com.azure.sonic.manifest="$$(cat $($*.gz_PATH)/manifest.json)" \ --label Tag=$(SONIC_IMAGE_VERSION) \ $($(subst -,_,$(notdir $($*.gz_PATH)))_labels) \ @@ -1357,6 +1425,7 @@ $(addprefix $(TARGET_PATH)/, $(DOCKER_DBG_IMAGES)) : $(TARGET_PATH)/%-$(DBG_IMAG TRUSTED_GPG_URLS=$(TRUSTED_GPG_URLS) \ SONIC_VERSION_CACHE=$(SONIC_VERSION_CACHE) \ DBGOPT='$(DBGOPT)' \ + ENABLE_SBOM=$(ENABLE_SBOM) \ scripts/prepare_docker_buildinfo.sh $*-dbg $($*.gz_PATH)/Dockerfile-dbg $(CONFIGURED_ARCH) $(LOG) docker info $(LOG) docker build \ @@ -1367,6 +1436,7 @@ $(addprefix $(TARGET_PATH)/, $(DOCKER_DBG_IMAGES)) : $(TARGET_PATH)/%-$(DBG_IMAG --build-arg docker_container_name=$($*.gz_CONTAINER_NAME) \ --build-arg SONIC_VERSION_CACHE=$(SONIC_VERSION_CACHE) \ --build-arg SONIC_VERSION_CACHE_SOURCE=$(SONIC_VERSION_CACHE_SOURCE) \ + --build-arg ENABLE_SBOM=$(ENABLE_SBOM) \ --label com.azure.sonic.manifest="$$(cat $($*.gz_PATH)/manifest.json)" \ --label Tag=$(SONIC_IMAGE_VERSION) \ --file $($*.gz_PATH)/Dockerfile-dbg \ @@ -1465,6 +1535,7 @@ $(addprefix $(TARGET_PATH)/, $(SONIC_RFS_TARGETS)) : $(TARGET_PATH)/% : \ CROSS_BUILD_ENVIRON=$(CROSS_BUILD_ENVIRON) \ MASTER_KUBERNETES_VERSION=$(MASTER_KUBERNETES_VERSION) \ MASTER_CRI_DOCKERD=$(MASTER_CRI_DOCKERD) \ + ENABLE_SBOM="$(ENABLE_SBOM)" \ ./build_debian.sh $(LOG) $(call SAVE_CACHE,$*,$@) @@ -1479,6 +1550,9 @@ $(addprefix $(TARGET_PATH)/, $(SONIC_INSTALLERS)) : $(TARGET_PATH)/% : \ onie-image.conf \ build_debian.sh \ scripts/dbg_files.sh \ + scripts/build_sbom.sh \ + scripts/install_sbom_tool.sh \ + scripts/sbom_fragment.py \ build_image.sh \ $$(addsuffix -install,$$(addprefix $(IMAGE_DISTRO_DEBS_PATH)/,$$($$*_DEPENDS))) \ $$(addprefix $(IMAGE_DISTRO_DEBS_PATH)/,$$($$*_INSTALLS)) \ @@ -1743,6 +1817,8 @@ $(addprefix $(TARGET_PATH)/, $(SONIC_INSTALLERS)) : $(TARGET_PATH)/% : \ MASTER_CRI_DOCKERD=$(MASTER_CRI_DOCKERD) \ MASTER_UI_METRIC_VERSION=$(MASTER_UI_METRIC_VERSION) \ MASTER_UI_DASH_VERSION=$(MASTER_UI_DASH_VERSION) \ + ENABLE_SBOM="$(ENABLE_SBOM)" \ + TARGET_PATH="$(TARGET_PATH)" \ ./build_debian.sh $(LOG) USERNAME="$(USERNAME)" \ @@ -1764,6 +1840,37 @@ $(addprefix $(TARGET_PATH)/, $(SONIC_INSTALLERS)) : $(TARGET_PATH)/% : \ CA_CERT="$(CA_CERT)" \ TARGET_PATH="$(TARGET_PATH)" \ ./build_image.sh $(LOG) + + # SBOM emit runs AFTER build_image.sh so the artifact (.bin / + # .swi / .img.gz) already exists on disk: sbom_emit_provenance.py + # needs it to compute the subject SHA-256 for the in-toto + # attestation. SBOM_TARGET_ARTIFACT is the actual basename + # of what was just built. The outer foreach iterates over + # $*_MACHINE + $*_DEPENDENT_MACHINE, and each iteration's + # build_image.sh writes to sonic-$(dep_machine) (per + # onie-image.conf: OUTPUT_ONIE_IMAGE=target/sonic-$TARGET_MACHINE.bin, + # OUTPUT_ABOOT_IMAGE=target/sonic-aboot-$TARGET_MACHINE.swi, etc.). + # We derive the per-variant artifact name by substituting the + # primary machine for the current dep_machine in $* so each + # variant gets its own .cdx.json / .spdx.json / .intoto.json: + # sonic-broadcom.bin -> sonic-broadcom{,-dnx,-legacy-th}.bin + # sonic-aboot-broadcom.swi -> sonic-aboot-broadcom{,-dnx}.swi + # sonic-vs.img.gz -> sonic-vs.img.gz (single machine) + ENABLE_SBOM="$(ENABLE_SBOM)" \ + SBOM_FORMAT="$(SBOM_FORMAT)" \ + SBOM_SCAN_TOOL="$(SBOM_SCAN_TOOL)" \ + SBOM_INCLUDE_LICENSES="$(SBOM_INCLUDE_LICENSES)" \ + SBOM_STRICT="$(SBOM_STRICT)" \ + SBOM_TARGET_ARTIFACT="$(subst $($*_MACHINE),$(dep_machine),$*)" \ + TARGET_PATH="$(TARGET_PATH)" \ + TARGET_MACHINE="$(dep_machine)" \ + CONFIGURED_ARCH="$(CONFIGURED_ARCH)" \ + CONFIGURED_PLATFORM="$(CONFIGURED_PLATFORM)" \ + SONIC_VERSION_CONTROL_COMPONENTS="$(SONIC_VERSION_CONTROL_COMPONENTS)" \ + SBOM_INSTALLER_DOCKERS="$($*_DOCKERS)" \ + SBOM_INSTALLER_DEBS="$($*_INSTALLS) $($*_LAZY_INSTALLS) $($*_LAZY_BUILD_INSTALLS)" \ + SBOM_INSTALLER_WHEELS="$($*_PYTHON_WHEELS)" \ + ./scripts/build_sbom.sh $(LOG) ) $(foreach docker, $($*_DOCKERS), \ diff --git a/sonic-slave-bookworm/Dockerfile.j2 b/sonic-slave-bookworm/Dockerfile.j2 index c3590c58571..6763e485ff1 100644 --- a/sonic-slave-bookworm/Dockerfile.j2 +++ b/sonic-slave-bookworm/Dockerfile.j2 @@ -805,6 +805,33 @@ ENV PATH $PATH:$RUST_ROOT/bin # (cargo-tarpaulin v0.35.2 pulled in gimli 0.33.0 which needs Rust 1.88+) RUN $RUST_ROOT/bin/cargo install --locked cargo-tarpaulin@0.35.1 +# Install cargo-auditable so every `cargo build` embeds the resolved +# Cargo.lock into the produced binary's .dep-v0 ELF section. Syft reads +# that section at SBOM scan time, giving SONiC SBOMs full per-crate +# visibility for Rust packages without source-tree changes to any +# debian/rules file. Pinned for reproducibility. CARGO_HOME=$RUST_ROOT +# ensures the binary lands at /usr/.cargo/bin/cargo-auditable (in PATH +# for all users) rather than /root/.cargo/bin/ (root-only). +RUN CARGO_HOME=$RUST_ROOT $RUST_ROOT/bin/cargo install --locked cargo-auditable@0.6.4 + +# rust-audit-info is the official CLI for reading the .dep-v0 section +# that cargo-auditable embedded. scripts/sbom_fragment.py invokes it +# on every ELF binary inside each SONiC-built .deb to recover the +# resolved Rust crate set that shipped in that .deb, then emits each +# crate as a CycloneDX component with sonic:source_deb= +# linking it back to the owning artifact. Same install pattern as +# cargo-auditable so the binary lands at /usr/.cargo/bin/rust-audit-info. +RUN CARGO_HOME=$RUST_ROOT $RUST_ROOT/bin/cargo install --locked rust-audit-info@0.5.4 + +# /usr/local/bin/cargo is a transparent shim that routes `cargo build` +# through `cargo auditable` (other subcommands pass through). Single +# source of truth: files/build/cargo-wrapper; staged into the slave's +# buildinfo/ directory by scripts/prepare_docker_buildinfo.sh. The +# shim must sit ahead of $RUST_ROOT/bin in PATH, which the default +# /usr/local/bin already does. +COPY buildinfo/cargo-wrapper /usr/local/bin/cargo +RUN chmod 0755 /usr/local/bin/cargo + {# Include vendor-defined rules for slave container if it exists. Contained in ../files #} {% with DEBIAN_VERSION='bookworm' %} {% include 'files/sonic-slave-Dockerfile.vendor.j2' ignore missing %} diff --git a/sonic-slave-bullseye/Dockerfile.j2 b/sonic-slave-bullseye/Dockerfile.j2 index 542feeddca7..0822aea2091 100644 --- a/sonic-slave-bullseye/Dockerfile.j2 +++ b/sonic-slave-bullseye/Dockerfile.j2 @@ -722,6 +722,33 @@ RUN mkdir -p /.cargo && $RUST_ROOT/bin/rustup target add aarch64-unknown-linux-g ENV RUSTUP_HOME $RUST_ROOT ENV PATH $PATH:$RUST_ROOT/bin +# Install cargo-auditable so every `cargo build` embeds the resolved +# Cargo.lock into the produced binary's .dep-v0 ELF section. Syft reads +# that section at SBOM scan time, giving SONiC SBOMs full per-crate +# visibility for Rust packages without source-tree changes to any +# debian/rules file. Pinned for reproducibility. CARGO_HOME=$RUST_ROOT +# ensures the binary lands at /usr/.cargo/bin/cargo-auditable (in PATH +# for all users) rather than /root/.cargo/bin/ (root-only). +RUN CARGO_HOME=$RUST_ROOT $RUST_ROOT/bin/cargo install --locked cargo-auditable@0.6.4 + +# rust-audit-info is the official CLI for reading the .dep-v0 section +# that cargo-auditable embedded. scripts/sbom_fragment.py invokes it +# on every ELF binary inside each SONiC-built .deb to recover the +# resolved Rust crate set that shipped in that .deb, then emits each +# crate as a CycloneDX component with sonic:source_deb= +# linking it back to the owning artifact. Same install pattern as +# cargo-auditable so the binary lands at /usr/.cargo/bin/rust-audit-info. +RUN CARGO_HOME=$RUST_ROOT $RUST_ROOT/bin/cargo install --locked rust-audit-info@0.5.4 + +# /usr/local/bin/cargo is a transparent shim that routes `cargo build` +# through `cargo auditable` (other subcommands pass through). Single +# source of truth: files/build/cargo-wrapper; staged into the slave's +# buildinfo/ directory by scripts/prepare_docker_buildinfo.sh. The +# shim must sit ahead of $RUST_ROOT/bin in PATH, which the default +# /usr/local/bin already does. +COPY buildinfo/cargo-wrapper /usr/local/bin/cargo +RUN chmod 0755 /usr/local/bin/cargo + {# Include vendor-defined rules for slave container if it exists. Contained in ../files #} {% with DEBIAN_VERSION='bullseye' %} {% include 'files/sonic-slave-Dockerfile.vendor.j2' ignore missing %} diff --git a/sonic-slave-buster/Dockerfile.j2 b/sonic-slave-buster/Dockerfile.j2 index 2b9040b1de0..ffec6e64cfe 100644 --- a/sonic-slave-buster/Dockerfile.j2 +++ b/sonic-slave-buster/Dockerfile.j2 @@ -688,6 +688,33 @@ RUN mkdir -p /.cargo && $RUST_ROOT/bin/rustup target add aarch64-unknown-linux-g ENV RUSTUP_HOME $RUST_ROOT ENV PATH $PATH:$RUST_ROOT/bin +# Install cargo-auditable so every `cargo build` embeds the resolved +# Cargo.lock into the produced binary's .dep-v0 ELF section. Syft reads +# that section at SBOM scan time, giving SONiC SBOMs full per-crate +# visibility for Rust packages without source-tree changes to any +# debian/rules file. Pinned for reproducibility. CARGO_HOME=$RUST_ROOT +# ensures the binary lands at /usr/.cargo/bin/cargo-auditable (in PATH +# for all users) rather than /root/.cargo/bin/ (root-only). +RUN CARGO_HOME=$RUST_ROOT $RUST_ROOT/bin/cargo install --locked cargo-auditable@0.6.4 + +# rust-audit-info is the official CLI for reading the .dep-v0 section +# that cargo-auditable embedded. scripts/sbom_fragment.py invokes it +# on every ELF binary inside each SONiC-built .deb to recover the +# resolved Rust crate set that shipped in that .deb, then emits each +# crate as a CycloneDX component with sonic:source_deb= +# linking it back to the owning artifact. Same install pattern as +# cargo-auditable so the binary lands at /usr/.cargo/bin/rust-audit-info. +RUN CARGO_HOME=$RUST_ROOT $RUST_ROOT/bin/cargo install --locked rust-audit-info@0.5.4 + +# /usr/local/bin/cargo is a transparent shim that routes `cargo build` +# through `cargo auditable` (other subcommands pass through). Single +# source of truth: files/build/cargo-wrapper; staged into the slave's +# buildinfo/ directory by scripts/prepare_docker_buildinfo.sh. The +# shim must sit ahead of $RUST_ROOT/bin in PATH, which the default +# /usr/local/bin already does. +COPY buildinfo/cargo-wrapper /usr/local/bin/cargo +RUN chmod 0755 /usr/local/bin/cargo + {# Include vendor-defined rules for slave container if it exists. Contained in ../files #} {% with DEBIAN_VERSION='buster' %} {% include 'files/sonic-slave-Dockerfile.vendor.j2' ignore missing %} diff --git a/sonic-slave-trixie/Dockerfile.j2 b/sonic-slave-trixie/Dockerfile.j2 index 26b19b9d936..5905cae1636 100644 --- a/sonic-slave-trixie/Dockerfile.j2 +++ b/sonic-slave-trixie/Dockerfile.j2 @@ -805,6 +805,33 @@ ENV PATH=$PATH:$RUST_ROOT/bin # (cargo-tarpaulin v0.35.2 pulled in gimli 0.33.0 which needs Rust 1.88+) RUN $RUST_ROOT/bin/cargo install --locked cargo-tarpaulin@0.35.1 +# Install cargo-auditable so every `cargo build` embeds the resolved +# Cargo.lock into the produced binary's .dep-v0 ELF section. Syft reads +# that section at SBOM scan time, giving SONiC SBOMs full per-crate +# visibility for Rust packages without source-tree changes to any +# debian/rules file. Pinned for reproducibility. CARGO_HOME=$RUST_ROOT +# ensures the binary lands at /usr/.cargo/bin/cargo-auditable (in PATH +# for all users) rather than /root/.cargo/bin/ (root-only). +RUN CARGO_HOME=$RUST_ROOT $RUST_ROOT/bin/cargo install --locked cargo-auditable@0.6.4 + +# rust-audit-info is the official CLI for reading the .dep-v0 section +# that cargo-auditable embedded. scripts/sbom_fragment.py invokes it +# on every ELF binary inside each SONiC-built .deb to recover the +# resolved Rust crate set that shipped in that .deb, then emits each +# crate as a CycloneDX component with sonic:source_deb= +# linking it back to the owning artifact. Same install pattern as +# cargo-auditable so the binary lands at /usr/.cargo/bin/rust-audit-info. +RUN CARGO_HOME=$RUST_ROOT $RUST_ROOT/bin/cargo install --locked rust-audit-info@0.5.4 + +# /usr/local/bin/cargo is a transparent shim that routes `cargo build` +# through `cargo auditable` (other subcommands pass through). Single +# source of truth: files/build/cargo-wrapper; staged into the slave's +# buildinfo/ directory by scripts/prepare_docker_buildinfo.sh. The +# shim must sit ahead of $RUST_ROOT/bin in PATH, which the default +# /usr/local/bin already does. +COPY buildinfo/cargo-wrapper /usr/local/bin/cargo +RUN chmod 0755 /usr/local/bin/cargo + {# Include vendor-defined rules for slave container if it exists. Contained in ../files #} {% with DEBIAN_VERSION='trixie' %} {% include 'files/sonic-slave-Dockerfile.vendor.j2' ignore missing %} diff --git a/src/sonic-build-hooks/scripts/collect_version_files b/src/sonic-build-hooks/scripts/collect_version_files index 6d14a7577d9..bd73ef3273f 100755 --- a/src/sonic-build-hooks/scripts/collect_version_files +++ b/src/sonic-build-hooks/scripts/collect_version_files @@ -32,6 +32,107 @@ while read -r line; do echo "$mirror==$date" >> ${TARGET_PATH}/versions-mirror done < <(grep Date: /var/lib/apt/lists/*_InRelease 2>/dev/null) +## SBOM enrichment: emit a richer per-package TSV alongside the simple +## name==version list. Gated by ENABLE_SBOM=y so non-SBOM builds incur +## zero overhead. Inside each container, /var/lib/apt/lists/ is still +## populated at this point (post_run_buildinfo runs before apt-get +## clean), so apt-cache can resolve SHA256 + Filename for the actual +## binary that was installed in *this* container — important because +## each container may have its own mirror snapshot pin. +## +## Performance: one bulk `apt-cache show pkg1=ver1 pkg2=ver2 ...` call +## per container (1-2 seconds total) instead of one fork per package. +## Output columns: Package, Version, Architecture, Source, SourceVersion, +## Maintainer, Homepage, Filename, SHA256. +if [ "${ENABLE_SBOM:-n}" = "y" ]; then + # License harvest: snapshot /usr/share/doc/*/copyright before any + # cleanup phase runs. apt-autoremove in docker-base layers does NOT + # strip docs by default, so the files survive into the container. + # We tar them up so they ride out via collect_docker_version_files.sh. + sbom_copyrights="${TARGET_PATH}/copyrights.tar.gz" + find /usr/share/doc -name copyright -print0 2>/dev/null \ + | tar --null --no-recursion -czf "$sbom_copyrights" \ + --files-from=- 2>/dev/null \ + || true + + # Lockfile harvest: snapshot any language-ecosystem lockfiles that + # describe transitive deps compiled into this container. Each format + # is the de-facto SBOM source for its ecosystem: + # go.sum (Go modules) — module name+version+h1-sha256 + # package-lock.json (npm) + # pnpm-lock.yaml (pnpm) + # yarn.lock (yarn) + # + # Cargo.lock is NOT harvested here — Rust crate attribution is + # done at recipe-emit time by scripts/sbom_fragment.py via + # rust-audit-info on the cargo-auditable .dep-v0 ELF section + # inside each SONiC-built .deb. That gives per-.deb precision + # (the exact resolved subset that actually shipped, scoped to + # the .deb that bundles the binary), which a workspace-level + # Cargo.lock walk cannot. Cargo.lock harvest is also noisy here + # because the slave container's filesystem contains build-tool + # lockfiles unrelated to anything SONiC ships. + # + # We skip /proc, /sys, /tmp, /var/cache, and the sonic build cache + # directories so we don't pick up build-tooling lockfiles. + sbom_lockfiles="${TARGET_PATH}/lockfiles.tar.gz" + # Walk only the paths where SONiC components actually put source + # trees with lockfiles, rather than the whole `/`. This avoids + # picking up unrelated lockfiles from system packages and is + # significantly faster on a large slave container. Any of these + # paths that don't exist in a given container are silently skipped + # by find. Specific exclusions still prune build-output dirs. + sbom_lockfile_roots=() + for r in /sonic /usr /etc /root /home /opt; do + [ -d "$r" ] && sbom_lockfile_roots+=("$r") + done + if [ ${#sbom_lockfile_roots[@]} -gt 0 ]; then + find "${sbom_lockfile_roots[@]}" \ + \( -path /sonic/target -o -path /sonic/dpkg \ + -o -path '*/node_modules' -o -path /var/cache \) -prune -o \ + \( -name 'go.sum' \ + -o -name 'package-lock.json' -o -name 'pnpm-lock.yaml' \ + -o -name 'yarn.lock' \) -type f -print0 2>/dev/null \ + | tar --null --no-recursion -czf "$sbom_lockfiles" \ + --files-from=- 2>/dev/null \ + || true + fi + + sbom_tsv_dpkg="${TARGET_PATH}/.sbom-dpkg-${DIST}-${ARCH}.tsv" + sbom_tsv_apt="${TARGET_PATH}/.sbom-apt-${DIST}-${ARCH}.tsv" + sbom_tsv_final="${TARGET_PATH}/sbom-pkgs-${DIST}-${ARCH}.tsv" + + # dpkg-known fields (cheap, single dpkg-query call). + dpkg-query -W -f='${Package}\t${Version}\t${Architecture}\t${source:Package}\t${source:Version}\t${Maintainer}\t${Homepage}\n' \ + | grep -Ev "${SKIP_VERSION_PACKAGE}" | sort -u > "$sbom_tsv_dpkg" 2>/dev/null || true + + # Bulk apt-cache call for SHA256 + Filename. One fork, one cache load. + # awk parses the multi-record RFC822 output keyed by Package=Version. + awk -F'\t' '{print $1"="$2}' "$sbom_tsv_dpkg" 2>/dev/null \ + | xargs -r apt-cache show 2>/dev/null \ + | awk ' + BEGIN { RS=""; FS="\n" } + { + pkg=""; ver=""; sha=""; fn=""; + for (i=1; i<=NF; i++) { + if ($i ~ /^Package: /) pkg = substr($i, 10); + else if ($i ~ /^Version: /) ver = substr($i, 10); + else if ($i ~ /^SHA256: /) sha = substr($i, 9); + else if ($i ~ /^Filename: /) fn = substr($i, 11); + } + if (pkg != "" && ver != "") print pkg"="ver"\t"sha"\t"fn; + } + ' | sort -u > "$sbom_tsv_apt" 2>/dev/null || true + + # Join: emit final 9-column TSV. + awk -F'\t' ' + FNR==NR { meta[$1"="$2] = $0; next } + { if ($1 in meta) print meta[$1]"\t"$3"\t"$2 } + ' "$sbom_tsv_dpkg" "$sbom_tsv_apt" > "$sbom_tsv_final" 2>/dev/null || true + + rm -f "$sbom_tsv_dpkg" "$sbom_tsv_apt" +fi + ## Print the unique and sorted result sort -u "${TARGET_PATH}/versions-deb-${DIST}-${ARCH}" -o "${TARGET_PATH}/versions-deb-${DIST}-${ARCH}" if [ -e "${TARGET_PATH}/versions-py2-${DIST}-${ARCH}" ]; then diff --git a/vex/README.md b/vex/README.md new file mode 100644 index 00000000000..88210ff864e --- /dev/null +++ b/vex/README.md @@ -0,0 +1,189 @@ +# SONiC VEX statements + +This directory holds **VEX** (Vulnerability Exploitability eXchange) +statements that suppress or contextualise specific CVE findings against +the SONiC SBOM. They are consumed by `scripts/sbom_vuln_scan.py` (via +the `--vex ` flag) and merged into the vulnerability report. + +## Layout + +``` +vex/ +├── README.md (this file) +├── .json (curated, hand-authored statements) +├── .json +├── ... +└── auto/ (auto-extracted from patch metadata) + └── /.json +``` + +VEX files use **OpenVEX in JSON form** (`.json`). YAML files are also +walked by the scanner but grype rejects YAML — JSON is the format the +toolchain actually consumes, so curated files should be JSON too. + +The top-level `*.json` files are **curated** — they require human +analysis and a written justification. The `auto/` subdirectory is +**machine-generated** by `scripts/sbom_extract_vex_from_patches.py`, +which scans the source tree for patches that mention CVEs in their +filename or header. Regenerate at any time; auto-VEX files are +overwritten idempotently. + +## Why VEX + +The SBOM records every component that ships, including the **upstream +ancestor versions** for SONiC's locally-patched packages (FRR, openssh, +kernel, etc.). Grype matches CVEs against ancestor versions, which +means: + +> Even though we backported the fix as a SONiC patch, grype flags the +> CVE because the *upstream* version in `pedigree.ancestors` is still +> the vulnerable one. + +A VEX statement says: "yes, we know about CVE-X. It applies to the +upstream version listed in our pedigree, but our build is `not_affected` +because we backported the fix as ``." + +Without VEX, every patched component generates noise in the vuln report. + +## Schema (OpenVEX) + +We use [OpenVEX v0.2.0](https://openvex.dev/) because it's natively +supported by grype and is the simplest of the three common VEX +formats (the others are CycloneDX VEX and CSAF VEX). + +Minimal example: + +```json +{ + "@context": "https://openvex.dev/ns/v0.2.0", + "@id": "https://github.com/sonic-net/sonic-buildimage/vex/2024-01-15-frr-cve-12345", + "author": "brad@nexthop.ai", + "timestamp": "2024-01-15T00:00:00Z", + "version": 1, + "statements": [ + { + "vulnerability": {"name": "CVE-2024-12345"}, + "products": [{"@id": "pkg:deb/sonic/frr@10.5.4-sonic-0"}], + "status": "not_affected", + "justification": "vulnerable_code_not_in_execute_path", + "impact_statement": "Fixed by SONiC patch src/sonic-frr/patch/0105-bgpd-Show-all-advertised-paths.patch applied at build time. Verified by manually walking the call graph from the daemon entry points.", + "references": ["https://github.com/FRRouting/frr/commit/abc123"] + } + ] +} +``` + +### Status values + +`status:` must be one of: + +| Status | Meaning | +|---|---| +| `not_affected` | The CVE does **not** apply to this build. Always pair with a `justification`. | +| `affected` | The CVE applies and a fix is pending. Include an `action_statement` describing the mitigation. | +| `fixed` | The CVE was applicable but is now resolved (a fix has shipped in this version). | +| `under_investigation` | Status not yet determined. (auto-VEX uses this for loose CVE mentions.) | + +### Justifications + +When `status: not_affected`, OpenVEX requires a `justification:`: + +| Justification | Meaning | +|---|---| +| `component_not_present` | The vulnerable component isn't actually in the build. | +| `vulnerable_code_not_present` | The component is here but the vulnerable code path was removed/replaced. | +| `vulnerable_code_not_in_execute_path` | The vulnerable code is present but never reachable. | +| `vulnerable_code_cannot_be_controlled_by_adversary` | Reachable but not exploitable. | +| `inline_mitigations_already_exist` | Reachable and exploitable upstream but mitigated by SONiC's wrapper / config. | + +`vulnerable_code_not_in_execute_path` is the typical choice when a +SONiC patch backports the upstream fix. + +### Products + +The `products:` array identifies which SBOM components the statement +applies to. The simplest form is an `@id` with a PURL: + +```yaml +products: + - '@id': pkg:deb/sonic/frr@10.5.4-sonic-0 +``` + +A PURL **prefix** also works (no version specified) — useful for +"applies to whatever version we ship" statements: + +```yaml +products: + - '@id': pkg:deb/sonic/frr +``` + +For statements covering the whole product family (e.g., affecting +every version of openssh-server SONiC has ever shipped), use: + +```yaml +products: + - '@id': pkg:generic/openssh +``` + +## Triage workflow + +When `sbom_vuln_scan.py` reports a new high/critical finding: + +1. Read the upstream advisory linked in the report (`dataSource` URL). +2. Check whether SONiC's pedigree includes a patch that fixes it. The + easiest way is to grep `src//patch/` and + `src//patches/` for the CVE id. +3. If a patch exists: + - Confirm the patch actually addresses the CVE (read the diff, not + just the filename). + - Add a curated VEX file under `vex/` with `status: not_affected, + justification: vulnerable_code_not_in_execute_path` and a brief + `impact_statement` linking the patch. +4. If no patch exists but the code path is unreachable in SONiC: + - Document the unreachability in `impact_statement` (e.g., "SONiC + does not enable feature X; the vulnerable function is never + compiled in" or "this CVE applies to the Windows DLL loader, not + used on Linux"). + - Use `vulnerable_code_not_in_execute_path` or + `vulnerable_code_cannot_be_controlled_by_adversary` as appropriate. +5. If the CVE is genuinely applicable: + - Decide on mitigation (patch backport, version bump, config + change, accept risk). + - Add an `affected` or `under_investigation` VEX entry with a + `action_statement`. + +## Re-running auto-VEX extraction + +``` +python3 scripts/sbom_extract_vex_from_patches.py +``` + +Re-run after pulling new patches. The script overwrites files in +`auto/` idempotently. Curated files under `vex/` itself are never +touched. + +## Consumption + +The vuln scanner picks up everything under the directory: + +```bash +python3 scripts/sbom_vuln_scan.py \ + --vex vex/ \ + --output target/sonic-broadcom.bin.vuln.json \ + target/sonic-broadcom.bin.cdx.json +``` + +grype walks the directory tree, parses both YAML and JSON VEX files, +and applies suppressions. + +To verify a VEX statement actually took effect, generate two reports +(with and without `--vex`) and diff them: + +```bash +python3 scripts/sbom_vuln_scan.py -o /tmp/no-vex.json sbom.cdx.json +python3 scripts/sbom_vuln_scan.py --vex vex/ -o /tmp/with-vex.json sbom.cdx.json +python3 scripts/sbom_vuln_diff.py /tmp/no-vex.json /tmp/with-vex.json +``` + +The diff's `Removed:` section is the set of findings the VEX +suppressed. From 64e05d3287886e70090a647b0deb53871211d132 Mon Sep 17 00:00:00 2001 From: Brad House Date: Wed, 27 May 2026 12:06:32 +0000 Subject: [PATCH 2/2] [sbom]: disable SBOM vulnerability scan in CI pipelines MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Gate all three CI vuln-scan sites behind `condition: false` so the SBOM-based vulnerability scan does not run in the build pipelines for now. SBOM artifacts (.cdx.json / .spdx.json / .intoto.json sidecars) continue to be produced by every ENABLE_SBOM=y build — only the post-build vulnerability scan and report publication is gated. Sites gated: * azure-pipelines.yml: the top-level VulnScan stage that scans every *.cdx.json sidecar from Build + BuildVS. * .azure-pipelines/build-template.yml: the per-platform inline scan step that runs after each platform's `Build sonic image` step. * .azure-pipelines/docker-sonic-mgmt.yml: the standalone job that scans the docker-sonic-mgmt container's SBOM. Each gate carries an inline comment explaining what to flip to re-enable the scan (typically removing `condition: false` or restoring the prior `succeededOrFailed()` / `not(canceled())` predicate). `scripts/sbom_vuln_scan.py` itself is unchanged and can still be run standalone against any committed SBOM: python3 scripts/sbom_vuln_scan.py --vex vex/ target/.cdx.json Signed-off-by: Brad House --- .azure-pipelines/build-template.yml | 7 ++++++- .azure-pipelines/docker-sonic-mgmt.yml | 6 ++++++ azure-pipelines.yml | 8 +++++--- 3 files changed, 17 insertions(+), 4 deletions(-) diff --git a/.azure-pipelines/build-template.yml b/.azure-pipelines/build-template.yml index c691b614458..580fd02ca7a 100644 --- a/.azure-pipelines/build-template.yml +++ b/.azure-pipelines/build-template.yml @@ -180,7 +180,12 @@ jobs: echo "##[endgroup]" done exit $OVERALL_RC - condition: succeededOrFailed() + # Disabled for now; change to `succeededOrFailed()` to re-enable + # the SBOM-based vulnerability scan step. SBOM emission itself is + # unaffected — the .cdx.json sidecars are still produced by the + # `Build sonic image` step above. Only the post-build scan is + # gated. + condition: false continueOnError: true displayName: 'SBOM vulnerability scan' - template: cleanup.yml diff --git a/.azure-pipelines/docker-sonic-mgmt.yml b/.azure-pipelines/docker-sonic-mgmt.yml index 3a7d0ffda82..fdfe99f9bdb 100644 --- a/.azure-pipelines/docker-sonic-mgmt.yml +++ b/.azure-pipelines/docker-sonic-mgmt.yml @@ -94,6 +94,12 @@ stages: jobs: - job: sbom_vuln_scan displayName: "[OPTIONAL] SBOM-based vulnerability scan (docker-sonic-mgmt)" + # Disabled for now; remove this `condition: false` (or change to + # `and(succeeded(), in(dependencies.Build.result, 'Succeeded'))`) + # to re-enable the docker-sonic-mgmt SBOM vulnerability scan. + # The SBOM itself is still produced by the Build stage above + # when ENABLE_SBOM=y; only the post-build scan is gated. + condition: false pool: sonic-ubuntu-1c continueOnError: true timeoutInMinutes: 30 diff --git a/azure-pipelines.yml b/azure-pipelines.yml index 29010d02793..21d68808a1f 100644 --- a/azure-pipelines.yml +++ b/azure-pipelines.yml @@ -247,9 +247,11 @@ stages: dependsOn: - BuildVS - Build - # Run even if a platform build had issues — surface findings for - # whatever SBOMs did make it to the artifact server. - condition: not(canceled()) + # Disabled for now; remove this `condition: false` (or change to + # `not(canceled())`) to re-enable the SBOM-based vulnerability + # scan stage. The SBOM artifacts themselves continue to be + # produced when ENABLE_SBOM=y — only the post-build scan is gated. + condition: false jobs: - job: sbom_vuln_scan displayName: "[OPTIONAL] SBOM-based vulnerability scan (all artifacts)"