Skip to content

feat(ci): cosign-sign sc.sh itself before publishing to dist.simple-container.com#290

Merged
Cre-eD merged 2 commits into
mainfrom
feat/sign-sc-sh
May 24, 2026
Merged

feat(ci): cosign-sign sc.sh itself before publishing to dist.simple-container.com#290
Cre-eD merged 2 commits into
mainfrom
feat/sign-sc-sh

Conversation

@Cre-eD

@Cre-eD Cre-eD commented May 24, 2026

Copy link
Copy Markdown
Contributor

Summary

Phase 2c (PR #186/#212-era) shipped cosign keyless verification for the
tarball that sc.sh downloads. Every release tarball at
dist.simple-container.com publishes a sibling .cosign-bundle and
verify_sc_tarball() aborts extraction on verification failure.

The remaining gap: sc.sh itself is not signed. A bucket-write or
CDN-swap attacker can replace sc.sh with a version whose
verify_sc_tarball is stripped or hard-coded to return 0 — downstream
consumers running bash <(curl ... sc.sh) then execute an unverified
tarball, silently regressing the Phase 2c protection.

This wires push.yaml's docker-finalize job to sign the
post-template sc.sh (the bytes that actually land at
dist.simple-container.com) under the same keyless cosign identity
already used for the tarballs:

^https://github\.com/simple-container-com/api/\.github/workflows/push\.yaml@refs/heads/main$

The resulting sc.sh.cosign-bundle is dropped next to sc.sh in the
dist bundle dir and shipped by the existing welder deploy -e prod
step. One regex, one identity, one verify pattern for both tarballs
and the bootstrap script.

Why hard-fail (no continue-on-error)

The existing tarball signing uses continue-on-error: true during the
Phase 2 14-day bake-in. This step deliberately does not — if signing
fails the deploy step never runs, so consumers never observe an sc.sh
whose .cosign-bundle is missing or stale. Atomic w.r.t. the script
bytes: both publish or neither. The cosign primitive is already proven
in production (same step shape on line 220-224 has signed tarballs since
Phase 2c), so the residual flake risk is sigstore-public being down —
that should block the release, not silently ship an unsignable script.

What's in the diff

.github/workflows/push.yamldocker-finalize job:

  • Grants id-token: write + attestations: write so keyless cosign works.
  • Adds install-attest-tools step (same composite action already used in build-platforms).
  • After finalize dist bundle setup (the step that runs the sed templating of VERSION), signs .sc/stacks/dist/bundle/sc.sh and writes .sc/stacks/dist/bundle/sc.sh.cosign-bundle.
  • The bundle is uploaded to dist.simple-container.com by the existing welder deploy -e prod step (which uploads the whole bundleDir).

docs/SECURITY.md — new "Verifying sc.sh itself" section parallel
to "Verifying tarballs", with a copy-pastable curl + cosign verify-blob

  • bash snippet against the production identity regex. Explicitly notes
    that the script does NOT self-verify (an attacker would tamper with the
    verifier too) — verification must happen out-of-band in the consumer's
    runner.

Pre-merge testing

  • YAML syntax — clean.
  • actionlint — only pre-existing findings (BlackSmith runner labels,
    pre-existing shellcheck noise on untouched steps). New lines clean.
  • Cosign command shape — locally validated with a key-pair flow on a
    copy of the actual sc.sh: cosign sign-blob --bundle … sc.sh
    cosign verify-blob --bundle … sc.sh returns Verified OK; flipping
    one byte (appending exit 0) → cosign verify-blob exits 1 with
    invalid signature when validating ASN.1 encoded signature. The
    bundle serialisation works.
  • Keyless OIDC path is the same primitive already running in
    production for tarballs (push.yaml:220-224), so the only delta is
    which file gets signed — mechanical.

Post-merge smoke test

On the first push.yaml run after merge:

curl -sI https://dist.simple-container.com/sc.sh.cosign-bundle
# expect: HTTP/2 200

curl -fsSL https://dist.simple-container.com/sc.sh -o /tmp/sc.sh
curl -fsSL https://dist.simple-container.com/sc.sh.cosign-bundle -o /tmp/sc.sh.cosign-bundle
cosign verify-blob \
  --bundle /tmp/sc.sh.cosign-bundle \
  --certificate-identity-regexp '^https://github\.com/simple-container-com/api/\.github/workflows/push\.yaml@refs/heads/main$' \
  --certificate-oidc-issuer https://token.actions.githubusercontent.com \
  /tmp/sc.sh
# expect: Verified OK

printf '\nexit 0\n' >> /tmp/sc.sh
cosign verify-blob \
  --bundle /tmp/sc.sh.cosign-bundle \
  --certificate-identity-regexp '^https://github\.com/simple-container-com/api/\.github/workflows/push\.yaml@refs/heads/main$' \
  --certificate-oidc-issuer https://token.actions.githubusercontent.com \
  /tmp/sc.sh
# expect: non-zero, signature failure

Welder follow-up (out of scope)

welder.sh at welder.simple-container.com has the same gap and is
NOT addressed here. Welder has no GitHub Actions today — its releases
run from somebody's laptop via mage + sc deploy. Signing welder.sh
from api/push.yaml would smear the trust root (api would attest to
bytes it didn't build) and create a snapshot-vs-live race the moment
welder ships a new release out-of-band. The proper fix is a real
release workflow in the welder repo, which is materially more work
than this PR. Captured as a follow-up.

Out of scope

  • verify_sc_tarball() logic in sc.sh — Phase 2c is fine, this is
    publishing-side only.
  • Mandating cosign on consumer machines — keeps the same soft-fail
    contract the in-script verifier already uses.

Test plan

  • CI on this PR passes (no behaviour change pre-merge; the new step only fires on push to main).
  • After merge, the first push.yaml run produces https://dist.simple-container.com/sc.sh.cosign-bundle (HTTP 200).
  • The post-merge smoke test above returns Verified OK on clean and exits non-zero after a tamper byte.
  • Follow-up issue / ClickUp item filed for welder release-workflow + welder.sh signing.

…ontainer.com

Phase 2c verifies the *tarball* sc.sh downloads, but a bucket-write or
CDN-swap attacker can replace sc.sh with a version whose
verify_sc_tarball is stripped or hard-coded to succeed. The in-script
protection regresses silently for every consumer running
`bash <(curl ... sc.sh)`.

This wires push.yaml's docker-finalize job to sign the post-template
sc.sh (the bytes that actually land at dist.simple-container.com) under
the same keyless cosign identity already used for the tarballs:
push.yaml@refs/heads/main. The resulting sc.sh.cosign-bundle is dropped
next to sc.sh in the dist bundle dir and shipped by the existing
`welder deploy -e prod` step.

Hard-fail by design: if `cosign sign-blob` fails, the deploy step never
runs, so consumers never observe an sc.sh whose bundle is missing or
points at a previous release. Atomic w.r.t. the script bytes.

docs/SECURITY.md gets a parallel "Verifying sc.sh itself" section
mirroring "Verifying tarballs", with a copy-pastable curl + verify-blob
+ bash snippet against the same regex.

Welder.sh (welder.simple-container.com) is out of scope: welder has no
GitHub Actions today and ships from a local mage flow, so giving it a
real trust root needs its own scoped PR. Signing welder.sh from
api/push.yaml would smear the trust root (api would attest to bytes
it didn't build).

Signed-off-by: Dmitrii Creed <creeed22@gmail.com>
@Cre-eD
Cre-eD requested a review from smecsia as a code owner May 24, 2026 14:40
@github-actions

github-actions Bot commented May 24, 2026

Copy link
Copy Markdown

Semgrep Scan Results

Repository: api | Commit: b57401a

Check Status Details
⚠️ Semgrep Warning 10 warning(s), 10 total

Scanned at 2026-05-24 15:37 UTC

@github-actions

github-actions Bot commented May 24, 2026

Copy link
Copy Markdown

Security Scan Results

Repository: api | Commit: b57401a

Check Status Details
✅ Secret Scan Pass No secrets detected
✅ Dependencies (Trivy) Pass 0 total (no critical/high)
✅ Dependencies (Grype) Pass 0 total (no critical/high)
📦 SBOM Generated 528 components (CycloneDX)

Scanned at 2026-05-24 15:38 UTC

@Cre-eD
Cre-eD merged commit 6884554 into main May 24, 2026
21 checks passed
@Cre-eD
Cre-eD deleted the feat/sign-sc-sh branch May 24, 2026 15:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants