Skip to content

ci: add R2-backed shared sccache#2855

Closed
UnArbosFive wants to merge 13 commits into
bittensor-core-explorationfrom
codex/subtensor-r2-sccache
Closed

ci: add R2-backed shared sccache#2855
UnArbosFive wants to merge 13 commits into
bittensor-core-explorationfrom
codex/subtensor-r2-sccache

Conversation

@UnArbosFive

Copy link
Copy Markdown
Contributor

Summary

  • use Firecracker MMDSv2 reader credentials for the shared Cloudflare R2 compiler cache on Fireactions turbo runners
  • retain GitHub Actions sccache as the legacy-runner fallback and ordinary Cargo as the fail-open path
  • add a protected main-only writer warmer using the sccache-writer environment
  • consolidate direct Rust compilation paths and retain Cargo registry/git caching without target tarballs where shared sccache applies

Depends on #2846 and intentionally targets its bittensor-core-exploration branch.

Security boundaries

  • pull requests receive only the bucket-scoped read credential from MMDS
  • writer credentials are available only through the protected sccache-writer environment on trusted main push/schedule/dispatch events
  • credentials are masked and never printed
  • R2 or MMDS failures disable the cache rather than failing compilation

Validation

  • .github/scripts/test-sccache-configure.sh
  • actionlint -config-file .github/actionlint.yaml
  • bash -n .github/scripts/sccache-configure.sh .github/scripts/test-sccache-configure.sh
  • git diff --check origin/pr-2846-head..HEAD

The R2 bucket, reader MMDS metadata, action allowlist, and protected writer environment are live. The BHS turbo canary proved reader GET succeeds and writer PUT is denied.

@vercel

vercel Bot commented Jul 11, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
subtensor Ready Ready Preview, Comment Jul 12, 2026 12:13am

Request Review

@github-actions

github-actions Bot commented Jul 11, 2026

Copy link
Copy Markdown
Contributor

🛡️ AI Review — Skeptic (security review)

VERDICT: SAFE

VERY HIGH account-age scrutiny, mitigated by admin repository permission; no Gittensor association or author/committer mismatch. Intentional stacked branch: codex/subtensor-r2-sccache → bittensor-core-exploration.

The writer workflow is restricted to trusted main events and its protected environment; ordinary PR jobs receive only the validated, bucket-scoped read credential. The trusted AI-review instruction files are unchanged.

Findings

No findings.

Conclusion

No malicious behavior or security vulnerability was identified. The shared-cache credential boundary remains fail-open without exposing a PR-accessible writer path.


🔍 AI Review — Auditor (domain review)

VERDICT: 👍

Gittensor association UNKNOWN; the author has limited public history but repository admin permission, so full correctness scrutiny was applied.

The PR is consistent with its description: R2 reader credentials are validated, writer credentials remain behind the protected main-only workflow, and cache failures degrade to GHA sccache or ordinary Cargo. No runtime code changed, and this base branch has no spec-version check.

The helper still permits writer mode on bittensor-core-exploration and codex/subtensor-r2-sccache, although no workflow supplies writer credentials there. This is a non-blocking description discrepancy and should be removed after rollout.

Overlapping PRs are stacked dependencies or separate CI experiments; none is a competing implementation.

Findings

No findings.

Conclusion

The shared-cache and multi-architecture build integration is coherent, fail-open, and adequately tested. No merge-blocking domain issues were found.

@github-actions

Copy link
Copy Markdown
Contributor

🔄 AI review updated — Skeptic: SAFE Auditor: 👍

@UnArbosFive

Copy link
Copy Markdown
Contributor Author

Live validation on fa-turbo is complete:

  • protected test-config-boundary workflow passes on Ubuntu
  • MMDS reader configuration validated
  • pinned sccache v0.15.0 started with SCCACHE_BACKEND=r2
  • endpoint, region=auto, TLS, and subtensor/v1 were exported
  • a short compile recorded 798 cache misses and 798 expected write denials under the read-only credential; compilation continued normally
  • no credentials appeared in the inspected logs

The runtime workflow was cancelled immediately after this proof, and all other active workflows from this PR were cancelled to free the self-hosted runners. Zero PR workflows are currently active.

@UnArbosFive

Copy link
Copy Markdown
Contributor Author

Temporary benchmark writer access is now included:

Merging this draft into bittensor-core-exploration will itself trigger the writer warmer and populate R2.

@github-actions github-actions Bot mentioned this pull request Jul 11, 2026
@UnArbosFive UnArbosFive marked this pull request as ready for review July 11, 2026 15:39
@github-actions

Copy link
Copy Markdown
Contributor

🔄 AI review updated — Skeptic: SAFE Auditor: 👍

@UnArbosFive

Copy link
Copy Markdown
Contributor Author

Final review and rollout status:

  • thermo-nuclear maintainability review: no structural or security blockers
  • latest bittensor-core: one Rust core (keys, keyfiles, timelock, codec, RFC-0078 digest, Ledger) under a thin Python SDK #2846 head re-fetched after rollout; still dcc180a99 and fully contained in this branch
  • writer warmer now asserts SCCACHE_BACKEND=r2 and the boundary test exercises writer activation
  • shared-cache changes now trigger real Check Rust, eco-tests, and Runtime Checks path filters
  • AX41 benchmarking now uses R2 sccache instead of target-directory tarballs
  • all 10 Fireactions hosts are converged: active service, config mode 0600, valid schema, and one read-only MMDS block per pool
  • final CI is running normally and has not been cancelled

PR is ready for review and merge. Merging into bittensor-core-exploration triggers the trusted writer warmer automatically.

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AI review — see the sticky summary comment for the verdict and the inline comments below for specific findings.

Comment thread .github/workflows/sccache-warm.yml Outdated
Comment on lines +9 to +11
# Temporary writer for this cache integration PR so it can populate R2
# before merging into #2846.
- codex/subtensor-r2-sccache

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[HIGH] PR head receives shared-cache writer execution

This enables the writer workflow directly on the branch whose code is still under review. Once the protected environment run is approved, this branch controls the checked-out local actions and all Cargo build scripts while writer credentials are present, so a hostile follow-up commit could exfiltrate the credentials or populate malicious artifacts in the cache consumed by other PRs. The hard-coded exception also remains reusable after merge. Remove the PR-head branch from this workflow and from prepare_writer’s allowlist; validate writer behavior only after merging into a protected trusted branch.

@github-actions

Copy link
Copy Markdown
Contributor

🔄 AI review updated — Skeptic: VULNERABLE

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AI review — see the sticky summary comment for the verdict and the inline comments below for specific findings.

Comment thread .github/workflows/sccache-warm.yml Outdated
- bittensor-core-exploration
# Temporary writer for this cache integration PR so it can populate R2
# before merging into #2846.
- codex/subtensor-r2-sccache

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[HIGH] PR head receives shared-cache writer execution

This still lets code under review execute with protected R2 writer credentials and populate the shared compiler cache. A hostile commit on this branch could exfiltrate the credentials or poison cached artifacts later consumed by ordinary PR jobs. Remove codex/subtensor-r2-sccache from this trigger, the job-level if, and prepare_writer in .github/scripts/sccache-configure.sh; only trusted post-review branches should receive writer access.

@github-actions

Copy link
Copy Markdown
Contributor

🔄 AI review updated — Skeptic: VULNERABLE

@github-actions

Copy link
Copy Markdown
Contributor

🔄 AI review updated — Skeptic: SAFE Auditor: 👍

@github-actions

Copy link
Copy Markdown
Contributor

🔄 AI review updated — Skeptic: SAFE Auditor: 👍

@github-actions

Copy link
Copy Markdown
Contributor

🔄 AI review updated — Skeptic: SAFE Auditor: 👍

@UnArbosFive

Copy link
Copy Markdown
Contributor Author

Superseded by #2857. The runner-density branch contains this PR’s complete cache commit history unchanged, followed by the runner-pool and try-runtime placement work. Consolidating on #2857 preserves the latest combined CI run and avoids another force-push/retest cycle. This branch is being retained for history until the unified PR lands.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant