Skip to content

[CI][ROCm] Make hf-xet reconstruction safe on shared NFS#49830

Closed
AndreasKaratzas wants to merge 1 commit into
mainfrom
agent/amd-nfs-xet-cache
Closed

[CI][ROCm] Make hf-xet reconstruction safe on shared NFS#49830
AndreasKaratzas wants to merge 1 commit into
mainfrom
agent/amd-nfs-xet-cache

Conversation

@AndreasKaratzas

Copy link
Copy Markdown
Member

Summary

  • Detect an NFS-backed HF_HOME in the AMD hardware test runner.
  • Keep Xet and the persistent Hub cache enabled while moving ephemeral Xet state to local storage and disabling high-performance vectored reconstruction only on NFS.

hf-xet 1.5.1 stalled on the large vectored write after the first tokenizer fragment, and later jobs convoyed behind the Hub lock. Non-NFS jobs are unchanged.

Validation

  • bash -n .buildkite/scripts/hardware_ci/run-amd-test.sh
  • Changed-file pre-commit passed.
  • Local syscall controls reconstructed Apeach and Qwen3-Next with scalar writes. Cold concurrent validation on Buildkite NFS remains pending.

Buildkite: Entrypoints Integration (Pooling)

Duplicate check: no open PR was found for hf-xet NFS.

AI assistance: Codex assisted with investigation, implementation, and validation; Andreas directed the design and reviewed the resulting changes.

Co-authored-by: OpenAI Codex <noreply@openai.com>
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
@mergify mergify Bot added ci/build rocm Related to AMD ROCm labels Jul 25, 2026
@github-project-automation github-project-automation Bot moved this to Todo in AMD Jul 25, 2026
@github-project-automation github-project-automation Bot moved this from Todo to Done in AMD Jul 25, 2026
@AndreasKaratzas
AndreasKaratzas deleted the agent/amd-nfs-xet-cache branch July 25, 2026 21:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci/build rocm Related to AMD ROCm

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

1 participant