Skip to content

Update Buildkite pipeline for the new JuliaGPU cluster#190

Open
maleadt wants to merge 1 commit into
LuxDL:mainfrom
maleadt:buildkite-queues
Open

Update Buildkite pipeline for the new JuliaGPU cluster#190
maleadt wants to merge 1 commit into
LuxDL:mainfrom
maleadt:buildkite-queues

Conversation

@maleadt

@maleadt maleadt commented Jun 12, 2026

Copy link
Copy Markdown

The JuliaGPU Buildkite agents have moved to a dedicated cluster, where the single juliagpu queue has been split into per-backend queues. Steps now select agents using queue: "cuda", queue: "rocm" or queue: "oneapi" instead of queue: "juliagpu" combined with a cuda/rocm/intel tag. The pipeline-upload trigger steps in pipeline.yml, which previously targeted the generic juliagpu queue without a backend tag, now run on the cuda queue. The commented-out AMDGPU group was updated as well.

This change only affects agent selection; the steps themselves are unchanged.

🤖 Generated with Claude Code

Switch to the per-backend queues of the new JuliaGPU cluster.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request updates the Buildkite pipeline configurations to use the new "cuda" and "rocm" queues instead of "juliagpu". The reviewer recommends using a lightweight CPU queue instead of the "cuda" queue for pipeline upload steps to avoid occupying valuable GPU resources for non-GPU tasks.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Comment thread .buildkite/pipeline.yml
if: build.branch != "main" && build.tag == null
agents:
queue: "juliagpu"
queue: "cuda"

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Running the pipeline upload step on the "cuda" queue occupies a GPU agent for a non-GPU task, which can lead to inefficient resource utilization and longer wait times for actual GPU tests. If a lightweight or default CPU queue is available in the new cluster, using it for pipeline uploads is highly recommended.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant