Enable Intel XPU via accelerator auto-detect (no per-guide duplication) by WenjiaoYue · Pull Request #1514 · llm-d/llm-d-benchmark

WenjiaoYue · 2026-06-17T15:50:50Z

Summary

Enables the benchmark guides to run on Intel XPU by extending the existing
accelerator auto-detect, instead of duplicating each guide into an -xpu
variant (per review feedback on this PR).

What changed

Auto-detect maps device-plugin resources to accelerator profiles in
cluster_resource_resolver.py:
- nvidia.com/gpu → nvidia
- amd.com/gpu → amd
- habana.ai/gaudi → intel-gaudi
- google.com/tpu → google
- gpu.intel.com/xe / gpu.intel.com/i915 / intel.com/gpu
  → intel-xe / intel-i915 (with a resolution priority for the Intel resources)
Single shared overlay config/templates/values/overlays/intel-xpu.yaml
holds the XPU-specific settings (vLLM XPU image, dtype/exec/block-size/memory
args, security context, storage and resource sizing). render_plans applies
the detected profile's overlay so image and command overrides are selected
automatically.
Guides are now accelerator-neutral (optimized-baseline,
pd-disaggregation, precise-prefix-cache-routing) — they run on XPU with
no per-guide copies. The previous precise-prefix-cache-routing-xpu
scenario/spec have been removed.
Cluster scan is skipped for the nok8s method.
Added tests/test_accelerator_profiles.py covering profile detection and
rendering across the three guides.

Result

A cluster exposing an Intel GPU device-plugin resource is detected
automatically and the matching profile is applied — full Intel XPU support
with zero guide duplication.

github-actions · 2026-06-17T15:51:09Z

Unsigned commits detected! Please sign your commits.

For instructions on how to set up GPG/SSH signing and verify your commits, please see GitHub Documentation.

Copilot

Pull request overview

Adds an Intel XPU (intel-xe device-plugin) variant of the precise-prefix-cache-routing benchmark configuration so it can be rendered and run using the existing template/spec pipeline.

Changes:

Added an Intel XPU scenario (accelerator.type=intel-xe, XPU vLLM image, Qwen3-0.6B, eager mode) with decode replicas set to 2 for prefix-hit routing across endpoints.
Added a matching specification Jinja template that points to the new XPU scenario file.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File	Description
config/specification/guides/precise-prefix-cache-routing-xpu.yaml.j2	New spec template wiring the XPU guide to defaults + template dir + XPU scenario.
config/scenarios/guides/precise-prefix-cache-routing-xpu.yaml	New Intel XPU scenario defining model, routing plugins, decode command, resources, storage mode, and kv-events publishing for precise prefix cache routing.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

maugustosilva · 2026-06-17T17:46:35Z

Hello @WenjiaoYue. I would like to explore the possibility of extending the "accelerator auto-detect" to include Intel XPU's. Ideally, I would like to avoid the need for replicating each guide for this particular accelerator (while maintaining full support for it, of course!)

github-actions · 2026-07-09T01:39:04Z

This PR is marked as stale after 21d of inactivity. After an additional 14d of inactivity (7d to become rotten, then 7d more), it will be closed. To prevent this PR from being closed, add a comment or remove the lifecycle/stale label.

Address review feedback on llm-d#1514 (maugustosilva, kalantar): rather than copying each guide into an -xpu variant, extend the accelerator auto-detect path so Intel XPU (i915/Xe device-plugins) is selected automatically and shares the canonical guides. - cluster_resource_resolver: map device-plugin resources to accelerator profiles (nvidia/amd/intel-gaudi/google/intel-i915/intel-xe) and add Intel XPU resource priority; accept an explicit profile/kubeconfig. - render_plans + config_schema: resolve and apply the accelerator profile overlay so image, security-context and command overrides are selected per profile. - values/overlays/intel-xpu.yaml: shared Intel XPU machine profile (vLLM XPU image, dtype/exec/memory/block-size args, storage, resource sizing). - Guides (optimized-baseline, pd-disaggregation, precise-prefix-cache-routing): use accelerator-neutral substitution points instead of duplicated XPU files. - step_02_admin_prerequisites: gateway CRD handling for the XPU path. - tests: accelerator profile rendering coverage. Signed-off-by: WenjiaoYue <wenjiao.yue@intel.com>

WenjiaoYue · 2026-07-16T10:15:11Z

I would like to explore the possibility of extending the "accelerator auto-detect" to include Intel XPU's. Ideally, I would like to avoid the need for replicating each guide for this particular accelerator (while maintaining full support for it, of course!)

Hi @maugustosilva Thanks for your suggestion. I've updated the PR to use auto-detection instead of duplicating the guides, as you suggested. Intel device-plugin resources are now detected and mapped to the corresponding intel-xe or intel-i915 profile, with the XPU-specific settings kept in a single shared overlay. This allows the existing guides to work on Intel XPU without requiring per-guide copies.

I've also closed the other two PRs, as their changes are now covered by this one. I've tested the updated implementation, and everything works as expected with no issues.

Please take a look when you have time, and let me know if you'd prefer a different approach.

Copilot

Pull request overview

Copilot reviewed 13 out of 13 changed files in this pull request and generated 2 comments.

Copilot

Pull request overview

Copilot reviewed 13 out of 13 changed files in this pull request and generated 2 comments.

WenjiaoYue · 2026-07-20T11:44:06Z

+    # The cluster resolver deliberately removes both fields when a device
+    # resource is available but no portable SKU label exists. In that case
+    # Kubernetes schedules from the accelerator resource request alone.
+    labelKey: str | None = None
+    labelValue: str | None = None


Fixed in 616701e. Standalone node affinity and affinity metadata now render only when non-empty accelerator label data exists; otherwise scheduling relies on the accelerator resource request. Added an end-to-end standalone render regression test for the no-SKU-label case.

WenjiaoYue · 2026-07-20T11:44:04Z

+          ${accelerator.dtypeArgs} \
+          ${accelerator.executionArgs} \
          --prefix-caching-hash-algo sha256_cbor \
          --kv-events-config '{"enable_kv_cache_events":true, "publisher":"zmq", "endpoint":"$(KV_EVENTS_ENDPOINT)", "topic":"kv@$(POD_IP):$(POD_PORT)@$(MODEL_NAME)"}' \


Fixed in 616701e. Replaced the undefined POD_PORT with the injected VLLM_INFERENCE_PORT and added a regression assertion for the rendered precise-routing command.

Address review feedback on llm-d#1514 (maugustosilva, kalantar): rather than copying each guide into an -xpu variant, extend the accelerator auto-detect path so Intel XPU (i915/Xe device-plugins) is selected automatically and shares the canonical guides. - cluster_resource_resolver: map device-plugin resources to accelerator profiles (nvidia/amd/intel-gaudi/google/intel-i915/intel-xe) and add Intel XPU resource priority; accept an explicit profile/kubeconfig. - render_plans + config_schema: resolve and apply the accelerator profile overlay so image, security-context and command overrides are selected per profile. - values/overlays/intel-xpu.yaml: shared Intel XPU machine profile (vLLM XPU image, dtype/exec/memory/block-size args, storage, resource sizing). - Guides (optimized-baseline, pd-disaggregation, precise-prefix-cache-routing): use accelerator-neutral substitution points instead of duplicated XPU files. - step_02_admin_prerequisites: gateway CRD handling for the XPU path. - tests: accelerator profile rendering coverage. Signed-off-by: WenjiaoYue <wenjiao.yue@intel.com>

…ache-routing-xpu

Signed-off-by: WenjiaoYue <wenjiao.yue@intel.com>

Copilot AI review requested due to automatic review settings June 17, 2026 15:50

WenjiaoYue requested review from Vezio, achandrasekar, kalantar, maugustosilva, mengmeiye and namasl as code owners June 17, 2026 15:50

Copilot started reviewing on behalf of WenjiaoYue June 17, 2026 15:51 View session

Copilot AI reviewed Jun 17, 2026

View reviewed changes

Comment thread config/scenarios/guides/precise-prefix-cache-routing-xpu.yaml Outdated

kalantar mentioned this pull request Jun 22, 2026

avoid duplicating env var from accelerator types llm-d-incubation/llm-d-modelservice#262

Merged

github-actions Bot added the lifecycle/stale label Jul 9, 2026

WenjiaoYue force-pushed the feat/precise-prefix-cache-routing-xpu branch 2 times, most recently from 9ec7f75 to afe7874 Compare July 16, 2026 10:03

github-actions Bot removed the lifecycle/stale label Jul 17, 2026

WenjiaoYue changed the title ~~Enable precise-prefix-cache-routing benchmark on Intel XPU~~ Enable Intel XPU via accelerator auto-detect (no per-guide duplication) Jul 17, 2026

WenjiaoYue requested a review from Copilot July 17, 2026 01:47

Copilot started reviewing on behalf of WenjiaoYue July 17, 2026 01:47 View session

Copilot AI reviewed Jul 17, 2026

View reviewed changes

Comment thread tests/test_accelerator_profiles.py Outdated

Comment thread tests/test_accelerator_profiles.py Outdated

WenjiaoYue requested a review from Copilot July 20, 2026 09:49

Copilot started reviewing on behalf of WenjiaoYue July 20, 2026 09:50 View session

Copilot AI reviewed Jul 20, 2026

View reviewed changes

WenjiaoYue force-pushed the feat/precise-prefix-cache-routing-xpu branch from e77b751 to 43472b0 Compare July 20, 2026 10:45

WenjiaoYue added 3 commits July 20, 2026 03:47

Merge remote-tracking branch 'origin/main' into feat/precise-prefix-c…

1964d40

…ache-routing-xpu

test: select rendered accelerator stack deterministically

43472b0

Signed-off-by: WenjiaoYue <wenjiao.yue@intel.com>

fix: handle accelerator routing defaults safely

616701e

Signed-off-by: WenjiaoYue <wenjiao.yue@intel.com>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Enable Intel XPU via accelerator auto-detect (no per-guide duplication)#1514

Enable Intel XPU via accelerator auto-detect (no per-guide duplication)#1514
WenjiaoYue wants to merge 4 commits into
llm-d:mainfrom
WenjiaoYue:feat/precise-prefix-cache-routing-xpu

WenjiaoYue commented Jun 17, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Jun 17, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

maugustosilva commented Jun 17, 2026

Uh oh!

github-actions Bot commented Jul 9, 2026

Uh oh!

WenjiaoYue commented Jul 16, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

WenjiaoYue Jul 20, 2026

Uh oh!

WenjiaoYue Jul 20, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

WenjiaoYue commented Jun 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

What changed

Result

Uh oh!

github-actions Bot commented Jun 17, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

maugustosilva commented Jun 17, 2026

Uh oh!

github-actions Bot commented Jul 9, 2026

Uh oh!

WenjiaoYue commented Jul 16, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

WenjiaoYue Jul 20, 2026

Choose a reason for hiding this comment

Uh oh!

WenjiaoYue Jul 20, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

WenjiaoYue commented Jun 17, 2026 •

edited

Loading