[Klaud Cold] minimaxm2.5-fp8-mi300x: add SHUFFLE_KV_CACHE_LAYOUT=1 + ROCM_AITER_FA backend#1550
Conversation
…AYOUT=1 + --attention-backend ROCM_AITER_FA Adopts the AITER recipe pattern (shuffle KV-cache layout + AITER FA attention backend) used by the mi355x sibling. Matches AMD's recommended recipe for MiniMax-M2.x on ROCm vLLM. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
Thanks for the contribution! For vLLM & SGLang, please ensure that your recipes is similar to the official vLLM recipes and/or the SGLang cookbook If it is not, please create a PR first before we can merge your single node PR into the master branch. Let's ensure that the documentation is first class such that the entire ML community can benefit from your hard work! Thank you PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. If re-running failed jobs is attempted, PR authors are responsible for ensuring it passes. See GitHub's docs on re-running failed jobs: https://docs.github.com/en/actions/how-tos/manage-workflow-runs/re-run-workflows-and-jobs#re-running-failed-jobs-in-a-workflow As a rule of thumb, generally, PR authors should request a review & get a PR approval from the respective companies' CODEOWNERS before requesting a review from core maintainers. If additional help is needed, PR authors can reach out to core maintainers over Slack. |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=26242949679 |
There was a problem hiding this comment.
LGTM, straightforward benchmark config change adopting the AITER recipe pattern from the mi355x sibling.
Extended reasoning...
Overview
This PR makes a small, isolated benchmark config change to benchmarks/single_node/minimaxm2.5_fp8_mi300x.sh: adds the VLLM_ROCM_SHUFFLE_KV_CACHE_LAYOUT=1 env var alongside the existing VLLM_ROCM_USE_AITER=1, and passes --attention-backend "ROCM_AITER_FA" to vllm serve. It also adds a corresponding perf-changelog.yaml entry. The two-line code change mirrors the AITER recipe already in use on the mi355x sibling script.
Security risks
None. This is a benchmark recipe configuration touching only environment variables and a CLI flag in a single-node benchmark script. No auth, crypto, network exposure, or permission surface is involved.
Level of scrutiny
Low — this is a benchmark/perf recipe change in benchmarks/single_node/, not production-critical code. The change is mechanical (matches a documented pattern from the mi355x sibling), self-contained, and easy to revert. The full-sweep-enabled label means the actual correctness signal will come from the sweep run itself.
Other factors
Verified that the mi355x sibling script (benchmarks/single_node/minimaxm2.5_fp8_mi355x.sh) does indeed use both VLLM_ROCM_SHUFFLE_KV_CACHE_LAYOUT=1 and --attention-backend "ROCM_AITER_FA", so the AMD-recommended-recipe claim checks out. The bug hunting system flagged no issues, and the perf-changelog entry follows the established format with a valid PR link.
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=26242951096 |
Summary
Changes
benchmarks/single_node/minimaxm2.5_fp8_mi300x.sh+ export VLLM_ROCM_SHUFFLE_KV_CACHE_LAYOUT=1(next to existingVLLM_ROCM_USE_AITER=1)+ --attention-backend "ROCM_AITER_FA"on thevllm servelineperf-changelog.yamlentryTest plan
🤖 Generated with Claude Code