Skip to content

[Klaud Cold] minimaxm2.5-fp8-mi300x: add SHUFFLE_KV_CACHE_LAYOUT=1 + ROCM_AITER_FA backend#1550

Open
functionstackx wants to merge 2 commits into
mainfrom
klaud-cold/minimaxm2.5-fp8-mi300x-aiter-fa-shuffle-kv
Open

[Klaud Cold] minimaxm2.5-fp8-mi300x: add SHUFFLE_KV_CACHE_LAYOUT=1 + ROCM_AITER_FA backend#1550
functionstackx wants to merge 2 commits into
mainfrom
klaud-cold/minimaxm2.5-fp8-mi300x-aiter-fa-shuffle-kv

Conversation

@functionstackx
Copy link
Copy Markdown
Collaborator

Summary

  • Adopts the AITER recipe pattern (shuffle KV-cache layout + AITER FA attention backend) used by the mi355x sibling.
  • Matches the AMD-recommended recipe for MiniMax-M2.x on ROCm vLLM.

Changes

  • benchmarks/single_node/minimaxm2.5_fp8_mi300x.sh
    • + export VLLM_ROCM_SHUFFLE_KV_CACHE_LAYOUT=1 (next to existing VLLM_ROCM_USE_AITER=1)
    • + --attention-backend "ROCM_AITER_FA" on the vllm serve line
  • perf-changelog.yaml entry

Test plan

  • full-sweep-enabled sweep finishes green on mi300x.

🤖 Generated with Claude Code

functionstackx and others added 2 commits May 21, 2026 13:42
…AYOUT=1 + --attention-backend ROCM_AITER_FA

Adopts the AITER recipe pattern (shuffle KV-cache layout + AITER FA
attention backend) used by the mi355x sibling. Matches AMD's recommended
recipe for MiniMax-M2.x on ROCm vLLM.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@github-actions
Copy link
Copy Markdown
Contributor

Thanks for the contribution! For vLLM & SGLang, please ensure that your recipes is similar to the official vLLM recipes and/or the SGLang cookbook

If it is not, please create a PR first before we can merge your single node PR into the master branch. Let's ensure that the documentation is first class such that the entire ML community can benefit from your hard work! Thank you

PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. If re-running failed jobs is attempted, PR authors are responsible for ensuring it passes. See GitHub's docs on re-running failed jobs: https://docs.github.com/en/actions/how-tos/manage-workflow-runs/re-run-workflows-and-jobs#re-running-failed-jobs-in-a-workflow

As a rule of thumb, generally, PR authors should request a review & get a PR approval from the respective companies' CODEOWNERS before requesting a review from core maintainers.

If additional help is needed, PR authors can reach out to core maintainers over Slack.

@github-actions
Copy link
Copy Markdown
Contributor

Copy link
Copy Markdown
Contributor

@claude claude Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, straightforward benchmark config change adopting the AITER recipe pattern from the mi355x sibling.

Extended reasoning...

Overview

This PR makes a small, isolated benchmark config change to benchmarks/single_node/minimaxm2.5_fp8_mi300x.sh: adds the VLLM_ROCM_SHUFFLE_KV_CACHE_LAYOUT=1 env var alongside the existing VLLM_ROCM_USE_AITER=1, and passes --attention-backend "ROCM_AITER_FA" to vllm serve. It also adds a corresponding perf-changelog.yaml entry. The two-line code change mirrors the AITER recipe already in use on the mi355x sibling script.

Security risks

None. This is a benchmark recipe configuration touching only environment variables and a CLI flag in a single-node benchmark script. No auth, crypto, network exposure, or permission surface is involved.

Level of scrutiny

Low — this is a benchmark/perf recipe change in benchmarks/single_node/, not production-critical code. The change is mechanical (matches a documented pattern from the mi355x sibling), self-contained, and easy to revert. The full-sweep-enabled label means the actual correctness signal will come from the sweep run itself.

Other factors

Verified that the mi355x sibling script (benchmarks/single_node/minimaxm2.5_fp8_mi355x.sh) does indeed use both VLLM_ROCM_SHUFFLE_KV_CACHE_LAYOUT=1 and --attention-backend "ROCM_AITER_FA", so the AMD-recommended-recipe claim checks out. The bug hunting system flagged no issues, and the perf-changelog entry follows the established format with a valid PR link.

@github-actions
Copy link
Copy Markdown
Contributor

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

1 participant