Update DSV4 GB300 8k1k MTP disagg configs#1529
Conversation
|
Thanks for the contribution! For vLLM & SGLang, please ensure that your recipes is similar to the official vLLM recipes and/or the SGLang cookbook If it is not, please create a PR first before we can merge your single node PR into the master branch. Let's ensure that the documentation is first class such that the entire ML community can benefit from your hard work! Thank you PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. If re-running failed jobs is attempted, PR authors are responsible for ensuring it passes. See GitHub's docs on re-running failed jobs: https://docs.github.com/en/actions/how-tos/manage-workflow-runs/re-run-workflows-and-jobs#re-running-failed-jobs-in-a-workflow As a rule of thumb, generally, PR authors should request a review & get a PR approval from the respective companies' CODEOWNERS before requesting a review from core maintainers. If additional help is needed, PR authors can reach out to core maintainers over Slack. |
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
| @@ -1,4 +1,4 @@ | |||
| name: "dsv4-pro-gb300-disagg-8k1k-mid-curve-1p1d-dep4-dep8-mtp" | |||
| name: "dsv4-pro-gb300-disagg-8k1k-mid-curve-8p1d-dep4-dep16-mtp" | |||
There was a problem hiding this comment.
🔴 The dep8 → dep16 mid-curve recipes were renamed in this PR but .github/configs/nvidia-master.yaml still references the old paths at lines 9128, 9158, and 9173 (disagg-mid-curve-{1p1d,2p1d,4p1d}-dep4-dep8-mtp.yaml). After merge those CONFIG_FILE entries are dangling and matrix sweeps that resolve them will fail with file-not-found; the orchestrator entries also hard-code decode tp/ep=8 and conc-lists [256]/[512]/[1024], which disagree with the new dep16 recipes (tp/ep=16, concurrencies 512/1024/3072). The newly added disagg-mid-curve-12p1d-dep4-dep12-mtp.yaml is also not registered anywhere in nvidia-master.yaml.
Extended reasoning...
What the bug is
This PR renames three mid-curve recipe files (git -M detected R090/R095/R095 renames) and adds a new 12p1d-dep12 recipe:
disagg-mid-curve-1p1d-dep4-dep8-mtp.yaml→disagg-mid-curve-8p1d-dep4-dep16-mtp.yaml(also bumps prefill from 1 to 8 nodes)disagg-mid-curve-2p1d-dep4-dep8-mtp.yaml→disagg-mid-curve-2p1d-dep4-dep16-mtp.yamldisagg-mid-curve-4p1d-dep4-dep8-mtp.yaml→disagg-mid-curve-4p1d-dep4-dep16-mtp.yaml- new:
disagg-mid-curve-12p1d-dep4-dep12-mtp.yaml
However .github/configs/nvidia-master.yaml is untouched and still references the three old dep8 filenames. AGENTS.md explicitly codifies the rule: Multi-node srt-slurm changes must edit the recipe yaml AND nvidia-master.yaml together — this PR is recipe-only.
Why existing code does not prevent it
CONFIG_FILE is a string passed through via additional-settings and resolved at sweep time by the launcher (runners/launch_gb300-*.sh) when it does srtctl apply -f "$CONFIG_FILE" against the cloned srt-slurm repo (overlaid by cp -rT of the local recipes). There is no compile-time check that the path exists in the local mirror, so the orchestrator YAML happily ships dangling references.
Impact
Two failure modes:
- File-not-found after merge. The local mirror (
benchmarks/multi_node/srt-slurm-recipes/sglang/deepseek-v4/8k1k/) no longer contains anydep4-dep8-mtpfiles, so the three matrix entries atnvidia-master.yaml:9120,:9149,:9164will fail to resolve theirCONFIG_FILEonce merged. The dsv4-fp4-gb300-dynamo-sglang-mtp matrix sweeps break. - Topology disagreement. Even if the paths are repaired, the per-entry orchestrator topology (
decode.tp:8,decode.ep:8,conc-list:[256]/[512]/[1024]) was calibrated for the deleted dep8 recipes. The new dep16 files specifytp:16, ep:16and runconcurrencies: 3072 / 1024 / 512— so the orchestrator-level sweep parameters are now mislabeled relative to the recipe content, which (perAGENTS.md) causes results to be mislabeled.
The new disagg-mid-curve-12p1d-dep4-dep12-mtp.yaml has no nvidia-master.yaml entry at all, so it is orphaned — only the non-MTP 12p1d variant (disagg-gb300-12p1d-dep4-dep12-15-c21504.yaml) is registered.
Step-by-step proof
git log --diff-filter=R HEADon this PR shows the three filesystem renames (R090, R095, R095).ls benchmarks/multi_node/srt-slurm-recipes/sglang/deepseek-v4/8k1k/ | grep dep8returns nomid-curvefiles — the dep8 mid-curve recipes are gone.- Reading
.github/configs/nvidia-master.yamlat the dsv4-fp4-gb300-dynamo-sglang-mtp scenario block shows three matrix entries that still embed the old paths:- L9128:
CONFIG_FILE=recipes/sglang/deepseek-v4/8k1k/disagg-mid-curve-1p1d-dep4-dep8-mtp.yaml - L9158:
CONFIG_FILE=recipes/sglang/deepseek-v4/8k1k/disagg-mid-curve-2p1d-dep4-dep8-mtp.yaml - L9173:
CONFIG_FILE=recipes/sglang/deepseek-v4/8k1k/disagg-mid-curve-4p1d-dep4-dep8-mtp.yaml
- L9128:
grep -r 'mid-curve-12p1d-dep4-dep12-mtp' .github/returns no hits — the new file is not registered.- The matrix entry at L9120-9133 sets
prefill.num-worker: 1, decode.tp: 8, decode.ep: 8, conc-list: [256], while the corresponding renamed file (disagg-mid-curve-8p1d-dep4-dep16-mtp.yaml) setsprefill_nodes: 8,tensor-parallel-size: 16,expert-parallel-size: 16,concurrencies: "3072"— confirming topology divergence.
How to fix
Update .github/configs/nvidia-master.yaml in this PR:
- L9120-9133 entry: rename the comment to
8p1d-dep4-dep16, fixCONFIG_FILEtodisagg-mid-curve-8p1d-dep4-dep16-mtp.yaml, bumpprefill.num-worker: 1 → 8,decode.tp/ep: 8 → 16,conc-list: [256] → [3072]. - L9149-9163 entry (2p1d): point at
-dep16-, bumpdecode.tp/ep: 8 → 16,conc-list: [512] → [512](matches new file). - L9164-9178 entry (4p1d): point at
-dep16-, bumpdecode.tp/ep: 8 → 16,conc-list: [1024] → [1024](matches new file). - Add a new matrix entry for the 12p1d-dep4-dep12 MTP recipe alongside the existing non-MTP 12p1d entry.
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=26171040711 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=26171500710 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=26171500710 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=26175909007 |
2 similar comments
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=26175909007 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=26175909007 |
- Add dep16 mid-curve configs (2p1d, 4p1d, 8p1d) and 12p1d-dep4-dep12 - Update 1p6d low-latency dynamo hash - Remove dep8 mid-curve configs (replaced by dep16)
yhyang201
force-pushed
the
update-dsv4-gb300-8k1k-mtp-configs
branch
from
d2d1535 to
8880ed0
Compare
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=26298935253 |
1 similar comment
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=26298935253 |
1 participant
Summary
Reference: https://github.com/elvischenv/srt-slurm/tree/dsv4-gb300-disagg-8k1k-mtp-2/recipes/dsv4-pro/sglang/gb300-fp4/8k1k/disagg/mtp