fix(ds4): use monolithic backend for single-device launches, not only for HIP#526
Open
Graffioh wants to merge 1 commit into
Open
fix(ds4): use monolithic backend for single-device launches, not only for HIP#526Graffioh wants to merge 1 commit into
Graffioh wants to merge 1 commit into
Conversation
5fc0d09 to
0171c69
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
DeepSeek4Backendon both CUDA and HIPLayerSplitBackendfor explicit multi-device placement or a configured remote target shardWhy
CUDA launches previously always selected
DeepSeek4LayerSplitAdapter. Without an explicit or remote split, its automatic sizing capped the local shard at 42 layers, so a single H200 executed a virtual[0, 42) + [42, 43)split on the same GPU.That virtual boundary adds split orchestration overhead and prevents the full-model fused decode path, whose eligibility requires the same range to start at layer 0 and own the output layer. A single local device should instead use the monolithic backend; devices that cannot fit the full model can use its existing hybrid-expert fallback.
Expected behavior
DeepSeek4BackendDeepSeek4BackendLayerSplitBackendLayerSplitBackendValidation
git diff --check