Skip to content

fix(ds4): use monolithic backend for single-device launches, not only for HIP#526

Open
Graffioh wants to merge 1 commit into
Luce-Org:mainfrom
Graffioh:codex/deepseek4-monolithic-single-device
Open

fix(ds4): use monolithic backend for single-device launches, not only for HIP#526
Graffioh wants to merge 1 commit into
Luce-Org:mainfrom
Graffioh:codex/deepseek4-monolithic-single-device

Conversation

@Graffioh

@Graffioh Graffioh commented Jul 15, 2026

Copy link
Copy Markdown
Contributor

Summary

  • route DeepSeek 4 single-device launches through DeepSeek4Backend on both CUDA and HIP
  • reserve LayerSplitBackend for explicit multi-device placement or a configured remote target shard
  • preserve automatic CUDA/Halo layer splitting whenever remote target-shard IPC is enabled

Why

CUDA launches previously always selected DeepSeek4LayerSplitAdapter. Without an explicit or remote split, its automatic sizing capped the local shard at 42 layers, so a single H200 executed a virtual [0, 42) + [42, 43) split on the same GPU.

That virtual boundary adds split orchestration overhead and prevents the full-model fused decode path, whose eligibility requires the same range to start at layer 0 and own the output layer. A single local device should instead use the monolithic backend; devices that cannot fit the full model can use its existing hybrid-expert fallback.

Expected behavior

Configuration Backend Behavior
H200/Halo single DeepSeek4Backend Full monolithic
3090 single DeepSeek4Backend Full attempt → hybrid fallback
3090 + Halo IPC LayerSplitBackend Auto-split CUDA/Halo
Multi-GPU explicit LayerSplitBackend Layer split

Validation

@Graffioh Graffioh force-pushed the codex/deepseek4-monolithic-single-device branch from 5fc0d09 to 0171c69 Compare July 15, 2026 06:04
@Graffioh Graffioh changed the title Route single-device DeepSeek4 through monolithic backend fix(ds4): use monolithic backend for single-device launches Jul 15, 2026
@Graffioh Graffioh changed the title fix(ds4): use monolithic backend for single-device launches fix(ds4): use monolithic backend for single-device launches, not only for HIP Jul 15, 2026
@Graffioh Graffioh marked this pull request as ready for review July 15, 2026 06:59

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 1 file

Re-trigger cubic

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant