[GG] fix(concurrency): isolate graph state and gate B12X MoE overlap

[voipmonitor]

Summary

• default B12X PCIe oneshot all-reduce to stream-isolated channels
• bind DCP A2A pools to enclosing CUDA graph capture and isolate target/draft pools
• give target and speculative execution independent vLLM workspace lanes
• right-size per-stream oneshot channels from actual dispatch cutoffs
• query the selected MoE kernel capability before scheduling shared experts on an auxiliary CUDA stream
• enqueue overlap-safe shared-expert work before the routed MoE launch so a resident routed grid cannot block its submission
• preserve overlap only for barrier-free B12X native-NVFP4/W4A8 micro launches; serialize dynamic and W4A16 resident launches
• add regression coverage for channel ownership, workspace isolation, launch order, and MoE capability scheduling

Root causes

Target and MTP graph managers could alias B12X oneshot/DCP channel state and process-global vLLM workspace storage. These resources now follow graph ownership and target/speculator execution use independent workspace lanes.

A separate MoE scheduling hazard existed when shared-expert GEMM work was submitted around a routed resident-grid kernel. Resident-grid kernels require all participating CTAs to become resident. The scheduler now asks the concrete B12X launch plan whether auxiliary-stream overlap is safe, serializes unsafe plans, and submits safe shared-expert work before routed MoE.

The deterministic Xid 31 after a 250k prefill was a distinct disposable graph-pool cleanup bug and is fixed in #131. It should not be attributed to tight DCP IPC output allocation: B12X returns normal PyTorch output tensors and uses IPC only for internal staging/signals.

Validation

• focused combined vLLM lifecycle and launch-order tests: 5 passed
• existing MoE capability tests: 3 passed
• paired B12X planning/channel tests passed
• native A4 split decode is bit-identical to the fused path and graph replay is stable with changed inputs/routes
• exact native-allocator TP8/DCP2/MTP3 integration passed short and 250k-token requests with shared-expert streams enabled
• Ruff and git diff --check passed

Dependencies

Requires local-inference-lab/sparkinfer#47. For the full v19 stack, merge #131 as well.

[coderabbitai]

Warning

Review limit reached

@voipmonitor, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 18 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 48c10db5-79f9-4755-bc38-f16cae7b205f

📥 Commits

Reviewing files that changed from the base of the PR and between 35ce29c and 73d96e6.

📒 Files selected for processing (11)

• tests/distributed/test_b12x_fused_all_reduce.py
• tests/distributed/test_dcp_a2a.py
• tests/model_executor/layers/fused_moe/test_shared_experts_stream.py
• tests/quantization/test_nvfp4_nf3_hybrid.py
• vllm/distributed/device_communicators/custom_all_reduce.py
• vllm/distributed/parallel_state.py
• vllm/envs.py
• vllm/model_executor/layers/fused_moe/runner/moe_runner.py
• vllm/model_executor/layers/quantization/nvfp4_nf3_hybrid.py
• vllm/v1/attention/ops/dcp_alltoall.py
• vllm/v1/worker/gpu/model_runner.py

📝 Walkthrough

Walkthrough

Changes

B12X PCIe oneshot sizing

Layer / File(s)	Summary
Oneshot limits and runtime sizing vllm/envs.py, vllm/distributed/device_communicators/custom_all_reduce.py, tests/distributed/test_b12x_fused_all_reduce.py	The single-channel option now defaults to disabled, cutoff values are validated centrally, and the runtime receives a separate oneshot buffer size covered by tests.

B12X DCP A2A graph capture

Layer / File(s)	Summary
DCP pool capture integration vllm/v1/attention/ops/dcp_alltoall.py, vllm/distributed/parallel_state.py, tests/distributed/test_dcp_a2a.py	DCP A2A pools use independent channels, matching pools are captured for the active DCP group, and global graph capture enters that context with tests covering the flow.

Independent workspace lanes

Layer / File(s)	Summary
Lane-aware workspace storage vllm/v1/worker/workspace.py, tests/v1/worker/test_workspace.py	Workspace buffers are selected by ubatch and lane, with lane validation and allocation behavior covered by tests.
Speculative execution lane wiring vllm/v1/worker/gpu_worker.py, vllm/v1/worker/gpu/model_runner.py, vllm/v1/worker/gpu/warmup.py	Eligible speculative V2 workers initialize two lanes, and speculator setup, warmup, capture, proposal, and capacity operations execute in lane one.

Estimated code review effort: 4 (Complex) | ~45 minutes

Sequence Diagram(s)

sequenceDiagram
  participant Speculator
  participant use_workspace_lane
  participant WorkspaceManager
  participant Worker
  Speculator->>use_workspace_lane: select workspace lane 1
  use_workspace_lane->>WorkspaceManager: scope allocation
  WorkspaceManager-->>Speculator: return ubatch/lane workspace
  Worker->>Speculator: run setup, warmup, capture, or sampling

Loading

Possibly related PRs

• local-inference-lab/vllm#130: Updates the related DCP A2A capture-active handling and packed-buffer lifetime tests.

Suggested reviewers: lukealonso

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 35.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title is concise and accurately summarizes the main concurrency isolation and B12X MoE overlap gating changes.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

🧹 Nitpick comments (1)

vllm/v1/attention/ops/dcp_alltoall.py (1)

154-174: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Add docstring arguments and improve type hint.

The type hint for stream is overly generic (object); using torch.cuda.Stream is more precise. Also, the docstring is missing the argument descriptions. As per coding guidelines, use Google-style docstrings in Python code, with Args:/Returns:/Raises: sections.

💡 Proposed fix

 `@contextmanager`
 def capture_b12x_dcp_a2a(
     cp_group: GroupCoordinator,
-    stream: object = None,
+    stream: torch.cuda.Stream | None = None,
 ):
-    """Bind each CUDA graph manager to independent B12X DCP channels."""
+    """Bind each CUDA graph manager to independent B12X DCP channels.
+
+    Args:
+        cp_group: The control-plane group coordinator to match.
+        stream: The CUDA stream to use for capturing.
+    """
     group_id = id(cp_group.device_group)
     matching_pools = sorted(

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@vllm/v1/attention/ops/dcp_alltoall.py` around lines 154 - 174, Update
capture_b12x_dcp_a2a to type stream as torch.cuda.Stream, preserving its
optional default, and expand the docstring with a Google-style Args section
describing cp_group and stream. Add Returns or Raises sections only if
applicable to the existing context-manager behavior.

Source: Coding guidelines

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@vllm/v1/attention/ops/dcp_alltoall.py`:
- Around line 154-174: Update capture_b12x_dcp_a2a to type stream as
torch.cuda.Stream, preserving its optional default, and expand the docstring
with a Google-style Args section describing cp_group and stream. Add Returns or
Raises sections only if applicable to the existing context-manager behavior.

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 2e710e54-84df-4178-be8f-c95bd609dafb

📥 Commits

Reviewing files that changed from the base of the PR and between 371085e and 6a2a043.

📒 Files selected for processing (6)

• tests/distributed/test_b12x_fused_all_reduce.py
• tests/distributed/test_dcp_a2a.py
• vllm/distributed/device_communicators/custom_all_reduce.py
• vllm/distributed/parallel_state.py
• vllm/envs.py
• vllm/v1/attention/ops/dcp_alltoall.py

[yatesdr]

@voipmonitor I opened dependent follow-up #134 for the hybrid-specific gap in the shared-expert overlap gate. NvFp4Nf3HybridMoEMethod uses resident-grid barriers in both its unified Grid188 and per-tier fallback paths, so it must conservatively reject auxiliary-stream shared-expert overlap. The focused add-on is commit 59730555; please fold/cherry-pick it into this concurrency stack if you agree. #134 remains draft pending the target-GPU gates.

[voipmonitor]

Folded the hybrid resident-grid protection from #134 into this stack (99695d4), added its parametrized regression coverage, and addressed the capture-stream type/docstring review note (3214348). Combined focused validation: 71 relevant tests passed, plus the TP4 backend-config gate passed with four visible GPUs.