Skip to content

[https://nvbugs/6507114][fix] Extend the per-model qwen3.5_moe_35b.yaml with…#16868

Open
trtllm-agent wants to merge 1 commit into
NVIDIA:mainfrom
tensorrt-cicd:repair-bot-bug6507114
Open

[https://nvbugs/6507114][fix] Extend the per-model qwen3.5_moe_35b.yaml with…#16868
trtllm-agent wants to merge 1 commit into
NVIDIA:mainfrom
tensorrt-cicd:repair-bot-bug6507114

Conversation

@trtllm-agent

@trtllm-agent trtllm-agent commented Jul 25, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • Root cause: Two independent AutoDeploy compile-pipeline defects stack on this test: multi_stream_moe/gemm stall the SM90 TP=4 GSM8K "Fetching responses" loop, and default piecewise CUDA-graph capture silently kills a worker on the first multi-image MMMU prefill for the Qwen3_5MoeForConditionalGeneration VLM.
  • Fix: Extend the per-model qwen3.5_moe_35b.yaml with compile_model.piecewise_enabled: false alongside the multi_stream_moe/gemm disables (scoped to this model config only, matching super_v3.yaml/super_v3_mtp.yaml precedents); remove the stale waives.txt entry.
  • Automated fix generated by repair-bot

Test plan

  • Verify fix on the same GPU type as the original failure
  • Check for regressions in related tests

Links

Dev Engineer Review

  • Disabled piecewise CUDA-graph capture and multi-stream MoE/GEMM transforms for Qwen3.5-35B, addressing known GSM8K stalls and MMMU prefill worker termination.
  • Changes are scoped to the model configuration, retain stage: compile, and align with model-specific precedents.
  • Removed the stale Qwen waiver entry from waives.txt; formatting and ordering remain consistent.
  • No public API or code changes; no apparent regressions.

QA Engineer Review

  • Test-list-only change: removed accuracy/test_llm_api_autodeploy.py::TestQwen3_5_397B_MoE::test_bf16_small[4] from tests/integration/test_lists/waives.txt.
  • No test-db/ or qa/ files were modified.
  • End-to-end validation passed on 4×H100: MMLU 84.75, GSM8K 88.97, and MMMU 59.89.
  • Verdict: needs follow-up because CBTS coverage data is unavailable.

…5-35B AutoDeploy

TestQwen3_5_397B_MoE::test_bf16_small[4] runs MMLU + GSM8K + MMMU on
Qwen3.5-35B-A3B via AutoDeploy on 4xH100. The default per-model config
(qwen3.5_moe_35b.yaml) opts into two performance transforms that both
break for this smoke test:

1) multi_stream_moe / multi_stream_gemm overlap shared-expert compute
   and all-reduce onto an aux stream; on SM90 TP=4 this stalls GSM8K
   "Fetching responses" at 0/1319 (the original reproduce failure).

2) compile_model.piecewise_enabled (inherited from default.yaml) hits
   an untested edge case in the VLM prefill path -- MMMU multi-image
   submits kill a worker silently at 84/900 with no Python traceback
   or CUDA error. Piecewise + VLM was previously hardened by NVIDIA#14006
   (commit 19b5d0e) but that fix did not cover this combination
   (Qwen3_5MoeForConditionalGeneration + mrope 3D position_ids +
   fewshot_as_multiturn chat template).

Disable both in the per-model YAML, matching the existing precedent in
model_registry/configs/super_v3_mtp.yaml (piecewise_enabled: false for
MTP incompatibility) and super_v3.yaml (multi_stream_moe/gemm: false).
The disables are scoped to this model config only and do not affect
qwen3.5_moe_400b.yaml or other AutoDeploy consumers.

Remove the stale waive entry now that the test passes end-to-end
(MMLU 84.75, GSM8K 88.97, MMMU 59.89 in 558s on 4xH100).

Signed-off-by: handongl <handongl@nvidia.com>
@coderabbitai

coderabbitai Bot commented Jul 25, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 7255872f-9321-4630-a51f-a8a87778f9ba

📥 Commits

Reviewing files that changed from the base of the PR and between cf44a1c and a90d01c.

📒 Files selected for processing (2)
  • examples/auto_deploy/model_registry/configs/qwen3.5_moe_35b.yaml
  • tests/integration/test_lists/waives.txt
💤 Files with no reviewable changes (1)
  • tests/integration/test_lists/waives.txt

Walkthrough

The Qwen 3.5 MoE auto-deploy configuration disables selected compile features and adds a compile-model setting. Its waived BF16 integration test is removed from the integration waiver list.

Changes

Qwen compile and test updates

Layer / File(s) Summary
Update Qwen compile settings
examples/auto_deploy/model_registry/configs/qwen3.5_moe_35b.yaml
Disables multi-stream MoE/GEMM compilation and piecewise compilation for the Qwen 3.5 MoE configuration.
Remove Qwen integration waiver
tests/integration/test_lists/waives.txt
Removes the waiver for the Qwen 3.5 397B MoE BF16 small test.

Estimated code review effort: 1 (Trivial) | ~3 minutes

Possibly related PRs

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title is related to the main change, but it is truncated and could be more specific about the config and waived test cleanup.
Description check ✅ Passed The description explains the root cause, fix, and test plan, though it doesn't follow the template's exact section headings.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants