From a90d01cfd022dbfdef95f5c6a2081daf152044c8 Mon Sep 17 00:00:00 2001 From: handongl Date: Sat, 25 Jul 2026 03:17:16 -0700 Subject: [PATCH] [nvbugs/6507114][fix] Disable piecewise CG and multi_stream on Qwen3.5-35B AutoDeploy TestQwen3_5_397B_MoE::test_bf16_small[4] runs MMLU + GSM8K + MMMU on Qwen3.5-35B-A3B via AutoDeploy on 4xH100. The default per-model config (qwen3.5_moe_35b.yaml) opts into two performance transforms that both break for this smoke test: 1) multi_stream_moe / multi_stream_gemm overlap shared-expert compute and all-reduce onto an aux stream; on SM90 TP=4 this stalls GSM8K "Fetching responses" at 0/1319 (the original reproduce failure). 2) compile_model.piecewise_enabled (inherited from default.yaml) hits an untested edge case in the VLM prefill path -- MMMU multi-image submits kill a worker silently at 84/900 with no Python traceback or CUDA error. Piecewise + VLM was previously hardened by #14006 (commit 19b5d0ea77) but that fix did not cover this combination (Qwen3_5MoeForConditionalGeneration + mrope 3D position_ids + fewshot_as_multiturn chat template). Disable both in the per-model YAML, matching the existing precedent in model_registry/configs/super_v3_mtp.yaml (piecewise_enabled: false for MTP incompatibility) and super_v3.yaml (multi_stream_moe/gemm: false). The disables are scoped to this model config only and do not affect qwen3.5_moe_400b.yaml or other AutoDeploy consumers. Remove the stale waive entry now that the test passes end-to-end (MMLU 84.75, GSM8K 88.97, MMMU 59.89 in 558s on 4xH100). Signed-off-by: handongl --- .../model_registry/configs/qwen3.5_moe_35b.yaml | 7 +++++-- tests/integration/test_lists/waives.txt | 1 - 2 files changed, 5 insertions(+), 3 deletions(-) diff --git a/examples/auto_deploy/model_registry/configs/qwen3.5_moe_35b.yaml b/examples/auto_deploy/model_registry/configs/qwen3.5_moe_35b.yaml index 46498142a957..3ed30967a715 100644 --- a/examples/auto_deploy/model_registry/configs/qwen3.5_moe_35b.yaml +++ b/examples/auto_deploy/model_registry/configs/qwen3.5_moe_35b.yaml @@ -44,11 +44,14 @@ transforms: # "shared_expert_down_proj": "rowwise" # gating layer should be replicated as well # "gate": "gather" + compile_model: + # https://github.com/NVIDIA/TensorRT-LLM/issues/13858 + piecewise_enabled: false multi_stream_moe: stage: compile - enabled: true + enabled: false # https://github.com/NVIDIA/TensorRT-LLM/issues/12954 multi_stream_gemm: stage: compile - enabled: true + enabled: false # https://github.com/NVIDIA/TensorRT-LLM/issues/12954 gather_logits_before_lm_head: enabled: true diff --git a/tests/integration/test_lists/waives.txt b/tests/integration/test_lists/waives.txt index 6dc4776bb633..5e5a0a2da902 100644 --- a/tests/integration/test_lists/waives.txt +++ b/tests/integration/test_lists/waives.txt @@ -21,7 +21,6 @@ accuracy/test_disaggregated_serving.py::TestQwen3_30B_A3B::test_mixed_ctx_gen_mo accuracy/test_epd_disagg_multimodal.py::TestVideoMMEEPD::test_disaggregated_videomme[nemotron_nano_v3_omni_fp8] SKIP (https://nvbugs/6478692) accuracy/test_llm_api_autodeploy.py::TestNemotronNanoV3::test_accuracy[nvfp4-4-attn_dp_off-trtllm] SKIP (https://nvbugs/6367792) accuracy/test_llm_api_autodeploy.py::TestNemotronSuperV3::test_mtp[nvfp4_ws8_80gb-trtllm] SKIP (https://nvbugs/6450341) -accuracy/test_llm_api_autodeploy.py::TestQwen3_5_397B_MoE::test_bf16_small[4] SKIP (https://nvbugs/6507114) accuracy/test_llm_api_pytorch.py::TestDeepSeekR1::test_fp8_blockscale[throughput_mtp] SKIP (https://nvbugs/6428101) accuracy/test_llm_api_pytorch.py::TestDeepSeekR1::test_fp8_blockscale[throughput_mtp_trtllm] SKIP (https://nvbugs/6426868) accuracy/test_llm_api_pytorch.py::TestDeepSeekR1::test_nvfp4_multi_gpus[throughput_pp4_mtp] SKIP (https://nvbugs/6481323)