Skip to content

[#16792][fix] Preserve Qwen3.5-VL multimodal content order#16845

Open
fallintoplace wants to merge 1 commit into
NVIDIA:mainfrom
fallintoplace:fix-qwen35-vl-placeholder-order
Open

[#16792][fix] Preserve Qwen3.5-VL multimodal content order#16845
fallintoplace wants to merge 1 commit into
NVIDIA:mainfrom
fallintoplace:fix-qwen35-vl-placeholder-order

Conversation

@fallintoplace

@fallintoplace fallintoplace commented Jul 24, 2026

Copy link
Copy Markdown

Description

Qwen3.5-VL uses string chat content, but its registration did not opt into the serving path that preserves ordered OpenAI content parts. Requests containing alternating text and images therefore bulk-prepended all vision placeholders, changing prompt tokenization and multimodal positions.

Enable the existing interleaved-placeholder path for the shared dense and MoE metadata. Requests without ordered content parts keep the existing BEFORE_TEXT fallback.

This addresses the placeholder-ordering part of #16792. The separate residual generation parity case described there remains out of scope.

Test Coverage

  • Extended the dense and MoE placeholder-registration tests to cover the interleaving contract.
  • pre-commit run --files tensorrt_llm/_torch/models/modeling_qwen3_5.py tests/unittest/_torch/modeling/test_modeling_qwen3_5_vl.py tests/unittest/_torch/modeling/test_modeling_qwen3_5_vl_moe.py
  • python3 -m compileall -q tensorrt_llm/_torch/models/modeling_qwen3_5.py tests/unittest/_torch/modeling/test_modeling_qwen3_5_vl.py tests/unittest/_torch/modeling/test_modeling_qwen3_5_vl_moe.py

PR Checklist

  • Reviewed the checklist in the pull request template as applicable.

Dev Engineer Review

  • Enabled interleaved placeholder registration for shared Qwen3.5-VL dense and MoE metadata.
  • Configuration is consistent with the intended OpenAI content-part ordering behavior and preserves the existing fallback path.
  • Added metadata assertions for both variants. No apparent API, performance, or error-handling regressions.
  • Residual generation parity remains out of scope.

QA Engineer Review

  • Modified placeholder-metadata tests for qwen3_5 and qwen3_5_moe, adding assertions that interleave_placeholders is enabled.
  • No corresponding test-db/ or qa/ coverage entries were identified.
  • Verdict: needs follow-up — CI/manual coverage data is unavailable.

Opt the dense and MoE Qwen3.5-VL registrations into the existing interleaved placeholder path so OpenAI content-part ordering is retained.

Signed-off-by: Minh Vu <vuhoangminh97@gmail.com>
@fallintoplace
fallintoplace requested a review from a team as a code owner July 24, 2026 18:27
@fallintoplace
fallintoplace requested a review from QiJune July 24, 2026 18:27
@coderabbitai

coderabbitai Bot commented Jul 24, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: ad2b34e9-e38c-4ef6-91c9-79994b136666

📥 Commits

Reviewing files that changed from the base of the PR and between f5c2a07 and 50864ce.

📒 Files selected for processing (3)
  • tensorrt_llm/_torch/models/modeling_qwen3_5.py
  • tests/unittest/_torch/modeling/test_modeling_qwen3_5_vl.py
  • tests/unittest/_torch/modeling/test_modeling_qwen3_5_vl_moe.py

Walkthrough

Qwen3.5 VLM placeholder metadata now enables interleaving for multimodal inputs. Registration tests for the dense and MoE variants verify the new metadata setting.

Changes

Qwen3.5 placeholder metadata

Layer / File(s) Summary
Enable and verify placeholder interleaving
tensorrt_llm/_torch/models/modeling_qwen3_5.py, tests/unittest/_torch/modeling/test_modeling_qwen3_5_vl.py, tests/unittest/_torch/modeling/test_modeling_qwen3_5_vl_moe.py
Shared Qwen3.5 VLM metadata sets interleave_placeholders=True, with registration tests covering the dense and MoE configurations.

Estimated code review effort: 1 (Trivial) | ~3 minutes

Suggested reviewers: qijune

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title matches the PR's main change and follows the required [issue][type] summary format.
Description check ✅ Passed The description includes the issue, solution, test coverage, and PR checklist sections required by the template.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant