[None][feat] Enable MM encoder cache on Qwen3.x and Gemma4 VLMs

[2ez4bz]

Summary by CodeRabbit

• New Features

  • Added multimodal encoder caching support for Gemma 4, Qwen3-VL, Qwen3-VL MoE, and Qwen3.5 models.
  • Improved reuse of image and video embeddings across requests, reducing repeated encoding work.

• Documentation

  • Expanded the supported-models documentation to include the newly supported multimodal optimization models.

• Tests

  • Added coverage confirming cached multimodal embeddings are reused correctly.

Description

Test Coverage

PR Checklist

Please review the following before submitting your PR:

• PR description clearly explains what and why. If using CodeRabbit's summary, please make sure it makes sense.

• PR Follows TRT-LLM CODING GUIDELINES to the best of your knowledge.

• Test cases are provided for new code paths (see test instructions)

• If PR introduces API changes, an appropriate PR label is added - either api-compatible or api-breaking. For api-breaking, include BREAKING in the PR title.

• Any new dependencies have been scanned for license and vulnerabilities

• CODEOWNERS updated if ownership changes

• Documentation updated as needed

• Update tava architecture diagram if there is a significant design change in PR.

• The reviewers assigned automatically/manually are appropriate for the PR.

• Please check this after reviewing the above items as appropriate for this PR.

GitHub Bot Help

To see a list of available CI bot commands, please comment /bot help.

[coderabbitai]

📝 Walkthrough

Walkthrough

Gemma4 and Qwen multimodal model wrappers now use encoder-cache-aware embedding retrieval, expose multimodal embedding metadata, and declare cache support. Tests verify reuse across requests, while documentation lists the expanded supported model set.

Changes

Multimodal encoder cache support

Layer / File(s)	Summary
Gemma4 mixin and cache flow tensorrt_llm/_torch/models/modeling_gemma4mm.py, tests/unittest/_torch/modeling/test_gemma4_multimodal.py	Gemma4 adopts MultimodalModelMixin, routes encoding through _get_or_encode_multimodal_embeddings, exposes embedding metadata, and tests reuse of cached image embeddings.
Qwen cache-aware embedding routing tensorrt_llm/_torch/models/modeling_qwen3vl.py	Qwen3-VL adds cache-aware embedding selection, direct encoder invocation, embedding metadata properties, and encoder-cache activation.
Model support declarations and documentation tensorrt_llm/_torch/models/modeling_qwen3_5.py, tensorrt_llm/_torch/models/modeling_qwen3vl_moe.py, docs/source/models/supported-models.md	Qwen3.5 and Qwen3-VL MoE declare encoder-cache support, and the supported-models table lists the expanded optimization coverage.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Possibly related PRs

• NVIDIA/TensorRT-LLM#16554: Adds the encoder-cache-aware multimodal mixin and related encoder side-stream/cache integration used by this model-side adoption.

Suggested labels: api-compatible

Suggested reviewers: yechank-nvidia

Sequence Diagram(s)

sequenceDiagram
  participant Request
  participant MultimodalModel
  participant MultimodalModelMixin
  participant EncoderCache
  participant MultimodalEncoder
  Request->>MultimodalModel: image or video inputs
  MultimodalModel->>MultimodalModelMixin: _get_or_encode_multimodal_embeddings
  MultimodalModelMixin->>EncoderCache: lookup embeddings
  alt cache miss
    MultimodalModelMixin->>MultimodalModel: encode_multimodal_inputs
    MultimodalModel->>MultimodalEncoder: encode multimodal inputs
    MultimodalEncoder-->>MultimodalModelMixin: multimodal embeddings
    MultimodalModelMixin->>EncoderCache: store embeddings
  else cache hit
    EncoderCache-->>MultimodalModelMixin: cached embeddings
  end
  MultimodalModelMixin-->>MultimodalModel: embeddings

Loading

🚥 Pre-merge checks | ✅ 3 | ❌ 2

❌ Failed checks (2 warnings)

Check name	Status	Explanation	Resolution
Description check	⚠️ Warning	The description is mostly a filled template and lacks the required issue/solution summary and test coverage details.	Add a short Description explaining the change and motivation, plus specific Test Coverage entries and any relevant checklist notes.
Docstring Coverage	⚠️ Warning	Docstring coverage is 37.50% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (3 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly states the feature and matches the multimodal encoder-cache changes for Qwen3.x and Gemma4.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 1

🧹 Nitpick comments (1)

tests/unittest/_torch/modeling/test_gemma4_multimodal.py (1)

765-776: 📐 Maintainability & Code Quality | 🔵 Trivial | 🏗️ Heavy lift

Add production-path cache coverage for Qwen and Gemma.

Coverage is insufficient: this harness calls _get_or_encode_multimodal_embeddings() directly, so it does not exercise Gemma4 forward() or Qwen’s new _get_qwen_multimodal_embeddings() routing. Add cache-hit/miss tests to tests/unittest/_torch/modeling/test_gemma4_multimodal.py and the Qwen3-VL model test module, asserting two identical raw requests invoke the encoder once; parameterize the Qwen test across dense, MoE, and Qwen3.5 wrappers. Run the targeted tests under pytest tests/unittest/.

As per path instructions, “Keep feedback actionable: suggest concrete list file names and whether coverage is sufficient, insufficient, or needs follow-up outside the PR.”

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests/unittest/_torch/modeling/test_gemma4_multimodal.py` around lines 765 -
776, The cache test only exercises the helper directly and does not cover
production routing. Add cache miss/hit tests through
Gemma4ForConditionalGeneration.forward and Qwen’s
_get_qwen_multimodal_embeddings, asserting identical raw requests invoke the
encoder once; parameterize Qwen coverage across dense, MoE, and Qwen3.5
wrappers, and run the targeted tests under pytest tests/unittest/.

Source: Path instructions

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@docs/source/models/supported-models.md`:
- Around line 129-138: Add Gemma4ForConditionalGeneration to the
MultimodalModelMixin optimization matrix, mark Multimodal Embeddings Cache as
Yes, and set Multimodal Encoder Side Stream according to its verified support
status.

---

Nitpick comments:
In `@tests/unittest/_torch/modeling/test_gemma4_multimodal.py`:
- Around line 765-776: The cache test only exercises the helper directly and
does not cover production routing. Add cache miss/hit tests through
Gemma4ForConditionalGeneration.forward and Qwen’s
_get_qwen_multimodal_embeddings, asserting identical raw requests invoke the
encoder once; parameterize Qwen coverage across dense, MoE, and Qwen3.5
wrappers, and run the targeted tests under pytest tests/unittest/.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 2af008bf-08c0-455b-8e97-c6273f5a8e3d

📥 Commits

Reviewing files that changed from the base of the PR and between 4fb31cb and 7da572a.

📒 Files selected for processing (6)

• docs/source/models/supported-models.md
• tensorrt_llm/_torch/models/modeling_gemma4mm.py
• tensorrt_llm/_torch/models/modeling_qwen3_5.py
• tensorrt_llm/_torch/models/modeling_qwen3vl.py
• tensorrt_llm/_torch/models/modeling_qwen3vl_moe.py
• tests/unittest/_torch/modeling/test_gemma4_multimodal.py