Skip to content
Open
2 changes: 2 additions & 0 deletions docs/source/models/supported-models.md
Original file line number Diff line number Diff line change
Expand Up @@ -165,6 +165,7 @@ For full documentation, see the [Visual Generation](./visual-generation.md) page
| `Lightricks/LTX-2` | Text-to-Video (with Audio), Image-to-Video (with Audio) |
| `Qwen/Qwen-Image` | Text-to-Image |
| `Qwen/Qwen-Image-2512` | Text-to-Image |
| `Qwen/Qwen-Image-Edit-2511` | Image Editing |

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure if we should name like text+image-to-image, and also some other models such as Flux also supports text+image-to-image.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe Image Editing (text+image-to-image)?

| `nvidia/Cosmos3-Nano` | Text-to-Image, Text-to-Video, Image-to-Video |
| `nvidia/Cosmos3-Super` | Text-to-Image, Text-to-Video, Image-to-Video |

Expand All @@ -178,6 +179,7 @@ For full documentation, see the [Visual Generation](./visual-generation.md) page
| **Wan 2.2** | Yes | Yes | No | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes |
| **LTX-2** | Yes | Yes | No | Yes | Yes | No | No | Yes | Yes | Yes | Yes | No |
| **Qwen-Image** [^2] | Yes | Yes | No | No | Yes | No | Yes | Yes | Yes | Yes | Yes | No |
| **Qwen-Image-Edit-2511** | Yes | Yes | No | Yes | No | No | Yes | Yes | Yes | No | No | No |
| **Cosmos3** | Yes | Yes | No | Yes | Yes | Yes | Yes | Yes | Yes | No | No | Yes |

[^vg1]: FLUX models use embedded guidance and do not have a separate negative prompt path, so CFG parallelism is not applicable.
2 changes: 2 additions & 0 deletions docs/source/models/visual-generation.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,7 @@ TensorRT-LLM **VisualGen** provides a unified inference stack for diffusion mode
| `Lightricks/LTX-2` | Text-to-Video (with Audio), Image-to-Video (with Audio) |
| `Qwen/Qwen-Image` | Text-to-Image |
| `Qwen/Qwen-Image-2512` | Text-to-Image |
| `Qwen/Qwen-Image-Edit-2511` | Image Editing |

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure if we should name like text+image-to-image, and also some other models such as Flux also supports text+image-to-image.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe Image Editing (text+image-to-image)?

| `nvidia/Cosmos3-Nano` | Text-to-Image, Text-to-Video, Image-to-Video |
| `nvidia/Cosmos3-Super` | Text-to-Image, Text-to-Video, Image-to-Video |

Expand All @@ -53,6 +54,7 @@ Models are auto-detected from the checkpoint directory. Diffusers-format models
| **Wan 2.2** | Yes | Yes | Yes [^3] | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | No |
| **LTX-2** | Yes | Yes | Yes [^4] | Yes | Yes | Yes | No | No | Yes | Yes | Yes | Yes | No | No |
| **Qwen-Image** [^5] | Yes | Yes | No | No | No | Yes | No | Yes | Yes | Yes | Yes | Yes | No | No |
| **Qwen-Image-Edit-2511** | Yes | Yes | No | No | Yes | No | No | Yes | Yes | Yes | No | No | No | No |
| **Cosmos3** | Yes | Yes | No | No | Yes | Yes | Yes | Yes | Yes | Yes | No | No | Yes | No |

[^1]: FLUX models use embedded guidance and do not have a separate negative prompt path, so CFG parallelism is not applicable.
Expand Down
2 changes: 2 additions & 0 deletions examples/visual_gen/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@ python models/flux1.py
python models/flux2.py
python models/cosmos3_ti2v.py --prompt "A robot arm picks fruit in a grocery store"
python models/qwen_image.py
python models/qwen_image_edit.py --image /path/to/source.png --prompt "Make the image look like a watercolor painting"

# With engine config (quant, parallelism, etc.)
python models/wan_t2v.py --visual_gen_args configs/wan2.2-t2v-fp4-1gpu.yaml
Expand All @@ -32,6 +33,7 @@ python models/flux1.py --visual_gen_args configs/flux1-dev-fp4-1gpu.yaml
python models/flux2.py --visual_gen_args configs/flux2-dev-fp4-1gpu.yaml
python models/cosmos3_ti2v.py --visual_gen_args configs/cosmos3-nano-1gpu.yaml --prompt "A robot arm picks fruit in a grocery store"
python models/qwen_image.py --visual_gen_args configs/qwen-image-fp8-1gpu.yaml
python models/qwen_image_edit.py --visual_gen_args configs/qwen-image-edit-2511-fp4-1gpu.yaml --image /path/to/source.png --prompt "Make the image look like a watercolor painting"
```

Install deps from the repo root: `pip install -r requirements-dev.txt`.
Expand Down
35 changes: 35 additions & 0 deletions examples/visual_gen/configs/qwen-image-edit-2511-fp4-1gpu.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

# 1-GPU Qwen-Image-Edit-2511 with selective NVFP4 dynamic quantization.
# Q/K/V projections remain in BF16 because they preserve accuracy
# Shared by offline examples (--visual_gen_args) and trtllm-serve.
quant_config:
quant_algo: NVFP4
dynamic: true
ignore:
- "transformer_blocks.*.attn.to_q"
- "transformer_blocks.*.attn.to_k"
- "transformer_blocks.*.attn.to_v"
- "transformer_blocks.*.attn.add_q_proj"
- "transformer_blocks.*.attn.add_k_proj"
- "transformer_blocks.*.attn.add_v_proj"
Comment on lines +19 to +28

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why these settings? When disclose such, we should have some docs to clarify the methodology.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I briefly mentioned in the comment above that

Q/K/V projections remain in BF16 because they preserve accuracy

Or we could use FP8 quant.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

modified to use FP8 quant config instead.

attention_config:
backend: VANILLA
parallel_config:
cfg_size: 1
ulysses_size: 1
cuda_graph_config:
enable: false
26 changes: 26 additions & 0 deletions examples/visual_gen/configs/qwen-image-edit-2511-fp8-1gpu.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
# Copyright (c) 2026, NVIDIA CORPORATION. All rights reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

# 1-GPU Qwen-Image-Edit-2511 with FP8 blockwise dynamic quantization.
# Shared by offline examples (--visual_gen_args) and trtllm-serve.
quant_config:
quant_algo: FP8_BLOCK_SCALES
dynamic: true
attention_config:
backend: VANILLA
parallel_config:
cfg_size: 1
ulysses_size: 1
cuda_graph_config:
enable: false
74 changes: 74 additions & 0 deletions examples/visual_gen/models/qwen_image_edit.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,74 @@
# Copyright (c) 2026, NVIDIA CORPORATION. All rights reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

"""Qwen-Image-Edit-2511 image editing.

Usage:
python qwen_image_edit.py --image input.png --prompt "Make the image look like a watercolor painting"
python qwen_image_edit.py \
--visual_gen_args ../configs/qwen-image-edit-2511-fp8-1gpu.yaml \
--image input.png
"""

import argparse

from tensorrt_llm import VisualGen, VisualGenArgs


def parse_args() -> argparse.Namespace:
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument(
"--model",
default="Qwen/Qwen-Image-Edit-2511",
help="Hugging Face model id or local checkpoint path.",
)
parser.add_argument(
"--visual_gen_args",
dest="visual_gen_args",
type=str,
default=None,
help="Path to YAML config (same as trtllm-serve --visual_gen_args)",
)
parser.add_argument(
"--image",
nargs="+",
Comment on lines +44 to +45

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This support is similar to Flux Kontext cc @karljang

required=True,
help="One or more input image paths or URLs.",
)
parser.add_argument(
"--prompt",
default="Make the image look like a watercolor painting while preserving the main subject.",
help="Text edit instruction.",
)
parser.add_argument(
"--output_path",
default="qwen_image_edit_output.png",
help="Edited image output path.",
)
return parser.parse_args()


def main() -> None:
args = parse_args()
extra_args = VisualGenArgs.from_yaml(args.visual_gen_args) if args.visual_gen_args else None
visual_gen = VisualGen(model=args.model, args=extra_args)
params = visual_gen.default_params
params.image = args.image if len(args.image) > 1 else args.image[0]
output = visual_gen.generate(inputs=args.prompt, params=params)
saved = output.save(args.output_path)
print(f"Saved edited image to {saved}")


if __name__ == "__main__":
main()
8 changes: 8 additions & 0 deletions tensorrt_llm/_torch/visual_gen/cuda_graph_runner.py
Original file line number Diff line number Diff line change
Expand Up @@ -77,6 +77,12 @@ def __init__(

self.graphs: Dict[KeyType, torch.cuda.CUDAGraph] = {}
self.graph_outputs: Dict[KeyType, Any] = {} # weak refs
# Keep the capture-created output tensors alive. Some pipelines keep
# outputs from multiple graph keys live at once, for example true CFG
# combines positive and negative Qwen-Image-Edit transformer outputs.
# Without strong refs, a later graph capture using the same memory pool
# can reuse the earlier output storage and make those outputs alias.
self._graph_output_refs: Dict[KeyType, Any] = {}
self.static_inputs: Dict[KeyType, Tuple[List[Any], Dict[str, Any]]] = {}
self.memory_pool = config.cuda_graph_mem_pool

Expand Down Expand Up @@ -147,6 +153,7 @@ def capture(

self.graphs[key] = graph
self.static_inputs[key] = (static_args, static_kwargs)
self._graph_output_refs[key] = output
self.graph_outputs[key] = make_weak_ref(output)
self.memory_pool = graph.pool()

Expand Down Expand Up @@ -209,5 +216,6 @@ def clear(self):
graph.reset()
self.graphs.clear()
self.graph_outputs.clear()
self._graph_output_refs.clear()
self.static_inputs.clear()
self.memory_pool = None
3 changes: 2 additions & 1 deletion tensorrt_llm/_torch/visual_gen/models/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -36,14 +36,15 @@
from .cosmos3 import Cosmos3OmniMoTPipeline
from .flux import Flux2Pipeline, FluxPipeline
from .ltx2 import LTX2Pipeline # noqa: F401
from .qwen_image import QwenImagePipeline
from .qwen_image import QwenImageEditPlusPipeline, QwenImagePipeline
from .wan import WanImageToVideoPipeline, WanPipeline

__all__ = [
"AutoPipeline",
"BasePipeline",
"FluxPipeline",
"Flux2Pipeline",
"QwenImageEditPlusPipeline",
"QwenImagePipeline",
"WanPipeline",
"WanImageToVideoPipeline",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@
"""

from .pipeline_qwen_image import QwenImagePipeline
from .pipeline_qwen_image_edit import QwenImageEditPlusPipeline
from .transformer_qwen_image import (
AdaLayerNormContinuous,
QwenEmbedRope,
Expand All @@ -25,6 +26,7 @@
__all__ = [
"AdaLayerNormContinuous",
"QwenEmbedRope",
"QwenImageEditPlusPipeline",
"QwenImagePipeline",
"QwenImageTransformer2DModel",
"QwenImageTransformerBlock",
Expand Down
Loading
Loading