v0.18.1 -> v0.19.0 by bxyu-nvidia · Pull Request #1 · bxyu-nvidia/vllm

bxyu-nvidia · 2026-05-26T19:45:24Z

PLEASE FILL IN THE PR DESCRIPTION HERE ENSURING ALL CHECKLIST ITEMS (AT THE BOTTOM) HAVE BEEN CONSIDERED.

Purpose

Test Plan

Test Result

Essential Elements of an Effective PR Description Checklist

The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
The test plan, such as providing test command.
The test results, such as pasting the results comparison before and after, or e2e results
(Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.

BEFORE SUBMITTING, PLEASE READ https://docs.vllm.ai/en/latest/contributing (anything written below this line will be removed by GitHub Actions)

…tion on cross-platform builds (vllm-project#37617) Signed-off-by: Andreas Karatzas <akaratza@amd.com>

…t#37733)

…project#37610) Signed-off-by: Andreas Karatzas <akaratza@amd.com>

…oject#37424) Signed-off-by: bongwoobak <bongwoobak@gmail.com> Co-authored-by: Chauncey <chaunceyjiang@gmail.com>

…(MI355) (vllm-project#37721) Signed-off-by: Andreas Karatzas <akaratza@amd.com>

…ct#34692) Signed-off-by: Tej Kiran <vpolamre@amd.com> Co-authored-by: Tej Kiran <vpolamre@amd.com>

…vllm-project#37318) Signed-off-by: Francesco Fusco <ffu@zurich.ibm.com>

…llm-project#37755) Signed-off-by: Mohammad Miadh Angkad <176301910+mmangkad@users.noreply.github.com>

…project#37756) Signed-off-by: Mohammad Miadh Angkad <176301910+mmangkad@users.noreply.github.com>

…2104) Signed-off-by: Brandon Pelfrey <bpelfrey@nvidia.com> Signed-off-by: Brandon Pelfrey <brandonpelfrey@gmail.com> Signed-off-by: Nick Hill <nickhill123@gmail.com> Co-authored-by: Nick Hill <nickhill123@gmail.com>

Merge awq.py and awq_marlin.py into a single file, eliminating the circular import between them. awq.py becomes a backward-compat shim. Follows the same structure as gptq_marlin.py. Co-authored-by: Claude Signed-off-by: Robert Shaw <robertgshaw2@gmail.com>

…llm-project#37768)

Signed-off-by: Robert Shaw <robshaw@redhat.com> Co-authored-by: Robert Shaw <robshaw@redhat.com>

…roject#37759) Signed-off-by: Robert Shaw <robertgshaw2@gmail.com>

vllm-project#37774) Signed-off-by: Andreas Karatzas <akaratza@amd.com>

…vllm-project#37775) Signed-off-by: Andreas Karatzas <akaratza@amd.com>

…project#37717) Signed-off-by: Andreas Karatzas <akaratza@amd.com>

…hts (vllm-project#37781) Signed-off-by: Andreas Karatzas <akaratza@amd.com>

…lback (vllm-project#37782) Signed-off-by: Andreas Karatzas <akaratza@amd.com>

Signed-off-by: Yang <lymailforjob@gmail.com>

… AOT support (vllm-project#37763) Signed-off-by: Andreas Karatzas <akaratza@amd.com>

…ect#35042 (vllm-project#37764) Signed-off-by: Andreas Karatzas <akaratza@amd.com>

…project#37778) Signed-off-by: Andreas Karatzas <akaratza@amd.com>

…evaluated in our nightly (vllm-project#37780) Signed-off-by: Andreas Karatzas <akaratza@amd.com>

…min acc threshold (vllm-project#37723) Signed-off-by: Andreas Karatzas <akaratza@amd.com>

vllm-project#36097) Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai> Signed-off-by: Woosuk Kwon <woosuk@inferact.ai> Co-authored-by: Woosuk Kwon <woosuk@inferact.ai>

Signed-off-by: yewentao256 <zhyanwentao@126.com>

…ce (vllm-project#37719) Signed-off-by: yewentao256 <zhyanwentao@126.com>

Signed-off-by: whyiug <whyiug@hotmail.com>

Signed-off-by: haosdent <haosdent@gmail.com>

…and retry (vllm-project#38415) Signed-off-by: Andreas Karatzas <akaratza@amd.com>

Signed-off-by: Kunshang Ji <jikunshang95@gmail.com> Co-authored-by: wang.yuqi <yuqi.wang@daocloud.io>

…v2/embed (vllm-project#38362) Signed-off-by: walterbm <walter.beller.morales@gmail.com>

…oling (vllm-project#35367) Signed-off-by: haosdent <haosdent@gmail.com>

…vllm-project#38111) Signed-off-by: Shubhra Pandit <shubhra.pandit@gmail.com>

…llm-project#38450) Signed-off-by: Andreas Karatzas <akaratza@amd.com>

…lm-project#38410) Signed-off-by: allgather <all2allops@gmail.com>

… 48.9% E2E throughput improvement (vllm-project#38139) Signed-off-by: yewentao256 <zhyanwentao@126.com>

Signed-off-by: Kyle Sayers <kylesayrs@gmail.com>

Signed-off-by: Andreas Karatzas <akaratza@amd.com>

…oject#33703) Signed-off-by: Stanislav Kirillov <stas@nebius.com> Co-authored-by: Stanislav Kirillov <stas@nebius.com> Co-authored-by: Chauncey <chaunceyjiang@gmail.com>

Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>

…audio_in_video tests (vllm-project#38492) Signed-off-by: Andreas Karatzas <akaratza@amd.com>

…torch. (vllm-project#38487)

…owlists (vllm-project#38329) Signed-off-by: Jaewon Lee <jaewon@meta.com>

…t#38495) Signed-off-by: haosdent <haosdent@gmail.com> (cherry picked from commit a08b773)

…#38535) Signed-off-by: jiang1.li <jiang1.li@intel.com> (cherry picked from commit 6557f49)

Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io> Signed-off-by: wang.yuqi <noooop@126.com> Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> (cherry picked from commit 719735d)

Signed-off-by: zjy0516 <riverclouds.zhu@qq.com> (cherry picked from commit ea7bfde)

…ect#38556) Signed-off-by: SandishKumarHN <sandishkumarhn@gmail.com> Signed-off-by: Matthew Bonanni <mbonanni@redhat.com> Co-authored-by: SandishKumarHN <sandishkumarhn@gmail.com> (cherry picked from commit 757068d)

…ng (vllm-project#36178) Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com> (cherry picked from commit eb47454)

…7160) Signed-off-by: Yifan Qiao <yifanqiao@berkeley.edu> Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai> (cherry picked from commit 91e4521)

Signed-off-by: jperezde <jperezde@redhat.com> (cherry picked from commit 58ee614)

…asoning_end_str (vllm-project#38242) Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com> (cherry picked from commit cbe7d18)

vllm-project#38730) Signed-off-by: Stefano Castagnetta <scastagnetta@nvidia.com> (cherry picked from commit 6183cae)

…103) hang (vllm-project#38730)" This reverts commit c284a66.

…timodal, Reasoning, Tool-Use) (vllm-project#38826) Signed-off-by: Luciano Martins <lucianommartins@users.noreply.github.com> Signed-off-by: Luciano Martins <lucianomartins@google.com> Co-authored-by: Luciano Martins <lucianommartins@users.noreply.github.com> Co-authored-by: Isotr0py <2037008807@qq.com> (cherry picked from commit 08ed2b9)

…llm-project#38847) Signed-off-by: Michael Hospedales <hospedales@me.com> (cherry picked from commit bb39382)

AndreasKaratzas and others added 30 commits March 21, 2026 11:58

[ROCm][CI] Guard CudaPlatform/RocmPlatform imports to fix test collec…

3ffa520

…tion on cross-platform builds (vllm-project#37617) Signed-off-by: Andreas Karatzas <akaratza@amd.com>

Revert "[compile] Initialize passes at VllmBackend init" (vllm-projec…

1fa1e53

…t#37733)

[ROCm][CI] Mark gemma3 as large GPU test to avoid OOM on MI250 (vllm-…

0d50fa1

…project#37610) Signed-off-by: Andreas Karatzas <akaratza@amd.com>

[Responses API] Add kv_transfer_params for PD disaggregation (vllm-pr…

17ee641

…oject#37424) Signed-off-by: bongwoobak <bongwoobak@gmail.com> Co-authored-by: Chauncey <chaunceyjiang@gmail.com>

[ROCm][CI] Update GSM8K eval config to use fp8-and-mixed models list …

02eec7e

…(MI355) (vllm-project#37721) Signed-off-by: Andreas Karatzas <akaratza@amd.com>

[ROCm] Enable DeepEP ROCm as all2allbackend for AMD GPUs. (vllm-proje…

3982bc2

…ct#34692) Signed-off-by: Tej Kiran <vpolamre@amd.com> Co-authored-by: Tej Kiran <vpolamre@amd.com>

[Hybrid] calling get_mamba_groups() once at MambaCopyBuffers.create() (…

298e510

…vllm-project#37318) Signed-off-by: Francesco Fusco <ffu@zurich.ibm.com>

[Core] Enable allreduce fusion by default for SM 10.3 (B300/GB300) (v…

88f1b37

…llm-project#37755) Signed-off-by: Mohammad Miadh Angkad <176301910+mmangkad@users.noreply.github.com>

[Perf] Add SM 10.3 (B300/GB300) all-reduce communicator tuning (vllm-…

61e381d

…project#37756) Signed-off-by: Mohammad Miadh Angkad <176301910+mmangkad@users.noreply.github.com>

Revert "Consolidate AWQ quantization into single awq_marlin.py file" (v…

5ad0446

…llm-project#37768)

[Quantization][Deprecation] Remove PTPC FP8 (vllm-project#32700)

eeee5b2

Signed-off-by: Robert Shaw <robshaw@redhat.com> Co-authored-by: Robert Shaw <robshaw@redhat.com>

[MoE] Move FlashInfer CuteDSL experts into fused_moe/experts/ (vllm-p…

6b2fa3a

…roject#37759) Signed-off-by: Robert Shaw <robertgshaw2@gmail.com>

[ROCm][CI] close missing quote in kernels/moe block in run-amd-test.sh (

e78bc74

vllm-project#37774) Signed-off-by: Andreas Karatzas <akaratza@amd.com>

[Bugfix] Fix pooling non-determinism from pinned prompt_lens aliasing (…

66f927f

…vllm-project#37775) Signed-off-by: Andreas Karatzas <akaratza@amd.com>

[ROCm][CI] Add large_gpu_mark to test_max_tokens_none for ROCm (vllm-…

c86b17c

…project#37717) Signed-off-by: Andreas Karatzas <akaratza@amd.com>

[CI] Skip ISAAC multimodal tests due to broken upstream HF model weig…

c862481

…hts (vllm-project#37781) Signed-off-by: Andreas Karatzas <akaratza@amd.com>

[Bugfix] Handle libsndfile sf_error(NULL) race condition in audio fal…

5dac719

…lback (vllm-project#37782) Signed-off-by: Andreas Karatzas <akaratza@amd.com>

[Perf] Optimize glm4.xv VIT (vllm-project#37779)

b050700

Signed-off-by: Yang <lymailforjob@gmail.com>

[ROCm][CI] Fix MEGA_AOT_ARTIFACT fallback when PyTorch < 2.10.0 lacks…

3b06c55

… AOT support (vllm-project#37763) Signed-off-by: Andreas Karatzas <akaratza@amd.com>

[ROCm][CI] get_cu_count was renamed to num_compute_units in vllm-proj…

6ecba84

…ect#35042 (vllm-project#37764) Signed-off-by: Andreas Karatzas <akaratza@amd.com>

[ROCm][CI] Added missing resampy dependency for MM audio tests (vllm-…

ffc8531

…project#37778) Signed-off-by: Andreas Karatzas <akaratza@amd.com>

[ROCm][CI] Make some duplicated tests optional so that they are only …

6eedec6

…evaluated in our nightly (vllm-project#37780) Signed-off-by: Andreas Karatzas <akaratza@amd.com>

[MoE] Move PF Methods to Folder (vllm-project#35927)

4383f15

[ROCm][CI] Stabilize ROCm speech-to-text translation test with lower …

cd1242d

…min acc threshold (vllm-project#37723) Signed-off-by: Andreas Karatzas <akaratza@amd.com>

[Model Runner V2] Support multi-modal embeddings for spec decode model (

b3e8460

vllm-project#36097) Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai> Signed-off-by: Woosuk Kwon <woosuk@inferact.ai> Co-authored-by: Woosuk Kwon <woosuk@inferact.ai>

[Bug] Fix fp8 deepgemm batch invariant (vllm-project#37718)

77d24c4

Signed-off-by: yewentao256 <zhyanwentao@126.com>

[Test] Only Run MLA model when user explicitly set for batch invarian…

eaf4978

…ce (vllm-project#37719) Signed-off-by: yewentao256 <zhyanwentao@126.com>

Enable NemotronHPuzzle + NemotronHMTP (vllm-project#37803)

e74c17e

whyiug and others added 30 commits March 28, 2026 10:28

[Misc]: clean up non-core lint issues (vllm-project#37049)

58c959a

Signed-off-by: whyiug <whyiug@hotmail.com>

[CI] Fix Ernie4.5-VL initialization test (vllm-project#38429)

b2bc736

Signed-off-by: haosdent <haosdent@gmail.com>

[ROCm][CI] Fix UV install in Dockerfile.rocm to detect curl failures …

af89140

…and retry (vllm-project#38415) Signed-off-by: Andreas Karatzas <akaratza@amd.com>

[CI]revert initialize_model context manager (vllm-project#38426)

aa4eb0d

Signed-off-by: Kunshang Ji <jikunshang95@gmail.com> Co-authored-by: wang.yuqi <yuqi.wang@daocloud.io>

[BugFix][Frontend] apply task instruction as system prompt in cohere …

fafca38

…v2/embed (vllm-project#38362) Signed-off-by: walterbm <walter.beller.morales@gmail.com>

[Feature] Add Qwen3-ForcedAligner support via token classification po…

d39b8da

…oling (vllm-project#35367) Signed-off-by: haosdent <haosdent@gmail.com>

[Spec Decode, BugFix] Propagate norm_before_fc from Eagle3 speculator (…

5b8c30d

…vllm-project#38111) Signed-off-by: Shubhra Pandit <shubhra.pandit@gmail.com>

[ROCm][CI] Fix cross-attention dispatch for encoder-decoder models (v…

43cc513

…llm-project#38450) Signed-off-by: Andreas Karatzas <akaratza@amd.com>

[Transformers v5] fix missing pixtral/voxtral multimodal dispatch (vl…

8c0b626

…lm-project#38410) Signed-off-by: allgather <all2allops@gmail.com>

[Perf] Remove redundant device copies for CPU-only pooling token IDs,…

995dea1

… 48.9% E2E throughput improvement (vllm-project#38139) Signed-off-by: yewentao256 <zhyanwentao@126.com>

[QeRL] Fix online quantized reloading (vllm-project#38442)

d28d86e

Signed-off-by: Kyle Sayers <kylesayrs@gmail.com>

[ROCm][CI] Enable hybrid chunked prefill test (vllm-project#38317)

4f2ed5f

Signed-off-by: Andreas Karatzas <akaratza@amd.com>

[Bugfix] Support multi-type params parsing for DeepSeek v3.2 (vllm-pr…

a6db99b

…oject#33703) Signed-off-by: Stanislav Kirillov <stas@nebius.com> Co-authored-by: Stanislav Kirillov <stas@nebius.com> Co-authored-by: Chauncey <chaunceyjiang@gmail.com>

Add @ZJY0516 to CODEOWNERS (vllm-project#38497)

c133f33

Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>

[CI] Add temperature=0.0, reduce max_tokens, and add debug prints to …

bea2353

…audio_in_video tests (vllm-project#38492) Signed-off-by: Andreas Karatzas <akaratza@amd.com>

[Misc] Always use forward_mulmat for Conv3d on newer versions of …

92f0db5

…torch. (vllm-project#38487)

[MoE] Add RoutingMethodType.Simulated to TRT-LLM FP8/NVFP4 kernel all…

d816834

…owlists (vllm-project#38329) Signed-off-by: Jaewon Lee <jaewon@meta.com>

[CI] Fix SPLADE pooler test broken by vllm-project#38139 (vllm-projec…

b92312d

…t#38495) Signed-off-by: haosdent <haosdent@gmail.com> (cherry picked from commit a08b773)

[Bugfix][CPU] Skip set_num_threads after thread binding (vllm-project…

12449f9

…#38535) Signed-off-by: jiang1.li <jiang1.li@intel.com> (cherry picked from commit 6557f49)

[CI] fix LM Eval Qwen3.5 Models (B200) (vllm-project#38632)

bcc0fdd

Signed-off-by: zjy0516 <riverclouds.zhu@qq.com> (cherry picked from commit ea7bfde)

[Bugfix][MLA] Add logits size budget to sparse indexer prefill chunki…

0ee3b7f

…ng (vllm-project#36178) Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com> (cherry picked from commit eb47454)

[Feat][v1] Simple yet General CPU KV Cache Offloading (vllm-project#3…

1dbbafd

…7160) Signed-off-by: Yifan Qiao <yifanqiao@berkeley.edu> Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai> (cherry picked from commit 91e4521)

(security) Enforce frame limit in VideoMediaIO (vllm-project#38636)

29982d4

Signed-off-by: jperezde <jperezde@redhat.com> (cherry picked from commit 58ee614)

[Misc] Rename think_start_str/think_end_str to reasoning_start_str/re…

3a30a1a

…asoning_end_str (vllm-project#38242) Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com> (cherry picked from commit cbe7d18)

[Bugfix] Restrict TRTLLM attention to SM100, fixing GB300 (SM103) hang (

c284a66

vllm-project#38730) Signed-off-by: Stefano Castagnetta <scastagnetta@nvidia.com> (cherry picked from commit 6183cae)

Revert "[Bugfix] Restrict TRTLLM attention to SM100, fixing GB300 (SM…

cfad6a5

…103) hang (vllm-project#38730)" This reverts commit c284a66.

[Bugfix]: Fix Gemma4ToolParser.__init__() missing tools parameter (v…

2a69949

…llm-project#38847) Signed-off-by: Michael Hospedales <hospedales@me.com> (cherry picked from commit bb39382)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

v0.18.1 -> v0.19.0#1

v0.18.1 -> v0.19.0#1
bxyu-nvidia wants to merge 466 commits into
bxyu/v0.18.1from
bxyu/v0.19.0

bxyu-nvidia commented May 26, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

20 participants

Conversation

bxyu-nvidia commented May 26, 2026

Purpose

Test Plan

Test Result

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

20 participants