Skip to content

fix(vllm_model): handle explicit null metadata in preprocessing#1727

Open
rodboev wants to merge 1 commit into
NVIDIA-NeMo:mainfrom
rodboev:pr/vllm-metadata-null
Open

fix(vllm_model): handle explicit null metadata in preprocessing#1727
rodboev wants to merge 1 commit into
NVIDIA-NeMo:mainfrom
rodboev:pr/vllm-metadata-null

Conversation

@rodboev

@rodboev rodboev commented Jun 25, 2026

Copy link
Copy Markdown

responses_api_models/vllm_model can still crash on a valid request that explicitly sends "metadata": null, because preprocessing expects a dict and dereferences .get() on a None value. This PR keeps metadata=None on the same code path as omitted metadata by normalizing it to {} before any metadata lookups.

This PR fixes that with a narrow adapter-only change:

  • Normalize only explicit metadata=None in _preprocess_chat_completion_create_params()
  • Keep absent metadata absent and leave present non-null metadata payloads untouched
  • Add a focused regression test that calls _preprocess_chat_completion_create_params() directly with metadata: None and asserts it returns with metadata == {}
  • Keep the existing metadata override tests unchanged as non-regression coverage

No request schema changes, no other model adapters, and no dependency updates.

Closes #1096

Checklist

  • Targeted tests pass: uv run pytest responses_api_models/vllm_model/tests/test_app.py -k "metadata_none or metadata_chat_template_kwargs_override or metadata_extra_body_override" -x
  • Touched-file hooks pass: uv run pre-commit run --files responses_api_models/vllm_model/app.py responses_api_models/vllm_model/tests/test_app.py
  • Touched-file lint passes: uv run ruff check responses_api_models/vllm_model/app.py responses_api_models/vllm_model/tests/test_app.py
  • Touched-file format check passes: uv run ruff format --check responses_api_models/vllm_model/app.py responses_api_models/vllm_model/tests/test_app.py
  • No request schema changes, other model adapter changes, or dependency changes
  • DCO sign-off on all commits

The local validation commands above were attempted on this Windows host, but uv stops while resolving uvloop with RuntimeError: uvloop does not support Windows at the moment. Upstream CI is the source of truth for the full matrix.

Signed-off-by: Rod Boev <rod.boev@gmail.com>
@copy-pr-bot

copy-pr-bot Bot commented Jun 25, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@nemo-automation-bot nemo-automation-bot Bot added the community-request Issue reported or requested by someone from the community label Jun 25, 2026
@svcnvidia-nemo-ci svcnvidia-nemo-ci added the waiting-on-maintainers Waiting on maintainers to respond label Jun 27, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

community-request Issue reported or requested by someone from the community waiting-on-maintainers Waiting on maintainers to respond

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bug: vllm model crashes on requests with metadata null

2 participants