Skip to content

standup: probe decode vllm port#1603

Open
hubunt wants to merge 1 commit into
llm-d:mainfrom
hubunt:hubunt/fix-decode-probe-port
Open

standup: probe decode vllm port#1603
hubunt wants to merge 1 commit into
llm-d:mainfrom
hubunt:hubunt/fix-decode-probe-port

Conversation

@hubunt

@hubunt hubunt commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

Summary

Fix decode pod probe ports so they target the actual vLLM bind port when routing proxy is enabled.

Motivation

Fixes #1435.

In pd-disaggregation, decode vLLM is started with --port $VLLM_METRICS_PORT, which resolves to 8200. However, the rendered ModelService values used
decode.vllm.servicePort (8000) as the default decode probe port. This can cause decode pod readiness checks to fail even though vLLM is listening on 8200.

What changed

  • Decode probe defaults now follow the vLLM bind port:
    • routing proxy enabled: decode.vllm.port
    • routing proxy disabled: decode.vllm.servicePort
  • Existing explicit decode.probes.*.port overrides are preserved.
  • Updated the port selection docs.
  • Added a regression test rendering the real pd-disaggregation scenario and verifying decode startup/liveness/readiness probes use 8200.

Testing

  • python -m pytest tests/test_decode_probe_port.py -q
  • python -m pytest tests -q
  • python -m py_compile tests/test_decode_probe_port.py
  • git diff --check

Backward Compatibility

Scenarios with explicit probe port overrides are unchanged. When routing proxy is disabled, decode probes continue to default to decode.vllm.servicePort.

Signed-off-by: hubunt <150658615+hubunt@users.noreply.github.com>
@mengmeiye mengmeiye added the do-not-merge Indicates that a PR should not merge label Jul 20, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

do-not-merge Indicates that a PR should not merge

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Decode pod readiness probe uses wrong port causing CrashLoopBackOff

2 participants