[GG] spec-decode: cap same-model draft position limit

[voipmonitor]

Summary

When a target model uses a dict hf_overrides to reduce
max_position_embeddings, that dict intentionally remains target-only. A
same-checkpoint speculative draft could therefore allocate RoPE tables for the
checkpoint default even though the scheduler cannot reach those positions.

After constructing the draft ModelConfig, this change lowers the draft text
config limit when, and only when:

• the target and draft resolve to the same model;
• both hf_text_config objects expose max_position_embeddings; and
• the target limit is smaller.

External draft checkpoints are untouched, and an already-smaller draft limit is
never increased.

Scope and prior work

This supersedes only the draft-context half of #146. It is an independent commit
directly on the current dev/gilded-gnosis head. Compared with #146, it uses
hf_text_config (including nested text configs), verifies same-model identity,
and includes regression tests. Searches of open upstream and local PRs found no
other implementation; #146 is the intentional predecessor being split.
PR #148 contains #146 in its stacked history, but its unique tip changes MLA
allocation order and is orthogonal to this configuration fix; it will need a
clean rebase after the predecessor replacement.

The original GLM-5.2 TP4 validation in #146 reported about 128 MiB/GPU reclaimed
when capping a 1M checkpoint default to a 600k target, with unchanged throughput,
KLD, and MTP acceptance. This patch does not alter any position the target can
schedule.

Validation

pytest tests/config/test_speculative_draft_hf_overrides.py -q
11 passed

ruff check vllm/config/speculative.py tests/config/test_speculative_draft_hf_overrides.py
ruff format --check vllm/config/speculative.py tests/config/test_speculative_draft_hf_overrides.py
git diff --check
all passed

AI assistance was used. This PR must not be merged until the human submitter has
reviewed every changed line and can defend the behavior end to end.

[voipmonitor]

@coderabbitai review

[voipmonitor]

@coderabbitai review

[coderabbitai]

✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

[coderabbitai]

Warning

Review limit reached

@voipmonitor, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 55 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: a4a55a6c-bcda-4eb5-a99f-34cfef8ded68

📥 Commits

Reviewing files that changed from the base of the PR and between 73e4a8c and b8d0967.

📒 Files selected for processing (2)

• tests/config/test_speculative_draft_hf_overrides.py
• vllm/config/speculative.py

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch fix/gg-draft-context-cap-20260721

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}