Skip to content

fix(examples): preserve geo3k response budget#2140

Open
zhangdw156 wants to merge 1 commit into
THUDM:mainfrom
zhangdw156:fix/geo3k-response-budget
Open

fix(examples): preserve geo3k response budget#2140
zhangdw156 wants to merge 1 commit into
THUDM:mainfrom
zhangdw156:fix/geo3k-response-budget

Conversation

@zhangdw156

Copy link
Copy Markdown

Summary

  • Fix geo3k VLM multi-turn rollout budgeting so max_new_tokens is treated as response budget, not total prompt+response budget.
  • When rollout_max_context_len is also set, use the tighter remaining context/response budget.
  • Add a CPU-only regression test for fresh prompts, existing response tokens, and combined context/response caps.

Why

sampling_params["max_new_tokens"] comes from rollout_max_response_len. The current multi-turn example subtracts len(sample.tokens), which includes the prompt, so long prompts can prematurely truncate or reduce the model's response budget before generation starts.

Tests

  • uvx --from pytest pytest tests/test_geo3k_vlm_multi_turn_budget.py -q
  • uvx ruff check examples/geo3k_vlm_multi_turn/rollout.py tests/test_geo3k_vlm_multi_turn_budget.py
  • git diff --check

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant