[Kimi-k2.5] fix(regen): carry the target's reasoning into the regenerated history#864
Conversation
|
Important Review skippedDraft detected. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Plus Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Merge Protections🔴 1 of 1 protections blocking · waiting on 👀 reviews
🔴 Require approval from approved reviewers listWaiting for any of
This rule is failing.All pull requests must have at least one approving review from a member of the approved reviewers list before merging.
|
On-policy regeneration appended each reply to the running prefix as `content` alone, dropping the `reasoning` the server had just parsed out. Chat templates read that field to decide whether a turn's reasoning is re-rendered into the next prompt, so the next turn was conditioned on a prefix serving would not produce. Qwen3 hid this: its template falls back to splitting `<think>` out of `content` when the field is missing. Kimi K2.5 has no fallback and preserves reasoning for exactly the in-flight tool loop, so a tool-calling conversation lost the reasoning behind every call. Forward the field instead. Reads either response spelling and sends the older one, which current vLLM normalizes and older builds pass through. A no-op without `--reasoning-parser`, since there is then no field to carry. Signed-off-by: Ranran Haoran Zhang <ranzhang@redhat.com>
6b602fd to
20bdc51
Compare
Warning!! This PR is from static code analysis, not tested end2end on kimi2.5!
Purpose
On-policy regeneration appends each reply to the running prefix as
contentalone, dropping thereasoningthe server just parsed out. Chat templates read that field to decide whether a turn's reasoning is re-rendered into the next prompt, so every turn after the first was conditioned on a prefix serving would not produce.Take a two-step tool call on Kimi K2.5. The target reasons, calls
get_weather, and the second request has to re-render that call as history:Kimi K2.5 preserves reasoning for exactly the in-flight tool loop and reads it only from
reasoning_content-- no fallback. Qwen3 hid the bug because its template splits<think>back out ofcontentwhen the field is missing, so the same defect is invisible there and silent here.This forwards the field. It reads either response spelling (current vLLM answers with
reasoning, older builds usedreasoning_content) and sends the older one, which current vLLM normalizes and older builds pass through untouched. Without--reasoning-parserthere is no field on the response, nothing is carried, and behaviour is unchanged.Tests
One test, on the tool-call path -- the branch where a real template actually loses the data.
Red without the source change, green with it:
ruff check,ruff format --check, andmypy --check-untyped-defs(174 files) all pass.Not yet verified end to end against a live thinking model -- the round trip is traced through vLLM's request normalization and template rendering, but no two-turn request has been run to watch the field survive. Draft until that is done.
Checklist
I have filled in: