fix(engine): key replay lookups by the served model#16
Open
flobernd wants to merge 1 commit into
Open
Conversation
The replay LOOKUP hashed on request.model (the raw client-facing alias), while the INSERT (via hash_visible_history in run_turn) hashes on served_model (the profile's resolved upstream_model). For any profile that sets upstream_model, request.model never equals served_model, so the two hashes disagree on every turn and a follow-up request never finds the prior turn's baseline. find_replay_baseline now takes the caller's already-resolved served_model and hashes on that instead of re-deriving it, keeping resolve_request_model the single resolution point. Catch-all profiles are unaffected since their served model already equals the request model. Closes local-inference-lab#32
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The replay-cache lookup now keys by the served model, matching the insert side. Profiles that set
upstream_modelpreviously never got a replay hit because the lookup hashed the request model while the insert hashed the served model.Testing
Full
cargo testgreen (0 failed across all binaries),cargo fmt --checkclean,cargo clippy --all-targetszero warnings. New wire test routes a web-search replay through a profile withupstream_modelset and asserts the turn-2 tool result is reconstructed from replay (RED before the fix: the lookup missed and the placeholder text appeared instead). Catch-all profiles are unaffected since their served model equals the request model.Closes local-inference-lab#32.
Notes
Stacked on local-inference-lab#36's branch; retarget or rebase once that merges.