fix (ai): serialize parallel tool results in tool-call order#16572
Open
Kripu77 wants to merge 1 commit into
Open
fix (ai): serialize parallel tool results in tool-call order#16572Kripu77 wants to merge 1 commit into
Kripu77 wants to merge 1 commit into
Conversation
1 task
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Background
Fixes #16567.
When an assistant step makes parallel tool calls, results are recorded in
completion order (whichever
executeresolves first).toResponseMessagesserializes that arrival order verbatim, so the same logical step produces
differently-ordered
tool-resultparts depending on tool timing.Providers pair tool calls and results by id, so responses are unaffected —
but provider prompt caching is byte-sensitive. A conversation served live
(completion order) and the same conversation rebuilt from persisted UIMessages
(call order, since tool parts are created at call time) serialize differently,
invalidating the entire cached prefix behind the first parallel batch. Measured
on a production agent loop: a 4-tool batch serialized in the exact reverse of
call order re-wrote 26 byte-identical cached blocks on the next request.
Summary
Sort the tool message's
tool-resultparts to match the order of thecorresponding
tool-callparts. Results without a matching call sort last,tie-broken by id, so the ordering is total and deterministic. Sorting is
skipped when the tool message contains
tool-approval-responseparts, sinceapproval responses and their synthesized execution-denied results carry
position-dependent semantics.
Within-step result order is not semantically meaningful to providers, so this
is behavior-preserving — it only makes serialization deterministic and
cache-friendly.
Manual Verification
Validated end-to-end against Bedrock Anthropic prompt caching: with this
change, a 36-request agent session with parallel batches up to size 8 produced
zero cache-prefix invalidations (previously invalidated on every fresh request
after a parallel batch).
Checklist
pnpm changesetin the project root)Related Issues
Fixes #16567