[Chat] Normalize non-text AssistantMessage content in MessageNormalizer#2265
Open
Amoifr wants to merge 1 commit into
Open
[Chat] Normalize non-text AssistantMessage content in MessageNormalizer#2265Amoifr wants to merge 1 commit into
Amoifr wants to merge 1 commit into
Conversation
normalizeAssistantParts() only handled Text, Thinking and ToolCall parts and silently dropped any other content. An AssistantMessage carrying an Image (e.g. a generated image returned by Gemini) was therefore lost when stored through the MessageNormalizer and could not be read back. Handle File/Document/Image/Audio (as data URL) and ImageUrl/DocumentUrl parts on both sides, mirroring what is already done for UserMessage content.
carsonbot
added
Bug
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The original round-trip issue for
UserMessageimages (base64 vs. data URL mismatch) was already fixed by 8ca2ab0. This PR addresses the remaining case reported in the issue's follow-up:AssistantMessagecontent.MessageNormalizer::normalizeAssistantParts()only handledText,ThinkingandToolCallparts and silently dropped anything else. So anAssistantMessagecarrying anImage(e.g. a generated image returned by Gemini) was lost on the way to the store and could not be read back.File/Document/Image/Audio(as data URL) andImageUrl/DocumentUrlparts are now handled on both the normalize and denormalize sides, mirroring what is already done forUserMessagecontent. Added a round-trip test for anAssistantMessageholding anImage.