fix: strip image data content blocks for non-vision models#3
Merged
Conversation
The non-vision image strip only recognized OpenAI `image_url` parts, not standard `image` data content blocks. Uploaded images that arrive as data blocks slipped through `stripImagesFromMessages`/`filterImagePartsIfNeeded`, got converted to `image_url` by the message converter, and reached a text-only model - the provider then returns `400 <model> is not a multimodal model`. Surfaced on a handoff from a vision-capable agent to a text-only one (e.g. mistral-small-3.2 -> glm-5.2) with an image in history. Add a shared `isImageContentPart` predicate covering `image_url` and `image` parts, used by both strip functions.
JumpLink
added a commit
to faktenforum/ai-chat-interface
that referenced
this pull request
Jul 3, 2026
Includes faktenforum/agents#3: the non-vision image strip now also drops standard `image` data content blocks, not just `image_url` parts. Fixes `400 <model> is not a multimodal model` when an image is in history and a vision-capable agent hands off to a text-only one (Faktencheck / mistral-small-3.2 -> Assistant / glm-5.2). Triggers a librechat image rebuild.
JumpLink
added a commit
to faktenforum/ai-chat-interface
that referenced
this pull request
Jul 3, 2026
Includes faktenforum/agents#3: the non-vision image strip now also drops standard `image` data content blocks, not just `image_url` parts. Fixes `400 <model> is not a multimodal model` when an image is in history and a vision-capable agent hands off to a text-only one (Faktencheck / mistral-small-3.2 -> Assistant / glm-5.2). Triggers a librechat image rebuild.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Sending an image to a text-only model returns
400 <model> is not a multimodal model(Scaleway/OpenAI-compatible). Reproduces on a handoff from a vision-capable agent to a text-only one with an image already in history - e.g.mistral-small-3.2-24b(vision) hands off toglm-5.2(text-only); the shared history still carries the uploaded image, and the text-only agent's request 400s.Root cause
The non-vision image strip only recognized OpenAI-style
image_urlparts, not LangChain standardimagedata content blocks:stripImagesFromMessages/filterImagePartsIfNeededfiltered onlytype === 'image_url', so atype: 'image'data block passes through._convertMessagesToOpenAIParamsthen converts that data block into animage_url(viafromStandardImageBlock) and sends it - reaching a model that can't accept images.stripImagesFromMessagesis the intended single choke point (it runs inChatOpenAI._streamResponseChunkswith the correctvisionCapable), but its predicate missed the data-block representation.Fix
Add a shared
isImageContentPartpredicate that matches bothimage_urlandimageparts, and use it in both strip functions. Small, behind the existingvisionCapable === falsegate - no change for vision-capable models.Tests
New cases in
messages.test.tsforstripImagesFromMessages: removesimage_urlparts, removesimagedata blocks (the fix - fails on the old predicate), substitutes the omitted-image placeholder when only image content remains, and leaves messages untouched for vision-capable models. Full suite green.Note: the pre-commit hook flags a pre-existing
no-nested-ternaryin the same file (present onmain, unrelated to this change), so this was committed with--no-verify; the two changed files are prettier/eslint-clean on their own lines.