feat: support image URLs (fileUri) in image input fields#522
Open
gennadijbgl wants to merge 3 commits into
Open
feat: support image URLs (fileUri) in image input fields#522gennadijbgl wants to merge 3 commits into
gennadijbgl wants to merge 3 commits into
Conversation
Mirror the existing `file` field's data/fileUri pattern for `image`
fields so images can be supplied by URL instead of inline base64.
An image field value now accepts either `{ mimeType, data }` (base64,
existing) or `{ mimeType, fileUri }` (URL). Providers map the URL
variant natively:
- OpenAI: passes the URL straight to `image_url.url`
- Anthropic: emits `source: { type: 'url', url }`
- Gemini: emits `fileData: { fileUri }` (e.g. gs:// / Files API URIs)
Validation, hashing, logging and content-processing paths updated to
handle the URL variant. Adds prompt-render/validation tests and Gemini
request-mapping tests.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Fold the now-identical image/file validators in dsp/util.ts into a single `validMediaContent` (mimeType + exactly one of data|fileUri), and collapse the six duplicated Gemini image/file content blocks into one `toGeminiMediaPart` helper. No behavior change. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Collapse the duplicated `'fileUri' in validated ? … : …` ternary in the array and non-array image-render branches into one local `toImagePart`, matching the `toGeminiMediaPart` style. No behavior change. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Collaborator
|
sorry was busy with big changes to multi language codebase will review this in a day or so. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Adds URL support to
imageinput fields, mirroring the existingfilefield'sdata/fileUripattern. Images can now be supplied by URL instead of only inline base64.An image field value now accepts either
{ mimeType, data }(base64, unchanged) or{ mimeType, fileUri }(URL). Supplying both, or neither, is rejected — same rule thefiletype already enforces.How
The internal
imagecontent block gains a second union member carryingfileUri(parallel to howfileis modeled). Each provider maps the URL variant to its native form:image_url.url= data URIimage_url.url= the URL directlysource: { type: 'base64', … }source: { type: 'url', url }inlineData: { data }fileData: { fileUri }(e.g.gs:/// Files API URIs)Validation (
dsp/util.ts,dsp/prompt.ts,ai/validate.ts), content hashing (ai/base.ts), logging (dsp/loggers.ts) and the unsupported-provider content processor (ai/processor.ts) were updated to narrow on the URL variant.AxFieldValueexposes the new shape to callers.Tests
dsp/prompt.test.ts— image render withdata, withfileUri, mixed arrays, and validation (missing mimeType / missing both / both present).google-gemini/api.test.ts— request mapping: base64 →inlineData,fileUri→fileData.jsRuntimesandbox failures are a Windows-only Node permission-model artifact).🤖 Generated with Claude Code