Skip to content

Add MIME hints to provider-file attachments (FIG-616 item 2)#133

Merged
SamGalanakis merged 1 commit into
mainfrom
samuel-fig-616-providerfile-mime
Jul 25, 2026
Merged

Add MIME hints to provider-file attachments (FIG-616 item 2)#133
SamGalanakis merged 1 commit into
mainfrom
samuel-fig-616-providerfile-mime

Conversation

@SamGalanakis

Copy link
Copy Markdown
Collaborator

AttachmentSource::ProviderFile was the only variant carrying no MIME (Inline, ExternalUrl and Stored's AttachmentRef all have one). Because source.media_type() returned None for it, the Anthropic adapter had an early return hardcoding a document block — so an image file id produced a wire error or the wrong modality, since Anthropic requires image file ids in an image block.

Design (grilled and ratified)

  • media_type: Option<MediaType> — optional, not required. Verified that only Anthropic needs it: OpenAI validates the provider scope and emits by file id, Google emits {"fileData":{"fileUri":id}}; both resolve the type server-side. Requiring it would tax the two providers that never consume it and invite invented values.
  • Open vocabulary preserved. No closed MIME enum — MediaType is deliberately a validated newtype over String ("Lash deliberately does not maintain a closed media catalog", FIG-564), and a closed catalog would contradict that and need a lash release per new provider MIME.
  • Anthropic: the fix is a deletion. The hardcoded-document early return is gone; provider file ids now flow through the normal is_image() dispatch. When the hint is absent it fails before the network (provider_file_media_type_required, Validation kind) rather than silently guessing document — unknown stays representable for the providers that don't care, but can never produce the wrong modality.

Verification

Type size 104 B, under the existing size_of <= 128 compile-time ceiling (unchanged). Serde omits the field when None, so existing payloads deserialize; core↔remote round-trip preserves both Some and None. Tests cover image/document block selection, the absent-hint refusal, and serde compatibility — independently re-run, 12/12 green. OpenAI and Google behaviour unchanged.

Breaking at source level only: the provider_file helper gains a third argument and direct enum construction gains a field; serialized payloads are unaffected.

Closes FIG-616 item 2. Item 3 was separately ratified as keep all 12 OpenAI MIMEs — OpenAI's live docs disprove the PDF-centric premise, and our list is in fact a subset of what they accept.

Carry optional validated media types through core and remote provider-file attachments. Anthropic uses the hint to select image or document file blocks and rejects missing hints before transport, while OpenAI and Google continue resolving file types server-side.

Release-Notes: Breaking: Provider-file attachments now carry an optional media type, and Anthropic provider file ids require that hint so Lash can choose the image or document modality.
@SamGalanakis
SamGalanakis merged commit 954a388 into main Jul 25, 2026
20 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant