Skip to content

[Platform][Mistral] Add OCR support via the /v1/ocr endpoint#2161

Merged
chr-hertel merged 1 commit into
symfony:mainfrom
tacman:mistral-ocr
Jun 21, 2026
Merged

[Platform][Mistral] Add OCR support via the /v1/ocr endpoint#2161
chr-hertel merged 1 commit into
symfony:mainfrom
tacman:mistral-ocr

Conversation

@tacman

@tacman tacman commented Jun 12, 2026

Copy link
Copy Markdown
Contributor
Q A
Bug fix? no
New feature? yes
Docs? yes
Issues Fix #2072
License MIT

Adds OCR support to the Mistral bridge via the dedicated /v1/ocr endpoint, as
discussed in #2072.

Mistral's mistral-ocr-latest model is not a chat completion — it uses a
separate endpoint and returns structured per-page output, so it can't be routed
through MessageBag/chat. Following the existing Whisper bridge pattern, this
adds a dedicated Ocr model with its own ModelClient and ResultConverter:

use Symfony\AI\Platform\Bridge\Mistral\Factory;
use Symfony\AI\Platform\Bridge\Mistral\Ocr\Result\OcrResult;
use Symfony\AI\Platform\Message\Content\DocumentUrl;

$platform = Factory::createPlatform($apiKey);

$result = $platform->invoke('mistral-ocr-latest', new DocumentUrl('https://example.com/document.pdf'));

$ocr = $result->asObject();
\assert($ocr instanceof OcrResult);

echo $ocr->getMarkdown();

The result is a typed OcrResult (pages with markdown, layout images + bounding
boxes, per-page annotations, usage info) via ObjectResult/asObject(), not a
text blob. Document URL, binary PDF and image inputs all work — the existing
Document, DocumentUrl and ImageUrl contract normalizers were widened to
also accept the Ocr model. Catalog entries (mistral-ocr-latest,
mistral-ocr-2505) plus a generator rule keep OCR models out of the chat class.

Verified end-to-end against the live API (a 29-page arXiv PDF by URL and a
binary PDF by base64), plus unit tests for the client, converter and catalog.

Also includes a Document OCR demo (demo/) showing the feature as a chat that
extracts a document's text and answers questions about it.

Docs: docs/components/platform.rst, the bridge README.md, examples under
examples/mistral/, and a CHANGELOG.md entry.

Companion PR #2160 (CacheableInputInterface) makes the ai.platform.cache.*
decorator able to cache content-object inputs like DocumentUrl, so OCR results
can be cached at the platform layer — independent of this PR.

@carsonbot carsonbot changed the title [Platform][Mistral] Add OCR support via the /v1/ocr endpoint [Mistral] Add OCR support via the /v1/ocr endpoint Jun 12, 2026
@carsonbot carsonbot added Feature New feature Platform Issues & PRs about the AI Platform component labels Jun 12, 2026
@carsonbot carsonbot changed the title [Mistral] Add OCR support via the /v1/ocr endpoint [Platform][Mistral] Add OCR support via the /v1/ocr endpoint Jun 12, 2026
Comment thread src/platform/src/Bridge/Mistral/README.md Outdated

@chr-hertel chr-hertel left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for working on this @tacman - including that in the demo is great, but UI needs some love there: gap between the rows and no bg color/gradient (also for detail page) is inconsistent

Image

Thanks already!

@tacman

tacman commented Jun 12, 2026

Copy link
Copy Markdown
Contributor Author

Yes. I'll tweak the UI and make getting the OCR layout a #[AsTool] -- then the agent can fetch the layout and respond.

Comment thread src/platform/src/Bridge/Mistral/Ocr/ResultConverter.php Outdated
@chr-hertel

Copy link
Copy Markdown
Member

Cherry-picked f5e8a48 from #2158 and made some UI patches in the demo.

@chr-hertel

Copy link
Copy Markdown
Member

Thank you @tacman.

@chr-hertel chr-hertel merged commit ddabafc into symfony:main Jun 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Feature New feature Platform Issues & PRs about the AI Platform component Status: Reviewed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Platform][Mistral] Add Mistral OCR (/v1/ocr) support to symfony/ai-mistral-platform

3 participants