Skip to content

[BIG FEATURE] Add image recognition fallback input flow#259

Open
musnows wants to merge 1 commit into
KunAgent:developfrom
musnows:codex/image-recognition-fallback-input
Open

[BIG FEATURE] Add image recognition fallback input flow#259
musnows wants to merge 1 commit into
KunAgent:developfrom
musnows:codex/image-recognition-fallback-input

Conversation

@musnows

@musnows musnows commented Jun 13, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • add an opt-in image recognition fallback for text-only chat models, with persisted settings and an image input settings section
  • pre-recognize attached images before sending to text-only models, then append path, MIME, dimensions, and recognized content to the user prompt without base64 or RecognitionModel
  • add a gated built-in recognize_image tool for local paths and URLs, plus runtime progress events shown in the chat UI

Root Cause / Behavior

Text-only models previously either rejected image attachments or relied on generic attachment fallback. This change lets users explicitly configure a multimodal recognition model so new eligible conversations can still attach or paste images while preserving historical conversation behavior by comparing the persisted enable time against the first user message time.

Validation

  • npm --prefix kun test -- tests/attachment-store.test.ts src/adapters/tool/image-recognition-tool-provider.test.ts
  • npm test -- src/renderer/src/lib/attachment-upload-availability.test.ts src/shared/app-settings.test.ts src/renderer/src/agent/kun-mapper.test.ts src/renderer/src/components/chat/MessageTimeline.tool-summary.test.ts src/renderer/src/components/settings-section-image-input.test.ts src/main/ipc/app-ipc-schemas.test.ts
  • npm --prefix kun run typecheck
  • npm run typecheck

@musnows

musnows commented Jun 13, 2026

Copy link
Copy Markdown
Collaborator Author

新增独立的多模态模型配置项,支持让多模态模型预处理图片后发给不支持多模态的模型。同时新增识图mcp工具,支持非多模态模型调用工具阅读图片内容。

image

@musnows musnows force-pushed the codex/image-recognition-fallback-input branch from 67b9c8f to 99429fa Compare June 13, 2026 12:18
@musnows musnows marked this pull request as ready for review June 13, 2026 12:21
@musnows musnows changed the title [codex] Add image recognition fallback input flow [BIG FEATURE] Add image recognition fallback input flow Jun 13, 2026
@musnows musnows force-pushed the codex/image-recognition-fallback-input branch 2 times, most recently from ba93a0d to 490aadb Compare June 14, 2026 06:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant