feat: image-input variants keep native pasted images for text-only models - #36
Open
Anionex wants to merge 5 commits into
Open
feat: image-input variants keep native pasted images for text-only models#36Anionex wants to merge 5 commits into
Anionex wants to merge 5 commits into
Conversation
…dels Register a sibling model-selector entry for every model the host declares text-only: the variant declares image input, so pasted images keep the native attachment flow (composer thumbnail, durable session image), and the variant's stream rewrites image blocks into Vision Toolkit descriptions only on the wire before delegating to the upstream route. The browser paste interception now asks the host before taking a paste over, and only converts to workspace paths when the current model is confirmed text-only. Add imageInputVariants config (enabled, providers), a same-origin paste-policy route, and unit coverage for the adapter, conversion, cache, sweep, and client verdict.
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
…rough, cache invalidation - Paste verdict: the model-selector label is now the authoritative route fact (the session header only updates on a request), with a label-keyed host cache invalidated on topology changes; the client sends the label and keys its verdict cache by session+label, so a model switch can never serve a stale takeover answer. - Sweep: release wrappers when restrict narrows or a route loses every eligible model; re-check disposal before registering. - resolveModel: carry context, defaultMaxTokens, and reasoning metadata through the variant so selectors and effort configs behave like upstream. - EvidenceCache: clear on runtime-instance change (reconfigured provider). - Paste-policy route: forward the model label; contain handler rejections. - Client: a 404 leaves the verdict unconfirmed and retries on next focus instead of standing down for the page lifetime. - Reuse the host's contentHasImage; tests for all of the above (209 total).
…pinned tests - labelTakeoverVerdict: an unreadable route catalog now vetoes the takeover instead of silently dropping that provider's vote — the unreadable route is exactly where an image-capable twin could hide. - Sweep: coalesce adapters-updated bursts into one extra pass and skip probing our own variant routes (they only re-probe their upstream). - Paste resolver: cache label misses too, so a label matching nothing stops paying a full catalog walk per paste. - Client tests: the 404-recovery test now actually pins the recovery (route comes back -> focus re-asks -> next paste is taken over), and the variant-label test drives the host veto by label instead of an unconditional stub. - Web spec: duplicate model query is refused with 400.
… sweep coalescing tests, docs - labelTakeoverVerdict: log the unreadable route so a broken provider is diagnosable; the veto stays (no header fallback here, since the header may describe a previous model after a switch). - Client: the variant-veto test now lets the veto response land before pasting, so 'native' is the host's verdict and not the unconfirmed default. - Sweep: tests pin burst coalescing and mid-sweep re-arm. - README (en/zh): document the label-authoritative verdict and the stale header fallback window; JSDoc the selector-label heuristic and the paste/media-type degradation.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Closes the paste-experience gap for text-only models: pasting an image into the DSH Web composer currently turns it into a text reference (capsule in the composer, plain path text in history), while the native flow keeps the image thumbnail and the durable session image.
This PR adds image-input variants:
<model> (Vision Toolkit)under a matching provider group. The variant declares image input, so pastes keep the native attachment flow — composer thumbnail, durable session image, history rendering — and the variant'sstream()rewrites image blocks into Vision Toolkit descriptions only on the wire before delegating to the upstream route. The session log is untouched; replay and the UI keep the real image.GET /_dsh/vision-toolkit/paste-policy?sessionId=&model=): pastes stay native for image-capable models (including the variants) and are converted to workspace paths only for models the host confirms text-only. The verdict uses the model-selector label the browser shows (authoritative — the session header only updates on a request), with a documented stale-header fallback when no label is readable.imageInputVariants.enabled(defaulttrue) andimageInputVariants.providers(restrict wrapped routes), validated and applied live.Conversion failures degrade to an explanatory wire block instead of failing the turn; descriptions are cached per attachment (64-entry LRU, concurrent readers join, failed reads are not memoized, cache cleared when the runtime reconfigures).
Files
src/image-input-variants.ts(new) — variant adapter, request-time image→description conversion (top-level andtool-result-nested), evidence cache, route sweep (releases on restrict narrowing / lost eligibility / vanished routes, coalesced on topology bursts), label- and header-based paste verdicts with a cached resolver.src/config.ts—imageInputVariantsschema + validation.src/web.ts,src/paste-images.ts,src/web-request.ts— same-originpaste-policyroute forwarding the selector label.src/client/paste-images.tsx— host-verdict-gated paste interception keyed by session+label, focus prefetch, 404 retry.src/index.ts— installer wiring + live settings reconciliation.tests/image-input-variants.spec.ts(new), updatedpaste-images-client.spec.ts,web.spec.ts,config.spec.ts.Verification
pnpm test: 214 passed (baseline 167), 1 pre-existing skip.tsc(host + both client projects) clean;pnpm run buildproduces clean artifacts;git diff --exit-code -- libclean.DSH_HOME): bundle installs and mounts; routing the headless session throughvision-toolkit-deepseek-officialreaches the upstream auth request (AUTH, same as the plain route) while an unregistered provider yieldsNO_ADAPTER— proving the variant is registered and delegates end-to-end on a real host.Browser path note
The paste/thumbnail flow is a Web profile behavior. This environment's running
dsh web(port 3080) serves the released 0.1.7 bundle; swapping in this branch requires restarting that profile, which interrupts the live GUI session, so the browser pass was deferred — the wire-level behavior is covered by the real-profile smoke above plus the client suite (jsdom, real Cordis context, real paste events, real selector-label DOM).