EDITOR_ADAPTER.md

ScrollSplice Editor Adapter

The editor adapter is ScrollSplice's stable, versioned control surface for programmatic inspection and manipulation. It lets Codex use exact document IDs and logical coordinates instead of trying to drag pixels on the screen. The optional authenticated local companion uses the same contract behind a strict browser-owned tool boundary.

The adapter does not add OAuth, OpenAI, a backend, credentials, or autonomous behavior. The human editor remains fully independent. Binary file selection and downloads remain host-mediated browser actions because JSON commands should not receive raw filesystem handles. An authorized host may now pass generated PNG/JPEG/WebP bytes as a Blob, base64 data URL, or raw base64 payload through the two asynchronous generated-asset commands.

Development browser access

Run the development app, open the browser developer console, and inspect the current editor:

const editor = window.scrollSpliceEditor
const snapshot = editor.inspect()
snapshot.episode
snapshot.planes
snapshot.elements

The browser bridge exists only in Vite development builds. The production-capable model tool dispatcher imports createEditorAdapter directly, validates network JSON, binds mutations to the inspected episode ID and revision, and rejects unknown or lifecycle commands. It never depends on a writable browser global.

Commands that persist generated bytes use await editor.executeAsync(...). Existing document and navigation commands remain synchronous through editor.execute(...).

Safe manipulation loop

Always use this sequence:

1. Call inspect().
2. Record snapshot.episode.id and snapshot.editor.currentRevision.
3. Find the target by its returned stable ID. Never invent or infer an ID from visible text alone.
4. Issue one execute(command). Network-dispatched mutations must also supply the recorded episode ID and expected revision.
5. Check result.ok, result.changed, and the returned snapshot.
6. Use undo if the result is valid but not what the creator wanted.

All positions and bounds use logical episode pixels. The episode is normally 800 px wide. Element coordinates are absolute within the full scroll; viewport coordinates only control what is currently visible.

set-active-plane and every plane-targeted creation command resolve the plane's composition group from its stable ID, activate that group first, verify that the requested plane became active, and only then continue. Callers do not need to issue a separate set-active-group; if activation fails, the adapter returns a rejected result before creating content on any fallback plane.

Common examples

Select and reveal an element:

editor.execute({
  type: 'select-element',
  elementId: 'element-id-from-inspect',
  reveal: true,
})

Move and resize it:

editor.execute({
  type: 'move-element',
  elementId: 'element-id-from-inspect',
  position: { x: 72, y: 1640 },
})

editor.execute({
  type: 'resize-element',
  elementId: 'element-id-from-inspect',
  bounds: { x: 72, y: 1640, width: 656, height: 420 },
})

Change opacity, layer plane, and stacking:

editor.execute({
  type: 'set-element-opacity',
  elementId: 'element-id-from-inspect',
  opacity: 0.65,
})

editor.execute({
  type: 'move-element-to-plane',
  elementId: 'element-id-from-inspect',
  planeId: 'ordinary-plane-id-from-inspect',
})

editor.execute({
  type: 'move-element-in-stack',
  elementId: 'element-id-from-inspect',
  direction: 'forward',
})

editor.execute({
  type: 'reorder-element-in-stack',
  elementId: 'element-id-from-inspect',
  targetIndex: 0,
})

Create a content plane and then place text:

const planeResult = editor.execute({ type: 'create-plane', group: 'content' })
const planeId = planeResult.ok ? planeResult.createdId : null

if (planeId) {
  editor.execute({ type: 'create-text', planeId })
}

Create precisely positioned editable lettering in one operation:

editor.execute({
  type: 'create-positioned-text',
  planeId: 'ordinary-plane-id-from-inspect',
  bounds: { x: 395, y: 145, width: 300, height: 80 },
  input: {
    text: 'CAUGHT YOU!',
    fill: '#111111',
    fontSize: 42,
    fontWeight: 700,
    align: 'center',
  },
})

The authenticated JSON tool presents the same operation as planeId, bounds, text, and a style object with fontSize, fontWeight, color, and textAlign. The dispatcher normalizes that public shape before calling the adapter.

Create a positioned shape with its complete initial style:

editor.execute({
  type: 'create-positioned-shape',
  planeId: 'ordinary-plane-id-from-inspect',
  name: 'Inset panel',
  bounds: { x: 64, y: 520, width: 672, height: 400 },
  shape: 'rectangle',
  fill: '#ffffff',
  stroke: '#111111',
  strokeWidth: 4,
  cornerRadius: 0,
})

Create an empty editable balloon body on an explicit plane:

const balloon = editor.execute({
  type: 'create-speech-balloon',
  planeId: 'ordinary-plane-id-from-inspect',
  presetId: 'wavy',
})

The stable preset IDs are standard, rounded, thought, whisper, shout, electric, rough, wavy, telepathic, and double-outline. New balloons contain no lettering; create a separate Text element when words are needed. update-speech-balloon accepts the complete current balloon property object and may include 6–32 normalized { x, y } bodyControlPoints for direct contour editing.

Undo:

editor.execute({ type: 'undo' })

Import generated image bytes and retain their provenance:

const imported = await editor.executeAsync({
  type: 'import-generated-asset',
  source: {
    kind: 'base64',
    base64: imageResultBase64,
    mediaType: 'image/png',
  },
  metadata: {
    displayName: 'Moonlit garden panel.png',
    provider: 'OpenAI',
    model: 'the-model-used-by-the-host',
    prompt: 'A moonlit garden in a vertical comic panel.',
  },
})

The same command accepts a Blob directly, { kind: 'blob', blob }, or { kind: 'data-url', dataUrl }. On success, createdId is the stable Asset Library ID and the returned snapshot already includes the new source metadata.

Place the generated source on an explicit ordinary plane with logical bounds:

const placed = await editor.executeAsync({
  type: 'place-generated-asset',
  assetId: imported.createdId,
  planeId: 'plane-id-from-inspect',
  bounds: { x: 72, y: 1640, width: 656, height: 420 },
})

The placement result returns the stable element ID and an updated snapshot. Placement is one document-history entry; Undo removes the placed instance while retaining the reusable generated source in the Asset Library.

Snapshot contents

inspect() returns JSON-safe data only:

• episode ID, title, format, and logical dimensions
• viewport, zoom, active group/plane, selection, history, and dirty state
• all three composition groups and their visibility
• ordered layer planes with stable IDs, element counts, and the pinned plane's base color where applicable
• every element's ID, type, plane, logical bounds, visibility, stacking, opacity, blend mode, transform, overflow, stable asset reference, text content where applicable, and element-group membership
• complete persisted type-specific properties for shapes, independent text, editable balloon bodies/tails, and image presentation/frame/mask/crop settings
• flat element groups with their stable member IDs
• built-in and imported asset metadata without blobs, object URLs, or filesystem handles; generated sources include provider, model, prompt, and generation timestamp provenance

Command families

• Navigation: set-active-group, set-active-plane, set-viewport, pan-viewport, set-zoom.
• Selection and organization: select/clear/select-all, group/ungroup, story-beat movement, alignment, one-step or direct-index stacking, and Move to Plane.
• Episode and planes: rename/extend/resize the episode; create/delete/rename/reorder/show/hide planes and groups; change the base color.
• Elements: create, select, rename, move, resize, duplicate, delete, show/hide, lock, opacity, blend, transforms, flips, and overflow.
• Type-specific editing: exact positioned shape/text creation, shape fill/style, independent text updates, ten empty editable balloon types plus normalized contour points, image presentation/frame/crop, and Background color regions.
• Assets: place an existing built-in or imported asset by stable ID.
• Generated assets: asynchronously import validated PNG/JPEG/WebP bytes with provenance, then place a generated source on a specified ordinary plane and logical bounds.
• Editor/project: magnet, slice guides, undo, redo, Save, Save As, Reopen, New Episode, and Reset Demo.

Use editor.commandTypes for every version-2 name and editor.asyncCommandTypes for commands that must be awaited. Version 2 adds generated-source provenance plus the two asynchronous generated-asset commands; existing synchronous version-1 command names and behavior remain intact. Type-specific update commands require the complete validated property object used by the corresponding human inspector.

Errors and boundaries

execute never throws for an ordinary targeting or validation failure. It returns ok: false with one of:

• not-found: the supplied stable ID does not exist
• wrong-type: the command does not apply to that entity
• invalid-command: a supplied option is unsupported
• invalid-source: generated bytes or their encoding are malformed or unsupported
• rejected: the normal editor command refused the operation

The adapter deliberately refuses raw Zustand setters, React/Konva objects, credentials, direct browser-storage access, WEBTOON automation, and filesystem handles. A model or image-generation host obtains the bytes; import-generated-asset validates and stores them through the normal Asset Library repository. Exporting or downloading files remains a separate authorized host/browser action.

The browser Asset Library—not an arbitrary filesystem folder—is the durable storage boundary. It keeps the unchanged generated source Blob in IndexedDB, and portable .scrollsplice projects retain both the source bytes and generation metadata. A future native or user-authorized folder export may mirror those bytes, but it is not required for an agent to place them on the canvas.

Persistent chat UI boundary

The chat bubble is application UI, not part of the editor adapter. It should be a small fixed launcher that toggles an overlay panel above the workspace without resizing or embedding itself in the canvas. Closing the panel hides it; it does not end the conversation or discard an active project history.

Conversation memory belongs in a separate browser-local AgentConversationRepository, keyed by the stable episode project key. It retains only user and finalized assistant text in local storage. Chat messages, authorization, Codex thread IDs, tool requests, and run status never enter the episode document, portable project, recovery, or Asset Library. The chat/runtime layer imports the bounded dispatcher, calls inspect(), runs approved commands, and renders returned results; Zustand subscriptions continue to update the ordinary canvas, minimap, and Layers UI.

Authenticated local model use

The local companion exposes exactly five model-visible operations: inspect the editor; preview the authoritative rendered episode; apply one runtime-validated reversible editor command; import the companion's latest staged generated source; and place an imported generated asset on an explicit ordinary plane and bounds. preview_editor is read-only and returns a bounded JPEG plus its rendered and logical dimensions alongside the current structural snapshot. The companion sends that image to the App Server as inputImage, allowing the model to check visual relationships inside flattened artwork—such as a blank speech balloon—without granting DOM, canvas, filesystem, or store access. Every mutation requires the inspected episode ID and revision. Project lifecycle actions such as New, Save, Reopen, and Reset remain human-owned. The companion enforces cancellation and keeps provider credentials outside the browser bundle, episode document, adapter inputs/results, logs, and git. It may pass only generated bytes plus non-secret provenance into the existing generated-asset intake seam; it never passes an OAuth token or reusable credential.