F15 — Drawing Control API for plugins, automation, and AI assistance

[Flow-Fly]

Status

Deferred later feature. This is an architectural and product-design tracker, not an implementation-ready issue.

Do not split or grant ready-for-agent until the internal editor-action boundary has been exercised by real product work, with F14 / #128 as a useful proving ground.

Product opportunity

Expose Pixel Forge's editing capabilities through a stable semantic API so that:

• first-party and third-party plugins can extend the editor;
• users can build macros and repeatable drawing workflows;
• an optional AI assistant can teach pixel-art, explain changes, draw with the user, or create and animate content;
• automation can control pixels, palettes, layers, frames, selections, and history without simulating pointer gestures.

The AI assistant is one possible client of the API. It must not define or own the core editor contract.

Architectural direction

1. Internal editor actions

Build an in-process action layer over ProjectContext, commands, and history.

Actions describe intent, for example:

• inspect a project or a bounded region;
• write or replace pixels;
• fill or transform a region;
• add, remove, reorder, or update layers;
• add, remove, reorder, or update frames and cels;
• read or change the palette;
• manage selections;
• group several edits into one undoable transaction;
• preview, commit, undo, or cancel a transaction.

Tools, plugins, and AI should call semantic actions. They should not reach into stores directly and should not remote-control mouse events.

2. Stable observations

Expose serializable, bounded project observations:

• project dimensions and metadata;
• stable layer, frame, and cel identifiers;
• palette and indexed pixel data;
• selected or visible ranges;
• thumbnails or bounded raster previews when needed;
• change events or revision identifiers.

Large image data must be requested explicitly and within practical bounds.

3. Transactions and safety

Every mutation must:

• target an explicit project context;
• validate identifiers, bounds, layer capabilities, and palette indices;
• be atomic and undoable;
• support grouped transactions;
• produce an inspectable before/after summary;
• avoid leaving partially applied changes after errors or cancellation.

4. Capabilities and permissions

Clients receive only declared capabilities, such as:

• read project;
• read pixels;
• write pixels;
• modify layers;
• modify animation;
• import or export files;
• access the network;
• use an external AI provider.

Third-party code must not gain arbitrary DOM, store, filesystem, credential, or network access through the drawing API.

5. Versioned adapters

Keep the editor-action core independent from its clients. Later adapters may include:

• first-party macros and automation;
• trusted local plugins;
• sandboxed third-party plugins;
• an AI tool adapter;
• a developer console or testing harness;
• optionally, a remote protocol such as MCP if a real use case justifies it.

No remote protocol is selected by this issue.

Proposed delivery phases

1. Internal action boundary

   • Consolidate semantic reads and undoable mutations.
   • Add transaction and observation tests.
   • Keep it private and free to evolve.

2. Trusted automation

   • Use the boundary for first-party macros or a real product feature.
   • Establish action names, errors, and revision behavior through use.

3. Plugin host

   • Define lifecycle, manifest, capability grants, subscriptions, compatibility policy, and sandbox.
   • Start with a small documented SDK rather than exposing the whole application.

4. AI assistant

   • Adapt the same observations and actions into model tools.
   • Offer distinct modes: explain, suggest with preview, and execute with confirmation.
   • Apply one reviewable undoable transaction at a time.
   • Keep provider choice replaceable and make external data transfer explicit.

AI experience principles

• The assistant should be useful without taking control away from the artist.
• Teaching and critique should work without granting write access.
• Proposed edits should be previewable before commit.
• Executed edits should remain visible in history and be undoable normally.
• The user chooses whether project data can leave the device.
• Destructive, broad, or expensive operations require explicit confirmation.
• Generated work should not silently overwrite hand-drawn work.
• Accessibility and keyboard workflows must remain available without the assistant.

Non-goals for the first phase

• a plugin marketplace;
• arbitrary JavaScript access to application internals;
• direct pointer or keyboard simulation;
• autonomous background editing;
• an AI-provider commitment;
• a public remote API before the in-process contract is proven;
• real-time collaboration or multi-user conflict resolution.

Dependencies and gates

• Use real feature consumers before freezing the public interface; F14 / F14 — Paint-by-number: guided pixel-art creation #128 can expose pixel-validation and transaction seams.
• Normalize remaining direct canvas/store mutation paths behind commands or actions where necessary.
• Decide the plugin isolation model before third-party execution.
• Complete a privacy and security review before any AI provider receives project content.
• Define browser support and local/offline behavior for plugins and AI separately.

Planning acceptance

This tracker is ready to split only when there is an approved architecture brief covering:

• action and observation schemas;
• transactions, history, cancellation, and errors;
• capability grants and plugin isolation;
• API versioning and compatibility;
• project-data privacy and external-provider consent;
• performance bounds for pixel reads and writes;
• a narrow first-party consumer proving the boundary.

Relationship to other roadmap features

• F14 / F14 — Paint-by-number: guided pixel-art creation #128 can help prove the internal pixel-action and validation seams.
• F5 share links and F6 version history may later improve previews and review workflows, but are not prerequisites for the internal action layer.
• F2 accounts and sync are not prerequisites.