ClipKnit Canvas

A minimal, self-hostable audio/video editor built around a freeform canvas instead of a fixed timeline. Clips are cards you drop anywhere on an infinite, pannable/zoomable surface; nudge them edge-to-edge and they snap into chains that play and export as one continuous piece. There are no locked, linear tracks — lay clips out however you think, and let adjacency be the timeline.

Everything happens on that one blank canvas with a single top toolbar: upload clips, arrange and trim them into chains, play the result gaplessly, and export it as one file.

No build step: the frontend is plain ES modules served by a FastAPI backend. The browser only edits an EDL (edit decision list); all heavy media work (probing, waveform/filmstrip generation, proxy transcoding, compositing) happens server-side where ffmpeg lives.

Status: v0.1.0 — the editor core plus an async job framework (exports run as background jobs with live progress: SSE push, poll fallback).

Features

• Freeform canvas. Clips live anywhere on a pannable, zoomable canvas. Scroll / two-finger drag (or drag empty space) to pan; Ctrl/⌘+wheel or trackpad pinch to zoom toward the cursor; minimap slider + Fit too. One-finger drag pans on touch.
• Snap into chains. Drag a clip near another's edge and it snaps to attach; attached clips form a chain that travels together (grabbing a mid-clip carries its right-tail). A chain is what plays / exports.
• Mode-locked. The first clip sets the project to video-only or audio-only (toolbar pill shows the mode); a video clip's audio is tied to it.
• Visual clips. Audio clips show a normalised waveform; video clips show a filmstrip with the waveform beneath. Peaks + thumbnails are generated by ffmpeg on upload and cached on disk.
• Edge-trim with re-expand. Drag a clip's edges to shorten it and back out to its original length; the chain's tail follows so it stays attached. A trim view (double-click a clip) gives precise in/out editing against the whole source with a filmstrip/waveform track and region playback.
• Gapless, media-clock playback. Select a clip → play its chain to the end. Two media elements ping-pong (the next clip is pre-decoded on the hidden one), and the playhead is driven by the playing element's own clock — so no load-time jump, no A/V drift, no black frames at cuts.
• Proxy preview (browser-agnostic, VPN-friendly). Playback streams a small, universally-decodable H.264 proxy (480p in the corner monitor by default, generated by ffmpeg on upload and cached) rather than the heavy original — so preview works in any browser and over limited bandwidth. The monitor has player controls (play/stop, scrubber, buffered indicator) and is resizable; a Fullscreen button plays the original quality. Export always uses the originals.
• Light / dark theme, applied before first paint and remembered across sessions.
• Export dialog. Server-side ffmpeg composites the selected chain into one file — video as mp4 / webm / gif, audio as m4a / mp3 / wav, with resolution and quality options — as an async job with a progress tray, then downloads the result. Undo/redo, copy/paste, and a project switcher round out the editing basics.
• Multi-user ready. Behind a forward-auth reverse proxy (e.g. Caddy + Authentik) each signed-in user gets a fully isolated workspace — own projects, media, and jobs, enforced server-side. Without a proxy it runs in single-user mode.

Run

docker compose up --build      # → http://127.0.0.1:3011

Data (uploads, renders, SQLite) lives in the clipknit-data volume.

The default compose runs single-user (CLIPKNIT_ALLOW_NO_AUTH=1) — no reverse proxy or identity provider needed; everything belongs to one local user. For multi-user, put a forward-auth proxy in front that strips client-supplied X-Authentik-* headers and injects the real X-Authentik-Uid, and unset CLIPKNIT_ALLOW_NO_AUTH — with neither the header nor the opt-in, requests are rejected (fail-closed). Local deployment tweaks belong in a docker-compose.override.yml (gitignored).

Develop

No build step — edit web/ and app/ directly (the Docker image bakes web/ in, so rebuild to see frontend changes in the container).

python3 -m venv .venv && .venv/bin/pip install -r requirements-dev.txt
.venv/bin/python -m pytest        # backend suite (ffmpeg/ffprobe are mocked)
npm test                          # frontend geometry suite (node --test, Node 18+, no deps)

Configuration (env)

Var	Default	Notes
CLIPKNIT_DATA_DIR	/data	uploads, renders, sqlite
CLIPKNIT_MAX_UPLOAD_BYTES	2147483648	0 = unlimited
CLIPKNIT_FFMPEG / CLIPKNIT_FFPROBE	ffmpeg / ffprobe	point at a jellyfin-ffmpeg build for NVENC
CLIPKNIT_VIDEO_CODEC	libx264	set h264_nvenc on the GPU deploy
CLIPKNIT_RENDER_WIDTH / _HEIGHT / _FPS / _SR	1280 / 720 / 30 / 44100	render target
CLIPKNIT_PEAKS_PER_SEC / CLIPKNIT_PCM_SR	100 / 8000	waveform resolution
CLIPKNIT_STRIP_H / _STRIP_FPS / _STRIP_MAX_FRAMES	60 / 1.0 / 240	filmstrip thumbnails

NVENC (GPU hardware export)

The default image bundles CPU ffmpeg. For hardware export, run under the nvidia runtime with NVIDIA_DRIVER_CAPABILITIES including video, mount/point CLIPKNIT_FFMPEG at an NVENC-capable ffmpeg, and set CLIPKNIT_VIDEO_CODEC=h264_nvenc.

API sketch

• POST /api/media?project_id= (multipart upload) · GET /api/media · DELETE …
• GET /api/media/{id}/file (Range) · /proxy/{h} · /peaks · /strip.json · /strip.jpg (derived assets, lazily generated)
• GET/POST/PUT/DELETE /api/projects[/{id}] — v2 EDL ({version, mode, clips}) stored as JSON
• POST /api/projects/{id}/clips — append a clip, enforcing the mode lock (409 on mismatch)
• POST /api/jobs ({type:"render", project_id, params}) · GET /api/jobs/{id} — export runs as an async job
• GET /api/jobs/events — one SSE stream of all your jobs' progress (push; the client falls back to polling)
• See OVERVIEW.md §"API surface" for the full table (sessions, sweep, /api/me, health)

License

0BSD.