feat(datatables): datatable migrations (GIT-890)

[diegoimbert]

Summary

Implements datatable migrations (Fixes GIT-890): schema changes to datatables are recorded as versioned SQL migration scripts that are tracked per datatable, synced to git as plain .sql files, applied by a migration runner, and carried across workspace forks — replacing the previous best-effort schema-diff generation on fork.

How it works

Storage & runner (backend)

• New datatable_migration table: versioned SQL scripts per (workspace, datatable) with name, content, checksum.
• Applied state is tracked inside the datatable's backing database in a _windmill_datatable_migrations table (like sqlx/flyway). Since a fork copies the database, the fork automatically sees the same migrations as applied — no state reconstruction needed.
• Runner applies pending migrations in version order; each migration runs in its own transaction together with its tracking insert (all-or-nothing per migration).
• Workspaced API under /datatable_migrations/{datatable}: list (with applied/drifted status), create, upsert, delete, run — declared in openapi.yaml, consumed via generated clients (frontend + CLI).

REPL warning modal (frontend)

• Running a query containing DDL (CREATE/ALTER/DROP/TRUNCATE/…) against a datatable in the SQL REPL surfaces a "Schema change detected" modal.
• Primary action "Save as migration" records the query as a migration (optional name) and applies pending migrations. "Run once without saving" remains as an escape hatch.
• Detection lives in a unit-tested sqlDdl.ts helper (leading-verb match only — SELECT 'create table' is not flagged).

CLI sync (_datatable_migrations/ folder)

• Migrations round-trip through git as _datatable_migrations/{datatable}/{version}_{name}.sql.
• wmill sync pull writes them from the workspace tarball export; wmill sync push upserts added/edited files by version and deletes removed ones.
• Run on git merge: after a push that touches migration files, pending migrations are applied to each affected datatable — so merging new migration files into a git-synced branch runs them on the target workspace.
• Migration files are excluded from the script push path and lockfile metadata generation (plain .sql is not a script language).

Fork behavior (replaces best-effort diff)

• create_workspace_fork no longer snapshots the parent datatable schema; the DatatableSchemaDiff three-way diff + generated ALTER/CREATE SQL component is removed.
• Migration scripts are cloned to the fork; the Compare & Deploy page now shows a Datatable migrations panel listing each datatable's migrations with applied/pending/drifted status and a "Run pending migrations" action.

Edit-after-apply warning (secondary requirement)

• Checksums detect when a stored migration was edited after having been applied (drifted). The UI shows an "edited after apply" badge (visible in the screenshot above), and wmill sync push logs a warning that the edited content will NOT be re-run. Checksums ignore trailing whitespace so editor-appended final newlines don't count as drift.

Validation

• cargo check (windmill-api, windmill-api-workspaces) — clean; SQLX_OFFLINE=true build passes (cache updated via the safe procedure, zero EE-cache deletions).
• Frontend npm run check — 0 errors; vitest sqlDdl tests pass; svelte-autofixer applied (keyed each blocks).
• CLI bun test unit suite — 815/815 pass (incl. new path-parsing tests); tsc introduces no new errors (pre-existing baseline verified by stash-diff).
• End-to-end in a real browser + real Postgres:
  • REPL: CREATE TABLE → modal → "Save as migration" → migration stored, runner applied it, tracking row written; "Run once without saving" executes without recording; SELECT unaffected.
  • CLI: sync pull writes the .sql file; adding a file + sync push upserts and auto-applies it (table created); deleting the file + push removes the remote migration; editing an applied migration + push logs the drift warning; re-push after workspace-side row loss correctly reports "no pending migrations" (applied state survives in the datatable DB).
  • Fork: created via API → migration rows cloned; Compare & Deploy renders the migrations panel with correct status badges.

Notes

• The _windmill_datatable_migrations tracking table is visible in the DB manager like any other table (same as flyway/sqlx conventions).
• DataTableForkedFrom.schema is kept as a legacy optional field so existing settings JSON still round-trips; it is no longer written.
• Migration create/upsert/delete/run require workspace admin.

🤖 Generated with Claude Code

[cloudflare-workers-and-pages]

Deploying windmill with    Cloudflare Pages

Latest commit:	5856435
Status:	✅  Deploy successful!
Preview URL:	https://3fc7b62a.windmill.pages.dev
Branch Preview URL:	https://diego-git-890-datatable-migr.windmill.pages.dev

View logs