Skip to content

feat(editor): y-prosemirror binding + markdown sidecar (PRSM-56)#14

Merged
nchopra merged 13 commits into
mainfrom
yjs
May 15, 2026
Merged

feat(editor): y-prosemirror binding + markdown sidecar (PRSM-56)#14
nchopra merged 13 commits into
mainfrom
yjs

Conversation

@nchopra

@nchopra nchopra commented May 15, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Replaces the lossy Y.Text("content")-as-JSON-string sync with TipTap's Collaboration extension bound to a real Y.Doc via y-prosemirror. Editor now has a true CRDT binding instead of a per-keystroke JSON re-serialize-and-stuff.
  • Repurposes notes.content as a one-way debounced markdown sidecar (1.5s after last edit) — used for export, search, RAG. The authoritative state lives in yjs_updates.
  • Fixes a long-standing race in compaction: compactUpToId(noteId, maxId, encoded) deletes only updates at or below a high-watermark id, so a renderer save concurrent with a compaction is never destroyed. Includes a regression test that demonstrates the original replaceYjsUpdates was lossy.
  • Adds snapshot scaffolding: note_snapshots table (self-contained encoded Y.Doc state, independent of yjs_updates compaction), saveNoteSnapshot, and pruneNoteSnapshots with two-axis retention (age + count, manual snapshots exempt). No capture sites wired — PRSM-58 builds the UI on top.

What's NOT in this PR

  • Hocuspocus / multi-device sync — Phase 2.
  • Snapshot UI / capture sites — PRSM-58 (the scaffolding here is purely the data layer + helpers).
  • Existing yXmlFragmentToProsemirrorJSON usage in collect-input.ts is @deprecated in y-prosemirror 1.3.7. Functional today; cleanup follow-up: migrate to yXmlFragmentToProseMirrorRootNode(fragment, schema).toJSON().
  • COLLAB_FRAGMENT_NAME constant currently lives in markdown-to-ydoc.ts; a more neutral home (editor-constants.ts) is a follow-up.

Architectural notes for reviewers

  • db/notes.ts YJS + snapshot functions now take db: DB as first parameter, matching the pre-existing pattern in db/artifacts.ts and db/skills.ts. Avoids a singleton in tests.
  • NoteSyncProvider keeps a 150ms batched persister (Y.mergeUpdates) and a 1.5s markdown debouncer. destroy() flushes BOTH before setting destroyed = true (load-bearing — both flush methods short-circuit on the guard).
  • Collaboration.configure({ document, field: COLLAB_FRAGMENT_NAME }) and markdownToYDocUpdate both reference the same "default" fragment constant. StarterKit undoRedo is disabled only when a Y.Doc is provided — non-collab callers (artifact editor) keep default undo.
  • Compaction triggers opportunistically on every save when row count > 100. Bounded by the high-watermark, so concurrent writes survive.
  • note_snapshots rows store the full encoded Y.Doc state (not a marker on yjs_updates), so aggressive compaction never invalidates a historical snapshot. Retention: 30 days / 50 newest per note, manuals never pruned.

Migrations included

  • 0011_overrated_bushwacker.sql — creates note_snapshots table + composite index on (note_id, created_at). Drizzle regenerated an unrelated artifacts index in the same generation pass; that duplicate was hand-removed so the migration applies cleanly.

Test plan

  • Unit: 343 vitest tests pass (pnpm --filter @prismical/desktop test). 18 new tests added across the 13 commits (compaction race regression, save-failure retry, markdown helper, sync-provider batching/debounce/destroy, artifact serializer regression, snapshot save round-trip, snapshot-independence-from-compaction, retention policies).
  • pnpm --filter @prismical/desktop type:check clean.
  • Manual smoke: dev DB wiped, app restarted, fresh note typed/closed/reopened — content survives. Onboarding seed loads with bullet content. Markdown sidecar converges within ~1.5s.
  • DB integrity verified post-edit: 29 yjs_updates rows decoded via Y.applyUpdateY.XmlFragment("default")yXmlFragmentToProsemirrorJSON produced JSON whose top-level shape (heading, paragraphs, artifact node with full attrs) matched the markdown sidecar.
  • Reviewer to spot-check: open a note with an artifact, run a skill, confirm the freshest (post-keystroke) note state is fed to the skill — proves Task 9's Yjs-replay path.
  • Reviewer to spot-check: ⌘Z works after editing — proves Collaboration's Y.UndoManager is wired correctly (StarterKit's undo is disabled).
  • Pre-merge: wipe dev DB again on review machine — old rows from before this branch are stale Y.Text JSON and will not decode under y-prosemirror. Repo memory says "delete prismical.db is a valid fix" pre-launch.

Linked

  • Spec: locally — docs/superpowers/specs/2026-05-14-prsm-56-yjs-pivot-design.md
  • Plan: locally — docs/superpowers/plans/2026-05-14-prsm-56-yjs-pivot.md
  • Follow-up: PRSM-58 (snapshot UI on top of the scaffolding shipped in this PR)

nchopra added 12 commits May 14, 2026 15:30
…-56)

Replace replaceYjsUpdates (full DELETE + INSERT) with compactUpToId that
deletes only rows with id <= maxId so concurrent renderer writes landing
after the snapshot are preserved; refactor the five YJS db functions to
take an injected db handle matching the artifacts.ts pattern.
Rewrites materializeNoteContent to replay yjs_updates rows into a transient
Y.Doc and read the canonical XmlFragment via yXmlFragmentToProsemirrorJSON,
replacing the stale Y.Text-as-JSON approach. Falls back to notes.content +
stripMarkdownLite for notes with zero Yjs rows. Updates extractPlainText to
accept ProseMirror JSON objects in addition to legacy Lexical JSON strings.
@nchopra nchopra requested a review from haritabh-z01 May 15, 2026 00:03
Adds the note_snapshots table (migration 0011), saveNoteSnapshot helper,
and pruneNoteSnapshots with two-axis retention + manual-snapshot protection.
6 new tests covering save round-trip, compaction-independence, and retention
semantics. No UI, no auto-capture, no integration sites (PRSM-58).
@nchopra nchopra merged commit cf49304 into main May 15, 2026
6 checks passed
@nchopra nchopra deleted the yjs branch May 15, 2026 01:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant