Unify Cogito and document analysis with resilient local workflows

.github/copilot-instructions.md

@@ -6,6 +6,10 @@
 - Keep `src/app.ts` as a tiny entrypoint and preserve single-file build output (`ink-app.html`).
 - Local git hooks are tracked under `.githooks/`; keep lint-plus-repomix pre-commit automation there and point setup docs at `git config core.hooksPath .githooks` instead of ad hoc hook creation snippets.
 - Regression coverage for refactor-sensitive flows lives in `cypress/e2e/workspace-actions.cy.js` and `tests/qunit/fs-api.test.js`.
+- Workspace opening must call `showDirectoryPicker()` directly from the user action, guard overlapping picker requests, coalesce duplicate rescans, and hydrate Markdown metadata/tags through the bounded worker pool in `src/app/workspace-io.ts` rather than reading every file serially or more than once.
+- Foreground note opens must cancel obsolete scan generations, and auto-refresh must remain idle-gated and low-frequency so background vault hydration cannot starve interactive file reads.
+- Unit coverage for auto-refresh I/O gating lives in `tests/qunit/auto-refresh.test.js`; preserve the invariant that recent interaction and dirty state return before permission queries or rescans. Cypress coverage in `workspace-actions.cy.js` verifies foreground note opens cancel slow scan generations.
+- Progressive large-vault loading is proposed in `openspec/changes/add-progressive-workspace-loading/`. Do not implement it before approval; its invariants are separate bounded directory/file queues, batched initial-open publication, atomic refreshes, and generation-token cancellation.
 - The app controller is in `src/app/app-controller.ts`, with UI wiring in `src/app/ui-events.ts` and workspace flows in `src/app/workspace-io.ts`.
 - Top menu visuals are authored directly in `ink.template.html`; when adding icons to dropdown rows, keep the text in a nested `.menu-label-text` span so dynamic updates like the Sort label do not remove the icon.
 - Keyboard shortcut precedence lives in `src/app/ui-events.ts`; editor-scoped handlers must not swallow longer chords like Cmd/Ctrl+Shift+S that are reserved for export.
@@ -23,13 +27,16 @@
 - Cogito Mode entrypoint must be a top-right menu bar button using a thinking-man glyphicon with accessible Cogito labeling.
 - Cogito runtime integration lives in `src/app/cogito.ts`; keep the prompt contract and JSON parsing helpers (`extractLastSentence`, `parseCogitoQuestionPayload`, `formatCogitoQuestionBlock`) stable and covered by QUnit tests.
 - Cypress end-to-end coverage for Cogito uses a test-only `globalThis.__INK_TEST_WEBLLM__` override; preserve that seam so the full panel/generate/insert flow remains deterministic under test.
+- Cogito configures WebLLM with the official IndexedDB cache backend to avoid brittle Cache API `Cache.add()` failures. Recoverable cache/network errors get one model-specific cleanup/retry for both Lite and Deep, and Deep additionally falls back to Lite; preserve both explicit Cypress scenarios.
 - The WebMCP `create_note` tool is agent-facing only. Keep its form out of the always-visible sidebar UI and surface it as a modal only when the tool is being used.
 - The canonical implementation for the WebMCP note popup lives in `ink.template.html`, `src/styles.scss`, and `src/app/ui-events.ts`; rebuild `ink-app.html` after source edits.
 - For local debugging, the WebMCP popup can be opened manually with `Cmd/Ctrl+Alt+N` or automatically via `?debugWebMcpNote=1`; keep those hooks available without making the tool permanently visible.
-- Document Linter runtime integration lives in `src/app/document-linter/document-linter.ts`. When extending or maintaining the Document Linter, make sure all associated DOM elements (e.g. `documentLinterPanel`, `documentLinterAnalyzeBtn`, etc.) are mapped in `src/app/dom.ts` and defined in `src/app/types.ts` to avoid bootstrapping reference errors.
+- Document Linter runtime integration lives in `src/app/document-linter/document-linter.ts`. Its controls and results render inside `#cogitoPanel`; keep associated DOM elements (e.g. `documentLinterAnalyzeBtn`) mapped in `src/app/dom.ts` and defined in `src/app/types.ts` to avoid bootstrapping reference errors.
 - The Document Linter now emits a markdown-aware review summary with prioritized fixes and section notes. Preserve the `analyzeDocumentText()` and `buildDocumentLinterReport()` contract when adjusting report output so the UI panel and export stay aligned.
 - Document Linter copy now lives in `src/app/document-linter/messages.ts`; keep suggestion/overview strings centralized there, preserve the `overallScore` field, and keep section-analysis `needsAttention` flags aligned between the panel and markdown export.
 - Document Linter panel behavior now includes a `Rerun on change` toggle plus clickable section line links back into the editor. Preserve `handleEditorChanged()` in `src/app/document-linter/document-linter.ts` and keep the panel's line-jump buttons wired to the textarea selection logic when changing the panel UI.
-- Cogito and the Document Linter are mutually exclusive panels. Opening one should close the other so the editor split layout stays stable and never tries to render both sidebars together.
+- Cogito is the single writing-assistance panel and contains both Document strength and Improve with Cogito. Do not restore a standalone Linter button or panel.
+- The unified writing-assistance flow is tracked in `openspec/changes/integrate-document-linter-into-cogito/`. Keep deterministic analysis independent from WebLLM, pass only compact linter findings into coaching, and preserve Cogito's strict three-question JSON contract.
+- Keep Document strength rerun-on-change debounced so analysis happens after typing pauses rather than on every keystroke, and keep Cogito prompt context bounded before invoking WebLLM.
 - Editor view mode lives in `src/app/editor-preview.ts` and is wired through the main editor header in `ink.template.html`. Keep the Source/Split/Preview toggle in sync with `editorSplit`, `editorPane`, and `previewPane`, and preserve the `ink-editor-view-mode` localStorage key when changing that flow.
 - Demo capture is recorded by `build/record-demo.js` using headless Chrome DevTools so Cypress UI is never present; use `npm run demo:record` for a slow, full-viewport app-only `assets/demo/ink-demo.webm` plus MP4 when conversion is available, and `npm run demo:gif` for the optional ffmpeg GIF.

README.md

@@ -71,6 +71,15 @@ The webapp is released as a single HTML file (`ink-app.html`).
 
 Canonical build entrypoint: `build/compile-and-assemble.js` (used by `npm run build`).
 
+## Cogito Writing Assistance
+
+Cogito is the single writing-assistance entrypoint in the top-right menu bar. Its side panel combines:
+
+- **Document strength**: deterministic analysis of clarity, structure, readability, engagement, and section-level priorities. Analysis can be run manually, rerun after edits, navigated by line, and exported as Markdown.
+- **Improve with Cogito**: exactly three question-only coaching prompts grounded in the latest sentence. When document analysis is available, Cogito uses a compact summary of its priorities and strengths to focus those questions.
+
+Document analysis does not require the language model. Cogito's local WebLLM model is loaded only when coaching questions are generated, and generated questions are never inserted without an explicit user action.
+
 ## Branding Assets
 
 - Canonical logo source: `assets/branding/logo.svg`

build/build-test.js

@@ -14,6 +14,10 @@ const bundles = [
     entryPoints: ["src/app/fs-api.ts"],
     outfile: "dist/test/fs-api.js",
   },
+  {
+    entryPoints: ["src/app/auto-refresh.ts"],
+    outfile: "dist/test/auto-refresh.js",
+  },
   {
     entryPoints: ["src/app/utils.ts"],
     outfile: "dist/test/utils.js",

cypress/e2e/cogito-mode.cy.js

@@ -97,7 +97,12 @@ describe("cogito mode", () => {
         win.__fakeWorkspace = root;
         win.__cogitoCreateEngineCalls = [];
         win.__cogitoCompletions = [];
+        win.__cogitoDeletedModels = [];
         win.__INK_TEST_WEBLLM__ = {
+          prebuiltAppConfig: { model_list: [] },
+          async deleteModelAllInfoInCache(modelId) {
+            win.__cogitoDeletedModels.push(modelId);
+          },
           async CreateMLCEngine(modelId, options = {}) {
             win.__cogitoCreateEngineCalls.push(modelId);
             options.initProgressCallback?.({ text: `Stub model ready: ${modelId}` });
@@ -132,7 +137,7 @@ describe("cogito mode", () => {
     });
   });
 
-  it("opens the panel, switches model, generates questions, and inserts one into the editor", () => {
+  it("assesses the document, uses findings for coaching, and inserts one question", () => {
     cy.get("[data-action=\"open-workspace\"]").click({ force: true });
     cy.get("[data-action=\"new-note\"]").click({ force: true });
     cy.get("#editor").clear().type(markdown);
@@ -143,6 +148,20 @@ describe("cogito mode", () => {
       .should("have.attr", "aria-expanded", "true");
     cy.get("#cogitoPanel").should("be.visible");
     cy.get(".split").should("have.class", "with-cogito");
+    cy.get("#documentLinterToggleBtn").should("not.exist");
+    cy.get("#documentLinterPanel").should("not.exist");
+
+    cy.get("#editorViewSourceBtn").click();
+    cy.get("#editorSplit").should("have.class", "view-source").and("have.class", "with-cogito");
+    cy.get("#cogitoPanel").should("be.visible");
+    cy.get("#editorViewPreviewBtn").click();
+    cy.get("#editorSplit").should("have.class", "view-preview").and("have.class", "with-cogito");
+    cy.get("#cogitoPanel").should("be.visible");
+    cy.get("#editorViewSplitBtn").click();
+
+    cy.get("#documentLinterAnalyzeBtn").click();
+    cy.get("#documentLinterStatus").should("contain", "Analysis complete");
+    cy.get("#documentLinterResults").should("contain", "Overall");
 
     cy.get("#cogitoDeepBtn").click().should("have.class", "active");
     cy.get("#cogitoLiteBtn").should("not.have.class", "active");
@@ -160,10 +179,131 @@ describe("cogito mode", () => {
       expect(win.__cogitoCreateEngineCalls).to.deep.equal(["Qwen3-8B-q4f16_1-MLC"]);
       expect(win.__cogitoCompletions).to.have.length(1);
       expect(win.__cogitoCompletions[0].messages[1].content).to.contain(markdown);
+      expect(win.__cogitoCompletions[0].messages[1].content).to.contain("Document strength:");
+      expect(win.__cogitoCompletions[0].messages[1].content).to.contain("Highest-priority improvements:");
     });
 
     cy.contains("#cogitoQuestionList .cogitoInsertBtn", "Insert").first().click();
     cy.get("#statusBadge").should("contain", "Inserted AI question");
     cy.get("#editor").should("have.value", `${markdown}> ### AI\nWhat problem does local-first editing solve here?\n`);
   });
+
+  it("marks existing questions stale when document analysis changes", () => {
+    cy.get("[data-action=\"open-workspace\"]").click({ force: true });
+    cy.get("[data-action=\"new-note\"]").click({ force: true });
+    cy.get("#editor").clear().type(markdown);
+    cy.get("#cogitoToggleBtn").click();
+    cy.get("#documentLinterAnalyzeBtn").click();
+    cy.get("#documentLinterStatus").should("contain", "Analysis complete");
+    cy.get("#cogitoGenerateBtn").click();
+    cy.get("#cogitoQuestionList .cogitoQuestionItem").should("have.length", 3);
+
+    cy.get("#editor").type(" More evidence is needed.");
+    cy.get("#documentLinterAnalyzeBtn").click();
+    cy.get("#cogitoStatus").should("contain", "Regenerate");
+    cy.get("#cogitoQuestionList .cogitoQuestionItem").should("have.length", 3);
+  });
+
+  it("repairs a failed cache and falls back from Deep to Lite", () => {
+    cy.window().then((win) => {
+      win.__INK_TEST_WEBLLM__.CreateMLCEngine = async (modelId, options = {}) => {
+        win.__cogitoCreateEngineCalls.push(modelId);
+        if (modelId === "Qwen3-8B-q4f16_1-MLC") {
+          throw new Error("Failed to execute 'add' on 'Cache': Cache.add() encountered a network error");
+        }
+        options.initProgressCallback?.({ text: `Stub model ready: ${modelId}` });
+        return {
+          chat: {
+            completions: {
+              async create(payload) {
+                win.__cogitoCompletions.push(payload);
+                return {
+                  choices: [{
+                    message: {
+                      content: JSON.stringify({
+                        questions: ["Fallback one?", "Fallback two?", "Fallback three?"],
+                      }),
+                    },
+                  }],
+                };
+              },
+            },
+          },
+        };
+      };
+    });
+
+    cy.get("[data-action=\"open-workspace\"]").click({ force: true });
+    cy.get("[data-action=\"new-note\"]").click({ force: true });
+    cy.get("#editor").clear().type(markdown);
+    cy.get("#cogitoToggleBtn").click();
+    cy.get("#cogitoDeepBtn").click();
+    cy.get("#cogitoGenerateBtn").click();
+
+    cy.get("#statusBadge", { timeout: 10000 }).should("contain", "Cogito questions ready");
+    cy.get("#cogitoLiteBtn").should("have.class", "active");
+    cy.get("#cogitoQuestionList .cogitoQuestionItem").should("have.length", 3);
+    cy.window().then((win) => {
+      expect(win.__cogitoDeletedModels).to.deep.equal(["Qwen3-8B-q4f16_1-MLC"]);
+      expect(win.__cogitoCreateEngineCalls).to.deep.equal([
+        "Qwen3-8B-q4f16_1-MLC",
+        "Qwen3-8B-q4f16_1-MLC",
+        "Llama-3.2-1B-Instruct-q4f32_1-MLC",
+      ]);
+    });
+  });
+
+  it("repairs the Lite model cache and retries Lite successfully", () => {
+    cy.window().then((win) => {
+      let liteAttempts = 0;
+      win.__INK_TEST_WEBLLM__.CreateMLCEngine = async (modelId, options = {}) => {
+        win.__cogitoCreateEngineCalls.push(modelId);
+        if (modelId === "Llama-3.2-1B-Instruct-q4f32_1-MLC") {
+          liteAttempts += 1;
+          if (liteAttempts === 1) {
+            throw new Error("Failed to execute 'add' on 'Cache': Cache.add() encountered a network error");
+          }
+        }
+        options.initProgressCallback?.({ text: `Stub model ready: ${modelId}` });
+        return {
+          chat: {
+            completions: {
+              async create(payload) {
+                win.__cogitoCompletions.push(payload);
+                return {
+                  choices: [{
+                    message: {
+                      content: JSON.stringify({
+                        questions: ["Lite retry one?", "Lite retry two?", "Lite retry three?"],
+                      }),
+                    },
+                  }],
+                };
+              },
+            },
+          },
+        };
+      };
+    });
+
+    cy.get("[data-action=\"open-workspace\"]").click({ force: true });
+    cy.get("[data-action=\"new-note\"]").click({ force: true });
+    cy.get("#editor").clear().type(markdown);
+    cy.get("#cogitoToggleBtn").click();
+    cy.get("#cogitoLiteBtn").should("have.class", "active");
+    cy.get("#cogitoGenerateBtn").click();
+
+    cy.get("#statusBadge", { timeout: 10000 }).should("contain", "Cogito questions ready");
+    cy.get("#cogitoLiteBtn").should("have.class", "active");
+    cy.get("#cogitoQuestionList .cogitoQuestionItem").should("have.length", 3);
+    cy.window().then((win) => {
+      expect(win.__cogitoDeletedModels).to.deep.equal([
+        "Llama-3.2-1B-Instruct-q4f32_1-MLC",
+      ]);
+      expect(win.__cogitoCreateEngineCalls).to.deep.equal([
+        "Llama-3.2-1B-Instruct-q4f32_1-MLC",
+        "Llama-3.2-1B-Instruct-q4f32_1-MLC",
+      ]);
+    });
+  });
 });

cypress/e2e/document-linter.cy.js

@@ -109,7 +109,9 @@ describe("document linter export", () => {
     cy.get("[data-action=\"new-note\"]").click({ force: true });
     cy.get("#editor").clear().type(noteContent);
 
-    cy.get("#documentLinterToggleBtn").click();
+    cy.get("#documentLinterToggleBtn").should("not.exist");
+    cy.get("#cogitoToggleBtn").click();
+    cy.get("#cogitoPanel").should("be.visible");
     cy.get("#documentLinterAnalyzeBtn").click();
     cy.get("#statusBadge").should("contain", "Analysis complete");
 

cypress/e2e/workspace-actions.cy.js

@@ -146,10 +146,111 @@ describe("workspace actions regression", () => {
     cy.get("#workspaceName").should("contain", workspaceName);
 
     cy.get("body").click("topLeft");
-    cy.get("#refreshBtn").click();
+    cy.get("body").click("bottomRight");
+    cy.get("#refreshBtn").click({ force: true });
     cy.get("#statusBadge").should("contain", "Refreshed");
   });
 
+  it("opens the folder picker immediately and ignores overlapping open requests", () => {
+    cy.window().then((win) => {
+      win.__pickerCalls = 0;
+      win.showDirectoryPicker = () => {
+        win.__pickerCalls += 1;
+        return new Promise((resolve) => {
+          win.__resolveWorkspacePicker = () => resolve(win.__fakeWorkspace);
+        });
+      };
+    });
+
+    cy.get("[data-action=\"open-workspace\"]").click({ force: true });
+    cy.get("#statusBadge").should("contain", "Choose a workspace folder");
+    cy.window().its("__pickerCalls").should("eq", 1);
+
+    cy.get("[data-action=\"open-workspace\"]").click({ force: true });
+    cy.window().its("__pickerCalls").should("eq", 1);
+    cy.get("#statusBadge").should("contain", "Folder picker already open");
+
+    cy.window().then((win) => {
+      win.__resolveWorkspacePicker();
+    });
+    cy.get("#statusBadge").should("contain", "Workspace ready");
+  });
+
+  it("loads workspace files concurrently and reads each file once", () => {
+    cy.window().then((win) => {
+      const tracker = { active: 0, maximum: 0, calls: 0 };
+      win.__workspaceReadTracker = tracker;
+
+      for (let index = 0; index < 32; index += 1) {
+        const name = `note-${index}.md`;
+        win.__fakeWorkspace.__entries.set(name, {
+          kind: "file",
+          name,
+          async getFile() {
+            tracker.calls += 1;
+            tracker.active += 1;
+            tracker.maximum = Math.max(tracker.maximum, tracker.active);
+            await new Promise((resolve) => win.setTimeout(resolve, 20));
+            tracker.active -= 1;
+            return new File([`---\ntags: [batch]\n---\n# Note ${index}`], name, {
+              type: "text/markdown",
+              lastModified: index,
+            });
+          },
+        });
+      }
+    });
+
+    cy.get("[data-action=\"open-workspace\"]").click({ force: true });
+    cy.get("#statusBadge").should("contain", "Workspace ready");
+    cy.get("#countsPill").should("contain", "32 notes");
+    cy.window().then((win) => {
+      expect(win.__workspaceReadTracker.calls).to.eq(32);
+      expect(win.__workspaceReadTracker.maximum).to.be.greaterThan(1);
+      expect(win.__workspaceReadTracker.maximum).to.be.at.most(4);
+    });
+  });
+
+  it("prioritizes opening a note over an in-progress background scan", () => {
+    cy.window().then((win) => {
+      win.__fakeWorkspace.__entries.set(
+        "priority.md",
+        createFakeFileHandle("priority.md", "# Priority\n\nOpen me immediately."),
+      );
+    });
+
+    cy.get("[data-action=\"open-workspace\"]").click({ force: true });
+    cy.get("#workspaceName").should("contain", workspaceName);
+    cy.get("#tree .node").contains("priority.md").should("be.visible");
+
+    cy.window().then((win) => {
+      const tracker = { calls: 0 };
+      win.__slowRefreshTracker = tracker;
+      for (let index = 0; index < 40; index += 1) {
+        const name = `slow-${index}.md`;
+        win.__fakeWorkspace.__entries.set(name, {
+          kind: "file",
+          name,
+          async getFile() {
+            tracker.calls += 1;
+            await new Promise((resolve) => win.setTimeout(resolve, 250));
+            return new File([`# Slow ${index}`], name, { type: "text/markdown" });
+          },
+        });
+      }
+    });
+
+    cy.get("#refreshBtn").click({ force: true });
+    cy.get("#tree .node").contains("priority.md").click();
+    cy.get("#currentFilename").should("contain", "priority.md");
+    cy.get("#editor").should("contain.value", "Open me immediately");
+
+    cy.wait(350);
+    cy.window().then((win) => {
+      expect(win.__slowRefreshTracker.calls).to.be.at.most(4);
+    });
+  });
+
   it("close workspace resets UI state", () => {
     cy.get("[data-action=\"open-workspace\"]").click({ force: true });
     cy.get("[data-action=\"new-note\"]").click({ force: true });