feat(wave4): reliability — threadpools, timeouts, WAL, batching, OCR guard

[vikranthreddimasu]

Context

Wave 4 of 5 from PLAN.md. This one targets the structural weaknesses the tech audit flagged as "threats to actually works": unbounded timeouts, event-loop-blocking inference, OOM-able embedding batches, reads blocking on writes in SQLite, silent no-op ingestions for scanned PDFs, conversation splits when a meta event drops, startup deadlocks on slow Ollama.

Event loop stays responsive

• `aembed` on the EmbeddingBackend protocol. `SentenceTransformersBackend` offloads `model.encode()` to the default threadpool (a 200–500ms encode was stalling concurrent health checks and sidebar refreshes). `HashEmbeddingBackend` runs inline.
• Async query variants. `VectorStoreManager.aquery` / `aquery_across_notebooks` / `aquery_document_summaries` use `aembed`. RAGService `prepare_prompt` / `prepare_prompt_cross_notebook` and `LlamaIndexRAGService.query` now actually async all the way down.
• LlamaCpp backend. `generate` and `stream_generate` use `loop.run_in_executor` for the blocking `llama.create_completion` calls. Streams pull chunks on a worker thread and yield on the event loop. Previously every second of inference froze all other requests.
• Ollama stream timeout. `httpx.AsyncClient(timeout=httpx.Timeout(connect=5, read=300, write=10, pool=5))` instead of `timeout=None`. A hung Ollama instance no longer pins the connection forever.

OOM and re-upload guards

• Batched embedding. `add_chunks` embeds in batches of 64 instead of one mega-call. A 500-page PDF no longer OOMs sentence-transformers.
• Upsert, not add. `collection.upsert` makes re-uploading a file succeed cleanly. The old path failed mid-batch with `DuplicateIDError` and left Chroma half-updated.
• OCR guard for scanned PDFs. `_load_pdf` now raises a clear `DocumentLoaderError` ("run it through OCR first") if extracted text is below a minimum threshold. Before, a scanned PDF "succeeded" with 0 chunks and queries returned "no relevant documents found" forever.

SQLite stays unblocked

• `PRAGMA journal_mode = WAL` + `busy_timeout = 5000` in `notebook_store`, `conversation_store`, `metrics_store`. Readers don't block on writers. The sidebar refresh during stream completion used to stall under default DELETE journal.

Data integrity on stream

• `conversation_id` echoed on `done` events too. If the `meta` event drops (parse error, transient glitch), the frontend still learns the id via done and doesn't fork the thread. `useChat` guards with `!activeConversationId` so it only sets when missing.

Startup correctness

• Lazy Ollama resolution. Model resolution moved to an `asyncio` lifespan; a slow `/api/tags` round-trip at boot no longer delays FastAPI startup or trips Electron's `waitForBackend` timeout. `app.py` imports consolidated at the top (were E402-violating before).
• Split token config defaults. `llm_context_window` 4096 (was 2048), `llm_max_tokens` 1024 (was 2048). One knob was doing two jobs — it capped replies AND silently truncated long RAG prompts.
• Chroma telemetry off. `Settings(anonymized_telemetry=False)`. Offline-first is the whole promise; also stops the recurring "Failed to send telemetry event" log noise.

Frontend hardening

• API timeouts. `request()` composes any caller-provided `AbortSignal` with a default 30s timeout. Uploads get their own 5-minute budget via `uploadDocument`. Previously a stalled endpoint hung the UI indefinitely.
• User-facing error. Timeout errors flow through the Wave-3 `humanizeError` translator so the user sees "The request took too long…" instead of an `AbortError` stack dive.

Verification

• `cd apps/desktop && npm run build` — clean (272 modules, 864ms)
• `cd backend && uv run ruff check .` — clean
• `cd backend && uv run pytest -q` — 33 passed

Intentionally not in this PR

• 4.1 Backend-crash recovery IPC (bigger Electron refactor)
• 4.3 Exponential backoff for 503/ECONNREFUSED (fiddly retry policy)
• 4.9 Stream-to-disk upload + size limit (multipart streaming rewrite)
• 4.13 Per-conversation asyncio lock (needs a store-level primitive)
• 4.15 `list_documents` via summaries collection (risks regressing docs that failed summary)
• 4.16 Defer summary LLM to a background task queue

These are planned for a follow-up or Wave 4b depending on priority.

🤖 Generated with Claude Code