Skip to content

Lazy cache Knowhere chunk pages - #121

Merged
suguanYang merged 4 commits into
stagingfrom
feature/wangbinqi/lazy-knowhere-chunk-cache
Jun 30, 2026
Merged

Lazy cache Knowhere chunk pages#121
suguanYang merged 4 commits into
stagingfrom
feature/wangbinqi/lazy-knowhere-chunk-cache

Conversation

@suguanYang

Copy link
Copy Markdown
Contributor

Summary

  • mark Knowhere-backed sources ready when a completed job exposes a document id instead of waiting on result.zip mirroring
  • load chunk pages lazily through Knowhere listChunks, key cached pages by document id plus current revision/job id, and mirror visible page assets into Vercel Blob in the background
  • keep full-tree loads structure-only, add paginated lightweight remote document rows, and materialize remote documents only when opened
  • update the Knowhere SDK to ^0.10.0 and pin Next dev root to this workspace

Verification

  • pnpm typecheck
  • pnpm lint
  • pnpm vitest run src/domains/chunks/server.test.ts src/domains/sources/route-service.test.ts src/domains/workspace/initial-state.test.ts
  • staging smoke: documents.list pagination returned page 1 / pageSize 2 / total 4 / totalPages 2
  • staging smoke: listChunks with includeAssetUrls false returned job_1d6de91af517 and no asset URLs
  • staging smoke: listChunks with includeAssetUrls true returned upstream asset URLs
  • local smoke: /api/sources returned 200 and selected Sydney Streets Code.pdf with document doc_2fb226371985
  • local smoke: /api/sources/ec815020-2238-4fa6-b37f-2395758cca7d/chunks?page=1&pageSize=2 returned 200, 2 chunks, total 1735, and Vercel Blob asset URLs after warm
  • local DB smoke: source revision key persisted as job_1d6de91af517

@vercel

vercel Bot commented Jun 30, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
knowhere-notebook-staging Ready Ready Preview, Comment Jun 30, 2026 2:14pm
1 Skipped Deployment
Project Deployment Actions Updated (UTC)
knowhere-notebook Ignored Ignored Jun 30, 2026 2:14pm

Request Review

@suguanYang
suguanYang changed the base branch from main to staging June 30, 2026 14:26
@suguanYang
suguanYang merged commit 0ce7aaf into staging Jun 30, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant