Skip to content

feat: add populateEmbedding option to search and searchByEmbedding#70

Merged
techiejd merged 3 commits into
mainfrom
feat/search-populate-embedding
Jun 22, 2026
Merged

feat: add populateEmbedding option to search and searchByEmbedding#70
techiejd merged 3 commits into
mainfrom
feat/search-populate-embedding

Conversation

@techiejd

Copy link
Copy Markdown
Owner

What

Adds an optional populateEmbedding?: boolean (default false) to both VectorSearchQuery and VectorSearchEmbeddingQuery. When true, each search result includes its embedding vector — mirroring the existing behavior of findByIds.

Why

findByIds can already return embeddings via populateEmbedding, but search / searchByEmbedding could not. This closes that gap so callers can retrieve vectors directly from a similarity search.

Changes

End-to-end plumbing (default false, so existing callers are unaffected):

  • src/types.ts — added the flag to both query interfaces and to the DbAdapter.search signature.
  • src/index.tssearch / searchByEmbedding pass it through.
  • src/endpoints/vectorSearch.ts — both handler signatures thread it to adapter.search (including the rerank path).
  • Adapters — return the embedding only when requested:
    • pg: selects the embedding column and parses it.
    • cf: sets returnValues: true and maps match.values.
    • mongodb: skips the embedding $project exclusion and maps it.
  • mongodb index.ts — fixed the search wrapper that dropped the new argument before reaching searchImpl (otherwise the option would have been a silent no-op on mongodb).

Not wired into the HTTP requestHandler — this is the programmatic plumbing only.

Tests

Added populate/omit coverage at each layer (core via VectorizedPayload, plus pg/cf/mongodb adapter compliance + the cf unit mock). The mongodb tests poll for the eventually-consistent vector index before asserting.

Full suites pass: CF 97 · PG 74 · mongodb 114 · core 96, zero failures; all packages typecheck clean.

Note

No changeset included — intentionally, to keep one minor changeset covering the stacked PRs rather than per-PR.

Adds an optional populateEmbedding flag (default false) to VectorSearchQuery
and VectorSearchEmbeddingQuery, threaded through the search handlers and the
DbAdapter.search signature into all three adapters (pg, cf, mongodb) so each
result can include its embedding vector, mirroring findByIds.

Also fixes the mongodb adapter's search wrapper, which dropped the argument
before reaching searchImpl.
@techiejd techiejd force-pushed the feat/search-populate-embedding branch from 58436ec to c1d1047 Compare June 22, 2026 06:20
techiejd added 2 commits June 22, 2026 14:05
- Extract the duplicated pg parseEmbedding into a shared module imported by
  both search.ts and findByIds.ts.
- Declare embedding?: number[] explicitly on VectorSearchResult and drop the
  `as number[]` casts from the populate-embedding tests.
- Note in the HTTP search handler that populateEmbedding is intentionally
  programmatic-only.
- Root README: add populateEmbedding to the search/searchByEmbedding Local
  API params and note it is Local-API-only (the REST endpoint never returns
  vectors); fix the findByIds note that claimed search never returns the vector.
- Adapters README: add populateEmbedding to the DbAdapter.search signature,
  search-path lifecycle, and method-reference row; add embedding? to the
  VectorSearchResult type; correct the EmbeddingRecord notes that said search
  never returns embeddings.
@techiejd techiejd merged commit a042169 into main Jun 22, 2026
8 checks passed
techiejd added a commit that referenced this pull request Jun 22, 2026
Consolidates the unreleased changes since 1.0.1 into one minor changeset:
findByIds (#62), searchByEmbedding (#68), populateEmbedding (#70), and the
mongodb search field-parity fix (#63).

Also sets onlyUpdatePeerDependentsWhenOutOfRange so the fixed package group
resolves to a minor (1.1.0) instead of a major peer-dependency cascade.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant