Skip to content

feat: add Search for vector, keyword, and hybrid search - #77

Open
lukekim wants to merge 5 commits into
trunkfrom
feat/search
Open

feat: add Search for vector, keyword, and hybrid search#77
lukekim wants to merge 5 commits into
trunkfrom
feat/search

Conversation

@lukekim

@lukekim lukekim commented Jul 27, 2026

Copy link
Copy Markdown
Contributor

What

Adds SpiceClient.Search, wrapping the runtime's POST /v1/search — vector similarity, keyword, and hybrid search over datasets with an embedding column.

resp, err := spice.Search(ctx, &gospice.SearchRequest{
    Text:     "tickets to Tokyo",
    Datasets: []string{"app_messages"},
    Limit:    3,
})

Only Text is required. Setting Keywords pre-filters the embedding column with a lexical search before the vector search runs, making the search hybrid.

Why

/v1/search works on a default spice run and is a distinctive Spice capability, but Go users had no way to reach it short of hand-rolling the HTTP call. spice.js is currently the only SDK that exposes it.

Two wire-format details the types handle so callers don't have to:

  • the similarity score is serialized as _score, not score
  • data, primary_key and metadata are omitted entirely when empty, so they decode to nil maps — safe to read from in Go

Part of aligning search support across the SDKs.

Verification

  • go build ./...
  • go vet ./...
  • gofmt clean
  • Unit tests pass — go test -run TestSearch ./..., covering request encoding, method/path, response decoding (including omitted objects), validation, and runtime error surfacing. All run against httptest, so they need no live runtime.
  • Verified against a live spice run: the search request reaches /v1/search, and a
    failure surfaces the runtime's own message ("Search cannot be run on <dataset> because it has no embeddings or full text search indexes.") rather than a bare status code
  • A search returning matches was not exercised end to end — that needs a dataset with an
    embedding column and a loaded embedding model, which this environment has no credentials for

lukekim added 2 commits July 27, 2026 08:46
Wraps POST /v1/search, which was previously unreachable from Go without
hand-rolling the HTTP call. spice.js is the only other SDK that exposes it.

Only Text is required; Datasets, Limit, Where, AdditionalColumns and Keywords
are optional. Supplying Keywords pre-filters the embedding column with a lexical
search before the vector search, making the search hybrid.

Decodes the runtime's wire format directly, including the `_score` field name
and the data/primary_key/metadata objects the runtime omits when empty.
The error path only unpacked a JSON {"error": ...} body and otherwise reported
a bare "failed with status=400", dropping the part that says what to fix — for
example "Search cannot be run on X because it has no embeddings or full text
search indexes", which the runtime returns as plain text.

Caught by running the new search path against a live runtime.
@lukekim lukekim self-assigned this Jul 27, 2026
@lukekim
lukekim requested review from Copilot and sgrebnov July 27, 2026 18:07
@lukekim lukekim added the enhancement New feature or request label Jul 27, 2026
@lukekim lukekim added this to the v8.0.0 milestone Jul 27, 2026

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds first-class Go SDK support for the runtime’s POST /v1/search endpoint by introducing SpiceClient.Search plus request/response types, tests, and README documentation so Go users can perform vector, keyword, and hybrid search without hand-rolling HTTP.

Changes:

  • Introduces SearchRequest, SearchResponse, and SearchMatch wire-format types and SpiceClient.Search implementation.
  • Adds unit tests covering request encoding, method/path, response decoding (including omitted objects), validation, and runtime error surfacing.
  • Documents Search usage and behavior in the README, including hybrid search via Keywords.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.

File Description
search.go Adds SpiceClient.Search and associated request/response types plus runtime error-message handling.
search_test.go Adds httptest-based unit tests for encoding/decoding, validation, and error surfacing.
README.md Documents the new Search API with examples and field behavior notes.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread search.go Outdated
lukekim added 3 commits July 27, 2026 15:39
…gistry

`spice add spiceai/quickstart` makes every run depend on the Spicepod registry.
That service is currently returning a body the CLI cannot unpack, so every job
fails at:

  Invalid argument: Failed to extract Spicepod archive: invalid Zip archive:
  Could not find EOCD

The runtime and the SDK are fine; only the fetch is broken. Declaring the same
dataset spiceai/quickstart provides removes the dependency.

Applies to both the Unix and WSL setup steps. Verified locally: taxi_trips
loads and is queryable within seconds (2,964,624 rows).

Mirrors spicepy#176.
An empty X-API-Key reads as a supplied-but-invalid credential to auth
middleware, which behaves differently from omitting the header. IsSpiceReady
already guards this; Search now matches.

Addresses review feedback on #77.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants