Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ on:
workflow_dispatch:
inputs:
version:
description: "Semver version (e.g. 0.2.1 or v0.2.1)"
description: "Semver version (e.g. 0.2.2 or v0.2.2)"
required: true
type: string

Expand Down
26 changes: 25 additions & 1 deletion Makefile
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
.PHONY: test-integration-all test-integration-stores test-integration-postgres test-integration-mssql test-integration-msql
.PHONY: test test-no-cache test-stores test-stores-no-cache test-integration-all test-integration-all-no-cache test-integration-stores test-integration-stores-no-cache test-integration-postgres test-integration-postgres-no-cache test-integration-mssql test-integration-mssql-no-cache test-integration-msql

GO ?= go
GOTOOLCHAIN ?= local
Expand All @@ -8,16 +8,40 @@ TEST_TIMEOUT ?= 30m
TEST_FLAGS ?=
GO_ENV ?= env -u GOROOT GOTOOLCHAIN=$(GOTOOLCHAIN)

test:
$(GO_ENV) $(GO) test -mod=$(MOD_MODE) -timeout=$(TEST_TIMEOUT) $(TEST_FLAGS) ./...

test-no-cache:
$(GO_ENV) $(GO) test -count=1 -mod=$(MOD_MODE) -timeout=$(TEST_TIMEOUT) $(TEST_FLAGS) ./...

test-stores:
$(GO_ENV) $(GO) test -mod=$(MOD_MODE) -timeout=$(TEST_TIMEOUT) $(TEST_FLAGS) ./stores/...

test-stores-no-cache:
$(GO_ENV) $(GO) test -count=1 -mod=$(MOD_MODE) -timeout=$(TEST_TIMEOUT) $(TEST_FLAGS) ./stores/...

test-integration-all:
$(GO_ENV) $(GO) test -mod=$(MOD_MODE) -tags=$(INTEGRATION_TAG) -timeout=$(TEST_TIMEOUT) $(TEST_FLAGS) ./...

test-integration-all-no-cache:
$(GO_ENV) $(GO) test -count=1 -mod=$(MOD_MODE) -tags=$(INTEGRATION_TAG) -timeout=$(TEST_TIMEOUT) $(TEST_FLAGS) ./...

test-integration-stores:
$(GO_ENV) $(GO) test -mod=$(MOD_MODE) -tags=$(INTEGRATION_TAG) -timeout=$(TEST_TIMEOUT) $(TEST_FLAGS) ./stores/...

test-integration-stores-no-cache:
$(GO_ENV) $(GO) test -count=1 -mod=$(MOD_MODE) -tags=$(INTEGRATION_TAG) -timeout=$(TEST_TIMEOUT) $(TEST_FLAGS) ./stores/...

test-integration-postgres:
$(GO_ENV) $(GO) test -mod=$(MOD_MODE) -tags=$(INTEGRATION_TAG) -timeout=$(TEST_TIMEOUT) $(TEST_FLAGS) ./stores/postgres

test-integration-postgres-no-cache:
$(GO_ENV) $(GO) test -count=1 -mod=$(MOD_MODE) -tags=$(INTEGRATION_TAG) -timeout=$(TEST_TIMEOUT) $(TEST_FLAGS) ./stores/postgres

test-integration-mssql:
$(GO_ENV) $(GO) test -mod=$(MOD_MODE) -tags=$(INTEGRATION_TAG) -timeout=$(TEST_TIMEOUT) $(TEST_FLAGS) ./stores/mssql

test-integration-mssql-no-cache:
$(GO_ENV) $(GO) test -count=1 -mod=$(MOD_MODE) -tags=$(INTEGRATION_TAG) -timeout=$(TEST_TIMEOUT) $(TEST_FLAGS) ./stores/mssql

test-integration-msql: test-integration-mssql
6 changes: 3 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -229,12 +229,12 @@ GitHub Actions workflows are configured for:

Release options:

1. Manual (recommended): run `Release` workflow via GitHub UI with `version` input (`0.2.1` or `v0.2.1`).
1. Manual (recommended): run `Release` workflow via GitHub UI with `version` input (`0.2.2` or `v0.2.2`).
2. Tag-driven: push a semver tag and the workflow publishes release notes automatically:

```bash
git tag v0.2.1
git push origin v0.2.1
git tag v0.2.2
git push origin v0.2.2
```

## Samples
Expand Down
5 changes: 3 additions & 2 deletions docs/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -138,10 +138,11 @@ The pgvector operators are documented in the [pgvector README](https://github.co

`MSSQLCollection.SearchByVector`:

1. Loads records from SQL Server
1. Streams records from SQL Server
2. Evaluates filters against records in-process
3. Computes distance in-process (cosine/l2/inner product)
4. Applies threshold, sorts by distance, returns top-k
4. Applies threshold and keeps a bounded in-memory top-k heap
5. Returns top-k sorted by distance

## 6) Filter System (AST -> SQL)

Expand Down
13 changes: 6 additions & 7 deletions docs/stores-implementation.md
Original file line number Diff line number Diff line change
Expand Up @@ -171,18 +171,18 @@ Write behavior:
- Chunk in batches of 500
- Use transaction per chunk
- Upsert strategy:
- attempt `UPDATE`
- if `RowsAffected == 0`, execute `INSERT`
- execute single-statement `UPDATE ... WITH (UPDLOCK, SERIALIZABLE)` + conditional `INSERT`
- avoids race conditions under concurrent upserts for the same ID

Search behavior (MVP):

1. Validate `topK` and vector dimension
2. Load records from SQL table into memory
2. Stream records row-by-row from SQL
3. Evaluate filter AST in-process (`matchesFilter`)
4. Compute distance in-process (`distanceBetween`)
5. Apply optional threshold
6. Sort by distance asc (stable tie-break by `ID`)
7. Apply projection and cut to top-k
6. Maintain bounded top-k max-heap in memory (`O(k)` memory)
7. Return results sorted by distance asc (tie-break by `ID`)

`EnsureIndexes` is intentionally a no-op in MSSQL backend for this MVP.

Expand All @@ -197,7 +197,7 @@ Filter AST supports:
Execution strategy by backend:

- Postgres: compile AST -> parameterized SQL via `CompileFilterSQL`
- MSSQL: evaluate AST in Go against materialized records
- MSSQL: evaluate AST in Go against streamed records

Important behavior:

Expand Down Expand Up @@ -257,4 +257,3 @@ To add a new store backend, follow the same contract shape:
4. Support filter handling (compiled or in-process)
5. Reuse `vectordata` errors and scoring semantics
6. Add integration tests matching existing backend test coverage style

Loading
Loading