Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 10 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -91,8 +91,9 @@ for hit in response.results:
print(hit.path, hit.score)
```

By default it reads `~/.vexor/config.json`. For runtime config overrides, cache
controls, and per-call options, see [`docs/api/python.md`](https://github.com/scarletkc/vexor/tree/main/docs/api/python.md).
Configuration follows the same global and project-level resolution as the CLI.
For runtime overrides, cache controls, and per-call options, see
[`docs/api/python.md`](https://github.com/scarletkc/vexor/tree/main/docs/api/python.md).

## AI Agent Skill

Expand Down Expand Up @@ -149,12 +150,16 @@ vexor init # guided setup (recommended)
vexor config --set-api-key "YOUR_KEY" # or env: VEXOR_API_KEY / OPENAI_API_KEY / ...
vexor config --set-provider openai # default; also gemini/voyageai/custom/local
vexor config --rerank hybrid # optional: fuse exact keyword + semantic ranking
vexor config --show # view current settings
vexor config --show # view effective settings and origins
```

Config lives in `~/.vexor/config.json`. Any field can also be injected via the `VEXOR_CONFIG_JSON` environment variable (useful for MCP client configs and CI), and fully offline use is supported through local embedding models.
Global config lives in `~/.vexor/config.json`; the nearest
`<project>/.vexor/config.json` can safely override selected behavior for that
project. Non-secret fields can also be injected via `VEXOR_CONFIG_JSON`
(useful for MCP clients and CI), and fully offline use is supported through
local embedding models.

See [`docs/configuration.md`](https://github.com/scarletkc/vexor/blob/main/docs/configuration.md) for the complete reference: all config commands, API keys and environment variables, rerank strategies (hybrid / BM25 / FlashRank / remote), remote vs local providers, embedding dimensions, and offline local model setup.
See [`docs/configuration.md`](https://github.com/scarletkc/vexor/blob/main/docs/configuration.md) for the complete reference: project config fields and precedence, all config commands, API keys and environment variables, rerank strategies (hybrid / BM25 / FlashRank / remote), remote vs local providers, embedding dimensions, and offline local model setup.

## CLI Reference

Expand Down
28 changes: 21 additions & 7 deletions docs/api/python.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,8 @@
# Python API

The Python API mirrors the CLI behavior and exposes the same search/index flow.
Configuration can come from disk, in-memory overrides, or per-call parameters.
Configuration can come from global or project files, environment variables,
in-memory overrides, or per-call parameters.

## Quick start

Expand Down Expand Up @@ -44,12 +45,22 @@ Configuration is resolved in this order (highest to lowest priority):
1. Explicit function arguments (e.g., `provider`, `model`, `api_key`)
2. Per-call override via `config=...`
3. In-memory runtime config set by `set_config_json(...)`
4. `~/.vexor/config.json` (unless `use_config=False`)
5. Built-in defaults
4. Environment overrides, including `VEXOR_CONFIG_JSON`
5. The nearest project `.vexor/config.json`
6. Global `~/.vexor/config.json`
7. Built-in defaults

For pure library usage that should ignore on-disk config, pass `use_config=False`
and set explicit arguments or `config=...`.

Project resolution starts at the call's resolved `path` and uses the nearest
`.vexor/` marker. The project file accepts exactly `rerank`, `auto_index`,
`model`, `embedding_dimensions`, `batch_size`, `embed_concurrency`, and
`extract_concurrency`. It rejects credentials and endpoints (`api_key`,
`base_url`, `remote_rerank`) and every other field. This restricted schema
applies only to `.vexor/config.json`; explicit API and runtime payloads retain
the full schema below.

Index cache location is resolved separately. Vexor walks upward from `path` and
uses the nearest project `.vexor/index.db` when a `.vexor/` directory exists.
Explicit per-call `cache_dir` or `data_dir` arguments, client-level overrides,
Expand Down Expand Up @@ -127,7 +138,7 @@ merging with the current runtime or on-disk config (avoids reading disk).

## Config payload schema

The `config` payload (dict/JSON) supports:
The per-call or in-memory `config` payload (dict/JSON) supports:

- `provider`: `openai`, `gemini`, `voyageai`, `custom`, `local`
- `model`: embedding model name
Expand All @@ -144,14 +155,17 @@ The `config` payload (dict/JSON) supports:
- `flashrank_model`: string or null
- `remote_rerank`: object with `base_url`, `api_key`, `model`

Unknown keys are ignored. `remote_rerank.base_url` is normalized to end with `/rerank`.
Unknown keys in these explicit payloads are ignored; project files reject
them. `remote_rerank.base_url` is normalized to end with `/rerank`.

API keys can also come from environment variables:
`VEXOR_API_KEY`, `OPENAI_API_KEY`, `GOOGLE_GENAI_API_KEY`, `VOYAGE_API_KEY`,
and `VEXOR_REMOTE_RERANK_API_KEY`.

Environment variables only supply API keys; other settings must be passed
explicitly via function arguments or `config=...`.
`VEXOR_CONFIG_JSON` supplies non-secret environment overrides; dedicated
variables supply API keys. Environment values override global and project
files, while explicit function, per-call, and in-memory values take precedence
over the environment.

## Cache behavior

Expand Down
34 changes: 23 additions & 11 deletions docs/cli.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@
| `vexor QUERY` | Shortcut for `vexor search QUERY` |
| `vexor search QUERY --path PATH` | Semantic search (auto-indexes if needed) |
| `vexor index --path PATH` | Build/refresh index manually |
| `vexor config --show` | Display current configuration |
| `vexor config --show` | Display effective configuration and each field's origin |
| `vexor config --clear-flashrank` | Remove cached FlashRank models under `~/.vexor/flashrank` |
| `vexor local --setup [--model MODEL]` | Download a local model and set provider to `local` |
| `vexor local --clean-up` | Remove local model cache under `~/.vexor/models` |
Expand Down Expand Up @@ -46,6 +46,18 @@ strategies (`off`, `bm25`, `flashrank`, `remote`, `hybrid`).
Porcelain output fields: `rank`, `similarity`, `path`, `chunk_index`,
`start_line`, `end_line`, `preview` (line fields are `-` when unavailable).

## Project Configuration

Search and index commands walk upward from their resolved `--path` and apply
`config.json` from the nearest `.vexor/` marker as a safe overlay on the
global config; credentials and endpoints are rejected. `vexor config --show`
and `vexor doctor` resolve from the current working directory: `--show`
labels each effective field's origin, `doctor` lists active overrides.
Mutating `vexor config` options always write `~/.vexor/config.json`; edit the
project file directly for project-specific values. See
[Configuration → Project configuration](configuration.md#project-configuration)
for the accepted fields and precedence.

## Ignore Files

Use `.vexorignore` for project-specific indexing exclusions. It supports full
Expand Down Expand Up @@ -86,18 +98,18 @@ Keep flags consistent to reuse cache; changing flags creates a separate index.

By default, indexes are stored in `~/.vexor/index.db`. Project-local caching is
opt-in: create a `.vexor/` directory in a project root, or run
`vexor index --local`. Either way, Vexor writes a self-ignoring `.gitignore`
(`*`) inside `.vexor/` on first use so the index database cannot be committed
by accident. For each index or search operation, Vexor walks upward
from the resolved target path and uses the nearest `.vexor/` directory it
finds. This means nested projects use their nearest marker. Searching a parent
directory above a project root does not discover markers in its children and
therefore uses the global database.
`vexor index --local`. Either way, Vexor writes a `.gitignore` inside
`.vexor/` on first use so generated indexes and caches cannot be committed by
accident while `config.json` remains eligible for version control. For each
index or search operation, Vexor walks upward from the resolved target path and
uses the nearest `.vexor/` directory it finds. This means nested projects use
their nearest marker. Searching a parent directory above a project root does
not discover markers in its children and therefore uses the global database.

Cache location precedence is: an explicit API/cache override, then the nearest
project `.vexor/`, then `~/.vexor/`. Only `index.db` moves into a project.
Configuration, update-check data, FlashRank assets, and local embedding models
remain under the global `~/.vexor/` directory.
project `.vexor/`, then `~/.vexor/`. The project directory may also contain the
safe `config.json` overlay described above. Global configuration, update-check
data, FlashRank assets, and local embedding models remain under `~/.vexor/`.

```bash
vexor config --show-index-all # list all cached indexes
Expand Down
77 changes: 67 additions & 10 deletions docs/configuration.md
Original file line number Diff line number Diff line change
@@ -1,22 +1,79 @@
# Configuration

Vexor is configured through `vexor config` commands (or the interactive
`vexor init` wizard). Settings persist in `~/.vexor/config.json`.
`vexor init` wizard). Global settings persist in `~/.vexor/config.json`.

## Where data lives

Configuration, update-check data, FlashRank assets, and local embedding models
stay in the global `~/.vexor/` directory. Indexes normally use
`~/.vexor/index.db`, but a project containing a `.vexor/` directory uses
`<project>/.vexor/index.db` for searches and indexing within that project.
Global configuration, update-check data, FlashRank assets, and local embedding
models stay under `~/.vexor/`. Indexes normally use `~/.vexor/index.db`, but a
project containing a `.vexor/` directory uses `<project>/.vexor/index.db` for
searches and indexing within that project. The same directory may contain a
tracked `config.json` with safe project-level overrides.

Run `vexor index --local` to create the project directory and its ignore file.
Only the index database, including embedding and query cache tables, is
project-local.
This does not create a project config. The generated index and caches remain
ignored while `.vexor/config.json` can be committed.

When an upgrade changes the index cache schema, existing index caches are
invalidated and rebuilt automatically on next use; cached embeddings are
reused, so the migration does not re-embed unchanged content.

## Project configuration

For each path-aware search or index operation, Vexor resolves the target path
and walks upward to the nearest `.vexor/` directory. If that marker contains
`config.json`, Vexor overlays it on the global configuration. Nested projects
therefore use their nearest marker; a search above a project does not discover
markers below it. If the nearest marker has no `config.json`, Vexor uses the
global configuration instead of continuing into an outer project.

Project config v1 accepts exactly these fields:

- `rerank`
- `auto_index`
- `model`
- `embedding_dimensions`
- `batch_size`
- `embed_concurrency`
- `extract_concurrency`

Example `<project>/.vexor/config.json`:

```json
{
"model": "text-embedding-3-small",
"embedding_dimensions": 1024,
"batch_size": 32,
"embed_concurrency": 4,
"extract_concurrency": 4,
"auto_index": true,
"rerank": "hybrid"
}
```

Repository files are untrusted input, so this is a strict allowlist. Project
config explicitly rejects credentials and endpoints (`api_key`, `base_url`,
and the entire `remote_rerank` object), as well as every other unsupported or
unknown field. Configure those values globally, through environment variables,
or with explicit Python API arguments. This is stricter than
`VEXOR_CONFIG_JSON`, which may set non-secret fields such as `base_url`.

Effective precedence, from lowest to highest, is:

1. Built-in defaults
2. Global `~/.vexor/config.json`
3. The nearest project `.vexor/config.json`
4. Environment overrides, including `VEXOR_CONFIG_JSON`
5. Explicit runtime or API arguments

`vexor config --show` and `vexor doctor` resolve the project from the current
working directory: `--show` labels every effective field with its origin, and
`doctor` reports the project config file plus any project or environment
overrides. All mutating `vexor config` options and `vexor init` still write
only the global config; edit `.vexor/config.json` directly to change a
project override.

## Commands

```bash
Expand Down Expand Up @@ -44,7 +101,7 @@ vexor config --set-remote-rerank-api-key $VEXOR_REMOTE_RERANK_API_KEY # or env
vexor config --clear-remote-rerank # clear remote rerank config
vexor config --set-base-url https://proxy.example.com # optional proxy
vexor config --clear-base-url # reset to official endpoint
vexor config --show # view current settings
vexor config --show # view effective settings and origins
```

Rerank defaults to `off`. **It is highly recommended to configure the
Expand All @@ -62,8 +119,8 @@ or `VOYAGE_API_KEY`; `VEXOR_API_KEY` takes precedence over a stored key.

Any non-secret config field can also be injected as a JSON object via the
`VEXOR_CONFIG_JSON` environment variable (useful for MCP client configs and
CI), merged over `~/.vexor/config.json`. Credential fields inside
`VEXOR_CONFIG_JSON` are rejected — use the dedicated variables above.
CI), merged over the effective global and project config. Credential fields
inside `VEXOR_CONFIG_JSON` are rejected — use the dedicated variables above.

## Rerank

Expand Down
6 changes: 5 additions & 1 deletion docs/development.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,4 +7,8 @@ pytest
```
Tests rely on fake embedding backends, so no network access is required.

Cache files and configuration live in `~/.vexor` (a project with a `.vexor/` directory keeps its index there instead). The text embedded for each chunk is built by the mode strategies in `vexor/modes.py`; adjust the strategy `label` construction there if you need to encode additional context.
Global cache files and configuration live in `~/.vexor`. A project with a
`.vexor/` directory keeps its index there and may add the restricted
`config.json` overlay documented in `docs/configuration.md`. The text embedded
for each chunk is built by the mode strategies in `vexor/modes.py`; adjust the
strategy `label` construction there if you need to encode additional context.
20 changes: 13 additions & 7 deletions docs/mcp.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,8 @@
`vexor mcp` runs a [Model Context Protocol](https://modelcontextprotocol.io)
server over stdio, exposing Vexor's semantic search to any MCP-capable agent
(Claude Code, Codex, Cursor, Windsurf, Zed, and others). It requires no extra
dependencies and reuses your existing Vexor configuration (`~/.vexor/config.json`)
and index cache.
dependencies and reuses your global Vexor configuration, the safe project
config selected by each tool path, and the corresponding index cache.

```bash
vexor mcp [--path PATH]
Expand Down Expand Up @@ -53,9 +53,10 @@ If `vexor` is not on the client's PATH, use an absolute command path, or
MCP clients can also launch Vexor without installing it, using
`"command": "uvx", "args": ["vexor", "mcp"]`. Notes for this mode:

- Configuration, index caches, and downloaded models live under `~/.vexor`,
so they persist across uvx's ephemeral environments. CLI commands from
error messages work with a prefix: `uvx vexor init`, `uvx vexor config --show`.
- Global configuration and downloaded models live under `~/.vexor`, so they
persist across uvx's ephemeral environments. Project config and local index
caches remain in the project's `.vexor/` directory. CLI commands from error
messages work with a prefix: `uvx vexor init`, `uvx vexor config --show`.
- uvx caches the resolved environment and does **not** auto-upgrade; use
`uvx vexor@latest mcp` to always run the newest release, or refresh
explicitly with `uvx --refresh vexor`. `vexor update --upgrade` applies
Expand All @@ -79,8 +80,8 @@ configuration kept on separate channels:
takes precedence over a stored reranker key and is only needed when
`rerank` is set to `remote`.
- `VEXOR_CONFIG_JSON` (non-secret) — any other Vexor config as a JSON
object, merged over `~/.vexor/config.json`. Accepts the same fields as
the config file: `provider`, `model`, `base_url`, `rerank`,
object, merged over the effective global and project config. Accepts the
full non-secret schema: `provider`, `model`, `base_url`, `rerank`,
`embedding_dimensions`, `auto_index`, and so on. Credential fields
(`api_key`, `remote_rerank.api_key`) are rejected with a clear error so
secrets stay on the dedicated variables above.
Expand Down Expand Up @@ -182,6 +183,11 @@ The tool returns `{path, mode, status, files_indexed}` where `status` is `stored
extra fields).
- Relative `path` arguments resolve against the server's default path
(`--path`, or the server's working directory).
- Each resolved tool path applies `config.json` from its nearest `.vexor/`
marker — a strict behavior-field allowlist that rejects credentials and
endpoints (see
[Project configuration](configuration.md#project-configuration)). MCP
client environment overrides take precedence over it.
- Index cache keys follow the same rules as the CLI (see
[Cache Behavior](cli.md#cache-behavior)): tool calls with the same
path/mode/filters share indexes with CLI usage. When a project contains
Expand Down
19 changes: 7 additions & 12 deletions docs/roadmap.md
Original file line number Diff line number Diff line change
Expand Up @@ -51,18 +51,13 @@ never leaving the machine.

- Add AST-aware `code` mode chunking for Go and Rust (tree-sitter support).
- Project-level config (`<project>/.vexor/config.json`).
- v1: overlay limited to behavior fields that already exist in the
Config schema (`rerank`, `auto_index`, `model`,
`embedding_dimensions`, batch/concurrency). Security constraint: repo
files are attacker-controllable input, so the loader must whitelist
those fields and reject credentials and endpoints (`api_key`,
`base_url`, `remote_rerank`) with an explicit error — stricter than
the `VEXOR_CONFIG_JSON` env override, which permits `base_url`.
Precedence: global config < project config < env overrides < explicit
arguments. Reuse the guarded `config_from_json(base=...)` merge; the
structural change is making config resolution directory-aware
(`load_config()` takes no path today). `vexor config --show` and
`vexor doctor` should display each field's origin (global vs project).
- v1 (shipped): the nearest project marker may override `rerank`,
`auto_index`, `model`, `embedding_dimensions`, `batch_size`,
`embed_concurrency`, and `extract_concurrency`. The strict allowlist
rejects credentials, endpoints, and every other field. Precedence is
global config < project config < environment overrides < explicit
arguments, and `vexor config --show` plus `vexor doctor` surface each
effective value's origin. Mutating config commands remain global-only.
- v2 (only if v1 sees real use): per-project scan defaults (`mode`,
`extensions`, `exclude_patterns`) — these are per-invocation CLI
arguments today, not config fields, so supporting them means new
Expand Down
Loading
Loading