diff --git a/README.md b/README.md index 892ed15..c77ab75 100644 --- a/README.md +++ b/README.md @@ -91,8 +91,9 @@ for hit in response.results: print(hit.path, hit.score) ``` -By default it reads `~/.vexor/config.json`. For runtime config overrides, cache -controls, and per-call options, see [`docs/api/python.md`](https://github.com/scarletkc/vexor/tree/main/docs/api/python.md). +Configuration follows the same global and project-level resolution as the CLI. +For runtime overrides, cache controls, and per-call options, see +[`docs/api/python.md`](https://github.com/scarletkc/vexor/tree/main/docs/api/python.md). ## AI Agent Skill @@ -149,12 +150,16 @@ vexor init # guided setup (recommended) vexor config --set-api-key "YOUR_KEY" # or env: VEXOR_API_KEY / OPENAI_API_KEY / ... vexor config --set-provider openai # default; also gemini/voyageai/custom/local vexor config --rerank hybrid # optional: fuse exact keyword + semantic ranking -vexor config --show # view current settings +vexor config --show # view effective settings and origins ``` -Config lives in `~/.vexor/config.json`. Any field can also be injected via the `VEXOR_CONFIG_JSON` environment variable (useful for MCP client configs and CI), and fully offline use is supported through local embedding models. +Global config lives in `~/.vexor/config.json`; the nearest +`/.vexor/config.json` can safely override selected behavior for that +project. Non-secret fields can also be injected via `VEXOR_CONFIG_JSON` +(useful for MCP clients and CI), and fully offline use is supported through +local embedding models. -See [`docs/configuration.md`](https://github.com/scarletkc/vexor/blob/main/docs/configuration.md) for the complete reference: all config commands, API keys and environment variables, rerank strategies (hybrid / BM25 / FlashRank / remote), remote vs local providers, embedding dimensions, and offline local model setup. +See [`docs/configuration.md`](https://github.com/scarletkc/vexor/blob/main/docs/configuration.md) for the complete reference: project config fields and precedence, all config commands, API keys and environment variables, rerank strategies (hybrid / BM25 / FlashRank / remote), remote vs local providers, embedding dimensions, and offline local model setup. ## CLI Reference diff --git a/docs/api/python.md b/docs/api/python.md index 61d3b2c..2aec5f7 100644 --- a/docs/api/python.md +++ b/docs/api/python.md @@ -1,7 +1,8 @@ # Python API The Python API mirrors the CLI behavior and exposes the same search/index flow. -Configuration can come from disk, in-memory overrides, or per-call parameters. +Configuration can come from global or project files, environment variables, +in-memory overrides, or per-call parameters. ## Quick start @@ -44,12 +45,22 @@ Configuration is resolved in this order (highest to lowest priority): 1. Explicit function arguments (e.g., `provider`, `model`, `api_key`) 2. Per-call override via `config=...` 3. In-memory runtime config set by `set_config_json(...)` -4. `~/.vexor/config.json` (unless `use_config=False`) -5. Built-in defaults +4. Environment overrides, including `VEXOR_CONFIG_JSON` +5. The nearest project `.vexor/config.json` +6. Global `~/.vexor/config.json` +7. Built-in defaults For pure library usage that should ignore on-disk config, pass `use_config=False` and set explicit arguments or `config=...`. +Project resolution starts at the call's resolved `path` and uses the nearest +`.vexor/` marker. The project file accepts exactly `rerank`, `auto_index`, +`model`, `embedding_dimensions`, `batch_size`, `embed_concurrency`, and +`extract_concurrency`. It rejects credentials and endpoints (`api_key`, +`base_url`, `remote_rerank`) and every other field. This restricted schema +applies only to `.vexor/config.json`; explicit API and runtime payloads retain +the full schema below. + Index cache location is resolved separately. Vexor walks upward from `path` and uses the nearest project `.vexor/index.db` when a `.vexor/` directory exists. Explicit per-call `cache_dir` or `data_dir` arguments, client-level overrides, @@ -127,7 +138,7 @@ merging with the current runtime or on-disk config (avoids reading disk). ## Config payload schema -The `config` payload (dict/JSON) supports: +The per-call or in-memory `config` payload (dict/JSON) supports: - `provider`: `openai`, `gemini`, `voyageai`, `custom`, `local` - `model`: embedding model name @@ -144,14 +155,17 @@ The `config` payload (dict/JSON) supports: - `flashrank_model`: string or null - `remote_rerank`: object with `base_url`, `api_key`, `model` -Unknown keys are ignored. `remote_rerank.base_url` is normalized to end with `/rerank`. +Unknown keys in these explicit payloads are ignored; project files reject +them. `remote_rerank.base_url` is normalized to end with `/rerank`. API keys can also come from environment variables: `VEXOR_API_KEY`, `OPENAI_API_KEY`, `GOOGLE_GENAI_API_KEY`, `VOYAGE_API_KEY`, and `VEXOR_REMOTE_RERANK_API_KEY`. -Environment variables only supply API keys; other settings must be passed -explicitly via function arguments or `config=...`. +`VEXOR_CONFIG_JSON` supplies non-secret environment overrides; dedicated +variables supply API keys. Environment values override global and project +files, while explicit function, per-call, and in-memory values take precedence +over the environment. ## Cache behavior diff --git a/docs/cli.md b/docs/cli.md index a7f24c4..55b97bc 100644 --- a/docs/cli.md +++ b/docs/cli.md @@ -8,7 +8,7 @@ | `vexor QUERY` | Shortcut for `vexor search QUERY` | | `vexor search QUERY --path PATH` | Semantic search (auto-indexes if needed) | | `vexor index --path PATH` | Build/refresh index manually | -| `vexor config --show` | Display current configuration | +| `vexor config --show` | Display effective configuration and each field's origin | | `vexor config --clear-flashrank` | Remove cached FlashRank models under `~/.vexor/flashrank` | | `vexor local --setup [--model MODEL]` | Download a local model and set provider to `local` | | `vexor local --clean-up` | Remove local model cache under `~/.vexor/models` | @@ -46,6 +46,18 @@ strategies (`off`, `bm25`, `flashrank`, `remote`, `hybrid`). Porcelain output fields: `rank`, `similarity`, `path`, `chunk_index`, `start_line`, `end_line`, `preview` (line fields are `-` when unavailable). +## Project Configuration + +Search and index commands walk upward from their resolved `--path` and apply +`config.json` from the nearest `.vexor/` marker as a safe overlay on the +global config; credentials and endpoints are rejected. `vexor config --show` +and `vexor doctor` resolve from the current working directory: `--show` +labels each effective field's origin, `doctor` lists active overrides. +Mutating `vexor config` options always write `~/.vexor/config.json`; edit the +project file directly for project-specific values. See +[Configuration → Project configuration](configuration.md#project-configuration) +for the accepted fields and precedence. + ## Ignore Files Use `.vexorignore` for project-specific indexing exclusions. It supports full @@ -86,18 +98,18 @@ Keep flags consistent to reuse cache; changing flags creates a separate index. By default, indexes are stored in `~/.vexor/index.db`. Project-local caching is opt-in: create a `.vexor/` directory in a project root, or run -`vexor index --local`. Either way, Vexor writes a self-ignoring `.gitignore` -(`*`) inside `.vexor/` on first use so the index database cannot be committed -by accident. For each index or search operation, Vexor walks upward -from the resolved target path and uses the nearest `.vexor/` directory it -finds. This means nested projects use their nearest marker. Searching a parent -directory above a project root does not discover markers in its children and -therefore uses the global database. +`vexor index --local`. Either way, Vexor writes a `.gitignore` inside +`.vexor/` on first use so generated indexes and caches cannot be committed by +accident while `config.json` remains eligible for version control. For each +index or search operation, Vexor walks upward from the resolved target path and +uses the nearest `.vexor/` directory it finds. This means nested projects use +their nearest marker. Searching a parent directory above a project root does +not discover markers in its children and therefore uses the global database. Cache location precedence is: an explicit API/cache override, then the nearest -project `.vexor/`, then `~/.vexor/`. Only `index.db` moves into a project. -Configuration, update-check data, FlashRank assets, and local embedding models -remain under the global `~/.vexor/` directory. +project `.vexor/`, then `~/.vexor/`. The project directory may also contain the +safe `config.json` overlay described above. Global configuration, update-check +data, FlashRank assets, and local embedding models remain under `~/.vexor/`. ```bash vexor config --show-index-all # list all cached indexes diff --git a/docs/configuration.md b/docs/configuration.md index c1132be..3658305 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -1,22 +1,79 @@ # Configuration Vexor is configured through `vexor config` commands (or the interactive -`vexor init` wizard). Settings persist in `~/.vexor/config.json`. +`vexor init` wizard). Global settings persist in `~/.vexor/config.json`. ## Where data lives -Configuration, update-check data, FlashRank assets, and local embedding models -stay in the global `~/.vexor/` directory. Indexes normally use -`~/.vexor/index.db`, but a project containing a `.vexor/` directory uses -`/.vexor/index.db` for searches and indexing within that project. +Global configuration, update-check data, FlashRank assets, and local embedding +models stay under `~/.vexor/`. Indexes normally use `~/.vexor/index.db`, but a +project containing a `.vexor/` directory uses `/.vexor/index.db` for +searches and indexing within that project. The same directory may contain a +tracked `config.json` with safe project-level overrides. + Run `vexor index --local` to create the project directory and its ignore file. -Only the index database, including embedding and query cache tables, is -project-local. +This does not create a project config. The generated index and caches remain +ignored while `.vexor/config.json` can be committed. When an upgrade changes the index cache schema, existing index caches are invalidated and rebuilt automatically on next use; cached embeddings are reused, so the migration does not re-embed unchanged content. +## Project configuration + +For each path-aware search or index operation, Vexor resolves the target path +and walks upward to the nearest `.vexor/` directory. If that marker contains +`config.json`, Vexor overlays it on the global configuration. Nested projects +therefore use their nearest marker; a search above a project does not discover +markers below it. If the nearest marker has no `config.json`, Vexor uses the +global configuration instead of continuing into an outer project. + +Project config v1 accepts exactly these fields: + +- `rerank` +- `auto_index` +- `model` +- `embedding_dimensions` +- `batch_size` +- `embed_concurrency` +- `extract_concurrency` + +Example `/.vexor/config.json`: + +```json +{ + "model": "text-embedding-3-small", + "embedding_dimensions": 1024, + "batch_size": 32, + "embed_concurrency": 4, + "extract_concurrency": 4, + "auto_index": true, + "rerank": "hybrid" +} +``` + +Repository files are untrusted input, so this is a strict allowlist. Project +config explicitly rejects credentials and endpoints (`api_key`, `base_url`, +and the entire `remote_rerank` object), as well as every other unsupported or +unknown field. Configure those values globally, through environment variables, +or with explicit Python API arguments. This is stricter than +`VEXOR_CONFIG_JSON`, which may set non-secret fields such as `base_url`. + +Effective precedence, from lowest to highest, is: + +1. Built-in defaults +2. Global `~/.vexor/config.json` +3. The nearest project `.vexor/config.json` +4. Environment overrides, including `VEXOR_CONFIG_JSON` +5. Explicit runtime or API arguments + +`vexor config --show` and `vexor doctor` resolve the project from the current +working directory: `--show` labels every effective field with its origin, and +`doctor` reports the project config file plus any project or environment +overrides. All mutating `vexor config` options and `vexor init` still write +only the global config; edit `.vexor/config.json` directly to change a +project override. + ## Commands ```bash @@ -44,7 +101,7 @@ vexor config --set-remote-rerank-api-key $VEXOR_REMOTE_RERANK_API_KEY # or env vexor config --clear-remote-rerank # clear remote rerank config vexor config --set-base-url https://proxy.example.com # optional proxy vexor config --clear-base-url # reset to official endpoint -vexor config --show # view current settings +vexor config --show # view effective settings and origins ``` Rerank defaults to `off`. **It is highly recommended to configure the @@ -62,8 +119,8 @@ or `VOYAGE_API_KEY`; `VEXOR_API_KEY` takes precedence over a stored key. Any non-secret config field can also be injected as a JSON object via the `VEXOR_CONFIG_JSON` environment variable (useful for MCP client configs and -CI), merged over `~/.vexor/config.json`. Credential fields inside -`VEXOR_CONFIG_JSON` are rejected — use the dedicated variables above. +CI), merged over the effective global and project config. Credential fields +inside `VEXOR_CONFIG_JSON` are rejected — use the dedicated variables above. ## Rerank diff --git a/docs/development.md b/docs/development.md index 834eded..62ade65 100644 --- a/docs/development.md +++ b/docs/development.md @@ -7,4 +7,8 @@ pytest ``` Tests rely on fake embedding backends, so no network access is required. -Cache files and configuration live in `~/.vexor` (a project with a `.vexor/` directory keeps its index there instead). The text embedded for each chunk is built by the mode strategies in `vexor/modes.py`; adjust the strategy `label` construction there if you need to encode additional context. \ No newline at end of file +Global cache files and configuration live in `~/.vexor`. A project with a +`.vexor/` directory keeps its index there and may add the restricted +`config.json` overlay documented in `docs/configuration.md`. The text embedded +for each chunk is built by the mode strategies in `vexor/modes.py`; adjust the +strategy `label` construction there if you need to encode additional context. diff --git a/docs/mcp.md b/docs/mcp.md index 43a4293..e505c08 100644 --- a/docs/mcp.md +++ b/docs/mcp.md @@ -3,8 +3,8 @@ `vexor mcp` runs a [Model Context Protocol](https://modelcontextprotocol.io) server over stdio, exposing Vexor's semantic search to any MCP-capable agent (Claude Code, Codex, Cursor, Windsurf, Zed, and others). It requires no extra -dependencies and reuses your existing Vexor configuration (`~/.vexor/config.json`) -and index cache. +dependencies and reuses your global Vexor configuration, the safe project +config selected by each tool path, and the corresponding index cache. ```bash vexor mcp [--path PATH] @@ -53,9 +53,10 @@ If `vexor` is not on the client's PATH, use an absolute command path, or MCP clients can also launch Vexor without installing it, using `"command": "uvx", "args": ["vexor", "mcp"]`. Notes for this mode: -- Configuration, index caches, and downloaded models live under `~/.vexor`, - so they persist across uvx's ephemeral environments. CLI commands from - error messages work with a prefix: `uvx vexor init`, `uvx vexor config --show`. +- Global configuration and downloaded models live under `~/.vexor`, so they + persist across uvx's ephemeral environments. Project config and local index + caches remain in the project's `.vexor/` directory. CLI commands from error + messages work with a prefix: `uvx vexor init`, `uvx vexor config --show`. - uvx caches the resolved environment and does **not** auto-upgrade; use `uvx vexor@latest mcp` to always run the newest release, or refresh explicitly with `uvx --refresh vexor`. `vexor update --upgrade` applies @@ -79,8 +80,8 @@ configuration kept on separate channels: takes precedence over a stored reranker key and is only needed when `rerank` is set to `remote`. - `VEXOR_CONFIG_JSON` (non-secret) — any other Vexor config as a JSON - object, merged over `~/.vexor/config.json`. Accepts the same fields as - the config file: `provider`, `model`, `base_url`, `rerank`, + object, merged over the effective global and project config. Accepts the + full non-secret schema: `provider`, `model`, `base_url`, `rerank`, `embedding_dimensions`, `auto_index`, and so on. Credential fields (`api_key`, `remote_rerank.api_key`) are rejected with a clear error so secrets stay on the dedicated variables above. @@ -182,6 +183,11 @@ The tool returns `{path, mode, status, files_indexed}` where `status` is `stored extra fields). - Relative `path` arguments resolve against the server's default path (`--path`, or the server's working directory). +- Each resolved tool path applies `config.json` from its nearest `.vexor/` + marker — a strict behavior-field allowlist that rejects credentials and + endpoints (see + [Project configuration](configuration.md#project-configuration)). MCP + client environment overrides take precedence over it. - Index cache keys follow the same rules as the CLI (see [Cache Behavior](cli.md#cache-behavior)): tool calls with the same path/mode/filters share indexes with CLI usage. When a project contains diff --git a/docs/roadmap.md b/docs/roadmap.md index 6deaee9..96e5628 100644 --- a/docs/roadmap.md +++ b/docs/roadmap.md @@ -51,18 +51,13 @@ never leaving the machine. - Add AST-aware `code` mode chunking for Go and Rust (tree-sitter support). - Project-level config (`/.vexor/config.json`). - - v1: overlay limited to behavior fields that already exist in the - Config schema (`rerank`, `auto_index`, `model`, - `embedding_dimensions`, batch/concurrency). Security constraint: repo - files are attacker-controllable input, so the loader must whitelist - those fields and reject credentials and endpoints (`api_key`, - `base_url`, `remote_rerank`) with an explicit error — stricter than - the `VEXOR_CONFIG_JSON` env override, which permits `base_url`. - Precedence: global config < project config < env overrides < explicit - arguments. Reuse the guarded `config_from_json(base=...)` merge; the - structural change is making config resolution directory-aware - (`load_config()` takes no path today). `vexor config --show` and - `vexor doctor` should display each field's origin (global vs project). + - v1 (shipped): the nearest project marker may override `rerank`, + `auto_index`, `model`, `embedding_dimensions`, `batch_size`, + `embed_concurrency`, and `extract_concurrency`. The strict allowlist + rejects credentials, endpoints, and every other field. Precedence is + global config < project config < environment overrides < explicit + arguments, and `vexor config --show` plus `vexor doctor` surface each + effective value's origin. Mutating config commands remain global-only. - v2 (only if v1 sees real use): per-project scan defaults (`mode`, `extensions`, `exclude_patterns`) — these are per-invocation CLI arguments today, not config fields, so supporting them means new diff --git a/plugins/vexor/skills/vexor-cli/SKILL.md b/plugins/vexor/skills/vexor-cli/SKILL.md index b0630e9..817a689 100644 --- a/plugins/vexor/skills/vexor-cli/SKILL.md +++ b/plugins/vexor/skills/vexor-cli/SKILL.md @@ -35,6 +35,19 @@ vexor "" [--path ] [--mode ] [--ext .py,.md] [--exclude-patte - `--no-cache`: in-memory only, do not read/write index cache - `vexor index --local`: create and use project-local `.vexor/index.db` storage +## Project Config + +- The nearest `.vexor/config.json` applies automatically for the resolved + search or index path. +- It accepts only `rerank`, `auto_index`, `model`, `embedding_dimensions`, + `batch_size`, `embed_concurrency`, and `extract_concurrency`. +- Credentials and endpoints (`api_key`, `base_url`, `remote_rerank`) and all + other fields are rejected. +- Precedence is global config, project config, environment overrides, then + explicit arguments. +- `vexor config --show` labels each field's origin and `vexor doctor` lists + active overrides; mutating `vexor config` commands remain global-only. + ## Modes (pick the cheapest that works) - `auto`: routes by file type (default) @@ -51,7 +64,7 @@ vexor "" [--path ] [--mode ] [--ext .py,.md] [--exclude-patte - Need ignored or hidden files: add `--include-hidden` and/or `--no-respect-gitignore`. - Scriptable output: use `--format porcelain` (TSV) or `--format porcelain-z` (NUL-delimited). - Get detailed help: `vexor search --help`. -- Config issues: `vexor doctor` or `vexor config --show` diagnoses API, cache, and connectivity (tell the user to set up). +- Config issues: `vexor doctor` or `vexor config --show` reports effective values and their origins. ## Examples diff --git a/tests/integration/test_cli.py b/tests/integration/test_cli.py index df36a3d..4cd4667 100644 --- a/tests/integration/test_cli.py +++ b/tests/integration/test_cli.py @@ -82,6 +82,129 @@ def fake_perform_search(request): assert captured["mode"] == "auto" +def test_search_and_index_use_project_config_from_target_path(tmp_path, monkeypatch): + runner = CliRunner() + project = tmp_path / "project" + project_config = project / ".vexor" / "config.json" + project_config.parent.mkdir(parents=True) + project_config.write_text( + json.dumps( + { + "model": "project-model", + "batch_size": 11, + "embed_concurrency": 3, + "extract_concurrency": 2, + "auto_index": False, + "rerank": "hybrid", + } + ), + encoding="utf-8", + ) + captured = {} + + def fake_perform_search(request): + captured["search"] = request + return SearchResponse( + base_path=project, + backend=None, + results=[], + is_stale=False, + index_empty=True, + ) + + def fake_build_index(_directory, **kwargs): + captured["index"] = kwargs + return IndexResult(status=IndexStatus.EMPTY) + + monkeypatch.setattr("vexor.cli.perform_search", fake_perform_search) + monkeypatch.setattr("vexor.cli.build_index", fake_build_index) + + search_result = runner.invoke( + app, ["search", "hello", "--path", str(project), "--mode", "name"] + ) + index_result = runner.invoke( + app, ["index", "--path", str(project), "--mode", "name"] + ) + + assert search_result.exit_code == 0 + assert index_result.exit_code == 0 + request = captured["search"] + assert request.model_name == "project-model" + assert request.batch_size == 11 + assert request.embed_concurrency == 3 + assert request.extract_concurrency == 2 + assert request.auto_index is False + assert request.rerank == "hybrid" + assert captured["index"]["model_name"] == "project-model" + assert captured["index"]["batch_size"] == 11 + + +def test_search_project_config_does_not_pollute_porcelain_output( + tmp_path, monkeypatch +): + runner = CliRunner() + project = tmp_path / "project" + sample = project / "alpha.txt" + sample.parent.mkdir() + sample.write_text("data", encoding="utf-8") + project_config = project / ".vexor" / "config.json" + project_config.parent.mkdir() + project_config.write_text(json.dumps({"batch_size": 11}), encoding="utf-8") + + def fake_perform_search(_request): + return SearchResponse( + base_path=project, + backend=None, + results=[SearchResult(path=sample, score=0.9, preview="alpha")], + is_stale=False, + index_empty=False, + ) + + monkeypatch.setattr("vexor.cli.perform_search", fake_perform_search) + + result = runner.invoke( + app, + [ + "search", + "hello", + "--path", + str(project), + "--mode", + "name", + "--format", + "porcelain", + ], + ) + + assert result.exit_code == 0 + assert result.stdout.count("\n") == 1 + assert len(result.stdout.rstrip("\n").split("\t")) == 7 + assert "(project)" not in result.stdout + assert "Field origins" not in result.stdout + + +def test_search_rejects_sensitive_project_config_without_traceback( + tmp_path, monkeypatch +): + runner = CliRunner() + project_config = tmp_path / "project" / ".vexor" / "config.json" + project_config.parent.mkdir(parents=True) + project_config.write_text( + json.dumps({"api_key": "must-not-load"}), encoding="utf-8" + ) + + result = runner.invoke( + app, + ["search", "hello", "--path", str(project_config.parent.parent)], + ) + + assert result.exit_code == 1 + assert result.stdout == "" + assert "must not contain: api_key" in result.stderr + assert str(project_config) in result.stderr + assert "Traceback" not in result.output + + def test_search_outputs_hybrid_reranker_label(tmp_path, monkeypatch): sample_file = tmp_path / "exact.py" sample_file.write_text("data") @@ -906,6 +1029,99 @@ def test_config_set_and_show(tmp_path): assert "FlashRank model" not in strip_ansi(result_show.stdout) +def test_config_show_reports_global_project_and_environment_origins( + tmp_path, monkeypatch, temp_config_home +): + runner = CliRunner() + temp_config_home.parent.mkdir(parents=True) + temp_config_home.write_text( + json.dumps({"provider": "gemini", "model": "global-model"}), + encoding="utf-8", + ) + project = tmp_path / "project" + project_config = project / ".vexor" / "config.json" + project_config.parent.mkdir(parents=True) + project_config.write_text( + json.dumps({"model": "project-model", "auto_index": False}), + encoding="utf-8", + ) + monkeypatch.setenv("VEXOR_CONFIG_JSON", json.dumps({"batch_size": 23})) + monkeypatch.chdir(project) + + result = runner.invoke(app, ["config", "--show"]) + output = strip_ansi(result.stdout) + + assert result.exit_code == 0 + assert "Default provider: gemini (global)" in output + assert "Default model: project-model (project)" in output + assert "Default batch size: 23 (environment)" in output + assert "Auto index: no (project)" in output + assert "Custom base URL: none (default)" in output + assert "Embedding dimensions: auto (default)" in output + + +def test_config_show_reports_provider_env_api_key(monkeypatch, temp_config_home): + runner = CliRunner() + temp_config_home.parent.mkdir(parents=True) + temp_config_home.write_text(json.dumps({"provider": "openai"}), encoding="utf-8") + monkeypatch.delenv("VEXOR_API_KEY", raising=False) + monkeypatch.setenv("OPENAI_API_KEY", "sk-env-key") + + result = runner.invoke(app, ["config", "--show"]) + output = strip_ansi(result.stdout) + + assert result.exit_code == 0 + assert "API key set: yes (environment)" in output + + +def test_config_show_remote_rerank_env_key_keeps_block_origin( + monkeypatch, temp_config_home +): + runner = CliRunner() + temp_config_home.parent.mkdir(parents=True) + temp_config_home.write_text( + json.dumps( + { + "rerank": "remote", + "remote_rerank": { + "base_url": "https://rerank.example.test/rerank", + "model": "rerank-model", + }, + } + ), + encoding="utf-8", + ) + monkeypatch.setenv("VEXOR_REMOTE_RERANK_API_KEY", "env-remote-key") + + result = runner.invoke(app, ["config", "--show"]) + output = strip_ansi(result.stdout) + + assert result.exit_code == 0 + flat = " ".join(output.split()) + assert "key from env) (global)" in flat + + +def test_config_setter_in_project_still_updates_only_global_config( + tmp_path, monkeypatch, temp_config_home +): + runner = CliRunner() + project = tmp_path / "project" + project_config = project / ".vexor" / "config.json" + project_config.parent.mkdir(parents=True) + project_config.write_text(json.dumps({"batch_size": 7}), encoding="utf-8") + monkeypatch.chdir(project) + + result = runner.invoke(app, ["config", "--set-batch-size", "42"]) + + assert result.exit_code == 0 + assert json.loads(project_config.read_text(encoding="utf-8")) == { + "batch_size": 7 + } + assert json.loads(temp_config_home.read_text(encoding="utf-8"))[ + "batch_size" + ] == 42 + + def test_config_hybrid_rerank_round_trip(tmp_path): runner = CliRunner() result = runner.invoke(app, ["config", "--rerank", "hybrid"]) @@ -1401,6 +1617,104 @@ def test_doctor_handles_malformed_config(monkeypatch, temp_config_home): assert "invalid json" in result.stdout.lower() +def test_doctor_reports_project_config_overrides( + tmp_path, monkeypatch, temp_config_home +): + from vexor.services import system_service + + runner = CliRunner() + temp_config_home.parent.mkdir(parents=True) + temp_config_home.write_text( + json.dumps({"provider": "local", "model": "global-model"}), + encoding="utf-8", + ) + project = tmp_path / "project" + project_config = project / ".vexor" / "config.json" + project_config.parent.mkdir(parents=True) + project_config.write_text( + json.dumps({"model": "project-model", "rerank": "hybrid"}), + encoding="utf-8", + ) + monkeypatch.setenv("VEXOR_CONFIG_JSON", json.dumps({"batch_size": 23})) + monkeypatch.chdir(project) + monkeypatch.setattr( + system_service, + "check_command_on_path", + lambda: system_service.DoctorCheckResult("Command", True, "ok"), + ) + monkeypatch.setattr( + system_service, + "check_cache_directory", + lambda: system_service.DoctorCheckResult("Cache Dir", True, "ok"), + ) + + result = runner.invoke(app, ["doctor", "--skip-api-test"]) + output = strip_ansi(result.stdout) + + assert result.exit_code == 0 + assert "Project config:" in output + assert ".vexor" in output + assert "config.json" in output + assert "Project overrides: model, rerank" in output + assert "Environment overrides: batch_size" in output + assert "provider: global" not in output + assert "base_url: default" not in output + + +def test_doctor_reports_disallowed_project_config_without_traceback( + tmp_path, monkeypatch +): + runner = CliRunner() + project_config = tmp_path / "project" / ".vexor" / "config.json" + project_config.parent.mkdir(parents=True) + project_config.write_text( + json.dumps({"base_url": "https://must-not-load.example"}), + encoding="utf-8", + ) + monkeypatch.chdir(project_config.parent.parent) + + result = runner.invoke(app, ["doctor", "--skip-api-test"]) + + assert result.exit_code == 1 + assert "must not contain: base_url" in result.stdout + assert ".vexor" in result.stdout + assert "config.json" in result.stdout + assert "Traceback" not in result.output + + +def test_doctor_project_config_error_falls_back_to_global_config( + tmp_path, monkeypatch, temp_config_home +): + from vexor.services import system_service + + runner = CliRunner() + temp_config_home.parent.mkdir(parents=True) + temp_config_home.write_text(json.dumps({"provider": "local"}), encoding="utf-8") + project_config = tmp_path / "project" / ".vexor" / "config.json" + project_config.parent.mkdir(parents=True) + project_config.write_text("{broken", encoding="utf-8") + monkeypatch.chdir(project_config.parent.parent) + monkeypatch.setattr( + system_service, + "check_command_on_path", + lambda: system_service.DoctorCheckResult("Command", True, "ok"), + ) + monkeypatch.setattr( + system_service, + "check_cache_directory", + lambda: system_service.DoctorCheckResult("Cache Dir", True, "ok"), + ) + + result = runner.invoke(app, ["doctor", "--skip-api-test"]) + output = strip_ansi(result.stdout) + + assert result.exit_code == 1 + assert "Invalid project config" in output + # The global provider=local still drives the remaining checks. + assert "no API key required" in output + assert "API key not configured" not in output + + def test_update_detects_newer_version(monkeypatch): runner = CliRunner() monkeypatch.setattr("vexor.cli.fetch_latest_pypi_version", lambda *_args, **_kwargs: "9.9.9") @@ -1806,7 +2120,9 @@ def embed_texts(self, texts): assert index_result.exit_code == 0 assert (project / ".vexor" / "index.db").is_file() - assert (project / ".vexor" / ".gitignore").read_text(encoding="utf-8") == "*\n" + assert (project / ".vexor" / ".gitignore").read_text(encoding="utf-8") == ( + "*\n!.gitignore\n!config.json\n" + ) assert not (global_cache / "index.db").exists() root_search = runner.invoke( diff --git a/tests/unit/test_api.py b/tests/unit/test_api.py index 9f77d4d..f4cc100 100644 --- a/tests/unit/test_api.py +++ b/tests/unit/test_api.py @@ -1,3 +1,5 @@ +import json + import numpy as np import pytest @@ -28,7 +30,7 @@ def test_search_uses_config_defaults(tmp_path, monkeypatch) -> None: model="rerank-model", ), ) - monkeypatch.setattr(api_module, "load_config", lambda: cfg) + monkeypatch.setattr(api_module, "load_config", lambda _directory=None: cfg) captured: dict[str, object] = {} def fake_perform_search(request): @@ -77,7 +79,7 @@ def test_search_overrides_config(tmp_path, monkeypatch) -> None: auto_index=True, local_cuda=False, ) - monkeypatch.setattr(api_module, "load_config", lambda: cfg) + monkeypatch.setattr(api_module, "load_config", lambda _directory=None: cfg) captured: dict[str, object] = {} def fake_perform_search(request): @@ -123,6 +125,205 @@ def fake_perform_search(request): assert req.embedding_dimensions == 512 +def test_project_config_applies_to_search_index_and_in_memory( + tmp_path, monkeypatch +) -> None: + config_dir = tmp_path / "global-config" + config_dir.mkdir() + (config_dir / "config.json").write_text( + json.dumps( + { + "provider": "openai", + "model": "text-embedding-3-small", + "batch_size": 4, + } + ), + encoding="utf-8", + ) + project = tmp_path / "project" + project.mkdir() + project_config = project / ".vexor" / "config.json" + project_config.parent.mkdir() + project_config.write_text( + json.dumps( + { + "model": "text-embedding-3-large", + "batch_size": 9, + "embed_concurrency": 3, + "extract_concurrency": 2, + "embedding_dimensions": 512, + "auto_index": False, + "rerank": "hybrid", + } + ), + encoding="utf-8", + ) + captured: dict[str, object] = {} + + def fake_perform_search(request): + captured["search"] = request + return SearchResponse( + base_path=project, + backend=None, + results=[], + is_stale=False, + index_empty=True, + ) + + def fake_build_index(_directory, **kwargs): + captured["index"] = kwargs + return IndexResult(status=IndexStatus.EMPTY) + + def fake_build_index_in_memory(directory, **kwargs): + captured["memory"] = kwargs + return [], np.empty((0, 0), dtype=np.float32), { + "include_hidden": False, + "respect_gitignore": True, + "recursive": True, + "mode": "name", + "exclude_patterns": (), + "extensions": (), + "chunks": [], + } + + monkeypatch.setattr(api_module, "perform_search", fake_perform_search) + monkeypatch.setattr(api_module, "build_index", fake_build_index) + monkeypatch.setattr( + api_module, "build_index_in_memory", fake_build_index_in_memory + ) + + api_module.search("hello", path=project, mode="name", config_dir=config_dir) + api_module.index(project, mode="name", config_dir=config_dir) + api_module.index_in_memory(project, mode="name", config_dir=config_dir) + + search_request = captured["search"] + assert search_request.model_name == "text-embedding-3-large" + assert search_request.batch_size == 9 + assert search_request.embed_concurrency == 3 + assert search_request.extract_concurrency == 2 + assert search_request.embedding_dimensions == 512 + assert search_request.auto_index is False + assert search_request.rerank == "hybrid" + for key in ("index", "memory"): + kwargs = captured[key] + assert kwargs["model_name"] == "text-embedding-3-large" + assert kwargs["batch_size"] == 9 + assert kwargs["embed_concurrency"] == 3 + assert kwargs["extract_concurrency"] == 2 + assert kwargs["embedding_dimensions"] == 512 + + +def test_api_explicit_and_per_call_config_override_project_config( + tmp_path, monkeypatch +) -> None: + config_dir = tmp_path / "global-config" + config_dir.mkdir() + project = tmp_path / "project" + project_config = project / ".vexor" / "config.json" + project_config.parent.mkdir(parents=True) + project_config.write_text( + json.dumps( + { + "model": "text-embedding-3-large", + "batch_size": 9, + "auto_index": False, + } + ), + encoding="utf-8", + ) + captured: dict[str, object] = {} + + def fake_perform_search(request): + captured["request"] = request + return SearchResponse( + base_path=project, + backend=None, + results=[], + is_stale=False, + index_empty=True, + ) + + monkeypatch.setattr(api_module, "perform_search", fake_perform_search) + + api_module.search( + "hello", + path=project, + mode="name", + config_dir=config_dir, + config={"batch_size": 18, "auto_index": False}, + model="text-embedding-3-small", + batch_size=20, + auto_index=True, + ) + + request = captured["request"] + assert request.model_name == "text-embedding-3-small" + assert request.batch_size == 20 + assert request.auto_index is True + + +def test_runtime_config_overrides_only_its_fields_after_project_config( + tmp_path, monkeypatch +) -> None: + config_dir = tmp_path / "global-config" + config_dir.mkdir() + (config_dir / "config.json").write_text( + json.dumps({"model": "text-embedding-3-small", "batch_size": 4}), + encoding="utf-8", + ) + project = tmp_path / "project" + project_config = project / ".vexor" / "config.json" + project_config.parent.mkdir(parents=True) + project_config.write_text( + json.dumps({"model": "text-embedding-3-large", "batch_size": 9}), + encoding="utf-8", + ) + captured: list[object] = [] + + def fake_perform_search(request): + captured.append(request) + return SearchResponse( + base_path=project, + backend=None, + results=[], + is_stale=False, + index_empty=True, + ) + + monkeypatch.setattr(api_module, "perform_search", fake_perform_search) + client = api_module.VexorClient(config_dir=config_dir) + client.set_config_json({"batch_size": 17}) + + client.search("hello", path=project, mode="name") + + assert captured[-1].model_name == "text-embedding-3-large" + assert captured[-1].batch_size == 17 + + client.set_config_json({"batch_size": 5}, replace=True) + client.search("hello", path=project, mode="name") + + assert captured[-1].model_name == DEFAULT_MODEL + assert captured[-1].batch_size == 5 + + +def test_api_wraps_project_config_errors(tmp_path) -> None: + config_dir = tmp_path / "global-config" + config_dir.mkdir() + project_config = tmp_path / "project" / ".vexor" / "config.json" + project_config.parent.mkdir(parents=True) + project_config.write_text( + json.dumps({"api_key": "must-not-load"}), encoding="utf-8" + ) + + with pytest.raises(api_module.VexorError, match="must not contain: api_key"): + api_module.search( + "hello", + path=project_config.parent.parent, + mode="name", + config_dir=config_dir, + ) + + def test_set_data_dir_updates_config_and_cache(tmp_path) -> None: from vexor import cache as cache_module from vexor import config as config_module @@ -359,7 +560,9 @@ def fake_perform_search(request): def test_client_config_context_scopes_runtime_config(tmp_path, monkeypatch) -> None: base_config = Config(provider="openai") - monkeypatch.setattr(api_module, "load_config", lambda: base_config) + monkeypatch.setattr( + api_module, "load_config", lambda _directory=None: base_config + ) captured: list[object] = [] def fake_perform_search(request): @@ -499,7 +702,9 @@ def test_search_rejects_unsupported_dimension_for_model(tmp_path) -> None: def test_config_context_yields_configured_client(tmp_path, monkeypatch) -> None: base_config = Config(provider="openai") - monkeypatch.setattr(api_module, "load_config", lambda: base_config) + monkeypatch.setattr( + api_module, "load_config", lambda _directory=None: base_config + ) captured: dict[str, object] = {} def fake_perform_search(request): diff --git a/tests/unit/test_cache.py b/tests/unit/test_cache.py index 0cf94f4..da67866 100644 --- a/tests/unit/test_cache.py +++ b/tests/unit/test_cache.py @@ -8,6 +8,7 @@ import vexor.cache as cache MODE = "name" +PROJECT_CACHE_GITIGNORE = "*\n!.gitignore\n!config.json\n" def _entries_for_files( @@ -920,7 +921,17 @@ def test_create_project_cache_dir_creates_marker_and_gitignore(tmp_path): assert marker == tmp_path / ".vexor" assert marker.is_dir() - assert (marker / ".gitignore").read_text(encoding="utf-8") == "*\n" + assert (marker / ".gitignore").read_text(encoding="utf-8") == PROJECT_CACHE_GITIGNORE + + +def test_create_project_cache_dir_migrates_legacy_gitignore(tmp_path): + marker = tmp_path / ".vexor" + marker.mkdir() + gitignore = marker / ".gitignore" + gitignore.write_text("*\n", encoding="utf-8") + + assert cache.create_project_cache_dir(tmp_path) == marker + assert gitignore.read_text(encoding="utf-8") == PROJECT_CACHE_GITIGNORE def test_create_project_cache_dir_preserves_existing_gitignore(tmp_path): @@ -933,6 +944,17 @@ def test_create_project_cache_dir_preserves_existing_gitignore(tmp_path): assert gitignore.read_text(encoding="utf-8") == "keep-this\n" +def test_create_project_cache_dir_preserves_custom_legacy_based_gitignore(tmp_path): + marker = tmp_path / ".vexor" + marker.mkdir() + gitignore = marker / ".gitignore" + custom = "*\n!custom.json\n" + gitignore.write_text(custom, encoding="utf-8") + + assert cache.create_project_cache_dir(tmp_path) == marker + assert gitignore.read_text(encoding="utf-8") == custom + + def test_project_cache_context_writes_self_ignore_for_manual_marker( tmp_path, monkeypatch ): @@ -946,7 +968,7 @@ def test_project_cache_context_writes_self_ignore_for_manual_marker( with cache.project_cache_context(project): pass - assert (marker / ".gitignore").read_text(encoding="utf-8") == "*\n" + assert (marker / ".gitignore").read_text(encoding="utf-8") == PROJECT_CACHE_GITIGNORE def test_project_cache_context_preserves_existing_gitignore(tmp_path, monkeypatch): diff --git a/tests/unit/test_config.py b/tests/unit/test_config.py index 0da9fac..c55d862 100644 --- a/tests/unit/test_config.py +++ b/tests/unit/test_config.py @@ -29,6 +29,245 @@ def test_load_config_defaults(tmp_path, monkeypatch): assert cfg.remote_rerank is None +def test_project_config_overlays_allowed_fields_and_tracks_origins( + tmp_path, monkeypatch +): + config_file = _prepare_config(tmp_path, monkeypatch) + config_file.parent.mkdir(parents=True) + config_file.write_text( + json.dumps( + { + "provider": "gemini", + "model": "global-model", + "batch_size": 8, + "embed_concurrency": 2, + "extract_concurrency": 3, + "auto_index": True, + "rerank": "off", + "base_url": "https://global.example.test", + } + ), + encoding="utf-8", + ) + project = tmp_path / "project" + project_config = project / ".vexor" / "config.json" + project_config.parent.mkdir(parents=True) + project_config.write_text( + json.dumps( + { + "model": "project-model", + "batch_size": 12, + "embed_concurrency": 5, + "extract_concurrency": 6, + "embedding_dimensions": 512, + "auto_index": False, + "rerank": "hybrid", + } + ), + encoding="utf-8", + ) + child = project / "src" / "package" + child.mkdir(parents=True) + + resolution = config_module.resolve_config(child) + cfg = resolution.config + + assert resolution.project_file == project_config.resolve() + assert cfg.provider == "gemini" + assert cfg.base_url == "https://global.example.test" + assert cfg.model == "project-model" + assert cfg.batch_size == 12 + assert cfg.embed_concurrency == 5 + assert cfg.extract_concurrency == 6 + assert cfg.embedding_dimensions == 512 + assert cfg.auto_index is False + assert cfg.rerank == "hybrid" + assert resolution.origin_for("provider") is config_module.ConfigOrigin.GLOBAL + assert resolution.origin_for("base_url") is config_module.ConfigOrigin.GLOBAL + for field in config_module.PROJECT_CONFIG_FIELDS: + assert resolution.origin_for(field) is config_module.ConfigOrigin.PROJECT + assert ( + resolution.origin_for("flashrank_model") + is config_module.ConfigOrigin.DEFAULT + ) + + +def test_project_config_null_clears_global_embedding_dimensions( + tmp_path, monkeypatch +): + config_file = _prepare_config(tmp_path, monkeypatch) + config_file.parent.mkdir(parents=True) + config_file.write_text( + json.dumps({"embedding_dimensions": 1024}), encoding="utf-8" + ) + project = tmp_path / "project" + project_config = project / ".vexor" / "config.json" + project_config.parent.mkdir(parents=True) + project_config.write_text( + json.dumps({"embedding_dimensions": None}), encoding="utf-8" + ) + + resolution = config_module.resolve_config(project) + + assert resolution.config.embedding_dimensions is None + assert ( + resolution.origin_for("embedding_dimensions") + is config_module.ConfigOrigin.PROJECT + ) + + +def test_nearest_project_marker_without_config_blocks_outer_config( + tmp_path, monkeypatch +): + config_file = _prepare_config(tmp_path, monkeypatch) + config_file.parent.mkdir(parents=True) + config_file.write_text(json.dumps({"model": "global-model"}), encoding="utf-8") + outer = tmp_path / "outer" + outer_config = outer / ".vexor" / "config.json" + outer_config.parent.mkdir(parents=True) + outer_config.write_text(json.dumps({"model": "outer-model"}), encoding="utf-8") + inner = outer / "inner" + (inner / ".vexor").mkdir(parents=True) + child = inner / "src" + child.mkdir() + + resolution = config_module.resolve_config(child) + + assert resolution.config.model == "global-model" + assert resolution.project_file == (inner / ".vexor" / "config.json").resolve() + assert resolution.origin_for("model") is config_module.ConfigOrigin.GLOBAL + + +def test_load_config_without_directory_keeps_legacy_global_scope( + tmp_path, monkeypatch +): + config_file = _prepare_config(tmp_path, monkeypatch) + config_file.parent.mkdir(parents=True) + config_file.write_text(json.dumps({"model": "global-model"}), encoding="utf-8") + project = tmp_path / "project" + project_config = project / ".vexor" / "config.json" + project_config.parent.mkdir(parents=True) + project_config.write_text( + json.dumps({"model": "project-model"}), encoding="utf-8" + ) + monkeypatch.chdir(project) + + assert config_module.load_config().model == "global-model" + assert config_module.load_config(project).model == "project-model" + + +@pytest.mark.parametrize( + ("field", "value"), + [ + ("api_key", "project-secret"), + ("base_url", "https://project.example.test"), + ("remote_rerank", {"base_url": "https://rerank.example.test"}), + ], +) +def test_project_config_rejects_sensitive_fields( + tmp_path, monkeypatch, field, value +): + _prepare_config(tmp_path, monkeypatch) + project_config = tmp_path / "project" / ".vexor" / "config.json" + project_config.parent.mkdir(parents=True) + project_config.write_text(json.dumps({field: value}), encoding="utf-8") + + with pytest.raises(config_module.ProjectConfigError) as exc_info: + config_module.load_config(project_config.parent.parent) + + message = str(exc_info.value) + assert str(project_config) in message + assert field in message + assert "project-secret" not in message + assert "Credentials and endpoints" in message + + +@pytest.mark.parametrize( + "field", + [ + "provider", + "extract_backend", + "update_check", + "local_cuda", + "flashrank_model", + "unknown_field", + ], +) +def test_project_config_rejects_non_whitelisted_fields( + tmp_path, monkeypatch, field +): + _prepare_config(tmp_path, monkeypatch) + project_config = tmp_path / "project" / ".vexor" / "config.json" + project_config.parent.mkdir(parents=True) + project_config.write_text(json.dumps({field: "value"}), encoding="utf-8") + + with pytest.raises(config_module.ProjectConfigError) as exc_info: + config_module.load_config(project_config.parent.parent) + + message = str(exc_info.value) + assert str(project_config) in message + assert field in message + assert "Allowed fields" in message + + +@pytest.mark.parametrize("payload", ["{", "[]"]) +def test_project_config_rejects_malformed_or_non_object_json( + tmp_path, monkeypatch, payload +): + _prepare_config(tmp_path, monkeypatch) + project_config = tmp_path / "project" / ".vexor" / "config.json" + project_config.parent.mkdir(parents=True) + project_config.write_text(payload, encoding="utf-8") + + with pytest.raises(config_module.ProjectConfigError, match="Invalid project config"): + config_module.load_config(project_config.parent.parent) + + +def test_project_config_wraps_invalid_allowed_field_value(tmp_path, monkeypatch): + _prepare_config(tmp_path, monkeypatch) + project_config = tmp_path / "project" / ".vexor" / "config.json" + project_config.parent.mkdir(parents=True) + project_config.write_text( + json.dumps({"auto_index": "sometimes"}), encoding="utf-8" + ) + + with pytest.raises(config_module.ProjectConfigError) as exc_info: + config_module.load_config(project_config.parent.parent) + + assert str(project_config) in str(exc_info.value) + assert "invalid value for auto_index" in str(exc_info.value) + + +def test_project_config_env_precedence_and_origins(tmp_path, monkeypatch): + config_file = _prepare_config(tmp_path, monkeypatch) + config_file.parent.mkdir(parents=True) + config_file.write_text( + json.dumps({"model": "global-model", "batch_size": 8}), + encoding="utf-8", + ) + project = tmp_path / "project" + project_config = project / ".vexor" / "config.json" + project_config.parent.mkdir(parents=True) + project_config.write_text( + json.dumps({"model": "project-model", "batch_size": 12}), + encoding="utf-8", + ) + monkeypatch.setenv( + config_module.ENV_CONFIG_JSON, + json.dumps({"model": "env-model"}), + ) + monkeypatch.setenv(config_module.ENV_API_KEY, "env-secret") + + resolution = config_module.resolve_config(project) + + assert resolution.config.model == "env-model" + assert resolution.config.batch_size == 12 + assert resolution.config.api_key == "env-secret" + assert resolution.origin_for("model") is config_module.ConfigOrigin.ENVIRONMENT + assert resolution.origin_for("batch_size") is config_module.ConfigOrigin.PROJECT + assert resolution.origin_for("api_key") is config_module.ConfigOrigin.ENVIRONMENT + + def test_config_dir_context_overrides_config_file(tmp_path): original_config_dir = config_module.CONFIG_DIR with config_module.config_dir_context(tmp_path): @@ -39,6 +278,24 @@ def test_config_dir_context_overrides_config_file(tmp_path): assert config_module.CONFIG_DIR == original_config_dir +def test_explicit_config_dir_is_not_reloaded_as_untrusted_project_config(tmp_path): + project = tmp_path / "project" + explicit_config_dir = project / ".vexor" + + with config_module.config_dir_context(explicit_config_dir): + config_module.save_config( + config_module.Config( + api_key="trusted-key", + base_url="https://trusted.example.test", + ) + ) + resolution = config_module.resolve_config(project) + + assert resolution.project_file is None + assert resolution.config.api_key == "trusted-key" + assert resolution.config.base_url == "https://trusted.example.test" + + def test_resolve_default_model_gemini_defaults() -> None: assert ( config_module.resolve_default_model("gemini", None) diff --git a/tests/unit/test_mcp_service.py b/tests/unit/test_mcp_service.py index 97c1f29..fbf5735 100644 --- a/tests/unit/test_mcp_service.py +++ b/tests/unit/test_mcp_service.py @@ -510,6 +510,51 @@ def test_index_tool_passes_local(tmp_path): assert client.index_calls[0]["local"] is True +def test_default_mcp_client_applies_project_config_for_tool_path( + tmp_path, monkeypatch +): + from vexor import api as api_module + from vexor import config as config_module + + global_config = tmp_path / "global" / "config.json" + global_config.parent.mkdir() + global_config.write_text( + json.dumps({"model": "global-model", "batch_size": 4}), + encoding="utf-8", + ) + monkeypatch.setattr(config_module, "CONFIG_DIR", global_config.parent) + monkeypatch.setattr(config_module, "CONFIG_FILE", global_config) + project = tmp_path / "project" + project_config = project / ".vexor" / "config.json" + project_config.parent.mkdir(parents=True) + project_config.write_text( + json.dumps({"model": "project-model", "batch_size": 13}), + encoding="utf-8", + ) + captured = {} + + def fake_perform_search(request): + captured["request"] = request + return SearchResponse( + base_path=project, + backend=None, + results=[], + is_stale=False, + index_empty=True, + ) + + monkeypatch.setattr(api_module, "perform_search", fake_perform_search) + server = VexorMcpServer(default_path=project) + + response = server.handle_message( + tool_call(SEARCH_TOOL, {"query": "config loader"}) + ) + + assert response["result"]["isError"] is False + assert captured["request"].model_name == "project-model" + assert captured["request"].batch_size == 13 + + def test_index_tool_rejects_non_boolean_local(tmp_path): server, client = make_server(tmp_path) diff --git a/tests/unit/test_system_service.py b/tests/unit/test_system_service.py index 1cade43..4e4f933 100644 --- a/tests/unit/test_system_service.py +++ b/tests/unit/test_system_service.py @@ -223,6 +223,70 @@ def test_command_config_api_key_and_cache_checks(monkeypatch, tmp_path): assert writable.passed is True +def test_check_config_exists_reports_override_summary(tmp_path): + from vexor.config import ( + CONFIG_FIELD_NAMES, + Config, + ConfigOrigin, + ConfigResolution, + ) + + global_file = tmp_path / "global" / "config.json" + global_file.parent.mkdir() + global_file.write_text("{}", encoding="utf-8") + project_file = tmp_path / "project" / ".vexor" / "config.json" + project_file.parent.mkdir(parents=True) + project_file.write_text("{}", encoding="utf-8") + origins = {field: ConfigOrigin.DEFAULT for field in CONFIG_FIELD_NAMES} + origins["provider"] = ConfigOrigin.GLOBAL + origins["rerank"] = ConfigOrigin.PROJECT + origins["model"] = ConfigOrigin.PROJECT + origins["batch_size"] = ConfigOrigin.ENVIRONMENT + resolution = ConfigResolution( + config=Config(), + origins=origins, + global_file=global_file, + project_file=project_file, + ) + + result = system_service.check_config_exists(resolution) + + assert result.passed is True + assert result.detail is not None + assert f"Project config: {project_file}" in result.detail + assert "Project overrides: model, rerank" in result.detail + assert "Environment overrides: batch_size" in result.detail + assert "provider: global" not in result.detail + + global_file.unlink() + project_only = system_service.check_config_exists(resolution) + assert "Project config file exists" in project_only.message + + +def test_check_config_exists_omits_override_lines_without_overrides(tmp_path): + from vexor.config import ( + CONFIG_FIELD_NAMES, + Config, + ConfigOrigin, + ConfigResolution, + ) + + global_file = tmp_path / "global" / "config.json" + global_file.parent.mkdir() + global_file.write_text("{}", encoding="utf-8") + origins = {field: ConfigOrigin.GLOBAL for field in CONFIG_FIELD_NAMES} + resolution = ConfigResolution( + config=Config(), + origins=origins, + global_file=global_file, + project_file=None, + ) + + result = system_service.check_config_exists(resolution) + + assert result.detail == "Project config: none" + + def test_check_cache_directory_reports_create_and_write_failures(monkeypatch, tmp_path): config_dir = tmp_path / "cache" monkeypatch.setattr("vexor.config.CONFIG_DIR", config_dir) diff --git a/vexor/api.py b/vexor/api.py index 37422cb..1961b2a 100644 --- a/vexor/api.py +++ b/vexor/api.py @@ -18,6 +18,7 @@ Config, RemoteRerankConfig, SUPPORTED_RERANKERS, + _coerce_config_payload, config_from_json, config_dir_context, load_config, @@ -169,7 +170,39 @@ def search( ) -_RUNTIME_CONFIG: Config | None = None +@dataclass(frozen=True, slots=True) +class _RuntimeConfigOverride: + payload: Mapping[str, object] + replace: bool = False + + +_RUNTIME_CONFIG: _RuntimeConfigOverride | None = None + + +def _update_runtime_config_override( + current: _RuntimeConfigOverride | None, + payload: Mapping[str, object] | str, + *, + replace: bool, +) -> _RuntimeConfigOverride: + """Validate and accumulate a deferred in-memory config override.""" + + try: + data = dict(_coerce_config_payload(payload)) + if current is None or replace: + combined = data + effective_replace = replace + else: + combined = {**current.payload, **data} + effective_replace = current.replace + base = Config() if effective_replace else load_config() + config_from_json(combined, base=base) + except (ValueError, OSError, UnicodeDecodeError) as exc: + raise VexorError(str(exc)) from exc + return _RuntimeConfigOverride( + payload=combined, + replace=effective_replace, + ) @contextmanager @@ -206,11 +239,11 @@ def set_config_json( if payload is None: _RUNTIME_CONFIG = None return - base = None if replace else (_RUNTIME_CONFIG or load_config()) - try: - _RUNTIME_CONFIG = config_from_json(payload, base=base) - except ValueError as exc: - raise VexorError(str(exc)) from exc + _RUNTIME_CONFIG = _update_runtime_config_override( + _RUNTIME_CONFIG, + payload, + replace=replace, + ) class VexorClient: @@ -228,7 +261,7 @@ def __init__( self.config_dir = config_dir self.cache_dir = cache_dir self.use_config = use_config - self._runtime_config: Config | None = None + self._runtime_config: _RuntimeConfigOverride | None = None def set_config_json( self, @@ -240,11 +273,11 @@ def set_config_json( if payload is None: self._runtime_config = None return - base = None if replace else (self._runtime_config or load_config()) - try: - self._runtime_config = config_from_json(payload, base=base) - except ValueError as exc: - raise VexorError(str(exc)) from exc + self._runtime_config = _update_runtime_config_override( + self._runtime_config, + payload, + replace=replace, + ) @contextmanager def config_context( @@ -744,7 +777,7 @@ def _search_with_settings( config: Config | Mapping[str, object] | str | None, temporary_index: bool, no_cache: bool, - runtime_config: Config | None, + runtime_config: _RuntimeConfigOverride | None, data_dir: Path | str | None, config_dir: Path | str | None, cache_dir: Path | str | None, @@ -768,6 +801,7 @@ def _search_with_settings( raise VexorError(Messages.ERROR_EXTENSIONS_EMPTY) settings = _resolve_settings( + directory=directory, provider=provider, model=model, batch_size=batch_size, @@ -836,7 +870,7 @@ def _index_with_settings( local: bool, use_config: bool, config: Config | Mapping[str, object] | str | None, - runtime_config: Config | None, + runtime_config: _RuntimeConfigOverride | None, data_dir: Path | str | None, config_dir: Path | str | None, cache_dir: Path | str | None, @@ -855,6 +889,7 @@ def _index_with_settings( raise VexorError(Messages.ERROR_EXTENSIONS_EMPTY) settings = _resolve_settings( + directory=directory, provider=provider, model=model, batch_size=batch_size, @@ -914,7 +949,7 @@ def _index_in_memory_with_settings( use_config: bool, config: Config | Mapping[str, object] | str | None, no_cache: bool, - runtime_config: Config | None, + runtime_config: _RuntimeConfigOverride | None, data_dir: Path | str | None, config_dir: Path | str | None, cache_dir: Path | str | None, @@ -930,6 +965,7 @@ def _index_in_memory_with_settings( raise VexorError(Messages.ERROR_EXTENSIONS_EMPTY) settings = _resolve_settings( + directory=directory, provider=provider, model=model, batch_size=batch_size, @@ -1051,6 +1087,7 @@ def _coerce_iterable(values: Sequence[str] | str | None) -> tuple[str, ...]: def _resolve_settings( *, + directory: Path | str | None, provider: str | None, model: str | None, batch_size: int | None, @@ -1063,14 +1100,20 @@ def _resolve_settings( embedding_dimensions: int | None, auto_index: bool | None, use_config: bool, - runtime_config: Config | None = None, + runtime_config: _RuntimeConfigOverride | None = None, config_override: Config | Mapping[str, object] | str | None = None, ) -> RuntimeSettings: - config = ( - runtime_config if (use_config and runtime_config is not None) else None - ) - if config is None: - config = load_config() if use_config else Config() + try: + if not use_config: + config = Config() + elif runtime_config is not None and runtime_config.replace: + config = config_from_json(runtime_config.payload, base=Config()) + else: + config = load_config(directory) + if runtime_config is not None: + config = config_from_json(runtime_config.payload, base=config) + except (ValueError, OSError, UnicodeDecodeError) as exc: + raise VexorError(str(exc)) from exc if config_override is not None: config = _apply_config_override(config, config_override) provider_value = (provider or config.provider or DEFAULT_PROVIDER).lower() diff --git a/vexor/cache.py b/vexor/cache.py index 4f2d3a3..ad77fc5 100644 --- a/vexor/cache.py +++ b/vexor/cache.py @@ -21,6 +21,8 @@ DEFAULT_CACHE_DIR = Path(os.path.expanduser("~")) / ".vexor" CACHE_DIR = DEFAULT_CACHE_DIR PROJECT_CACHE_DIRNAME = ".vexor" +_LEGACY_PROJECT_CACHE_GITIGNORE = "*\n" +_PROJECT_CACHE_GITIGNORE = "*\n!.gitignore\n!config.json\n" _CACHE_DIR_OVERRIDE: ContextVar[Path | None] = ContextVar( "vexor_cache_dir_override", default=None, @@ -227,18 +229,22 @@ def find_project_cache_dir(path: Path) -> Path | None: def _ensure_project_cache_self_ignore(project_cache_dir: Path) -> None: - """Write the self-ignoring .gitignore into *project_cache_dir* if missing. + """Write or migrate the generated .gitignore in *project_cache_dir*. Users can opt in by creating `.vexor/` by hand; without this marker the - index database would show up as untracked and could get committed. + index database would show up as untracked and could get committed. The + project config and the ignore file itself remain available to version + control. Custom ignore files are left untouched. """ gitignore_path = project_cache_dir / ".gitignore" - if gitignore_path.exists(): - return try: - gitignore_path.write_text("*\n", encoding="utf-8") - except OSError: + if gitignore_path.exists(): + current = gitignore_path.read_text(encoding="utf-8") + if current != _LEGACY_PROJECT_CACHE_GITIGNORE: + return + gitignore_path.write_text(_PROJECT_CACHE_GITIGNORE, encoding="utf-8") + except (OSError, UnicodeDecodeError): # Hygiene write only: a search against a read-only tree must not # fail because the ignore marker could not be created. pass diff --git a/vexor/cli.py b/vexor/cli.py index 32b63fa..0bea835 100644 --- a/vexor/cli.py +++ b/vexor/cli.py @@ -55,11 +55,16 @@ DIMENSION_SUPPORTED_MODELS, SUPPORTED_PROVIDERS, get_supported_dimensions, + resolve_api_key, resolve_default_model, supports_dimensions, ) from .services.cache_service import is_cache_current, load_index_metadata_safe -from .services.config_service import apply_config_updates, get_config_snapshot +from .services.config_service import ( + apply_config_updates, + get_config_origin_labels, + get_config_resolution, +) from .services.init_service import run_init_wizard, should_auto_run_init from .services.index_service import IndexStatus, build_index, clear_index_entries from .services.search_service import SearchRequest, perform_search, _select_cache_superset @@ -203,6 +208,16 @@ def _parse_boolean(value: str) -> bool: raise ValueError(Messages.ERROR_BOOLEAN_INVALID.format(value=value)) +def _load_config_or_exit(directory: Path | str | None = None) -> config_module.Config: + """Load config for a CLI command and render failures without a traceback.""" + + try: + return load_config(directory) + except (ValueError, OSError, UnicodeDecodeError) as exc: + typer.echo(str(exc), err=True) + raise typer.Exit(code=1) from exc + + def _prepare_flashrank_model(model_name: str | None) -> None: try: from flashrank import Ranker @@ -417,7 +432,8 @@ def search( ), ) -> None: """Run the semantic search.""" - config = load_config() + directory = resolve_directory(path) + config = _load_config_or_exit(directory) provider = (config.provider or DEFAULT_PROVIDER).lower() model_name = resolve_default_model(provider, config.model) batch_size = config.batch_size if config.batch_size is not None else DEFAULT_BATCH_SIZE @@ -443,7 +459,6 @@ def search( except ValueError as exc: # pragma: no cover - validated by Typer raise typer.BadParameter(str(exc)) from exc - directory = resolve_directory(path) mode_value = _validate_mode(mode) recursive = not no_recursive normalized_exts = normalize_extensions(extensions) @@ -605,7 +620,8 @@ def index( ), ) -> None: """Create or refresh the cached index for the given directory.""" - config = load_config() + directory = resolve_directory(path) + config = _load_config_or_exit(directory) provider = (config.provider or DEFAULT_PROVIDER).lower() model_name = resolve_default_model(provider, config.model) batch_size = config.batch_size if config.batch_size is not None else DEFAULT_BATCH_SIZE @@ -615,7 +631,6 @@ def index( base_url = config.base_url api_key = config.api_key - directory = resolve_directory(path) if local: project_cache_dir = cache.create_project_cache_dir(directory) console.print( @@ -879,7 +894,7 @@ def config( help=Messages.HELP_CLEAR_INDEX_ALL, ), ) -> None: - """Manage Vexor configuration stored in ~/.vexor/config.json.""" + """Manage global config and inspect effective project settings.""" if set_batch_option is not None and set_batch_option < 0: raise typer.BadParameter(Messages.ERROR_BATCH_NEGATIVE) if set_embed_concurrency_option is not None and set_embed_concurrency_option < 1: @@ -982,7 +997,7 @@ def config( raise typer.BadParameter(Messages.ERROR_FLASHRANK_MISSING) set_rerank_option = normalized_rerank - config_snapshot = load_config() + config_snapshot = _load_config_or_exit() current_provider = (config_snapshot.provider or DEFAULT_PROVIDER).lower() pending_provider = set_provider_option or current_provider pending_model = set_model_option if set_model_option is not None else config_snapshot.model @@ -1158,7 +1173,7 @@ def config( flashrank_model = ( set_flashrank_model_option if set_flashrank_model_option is not None - else get_config_snapshot().flashrank_model + else _load_config_or_exit().flashrank_model ) console.print(_styled(Messages.INFO_FLASHRANK_SETUP_START, Styles.INFO)) try: @@ -1276,16 +1291,24 @@ def config( return if show: - cfg = get_config_snapshot() + try: + resolution = get_config_resolution(Path.cwd()) + except (ValueError, OSError, UnicodeDecodeError) as exc: + typer.echo(str(exc), err=True) + raise typer.Exit(code=1) from exc + cfg = resolution.config + origins = get_config_origin_labels(resolution) provider = (cfg.provider or DEFAULT_PROVIDER).lower() rerank = (cfg.rerank or DEFAULT_RERANK).lower() flashrank_line = "" remote_rerank_line = "" if rerank == "flashrank": model_label = cfg.flashrank_model or f"default ({DEFAULT_FLASHRANK_MODEL})" - flashrank_line = ( - f"{Messages.INFO_FLASHRANK_MODEL_SUMMARY.format(value=model_label)}\n" + flashrank_summary = Messages.INFO_FLASHRANK_MODEL_SUMMARY.format( + value=model_label, + origin=origins["flashrank_model"], ) + flashrank_line = f"{flashrank_summary}\n" if rerank == "remote": remote_cfg = cfg.remote_rerank if remote_cfg is None: @@ -1293,19 +1316,34 @@ def config( else: url_label = remote_cfg.base_url or "unset" model_label = remote_cfg.model or "unset" - key_label = "yes" if remote_cfg.api_key else "no" + if os.getenv(config_module.REMOTE_RERANK_ENV): + key_label = "from env" + else: + key_label = "yes" if remote_cfg.api_key else "no" remote_label = f"{url_label} (model {model_label}, key {key_label})" - remote_rerank_line = ( - f"{Messages.INFO_REMOTE_RERANK_SUMMARY.format(value=remote_label)}\n" + remote_rerank_summary = Messages.INFO_REMOTE_RERANK_SUMMARY.format( + value=remote_label, + origin=origins["remote_rerank"], ) + remote_rerank_line = f"{remote_rerank_summary}\n" + embedding_dimensions = cfg.embedding_dimensions or "auto" + batch_size = ( + cfg.batch_size + if cfg.batch_size is not None + else DEFAULT_BATCH_SIZE + ) + effective_api_key = cfg.api_key or resolve_api_key(None, provider) + api_origin = origins["api_key"] + if effective_api_key and not cfg.api_key: + api_origin = config_module.ConfigOrigin.ENVIRONMENT.value console.print( _styled( Messages.INFO_CONFIG_SUMMARY.format( - api="yes" if cfg.api_key else "no", + api="yes" if effective_api_key else "no", provider=provider, model=resolve_default_model(provider, cfg.model), - embedding_dimensions=cfg.embedding_dimensions if cfg.embedding_dimensions else "default", - batch=cfg.batch_size if cfg.batch_size is not None else DEFAULT_BATCH_SIZE, + embedding_dimensions=embedding_dimensions, + batch=batch_size, concurrency=cfg.embed_concurrency, extract_concurrency=cfg.extract_concurrency, extract_backend=cfg.extract_backend, @@ -1316,6 +1354,19 @@ def config( remote_rerank_line=remote_rerank_line, local_cuda="yes" if cfg.local_cuda else "no", base_url=cfg.base_url or "none", + api_origin=api_origin, + provider_origin=origins["provider"], + model_origin=origins["model"], + embedding_dimensions_origin=origins["embedding_dimensions"], + batch_origin=origins["batch_size"], + concurrency_origin=origins["embed_concurrency"], + extract_concurrency_origin=origins["extract_concurrency"], + extract_backend_origin=origins["extract_backend"], + auto_index_origin=origins["auto_index"], + update_check_origin=origins["update_check"], + rerank_origin=origins["rerank"], + local_cuda_origin=origins["local_cuda"], + base_url_origin=origins["base_url"], ), Styles.INFO, ) @@ -1607,15 +1658,35 @@ def doctor( console.print() config_load_error: DoctorCheckResult | None = None + config_resolution = None try: - config = load_config() - except (json.JSONDecodeError, OSError, UnicodeDecodeError) as exc: - config = config_module.Config() + config_resolution = get_config_resolution(Path.cwd()) + config = config_resolution.config + except (ValueError, OSError, UnicodeDecodeError) as exc: + # Fall back to the global-only config so the remaining checks reflect + # the user's real settings instead of blank defaults. + try: + config = get_config_resolution(None).config + except (ValueError, OSError, UnicodeDecodeError): + config = config_module.Config() + if isinstance(exc, config_module.ProjectConfigError): + message = str(exc) + detail = None + elif isinstance(exc, ValueError) and not isinstance( + exc, json.JSONDecodeError + ): + message = str(exc) + detail = None + else: + message = Messages.DOCTOR_CONFIG_INVALID.format( + path=config_module.CONFIG_FILE + ) + detail = str(exc) config_load_error = DoctorCheckResult( name="Config JSON", passed=False, - message=Messages.DOCTOR_CONFIG_INVALID.format(path=config_module.CONFIG_FILE), - detail=str(exc), + message=message, + detail=detail, ) provider = (config.provider or DEFAULT_PROVIDER).lower() @@ -1636,6 +1707,7 @@ def doctor( rerank=config.rerank, flashrank_model=config.flashrank_model, remote_rerank=config.remote_rerank, + config_resolution=config_resolution, ) ) @@ -1645,9 +1717,13 @@ def doctor( if not result.passed: has_failure = True - console.print(f" {icon} [bold]{result.name}:[/bold] {result.message}") + console.print( + f" {icon} [bold]{result.name}:[/bold] {result.message}", + soft_wrap=True, + ) if result.detail: - console.print(f" [dim]{result.detail}[/dim]") + for line in result.detail.splitlines(): + console.print(f" [dim]{line}[/dim]", soft_wrap=True) console.print() if has_failure: diff --git a/vexor/config.py b/vexor/config.py index c0e8e03..303ee51 100644 --- a/vexor/config.py +++ b/vexor/config.py @@ -8,6 +8,7 @@ from contextlib import contextmanager from contextvars import ContextVar from collections.abc import Mapping +from enum import Enum from pathlib import Path from typing import Any, Dict from urllib.parse import urlparse, urlunparse @@ -60,6 +61,21 @@ ENV_CONFIG_JSON = "VEXOR_CONFIG_JSON" ENV_NO_UPDATE_CHECK = "VEXOR_NO_UPDATE_CHECK" REMOTE_RERANK_ENV = "VEXOR_REMOTE_RERANK_API_KEY" +PROJECT_CONFIG_FILENAME = "config.json" +PROJECT_CONFIG_FIELDS = frozenset( + { + "auto_index", + "batch_size", + "embed_concurrency", + "embedding_dimensions", + "extract_concurrency", + "model", + "rerank", + } +) +PROJECT_CONFIG_SENSITIVE_FIELDS = frozenset( + {"api_key", "base_url", "remote_rerank"} +) @dataclass @@ -88,6 +104,39 @@ class Config: embedding_dimensions: int | None = None +class ConfigOrigin(str, Enum): + """Source of an effective configuration field.""" + + DEFAULT = "default" + GLOBAL = "global" + PROJECT = "project" + ENVIRONMENT = "environment" + + +CONFIG_FIELD_NAMES = tuple(Config.__dataclass_fields__) + + +@dataclass(frozen=True, slots=True) +class ConfigResolution: + """Effective configuration plus source metadata for each field.""" + + config: Config + origins: Mapping[str, ConfigOrigin] + global_file: Path + project_file: Path | None = None + + def origin_for(self, field: str) -> ConfigOrigin: + return self.origins.get(field, ConfigOrigin.DEFAULT) + + +class ProjectConfigError(ValueError): + """Raised when a project-controlled config cannot be applied safely.""" + + def __init__(self, message: str, *, path: Path) -> None: + super().__init__(message) + self.path = path + + def _parse_remote_rerank(raw: object) -> RemoteRerankConfig | None: if not isinstance(raw, dict): return None @@ -115,6 +164,27 @@ def _resolve_config_file() -> Path: return CONFIG_FILE +def _resolve_project_config_file( + directory: Path | str | None, +) -> Path | None: + """Return the config candidate under the nearest project marker.""" + + if directory is None: + return None + + # Import lazily so importing the standalone config module does not pull in + # the cache's NumPy/SQLite dependencies. + from .cache import find_project_cache_dir + + project_dir = find_project_cache_dir(Path(directory)) + if project_dir is None: + return None + project_file = project_dir / PROJECT_CONFIG_FILENAME + if project_file.resolve() == _resolve_config_file().resolve(): + return None + return project_file + + @contextmanager def config_dir_context(path: Path | str | None): """Temporarily override the config directory for the current context.""" @@ -132,7 +202,11 @@ def config_dir_context(path: Path | str | None): _CONFIG_DIR_OVERRIDE.reset(token) -def _apply_env_overrides(config: Config) -> Config: +def _apply_env_overrides( + config: Config, + *, + origins: dict[str, ConfigOrigin] | None = None, +) -> Config: """Merge the VEXOR_CONFIG_JSON environment override over *config*. Lets MCP client configs (and CI) inject any non-secret config field via @@ -159,6 +233,10 @@ def _apply_env_overrides(config: Config) -> Config: ) ) config = config_from_json(data, base=config) + if origins is not None: + for field in data: + if field in origins: + origins[field] = ConfigOrigin.ENVIRONMENT except ValueError as exc: raise ValueError( Messages.ERROR_ENV_CONFIG_JSON_INVALID.format(reason=exc) @@ -170,18 +248,31 @@ def _apply_env_overrides(config: Config) -> Config: api_key = os.getenv(ENV_API_KEY) if api_key: config.api_key = api_key + if origins is not None: + origins["api_key"] = ConfigOrigin.ENVIRONMENT remote_rerank_api_key = os.getenv(REMOTE_RERANK_ENV) if remote_rerank_api_key and config.remote_rerank is not None: + # Key-only injection: the endpoint/model still come from the stored + # block, so the field's origin is left unchanged. config.remote_rerank.api_key = remote_rerank_api_key return config -def _load_stored_config() -> Config: - """Load the persisted config without applying environment overrides.""" +def _load_stored_config_payload() -> Mapping[str, object]: + """Load the persisted global JSON object without applying it.""" + config_file = _resolve_config_file() if not config_file.exists(): - return Config() + return {} raw = json.loads(config_file.read_text(encoding="utf-8")) + if not isinstance(raw, Mapping): + raise ValueError(Messages.ERROR_CONFIG_JSON_INVALID) + return raw + + +def _config_from_stored_payload(raw: Mapping[str, object]) -> Config: + """Build a Config while preserving the global file's legacy coercions.""" + rerank = (raw.get("rerank") or DEFAULT_RERANK).strip().lower() if rerank not in SUPPORTED_RERANKERS: rerank = DEFAULT_RERANK @@ -206,9 +297,103 @@ def _load_stored_config() -> Config: ) -def load_config() -> Config: - """Load persisted configuration and apply process-local environment overrides.""" - return _apply_env_overrides(_load_stored_config()) +def _load_stored_config() -> Config: + """Load the persisted config without applying environment overrides.""" + + return _config_from_stored_payload(_load_stored_config_payload()) + + +def _load_project_config( + base: Config, + directory: Path | str | None, +) -> tuple[Config, Path | None, tuple[str, ...]]: + project_file = _resolve_project_config_file(directory) + if project_file is None or not project_file.exists(): + return base, project_file, () + + try: + raw = json.loads(project_file.read_text(encoding="utf-8")) + except (json.JSONDecodeError, OSError, UnicodeDecodeError) as exc: + raise ProjectConfigError( + Messages.ERROR_PROJECT_CONFIG_INVALID.format( + path=project_file, + reason=exc, + ), + path=project_file, + ) from exc + if not isinstance(raw, Mapping): + raise ProjectConfigError( + Messages.ERROR_PROJECT_CONFIG_INVALID.format( + path=project_file, + reason=Messages.ERROR_CONFIG_JSON_INVALID, + ), + path=project_file, + ) + + fields = set(raw) + sensitive = sorted(fields & PROJECT_CONFIG_SENSITIVE_FIELDS) + if sensitive: + raise ProjectConfigError( + Messages.ERROR_PROJECT_CONFIG_SENSITIVE.format( + path=project_file, + fields=", ".join(sensitive), + ), + path=project_file, + ) + unsupported = sorted(fields - PROJECT_CONFIG_FIELDS) + if unsupported: + raise ProjectConfigError( + Messages.ERROR_PROJECT_CONFIG_UNSUPPORTED.format( + path=project_file, + fields=", ".join(unsupported), + allowed=", ".join(sorted(PROJECT_CONFIG_FIELDS)), + ), + path=project_file, + ) + + try: + config = config_from_json(raw, base=base) + except ValueError as exc: + raise ProjectConfigError( + Messages.ERROR_PROJECT_CONFIG_INVALID.format( + path=project_file, + reason=exc, + ), + path=project_file, + ) from exc + return config, project_file, tuple(sorted(fields)) + + +def resolve_config( + directory: Path | str | None = None, +) -> ConfigResolution: + """Resolve global, project, and environment config with field origins.""" + + global_file = _resolve_config_file() + stored = _load_stored_config_payload() + config = _config_from_stored_payload(stored) + origins = {field: ConfigOrigin.DEFAULT for field in CONFIG_FIELD_NAMES} + for field in stored: + if field in origins: + origins[field] = ConfigOrigin.GLOBAL + + config, project_file, project_fields = _load_project_config(config, directory) + for field in project_fields: + origins[field] = ConfigOrigin.PROJECT + + config = _apply_env_overrides(config, origins=origins) + return ConfigResolution( + config=config, + origins=origins, + global_file=global_file, + project_file=project_file, + ) + + +def load_config(directory: Path | str | None = None) -> Config: + """Load effective config for *directory* and apply environment overrides.""" + + return resolve_config(directory).config def save_config(config: Config) -> None: diff --git a/vexor/services/config_service.py b/vexor/services/config_service.py index d09b31d..0489d0f 100644 --- a/vexor/services/config_service.py +++ b/vexor/services/config_service.py @@ -3,10 +3,15 @@ from __future__ import annotations from dataclasses import dataclass +from pathlib import Path from ..config import ( + CONFIG_FIELD_NAMES, Config, + ConfigOrigin, + ConfigResolution, load_config, + resolve_config, set_api_key, set_base_url, set_batch_size, @@ -190,7 +195,38 @@ def apply_config_updates( return result -def get_config_snapshot() -> Config: +def get_config_snapshot(directory: Path | str | None = None) -> Config: """Return the current configuration dataclass.""" - return load_config() + return load_config(directory) + + +def get_config_resolution( + directory: Path | str | None = None, +) -> ConfigResolution: + """Return effective config and per-field source metadata.""" + + return resolve_config(directory) + + +def collect_config_overrides( + resolution: ConfigResolution, +) -> dict[ConfigOrigin, tuple[str, ...]]: + """Return fields overridden by the project or environment, sorted by name.""" + + grouped: dict[ConfigOrigin, list[str]] = {} + for field in sorted(CONFIG_FIELD_NAMES): + origin = resolution.origin_for(field) + if origin in (ConfigOrigin.PROJECT, ConfigOrigin.ENVIRONMENT): + grouped.setdefault(origin, []).append(field) + return {origin: tuple(fields) for origin, fields in grouped.items()} + + +def get_config_origin_labels( + resolution: ConfigResolution, +) -> dict[str, str]: + """Return stable user-facing origin labels keyed by config field.""" + + return { + field: resolution.origin_for(field).value for field in CONFIG_FIELD_NAMES + } diff --git a/vexor/services/system_service.py b/vexor/services/system_service.py index 92d2ff3..64e0802 100644 --- a/vexor/services/system_service.py +++ b/vexor/services/system_service.py @@ -20,6 +20,8 @@ from ..config import ( Config, + ConfigOrigin, + ConfigResolution, DEFAULT_FLASHRANK_MODEL, DEFAULT_RERANK, ENV_NO_UPDATE_CHECK, @@ -30,6 +32,7 @@ update_check_file, ) from ..text import Messages +from .config_service import collect_config_overrides EDITOR_FALLBACKS = ("nano", "vi", "notepad", "notepad.exe") @@ -61,22 +64,62 @@ def check_command_on_path() -> DoctorCheckResult: ) -def check_config_exists() -> DoctorCheckResult: +def check_config_exists( + resolution: ConfigResolution | None = None, +) -> DoctorCheckResult: """Check if config file exists.""" from ..config import CONFIG_FILE - if CONFIG_FILE.exists(): - return DoctorCheckResult( + config_file = resolution.global_file if resolution is not None else CONFIG_FILE + project_file = resolution.project_file if resolution is not None else None + project_exists = project_file is not None and project_file.exists() + if config_file.exists(): + result = DoctorCheckResult( name="Config", passed=True, - message=Messages.DOCTOR_CONFIG_EXISTS.format(path=CONFIG_FILE), + message=Messages.DOCTOR_CONFIG_EXISTS.format(path=config_file), ) - return DoctorCheckResult( - name="Config", - passed=True, - message=Messages.DOCTOR_CONFIG_DEFAULT, - detail=str(CONFIG_FILE), + elif project_exists: + result = DoctorCheckResult( + name="Config", + passed=True, + message=Messages.DOCTOR_PROJECT_CONFIG_EXISTS.format(path=project_file), + ) + else: + result = DoctorCheckResult( + name="Config", + passed=True, + message=( + Messages.DOCTOR_CONFIG_NO_GLOBAL + if resolution is not None + else Messages.DOCTOR_CONFIG_DEFAULT + ), + detail=str(config_file), + ) + + if resolution is None: + return result + + project_label = str(project_file) if project_exists else "none" + detail_lines = [Messages.DOCTOR_CONFIG_PROJECT.format(project=project_label)] + overrides = collect_config_overrides(resolution) + project_fields = overrides.get(ConfigOrigin.PROJECT) + if project_fields: + detail_lines.append( + Messages.DOCTOR_CONFIG_PROJECT_OVERRIDES.format( + fields=", ".join(project_fields) + ) + ) + env_fields = overrides.get(ConfigOrigin.ENVIRONMENT) + if env_fields: + detail_lines.append( + Messages.DOCTOR_CONFIG_ENV_OVERRIDES.format(fields=", ".join(env_fields)) + ) + source_detail = "\n".join(detail_lines) + result.detail = ( + f"{result.detail}\n{source_detail}" if result.detail else source_detail ) + return result def check_api_key_configured(provider: str, api_key: str | None) -> DoctorCheckResult: @@ -387,11 +430,17 @@ def run_all_doctor_checks( rerank: str | None = None, flashrank_model: str | None = None, remote_rerank: RemoteRerankConfig | None = None, + config_resolution: ConfigResolution | None = None, ) -> list[DoctorCheckResult]: """Run all doctor checks and return results.""" + config_check = ( + check_config_exists(config_resolution) + if config_resolution is not None + else check_config_exists() + ) results = [ check_command_on_path(), - check_config_exists(), + config_check, check_cache_directory(), check_api_key_configured(provider, api_key), ] diff --git a/vexor/text.py b/vexor/text.py index 61cde89..f17caf4 100644 --- a/vexor/text.py +++ b/vexor/text.py @@ -150,7 +150,7 @@ class Messages: "Set the remote rerank API key (or use VEXOR_REMOTE_RERANK_API_KEY)." ) HELP_CLEAR_REMOTE_RERANK = "Clear remote rerank configuration." - HELP_SHOW_CONFIG = "Show current configuration." + HELP_SHOW_CONFIG = "Show effective configuration and each field's origin." HELP_SHOW_INDEX_ALL = "Show metadata for every index in the active cache database." HELP_CLEAR_INDEX_ALL = "Delete every index in the active cache database." HELP_INSTALL_SKILLS = ( @@ -412,26 +412,37 @@ class Messages: ERROR_ENV_CONFIG_JSON_SECRET = ( "must not contain '{field}'; use the {env} environment variable instead" ) + ERROR_PROJECT_CONFIG_INVALID = "Invalid project config at {path}: {reason}" + ERROR_PROJECT_CONFIG_SENSITIVE = ( + "Project config at {path} must not contain: {fields}. Credentials and " + "endpoints must be configured globally or through environment variables." + ) + ERROR_PROJECT_CONFIG_UNSUPPORTED = ( + "Project config at {path} contains unsupported fields: {fields}. " + "Allowed fields: {allowed}." + ) ERROR_CONFIG_VALUE_INVALID = "Config JSON has invalid value for {field}." INFO_CONFIG_SUMMARY = ( - "API key set: {api}\n" - "Default provider: {provider}\n" - "Default model: {model}\n" - "Embedding dimensions: {embedding_dimensions}\n" - "Default batch size: {batch}\n" - "Embedding concurrency: {concurrency}\n" - "Extract concurrency: {extract_concurrency}\n" - "Extract backend: {extract_backend}\n" - "Auto index: {auto_index}\n" - "Update check: {update_check}\n" - "Rerank: {rerank}\n" + "API key set: {api} ({api_origin})\n" + "Default provider: {provider} ({provider_origin})\n" + "Default model: {model} ({model_origin})\n" + "Embedding dimensions: {embedding_dimensions} " + "({embedding_dimensions_origin})\n" + "Default batch size: {batch} ({batch_origin})\n" + "Embedding concurrency: {concurrency} ({concurrency_origin})\n" + "Extract concurrency: {extract_concurrency} " + "({extract_concurrency_origin})\n" + "Extract backend: {extract_backend} ({extract_backend_origin})\n" + "Auto index: {auto_index} ({auto_index_origin})\n" + "Update check: {update_check} ({update_check_origin})\n" + "Rerank: {rerank} ({rerank_origin})\n" "{flashrank_line}" "{remote_rerank_line}" - "Local CUDA: {local_cuda}\n" - "Custom base URL: {base_url}" + "Local CUDA: {local_cuda} ({local_cuda_origin})\n" + "Custom base URL: {base_url} ({base_url_origin})" ) - INFO_FLASHRANK_MODEL_SUMMARY = "FlashRank model: {value}" - INFO_REMOTE_RERANK_SUMMARY = "Remote rerank: {value}" + INFO_FLASHRANK_MODEL_SUMMARY = "FlashRank model: {value} ({origin})" + INFO_REMOTE_RERANK_SUMMARY = "Remote rerank: {value} ({origin})" INFO_SEARCH_RUNNING = "Searching cached index under {path}..." INFO_SEARCH_RUNNING_NO_CACHE = "Searching in-memory index under {path}..." INFO_DOCTOR_CHECKING = "Checking if `vexor` is on PATH..." @@ -445,8 +456,13 @@ class Messages: DOCTOR_CMD_MISSING = "`vexor` not found on PATH" DOCTOR_CMD_MISSING_DETAIL = "Install with pip or add the script directory to PATH." DOCTOR_CONFIG_EXISTS = "Config file exists at {path}" + DOCTOR_PROJECT_CONFIG_EXISTS = "Project config file exists at {path}" DOCTOR_CONFIG_DEFAULT = "No config file (using defaults)" + DOCTOR_CONFIG_NO_GLOBAL = "No global config file" DOCTOR_CONFIG_INVALID = "Config file is invalid JSON at {path}" + DOCTOR_CONFIG_PROJECT = "Project config: {project}" + DOCTOR_CONFIG_PROJECT_OVERRIDES = "Project overrides: {fields}" + DOCTOR_CONFIG_ENV_OVERRIDES = "Environment overrides: {fields}" DOCTOR_CACHE_CREATED = "Created cache directory at {path}" DOCTOR_CACHE_WRITABLE = "Cache directory writable at {path}" DOCTOR_CACHE_NOT_WRITABLE = "Cache directory not writable at {path}"