From 99c484109cb9068e4abc2cec99a86da754edfc85 Mon Sep 17 00:00:00 2001 From: John Morrissey Date: Sun, 7 Jun 2026 19:26:16 +1000 Subject: [PATCH 001/186] chore: untrack local filigree/loomweave workspace files Stop tracking per-clone tooling state (.filigree.conf, loomweave-workflow skill fingerprints/SKILL.md) and add ignore rules (incl. .weft/). Files remain on disk; only removed from the index. Co-Authored-By: Claude Opus 4.8 --- .../skills/loomweave-workflow/.fingerprint | 1 - .agents/skills/loomweave-workflow/SKILL.md | 211 ------------------ .../skills/loomweave-workflow/.fingerprint | 1 - .claude/skills/loomweave-workflow/SKILL.md | 211 ------------------ .filigree.conf | 6 - .gitignore | 8 + 6 files changed, 8 insertions(+), 430 deletions(-) delete mode 100644 .agents/skills/loomweave-workflow/.fingerprint delete mode 100644 .agents/skills/loomweave-workflow/SKILL.md delete mode 100644 .claude/skills/loomweave-workflow/.fingerprint delete mode 100644 .claude/skills/loomweave-workflow/SKILL.md delete mode 100644 .filigree.conf diff --git a/.agents/skills/loomweave-workflow/.fingerprint b/.agents/skills/loomweave-workflow/.fingerprint deleted file mode 100644 index b8934d20..00000000 --- a/.agents/skills/loomweave-workflow/.fingerprint +++ /dev/null @@ -1 +0,0 @@ -8af48023ff74748434eec046b718fe586bce8784e51d474c9c58daf8f292326b \ No newline at end of file diff --git a/.agents/skills/loomweave-workflow/SKILL.md b/.agents/skills/loomweave-workflow/SKILL.md deleted file mode 100644 index fd7ab55c..00000000 --- a/.agents/skills/loomweave-workflow/SKILL.md +++ /dev/null @@ -1,211 +0,0 @@ ---- -name: loomweave-workflow -description: > - Use when orienting in an unfamiliar or large codebase and you want to avoid - re-reading or grepping the whole source tree: answering "what calls X", - "where is X defined", "what does X depend on", "what subsystem is X in", or - "find the function/class/module that does Y". Applies whenever a Loomweave - code-archaeology MCP server (loomweave serve / mcp__loomweave__* tools) is - available for the project. ---- - -# Loomweave Workflow - -## Overview - -Loomweave pre-extracts a codebase into a queryable map — entities (functions, -classes, modules, files), the call/reference/import edges between them, and -subsystem clusters — and serves it over MCP. **Ask Loomweave instead of -re-exploring the tree.** One `find_entity` + one `callers_of` answers "what -calls this?" without reading a single file. - -## When to use - -- You're dropped into a codebase and need to locate a symbol or trace its callers/callees. -- You'd otherwise `grep`/read many files to answer a structural question. -- You need a function's neighborhood, execution paths, or which subsystem it belongs to. - -**Not for:** editing code, reading exact implementation bodies (use `summary` or -read the file once you have its path), or codebases with no `.loomweave/` index. - -## Entity IDs — the model - -Every entity has an ID: `{plugin}:{kind}:{qualified_name}` -(e.g. `python:function:pkg.mod.func`, `python:class:pkg.mod.Cls`, -`python:module:pkg.mod`). Subsystems are `core:subsystem:{hash}`. - -**You almost never type IDs.** Get one from `find_entity` / `entity_at`, then -**copy it verbatim** into the next tool. Don't hand-construct or guess IDs. - -### `id` vs `sei` — which one to bind on - -Every entity in a tool response now carries an `sei` field alongside its `id`. -They are not interchangeable: - -- **`id`** is the entity's *locator* — a mutable address. It changes when the - code is renamed or moved, and it's the right thing to feed into the next - Loomweave tool call (above). -- **`sei`** is the entity's *durable, stable identity*. It survives renames and - moves. **When you record a cross-tool binding** — e.g. attaching a Filigree - issue to a Loomweave entity — **bind on the `sei`, not the `id`.** A binding - keyed on the mutable `id` silently breaks the first time the entity moves. - -`sei` is `null` when the index predates SEI support or the entity has no binding -yet; `project_status` and `orientation_pack` report `sei.populated` so you can -tell which case you're in. - -## Tools - -| Tool | Use when | Args | -|------|----------|------| -| `find_entity` | locate an entity by name/text | `{"pattern": ""}` | -| `entity_at` | what's at a file:line | `{"file": "rel/path.py", "line": 42}` | -| `callers_of` | what calls this entity | `{"id": ""}` | -| `neighborhood` | one-hop callers+callees+container+contained+references+imports | `{"id": ""}` | -| `execution_paths_from` | bounded call paths out of an entity | `{"id": "", "max_depth": 5}` | -| `subsystem_members` | modules in a subsystem | `{"id": "core:subsystem:"}` | -| `subsystem_of` | the subsystem an entity belongs to (reverse of `subsystem_members`) | `{"id": ""}` | -| `summary` † | on-demand prose summary of one entity | `{"id": ""}` | -| `summary_preview_cost` | preview a `summary` call's cache status / cost before spending | `{"id": ""}` | -| `issues_for` | Filigree issues attached to an entity | `{"id": ""}` | -| `source_for_entity` | an entity's exact indexed source span + bounded context | `{"id": "", "context_lines": 10}` | -| `call_sites` | the source line(s) behind a calls/references edge | `{"id": "", "role": "caller"}` | -| `orientation_pack` | one deterministic orientation packet for an entity or file:line (entity + context + neighbors + paths + issues + freshness) | `{"file": "rel/path.py", "line": 42}` | -| `index_diff` | index freshness / drift vs. the current working tree | `{}` | -| `analyze_start` † | launch a background re-index, return its `run_id` | `{}` | -| `analyze_status` | poll a started analyze (queued/running/terminal + progress) | `{"run_id": ""}` | -| `analyze_cancel` † | stop a running analyze (group-kills plugin + Pyright) | `{"run_id": ""}` | -| `project_status` | index freshness, counts, LLM + Filigree status | `{}` | - -† **Write-gated.** `summary` (`entity_summary_get`), `analyze_start`, -`analyze_cancel`, `propose_guidance`, and `promote_guidance` are registered only -when `serve.mcp.enable_write_tools: true` is set in `loomweave.yaml` (default -`false`). When the gate is off they do not appear in `tools/list` and a call -returns a tool-disabled error — run `loomweave config check` to see the active -policy. `summary` additionally requires the live LLM provider to be enabled -(`llm_policy.enabled: true` + `allow_live_provider: true`), or it serves cache -only. - -`callers_of` / `neighborhood` / `execution_paths_from` take a `confidence` -tier — one of `"resolved"` (default; only high-confidence edges), -`"ambiguous"`, or `"inferred"`. There is no `"all"` value. When you suspect an -edge is missing (e.g. dynamic dispatch), re-query at `"ambiguous"` and -`"inferred"` and union the results — a default `resolved` count can understate -the true caller set. - -These three tools also return a `scope_excludes` array listing static blind -spots the query did **not** search (e.g. `"attribute-receiver-calls"` like -`ctx.svc.run()`). A non-empty -`scope_excludes` means an empty/short result is **not** a guaranteed true -negative — re-query at `"inferred"` (which searches those categories and returns -`scope_excludes: []`) before concluding "nothing calls this." - -`execution_paths_from` returns a compact shape: `root`, a deduplicated `nodes` -table (id + short_name + location, each node once), and `paths` as arrays of -node-id strings ranked longest-first. Resolve a path id against `nodes`, not by -re-reading each path element. `truncated`/`truncation_reason` report `edge-cap` -(traversal stopped early) or `path-cap` (ranked output trimmed for size). - -## Catalogue tools — inspection · faceted search · shortcuts - -Beyond navigation, Loomweave serves a **stateless catalogue** of read tools. All -of them: take explicit ids/scopes (no cursor/session — there is no `goto`/`back` -state to manage); **paginate** (`limit`/`offset`, with a `page` block reporting -`total`/`returned`/`truncated` — no silent caps); carry `sei` on every entity -they return; and are **honest-empty** — where a signal isn't present they return -an empty result with a `signal` note (`available:false`, the reason), never a -fabricated answer. - -`scope?` (where accepted) takes **either** an entity id (→ that entity's -descendants) **or** a path glob (`"src/auth/**"`); omit it for the whole project. - -**Inspection (read):** - -| Tool | Use when | Args | -|------|----------|------| -| `guidance_for` | guidance sheets applicable to an entity, scope-ranked | `{"id": ""}` | -| `findings_for` | findings anchored to an entity (filter kind/severity/status) | `{"id": "", "filter": {"status": "open"}}` | -| `wardline_for` | the entity's Wardline metadata (verbatim, opaque) | `{"id": ""}` | - -**Faceted search:** - -| Tool | Use when | Args | -|------|----------|------| -| `find_by_tag` | entities carrying a categorisation tag | `{"tag": "", "scope": "src/**"}` | -| `find_by_kind` | entities of a kind (`function`/`class`/`module`/…) | `{"kind": "function"}` | -| `find_by_wardline` | entities by Wardline tier/group (best-effort) | `{"tier": "exact"}` | - -**Exploration-elimination shortcuts** (on-demand graph/index queries — no -analyze-time precompute): - -| Tool | Use when | -|------|----------| -| `find_circular_imports` | import cycles (SCCs over `imports` edges) | -| `find_coupling_hotspots` | entities ranked by fan-in + fan-out | -| `find_entry_points` / `find_http_routes` / `find_data_models` / `find_tests` | entities by categorisation tag | -| `find_deprecations` / `find_todos` | deprecated / TODO-tagged entities | -| `what_tests_this` | test-tagged callers of an entity | -| `high_churn` | entities ranked by git churn | -| `recently_changed` | entities changed since a timestamp | - -`find_circular_imports` and `find_coupling_hotspots` are edge-derived, so they -take a `confidence` tier (default `resolved`, a ceiling) and echo it. The -categorisation shortcuts read plugin-emitted tags. The Python plugin emits -conservative tags for common conventions (`entry-point`, `http-route`, `test`, -`data-model`, `cli-command`, `exported-api`), so root/tag shortcuts and -`find_dead_code` light up on freshly analyzed Python projects where those -signals are present. `find_deprecations` / `find_todos` still return -honest-empty unless a plugin emits those tags. Likewise `high_churn` and -`recently_changed` are honest-empty until churn/change signals are populated (use -`index_diff` for repo-level freshness). - -`search_semantic` is also in the catalogue. It is opt-in under -`semantic_search:`; when enabled, `loomweave analyze` populates the git-ignored -`.loomweave/embeddings.db` sidecar and the query path filters stale vectors by -content hash. - -> Not in this catalogue: `emit_observation` as a general-purpose write surface. - -**Guidance authoring has an operator boundary.** Operators can manage sheets via -`loomweave guidance create/edit/show/list/delete/promote` (plus `export`/`import` -for team sharing). Agents may call `propose_guidance` to create a Filigree -observation, but that proposal is inert until an operator promotes it through -`promote_guidance` or the CLI. Promoted sheets reach you through `guidance_for` -and are composed into `summary` prompts with a real guidance fingerprint. -(`propose_guidance` and `promote_guidance` are write-gated — see the † note above.) - -## Workflow: orient, then navigate - -1. **Anchor.** `find_entity` by name (or `entity_at` for a file:line) to get the - entity and its `id`. For a code location you're about to dig into, prefer - `orientation_pack` — it returns the entity, its context, one-hop neighbors, - execution paths, attached issues, and index freshness in one deterministic - call, instead of hand-composing those queries. -2. **Navigate.** Feed that `id` into `callers_of`, `neighborhood`, - `execution_paths_from`, or `summary`. Chain results' IDs to keep walking. - -## Gotchas (read before hunting for a subsystem) - -- **To find a package's subsystem, search the package NAME with `kind`.** - Subsystems are *named after* their dominant package (e.g. `mypkg`), so - `find_entity {"pattern":"subsystem"}` returns nothing. Search the package name - and pass `{"kind":"subsystem"}` to return only subsystem entities, then call - `subsystem_members`. (`find_entity` accepts an optional `kind` filter — - `"subsystem"`, `"function"`, `"class"`, `"module"`, …; omit it for no filter.) -- **To go from an entity to its subsystem, use `subsystem_of`.** - `neighborhood` does **not** return the entity's subsystem. Call - `subsystem_of {"id": ""}` — it accepts any entity (a function/class - resolves through its containing module) and returns the subsystem plus the - module it resolved through. `subsystem_members` is the forward direction. -- **`find_entity` is paginated** (~20/page, `next_cursor`); narrow the pattern - rather than paging if you can. - -## Launch - -`loomweave serve --path ` where `` contains `.loomweave/loomweave.db` -(built by `loomweave analyze `). In an MCP client the tools appear as -`mcp__loomweave__find_entity`, etc. - -Besides the tools, the server exposes a `loomweave://context` **resource** — live -entity/subsystem/finding counts and index freshness as JSON, a lightweight read -when you only want the numbers (`project_status` is the fuller tool-based view). diff --git a/.claude/skills/loomweave-workflow/.fingerprint b/.claude/skills/loomweave-workflow/.fingerprint deleted file mode 100644 index b8934d20..00000000 --- a/.claude/skills/loomweave-workflow/.fingerprint +++ /dev/null @@ -1 +0,0 @@ -8af48023ff74748434eec046b718fe586bce8784e51d474c9c58daf8f292326b \ No newline at end of file diff --git a/.claude/skills/loomweave-workflow/SKILL.md b/.claude/skills/loomweave-workflow/SKILL.md deleted file mode 100644 index fd7ab55c..00000000 --- a/.claude/skills/loomweave-workflow/SKILL.md +++ /dev/null @@ -1,211 +0,0 @@ ---- -name: loomweave-workflow -description: > - Use when orienting in an unfamiliar or large codebase and you want to avoid - re-reading or grepping the whole source tree: answering "what calls X", - "where is X defined", "what does X depend on", "what subsystem is X in", or - "find the function/class/module that does Y". Applies whenever a Loomweave - code-archaeology MCP server (loomweave serve / mcp__loomweave__* tools) is - available for the project. ---- - -# Loomweave Workflow - -## Overview - -Loomweave pre-extracts a codebase into a queryable map — entities (functions, -classes, modules, files), the call/reference/import edges between them, and -subsystem clusters — and serves it over MCP. **Ask Loomweave instead of -re-exploring the tree.** One `find_entity` + one `callers_of` answers "what -calls this?" without reading a single file. - -## When to use - -- You're dropped into a codebase and need to locate a symbol or trace its callers/callees. -- You'd otherwise `grep`/read many files to answer a structural question. -- You need a function's neighborhood, execution paths, or which subsystem it belongs to. - -**Not for:** editing code, reading exact implementation bodies (use `summary` or -read the file once you have its path), or codebases with no `.loomweave/` index. - -## Entity IDs — the model - -Every entity has an ID: `{plugin}:{kind}:{qualified_name}` -(e.g. `python:function:pkg.mod.func`, `python:class:pkg.mod.Cls`, -`python:module:pkg.mod`). Subsystems are `core:subsystem:{hash}`. - -**You almost never type IDs.** Get one from `find_entity` / `entity_at`, then -**copy it verbatim** into the next tool. Don't hand-construct or guess IDs. - -### `id` vs `sei` — which one to bind on - -Every entity in a tool response now carries an `sei` field alongside its `id`. -They are not interchangeable: - -- **`id`** is the entity's *locator* — a mutable address. It changes when the - code is renamed or moved, and it's the right thing to feed into the next - Loomweave tool call (above). -- **`sei`** is the entity's *durable, stable identity*. It survives renames and - moves. **When you record a cross-tool binding** — e.g. attaching a Filigree - issue to a Loomweave entity — **bind on the `sei`, not the `id`.** A binding - keyed on the mutable `id` silently breaks the first time the entity moves. - -`sei` is `null` when the index predates SEI support or the entity has no binding -yet; `project_status` and `orientation_pack` report `sei.populated` so you can -tell which case you're in. - -## Tools - -| Tool | Use when | Args | -|------|----------|------| -| `find_entity` | locate an entity by name/text | `{"pattern": ""}` | -| `entity_at` | what's at a file:line | `{"file": "rel/path.py", "line": 42}` | -| `callers_of` | what calls this entity | `{"id": ""}` | -| `neighborhood` | one-hop callers+callees+container+contained+references+imports | `{"id": ""}` | -| `execution_paths_from` | bounded call paths out of an entity | `{"id": "", "max_depth": 5}` | -| `subsystem_members` | modules in a subsystem | `{"id": "core:subsystem:"}` | -| `subsystem_of` | the subsystem an entity belongs to (reverse of `subsystem_members`) | `{"id": ""}` | -| `summary` † | on-demand prose summary of one entity | `{"id": ""}` | -| `summary_preview_cost` | preview a `summary` call's cache status / cost before spending | `{"id": ""}` | -| `issues_for` | Filigree issues attached to an entity | `{"id": ""}` | -| `source_for_entity` | an entity's exact indexed source span + bounded context | `{"id": "", "context_lines": 10}` | -| `call_sites` | the source line(s) behind a calls/references edge | `{"id": "", "role": "caller"}` | -| `orientation_pack` | one deterministic orientation packet for an entity or file:line (entity + context + neighbors + paths + issues + freshness) | `{"file": "rel/path.py", "line": 42}` | -| `index_diff` | index freshness / drift vs. the current working tree | `{}` | -| `analyze_start` † | launch a background re-index, return its `run_id` | `{}` | -| `analyze_status` | poll a started analyze (queued/running/terminal + progress) | `{"run_id": ""}` | -| `analyze_cancel` † | stop a running analyze (group-kills plugin + Pyright) | `{"run_id": ""}` | -| `project_status` | index freshness, counts, LLM + Filigree status | `{}` | - -† **Write-gated.** `summary` (`entity_summary_get`), `analyze_start`, -`analyze_cancel`, `propose_guidance`, and `promote_guidance` are registered only -when `serve.mcp.enable_write_tools: true` is set in `loomweave.yaml` (default -`false`). When the gate is off they do not appear in `tools/list` and a call -returns a tool-disabled error — run `loomweave config check` to see the active -policy. `summary` additionally requires the live LLM provider to be enabled -(`llm_policy.enabled: true` + `allow_live_provider: true`), or it serves cache -only. - -`callers_of` / `neighborhood` / `execution_paths_from` take a `confidence` -tier — one of `"resolved"` (default; only high-confidence edges), -`"ambiguous"`, or `"inferred"`. There is no `"all"` value. When you suspect an -edge is missing (e.g. dynamic dispatch), re-query at `"ambiguous"` and -`"inferred"` and union the results — a default `resolved` count can understate -the true caller set. - -These three tools also return a `scope_excludes` array listing static blind -spots the query did **not** search (e.g. `"attribute-receiver-calls"` like -`ctx.svc.run()`). A non-empty -`scope_excludes` means an empty/short result is **not** a guaranteed true -negative — re-query at `"inferred"` (which searches those categories and returns -`scope_excludes: []`) before concluding "nothing calls this." - -`execution_paths_from` returns a compact shape: `root`, a deduplicated `nodes` -table (id + short_name + location, each node once), and `paths` as arrays of -node-id strings ranked longest-first. Resolve a path id against `nodes`, not by -re-reading each path element. `truncated`/`truncation_reason` report `edge-cap` -(traversal stopped early) or `path-cap` (ranked output trimmed for size). - -## Catalogue tools — inspection · faceted search · shortcuts - -Beyond navigation, Loomweave serves a **stateless catalogue** of read tools. All -of them: take explicit ids/scopes (no cursor/session — there is no `goto`/`back` -state to manage); **paginate** (`limit`/`offset`, with a `page` block reporting -`total`/`returned`/`truncated` — no silent caps); carry `sei` on every entity -they return; and are **honest-empty** — where a signal isn't present they return -an empty result with a `signal` note (`available:false`, the reason), never a -fabricated answer. - -`scope?` (where accepted) takes **either** an entity id (→ that entity's -descendants) **or** a path glob (`"src/auth/**"`); omit it for the whole project. - -**Inspection (read):** - -| Tool | Use when | Args | -|------|----------|------| -| `guidance_for` | guidance sheets applicable to an entity, scope-ranked | `{"id": ""}` | -| `findings_for` | findings anchored to an entity (filter kind/severity/status) | `{"id": "", "filter": {"status": "open"}}` | -| `wardline_for` | the entity's Wardline metadata (verbatim, opaque) | `{"id": ""}` | - -**Faceted search:** - -| Tool | Use when | Args | -|------|----------|------| -| `find_by_tag` | entities carrying a categorisation tag | `{"tag": "", "scope": "src/**"}` | -| `find_by_kind` | entities of a kind (`function`/`class`/`module`/…) | `{"kind": "function"}` | -| `find_by_wardline` | entities by Wardline tier/group (best-effort) | `{"tier": "exact"}` | - -**Exploration-elimination shortcuts** (on-demand graph/index queries — no -analyze-time precompute): - -| Tool | Use when | -|------|----------| -| `find_circular_imports` | import cycles (SCCs over `imports` edges) | -| `find_coupling_hotspots` | entities ranked by fan-in + fan-out | -| `find_entry_points` / `find_http_routes` / `find_data_models` / `find_tests` | entities by categorisation tag | -| `find_deprecations` / `find_todos` | deprecated / TODO-tagged entities | -| `what_tests_this` | test-tagged callers of an entity | -| `high_churn` | entities ranked by git churn | -| `recently_changed` | entities changed since a timestamp | - -`find_circular_imports` and `find_coupling_hotspots` are edge-derived, so they -take a `confidence` tier (default `resolved`, a ceiling) and echo it. The -categorisation shortcuts read plugin-emitted tags. The Python plugin emits -conservative tags for common conventions (`entry-point`, `http-route`, `test`, -`data-model`, `cli-command`, `exported-api`), so root/tag shortcuts and -`find_dead_code` light up on freshly analyzed Python projects where those -signals are present. `find_deprecations` / `find_todos` still return -honest-empty unless a plugin emits those tags. Likewise `high_churn` and -`recently_changed` are honest-empty until churn/change signals are populated (use -`index_diff` for repo-level freshness). - -`search_semantic` is also in the catalogue. It is opt-in under -`semantic_search:`; when enabled, `loomweave analyze` populates the git-ignored -`.loomweave/embeddings.db` sidecar and the query path filters stale vectors by -content hash. - -> Not in this catalogue: `emit_observation` as a general-purpose write surface. - -**Guidance authoring has an operator boundary.** Operators can manage sheets via -`loomweave guidance create/edit/show/list/delete/promote` (plus `export`/`import` -for team sharing). Agents may call `propose_guidance` to create a Filigree -observation, but that proposal is inert until an operator promotes it through -`promote_guidance` or the CLI. Promoted sheets reach you through `guidance_for` -and are composed into `summary` prompts with a real guidance fingerprint. -(`propose_guidance` and `promote_guidance` are write-gated — see the † note above.) - -## Workflow: orient, then navigate - -1. **Anchor.** `find_entity` by name (or `entity_at` for a file:line) to get the - entity and its `id`. For a code location you're about to dig into, prefer - `orientation_pack` — it returns the entity, its context, one-hop neighbors, - execution paths, attached issues, and index freshness in one deterministic - call, instead of hand-composing those queries. -2. **Navigate.** Feed that `id` into `callers_of`, `neighborhood`, - `execution_paths_from`, or `summary`. Chain results' IDs to keep walking. - -## Gotchas (read before hunting for a subsystem) - -- **To find a package's subsystem, search the package NAME with `kind`.** - Subsystems are *named after* their dominant package (e.g. `mypkg`), so - `find_entity {"pattern":"subsystem"}` returns nothing. Search the package name - and pass `{"kind":"subsystem"}` to return only subsystem entities, then call - `subsystem_members`. (`find_entity` accepts an optional `kind` filter — - `"subsystem"`, `"function"`, `"class"`, `"module"`, …; omit it for no filter.) -- **To go from an entity to its subsystem, use `subsystem_of`.** - `neighborhood` does **not** return the entity's subsystem. Call - `subsystem_of {"id": ""}` — it accepts any entity (a function/class - resolves through its containing module) and returns the subsystem plus the - module it resolved through. `subsystem_members` is the forward direction. -- **`find_entity` is paginated** (~20/page, `next_cursor`); narrow the pattern - rather than paging if you can. - -## Launch - -`loomweave serve --path ` where `` contains `.loomweave/loomweave.db` -(built by `loomweave analyze `). In an MCP client the tools appear as -`mcp__loomweave__find_entity`, etc. - -Besides the tools, the server exposes a `loomweave://context` **resource** — live -entity/subsystem/finding counts and index freshness as JSON, a lightweight read -when you only want the numbers (`project_status` is the fuller tool-based view). diff --git a/.filigree.conf b/.filigree.conf deleted file mode 100644 index bfc98f3f..00000000 --- a/.filigree.conf +++ /dev/null @@ -1,6 +0,0 @@ -{ - "version": 1, - "project_name": "wardline", - "prefix": "wardline", - "db": ".filigree/filigree.db" -} diff --git a/.gitignore b/.gitignore index e6c7ee05..47e5c355 100644 --- a/.gitignore +++ b/.gitignore @@ -42,3 +42,11 @@ CLAUDE.md .coverage coverage.json loomweave.yaml + +# Filigree issue tracker +.weft/ +.filigree.conf +.agents/skills/loomweave-workflow/.fingerprint +.agents/skills/loomweave-workflow/SKILL.md +.claude/skills/loomweave-workflow/.fingerprint +.claude/skills/loomweave-workflow/SKILL.md From b831db90f1ade33ef282eae027a501b7911248ad Mon Sep 17 00:00:00 2001 From: John Morrissey Date: Sun, 7 Jun 2026 19:35:18 +1000 Subject: [PATCH 002/186] delete: remove the comprehensive read-only audit report for the Wardline codebase conducted on 2026-06-04 --- .gitignore | 3 + .pre-commit-config.yaml | 11 +- ...nly-audit-2026-06-04-subagent-synthesis.md | 335 ----------- wardline-readonly-audit-2026-06-04.md | 568 ------------------ 4 files changed, 11 insertions(+), 906 deletions(-) delete mode 100644 wardline-readonly-audit-2026-06-04-subagent-synthesis.md delete mode 100644 wardline-readonly-audit-2026-06-04.md diff --git a/.gitignore b/.gitignore index 47e5c355..1eaa00ef 100644 --- a/.gitignore +++ b/.gitignore @@ -50,3 +50,6 @@ loomweave.yaml .agents/skills/loomweave-workflow/SKILL.md .claude/skills/loomweave-workflow/.fingerprint .claude/skills/loomweave-workflow/SKILL.md +wardline.yaml +.agents/skills/wardline-gate/SKILL.md +.claude/skills/wardline-gate/SKILL.md diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml index cbc24475..c571a86c 100644 --- a/.pre-commit-config.yaml +++ b/.pre-commit-config.yaml @@ -5,7 +5,12 @@ repos: - id: ruff-check args: [--fix] - id: ruff-format - - - + - repo: local + hooks: + - id: wardline-scan + name: wardline scan + entry: wardline scan + language: system + types: [python] + pass_filenames: false diff --git a/wardline-readonly-audit-2026-06-04-subagent-synthesis.md b/wardline-readonly-audit-2026-06-04-subagent-synthesis.md deleted file mode 100644 index ee73a89d..00000000 --- a/wardline-readonly-audit-2026-06-04-subagent-synthesis.md +++ /dev/null @@ -1,335 +0,0 @@ -# Wardline Read-Only Audit Synthesis - -Date: 2026-06-04 -Repository: `/home/john/wardline` -Mode: strictly read-only code review and research. The only workspace write was this requested report artifact. - -## Scope And Method - -Seven specialized read-only subagents reviewed the codebase with `enable_write_tools=false` and `enable_mcp_tools=false` in their task contracts: Architecture Critic, Systems Thinker, Python Engineer, Quality Engineer, Security Architect, Static Tools Analyst, and MCP & CLI Specialist. I then verified and de-duplicated the strongest claims against the live tree using read-only inspection commands. - -No tests were run because even normal pytest/mypy runs can write caches or coverage state. Existing `AUDIT.md` and `wardline-readonly-audit-2026-06-04.md` were left untouched. - -## Executive Summary - -No Critical findings were confirmed. - -The highest-risk issues cluster around four themes: - -- Several public non-scan CLI/core flows do not enforce the source-root confinement that `scan` and MCP now enforce. -- Scanner soundness has false-negative gaps around modern Python constructs and call argument unpacking. -- Trust/evidence artifacts can diverge from the effective scan policy, especially baseline generation and attestation policy identity. -- Live CI oracles can pass as skipped, so external integration drift can be missed. - -## Critical - -No Critical findings confirmed. - -## High - -### H1. Non-scan trust/evidence entrypoints can scan outside the selected project root - -Locations: -- [src/wardline/core/discovery.py:17-48](/home/john/wardline/src/wardline/core/discovery.py:17) -- [src/wardline/cli/scan.py:99-153](/home/john/wardline/src/wardline/cli/scan.py:99) -- [src/wardline/cli/assure.py:35-40](/home/john/wardline/src/wardline/cli/assure.py:35) -- [src/wardline/core/assure.py:233-251](/home/john/wardline/src/wardline/core/assure.py:233) -- [src/wardline/cli/attest.py:90-119](/home/john/wardline/src/wardline/cli/attest.py:90) -- [src/wardline/core/attest.py:173-193](/home/john/wardline/src/wardline/core/attest.py:173), [src/wardline/core/attest.py:226-255](/home/john/wardline/src/wardline/core/attest.py:226) -- [src/wardline/cli/dossier.py:66-72](/home/john/wardline/src/wardline/cli/dossier.py:66) -- [src/wardline/loom_dossier.py:60-69](/home/john/wardline/src/wardline/loom_dossier.py:60) -- [src/wardline/core/dossier.py:628-650](/home/john/wardline/src/wardline/core/dossier.py:628) -- [src/wardline/cli/judge.py:116-129](/home/john/wardline/src/wardline/cli/judge.py:116) -- [src/wardline/core/judge_run.py:128-182](/home/john/wardline/src/wardline/core/judge_run.py:128) -- [src/wardline/core/baseline.py:77-131](/home/john/wardline/src/wardline/core/baseline.py:77) -- MCP contrast: [src/wardline/mcp/server.py:240-323](/home/john/wardline/src/wardline/mcp/server.py:240) - -Evidence: `discover()` only rejects escaping `source_roots` and symlinked Python files when `confine_to_root=True`. The canonical `scan` CLI passes `confine_to_root=not allow_source_root_escape`, and tests pin that default. MCP wrappers also pass `confine_to_root=True` for dossier, assure, attest, verify, and judge. The CLI/core paths for assure, attest, dossier, judge, and baseline use defaults or calls that leave confinement off. - -Impact: A poisoned in-repo `wardline.yaml` can point `source_roots` outside the selected project. That can make assurance posture, attestations, judge excerpts, dossiers, or baselines incorporate out-of-root files while the canonical scan/MCP surfaces would reject the same scope. The judge path is especially sensitive because excerpts may be sent to OpenRouter. - -Remediation: Make public builder defaults `confine_to_root=True`. Pass `confine_to_root=True` from all CLI entrypoints. Add explicit opt-out flags only where intentionally supported, mirroring `scan --allow-source-root-escape`. Add regression tests for escaping `source_roots` across `assure`, `attest --reproduce`, `dossier`, `judge`, and baseline create/update. - -### H2. Persistent taint summary cache can be poisoned into false-green scans - -Locations: -- [src/wardline/scanner/taint/summary_cache.py:217-270](/home/john/wardline/src/wardline/scanner/taint/summary_cache.py:217) -- [src/wardline/scanner/taint/project_resolver.py:113-143](/home/john/wardline/src/wardline/scanner/taint/project_resolver.py:113) - -Evidence: `SummaryCache.load()` accepts any syntactically valid `/.json` payload and deserializes `FunctionSummary.cache_key` from the body, but does not verify that every loaded summary's `cache_key` matches the filename key. The resolver then trusts `summary_cache.get(cache_key)` directly for clean modules. - -Impact: If a CI cache or project cache directory is attacker-controlled or stale in a malicious way, forged summaries can replace fresh analysis and suppress real findings. - -Remediation: Treat persistent cache files as untrusted. On load, require `summary.cache_key == path.stem` for every summary and reject mixed-FQN or mismatched records. Consider authenticated cache records or disabling persistent cache for security gates. Add tests where a valid JSON cache file has a mismatched internal key and must fall back to fresh summarization. - -### H3. Attestation `ruleset_hash` omits effective scan policy inputs - -Locations: -- [src/wardline/core/attest.py:100-114](/home/john/wardline/src/wardline/core/attest.py:100) -- [src/wardline/core/attest.py:188-223](/home/john/wardline/src/wardline/core/attest.py:188) -- [src/wardline/core/config.py:31-49](/home/john/wardline/src/wardline/core/config.py:31) -- [src/wardline/core/config.py:197-214](/home/john/wardline/src/wardline/core/config.py:197) - -Evidence: `ruleset_hash()` hashes only sorted `rules_enable`, sorted `rules_severity`, and Wardline version. It omits fields that materially change scan results, including `source_roots`, `exclude`, `untrusted_sources`, `sanitisers`, `provenance_clash`, custom packs, and pack grammar/config effects. - -Impact: Two attestations can share the same policy identity while scanning different files or using different trust semantics. Downstream governance consumers may treat non-equivalent evidence bundles as comparable. - -Remediation: Replace `ruleset_hash()` with a canonical effective-scan-policy hash. Include source scope, excludes, rules, severity, provenance policy, custom sources/sanitisers, trusted pack names and versions/hashes, and grammar-affecting pack data. Add tests proving each policy-affecting field changes the signed policy identity. - -### H4. Baseline generation bypasses the shared scan pipeline - -Locations: -- [src/wardline/core/run.py:78-152](/home/john/wardline/src/wardline/core/run.py:78) -- [src/wardline/core/baseline.py:77-108](/home/john/wardline/src/wardline/core/baseline.py:77) -- [src/wardline/cli/main.py:59-105](/home/john/wardline/src/wardline/cli/main.py:59) -- [src/wardline/mcp/server.py:348-367](/home/john/wardline/src/wardline/mcp/server.py:348) - -Evidence: `run_scan()` constructs the configured grammar, summary cache, trust-pack behavior, strict defaults, and analyzer. `collect_and_write_baseline()` loads config with default trust flags, calls `discover()` directly, and constructs `WardlineAnalyzer()` directly. - -Impact: A baseline can differ from the scan/gate population. Custom grammar findings can be omitted or baseline generation can fail/differ where scan succeeds with explicit trust options. That weakens baseline suppression as an auditable snapshot of the actual gate. - -Remediation: Generate baselines from `run_scan()` or a shared `ScanOptions` pipeline. Thread `trust_local_packs`, `trusted_packs`, `strict_defaults`, cache options, and confinement through baseline CLI/MCP APIs. Add a regression test where a trusted pack emits a custom finding and baseline creation captures the same finding as `scan`. - -### H5. Multiple `**kwargs` unpackings overwrite earlier taints - -Locations: -- [src/wardline/scanner/taint/variable_level.py:430-447](/home/john/wardline/src/wardline/scanner/taint/variable_level.py:430) -- [src/wardline/scanner/analyzer.py:380-388](/home/john/wardline/src/wardline/scanner/analyzer.py:380) -- [src/wardline/scanner/rules/_sink_helpers.py:172-179](/home/john/wardline/src/wardline/scanner/rules/_sink_helpers.py:172) - -Evidence: Every keyword unpack records `resolved_args[kw.arg] = t`. For `**kwargs`, `kw.arg is None`, so `callee(**raw_kwargs, **clean_kwargs)` records only the last unpack. Interprocedural binding and sink helpers consume the single `None` value. - -Impact: Raw keyword flows can disappear from PY-WL-105, sink rules, and callee parameter propagation. This is a scanner false negative. - -Remediation: Store each `**` unpack separately or combine duplicate `None` entries with `combine()`. Update `_bind_call_site_arguments_to_parameters()` and `worst_arg_taint()` to aggregate all unpack taints. Add regression tests where raw unpack appears before a clean unpack. - -### H6. Starred unpack targets can lose raw taint - -Locations: -- [src/wardline/scanner/taint/variable_level.py:228-237](/home/john/wardline/src/wardline/scanner/taint/variable_level.py:228) -- [src/wardline/scanner/taint/variable_level.py:724-744](/home/john/wardline/src/wardline/scanner/taint/variable_level.py:724) -- [tests/unit/scanner/taint/test_variable_level.py:1089-1097](/home/john/wardline/tests/unit/scanner/taint/test_variable_level.py:1089) - -Evidence: Element-wise unpack handles `ast.Name` and nested tuple/list targets, but skips `ast.Starred`. Later reads of the starred target fall back to function-level taint if no binding exists. The existing test documents that the middle starred target is skipped and does not assert `b`. - -Impact: `(a, *rest, c) = (clean, raw, clean); return rest` inside a trusted producer can suppress PY-WL-101 and downstream sink findings. - -Remediation: Bind starred targets to the captured RHS slice when statically available, or conservatively bind to the whole RHS taint. Add PY-WL-101 and variable-level regression tests for starred unpack targets. - -### H7. `AsyncFor`, `TryStar`, and `except*` handlers are skipped by taint/rule traversal - -Locations: -- [src/wardline/scanner/taint/variable_level.py:602-619](/home/john/wardline/src/wardline/scanner/taint/variable_level.py:602) -- [src/wardline/scanner/taint/variable_level.py:627-629](/home/john/wardline/src/wardline/scanner/taint/variable_level.py:627) -- [src/wardline/scanner/taint/variable_level.py:1226-1251](/home/john/wardline/src/wardline/scanner/taint/variable_level.py:1226) -- [src/wardline/scanner/rules/_ast_helpers.py:32-37](/home/john/wardline/src/wardline/scanner/rules/_ast_helpers.py:32) -- [src/wardline/scanner/rules/broad_exception.py:49-56](/home/john/wardline/src/wardline/scanner/rules/broad_exception.py:49) -- [src/wardline/scanner/rules/silent_exception.py:49-56](/home/john/wardline/src/wardline/scanner/rules/silent_exception.py:49) - -Evidence: L2 statement dispatch handles `For`, `Try`, `With`, `AsyncWith`, and `Match`, but not `AsyncFor` or `TryStar`; unhandled statements only get walrus scanning. `own_except_handlers()` only yields handlers from `ast.Try`, so PY-WL-103/PY-WL-104 miss `except*`. - -Impact: Raw assignments inside `async for` or `except*` paths can be missed, and trusted-tier `except* Exception: pass` can evade broad/silent exception rules. - -Remediation: Route `ast.AsyncFor` through loop handling and `ast.TryStar` through try handling. Update `_ast_helpers.own_except_handlers()` to include `ast.TryStar`. Add fixtures for `async for` taint propagation and `except*` PY-WL-103/PY-WL-104 findings. - -### H8. Comprehension walrus writeback ignores existing outer variables - -Location: -- [src/wardline/scanner/taint/variable_level.py:361-404](/home/john/wardline/src/wardline/scanner/taint/variable_level.py:361) - -Evidence: The PEP 572 writeback loop only writes `var_taints[name] = taint` when `name not in var_taints`. That handles new walrus targets but skips rebinding an existing outer variable. - -Impact: If `x` starts trusted and a comprehension executes `(x := read_raw(p))`, `x` can remain trusted in the outer taint map. Later trusted returns or sinks using `x` can be missed. - -Remediation: For names proven by `_name_bound_by_walrus()`, write back the local taint whether the name is new or existing. Add tests for existing clean variables overwritten inside list/set/dict/gen comprehensions. - -### H9. Live oracle CI can pass without running the live oracle - -Locations: -- [.github/workflows/ci.yml:84-135](/home/john/wardline/.github/workflows/ci.yml:84) -- [tests/e2e/test_judge_live.py:12-38](/home/john/wardline/tests/e2e/test_judge_live.py:12) -- [tests/e2e/test_legis_live.py:55-65](/home/john/wardline/tests/e2e/test_legis_live.py:55) -- [tests/e2e/test_filigree_promote_live.py:44-47](/home/john/wardline/tests/e2e/test_filigree_promote_live.py:44) -- [tests/unit/test_ci_live_oracles.py:6-14](/home/john/wardline/tests/unit/test_ci_live_oracles.py:6) - -Evidence: Scheduled/manual jobs run marker-selected live tests, but tests skip when secrets, services, or routes are absent. The workflow summary explicitly says missing local services or secrets are reported as skipped tests. The default workflow guard only asserts some live markers and summary text, not a required no-skip mode. - -Impact: Weekly/manual CI can be green when OpenRouter, Clarion, Legis, or Filigree coverage did not actually execute, so integration drift is not caught. - -Remediation: Add a required live-oracle mode such as `WARDLINE_LIVE_ORACLE_REQUIRED=1` that turns missing secrets/services/capabilities into failures. Add workflow preflights or pytest no-skip enforcement. Extend `test_ci_live_oracles.py` to assert the network judge job, required secret env, live-oracle matrix, and required-mode behavior. - -## Medium - -### M1. MCP request validation accepts invalid IDs and maps malformed params to internal errors - -Locations: -- [src/wardline/mcp/protocol.py:55-102](/home/john/wardline/src/wardline/mcp/protocol.py:55) -- [src/wardline/mcp/server.py:834-853](/home/john/wardline/src/wardline/mcp/server.py:834) -- [tests/unit/mcp/test_protocol.py:96-105](/home/john/wardline/tests/unit/mcp/test_protocol.py:96) - -Evidence: The server treats presence of the `id` key as a request, so `id: null` is accepted and tested as a valid request. It handles `initialize` before the notification gate, so an `initialize` notification can produce a response. It also assigns `params = message.get("params") or {}` without validating that `params` is an object; `_tools_call()` then assumes `.get()`, so array/string params can become `-32603` internal errors. - -Protocol reference: the current MCP spec says requests must include a string or integer ID, IDs must not be null, and notifications must not include IDs or receive responses. See [MCP messages](https://modelcontextprotocol.io/specification/2025-06-18/basic/index). MCP tool malformed-request errors should be protocol errors rather than tool execution errors; see [MCP tools error handling](https://modelcontextprotocol.io/specification/draft/server/tools). - -Impact: Strict clients can reject the server, and malformed tool envelopes produce opaque internal errors instead of stable `-32602` invalid-params responses. - -Remediation: Validate request IDs before method dispatch. Reject `id is None` and non-string/non-integer IDs with `-32600`. Treat messages without `id`, including `initialize`, as notifications with no response or reject them via documented policy. Validate `params`, `name`, and `arguments` shape before handler calls and return `McpError(..., code=-32602)` for envelope faults. Replace the `id:null` test with conformance rejection tests. - -### M2. Filigree dossier URL scheme and network body sizes are not consistently bounded - -Locations: -- [src/wardline/core/config.py:237-249](/home/john/wardline/src/wardline/core/config.py:237) -- [src/wardline/filigree/dossier_client.py:41-56](/home/john/wardline/src/wardline/filigree/dossier_client.py:41) -- [src/wardline/core/judge.py:276-286](/home/john/wardline/src/wardline/core/judge.py:276) -- [src/wardline/clarion/client.py:48-59](/home/john/wardline/src/wardline/clarion/client.py:48) -- [src/wardline/core/filigree_emit.py:107-122](/home/john/wardline/src/wardline/core/filigree_emit.py:107) - -Evidence: `_is_safe_url()` checks localhost hostnames but not scheme. The Filigree dossier client does not enforce `http`/`https` before `urllib.request.urlopen()`. Several network transports read response and error bodies with unbounded `resp.read()` or `exc.read()`. - -Impact: Config URLs like `file://localhost/...` can pass the localhost check in some paths, and compromised or misconfigured endpoints can return oversized bodies that exhaust memory or produce excessive exception text. - -Remediation: Require `http`/`https` in `_is_safe_url()` and in `FiligreeWorkProvider` transport. Add a shared bounded-read helper for normal and error bodies. Truncate logged/raised response text. Add tests for `file://`, `ftp://`, schemeless URLs, and oversized response bodies. - -### M3. Autofix can report success when the write failed - -Location: -- [src/wardline/core/autofix.py:152-220](/home/john/wardline/src/wardline/core/autofix.py:152) - -Evidence: `applied[rel_path].append(...)` happens before the file write, and the write is wrapped in `contextlib.suppress(Exception)`. - -Impact: `wardline fix` or MCP autofix can tell automation a fix was applied even when the file was not changed, leaving the finding in place and making follow-up scan state confusing. - -Remediation: Write first and handle `OSError` explicitly. Only append/report applied fixes after a successful write. Return structured failures or raise `WardlineError` when a requested write fails. - -### M4. MCP Filigree emission softens protocol rejection into a nested warning - -Locations: -- [src/wardline/core/filigree_emit.py:141-148](/home/john/wardline/src/wardline/core/filigree_emit.py:141) -- [src/wardline/cli/scan.py:196-216](/home/john/wardline/src/wardline/cli/scan.py:196) -- [src/wardline/mcp/server.py:44-58](/home/john/wardline/src/wardline/mcp/server.py:44) -- [src/wardline/mcp/server.py:141-180](/home/john/wardline/src/wardline/mcp/server.py:141) - -Evidence: Core/CLI treat Filigree 3xx/4xx rejection as loud `FiligreeEmitError`. MCP `_emit_filigree()` catches `FiligreeEmitError` and returns `filigree.reachable=false` inside an otherwise successful scan payload. - -Impact: Agents can consume a successful MCP scan summary/gate while tracker emission or reconciliation was rejected, creating drift between local scan state and work-tracker state. - -Remediation: Preserve loud failure semantics for Filigree protocol/client errors in MCP, or expose a top-level `tracker_reconciled=false` / `emission_error` contract that consumers must handle. Align docs and tests with the chosen behavior. - -### M5. Interprocedural call binding over-taints impossible parameters - -Locations: -- [src/wardline/scanner/analyzer.py:334-388](/home/john/wardline/src/wardline/scanner/analyzer.py:334) -- [tests/unit/scanner/rules/test_wave2_engine_precision.py:70-99](/home/john/wardline/tests/unit/scanner/rules/test_wave2_engine_precision.py:70) - -Evidence: Starred taint is appended to every positional parameter, and `**kwargs` taint is appended to positional-only, already-filled, vararg, kw-only, and kwargs slots. The existing test explicitly expects all positional parameters to become contaminated from `*args`. - -Impact: PY-WL-105 and sink rules can report raw flow into parameters that Python call binding could not actually populate. This is a precision regression and can inflate false positives. - -Remediation: Model `inspect.Signature` binding more closely: explicit positional args first, star args only remaining positional/vararg slots, kwargs only unfilled keyword-capable slots. Keep conservative fallback only when static binding cannot determine a safe subset. - -### M6. `AnalysisContext` read-only contract is shallow - -Locations: -- [src/wardline/scanner/context.py:28-45](/home/john/wardline/src/wardline/scanner/context.py:28) -- [src/wardline/scanner/context.py:85-115](/home/john/wardline/src/wardline/scanner/context.py:85) - -Evidence: The docstring says inner mappings are wrapped read-only, but also notes `function_var_taints` inner dicts are left by convention. `__post_init__()` wraps several outer mappings only; nested maps remain mutable for some fields. - -Impact: A rule can mutate nested context state and affect later rules, making rule ordering a hidden input. - -Remediation: Deep-freeze nested mappings or provide isolated per-rule views. Add a test rule that attempts to mutate nested context and assert later rules are unaffected. - -### M7. SARIF serialization reaches into scanner internals - -Locations: -- [src/wardline/core/sarif.py:66-93](/home/john/wardline/src/wardline/core/sarif.py:66) -- [src/wardline/core/sarif.py:117-125](/home/john/wardline/src/wardline/core/sarif.py:117) - -Evidence: `core.sarif` imports private scanner/rule helpers and re-derives sink provenance from `AnalysisContext` internals. - -Impact: `core` is not a pure shared contract layer for SARIF; scanner refactors can silently break SARIF code-flow/provenance output. - -Remediation: Move code-flow/provenance projection to a public scanner explain API or stable DTO, and let SARIF serialize that public contract. - -## Low - -### L1. PY-WL-109 treats `Any` as a non-None promise - -Location: -- [src/wardline/scanner/rules/none_leak.py:66-124](/home/john/wardline/src/wardline/scanner/rules/none_leak.py:66) - -Evidence: `_annotation_allows_none()` recognizes `None`, `Optional`, `Union`, and `| None`, but not `Any` or `typing.Any`. Any other explicit annotation is treated as a non-None promise. - -Impact: Functions annotated `-> Any` can get false-positive None-leak findings. - -Remediation: Treat `Any` and `typing.Any` as not promising non-None. Add direct and string-annotation tests. - -### L2. Live judge cache oracle is tautological and CI guard misses the network job - -Locations: -- [tests/e2e/test_judge_live.py:14-38](/home/john/wardline/tests/e2e/test_judge_live.py:14) -- [tests/unit/test_ci_live_oracles.py:6-14](/home/john/wardline/tests/unit/test_ci_live_oracles.py:6) -- [.github/workflows/ci.yml:84-98](/home/john/wardline/.github/workflows/ci.yml:84) - -Evidence: The live judge test docstring says the second call hits cache, but the assertion allows `prompt_tokens_cached is None` or `>= 0`, including zero. The CI guard checks the live-oracle matrix markers but not the scheduled `network` job. - -Impact: Prompt-cache telemetry or network-job workflow drift can pass unnoticed. - -Remediation: Either require a positive cached-token signal where provider cache is contractual, or remove the cache-hit claim and assert only schema-critical fields. Extend the workflow guard to include the `network` job, marker, schedule condition, and API-key environment. - -### L3. Clarion live oracle setup is fragile - -Locations: -- [tests/e2e/test_clarion_live.py:39-69](/home/john/wardline/tests/e2e/test_clarion_live.py:39) -- [tests/e2e/test_clarion_live.py:142-164](/home/john/wardline/tests/e2e/test_clarion_live.py:142) - -Evidence: Route support is inferred by running `strings` over the binary and searching for a literal route. `_free_port()` binds port `0`, releases it, then the subprocess later tries to bind the chosen port. - -Impact: Valid Clarion builds can be skipped falsely, and port reuse races can make the live oracle flaky. - -Remediation: Prefer runtime capability probing after launch. Let explicit `WARDLINE_CLARION_BIN` proceed to runtime probing. Use a bind-to-port-0 server mode if Clarion supports it, or otherwise remove the open-port race. - -### L4. Protocol/package boundaries need clearer ownership - -Locations: -- [src/wardline/mcp/lsp.py:1-2](/home/john/wardline/src/wardline/mcp/lsp.py:1) -- [src/wardline/cli/lsp.py:1-10](/home/john/wardline/src/wardline/cli/lsp.py:1) -- [src/wardline/mcp/server.py:517-807](/home/john/wardline/src/wardline/mcp/server.py:517) -- [src/wardline/mcp/server.py:834-877](/home/john/wardline/src/wardline/mcp/server.py:834) - -Evidence: LSP lives under the MCP package, and the MCP registry mixes read-only, mutating, and network tools in one registration/dispatch path without central capability enforcement despite tool metadata such as `network=True`. - -Impact: Future read-only/no-network MCP modes have no central enforcement point, and package ownership is muddy. - -Remediation: Move LSP to `wardline.lsp` or `wardline.protocols.lsp` with a compatibility re-export. Add tool capability classes and enforce read/write/network policy at dispatch. - -### L5. Scanner orchestration has hidden global seams - -Locations: -- [src/wardline/scanner/analyzer.py:76-152](/home/john/wardline/src/wardline/scanner/analyzer.py:76) -- [src/wardline/scanner/analyzer.py:319-485](/home/john/wardline/src/wardline/scanner/analyzer.py:319) -- [src/wardline/scanner/analyzer.py:504-778](/home/john/wardline/src/wardline/scanner/analyzer.py:504) -- [src/wardline/scanner/taint/variable_level.py:78-92](/home/john/wardline/src/wardline/scanner/taint/variable_level.py:78) - -Evidence: Analyzer orchestration combines parsing, cache, L1/L2/L3 flow, diagnostics, and rule dispatch. It mutates private contextvars from `variable_level.py` to pass call-site state. - -Impact: Stage boundaries are hard to test independently, and private global/contextvar state increases refactor risk. - -Remediation: Split the scanner into explicit pipeline stages with typed inputs/outputs, and pass taint-analysis context explicitly instead of mutating private globals. - -## Suggested Remediation Order - -1. Fix H1 root confinement first because it is a trust-boundary and possible data-exfiltration issue, especially for `judge`. -2. Fix scanner false negatives next: H5, H6, H7, H8. Add focused failing tests before implementation. -3. Fix evidence identity drift: H3 and H4, then add parity tests between scan, baseline, attest, CLI, and MCP. -4. Harden cache trust (H2) and network/MCP protocol behavior (M1, M2, M4). -5. Tighten CI live-oracle required mode (H9) so future integrations fail loudly when not actually exercised. - -## Verification Notes - -- Verified current tree with read-only commands only. -- No source files were modified. -- No tests were run to avoid cache/coverage writes. -- External protocol references used only official Model Context Protocol documentation: - - [MCP current basic messages](https://modelcontextprotocol.io/specification/2025-06-18/basic/index) - - [MCP tools error handling](https://modelcontextprotocol.io/specification/draft/server/tools) diff --git a/wardline-readonly-audit-2026-06-04.md b/wardline-readonly-audit-2026-06-04.md deleted file mode 100644 index 9bf1e3e3..00000000 --- a/wardline-readonly-audit-2026-06-04.md +++ /dev/null @@ -1,568 +0,0 @@ -# Wardline Read-Only Codebase Audit - -Date: 2026-06-04 -Scope: `/home/john/wardline` -Mode: Comprehensive read-only audit of source, tests, CLI, MCP, static-analysis logic, integrations, and security boundaries. - -## Read-Only Boundary - -This audit was conducted as a read-only review of the codebase. No source files, tests, configs, or tracker state were modified as part of the audit. The only write performed was this requested markdown artifact. - -Seven specialized subagents were dispatched with explicit read-only instructions: - -| Agent | Focus | -| --- | --- | -| Architecture Critic | Package boundaries, cohesion, coupling, structural fragility under `src/` | -| Systems Thinker | Feedback loops, dependency chains, propagation flows, failure modes | -| Python Engineer | Python implementation, typing posture, AST parsing/manipulation, idioms | -| Quality Engineer | Tests, CI, coverage structure, maintainability, validation paths | -| Security Architect | Trust boundaries, external data handling, secure defaults | -| Static Tools Analyst | Static-analysis rules, taint propagation, trust lattice, SCC logic | -| MCP & CLI Specialist | CLI and MCP protocol/server behavior, path confinement, tool parity | - -Tooling note: the available subagent spawning interface did not expose literal `enable_write_tools=false` or `enable_mcp_tools=false` switches. The equivalent boundary was enforced in each subagent prompt: no edits, no `apply_patch`, no write commands, no MCP tools, no tracker changes, and no escaped double quotes in tool arguments. - -## Executive Summary - -No Critical issues were found. The audit identified 5 High, 15 Medium, and 5 Low findings. The highest-risk themes are: - -- Warm-cache scan behavior can diverge from cold scans for L2-backed rules. -- A default MCP waiver write path can bypass root confinement through symlinked `wardline.yaml`. -- Project-controlled config can influence autofix code generation and LLM judge suppression behavior. -- Filigree close-on-fixed feedback lacks a clean-file heartbeat, so fixed findings may stay open. -- MCP/CLI hardening has several retry, schema, notification, and confinement gaps. - -The codebase also shows several strong foundations: shared CLI/MCP orchestration through `core.run`, generally coherent scanner layering, strict mypy/coverage-oriented CI, safe YAML loading/schema validation, fail-soft external integrations, and explicit trust-lattice modeling in the analyzer. - -## Remediation Status - -Resolved in the current remediation pass: - -- H-01: warm-cache L2 bypass no longer skips flow-sensitive call-site state; the warm/cold cache parity test now uses a sink fixture. -- H-02: MCP `waiver_add` passes the project root into waiver writes, rejecting symlinked default config escapes. -- H-03: `autofix.boundary_exception` is validated as an identifier/dotted identifier, and MCP `fix` requires `apply: true` before modifying files. -- H-04: project judge config no longer controls model or write confidence floor unless `trust_judge_config` is explicitly set. -- H-05: Filigree scan-results payloads now carry `scanned_paths`, allowing clean scanned files to participate in close-on-fixed reconciliation. -- M-01: JSON-RPC notifications no longer invoke registered handlers except for initialization lifecycle notifications. -- M-02: MCP tool schemas are closed with `additionalProperties: false`, and unknown tool arguments return `isError`. -- M-03: retrying `baseline_create` and `waiver_add` is idempotent and returns existing state instead of failing. -- M-04: CLI/core scans reject escaping `source_roots` by default; CLI escape now requires `--allow-source-root-escape`. -- M-05: attestation verification now checks `signature.alg`, `signature.key_id`, and signature value. -- M-06: Filigree dossier reads normalize scan-results URLs to the Filigree API base before querying entity associations. -- M-07: Clarion-backed explanations select the exact stored finding by fingerprint/path/line and fall back to local analysis on mismatch. -- M-08: discovery skip rules apply relative to the configured source root, not absolute parent directories. -- M-09: callgraph receiver type tracking no longer collects assignments from nested scopes. -- M-10: sink discovery no longer descends into lambda bodies. -- M-11: `NoneLeak` fall-through analysis handles all-return `try`/`except` paths. -- M-12: shared dossier identity types live in neutral `core.identity`, with Clarion compatibility re-exports. -- M-13: MCP resource/prompt catalogs and shared tool plumbing were split out of `mcp/server.py`, with advertisement snapshot coverage. -- M-14: CI now exposes scheduled/manual live-oracle jobs for Clarion, Legis, and Filigree e2e markers. -- M-15: Filigree unsafe config URL rejection now has mirrored negative coverage. -- L-01: analyzer cache freshness checks use `SummaryCache.has_current()` instead of private `_entries`. -- L-02: `core.protocols` is wired into scan orchestration and the rule registry seam. -- L-03: pack tests use `monkeypatch.syspath_prepend` instead of direct `sys.path` mutation. -- L-04: LSP `Content-Length` framing has a maximum body size and drains oversized frames deterministically. -- L-05: CLI `scan --fix` preserves `strict_defaults` during the post-fix rescan. - -Verification after remediation: - -- `uv run pytest` — 2173 passed, 8 live/network tests deselected, 1 expected symlink-skip warning. -- `uv run mypy src tests` — success. -- `uv run ruff check src tests` — success. - -## Critical Findings - -None found. - -## High Findings - -### H-01: Warm-cache L2 bypass changes rule behavior - -Severity: High -Areas: Static analysis correctness, scanner cache, taint rules -Locations: - -- [src/wardline/scanner/analyzer.py:518](/home/john/wardline/src/wardline/scanner/analyzer.py:518)-525 -- [src/wardline/scanner/rules/_sink_helpers.py:174](/home/john/wardline/src/wardline/scanner/rules/_sink_helpers.py:174)-187 -- [src/wardline/scanner/rules/_sink_helpers.py:268](/home/john/wardline/src/wardline/scanner/rules/_sink_helpers.py:268)-270 -- [src/wardline/scanner/rules/untrusted_to_trusted_callee.py:117](/home/john/wardline/src/wardline/scanner/rules/untrusted_to_trusted_callee.py:117)-119 -- [tests/unit/cli/test_cli.py:191](/home/john/wardline/tests/unit/cli/test_cli.py:191)-218 - -Finding: cached modules with `bypass_l2` restore only selected summary fields, while L2-derived local variable and callsite taints are not rebuilt. Sink rules then fall back to `UNKNOWN_RAW`, and PY-WL-105 can lose caller/callee evidence. Cold and warm scans can therefore produce different diagnostics for the same code. The existing CLI warm-cache test covers a trivial non-sink fixture, so it does not prove the L2-backed sink/callee invariant. - -Impact: false positives and false negatives in the scanner after cache reuse. This is especially risky because cache reuse is a normal operational path, so CI or local dogfooding can disagree with a fresh scan. - -Remediation: - -1. Persist and reload the L2 artifacts that rules consume, including local variable taints and callsite taints. -2. If full L2 persistence is not desired, recompute L2 for cached modules before running rules. -3. Add a cold-vs-warm invariant test that scans the same fixture twice and asserts identical diagnostics. -4. Include fixtures for a safe literal sink, a raw sink, and PY-WL-105 caller/callee propagation. - -### H-02: MCP `waiver_add` can write through an out-of-root symlinked default config path - -Severity: High -Areas: MCP mutating tool, path confinement, waiver persistence -Locations: - -- [src/wardline/mcp/server.py:417](/home/john/wardline/src/wardline/mcp/server.py:417)-429 -- [src/wardline/core/waivers.py:88](/home/john/wardline/src/wardline/core/waivers.py:88)-117 - -Finding: the MCP `_waiver_add` fallback writes to `root / "wardline.yaml"` but calls `add_waiver` without passing `root=root`. `add_waiver` only applies `safe_project_file` when `root` is supplied. A symlinked default config file can therefore redirect writes outside the project root. - -Impact: a project can cause the MCP tool to append waiver data to an arbitrary symlink target reachable by the process. This breaks the MCP server's otherwise explicit path-confinement posture. - -Remediation: - -1. Pass `root=root` from `_waiver_add` to `add_waiver` for the default path. -2. Make `add_waiver` require a root for all writes, or require callers to pass a prevalidated path object. -3. Add symlink escape tests for `waiver_add`, default config reads, default config writes, `resources/read wardline://config`, `scan`, and `fix`. -4. Treat a symlinked config target outside the root as an MCP `isError` response with a clear confinement error. - -### H-03: Config-controlled autofix exception name can generate unsafe Python output - -Severity: High -Areas: Autofix, config trust boundary, MCP fix tool -Locations: - -- [src/wardline/core/autofix.py:80](/home/john/wardline/src/wardline/core/autofix.py:80) -- [src/wardline/core/autofix.py:175](/home/john/wardline/src/wardline/core/autofix.py:175) -- [src/wardline/core/config_schema.py:59](/home/john/wardline/src/wardline/core/config_schema.py:59) -- [src/wardline/core/config.py:36](/home/john/wardline/src/wardline/core/config.py:36)-38 -- [src/wardline/mcp/server.py:437](/home/john/wardline/src/wardline/mcp/server.py:437)-456 - -Finding: `autofix.boundary_exception` is accepted as an arbitrary string and later inserted into an AST node as a name before being unparsed. The MCP fix path can apply changes without a confirmation callback. - -Impact: untrusted project configuration can steer generated source text. Even if malformed names usually fail at AST construction or unparsing time, this is a codemod trust-boundary violation and creates brittle, surprising behavior. - -Remediation: - -1. Validate `autofix.boundary_exception` as either a single valid identifier or a dotted qualified name where every segment passes `str.isidentifier()`. -2. Prefer an allowlist for known safe boundary exception names if project policy permits it. -3. Return a config validation error before any fix planning when the value is invalid. -4. Make MCP fix dry-run by default, or require an explicit `apply: true` argument for file mutation. -5. Add negative tests for malformed names, dotted names, keywords, whitespace, and punctuation. - -### H-04: Project config can steer LLM judge model and suppression threshold - -Severity: High -Areas: LLM judge integration, suppression policy, untrusted project config -Locations: - -- [src/wardline/core/config_schema.py:37](/home/john/wardline/src/wardline/core/config_schema.py:37)-42 -- [src/wardline/core/config.py:324](/home/john/wardline/src/wardline/core/config.py:324)-347 -- [src/wardline/cli/judge.py:95](/home/john/wardline/src/wardline/cli/judge.py:95) -- [src/wardline/core/judge_run.py:81](/home/john/wardline/src/wardline/core/judge_run.py:81)-92 -- [src/wardline/core/judge_run.py:150](/home/john/wardline/src/wardline/core/judge_run.py:150) -- [src/wardline/core/judge_run.py:200](/home/john/wardline/src/wardline/core/judge_run.py:200) -- [src/wardline/core/judge.py:211](/home/john/wardline/src/wardline/core/judge.py:211)-237 -- [src/wardline/core/judge.py:322](/home/john/wardline/src/wardline/core/judge.py:322) - -Finding: project config can set judge model and false-positive floor values. In a hostile or low-trust checkout, config can direct analysis to an unintended model or lower suppression thresholds. - -Impact: a repository under scan can influence the external judge used to evaluate its own findings and can make suppression easier. This weakens judge-based assurance and creates an operator trust-boundary issue. - -Remediation: - -1. Treat model selection and suppression threshold as operator-controlled settings by default. -2. Require an explicit CLI flag such as `--trust-judge-config` before project config can influence these values. -3. Do not allow project config to lower the built-in false-positive floor unless an operator override is present. -4. Record effective judge model, threshold source, and override source in output metadata. -5. Add tests showing that project config is ignored without the trust flag and honored only with the flag. - -### H-05: Filigree close-on-fixed feedback lacks a clean-file heartbeat - -Severity: High -Areas: Filigree integration, feedback loops, finding lifecycle -Locations: - -- [src/wardline/core/filigree_emit.py:58](/home/john/wardline/src/wardline/core/filigree_emit.py:58)-69 -- [src/wardline/scanner/diagnostics.py:39](/home/john/wardline/src/wardline/scanner/diagnostics.py:39)-48 - -Finding: close-on-fixed appears to rely on emitted findings and a `mark_unseen` behavior, but the emitted payload does not include a complete scanned-file inventory or per-file clean heartbeat. The code comment indicates absent fingerprints are swept only for files still represented in the batch. If a file has no current findings, it may not be represented as a cleaned source file. - -Impact: fixing the only finding in a file may fail to close the linked Filigree issue. The external tracker can drift toward stale open issues and reduce trust in scan automation. - -Remediation: - -1. Extend the Filigree payload with explicit scanned source scopes for every analyzed file. -2. Alternatively, emit a per-file clean/reconciliation fact when a file was scanned and produced no findings. -3. Ensure `mark_unseen` receives enough file-level scope to close findings removed from otherwise-clean files. -4. Add a live or contract e2e test: create a finding, bind or emit it to Filigree, fix it, rescan, and assert the file_finding issue closes. - -## Medium Findings - -### M-01: JSON-RPC notifications can silently execute side-effecting MCP handlers - -Severity: Medium -Locations: - -- [src/wardline/mcp/protocol.py:58](/home/john/wardline/src/wardline/mcp/protocol.py:58)-99 - -Finding: requests without an `id` are treated as notifications and no response is sent, but the handler is still invoked. A no-id `tools/call` notification can therefore run side-effecting handlers silently. - -Remediation: - -1. Whitelist legitimate notifications only, such as initialization lifecycle notifications if needed. -2. Reject or ignore no-id calls to `tools/*`, `resources/*`, and `prompts/*`. -3. Add a test proving no-id `tools/call` cannot mutate baseline, waiver, or fix state. - -### M-02: MCP tool schemas allow unknown arguments on mutating tools - -Severity: Medium -Locations: - -- [src/wardline/mcp/server.py:824](/home/john/wardline/src/wardline/mcp/server.py:824)-835 -- [src/wardline/mcp/server.py:922](/home/john/wardline/src/wardline/mcp/server.py:922)-956 -- [src/wardline/mcp/server.py:437](/home/john/wardline/src/wardline/mcp/server.py:437)-456 - -Finding: tool schemas do not consistently set `additionalProperties: false`. Unknown arguments, typoed safety flags, and accidentally ignored user intent can pass through mutating tool calls. - -Remediation: - -1. Add `additionalProperties: false` to all MCP input schemas. -2. Reject unknown arguments in server-side validation before dispatch. -3. Add negative schema tests for typoed mutating-tool arguments. -4. Require explicit `apply: true` or equivalent for mutating tools that change files. - -### M-03: Mutating MCP tools are not retry-safe - -Severity: Medium -Locations: - -- [src/wardline/mcp/server.py:401](/home/john/wardline/src/wardline/mcp/server.py:401)-429 -- [src/wardline/core/baseline.py:100](/home/john/wardline/src/wardline/core/baseline.py:100)-109 -- [src/wardline/core/waivers.py:111](/home/john/wardline/src/wardline/core/waivers.py:111)-117 - -Finding: repeated identical mutating calls can append duplicates or overwrite state without a stable idempotency story. - -Remediation: - -1. Make repeated identical calls return structured success such as `already_exists: true`. -2. Deduplicate waiver entries by stable fingerprint/rule/path tuple. -3. Add expected-version or idempotency-key support for baseline writes. -4. Add retry tests that issue the same MCP mutating call twice. - -### M-04: CLI/core scans are unconfined by default - -Severity: Medium -Locations: - -- [src/wardline/core/run.py:76](/home/john/wardline/src/wardline/core/run.py:76)-93 -- [src/wardline/core/discovery.py:17](/home/john/wardline/src/wardline/core/discovery.py:17)-27 -- [src/wardline/cli/scan.py:137](/home/john/wardline/src/wardline/cli/scan.py:137) -- [tests/unit/mcp/test_server_security.py:74](/home/john/wardline/tests/unit/mcp/test_server_security.py:74)-99 - -Finding: MCP has explicit path-escape tests, but CLI/core scanning defaults to unconfined source roots. That creates different trust behavior across entry points. - -Remediation: - -1. Default `confine_to_root=True` in CLI/core scan paths. -2. Add an explicit `--allow-source-root-escape` CLI option for the less-safe behavior. -3. Include the confinement mode in scan metadata. -4. Add CLI tests matching the MCP path-escape coverage. - -### M-05: Attestation signature metadata is mutable - -Severity: Medium -Locations: - -- [src/wardline/core/attest.py:126](/home/john/wardline/src/wardline/core/attest.py:126)-135 -- [src/wardline/core/attest.py:300](/home/john/wardline/src/wardline/core/attest.py:300)-302 - -Finding: HMAC verification covers the payload/value, but the outer signature metadata such as algorithm and key id is not itself bound or strictly revalidated. - -Remediation: - -1. Require `signature.alg == "HMAC-SHA256"` during verification. -2. Require the stored `key_id` to match the verifying key's derived id. -3. Prefer including signature metadata in the signed canonical envelope. -4. Add tamper tests for algorithm, key id, schema tag, and payload. - -### M-06: Dossier Filigree read path treats a scan-results URL as an API origin - -Severity: Medium -Locations: - -- [src/wardline/filigree/dossier_client.py:78](/home/john/wardline/src/wardline/filigree/dossier_client.py:78)-86 -- [src/wardline/loom_dossier.py:108](/home/john/wardline/src/wardline/loom_dossier.py:108)-109 -- [tests/unit/core/test_config.py:185](/home/john/wardline/tests/unit/core/test_config.py:185)-193 - -Finding: configuration examples/tests treat `filigree.url` as a Loom scan-results endpoint, while dossier association reads append `/api/entity-associations` to that value as though it were an API origin. - -Remediation: - -1. Split configuration into distinct values such as `filigree.scan_results_url` and `filigree.api_base_url`. -2. Or normalize the configured scan-results URL back to an origin before appending association routes. -3. Add tests for the documented URL shape and the dossier association lookup URL. - -### M-07: Clarion-backed explain can pair a requested fingerprint with the wrong stored finding - -Severity: Medium -Locations: - -- [src/wardline/core/explain.py:152](/home/john/wardline/src/wardline/core/explain.py:152)-177 -- [src/wardline/core/explain.py:273](/home/john/wardline/src/wardline/core/explain.py:273)-280 -- [src/wardline/clarion/facts.py:59](/home/john/wardline/src/wardline/clarion/facts.py:59)-91 - -Finding: the Clarion-backed explanation path can collapse an entity query to the first finding in a stored blob instead of selecting the finding matching the requested fingerprint/path/line. - -Remediation: - -1. Pass requested fingerprint, path, line, and rule id into the blob extraction function. -2. Select the exact matching finding when present. -3. Fall back to local explanation when no matching entry exists. -4. Add tests with multiple findings bound to one entity. - -### M-08: Absolute-path skip filter can false-green scans based on checkout location - -Severity: Medium -Locations: - -- [src/wardline/core/discovery.py:17](/home/john/wardline/src/wardline/core/discovery.py:17)-34 -- [src/wardline/core/run.py:209](/home/john/wardline/src/wardline/core/run.py:209)-223 - -Finding: skip logic checks path components directly. If an absolute parent directory contains a skipped component such as `.venv`, `venv`, `.git`, or `.mypy_cache`, an otherwise valid checkout can be skipped entirely. - -Remediation: - -1. Apply skip logic to components relative to the project or source root. -2. Emit a structured diagnostic or run metadata flag when all candidate files are skipped. -3. Add tests for checkouts under paths containing skipped directory names. - -### M-09: Callgraph receiver type inference is polluted by nested-scope assignments - -Severity: Medium -Locations: - -- [src/wardline/scanner/taint/callgraph.py:91](/home/john/wardline/src/wardline/scanner/taint/callgraph.py:91)-114 -- [src/wardline/scanner/taint/callgraph.py:145](/home/john/wardline/src/wardline/scanner/taint/callgraph.py:145)-154 - -Finding: `ast.walk` traverses nested functions, lambdas, and classes while collecting local assignment type information. Inner-scope assignments can pollute outer-scope receiver inference. - -Remediation: - -1. Replace broad `ast.walk` collection with an own-scope visitor. -2. Stop recursion at nested `FunctionDef`, `AsyncFunctionDef`, `ClassDef`, and `Lambda` boundaries. -3. Add tests where nested-scope assignments use the same variable name as the outer scope. - -### M-10: Sink discovery enters lambda bodies without matching taint snapshots - -Severity: Medium -Locations: - -- [src/wardline/scanner/rules/_sink_helpers.py:81](/home/john/wardline/src/wardline/scanner/rules/_sink_helpers.py:81)-97 -- [src/wardline/scanner/rules/_sink_helpers.py:191](/home/john/wardline/src/wardline/scanner/rules/_sink_helpers.py:191)-205 -- [src/wardline/scanner/ast_primitives.py:122](/home/john/wardline/src/wardline/scanner/ast_primitives.py:122)-124 - -Finding: sink discovery can traverse lambda bodies, but the taint state associated with those expressions does not have a matching scope snapshot. This can misattribute trust state in nested expression scopes. - -Remediation: - -1. Apply the same scope-boundary policy used by callgraph/L2 analysis. -2. Or model lambdas as first-class scopes with their own taint snapshots. -3. Add lambda sink tests for trusted and untrusted captures. - -### M-11: PY-WL-109 NoneLeak fall-through analysis is too shallow - -Severity: Medium -Locations: - -- [src/wardline/scanner/rules/none_leak.py:138](/home/john/wardline/src/wardline/scanner/rules/none_leak.py:138)-154 -- [src/wardline/scanner/rules/none_leak.py:183](/home/john/wardline/src/wardline/scanner/rules/none_leak.py:183)-190 -- [tests/unit/scanner/rules/test_none_leak.py:106](/home/john/wardline/tests/unit/scanner/rules/test_none_leak.py:106)-229 - -Finding: fall-through detection handles simple terminal statements but does not robustly model `try`, `match`, and common loop terminal forms. This can over-report missing returns. - -Remediation: - -1. Expand terminal-path analysis for `try`/`except`/`finally`, `match`, and loops whose bodies unconditionally return/raise. -2. When full certainty is not available, prefer a conservative unknown path instead of a confident diagnostic. -3. Add positive and negative tests for `try`, `match`, loop, and nested branch cases. - -### M-12: Dossier identity model depends on Clarion-owned types - -Severity: Medium -Locations: - -- [src/wardline/core/dossier.py:39](/home/john/wardline/src/wardline/core/dossier.py:39) -- [src/wardline/core/dossier.py:428](/home/john/wardline/src/wardline/core/dossier.py:428)-442 -- [src/wardline/filigree/dossier_client.py:27](/home/john/wardline/src/wardline/filigree/dossier_client.py:27) -- [src/wardline/loom_dossier.py:5](/home/john/wardline/src/wardline/loom_dossier.py:5)-8 - -Finding: core dossier and Filigree dossier paths depend on identity/status types owned by the Clarion integration. This weakens package boundaries and makes a product-neutral dossier model depend on one provider. - -Remediation: - -1. Move shared identity types such as `IdentityStatus`, `ContentStatus`, `EntityBinding`, and `content_status` into a neutral module such as `core.identity`. -2. Re-export from Clarion only for compatibility. -3. Update core, Clarion, Filigree, and Loom import paths. -4. Add a small dependency-direction test or import-lint check if the project already uses such tooling. - -### M-13: MCP server is over-concentrated - -Severity: Medium -Locations: - -- [src/wardline/mcp/server.py:132](/home/john/wardline/src/wardline/mcp/server.py:132) -- [src/wardline/mcp/server.py:401](/home/john/wardline/src/wardline/mcp/server.py:401) -- [src/wardline/mcp/server.py:469](/home/john/wardline/src/wardline/mcp/server.py:469) -- [src/wardline/mcp/server.py:551](/home/john/wardline/src/wardline/mcp/server.py:551) -- [src/wardline/mcp/server.py:854](/home/john/wardline/src/wardline/mcp/server.py:854) -- [src/wardline/mcp/server.py:913](/home/john/wardline/src/wardline/mcp/server.py:913) - -Finding: MCP protocol, dependency construction, tool handlers, resource handlers, prompts, and schema advertisement are concentrated in one large server module. - -Impact: local changes to one tool can accidentally affect unrelated MCP surfaces, and security validation rules are harder to enforce consistently. - -Remediation: - -1. Split handlers into modules such as `mcp/tools/scan.py`, `mcp/tools/fix.py`, `mcp/resources.py`, `mcp/prompts.py`, and `mcp/schemas.py`. -2. Keep the top-level server as a dispatcher and registry assembler. -3. Centralize common argument validation, root confinement, and error mapping. -4. Preserve public tool names and add advertisement snapshot tests during the split. - -### M-14: Live Clarion/Legis/Filigree e2e oracles are not CI-gated - -Severity: Medium -Locations: - -- [pyproject.toml:105](/home/john/wardline/pyproject.toml:105)-112 -- [.github/workflows/ci.yml:47](/home/john/wardline/.github/workflows/ci.yml:47)-95 -- [tests/e2e/test_clarion_live.py:1](/home/john/wardline/tests/e2e/test_clarion_live.py:1)-12 - -Finding: important live integration tests are marked as opt-in and are not represented as scheduled or manually triggered CI jobs. - -Remediation: - -1. Add scheduled or workflow-dispatch jobs for `clarion_e2e`, `legis_e2e`, and `filigree_e2e`. -2. Gate those jobs on required service secrets or local service availability. -3. Publish skipped/live status clearly in CI summaries. -4. Keep normal PR CI dependency-free, but run live oracles often enough to catch drift. - -### M-15: Filigree unsafe-config URL guard lacks mirrored negative tests - -Severity: Medium -Locations: - -- [src/wardline/core/config.py:257](/home/john/wardline/src/wardline/core/config.py:257)-285 -- [tests/unit/core/test_config.py:217](/home/john/wardline/tests/unit/core/test_config.py:217)-233 - -Finding: Clarion unsafe URL configuration has negative test coverage, but Filigree's equivalent unsafe-config guard does not have mirrored tests. - -Remediation: - -1. Add Filigree tests that reject unsafe scheme/host combinations under unsafe config. -2. Mirror the Clarion test shape to make the two trust-boundary policies easy to compare. -3. Include a positive test for an explicitly permitted safe Filigree URL. - -## Low Findings - -### L-01: Analyzer reaches into `SummaryCache` private state - -Severity: Low -Locations: - -- [src/wardline/scanner/analyzer.py:173](/home/john/wardline/src/wardline/scanner/analyzer.py:173) -- [src/wardline/scanner/taint/summary_cache.py:88](/home/john/wardline/src/wardline/scanner/taint/summary_cache.py:88)-97 -- [src/wardline/scanner/taint/summary_cache.py:125](/home/john/wardline/src/wardline/scanner/taint/summary_cache.py:125)-134 - -Finding: analyzer logic reaches into `SummaryCache._entries`, which couples orchestration code to cache internals. - -Remediation: - -1. Add a public cache API such as `has_current(path, digest)` or `lookup_current(path, digest)`. -2. Move freshness checks into `SummaryCache`. -3. Update analyzer tests to assert behavior through the public API. - -### L-02: `core.protocols` abstractions are declared but not wired - -Severity: Low -Locations: - -- [src/wardline/core/protocols.py:14](/home/john/wardline/src/wardline/core/protocols.py:14) -- [src/wardline/core/protocols.py:18](/home/john/wardline/src/wardline/core/protocols.py:18) -- [src/wardline/core/run.py:96](/home/john/wardline/src/wardline/core/run.py:96) - -Finding: protocol abstractions exist but are not meaningfully used by the orchestration surface. - -Remediation: - -1. Wire the protocols into `run_scan`/dependency construction if they represent intended extension seams. -2. Otherwise remove or deprecate them to reduce design noise. -3. Add tests only if the protocols are kept as supported extension points. - -### L-03: Some tests mutate `sys.path` without cleanup - -Severity: Low -Locations: - -- [tests/unit/core/test_packs.py:11](/home/john/wardline/tests/unit/core/test_packs.py:11)-14 -- [tests/unit/core/test_judge_run.py:96](/home/john/wardline/tests/unit/core/test_judge_run.py:96)-105 -- [tests/unit/cli/test_cli.py:114](/home/john/wardline/tests/unit/cli/test_cli.py:114)-143 - -Finding: some tests insert paths into `sys.path` directly. This can leak import state across tests. - -Remediation: - -1. Use `monkeypatch.syspath_prepend`. -2. Add fixtures that clean up import/module state after each test. -3. Keep dynamically created modules isolated per test. - -### L-04: LSP input framing lacks a maximum `Content-Length` - -Severity: Low -Locations: - -- [src/wardline/mcp/protocol.py:122](/home/john/wardline/src/wardline/mcp/protocol.py:122)-129 -- [src/wardline/mcp/lsp.py:78](/home/john/wardline/src/wardline/mcp/lsp.py:78) -- [src/wardline/mcp/lsp.py:83](/home/john/wardline/src/wardline/mcp/lsp.py:83)-91 - -Finding: MCP line input has a 10 MB guard, but LSP-style `Content-Length` framing does not appear to enforce a comparable maximum body size. - -Remediation: - -1. Add a maximum LSP body size. -2. Reject or drain oversized frames deterministically. -3. Add tests for exactly-at-limit, over-limit, malformed, and missing-length frames. - -### L-05: CLI `scan --fix` drops `strict_defaults` on post-fix rescan - -Severity: Low -Locations: - -- [src/wardline/cli/scan.py:137](/home/john/wardline/src/wardline/cli/scan.py:137)-178 - -Finding: the first scan honors `strict_defaults`, but the post-fix rescan does not pass that setting through. - -Remediation: - -1. Pass `strict_defaults=strict_defaults` to the post-fix `run_scan` call. -2. Add a CLI regression test where strict defaults affect diagnostics before and after `--fix`. - -## Prioritized Remediation Plan - -1. Close direct trust-boundary vulnerabilities first: fix MCP waiver symlink confinement, validate autofix config values, and prevent untrusted project config from lowering judge assurance. -2. Restore scanner determinism next: make cold and warm scans produce identical L2-backed rule behavior, then add invariant tests. -3. Repair feedback loops: add Filigree clean-file heartbeat/reconciliation and validate close-on-fixed with live or contract e2e coverage. -4. Harden MCP contracts: block side-effecting notifications, reject unknown schema arguments, and make mutating tools idempotent. -5. Improve structural maintainability: split the MCP server, move identity types into a neutral module, and replace private cache access with public APIs. -6. Expand CI and tests: schedule live integration oracles, add Filigree unsafe-config tests, add nested-scope static-analysis fixtures, and clean `sys.path` mutation patterns. - -## Suggested Acceptance Tests - -- Cold scan and warm cached scan of the same fixture produce byte-for-byte equivalent diagnostics for L2-backed sink and PY-WL-105 cases. -- MCP `waiver_add` rejects an out-of-root symlinked `wardline.yaml`. -- Invalid `autofix.boundary_exception` values fail config validation before any AST edit is planned. -- Judge config cannot lower suppression threshold unless an explicit trust flag is supplied. -- A file with one Filigree-backed finding is fixed, rescanned clean, and the associated issue closes. -- No-id JSON-RPC `tools/call` notifications cannot mutate project state. -- MCP mutating tool calls with unknown keys return `isError`. -- CLI scans reject out-of-root source paths by default and allow them only with an explicit escape flag. - -## Audit Limitations - -- This was a static read-only audit. No source edits or test runs were performed as part of remediation. -- Live Clarion, Legis, Filigree, and external LLM judge services were not exercised. -- Findings were synthesized from code inspection and seven specialized read-only reviewer reports. -- The report should be treated as a remediation backlog plus a verification guide, not as proof that every listed bug is currently reproducible under all runtime configurations. From 33b80594e742551281fc84a7b1dca295d4147e3d Mon Sep 17 00:00:00 2001 From: John Morrissey Date: Sun, 7 Jun 2026 22:58:34 +1000 Subject: [PATCH 003/186] docs(doctor): spec filigree federation-token check + repair wardline doctor gains a filigree.auth check: probe the configured daemon (URL sourced from .mcp.json) with the token wardline would emit, detect 401/403 token-mismatch, and under --repair recover the accepted token from local mints and pin it as WEFT_FEDERATION_TOKEN in .env. Co-Authored-By: Claude Opus 4.8 --- ...line-doctor-filigree-token-check-design.md | 182 ++++++++++++++++++ 1 file changed, 182 insertions(+) create mode 100644 docs/superpowers/specs/2026-06-07-wardline-doctor-filigree-token-check-design.md diff --git a/docs/superpowers/specs/2026-06-07-wardline-doctor-filigree-token-check-design.md b/docs/superpowers/specs/2026-06-07-wardline-doctor-filigree-token-check-design.md new file mode 100644 index 00000000..8738346e --- /dev/null +++ b/docs/superpowers/specs/2026-06-07-wardline-doctor-filigree-token-check-design.md @@ -0,0 +1,182 @@ +# `wardline doctor` — filigree federation-token check + repair + +**Date:** 2026-06-07 +**Status:** Approved (design) +**Branch:** `rc4` + +## Problem + +When wardline emits scan findings to a Filigree daemon over the Weft federation +surface (`POST /api/weft/scan-results`), the request carries +`Authorization: Bearer ` where the token comes from +`load_filigree_token(root)` (env → `root/.env`, federation name then legacy +fallback). If that token is not the value the **running daemon** accepts, every +emit returns `401` and fails *soft and silent*: the scan still succeeds, findings +are written locally, and the only signal is a `filigree_emit` block the operator +must read. The Filigree tracker looks empty even though wardline holds active +defects. + +### Root cause (observed in lacuna, 2026-06-07) + +The failure is **not** a missing token or an unauthenticated POST. wardline reads +`root/.env` itself and *does* send a bearer token — just the wrong value. The +real cause is **two independently-minted federation-token stores that disagree**, +with no tier-1 env override to unify them: + +- Filigree's federation token is auto-minted **per store-dir** + (`filigree/federation_token.py`). A daemon launched with `--server-mode` + resolves its token from `~/.config/filigree/federation_token`; a single-project + resolution uses `/.weft/filigree/federation_token`. These are minted + independently and hold **different** secrets. +- In lacuna, the live daemon on `:8749` runs `--server-mode` with no + `WEFT_FEDERATION_TOKEN` in its environment, so it accepts the + `~/.config/filigree/` value (call it `W`). lacuna's `.env` carries + `WARDLINE_FILIGREE_TOKEN` set to the *project-store* value (call it `D`). +- wardline emits `Bearer D`; the daemon only knows `W` → `401`. The Filigree + MCP client's `.mcp.json` bearer happens to be `W`, so MCP reads work — masking + the rotation. Empirically: probing the live daemon, `W → HTTP 400` (auth + passed, sentinel body rejected), `D → HTTP 401` (auth rejected). + +This is **not** a stale in-memory daemon and a restart does **not** fix it: the +daemon's token file is unchanged since boot; a restart re-reads `W` and still +rejects `D`. Git history shows recurring `fix(filigree): update authorization +token` commits — the churn of hand-pasting a rotated value with no single source +of truth. + +`wardline doctor` should detect this mismatch and repair it. + +## Goals + +- Detect, from `wardline doctor`, that the token wardline **will emit** is not the + token the configured Filigree daemon **accepts**. +- Under `--repair`, recover the correct token from local mints and pin it as + `WEFT_FEDERATION_TOKEN` in `/.env`, removing the stale legacy line. +- Emit a message that distinguishes *token absent* from *token present but + rejected* — the existing `filigree_emit.py` 401 string ("set + WEFT_FEDERATION_TOKEN") reads as "no token," which is what originally + misdirected diagnosis. + +## Non-goals (YAGNI) + +- Reconciling the **Filigree MCP-client** bearer in `.mcp.json` — that is + Filigree's client config, not wardline's emit path. doctor fixes only the token + **wardline emits** (`.env`). A drifted MCP bearer is out of scope (noted, not + touched). +- Re-minting or editing Filigree's store files (`federation_token`). +- Cross-host recovery: if the daemon authenticates against a `WEFT_FEDERATION_TOKEN` + env override that no local file matches, doctor cannot recover the value — it + reports the situation and the operator action, and writes nothing. + +## Design + +### New check: `filigree.auth` + +A `DoctorCheck` added to `machine_readable_doctor`, alongside the existing +checks, and surfaced in the human `doctor` output. + +#### Probe-URL resolution (precedence) + +doctor must probe the **same daemon the emit path hits**. The emit URL in the +real (MCP) setup lives in `.mcp.json`, not in env/config, so a plain +`resolve_filigree_url(None, root)` returns `None` and would never probe. The +probe URL therefore resolves by: + +1. `--filigree-url` flag (new, optional on `doctor`, mirrors `scan`) +2. `WARDLINE_FILIGREE_URL` env var +3. **`.mcp.json` → `mcpServers.wardline.args` → value after `--filigree-url`** + (doctor already parses `.mcp.json` for `_check_project_mcp`) +4. published-port rung (`.weft/filigree/ephemeral.port`, legacy + `.filigree/ephemeral.port`) + +If none resolve → `ok`, message `"filigree not configured; nothing to verify"`. + +#### Token + +`load_filigree_token(root)` — exactly the value emit would send. + +#### Detection (read-only; runs without `--repair`) + +| Condition | Result | +|---|---| +| URL resolves, token is `None` | `error` — `"no federation token set; export WEFT_FEDERATION_TOKEN or add it to .env"` | +| Resolved URL is **non-loopback** | `ok` — `"non-loopback filigree; token not probed"` (never send a bearer off-box) | +| Probe → `401`/`403` | `error` — `"emit token rejected by filigree (); the configured token is not what the daemon accepts"` | +| Probe → unreachable (conn refused / timeout) | `ok` — `"filigree daemon not reachable; token not verified"` | +| Probe → any other status (e.g. `400`, `2xx`) | `ok` | + +**Probe mechanism:** `POST ` with a **sentinel body `{}`** and the bearer, +~2 s timeout. Filigree's auth middleware runs *before* body validation, so a good +token yields `400` (request rejected, **nothing recorded**) and a bad token +yields `401/403`. This is **not** `emit([])`, which would POST a valid +empty-findings body and could register an empty scan. + +#### Repair (`--repair`; only when detection saw `401`/`403`) + +1. Collect candidate tokens from locally-readable mints, in order: + - `~/.config/filigree/federation_token` (server-mode store) + - `/.weft/filigree/federation_token` (project store) + + The already-rejected `.env`/env value is skipped. +2. Probe each candidate against the daemon (same sentinel POST). A daemon accepts + exactly one token, so at most one **distinct** value can authenticate. +3. Outcome: + - **Exactly one accepted** → surgically rewrite `/.env`: set + `WEFT_FEDERATION_TOKEN=`, remove any stale `WARDLINE_FILIGREE_TOKEN=` + line, preserve all other lines, `chmod 0600`. Mark `fixed`; re-probe to + confirm `ok`. + - **None accepted** → `error`, **no write**: `"no local federation_token matched + the daemon — it likely uses a WEFT_FEDERATION_TOKEN env override; set that + same value in .env"`. + +### Code surface + +- **`core/filigree_emit.py`** — add `FiligreeEmitter.verify_token() -> ProbeResult` + (a small frozen dataclass: `accepted: bool`, `reachable: bool`, + `status: int | None`). Reuses the existing auth-header construction and the + injectable `Transport` seam. Sends the sentinel body. Does **not** reuse + `emit()`. +- **`install/doctor.py`** — `_check_filigree_auth(root, *, repair: bool, + filigree_url: str | None)` returning a `DoctorCheck`; helpers: + - `_resolve_probe_url(root, flag)` (precedence above, incl. `.mcp.json` arg + extraction) + - `_filigree_token_candidates(root)` (the two store-dir files) + - `_rewrite_env_token(env_path, value)` (surgical `.env` update + legacy-line + removal + `0600`) + + Wire `_check_filigree_auth` into `machine_readable_doctor`. The repair path runs + inside the existing `fix` flow. +- **`cli/doctor.py`** — add optional `--filigree-url` passthrough; thread it to + `machine_readable_doctor` / `_check_filigree_auth`. + +### Loopback discipline + +The bearer is only sent to loopback origins. A non-loopback resolved URL skips the +probe entirely (reports `ok` / not-probed) rather than transmitting the token +off-box — mirroring the existing Loomweave token-origin discipline. + +## Testing + +Unit tests with an **injected prober / `Transport` stub** — no real network: + +- Detection: rejected (`401`) → `error`; token `None` → absent-message `error`; + unreachable → non-failing `ok`; non-loopback → skipped `ok`; URL unresolved → + `ok`. +- Probe-URL resolution: flag > env > `.mcp.json` arg > published port; the + `.mcp.json`-arg rung exercised explicitly (the lacuna shape). +- Repair: rejected + exactly-one-candidate-accepted → `.env` rewritten, + `WEFT_FEDERATION_TOKEN` set, legacy `WARDLINE_FILIGREE_TOKEN` line removed, + unrelated lines preserved, mode `0600`, re-probe `ok`/`fixed`. +- Repair: rejected + no-candidate-accepted → no write, guidance `error`. +- `verify_token()`: maps `401/403 → accepted=False`, `400/2xx → accepted=True`, + transport error → `reachable=False`. + +**Acceptance oracle (manual, already performed):** against the live lacuna daemon, +`W → HTTP 400`, `D → HTTP 401`, garbage `→ 401`. No real-network test enters the +default suite. + +## Rollout + +- Lands on `rc4` with the rest of the release-candidate work (single-RC-branch + discipline). +- Pure addition: a new check + new emitter method + a new optional flag. No + behavior change to existing checks, `scan`, or `emit`. From 665d0c6d21212c96372c529818f73fac062416f7 Mon Sep 17 00:00:00 2001 From: John Morrissey Date: Sun, 7 Jun 2026 23:17:03 +1000 Subject: [PATCH 004/186] docs(doctor): implementation plan for filigree.auth check 7-task TDD plan: verify_token probe, .env rewriter, probe-URL resolution (flag>env>.mcp.json arg; no published-port rung), detection (probe-always; absent token only errors on a real 401), repair-from-local-mints, CLI wiring. Reconciles the spec with two planning refinements. Co-Authored-By: Claude Opus 4.8 --- ...07-wardline-doctor-filigree-token-check.md | 853 ++++++++++++++++++ ...line-doctor-filigree-token-check-design.md | 20 +- 2 files changed, 868 insertions(+), 5 deletions(-) create mode 100644 docs/superpowers/plans/2026-06-07-wardline-doctor-filigree-token-check.md diff --git a/docs/superpowers/plans/2026-06-07-wardline-doctor-filigree-token-check.md b/docs/superpowers/plans/2026-06-07-wardline-doctor-filigree-token-check.md new file mode 100644 index 00000000..ac9d1721 --- /dev/null +++ b/docs/superpowers/plans/2026-06-07-wardline-doctor-filigree-token-check.md @@ -0,0 +1,853 @@ +# wardline doctor — filigree federation-token check Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Add a `filigree.auth` check to `wardline doctor` that probes the configured Filigree daemon with the token wardline would emit, detects a 401/403 token mismatch, and under `--repair` recovers the accepted token from local mints and pins it as `WEFT_FEDERATION_TOKEN` in `.env`. + +**Architecture:** A new `FiligreeEmitter.verify_token()` probe (sentinel-body POST, reusing the existing injectable `Transport` seam) provides the live auth check. `install/doctor.py` gains `_check_filigree_auth` plus helpers for probe-URL resolution (flag → env → `.mcp.json` arg → published port), candidate-token gathering, and a surgical `.env` rewrite. The check is wired into `machine_readable_doctor` (JSON path) and surfaced in the human `doctor` / `--repair` output. + +**Tech Stack:** Python 3.13, click, stdlib `urllib`/`json`/`tomllib`, pytest. Zero new deps. + +**Spec:** `docs/superpowers/specs/2026-06-07-wardline-doctor-filigree-token-check-design.md` + +--- + +## File Structure + +- `src/wardline/core/filigree_emit.py` — add `ProbeResult` dataclass + `FiligreeEmitter.verify_token()`. +- `src/wardline/install/doctor.py` — add `_check_filigree_auth` + helpers (`_resolve_probe_url`, `_mcp_filigree_url`, `_is_loopback`, `_filigree_token_candidates`, `_rewrite_env_token`); wire into `machine_readable_doctor`. +- `src/wardline/cli/doctor.py` — add optional `--filigree-url`; surface `filigree.auth` in human output. +- `tests/unit/core/test_filigree_verify_token.py` — probe classification. +- `tests/unit/install/test_doctor_filigree_auth.py` — resolution, detection, repair, `.env` rewrite. +- `tests/unit/cli/test_doctor.py` — CLI flag + not-configured integration (append). + +--- + +## Task 1: `FiligreeEmitter.verify_token()` probe + +**Files:** +- Modify: `src/wardline/core/filigree_emit.py` (add `ProbeResult` near `EmitResult`; add `verify_token` method on `FiligreeEmitter`) +- Test: `tests/unit/core/test_filigree_verify_token.py` + +- [ ] **Step 1: Write the failing test** + +```python +# tests/unit/core/test_filigree_verify_token.py +from collections.abc import Mapping + +import pytest + +from wardline.core.filigree_emit import FiligreeEmitter, Response + + +class _FakeTransport: + """Records the last POST and returns a canned Response or raises.""" + + def __init__(self, *, status: int | None = None, exc: Exception | None = None) -> None: + self._status = status + self._exc = exc + self.calls: list[tuple[str, bytes, dict[str, str]]] = [] + + def post(self, url: str, body: bytes, headers: Mapping[str, str]) -> Response: + self.calls.append((url, body, dict(headers))) + if self._exc is not None: + raise self._exc + assert self._status is not None + return Response(status=self._status, body="") + + +def test_verify_token_401_is_rejected() -> None: + t = _FakeTransport(status=401) + result = FiligreeEmitter("http://127.0.0.1:8749/api/weft/scan-results", transport=t, token="bad").verify_token() + assert result.reachable is True + assert result.accepted is False + assert result.status == 401 + + +def test_verify_token_400_is_accepted() -> None: + # Auth middleware runs before body validation: a good token + sentinel body => 400. + t = _FakeTransport(status=400) + result = FiligreeEmitter("http://127.0.0.1:8749/api/weft/scan-results", transport=t, token="good").verify_token() + assert result.accepted is True + assert result.status == 400 + # Sentinel body present; bearer attached. + url, body, headers = t.calls[0] + assert headers["Authorization"] == "Bearer good" + assert body # non-empty sentinel + + +def test_verify_token_403_is_rejected() -> None: + result = FiligreeEmitter("http://x/y", transport=_FakeTransport(status=403), token="t").verify_token() + assert result.accepted is False + + +def test_verify_token_transport_error_is_unreachable() -> None: + t = _FakeTransport(exc=OSError("connection refused")) + result = FiligreeEmitter("http://127.0.0.1:8749/api/weft/scan-results", transport=t, token="t").verify_token() + assert result.reachable is False + assert result.accepted is False + assert result.status is None +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: `.venv/bin/pytest tests/unit/core/test_filigree_verify_token.py -q` +Expected: FAIL — `AttributeError: 'FiligreeEmitter' object has no attribute 'verify_token'`. + +- [ ] **Step 3: Write minimal implementation** + +In `src/wardline/core/filigree_emit.py`, add the `ProbeResult` dataclass immediately after the `EmitResult` class (before `filigree_disabled_reason`): + +```python +@dataclass(frozen=True, slots=True) +class ProbeResult: + """Outcome of an auth probe (verify_token). ``accepted`` is True when the daemon + authenticated the bearer (any non-401/403 status, e.g. a 400 from the sentinel body). + ``reachable`` is False only on a transport failure (connection refused / timeout).""" + + reachable: bool + accepted: bool + status: int | None = None +``` + +Add the method to `FiligreeEmitter` (after `emit`): + +```python + def verify_token(self) -> ProbeResult: + """Probe whether the daemon accepts this emitter's bearer token, WITHOUT + recording anything. Auth runs in middleware before body validation, so a + deliberately-incomplete sentinel body yields 400 (auth passed) or 401/403 + (rejected). Never reuses emit() — that would POST a valid empty scan.""" + body = b"{}" # parses as JSON, missing required scan-results fields => 400 when authed + headers = {"Content-Type": "application/json"} + if self._token: + headers["Authorization"] = f"Bearer {self._token}" + try: + resp = self._transport.post(self._url, body, headers) + except (urllib.error.URLError, OSError): + return ProbeResult(reachable=False, accepted=False) + accepted = resp.status not in (401, 403) + return ProbeResult(reachable=True, accepted=accepted, status=resp.status) +``` + +(`urllib` is already imported at the top of the module.) + +- [ ] **Step 4: Run test to verify it passes** + +Run: `.venv/bin/pytest tests/unit/core/test_filigree_verify_token.py -q` +Expected: PASS (4 passed). + +- [ ] **Step 5: Commit** + +```bash +git add src/wardline/core/filigree_emit.py tests/unit/core/test_filigree_verify_token.py +git commit -m "feat(emit): add FiligreeEmitter.verify_token auth probe" +``` + +--- + +## Task 2: surgical `.env` token rewriter + +**Files:** +- Modify: `src/wardline/install/doctor.py` (add `_rewrite_env_token`) +- Test: `tests/unit/install/test_doctor_filigree_auth.py` (new file; first tests) + +- [ ] **Step 1: Write the failing test** + +```python +# tests/unit/install/test_doctor_filigree_auth.py +import stat +from pathlib import Path + +from wardline.install.doctor import _rewrite_env_token + + +def test_rewrite_env_sets_new_name_and_drops_legacy(tmp_path: Path) -> None: + env = tmp_path / ".env" + env.write_text( + "WARDLINE_ATTEST_KEY=keep-me\nWARDLINE_FILIGREE_TOKEN=stale\nOTHER=x\n", + encoding="utf-8", + ) + _rewrite_env_token(env, "GOODTOKEN") + text = env.read_text(encoding="utf-8") + assert "WEFT_FEDERATION_TOKEN=GOODTOKEN" in text + assert "WARDLINE_FILIGREE_TOKEN" not in text # stale legacy line removed + assert "WARDLINE_ATTEST_KEY=keep-me" in text # unrelated line preserved + assert "OTHER=x" in text + assert stat.S_IMODE(env.stat().st_mode) == 0o600 + + +def test_rewrite_env_updates_existing_new_name(tmp_path: Path) -> None: + env = tmp_path / ".env" + env.write_text("WEFT_FEDERATION_TOKEN=old\nKEEP=1\n", encoding="utf-8") + _rewrite_env_token(env, "NEW") + text = env.read_text(encoding="utf-8") + assert text.count("WEFT_FEDERATION_TOKEN=") == 1 + assert "WEFT_FEDERATION_TOKEN=NEW" in text + assert "KEEP=1" in text + + +def test_rewrite_env_creates_file_when_absent(tmp_path: Path) -> None: + env = tmp_path / ".env" + _rewrite_env_token(env, "NEW") + assert env.read_text(encoding="utf-8").strip() == "WEFT_FEDERATION_TOKEN=NEW" + assert stat.S_IMODE(env.stat().st_mode) == 0o600 +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: `.venv/bin/pytest tests/unit/install/test_doctor_filigree_auth.py -q` +Expected: FAIL — `ImportError: cannot import name '_rewrite_env_token'`. + +- [ ] **Step 3: Write minimal implementation** + +In `src/wardline/install/doctor.py`, add near the other helpers: + +```python +def _rewrite_env_token(env_path: Path, value: str) -> None: + """Surgically pin ``WEFT_FEDERATION_TOKEN=`` in *env_path*. Removes any + existing ``WEFT_FEDERATION_TOKEN`` or legacy ``WARDLINE_FILIGREE_TOKEN`` line, + preserves all other lines/order, creates the file if absent, and sets mode 0600 + (the file holds a secret).""" + drop = ("WEFT_FEDERATION_TOKEN=", "WARDLINE_FILIGREE_TOKEN=") + kept: list[str] = [] + if env_path.is_file(): + for raw in env_path.read_text(encoding="utf-8", errors="replace").splitlines(): + if raw.strip().startswith(drop): + continue + kept.append(raw) + kept.append(f"WEFT_FEDERATION_TOKEN={value}") + env_path.write_text("\n".join(kept) + "\n", encoding="utf-8") + env_path.chmod(0o600) +``` + +- [ ] **Step 4: Run test to verify it passes** + +Run: `.venv/bin/pytest tests/unit/install/test_doctor_filigree_auth.py -q` +Expected: PASS (3 passed). + +- [ ] **Step 5: Commit** + +```bash +git add src/wardline/install/doctor.py tests/unit/install/test_doctor_filigree_auth.py +git commit -m "feat(doctor): add surgical .env federation-token rewriter" +``` + +--- + +## Task 3: probe-URL resolution + loopback helper + +**Files:** +- Modify: `src/wardline/install/doctor.py` (add `_mcp_filigree_url`, `_resolve_probe_url`, `_is_loopback`) +- Test: `tests/unit/install/test_doctor_filigree_auth.py` (append) + +- [ ] **Step 1: Write the failing test** + +```python +# append to tests/unit/install/test_doctor_filigree_auth.py +import json + +from wardline.install.doctor import _is_loopback, _mcp_filigree_url, _resolve_probe_url + + +def _write_mcp_with_filigree_url(root: Path, url: str) -> None: + root.joinpath(".mcp.json").write_text( + json.dumps( + {"mcpServers": {"wardline": {"type": "stdio", "command": "wardline", + "args": ["mcp", "--root", ".", "--filigree-url", url]}}} + ), + encoding="utf-8", + ) + + +def test_mcp_filigree_url_extracts_arg(tmp_path: Path) -> None: + _write_mcp_with_filigree_url(tmp_path, "http://127.0.0.1:8749/api/weft/scan-results") + assert _mcp_filigree_url(tmp_path) == "http://127.0.0.1:8749/api/weft/scan-results" + + +def test_mcp_filigree_url_none_when_absent(tmp_path: Path) -> None: + tmp_path.joinpath(".mcp.json").write_text( + json.dumps({"mcpServers": {"wardline": {"args": ["mcp", "--root", "."]}}}), encoding="utf-8" + ) + assert _mcp_filigree_url(tmp_path) is None + assert _mcp_filigree_url(tmp_path / "nope") is None # missing file + + +def test_resolve_probe_url_precedence(tmp_path: Path, monkeypatch) -> None: + monkeypatch.delenv("WARDLINE_FILIGREE_URL", raising=False) + _write_mcp_with_filigree_url(tmp_path, "http://127.0.0.1:8749/api/weft/scan-results") + # flag wins + assert _resolve_probe_url(tmp_path, "http://flag/x") == "http://flag/x" + # env beats .mcp.json + monkeypatch.setenv("WARDLINE_FILIGREE_URL", "http://env/y") + assert _resolve_probe_url(tmp_path, None) == "http://env/y" + # .mcp.json arg is the fallback that makes the real setup work + monkeypatch.delenv("WARDLINE_FILIGREE_URL", raising=False) + assert _resolve_probe_url(tmp_path, None) == "http://127.0.0.1:8749/api/weft/scan-results" + + +def test_resolve_probe_url_none_when_unconfigured(tmp_path: Path, monkeypatch) -> None: + monkeypatch.delenv("WARDLINE_FILIGREE_URL", raising=False) + assert _resolve_probe_url(tmp_path, None) is None + + +def test_is_loopback() -> None: + assert _is_loopback("http://127.0.0.1:8749/x") is True + assert _is_loopback("http://localhost:8749/x") is True + assert _is_loopback("http://[::1]:8749/x") is True + assert _is_loopback("https://filigree.example.com/x") is False +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: `.venv/bin/pytest tests/unit/install/test_doctor_filigree_auth.py -q` +Expected: FAIL — `ImportError: cannot import name '_mcp_filigree_url'`. + +- [ ] **Step 3: Write minimal implementation** + +In `src/wardline/install/doctor.py` (the module already imports `json`, `os`, and `urlsplit` from `urllib.parse`), add the helpers. Note the probe URL is resolved ONLY from a deliberately-configured emit target (flag → env → `.mcp.json --filigree-url` arg); the published-port rung is intentionally NOT used, so doctor does no network unless filigree emit is explicitly wired (this also keeps the existing doctor tests network-free): + +```python +_FILIGREE_URL_ENV = "WARDLINE_FILIGREE_URL" + + +def _mcp_filigree_url(root: Path) -> str | None: + """The ``--filigree-url`` value from the wardline server entry in ``.mcp.json``, + or None. This is the URL the agent's MCP server actually emits to, and the only + place it is recorded in the common (MCP) setup.""" + path = root / ".mcp.json" + if not path.is_file(): + return None + try: + raw = json.loads(path.read_text(encoding="utf-8")) + except (json.JSONDecodeError, OSError): + return None + try: + args = raw["mcpServers"]["wardline"]["args"] + idx = args.index("--filigree-url") + value = args[idx + 1] + except (KeyError, TypeError, ValueError, IndexError): + return None + return value if isinstance(value, str) else None + + +def _resolve_probe_url(root: Path, flag: str | None) -> str | None: + """Probe-URL precedence: flag > WARDLINE_FILIGREE_URL env > .mcp.json wardline + --filigree-url arg. None when nothing resolves. The published-port rung is + deliberately excluded: doctor probes only a configured emit target, so it does + no network unless filigree emit is explicitly wired.""" + if flag: + return flag + env = os.environ.get(_FILIGREE_URL_ENV) + if env: + return env + return _mcp_filigree_url(root) + + +def _is_loopback(url: str) -> bool: + """True when *url*'s host is loopback — the only origins a bearer is probed against.""" + host = (urlsplit(url).hostname or "").lower() + return host in {"localhost", "::1"} or host.startswith("127.") +``` + +- [ ] **Step 4: Run test to verify it passes** + +Run: `.venv/bin/pytest tests/unit/install/test_doctor_filigree_auth.py -q` +Expected: PASS. + +- [ ] **Step 5: Commit** + +```bash +git add src/wardline/install/doctor.py tests/unit/install/test_doctor_filigree_auth.py +git commit -m "feat(doctor): resolve filigree probe URL (incl. .mcp.json arg) + loopback guard" +``` + +--- + +## Task 4: `_check_filigree_auth` — detection (no repair) + +**Files:** +- Modify: `src/wardline/install/doctor.py` (add `_filigree_token_candidates`, `_check_filigree_auth`) +- Test: `tests/unit/install/test_doctor_filigree_auth.py` (append) + +- [ ] **Step 1: Write the failing test** + +```python +# append to tests/unit/install/test_doctor_filigree_auth.py +from collections.abc import Mapping + +from wardline.core.filigree_emit import Response +from wardline.install.doctor import _check_filigree_auth + + +class _ScriptedTransport: + """Returns a Response per token: maps Authorization bearer -> status.""" + + def __init__(self, status_by_token: dict[str, int], *, unreachable: bool = False) -> None: + self._status_by_token = status_by_token + self._unreachable = unreachable + + def post(self, url: str, body: bytes, headers: Mapping[str, str]) -> Response: + if self._unreachable: + raise OSError("connection refused") + token = headers.get("Authorization", "").removeprefix("Bearer ") + return Response(status=self._status_by_token.get(token, 401), body="") + + +def _setup_lacuna(root: Path, monkeypatch, env_token: str) -> None: + monkeypatch.delenv("WARDLINE_FILIGREE_URL", raising=False) + monkeypatch.delenv("WEFT_FEDERATION_TOKEN", raising=False) + monkeypatch.delenv("WARDLINE_FILIGREE_TOKEN", raising=False) + _write_mcp_with_filigree_url(root, "http://127.0.0.1:8749/api/weft/scan-results") + root.joinpath(".env").write_text(f"WARDLINE_FILIGREE_TOKEN={env_token}\n", encoding="utf-8") + + +def test_check_detects_rejected_token(tmp_path: Path, monkeypatch) -> None: + _setup_lacuna(tmp_path, monkeypatch, env_token="STALE") + t = _ScriptedTransport({"GOOD": 400}) # daemon accepts GOOD; STALE -> 401 + check = _check_filigree_auth(tmp_path, repair=False, transport=t) + assert check.status == "error" + assert "rejected" in (check.message or "") + assert check.fixed is False + + +def test_check_ok_when_token_accepted(tmp_path: Path, monkeypatch) -> None: + _setup_lacuna(tmp_path, monkeypatch, env_token="GOOD") + t = _ScriptedTransport({"GOOD": 400}) + check = _check_filigree_auth(tmp_path, repair=False, transport=t) + assert check.status == "ok" + + +def test_check_error_when_token_absent(tmp_path: Path, monkeypatch) -> None: + monkeypatch.delenv("WARDLINE_FILIGREE_URL", raising=False) + monkeypatch.delenv("WEFT_FEDERATION_TOKEN", raising=False) + monkeypatch.delenv("WARDLINE_FILIGREE_TOKEN", raising=False) + _write_mcp_with_filigree_url(tmp_path, "http://127.0.0.1:8749/api/weft/scan-results") + check = _check_filigree_auth(tmp_path, repair=False, transport=_ScriptedTransport({})) + assert check.status == "error" + assert "no federation token" in (check.message or "") + + +def test_check_ok_when_auth_off_and_no_token(tmp_path: Path, monkeypatch) -> None: + # Daemon has auth OFF: it accepts an unauthenticated emit ("" bearer -> 400). No token + # configured is fine — not an error. + monkeypatch.delenv("WARDLINE_FILIGREE_URL", raising=False) + monkeypatch.delenv("WEFT_FEDERATION_TOKEN", raising=False) + monkeypatch.delenv("WARDLINE_FILIGREE_TOKEN", raising=False) + _write_mcp_with_filigree_url(tmp_path, "http://127.0.0.1:8749/api/weft/scan-results") + t = _ScriptedTransport({"": 400}) # empty (no) bearer is accepted + check = _check_filigree_auth(tmp_path, repair=False, transport=t) + assert check.status == "ok" + + +def test_check_ok_when_unreachable(tmp_path: Path, monkeypatch) -> None: + _setup_lacuna(tmp_path, monkeypatch, env_token="STALE") + check = _check_filigree_auth(tmp_path, repair=False, transport=_ScriptedTransport({}, unreachable=True)) + assert check.status == "ok" + assert "not reachable" in (check.message or "") + + +def test_check_ok_when_non_loopback(tmp_path: Path, monkeypatch) -> None: + monkeypatch.delenv("WARDLINE_FILIGREE_URL", raising=False) + monkeypatch.setenv("WEFT_FEDERATION_TOKEN", "T") + check = _check_filigree_auth(tmp_path, repair=False, filigree_url="https://remote.example.com/api/weft/scan-results", + transport=_ScriptedTransport({})) + assert check.status == "ok" + assert "non-loopback" in (check.message or "") + + +def test_check_ok_when_url_unresolved(tmp_path: Path, monkeypatch) -> None: + monkeypatch.delenv("WARDLINE_FILIGREE_URL", raising=False) + check = _check_filigree_auth(tmp_path, repair=False, transport=_ScriptedTransport({})) + assert check.status == "ok" + assert "not configured" in (check.message or "") +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: `.venv/bin/pytest tests/unit/install/test_doctor_filigree_auth.py -q` +Expected: FAIL — `ImportError: cannot import name '_check_filigree_auth'`. + +- [ ] **Step 3: Write minimal implementation** + +In `src/wardline/install/doctor.py`, add the import for the emitter and token loader at the top with the other imports: + +```python +from wardline.core.filigree_emit import FiligreeEmitter, Transport, UrllibTransport +from wardline.filigree.config import load_filigree_token +``` + +Add the candidate helper and the check (detection only; the `repair` branch is filled in Task 5): + +```python +def _filigree_token_candidates(root: Path) -> list[str]: + """Locally-readable federation-token mints, in precedence order: the server-mode + store (~/.config/filigree) then the project store (/.weft/filigree). Returns + distinct, non-empty values.""" + paths = [ + Path.home() / ".config" / "filigree" / "federation_token", + root / ".weft" / "filigree" / "federation_token", + ] + out: list[str] = [] + for p in paths: + try: + value = p.read_text(encoding="utf-8").strip() + except OSError: + continue + if value and value not in out: + out.append(value) + return out + + +def _check_filigree_auth( + root: Path, + *, + repair: bool, + filigree_url: str | None = None, + transport: Transport | None = None, +) -> DoctorCheck: + """Verify the token wardline would emit is accepted by the configured Filigree + daemon. Read-only probe; under *repair*, recover the accepted token from local + mints and pin it in .env. The probe targets only loopback origins.""" + probe_transport = transport if transport is not None else UrllibTransport(timeout=2.0) + url = _resolve_probe_url(root, filigree_url) + if url is None: + return DoctorCheck("filigree.auth", "ok", message="filigree not configured; nothing to verify") + if not _is_loopback(url): + return DoctorCheck("filigree.auth", "ok", message="non-loopback filigree; token not probed") + token = load_filigree_token(root) # may be None — probe anyway (the daemon may have auth off) + probe = FiligreeEmitter(url, transport=probe_transport, token=token).verify_token() + if not probe.reachable: + return DoctorCheck("filigree.auth", "ok", message="filigree daemon not reachable; token not verified") + if probe.accepted: + return DoctorCheck("filigree.auth", "ok") + # Rejected (401/403): filigree auth is on and our credential is not accepted. + if repair: + return _repair_filigree_auth(root, url, probe_transport) # implemented in Task 5 + if token: + return DoctorCheck( + "filigree.auth", "error", + message=f"emit token rejected by filigree ({probe.status}); " + "the configured token is not what the daemon accepts", + ) + return DoctorCheck( + "filigree.auth", "error", + message="filigree rejected an unauthenticated emit and no federation token is set; " + "export WEFT_FEDERATION_TOKEN or add it to .env", + ) +``` + +For this task, add a temporary stub so the module imports while Task 5 is pending (it is replaced in Task 5): + +```python +def _repair_filigree_auth(root: Path, url: str, transport: Transport) -> DoctorCheck: + return DoctorCheck("filigree.auth", "error", message="repair not yet implemented") +``` + +- [ ] **Step 4: Run test to verify it passes** + +Run: `.venv/bin/pytest tests/unit/install/test_doctor_filigree_auth.py -q` +Expected: PASS (detection tests green; repair path not yet exercised). + +- [ ] **Step 5: Commit** + +```bash +git add src/wardline/install/doctor.py tests/unit/install/test_doctor_filigree_auth.py +git commit -m "feat(doctor): detect filigree emit-token mismatch via live probe" +``` + +--- + +## Task 5: `_repair_filigree_auth` — recover + pin the accepted token + +**Files:** +- Modify: `src/wardline/install/doctor.py` (replace the `_repair_filigree_auth` stub) +- Test: `tests/unit/install/test_doctor_filigree_auth.py` (append) + +- [ ] **Step 1: Write the failing test** + +```python +# append to tests/unit/install/test_doctor_filigree_auth.py +def test_repair_writes_accepted_candidate(tmp_path: Path, monkeypatch) -> None: + _setup_lacuna(tmp_path, monkeypatch, env_token="STALE") + # server-mode store holds the accepted token + cfg = tmp_path / "home" / ".config" / "filigree" + cfg.mkdir(parents=True) + (cfg / "federation_token").write_text("GOOD\n", encoding="utf-8") + monkeypatch.setattr("wardline.install.doctor.Path.home", lambda: tmp_path / "home") + t = _ScriptedTransport({"GOOD": 400}) # GOOD accepted, STALE -> 401 + + check = _check_filigree_auth(tmp_path, repair=True, transport=t) + + assert check.status == "ok" + assert check.fixed is True + env_text = (tmp_path / ".env").read_text(encoding="utf-8") + assert "WEFT_FEDERATION_TOKEN=GOOD" in env_text + assert "WARDLINE_FILIGREE_TOKEN" not in env_text + + +def test_repair_no_candidate_matches_does_not_write(tmp_path: Path, monkeypatch) -> None: + _setup_lacuna(tmp_path, monkeypatch, env_token="STALE") + cfg = tmp_path / "home" / ".config" / "filigree" + cfg.mkdir(parents=True) + (cfg / "federation_token").write_text("ALSO-WRONG\n", encoding="utf-8") + monkeypatch.setattr("wardline.install.doctor.Path.home", lambda: tmp_path / "home") + t = _ScriptedTransport({"GOOD": 400}) # neither STALE nor ALSO-WRONG is accepted + + check = _check_filigree_auth(tmp_path, repair=True, transport=t) + + assert check.status == "error" + assert "no local federation_token matched" in (check.message or "") + assert "WARDLINE_FILIGREE_TOKEN=STALE" in (tmp_path / ".env").read_text(encoding="utf-8") # untouched +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: `.venv/bin/pytest tests/unit/install/test_doctor_filigree_auth.py -q` +Expected: FAIL — repair returns the "not yet implemented" stub. + +- [ ] **Step 3: Write minimal implementation** + +Replace the `_repair_filigree_auth` stub in `src/wardline/install/doctor.py`: + +```python +def _repair_filigree_auth(root: Path, url: str, transport: Transport) -> DoctorCheck: + """A 401/403 was seen. Probe each locally-readable mint; if exactly one is + accepted, pin it as WEFT_FEDERATION_TOKEN in .env and confirm. Otherwise write + nothing and report (the daemon likely uses an env override we cannot read).""" + for candidate in _filigree_token_candidates(root): + probe = FiligreeEmitter(url, transport=transport, token=candidate).verify_token() + if probe.reachable and probe.accepted: + _rewrite_env_token(root / ".env", candidate) + return DoctorCheck( + "filigree.auth", "ok", fixed=True, + message="wrote WEFT_FEDERATION_TOKEN to .env (was a stale/mismatched token)", + ) + return DoctorCheck( + "filigree.auth", "error", + message="no local federation_token matched the daemon — it likely uses a " + "WEFT_FEDERATION_TOKEN env override; set that same value in .env", + ) +``` + +- [ ] **Step 4: Run test to verify it passes** + +Run: `.venv/bin/pytest tests/unit/install/test_doctor_filigree_auth.py -q` +Expected: PASS (all detection + repair tests green). + +- [ ] **Step 5: Commit** + +```bash +git add src/wardline/install/doctor.py tests/unit/install/test_doctor_filigree_auth.py +git commit -m "feat(doctor): repair recovers accepted federation token into .env" +``` + +--- + +## Task 6: wire into machine_readable_doctor + CLI + +**Files:** +- Modify: `src/wardline/install/doctor.py` (`machine_readable_doctor` signature + append check) +- Modify: `src/wardline/cli/doctor.py` (`--filigree-url` option; surface `filigree.auth` in human output; pass through to JSON path) +- Test: `tests/unit/cli/test_doctor.py` (append) + +- [ ] **Step 1: Write the failing test** + +```python +# append to tests/unit/cli/test_doctor.py +def test_doctor_accepts_filigree_url_flag_and_reports_not_configured(tmp_path: Path, monkeypatch) -> None: + # No filigree wiring (no .mcp.json arg, no env, no port) => filigree.auth is ok/not-configured, + # so doctor does no network and the new flag is accepted. + home = tmp_path / "home" + monkeypatch.delenv("WARDLINE_FILIGREE_URL", raising=False) + monkeypatch.delenv("WEFT_FEDERATION_TOKEN", raising=False) + monkeypatch.delenv("WARDLINE_FILIGREE_TOKEN", raising=False) + monkeypatch.setattr("wardline.install.mcp_json.Path.home", lambda: home) + monkeypatch.setattr("wardline.install.mcp_json._find_wardline_command", lambda: "/bin/wardline") + monkeypatch.setattr("wardline.install.detect.shutil.which", lambda _: None) + + repair = CliRunner().invoke(cli, ["doctor", "--root", str(tmp_path), "--repair"]) + assert repair.exit_code == 0, repair.output + + result = CliRunner().invoke(cli, ["doctor", "--root", str(tmp_path)]) + assert result.exit_code == 0, result.output + assert "filigree.auth" in result.output + + +def test_doctor_fix_json_includes_filigree_auth_check(tmp_path: Path, monkeypatch) -> None: + import json as _json + + home = tmp_path / "home" + monkeypatch.delenv("WARDLINE_FILIGREE_URL", raising=False) + monkeypatch.delenv("WEFT_FEDERATION_TOKEN", raising=False) + monkeypatch.delenv("WARDLINE_FILIGREE_TOKEN", raising=False) + monkeypatch.setattr("wardline.install.mcp_json.Path.home", lambda: home) + monkeypatch.setattr("wardline.install.mcp_json._find_wardline_command", lambda: "/bin/wardline") + monkeypatch.setattr("wardline.install.detect.shutil.which", lambda _: None) + + result = CliRunner().invoke(cli, ["doctor", "--root", str(tmp_path), "--fix"]) + payload = _json.loads(result.output) + ids = [c["id"] for c in payload["checks"]] + assert "filigree.auth" in ids +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: `.venv/bin/pytest tests/unit/cli/test_doctor.py -q` +Expected: FAIL — `--filigree-url` is an unknown option / `filigree.auth` not in output. + +- [ ] **Step 3: Write minimal implementation** + +In `src/wardline/install/doctor.py`, update `machine_readable_doctor`: + +```python +def machine_readable_doctor( + root: Path, + *, + fix: bool = False, + filigree_url: str | None = None, + transport: Transport | None = None, +) -> dict[str, Any]: +``` + +and append the new check to the `checks` list (after `_check_auth_token`): + +```python + checks.append(_check_auth_token(root)) + checks.append(_check_filigree_auth(root, repair=fix, filigree_url=filigree_url, transport=transport)) +``` + +In `src/wardline/cli/doctor.py`, add the import and option, and surface the check in the human paths: + +```python +from wardline.install.doctor import ( + _check_filigree_auth, + check_install, + machine_readable_doctor, + repair_install, +) +``` + +Add the option decorator (after `--fix`): + +```python +@click.option("--filigree-url", default=None, help="Filigree Weft URL to probe (default: resolve from .mcp.json/env).") +``` + +Update the signature: `def doctor(root: Path, repair: bool, fix_json: bool, filigree_url: str | None) -> None:` + +In the `--fix` branch, pass it through: + +```python + payload = machine_readable_doctor(root, fix=True, filigree_url=filigree_url) +``` + +In the `--repair` branch, after the install-check loop and before the exit decision, add: + +```python + fcheck = _check_filigree_auth(root, repair=True, filigree_url=filigree_url) + status = ("fixed" if fcheck.fixed else fcheck.message) if fcheck.ok else f"failed ({fcheck.message})" + click.echo(f" filigree.auth: {status}") + if not all(check.ok for check in after) or not fcheck.ok: + raise SystemExit(1) + return +``` + +(Replace the existing `if all(check.ok for check in after): return / raise SystemExit(1)` tail of the `--repair` branch with the block above.) + +In the default branch, surface detection too. Replace the default tail: + +```python + checks = check_install(root) + fcheck = _check_filigree_auth(root, repair=False, filigree_url=filigree_url) + ok = all(check.ok for check in checks) and fcheck.ok + click.echo("wardline doctor: ok" if ok else "wardline doctor:") + for check in checks: + if ok: + continue + click.echo(f" {check.name}: {check.message}") + if not ok: + click.echo(f" filigree.auth: {fcheck.message}" if not fcheck.ok else "") + raise SystemExit(1) +``` + +To always show the `filigree.auth` line (the CLI test asserts its presence even when ok), simplify the default tail to: + +```python + checks = check_install(root) + fcheck = _check_filigree_auth(root, repair=False, filigree_url=filigree_url) + ok = all(check.ok for check in checks) and fcheck.ok + click.echo("wardline doctor: ok" if ok else "wardline doctor:") + for check in checks: + if not check.ok: + click.echo(f" {check.name}: {check.message}") + fmsg = fcheck.message or ("ok" if fcheck.ok else "error") + click.echo(f" filigree.auth: {fmsg}") + if not ok: + raise SystemExit(1) +``` + +- [ ] **Step 4: Run test to verify it passes** + +Run: `.venv/bin/pytest tests/unit/cli/test_doctor.py -q` +Expected: PASS. The existing `test_doctor_fix_emits_shared_machine_readable_shape` must stay green: with the published-port rung excluded from probe resolution and no `--filigree-url` arg in `_local_mcp_entry()`, `filigree.auth` resolves no URL → `ok`/"not configured" → no network, so `payload["ok"]` stays `True` and `next_actions` stays `[]`. + +- [ ] **Step 5: Commit** + +```bash +git add src/wardline/install/doctor.py src/wardline/cli/doctor.py tests/unit/cli/test_doctor.py +git commit -m "feat(doctor): wire filigree.auth check into machine-readable + CLI surfaces" +``` + +--- + +## Task 7: full-suite gate + changelog + +**Files:** +- Modify: `CHANGELOG.md` (under `[Unreleased] / Added`) +- Test: full suite + linters + +- [ ] **Step 1: Run the whole suite + linters** + +Run: +```bash +.venv/bin/pytest -q +.venv/bin/ruff check src tests +.venv/bin/mypy src +``` +Expected: all green. Fix any failures before continuing (no "pre-existing" excuse — red is red). + +- [ ] **Step 2: Add the changelog entry** + +In `CHANGELOG.md`, under `## [Unreleased]` → `### Added`: + +```markdown +- `wardline doctor` now verifies the Filigree federation token: it probes the + configured daemon (URL resolved from `.mcp.json`/env) with the token wardline + would emit and reports a `filigree.auth` check. `--repair` recovers the + daemon-accepted token from local mints and pins it as `WEFT_FEDERATION_TOKEN` + in `.env`, removing a stale `WARDLINE_FILIGREE_TOKEN` line. +``` + +- [ ] **Step 3: Commit** + +```bash +git add CHANGELOG.md +git commit -m "docs(changelog): note doctor filigree.auth check" +``` + +- [ ] **Step 4: Live acceptance against lacuna (manual, optional but recommended)** + +Run a real probe against the running daemon to confirm end-to-end (the spec's oracle): +```bash +wardline doctor --root ~/lacuna --repair +``` +Expected: `filigree.auth: fixed` (writes `WEFT_FEDERATION_TOKEN` into `~/lacuna/.env`), then `wardline scan` from lacuna emits successfully and a finding lands in Filigree. Do not claim done on unit tests alone — confirm the emit block flips to success. + +--- + +## Self-Review Notes + +- **Spec coverage:** probe-URL-from-`.mcp.json` (Task 3), sentinel-`{}` probe not `emit([])` (Task 1), detection table incl. absent/non-loopback/unreachable (Task 4), repair-from-local-mints + no-candidate guard (Task 5), `.env`-only write leaving the MCP bearer alone (Tasks 2/5; no `.mcp.json` write anywhere), loopback discipline (Tasks 3/4), CLI surface (Task 6). All covered. +- **Type consistency:** `ProbeResult(reachable, accepted, status)`, `DoctorCheck(id, status, fixed, message)`, `Response(status, body)`, `_check_filigree_auth(root, *, repair, filigree_url=None, transport=None)`, `_repair_filigree_auth(root, url, transport)` — names match across tasks. +- **Note for executor:** Task 4 introduces a temporary `_repair_filigree_auth` stub that Task 5 replaces — if executing out of order, do Task 5 before relying on repair. diff --git a/docs/superpowers/specs/2026-06-07-wardline-doctor-filigree-token-check-design.md b/docs/superpowers/specs/2026-06-07-wardline-doctor-filigree-token-check-design.md index 8738346e..b682e1a5 100644 --- a/docs/superpowers/specs/2026-06-07-wardline-doctor-filigree-token-check-design.md +++ b/docs/superpowers/specs/2026-06-07-wardline-doctor-filigree-token-check-design.md @@ -85,24 +85,34 @@ probe URL therefore resolves by: 2. `WARDLINE_FILIGREE_URL` env var 3. **`.mcp.json` → `mcpServers.wardline.args` → value after `--filigree-url`** (doctor already parses `.mcp.json` for `_check_project_mcp`) -4. published-port rung (`.weft/filigree/ephemeral.port`, legacy - `.filigree/ephemeral.port`) If none resolve → `ok`, message `"filigree not configured; nothing to verify"`. +The published-port rung (`.weft/filigree/ephemeral.port`) is **deliberately +excluded**: it is a dynamic discovery convenience for live `scan`, not a +configured emit target. doctor probes only a deliberately-configured URL, so it +performs **no network I/O unless filigree emit is actually wired** — which also +keeps the existing doctor test suite network-free. A CLI-only operator relying on +the port rung can pass `--filigree-url` or set `WARDLINE_FILIGREE_URL`. + #### Token `load_filigree_token(root)` — exactly the value emit would send. #### Detection (read-only; runs without `--repair`) +The token (`load_filigree_token(root)`, possibly `None`) is **always probed** — an +absent token is not inherently an error, because the daemon may have auth off and +accept unauthenticated emits. + | Condition | Result | |---|---| -| URL resolves, token is `None` | `error` — `"no federation token set; export WEFT_FEDERATION_TOKEN or add it to .env"` | +| URL does not resolve | `ok` — `"filigree not configured; nothing to verify"` | | Resolved URL is **non-loopback** | `ok` — `"non-loopback filigree; token not probed"` (never send a bearer off-box) | -| Probe → `401`/`403` | `error` — `"emit token rejected by filigree (); the configured token is not what the daemon accepts"` | | Probe → unreachable (conn refused / timeout) | `ok` — `"filigree daemon not reachable; token not verified"` | -| Probe → any other status (e.g. `400`, `2xx`) | `ok` | +| Probe → accepted (any non-`401/403`, e.g. `400`/`2xx`) | `ok` | +| Probe → `401`/`403`, **token present** | `error` — `"emit token rejected by filigree (); the configured token is not what the daemon accepts"` | +| Probe → `401`/`403`, **no token set** | `error` — `"filigree rejected an unauthenticated emit and no federation token is set; export WEFT_FEDERATION_TOKEN or add it to .env"` | **Probe mechanism:** `POST ` with a **sentinel body `{}`** and the bearer, ~2 s timeout. Filigree's auth middleware runs *before* body validation, so a good From 28b6fbfadcef944147dc3ab3aac22df467445c18 Mon Sep 17 00:00:00 2001 From: John Morrissey Date: Mon, 8 Jun 2026 00:09:13 +1000 Subject: [PATCH 005/186] feat(emit): add FiligreeEmitter.verify_token auth probe A read-only probe that POSTs a sentinel body ({}) so the daemon's auth middleware runs before body validation: a good token yields 400 (nothing recorded), 401/403 means rejected. Returns ProbeResult(reachable, accepted, status). Does not reuse emit() (which would register an empty scan). Co-Authored-By: Claude Opus 4.8 --- src/wardline/core/filigree_emit.py | 27 +++++++++ tests/unit/core/test_filigree_verify_token.py | 57 +++++++++++++++++++ 2 files changed, 84 insertions(+) create mode 100644 tests/unit/core/test_filigree_verify_token.py diff --git a/src/wardline/core/filigree_emit.py b/src/wardline/core/filigree_emit.py index 3da09687..0f591f0f 100644 --- a/src/wardline/core/filigree_emit.py +++ b/src/wardline/core/filigree_emit.py @@ -122,6 +122,17 @@ def __post_init__(self) -> None: raise ValueError("an unreachable EmitResult must have zero created/updated/failed") +@dataclass(frozen=True, slots=True) +class ProbeResult: + """Outcome of an auth probe (verify_token). ``accepted`` is True when the daemon + authenticated the bearer (any non-401/403 status, e.g. a 400 from the sentinel body). + ``reachable`` is False only on a transport failure (connection refused / timeout).""" + + reachable: bool + accepted: bool + status: int | None = None + + def filigree_disabled_reason(*, reachable: bool, status: int | None) -> str | None: """The ``disabled_reason`` for an emit attempt, or None when Filigree was reached. @@ -230,3 +241,19 @@ def emit(self, findings: Sequence[Finding], *, scanned_paths: Sequence[str] = () failed=len(failed) if isinstance(failed, list) else 0, warnings=tuple(warnings), ) + + def verify_token(self) -> ProbeResult: + """Probe whether the daemon accepts this emitter's bearer token, WITHOUT + recording anything. Auth runs in middleware before body validation, so a + deliberately-incomplete sentinel body yields 400 (auth passed) or 401/403 + (rejected). Never reuses emit() — that would POST a valid empty scan.""" + body = b"{}" # parses as JSON, missing required scan-results fields => 400 when authed + headers = {"Content-Type": "application/json"} + if self._token: + headers["Authorization"] = f"Bearer {self._token}" + try: + resp = self._transport.post(self._url, body, headers) + except (urllib.error.URLError, OSError): + return ProbeResult(reachable=False, accepted=False) + accepted = resp.status not in (401, 403) + return ProbeResult(reachable=True, accepted=accepted, status=resp.status) diff --git a/tests/unit/core/test_filigree_verify_token.py b/tests/unit/core/test_filigree_verify_token.py new file mode 100644 index 00000000..2ee3916e --- /dev/null +++ b/tests/unit/core/test_filigree_verify_token.py @@ -0,0 +1,57 @@ +# tests/unit/core/test_filigree_verify_token.py +from collections.abc import Mapping + +from wardline.core.filigree_emit import FiligreeEmitter, Response + + +class _FakeTransport: + """Records the last POST and returns a canned Response or raises.""" + + def __init__(self, *, status: int | None = None, exc: Exception | None = None) -> None: + self._status = status + self._exc = exc + self.calls: list[tuple[str, bytes, dict[str, str]]] = [] + + def post(self, url: str, body: bytes, headers: Mapping[str, str]) -> Response: + self.calls.append((url, body, dict(headers))) + if self._exc is not None: + raise self._exc + assert self._status is not None + return Response(status=self._status, body="") + + +def test_verify_token_401_is_rejected() -> None: + t = _FakeTransport(status=401) + result = FiligreeEmitter("http://127.0.0.1:8749/api/weft/scan-results", transport=t, token="bad").verify_token() + assert result.reachable is True + assert result.accepted is False + assert result.status == 401 + + +def test_verify_token_400_is_accepted() -> None: + # Auth middleware runs before body validation: a good token + sentinel body => 400. + t = _FakeTransport(status=400) + result = FiligreeEmitter("http://127.0.0.1:8749/api/weft/scan-results", transport=t, token="good").verify_token() + assert result.accepted is True + assert result.status == 400 + # Sentinel body present; bearer attached. The body MUST be the deliberately-incomplete + # b"{}" sentinel, NOT a real scan-results body — a regression to emit()-style content + # (build_scan_results_body([]) is also truthy) would register an empty scan on the + # daemon every doctor probe. Pin the exact content so that regression fails here. + url, body, headers = t.calls[0] + assert headers["Authorization"] == "Bearer good" + assert body == b"{}" # sentinel only; no findings/scan_source body + assert b"findings" not in body and b"scan_source" not in body + + +def test_verify_token_403_is_rejected() -> None: + result = FiligreeEmitter("http://x/y", transport=_FakeTransport(status=403), token="t").verify_token() + assert result.accepted is False + + +def test_verify_token_transport_error_is_unreachable() -> None: + t = _FakeTransport(exc=OSError("connection refused")) + result = FiligreeEmitter("http://127.0.0.1:8749/api/weft/scan-results", transport=t, token="t").verify_token() + assert result.reachable is False + assert result.accepted is False + assert result.status is None From ca08a8e767dab81edce37fb5c9df3daead798bd0 Mon Sep 17 00:00:00 2001 From: John Morrissey Date: Mon, 8 Jun 2026 00:09:13 +1000 Subject: [PATCH 006/186] fix(install): preserve operator-pinned --filigree-url/--loomweave-url on repair merge_mcp_entry normalized the wardline .mcp.json entry to the canonical local form, stripping a fixed-port/server-mode --filigree-url the published-port rung cannot reconstruct -- silently disabling runtime emit. Preserve those flag pairs in the operator's original order (idempotent); _check_project_mcp accepts them. Co-Authored-By: Claude Opus 4.8 --- src/wardline/install/mcp_json.py | 52 +++++++++++++++++++++++++++-- tests/unit/install/test_mcp_json.py | 47 ++++++++++++++++++++++++++ 2 files changed, 96 insertions(+), 3 deletions(-) diff --git a/src/wardline/install/mcp_json.py b/src/wardline/install/mcp_json.py index 65a713cb..a66e4edf 100644 --- a/src/wardline/install/mcp_json.py +++ b/src/wardline/install/mcp_json.py @@ -35,12 +35,57 @@ def _local_mcp_entry() -> dict[str, object]: return {"type": "stdio", "command": _find_wardline_command(), "args": ["mcp", "--root", "."]} +# Operator-pinned sibling-endpoint flags. When an existing .mcp.json entry carries +# these (e.g. a fixed-port / server-mode filigree whose URL the published-port rung +# cannot reconstruct), they ARE the runtime emit/discovery target — repair must keep +# them, not normalize them away. +_PRESERVED_ARG_FLAGS = ("--filigree-url", "--loomweave-url") + + +def _preserved_sibling_args(entry: object) -> list[str]: + """Extract operator-pinned ``--filigree-url``/``--loomweave-url`` flag pairs from an + existing wardline entry's args, in the order the operator wrote them. Returns ``[]`` + for any shape that isn't a list-of-args (a malformed entry contributes nothing to + preserve). Original order is kept so an already-correct entry is recognized as + ``unchanged`` and is never needlessly reordered on repair.""" + if not isinstance(entry, dict): + return [] + args = entry.get("args") + if not isinstance(args, list): + return [] + preserved: list[str] = [] + i = 0 + while i < len(args): + flag = args[i] + if flag in _PRESERVED_ARG_FLAGS and i + 1 < len(args) and isinstance(args[i + 1], str): + preserved.extend((flag, args[i + 1])) + i += 2 + continue + i += 1 + return preserved + + +def _desired_local_entry(existing: object) -> dict[str, object]: + """The canonical local entry, augmented with any operator-pinned sibling-URL args + carried by *existing*. Idempotent: re-running over the desired entry reproduces it.""" + entry = _local_mcp_entry() + preserved = _preserved_sibling_args(existing) + if preserved: + base_args = entry["args"] + assert isinstance(base_args, list) + entry["args"] = [*base_args, *preserved] + return entry + + def merge_mcp_entry(root: Path) -> str: - """Add/replace the `wardline` entry under mcpServers. Returns created|updated|unchanged.""" + """Add/replace the `wardline` entry under mcpServers. Returns created|updated|unchanged. + + An existing entry's operator-pinned ``--filigree-url``/``--loomweave-url`` args are + preserved (they are the runtime emit/discovery target when the published-port rung + cannot reconstruct it).""" path = safe_project_file(root, root / ".mcp.json", label=".mcp.json") - entry = _local_mcp_entry() if not path.exists(): - payload = {"mcpServers": {"wardline": entry}} + payload = {"mcpServers": {"wardline": _local_mcp_entry()}} path.write_text(json.dumps(payload, indent=2) + "\n", encoding="utf-8") return "created" try: @@ -55,6 +100,7 @@ def merge_mcp_entry(root: Path) -> str: data["mcpServers"] = servers if not isinstance(servers, dict): raise WardlineError(".mcp.json mcpServers must be an object") + entry = _desired_local_entry(servers.get("wardline")) if servers.get("wardline") == entry: return "unchanged" servers["wardline"] = entry diff --git a/tests/unit/install/test_mcp_json.py b/tests/unit/install/test_mcp_json.py index c1dbd385..b52d42d5 100644 --- a/tests/unit/install/test_mcp_json.py +++ b/tests/unit/install/test_mcp_json.py @@ -68,6 +68,53 @@ def test_mcpservers_null_is_treated_as_absent(tmp_path: Path, monkeypatch: pytes assert data["other"] == 1 # unrelated top-level keys preserved +def test_preserves_operator_pinned_sibling_url_args(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None: + # A fixed-port / server-mode emit target (e.g. lacuna) pins --filigree-url / + # --loomweave-url in the wardline entry's args. The published-port rung cannot + # reconstruct such a URL, so these args ARE the runtime emit/discovery target. + # merge_mcp_entry must keep them rather than normalizing to the bare canonical entry. + monkeypatch.setattr("wardline.install.mcp_json._find_wardline_command", lambda: "/bin/wardline") + (tmp_path / ".mcp.json").write_text( + json.dumps( + {"mcpServers": {"wardline": {"type": "stdio", "command": "OLD", "args": [ + "mcp", "--root", ".", + "--loomweave-url", "http://127.0.0.1:9730", + "--filigree-url", "http://127.0.0.1:8749/api/p/lacuna/weft/scan-results", + ]}}} + ), + encoding="utf-8", + ) + # command is refreshed to canonical (OLD -> /bin/wardline), but the pinned + # sibling-URL args survive in the operator's ORIGINAL order (loomweave-first here). + assert merge_mcp_entry(tmp_path) == "updated" + entry = json.loads((tmp_path / ".mcp.json").read_text(encoding="utf-8"))["mcpServers"]["wardline"] + assert entry["command"] == "/bin/wardline" + assert entry["args"] == [ + "mcp", "--root", ".", + "--loomweave-url", "http://127.0.0.1:9730", + "--filigree-url", "http://127.0.0.1:8749/api/p/lacuna/weft/scan-results", + ] + # idempotent: re-running over the already-preserved entry is a no-op (no reorder churn). + assert merge_mcp_entry(tmp_path) == "unchanged" + + +def test_already_canonical_lacuna_entry_is_unchanged(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None: + # The real lacuna ordering (loomweave-first) with the canonical command must be a + # no-op for merge_mcp_entry — no spurious reorder churn on every repair. + monkeypatch.setattr("wardline.install.mcp_json._find_wardline_command", lambda: "/bin/wardline") + (tmp_path / ".mcp.json").write_text( + json.dumps( + {"mcpServers": {"wardline": {"type": "stdio", "command": "/bin/wardline", "args": [ + "mcp", "--root", ".", + "--loomweave-url", "http://127.0.0.1:9730", + "--filigree-url", "http://127.0.0.1:8749/api/p/lacuna/weft/scan-results", + ]}}} + ), + encoding="utf-8", + ) + assert merge_mcp_entry(tmp_path) == "unchanged" + + def test_replaces_stale_wardline_entry(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None: monkeypatch.setattr("wardline.install.mcp_json._find_wardline_command", lambda: "/bin/wardline") (tmp_path / ".mcp.json").write_text( From 3cf87c380b9ef9a8a87f2b3eca7c882f3b07b926 Mon Sep 17 00:00:00 2001 From: John Morrissey Date: Mon, 8 Jun 2026 00:09:13 +1000 Subject: [PATCH 007/186] feat(doctor): add filigree.auth federation-token check + repair wardline doctor probes the configured Filigree daemon (URL resolved flag > env > .mcp.json --filigree-url arg; loopback-only via strict ipaddress parsing) with the token wardline would emit. Reports filigree.auth; under --repair, recovers the daemon-accepted token from local mints and pins it as WEFT_FEDERATION_TOKEN in .env (surgical, byte-preserving, 0600 from creation). Distinguishes token absent from token-present-but-rejected. Wired into machine_readable_doctor and the CLI (--filigree-url flag, human + JSON + --repair surfaces). Co-Authored-By: Claude Opus 4.8 --- CHANGELOG.md | 12 + src/wardline/cli/doctor.py | 37 +- src/wardline/install/doctor.py | 181 +++++++++- tests/unit/cli/test_doctor.py | 36 ++ .../unit/install/test_doctor_filigree_auth.py | 338 ++++++++++++++++++ 5 files changed, 591 insertions(+), 13 deletions(-) create mode 100644 tests/unit/install/test_doctor_filigree_auth.py diff --git a/CHANGELOG.md b/CHANGELOG.md index 2993d617..a9ca030d 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,6 +7,13 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] +### Added +- `wardline doctor` now verifies the Filigree federation token: it probes the + configured daemon (URL resolved from `.mcp.json`/env) with the token wardline + would emit and reports a `filigree.auth` check. `--repair` recovers the + daemon-accepted token from local mints and pins it as `WEFT_FEDERATION_TOKEN` + in `.env`, removing a stale `WARDLINE_FILIGREE_TOKEN` line. + ### Changed - **BREAKING: Weft config/store consolidation.** Operator config moved from `wardline.yaml` (YAML) to the `[wardline]` table of a shared, operator-authored @@ -43,6 +50,11 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 is honored as a **deprecated fallback** (read after the new name), so existing deployments keep working with no change; migrate at leisure. Only the token *value* must match what the Filigree operator configured. +- **`wardline doctor --repair`/`install` now preserves operator-pinned + `--filigree-url`/`--loomweave-url` args** in the `.mcp.json` wardline server entry + (in the order the operator wrote them) instead of normalizing them away. Previously + every repair rewrote the entry to the bare canonical args, silently stripping a + fixed-port sibling emit/discovery target the published-port rung cannot reconstruct. ### Fixed - **Explicit `--config` pointing at a malformed (but existing) `weft.toml` no longer diff --git a/src/wardline/cli/doctor.py b/src/wardline/cli/doctor.py index 94dae2c4..2b031a93 100644 --- a/src/wardline/cli/doctor.py +++ b/src/wardline/cli/doctor.py @@ -8,7 +8,13 @@ import click from wardline.core.errors import WardlineError -from wardline.install.doctor import check_install, machine_readable_doctor, repair_install +from wardline.install.doctor import ( + _check_filigree_auth, + _resolve_probe_url, + check_install, + machine_readable_doctor, + repair_install, +) @click.command() @@ -20,14 +26,17 @@ ) @click.option("--repair", is_flag=True, help="Repair missing or stale install artifacts.") @click.option("--fix", "fix_json", is_flag=True, help="Repair install bindings and emit machine-readable JSON.") -def doctor(root: Path, repair: bool, fix_json: bool) -> None: +@click.option( + "--filigree-url", default=None, help="Filigree Weft URL to probe (default: resolve from .mcp.json/env)." +) +def doctor(root: Path, repair: bool, fix_json: bool, filigree_url: str | None) -> None: """Check Wardline agent install artifacts and sibling bindings.""" if repair and fix_json: click.echo("error: use only one of --repair or --fix", err=True) raise SystemExit(2) if fix_json: try: - payload = machine_readable_doctor(root, fix=True) + payload = machine_readable_doctor(root, fix=True, filigree_url=filigree_url) except WardlineError as exc: click.echo(f"error: {exc}", err=True) raise SystemExit(2) from exc @@ -37,6 +46,9 @@ def doctor(root: Path, repair: bool, fix_json: bool) -> None: raise SystemExit(1) if repair: + # Resolve the probe URL BEFORE repair_install rewrites .mcp.json (which would + # erase a configured --filigree-url arg), so repair can still probe/recover. + probe_url = _resolve_probe_url(root, filigree_url) try: statuses = repair_install(root) except WardlineError as exc: @@ -47,16 +59,21 @@ def doctor(root: Path, repair: bool, fix_json: bool) -> None: for check in after: status = statuses.get(check.name, "checked") if check.ok else f"failed ({check.message})" click.echo(f" {check.name}: {status}") - if all(check.ok for check in after): - return - raise SystemExit(1) + fcheck = _check_filigree_auth(root, repair=True, filigree_url=probe_url) + fstatus = ("fixed" if fcheck.fixed else fcheck.message) if fcheck.ok else f"failed ({fcheck.message})" + click.echo(f" filigree.auth: {fstatus}") + if not all(check.ok for check in after) or not fcheck.ok: + raise SystemExit(1) + return checks = check_install(root) - ok = all(check.ok for check in checks) + fcheck = _check_filigree_auth(root, repair=False, filigree_url=filigree_url) + ok = all(check.ok for check in checks) and fcheck.ok click.echo("wardline doctor: ok" if ok else "wardline doctor:") for check in checks: - if ok: - continue - click.echo(f" {check.name}: {check.message}") + if not check.ok: + click.echo(f" {check.name}: {check.message}") + fmsg = fcheck.message or ("ok" if fcheck.ok else "error") + click.echo(f" filigree.auth: {fmsg}") if not ok: raise SystemExit(1) diff --git a/src/wardline/install/doctor.py b/src/wardline/install/doctor.py index b9bc44cd..45cccb46 100644 --- a/src/wardline/install/doctor.py +++ b/src/wardline/install/doctor.py @@ -2,6 +2,7 @@ from __future__ import annotations +import ipaddress import json import os import tomllib @@ -13,7 +14,9 @@ from wardline.core.config import load from wardline.core.errors import ConfigError +from wardline.core.filigree_emit import FiligreeEmitter, Transport, UrllibTransport from wardline.core.paths import weft_config_path, weft_state_dir +from wardline.filigree.config import load_filigree_token from wardline.install.block import inject_block from wardline.install.detect import ( _detect_filigree, @@ -23,7 +26,7 @@ from wardline.install.mcp_json import ( _codex_config_path, _codex_mcp_entry, - _local_mcp_entry, + _desired_local_entry, install_codex_mcp, merge_mcp_entry, ) @@ -80,7 +83,10 @@ def _check_project_mcp(root: Path) -> CheckResult: if not isinstance(servers, dict): return CheckResult(".mcp.json", False, "missing mcpServers object") entry = servers.get("wardline") - if entry != _local_mcp_entry(): + # An entry carrying operator-pinned --filigree-url/--loomweave-url args is well-formed: + # compare against the canonical entry augmented with those preserved args, not the bare + # canonical entry (which would flag a configured emit target as misconfiguration). + if entry != _desired_local_entry(entry): return CheckResult(".mcp.json", False, "missing wardline server") return CheckResult(".mcp.json", True, "configured") @@ -216,10 +222,178 @@ def _check_auth_token(root: Path) -> DoctorCheck: return DoctorCheck("auth.token", "ok", message="optional Loomweave token not configured") -def machine_readable_doctor(root: Path, *, fix: bool = False) -> dict[str, Any]: +def _rewrite_env_token(env_path: Path, value: str) -> None: + """Surgically pin ``WEFT_FEDERATION_TOKEN=`` in *env_path*. Removes any + existing ``WEFT_FEDERATION_TOKEN`` or legacy ``WARDLINE_FILIGREE_TOKEN`` line, + preserves all other lines/order byte-for-byte, creates the file if absent, and + keeps mode 0600 (the file holds a secret). + + Operates on raw bytes: an unrelated line carrying a non-UTF8 byte (e.g. a sibling + secret) is preserved verbatim rather than corrupted to U+FFFD on a decode round-trip. + The file is created with mode 0600 from the outset (``os.open`` with O_CREAT), so the + secret is never briefly written to a world-readable file; the trailing ``chmod`` still + tightens an already-existing loose-permission file.""" + drop = (b"WEFT_FEDERATION_TOKEN=", b"WARDLINE_FILIGREE_TOKEN=") + kept: list[bytes] = [] + if env_path.is_file(): + for raw in env_path.read_bytes().splitlines(): + if raw.strip().startswith(drop): + continue + kept.append(raw) + kept.append(b"WEFT_FEDERATION_TOKEN=" + value.encode("utf-8")) + payload = b"\n".join(kept) + b"\n" + fd = os.open(env_path, os.O_WRONLY | os.O_CREAT | os.O_TRUNC, 0o600) + with os.fdopen(fd, "wb") as fh: + fh.write(payload) + env_path.chmod(0o600) + + +_FILIGREE_URL_ENV = "WARDLINE_FILIGREE_URL" + + +def _mcp_filigree_url(root: Path) -> str | None: + """The ``--filigree-url`` value from the wardline server entry in ``.mcp.json``, + or None. This is the URL the agent's MCP server actually emits to, and the only + place it is recorded in the common (MCP) setup.""" + path = root / ".mcp.json" + if not path.is_file(): + return None + try: + raw = json.loads(path.read_text(encoding="utf-8")) + except (json.JSONDecodeError, OSError): + return None + try: + args = raw["mcpServers"]["wardline"]["args"] + if not isinstance(args, list): + return None + idx = args.index("--filigree-url") + value = args[idx + 1] + except (KeyError, TypeError, ValueError, IndexError): + return None + return value if isinstance(value, str) else None + + +def _resolve_probe_url(root: Path, flag: str | None) -> str | None: + """Probe-URL precedence: flag > WARDLINE_FILIGREE_URL env > .mcp.json wardline + --filigree-url arg. None when nothing resolves. The published-port rung is + deliberately excluded: doctor probes only a configured emit target, so it does + no network unless filigree emit is explicitly wired.""" + if flag: + return flag + env = os.environ.get(_FILIGREE_URL_ENV) + if env: + return env + return _mcp_filigree_url(root) + + +def _is_loopback(url: str) -> bool: + """True when *url*'s host is loopback — the only origins a bearer is probed against. + + Uses strict IP parsing, never a string-prefix test: ``127.attacker.com`` / + ``127.0.0.1.evil.com`` are registrable hostnames that resolve off-box via DNS, so + a ``startswith("127.")`` check would send the federation bearer to an attacker. Only + the literal ``localhost`` and addresses in the 127/8 + ``::1`` loopback ranges pass.""" + host = (urlsplit(url).hostname or "").lower() + if host == "localhost": + return True + try: + return ipaddress.ip_address(host).is_loopback + except ValueError: + return False + + +def _filigree_token_candidates(root: Path) -> list[str]: + """Locally-readable federation-token mints, in precedence order: the server-mode + store (~/.config/filigree) then the project store (/.weft/filigree). Returns + distinct, non-empty values.""" + paths = [ + Path.home() / ".config" / "filigree" / "federation_token", + root / ".weft" / "filigree" / "federation_token", + ] + out: list[str] = [] + for p in paths: + try: + value = p.read_text(encoding="utf-8").strip() + except OSError: + continue + if value and value not in out: + out.append(value) + return out + + +def _repair_filigree_auth(root: Path, url: str, transport: Transport) -> DoctorCheck: + """A 401/403 was seen. Probe each locally-readable mint; if exactly one is + accepted, pin it as WEFT_FEDERATION_TOKEN in .env and confirm. Otherwise write + nothing and report (the daemon likely uses an env override we cannot read).""" + for candidate in _filigree_token_candidates(root): + probe = FiligreeEmitter(url, transport=transport, token=candidate).verify_token() + if probe.reachable and probe.accepted: + _rewrite_env_token(root / ".env", candidate) + return DoctorCheck( + "filigree.auth", "ok", fixed=True, + message="wrote WEFT_FEDERATION_TOKEN to .env (was a stale/mismatched token)", + ) + return DoctorCheck( + "filigree.auth", "error", + message="no local federation_token matched the daemon — it likely uses a " + "WEFT_FEDERATION_TOKEN env override; set that same value in .env", + ) + + +def _check_filigree_auth( + root: Path, + *, + repair: bool, + filigree_url: str | None = None, + transport: Transport | None = None, +) -> DoctorCheck: + """Verify the token wardline would emit is accepted by the configured Filigree + daemon. Read-only probe; under *repair*, recover the accepted token from local + mints and pin it in .env. The probe targets only loopback origins.""" + probe_transport = transport if transport is not None else UrllibTransport(timeout=2.0) + url = _resolve_probe_url(root, filigree_url) + if url is None: + return DoctorCheck("filigree.auth", "ok", message="filigree not configured; nothing to verify") + if not _is_loopback(url): + return DoctorCheck("filigree.auth", "ok", message="non-loopback filigree; token not probed") + token = load_filigree_token(root) # may be None — probe anyway (the daemon may have auth off) + probe = FiligreeEmitter(url, transport=probe_transport, token=token).verify_token() + if not probe.reachable: + return DoctorCheck("filigree.auth", "ok", message="filigree daemon not reachable; token not verified") + if probe.accepted: + return DoctorCheck("filigree.auth", "ok") + # Rejected (401/403): filigree auth is on and our credential is not accepted. + if repair: + return _repair_filigree_auth(root, url, probe_transport) + if token: + return DoctorCheck( + "filigree.auth", "error", + message=f"emit token rejected by filigree ({probe.status}); " + "the configured token is not what the daemon accepts", + ) + return DoctorCheck( + "filigree.auth", "error", + message="filigree rejected an unauthenticated emit and no federation token is set; " + "export WEFT_FEDERATION_TOKEN or add it to .env", + ) + + +def machine_readable_doctor( + root: Path, + *, + fix: bool = False, + filigree_url: str | None = None, + transport: Transport | None = None, +) -> dict[str, Any]: """Return the shared machine-readable doctor shape, optionally repairing install bindings.""" before = {check.name: check for check in check_install(root)} bindings_fixed = False + # Resolve the probe URL before repair_install as belt-and-suspenders. merge_mcp_entry + # now PRESERVES an operator-pinned --filigree-url arg (it no longer normalizes sibling + # URLs away), so resolution is order-independent: resolving before vs after repair + # yields the same value. We resolve early so a future change to repair can't surprise + # the probe target. + probe_url = _resolve_probe_url(root, filigree_url) if fix: repair_install(root) bindings_fixed = not before.get("bindings", CheckResult("bindings", True, "")).ok @@ -233,6 +407,7 @@ def machine_readable_doctor(root: Path, *, fix: bool = False) -> dict[str, Any]: checks.append(_check_decorator_grammar()) checks.append(_check_scan_output_path(root)) checks.append(_check_auth_token(root)) + checks.append(_check_filigree_auth(root, repair=fix, filigree_url=probe_url, transport=transport)) next_actions = [f"{check.id}: {check.message}" for check in checks if not check.ok and check.message] return { diff --git a/tests/unit/cli/test_doctor.py b/tests/unit/cli/test_doctor.py index 17e03e25..be5f5ed2 100644 --- a/tests/unit/cli/test_doctor.py +++ b/tests/unit/cli/test_doctor.py @@ -142,3 +142,39 @@ def test_doctor_fix_reports_filigree_url_ok_from_env(tmp_path: Path, monkeypatch checks = {check["id"]: check for check in payload["checks"]} assert checks["filigree.url"]["status"] == "ok" assert not (tmp_path / "weft.toml").exists() + + +def test_doctor_accepts_filigree_url_flag_and_reports_not_configured(tmp_path: Path, monkeypatch) -> None: + # No filigree wiring (no .mcp.json arg, no env, no port) => filigree.auth is ok/not-configured, + # so doctor does no network and the new flag is accepted. + home = tmp_path / "home" + monkeypatch.delenv("WARDLINE_FILIGREE_URL", raising=False) + monkeypatch.delenv("WEFT_FEDERATION_TOKEN", raising=False) + monkeypatch.delenv("WARDLINE_FILIGREE_TOKEN", raising=False) + monkeypatch.setattr("wardline.install.mcp_json.Path.home", lambda: home) + monkeypatch.setattr("wardline.install.mcp_json._find_wardline_command", lambda: "/bin/wardline") + monkeypatch.setattr("wardline.install.detect.shutil.which", lambda _: None) + + repair = CliRunner().invoke(cli, ["doctor", "--root", str(tmp_path), "--repair"]) + assert repair.exit_code == 0, repair.output + + result = CliRunner().invoke(cli, ["doctor", "--root", str(tmp_path)]) + assert result.exit_code == 0, result.output + assert "filigree.auth" in result.output + + +def test_doctor_fix_json_includes_filigree_auth_check(tmp_path: Path, monkeypatch) -> None: + import json as _json + + home = tmp_path / "home" + monkeypatch.delenv("WARDLINE_FILIGREE_URL", raising=False) + monkeypatch.delenv("WEFT_FEDERATION_TOKEN", raising=False) + monkeypatch.delenv("WARDLINE_FILIGREE_TOKEN", raising=False) + monkeypatch.setattr("wardline.install.mcp_json.Path.home", lambda: home) + monkeypatch.setattr("wardline.install.mcp_json._find_wardline_command", lambda: "/bin/wardline") + monkeypatch.setattr("wardline.install.detect.shutil.which", lambda _: None) + + result = CliRunner().invoke(cli, ["doctor", "--root", str(tmp_path), "--fix"]) + payload = _json.loads(result.output) + ids = [c["id"] for c in payload["checks"]] + assert "filigree.auth" in ids diff --git a/tests/unit/install/test_doctor_filigree_auth.py b/tests/unit/install/test_doctor_filigree_auth.py new file mode 100644 index 00000000..92cf9163 --- /dev/null +++ b/tests/unit/install/test_doctor_filigree_auth.py @@ -0,0 +1,338 @@ +# tests/unit/install/test_doctor_filigree_auth.py +import json +import stat +from collections.abc import Mapping +from pathlib import Path + +from wardline.core.filigree_emit import Response +from wardline.install.doctor import ( + _check_filigree_auth, + _check_project_mcp, + _is_loopback, + _mcp_filigree_url, + _resolve_probe_url, + _rewrite_env_token, + machine_readable_doctor, +) + + +def test_rewrite_env_sets_new_name_and_drops_legacy(tmp_path: Path) -> None: + env = tmp_path / ".env" + env.write_text( + "WARDLINE_ATTEST_KEY=keep-me\nWARDLINE_FILIGREE_TOKEN=stale\nOTHER=x\n", + encoding="utf-8", + ) + _rewrite_env_token(env, "GOODTOKEN") + text = env.read_text(encoding="utf-8") + assert "WEFT_FEDERATION_TOKEN=GOODTOKEN" in text + assert "WARDLINE_FILIGREE_TOKEN" not in text # stale legacy line removed + assert "WARDLINE_ATTEST_KEY=keep-me" in text # unrelated line preserved + assert "OTHER=x" in text + assert stat.S_IMODE(env.stat().st_mode) == 0o600 + + +def test_rewrite_env_updates_existing_new_name(tmp_path: Path) -> None: + env = tmp_path / ".env" + env.write_text("WEFT_FEDERATION_TOKEN=old\nKEEP=1\n", encoding="utf-8") + _rewrite_env_token(env, "NEW") + text = env.read_text(encoding="utf-8") + assert text.count("WEFT_FEDERATION_TOKEN=") == 1 + assert "WEFT_FEDERATION_TOKEN=NEW" in text + assert "KEEP=1" in text + + +def test_rewrite_env_creates_file_when_absent(tmp_path: Path) -> None: + env = tmp_path / ".env" + _rewrite_env_token(env, "NEW") + assert env.read_text(encoding="utf-8").strip() == "WEFT_FEDERATION_TOKEN=NEW" + assert stat.S_IMODE(env.stat().st_mode) == 0o600 + + +def test_rewrite_env_preserves_non_utf8_unrelated_line(tmp_path: Path) -> None: + # An unrelated line carrying a raw non-UTF8 byte (0xE9, Latin-1 'é' in another + # secret) must survive byte-for-byte. A decode round-trip with errors="replace" + # would corrupt it to U+FFFD; the bytes-based rewrite preserves it. + env = tmp_path / ".env" + env.write_bytes(b"WARDLINE_ATTEST_KEY=caf\xe9-secret\nWARDLINE_FILIGREE_TOKEN=stale\n") + _rewrite_env_token(env, "GOODTOKEN") + data = env.read_bytes() + assert b"WARDLINE_ATTEST_KEY=caf\xe9-secret" in data # non-UTF8 byte intact + assert b"\xef\xbf\xbd" not in data # no U+FFFD replacement char introduced + assert b"WEFT_FEDERATION_TOKEN=GOODTOKEN" in data + assert b"WARDLINE_FILIGREE_TOKEN" not in data + assert stat.S_IMODE(env.stat().st_mode) == 0o600 + + +# --- Task 3: probe-URL resolution + loopback --------------------------------- + + +def _write_mcp_with_filigree_url(root: Path, url: str) -> None: + root.joinpath(".mcp.json").write_text( + json.dumps( + {"mcpServers": {"wardline": {"type": "stdio", "command": "wardline", + "args": ["mcp", "--root", ".", "--filigree-url", url]}}} + ), + encoding="utf-8", + ) + + +def test_mcp_filigree_url_extracts_arg(tmp_path: Path) -> None: + _write_mcp_with_filigree_url(tmp_path, "http://127.0.0.1:8749/api/weft/scan-results") + assert _mcp_filigree_url(tmp_path) == "http://127.0.0.1:8749/api/weft/scan-results" + + +def test_mcp_filigree_url_none_when_absent(tmp_path: Path) -> None: + tmp_path.joinpath(".mcp.json").write_text( + json.dumps({"mcpServers": {"wardline": {"args": ["mcp", "--root", "."]}}}), encoding="utf-8" + ) + assert _mcp_filigree_url(tmp_path) is None + assert _mcp_filigree_url(tmp_path / "nope") is None # missing file + + +def test_mcp_filigree_url_none_when_args_not_list(tmp_path: Path) -> None: + # A hand-corrupted config where args is a string: str.index("--filigree-url") + # is a substring search that would otherwise return a char index, and + # args[idx + 1] a single bogus character. The list-type guard returns None. + tmp_path.joinpath(".mcp.json").write_text( + json.dumps( + {"mcpServers": {"wardline": {"args": "mcp --filigree-url http://x"}}} + ), + encoding="utf-8", + ) + assert _mcp_filigree_url(tmp_path) is None + + +def test_check_project_mcp_accepts_pinned_sibling_args_in_operator_order(tmp_path: Path, monkeypatch) -> None: + # Plain `wardline doctor` (no --repair): an entry pinning --loomweave-url AND + # --filigree-url in the real lacuna order (loomweave-first) must read as configured, + # not "missing wardline server". The order-preserving preserve fix guards this. + monkeypatch.setattr("wardline.install.mcp_json._find_wardline_command", lambda: "/bin/wardline") + tmp_path.joinpath(".mcp.json").write_text( + json.dumps( + {"mcpServers": {"wardline": {"type": "stdio", "command": "/bin/wardline", "args": [ + "mcp", "--root", ".", + "--loomweave-url", "http://127.0.0.1:9730", + "--filigree-url", "http://127.0.0.1:8749/api/p/lacuna/weft/scan-results", + ]}}} + ), + encoding="utf-8", + ) + check = _check_project_mcp(tmp_path) + assert check.ok, check.message + + +def test_resolve_probe_url_precedence(tmp_path: Path, monkeypatch) -> None: + monkeypatch.delenv("WARDLINE_FILIGREE_URL", raising=False) + _write_mcp_with_filigree_url(tmp_path, "http://127.0.0.1:8749/api/weft/scan-results") + # flag wins + assert _resolve_probe_url(tmp_path, "http://flag/x") == "http://flag/x" + # env beats .mcp.json + monkeypatch.setenv("WARDLINE_FILIGREE_URL", "http://env/y") + assert _resolve_probe_url(tmp_path, None) == "http://env/y" + # .mcp.json arg is the fallback that makes the real setup work + monkeypatch.delenv("WARDLINE_FILIGREE_URL", raising=False) + assert _resolve_probe_url(tmp_path, None) == "http://127.0.0.1:8749/api/weft/scan-results" + + +def test_resolve_probe_url_none_when_unconfigured(tmp_path: Path, monkeypatch) -> None: + monkeypatch.delenv("WARDLINE_FILIGREE_URL", raising=False) + assert _resolve_probe_url(tmp_path, None) is None + + +def test_is_loopback() -> None: + assert _is_loopback("http://127.0.0.1:8749/x") is True + assert _is_loopback("http://127.255.255.255:8749/x") is True + assert _is_loopback("http://localhost:8749/x") is True + assert _is_loopback("http://[::1]:8749/x") is True + assert _is_loopback("https://filigree.example.com/x") is False + + +def test_is_loopback_rejects_127_prefixed_registrable_hosts() -> None: + # The host literally STARTS with "127." but is a registrable name that resolves + # off-box via DNS. A startswith("127.") check would wrongly send the federation + # bearer to the attacker; strict IP parsing rejects it (no probe, no leak). + assert _is_loopback("http://127.attacker.com/x") is False + assert _is_loopback("http://127.evil/x") is False + assert _is_loopback("http://127.0.0.1.evil.com/x") is False + + +# --- Task 4: detection (no repair) ------------------------------------------- + + +class _ScriptedTransport: + """Returns a Response per token: maps Authorization bearer -> status.""" + + def __init__(self, status_by_token: dict[str, int], *, unreachable: bool = False) -> None: + self._status_by_token = status_by_token + self._unreachable = unreachable + + def post(self, url: str, body: bytes, headers: Mapping[str, str]) -> Response: + if self._unreachable: + raise OSError("connection refused") + token = headers.get("Authorization", "").removeprefix("Bearer ") + return Response(status=self._status_by_token.get(token, 401), body="") + + +def _setup_lacuna(root: Path, monkeypatch, env_token: str) -> None: + monkeypatch.delenv("WARDLINE_FILIGREE_URL", raising=False) + monkeypatch.delenv("WEFT_FEDERATION_TOKEN", raising=False) + monkeypatch.delenv("WARDLINE_FILIGREE_TOKEN", raising=False) + _write_mcp_with_filigree_url(root, "http://127.0.0.1:8749/api/weft/scan-results") + root.joinpath(".env").write_text(f"WARDLINE_FILIGREE_TOKEN={env_token}\n", encoding="utf-8") + + +def test_check_detects_rejected_token(tmp_path: Path, monkeypatch) -> None: + _setup_lacuna(tmp_path, monkeypatch, env_token="STALE") + t = _ScriptedTransport({"GOOD": 400}) # daemon accepts GOOD; STALE -> 401 + check = _check_filigree_auth(tmp_path, repair=False, transport=t) + assert check.status == "error" + assert "rejected" in (check.message or "") + assert check.fixed is False + + +def test_check_ok_when_token_accepted(tmp_path: Path, monkeypatch) -> None: + _setup_lacuna(tmp_path, monkeypatch, env_token="GOOD") + t = _ScriptedTransport({"GOOD": 400}) + check = _check_filigree_auth(tmp_path, repair=False, transport=t) + assert check.status == "ok" + + +def test_check_error_when_token_absent(tmp_path: Path, monkeypatch) -> None: + monkeypatch.delenv("WARDLINE_FILIGREE_URL", raising=False) + monkeypatch.delenv("WEFT_FEDERATION_TOKEN", raising=False) + monkeypatch.delenv("WARDLINE_FILIGREE_TOKEN", raising=False) + _write_mcp_with_filigree_url(tmp_path, "http://127.0.0.1:8749/api/weft/scan-results") + check = _check_filigree_auth(tmp_path, repair=False, transport=_ScriptedTransport({})) + assert check.status == "error" + assert "no federation token" in (check.message or "") + + +def test_check_ok_when_auth_off_and_no_token(tmp_path: Path, monkeypatch) -> None: + # Daemon has auth OFF: it accepts an unauthenticated emit ("" bearer -> 400). No token + # configured is fine — not an error. + monkeypatch.delenv("WARDLINE_FILIGREE_URL", raising=False) + monkeypatch.delenv("WEFT_FEDERATION_TOKEN", raising=False) + monkeypatch.delenv("WARDLINE_FILIGREE_TOKEN", raising=False) + _write_mcp_with_filigree_url(tmp_path, "http://127.0.0.1:8749/api/weft/scan-results") + t = _ScriptedTransport({"": 400}) # empty (no) bearer is accepted + check = _check_filigree_auth(tmp_path, repair=False, transport=t) + assert check.status == "ok" + + +def test_check_ok_when_unreachable(tmp_path: Path, monkeypatch) -> None: + _setup_lacuna(tmp_path, monkeypatch, env_token="STALE") + check = _check_filigree_auth(tmp_path, repair=False, transport=_ScriptedTransport({}, unreachable=True)) + assert check.status == "ok" + assert "not reachable" in (check.message or "") + + +def test_check_ok_when_non_loopback(tmp_path: Path, monkeypatch) -> None: + monkeypatch.delenv("WARDLINE_FILIGREE_URL", raising=False) + monkeypatch.setenv("WEFT_FEDERATION_TOKEN", "T") + check = _check_filigree_auth(tmp_path, repair=False, filigree_url="https://remote.example.com/api/weft/scan-results", + transport=_ScriptedTransport({})) + assert check.status == "ok" + assert "non-loopback" in (check.message or "") + + +def test_check_ok_when_url_unresolved(tmp_path: Path, monkeypatch) -> None: + monkeypatch.delenv("WARDLINE_FILIGREE_URL", raising=False) + check = _check_filigree_auth(tmp_path, repair=False, transport=_ScriptedTransport({})) + assert check.status == "ok" + assert "not configured" in (check.message or "") + + +# --- Task 5: repair ----------------------------------------------------------- + + +def test_repair_writes_accepted_candidate(tmp_path: Path, monkeypatch) -> None: + _setup_lacuna(tmp_path, monkeypatch, env_token="STALE") + # server-mode store holds the accepted token + cfg = tmp_path / "home" / ".config" / "filigree" + cfg.mkdir(parents=True) + (cfg / "federation_token").write_text("GOOD\n", encoding="utf-8") + monkeypatch.setattr("wardline.install.doctor.Path.home", lambda: tmp_path / "home") + t = _ScriptedTransport({"GOOD": 400}) # GOOD accepted, STALE -> 401 + + check = _check_filigree_auth(tmp_path, repair=True, transport=t) + + assert check.status == "ok" + assert check.fixed is True + env_text = (tmp_path / ".env").read_text(encoding="utf-8") + assert "WEFT_FEDERATION_TOKEN=GOOD" in env_text + assert "WARDLINE_FILIGREE_TOKEN" not in env_text + + +def test_fix_probes_and_repairs_with_mcp_filigree_url(tmp_path: Path, monkeypatch) -> None: + # Regression (primary lacuna path): a .mcp.json pinning --filigree-url + a stale .env + # token. machine_readable_doctor(fix=True) runs repair_install (which rewrites the + # wardline entry) and must still probe/repair the token. Two guarantees: + # 1. repair_install PRESERVES the operator-pinned --filigree-url arg (it is the + # runtime emit target; the published-port rung cannot reconstruct a fixed-port + # server-mode URL, so stripping it would silently disable emit). + # 2. filigree.auth detects the stale token and pins the accepted mint in .env. + home = tmp_path / "home" + monkeypatch.setattr("wardline.install.mcp_json.Path.home", lambda: home) + monkeypatch.setattr("wardline.install.doctor.Path.home", lambda: home) + monkeypatch.setattr("wardline.install.mcp_json._find_wardline_command", lambda: "/bin/wardline") + monkeypatch.setattr("wardline.install.detect.shutil.which", lambda _: None) + _setup_lacuna(tmp_path, monkeypatch, env_token="STALE") + # local mint holds the accepted token + cfg = home / ".config" / "filigree" + cfg.mkdir(parents=True) + (cfg / "federation_token").write_text("GOOD\n", encoding="utf-8") + t = _ScriptedTransport({"GOOD": 400}) # GOOD accepted, STALE -> 401 + + payload = machine_readable_doctor(tmp_path, fix=True, transport=t) + + checks = {c["id"]: c for c in payload["checks"]} + auth = checks["filigree.auth"] + assert auth["status"] == "ok" + assert auth["fixed"] is True + # repair preserved the pinned emit target rather than normalizing it away. + args = json.loads((tmp_path / ".mcp.json").read_text(encoding="utf-8"))["mcpServers"]["wardline"]["args"] + assert "--filigree-url" in args + assert args[args.index("--filigree-url") + 1] == "http://127.0.0.1:8749/api/weft/scan-results" + # the .mcp.json check accepts the pinned arg (not "missing wardline server"). + mcp_check = checks["mcp.registration"] + assert mcp_check["status"] == "ok" + assert "WEFT_FEDERATION_TOKEN=GOOD" in (tmp_path / ".env").read_text(encoding="utf-8") + + +def test_repair_writes_project_store_candidate_when_config_store_rejected(tmp_path: Path, monkeypatch) -> None: + # Exercises the SECOND candidate rung and the "first rejected -> second accepted" + # iteration: ~/.config/filigree holds a rejected mint, the project store + # /.weft/filigree/federation_token holds the accepted one. The project-store + # value must be the one pinned in .env. + _setup_lacuna(tmp_path, monkeypatch, env_token="STALE") + cfg = tmp_path / "home" / ".config" / "filigree" + cfg.mkdir(parents=True) + (cfg / "federation_token").write_text("WRONG\n", encoding="utf-8") # first rung: rejected + proj = tmp_path / ".weft" / "filigree" + proj.mkdir(parents=True) + (proj / "federation_token").write_text("GOOD\n", encoding="utf-8") # project rung: accepted + monkeypatch.setattr("wardline.install.doctor.Path.home", lambda: tmp_path / "home") + t = _ScriptedTransport({"GOOD": 400}) # only GOOD accepted; STALE/WRONG -> 401 + + check = _check_filigree_auth(tmp_path, repair=True, transport=t) + + assert check.status == "ok" + assert check.fixed is True + env_text = (tmp_path / ".env").read_text(encoding="utf-8") + assert "WEFT_FEDERATION_TOKEN=GOOD" in env_text + assert "WARDLINE_FILIGREE_TOKEN" not in env_text + + +def test_repair_no_candidate_matches_does_not_write(tmp_path: Path, monkeypatch) -> None: + _setup_lacuna(tmp_path, monkeypatch, env_token="STALE") + cfg = tmp_path / "home" / ".config" / "filigree" + cfg.mkdir(parents=True) + (cfg / "federation_token").write_text("ALSO-WRONG\n", encoding="utf-8") + monkeypatch.setattr("wardline.install.doctor.Path.home", lambda: tmp_path / "home") + t = _ScriptedTransport({"GOOD": 400}) # neither STALE nor ALSO-WRONG is accepted + + check = _check_filigree_auth(tmp_path, repair=True, transport=t) + + assert check.status == "error" + assert "no local federation_token matched" in (check.message or "") + assert "WARDLINE_FILIGREE_TOKEN=STALE" in (tmp_path / ".env").read_text(encoding="utf-8") # untouched From eea17d5b313c60d23bebe1faa459d2cb556e9a53 Mon Sep 17 00:00:00 2001 From: John Morrissey Date: Mon, 8 Jun 2026 01:06:27 +1000 Subject: [PATCH 008/186] feat(weft): bounded scan default, gate verdict, suppression_state, C-7 token msg MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Resolve the four wardline-owned Dogfood-2 federation tickets (weft tracker): - W1 (weft-439d09fc8d): MCP `scan` default is now BOUNDED — agent_summary is the single finding carrier (top-level `findings` array dropped), default page ≤25 bodies, `full=true` lifts the cap, `offset` paginates via agent_summary.truncation.next_offset; `explain=true` inlines provenance into the active_defects entries (capped/announced). Kills the ~123KB single-line dump. - W2 (weft-b937e53854): GateDecision gains `verdict` (NOT_EVALUATED/PASSED/FAILED) + `would_trip_at`; a bare scan (no --fail-on) is NOT_EVALUATED, never a vacuous green. Invariants re-keyed (not removed) so a tripped gate still cannot read as passed. Threaded to all 4 gate surfaces. - W3 (weft-f506e5f845): per-finding key suppressed -> suppression_state (to_jsonl, filigree metadata, legis artifact); summary gains `informational` so active+baselined+waived+judged+informational==total (unanalyzed = overlay). Identity corpus + grammar golden regenerated (rename-only, fingerprints unchanged). legis-side ingest adoption tracked in weft-ef79348eb2; the signed legis golden was rekeyed and legis_e2e stays red until legis adopts the key. - C-7 (weft-23574069a1/F1): filigree emit distinguishes no-token-sent from token-sent-but-rejected (401) and names the URL it tried. Suite 2589 green; ruff/mypy/mkdocs --strict clean; live-verified against a freshly spawned `wardline mcp`. Glossary (the vocabulary SoT) updated for the rename + new informational/verdict/would_trip_at vocabulary. Co-Authored-By: Claude Opus 4.8 (1M context) --- CHANGELOG.md | 32 ++++ docs/getting-started.md | 2 +- docs/index.md | 2 +- .../reference/finding-lifecycle-vocabulary.md | 164 +++++++++++------- src/wardline/cli/doctor.py | 4 +- src/wardline/cli/scan.py | 25 ++- src/wardline/core/agent_summary.py | 96 ++++++++-- src/wardline/core/filigree_emit.py | 47 +++-- src/wardline/core/finding.py | 4 +- src/wardline/core/legis.py | 2 +- src/wardline/core/run.py | 103 ++++++++--- src/wardline/core/scan_file_workflow.py | 9 +- src/wardline/install/doctor.py | 13 +- src/wardline/mcp/server.py | 153 +++++++++------- .../conformance/test_legis_intake_contract.py | 12 +- tests/conformance/test_mcp_handshake.py | 2 +- tests/docs/test_glossary_vocabulary.py | 61 ++++--- tests/golden/identity/corpus/META.json | 2 +- tests/golden/identity/corpus/sampleapp.json | 4 +- tests/golden/identity/corpus/stress.json | 8 +- tests/grammar/golden/builtin_findings.jsonl | 68 ++++---- tests/unit/cli/test_cli.py | 2 +- tests/unit/core/test_agent_summary.py | 6 +- tests/unit/core/test_cli_mcp_parity.py | 21 ++- tests/unit/core/test_filigree_emit.py | 46 ++++- tests/unit/core/test_finding.py | 8 +- tests/unit/core/test_legis_artifact.py | 29 ++-- tests/unit/core/test_run.py | 75 ++++++-- .../unit/install/test_doctor_filigree_auth.py | 45 +++-- tests/unit/install/test_mcp_json.py | 54 ++++-- tests/unit/mcp/test_server_filigree_emit.py | 4 + tests/unit/mcp/test_server_loomweave_write.py | 2 +- tests/unit/mcp/test_server_query_explain.py | 92 +++++++--- tests/unit/mcp/test_server_suppression.py | 10 +- tests/unit/mcp/test_server_tools.py | 11 +- 35 files changed, 863 insertions(+), 355 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index a9ca030d..f629d3e8 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -8,6 +8,20 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] ### Added +- **Gate verdict is now explicit (no vacuous green).** `GateDecision` carries a + `verdict` (`NOT_EVALUATED` / `PASSED` / `FAILED`) and a `would_trip_at` (the + highest severity that would trip on the evaluated population, or null). A bare + scan with no `--fail-on` reports `verdict: NOT_EVALUATED` + `would_trip_at` + instead of a clean-looking `tripped: false`, so an agent's first scan is never a + false green. Surfaced on every gate block (MCP `scan`, agent-summary, + `scan_file_findings`) and on the CLI as a `gate: NOT_EVALUATED — …` line + (weft-b937e53854). +- **Bounded default scan output + pagination.** The MCP `scan` tool returns a + bounded page (≤25 finding bodies) by default so a bare call cannot overflow an + agent's context (previously ~123KB on one line). New `full: true` lifts the cap; + new `offset` pages through the rest via `agent_summary.truncation.next_offset`. + `explain: true` inlines provenance into the `agent_summary.active_defects` + entries (capped, announced) (weft-439d09fc8d). - `wardline doctor` now verifies the Filigree federation token: it probes the configured daemon (URL resolved from `.mcp.json`/env) with the token wardline would emit and reports a `filigree.auth` check. `--repair` recovers the @@ -15,6 +29,24 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 in `.env`, removing a stale `WARDLINE_FILIGREE_TOKEN` line. ### Changed +- **BREAKING (wire): per-finding `suppressed` key renamed to `suppression_state`.** + The JSONL stream, the Filigree `metadata.wardline.*` subtree, and the signed + **legis scan artifact** now emit `suppression_state` (values unchanged: + `active`/`baselined`/`waived`/`judged`). This eliminates the "active" overload + (the per-finding *state* vs the summary `active` *count*). Because the legis + artifact is signed, the canonical bytes — and the golden signature — change; + **legis must adopt `suppression_state` on its ingest/co-sign side** before the + signed hop verifies again (the opt-in `legis_e2e` oracle stays red until then) + (weft-f506e5f845). +- The MCP `scan` response no longer carries a top-level `findings` array; finding + bodies live solely in `agent_summary` (the single canonical carrier). The + `truncation` block moved under `agent_summary` (weft-439d09fc8d). +- The MCP `summary` block adds an `informational` bucket so + `active + baselined + waived + judged + informational == total` + (weft-f506e5f845). +- The Filigree emit `disabled_reason` now distinguishes *no token sent* from + *token sent but rejected (401)* and names the URL it tried, instead of a flat + "set WEFT_FEDERATION_TOKEN" that implied absence (weft-23574069a1 / C-7). - **BREAKING: Weft config/store consolidation.** Operator config moved from `wardline.yaml` (YAML) to the `[wardline]` table of a shared, operator-authored `weft.toml` (TOML), read via stdlib `tomllib` (zero new dependency). An diff --git a/docs/getting-started.md b/docs/getting-started.md index 176a905c..ee2d6e07 100644 --- a/docs/getting-started.md +++ b/docs/getting-started.md @@ -61,7 +61,7 @@ readability): "location": {"path": "service.py", "line_start": 7, "line_end": 8, "col_start": 0, "col_end": 26}, "message": "service.current_user declares return trust INTEGRAL but actually returns EXTERNAL_RAW (less trusted) — untrusted data reaches a trusted producer", "properties": {"declared_return": "INTEGRAL", "actual_return": "EXTERNAL_RAW"}, - "suppressed": "active", + "suppression_state": "active", "suppression_reason": null, "confidence": null, "suggestion": null, diff --git a/docs/index.md b/docs/index.md index abf67db1..2d2e83f7 100644 --- a/docs/index.md +++ b/docs/index.md @@ -41,7 +41,7 @@ directory; the line above is the run summary. One of those findings flags a trust-boundary violation: ```json -{"rule_id": "PY-WL-101", "severity": "ERROR", "kind": "defect", "qualname": "service.current_user", "location": {"path": "service.py", "line_start": 7, "line_end": 8, "col_start": 0, "col_end": 26}, "message": "service.current_user declares return trust INTEGRAL but actually returns EXTERNAL_RAW (less trusted) — untrusted data reaches a trusted producer", "properties": {"declared_return": "INTEGRAL", "actual_return": "EXTERNAL_RAW"}, "suppressed": "active"} +{"rule_id": "PY-WL-101", "severity": "ERROR", "kind": "defect", "qualname": "service.current_user", "location": {"path": "service.py", "line_start": 7, "line_end": 8, "col_start": 0, "col_end": 26}, "message": "service.current_user declares return trust INTEGRAL but actually returns EXTERNAL_RAW (less trusted) — untrusted data reaches a trusted producer", "properties": {"declared_return": "INTEGRAL", "actual_return": "EXTERNAL_RAW"}, "suppression_state": "active"} ``` That is Wardline reporting that a function annotated as a trusted producer diff --git a/docs/reference/finding-lifecycle-vocabulary.md b/docs/reference/finding-lifecycle-vocabulary.md index b6d50c1a..86f68ec1 100644 --- a/docs/reference/finding-lifecycle-vocabulary.md +++ b/docs/reference/finding-lifecycle-vocabulary.md @@ -1,7 +1,7 @@ # Finding lifecycle & gate vocabulary This is the single source of truth for the words Wardline uses to describe the -**state and lifecycle of a finding** — `new`, `active`, `suppressed`, +**state and lifecycle of a finding** — `new`, `active`, `suppression_state`, `baselined`, `waived`, `judged` — and how each one maps onto the three surfaces an agent reads: the **CLI summary line**, the **MCP / agent-summary JSON**, and the **Filigree store**. @@ -45,11 +45,21 @@ waiver > judged > baseline** — explicit human intent wins, then the LLM verdic (so its rationale is the visible reason), then the silent baseline (`src/wardline/core/suppression.py:61-70`). -**"suppressed"** is the umbrella term for "any state other than `active`": -`baselined` + `waived` + `judged`. The CLI prints this sum as the `suppressed` -count (`src/wardline/cli/scan.py:360`), and `to_filigree_metadata` only writes a -`suppressed` key when the state is not `active` -(`src/wardline/core/finding.py:184-187`). +### The per-finding key is `suppression_state` (not `suppressed`) + +Each serialized finding carries its state under the key **`suppression_state`** +(`src/wardline/core/finding.py:126` in `to_jsonl`; `src/wardline/core/finding.py:185` +in the Filigree `metadata.wardline.*` subtree; the agent-summary entries and the +legis artifact use the same key). The key was renamed from `suppressed` → +`suppression_state` (weft-f506e5f845) so the per-finding **state** never reads as +the opposite of the summary's `active` **count**: `suppression_state: "active"` +clearly names a state, while `summary.active` is a count of unsuppressed defects. +The Filigree metadata only carries the key when the state is not `active` +(`src/wardline/core/finding.py:185`). + +**"suppressed"** survives only as the umbrella *word* for "any state other than +`active`": `baselined` + `waived` + `judged`. The CLI prints this sum as the +`suppressed` count (`src/wardline/cli/scan.py:362`). ## `active` is the one word for "non-suppressed defect" @@ -59,37 +69,39 @@ consistently, on every surface: | Surface | Where | Term | | --- | --- | --- | | Enum | `src/wardline/core/finding.py:68` | `SuppressionState.ACTIVE = "active"` | -| Summary field | `src/wardline/core/run.py:50`, built at `src/wardline/core/run.py:288` | `ScanSummary.active` | -| CLI summary line | `src/wardline/cli/scan.py:361` | `… {s.active} active` | -| MCP scan response | `src/wardline/mcp/server.py:313` | `summary.active` | -| Agent-summary JSON | `src/wardline/core/agent_summary.py:99` | `summary.active_defects` | +| Summary field | `src/wardline/core/run.py:50`, built at `src/wardline/core/run.py:312` | `ScanSummary.active` | +| CLI summary line | `src/wardline/cli/scan.py:370` | `… {s.active} active` | +| MCP scan response | `src/wardline/mcp/server.py:328` | `summary.active` | +| Agent-summary JSON | `src/wardline/core/agent_summary.py:117` | `summary.active_defects` | | `wardline:loop` prompt | `src/wardline/mcp/prompts.py:13` | "Read `summary.active`" | The agent-summary key is `active_defects` rather than bare `active` — that is a descriptive-suffix convention alongside `total_findings` / `suppressed_findings` -(`src/wardline/core/agent_summary.py:98-105`), not a different concept. It counts +(`src/wardline/core/agent_summary.py:116-124`), not a different concept. It counts the same population. The discipline test `tests/cli/test_scan_summary_vocab.py` pins this: the CLI line says `active` (never `new`), and the count matches the agent-summary and MCP surfaces. -## The three meanings of "new" +## The summary buckets partition the total -"new" is overloaded across the suite. Wardline's own surfaces no longer use it -for the active count (that was a historical CLI mislabel, now `active`). The word -still legitimately means three different things depending on the surface: +`ScanSummary` (`src/wardline/core/run.py:47-68`) counts split the whole scan into +buckets that **sum to `total`** exactly (weft-f506e5f845): -| "new" on this surface | Means | Owner / anchor | -| --- | --- | --- | -| Filigree store | An **unseen fingerprint** — first time this finding identity is seen for a `(file, scan_source)`. Driven by `mark_unseen` / the absent-fingerprint sweep. | **Filigree-owned** lifecycle (`src/wardline/core/filigree_emit.py:68-76`) | -| `wardline scan --new-since ` | **Delta-scope**: the gate fires only on defects in files/entities changed since a git ref; everything else is re-marked `baselined`. | `src/wardline/core/run.py:264-283`; help text `src/wardline/cli/scan.py` (`--new-since`, "new findings only") | -| (historical) CLI summary | Formerly relabelled the `active` count as "N new". **Corrected to "N active"** so the CLI matches every other surface. | `src/wardline/cli/scan.py:360` | +- the defect buckets partition the `DEFECT`s by state — + `active` (`src/wardline/core/run.py:50`) + `baselined` (`src/wardline/core/run.py:52`) + + `waived` (`src/wardline/core/run.py:53`) + `judged` (`src/wardline/core/run.py:54`); +- `informational` (`src/wardline/core/run.py:60`) is **every non-defect finding** + (facts, metrics, classifications) — the rest of `total`. -The first-seen Filigree sense and the delta-scope `--new-since` sense are -genuinely distinct concepts; neither is "active". An agent should read the CLI / -MCP `active` count as "live defects now", Filigree's first-seen status as "is this -identity new to the tracker", and `--new-since` as "only gate on what changed". +So `active + baselined + waived + judged + informational == total` +(`src/wardline/core/run.py:49` for `total: int`). `unanalyzed` +(`src/wardline/core/run.py:68`) is an **overlay** — a subset of `informational` +that surfaces a silent under-scan — and is deliberately **not** a partition member. +The MCP `summary` block exposes `informational` (`src/wardline/mcp/server.py:336`) +and `unanalyzed` (`src/wardline/mcp/server.py:340`); the agent-summary block mirrors +both (`src/wardline/core/agent_summary.py:123`, `src/wardline/core/agent_summary.py:124`). ## Emitted-active vs the gate population @@ -97,35 +109,66 @@ There are **two distinct populations** of defects in one scan, and they can differ on purpose: 1. **Emitted-active** — `summary.active` counts `active` defects in the - **emitted** (post-annotation) findings (`src/wardline/core/run.py:285-293`). + **emitted** (post-annotation) findings (built at `src/wardline/core/run.py:312`). Baseline / waiver / judged annotate these findings in place; a suppressed defect is still emitted, just not counted as `active`. 2. **Gate population** — the `--fail-on` gate evaluates a **separate** `ScanResult.gate_findings` list: the *unsuppressed* population - (`src/wardline/core/run.py:250-254`). By default, repository-controlled + (`src/wardline/core/run.py:278`). By default, repository-controlled baseline / waiver / judged entries **annotate** the emitted findings but do **not** clear the gate — so a malicious PR cannot green the gate by committing a suppression keyed to its own new defect. `gate_decision` evaluates `gate_findings` when present, else falls back to `findings` (the trusted - `--trust-suppressions` / directly-constructed path) - (`src/wardline/core/run.py:315-316`). + `--trust-suppressions` / directly-constructed path), selected at + `src/wardline/core/run.py:363` (`honors_suppressions`). This is why **`summary.active: 0` can co-exist with `gate.tripped: true`**: every defect was suppressed by a committed baseline (so emitted-active is 0), but those suppressions do not clear the unsuppressed gate population. It is by design, not a -bug. The gate result is reported separately from `summary.active`: `GateDecision` -carries `tripped` / `fail_on` / `exit_class` **plus** a human `reason` and the -`evaluated` population it judged (`src/wardline/core/run.py:86-96`), so the -`0 active + tripped` case explains itself instead of reading as a defect. The MCP -`scan` block exposes `gate.tripped` / `gate.reason` / `gate.evaluated` / -`gate.migration_hint` (`src/wardline/mcp/server.py:332-338`); the CLI prints -`gate: FAILED (--fail-on …) — ` then `gate: evaluated <…>` on stderr -(`src/wardline/cli/scan.py:375-376`). +bug. + +### The gate verdict is explicit (never a vacuous green) + +`GateDecision` (`src/wardline/core/run.py:97`) carries `tripped` / `fail_on` / +`exit_class` **plus** an explicit `verdict` (`src/wardline/core/run.py:106`) and a +`would_trip_at`, alongside a human `reason` and the `evaluated` population it +judged. The `verdict` is one of: + +- **`NOT_EVALUATED`** — no `--fail-on` threshold was set, so the gate never judged. + A bare scan is **not** a clean pass; `would_trip_at` names the highest severity + that *would* trip so the agent's first call is not a false green (weft-b937e53854). +- **`PASSED`** — a threshold ran and nothing tripped. +- **`FAILED`** — a threshold ran and tripped. + +The MCP `scan` gate block exposes `gate.tripped` (`src/wardline/mcp/server.py:343`), +`gate.verdict` (`src/wardline/mcp/server.py:346`), `would_trip_at`, `reason`, +`evaluated`, and `migration_hint`, opened at `src/wardline/mcp/server.py:342` +(`"gate": {`); the agent-summary mirrors them at +`src/wardline/core/agent_summary.py:127` (`tripped`) and +`src/wardline/core/agent_summary.py:130` (`verdict`). The CLI prints +`gate: FAILED (--fail-on …) — ` then `gate: evaluated <…>`, or a +`gate: NOT_EVALUATED — …` line for a bare scan +(`src/wardline/cli/scan.py:388`). `--new-since` scopes **both** populations identically: any `active` defect outside the delta is re-marked `baselined` in both the emitted and gate lists -(`src/wardline/core/run.py:264-283`). +(`src/wardline/core/run.py:288`, `def apply_delta_scope`). + +## The three meanings of "new" + +"new" is overloaded across the suite. Wardline's own surfaces no longer use it +for the active count (that was a historical CLI mislabel, now `active`). The word +still legitimately means three different things depending on the surface: + +| "new" on this surface | Means | Owner / anchor | +| --- | --- | --- | +| Filigree store | An **unseen fingerprint** — first time this finding identity is seen for a `(file, scan_source)`. | **Filigree-owned** lifecycle (`src/wardline/core/filigree_emit.py:68-76`) | +| `wardline scan --new-since ` | **Delta-scope**: the gate fires only on defects in files/entities changed since a git ref; everything else is re-marked `baselined`. | `src/wardline/core/run.py:288`; help text `src/wardline/cli/scan.py` (`--new-since`) | +| (historical) CLI summary | Formerly relabelled the `active` count as "N new". **Corrected to "N active"**. | `src/wardline/cli/scan.py:362` | + +The first-seen Filigree sense and the delta-scope `--new-since` sense are +genuinely distinct concepts; neither is "active". ## Cross-surface mapping table @@ -133,36 +176,39 @@ How each concept appears on each surface: | Concept | CLI summary text | `ScanSummary` field | MCP `summary` key | Agent-summary key | Filigree store | | --- | --- | --- | --- | --- | --- | -| every finding | `N finding(s)` | `total` (`run.py:49`) | `total` (`server.py:312`) | `total_findings` (`agent_summary.py:98`) | one finding per wire entry | -| live defect | `N active` (`scan.py:361`) | `active` (`run.py:50,288`) | `active` (`server.py:313`) | `active_defects` (`agent_summary.py:99`) | no `suppressed` key (`finding.py:184`) | -| suppressed (sum) | `N suppressed` (`scan.py:360`) | `baselined+waived+judged` | the three keys | `suppressed_findings` (`agent_summary.py:100`) | `metadata.wardline.suppressed` (`finding.py:184-187`) | -| baselined | `N baseline` | `baselined` (`run.py:52`) | `baselined` (`server.py:314`) | `baselined` (`agent_summary.py:102`) | `suppressed: "baselined"` | -| waived | `N waiver` | `waived` (`run.py:53`) | `waived` (`server.py:315`) | `waived` (`agent_summary.py:103`) | `suppressed: "waived"` | -| judged | `N judged` | `judged` (`run.py:54`) | `judged` (`server.py:316`) | `judged` (`agent_summary.py:104`) | `suppressed: "judged"` | -| under-scan | `N file(s) could not be analyzed` | `unanalyzed` (`run.py:60`) | `unanalyzed` (`server.py:320`) | `unanalyzed` (`agent_summary.py:105`) | `WLN-ENGINE-*` facts | -| gate verdict | exit code + `--fail-on` | (`gate_findings`, `run.py:79`) | `gate.tripped` (`server.py:333`) | `gate.tripped` (`agent_summary.py:108`) | not emitted to Filigree | +| every finding | `N finding(s)` | `total` (`run.py:49`) | `total` (`server.py:327`) | `total_findings` (`agent_summary.py:116`) | one finding per wire entry | +| live defect | `N active` (`scan.py:370`) | `active` (`run.py:50,312`) | `active` (`server.py:328`) | `active_defects` (`agent_summary.py:117`) | no `suppression_state` key (`finding.py:185`) | +| suppressed (sum) | `N suppressed` (`scan.py:362`) | `baselined+waived+judged` | the three keys | `suppressed_findings` (`agent_summary.py:118`) | `metadata.wardline.suppression_state` (`finding.py:185`) | +| baselined | `N baseline` | `baselined` (`run.py:52`) | `baselined` (`server.py:329`) | `baselined` (`agent_summary.py:120`) | `suppression_state: "baselined"` | +| waived | `N waiver` | `waived` (`run.py:53`) | `waived` (`server.py:330`) | `waived` (`agent_summary.py:121`) | `suppression_state: "waived"` | +| judged | `N judged` | `judged` (`run.py:54`) | `judged` (`server.py:331`) | `judged` (`agent_summary.py:122`) | `suppression_state: "judged"` | +| informational | (the remainder of `total`) | `informational` (`run.py:60`) | `informational` (`server.py:336`) | `informational` (`agent_summary.py:123`) | facts/metrics | +| under-scan | `N file(s) could not be analyzed` | `unanalyzed` (`run.py:68`) | `unanalyzed` (`server.py:340`) | `unanalyzed` (`agent_summary.py:124`) | `WLN-ENGINE-*` facts | +| gate verdict | exit code + `--fail-on` | (`gate_findings`, `run.py:87`; `GateDecision`, `run.py:97`, `verdict` `run.py:106`) | `gate.tripped` (`server.py:343`), `gate.verdict` (`server.py:346`) | `gate.tripped` (`agent_summary.py:127`), `gate.verdict` (`agent_summary.py:130`) | not emitted to Filigree | + +The unsuppressed gate population is built from `Baseline(frozenset())` +(`src/wardline/core/run.py:278`). ## For the suite This page is the **Wardline-anchored** glossary. Two pieces of the vocabulary are -owned by sibling tools and are intentionally **not** renamed by Wardline — they -are recorded here as coordination context, not as a change Wardline executes: +owned by sibling tools and are recorded here as coordination context: - **Filigree's "new" / `seen_count` lifecycle is Filigree-owned.** Filigree decides first-seen vs returning purely from fingerprint presence across scans (`mark_unseen`, `src/wardline/core/filigree_emit.py:68-76`). Wardline emits the - fingerprint and `scanned_paths`; it does not, and should not, rename Filigree's - first-seen concept to match its own `active`. The two words mean different - things and that distinction is correct. + fingerprint and `scanned_paths`; it does not rename Filigree's first-seen concept. -- **legis receives the gate population as `active`.** The legis scan artifact - projects the *whole scan*, mapping `baselined` / `judged` onto legis's own - `suppressed` while `active` stays `active`, so legis reproduces Wardline's gate - population exactly (the "one judge" property). This is a contract Wardline - conforms to, not a rename of any other tool's fields (see the CHANGELOG legis - handoff entry and [Signed scan handoff to legis](../guides/legis-handoff.md)). +- **legis receives the gate population, keyed by `suppression_state`.** The legis + scan artifact projects the *whole scan*, mapping `baselined` / `judged` onto + legis's own `suppressed` value while `active` stays `active`, so legis reproduces + Wardline's gate population exactly (the "one judge" property). The per-finding key + was renamed `suppressed` → `suppression_state` (weft-f506e5f845); legis adopts the + same key on its side (tracked as a federation contract change). See the CHANGELOG + legis handoff entry and [Signed scan handoff to legis](../guides/legis-handoff.md). In short: **within Wardline, `active` is the single word for a non-suppressed -defect, on every surface.** The remaining divergence is genuine cross-tool -semantics (Filigree's first-seen lifecycle, `--new-since` delta-scope) that this -glossary documents rather than collapses. No cross-repo rename is implied. +defect, and `suppression_state` is the single per-finding key for its state, on +every surface.** The remaining divergence is genuine cross-tool semantics +(Filigree's first-seen lifecycle, `--new-since` delta-scope) that this glossary +documents rather than collapses. diff --git a/src/wardline/cli/doctor.py b/src/wardline/cli/doctor.py index 2b031a93..43e8a3c8 100644 --- a/src/wardline/cli/doctor.py +++ b/src/wardline/cli/doctor.py @@ -26,9 +26,7 @@ ) @click.option("--repair", is_flag=True, help="Repair missing or stale install artifacts.") @click.option("--fix", "fix_json", is_flag=True, help="Repair install bindings and emit machine-readable JSON.") -@click.option( - "--filigree-url", default=None, help="Filigree Weft URL to probe (default: resolve from .mcp.json/env)." -) +@click.option("--filigree-url", default=None, help="Filigree Weft URL to probe (default: resolve from .mcp.json/env).") def doctor(root: Path, repair: bool, fix_json: bool, filigree_url: str | None) -> None: """Check Wardline agent install artifacts and sibling bindings.""" if repair and fix_json: diff --git a/src/wardline/cli/scan.py b/src/wardline/cli/scan.py index 3bf1e9d2..e4af9f0e 100644 --- a/src/wardline/cli/scan.py +++ b/src/wardline/cli/scan.py @@ -307,10 +307,19 @@ def confirm_cb(rel_path: str, orig: str, replacement: str, f: Finding) -> bool: "Findings written locally only.", err=True, ) + elif emit_result.token_sent: + # A token WAS sent and rejected — the value is wrong, not absent. Saying + # "set the token" here is the C-7 misdiagnosis (weft-23574069a1). + click.echo( + f"warning: Filigree rejected the token (401) at {filigree_url}; a token WAS sent but " + "its value is wrong — align WEFT_FEDERATION_TOKEN (env or .env) to the canonical " + "federation token. Findings written locally only.", + err=True, + ) else: click.echo( - f"warning: Filigree returned {emit_result.status} (auth rejected) at {filigree_url}; " - "set WEFT_FEDERATION_TOKEN (or .env) to the project token. Findings written locally only.", + f"warning: Filigree returned 401 (auth rejected) at {filigree_url}; no token was sent — " + "set WEFT_FEDERATION_TOKEN (env or .env) to the project token. Findings written locally only.", err=True, ) elif emit_result.status is not None: @@ -368,9 +377,13 @@ def confirm_cb(rel_path: str, orig: str, replacement: str, f: Finding) -> bool: f"(see WLN-ENGINE-* facts in {output}).", err=True, ) - gate_dec = gate_decision(result, Severity(fail_on)) if fail_on is not None else None - gate_tripped = gate_dec is not None and gate_dec.tripped - if gate_dec is not None and gate_dec.tripped: + gate_dec = gate_decision(result, Severity(fail_on)) if fail_on is not None else gate_decision(result, None) + gate_tripped = gate_dec.tripped + if gate_dec.verdict == "NOT_EVALUATED": + # A bare scan never ran the gate — say so explicitly so a clean-looking exit is not + # mistaken for a PASS (weft-b937e53854). Carries would_trip_at via the reason. + click.echo(f"gate: NOT_EVALUATED — {gate_dec.reason}", err=True) + elif gate_dec.tripped: # Never let "0 active + gate FAILED" read as a bug: say why and which population. click.echo(f"gate: FAILED (--fail-on {gate_dec.fail_on}) — {gate_dec.reason}", err=True) click.echo(f"gate: evaluated {gate_dec.evaluated}", err=True) @@ -405,6 +418,8 @@ def _filigree_status(result: EmitResult | None) -> dict[str, object]: "disabled_reason": filigree_disabled_reason( reachable=result.reachable, status=result.status, + token_sent=result.token_sent, + url=result.url, ), } diff --git a/src/wardline/core/agent_summary.py b/src/wardline/core/agent_summary.py index e11bc5a8..b32e0b85 100644 --- a/src/wardline/core/agent_summary.py +++ b/src/wardline/core/agent_summary.py @@ -53,6 +53,10 @@ class AgentSummary: display_findings: list[Finding] | None = None summary_only: bool = False max_findings: int | None = None + # Offset into the flattened ordered union (active → suppressed → engine_facts) for + # pagination. The default scan returns a bounded first page; the agent walks the rest + # with offset = truncation.next_offset (weft-439d09fc8d). + offset: int = 0 include_suppressed: bool = True # The secure-gate-default rollout hint (or None), surfaced in the gate block so the # "see gate.migration_hint" pointer in next_actions resolves on this surface too — the @@ -67,27 +71,41 @@ def __post_init__(self) -> None: # is too costly for the hot scan path). if self.max_findings is not None and self.max_findings < 0: raise ValueError(f"max_findings must be >= 0, got {self.max_findings}") + if self.offset < 0: + raise ValueError(f"offset must be >= 0, got {self.offset}") def to_dict(self) -> dict[str, Any]: # Counts are whole-project (summary describes the whole project, per the `where` - # contract); arrays come from the displayed/filtered view, then bounded. + # contract); arrays come from the displayed/filtered view, then paginated. count_active = len(_active_defects(self.result.findings)) count_suppressed = len(_suppressed_defects(self.result.findings)) count_facts = len(_engine_facts(self.result.findings)) base = self.result.findings if self.display_findings is None else self.display_findings + # ONE ordered sequence is the pagination unit (weft-439d09fc8d): active defects first + # (most urgent), then suppressed debt, then engine facts — each internally sorted by + # _sort_key. A single offset+max_findings window slices this union so one truncation + # block describes the whole page, not three independently-sliced arrays. + union = _active_defects(base) + if self.include_suppressed: + union = union + _suppressed_defects(base) + union = union + _engine_facts(base) + findings_total = len(union) if self.summary_only: - shown_active: list[Finding] = [] - shown_suppressed: list[Finding] = [] - shown_facts: list[Finding] = [] + window: list[Finding] = [] + elif self.max_findings is not None: + window = union[self.offset : self.offset + self.max_findings] else: - shown_active = _active_defects(base) - shown_suppressed = _suppressed_defects(base) if self.include_suppressed else [] - shown_facts = _engine_facts(base) - if self.max_findings is not None: - shown_active = shown_active[: self.max_findings] - shown_suppressed = shown_suppressed[: self.max_findings] - shown_facts = shown_facts[: self.max_findings] + window = union[self.offset :] + findings_returned = len(window) + end = self.offset + findings_returned + findings_truncated = (not self.summary_only) and end < findings_total + next_offset = end if findings_truncated else None + # Re-split the window back into the three display arrays by category (the union was + # built in category order, so this preserves both order and the page boundary). + shown_active = [f for f in window if f.kind is Kind.DEFECT and f.suppressed is SuppressionState.ACTIVE] + shown_suppressed = [f for f in window if f.kind is Kind.DEFECT and f.suppressed is not SuppressionState.ACTIVE] + shown_facts = [f for f in window if f.kind is Kind.FACT] active_defects = [_finding_entry(f, include_next=True) for f in shown_active] suppressed = [_finding_entry(f, include_next=False) for f in shown_suppressed] engine_facts = [_finding_entry(f, include_next=False) for f in shown_facts] @@ -102,12 +120,15 @@ def to_dict(self) -> dict[str, Any]: "baselined": self.result.summary.baselined, "waived": self.result.summary.waived, "judged": self.result.summary.judged, + "informational": self.result.summary.informational, "unanalyzed": self.result.summary.unanalyzed, }, "gate": { "tripped": self.gate.tripped, "fail_on": self.gate.fail_on, "exit_class": self.gate.exit_class, + "verdict": self.gate.verdict, + "would_trip_at": self.gate.would_trip_at, "reason": self.gate.reason, "evaluated": self.gate.evaluated, "migration_hint": self.migration_hint, @@ -119,6 +140,20 @@ def to_dict(self) -> dict[str, Any]: "active_defects": active_defects, "suppressed_findings": suppressed, "engine_facts": engine_facts, + # Every cut is explicit so a bounded page never reads as "covered all". This is the + # single pagination descriptor for the union above; ``explanations_truncated`` is + # set by the MCP server when explain=true inlines provenance (core never explains). + "truncation": { + "summary_only": self.summary_only, + "include_suppressed": self.include_suppressed, + "max_findings": self.max_findings, + "offset": self.offset, + "findings_total": findings_total, + "findings_returned": findings_returned, + "next_offset": next_offset, + "findings_truncated": findings_truncated, + "explanations_truncated": False, + }, # next_actions follow the whole-project active count, not the displayed slice, # so a summary_only/filtered view does not falsely say "no active defects" — and # they are GATE-AWARE so a baselined-only trip (0 active + gate FAILED) never @@ -195,11 +230,46 @@ def _finding_entry(finding: Finding, *, include_next: bool) -> dict[str, Any]: def _next_actions_for(active_count: int, gate: GateDecision) -> list[dict[str, Any]]: if active_count > 0: - return [ + actions = [ {"tool": "explain_taint", "reason": "inspect each active defect before editing"}, {"tool": "file_finding", "reason": "promote confirmed true positives after Filigree emission"}, {"tool": "scan", "reason": "rescan after fixes to verify closure"}, ] + if gate.verdict == "NOT_EVALUATED": + # Active defects AND no threshold ran — name the enforcement step so a green-looking + # exit is not mistaken for a pass (weft-b937e53854). + actions.append( + { + "tool": "scan", + "reason": ( + f"gate NOT_EVALUATED (no --fail-on ran); pass --fail-on " + f"{gate.would_trip_at or 'ERROR'} to enforce" + ), + } + ) + return actions + if gate.verdict == "NOT_EVALUATED": + # 0 active defects but the gate never ran. If would_trip_at is set, suppressed/baselined + # defects would re-enter an unsuppressed gate (the dogfood-#2 "worse" case); never let + # this read as a clean pass. + if gate.would_trip_at is not None: + return [ + { + "tool": "scan", + "reason": ( + f"gate NOT_EVALUATED (no --fail-on ran); 0 active defects but suppressed/baselined " + f"{gate.would_trip_at}+ finding(s) would trip an unsuppressed gate — pass --fail-on " + f"{gate.would_trip_at} to enforce, or trust_suppressions / new_since to scope" + ), + } + ] + return [ + { + "tool": "scan", + "reason": "gate NOT_EVALUATED (no --fail-on ran); no defect would trip at any threshold — " + "pass --fail-on ERROR to lock it in", + } + ] if gate.tripped: # 0 active defects but the gate FAILED — it tripped on suppressed/baselined findings. # Do NOT say "rescan after edits" (which reads as passed); point at the gate verdict. @@ -226,6 +296,7 @@ def build_agent_summary( display_findings: list[Finding] | None = None, summary_only: bool = False, max_findings: int | None = None, + offset: int = 0, include_suppressed: bool = True, migration_hint: str | None = None, ) -> AgentSummary: @@ -237,6 +308,7 @@ def build_agent_summary( display_findings=display_findings, summary_only=summary_only, max_findings=max_findings, + offset=offset, include_suppressed=include_suppressed, migration_hint=migration_hint, ) diff --git a/src/wardline/core/filigree_emit.py b/src/wardline/core/filigree_emit.py index 0f591f0f..24d452bc 100644 --- a/src/wardline/core/filigree_emit.py +++ b/src/wardline/core/filigree_emit.py @@ -104,6 +104,14 @@ class EmitResult: # and a 2xx success. It is the *error* status: a reached/success result carries none. # All of these stay SOFT (reachable=False); only the message differs. status: int | None = None + # Whether a bearer token was actually sent on the attempt. A 401 means different things + # by this flag: token_sent=False → none configured (set one); token_sent=True → the value + # was REJECTED (it is wrong; align it to the canonical source). The original "set + # WEFT_FEDERATION_TOKEN" message implied absence and is what steered F1's wrong root-cause + # (weft-23574069a1 / C-7). ``url`` is the endpoint attempted, so the actionable message can + # name WHERE it tried without the caller threading it separately. + token_sent: bool = False + url: str | None = None @property def auth_rejected(self) -> bool: @@ -133,7 +141,9 @@ class ProbeResult: status: int | None = None -def filigree_disabled_reason(*, reachable: bool, status: int | None) -> str | None: +def filigree_disabled_reason( + *, reachable: bool, status: int | None, token_sent: bool = False, url: str | None = None +) -> str | None: """The ``disabled_reason`` for an emit attempt, or None when Filigree was reached. Single source of the auth-rejected (401/403) vs server-error (5xx) vs unreachable @@ -149,14 +159,23 @@ def filigree_disabled_reason(*, reachable: bool, status: int | None) -> str | No """ if reachable: return None + at = f" at {url}" if url else "" if status in (401, 403): - # 401 → set a token; 403 → token present but lacks access (a token won't help). + # 403 → token present but lacks access (a token won't help). 401 → split by whether a + # token was actually SENT: absent (set one) vs rejected (the value is wrong). The old + # flat "set WEFT_FEDERATION_TOKEN" implied absence even when a token was sent and + # rejected — the C-7 misdiagnosis (weft-23574069a1). if status == 403: - return "filigree forbidden (403); token present but lacks access / blocked" - return f"filigree auth-rejected ({status}); set WEFT_FEDERATION_TOKEN" + return f"filigree forbidden (403){at}; token present but lacks access / blocked" + if token_sent: + return ( + f"filigree rejected the token (401){at}; a token WAS sent but its value is wrong — " + "align WEFT_FEDERATION_TOKEN (env or .env) to the canonical federation token" + ) + return f"filigree auth-rejected (401){at}; no token sent — set WEFT_FEDERATION_TOKEN (env or .env)" if status is not None: - return f"filigree server error ({status})" - return "filigree unreachable" + return f"filigree server error ({status}){at}" + return f"filigree unreachable{at}" class Transport(Protocol): @@ -196,7 +215,8 @@ def __init__(self, url: str, *, transport: Transport | None = None, token: str | def emit(self, findings: Sequence[Finding], *, scanned_paths: Sequence[str] = ()) -> EmitResult: body = json.dumps(build_scan_results_body(findings, scanned_paths=scanned_paths)).encode("utf-8") headers = {"Content-Type": "application/json"} - if self._token: + token_sent = bool(self._token) + if token_sent: headers["Authorization"] = f"Bearer {self._token}" try: resp = self._transport.post(self._url, body, headers) @@ -204,16 +224,17 @@ def emit(self, findings: Sequence[Finding], *, scanned_paths: Sequence[str] = () # Connection refused / DNS / timeout — sibling absent. Enrichment is # non-load-bearing: warn (at the CLI) and continue. No status reached us, so # this is the genuine "could not reach" case (status=None). - return EmitResult(reachable=False) + return EmitResult(reachable=False, token_sent=token_sent, url=self._url) if resp.status in (401, 403): # Filigree is present but its opt-in bearer auth is on and refusing us. Stays - # SOFT (enrichment unavailable, never exit-2) — but distinguished as auth so the - # caller can say "401 (set WEFT_FEDERATION_TOKEN)" instead of "could not reach". - return EmitResult(reachable=False, status=resp.status) + # SOFT (enrichment unavailable, never exit-2) — but distinguished as auth (and by + # token_sent: no-token vs token-rejected) so the caller can say the actionable + # thing instead of "could not reach". + return EmitResult(reachable=False, status=resp.status, token_sent=token_sent, url=self._url) if resp.status >= 500: # Server-side outage (5xx) — the sibling is degraded, not a Wardline payload bug. # Treat like absent (warn + continue), carrying the status for an honest message. - return EmitResult(reachable=False, status=resp.status) + return EmitResult(reachable=False, status=resp.status, token_sent=token_sent, url=self._url) if not 200 <= resp.status < 300: # 3xx (a redirect reached the client) or any remaining 4xx (notably 400): Wardline # sent a request the server would not accept — bad payload / wrong endpoint. Loud. @@ -240,6 +261,8 @@ def emit(self, findings: Sequence[Finding], *, scanned_paths: Sequence[str] = () updated=_safe_int(stats.get("findings_updated")), failed=len(failed) if isinstance(failed, list) else 0, warnings=tuple(warnings), + token_sent=token_sent, + url=self._url, ) def verify_token(self) -> ProbeResult: diff --git a/src/wardline/core/finding.py b/src/wardline/core/finding.py index a4a11289..83477826 100644 --- a/src/wardline/core/finding.py +++ b/src/wardline/core/finding.py @@ -123,7 +123,7 @@ def to_jsonl(self) -> str: "confidence": self.confidence, "related_entities": list(self.related_entities), "properties": dict(self.properties), - "suppressed": self.suppressed.value, + "suppression_state": self.suppressed.value, "suppression_reason": self.suppression_reason, "maturity": self.maturity.value, } @@ -182,7 +182,7 @@ def to_filigree_metadata(finding: Finding) -> dict[str, Any]: if finding.properties: wardline["properties"] = dict(finding.properties) if finding.suppressed is not SuppressionState.ACTIVE: - wardline["suppressed"] = finding.suppressed.value + wardline["suppression_state"] = finding.suppressed.value if finding.suppression_reason is not None: wardline["suppression_reason"] = finding.suppression_reason return {"wardline": wardline} diff --git a/src/wardline/core/legis.py b/src/wardline/core/legis.py index f793c4f5..26c0a90c 100644 --- a/src/wardline/core/legis.py +++ b/src/wardline/core/legis.py @@ -169,7 +169,7 @@ def project_finding(finding: Finding) -> dict[str, Any]: "fingerprint": wire["fingerprint"], "qualname": wire["qualname"], "properties": properties, - "suppressed": suppressed, + "suppression_state": suppressed, } diff --git a/src/wardline/core/run.py b/src/wardline/core/run.py index 47861b70..bd043788 100644 --- a/src/wardline/core/run.py +++ b/src/wardline/core/run.py @@ -31,7 +31,7 @@ from wardline.core.judged import load_judged from wardline.core.paths import baseline_path, judged_path, weft_config_path from wardline.core.protocols import Analyzer -from wardline.core.suppression import apply_suppressions, gate_trips, severity_gates +from wardline.core.suppression import SEVERITY_ORDER, apply_suppressions, gate_trips, severity_gates from wardline.core.waivers import WaiverSet, load_project_waivers if TYPE_CHECKING: @@ -52,11 +52,19 @@ class ScanSummary: baselined: int waived: int judged: int + # Every NON-DEFECT finding (facts, metrics, classifications). The defect buckets + # above (active/baselined/waived/judged) partition the DEFECTs; this is the rest, + # so the five together sum to ``total`` exactly (the buckets-sum-to-total invariant — + # weft-f506e5f845). Before this bucket existed, non-defect facts/metrics were silently + # uncounted and total != sum(buckets). + informational: int = 0 # Files DISCOVERED but NEVER analysed despite being analysable — a genuine # under-scan (parse errors, too-deep skips, missing source roots). Benign # no-module skips (WLN-ENGINE-NO-MODULE) are EXCLUDED — see UNANALYZED_RULE_IDS. # These are Severity.NONE FACTs that never trip the severity gate, so they are - # counted separately to surface a silent under-scan / false-green. + # counted separately to surface a silent under-scan / false-green. This is an + # OVERLAY (a subset of ``informational``), NOT a partition member — it is not added + # into the sum-to-total identity. unanalyzed: int = 0 @@ -82,18 +90,29 @@ class ScanResult: _SEVERITY_VALUES: frozenset[str] = frozenset(s.value for s in Severity) +_VERDICT_VALUES: frozenset[str] = frozenset({"NOT_EVALUATED", "PASSED", "FAILED"}) + + @dataclass(frozen=True, slots=True) class GateDecision: tripped: bool fail_on: str | None exit_class: int # 0 clean, 1 gate tripped, 2 reserved for tool errors (CLI layer) + # An explicit verdict so a bare scan (no --fail-on) never reads as a clean PASS: a + # vacuous green is the worst false signal for a governance suite (weft-b937e53854). + # NOT_EVALUATED — no threshold ran (fail_on is None); the gate did not judge. + # PASSED — a threshold ran and nothing tripped. + # FAILED — a threshold ran and tripped. + verdict: str # A human-readable verdict so "summary.active:0 + gate.tripped:true" never reads as - # a bug: ``reason`` names the count and class of defects that decided it (and the - # escape hatches when the trip is solely from suppressed-but-gated findings); - # ``evaluated`` names the population it judged (unsuppressed by default vs honored - # under --trust-suppressions). Both None when no threshold is set (no gate). + # a bug: ``reason`` names the count and class of defects that decided it (and, for + # NOT_EVALUATED, what WOULD trip); ``evaluated`` names the population it judged + # (unsuppressed by default vs honored under --trust-suppressions). ``would_trip_at`` is + # the highest severity at which the gate WOULD trip on that population (None if nothing + # would), computed in every branch so a bare scan still tells the agent the worst it found. reason: str | None = None evaluated: str | None = None + would_trip_at: str | None = None def __post_init__(self) -> None: # Enforce the invariants the ``gate_decision`` factory upholds so a *second* @@ -102,18 +121,23 @@ def __post_init__(self) -> None: # GateDecision value. if self.exit_class != (1 if self.tripped else 0): raise ValueError(f"exit_class {self.exit_class} contradicts tripped={self.tripped}") - # A tripped gate must always carry its verdict — never silently None. - if self.tripped and self.reason is None: - raise ValueError("a tripped gate must carry a reason") - # No threshold (fail_on None) ⟺ no verdict; a threshold always produces both. - if (self.fail_on is None) != (self.reason is None): - raise ValueError("reason must be present iff fail_on is set") - if (self.fail_on is None) != (self.evaluated is None): - raise ValueError("evaluated must be present iff fail_on is set") + if self.verdict not in _VERDICT_VALUES: + raise ValueError(f"verdict {self.verdict!r} is not one of {sorted(_VERDICT_VALUES)}") + # The verdict is keyed to the gate state — these guards are what stop a tripped gate + # from ever serialising as a pass (the dogfood #2 regression). + if (self.verdict == "NOT_EVALUATED") != (self.fail_on is None): + raise ValueError("verdict NOT_EVALUATED iff no --fail-on threshold is set") + if (self.verdict == "FAILED") != self.tripped: + raise ValueError("verdict FAILED iff the gate tripped") + # Every decision carries its reason now — including NOT_EVALUATED (what would trip). + if self.reason is None: + raise ValueError("a gate decision must always carry a reason") # fail_on is always a Severity value (the factory passes Severity.value); an - # arbitrary string satisfies the iff-guards above but is still an illegal state. + # arbitrary string satisfies the guards above but is still an illegal state. if self.fail_on is not None and self.fail_on not in _SEVERITY_VALUES: raise ValueError(f"fail_on {self.fail_on!r} is not a valid Severity value") + if self.would_trip_at is not None and self.would_trip_at not in _SEVERITY_VALUES: + raise ValueError(f"would_trip_at {self.would_trip_at!r} is not a valid Severity value") def run_scan( @@ -289,6 +313,7 @@ def apply_delta_scope(candidates: list[Finding]) -> list[Finding]: baselined=sum(1 for f in defects if f.suppressed is SuppressionState.BASELINED), waived=sum(1 for f in defects if f.suppressed is SuppressionState.WAIVED), judged=sum(1 for f in defects if f.suppressed is SuppressionState.JUDGED), + informational=len(findings) - len(defects), unanalyzed=sum(1 for f in findings if f.rule_id in UNANALYZED_RULE_IDS), ) resolved_root = root.resolve() @@ -305,30 +330,68 @@ def apply_delta_scope(candidates: list[Finding]) -> list[Finding]: ) +def _would_trip_at(gate_population: list[Finding]) -> str | None: + """The HIGHEST severity at which the gate would trip on this population, or None. + + ``gate_trips`` is monotonic in the threshold (a lower threshold catches a superset), so + the highest tripping threshold equals the max severity of any active gating defect — the + single most useful "set --fail-on X to catch the worst thing here" signal. + """ + for sev in reversed(SEVERITY_ORDER): # CRITICAL → ERROR → WARN → INFO + if gate_trips(gate_population, sev): + return sev.value + return None + + +def _not_evaluated_reason(would_trip_at: str | None, evaluated: str) -> str: + base = "no --fail-on threshold set; gate did not evaluate" + if would_trip_at is None: + return f"{base}. No active defect would trip at any threshold; evaluated {evaluated}" + return ( + f"{base}. would_trip_at {would_trip_at} — pass --fail-on {would_trip_at} (or lower) to " + f"enforce; evaluated {evaluated}" + ) + + def gate_decision(result: ScanResult, fail_on: Severity | None) -> GateDecision: """Translate a scan into a pass/fail verdict. A trip is data, not an error.""" - if fail_on is None: - return GateDecision(tripped=False, fail_on=None, exit_class=0) # None SENTINEL: evaluate the unsuppressed gate population when present (secure # default), else the suppressed ``findings`` (trusted ``--trust-suppressions`` / - # a directly-constructed ScanResult with no gate_findings). + # a directly-constructed ScanResult with no gate_findings). Population selection is + # LIFTED above the no-threshold branch so even a bare scan computes would_trip_at over + # the SAME population an actual --fail-on would judge (weft-b937e53854). honors_suppressions = result.gate_findings is None gate_population = result.findings if honors_suppressions else result.gate_findings assert gate_population is not None # narrow for mypy; the sentinel branch set findings - tripped = gate_trips(gate_population, fail_on) - sev = fail_on.value + would_trip_at = _would_trip_at(gate_population) evaluated = ( "post-suppression (repository baseline/waiver/judged honored — trusted-local)" if honors_suppressions else "unsuppressed (repository baseline/waiver/judged ignored)" ) + if fail_on is None: + # NOT a clean pass — the gate never ran. The verdict says so; would_trip_at names the + # worst severity present so the agent's first bare scan is not a false green. + return GateDecision( + tripped=False, + fail_on=None, + exit_class=0, + verdict="NOT_EVALUATED", + reason=_not_evaluated_reason(would_trip_at, evaluated), + evaluated=evaluated, + would_trip_at=would_trip_at, + ) + tripped = gate_trips(gate_population, fail_on) + sev = fail_on.value reason = _gate_reason(result, fail_on, tripped=tripped, honors_suppressions=honors_suppressions) return GateDecision( tripped=tripped, fail_on=sev, exit_class=1 if tripped else 0, + verdict="FAILED" if tripped else "PASSED", reason=reason, evaluated=evaluated, + would_trip_at=would_trip_at, ) diff --git a/src/wardline/core/scan_file_workflow.py b/src/wardline/core/scan_file_workflow.py index 29969fe1..ef1e6e56 100644 --- a/src/wardline/core/scan_file_workflow.py +++ b/src/wardline/core/scan_file_workflow.py @@ -193,9 +193,16 @@ def scan_file_findings( "baselined": result.summary.baselined, "waived": result.summary.waived, "judged": result.summary.judged, + "informational": result.summary.informational, "unanalyzed": result.summary.unanalyzed, }, - "gate": {"tripped": decision.tripped, "fail_on": decision.fail_on, "exit_class": decision.exit_class}, + "gate": { + "tripped": decision.tripped, + "fail_on": decision.fail_on, + "exit_class": decision.exit_class, + "verdict": decision.verdict, + "would_trip_at": decision.would_trip_at, + }, "filigree_emit": _emit_to_dict(emit_result, configured=filigree_emitter is not None), "active_defects": active_out, "selected_count": len(selected & known_active), diff --git a/src/wardline/install/doctor.py b/src/wardline/install/doctor.py index 45cccb46..951fba14 100644 --- a/src/wardline/install/doctor.py +++ b/src/wardline/install/doctor.py @@ -330,11 +330,14 @@ def _repair_filigree_auth(root: Path, url: str, transport: Transport) -> DoctorC if probe.reachable and probe.accepted: _rewrite_env_token(root / ".env", candidate) return DoctorCheck( - "filigree.auth", "ok", fixed=True, + "filigree.auth", + "ok", + fixed=True, message="wrote WEFT_FEDERATION_TOKEN to .env (was a stale/mismatched token)", ) return DoctorCheck( - "filigree.auth", "error", + "filigree.auth", + "error", message="no local federation_token matched the daemon — it likely uses a " "WEFT_FEDERATION_TOKEN env override; set that same value in .env", ) @@ -367,12 +370,14 @@ def _check_filigree_auth( return _repair_filigree_auth(root, url, probe_transport) if token: return DoctorCheck( - "filigree.auth", "error", + "filigree.auth", + "error", message=f"emit token rejected by filigree ({probe.status}); " "the configured token is not what the daemon accepts", ) return DoctorCheck( - "filigree.auth", "error", + "filigree.auth", + "error", message="filigree rejected an unauthenticated emit and no federation token is set; " "export WEFT_FEDERATION_TOKEN or add it to .env", ) diff --git a/src/wardline/mcp/server.py b/src/wardline/mcp/server.py index bd931312..df7ba491 100644 --- a/src/wardline/mcp/server.py +++ b/src/wardline/mcp/server.py @@ -22,7 +22,7 @@ from wardline.core.errors import WardlineError from wardline.core.explain import explain_chain, explain_finding, explanation_from_context from wardline.core.filigree_emit import FiligreeEmitter, filigree_disabled_reason -from wardline.core.finding import Finding, Kind, Severity, SuppressionState +from wardline.core.finding import Finding, Severity from wardline.core.finding_query import filter_findings from wardline.core.judge_run import run_judge from wardline.core.paths import baseline_path as baseline_file @@ -36,7 +36,6 @@ from wardline.mcp.tooling import Tool, ToolCapability, ToolError, ToolPolicy from wardline.mcp.tooling import cfg as _cfg from wardline.mcp.tooling import explanation_to_dict as _explanation_to_dict -from wardline.mcp.tooling import finding_to_dict as _finding_to_dict from wardline.mcp.tooling import require as _require from wardline.mcp.tooling import resolve_under_root as _resolve_under_root @@ -60,9 +59,12 @@ def _emit_filigree( "failed": er.failed, "warnings": list(er.warnings), # Distinguish auth-rejected (401/403) from transport-unreachable so the agent reads - # an actionable reason, not a flat "unreachable" (dogfood #5). + # an actionable reason, not a flat "unreachable" (dogfood #5). token_sent + url further + # split a 401 into no-token vs token-rejected, naming where it tried (C-7). "status": er.status, "auth_rejected": er.auth_rejected, + "token_sent": er.token_sent, + "url": er.url, } @@ -80,6 +82,8 @@ def _filigree_emit_status(block: dict[str, Any] | None) -> dict[str, Any]: disabled_reason = filigree_disabled_reason( reachable=bool(block.get("reachable")), status=block.get("status"), + token_sent=bool(block.get("token_sent")), + url=block.get("url"), ) return {"configured": True, "disabled_reason": disabled_reason, **block} @@ -255,84 +259,92 @@ def _scan( # An unknown filter key or SEI resolution failure is agent-actionable -> isError result. raise ToolError(str(exc)) from exc - # Payload-shrinking controls (dogfood #4). The `summary`/`gate` blocks always - # describe the WHOLE project; these only bound the returned finding bodies. + # Payload-shrinking controls. The `summary`/`gate` blocks always describe the WHOLE + # project; these only bound the returned finding BODIES (which live solely in + # agent_summary now — there is no separate top-level findings array). The DEFAULT scan is + # BOUNDED (weft-439d09fc8d): a bare call returns at most _DEFAULT_MAX_FINDINGS bodies so + # an agent's first natural call cannot overflow its own context. full=true lifts the cap; + # offset pages through the rest via truncation.next_offset. summary_only = _bool_arg(args, "summary_only", False) include_suppressed = _bool_arg(args, "include_suppressed", True) + full = _bool_arg(args, "full", False) max_findings = args.get("max_findings") if max_findings is not None and ( not isinstance(max_findings, int) or isinstance(max_findings, bool) or max_findings < 0 ): raise ToolError("max_findings must be a non-negative integer") + offset = args.get("offset", 0) + if not isinstance(offset, int) or isinstance(offset, bool) or offset < 0: + raise ToolError("offset must be a non-negative integer") explain = _bool_arg(args, "explain", False) - # include_suppressed:false drops the suppressed DEFECT bodies (counts stay whole). - if not include_suppressed: - selected = [f for f in selected if not (f.kind is Kind.DEFECT and f.suppressed is not SuppressionState.ACTIVE)] - findings_total = len(selected) - - # summary_only returns no finding bodies at all (the smallest "did the gate pass?" - # payload); otherwise an explicit max_findings bounds the list (default: uncapped). - display = [] if summary_only else selected - findings_truncated = False - if max_findings is not None and len(display) > max_findings: - display = display[:max_findings] - findings_truncated = True - - # explain has a DEFAULT ceiling: inlining EVERY active defect's provenance is the - # 56KB-on-one-line blowup the dogfood report hit. Cap the number of explanations (an - # explicit max_findings tightens it further); findings past the cap are still - # returned, just without inline provenance. The cut is announced, never silent. - explain_cap = max_findings if max_findings is not None else _EXPLAIN_DEFAULT_CAP - explanations_attached = 0 - explanations_truncated = False - findings_out: list[dict[str, Any]] = [] - for f in display: - d = _finding_to_dict(f) - if ( - explain - and f.kind is Kind.DEFECT - and f.suppressed is SuppressionState.ACTIVE - and f.qualname is not None - and result.context is not None - ): - if explanations_attached < explain_cap: - exp = explanation_from_context(f, result.context) - d["explanation"] = _explanation_to_dict(exp) - explanations_attached += 1 + # Effective page size: full=true → uncapped; explicit max_findings → that; else the + # bounded default. summary_only short-circuits to no bodies inside agent_summary. + if full: + limit: int | None = None + elif max_findings is not None: + limit = max_findings + else: + limit = _DEFAULT_MAX_FINDINGS + + from wardline.core.agent_summary import build_agent_summary + + agent_summary = build_agent_summary( + result, + decision, + filigree_emit=filigree_status, + loomweave_write=loomweave_status, + display_findings=selected, + summary_only=summary_only, + max_findings=limit, + offset=offset, + include_suppressed=include_suppressed, + migration_hint=migration_hint, + ).to_dict() + + # explain inlines each SHOWN active defect's provenance into its agent_summary entry (one + # call instead of an explain_taint per finding). Capped — inlining EVERY provenance is the + # 56KB-on-one-line blowup the dogfood report hit; the cut is announced in truncation. + if explain and result.context is not None: + explain_cap = max_findings if max_findings is not None else _EXPLAIN_DEFAULT_CAP + by_fp = {f.fingerprint: f for f in selected} + attached = 0 + explanations_truncated = False + for entry in agent_summary["active_defects"]: + f = by_fp.get(entry["fingerprint"]) + if f is None or f.qualname is None: + continue + if attached < explain_cap: + entry["explanation"] = _explanation_to_dict(explanation_from_context(f, result.context)) + attached += 1 else: explanations_truncated = True - findings_out.append(d) - from wardline.core.agent_summary import build_agent_summary + agent_summary["truncation"]["explanations_truncated"] = explanations_truncated response: dict[str, Any] = { "files_scanned": result.files_scanned, - "findings": findings_out, "summary": { "total": result.summary.total, "active": result.summary.active, "baselined": result.summary.baselined, "waived": result.summary.waived, "judged": result.summary.judged, + # Non-defect findings (facts/metrics/classifications). active+baselined+ + # waived+judged+informational == total (the buckets-sum-to-total invariant, + # weft-f506e5f845); unanalyzed is an overlay (subset of informational), not a + # partition member. + "informational": result.summary.informational, # Files discovered but NOT analysed (parse error / too-deep / missing # source root — benign no-module skips are excluded). Surfaced so the # silent under-scan reaches the agent, not just the human-facing stderr. "unanalyzed": result.summary.unanalyzed, }, - # Make every cut explicit so a bounded payload never reads as "covered all". - "truncation": { - "summary_only": summary_only, - "include_suppressed": include_suppressed, - "max_findings": max_findings, - "findings_total": findings_total, - "findings_returned": len(findings_out), - "findings_truncated": findings_truncated, - "explanations_truncated": explanations_truncated, - }, "gate": { "tripped": decision.tripped, "fail_on": decision.fail_on, "exit_class": decision.exit_class, + "verdict": decision.verdict, + "would_trip_at": decision.would_trip_at, "reason": decision.reason, "evaluated": decision.evaluated, "migration_hint": migration_hint, @@ -341,17 +353,7 @@ def _scan( "filigree": filigree_block, "loomweave_write": loomweave_status, "filigree_emit": filigree_status, - "agent_summary": build_agent_summary( - result, - decision, - filigree_emit=filigree_status, - loomweave_write=loomweave_status, - display_findings=selected, - summary_only=summary_only, - max_findings=max_findings, - include_suppressed=include_suppressed, - migration_hint=migration_hint, - ).to_dict(), + "agent_summary": agent_summary, } _attach_legis_artifact( response, @@ -726,6 +728,10 @@ def _fix(args: dict[str, Any], root: Path) -> dict[str, Any]: # on the MCP `scan`. Bounds the one-shot payload (the dogfood report hit 56,820 chars on # one line over a whole repo); an explicit `max_findings` tightens it further. _EXPLAIN_DEFAULT_CAP = 10 +# The bounded-default page size for `scan` (weft-439d09fc8d). A bare scan returns at most +# this many finding bodies so an agent's first natural call cannot overflow its context; +# full=true lifts the cap and offset pages through the rest. +_DEFAULT_MAX_FINDINGS = 25 class WardlineMCPServer: @@ -856,19 +862,32 @@ def _register_tools(self) -> None: "(immediate tainted callee, source boundary, trust tiers, resolution " "counts) — one call instead of an explain_taint per finding. Inlining is " "capped at 10 provenances by default (raise/lower with max_findings); the cut " - "is reported at truncation.explanations_truncated.", + "is reported at agent_summary.truncation.explanations_truncated.", }, "summary_only": { "type": "boolean", "description": "Return counts + gate only, no finding bodies — the smallest " "'did the gate pass?' payload. summary/gate still describe the whole project.", }, + "full": { + "type": "boolean", + "description": "Default false. The default scan is BOUNDED (≤25 finding bodies) so " + "it cannot overflow your context; set full=true to return ALL bodies in one call " + "(or page with offset). summary/gate counts are always whole-project.", + }, "max_findings": { "type": "integer", "minimum": 0, - "description": "Cap the number of returned finding bodies (and inlined " - "explanations). Must be a non-negative integer. The cut is reported in the " - "truncation block; summary counts stay whole-project.", + "description": "Override the page size for returned finding bodies (and the inlined-" + "explanation cap). Default 25; full=true ignores it. Must be a non-negative integer. " + "The cut + next page are reported in agent_summary.truncation; counts stay whole.", + }, + "offset": { + "type": "integer", + "minimum": 0, + "description": "Pagination cursor into the ordered finding union (active → suppressed " + "→ engine_facts). Pass agent_summary.truncation.next_offset from the previous call to " + "fetch the next page. Default 0.", }, "include_suppressed": { "type": "boolean", diff --git a/tests/conformance/test_legis_intake_contract.py b/tests/conformance/test_legis_intake_contract.py index 51aae85f..e113a9df 100644 --- a/tests/conformance/test_legis_intake_contract.py +++ b/tests/conformance/test_legis_intake_contract.py @@ -126,9 +126,13 @@ def from_wire(cls, d: Mapping[str, Any]) -> _LegisFinding: qualname = d.get("qualname") if qualname is not None and not isinstance(qualname, str): raise _LegisPayloadError("finding qualname must be a string or null") - suppressed = d.get("suppressed", "active") + # W3 (weft-f506e5f845): the per-finding wire key was renamed suppressed -> + # suppression_state. This vendored guard tracks the NEW contract; the real legis + # ingest must adopt the same key (tracked in the W3 legis weft ticket) for the live + # legis_e2e oracle to pass again. + suppressed = d.get("suppression_state", "active") if not isinstance(suppressed, str): - raise _LegisPayloadError("finding suppressed must be a string") + raise _LegisPayloadError("finding suppression_state must be a string") for key in ("rule_id", "message", "kind", "fingerprint"): if not isinstance(d[key], str) or not d[key]: raise _LegisPayloadError(f"finding {key} must be a non-empty string") @@ -318,7 +322,7 @@ def test_secure_default_gate_defect_is_enforced_by_legis(tmp_path: Path) -> None scan = wl_legis.build_legis_artifact(result, root=repo, config=cfg, key=None) # gate_findings != findings here (active vs baselined) — that asymmetry is the point. (projected,) = scan["findings"] - assert projected["suppressed"] == "active" + assert projected["suppression_state"] == "active" legis_active = active_defects(scan) assert len(legis_active) == 1 assert legis_active[0].fingerprint == "b" * 64 @@ -342,7 +346,7 @@ def test_trust_suppressions_path_projects_the_suppressed_view(tmp_path: Path) -> cfg = load_config(repo / "weft.toml") scan = wl_legis.build_legis_artifact(result, root=repo, config=cfg, key=None) (projected,) = scan["findings"] - assert projected["suppressed"] == "suppressed" + assert projected["suppression_state"] == "suppressed" assert _has_suppression_proof(projected["properties"]) assert active_defects(scan) == [] diff --git a/tests/conformance/test_mcp_handshake.py b/tests/conformance/test_mcp_handshake.py index 382439fb..81c027ae 100644 --- a/tests/conformance/test_mcp_handshake.py +++ b/tests/conformance/test_mcp_handshake.py @@ -80,7 +80,7 @@ def test_full_client_handshake_and_every_surface() -> None: call = by_id[5]["result"] assert call["content"][0]["type"] == "text" payload = json.loads(call["content"][0]["text"]) - assert {"findings", "summary", "gate"} <= set(payload) + assert {"agent_summary", "summary", "gate"} <= set(payload) # resources/read: the vocab resource round-trips non-empty text through the loop read = by_id[6]["result"] assert read["contents"][0]["uri"] == "wardline://vocab" diff --git a/tests/docs/test_glossary_vocabulary.py b/tests/docs/test_glossary_vocabulary.py index a9b086a8..1453496f 100644 --- a/tests/docs/test_glossary_vocabulary.py +++ b/tests/docs/test_glossary_vocabulary.py @@ -31,35 +31,44 @@ ("src/wardline/core/run.py", 52, "baselined: int"), ("src/wardline/core/run.py", 53, "waived: int"), ("src/wardline/core/run.py", 54, "judged: int"), - ("src/wardline/core/run.py", 60, "unanalyzed: int"), - ("src/wardline/core/run.py", 79, "gate_findings:"), - ("src/wardline/core/run.py", 86, "class GateDecision"), - ("src/wardline/core/run.py", 254, "Baseline(frozenset())"), - ("src/wardline/core/run.py", 264, "def apply_delta_scope"), - ("src/wardline/core/run.py", 288, "active=sum"), - ("src/wardline/core/run.py", 315, "honors_suppressions"), + ("src/wardline/core/run.py", 60, "informational: int"), + ("src/wardline/core/run.py", 68, "unanalyzed: int"), + ("src/wardline/core/run.py", 87, "gate_findings:"), + ("src/wardline/core/run.py", 97, "class GateDecision"), + ("src/wardline/core/run.py", 106, "verdict: str"), + ("src/wardline/core/run.py", 278, "Baseline(frozenset())"), + ("src/wardline/core/run.py", 288, "def apply_delta_scope"), + ("src/wardline/core/run.py", 312, "active=sum"), + ("src/wardline/core/run.py", 363, "honors_suppressions"), # src/wardline/cli/scan.py — CLI summary line + gate stderr - ("src/wardline/cli/scan.py", 360, "suppressed"), - ("src/wardline/cli/scan.py", 361, "{s.active} active"), - ("src/wardline/cli/scan.py", 375, "gate: FAILED"), + ("src/wardline/cli/scan.py", 362, "suppressed"), + ("src/wardline/cli/scan.py", 370, "{s.active} active"), + ("src/wardline/cli/scan.py", 388, "gate: FAILED"), # src/wardline/mcp/server.py — MCP scan summary + gate block - ("src/wardline/mcp/server.py", 312, '"total": result.summary.total'), - ("src/wardline/mcp/server.py", 313, '"active": result.summary.active'), - ("src/wardline/mcp/server.py", 314, '"baselined": result.summary.baselined'), - ("src/wardline/mcp/server.py", 315, '"waived": result.summary.waived'), - ("src/wardline/mcp/server.py", 316, '"judged": result.summary.judged'), - ("src/wardline/mcp/server.py", 320, '"unanalyzed": result.summary.unanalyzed'), - ("src/wardline/mcp/server.py", 332, '"gate": {'), - ("src/wardline/mcp/server.py", 333, '"tripped": decision.tripped'), + ("src/wardline/mcp/server.py", 327, '"total": result.summary.total'), + ("src/wardline/mcp/server.py", 328, '"active": result.summary.active'), + ("src/wardline/mcp/server.py", 329, '"baselined": result.summary.baselined'), + ("src/wardline/mcp/server.py", 330, '"waived": result.summary.waived'), + ("src/wardline/mcp/server.py", 331, '"judged": result.summary.judged'), + ("src/wardline/mcp/server.py", 336, '"informational": result.summary.informational'), + ("src/wardline/mcp/server.py", 340, '"unanalyzed": result.summary.unanalyzed'), + ("src/wardline/mcp/server.py", 342, '"gate": {'), + ("src/wardline/mcp/server.py", 343, '"tripped": decision.tripped'), + ("src/wardline/mcp/server.py", 346, '"verdict": decision.verdict'), # src/wardline/core/agent_summary.py — agent-summary JSON keys - ("src/wardline/core/agent_summary.py", 98, '"total_findings"'), - ("src/wardline/core/agent_summary.py", 99, '"active_defects"'), - ("src/wardline/core/agent_summary.py", 100, '"suppressed_findings"'), - ("src/wardline/core/agent_summary.py", 102, '"baselined"'), - ("src/wardline/core/agent_summary.py", 103, '"waived"'), - ("src/wardline/core/agent_summary.py", 104, '"judged"'), - ("src/wardline/core/agent_summary.py", 105, '"unanalyzed"'), - ("src/wardline/core/agent_summary.py", 108, '"tripped": self.gate.tripped'), + ("src/wardline/core/agent_summary.py", 116, '"total_findings"'), + ("src/wardline/core/agent_summary.py", 117, '"active_defects"'), + ("src/wardline/core/agent_summary.py", 118, '"suppressed_findings"'), + ("src/wardline/core/agent_summary.py", 120, '"baselined"'), + ("src/wardline/core/agent_summary.py", 121, '"waived"'), + ("src/wardline/core/agent_summary.py", 122, '"judged"'), + ("src/wardline/core/agent_summary.py", 123, '"informational"'), + ("src/wardline/core/agent_summary.py", 124, '"unanalyzed"'), + ("src/wardline/core/agent_summary.py", 127, '"tripped": self.gate.tripped'), + ("src/wardline/core/agent_summary.py", 130, '"verdict": self.gate.verdict'), + # per-finding suppression_state output key (renamed from `suppressed`, weft-f506e5f845) + ("src/wardline/core/finding.py", 126, '"suppression_state"'), + ("src/wardline/core/finding.py", 185, 'wardline["suppression_state"]'), # stable-file anchors (lower churn, but locked for free) ("src/wardline/core/finding.py", 68, 'ACTIVE = "active"'), ("src/wardline/core/suppression.py", 70, "SuppressionState.BASELINED"), diff --git a/tests/golden/identity/corpus/META.json b/tests/golden/identity/corpus/META.json index 909514fb..37bbb2d8 100644 --- a/tests/golden/identity/corpus/META.json +++ b/tests/golden/identity/corpus/META.json @@ -1,4 +1,4 @@ { "corpus_version": 1, - "reason": "Federation rebrand: corpus docstring brand text changed the whole-file content_hash (no identity/structure change)" + "reason": "W3: rename per-finding output key suppressed->suppression_state (weft-f506e5f845); representation refresh, fingerprints unchanged" } diff --git a/tests/golden/identity/corpus/sampleapp.json b/tests/golden/identity/corpus/sampleapp.json index 55532705..035656b1 100644 --- a/tests/golden/identity/corpus/sampleapp.json +++ b/tests/golden/identity/corpus/sampleapp.json @@ -1893,8 +1893,8 @@ "rule_id": "PY-WL-101", "severity": "ERROR", "suggestion": null, - "suppressed": "active", - "suppression_reason": null + "suppression_reason": null, + "suppression_state": "active" } ], "sarif": { diff --git a/tests/golden/identity/corpus/stress.json b/tests/golden/identity/corpus/stress.json index 87e58b5d..24f16b45 100644 --- a/tests/golden/identity/corpus/stress.json +++ b/tests/golden/identity/corpus/stress.json @@ -508,8 +508,8 @@ "rule_id": "PY-WL-111", "severity": "ERROR", "suggestion": null, - "suppressed": "active", - "suppression_reason": null + "suppression_reason": null, + "suppression_state": "active" }, { "confidence": null, @@ -533,8 +533,8 @@ "rule_id": "PY-WL-101", "severity": "ERROR", "suggestion": null, - "suppressed": "active", - "suppression_reason": null + "suppression_reason": null, + "suppression_state": "active" } ], "sarif": { diff --git a/tests/grammar/golden/builtin_findings.jsonl b/tests/grammar/golden/builtin_findings.jsonl index ccc9dfe2..076e0738 100644 --- a/tests/grammar/golden/builtin_findings.jsonl +++ b/tests/grammar/golden/builtin_findings.jsonl @@ -1,34 +1,34 @@ -{"confidence": null, "fingerprint": "97fcbb3547c624ddeb39c3c37c11fc5e5b0d2d1eda1b6465bc32ad81952aca07", "kind": "metric", "location": {"col_end": null, "col_start": null, "line_end": null, "line_start": null, "path": ""}, "maturity": "stable", "message": "L3 resolver run metrics", "properties": {"cache_hit_rate": 0.0, "convergence_iterations_histogram": [[1, 5]], "convergence_iterations_max": 1, "scc_size_distribution": [[1, 5]], "taint_source_counts": {"anchored": 39, "fallback": 5, "module_default": 0}}, "qualname": null, "related_entities": [], "rule_id": "WLN-ENGINE-METRICS", "severity": "NONE", "suggestion": null, "suppressed": "active", "suppression_reason": null} -{"confidence": null, "fingerprint": "5a0d307a403219d45d1701776b71a05840a09f9395d007b4245075ecf51dede3", "kind": "metric", "location": {"col_end": null, "col_start": null, "line_end": null, "line_start": null, "path": ""}, "maturity": "stable", "message": "Function tests.corpus.fixtures.aliased_stdlib.aliased_sink has 100% unresolved calls (1/1)", "properties": {}, "qualname": null, "related_entities": [], "rule_id": "WLN-L3-LOW-RESOLUTION", "severity": "INFO", "suggestion": null, "suppressed": "active", "suppression_reason": null} -{"confidence": null, "fingerprint": "f02a6f8ac9963cc361ba887bad5c5ef18c9a178bddebb44bc74a6bebfdc0ac55", "kind": "metric", "location": {"col_end": null, "col_start": null, "line_end": null, "line_start": null, "path": ""}, "maturity": "stable", "message": "Function tests.corpus.fixtures.aliased_stdlib.indirect_return has 100% unresolved calls (1/1)", "properties": {}, "qualname": null, "related_entities": [], "rule_id": "WLN-L3-LOW-RESOLUTION", "severity": "INFO", "suggestion": null, "suppressed": "active", "suppression_reason": null} -{"confidence": null, "fingerprint": "3afa077d8f0b2ed588e2452b98c7afd6d6634c9d1f95cba527cf4ba3d5bca7ee", "kind": "metric", "location": {"col_end": null, "col_start": null, "line_end": null, "line_start": null, "path": ""}, "maturity": "stable", "message": "Function tests.corpus.fixtures.cf_joins.validate has 100% unresolved calls (1/1)", "properties": {}, "qualname": null, "related_entities": [], "rule_id": "WLN-L3-LOW-RESOLUTION", "severity": "INFO", "suggestion": null, "suppressed": "active", "suppression_reason": null} -{"confidence": null, "fingerprint": "3eeafba571222e05ca13a57f17c5e7b7d797794c26100a924c7369bb5b8d544f", "kind": "metric", "location": {"col_end": null, "col_start": null, "line_end": null, "line_start": null, "path": ""}, "maturity": "stable", "message": "Function tests.corpus.fixtures.match_arms.validate has 100% unresolved calls (1/1)", "properties": {}, "qualname": null, "related_entities": [], "rule_id": "WLN-L3-LOW-RESOLUTION", "severity": "INFO", "suggestion": null, "suppressed": "active", "suppression_reason": null} -{"confidence": null, "fingerprint": "d1c0002b30f0e4f8a0506e830c6f05364e691b946120b92cacf80fff80c28b94", "kind": "metric", "location": {"col_end": null, "col_start": null, "line_end": null, "line_start": null, "path": ""}, "maturity": "stable", "message": "Function tests.corpus.fixtures.more_shapes.validate has 100% unresolved calls (1/1)", "properties": {}, "qualname": null, "related_entities": [], "rule_id": "WLN-L3-LOW-RESOLUTION", "severity": "INFO", "suggestion": null, "suppressed": "active", "suppression_reason": null} -{"confidence": null, "fingerprint": "1f79bbef4441f05d6c0795f8bb1c29c67611cc3cdc495a6fc34a95cd982230ef", "kind": "metric", "location": {"col_end": null, "col_start": null, "line_end": null, "line_start": null, "path": ""}, "maturity": "stable", "message": "Function tests.corpus.fixtures.validators.has_rejection has 100% unresolved calls (1/1)", "properties": {}, "qualname": null, "related_entities": [], "rule_id": "WLN-L3-LOW-RESOLUTION", "severity": "INFO", "suggestion": null, "suppressed": "active", "suppression_reason": null} -{"confidence": null, "fingerprint": "582fbe157c5771c206383f81299f572d7fb29d11551f5d4c06e824768a0d4a02", "kind": "defect", "location": {"col_end": 28, "col_start": 0, "line_end": 13, "line_start": 12, "path": "tests/corpus/fixtures/aliased_stdlib.py"}, "maturity": "stable", "message": "tests.corpus.fixtures.aliased_stdlib.aliased_sink declares return trust ASSURED but actually returns UNKNOWN_RAW (less trusted) — untrusted data reaches a trusted producer", "properties": {"actual_return": "UNKNOWN_RAW", "declared_return": "ASSURED"}, "qualname": "tests.corpus.fixtures.aliased_stdlib.aliased_sink", "related_entities": [], "rule_id": "PY-WL-101", "severity": "ERROR", "suggestion": null, "suppressed": "active", "suppression_reason": null} -{"confidence": null, "fingerprint": "c61ddc0aead2da70944cfd52a0c27e6ccb94a194513f2a5c641a843b2a906e79", "kind": "defect", "location": {"col_end": 15, "col_start": 0, "line_end": 19, "line_start": 17, "path": "tests/corpus/fixtures/aliased_stdlib.py"}, "maturity": "stable", "message": "tests.corpus.fixtures.aliased_stdlib.indirect_return declares return trust ASSURED but actually returns UNKNOWN_RAW (less trusted) — untrusted data reaches a trusted producer", "properties": {"actual_return": "UNKNOWN_RAW", "declared_return": "ASSURED"}, "qualname": "tests.corpus.fixtures.aliased_stdlib.indirect_return", "related_entities": [], "rule_id": "PY-WL-101", "severity": "ERROR", "suggestion": null, "suppressed": "active", "suppression_reason": null} -{"confidence": null, "fingerprint": "7867deb10c7761beab4baa764ea8c46780e328860174a9579d3599513818fb40", "kind": "defect", "location": {"col_end": 22, "col_start": 0, "line_end": 23, "line_start": 20, "path": "tests/corpus/fixtures/cf_joins.py"}, "maturity": "stable", "message": "tests.corpus.fixtures.cf_joins.if_branch_leaks declares return trust ASSURED but actually returns UNKNOWN_RAW (less trusted) — untrusted data reaches a trusted producer", "properties": {"actual_return": "UNKNOWN_RAW", "declared_return": "ASSURED"}, "qualname": "tests.corpus.fixtures.cf_joins.if_branch_leaks", "related_entities": [], "rule_id": "PY-WL-101", "severity": "ERROR", "suggestion": null, "suppressed": "active", "suppression_reason": null} -{"confidence": null, "fingerprint": "a7552cfd1721d7df78d2f882d7f0087bddae41c5c7cb71d21fcde50f91151009", "kind": "defect", "location": {"col_end": 26, "col_start": 0, "line_end": 31, "line_start": 27, "path": "tests/corpus/fixtures/cf_joins.py"}, "maturity": "stable", "message": "tests.corpus.fixtures.cf_joins.try_branch_leaks declares return trust ASSURED but actually returns UNKNOWN_RAW (less trusted) — untrusted data reaches a trusted producer", "properties": {"actual_return": "UNKNOWN_RAW", "declared_return": "ASSURED"}, "qualname": "tests.corpus.fixtures.cf_joins.try_branch_leaks", "related_entities": [], "rule_id": "PY-WL-101", "severity": "ERROR", "suggestion": null, "suppressed": "active", "suppression_reason": null} -{"confidence": null, "fingerprint": "b5254770c2ad34076f7f44a20a42031156f1d6e04190023126ed80334d7f668f", "kind": "defect", "location": {"col_end": 19, "col_start": 0, "line_end": 17, "line_start": 13, "path": "tests/corpus/fixtures/exceptions.py"}, "maturity": "stable", "message": "tests.corpus.fixtures.exceptions.broad_handler declares return trust INTEGRAL but actually returns UNKNOWN_RAW (less trusted) — untrusted data reaches a trusted producer", "properties": {"actual_return": "UNKNOWN_RAW", "declared_return": "INTEGRAL"}, "qualname": "tests.corpus.fixtures.exceptions.broad_handler", "related_entities": [], "rule_id": "PY-WL-101", "severity": "ERROR", "suggestion": null, "suppressed": "active", "suppression_reason": null} -{"confidence": null, "fingerprint": "58d893ebfaee68fe96b8a5930072c58fa2f6fb32e5e2fe799e10d270502fd8e7", "kind": "defect", "location": {"col_end": 15, "col_start": 0, "line_end": 26, "line_start": 21, "path": "tests/corpus/fixtures/exceptions.py"}, "maturity": "stable", "message": "tests.corpus.fixtures.exceptions.silent_handler declares return trust INTEGRAL but actually returns UNKNOWN_RAW (less trusted) — untrusted data reaches a trusted producer", "properties": {"actual_return": "UNKNOWN_RAW", "declared_return": "INTEGRAL"}, "qualname": "tests.corpus.fixtures.exceptions.silent_handler", "related_entities": [], "rule_id": "PY-WL-101", "severity": "ERROR", "suggestion": null, "suppressed": "active", "suppression_reason": null} -{"confidence": null, "fingerprint": "65882a616c5d38981743aab237001a7877b18442738c979e185838462ed74cc4", "kind": "defect", "location": {"col_end": 44, "col_start": 0, "line_end": 34, "line_start": 30, "path": "tests/corpus/fixtures/exceptions.py"}, "maturity": "stable", "message": "tests.corpus.fixtures.exceptions.narrow_logged declares return trust INTEGRAL but actually returns UNKNOWN_RAW (less trusted) — untrusted data reaches a trusted producer", "properties": {"actual_return": "UNKNOWN_RAW", "declared_return": "INTEGRAL"}, "qualname": "tests.corpus.fixtures.exceptions.narrow_logged", "related_entities": [], "rule_id": "PY-WL-101", "severity": "ERROR", "suggestion": null, "suppressed": "active", "suppression_reason": null} -{"confidence": null, "fingerprint": "b451b97cc9a21f0e0a8652de37232a2de5bb722e68e7c8353806e8a11bc872c4", "kind": "defect", "location": {"col_end": 12, "col_start": 0, "line_end": 26, "line_start": 20, "path": "tests/corpus/fixtures/match_arms.py"}, "maturity": "stable", "message": "tests.corpus.fixtures.match_arms.match_arm_leaks declares return trust ASSURED but actually returns UNKNOWN_RAW (less trusted) — untrusted data reaches a trusted producer", "properties": {"actual_return": "UNKNOWN_RAW", "declared_return": "ASSURED"}, "qualname": "tests.corpus.fixtures.match_arms.match_arm_leaks", "related_entities": [], "rule_id": "PY-WL-101", "severity": "ERROR", "suggestion": null, "suppressed": "active", "suppression_reason": null} -{"confidence": null, "fingerprint": "7bd22330169739f26817f110cfc7a944d09bb92ddec1604fd31cf947b97b2876", "kind": "defect", "location": {"col_end": 22, "col_start": 0, "line_end": 22, "line_start": 21, "path": "tests/corpus/fixtures/more_shapes.py"}, "maturity": "stable", "message": "tests.corpus.fixtures.more_shapes.direct_raw declares return trust ASSURED but actually returns UNKNOWN_RAW (less trusted) — untrusted data reaches a trusted producer", "properties": {"actual_return": "UNKNOWN_RAW", "declared_return": "ASSURED"}, "qualname": "tests.corpus.fixtures.more_shapes.direct_raw", "related_entities": [], "rule_id": "PY-WL-101", "severity": "ERROR", "suggestion": null, "suppressed": "active", "suppression_reason": null} -{"confidence": null, "fingerprint": "3ecd4fbbe6886c8a482989834496b303dad1f3302ba9e37db74521ecb20ce140", "kind": "defect", "location": {"col_end": 53, "col_start": 0, "line_end": 27, "line_start": 26, "path": "tests/corpus/fixtures/more_shapes.py"}, "maturity": "stable", "message": "tests.corpus.fixtures.more_shapes.dict_of_raw declares return trust ASSURED but actually returns UNKNOWN_RAW (less trusted) — untrusted data reaches a trusted producer", "properties": {"actual_return": "UNKNOWN_RAW", "declared_return": "ASSURED"}, "qualname": "tests.corpus.fixtures.more_shapes.dict_of_raw", "related_entities": [], "rule_id": "PY-WL-101", "severity": "ERROR", "suggestion": null, "suppressed": "active", "suppression_reason": null} -{"confidence": null, "fingerprint": "ab1266baf6c0487e4c091a6a604df2ec93caaad65fe9a19cfcd8efbf9ab82c84", "kind": "defect", "location": {"col_end": 27, "col_start": 0, "line_end": 32, "line_start": 31, "path": "tests/corpus/fixtures/more_shapes.py"}, "maturity": "stable", "message": "tests.corpus.fixtures.more_shapes.str_wrapped_raw declares return trust ASSURED but actually returns UNKNOWN_RAW (less trusted) — untrusted data reaches a trusted producer", "properties": {"actual_return": "UNKNOWN_RAW", "declared_return": "ASSURED"}, "qualname": "tests.corpus.fixtures.more_shapes.str_wrapped_raw", "related_entities": [], "rule_id": "PY-WL-101", "severity": "ERROR", "suggestion": null, "suppressed": "active", "suppression_reason": null} -{"confidence": null, "fingerprint": "ef809807c4383a2f04bb7c6445a151dfaae92d6b06ce730a9b682723b8bcdcef", "kind": "defect", "location": {"col_end": 12, "col_start": 0, "line_end": 39, "line_start": 36, "path": "tests/corpus/fixtures/more_shapes.py"}, "maturity": "stable", "message": "tests.corpus.fixtures.more_shapes.chained_indirection declares return trust ASSURED but actually returns UNKNOWN_RAW (less trusted) — untrusted data reaches a trusted producer", "properties": {"actual_return": "UNKNOWN_RAW", "declared_return": "ASSURED"}, "qualname": "tests.corpus.fixtures.more_shapes.chained_indirection", "related_entities": [], "rule_id": "PY-WL-101", "severity": "ERROR", "suggestion": null, "suppressed": "active", "suppression_reason": null} -{"confidence": null, "fingerprint": "2eb8b17bec9fd041446d1353de9917d36c2e20cd90883722c3ee9d6e5cca115a", "kind": "defect", "location": {"col_end": 25, "col_start": 0, "line_end": 45, "line_start": 43, "path": "tests/corpus/fixtures/more_shapes.py"}, "maturity": "stable", "message": "tests.corpus.fixtures.more_shapes.fstring_raw declares return trust ASSURED but actually returns UNKNOWN_RAW (less trusted) — untrusted data reaches a trusted producer", "properties": {"actual_return": "UNKNOWN_RAW", "declared_return": "ASSURED"}, "qualname": "tests.corpus.fixtures.more_shapes.fstring_raw", "related_entities": [], "rule_id": "PY-WL-101", "severity": "ERROR", "suggestion": null, "suppressed": "active", "suppression_reason": null} -{"confidence": null, "fingerprint": "7d50fefe0225e23c8650ce8064e8abc6e54c64d69cfcb181f3a4463b5f63dac3", "kind": "defect", "location": {"col_end": 37, "col_start": 0, "line_end": 50, "line_start": 49, "path": "tests/corpus/fixtures/more_shapes.py"}, "maturity": "stable", "message": "tests.corpus.fixtures.more_shapes.list_of_raw declares return trust ASSURED but actually returns UNKNOWN_RAW (less trusted) — untrusted data reaches a trusted producer", "properties": {"actual_return": "UNKNOWN_RAW", "declared_return": "ASSURED"}, "qualname": "tests.corpus.fixtures.more_shapes.list_of_raw", "related_entities": [], "rule_id": "PY-WL-101", "severity": "ERROR", "suggestion": null, "suppressed": "active", "suppression_reason": null} -{"confidence": null, "fingerprint": "787ebaa087db047519c7460540b867b00b1f0f556a029ea58de4ac13d7bca0bb", "kind": "defect", "location": {"col_end": 14, "col_start": 0, "line_end": 57, "line_start": 54, "path": "tests/corpus/fixtures/more_shapes.py"}, "maturity": "stable", "message": "tests.corpus.fixtures.more_shapes.augassign_raw declares return trust ASSURED but actually returns UNKNOWN_RAW (less trusted) — untrusted data reaches a trusted producer", "properties": {"actual_return": "UNKNOWN_RAW", "declared_return": "ASSURED"}, "qualname": "tests.corpus.fixtures.more_shapes.augassign_raw", "related_entities": [], "rule_id": "PY-WL-101", "severity": "ERROR", "suggestion": null, "suppressed": "active", "suppression_reason": null} -{"confidence": null, "fingerprint": "490fd3d5f75b21a869f73297d8ee0b6a0f6116e87fc2916be50aa7b6b81d03d2", "kind": "defect", "location": {"col_end": 22, "col_start": 0, "line_end": 65, "line_start": 61, "path": "tests/corpus/fixtures/more_shapes.py"}, "maturity": "stable", "message": "tests.corpus.fixtures.more_shapes.launders_through_broken_boundary declares return trust ASSURED but actually returns UNKNOWN_RAW (less trusted) — untrusted data reaches a trusted producer", "properties": {"actual_return": "UNKNOWN_RAW", "declared_return": "ASSURED"}, "qualname": "tests.corpus.fixtures.more_shapes.launders_through_broken_boundary", "related_entities": [], "rule_id": "PY-WL-101", "severity": "ERROR", "suggestion": null, "suppressed": "active", "suppression_reason": null} -{"confidence": null, "fingerprint": "9e97be58ccce035dba9d6a5bf276c5dffe4b2516368648b1fb05da9e8197592a", "kind": "defect", "location": {"col_end": 18, "col_start": 0, "line_end": 71, "line_start": 69, "path": "tests/corpus/fixtures/more_shapes.py"}, "maturity": "stable", "message": "tests.corpus.fixtures.more_shapes.passthrough_no_check declares a trust boundary (EXTERNAL_RAW -> ASSURED) but has no rejection path (no raise / no falsy return) — it cannot validate", "properties": {"body_taint": "EXTERNAL_RAW", "return_taint": "ASSURED"}, "qualname": "tests.corpus.fixtures.more_shapes.passthrough_no_check", "related_entities": [], "rule_id": "PY-WL-102", "severity": "ERROR", "suggestion": null, "suppressed": "active", "suppression_reason": null} -{"confidence": null, "fingerprint": "3042b0dda717f3cc0b6e0e5e798da1a4f220ab7a741c55b9655094f580e552af", "kind": "defect", "location": {"col_end": 12, "col_start": 0, "line_end": 9, "line_start": 8, "path": "tests/corpus/fixtures/validators.py"}, "maturity": "stable", "message": "tests.corpus.fixtures.validators.no_rejection declares a trust boundary (EXTERNAL_RAW -> ASSURED) but has no rejection path (no raise / no falsy return) — it cannot validate", "properties": {"body_taint": "EXTERNAL_RAW", "return_taint": "ASSURED"}, "qualname": "tests.corpus.fixtures.validators.no_rejection", "related_entities": [], "rule_id": "PY-WL-102", "severity": "ERROR", "suggestion": null, "suppressed": "active", "suppression_reason": null} -{"confidence": null, "fingerprint": "e27672ff7bc798accf1edd645163de014744e59a0eeb495a59dfeb2afc0b8826", "kind": "defect", "location": {"col_end": 12, "col_start": 0, "line_end": 14, "line_start": 13, "path": "tests/corpus/fixtures/validators.py"}, "maturity": "stable", "message": "tests.corpus.fixtures.validators.no_rejection_guarded declares a trust boundary (EXTERNAL_RAW -> GUARDED) but has no rejection path (no raise / no falsy return) — it cannot validate", "properties": {"body_taint": "EXTERNAL_RAW", "return_taint": "GUARDED"}, "qualname": "tests.corpus.fixtures.validators.no_rejection_guarded", "related_entities": [], "rule_id": "PY-WL-102", "severity": "ERROR", "suggestion": null, "suppressed": "active", "suppression_reason": null} -{"confidence": null, "fingerprint": "da2a549c6e8d7be9511bc565ae169493d5b820d6721f11ee0951bcbf5a4824d5", "kind": "defect", "location": {"col_end": null, "col_start": null, "line_end": null, "line_start": 16, "path": "tests/corpus/fixtures/exceptions.py"}, "maturity": "stable", "message": "tests.corpus.fixtures.exceptions.broad_handler: broad exception handler at line 16", "properties": {"tier": "INTEGRAL"}, "qualname": "tests.corpus.fixtures.exceptions.broad_handler", "related_entities": [], "rule_id": "PY-WL-103", "severity": "WARN", "suggestion": null, "suppressed": "active", "suppression_reason": null} -{"confidence": null, "fingerprint": "f743355684ee7af2e9bf83dd94951678ffdafcb00b74efedfb31da8489831a94", "kind": "defect", "location": {"col_end": null, "col_start": null, "line_end": null, "line_start": 24, "path": "tests/corpus/fixtures/exceptions.py"}, "maturity": "stable", "message": "tests.corpus.fixtures.exceptions.silent_handler: exception silently swallowed at line 24", "properties": {"tier": "INTEGRAL"}, "qualname": "tests.corpus.fixtures.exceptions.silent_handler", "related_entities": [], "rule_id": "PY-WL-104", "severity": "WARN", "suggestion": null, "suppressed": "active", "suppression_reason": null} -{"confidence": null, "fingerprint": "a691d4bd0b118f5d73f2cc65429b7d77c688b2b740e2ce4a383362c14c08b959", "kind": "defect", "location": {"col_end": 12, "col_start": 0, "line_end": 8, "line_start": 7, "path": "tests/corpus/fixtures/contradictory.py"}, "maturity": "stable", "message": "tests.corpus.fixtures.contradictory.conflicting carries contradictory trust markers (external_boundary+trusted); the engine resolves the clash to the least-trusted seed, silently ignoring the rest", "properties": {"markers": "external_boundary+trusted"}, "qualname": "tests.corpus.fixtures.contradictory.conflicting", "related_entities": [], "rule_id": "PY-WL-110", "severity": "WARN", "suggestion": null, "suppressed": "active", "suppression_reason": null} -{"confidence": null, "fingerprint": "385a3b2dd4dbc349dcb2656cf79ea83b6d0820faa9fe2530b9d11f357ca363e0", "kind": "defect", "location": {"col_end": 10, "col_start": 0, "line_end": 9, "line_start": 6, "path": "tests/corpus/fixtures/none_leak.py"}, "maturity": "stable", "message": "tests.corpus.fixtures.none_leak.maybe_none declares trusted return ASSURED but a path returns None (bare return / return None) — None leaks from a trusted producer", "properties": {"declared_return": "ASSURED"}, "qualname": "tests.corpus.fixtures.none_leak.maybe_none", "related_entities": [], "rule_id": "PY-WL-109", "severity": "WARN", "suggestion": null, "suppressed": "active", "suppression_reason": null} -{"confidence": null, "fingerprint": "7818cf51b8302c0dafc1317e9867645768ad59e6f16a55f4534e7f014ecd2b49", "kind": "defect", "location": {"col_end": null, "col_start": null, "line_end": null, "line_start": 16, "path": "tests/corpus/fixtures/trusted_callee.py"}, "maturity": "stable", "message": "tests.corpus.fixtures.trusted_callee.handler: EXTERNAL_RAW (untrusted) data passed to trusted producer tests.corpus.fixtures.trusted_callee.store() at line 16", "properties": {"arg_taint": "EXTERNAL_RAW", "callee": "tests.corpus.fixtures.trusted_callee.store", "callee_body": "ASSURED"}, "qualname": "tests.corpus.fixtures.trusted_callee.handler", "related_entities": [], "rule_id": "PY-WL-105", "severity": "ERROR", "suggestion": null, "suppressed": "active", "suppression_reason": null} -{"confidence": null, "fingerprint": "ea3883a7f6cd5bef0a3c95adfa9f3c9c1a68405448255d804eb5e253cb685de4", "kind": "defect", "location": {"col_end": null, "col_start": null, "line_end": null, "line_start": 15, "path": "tests/corpus/fixtures/deser_sink.py"}, "maturity": "stable", "message": "tests.corpus.fixtures.deser_sink.loads_untrusted: EXTERNAL_RAW (untrusted) data reaches the deserialization sink pickle.loads() at line 15", "properties": {"arg_taint": "EXTERNAL_RAW", "sink": "pickle.loads", "tier": "ASSURED"}, "qualname": "tests.corpus.fixtures.deser_sink.loads_untrusted", "related_entities": [], "rule_id": "PY-WL-106", "severity": "WARN", "suggestion": null, "suppressed": "active", "suppression_reason": null} -{"confidence": null, "fingerprint": "313e1050416b8b245d991c7d6ae06428e9a897c804e71b9f34bccb7e853caf89", "kind": "defect", "location": {"col_end": null, "col_start": null, "line_end": null, "line_start": 13, "path": "tests/corpus/fixtures/exec_sink.py"}, "maturity": "stable", "message": "tests.corpus.fixtures.exec_sink.evals_untrusted: EXTERNAL_RAW (untrusted) data reaches the dynamic-code-execution sink eval() at line 13", "properties": {"arg_taint": "EXTERNAL_RAW", "sink": "eval", "tier": "ASSURED"}, "qualname": "tests.corpus.fixtures.exec_sink.evals_untrusted", "related_entities": [], "rule_id": "PY-WL-107", "severity": "WARN", "suggestion": null, "suppressed": "active", "suppression_reason": null} -{"confidence": null, "fingerprint": "d2314f3580695093b23e89b87ffebc59a67ed757f3b26ea1b9a29970cd6e02b1", "kind": "defect", "location": {"col_end": null, "col_start": null, "line_end": null, "line_start": 15, "path": "tests/corpus/fixtures/command_sink.py"}, "maturity": "stable", "message": "tests.corpus.fixtures.command_sink.runs_untrusted: EXTERNAL_RAW (untrusted) data reaches the OS-command sink os.system() at line 15", "properties": {"arg_taint": "EXTERNAL_RAW", "sink": "os.system", "tier": "ASSURED"}, "qualname": "tests.corpus.fixtures.command_sink.runs_untrusted", "related_entities": [], "rule_id": "PY-WL-108", "severity": "WARN", "suggestion": null, "suppressed": "active", "suppression_reason": null} \ No newline at end of file +{"confidence": null, "fingerprint": "97fcbb3547c624ddeb39c3c37c11fc5e5b0d2d1eda1b6465bc32ad81952aca07", "kind": "metric", "location": {"col_end": null, "col_start": null, "line_end": null, "line_start": null, "path": ""}, "maturity": "stable", "message": "L3 resolver run metrics", "properties": {"cache_hit_rate": 0.0, "convergence_iterations_histogram": [[1, 5]], "convergence_iterations_max": 1, "scc_size_distribution": [[1, 5]], "taint_source_counts": {"anchored": 39, "fallback": 5, "module_default": 0}}, "qualname": null, "related_entities": [], "rule_id": "WLN-ENGINE-METRICS", "severity": "NONE", "suggestion": null, "suppression_reason": null, "suppression_state": "active"} +{"confidence": null, "fingerprint": "5a0d307a403219d45d1701776b71a05840a09f9395d007b4245075ecf51dede3", "kind": "metric", "location": {"col_end": null, "col_start": null, "line_end": null, "line_start": null, "path": ""}, "maturity": "stable", "message": "Function tests.corpus.fixtures.aliased_stdlib.aliased_sink has 100% unresolved calls (1/1)", "properties": {}, "qualname": null, "related_entities": [], "rule_id": "WLN-L3-LOW-RESOLUTION", "severity": "INFO", "suggestion": null, "suppression_reason": null, "suppression_state": "active"} +{"confidence": null, "fingerprint": "f02a6f8ac9963cc361ba887bad5c5ef18c9a178bddebb44bc74a6bebfdc0ac55", "kind": "metric", "location": {"col_end": null, "col_start": null, "line_end": null, "line_start": null, "path": ""}, "maturity": "stable", "message": "Function tests.corpus.fixtures.aliased_stdlib.indirect_return has 100% unresolved calls (1/1)", "properties": {}, "qualname": null, "related_entities": [], "rule_id": "WLN-L3-LOW-RESOLUTION", "severity": "INFO", "suggestion": null, "suppression_reason": null, "suppression_state": "active"} +{"confidence": null, "fingerprint": "3afa077d8f0b2ed588e2452b98c7afd6d6634c9d1f95cba527cf4ba3d5bca7ee", "kind": "metric", "location": {"col_end": null, "col_start": null, "line_end": null, "line_start": null, "path": ""}, "maturity": "stable", "message": "Function tests.corpus.fixtures.cf_joins.validate has 100% unresolved calls (1/1)", "properties": {}, "qualname": null, "related_entities": [], "rule_id": "WLN-L3-LOW-RESOLUTION", "severity": "INFO", "suggestion": null, "suppression_reason": null, "suppression_state": "active"} +{"confidence": null, "fingerprint": "3eeafba571222e05ca13a57f17c5e7b7d797794c26100a924c7369bb5b8d544f", "kind": "metric", "location": {"col_end": null, "col_start": null, "line_end": null, "line_start": null, "path": ""}, "maturity": "stable", "message": "Function tests.corpus.fixtures.match_arms.validate has 100% unresolved calls (1/1)", "properties": {}, "qualname": null, "related_entities": [], "rule_id": "WLN-L3-LOW-RESOLUTION", "severity": "INFO", "suggestion": null, "suppression_reason": null, "suppression_state": "active"} +{"confidence": null, "fingerprint": "d1c0002b30f0e4f8a0506e830c6f05364e691b946120b92cacf80fff80c28b94", "kind": "metric", "location": {"col_end": null, "col_start": null, "line_end": null, "line_start": null, "path": ""}, "maturity": "stable", "message": "Function tests.corpus.fixtures.more_shapes.validate has 100% unresolved calls (1/1)", "properties": {}, "qualname": null, "related_entities": [], "rule_id": "WLN-L3-LOW-RESOLUTION", "severity": "INFO", "suggestion": null, "suppression_reason": null, "suppression_state": "active"} +{"confidence": null, "fingerprint": "1f79bbef4441f05d6c0795f8bb1c29c67611cc3cdc495a6fc34a95cd982230ef", "kind": "metric", "location": {"col_end": null, "col_start": null, "line_end": null, "line_start": null, "path": ""}, "maturity": "stable", "message": "Function tests.corpus.fixtures.validators.has_rejection has 100% unresolved calls (1/1)", "properties": {}, "qualname": null, "related_entities": [], "rule_id": "WLN-L3-LOW-RESOLUTION", "severity": "INFO", "suggestion": null, "suppression_reason": null, "suppression_state": "active"} +{"confidence": null, "fingerprint": "582fbe157c5771c206383f81299f572d7fb29d11551f5d4c06e824768a0d4a02", "kind": "defect", "location": {"col_end": 28, "col_start": 0, "line_end": 13, "line_start": 12, "path": "tests/corpus/fixtures/aliased_stdlib.py"}, "maturity": "stable", "message": "tests.corpus.fixtures.aliased_stdlib.aliased_sink declares return trust ASSURED but actually returns UNKNOWN_RAW (less trusted) — untrusted data reaches a trusted producer", "properties": {"actual_return": "UNKNOWN_RAW", "declared_return": "ASSURED"}, "qualname": "tests.corpus.fixtures.aliased_stdlib.aliased_sink", "related_entities": [], "rule_id": "PY-WL-101", "severity": "ERROR", "suggestion": null, "suppression_reason": null, "suppression_state": "active"} +{"confidence": null, "fingerprint": "c61ddc0aead2da70944cfd52a0c27e6ccb94a194513f2a5c641a843b2a906e79", "kind": "defect", "location": {"col_end": 15, "col_start": 0, "line_end": 19, "line_start": 17, "path": "tests/corpus/fixtures/aliased_stdlib.py"}, "maturity": "stable", "message": "tests.corpus.fixtures.aliased_stdlib.indirect_return declares return trust ASSURED but actually returns UNKNOWN_RAW (less trusted) — untrusted data reaches a trusted producer", "properties": {"actual_return": "UNKNOWN_RAW", "declared_return": "ASSURED"}, "qualname": "tests.corpus.fixtures.aliased_stdlib.indirect_return", "related_entities": [], "rule_id": "PY-WL-101", "severity": "ERROR", "suggestion": null, "suppression_reason": null, "suppression_state": "active"} +{"confidence": null, "fingerprint": "7867deb10c7761beab4baa764ea8c46780e328860174a9579d3599513818fb40", "kind": "defect", "location": {"col_end": 22, "col_start": 0, "line_end": 23, "line_start": 20, "path": "tests/corpus/fixtures/cf_joins.py"}, "maturity": "stable", "message": "tests.corpus.fixtures.cf_joins.if_branch_leaks declares return trust ASSURED but actually returns UNKNOWN_RAW (less trusted) — untrusted data reaches a trusted producer", "properties": {"actual_return": "UNKNOWN_RAW", "declared_return": "ASSURED"}, "qualname": "tests.corpus.fixtures.cf_joins.if_branch_leaks", "related_entities": [], "rule_id": "PY-WL-101", "severity": "ERROR", "suggestion": null, "suppression_reason": null, "suppression_state": "active"} +{"confidence": null, "fingerprint": "a7552cfd1721d7df78d2f882d7f0087bddae41c5c7cb71d21fcde50f91151009", "kind": "defect", "location": {"col_end": 26, "col_start": 0, "line_end": 31, "line_start": 27, "path": "tests/corpus/fixtures/cf_joins.py"}, "maturity": "stable", "message": "tests.corpus.fixtures.cf_joins.try_branch_leaks declares return trust ASSURED but actually returns UNKNOWN_RAW (less trusted) — untrusted data reaches a trusted producer", "properties": {"actual_return": "UNKNOWN_RAW", "declared_return": "ASSURED"}, "qualname": "tests.corpus.fixtures.cf_joins.try_branch_leaks", "related_entities": [], "rule_id": "PY-WL-101", "severity": "ERROR", "suggestion": null, "suppression_reason": null, "suppression_state": "active"} +{"confidence": null, "fingerprint": "b5254770c2ad34076f7f44a20a42031156f1d6e04190023126ed80334d7f668f", "kind": "defect", "location": {"col_end": 19, "col_start": 0, "line_end": 17, "line_start": 13, "path": "tests/corpus/fixtures/exceptions.py"}, "maturity": "stable", "message": "tests.corpus.fixtures.exceptions.broad_handler declares return trust INTEGRAL but actually returns UNKNOWN_RAW (less trusted) — untrusted data reaches a trusted producer", "properties": {"actual_return": "UNKNOWN_RAW", "declared_return": "INTEGRAL"}, "qualname": "tests.corpus.fixtures.exceptions.broad_handler", "related_entities": [], "rule_id": "PY-WL-101", "severity": "ERROR", "suggestion": null, "suppression_reason": null, "suppression_state": "active"} +{"confidence": null, "fingerprint": "58d893ebfaee68fe96b8a5930072c58fa2f6fb32e5e2fe799e10d270502fd8e7", "kind": "defect", "location": {"col_end": 15, "col_start": 0, "line_end": 26, "line_start": 21, "path": "tests/corpus/fixtures/exceptions.py"}, "maturity": "stable", "message": "tests.corpus.fixtures.exceptions.silent_handler declares return trust INTEGRAL but actually returns UNKNOWN_RAW (less trusted) — untrusted data reaches a trusted producer", "properties": {"actual_return": "UNKNOWN_RAW", "declared_return": "INTEGRAL"}, "qualname": "tests.corpus.fixtures.exceptions.silent_handler", "related_entities": [], "rule_id": "PY-WL-101", "severity": "ERROR", "suggestion": null, "suppression_reason": null, "suppression_state": "active"} +{"confidence": null, "fingerprint": "65882a616c5d38981743aab237001a7877b18442738c979e185838462ed74cc4", "kind": "defect", "location": {"col_end": 44, "col_start": 0, "line_end": 34, "line_start": 30, "path": "tests/corpus/fixtures/exceptions.py"}, "maturity": "stable", "message": "tests.corpus.fixtures.exceptions.narrow_logged declares return trust INTEGRAL but actually returns UNKNOWN_RAW (less trusted) — untrusted data reaches a trusted producer", "properties": {"actual_return": "UNKNOWN_RAW", "declared_return": "INTEGRAL"}, "qualname": "tests.corpus.fixtures.exceptions.narrow_logged", "related_entities": [], "rule_id": "PY-WL-101", "severity": "ERROR", "suggestion": null, "suppression_reason": null, "suppression_state": "active"} +{"confidence": null, "fingerprint": "b451b97cc9a21f0e0a8652de37232a2de5bb722e68e7c8353806e8a11bc872c4", "kind": "defect", "location": {"col_end": 12, "col_start": 0, "line_end": 26, "line_start": 20, "path": "tests/corpus/fixtures/match_arms.py"}, "maturity": "stable", "message": "tests.corpus.fixtures.match_arms.match_arm_leaks declares return trust ASSURED but actually returns UNKNOWN_RAW (less trusted) — untrusted data reaches a trusted producer", "properties": {"actual_return": "UNKNOWN_RAW", "declared_return": "ASSURED"}, "qualname": "tests.corpus.fixtures.match_arms.match_arm_leaks", "related_entities": [], "rule_id": "PY-WL-101", "severity": "ERROR", "suggestion": null, "suppression_reason": null, "suppression_state": "active"} +{"confidence": null, "fingerprint": "7bd22330169739f26817f110cfc7a944d09bb92ddec1604fd31cf947b97b2876", "kind": "defect", "location": {"col_end": 22, "col_start": 0, "line_end": 22, "line_start": 21, "path": "tests/corpus/fixtures/more_shapes.py"}, "maturity": "stable", "message": "tests.corpus.fixtures.more_shapes.direct_raw declares return trust ASSURED but actually returns UNKNOWN_RAW (less trusted) — untrusted data reaches a trusted producer", "properties": {"actual_return": "UNKNOWN_RAW", "declared_return": "ASSURED"}, "qualname": "tests.corpus.fixtures.more_shapes.direct_raw", "related_entities": [], "rule_id": "PY-WL-101", "severity": "ERROR", "suggestion": null, "suppression_reason": null, "suppression_state": "active"} +{"confidence": null, "fingerprint": "3ecd4fbbe6886c8a482989834496b303dad1f3302ba9e37db74521ecb20ce140", "kind": "defect", "location": {"col_end": 53, "col_start": 0, "line_end": 27, "line_start": 26, "path": "tests/corpus/fixtures/more_shapes.py"}, "maturity": "stable", "message": "tests.corpus.fixtures.more_shapes.dict_of_raw declares return trust ASSURED but actually returns UNKNOWN_RAW (less trusted) — untrusted data reaches a trusted producer", "properties": {"actual_return": "UNKNOWN_RAW", "declared_return": "ASSURED"}, "qualname": "tests.corpus.fixtures.more_shapes.dict_of_raw", "related_entities": [], "rule_id": "PY-WL-101", "severity": "ERROR", "suggestion": null, "suppression_reason": null, "suppression_state": "active"} +{"confidence": null, "fingerprint": "ab1266baf6c0487e4c091a6a604df2ec93caaad65fe9a19cfcd8efbf9ab82c84", "kind": "defect", "location": {"col_end": 27, "col_start": 0, "line_end": 32, "line_start": 31, "path": "tests/corpus/fixtures/more_shapes.py"}, "maturity": "stable", "message": "tests.corpus.fixtures.more_shapes.str_wrapped_raw declares return trust ASSURED but actually returns UNKNOWN_RAW (less trusted) — untrusted data reaches a trusted producer", "properties": {"actual_return": "UNKNOWN_RAW", "declared_return": "ASSURED"}, "qualname": "tests.corpus.fixtures.more_shapes.str_wrapped_raw", "related_entities": [], "rule_id": "PY-WL-101", "severity": "ERROR", "suggestion": null, "suppression_reason": null, "suppression_state": "active"} +{"confidence": null, "fingerprint": "ef809807c4383a2f04bb7c6445a151dfaae92d6b06ce730a9b682723b8bcdcef", "kind": "defect", "location": {"col_end": 12, "col_start": 0, "line_end": 39, "line_start": 36, "path": "tests/corpus/fixtures/more_shapes.py"}, "maturity": "stable", "message": "tests.corpus.fixtures.more_shapes.chained_indirection declares return trust ASSURED but actually returns UNKNOWN_RAW (less trusted) — untrusted data reaches a trusted producer", "properties": {"actual_return": "UNKNOWN_RAW", "declared_return": "ASSURED"}, "qualname": "tests.corpus.fixtures.more_shapes.chained_indirection", "related_entities": [], "rule_id": "PY-WL-101", "severity": "ERROR", "suggestion": null, "suppression_reason": null, "suppression_state": "active"} +{"confidence": null, "fingerprint": "2eb8b17bec9fd041446d1353de9917d36c2e20cd90883722c3ee9d6e5cca115a", "kind": "defect", "location": {"col_end": 25, "col_start": 0, "line_end": 45, "line_start": 43, "path": "tests/corpus/fixtures/more_shapes.py"}, "maturity": "stable", "message": "tests.corpus.fixtures.more_shapes.fstring_raw declares return trust ASSURED but actually returns UNKNOWN_RAW (less trusted) — untrusted data reaches a trusted producer", "properties": {"actual_return": "UNKNOWN_RAW", "declared_return": "ASSURED"}, "qualname": "tests.corpus.fixtures.more_shapes.fstring_raw", "related_entities": [], "rule_id": "PY-WL-101", "severity": "ERROR", "suggestion": null, "suppression_reason": null, "suppression_state": "active"} +{"confidence": null, "fingerprint": "7d50fefe0225e23c8650ce8064e8abc6e54c64d69cfcb181f3a4463b5f63dac3", "kind": "defect", "location": {"col_end": 37, "col_start": 0, "line_end": 50, "line_start": 49, "path": "tests/corpus/fixtures/more_shapes.py"}, "maturity": "stable", "message": "tests.corpus.fixtures.more_shapes.list_of_raw declares return trust ASSURED but actually returns UNKNOWN_RAW (less trusted) — untrusted data reaches a trusted producer", "properties": {"actual_return": "UNKNOWN_RAW", "declared_return": "ASSURED"}, "qualname": "tests.corpus.fixtures.more_shapes.list_of_raw", "related_entities": [], "rule_id": "PY-WL-101", "severity": "ERROR", "suggestion": null, "suppression_reason": null, "suppression_state": "active"} +{"confidence": null, "fingerprint": "787ebaa087db047519c7460540b867b00b1f0f556a029ea58de4ac13d7bca0bb", "kind": "defect", "location": {"col_end": 14, "col_start": 0, "line_end": 57, "line_start": 54, "path": "tests/corpus/fixtures/more_shapes.py"}, "maturity": "stable", "message": "tests.corpus.fixtures.more_shapes.augassign_raw declares return trust ASSURED but actually returns UNKNOWN_RAW (less trusted) — untrusted data reaches a trusted producer", "properties": {"actual_return": "UNKNOWN_RAW", "declared_return": "ASSURED"}, "qualname": "tests.corpus.fixtures.more_shapes.augassign_raw", "related_entities": [], "rule_id": "PY-WL-101", "severity": "ERROR", "suggestion": null, "suppression_reason": null, "suppression_state": "active"} +{"confidence": null, "fingerprint": "490fd3d5f75b21a869f73297d8ee0b6a0f6116e87fc2916be50aa7b6b81d03d2", "kind": "defect", "location": {"col_end": 22, "col_start": 0, "line_end": 65, "line_start": 61, "path": "tests/corpus/fixtures/more_shapes.py"}, "maturity": "stable", "message": "tests.corpus.fixtures.more_shapes.launders_through_broken_boundary declares return trust ASSURED but actually returns UNKNOWN_RAW (less trusted) — untrusted data reaches a trusted producer", "properties": {"actual_return": "UNKNOWN_RAW", "declared_return": "ASSURED"}, "qualname": "tests.corpus.fixtures.more_shapes.launders_through_broken_boundary", "related_entities": [], "rule_id": "PY-WL-101", "severity": "ERROR", "suggestion": null, "suppression_reason": null, "suppression_state": "active"} +{"confidence": null, "fingerprint": "9e97be58ccce035dba9d6a5bf276c5dffe4b2516368648b1fb05da9e8197592a", "kind": "defect", "location": {"col_end": 18, "col_start": 0, "line_end": 71, "line_start": 69, "path": "tests/corpus/fixtures/more_shapes.py"}, "maturity": "stable", "message": "tests.corpus.fixtures.more_shapes.passthrough_no_check declares a trust boundary (EXTERNAL_RAW -> ASSURED) but has no rejection path (no raise / no falsy return) — it cannot validate", "properties": {"body_taint": "EXTERNAL_RAW", "return_taint": "ASSURED"}, "qualname": "tests.corpus.fixtures.more_shapes.passthrough_no_check", "related_entities": [], "rule_id": "PY-WL-102", "severity": "ERROR", "suggestion": null, "suppression_reason": null, "suppression_state": "active"} +{"confidence": null, "fingerprint": "3042b0dda717f3cc0b6e0e5e798da1a4f220ab7a741c55b9655094f580e552af", "kind": "defect", "location": {"col_end": 12, "col_start": 0, "line_end": 9, "line_start": 8, "path": "tests/corpus/fixtures/validators.py"}, "maturity": "stable", "message": "tests.corpus.fixtures.validators.no_rejection declares a trust boundary (EXTERNAL_RAW -> ASSURED) but has no rejection path (no raise / no falsy return) — it cannot validate", "properties": {"body_taint": "EXTERNAL_RAW", "return_taint": "ASSURED"}, "qualname": "tests.corpus.fixtures.validators.no_rejection", "related_entities": [], "rule_id": "PY-WL-102", "severity": "ERROR", "suggestion": null, "suppression_reason": null, "suppression_state": "active"} +{"confidence": null, "fingerprint": "e27672ff7bc798accf1edd645163de014744e59a0eeb495a59dfeb2afc0b8826", "kind": "defect", "location": {"col_end": 12, "col_start": 0, "line_end": 14, "line_start": 13, "path": "tests/corpus/fixtures/validators.py"}, "maturity": "stable", "message": "tests.corpus.fixtures.validators.no_rejection_guarded declares a trust boundary (EXTERNAL_RAW -> GUARDED) but has no rejection path (no raise / no falsy return) — it cannot validate", "properties": {"body_taint": "EXTERNAL_RAW", "return_taint": "GUARDED"}, "qualname": "tests.corpus.fixtures.validators.no_rejection_guarded", "related_entities": [], "rule_id": "PY-WL-102", "severity": "ERROR", "suggestion": null, "suppression_reason": null, "suppression_state": "active"} +{"confidence": null, "fingerprint": "da2a549c6e8d7be9511bc565ae169493d5b820d6721f11ee0951bcbf5a4824d5", "kind": "defect", "location": {"col_end": null, "col_start": null, "line_end": null, "line_start": 16, "path": "tests/corpus/fixtures/exceptions.py"}, "maturity": "stable", "message": "tests.corpus.fixtures.exceptions.broad_handler: broad exception handler at line 16", "properties": {"tier": "INTEGRAL"}, "qualname": "tests.corpus.fixtures.exceptions.broad_handler", "related_entities": [], "rule_id": "PY-WL-103", "severity": "WARN", "suggestion": null, "suppression_reason": null, "suppression_state": "active"} +{"confidence": null, "fingerprint": "f743355684ee7af2e9bf83dd94951678ffdafcb00b74efedfb31da8489831a94", "kind": "defect", "location": {"col_end": null, "col_start": null, "line_end": null, "line_start": 24, "path": "tests/corpus/fixtures/exceptions.py"}, "maturity": "stable", "message": "tests.corpus.fixtures.exceptions.silent_handler: exception silently swallowed at line 24", "properties": {"tier": "INTEGRAL"}, "qualname": "tests.corpus.fixtures.exceptions.silent_handler", "related_entities": [], "rule_id": "PY-WL-104", "severity": "WARN", "suggestion": null, "suppression_reason": null, "suppression_state": "active"} +{"confidence": null, "fingerprint": "a691d4bd0b118f5d73f2cc65429b7d77c688b2b740e2ce4a383362c14c08b959", "kind": "defect", "location": {"col_end": 12, "col_start": 0, "line_end": 8, "line_start": 7, "path": "tests/corpus/fixtures/contradictory.py"}, "maturity": "stable", "message": "tests.corpus.fixtures.contradictory.conflicting carries contradictory trust markers (external_boundary+trusted); the engine resolves the clash to the least-trusted seed, silently ignoring the rest", "properties": {"markers": "external_boundary+trusted"}, "qualname": "tests.corpus.fixtures.contradictory.conflicting", "related_entities": [], "rule_id": "PY-WL-110", "severity": "WARN", "suggestion": null, "suppression_reason": null, "suppression_state": "active"} +{"confidence": null, "fingerprint": "385a3b2dd4dbc349dcb2656cf79ea83b6d0820faa9fe2530b9d11f357ca363e0", "kind": "defect", "location": {"col_end": 10, "col_start": 0, "line_end": 9, "line_start": 6, "path": "tests/corpus/fixtures/none_leak.py"}, "maturity": "stable", "message": "tests.corpus.fixtures.none_leak.maybe_none declares trusted return ASSURED but a path returns None (bare return / return None) — None leaks from a trusted producer", "properties": {"declared_return": "ASSURED"}, "qualname": "tests.corpus.fixtures.none_leak.maybe_none", "related_entities": [], "rule_id": "PY-WL-109", "severity": "WARN", "suggestion": null, "suppression_reason": null, "suppression_state": "active"} +{"confidence": null, "fingerprint": "7818cf51b8302c0dafc1317e9867645768ad59e6f16a55f4534e7f014ecd2b49", "kind": "defect", "location": {"col_end": null, "col_start": null, "line_end": null, "line_start": 16, "path": "tests/corpus/fixtures/trusted_callee.py"}, "maturity": "stable", "message": "tests.corpus.fixtures.trusted_callee.handler: EXTERNAL_RAW (untrusted) data passed to trusted producer tests.corpus.fixtures.trusted_callee.store() at line 16", "properties": {"arg_taint": "EXTERNAL_RAW", "callee": "tests.corpus.fixtures.trusted_callee.store", "callee_body": "ASSURED"}, "qualname": "tests.corpus.fixtures.trusted_callee.handler", "related_entities": [], "rule_id": "PY-WL-105", "severity": "ERROR", "suggestion": null, "suppression_reason": null, "suppression_state": "active"} +{"confidence": null, "fingerprint": "ea3883a7f6cd5bef0a3c95adfa9f3c9c1a68405448255d804eb5e253cb685de4", "kind": "defect", "location": {"col_end": null, "col_start": null, "line_end": null, "line_start": 15, "path": "tests/corpus/fixtures/deser_sink.py"}, "maturity": "stable", "message": "tests.corpus.fixtures.deser_sink.loads_untrusted: EXTERNAL_RAW (untrusted) data reaches the deserialization sink pickle.loads() at line 15", "properties": {"arg_taint": "EXTERNAL_RAW", "sink": "pickle.loads", "tier": "ASSURED"}, "qualname": "tests.corpus.fixtures.deser_sink.loads_untrusted", "related_entities": [], "rule_id": "PY-WL-106", "severity": "WARN", "suggestion": null, "suppression_reason": null, "suppression_state": "active"} +{"confidence": null, "fingerprint": "313e1050416b8b245d991c7d6ae06428e9a897c804e71b9f34bccb7e853caf89", "kind": "defect", "location": {"col_end": null, "col_start": null, "line_end": null, "line_start": 13, "path": "tests/corpus/fixtures/exec_sink.py"}, "maturity": "stable", "message": "tests.corpus.fixtures.exec_sink.evals_untrusted: EXTERNAL_RAW (untrusted) data reaches the dynamic-code-execution sink eval() at line 13", "properties": {"arg_taint": "EXTERNAL_RAW", "sink": "eval", "tier": "ASSURED"}, "qualname": "tests.corpus.fixtures.exec_sink.evals_untrusted", "related_entities": [], "rule_id": "PY-WL-107", "severity": "WARN", "suggestion": null, "suppression_reason": null, "suppression_state": "active"} +{"confidence": null, "fingerprint": "d2314f3580695093b23e89b87ffebc59a67ed757f3b26ea1b9a29970cd6e02b1", "kind": "defect", "location": {"col_end": null, "col_start": null, "line_end": null, "line_start": 15, "path": "tests/corpus/fixtures/command_sink.py"}, "maturity": "stable", "message": "tests.corpus.fixtures.command_sink.runs_untrusted: EXTERNAL_RAW (untrusted) data reaches the OS-command sink os.system() at line 15", "properties": {"arg_taint": "EXTERNAL_RAW", "sink": "os.system", "tier": "ASSURED"}, "qualname": "tests.corpus.fixtures.command_sink.runs_untrusted", "related_entities": [], "rule_id": "PY-WL-108", "severity": "WARN", "suggestion": null, "suppression_reason": null, "suppression_state": "active"} \ No newline at end of file diff --git a/tests/unit/cli/test_cli.py b/tests/unit/cli/test_cli.py index 38907eed..f9a6df25 100644 --- a/tests/unit/cli/test_cli.py +++ b/tests/unit/cli/test_cli.py @@ -462,7 +462,7 @@ def test_scan_baseline_annotates_but_does_not_clear_gate(tmp_path) -> None: assert res.exit_code == 1, res.output findings2 = [_json.loads(ln) for ln in out.read_text().splitlines() if ln.strip()] leak = next(f for f in findings2 if f["rule_id"] == "PY-WL-101") - assert leak["suppressed"] == "baselined" # annotate-and-keep + assert leak["suppression_state"] == "baselined" # annotate-and-keep assert "1 suppressed" in res.output diff --git a/tests/unit/core/test_agent_summary.py b/tests/unit/core/test_agent_summary.py index c3ada084..12857f6f 100644 --- a/tests/unit/core/test_agent_summary.py +++ b/tests/unit/core/test_agent_summary.py @@ -108,7 +108,11 @@ def test_agent_summary_no_active_defects_still_has_next_actions(tmp_path: Path) out = build_agent_summary(scan, gate_decision(scan, None)).to_dict() assert out["active_defects"] == [] - assert out["next_actions"] == [{"tool": "scan", "reason": "no active defects; rescan after edits"}] + # A bare scan is NOT_EVALUATED (weft-b937e53854): next_actions point at enforcing a + # threshold, never the bland "rescan after edits" that reads as a pass. + assert len(out["next_actions"]) == 1 + reason = out["next_actions"][0]["reason"].lower() + assert "not_evaluated" in reason and "--fail-on" in reason def test_agent_summary_next_actions_do_not_say_passed_when_gate_tripped(tmp_path: Path) -> None: diff --git a/tests/unit/core/test_cli_mcp_parity.py b/tests/unit/core/test_cli_mcp_parity.py index 4ec93e83..4de37586 100644 --- a/tests/unit/core/test_cli_mcp_parity.py +++ b/tests/unit/core/test_cli_mcp_parity.py @@ -16,28 +16,37 @@ from pathlib import Path +from wardline.core.agent_summary import build_agent_summary from wardline.core.finding import Severity from wardline.core.run import baseline_migration_hint, gate_decision, run_scan -from wardline.mcp.server import _finding_to_dict, _scan +from wardline.mcp.server import _scan _CORPUS = Path(__file__).resolve().parents[3] / "tests" / "corpus" / "fixtures" def test_cli_and_mcp_scan_agree_on_findings_and_gate() -> None: - # Shared scan default: confine_to_root=True. + # Shared scan default: confine_to_root=True. The finding BODIES live solely in + # agent_summary now (the bloat-causing top-level `findings` array was removed, W1). + # The CLI `--format agent-summary` is uncapped; MCP defaults bounded, so parity is + # asserted against MCP full=true (engine parity preserved; only the default page size + # differs by surface, by design). cli_result = run_scan(_CORPUS) - cli_findings = [_finding_to_dict(f) for f in cli_result.findings] cli_gate = gate_decision(cli_result, Severity.ERROR) + cli_ag = build_agent_summary(cli_result, cli_gate).to_dict() # MCP parameterization: the real _scan handler (confine_to_root=True, no loomweave). - mcp = _scan({"fail_on": "ERROR"}, root=_CORPUS) + mcp = _scan({"fail_on": "ERROR", "full": True}, root=_CORPUS) + mcp_ag = mcp["agent_summary"] - assert mcp["findings"] == cli_findings + for key in ("active_defects", "suppressed_findings", "engine_facts"): + assert mcp_ag[key] == cli_ag[key], key cli_hint = baseline_migration_hint(cli_result, cli_gate, root=_CORPUS, new_since=None) assert mcp["gate"] == { "tripped": cli_gate.tripped, "fail_on": cli_gate.fail_on, "exit_class": cli_gate.exit_class, + "verdict": cli_gate.verdict, + "would_trip_at": cli_gate.would_trip_at, "reason": cli_gate.reason, "evaluated": cli_gate.evaluated, "migration_hint": cli_hint, @@ -46,7 +55,7 @@ def test_cli_and_mcp_scan_agree_on_findings_and_gate() -> None: assert mcp["summary"]["active"] == cli_result.summary.active assert mcp["files_scanned"] == cli_result.files_scanned # Sanity: the labeled corpus is a non-trivial substrate (it fires real defects). - assert any(f["kind"] == "defect" for f in cli_findings) + assert any(e["kind"] == "defect" for e in cli_ag["active_defects"]) def test_cli_and_mcp_emit_identical_filigree_body() -> None: diff --git a/tests/unit/core/test_filigree_emit.py b/tests/unit/core/test_filigree_emit.py index a9c90a24..4ca345b2 100644 --- a/tests/unit/core/test_filigree_emit.py +++ b/tests/unit/core/test_filigree_emit.py @@ -10,6 +10,7 @@ FiligreeEmitter, Response, build_scan_results_body, + filigree_disabled_reason, ) from wardline.core.finding import Finding, Kind, Location, Severity, SuppressionState @@ -72,7 +73,7 @@ def test_fingerprint_is_top_level_and_severity_lowercased() -> None: def test_metadata_namespaced_and_carries_suppression() -> None: wire = build_scan_results_body([_f(suppressed=SuppressionState.WAIVED, suppression_reason="fp")])["findings"][0] assert set(wire["metadata"]) == {"wardline"} - assert wire["metadata"]["wardline"]["suppressed"] == "waived" + assert wire["metadata"]["wardline"]["suppression_state"] == "waived" assert wire["metadata"]["wardline"]["suppression_reason"] == "fp" @@ -214,6 +215,47 @@ def test_no_authorization_header_when_no_token() -> None: assert "Authorization" not in t.calls[0][2] +# --- C-7: token-absent vs token-rejected (weft-23574069a1) ------------------- + + +def test_emit_stamps_token_sent_and_url() -> None: + url = "http://x/api/weft/scan-results" + t = _FakeTransport(response=Response(status=401, body="no")) + with_token = FiligreeEmitter(url, transport=t, token="wrong").emit([_f()]) + assert with_token.token_sent is True and with_token.url == url + t2 = _FakeTransport(response=Response(status=401, body="no")) + no_token = FiligreeEmitter(url, transport=t2).emit([_f()]) + assert no_token.token_sent is False and no_token.url == url + # success path also stamps token_sent + url + t3 = _FakeTransport(response=Response(status=200, body=_ok_body())) + ok = FiligreeEmitter(url, transport=t3, token="good").emit([_f()]) + assert ok.token_sent is True and ok.url == url + + +def test_disabled_reason_401_distinguishes_no_token_from_rejected() -> None: + url = "http://h/api/weft/scan-results" + # A token WAS sent and rejected — say the value is wrong, not "set a token" (the C-7 + # misdiagnosis). Names the URL it tried. + rejected = filigree_disabled_reason(reachable=False, status=401, token_sent=True, url=url) + assert rejected is not None + assert "401" in rejected and "wrong" in rejected and url in rejected + assert "no token sent" not in rejected + # No token sent — that is the "set WEFT_FEDERATION_TOKEN" case. + absent = filigree_disabled_reason(reachable=False, status=401, token_sent=False, url=url) + assert absent is not None + assert "no token sent" in absent and "WEFT_FEDERATION_TOKEN" in absent and url in absent + + +def test_disabled_reason_403_and_unreachable_unchanged_in_shape() -> None: + url = "http://h/api/weft/scan-results" + forbidden = filigree_disabled_reason(reachable=False, status=403, token_sent=True, url=url) + assert forbidden is not None and "403" in forbidden and "lacks access" in forbidden + unreachable = filigree_disabled_reason(reachable=False, status=None, token_sent=False, url=url) + assert unreachable is not None and "unreachable" in unreachable and url in unreachable + # reached/success -> no disabled_reason + assert filigree_disabled_reason(reachable=True, status=None) is None + + def test_2xx_with_unparseable_body_warns_not_crashes() -> None: # POST accepted (2xx) but the body is not a JSON object -> surface a warning, # reachable=True, zeroed stats; must NOT raise. @@ -328,5 +370,5 @@ def test_judged_finding_carries_suppression_metadata() -> None: wire = build_scan_results_body([_f(suppressed=SuppressionState.JUDGED, suppression_reason="over-taint floor")])[ "findings" ][0] - assert wire["metadata"]["wardline"]["suppressed"] == "judged" + assert wire["metadata"]["wardline"]["suppression_state"] == "judged" assert wire["metadata"]["wardline"]["suppression_reason"] == "over-taint floor" diff --git a/tests/unit/core/test_finding.py b/tests/unit/core/test_finding.py index 7c1c8ee4..606cd8ae 100644 --- a/tests/unit/core/test_finding.py +++ b/tests/unit/core/test_finding.py @@ -87,13 +87,13 @@ def test_suppressed_serializes_in_jsonl() -> None: f = _finding(suppressed=SuppressionState.WAIVED, suppression_reason="reviewed") obj = json.loads(f.to_jsonl()) - assert obj["suppressed"] == "waived" + assert obj["suppression_state"] == "waived" assert obj["suppression_reason"] == "reviewed" def test_active_suppression_serializes_too() -> None: obj = json.loads(_finding().to_jsonl()) - assert obj["suppressed"] == "active" + assert obj["suppression_state"] == "active" assert obj["suppression_reason"] is None @@ -112,7 +112,7 @@ def test_filigree_metadata_includes_suppression_only_when_suppressed() -> None: from wardline.core.finding import SuppressionState, to_filigree_metadata active = to_filigree_metadata(_finding())["wardline"] - assert "suppressed" not in active + assert "suppression_state" not in active waived = to_filigree_metadata(_finding(suppressed=SuppressionState.WAIVED, suppression_reason="ok"))["wardline"] - assert waived["suppressed"] == "waived" + assert waived["suppression_state"] == "waived" assert waived["suppression_reason"] == "ok" diff --git a/tests/unit/core/test_legis_artifact.py b/tests/unit/core/test_legis_artifact.py index e064ad4b..7123febf 100644 --- a/tests/unit/core/test_legis_artifact.py +++ b/tests/unit/core/test_legis_artifact.py @@ -28,11 +28,16 @@ from wardline.core.finding import Finding, Kind, Location, Severity, SuppressionState # --------------------------------------------------------------------------- -# Golden vector — the literal hex was produced by the REAL legis signer -# (`from legis.enforcement.signing import sign`) over the fields below with the -# key below. Vendoring the canonical_json+HMAC *formula* cannot catch a shared -# misreading of the contract; a hardcoded hex captured from legis can. The live -# `legis_e2e` oracle re-confirms the verify path against a running legis. +# Golden vector — the literal hex pins Wardline's signer over the fields below +# with the key below. Vendoring the canonical_json+HMAC *formula* cannot catch a +# shared misreading of the contract; a hardcoded hex can. The live `legis_e2e` +# oracle re-confirms the verify path against a running legis. +# +# REKEYED for W3 (weft-f506e5f845): the per-finding key was renamed +# ``suppressed`` -> ``suppression_state``, which changes the SIGNED bytes. legis +# (the consumer + co-signer) must adopt the same key and regenerate its matching +# vector before the signed hop verifies again — tracked in the W3 legis weft ticket; +# the opt-in `legis_e2e` oracle stays red against an un-updated legis until then. # --------------------------------------------------------------------------- _GOLDEN_KEY = b"test-shared-secret-key" _GOLDEN_FIELDS = { @@ -49,11 +54,11 @@ "fingerprint": "a" * 64, "qualname": "svc.leaky", "properties": {"declared_return": "INTEGRAL", "actual_return": "EXTERNAL_RAW"}, - "suppressed": "active", + "suppression_state": "active", } ], } -_GOLDEN_SIG = "hmac-sha256:v2:73eb9f0c8b7ba898aa4b5fd62fa56ade0cbd9755d2c42eee209012675537f81d" +_GOLDEN_SIG = "hmac-sha256:v2:2b2cf09548572b58fd01c359d1b6a16c3c1181f1cbfe8e4f5ada6fcd21f35ac4" def test_golden_signature_matches_real_legis() -> None: @@ -121,7 +126,7 @@ def test_projection_minimal_legis_read_surface() -> None: "fingerprint", "qualname", "properties", - "suppressed", + "suppression_state", } @@ -130,7 +135,7 @@ def test_baselined_maps_to_suppressed_with_synthesized_proof() -> None: # non-active defect. A baselined finding with no reason must still carry proof. f = _finding(suppressed=SuppressionState.BASELINED, suppression_reason=None) out = legis.project_finding(f) - assert out["suppressed"] == "suppressed" + assert out["suppression_state"] == "suppressed" assert isinstance(out["properties"].get("suppression_reason"), str) assert out["properties"]["suppression_reason"].strip() @@ -138,20 +143,20 @@ def test_baselined_maps_to_suppressed_with_synthesized_proof() -> None: def test_judged_maps_to_suppressed_and_carries_reason() -> None: f = _finding(suppressed=SuppressionState.JUDGED, suppression_reason="LLM: false positive") out = legis.project_finding(f) - assert out["suppressed"] == "suppressed" + assert out["suppression_state"] == "suppressed" assert out["properties"]["suppression_reason"] == "LLM: false positive" def test_waived_keeps_state_and_injects_proof_into_properties() -> None: f = _finding(suppressed=SuppressionState.WAIVED, suppression_reason="WAIVE-123") out = legis.project_finding(f) - assert out["suppressed"] == "waived" + assert out["suppression_state"] == "waived" assert out["properties"]["suppression_reason"] == "WAIVE-123" def test_active_finding_carries_no_suppression_proof() -> None: out = legis.project_finding(_finding(properties={"tier": "INTEGRAL"})) - assert out["suppressed"] == "active" + assert out["suppression_state"] == "active" assert "suppression_reason" not in out["properties"] diff --git a/tests/unit/core/test_run.py b/tests/unit/core/test_run.py index 0c917ab5..347385e7 100644 --- a/tests/unit/core/test_run.py +++ b/tests/unit/core/test_run.py @@ -251,17 +251,26 @@ def test_gate_decision_reason_names_both_active_and_suppressed_on_mixed_trip(tmp def test_gate_decision_rejects_contradictory_construction() -> None: # The __post_init__ invariant guard: GateDecision must make "tripped gate that reads - # as passed" (dogfood #2) unconstructible, not merely avoided by the factory. + # as passed" (dogfood #2) unconstructible, not merely avoided by the factory. The guards + # are now verdict-keyed (weft-b937e53854). with pytest.raises(ValueError, match="exit_class"): - GateDecision(tripped=True, fail_on="ERROR", exit_class=0, reason="x", evaluated="y") + GateDecision(tripped=True, fail_on="ERROR", exit_class=0, verdict="FAILED", reason="x", evaluated="y") with pytest.raises(ValueError, match="reason"): - GateDecision(tripped=True, fail_on="ERROR", exit_class=1, reason=None, evaluated="y") - with pytest.raises(ValueError, match="reason"): - # fail_on set but no verdict — the no-gate shape leaking into a gated decision. - GateDecision(tripped=False, fail_on="ERROR", exit_class=0, reason=None, evaluated=None) - # The two legitimate shapes the factory produces still construct cleanly. - GateDecision(tripped=False, fail_on=None, exit_class=0) - GateDecision(tripped=True, fail_on="ERROR", exit_class=1, reason="1 active", evaluated="unsuppressed") + GateDecision(tripped=True, fail_on="ERROR", exit_class=1, verdict="FAILED", reason=None, evaluated="y") + with pytest.raises(ValueError, match="NOT_EVALUATED"): + # NOT_EVALUATED but a threshold IS set — the no-gate shape leaking into a gated decision. + GateDecision(tripped=False, fail_on="ERROR", exit_class=0, verdict="NOT_EVALUATED", reason="x", evaluated="y") + with pytest.raises(ValueError, match="FAILED"): + # tripped but verdict says PASSED — the dogfood #2 regression made unconstructible. + GateDecision(tripped=True, fail_on="ERROR", exit_class=1, verdict="PASSED", reason="x", evaluated="y") + # The three legitimate shapes the factory produces still construct cleanly. + GateDecision(tripped=False, fail_on=None, exit_class=0, verdict="NOT_EVALUATED", reason="no threshold") + GateDecision( + tripped=False, fail_on="ERROR", exit_class=0, verdict="PASSED", reason="clean", evaluated="unsuppressed" + ) + GateDecision( + tripped=True, fail_on="ERROR", exit_class=1, verdict="FAILED", reason="1 active", evaluated="unsuppressed" + ) def test_gate_decision_evaluated_reflects_trust_suppressions(tmp_path: Path) -> None: @@ -272,10 +281,16 @@ def test_gate_decision_evaluated_reflects_trust_suppressions(tmp_path: Path) -> assert decision.evaluated is not None and "honored" in decision.evaluated -def test_gate_decision_no_threshold_has_no_reason() -> None: +def test_gate_decision_no_threshold_is_not_evaluated() -> None: + # weft-b937e53854: a bare scan is NOT a clean pass — it never ran the gate. result = ScanResult(findings=[], summary=ScanSummary(0, 0, 0, 0, 0), files_scanned=0, context=None) decision = gate_decision(result, None) - assert decision.reason is None and decision.evaluated is None + assert decision.verdict == "NOT_EVALUATED" + assert decision.tripped is False and decision.exit_class == 0 + # Empty tree: nothing would trip, but the decision still carries an honest reason + population. + assert decision.would_trip_at is None + assert decision.reason is not None and "did not evaluate" in decision.reason + assert decision.evaluated is not None def _hint(proj: Path, *, new_since=None, trust=False): @@ -522,14 +537,48 @@ def test_gate_decision_rejects_unknown_fail_on() -> None: # fail_on is always a Severity value; an arbitrary string is an illegal state the # other guards would otherwise let through (it satisfies "reason iff fail_on"). with pytest.raises(ValueError, match="fail_on"): - GateDecision(tripped=True, fail_on="banana", exit_class=1, reason="x", evaluated="y") + GateDecision(tripped=True, fail_on="banana", exit_class=1, verdict="FAILED", reason="x", evaluated="y") def test_gate_decision_accepts_valid_severity_value() -> None: - dec = GateDecision(tripped=True, fail_on=Severity.ERROR.value, exit_class=1, reason="x", evaluated="y") + dec = GateDecision( + tripped=True, fail_on=Severity.ERROR.value, exit_class=1, verdict="FAILED", reason="x", evaluated="y" + ) assert dec.fail_on == "ERROR" +def test_would_trip_at_names_highest_severity_on_bare_scan(tmp_path: Path) -> None: + # weft-b937e53854: a bare scan reports would_trip_at = the worst active severity, so the + # agent's first call is not a vacuous green. The leaky proj has a PY-WL-101 ERROR defect. + proj, _ = _leaky_proj(tmp_path) + decision = gate_decision(run_scan(proj), None) + assert decision.verdict == "NOT_EVALUATED" + assert decision.would_trip_at == "ERROR" + assert decision.tripped is False and decision.exit_class == 0 + assert decision.reason is not None and "ERROR" in decision.reason + + +def test_verdict_passed_vs_failed_and_would_trip_at_is_threshold_independent(tmp_path: Path) -> None: + proj, _ = _leaky_proj(tmp_path) + result = run_scan(proj) + failed = gate_decision(result, Severity.ERROR) + assert failed.verdict == "FAILED" and failed.tripped is True and failed.would_trip_at == "ERROR" + # A threshold ABOVE the worst active severity passes; would_trip_at still names the worst. + passed = gate_decision(result, Severity.CRITICAL) + assert passed.verdict == "PASSED" and passed.tripped is False + assert passed.would_trip_at == "ERROR" + + +def test_summary_buckets_sum_to_total(tmp_path: Path) -> None: + # weft-f506e5f845: active+baselined+waived+judged+informational == total exactly; + # unanalyzed is an overlay, not a partition member. + proj, fp = _leaky_proj(tmp_path) + _write_baseline(proj, fp) + s = run_scan(proj).summary + assert s.active + s.baselined + s.waived + s.judged + s.informational == s.total + assert s.informational >= 1 # the engine metric/fact is a non-defect finding + + def test_run_scan_explicit_malformed_config_raises(tmp_path: Path) -> None: # (d) An EXPLICIT --config that EXISTS but is malformed must NOT silently fall # back to default policy either — that is the same false-green as a missing path. diff --git a/tests/unit/install/test_doctor_filigree_auth.py b/tests/unit/install/test_doctor_filigree_auth.py index 92cf9163..fc064080 100644 --- a/tests/unit/install/test_doctor_filigree_auth.py +++ b/tests/unit/install/test_doctor_filigree_auth.py @@ -69,8 +69,15 @@ def test_rewrite_env_preserves_non_utf8_unrelated_line(tmp_path: Path) -> None: def _write_mcp_with_filigree_url(root: Path, url: str) -> None: root.joinpath(".mcp.json").write_text( json.dumps( - {"mcpServers": {"wardline": {"type": "stdio", "command": "wardline", - "args": ["mcp", "--root", ".", "--filigree-url", url]}}} + { + "mcpServers": { + "wardline": { + "type": "stdio", + "command": "wardline", + "args": ["mcp", "--root", ".", "--filigree-url", url], + } + } + } ), encoding="utf-8", ) @@ -94,9 +101,7 @@ def test_mcp_filigree_url_none_when_args_not_list(tmp_path: Path) -> None: # is a substring search that would otherwise return a char index, and # args[idx + 1] a single bogus character. The list-type guard returns None. tmp_path.joinpath(".mcp.json").write_text( - json.dumps( - {"mcpServers": {"wardline": {"args": "mcp --filigree-url http://x"}}} - ), + json.dumps({"mcpServers": {"wardline": {"args": "mcp --filigree-url http://x"}}}), encoding="utf-8", ) assert _mcp_filigree_url(tmp_path) is None @@ -109,11 +114,23 @@ def test_check_project_mcp_accepts_pinned_sibling_args_in_operator_order(tmp_pat monkeypatch.setattr("wardline.install.mcp_json._find_wardline_command", lambda: "/bin/wardline") tmp_path.joinpath(".mcp.json").write_text( json.dumps( - {"mcpServers": {"wardline": {"type": "stdio", "command": "/bin/wardline", "args": [ - "mcp", "--root", ".", - "--loomweave-url", "http://127.0.0.1:9730", - "--filigree-url", "http://127.0.0.1:8749/api/p/lacuna/weft/scan-results", - ]}}} + { + "mcpServers": { + "wardline": { + "type": "stdio", + "command": "/bin/wardline", + "args": [ + "mcp", + "--root", + ".", + "--loomweave-url", + "http://127.0.0.1:9730", + "--filigree-url", + "http://127.0.0.1:8749/api/p/lacuna/weft/scan-results", + ], + } + } + } ), encoding="utf-8", ) @@ -229,8 +246,12 @@ def test_check_ok_when_unreachable(tmp_path: Path, monkeypatch) -> None: def test_check_ok_when_non_loopback(tmp_path: Path, monkeypatch) -> None: monkeypatch.delenv("WARDLINE_FILIGREE_URL", raising=False) monkeypatch.setenv("WEFT_FEDERATION_TOKEN", "T") - check = _check_filigree_auth(tmp_path, repair=False, filigree_url="https://remote.example.com/api/weft/scan-results", - transport=_ScriptedTransport({})) + check = _check_filigree_auth( + tmp_path, + repair=False, + filigree_url="https://remote.example.com/api/weft/scan-results", + transport=_ScriptedTransport({}), + ) assert check.status == "ok" assert "non-loopback" in (check.message or "") diff --git a/tests/unit/install/test_mcp_json.py b/tests/unit/install/test_mcp_json.py index b52d42d5..84f4df96 100644 --- a/tests/unit/install/test_mcp_json.py +++ b/tests/unit/install/test_mcp_json.py @@ -76,11 +76,23 @@ def test_preserves_operator_pinned_sibling_url_args(tmp_path: Path, monkeypatch: monkeypatch.setattr("wardline.install.mcp_json._find_wardline_command", lambda: "/bin/wardline") (tmp_path / ".mcp.json").write_text( json.dumps( - {"mcpServers": {"wardline": {"type": "stdio", "command": "OLD", "args": [ - "mcp", "--root", ".", - "--loomweave-url", "http://127.0.0.1:9730", - "--filigree-url", "http://127.0.0.1:8749/api/p/lacuna/weft/scan-results", - ]}}} + { + "mcpServers": { + "wardline": { + "type": "stdio", + "command": "OLD", + "args": [ + "mcp", + "--root", + ".", + "--loomweave-url", + "http://127.0.0.1:9730", + "--filigree-url", + "http://127.0.0.1:8749/api/p/lacuna/weft/scan-results", + ], + } + } + } ), encoding="utf-8", ) @@ -90,9 +102,13 @@ def test_preserves_operator_pinned_sibling_url_args(tmp_path: Path, monkeypatch: entry = json.loads((tmp_path / ".mcp.json").read_text(encoding="utf-8"))["mcpServers"]["wardline"] assert entry["command"] == "/bin/wardline" assert entry["args"] == [ - "mcp", "--root", ".", - "--loomweave-url", "http://127.0.0.1:9730", - "--filigree-url", "http://127.0.0.1:8749/api/p/lacuna/weft/scan-results", + "mcp", + "--root", + ".", + "--loomweave-url", + "http://127.0.0.1:9730", + "--filigree-url", + "http://127.0.0.1:8749/api/p/lacuna/weft/scan-results", ] # idempotent: re-running over the already-preserved entry is a no-op (no reorder churn). assert merge_mcp_entry(tmp_path) == "unchanged" @@ -104,11 +120,23 @@ def test_already_canonical_lacuna_entry_is_unchanged(tmp_path: Path, monkeypatch monkeypatch.setattr("wardline.install.mcp_json._find_wardline_command", lambda: "/bin/wardline") (tmp_path / ".mcp.json").write_text( json.dumps( - {"mcpServers": {"wardline": {"type": "stdio", "command": "/bin/wardline", "args": [ - "mcp", "--root", ".", - "--loomweave-url", "http://127.0.0.1:9730", - "--filigree-url", "http://127.0.0.1:8749/api/p/lacuna/weft/scan-results", - ]}}} + { + "mcpServers": { + "wardline": { + "type": "stdio", + "command": "/bin/wardline", + "args": [ + "mcp", + "--root", + ".", + "--loomweave-url", + "http://127.0.0.1:9730", + "--filigree-url", + "http://127.0.0.1:8749/api/p/lacuna/weft/scan-results", + ], + } + } + } ), encoding="utf-8", ) diff --git a/tests/unit/mcp/test_server_filigree_emit.py b/tests/unit/mcp/test_server_filigree_emit.py index 51678300..6d5ff82d 100644 --- a/tests/unit/mcp/test_server_filigree_emit.py +++ b/tests/unit/mcp/test_server_filigree_emit.py @@ -63,6 +63,8 @@ def test_scan_emits_to_filigree_when_emitter_present(tmp_path): "warnings": [], "status": None, "auth_rejected": False, + "token_sent": False, + "url": None, "disabled_reason": None, } assert emitter.scanned_paths == ("svc.py",) @@ -83,6 +85,8 @@ def test_scan_reports_both_integrations_successful(tmp_path): "warnings": [], "status": None, "auth_rejected": False, + "token_sent": False, + "url": None, "disabled_reason": None, } diff --git a/tests/unit/mcp/test_server_loomweave_write.py b/tests/unit/mcp/test_server_loomweave_write.py index 82289812..e1630595 100644 --- a/tests/unit/mcp/test_server_loomweave_write.py +++ b/tests/unit/mcp/test_server_loomweave_write.py @@ -56,7 +56,7 @@ def test_scan_tool_survives_loomweave_write_error(tmp_path): # The scan payload itself is intact, NOT discarded — assert on real scan keys # that _scan always returns. assert "summary" in out - assert "findings" in out + assert "agent_summary" in out assert "gate" in out # PY-WL-101 fires on _LEAKY, so the scan found real findings. assert out["summary"]["total"] >= 1 diff --git a/tests/unit/mcp/test_server_query_explain.py b/tests/unit/mcp/test_server_query_explain.py index 7a8a3fed..afbd7b33 100644 --- a/tests/unit/mcp/test_server_query_explain.py +++ b/tests/unit/mcp/test_server_query_explain.py @@ -36,13 +36,13 @@ def _baseline_all(tmp_path) -> None: def test_where_filters_findings_by_qualname(tmp_path): (tmp_path / "svc.py").write_text(_SRC, encoding="utf-8") - full = _scan({}, tmp_path) - qualnames = {f["qualname"] for f in full["findings"] if f["rule_id"] == "PY-WL-101"} + full = _scan({"full": True}, tmp_path) + qualnames = {e["qualname"] for e in full["agent_summary"]["active_defects"] if e["rule_id"] == "PY-WL-101"} assert "svc.leak_a" in qualnames and "svc.leak_b" in qualnames - filtered = _scan({"where": {"qualname": "svc.leak_a"}}, tmp_path) - got = [f for f in filtered["findings"] if f["rule_id"] == "PY-WL-101"] - assert {f["qualname"] for f in got} == {"svc.leak_a"} + filtered = _scan({"where": {"qualname": "svc.leak_a"}, "full": True}, tmp_path) + got = [e for e in filtered["agent_summary"]["active_defects"] if e["rule_id"] == "PY-WL-101"] + assert {e["qualname"] for e in got} == {"svc.leak_a"} def test_where_summary_and_gate_describe_whole_project(tmp_path): @@ -63,7 +63,7 @@ def test_where_unknown_key_is_toolerror(tmp_path): def test_explain_inlines_provenance_on_active_defects(tmp_path): (tmp_path / "svc.py").write_text(_SRC, encoding="utf-8") out = _scan({"explain": True}, tmp_path) - by_q = {f["qualname"]: f for f in out["findings"] if f["rule_id"] == "PY-WL-101"} + by_q = {e["qualname"]: e for e in out["agent_summary"]["active_defects"] if e["rule_id"] == "PY-WL-101"} exp = by_q["svc.leak_a"]["explanation"] assert exp["immediate_tainted_callee"] == "read_a" assert exp["source_boundary_qualname"] == "svc.read_a" @@ -73,7 +73,7 @@ def test_explain_inlines_provenance_on_active_defects(tmp_path): def test_explain_absent_by_default(tmp_path): (tmp_path / "svc.py").write_text(_SRC, encoding="utf-8") out = _scan({}, tmp_path) - assert all("explanation" not in f for f in out["findings"]) + assert all("explanation" not in e for e in out["agent_summary"]["active_defects"]) def test_explain_matches_single_finding_explain(tmp_path): @@ -82,7 +82,11 @@ def test_explain_matches_single_finding_explain(tmp_path): (tmp_path / "svc.py").write_text(_SRC, encoding="utf-8") out = _scan({"explain": True}, tmp_path) - f = next(f for f in out["findings"] if f["qualname"] == "svc.leak_a" and f["rule_id"] == "PY-WL-101") + f = next( + e + for e in out["agent_summary"]["active_defects"] + if e["qualname"] == "svc.leak_a" and e["rule_id"] == "PY-WL-101" + ) single = _explain_taint({"fingerprint": f["fingerprint"]}, tmp_path) # All six explanation keys must match the single-finding explain projection. assert f["explanation"] == {k: single[k] for k in f["explanation"]} @@ -97,9 +101,8 @@ def test_where_filters_agent_summary_arrays(tmp_path): (tmp_path / "svc.py").write_text(_many_leaks(5), encoding="utf-8") _baseline_all(tmp_path) out = _scan({"where": {"suppression": "active", "severity": "CRITICAL"}}, tmp_path) - assert out["findings"] == [] # 0 active CRITICAL summ = out["agent_summary"] - assert summ["suppressed_findings"] == [] + assert summ["suppressed_findings"] == [] # 0 active CRITICAL assert summ["active_defects"] == [] # but the whole-project counts are preserved assert summ["summary"]["suppressed_findings"] == 5 @@ -112,9 +115,9 @@ def test_explain_true_has_default_cap(tmp_path): # truncation is announced — never silent. (tmp_path / "svc.py").write_text(_many_leaks(40), encoding="utf-8") out = _scan({"explain": True}, tmp_path) - explained = [f for f in out["findings"] if "explanation" in f] - assert 0 < len(explained) <= 10 # default cap - assert out["truncation"]["explanations_truncated"] is True + explained = [e for e in out["agent_summary"]["active_defects"] if "explanation" in e] + assert 0 < len(explained) <= 10 # default explanation cap (independent of the body page size) + assert out["agent_summary"]["truncation"]["explanations_truncated"] is True # the true total is still reported, so nothing is silently hidden assert out["summary"]["active"] == 40 @@ -124,22 +127,21 @@ def test_max_findings_can_raise_explain_cap_above_default(tmp_path): # the conservative default (10) when the agent accepts the larger payload. (tmp_path / "svc.py").write_text(_many_leaks(20), encoding="utf-8") out = _scan({"explain": True, "max_findings": 20}, tmp_path) - explained = [f for f in out["findings"] if "explanation" in f] + explained = [e for e in out["agent_summary"]["active_defects"] if "explanation" in e] assert len(explained) > 10 # exceeded the default cap - assert out["truncation"]["explanations_truncated"] is False + assert out["agent_summary"]["truncation"]["explanations_truncated"] is False def test_summary_only_omits_finding_arrays(tmp_path): # (d): the "did the gate pass?" payload — counts + gate, no finding bodies. (tmp_path / "svc.py").write_text(_many_leaks(5), encoding="utf-8") out = _scan({"summary_only": True, "fail_on": "ERROR"}, tmp_path) - assert out["findings"] == [] summ = out["agent_summary"] assert summ["active_defects"] == [] and summ["suppressed_findings"] == [] and summ["engine_facts"] == [] # counts + gate intact assert out["summary"]["active"] == 5 assert out["gate"]["tripped"] is True - assert out["truncation"]["summary_only"] is True + assert summ["truncation"]["summary_only"] is True def test_include_suppressed_false_drops_suppressed(tmp_path): @@ -147,7 +149,8 @@ def test_include_suppressed_false_drops_suppressed(tmp_path): (tmp_path / "svc.py").write_text(_many_leaks(5), encoding="utf-8") _baseline_all(tmp_path) out = _scan({"include_suppressed": False}, tmp_path) - assert all(f["suppressed"] == "active" for f in out["findings"]) + # suppressed bodies are dropped from the page; any shown defect body is active. + assert all(e["suppression_state"] == "active" for e in out["agent_summary"]["active_defects"]) assert out["agent_summary"]["suppressed_findings"] == [] # whole-project count still visible assert out["summary"]["baselined"] == 5 @@ -157,10 +160,12 @@ def test_max_findings_caps_and_marks(tmp_path): # (b): bound the returned list and announce the cut. (tmp_path / "svc.py").write_text(_many_leaks(10), encoding="utf-8") out = _scan({"max_findings": 3}, tmp_path) - assert len(out["findings"]) == 3 - assert out["truncation"]["findings_truncated"] is True - assert out["truncation"]["findings_returned"] == 3 - assert out["truncation"]["findings_total"] >= 10 + ag = out["agent_summary"] + shown = len(ag["active_defects"]) + len(ag["suppressed_findings"]) + len(ag["engine_facts"]) + assert shown == 3 + assert ag["truncation"]["findings_truncated"] is True + assert ag["truncation"]["findings_returned"] == 3 + assert ag["truncation"]["findings_total"] >= 10 @pytest.mark.parametrize("bad", [-1, 1.5, "3", True]) @@ -179,3 +184,46 @@ def test_boolean_payload_controls_reject_non_bool(tmp_path, name): (tmp_path / "svc.py").write_text(_SRC, encoding="utf-8") with pytest.raises(ToolError, match=name): _scan({name: "false"}, tmp_path) + + +# --- W1: bounded default + offset pagination (weft-439d09fc8d) --------------- + + +def _shown(ag) -> int: + return len(ag["active_defects"]) + len(ag["suppressed_findings"]) + len(ag["engine_facts"]) + + +def test_default_scan_is_bounded_to_25(tmp_path): + # A bare scan over 30 active defects returns at most the bounded page (25), so the + # agent's first natural call cannot overflow its context — the ~123KB dump is gone. + (tmp_path / "svc.py").write_text(_many_leaks(30), encoding="utf-8") + out = _scan({}, tmp_path) + ag = out["agent_summary"] + assert _shown(ag) == 25 + t = ag["truncation"] + assert t["findings_returned"] == 25 and t["findings_truncated"] is True and t["next_offset"] == 25 + # whole-project count stays honest regardless of the page bound + assert out["summary"]["active"] == 30 + + +def test_offset_pages_are_disjoint_and_full_is_uncapped(tmp_path): + (tmp_path / "svc.py").write_text(_many_leaks(30), encoding="utf-8") + page1 = _scan({}, tmp_path)["agent_summary"] + nxt = page1["truncation"]["next_offset"] + assert nxt == 25 + page2 = _scan({"offset": nxt}, tmp_path)["agent_summary"] + fp1 = {e["fingerprint"] for e in page1["active_defects"]} + fp2 = {e["fingerprint"] for e in page2["active_defects"]} + assert fp1 and fp2 and fp1.isdisjoint(fp2) # no finding appears on both pages + assert page2["truncation"]["offset"] == 25 + assert page2["truncation"]["next_offset"] is None # 30 actives → page 2 is the last + # full=true returns every body in one call, untruncated. + full = _scan({"full": True}, tmp_path)["agent_summary"] + assert len(full["active_defects"]) == 30 + assert full["truncation"]["findings_truncated"] is False and full["truncation"]["next_offset"] is None + + +def test_offset_rejects_non_negative_integer(tmp_path): + (tmp_path / "svc.py").write_text(_SRC, encoding="utf-8") + with pytest.raises(ToolError, match="offset"): + _scan({"offset": -1}, tmp_path) diff --git a/tests/unit/mcp/test_server_suppression.py b/tests/unit/mcp/test_server_suppression.py index 8e91a675..f5f0b681 100644 --- a/tests/unit/mcp/test_server_suppression.py +++ b/tests/unit/mcp/test_server_suppression.py @@ -48,7 +48,7 @@ def test_mcp_scan_gate_trips_on_baselined_defect_by_default(tmp_path: Path) -> N proj = _leaky_project(tmp_path) server = WardlineMCPServer(root=proj) first = _call(server, "scan", {}) - fp = next(f["fingerprint"] for f in first["findings"] if f["rule_id"] == "PY-WL-101") + fp = next(e["fingerprint"] for e in first["agent_summary"]["active_defects"] if e["rule_id"] == "PY-WL-101") bl = baseline_path(proj) bl.parent.mkdir(parents=True, exist_ok=True) bl.write_text( @@ -57,8 +57,8 @@ def test_mcp_scan_gate_trips_on_baselined_defect_by_default(tmp_path: Path) -> N ) # Default: annotated baselined, but the gate trips. default = _call(server, "scan", {"fail_on": "ERROR"}) - leak = next(f for f in default["findings"] if f["rule_id"] == "PY-WL-101") - assert leak["suppressed"] == "baselined" + leak = next(e for e in default["agent_summary"]["suppressed_findings"] if e["rule_id"] == "PY-WL-101") + assert leak["suppression_state"] == "baselined" assert default["gate"]["tripped"] is True # trust_suppressions restores the trusted-local behaviour: the gate clears. trusted = _call(server, "scan", {"fail_on": "ERROR", "trust_suppressions": True}) @@ -104,7 +104,9 @@ def test_baseline_create_trusted_pack_matches_scan_mcp(tmp_path: Path, monkeypat server = WardlineMCPServer(root=proj) scan = _call(server, "scan", {"trust_packs": ["baseline_mcp_pack"], "trust_local_packs": True}) - assert any(f["rule_id"] == "PY-WL-901" for f in scan["findings"]) + ag = scan["agent_summary"] + shown = ag["active_defects"] + ag["suppressed_findings"] + ag["engine_facts"] + assert any(e["rule_id"] == "PY-WL-901" for e in shown) baseline = _call( server, diff --git a/tests/unit/mcp/test_server_tools.py b/tests/unit/mcp/test_server_tools.py index fc7ea68d..f7d06dc7 100644 --- a/tests/unit/mcp/test_server_tools.py +++ b/tests/unit/mcp/test_server_tools.py @@ -75,8 +75,11 @@ def test_scan_tool_returns_summary_and_gate(tmp_path: Path) -> None: root = _leaky_project(tmp_path) server = WardlineMCPServer(root=root) out = _call(server, "scan", {"fail_on": "ERROR"}) - assert "findings" in out and "summary" in out and "gate" in out - assert out["summary"]["total"] == len(out["findings"]) + assert "agent_summary" in out and "summary" in out and "gate" in out + # Finding bodies live in agent_summary now (no top-level findings array, W1). The + # whole-project buckets partition the total exactly (weft-f506e5f845). + s = out["summary"] + assert s["active"] + s["baselined"] + s["waived"] + s["judged"] + s["informational"] == s["total"] assert out["summary"]["active"] >= 1 assert out["gate"]["tripped"] is True # The agent-facing dogfood gate fields are assembled in server.py separately from @@ -88,7 +91,7 @@ def test_scan_tool_returns_summary_and_gate(tmp_path: Path) -> None: # No committed baseline here, so the migration hint must be present AND None (a # spurious fire would be a regression in the secure-default rollout signal). assert "migration_hint" in out["gate"] and out["gate"]["migration_hint"] is None - assert any(f["rule_id"] == "PY-WL-101" for f in out["findings"]) + assert any(e["rule_id"] == "PY-WL-101" for e in out["agent_summary"]["active_defects"]) def test_scan_tool_summary_includes_unanalyzed(tmp_path: Path) -> None: @@ -127,7 +130,7 @@ def test_explain_taint_success_through_mcp(tmp_path: Path) -> None: root = _leaky_project(tmp_path) server = WardlineMCPServer(root=root) scan_out = _call(server, "scan", {}) - leak = next(f for f in scan_out["findings"] if f["rule_id"] == "PY-WL-101") + leak = next(e for e in scan_out["agent_summary"]["active_defects"] if e["rule_id"] == "PY-WL-101") fp = leak["fingerprint"] resp = server.rpc.dispatch( From 25971c98f1cfd82a07bf96df5955547bd9c8b5f0 Mon Sep 17 00:00:00 2001 From: John Morrissey Date: Mon, 8 Jun 2026 05:32:09 +1000 Subject: [PATCH 009/186] feat(emit): echo emit destination so a wrong-project write is visible (N1 / C-10a) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Every Filigree emit status block (MCP scan, agent-summary, CLI) now carries a destination {url, project, project_pinned}, and the CLI success line names the destination project. An unpinned URL reports project_pinned:false — surfacing that Filigree resolves the project server-side (the silent-misroute shape behind the lacuna->filigree contamination) so a misroute no longer reads as success. Pairs with filigree's fail-closed (weft-7a399b8124); filigree's response carries no project echo yet, so the cross-check half remains gated there. Reconciled the finding-lifecycle glossary anchors shifted by the scan.py/server.py insertions. Co-Authored-By: Claude Opus 4.8 --- CHANGELOG.md | 7 ++++ .../reference/finding-lifecycle-vocabulary.md | 38 +++++++++--------- src/wardline/cli/scan.py | 22 ++++++++++- src/wardline/core/filigree_emit.py | 30 ++++++++++++++ src/wardline/mcp/server.py | 5 ++- tests/docs/test_glossary_vocabulary.py | 26 ++++++------- tests/unit/core/test_filigree_destination.py | 39 +++++++++++++++++++ tests/unit/mcp/test_server_filigree_emit.py | 3 ++ 8 files changed, 135 insertions(+), 35 deletions(-) create mode 100644 tests/unit/core/test_filigree_destination.py diff --git a/CHANGELOG.md b/CHANGELOG.md index f629d3e8..563cd9a5 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -22,6 +22,13 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 new `offset` pages through the rest via `agent_summary.truncation.next_offset`. `explain: true` inlines provenance into the `agent_summary.active_defects` entries (capped, announced) (weft-439d09fc8d). +- **Emit destination is now echoed (no silent misroute).** Every Filigree emit + status block (MCP `scan`, agent-summary, CLI) carries a `destination` + (`{url, project, project_pinned}`) naming where findings were sent; the CLI + success line names the destination project. When the URL pins no project, + `project_pinned: false` surfaces that Filigree resolves it server-side — the + silent-misroute shape behind the lacuna→filigree contamination — so a + wrong-project write is visible at the caller (C-10(a)). - `wardline doctor` now verifies the Filigree federation token: it probes the configured daemon (URL resolved from `.mcp.json`/env) with the token wardline would emit and reports a `filigree.auth` check. `--repair` recovers the diff --git a/docs/reference/finding-lifecycle-vocabulary.md b/docs/reference/finding-lifecycle-vocabulary.md index 86f68ec1..63815793 100644 --- a/docs/reference/finding-lifecycle-vocabulary.md +++ b/docs/reference/finding-lifecycle-vocabulary.md @@ -59,7 +59,7 @@ The Filigree metadata only carries the key when the state is not `active` **"suppressed"** survives only as the umbrella *word* for "any state other than `active`": `baselined` + `waived` + `judged`. The CLI prints this sum as the -`suppressed` count (`src/wardline/cli/scan.py:362`). +`suppressed` count (`src/wardline/cli/scan.py:384`). ## `active` is the one word for "non-suppressed defect" @@ -70,8 +70,8 @@ consistently, on every surface: | --- | --- | --- | | Enum | `src/wardline/core/finding.py:68` | `SuppressionState.ACTIVE = "active"` | | Summary field | `src/wardline/core/run.py:50`, built at `src/wardline/core/run.py:312` | `ScanSummary.active` | -| CLI summary line | `src/wardline/cli/scan.py:370` | `… {s.active} active` | -| MCP scan response | `src/wardline/mcp/server.py:328` | `summary.active` | +| CLI summary line | `src/wardline/cli/scan.py:385` | `… {s.active} active` | +| MCP scan response | `src/wardline/mcp/server.py:331` | `summary.active` | | Agent-summary JSON | `src/wardline/core/agent_summary.py:117` | `summary.active_defects` | | `wardline:loop` prompt | `src/wardline/mcp/prompts.py:13` | "Read `summary.active`" | @@ -99,8 +99,8 @@ So `active + baselined + waived + judged + informational == total` (`src/wardline/core/run.py:49` for `total: int`). `unanalyzed` (`src/wardline/core/run.py:68`) is an **overlay** — a subset of `informational` that surfaces a silent under-scan — and is deliberately **not** a partition member. -The MCP `summary` block exposes `informational` (`src/wardline/mcp/server.py:336`) -and `unanalyzed` (`src/wardline/mcp/server.py:340`); the agent-summary block mirrors +The MCP `summary` block exposes `informational` (`src/wardline/mcp/server.py:339`) +and `unanalyzed` (`src/wardline/mcp/server.py:343`); the agent-summary block mirrors both (`src/wardline/core/agent_summary.py:123`, `src/wardline/core/agent_summary.py:124`). ## Emitted-active vs the gate population @@ -141,15 +141,15 @@ judged. The `verdict` is one of: - **`PASSED`** — a threshold ran and nothing tripped. - **`FAILED`** — a threshold ran and tripped. -The MCP `scan` gate block exposes `gate.tripped` (`src/wardline/mcp/server.py:343`), -`gate.verdict` (`src/wardline/mcp/server.py:346`), `would_trip_at`, `reason`, -`evaluated`, and `migration_hint`, opened at `src/wardline/mcp/server.py:342` +The MCP `scan` gate block exposes `gate.tripped` (`src/wardline/mcp/server.py:346`), +`gate.verdict` (`src/wardline/mcp/server.py:349`), `would_trip_at`, `reason`, +`evaluated`, and `migration_hint`, opened at `src/wardline/mcp/server.py:345` (`"gate": {`); the agent-summary mirrors them at `src/wardline/core/agent_summary.py:127` (`tripped`) and `src/wardline/core/agent_summary.py:130` (`verdict`). The CLI prints `gate: FAILED (--fail-on …) — ` then `gate: evaluated <…>`, or a `gate: NOT_EVALUATED — …` line for a bare scan -(`src/wardline/cli/scan.py:388`). +(`src/wardline/cli/scan.py:403`). `--new-since` scopes **both** populations identically: any `active` defect outside the delta is re-marked `baselined` in both the emitted and gate lists @@ -165,7 +165,7 @@ still legitimately means three different things depending on the surface: | --- | --- | --- | | Filigree store | An **unseen fingerprint** — first time this finding identity is seen for a `(file, scan_source)`. | **Filigree-owned** lifecycle (`src/wardline/core/filigree_emit.py:68-76`) | | `wardline scan --new-since ` | **Delta-scope**: the gate fires only on defects in files/entities changed since a git ref; everything else is re-marked `baselined`. | `src/wardline/core/run.py:288`; help text `src/wardline/cli/scan.py` (`--new-since`) | -| (historical) CLI summary | Formerly relabelled the `active` count as "N new". **Corrected to "N active"**. | `src/wardline/cli/scan.py:362` | +| (historical) CLI summary | Formerly relabelled the `active` count as "N new". **Corrected to "N active"**. | `src/wardline/cli/scan.py:384` | The first-seen Filigree sense and the delta-scope `--new-since` sense are genuinely distinct concepts; neither is "active". @@ -176,15 +176,15 @@ How each concept appears on each surface: | Concept | CLI summary text | `ScanSummary` field | MCP `summary` key | Agent-summary key | Filigree store | | --- | --- | --- | --- | --- | --- | -| every finding | `N finding(s)` | `total` (`run.py:49`) | `total` (`server.py:327`) | `total_findings` (`agent_summary.py:116`) | one finding per wire entry | -| live defect | `N active` (`scan.py:370`) | `active` (`run.py:50,312`) | `active` (`server.py:328`) | `active_defects` (`agent_summary.py:117`) | no `suppression_state` key (`finding.py:185`) | -| suppressed (sum) | `N suppressed` (`scan.py:362`) | `baselined+waived+judged` | the three keys | `suppressed_findings` (`agent_summary.py:118`) | `metadata.wardline.suppression_state` (`finding.py:185`) | -| baselined | `N baseline` | `baselined` (`run.py:52`) | `baselined` (`server.py:329`) | `baselined` (`agent_summary.py:120`) | `suppression_state: "baselined"` | -| waived | `N waiver` | `waived` (`run.py:53`) | `waived` (`server.py:330`) | `waived` (`agent_summary.py:121`) | `suppression_state: "waived"` | -| judged | `N judged` | `judged` (`run.py:54`) | `judged` (`server.py:331`) | `judged` (`agent_summary.py:122`) | `suppression_state: "judged"` | -| informational | (the remainder of `total`) | `informational` (`run.py:60`) | `informational` (`server.py:336`) | `informational` (`agent_summary.py:123`) | facts/metrics | -| under-scan | `N file(s) could not be analyzed` | `unanalyzed` (`run.py:68`) | `unanalyzed` (`server.py:340`) | `unanalyzed` (`agent_summary.py:124`) | `WLN-ENGINE-*` facts | -| gate verdict | exit code + `--fail-on` | (`gate_findings`, `run.py:87`; `GateDecision`, `run.py:97`, `verdict` `run.py:106`) | `gate.tripped` (`server.py:343`), `gate.verdict` (`server.py:346`) | `gate.tripped` (`agent_summary.py:127`), `gate.verdict` (`agent_summary.py:130`) | not emitted to Filigree | +| every finding | `N finding(s)` | `total` (`run.py:49`) | `total` (`server.py:330`) | `total_findings` (`agent_summary.py:116`) | one finding per wire entry | +| live defect | `N active` (`scan.py:385`) | `active` (`run.py:50,312`) | `active` (`server.py:331`) | `active_defects` (`agent_summary.py:117`) | no `suppression_state` key (`finding.py:185`) | +| suppressed (sum) | `N suppressed` (`scan.py:384`) | `baselined+waived+judged` | the three keys | `suppressed_findings` (`agent_summary.py:118`) | `metadata.wardline.suppression_state` (`finding.py:185`) | +| baselined | `N baseline` | `baselined` (`run.py:52`) | `baselined` (`server.py:332`) | `baselined` (`agent_summary.py:120`) | `suppression_state: "baselined"` | +| waived | `N waiver` | `waived` (`run.py:53`) | `waived` (`server.py:333`) | `waived` (`agent_summary.py:121`) | `suppression_state: "waived"` | +| judged | `N judged` | `judged` (`run.py:54`) | `judged` (`server.py:334`) | `judged` (`agent_summary.py:122`) | `suppression_state: "judged"` | +| informational | (the remainder of `total`) | `informational` (`run.py:60`) | `informational` (`server.py:339`) | `informational` (`agent_summary.py:123`) | facts/metrics | +| under-scan | `N file(s) could not be analyzed` | `unanalyzed` (`run.py:68`) | `unanalyzed` (`server.py:343`) | `unanalyzed` (`agent_summary.py:124`) | `WLN-ENGINE-*` facts | +| gate verdict | exit code + `--fail-on` | (`gate_findings`, `run.py:87`; `GateDecision`, `run.py:97`, `verdict` `run.py:106`) | `gate.tripped` (`server.py:346`), `gate.verdict` (`server.py:349`) | `gate.tripped` (`agent_summary.py:127`), `gate.verdict` (`agent_summary.py:130`) | not emitted to Filigree | The unsuppressed gate population is built from `Baseline(frozenset())` (`src/wardline/core/run.py:278`). diff --git a/src/wardline/cli/scan.py b/src/wardline/cli/scan.py index e4af9f0e..96d2824c 100644 --- a/src/wardline/cli/scan.py +++ b/src/wardline/cli/scan.py @@ -11,7 +11,13 @@ from wardline.core.config import resolve_filigree_url, resolve_loomweave_url from wardline.core.emit import JsonlSink from wardline.core.errors import WardlineError -from wardline.core.filigree_emit import EmitResult, FiligreeEmitter, filigree_disabled_reason +from wardline.core.filigree_emit import ( + EmitResult, + FiligreeEmitter, + filigree_destination, + filigree_disabled_reason, + filigree_url_project, +) from wardline.core.finding import Severity from wardline.core.paths import weft_config_path from wardline.core.run import baseline_migration_hint, gate_decision, run_scan @@ -334,8 +340,17 @@ def confirm_cb(rel_path: str, orig: str, replacement: str, f: Finding) -> bool: err=True, ) else: + # N1 / C-10(a): name the destination project so a wrong-project write is visible + # rather than reading as silent success. An unpinned URL means Filigree resolves + # the project server-side — surface that ambiguity explicitly. + dest_project = filigree_url_project(filigree_url) + where = ( + f"project {dest_project!r}" + if dest_project + else "server-default project (URL pins none — add ?project= to make it explicit)" + ) line = ( - f"emitted {len(findings)} finding(s) to {filigree_url} — " + f"emitted {len(findings)} finding(s) to {filigree_url} [{where}] — " f"{emit_result.created} created / {emit_result.updated} updated" ) if emit_result.failed: @@ -407,6 +422,7 @@ def _filigree_status(result: EmitResult | None) -> dict[str, object]: "failed": 0, "warnings": [], "disabled_reason": "not configured", + "destination": filigree_destination(None), } return { "configured": True, @@ -421,6 +437,8 @@ def _filigree_status(result: EmitResult | None) -> dict[str, object]: token_sent=result.token_sent, url=result.url, ), + # N1 / C-10(a): name where findings went so a wrong-project write is visible. + "destination": filigree_destination(result.url), } diff --git a/src/wardline/core/filigree_emit.py b/src/wardline/core/filigree_emit.py index 24d452bc..9beaab2e 100644 --- a/src/wardline/core/filigree_emit.py +++ b/src/wardline/core/filigree_emit.py @@ -81,6 +81,36 @@ def build_scan_results_body( return body +# --- destination echo (N1 / C-10(a)) ----------------------------------------- + + +def filigree_url_project(url: str | None) -> str | None: + """The destination project pinned in a Filigree Weft URL, or None when none is pinned. + + An unpinned URL means Filigree resolves the project server-side (ambient/default) — the + silent-misroute shape behind the lacuna→filigree contamination. Recognizes both the + ``?project=

`` query and the ``/api/p/

/`` path conventions.""" + if not url: + return None + parts = urllib.parse.urlsplit(url) + project = urllib.parse.parse_qs(parts.query).get("project", [None])[0] + if project: + return project + segments = [s for s in parts.path.split("/") if s] + for i, seg in enumerate(segments): + if seg == "p" and i + 1 < len(segments): + return segments[i + 1] + return None + + +def filigree_destination(url: str | None) -> dict[str, Any]: + """The destination echo for the emit status block (N1 / C-10(a)): name where findings + were sent so a wrong-project write is visible at the caller instead of reading as + success. ``project_pinned`` is False when Filigree will resolve the project itself.""" + project = filigree_url_project(url) + return {"url": url, "project": project, "project_pinned": project is not None} + + # --- transport + emitter ----------------------------------------------------- diff --git a/src/wardline/mcp/server.py b/src/wardline/mcp/server.py index df7ba491..66850983 100644 --- a/src/wardline/mcp/server.py +++ b/src/wardline/mcp/server.py @@ -21,7 +21,7 @@ from wardline.core.baseline import generate_baseline, load_baseline from wardline.core.errors import WardlineError from wardline.core.explain import explain_chain, explain_finding, explanation_from_context -from wardline.core.filigree_emit import FiligreeEmitter, filigree_disabled_reason +from wardline.core.filigree_emit import FiligreeEmitter, filigree_destination, filigree_disabled_reason from wardline.core.finding import Finding, Severity from wardline.core.finding_query import filter_findings from wardline.core.judge_run import run_judge @@ -65,6 +65,8 @@ def _emit_filigree( "auth_rejected": er.auth_rejected, "token_sent": er.token_sent, "url": er.url, + # N1 / C-10(a): name where findings went so a wrong-project write is visible. + "destination": filigree_destination(er.url), } @@ -78,6 +80,7 @@ def _filigree_emit_status(block: dict[str, Any] | None) -> dict[str, Any]: "failed": 0, "warnings": [], "disabled_reason": "not configured", + "destination": filigree_destination(None), } disabled_reason = filigree_disabled_reason( reachable=bool(block.get("reachable")), diff --git a/tests/docs/test_glossary_vocabulary.py b/tests/docs/test_glossary_vocabulary.py index 1453496f..0a543943 100644 --- a/tests/docs/test_glossary_vocabulary.py +++ b/tests/docs/test_glossary_vocabulary.py @@ -41,20 +41,20 @@ ("src/wardline/core/run.py", 312, "active=sum"), ("src/wardline/core/run.py", 363, "honors_suppressions"), # src/wardline/cli/scan.py — CLI summary line + gate stderr - ("src/wardline/cli/scan.py", 362, "suppressed"), - ("src/wardline/cli/scan.py", 370, "{s.active} active"), - ("src/wardline/cli/scan.py", 388, "gate: FAILED"), + ("src/wardline/cli/scan.py", 384, "suppressed"), + ("src/wardline/cli/scan.py", 385, "{s.active} active"), + ("src/wardline/cli/scan.py", 403, "gate: FAILED"), # src/wardline/mcp/server.py — MCP scan summary + gate block - ("src/wardline/mcp/server.py", 327, '"total": result.summary.total'), - ("src/wardline/mcp/server.py", 328, '"active": result.summary.active'), - ("src/wardline/mcp/server.py", 329, '"baselined": result.summary.baselined'), - ("src/wardline/mcp/server.py", 330, '"waived": result.summary.waived'), - ("src/wardline/mcp/server.py", 331, '"judged": result.summary.judged'), - ("src/wardline/mcp/server.py", 336, '"informational": result.summary.informational'), - ("src/wardline/mcp/server.py", 340, '"unanalyzed": result.summary.unanalyzed'), - ("src/wardline/mcp/server.py", 342, '"gate": {'), - ("src/wardline/mcp/server.py", 343, '"tripped": decision.tripped'), - ("src/wardline/mcp/server.py", 346, '"verdict": decision.verdict'), + ("src/wardline/mcp/server.py", 330, '"total": result.summary.total'), + ("src/wardline/mcp/server.py", 331, '"active": result.summary.active'), + ("src/wardline/mcp/server.py", 332, '"baselined": result.summary.baselined'), + ("src/wardline/mcp/server.py", 333, '"waived": result.summary.waived'), + ("src/wardline/mcp/server.py", 334, '"judged": result.summary.judged'), + ("src/wardline/mcp/server.py", 339, '"informational": result.summary.informational'), + ("src/wardline/mcp/server.py", 343, '"unanalyzed": result.summary.unanalyzed'), + ("src/wardline/mcp/server.py", 345, '"gate": {'), + ("src/wardline/mcp/server.py", 346, '"tripped": decision.tripped'), + ("src/wardline/mcp/server.py", 349, '"verdict": decision.verdict'), # src/wardline/core/agent_summary.py — agent-summary JSON keys ("src/wardline/core/agent_summary.py", 116, '"total_findings"'), ("src/wardline/core/agent_summary.py", 117, '"active_defects"'), diff --git a/tests/unit/core/test_filigree_destination.py b/tests/unit/core/test_filigree_destination.py new file mode 100644 index 00000000..86e02db7 --- /dev/null +++ b/tests/unit/core/test_filigree_destination.py @@ -0,0 +1,39 @@ +"""N1 / C-10(a): the emit destination must be visible at the caller so a wrong-project +write cannot read as silent success.""" + +from __future__ import annotations + +from wardline.core.filigree_emit import filigree_destination, filigree_url_project + + +def test_url_project_from_query() -> None: + assert filigree_url_project("http://127.0.0.1:8749/api/weft/scan-results?project=lacuna") == "lacuna" + + +def test_url_project_from_path() -> None: + assert filigree_url_project("http://127.0.0.1:8749/api/p/lacuna/weft/scan-results") == "lacuna" + + +def test_url_project_none_when_unpinned() -> None: + # The contamination shape: a bare endpoint pins no project, so Filigree resolves it + # server-side and a misroute is invisible unless surfaced. + assert filigree_url_project("http://127.0.0.1:8749/api/weft/scan-results") is None + assert filigree_url_project(None) is None + + +def test_destination_pinned() -> None: + d = filigree_destination("http://127.0.0.1:8749/api/weft/scan-results?project=lacuna") + assert d == { + "url": "http://127.0.0.1:8749/api/weft/scan-results?project=lacuna", + "project": "lacuna", + "project_pinned": True, + } + + +def test_destination_unpinned_is_visible() -> None: + d = filigree_destination("http://127.0.0.1:8749/api/weft/scan-results") + assert d == { + "url": "http://127.0.0.1:8749/api/weft/scan-results", + "project": None, + "project_pinned": False, + } diff --git a/tests/unit/mcp/test_server_filigree_emit.py b/tests/unit/mcp/test_server_filigree_emit.py index 6d5ff82d..993f8db1 100644 --- a/tests/unit/mcp/test_server_filigree_emit.py +++ b/tests/unit/mcp/test_server_filigree_emit.py @@ -66,6 +66,7 @@ def test_scan_emits_to_filigree_when_emitter_present(tmp_path): "token_sent": False, "url": None, "disabled_reason": None, + "destination": {"url": None, "project": None, "project_pinned": False}, } assert emitter.scanned_paths == ("svc.py",) @@ -88,6 +89,7 @@ def test_scan_reports_both_integrations_successful(tmp_path): "token_sent": False, "url": None, "disabled_reason": None, + "destination": {"url": None, "project": None, "project_pinned": False}, } @@ -103,6 +105,7 @@ def test_scan_filigree_block_null_when_no_emitter(tmp_path): "failed": 0, "warnings": [], "disabled_reason": "not configured", + "destination": {"url": None, "project": None, "project_pinned": False}, } From c58da865740832a5d610eba5e2656d371fdbde02 Mon Sep 17 00:00:00 2001 From: John Morrissey Date: Mon, 8 Jun 2026 07:21:43 +1000 Subject: [PATCH 010/186] fix(install): make managed-block injector C-4 foreign-safe (wardline-906d19b29a) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit block.py was the only non-conforming writer of the weft C-4 multi-owner managed-block contract: in a shared CLAUDE.md/AGENTS.md co-resident with a sibling tool's block (e.g. filigree in ~/lacuna) it could swallow a foreign block, delete an own duplicate sitting beyond a foreign block, or truncate the shared doc on a mid-write crash. Mirrors the legis/filigree reference algorithm: - _INSTR_FENCE_RE: generic namespaced-fence detector (open/close), foreign = ns.lower() != "wardline", case-insensitive per (h). - _first_own_open_fence_pos walks fences tracking the enclosing foreign block, so an own marker quoted inside a sibling block is never claimed (foreign-safe). - Three explicit branches: own_end==-1 -> append-preserve (d); own_end well-formed replace + canonicalise own dups up to the first foreign fence (e); own_end>=foreign -> bounded recovery, cut at foreign (c). Never spans a foreign fence. - _canonicalise_tail collapses own dups only before the next foreign fence; an own duplicate beyond a foreign block is preserved and surfaced (warning). - All writes routed through _atomic_write_text (mkstemp + os.replace, mode preserved) with a refuse-to-empty guard (g). Goes one better than the references: a convergence guard in the append branch declines to re-append when the exact current block already exists but is shielded inside an unclosed foreign block — the references grow the file unboundedly across repeated SessionStart hook runs on such input; wardline converges. Foreign-safety preserved (the guard is read-only). Preserves (b) namespaced match, (d) append-on-missing-end, idempotent convergence. 11 new tests cover (c)/(d)/(e)/(f)/(g)/(h) + convergence; full suite 2604 green, ruff + mypy clean. Co-Authored-By: Claude Opus 4.8 (1M context) --- src/wardline/install/block.py | 203 ++++++++++++++++++++++++++++--- tests/unit/install/test_block.py | 199 +++++++++++++++++++++++++++++- 2 files changed, 387 insertions(+), 15 deletions(-) diff --git a/src/wardline/install/block.py b/src/wardline/install/block.py index 7649c29b..26c4b733 100644 --- a/src/wardline/install/block.py +++ b/src/wardline/install/block.py @@ -1,13 +1,31 @@ -"""Render + idempotently inject the hash-fenced wardline instruction block.""" +"""Render + idempotently inject the hash-fenced wardline instruction block. + +The block lands in shared agent docs (``CLAUDE.md`` / ``AGENTS.md``) that may +also carry a co-resident sibling tool's managed block (filigree / legis / +loomweave). The injector therefore obeys the weft C-4 multi-owner managed-block +contract: it mutates only its own vendor-namespaced span, never lets a rewrite +cross a foreign-namespace fence, never reorders or relocates a foreign block, +canonicalises its own duplicates only when doing so cannot reach across a +foreign block (preserving + surfacing any duplicate beyond one), and writes +atomically with a refuse-to-empty guard so a crash can never truncate the shared +doc. +""" from __future__ import annotations import hashlib +import logging +import os import re +import stat +import tempfile from pathlib import Path +from wardline.core.errors import WardlineError from wardline.core.safe_paths import safe_project_file +logger = logging.getLogger(__name__) + _BLOCK_VERSION = "1" _BODY = ( @@ -19,43 +37,200 @@ "in `docs/agents.md`." ) +_OWN_NS = "wardline" +_END_MARKER = f"" + +# A complete, well-formed wardline block (open .. close). Own-namespace only, so +# it can only ever match wardline's own spans (C-4 (b) own-namespace mutation). _FENCE_RE = re.compile( r".*?", re.DOTALL, ) +# Recognises ANY tool's instruction fence (open OR close, via the optional +# leading ``/``) and captures its vendor namespace. wardline uses it to bound its +# own rewrite at a *foreign* fence and never delete a co-resident sibling block +# in a shared CLAUDE.md / AGENTS.md. The namespace is compared case-insensitively +# (via ``.lower()``) so an uppercase-namespaced sibling (e.g. ``FILIGREE``) still +# registers as a boundary (C-4 (h)). The cross-tool multi-owner block contract +# lives in weft conventions.md (C-4). +_INSTR_FENCE_RE = re.compile(r"\n{_BODY}\n" + return f"\n{_BODY}\n{_END_MARKER}" + + +def _first_foreign_fence_pos(content: str, search_from: int) -> int: + """Index of the first non-wardline instruction fence at/after *search_from*. + + Own-namespace fences are absorbed — never treated as a boundary — so + duplicate or unclosed wardline blocks still collapse to one clean block. When + no foreign fence follows, returns ``len(content)`` (i.e. bound at EOF). + """ + for m in _INSTR_FENCE_RE.finditer(content, search_from): + if m.group("ns").lower() != _OWN_NS: + return m.start() + return len(content) + + +def _first_own_open_fence_pos(content: str) -> int: + """Index of wardline's *own* top-level open instruction fence, or -1 if none. + + A wardline open marker quoted *inside* a co-resident sibling block (a worked + example, documentation) is textually identical to a real one, so a bare regex + anchor would splice there and gut the sibling. This walks fences in document + order, tracking the foreign block we are currently inside, and only returns a + wardline open fence found at top level (not enclosed by an unclosed foreign + block). An unclosed foreign block therefore shields any wardline marker + beyond it: we decline to claim content we cannot prove is ours, and the + caller falls back to an append (which deletes nothing). + """ + inside_foreign: str | None = None + for m in _INSTR_FENCE_RE.finditer(content): + ns = m.group("ns").lower() + is_close = bool(m.group("close")) + if inside_foreign is not None: + if is_close and ns == inside_foreign: + inside_foreign = None + continue + if ns == _OWN_NS and not is_close: + return m.start() + if ns != _OWN_NS and not is_close: + inside_foreign = ns + return -1 + + +def _canonicalise_tail(tail: str) -> tuple[str, bool]: + """Collapse duplicate own blocks in *tail* that precede any foreign fence. + + Returns ``(cleaned_tail, foreign_shielded_dup)``. Own blocks before the first + foreign fence are removed (own-duplicate canonicalisation, C-4 (e)). + Everything from the first foreign fence onward is preserved verbatim — + including any own duplicate beyond it, which foreign-safety forbids reaching + across; the bool flags such a shielded duplicate so the caller can surface it. + """ + foreign = _first_foreign_fence_pos(tail, 0) + head, rest = tail[:foreign], tail[foreign:] + cleaned = _FENCE_RE.sub("", head) + shielded_dup = _first_own_open_fence_pos(rest) != -1 + return cleaned + rest, shielded_dup + + +def _atomic_write_text(path: Path, content: str) -> None: + """Write *content* to *path* atomically (temp + ``os.replace``), preserving mode. + + Refuse-to-empty guard (C-4 (g)): every caller always has non-empty content (a + rendered block, or existing text plus a block), so an empty / whitespace-only + payload can only be corruption or a logic bug — refuse loudly rather than + rename an empty temp file over a populated CLAUDE.md / AGENTS.md. The + write-temp-then-replace makes truncation structurally impossible: a crash + leaves the prior file intact, never a partial shared agent doc. + + ``tempfile.mkstemp`` creates 0o600 files; without an explicit chmod the + rename would leak that owner-only mode onto a user-visible file, so the + destination's existing mode is preserved (or the process umask is respected + for a new file). + """ + if not content.strip(): + raise WardlineError(f"refusing to write empty content to {path}") + existing_mode: int | None + try: + existing_mode = stat.S_IMODE(path.stat().st_mode) + except FileNotFoundError: + existing_mode = None + fd, tmp = tempfile.mkstemp(dir=path.parent, suffix=".tmp", prefix=path.name) + try: + with os.fdopen(fd, "w", encoding="utf-8") as f: + f.write(content) + if existing_mode is not None: + os.chmod(tmp, existing_mode) + else: + umask = os.umask(0) + os.umask(umask) + os.chmod(tmp, 0o666 & ~umask) + os.replace(tmp, path) + except BaseException: + Path(tmp).unlink(missing_ok=True) + raise def inject_block(file_path: Path) -> str: """Create / append / replace the block, collapsing to one canonical block. - Returns created|updated|unchanged. + Foreign-safe per the weft C-4 multi-owner managed-block contract: never + deletes, reorders, or relocates a co-resident sibling tool's block, and never + lets a rewrite span across a foreign-namespace fence. Returns + created|updated|unchanged. """ file_path = safe_project_file(file_path.parent, file_path, label=file_path.name) block = render_block() if not file_path.exists(): - file_path.write_text(block + "\n", encoding="utf-8") + _atomic_write_text(file_path, block + "\n") return "created" text = file_path.read_text(encoding="utf-8") - matches = list(_FENCE_RE.finditer(text)) - if not matches: + start = _first_own_open_fence_pos(text) + own_end = text.find(_END_MARKER, start) if start != -1 else -1 + + if start == -1 or own_end == -1: + # No own block we can claim — either none at all, an own open with no + # close (so we cannot prove its extent), or an open marker shielded + # inside an unclosed foreign block. Append a fresh block and preserve all + # existing text verbatim (C-4 (d) append-on-missing-end). This is NOT + # recovery-to-EOF: trailing user text after an unclosed own marker is + # kept, never deleted. + if block in text: + # An identical current block already exists but is unreachable to the + # claim logic (shielded inside an unclosed foreign block, whose own + # close we must not invent). Appending another copy each run would + # grow the file unboundedly across repeated hook invocations; decline + # instead. Purely read-only, so foreign-safety is untouched. + return "unchanged" sep = "" if text.endswith("\n") else "\n" - file_path.write_text(f"{text}{sep}\n{block}\n", encoding="utf-8") + _atomic_write_text(file_path, f"{text}{sep}\n{block}\n") return "updated" - # Canonicalise to a single block at the position of the first existing one; - # drop any additional blocks (e.g. from a botched prior write) so repeated - # calls converge. - prefix = text[: matches[0].start()] - suffix = _FENCE_RE.sub("", text[matches[0].end() :]) - candidate = prefix + block + suffix + + # An own block exists with a close marker. Bound the span we rewrite so it + # never crosses a foreign fence (C-4 (c)). + foreign = _first_foreign_fence_pos(text, start) + if own_end < foreign: + # Well-formed own block, closing before any foreign fence: replace it in + # place, then canonicalise duplicate own blocks in the tail up to (but + # never across) the first foreign fence (C-4 (e)). + bound = own_end + len(_END_MARKER) + tail, shielded_dup = _canonicalise_tail(text[bound:]) + sep = "" + else: + # Bounded recovery (C-4 (c)): the own open is malformed, or its close + # lies beyond a foreign fence (so a naive open..close match would swallow + # the foreign block). Cut at the foreign fence (or EOF) instead. Re-insert + # the separating newline we may have eaten so our close marker is never + # glued mid-line against a following foreign fence — keeping us + # independent of whether a sibling's detector is line-anchored. + bound = foreign + tail = text[bound:] + sep = "\n" if (bound < len(text) and not tail.startswith("\n")) else "" + shielded_dup = _first_own_open_fence_pos(tail) != -1 + + if shielded_dup: + # A second own block survives beyond a foreign block because + # canonicalising it would mean reaching across a block we don't own. It + # is STALE, conflicting guidance — not a harmless duplicate — so surface + # it instead of silently shipping a split brain (foreign-safety wins over + # own-dedup, C-4 (e)). + logger.warning( + "wardline instruction block in %s has a duplicate beyond another " + "tool's block that could not be canonicalised without crossing it; " + "the stale copy was left in place. Resolve it by hand.", + file_path, + ) + + candidate = text[:start] + block + sep + tail if candidate == text: return "unchanged" - file_path.write_text(candidate, encoding="utf-8") + _atomic_write_text(file_path, candidate) return "updated" diff --git a/tests/unit/install/test_block.py b/tests/unit/install/test_block.py index 8a0ac5e1..e721ce72 100644 --- a/tests/unit/install/test_block.py +++ b/tests/unit/install/test_block.py @@ -1,9 +1,17 @@ +import logging from pathlib import Path import pytest from wardline.core.errors import WardlineError -from wardline.install.block import inject_block, render_block +from wardline.install.block import _atomic_write_text, inject_block, render_block + +# A co-resident sibling tool's managed block (filigree), used to assert +# wardline's writer never deletes, truncates, or spans across a foreign block +# (weft C-4 multi-owner managed-block contract). +_FOREIGN = "\nfiligree body — DO NOT TOUCH\n" +_OPEN = "" +_CLOSE = "" def test_render_block_is_fenced_and_mentions_the_gate() -> None: @@ -88,3 +96,192 @@ def test_inject_is_idempotent_after_collapse(tmp_path: Path) -> None: f.write_text(f"{block}\n\n{block}\n", encoding="utf-8") inject_block(f) # collapses to one assert inject_block(f) == "unchanged" # now converged + + +# --------------------------------------------------------------------------- +# C-4 multi-owner managed-block contract +# --------------------------------------------------------------------------- + + +def test_foreign_block_between_own_head_and_own_trailing_is_preserved( + tmp_path: Path, caplog: pytest.LogCaptureFixture +) -> None: + """(c)+(e): a foreign block sandwiched between an own head and an own trailing + duplicate survives, AND the trailing own duplicate (beyond the foreign block) + is preserved + surfaced, not deleted.""" + block = render_block() + # Stale head block (old hash) so a real rewrite is exercised; the trailing + # block is current and sits BEYOND the foreign fence. + stale = f"{_OPEN}\nOLD BODY\n{_CLOSE}" + f = tmp_path / "CLAUDE.md" + f.write_text( + f"head\n\n{stale}\n\n{_FOREIGN}\n\n{block}\n\ntail\n", + encoding="utf-8", + ) + with caplog.at_level(logging.WARNING): + assert inject_block(f) == "updated" + text = f.read_text(encoding="utf-8") + # Stale head body is gone (replaced in place). + assert "OLD BODY" not in text + # Foreign block survives intact. + assert _FOREIGN in text + assert "filigree body — DO NOT TOUCH" in text + # Surrounding user text survives. + assert "head" in text and "tail" in text + # The own duplicate beyond the foreign block is preserved (foreign-safety > + # own-dedup), so there are still two own blocks — NOT collapsed to one. + assert text.count("wardline:instructions:v") == 2 + # ...and it is surfaced. + assert any("duplicate" in r.message for r in caplog.records) + + +def test_unclosed_own_then_foreign_then_complete_own_preserves_foreign( + tmp_path: Path, +) -> None: + """(c): an unclosed own open before a foreign block, followed by a complete + own block, must not let the rewrite span across the foreign block.""" + block = render_block() + f = tmp_path / "CLAUDE.md" + # Orphan own open (no close), then foreign, then a complete own block. + f.write_text( + f"intro\n\n{_OPEN}\nORPHAN BODY\n\n{_FOREIGN}\n\n{block}\n", + encoding="utf-8", + ) + assert inject_block(f) == "updated" + text = f.read_text(encoding="utf-8") + # Foreign block survives. + assert _FOREIGN in text + assert "filigree body — DO NOT TOUCH" in text + # A valid fresh wardline block is present at the front (orphan recovered). + assert text.lstrip().startswith("intro") + assert _CLOSE in text + # Two own blocks remain: the recovered front block + the complete one beyond + # the foreign fence. We do NOT assert count == 1 — that would contradict + # foreign-safety (the trailing block sits beyond the foreign block). + assert text.count("wardline:instructions:v") == 2 + + +def test_two_call_orphan_open_plus_foreign_keeps_foreign(tmp_path: Path) -> None: + """(c): the reachable 2-call sequence — orphan own open + foreign, then inject + twice — must keep the foreign block across both calls.""" + f = tmp_path / "CLAUDE.md" + f.write_text(f"{_OPEN}\nORPHAN BODY\n\n{_FOREIGN}\n", encoding="utf-8") + # Call 1: append path manufactures a later own close. + inject_block(f) + assert _FOREIGN in f.read_text(encoding="utf-8") + # Call 2: must not swallow the foreign block now that a later close exists. + inject_block(f) + text = f.read_text(encoding="utf-8") + assert _FOREIGN in text + assert "filigree body — DO NOT TOUCH" in text + + +def test_uppercase_foreign_namespace_registers_as_boundary(tmp_path: Path) -> None: + """(h): an uppercase-namespaced sibling fence must register as a foreign + boundary (case-insensitive namespace match).""" + block = render_block() + upper_foreign = "\nFILIGREE BODY\n" + f = tmp_path / "CLAUDE.md" + f.write_text( + f"{_OPEN}\nORPHAN BODY\n\n{upper_foreign}\n\n{block}\n", + encoding="utf-8", + ) + assert inject_block(f) == "updated" + text = f.read_text(encoding="utf-8") + assert upper_foreign in text + assert "FILIGREE BODY" in text + + +def test_append_on_unclosed_own_with_trailing_text_preserves_all( + tmp_path: Path, +) -> None: + """(d): an own open with no close marker, followed by user text, must append a + fresh block and preserve all existing text verbatim (no recovery-to-EOF that + would delete the trailing user text).""" + f = tmp_path / "CLAUDE.md" + original = f"intro\n\n{_OPEN}\nORPHAN BODY\n\nIMPORTANT USER TEXT\n" + f.write_text(original, encoding="utf-8") + assert inject_block(f) == "updated" + text = f.read_text(encoding="utf-8") + # Every byte of the original is preserved (append-on-missing-end). + assert original in text + assert "IMPORTANT USER TEXT" in text + assert "ORPHAN BODY" in text + # A fresh, well-formed block was appended. + assert _CLOSE in text + + +def test_append_on_unclosed_own_no_trailing_text(tmp_path: Path) -> None: + """(d): an own open with no close marker and nothing after it still appends a + fresh block, preserving the orphan.""" + f = tmp_path / "CLAUDE.md" + original = f"intro\n\n{_OPEN}\nORPHAN BODY\n" + f.write_text(original, encoding="utf-8") + assert inject_block(f) == "updated" + text = f.read_text(encoding="utf-8") + assert original in text + assert "ORPHAN BODY" in text + assert _CLOSE in text + + +def test_atomic_write_refuses_empty_content(tmp_path: Path) -> None: + """(g): the atomic writer refuses empty / whitespace-only content rather than + truncating an existing populated file.""" + f = tmp_path / "CLAUDE.md" + f.write_text("PRECIOUS CONTENT\n", encoding="utf-8") + with pytest.raises(WardlineError, match="empty"): + _atomic_write_text(f, "") + with pytest.raises(WardlineError, match="empty"): + _atomic_write_text(f, " \n\t ") + # File untouched. + assert f.read_text(encoding="utf-8") == "PRECIOUS CONTENT\n" + + +def test_atomic_write_preserves_existing_file_mode(tmp_path: Path) -> None: + """(g): the atomic writer preserves the destination's permission bits.""" + f = tmp_path / "CLAUDE.md" + f.write_text("old\n", encoding="utf-8") + f.chmod(0o640) + _atomic_write_text(f, "new content\n") + assert f.read_text(encoding="utf-8") == "new content\n" + assert (f.stat().st_mode & 0o777) == 0o640 + + +def test_foreign_block_before_own_is_preserved(tmp_path: Path) -> None: + """(f): a foreign block before the own block is never reordered/relocated and + a stale own block is still replaced in place.""" + f = tmp_path / "CLAUDE.md" + f.write_text( + f"{_FOREIGN}\n\n{_OPEN}\nOLD BODY\n{_CLOSE}\n", + encoding="utf-8", + ) + assert inject_block(f) == "updated" + text = f.read_text(encoding="utf-8") + assert _FOREIGN in text + assert "OLD BODY" not in text + # Foreign block stays first; own block replaced in place. + assert text.index("filigree body") < text.index("wardline:instructions:v") + assert text.count("wardline:instructions:v") == 1 + + +def test_own_marker_shielded_in_unclosed_foreign_converges(tmp_path: Path) -> None: + """(d)+idempotency: a wardline open marker shielded inside an UNCLOSED foreign + block is not claimable (we must not invent the foreign close), so the first + inject appends a fresh block. Repeated injects must then CONVERGE rather than + append a new copy every run (unbounded growth under repeated hook calls).""" + f = tmp_path / "CLAUDE.md" + # Unclosed filigree fence with a wardline open quoted inside it. + f.write_text( + f"\nexample: {_OPEN}\nsome text\n", + encoding="utf-8", + ) + assert inject_block(f) == "updated" # appends a fresh block + after_first = f.read_text(encoding="utf-8") + # The (malformed) foreign marker and quoted text are preserved verbatim. + assert "" in after_first + assert "some text" in after_first + # Second and third runs converge — no unbounded growth. + assert inject_block(f) == "unchanged" + assert f.read_text(encoding="utf-8") == after_first + assert inject_block(f) == "unchanged" + assert f.read_text(encoding="utf-8") == after_first From 779e3c5f837c81a47b7290c5a2f6e5163674e27a Mon Sep 17 00:00:00 2001 From: John Morrissey Date: Mon, 8 Jun 2026 07:32:33 +1000 Subject: [PATCH 011/186] fix(install): close two C-4 review-panel findings in the block injector MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Adversarial review panel (6 reviewers) against the C-4 contract surfaced two real, reachable defects in the prior commit: HIGH (b,c): a foreign marker merely QUOTED inside wardline's own block body was mistaken for a real foreign boundary by _first_foreign_fence_pos (no balance tracking), forcing spurious bounded recovery — the rewrite truncated at the quoted marker, leaving a dangling own close + a stray foreign open that read as a real foreign block on the next run (sticky corruption). Replaced with _first_real_foreign_block_pos: a foreign OPEN counts as a boundary only when it has a matching foreign CLOSE (a genuine sibling block). A lone quoted marker is our own content -> the block is replaced in place; a genuinely nested balanced foreign block still triggers bounded recovery and is preserved (NOT swallowed). MEDIUM (e,h): _FENCE_RE was case-sensitive while boundary detection compares case-insensitively, so an uppercase-namespaced own duplicate (WARDLINE) was neither collapsed nor surfaced. Added re.IGNORECASE so own-duplicate canonicalisation matches the case-insensitive namespace rule. +4 tests (quoted-foreign-in-body replace-in-place + converge; genuinely nested foreign preserved; uppercase own dup canonicalised). Full suite 2607 green, ruff + mypy clean. Co-Authored-By: Claude Opus 4.8 (1M context) --- src/wardline/install/block.py | 62 ++++++++++++++++++++----------- tests/unit/install/test_block.py | 63 ++++++++++++++++++++++++++++++++ 2 files changed, 103 insertions(+), 22 deletions(-) diff --git a/src/wardline/install/block.py b/src/wardline/install/block.py index 26c4b733..7d117ef2 100644 --- a/src/wardline/install/block.py +++ b/src/wardline/install/block.py @@ -42,9 +42,12 @@ # A complete, well-formed wardline block (open .. close). Own-namespace only, so # it can only ever match wardline's own spans (C-4 (b) own-namespace mutation). +# IGNORECASE so an uppercase-namespaced own duplicate (e.g. ``WARDLINE``) is still +# matched for canonicalisation, consistent with the case-insensitive namespace +# comparison used for boundary detection (C-4 (e)+(h)). _FENCE_RE = re.compile( r".*?", - re.DOTALL, + re.DOTALL | re.IGNORECASE, ) # Recognises ANY tool's instruction fence (open OR close, via the optional @@ -65,15 +68,28 @@ def render_block() -> str: return f"\n{_BODY}\n{_END_MARKER}" -def _first_foreign_fence_pos(content: str, search_from: int) -> int: - """Index of the first non-wardline instruction fence at/after *search_from*. +def _first_real_foreign_block_pos(content: str, search_from: int) -> int: + """Index of the first *real* foreign block at/after *search_from*, else EOF. - Own-namespace fences are absorbed — never treated as a boundary — so - duplicate or unclosed wardline blocks still collapse to one clean block. When - no foreign fence follows, returns ``len(content)`` (i.e. bound at EOF). + A real foreign block is a foreign-namespace OPEN fence that has a matching + foreign CLOSE fence somewhere after it — i.e. genuine co-resident sibling + content we must never delete or split. A *lone* foreign open (a marker quoted + in prose or inside wardline's own body) and a stray foreign close are NOT + boundaries: they are our own content, so a well-formed own block whose body + happens to mention a sibling's marker is replaced in place rather than + truncated at the quoted marker (C-4 (b)+(c)). Own-namespace fences are always + absorbed, so duplicate / unclosed wardline blocks still collapse. + + Returns ``len(content)`` when no real foreign block follows (bound at EOF). + The namespace match is case-insensitive (C-4 (h)). """ - for m in _INSTR_FENCE_RE.finditer(content, search_from): - if m.group("ns").lower() != _OWN_NS: + fences = list(_INSTR_FENCE_RE.finditer(content, search_from)) + for i, m in enumerate(fences): + ns = m.group("ns").lower() + if ns == _OWN_NS or m.group("close"): + continue + # Foreign open: a boundary only if a matching foreign close follows it. + if any(n.group("ns").lower() == ns and n.group("close") for n in fences[i + 1 :]): return m.start() return len(content) @@ -106,15 +122,15 @@ def _first_own_open_fence_pos(content: str) -> int: def _canonicalise_tail(tail: str) -> tuple[str, bool]: - """Collapse duplicate own blocks in *tail* that precede any foreign fence. + """Collapse duplicate own blocks in *tail* that precede any real foreign block. Returns ``(cleaned_tail, foreign_shielded_dup)``. Own blocks before the first - foreign fence are removed (own-duplicate canonicalisation, C-4 (e)). - Everything from the first foreign fence onward is preserved verbatim — - including any own duplicate beyond it, which foreign-safety forbids reaching - across; the bool flags such a shielded duplicate so the caller can surface it. + real foreign block are removed (own-duplicate canonicalisation, C-4 (e)). + Everything from that foreign block onward is preserved verbatim — including + any own duplicate beyond it, which foreign-safety forbids reaching across; the + bool flags such a shielded duplicate so the caller can surface it. """ - foreign = _first_foreign_fence_pos(tail, 0) + foreign = _first_real_foreign_block_pos(tail, 0) head, rest = tail[:foreign], tail[foreign:] cleaned = _FENCE_RE.sub("", head) shielded_dup = _first_own_open_fence_pos(rest) != -1 @@ -195,19 +211,21 @@ def inject_block(file_path: Path) -> str: return "updated" # An own block exists with a close marker. Bound the span we rewrite so it - # never crosses a foreign fence (C-4 (c)). - foreign = _first_foreign_fence_pos(text, start) + # never crosses a *real* foreign block (C-4 (c)). A foreign marker merely + # quoted inside our own body is not a boundary (see + # _first_real_foreign_block_pos) — that block is replaced in place. + foreign = _first_real_foreign_block_pos(text, start) if own_end < foreign: - # Well-formed own block, closing before any foreign fence: replace it in - # place, then canonicalise duplicate own blocks in the tail up to (but - # never across) the first foreign fence (C-4 (e)). + # Well-formed own block, closing before any real foreign block: replace it + # in place, then canonicalise duplicate own blocks in the tail up to (but + # never across) the first real foreign block (C-4 (e)). bound = own_end + len(_END_MARKER) tail, shielded_dup = _canonicalise_tail(text[bound:]) sep = "" else: - # Bounded recovery (C-4 (c)): the own open is malformed, or its close - # lies beyond a foreign fence (so a naive open..close match would swallow - # the foreign block). Cut at the foreign fence (or EOF) instead. Re-insert + # Bounded recovery (C-4 (c)): the own open is malformed, or its close lies + # beyond a real foreign block (so a naive open..close match would swallow + # the foreign block). Cut at the foreign block (or EOF) instead. Re-insert # the separating newline we may have eaten so our close marker is never # glued mid-line against a following foreign fence — keeping us # independent of whether a sibling's detector is line-anchored. diff --git a/tests/unit/install/test_block.py b/tests/unit/install/test_block.py index e721ce72..925be143 100644 --- a/tests/unit/install/test_block.py +++ b/tests/unit/install/test_block.py @@ -264,6 +264,69 @@ def test_foreign_block_before_own_is_preserved(tmp_path: Path) -> None: assert text.count("wardline:instructions:v") == 1 +def test_quoted_foreign_marker_in_own_body_is_replaced_in_place( + tmp_path: Path, +) -> None: + """(b)+(c): a foreign marker merely quoted inside wardline's own well-formed + block body is OUR content, not a real foreign boundary — the block is replaced + in place, not truncated at the quoted marker (which would leave a dangling own + close + a stray foreign open that sticks as corruption).""" + f = tmp_path / "CLAUDE.md" + body = "Docs: the filigree marker is " + f.write_text( + f"intro\n{_OPEN}\n{body}\nOLD BODY\n{_CLOSE}\ntail\n", + encoding="utf-8", + ) + assert inject_block(f) == "updated" + text = f.read_text(encoding="utf-8") + # Exactly one own block, one own close — no dangling/orphaned markers. + assert text.count(_CLOSE) == 1 + assert text.count("wardline:instructions:v") == 1 + # The quoted foreign open (which was our body content) is gone with the old + # body; no stray foreign block left behind. + assert "" not in text + assert "OLD BODY" not in text + assert "intro" in text and "tail" in text + # And it converges. + assert inject_block(f) == "unchanged" + + +def test_genuinely_nested_foreign_block_is_preserved(tmp_path: Path) -> None: + """(c): a genuinely balanced foreign block nested inside the own span (own + open .. foreign open .. foreign close .. own close) is real foreign content — + bounded recovery must cut at it and preserve it, NOT replace across it.""" + f = tmp_path / "CLAUDE.md" + f.write_text( + f"intro\n{_OPEN}\nOLD BODY\n{_FOREIGN}\nMORE OLD\n{_CLOSE}\ntail\n", + encoding="utf-8", + ) + assert inject_block(f) == "updated" + text = f.read_text(encoding="utf-8") + # The nested foreign block survives intact. + assert _FOREIGN in text + assert "filigree body — DO NOT TOUCH" in text + assert "intro" in text and "tail" in text + + +def test_uppercase_own_duplicate_is_canonicalised(tmp_path: Path, caplog: pytest.LogCaptureFixture) -> None: + """(e)+(h): an uppercase-namespaced own duplicate (no foreign block between) + is collapsed, not silently left in place — own-block matching is + case-insensitive, consistent with the boundary comparison.""" + block = render_block() + upper_dup = "\nBODY2\n" + f = tmp_path / "CLAUDE.md" + f.write_text(f"{block}\n\nmid\n\n{upper_dup}\n", encoding="utf-8") + with caplog.at_level(logging.WARNING): + assert inject_block(f) == "updated" + text = f.read_text(encoding="utf-8") + import re + + # No own block remains uncollapsed (case-insensitive count). + assert len(re.findall(r"guX5l;Rx8a?pxp>Q=9&be~z%%_u zzK}2Ci+K|!yGt>jUB+8@D{tes^LF0BJ9!sh&hOwW_?>vdyprF=SK$n&)qD-saqr>X zd>vnpHO3A6KE4s}L2Tk*=bQQc`~m(A%xGhHGd@KJZzIXC1 zzMX%YKZK_wJNUzVC;u*(qkMkcI5q!V#l<_qG5r#y^@D;d& z_#)gh_=f2ZjK?w0{RQ4k4dPGmAM*qJN&Xannjhpp;m`1&@@M(a_#yst{v4izALhT{ zNBA%KQT{wX#(%|+^B4Gw{3U*Z|C+zdU*RYDt2m+eH~e+}TYifFj-Teg=V$mI_#6C> z{7wER{uY0mpXGn%=lEavdHz@a4u6+l;P3H^{BQhyL+}syCH{AQ*;tP+)%o)e`A5cH z{tx3`{;{#0|C3+gpYTulXZ$MvoL}Sr!V|rJ<3UpxzL0s@corS_H;hz#i|Sj(Hhin~ zzc3tl6oZ?sf(dTyG2ok{}M1+bk5iTM`q=*vH zB1XiDI1w)rL?4kT`idmcPb7=}B1NQ%G@O}~E;2->$P(EiM+_8$M6Spa`C_neiUQ#h zg~Ba7qDT~r5>YCKh%!+whKgZgxTp}7qDoYY8c}QfOw@@HVx$-)MvF0GtQaT8iwXF) z{4HXVm@KAjJx#76Nou}OSg zY!>&62gEnT7V)6iD!wVUiEoMR;@jdO@g1>4JS=vK?}}aGdt$eEMC=jY7mtd^#9r}V zVxRbdcwGEY>=!>0Plz9j1L8^Xlz3Vk6h9Hqh@XmQ#m~ec@pJK<_-}Do{6ZWNzZ6Hs z^WvEJl{hY55HE_C#0l|h@v?YDoD{E$*Tiqc>*BZKl=z)EEq*V~h(Cxo#2>|*;!ol& z@wPZC{w&UkzlihVui_o?uDF1`y@AF?V*|d?y9r+qS|{EU7qQNB!RQu$GmeV)#RuY& z_`A3)J`^8`e~6F8KgAXCiTG4}Ca#Ll#WnFSab5h|G)!i4Q{WqEex}3p$IiDvGsp}! zL(EV!%nUap%t$lJj5cF%R$ZJKZzh<1%tW)VnPm1elg<8SikWJrnFGvpGsDa@v&?KW z#~f%5GIPy5Gv6F+I?V#pWfq!l(_B2_ zbA&n49A%C+$CzWyaprh)f;rK=#hheLHm8_V&1vRzbB1}VIn$hF&Ngo|=a_TNd1k#i z-&|len2qK_bCL0Y@g}Cs_Zx2;XN*7NyTpIO`qdl8TgF*)vDsuUF_)Un<}$OzY&F}= z+s$^f!|XJ>%;n}C<_hypbESEgxyroTTy3r~*P8d3-R3%Ty?L*>!MxAhXnxJyWPaV; zY~F7^V1C2gVm@eYHNRahtzXd9*%(~Cps9U9*RqApjVpqx8(KT-7c6LO>2%aA zs7GpC+FR>89kr5IN38@Z)K!}0YE8OYld9G%SG$91z36ln&0STcf34=GR&yh@P$i{t z7Sy>NBP6BZ5kBS!g|oor3L4?XYL1*=-yS^Dm&P&5rqgPzgX$9hQJUH)Ma?m)v#GhE zF?h5ur@~%X0BNndOXdx{*BB{X>Q4Z^z$G{dEml(V3u&YS}MI;PmN7fH69u7XO(RGZAT zTDICEzo|Z1YBj<-t&v(wHZ4AE4;Pi&3 z#`eaJrVhvSMeX&=8-s84MJjc|l5UkzIw*DTbQNmt)|EJBO0+>Uy+-Azw+U8Rr!rO* z>uZ6&7V4{quBex~cq(NW_0p=RI$vd}N?%!}uNqHfp?|#=x?b0W`bH(_0-r2F3w+!) z$kK2$N+unR-m)pEtvMwnTvt=Ma{>T0xpYP1S#bn(_$D%QnZUFX;8v-=t?Rjrjvr>fBf zR9g_*+QaT!wd|@6R|U7)>TtV~tKG)1r`XYM770!T|NnFwcK@D54Dy=T9Vqj(5@a5bXgMUn%gC7ZkMXLcPM7>@G%>B zhqvb5p=$1(zDQ*nFtt1FHCJjJHCOAjuGFzoq77QO0+qTFyc(r)qNn*K&HSa##pYku6CACHA!(|2qB+?p@lmbyI_-Q~v^wz%G`YV54{8&SV(S-q?vg{5x)+P01+v=in`M2T8NPg#sRa}-8@eq-v}+UhY3 zS~kC-o{#I|HV;W}U?kGUbNa%h%hAV^YEw^DGG zL<=he0T5Q+DE?K~msZ_Ou@Bj7pW@aRHHVR zv%nd(_)ZKHTIQ-tesj~J`aV5C_L#(;F&LH9gE5&YTt}fdmf>g}wx{Xa6E@c#7op>t z8#_8Af>;}1t`{1oq4gLLsDYDA95o-KhsNf{g`IOP24V=bw6U`XFyjUwrJk!{?euJ3 zk+!Gul1{Rxo7>X0Y<^=qDoBfBK$XT^b&;SxJwW!D_?|Id3NadfdskyeXH%=f9I1hg zw|CVyOT-ZiTG6D@j3S(okidzP3Czqf?Y^aW>Ix zP@57HY~UU;C)y%>w~|1HFKjbs+pSFH{GU?Ym?74d!KY_CA6+eI+d3Aswl_)&>9!2M zs2g+`Efwu8B$+Wn!F)vGYyg|xL@BVn0CiRwA#v%fHUu|=eW{aed{PtQ3(8!HzVMAv zs(n^NM^|xeX<&T|+WY3lCOu{;R)ZZUc1BcBqQrHzb+XZRIjic-226R(+AgajtCAxq zms5>MU6>R}^e#0saJke-8*q<5NGwA)5)5TEEw68GY+2A4+_<6%#^2J}RNt)fsga$_ z<*W`;Tv3)l+;()$?`Z5)iAp>{&8>@?7Sz*xG*I8Rwo|0jS*5x-mm15XYm{R@Fj5@S z-bgEN3R9RBK;z~`jCn^#)3PSKb`;Wv$tz+c)qsw!1&ccu*LQ|lP?aeV1eYTbn6Gbd zZ@r_dP4ZVzS{x+9vOU+Jl0XS+XuU(OK0#4XJY}Lfs?!uH$o$sM#pxic3$a=CV``H$0`vT0EsX zf2qb-D#!C!3Q^YOQOjfq*ZDVhV#TeY(NFsR4NKKka$Bvrt<_hx3h45vl?0ckT4PlU zNC<1Ls*3{`HZ?alHq38bq2#VqgB4Gu=1VOlxjecQJi79D)Z86#oep#N&i1DIMcNQN zSYno8U5XyHZtL>sVVOrQbs?Y{)BW|FGX6DYM}%8tbCe3 zHJ?IyHT?9b`KimJ)(2c3HNSRwoZ7C`eBI?y^FzdIe$~1H!kQna=10xvkzU(Vk;T77 zU+clG)4MgkVy#CtF9tuFKQ*5RKUzLL9QLTD5O9mW&ac}Pk5kiA^LLl0Sf^9-V&J#Z zTk>dr)w&4y*YRrI0CH-6)chA=&A*zbyF7(fSj(s8H^t^YuOI;_?`Aiu`1)@u-N@vH4wt(PF34y$ztgmt;9bqwfBmy=p=fWCA+ zQtKM9TPt46r`ADG9vZJ&j{<$mel=XJPk^4KH!Z*J6+LSG3Hh|#*!ijInYIJ9?f`yt zSUmy&KiYoOx(WEu{HpaW(9`j{&+`;m<*w-$Y56_1!Kxi8%6B>o^nEeiOFjzqG_j~s z$z7z|!QujsV_8!Ro?tmT8W*&-G|1Eiy7m+l6a~_3VHsv(DyGm`=(o7FtD{k2)m#Ab;cM7%=O;M5m1^p z=!@{($s{qpB)$<`IE|*%98V^S_l0{*N}{ca8^-RwXk!`ejjT8oX9 zC>d={ekF#)9-tnj5h|lB&LXXlcZo)^M72XZ8>0lVv#+$GYKTmx1ns5Dq)9rqIpxh_ zgvlh{S{3IF_8_Tr<14jjnM_%RC6p3D!F(zE*;1-FyC|wWZ-%S#RB%Ed#Z+3eXIIR7 z6DSTOL0ds7Ts}4=E+v2jdb6v9H!P6bTLc)5e(TzIA{uX_2nP(Md1bQTB7=XwiTm(359kMQVD;dfRkyw2+U`sPjzljA{W zp?=O*sGm<2Ix7R^<3(F;cj)5Q)}{6HTbE-*;KeD&A+Y0qVwn8;{a~2ksCuiC70(-ztyg;!PX;+dhtxHi+(n1Zi=&l1wbRDoO z#Z_xdR;Vb4S`Rs7RG9Z6t_%iA^{HE_I(Zvuf8;D7;l8KFDzSXvt#0K*Zh0F>52F=O z>P_ATt0&g--cLP1k-h|YnUkPkiHeM6!JXPZi(rp(tcok$J8ke%+kK(FciGa@y_KHs zt@LzX7@RnUnJ)*qXP9R>&JS*iYdf!`tYLy}(|Ak9W=CpJLb_OK+FMe+%!IGrU)R7XCT* z7x;h0o8>q$jeP+B?+g;*z46a+x)yD+V0c^H0Y88Tzz@S7SBAI5vHO?*2=6L0yo3A< z_E{F2*i(x49oymGW6FJ)&}`E~3UNbZJ5#tDM4Wj}-yy;8jVZ8hCmFwK$ zpM+h)6lU*#aK$efVEu@3`3{`kt3&!eyZ`#K8@3c@aY%7-%|W?-iTuc^Zs z!(q+!oh>L)oHzw)0hKdrQw%qrzmY$Q{4td?;g6dz7XG9OuvLNjGfCXW{2P}w9Aq&Ee4_FX(eZYc%1s23+k_BPU2P_C!VL`wP3&PG1SP*u9 zz=E&?1QvunAg~~|Q0`V}A`1e2SP;9NEC@S3U_sdR0Sf{jupsRHfCXXy2P_EOz=Bve zSrA)C7R1(*1+ja{g4hPKAhwAth<%+bh;62`*h5fu%gTF%6^~wH0B;rpdI(s^cgvV>5UryFWM$y3GE6#Rr3vzz_d&pSVS; zoBW`I$babnj<2W(PQ=*=^S+9IGq(i!PiQ`925Ce2cASB2;BD;}`Wq0t-Jbr7?}JVd zpZ^&@a4+Hca=%wv2PF^6m;Cg!f&YXid(%MRal-sn{lI4^<(K>a6E;h(FPBK!|Cguv zqS#*R&NV=J(EkhmJJOz|AB^5VIO@y&Afyb={=ee~zDB%#6+dVb>g2ZnnE$BMiQSj_ z3A_kyuJ+=e2K=mo2fY*YKI!K_LFigTB<4y2$(@ACjJy z@_to67Zl5bzTE#Ga7tceiI5!v-~Tb+*G7Q~#uHY5=Y!j8J=Jv8ZcDCY-|9VOA zPmjZ7{uG5GL?i!O{^@o zC@^-X^bbhCOLB87d?_cUiz){<6pebKLF0E|U+7I0?!`-^Si$xqMrnhj$Ltrl1?B@5 zbuOXn@O2%A5irUBsyBxk2NUN4C#_p}RF2sQGtA40#w^?`9<37NQZIy&q(UNSgD9TC zPTrpL$xVP;^T`czJ~>{_CpXCXFuP z#f@#F?JsPH^mj{tZ)mzBn97-Bc}8gDSZhMl7i7!P9K@OJDI-f8SI9>H$meb^bj zALlkaiM`U#8i%k0`xn@2ecX7__%&wOuNkl7go87vBX47e4|DEd7KSq*qH)qg0#0{GV*PQRLpsZ3+4x3f9!_g;VJ=>Ta~X!4Fy6BL$W)dx!j>Q3G> zc85%HyYw-dN9tDi9P{UuGSnsgPU$a~KF(9%*Tfr+OvehxR>%2(2**eM^8+FRBK+Gh zwZ;{{gF^GXpCEw+8MHstcMI^iY=pcG-q0O+Lp9k1Hy`y4@en+_yLcn*Q8HNKUB0l{l$!VMazqpXS8Iz1^;s9 zl+1;h&t|@!m7X;$YtKMu)`4uE9g&@$y=|a#pfh_`vb#^drFEn4n5ThsvF*B5n}`C<6)44pQ#bLjbDd|3OiEyEqd(}(Y?I5hldMP@}w z1@0;-(1=qbT_eYie01dDQE{W3qq;}!9QE<&u+ixe zw@kZb#VyxlTf?v}$8qpaA)P{k71_3SJ9a_1ASfh)_`dc;D|Hk69ir?6FiTA!oT(FP6^Jvd5gDV4VRyNL+ z8G@C;gY0FTCh`_e2@&$N4y;My#EyLU8GJDOOzwoA#S7qPa~J#^UI>36cf%jVJ@9jR zkzvXcG&Y($aLxmrpYb)m6+7@l5$1e7e1S7GzKN?JPS{w3Q#7{87@X#ToSXT>*pW|h z%hjnF^8NvwmhlMo-4ljyaPqgv`v>8hyi>f3h1MPTdNxk=!07?+iR;+uPrLlXu$w;( zd-Vz5cj&AQahCF7pFcs~qEiE~+kc1A2Pb{3#t9#Evd7nPPRCB1(6JjQ5xkG{1*|g# z{)ICH95@#s2`2#zMxHdBA3~>xti`z?_w!viv*2$yk3gM9@UcFD;NSAp0e_q|5P{PL z;&842ohXokQv`BwdO$v~1k2M$y7--Zow$Y*3Hsx#fpp2;?Ko{93nvT^zk}clIvoMm zAe>r)vr5DToME8OE$9m<13$f;x>6=jP^sWmyhfg!G72ZAOpvFgOvmXcKgL-fFUT`K zUX|xIydqc8O`N-tjGngwmc`0WC|9ROjj*(`Lk6!4tHZa3DZXy8!D_Ed3#72L#WFm1 zsC^$i$rRc;ImR$t>K|hlK3hD0Gp)KIz^5GL1g*M(pfwP_#8iu#T!)#V#O$Y2!0zj4 zeVxt~C12Kr+DjPqIuP>1$RZ6Rh>;j6fZN(L(s8stXAmoC)E+lP?Q!}^2(R5+ahnTA>PT+wqiTWYpQn5b!> zA2?YhDd1>Xq~tZ@YsO>Ad@JA!B{hPwni9ZP8N6s!jp89nV_gn93yT)kp!|UaPl_Zi zYO^7`xr28~D;H$Dq8B?YN(Wvzj1e`TMKPcfgR-YEt3HCbbcxe?w!}R2{0&+LGzLE7 zKCLKgMV|mT=M#IQIQ*!=$a$W!C=P$h;GR96J(?n<_iWV^XlG4 zAYq53b&IAIfYXFB@txT^91^$??Kqw8&xQjhxuG9Y6oz4M0;YIQxQXX3#WO!4EvQG! zG0bT&a&E#Wc^Q^saCf)6TZUOxKH}POPx?<(DTZDLY@UYrEI{d}UDBG+TUodLayH(ze((a|*DsM*WX5egw zZ_$`(i1L~AlMr65YNwMW9hw>Eo1RG$ZZerwMQMbuk zb_uOM-0)3E53IviA`Du%YF^49|L`Tlm#FxPDTpJk2!Dp6S+*arks3m9Y57@OrRDGR zwm<2xcxtQZi%qNy&$!`u+4A903iEK}D{BCx0e2Q{lu6m7VV8zok`O-p^zhTLJX+r% zI`Zsds;6bSfU5E)7-32Pri5V!Wx8zXORh4nNv?K#A#BJ+ev4N#w*F+uDTBfBVQs@` z9wxC4IckVubLb}_e0bgPI#9-%oWwVfBpdDm1T7&szz6A^8-Xc(7`0*s&a3xoDHA1+ z_Tjsd1Anwt>1fqzaC%$=+67=6f);^X3CWCb$;${2lUIu;vKp9O4^hWI#B zZN>wDbBugO^;UYt;0BHtKMG%KqIjzzinr2FLio^)LpLJTT7(F90ZB2m6Od-*0CQjr}P{08*8s4floe`4q`ZkG9hD1o{Q#G#{Q_=u39NmwFn?5&wY1Xy9aY zX=k2QfWZM;PKA{LR&Ap`5i4Q6Q761ZkY|X;JF-p92K8*u zanEt!z=&+fb3>j(*->32thGda$UfvF>)P#wAvWcb!BZ}IHcOb#dgZ7HEfKxd#Bn|& zdKZZcWu5XYpDX#gY0T?EO9UrM77vaX9D(%8f(9C9$&~zS^ph|#B$<8&%fBFT8t4az z=YS!=p8vXXh#5s(mwtqn0^_^<=kg&Dpp+JTyHBC^UeDi~zgI&o{`4pkGr}E+5xfGY zM5-~%4KoBcEEA5qx8%6C;C+1I-5VS|7XaVwv5k-h!3OEribvsmbNNfjkmd;im@AJCS$7 z8@^zOyhBD5#!JKpzGtY@y@qsq3SaZ4JA&}mK_TAo0ff8V*)q&Z_JM{q3LCF6pw-NS zJV-I##b9h6C4BK;(UmkcX`CFBJlM3^U%lqJws?#Idj zc8jKu-YEBM?pdAAtF^54;2=^Xz#K-J4iNBXINC9kL zJCcYENf;7?9*3`KI1d`ad60e*!aa|B9!DxF1(NbM;@qz(@##mEQEy)eFa_}+aJZA8m?ap%KkUR{2 z$`Cp0u{=FcLb!XMdmpGJNQ?%~vW5oS+YFX72T)VeBN*|p0+>b*zN}$h9<;>=Ym1-m zwZfZf=wLUnxDAoHmVS~x;Y~5j%x3kIFyW4&pTRO|L_l;J!2vJ4UU(faj3&tex{!D+ zynt}7!83E=6LgW{Hva_Z#L-V<@bb&39ou2z8!bppJU)u?bP?7~siavx+J88n@n*)G zh?kP*qy5h~k#QpbiiGgO&O($#2KDlUwMp_Z4-kwe$pN-R=S2ITu_XgIWjZepSd%3l zqP&(vAH1kiZBBj|IDpIW*4Mo2pg)>^z#=)ydmng9@@XUto?YjWn(7hZN6SQa=qFf) zlD%2NrM=J^9O%;O3g;*EO!|h|Z5AXUm#%sqmbV(=i3odpo~#Okryoi`q^U1N+~b-$ zR*UE^3l^OIkfht;|eJ;-YMNeC~vR&WiH zHEUXOj#z;5&qLco4)EI*{Y;FW^J?;HB&^NHLGljCL5nK;=v63vTiItq5;&f@Fwfgt zW}+X+oRWJr_p0V$MZpU65f?NM@@p~$4ak=d=qX8wV8lII^vuNEBN7ICkvY)!Cu3y- zT39N1u=@TUt2}v&QFdD9F1J&Kb3rfTFv7b02V^7s1j18fJZNV?{@j|} z8kIf=rI+yl;@v8Y){o_;gU+C>S5DjJpcf53 z=tcNe7-Ql=&ki2%4evvEUj~<9HaH*oACgYm@}O?qZ?Z}F6Fh(YP}lMp_5&)_{s6(w;Qc;6beYA`f5@Vat< z)A}ta4TOyjya=D@-tD74@FW|itO_=jCTsH~%+G;m;4{kQk3|lJV=qRo<@w$=dEj=0 zbKzU@8xT$!7$(DPumkM@*&|V0qcOk&@~&!#r2(FpjnaZ`Nj`dv0S3Cj16T;yO4C_gzrS;gmFK^*CZcv`1;&6xsT(Kbkk zmsi=^8M0sEI3NC{Udsn~vZ=Q0?|)I@&qleE%cXuulQIK3n zm`nABFy2SH%7q+o2U?ya!144-*$4G}JyzSYPzG>( z(4&JMmGlVHdeoDx^z%a60a-g}ZI;p&$MfsOpijCWI-4X{D2+joo*ZCbS31cm0q(j% zC~cX}#}(jss@CfWCX0gytNr*vDS@9m@c9?1(mCsWTl zZs??3|-|51w-|2Q?u3vZgA(3zu^m5ZFFBz#meaWSs}E zFJ`|eVc+&r=Xg7aadR;aU{fNycI)(}}cv)4*VF)GbjoCEupZ2?i+0cwm| zqpanCD*GoG>AMb?=EJ9S58F79V={3C5Y8+33zCywThD1bQPS!`trO#I3 zbhJ9u8sIA9`$iy)Qcm~wG+9S6a?6FUrOA2<;iU8=l|FkX(uUxkIHYd`;w}r8IpC^> z5M0`5^kLGPxa!rKZRIU%HgYLvQ5g%#$7U(62xmFMGn21SD5tHVpdm9;HZB z4)?nyZP~+TVM-z?()=?MyR}BkCk1GuP`A+r!12s?;7hz|s9Bj=XDIp3OSI?|@@OWmhwWFzh&P2-1(24)j1b%Uhw zL+N|lMby-8KserNP`!CiT5X^Om*Oq&^mztPIhJxv(aLxZaR)W6A8A^{0NW#}U`L{i zgBI01iLcYNd@F@EZKiWjQ%;xWfR7A5;3N1}_#DFL(x=EU%K)VTuS*IBzPKdqe83UF z-~c_~rOr?@hTp77V;)i{r}1MAjl)e;_|xA)e1*o3S-!lZzK)z>JS}}$k5GH+Y#|3n zZN>Vb8o%Dq3J*vEhe?nl3BIN~Ajl8{g6Jn9JiQ^k!C-0B7ZIhOfHq5qou!>u4zOnw zKWVRF&HXj_ln%R1WsU%(Itb1k2;I=nrR_o-T&lMPOM3v;lbcHIg_JN2W3$wx{xfBm z4R`}-trqNOIt8RlxhiS>%uA8`FO(L$#!J;!Q`2CRa6ILBf744R&JZaFlRrvUrOOAj z4S=O3pG7=L@pI5(1BlWCfWGX7AvX0*U|N~{tcF=KTPxB#_*%pWSag-P=^IKQ6?Gqu zr|w82$)wb&8?my#G4-l^Vj-Z@b7|+0ih6&-|KG$-+DSm>!=3WN5SxlIXzKXXBNBF4 z`u5tj8W5^oFL+~EY68DaYU!r+oEX?FoQOFPbD-aw7H=^-4KrqE%+7u<_M;JpFhp7n z{V)cy>fRARd0KuNEHkDZaFj4o5|pJS05;p;F|%pJ1n4gT1>Z4spi>u9*AM9tKI-L< zGZXI>{Y{S_ZyMFcpw)!qF(Lg*kPj4+o-kPSRf9)g?UyMbz#fgY^ymxyV*15M2v6Ob zx)pqJgoxTP%9je=M1yY9B`=I}J^@TmB*E^G?pHppj23Y9f##}5D;2#Gx!{uaN?L>i zv+ShpNoxMbQsWS}K~sJ~Ngq8Duuh5HS2F4ymCJgh_>yg;Xj{{WhWv0MIwW>$EVboQ zn^B({BI?tqPh(e02v6BVZD4G(ruq`eOO_mk(L?MENta;YH3~g{)Zti^CPF8C^2$

XdY~vvcB?%{@HufLge>CzbOr3yqMs@b5c_-{4B{?P8VExmH3f#+pv6LX>91m!O z7e+h`0S`lxuSwV|J|2wg(E0l|9&D|3l!<(BF`FbcV2Q*CIc8;ycU}>R5n{{|jG3gB z^xuNC)T$6KuPTlsQTj0`2?-(JTXV#DT?*4}rC_5TiQbG{@<`;8(nn(SkX(|edRpQJ z?NM?zcCIVxQAnF0DHz6YbPnh|*&(T8SAGxbmo)VkGk7m>^ zNKZ6XdqtzfIoKBfYf6L_MQwo9ZHU^?NA(~q3hfCTk7|c+QJ#l1^XMlbyx;nM>yc_a zLPYBigsWd0AVZV`>ins^@SrPk*P@Aa*oK4%R^u5XpZ54?TQJNI?< zb$P=_5Kim6O@>+0YNUC9xh^phCag&IxzvYN9HexxMc(JFIMvdy#21nF zB;pAFdpZT`OrPg8gy6n*>@VJy6>|6NmsZ6zGm5E}QEc_~}gbQd60!fGzwgJPdB=uSB;e*$`) zgzNAr%=*j#c5C(LLq##+O~Bxg;v+9DmT(linY=P59PEAkod8;JJfW1)HR z*Km~@PdJ|dD~-R15aIin_<+GR3#vcWyO!x%*r8ZW@=HA-!2&)W>LV(QC(Z_?m+#v<`8Xqt4+fIfy?Ke+H?nGQUb3#2+(Q z94sM9**w9(LDW8j$2|j|Xn$_w0B~^7UvglLdBKSt3tiO@8!0~nr<}eQ7l^Z*w@fkk$zY#EFX@?9*$CNEhxqAH$?3I zsJN&&nTE%0i`xeJH1`2b{3Bl+xQN|=QN(30jB=vqjBSsUwvDZjUJmgUU0NUd%nM<$ zHSD*&m1%4~u;d#&@}bCwBz3}?WQfQO^wa!c>jWrwYJM=IKpOB7IZrucF_q3@FyoFK z8#z|;h*UBMtcsxqfuGNN;~+8)SmIDG_QTg!6$yMXJ0h{cLU9n=9ovo3&PSR9hvc9I zHLMfRw3 z_`)$)2>QXti8a1ygT1}#Bgh2{iv3ve0SpkS8_)%$_YCQKOC4KVWB=wSPVGv z11Huft;bk399Bjy;4E+`JZNFQwD?M!DoZdT6Fw&8zT55{Un4( z9E(5?82lVUByB&^O2h#`_9+J#o~ux-A8J!G&t32#7+ zX+ZvI!Ab+b7d{D`)X-0JG9&_Hl3$P2}rVX$dal!2~Pl~pkUd4u@vG5&R1iu zNlsn>Csqs#I;G!EyJ6fCgmQr6<`y0#xd5gh^myiKeCPaA3E|;;!uMbl{*I(#umQwZ zINA#nwV9z^Ll%NjPV{(Y4FgUIv#5Jl^GxDckdACC0X8iKIe`P?-mqbeYJBgZV1NW-%_8(8JQX2sGRTJ`EujS_8#c*`XoT>=Junl1wKaR zG>yxErAvywof@h)&XpEH@6xJvzXvTm$6(T__KbA37q!ieAL*Y3WA=lUPlDq`QcY_; zLQxt*^UdjW4`;Pzv_Dz3tG!XIK^!^i)0ADiL2(Fvl48&W#0}IGi**X3mH>yBm*F!= zLe~F0LEtR9JIp}{agM#eH*3|JSqE;n(+%7ZxIxoxLw#GW>6ZHFV%$h{OW`wq!#bth zH%$~@K#T)?Pt|{d zfd%tL=;^Xwh2*DHa7wT9G=T*y_+kLcQ<}g?}3ot*dlRa zLy)G35rBc#0)&Q=4`=C`cAS&sKh}L`4_cP4&217l%}8}k;x$+$ z={f-Qmvk-2?i36cF`INPPHC8r6Q%1uTG!9_u4|D53~)YOix9i6F@E*wx?0K4F9N1? zT}@QsEXip{KS}=lm+4w$TT+v*MS@WCEJp2+pq`JUtM%!cU(pcK^-VQNe5@(4S4r9g15@lW>6$OrFrVy7*AZIR zb-n9ax7{{fv)y)GJ0QQ;u19H#q-Uk;QA8Ea(lwpcK)SB`GF>wod3xI}oEy7ehM6N5 z((IJ@3^qomK-IE=^ z^*iWy5LB#odK_TPZ#N(qyRitO#|TDL*F!f~aNduC`8ZL!&e6Ia)w`~(dW{~#CO>+N zy)w-FAU{qeB02_};FBLz6@=tJLHBT${B#Ni$v^7LH2y`sg?R{ht={4m8^r{o!ft`j zuve0N-(1vM72O3m(e35U3v(@6C^$&0B{eaA@b!y$E5+9@PF8fy@qnp*aWd{0p8(%3 z=?>HeqBsIE)LZmuGtovdaz2XUx}BmRJFrT4s-h^~1WZw!iaUlqy*E;P5D=nx%gq$U zPP7c<5-k}8tsex5AQ@(24lrJo5*XN#BJD>k0Zg@&H}?8F_%}4IalO!Tf)1QbI*by1 z`s5!W&XVp{MT=htOi6bu?ieSX+w>{VSt04F_y|eyhOs1n4tdC_u?%wZ{Yah8iH?+p zV}{RTxRRgo7Njt>yt8zQm8e0)o55!Fq>kC+cK{*X&c0b*UIAQiK6&Y^8JoNuahANd zXEnw*JO%>aM>O)>^>p923vqwhId4nkqwvcg;6ZZAG4}N4I>SZJdBSc2pqLh zvSsAfyR;*4-b~N(u9NsjBQ?2+Uas&=->ATkxKvtpN}L93M3~+3n+`E;h>$9%|NNZkjD>kPb1iZ74@gtZ?T4cp1sT7HU5f+An|yDh@5bw_h~4vPnh{1J;RC{ z{^ttjB)ZS2D`pQPS!SBF_`6z2sXMHmx-KbNq#^Yqp<;Ess7tYzF^t6cCfD1Vq^(t6| z@uCjX*h#@6SuXAoMjNbPoJFBWmSJ|(n=YDt3>dtVjV2VYHCxPznl7p9T$Ii>F)%>5v7GrBjff8qk@8zE&}{g9q>1TNCzYt zaVe=FLLLrFumJgj3FKsfXh&(~4~xK{C{yDMQ9s;~Bq0oKDDlKX@$69I!IMh_rQ?PX zI98PdeOUm1Ife@-qt@U=)M>EC#n}D05;c4?_FC-1JK;|nhj6CpE5>P-#s=XW(nj!F#uw#;A zmZQ~im*YOi7ROG-;zSpYs2c|2zJF_y072VW~MfASoa-U~oW5 zKvlr*fX4%#4*2hY<9O!xhk$c|69ab!J{I@{9tI2#S|0RDa3D^8?;D&Rd{6ME;BSV+ zhopq$gmi@*4>=pkLxV%dgiZ}z7rHt0TcNu`_k}(gdMNbyF#oW~u*9(Ru)MJ1u!^uz zVUL789`W6_#J-3pBMwD8AMtX;?;_rccsJs5#AlH_GB`3OGC49Ua(m?WBKJjAMvaNu z8TEG5-=aQ=u8&?4-4T5(W>Czkm_Nkk#umj6j~y9%OYF?phS+7X%VXEXZj5~}c1P^@ zWB14YBrZ436SpDm#rWL#lK9H_(eaaU;%Q_2L-CKqKOTQD{`L5`;xEL16n`zjkr0{C zHz6ZoaKgrftqHplewFZ>gnuMl>*LpFU>|p%VSPsSnbK!XpHqo`iD8NHi7AOWiAxhZ z6Q4^wp7?6wnZ!RQUQGN);2Z%W^sz8m{)?fY=w$NE0e_u0O`?0X`qBx!xp z14-ZRm)x(l-|^&xLbj zC8?FEqf;lR&Q5JiZApD1^-$`u)RU>dPyKW1#ngYKjY^x8HY@F}v|psXl=gbsAJfhc zNFK0Yz_J0$2do+JwE>TjefF}mLJmBtFz=sC@W#IdRN(NO9dU(*gxg&EYVF%mM z++XLO$}{uY^S+7KmRBBhX>~jb`7o{yk_us2R}df zPtFj0O6m^huL^<-VhZLKJYI0H;BdhSSEy@_>szjiu74EP6pkyLRyeotp2AIq-z?l& z_(9>7!hgF5xO3ee_Y8Nvdx^Wly~F)|_kQ-{G{@TZYUWvUA8| zLr&v-nYywGWi!g=mo=Alm8~iJTG_U;-DQuLeNgs6`MC0B<@?M3IW%_Yf}sb7zC5gE z*n`8)4tsCdN5eiJZVnF}9yh%I@a*9Q!-ovNYxu*%PglfOlvb>*c&Xxi<epek>U06N8`kU3qYa(jq)-0*Hzc!+FR&7IVd+plVujBm0>Z`BX^E`X5=47eljX))WT8Ek9vD_=;-*-t)o|tzHju_(K|;!Hu{OtKOOzc(Jznw z-I$ayIb&R7%Er`=xns;v$2>pg5oi*e)?~wzcHiVjEWiaX1sgr;9Kvw_32yxJ~L`&-C&O`4lO_kp>u z%nQLUcHY=|-o3i3pa0hU&lfzh;7Y@|hCK~O8csHxZMf8EG^RDKX?$|wz=f`bb@(k<_=)egC}C0R zqB)DsF8)E&t4qc#S+Hc?lCw*$Ep;qSTUxSo(bBIkeP-$DX4ahD{ABa_Wm(G_mi?k7 zqGeFa_?ERTPqqBHHKMhubxvz{>*m&PwLahaQR}s~w6;lYv)Ve_zSH)zwl~^7zuoWl zS+{@I{^Rx&?eBGjbQE_yjRT7w?mW|#-*sEpeO-@tomuW!UbuY8@{Z--T7G!>3wMma zCPV znY3osnuayY)+}GMX3fSm53V`7=Fe;XzUE(R!`BX2TfBDG+Kp>pTKncb0rw={Q*qDu zds^<Z%*19q4 zR<8TOx)0ZnU%z;L_xgkDuicw>@7#O$ZSdQWwqf{&&J9Q8?}ZKO_v!}xDln>BORE%V z7*(ZV3>6X8@7$%Bun*sTCjw&{({u!`TOS+|VSM;uaUQlLoBRu@A|3%)L#z>Z3XJT)Sx5bMW`@UsFW>wbK)m>FAHp!;ANR%|P zwb-(?(DDL9m;v;|n1NwHhG)hZ@v^#UjDHvdW8h!5e~bmghAjhzZIK#4rqygNO*VTg zc2(sb`yTOPd9l1${N2ci%&f|)UPuWt;AAH&^JTpE?mhS3bIb zClXr_HX8K+U!VQT>-hTYcl~_tx_s9!`gVd-kiPFwjE<)-CH!gS0qYks+o5Lqi|+dd zp8gGdYGW*fg29Bp(-|0DquuKEFAp9sigRnmXT$}C!7@!#d}cT|-h2B;Z@--{8IP;I zfoZgc?aSlqtkt>}%yGIoWMo$$5UBm!BYquiCy2X;yS?D5C&sXRI9`ZybFa*+Zaph( zZM9k+&)#0fX17zV*4CEp$vu%-U);|5(=K_?X8dx$HNYnyK6E;dj&9z}*q%J0Mx%!h zUpVgwvi7gxotrM#d|((xf3lcQ`bM{F7~P`D`GaBmyjht_!BA9VJG!pSX=*wi^}7A> zY@T%;92_*poZW7(*Xv`!mrkek&0>+ZClbk6;@Xv1IG#xOcx#g%7IVi$z0EJmMQZ%oN%$oDB9-crbjXx`}_OdEJg3`#K(UCJ0>fwnArm46Wq9R<@&W$(6b#CjaIEVkbL;?bYN(9 zJ|n&M+G|%dehPWPPt=Z-EVF-n=aS>{;tw!$Vd{i^BcEZGXC)X`%dS{$xAPNhFN^Mr=R7@ z-{mgx*>~30AkWtz&*)T+Mk1+DuQ%!s`dwo%d`iHVH;fA+kCso4Pfo^Dp}mk4yTGaZ zQ)6&>pZFA`Lb$HvoL9-~rTAWBV09Y$j<8sy)7`G$zblo?8Xvmqk>ly*|#p1@5!cio0 zv=`gJBfUub`k`_`X&v6V)mp$^PCxI z)CFloje4zKXKk*rLQO*3>p|H6@1 z@mv)>;d%bxBeFZtm_g7<+pjoy{PM*wuI?nGPeyEqGBB!?q^~@Z7hS(RGtNJ0ktYP# zPw?T%emq%Z=5t2V$|sXu)^7B%P|%VcQI=`byz`}?rn8IJ=r@;5TCR<+;=V87zNph# z9F4{kYdVT{I)S^dTn&nJ&C9V2!}3_Pwhs?sE*^fMe;q|K{*2Pa-{1D~y!Yzw{^wt# z{w4M2yV1i#rzkQDt3PO^QbxH_t&~e@QxGKBVjAwZF^WrJcwPOg@4x%-!w*dJgNF~_ z`~F{@YA7-jQfP(C>pg}HDa+GnB&M?k%Qy8b`_2#lw>Q4J_!FJE^R<8QHPo&87rRO> zhfoR2GDbXJ$ucyJH*l~0`hV~b|0A7y^I!ch|MJgXI8v7-die@jF|VRE;a+PdyFAu( z&TtKn=Y(2JCnImOUhgj$>;ZdwdwFBsB%S`B|J$!G{j&gYmssKI@+`ox2a3Z z+(wPN9i!RmP9~hk7Yb{(DJ=A2ThDNd2~6`{4=Y;pP9ZQ0Sd72IVQUjViXe@-ZIUufP-gd^qs=t!Qt^JbnQlf9yd#+e7-)WC0`^G zkE>jJZWASrmKn(^=8g`U6I(2i+Vbln*|7M)Mk1J9dZG|(j7)c7K5b1{Wr;03cixj2 z`mbVcq&1&WN3S&EC1Wl245Qgxt;Gn|Vsp6`AAIaQNH_YZfpR|nzR2l%w5p~23D#pE z6is;b2kQp@!t0Un`Th`XzeRJ{C6Ix_(KLsxL7psH9?Al{%|xH^xBPPv)O1g__+fc zh%0L6$%S@HqiU66uCvfKuU@~E&;^@KP_&_iXFh!B@s!KJPuw>0b@y&U8atgy% zG<9Xv+bpeJ?{oe(a!ynTdY%W=s`73Yfh_vJh2BHz4NRM09TCv=pjoMRQQQcllC$ryfc7f4gJI?Ym z)6irt>!$_H7y}HW4i1Wj7Dk^Vno&H^uQV$C(TuWsP3%=&i{ey|$LDjfj+i*zfBeyh z)seV)_1aAzo+r!~S)So}>~UHD+_HFPv4~K2?*@3Q`p!G=RIx_ZWAHh&7Dr1^A-8)r zn~X-oe!Ed@#^h$LJW)2%n;r(^AaH^x2#Rx(fxAp~I-R-fCu0#R7F(9$al18D^?8pP zkz}eVpRvMFXRD^By)?!f@N7J`rwlmM%x_r zOjx4Vru`Bu(U35NQps1T#v|AQ?U5MpJ${&L&prNtgEz!@lo+U)D}pRXiG8Zf8|BB3 z3hn+BPqj=l4+$SN#}02Q9d*j=XRulR33~TW(7V?zpk7Xbdf8mz&(PuL;a{I#2(LW{ z4Eqdnc`hvWw;-!;Kvv&?tTx;}uO_hl$&-gi@MT&Z!|V(ge=wxX+4)s*)^5TPU_1^! zD_o~KEz`2*SqE&q#@Gg=xep5}?AhHoK0GRQCoa+IG9HgdnRRJ#-aS5eQto$$`K~1? zDjw1s&9cJVC5B;eFmTp!tdbg*!AKo3)zc!A#ULKwYyMC!6Crb#l@&VOyH*6Fp6jYx#V! zNbpu$G@i>h#nX3iQ>EWY=>RjKT_UiZ7rg#7hZJ$1E>NGCiL)8?|CSmun5rUUFeJJJx9K z`ki5hbGZCIzf+Opv5gydUU_*la@K{jnTWbeYk>B0i`G^m@w-T*>MGi_jW)Sy&FzFc zSvIGv$Lnu3nr&0h&{H#GGpq1lt7e8atp_`a{MbNYW~+rB^!=RN9wDr@!^|I}qUlWc^41G2bs(VBI*$dH%X6V6SBym7&v)h}4E z3|uixgBsLG&CJ-JGGp~C`Gfs~qhWCSx}O@=^Z8snk@ArU34Tb}+188~M3$21i^eL0 zut~+DzcK!60 zSMT2c!u`9iW~fER_T%^7edpbG-(IzT1CrfVc~?41;I zc-5{WcigznFf1#$eIIzEP{dR(WBZ+7$En%B{yQ0Jy~|vl&zG-!I%!7(8lTjrz9%?z^vtV$X#{@3Vq z>2~|$v2f7WZ2FwGjSc^Zb9({-CpW3pO2w9*VHXxp>+Gl-kL&JS&LP3Tt__xktXCd1 z4)o1ZF<;2#3#C%&^2eU`g^QRL>cyz87H zaV|IYW90THkkUVZknR!P?T>j{FsqiJ!>5}GYfM}Kc)H{)eDc(|HyfJ6q2`Xl)=I^~ z>By&>D%uJ_r)Dc+!>5@$pGI;dbMC)}1l*v|*f`E?YHklbY487RJYVdhS{=)?^XY8X zYqu;>3&kyyvGEKz(itDqo2m|Xt9Gz|xL@wXk{dy%(C_IP%_dnR#LX3TV`IY!9iQdr z)5!!5q@9yh)odOgA9pNkmW|K=JvkbSr)jFH%2?9qpx>8WxD9bqmSKUnZ%S&b-RRLa zhob{(2Q{BU-yvWPU^LCsRJj*-!dj^ zbXdpEqiEm)ctP6$5@L|3V{AdNsC>RLTBxDW!Cqy^M+hV=jEcEHVQ@M%`%*?f9h4+So4 zNiI(yh%KpJD;3VRB#vxJ^7Hqktx^&GMQ*((9UXk~rj()N2QS)}{(X$he~6KJiH5f* z%aSO$;#aSOY2?$mSt8P!%-Gsd17O>DJ{A0-uurj#0F<>5t|4@jtUM?k9pxuV$knhi zflAA33({kAJfm_sxXgY698$myUt-7g0_NQDLB7!&G`quUcQiT4$GoY;6-OkMPK8x@ zS}7K)gM~r}M7mc7@y3Vc?Dg7d3)ZDH;1g0N;Ia3qo1s|Lr^=uIeA+8b0J!nq^yg34 zG>Ok6ABWnxD(4PTi3q+DsT+6i+`e@aHq`a4$o=~_MDO;s8#j|)k&qFl`hUrd&x9>N zhqWfW3yasqPg)21wq*e!tW@hH%mp+4C;NMk6@XViIX=z(@i_u>7;1{j5BcL)TYUJIP4;x6r~H4yV)Y@yKd85{jq@h;~~2ajz5f zxJ1e0(Te%@nD_d_5n`=Wb%q!!1crq}-|c4DZrAC|Fu7bO>sY{0xnQ>FH}c0Pg>t!Y zlFy$U_RZttXq1|Vqs1bwl)=hM`GpbUKv?v)P*i7^h6}|YCS@28Xv4so$~tT?TP_$e zr?UOmSF+utd4*^mJ5SI09l+t$N^eRz0_}R+at8vcZPsj7q0jgkfg6}?h!JGF-E3pK zFtAUnZ2GI zxO(+!7zj2?aiiLkCr>ISXIshl++O%>cGjm^&BPWX9yK+60AuLy&JsJfjy_GR{~=oa z57FrRoO?69nZA~^zi0qkGtnZz$YeZQ_|_wdc}F5O)`Dk&NS_XA>Zu`R75R)yz@Lxp z{dN@#$NlQ(LoXU+WXB8zm&dEweQIqz+CR_*Vj~H$s|eYwhCTJvuxoDZj2F83?D$`n z$kX_n>I;SJ9NjK!zm666LwPd@#DuO(;R1=OX;2BZ`0{q4zBSx zHNKIO>CpBT!Wlct6|exWe)qc|C(TZ)*=+g!At!G&z&got@ldEc$O6pF@*1pS zkG$)LIhcyw$MzJD1Ows2y~pJ~7u!hb59sbNOPfZ<*4Rh9>co5o?byZF$MtB9Uh*sS z^2`g@BrW38nq|U&V)yp8wlehI-mYXWCe@aizDc}r_Wu31-_FqY@2}o)Zp{&H5`{;e zzxmnskahQqTtnt1_4?~OFz)pWn)TzJc##S9?B__1^uLKGUb)0QJQst6#U@M#t!CF` z^tA`5o@p`=Dj*{S1}IQk=3_WQ<2hr~ub7>7rz0wU#680izbEQFmmq5y-5w=6L&1Qe zKZh!{6tCm{eX8G|vUt5yVyFEJXx_8EIF~;NL`#C$sr-}ZT@t+mj4XRX;j|_}k4+^f z_)XXhb9J$bXy2lY+t*!ePs)O|H2o(kaiHPMdInzXujwzYlMHvp;9C5 zS|_F34YR9;Uv6f+c zHUBTp%jaF&UVZLd3S{L_{`Fi>qjgu%y4`15$I6BY63E3SW;9uTrj*HlBJ_Rcn>~TBX*gwueG-WDbUd{-Edc2b~IwZOE=VByQ2J*Yzwj9MU@HOpt$Q zNzK@78mwED#}5wt;NW+!Rw|m^t!X|0x~kXa24d|Eg+ujPwLPL#htue^5XMI60=9d% zJB2cuSj&HDmdV(LjiaNZI#BFkefh&mrZ>=1!rcQMyw>O(A2*&=ZmcXfx=-mg#N%Vn zQ*FDKRNLPumPI^cvWv!fsy6cngAu6TTCL)AR%)%oL$te%h1==0m+Eh?m077`{j>w@ zd`2x^kK3H;yj(cggEo#Nl7&JfaxWBujt+zD<7NP0DoaM!N;fZ+HD?;1kPiNQ_0_6xirIP9{SgZrKX40-2o z%|8CS!3EcPU? zu}_w4o{t+1>L5$e+2<3wo*9^D&?uLVKD}UN5G_SK5CjQ+;ZI$b;2EEbai%s>+gFkv z+q3dBXXE+Anv90l?0I~y8OO2oMLriUob9O3$n(z9KL~yK%U7glq!|xK&-?=M=}Eri zbBSR*>p5;yv9n&-h$Hccp#WD-Ypwb=#+Fk##t|eur(|?KEoO>^{xs z0#gLsN~}e2E35l@+U(2fX`2gVJ+Aqc;BysD>LZ>20=~<9F2dJeugBPDTaP;!_Z^J; zS0*!R4jL@z27pz0Nd&ga&<1cx!GGhXq$)O1;90P~tlk2#j*znG=Z^RHDy@yJtI&si z_&4liAi|SdLAeqK(8p{-<6~2&v#}MFdvw$^We+h`#X;i;9{2KRn25i`koPmx%$hG) zj>B)K0IqE|Zj^aMP{3TGXH#n?(_J_(7Uzg3qj4{RaKWJ>5H344!D(8g{0O66qA6oWXmGpEVd1Rv*BPc0MIs(Os%J%jexX%9b4EW z%Z&zr9oo=gK?$_FZK-)0;`2t8t*y8)WE+q$% z%ICK)x<<2(i1sIA$=B+&jZ{3I-n^Si)$3QUZQf08#Nr?ur&7s;V}a>++@9N0SGI3N z^&NT$N8(dq$^@k(`HEY9ujWdn0v<1--fo~%fkfOD@wZxI-s26zzq3egAU06WM6^sK zs1&2KL%?TQpx$*;A#^;SVm2Yr!~K)@^SN@lOyHd&{yjc=Kc5eWg%Oxkj{!cBO$$CP z1=a+pIGi94M$;@k>F%m$$P?MO?ydGMQblNKhd7-z?Bh*vqB|YK>+{>0QN31Kj&a?;`yz5ZDff6R9hksg1pZ$Atk4iDMKAa0yVjVdh9i;@ZbL*RG!pJNkXJ z?e`(S8x!Oe9v?La4re^Ekw{WYQpA)3n>?9z+n8CgkZ+Cr3v)PLc=QZVH%AxP-9dFQ z(@-MCB{I5;;Wz?6raf3dMyppbMh|W*T8~a*0gV?d7+-@4+X4@^Go8A(qsPUjX@V9G z&BeI4HaDHLo^ff2H~<^i@~bc*Vekwr-PNyF^Dv|LYC}d+9GXMAX#Jn=C?D77e~KRd zQ}pnca+Pl1n$gp-Fly9tPaeMe;lXF>^mEbtPb-LDG7M{uu>+;fCaMHm_Q;<}HqQn6 zKbyQR@0#bHgF-w9{NJe#ATh~5fy{R9S`Gl99*+ZQ6DF)sV>F+Q2~o5^G$;Lu8ji(% ztiVu8BnAX&)~pxGl=n36;OIcIFMkoxNid@7N?#bct?P*r$O+)-ZdHp0VF9^pU%K z`N^!`H2u#!-rwrG?bLH#!8U!qCK3WJzlY?zkXl-PkJM^C?>!Hm?tx#J#jZZ@6+46p z_`El)bvM8_4Oul_SrA4b5hKny%sfwI&s~5|H)88K%j|h=dhTj^UK=m386p?LsoQbc z+%9*mTJBgjXffbs{VCzpO~AHkB3#*=WSEJy*mcuJrGfzJRlhXJA0LLJ5x=}9)g|r{ zDmh4X;+k3e^Xl=-?fm(!d=b4PEXK%%uF;%bRVwvH4P@>9l(M;X>b#Rn7sN&qO3>LX zHPs{KQYwALJ~;T!|Mh-LOhxvUd1k}ZsGhizFHZOHdOH~^l5Ba5EJ zK$o~(W37=$Z`SxobA@|cyoL|H!0-6b z6G6`iqmGmsXb?u-MMpy4y5KtW%S%}W#BuKEFdT_q@EuMGb*J6}z2ws+NEBNJpYEc+ zZ~=jPy7i>jPhUCb(>W_h@g<&6u#*FA?$IH_TZfOpr#p&+57 zfBm%;rn%11ghkxJ&6CM|?rF^l>0z-@jkXhlXaEOp;D=^Yw<)qO z;**u#_)!_ypc_ZG!6dM)3!;NAi#@jeJ2z(JH-i#31>Wz8{ zgf<{e@u<(^i^g{Yh@m7%%@_NQR-rLKaJbLoo{k=zz_l}MakcQ)8dyjwi&=vCckwV8WHzflGs<>LA z>HGKZWBwreaJo`{5lypRiOBr<6su1A*iSt@OV2QW)wYGwI-|p84~pSK0t??Dn17@9 z-7;$K{Av_d2D6QHXXepaw)4;1BeK#-t zBP~C}>N-cfPabG;78DX~0YcDXAxRQ=c~E&%9aI(6TtEn=H{)EM!6*{}R)=aL_~RV> zpoIkUUu8!m4XIH_9Tj1SANqLX@^Lf+TbjcfLlzHACF#FdO8N82Q^vE)3C9^9fjJ3} zmBT-@ljTK$wx;7TE7@nG(V)MhnVP~Ynu!!;q^oGA8KFn*cC9|5We;NAD2y^0_j{u$ zC5S+3^$5>9edY{tu$k#|ffCe4F z9dEq_9pO+Ufz608;(~%%EEb>9bmwhQ(S~JSGsJ6^q0o>UzsrMJVWn^3m$%-!fUrM3 z*EHOT!;vVEM?#_Hy}RUI!7zUK!{yz(=Xbw?&%Q!>8gL7eGZ2gyk8@>N?)MKx_oCaW z<*?+SMGy0}41IjOD;yuE(={*|^uUZKgKoA|pTd6s`(OCN&HM-9>(_7G#0_`vzH;;0 z^$dOY?yhk6?z`{ac=_JvUf9p`*?rCjzKvGhMyq@-ud`%0w027gx2i3_Vzf0RqC*+6 zAode-znP)93~jfQ?_J;gqlstKF>pp!joDzTtl~6{X#^i1XW)7-zoGdsugLdGeuQm9 zYO1~@JHyFQXTWRWNFwa;sFDV*M=YpOZP=17yF6>PTE=K0MFvg4vh*Xdy@@4HKRM#O6LVGY7Pj_FE7rntNuO!`zTDds14JoHMZZ)bV8@P47 z)u-$(;yZ<1qUZ?1OCUyafJg2U*&Nj==4&(>buD@)fR5{=@mYnDpv{ryf)&D@YAa&F zrO^|Y$J6-mXNTo_qS^G`yqXC5kM-==7Jst%PniGauinwWO8pD=KmG0dcXnO5oED2m ze4b)FZX6#MYRw1&qu>0a-~E->bnc)3SO48#y!lBP5$CgYy@t;1j1*dvLTjA-Lg8iB z+35shtvxW&?EouaN51=KI=8v`XK(i&I4xG24H}(j^!R5<5S<+46ELNs6`gzQt;(3W z5aT2>{qkEs?#kI}a4wtBtN?2us{soY@AD_|>!r2yanD=#^!zg}t4GC$lum(S!DYenZV2l&`9UUvHxhLkaw8j@|TEzwy;?{My}EsQ&D%uX8~$V$q@ znm^#@)e?b6EC%0Lxe!4POo zJ2uSpSj7Y@Te}y9|MUc$*$}6ug_4ayC zMM5YdN;y}cU4;X>OCw(!-@D3t?-dHkYfs)&{eBgfUwbWs@71b5yH~bX3WCyWdjfq( zk1m%p_WK{4Eq(AnmV$Ee10D zE-G1k>lDQxClI^@slvstlHha&QHMtu3@mqUUHdu%nL_wc5BcoS5SS3B50d)G&+ zcbx#zglL}oz5wcAQ0c9+H{Mvi5!n*-lP8&vdFy)j4QX1lfoxZFWn`5H9LfPC)Op8) zJYd+<=wlzJ3M`9j8I;KbCw?PTE<4=}1$!5uypK1YJbLn|GPQR)m_%6{F&7-r{StJv zleW)L%k7O*iIOY@l2fy@VKI`+8SZr_L*O`_k>&Eyy@AEaEt3Wf$R*89=q98QAMV2w zg5?B8Gq@!2{mLjK;d0BeM0osq3Kka2Gc5W)mj#D?+OP(jH*Tan^0v_`myV80)g&GS zP%)q&2~E?OPF&4$2M3CT3ShV#PqaF)oL1wo_EBPzRIfQ*z3u=8NvG2@UEWr`WlTxa zxG|}Hf(?N7vkv7G8y+~qVrLT4tcFAG48rCEn*@pL8IzB1Y2~{8z z%Gg37#LMGa)^3_e;7cbHWE(SIxWa#JB{@`IQXQ(xsiJBfq@UXK2-8}%(CXKaK{=C? zDL7_NSFY1EJf2`M9?!DY%xU$y)EL=V$Hz4wK>kELcU&9U!qJq&2EZ5<>?*~LjlhKp zcBjRcuqWmR;c&6IWJF)AVt3X$)+Q@T{sbw}k?2Oi6-gqBViog9eX_B$(GXRzhBKW+ zTQq|HHcXGF)w+=H$M+2@L9I!G|R)M@N;0H;7gQy^YF|o@PZwmfgS&5#V#E z5_r0HQR2m6{;+t`m^(MOH?D;7oGCUsoweH)F6%}<&y-3gZ4KW9e*MJPuq9Ws@+>!V|@{k={bQMi9%TMz23!?2_zwrxP>nq&ab}-uugt$~PdD4S^|Hu5jmOgib^UtW!RtHrCC_0!GMV!cj%x|GJV2}aAUGlD1b0Hpq$ZYG8vM^1vDDUuRDXSO0hjcrYaI0WJk9@$p!{phuwyf z23I|mis$NGpRe)wCqMbgW6aR!`(3|_mi{hUx=nK$tObEGM#8KCL#0@(89mhM>K2qad@Ws;oDT9((I6=97;y)079vmTCu_Cwp-7NsVWNr|%B2l~#{K4^`(E){Zt zoe_`4l5tcii7y$VB=Pg)pbGXi7>)rn4&L!@;)g%{VF^#7i3dEJpfb|p&YcK9DWV63 z2_IQUi-~lXv%rZq4H1Q(gPGPFjM&1A4TYn2TeHI1q? zGd`aWL^f7-BO^rL$md%#pU;n!8at{;a--CsmODNwc4CQ6$ z85g#8uM9T61rK*VKn<4CYIc4T?fQ2h%UiZ4BZ@p#V$(&ITl{`k;%~)4R9mMd#q? zxC_oX7`DcIZgU0qKROz4phU%E4!G<-f562HbEAIs#)e(bx@igYRaIezBrA6>U#)f~ zy4xDGn@yv~s-#lV7I>L%;Z*CKdxqz6urq63)8cW9ZkAyfo%*)#yd%?_xA5zkHNT!s zJ6Bm5CZfyX^n=(Ri6`8G-$_qU>DHJ|xB&8*T>KQt1VuzIIG`gGmlY>{mi0z4sfYr@ zrR8jrFq_onY}%%gHCzBqHjnUj4s6-#=%_+#CAJr!k*h=Xj__7PsCY#JK zi{=<7s)Tv9i^9sxayAt(n~E1^)8*B>zH+`N5UctcMi$<&t7U4=PFwc08*RU2bet{_ zarKpJyh{_xuZ*CpkuxGnx2M|sbuF$n`l{ONsZo5;vNHJI1;sOi@72CUqGW%J=igbd zes8&kqC7Tt1dIWf%sJf2R64BLZT)W5A5<2!FgH;_68hh4p>!N2>UU-YUuuB#_$1#k z=P1;G0v{+77Y1;5B?PQUKur&pqInvr&z1qSbvD!Q%*RG2m3Ff#(?jH;$In=Cj3Q>W z!JJWI(OPRjiJTI^a+Pqpc_I-HX|Mv~PQ=-lbH;6-!#PKMoE|h!$kal%g<hPX*fOWE4iPn=gYm&}xwZDN_x=x^+uHi}{`i4&$~mWl zx&e>(qtulf2^Ezxh=(0GBp&vo?pito#r-@+K7uN$L*hbAhNwE^zw^q?%}B83?GNXi z4cM3Mr;iJT!sB;zQjqR{r2ZN6_rH2O!@Tm#sHOJbqaI!4@K6;5j#-RGy;Q2xGm!S# zbhKua=<+ymw+&bxwtU^Lv5m)Xzh#;~`0?>fJC)vHgmaEMHF=wScaP6x)>E?BB{~n!5ehio8_>jB-3xu2hA*t&E`nI zJF22OaVV+;xdt-E1`_O^R;z(HK%+qk4xu)4`+Q!P8}5S7>jV>P>}MLf^Y;7imnVGi zHX-*<%J08VXIPe>pe!SgLSY|$^igdAvthymlSADW_S&^;{?#6|?&rDnN_5u}5eP@) zn3Zm~tGbMUy941aiYyT`9j2fS!L)vbe7)|;9p*ay=1IPgOd?MTS^<${e6Wl^Rt+=D zJFtt#mNd$5YG@J^&< zOVswfb9As*>c@ff;BOI^w1fdt9u^6Pr*;~N%#+GoK8w=iM~_U@Tf_4|Fy_8MNVH&D zp-SiO={sPV_m@VO?rY?S)b>lOx4gW1%N2b1%{MbIdK1BY$(w@2^S8#ZM-Y9gj(A@n zXkV-xe;$Cq7#YtZz;k0NV1s!h2)7@p#3;dpjU^xsk+0I6t1GV`29guj@e*@k%Wpc3By5sHl_eWH?r&VZ;^LCR@Z<E=m}C{PoJ9N28`#P_ky& zDB$oD_yv0|@%a6I6eT)oT9yPvT8~#MPVan@zN$J=9nGmJ@?@www;l6(rBO5CKV_j) zGXhq-$7xL$Svv})!QxY=ZWcrQ!Zl2In@fv#&HY#aY!#p*N})3+)jf z3U+!=XgAAu&;wu??P}VnXCbxQU*su)&{ zg&kQ~sQsK_XHxj26p7Jf6o6zFHw_COFWyon6$BGC7rX=P%tf5e& zW%I;cOr=yHupMf3@yr7S*5}Cyrn!UZBa8)YNzQ;^-O=dDf%jH4zwe7}U%!daAkUP` zt%kX)oSaZBi;TfSYsj8zM1std$m-QbiAS}VU>M^7#-r0?$Jq^)Qg4L)v~zfbtTU2U zFys{Y|I0KW>NM9Z!!C1%MKWHeiD=w>M$&_Ju1EoPGvLBA2{EDc|%aW2~t zm#>^z+2jC}y#S8f;BA7e>*B$|=WlPsH(t80+sUye_x9S|UaxQQ_|u&`hd=m>@BGF0 zpFOYU++Ml-p37JM9%cjK>ZYwB&Mkbp+Un&R!7S>`dq2KNZ915$99}3+x+b$fT`ZW* zMWeaQvl%AOt(O3u=Z;#~@+q|;5)1snA4vH9OWgM|!uP)S1FiYRFMa7V^!Bo>FJB?3?y|zC%VmcJ zTtiUL?rJaNdj3A9E24eat*r@i1rW*0aLZsGA>z$m54Bo#7||Yt?IwNuGIG}phQmRx z8%P|(h)RYf!8+vSk$4?`fi7Kn!{p5Of1AkS>crvlAVDjc2+H)ZmM`q@AD1en@q(J- zY!*8~@g~@O&FSurbZ}5Eo*WziwarSb1eW=6p@fi|qf|P{sUfI@keWLwmv?vks20c$ zs@0>T>W~rxiNv*Q2`9KD8HYze3gmGE*m>m2^@Ilx#i1wUYdD@|rP+9R`rB(6$DjV` z@4YrG0Cz16U;Dj3{nM4y&v(l-I5<>!&89oQPI3q-bA6pdKZz$tZSOvyOnfhO<0o8A z<8#KP&&F}((faE7`DdA0u2kRIVXz+H7#hA7wu8<^s`SrCyk zU6&6U{Wd5%?SA9n4R@h%9Fq}$M}a^I=w~j46EAf6)i3|jmtKAK)i3?hYoA|50)Ge3 zBzXhiOxXcJ1GNgc%t_an&&c^93sSTB=&*n!8ZXibmYDg84un`|k%DzK!^h*|pwfLq zv$|M8`{Z-(a5(Jd^lOFu@&1#8!;?azs|jE6MpNl@3TtiL$Zw?MZi(s?^Cze~Z1W~` zNwzHrXU`n(WyE*ibt{V7qjFgn7F+Nd49;sol>zs*VxST}!O{O1Z6*ws+Y6J?9Cj2@ zp%5~W+3|v@HA3DXC||APqr*Ci@bnF!$qi#-(*m`MrQar`@`(Y~;~0P(tQ;2FK@Jh9 zE{sON<$!B5H=6{_LpcFB9JlFV6M*MzG3%Y=PAUUy$lBFC6eJzY72s(088hwMmgRoM zQMnxQfItAK2U$qga0N=@wUI{0f>q=6)fzxvap3dSMkHcYN zlP&8>y)ZYG6_!d3wQMwYri(9PNvveJU$?Jt#!fuGfs&<&1D{P$)mH z8nN_NGU5gu`~?}|Pg|sC{28~`U2la|P!%mi5}e@f=!rHbbmm^~qYn|^*W6ygQkX6n z5*w~X_~z$cehFSpW7sO{H;_Gua2C6~c|NZqDelN%p_71~jO7RjA1`m&=kw9&vrl)g z%l*2Jel78(=ECEQbsJvP1Q<^H)mo!Y3huI{4{%NliD}vriN||zm3BRJuUt6{Z`5*C z!)y0$Y^I{1wxTwh7u;6*r$0sQ%^xSOL$s^WThD(K@bth_N6~fo>PD-Lnr|d~aE8&0 z#XZuZfl>ru4lzLVb(R!&yOcfXU1kraOQh;K0}??3q3=ipIc6kxP(eYf{1F&2BzF)_ z6f+niDnG^P1JB7GM6Mf3yq(JAtB}CgFtUkNeK&aDA>bV-5msuC9K&*>?(urObJ3|Y zH~<6_ETa;BgBLNG3mR#nCssAy!s{Aun0Z3*`n}t?jJ8o*42n7{lC$}oXHbVTbwME^ z3QRQqx>mu~=?MrXve|0Ak!dXRW{C%a1#p^h87D`RoVH2X+=Lm3=nXlWB&(r3?kSUh zmC>$m6;Dp8<~-oEu|>x3b7|zWL_={(~QU=YRSi z{v3>ktcFDQQzm~S_vV{#K3X$Z&a7?qG^6L4JRyPvp5_UuB#DR26_O*goGK)DC-ZcU z(B+v)@;XShpwWPsJeE{*U;qLIMW#)=y&kYNhI*Eo>rAg$%!2L#ZZp8c9yOA% z4)hp$gJU2=;h25cVMGT`I(2%z0j^%jpVS8utZtUiuzc|5O_XRTq56<`^JaiA9padT z)!ew;sx7pN@K?bf#x61W2>u@UZ-V55|J;jS{T(1Sun1ryVr-{ zU$ok+TXA^(p@>HWBe)5@rZA-5T&~vaGpgO?Liu2p%TQxHTvqD+8TG*j$MuB+!BWSf ze)hxhqenc`uPbtvwTYtbk|l2^7JGPYL9b;B>v?5UU%ql~QJ=?$&##?Z-kO#@y{+>{p52`S6V!^`p?@#DqxWLiIiN1&jf7VNCb}AvD4Wo6f5;+qSHLu zKkf@ar?4^YA7!{Zx3|+NoqOl~NBajEmYDqyVvMxqg)z{A&TqD_77A`uxp63uqZ{dH zEVYe$iMdZ_xOd)u=c9d{yL10W5akny`M;7K;R5_Y68jQzyHO4lXDiS%oJ5A>|4=~? zbYT$VCkVpV4OYZoMRtKxGjS#ZIbf6q*BIZxffiJ;*vW8C0;p$HqiBayIE5ZS=9FEJ zMrcCb=kR_0wfuz`m5~VwOV(|8kI8b+LYQ6L2}6%tU|Jw1PJQlkLEfso z{dUFTgC;LePqptl$48bqH4t!a){8JbcDTL0c(T8&nIxO@&S7IP*}O>-D7cqj{(pXq zXw+E-#jEK5@>D5qF1S)#SG??~TyiKjlz$>$y}?iBoW27)_HZH)EZB!IHIo$pdEq<( zq-TU{#<=*ihwtb6!bZAWst+l^eqQ9Q|4I-Qe$-bswSMOaasB;91Ig$DQiuAL!C4KQ zujB1YdlsqRh&flg7VCuB>EL7jD_(81m|BBot5!wf!;FiZCRoFcQ886N=u$ZPAR}(G z^SPj!qv>t}xLT`q{Z>q!6+t>^Qv>V$Mlax+1LjPCJ`R)(&PYb}-MV$lMTtGEZ}#EC zjQAK85)}#6U017aEl(vB@oRp-@4h7;7YPF=iv@Ph*Oi0E`R1c|G zp`(MCqrrjqj}1bEm$;ZC!<976;AETPQcVu^1z{q~7fxB>llyC=&GJL<*M;IrE1o z6f4G($e0JTX?NK}k&M_iOmg7t4E+3Cx4erwHi`Ped+T)9_Mv4yv`h>n%SQ}vTBPh( zBGKz*sEu9mC*RkcA0B8}OFQgaZ~gFx8S2NY9{nTS_m6Pj&gVE316ABgBE5OV!;`Xh zwPCy0hvz%9M3lZ#Eyk{aCN5Ga7dT(cM8_zfJ3%+^S)<`_GNf>bAO^rI(2L=yi_-Cg z;e|?(ve#u-g;SyvC1{Gvhr=~02<&|hJ7%#_n6*z30YMlp7;;M+33&28Q5eEuEw+A$ zDYjoSDcO$%g^e$1VJvFYy}3sU%5yT?tscDh-g}3&9w%dnM6ixtdTGPWkGoXAXLN?( z#0*h1piB40yes+AYCk<2K~QxzBW@U0Zk+Q(PoV<4Bda|Z{h^o}fsb0HukRpW?1Rp- zi#)2FxM`MmL}EDnW@;nkB@aLC=o`mI3!YQC22ia%p+zUmZdFN( zW4N_E$aiTKuJ^ffWat0;SBU*a;#)gmkpv@PZxV4Xv$yOjj>8&{`lwM4l3!P~fdi`I zNF|J7S0$%}xzuhOUSfDUGzWm7IKYhx$6AXAau;pA@G&wyuDJ^tP;x1Gdt@ql3KxHN|mG z7LJp04*M2|7C0S3(g}VJXuUWoY=DEV2G~!sw)w0#9HGG5>5h0VH%<`REu4R%Io%@q zC_5yabZy~0Fw~gH1W_fl2N0~cRA^QpuvjD&#M`X-?oQZ3rWA-c3!)of9JY*}B0Js4 zVpB2cf>^Jpdh3yJi~=}XjRsO+rw9oM?5YLpc#C3pI8_bBQ)DpAu#iw*6RR3$E(~`S z9G^aH@lLJZvqP;(D3S!m*=aAnhS4W+?gYb0_8#bB9B5epp2lOypGy{c(w`E#yad7;7YvG+6ktOxb`$eay92Ksp{!na%-9fK>&b$IFgzy;fV5bB z7ojImPoR_j6i+U<a zb!Hq^zRb?Z27^hbC^V9uk*bH(C6=do5v3l{eIST-@YnT((XRK6;le^tkff!;PQo#^ zGt~SW%@|QqV46$FK0+g>SO-}b*5YE01`6{5gg%%H>oy6Hx_~yZvS5z&H`vD9%jhdq z`4928Z{lyFrb0(4SlXGCW|uIXrq;+{7~7&Vr$ijfhiM{<(qv>44}bfYexmP)z42Tm zxw#zJ30awwSc`<)u_Oo=MFK>PQ)W%`Teqx5u;1-iygd-IMzlzrSOuUCxL)g<1CQ6& ztclXHIj%)8SSjW5HQNF|k?LeJIOMWCou0m*#M=ok{KmyMQpk_Q!UHA`<`0&SC zro(|bm@ZqBJ!`-NOm5R$@Dk)jNU=5h)nn*yI`=nJTv1zQn1jIQ^GitKW0 zm|V6~sWImbdtfK058C-Td9Z7YZdX?2GKATSwKaL-KgRNEr0n8_g=!F|eupN?=YKQW| z7fX}>XgdS~(Lr#?jkfGIAo_?$FOb2%ph)5qO+(p{Zw~)BdZ7LZPKP?`bnQMY^IY8+ zQ6n6zDQR)PCN0N(9bs{KwOX5G^UFA{&>9Yh(;2W-I7vJY$_x*`LB|>Gp%8IoI^ztF zj3 z#f47Ca~V1p&q%yX5FD>2u`6GiRJ|hgV?3b*BC|~q0EkNwMt!7G4#xV9J1t3RN|IoX z!n$2hI5kSrJ*7t8lMIglrgQ*E!w0tu(A(}bh$vLO-Wg#3T`WdD5YSdO@UP%4M8j>w zqtUIq_wL`^yq1o8p&le^Z2I7Bg?wI;x^G%jXbJqwX%WI!=S5AiyS&+pJ=`djg7nG3 z zw3H$cq@{DH*+lZ4$eLJVHM>B$b`kqIwJ@T|Q>Fo`2w;Vf{|Rb1$fJ7qLL(=ecU*dM#7v)jfW(erw{%o6oa`JM+5-9AvL28}uArNvNdQvx_9~?&9I$cWql3)G z!AFHLH3QaK?z(U$s36dsE9f*OoM`2gWf%U@?`g)v$JNnvHm*MUz?3T}(z3u#fMVou zz1&!df;+J}v)#H95V0VGTi5&@7#JB2HsQFh2l{sKa2Dhg?cuoLNya^}I&7ZQcE-7# z^4K_D^u&{%#uynP(~M(cH0=Ek9@lYL=CuBJjdPMY`74k@LVy{~JLtKn4!g28g*81f z=Jc>9sRN@eI7djFlI6ibQLs&|2#Sg_7%Ua9j8kGU@`--A(FG`EjOt2o+^~nd4u{95 ziT`hRZyFp)dY%VnR%TXKW!7C;H&D16K%X<+Gd%}~(;O~IQIf+Ra&?7d%VAp)Ys(H< zvGVT5dc(UYB<ZCDVsCEL%UkTA7mdW7i6nNup=pC>^SF&OTsajVeBW<#$(7KIu@eu!^C-CtzFSEkMPsl_8zTsw%q7N+ z*O74rCZ8p65Dz{j95Y2k7Tg?4TKR1(WA4-{DCLu{5a_W~gC34V&hZg1s2sOfA0iE; zfb^rSM1`q!oya_z08tt~gs+Y1K(~75TX*tI&voJTYyHt|tTIqOSgXwrP=Fnv=z1;( zBK{6kl839?L%j8_h%p-0*zbgl0o@K4>0(q~NX15Y3XPMmIV2YZwzn|PiZIn1RC+kRbd{U8Q z{|pEbqY^egcduu5fD{3+Rq`wJz-71SK?LE>db&z2f;jN`PkiDNz2iN0N?vamq*D$S zbuRqCHI6FIAnGPJa@le|Pre)kkUDnCzd?sm1-F&Of?UtJs3tB7c*-txKbOOefU>RH zervYMpgzE*VYiL1Y8Cmy!YNPGktix zfK&nrgdfoF*Vd+{u8YYc*d~&w=)1tZ_S$>z4SaErkN?L%v7rw(^nv3#9Zuyu`MgKv zGF6N~4rqF#Ch`-`gAK^590D1N%4s~W9XW5im{*H!?|7aVCLpz^MERAkcGj4<0}d`AT2ve^JLg-s{IdG5mUR z*d5&4D~u|Ubh@mn4D3qNlDuMCcfIJiQM}okhSG#60{ySAEW`5@pM5*Ci<(1;JaL=A zOWS>WL0aLiIf&<*th%V8S)0e*}IcNiASclBs{DOa!td8xfEnXk(y4Rlh3NQ`NcAGmmaQkQplu|Bo@_ z|3T|UW{u$W_^|lk5E!7DB_Wr}z(0sT={XLLRd5Z3o)?WieFTa$lJwCe(rD60lt?4- z&}bZD1dX%wGiBpU{cPDdTW>)&S~%1S$=HKyTa^9Ue&~wLJgXA;8;dpob(Z&Cg6sX? zsdoRJy2R17`)luT1}Xur<3^G)CLHzly14d0S67x%+p1LF-%aep^~D3u z?MBoTngRmcsZCV@xg1GALNL|oMhZ$Oq`(Rmsp=FcOTaKj+Tc&4PdfkW9w7UcgPqM` zv;Lm(CHc;sVp0fcxYr9{fztPlfyFtC zJD$Zz2Lo1U$_dtaoi_?4W$1jpjs8CJmcIM^dLW+3Ja|AeB_36jxw-z_)8~M`yS$~h z{pTdC5J_@xPjDi)k`*aquhfNt7oc#1rFMv)zgTt6~SM1 z>(=R7AslqNR)}Z>_PN<|!V@e)tF5Q(VR|VJCn~a9{pEz!uM{)RgMIKc6%o`ax{Amg z+9!wFT^Dg&dg*sz#$8aI6VsXA1NFKo)l&}FI-zf_UkA}Ad-zjB62#<4W6jtRo)oJ# zjycNAPc2=)4wHPTpT9QjBVkk}qmPfDOEXT_#Nqq|QyTdCOg+q{HsWCL&9Nxg2)~HF zXOBrq(0NG$&>qC|1`mN`W{Ap7W6V;hR>O>=ow`$jH$daHg`G;AEeLgF}J|h z1{B@Y6lC7)SpiG1_L{|Xy3#^|zLRfN(iq`u$9C#xk;B0;bk7tur)Ofs`U!7GEVf?n zEZ?<8Ozo~a{h)2CZg+V$i*W>6vl>#P_G04R9$6Wo(2{rk+bzCyqb)So!!HVL^F_YZ z0Su0mEU>}=o+RQX;G%%y)qnCY;TjZ~k2oA)gASOw+&ItJ9K}5BwM5s{Tx}za1oUDu zDa)mj-{0xPteuYEUn?{^|E?I0hPc{uW~l92^Z$GhA54(G=My z9LU1C4t$Ag@pk>;`m?wUyqWDyOcG#q;&2%~$$Mb>+rRujyuthi^Ur2K^NF#!&%Tz; zj#>70HmmmCoXtNzJ{EDyu68At%fkTRL9+?vP*G;jTzm6PCHnI}AAJVD0*&wfOrR93 z{!I8SeD*JKU)R{s)Y%E3#>wK2f*VcIEp8d=JK{jFxLe7U&HNr&+>Lr^cVj0F&o6?? z;7=eQa4Ku*aJ_PY>y|Y2T2yug+&uOJq@uw7-bvRxy8T_N>GpTK4w~nZS448FpxfWl z=+wDM-Tpp19%vZBkW;*Ae1%PS9V-bm&gqRQ*GdCb*r)lam|dBZlZ zH?>&gh;9DhPGLePOzibG96mVWPhO9H8PcSPz-6nQg~!^|lbp38t=HvRBVPtwN^(>R zIg1++;>WGoryBDonY)+EiA)K;nUb-H3q7a8A}!Ha#F>-K;nUW0)OhZ%=O>uU{aN!b zaEJe##>??L3HTlm^>Q9O6nsmzQWhyu3qek!rO$?g?(TX}b2 zE03az06=XHan3N7PxXle2Lc)x(?Dnw&|WvD&3$7!rHYDSO6#NKi;&nCA+g!x66-KW zB{sBMpDK{PlCp#i{K?OLX!xFbKZcLzh-Lf_(BprA9>0Xpq_N|X_kXbCwYyLCON}4i z{SNPn2xUCkU-go{E4DZr?TUDx0V{^huXxN)^<#}6-YpOBqRg}TxBf2f`tRbduUJg( z-TMg{2oHzF0{nFvxW%X5@d+O9#iu&wu9R02VdoUn03fC-iZ;dLvh<#MqbIn`7f+ke z#`7I$4*wMQ_)kuny(ypK1YHq4HL0!D2TZTZ1%jyI*9KUvk?*nx)4D_%(AA#(1Rati zone~T3PIcNLgtKuH1px|R!GCWo-3O>aJKP1#08E@0*T;=xi9(K%{Icm$aV@d3Cz=N z^y`R^GX}hm9)EDj=7D}NoCW=P%XW>m-1yxEAFd%9^2i!%J9%bljXy4%0b2~Ekq=}s z;Ejo)Y`3nL)6~$twlDQ(cDiV+>c;QGDmtz3#wt(hI=znHKY4VIdUZkLfq+XqhEg#4 zQcoVVlaN6CXUm^Vu;6oOguv|4TjC*VtSIe4vS5f znIXF;$?tSI8uIiXljkwH-h*84J&RnQCSRC=mBL{`8}dE=`)Triy4(% zj2RJ7LGDgmJp5uSfoXc3{5-tlodUf^&(4ai(X+;DPGF{nFReBOh({<|gah&k6B)LYjO}UGiOh|3MUn#52We3?9J=pFUVf z7d@EC#=2G0^i|4)#~V9stKy^jdNi@2uQjA?985N2Edmt}JsHzGSXGX$z6Vc%e~08c zkSXo7e-ARf@}%CqdU1Lj7|I}k?|tYg=kPwYzl%P>6IFskgi8NYZVaspJU_EFRnnGI-qls(5;97EWGCA;84bKMk5B9mo&pvyFYn_Kd z&QvbHt{y+zz!$hY?i~^Xx{P`|wIIf=9Wu>JbRNvL9bc*tMKn1E_L+|dc0F0jy5}9E*FkRwF@?wdFfLrng=>k3)xgW9M={<)6ho|GLPVo_B;&1NB*XB?GdxJk zRVEYzCCszLJB;fd;5rFU^fS2bb9OjK?RL~3Wz*>_=n?av8%H(^q3u~Pp6}Lp6u!1~ zvwLl8Ff`)i3hI{~CIOJhMzz+^2>ae!as$DeX%aFEgt$%72Na6>T(`Tsi)=Lfzt&O!*BcT3#}g&I!J>M{cySw zJ_R_^YBjsd$u=PP^K8#2X4?RiYsjwcRa_xG39r*hf>;>EW^`=2O#t++piXyED6qhE z>$MQqMqD0v%0`7SVXC5$3~TuZ3w6M`RZKy58_$-TQlx0eJ^rKnA*O z+u5oLb~}oUgDwbO|7sAr;8Zv}`VB4~FU^;QLWr5R8AT$XRmQ zQOJGK0UQ*E+Gm9dQ5Q72G3V;TBvkjvhbB}#o#0&)+H3! zi!st$oI^wm&6#=Og^-{nk*B$<36U2t)G6H&{=&bBJNO&8gG+4_SeF3HbhvC5lrXa+ zE~b^z-cc{`eCeHXeJ@dMf^|7pw}b*HGzZk#`3@M$JU=NZ+CP5w{ zip{3SQ`|$Cyygr!TpdMX}Yt2E>5)z-F7Zj8e9p|yCv&IQ6_c1t*F6GPDnWC!Mw zQ}dI6=-4?6JexFL;ORs?_(&bNo@Kyw;`9MR5%@9++|GP78YC(W+|MU)KRTu`qLp~J z60nPIzhY@unn?c5?-%R&R4P}2u_96Wt44&(4slV2e(b0f;!rAsK*hf{UJ+?^doSxqgkIrNL_ z<}TP9ni(-!{9wUBO~xS0UVJ_xbh2Q{+y$s28Gp3gV)V%9g6-B1K3vDbkK*Y_C0+zL zA?Blw*|chMb$MkcvAemp#vqgiYiu5HDlFq#eg{t-=^KK>ABwN;Re2>49GRR0>B5BD zYGukz4qy}8YgZ-|#ckv3V~mlo#D*^U!{heVe8~1oM( zYt$zABNDH`lW+a_0n=KIOFXz0t?$0;lHz-2=7GUF^E|Fe{7(y98<_wxCy$6x0*NuL zHq#3ivlPNjCb8w3LW~Fm5jL-_U6ei&m;Bh|x*dSk;+vVXc&?o2@!e5IqNJ&VaIe>OyiY z1Uw(UdiB-V;5BG@h)~jFc5mz9Rs*5*xF+x#?pu#yn0i3#`+TC-%~!ZaA)ig*}w=*wlG@LX~rDF25<1f8~T-uj0F?2)j0aC$_RkKY^*sQ)v8044MBHI zrj9Bb(nW~Gke-o=0W7zJmIEq0j|Yd6g&t2fJ2P`#u5~x>6q_rzb2uAcJontzmfL;% z_8<%U?iuAZM^|BeJ)qiVs?#1Fw8^v zpb^C>sn@i$;6p`*mKiW?JCYB8s>4nrbpe%zKETR$+wR_8%_IWNX*Tyzcoq;~a|?Ar z%k_Fkv|3d46H+LOsf)~YgtuJ;bVg_a%My6~H37k00ffd*JK=8yxUXwT@bRo|AXA=b zZwK+EHy{ntv&KjoHnG9Uc8AAqLHaJL7}zby3r6|AM5PB%N`$hKQ+bW}I0UT1dWas3 z*Ba$osAF_H)ponQ@!_o-rPjx{ZZ0QL8Ughzad}=eRaMmo^vztz3@9tO)mupn6ATuc zHOAF3t_Ew`k^Ve!v_56H>iv1*kbR~x`y%A?BIFYy`Ivw~1NEuh)Xi6c|3)@uxs3{4 zDiT5YR!uS!d>nbN9;uc`Mx3OENG28xFepgit`oar_|oOeFV6aGoT`F6v8u&DSd3?M z@7}#sjQ3+fHAeZ7>$>H(;0Aus&U@`$B*|uxMS_`w>*6x`n&iPnap|@hNl3sz_pjSD zHIpCRy7hwx$hB_3>5U?5T&n`KbrF~G$1Eva6qlBHq)lPi4CFtYqsV>}UEXY}+$`jP z-lm5g17E(P2&%lfT5RpcK}9TtqpGZt5hH-qCLT9z~%W2rM@tE>@<7ABCS z73vnYW(Jp+H+geCETsc4PVF^RdCk?>4PEeVwjC?q8}#locp7?+EUg@bsdh=`dRfO* z^A=Gy&lv6E0WwE> z*>CtrPxXe6_O{>f2|b=f%ft+|U~k(M3+Hr$P`R5;fW1se0_lSR4_u`JB+-&nGi5ic zy;Nd)&TEaHvw>hkn4c%mKujR8!2En)Tcusuf))XKVnH3qnz|}04OUvRXeoT9)134F&*#)Jx! zkQ-28Fz?JvuU173h6dj%f~>Hq>Dha@WT#B|dmXdOXKCxXxu_oqw z0|^a}i@NsPA*xHtB(IxMkyPHEK*=5E)q(C}AYK-{x;1oQ2@R|uvu*{!0s@Q$7LacJ zcy;UN!15VbJ!akNfyE>07SDmTGt^F<&{7cdE{38Rz~xXsAvSm|$?*sAj3Oz~_ye8C z9K2Ylr$Dd4{TQn~*b@~2^ieCC@PUS*x5iAxP+W080({MaNm{MNg!&t!!Jm=v7x5+v z-}#QWIAl#U#+@AWn3)3nP~C6?-~|>i9po~`bM_NQdXnzi{o9 zfwTzq@>enL&Y^H6_5$Re`a%E#2KLyvpT0K0_kKbY;A)-ZF6h#)z5umHWV3y39MHP(fM%niXlg9JV>of zwxPHE!D{BTN)KeQ=zLWYobtPvz_O1 zP32KX97wfCI*}+lT~y@CHw99qDdMZL9p|f(am7Av+CT(;Ma*OfHw|ri;N$#N9P>+% z>?>wpE3Lt+1uIa4#+=QT8g&tyt;pol`@4dKOlL3Ba9N~i60;kCm=UW9&LZ;Kaa|2` zy)Mvtbci$rvj{{xg0>r@FF4iR|00pl1bj}jE1|x!&5W`dsz4O-n5`&NiX6-?u9_`0 zLD(n5bioe-BLV>XNDdW|vs^P#ezM{K6iQz)u!ztC<^=Ug6Dwkjaeftgd5oy6QT|bM z4tBGDgFDnCiI%9d*>`cmgACS&CU!k6IudeTpcH^|8>>yGMjC|%f++Hp0fG&qjVCa7)WCbsBNP04XjiU-o6Bf5GQ6GEP@l`j*Aq`4~eU7hy z@-|IaamaoVMFEaww1 z3xya6pk1#1zVtV6%ooSuy%KFmuoIDV-fVF_%1=fDJUrGKCn6~hsmK`mJ+ROKN7mDj zGYLv=zuzT`i{N8(LAPyHi^$(ER$GgX;alYO506m8Ax5tqi@bl}Dv?BN{6f-WD8IhZ z<~e)2(+f{cRJL~#EQ&SP8z3zXttT=}HN#jOfysbl@T>os?yY(m{U-()xX;qXQ>=xB z;K~YuYbP5FwVp#b#b8)4gV*+nHpB4hPHQ4_ip|i!`e|kZ@0Ct58x};m`jae&$94Ya zAPv1Dz$ubA&1^c&WO}m2^ct?F`#7Fx#Xi}HjXcqSeHQz5c+aQE|1^`$cZ%GfY^@n= z%=V}!Q?iG0+dZ)2hJZ(9bGb!fd-Ki>)NrwHBxgpltD$L|jc;v^;>$>6QTW`Cy)ud9 zhQ23}tj$S`+Zg;vROAOa4k-AenuTzpJ;SW33*Ky&v1hY(#`wLp#bjfs-R1(#q<(lj z{_7n3ZRXR+E!l}NAdBCLaXZYJGjB2PFD=mg*PqR!Oy<$FqD&(xu~mQY03OI*r^5n9fME%`s)9NHvShh`8q2&eLgRI$qdL~K>$J)u@yPof(W1zuw??OX1i==z5@;j zq^z^%1~?+zZqyb-8ebx>B99i#57^ijO~imfdcUy9u?+qlM+6H$JtuZ`Ni)~6gJ86R zra*$*%q}sFoccNoejB*Gi!DZUf&Vg7E9Jm_ukX7I4JtydoEqRdYBpH8rZnWM!^sjrx}#?iWzY&e4pv?(! zj*8CJjN%Z!g7?M`c}$R)@-Or=Ngw(*IP+SmDLb~-cXD1g)adYp*BcCl+@MUBJ>Vj= zGzGR&8UtlXoDvcS!y%gBZ=6-$*`&*pS{9f)2c zf%#fDxxN|j+jt(uMx`AXYL5oE?GO4Vf7&i~>xWA95 zei=Poc$KoviTH}?qA%zToImIg<&oec>XZgA1wzk_MjNw7%D1?-=c&-AqLeQ)P5+3OB4?3?*>p-mevfm?T}EKO_Et+Z))|d z>hGg+b+dt3sEng=1Q_SBnDj^YUqNsGGI~1)zFx7}u2*Ng*@7Gp#WkqN2D09uuP^M40-hvS6=X)>?HmRI0{K0sn|Cy!hxhkt5qBx#E;&7?6?yAO zS1}?#2e#YkIj_U>Ply~HoPS{Zk$0C~hV3`>4=6#01;trYl@`@E({mrmj8YW%2UZJB zU@IoOm{~QEo7-y1&UmO*s8Xp=7f~uoYlZY1FFKKyrKDN?%GtVx~o4YyG8raxJ3?vd7 z9ren)>%Dq5olIbV2!^OKp`}wP1`*5$1gF-Klz6(i7|q3N&3Yvv`W3Dge|Y=$?T2w? z918E5u?XmjjQE7%f${30=m7X?NdnFMei;iMf=%Q{^CGu7CL+Jtt4qVHr?%*8p#Od#onL`;{zp7v!RgE)h#b2F_9#t5eQGx$ae54v2)>Nu6bt4s za!}w4hhq(vw&Em8zdESI7Oj8q2j6^{c=XQi{qO%=wf^LfzcLw|di`gA`BxUXM$u_q z^jMu>ugIh1Gg;_vl-JPm>PxlS6bRulO{}uRwHpYeIvs`sZl3LR<`_y!VSpL|UL!KHYl>Y)GX=1kpgs;-SR` zTaTs9x3S0IUkh2=q{+~D{;CNK2fz9I`fhXJ4~y)^olF6)GS`y%!Sejv17;q7JQ_&K z4@voPF6$9vY_9o_`@SK>ucu__H*2hPv3hlhIB>dy%_VJ%IGujUyrlQ;y82p zdDwxJ8e`&FI5vs-gZ>2TR{vxCfp8P$yE zff@1POZsMe86I5v5}!UlZ*vDmM*|8xEAiz@Zf|>gFI`pA%N3>yiplN0TxB^vFEa`q z7t%LX6puN&Yb?G)_ywIp&LrPRZjNvuxF47Ss#J9=KI- z>Q90+24X0}Qd_G<2TudyB?lvZu1u+5R}uu{(J&8UBI0`Z`MQBpz>tB&sXpS;;Hgng z9mUkF&?d2=72!2qlt5e#Ltk>kle)-&SyKDf?kFv`+voi|7TqJ0!ZrsbJF!& z@Ia7$>+0*b*o~my+M(atQNNV*+jg@Ra|n%|10G}Ow|l4n3;h-~MJJ|C)NdU}^jp+d zbVQG-w-;(T;C=e)Epk>#z0K9??49Mg$%ACE@c&^@buJ3Fc?rAhz?e3(XoifVbK5) zY_gV-LcKNTmJaE*N7Uho6ZB!yg$r%>xK)d9Y;Aya;D^$M4CM-kRd1df z6Eti1;-!~fdOjqZgbo{=_xnH~I1!wjJAe7b7cHVKFfsc)U`MZida@=5|MT?hW12ZQ za*k-`Cu!v@c2stno(O?6+{*OD3uh)LC!#@=^XM^i^D`qs@R+umoa{N%V)@Gv<+LYq z?UW~S+VPJa?rNFn3AW6UvtA*ypWa8j`0zt{sx@=?;HN#oALiI+e%gng-jM-v?F7r} zm{oP;_{R=+oxOPM_#eRv`{72%4|jB52d2Cq(j8tQ3a>!WNoZ3JMFYU9Xq&p^Z<{1I z>tDWnZf$0ieVD4y=dee3)=&}#QP z9CWe8zAj2?)+Db%<67iM&l>DOq{R}R9lrVa#u?(&2wvJj+>I1?sNm?ylC4&|eD&Y; z$v-Y!##i9eKpsZlQ*CIl=R%6t3;R)L7zzf)C!^7^aL56Mp*%4R!LI;xYo%)kp#`Co zo{44XAD??xmLXj)e^#cUxBK>k4|h&%p(ya>)Ui|jp6TpCUzVLA4eW0hF5I~TOi!Od z=*#`{hKP9twhhGmuq_{08arGeOT|X-8uA^Y5@aqG(2{%klEd<+B}YSv8;Ys-t#jbZl_yalRJD~MJiKY zp9cBGsQ<#t8K87`lBpRS=T9-OgU?+sHS6CW*y4r<70O0-|IubXy&5kL?qbLq@Fn0E zqEqHhcv8@UeOrhEtbLwx*U zlX^FdfTGpCwkenD^02iQhdqO(g`NrR@FWkKIecOCYB0(PJ3$BsH%9sTS)F+1#YBA5 zH{svGcm^{9hB{~xklT+0N7ao1|I)v|L2Nf)JAe7=!*AZch@+;ah=U6CLSbq4*O+m% zI*wLBZl5`FpNmKD({cO>ES_|aa9=Qs)JK%l@cDD;9xYsfFqjLK#bCYI7Pxk$COWJr zB4!cIopJ-VP#a)z4JBaD1ZA^7Yy*dDCUJl;_xrpQC1PV?mNQ$CAF`Lu)Z47nJwH7) zl?U5!x!M|_&Bb73#5Fae=+LNZwHl$#XO&)|w|5_Cb1InIs8&Y(slvlY&}UX3E`#c~ zSGGk*A|4LOp#Yw5e)k(}HOHd%Z~(me4#J&BUwmQx!6V?#A>9z{CSK1!q1rkD#ECZLdxe9EScZkgz;Ir_zfzcYqOAMqJp}h>mm*Owv=ymvNbVbzu z#ntGF`c)%`uX^~(`qd*xu70Qohk9`p_j(miKR7-<5ebig91ny)#K>?!$N*|&7}Y>m z;?mBeFrwhrs>bhbjZN`dc#QZG4(JjF3`A~P3y#5x`s>OtHx90!9^67~uT_P5HOv`5 z!jgg_5N35&TU*oSBx#O*!b%)ba6iudS})iBx%U5I+Bf~5!)yGpevL2EHC{OJ8sue~!F_ly{J6t1 zfBCG<{JKv5744IM9l0A|1F(!QdB5>btG#!B|3B85f34GieX~ykhPt*`-+=|0f=_aw z0T?@~ci!u;%#56~ndSLbu1&;VIpT>WSq<((=kN0V_8(PyzxjK=UT5CgFR>HA#;v#U2_nuJkH`~+tk|Yax5iC7?~b1pi26wAWPou?ox^B541n>X$WZN zt+!Zb=v00ulJhK%a!-zc(G$c2d%I)d{lZK%Q&QbQFz9udE(_GkUMete;ilCEm-wys z9)Ti)-|1bTvzBcm4o6CQ4^Yd3#^=MM9i9TBh2Kt#UJQ!{L|K)9bj{K=LuX!ANwp(vY5&8F_==TqW@ho&pYdTW^cT z2zn3kg*P& zvA?zQ@vU;>-u*{wfLmEv4R!bIRDp2GL1+K9zxIp2Xzl&{&;8UF(1vvsl&Vnx?RWPE zGs4i~Y>_B@XcHgpQH!@!Ca=<9!!O&%w3EA>Dh_j(KV`DUs;?-{hPq>p(0%Af!YNbjx> zPA9zO(B2`5^5_G&w={!oisCJQ1)t4MC=*EA0i!d(kf|MFZ!9j-E~H3)OT-lHS0=9=7l$qXieWaI}>#9c{(1fZesc zw^J*(B!?)ub3C;PO!|lHERzYd!F}c;xBRwyZ0^24>JYcft@wj;VEspU(av5oe-BU; z6JROdVP7BUD*qGuM%eLJ1l~I~D~h+`oDXoM1ZO>R4GTGc&{EIsAeSnW2Zet| z^C$tST@z((C%zruRu`0Buhqan@pBvNtE+yp5Bmwo2%~ssrePA6VG<%Ocx^3HMS+V^ z*Yr%tZt2z3SHRJ0^6sQW#y78Y%Zcj7t=k{1gNwh=wMn;r@Nlo3Xrma9rG-?d{rh+C ze)IQhsqG3<$ULaa=xOBewAZ^MvzITgKV01OPO10qo!9qPjLf$ZxHpUb@MWLp7EBehv{elt(CL zI$#&Ivvm~P0M>H@yEATHqnr1!>$v%VKM0TwZ;B}<)NX-oobJs|&rG2pieBdrIoK=d zswp%w2CFy>5j0dk>|D=Ox#@RKzcTIMs(b+$=bWa+yjr>B5L3k!l%Z?zb?@}VcuoVk2?;}H^s8?1@oy)cU>*!AVxx55(>lS5%xdnqw= z^X7K3Gzec26UgaA^XK*Llh=Ow zDyl0nPh~Vw`(`CgiD%?A8QQrf6W3{XAKbsUf~tGiPY$+|qjoN>YsRvs9C*Kfz8jA(dx))$>qIMOk&Byq2tu1+aAPIv<#7$bz55lC3 zO@5q8jEtvOp)c+2seM-%5ZS6**Fo-(TK4@fT)q79jdP#X6=kq}kSi>iWDn_PL(S?& zsgYVqR$8Q49VoC%cD*r)J^y1bSbJZDZg!xt4m3on6QFDJ7~#dceMjBZwMx;C;vu_w z-=J8>`l>vvSg7)Plm9oV^56g6H?Lm4_op|{e@5@&z**OWD&N;0scp*3^5U-dZs?Xm z@Wp$nePz^_T0hoy-%Oyk|Ek{h`@j3et1sO9=B;x-)^8goAMPGz0^6UNMs60hoxCB> zH}maz@0}Fzrsq<9nJx|OkKUluq-0v(U48TF^WVJnN9P8e9yV<7;Tt&G1qDd-a;xL< zgh#`Qk4u075#>a%tSJb-E05}pb{F{vXQ+%#54FmDoet0O9n+t`cjwWaKmT^)>I-Ma z&s=#?ML9N~;!F8^C~w(i*%Y8v;PL1Vo}ytdz^^^vPoww%p(!0)jl1(7%8hsL(=D!( z)j+C0;h7w~lmE&8-rD<>pCp0(7@jN42O^F&7_TSM2BB)bT+MvvkHX`3zkhdR@>}=l zIjkBZ1h!cy^kpwK_4hB&fAVKP|NQKKr?>Z+BjaS?gE#tlVAsGvV-|3YYt+5v?RvdI zb+mHn{ry}SoFqLkk(5id4k}N2lw!Wn6)iG~mUf!hDv(nB08$VEN(NR(x~i?(G}da=X{8 zX5y=>@x4k{kPp^cCvq{5F1FVU`Q$)CV{fY z$@SjD&E3Rq98?u6cR%?4jobH7$9?(!y}S5)XTMNu)Tv-+snKCAs15@12yndzBZQP_ zww#fXXk-TH77N>L13U=u0UW5=JrKBO^0^d7$cHy?eN4l6Eu})-R-QcOfOV~JBjfSN zU_3l955F(b1L)`vVC?uPJv?IRJ+h^iu3f(J!j&s8zZRYN)N8MP;*+1ading^%+zEw zIypN#8AU-^@X&jG!GITdLg1bW{8&RV$-N}9Wm?F=YRxoST@F?I?7*H-(A!cqu zyBlY2_WenpLCgOQS{}2?X04D)=c?7p$moczlGs>Fl)+@{5{r(x`z)zf&4BS?!>MR;GTui5Wp$}__V$L5UH|{w$dnQqeJCAh$aZk zF~rcdBiMIM|iEBc&1lV~hg)BOD>Dv2W|N1xn_lsmClUYz72ncCPATChfvu^{kPF97MgU*S1nZROE=9HdUb;g)jwa&w}C z#TAt{1x}DX{>a3c+1WGGljFFZ%U3SU&h`h&wH|2LaM+<;_w@H28akz;LucgFy(N>& zr1~T2PyXI-h1!;n;+l1fB)X{gXKH7?<}uwZtoD6*Y56qwp0bB5)k1P zFi${r!F;cPYA;tTUv+{VuDe?|C=8+P2rGx?pjHHU~-(US*`1 z6`?{&=~sKOWwkLo);n%Q57sf@y&xz1jl&EUgd^9hMNxev)JW5D=p4MeC^1GDcSBb_ zz9;^fHj?Khzb`KN~YM&N45I0}8_s z#8ywb5F~I}Yxz8ASuAzQ+-+96Ci>Upf^O7@3imO>tjWo?G_!vs;!f}E7C<4ZIV)|X zv|5qqF!;^rS0S|j2%-H6K}`RNC=A=qWx?%ZHZ_r!VKUQq{l-OLKQ$QzHbkQlU7ADL zI6bK41iq)ui4g5gHavPpcadw=a z$v~z9xet-pW47v+m(Qw8%-E87Z)Y@E?bz@~W1w{mkCVsp9sgDZqy<~o&D(pjsS+H) zAC~KV?S#f3WWm^3kOtMNjV><(Q_xmfj$1?lP*hc1WH=S#fRG9IEY^}9#g@y2uw(La zP-eRQcE%u6V)8;}Fr0V*O0y=5$x$yb;D~Hl#B#b&wvxiJEorE2}6tu`LQM+oO zUhncc^que4Q4yAsb$W1!SqAptE9j-3uLy)P*XveD<->Xjh_cBnIzS%)1@MU&s!kU@b2d<$5&>ILN0DAE_of3x-DUD1UuBXo~ zzk5HYat`p?saz@_Kt(pFLB>iugQLiA;8I3LVb8#$h;rB{;9NoZ6Nep?aA+196lgl^ z@{$WMK#Id4Nt{7>ZIMR|*8%T16#=(etuTukMlG)4sHp}c%^gwN!>Hl!-?yG^w=X?5 zYHG=lgi$j#mPi<*hOB2ojm!CzD50zp*1)caxYi;oA=O?9!QCsrcLV;~OePc#bkZub zw#cxHoB*^TDx%@e59W-Hyk}X_Y8Oz6qO6G)*vv?1!ei5i2RK#I-cG#83J()y631mb6up{llHoXjz8)T5=rL<0{mS*r?G zo(ehRCD%{|1k746tXj?f7#z+UWa7{YSS*68Z7^@nLIP(Y0go9pCSI>1qwH|KlFx$j zfmJQY#X@;;RuEz$i!vf6+>GkU=9_Jn0SQqR9Af^_k%-f*np2c))$GNrpm)LpDi>0z z!5B4sFRr#U5{XQP%4H;aRZ4igD)!Yb=n-NTs*#N3nVLm)6;TQx&6(XztJmjuI|5GC z1==@_MRDM4-6WyJMZjozIQB5~O)96;R&tujMD_e+1fY-IJxH8dxO;6+I8q~%F#q7M z9HRaoU!DIe{6rsjlNfi{RU|8R6$~c~CC=KjSgHk^%hhgR#nNiL{?Exvi6Wy(HI?P<}!zHBr}IY?k!1 zhvU}a&hF#M{Rn$Wd^Z{%>?LQ<-v8)eFB$Fw`hFq&|GHmb;$pwp^ZHsx_KV>&IZw|- zpWr6iFRZe&QAF}NYGPR(t&-W%?x7m91KhE?B0xJebON!O%9v33S|kDvJBq*KVYJFT zI7%2!m*&fu>9ki+HjBPjAe+TOTLmka?LW*`(eCW0XsbYy;YYw!W-woAuOL70kGfa5+Q=lWV28%S*=aT^R0EJ!>Sx(22zN7vYuk}RLz#ZE&0+#u zioWX;Q#5fe%A9Hk-ytK1LaDuC3v^uIAm}b)CDV%VoD>b*;&AkyvhR@ZLv~oFW-mT> zL2@`kQ~JJR9qv0Cs#wBWc=?s9BX9z(T_0>Z`Y8E}+I2Ad?KXy{zMe6L5oPOHG{jwO zE(^riuwf6QEv&zjJ-!SqIGQb9iG*FL?VVg#;9E{(V>5mqTZ04OWW-E{<~Ga){SkJc z5^6M?1yp`(!HA4ZPK+z^f0-SKw8XJ-hFP#72eJxoXkJ3WZS9zrh?Ue(6D_@J((7~N zvmx+S3=~CubPnf6(3>u(T&UR$soY);B+(qqo#bF{{MWG)4QGbmKd=KPB8EOfa_HMu z-Zb<{cvibu9de3C?c0SH$NQ_8(}83pT>--0h2bkWoSt&|kV&K-wuvI?{m7^lW>DMg zfFTrimthJe3Jn-TDDW0Rt{#JVl9e-`qc;np%Q$# zt$?&tMW#AP=cpW}7Q>n@yuILV7E7V9-PG7ig25bqzL}`%K}NYSpmGyT9W6vfm;3YV zSnJU02dyiGw9z_L>r-0a+qIaD*5S}{pf^Wae*=C0G*+!B^}W^9+HH=fzT5p^2MMzs zRFbNK0V8$fN^aZ-2#6?70y6{i17Ng;QQTHAWMk{e1l<-2E#PNA=L?NRqalA#MR9bx z#bGDFvVah)6UjDXA-~fGwp?1#DYxb4(4U1lK_b1ghf*rlYO@3T2+h|vM-vnHmA<_YDr908>eN{p&%eY zz|r6ey$&2m;OCOnhkMV55%oYn~e2w-S8DCEx?6o zjtOm&bQ^C~L1%gfC7eZAG=s9SSgbU_1JYhPn?vfK1zfVDwtB_x^!Qi-nkx{Rh+>y( zTD{qW>jUeD3uHbJ8z@DHHWLLjYTX#%GGz(_Ba)J@Ey|qE+X12q5x6LQ<`~)cQm6az z2T@yPOM!n#e=aocB#1rJ6{OCfc+_55| u;jp|Nhvm(*T3`oUS|F|j0e$*yhW%H6tp5c}EkHB3^Lug_l{e7oH%is<2aY@IPK2q z7pGm~B$r$gr`Jny%GpjYmtLJ9{-4>EgdimMz2E=)b+UTz&D%FSJ3BiwJ3H?oF-ej% z_)$qMRc&=OxMGt0mKDcTW7 zl*KF74;T7hceNzRz#qPR`KtK?oe7tM-h{6tX-e^mf#KDD5%Mb3zYq13Rt~IKP&YpW z?WaA9`+2KZty{mh_`O_-&HPo8@=UAOE?B+Tf7h9K{$tcvgC?LZQb~W>($hD!{Ci1J z2rj+#@{O1ClIptT^s~D6!el8igK((7=68HG8>I#CW zt^i#trGt2qS{nPO0Fz2~vI`|Y5O9v%iCYcg$avcS3kdNSQlH>=RPL%xr!;%2pRC-W4f2VW<9*E*O4&nTN zqxAhwdmcK7CTf?COa9A}jrvrGW-9AY7P*gfP9+kh5rud>S@F>EzkSr9Jt~AZ4Tb2V z_FLTN9-8S``#Z(+j0yKAmdX0^{|Y17i8_PNh5GE#oxwOaqIht4#>%6YJUZDkUc^6Y-(wGoa8C2E z0fp*V-G$CkMt{K(bB!OV9nvM#)*KX%J|!IqndGut>^>(xP(7NbbR>MI`IH2ldt@aX z=Ty(#CYMK&N$Mx*R)SMqI#U1IP<(kveW3m{qfi~+J`CcTXe0b&3+bMRSJVd&4RoY> zW|VUGIUT7k;r5IHofGe=zK6HO2cnDad29jcVdAxiW)Ho@7Z1-pbYrfKp665U!1?JY zIC37-GlZXLp(D{n`iI)6LLr!ET#0WUdZ|sCzeFqXl4zlNp1OpO>d^QTEgl*@M-QLq zNH_>i=hQB>N%%b1bR^mE&_(r$Hmd8Plj>ZAas|qzC}*SWM%jttf$w$0JlFrJ9^p)I zmslL%M4>rP#{($Dt7GnR7LGKJ=-xpT;wu%pCi!oWq|q3ZVVvKOLiU~dLFG9VqK%%X zBaQjRD7)N+`bhFa*VHD_K+h3gI(~?<55+?(Jxhgf{UCR`3gymU-bT3*=iu$g2+o1S zb999PUWxL6-1+mbIG&GUMtMc<{PhBq)i^&D1<$%(!u3-q;J@oG9J6q2MWOpt_M{gzkfek!OCVe1duyqn}I0?*Fn5=idScXytgLa1I)OIRP3X zP_(%JKKi>I1$Dh+M113gyjq;oyds`uqImQTp6B<7f5cPLIi9g2`5i?eSs<84=4|e3 zf>EJo2}WZ?<2Qst?Mzmvo~L+ZnsAi63(-mK(wrc?)HW58?NH2J@RpA^-S@OjGUrPZ zJx6nuY%LYS=Q*eT(KU`-4-kAR-B;ZqrJ*Fy5%(ZF|9=aEe8?dJ51mhj&lk>l;2yKd z>iWWb>vCBqd3QhSd!%RRdfc-<*PeUcBjNEB8c)wT9dSMSxhMqpoYT=$&ja(l?|aVc z9!7l>PhHPZf;>-za~R3xglB$JFUDPmaOfwm|DSMp=$LpU8VKe&dhTdCqx0h&`@Q2t-f|iy@XGTn@4pZ3adi8Pr!Q~E)uCs95QS_Mm#s+j zk8B~$E1Huu51ZWA1f#h{vICvV^#SPE3DVni{u=K2)`R|YCE+>HG;##TeWG}D zxUa3C^Iy=eD;USSa2!C{iSmN`Izt@k-mfcw?^|3$Z}}Fg|8phoeE~T7;rbKmZUTKL z;rc^dt8sj2LLI{I#zW6~&dIj9YR0aA0grf_z)N*pL=RwOH%2}YZE`z=@o>Su@G+rr z@e~?A^iR40=3+a_2PpTTAPys4jG)XR95=bkE*uY{(2>f^D7E4^x(`QsmhL}<@*K+j zk~DG-j^qQ7o%#Uf9+X#6E=B){YQG-w6S%-K{OE#ADRUH zXyd=3iD*7X&j1(UN4(Bk=z4fUISbdmW!!z>aiT0iA)5u=JXY`=zor9kkBUkOR;DW9$_z!Pq$x#8v9duqNjXQ^rCg|7tNcZ|LwQPhL3v#jtcq3XRHdp) z)ht!Fs!w&1>UPy931=o;o$zzgw4}(S*rbFcZBj~-At^7ZAgMH|Drt4n=A|i2YP7N11g%c1*QRMRwI*%3woW^sU7$Tn z=ck*l)9cpjT>3rwYxLLZZ`MDle_H>n{ssL%^`Gg#G+djiNnMlrN$QW8ok#Z{y~{N^ zItreIATF0Dl}h_qA?Lw9b~}5HeaJp#-?3li>2f~j!TsRD6X3xcoClKP@8yA>^I$!A za0YmAzH*UropM0AQ+XOZV5(5n3{{G%OjV_tt?E@>sJcz{al$F!!9MUHB58V3TvC#o z2l+`QoCoW{gHw`j_VPf(d2k`;fvgSC25F~)2h-g=NOkj|R@<)~(w?T1-8}e3FX=DQ z@6#X9|CRILdHt*UkHCX{;K9n&k5j(~5B43sgLoic1&QDhR!MUG#FD_13IANrxh_JT zP3%0j9JfA`cx`561uT=LFdd6x8tK8&bUfdWVsV-AYojwlG%|=nGs)-QF<>~}aNHq4 z$5W2S9KZFw%Z_Iq`}WwOV~36p9lP$h-*NS^ACCR|_#xE1={U5@v7n=>_fB5K~N3nX4q<7Z8bILoIH}AB+GYh}f?^L{#{!Y@{w@T8H z2ak-tahmQM-3Q7oG&AtO#NycX@)Pu&(y2I=Zlzc0R|b_KWvN2(b4giAzc?=CzpE8s zp7>8$i~HO7ef%mrP|jBnl~?|T^1gCHHBA++id03ZrXvoWMDiXQVxS{(0`G-Vi|58>)B3nLj`H8^$X8CybrbnfzzjDld|2Kv)Os8LI#1dsT`S!v-6Y*1&z9@uR{15lTh#&`|D5zZEW{g-rO%~rr5~8W z{Mi&Xl}%&O&_)HAi^a^!s#z!NV!f=7EtY2~&&e(F3-VuNn|!N$P`*vRM|oa(Le;E1 z$(GCW&Xx02v*jbo<8qa%OU_{{N+}$AFHnlcen^xw zT}qJhC4-a>Jy<7IO4X7>(n`J3QmJ1Wgq2tP8AB&AA6pqJj0jF@d{(pyrw^cA$! z7m``}Myim$mTc1Zl2!UnYGSg~0Qs$wewG>;lWL?NrCH2Rn$6TKKx$)wQacNl=CB~C zgM~VNy4Xl;*Jr7B2N*?yh5*(g2H**0Ky~nB_^^nNd2KnP3@;q*GambOtMz zPG@D(X{=N_msLvVFuSyq*`#aOT)1T$FKj?MAl=E9 zNO!Si(qGw9W|5Y&6zLMyggul8r84PDX+6t={rR^PBRwl^WI58AtU}tws-)Z4LRFEf zP}L6G(V=px=Bip%C9owGDx1o#vZ#txCfF4#^r~5PlIj%pv3w)@1RDD@_HXuu9Kyb2 zU&#@2q^yyHpL*;PxHQUIpWNX=t$(yPrM4{>C0;hu94+ z?0R+;?C#a<1?dLXBkgBS>3ZzFg-QRABBUpzAow03(j!u+^ml2h^r#dpJ&b+Fd$E6Y zKdkuQu>bS`cC7A`W=ij47x6=>Ncu#|klvB9q<=}-(orc#IwoaG?@IO3D7>u^X$eb~ zmNA_)pUsdKuy|=9OOO_^L}`e{Nh_FMTE$YO)hta~!_uXd%pmP%_0k^JDD7p<(w|t1 zbQzm1UCvsiOW7>x7B(c^%;rnCvIWxZY>_-556V4qpWH9^!V7pwu9l;fm*p?ykL6G0 z&*iV=ujMc0PvpdtqUpmmiW3!`?nCKO?^{ ze<=T5epEgxA5;FRyefaIyraCKys5mUyshj|`jv~76|lMsmGhKE%1&iI{E#8#Ol6MJ ztn|Siw3Rqj`ARAwpnDDBEE%4+2j`9$shG%rGQmNdo>{IN@gUVe>jq-rs zRce(AIS#islXoib?oCO%cgSsWyCmJmM$f!Mbsm|BSj?xmK%ON#=K->cG81(@A+syXl-m#R)tU9I|y>LJxrs#jJ2 zQhloW-cRxi@{96I_G|L%^xNons^1lUH~QV}_lVy!ey{l*_xssD&A-OK)xXDovHx!W ztNdT^KjQy^|5yIMsHdnS)rsm1wNY(VH>f+*m#X)xZ&lx?eoXzm`c3uw0oehI0@em> z4>%NXbc!;iV#?5zRa3T1IdjTCro1raNMK%IS)e0uQ{b(E4+Xv!cq}L=C@QEh$P!c^ z)E?9yv@B?2(5XSU2mLMR@t_xj-U|9K= zq7BIoDGsR&X%6WM846hya%RYdAyq}W#Eb|-#LS5D zh?9xhe#7)9roS}(?dc!Kq{kG- zSYj@X*&lOj%xAGuY+S54b};tR*!{7O#6A=ITI{jd&trd_AC-YF|<5@~p zP*!GEU)K3q|HyhL>x1mD?AUBe_V(r(8p>Id^UDO}P)`1?5@uw&b0jw=3_GysPtW%DW@){=7%?p2>SP&owh*X6elN zGdIut%giG)zsN7mUzWc)|L^(V6yz7w7IYMBFSx4Uje-+}afQu=rxu=5xUcYpG2J-R zXf@67!!5;-ccF;x)zB6dx`Axg@zHyJShphLST&_Ll4~ zxvk`HCC`++UGi~hTj`F{-KE!*-duWb>5;O4vdFTevaGUIWha+?QFfx-QGR>*=N07@ z=Tv-dE;rv{sj-}4xz{RNr&_bDb=DQu&DN`}w^<*tK4(2<{oXoa^Yi{!W7}$b${ua+ zwExNei2aMotjdSwDzc1&?(IBbq4$0o-P$9~6;H4!zrHM43K)SOzgujZXv|Js<^1+{P2eqC2w*I##b z-EDRE)V*1+)a&bO>Q~fXQh%ua!-n7nV?$5Fj)tci0~$*k8yeR&Uf%dl3HTgG1 zHffvkn#@g&P0pr8P3xOZZ92c{%BBNN_ccAy^lH=brmvb@&B4v_%`=k<=Ug`D&N;{Ce9XSdth=d>?wKfV2;_N&?t zwBOhMZ2Q03zwHR3FT_@XnETY+Pq1;O>CEUX?`-TG=v>peqw}iHJ31ff{AcF}ohQ1&x-z@0UF}_qy0&(m z-F11_fv&%GJ=yhY*N0BondHoNRyq5e%bnYuyPQ`$Z*~67`J(f<^SgN=^R)9!^JdMP zKX2o_UGr|3_t3o8=Y7>3*qzl~(cRqL-@U&3)b2gq*LNT6{(JZH-AB8B?TPJ4?J@Us z^(^ce?m4ID@}2`d_x8NdbG+yK-q2oMZ*gx+?@;fi-t&5I?0vBJ#oqUOf9VVEOYY0- ztL~fAx2$h_-|oKsefRb~*>|+>yZ(Uww0>)USO2R1Q~NLOzq$XR{+If{9S9oG4HOJC z44gdh=YakJ+vTnLG^-;1uGVu zxiDbiyoLK0zOnG@MKOzJE-G0xu;|{!_9Y=p<}cZ}?y}8=H@A{&@3On}6COZ3);iZOim6Nn6sk2=qPTq6!lc$89GUJroQz}lGP5)t~=R{<|h5RoC{yK8U3c`}Z^vPk2B|F*muKg^_ z^$5zV@{tikzFg*#8e9C*VKOG=@cQIH>{iS+kRH5Ntj6J2USevz@zpwk&WCp|9=t;uLADs9g# z?62Ih%~Drq*|xE?(q6hrJ~GsmU!R5;Qcg#ixqIg9%F^~#srjT-^pKoAJG=# zt-%;pSX6}OqBJ_7*ZOP1z-6>mfaY1AeZXiOu-p3!owl;F%F41bo3pS#Fu%{n>g~Px z1-&+NrQPhh+gw>`w)PdEe;hmS-waRxCV>w65jQ(A(nLu(&CzI2$1v0hGCHp_{Riq^ zc0|oE5AF5eYQnILK{(O5wgMGeyq-e0x^-ksdqLz9))7k1V z>Va-73gij-9TCQ4jY1>;8(e{G`V-^Pu~`ibt~-B=6#d||@e#o6frLSy{NJ<%S;Abw znwI!L+FZl`!*K}u{BD-!dV@7!1{|O58fhh7^4gp-y^xLk$)NKiCxZ@73LWr5p%pPk z0xxtUwFV6pz_qr}isD|4_lY>4C!ibM^g%0n@jy3vsUCwuH+rF@9|YcRFWvx!q(wk` z1e7S@x^f~7pwC;k_lGgOuX>@2d|D$-DKIRIB+(hd{0RZ~ry+VeiJtTs5caT-ige8# z8=i^EQ4H23{ab(*86w&eT71iGz1`#8$8K{SVCAkSCW=rqzx;I_c9QYE0! zn|y2q6gnF{#V`BCGXtQv@t+R=^*BFuIzN?*}6 zfq^7mU|1B+G4Q%1@d8S|fegGx-rwJmcpnz90$+*$STJO~w0qb?^zzom`nG^`H+%F4 z-ddQLT?N805|^RtS?H2sXN}cbb32=|IdJ2`%DTErc2DcbPYX8^{diW6=Y4JF{2nMr z6I?gvVZ4I5)d+(>t3>D&U0Ans`ToJXA6|4|35NYiR^qzWbtY?d-A0(YA&qzjoM_>g zR&v^~izr6}lb^tptXbz|^>A*6`DjCHAlgj2rIkO^cQIjdKjVd#-^8K74@_R@O0KiU z*Hz0^*!7_iAjL!NDx?EUM!kk)I9U_P*1G-;3O;N((Sm#sP6ztih}yxZ4gG|f5gip3 zT~uT;k-!(4WfQi|6BJEYj{hxut(&`Mw)#JyoWI7@V{g~W$y61vPuCp7sS-f zPGOgi800n6>t^d+3&9QkH0M#5;1Smz6QP4#a-E~oXypPPiMn-s44k8ld_5EE@m_e- zn1F6l$RYU3W;Fi_BIXK$3cAe)TPq6XK|uycdn=}{HH9^e7|S~Iyi}1Sik3(c$y53P z{>J;*2l?0p3VeRlC%X0eo&TCHB6Ex*_1y4%*5KfBV*>d9|& z-D9pKIpA$@en88UXX@3ak`HuDoBBdMdh-Wa*kOS|*x^oCKh(uq2({2JaZhLjYE#%@ z0Tp(bpu!FdsIbF=KC;6CD(tX;62AmgXeHhz*WdywRwtq^%?be(`bR)X{|G4b52Tx* z;YtQPn|SOPNsN1^+*;|BaenC}{J$`AT0}!h-u6j`LEdVMnAXseSva!Q*GrVdR`2^+_wGH2{`v5I5is}hlXGmtZ|2LGJc{|72JTw;nkyUB`BD~LKsK9JktkS2 zhRWk7^J94qz21?h%ZXL|*pxf}^rt&rKl&-LIe{5@c^N2yIUS{JZCQIxe0pR{>xZqb z)|QBj_?EH)wzi$+DR9N^@dnic7IkiWHJp^U35`dO-EFs7jAdRcxYZa{2bQ&)jr}HxVK+(qemt9tBuPnJt zKC(C8Sw3&YL8h{qimimDWwa4H(C;JfKo99jjn?}AR&b)nGlAW>EKn5^fW=eM{K~4q zqEh3A<3uIO!8`q#alpo`_I@KMc0E!#6!Z5J^%cx&uB$V<9;v9OHmU!~utT9}l_;k* z7UYxLUN7n0$8Cc}P4>%{?%Hs6aaC3E+2BF1rG9B7;!;Jfr`cIGrDYD9R~#E3`E5QU zKM^V{2U%~lj2R^A3d>PTvrYa0d(+%MPIbE$g@#lFwh->11$nf4Jsueq4Z|tiT2 zjLXCx2&+_!5kL)_3;QhZ-Br?&Thwo> zEGc(5%CoOO!(@|#%{>L3-$32;TCD>`t+f^9j^<CdD*|yK9DoaH~ z;Z5-Xc*!wx89AHFh>!*oq#+sWzi{-*M;fBt5t~$Z+dqk+&I{&|nPx)o9?n>uh{QbO zri@zRGvqCdk7%jx9q!w4?j3>0!z~OPJGb|Z6MvSqaJjRrqP*O>EO5<$96s^|4GA3w zR|6yGJg1$u>ph;#M5yp&CPIbG2qrs}NAKy`nB`uaF73 zO{wKx#YCTioLApI-j&b2h=;sgl`r%74m}e)EvWkexXHV@WL&Q^d5Z=WJO#vbaZhk) zTwO2CfD+9DgYXy76F`X<0t$=f#vuF!(Z7BVl)1RSAfV(gz)#1jjpF_WF_sX1ueUI+ z%VYim8}kr)^PT3>vI=Lq+d~-h3~0Q`fu-Y^Z{Qt&is%X|5O?xI$HtT_GxP2*1Y_nX z1rcI}f@Rqk5B6*e30JAYLblF3ghko3!mjA(u0r2PtZZ`R)KFQiteDebqBRJTQL1{(ik}~2il1E4+Oftg`>7c7Qz?~L1-u`Xn-LnC zksBq)jXO8UhUhFqaIhgOdgKW8g15{^`C6Y*o(L6EETGh5QCCRu#JU)J^e7EIqLnGx z#|UL?sfP8bd#^>E_VSa9*G!4<_luaaZvK-mFWNpO!Vl*sA&YGB8)2zIL8)PHxUP1c z{c%_-2oC!MSmBGoFC)7Pz2wW@u+g`BE(#8n<*C8v_q@%**i)`oV{`!lx)^K$nOt9@ zr~Fyo)0=#HIuR;*DxlO;QCIX-2VWVjd!fQto`^xnzJQYK3k*W`1(f_>0fjt!DI04| zNK!lcinS(>jTu=o+&Y%5X=in=2iY~Qds&-P&S`ayyv*07)wfiN#>9rqtJP{uk_BzJp_WOX8=ew{ls)PA6%c&6RY zIc5FazLlL*r}mhvt$)rgv;=lFHg%c`OzFCsIfHXtJC;l>chp$(oAWbsjG%(Ir-Xru zgPh`09nm%sYT+?+0o~%Qdm8VD4KX2?5{7l4MMdjCfv0BDCr)=i0Sl_^_trhb2fEJ- zJ<|s|MxWUS?-;bg1C`e(XLvE}m_Q$DdGC4Ylaqa(ILp%})~RFk!G3vpGDhD{ALtY> zUXz;=vM708sJAuZlowj^(HPzmFSOFr7hw3r3$^>v>eU82JyXs@oH2_mxmcr_qP<&a zd{6B^%L{vtf@oI0vF3W_4}?bq)&k?QXpimh{}V~7@=|Oa>xUBTh1z&Obg(@}^rKL+ zA}g6zPDUR=CqIf@`RD?(jH3o?DDU1CttnF7t0^x1xE0}^AesZ_x;96Bk}giyURR!4 zI0Jctu`~5mP4DUxk__!FHhq2!KnbSkq_o;Zt?u5P`sBovz(#AixvoAdO&f1%iq(gO z>0>+UFIu9{8M|)E(J#>@l*enC++fJgHb5G95BOLb}P zb%C09NdtGEmOyTQk$l6(s@Cvur&Ep$Z>ww@8d_6gx0je~w!k`z20_Dp+coC;m3`Ot zu57Nd?zL7q%zMoa((CPL$sZhy_xPZm#f5=;qJm+j0v>~><(0i}J#sc}#bpjhS+NbL z>nU=y7n^+xJB=lk=1a|0rG=doKjWBr&z=`O({gP^?fF2>D(Y`L_tX#@iQ~ouV%k;U}tpz>QGfu zob9^d+QnIWmNsV@OBXq;O*%`axv43zcwuGbd{cOAaG+5?ud$_gzO}k#<{InyR>zXk zmKIxm1yqmk1dzoOaZ>}ugS6<0~&>H!f)Gl!w?UK%-x&DW3(n1D2@9%4?YH_#O zVkkD7i%Tjhrm$YHeSu*6+g`Ri&5kOo(^|!TXlM{)jG34RzY4Y){a7W zgl0#CXC@|Nkw$ysHj0fLuvZjX)y^p!(ro4mJM*Ju+FomKVPUUzz-X)1>#Jw8O1;hD zu*G|%OyP6m4UaBx%bqXP;+05WD7Nk=sXHdU!k!7d!k)Fm`vC^n+X`-P#l0FY9%0V} zRM<0`(ZZeysIX@QRTQ#k0xIm8fYOW+P*@o+ePhsKHx!g`D<|rX(cs&|5}qp}@RF4i zbt}j4aw{jGc0NBv95N3!u1LrW@o`-15yZn=33V2{taeAa>#iXIdjd+(F6rt0#;!Su;Vejnm85gyUt+ z9T9N5Y#`Si&G!umG?tqSO9Px!R?o%apHeQ$e^FPEsCi@z;xREfZWAhQ}4-ks#dKeiQb?xmLsd9K&OJ)1uV0&dtSXkG**?O-3ES7_w8x^%BLLMbVtJ?;hmZD;teB|H~jjd)OK77Fz zm)RUv*M6c7{|Jlh&z>gv9X&?Zd=33AZmCrjqf4cIG+!`InF`VHBkyrBHilud2@2R0 zjGoxiM69*_u>(wXe=9q!)s^L4Lu7JpiDyf*sbztYd`AJbOq~o`;R79O-83l_G!b5C zlQBvSDx%b1Rpax-SeqqZkF`lNSx{0L?uGK1ETDGaQMi8q?8sz@9g7n9=6reCpg;_T zRX`ZX?z_kPIA_pT$p%9*ittE1Szm6p$vd_22|8Uuyw=Mpnxldeo4cQl>@bZ8gX4l8 z7K5SWMK!X!P&33cCn(TGQ=t7+bM=SJI=tXuTV(~iyLIH}fwi1ljqC)CjT=h^7TqFy zA{(4RJ+Sm4GKTuV;?HS;2Mj1y zys{2)cz<>app0jfQHCVe*vWinfs+=6`OY@w)u!FgA`D5l8yvap->oBm*QRpWAtw|W zjlc{UqX^mOgqcHmdjbkALQqh_GyU9k>-attgr|JX$^X38b0UVuJTdY;FQy|uYytcmzqYdhMA$h-Y zeeBl89yLUH9$uF4oo0*CTv1ePaY8>eR=4%Lx0+d(P)o9h@T;)VOyer3fc(4LqZJqV zyg$KK=lJY2kC{@BZ5nfTv&628_${;B%StON1K@h^#(Dw!nipf=ti^8KZME|sYL(;X zoQg*O{8goVmw6)8;{$;fpSm7z=O?jxX9B}-yd8AI;Jzm!K7^fnJ8r1Q+xba&J8r1Q z+o3k$?YN;k-TntbNh^ppO>QXpH921BSZlsid;A*00Bz{T0KbOoy`QiPamw%RGXD|p zcWJ}I8%lG}nq&yrOSbSZ+GTd7{_bG7$0CmN7wa1u3qG?+(}N~ZDxhP%@~vCKcbJK@ zalBWeuE$#=C`p!p+V~DL)t%-unq_00C41oJ%XrVuXNTF8A;Sgnb(5u$pjd0uR!+mQ>iV+jqB#Sg|vH+mc~h-RFP z-&_8JvC*-6=ecTc_co5bWm@N;B*@E0_C)$LOEaXpVy_u_ZF?)KX@>CjIG?hAGed}{ zLITO7B-%uLBn=)g=8PvkGD%E?7BR#>Z%o8HiM>&ggO$zAm1zaBOeMz@q*XS*o01r- zSG5N=7N-@&$SM|FkdT;?l9*&L+`rS1gn-E0x)b#so=C{vAwzajthr8HH)R`6Py_@& zOlqPt=$3Bm{L)&B@BBvl=r8hmK^&qW(Hhe@r?MNX=(ei1`Ps9JE9YQ^x4Fb-D>2#a z2jwHQj9$@qt))AkFDZZWje984+|ds1$GKOr$S&5Hcnkj_&NA1zck)2Xfe${|_lL0- z;X9MA7V(0~-R22Vo&AXCv{tnZ47T!CU-tORo_J2*3Tku2@RM)6fJv*|Hc~U*Mrw5K zIS{t0Qxo6RICqiLQd(kz>bG^xThWsoW2^Rfs2l^AvO*zc`6PQ2p=PcTV&DP5B1875 zF4?hpIMTk(PsnS{p_KxyoKp*;;C1lOgnJtsSJ=izphAS7-TT-&Hlw(0rk~T_n3Z8H z%+0BvJ*&6ZFsnb%>7SQwD9BCAtnZpVFc_e-X2okWQ*`O-;Ltip{p@;6Y@)%SO~Pek zZR1>`ms7xdQ7n3qHW}1R&I;{9acaZ)O!-Z=dFp@DgV)D}q{99u-5DRBdh9=RZj!!L z@b^uSiAKG^Sco3ieed>R5I=gKMnddniv$Os_Xm8owA~pWlq`mQZL784m^q7HBf#Fa zy|~;_U6GYhY_Xb5b~y^$+nH_U`OR74U4n{UV{1)mslC}@EVW{r`%`NB2K>y zt~0}K=xMy;4qcX9R@2#%l``@KYQk!Q_P>*7n2fo|7p8DxFe4+$TU}f;>xwI!Z@*o> zx3QX4wq9{XtLu=Z6(i?)PNBRL6GA@f-#*>WcC7P0|7AnNmy_T~Qv#qffrDXbq5KvH zTkmuZb$hYM%dfn0WY0uAT#Bdp( zxoiu6P{a@nzq2o%*|*kdH5J>O>#;A+dR_a?)zxM?4y@r4Mf)kVDybH;MPxkEPLdB) z_+GpnAE@xX1eE+BfkF5|PTJYQ8o>h<@tuhngdZfJbz^b!IzPE0EUY4#t;KeFs zEz47sijE3+(l{w;?4Dny3Y6slKQl0Ii7G%gS`^zcb~bW^ZHp_OHmx|$bq4sXgZ?n` zQSAcFp0&e7sE||vB}o-^g``fbi?K#8ki|&S`$v1<`H$VZbcH%ZrG%(gF1h#7r5n{D zeyXVfo7fB%bRZ};ARspAfa^QghX;e=r%Z_tI!GLax0CFS8JK)n%Kke0*`ZU^!75d- z|L}n8>w~Y!D%UMxxzncQhOs$FFW_?@&+?x3`Sg@y^nr?=3Mlnd)D=B7!HY-hZm3#S zBkkj(KM{wJb^#@67dR@%>T(GdP)M+sHj*C!74n1JK+*x@cfXyi5cz`_xt?IvPPV1h z>Ds}g54q zS-p*XKwCd7bcNXg%sm1VFvkKj!{2!jF=0RQ5r8*rCHo7#Le@GmjN1Gzj|iIBn$u2?_1!r#40>Zm*Yix6sbp8-WmEb~GF*E~E~7+$#os?BJ7{`1 zKEbrn(c;=t_RAA~ENR!9kyRbJ?2eIjZ|>Ulrd;9L*TmnN7y^y-rj$QfS5U>TdE6A& zpbBqUfvRBCY6OqS`$et_-n+zqo>wXG_KgyJ#+Kg28I75XdWTQzS)JXOy$0Vf8bl&S z6)#u0cCrqgx7)l%u~ zs;I9sSJXR;7gR0TCS0nr6@hu3Wkz#tX<1Ecqp7%}y4c)Z(BG9mi#)2_xn-6C2sEq$ ze#nOsZ!*BzAvYDwqr^m$;+(6foKN#P_YmQsoE~tqW$c<~;v=_TK+$&O=r8O9+Meo;ZPNG=6R{+V49`l%cWj28 zkE`X`9e#6W^)qaw$!EJZ?%0l28-X8hcK}l-#}A34ccE}?;nzH~9$EPQJbN6mEZnD- z)La6ok+5ENrSoItKLIJk2{?&niZ=S=+i7%7IOtt0f0}c+CSMR4h`2_)2GqOf4@G7R z#%bAy>t7!{J!G0JhX$=1bp3otj(1(5t(ZEsLdzD6(9S%6l1u!V9-hHFEE@et)+#b= zK{Dn2`x&t;pm|<|NW6w7B44X&Qp2=JrojFnZ4mNZ5QC1%72O5--R3#w?)-vz<~jPJ z`1qpD24j4@(GcS(?6G`m>oYnW#y;DpmYzb#ZuVnxMdqjJWl2e8>7Qm+B!i|j?}**V zd)I(2{tS*`A166XtdYqf;=7U%dUj1nO+oJ}XH8k2Inc>s6!qHvpMNf^Q7a7c0q*o- zc2fQod~-hptocO2VlFi-Fhn&U|0o^pu8u>6$%zy<`3CT;9T3`TP-H-k|4tM^EQc0GgtWh+^{pbehp? zogX`|-M$^o3_}xx8S*Mt zGL(*6S6&XMSswibKZM1tQ?V491kYK!K3pOYEg2$2L-N_J6EGpK2AF7Hjo$l>At~mV zNG8OazhsAW^hT6zPWrqhi@Vd)jU^?B9_jYKCj(u=%d58Z!06h76jG?mTwP##6Ywl7Fm%E`{;nd+8J3E!V&>2=Vp2 zU*2C>*iUaNIW2g%{Ma$PWo~g6^m*S_qV3MAO7oW1lPN-m*FH&t=J4FNu~yvK^Yk{U zJN_{)!J4Jn`U{-aQh~qJ>MZEDRnl&6U_qai(4O47#W%GYAvx;X5;R5cF#6Mugy))U z5!;Kt!WMC;`yD~lYUUik50{W=;xidRoyNqS1TdML6ty~R3N6)9%d9C@#&C z&z&`EG^}-?)%6dSR$(^hy0B>|OXc8Y9(c*4jRLeCl${HkON-drqSDeL*ZD=I)DoE>(lTOx zL_pz@z9ajiQ^*YQ+`;0#_Qs+?+fnEGb2snUVXCfT*2tyxmR=(a5xpI;gQ;rDN^5C+ zBzV3wOOiY*WKYk5(3c|=^dXa}gt$F~*-)#5;13E}qxv&$BXl<_N6k z`<#Ailh)wEGb2mNBW@&hibpzF*5K;ZgM+IfTxI5OES7J~Wo72THG`~<;I4aZeFd|s ztY@ByB$2N5*2+pNM&F%piE*a9MNfAB5s@RqN5|`MVp~QeU~*@{J#%U69Yd5gu;z@d zyLQc<6B^2zE9VX%m%r3(E=37kH+15R3-C1_TV$Di^(D?Iv#)Q=}7q3ivkqqo`WXdiN3c3Ga$n0MLUjG1|vdu2ltzMK@` zD64Oqm0?KDXiiH>Nyn(2=EmogQ2AfQvLw0=Ga0j19as_=j+4#`%YMMJypZOpkA#E`ocwv;4FBROB9p4K=Z7 zS1NgjZ0A$620;PO48e@>r<@AUHTH}RD)I?96cmNy+yO4ReM7ung$R>pqZJ9U>hR&! zbNh{r1@k*r4?Bm78jBX-`=r5N;h*bA*Lx@M4_IuYKf@2r#&pgwg)PVP*Tv0(GYoIFiNLVjAQe_!CrS@~sivaO{x*5&im^_G~V z9DQ0&b5=@>HasQP+@i0_ooP|2{Osns4$vUHB_8P^jXmBc?Cn58`Mc{n6o2_$teszt zx0Xb**RWpBino?UxYEHN{yZPC1AN4gbqlNm;170>-FwhutQm0jiaqE)+3=ow3!1ph z>vJr(h;dIKEHR@$!-gB^n>-j(VY4u~)uFO3COn5mbA$6yKUr5-rk+#V!wNyzs~^r? z4wuXGPB8zT2KwdmPO#DQPOz-eG;XOy)Ik*%ShZ!Ytf<&v-+XMJ$%a?NE5R?19BI#9 zx_m&7m!(bFJ9gXK3A2Y|@GO8?BXTEnnqD{-a54=ad@yq4OUf+gyN!sU`t!Y%vFCo{ zml}v}h%ugV?_J=TnOyqXxTVX(myDiwQQN{VHT>oc9QOLGS*~=#z}MLDL;Pvo?78;l zLvtwiKO1E&x70JZ6@~xFmzmuA)F%0{NOL_PgNCrDjW{{RH1F|`*K>qGf)rP#lFLMlAmdWR`y4uzJBKA z(`+dV*Ep*yGRlIShgE8}VBUcP1!eFvLwOuX_?d=3@G}jr4ECBU!^h9$wI}m44Xy-s zoC;8bHY@itN$>r!pJ{Lz*bz~D{7l};q<*HsmBn5iIl}$SLCgw@00oMSDag(Qr8?-` z-?MCa_x}BbCR5>l+3GGyKKC4ci$Qpm*sM7xKHVs zpPZ!M@hM{x7`FF>Pq|fLbvD-J_9@%Vg>2|>dv!*2JA2dh$-L#WvQjeOQlb?re(YRk zgilHH&9@OBpVHd4v#x?X)*U-vu1HBwPpOa%hYufacceMmS%mA~4OuB^DP(2bJ4%=% z@JTTC!k^3|-xzw>y%2NnDUn6>x|6iMiLZ!oZ?bPK(v~g7Wwys2vy~NFaItoL4s&8l zn#f{C3&NN5wBT9%kGYcLGV@p($89wU+$JmG?i0S`!p_3dDsx&(qPeof*trmfeJ>nI z^a-lTCvyrJMLs!Gh-_BOVCD;qv*XzJ-{`mnLrVa)w7*T{d+2Du1i zCB*x}Chk{~Tgktb#gpkpsx__WooLm>Ia>rs>^@g5%0jjcu>pilTU%LOE01W8i*%tC;fc_AoyxgM z)O({^pq{)1diyvOqi1A`t}!4Qwfj5N}>Slq8L@DHP@NoN_y zi3h@I7fmnH7x6dx^~RXpucYPEEB*OtEq;ow+Q_Dh%qG{z#wtW3TUOSUuz3Db|9d5M zD`_kP-CQ-W7cq8f)yedCH;`?69^e#37GKK(+y&oN!1=eDQUwIE&f}C+pMU3P3~uN3 z?|_b^?_3c41>odyOyD^k=M+^V_*aA2$wt z&jZEGCO?xRH9=VA!QbIN9^V4hV_>{NZniu=ySSsT+x3d~!$AjUnR04sI=hh>GwusQ zz)3riC&uY|u{ws<;@a5rlHi!G4{q!0Z^bJWc;P5K-1|mFAYL$XC!`?z2Jdh9B&7(- zDF;DAG7#+@ePn65w=aCY62bc=nscWhm`xQ9gze7U>YIRYP<~ouAXIcro7Pv0^n-)U z-zNb9a_83BwC;;GVlN;;nVBbE!PSgD);$~$6riZo!Rih39y@pt3#E&vrzeJoC#Fwl zD@KkS1aIign`z*83{h2s1|EcKwF3KAC& z4kS|l&5DWMTvdlEw7>ujYPHT(gte}rDl(^eW2@^ScFkY%t1`k(xjX9}{=v>*KjS&` zhVIeY>wAW0wK{JrXsIlT&oz}g&Nw5lAJ`$!@Lm4f>{|ROu{dt0F}*ythjvF2y>{3u zU!D-pnB7q@c9VVsshNeizU84stH@7vJuDJbnUyxsU5{Apte%!aTtCx#0QxzZTRE6j zdO07V?Q4*|e3)qsNoP^sa!IRegFB~s0J9$}Yv^5CDUVqw+L?$*EnwcQ@ch^7`eO^-yCtjE;2y~?A7*C%hts;3rx=Xii-OBimtAjrly)sk0^9*-2OsMZ$#K!bI1Jr zZcA}>MX{-|)mC0^;{vgYXqmYBj=^e$-gdy+lj3L;brfMj!Ekwce3~3=@(5Pd2hk|c3rGA@J*o&`q*t+vGXBAh@MoQsEyen2x zQeinLPq%c>oH@@rA*B#`g=Pmv1#hRwiIC(Z@XYH4y7@J6vfx|nc58o;v#OXrn%d`K zPv6k`=P!il<+PX4yObANteh>&(F^h*!hxOVMhg7o<)`LD3AwvMTFA5Jhfm&#tkMtH zX(G~7qQu9(D8$V__H~$E*Tnm9k-EINOCjNKwn)BtPOC?{1H95r`AYQJK%RqjYR#;J z?4`%bxASZyk${CU6S*uT+trXCWMqLCl-m%=-z$X|;m%*7eHC|*!X5Y2F{OWbo4tH_ zOV9Ftzp~1_jFf{1SM;!M*Yy<+_OFim2YXf={KZl6U*81s@j$h_#$^DJ1VFlv;a&da5xzp&W4l#5$Rfu z*u(P}Sr2zpjhG3vR7N#?2KRn^)WsZ=l$?-|TuVQQ)wMawi>&_Ez>emp+g1hksmn5- zNzKYieMbCd7D4kaZF{=8qo$&w#xws6Jc@P#{PS0x#Qg%Al?I+wE}*z}&s77xYt97p z6b|Jhz%kGW0E18N7G*~WD3q6Hv^WL<6&U!u;~2n8F9yyPg39ow2#OtcvGT$DE8N#l z74s7OqQU)LpFu%R9h=;5@LM{t>JNMe*|uvJsOGki)Axg*B$WEX?FWbcB%tIr(VPdx z6x+d?c%qjyK2pk>-LBHNJ|WXFXK9kyDe${j1&BF8E0NztD;@Ys3N3W>(B;lk@rSx* zp8A6J1?dw&efVK@8atTNL>KDBC=hXoRh%h>lu$1&`i%_UV9F zXB2)0ZC;RiW65}}8}UWjj%&CZ9}YhKv7wEop+J0qtD?HLw9s7Tyrb25C}?eeA-0@= z8|ySgEcwnh92WZfdlY{PW0$+`VabE8*S~V^JCEJ!vZno2yyYWy5BT?026i`R=2n?&%vI@H zwx{*m*xVwvYJSbY8c0qR#yb*pX}mEj=o#du;KP@gK0J?yMqKa~8$#K5{W>x6-k}#% z5%%=*&b-n^HP+^4D@tJ5Qio~jOqS|8sKg}{FLXGTmbJ7y9PLyD2ly^q0LBYn8X+aF z+|T@6pF7!;Bl6kHAL<-|nxXHlkpzB#7zS2;d>la0*!X%sFDA(H`2d+1bNIhp+}wXpNZ!LM!WPtUFi%)>Vt`@utaWjlH5}y`T*DNV3TT%nox!z>GUWU1~8toJQ5B*+8PTC#(( z)H>F`5x#mzggfb_p90N|`=eqqbh-?O(qgS@&d{H7gk3Us^R(d!&IVP_Gu-r!20jr8+~eOHb1!^xXT^277Z0JGarQ%RnpXm0i|Z ztKw?S)<(j(8}!`*jED|V<{`|U&^!xYJ<$Bu^BwGx?w+qlr!IE=in{c-^6o;fp#iaE z@Ms-A`PJO((zv&Zq3z=iRQK#DKo5;Xn~v>5aii_78wFQx~3y4d|BgmtvM)3nKk8Bz5)#@W^S%$TUU zTvLtFFQ2psbG3JU4r0_J23EUqhfc8|i}sHMLb5r;25VzwtwF3%c>HddAhn<0Sd zw$BPFpR!?{E(Q6c<#KgVadmcjPf$kGijns>s=|ZJj`n58cvIHu$n>aqoi4u6Y|c+C zk1h<9{qxx+`JbybfkloA*T3^!c&`}AM$mu>7%GZ8hG3lFIbkOHG#N1_%~(JeumpEI z1GnFPQuSHu%3WsO-UQT+mfH_a>w3x40=x&_LJ;2Hg};L@ntEo4hNR;6<{$6wQFl8h zl{(7xi3xgre8TK2Uteaba%|=;yPIMcmgdIbUA~xv5=*|RExa^0x74+o6TqZ5;d7iU zT}iVOw$8_wqIb|x0<-j~IXS6%lIT`1RgxP&$m#1sB=~P_Zo-@LB04<#gYzk}-Z!1Dz5h!&B&~jRGMrKlE zQf5Xnr$H{wNKVSkOiIq6aid(IIQ$J!$~;D#ihAbm8YWQkxT}J3SM1ZMxVWh388cif z+$T4=kAVp>4&vhP{1Da0kQurO(&9&6uwJ2%zj_?qELgKtlN+nm#wNrkYGYq#=ME3M zHoUEkjnyW{#&S>n7-AWlSUdgvMG}yLpcPuK&YcHunKi`QZM4pgZS%{k}1`MC}%1@GMH*YoHE*kfIEn#r4N09&=ieS3^l&W z86B;rO8*~uZvtOcasCgVnK}2~>yrBQqUgweoQ^LKV}HD?b>cXq>V;Yiqu+he{*3*7F8srveqq6VG&?kUJ1m4j6Husm=o%i5(TaLB zBv>%oTl{PgD@6fsEhJv=*KS?TR?62{)lXpR&~Mp6@@1ewb-NimgLw6SKc;b*Xh{$0}E_(ymbAc75lp55$Wyxxrb(- zKR*%}^dkR0J%4=ZR5TgZle0_pC7xc|9$Z$t^wJr(ZCesK|NPm9=AMtt@!k7Y9J>A` z!oL-@6R4d&BhP{oZQX_rF?cokw1-<{oDa-b9k6iN@2&dhe_<{og(w#;!I$jx;JmMV8&sjy;uy07>+ zEE3DYjN;)HLoZ=^O>LuILcR}++H4(*ZduM8%WPRr_Dqy5F(_$9^_cP^w9><UN7Tx8B~}NJh~yEZ{sCGK!dg z@7y`Kb0=HRHfq;u*Afh*cver5CR4EFUAykxwac3XJl2UcBz>kM4lGtrpHgax^RC6y z!}93@^C{r%M!f5z2=oO0qn*Ru2;7~n@3$Inz^RKQC%IWV5AvnN-l9wi7z*Aj*55<} z2JQg@B@TyBHFOJ48v01`I?`3K_d1%Jk z7xbap1?C|)X|^=^&4P}+(0Sx}zdk>4fL6vij)D5^&)A zatUkC(U&`q9Lb?lsmwj>e%baCPX=Fbc3YbKW*frVL$y!w#qK}W)^?0OfcG`hFVeY8 zPtCYYvIl>Fr}$p)Htvwr_APLTjV(!9j``c-t=(=wqfUM%qaBM1K@}V!6&l zG6Ng9TXDc%^i0~rz^E&*jZ;QzE)Y16ulwlZ&L`0g`;#p1s}S60YJb-L{5iU+8^1sy z?ew9avL*1RFMA&{VeMs+VegaFn-}EGc1Pshn9|4p{f%Vf*#w1!SNp`w-e6H#MKb6?WC#jb4 z3GyB_AndR{bSBL=GqX_xFq17lo?YJOnZloazVpdR&PN?X?xD}it3TE8F;YXam;2I_ z*r0sN`UWG#zw~T}iFkh5SvJd-U_iHMh6 zJw_|z;o0L8vmFyECKfbgw`Qe|%g&fE`R$B!w?;4JBxgv5TbHZ)u|UdbT1kZ_jS+-wb`bedm+8b=^jQwc{ms1)@N8`x`=ref;<~UW zVDuRp(yj!&eMAF3&+|J-H!DAKOQa&SsTdhc@tm}bY|=BR#r0MnP0zqoOJXrUQ@em& zk)h3HPmHQ_c-%m1gyP}(oj$|>pI0&4D8OtZn@{odOMIq02-!CH6FeWMKOcAm&*}Yr z`ukhx{U*F`;PdhP$WWdBe&7aNp%;2Eu?OUfkOGD~GUvTAXC39lV&*{vH2(z88$9<8 z{RL9~Ihp*7DCgibB_DH>HS}g+ zJ$4ESLIJKe{@jqv-jxy$<=sP7LyhfoE{C!wfp;@_fC&uy?5;5lPLPGMK=bAa~>XiQ`;LI+B*}C znG`!<-kp3VplU5j2YX%4)gpDRRiGE2Yz7#QcW$0Fc$ALd_x>Cd<7t>M8FQ2WTxX4%4bU=B$>eY5#g zqlKqZ3v16}b~HtDO0DO8w2PJ3K0^`^T!3}on`rK)_XWK_&`LLbw83upth;*+B8|Du zY&(-}(H5=YQJ%-U54!ieA9w&?2LWYZ|IkZ-@(?JYz&y$5Re_KQn&sw zHh1tUam(U)i}|^pC3E@LgYV82n+7`;J9js3$G_di1(;tC{+T2(uhTF}0rM#jP>tyM z?11OhFVXXM)D*thL4i8SfmUAks-mdcB0N8e=T=Dj@wC5(u$dU^?3rks@4;ugW5=pI zhfKQ$-#7_Fy_gXa7v?_UcAq%O6mn>2U@vB8;W#;%%^L@{p*Fap1^1&?jekN@c%D^i zvBP^IW^YYWyZNE>H;aGQ(pEIh40mKk%Guo3l&vFXb9m;R%{4bQ(`+ut8WR@|6~V7t z)jGTMxt%q{HgWz$@7!+tcADFXP|WSWIA=LKWNM3UJ_oESI*G`Z270kRRZD$f8-1Xpe}1T;HO+m^w8rZvgmG%*x& z{iaJK7Eq5FQYn>TX;+PeKb!aO6xNwV+{Px8W|(- zaD+$DmzQH$9t9tBCZ>8`fS@=J{g2klipkdGWYW?yto4V_|5=`&2qkek_*1`CfZ;0w z?0!?EYOk4McE<=r72pT?@MGXd>sSS}K3yIIb&%GwplV7ZCoBqQeg|&HCdaRfU(uF0 z&SII6kdmB$OQI+96zC0K4YQ0yO}5&A+#fDgDwOFX`H(#Pc{dJiP#}%PFS5zTjwRMl zygesiRc%UwiJsxX6 z7JRgX0D7J<`n><5kp7JN{oxZCC)gOjl_B5i5ZwVw`opL)aJ7No6xe&#gUZdy+>toa z41w@8ul~I0C{L5Pnpaw@ipQWzIGfjj-v!{#A`IlcH~zKIzA)YzH?lq~B18Lv6;joF zw`Oa|%HYdb1v7pcm84%v(6<#vJ*j^4l|{;FICsBNS@Yd>{XTU*RozV*eA^jFgRv$J zM%)At<71N;!)|n{jLpzKW|fjEqumaph$KuPk?4^hq39=ZEu%Nus!1OE=Ftgl9dHaqM_Buuxry?m>d&;(wfl^c|ttfbM3jO+)au8TtN zq+apT@88RQ1paTmqKgD0V|oZ*uk?- zQmopWSDUUcbUK&T^w*DqLQ>o2x`W=;&x5nhTM$K9K~MO%xrWJSKs%z@MB~s)I7ce9 zXpCQnJk{6k@PnLi?NZ7raCk7q8J;bd4!(jFznRV}kp0?bD7o+HMFA_|A@IVH^Cm88d?S zeJh4YWB0tTPR5cUax2@1uE4r>0W@gEBCtur7(07#a5Xw!N-w+q9CodiuhB`IUZi_V zPZy7d!Cp=;OVK`FJvi9QUS_Xq8_vudHgGGqtpIDpv7YfrAw@ z8zFlkD3(o@zSI%0SR=g_>Ia`ty0o_*Ta|H=&=b4G)WO3$45yv}VKN|MFe;vjv!>@_ z?+W^Frf%MnZOhh;Zn2LOPX)$f$41uN7N1F{Fbq3$+E-woFN!Nq8DAdXI(2?j^q8oH zHH+iRQfdkpRL_r$ijHj6hBBw5r%%buta3Q2&Ri4`85z+!bwPaP_|%HANE;C`9x2C8 z9zVWfY-_EfDl3c9(Kl+|1R6w%)!3!UCB4W3<8zAU7ay&MA*B5;Y;3^DHQTO-p0L?5 zwE}hp9PUVfT|A3fww||5DI3>3Z{>oj`5mmSC_iV0yK`|-SwUV{RbEkT)$WUfIu{mA zwJ~cDViLO723>M-(At)QDYkFI5tFc`EBNBqf~pD;vQVAL!fj=_CDVh^cVKh+pb-y( z;!wa+nfAwM;}9EU{t-BAU}f6GmuqYCs2U0K;ZY&JW(Qs~0z*t;dCu--6U@L6E69;3 z7Pnt%oSH(;=u>d)|50=cro_;$E95k+a^#L6_OgrR$I!{VOo9=@P9JHXTV5Q&SuITW z=HU+t)YN4awl-@+c~#?G_-m*xZ?Lx6DwdG}UyiH8l>;@aU{Q5hbyJP}S<{63MJN(G z0kCo;y(&pj>{9WgMJu8J$|Byxg{|+hq3D&sZ@rgw(?asQh~`<=UbpkZ4|i77*H`S^>uzqH2FS}}5Ma4A0h8O{^@56gLHmr+jZ>wYyqAcjjt2dM-!adU#C7K6K)I8Sd zh)qt8B?lm9l(6M><(0Mc+fh6g(WFb*+K!mgiiF&nlER9tiuw8ZtyRvdiqi1~dCvO6 z+!>`2ZdODqo`AsATV+nHd?e)eEt>?!7QS=c~5imwoxOQ{aHDIUEe+WuCLM$jCDI_ zX}w4*qQ*aaj;rlho9j@Es2Y5RJ}p$cA$%=C?Xagr>4w_K(g8BH4bNqa0S=8D=^RQU zc26#+l^XOELy}>BEgco%O0uLFlnoq}heEuDw!hL)6tyMXg>G>)0);jnPR-fL{su54`lO(R*#Q==HRnb@6(#%dA$ ziw57&A~6g!5414e^4AvKCdD=M3}Tgsp@cN)nH8fVF;1~F1*_(!1IYAALbAN=q;Iyd z2{qaqXEjx3mLQ5#Z2Z{P=~MLExP;bGpH_*c!C$j|jZ>y4W@jfRjcaVIY%=b?8toyt z+yPuGC9YWEryXV-6q5d*sD>o7aV4V!;&O+M(RHjSrzTUY#}ac*j`lJus##lu|FZBM ze#AD&PyKX6DcDOmX?yCWw2a=UdMTr|tbZwV&IENCRtjxugI^5z4QnbZ*EA5*$Uyf+ z67r$4m9D(H6$pG#`@he41m(j>i?J5NEG!~! zDZb*0;w!H%zWU0ND}GdR#g)ZZTwQYY6(v`K2RpE@`6aX#veZEv(@+v<^Afm>-NCLp z=AAi93l{lB!Bmf#Q)IT)yaRn&jrDu#Pu>|vD@+kuut|s&V<@&LyA@Ovl8Um|QR~nu zG-+*md#uoNNP5AUfXoIuwJI*0;+;?4J<7eBELP%Mbrn`$OAlBbC+hm;%FHayv|Ave zG3;BoAgr__`Z7E#bG^~=hIiRd)`iXtzXE)tNamBiZfei6OsbUMVx!yrq&GL#<%wJL zQ^$S~O8v_>$l^l)5CTw;oI&GGC3%N{_UOkO#rDQWC3ig9c%0bdIQc|$DV<1-(?NMveG5$U z=zKk$7mx`1g5b$>E4rI^T4KhAqg09@r*stwra;i;?w-;#tGc)`T3C{cGo~~(Po;Y` z#hH|xVGm*BqEB0j^=eCAP07@fbREgika(@sk37GV+EL(`Y#fQA zqiEfYJgBj)?~tf!Y0*bzltt&CNHavi@um9tZARff&C}bL;qdmyZ2O@WwwayP;?|y6 zqdiWxG^ByeKt6D5nt<*FN4d7*tTNC?u&@g@KCm*#%7U3Kc>jXe(5tm`(~CnxOVW4+ zjg&s0Q^{mmscWRft8ww~L30tth#x*J@tR~J7V>9{4?toH`L=aTy)*cUUk2w0KMnpHy|m&GU3JXudo-=yRiBpbN>5X)L*D?e#Rguva^)71%mK_WdDDBDEH&PX zkDTP0hsrPq42*I>>*fA<`@;xgS5#@eEGjW6DKTN(ID1iLWl>%Z=M!_u)H!zo=M%Ok z(qB@Nb{_&lmQx4;gubCas#HNrQd@E|zLHW(rWCgoPsz30bGXdd{Tz&X_{?wWGt`(ZPy38$0oz=4^DDER|_;C%{4T zCjXrP|NAu5Sn@8|)&DtrDPz*441Ba3j2o6V;>qdx85#NX(QY>H%qv@M-hszRpReTi zf_tEg$nhyqj{=jBhU4Tx+RMZBo0gWoVakm@e<}@tN7|okZ-3n;g$H@Q8Lrs?-Y~U> z!}jwSY(Ec6Y6Vz-`rD*?EkR2i`S}ifj+qvrIr1kS(F%)8ks}yZG%%*2%^b=$)O|5d z)eN5v7`0}>!>hb)ScN4N@0Dz;k4T}f^t|% zkuzD{|B3%7L2Apccno}&ho5u$lh3gT9E*-FX*0ujvu9?_A_~DWh|iJdjxevn zxd{i^-e+3u6zYhxbHZ&2 z;dgX?{E_3u@B~{h6K=tSt%;FG$udgEui1+qnN7!&ffXxf(DG;<3RBSCPbcJsvfz{m zZC^E88If$WajUy|jsTvC;jE)ZyE!5)l!q52Jg2owi0J8aAiu$1mdsM};)Y12`iWdy zF%F^FS+4f(qj5=U2vgexCqjmJiAEj7pd`V{SDFE9ovbZ(C2=GK;VPKd(1^pU6ul7@V#i^3DDvEF+|&#O)M?; zWXqdK-{^<)T3@T{^#Cv2I7%)2IKf~Jvs%QrLksyFaS82y__yL{y#cMFw`-pEj?u10 z|43))M*q;;`Jk_z&l=fZE;k$AN?MTgQ>L#ikcC(xxc|H2XdZ4> z&A=tkuiYq{K`R}m{>}$ezLYc`RzE))8|9&o5)aAR>*#p8h$FbpWwIt%H9?vLZY0uy zc)H9|MnD3m9-&(6kW3yKBzZtMQq8p^#LoOWejV5WCw_h0VAjGwbr>G4mgrCJmtwJFU$wHQ_cW5`-w59O3S@7S}UyoQ_y~ z(Ep!!#*?Z%?^l_B|C_hsDYMk7s%`(Px8xlV{QJMpN$)CuQ$A8YRX$h#4KJ1h$`BI> z^AU>37BNuX#=B3LxAvORJ+#w}n=1cKYbKWOjEvv^-q-oP(2eq_aO$K3 zv3?!&gck#O!`24=)_<}0MtiD1|f|xGbm5Y>}%4NzG%GJvC%1z3x%I%0Cc%Slf<6@>BlI4{ePl|hm3zL z&%8PCU)`A5sh>)%9Qy|=_-!;xl>6C+Edgz5ATb z*uNW|6Dw;WV|vT8@( zgqdgP*+SS_U((D?$ecHU?U2dtGhloEWE8;@kT0QHTPjO>!3Q@}QN1;dgif7HcIm?@ zIGh`3H_cp?4qSuv<2QdUnOQbtMj3q^lO{RR3km{A9>6s_^>k%3CKaUXiG4i2k7lKr z!JMWdG)Hjai`-C{yiD&g??DPndA7|=r5*YBrq6m+A!RIY1cAiWz;w}Z8~ev9-FQ` zUKLT6{Mh)%lglD-^Z5A3QpoF;qEyK^oxrygGan3od@-4Ea=$mgl-^}xa)p;~(gvSQ zg)KWk*n>6i!`LAJgmR?X!}5;Nyb%z5ZvcVbk(9*y{=Pn$#3@hwn;1+NnOH+sp2BJD zQwWgIT zv*yR0CScF|e46KRnn5EM$u(e;%@`*7tPBr4x1lIN)MXytl}0nZ!b#ItnTMWVM|md$o{!k?|0c_Hvfs-3nx!Hg z%C%%>0-;yfQ(CP=;BSBXo0k^}2c#k_VhlvIBY)E>0YQr-XtLMo7|C=@L|b&UoB&6) zmBs6|J;qmn7i4WMd!FMb7cLCUTkOqi za2*e#lFDOZ_Uzen3^VZ~v~NOOcNAq^DoUS0@2SN63gqU6g{sUa1AAk$s85 zy}Lvytrrk(Pa!?vO74{<06UZc3q=(3kDo1+CFMM1rlGPV1@i!LG87JEJFK&Ur1>?JP799;kk~#($RrdM1FSI$WOV+j!VzI495mv z*R#Hc#Dhm=Hd#NvGL;y4KA7x~9q8q+Ygcy25=BV|EV-#P{YzLQp?B)}Vp&gzHt}G`+a3Jz!S~GcXaTb{vKm5a*tOuh*uJL&W{F>P z9DEzb=+ip-WI@#OL@9G^icxxefRpwb{?m$}ToqLH7BTRkk#|^|palJ&o%yxR!H$j& zv{OLyP`&iR^>#LDCutKS+lU}xR)+6AXyFc7O8}wfC1lWQk~V^H52UQlCB@M(go}xZ z9}aqUqjaX%i5(!y%97F6^l#5WjnB4)m2kn7j)qF=m(;^u5h5eA_JBEcYs;LC@67?R%RYkO@46-uGx3FJ44B zq>1Qo%2Xu#WlE)wew&a8uReBwiXw~vNq=ubA}BJA6Ra;&5gv}EG7wu`${j%eJfx;{M`Vd~-pzkvFx?WG}u}A}!r4)mj@%}0kyEMrk4odTO zAJwz65AkEhNJvUQes_5xQE9~b+G~PSQN!vfgVIqNXW!eD2s@XbdEWHlBsomhQ!kUE z$jVZU)KsoaE~So?2fD2(NgbqR_%>^Mr4&{{;Hqz=pfYO>NhMnkIq09PIYgVRJyJtT z0c%++Z%RAa#2PgtF^JlgASDD7teLTC9FF=+xkT$5z}(KZ?$M_1VG;5G5m)+#HnOWk zr=EuVbPM7d{9VoVY3+O|yH#6h@B}07gT6q$s*iZ+6|mE=R)u6HelmDBpiQqbTY{vw z!Mn0a&6Y6V`#4f>C@KRrp_WAHBwrz0sREK)SOg@E?+xqmdn5CpPf;HF2U$rpLJDUG zHs4hx0dQ+%ZfYx%$_ACpRb}>8hv&@>qy{?^Rp<}-`i81OwRTs?FeT+)O)<`$ZGlH9{A<0G5P=EtcVZKHQYI zluo9UGS#nKr@u5r`1cutpIdd~!!GO3T&ND1Bqt#s}rp()uDvgBUk&|x(vnIO~Z zl0l|6YcEBzk>13C)|&MGh@nCsOq`}fGg^XhrKKgnMQac}S%i~h_M3gfOE**b(OpV+ zAKhi!N_v?x623MKDfzZnI-_R(lKuev0Yavb-8o7iyYuM&%FZT-WA&c-P)g3jf!~e= zH?7ewp}vFK(#WOfkVX}l&m(G7|NI+PIULj1RZ@;MH!bMENny$XuC8Uvz&Xev==6y< zG)%0csk5?U{+`vWmJ+kI+KtFZWuQk8k0J+bx(r|KaWLQDGgjR&zoT;9bO);iLX?AW zldS~rR9#{dNe~-ZQWnD+*3OvSAw(>NYjB@9;nx1m!g1Z7!`@@>DBV zLt6n)>RgU7jfPFBE6Ms8`DI@>N3$N8g2oMRTk7u#hCVv$YKUw(+17qCA61Z$sZDjv zV2b+6V$KuF_gw!3=xpA5aJjyNaoi<=2N}CTjooeNxG?^TQ%r*3Ykwdq`y&R zK6uKYr6Kbr#w7D=A@faaylqC~k5}d!SODXG$oyqKnID6eGiAPBI%b?L+WmtsvU%DA z>=rghdr0SK*#``{+TVdPrOYQe=#!OF21L`)NG)xsi$*)3KY*sDtcsQD9f&S>mxN`j}1w z5b&f9YnBcg=;?-yy#8Yk(i-$KQ=?})X<*T{YGZIQMgZ`D0o}ks!jF{`MuWfnd5mVL za?I3u6&eVo^d%(!qI(ROL|K9+A?`u4gF3@`;rZ9h9IxbfZQ>fhR|Zts3v`%LJM!xT zMqiTU>hS#LB@z}XK}K&O9ZdEXf<=0i(OJH0RzkXz!Jp8kbm|$sO4p{SYok+*XgQs4 zXzcdR%>uM0gBwQnazFjgzlPu}Nm~ieJGQ@xDVnSyv~shClunlza&AWbUCu!Dwv3Tb zWUn&%l;FM|rEAMNBuZ&W4nt)$*Iw{m&b3VWB-dYLHH{@0TE3ObFoYYt+YxSPC#<8; zKOH|BC$NeFcKUMgJ(5Yr3JTKc>nEh6k=&w$SUmwm3`agX#}L06JWq9k=A*9+NM_dO z{5nTT4m1`N^zm!3gADKORX}R+@^HjM{W<-++~Z%7M5*H?I$HY zHK1&5{UYTOO|K=F$hotmnZ(D$-{2D|p=DmQi=HPCGnrnFD0*22zfc*L9LX}$*4o(B z`JdY}K`a>~LYV`ndAzY%XrB?*z5i43qi8ou!5aPw^qxrg50-ZMr6uq_WRG=ZOYfl; zxkAtKa2O>mEiBR*sXgUgEOe_mN->!h}^dE>`Th_lE1Z2Gb@x2&w0xjell?%MI=3qlsp z-d;K*GGt;?MqyL-#95`u6XRkt6B8;E|9Qc4w=ZgU=jXg&$w}LuLpZs>Pv62aK7>54 z#zz~JqQMG>w_7_qTitE$L+q1fv)r?n4)%yE2YYD#F?(n=4@FI?C_*XGJexw)=tuF5 zm@MUpe`R`3m#zm;G1LJbpDB!j!!E-WI%4AF4ma}Xd$}yRY)M7X^5CfxQ!1*8ou@Y} zm{~rqGN*G{%3K5uv{huM6ju~aYMb6Nqbj*3w|hloN`69~VWkrj{Hm;_0<|4N%IU_GC6;C4Bl+S;Dk zRc%gZTb2BFO>StIJb9WcEs+8rC8oKibysyY?P=<&GA_=7g%x`$S_=zXaamZhYuWbjKQ>ydudZ8$3;M^DlN zz+!X~gSLCIh(u&YxC=yZq^)0aGTz&PGh4gd>6T#5ZI-k-Xf2$52pCL*jjaE!@b znbV1_q_wLfawG76wRYoQVkTsU#TEyzSsUz(4a=Mm^B0u92kZEU(W)b&BcL^@=8`cH zG-&?(`e`jK4HKNs2{}bYaEjz|x#?3}GBLNLBzIy70ocq0zW28Rw8)|E7LoLqJF2PPGz5&4vc^U)=kd*PH zk6260 zQMI|P4y@~Fi!vv~CWTM57o<_SCWaWF;WGzw3Lloh)DBD~uE4e*4#wr$$vf_T>W6gl zlukR*DN0yEfKy@+9RMymz%&kl!=pGl8jV#Ov*)2I)RsnLLd>Kr-(ro5wsKYFjmvv1 z(a|x`OqT8Qhc*&oi?;&N@Cd$u#^1AAVc-Se3}uexYb zV>XqB-a}`Tt>`=Y*^;U!JD%Kl@rA7C(4n)=0?*6?y(s#UO4@F5ba^}(G-TW6z4exM z9&5SpK6ct4|ES&NWYKrsh1B|4vMgPf)=&28E81-1;XB_weCN@lbewmhn~v%$h@}&W zacc&k#~7C0^@g`LtYUaOki)b@p;p1YU3MxGEs%qdHqp?*?eN3z7gNQEWu&JEKl-A1 z+L6|VJnYTbv14On#@1ICIjgI+GmEOLi(u;`h(u3IedP>BWQlcs+WkWgj z^`B8Pnt^}{UHp$aM_SV1h71Rf%E}PI00*|nZVHtpaRTvGUKS^Zxd;v~O%yQJ-4W6@#gxtw6wVR)YOo|MLl({p4>QF zbW~w{QgdGQ^0Kn!)p^ZH@r6;*wz!GCuDYH@g`YYS*m(&KM}l@q0u(~*K1(dS&{htm zh`5vRq@Z{--U1&TM^nr2AHC3rbo>Yl!jrC5D=I2hR8_TCR<>99V@72YhICy%rK7sK zV~YOW=tww3_~6S+GaZ}n#O0KwOM0BNRFXDd!EfeDzJ^vMppgVn-N4)eVtpxz6$b1W zPDcNSujY$LyT!Re3_mmgQOn_qh$LAZ1qH0IFgrKJGHFu5j>{_JvJ1DIx23fJUxnFm z6_@XrR4~bslAB#v$O=euS1BLLek!Pk(VmY*@2%t+qQdC0QH)RQ3L(MCNfS&3t#!=qi!9C;TJZ&2n367t~a1TyC(1D!vFkpC}H`}?;F`F_-+ zq>h6vYK1OssFy;gp&qNdRg1w2gn;J+auGe+^p4>)xm|mzW$DtE$cTw~1-YQnV>p(Z zd~?>ihqUKLO!8Q3GJA_=cGFrKCuC#pmY>7sy0uTH%?x1=jGXDgQb>9m)9yiRj01+J zMU1GlE~4Gj#_#kjZ58#}C2s99NS=XXINwXrH^yKFffcPZXy9~gTaUtFA2`-#In9V^ zfk+%S(SMhN&ly5=3_X|yB3leW{>g|76n#&|QnS�`t1>ioQcDTIG6x2X6$P-d@w| zba%HIfk(RlaTX}p&!|Cor1Cs0=7SBZLA2^eB~-za*Pq!%p0j+x8lY{%7v|rfhbIH) zEb-zq*lrUqO0V>M@Brx6Bk2bH+G0?SAZaaa{P&(p&o^x*0YyC`jEKSKhOSYLOO0Nx z*pZC|rKiuXwt)}Z*rmFTPpIm;zS{r%&`)7K_!%@_8^$eoag!zKx!|Iuln&>kE!g1J zHWVm$8=vp_IeQ{5J0v7KPOAq7gHPxmSgdK+u7%SSyK0FKXJ|-Y9i-)gQ3t!eHs-B| zUtI>_MSnJajJiq zJ4)>xe|MCrvobk6E3oXC)SzH=)9|d)gzt1td5JnURy#r-UPBN=A|#jv#m4eexT<7Y zs4&B$mMNqZN5sMs}dRJK^_mcGi!TiMf3LrM%G=Y7Zt{lq^f z_CRRnDRUEq!6$VHCJ-MS9TH%1L`29(e1SEXy^i%kBPao0xwIGl^{>VM`WJKgd*;gj z%Ka7o#|F6PvJULW%3+X?M@dOy7Em>!y`$Jv1a?UH)Sr`?Lt;u89m4L2ca zgFg!hQ|K!ir6XkR@R)^IWPV zfwwRFz<%r6Z*SB;(mCQejg3#kIU?HJjC}-dDVGs7%-&a;MqTpp_5J5t4tu*^jb#UQ zNXIU>+3(!4BLb2)0@Fz$4ZbecnUDr=5ED-V$rq1FMoG;c1*>9HMo>_u^#cdhJto|N z_3A=`8xWdlIJTb$L}!xaVg2Mxv?-*KjJKp0DuXSVL6>^g4cw|`*3=BVrFN{7A(!qq zLoe-uXHxerin`<%eQ97fi1OvhBc33(iDv~rwr21-u>lcJ`0bwDHSn@|Rba#uSoS;M z=|#rP%JkMc9zx!xg?YnI_3>vtQ=WI2^+F^mwCuyim=AA5{y77Ne%rvq5=e6FqwD(A z+f|#4YbAAnfn^53l*CU1RRJ0ML|@&?=FbDeNJJHBkugy&LmZh4(Cg`W2}H7$S7UvL zYcCHyg3;!2jDc{spi2M}tTHSQT>Qc1;wRDLZMN~zS{!XC@D+R7?!`eOIO(H zt)}gG8^C$%L=j_^Oho5GNH(d*z%t!8aKsNh{OPCb)~WlA!DIQl`_wzU9J0K|!LnG+ zh!KS1;7n9%*&BR5#ld-vJZC?TIo6O-F111sjVyY(Xn`~!5F-ZPMPYs7i@}KJwe`3~ z_aOLa1iI)xZ*V zcrGyS;0`0N<*)sTmJq8iZ?h%bX5>Y@qBF@C9qg8{BA6EMAQ!#X_&t%OmX+Gkb}v*{1QbxECy|Uy#}Q_a2d`qMb1Iqg`Z9*>k-)%h6pO*=a^j z_4emGKl*50-@qB(oNsD>!kLaCsKq9y9`WN;%eVEA@&fxMB;jMgWf)^j`VsL+nnLHa z!!J3O)B=5R_D>|@#Kfy=g8fl#2&2kX>Tmm(1W0?G*74&k>UeZ3k{iv#MLzd@hS)sQ z_=S3G9_*6+;MhF=ZY!_(FGKTaPX)T${J#;LC(ui#7&wGp_9g09iF1Bf5=U%B3_3q3 zdMuph_#A^;{QZ1tGn!_`Oo{u!k%+XH1FOs5_g6_!`K0nOaHsTx0~7hzy)8ha;o6PR z!5d6{H6F78tfEGfn<;SGr=Fkl`E6}ly*?LcWgBqbWAKT!e3SNBc)UfmqhpJ~t={>C z)Z*owBoNLvOe}_hqA9>A;MD%}0d1H6Y{fgr351zXla*1yo&+?slMgWuI}9JPVS21@ z*gYCkpHgV)0B<+y&8W<)*%Pre1&Y<``TW7HguhlFlNB186~mtBZd0_`1Lab-Og`l9^LJ{7~)S1)5lU>SjuX|xiD%b7s!%dCYRPP;5Pd}d)qq;Y9+(J4Ve zDbef`sec{sIIgQ;(7?3k{<`?D(1M^B7-KtWak7PsA)aE3+T>`xUL9y_8?dZDXsPUf zwpvnq*DlXPn5pldYmW809%Mj(0z~SpK4P!8?C76bJ=OBN{`1%CdEIm8xpyfX9HOsa z>H3o_+tB>tvk_u9%_f;(y;ak7zHj;ZgSPFix^FHvaWK{k=FXd|*F>&|ofMi;m%9xO zvJLggbk+60HPxumaA<}WLUV43o2iFnBV@3bGW*wce*E#ebp!3|532hylUVM>4O$5( zJinzcdF-85iraK&3*&DMaRx2YeP_3EU7YierM4Fkv4OJINh zyI%u)!69-rzcX!IirqdwC8Ibi zvlvP2W0Dh-9Ah7m2y1_~j~{O*&-~^md0b9T&^hA$7 zKu-?%OS56JMe+}|#&Tu7Q9l>~1T==(DCYGfjpJ=3;viiIiXVI(l}bvnCnRJQ(N%9Y z|NHOVgp{=?%a^t296x1UlA+nG{b<)fiVRP2Sk9mftsXq8O@y{X(WBsrKE9m!>8kKO zm6PjvgLH`KB%3WMn!U)9mTR|mpULbpj1&58Sn!AZ1lfwPfv*7PVZebWc*`(2f#IP% z{~igonosc!<2a)7pAm)JsU7zX(Y4TBy>WWI7~$yY})Ibg|T zT6cJj_984g7XIpdQ2pOBEEJrU_~7YCC*L_mj!$zypys|NPe zVLfyC4-zyV>XbXRC?THSV^D1cs1}nQ0kY}RdE^xgG8zf}NQ#I8M9t5}7!;GzLhtIj zc<6a7gMGkmX3MlM+{a^6Y_^ox z<0r83M|NsAv#tCOSQYl!sohL|TCgs17h*GSRc^rQ8+=HdDy@8Xa(lOOc9ZnqBFLH> z*oriY3^3EZV~2JtzSvDvo>As*ejBuwzjLb!9~q}ClPlZ#Wr*7rELZUr|5bbHHM%hk z>>&0gI~>XQ3>a5_S$cAc!;zAlZeFRJ4*;uI!U}){yXCB`G<BAZd$5UUc{TNB+ z%#k?m#)0XhfP(cdq8oyvkw})7s7;Hf7v@YTn!cD{=yt(mCEEod3$O0{40b?F8%gUx z+sLxIx-$BZ?I}L|jGE%&8v2x`B_*Y&Cnco;!`=LHlpaR=L+ppyukg#gb}QZ+qKd&a zl8eFqyKxZaHa!O5G0^KOQ9YsYlRwz$OOHEEpOY1Ms&o8G>@wm9evE$<7i15mjLs9I~q1y?2Vv;A9cXpn*`f7Uf z;!r8O8D6U)Ti`(zxWo1pDRfKh7aQZol{+03_Qc6qb!9J<$A?EJF0U4`8Hwo>OYt|z zdlBrKpt0!qA)_>VgV_VZe%5CdG>4|Aq$E_mP@0omnHHRsZ;wkN&uE^@S-O(Kr0-VHZ#&46mt*XoTEhQSPx9a3? zle$)Qru=PE%&Ioa-$uJY0%D~G+_+s7LCXDNWY0ULloLfJ+n85s*$+e^q}+qVYyR@m zE9FF?6qOK{l==}d5mN3E$={}w6B7+7hj;t*cczq+uOQ{bL`XTxHL8@OR3zn;t$=VG z=|)I0N})?KUy5Op3`tB$CNd#Q9yz5X6PbaMOoJr*>qvTz+K0T7OdC~_i2{`Vi~mBB zi2|9!^n2`;wB6TMH6xgMJ5@!(|`ahj{&aXH9|_ht6bZA z#o)1HfW>^W~I9;SMYIvStwvM||yRwMn7q~z9l-9vp-waW%BoHXhrJ7R&}}jZT4KNdWQV%?V&2{ECJv58#*^hI-~Dd)s^!1A+Gl< ztBOP1^1a?Wtm-P`x6a#Eb+y#(Oi4_L8 zI3=^Jkp|tyrMr92*v2nxszmUy@s&;6cI<#uKUHp$wZ7b)+s^16J@*|bJ7r|qa=fP1 z6FNWzJ;w*?y!_yZ@N!pfad42s4`g*%cxZT4c4~M+sFOyV-Oy`}iv56&lOV%tHd0?R z%#HJ!7Q1l%rPQ_9IzDr3Tr|v$OD3jf#$!3yRl;|o_BlDI{XH=WF_Do8u{njfNuev1 zxDA@|nSxps7SW{h8Wz#)=H8BN+69*{`0*9`Twrh({ zk|Rsp0nG&RL+XM!%jBh!nf;#p;OIza`J}R-V25_838p$CBrLot#}R4|cj_oeoH2gw zhn_@-SWZS46K?lNqG<2w;6>gDn%pC+$(3CqjQs(pU7LVMfN%njpBggW-iUea;KauH_c2dDrD z;|T{p9oibnhoPet*g|HsCng64rIz`D6`Ljp1qD?l#D^z^=5(0g17PQl#IAfKO^ua> z0Gvz5PfQs{cJ4(JQYR!Q65|z!u&S!mq$DgJ?T(0vh>Eb;vhr~g9ZomIJtB;+k#rr# zZ#QN($dRs$dTmD;m2=JZ0qRNPB&VTsPCU63ppamUNd0sL6dad;-4KT42rC4 zP0b1l$}qW2Y$^x}jjBvf3Xcy;@9={L9e5BjFMU*QGbm%iJ$n?QXs>U+aPz9U^TdrK zX|lV!dj`}^MB{@4w#DKIa85*PQv@k&s+X2J%H!b-0*FJGmo7%y@-nYZNbWv(usbJp zZ$c8R&AwpU^OlV38j~M$@ZiBinayK+V@i^>EBQHmnP&iyJBRAn=a83UNoHYp7EuD@ zi=A}LjLvFd@j`~DBKIZ5_Pdr|6lS-Cu%Mvk(=QB*w}x0kce(%aB>O~rD`|Rsp=%tA zK_L4~+>e`=s{NDC!JBrd&q8q`zIK*a59}wHr)TM04F#CS(j(54+Kn?dP=<6XOIk?B zc~f|@){ppg)5(vJ!g|=P54Sd zmUX<_b6sFMK)~@va@PtVIA7G|2OThi;WUOoq_Dqk(lT%((|E4&?7>J&{#Tb#di*8O ztWLkj11o;X=ZxV@*jVG9PJSgB_X7JzS;oBzjXu}7w;(V4fe=V$Mc&EAy$vVJs*L*} zNZJPDJ{W7yryKVnDCunDJ`^o=wQ)~-`ad=9!;$wH<30k|yk*?OMhsCI^m3zsn+I*q z)L@|TIOE1WQvtja{h?9%UT3VEm1 z?}L;v{A=SrSeYqLc2fFK#VJ~h=V5;L;VfN_PLw7>DO2w@(j+Ju>f8D~DiMaJlkh?R zAPaflMS9dD))H&nV??y%8275LF3f_wPUE>1d8ZopHf4;Z(YOy%CRr96_rXe~WhtXPHwqr*jgHGGFHVA$7vmY5+q*hf%K9qxh+4A=9NmM60Be-B z%4Q`MeWOr*Qem027O?_Sm6`b7jQzVFoLcHcbdz2@n~j{^_@rVdqX*Y@NV8JrcOkC8 zX5`$6=ae!>!rhE=dSUNWfPZTMg;H!$R=~=9B~os{(~+f&_RK7I!YJu|ShWC~HGtm* zjC)X+x)(vKrqb%SuJANk}sZDJG5b-i@~C1g$#JMwK(8 z@~i*|RIlM}Rs?M6qjUdg<(k;EqrTK1h{sk)Dx?BqowHN%jLLQ4+%ny72Ir@apiVDp zLp^U3-ct(^=XBvpJ!cJi#9Tm_^Im`Q-bQoiC*eao6`t?JLEH&G*asL(_j({~t-md`Lqd zUA`YyKa5ts+MD@ipq1J|f9esw7KN@we?zMvdkkGkJ}|!2y|~Fxo@X!#WGb^TGB6KP zomez{$y6cDj^e`$}4P&@&{JMs@YV; zuc}qnFqiTwt3xEKX{>=YBA!(fo55zXX60%&3o+LoREm{C9I@vmmBUQ?$Y`yXr>tY*NH{xb(QmWaR*lF%jrYW6>qP3ZAVP_$h);4ywvYwrT zm|Ewt?d*J|i(R0sV;8cE*bel74a$CYG5S%XqA^9;$Sz?!VHf-fyA--rld=ghxPGMc zu*=xxh{d&=U7_@{E7?z!GvUSfYIY4`b6p3CHCiqLfne9anenZ*J zZh@TI%5G)1DO(W1Yd^c4-J#4zB(DR?9CjDGTbYZPUiY&5*g-@Ob1U-@-|J`W=h!QM z0Ft0ZIh#GGoXdW}9zx8oL+qEzdF&D89QG^rYxW!VD0_@O&YoaTvfr|&l-t?w5KZV8 zkiVayZ@0pi$J6W?Wg&Z3`9K+Dzh}=Wm$5!&5qq8;VK1S#QqHZcZt%*-ePaFzp!_d)1Zg{gT1Rf%HCtg*{flmK1AH2+t^3!WA=CU3Hy}&gZ-0z#y)2!*uU5p?BDE5_7(e@{fB+SzGeMvfDJMa z)7TJKIO804mnyeluh@puL%}=*et*JX8yCSNc@&R^rBw`%<#zaNipTlw1fIypVUIJJ zr||JSm8bD^?%)|b6FZ;TJcm!<6X9PckLUAAya2WyMcm1Yc?o>Ol<{&tnODFY_Y_{m ztNB!3!)v*V*YSEjjW_T{KAktg3&l*{%xCf0@D4JUyZJoc!sqh^yp=EHix6322~J3! z#!u%<`7*wopTXPV!)hh(;H&s*zJ{;motT2H$1H0D@8%o%CVnRG;k|q_-@?!0TlqGA zwz3!V+H?7Nd^cHy{Br(dzMEgcujD_$+Dj~~@ZvD1 z8H+hiB4+VP{3?DmzlLASujAMAJ^TiKBfp8?%=hwJ_^tdlzK`$cxAQyro&13Exbmg) zmGU*#86z=^O2HgSRqn((z#WKzxC^UvR(==eaW(vIeh=m&VahV)44gTvMI_$)_(A?t zC5rz{c}RJf{~V*uGl)ZV2r;Q1K@`RNl!MBXh@@yyLiqjs0sbKW1%HS?%n$Kj@<;fu z_^L;ex}82kL6@K5jdgVGqi@XOC{Qz`^YXlQqxkb4d@wo0%ZWKbOunYdZum~#-tptf+5h6lG zmY!+L@Sz@c$Ce9Y;h;zkxV!JqBTp%tK7l|F>VsVMs zDJ~Ve#E-;f;&Sn0v0Gdrt`t8JSBa~|HR4)vow#1?5jTh%#ZBU7u~*z8ZWXtQePX}3 zUECq=6bHmz;%;$|xL4dK4vL?OpNXG~`^5v|LGcUmka$=e62BCWh+m0ci{FSx#be@e z@q~C%{8l_AekTr#r^PekS@C=EoahtJizDI%aa6o0UJ@^hSHvI0G4ZN+P5e>3F5VD- z5`PwNinqku;xFPI@veAJ92b8Te-rPE55$M!Bk?iLO$IAJQZ7Z5(903SaW@vZ0=17c8kgeHbmMP({i1)?-r zRI6&k`K4erL=9EL)NnOIjZ~x5XmyMlqsFRsHBOCJ$EpcxqB>4ZQj^sbb-bFYrm5+w zL(Nb#)hsnz%~2<)6V+TbPt8{+sRe4GTBJJFVzopqRm;?Jb+THaR;p9fDz#djs@ABr zs!Od?>(yy$gW9N0SDVxs>P)p+ou$rJ=csdWnrfcfqRv+rsIBTkb&Uo_ zOVwrSa`gS}e3a;5S%W~5gr?<#*%j$_^AFNinyrt*&Rp1M};RM)BN z)h=~|+O2L>H>qc;J!-GIS>2+ZrEXQXsb{O_sOPHZsoT}_)eF=M)r-^}>c#3M>Q41i zb(i`h^)mHx^~dUN^$PV$^(X38>ecEs>b2^1>hoqr|G|4@;CC; z8^4WwuF1x4mvJ@VxSTfEhW3>`8@p|;jcYb`uUc<&HMFnXvUyc#{mRasm0LEf?pn1i zq`qV0=Ju5O$$EwhozMMMr;xg+jNfqlXZ%qq}itE%_DAPJee>KPNRY|F6 z=w4%;>sRbFgAUV-%1$%rG0kYQX(qjlrmJtT%w4;&r&}k{Gy~}dGnbKSnnCS`!tl9k zw{)**@7c1Ut9{F6+gzh+k~$5p(7C-`?Y(PtOk0eLAHt<&*7@>j==@=ocNQ54SID9Y zizYX!3qW7%0>A8K60JgKk;~fZSLX(!ZVhFYR$pBj41kRWMHmnWPMLx8H7j<;5>RHv>*=t?2rl&WNhIN%h(z?pqHbo6}2Cnr+ne|2)^#-o> zr6H@la18VfwCd_?s|++&1$0PRjY4OsOoxtHYF#5s3|-^HPN#sgIAo0%y1JG+=2~AG zYp29FxN}n1#+6+QTAKCp>kV?$SJ*m@@;i+#+9_LI!gdyyhpzL%(#tC@4_m*cXVt3i zuJ-PZ&Xv|KNi}Pimmui3^=cPTvUckP?QY++v3GON#!YKiS-bs`m6VB9V6%;W%|6W_ z-86$f(+m<$Gn#jrNn)d!>l-Z_eSAO7s8xfR%SbiNXrP9o@Qnd{k7Z&hn)NhVLkwA|XOSE+XxRh&f?madK6YcPmW&)w{osZ1h+o?B<#;#boKqeczomMy*- zG#K@6H0aP^*2t(yLu2@sfEsKuYhd);Ewbls(R=Pz9oeluWP`VQd+t`f=br6*sj~)H z?QH*^D=ChiYtXc@!g`Jb8**9Ja)6$yx;GT7n@BsF9L*+golWT zND-B1K)~=2f?5R311YuCT57GAQp)9et!=HfE!V5opQRSjQZJ>7UM>Q~@D3p)JOU&k z`F*}Kb9Of&c9{3-ZjC=#-YxlZOTOHa54YsY%|}<@ll-_e{F1&~^5>R(xwLW` z-s?D1JR}QIac$tR$wFFM z5XC?|mkT`)@)#Z>p$)YQ>gwm37D$oTAVWu)^<3n|B=W;tle(eU9d7{#e$WKm)FuV1@2!hcPYk>nU*(y?y3dhL+5b`iv>sS;v(L4 z+{ML0(&8e~Wn98*bG!0gjzwRGo3L!Q_{ytW{LtK}0U-aJ=z%%#O6GzwUdbl2FdOr9 zs7(8pi5Uo+?Vo3td36g`t>Os6`~b7P&~OQz3x_~BPQ0*V9=wMIbqf~O&(;`l66oOt z^#g!ugMd_ewuG&u`_+Q9e2_%U2w3ks1Ikt+*L5)#j75keai)}6Xpf`%Ny#2atD@|jPjYp>=koYQXgI@?8!fe zjEEa5uM^`}$Is{lWxqT>(fpu(N{sXa50E+9FT;1q38eYbe&+0V$xFHZvr>aAg!xtQ z>Dix;)ypupt(w1lW`>Iu)z?RJf|3a+S%1(&N<&s(*iUKA?vSnHNQw0QnpT8~=f z2g(yDb6i?c}$Jm_X8uvfh!l#&YQp#tR*nGc@bmXv1;+s#duOEa0M2x$l+AY zt5(lnRKI9$eXs@3Mv6ZL!crHOD77nUi{utkVf})^! zsziNMmMIdDdCTh;iL2EsYGvtaF;N$|#6(_PBcv>@5fgcFjmY;%hr%Nr3J)BL+GX>w z5%4gl;gJr7M>-Uql466*UEx;>ZY#Mdo)XdSo)Xczo{}n4-9o=|CEPzoIiX_@+>w>_ ze)*+>OHZlb(o-tAEEVpCr&L&rr&QK2mH0}zKaZ^tVO<`vO@?%hsjeP7ZnX;xJT}!n zEWSCnV}czx;;Jta!#qzQk6N z+ap`SBRh{rtla^Z<*;_IU%7bhL(&jD*ka~s*@_;qZ|n9*x6C89x{#J~VjGElO-ruxBJBl=`fc?NcfBDE5aTk#HkDvJZHKI|%$zBVv2Xji*lqKC#|$<1KcX7QNO} zB-_O!`4a0Vl#_ZXlA0779l)p6ll+PG6v_+t(<9cWZjaa>aC^l1+U;>kyAtblw@0iG zkuUib`wB=)eq53tv7SeHX-`EO{~CR%heBDtP~t0=dKBwo@FV#X>v`}a<&*BPM+}94 zYxHG(Ii`4AlAc(`#E6rZ*|S zoE1G{{|WV^+(ojS*k=G;$w!g23vmkoaA`kc-voTfa$Q^NTFB zT3Cven8+z`6&MyRU%hI9z$&>akgZx!Bcv;+5v^A#r7oxuTo=>`trm#h<#N|>YsGi5 zV{ZLmZzgi?Qksn62+a0gaStfjoAhP)u6PlruZYisvynF?)_A;7gfBf{Q4*>x+~B+W zmTf~V<^p+1ZB;50{v_-g4Bwn)q#s}aQHPe{Q%ET;;e;wlRj7ohxsE4=d{I2(2tJ@0rp>9ain$$_pZNW{lVZ!KfnMc z9U{Y@h3EmG0i~THi`I&M>v{nk?0$zzoc`?T_%~+@mo{gDn>9b;_X%8-sAw1LN~rv3+2UxWoSYP5d_RvGS080 z$nzIPwCAmG(VhZM2)LL+YyQ;*^WFl215VJdp#&};8yuGqfCIhNMZrPaX|cSF4>&+A z34=3z=vTWiXwmZJ56_*q{84xW zUYy(yaTUn>fCc%Lf!Ir)x1etMW4=Q2CL7)a6-4FDwgP#Rtw1<$u7ZLR!JxZ{7!2e} zaFy_#XPGGGiD0jYBI%XWzi6d@!tI|J>z^p{PmJM-V6WuYb*-$8{XoQ-{j_(7P=WNfLw3DuK7FZH`Fkk@X%@^r10r)#Y|T^HDeNpbv~6+eJ$FraiWpuiu%X$}Sq zz`>~xE+KGGH>k#ToBZa!IMTc?R=X-Cyi&r&tsTG%^po$MG)Z^Ql(7olN@O_oz=m%f zf6G$7FM#LRRlK>v@boXkv%mBf%hPMfvh{M@%#D#1m;tKXP#P_iW5zodm^bF6>FGIWn&(Je^Y92A3mPfpv zH6U(c8xU{8lkf~rynhez^Xvu0KVm;Z{A2cG#5>tPA%2Pd1TmgrNBk@HE5y6lF2t|1 ze?`0p&!jWpID;>k7B79()PQxdnPgpf`vca6H$Y%rs1NJHTOhD5ya@v9!n+`_F1!r_>%zMrur9n0 z0_(y%A+Rn`gLUDJ5Lg%92!VC6NBA2IkCAn;uakA*O%PZYIDmEGeGpg|_<(hR3s@Jp zfOWBNl6A4C$-3CL$hz2OvM%;*vM%;rvM#oTtc!h*tP5{};FPC8oD4-F;AQBvTU;}o z6rjPs;2XVEQ3GKeLj;8-0>>OQD-ONDdpZ1?(qAGllo^1Nl@CaN8Q%UgIy#0QGo%wZ z@Q2|7Hq}jMu;A=-0mUQtfa+KdprVvYrPxHI8S0`e-bAAQ!T%Z*_Wb}Sv>5%rB?fB~ z5vG4}Ox%L&TVm_4QTL7iH?g2@X~3KP&wiQsR&ELMRcPLd(`tI5p#Y>YT{YMUxg-z(tzbt!u6N3brWRR@|DFqxla6Jt{+Pm z@8h?|ZKMs6ODzk{d~w`vJx1F5isC?y=ZoVnLklONSR<{s#%lp@vVLLA^@Ve6P;!VE zlJ4j7e%Uw`cnYmw9RDXcIWNMh|C?jqI0mk(5agWbgZqz`unsx4a}N3Z>nlmQWIz6~ z|6C)-4V-5OIwBs_69Jd=Lmm{VnW zvYslrDk4hcXQR{zZz-JeDzOU*8<9i2s*16J^9l+&oLsM!5hnXe}|^7Z6K zzMfpd*OMFhdU7LQPwwFBNuhaA!wS~|?DS4B&X&a9L;$CVRU~qZUyw(qIQiGpFjBpN zIsO>UetE%jv0bQ4pHJ~D+A?F0&Y!<@1zXGGjSKPk8{0y!#;_N7yq(7{E?-%@jQwoY z;$;ijuj#QL_Qona63pJAhjy{&AeZR)ej4pX=TQYHcD!93hShHr-l~pM5^*v^I!;W; z!3iKPXs{UP3yf4MaFW0{oRM)SP6L^$+>0|i?!!LJ9Go~(i<3bf#@PXMTF7eb%B)e= z!w8%39`Z)zn|Q1A8Jte_J>_}4L%dDduKXk3Bz_rhO}~OO9e#!PsNYcD!rR%u!+Y3! zl@F9ZVTFA_`3R>U9Kt)8f5qFEC-Bz(S)~>4@^>hg@doz|rB~_4xe*4exdT`**4&{u z1tJn_?O2w8vmR0ze(H~9;Vg&Y%#Bre5zcHV!|Hn!*5Z{oo1q5hGE8KXum+!klNYAp z%!L_PYd?S!7Utk|g<5J|oRL75xe)8P&s~jZd`smApmfeU@ZPx#bzl%hpQk z1J+mhSXUVELXa!qrNHnYSCA_(HE>hl)}VM?j|zIVXl2mu;PJsTgIj`oZFROa_-wKr zwlB1uwa>6G#HZfA*M2CZ+%Yv|V#r&LsrdfDQG)L|j!VPWhuT9M$`6KaaTYpjoUadG z@BAQa=J54lkB99GI~iUSUKRdf`N8nR5fjS~mLH6m5%EF9k;qArOCo#o7UeC93XW<( zyd~;-v?Kb_=uOerOLxXtW7ZB|AM;FXW^7sP!P1?vEpbyzcb4vqn-lkG+}ZfD_(}1n z6O@EU6E-Dul^;wrCvGlUnYcYEI4LJ-Tfvs3*OQ}@bCTC5Z%*EtVl7&k5}Shjl$TN~ zQm3ZAUbHfGPuc@TD~nd9ElE3!xGjB3`n>dA>H9MhGs-fyXXRwPlF^qLn3M-Z94a0V3Igl>P`jEKJI+i^zZ&7xA_TKE{IaBi%<<#bUiui2qq}+wM`*Kg_ z%|xk3^NtK(pVvBk#qjm`Y#x4Wc-!!9*DTlLu6?eP`9=9t@^`vl$$!o5bmzIZyI;X) zcR^%!aVPNUkWXz*WVbG2vreV>XT1S*cV;SFWtwT-iG|dTia; zjbo3EyAs+m>udd-1x72{@&dwtx2@p@Koi>@H5ks5k@rhP60jY6VU@S&I>Hld=UTOuUS< zU*1;U#qRVWTL{&hWr)B+l{3L!71#N1U#@ z5ND|Qh%;3;;$dn5;w%;CQ{#*e58@oPNYV3iH8$zD;`9ePRpXoL7QB}qgtV$|K&->b z8sEdW0q1T!iL*7H<2iSzlTdTB`a`^@PkE1uvo!eiGdM%zAMrLmVfcWumtckNmi?0SDSUc zIHw>1ClI7^?jFGz1Q|GYfcVWutfMm%@NLCeCOE-FcNQlbh|>&W0A=83sIyo`@N-ov z)JnCApQkbb=cwGp&rtaq&P@3wP5^nIpZxJ>ewxF-@m+PCO^B08@-Wk4H-5xE5o%~{ z#4T;aTZ&o}TocUGtY(qIs{6PFQobmG`w-P_;E5mtEv}(!} zCWE9^lLcB?h&iU}UgSqhS_VO@26j&YP@|U?t8T}cZAQQh@F|kvJB){q0A8vO@t{^8 z!j+^^y;spy@1+lisMXI`KaWzokRq&RUatv|9U=fm=T9QHdb6TdZ$?bz=;nfulvS^Q zZ$i}ROwM%NFPfq(L_Z|@GkDNd&!7)V2s~AazPd_$I80aVpbyHDze#xMJV~{xw+b4l zjurvXAi`O78fiT^!J!kpY_Pyu)voBO+UdigbT3QtS9Jg!`BLwy_YBQ#1@1o~=!y;$ z9jJN{wK=yHGZlT&+eL3zJzMoGhv}+n=>tx%KI3@lcTv=;$yJlFHNFM#gK5V>hZ&mBL<><(Ebwk)`Q7Z5vz&n(D7CAx1 zN%v(yfsNgbyi|@;fiKS`@z7m4Xc4tT+X} z(Z5zy&rZ)yNfFX}wn&QMvc$81HE@dIJofesNVtmAx?9pR(awodQ>0MJ4f-=jlwD+ z??{d3NWt?c+y@*fc#gb<=hcE|bsx8&0VBsKm%=C%5E z`i~YRM)mKfSgTPOgB7#DR2Z(PV_L_w3R+{(PGd-B!Wk<{ z6kGx90|^;G3-gI+cg@mh6iLJyqylp*TA>{GL}ifF%J(a3`F_M&`W>X-K~JGH8&iey zuS)9if_nKjzz_gU;4z=U%bU>JLyTMHI3pNAEw4kY(U_^|%4gDtL)4066~xzgNhgUY zS77umFBAcA+h&uYy(eU_SwSs;I=NRojA_q{>-UK5|k-!u=Y8U4! zlgFH^^j^-@4ljh2wW@b}HRCs*lpRzUf;wu&s1+JNW$!AwQM2eH`KcLQ1Im*mKUtg~ zHy{`ZDFB`=_$k{2OsS)26jPKeFLgB?Ef4$u5$C{!u__g#S{2TetHrngJ(XcZpin|G zBVF<#(xdpd)KM8mg_5^R-sbs`ybPm4$t#Gpe2fYu+fr#BNBvRHq3&i}6Cb&Px6(Gi z(C!>Uu^72t#OU}#^cJ$3Avi0kL@k6;noWVZ^hKm0JHB~3bpz7LNI%EZz*^!^)Y1*5 z8@y@U^ewF~QAPQY-6&g&YvN$I;H>yKU{fW;$C(&2o&j8i=QClb)++`#2x{?*h@~cq zwyazn^39&DZ-skQjDwzq)r6D+>(zOz%-+H1}{gxKh-d!R5`|QUn)40NAy$r=u~Zwy$_2gTS%3*C%f0y3#5KHMtnte0 zH)bWH|HvSvO^#MntnH&gk@x;&SV*$9=w+T}BX%e(she}I;JE=_BIzt(2!M|EQe{c| zamuAOO8SijNpGO^8;Y9r24YDg=_N&%^b&nIL>*B!0@_LMCJ@)9moQSm zzsq|&4-x@NWx;phLG<3fyqEG`l2DC5=|y5ixD`1%wE|~H3ZG@r3ZW2|iJ%rfn>P;_ z1=jV7u5f+cl)Nb%g7K&nBS79LqKI$@=V%Vpw^rl}!x~VXer&-C%prLtTlO+a>nz@h%B_{jvYy{6{$@3ts70gr^ zf|{F?o8zT}Hpz{q4~M8Fqe@^2xn`sY^AwV@1lr2EA_8D{OZu3Na*pR5m*u=#%h&)8 zERus;?9BbC!Vq-Dy*V%Eh+Yj!?spY^@!6c`=)?KY6~9g&&}=}OaNh%}YVnTZ9e^zn z0nlm0U-8ojFN2$DIn%sl32PnA5;>!CAQ8vl;T6<#7v(L+n0ePV%gD1kLQ7>Js3su zOL&kH=VvNvekNiq4R0(z(vvQ^vZJhKEAVuQD_55YxW1qB#^+bp5j;6L#E9tHi_&`) z)wLI~q~Y47=v=$#!y&5YXP%#-6tx0L`5F z$uTN8;Tjq!d`@A*W&x__^aw^gJPu3~hj&Prmj`L_0n*~X=C#6Gdg$;%U@25|=?(PZ z^a*dGqED|A9}d$M;#NSRQ(@`k5fGgP;6N?tE9e6Z-XsM;7ZR@pXOYfP)bt$01bs+w zo3{^i!s#P1c==^Cj&0S4@>U}rUxYthgnd(LX_klaA3;q!l6C~xxa4^l|I_xR?aRZ{ z6M(1%^#y2&G@9iJ>td=`Fb5F$lN11>-yK129>)K)XVZX_m-F&~Jz3(x;k6w4ZG=iO z=HvyV9zvS8zvlLV{zUo!3&)$=2D~MC0BD%_e1*g8i z=`Qn8cN{0F)jF@#e%@1hE84LYGsv^K&vLxLl70v?$a?y4h??J<-wVm=BrU#1%t!m@ zV(g&+_#=XTI=tuHs@y6LTjAp%ekg|j(H$`T26OP zx8&jR{KqjzoRvJRBK_yrqFx@L2RS8z5%(C;)1z~Ca~SM}*TCE#kDUo<;bG2$Hun$M z<;h)ywmSxV8#qmtwjXJPoO+SY0ll<$ke2PAl8N-INKfPWpq&Q!bE?T}q;(@1KW?Vmxvc%Klg4b-c69quTZx8W#AbS!{ z!XIKJ&MrdfBG4>Ctnq~3#>q~l4~MA3cMji)QVygD>l4(f!=bUPJ`n(;{afT_!A7%M z5mUK!Jm%E1_OnsKs$f&eyf;t9`W$$&UKb&65^4w>FTryy&-0GSSuf$4PP|qleVw*6=jT zgY>zi9m3TpX&{xGDj{APX7a(9q%oxR4%>^_10ipjz@7dP($GQPERoJciw}DVX<-p8 ztp@4mktTl{`Wn`Nuo3Azo(31g9)&*Q^WsH*ZWr=MH^fu3;C0wE@QSg4L%h7w9xlH# zM5EnQ^^Zfg5AbACZ+Ru5Rp8G=yHm)ad6~CY=6d9B&e`luFGG55rj##tAwVs!U^!Q$TVcVGi{h!uSo+ zRSxVhqXm}R;e{bLW51$i?9V>JVdzV|21cG(z;X9V*ayvg19sao&;|%<_KVpsa(aXb zzZI3eg+5-$N$Q3BjG*=TKE&c_wt zxT_|(8nmv+P{9LsfZ``-tU)c_8tEO1F1|ctQN|(;QL|5HpGGONY{L0%;vgF~hjEr= zRC-~^O@AGj)C|}kFX!cedoWJS$6Jx3Bmc#3X7rn*Fh0;2p~me-EV;o0PP({Pv$2K8 zA!_!F>>21??MM;+?{IFa6c&ee$U5wWAvbF;Fx4Zba?kSEJ4dTKIWKgdOx)`jG)E6} zqE}G3&Z)9tXK}4Zpm|By>=k%xhlU*z`C0EGk49vog&i)s%9Q<*L%h0RxHp5@Y2&S9 zNahaI5kaE?KPmIsOfg@xtXar!Aj}BQNgB%lLjdIaUI|&|H0(WS6j^4CI;01UpA%&W z^!$Xdf8seF-Yaf94->P8I;?dVdO&7}r1}HW_OMR@f$dWO`~|^D#!2w{LFNY>_5&|< z{6Z?1_ZwW^LDrYK1z4~{o$)GSP38tgm$5CgflCTsaM+?@uzx-l5VftG1K5AYqkszg zCm8Ww15EP}Q@J1daX_QgR$Uh$s#>AG&p8=#WKF|7o{1ijN}~-V&A22&>H{L3icyC~ z1AO!R7(3EvDZf3KZsazl8(Lk^mpmwg}esHYw540 zM01Fm@j?c?5%`Zn3p=@ce*>>%^_+=G)bMFh!kPP?jiJGgm<wi{9Mq$YJw$g+N!%l7rM->3U6R%>B&|_^?c}uZRwB=X7P;rJRc)5E zd^?4H+DuhJO$6DRDVG#ANK8aX`qzy5vQQwcbNE?kg^*v1VCT&QfKHH zqi@xuG6y9@kod8OrXoxc_*0J|ze3{2DxY7`Tt`7se#2wlkI;K+{8C00y%qb1!han! z2BySi&)Z4u`0S0D}6XbO|4C>Rai33MMUXeK%1q)&XP}x0NCq-pX3ABb3cHX z%Hi!(Uc-!1yTG{-sX_Bx@^<7QBzZ@$6{d?DXK&W@U z@6BOJk?Mn7OSkOjIAOC0I_GQ7*W!+7yg9cidgnIhwzv=C$m7r{y5uVQzz5R$-flqE z;Xk}K0qE5puYzcd^>AEr^Z*Y^ndOWFdUiy!> zc(qQ9nh2^hFs=mkKq2;3g@tx2YG`*{I)?yzDE87r&&E09oE)MiZAsb!zEq@$+8(Nx z1l@#!Zfv_3Mm1jrrdMNOcPRIsk1J&vxVj*@8Zb(Qu0bt?*q1mh!huzG?DMf={l}8R zk+)G&eqTr*Iv%ikj@{QX;u)0==_&rv&r|f<(+Gw92)fX~uq|OUmUC@7t}8mnb;tFv z^&Fxm?xZm=tWHw>Bgspa?11+WHiOe87)Aji3t-?PhhGCq~1}VK1C|hY$j&uR@46xc9-?yzd?s!s3D2S|jVi>;p9RMu zXniMIf7Yx{g2thp=Bz=3AqT0IV5Ci3H(khfRz^ReK^z6&q}K*b2d=N9``JiD)P$LyQ&tA<9D5M!Emk24y&aP*=T zd!vK~n4=XC5hMqW!^l63m~)85KXdGp9DFJ`a9~WO0O-H_Xgan-dZH=DEAkR&;f(-T zQ#7o|u@P3c(XlZ~%plBx@dQD2tVFC)o`W)T=))mu+=jRfC^Z==qV*qyD{ciKWg-A} zSaOsOOzDnvUhXqLo~(GY!K;;!UUe+zaKN|_Jp~+!Ao+-{P}JxO#9RZheaLs=n(!PI z^rF#YC;)QIhd)}6`=MTHQC;egPxV9{R~UtuN{%0RqcA(ixMSSj^lqe+W1iz_7P}s0 z9k?dU#|8bU?SN?zjUpeH)Ne|g#JqN^tWmX~h7cphWTHI{v@&C){l~)oV>J73k+j@^ z!Hz+_#VeiWseKC&Rjrh(rvZA6ysj_`Y9ea#z885GW$mi$=`k-M4_O@NNm*hqo~9P# z{KBJR@Z-%C3dixfUOd9^LC$!aoF2qo+qtN2wW-=6S$abR{5FGAK%$ zJXPlcT)}UWYMe0mr^2TBhyp%7MIO@$js~=?hBLCm%T1x8Mjnqm&eJSvAKLx^t_k-= zNfovg`HqB;-#fxfS+k}m0HGEZYw-aTqk^G35v2T)m~|ri5L22(%>Z_7_oq`(j64Du z0!m!+(qfVC;yos>%#pi>_P--R3qf_RL@X&r^uiaK6VZ!ruJOo|k+9N;R-_2u72*R@ zGbgMqf^;Wf$c=axn9>mw?5dZ#lv{Tb5EZ|;DdZmHhlV~1PXe=6(0Ig>lh9g47gQ8l zi*Lcn%*dH2H5DnMaE)_Pqp%=5prgDnazrFSi5s zbVJdP!Z50b9l%^g`e9+Pd;~S@9fufeK`HDNMHlvpBis?r%c$Ycg+B-SwDti_{G(nt zxCq+_FQUT>qnemG!&ZiH+s1G6dO5^z>(Va74KIX+Rk4qUYSXYhV98U|kQYK;;M56g ztfC9qNFT`$UY!8tddUw~6et5eLgt8oDte(aCsy1clR_qO9#M+dfK@T{An?;O6bB*U zz!Hvr@d{#TRUyFV+!}%xECdH(O<_&&b}mT{jGTjI=wbDM9^=#q21e}rfN2h5(wxc1 zf!Jr%>xSaM4s9c-_UD{g8VB|!MQ3kvMmlL0gzbdk#YoU2zlJbdhy!OE>OFwI#77kl zLzaC!FpUotTv`1%utEn$ZG0bkt!YPj1l4Y}h#n1mHmntF*DX^m)T_aX^HC>80!xMD z1ZxX$G7sfG0yKwnM=+`hJF~rGiR7>#A1BNSP7L5g^U?5O^@^m=Bn# zPm45sW%F*^LT`FH(ys*{<7pQ9DeCUPHR;JFYZL9p79Np>?5k3Ja!{FVPACEf$LL-<%96=3NQHSMV-i3PwyY%s;#a>{9GA z>G}at@uU#W;F38kcs(3X$qThdMT=gt<*>KX+yV)31g=WM3)6ELi!aRBp;@27`?Y4*3!_fX>-tSo@R~; zlqH{mG=RMnx-u;h0hbY8KA33=o8^_<&-V!IQW%12Dhpc1aiUrv{54Z%&;vmaaEKam zIOH&Rn<(kf6LBnLFQ}-17IE70DY?L(DXOtO2rUIPQC{j|S0>&6UXc3dp*3PGQ5c0l zVTUYmI?|a)Pv^X_K#a6T*nVJ*#;p-89#|AuB=SQr@&%?#PGf{d45tC(Z34Givjy!Q z#qy!FV)(Vf5Y&L<0mn5uh8>D7U_X60M78g+V+IU(8!3`Dj;rk8W_J~+{GnDwB1I&F~OVnsV3sQPDBE9U~3 zteEli>-oubWUZ=gr)?*^@Y9m76ynQ<@j{Q@%rLH@3c;u*W;}fr15OUpsC#$w^u)1_ zbmX@a;HM?43pn8S29ILY8w5^hKRAaiVM{nfwK;9D0VX|YMsHAHAj2qP1hk4%CKw{$d|!d(&KCXHeQ(h^2svSB@05qSu=;RbKBGXyp0HEXoSjdBD&-FB#mi-boh#TESpT9D+Zj?EwIR_SY+OpZp4W3C= z4_ek()(CEbt_NKQ-9?g{VZ=?)NqF(E0y>RTCKz#pSw`K8d5Sn3hM0lsV4hA%o-p?M z?VSwpq0H+sg02Uychz?_uFQ4d>NWa!Au|Elu5LGf#35?Xl%Of#isT^r*__XEK%g}W zfaM52&56KWtD^Vtay|}upVMamj%T&xC^Tp`;O98*0!A7~xYX=Llnz(Jy=nLrs?jXH zzreR(L0U`Z$r@-&1`UFG9>oZ-Ahv?{_(z5sbJ%OB@fym!W;*Gmvt7}#?WT`RQvSdx zfso&{o8zPVc4((S$j=}<%^M&?FnlqL8LZay0*B@MIKT>17NB)^@E*Hg5yC_NoxaA4hf<4=GW*NK^x8J{bVt&4n?sFc@$_)6Lwt) zie>n4iq^Bp%aRm}WeK7diGYWf5i>|a*8c~BAZT>A>a&qjRlNIqt5&s@wdHw#x|WTW zjgsyP^tVSP-BKT2_>Dxj6fsi=?NjpiritSF$T6bsppn{uQ4c|7j|yEW;Xu z?EjYQmg^{`_5Tr|Ors#f3V<0z|0ft&(0>3ubr>o*KO-ncAlb#(7%qGMFTKyJ4rpop zCSX9oeL1Oj{o|;+10Sw;3*=Xxzocs&*`0vlOZNcjT6a*we4GefM@e1( zVQ5|JVu1m{r)yoHzpmlG`gA>3$gj2nCUiZPs3K^R(;NLH`5(SW*SbtiYSOhXQYY3~ zjK(1y&3q(XwNKY-mxPe6Z|O<8E0W@nx>jl2FDG5AyELk57U>#i671v<)s$#TL@CXp z#*wZ~fkM~nT27l_V2X8+uGK{n=969M+AeinGqkSd*zKolh8-85T^k|4*RCf>ilk?u z>j^{^LDMyz)j+zg`6692@;tp`7tV@(g{PS@2hwcg_!M@BEP=6$Jxx>qAzjzdHG)QQ ztNLxEh~kjCRt_Nd0CZi+#(Sy016{i?d(bQZZ_lvHunSbQar#ccm|+JX@ZFf5=rMv3 z)eX>17f$*SFdrvE*TbZ)Ck(A?tzTot@RJ`i#!EcS43Hn^5fL4Q-Q|-XR0V|O-$&O7 zn*4MM2FX9+i!^?_*+Tyo>T0va-F_4!i3+Fd^N2xMC{l z+)tnUoE4I;QXR)B4)P_{w^4_J@MTa_eFddcRieYCQL(~TovM(ZsmoA8FXg>omRN%x zM7$~N{(;o7di)1KNVgB%Dz91rTnIjS>8u$)c~#_T@;)eOQR@qNAHFh_OneNN)f!bgT-#!koZc-nWEin%H z+Srk(Jb>HG0e>`iXQ>Jx>?waqggg zq`n&dD|lsC9V3wZ0vqR^Y5v+G)FJ=nH^^OveevJm9Q01*ByM6vFeBALAid4NNq5(n z{wqF(N(JsNd@!6q8ai)(1SA@%AE%h~6L9*xLq8dN9AWzP`t?epzDfU0C5h{c#(eC; zbM9UShpEIV_9wKq(PpiO!MPDUFM>vv=|isx0rm00~o=%!5n z4aBATO^C-rAF6(gUx_AOqM4U~Z-5d{4Jz>r(zEs7K|D+UEaHbCy{cc}hvR8p;#<4~ zW)PJ4HZOq_Ho^aB{apQgueKOkgRBI<9^r+l;9Sp7FvHnpfa>XtF*;q$+j6PAot(&< zf{TcjFhzm!p-`xs0mTAKb|!D)?Jiuz4}nycoi!&R4> zue#L&wNUk_Md~r?FF2PfNm1a9{{edTwLlK?tP0I)MkQFq3KE`0tXjgqX@t?MWdeq4 zRgF?(21!Rf)xr5u*ke=%VVBoFW4$?m3l4ah*?v3CT#s}b96u3`^hoEm?LrA3m!vz= zgoXubAP)NuTV~l}z=HuiTzZI4N!t`V{q7#;w}OtvZBTwUb+Qf}Z+oqL&x4 z6)RGZm%|jaqyHcA9LRHG5v#kE-8{DfgRcIEvLIOkjYd@0f;d%CAH$FFQPsuubL+4T zNm{2fttR8d%}14|mF+k`lgh)maF0~sfkp=bQJO=mnINnzCwJlAlx0~cyMnLNgja1GD)6dbwAPlO7|8{efv=NiS8Iqoy*c!=r`-1*YD8(OutM2 zj{XDvaec4BV2CrM8-^Q743&mE4EGvl85S8HG5m|+9m9FU6=Q@k(U@->X`Eo3YMfhDkM5neH@w-?ZKI6Vq#^H%;%F{$x66I%+y&>M-3f>&$b_OU$dxYt2uY zpE3Wyyu%V~nQ57Csk7|0T()Le*IQq={?7V=^)CU#14;tM1ndp?bHJg%^uVtNZVr4g z@E3s>0&c52d-u?~yv-W?m@3jBizRUiO{r!+JIKT9sknhv5=bcxaePO0BdstLha#(g)Vc5v9ny|aV_J;jAJTd(4@ZW_0 zF8qV=zl0wSKM~#*kr0s?ksmQ4;_iqCBOZ!)B;tvPry`z>*c$O-#H*2pNLyq~WCcz< zT^9LRWJBanB7YV6+sHkU2O|5T?uptE^-R76o*6wqx-NQk z^!n&;L_Zt-579fLe;yMM6BZL6lM&;JDUGR&xg+MySVOEWHZpcw?1i}MxbMfkANQBI z!*M6#$Hz~Nzd!zT{Kfd5ga;GWC+tl4FyUt6U5PUiHzYoj_=Ch9i9b#Jb>eRm_aw2T zprnYTq@=8*!laQ&HA&l&{yFK@q<>2eNsdWQP0md&N?w+{C3#!&KPSJM{6_NclmC=_ zF!^Zm)#UyZb4p0cf|PA3|D19-Hs2L+O9?keerYX~&IXm;!%zw?? zo%zRM%CNhJePh^xVgHepnw6XNjjWHePG+^^?X1G=k$Cs)(d=V6=A4k6f}D{#3v+&w z^Shi6at`O5$hnZK%MHv8%bl4!KX*&+E4lCGp2)qPmzH;L-ln`yhDQxg9=>$=8^hln z{^9ULt|-?c*NV!PtVk3hpmhSnx=}+JbKse6Qd~1wSb`UYJw3sqj}G)w9I&8_&ri zchP-C^NOA-dZ+09;+$en@j{$c(O)vHWKBtP$sbBSEa@$sUHUN2A6ZxWjneOyZY_PW z^wrY7GAqt&y%(R~lB^(e0yqD`r(}ujm}(7?U_=+L(1?UKw+=GPrVP zns{DBDy<>Nb{pr|$9sAzc17km{vR0K=ZK*n6&8j2#C$V~1b#Zk?^z(#Hzjf!$$=|=re%EiOJU^xN?$Wz|arf(Ye{%Qbsj*Yjr%sqU zb?U;Y_58DL>PGM9#2l z*T#MAJ74?v8KE;K&saEP`HU?yKALfAM*n@O_f5F3{=OgI_x61s%nY6xKXcT~-_HDO zX7~Ns_m91Q-u+MC-}XS-1H}*gSxt&ssCa9Z|hIhx37Ne(d5T;kL5h}$YZ~Ky!P>n zUth4svu56!m1~|@^Q|@iu;%A$cCFd7=He3pPeeVD@kHqpW;$bKUfHv)3(J_sF_6>zdbXUibZV+t>YM z-D~UKTz7K4Zhgr5g!TFBtJmMR{)zS5*MGMD+>`bvlb;;_YqV~y_4iUYNZA{pK49ni{j&Oi?+cwZC^wsLS8pt^ zSEE$ybz@XiR6=-1N7v=cmpd+AzC5_;3}$m*&rL(4xnJq(y4uy%WggU+NuT`r!w)}f zIo)~WJWyO{y?puB*0r~`Ug}|ij9s{3pvh8ngDEKi?l185wEqJKx+K(8rYC8`m#i zx^m-YUsK4@qet6r=mP@-&!0bk!xWa1lF}R*0XDHCq|%obsW{g`jX1>u6Jp~t6Jp}y z!wfy~{l||V?~nid^Zq7tztMR4a*x5RYqVr$Mg{es3%Ppkkl>l_Kvsam1l1HAg1=ys z+Sh&kdSYU*dhHzkuBo=f#Ov3)`1sQ27(# z5Ad}dZ|Zf;6qVu&rsjMa`k>OU31Y`${k%C&B0Ff^K&&S1cyXyz{z%iYzb z=!{0A($fx~;Xx^MJ1TDrP=Z}wil z+|k&ef?P6Hd>AyJ9b&=4@^!@ZVo$fA~Z5N6MxB(p$Vb=ZTRaCO+ew^tDUw$ zL!+g0#_%!HKFW_NmLz^`np9cL$_RsUae5>6m+1~>BpPT{y_F1zW`YVU!#P(QX zV`Gz}E?>TZ!T$24t}dsutLwrA5t?tE4w_70m2YE4fUq50NXpvm;nM|S(?pXcS>wlC5CSs)%Spn0zD;f&M};?U`_wmCvn>22n+4>kyS1wOXpvuimY_VlZV}xUI9T({Ln> zrJ*;TIdLW`F3R9Kpd4`hJ-yK#}yZuyu@6EnG)*su^fhpZ#Rc^LA zmEJ4u@Wq;&j0cb5``E|Ll{kjtr^-h2j7+D&;LNz^$uY`{$}cy>9y=CdHnYC&t5+{3 zC&S_FXuoj5b}=|O*oHsUH>v?40ji-n=y&@*{`BC-U0ol4^2uM`|J?@OKAGSm1zdzU zoyUWMLts*EZ3$N{U%YVPLUWUD{~urbMn(UUX4O6Bp)uIUetttpOH0(XuFD`3AAiOg z5X4F7%g4`Iv8-9Y=j~s-_>#bTtM0xGgn9_+B>M7Y)fN_Z`BFPFP` zezm#5^x0>ho#@n9YGyy(R@uL%|GxgojcVuR?x&)yRzp`yi(t-=-FT*=TjKVnag>8_1;X*)QKtMoANlB93&#fQ2J(Qyxb%x7FkN(fJ%dIW-MG)^QsY?%& zjEs&6Hw@O>nZI7XhMwDgp|f+4;hJfo-mf$0)h4OFSeP)%GM!ww53*(Vna^pl5t@P% zTL)QnkiN4IYqNgbjv8d%W}2J~5lIP=O?+(xLysdvH}&VEIJXl5t&}soUYR| zrZ>Z?ySvYwyLj>3x$bTx%zAdY2jAA7%je-l9zBndfc4?dT>mChVq!;!-G2P|#-Ns# zi#JRm@$vCC(~ZuS=GYkEB;5N(Jlf0D2i#j0vEjNN?ZGdHbWRbZ5P_FpT=JV zmV`#5DZqBEr@z_$N$8mz1*_eH1lzSUAD_8qQ)n+PHIXU95*6|Q^t7mlH^8ipzRteZ z*1p)-puR?PV5D;ODvL--3Btw<%CQ3Rg8a?|MyxRRWF<+IPOG+8d6KXD8$@x`P2 z#OO*-$+^&yaCgp}QF<<6e7~-o!6gl(0m_-?rchX2*GiWY`LH6R2FlGD41&lR*vl<(0IU$hasoGlydLB`Va+ zZnT5snG2UXm=(2cRz|fuuVIeH?bjy5&8|+HRo@iGOt#LeJvWu3M^AOyqOlVYW9vM1 zv^lNyRCo7H!F=1L9z`DvuAC8(@yeA$`0G^cwCoOQZct1c464D<)7#e+*4NWxFu^%4 zD2OohoZG*D|2g!80?B7S_{;~N5e~=Ao3wOl{cC4}v9-1JW-#vB1>dAKzTRZk=>i)A zoKB^qqodCn4t|;=PMvD+0mvDf;{2==b4dGHu9rmG6u5EaT4!5hz_DZ48?|*`_#8VB zf7c8`&qUI_{VZm*1_qrvb&Um~ub6wRLHM)imEOyjqN6R!73xPl=i9KMcfG&4DIJox zW6$W^Rb50(n9~+y)%W-J>vTFh1PHSzo$c75>AHS%Lssv#w$pGXL;5eG3!H4Df%oW% zi`UH&;fD{mUZcInVBG}+z8uEp#tiC3v9y~K(0~5)p?`0=c%>J3d%C(#K)HXt&}|D# zNlmZ^>c6}{vL5n13He42vCs6kh&<$$)0bAm$RYNh{ua#)xfS)_pe=faZydB0?Ql4P zO}Z|i?lUjv$M(04A!n2eah#bw|5x~#;%K>H>a{2=P|P2?=r*g-^r5~ zx|ul~BTjsTm0blGd$95L=Eh*1$!fP>B`ZAH1-ovt_gw3;nsiELW~BA(*;Z>Yvdlnv2FqdgR78`uK^ z;^UID3knJ{6Ixn2J8ic3c;D71PAAf~K9%?0>+gT>J-@Y@wuYv46U{K`;3*v~6vAw< zADBoFlo=l0-rnBX+{k*n8jW4&(EZPKHL|Xo8yf7_z%BIaP};?X8Kq3YjYeCX+wG1+ zqXk?1sQv9bShTXES`Gh&J0=6E(Q~R;aVl{pQ&wJKVIelyW739@a)8fzi^Os8QGI^u zt1;#sCRg4k+{e6+` zC$PTW_wip_{`&F0je*C{qy-x>vGsKwjY>gA%IL{=Odg#o?;}>D{sh!d4YQrW9f_Ws zeL9oTcB!vV(HTsEHk-4vEzD{$>R5m8^=p{$jryMRE$4gm-ir<9wu^oC2z&pN&b~gdhM)vBH<=5|OR`h4OUeryeV5JV-@Uo#@TtRl z-u&I)&}Y6}57Bn-BZ8bEpSQMN@6$U%V=uN{PR1%(?P+g64zCEU0q^%XT;E2;*l2Wy zrsk$PLN`RT9c#}v;>OU;>o*QMlAWLSbl>b#47y?DNHrQu3W`dL3QDlG+GzanF9-G? zIPfRQeG)XA1kJ(?vz_k7sI4;^qOq4I>b0C|IsW;nmPR;Z!7VLqIoEFBl_HDv6DP*5 z2IX?2amx54N7DEyjfysVd>Nbna@zPZn7&LI-yFbQZG$n%&~@yF;ZUlQgdNeYH*Z*~ ztp@jAWq;Z2_AtH-6?&$4a-yPSgoI+y3ZUa`0$5k;S$l}N|0+gir!zVxCd^_7T%j9< zH=7#+SUA6=w2b2Ey2r6zyJI3GnbnzJ=c{^Lq_1is&AE>o*TxKo>-7QwNWu4+-BG3)|y{6M^y*TuG>~R>bwcG9Xo-5}Le|n>@zcHqtv3{mE=&bd~exTB}%W>XAs<6ailqZhi+wbUNV^`lp=SVK@P#M;bMU2}vv5NqZxy~W|Ua;1l6 zgdR)={mRWI6E=j^?hY)CV`=UJI>pk5vQ`J&;3zwW#+HlSy5L~2V}fG}+5RtgZyw}G zn%#$GA`bwW0223ARRF3`-Cfi7cK6J1XL@(IyCiohncS7>OJms)lGhH23M;IT93k0O z7>TUzS$U`q^Gp)M5eGjSvi|6V3g zsH+bi4l0apRG|`ieCPY#`@P?LpWKBuUoMr}Svbw{TpJa~FmxNtO^+mr*}2icfNcx& z-ln6%V@C1-nU+SlY$#3IbT-;%vy>thzlJtkE+1@ewh9H(|4N0;&3qY8!y{QTXoE6~ zuL=c#s+<$0(|9S7XmmB7fCrKDK?*`!4@>RBnzl^NkE;s{dwWYuN&JpdmGEL4)YJ`D zfkS*rfyO9$?qKy{8%EaO(`Avlbt@_9TVSAU175ua^8%VF`7+5%voNykdJjM9*DK+e zJZe+v+9Np@uGITetXE@*8iD%V4U$de#5~wuAy_+Yhw>}BxVUG%6bptq@gm>9WQdd+`0v*ej8(H zOP7+9u|#o)e+|9jLy$Vbd=95CdFk5na?H)ETr5x?s=jo?n;`xuut1 zP6wTR2r(g;e%YECC&xR7gsJ`wAn6ROx?P5$rHbbSKMO* z`qQ*&|A_A$Hnkm1UFi!xw4YDtSJt#h*wO~zvLl6n)LH!CI5uo3`_M1;*VjLI{6Jp+ zV10f3;K6uq4ydu*f$s%P4tZw3D0P0?k*u*o7RN86rB?!k2Pzl`OH0vkG+L`gqgXRA z4#2mHVjNJNZe#k+L)~gelOcw(rdh#64m|=pBp?(jRu_jt<^2xLV77|`VIaZ7F7N|eHfpm= zE|*@#W9uZo81jALeyle2UcmlW7~LuOj`6EM{hK=g*a6`7~e3=~7cX-v59?zr^`K zgG;X@U98|D@|+8u*8=_ox*j^u7p;{I(O^fTk+FX;YENNj_zXK z0#I`UP$SI*h_oW0&R#0R1W$9jJ%p#F$EMK2#HtTH3yIyzaBRV8VaJ$+U>M_A#+`5_ zbfAL=4<794khos6^(-`PFc?YEDVlZ>Jw&+@unxdy7!2A6t=53Veyc^Mps>6=9xKkQ z18`$h8IPBjSHYZUlROv!Yw;4%PnH3xXN2#6U)OV$tmC7PKKgV`*T4V$iJeK&)Bh!& zaM|syGN@|^i(kk~wvsR4#pif9jDgUs9j-ZO3pj-{3ZRUJP%?~|4{8-R<~6PbYLm(e zLzSF5{mJsvhbX^Xt37^90_WrH?cJR{>hZmu-RLmGH{h+ng0&rE+X^$PrW{sCb?u#9pNBmck~? zSJ~b!*C5xhJclI)plbmn(?l%RXlL2cFe{KD8IV@Pq{=P4w6Ks&%*EqzT6u?VqBY)O z8=Wkx*Hy9f_`Ubu+tN=U7^`?atwvKEQkdY7Cm3RqSIU?8N9UMF3=hsxLf~I|AG=Lj zcbv5yF@YG|V)mqy_obHk(;P$GA#?X`270)5#6o@A9gp&kj+_Ts>tep1Ld0;2i6PR| zd||<*|A#M*jZT?+Fz3PBGuy%Lw)dL7sKw!CAf3SAc9?A=j#@377zZRavF5G29^MZT zzFbwj{ox``E|NQ6Eo_F3+ z*+bsPlb$f)j9Yvq-n+ZI7vN~rvJL1VtjY!gt1cR?wx5Jv41Ed|178IKD${!^TWLCT zX5%%5nxs$@kKW^;D+YrQ0tKjsFf@9#5-P%%au6h|9z`IjW>zJg0K-S+<%N~^-&gsS zYcIa|;*rrmZNuPXtREedM~@xHuV?K*bA$;`TbdC26_igqRX~&ObfT21SCUQ^)MP-N zS=wL1y}yooUvRq%h4sQgp>$AcIQP4qcBc(J63eK|K8A%1TD-s*7xH;U33+4f8jEDOK#0Ha0ejK%zTE^TVWFDF+*^GN4Eo3J_+clfn#rS87aT8I}~tve4e% z!daqh^@u1l&4!ufM?GQ=@;lFuYfVSgOs^qJ_Ep>|AqZ6)slO1o`F!5%&F4#7TPQhT z6}+lihQq+Bs1xC|w2W*LswUFd z6wka~Cpy*PTs}U&zLK(Xs)P`%Z%Ge=1Woz?lz%o8uq#--iQ)h@B!hXN!{`o5)Z;od>8eG*^9tgxxRj|%ekFk!gDr&q(l3N;=e{Fc`M-~thHugt=vEd z(P)aVvb>}^bPnRr3#vkP;SgU%6pe+d;!C^zIKn!$%+Aq@1W`>K47`fP(%$AKaMc3I zP}oF_38%KO*a1iFXg0wZX#ql3g9NAQGKHE&WF2toIa;s?MvYoPYBUZme77y3{eWAjif#d_xkEtJ(&xypMENmRA`x(AE4#bBYL!0LtGl~pgvH_2 z5WQHMV_p$oRjYlC(N+9^Ch)sbEG{i25=%=r$t`fNKW%yM+^exxDeDb zYzKzJGXP+#yo4#5p4u>wL|}-4plRX}wCEya@uYpngKg|_Xe^LLt^>F$>jr~QmqCt8 zhh)7BOmTgE>yxc5n3|?Nx4*x&1xPU-KWWUN$}DIyuz1_uEZc9ax(`7`nzy$m^Y${z zXX+qc1jTIa=bN=UMm#t(_F)t2J9=1>d;kwwaL>$v8iW$Ey&Z|H#N)}O^O&=k0;Qo0 z^ay=OFyk0dzn-HAwW&Szd~@)$y!@|GHp&xoxorT+d$d+=d%VyTqVXR2OT-SiFwMC+ zyU}S-8CroJ)VrQX+r{g|1x3OF4bOcB)Cu6{6kdO!3<;Iu z+o0RG4i2_SVJIN9)TMYmP9QgbEDUTF^<71MUlRlc9QvSy03p3aAsQX#kJki8V0d{- z!1?2~%4_wyF+zWW11CD%L`@1vf4qh|1nOU!g)#m~+V1qYm{}DvSrzzU%{9t9NSF$u zLHvrtHr6qc8$ESlK)jE3NALHU04m}~vMrk%1IcUa5N43EIYnF6J?IaBGd1Ct>@x_S z4Lo}EaIei{w&E@WXAr|C=v=*w#d5y^`H%5FAAngB#SO#up)A|t4wSH%72CE8HT84gG#+#E@DhV z;6#-yj}f;JTa_V+`jBvO;)vm!PY$v?7UPfaP3{oE9U{1aME1GOil53j9Z-XVrFw~} ztF~6@oXVXmJz_FWAo-x0?@IOb4c}_@xUW9W#+ItUWjvN zOog+`bU5IT#_gU}#?vGET#7XrY($ba4H%d(Hc6yaxGiIqFfz_TpXsl!;|Cl-b&N1t zHCYmTfV99Aw5VH2(CLyyU=#nW?Z4Aye#Q}KT`uNj)a&IG<3lbj6Cs+_&TCbT>+=Ax#4mjL z3$J|P%eU{ES5M;>6G5d)fok9r1gbV3FB|8nSG}UhLj@kfYbln?TWedT642QaK7dv# z%2H@9B+09ZjeW!xK(-0?uG9nj4Y#Sqmh0flL2VufaPjM3zq)+&mtR-K*MIp+`sz1c z*Tgs9{OI9FZ@xK^l7CX2f>Q#XK(Hd2W~>2lY0tmpq|j9+wFiEXCY*uGGEjF79qGwL55CyOdj4!IwQ%{O0So zZ{2?Vn-dK9PtyX^8KgYdYY?+Yjno^51Yx*{le4*9Ef z-K#**(z&K1oFu10kD}lgLFo-0lX^2G`4XN_d-jkTFu5OO zGVk&>^582#EA=cawzTjNDjOiqirdX`Km6h6KfnCw(GqkroPP7>=Pti+QDtx5TorHL{OF@gH(op; zQD?Q9jo4YN__RbJIREw_jYr(k2nYfxAxpb=b1J%DPY$4akiBFE*w08#e(qwW#khQ7JT>NZINTAa;t{wWA~q3GIJ7#s2T*!( zRA_ENvn^8Lvof@Fo{|A@f`hl#=)-tp3N$WN8qH+6j3MqAFP|H(i9?%S5ubb3@xkIZ^; zdSo{8QfSu{+7&)(A5CCli@^KPbZHqL%koFx(YWR1?`+&r48z_=i_U|~b@9?=FMSWS zZ9cDY@4S<@9q|K_WIX-EryV&y`n|Pi-W>fq^P8!|(LXKu)9!Qn9P(V5tByb5Kd9Au zBhKZHSBe;FSm{~4rn7IjbKCnDc!Zx1`}%q;iLVA&$A01GfA;6Udi#jq@wmLFZ2bQz z>UjyNgW!?D3xFt)9XM$4LS(ehXKzDQD(H@?IqJf4x2l2nVDu3tc1Pfy?mpOzfvsdR zs=&}HY;A27^uAk(0+6^=sgw&qG4e%>Jik8>BF9BIKzzor3m8YhHFmKr5EsiW)`gpL zP8rzs9RYj`*~w)CD-RxQR{C;?Qr|3vlaXyWek8oEFSa(C>9WdSob&q z7QnE{h#2k*3qFb-utCW8`*%%cAHW9bxHf5Ry@>1wQhQ@9=Af?`C*sjmQ(n1oXMbvz zQAY9RBh|Xftzj?dX*!aRcTyj_Rip*DZOORFI*KKY77QDvmMbqeBQ4do&+Mz>US6px zwWw-KA$tbH&`J|cL9a_OcG0@PCI($ammbkoeuKfk>2XecxNpymAG%aN3Oz7tmXI<+ z%SUqF_`T&ot*SDcSs}NR+MGwig;%oXnOAYN@&z>*WF2dZV1oho;ZXSj)C6e&=P*ck5h-0dz-hUI z654=}h=`1UDMhi>67DRwwkA7OCypkPl^SHRwW3dTeK^vKMW3(PG;=*b?)rSiVpC_S zPOJ`9pO!T$Q;&$rE+{E@P}ML*D$qg`7@uLFJXTOM$}=~YF6F5(`zTD7jm1=o=al*h zCF_EAgJPtTv>546oU30&oBd)6tT~&VvQ8*&WGyrLeVEj?4n}0XjvvTewzSlTJUEuQ zOd)4rz*(sT0+hF8&CFf~^5Nt)F>|3RA&8n~!&aDIvF8YQS?F=+QJMN;1+BwmE`5f(SAJ6HnjXmSsugps?cAMcf2j3ks2wOL`zs zsT>JB z99S4D@Q)-J8W@3KB>+q6odkj6oKkh6YQu+(BUHu9xl z6lI78OZg2g%{g#6AyvU3J~%lj-cY?c3Kq-Nxe$TMA>3yGDx}7_Tr!{agVQ@;QYo<% z>ccMXhe~2w74%ZVxOhAWbj@j?E>pFvZ#0uWdxFAuaaz_X>4N zWrKVJtW5qVnKk*rATV-<5p-o8P$K4o9M|e+13VG}QYwI|!4MMy-i0cbqYDIdVs6G_ z3{Ch%DnSr0ULoXmvA!~o1O9}qD6A&mefQlx+=(SSbl69VjPZ>db3%U?RoLkZb5kGa z&!fygk1{WMy<#6>Dh70VC|6ysz&WwHIzd4aiF)i75h}=!Ny}Dk1A|q4&|B+08=4NM zbCMqzlV;!Et_+EAr$%J(7WrG*pwUh zon~)75-Gw`#R>)(fBS=(efQ&yT`kn5@bbbzB;s~|{PC?@somW~GMU23uu0MZmA!Fe zRlITI$3G5Vdg+PdV=DP(vXJz`7ts2!?R=Gv7Yn-6S1nhABBkQ!B5CKuuv=w`#wo$Q zR?pSv-OZ-k-7W%|FXBV9=~}>vhN0rU3L>A1_e9ox6Za>3E_0O+hw|kX%VyaR%9K0OC(`7&e_~l1r^*knF zisWP1H8QnuKie*ELCQSZ!5vnetZmRSoE~slCzekd^)a(ZA-oAzUA>vC)ktP}+_38S z4?ns8cunJPTw6&mt@?I$$kih|AI#6=-t}ht0#3Q{(xs@P@gKhb;b)IetXZ>OIb`CI z9_oq1uwQVKyYna$kCaZ=M)I}fx|D*hd5Vd5*63V9>6g$Opse|Tw;-QwG+Hf|%ErVD zeDr%7f9~ABd)&L@9Sj^42uBO?rOQ_qC?x?If{}HNf9IY14R+EV6a)S;?l_11gy;i6 zdShg9!3+$+m0ZA}E@IF^L;UdrID{X3Xr}i1UFO@i|L$k6skYC35OfGS(xBG@N6frrRiHwD#x^B<~T5|o>y_ducR2tKMT3XBDo{P7AYpAodh5}Fkc(B z8q%hKX=+2O+p>W9*qwQmYwYZR*D;UWjAX%)LPPMx^x9w%D|Io1Srw{rSi8IfOq4;! zCiDlYXV4dzYoGLmDAB6>pO}`>v~06hBgsK5$v}K=-qW|iLW1lnw7GCR-e|z0gmNgm z9Ef@rTex>Ve;4||UaMQjIKZOfv$b}}XND8^@#R5X&kAOMNy0+n|NmrFUO-EoOQEHx ziF9ItH1z{r1cZdt5853Y0-#KD5#Qb{XSi#lm0d66+1M)sXI!HPh7ZbCC!XRt>?LDpr-2JB7@L<~X3i1d3saS)u!{ z=no(28R>0?9&nu=VD|b&+@I{>iJO(i4?H0@mr_R5W`9r~LSL(2 zu5lg1iiat&35WpJh0z}vSuT~@uc^-M?d|HI@4$H07TfI(nU8aGHd7B4xn2)ThL4}K zHDG^lG;Dw%P5BQS5t>jm2tq8ku_40Xh570khD5AG6B`lrrwGY_pBf7-Y@#8PhW5+( zK)>5T?w@WT-|u4fg}WIMo0_3$hUy3-rApZ_oZ$r`qW1ggw9^WsK|(CAH|^|Ux*YVJ zWFtb?C2KO^MY8KFSV?IHWEmnp+4LTQNDTNNB~uVs zuko-8`{UI`wz*ACbu*sN;edwOWU*T3wwqQwU?Pn5JLFIlav3h*4jzwb6>b1sI5Y|m zWtV_nyaPJMG#Y~B*~w$HV$v})VdfF${cVf_$|@7U_T4lLXdqx}niJ0Eclrx}0Lz^& z_|b^JWDwl0&5HOvg4s1dr&#+cMu>APs_20kHa{+ zsl}LajZuJN(c>u=@j(p?F;rVai;M~94SIc_!iWd3vNS}mGF%qeJU>9*u{P-F)DJ2Y ze=RK{p4o6XHa8s(d>D$b9syp76bD`Eio`=n1aP0>fGkGXlD4d$69z+M_VEvd3b3Dw zFz`>~kq9+2Jr;A}Al7R|Cbi9$O0z3w^7I!gK!l z`K_(!OLIsoltvoFB}`6GwM(#-V20yfw%gxW{EQ$EM z8BAw-)9IEaRuy?KM5`S~s~94AZPoGewPno1-2-{@~3w z-~4~xTeq~}(|0|62i8Xj_uqa}y5^6lEP#qiVTpNP5DHgBTllY*gNK(5SrWqfu?}+}xe3R7wXBDyBDv zqH;`!3XQ;}<&h6XG-@D!UuygzGnY}w( z?b6^$ARpz#qMIFGq2v1WI21aD>>Xxh^$w%sZQkV>N7Fd3e(Kx=j@+LfojuPi2UvF2 zo#{Av{mjCGK3F}HlG`i=`80kGPfQ0u5LA=^Mc{j^0Q~5w0DP!- zQ?|63hQUFY;8DtSdVinLGvs2}-bT2+8Fg=FVK7&uNz^@t+LS8&hSwv^QZO84OHaL( zFC1Z*R_kPMfL@A$e+Iu4;-BcH66mF!YP~%Gw^ReS^wBA9srCYjGv$`X0|cXhSV}nc zX=W*IO2kr$h^76(UjNOLvRo~oFDk>Fok#1qDAVrdKN#wy5W4K4mNAqDH5LM3k!i^Vf zdPAS&bowUhY~f-ZWKvfgx(><&koi8t9TcN}Khy6NV62p?JtW}9@Mv@xS0GwE7!WQt z9QW&R2=#z|K||soaU=rT&*QOUpkxqMBfU((8Qf9EP)pjn72Eh6+`?&#V8&BDL1M_YfsFNj;z_tD*Y~*; z(m+n+^_NkfFQX5WUautBgJ~pX(XV;y5S{s!5FVGxW#F`7(dpIf?aKav?nRiVd+EQp zC3Tzaeb{tsMN$zprmH0p2fn-h?(er+PN!Q5EdV8qMFyT&G`9Bm@lT_}9nJ0!M}4Dp z#I(9U*(vB^C~*!Zz>NzYl|ej(eKM+^7{3Ue(Zr;n*B-_%;vvr&zi5SGn{t4(a>~`5 zSvnCXbuv%$!Q9-zG>gt^1nH)cG5vTvRIe9@o8*kF>Bj<~-Z<1bqLZxY$D^H~bstP> zpF-JED4T~9y-+5Jv{UUM!}X~cTfR@EgnAwB|0pQqPmZxAxjS2pk1{TbmAJ=^Xw`b1 zxIm*-euSXv(H-~xb}kuPNAhJ54TTH%3K{<1{_Y?A;UE0Fw+d% zTo_scQn?fY04QZe!G%RZny$#FHH5=B4nPlHOr3yz)NPoioEdRZfq~ztfh1Jx91i1- z!Sk(VGX3S^|i};d&O!Iw>hZy;WoQ} zztExRC$3|+cbuq!v*TaBcJ*2lmQSa$r(H%g7Q85&c{OfufIy>-ItMm2ZqbwGKqt6< z&NUjD8X*R~IUY~f&CC>mVGt;fF$l=12F0K31lq6$Ys%iebXp}pnwU&Jcmz{xl{O56 z%rjs3?P8HEe6dhR()c(~3dtG#i1{(|ql-7uYlm}yESoeMS>_pvg-7&?; zBADQQq_ydnP$H9YjS_k6K#RcaD>drq6xwA0k^}{_4EiW-wqO-YUS5u`tfb=#$&Rdo z#Gw}NdJ<8oowJ$^9#EwsFD?OEq%9X<$Dp;21W{^!;3P;|tiS#Czx|^>dh=iZ%l`|W z^Q;1B$KqX=a&N!=_Wh}UF_~gb8RB$ULjuFXLD3aJ=a_oNj#voBjMH$FXOLW2=rROM zU|dn?gE|d;c*59x1?}*8jLikW*jNFfwn3fH%?vAmw3`AU5t04h})w5T)2ouWM+) ztJJj7h$|!JbC(w5%{|~{d(HUahsEnJ37ir~nm2CH1F!K)hoXtr@7`u{Ogp`K#@MlM@?rD-eZkf$%2JjS9S(6K zvuFCci|m8=R9~M;AL!~64gU0znNC0Rn9h#P^!(?$lFpi9c+;F96eCL9iVM%9n}=wHW#PqYq7znsx~ge zLFl#dJ~;vi8dvO1!tR@tH*(Q4&;=2}~4f?By;)Wt`MR0~;eL_ObX^_PR<^?&~(j&^D=ulgBW^)nbhah`Wa36UN( zcXnJ7SsRFaAGDhv@w}FCKf1r!wG(0;g7I$4+HInfZ|aB%ix&#bZQ{lak*}zbM{K0v zM)|#-F9MOxxIee7v}zk*Mm#Q+5K?RhoTFFThw))w#?wtLnib8+Gr)MbIOt@e(UhV< zE*K_1zkxRP(N>EEtWbH~9Y&)dxxDIFK~SgNt#=2jN&Bc(sa(7m7Ku@nYm0{QpxFc? zg7?wNY6fK?kEc5X$koRUN1Cx2|Wa=;nk~)excW3TFrW`12~|A z!0!gz>OB%^YL`a>(sQW*tuO z%q=jrfe0S0;P*{*Uf^kT58TV<+pLxjb`B04*s90pO`DwtJw>*QV`flo@9#2K*n6zi zV>FUT2ZkLu;cGHIlWaw!h(;x#d@R2EZOVLpXa|PRvIAk8u4a~|yi9WsR*PUoiUPLm zA!*>&4JfLoOu;KxDDlO;d-n(ux?*ld`UE{^y1LG+x&P3POjo{49R59^BaB5FW+=jm z!l7s|gImfO4Yvp`qZ?b(03d3(XT)niWn*thBRNzww|)O-8|o+49F-r z2vwr$7{EndD}XhuWn7OR?hXVuN-9HOWC0XGMC9|U(PndO;52wiyxvaBpv=a8mvfGo zH&}q6uC=k!BFF7GD~=Gv&_y~f>#46`CJ@Fw)rV;i0}nR;`*=qUR)x=paVtt41d;0* z@5q4lL!!%wjF;qrG0MnBt%gL$=DKWTWdu05;H)Mo!E0(1a}r6V0!}**QxvK$fF#=Y z%FeBtwT|KWb9nx2jc}o<+ODSP)^Tcwk(*6y+2`_LC}f#q=om5>;xpzku>0{S8-{*o#+lk|Ai>JE9<;WeO-42k(q#fD zQ4l2(q)4LyA|3+{I~$E^OnTK`=(237gm|kAM?yknDps%jDz5oes>Kb8^W#P`%H*=M zeKbt9GN9nWAw$!_!PbZPhBh^7HW_I^t1(!|X`|n5cX*_vq+}0%4-ieg30^+xS4LbX zkF>8mmz6kPls(+IT`QxqH^^?O==C;+@qh&aD%vHdAI|_dghS10eh<|`2qDH5B3%?j zCz5@Gov7>rY`9NL#Mbow{}fDHq`xqu`mVd7nD$$G8SZgj>5 z+JHhFp#wRvFSgZBW-+l9C>SqzVG3k@n1ukfyk5JRD!V;G9zJU>+EFGu%EU=_y{Emw zm2AP0Hqrgb{`)KV&Itp|g|dQEuX+@>(_sTl+ih^1ooyfu0HtwvI`OODcR0-3AQM7w z$T^%;KQJV+67S@URu^T7L_9VqAeCw)Vn7omZ18iuc$2uregPTWQn8hggz;+CC zkn~0zvafs1qGg=1$oFyjVmFL$VZ$*aMkb z$~Oj|dM~$K5XbmL$)hBdB=}(PEvtRMf#-iM1#i#MN0Rk{m>h;eRJkmwiLHx=!!GVU zFiVp?EWjNXwb@ue7bwJN6(u`zVhG%~u=6~LO%6*xXL{2}dYalm1Z6}Cot;p(#Ilu+ z>{ld_3d^EwtTZ4{Y^0Ct4Cvm%M1sUoO3IUwk*M&PT6idAmt)wc-6h)n;c7;rWK&%; z^OO_c$iaD76Z1iNa)-ll6hmGNj3e6ib~=t$QdYEriBk|s%=`~MN`e>B`YB_BZ;)e| zvC%iinQ#|c5TYp{RPD@>&l)0bieSrd-Fh*nI-H6b6RS-37b3sR5^CWKD*Gh?au?DT zS@9`42dFFzAEfNWjDohQIjMXMCDHDwJOv<$tCO1}1vze>gS^6?wf_n3_QIHCXT3MJ z0)L+N7q+VsC36Mm8D!yq;vbVXGto{wA>a##3x%rJ2bY7-+X1H9&^sV9-&fiDy6PO5 zk<)swZ`_gddwcPC)Ko9J(5O&w$5>+%2+PeSRHq<$N#RnQ+yv2}mZuCM=Jvs+qDZ3z zyqaL6Z1>sGrQM8_b~@8e0ip9)H23!M(Rkb-PzInj1rI=V%lOlv!7#;Qtxc}3akp7R zo=0nZPkoEX^^`y)$&K?!<&%hFf?+csT@Qg=84riBZ4P+y11$r$ki(*?568GQNWmt#yGR_PtS-{~5`GipxS z_E%AJ%4KLy3sf8;0JFs85eCf!o_$Wro0ikBOm-2=1ushQ?!G zGdKpV_L};iSAmqR!h2-d2Jb?X#U8BKu3b`;CB| zhX;e@eNnIOOkYkDKO4Wao8HQ2j}Fk)y@UJrx?*m>VZdN(SOFun0P~($qUX4BA>!ag zXXL`AKpnBp=n-s^tVLR`Vy!nG4?t4vl>(^*p2CZPr4`k?vJ?<`!4XKL0;OK3OQc^w zEw0wR2Ss=Z2gL{XEWP9B&<5n`W}OJ>#^aenS4A!1-v*-y2YGuG9b~1Pk z13TN2%iW$x5UnSdQ-ORj^u?slPg^U8(lQuAm=L5TZA&#-bq~J_1&bwo_jqq9I{j`s zdKd$P$A`Ds^M^-iNQI|IFQ6?bQX(VpBovUB-?S;}&EBfYBag6$VhmGAyQG)5A3p%E zncTLyd|S0UOlL;ev6}HYu-m~9O4RrM@LrV}d#`-)wMm^;CU;luqG>~V9I7}*KG0GL z)7<&s@it}Un@^=x-szF_#;R&JbBgDR$ebZlnx}+M%lKaV;uY_hsowj;?=jd?iWzeb zs~`Uo?oM$*^ZN~~dJ1yx;pOr{tgsD=C(Li=_VXadHS_zq%{-1k&mOdEJ|AotEK{Wl zd^=g%&LHfqz#L~;r>{oX)9M9m7XaqM-VSgIVaME0VlRgtr~L&H>}0almV-#?fR!B@ znSV^qM!35|<3Z@t*2J0gAC=U#pF)$yb~rX_}k3WNvyy}Q+U4>mtZMxn<< z-6J>(YNoHCUaPnDkvodoqdnbt;g8 zG!h8EjiE;OJw4h_ADNEvH#mQ_>c=1ladx^h?Ad2aDTaB23&LxoL7dybQNgNbK5urf z08;He&%H?BeS<+1bprG1dn)t2C(hGJBsOCwjl$Gyz70vSvU2}EM##6V`u z{Y^Z3k8?nUJ#OIvoCQDchF*J!j*UhEsXvU2|6{rj+6aQd6- z^#A1U(|QVlPzbOEPJx|RS{1PAd@{L!KPd_27yI#Tj?svZP zJKyQ|fBUz8=U-{7&aEwo&_rTk4&(@$3r^h2$xP>1kO`L9pp|MMMhcsG&dKLpx+Y0a zdEW2-?(clZFn;?x|LWi1dE49O^X6j^C-l6}<9WnSh+n$#@@rp_q%VE(<(rp=(lC-p zCKeEWM^%D&o>hYC1d#yEg0dyCtMT9ajoGKuCLEfZrOqLNoB|RjVrYWMo&Rx3&#q`z`5G zY(>Hs)J!U3jq;OTJdf*X*HQl$(vr0N1;vg8Rxiw-aUb(${@?npVSM+uHkitAVhTUfk`!Elfl=?k_@^*E!{CEv zmOMbi*z6#tCD%4LAComuhna~GRj{8(N5*ZCc&OK|>}WwOE>t&(Rz+VZ0x=5G9he}v z1_&*Z4^lr7^X)%Jndb9Nn=DV)H!#s<8F{J|Eszip^4_#L*l4se7*vz~K}`_`qk zTn^)KYlEU`H)urWw$`AInBhvurviUNBB?-AFBA47to?zR zW@>-%Af(fGFEophNUa=+JV^y4lpL#H`pK#w>SC&b%|uNdBP6MNX%JjTE>5dGk+6jBByGjWpi+PgVfyPruOl_(2Hl8h4L}G5b;^n`hXRE zfpC3yPIV%X7Sj|_Nkalt$ZSVMD-?EUK2J3tUn`Z~`}l)LAO83~MEXdk?-BFT24fU~ zzUGEA)4z&)Q5K;d40fUNgpj3pDG~B`M!F0WNeA2nyiXizuG*pv?Enwbh9^`|ofh7Z z2rEw20wOezi4{DLJ$f50cOwNxT9{xxj-hy#A#>#O8%c@MW=R6LGC(ZG}Rlz@L!Ui^J+@C*ea=N42NHC?+T2WF;WP4r(XEdORvS0x3>Fbf^p23{`fF1WcVx zl|^#!ED#)7ZZyKkBZdKk82Uy-miPDXviU6(2!Kd?N9FWtG^t{bEj)J)2BBA*gNY(Z zv_H146S)j~7$D1Ay-pEbFbZi7O&JfY&SFm+2Yy(scsL`vm<^8UjF_Db;yG-U>Dh#x z5?_{l!AQWOF4LlL=2SQ~6EcC)_9jFIcul!yjdt5NzxlVm#AKP(m6uEr!JUYDcE){*MJf-wFUzg5HW9P!Qh4SuYKbi!NlvYC!RiR&9<0H1`2euB*Rn& zygG=a&L;K8=bc-OoWL23!p2U07dS($W8w@4!cgzQc`&vE6&B0YAvda*$@O}CfBNXN4@~i)@g^RjdI!kza26iHPw|LO&rs)%;Su-mL39x8 zG&V)Yk+tjiJpAhzC4Ym)I`E1mC-MwVS1!ba*4ypb%Eg?r;hza$oMq%I7N$`yPZuS-f+ZiB%IeLB*wfl`p?Or=tJ5lV^$KzS;>H`;;XaHA`znHu? zA@=}aTfG7BrC9E;EU{{Ma1Jq4o?UGM0{ngp?kZOa2!MnE5CA^0ip1pszl8+6cCslh zUGfGHPJ0ON0Aaw|4y2$|GLeEk;QWLXEaPrbFaRAMe6p~DCjvpyNfS-51XEd0SXhGH z!V<8Hsy#S`63kYkoc#asq}9?#TMa0|u~t)%`)+@qT(r-^va(z{4f*=a-Y1z%FWm7N z$X1qn`|aDeCldSbqQrj}CC)^TtY=5pwr8Soh+Ukmbw}4gATduz=`t_T>W4Hm7&Ztg zoPB;KdY8g(Ls%yjS}B3_^~T6#6LJFHhq!(oRPS%1)_)VV#@6t@-LfedP`lqPlKb?%l_|dTl zvq?dZ(ZtNJ-@lJ^$~;_?vef}~H8Pvq4IMbS-q_v%TYF>sb23TzIkZ=xkos zictKD_F8`oMQq$VIo1(rkANO6$}$6B<-y?X+qdJ)9wsZy*W>7}>U7Aazr`zy6CGhX z6Q=mHGL`4k-zE+nvs%aLnzd5hsdUc5r6<{Jj^NYe1*A9jc{VK-njR)z{qo6yFqQE~ zacjb^r}#Bt*k|F`gk>McvsoxG^v0GXrZHnG{WE;~w=qioW)0Hmi=||w^{oa zj@SOIR(MuxJh@e-trI@cI)|+^88|akf7(jZ-)FVf(N{g$)e&PwOf7lCxeJmid zLVZmfExvH{J!nOe^gg>cqN!n<-b)wlW9yxy4;OfL`S9KEEHCss@r7DB{5{w*V*}8~oGWn*O$|2a1;XCG^@GQxWiF7nGNbIx_m6xyZedJUO?Ij#9IlUYHkT zV!(8GuGQ;nC?_#p1ur~eXRcQE7W6GsCfU%R>XQ~HK4D6q#RF5aob54_B(sdR??Hw< zY31R+M-4eycm8{n?r%M{bn#aGm1}88`vj~wi3lkmRDeSB%mQb_fS>mzMS?=p0>*-AWjcW?+#c zgZRm-ufqhtem!f`Z`{^q7n@}v_QWEt<${6Sfr>`$^z3d2V%oN~tjoVUrxeOnm|20) z^;Iu+IRP$OumAYt-SXZiKUP+dJ9Zc<)DkdxG-kSY$~01fKr289=2NE8_Q<%S(%ViY zJYLFgvxS06Z~N9~fdzWciaPPRoW`)+D7_n{N0kAU-eD5>53U+bi95{q@P9J04!uQ8 zOl63n@>KuN^f@!;>0ick{xdx1x*H^Hw;Ow0vc)1|c$!&9$>W5~lp;~P16V&O`?KiyLf4j)w}a(mFJmk`!y<=-K; z(zt7F@Yo%@=d1uT*hOhKZN5H#xSt$Fm%;071(%)3z4Of!%lx+}O$v!gLCe6%q1TkF zVf|raK{B*qrYBl3>H{cx|G-`gMW+Z*U2NCxKhNQuOj#jC*jAly# zaoT9J4?gP#z6h8Z8s1<)+NU3+K&D5j6w5vAH2^>+VApKmon_sl zp#htQg%7;j!_ba%ywM}#mLT#t3lVFRs~mxe$A?&qe;u{D2^Od9-bJc*$t14>SO%2W zY{cX6JQvHL?P1Ga-45!x!vGDfk1)yY$C~OH0vIekd>je|QGA}wx;cU4{O6WytyZ(y z8mP+S2m5UE}ZQi(c0HRFa83{jpSnT)90!AS*0m+5@ zgMZt{IXDmksnyN0(7BT@-del=S+3f>;{w#U{lN$G>E)!?K30_**XCpl-*cl6KG-Pt z+_B00nU4R<0Vvfl*D04kG;Adkz~*xis^%D(bdc(1Y^C=Zk12H1HbzzfzKcEBBn=b^ zkCRze{mXEGwr18lK;d^FR;7&={9BB#cymWKr?x4rZs7d-6Z$jx_(DR z&mqY%He^#7=?>?y!%wJ`ZrzFr#y%oYcMW0g7KUuy8vlO{ee+k*H`fL>q8Y;RV!jm_ zRNQ{E6pXtD4Ve6tjLJ+%)f~9v!4e3bntFMtBfD3xXSY`>u^@DB32Ds@Kjr_4RVsFS zujlvI^JwE9cnZSmwQ=VFv$WH(-kS$q#+^Es$mP3?3%(>I<~QH-u5l-yOPou(X*YET zFi2`XV3OToc8=0+d0u zw|c4CLc$zMLVI9PHe-)Hi+$A75rq;iLt@-f<3Rzk3s?(WMTlF9pGS>iG5o;xxVX~X zKz3&*7<^pF%1DFz=~@t}hGv;(iOKy{v?!R|Lpe7}P2R!p6NTy@zHHU}Y2 zPKkInlzrEx7Y^kc`GIIRL)FDo43qUzK)7ewt5?HJ7edy0?QO38!;I_0_kX+&X12p7E>$r^e{^QRcA*g?IZH<9%Ptoqq_iMX|2+1QlN{P+yZquN=v@l2=FeT zU-YIW6-oBX)*JrTmGhhs4L93}cy=x=l`1}EFj`EmM)vk}q_MExk`#|m-?-$H`%M3F zPU4Z!+4rVxX3orYe5pYm5HHuYE&hnv*9?QF4plHq8fjGLw%) z&Ocpvgs?W+hh{&$77GfmAb9W<3u~XQ(&l1LlnzXOtkq__onDj4w?ZC+OnhjDpE-u) z`E1iLF*Ea(+xT0X1Icp$3!Xf&=#8*Xpj`}%Y5}Jy+MLiK!oJ?@B2YkZr!WNs0Y6f1 z{HY-kL93&NoGwg!47dG0hF-DAh%xm4G%xy$V*vkx1pk8KX}3KfJ51XPPKRlGOth6l zFRB#8zY_PzUp}V_`hENha@V(fA6X264C<$%eyet_6;vec zHRC`)zC`3b5Gl{L0D>nxJTVu8Wd)&ru>N4pJndUedsQ;`dR$CU{F|3(&_s%AQm%SK zP$UFH;)!l@`3VF8?<2F&+C~l=%Iw#TiGSuxXbsY0lGGZwfk@HVAc+Cv5s43FMGiNI z;QY>!XC6^3lmkTYh0!>87&1E>6pNuHsXC}anP#c3MK%O_V>~CsH4lNa-9|yHETQ`D-ZLu5%`QDXqM-x!|=Jr;frXW7tua3Y9E9ypw>Y* zPcwqhQ9z}b3mU>ekv&QhiRew?Z<3B3Uw0)fMr|VTmiv_lOWw0c839tzuQUw_9@|J1 zX(n*QE{g?~4`bJ%>6pGXd2W=Rt0*QS3rH591~f8D&A}=ji>Fj|$&Y8^$paCY6XD^t z9%~qlO?1MC8x6RB@fBVuADG*9JSD6;3V0@ZS>_Q(g;8Tk(UX!TAO*^LSqRpddQcFy z^sS_n6mkh6kpsQ$h=+Wp(N8TBxnuX+JFKJE7`RwXw(Bf{j5-chxBE0Xy7S|msO@0k z#Z(#F3O`o*JJBn_t-kX3hm%ooGi5~tk&o6~3N4;7=*FrWG;IMiiODaB1;qnqx>GUw zs7Bk+=1jh|ZdiP4G(JY@;a{3>Es;RXYPHcJ8@JPd+I>A3NM|>y{v5)xtk=ppE&t7xE82sqb z=HpL(OwLu)b$-hKb$BO8VZ-paM%|(0F>t53-Fm>MAQYxrF5pkP!{@G_OD$jh9Np<4 z=vfm?OHNvwY*k_lX(tMmQJ@tZFPN9^cD!&I1q<Z;TBsRyMh#-|}zI_PAjMtmXy!P3~Gv@XG2XUo+p4Ryl?;B??_-}7URc2wAKk+|zxUBZpZK3p*S~|hW(0}~lWZ!FD4EVc zg)bQ|@Iye9bqVT12b#6lhrQWRwmR~$pu})cI1n~ft5WzmGg4`9C0qmDn$mlYvJ`&h z;6T#f+iOdtOM=#6I*=$r2D)%u)*&*r+j`5or_Cw5Wkficd}omvciK$^jxjpefXxA- z6|F~33gGnl5XZz~pD#GA%{}SvatK`(vLM+Zm8ZfpLIsW4Fa)rKV;;dE%cDME3@o>n zWIJp*Zz5NVg7!fsG14rGE9qn^xxBLK+E`C5=3vk=2whvi7jJ#<-@o(y@4Y=~Il_X7 zktZg+`4J_%M1C@JV@Yt>HrMXob!~1W1P)Q=gZ}oSxwGV#zVONyzckrda;hn+e$%#? z*%g*eA(}P?>8%#N^Z2+_15U_aO)+CN>J*$yWqH&*|8Y+pc*7W@-( zr69;(!aEH8Gw0D18B>{{#(bhO_W&rZ+0v?zPVdgayQwk{HSTNi_%Gx5tK%Q2!Vi8B zs2}EMTtsUuqBYp4OO-1lY)L;xfL*j((7;^-31wpw8z&uXY}{4e{s}Z;_?@ezO11n+ zFoY8jwr>CY&%Su&%8Rdke!`i09rvTyxJzS`0jMDJ7kJos!7hkzjsnV<28}T`69~v- zGoG4~Spj-r_^hMf;*fzAS?APn^|K#gb8j3rH?VMmx8b%nu()W2Rp&?>CzBX7zOAJ% z{6}B8M%J&(+QR}Vxalus81f;0N9d!U$0LxPq<*Q~HShhS$0cAK=1vwn*vVq|$W9iY zLmkKhLB?YYK192t_?RF_T4f)$+Uvopg{TPyyF?M2CbVl33Ba zw7F6UNmFm8DVvq+WHzfQSJzI8E4{UE+e;1ehUKAt!1Hd_WF-8X( zX{BQTDRUW!d@cyI=POiJWW(!rSpmMsurNX(Wov{@k4MO{kpBm$<7 zDMlb!uum#&`HT)%8RZ{CX;vv0b`RSKQ|c#Z9seA4`WL9vYmS712p7Wf&`~|S@QdUc zf>^M)_c~?-g$_cAVGKGtp@ce88sOFSkhUKTDIT&xB$p=)P;M#YaR9>_h}a_p5xrRs zBp(|{Q*E$yAnv|J1svbSS12M8fMDd}D;5Wlxfv0W&})%J)DFM5LuLC#Ew$^PuT~4t z(RPc-LV#CezW`~m38`PHHM+yT7VZzbjaucPQO25bP-L9d#Nb zGdgY?@@pF6(6O&e0i&n@+2BzT@PQbJ>8n;}Yp#WiL3W4&DOxNi5UaXa%JAMAvcj6J{a4Z2w^G1)ID~L}?A#a?L8GJDu(v3XH-j@n zswc;xe9Qf?p$MkqP` zt*n=r660C&(3?J+CHxtbkg~SK$Rc@yj-H7=dIZnV*=jo6=E4)y^o;n*E70+uqNx+Z zj-HOl6qvqeYU+zZKo5bEd9(^f8nd2U&?aTxjE!C zG)FI4m`k`LxmRfO9AGn8ax@@GdxLY#V=WUXml=1t>}ICFx3-wFiXFkcWh7go({s6h z#IfIJzKn>H9hKRkpWF^}{`?*0-PH`uaKdxu9+pZEmxICO;ds>V+`kW)4qLXc!pe%k z46!Wd!)iEO#71z7$>?Vn^IKRYfze(3EV{HfDe39a_W#`x@~Bc)mH!Yk3sQ@6P<~Ln zamMUJvU<(PoR%vhH^;8lep4Q6va2f;6>TZKw=w1VqEW&7v0S2W;2!E{f36HR};vG zj!M}nNKy+~sHXUt8qtQ&`dGno4XmewHG%~=r8`Lha7#?LqGc|3|KIMuH8_&%I=KJ5mb*(+G{GfhDk;%PKKDGn^Bom+1>O+yv!;shH-pNGeeJ=$cR;%;^2tf7_nQ?v^Mm9$ zMqX!AfTreJ{53Z7QgEN7gDGkPq ztEO2i-@?(~!YIt;N=Ao!Yx7>Nk8;1m;IV{mW;K~CPE43s5%^qsDB=?;L|}82h4hy7<{kOw zJlx;JETi~|H!`$~^yQif91nca7=Vg35)Kc5&~w<`N*^EEv5>L_(eA}P*#*|QrorGs zlr|bbg_z?sm^|`36i2M(D#|&&Qc02elq&OX6xM=f=|Q zG8KlS)B`h4rr<1bHqbA$gcs~hK1y9NwbJ>H?(nrz4SNAEpgbf$4-PYAu9&%#e#K!X zKuVAigfuZ1A~QUa?pef$SRq`+vRx`tjd&wRfgdz zn*=`$B5(P;$+Vx&R6q}7h1;FDNX*a<>lMTtiv=)pk*|=-jo~K&Clla(36Ep~!5)a? zY&H(uCT4?4N}%5|{G6omQ*J-}P0fnzgYbb>&}vi+6%^lSwg}>i=Ai212U(NQq!Y&1 zUb~6Ek#%6%$kuD=#kXLD(`=4nERi?MDkM?rKBj;%hH;?b;Uykfj9`~72)3HZ=s;$N zv6|ssMrS2sW-{7>`?TObji+#GoLEv1&P;8e&n&I(D9XwCCzcpRSy-e+v&4EWbOb3< z#gyt>g@s<+LV}&0?jL-?aq&#^UDkjT<+XH~quVO~<1HW$Y@N zju1qzgGgLN?L7U)??|Xl`yu0%HcYpWa zDb~;b&0iYri#_$FfASS#I&oR!UaJd)567V65+qF%(iz)I*_t z3)!x=5Y^X)eq?BH2E}Pm=K#+e&4>va-IKpG6>?qp?Ps{Zz|-jj!}Z%M`{@Pwnbs9B z1!dz#3JZXvjN}49?sZe0O<`ShX|@-w$Zr`y3h?qDuvMxfZ_JT_olO^)RuY^$VP0CC znq16v4ZX%rx&?j4q@R8Doc=ypMyC;O5(~lb5n~V9Qy{)aigM$BHG0zna=#QiP;^vf% z^V30)=TrWJvr|aS;sNe~kR>sO&S{;1gAI>0$xVDn?-`~t`C@(pb}q=)M7QJ%L<3QH zlkaWd?F`g3(H#x=z^InlI52IXu5()EfW`_&p-{NQv@04wy=>UlQE>z53~79RaIlef zv6<+{DkK&ktTIppXtROgB_AEWF2)p?ZGxar4Ywp)5YfWd|5|#WzD%|vMU$)o3LMmH zRLVInfh4?Tv`QX4(s2%iYi6?xw!S1FR|{C~Y6dzA!iRjVnV5&nY%~)hp41Y?$Y@9q zi?~&>Lsh?nD=)5qSmg-#Z@>%TjFnMtOXmUjZCv?Z;>z=nd?M{K?&wu6Q89BQM!28* zluKuqavAf-osmyjw|xEeckis7rd!6Xx@x($A9ucfs%H5*M&K=sz+9n4uBg7)_z-Y4 zey0`BXc47&QF8@!eQA*M4W>)gNxQ8>v+PnvM^4d3NfRybfnlKp6=>u5lWL+JYOAwH zZIvIQwaz_6YaI<6>yGH7_&SGjd-(E?LlyM}b_){-=1ZNm~d2OoT6_1|7qkS=_)NbX5-k?%m?pPv{C`^31x9*|yEg4r5&J6Kl+vI>b2hO%wBhAbSz&j~})*YiYj1Rn|pA;r-itWLElm9v=>I0Wj^74%{7?_A?hKv?)-&aFKa~u0c&; zOvW%KUI~<3=cYy(RRytcJOVDZI7+(1po6xIbXuxBm+%&gv$I5k!8kaG69ZN3BPAH_a%1mNBCo!G?)0`Yjig?-Kb%2#o`I>Dn$lcsM zf}hSmG@~HLMKvnG3_-xcz1=NYik?Zlgdk{_2oY62P+ru zdlSJbVEteWv3^Yitx#=%S@dESBLfKR=H?6|kGTMgC8$xa1h9;GWX59+J3u`MmsSf8 zajOMKtdkb=AF!4DJ%DWjPq+THb?#@MHpI)lkR&zb_-W7yCjosg2e! zt~iV<0?bEud@={|?t6Cjj(z;J`z3u#Mr%mG4F*ET$5hY)K!Ff&i2yhrE7np)bXu#W zB4`JMLW%Ggai<5CnpZ&Z+yv%?gWdZ9%yO7FnyoB*;Hc6xdVJ^3#A5e@nPX>ASsh^J zr;u{Oz|3%JFfcQ*KvW93a5$A(Uion2?&|UyoGUdI*9<=bQDVLI$8Z%;Y21Jl9Uxe9 zM29tFw8I!^6Xhm=u7(PJ-(6f-U?^vip_~V8w%XnpLphh7I?DMpo3PK-mv!!C_MBPR zgdD>c`2{N>j0j`25rs`CFr1NwG}6#6PREH|N&3Ho5K8o2WWOD1TZC4kZ-?5R+Tupk zp|(lrCHiJh>t)Q-Wy}-&dte$GD#4O1mAdok`w%P4FjOyLO1p46{KkfC+}MalWoI7h(3+6VRjPodZ3@o0fLRyGnfg?u{MrU#*K(DjAa^#4yG z-(>Todsx11p~M}w1ApD3pQJs(y5#+a4Kx-(4qGZHEo`S_M$h5?hZub7DkuMtduB`9=+gk0FZ*CfqIetP$ zCDI6EMgcBiWK z@ON!RFX5q{0n$H;L+K*y8WvK>I#^UjFv=qsWv2jNy3Ouz2#XL1=|%87>U3X5Q15vY z&g*~le_s2u*M9InulsU>kn`aHjoC{E)#>J@=Di_5z+-4HY;n4r4yW5?S?edJaYHIeh*yPDRa``?q#gYt=QnYq$VZI@TYWQN~MG=E`qxBU$HdyWhTk zu+BQc=90!6@G>V|ctj9JuF{Lk7+_qA%_V>zFSE921(Fu$E4R>Asr1g`JEhY0+jd{O zL6J-fGCfTaN)S$^qgFga_9n-d z_Bc)ThnxHX{M5gPp8iMl^r@0}V5A>08EI&8G-wMBdigp^kivydQ9@+(6?axXOx=C| z-K7H*99+A-x(yDDe4|mV8iW@7owC{IF*Xb?U*Z>|5(36oJV7GQIS$@m+(k8n;_e4G zilz^@kIOKhkN5WW;JddWK8^Al14ZHNm|uXB>mQqR9|_311t7CkD-k$YJb^KPG?|cJ zBR4*cOz3`8@%J%u{{tg82NtHY-k*A?ynIepxsS9r1jpz=7s_9;idoWW|$KX!(GBtG$ZnxUomb+{xg$BxOUcZ<- z%896S!3X!^;?0>MOn(D4FrBxO$y+Yhty`gfq5uUYC%+$zi=TN5L1=@KVb^G1IfI!y zgPBA1V3x1b)n=lnO=hn4wwpM$-6^z7e9s-0d3S%m+S7L1H1BQytEaZFBc~8|@`n)r z4na#HDS$7O9LIMG`G?qV=lkBP)B&y)r3o(8&-dNqX6#mr8MQ1|UWP0O4a^GuArDky zY>X2VR0((-mNIZbVg6~SL$sLcwVwcHVgTgi&r#o^ff(`=nuZFw%Ka-dUxzTX%2TZ8lecp$PtEJ?%Ec(r%wRILH};ps5!o#s`A^V-sgw0B2N_sLs6( zru=lvZu{BK?xxe5RbUXCaMN$%lUM%`;hGxWKw+17U-?L1dWgIYjL*&@6?Y3b6<%S< zxYw`4OpUltHB;Hj{C$kYzb6?W2!XNl?VJjmbD+-$cp#KM2k@&Hin}Wm_tCO9_Gma* z1~OrF49LVmff|fU2|+}pAk(cK$J?SOce!zg$o!J5}?%Bk@)v1}goZgLHz zCf_-rF^c%enzz&o${@&meB+eSdim1XGgB8YUy35ZOe$frj}UBk*K&+_Pa|F1I)n!^ zD`C5L#+_6@^_^FM6tLm%e&6Y%wT#gM#6DycVBe3h7+|xtzI21^`;LLD@0kZFi}}R9 zCw9IUX%*29c?V?{vfv6t0T6HmK8!S80fjzFUDFQvk(?bSw(-mSi&tdmVjs!hp{Qmz z?U)~v!Q&F{f&2j2x@F*pmzS58mY2Ks!frQ!zSLQgO_v*>OR(zPyqX(8U!g4dhON54 zgX>b3?HL~k+kAlfrBE!JZ6&AgBxkyu5uLqguc9DWKb12F2NH@KO~Lq@n(FFOx2)IP z6ih4Rx{`ysg}17swJif?3{1qp;va}jFw+2*L1=$Za@av*-Yn#bQ!#_6BrH!qd1i3% z%#%-l6)wjR5)C9Hus4Emp}HZCsFnfc2@(F$Lwes0Re5}OFBysKuR_1u+f%@U30uGh zK%IwH!IxAu@`Woi3k&Bzi$g#dYi1t3Wb#B097Kyule~dCiL1y!e;1l0Q^S!!4rw0^?^*ZaNsh7^+C*u=2@VBrVnpY7~7>~5{!WZ9F~WFG19xa}sfJr5vle4COg zcD9vy>{&?4UMW{lAkiX@3ntW(tk;xluJVt6d~4;_8$aSNUm71CzjRr#ruGB=z9LT)7Wdb{dPu5B{uJeEoeIx^+w{j5UV?(4mAAG_9Zi ziXi-p&(lOahdG6rwzl>)_MDNPK&#A0=l^LWa{DK@B9R}mMpnC523qn_OTRfi{mcu` zPEY?9YwlONmWbM>-pdoaERL%HX2H!-=k6RLK15lGY1ryW7z5s-iFDusSTQL30NX`f zFpFkmvyunfNe-!0twt$xZ+m-Z;|_<^q(Nxwhv_`v_W2BeZ795T@;Y0T2Z~K3AvnDO zM6m;2XZy9+YG@e~b2J8z^f^Wzojrec`jLs)5DMAAVVKRNkAS=Qa2cfnp>QNOx3)L8 z?j56|o7I7m$$gQD>Dga??pMF?1r9enwC*4C@GUA;=5T{NKHzgZ+OO>*dV;24D*DxqRwi|KKQJZeaft1>pCLfIk3K0T;B{ z<@UNAqOn=ZWpWP^ganYD({J95g3UI=sl${5p-c%ufsSE1Nsn|PKe9(?Vm@pRUvL28 z5M41yu0Y6-qL!xz)n!^|w-!I#+TGdAW;X9GzjyuJTT82JtIJD(qpofx zQ@L`rf}UqtwK+-z7X^zQPpmr7j&T1#-#MQHh4zpy2ZFLnzKxxRWCMrq~DB1c}vS&?puFuZbS(*euARHMUjtoZdGZ>9cJ~DIR0zDruXa=}4$Mdx;Wy}Dnh_ZVK z2BQeyN51x<*hswze^)IVW$897lB_m#ujjR5*35WKl8#~qW0*nS?+4vu(JEvQvM6{& z1#Dy+w)LRT1Hj+$;oX$?nijY=l5_d5XC`8w~_!3(aPoYIE zkCiE}QYl@MoMi_C?uV zX)BAZ)`I%Xe-=FJk<#3A3(XeK_bA4 zfQq43t5zxi@e#}v0W;JHhz)veATS_!Io?CH0kbREKR873bx9hJjVL0JW>BC(MGyyx z&Pj@`6}s;x$PUyU<=g1ff2Akp6d-*yhidCBBapud(SS=6?G9@Bi;t-dMP`yt#*rYGn5tY``7jagU%@paklX ziIE7lNw*87EV)u9wY9eV{;dygzWcNFO!5Z&i5v=Yd2UK@M zWX^d;z3ZAqO=y&VhIQ7p-%WJycgR+Jz<#Ho$fKT7F82Y_tmm7jfKAk2WAAqso470W zNWaG+DVO|#Q2*fQnX|Z4eBud5#>QiVLAMox5s_L7`hq|Q>1jygrW)?3GmE9{RmRjh z8p-4#aQXsby%J^+i_7bx#7OnE6Ll&9Rnd>6jCJ*ydI$IV6MCjXkv>#dj!sU`Ja+l9 z^OK|f0Y7L?h6aL2+i0M`H8vNsu~E(+?tyv=MGjCaV{Kz6MOa>HxG-7=2a3~agD4jr zkkw<8XQw8|$6{ks)0ZIlwV2!-FIw;vbnWJyJA3!*1}&Rx*RmOU@ES^CbB8*Qb@a-g z{P49O{rQjIc=N5l`02vU#pUJ24;GgeKlos2bz^&H2Su)+>KtWD)k?YA0#i!??gv!H z#mzxMI6giaJ=g{1{MOyomDTn2)x}%yy}x+-=8YRayZ)0mUVHV`SARgRYh`|pKRiML zDBC(dYUe=3ggQ!Y5s})-rZm*~NB8T1+bXJY39{D6f-yc3YZhh6G0^XIIm`lr*eF7Q z@&-v{?KPc#XS1eOL@;J*Q88}8OgNn$KcbjP8CfdHBou*n-no90#c9=kDljj!1=`e__3~t zBh_qz$zdSXIpDqtOl;(>4Y2{rOQ~6ITFuS80YCWFQ*p%zFU)}NaObcgG;DdJ+ilFN zzgernVc_3~ITa}c6te}%I4E4Mw$#QCl@H6JPZZSim^5wQzuTAEBaWuKBZG=||LxC$ z!SpeGwlOlGe`dpd+i>3&tFW~@6fRYf170tkJ$w5-Bvxpd2krA|)4=(u6vj4B zCxwS$A=*LE!U`taY{=YKc%w0nTA<)Bi*rWBg7PeIFqsVVLNV!+4#Q@oq}cj6Z~SL{cl?sP)$CvVf$w zwq>>^G5@C9x2msF&hU)k?;HN1;UDAs zIedRPZjxM_KZ3PO9~*_Yv5$mNb_zt>1C9PkcJ=niZ<;XgHUv z(B=(BG(`}_X00K}A{93#&Pq{n7wF0cv!`wY2-+d}e2RO2ALfId=MPH;iK^P~YcW#& zNq*VL*c~AXY&IR?uYac3o+k1k{fs_H6wn z$osae{wt8_*hnGM_x$CLFVmgXthwGYwoZ}hIkTf`0@Vo8t=Xvkemo-UA@Qlj{Xpd z9@VZ0@+6iPXoHOgWt8o*#A|G0A$0}UBcV-KgzLL#3m+azek2khPq3ER%JfLe>}JH> zvpbO271|7ELs9Dq)wF>7fs4c`vSajz5oAP0GeHau{JdbZ+a4crD+Z|d?74e@F7%Q6 z!l&}Qs{4%kyZedzRCWZQ*I?A3Xh+TF@D9ME*5~>w*%5w0eT$7ukG@g~lfEJ)*=K(} z_Jp>yJw<~iY`qbfB8{GU**NtRz zIJQkf0PqG{M6&_TJqT|&o$CVlX)~>Psg^&?*Nn`CBEuaG;8LxK1kAc|-iHzP`{56S z-W;}~LUgOi@o`Gx1m$`Y5qC=Bv@0H%K*(f}P5tC~Cvr(CECjvVqt=+e0a4lL3BPMNdsg58UUbtn^91XF4RjOM@jw%63pKO@R zP^05C(#&){WqgUr+phPb-|2La@6m!BS=YeXvuDpupFe*d`CC&%PBM4dYS8)meC&w< z0Sl}!p$S_TzhBjOPMbf~pCCK<^gB?MO z(T|BWYVl0Mj57Q=JfdB#hlA-OgaA4lr>mhxy|1_VOwoyS;jY%{a-5*8kf*g#jk*A* z4O>O=2=zPR$IkX%Q<)dQ9g1UMPqjeOAMd*SwP=kfgSV2&{R*7f219>uM-;P^&Shbn z#)ab1K9r2^PN|67FQGSI#vGsPy*j8WP4qyqIk$q-5~T$wftj2IpqezxLLKWaRmV%P zA`$rwNX&Rqh$HU-$AAulFV_?2jt~Qor1D-r>@jkagNvJ|+5!x}RIHUMO#_G`YBiiM zV$?vl%ouENbdw@5ISR278Ru~+Zew6cE5Z(zmd2B;<*;y4qjB=Iv3a3YC)w>++K(KY zf)me)e&)QDU%sU;;*&EwAJ(2TAHj%CW5j|k2W&vQ34EI{P7n8w^;YHuvyZZh;wI5y zH}mxP1-&LX2(WsDz|_>#*hsrq=Hk60P@x~ig)$!EZTJtgwf;F=g)%)xA2tt>(e=$B zK#qjDG!JyoNm| literal 0 HcmV?d00001 diff --git a/www/fonts/SpaceGrotesk-OFL.txt b/www/fonts/SpaceGrotesk-OFL.txt new file mode 100644 index 00000000..cb512b9a --- /dev/null +++ b/www/fonts/SpaceGrotesk-OFL.txt @@ -0,0 +1,93 @@ +Copyright 2020 The Space Grotesk Project Authors (https://github.com/floriankarsten/space-grotesk) + +This Font Software is licensed under the SIL Open Font License, Version 1.1. +This license is copied below, and is also available with a FAQ at: +http://scripts.sil.org/OFL + + +----------------------------------------------------------- +SIL OPEN FONT LICENSE Version 1.1 - 26 February 2007 +----------------------------------------------------------- + +PREAMBLE +The goals of the Open Font License (OFL) are to stimulate worldwide +development of collaborative font projects, to support the font creation +efforts of academic and linguistic communities, and to provide a free and +open framework in which fonts may be shared and improved in partnership +with others. + +The OFL allows the licensed fonts to be used, studied, modified and +redistributed freely as long as they are not sold by themselves. The +fonts, including any derivative works, can be bundled, embedded, +redistributed and/or sold with any software provided that any reserved +names are not used by derivative works. The fonts and derivatives, +however, cannot be released under any other type of license. The +requirement for fonts to remain under this license does not apply +to any document created using the fonts or their derivatives. + +DEFINITIONS +"Font Software" refers to the set of files released by the Copyright +Holder(s) under this license and clearly marked as such. This may +include source files, build scripts and documentation. + +"Reserved Font Name" refers to any names specified as such after the +copyright statement(s). + +"Original Version" refers to the collection of Font Software components as +distributed by the Copyright Holder(s). + +"Modified Version" refers to any derivative made by adding to, deleting, +or substituting -- in part or in whole -- any of the components of the +Original Version, by changing formats or by porting the Font Software to a +new environment. + +"Author" refers to any designer, engineer, programmer, technical +writer or other person who contributed to the Font Software. + +PERMISSION & CONDITIONS +Permission is hereby granted, free of charge, to any person obtaining +a copy of the Font Software, to use, study, copy, merge, embed, modify, +redistribute, and sell modified and unmodified copies of the Font +Software, subject to the following conditions: + +1) Neither the Font Software nor any of its individual components, +in Original or Modified Versions, may be sold by itself. + +2) Original or Modified Versions of the Font Software may be bundled, +redistributed and/or sold with any software, provided that each copy +contains the above copyright notice and this license. These can be +included either as stand-alone text files, human-readable headers or +in the appropriate machine-readable metadata fields within text or +binary files as long as those fields can be easily viewed by the user. + +3) No Modified Version of the Font Software may use the Reserved Font +Name(s) unless explicit written permission is granted by the corresponding +Copyright Holder. This restriction only applies to the primary font name as +presented to the users. + +4) The name(s) of the Copyright Holder(s) or the Author(s) of the Font +Software shall not be used to promote, endorse or advertise any +Modified Version, except to acknowledge the contribution(s) of the +Copyright Holder(s) and the Author(s) or with their explicit written +permission. + +5) The Font Software, modified or unmodified, in part or in whole, +must be distributed entirely under this license, and must not be +distributed under any other license. The requirement for fonts to +remain under this license does not apply to any document created +using the Font Software. + +TERMINATION +This license becomes null and void if any of the above conditions are +not met. + +DISCLAIMER +THE FONT SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, +EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTIES OF +MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT +OF COPYRIGHT, PATENT, TRADEMARK, OR OTHER RIGHT. IN NO EVENT SHALL THE +COPYRIGHT HOLDER BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, +INCLUDING ANY GENERAL, SPECIAL, INDIRECT, INCIDENTAL, OR CONSEQUENTIAL +DAMAGES, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING +FROM, OUT OF THE USE OR INABILITY TO USE THE FONT SOFTWARE OR FROM +OTHER DEALINGS IN THE FONT SOFTWARE. diff --git a/www/fonts/SpaceGrotesk-Variable.ttf b/www/fonts/SpaceGrotesk-Variable.ttf new file mode 100644 index 0000000000000000000000000000000000000000..a1b2e6c26093066510a31147e7aec9abdc8d6c5e GIT binary patch literal 136676 zcmcG%31HJj7C%1o{kBaBX?ms?ZJIW1(j&dwG)>bsz3+q4mX@PZ?wfK|g_KU=Q8$XA7j$AhR%*jxyw6y82frJW83d; znAB4}?~#l^#?PCIOEqO;Jb0wlEI)%z>|{ZGbTNJVy8Lo4PC%@5jo5z@~Fz6P8`)(FS$-F{Z6 z>t9TkKf^cn&d!t@l;`EMMqcuK6Yx+#DZU2@T#7(C`ArXa5b*WLODE3?cu3?#G3uZE zxxo3HO=H1qHbV<|W5iQtG7FG$xeYN)x>Q2zu@>dY2Ur0&`(xkA*hnWct@Fa_Tk!TY z3;88LrC}paHKEBUo*$F;;bB5eCbFrNF9nV|N~>6cbPdas&S!3E2c9LYS27}8jqlr7 zFZ&Abt;{A(!_&-qbZH$+MY=GlniWYE ztVG((`lP8yw;7?Ax%ob3ls4ggHtRz<)$E&*AEi+7-X{+L7jWc7_C+4+k| zH;YY_VwpuwXHKb$6^QRz;BExoH7rUR0DLOoi&+ZFa!4}~=U`@O4d|T79K4?OAw==p zSUN%!Ux{=pnU*g?dwzy`Bj8)&-gh&>VT9J*Z925ab%fQ+u5nsQtWw+VQsnwXrv# zt%) zfJ3t54J1SUf#k_6QzTcv5roWe+ zCRsjXAQ|_D*&{DOu7kl(4Fd7=RfP8tK1WdbJn|*_!fR?@Xy6v=6C+2V4HNh_wO>%* zPy+Qg_iu%O5T*a159K11Q$HFL7NS42LnhI;eEp01*lz^U|NL>N4|>BUg+~51t43%+ z`?=9C-Fy$r#$JOX-x!qaNF6VL4kdk`*1C`Gs$y5T6ERd{{{U9lZ{ zjKX5J1lrlfeuN%chP*a1BYTZi@cjsPuuRe4&!7*^q$_-JYasii=hv`kp*Oa$-hdZa zUch63<6XLsB?{d^@1#R0emCN`%W||cy(>DVmtThb8v*NP4fIX{ z^8FH?MhbX;8P6mY{s}#mcgWG`d%PO|r3%YcpJ`Gmfug zd8hF#-~$BWgI9OqDPO35{|z1H({UsR|10k4@Vq+E|LN6*TfjT-yZ7nUjo#;P=tlId z&eI0JbfiDM!KXvLy43q5eM`@??go%*T|hwAbuvw6SM_~U)2CqIV;i81NrQ{a8jp^I5Tz&5ngZ;e%Cmu-dZ z20KYSSF$R4dIR#wL#Xh5_sfs$8tFXfY1lZ@0=2CBV09ARC!mjC9SuV79TB>(8~IUS zLRYRqyTD$9&P~Nw(L_&g08SI(dcX5aq03<3iRS<_(bF3MC!KBaeka`w88S&Jpq~Pb z$7gu==PSJn{k0RauY`I$>%Ddzv=>vyVp!FRyn3I+6JhV20w-J08=&vV26!0(HUR8k zC6G-V57?KeM*`W#Ui(;D51xL-Y6(v^F`IwDSd-3s(0*iFno-}yY$<=63H!lIlc>+V ztW((7i&+ob5BYeDC39)C9#kLjE>$91nV!oa2Y7k|@-0Uw^M04RRC{D5a2JYn)sXk< zfUnUX{s;Sna0Fe1JN#dQUp`{2gq<&*Yng?f-hjL+q@~PB-zi^y1onhqS^>Kj^~+?< zB3z4cH>-dN3_9i!F+|&vzpoH=?$Qt>>|7O zJ9JqN+KF^p4%(@Up56ewdW2r@cRVrYRcQ3eAJS4gR}-8+!7wjpE?GZn3wmurhB+kJ zCTQ0J@oZO~^e(=aKY=p-9Q4WO+CVX}zxwWD!0sV{67{ojIqKN=`4eg9vdux&BFv{9hBraIqLzOTo31aJJQPtaDP zhEhCB{nL7J!8iJo5M+cXd~)#&2wsKfhwMDePiL|amdi?+n@wRe*jl!cpU1D{+xe~h z0sb!klz%Brkg}yhsYI%gCQAL%KItjxi1ZIRK-S5ja)dlVPLJi;PzquQpz1+-3ZaX}l@SWH7~>jHVQm z#gu0%G&xPRrY_TR(>bO~Ojnq$G2Lp~W4gn1m+3y!gQkZ~kDHz}J(V1p9Gz@RPD!>T z=OlM04j44?ujVUuy=A<36e*Qt>F99PXsA~v%ULJaJ6`RTi z**dmc)Nwce13$oz@h|w#Ql^CU3bl@pN&BVer8lvJ5`;R2qmE|TBI;P|t7EWHXN)o? z7){1hW2Q03Xg5|HTa8nUi;ZiH=NYdsZWndjZPK8Q5vEvEg2^Q6SYRp@bsRFSGHozj zF6wx@>G!DPJ*GdRj*pr4ol?h1$y1ZJp^ky5V>s#<=d0sv)bXkp~dioc_8esGvxK8mK#5AG447k}{ihe1cRM>R(u_+at-AHV+| zWAC>j)FL(Lc|Yg7sNA#annqh)udBxfqYq_*8D?$FyKmBjgM0b9rxvBhi|Th3NMAFN~R*+#a7UBWJ9TiMy{26iL6iS1-J zvt7KB-Nx>NX8Z&DBYTwXN4q@Bo?{2tpV{;5FYGWof|=*v+3W00=$en%#~8m}Vc)ZV zVJ!VO`-%OByV*&u<>R>yt!dyBcnn*@USLbvK{mt=v6bwvY!!Qv4YQZoYW5Oa!(L_Q zu)nc$*;~+DZ?p4ZlV8BzVH?;R>_YYrwu!yRE@JQVozkD#X2{CL>;rZg`-EN2K4sh3 z=j=+X!(AbL!Jg)c>?-yJe^UCEeaW`7ulUo_adtKPn(biUvTNBt*>&tY{(>CLu4l*D z59}$Pz;0nbvs)p1yV)=7cIIJw*a$xcY5N_-@(wPsJGsp6=7H=k9>9;wm$7?zFuRwJ zV-N5U_8<>s5AiUzmxr^5F&;m{BiWxIy^rx|_BeZj$Fe7RJln_PxJK&dZ^{|mBmaS4 z$S;%5<@@=w(np>!lKYwX~VTmygJ8(gtanM& zJpO}x1^<`4QwowFlLF;Oq;c|oNh9x*g5@VAt$aWVlV9d*rF_0lD(0)CWIimV@v|i> zUn6Dk)smSnm7@5N6w8-O@qC2@%R`Fh*GL!hYo#sxM(HMgg>*jOCSAy{k~Z<}(nb7A z=>qHM#$Ejg3FBis19ayI{moWtLi zv-k=5K7LZZpO457NL+qUlH`XZS>7u#`HzxLJ}8ZspO-@9LsF>xqV#*{yDO!?%b!Vm zFp3XK@3MB*!Ma!{o5&_{Gf!o+@+pP^S;$Q7RN6BG%SuUrC`YbL36(ttjc`fT)1xfO7*L4){DUFt9ps zPT)}By1;7!cL%;6_)}1N&_K{NLH7lHr3uz#X(}|+HN%=)HIHdN()<{lA3QI3bMVvS zg2tK0Z5(%C+|h9-v^MQD?KhkY6LV>k~F36Bjohv$Zuh1Z8q44)Q0FMMVAx#5?EUl)FR_#eU_ z5C3!cE8*{ke-;rIQ5-QdVrRr15f4S|kN8W(YZ329d=c@lNFKQ$a&_eSky|6Li@ZJZ zSmg0YPn0$)Ix0EJ7F7~e9n}`qA2laxC~AGwmZ+EqiX}HO7hv7lPlZJzazZu>)d|~)kbX#;^^sMOR(Ho+- zL|+}fEBeP6eN22zc}!DGPt1&%MKQxM=f`Y~xjyFhm_Nik5%WUKt1hUt!Nj4&a}%#fygBiYiTe}ZFv`YgSZw9SKI01GR^u(kCyZYuH6-;W-InyC zDa4dvsxwV7oo%|*wB2;0={D2dusk0(J!?8-dc*X-=~L4`P5(BHBx{nL$`8ejWq-<{l-E-}O!>x~VJ

aNtgQXft|nHHQDm1asSPMeapHSM~z+tTh!do=C2v=`IfO8YqNdy8ZVvBX-;mK;l| zrPk7EnP!=1Sz*~=xzX}_%Y&BvmP3};EgxFGvHWZevW8m|tr^y0Yn8Rl+Haj>9kQ;o zUTVF;dZ+bI*5|FSSU<3SXFZv2Pp?jIPoI)LH+@C=#`H_luT9^R{!sc;>4($bPX9dp zr;LD%(2NNg){KITs*KKzb2Bc>xGv-NjE6Fw$~c_yM#hI3UuXQ3smYAaOv^0FbZ53^ z_GQk_9Lijuxh3=J%v&<=&wMiTFPVSO{2=q|%>QHsW`$=ZWLdKcvzBG8%epvgd)BV3 z`?8+QI+*oZ*85psX8ml_+6*?6Ez9P#HQD-X^K8Smi)`1}Znr&Xd&c&n?H$`^w(o78 zY;AUQc1m_mwlljvyDNKo_JZtH+2>{N%Km-!gV|4J|0TzsQ?7R!}uFShB?~c5O@}9~&ocBiF_xYOqi2R)V=KN{-3-Z_HUzWc= z|Kt203xW%x3QPsI0!Kk@L1)3Vg82ov7d%_=VZj%LafP*D z`1|4$c8xvSo@Otw57{@^pSB+^2`-5&sVlj%eTQWw<(At6ev_o^k!f^@{75 z>z8tUd2)G9xxKuiys5mqd{+6=@^i{BD&JOqQ~3ksPn5q@{!aPHir|XKih_!X6`L!b zsCcE~lgh|SSLKSzjg^;FUR!xvunRo1G!DpyrQ)tsset8TA)u3B4ds7|detgfz}SUsb9QT5vDORINOKUlrL`cU=j z)gM-WQ~h&IU`<#}QcX@xWlcj(XU$;Ef|?aI>uRp4xwq!wnrCWG){d)Wk|K>o2aqul|Ml z59+_F|93-r!_bKnHhkZ3qA|WPr7^3qsBvQBvc@%y z=Qlpy_-y0hrm&{irsSr~rh+DCQ%zHA)0C$9O~Xwan>IK7uIWJ2D^2e;ec7DYoZej2 z>~3zwzjK<;Z@#qondZMVzuNpx^GD5JHviC)*3#Rure#mdy)6&7eAe=9%YRy>R&8rU z>x5QIYksS{wWD>QbxG?vtrxXk)p~R5U9AtdKGS-n^^MkJZGmmBwwAWZZL`{zwXJKr zxNUFS-`d`9`=ae%?YuptJ+3{iJ-^-6-q=2=eV~0o`>OVH+b?V1+x|iO=k4FM|J)&W z6n8A^IJ@KAjx8Nmb==r-d&fN;4|P1z@m$9b9Va`3Izu~SJ7eKDGIiv7*aGn^lqIn& zSRO1e#ZnwzWG_rEh&F^r1)0;53+w<`%*iQ107RD{9?^)63bNv@q{MD7Ml@m#W_+ip zXagdHG@2m5czEByKp)TTpFX`GGg!OZ>2!Mzxy#C4i;RztjEalPiAoMn&8o@GX(+O_ zb z+`~;|ATcM9d!>nrADW%qbNAvof@nQN^|FHOS`h1 z^<}|j_0Bqn)=`I59{0#`sYyD7)(|bBvYI%nusAAO6jlfZ^Y(RDU%hVa`R=JR2A3=u zoH5mXzW%DMyS83+$#9^icggUg+ZGKk=?x7Tz66}0T1`W(LRgGnov0>$6(RDqm)vl} zC3EL4UA1cITq5|gtIxmsQs=_qi_czw$vbAUOybCg_!*vTNLE8Hzr%9_Z}B|L7fJh^ zo}ZnbpPULd{}I^Z{IPraZqIIB<9SlW+)w#Zz3WhKtEjgHj1@8_Dn6#>XbZH5SugSC z&NbDj{|2{ZqMI0S)|5|qnp3BB$$2SUNq*;d!znHY?s+3#Ydh zs7E;37@xgvKHKf~?3DI-p62e8wY=K1U#SJ#0c@<06aur?P4WV`4Ky18H>hoeMiH$; z@?tQf^YFmgtKF^bofYeOQ1d+f$|d>vP1C10@dunvPv?@AUP@Y#hfc}6kQZ`T{bm%R zj~DP50V{3f*ffUL(0*DD5~|i$KqXYQB|7DcTp*5NC*-bGcFHKG*Z%XqxLzE`VH7 zyG=p6g|G=@B^KI2?H{BWqM~`*y6xN7QJA@C(agccixpWF0_zk_Bt9TsLdJ?fgCeWp zzP@KRhll&RS}&I?W;x4dl|Q$yWpYp3K51WWU!|_HKgaVfA84-8);6KN+#_>+)LG6< zUkvnGFK&@i&=^Q}h#s6grUDUF zvr21o%z19J+4JL(_MSDm|bYY>{O{JyX*?H6Jb8|~-O*NI71x8(1NLEXE z`ILN;G}5qnsI!JK>X#(I>-tr}N}SV041&Mqos$R70ZFZh{w()!MS$?Vi=& zKg(UYLSMYNi@)K?7??VBfSz58i`6nZl`;xt(&B~k$q#B?(Y;dh0VH^Vbe-sp=}_L( zr%C1&EP+WCEOSLueT)}4Ko?kZL8eX@@-Qc;h|NF9RIk!m54ls5H?V{Oms zo@+byMcWIy=dtcK)$4Vm)BMBU~QGVCB&&K>UB+< zlaT99O4WvBROhsv^~a*R2EDm9FW+slmQ`04hiBDg%;^2fSd?bcL}$j^r}yZxb8`!c zjy}*)54xj?Q?L=NX0$Z4gCIZ|8c*r((>nVNI;Tq)lG{>Ty|AKkVaKF-^Cor8pRcb> z$8Ln@?{V3Q<;&Z2ZOhB;3#SK9Uno9^N2F7dQ0Ek{oU2wvj3!4y#e_N`MOzYsQNZ4? z%i^A($ePE})>RSPRC7FF4sb3=44r_Rt{?X8wtnZ)cko^`Ue zKfThwT1r{*s8t+k2XJt-EQ29A+FKtvS#(_PXjxWKv829sfy*;fUeH!EU`{b$ zmTwND1-AQQhW_#75il0=ze5Y$$>OzyB+(o6m5}T9gTDE%r)Zl6J z=1mKpIv*5czg+Z*Xrg#jrZ8lwMo*thr^~7={k@o`-ltQDsY`=PmEk%O{it zdjnij%knG4fV?BR+P!Sz$EH}zZT$+9>SxXV-YfCc%HgVo=`V%Q#;5dQd-JmL@@37+ zXYcIUy*+d0=qs1EknE`s`;sZaQ9;-RLWQ_cPXhVM| z^{R!{#VzPnkE?=nP`V$TsC=lMR77Gh#Er~on@sS69yJ*7U! zy~3%=tn){2Sv(PfNj+#I*6xn0}P^)TuBf0{!}u{8D#GH9uCKY;;sq zdtN4q!D!9j26dT=zAQ09RntsK9jS>zjdze%$7mkcw?SVPoZpb{YN{*E$!X4-)ir5O zh(p`rD5-W8vm+PL80Bu7qN3rZ&~ zup9m<&;rRcqa#HCks-5`ESzAgb>~jk-FF*Tq^I)P_D3GEd-fRZNH-DQ*W>vy)uu&4 z?`SQiK~RT0vZ!C_hR}WyU0MhjPxrjr(EtJS#6#k!?y$A^apVH4z&m2U`-pqDZnygp ze$3OzAN9loiw3xFfOVP?`_82mGjto~ZqUuBSh`??ZUa(P^Zg>k(-Wx+klKXQG-8)n z3u6jP3NzQ>*o$-FQ&ZIXuJjPDlOB**}m%L#}Sr^$6%VWkh%L2G4VRt>;nR*~PoH z_Abx8TD!LgR{T#EbLH?Qmkd*wF*rB_L4W1tyDz^|d^pZtaOZ-v#Ru^_3Lh2FPKp&K zn_=&B7IH)VAJ36?cuw$4s48F)zcWyB1~^8Yq@<)m5;NG-&E55w<|Z4Q_QC3c{Fcl) z?UNR|+{+sqm$-^&R2A4;()m4gEt89)!a}oZivpUu`=^G4v@R$qKC5zE?c@?eXn0z= zJ;p($7GR7ugEnYLWz_Rpt-?k%7pDm$D9}Kou9#W#+@_UP6$^WMXXH;Rn8y=51N^Ax z(~_#WF4t1MV^M2lRL|hF`k*3rQui}j$CRhE*^{fZRfBeDA6N_$S&CYcDU4agG=VXg z4Qa4VIZUiexogeZwVum)vCi|xwr2jJN9WjfkiYB^vnB^9O-5>1lE0#q><$g)h?w#E z+dMqGd1-mY+|J&?>8&NB$Km049WO!%^;?QIL~75%+^ zgSyE6_Nw{iRmKzN4|J}O8mKY`lu}wcuR+%^-{BbO3NGo) z&h9J;?i!$$GJ|{3s6X{b$VZ84o>ED>b>Te5)4{*@ygPneY)ZC?|1i$cbn=I&B z8f$VZS9eWavSca+8jBG0`Tb>Ddwp7fPLo!dqjh#>Wpz5WIhAP|T|ipBU0c?l?`W(K zZopq~eIuV%i}lS~@j*S2+B;$3+t+Ty!iwQFbS$7a32yZNyXpJ%1#geWRwsIusI$ z^(bZSE{c-upv1?DjIza8l%9Cv=|qS(XXz;K3Zz4uE9Hq~{^bUl&D19q3VuqoR<*QL zSrb7;TB@zXi)Jlgsy~H_T}t*PDD`H$ z!&?++=H((&UQVQY9z|iEM-}?rpnri8p&6-m!-^FfL_uM^nStaoMSj1;YpUZmqCc-)u|G(uvvK2lgv!x&^ zmvSE2${TT_%|hQ5D)8ihgJQrjLXxa`SC3c6nM`pAdhv-j1y|%rc1sM&jSv}iEilU5 z9BT@RO-_!DO>SA0X)-0n#wI1lPHGipqYVCoWP!g8m5oonO`%D^tqKHwb6#4HoDCs{g}g{z@;dE_|rbPT|=Fo zlHcfB=rx(lBwn=kGiK0I=I+tJ62lfYDtv1vg@&=iuM~5^Ux$|-I5%qjzJ+J?#8|Al zajBIB&a+BN2Aug7sal9#L!&>a~@jx^qY14{dEWj=N`Bs>WTw%XFYyl z!>&5e-W#6EdP&JWE2fv(FyC`nI0Q_njAZ-hGqp-_4z>ojP}ZpRR8{ z^)s^KYf-ZlT6MroMewSmSml{A#Ue@b&R(oo(-3Q0oH+&@sU=2RonxB)qVrrm>DAVo zT(f7tbIUCill!u|%52rP@mhULjcZb-y{az9nXq!D-SfDehgH|t)BGWiC-9e1axQdn zabZk})mjpRPYAdsrlcfBgHIH1)x^YTtoXzNZi!W+v6kR7udg=HUSX=p(rR)v+AO@; z18e(E2jqYM4FtbbkR-8RB@aQ11&}t94#QiIG^+w%?1N{)tM1qEAZga*QRoU4J4n81 zU=%vT3zdeKj6$7WsLbb&LYI1>Qv0$|=$$H5BTr$w6e>mrr1Ri|q*5(g;Zsg+3##Q1 zaCkh;CcQgF;>p2T89~D@<>z{y;bzZ2c+JUBUf>$P_o+8w!K+(H(mso4!1*{h;2yEl zz8K1Y4=Fe^h!KToq7}R*nK+}sv*54zYdAQw=ycElKj=(f9_!$bNZ8QBv}Xo5#%mTN zZ~4dv6(tI&G%&>n1#S>LE$O@1Clf7+1t-yxSNp++?l=3F1MrwyA=O~KFXG`PG@lqsO{#cxv{kQnwm!9T3g z0r&7*1Z}|k8zKhx(GNQ7YJ$EFXtYedK;X(Z-AK?+0M*FEJ0*wVy9jy| zP#ty;ytp`ogwn){qD=KraM#g_B5N5LU{48pm3*zCl*6aUAxI`(D7fwS5H9TyRtw6o zN&ybdg10Tf>rv+*0mle|+GS}8N^ONbJ>)@9jWnIzP1qyvkJJL2s3W))rwjH|Dw*Go z1T^=T?C52I$}TJI%BEST3Y=tHfoH+1cog3YhdeAC zg;Ki_Hu7FR3Z-^a@)&04`Q<^pQLy>CQEbwq3ib!^d>(~UD=F}H_#rFsdqH`CFPHNt z6KCDfU*ST3VQ+Jc{-WU)MyY`oPB3NUSBz>g&xTzOJg{ra=FMAtT5R`C^*8NyY##0& z-mG$3=}RQz#OcF=2IA=`9Gsp+l=Y(R>qOfVHnlyZo!Wjj#flw4(Q46ZUI#^9W8vUz z=dZC*f(o_Mt|IELH6E<3YXcS5HD^CvF8DNh7J#5eR0N&R(>~pCwhm5`D&hkHz=t9C1-iD$T?YEXFNmh zG8T%H5q@1tM|C^=pz-lvi{smd@rTwD`?5kpjG_$>2@6SqK7dvssD{sF4+%LK3lEZJ zI;jm6=uIkikbGnPD0H_MDy^P43LW-BWgIH;<=`zZNNN*0L&^7DA8!9f1$TgUl2xb% zWBV%57(i6W!|jc}niiLp zEvTuPQ|cKCoYhg)5Lpya*zQSb8+@dsxxKV|>7vd*wNzDi>K!_Je|hbcNJDgJTu8yh zhxjD9zp?QwecjnjjVo$O7dEF(o?V$W!CanJcvg=#J13{Ou(heFKC7&{q9{DGHp7v- z?aE{aT;;oZ`oR;j6tR1ZdYY&WU@79h9s$R$vP8TRaI_6H>962uo8Hqwr};r|^X4I~ z6{`m1XL4M!NpFCMR+`6%8M8X+iz&pO7dC&?)mt$YBPL?ph{TM8(-xa+QJu3hZhX|a zt0LxYZ!cfk;^@wq(}TUXI?GavvfnmTX)(5~PtGwef9Sl)>#FPpQ_6z-moDwc2lvEZ z7w@I{AbOYutI1UJ=@Kj$wT2sotO~e%3Fkg|ynv?&{6jc>dWUD1WBvKyh6bxDHc!}$D>sG{Bvki?SKKC)u471R6t%~S^2Q34Fnu* z@Qez_{g@yK*NCI(+TF^dH ztJ>EDw@B*+m9VlgE?^Ag@TkL@2l!b6%av9|3!=F``T^IGjH@y{w~Mg|WfF}m{BmE* zUi3Vuz|lI?v(fj2)>V2o@crPBOworGC}~~7hSr@43US_sa6k1zrLri3D*c+;-6raD z29 z6l2$`d}(vr(s*6svc}r^Wu939^Cs3dN0o#XO{!{NF+mr-S zL6J3wz;U3lsGXvsv60Qpnu7dez+1G@96HMI#X-6iMr(yFZo>2 zpYe`!T~uQMm8N}9xaE+q1=KeL)GJ>QYs86tc#$HH+7NBEOSIuwINB=h*VvF56>6uw zeB@7g1WALdL5A>&YQc)F%1vvDSP`XdO{-NbVEJ65u`K~l5S7Waev6HYD4%2A_R{_m zjYBuFtgI&0k(8Q}VofyLn~O7hO3NyiHk3}vNhwVU*C%IL>aubs=`Bq~I=8bZ-;`z2 zhlgdxC0oLxinDSHGN-k9a9dFK8h3haMy4w(a$Lmt7`-7p!1s#P`3kx{f>xI+6(O&JN*wnWO z8+~iaD3tn^fJ&>yN`q1&Nr{4sW3xUwyyc+}%^QW14JD|+ty0@hOU2O*3ig0x$NhR< zY%5nrQrdCl7xqEuII1T8`F!d))&!Dgue{=&cJz2!cAt~*iC&F7D+Z%lhGr{ z=T6xw)>uUqVehF~!9*5|?}{b;KuJeO@mhFU<^>n(T54)qqDB{*GSnP@uK~(fR z9W!TkbPf)7I_m2k2)ui+w|9_&t+leUl>+eq;~@V6@-SZEfp^*6bN~Gn_utR&cf9_( z!_(!57YV##(j`6zBXTk_0AHlI|NiG4&pHkqz{m5TG!H&e&q8CCsPl}`SsFHA6_x4p z-%?zrY4;1Jat!unqcnKu(pptHYdR$&by> zE6&zOl)3U#BGY3u^;39Ka8!uZ5)u@tjfk_P>ot1R!~{OQ3qFOD-#ffZe2!<39Q>1& zZ37oA+khq$4&EXdg|#^@h#{`w$% zrOgb3_UJsqyVK);T0@ycl&EV_wKzh$lX=$_cQaZ>lBD#L`d zqJn}Ob8>CD(FS)HW{+h&lfRD^4Fw-^nFCxoWxtDL-7AB%9zmO6K96{~7>FebAeecG z3;RuHxs=bg>C@X<2L=Lq(kzjN@KB4PBtJYNB0QgG77vt{4-_k(mg#MT*fzcVy|lQf zgy0}<2&t?vOj;eS*BeC7cu$;?2&JCEvC9J|F>uMOfu9&XX|(}ERf9dq-e3tTah3!< z(z?HuzxUn~PrSE}{!&u*k(MmrZ%a&i8nvVvo{|r3!y+Tab%gx;gdkH*sX^ewfsvlX zH;6O8f_7PR)8b=d3^6fUQ(8uWdbX<%-$IhKQU&DdM z*YKx2c<{ZHql3+mieb5%;mZSiyhQ1in5qgr7K5EJQ5g-k#e|oP74DqW#L7N3_UdM4 zIw#=IndvljWZ62B$}-F15)sv|gpK5nd^@r~iga$+0|8^2$p6mbSEW})S z&xvW!ixnU;3;h9WT+p4uFjBo2e4P_5f;IYz=KA_&GiEHSXl|%)x%gr}$H4=OHZ54Z zctKfdb5Bq61x21;P(_(mETwLtkwfVWMRK@RJ!GH>x5Cg8##VI5cjCCgcOf70O`i39 z6Hg7|+r_zqr9qx|?=C7DDk=i^+~8i4G#ld-M#xd^LjLZj@vRrUI-`I6pz}-~_H(5- zPm+h^gpnU`r^G?jg~eETOtn=iwCa+ihm){pVZj*uG5945kK}Atrr^kaTJ2XSiOvvr zzD+8!Eo+!Oxq(8S!;y#Jx&IV^-Yh+AKG`z4DX6)(w>hY3a$b49w!q~o(B_x(nWqKQ z9X+5Gw@vudT8VqIrvAU97}~@z)}cOTO#MWSHo7}H+*OT@<+l8MTUJ4VXYVOBl43)a zt8uyRrY3h)d%If6-lpK@p5CUQroQ~j{NMtovp|<$#s^PZNzbNz`wV6II>HJDdLNSY z_B01K^}@P}8u>wvK`UBhvSlqiANoflJ;ZMjHr8ol<(s);*BX&_sYnYen$oUU(*CHX zJuVlXnzOu(zawH3!1bNrx&=46Vh4ohvn%k_$lIh^Uuw&#sijN!vm(}r)Ej5oKzoz3^~YB+TuW|-ZK(~(J(1N1C3EOaZh>aO$~ zZb-lWdhvNPeJcNNL=LmrWhgTrZ#3SD{Al!1G)z!3ZY35YcA}4?Zn@3r|bmCjy!n zaw^rF&1GLpLFg-i)K2iAFX2wb0k`A^6e>jtuGj%(cICrtkSi((cTrV)lc0} zGyajMN|&^kb!FvEDzcI56$NfkN%wcWdv)6;cD9+dQ}B;P$e751q2O z2D9xqkq&ceF_)*y1Yq)zD|3+fngtUq#`W}Z&|#E5PqH|SCu5RejNrTs%Bz+8gcpMh zeJW0Gel_{5t<$vcReZtS4$pmZ;fXs56MK@GC;>hdF~yuZ90Y| z9>iZ4<3IwwW8DlhiLb?X7w+VI9k~Rd%?aNn+3@`VTBG!ShaWAzW4%X>SHDkFzZ>~t z=_XpOQolQSue6(1cErfylB)T9X&4eObg_%uq-vONqNGgR(L_17sVPnDB<_p~!upiA zq{$-ws5fV}2P+~t_ojRwvGYKz+F=D#{XT%5@bO4*MaeR)>`_g{%3g|h1;E>WU@)8@ zU_bj0Gv4!rJOJlR0$jME#~2-xnCM+dq{D?cyAv-t1vn1bJ(vy9vH_l^1|<(MqnSMlnbP7$<_%MU6f|L?TQIox27f8 zjXbUBP?E)Nd;*FYLI=-ewdIJ@*b#QMw@#C6n>K;dQE;Dn7HzJPvHr&g07-zTcaYz; ze6tF{{m>k@O8~NsAgEadK6l{edExylJiPrz@ z3m@s!vwi9W5}Sn;dAiV&{ED7}MFn+~+8&qI-1_q2<+W9R0yA0WNDkk?cB5r6q6qsj zS?D`;@d`)}gB=pOKLyla6i0h)8Tn{rHsH~KgBr<# zn}HShUj$ruc8Yck0^ADC1kX;AGO=2#c(lM52jc?z2G*x(T?Ku6^!k$X}Rzhqu3y6 zh1kmx)CkETC~+0k_^_!zDDby%zvn2N`hxSI`Mj^b=X^+smZpE+bneVPIag>*)6(z`oOhT)q{@O#vi^Ocf(@Pqu$P^qE^6G*1PBikU~S= z@G#FB9;OR~oF^Y8gNl3v`A@(MhdvS5TFIWA$M7@L_xRtmg zWr=-71&Y4L6=?gphW!s1=}U zX>Vl{{a*sgWa`U=tC1#;{6X|(h2ww=Y=V0PocsW$l3%Qni_C=|K;FoJKXtNI_`s(D zcMf?gtGG9PPKCiYn`1bmdCF=+wa;YA%>`zxc5nD%@9Fr7ROvkg&RbRKc>Q) zeDJA$@L9M`necm|DWDNhXY9zM&P!B3tKO000ac((^#f-il3B$+F8_NJ&L8u^gU0xc zFJVus`0yJS`D5J!`3t{sQLaOJX*3^ zz(I_lA^7DSrB=rS=yaroIE%gQ$9(W0@+3t*0*=}qRq&x4DutK50{5L3fX4C1U$36Z zfJdsVa7QWE>dSw~4mt>X{FPu1WVX-}P@ymr}@l*Tzrp3Do?Aq9~I_~e;P+Pm9=l+hf zVzqW>sI#l^?)huh%)h&^3tvXtg7kqx$y@AUApul_RkF?pe;+GjG>$89vc+huA&c-s z6~9)&SK*>YK77)b3cmmOgH@!5RQT9(NiPuo*pmjNi4-a)!z#{>Y^;zLiY1+c{X)cI6h@v)yt)XzuuhCX1cyZ?EF2p3{jtUA1Ik5U6OWW!CgWyk zVR^x#9mPi<6h3WZPa{|{e~cAntXoM^p~QZLyrKcM(5VE_wGQ{uUkWKDzS7y|WK~Pk zT90Bis4IH3$4XL5lLB75ehV(qS~DduI&FDF%`vg|7pCrzuFAD9{x`^9fvG zClC~tz`2TKeIRtn>0<|ASCghuVguA1y=m#Jhc8wpJ>{1(ENP+lL`@ai z_o%5f_ywZ9xU)$q>k*kec_=rH^qv$fB#iJM5o4)Gz(-G%)Bjq^EM0@vVf@ko^?HS0)Qbg2=sXKz ziDK%B>6oJXpC!rS^@1Y`9QK{CpD4GF{ovG|WLM#I6^|0|F!aS%A$wS-0el^u4nzK7 z=(~z_p}@VPzk*NsDDcDhO$C83@}ZVc;BT=Vq9qi3nkNt(^8{fPC{XZ8h2GRfc@&{c z#ZMYmIcQnW?!}Y#orR|ra5F_e_Kh{l8D-Tos`yv_raTiDiFxm8!p(K2v{Nr!Q?6_B zEH)NdQ^LkxxdwVfx#S}!JiN%y&O3sDo>4efQ?UvFd~tF@?n14MR9C5E5&i50UnCTs?ESN4V__ei|Jt<%4iz#iam&dJL>#c=PCWEBq4xa@3qtJspFr^25@;G@^_6e$DPac!YunG^7aUfgJKHyI~ zPLuovv=aAo*ly`lvF@&%mNWa7>YP8#pVIkX?s}-O_wHO6v2b_)jJuXRYQgTi`w$U;J_a@D0e+X z*-mrU!_pf&C*8g_Z2jGn?O*ua^&r}S_yb>UQDf|Wt)>=8#m7QL%9EAbZcbElOgQQM zVF1<}wn|Ik@1@+vo$p&^@?I8p=9TP7tVPkK*310YvkyK?>)FpfPwUyw?{m3+^IGnc zT6quA(hFKDK#Q`To=vOK-aCGOV?~|lIMb^7b1v6!EUYUu(CtFAal23-^?c=ap`G#% zC$MCq5!9XM$EjlcHXLT^>Z;uT$S+L#BC(SegFyE);EJs?+-F|opubh(^S7C}bGsa| zef#zk-)#3%>(9%4?C6*N-*EI)O3$!oQRheJZ?V~*O1dTMb@+83gD<4AtEOD8CC+}M zd|auDjR@tIEaBdLx?8d`(-XhMy;Wb*9a%@kZB|E>8?ubZdy|?s?O=-CORNEtZDa=j z#gs*+k!`iqqm8HHfb{vcJdk@P3$K%o!C|N zQbN0_pg_G%4CP9&xY83(u2pX1a*gEii6f7A%k!**%PH@=hGuYN&<6|B=0ah{PGZz0ae4lAPspL=g zn8;sx6KBlG)))A+zp3W`7EaV$rs6M_YcLOM3uD+vz7eJ0sp78>7zcR!c!J*q_;q5m zvSI)KK~Zn-wAbfX>S`}9Z*zdrnUXE2qPn_5b2eB`W)%IQ&ZXVl94oW8< zpih(=3_BgSJY%H7+B0l@oO48}73?@}Ga~;Mf1exSed2$c(eYDnGdhTLdz5r(r+CG1 zyWcfPV-r&A+JwIZ(W+kT`S`}>Ze4ftM(ob_^=%TI2xgo8Y@=~i@lx>Ky&~pEektrJ zztm4}n%Jr9oVW>gRcq@o>M70$3oFTldeWXF_VvU$oR{!a{A8P;OOvSQ2Z}DKtTFhQ zt*$|kdo_*{)OXF~R&$k23MSa@W3#G-m@TD%#ZU@Xr{pugd-=GxdQxJDigeSN2J#*4G5 zUhsx2&Q@`c8hHOQZoPn2P8`9v2-YJc@<+g*1bmi&XTvUb3b>8m<%fR+zuV`<-{Xh> zSHQja|HS&OH~+Wso4Q_n{B)%+|99|feqQ`da*Y=sJ!G8PL(s~j_~;=a@C&6=j$3gb zkkWG2-)_4=+$B%<$)D!8G(4~DE))OG)Qqavw`(K_E4cgVE*A2666fT-u9&2}{me*{ z`Dd7s5sDdEk&-Ohr2sbQmy|jZqYB(TNckg~DtOl$!a`LeG&x23(h?FbtWaZIC6ZGc zs5b%z;kV;x-=Fp;wnA5_UZUi4Eh8&snGCCBe(!zcFFh+ICnp8LGevz-U59NcDK-j{ z%lk%sgL)_V7%9G@)ISlnbvHD2HPuy+tZmvPuzu!!^3YhU>!)=#bhI=$rJvPZHku>j zyfE|)*7w9r0}6*ykJR(tk$bVLh`vGn&r3V8#QV<8(TR!CF$oEW)#u~lTa5Y*PeF&` zTP1SxulX5slAOQFO0Cm?JVT-Czvh*tmj5^PDgFHGyyyptbu+E45i2%s$k!9mV|Y}hLew3eeD8dFNOoFr#Aymz zNu4=S(U)t~^@7?lY1h*UPoxEwQlb?9tF%YpSLI!6$d$2PFec}>d8RKm{?u6L#p%qu z*5H4a!g6tfl=_+9T?&}dgBmX#63xV^4jR}iG!DwJdyn(kd@-yYE1d$u=!AB*tJm@= z%1KPTl%pA5*IK&Q-TVH0%cPi;P(yZiZEbh9AuKs&QVX3bppaWqf*%tp(K~Yzk|SkZ zpd~G*xHu=x5~!0SlM`~BHd7MLHR9uLQ$k`=QeuJ$l+rE%ZWxw=m|IYu4&S@k_zrJ< z2*3bnCU4n%Xubch?jSC08mi1LkBcj@mM^TUTUc%_iHj@Gt{iHbxM@oS z`zo}>4VHkAV2e8!H!9m~xKTORZ3zwuurw5FEBdnS&A5ZOu@QF=H}mN=?%>M0y2@a; zxKAm4BokLT?U8~cOp8!Y?{`tJ5xQOSl$rvPhnl)jQ}BZP4_F1h=g@a)>&gE$4_bYQ z7i%ii@hKMRgnh2CaTJ%5{q&-<5zldn_G=-vDt<@_ky*E^9FlKRg%c`21RV^*6tE*SHIHwd9 zP5D3Uy$5(zRrWr7_TK3UA@mNRh7w}#O$7pykc28A(o|X+H6)mXYDX+s7tW>$l2J z@QiAm+J4r071p?Z%Yg|A16$UQ?O!>3cx8WRNod-5V$1rins@A;*-JHPe0t0JEnBnz zoZeXK%BctWex3DTZki}17LVGQi;|VwjJh#(vOKIY1?3GgoNCA%)c@7NqO<=xVBsh# z1|mK*NBX#i z5!T>G2qdd&H#U(-u3qA*J6W#m`i($>M1#4S0* z2O0X2j8DG9s2!iwk|XwiMD~60NyV%lM&<7oK`|r!{Ex)*prg<>Oc$pkS`i3Ky4=K>#3*?N#-8}v>o>3o zULW9B(b70QCLusUcJ9* zkG2`p21N~=(Tl0Th;;_^;f3i(oK(5*Och70TIoc}2ZNt^QG3Wq0|{d^(w&5Khw1yH zgTp*(7~nxTSyu~C+EKC(AzwPaRIzIiKjfvxzJoFC< z7d%zkDoPN27w=6WJHLP>_$W&d-|?e0=O85v6W{fvfFH@PK zqvC4ijj7&PQ%42D>w@$SV#%$cJf1HFM~xAF-0xM;ZXvgUl0G;8O2rQN^0(k zQal@zL0Qh?rn8`#Aw9aISu?3Xe#f*fc|ga6Mh)6@YMR-sb3%uI%I5s42YYxtJp@Yo z3?14>{xk3`__&N_-RiF@tlxOX`Zj5ux+g|OMJB{|>=DtpTS)8Hty^W|E`=O#Lgy~{ z>ztq_;;qzX7-f!rY0=z4&iJ9rW1Jc zP;IvpdPs|^ZCXfECxqP?)g^>*UFJ$VIIdx?B|b5~h3&>MU+Fhr}ZYi7RhJ&-L zMS(bB%1~g6jdNLRIbS8o{0l4c8z!7`iq2f(?3Mk36GmB!(`MPI;W@Gbr=KM2W%Nte z=86&dIYB3k&$}m#2<_s*gZjt`BU+E_u;gKfrN&KntajNPlQ|L^0C%$Ft5}TTxbTEq zWJl`joOmPO(HkvPNPMm=n%CJ>8!kzwcv(Gw;Z7H?f+Mmpcq9&u0pf*SYa zFHqMyZtJ=ye}Vs=e6;cLkO_CBIxzb~e;8IKt#CZnlmSyYC4L3A(hfVRP@ExP`i*wz z>-hdJk+7}R?Ba+{&04i-(zJ72%f=~fTlZ|0ln|mj4Q|x1al^QnuJs!=?AQudVlb>d zD%3pqyTjU3XIR~lu06jytnPJ%WkQgyb-y<(6H+@Ylq2RLSU0uB7*#-C;I5-EzW&`Q zPqhX!@{!y_arPEv!rw6~u1S;3lrCyba-$aga=iSSOK>iaI&@0z)dDTI<{lmU+tGS_$YgLuB#UZj-2p9?}0;|Jv`JA z(hDsLK`X4TzWEYzCi)eSGb|$|r;bbHY^%K6khQCEC*lO&iTDn3y_-CF7hBs|A`9=_ ziD<@y=ZR?-jXM#!GOf?u;2fA8q5BUf=_0`nt}62*v|G7#kT)G#*I@(&&%@V54aR55 zv>&Mxsj6+V7vkQ!_=!lXJ96oiGH_Be?jh;1tpPFz<{qegsoGg|O7EVuySlZy?v*H+ zDW~HI2>uU3AIR6t&3BRpHC1-&X?i_p9aeZ?tq{dYv3~;0J{BflIU)yL;aj9NG4n(G z@qkjqA0sl^k_*4yU?YRLqdWKpRVBKEuU9WYli~9!o9crHRzC@if~g2Q7kk$CFl@}? zQ983DkY2aUBu2;7DYHb|)_ZlR%)CMq((sKs_SP_E5akG})py@HTWa}-pOP&#^<7sO zV8<;}cHE@I6Rp$Xd$RcMd_3Q$*Yb@UZV>l=#*H&6?CwGT0>%|bUqcgrYA}(I)aX#A zW#>+A?tNFC;3m#7P=RzMAfB30$z3|9BPK~FV(R>F#dP|yVsf-EoeTX#IHL+xm;fq# zznoRQM>WodB7}}R)Vh9ZU3Gj^bv#N>&Y<&Lw(hw~-lgiX8jObj){XB{##PdB}giA^y8(a5F@D^j+^8C8>o!|!7EGsu1b+}6Ecp8RYRrITT&we| zY_Tz6YP#tyKIHBU_%LDcf?SRSOu$^pDKlhEDkMl_XM+rs-N>AI!-vny=|6qar0Mr$ zsW--+m6>_g*y=R(MsYzwadlc17v9WAu7R#m^v&*)4}A`#&!m5i!J1$=xCwRwL?*;{ zG)dKAa?Bxem3ZRHt1(%_vj17YBqi98*!%JIlfVstoWw;C0y*X z-V*(HC}@U~X5_+A zxQsUy$EL+Rh2hRU$x{#JX|{XY<9Ni-ki6S5=S|B_8Pss^%sMpJ;*;DRYgwE4m%>io zvEFc3Cc*cStsO8T)DxQQ@n*(EcvAa}9-o&tete#{S1+%pckkHViJ>jJq;_c+-nM_A zKDh(>%eodMC+_w83cy)~RxTLwDU4Yo-qJm7MG_ z1`<_ERgWy0ph7A5a%h#DUz{156*_Lj=*jhBk_M&po!Ym5>44l>C-<5;bVPCS2>gHK z9hV-SJ)&^z#3tQaXHFmJDwz956wGxg7!@(9fFOOK6ISVyWS`mKlriX?sr9eggVGN- zeX(j!Kp#a8L&Z-YFCSHjFX~zLq=Ro24%~g-$H;jXjsfMo3yBu_fOGxlT>(EL?Ys`I z!h&x=KEm$?Xp8fmcct<~D>&qh>LkByVr^TTcNO~2y8`h-vX$ShFbAfsKdrc18S>2* zZp9Jlnwqu=xEn*#8f>47e93LUk{(&vh&}i`?%I~yGBD=?hmp+>%!y^&1LM>M@ts0L zV!W}+H0`#seY@D!aVf(hqrw}>Ov@=xIc0Og`^1%l74y-UulvluP*<>TIOdSQHXN^1 z;HLUqaqlyC(&dIfTE;QQJ$h_#IEdFZuNg1S9B2+X%tGniPAXTssl z*us!^xDL8?17UQ5iw@|SL1FBOF3-QZR6Tz8N1V4e#teI{?7^8nP8jYxZ|)#xs~sxt z>A>iMDG5Raug0D}HReh`Q#tkil6Uul>gRNnPjhZBBu(>9m?2DuGQE0Rlf>4|REy4ugZhVM4;*}Q z*vW&P`j%V5MauuMwDO=+AU`%d%X znKO$|9Xj;X>b=uujGQqoapY+Hq{j|Xja73Ka%^9!aV=kskb*_^6B#X1*$`jyY;jU~ zG(W0wupji4wyGO+4&ICw44s221EIv@R^!XmXlc9y^Md$pU2@g)%QW>Sb=w zm+5DqQ6>f&WvVx+NkaxiIxs%i2B--O*w(>iSo~JyAkIchBpyeh|0VPpIM6SsQ!(>r;1i z3a;R;PTke$;iq)IvmULv0F@T%^{+YI73VR0*IZPJ?>OkOEpgU{xOk51$T(4^mn}Ec z?ND2ypgFsDKS#TD99nfx*KMOTNWC>Nal3bCZ`#ydEsj!)t8eI6eS@4mXF;oaYm5xO zI|ebvg$K2#;Q8|n{d0$;cg@V~+ATA)TeFren>B6OQa@c*b~A4_!|i5Ct(r7xCI1X} zk7{Rr{Vn1CHFwa+lv?4Mi|5U|@yKvxFUz?D>zKD+FRvAvjz2Oe@Zv1&4<0u6pkK)D z41<0cx%k8lrR9?msy>EgW4(2jl84mwU!efxFGY=u@i2n}H(B{`fHJT3-pm22DJ zb}egLXWs}j1%6E-+M&2Z@BfBDMA>;;aE8BE$B!L5R)s>BdGDgER}LTU-~WXnqel;U zVQp$wsv4U#wqH!&(LJ(K*RH*D5LB=aLJUI@%^bwgR?gnqO9t9=(-K5Sa0vIb$GHGn z$$-1EW5??6-X1w_+{m|e=a)Kqa+5kHSI^L$x5szrJ!H_JA(^Q$jq-7kcT9}xp0rzi z!o4d<1K~zy8N0Bu!kwT~6FV-0Q!38?t zG+_t1KsvZUK)-|Rv!S@))`>@U5G5Wb497gThWm5m`)#_1IajSr7iMTM_xVHVxN>Xg z1DMaqVYTeLzS%sx+bi|6+soSezyV32b1NCn>&C%1dO4sEW=8eEC1uiyT5HeiVG3dz zUUES7xGp9lKC$K6sKl&~8@6pF>F9`BfF^g$t4?OR;&ATS2k+r#Cc>J8FcYz7Aa~tk zJZ&a3S=5Eph7885pzA%V`gClI?u?Fnt&u2u|28r`@9@t}+P7>dp~vBrvkzxnl}_mH zJ45G;tB&}PB7EZ_@zvHX|3p+H)uq~_8_4f9ZxXA*V}rgP^&K8TW?gHdpC5=JWj&Vf(- zCn#C{H20EZPEvC%sQM2ml}&7sc zFso^|)|t}=#SEODnI70UkhEdnU^@2=&U4c0zK0v6m33V!_3)FA)I*~n_6_>+jCw5I zQ{>$&2w{GY7Y5pACzOQyRhaB;cTyoa)rYtdChM?O>S4^6FtHTb=2<7H{G`zhO@2f8 z%d*$0dvUw3Ygf`u<#y}cOlLJix{i4s;lv5f@V(PUWrta1C?Gac!|#%{fAvyl;~J~r^-}Ce?5chc+pV$(U||oz@tuGxkg!D#88&JWp3E%^pNDVSs8OjzNbm}i@461hilM%B7emRd;r<+OX~`SzZC2ya7f`fV zYx_3NIEuLZ&ygDld#?Y45U)8oF{yV(N}tQNDbLdE!CmuXv-^egNbJ_3OM7pARAR6E zD3y}gDXp!f#EaJ0k-X7aFV?=rQKX5x>sW9cw85ZVJI-@u6vJ#3tTMETzx%0f`K8$* zeZr=WOdZ-idH6|FAmKD$y;Xgvd*8x7eWripnUWhZW>k;CJtMr?@xz{rN+0=LRMNQq zG5sc|vjn=U@#LmAIwghPA@96Zl0B?e3gJg~=0wj{nAL~!Gbba(t&rj{^VfP#-P8y} z+81^r<<3!*=do`Cp4z=_RD9DW?Ha~5ii(N}ZH?D<&G1^UQ8Zq4$7Y!gn>A<}*ElLN zB|ti@@t``#R2)Bs;;!*g@hzHlY8=}zD!N`+TfDYwDX$Hpqw0t0Uz?{jXwj@`T)pV1 zpYaozws1 z^o;b_*becs8{8WS>_(mr3h%XU3q{QR5wneXM|(GBT)HgNBgl#+xU3wUJY4pZ+BeBL zbJUQ2Lq_K1O^)sx*}tS8T1wLBUc*vz)wx|yIlX>tw8!I3?UNhbYiO5F`Ce~!m#BJC z>0W6~tuYplAiuuLNr(H+TT9uRYNZT*1|aQn6Hu7dJ5th)SqEoGVP-`$=H-Ym98F3x z{{kaI-=7eqiz#4w)-dr+w{mhzw>Y^iarP5lHy!XI7gji&s zFS9aqh9PGxO;V(7L>YWlE7tl&&l6^Tg z$SP)+bFC6eV8bQ8f_PcRul5bFWsZxzQyk^>4O9!~^M|g^#}Im;^YMf8F~s2~;nDqu zLVRO2u3TmJuKy;+`Kkha&i$&QW_X?&-&m+g-vkH$P4!JqJm0v+{`_&5iFGHrcFR_F zin8ojs{-GX$|#OFv(oq7BO;HS8CU2JoDomVb>cnJPNeq*>Dj^!1X#iQ%~znjQl;)t zQcj&Z?omgs`FiUc*mJMLRB)E+oY*;?_6UtESY$8a>9e!v#9TA)nwUA+XV1ST z=9=ogX4Ihv@m;BN#N`}wU4M9{d(0&%huoez1P+b4yokqQ7Vm2Dz|q{W(;7Dp@3bNN z&X_y1e~#&z@LGDk2IIA^9;@ybCEe`fuz%j{9@e5B5jC(8b(!e~xn27^h-I z5iRZB4S&70Bq)6850ZA=nN4l=j_XZ+*PmE_=WR1i+F|ax{$$=k#Ob)6=emZ(+r&UV zVd}WW)|TzmcCF=GbHlGy#YC2GvzG|}FYHh4^8Ur#^LDbml=7cDCgViOsA>If5R zb)alVPP&5v-B8anw(q#Lhsp>aOKD!upO2x@1SDeyk zTypZbKE&AUs$s*bvK>rMJu@pyzSac=?s7OvED50~1&D4saUBs;!|Eqm48K4P zZgGkiF&+_LYS*ptoy+l$SiMpfiMSVgI=-z8GuxfA@V|dm@COKU9-tNOdJc2VHBVI>HwLD%DienxkRxf1cA)TGyJfwMUf|Gj#NjrIvwMBymcDf6j(*1 z2f}SR+?C+w?^&=L$@iIfFQV^k+$mHAf8{W*gnd_rT7?iQF-D$*-!#M}F6QF=Gzq@S zadLB#HPtGmpBad$3?UcLzr<4p+e&{5dcuEj3c4LLb;JDm8ikOiA_yDU{PnT& zv18aLD3qEx>}dtU7>h6@S2K{(GM1Dtlt2j`-FJSC9L6Ko;5YVtHtQ$fvVOj47Mi&8wLHp}3?1jniID3>xkC3o>~@2mB}$a}ms z0%b35gF}`w351XcyV_;rLAvGNX+?F#>q|`mv=9{_JgJ#Nprms?-X$IJFq3$*ty4)? zp$)~uyk=QdA}!K_=fGUbOKPYTPiceG(0WE7gkgWsSD`@2DKhZ=Lobv>A51s;LQ6^>@;A^LWDT~4pj8cnMzG^z zIBPt!X{wq(g^3?O|XV-27cHA5^$}+3;SVxILVT4%VWSXCtEwQa~lP&x(2J3 z$8b;T%hoISezHDR*KMu)!AbAH-r!2y^!Gewl&h^rt(UAzk>ie#{jP_5q+Z7wopZ^tyeLso{ClVRh;#cYM>h8UWd1>4c43B zy>F=|)-S3lcCnhP7PtYVm9n41U@X8+Dyk z7wpFWsuEOJ>m%y}>qDGWPKTDr9x74wR7vV2Yqv_ao>m@62Yan=vALJ3(yW=NwOOdO zQkAYUtTL5ptw8-%t6s`dy{%o?|5msqcaBw|vaCwgSM^gTtNtq6`dQ_u0V)?YUS-Ww zdDeWDuLi0?YOorjhN@xe6lQ}Jma5Cu73xZLm0G6$qOMlgsB3Y>`gQ7W>U#Beb%VN5-K1_-%hfH= zZgiVkp>9_z)g8FcM`|CwMIRzo={J! zr_|Hx8TG7sPCbv$ZeLVu)l2GS^@>`DulHY5udDUy4a~mZRBx%b)jR54^`3fPeV{&6 zAE}SkC+bt2uzaRAsn6AB^@ZA^zEoS)Hnm-SrFN*V)lT(|+NE}@J?dMvSAD0xS3jtI z>R)QVI-m}!AJtFLX8()&6>F};s#@95-l??KMu+H79fmLHBA|OL3hR{^T~EjA`Z`WG zfV^QNXa{P7lapq;IrOKu)U9-D-A1>C&ct|V1M7f&!%n(0B*YV-8L1og@Owbop(l3Y zPQre-2YV>w%Cy8LWrsp}3** z6g^yzz&^sMdXzp5@;zhpSUpaU*Qevg;4`5$Vxpd;C+jJ?Ko{yFU93y=R6Pyfi_Fk7 z^(;t%&(>wS9A|E4=?dtmsKR%{^YnbZ0DBXQ^x66xeJ<{dIA33&FVu_mMS6+8SYM(q z#liVfeYw5@_k>)fm+8OgtMxVdTK!jjo&KA?9>SV8=o|G-xT|fszD3`vZ-e&f+x1F) zhrUz)L*J$EhD5-<`aXTXUZo$<59)`oHhl!U6_4u2^gs0){WzpUpVUw3r?DP`-R@3ztmgxHoaYc1xfv{^-leb-lcczJ^EX$|G(4U>mT$!{V%=W z`dA;(2lbEoC;c-tnEwiyl*77O+hFs`Xk$!>2{mCR+(ej26J?@J3^d%tn))WrG%yWK zBh%P4F-=V~)7-Q$Eln%a+O#okO*<2B+M5p83g~1yn=U57bT!>fchkcpnw}=foMe)X z$9PSONi}IE-DH?d(+jt}_Ayzeujyw_HvLVu$uR>=uE{g`W}q2l2Ad&fs2OHXF~iLW zGt%IGDs!3{ZN`|fW}F#sPB&+mGtC4u(M&Rv%@k8$3QdtIHYH}NnP#S&8D^%LWlGI# zQ)bG|9CMbbFqNjt%r*1Oe6zqTG>gpH<{WdbInSJLE-)9G#pWWj#9VAHF_)Ul%u;hX z?i;+)TxFJ-znH7dHRf9LS96{Do4MZn-P~YqG&h->&2n>#xz*feR+!t(N^^&~)BMBS zW$rfjn0w8A=6%@gKH^OSkoJY$|U&za}V3+6?$ z*1Tk1Hm{g<=2i2WdEKlxZY5*$hu!N@mC~gdH~Au^R4Jjc~%MH)8tSvS|esb7z+p%&iI= z!4ShnIZj6fg_1TP>@=E2pB5C|Xu1d)Er}fCByxoE)}bWFvJvT60k1jiCS#?Ah? zKuFQ22k5Xf7*W(2{t_{HWz+QFA>o6E&?daVjiZ2ZgcrEs6$QeGD)M^{D`q0YN}Sx3 zxPD6X=xIU0ObZxK4;Y6`mkiHvGCacxFnoqvD>E2y^vr*N=nK~ z3(AUT6or*Kg>F37NQMio}! zc&i8sF)cT&lBUs>fpmnHmX}Sd46CBMkSfW;Tqh56-9nkm07K`p(&h%o6+YK3cN&s2uz8IbGPXY(EY2DtC8zwAu+J)7>m$!<8= zzNeEeZ?ZQkTZ)@W2+K*TtW0)d@TR83u*_0Z{Dz*qDCp zcI|Ppia?lXx6`DiIi6G9PLt}3$qh7>fhBu#*-1Tlxo*?SbtB1jn?kPZKiBc^O=iD?A)j(c5b&x<*-qME3r}g3_VW2%<%+y&msSXOY&cz zOL9gSk~7i}ZJd2yp7b79P?L+eUb9`V*#WOeTdosxHo3abrJKg=;526Ua$WWcN@I48 zYsi7a6U*X|K`^PPpt2m!7SM^jB%%M3_r%5!0{zEFzfLT=H8qEQ3Wzc(uG4Z^7=F- z>7*3non$<^z|ROhv4sUiv)D}o8O}>__&%QEFEs*lA-}CRIfu6Lq-|^=W>_VqB~zWi z>+`ijcIN3PPTpsgRQVk=a18=ZSjs70r+1_zr%R=QBd0rwbzl4}8Wuv=woE4fb}W-? zTlNaF>{Z(`CCD;`mQI2^D4dG&StVt`*-!RlCPx>}EiEl^vL{ZxUXMG`N*=%wB?WVc zg3>BC9|QbhVsb%;C_6Hed-FSaS0A;XIcz=nI>J^r!UiDStHY8 z;L9{=`m9$}jv4T5Of;(sDi*qR;;@($hYu%vbI1q1$quhaFiu0kGdPc&`yp=*Gfi>l z2YC7&p~PHHnw%d}jCr>cU9vNzrX*+3Weg?>g(aos^Mjgtc4R?CMfv>V^7$O5z*U$; z%)i_jXAXx1%n?|!c)I;=09icVosvf$Bu=m@)(DgoI9t|^R53vZN}yq|9#^SyEYrRaI3=;QYxfBY?N6o2pJXX1OK_$d`LBXU&PA#8X;YK2qL?TZo5}_E05Kbhd z7>PqkBtl6fdteqnOo81I!pjlD8_9t|T6V0TVp#`XongsuTXR?f5AimfHWwEQl*}#jf9P2Tjrukb}a*8L%;hmnGVu@K6iV{b+tmoxIDVMytWiUKEF&SKE( z@i_BRkCRU9RxzE)DJf3cQj(noE}ns4lXKjJr8=H5)`ZQTQHG5#C+uv8rzbm0?PQO$ z%JAg*S05RXWi!g$-c1S}k56_yQqu8Au;V$DbUc?r$0LP~=R5-)j|Gm7=M=nWfG@3% zxu+LL0F0;32*4D3dT~YoLuW?d7&?<^Ov&BZuYXa*kgzy%`Mqz-?oR2w$xiioo%W6D zWl^F@IOZu{cNRS$KeRkBwRaW(p1eF?sC123+|ee_Z7j*|yR&oS$xHJ@U_d1k3lpMr zD&M6f!{r?XLjx&Chd?p9#GFccsd6lWIHVI7hOD^qisGpys9dQp&iKoZSaG=tO?7&d z*P9zVr=SAcZ9Y!mG>S|*L%rnIr*&1zYB zN4Kimm|nu>lwe%*HFvl4oUs_2L75HCnk&Oxd6|DalXQY%IGqtaL*7xoIhgNaKEU*1 zzUw0c-6127D+Ba0XAV@)znw&GR^!d@3%%djRlqY~=ym26m43djC42EX-=zmr;DDvbT=32ZaC?^!YaXJ%Ziy#Bl)}dIiE-3}; zBT=vpiGp>BK;}~-xz6Gk@2C8^Lvr#Xs^*tFB`yR?TplcOA*96R$r2|aCGHR?aUoFR zd;}sTI|N&FXEx3}8}CkWdz@Vzyp!?b=}w>X+@_kB5jr=(_VQe2n(sE$JhwpeSdd_c zE?4nQUo&#T*$3y&afZIRbBf*FB$wfP9fprNWMNka0yjk08(a?&d;1 zHy7v`V}!drPvQK|7oxl4kk1_l*RWSgL=5k zcx5Kj+A+*<4Ksoblkou;giRnTSszaWDI5zo4p{0+{O^GjQ4dJnM8GDB5{*qDyQ3hx z(-PPkvN-a^T4$i>Wl)eA5-G`4{P6uEB!)zG0e;u(H(~M#PS6d`&c6ii!p(LHr{v!OZG$}G{5k=M6J{XJg@KovOMy$xQsCw0 za!bqUFX$H_H?1JE6bUJ((a597@AW2K4D(3IQ?Q%>e9HYF8q!IS3&l4H8qXNycL^k_ z6uxj)C{1~bbE#WlZ&~dwKw;D71-`f9yGU@w;6EOA5tO(PNr(-Q8xZ*bd@H7=qNGHQ zVw{u`20KyhY1at=qJ17vG^{mxS(#Jx@XhP0eI{QINGm+2-aC8*Rmw7F3m?P2pQm zlxSr38F}$mZ_G*rM+pwiJ_C5lh+)9t!*YS6Mw|j1O*dF8fTlDvzLr4f+A_0BD$1D3(ou2!hpG&RoR>t!3l^IJi$X7l} z5#{Lox}DQK7o|BoscZM^5LSB9B}uDM9yQ9Uj<+Xz;Fnm2ni^fR6pkoMmM%-#Emh3`#F`p>mqLmVpZs%`L&!8Xc zkZspjyG~uxr_`a;tgVdZWz58g!_8e8wER;so2uJS;F z^nHvwMb-z9#GC;+)7k0*>l2KnmqSkTDz)4?0LjG_DwJ}I5s*$?t)eixeyF0=$GEJi zD`fo|t3pWoHC08B_iK(3tR=MMltAXMHAa-S_+oY%r2g9D>zR(YWgmA^=+5{iJwYd^ zS&;tghVN;UaEtwHNd9?L8D#%bR5_&o($pNt|7EDNMEXxvK>Dwrs)97%>1rPpDj-KUm8%I-n+ z7s%MHR#!vD?w{%!NZ37zJDi`@&#J$Qe4V;Z02OtU8Ks^LGxMuj; zzK!Xk9)+}9SM@BU+&tKaz+BEeR zWYbF3hmcI0qdtOMT9x`(B-7Ld&dU#4Eojox{t4!-?R^*Tz@0!_ zfurorz|)AMiDQW4iIeGO3bBAVo#D(N&LqwvmJ(+Z%ZTN~ImEMwcQZZsk`bx8mv|rX zex`60@o}c`38wH#;!}c(AuEQg7_wr>iXp2C313wbtB7-n^N90_3y2Gei->0v&mo>m z+%4%*dx(39-w~ld3+6u%_ena{zli&Z2LyGvq+CZ3BZ*Oxave>KA=V?t66+H;NvZ13 ziQ5FtMJ#7+4YFUmY$doz2vA!C?So4i!)*29nc!c>-nV!H-k!MdJ=C_jXf@o$EVit_ z+1oGb3f|Vm-gVhQD}mUJ=&|=)CjP%#@&bG`T%2WnOLJ%|vbS7z$nw|~7ha;mi4nv| zViB=eP~S?tjktn%J8>oP4ncDT@k-({;&MUE*b=N}i>o1P9%3H^{|&PbUK(aa5$oBz z7MEHLNjD}owST#^sobWpWH@wJ99YuHYESGy#CLTt??f6r`@&p&0Vi&fh`5smwz#bU z=(VdaiU)tT&h7_ZZY>^!yKclJlWFK>Z@y><^e_H$;R<|k>Us7^e7_*L zgZMS^8{#hF0lGg({E_$*@n_;M#9xVrh=+;Qf=Wvs@r{gN2oW;{(BZ@gBEIT_NtCoA z6-|sG)+5Fe>l5RM4Tueijfl7na`7P5gmhD4Gh!QJTVgxLQba5!vW0}~#n)Z?gPeZ(^ zq|->Jlg1a`FwZ32i`bjkmx$FjY)>YgO*~zCmYz(ylDL3)KJfzLg~Y|gi-=2z7ZWcb zUP`=-c)7GCeI4n)5w9oSOuUt8xQ)1icsp?=@ebl+!j1J>x_^cE8j*b;cUFP}X!d>F zWCz;BNQ{QHVA_gH!&)$`1;eq!uog^5+Oif*XVR<%!}2%XXu|S0JxFt8F+E9hd@-yM z!x}Ljny1J(VzNm0BeH}|f6^>xQy}>=g~TG7vtOGMnoK25BeKt$D@bEk4!)KVmlJQ5 z(aPK{T*BN#yq9<%@qXee;seA7i4PGUCO$%3O?;I281bLPHN+>0&j_b7&k~;_z91v1 zd69^51oTV9m+2nk2u#+I#yA4{HPRSIK(8l_aRl@R(r=P}i}c&1!6i_S;1WReYs?cq z#e8A^f*n>p`@{1yu~svS*5PYH!M67I3+G^V@a_38V|F0ef!K-I#i|E43B*L2BoUKs z`|SM~&n3(m#F@ld#8TpHVi~cVIEQ!^@lvbl!p7D$OwavH$tvPnrspN%%fwfR>xi!s zUn9OwTu*$1xPfVVllT_#ZQ?t`cZu&2-zR=R{E+w&@nhmA#7~JEiJuWS5kDtxCVoNu z(&~Xy+R9SeM%-?l29vLdSQmi)nz)m;-;mx#{LVgrl03k04ibMP{zUwl_zUq@;vwQ; zVzr>MBusqqD5#}glp%%?Ly7IAd@%b*ePQ$yWQ}0-6Ph)G(NE|~Vij>NaUO9#aRG55 zaS`!s;yJ`~iBC$MVw4knn)nQHpVY7V7jZuk+5r}=!{~urVDu1dL~Jan#pod>O-VN+ z-JEm_(k)51BHfyF8`5mg7(K)*{Mr13+Xn!l>K}=ZA7$1c0Mq~+Nd=L}z7K{%V; zVm~5F4MQs-dQysD=5Kw_Rn+Q!wO31 z#>A%fLD0?YRr9pf+};bih5bb3TUJZ^?M1kS+rF#v5IE?+7UBDM``bmUgo6T62EdN? zx`oeSeI;~)m9}U()>lF!e?V~1d7H3a6gOUb4=~043gM*MKLFEdQo%G-603-FiSvjT z5tk4zCSJqz-$eRm;&S3G#9N8C5myjzC$1#kLA;y!y@z-&@jl}HEP++T2Urdd5+5Qy zOnij6n)oR3G2%aoYlx4t+@4^$JxP3uxRz!367gl?E5vogSBb9?Unj07zCi@nM|pzl z1Htuy;QByteIU3#5L_P!t`7v)2ZHMZ!S#XQ`ap1fAhWOa$j%v`V=3yiLNbf#B9aaBE<=vec}hi4~ZWWKOuff z+(`V4xQX~VaWnA?;uhkU#I3|_#O=hdh`XgFV?87IElu{4{*L%P@dx5QY3JCR65LNb zKs?A?|496a_%rbr;;+O*#KXjDqAjQ`q9SUdA%+k`iDATW=_5LV7)gwhK7t)E!5Csa zVl1&f5u@m$RWgbKF^U2kOP$In3c4w2jG~~Mlg20tx+Q6hqM%!o26qQdo+hIxXmU1; zq7oW88%9x~$=NW93JvZKmkFf7<3V>L4IU3Vi8MGmXmE6(hb9i18DM z@e_#g6NvE>i18DM@e_#g6L_oi4t*PO1@U&`O5z>FwRDLw7Va^|E?OmHCJCtmM@j2oPGFF-wiED{35nrbJS4giT z{VM6#NWV^cJ?S?{Zy^09>9Cxto|2Rsy=qTL(aMu@)8ZsT7VsHp(Cs&6`7FL5R9UEH0c;(EU`W@j@W?M zkS-gMZcMrf>1I}FNpr}%NXRX$m{~D6Aq)Xxhv%&Ra^LIhhH_^l{yJjAEfeR1^4EoQ z0s@gd^F#7Bs$iH{N=BmR@PhWI#Z>Iv2quK7fL zJw<$)wT68R)E)LQfX^}B=ZRZbQ(s!)WuwtfrB1gIcM|u|_FE$R0`9fJAG!-%qSpw9 z6Cp1JI+BQ+6yczVG-QTg&Kgmy5p|)I1EiqPV%0^Ga>xb=eKC=Bp>W~@^UG+ml=S7K zuONLT>8ps#h<_noO}vK4dQ_}ObsbIqMr55T)~RBhD%PpGiMFg$oF_{PS)1xMny(<< z&e&HH?;zeu{0H%F;yuLsh`7B8xnP|t)~RA`D%PfAJu229^uCHq)FAK);IIs-NL)*NiTE<{72-PLtHjrcuM?pw5+(cwaRYPrCh;xe+r)Q>?-Ji5zEAvs_#yFQ z;wQvUi5rQZ5jPP(CvGNwLEJ+8lDL()jkulo6>$gC_BC-Q@f+eU;v_THPIH-P{#&@BpOf?AuR@H@Y`82I)WHU z#8uU2_1DovNUedcM~o%nZeEz+7TDPhbsx!v&LW;JeNC5>zEygMzKyto2)RI*tR&t+ z{78C}{+P&Ki4jWN>?ZCdeoxzd#D5VF6RQO=e!)G)FTpE_R}z;ImkVO-xLn+dJ~8}Z z&lP`H5w9j*Cy3h>+s;~}?j^m7_?RF@q|5kRN?c}zPC5^FTj5VVL|jet+Hr(V8eeA| z;_t{fVlme4oYe{I_7z22Fv|V}XG85*C&gP&(Bw(tQ-V5*7)^{J)+5Fe>kFD8c24Le zxZf?r{=WDb>=Jx5btd))gyZ3Cqa@2}WpAB$6Jqy=v*KGZ6BF!8lWxQTmMLzCd-VhdS`CY*;#|H5=wVjR;?1~U zPjEPK1aTzsRN@wf@TC=3{1`?q33(fFCvgvLza{P^?ia+!1(z7P1Y3y-`X*vQ-xNGu zLPOsa`XkzYOk|kockqRNC%BjRJ#F_9|3y4ZtQJJSgG=-~Am)vO>;uKgXt6f1J@%Kr zE$)CFB*7GWd%>;PJK9@tD|QcHg53kbNMd7}V}=itVnM881aBj*Al^<~NxVZ4bAG`q ziOYz~iO)!An2ifQM|?qI$8211E%7DdE5uicuM^)OzDayr5H}DGu$mPnq6P&~gVR=_ z2Aks$XPDJ+irkPWIGi|wIFfj(AWB{^i&#$c9(t)=Z`xRW6>%ByTH-zSNibhUe1P}} z@zLsCQ@7~HiBAxp5j4@&)xbDnW1`pY0+V!N2C)xurrjInWyEq~C2=l&EhH`?o=3cz zzOEzwjd&yR7Q0cQWo{>~B;G}Qm%iR3eoEX({EWDXxSjZwpvXkq`z9|AtRp7)^2p>zZBTF{=?? zTk=GJ%Z`6c_$AX2egV>f*gqM66Y5$|8qE}lygX*mB{$(Y$lh5v$4Td;r^K{)J$ztY zgq{t%NvmMi0GyOLDUOBN!AUo8)wZ)F7jLVkq#|T9c?;Zyj_-{$eOVA|ZL_yedJyst z@zwu6PLZ`qLyy+1thEh}pz7h;ctsj(dVp6DwhJ*Tm(pv5MY-&Bx;&Q%`D*+*=G^HWRq2a`3BBunCF!Zp*aou6_D>&*VN3s?0fjbtrVPpUvJ_7sk1h$j_22I=uUtMQ}|mg#2rkor9^c_h;IZBabj z3c}2?CQpT#JvmD_9}p?3{(RCp_%rs_N%;(~UExlN#j+-*!tD=*QjT~V+Zu0Q6=Iem zZ3l6P-|E7(md?LHB-aPMGh!0HOXv%m-}GniE_7xqu50_7qU2zY_NFsA*9w>wKprUY zXa6)|ioDtXDrjr(nh+AOsY%&~ilndpX@B+?Mfpby+&)nFn7v~XW~Rsfv%f-(V$N#s zbMj$tsp&`ZYJZ2YzD7^}6aMV)5&q5-6EIp#je0$1s7)sev2$GFDo!chhX|E@;d5)dlRw_KGnZ`Tp>P zB?!acj_qxZJKB9X7N>-cdwD+*fA;G&`O1k1^EZXjqLBIZxwU@|x3q?*`%Bk8s0GiI~Fgtxn~pbEaPI}4Q+k%@y9E1f4shKb0;V`V1FSesXLTXx4u8bI&re->wfEO-m)K#cxrR98%vpS4V9j$`fDTah{@QDifZczBvVXz) z0DniUk+3E>R{Zw&$DTiOO|h?58GUeUKfd_vga75@-s~~2`O^Bo_O$OSSSM>jdyU&S zAD(!b?VCqG=gzY?I(^3eZsJD!VYH2xgW7I<;ZwnRwjUbX8apCe?A2r5@|oK!ro`IM z!QT^wnedzFGygN5_J@1uPKE512d03rcO&Ifmm0swao7CyxcNiuWrT-4s zf7q%0E7tUTi&|oy`jP#wN&D*x$o{ydx&65_SF#s7JjuS^>A&`UlVk070({DTtH{6a zV6O*{{l2K~a;q!S-(Y4x7>MgPI6omy?EP4KbPkjU)`~Uw`3Z^S7=hSz*3dCLxK^^4 z6+DL6m_7T}!+zf?uuiq@TK(lH?b^Qc==)y&aO)y^PFgI+cC8(@x}4T!W^X$F7-8?N z>C^u9*y~^W$79cv?60xwTw|xH-h{QZf1jj| z=>Bvg>_<-d(9=>*F7v=X+Mkg>d*2@m-`+Jb0&|hY_AYmRiKo5YopZWo$4qa?*bc$=_9vKU?m!s3 z$NJA`>_^;v?2Ti#*gv7~ z?1tMr(MF#N;*E&~Q-Z^{@5PzaT}8uW9`7@^S0Jo)mAU z>+N@I@)KWNbv^Bk?i_hn&9epJg>||2nPI=+ch8XQ{cc~uoZc7C|Had{$9Tj%td27! zyW%)=`QW+f56;~2{~}Kkr~T`(=ZQ|bCPrA+6wDXpDSzN;yM28kU}jn1Lu>lNlLXJPTt{J|QHhc5r&GxP_JLD<$e1B)s8XwK) z&N1BKw)cULiF?|75P^dzZZhsayM_Bh%$`bF8O*pw|566?dLkjXk%Y9r+*o zlftL$b-$N)qg-m8g9q;u)s?2>o-N8;3v0DtzelUzU^ha`K4fjP-}`Rw`u%0VxR0@} z3{C_4LgzrcKh_z?Z|P^jVgD8yvSzqz=KppM^jp&TN8J1;_DTLoy8cW*zVpvNGmaA- zwr?GFyl{^--;r~pjjlDV0=jwKWH1BrKI*w+4G;{k~XFhLlueokLn*0A2 z^V;j)qowPZ{kfJ*+R?)JPn!RpF;v__2K*y?OTiSMzyHNKwXb~b=L^^SXy4O*9{UGx zkB#%a*M8RKi!v73FJM2Q8haxcw|sv-caUQ_p|Uu>@K5aX82f2K5!J<;Y}=VjVlEUo z|N0HGs6oO0&yha9{okDM&*0jsno*lH`YX9ukH{{uVYZ(83a94Xj*>`s3D=%Le z*gsFo3I3i8?-lk3KHDRoVqe3wL(T6o>e@rJSO2knxf=1Dz?a%*t1_>Z{nNVSxFuTO zw@IYsyPnR0^|zdL+Mi+n{wM6mzUc68$ev8Gm;JH*-s5G{{t4;)w&r@-{-NfXcwJ#& zk2sUxT0)M1`Pc{<0r&efA0PFtz3|O*l)Vw>6hS=HN~*P%lymS8oVBMfKG}QcUNrU_ zg4>kO{zN|Q?~g9Gk2NsQ808FGrpO*gIW2*-ytB#NHB=&$ycVIlgm$ zUwQfR!?sgv-`i*R|DJ!W1lZ2q^LGE3ZU^-{cbxxYq5pUIveD*zDXHD2#N6J6euVE+ zAd|~=na}P2@-vt(*md>C-IMmqo>pCUCnDcCpL*(8;np*FP$8!7s?EbwHIFb8F z|6WQ?jPJVU4|U z4ZVoVt#6J**y4-PNPef|v*`h~3o)=rBB4-J!E}w)!Wv6t1C`!pEtl@CjY1 zE7g;FfnK1V(&y{T)zkV)y-dBTuhrM8_4+#fclCz8QQxTErryGLL<6yUSKq7eMd&`t>bgx_oY z)DL=}{+HSZU4_-q<|n!e4?*KtC?+P>uzSQnWwv(1!j>>H0PP~ z^hwZ9xI`zLE6kPJYnGX1I@MfbuF+}cuja2h-7GiDb%wds+^RFp3bR7@G7p)@bZ^mH zsQZiFLOnqA7V3ebw@?ofy@h(P=q=PkL~o%U3cZCN=waq#vq_JF#=@`kcxZ@jhIS*r zQ2e^$mZmV=!5fYrv>@Xbiu-*dtt{N_6@?qPqH%9eIBs66XSK3o@jD4OXVr&R;W(=S z%p2mTtVa0tg;wUqpmDP=-kRbU1KlyraF?&#@T+mZZwsp*!fOeA)~)b688)pEN*nxI zSZ(nO!##EF5no6AylB(7%MZ6hB_Ks|-&9Yu?QXE`4xONpNX1ErM|4HkLksud_P-ST zVxe&^6+Y7N8;m}XfmDbt(H?03^6P_NFXSu>uKMDaf)w?G{mJ-cSpD(ySlRfcq2J^n zZ0R}4=sBk%XQR-6I|^!O$F4@nU4vgM>ssr2l-%E~8)18sh5I+HJMas&?zHYg>D_JJhtj(r`bt|{4_IqZ zdXMAR2|7!kLWw<%zS;tP^?CSx!Fmy)t+m#p^xnfS(Rv@hX!PMPQA%5JM_~l+F#8&H zwG+Px>l7*5oTMVs!W}0g+K%O1y)yeA#UJp4&AAXai7*jxO=yS zx>!Mnx4J@IiEysMt+=7;M%;?4)J?b*H&opWp3)LLpez&tG`5` z&@BtPnTB>I(J%|Tg@%qM(K8FWm2L$Z+Gatw(QQCO=Pc-U8n>j0=2_4kbO+EKp(8C! zcha3eLklhFF1ib7=%NMPRd)pqjkKV<>+Ya&^Bw4ubTVjYrv>fRUeKvJ6$l-*FiF?x zprNT2bf)eN8v1Ij#=5WW3mRH$VSchc88pTQAhg!P7UO{xE#m?3e0{#vR4>&_t!~g` zd%5Mo*l?xQUtgt{!2}}&@bCKXRu>r|fVb*fEw8>!uYeZ!+x6`TVWnOP`VM`El_+C| zl_Fz?l`Uh2l`3O~l_O(@)mFv~AT-+oq1hG)&9+vgj2TuNj2T-IKJ?py-mbTU{t9|; z8t9$4oihSs$!_2tXuN44BMNAYC{|AyQIG~`zlBM)uC}6OT(KfW2QDz$L|aMFgd1a} z$rxkh$`}I^j4{Bbrl}QS{y)yX1hB2@O8YH(TAp@Uk|kM^CGYVTZ}CnNLJ~p>A*?N( zmSHPUN`V&WFr6-voh~fX;cxp-8QRWtp&h0$9a>tZ1xnL|gpi~m3)vhealFTtZP}J( zNtWdQyZ1@5q@>XP&(_sDcVEvr_ndRD3i(3BT*cQjC4{apt%R;H7D89x-pSv^X!(2i z`{07QFl~goFq=SK9%riY8@OL!+CgKUU^esr$bS(&pfqstOSp`S-@)%-DhRz{iV3}e z3wpzJ5PHL0P3R4LKyR2%LT{KYgx)Zvgx)Zfgx)YZLT})L-oORDfeU&A7xV@$ekT|1 zVg4{<<&W@3nL_+#?)yv)D9|aUi$Bfx!w0{d3!hanO6MqlmfXZ1u6mE=|J9u z3e{6OR7dGhC8a>Mlme08x?T^8M<`GgZaTdRwcCWhGD?9=lma;@1#(adWB~>G9D0k; z9*%hee+o)i=rq&PjL4wr6OxUMds0OAxcXYm?ZL9#-EY}`U9Pi=!$|>0V9hnup6Z= zPS(N#qfr{;!mn}T##~Bil&p(&!=Lb;0_8nLP?|3@a!Oq|(3M``-*2#>wd|AZlW?DA zpN30l2}fxON2!Q{QV|)YB8n_3qR65miYzLkpj1RbsfYqpO(QQp|3mn4P7VU7(nqrI?*1Tt8DzxPFGESl&djJWH`W3oQRl z_!BJerdYm$V)-VD<=qs^yRi@WSHvN>zKG)bBC;ER4|W5LNO5{I_bT@)Q%o^?CB^J5 z-2ZaFgU_4Xn{Wx1w{p9;h z;e9nX!A+#Fzd8&1tGO9&CPf2Olm@8Ct^#TNaVZd$Tn-~w@%r`Tu_%-|*;QCg+4L)6b7hLQ=5C{7YxSRQ_f#tXJ+mXW! z{0*QwH-cs;K{Gyy_uD}^l)OYa?&k4p30d@`hSHCAN(Lw2lm(mX}r5_eb zKZ+>*5Gnl-DgCIU^rMr~50TQ3dP+aK`2XZzV|f1e{O{osD&kC0k+=D`;Um$Kz5HIh zOB7`vzYp&cUD?m$Cf^iwImjQxyF_CS@rUrfLTOx-(iBiiQ$i`tYD#GeD5WW(l%@dt z*?84bI&3ls0#)bkCwVt>YAv#nYwM%HB+}6u8bkK*pi!xHOxK0 z#Ru`L0W4F^-pkc+*K*f!+qjQ%pWr^p-Ok;~-NoI_eU^KO`yBUq?lJCh?hD)(xqsrm z#O>z}aHqJ_+&S(%H^5!smU*B(zLLL<|2XgGd-!kgPx8<6SOK6rWd9~dIZ9A6vP;uq z{{x;8{JjZ#BP(dD4HVId{goS(&x5^H31~?#c$#B9Q^w6{S$m%#^*JBc7j@t;uFOCVSEMJ3Qy4&v;`81Tyh?oByLhu;t3r=f2T;SHt1?_InRTMZujH%9$2 z`nH}Ol~;+k!cWDH3O^99#T)9aE~BCM;U@<^Pu{Sz8RcVQJwl&hM;TEh;YuF^?(;2R z5W#kVl`VADK*s`lTP4P3BI& zW2#q?Tx1($*AhS3tt%mb?vxmix;N9vJx$~UIp5q9&+JD_`V zpvUB2kzhG$A%9LzOHa8ueI$QPzL&(5pP=planp}HD5s+-xe8g@pL{FKimcJZyzb=Q zH1u)!?Vx`3h(g=K{44h+`z`AC-*Ba}8+;@Olnow(yj_R3 z(Hz+8P~zLv?>8t@irqwGe-fpjB*bn-I$$citwZcMeQQPzbM$RJ;%&zp%T~~~RRV6H zA=jcNZ_&4HaHYH}e8`7GtLc2S>|yqga9PLddm<`H0}bhQ{$s74y9x30CAViM7XDLj0B`4m?AQ5FD7pmpzxP#oExjya;V2)B zi}d#{ydT7etfT-wSbr?OSd0J}O|lBuDtv&yS*#!SMtnYj&%O9Of)C1ojF!PKnK3N# z$G(?m?j!xhwji5EAL;4OWcqV8g%1E9mLL>-7)E^q%HkQ!SI&%*d1eSR?(;Mh928rI zTJg*v$q^C-j4I7xFq7kz6n+;mZ-#7}Xa0=TAE&7wf#1*3PVjE<`vLWP34Y&3Ux3>~ z4sq)D9Q?iu={wInm&vu1=K3pIDp?WL6#g~BPti0-k){Pw7I1#ZyPEp_0I|)?TKN4S zlaHOI3nLD}2s|@Sb9CVS3L5f0^5N(#tfz24Vk!};!}!%h8ngvv+y!aPH<{;{7cp-8 zm~)Vlpx>~w6;T$$)X;PX_*_2R=D7JAvYE2`st$ZpRq@j;B^#!1mVO7*i?d%|9T2itfDg!N-^v?!NQUPcgr^`|i&^!n{J=*YA1g;m4)xR z9gjZR(8zk=HnA0Oo7q~pEo>9qHEjE%kKgquyY|t?A9aIzDhP zt{%8NzGc|=5;jD7Bkx2DNy4P@C9s$JI;l(ME(_TtakbPXE0V=dnYe^UA{t8aOOh=1 z%fv0EE?G&0IuW;=x)sy~y@d~OC|o^tjnpL=nq^8L>Aji#39$I>+(F23s`zio+GW?u z9+JH*dtG)&FbnngyII&NJcZrFPGL}9kH7onugH(eLyCIEvx+(8&8pYM3h{pYjjGM+ zZR(eiFUebpn!dnX5A9lJug>p;2JLI?*P(%bfqjwv9s3e=UB6`im;D+0CXskThG*hD z+?8Ad*TEG)TGz?F&HagchyN45i{H&(!TQ)8>^Iq`*k_;i9WV=s8YWrW1(EbnmYsh0?ht_TnWT;liOC6Anx*-QGh0L=O^38h4 zGS@&>xt6<%+YI^N9&R81HouqO$M5G4@`pgd$^6k_Mm++3EtnM48B#akJ<`uy%Eyd1 zC$Kj0BlC_!oJxEvAkn!KlAA9H_Qbz<3V z^g~LLN+9!(WxHwFB!9wt$msmLAHCvWZ)R@+hx1wXC(Pp*+2hPhuzfI~XGnT!CbTex zL{b9@(>889cO!V6TQTSE=03&U$9DDY!=BXr5iIJxT~zrUBe0sQ8E?k2dWxLX(wW400lze|Ny z-3a#pcQf46+^vZ5{j3=G&^+!%*naMQxTm-W^78l;d~3M-;2z*U4fizn8JY)h80Gp1 zubUoO!V5!UjXyFc(|x3aBy5_$hn%@s(_jnf!+W%7qM7(QgYZK?6r(HKM2#V=o9HuOmvs)d_w z2ef*y`k96g(e9DelMdO)Xdtca;(yNn8~DjzLGt`4_if0=ui}Z6F&)c|SflKH?EUPg zxj%5f=D*56#(jsk@Eaik=kV7K4I0C2#;#OE@jv9Bh0fqbNV^~4o`fzo zoiF+v+APvrTQYtegB=@xoNwX(ga0=7TkaS9KkyH7PxIPLpOj=;eGjc4_6H;nwlU*> zZOWg>P(|oFUgTbd7NCiL05y0PwK#5JTji=6=Ebme-;#526;|MvabVY9Wn$ zEpn158rd7O5__)>Xg${PpJmqaPe6nG9io|rew(B+bI)@Bhx;Gy74CKJ z4c^2T@Ev?7|2h8i{FnGI^WWp2<6q!^#-HaeDYrhPef9k&A+Mz+M4=cD{({qVTtsznQ-k z6b1GN_=C*>{&o;u5dOYTG(h+}M*dLmTOg&m3j6j=bWLu7ZPr%&+TX{SPcWZ^4ddT4 zcQAKiM|3yysSm;F&(QIH0W*k<`D?n)*W^~STRrG4O%_8rSv;J zq6DK2@kjahA&Wo7_woJwIevg2ulO(y4tW$|g6pL)_F%wqq#x1>14f=;UvD?oFH8 z8(Q1iTDRV`b?Z$xZN2Hbt=C=WzwWxCqP8}<+-|oSkIz@0dAb`L8=X$4w_sE)-<@EW4tDDstO_k4F<8$dWe3RQ= zs<#RXnJ@2Ioras6>*`i*ET~m=tS&3AtMmElinIkjvoYNXnQw2Y$5G~Tlz1IphtHbs z5%OJP@_5?NF`FIU&6_)N{sC>?T-;n-++1H|x3}8YbXB3hu3YJ{N?&CgebQZs9wdF} zmAY}IBRA3qz1rPPdX#i38cr&c{>!|eem`b?w ze79MzAzykAaZ^u=&$kt$Xa(49HNf5OD@YHOT9k_xQ51bXQG}n|Cl}O(*EyTflbaB; zz))=P84O5dFgWTQHK-*fuA_O4PG?K~lR`XK<7sPidc6da%OzKyo&i40<1JD>w{7!X z@7qeM++p=@xWR#0A(o4tR1Ma8D|MCiRq1JBwfd~9=mVd+O?6k)VN9KM)=CngNffI^ z@v64&jaO~Bt_Ty#=i9ceqRHp0tZZ*z$2lBoRMxS**<`~c|WPgwi z3kGO|Zxnik9|*4t?+OFLg4`jmlW&*bF8>$#8}h?2M(j{*SNxmes3M@0E1Q(JE1yt) zTlszE3(Egd{!Y15xnFrqc}6*+oKc3ANtI5uO4X$5Qf*RwO!c7Z1=U&A49wg#Vxd?f z)`%O#kBC1PPpge;yV|Q>rEXMrsIOFCqrO3XoBA>Jm()Adqv{(<@h^~>s4)o-YG zs}HD;sn4i~)RXFY^%B@HwWdJh(0DbgG>w`L&6S#KG&g8&)7+`~wB~b~f7JLjPidal z{8aOj=2gubn%$ZMnxmRN&7fvNGpmVeS#7DdM%%3I)^5~ZtG!wK3GLn52ehBpeo@<_ zeMf7kz9|4aRA`nU9Z^@sE)^yl=W`e}Vw zpESq~I)l~VHdGi^8`=!(3^y76&TzNkLBkgfy@sa^-!XjO@KeJ}hF1-57*gv z7={d!#?{6)<2vIO<96f6jCUCCH@;;2lX1Us)HrPn8zA!xxBiRuIqQ$Dzp(z^y36{$b)k?g)E3$cR~2>^UR(IF z!UqcfrSM2$zs+vzwmoS3lI=UTpV(fr?XvB&9kLy__1lJQ6Si4f#LnAAyU}j57uzfB zb@q?iAGLqm{zLl<_LuCxw(qkK;829pVQ|=iURODq9bJx1j%|*QIqq^i=y=TWCC3iO zKRf=fV~^vI$Tkd`CqwYTUpnJkS?~WDAigm@U#cPW<7jG;6Xz}gEpDKQ^_|f8jD()@*=i=`b z|G4<2;#Z5`D1N*6VDbCK{l&w@f#QYYWseH@)$Z|nR(Tpd9iA&a*LZI5+~#@O^BvFk zJwNrlWg&J>wnnPI~7{^d;63cS%J_eaTZL-!A!n$kUaono z)>`YXt*EW9ZLM8fySa8-?X9)9*M7S8vD&ZJK2`hs+T*peb=taBbzODW)!kh8iMo60 zdh32%x3lhi-Duqu45&@@-uj06uKEr2*VNxoe_Q?C^`EJKwEoNWU#tJ;`seF^QvYiG z8};wk_ty{CFReDM?pS@t>PJ@lR===%*XpU&3#$_i@&F-HqjqwT;bAt3ioBpBct4-f%`j@8fHT_4^e>VN9>5Zm+P46|GXc}n>G|e}~ zn$^wj=JMvc=D%;gulaM$|J3}o=KpAZt@+Q*W6g<{f)-~>L(5eypK5u!<=ZX)*7D<) z|7!VF%kNv>YZ+--SR_q*>swn}*S22W z`q9?gTOV!xVynOPxz_(^{dwzux9)B|-8#@Z)>hc&ZYyv5WZOM$pK1Gi+dsDX+P=~D zt+wyA{iN-`+kVsbM%&T0i*0jliFSGW>h`wwb?sZ)x3_<^{mbn;+MjO!-}blK_p~2u zKi1yYexXC?sP5R@@j%DtI-cnGamR}t|I_hW#~(W0?l{nKtm90_NXJY^w3F`?J58OA z&XUemosFFxomX~V(|J?pr#qkRe7^H%ov(EMu5)MSzRuCkWS7vT?Xq;ay2`uiyIQ-} zc5Uw3)^%&w?OpeFJ=FDuuCH|csO#;n16@bEPInD-E4rJyJG$3*U){Z}`s?m4V8jZ$Wpi?Fi35&O8!-fsm8WbD7-m)^vXqT_5Z)|D75ikF`(b3?n z-ar^KgMKzRItoTxE04`hKh2hhuPmhE-F0vTa zFD)%a7sHW-`Gv?*lHonQQjG=F*vG{Ey|n&hKPQIYy~1$dLU~uy`*J@c_wSGo3@{$g zv16$ecOuT}WHKC1Ce?aywg!W#u&AhzaOh@>#nh=BKaA(1va^#LJ&0!vCud}SAK%%z zcW;lLWs~s)!^@IMPR8>b>n~Wlc5R8y5FZ#A0ISajgXebb+9lQIN|fYHF3#auObSM$ z@xAxn8%f&HZ>{#^2mp9Na_{e9mxNwRd@&r3UU>WMw}*u#7q$f+g(#|IJ7~MsASb(2 zXS0oLQf%fu3dh1 z*RHQiIT1agj^xyOyzGIJ_gayq6`fvD;b(Aep|4NsW7LM>Nvs}b388D(?vPS@;cpBC z?osJ~Q(HSRF}(g%OMN|KLzI@5UUttOKR!|pcEr>M0{0(3bH?TRF^Q#0#X4|6iqU}> zE+J03GA`)IzC$NWy^m)~aFJ=@Vnn1^5_Yh{l3&~1?TjjhNe?Qb&TeTen~{S(5Kw9L zScE#AM%B>Jz>n?6Gr~78GJkJDY;I<1a;U$*pTT02O;4Z9%*X4Iw(5gQ*TzRiMzEu; za5}RqV$7|`=x9Rcmth`QMx{D!LcS$|fT$J~c=Sdi*-v9`o;ruhaYLAgy9Z&w#QJ;1 zWMVlXDEUNue$lU=96EpgJmYe?6jO7jGPV3Ha%+u+rUQXMeSN()Hik9${<#a{()8ry zld3E(mcq7LZr792_4#(?2jay%6u= z5(_ha)6Br1(Vu;DoWW`@^EwI)TDdeb&`ZmBvIh8baajq!3t~7r=g5A{B8$!dPXYJY zWWf_BPM}|HdPeE@b7FzfFW1(fuo|tOGZw(Knx-N;Xw^I6t|YTt*~TO>rVR~Wry+DU zoeZ~YOsB8bna8WI=`YZ3WITw`+Um!rmpye#su9tF>*tOKJ2>TGdrRRVCD(=d)XO$C z?O+cblH!mVryd`VtW#)@J8Nqhv){+``w_*@^h+_w+}FlpZ@w9ch~kzl9EaNY`0a1K z{kETfWkYwj6zhJ(YM!4B&dkgN7nZQ1FUHhW^`X#0B+gOcj#3$#9vmDzJvg%( zp~MQaw+V}JdKxRUrP^uMk^Qo$*2jm)d>)RQG5+;tCW4Nh85-&>s;DS7A%aG&;BgOE zd%ImGpFMi?=&W33ZwCxNqOC zy{9g%sV0eB)uand#T6C3#-X8^%hq%=YFZMUCG%=}dUhcO3YutYG=xSD?g=k0&NsdO z`k4^S+b8`!E>=JfU@-Pwh!>LiTS%6IE0N?C8ZAP#8U>$BFrch5`5b2NoLpw> zl-2>k^cA!oOG~qS z52^Hek`y!MY-&7jL)_XWNaX8ko${qgP=||QB^REWnhM9`g6iO*@wmRGrlwGskIIme z1#{R_F~yiFk*8Z+2uzG!oSco2{#9F(6By~qq|RAf;&I!7zUe#z7pa9#o|RZ4;%ee^ z{r&NNciioEiZQ>^Qg%w57Ecj8OH(wVPKCjV!0hZS1g8!$ipfz}2$tyh2qXCQ9H%y* zJVPZ&yoVJcMoOvTw z^7`UzJu~hnbE&RQ`uN$iXQi4G>Y^V%GE^Ps-Y=@N9^&E+1m7#r+S*#oCx3>ztVhh^ zv9Z%9Po6w-{M^M$>R!k#qUJujyr-r@33LMFWTI%SLL0msS^Ft2j#8LV=nsDgg+y_~ z28?hVwcB`jt9yzfQH$s4t?b|m!Em78wA*?1C3{DIWFI+tQCp?1- zI#4N0jM;^Sg*Z9<$MVXU;90X~4bB0E{633bTUb(3S>_bO;qxeJfLU5fCYO73$yj(H z91KifoSIixc6O%l5t;p!>{1AYfETc`0WiYHhFLI&zy9^y9LH_h(ppwX_#!>dQ2Tq# z$yhYH>;yS+3aW*(pqi?kd-wXqSC*sED5DgW0xR_mYzsvdlNJx1t8sTx=E#4>i!UQ=|qL8GUJO{mhI?)!5k9TJO^8z>ZriMi{91 z{pMwcMU&8o&SI7y2QoQ6IyL3jz5RAom0JX&VFjK|q-h=H1Ng`+8K$^Gp)@)iCr&IV zoEV_a_Hu{SLI$o-C(M(P8;%O53Nm&$De3nMGTHJXBkQSsi3l~ ztc)*{GXs9VewmfYnP_AQJ)exlqw$_vII$$arAU;K%h+X9x>Bp1!`5z2tF7z=I~@#? zru7GdGKYj|X^*kYA{meZHUN}<=&&Mad`dkPO}cQ89mobEd?uyI2%sYS7YrXlf2d?Z_*F|xsQnr-Oo za*u)ZYgs&AM!pSLs@PnvUEAJKTiel2U$MP!@RyR3moHqnP+VNhW@^C8vE;zl#1h$c zq4}oj5?PeV79(V~ky?33tu!qy8uINi(0oZsWROT8D>Gbss2<7c}4 z%xltY5&`?T4a`^k>}#*3>ZpVthpTe@%!QQCZr$2mE@bLKYTv`G_xqT)Qne?&;KM#-y`OnYs)Z4q~GF>B~n^~z7^!mB6P-#S5uRO$v$KHBt0Xfs2zZYdTVe$-(o;>u&KmPIPbX-*q zz7>*$2vFcMr3`8Z(RVKgA#(~Y>w$&zOu}#JRvAhvJjL3`*bywm!y{ppYCg2oqh4N$ z%+H1vKrgs%@B>-WgujYRwJs^2(Xowms_n@Uxh*F*1x9-p9Md9^z4FCS=0m{=87vyMa?`W&VpL) zy&~1>PoUOylOv-O6B8rDV`CRUvW7<{j~z>BAYds{CU)-@ghT?L-MbU|>gsAEGX)tI zCkQf*S>|OtEKPds*sz^n;7pJ>u&l!|KF%~XVILjQdYe+EkeLlKJ{JxL7u5nU_ZK>y zR(qkbytbmE+T*FNsHiPB7TT>&r(cxwf_gC+4$onts0ZTlaCC82(8%SY-z&=H8ew)Z z3Kn``V036?^5o$YCyw^_A3bs6@X5)Mq0xaeb|!H8ol}HDzxD7ty2Gwae=r8d|%yR~Q)Zcm@Wt zaUaana%wf72S+~yUdRjAGqRlgk{MXSyfS}w=|jwXPVA{Xc>Y7+vw7jkJa{n=p2>ql zdGJzRYnJoi#SBbZ<71Y3r5PYWb16ugA5qM(?D@gx!Kyr1n+MP5!R99ey0cF=ImVrqR_hcX`cUZ*d#Gc55)fxD&Nu9{V?m}6v zNW^ID>?EIpf=GxkRG~=3Km^o=2#D0ms_G$lD&be`+cy-Kx!O}wIH`sK2z;~KGnav- zKAug$Bh|-JiWvxSvr{a52%N}-Uv5S&=FCXk2a}UGHefk0ZbZs~aZgMRj9bQXU@;HQ zk54%K?6&0R@5l>RgBQ<@Z_a@Q$ab^CXY$}>?DMn3g*=$ez^xeJqAaCB!-_6}YFlJ} zO914^oLnBq<`OC@HBLq^%?ASCkY;y1(%YcrIClCpv>Xs9O-%HkAsP>txP&P!sYMPP z@biTK_OlhXl$>A#(iJP?($>|@4?*%WIB2zk=Tc8khr`v?O-+o+Zv&sO#BocutFF4L zaB25$mfgL3DarOhp~LOk)t90|RD$=xdCe|#l%Abx)MMp;K%N5fWHcH-tI=fR=5Pjc zOEYsW55`6}r?>OtkLQI8d2l2TR_DR74E!8=ZQZ*EAmuuEaR2`I4ncG$aOdBC?S+4T zXYbp4m&e|D=N*O!D)sY3ct0BfJTNPiR?f`M24^SE42+FkI5(&Wgyc{E^UHhA_?h4N z{a#)WnC52V6quEm*3;}Sk*lJ(?s z&LkL%WeziAA%`Pa0xOk0{{ngN?1#X?O!)0+aa|%dPd2zgD6$S8I)38B#bu=y;%NmF zpF9Q;!oI`M-G-(W@1P+S}B2?{?t(^aF#sZCnrMl z9ErY^4kv1IdZLzDn}Sm%ld7ajCTDR}d}!0nJiloZv&GNP7;D88K1MLlzXDjsB#;=$F} zSPb+5eN-So&_`8aWSQ-;Po6$KC21q(A?utc+6d^VF2NKLNMy2CP$C0Me!p8tOz>XBt0+(`P7DtYPRxatxaGyk(+3V5cz-zP=Yrux zk14S*JrxK{%|;V)b)ns6HW`#V=>D|?wJ?OSwF-yB8f#fuEBcHnThYt$L@$?;9HeKX zA;vCAvk|L8kJEiC+6YK!bje^^AK}98FXq=+D^eN>CTPx{#Y>PxXZIbE>Sx38JUBo8 zR9?842Pg7iLmr&WgClux0J@Cq`h@es^Xn7H3y)>STEYs;d2liVQ|bNmbYo-tni`iy z4{Cx(Z*kSEX>V+d4`V0~K;^s|j~O_$2fOqvZ8LOYfsv8R>aI3nbW14KxB=nWt#!wkX9& zCwA^E8}`Xg5iL~@4~3O9EABx0)w8n@aLh-b`?(lV+KLkK_)?4{XK+U{q!h#BQ^Dm) zgf8nK?9(yJet#hteRP*>0pX!#of_I+C07JtQ;z{cE@-2jPDKdQJF2ueQ}ne3CE1r_ z@Qf|%f4i67UId$5uOUJC(Bn?atcuXFl z54X_vNv%j}OsK*HjtVho9Fs>i?HO7&yS3^ZRW)G#m*l zn63gLG(I{q6N|-{7O_Q2^i)Tel*)yn_aT=$522l>L?eexMx_=7rp&FEM;B(NhtE&O zSZzUJu_W=O`>k+z^x_neZp};uwrsK4Dk{!HAtm?V5t$ahPYF~tITxIpi|)`IIl`Ds zz^@~-vvHG@$1|jjLV{XoEhGfJ)dMX@na8S^>jZRvSkTE0sDH6Ru7;4`pi+|IgT#AQ zQ3zGo8nYq-fQ*Hbm{AD|A*zT)dxdCB5fv2v)v(Gd3X3B|e=#D4i)!8OqJsHxqIQ{> zot-%Shd=z`(TJ5|d1l`MnGkP)a`7Naj^Iyga`+8Gpo1)SlX;$5Eim~ZwIhEp=93|%Tgq6q*7P}mw!LhfT2jN{K4tcHbtbr8ngYQv~| zkle!W7<7*^xWZ{L)xP`DKHfMz8|~3WXQz#3mC z{y9cp2Fg^X?bYPQ)n&z{xf!FgEiF|>X8I&B-N_low5laFQlCSKop}~l#|MTqi6C^R zGvPQ#%um1-oSu!VfF4UtnxRD%wYci*vs(ZEtggOE`)WN(E+!*KA5Lw?jEA;uUR?^L zSWwW~T3pQNpug_yyl|nQAR6g0M4}k_5s2bPmJ13lT*&BViFTsk+_@8@4GI3Rqq?Nz z{CTgJe6U2uCjBr+@{5yW%+9&FodLHykZL*6MO7u@lFkTc`*^kPy~F*pjGpQ>EA-68 z!-upgJ~=pOvk^4-p*BO`l;!}i7shDeY?PVy<1YB*XsSHIiCs}Je>xhPq02plsTGMt zLl=li#)Xgq2*|C7%tJ2`nvcMM5+;<95yBmeMASgVC2Gv?F1BcSL?%lv#iNK8jV~o- z6&2d(@&riD#Bx+yq0z{B876<6mGN>-eZ7TM%wZchr(iAhse0^0J=T$W1gD5}XDTRJ zOf5=_d{GH`vqSko^%x(YnsS{x2=w^I!E-ZYNzIH5g1s6W92qCt+;Qflf1OlO2oEBp zqEMB{%FES}I1ww1$K(3)531;#Og2{p9ZWnGJ)$lrZC)a6jw~e@S*j%x&uF(ZmNd_p z@{z4bmn0utFqDz$Wm3s=Q-N~MavuU4fN;b$2 zx|iL;kZajBgu}A1xK8tX>@w>bqCUj#Pb`=?K2g9-S!=9QJJeHFW};wK!31NSIw4{F z3WhO3J(5DI5(0y3WS19)h8EN=mpc7zbPc_AWF!(9Ir5gW2nxH^MM_!T&`=#7zT5v5ixuWR&u$Qz(F|y!PYmU~3Xp7b(t!w<9so)aJs^mE1s5LtZ+f6!{ zaEGG{q0rpKNU!Ud+*ORVSGl?t2(G2AE;cj(0(bhzaZuWW@22M37m-uf2j`RuRmv`3 zj%l{6>k4R$Hj$IM*WGx_M{I1eN4+>R582Jc*l)+Njk2oDItbmYc76YzU9eSstEwu)SAhZ)2Wz?i)T#clq9QdDf?05g zQR779%6qClRY1EYkZKT8Lf7exBenX1p_zbH z7&~zyEP#)rWAstfe^pcxHBXJ4JNee1-raLzBCx=M%D^yi>^-PU-=A7iml1Q5GIg)N zk|jbcSjda6k`edXSGvNUXvW+}>H)bEbJ+h%Kv%ifd4 z8+*SdgW&?qvZeAtvl+waDAcs)&(Fhd8Z0W9)Ezzn))}4WCwl}CoW0((#S^vIIvtsq zXf_tUtJ~J=6DB6g%F-6TM2ex6%SYsLhvU{;uekMA2TVn{Es(s)_wL=vtS4)WtbFLf zQ!8-J-QVJ8_on)SL;4-5dS$;$oWC{2`Ckq)rynw+ru6a~GuSPtaOflN7&A_wA95N@eu$};w;u^6#W9T*^=k`i_S^LByNVgYIy!^*xmW3TFm zWiS&6#9((&X;3O37_WBr@^OtOo>~V2eEld?8e0iUvv4`jo|kQVS~)w;&YqcuJhs0xLFPV2&6Sn55nw zPN@XP8SW7hF!jF}mC9`+dB?yah5PzOBQk?&aVZvwCVN%!Sl~>b)1?Bl)+3_CFq%CR zQ3~Ibqt7D|ty?Nj+CvASInCaEu4rQ=3K%0EITD(c7*Y-^uo^Y;0rw+?pDD^NWkp9U z#YuM4ICD~*R7oUKs-#pb{AdkwO2ta~)m+9;DlxS~l4>=IXxZ(;QGy&8x45U^G*ynv zab+%D6H*Q`=M}5Mhf_4bna{^0bfCM?6tmr~0S{wXy^P8e(lK8Z-^MAVFL~T>~Trn68&S9VE3( z#VB(60>2WH2d8Cl(%cqOvsp^LqOXwPL`Le8&OKLOqRE7l+eZA3qAU7K0Nh(JT(1KYtF?er6Wdse2Ee z9`JJmBMUv+-<&ubn2)P;7K=%*OGz@N(UfOuAdOx$HF^@1*1LjdTRC->3@4krnj2-E zdzrnRl06x*-0)c=s3I!rSUQOv5Sp?~K+F!m3U}2GF*evsL4o*&pLs*-M`A}@oD4&B z1Nnp~0 z=L9h!$;?E_FP`Hk^=3Si?iSQmH9d{pKNj`$VvkcKw9)2tnoN<%Og|xoLyQxv9R|=! zWv@@!jjLNaTX6Mnj}p7{mX517c6a*?x8AsU-9}jed!K-8`%?irFDDpo6Rz06gQYf{Zq?bnq7{Rb^$Pqk;Xj`iqz8d2wCgXBa}{2pNesL zc!tRhm*>HJ22PidP^L?e+9(pUrp&CB{CKI|IkQd3vrbybgO~DPHqSaKA701{*W|&5 zJXnu_;JveX=hDDh7;{gg5t%|y0n5$Lhw~MD+4ylH z!?Q}G5zK@0ZF*M1rI{Nb5mGZ(@(GeCDIY1%znsU1g-c@>A%;kPMxBjOg42YZO8ZIq z{lU8fuPP( z3U0aqzv=8Rh~N@5Vl$kEDBt;9GK9Wk(j*z~|K}-;!-5{Fg|*PK?4* z9f~@fmAAoS(B<+LMP?xVId^_|o>^KJ@Fc>ElV?w!JbM!!?HrA3h&sQu{faBEyn@IMJG;8tJL`3%&HDPz&MFNk<(#&vb1jEo->v=|ZxxYF zFaI4%TOA14T|`;r_LQ$$RZdhxCD_9VAWfLgKrJK z_10kc=+W@eaQG-W)%Q2uqVgR&&+bFXU4Ol#WkjVKQH_kKT+yg2>T=;A-(TH!sl06{ z@2@-eR_`wz9bHtU&%H%ekf#%&;>x)f}! zzl(x2BwnVxta7SQj#06Ioi{|cdd2em{5*sS_FiKwCSVJ$quK@?-h^0c<^1g_Do7bD z%*q02t(px(ECDN2vDdHTV6dtxfNueL;ZpK(?0;DNC2@H?ohvCAEPoEXgiC|7!$)w< zZZ@pSi_hWee;Q6OFSESE$JivwM`1Z&mju%mqP7bPVmpQ&Syt%LJH192Fc7gWl&~>a zP6y^=X*u!bPX%0OYPRIS9N z?ZJ6DMllrd3hJ4%x?Tn3*AB;u3|pdplO?8JX2Q>;PlS;(*NVP?ms4JJH$c~+(O4mg zmS}(yVLpC|82hGZ{3iI6f7qQz!^0jAWJn)$?~!C5KyJIS#v;>!(Tg)7a;7#kJ2Ppr zn4yU>>?jO{rbd)H2p5c65rhNhe^qiW9u7%kL)#h{swHJ<3L*#=xcCa zRC2^dfY^YC!&0kKvuHhjjwE}QtivVL+3@PT@U?m2HF@Foyl|M>=H!=T$|U$BX!RHg*Z1a7nYWKEQ_!qnmL2b*10921Y56i zqrp%hK11tFXuf%2At{3;mO$*EVsdhlvqYQdHJ`wxA2ahVA|pa zT0&doTyo;fnen80Y5M&60H@s!`$m@?3Z3&6OhXT zVS^!@D%FWn)GEkd0|}J{O|}p7mx%>yLynFE z7fRDRX=ZKlOKY_ZAN;0 zJR54c916wQ_+ls=+P^>Kc83P{?;nI= z(YS}2xXBoBZxE5o2vkxSpNkioO!CQd=O!T|pqN7H-+@&BO3-);dJ?_84!6|6Tw!LU z@AT<2!>|y8#dDldkShYsuHs_18K-YmxZ!Jp*pUa~TEbfB;sIjeG!fU~t^&SZ%Pivl z1;UXt(P$iZ4<}+viRGT!<-}4f0f}ZD_hZQAIL!fNzg80>nhlM%ti@vCXM^W)5axUk z7b!?sKDT}!1Ocl<$=|!eIY9rOhCV2rG-p=MX-MO|L=J-Z8K+c>l%F)tDL>4um6OyH zn5VRNZ$c$}70bkoDHO%s9C9#B8ac}Z`-n8HF9_2!A|S2MJBNl{z5a4^l1xDk>!=hm z(j(p7tF62uL}K@c6}+vndqHD?;SG@>UcSV39QVTMVOTgyAA)^hkV##<$QLg#PX}qi{dGUz6;X3G)fO+loSIvB2Cit)OtYoMn_9Z@@x&_ai?>1w5m$6gkw`n ziUL><6hJj`>3v)NOK#x$3l<-$~uqcAEe#r2#{gYfSw3YGSy(cZnup+$tQMk*!PLVgLuB?5 zwG5eume72t*sVOj|AO5W3VTYtxOp@j4qC_rHI$UhT?mGl0l8cN;sd(K$~$d2BhC0= zs-*2`T}6Cxc4n%fz92ph6X5qolN$G`+9}8^R_(OKNj5QZT$$^NacUP35KM^3FHLGvRdi8Vl4__kxjS(wrytYFn?Lu4N%{yfX8;J^miy= zQ{+g0Z^pRPeenIaGTeqL-G7_j*?#yPxKB!>PWD$~YdyJ$`r(ILTa|w7&y-hQsZ`?J zM?qlFW3|2~B$@*&Zz;a}1Jg|m;5SG9%r7-gth{BVv((nk&*M}_OG|Asxn~b#d2)G@ zme-sR-m_XggMmQi?XO8U(YVC_*=%+XOsu@6(p~0Wn(_V;tMw4S)G(eEC+}XG@rEV- zkkwk!pN>OTuMb)uiPMp|j(n@56^S_Ap9N-S$yIIhp_z-LfoWVRXMotiY%%v1lEY4k zC^;_^NqQa7S{g(dE$=Gr|%a&7A#}IPVBKww%#p zN^$Z}D(AqG)$4SF8`^eCbEPyy0-BA@R?aFLNvo4P*K=E6lf`>``L%109?hPME9a&S z)Sev$5N1FT!o{NvMFzJF>=v#|%_=L&l&8vCg}kv3*uMSN#f|HkXT=S(w{G7qmp}0Y z(+U|`aj_q|eEEe7%$VIimR;sb?ebe(h8&p%zp&uG5qY#KiR2a78NiMr19h2Y>X7|NNm}_~C#4;zxsO z|D|OW&pYXs+~KggJm(aq@xM^^UzuB?Vt8ztSXp~2tH+JbxxY|1@CJ-Z9*@hnTyez~ zW&_B8AD6zZP=iatV^Vl}<*!3}7a@h=yt7EfIHgojKt8=hhk=twmaLOVIR0?XU^pjs zcc&x<1cTVGy|&v6KJ1oTZu#g;LQQ+G5+N((ugqK8_PR7blME#C9%iQqrUsU>n#rq7|!^mTtuZ(mt%j3yWA*NFed< zNwiPrrg3v=Qa8TDu_y7}-tV6?<9Mc_Y}X>9 z=ghgxnK|>H|N8x^j`=0@?JsNsO)47CWa3f$BWGzR_hgVSpUdU9;_EEhJ%0a`%jpb` zmTQ7AKK|^nACi-8+Zm_R_DlSnr%nd)1YbXYzj4PH<4Sj<^)(JRVduT;$}VlF>1UA? zkJgOQIw3V?xaPnx*bvaQT5|v#MLG>M>U~l~SX|ng6tCLtnbGBBqI z+4XgmlT|ci`SjQbENqCw;TF}M3#3w46MdB=04qPPd4l_kD3?xt`MZ}LryZwXlDK1& z*6W;kc1*?cVdsvPUx3}U@!T;Y0qmYbuV7U}*_|CZ9JFVn(1%?6_qVZ4V$|3#D-*u& zl5B%Zw+)fd`vt_rT3aCowFtmGgd&Bnkt|&*nmHjDY^kMS-b>Y5v?y6dIqecVdss=+ z)x3_mj|kjO60VW$LJHx$v)?Yn(!^B)D5PZd_+U-xcAL#5^?+=(#4!(C9zONW5A zXd|g%pLKIHXMvv5CED#uv;~rb5j5q*1P~dvU_0FcIKu?nVfxu0S2E+y03W5ktw&=V zv=!*bV9(wRt7{{LwqaAS@*oEF95sLV$)yXjoN~M8=NIN?K;B!bBVXV8Ze~Ul5wmvg z;z#Gz-LQB2lw_6rzPNh*+BL}a-f=7Apa1CM zxnvSb)R~#@ZhbxS*!CXDCcpo|x%XCAqY|P@u3!D4Pi=ofjZW2erz$`)F_M%v`0JtM z0+d>y5^AyYyuGEx8whw?vI`6H@TpV7AjHQe?bnee)ru%63&6T`N~3xmdU9aZd2u{LW2Xi-12SzT6F z$KLoZE(X0Z_l^u*Ghh~)$kOljc)e3@V9ZvkW`sQT1pR*SW12ph^)J=uW#uX`mK%4_ zrlFkunR4l8uJ7V-ur&z;{9(`(3heo)nnL0&9GOhIyZkR8HzEKAAgs*(#lCP zq+%rs2?5xHiGLnF3YdzB<=#C3(2#qzW#ec)^(7x>cSfT>9?AbT9JX3zn9odKeU(a0 z@h4BRu|PLD*Yfh6&c(m=^--iX`0xnu^k_B-k+plxs6A`#ohyD?X_>@R$hYZ@b(olm zurxPsrEzL(BvL|_tj<{h>0?46`miJ?6xAk+6&~$Kt0$QPu&hqD0jvO_;D9CIP1|Un zGx4sl`USB$cX3wKJ;k1Xm{BRUj=)tUPG86No?53^$bA+T7i;nP4&Zb1o5b|}`{@b^bd?yq76mc3EeTGvQqI>WYpvXm zEU!_7+L|mo{1njdcW^GU*zfD;hy#}dCiCqLuehyQO2F+nDZ@AYpq4U=`8afJOYywf z920iBoRY8|6IH$3le8tP2}Ah2uRi%6+957%+cfX;L9^h453O@$C0(TyjLvF$MQukr zi4S-t?Vi98oMVPy!UFhLSyYT=VmZa$(Q{}O04DtiX~K^jero@jGiO3}(oS1LXROxY zVR%SPbrw<&AFTcWqy-N9s1%FMK^bzJ=WnwyPGO7?(iHTKkp1Cc$cHi7YD77zZ5RWP z(D(yL>6Cxx!M(-#e?zUp!4oPOjo!vM^NGa#%=-FFzeIrF&Gt*$k!2)ls{RjqB^4z*LJv~3Wnux6||8SSPQLWxk*TU!} z{}66|5zfjeJ*&&-P=}C5HoHO-;;@OqXv%f~dkRLcN9xr0W$gOZ(FhS!iW;aqhq8x# zt!G}!Ck_k&wF|sFG9B#l^XwecbgUutZz238AR__0o`BcE%tkD{*>-nnNixmmc1rC2 zS-(uf_bj0$yEcGPJcy`_E`9lBv1qe}C@dVYa1ocS$?Lm4KQD?`Y;b#P97*fT#@)L_ zQ86tL++y)7SBgbZ96JPu!zn_!ceh`@dQ~$2eHZ0kX$$YYyV1`-&x2U^_K;7}li$HB zm~)5}$(f9db$|Qj7X_A>zUOkX_ z-!tlNWA~!X{jw+CjGO>QsDoNK{sYW;zVmzu`E5&+IP1ERYaI$gix_R{Dkh{;<0tH_D`= zo+hP{BCHuS_;>4N?H<;Pyo%n$Xvm#<3wmz5+5TQYsW-Qs+X;mnx%HbjZ?5Lj>C2b5 zxAXbVP6srtAW=!F6*pfobwH->6v<7Ai>|Cl_IRySgxcGzSu@xh(V|*eNsHJ%gLwvX znZ(1`@Q^E)_z^SRyjQTVu738}#)crke$wTkELkq3zL2Dl&04k+e$9T!*77~rQksCJ zja(??4T#Kvu%hcMZrUvRIrSxFmlhtkXSGovMREaKVpFaOCW{G)5&;8b2?n}ca3FBF znjFATnQAS$J@DG{`9eb=LeC!Wf$eL`nor|&ThTPZ6Eu!Fk&uDX3;dQ`nYTNuUT-rT zscmAjXyr=zslcYl*}&a;d?EO15AJtm%B0qauP%T|{a*7RQ}FAs0~J<+?H-|dPSj=8 zfzGL6<1mdZ>Ee1oi}LasC55WC4?l%Sv1$e5lnSl`BRmhTE7INO_jh~6$`;5~x=I-| z03Jo@B9QLBkw$BlTJ~JE$oyYpjy^1#a}*vz=@hSe21nAVY_V!;fvVl(bhd;E1DU?A z-Y$P{kFUdCpp-!y1p#@m07}UhaHb|w*|xE<#@V#-^|R?q$~&#wO_fqbd&Byc>?<%L z^`$rs$&a1KNrLt>$kR#cWUM1dJ!2ig`qk@b_DNJUp3^2nxy=v*nX8vsZ+(PhseYBM z#bhWOukWG?vbBi>Ls*xgmGe>2V0R=Q_}b%xefxlj5OJOV_|k|Ykepj*XfX75dpGHpbi-Z8{s6%D(>T)(c-@?i@< z+(`ZV>n+%`_~{YYb-^%$F;dK7xZd`Jo3#U#F9n{?U=TrlTYT5{xq>iZ??+e7WChw|Hz2~V}FwHBO-lA0H+! zMfvmr!iO+GSWZE>b_r6;`1pv|B2zxqMcLvV8CUOuB#MDYM%$Rfg60k%bcL?`XT2x} zVMu;tL?E{v^jeaOt>I@z@-gU%?`>QBy0z_n|Fj1sJfg;R|1Twlde$|{dzAEawqBaR ze7%OOWaRyDbhPW(_jfieJpc)UBFa?AIn0yoW=F07fZO)mj4d!enp*+$oY175|KrXv IH;+H=zZFcdV*mgE literal 0 HcmV?d00001 diff --git a/www/index.html b/www/index.html new file mode 100644 index 00000000..0d5fb346 --- /dev/null +++ b/www/index.html @@ -0,0 +1,528 @@ + + + + + +Wardline — trust-boundary analysis + + + + + + + + + + + + +
+
+ + + + + + + + + + + Wardline + ~/wardline +
+ +
+ +
+ + +
+
+ + Trust-boundary analysis · Python · zero runtime dependencies +
+ +

Does this function's data
match its trust claim?

+ +

+ Wardline reads your Python source statically — it never runs your code — and asks one question + of every trust-annotated function: is the data it works with as trusted as it claims? + It propagates a taint (a trust level) across the whole call graph and flags untrusted data + reaching a trusted producer with no validation in between. +

+ +
+
+ $ + pip install "wardline[scanner]" + +
+ Get started +
+ + +
+
+ + + + demo.py +
+ 
1from weft_markers import trusted, external_boundary
+2
+3@external_boundary
+4def read_request(req):
+5    return req.body            # raw, untrusted (EXTERNAL_RAW)
+6
+7@trusted(level="ASSURED")
+8def build_record(req):
+9    return read_request(req)   # claims ASSURED, returns raw — no validation
+ + +
+ EXTERNAL_RAW + + @trusted ASSURED + + ✗ no validation +
+ + +
+ +
+ demo.build_record declares return trust ASSURED + but actually returns EXTERNAL_RAW (less trusted) — + untrusted data reaches a trusted producer. +   demo.py:8 · PY-WL-101 +
+
+
+ + +
+
+ Runtime deps + 0 +
+
+ Trust decorators + 3 +
+
+ Lattice states + 8 +
+
+ Output formats + SARIF + JSONL +
+
+ + +
+
Gate CI with a single command
+ $ wardline scan . --fail-on ERROR +
+ exit 0 — clean + exit 1 — gate tripped + exit 2 — wardline error +
+
+ +
+ + +
+
The trust model
+

Three decorators. Eight states.

+

+ Declare trust at the source, not in a config file. Wardline reads those declarations and + propagates them across the whole call graph. Everything else is inferred. +

+ + +
+
+ @external_boundary +

+ Marks a function as the entry point for untrusted external data — network requests, file + reads, environment variables. Its return value is tainted EXTERNAL_RAW. +

+
+
+ @trust_boundary +

+ Marks a function that validates or transforms data before passing it inward. The function + is expected to reject or sanitize; Wardline checks that it can. +

+
+
+ @trusted(level="…") +

+ Declares the return trust level a function claims to produce. Wardline verifies the actual + propagated trust level meets that claim — violations become findings. +

+
+
+ + +
+
+ Most trusted + 8-state trust lattice + Least trusted +
+
+ +
+ + INTEGRAL +
+
+ + ASSURED +
+
+ + GUARDED +
+
+ + UNKNOWN_ASSURED +
+
+ + UNKNOWN_GUARDED +
+
+ + EXTERNAL_RAW +
+
+ + UNKNOWN_RAW +
+
+ + MIXED_RAW +
+
+
+ + +
+

+ Opt-in by design. Undecorated code sits in the developer-freedom zone — + unknown-trust, no findings. You declare trust only on the functions that matter, + which is what lets Wardline scan a large untouched codebase (including its own) with zero + noise. Start by annotating the edges where data crosses trust levels; the engine infers the rest. +

+
+ +
+ + +
+
The command surface
+

CLI + MCP + agent integration

+

+ Wardline is agent-first: the MCP server exposes scan, explain, judge, baseline, and waiver + tools over JSON-RPC with no SDK. Agents run the full scan → explain → fix-at-boundary → rescan loop natively. +

+ +
+ +
+
CLI
+ + + + + + + + + + + + + + + + + + + + + + + + + +
wardline scanscan for trust-boundary violations (SARIF / JSONL / human output)
wardline assuregate: assert the trust posture still holds (pass/fail)
wardline attestproduce a signed assurance bundle
wardline dossierper-entity trust dossier
wardline judgeopt-in LLM triage — labels findings TRUE/FALSE positive (never runs automatically)
wardline installwire Wardline into your coding agent (CLAUDE.md, .mcp.json, Loomweave / Filigree bindings)
+

The CLI reference in the docs is authoritative. --fail-on level, output format flags, and suppression options are covered there.

+
+ + +
+
MCP (JSON-RPC · no SDK)
+ + + + + + + + + + + + + + + + + + + + + + + + + +
scanscan with a conjunctive where filter + inline explain
scan_file_findingsdry-run findings for one file
decorator_coveragetrust-decorator coverage posture report
dossierthe per-entity trust dossier over MCP
assuregate the trust posture over MCP
attestproduce a signed assurance bundle over MCP
+

Launch with wardline mcp. Run wardline install to wire it automatically. See agent integration docs.

+
+
+ + +
+
Installation layers
+ + + + + + + + + + + + + + + + + + + + + + + + + +
CommandWhat it provides
pip install weft-markersTiny runtime marker package — import @trusted, @external_boundary, @trust_boundary in application code without depending on the full scanner
pip install wardlineZero-dependency base — the library and decorators, no CLI
pip install "wardline[scanner]"The full scanner — wardline CLI, wardline mcp server (adds pyyaml, jsonschema, click)
pip install "wardline[loomweave]"Persist per-entity taint facts to a Loomweave store (adds blake3)
+
+ +
+ + +
+
The federation
+

Wardline in the Weft Federation

+

+ Wardline is the Weft Federation's trust-policy surface — it answers, statically + and deterministically, whether the data each trust-annotated function works with is as trusted + as it claims. The siblings enrich this picture; they are never required. +

+ +
+

+ Enrich-only, never load-bearing. Wardline scans and analyzes with all siblings absent. + The Filigree emitter is fail-soft (core/filigree_emit.py). Remove any sibling — the scanner keeps working. +

+
+ +
+
+
The connective tissue
+
SEI — Stable Entity Identity
+
+ Wardline keys taint facts and cross-tool bindings on SEI, the durable per-entity identity + owned by Loomweave. A link survives a rename because it keys on the identity, not the name. + Degrades gracefully when the sei capability is absent from the Loomweave instance. + LOCKED · 2026-06-05 +
+
+
+
The authority
+
Wardline owns trust policy
+
+ The trust lattice and its states, the decorator vocabulary, the rule IDs (PY-WL-1xx), the scanner + engine, and the corpus of ground-truth specimens are all Wardline's authority. + The federation hub does not restate rule counts or decorator names — the repo is the source of truth. +
+
+
+ +
Bindings
+
    +
  • + Wardline + → + Loomweave + taint facts enrich the entity graph, keyed on SEI — survives any rename +
    • +
  • + Wardline + → + Filigree + findings become tracked work via the native Filigree emitter + A-1 · LIVE — until Loomweave-absent path demonstrated end-to-end +
    • +
  • + Wardline + → + Legis + findings are governed by Legis (git/CI governance & attestations) +
    • +
+ +

+ Full federation context and the asterisk register: + federation-map.md · + asterisk-register.md · + members/wardline.md +

+ + +
The Weft Federation members
+ + +
+ +
+ + + + + + + diff --git a/www/main.js b/www/main.js new file mode 100644 index 00000000..cfeeff27 --- /dev/null +++ b/www/main.js @@ -0,0 +1,87 @@ +/* ============================================================================ + WARDLINE — front-door site interactions (progressive enhancement only) + The page is content-complete without JS. This script only layers in: + · copy-to-clipboard on the install command strip + · hover-reveal anchor links on section headings (keyboard accessible) + ============================================================================ */ +(function () { + "use strict"; + + /* ---- Copy to clipboard for the install command strip ---------------- */ + var copyBtn = document.getElementById("install-copy-btn"); + var installText = document.getElementById("install-text"); + + if (copyBtn && installText && navigator.clipboard) { + copyBtn.addEventListener("click", function () { + var text = installText.textContent.trim(); + navigator.clipboard.writeText(text).then(function () { + var original = copyBtn.textContent; + copyBtn.textContent = "copied!"; + copyBtn.setAttribute("aria-label", "Copied!"); + setTimeout(function () { + copyBtn.textContent = original; + copyBtn.setAttribute("aria-label", "Copy install command"); + }, 1600); + }).catch(function () { + /* clipboard unavailable — button does nothing, page still works */ + }); + }); + } + + /* ---- Hover-reveal anchor links on section headings ------------------ * + * For each h2/h3 with an id in the page sections, inject a lightweight * + * "§" anchor link that appears on hover and is keyboard-reachable. */ + var headings = Array.prototype.slice.call( + document.querySelectorAll("main h2[id], main h3[id]") + ); + headings.forEach(function (h) { + var id = h.id; + if (!id) return; + var a = document.createElement("a"); + a.href = "#" + id; + a.textContent = " §"; + a.setAttribute("aria-label", "Link to section: " + (h.textContent.replace(/\s§$/, "").trim())); + a.style.cssText = [ + "font-size: 0.72em", + "font-weight: 400", + "color: var(--text-muted)", + "text-decoration: none", + "opacity: 0", + "transition: opacity 0.15s", + "vertical-align: middle", + "margin-left: 0.35em" + ].join(";"); + h.appendChild(a); + h.style.position = "relative"; + + /* show on parent heading hover */ + h.addEventListener("mouseenter", function () { a.style.opacity = "1"; }); + h.addEventListener("mouseleave", function () { + if (document.activeElement !== a) a.style.opacity = "0"; + }); + /* always visible when focused */ + a.addEventListener("focus", function () { a.style.opacity = "1"; }); + a.addEventListener("blur", function () { a.style.opacity = "0"; }); + }); + + /* ---- Sibling member row hover effect -------------------------------- */ + var memberRows = Array.prototype.slice.call( + document.querySelectorAll(".bindings-member") + ); + memberRows.forEach(function (row) { + row.addEventListener("mouseenter", function () { + row.style.background = "var(--surface-overlay)"; + row.style.textDecoration = "none"; + }); + row.addEventListener("mouseleave", function () { + row.style.background = "var(--surface-raised)"; + }); + row.addEventListener("focus", function () { + row.style.outline = "2px solid var(--accent)"; + row.style.outlineOffset = "2px"; + }); + row.addEventListener("blur", function () { + row.style.outline = ""; + }); + }); +})(); diff --git a/www/styles.css b/www/styles.css new file mode 100644 index 00000000..8959001d --- /dev/null +++ b/www/styles.css @@ -0,0 +1,428 @@ +/* ============================================================================ + WARDLINE — front-door site layout + ---------------------------------------------------------------------------- + Layered on colors_and_type.css (the token + type source of truth, copied + verbatim from the Weft design system). Coral (--thread-wardline) is + Wardline's identity color — used for left-rules, glyph, eyebrow accents. + Amber (--accent) is reserved for interactive affordances (links, focus + rings, hover fills), exactly as the Weft hub does. + ============================================================================ */ + +*, *::before, *::after { box-sizing: border-box; } + +html, body { margin: 0; } +body { + background: var(--surface-base); + color: var(--text-primary); + font-family: var(--font-mono); + -webkit-font-smoothing: antialiased; + font-feature-settings: "liga" 1, "calt" 1; +} + +::-webkit-scrollbar { width: 10px; } +::-webkit-scrollbar-track { background: var(--surface-base); } +::-webkit-scrollbar-thumb { background: var(--border-strong); border-radius: 5px; } + +/* ---- skip link ------------------------------------------------------------ */ +.skip-link { + position: absolute; left: -9999px; top: var(--space-2); + padding: 6px 14px; background: var(--accent); color: var(--text-on-accent); + border-radius: var(--radius); font-size: 12px; font-weight: 600; z-index: 100; +} +.skip-link:focus { left: var(--space-4); } + +.mark { display: block; flex: 0 0 auto; } + +/* ---- links --------------------------------------------------------------- */ +a { color: var(--accent); text-decoration: none; } +a:hover { text-decoration: underline; } +a:focus-visible { outline: 2px solid var(--accent); outline-offset: 2px; border-radius: 2px; } +/* external-link affordance */ +.ext::after { content: " \2197"; font-size: 0.85em; color: var(--text-muted); } + +/* ---- shared section width ------------------------------------------------ */ +.hero, .trust-model, .commands, .federation, .hub-footer { + max-width: 980px; margin: 0 auto; +} + +/* ============================ HEADER ============================ */ +.hub-header { + display: flex; align-items: center; gap: 22px; + padding: 12px 30px; + border-bottom: 1px solid var(--border-default); + background: var(--surface-raised); + position: sticky; top: 0; z-index: 10; +} +.brand { display: flex; align-items: center; gap: 11px; min-width: 0; } +.brand .mark { color: var(--thread-wardline); } /* coral glyph */ +.wordmark { + font-family: var(--font-display); font-size: 19px; font-weight: 700; + letter-spacing: -0.02em; color: var(--text-primary); white-space: nowrap; +} +.path-hint { font-size: 11px; color: var(--text-muted); margin-left: 2px; } + +.hub-nav { display: flex; gap: 4px; margin-left: auto; flex-wrap: wrap; } +.hub-nav a { + font-size: 12px; color: var(--text-secondary); text-decoration: none; + padding: 6px 11px; border-radius: var(--radius); white-space: nowrap; + transition: background var(--dur-fast) var(--ease), color var(--dur-fast) var(--ease); +} +.hub-nav a:hover, .hub-nav a:focus-visible { + background: var(--surface-overlay); color: var(--text-primary); text-decoration: none; +} +.hub-nav a.ext::after { content: " \2197"; font-size: 0.8em; opacity: 0.6; } + +/* ============================ HERO ============================ */ +.hero { padding: 64px 30px 40px; } +.eyebrow { display: flex; align-items: center; gap: 9px; margin-bottom: 22px; } +.eyebrow-dot { + width: 7px; height: 7px; border-radius: 50%; + background: var(--thread-wardline); /* coral identity color */ +} + +.hero-title { + font-family: var(--font-display); font-weight: 700; + font-size: clamp(36px, 6vw, 52px); letter-spacing: -0.03em; line-height: 1.02; + margin: 0; color: var(--text-primary); +} +.hero-lede { font-size: 16px; max-width: 680px; margin-top: 22px; line-height: 1.6; color: var(--text-secondary); } +.hero-lede strong { color: var(--text-primary); font-weight: 600; } + +/* install strip */ +.install-strip { + display: flex; align-items: center; gap: 12px; + margin-top: 26px; flex-wrap: wrap; +} +.install-cmd { + font-family: var(--font-mono); font-size: 14px; font-weight: 500; + padding: 10px 16px; + background: var(--surface-overlay); + border: 1px solid var(--border-strong); + border-radius: var(--radius); + color: var(--text-primary); + display: flex; align-items: center; gap: 10px; +} +.install-cmd .prompt { color: var(--text-muted); user-select: none; } +.install-copy { + font-family: var(--font-mono); font-size: 11px; font-weight: 600; + padding: 4px 10px; border: 1px solid var(--border-strong); + border-radius: var(--radius-sm); background: transparent; + color: var(--text-secondary); cursor: pointer; + transition: background var(--dur-fast) var(--ease), color var(--dur-fast) var(--ease); +} +.install-copy:hover { background: var(--surface-hover); color: var(--text-primary); } +.install-copy:focus-visible { outline: 2px solid var(--accent); outline-offset: 2px; } +.install-link { + font-size: 13px; color: var(--accent); font-weight: 600; display: inline-flex; align-items: center; gap: 4px; +} + +/* ---- Hero finding panel (the PY-WL-101 motif) --- */ +.finding-panel { + margin-top: 36px; + background: #131E24; /* editor-dark surface, pinned dark in both themes */ + border: 1px solid #223040; + border-left: 3px solid var(--thread-wardline); /* coral left-rule */ + border-radius: var(--radius-lg); + overflow: hidden; + font-size: 13px; + box-shadow: 0 8px 30px rgba(0, 0, 0, 0.40); +} +.finding-panel__bar { + display: flex; align-items: center; gap: 8px; + padding: 10px 16px; + background: #0F1A20; + border-bottom: 1px solid #223040; + color: #8FAAB8; font-size: 11px; +} +.finding-panel__dot { + width: 9px; height: 9px; border-radius: 50%; + background: currentColor; opacity: 0.4; +} +.finding-panel__filename { margin-left: auto; letter-spacing: 0.02em; } + +/* Code block inside the finding panel */ +.fp-code { + margin: 0; padding: 16px; + font-family: var(--font-mono); font-size: 13px; line-height: 1.65; + color: #C8D8E0; overflow-x: auto; tab-size: 4; + background: #131E24; +} +.fp-ln { + display: inline-block; width: 1.5rem; text-align: right; + color: #3A5A6E; user-select: none; margin-right: 14px; +} +/* Syntax tokens re-colored onto the warm-Loom palette */ +.fp-kw { color: #56B7E2; } /* sky — keyword */ +.fp-dec { color: #E9B04A; } /* amber — decorator (Wardline's accent) */ +.fp-fn { color: #52C9B8; } /* aqua — function name */ +.fp-str { color: #5FB98E; } /* warm-emerald — string */ +.fp-cmt { color: #4A6070; font-style: italic; } /* comment — muted */ +.fp-raw { color: #E2604E; font-weight: 600; } /* the trust leak — stale red */ +.fp-arg { color: #B79BF2; } /* violet — argument */ + +/* trust-flow strip */ +.fp-flow { + display: flex; align-items: center; gap: 8px; + padding: 10px 16px; + border-top: 1px solid #223040; + font-size: 11px; flex-wrap: wrap; +} +.fp-flow__node { + padding: 3px 10px; border-radius: 999px; + border: 1px solid currentColor; white-space: nowrap; font-weight: 600; +} +.fp-flow__node--raw { color: #E2604E; } /* untrusted — stale red */ +.fp-flow__node--claim { color: #5FB98E; } /* claimed trusted — warm emerald */ +.fp-flow__node--bad { color: #E2604E; } /* bad outcome */ +.fp-flow__arrow { color: #4A6070; flex: none; font-size: 14px; } + +/* verdict strip */ +.fp-verdict { + display: flex; align-items: flex-start; gap: 10px; + padding: 12px 16px; + border-top: 1px solid #223040; + background: rgba(226, 96, 78, 0.10); /* stale-red wash on the dark panel */ +} +.fp-verdict__mark { flex: none; font-weight: 700; color: #E2604E; font-size: 14px; line-height: 1.5; } +.fp-verdict__body { + font-family: var(--font-mono); font-size: 12px; line-height: 1.55; + color: #C8D8E0; +} +.fp-verdict__rule { font-weight: 700; color: #E2604E; } +.fp-verdict__body .vt-bad { color: #E2604E; font-weight: 600; } +.fp-verdict__body .vt-good { color: #5FB98E; font-weight: 600; } + +/* ---- Hero metric strip ---- */ +.hero-stats { + display: flex; gap: 44px; flex-wrap: wrap; + margin-top: 32px; padding-top: 26px; + border-top: 1px solid var(--border-default); +} +.stat { display: flex; flex-direction: column; gap: 5px; } +.stat-label { + display: flex; align-items: center; gap: 7px; + font-size: 11px; font-weight: 600; letter-spacing: 0.1em; + text-transform: uppercase; color: var(--text-muted); +} +.stat-dot { width: 7px; height: 7px; border-radius: 50%; flex: 0 0 auto; } +.stat-dot.t-wardline { background: var(--thread-wardline); } +.stat-dot.t-accent { background: var(--accent); } +.stat-dot.t-ready { background: var(--ready); } +.stat-dot.t-muted { background: var(--text-muted); } +.stat-value { + font-family: var(--font-display); font-weight: 700; + font-size: 34px; letter-spacing: -0.02em; line-height: 1; + color: var(--text-primary); +} + +/* ---- CI gate example ---- */ +.ci-gate { + margin-top: 30px; padding: 16px 20px; + background: var(--surface-raised); + border: 1px solid var(--border-default); + border-left: 3px solid var(--thread-wardline); + border-radius: 0 var(--radius) var(--radius) 0; +} +.ci-gate__label { margin-bottom: 8px; } +.ci-gate__cmd { + font-family: var(--font-mono); font-size: 13px; color: var(--text-primary); + padding: 8px 12px; background: var(--surface-overlay); + border-radius: var(--radius-sm); display: inline-block; +} +.ci-gate__exit { + display: flex; gap: 16px; margin-top: 10px; flex-wrap: wrap; +} +.ci-exit-item { font-size: 11.5px; color: var(--text-secondary); } +.ci-exit-item .exit-code { font-family: var(--font-mono); font-weight: 600; color: var(--text-primary); } +.ci-exit-item .exit-ok { color: var(--ready); } +.ci-exit-item .exit-trip { color: var(--stale); } +.ci-exit-item .exit-err { color: var(--aging); } + +/* ============================ TRUST MODEL ============================ */ +.trust-model { padding: 32px 30px 40px; } +.section-label { margin-bottom: 14px; } + +.comp-title { + font-family: var(--font-display); font-size: 26px; font-weight: 600; + letter-spacing: -0.015em; color: var(--text-primary); margin: 0 0 4px; +} +.comp-lede { max-width: 660px; margin-bottom: 20px; font-size: 15px; color: var(--text-secondary); line-height: 1.6; } + +/* Decorators */ +.decorators-grid { + display: grid; grid-template-columns: repeat(3, 1fr); gap: 12px; + margin-bottom: 28px; +} +.decorator-card { + background: var(--surface-raised); + border: 1px solid var(--border-default); + border-left: 3px solid var(--thread-wardline); + border-radius: var(--radius-lg); padding: 16px 18px; +} +.decorator-name { + display: block; font-family: var(--font-mono); font-size: 13px; font-weight: 600; + color: var(--thread-wardline); margin-bottom: 8px; +} +.decorator-body { font-size: 12.5px; color: var(--text-secondary); line-height: 1.55; } + +/* Lattice ramp */ +.lattice-block { + background: var(--surface-raised); border: 1px solid var(--border-default); + border-radius: var(--radius-lg); padding: 20px 22px; +} +.lattice-caption { + display: flex; justify-content: space-between; align-items: center; + font-size: 10.5px; font-weight: 600; letter-spacing: 0.08em; + text-transform: uppercase; color: var(--text-muted); margin-bottom: 12px; +} +.lattice-track { display: flex; flex-direction: column; gap: 2px; } +.tier-row { + display: flex; align-items: center; gap: 10px; + padding: 7px 10px; border-radius: var(--radius-sm); + background: var(--surface-overlay); border: 1px solid var(--border-default); + font-size: 12px; +} +.tier-swatch { + flex: none; width: 12px; height: 12px; + border-radius: 3px; border: 1px solid rgba(0,0,0,0.25); +} +.tier-name { font-weight: 600; color: var(--text-primary); } + +/* Opt-in zone callout */ +.optin-block { + margin-top: 22px; padding: 16px 20px; + background: var(--surface-raised); + border: 1px solid var(--border-default); + border-left: 3px solid var(--accent); + border-radius: 0 var(--radius) var(--radius) 0; +} +.optin-block__body { font-size: 14px; color: var(--text-secondary); line-height: 1.6; } +.optin-block__body strong { color: var(--text-primary); } + +/* ============================ COMMANDS ============================ */ +.commands { padding: 16px 30px 40px; } + +.cmd-grid { + display: grid; grid-template-columns: 1fr 1fr; gap: 12px; margin-top: 16px; +} +.cmd-card { + background: var(--surface-raised); border: 1px solid var(--border-default); + border-left: 3px solid var(--thread-wardline); border-radius: var(--radius-lg); + padding: 16px 18px; +} +.cmd-card-label { margin-bottom: 12px; } +.cmd-table { width: 100%; border-collapse: collapse; font-size: 12px; } +.cmd-table tr + tr td { border-top: 1px solid var(--border-default); } +.cmd-table td { padding: 7px 6px; vertical-align: top; } +.cmd-table td:first-child { + font-family: var(--font-mono); font-weight: 600; color: var(--text-primary); + white-space: nowrap; padding-right: 14px; +} +.cmd-table td:last-child { color: var(--text-secondary); } +.cmd-note { font-size: 11px; color: var(--text-muted); margin-top: 12px; line-height: 1.5; } + +.install-table-wrap { margin-top: 22px; } +.install-table { + width: 100%; border-collapse: collapse; font-size: 12px; + background: var(--surface-raised); border: 1px solid var(--border-default); + border-radius: var(--radius-lg); overflow: hidden; +} +.install-table th { + background: var(--surface-overlay); padding: 8px 14px; + text-align: left; font-size: 10px; font-weight: 700; + letter-spacing: 0.1em; text-transform: uppercase; color: var(--text-muted); + border-bottom: 1px solid var(--border-default); +} +.install-table td { padding: 8px 14px; color: var(--text-secondary); vertical-align: top; } +.install-table tr + tr td { border-top: 1px solid var(--border-default); } +.install-table td:first-child { font-family: var(--font-mono); font-weight: 600; color: var(--text-primary); white-space: nowrap; } + +/* ============================ FEDERATION ============================ */ +.federation { padding: 28px 30px 44px; } + +.fed-intro-block { + padding: 16px 20px; margin: 16px 0 28px; + background: var(--surface-raised); + border: 1px solid var(--border-default); + border-left: 3px solid var(--thread-wardline); + border-radius: 0 var(--radius) var(--radius) 0; +} +.fed-intro__body { font-size: 15px; color: var(--text-primary); line-height: 1.55; } +.fed-intro__body em { font-style: normal; color: var(--thread-wardline); font-weight: 600; } + +.facts { display: grid; grid-template-columns: 1fr 1fr; gap: 12px; margin: 0 0 28px; } +.fact { + background: var(--surface-raised); border: 1px solid var(--border-default); + border-left: 3px solid var(--accent); border-radius: var(--radius-lg); + padding: 18px 20px; +} +.fact-title { font-size: 15px; font-weight: 600; color: var(--text-primary); margin: 7px 0 8px; } +.fact-body { font-size: 13px; color: var(--text-secondary); line-height: 1.55; } + +.bindings { list-style: none; margin: 12px 0 0; padding: 0; display: flex; flex-direction: column; gap: 8px; } +.bindings li { + font-size: 13px; color: var(--text-secondary); line-height: 1.5; + padding: 11px 14px; background: var(--surface-raised); + border: 1px solid var(--border-default); border-radius: var(--radius); + display: flex; align-items: baseline; gap: 7px; flex-wrap: wrap; +} +.b-name { font-weight: 600; } +.b-arrow { color: var(--text-muted); } +.b-desc { color: var(--text-secondary); } + +/* ---- small inline status tags ---- */ +.tag { + display: inline-block; font-size: 10px; font-weight: 600; letter-spacing: 0.04em; + padding: 2px 7px; border-radius: var(--radius-sm); white-space: nowrap; margin-left: 2px; +} +.tag-ok { color: var(--ready); background: color-mix(in oklab, var(--ready) 16%, transparent); } +.tag-warn { color: var(--aging); background: color-mix(in oklab, var(--aging) 16%, transparent); } +.tag-dim { color: var(--text-muted); background: var(--surface-overlay); } +.tag-live { color: var(--thread-wardline); background: color-mix(in oklab, var(--thread-wardline) 14%, transparent); } + +.weave-foot { margin-top: 16px; } + +/* ============================ FOOTER ============================ */ +.hub-footer { + border-top: 1px solid var(--border-default); + padding: 20px 30px; display: flex; gap: 14px; align-items: center; flex-wrap: wrap; + max-width: 980px; margin: 0 auto; +} +.hub-footer .mark { color: var(--text-muted); } +.foot-note { font-size: 11px; color: var(--text-muted); } +.foot-links { display: flex; gap: 14px; flex-wrap: wrap; margin-left: auto; } +.foot-links a { font-size: 11px; color: var(--text-secondary); } +.foot-links a:hover { color: var(--text-primary); } +.foot-meta { font-size: 11px; color: var(--text-muted); } +.foot-org { + display: flex; align-items: center; gap: 8px; font-size: 11px; color: var(--text-muted); + text-decoration: none; transition: color 0.15s var(--ease); +} +.foot-org .mark { color: inherit; } +.foot-org:hover { color: var(--text-secondary); } +.foot-divider { + width: 100%; height: 0; border: none; + border-top: 1px solid var(--border-default); + margin: 0; +} + +/* ============================ RESPONSIVE ============================ */ +@media (max-width: 720px) { + .facts { grid-template-columns: 1fr; } + .decorators-grid { grid-template-columns: 1fr; } + .cmd-grid { grid-template-columns: 1fr; } +} +@media (max-width: 640px) { + .hub-header { gap: 12px; padding: 12px 18px; } + .path-hint { display: none; } + .hero { padding: 44px 18px 32px; } + .hero-stats { gap: 24px 32px; } + .stat-value { font-size: 28px; } + .trust-model, .commands, .federation { padding-left: 18px; padding-right: 18px; } + .hub-footer { padding: 18px; } + .foot-links { margin-left: 0; flex-basis: 100%; } +} + +@media (prefers-reduced-motion: reduce) { + *, *::before, *::after { transition: none !important; animation: none !important; } +} From 93fcdb7357518dcb2aac41f462e3d8158a76fc3f Mon Sep 17 00:00:00 2001 From: John Morrissey Date: Wed, 10 Jun 2026 09:27:38 +1000 Subject: [PATCH 078/186] docs(plan): Rust SP2 + Phase-1b full-producer sprint plan (5-lens panel-reviewed, findings folded) --- .../2026-06-10-rust-sp2-phase1b-producer.md | 336 ++++++++++++++++++ 1 file changed, 336 insertions(+) create mode 100644 docs/superpowers/plans/2026-06-10-rust-sp2-phase1b-producer.md diff --git a/docs/superpowers/plans/2026-06-10-rust-sp2-phase1b-producer.md b/docs/superpowers/plans/2026-06-10-rust-sp2-phase1b-producer.md new file mode 100644 index 00000000..2accb830 --- /dev/null +++ b/docs/superpowers/plans/2026-06-10-rust-sp2-phase1b-producer.md @@ -0,0 +1,336 @@ +# Rust Frontend → Full ADR-049 Producer + SP2 Whole-Tree — Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. +> +> **Panel-reviewed 2026-06-10** (reality/contract/architecture/quality/systems, all "execute-with-fixes") — every finding folded below. Where a step cites oracle source (file:line in /home/john/loomweave), the executor MUST read that source before implementing; the citations are the contract for behaviors the corpus does not pin. + +**Goal:** Take the Rust plugin from "preview / provisional identity, taint-only producer" to a full second ADR-049 producer with real crate-prefixed finding identity — RS-WL-* findings become baseline-eligible. + +**Architecture:** Wardline's tree-sitter Rust frontend (`src/wardline/rust/`) grows from callables-only to the full ten-kind ADR-049 entity surface (leaf kinds, the `impl` entity, `module → impl → method` containment) plus the two anchored edge kinds; an SP2 whole-tree pass (Cargo.toml crate roots, cross-file module routes) replaces the directory-name crate stub; identity then graduates (frozen `tests/golden/identity/rust/` corpus, `provisional_identity` plumbing removed). The Loomweave extractor + vendored corpus remain the oracle: where upstream behavior is decided-and-emitted but un-oracled (stacked-cfg fold, cfg reserved-char escape, leaf kinds), we add the corpus rows **upstream first** (verified by Loomweave's own cargo gate), re-vendor, then conform. Where upstream has no decision (reserved-colon path-typed generic args; const-arg spacing), we draft the ADR-049 decision letter and record the dependency — no unilateral normalization. + +**Tech Stack:** Python 3.12 (stdlib `tomllib` for manifests — zero new dep), tree-sitter / tree-sitter-rust (preview extra), pytest; Rust/cargo only to run Loomweave's own gates upstream. + +**Oracle ground truth (verified 2026-06-10, panel-corrected):** +- Loomweave repo: `/home/john/loomweave`, branch `rc4` @ `510a032` (`feat/rust-plugin-spec` is merged into rc4; the stale branch ref still exists locally — ignore it; the Rust plugin is live on-by-default). Working tree has ONE unrelated dirty file (`.agents/skills/loomweave-workflow/SKILL.md`) — never touch or stash it. +- Upstream corpus `fixtures/qualnames_rust.json` @ blob `a0aaa341041dc66...` (HEAD): 22 entity cases (all slice-1), 6 module_route cases (5 slice-1 + `path_attr_known_gap` sp2/known_gap). vs our vendored copy: ONE new case `generic_self_nested_param`; zero diffs in shared cases. +- The upstream corpus has **no `enum`/`trait`/`type_alias`/`const`/`static` rows** (`macro` IS pinned by `macro_invocation_generates_no_entity`) and **no stacked-cfg or cfg-escape rows** — but the extractor implements all of it (extract.rs:830-843 `cfg_predicates` collects ALL cfg attrs raw; qualname.rs:299-302 `cfg_discriminant` normalises each, sorts, joins `&`; qualname.rs:314-336 `normalise_pred` strips ws → `escape_reserved` (`%`→`%25` then `:`→`%3A`) → any()/all() 1-level arg sort; plugin.toml `entity_kinds` = the ten kinds, `ontology_version = "0.4.0"`). +- **Trait bodies are NOT walked** by the extractor (extract.rs:457 "Trait *bodies* are deliberately NOT walked here"); a trait definition emits only the `trait` entity. +- **cfg-twin suffixing is per-(kind, name) across ALL nine named item kinds** (extract.rs:326-358 `twin_counts`; struct arm :397-400, inline-mod arm :436-439) — suffix applied only on collision, per-kind counted. +- Crate roots (crate_roots.rs:47-95): a dir registers iff (a) `Cargo.toml` **parses as TOML** AND has `[package].name` as a string (toml::Value parse — `name.workspace = true` is a table → falls through), ELSE (b) `src/lib.rs` OR `src/main.rs` exists → directory-name fallback. Virtual workspace roots (no package.name, no src/lib|main.rs) register NOTHING. Names `-`→`_` normalised. Walk skips symlinked dirs (crate_roots.rs:83-94 + dedicated test). File→crate by longest path-prefix. +- Out-of-src files (scope.rs:21-32 `emittable_scope`): loomweave EXCLUDES tests/, benches/, examples/, build.rs, a `src/main.rs` shadowed by sibling lib.rs, and files under no crate root — it emits NOTHING for them. Wardline must NOT mirror that for scan coverage (see Task 4 design). +- Edges (resolve.rs:13-17, :40-49; extract.rs:634-665, :788-794): glob `use a::*` → Ambiguous(in-project module id) else dropped; multi-kind ambiguity to_id = FIRST id by sorted order; use-tree groups fan out, `as` aliases resolve the REAL path, `self` group leaf → the prefix module; trait lookup STRIPS generic args; negative impls emit no edge. +- Reserved-colon (`impl From`): upstream renders the colon then REJECTS at `entity_id` construction (`validate_no_colon`, entity_id.rs:140) and degrades the file — **no canonical colon-free form exists**. Genuine ADR-049 amendment needed (Task 9). +- Wardline already byte-conforms on every vendored slice-1 row (suite 3046 passed, 1 xfailed = the `path_attr_known_gap` sp2 module_route row; both xfail code branches exist at test_loomweave_rust_qualname_parity.py:144-145 + :154-155 but only the module_route one fires today). +- **Wardline's `cfg_predicate_of` (qualname.py:99-114) returns the NORMALIZED predicate and index.py stores it** — the stacked-cfg fix must restructure to collect RAW and normalize exactly once (Task 2). +- **analyzer.py has NO kind filter** (analyzer.py:138-153 iterates ALL entities, calls `taint_for(entity.node)` + `child_by_field_name("body")` unconditionally) — Task 3 MUST add the callable guard. +- weft.toml severity overrides still do NOT apply to Rust (analyzer.py:84-85) — the banner's severity-override warning stays after graduation. +- Loomweave CI gate is `cargo nextest run --workspace --all-features --no-tests=pass` (.github/workflows/verify.yml:91) plus lockstep scripts — Task 1 must run the workspace gate, not just the plugin suite. +- No stored state holds RS-WL fingerprints (no `.weft/` in the worktree; no RS-WL entries in any baseline.yaml) — the Task 4 rekey orphans nothing; keep the cheap defensive grep. + +**Branch/commit discipline:** all Wardline work on `feat/rust-gold` (worktree `/home/john/wardline/.worktrees/rust-gold`), scoped commits per task, merged to `rc5` at the end. Loomweave-side work is ONE scoped commit on their `rc4` touching only the fixture (+ its header text). The rc5 main checkout has unrelated uncommitted changes (ci.yml, mkdocs.yml, docs/index.md, .gitignore, docs/arch-analysis-2026-06-10/) — never touch them; they are disjoint from this sprint's paths (the sprint touches docs/guides/, docs/reference/, docs/integration/, docs/superpowers/, CHANGELOG.md — NOT docs/index.md or mkdocs.yml). + +**Filigree:** sprint label `rust-sp2-2026-06-10` on every issue touched (already applied). Umbrella task `wardline-9f00d5b44b` (claimed). Claim with `--actor rust-gold-sprint` before starting the relevant task; close on completion. Issues: `wardline-be5ee9cc34` (reserved-colon, Task 9), `wardline-4fdad782a7` (stacked-cfg, Tasks 1–2), `wardline-e8f7c0508f` (cfg escape + const spacing, Tasks 1–2 + 9), `wardline-868908944b` (drift alarm, Task 8). + +--- + +## Dependency graph + +``` +Task 0 (spec amendment) ──────────────────────────┐ +Task 1 (loomweave oracle rows) → Task 2 (re-vendor + cfg conformance) + → Task 3 (producer surface: leaf kinds + impl entity + containment + per-kind twins) + → Task 4 (SP2 whole-tree: crate roots + analyzer wiring + un-xfail) + → Task 5 (edges: imports/implements) + → Task 6 (identity freeze: tests/golden/identity/rust/) + → Task 7 (graduation: provisional plumbing removal + docs) +Task 1 → Task 8 (drift alarm) +Task 9 (reserved-colon + const-spacing decision letter) — independent +Task 10 (hard-gates sweep + merge to rc5 + filigree closes) — last +``` + +--- + +### Task 0: Spec amendment — fold Phase 1b into SP2 + +**Files:** +- Modify: `docs/superpowers/specs/2026-06-08-wardline-rust-frontend-design.md` (§6.3, §6.4, §11) + +- [ ] **Step 0.1:** Amend §11's SP2 bullet to record the user-approved fold: SP2 now also comprises the Phase-1b producer surface (six leaf kinds, the `impl` entity + method re-parenting, the `imports`/`implements` anchored edges, `ontology_version 0.4.0`, `plugin_id rust`), citing `docs/integration/2026-06-09-loomweave-rust-qualname-phase1b-changeset.md`. Amend §6.3/§6.4 where they say "Wardline never emits struct/module rows" — after this sprint Wardline emits the full ten-kind surface and the conformance comparison graduates from the subset-consumer rule to the full-set rule (§7 rule 1 of the changeset). Note the oracle ground-truth corrections: `feat/rust-plugin-spec` is merged into loomweave `rc4`; the corpus gains `generic_self_nested_param`; leaf-kind/stacked-cfg/cfg-escape rows are added upstream by this sprint (Task 1). Add a §13 changelog entry dated 2026-06-10. +- [ ] **Step 0.2:** `.venv/bin/mkdocs build --strict` passes. Commit: `docs(spec): fold Phase 1b producer surface into SP2 (user-approved) + oracle ground-truth corrections`. + +--- + +### Task 1: Loomweave-side oracle rows (upstream, `/home/john/loomweave`, branch rc4) + +Pin four already-emitted-but-un-oracled behaviors with corpus rows, verified by Loomweave's own byte-for-byte gate. **Hand-author the `source`; the extractor dictates `expected`** — pre-derive expected by READING extract.rs/qualname.rs (not guessing), run the gate, and on mismatch adopt the extractor's actual output. Never weaken their test. + +**Files:** +- Modify: `/home/john/loomweave/fixtures/qualnames_rust.json` (append 4 entity cases + fix the stale `_dialect_summary.free_items` header text "enum/trait/const/static/type_alias/macro are Phase 1b, not yet emitted" — it already contradicts the live extractor and would contradict the new rows in the same file) +- Read first: `/home/john/loomweave/crates/loomweave-plugin-rust/tests/qualname_conformance.rs` (case schema + how the gate enumerates JSON cases), `src/extract.rs` (esp. :326-358 twin_counts, :457 trait-body comment, :476-492 Item::Trait arm, :830-843 cfg_predicates), `src/qualname.rs` (:299-336 cfg_discriminant/normalise_pred/escape_reserved) + +- [ ] **Step 1.1:** Append four cases to `entities` (all `"reproducibility": "slice-1"`, crate `demo`, `rel_path` `src/m.rs`, `module_path` `demo.m`): + +1. **`leaf_item_kinds`** — the five missing free-item kinds (NO `macro_rules!` — already pinned by `macro_invocation_generates_no_entity`; NO trait-body method — extract.rs:457 never walks trait bodies): +```rust +pub enum Color { Red } +pub trait Greet {} +pub type Alias = u8; +pub const LIMIT: u32 = 10; +pub static NAME: &str = "x"; +``` +expected (module row first, then source order): `demo.m` (module), `demo.m.Color` (enum), `demo.m.Greet` (trait), `demo.m.Alias` (type_alias), `demo.m.LIMIT` (const), `demo.m.NAME` (static). + +2. **`stacked_cfg_twin`** — source: +```rust +struct Foo; +#[cfg(feature = "a")] +#[cfg(unix)] +impl Foo { pub fn go(&self) {} } +#[cfg(feature = "b")] +#[cfg(unix)] +impl Foo { pub fn go(&self) {} } +``` +expected: per `cfg_discriminant` (each pred normalised, set sorted, joined `&`): module + struct rows, then `demo.m.Foo.impl#<>@cfg(feature="a"&unix)` (impl) + `….go` (function), `demo.m.Foo.impl#<>@cfg(feature="b"&unix)` (impl) + `….go` (function). **Adopt the extractor's exact bytes.** + +3. **`cfg_escape_reserved_char`** — source: +```rust +#[cfg(feature = "a:b")] +pub fn f() {} +#[cfg(feature = "c")] +pub fn f() {} +``` +expected: module row + `demo.m.f@cfg(feature="a%3Ab")` and `demo.m.f@cfg(feature="c")` (function rows; verify exact quoting/escape bytes from `normalise_pred`). + +4. **`leaf_kind_cfg_twin`** — pins the per-kind twin counter on a newly-oracled leaf kind (it becomes load-bearing under Wardline's full-set comparison): +```rust +#[cfg(unix)] +pub const LIMIT: u32 = 1; +#[cfg(windows)] +pub const LIMIT: u32 = 2; +``` +expected: module row + `demo.m.LIMIT@cfg(unix)` and `demo.m.LIMIT@cfg(windows)` (const rows). + +- [ ] **Step 1.2:** Run the gates: `cd /home/john/loomweave && cargo test -p loomweave-plugin-rust --test qualname_conformance`, then the full plugin suite `cargo test -p loomweave-plugin-rust`, then **the actual CI gate** `cargo nextest run --workspace --all-features --no-tests=pass` (fall back to `cargo test --workspace --all-features` if nextest is absent). On mismatch, correct `expected` to the extractor's emission. All green before committing. +- [ ] **Step 1.3:** Commit upstream (ONLY the fixture; never the dirty `.agents/...` file): `git add fixtures/qualnames_rust.json && git commit -m "test(plugin-rust): pin leaf kinds, stacked-cfg fold, cfg reserved-char escape, leaf-kind cfg twin with corpus rows (wardline second-producer conformance)"`. Record the new commit hash and `git rev-parse HEAD:fixtures/qualnames_rust.json` blob hash for Tasks 2/8. + +--- + +### Task 2: Re-vendor + Wardline cfg conformance (stacked-cfg fold, reserved-char escape, nested-param row) + +Claims `wardline-4fdad782a7` and the escape half of `wardline-e8f7c0508f` (`filigree start-work --actor rust-gold-sprint --advance`). + +**Files:** +- Modify: `tests/conformance/qualnames_rust.json` (verbatim copy of upstream blob) +- Modify: `tests/conformance/test_loomweave_rust_qualname_parity.py` (provenance header; `_KNOWN_KINDS` → the ten-kind set `{module, struct, function, enum, trait, type_alias, const, static, macro, impl}`) +- Modify: `src/wardline/rust/qualname.py` (`_escape_reserved`; `cfg_discriminant`; **`cfg_predicate_of` gains a raw mode** — see the layering note) +- Modify: `src/wardline/rust/index.py` (`_walk_scope` cfg accumulation: last-wins scalar → collect-all RAW list) +- Test: `tests/unit/rust/test_qualname.py`, `tests/unit/rust/test_index.py` + +**Layering (locked — mirrors the oracle, prevents the double-escape trap):** loomweave collects predicates RAW (extract.rs:830-843) and normalizes exactly once inside `cfg_discriminant` (qualname.rs:299-302). Wardline's `cfg_predicate_of` currently RETURNS `normalize_cfg_predicate(...)` (qualname.py:99-114) and index.py stores that. Restructure: `cfg_predicate_of` (or a new `raw_cfg_predicate_of`) returns the RAW predicate text; `pending_cfgs: list[str]` accumulates raw strings; `cfg_discriminant(predicates)` = `normalize_cfg_predicate(p)` for each (which now includes `_escape_reserved`), `sorted()`, `"&".join(...)`, wrapped `@cfg(...)`. Normalization+escape happens EXACTLY once. `_escape_reserved` runs after whitespace/paren strip, BEFORE the any()/all() split (qualname.rs:319-336 order). + +- [ ] **Step 2.1:** Copy the upstream fixture verbatim: `cp /home/john/loomweave/fixtures/qualnames_rust.json tests/conformance/qualnames_rust.json`. Update the provenance header (source commit + blob hash from Task 1.3). Extend `_KNOWN_KINDS` to the ten-kind set. Run `.venv/bin/pytest tests/conformance/test_loomweave_rust_qualname_parity.py -q` — expect FAILURES on `stacked_cfg_twin` + `cfg_escape_reserved_char` (today: collide on `@cfg(unix)` / no escape); `generic_self_nested_param` must pass immediately (nested-literal rendering already implemented); `leaf_item_kinds`/`leaf_kind_cfg_twin` rows are invisible to the still-function-only comparison (graduates in Task 3). +- [ ] **Step 2.2:** Failing unit tests first (TDD), expected bytes from Task 1's locked gate output: + +```python +# tests/unit/rust/test_qualname.py +def test_normalize_cfg_predicate_escapes_reserved_chars() -> None: + # % before : (order matters — injective, mirrors loomweave escape_reserved) + assert normalize_cfg_predicate('feature = "a:b"') == 'feature="a%3Ab"' + assert normalize_cfg_predicate('feature = "a%3Ab"') == 'feature="a%253Ab"' + +def test_escape_happens_before_any_all_split() -> None: + # escape applies to the whole stripped pred BEFORE arg sorting (oracle order) + assert normalize_cfg_predicate('any(feature = "a:b", unix)') == 'any(feature="a%3Ab",unix)' + +def test_cfg_discriminant_folds_all_predicates_sorted() -> None: + assert cfg_discriminant(['unix', 'feature = "a"']) == '@cfg(feature="a"&unix)' + assert cfg_discriminant(['feature = "a"', 'unix']) == '@cfg(feature="a"&unix)' # order-independent + +def test_cfg_discriminant_normalizes_exactly_once() -> None: + # raw input with a reserved char escapes ONCE (no double-escape through the pipeline) + assert cfg_discriminant(['feature = "a:b"']) == '@cfg(feature="a%3Ab")' +``` + +```python +# tests/unit/rust/test_index.py +def test_stacked_cfg_twins_get_distinct_folded_suffixes() -> None: + src = ( + 'struct Foo;\n' + '#[cfg(feature = "a")]\n#[cfg(unix)]\nimpl Foo { pub fn go(&self) {} }\n' + '#[cfg(feature = "b")]\n#[cfg(unix)]\nimpl Foo { pub fn go(&self) {} }\n' + ) + names = {e.qualname for e in discover_rust_entities(src, module="demo.m")} + assert 'demo.m.Foo.impl#<>@cfg(feature="a"&unix).go' in names + assert 'demo.m.Foo.impl#<>@cfg(feature="b"&unix).go' in names + +def test_single_stacked_cfg_impl_without_twin_gets_no_suffix() -> None: + # dialect: @cfg is a COLLISION discriminator — a lone stacked-cfg impl stays bare + src = '#[cfg(feature = "a")]\n#[cfg(unix)]\nstruct Foo;\nimpl Foo { pub fn go(&self) {} }\n' + names = {e.qualname for e in discover_rust_entities(src, module="demo.m")} + assert 'demo.m.Foo.impl#<>.go' in names +``` +Run; verify both new behaviors fail for the right reason. + +- [ ] **Step 2.3:** Implement per the locked layering. In index.py the per-item state becomes `pending_cfgs: list[str]` (reset on each non-attribute item; appended per `#[cfg]` attribute); the collision guard is the truthy-list check (`if pending_cfgs:` — NOT `len > 1`); suffix = `cfg_discriminant(pending_cfgs)` applied on collision exactly as today. Run the unit tests, then conformance — `stacked_cfg_twin` + `cfg_escape_reserved_char` rows green. +- [ ] **Step 2.4:** Full quick gate: `.venv/bin/pytest tests/unit/rust tests/conformance -q` green; `.venv/bin/ruff check . && .venv/bin/mypy src` clean. Commit: `fix(rust): fold ALL stacked #[cfg] predicates + mirror cfg reserved-char escape (oracle rows landed upstream) — closes wardline-4fdad782a7`. Filigree: close `wardline-4fdad782a7`; comment on `wardline-e8f7c0508f` (escape half done with oracle row; const-spacing → Task 9 letter). + +--- + +### Task 3: Phase 1b producer surface — leaf kinds, `impl` entity, containment, per-kind twins, full-set conformance + +**Files:** +- Modify: `src/wardline/rust/index.py` (emit all ten kinds; parent links; impl entity rows; **generalized per-kind twin counter**) +- Modify: `src/wardline/rust/qualname.py` (constants `RUST_PLUGIN_ID = "rust"`, `RUST_ONTOLOGY_VERSION = "0.4.0"`, and `entity_id(kind, qualname)` — qualname.py is the dialect home, NOT `__init__.py`) +- Modify: `src/wardline/rust/analyzer.py` (**add the callable filter — it does not exist today**) +- Modify: `tests/conformance/test_loomweave_rust_qualname_parity.py` (graduate subset-consumer → full-set comparison) +- Test: `tests/unit/rust/test_index.py`, `tests/unit/rust/test_qualname.py` + +**Design (locked):** +- `RustEntity.kind` extends over the full id-kind set (`module|struct|function|enum|trait|type_alias|const|static|macro|impl`; Wardline's semantic `method` kept, mapped to id-kind `function` at emission/comparison) + new field `parent: str | None` (qualname of the containing module or impl entity). All entities carry `node_id`/`location` (needed for edges/federation). +- `entity_id(kind, qualname)` returns `f"rust:{kind}:{qualname}"`, maps `method`→`function` ITSELF, and raises on kinds outside the ten-kind set (mirrors loomweave's build_entity_id validation posture). +- **Emit ordering (matches the corpus — verified against `free_fns_and_struct`, `nested_inline_mod`, `same_type_name_distinct_module_scopes`):** the FILE-SCOPE module entity is emitted FIRST (before any items); inline `mod` entities are emitted AT their source position (before recursing into them); the merged `impl` entity is emitted once at the first contributing block in source order; everything else at its source position. +- **Twin counter generalized per-(kind, name)** over all nine named item kinds (mirror extract.rs:326-358) — `fn S` and `struct S` never interfere; the `@cfg` suffix applies per-kind on collision (struct_cfg_twin + new leaf_kind_cfg_twin rows are the trip-wires). +- **The taint path gets an explicit guard:** analyzer.py:138-153 currently iterates ALL entities calling `taint_for(entity.node)` and `child_by_field_name("body")` unconditionally — add `if entity.kind not in ("function", "method"): continue` at the loop top, and count `functions_total` (the coverage metric) over callables only. + +- [ ] **Step 3.1:** Failing unit tests: + - leaf kinds emission (enum/trait/type_alias/const/static with qualname `.`, parent = module; `macro_rules! name` → `macro` entity; bare invocation → nothing — keep the existing guard test), + - impl entity rows (post-merge, `…Foo.impl#<>` / `…Foo.impl[Display]` incl. `@cfg` forms), + - containment (method.parent == impl entity qualname; impl.parent == module; free items' parent == module), + - `test_merged_impl_emitted_at_first_block_in_source_order` — two same-key inherent blocks: the ONE impl entity appears before both blocks' methods and its `location.line_start` is the FIRST block's line, + - `test_file_module_entity_emitted_first_and_inline_mods_at_source_position` — pin the ordering rules above, + - `test_per_kind_twin_counting` — `fn S` + `struct S` with cfg on one: no cross-kind suffix interference, + - `test_non_callable_entities_do_not_enter_taint_analysis` — a file with struct/enum/const/trait + one tainted `Command::new` function: `RustAnalyzer.analyze` returns exactly the function's findings; no crash, no extra findings, + - `test_emission_is_deterministic` — `discover_rust_entities` twice on the same source → byte-identical ordered lists, + - `test_entity_id_maps_method_and_validates_kind` — `entity_id("method", q) == f"rust:function:{q}"`; unknown kind raises. +- [ ] **Step 3.2:** Implement in `index.py` (lift the existing impl-key/merge-triple machinery into emitted `impl` entities; emit the file-scope module first; generalize the twin counter), `qualname.py` (constants + `entity_id`), `analyzer.py` (callable guard + callable-scoped coverage counter). Unit tests green. +- [ ] **Step 3.3:** Graduate the conformance comparison: rewrite `test_entity_qualnames` (and its helper `_expected_function_qualnames` → `_expected_all_pairs(case)`) to compare Wardline's FULL emission as an **ordered list of `(qualname, kind)`** (kind-mapped `method`→`function`) against the corpus `expected` list exactly — this matches loomweave's own ordered gate; the subset-consumer rule in `_consumer_comparison` forbids list-equality only for function-only consumers, which Wardline no longer is. The Step-3.1 ordering tests de-risk this; if a case still fails ONLY on order (qualname sets equal), STOP and check the corpus row's order against `discover_rust_entities` — fix the emitter, never the comparison. Run conformance: ALL rows green including `leaf_item_kinds` + `leaf_kind_cfg_twin`. +- [ ] **Step 3.4:** Full suite + lints (`.venv/bin/pytest -q`, ruff, mypy) — eyeball the conformance output for source-order alignment, not just pass/fail. Commit: `feat(rust): full ADR-049 producer surface — six leaf kinds, impl entity, module→impl→method containment, per-kind cfg twins, full-set conformance`. + +--- + +### Task 4: SP2 whole-tree — crate roots from Cargo.toml, cross-file routes, un-xfail + +**Files:** +- Create: `src/wardline/rust/crate_roots.py` +- Modify: `src/wardline/rust/analyzer.py` (`_module_for` → crate-root-aware) +- Modify: `tests/conformance/test_loomweave_rust_qualname_parity.py` (remove BOTH sp2 auto-xfail branches, :144-145 and :154-155) +- Test: `tests/unit/rust/test_crate_roots.py` (new) + +**Design (locked, panel-corrected — mirror `/home/john/loomweave/crates/loomweave-plugin-rust/src/crate_roots.rs` exactly; read it first):** +- **Manifest read:** parse `Cargo.toml` with stdlib `tomllib` (the oracle does a real `toml::Value` parse — "read as text" in ADR-049 means "not cargo-metadata", NOT a hand-rolled scan). Take `package.name` only if it parses AND is a string (`name.workspace = true` is a table → falls through to fallback). Unparseable TOML → fallback path. +- **Registration rule (two branches):** a dir is a crate root iff (a) its Cargo.toml yields a string `[package].name` → that name `-`→`_` normalised; ELSE (b) `src/lib.rs` or `src/main.rs` exists → directory-name normalised. A virtual workspace root (no package.name, no src/lib|main.rs) registers NOTHING — member crates own their files outright. +- **Walk:** skip symlinked directories (crate_roots.rs:83-94; use `os.scandir` + `entry.is_symlink()`, never a follow-links walk). File→crate by longest path-prefix match. +- **Scan coverage is NOT narrowed (the panel's must-fix):** loomweave's `emittable_scope` (scope.rs:21-32) EXCLUDES tests/, benches/, examples/, build.rs, shadowed src/main.rs, and no-crate-root files — that is its *federation entity surface*, not a scan filter. Wardline keeps scanning ALL discovered `.rs` files. Routing: files under a crate root's `src/` get the oracle route (`rust_module_route(crate, src_root=/src, file)`); all other files (tests/, build.rs, no-Cargo trees — including today's entire preview population) get a documented wardline-local fallback route (current behavior: crate = owning crate name if any else directory name, mechanical path route) with a module docstring stating those qualnames carry no cross-tool conformance claim (loomweave emits nothing there, so no collision is possible). `#[path]` stays un-honoured (shared known gap — `path_attr_known_gap` pins the mechanical form). + +- [ ] **Step 4.1:** Failing tests (`tmp_path` fixtures): + - single crate: `Cargo.toml` `name = "my-app"` → crate `my_app`; `src/lib.rs` → `my_app`; `src/a/b.rs` → `my_app.a.b`; `src/a/mod.rs` → `my_app.a`, + - virtual workspace root (`[workspace]`-only Cargo.toml, no src/) with two members → ONLY the members register; `members' src files route to their own crates`, + - NESTED crates: `outer/Cargo.toml` (`outer`) + `outer/inner/Cargo.toml` (`inner`) → `outer/src/lib.rs` → `outer`, `outer/inner/src/main.rs` → `inner` (longest-prefix), + - `name.workspace = true` manifest WITH `src/lib.rs` → dir-name fallback; package-less manifest WITHOUT src/lib|main.rs → not a root, + - symlinked external crate dir under the scan root → NOT registered (mirror loomweave's `does_not_register_crate_roots_reached_through_symlinked_dirs`), + - **coverage preservation:** `build.rs`, `tests/integration.rs`, and a bare no-Cargo `.rs` tree still produce RS-WL findings via the fallback route (pin that scan population is unchanged). +- [ ] **Step 4.2:** Implement `crate_roots.py`; wire `analyzer.py::_module_for` through it (keeping per-file isolation/error posture). Existing analyzer tests that assumed `crate=root.name` update only where a fixture grows a `Cargo.toml`. +- [ ] **Step 4.3:** Remove BOTH `pytest.xfail("sp2 …")` branches; sp2 rows assert for real (`path_attr_known_gap` passes mechanically). Run conformance: **0 xfail**. Full suite: xfail count 1→0. Defensive check: `grep -rn "RS-WL" .weft/ 2>/dev/null || echo clean` → clean (no stored fingerprints to orphan). +- [ ] **Step 4.4:** Commit: `feat(rust): SP2 whole-tree — Cargo.toml crate roots (tomllib, two-branch registration, symlink-safe), real crate-prefixed routes, sp2 conformance rows un-xfailed`. + +--- + +### Task 5: Anchored edges — `imports` + `implements` + +**Files:** +- Create: `src/wardline/rust/edges.py` +- Test: `tests/unit/rust/test_edges.py` (new) + +**Design (locked, per changeset §6 + the oracle source for what §6 leaves open — the corpus is entity-only; pin these citations in test docstrings):** `RustEdge` frozen dataclass `{kind: Literal["imports","implements"], from_id: str, to_id: str, source_byte_start: int, source_byte_end: int, confidence: Literal["resolved","ambiguous"]}` (never `inferred`). `discover_rust_edges(...)` resolves against the whole-tree entity index (Tasks 3+4). Semantics: +- `imports`: one per file-scope `use` leaf; `from_id` = enclosing module entity; use-tree GROUPS fan out (`use a::{B, C}` → two edges); `use a::B as C` resolves the REAL path `a::B` (alias dropped); a `self` group leaf (`use a::{self, B}`) → the prefix module `a` (extract.rs:634-665). Glob `use a::*`: in-project module → `ambiguous` edge to that module entity; else dropped (resolve.rs:40-49). Unique in-project target → `resolved`; multi-kind candidate set → `ambiguous` with `to_id` = FIRST id by sorted order — deterministic, never null (resolve.rs:13-17); external/unresolvable → DROPPED. Span = the `use` statement. +- `implements`: one per trait impl whose trait resolves in-project; `from_id` = the trait-impl entity id; trait lookup STRIPS generic args (`impl MyTrait for Foo` resolves `MyTrait` — extract.rs:788-794); negative impls (`impl !Send for X`) emit NO edge; merged twin blocks emit exactly ONE edge per impl entity. Span = the implemented-trait path node only. +- Path resolution: `crate::`/`self::`/`super::` prefixes + plain relative paths against the module routes; when in doubt, drop (D1). ids via `entity_id()`. + +- [ ] **Step 5.1:** Failing tests (multi-file `tmp_path` fixtures): the base two-file resolved case (`use crate::Greet` + `impl Greet for Foo` → one `imports` + one `implements`, both `resolved`, spans pinned); `super::` and nested `super::super::` resolution; use-tree group fan-out + `as`-alias real-path + `self` group leaf; glob in-project → `ambiguous(module)`, glob external → dropped; multi-kind ambiguity → `ambiguous` + first-by-sorted-order to_id; external `use std::fmt` → no edge; generic in-project trait `impl MyTrait for Foo` → resolved to `MyTrait`; negative impl → no edge; two merged same-key trait-impl blocks → exactly one `implements` edge; `confidence` never `inferred`. +- [ ] **Step 5.2:** Implement `edges.py`. Resolved-or-dropped throughout. +- [ ] **Step 5.3:** Full suite + lints. Commit: `feat(rust): anchored imports/implements edges (resolved-or-dropped, §6 contract + oracle-cited semantics)`. + +--- + +### Task 6: Freeze `tests/golden/identity/rust/` (the SP2 completion gate) + +**Files:** +- Create: `tests/golden/identity/rust/{__init__.py,README.md,_capture.py,regen.py,test_rust_identity_parity.py,fixtures/rustapp/...,corpus/...}` — follow the Python oracle's conventions (read `tests/golden/identity/{README.md,_capture.py,regen.py}` first: LF-pinned fixtures via `.gitattributes`, no `.weft/`/`weft.toml` in fixtures, META.json with corpus_version/scheme/reason) + +**Design (locked):** the Rust capture is a PARTIAL mirror by necessity: `RustAnalysisContext` is not the Python `AnalysisContext` (analyzer.py last_context → None), so SARIF code-flows / taint facts / explain are NOT capturable — the Rust identity surface is **findings** (`Finding.to_jsonl()` for `RS-WL-* ∧ Kind.DEFECT` — the `is_identity_bearing` predicate filters `RS-WL-*`, not `PY-WL-*`) + **entity rows** (qualname, id-kind, parent, span for EVERY emitted entity) + **edges**. State this in the rust README. Fixture = vendored crate `fixtures/rustapp/` (`Cargo.toml` `name = "rust-app"` → crate `rust_app`; `src/main.rs` + `src/cmd/runner.rs`) exercising: RS-WL-108 (tainted program), RS-WL-112 (tainted sh -c arg), a `/// @trusted(level=ASSURED)` marker, an impl method, a cfg twin, and at least one leaf kind. **Constraint: NO path-typed generic args anywhere in the fixture** (`impl From` is the un-decided reserved-colon case — freezing it would pre-empt Task 9's cross-tool decision; record the constraint in the rust README). + +- [ ] **Step 6.1:** Write `_capture.py` + `regen.py` + the fixture crate + the parity test; run `regen.py` ONCE to seed `corpus/`. +- [ ] **Step 6.1b (non-vacuity — must pass before freezing):** assert over the seeded JSON: (a) ≥1 finding row per rule (`RS-WL-108` AND `RS-WL-112`), each `fingerprint` non-empty; (b) ≥1 entity row with `kind == "impl"`; (c) ≥1 qualname containing `@cfg(`; (d) ≥1 qualname prefixed `rust_app.cmd.runner` (real crate prefix + cross-file route — NOT a directory name); (e) ≥1 edge. Encode these as permanent structural tests in `test_rust_identity_parity.py` (the analogue of the Python oracle's non-vacuity tests), not a one-off eyeball. +- [ ] **Step 6.2:** Parity test green on a fresh run. Trip-wire proof: mutate a **fingerprint hex char** in the seeded corpus findings (NOT a message string), run → test FAILS; revert; mutate one **qualname** in the entity rows, run → FAILS; revert. Commit: `test(identity): freeze the Rust finding-identity corpus (SP2 completion gate) — crate-prefixed RS-WL-* identity`. + +--- + +### Task 7: Identity graduation — remove provisional plumbing, retire the banner claim, docs + +**Files:** +- Modify: `src/wardline/rust/rules.py:141` (remove the `"provisional_identity": True` entry from the `properties` dict in `_finding()`) +- Modify: `src/wardline/core/suppression.py:68-77` (drop the provisional short-circuit — the whole if-block incl. the `continue`; the SEPARATE `Maturity.PREVIEW` guards at :104/:127 are a different mechanism used by Python preview rules — LEAVE THEM) +- Modify: `src/wardline/core/baseline.py:55-62` (drop the provisional exclusion) +- Modify: `src/wardline/cli/scan.py:157-168` (banner: drop ONLY the "provisional identity (baseline-ineligible)" claim. The severity-override claim is STILL TRUE — analyzer.py:84-85: weft.toml severity overrides do not apply to Rust — so KEEP it. New banner: `note: --lang rust covers the command-injection slice (RS-WL-108/112); config severity overrides do not yet apply to Rust findings.`) +- Modify: `tests/unit/rust/test_provisional_identity.py` → rename `test_rust_identity_graduated.py`, INVERT (a matching baseline entry now suppresses; `build_baseline_document` now includes RS findings) +- Modify: `docs/guides/rust-preview.md` (graduate: provisional/baseline-ineligible warning → baseline-eligible statement; KEEP coverage-scope + severity-override warnings), `docs/guides/agents.md:294` (same stale sentence — panel-found), `docs/reference/cli.md`, `CHANGELOG.md` [Unreleased] +- Modify: `src/wardline/rust/qualname.py:20` + `analyzer.py:176` docstrings (stale provisional references) + +- [ ] **Step 7.1 (TDD):** Invert the two provisional tests first; watch them fail against current code. +- [ ] **Step 7.2:** Remove the plumbing (4 code sites); tests pass. Scoped grep: `grep -rn "provisional" src/ tests/ docs/guides/ docs/reference/ CHANGELOG.md` — zero hits outside the CHANGELOG's historical entries (archived plans/specs under docs/superpowers/ are exempt history). Verify no other consumer of `provisional_identity` exists anywhere in `src/`. +- [ ] **Step 7.3:** Docs + CHANGELOG (one [Unreleased] entry: full producer surface, SP2 whole-tree identity, baseline eligibility, BREAKING note that RS-WL fingerprints change once — they were never baseline-eligible, no migration needed; severity-override gap persists and is tracked). `mkdocs build --strict` green. Commit: `feat(rust)!: graduate RS-WL-* identity — baseline-eligible, provisional plumbing removed`. + +--- + +### Task 8: Corpus drift alarm (closes `wardline-868908944b`) + +**Files:** +- Modify: `tests/conformance/test_loomweave_rust_qualname_parity.py` +- Modify: `pyproject.toml` (register `loomweave_drift` in `[tool.pytest.ini_options].markers` AND append `and not loomweave_drift` to the `addopts` `-m` exclusion — BOTH are required; markers-only would run it in the default suite and fail in CI where the sibling checkout is absent) + +**Design (locked, deliberately light):** +1. **Byte-pin:** constant `UPSTREAM_BLOB_SHA = ""` + a test computing the git blob hash of the vendored file — read in BINARY mode, `hashlib.sha1(b"blob %d\x00" % len(data) + data).hexdigest()` — asserting equality, plus a sanity assert (40 lowercase hex chars). Cross-check the constant once via `git hash-object tests/conformance/qualnames_rust.json` (the reference oracle for the formula) before committing. +2. **Live recheck (opt-in):** `@pytest.mark.loomweave_drift` test: locate the sibling checkout (`WARDLINE_LOOMWEAVE_REPO` env override, default `/home/john/loomweave`); absent → `pytest.skip`; present → byte-compare `fixtures/qualnames_rust.json` to the vendored copy, FAIL on drift with a "re-vendor + conform (see test header)" message. Document the re-vendor as a release-gate item in the test header. + +- [ ] **Step 8.1:** Write both tests; run the blob-pin in the default suite and `-m loomweave_drift -v` live — green against the Task-1 upstream commit. Verify the default suite still deselects it (`pytest -q` collects no loomweave_drift test). Commit: `test(conformance): corpus drift alarm — upstream blob byte-pin + opt-in live recheck (closes wardline-868908944b)`. Filigree: close `wardline-868908944b`. + +--- + +### Task 9: Reserved-colon + const-arg-spacing ADR-049 decision letter (no implementation) + +Claims `wardline-be5ee9cc34` (`--advance` walks triage→confirmed; do NOT move to fixing — it stays blocked on the cross-tool decision; release the claim after the letter lands and record the dependency instead). + +**Files:** +- Create: `docs/integration/2026-06-10-wardline-loomweave-rust-qualname-amendment-requests.md` + +- [ ] **Step 9.1:** Draft the letter (Wardline → Loomweave), three sections: +1. **Reserved-colon path-typed generic args** (the decision request): today `impl From for Foo` renders a `:`-bearing locator that Wardline emits un-gated and Loomweave rejects-and-degrades whole-file (`validate_no_colon`, entity_id.rs:140) — both bad endpoints, ubiquitous in real Rust. **Propose:** extend the existing injective `escape_reserved` (`%`→`%25`, `:`→`%3A`) — already the dialect's cfg-predicate precedent — to `type_textual` rendering inside `impl[...]`/`<...>` fragments, so `From` → `From`: collision-free (unlike last-segment: `io::Error` vs `fmt::Error`), reversible, one normalizer for both producers. Name the rejected alternatives (last-segment collision; degrade-whole-file loses every entity in the file for one impl). Request corpus rows (`path_typed_generic_arg_inherent`, `path_typed_generic_arg_trait`) and an ADR-049 §2 amendment. +2. **Const-generic-arg spacing**: propose the oracle strip_ws const args (`Foo<{N+1}>` not `Foo<{ N + 1 }>`), converging on the whitespace-free form Wardline already renders; request a corpus row. +3. Record that leaf-kind, stacked-cfg, cfg-escape, and leaf-kind-cfg-twin rows were vendored upstream by this sprint (Task 1 commit hash) — closing those from the amend3 tail. +- [ ] **Step 9.2:** Filigree: comment on `wardline-be5ee9cc34` + `wardline-e8f7c0508f` with the letter path; create a blocker issue "Loomweave ADR-049 reserved-colon decision (letter sent, awaiting amendment + corpus rows)" and `dependency_add` from be5ee9cc34 to it. Commit: `docs(integration): ADR-049 amendment requests — reserved-colon escape proposal + const-arg spacing`. +- [ ] **Step 9.3:** Surface in the final report: the letter needs the user's sign-off before sending/implementing. + +--- + +### Task 10: Hard-gates sweep, merge, filigree closes + +- [ ] **Step 10.1:** The full gate matrix, all green, output captured: + - `.venv/bin/pytest -q` (full suite) — then machine-check the xfail gate: `.venv/bin/pytest -q 2>&1 | tail -2` must show **0 xfailed**, and `grep -c "pytest.xfail" tests/conformance/test_loomweave_rust_qualname_parity.py` must be 0 + - `.venv/bin/pytest -m rust_e2e -v` + - `.venv/bin/pytest -m loomweave_drift -v` (live oracle check) + - Python identity oracle: `.venv/bin/pytest tests/golden/identity -q` — all green incl. the 3 byte-identity corpus checks (any byte diff is a sprint-stopping regression) + - Rust identity oracle: `.venv/bin/pytest tests/golden/identity/rust -q` + - `.venv/bin/ruff check . && .venv/bin/ruff format --check .` + - `.venv/bin/mypy src` + - `.venv/bin/mkdocs build --strict` + - Self-scan: `.venv/bin/wardline scan . --fail-on ERROR` (exit 0) +- [ ] **Step 10.2:** Merge `feat/rust-gold` → `rc5`: from the MAIN checkout (`/home/john/wardline`), verify `git status` (the dirty files are ci.yml/.gitignore/docs/index.md/mkdocs.yml/docs/arch-analysis — all disjoint from sprint paths), `git merge --no-ff feat/rust-gold`, verify the dirty files are untouched after. **Re-run the gate on the post-merge rc5 tree** (`pytest -q tests/unit/rust tests/conformance tests/golden` + `mkdocs build --strict` — the rc5 checkout's mkdocs.yml differs from the worktree's). +- [ ] **Step 10.3:** Filigree: verify closes (4fdad782a7, 868908944b), comments + dependency on be5ee9cc34/e8f7c0508f, close the umbrella `wardline-9f00d5b44b` only if every gate held; `observation_list` skim per CLAUDE.md. + +--- + +## Self-review notes (spec-coverage check) + +- Prompt scope 1 (SP2 whole-tree) → Task 4. Scope 2 (Phase 1b surface) → Tasks 1–3, 5. Scope 3 (identity graduation) → Tasks 4 (rekey), 6, 7. Scope 4 bugs → Tasks 1–2 (4fdad782a7), 2+9 (e8f7c0508f), 9 (be5ee9cc34), 8 (868908944b). +- Known deviations from the sprint prompt, justified by verified oracle ground truth: (a) the `_KNOWN_KINDS` guard does NOT trip on today's upstream corpus (no leaf-kind rows exist upstream) — Task 1 creates the rows the prompt assumed, then Tasks 2–3 do the guard/producer work; (b) "worktree + work directly on rc5" contradiction resolved as worktree branch `feat/rust-gold` merged to rc5 at the end (Task 10). +- Edges have no corpus rows (entity-only corpus) — changeset §6 + cited oracle source (resolve.rs/extract.rs) are the contract; tests pin both, citations in docstrings. From 4c31433ab7f2602ea3eb6ac0cff89ab339851972 Mon Sep 17 00:00:00 2001 From: John Morrissey Date: Wed, 10 Jun 2026 09:33:01 +1000 Subject: [PATCH 079/186] docs(spec): fold Phase 1b producer surface into SP2 (user-approved) + oracle ground-truth corrections Co-Authored-By: Claude Fable 5 --- ...026-06-08-wardline-rust-frontend-design.md | 42 ++++++++++++++++++- 1 file changed, 41 insertions(+), 1 deletion(-) diff --git a/docs/superpowers/specs/2026-06-08-wardline-rust-frontend-design.md b/docs/superpowers/specs/2026-06-08-wardline-rust-frontend-design.md index eb03180f..892a40a0 100644 --- a/docs/superpowers/specs/2026-06-08-wardline-rust-frontend-design.md +++ b/docs/superpowers/specs/2026-06-08-wardline-rust-frontend-design.md @@ -359,6 +359,14 @@ this is committed at `docs/integration/2026-06-09-loomweave-rust-qualname-dialec ### 6.3 Module-route resolution + the slice-1 / SP2 reproducibility line +> **Phase-1b fold amendment, 2026-06-10 (user-approved).** SP2 is no longer the qualname-prefix work +> alone: the Phase-1b producer surface folds into it (§11; the change-set is +> `docs/integration/2026-06-09-loomweave-rust-qualname-phase1b-changeset.md`), so the SP2 entity +> surface is the **full ten-kind set** — Wardline emits `module` and `struct` (and the other leaf) +> rows itself, not just callables. The *reproducibility* line below is unchanged by the fold (the +> Phase-1b rows are all slice-1 modulo the shared crate-prefix caveat); `#[path]` remains the shared +> known gap. + Rust modules are **not 1:1 with files**. The route is the `mod` tree from the crate root (§6.2). Per the corpus's reproducibility tags, the line between what Wardline reproduces **now** vs at **SP2** is: @@ -379,6 +387,19 @@ fingerprints once. ### 6.4 Identity pinning — the corpus is INVERTED (Loomweave seeds, Wardline vendors) +> **Phase-1b fold amendment, 2026-06-10 (user-approved — supersedes the comparison rule in the second +> bullet).** With the Phase-1b producer surface folded into SP2 (§11), Wardline emits the **full +> ten-kind surface** (`module`, `struct`, `function`, `enum`, `trait`, `type_alias`, `const`, +> `static`, `macro`, `impl`), so "rows Wardline never emits" no longer holds and the conformance +> comparison **graduates from the subset-consumer rule to the full-set rule** (change-set §7 rule 1): +> compare Wardline's full ordered emission of `(qualname, kind)` pairs — semantic `method` mapped to +> id-kind `function` — list-equal against `expected`. Oracle ground-truth corrections (verified +> 2026-06-10): `feat/rust-plugin-spec` is **merged into Loomweave `rc4`** (the plugin is live, +> on-by-default; the stale local branch ref is to be ignored); the upstream corpus gained +> `generic_self_nested_param` (zero diffs in shared cases); and the leaf-kind, stacked-cfg, +> cfg-escape, and leaf-kind-cfg-twin rows are added **upstream by this sprint** (the extractor +> already emits all of it, un-oracled), verified by Loomweave's own cargo gate, then re-vendored. + - **Loomweave hosts** `fixtures/qualnames_rust.json` (extractor-**generated**, locked by `crates/loomweave-plugin-rust/tests/qualname_conformance.rs`, passing). **Wardline vendors** a pinned copy to `tests/conformance/qualnames_rust.json` and reproduces `expected` **byte-for-byte** from its @@ -628,7 +649,13 @@ Six sub-projects, each its own spec→plan→build cycle: identity oracle stay byte-green.** - **SP2 — Rust parse + index** . tree-sitter-rust → entities; the ADR-049 qualname dialect (§6) incl. the **whole-tree** pieces that are SP2 by construction — crate-name-from-`Cargo.toml`, cross-file - module route, `#[path]` (shared known gap) (§6.3); and **the + module route, `#[path]` (shared known gap) (§6.3); **the full Phase-1b producer surface** + *(amended 2026-06-10 — user-approved fold; + `docs/integration/2026-06-09-loomweave-rust-qualname-phase1b-changeset.md`)* — the six leaf kinds + (`enum`/`trait`/`type_alias`/`const`/`static`/`macro`), the `impl` entity with + `module → impl → method` re-parenting, the anchored `imports`/`implements` edges, and + `ontology_version 0.4.0` under `plugin_id rust`, taking the conformance comparison from + subset-consumer to full-set (§6.4); and **the frozen `tests/golden/identity/rust/` finding corpus** (§6.4). These are the SP2 *completion gates*, the point at which the crate-prefixed `RS-WL-*` identity stops being provisional. - **SP3 — Rust trust vocabulary** . `rust_taint.yaml` + `RustTrustProvider` (doc-comment markers) + @@ -692,3 +719,16 @@ rows). Reproducibility tiers pinned: slice-1 = the file-module-rooted suffix; ** prefix + cross-file route + `#[path]`** (so R-2 downgrades to Med). SEI fold resolved: keep folding the qualname, never Loomweave's SEI token. Reply committed at `docs/integration/2026-06-09-loomweave-rust-qualname-dialect-reply.md`. + +**Round 3 — Phase 1b folded into SP2 + oracle ground-truth corrections (2026-06-10).** The +user-approved fold: SP2 (§11) now also comprises the **Phase-1b producer surface** from +`docs/integration/2026-06-09-loomweave-rust-qualname-phase1b-changeset.md` — six leaf kinds, the +`impl` entity + method re-parenting, the anchored `imports`/`implements` edges, `ontology_version +0.4.0` under `plugin_id rust`. Consequence for §6.3/§6.4: Wardline emits the **full ten-kind +surface**, so the "module/struct rows Wardline never emits" premise is retired and the conformance +comparison graduates from the subset-consumer rule to the **full-set rule** (change-set §7 rule 1). +Oracle ground truth corrected (verified 2026-06-10): `feat/rust-plugin-spec` is **merged into +Loomweave `rc4`** (plugin live, on-by-default); the upstream corpus gained +`generic_self_nested_param`; the leaf-kind / stacked-cfg / cfg-escape / leaf-kind-cfg-twin rows — +emitted by the extractor but un-oracled — are added **upstream by this sprint**, gated by +Loomweave's own cargo suite, then re-vendored. From ee233da3ac63dfd7d39d039a922a3ae8df861f20 Mon Sep 17 00:00:00 2001 From: John Morrissey Date: Wed, 10 Jun 2026 09:39:45 +1000 Subject: [PATCH 080/186] =?UTF-8?q?fix(rust):=20fold=20ALL=20stacked=20#[c?= =?UTF-8?q?fg]=20predicates=20+=20mirror=20cfg=20reserved-char=20escape=20?= =?UTF-8?q?(oracle=20rows=20landed=20upstream)=20=E2=80=94=20closes=20ward?= =?UTF-8?q?line-4fdad782a7?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Re-vendor qualnames_rust.json from loomweave rc4 @ a209fc7 (blob 56cba0fe): +5 cases (generic_self_nested_param, leaf_item_kinds, stacked_cfg_twin, cfg_escape_reserved_char, leaf_kind_cfg_twin); _KNOWN_KINDS extended to the ten-kind ADR-049 set. cfg layering now mirrors the oracle: index.py collects predicates RAW (pending_cfgs list), qualname.cfg_discriminant normalises+ escapes exactly once (%->%25 then :->%3A, before the any()/all() arg sort). leaf-kind rows stay invisible to the function-only comparison until the full-set graduation (next task). Co-Authored-By: Claude Fable 5 --- src/wardline/rust/index.py | 26 +++---- src/wardline/rust/qualname.py | 51 +++++++++++--- tests/conformance/qualnames_rust.json | 69 ++++++++++++++++++- .../test_loomweave_rust_qualname_parity.py | 36 +++++++--- tests/unit/rust/test_index.py | 23 +++++++ tests/unit/rust/test_qualname.py | 34 ++++++++- 6 files changed, 207 insertions(+), 32 deletions(-) diff --git a/src/wardline/rust/index.py b/src/wardline/rust/index.py index af8265ac..930d565c 100644 --- a/src/wardline/rust/index.py +++ b/src/wardline/rust/index.py @@ -81,19 +81,21 @@ def _walk_scope( entities: list[RustEntity], path: str, ) -> None: - # Attributes are *preceding siblings* of the item they decorate, so fold each - # pending cfg predicate onto the next item. Any non-attribute node resets it. - items: list[tuple[Node, str | None]] = [] - pending_cfg: str | None = None + # Attributes are *preceding siblings* of the item they decorate, so accumulate + # every pending cfg predicate RAW onto the next item (mirrors extract.rs + # `cfg_predicates` — ALL stacked #[cfg]s feed the discriminant, normalisation + # happens exactly once in `cfg_discriminant`). Any non-attribute node resets it. + items: list[tuple[Node, list[str]]] = [] + pending_cfgs: list[str] = [] for child in child_nodes: if child.type == "attribute_item": pred = q.cfg_predicate_of(child) if pred is not None: - pending_cfg = pred + pending_cfgs.append(pred) continue if child.type in _ITEM_TYPES: - items.append((child, pending_cfg)) - pending_cfg = None + items.append((child, pending_cfgs)) + pending_cfgs = [] # The @cfg suffix is added only on a bare-path COLLISION (ADR-049): a lone # cfg-gated item gets no suffix. Collisions are per-kind; we emit only callables, @@ -115,12 +117,12 @@ def _walk_scope( impl_segments[node.id] = seg impl_twin_counts[seg] += 1 - for node, cfg in items: + for node, cfgs in items: if node.type == "function_item": name = _name(node) qualname = f"{module}.{name}" - if cfg is not None and fn_name_counts[name] > 1: - qualname += f"@cfg({cfg})" + if cfgs and fn_name_counts[name] > 1: + qualname += q.cfg_discriminant(cfgs) entities.append(_entity(qualname, "function", node, nmap, path)) elif node.type == "mod_item": body = node.child_by_field_name("body") @@ -130,8 +132,8 @@ def _walk_scope( seg = impl_segments.get(node.id) if seg is None: continue - if cfg is not None and impl_twin_counts[seg] > 1: - seg += f"@cfg({cfg})" + if cfgs and impl_twin_counts[seg] > 1: + seg += q.cfg_discriminant(cfgs) _emit_impl_methods(node, f"{module}.{seg}", nmap, entities, path) # struct_item: scope-only, never a callable -> not emitted. diff --git a/src/wardline/rust/qualname.py b/src/wardline/rust/qualname.py index cfcee550..ece7c206 100644 --- a/src/wardline/rust/qualname.py +++ b/src/wardline/rust/qualname.py @@ -43,6 +43,7 @@ from tree_sitter import Node __all__ = [ + "cfg_discriminant", "cfg_predicate_of", "impl_type_param_names", "normalize_cfg_predicate", @@ -76,17 +77,23 @@ def normalize_cfg_predicate(text: str) -> str: """Canonicalise a ``cfg(...)`` predicate, mirroring loomweave ``qualname.rs`` ``normalise_pred`` BYTE-FOR-BYTE (the @cfg discriminant is a parity surface). - ``text`` is the argument token-tree text incl. its parens, e.g. ``"(unix)"`` or - ``"(any(windows, unix))"``; the outer cfg-argument parens are stripped to the bare - predicate, then: all whitespace removed, and a single top-level ``any(...)``/ - ``all(...)`` wrapper's args sorted by a **naive** ``split(',')`` (NOT paren-aware — - this is exactly the oracle's algorithm; deeper nesting is left as the deterministic - stripped string, even though that mangles, because the contract is byte-equality - with the oracle, not a "nicer" canonical form). + ``text`` is the RAW argument token-tree text, with or without its outer parens + (``"(unix)"`` / ``"unix"`` / ``"(any(windows, unix))"``); the outer cfg-argument + parens (if present) are stripped to the bare predicate, then — in the oracle's + exact order: all whitespace removed; every reserved entity-id char escaped + (``_escape_reserved``: ``%`` -> ``%25`` first, then ``:`` -> ``%3A`` — injective, + so ``feature="a:b"`` and a literal source ``feature="a%3Ab"`` stay distinct); and + a single top-level ``any(...)``/``all(...)`` wrapper's args sorted by a **naive** + ``split(',')`` (NOT paren-aware — this is exactly the oracle's algorithm; deeper + nesting is left as the deterministic stripped string, even though that mangles, + because the contract is byte-equality with the oracle, not a "nicer" canonical + form). The escape runs on the whole stripped predicate BEFORE the any()/all() + split, exactly as in ``normalise_pred``. """ stripped = "".join(text.split()) if stripped.startswith("(") and stripped.endswith(")"): stripped = stripped[1:-1] + stripped = _escape_reserved(stripped) for fn in ("any", "all"): prefix = fn + "(" if stripped.startswith(prefix) and stripped.endswith(")"): @@ -96,9 +103,33 @@ def normalize_cfg_predicate(text: str) -> str: return stripped +def _escape_reserved(s: str) -> str: + """Escape every reserved entity-id char so a cfg predicate can never inject the + reserved ``:`` separator into a qualname (mirrors qualname.rs ``escape_reserved``). + Order matters for injectivity: the ``%`` introducer is encoded FIRST.""" + return s.replace("%", "%25").replace(":", "%3A") + + +def cfg_discriminant(predicates: Sequence[str]) -> str: + """Fold ALL of an item's RAW ``#[cfg(...)]`` predicates into the stable ``@cfg(...)`` + discriminant suffix (mirrors qualname.rs ``cfg_discriminant`` BYTE-FOR-BYTE): each + predicate normalised+escaped exactly once (``normalize_cfg_predicate``), the set + sorted (order-independent — NOT source order), joined with ``&``. Folding every + predicate is what keeps stacked cfg-twins (``#[cfg(feature="a")] #[cfg(unix)]`` vs + ``#[cfg(feature="b")] #[cfg(unix)]``) distinct. Applied by ``index.py`` only on a + bare-path COLLISION, exactly like the oracle's extract.rs twin counter.""" + return f"@cfg({'&'.join(sorted(normalize_cfg_predicate(p) for p in predicates))})" + + def cfg_predicate_of(attribute_item: Node) -> str | None: - """The normalised cfg predicate of an ``attribute_item`` node, or ``None`` if it is - not a ``#[cfg(...)]`` attribute. e.g. ``#[cfg(unix)]`` -> ``"unix"``. + """The RAW cfg predicate text of an ``attribute_item`` node (outer argument parens + included, source spacing intact — e.g. ``#[cfg(feature = "a")]`` -> + ``'(feature = "a")'``), or ``None`` if it is not a ``#[cfg(...)]`` attribute. + + Deliberately UN-normalised, mirroring extract.rs ``cfg_predicates`` (raw token + text): collection is raw, and ``cfg_discriminant`` is the single place predicates + are normalised + reserved-char-escaped — normalising here too would double-escape + (``a:b`` -> ``a%253Ab``). """ if not attribute_item.named_children: return None @@ -111,7 +142,7 @@ def cfg_predicate_of(attribute_item: Node) -> str | None: args = attribute.child_by_field_name("arguments") if args is None or args.text is None: return None - return normalize_cfg_predicate(args.text.decode("utf-8")) + return args.text.decode("utf-8") # Generic args that are NOT part of the locator key (qualname.rs trait_generic_args / diff --git a/tests/conformance/qualnames_rust.json b/tests/conformance/qualnames_rust.json index be9f8e28..56cba0fe 100644 --- a/tests/conformance/qualnames_rust.json +++ b/tests/conformance/qualnames_rust.json @@ -4,7 +4,7 @@ "delimiter": ".", "crate_rooted": "first segment is the crate name, '-' normalised to '_' (loomweave-cli -> loomweave_cli); discovered from Cargo.toml [package].name read as TEXT, longest-path-prefix match (crate_roots.rs). Falls back to the dir holding src/lib.rs|src/main.rs.", "module_route": "crate root + mod tree to the item; lib.rs/main.rs/mod.rs contribute NO segment; inline `mod foo {}` nests. #[path] is NOT yet honoured (KNOWN-GAP, sp2).", - "free_items": ".. (function, struct; enum/trait/const/static/type_alias/macro are Phase 1b, not yet emitted).", + "free_items": ".. for EVERY named free-item kind: function, struct, enum, trait, type_alias, const, static, macro (the macro_rules! definition). Trait BODIES are deliberately not walked: a trait definition emits only its own `trait` entity, never its method items.", "self_type_generic_args": " in every impl locator carries the self type's CONCRETE generic arguments (ADR-049 §2 amendment): impl Foo -> Foo, impl Foo -> Foo, so the two impls (and their like-named methods) get distinct ids instead of colliding on a bare `Foo` key and being spuriously merged (silent data loss at the writer's ON CONFLICT(id) DO UPDATE). A self-type arg that is the impl's OWN declared generic param renders POSITIONALLY ($0,$1,... De Bruijn) — impl Foo -> Foo<$0> — so a param rename does not churn; concrete args (i32, String, const, nested generics) render whitespace-free; lifetimes/associated-type bindings dropped. A non-generic self type (impl Foo) renders bare `Foo`. Composition order: .|impl[Trait]>[@cfg(...)].", "trait_impl_method": ".impl[]. e.g. Foo.impl[Display].fmt, Foo.impl[From].from, Foo.impl[Display].fmt. Lifetimes dropped; concrete type/const generic args KEPT in BOTH the self-type prefix and the trait fragment (they are part of the key).", "inherent_impl_method": ".impl#. e.g. Foo.impl#<>.bar, Foo.impl#<$0>.get, Foo.impl#<>.get. The self-type prefix carries concrete instantiation args (Foo vs Foo are distinct) with the impl's own declared params rendered positionally ($0). The trailing #<...> signature renders the impl's declared generic params POSITIONALLY ($0,$1,... De Bruijn) so a param rename does not churn. There is NO source-order ordinal (ADR-049 amend, Option b): same-(self-type-args, positional-generic-signature, cfg) inherent impls MERGE into ONE `impl` entity and their methods all hang off it, so the discriminator is genuinely source-order-independent (reorder-stable). DIFFERENT concrete self-type args (Foo vs Foo) do NOT merge. cfg-twin inherent impls (same type+args+sig, mutually-exclusive cfgs) are split by an @cfg(...) suffix on the impl key (Foo.impl#<>@cfg(unix) vs Foo.impl#<>@cfg(windows)); their methods inherit it.", @@ -175,6 +175,19 @@ {"qualname": "demo.m.Foo<$0>.impl#<$0>.get", "kind": "function"} ] }, + { + "name": "generic_self_nested_param", + "crate": "demo", "module_path": "demo.m", "rel_path": "src/m.rs", + "reproducibility": "slice-1", + "note": "F2 NESTED-PARAM trip-wire (ADR-049 §2 self-type-args amendment, rename-stability limitation). The impl's declared param T appears NESTED inside another type arg (Vec), not at top level. A nested declared param is NOT positionally substituted: the whole self-type arg renders via type_textual (literal `T`), so the self-type prefix is Foo>, NOT recursive Foo>. Contrast positional_generic_param, where the TOP-LEVEL declared param renders positionally ($0). The impl's own inherent signature still renders its declared param positionally (#<$0>). CROSS-TOOL CONTRACT: a second producer (Wardline) MUST mirror this type_textual rendering of nested params; implementing recursive positional substitution (Foo>) would diverge, and THIS row is the conformance trip-wire that catches it. KNOWN LIMITATION: nested-param rendering is therefore NOT rename-stable (impl Foo> renders the distinct key Foo>) — strictly better than the pre-amendment behaviour (which silently dropped a method), but full recursive positional rendering is a deferred follow-up. See ADR-049 §2 and the federation change-set F2.", + "source": "struct Foo(T);\nimpl Foo> { fn get(&self) {} }\n", + "expected": [ + {"qualname": "demo.m", "kind": "module"}, + {"qualname": "demo.m.Foo", "kind": "struct"}, + {"qualname": "demo.m.Foo>.impl#<$0>", "kind": "impl"}, + {"qualname": "demo.m.Foo>.impl#<$0>.get", "kind": "function"} + ] + }, { "name": "cfg_twin", "crate": "demo", "module_path": "demo.m", "rel_path": "src/m.rs", @@ -304,6 +317,60 @@ {"qualname": "demo.m", "kind": "module"}, {"qualname": "demo.m.make", "kind": "macro"} ] + }, + { + "name": "leaf_item_kinds", + "crate": "demo", "module_path": "demo.m", "rel_path": "src/m.rs", + "reproducibility": "slice-1", + "note": "Pins the five Phase 1b leaf kinds the corpus did not previously carry: enum, trait, type_alias, const, static (the macro_rules! definition is already pinned by macro_invocation_generates_no_entity). Each is a free item riding the same . qualname pattern as struct/function. The trait emits ONLY its own `trait` entity — trait bodies are deliberately not walked (extract.rs), so a trait method item would produce no entity.", + "source": "pub enum Color { Red }\npub trait Greet {}\npub type Alias = u8;\npub const LIMIT: u32 = 10;\npub static NAME: &str = \"x\";\n", + "expected": [ + {"qualname": "demo.m", "kind": "module"}, + {"qualname": "demo.m.Color", "kind": "enum"}, + {"qualname": "demo.m.Greet", "kind": "trait"}, + {"qualname": "demo.m.Alias", "kind": "type_alias"}, + {"qualname": "demo.m.LIMIT", "kind": "const"}, + {"qualname": "demo.m.NAME", "kind": "static"} + ] + }, + { + "name": "stacked_cfg_twin", + "crate": "demo", "module_path": "demo.m", "rel_path": "src/m.rs", + "reproducibility": "slice-1", + "note": "STACKED-cfg twins: two inherent impls of one type each carry TWO #[cfg] attributes, differing only in the first. cfg_discriminant folds EVERY predicate (each normalised, the set sorted, joined with `&`), so the discriminants stay distinct (@cfg(feature=\"a\"&unix) vs @cfg(feature=\"b\"&unix)). Folding only the FIRST #[cfg] would hand both blocks the same suffix and merge them at the writer's ON CONFLICT, silently dropping one `go`. The suffix is order-independent: the predicates are sorted AFTER normalisation, not in source order. Each block's method inherits its impl key.", + "source": "struct Foo;\n#[cfg(feature = \"a\")]\n#[cfg(unix)]\nimpl Foo { pub fn go(&self) {} }\n#[cfg(feature = \"b\")]\n#[cfg(unix)]\nimpl Foo { pub fn go(&self) {} }\n", + "expected": [ + {"qualname": "demo.m", "kind": "module"}, + {"qualname": "demo.m.Foo", "kind": "struct"}, + {"qualname": "demo.m.Foo.impl#<>@cfg(feature=\"a\"&unix)", "kind": "impl"}, + {"qualname": "demo.m.Foo.impl#<>@cfg(feature=\"a\"&unix).go", "kind": "function"}, + {"qualname": "demo.m.Foo.impl#<>@cfg(feature=\"b\"&unix)", "kind": "impl"}, + {"qualname": "demo.m.Foo.impl#<>@cfg(feature=\"b\"&unix).go", "kind": "function"} + ] + }, + { + "name": "cfg_escape_reserved_char", + "crate": "demo", "module_path": "demo.m", "rel_path": "src/m.rs", + "reproducibility": "slice-1", + "note": "Reserved-char ESCAPE in cfg predicates: `feature = \"a:b\"` carries the reserved entity-id separator `:`; flowed verbatim into a qualname it would make build_entity_id reject the id and degrade the whole file. escape_reserved encodes injectively — `%` first (% -> %25), then `:` -> %3A — so feature=\"a:b\" renders feature=\"a%3Ab\" and a literal source `%3A` cannot alias it. Escape runs on the whitespace-stripped predicate BEFORE any any()/all() arg sorting.", + "source": "#[cfg(feature = \"a:b\")]\npub fn f() {}\n#[cfg(feature = \"c\")]\npub fn f() {}\n", + "expected": [ + {"qualname": "demo.m", "kind": "module"}, + {"qualname": "demo.m.f@cfg(feature=\"a%3Ab\")", "kind": "function"}, + {"qualname": "demo.m.f@cfg(feature=\"c\")", "kind": "function"} + ] + }, + { + "name": "leaf_kind_cfg_twin", + "crate": "demo", "module_path": "demo.m", "rel_path": "src/m.rs", + "reproducibility": "slice-1", + "note": "cfg-twin discrimination on a Phase 1b LEAF kind: the twin counter is per-(kind, name) across ALL nine named item kinds (extract.rs twin_counts), so two cfg-gated `const LIMIT` siblings get @cfg suffixes exactly like the function (cfg_twin) and struct (struct_cfg_twin) cases. Load-bearing for a second producer doing full-set comparison: a twin counter that only covers fn/struct would emit two colliding demo.m.LIMIT rows here.", + "source": "#[cfg(unix)]\npub const LIMIT: u32 = 1;\n#[cfg(windows)]\npub const LIMIT: u32 = 2;\n", + "expected": [ + {"qualname": "demo.m", "kind": "module"}, + {"qualname": "demo.m.LIMIT@cfg(unix)", "kind": "const"}, + {"qualname": "demo.m.LIMIT@cfg(windows)", "kind": "const"} + ] } ] } diff --git a/tests/conformance/test_loomweave_rust_qualname_parity.py b/tests/conformance/test_loomweave_rust_qualname_parity.py index ea3c64e5..cd4c081e 100644 --- a/tests/conformance/test_loomweave_rust_qualname_parity.py +++ b/tests/conformance/test_loomweave_rust_qualname_parity.py @@ -8,10 +8,18 @@ the *second producer*, it MINTS the same locator string and never parses it. Provenance — re-vendor when Loomweave bumps the corpus: - source: loomweave feat/rust-plugin-spec @ 8f4f85f (fixtures/qualnames_rust.json, - blob be9f8e2, extractor-generated, locked by + source: loomweave rc4 @ a209fc7603b09bb06564e09c8c99390d410ea5b2 + (fixtures/qualnames_rust.json, blob 56cba0fe2d6c449ebc841c52a2800368b2e389e4, + extractor-generated, locked by crates/loomweave-plugin-rust/tests/qualname_conformance.rs) - vendored byte-identical to tests/conformance/qualnames_rust.json (2026-06-09). + vendored byte-identical to tests/conformance/qualnames_rust.json (2026-06-10). + The a209fc7 re-vendor (rust-sp2 sprint, Task 1 upstream) adds FIVE cases: + ``generic_self_nested_param`` (the F2 nested-param trip-wire — the unit-only + guard now has its corpus row), ``leaf_item_kinds`` (enum/trait/type_alias/ + const/static), ``stacked_cfg_twin`` (ALL #[cfg] predicates folded — normalised, + sorted, ``&``-joined), ``cfg_escape_reserved_char`` (injective escape ``%``->``%25`` + then ``:``->``%3A``, applied to the whitespace-stripped predicate BEFORE any + any()/all() arg sort), and ``leaf_kind_cfg_twin`` (per-(kind, name) twin counter). NOTE: this is the **ADR-049 amendment 3 (self-type generic args)** corpus, layered on amendment Option b. The impl ```` segment now carries the self type's CONCRETE generic args (``Foo`` vs ``Foo`` are distinct keys; the impl's own top-level @@ -60,11 +68,23 @@ _CORPUS: dict[str, Any] = json.loads((Path(__file__).parent / "qualnames_rust.json").read_text("utf-8")) _KNOWN_TIERS = {"slice-1", "sp2"} -# The ADR-049-amendment corpus adds `impl` entities (one per merged impl block) and -# `macro` rows. Wardline emits only `function` callables, so the comparison rule -# (set-equality on function-kind qualnames) is unchanged; the extra kinds must be -# *known* so test_expected_kinds_are_known stays a real guard, not a false failure. -_KNOWN_KINDS = {"function", "struct", "module", "impl", "macro"} +# The a209fc7 corpus carries the FULL ten-kind ADR-049 surface (leaf_item_kinds / +# leaf_kind_cfg_twin pin enum/trait/type_alias/const/static; macro + impl were already +# present). Wardline still compares only `function`-kind rows here (the subset-consumer +# rule); the full-set graduation is the producer-surface task. The kinds must be *known* +# so test_expected_kinds_are_known stays a real guard, not a false failure. +_KNOWN_KINDS = { + "module", + "struct", + "function", + "enum", + "trait", + "type_alias", + "const", + "static", + "macro", + "impl", +} def _expected_function_qualnames(case: dict[str, Any]) -> set[str]: diff --git a/tests/unit/rust/test_index.py b/tests/unit/rust/test_index.py index b9b24cdb..37e1ffad 100644 --- a/tests/unit/rust/test_index.py +++ b/tests/unit/rust/test_index.py @@ -53,6 +53,29 @@ def test_semantic_kind_split_rides_metadata_not_the_qualname() -> None: assert entities["demo.m.Foo.impl#<>.bar"].kind == "method" +def test_stacked_cfg_twins_get_distinct_folded_suffixes() -> None: + # ALL stacked #[cfg] attributes fold into the discriminant (loomweave extract.rs + # cfg_predicates collects every cfg; folding only the FIRST/LAST would hand both + # blocks the same suffix and silently merge them). Pinned upstream by the + # stacked_cfg_twin corpus row. + src = ( + "struct Foo;\n" + '#[cfg(feature = "a")]\n#[cfg(unix)]\nimpl Foo { pub fn go(&self) {} }\n' + '#[cfg(feature = "b")]\n#[cfg(unix)]\nimpl Foo { pub fn go(&self) {} }\n' + ) + names = {e.qualname for e in discover_rust_entities(src, module="demo.m")} + assert 'demo.m.Foo.impl#<>@cfg(feature="a"&unix).go' in names + assert 'demo.m.Foo.impl#<>@cfg(feature="b"&unix).go' in names + + +def test_single_stacked_cfg_impl_without_twin_gets_no_suffix() -> None: + # dialect: @cfg is a COLLISION discriminator — a lone stacked-cfg impl stays bare + # (the suffix applies only when extract.rs's twin counter sees a path collision). + src = '#[cfg(feature = "a")]\n#[cfg(unix)]\nstruct Foo;\nimpl Foo { pub fn go(&self) {} }\n' + names = {e.qualname for e in discover_rust_entities(src, module="demo.m")} + assert "demo.m.Foo.impl#<>.go" in names + + def test_node_id_zero_is_the_root() -> None: # Sanity that the stamped NodeId comes from the shared mint authority (pre-order # from the root): the first function's id is well past 0 (root + items precede). diff --git a/tests/unit/rust/test_qualname.py b/tests/unit/rust/test_qualname.py index 9ffd7a27..c9b66613 100644 --- a/tests/unit/rust/test_qualname.py +++ b/tests/unit/rust/test_qualname.py @@ -15,7 +15,11 @@ pytest.importorskip("tree_sitter", reason="wardline[rust] extra not installed") from wardline.rust.index import discover_rust_entities # noqa: E402 -from wardline.rust.qualname import normalize_cfg_predicate, rust_module_route # noqa: E402 +from wardline.rust.qualname import ( # noqa: E402 + cfg_discriminant, + normalize_cfg_predicate, + rust_module_route, +) def test_module_route_root_files_contribute_no_segment() -> None: @@ -46,6 +50,34 @@ def test_normalize_cfg_sorts_a_single_flat_any_all_like_the_oracle() -> None: assert normalize_cfg_predicate("(all(unix, windows))") == "all(unix,windows)" +def test_normalize_cfg_predicate_escapes_reserved_chars() -> None: + # % before : (order matters — injective, mirrors loomweave escape_reserved: + # the introducer is encoded first so a literal source `%3A` cannot alias a + # real escaped `:`; qualname.rs escape_reserved). + assert normalize_cfg_predicate('feature = "a:b"') == 'feature="a%3Ab"' + assert normalize_cfg_predicate('feature = "a%3Ab"') == 'feature="a%253Ab"' + + +def test_escape_happens_before_any_all_split() -> None: + # escape applies to the whole stripped pred BEFORE arg sorting (oracle order: + # qualname.rs normalise_pred — strip ws -> escape_reserved -> any()/all() sort). + assert normalize_cfg_predicate('any(feature = "a:b", unix)') == 'any(feature="a%3Ab",unix)' + + +def test_cfg_discriminant_folds_all_predicates_sorted() -> None: + # ALL predicates fold (each normalised, the set sorted, joined `&`) — mirrors + # loomweave cfg_discriminant (qualname.rs); pinned upstream by stacked_cfg_twin. + assert cfg_discriminant(["unix", 'feature = "a"']) == '@cfg(feature="a"&unix)' + assert cfg_discriminant(['feature = "a"', "unix"]) == '@cfg(feature="a"&unix)' # order-independent + + +def test_cfg_discriminant_normalizes_exactly_once() -> None: + # raw input with a reserved char escapes ONCE (no double-escape through the + # pipeline): index.py collects RAW predicates, cfg_discriminant is the single + # normalisation point — the layering that prevents `a:b` -> `a%253Ab`. + assert cfg_discriminant(['feature = "a:b"']) == '@cfg(feature="a%3Ab")' + + def test_positional_generics_are_rename_stable() -> None: # and render to the identical positional ($0) locator — a param rename # must not churn the qualname (ADR-049 De Bruijn rendering) — in BOTH the self-type From b189b1c8c77b1b6ebaefa0af4ec2fd9577632742 Mon Sep 17 00:00:00 2001 From: John Morrissey Date: Wed, 10 Jun 2026 09:40:10 +1000 Subject: [PATCH 081/186] =?UTF-8?q?docs(integration):=20ADR-049=20amendmen?= =?UTF-8?q?t=20requests=20=E2=80=94=20reserved-colon=20escape=20proposal?= =?UTF-8?q?=20+=20const-arg=20spacing?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-Authored-By: Claude Fable 5 --- ...mweave-rust-qualname-amendment-requests.md | 108 ++++++++++++++++++ 1 file changed, 108 insertions(+) create mode 100644 docs/integration/2026-06-10-wardline-loomweave-rust-qualname-amendment-requests.md diff --git a/docs/integration/2026-06-10-wardline-loomweave-rust-qualname-amendment-requests.md b/docs/integration/2026-06-10-wardline-loomweave-rust-qualname-amendment-requests.md new file mode 100644 index 00000000..c8a5e746 --- /dev/null +++ b/docs/integration/2026-06-10-wardline-loomweave-rust-qualname-amendment-requests.md @@ -0,0 +1,108 @@ +# Wardline → Loomweave: Rust qualname dialect — two ADR-049 amendment requests + one closed tail + +**From:** Wardline engineering (Rust tree-sitter frontend, second ADR-049 producer) +**To:** Loomweave maintainers (Rust plugin) +**Date:** 2026-06-10 +**Re:** Two un-decided dialect gaps both producers share — (1) reserved-colon path-typed generic args, (2) const-generic-arg spacing — plus the record of the amend-3 corpus tail your `rc4` now carries. +**Status:** **DRAFT — awaiting Wardline-owner sign-off before sending; proposal only, neither producer implements until ADR-049 is amended and corpus rows land.** + +--- + +## 0. Why this letter + +Your Phase 1b change-set (`docs/integration/2026-06-09-loomweave-rust-qualname-phase1b-changeset.md`) and the corpus rows landed since put both producers byte-for-byte on every *decided* surface. Two surfaces remain **undecided** — not "Wardline diverges from the oracle" but "the oracle has no good output either." Per ADR-049's authority model, the decision is yours; per the second-producer discipline, we will not normalize unilaterally. This letter proposes the decision, names the alternatives we rejected, and requests the corpus rows that would gate both producers' conformance. + +Nothing here is implemented on the Wardline side, and nothing should be implemented on the Loomweave side, until (a) the Wardline owner signs this off, (b) ADR-049 §2 is amended, and (c) the corpus rows exist. Tracked Wardline-side as `wardline-be5ee9cc34` (§1) and the const-spacing half of `wardline-e8f7c0508f` (§2), both blocked on this decision. + +--- + +## 1. Reserved-colon path-typed generic args — the decision request + +### The shared defect (both endpoints are bad) + +A trait or self-type concrete generic argument that is itself a `::`-path mints a locator segment containing the dialect's one reserved character: + +```rust +impl From for Foo { fn from(e: std::io::Error) -> Self { … } } +``` + +- **Loomweave:** `trait_generic_args` → `type_textual` → `strip_ws` (qualname.rs:158-172, :258-269) renders the arg as `std::io::Error`, so the assembled locator is `…Foo.impl[From]` — colon-bearing. `entity_id()` then rejects it (`validate_no_colon`, entity_id.rs:140-148, invoked at :102), `build_id` maps the `EntityIdError` into a `syn::Error` (extract.rs:861-865), and `degraded_aware` (extract.rs:232-274) collapses the **whole cleanly-parsed file** to a single `syntax_error` module plus one Warning finding. One ubiquitous impl erases every entity in its file. +- **Wardline:** our tree-sitter frontend renders the byte-identical `From` segment and emits it **un-gated** — a locator that violates the dialect's own reserved-separator invariant and that your host would refuse. + +So today the dialect has *no canonical colon-free form* for this construct, and the construct is everywhere in real Rust (`From`, `TryFrom`, `From`, `AsRef`). The vendored corpus only exercises single-segment generic args (`From`, `From`), so neither producer's conformance gate can see the gap. + +### The proposal: extend `escape_reserved` to generic-arg rendering + +ADR-049 already owns an injective reserved-char escape — `escape_reserved` (qualname.rs:314-317): `%` → `%25` **first**, then `:` → `%3A` — applied today to cfg predicates inside `normalise_pred` (qualname.rs:319-336), and now corpus-pinned by `cfg_escape_reserved_char`. We propose applying **the same function** to the **full `type_textual` rendering of each concrete generic argument**, in **both** places that rendering reaches the locator: + +| construct | today (rejected / un-gated) | proposed canonical form | +|---|---|---| +| trait fragment | `Foo.impl[From]` | `Foo.impl[From]` | +| self-type prefix | `Foo.impl#<>` | `Foo.impl#<>` | +| composed | `Foo.impl[From].from` | `Foo.impl[From].from` | + +Precisely: wherever a concrete generic argument is rendered for the locator — the `GenericArgument::Type` arms of `trait_generic_args` (qualname.rs:165) and `self_ty_locator`/`self_ty_arg` (qualname.rs:222, :250) — the rendered string becomes `escape_reserved(type_textual(ty))`. Nested args are covered for free because the escape runs over the full rendered text (`Vec` → `Vec`), as are qself paths (`::Item`). The dialect's structural characters `[ ] # < > @ $ , &` are untouched — they are already legal id bytes; only `%` and `:` rewrite. + +Properties: + +- **Collision-free.** Unlike last-segment truncation, distinct paths stay distinct: `From` ≠ `From` (`…io%3A%3AError` vs `…fmt%3A%3AError`). +- **Injective, hence reversible.** Because `%` is escaped before `:` (the same ordering argument documented at qualname.rs:308-313), a literal `%3A` in source type text cannot alias an escaped `::`; the original rendering is recoverable byte-for-byte. +- **One normalizer, already shared.** It is the SAME `escape_reserved` both engines already mirror for cfg predicates (Wardline's copy is gated by your `cfg_escape_reserved_char` row as of rc4 `a209fc7`). No new escape grammar enters the dialect. +- **Rename-stability unchanged.** Positional params (`$0`) never carry `:`; only concrete instantiations are affected, and they were already churn-on-edit by design. +- **`implements`-edge resolution unaffected.** Trait lookup strips generic args before resolving (extract.rs:788-794), so the escape never reaches the resolver. + +### Rejected alternatives (so the ADR can record them) + +1. **Last-segment truncation** (`From` → `From`): collides `io::Error` with `fmt::Error` — re-opening exactly the silent-data-loss family (writer `ON CONFLICT(id) DO UPDATE` overwrite) that the self-type-args amendment just closed. Rejected. +2. **Keep the degrade-whole-file behavior as canonical:** one `impl From` erases every entity in an otherwise clean file — and Wardline mirroring it would mean a single-file frontend discarding a whole file's findings surface over one impl. Disproportionate, and it leaves the dialect with no rendering at all for a ubiquitous construct. Rejected. +3. **Wardline-only normalization:** breaks the byte-for-byte parity that is the second-producer arrangement's entire reason to exist. Rejected without discussion. + +### Requested artifacts + +- **ADR-049 §2 amendment** specifying `escape_reserved` over the `type_textual` rendering of concrete generic args in both the trait fragment and the self-type prefix. +- **Two corpus rows** (extractor-generated as always; suggested sources): + - **`path_typed_generic_arg_inherent`** — `struct Foo(T); impl Foo { pub fn get(&self) {} }` → self-type prefix escape: `demo.m.Foo.impl#<>` (+ `.get`). + - **`path_typed_generic_arg_trait`** — `struct Foo; impl From for Foo { fn from(_: std::io::Error) -> Self { Foo } }` → trait-fragment escape: `demo.m.Foo.impl[From]` (+ `.from`). + +Both producers then conform in lockstep against the rows; Wardline re-vendors and implements only after they land. + +--- + +## 2. Const-generic-arg spacing — converge on `strip_ws` + +### The divergence + +Const generic arguments are the one place the oracle's rendering is **not** whitespace-free: both `trait_generic_args` (qualname.rs:166-168) and `self_ty_locator` (qualname.rs:223-225) render `GenericArgument::Const` via `to_token_stream().to_string()` — proc-macro2 canonical spacing — **without** the `strip_ws` pass every `Type` arg gets. So a multi-token const arg renders spaced: + +| source | oracle today | Wardline today | +|---|---|---| +| `Foo<{ N + 1 }>` | `Foo<{ N + 1 }>` | `Foo<{N+1}>` | +| `Foo<-1>` | `Foo<- 1>` | `Foo<-1>` | +| `Foo<3>` | `Foo<3>` | `Foo<3>` (already matches) | + +Wardline cannot faithfully match the spaced form without reimplementing proc-macro2's token-spacing algorithm inside a tree-sitter frontend — the same impossibility class as the SEI token. And the spaced form is the odd one out in a dialect whose own normalizer comment calls `strip_ws` "the crate's one path/type normaliser" (qualname.rs:253-257). + +### The proposal + +Route `Const` args through the same `strip_ws` as `Type` args (in both call sites), converging the dialect on the whitespace-free form — which is also embedded whitespace's only sane treatment in a locator. Plain const args (`Foo<3>`, `Foo`, bare const-param idents) are byte-identical under both renderings, so the blast radius is multi-token const expressions only — currently un-oracled, so no corpus row churns. + +Note the composition with §1: a const expression can also carry `::` (`Foo<{ usize::MAX }>`), so the §1 escape should apply to the const rendering after `strip_ws` — i.e. one shared pipeline `escape_reserved(strip_ws(arg))` for every concrete generic argument, type or const. + +### Requested artifact + +- **One corpus row** (suggested name **`const_generic_arg_spacing`**), e.g. `struct Foo; impl Foo<{ 1 + 2 }> { pub fn get(&self) {} }` → `demo.m.Foo<{1+2}>.impl#<>` (+ `.get`) — pinning the stripped form in the self-type prefix. (If you prefer, a trait twin `impl From> …` would pin the trait fragment too; one row is enough to gate the normalizer.) + +--- + +## 3. Record: the amend-3 corpus tail is CLOSED (no action needed) + +For completeness against the open list in `wardline-e8f7c0508f` / the amendment-3 fidelity review — these are **done**, landed on your `rc4` and already vendored + conformed on our side: + +- **Loomweave rc4 commit `a209fc7603b09bb06564e09c8c99390d410ea5b2`** ("test(plugin-rust): pin leaf kinds, stacked-cfg fold, cfg reserved-char escape, leaf-kind cfg twin with corpus rows", fixture blob `56cba0fe2d6c449ebc841c52a2800368b2e389e4`) added four extractor-verified rows to `fixtures/qualnames_rust.json`: + - **`leaf_item_kinds`** — the five previously un-oracled free-item kinds (`enum`/`trait`/`type_alias`/`const`/`static`); + - **`stacked_cfg_twin`** — the all-cfg fold (`impl#<>@cfg(feature="a"&unix)`), pinning `cfg_discriminant`'s sort-and-join over EVERY stacked predicate; + - **`cfg_escape_reserved_char`** — `feature="a:b"` → `f@cfg(feature="a%3Ab")`, pinning `escape_reserved` on the cfg path (the precedent §1 extends); + - **`leaf_kind_cfg_twin`** — per-(kind, name) twin counting on a leaf kind (`LIMIT@cfg(unix)` / `LIMIT@cfg(windows)`). +- The owed **nested-param row** (`generic_self_nested_param`, the F2 follow-up from the Phase 1b change-set) was already present upstream before that commit and is likewise vendored. + +That closes every item of the amend-3 tail except the two decisions requested above. Once ADR-049 is amended and the §1/§2 rows land, ping us; Wardline re-vendors, conforms byte-for-byte, and `wardline-be5ee9cc34` / the spacing half of `wardline-e8f7c0508f` close in lockstep. From 457a92047727e7c8918abfae9a9242bbb268122e Mon Sep 17 00:00:00 2001 From: John Morrissey Date: Wed, 10 Jun 2026 09:52:19 +1000 Subject: [PATCH 082/186] =?UTF-8?q?feat(rust):=20full=20ADR-049=20producer?= =?UTF-8?q?=20surface=20=E2=80=94=20six=20leaf=20kinds,=20impl=20entity,?= =?UTF-8?q?=20module=E2=86=92impl=E2=86=92method=20containment,=20per-kind?= =?UTF-8?q?=20cfg=20twins,=20full-set=20conformance?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-Authored-By: Claude Fable 5 --- src/wardline/rust/analyzer.py | 10 +- src/wardline/rust/index.py | 157 ++++++++++++----- src/wardline/rust/qualname.py | 30 ++++ .../test_loomweave_rust_qualname_parity.py | 44 +++-- tests/unit/rust/test_analyzer_protocol.py | 16 ++ tests/unit/rust/test_index.py | 165 +++++++++++++++++- tests/unit/rust/test_qualname.py | 46 +++-- 7 files changed, 391 insertions(+), 77 deletions(-) diff --git a/src/wardline/rust/analyzer.py b/src/wardline/rust/analyzer.py index f17345cf..c2b0f19f 100644 --- a/src/wardline/rust/analyzer.py +++ b/src/wardline/rust/analyzer.py @@ -113,7 +113,10 @@ def analyze(self, files: Sequence[Path], config: WardlineConfig, *, root: Path) continue self._last_rust_context = context files_analyzed += 1 - functions_total += len(context.entities) + # The coverage METRIC counts CALLABLES only — `context.entities` now carries + # the full ten-kind surface (Phase 1b), but the trust-surface denominator is + # still "functions that could have declared @trusted". + functions_total += sum(1 for e in context.entities.values() if e.kind in ("function", "method")) # Declared = seeded from a `/// @trusted` marker (tier is a real trust level, not # the fail-closed default). This is the trust SURFACE — the denominator that stops # a default-clean scan over an un-annotated repo from reading as a clean PASS. @@ -136,6 +139,11 @@ def _analyze_tree(self, tree: Tree, *, module: str, path: str) -> tuple[list[Fin project_taints: dict[str, TaintState] = {} triggers: list[RustTriggerContext] = [] for entity in entities: + if entity.kind not in ("function", "method"): + # Phase 1b: the index emits the full ten-kind surface; the taint path + # judges CALLABLES only (a module/struct/const has no body to seed or + # walk — feeding one to taint_for/dataflow would be a category error). + continue try: seed = self._provider.taint_for(entity.node) except ValueError: diff --git a/src/wardline/rust/index.py b/src/wardline/rust/index.py index 930d565c..cd845c6a 100644 --- a/src/wardline/rust/index.py +++ b/src/wardline/rust/index.py @@ -1,16 +1,31 @@ -"""The minimal Rust index: parse tree -> callable ``RustEntity`` list + NodeId stamping. +"""The Rust index: parse tree -> full ADR-049 entity surface + NodeId stamping. -Walks the module scope in source order, minting each callable's qualname via the +Walks the module scope in source order, minting every entity's qualname via the ``qualname`` dialect (ADR-049) and stamping its ``NodeId`` from the shared keying -authority. **Only callables are emitted** (free fns, inherent methods, trait methods, -assoc fns) — structs and modules are scope-only, closures and nested ``fn``s are never -emitted because the walk never descends a ``function_item`` body (ADR-049). A finding -inside a closure/nested fn attributes to the enclosing named fn via ``line_start``. - -This is the single-file, file-module-rooted slice-1 view: the caller supplies ``module`` -(crate name from ``Cargo.toml`` + cross-file route are SP2). tree-sitter types appear -only under ``TYPE_CHECKING`` so importing this module never pulls the ``wardline[rust]`` -extra. +authority. **The full ten-kind surface is emitted** (Phase 1b): the file-scope +``module`` entity FIRST, then — at their source positions — free items over the +nine named kinds (function/struct/enum/trait/type_alias/const/static/macro plus +inline ``mod``), the merged ``impl`` entity (once, at its FIRST contributing +block), and impl methods re-parented onto the impl entity (``module -> impl -> +method`` containment, mirroring extract.rs ``emit_impl``). + +Not emitted, matching the oracle: closures and nested ``fn``s (the walk never +descends a ``function_item`` body — a finding inside one attributes to the +enclosing named fn via ``line_start``), trait-body items (extract.rs deliberately +never walks trait bodies — a trait definition is only its ``trait`` entity), bare +macro INVOCATIONS, external ``mod foo;`` declarations, and ``union`` items +(outside the ten-kind set, the oracle's ``_ => None`` arm). + +cfg twins are counted per-(kind, name) over the nine named item kinds (extract.rs +``twin_counts``): ``fn S`` and ``struct S`` never interfere — the entity id's kind +segment already separates them — and the ``@cfg(...)`` suffix is applied only on a +within-kind collision. Impl blocks have their own pre-cfg twin counter keyed on +the full impl segment. + +This is still the single-file, file-module-rooted view: the caller supplies +``module`` (crate name from ``Cargo.toml`` + cross-file route are SP2). +tree-sitter types appear only under ``TYPE_CHECKING`` so importing this module +never pulls the ``wardline[rust]`` extra. """ from __future__ import annotations @@ -34,38 +49,60 @@ __all__ = ["RustEntity", "discover_rust_entities", "index_entities"] -_ITEM_TYPES = frozenset({"function_item", "mod_item", "impl_item", "struct_item"}) +# tree-sitter node type -> ADR-049 id-kind, for the free leaf items emitted directly +# at their source position. `mod_item` (recursed) and `impl_item` (merged) are handled +# by their own arms; a bare macro invocation is an `expression_statement`, never here. +_LEAF_KINDS: dict[str, str] = { + "function_item": "function", + "struct_item": "struct", + "enum_item": "enum", + "trait_item": "trait", + "type_item": "type_alias", + "const_item": "const", + "static_item": "static", + "macro_definition": "macro", +} +_ITEM_TYPES = frozenset(_LEAF_KINDS) | frozenset({"mod_item", "impl_item"}) @dataclass(frozen=True, slots=True) class RustEntity: - """A callable Rust entity. ``kind`` is Wardline's *semantic* split - (``function``/``method``); the qualname id-kind is ``function`` for both (ADR-049). + """One emitted Rust entity. ``kind`` spans the full ADR-049 id-kind set + (``module``/``struct``/``function``/``enum``/``trait``/``type_alias``/``const``/ + ``static``/``macro``/``impl``) plus Wardline's *semantic* ``method`` for impl fns — + the qualname id-kind is ``function`` for both (``qualname.entity_id`` maps it). - ``node`` is the ``function_item`` CST node (valid while the source tree is alive) — - the analyzer reuses it (under the *same* ``NodeIdMap``, spec §5) to seed the fn's - trust tier and run dataflow over its body.""" + ``parent`` is the qualname of the containing module or impl entity (``None`` for + the file-scope module) — the ``module -> impl -> method`` containment chain. + + ``node`` is the entity's CST node (valid while the source tree is alive) — for + callables the analyzer reuses it (under the *same* ``NodeIdMap``, spec §5) to seed + the fn's trust tier and run dataflow over its body.""" qualname: str kind: str node_id: NodeId location: Location node: Node + parent: str | None def index_entities(tree: Tree, nmap: NodeIdMap, *, module: str, path: str = "") -> list[RustEntity]: - """Emit the callable entities of an already-parsed ``tree`` under its ``nmap``. + """Emit the entities of an already-parsed ``tree`` under its ``nmap``. The analyzer calls this so index/dataflow/rules share ONE tree and ONE keying authority (spec §5 — re-parsing would mint divergent NodeIds and fail quietly). + The file-scope ``module`` entity is emitted FIRST (corpus row order), spanning + the root node. """ - entities: list[RustEntity] = [] - _walk_scope(tree.root_node.children, module, nmap, entities, path) + root = tree.root_node + entities: list[RustEntity] = [_entity(module, "module", root, nmap, path, parent=None)] + _walk_scope(root.children, module, nmap, entities, path) return entities def discover_rust_entities(source: str, *, module: str, path: str = "") -> list[RustEntity]: - """Parse ``source`` and emit its callable entities, qualname-rooted at ``module``. + """Parse ``source`` and emit its entities, qualname-rooted at ``module``. The corpus-facing API: parses internally (``module`` is the supplied file-module root, since deriving it from ``Cargo.toml`` is SP2). ``path`` only labels ``Location``. @@ -98,11 +135,14 @@ def _walk_scope( pending_cfgs = [] # The @cfg suffix is added only on a bare-path COLLISION (ADR-049): a lone - # cfg-gated item gets no suffix. Collisions are per-kind; we emit only callables, - # so count function names. Inherent impls carry NO source-order ordinal (ADR-049 - # amend, Option b): same-(type, generic-sig) impls share one `impl#<...>` key and - # their methods merge under it. - fn_name_counts = Counter(_name(node) for node, _ in items if node.type == "function_item") + # cfg-gated item gets no suffix. Counting is per-(kind, name) over the nine + # NAMED item kinds (extract.rs `twin_counts`) — the id's kind segment already + # separates `fn S` from `struct S`, so a unique (kind, name) keeps the bare path. + twin_counts: Counter[tuple[str, str]] = Counter() + for node, _cfg in items: + key = _named_item_key(node) + if key is not None: + twin_counts[key] += 1 # Pre-cfg impl-segment twin counts (mirrors extract.rs `impl_twin_counts`): two # impls of the same (type, generic-sig) — incl. cfg-twins — share one key and would @@ -117,25 +157,56 @@ def _walk_scope( impl_segments[node.id] = seg impl_twin_counts[seg] += 1 + # First block with a given (cfg-augmented) impl qualname emits the ONE merged impl + # entity; later same-key blocks only append methods (extract.rs `seen_impl_ids`). + # Per-scope is equivalent to per-file: the qualname embeds the full module path. + seen_impl_quals: set[str] = set() + for node, cfgs in items: - if node.type == "function_item": - name = _name(node) - qualname = f"{module}.{name}" - if cfgs and fn_name_counts[name] > 1: - qualname += q.cfg_discriminant(cfgs) - entities.append(_entity(qualname, "function", node, nmap, path)) - elif node.type == "mod_item": + if node.type == "mod_item": body = node.child_by_field_name("body") - if body is not None: # `mod foo;` (external) has no body to descend - _walk_scope(body.children, f"{module}.{_name(node)}", nmap, entities, path) + if body is None: # `mod foo;` (external) has no body to descend + continue + name = _name(node) + nested = f"{module}.{name}" + if cfgs and twin_counts[("module", name)] > 1: + nested += q.cfg_discriminant(cfgs) + # The inline-mod entity is emitted AT its source position, BEFORE its + # members (corpus nested_inline_mod row order; extract.rs inline-mod arm). + entities.append(_entity(nested, "module", node, nmap, path, parent=module)) + _walk_scope(body.children, nested, nmap, entities, path) elif node.type == "impl_item": seg = impl_segments.get(node.id) if seg is None: continue if cfgs and impl_twin_counts[seg] > 1: seg += q.cfg_discriminant(cfgs) - _emit_impl_methods(node, f"{module}.{seg}", nmap, entities, path) - # struct_item: scope-only, never a callable -> not emitted. + impl_qualname = f"{module}.{seg}" + if impl_qualname not in seen_impl_quals: + seen_impl_quals.add(impl_qualname) + entities.append(_entity(impl_qualname, "impl", node, nmap, path, parent=module)) + _emit_impl_methods(node, impl_qualname, nmap, entities, path) + else: + kind = _LEAF_KINDS[node.type] + name = _name(node) + qualname = f"{module}.{name}" + if cfgs and twin_counts[(kind, name)] > 1: + qualname += q.cfg_discriminant(cfgs) + entities.append(_entity(qualname, kind, node, nmap, path, parent=module)) + + +def _named_item_key(node: Node) -> tuple[str, str] | None: + """The per-(kind, name) twin-counter key of a named item, or ``None`` for items + that don't participate (impl blocks have their own counter; an external ``mod foo;`` + emits nothing, matching extract.rs counting only ``content: Some(_)`` mods).""" + if node.type == "mod_item": + if node.child_by_field_name("body") is None: + return None + return ("module", _name(node)) + kind = _LEAF_KINDS.get(node.type) + if kind is None: + return None + return (kind, _name(node)) def _impl_segment(impl_node: Node) -> str | None: @@ -161,7 +232,11 @@ def _emit_impl_methods( return for child in body.named_children: if child.type == "function_item": - entities.append(_entity(f"{impl_qualname}.{_name(child)}", "method", child, nmap, path)) + # Methods re-parent onto the impl ENTITY (module -> impl -> method), and the + # method qualname builds from the cfg-AUGMENTED impl qualname (extract.rs). + entities.append( + _entity(f"{impl_qualname}.{_name(child)}", "method", child, nmap, path, parent=impl_qualname) + ) def _name(node: Node) -> str: @@ -171,7 +246,7 @@ def _name(node: Node) -> str: return "" -def _entity(qualname: str, kind: str, node: Node, nmap: NodeIdMap, path: str) -> RustEntity: +def _entity(qualname: str, kind: str, node: Node, nmap: NodeIdMap, path: str, *, parent: str | None) -> RustEntity: start = node.start_point end = node.end_point location = Location( @@ -181,4 +256,6 @@ def _entity(qualname: str, kind: str, node: Node, nmap: NodeIdMap, path: str) -> col_start=start[1], col_end=end[1], ) - return RustEntity(qualname=qualname, kind=kind, node_id=nmap.node_id(node), location=location, node=node) + return RustEntity( + qualname=qualname, kind=kind, node_id=nmap.node_id(node), location=location, node=node, parent=parent + ) diff --git a/src/wardline/rust/qualname.py b/src/wardline/rust/qualname.py index ece7c206..82c48340 100644 --- a/src/wardline/rust/qualname.py +++ b/src/wardline/rust/qualname.py @@ -43,8 +43,11 @@ from tree_sitter import Node __all__ = [ + "RUST_ONTOLOGY_VERSION", + "RUST_PLUGIN_ID", "cfg_discriminant", "cfg_predicate_of", + "entity_id", "impl_type_param_names", "normalize_cfg_predicate", "render_positional_generics", @@ -55,6 +58,33 @@ _ROOT_STEMS = frozenset({"lib", "main", "mod"}) +# The ADR-049 producer identity (mirrors loomweave plugin.toml: `plugin_id = "rust"`, +# `ontology_version = "0.4.0"`) — Wardline mints the SAME entity ids as the oracle. +RUST_PLUGIN_ID = "rust" +RUST_ONTOLOGY_VERSION = "0.4.0" + +# The ten ADR-049 id-kinds (plugin.toml `entity_kinds`). Wardline's semantic `method` +# is NOT an id-kind — `entity_id` maps it to `function` itself. +_ID_KINDS = frozenset( + {"module", "struct", "function", "enum", "trait", "type_alias", "const", "static", "macro", "impl"} +) + + +def entity_id(kind: str, qualname: str) -> str: + """The federation entity id ``{plugin}:{kind}:{qualname}`` for an emitted entity. + + Wardline's semantic ``method`` maps to the id-kind ``function`` HERE (callers pass + ``RustEntity.kind`` verbatim, never pre-mapping); any kind outside the ten-kind + ADR-049 set raises ``ValueError`` — mirroring loomweave's ``build_entity_id`` + validation posture (reject, never silently coin a new kind). + """ + if kind == "method": + kind = "function" + if kind not in _ID_KINDS: + msg = f"unknown Rust entity kind {kind!r} (not in the ADR-049 ten-kind set)" + raise ValueError(msg) + return f"{RUST_PLUGIN_ID}:{kind}:{qualname}" + def rust_module_route(*, crate: str, src_root: str, file: str) -> str: """Route a source ``file`` to its dotted module path (ADR-049 §module_route). diff --git a/tests/conformance/test_loomweave_rust_qualname_parity.py b/tests/conformance/test_loomweave_rust_qualname_parity.py index cd4c081e..1c24e099 100644 --- a/tests/conformance/test_loomweave_rust_qualname_parity.py +++ b/tests/conformance/test_loomweave_rust_qualname_parity.py @@ -46,14 +46,15 @@ ``wardline.rust.index`` — they no longer skip). The *structural* self-tests also run and catch a malformed / stale re-vendor. SP2 rows still ``xfail`` (whole-tree view). -The comparison rule (from the corpus ``_consumer_comparison`` key — do NOT use raw -list-equality against ``expected``): - 1. The byte-exact obligation is the ``qualname`` of every NON-``module`` row. - 2. ``kind`` is the locator id-kind (``function`` for every callable); Wardline's - semantic ``method`` ↔ id-kind ``function`` for impl fns. Compare qualname-only. - 3. ``module`` rows are validated via the ``module_route`` section, not re-emitted. -Wardline emits only callables (functions), so the non-vacuous form of (1) is -*set-equality on the function-kind qualnames* (catches both missing and extra). +The comparison rule — GRADUATED (Phase 1b): Wardline now emits the FULL ten-kind +ADR-049 surface (changeset §7 rule 1 for full-surface producers), so the gate is +**ordered-list equality of `(qualname, kind)` pairs** against the corpus +``expected`` — exactly loomweave's own ordered conformance gate. Wardline's +semantic ``method`` maps to the id-kind ``function`` (the only mapping applied); +``module`` rows are compared like every other row AND the ``module_route`` +section separately validates the file->module routing. The corpus +``_consumer_comparison`` subset rule (set-equality over function rows) remains +valid for function-only CONSUMERS; Wardline is no longer one. """ from __future__ import annotations @@ -70,9 +71,9 @@ _KNOWN_TIERS = {"slice-1", "sp2"} # The a209fc7 corpus carries the FULL ten-kind ADR-049 surface (leaf_item_kinds / # leaf_kind_cfg_twin pin enum/trait/type_alias/const/static; macro + impl were already -# present). Wardline still compares only `function`-kind rows here (the subset-consumer -# rule); the full-set graduation is the producer-surface task. The kinds must be *known* -# so test_expected_kinds_are_known stays a real guard, not a false failure. +# present), and Wardline now compares its FULL ordered emission against it (Phase 1b +# graduation). The kinds must be *known* so test_expected_kinds_are_known stays a real +# guard, not a false failure. _KNOWN_KINDS = { "module", "struct", @@ -87,10 +88,11 @@ } -def _expected_function_qualnames(case: dict[str, Any]) -> set[str]: - """The contract's obligation surface for a function-only producer: the set of - ``function``-kind qualnames in the case (``module``/``struct`` rows excluded).""" - return {row["qualname"] for row in case["expected"] if row["kind"] == "function"} +def _expected_all_pairs(case: dict[str, Any]) -> list[tuple[str, str]]: + """The full-surface obligation: the case's ``expected`` rows as an ORDERED list of + ``(qualname, kind)`` pairs — order is part of the contract (loomweave's own gate + compares the emission list in order).""" + return [(row["qualname"], row["kind"]) for row in case["expected"]] # --------------------------------------------------------------------------- # @@ -163,9 +165,15 @@ def test_entity_qualnames(case: dict[str, Any]) -> None: rust_index, _ = _rust_producer() if case["reproducibility"] == "sp2": # pragma: no cover - flips to hard assert at SP2 pytest.xfail("sp2 row: needs the whole-tree view (crate name / cross-file route)") - # WP2 contract: emit callable entities for `source` rooted at `module_path`. - found = {e.qualname for e in rust_index.discover_rust_entities(case["source"], module=case["module_path"])} - assert found == _expected_function_qualnames(case) + # Phase 1b contract: the FULL ordered ten-kind emission for `source` rooted at + # `module_path`, kind-mapped semantic `method` -> id-kind `function` (the one + # legal projection; everything else must match the corpus byte-for-byte AND + # row-for-row in order). + found = [ + (e.qualname, "function" if e.kind == "method" else e.kind) + for e in rust_index.discover_rust_entities(case["source"], module=case["module_path"]) + ] + assert found == _expected_all_pairs(case) @pytest.mark.parametrize("route", _CORPUS["module_route"], ids=lambda r: r["name"]) diff --git a/tests/unit/rust/test_analyzer_protocol.py b/tests/unit/rust/test_analyzer_protocol.py index 7839635e..911836ea 100644 --- a/tests/unit/rust/test_analyzer_protocol.py +++ b/tests/unit/rust/test_analyzer_protocol.py @@ -104,6 +104,22 @@ def test_coverage_posture_flags_empty_trust_surface(tmp_path) -> None: assert cov.properties["functions_total"] == 1 +def test_non_callable_entities_do_not_enter_taint_analysis(tmp_path) -> None: + # Phase 1b: the index now emits struct/enum/const/trait (etc.) rows. The taint + # path must judge CALLABLES ONLY — leaf entities have no body/trust marker, so + # feeding them to taint_for/dataflow would crash or fabricate findings, and the + # coverage METRIC's functions_total must keep counting callables only. + src = 'struct Cfg;\npub enum Mode { A }\npub const NAME: &str = "x";\npub trait Run {}\n' + _INJECTION + (tmp_path / "m.rs").write_text(src, encoding="utf-8") + findings = list(RustAnalyzer().analyze([tmp_path / "m.rs"], _cfg(), root=tmp_path)) + rs = [f for f in findings if f.rule_id.startswith("RS-WL-")] + assert [f.rule_id for f in rs] == ["RS-WL-108"] # exactly the one fn's finding + assert all(f.qualname is not None and f.qualname.endswith(".run") for f in rs) + (cov,) = [f for f in findings if f.rule_id == "WLN-RUST-COVERAGE"] + assert cov.properties["functions_total"] == 1 # callables only, not 5 + assert cov.properties["functions_declared"] == 1 + + def test_clean_file_yields_no_findings(tmp_path) -> None: (tmp_path / "clean.rs").write_text( _TRUSTED + 'fn run() {\n Command::new("ls").arg("-la").output();\n}\n', encoding="utf-8" diff --git a/tests/unit/rust/test_index.py b/tests/unit/rust/test_index.py index 37e1ffad..e3f8e85b 100644 --- a/tests/unit/rust/test_index.py +++ b/tests/unit/rust/test_index.py @@ -1,10 +1,12 @@ -"""WP2: the minimal Rust index — ``function_item`` -> ``RustEntity`` + NodeId stamping. +"""The Rust index — the full ADR-049 entity surface + NodeId stamping. Pins the entity-shape contract the corpus parity gate does not: that every emitted -entity carries a ``Location`` and a ``NodeId``, that closures and nested ``fn``s are -NOT emitted (ADR-049 — the walk never descends a ``function_item`` body), and that -Wardline keeps its semantic ``function``/``method`` split in entity metadata while the -qualname id-kind stays ``function`` for both. +entity carries a ``Location``, a ``NodeId``, and a ``parent`` containment link; that +closures and nested ``fn``s are NOT emitted (ADR-049 — the walk never descends a +``function_item`` body); that Wardline keeps its semantic ``function``/``method`` +split in entity metadata while the qualname id-kind stays ``function`` for both; and +(Phase 1b) the leaf kinds, the merged ``impl`` entity, the corpus-pinned emit +ordering, the per-(kind, name) twin counter, and emission determinism. """ from __future__ import annotations @@ -29,7 +31,7 @@ def test_emits_one_entity_per_callable_excluding_bodies() -> None: entities = discover_rust_entities(_SPECIMEN, module="demo.m") - quals = {e.qualname for e in entities} + quals = {e.qualname for e in entities if e.kind in ("function", "method")} assert quals == {"demo.m.top", "demo.m.Foo.impl#<>.bar", "demo.m.outer"} # the nested fn and the closure produced no entity of their own assert not any("inner" in q for q in quals) @@ -80,5 +82,154 @@ def test_node_id_zero_is_the_root() -> None: # Sanity that the stamped NodeId comes from the shared mint authority (pre-order # from the root): the first function's id is well past 0 (root + items precede). entities = discover_rust_entities("fn only() {}\n", module="demo.m") - (only,) = entities + only = next(e for e in entities if e.kind == "function") assert NodeId(only.node_id) > NodeId(0) + + +# --------------------------------------------------------------------------- # +# Phase 1b producer surface: the full ten-kind ADR-049 emission (leaf kinds, +# the `impl` entity, module -> impl -> method containment, per-kind cfg twins, +# corpus-pinned emit ordering). Oracle: loomweave extract.rs (twin_counts +# :326-358; struct arm :397-400; inline-mod arm :436-439; trait arm :476-492 — +# trait BODIES are never walked) + the vendored qualnames_rust.json rows. +# --------------------------------------------------------------------------- # + + +def test_leaf_item_kinds_emitted_with_module_parent() -> None: + # The five Phase-1b leaf kinds + macro_rules!, each at its source position with + # qualname `.` and parent = the file module; the bare macro + # INVOCATION emits nothing (corpus macro_invocation_generates_no_entity). + src = ( + "pub enum Color { Red }\n" + "pub trait Greet {}\n" + "pub type Alias = u8;\n" + "pub const LIMIT: u32 = 10;\n" + 'pub static NAME: &str = "x";\n' + "macro_rules! make { () => { fn generated() {} } }\n" + "make!();\n" + ) + rows = [(e.qualname, e.kind, e.parent) for e in discover_rust_entities(src, module="demo.m")] + assert rows == [ + ("demo.m", "module", None), + ("demo.m.Color", "enum", "demo.m"), + ("demo.m.Greet", "trait", "demo.m"), + ("demo.m.Alias", "type_alias", "demo.m"), + ("demo.m.LIMIT", "const", "demo.m"), + ("demo.m.NAME", "static", "demo.m"), + ("demo.m.make", "macro", "demo.m"), + ] + + +def test_impl_entity_rows_inherent_trait_and_cfg_forms() -> None: + # The merged `impl` entity is a real row now: inherent `…impl#<>`, trait + # `…impl[Display]`, and the @cfg-split twin forms (corpus inherent_method / + # trait_method / trait_impl_cfg_twin). + src = ( + "struct Foo;\n" + "impl Foo { fn a(&self) {} }\n" + "impl std::fmt::Display for Foo {" + " fn fmt(&self, f: &mut std::fmt::Formatter) -> std::fmt::Result { Ok(()) } }\n" + "#[cfg(unix)]\nimpl Clone for Foo { fn clone(&self) -> Foo { Foo } }\n" + "#[cfg(windows)]\nimpl Clone for Foo { fn clone(&self) -> Foo { Foo } }\n" + ) + impls = [e.qualname for e in discover_rust_entities(src, module="demo.m") if e.kind == "impl"] + assert impls == [ + "demo.m.Foo.impl#<>", + "demo.m.Foo.impl[Display]", + "demo.m.Foo.impl[Clone]@cfg(unix)", + "demo.m.Foo.impl[Clone]@cfg(windows)", + ] + + +def test_containment_parents_module_impl_method() -> None: + # module -> impl -> method containment: a method's parent is the impl ENTITY + # qualname; the impl's parent is the module; free items parent on the module. + src = "fn free() {}\nstruct Foo;\nimpl Foo { fn bar(&self) {} }\n" + by_q = {e.qualname: e for e in discover_rust_entities(src, module="demo.m")} + assert by_q["demo.m"].parent is None + assert by_q["demo.m.free"].parent == "demo.m" + assert by_q["demo.m.Foo"].parent == "demo.m" + assert by_q["demo.m.Foo.impl#<>"].parent == "demo.m" + assert by_q["demo.m.Foo.impl#<>.bar"].parent == "demo.m.Foo.impl#<>" + + +def test_merged_impl_emitted_at_first_block_in_source_order() -> None: + # Two same-key inherent blocks merge to ONE impl entity, emitted at the FIRST + # contributing block (its location.line_start = block 1's line); both blocks' + # methods follow in source order (corpus multiple_inherent_merge + oracle + # emit_impl's seen_impl_ids first-block guard). + src = "struct Foo;\nimpl Foo { fn a(&self) {} }\nimpl Foo { fn b(&self) {} }\n" + entities = discover_rust_entities(src, module="demo.m") + rows = [(e.qualname, e.kind) for e in entities] + assert rows == [ + ("demo.m", "module"), + ("demo.m.Foo", "struct"), + ("demo.m.Foo.impl#<>", "impl"), + ("demo.m.Foo.impl#<>.a", "method"), + ("demo.m.Foo.impl#<>.b", "method"), + ] + (impl_entity,) = [e for e in entities if e.kind == "impl"] + assert impl_entity.location.line_start == 2 # the FIRST contributing block + + +def test_file_module_entity_emitted_first_and_inline_mods_at_source_position() -> None: + # Emit ordering (corpus nested_inline_mod / same_type_name_distinct_module_scopes): + # the file-scope module entity comes FIRST, before any item; an inline `mod` entity + # is emitted at its source position, BEFORE its members. + src = "pub fn top() {}\nmod inner {\n pub fn g() {}\n}\npub fn tail() {}\n" + entities = discover_rust_entities(src, module="demo") + rows = [(e.qualname, e.kind) for e in entities] + assert rows == [ + ("demo", "module"), + ("demo.top", "function"), + ("demo.inner", "module"), + ("demo.inner.g", "function"), + ("demo.tail", "function"), + ] + by_q = {e.qualname: e for e in entities} + assert by_q["demo.inner"].parent == "demo" + assert by_q["demo.inner.g"].parent == "demo.inner" + + +def test_per_kind_twin_counting() -> None: + # The twin counter is per-(kind, name) — extract.rs twin_counts. `fn S` and + # `struct S` share a name but NOT a kind, so the cfg-gated fn is no twin and + # gets NO @cfg suffix (the id's kind segment already separates them). + src = "#[cfg(unix)]\nfn S() {}\nstruct S;\n" + rows = {(e.qualname, e.kind) for e in discover_rust_entities(src, module="demo.m")} + assert ("demo.m.S", "function") in rows + assert ("demo.m.S", "struct") in rows + assert not any("@cfg" in qual for qual, _ in rows) + + +def test_per_kind_twin_suffix_applies_within_a_leaf_kind() -> None: + # Within one (kind, name) the @cfg suffix DOES split (corpus leaf_kind_cfg_twin). + src = "#[cfg(unix)]\npub const LIMIT: u32 = 1;\n#[cfg(windows)]\npub const LIMIT: u32 = 2;\n" + rows = [(e.qualname, e.kind) for e in discover_rust_entities(src, module="demo.m")] + assert rows == [ + ("demo.m", "module"), + ("demo.m.LIMIT@cfg(unix)", "const"), + ("demo.m.LIMIT@cfg(windows)", "const"), + ] + + +def test_emission_is_deterministic() -> None: + # Two runs over the same source produce byte-identical ordered emissions + # (qualname, kind, parent, span) — the property the full-set ordered + # conformance comparison and the identity freeze both lean on. + src = ( + "pub enum Color { Red }\n" + "struct Foo;\n" + "impl Foo { fn a(&self) {} }\n" + "impl Foo { fn b(&self) {} }\n" + "mod inner {\n pub fn g() {}\n}\n" + "#[cfg(unix)]\nfn f() {}\n#[cfg(windows)]\nfn f() {}\n" + ) + + def run() -> list[tuple[str, str, str | None, int, int]]: + return [ + (e.qualname, e.kind, e.parent, e.location.line_start, e.location.line_end) + for e in discover_rust_entities(src, module="demo.m", path="m.rs") + ] + + assert run() == run() diff --git a/tests/unit/rust/test_qualname.py b/tests/unit/rust/test_qualname.py index c9b66613..f5cd414c 100644 --- a/tests/unit/rust/test_qualname.py +++ b/tests/unit/rust/test_qualname.py @@ -16,12 +16,36 @@ from wardline.rust.index import discover_rust_entities # noqa: E402 from wardline.rust.qualname import ( # noqa: E402 + RUST_ONTOLOGY_VERSION, + RUST_PLUGIN_ID, cfg_discriminant, + entity_id, normalize_cfg_predicate, rust_module_route, ) +def test_entity_id_maps_method_and_validates_kind() -> None: + # entity_id mirrors loomweave's build_entity_id posture: `{plugin}:{kind}:{qualname}`, + # Wardline's semantic `method` maps to the id-kind `function` HERE (callers never + # pre-map), and a kind outside the ten-kind ADR-049 set raises. + assert entity_id("function", "demo.m.f") == "rust:function:demo.m.f" + assert entity_id("method", "demo.m.Foo.impl#<>.bar") == "rust:function:demo.m.Foo.impl#<>.bar" + assert entity_id("impl", "demo.m.Foo.impl#<>") == "rust:impl:demo.m.Foo.impl#<>" + assert entity_id("module", "demo.m") == "rust:module:demo.m" + assert entity_id("type_alias", "demo.m.Alias") == "rust:type_alias:demo.m.Alias" + with pytest.raises(ValueError, match="union"): + entity_id("union", "demo.m.U") + assert RUST_PLUGIN_ID == "rust" + assert RUST_ONTOLOGY_VERSION == "0.4.0" + + +def _callable_quals(source: str, module: str) -> set[str]: + """The CALLABLE qualnames of a source — these rendering tests pin method/fn + qualname bytes; the full ten-kind emission surface is test_index.py's job.""" + return {e.qualname for e in discover_rust_entities(source, module=module) if e.kind in ("function", "method")} + + def test_module_route_root_files_contribute_no_segment() -> None: assert rust_module_route(crate="demo", src_root="/p/src", file="/p/src/lib.rs") == "demo" assert rust_module_route(crate="demo", src_root="/p/src", file="/p/src/main.rs") == "demo" @@ -85,8 +109,8 @@ def test_positional_generics_are_rename_stable() -> None: # §2). No source-order ordinal (ADR-049 amend, Option b). with_t = "struct Foo(T);\nimpl Foo { fn get(&self) {} }\n" with_u = "struct Foo(U);\nimpl Foo { fn get(&self) {} }\n" - t = {e.qualname for e in discover_rust_entities(with_t, module="demo.m")} - u = {e.qualname for e in discover_rust_entities(with_u, module="demo.m")} + t = _callable_quals(with_t, "demo.m") + u = _callable_quals(with_u, "demo.m") assert t == u == {"demo.m.Foo<$0>.impl#<$0>.get"} @@ -95,7 +119,7 @@ def test_positional_generics_count_type_params_only() -> None: # only `T` counts. impl<'a, const N: usize, T> -> one positional $0 in both prefix # (Foo<$0>) and signature (impl#<$0>). src = "struct Foo(T);\nimpl<'a, const N: usize, T> Foo { fn get(&self) {} }\n" - quals = {e.qualname for e in discover_rust_entities(src, module="demo.m")} + quals = _callable_quals(src, "demo.m") assert quals == {"demo.m.Foo<$0>.impl#<$0>.get"} @@ -104,7 +128,7 @@ def test_self_type_concrete_args_split_distinct_impls() -> None: # keys, so two like-named `get` methods do NOT collide/merge (the silent-data-loss # family the amendment closes). Mirrors corpus generic_self_inherent_concrete_args. src = "struct Foo(T);\nimpl Foo { fn get(&self) {} }\nimpl Foo { fn get(&self) {} }\n" - quals = {e.qualname for e in discover_rust_entities(src, module="demo.m")} + quals = _callable_quals(src, "demo.m") assert quals == {"demo.m.Foo.impl#<>.get", "demo.m.Foo.impl#<>.get"} @@ -115,8 +139,8 @@ def test_nested_self_type_param_renders_literal_not_positional() -> None: # only guard against accidentally implementing recursive positional substitution. vec = "struct Foo(T);\nimpl Foo> { fn get(&self) {} }\n" ref = "struct Foo(T);\nimpl Foo<&T> { fn get(&self) {} }\n" - v = {e.qualname for e in discover_rust_entities(vec, module="demo.m")} - r = {e.qualname for e in discover_rust_entities(ref, module="demo.m")} + v = _callable_quals(vec, "demo.m") + r = _callable_quals(ref, "demo.m") assert v == {"demo.m.Foo>.impl#<$0>.get"} assert r == {"demo.m.Foo<&T>.impl#<$0>.get"} @@ -125,7 +149,7 @@ def test_non_generic_self_type_renders_bare() -> None: # A non-generic self type renders the bare name (no empty brackets) — unchanged by # the self-type-args amendment. src = "struct Foo;\nimpl Foo { fn bar(&self) {} }\n" - quals = {e.qualname for e in discover_rust_entities(src, module="demo.m")} + quals = _callable_quals(src, "demo.m") assert quals == {"demo.m.Foo.impl#<>.bar"} @@ -136,7 +160,7 @@ def test_empty_turbofish_self_type_does_not_emit_empty_brackets() -> None: # (Not asserted as oracle-convergence: whether syn accepts `impl Foo<>` at all is # unverified; either way neither producer emits a `Foo<>` entity for this input.) src = "struct Foo;\nimpl Foo<> { fn bar(&self) {} }\n" - quals = {e.qualname for e in discover_rust_entities(src, module="demo.m")} + quals = _callable_quals(src, "demo.m") assert quals == {"demo.m.Foo.impl#<>.bar"} @@ -145,8 +169,8 @@ def test_trait_args_drop_lifetimes_and_bindings_and_omit_empty_brackets() -> Non # associated-type bindings dropped), and trait_impl omits <> when none survive. binding = "struct F;\nimpl Iterator for F { fn next(&mut self) {} }\n" lifetime = "struct F;\nimpl<'a> MyTrait<'a> for F { fn m(&self) {} }\n" - b = {e.qualname for e in discover_rust_entities(binding, module="m")} - lt = {e.qualname for e in discover_rust_entities(lifetime, module="m")} + b = _callable_quals(binding, "m") + lt = _callable_quals(lifetime, "m") assert b == {"m.F.impl[Iterator].next"} # binding dropped, no brackets assert lt == {"m.F.impl[MyTrait].m"} # lifetime dropped, no empty <> @@ -155,5 +179,5 @@ def test_cfg_twin_inherent_impls_split_by_at_cfg() -> None: # Post-amendment the ordinal is gone, so a cfg-gated pair of same-signature inherent # impls would COLLIDE to one qualname (silent finding-masking) without the @cfg split. src = "struct Foo;\n#[cfg(unix)]\nimpl Foo { fn run(&self) {} }\n#[cfg(windows)]\nimpl Foo { fn run(&self) {} }\n" - quals = sorted(e.qualname for e in discover_rust_entities(src, module="demo.m")) + quals = sorted(_callable_quals(src, "demo.m")) assert quals == ["demo.m.Foo.impl#<>@cfg(unix).run", "demo.m.Foo.impl#<>@cfg(windows).run"] From 7374b94d9c18cd09b13eaeba0b4c26d9ba079f13 Mon Sep 17 00:00:00 2001 From: John Morrissey Date: Wed, 10 Jun 2026 10:12:06 +1000 Subject: [PATCH 083/186] fix(rust): cfg discriminants survive comment interposition + in-predicate comments; kind-disambiguated context keys (keystone panel, oracle rows pinned upstream) --- src/wardline/rust/analyzer.py | 42 ++++++++++------ src/wardline/rust/context.py | 4 ++ src/wardline/rust/index.py | 17 ++++++- src/wardline/rust/qualname.py | 49 ++++++++++++++++++- tests/conformance/qualnames_rust.json | 24 +++++++++ .../test_loomweave_rust_qualname_parity.py | 12 +++-- tests/unit/rust/test_analyzer_protocol.py | 29 +++++++++++ tests/unit/rust/test_index.py | 43 ++++++++++++++++ tests/unit/rust/test_provider.py | 28 +++++++++++ tests/unit/rust/test_qualname.py | 31 ++++++++++++ 10 files changed, 258 insertions(+), 21 deletions(-) diff --git a/src/wardline/rust/analyzer.py b/src/wardline/rust/analyzer.py index c2b0f19f..8488ea33 100644 --- a/src/wardline/rust/analyzer.py +++ b/src/wardline/rust/analyzer.py @@ -103,7 +103,7 @@ def analyze(self, files: Sequence[Path], config: WardlineConfig, *, root: Path) continue module = _module_for(file, resolved_root) try: - file_findings, context = self._analyze_tree(tree, module=module, path=relpath) + file_findings, context, file_callables = self._analyze_tree(tree, module=module, path=relpath) except Exception as exc: # noqa: BLE001 — per-file isolation, see below # One pathological file (e.g. a RecursionError on a deeply-nested expression) # must not abort the whole scan and lose every other file's findings. Mirror @@ -113,10 +113,12 @@ def analyze(self, files: Sequence[Path], config: WardlineConfig, *, root: Path) continue self._last_rust_context = context files_analyzed += 1 - # The coverage METRIC counts CALLABLES only — `context.entities` now carries - # the full ten-kind surface (Phase 1b), but the trust-surface denominator is - # still "functions that could have declared @trusted". - functions_total += sum(1 for e in context.entities.values() if e.kind in ("function", "method")) + # The coverage METRIC counts CALLABLES only — the emission carries the full + # ten-kind surface (Phase 1b), but the trust-surface denominator is still + # "functions that could have declared @trusted". `_analyze_tree` counts them + # over the entity LIST (never the context mapping, which dict-ification + # could collapse). + functions_total += file_callables # Declared = seeded from a `/// @trusted` marker (tier is a real trust level, not # the fail-closed default). This is the trust SURFACE — the denominator that stops # a default-clean scan over an un-annotated repo from reading as a clean PASS. @@ -128,22 +130,27 @@ def analyze(self, files: Sequence[Path], config: WardlineConfig, *, root: Path) def analyze_source(self, source: str, *, module: str, path: str = "") -> list[Finding]: """Analyze a single in-memory source string (the WP5 rule-test entry).""" tree = parse_rust(source) - findings, context = self._analyze_tree(tree, module=module, path=path) + findings, context, _ = self._analyze_tree(tree, module=module, path=path) self._last_rust_context = context return findings - def _analyze_tree(self, tree: Tree, *, module: str, path: str) -> tuple[list[Finding], RustAnalysisContext]: + def _analyze_tree(self, tree: Tree, *, module: str, path: str) -> tuple[list[Finding], RustAnalysisContext, int]: + """Run the per-file pipeline; returns ``(findings, context, callables_total)``. + + ``callables_total`` (the coverage-metric denominator) is counted over the + entity LIST, before dict-ification — the context mapping is keyed and a + pathological duplicate could collapse there. + """ nmap = mint_node_ids(tree) entities = index_entities(tree, nmap, module=module, path=path) + callables = [e for e in entities if e.kind in ("function", "method")] project_taints: dict[str, TaintState] = {} triggers: list[RustTriggerContext] = [] - for entity in entities: - if entity.kind not in ("function", "method"): - # Phase 1b: the index emits the full ten-kind surface; the taint path - # judges CALLABLES only (a module/struct/const has no body to seed or - # walk — feeding one to taint_for/dataflow would be a category error). - continue + for entity in callables: + # Phase 1b: the index emits the full ten-kind surface; the taint path + # judges CALLABLES only (a module/struct/const has no body to seed or + # walk — feeding one to taint_for/dataflow would be a category error). try: seed = self._provider.taint_for(entity.node) except ValueError: @@ -163,12 +170,17 @@ def _analyze_tree(self, tree: Tree, *, module: str, path: str) -> tuple[list[Fin context = RustAnalysisContext( triggers=tuple(triggers), project_taints=project_taints, - entities={e.qualname: e for e in entities}, + # Keyed by the kind-disambiguated FEDERATION id (`rust:{kind}:{qualname}`, + # semantic `method` mapped to id-kind `function` by entity_id itself) — a + # qualname-only key would silently drop one of `fn S` / `struct S`, whose + # qualnames legitimately collide (the per-kind twin counter never suffixes + # ACROSS kinds; the id's kind segment is what separates them). + entities={q.entity_id(e.kind, e.qualname): e for e in entities}, ) findings: list[Finding] = [] for rule in self._rules: findings.extend(rule.check(context)) - return findings, context + return findings, context, len(callables) def _relpath(file: Path, resolved_root: Path) -> str: diff --git a/src/wardline/rust/context.py b/src/wardline/rust/context.py index 53807d3b..0bbd5f56 100644 --- a/src/wardline/rust/context.py +++ b/src/wardline/rust/context.py @@ -39,6 +39,10 @@ class RustAnalysisContext: triggers: Sequence[RustTriggerContext] project_taints: Mapping[str, TaintState] # qualname -> body taint + # Keyed by the kind-disambiguated federation entity id (`rust:{kind}:{qualname}`, + # qualname.entity_id — semantic `method` maps to id-kind `function`). NOT keyed by + # bare qualname: `fn S` and `struct S` legitimately share one (the per-kind twin + # counter never suffixes across kinds), and a qualname key would drop one of them. entities: Mapping[str, RustEntity] diff --git a/src/wardline/rust/index.py b/src/wardline/rust/index.py index cd845c6a..441c8d9e 100644 --- a/src/wardline/rust/index.py +++ b/src/wardline/rust/index.py @@ -64,6 +64,14 @@ } _ITEM_TYPES = frozenset(_LEAF_KINDS) | frozenset({"mod_item", "impl_item"}) +# Comment nodes are token-stream-INVISIBLE to the oracle (syn/proc-macro2 drop them +# before extract.rs ever runs), so a comment interposed between a #[cfg] attribute +# and its item must not reset the pending-cfg accumulation. Covers `//`, `/* */`, +# AND `///` doc comments (tree-sitter parses a doc comment as a line_comment too; +# to syn it is a #[doc] attribute — either way, never a cfg). Corpus row: +# cfg_attr_comment_interposition. +_COMMENT_TYPES = frozenset({"line_comment", "block_comment"}) + @dataclass(frozen=True, slots=True) class RustEntity: @@ -125,6 +133,11 @@ def _walk_scope( items: list[tuple[Node, list[str]]] = [] pending_cfgs: list[str] = [] for child in child_nodes: + if child.type in _COMMENT_TYPES: + # Token-stream-invisible (see _COMMENT_TYPES): skip WITHOUT resetting + # pending_cfgs — a `// note` between #[cfg] and the fn must not detach + # the cfg and hand two twins the same bare colliding path. + continue if child.type == "attribute_item": pred = q.cfg_predicate_of(child) if pred is not None: @@ -159,7 +172,9 @@ def _walk_scope( # First block with a given (cfg-augmented) impl qualname emits the ONE merged impl # entity; later same-key blocks only append methods (extract.rs `seen_impl_ids`). - # Per-scope is equivalent to per-file: the qualname embeds the full module path. + # The set is PER-INVOCATION (each nested scope gets a fresh one); that is sound + # because the impl qualname embeds the full module path, so two impls in different + # scopes can never share a key — dedup only ever needs to see one scope at a time. seen_impl_quals: set[str] = set() for node, cfgs in items: diff --git a/src/wardline/rust/qualname.py b/src/wardline/rust/qualname.py index 82c48340..28760291 100644 --- a/src/wardline/rust/qualname.py +++ b/src/wardline/rust/qualname.py @@ -147,7 +147,15 @@ def cfg_discriminant(predicates: Sequence[str]) -> str: sorted (order-independent — NOT source order), joined with ``&``. Folding every predicate is what keeps stacked cfg-twins (``#[cfg(feature="a")] #[cfg(unix)]`` vs ``#[cfg(feature="b")] #[cfg(unix)]``) distinct. Applied by ``index.py`` only on a - bare-path COLLISION, exactly like the oracle's extract.rs twin counter.""" + bare-path COLLISION, exactly like the oracle's extract.rs twin counter. + + Raises ``ValueError`` on an empty input: ``@cfg()`` is never a meaningful + discriminant (the oracle's ``cfg_suffix`` guards the empty case BEFORE calling + ``cfg_discriminant``), and rendering it would silently collide every + "discriminated" twin onto one key — a caller bug, surfaced loudly.""" + if not predicates: + msg = "cfg_discriminant() requires at least one raw #[cfg] predicate" + raise ValueError(msg) return f"@cfg({'&'.join(sorted(normalize_cfg_predicate(p) for p in predicates))})" @@ -160,6 +168,13 @@ def cfg_predicate_of(attribute_item: Node) -> str | None: text): collection is raw, and ``cfg_discriminant`` is the single place predicates are normalised + reserved-char-escaped — normalising here too would double-escape (``a:b`` -> ``a%253Ab``). + + The ONE exception to "raw source span": comment nodes inside the predicate + (``#[cfg(any(unix, /* why */ windows))]``) are excised — the oracle's predicate + is ``list.tokens.to_string()`` and proc-macro2 token streams carry no comments, + so comment bytes were never part of the oracle's raw text either (corpus row + ``cfg_predicate_internal_comment``). Everything else (parens, spacing, unescaped + reserved chars) is kept verbatim. """ if not attribute_item.named_children: return None @@ -172,7 +187,37 @@ def cfg_predicate_of(attribute_item: Node) -> str | None: args = attribute.child_by_field_name("arguments") if args is None or args.text is None: return None - return args.text.decode("utf-8") + return _text_excluding_comments(args) + + +# tree-sitter comment node types — `///` doc comments parse as line_comment too. +_COMMENT_TYPES = frozenset({"line_comment", "block_comment"}) + + +def _text_excluding_comments(node: Node) -> str: + """``node``'s raw source text with every comment node's byte span excised + (the comment's SURROUNDING whitespace survives — ``normalize_cfg_predicate`` + strips all whitespace downstream, so only the comment bytes themselves matter).""" + raw = node.text or b"" + base = node.start_byte + spans: list[tuple[int, int]] = [] + stack = [node] + while stack: + current = stack.pop() + if current.type in _COMMENT_TYPES: + spans.append((current.start_byte - base, current.end_byte - base)) + continue + stack.extend(current.children) + if not spans: + return raw.decode("utf-8") + spans.sort() + kept = bytearray() + pos = 0 + for start, end in spans: + kept += raw[pos:start] + pos = end + kept += raw[pos:] + return kept.decode("utf-8") # Generic args that are NOT part of the locator key (qualname.rs trait_generic_args / diff --git a/tests/conformance/qualnames_rust.json b/tests/conformance/qualnames_rust.json index 56cba0fe..ed436c82 100644 --- a/tests/conformance/qualnames_rust.json +++ b/tests/conformance/qualnames_rust.json @@ -371,6 +371,30 @@ {"qualname": "demo.m.LIMIT@cfg(unix)", "kind": "const"}, {"qualname": "demo.m.LIMIT@cfg(windows)", "kind": "const"} ] + }, + { + "name": "cfg_attr_comment_interposition", + "crate": "demo", "module_path": "demo.m", "rel_path": "src/m.rs", + "reproducibility": "slice-1", + "note": "COMMENT INTERPOSITION between a #[cfg] attribute and its item: a `//` line comment (and, on the twin, a `///` doc comment) sits between the cfg and the fn. Comments are token-stream-invisible to syn — the cfg still attaches to the fn, so BOTH twins keep their @cfg discriminant. A `///` doc comment IS a syn attribute (#[doc = \"...\"], Meta::NameValue) but cfg_predicates filters Meta::List path=cfg only, so it contributes nothing. A producer whose attribute-to-item association resets on an interposed comment would emit a bare colliding `demo.m.f` here; this row is the conformance trip-wire.", + "source": "#[cfg(unix)]\n// platform note\npub fn f() {}\n#[cfg(windows)]\n/// doc comment\npub fn f() {}\n", + "expected": [ + {"qualname": "demo.m", "kind": "module"}, + {"qualname": "demo.m.f@cfg(unix)", "kind": "function"}, + {"qualname": "demo.m.f@cfg(windows)", "kind": "function"} + ] + }, + { + "name": "cfg_predicate_internal_comment", + "crate": "demo", "module_path": "demo.m", "rel_path": "src/m.rs", + "reproducibility": "slice-1", + "note": "COMMENT INSIDE the cfg predicate: `any(unix, /* why */ windows)`. proc-macro2 token streams carry no comments, so cfg_predicates' tokens.to_string() never sees `/* why */`; normalise_pred then strips whitespace and sorts the any() args, yielding any(unix,windows). A producer that collects the predicate as raw source TEXT between the parens would smuggle the comment bytes into the discriminant and diverge; this row pins that the predicate is the TOKEN stream, not the source span.", + "source": "#[cfg(any(unix, /* why */ windows))]\npub fn g() {}\n#[cfg(target_os = \"macos\")]\npub fn g() {}\n", + "expected": [ + {"qualname": "demo.m", "kind": "module"}, + {"qualname": "demo.m.g@cfg(any(unix,windows))", "kind": "function"}, + {"qualname": "demo.m.g@cfg(target_os=\"macos\")", "kind": "function"} + ] } ] } diff --git a/tests/conformance/test_loomweave_rust_qualname_parity.py b/tests/conformance/test_loomweave_rust_qualname_parity.py index 1c24e099..ae48b96e 100644 --- a/tests/conformance/test_loomweave_rust_qualname_parity.py +++ b/tests/conformance/test_loomweave_rust_qualname_parity.py @@ -8,12 +8,18 @@ the *second producer*, it MINTS the same locator string and never parses it. Provenance — re-vendor when Loomweave bumps the corpus: - source: loomweave rc4 @ a209fc7603b09bb06564e09c8c99390d410ea5b2 - (fixtures/qualnames_rust.json, blob 56cba0fe2d6c449ebc841c52a2800368b2e389e4, + source: loomweave rc4 @ cab95a1695a45f875933d8c4ac0e800e793c9305 + (fixtures/qualnames_rust.json, blob ed436c825861ad2b9e313f9211f5a55583b80c7c, extractor-generated, locked by crates/loomweave-plugin-rust/tests/qualname_conformance.rs) vendored byte-identical to tests/conformance/qualnames_rust.json (2026-06-10). - The a209fc7 re-vendor (rust-sp2 sprint, Task 1 upstream) adds FIVE cases: + The cab95a1 re-vendor (keystone-panel rows) adds TWO cases pinning syn's + token-stream comment semantics: ``cfg_attr_comment_interposition`` (a ``//`` or + ``///`` comment between ``#[cfg]`` and its item never detaches the cfg — both + twins keep their ``@cfg`` discriminant) and ``cfg_predicate_internal_comment`` + (a ``/* ... */`` inside the predicate is invisible to the token stream — + ``any(unix, /* why */ windows)`` renders ``any(unix,windows)``). + The prior a209fc7 re-vendor (rust-sp2 sprint, Task 1 upstream) added FIVE cases: ``generic_self_nested_param`` (the F2 nested-param trip-wire — the unit-only guard now has its corpus row), ``leaf_item_kinds`` (enum/trait/type_alias/ const/static), ``stacked_cfg_twin`` (ALL #[cfg] predicates folded — normalised, diff --git a/tests/unit/rust/test_analyzer_protocol.py b/tests/unit/rust/test_analyzer_protocol.py index 911836ea..fd95fa2a 100644 --- a/tests/unit/rust/test_analyzer_protocol.py +++ b/tests/unit/rust/test_analyzer_protocol.py @@ -120,6 +120,35 @@ def test_non_callable_entities_do_not_enter_taint_analysis(tmp_path) -> None: assert cov.properties["functions_declared"] == 1 +def test_fn_struct_same_name_keeps_both_entities_and_counts_one_callable(tmp_path) -> None: + # Keystone panel: `fn S` and `struct S` share a qualname (the per-kind twin counter + # deliberately adds no @cfg suffix across kinds), so a context map keyed on qualname + # alone silently drops one of them at dict-ification. The context must keep BOTH + # (keys are kind-disambiguated entity ids) and the coverage denominator counts the + # ONE callable — computed from the entity list, never the (collapsible) mapping. + src = ( + "struct S { x: i32 }\n" + + _TRUSTED + + 'fn S() {\n let t = std::env::var("X").unwrap();\n Command::new(t).output();\n}\n' + ) + (tmp_path / "m.rs").write_text(src, encoding="utf-8") + analyzer = RustAnalyzer() + findings = list(analyzer.analyze([tmp_path / "m.rs"], _cfg(), root=tmp_path)) + + rs = [f for f in findings if f.rule_id.startswith("RS-WL-")] + assert [f.rule_id for f in rs] == ["RS-WL-108"] # the fn IS exercised + ctx = analyzer.last_rust_context + assert ctx is not None + qual = f"{tmp_path.name}.m.S" + # BOTH entities survive, addressable by their kind-disambiguated entity-id keys. + assert f"rust:struct:{qual}" in ctx.entities + assert f"rust:function:{qual}" in ctx.entities + assert {(e.kind, e.qualname) for e in ctx.entities.values()} >= {("struct", qual), ("function", qual)} + (cov,) = [f for f in findings if f.rule_id == "WLN-RUST-COVERAGE"] + assert cov.properties["functions_total"] == 1 # one callable, not two, not zero + assert cov.properties["functions_declared"] == 1 + + def test_clean_file_yields_no_findings(tmp_path) -> None: (tmp_path / "clean.rs").write_text( _TRUSTED + 'fn run() {\n Command::new("ls").arg("-la").output();\n}\n', encoding="utf-8" diff --git a/tests/unit/rust/test_index.py b/tests/unit/rust/test_index.py index e3f8e85b..1b9cf52c 100644 --- a/tests/unit/rust/test_index.py +++ b/tests/unit/rust/test_index.py @@ -213,6 +213,49 @@ def test_per_kind_twin_suffix_applies_within_a_leaf_kind() -> None: ] +def test_line_comment_interposition_does_not_detach_cfg() -> None: + # Keystone-panel repro: a `//` comment between #[cfg] and its item. Comments are + # token-stream-invisible to the oracle (syn never sees them — extract.rs + # cfg_predicates operates on syn attributes, which attach across comments), so + # the cfg must stay pending: both twins keep their @cfg discriminant, and there + # is NO bare colliding `demo.m.f`. + src = "#[cfg(unix)]\n// comment\npub fn f() {}\n#[cfg(windows)]\npub fn f() {}\n" + rows = [(e.qualname, e.kind) for e in discover_rust_entities(src, module="demo.m")] + assert rows == [ + ("demo.m", "module"), + ("demo.m.f@cfg(unix)", "function"), + ("demo.m.f@cfg(windows)", "function"), + ] + + +def test_doc_comment_interposition_does_not_detach_cfg() -> None: + # The corpus row (cfg_attr_comment_interposition): a `///` doc comment IS a syn + # attribute (#[doc = "..."], Meta::NameValue) but NOT a cfg, so it contributes + # nothing to the discriminant — and in tree-sitter it is a plain line_comment + # sibling that must not reset the pending cfg either. + src = "#[cfg(unix)]\n// platform note\npub fn f() {}\n#[cfg(windows)]\n/// doc comment\npub fn f() {}\n" + rows = [(e.qualname, e.kind) for e in discover_rust_entities(src, module="demo.m")] + assert rows == [ + ("demo.m", "module"), + ("demo.m.f@cfg(unix)", "function"), + ("demo.m.f@cfg(windows)", "function"), + ] + + +def test_in_predicate_comment_is_token_invisible() -> None: + # The corpus row (cfg_predicate_internal_comment): a /* */ comment INSIDE the + # predicate never reaches the oracle's token stream (proc-macro2 drops comments), + # so the discriminant renders from tokens only — comment stripped, any() args + # sorted, whitespace gone. + src = '#[cfg(any(unix, /* why */ windows))]\npub fn g() {}\n#[cfg(target_os = "macos")]\npub fn g() {}\n' + rows = [(e.qualname, e.kind) for e in discover_rust_entities(src, module="demo.m")] + assert rows == [ + ("demo.m", "module"), + ("demo.m.g@cfg(any(unix,windows))", "function"), + ('demo.m.g@cfg(target_os="macos")', "function"), + ] + + def test_emission_is_deterministic() -> None: # Two runs over the same source produce byte-identical ordered emissions # (qualname, kind, parent, span) — the property the full-set ordered diff --git a/tests/unit/rust/test_provider.py b/tests/unit/rust/test_provider.py index efef2c14..9377b3d2 100644 --- a/tests/unit/rust/test_provider.py +++ b/tests/unit/rust/test_provider.py @@ -77,6 +77,34 @@ def test_marker_with_no_leading_space_still_matches() -> None: assert seed is not None and seed.body_taint is TaintState.ASSURED +def test_trusted_marker_on_impl_method_seeds_only_that_method() -> None: + # The marker works on impl METHODS, not just free fns: inside the impl body the + # doc comment is the method's preceding sibling exactly as at file scope. The + # marked method seeds its level; its unmarked sibling stays None (fail-closed). + src = ( + "struct Foo;\n" + "impl Foo {\n" + " /// @trusted(level=GUARDED)\n" + " fn marked(&self) {}\n" + " fn sibling(&self) {}\n" + "}\n" + ) + tree = parse_rust(src) + impl_node = next(c for c in tree.root_node.children if c.type == "impl_item") + body = impl_node.child_by_field_name("body") + assert body is not None + fns: dict[str, Node] = {} + for child in body.named_children: + if child.type == "function_item": + name = child.child_by_field_name("name") + assert name is not None and name.text is not None + fns[name.text.decode("utf-8")] = child + provider = RustTrustProvider() + seed = provider.taint_for(fns["marked"]) + assert seed is not None and seed.body_taint is TaintState.GUARDED + assert provider.taint_for(fns["sibling"]) is None + + def test_malformed_level_is_surfaced_not_silently_ignored() -> None: fn = _first_fn("/// @trusted(level=BOGUS)\nfn f() {}\n") with pytest.raises(ValueError, match="level"): diff --git a/tests/unit/rust/test_qualname.py b/tests/unit/rust/test_qualname.py index f5cd414c..3ac19197 100644 --- a/tests/unit/rust/test_qualname.py +++ b/tests/unit/rust/test_qualname.py @@ -15,10 +15,12 @@ pytest.importorskip("tree_sitter", reason="wardline[rust] extra not installed") from wardline.rust.index import discover_rust_entities # noqa: E402 +from wardline.rust.parse import parse_rust # noqa: E402 from wardline.rust.qualname import ( # noqa: E402 RUST_ONTOLOGY_VERSION, RUST_PLUGIN_ID, cfg_discriminant, + cfg_predicate_of, entity_id, normalize_cfg_predicate, rust_module_route, @@ -102,6 +104,35 @@ def test_cfg_discriminant_normalizes_exactly_once() -> None: assert cfg_discriminant(['feature = "a:b"']) == '@cfg(feature="a%3Ab")' +def test_cfg_discriminant_rejects_empty_input() -> None: + # An empty predicate list would render the meaningless `@cfg()` and silently + # collide every "discriminated" twin onto it — a caller bug, surfaced loudly. + with pytest.raises(ValueError, match="predicate"): + cfg_discriminant([]) + + +def test_cfg_predicate_of_returns_raw_unnormalized_text() -> None: + # The collection-is-raw invariant (mirrors extract.rs cfg_predicates): the + # returned predicate keeps its outer argument parens, source spacing, and the + # UNESCAPED reserved `:` — cfg_discriminant is the single normalisation point + # (normalising here too would double-escape `a:b` -> `a%253Ab`). + tree = parse_rust('#[cfg(feature = "a:b")]\nfn f() {}\n') + attr = next(c for c in tree.root_node.children if c.type == "attribute_item") + assert cfg_predicate_of(attr) == '(feature = "a:b")' + + +def test_cfg_predicate_of_excludes_comment_tokens() -> None: + # A /* */ comment inside the predicate is NOT part of the oracle's token stream + # (proc-macro2 drops comments before cfg_predicates ever sees them), so the raw + # collected text excludes the comment bytes — everything else stays raw + # (the comment's surrounding source whitespace survives; normalize strips it). + tree = parse_rust("#[cfg(any(unix, /* why */ windows))]\npub fn g() {}\n") + attr = next(c for c in tree.root_node.children if c.type == "attribute_item") + raw = cfg_predicate_of(attr) + assert raw == "(any(unix, windows))" + assert normalize_cfg_predicate(raw) == "any(unix,windows)" + + def test_positional_generics_are_rename_stable() -> None: # and render to the identical positional ($0) locator — a param rename # must not churn the qualname (ADR-049 De Bruijn rendering) — in BOTH the self-type From 175514b8dc519b46d91513a93e43396b94480a3c Mon Sep 17 00:00:00 2001 From: John Morrissey Date: Wed, 10 Jun 2026 10:19:34 +1000 Subject: [PATCH 084/186] =?UTF-8?q?feat(rust):=20SP2=20whole-tree=20?= =?UTF-8?q?=E2=80=94=20Cargo.toml=20crate=20roots=20(tomllib,=20two-branch?= =?UTF-8?q?=20registration,=20symlink-safe),=20real=20crate-prefixed=20rou?= =?UTF-8?q?tes,=20sp2=20conformance=20rows=20un-xfailed?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-Authored-By: Claude Fable 5 --- src/wardline/rust/analyzer.py | 40 +++- src/wardline/rust/crate_roots.py | 127 +++++++++++ src/wardline/rust/index.py | 10 +- .../test_loomweave_rust_qualname_parity.py | 14 +- tests/unit/rust/test_analyzer_protocol.py | 6 +- tests/unit/rust/test_crate_roots.py | 215 ++++++++++++++++++ 6 files changed, 391 insertions(+), 21 deletions(-) create mode 100644 src/wardline/rust/crate_roots.py create mode 100644 tests/unit/rust/test_crate_roots.py diff --git a/src/wardline/rust/analyzer.py b/src/wardline/rust/analyzer.py index 8488ea33..f4555541 100644 --- a/src/wardline/rust/analyzer.py +++ b/src/wardline/rust/analyzer.py @@ -6,9 +6,10 @@ §5; a re-parse would mint divergent NodeIds and fail quietly). WP6 adds ``analyze(files, config, *, root)`` — the engine ``Analyzer`` protocol method -``run_scan`` drives under ``--lang rust``. It reads each ``.rs`` file, derives a deterministic -slice-1 module route (``crate=root.name, src_root=root`` — full Cargo-aware routing is SP2), -runs the per-file pipeline, and surfaces a ``WLN-ENGINE-PARSE-ERROR`` FACT for any file +``run_scan`` drives under ``--lang rust``. It discovers the tree's Cargo crate roots ONCE +(SP2, ``wardline.rust.crate_roots`` — the loomweave-oracle-mirroring whole-tree pass), +routes each ``.rs`` file to its real crate-prefixed module (``_module_for``), runs the +per-file pipeline, and surfaces a ``WLN-ENGINE-PARSE-ERROR`` FACT for any file tree-sitter cannot fully parse (then contributes no findings for it — never half-analyze). ``last_context`` is the engine-shaped ``AnalysisContext | None`` (None in slice-1: the @@ -25,6 +26,7 @@ from wardline.core.taints import TaintState from wardline.rust import qualname as q from wardline.rust.context import RustAnalysisContext, RustTriggerContext +from wardline.rust.crate_roots import CrateRoots, discover_crate_roots from wardline.rust.dataflow import analyze_command_dataflow from wardline.rust.index import index_entities from wardline.rust.nodeid import mint_node_ids @@ -86,6 +88,9 @@ def analyze(self, files: Sequence[Path], config: WardlineConfig, *, root: Path) ``WLN-ENGINE-PARSE-ERROR`` FACT and no ``RS-WL-*`` findings. """ resolved_root = root.resolve() + # SP2 whole-tree pass: discover Cargo crate roots ONCE per scan; every file's + # module route resolves against this map (longest-prefix, symlink-safe walk). + crate_roots = discover_crate_roots(resolved_root) findings: list[Finding] = [] functions_total = 0 functions_declared = 0 @@ -101,7 +106,7 @@ def analyze(self, files: Sequence[Path], config: WardlineConfig, *, root: Path) if has_errors(tree): findings.append(_parse_error_finding(relpath, "tree-sitter recovered from a syntax error")) continue - module = _module_for(file, resolved_root) + module = _module_for(file, resolved_root, crate_roots) try: file_findings, context, file_callables = self._analyze_tree(tree, module=module, path=relpath) except Exception as exc: # noqa: BLE001 — per-file isolation, see below @@ -190,13 +195,30 @@ def _relpath(file: Path, resolved_root: Path) -> str: return resolved.as_posix() -def _module_for(file: Path, resolved_root: Path) -> str: - """The slice-1 module route: ``crate=root.name``, ``src_root=root`` (no ``src/`` strip). - - Deterministic and stable per function — all slice-1 needs (``provisional_identity`` - disclaims qualname/baseline stability). Cargo-aware crate/route derivation is SP2. +def _module_for(file: Path, resolved_root: Path, roots: CrateRoots) -> str: + """The SP2 module route. Three file classes: + + 1. **In-src** (under a crate root's ``src/``): the ADR-049 oracle route — + ``rust_module_route(crate=, src_root=/src, file)``. + Conformance-bearing: byte-identical to loomweave's emission for the same file. + 2. **Under a crate root but OUTSIDE its src/** (``tests/``, ``benches/``, + ``build.rs``, ...): wardline-local fallback — the owning crate's real name + + the mechanical path route from the crate dir (no ``src/`` to strip). Loomweave's + ``emittable_scope`` emits NOTHING for these files, so this qualname carries no + cross-tool conformance claim and cannot collide with a loomweave locator — + wardline scans them anyway (coverage is never narrowed to the entity surface). + 3. **Under no crate root** (a bare no-Cargo tree — the whole pre-SP2 preview + population): the pre-SP2 mechanical route, byte-unchanged — ``crate`` = scan-root + name, ``src_root`` = the scan root (no ``src/`` strip). Same no-conformance-claim + disclaimer as class 2; the preview scan population is unchanged. """ resolved = file.resolve() + crate_dir = roots.crate_dir_for(resolved) + crate_name = roots.crate_name_for(resolved) + if crate_dir is not None and crate_name is not None: + src_root = crate_dir / "src" + base = src_root if resolved.is_relative_to(src_root) else crate_dir + return q.rust_module_route(crate=crate_name, src_root=str(base), file=str(resolved)) crate = resolved_root.name or "crate" try: return q.rust_module_route(crate=crate, src_root=str(resolved_root), file=str(resolved)) diff --git a/src/wardline/rust/crate_roots.py b/src/wardline/rust/crate_roots.py new file mode 100644 index 00000000..27fd133a --- /dev/null +++ b/src/wardline/rust/crate_roots.py @@ -0,0 +1,127 @@ +"""SP2 crate-root discovery — ``Cargo.toml [package].name`` as the crate name. + +Mirrors the loomweave oracle (``crates/loomweave-plugin-rust/src/crate_roots.rs``) +exactly; that source is the contract for behaviors the corpus does not pin: + +* **Manifest read:** a real TOML parse (stdlib ``tomllib``, mirroring the oracle's + ``toml::Value`` — ADR-049's "read as text" means *not cargo-metadata*, not a + hand-rolled scan). ``[package].name`` is taken only if the manifest parses AND + the name is a string: ``name.workspace = true`` parses as a table and falls + through; unparseable TOML falls through. +* **Two-branch registration:** a dir is a crate root iff (a) its manifest yields a + string ``[package].name`` -> that name ``-``->``_`` normalised; ELSE (b) + ``src/lib.rs`` or ``src/main.rs`` exists -> the directory name normalised. A + virtual workspace root (neither) registers NOTHING — member crates own their + files outright. +* **Walk:** symlinked directories are never followed (an out-of-tree escape would + register an outside crate; a cycle would re-register through an alias); an entry + whose type cannot be determined is not recursed into (can-not-determine => + do-not-recurse). Vendored/build/store dirs the host also skips are skipped. +* **Lookup:** file -> owning crate by longest directory-prefix match. + +SCAN-COVERAGE NOTE (the distinction from loomweave's ``scope.rs emittable_scope``): +loomweave additionally EXCLUDES out-of-src files (``tests/``, ``benches/``, +``examples/``, ``build.rs``), a ``src/main.rs`` shadowed by a sibling ``lib.rs``, +and files under no crate root — it emits no federation entity for them. That is +its *entity surface*, not a scan filter: wardline keeps scanning ALL discovered +``.rs`` files. Files outside any crate's ``src/`` tree get a wardline-local +FALLBACK module route (see ``analyzer._module_for``) whose qualnames carry **no +cross-tool conformance claim** — loomweave emits nothing there, so no locator +collision is possible. +""" + +from __future__ import annotations + +import os +import tomllib +from pathlib import Path + +__all__ = ["CrateRoots", "discover_crate_roots"] + +# Vendored / build / store directories the host also skips (oracle `is_ignored`). +_IGNORED_DIRS = frozenset({"target", ".git", ".weft", "node_modules"}) + + +def _normalise(name: str) -> str: + """Underscore a crate name the way Rust does (``a-b`` -> ``a_b``).""" + return name.replace("-", "_") + + +class CrateRoots: + """Crate roots discovered under a project root: each crate's root directory + mapped to its (underscored) crate name, longest-prefix matched.""" + + def __init__(self, roots: dict[Path, str]) -> None: + # Sorted by path so longest-prefix lookup is deterministic (oracle: BTreeMap). + self._roots: list[tuple[Path, str]] = sorted(roots.items()) + + def crate_name_for(self, file: Path) -> str | None: + """The crate name owning ``file``, by longest directory-prefix match.""" + owner = self._owning_root(file) + return owner[1] if owner is not None else None + + def crate_dir_for(self, file: Path) -> Path | None: + """The crate root directory owning ``file`` (the dir holding ``Cargo.toml`` + / ``src/``), by the same longest-prefix match as ``crate_name_for``. Join + ``src`` onto this to get the source root for ``rust_module_route``.""" + owner = self._owning_root(file) + return owner[0] if owner is not None else None + + def _owning_root(self, file: Path) -> tuple[Path, str] | None: + candidates = [(d, n) for d, n in self._roots if file.is_relative_to(d)] + if not candidates: + return None + return max(candidates, key=lambda item: len(str(item[0]))) + + +def discover_crate_roots(project_root: Path) -> CrateRoots: + """Walk ``project_root`` and discover every crate root directory and its + (underscored) crate name (oracle ``discover_crate_roots``).""" + roots: dict[Path, str] = {} + _visit(project_root, roots) + return CrateRoots(roots) + + +def _package_name(manifest: Path) -> str | None: + """``[package].name`` iff ``manifest`` parses as TOML and the name is a string. + + ``name.workspace = true`` parses as a TABLE -> ``None`` (falls through to the + dir-name branch); unparseable/unreadable TOML -> ``None`` likewise. + """ + try: + with manifest.open("rb") as fh: + value = tomllib.load(fh) + except (OSError, tomllib.TOMLDecodeError): + return None + package = value.get("package") + if not isinstance(package, dict): + return None + name = package.get("name") + return name if isinstance(name, str) else None + + +def _visit(directory: Path, out: dict[Path, str]) -> None: + # Mirror the oracle's order: an unreadable dir registers nothing and is not walked. + try: + entries = list(os.scandir(directory)) + except OSError: + return + cargo = directory / "Cargo.toml" + name = _package_name(cargo) if cargo.is_file() else None + if name is not None: + out[directory] = _normalise(name) + elif ((directory / "src" / "lib.rs").is_file() or (directory / "src" / "main.rs").is_file()) and directory.name: + out.setdefault(directory, _normalise(directory.name)) + for entry in entries: + # Do NOT follow symlinked directories (oracle crate_roots.rs:83-94): a + # symlinked dir is an out-of-tree escape or a cycle. `entry.is_symlink()` + # reports the link itself; on an OSError we must not fall through to a + # follow-links check — can-not-determine => do-not-recurse. + try: + if entry.is_symlink() or not entry.is_dir(follow_symlinks=False): + continue + except OSError: + continue + if entry.name in _IGNORED_DIRS: + continue + _visit(Path(entry.path), out) diff --git a/src/wardline/rust/index.py b/src/wardline/rust/index.py index 441c8d9e..b9554004 100644 --- a/src/wardline/rust/index.py +++ b/src/wardline/rust/index.py @@ -22,8 +22,9 @@ within-kind collision. Impl blocks have their own pre-cfg twin counter keyed on the full impl segment. -This is still the single-file, file-module-rooted view: the caller supplies -``module`` (crate name from ``Cargo.toml`` + cross-file route are SP2). +This is the single-file, file-module-rooted view: the caller supplies ``module`` +(the SP2 whole-tree pass — ``Cargo.toml`` crate roots + cross-file routes — lives +in ``crate_roots.py``/``analyzer._module_for`` and feeds it in). tree-sitter types appear only under ``TYPE_CHECKING`` so importing this module never pulls the ``wardline[rust]`` extra. """ @@ -112,8 +113,9 @@ def index_entities(tree: Tree, nmap: NodeIdMap, *, module: str, path: str = "") def discover_rust_entities(source: str, *, module: str, path: str = "") -> list[RustEntity]: """Parse ``source`` and emit its entities, qualname-rooted at ``module``. - The corpus-facing API: parses internally (``module`` is the supplied file-module root, - since deriving it from ``Cargo.toml`` is SP2). ``path`` only labels ``Location``. + The corpus-facing API: parses internally (``module`` is the supplied file-module root — + the scan path derives it via ``crate_roots``/``analyzer._module_for``; corpus cases + supply it directly). ``path`` only labels ``Location``. """ tree = parse_rust(source) return index_entities(tree, mint_node_ids(tree), module=module, path=path) diff --git a/tests/conformance/test_loomweave_rust_qualname_parity.py b/tests/conformance/test_loomweave_rust_qualname_parity.py index ae48b96e..da0fcc87 100644 --- a/tests/conformance/test_loomweave_rust_qualname_parity.py +++ b/tests/conformance/test_loomweave_rust_qualname_parity.py @@ -50,7 +50,8 @@ Status: the Rust frontend (``wardline.rust.*``) now EXISTS (slice-1 WP2 landed), so the *producer-parity* tests below run live (``_rust_producer`` resolves ``wardline.rust.index`` — they no longer skip). The *structural* self-tests also run -and catch a malformed / stale re-vendor. SP2 rows still ``xfail`` (whole-tree view). +and catch a malformed / stale re-vendor. SP2 rows assert for real (the whole-tree +crate-root pass landed — ``wardline.rust.crate_roots``); no xfail tier remains. The comparison rule — GRADUATED (Phase 1b): Wardline now emits the FULL ten-kind ADR-049 surface (changeset §7 rule 1 for full-surface producers), so the gate is @@ -150,7 +151,8 @@ def _rust_producer() -> tuple[Any, Any]: Expected slice-1 surface (pinned in the plan's WP2): - ``wardline.rust.index.discover_rust_entities(source: str, *, module: str) -> Sequence[RustEntity]`` (parses internally; entities carry ``.qualname``; - deriving ``module``/crate from Cargo.toml is SP2, so the case supplies it); + the case supplies ``module`` directly — the scan path derives it from + Cargo.toml crate roots, ``wardline.rust.crate_roots``); - ``wardline.rust.qualname.rust_module_route(*, crate: str, src_root: str, file: str) -> str``. @@ -169,8 +171,7 @@ def _rust_producer() -> tuple[Any, Any]: @pytest.mark.parametrize("case", _CORPUS["entities"], ids=lambda c: c["name"]) def test_entity_qualnames(case: dict[str, Any]) -> None: rust_index, _ = _rust_producer() - if case["reproducibility"] == "sp2": # pragma: no cover - flips to hard assert at SP2 - pytest.xfail("sp2 row: needs the whole-tree view (crate name / cross-file route)") + # SP2 landed: sp2 rows assert for real alongside slice-1 (no xfail tier remains). # Phase 1b contract: the FULL ordered ten-kind emission for `source` rooted at # `module_path`, kind-mapped semantic `method` -> id-kind `function` (the one # legal projection; everything else must match the corpus byte-for-byte AND @@ -185,8 +186,9 @@ def test_entity_qualnames(case: dict[str, Any]) -> None: @pytest.mark.parametrize("route", _CORPUS["module_route"], ids=lambda r: r["name"]) def test_module_route(route: dict[str, Any]) -> None: _, rust_qualname = _rust_producer() - if route["reproducibility"] == "sp2": # pragma: no cover - flips to hard assert at SP2 - pytest.xfail("sp2 row (e.g. #[path] known gap): correct routing is a shared SP2 task") + # SP2 landed: the sp2 `path_attr_known_gap` row asserts for real — #[path] stays + # un-honoured by BOTH producers (the shared known gap), so the mechanical route IS + # the expected byte form and passes without special-casing. # WP2 contract: route a file to its dotted module given the crate + src_root. got = rust_qualname.rust_module_route(crate=route["crate"], src_root=route["src_root"], file=route["file"]) assert got == route["expected_module"] diff --git a/tests/unit/rust/test_analyzer_protocol.py b/tests/unit/rust/test_analyzer_protocol.py index fd95fa2a..868d4aac 100644 --- a/tests/unit/rust/test_analyzer_protocol.py +++ b/tests/unit/rust/test_analyzer_protocol.py @@ -41,8 +41,10 @@ def test_analyze_over_files_finds_injection_with_repo_relative_path(tmp_path) -> assert [f.rule_id for f in rs] == ["RS-WL-108"] # repo-relative POSIX path (the Filigree/Location anchor), not the absolute fs path. assert rs[0].location.path == "src/m.rs" - # dumb-but-deterministic slice-1 route: crate=root.name, src_root=root (src/ NOT - # stripped — full Cargo-aware routing is SP2; provisional_identity disclaims it). + # No Cargo.toml and no src/lib|main.rs anywhere -> no crate root registers, so the + # SP2 router falls back to the pre-SP2 mechanical route (crate=root.name, src/ NOT + # stripped) — byte-unchanged for no-Cargo trees. Real crate-prefixed routes are + # pinned in tests/unit/rust/test_crate_roots.py. assert rs[0].qualname == f"{tmp_path.name}.src.m.run" diff --git a/tests/unit/rust/test_crate_roots.py b/tests/unit/rust/test_crate_roots.py new file mode 100644 index 00000000..b4136363 --- /dev/null +++ b/tests/unit/rust/test_crate_roots.py @@ -0,0 +1,215 @@ +"""Task 4 (SP2): Cargo.toml crate-root discovery + crate-prefixed module routes. + +``wardline.rust.crate_roots`` mirrors the loomweave oracle +(``crates/loomweave-plugin-rust/src/crate_roots.rs``) exactly: + +* **Two-branch registration:** a dir is a crate root iff (a) its ``Cargo.toml`` + parses as TOML AND ``[package].name`` is a string -> that name ``-``->``_`` + normalised; ELSE (b) ``src/lib.rs`` or ``src/main.rs`` exists -> the directory + name normalised. A virtual workspace root (no package name, no src/lib|main.rs) + registers NOTHING — member crates own their files outright. +* **Walk:** symlinked directories are never followed (out-of-tree escape / cycle). +* **Lookup:** file -> crate by longest path-prefix match. + +The COVERAGE tests pin the panel's must-fix: wardline does NOT mirror loomweave's +``emittable_scope`` (scope.rs:21-32) for scan coverage — out-of-src files +(tests/, build.rs) and no-Cargo trees keep producing RS-WL findings via the +documented wardline-local fallback route; only the *route shape* differs. +""" + +from __future__ import annotations + +import os +from pathlib import Path + +import pytest + +from wardline.rust.crate_roots import discover_crate_roots + +_TRUSTED = "/// @trusted(level=ASSURED)\n" +_INJECTION = _TRUSTED + 'fn run() {\n let t = std::env::var("X").unwrap();\n Command::new(t).output();\n}\n' + + +def _write_crate(root: Path, rel: str, manifest: str, *, lib: bool = True) -> Path: + crate = root / rel + (crate / "src").mkdir(parents=True) + (crate / "Cargo.toml").write_text(manifest, encoding="utf-8") + (crate / "src" / ("lib.rs" if lib else "main.rs")).write_text("pub fn f() {}\n", encoding="utf-8") + return crate + + +# --------------------------------------------------------------------------- # +# Registration + lookup (pure discovery — mirrors the oracle's branches) +# --------------------------------------------------------------------------- # + + +def test_single_crate_registers_normalised_package_name(tmp_path: Path) -> None: + crate = _write_crate(tmp_path, "app", '[package]\nname = "my-app"\nversion = "0.1.0"\n') + roots = discover_crate_roots(tmp_path) + assert roots.crate_name_for(crate / "src" / "lib.rs") == "my_app" # - -> _ normalised + assert roots.crate_dir_for(crate / "src" / "lib.rs") == crate + + +def test_virtual_workspace_root_registers_nothing(tmp_path: Path) -> None: + # A [workspace]-only manifest with no src/lib|main.rs is NOT a crate root — + # member crates own their files outright (oracle: no package.name -> fall + # through; no src roots -> nothing registered). + (tmp_path / "Cargo.toml").write_text('[workspace]\nmembers = ["m1", "m2"]\n', encoding="utf-8") + m1 = _write_crate(tmp_path, "m1", '[package]\nname = "m-one"\n') + m2 = _write_crate(tmp_path, "m2", '[package]\nname = "m_two"\n', lib=False) + roots = discover_crate_roots(tmp_path) + assert roots.crate_dir_for(tmp_path / "stray.rs") is None # the workspace root owns nothing + assert roots.crate_name_for(m1 / "src" / "lib.rs") == "m_one" + assert roots.crate_name_for(m2 / "src" / "main.rs") == "m_two" + + +def test_nested_crates_resolve_by_longest_prefix(tmp_path: Path) -> None: + outer = _write_crate(tmp_path, "outer", '[package]\nname = "outer"\n') + inner = _write_crate(tmp_path, "outer/inner", '[package]\nname = "inner"\n', lib=False) + roots = discover_crate_roots(tmp_path) + assert roots.crate_name_for(outer / "src" / "lib.rs") == "outer" + assert roots.crate_name_for(inner / "src" / "main.rs") == "inner" # longest prefix wins + assert roots.crate_dir_for(inner / "src" / "main.rs") == inner + + +def test_name_workspace_true_with_src_falls_back_to_dir_name(tmp_path: Path) -> None: + # `name.workspace = true` parses as a TABLE, not a string -> branch (a) falls + # through; src/lib.rs exists -> branch (b) dir-name fallback (normalised). + crate = _write_crate(tmp_path, "member-a", "[package]\nname.workspace = true\n") + roots = discover_crate_roots(tmp_path) + assert roots.crate_name_for(crate / "src" / "lib.rs") == "member_a" + + +def test_package_less_manifest_without_src_is_not_a_root(tmp_path: Path) -> None: + nodir = tmp_path / "meta" + nodir.mkdir() + (nodir / "Cargo.toml").write_text('[package]\nversion = "0.1.0"\n', encoding="utf-8") # no name + roots = discover_crate_roots(tmp_path) + assert roots.crate_name_for(nodir / "x.rs") is None + assert roots.crate_dir_for(nodir / "x.rs") is None + + +def test_unparseable_manifest_falls_back_to_dir_name(tmp_path: Path) -> None: + crate = _write_crate(tmp_path, "broken", "this = = is not toml [\n") + roots = discover_crate_roots(tmp_path) + assert roots.crate_name_for(crate / "src" / "lib.rs") == "broken" + + +@pytest.mark.skipif(os.name != "posix", reason="symlinks: posix-only fixture") +def test_symlinked_external_crate_dir_is_not_registered(tmp_path: Path) -> None: + # Mirrors loomweave's does_not_register_crate_roots_reached_through_symlinked_dirs: + # a symlinked dir is an out-of-tree ESCAPE (an outside Cargo.toml would mint an + # outside crate root) or a CYCLE (re-registration under an aliased path). + proj = tmp_path / "proj" + real = _write_crate(proj, "c", '[package]\nname = "c_crate"\n') + outside = _write_crate(tmp_path, "outside", '[package]\nname = "evil_crate"\n') + (proj / "evil").symlink_to(outside) + (proj / "loop").symlink_to(proj) # the walk must RETURN, not recurse forever + + roots = discover_crate_roots(proj) + assert roots.crate_name_for(real / "src" / "lib.rs") == "c_crate" + assert roots.crate_name_for(proj / "evil" / "src" / "lib.rs") is None, ( + "out-of-tree crate reached via symlink must not be a registered root" + ) + + +# --------------------------------------------------------------------------- # +# Module routing through the analyzer (the three file classes) + coverage +# preservation. These need the tree-sitter extra (they drive RustAnalyzer). +# --------------------------------------------------------------------------- # + + +def _analyzer_module(): # noqa: ANN202 - dynamic import behind importorskip + pytest.importorskip("tree_sitter", reason="wardline[rust] extra not installed") + from wardline.rust import analyzer + + return analyzer + + +def test_in_src_files_get_the_oracle_crate_prefixed_route(tmp_path: Path) -> None: + analyzer = _analyzer_module() + crate = _write_crate(tmp_path, "app", '[package]\nname = "my-app"\n') + (crate / "src" / "a").mkdir() + (crate / "src" / "a" / "b.rs").write_text("", encoding="utf-8") + (crate / "src" / "a" / "mod.rs").write_text("", encoding="utf-8") + roots = discover_crate_roots(tmp_path) + + route = analyzer._module_for # noqa: SLF001 - the unit under test + assert route(crate / "src" / "lib.rs", tmp_path, roots) == "my_app" + assert route(crate / "src" / "a" / "b.rs", tmp_path, roots) == "my_app.a.b" + assert route(crate / "src" / "a" / "mod.rs", tmp_path, roots) == "my_app.a" + + +def test_workspace_member_files_route_to_their_own_crates(tmp_path: Path) -> None: + analyzer = _analyzer_module() + (tmp_path / "Cargo.toml").write_text('[workspace]\nmembers = ["m1", "m2"]\n', encoding="utf-8") + m1 = _write_crate(tmp_path, "m1", '[package]\nname = "m-one"\n') + m2 = _write_crate(tmp_path, "m2", '[package]\nname = "m_two"\n', lib=False) + roots = discover_crate_roots(tmp_path) + + route = analyzer._module_for # noqa: SLF001 + assert route(m1 / "src" / "lib.rs", tmp_path, roots) == "m_one" + assert route(m2 / "src" / "main.rs", tmp_path, roots) == "m_two" + + +def test_out_of_src_files_get_the_crate_named_fallback_route(tmp_path: Path) -> None: + # Class 2: under a crate root but OUTSIDE its src/ — loomweave's emittable_scope + # emits NOTHING here; wardline routes mechanically from the crate dir under the + # real crate name (documented: no cross-tool conformance claim, no collision risk). + analyzer = _analyzer_module() + crate = _write_crate(tmp_path, "c", '[package]\nname = "c-app"\n') + (crate / "tests").mkdir() + (crate / "tests" / "integration.rs").write_text("", encoding="utf-8") + (crate / "build.rs").write_text("", encoding="utf-8") + roots = discover_crate_roots(tmp_path) + + route = analyzer._module_for # noqa: SLF001 + assert route(crate / "build.rs", tmp_path, roots) == "c_app.build" + assert route(crate / "tests" / "integration.rs", tmp_path, roots) == "c_app.tests.integration" + + +def test_no_crate_files_keep_the_pre_sp2_mechanical_route(tmp_path: Path) -> None: + # Class 3: no owning crate root anywhere — today's entire preview population. + # The route is BYTE-IDENTICAL to the pre-SP2 behavior (crate = scan-root name, + # src_root = scan root, no src/ strip) so the preview scan population is unchanged. + analyzer = _analyzer_module() + (tmp_path / "src").mkdir() + (tmp_path / "src" / "m.rs").write_text("", encoding="utf-8") + roots = discover_crate_roots(tmp_path) + + assert roots.crate_dir_for(tmp_path / "src" / "m.rs") is None # no lib.rs/main.rs -> no root + got = analyzer._module_for(tmp_path / "src" / "m.rs", tmp_path, roots) # noqa: SLF001 + assert got == f"{tmp_path.name}.src.m" + + +def test_scan_coverage_is_not_narrowed_to_emittable_scope(tmp_path: Path) -> None: + # The panel's must-fix, end-to-end: build.rs, tests/, and a bare no-Cargo tree + # all still produce RS-WL findings (loomweave EXCLUDES them from its federation + # entity surface; wardline keeps scanning them via the fallback route). + analyzer = _analyzer_module() + from wardline.core.config import WardlineConfig + + crate = _write_crate(tmp_path, "c", '[package]\nname = "c-app"\n') + (crate / "src" / "cmd.rs").write_text(_INJECTION, encoding="utf-8") + (crate / "build.rs").write_text(_INJECTION, encoding="utf-8") + (crate / "tests").mkdir() + (crate / "tests" / "integration.rs").write_text(_INJECTION, encoding="utf-8") + bare = tmp_path / "bare" + bare.mkdir() + (bare / "m.rs").write_text(_INJECTION, encoding="utf-8") + + files = [crate / "src" / "cmd.rs", crate / "build.rs", crate / "tests" / "integration.rs", bare / "m.rs"] + findings = list(analyzer.RustAnalyzer().analyze(files, WardlineConfig(), root=tmp_path)) + + rs = [f for f in findings if f.rule_id == "RS-WL-108"] + assert sorted(f.location.path for f in rs) == [ + "bare/m.rs", + "c/build.rs", + "c/src/cmd.rs", + "c/tests/integration.rs", + ] + by_path = {f.location.path: f.qualname for f in rs} + assert by_path["c/src/cmd.rs"] == "c_app.cmd.run" # class 1: the oracle route + assert by_path["c/build.rs"] == "c_app.build.run" # class 2: crate-named fallback + assert by_path["c/tests/integration.rs"] == "c_app.tests.integration.run" + assert by_path["bare/m.rs"] == f"{tmp_path.name}.bare.m.run" # class 3: pre-SP2 route From 36c8adcf5312baf3458ea4d56142ad8e1106a8be Mon Sep 17 00:00:00 2001 From: John Morrissey Date: Wed, 10 Jun 2026 10:23:22 +1000 Subject: [PATCH 085/186] =?UTF-8?q?test(conformance):=20corpus=20drift=20a?= =?UTF-8?q?larm=20=E2=80=94=20upstream=20blob=20byte-pin=20+=20opt-in=20li?= =?UTF-8?q?ve=20recheck=20(closes=20wardline-868908944b)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-Authored-By: Claude Fable 5 --- pyproject.toml | 3 +- .../test_loomweave_rust_qualname_parity.py | 68 ++++++++++++++++++- 2 files changed, 69 insertions(+), 2 deletions(-) diff --git a/pyproject.toml b/pyproject.toml index a762acd1..21ee4738 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -119,13 +119,14 @@ disable_error_code = [ [tool.pytest.ini_options] testpaths = ["tests"] -addopts = "-m 'not network and not loomweave_e2e and not legis_e2e and not filigree_e2e and not rust_e2e'" +addopts = "-m 'not network and not loomweave_e2e and not legis_e2e and not filigree_e2e and not rust_e2e and not loomweave_drift'" markers = [ "network: tests that need network (live OpenRouter judge e2e — SP5)", "loomweave_e2e: tests that need a real `loomweave serve` binary (SP9 round-trip)", "legis_e2e: tests that need a real `legis` server (Track 5 intake round-trip)", "filigree_e2e: tests that need a real Filigree with the promote route (WS-A2 round-trip)", "rust_e2e: live `wardline scan --lang rust` CLI subprocess over the .rs corpus (WP6)", + "loomweave_drift: live recheck of the vendored Rust qualname corpus against the sibling loomweave checkout (release-gate drift alarm)", ] [tool.coverage.run] diff --git a/tests/conformance/test_loomweave_rust_qualname_parity.py b/tests/conformance/test_loomweave_rust_qualname_parity.py index da0fcc87..7b764e83 100644 --- a/tests/conformance/test_loomweave_rust_qualname_parity.py +++ b/tests/conformance/test_loomweave_rust_qualname_parity.py @@ -13,6 +13,25 @@ extractor-generated, locked by crates/loomweave-plugin-rust/tests/qualname_conformance.rs) vendored byte-identical to tests/conformance/qualnames_rust.json (2026-06-10). + +Drift alarm (two layers — wardline-868908944b): + 1. Byte-pin (default suite): ``UPSTREAM_BLOB_SHA`` below pins the vendored file's + git blob hash. ANY byte change to the vendored copy fails loudly — re-vendors + are deliberate, atomic, and update the constant in the same commit. + 2. Live recheck (opt-in, ``-m loomweave_drift``): byte-compares the vendored copy + against the sibling checkout (``WARDLINE_LOOMWEAVE_REPO``, default + ``/home/john/loomweave``); skips when the checkout is absent (CI). + +RE-VENDOR PROCEDURE — a RELEASE-GATE item (run ``pytest -m loomweave_drift -v`` +before every release; on drift, or on any deliberate corpus bump upstream): + 1. ``cp $WARDLINE_LOOMWEAVE_REPO/fixtures/qualnames_rust.json + tests/conformance/qualnames_rust.json`` — byte-verbatim. NEVER hand-edit the + vendored copy; the upstream extractor + its cargo gate are the only authors. + 2. Update ``UPSTREAM_BLOB_SHA`` to ``git hash-object tests/conformance/qualnames_rust.json`` + and refresh the provenance lines above (source commit + blob) — all in the + SAME commit as the new bytes. + 3. Re-run conformance (``pytest tests/conformance -q``) and CONFORM the producer + until byte-green — fix ``wardline.rust.*``, never weaken the comparison. The cab95a1 re-vendor (keystone-panel rows) adds TWO cases pinning syn's token-stream comment semantics: ``cfg_attr_comment_interposition`` (a ``//`` or ``///`` comment between ``#[cfg]`` and its item never detaches the cfg — both @@ -66,14 +85,22 @@ from __future__ import annotations +import hashlib import importlib import json +import os from pathlib import Path from typing import Any import pytest -_CORPUS: dict[str, Any] = json.loads((Path(__file__).parent / "qualnames_rust.json").read_text("utf-8")) +_CORPUS_PATH = Path(__file__).parent / "qualnames_rust.json" +_CORPUS: dict[str, Any] = json.loads(_CORPUS_PATH.read_text("utf-8")) + +# The git blob hash of the vendored corpus as committed upstream (loomweave rc4 +# @ cab95a1695a45f875933d8c4ac0e800e793c9305). Re-vendors update this constant in +# the SAME commit as the new bytes — see the RE-VENDOR PROCEDURE in the header. +UPSTREAM_BLOB_SHA = "ed436c825861ad2b9e313f9211f5a55583b80c7c" _KNOWN_TIERS = {"slice-1", "sp2"} # The a209fc7 corpus carries the FULL ten-kind ADR-049 surface (leaf_item_kinds / @@ -139,6 +166,45 @@ def test_corpus_exercises_functions() -> None: ) +# --------------------------------------------------------------------------- # +# Corpus drift alarm (wardline-868908944b) — layer 1 runs in the default suite; +# layer 2 is the opt-in live recheck against the sibling checkout. +# --------------------------------------------------------------------------- # + + +def test_vendored_corpus_matches_upstream_blob_pin() -> None: + """Layer 1: the vendored corpus byte-pins to the upstream git blob hash.""" + assert len(UPSTREAM_BLOB_SHA) == 40 and set(UPSTREAM_BLOB_SHA) <= set("0123456789abcdef"), ( + f"UPSTREAM_BLOB_SHA must be 40 lowercase hex chars (a git blob SHA-1): {UPSTREAM_BLOB_SHA!r}" + ) + data = _CORPUS_PATH.read_bytes() + actual = hashlib.sha1(b"blob %d\x00" % len(data) + data).hexdigest() + assert actual == UPSTREAM_BLOB_SHA, ( + f"the vendored corpus changed (git blob {actual}, pinned {UPSTREAM_BLOB_SHA}) — " + "if this was a deliberate re-vendor, update UPSTREAM_BLOB_SHA in the same commit " + "and re-run conformance; if not, someone edited the vendored copy (forbidden — " + "the upstream extractor is the only author; see the RE-VENDOR PROCEDURE in this " + "module's header)" + ) + + +@pytest.mark.loomweave_drift +def test_vendored_corpus_matches_live_sibling_checkout() -> None: + """Layer 2 (opt-in, ``-m loomweave_drift``): the sibling loomweave checkout's + fixture must be byte-identical to the vendored copy — the release-gate drift + alarm. Absent checkout (CI) skips; drift FAILS.""" + repo = Path(os.environ.get("WARDLINE_LOOMWEAVE_REPO", "/home/john/loomweave")) + upstream = repo / "fixtures" / "qualnames_rust.json" + if not upstream.is_file(): + pytest.skip(f"no loomweave sibling checkout at {repo} (override via WARDLINE_LOOMWEAVE_REPO)") + if upstream.read_bytes() != _CORPUS_PATH.read_bytes(): + pytest.fail( + f"upstream {upstream} has drifted from the vendored tests/conformance/qualnames_rust.json — " + "re-vendor + conform: follow the RE-VENDOR PROCEDURE in this module's header " + "(byte-verbatim copy, bump UPSTREAM_BLOB_SHA in the same commit, re-run conformance)" + ) + + # --------------------------------------------------------------------------- # # Producer-parity tests — SKIP until the Rust frontend exists (slice-1 WP2). # --------------------------------------------------------------------------- # From 226c4d2bef805a86fb1b46a4231b9bf65773c559 Mon Sep 17 00:00:00 2001 From: John Morrissey Date: Wed, 10 Jun 2026 10:33:40 +1000 Subject: [PATCH 086/186] =?UTF-8?q?feat(rust):=20anchored=20imports/implem?= =?UTF-8?q?ents=20edges=20(resolved-or-dropped,=20=C2=A76=20contract=20+?= =?UTF-8?q?=20oracle-cited=20semantics)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-Authored-By: Claude Fable 5 --- src/wardline/rust/edges.py | 366 +++++++++++++++++++++++++++++++ tests/unit/rust/test_edges.py | 400 ++++++++++++++++++++++++++++++++++ 2 files changed, 766 insertions(+) create mode 100644 src/wardline/rust/edges.py create mode 100644 tests/unit/rust/test_edges.py diff --git a/src/wardline/rust/edges.py b/src/wardline/rust/edges.py new file mode 100644 index 00000000..13586a9a --- /dev/null +++ b/src/wardline/rust/edges.py @@ -0,0 +1,366 @@ +"""Anchored Rust edges — ``imports`` + ``implements`` (changeset §6, Phase 1b). + +The shared conformance corpus is entity-only, so the contract here is changeset +``docs/integration/2026-06-09-loomweave-rust-qualname-phase1b-changeset.md`` §6 plus +the loomweave oracle source for what §6 leaves open +(``crates/loomweave-plugin-rust/src/{resolve.rs,extract.rs}``): + +* Both edge kinds are **anchored** (carry the source byte span) and therefore never + ``inferred`` confidence (ADR-026 decision 3); both are **resolved-or-dropped** — an + external or unresolvable target yields NO edge, never a dangling one (D1). +* ``imports``: one per module-scope ``use`` leaf. ``from_id`` is the ENCLOSING module + entity (a file-scope ``use`` → the file module; a ``use`` inside an inline ``mod`` → + that mod's entity; a fn-body ``use`` is not a module property and emits nothing). + Use-tree groups fan out, ``as`` aliases resolve the REAL imported path, a ``self`` + group leaf names the prefix module (extract.rs ``collect_use_leaves``). A glob + ``use a::*`` over an in-project module → ``ambiguous`` to that module entity, else + dropped (resolve.rs ``resolve_use_path``). Span = the whole ``use`` statement. +* ``implements``: one per trait-impl ENTITY whose trait resolves in-project — + merged same-key twin blocks share one impl entity and so emit exactly ONE edge + (extract.rs ``seen_impl_ids``). Trait lookup STRIPS generic args + (``trait_path_for_lookup``); a negative impl (``impl !Tr for Foo``) asserts + NON-implementation and emits nothing. Span = the implemented-trait path node only. +* Resolution (resolve.rs ``resolve_non_glob``/``resolve_ids``): attempt 1 looks the + normalized dotted path up as-is; attempt 2 (ONLY when attempt 1 found nothing AND + the original path is a BARE single segment) retries crate-root-relative — the + bare-segment gate is the H5 guard: a multi-segment miss (``serde::Serialize``) + stays dropped, never re-prefixed. 0 candidates → drop; exactly 1 → ``resolved``; + >1 (a legal value/type-namespace qualname collision) → ``ambiguous`` with ``to_id`` + = FIRST id by sorted order (deterministic, never null). +* ``crate::``/``self::``/``super::`` resolve against the module routes. Upstream 1b + maps ``self`` to the crate root and DEFERS ``super::`` to External because it does + not thread the defining module through (resolve.rs ``normalize_path``); wardline + DOES thread the enclosing module (it is the imports ``from_id``), so ``self::`` is + module-relative and ``super::`` pops module segments — the semantics that same + oracle comment names as correct (``super::a::S`` from ``c.m.n`` means ``c.m.a.S``). + A ``super::`` walking above the crate root drops. + +WIRING NOTE: edges are deliberately NOT in the analyzer/scan output surface yet. +The identity corpus (``tests/golden/identity/rust/``) captures them by calling +``index_rust_file`` + ``discover_rust_edges`` directly; runtime/federation wiring +(emitting them over the Weft wire) is future work. + +tree-sitter types appear only under ``TYPE_CHECKING`` so importing this module never +pulls the ``wardline[rust]`` extra. +""" + +from __future__ import annotations + +from dataclasses import dataclass +from typing import TYPE_CHECKING, Literal + +from wardline.rust import qualname as q +from wardline.rust.index import RustEntity, index_entities +from wardline.rust.nodeid import mint_node_ids +from wardline.rust.parse import parse_rust + +if TYPE_CHECKING: + from collections.abc import Callable, Iterable, Sequence + + from tree_sitter import Node, Tree + +__all__ = ["RustEdge", "RustParsedFile", "discover_rust_edges", "index_rust_file"] + +Confidence = Literal["resolved", "ambiguous"] +_RESOLVED: Confidence = "resolved" +_AMBIGUOUS: Confidence = "ambiguous" + +# tree-sitter node types that form a `use`/trait path; comments may interpose inside +# a use_list and are token-stream-invisible to the oracle (syn drops them). +_COMMENT_TYPES = frozenset({"line_comment", "block_comment"}) + + +@dataclass(frozen=True, slots=True) +class RustEdge: + """One anchored edge (the §6 ``RawEdge`` wire shape). ``confidence`` is only ever + ``resolved`` or ``ambiguous`` — an anchored edge may never be ``inferred``.""" + + kind: Literal["imports", "implements"] + from_id: str + to_id: str + source_byte_start: int + source_byte_end: int + confidence: Confidence + + +@dataclass(frozen=True, slots=True) +class RustParsedFile: + """One file's parse products: the tree (alive — entities' nodes point into it), + its SP2 module route (``analyzer._module_for`` / ``rust_module_route``), and the + entities ``index_entities`` emitted for it. Build via :func:`index_rust_file` + (or assemble from an existing tree/entity pass — never re-parse).""" + + tree: Tree + module: str + entities: tuple[RustEntity, ...] + + +def index_rust_file(source: str, *, module: str, path: str = "") -> RustParsedFile: + """Parse + index one file into a :class:`RustParsedFile` (ONE parse, one + ``NodeIdMap`` — the standalone helper the identity-corpus capture calls so it + gets the tree AND the entities without re-parsing).""" + tree = parse_rust(source) + entities = index_entities(tree, mint_node_ids(tree), module=module, path=path) + return RustParsedFile(tree=tree, module=module, entities=tuple(entities)) + + +def discover_rust_edges(files: Sequence[RustParsedFile]) -> list[RustEdge]: + """Discover the anchored ``imports``/``implements`` edges of a whole-tree pass. + + ``files`` are the per-file parse products of the SAME scan (one entry per ``.rs`` + file, each already parsed + indexed — see :class:`RustParsedFile`). The whole-tree + symbol table is built from the UNION of every file's entities, so cross-file + ``use crate::…`` paths resolve against the real crate-prefixed routes. Returns + edges per file in input order (each file's imports in source order, then its + implements in entity order). Resolved-or-dropped throughout. + """ + table = _symbol_table(file_entities for f in files for file_entities in f.entities) + edges: list[RustEdge] = [] + for f in files: + from_crate = f.module.split(".", 1)[0] + modules_by_node = {e.node.id: e for e in f.entities if e.kind == "module"} + file_module = modules_by_node.get(f.tree.root_node.id) + if file_module is not None: + _imports_in_scope(f.tree.root_node.children, file_module, from_crate, modules_by_node, table, edges) + for entity in f.entities: + if entity.kind == "impl": + _implements_for(entity, f.module, from_crate, table, edges) + return edges + + +# --------------------------------------------------------------------------- # +# Symbol table + resolution (mirrors resolve.rs) +# --------------------------------------------------------------------------- # + + +def _symbol_table(entities: Iterable[RustEntity]) -> dict[str, list[str]]: + """qualname -> sorted entity ids (the resolver's reverse index). Two entities may + legally share a qualname across kinds (``fn S`` / ``struct S`` — the id's kind + segment separates them); the sorted id list is what makes the multi-kind + Ambiguous target deterministic (resolve.rs ``resolve_ids``: first by sorted order).""" + table: dict[str, set[str]] = {} + for entity in entities: + table.setdefault(entity.qualname, set()).add(q.entity_id(entity.kind, entity.qualname)) + return {qualname: sorted(ids) for qualname, ids in table.items()} + + +def _normalize_path(path: str, from_crate: str, enclosing_module: str) -> str | None: + """``::``-path -> dotted qualname against the module routes, or ``None`` (drop). + + ``crate::a::B`` -> ``.a.B``; ``self::B`` -> ``.B``; + ``super::X`` pops one module segment per ``super`` (above the crate root -> + ``None``); any other leading segment is kept as-is (a real crate-qualified path + resolves, an external one misses the table -> dropped by the caller). + """ + segs = path.split("::") + if segs[0] == "crate": + return ".".join([from_crate, *segs[1:]]) + if segs[0] == "self": + return ".".join([enclosing_module, *segs[1:]]) + if segs[0] == "super": + supers = 0 + while supers < len(segs) and segs[supers] == "super": + supers += 1 + module_parts = enclosing_module.split(".") + if supers >= len(module_parts): # walked above the crate root + return None + return ".".join(module_parts[: len(module_parts) - supers] + segs[supers:]) + return ".".join(segs) + + +def _resolve_non_glob( + path: str, + from_crate: str, + enclosing_module: str, + table: dict[str, list[str]], + keep: Callable[[str], bool], +) -> tuple[Confidence, str] | None: + """resolve.rs ``resolve_non_glob`` + ``resolve_ids``: attempt 1 as-is; attempt 2 + (crate-root-relative) ONLY when attempt 1's RAW candidate slice is empty AND the + original path is a bare single segment (the H5 guard). Then 0 -> drop, 1 -> + resolved, >1 -> ambiguous(first by sorted order).""" + dotted = _normalize_path(path, from_crate, enclosing_module) + if dotted is None: + return None + ids = table.get(dotted, []) + if not ids and "::" not in path: + ids = table.get(f"{from_crate}.{dotted}", []) + matched = sorted(candidate for candidate in ids if keep(candidate)) + if not matched: + return None + if len(matched) == 1: + return (_RESOLVED, matched[0]) + return (_AMBIGUOUS, matched[0]) + + +def _resolve_use_path( + path: str, from_crate: str, enclosing_module: str, table: dict[str, list[str]] +) -> tuple[Confidence, str] | None: + """resolve.rs ``resolve_use_path``: a glob (``a::*``) over an in-project module -> + ambiguous(module id), else dropped; a non-glob path -> :func:`_resolve_non_glob` + with no kind filter.""" + if path.endswith("::*"): + dotted = _normalize_path(path[: -len("::*")], from_crate, enclosing_module) + if dotted is None: + return None + module_id = next( + (candidate for candidate in table.get(dotted, []) if candidate.startswith("rust:module:")), + None, + ) + return (_AMBIGUOUS, module_id) if module_id is not None else None + return _resolve_non_glob(path, from_crate, enclosing_module, table, lambda _candidate: True) + + +# --------------------------------------------------------------------------- # +# imports — module-scope use statements (mirrors extract.rs emit_use_edges) +# --------------------------------------------------------------------------- # + + +def _imports_in_scope( + children: Iterable[Node], + module_entity: RustEntity, + from_crate: str, + modules_by_node: dict[int, RustEntity], + table: dict[str, list[str]], + edges: list[RustEdge], +) -> None: + """Walk ONE module scope's item list: emit an edge per resolving use leaf, recurse + into inline-mod bodies under THAT mod's entity. Only module scopes are walked — + a use inside a fn/impl body is not a module property and emits nothing.""" + from_id = q.entity_id("module", module_entity.qualname) + for child in children: + if child.type == "use_declaration": + argument = child.child_by_field_name("argument") + if argument is None: + continue + leaves: list[str] = [] + _collect_use_leaves(argument, "", leaves) + for leaf in leaves: + resolution = _resolve_use_path(leaf, from_crate, module_entity.qualname, table) + if resolution is None: + continue # external / unresolvable — dropped, never dangling + confidence, to_id = resolution + # Span = the whole `use` statement (extract.rs source_range_of(it)). + edges.append(RustEdge("imports", from_id, to_id, child.start_byte, child.end_byte, confidence)) + elif child.type == "mod_item": + body = child.child_by_field_name("body") + nested = modules_by_node.get(child.id) + if body is None or nested is None: # `mod foo;` external decl / no entity + continue + _imports_in_scope(body.children, nested, from_crate, modules_by_node, table, edges) + + +def _collect_use_leaves(node: Node, prefix: str, out: list[str]) -> None: + """Flatten a use tree into ``::``-joined leaf paths (extract.rs + ``collect_use_leaves``): a Group fans out per branch under the shared prefix; a + Rename (``a::B as C``) contributes the REAL path ``a::B`` (alias dropped); a + Glob terminates a ``::*`` leaf; a ``self`` group leaf terminates the + prefix path UNCHANGED (it names the enclosing module — appending the literal + segment would miss the table and silently drop the module edge).""" + + def joined(seg: str) -> str: + return f"{prefix}::{seg}" if prefix else seg + + t = node.type + if t == "use_list": + for item in node.named_children: + if item.type not in _COMMENT_TYPES: + _collect_use_leaves(item, prefix, out) + elif t == "scoped_use_list": + path = node.child_by_field_name("path") + inner = node.child_by_field_name("list") + if inner is not None: + new_prefix = joined("::".join(_path_segments(path))) if path is not None else prefix + _collect_use_leaves(inner, new_prefix, out) + elif t == "use_as_clause": + path = node.child_by_field_name("path") + if path is not None: + out.append(joined("::".join(_path_segments(path)))) + elif t == "use_wildcard": + path = next((c for c in node.named_children if c.type not in _COMMENT_TYPES), None) + glob_prefix = joined("::".join(_path_segments(path))) if path is not None else prefix + out.append(f"{glob_prefix}::*" if glob_prefix else "*") + elif t == "self": + if prefix: # a bare `use self;` carries an empty prefix and contributes nothing + out.append(prefix) + else: # identifier / scoped_identifier / crate / super — a plain path leaf + out.append(joined("::".join(_path_segments(node)))) + + +def _path_segments(node: Node) -> list[str]: + """A (possibly scoped) path node -> its ``::`` segments, leading + ``crate``/``self``/``super`` kept verbatim for ``_normalize_path`` to map.""" + if node.type in ("scoped_identifier", "scoped_type_identifier"): + path = node.child_by_field_name("path") + name = node.child_by_field_name("name") + segments = _path_segments(path) if path is not None else [] + if name is not None: + segments.append(_text(name)) + return segments + return [_text(node)] + + +# --------------------------------------------------------------------------- # +# implements — one per trait-impl entity (mirrors extract.rs emit_impl) +# --------------------------------------------------------------------------- # + + +def _implements_for( + entity: RustEntity, + file_module: str, + from_crate: str, + table: dict[str, list[str]], + edges: list[RustEdge], +) -> None: + """Emit at most one ``implements`` edge for an ``impl`` ENTITY. The index already + merged same-key twin blocks to one entity (anchored at the FIRST block), so + per-entity emission IS the one-edge-per-impl rule (extract.rs ``seen_impl_ids``).""" + impl_node = entity.node + trait_node = impl_node.child_by_field_name("trait") + if trait_node is None: # inherent impl — nothing to implement + return + # A negative impl (`impl !Tr for Foo`) asserts NON-implementation: no edge + # (extract.rs bang guard). The `!` is a direct child between `impl` and the trait. + if any(child.type == "!" for child in impl_node.children): + return + lookup = "::".join(_trait_path_segments(trait_node)) + if not lookup: + return + # `self::`/`super::` in the trait path resolve against the impl's OWN enclosing + # module (its parent qualname — the module the index parented it to). + resolution = _resolve_non_glob( + lookup, + from_crate, + entity.parent or file_module, + table, + lambda candidate: candidate.startswith("rust:trait:"), + ) + if resolution is None: + return # external trait — dropped at emit + confidence, to_id = resolution + # Span = the implemented-trait path node ONLY (extract.rs source_range_of(trait_path)), + # generic args included (`MyTrait`) — never the whole impl block. + edges.append( + RustEdge( + "implements", + q.entity_id("impl", entity.qualname), + to_id, + trait_node.start_byte, + trait_node.end_byte, + confidence, + ) + ) + + +def _trait_path_segments(trait_node: Node) -> list[str]: + """The implemented-trait path's segments with generic args STRIPPED (extract.rs + ``trait_path_for_lookup``: the resolver keys on the trait's bare-ident qualname, + so ``impl MyTrait for Foo`` MUST look up ``MyTrait``).""" + if trait_node.type == "generic_type": + base = trait_node.child_by_field_name("type") + return _path_segments(base) if base is not None else [] + return _path_segments(trait_node) + + +def _text(node: Node) -> str: + return node.text.decode("utf-8") if node.text is not None else "" diff --git a/tests/unit/rust/test_edges.py b/tests/unit/rust/test_edges.py new file mode 100644 index 00000000..1e2195f9 --- /dev/null +++ b/tests/unit/rust/test_edges.py @@ -0,0 +1,400 @@ +"""Task 5: anchored ``imports``/``implements`` edges (changeset §6). + +The shared corpus is entity-only, so the contract for edges is changeset +``docs/integration/2026-06-09-loomweave-rust-qualname-phase1b-changeset.md`` §6 plus +the loomweave oracle source for what §6 leaves open (citations pinned per test): + +* ``resolve.rs`` — ``Resolution`` (:10-20: Ambiguous always carries a REAL id, never + null), ``resolve_use_path`` (:39-50: glob ``a::*`` → Ambiguous(in-project module id) + else External), ``resolve_non_glob`` (:106-125: attempt-1 as-is, attempt-2 crate-root + fallback ONLY for a bare single segment — a multi-segment miss stays External, the H5 + guard), ``resolve_ids`` (:127-139: 0 → External, 1 → Resolved, >1 → Ambiguous(first + by sorted order)), ``normalize_path`` (:141-168: ``crate``/``self`` map to the crate; + ``super::`` is a DELIBERATE 1b deferral upstream — see the super tests below for + wardline's plan-locked extension). +* ``extract.rs`` — ``emit_use_edges``/``collect_use_leaves`` (:600-667: groups fan out, + ``as`` aliases resolve the REAL path, a ``self`` group leaf → the prefix module, span + = the whole ``use`` statement), the negative-impl bang guard + one-edge-per-impl- + entity ``seen_impl_ids`` gate (:707-748), ``trait_path_for_lookup`` (:780-794: trait + generic args STRIPPED for lookup). + +Edges are anchored (byte spans) and resolved-or-dropped; ``confidence`` is only ever +``resolved`` or ``ambiguous`` — never ``inferred``. + +Fixtures are real ``tmp_path`` crates with a ``Cargo.toml`` so the module routes come +from the SP2 whole-tree pass (``discover_crate_roots`` + ``rust_module_route``), not a +hand-typed module string. +""" + +from __future__ import annotations + +from pathlib import Path + +import pytest + +pytest.importorskip("tree_sitter", reason="wardline[rust] extra not installed") + +from wardline.rust import qualname as q # noqa: E402 +from wardline.rust.crate_roots import discover_crate_roots # noqa: E402 +from wardline.rust.edges import ( # noqa: E402 + RustEdge, + RustParsedFile, + discover_rust_edges, + index_rust_file, +) + +_MANIFEST = '[package]\nname = "demo"\nversion = "0.1.0"\n' + + +def _parse_crate(tmp_path: Path, files: dict[str, str]) -> list[RustParsedFile]: + """Write a real crate under ``tmp_path`` and produce the per-file parse products. + + Routes are REAL: ``discover_crate_roots`` reads the ``Cargo.toml`` package name and + ``rust_module_route`` derives each file's module — the same SP2 pass the analyzer + runs, so the edge tests exercise crate-prefixed cross-file resolution end to end. + """ + crate = tmp_path / "app" + (crate / "src").mkdir(parents=True, exist_ok=True) + (crate / "Cargo.toml").write_text(_MANIFEST, encoding="utf-8") + for rel, source in files.items(): + target = crate / rel + target.parent.mkdir(parents=True, exist_ok=True) + target.write_text(source, encoding="utf-8") + roots = discover_crate_roots(tmp_path) + parsed: list[RustParsedFile] = [] + for rel in files: + file = (crate / rel).resolve() + crate_dir = roots.crate_dir_for(file) + crate_name = roots.crate_name_for(file) + assert crate_dir is not None and crate_name is not None + module = q.rust_module_route(crate=crate_name, src_root=str(crate_dir / "src"), file=str(file)) + parsed.append(index_rust_file(files[rel], module=module, path=rel)) + return parsed + + +def _edges(tmp_path: Path, files: dict[str, str]) -> list[RustEdge]: + return discover_rust_edges(_parse_crate(tmp_path, files)) + + +def _span(source: str, fragment: str) -> tuple[int, int]: + start = source.encode("utf-8").find(fragment.encode("utf-8")) + assert start >= 0, f"fixture fragment {fragment!r} not found" + return start, start + len(fragment.encode("utf-8")) + + +# --------------------------------------------------------------------------- # +# Base case: cross-file resolved imports + implements, spans pinned +# --------------------------------------------------------------------------- # + + +def test_base_two_file_resolved_imports_and_implements(tmp_path: Path) -> None: + """One ``use crate::Greet`` + one ``impl Greet for Foo`` across two files. + + §6: ``imports`` from the enclosing MODULE entity, span = the use statement; + ``implements`` from the trait-IMPL entity, span = the implemented-trait path node + only (extract.rs:746 ``source_range_of(trait_path)``). The bare ``Greet`` trait + name resolves via the crate-root-relative bare-segment fallback + (resolve.rs:106-125 attempt 2). + """ + foo_rs = "use crate::Greet;\nstruct Foo;\nimpl Greet for Foo {\n fn hi(&self) {}\n}\n" + edges = _edges( + tmp_path, + { + "src/lib.rs": "pub trait Greet {}\npub mod foo;\n", + "src/foo.rs": foo_rs, + }, + ) + imports = [e for e in edges if e.kind == "imports"] + implements = [e for e in edges if e.kind == "implements"] + assert len(imports) == 1 + assert len(implements) == 1 + + imp = imports[0] + assert imp.from_id == "rust:module:demo.foo" + assert imp.to_id == "rust:trait:demo.Greet" + assert imp.confidence == "resolved" + assert (imp.source_byte_start, imp.source_byte_end) == _span(foo_rs, "use crate::Greet;") + + impl = implements[0] + assert impl.from_id == "rust:impl:demo.foo.Foo.impl[Greet]" + assert impl.to_id == "rust:trait:demo.Greet" + assert impl.confidence == "resolved" + # span anchors the implemented-trait path node ONLY — the `Greet` of the impl + # header, not the whole impl block and not the use statement. + start, end = impl.source_byte_start, impl.source_byte_end + assert foo_rs.encode("utf-8")[start:end] == b"Greet" + assert start > foo_rs.encode("utf-8").find(b"impl ") + + +def test_inherent_impl_emits_no_implements_edge(tmp_path: Path) -> None: + edges = _edges(tmp_path, {"src/lib.rs": "struct Foo;\nimpl Foo {\n fn go(&self) {}\n}\n"}) + assert [e for e in edges if e.kind == "implements"] == [] + + +# --------------------------------------------------------------------------- # +# super:: resolution (wardline extension — plan-locked; upstream 1b defers) +# --------------------------------------------------------------------------- # + + +def test_super_and_nested_super_resolve_against_module_routes(tmp_path: Path) -> None: + """``super::X`` from ``demo.a.b`` → ``demo.a.X``; ``super::super::Y`` → ``demo.Y``. + + Upstream 1b deliberately defers ``super::`` to External (resolve.rs:141-168: the + defining-module path is not threaded through). Wardline DOES thread the enclosing + module (it is the imports ``from_id``), so it implements the semantics that same + oracle comment names as correct ("``super::a::S`` from module ``c.m.n`` means + ``c.m.a.S``") — the plan-locked Design for what §6 leaves open. + """ + edges = _edges( + tmp_path, + { + "src/lib.rs": "pub struct Y;\npub mod a;\n", + "src/a.rs": "pub struct X;\npub mod b;\n", + "src/a/b.rs": "use super::X;\nuse super::super::Y;\n", + }, + ) + imports = {e.to_id: e for e in edges if e.from_id == "rust:module:demo.a.b"} + assert set(imports) == {"rust:struct:demo.a.X", "rust:struct:demo.Y"} + assert all(e.confidence == "resolved" for e in imports.values()) + + +def test_super_past_crate_root_drops(tmp_path: Path) -> None: + # `super::` at the crate root walks above the crate — resolved-or-dropped (D1). + edges = _edges(tmp_path, {"src/lib.rs": "pub struct X;\nuse super::X;\n"}) + assert [e for e in edges if e.kind == "imports"] == [] + + +def test_self_prefix_resolves_module_relative(tmp_path: Path) -> None: + """``use self::B;`` in module ``demo.a`` → ``demo.a.B`` (module-relative). + + Interpreted §6 gap, pinned here: upstream 1b maps a leading ``self`` to the CRATE + root (resolve.rs normalize_path — it lacks the defining module), which is wrong + Rust semantics; wardline threads the enclosing module, so ``self::`` resolves + module-relative (at the crate root the two agree byte-for-byte). + """ + edges = _edges( + tmp_path, + { + "src/lib.rs": "pub mod a;\n", + "src/a.rs": "pub struct B;\nuse self::B;\n", + }, + ) + imports = [e for e in edges if e.kind == "imports"] + assert [(e.from_id, e.to_id, e.confidence) for e in imports] == [ + ("rust:module:demo.a", "rust:struct:demo.a.B", "resolved") + ] + + +# --------------------------------------------------------------------------- # +# Use-tree expansion: group fan-out, `as` alias, `self` group leaf +# --------------------------------------------------------------------------- # + + +def test_group_fanout_alias_real_path_and_self_leaf(tmp_path: Path) -> None: + """``use crate::a::{self, B, C as Zed};`` → three edges from ONE statement. + + extract.rs:621-667 ``collect_use_leaves``: a Group fans out per branch; a Rename + resolves the REAL path (``crate::a::C`` — the alias ``Zed`` is dropped); a ``self`` + group leaf terminates the PREFIX path unchanged (``crate::a`` → the module itself). + All three share the one use-statement span. + """ + lib_rs = "pub mod a;\nuse crate::a::{self, B, C as Zed};\n" + edges = _edges( + tmp_path, + { + "src/lib.rs": lib_rs, + "src/a.rs": "pub struct B;\npub struct C;\n", + }, + ) + imports = [e for e in edges if e.kind == "imports"] + assert {(e.to_id, e.confidence) for e in imports} == { + ("rust:module:demo.a", "resolved"), # the `self` leaf → the prefix module + ("rust:struct:demo.a.B", "resolved"), + ("rust:struct:demo.a.C", "resolved"), # real path, alias dropped + } + span = _span(lib_rs, "use crate::a::{self, B, C as Zed};") + assert all((e.source_byte_start, e.source_byte_end) == span for e in imports) + assert all(e.from_id == "rust:module:demo" for e in imports) + assert not any("Zed" in e.to_id for e in imports) + + +# --------------------------------------------------------------------------- # +# Globs: in-project → ambiguous(module); external → dropped +# --------------------------------------------------------------------------- # + + +def test_glob_in_project_is_ambiguous_to_the_module(tmp_path: Path) -> None: + """``use crate::a::*;`` → ONE ``ambiguous`` edge to the in-project module entity + (resolve.rs:39-50: a glob can never be promoted to Resolved from syntax alone, but + carries the REAL module id — never null).""" + edges = _edges( + tmp_path, + { + "src/lib.rs": "pub mod a;\nuse crate::a::*;\n", + "src/a.rs": "pub struct B;\npub struct C;\n", + }, + ) + imports = [e for e in edges if e.kind == "imports"] + assert [(e.to_id, e.confidence) for e in imports] == [("rust:module:demo.a", "ambiguous")] + + +def test_glob_external_is_dropped(tmp_path: Path) -> None: + edges = _edges(tmp_path, {"src/lib.rs": "use std::collections::*;\n"}) + assert [e for e in edges if e.kind == "imports"] == [] + + +# --------------------------------------------------------------------------- # +# Multi-kind ambiguity: deterministic first-by-sorted-order target +# --------------------------------------------------------------------------- # + + +def test_multi_kind_ambiguity_targets_first_id_by_sorted_order(tmp_path: Path) -> None: + """``fn S`` + ``struct S`` legally share a qualname (value vs type namespace); + ``use crate::S`` → ``ambiguous`` with ``to_id`` = FIRST entity id by sorted order + (resolve.rs:127-139 ``resolve_ids`` — ``rust:function:…`` < ``rust:struct:…``).""" + edges = _edges( + tmp_path, + { + "src/lib.rs": "pub fn S() {}\npub struct S;\npub mod b;\n", + "src/b.rs": "use crate::S;\n", + }, + ) + imports = [e for e in edges if e.kind == "imports"] + assert [(e.to_id, e.confidence) for e in imports] == [("rust:function:demo.S", "ambiguous")] + + +# --------------------------------------------------------------------------- # +# External / unresolvable → dropped (resolved-or-dropped, D1) +# --------------------------------------------------------------------------- # + + +def test_external_use_is_dropped(tmp_path: Path) -> None: + edges = _edges(tmp_path, {"src/lib.rs": "use std::fmt;\n"}) + assert [e for e in edges if e.kind == "imports"] == [] + + +def test_multi_segment_miss_never_falls_back_to_crate_root(tmp_path: Path) -> None: + """The bare-segment gate (resolve.rs:106-125): a MULTI-segment path that misses + attempt 1 stays dropped — never re-prefixed with the crate (the H5 guard: an + in-project ``mod serde`` must not capture an external ``use serde::Serialize``).""" + edges = _edges( + tmp_path, + { + "src/lib.rs": "pub mod serde;\nuse serde::Serialize;\n", + "src/serde.rs": "pub struct Serialize;\n", + }, + ) + # `serde::Serialize` attempt 1 looks up `serde.Serialize` (no crate prefix) — miss; + # multi-segment, so NO crate-root fallback: dropped, NOT `demo.serde.Serialize`. + assert [e for e in edges if e.to_id == "rust:struct:demo.serde.Serialize"] == [] + + +def test_external_trait_impl_is_dropped(tmp_path: Path) -> None: + edges = _edges( + tmp_path, + {"src/lib.rs": "use std::fmt;\nstruct Foo;\nimpl fmt::Display for Foo {}\n"}, + ) + assert [e for e in edges if e.kind == "implements"] == [] + + +# --------------------------------------------------------------------------- # +# Trait lookup: generic args stripped; negative impls emit nothing +# --------------------------------------------------------------------------- # + + +def test_generic_trait_impl_resolves_with_args_stripped(tmp_path: Path) -> None: + """``impl MyTrait for Foo`` resolves ``MyTrait`` — generic args STRIPPED for + the lookup (extract.rs:780-794 ``trait_path_for_lookup``: the in-project trait + entity is keyed on its bare ident). The span still anchors the full implemented- + trait path node (``MyTrait``, syn Path spans include the args).""" + lib_rs = "pub trait MyTrait {}\nstruct Foo;\nimpl MyTrait for Foo {}\n" + edges = _edges(tmp_path, {"src/lib.rs": lib_rs}) + implements = [e for e in edges if e.kind == "implements"] + assert [(e.from_id, e.to_id, e.confidence) for e in implements] == [ + ("rust:impl:demo.Foo.impl[MyTrait]", "rust:trait:demo.MyTrait", "resolved") + ] + assert (implements[0].source_byte_start, implements[0].source_byte_end) == _span(lib_rs, "MyTrait") + + +def test_negative_impl_emits_no_implements_edge(tmp_path: Path) -> None: + """``impl !Marker for Foo`` asserts NON-implementation → no (positive) edge + (extract.rs:729-738: the bang guard; the impl ENTITY itself is still emitted).""" + edges = _edges( + tmp_path, + {"src/lib.rs": "pub trait Marker {}\nstruct Foo;\nimpl !Marker for Foo {}\n"}, + ) + assert [e for e in edges if e.kind == "implements"] == [] + + +# --------------------------------------------------------------------------- # +# Merged twin impl blocks → exactly ONE implements edge per impl entity +# --------------------------------------------------------------------------- # + + +def test_merged_twin_impl_blocks_emit_exactly_one_edge(tmp_path: Path) -> None: + """Two same-key ``impl Greet for Foo`` blocks merge to ONE impl entity, and the + ``implements`` edge is emitted once per impl ENTITY (extract.rs:707-727 + ``seen_impl_ids`` — the second block only appends methods).""" + edges = _edges( + tmp_path, + { + "src/lib.rs": ( + "pub trait Greet {}\n" + "struct Foo;\n" + "impl Greet for Foo {\n fn a(&self) {}\n}\n" + "impl Greet for Foo {\n fn b(&self) {}\n}\n" + ) + }, + ) + implements = [e for e in edges if e.kind == "implements"] + assert [(e.from_id, e.to_id) for e in implements] == [("rust:impl:demo.Foo.impl[Greet]", "rust:trait:demo.Greet")] + + +# --------------------------------------------------------------------------- # +# Enclosing-module from_id: inline mods; non-module scopes emit nothing +# --------------------------------------------------------------------------- # + + +def test_use_inside_inline_mod_is_from_that_mod_entity(tmp_path: Path) -> None: + edges = _edges( + tmp_path, + {"src/lib.rs": "pub trait Greet {}\nmod inner {\n use crate::Greet;\n}\n"}, + ) + imports = [e for e in edges if e.kind == "imports"] + assert [(e.from_id, e.to_id) for e in imports] == [("rust:module:demo.inner", "rust:trait:demo.Greet")] + + +def test_use_inside_a_function_body_emits_nothing(tmp_path: Path) -> None: + # §6: "one per FILE-SCOPE use leaf" — the oracle walks module item lists only + # (a fn-body use is not a module property; extract.rs never visits it). + edges = _edges( + tmp_path, + {"src/lib.rs": "pub trait Greet {}\nfn f() {\n use crate::Greet;\n}\n"}, + ) + assert [e for e in edges if e.kind == "imports"] == [] + + +# --------------------------------------------------------------------------- # +# Confidence is never `inferred` (anchored edges, ADR-026 decision 3) +# --------------------------------------------------------------------------- # + + +def test_confidence_is_never_inferred(tmp_path: Path) -> None: + edges = _edges( + tmp_path, + { + "src/lib.rs": ( + "pub trait Greet {}\n" + "pub mod a;\n" + "use crate::a::*;\n" # ambiguous + "use crate::Greet;\n" # resolved + "use std::fmt;\n" # dropped + "struct Foo;\n" + "impl Greet for Foo {}\n" # resolved + ), + "src/a.rs": "pub struct B;\n", + }, + ) + assert edges, "fixture must produce edges" + assert {e.confidence for e in edges} <= {"resolved", "ambiguous"} + assert {e.kind for e in edges} == {"imports", "implements"} From a053ff267f22a50a8b06214fd5791beea44c39ae Mon Sep 17 00:00:00 2001 From: John Morrissey Date: Wed, 10 Jun 2026 10:45:26 +1000 Subject: [PATCH 087/186] =?UTF-8?q?test(identity):=20freeze=20the=20Rust?= =?UTF-8?q?=20finding-identity=20corpus=20(SP2=20completion=20gate)=20?= =?UTF-8?q?=E2=80=94=20crate-prefixed=20RS-WL-*=20identity?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-Authored-By: Claude Fable 5 --- .gitattributes | 4 + tests/golden/identity/rust/README.md | 68 +++++ tests/golden/identity/rust/__init__.py | 0 tests/golden/identity/rust/_capture.py | 145 ++++++++++ tests/golden/identity/rust/conftest.py | 41 +++ tests/golden/identity/rust/corpus/META.json | 5 + .../golden/identity/rust/corpus/rustapp.json | 254 ++++++++++++++++++ .../identity/rust/fixtures/rustapp/Cargo.toml | 6 + .../rust/fixtures/rustapp/src/cmd/mod.rs | 3 + .../rust/fixtures/rustapp/src/cmd/runner.rs | 45 ++++ .../rust/fixtures/rustapp/src/main.rs | 21 ++ tests/golden/identity/rust/regen.py | 46 ++++ .../rust/test_rust_identity_parity.py | 148 ++++++++++ 13 files changed, 786 insertions(+) create mode 100644 tests/golden/identity/rust/README.md create mode 100644 tests/golden/identity/rust/__init__.py create mode 100644 tests/golden/identity/rust/_capture.py create mode 100644 tests/golden/identity/rust/conftest.py create mode 100644 tests/golden/identity/rust/corpus/META.json create mode 100644 tests/golden/identity/rust/corpus/rustapp.json create mode 100644 tests/golden/identity/rust/fixtures/rustapp/Cargo.toml create mode 100644 tests/golden/identity/rust/fixtures/rustapp/src/cmd/mod.rs create mode 100644 tests/golden/identity/rust/fixtures/rustapp/src/cmd/runner.rs create mode 100644 tests/golden/identity/rust/fixtures/rustapp/src/main.rs create mode 100644 tests/golden/identity/rust/regen.py create mode 100644 tests/golden/identity/rust/test_rust_identity_parity.py diff --git a/.gitattributes b/.gitattributes index bd2169b5..fc180e62 100644 --- a/.gitattributes +++ b/.gitattributes @@ -3,3 +3,7 @@ # and treat the frozen corpus as binary so no autocrlf/normalization rewrites it. tests/golden/identity/fixtures/** text eol=lf tests/golden/identity/corpus/** -text +# Same discipline for the Rust oracle (tests/golden/identity/rust/): LF-pinned +# fixtures (tree-sitter byte offsets are frozen in edge spans), binary-frozen corpus. +tests/golden/identity/rust/fixtures/** text eol=lf +tests/golden/identity/rust/corpus/** -text diff --git a/tests/golden/identity/rust/README.md b/tests/golden/identity/rust/README.md new file mode 100644 index 00000000..f34cf6d0 --- /dev/null +++ b/tests/golden/identity/rust/README.md @@ -0,0 +1,68 @@ +# Rust identity parity oracle (the SP2 completion gate) + +A byte-exact golden corpus of the Rust frontend's externally-observable +**identity** — the freeze on which `RS-WL-*` findings graduated from +provisional (baseline-ineligible) to real, baseline-eligible, crate-prefixed +identity. Any byte drift here is either a real regression or a deliberate, +`--reason`-stamped rekey — exactly the parent oracle's discipline +(`tests/golden/identity/README.md`, ADR +`docs/decisions/2026-06-05-wardline-finding-identity-frozen-contract.md`). + +## What it covers — a PARTIAL mirror, by necessity + +`RustAnalyzer.last_context` is `None`: the Rust-native `RustAnalysisContext` is +**not** the Python `AnalysisContext`, so the parent oracle's SARIF code-flows, +taint facts, assure posture, and explain surfaces are *not capturable* for Rust. +The Rust identity surface, captured per fixture crate by `_capture.py`: + +- **findings** — the real wire format (`Finding.to_jsonl()`) for the + identity-bearing population (`RS-WL-* ∧ Kind.DEFECT`), produced by the REAL + analyzer path (`run_scan(root, lang="rust")`: discovery → Cargo crate roots → + module routes → per-file pipeline → suppression). +- **entities** — qualname, ADR-049 id-kind (via the `entity_id` mapping, so the + semantic `method` freezes as `function`), parent, and full span of **every** + emitted entity (the full ten-kind producer surface, `module → impl → method` + containment included). +- **edges** — every anchored `imports`/`implements` edge + (`discover_rust_edges` over the same whole-tree parse products). + +Engine diagnostics (`WLN-ENGINE-*`, `WLN-RUST-COVERAGE`, `Kind.METRIC`/`FACT`) +are excluded — same rationale as the parent oracle. + +## Inputs + +- `fixtures/rustapp/` — vendored crate (`Cargo.toml` `name = "rust-app"` → + crate `rust_app`; `src/main.rs` + `src/cmd/mod.rs` + `src/cmd/runner.rs`). + Exercises: RS-WL-108 (tainted program reaching `Command::new`, inside an impl + method), RS-WL-112 (tainted `sh -c` arg), `/// @trusted(level=ASSURED)` + markers, an inherent impl + a trait impl (`implements` edge), cross-file + `use crate::…` imports, a `#[cfg(unix)]`/`#[cfg(windows)]` twin, and the + `const`/`enum`/`trait`/`struct` leaf kinds. + +Fixtures carry **no** `.weft/` or `weft.toml` (a baseline/waiver would +date-poison the corpus via `date.today()`); `.gitattributes` pins them to LF so +tree-sitter byte offsets (frozen in edge spans) stay reproducible across OSes. + +### Reserved-colon constraint (do not "fix" this) + +The fixture contains **no path-typed generic args** (e.g. +`impl From for …`). That rendering is an **un-decided +cross-tool ADR-049 case**: today Wardline renders the `:`-bearing locator +un-gated while Loomweave rejects it at `entity_id` construction and degrades +the whole file — no canonical colon-free form exists yet (see +`docs/integration/2026-06-10-wardline-loomweave-rust-qualname-amendment-requests.md`). +Freezing such a qualname here would unilaterally pre-empt that decision. +`test_fixture_has_no_path_typed_generic_args` guards the constraint; lift it +only after the ADR-049 amendment lands (which will be a versioned rekey). + +## Determinism (verified before freezing) + +The capture was run twice in separate processes (fresh interpreter each) and +byte-compared before the corpus was committed — mirroring the parent oracle's +discipline. All sorts are content-derived (no engine-emission-order artifacts). + +## Regenerating (intentional rekey ONLY) + +```bash +cd tests && PYTHONPATH=. python -m golden.identity.rust.regen --reason "" +``` diff --git a/tests/golden/identity/rust/__init__.py b/tests/golden/identity/rust/__init__.py new file mode 100644 index 00000000..e69de29b diff --git a/tests/golden/identity/rust/_capture.py b/tests/golden/identity/rust/_capture.py new file mode 100644 index 00000000..5f0a11a9 --- /dev/null +++ b/tests/golden/identity/rust/_capture.py @@ -0,0 +1,145 @@ +"""Deterministic Rust identity-capture harness (the SP2 completion gate). + +The Rust sibling of ``golden.identity._capture`` — same canonical-JSON discipline +(stable named-array sorts, ``sort_keys``, relative paths only, no timestamps/host +data), reusing its ``to_json`` serializer and finding sort key. + +**A PARTIAL mirror by necessity:** ``RustAnalyzer.last_context`` is ``None`` — +``RustAnalysisContext`` is not the Python ``AnalysisContext`` — so the Python +oracle's SARIF code-flows, taint facts, and explain surfaces are NOT capturable +here. The Rust identity surface is: + +- **findings** — the real wire format (``Finding.to_jsonl()``) for the + identity-bearing population (``RS-WL-* ∧ Kind.DEFECT``), produced by the REAL + analyzer path (``run_scan(root, lang="rust")`` — discovery, crate roots, module + routes, per-file pipeline, suppression — exactly what a scan emits). +- **entities** — qualname, ADR-049 id-kind (via the ``entity_id`` mapping, so the + semantic ``method`` freezes as ``function``), parent, and full span of EVERY + emitted entity across the fixture crate. +- **edges** — every anchored ``imports``/``implements`` edge + (``discover_rust_edges`` over the same whole-tree parse products). + +Engine diagnostics (``WLN-ENGINE-*`` / ``WLN-RUST-COVERAGE`` / ``Kind.METRIC`` / +``Kind.FACT``) are deliberately excluded, mirroring the Python oracle's rationale. + +Imported by both ``regen.py`` (freeze) and ``test_rust_identity_parity.py`` (gate) +via ``from golden.identity.rust import _capture`` (``tests/`` is on ``sys.path``). +""" + +from __future__ import annotations + +import dataclasses +import json +from typing import TYPE_CHECKING, Any + +from golden.identity._capture import _finding_sort_key, to_json +from wardline.core import config as config_mod +from wardline.core.discovery import discover +from wardline.core.finding import Finding, Kind +from wardline.core.paths import weft_config_path +from wardline.core.run import run_scan +from wardline.rust import qualname as q +from wardline.rust.analyzer import _module_for +from wardline.rust.crate_roots import discover_crate_roots +from wardline.rust.edges import RustParsedFile, discover_rust_edges, index_rust_file + +if TYPE_CHECKING: + from pathlib import Path + +__all__ = ["capture", "is_identity_bearing", "to_json"] + + +def is_identity_bearing(f: Finding) -> bool: + """The Rust analogue of the Python oracle's predicate: ``RS-WL-* ∧ Kind.DEFECT``. + + A positive allowlist so engine diagnostics (``WLN-ENGINE-*`` FACTs, the + ``WLN-RUST-COVERAGE`` METRIC) can never silently enter the frozen corpus. + """ + return f.rule_id.startswith("RS-WL-") and f.kind is Kind.DEFECT + + +def _capture_findings(result: Any) -> list[dict[str, Any]]: + # The REAL wire format (Finding.to_jsonl), re-parsed for canonical + # re-serialization — identical discipline to the Python oracle. + recs = [json.loads(f.to_jsonl()) for f in result.findings if is_identity_bearing(f)] + return sorted(recs, key=_finding_sort_key) + + +def _parsed_files(root: Path) -> list[RustParsedFile]: + """Parse + index every discovered ``.rs`` file exactly the way the analyzer + does: same ``discover`` sweep (suffix ``.rs``, ``target/`` skipped), same + crate-root pass, same ``_module_for`` route, relative ``path`` labels only.""" + resolved_root = root.resolve() + cfg = config_mod.load(weft_config_path(resolved_root), explicit=False) + files = discover(resolved_root, cfg, confine_to_root=True, suffixes=frozenset({".rs"})) + crate_roots = discover_crate_roots(resolved_root) + parsed: list[RustParsedFile] = [] + for file in files: + source = file.read_text(encoding="utf-8") + module = _module_for(file, resolved_root, crate_roots) + relpath = file.resolve().relative_to(resolved_root).as_posix() + parsed.append(index_rust_file(source, module=module, path=relpath)) + return parsed + + +def _capture_entities(parsed: list[RustParsedFile]) -> list[dict[str, Any]]: + # EVERY emitted entity: qualname, id-kind (entity_id maps method -> function), + # parent, full span. (path, qualname, kind) is a total key — within one file a + # (kind, qualname) pair is unique (the per-kind twin counter guarantees it) — + # but keep the canonical-JSON tiebreaker anyway, mirroring the Python oracle. + rows: list[dict[str, Any]] = [] + for f in parsed: + for e in f.entities: + id_kind = q.entity_id(e.kind, e.qualname).split(":", 2)[1] + loc = e.location + rows.append( + { + "qualname": e.qualname, + "kind": id_kind, + "parent": e.parent, + "location": { + "path": loc.path, + "line_start": loc.line_start, + "line_end": loc.line_end, + "col_start": loc.col_start, + "col_end": loc.col_end, + }, + } + ) + return sorted( + rows, + key=lambda r: ( + r["location"]["path"], + r["qualname"], + r["kind"], + json.dumps(r, sort_keys=True, ensure_ascii=False), + ), + ) + + +def _capture_edges(parsed: list[RustParsedFile]) -> list[dict[str, Any]]: + # The full RustEdge field set, totally ordered by its own content (the dataclass + # fields are the whole record, so the tuple key is total). + rows = [dataclasses.asdict(e) for e in discover_rust_edges(parsed)] + return sorted( + rows, + key=lambda r: ( + r["kind"], + r["from_id"], + r["to_id"], + r["source_byte_start"], + r["source_byte_end"], + r["confidence"], + ), + ) + + +def capture(root: Path) -> dict[str, Any]: + """Capture the full Rust identity surface for one fixture crate root.""" + result = run_scan(root, lang="rust") + parsed = _parsed_files(root) + return { + "findings": _capture_findings(result), + "entities": _capture_entities(parsed), + "edges": _capture_edges(parsed), + } diff --git a/tests/golden/identity/rust/conftest.py b/tests/golden/identity/rust/conftest.py new file mode 100644 index 00000000..43e7c9e8 --- /dev/null +++ b/tests/golden/identity/rust/conftest.py @@ -0,0 +1,41 @@ +"""On a Rust parity failure, dump the freshly-captured ``actual`` to /tmp and emit +a unified-diff head (the Rust sibling of ``golden/identity/conftest.py`` — the +parent hook keys on the Python oracle's stash key and ignores these tests).""" + +from __future__ import annotations + +import difflib +from pathlib import Path + +import pytest + +_HERE = Path(__file__).parent + + +@pytest.hookimpl(hookwrapper=True) +def pytest_runtest_makereport(item: pytest.Item, call: pytest.CallInfo): # type: ignore[no-untyped-def] + outcome = yield + report = outcome.get_result() + if report.when != "call" or report.passed: + return + from golden.identity.rust.test_rust_identity_parity import _ACTUAL_KEY # type: ignore[import-not-found] + + stashed = item.stash.get(_ACTUAL_KEY, None) + if stashed is None: + return + name, actual = stashed + dump = Path("/tmp") / f"corpus_actual_rust_{name}.json" + dump.write_text(actual, encoding="utf-8") + golden_path = _HERE / "corpus" / f"{name}.json" + golden = golden_path.read_text(encoding="utf-8") if golden_path.exists() else "" + diff = "".join( + difflib.unified_diff( + golden.splitlines(keepends=True), + actual.splitlines(keepends=True), + fromfile=f"corpus/{name}.json (committed)", + tofile=f"/tmp/corpus_actual_rust_{name}.json (now)", + n=2, + ) + ) + head = "\n".join(diff.splitlines()[:60]) + report.sections.append((f"rust identity corpus diff [{name}]", head or "(no line diff; check byte/encoding)")) diff --git a/tests/golden/identity/rust/corpus/META.json b/tests/golden/identity/rust/corpus/META.json new file mode 100644 index 00000000..ca8f2089 --- /dev/null +++ b/tests/golden/identity/rust/corpus/META.json @@ -0,0 +1,5 @@ +{ + "corpus_version": 1, + "fingerprint_scheme": "wlfp2", + "reason": "initial freeze (rust-sp2-2026-06-10 Task 6): SP2 completion gate — crate-prefixed RS-WL-* identity over the vendored rust-app crate" +} diff --git a/tests/golden/identity/rust/corpus/rustapp.json b/tests/golden/identity/rust/corpus/rustapp.json new file mode 100644 index 00000000..b3f2f67b --- /dev/null +++ b/tests/golden/identity/rust/corpus/rustapp.json @@ -0,0 +1,254 @@ +{ + "edges": [ + { + "confidence": "resolved", + "from_id": "rust:impl:rust_app.cmd.runner.Runner.impl[Describe]", + "kind": "implements", + "source_byte_end": 804, + "source_byte_start": 796, + "to_id": "rust:trait:rust_app.Describe" + }, + { + "confidence": "resolved", + "from_id": "rust:module:rust_app", + "kind": "imports", + "source_byte_end": 203, + "source_byte_start": 172, + "to_id": "rust:struct:rust_app.cmd.runner.Runner" + }, + { + "confidence": "resolved", + "from_id": "rust:module:rust_app.cmd.runner", + "kind": "imports", + "source_byte_end": 393, + "source_byte_start": 373, + "to_id": "rust:trait:rust_app.Describe" + } + ], + "entities": [ + { + "kind": "module", + "location": { + "col_end": 0, + "col_start": 0, + "line_end": 4, + "line_start": 1, + "path": "src/cmd/mod.rs" + }, + "parent": null, + "qualname": "rust_app.cmd" + }, + { + "kind": "module", + "location": { + "col_end": 0, + "col_start": 0, + "line_end": 46, + "line_start": 1, + "path": "src/cmd/runner.rs" + }, + "parent": null, + "qualname": "rust_app.cmd.runner" + }, + { + "kind": "const", + "location": { + "col_end": 41, + "col_start": 0, + "line_end": 12, + "line_start": 12, + "path": "src/cmd/runner.rs" + }, + "parent": "rust_app.cmd.runner", + "qualname": "rust_app.cmd.runner.DEFAULT_TIMEOUT_SECS" + }, + { + "kind": "enum", + "location": { + "col_end": 1, + "col_start": 0, + "line_end": 17, + "line_start": 14, + "path": "src/cmd/runner.rs" + }, + "parent": "rust_app.cmd.runner", + "qualname": "rust_app.cmd.runner.Mode" + }, + { + "kind": "struct", + "location": { + "col_end": 18, + "col_start": 0, + "line_end": 19, + "line_start": 19, + "path": "src/cmd/runner.rs" + }, + "parent": "rust_app.cmd.runner", + "qualname": "rust_app.cmd.runner.Runner" + }, + { + "kind": "impl", + "location": { + "col_end": 1, + "col_start": 0, + "line_end": 29, + "line_start": 21, + "path": "src/cmd/runner.rs" + }, + "parent": "rust_app.cmd.runner", + "qualname": "rust_app.cmd.runner.Runner.impl#<>" + }, + { + "kind": "function", + "location": { + "col_end": 5, + "col_start": 4, + "line_end": 28, + "line_start": 23, + "path": "src/cmd/runner.rs" + }, + "parent": "rust_app.cmd.runner.Runner.impl#<>", + "qualname": "rust_app.cmd.runner.Runner.impl#<>.launch" + }, + { + "kind": "impl", + "location": { + "col_end": 1, + "col_start": 0, + "line_end": 35, + "line_start": 31, + "path": "src/cmd/runner.rs" + }, + "parent": "rust_app.cmd.runner", + "qualname": "rust_app.cmd.runner.Runner.impl[Describe]" + }, + { + "kind": "function", + "location": { + "col_end": 5, + "col_start": 4, + "line_end": 34, + "line_start": 32, + "path": "src/cmd/runner.rs" + }, + "parent": "rust_app.cmd.runner.Runner.impl[Describe]", + "qualname": "rust_app.cmd.runner.Runner.impl[Describe].describe" + }, + { + "kind": "function", + "location": { + "col_end": 1, + "col_start": 0, + "line_end": 40, + "line_start": 38, + "path": "src/cmd/runner.rs" + }, + "parent": "rust_app.cmd.runner", + "qualname": "rust_app.cmd.runner.shell_name@cfg(unix)" + }, + { + "kind": "function", + "location": { + "col_end": 1, + "col_start": 0, + "line_end": 45, + "line_start": 43, + "path": "src/cmd/runner.rs" + }, + "parent": "rust_app.cmd.runner", + "qualname": "rust_app.cmd.runner.shell_name@cfg(windows)" + }, + { + "kind": "module", + "location": { + "col_end": 0, + "col_start": 0, + "line_end": 22, + "line_start": 1, + "path": "src/main.rs" + }, + "parent": null, + "qualname": "rust_app" + }, + { + "kind": "trait", + "location": { + "col_end": 1, + "col_start": 0, + "line_end": 12, + "line_start": 10, + "path": "src/main.rs" + }, + "parent": "rust_app", + "qualname": "rust_app.Describe" + }, + { + "kind": "function", + "location": { + "col_end": 1, + "col_start": 0, + "line_end": 21, + "line_start": 15, + "path": "src/main.rs" + }, + "parent": "rust_app", + "qualname": "rust_app.main" + } + ], + "findings": [ + { + "confidence": null, + "fingerprint": "8eb7f8775b3756ecb5eb5047ee29433a282f169aef4caecb493596e238f167fa", + "kind": "defect", + "location": { + "col_end": null, + "col_start": null, + "line_end": 27, + "line_start": 27, + "path": "src/cmd/runner.rs" + }, + "maturity": "stable", + "message": "Untrusted data selects the program executed by Command::new (constructed at line 26, run at line 27): an attacker controls which executable runs (CWE-78).", + "properties": { + "constructor_line": 26, + "provisional_identity": true, + "taint_path": "EXTERNAL_RAW->Command::new(program)@L26->exec@L27", + "trigger_node_id": 146 + }, + "qualname": "rust_app.cmd.runner.Runner.impl#<>.launch", + "related_entities": [], + "rule_id": "RS-WL-108", + "severity": "ERROR", + "suggestion": null, + "suppression_reason": null, + "suppression_state": "active" + }, + { + "confidence": null, + "fingerprint": "b34c868714b59c4aa6be57cbef8d1e29662ba853d92dd92cd9e87067e6ac2239", + "kind": "defect", + "location": { + "col_end": null, + "col_start": null, + "line_end": 18, + "line_start": 18, + "path": "src/main.rs" + }, + "maturity": "stable", + "message": "Untrusted data reaches a shell command line ('sh -c ...', run at line 18): an attacker can inject shell syntax (CWE-78).", + "properties": { + "constructor_line": 18, + "provisional_identity": true, + "taint_path": "EXTERNAL_RAW->arg->'sh -c'->exec@L18", + "trigger_node_id": 106 + }, + "qualname": "rust_app.main", + "related_entities": [], + "rule_id": "RS-WL-112", + "severity": "WARN", + "suggestion": null, + "suppression_reason": null, + "suppression_state": "active" + } + ] +} diff --git a/tests/golden/identity/rust/fixtures/rustapp/Cargo.toml b/tests/golden/identity/rust/fixtures/rustapp/Cargo.toml new file mode 100644 index 00000000..d7d670b0 --- /dev/null +++ b/tests/golden/identity/rust/fixtures/rustapp/Cargo.toml @@ -0,0 +1,6 @@ +[package] +# The hyphenated package name is deliberate: the SP2 crate-root pass must +# normalise it to the underscored crate `rust_app` (the identity the corpus pins). +name = "rust-app" +version = "0.1.0" +edition = "2021" diff --git a/tests/golden/identity/rust/fixtures/rustapp/src/cmd/mod.rs b/tests/golden/identity/rust/fixtures/rustapp/src/cmd/mod.rs new file mode 100644 index 00000000..21adf791 --- /dev/null +++ b/tests/golden/identity/rust/fixtures/rustapp/src/cmd/mod.rs @@ -0,0 +1,3 @@ +//! `mod.rs` contributes no segment: this file is the `rust_app.cmd` module. + +pub mod runner; diff --git a/tests/golden/identity/rust/fixtures/rustapp/src/cmd/runner.rs b/tests/golden/identity/rust/fixtures/rustapp/src/cmd/runner.rs new file mode 100644 index 00000000..2f7e3fd8 --- /dev/null +++ b/tests/golden/identity/rust/fixtures/rustapp/src/cmd/runner.rs @@ -0,0 +1,45 @@ +//! Tool runner — the cross-file module route (`rust_app.cmd.runner`), the impl +//! surface, a cfg twin, and the leaf kinds the corpus freezes. +//! +//! Deliberately NO path-typed generic args anywhere (e.g. `impl From`): +//! the reserved-colon rendering is an un-decided cross-tool ADR-049 case — see the +//! corpus README. + +use std::process::Command; + +use crate::Describe; + +pub const DEFAULT_TIMEOUT_SECS: u64 = 30; + +pub enum Mode { + Direct, + Shell, +} + +pub struct Runner; + +impl Runner { + /// @trusted(level=ASSURED) + pub fn launch(&self) { + let tool = std::env::var("RUSTAPP_TOOL").unwrap(); + // RS-WL-108: untrusted data selects the program run by Command::new. + let mut command = Command::new(tool); + command.output(); + } +} + +impl Describe for Runner { + fn describe(&self) -> &'static str { + "runner" + } +} + +#[cfg(unix)] +pub fn shell_name() -> &'static str { + "sh" +} + +#[cfg(windows)] +pub fn shell_name() -> &'static str { + "cmd.exe" +} diff --git a/tests/golden/identity/rust/fixtures/rustapp/src/main.rs b/tests/golden/identity/rust/fixtures/rustapp/src/main.rs new file mode 100644 index 00000000..5872f736 --- /dev/null +++ b/tests/golden/identity/rust/fixtures/rustapp/src/main.rs @@ -0,0 +1,21 @@ +//! Vendored identity fixture — the `rust-app` crate root (`main.rs` contributes +//! no module segment: this file IS the `rust_app` module). + +use std::process::Command; + +use crate::cmd::runner::Runner; + +mod cmd; + +pub trait Describe { + fn describe(&self) -> &'static str; +} + +/// @trusted(level=ASSURED) +fn main() { + let tool = std::env::var("RUSTAPP_TOOL").unwrap(); + // RS-WL-112: untrusted data reaches a `sh -c` shell command line. + Command::new("sh").arg("-c").arg(tool).output(); + let runner = Runner; + runner.launch(); +} diff --git a/tests/golden/identity/rust/regen.py b/tests/golden/identity/rust/regen.py new file mode 100644 index 00000000..b507e478 --- /dev/null +++ b/tests/golden/identity/rust/regen.py @@ -0,0 +1,46 @@ +"""Regenerate the Rust identity golden corpus. + +Run ONLY for an intentional, versioned rekey — NEVER to paper over an accidental +drift (mirrors the Python oracle's discipline; see +``docs/decisions/2026-06-05-wardline-finding-identity-frozen-contract.md``). +Requires ``--reason`` and stamps it into ``corpus/META.json``. + +Run with ``tests/`` on the path so the ``golden.identity.rust`` package resolves: + + cd tests && PYTHONPATH=. python -m golden.identity.rust.regen --reason "" +""" + +from __future__ import annotations + +import argparse +from pathlib import Path + +from golden.identity.rust import _capture as c # type: ignore[import-not-found] +from wardline.core.finding import FINGERPRINT_SCHEME + +_HERE = Path(__file__).parent +_INPUTS = { + "rustapp": _HERE / "fixtures" / "rustapp", +} +# 1: initial freeze (SP2 completion gate) — crate-prefixed RS-WL-* identity. +CORPUS_VERSION = 1 + + +def main() -> None: + parser = argparse.ArgumentParser(description="Regenerate the Rust identity golden corpus.") + parser.add_argument("--reason", required=True, help="Why the corpus is being rekeyed (accountability record).") + args = parser.parse_args() + + out = _HERE / "corpus" + out.mkdir(exist_ok=True) + for name, root in sorted(_INPUTS.items()): + (out / f"{name}.json").write_text(c.to_json(c.capture(root)), encoding="utf-8") + (out / "META.json").write_text( + c.to_json({"corpus_version": CORPUS_VERSION, "fingerprint_scheme": FINGERPRINT_SCHEME, "reason": args.reason}), + encoding="utf-8", + ) + print(f"wrote Rust identity corpus ({len(_INPUTS)} inputs + META) to {out}") + + +if __name__ == "__main__": + main() diff --git a/tests/golden/identity/rust/test_rust_identity_parity.py b/tests/golden/identity/rust/test_rust_identity_parity.py new file mode 100644 index 00000000..99aba801 --- /dev/null +++ b/tests/golden/identity/rust/test_rust_identity_parity.py @@ -0,0 +1,148 @@ +"""Parity gate: re-running the Rust frontend over the frozen fixture crate must +reproduce the committed corpus BYTE-for-BYTE (the SP2 completion gate — RS-WL-* +finding identity graduated from provisional to baseline-eligible on this freeze). + +The Rust sibling of ``golden.identity.test_identity_parity`` — same byte-equality +discipline, same regen accountability (``--reason`` stamped into META.json), same +dump-on-failure conftest. The captured surface is the PARTIAL mirror documented in +``_capture.py``/``README.md``: findings + entities + edges (no SARIF/facts/explain — +``RustAnalysisContext`` is not the Python ``AnalysisContext``). + +Import is ``from golden.identity.rust import _capture`` — the repo puts ``tests/`` +on ``sys.path`` (pytest prepend mode). +""" + +from __future__ import annotations + +import json +from pathlib import Path + +import pytest + +pytest.importorskip("tree_sitter", reason="Rust identity capture needs wardline[rust] (tree-sitter)") + +from golden.identity.rust import _capture as c # type: ignore[import-not-found] # noqa: E402 (after importorskip) + +_HERE = Path(__file__).parent +_INPUTS = { + "rustapp": _HERE / "fixtures" / "rustapp", +} +_REGEN_HINT = ( + "Rust identity corpus drift. If this is an INTENTIONAL, versioned rekey, regenerate with:\n" + " cd tests && PYTHONPATH=. python -m golden.identity.rust.regen --reason ''\n" + "Otherwise this is a real regression — see /tmp/corpus_actual_rust_.json for the diff." +) + + +def test_corpus_meta_has_engine_scheme() -> None: + from wardline.core.finding import FINGERPRINT_SCHEME + + meta = json.loads((_HERE / "corpus" / "META.json").read_text(encoding="utf-8")) + assert meta["fingerprint_scheme"] == FINGERPRINT_SCHEME + + +@pytest.mark.parametrize("name", sorted(_INPUTS)) +def test_rust_identity_corpus_is_byte_identical(name: str, request: pytest.FixtureRequest) -> None: + root = _INPUTS[name] + golden = (_HERE / "corpus" / f"{name}.json").read_text(encoding="utf-8") + actual = c.to_json(c.capture(root)) + request.node.stash[_ACTUAL_KEY] = (name, actual) # for the conftest dump-on-failure + assert actual == golden, f"{name!r}: {_REGEN_HINT}" + + +# --- Non-vacuity (Step 6.1b): a silently-empty/shallow corpus must NOT pass --- +# Permanent structural tests over the FROZEN JSON (not the live capture), so a +# fixture or harness change that hollows out a surface fails loudly instead of +# freezing a vacuous oracle. + + +def _corpus(name: str) -> dict: + return json.loads((_HERE / "corpus" / f"{name}.json").read_text(encoding="utf-8")) + + +def test_corpus_freezes_a_finding_per_rule_with_real_fingerprints() -> None: + findings = _corpus("rustapp")["findings"] + by_rule: dict[str, list[dict]] = {} + for f in findings: + by_rule.setdefault(f["rule_id"], []).append(f) + for rule in ("RS-WL-108", "RS-WL-112"): + assert by_rule.get(rule), f"corpus freezes no {rule} finding (rules={sorted(by_rule)})" + for f in by_rule[rule]: + assert f["fingerprint"], f"{rule}: empty fingerprint frozen" + # The whole point of SP2: identity is CRATE-prefixed (Cargo.toml name = "rust-app" + # -> crate rust_app), not directory-named. + assert all(f["qualname"].startswith("rust_app.") for f in findings), ( + f"finding qualnames are not crate-prefixed: {[f['qualname'] for f in findings]}" + ) + + +def test_corpus_freezes_the_impl_entity_surface() -> None: + entities = _corpus("rustapp")["entities"] + assert any(e["kind"] == "impl" for e in entities), "no impl entity row frozen" + # The semantic `method` kind freezes as the id-kind `function`, re-parented onto + # the impl entity (module -> impl -> method containment). + methods = [e for e in entities if e["kind"] == "function" and e["parent"] and ".impl" in e["parent"]] + assert methods, "no impl-method entity row (parent = impl entity) frozen" + + +def test_corpus_freezes_a_cfg_twin() -> None: + qns = {e["qualname"] for e in _corpus("rustapp")["entities"]} + assert any("@cfg(" in qn for qn in qns), f"no @cfg(...) twin qualname frozen (qualnames={sorted(qns)})" + + +def test_corpus_freezes_the_cross_file_crate_route() -> None: + # A real crate prefix + cross-file module route (src/cmd/runner.rs -> + # rust_app.cmd.runner) — NOT a directory-name stub. + qns = {e["qualname"] for e in _corpus("rustapp")["entities"]} + assert any(qn.startswith("rust_app.cmd.runner") for qn in qns), ( + f"no rust_app.cmd.runner-prefixed entity frozen (qualnames={sorted(qns)})" + ) + + +def test_corpus_freezes_edges() -> None: + edges = _corpus("rustapp")["edges"] + assert edges, "no edges frozen" + kinds = {e["kind"] for e in edges} + assert kinds == {"imports", "implements"}, f"expected both edge kinds frozen, got {sorted(kinds)}" + for e in edges: + assert e["confidence"] in ("resolved", "ambiguous"), f"illegal confidence frozen: {e}" + + +def test_corpus_entity_spans_carry_real_coordinates() -> None: + for e in _corpus("rustapp")["entities"]: + loc = e["location"] + assert loc["path"] and not loc["path"].startswith("/"), f"non-relative path frozen: {e['qualname']!r}" + assert loc["line_start"] is not None and loc["col_start"] is not None, f"null span {e['qualname']!r}" + + +def test_corpus_fingerprints_are_collision_free() -> None: + # The join-key soundness gate, mirrored from the Python oracle: two distinct + # active findings sharing a fingerprint would silently drop one on the join. + fps = [f["fingerprint"] for f in _corpus("rustapp")["findings"]] + assert len(fps) == len(set(fps)), f"fingerprint collision in the frozen corpus: {fps}" + + +# --- Fixture hygiene: nothing date/env dependent may travel with the fixture --- + + +@pytest.mark.parametrize("name", sorted(_INPUTS)) +def test_fixture_has_no_local_config(name: str) -> None: + root = _INPUTS[name] + assert not (root / ".weft").exists(), f"{name}: fixture must not carry a .weft/ dir" + assert not (root / "weft.toml").exists(), f"{name}: fixture must not carry a weft.toml" + + +def test_fixture_has_no_path_typed_generic_args() -> None: + # Reserved-colon constraint (see README.md): `impl From`-style + # path-typed generic args are an UN-DECIDED cross-tool ADR-049 case — freezing + # one would pre-empt the pending decision. Guard the fixture against a future + # edit re-introducing one: no `impl`/`<...>` qualname may carry a `::` path. + src_files = sorted((_INPUTS["rustapp"] / "src").rglob("*.rs")) + for f in src_files: + for lineno, line in enumerate(f.read_text(encoding="utf-8").splitlines(), start=1): + stripped = line.strip() + if stripped.startswith("impl") and "<" in stripped and "::" in stripped.split("for")[0]: + pytest.fail(f"{f.name}:{lineno}: path-typed generic arg in an impl header: {stripped!r}") + + +_ACTUAL_KEY = pytest.StashKey[tuple]() From 2c541ac0c113a7b47de86964c0fd3ed142639079 Mon Sep 17 00:00:00 2001 From: John Morrissey Date: Wed, 10 Jun 2026 14:22:33 +1000 Subject: [PATCH 088/186] =?UTF-8?q?feat(rust)!:=20graduate=20RS-WL-*=20ide?= =?UTF-8?q?ntity=20=E2=80=94=20baseline-eligible,=20provisional=20plumbing?= =?UTF-8?q?=20removed?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- CHANGELOG.md | 38 +++++++-- docs/guides/agents.md | 10 ++- docs/guides/rust-preview.md | 38 +++++---- docs/reference/cli.md | 2 +- src/wardline/cli/scan.py | 12 +-- src/wardline/core/baseline.py | 10 +-- src/wardline/core/suppression.py | 10 --- src/wardline/rust/qualname.py | 8 +- src/wardline/rust/rules.py | 1 - tests/golden/identity/rust/corpus/META.json | 4 +- .../golden/identity/rust/corpus/rustapp.json | 2 - tests/golden/identity/rust/regen.py | 4 +- tests/unit/cli/test_scan_rust.py | 11 +-- tests/unit/rust/test_provisional_identity.py | 59 -------------- .../unit/rust/test_rust_identity_graduated.py | 79 +++++++++++++++++++ 15 files changed, 162 insertions(+), 126 deletions(-) delete mode 100644 tests/unit/rust/test_provisional_identity.py create mode 100644 tests/unit/rust/test_rust_identity_graduated.py diff --git a/CHANGELOG.md b/CHANGELOG.md index 5039cca8..6997ec2b 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -8,18 +8,44 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] ### Added -- **Rust support (preview).** A new `--lang rust` frontend (behind the +- **Rust support.** A new `--lang rust` frontend (behind the `wardline[rust]` extra: tree-sitter, no base dependency) sweeps `*.rs` and flags command-injection trust-boundary defects — `RS-WL-108` (program injection, ERROR: untrusted data chooses the executable of `std::process::Command`) and `RS-WL-112` (shell injection, WARN: untrusted data reaches a `sh -c` command line). Trust is declared with a `/// @trusted(level=ASSURED|GUARDED)` doc-comment marker; analysis is default-clean and reuses the Python engine's taint lattice, severity modulation, - and finding/gate machinery. Findings carry **provisional identity** (baseline- - ineligible) and `weft.toml` severity overrides do not yet apply; a `.rs` file that - does not fully parse is surfaced as `WLN-ENGINE-PARSE-ERROR` and never half-analyzed. - The Python default path is byte-identical (identity oracle green). See the - [Rust support guide](docs/guides/rust-preview.md). + and finding/gate machinery. A `.rs` file that does not fully parse is surfaced as + `WLN-ENGINE-PARSE-ERROR` and never half-analyzed. The Python default path is + byte-identical (identity oracle green). Rule coverage is the command-injection + slice and `weft.toml` severity overrides do not yet apply to Rust findings. See + the [Rust support guide](docs/guides/rust-preview.md). +- **Rust finding identity is graduated — RS-WL-* findings are baseline-eligible.** + The whole-tree SP2 pass reads the real crate name from `Cargo.toml` + (`[package].name`, `-`→`_`, two-branch crate-root registration mirroring the + Loomweave extractor, symlink-safe) and routes cross-file modules, so every + qualname/fingerprint carries its real crate prefix. Identity is frozen by a new + byte-exact golden corpus (`tests/golden/identity/rust/`, the SP2 completion + gate); the pre-SP2 `provisional_identity` plumbing (never-baseline-match / + never-baseline-capture) is removed — baseline, waivers, and judged verdicts now + apply to Rust findings exactly as for Python. (Pre-graduation RS-WL-* + fingerprints change once; they were never baseline-eligible, so no migration.) +- **Rust frontend is a full ADR-049 producer (Loomweave Phase 1b).** The entity + surface grows from callables-only to the full ten-kind contract set — + `enum`/`trait`/`type_alias`/`const`/`static`/`macro` leaf entities, the `impl` + block as its own entity with `module → impl → method` containment, per-kind + `@cfg` twin discrimination (stacked `#[cfg]` attributes fold sorted-`&`-joined; + reserved chars escape `%`→`%25`, `:`→`%3A`; comments are token-stream-invisible) + — plus the two anchored edge kinds (`imports`, `implements`; resolved-or-dropped, + never `inferred`), under `plugin_id rust` / `ontology_version 0.4.0`. Conformance + against the Loomweave-hosted corpus graduates from the subset-consumer rule to + the full-set ordered byte-for-byte rule, with eight new oracle rows vendored + upstream this sprint and a corpus **drift alarm** (upstream blob byte-pin in the + default suite + an opt-in `loomweave_drift` live recheck against a sibling + checkout). The path-typed-generic-arg reserved-colon case and const-generic-arg + spacing remain a pending cross-tool ADR-049 decision (drafted amendment-request + letter in `docs/integration/`); the frozen identity corpus deliberately avoids + both shapes. - **Gate verdict is now explicit (no vacuous green).** `GateDecision` carries a `verdict` (`NOT_EVALUATED` / `PASSED` / `FAILED`) and a `would_trip_at` (the highest severity that would trip on the evaluated population, or null). A bare diff --git a/docs/guides/agents.md b/docs/guides/agents.md index 3783e991..cd16bdf8 100644 --- a/docs/guides/agents.md +++ b/docs/guides/agents.md @@ -285,14 +285,16 @@ versioned handoff shape (`wardline-agent-summary-1`) to disk: active defects first with fingerprints and next tool calls, plus suppressed findings, engine facts, and Loomweave/Filigree write status when configured. -## Scanning Rust (preview) +## Scanning Rust For a Rust codebase, add `--lang rust` (install the `wardline[rust]` extra first). It sweeps `*.rs` and flags command-injection defects (`RS-WL-108` program injection / `RS-WL-112` shell injection) through the same gate, formats, and -emission paths as the Python frontend. Treat the result as a **signal, not a -contract**: Rust findings carry provisional identity (baseline-ineligible) and -`weft.toml` severity overrides do not apply yet. Declare a function's trust tier +emission paths as the Python frontend. Rust finding identity is frozen and +crate-prefixed, so RS-WL-* findings are **baseline-eligible** — baseline, waivers, +and judged verdicts apply exactly as for Python findings. Read the result at the +right scope: rule coverage is the command-injection slice, and `weft.toml` +severity overrides do not yet apply. Declare a function's trust tier with a `/// @trusted(level=ASSURED|GUARDED)` doc-comment marker so the default-clean analysis knows which functions are part of your trust surface. See the [Rust support guide](rust-preview.md) for the boundary sources, the trust diff --git a/docs/guides/rust-preview.md b/docs/guides/rust-preview.md index 57b5b6e2..818f93a6 100644 --- a/docs/guides/rust-preview.md +++ b/docs/guides/rust-preview.md @@ -1,20 +1,23 @@ -# Rust support (preview) +# Rust support -Wardline ships a **preview** language frontend for Rust. Point it at a tree of +Wardline ships a language frontend for Rust. Point it at a tree of `.rs` files and it flags command-injection trust-boundary defects — untrusted data reaching the program or shell command line of `std::process::Command`. -!!! warning "Preview — provisional identity, narrow scope" - The Rust frontend is an early slice. Its findings (`RS-WL-108` / `RS-WL-112`) - carry **provisional identity**: their `qualname`/fingerprint is not yet stable - across releases. They are **baseline-ineligible** — and this is *enforced*, not - just advised: the engine never matches them against a committed - baseline/waiver/judged entry and never writes them into a generated baseline, so - a stale committed suppression can never silently clear one. (A `--new-since` - delta-scope still scopes them like any other defect; that scoping is computed per - run, never persisted.) `weft.toml` severity overrides do **not** apply to Rust - rules yet, and the frontend is **CLI-only** — the MCP `scan` tool analyzes - Python. Treat a Rust scan as a signal, not a contract. +!!! note "Identity graduated — RS-WL-* findings are baseline-eligible" + Rust finding identity is **frozen**: qualnames are real crate-prefixed module + routes (`Cargo.toml`-aware, whole-tree) and fingerprints are stable across + releases, gated byte-for-byte by the frozen identity corpus in + `tests/golden/identity/rust/`. RS-WL-* findings participate fully in the + suppression machinery — they match committed baseline/waiver/judged entries + and are captured by `wardline baseline` like any Python finding. + +!!! warning "Scope — command-injection slice; no config severity overrides yet" + Rule coverage is the **command-injection slice** (`RS-WL-108` / `RS-WL-112`) — + a Rust scan says nothing about other defect families. `weft.toml` severity + overrides do **not** apply to Rust rules yet (they carry hardcoded base + severities), and the frontend is **CLI-only** — the MCP `scan` tool analyzes + Python. ## Running it @@ -89,7 +92,7 @@ terminators. ## Known limitations (this slice) The frontend reports **provable** taint, not fail-closed unknowns. The following -are **known false-negative families** — deliberately out of scope for the preview +are **known false-negative families** — deliberately out of scope for this slice, documented so you do not mistake silence for safety: - **Iterator extraction of `args`/`vars`.** `env::args()`/`env::vars()` are in the @@ -108,9 +111,10 @@ slice, documented so you do not mistake silence for safety: - **Closures and nested `fn`s.** A finding inside a closure or nested function attributes to the enclosing named function by line; the inner scope is not walked separately. -- **Module routing is path-based, not Cargo-aware.** Qualnames are rooted at the - scan directory name with no `Cargo.toml`/`#[path]` resolution — another reason - the identity is provisional. +- **`#[path]` module attributes are not honoured.** Module routing is + `Cargo.toml`-aware (real crate names, `src/` roots, workspace members, + longest-prefix nesting), but a `#[path = "..."]` override is routed + mechanically by file path — a known gap shared with the Loomweave extractor. A `.rs` file that tree-sitter cannot fully parse is **not** half-analyzed: it is surfaced as a `WLN-ENGINE-PARSE-ERROR` fact, counts toward the "could not be diff --git a/docs/reference/cli.md b/docs/reference/cli.md index b52c825a..d7048170 100644 --- a/docs/reference/cli.md +++ b/docs/reference/cli.md @@ -94,7 +94,7 @@ it at a package root, not a single file. | --- | --- | | `--config FILE` | Path to a `weft.toml` config file; Wardline reads its `[wardline]` table for rule enable/severity and judge settings (defaults to `weft.toml` in the scan path). | | `--format [jsonl\|sarif\|agent-summary\|legis]` | Output shape. `jsonl` is one finding per line; `sarif` is SARIF 2.1.0 for GitHub code-scanning and other generic SARIF consumers; `agent-summary` is stable versioned JSON for agents (`schema: wardline-agent-summary-1`) with active defects first, suppressed findings, engine facts, integration status, and suggested next tool calls; `legis` is the signed, verbatim-postable `scan` for legis's `POST /wardline/scan-results` (signed when `WARDLINE_LEGIS_ARTIFACT_KEY` is provisioned — write it **outside** the working tree, see the [legis handoff guide](../guides/legis-handoff.md)). SARIF carries Wardline identity in `partialFingerprints["wardlineFingerprint/v2"]`; downstream Filigree lifecycle quality depends on importers preserving that field. | -| `--lang [python\|rust]` | Language frontend (default `python`). `rust` is a **preview** that sweeps `*.rs` for command-injection findings (`RS-WL-108`/`RS-WL-112`); needs the `wardline[rust]` extra. Findings carry provisional identity (baseline-ineligible) and config severity overrides do not apply — see the [Rust support guide](../guides/rust-preview.md). | +| `--lang [python\|rust]` | Language frontend (default `python`). `rust` sweeps `*.rs` and covers the **command-injection slice** (`RS-WL-108`/`RS-WL-112`); needs the `wardline[rust]` extra. Finding identity is frozen and crate-prefixed (baseline-eligible); config severity overrides do not yet apply to Rust findings — see the [Rust support guide](../guides/rust-preview.md). | | `--output PATH` | Write findings to a file instead of stdout. | | `--fail-on [CRITICAL\|ERROR\|WARN\|INFO]` | Exit non-zero when any finding at or above this severity survives the baseline. Use this as your CI gate. | | `--cache-dir PATH` | Persist the L3 inter-procedural summary cache here so the next scan reuses unchanged summaries. | diff --git a/src/wardline/cli/scan.py b/src/wardline/cli/scan.py index 9a40873b..da038817 100644 --- a/src/wardline/cli/scan.py +++ b/src/wardline/cli/scan.py @@ -159,13 +159,13 @@ def scan( ) -> None: """Scan PATH for findings.""" if lang == "rust": - # Loud posture banner: the Rust frontend is a preview slice. RS-WL-* findings carry - # provisional identity (baseline-ineligible until SP2) and weft.toml severity - # overrides do not yet apply to them. Surface this so a green/red gate is not - # mistaken for the stability of the Python path. + # Posture banner: RS-WL-* identity is graduated (frozen, baseline-eligible) but + # rule coverage is the command-injection slice and weft.toml severity overrides + # do not yet apply to Rust findings (analyzer accepts config for protocol parity + # only). Surface the remaining gaps so a green gate is read at the right scope. click.echo( - "note: --lang rust is PREVIEW — RS-WL-* findings have provisional identity " - "(baseline-ineligible) and config severity overrides do not apply.", + "note: --lang rust covers the command-injection slice (RS-WL-108/112); " + "config severity overrides do not yet apply to Rust findings.", err=True, ) if fmt == "sarif": diff --git a/src/wardline/core/baseline.py b/src/wardline/core/baseline.py index c20c06de..86f97474 100644 --- a/src/wardline/core/baseline.py +++ b/src/wardline/core/baseline.py @@ -50,17 +50,9 @@ def contains(self, fingerprint: str) -> bool: def build_baseline_document(findings: Iterable[Finding]) -> dict[str, Any]: - """Pure: the YAML-shaped dict for the given findings (deduped, severity-sorted). - - Provisional-identity findings (preview rules whose fingerprint is not yet stable — - e.g. the Rust RS-WL-* frontend) are EXCLUDED: writing them to a baseline would pin a - suppression to an identity that shifts in a later slice. They are baseline-ineligible - by contract (the suppression path keeps them ACTIVE), so they must never be captured. - """ + """Pure: the YAML-shaped dict for the given findings (deduped, severity-sorted).""" unique: dict[str, Finding] = {} for f in findings: - if f.properties.get("provisional_identity") is True: - continue unique.setdefault(f.fingerprint, f) ordered = sorted( unique.values(), diff --git a/src/wardline/core/suppression.py b/src/wardline/core/suppression.py index f2709e3a..44a2c228 100644 --- a/src/wardline/core/suppression.py +++ b/src/wardline/core/suppression.py @@ -65,16 +65,6 @@ def apply_suppressions( ) ) continue - # Provisional-identity findings (preview rules whose fingerprint will shift in a - # later slice — e.g. the Rust RS-WL-* frontend) are baseline-INELIGIBLE by contract: - # a fingerprint-keyed baseline/waiver/judged entry pinned to them now would silently - # orphan when the identity changes. Never match them — they stay ACTIVE, so they - # always gate and a committed suppression can never (even under --trust-suppressions) - # clear them. This is what the CLI's "provisional identity (baseline-ineligible)" - # banner promises; the guarantee lives here, not just in prose. - if f.properties.get("provisional_identity") is True: - out.append(f) - continue # Precedence (waiver > judged > baseline) lives in resolve_identity — the # single JOIN predicate. BASELINED carries no reason (matches the historical # inline behaviour); WAIVED/JUDGED carry the resolver's reason. diff --git a/src/wardline/rust/qualname.py b/src/wardline/rust/qualname.py index 28760291..9b461923 100644 --- a/src/wardline/rust/qualname.py +++ b/src/wardline/rust/qualname.py @@ -16,9 +16,11 @@ faithful in *rendering* but lacks that validate-and-degrade gate, so it currently emits a ``:``-bearing locator. The correct fix is an ADR-049 amendment defining a colon-free canonical form for path-typed generic args, adopted by both producers in lockstep — a Wardline-only -normalization would itself diverge from the (still-unreleased) oracle. Low slice-1 blast radius -(RS-WL-* findings are ``provisional_identity`` and Wardline emits no federation entity yet). -Tracked: see the ``rust-bug-hunt-2026-06-09`` reserved-colon issue. +normalization would itself diverge from the (still-unreleased) oracle. The frozen Rust +identity corpus (``tests/golden/identity/rust/``) deliberately contains no path-typed +generic args, so graduation does not pre-empt the pending cross-tool decision. +Tracked: see the ``rust-bug-hunt-2026-06-09`` reserved-colon issue and the 2026-06-10 +ADR-049 amendment-request letter. KNOWN GAP (const-generic-arg spacing): a multi-token *const* generic arg (``Foo<{N + 1}>``, ``Foo<-1>``) is rendered by the oracle via ``to_token_stream().to_string()`` — proc-macro2 diff --git a/src/wardline/rust/rules.py b/src/wardline/rust/rules.py index e10af409..0e438003 100644 --- a/src/wardline/rust/rules.py +++ b/src/wardline/rust/rules.py @@ -138,6 +138,5 @@ def _finding(rule_id: str, tc: RustTriggerContext, severity: Severity, taint_pat "taint_path": taint_path, "constructor_line": trig.constructor_line, "trigger_node_id": int(trig.trigger_node_id), - "provisional_identity": True, # RS-WL-* are baseline-ineligible until SP2 (spec §3.6) }, ) diff --git a/tests/golden/identity/rust/corpus/META.json b/tests/golden/identity/rust/corpus/META.json index ca8f2089..bd2694c5 100644 --- a/tests/golden/identity/rust/corpus/META.json +++ b/tests/golden/identity/rust/corpus/META.json @@ -1,5 +1,5 @@ { - "corpus_version": 1, + "corpus_version": 2, "fingerprint_scheme": "wlfp2", - "reason": "initial freeze (rust-sp2-2026-06-10 Task 6): SP2 completion gate — crate-prefixed RS-WL-* identity over the vendored rust-app crate" + "reason": "graduation (rust-sp2-2026-06-10 Task 7): drop provisional_identity property (rules.py no longer emits it; RS-WL-* baseline-eligible)" } diff --git a/tests/golden/identity/rust/corpus/rustapp.json b/tests/golden/identity/rust/corpus/rustapp.json index b3f2f67b..a16a8f86 100644 --- a/tests/golden/identity/rust/corpus/rustapp.json +++ b/tests/golden/identity/rust/corpus/rustapp.json @@ -211,7 +211,6 @@ "message": "Untrusted data selects the program executed by Command::new (constructed at line 26, run at line 27): an attacker controls which executable runs (CWE-78).", "properties": { "constructor_line": 26, - "provisional_identity": true, "taint_path": "EXTERNAL_RAW->Command::new(program)@L26->exec@L27", "trigger_node_id": 146 }, @@ -238,7 +237,6 @@ "message": "Untrusted data reaches a shell command line ('sh -c ...', run at line 18): an attacker can inject shell syntax (CWE-78).", "properties": { "constructor_line": 18, - "provisional_identity": true, "taint_path": "EXTERNAL_RAW->arg->'sh -c'->exec@L18", "trigger_node_id": 106 }, diff --git a/tests/golden/identity/rust/regen.py b/tests/golden/identity/rust/regen.py index b507e478..420a7a74 100644 --- a/tests/golden/identity/rust/regen.py +++ b/tests/golden/identity/rust/regen.py @@ -23,7 +23,9 @@ "rustapp": _HERE / "fixtures" / "rustapp", } # 1: initial freeze (SP2 completion gate) — crate-prefixed RS-WL-* identity. -CORPUS_VERSION = 1 +# 2: graduation — drop the provisional_identity property (rules.py no longer emits it; +# RS-WL-* baseline-eligible). Fingerprints unchanged (the property was never folded in). +CORPUS_VERSION = 2 def main() -> None: diff --git a/tests/unit/cli/test_scan_rust.py b/tests/unit/cli/test_scan_rust.py index 82fd566f..fe63a9dc 100644 --- a/tests/unit/cli/test_scan_rust.py +++ b/tests/unit/cli/test_scan_rust.py @@ -1,7 +1,7 @@ """WP6: ``wardline scan --lang rust`` end-to-end through the CLI. Drives the real Click command over a tmp ``.rs`` tree: the gate fires (exit 1) on an -``RS-WL-108`` injection, a clean tree exits 0, the preview banner prints to stderr, and a +``RS-WL-108`` injection, a clean tree exits 0, the posture banner prints to stderr, and a malformed file is surfaced via ``--fail-on-unanalyzed``. """ @@ -28,8 +28,9 @@ def test_scan_lang_rust_gate_trips_on_injection(tmp_path) -> None: assert result.exit_code == 1 # gate tripped rule_ids = [json.loads(line)["rule_id"] for line in out.read_text().splitlines() if line.strip()] assert "RS-WL-108" in rule_ids - # The preview posture banner is loud on stderr (provisional identity / no config parity). - assert "preview" in result.output.lower() + # The posture banner is loud on stderr (slice coverage + no severity-override parity). + assert "command-injection slice" in result.output + assert "severity overrides do not yet apply" in result.output def test_scan_lang_rust_clean_tree_exits_zero(tmp_path) -> None: @@ -67,9 +68,9 @@ def test_scan_lang_rust_reports_coverage_when_markers_present(tmp_path) -> None: def test_scan_default_lang_python_ignores_rs(tmp_path) -> None: - # Without --lang rust, a .rs file is not swept and no preview banner prints. + # Without --lang rust, a .rs file is not swept and no Rust posture banner prints. (tmp_path / "m.rs").write_text(_INJECTION, encoding="utf-8") out = tmp_path / "findings.jsonl" result = CliRunner().invoke(scan, [str(tmp_path), "--output", str(out)]) assert result.exit_code == 0 - assert "preview" not in result.output.lower() + assert "command-injection slice" not in result.output diff --git a/tests/unit/rust/test_provisional_identity.py b/tests/unit/rust/test_provisional_identity.py deleted file mode 100644 index 48044696..00000000 --- a/tests/unit/rust/test_provisional_identity.py +++ /dev/null @@ -1,59 +0,0 @@ -"""WP6 fold: the CLI banner's "provisional identity (baseline-ineligible)" claim is -ENFORCED, not just printed. - -A finding carrying ``properties["provisional_identity"] = True`` (every RS-WL-* finding): -* is never matched against a committed baseline/waiver/judged entry (stays ACTIVE, so it - always gates — even under ``--trust-suppressions`` a stale committed suppression cannot - clear it); and -* is never written INTO a generated baseline (its unstable fingerprint would orphan). -""" - -from __future__ import annotations - -from datetime import date - -from wardline.core.baseline import Baseline, build_baseline_document -from wardline.core.finding import Finding, Kind, Location, Severity, SuppressionState -from wardline.core.suppression import apply_suppressions -from wardline.core.waivers import WaiverSet - - -def _rs_finding(*, provisional: bool) -> Finding: - props = {"taint_path": "x"} - if provisional: - props["provisional_identity"] = True - return Finding( - rule_id="RS-WL-108", - message="program injection", - severity=Severity.ERROR, - kind=Kind.DEFECT, - location=Location(path="src/m.rs", line_start=3, line_end=3), - fingerprint="a" * 64, - qualname="m.f", - properties=props, - ) - - -def test_provisional_finding_is_not_suppressed_by_a_matching_baseline() -> None: - f = _rs_finding(provisional=True) - baseline = Baseline(frozenset({"a" * 64})) # a committed entry keyed to its exact fingerprint - (out,) = apply_suppressions([f], baseline, WaiverSet([]), today=date(2026, 6, 9)) - assert out.suppressed is SuppressionState.ACTIVE # baseline-ineligible: stays ACTIVE - - -def test_non_provisional_finding_is_still_baselined() -> None: - # The guard must be SCOPED to provisional findings — a normal defect still baselines. - f = _rs_finding(provisional=False) - baseline = Baseline(frozenset({"a" * 64})) - (out,) = apply_suppressions([f], baseline, WaiverSet([]), today=date(2026, 6, 9)) - assert out.suppressed is SuppressionState.BASELINED - - -def test_provisional_finding_is_excluded_from_a_generated_baseline() -> None: - doc = build_baseline_document([_rs_finding(provisional=True)]) - assert doc["entries"] == [] # never captured — its fingerprint is not stable - - -def test_non_provisional_finding_is_captured_in_a_baseline() -> None: - doc = build_baseline_document([_rs_finding(provisional=False)]) - assert [e["fingerprint"] for e in doc["entries"]] == ["a" * 64] diff --git a/tests/unit/rust/test_rust_identity_graduated.py b/tests/unit/rust/test_rust_identity_graduated.py new file mode 100644 index 00000000..3822b680 --- /dev/null +++ b/tests/unit/rust/test_rust_identity_graduated.py @@ -0,0 +1,79 @@ +"""SP2 graduation: RS-WL-* findings are full citizens of the suppression machinery. + +The pre-SP2 ``properties["provisional_identity"]`` short-circuits (never baseline-match, +never baseline-capture) are GONE — Rust identity is frozen (crate-prefixed, gated by +``tests/golden/identity/rust/``), so an RS-WL-* finding: +* IS matched against a committed baseline entry like any other defect; and +* IS written into a generated baseline. + +These tests are the inversion of the retired ``test_provisional_identity.py``. A stray +``provisional_identity`` property (e.g. from a stale producer) must be inert — no code +path consults it anymore. +""" + +from __future__ import annotations + +from datetime import date + +import pytest + +from wardline.core.baseline import Baseline, build_baseline_document +from wardline.core.finding import Finding, Kind, Location, Severity, SuppressionState +from wardline.core.suppression import apply_suppressions +from wardline.core.waivers import WaiverSet + + +def _rs_finding(*, stray_provisional_prop: bool = False) -> Finding: + props: dict[str, object] = {"taint_path": "x"} + if stray_provisional_prop: + props["provisional_identity"] = True + return Finding( + rule_id="RS-WL-108", + message="program injection", + severity=Severity.ERROR, + kind=Kind.DEFECT, + location=Location(path="src/m.rs", line_start=3, line_end=3), + fingerprint="a" * 64, + qualname="m.f", + properties=props, + ) + + +def test_rs_finding_is_suppressed_by_a_matching_baseline() -> None: + f = _rs_finding() + baseline = Baseline(frozenset({"a" * 64})) # a committed entry keyed to its exact fingerprint + (out,) = apply_suppressions([f], baseline, WaiverSet([]), today=date(2026, 6, 10)) + assert out.suppressed is SuppressionState.BASELINED # graduated: baseline-eligible + + +def test_rs_finding_is_captured_in_a_generated_baseline() -> None: + doc = build_baseline_document([_rs_finding()]) + assert [e["fingerprint"] for e in doc["entries"]] == ["a" * 64] + + +def test_producer_no_longer_emits_the_provisional_property() -> None: + # The real RS-WL-* producer (rules.py) emits no retired identity flag. + pytest.importorskip("tree_sitter", reason="wardline[rust] extra not installed") + from wardline.rust.analyzer import RustAnalyzer + + source = ( + "/// @trusted(level=ASSURED)\n" + "fn f() {\n" + ' let t = std::env::var("X").unwrap();\n' + " Command::new(t).output();\n" + "}\n" + ) + (finding,) = RustAnalyzer().analyze_source(source, module="demo.m", path="src/m.rs") + assert finding.rule_id == "RS-WL-108" + assert "provisional_identity" not in finding.properties + + +def test_stray_provisional_property_is_inert() -> None: + # The plumbing is removed, not bypassed: a finding still carrying the retired + # property baselines and captures exactly like any other defect. + f = _rs_finding(stray_provisional_prop=True) + baseline = Baseline(frozenset({"a" * 64})) + (out,) = apply_suppressions([f], baseline, WaiverSet([]), today=date(2026, 6, 10)) + assert out.suppressed is SuppressionState.BASELINED + doc = build_baseline_document([f]) + assert [e["fingerprint"] for e in doc["entries"]] == ["a" * 64] From b30d3d79cafdf0a851634f20045c495ca3df46df Mon Sep 17 00:00:00 2001 From: John Morrissey Date: Wed, 10 Jun 2026 14:54:13 +1000 Subject: [PATCH 089/186] fix(rust)!: wlfp2 move-stable RS-WL-* fingerprints (entity-relative discriminators) + #out non-conformance routes + RS-WL rejoins rekey population (identity keystone panel) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Pre-release rekey, done in the one window where nothing stores RS-WL fingerprints: - FIX 1 (critical): the fingerprint discriminator folded ABSOLUTE lines and a file-wide pre-order NodeId — any edit above the function rekeyed the join key, violating wlfp2 move-stability (core/finding.py:170). Lines now fold as (line - entity_line_start) and the NodeId as (trigger_node_id - entity_node_id); resolved taint tiers are dropped from the join key per the wlfp2 invariant (weft-4a9d0f863c). properties["taint_path"] keeps the absolute-line display form. New tests/unit/rust/test_fingerprint_stability.py (TDD: comment-at-top / comment-above-fn / sibling-above keep both rules' fingerprints; in-fn edit rekeys, documented; same-line twins stay distinct). - FIX 2 (critical): core/rekey.is_join_population no longer hard-excludes RS-WL-* (a live orphaning path post-graduation); P5-REVISIT decided 2026-06-10; population test inverted. - FIX 3 (critical): rust e2e asserts the graduated banner ("command-injection slice"), not the retired "preview". - FIX 4 (major): class-2 routes = {crate}.#out.{literal stems from crate dir}, class-3 = crate.#out.{literal stems from scan root} (constant "crate" segment — cargo forbids the keyword as a package name). The reserved #out segment is impossible in loomweave's locator grammar, so the no-collision claims are now true; class-3 identity is relpath-pure (scan-root rename does not rekey — the panel's exp_e4e repro is a test now). - FIX 5/6: --lang help de-previewed; graduation tests extended (waiver -> WAIVED, judged -> JUDGED, gate_trips on an active RS-WL-108 ERROR). - FIX 7: corpus v3 (regen --reason stamped) + class-2 fixture (tests/integration.rs) freezing a rust_app.#out.* RS-WL-112 finding; non-vacuity test for .#out.; META corpus_version pinned to regen.CORPUS_VERSION (the panel's discipline gap). - FIX 8: rust guide documents crate-name-keyed identity (manifest add/remove/ rename rekeys; scan from the crate root); CHANGELOG bullet extended. BREAKING: every pre-rekey RS-WL-* fingerprint changes (never stored anywhere; no migration needed). Co-Authored-By: Claude Fable 5 --- CHANGELOG.md | 6 + docs/guides/rust-preview.md | 10 ++ .../reference/finding-lifecycle-vocabulary.md | 12 +- src/wardline/cli/scan.py | 5 +- src/wardline/core/rekey.py | 12 +- src/wardline/rust/analyzer.py | 63 ++++++-- src/wardline/rust/context.py | 11 +- src/wardline/rust/crate_roots.py | 10 +- src/wardline/rust/rules.py | 48 ++++-- tests/docs/test_glossary_vocabulary.py | 6 +- tests/e2e/test_rust_live.py | 4 +- tests/golden/identity/rust/corpus/META.json | 4 +- .../golden/identity/rust/corpus/rustapp.json | 54 ++++++- .../fixtures/rustapp/tests/integration.rs | 13 ++ tests/golden/identity/rust/regen.py | 7 +- .../rust/test_rust_identity_parity.py | 20 +++ tests/unit/core/test_rekey_population.py | 10 +- tests/unit/rust/test_analyzer_protocol.py | 10 +- tests/unit/rust/test_crate_roots.py | 48 ++++-- tests/unit/rust/test_fingerprint_stability.py | 140 ++++++++++++++++++ .../unit/rust/test_rust_identity_graduated.py | 45 +++++- 21 files changed, 463 insertions(+), 75 deletions(-) create mode 100644 tests/golden/identity/rust/fixtures/rustapp/tests/integration.rs create mode 100644 tests/unit/rust/test_fingerprint_stability.py diff --git a/CHANGELOG.md b/CHANGELOG.md index 6997ec2b..2eed29f0 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -30,6 +30,12 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 never-baseline-capture) is removed — baseline, waivers, and judged verdicts now apply to Rust findings exactly as for Python. (Pre-graduation RS-WL-* fingerprints change once; they were never baseline-eligible, so no migration.) + Finding identity is keyed to the crate name: adding/removing a `Cargo.toml` or + renaming the crate in the manifest rekeys RS-WL-* fingerprints (re-baseline + after such a change); non-conformance files — outside `src/`, or in a + manifest-less tree — carry a reserved `#out` route segment + (`{crate}.#out.{...}` / `crate.#out.{...}`) so their qualnames can never + collide with a Loomweave-conformant locator. - **Rust frontend is a full ADR-049 producer (Loomweave Phase 1b).** The entity surface grows from callables-only to the full ten-kind contract set — `enum`/`trait`/`type_alias`/`const`/`static`/`macro` leaf entities, the `impl` diff --git a/docs/guides/rust-preview.md b/docs/guides/rust-preview.md index 818f93a6..e5f58175 100644 --- a/docs/guides/rust-preview.md +++ b/docs/guides/rust-preview.md @@ -12,6 +12,16 @@ data reaching the program or shell command line of `std::process::Command`. suppression machinery — they match committed baseline/waiver/judged entries and are captured by `wardline baseline` like any Python finding. +Finding identity is **keyed to the crate name**: the qualname every fingerprint +hashes starts with the crate read from `Cargo.toml` (`[package].name`, +underscored). Adding or removing a `Cargo.toml`, or renaming the crate in the +manifest, therefore **rekeys** every fingerprint under that crate — re-baseline +after such a change. For manifest-less trees the identity is path-pure (stable +under a scan-root directory rename), but adding a manifest later rekeys those +findings too. Relatedly, scanning a **subtree** of a crate (e.g. its `src/` +directory directly) loses sight of the manifest and degrades identity to the +manifest-less form — always scan from the crate root. + !!! warning "Scope — command-injection slice; no config severity overrides yet" Rule coverage is the **command-injection slice** (`RS-WL-108` / `RS-WL-112`) — a Rust scan says nothing about other defect families. `weft.toml` severity diff --git a/docs/reference/finding-lifecycle-vocabulary.md b/docs/reference/finding-lifecycle-vocabulary.md index f264c21f..ce724fff 100644 --- a/docs/reference/finding-lifecycle-vocabulary.md +++ b/docs/reference/finding-lifecycle-vocabulary.md @@ -61,7 +61,7 @@ The Filigree metadata only carries the key when the state is not `active` **"suppressed"** survives only as the umbrella *word* for "any state other than `active`": `baselined` + `waived` + `judged`. The CLI prints this sum as the -`suppressed` count (`src/wardline/cli/scan.py:403`). +`suppressed` count (`src/wardline/cli/scan.py:406`). ## `active` is the one word for "non-suppressed defect" @@ -72,7 +72,7 @@ consistently, on every surface: | --- | --- | --- | | Enum | `src/wardline/core/finding.py:72` | `SuppressionState.ACTIVE = "active"` | | Summary field | `src/wardline/core/run.py:51`, built at `src/wardline/core/run.py:320` | `ScanSummary.active` | -| CLI summary line | `src/wardline/cli/scan.py:404` | `… {s.active} active` | +| CLI summary line | `src/wardline/cli/scan.py:407` | `… {s.active} active` | | MCP scan response | `src/wardline/mcp/server.py:331` | `summary.active` | | Agent-summary JSON | `src/wardline/core/agent_summary.py:135` | `summary.active_defects` | | `wardline:loop` prompt | `src/wardline/mcp/prompts.py:13` | "Read `summary.active`" | @@ -151,7 +151,7 @@ The MCP `scan` gate block exposes `gate.tripped` (`src/wardline/mcp/server.py:34 `src/wardline/core/agent_summary.py:153` (`verdict`). The CLI prints `gate: FAILED (--fail-on …) — ` then `gate: evaluated <…>`, or a `gate: NOT_EVALUATED — …` line for a bare scan -(`src/wardline/cli/scan.py:438`). +(`src/wardline/cli/scan.py:441`). `--new-since` scopes **both** populations identically: any `active` defect outside the delta is re-marked `baselined` in both the emitted and gate lists @@ -167,7 +167,7 @@ still legitimately means three different things depending on the surface: | --- | --- | --- | | Filigree store | An **unseen fingerprint** — first time this finding identity is seen for a `(file, scan_source)`. | **Filigree-owned** lifecycle (`src/wardline/core/filigree_emit.py:68-76`) | | `wardline scan --new-since ` | **Delta-scope**: the gate fires only on defects in files/entities changed since a git ref; everything else is re-marked `baselined`. | `src/wardline/core/run.py:296`; help text `src/wardline/cli/scan.py` (`--new-since`) | -| (historical) CLI summary | Formerly relabelled the `active` count as "N new". **Corrected to "N active"**. | `src/wardline/cli/scan.py:403` | +| (historical) CLI summary | Formerly relabelled the `active` count as "N new". **Corrected to "N active"**. | `src/wardline/cli/scan.py:406` | The first-seen Filigree sense and the delta-scope `--new-since` sense are genuinely distinct concepts; neither is "active". @@ -179,8 +179,8 @@ How each concept appears on each surface: | Concept | CLI summary text | `ScanSummary` field | MCP `summary` key | Agent-summary key | Filigree store | | --- | --- | --- | --- | --- | --- | | every finding | `N finding(s)` | `total` (`run.py:50`) | `total` (`server.py:330`) | `total_findings` (`agent_summary.py:134`) | one finding per wire entry | -| live defect | `N active` (`scan.py:404`) | `active` (`run.py:51,320`) | `active` (`server.py:331`) | `active_defects` (`agent_summary.py:135`) | no `suppression_state` key (`finding.py:285`) | -| suppressed (sum) | `N suppressed` (`scan.py:403`) | `baselined+waived+judged` | the three keys | `suppressed_findings` (`agent_summary.py:136`) | `metadata.wardline.suppression_state` (`finding.py:285`) | +| live defect | `N active` (`scan.py:407`) | `active` (`run.py:51,320`) | `active` (`server.py:331`) | `active_defects` (`agent_summary.py:135`) | no `suppression_state` key (`finding.py:285`) | +| suppressed (sum) | `N suppressed` (`scan.py:406`) | `baselined+waived+judged` | the three keys | `suppressed_findings` (`agent_summary.py:136`) | `metadata.wardline.suppression_state` (`finding.py:285`) | | baselined | `N baseline` | `baselined` (`run.py:53`) | `baselined` (`server.py:332`) | `baselined` (`agent_summary.py:138`) | `suppression_state: "baselined"` | | waived | `N waiver` | `waived` (`run.py:54`) | `waived` (`server.py:333`) | `waived` (`agent_summary.py:139`) | `suppression_state: "waived"` | | judged | `N judged` | `judged` (`run.py:55`) | `judged` (`server.py:334`) | `judged` (`agent_summary.py:140`) | `suppression_state: "judged"` | diff --git a/src/wardline/cli/scan.py b/src/wardline/cli/scan.py index da038817..fd8fd0a6 100644 --- a/src/wardline/cli/scan.py +++ b/src/wardline/cli/scan.py @@ -41,7 +41,10 @@ "--lang", type=click.Choice(["python", "rust"]), default="python", - help="Language frontend. 'rust' (PREVIEW) scans .rs files for RS-WL-* command-injection findings.", + help=( + "Language frontend. 'rust' scans .rs files for RS-WL-* command-injection findings " + "(frozen identity, baseline-eligible; config severity overrides not yet applied)." + ), ) @click.option("--output", type=click.Path(path_type=Path), default=None) # exit 1 if any non-suppressed DEFECT has severity >= this threshold (SP3b) diff --git a/src/wardline/core/rekey.py b/src/wardline/core/rekey.py index 15ad8a33..bca5c58e 100644 --- a/src/wardline/core/rekey.py +++ b/src/wardline/core/rekey.py @@ -55,11 +55,11 @@ def is_join_population(f: Finding) -> bool: gating ERROR DEFECTs at ENGINE_PATH) silently orphans on migration and resurfaces ACTIVE (the P4-review gate regression). - ``RS-WL-*`` (Rust) is EXCLUDED: those findings do not appear on rc4 at all (the - plugin lives in a worktree), so the exclusion is a no-op here. **P5-REVISIT:** when - the Rust worktree rebases, decide whether RS-WL DEFECTs enter the stores and should - migrate (see the skipped test in test_rekey_population.py).""" - return f.kind is Kind.DEFECT and not f.rule_id.startswith("RS-WL-") + ``RS-WL-*`` (Rust) is INCLUDED — P5-REVISIT decided 2026-06-10 (identity keystone): + Rust identity graduated to baseline-eligible, so an RS-WL DEFECT enters the stores + like any other and a stored RS-WL verdict must migrate, not orphan. (The former + hard exclusion was a no-op pre-merge but a live orphaning path post-graduation.)""" + return f.kind is Kind.DEFECT def _is_scheme_independent(rule_id: str) -> bool: @@ -111,7 +111,7 @@ class FingerprintRemap: def compute_old_new_fingerprints(findings: Iterable[Finding]) -> list[FingerprintRemap]: """The dual-fingerprint contract from one scan, over the join population (every - DEFECT minus RS-WL). ``old_fp`` is ``_old_fingerprint(f)`` (v0 reconstruction for + DEFECT). ``old_fp`` is ``_old_fingerprint(f)`` (v0 reconstruction for scheme-sensitive rules, identity for scheme-independent engine diagnostics); ``new_fp`` is the live ``finding.fingerprint``. The v0 reconstruction is validated NON-CIRCULARLY against the real pre-P3 corpus in ``tests/unit/core/test_rekey_dual_fp.py``. diff --git a/src/wardline/rust/analyzer.py b/src/wardline/rust/analyzer.py index f4555541..cc64eb6f 100644 --- a/src/wardline/rust/analyzer.py +++ b/src/wardline/rust/analyzer.py @@ -170,7 +170,19 @@ def _analyze_tree(self, tree: Tree, *, module: str, path: str) -> tuple[list[Fin if body is None: continue for trig in analyze_command_dataflow(body, nmap): - triggers.append(RustTriggerContext(trigger=trig, qualname=entity.qualname, tier=tier, path=path)) + triggers.append( + RustTriggerContext( + trigger=trig, + qualname=entity.qualname, + tier=tier, + path=path, + # The entity's OWN anchors — the rules fold trigger positions into + # the fingerprint entity-relative (wlfp2 move-stability), so the + # containing fn's line/NodeId travel with each trigger. + entity_line_start=entity.location.line_start or 0, + entity_node_id=entity.node_id, + ) + ) context = RustAnalysisContext( triggers=tuple(triggers), @@ -202,29 +214,52 @@ def _module_for(file: Path, resolved_root: Path, roots: CrateRoots) -> str: ``rust_module_route(crate=, src_root=/src, file)``. Conformance-bearing: byte-identical to loomweave's emission for the same file. 2. **Under a crate root but OUTSIDE its src/** (``tests/``, ``benches/``, - ``build.rs``, ...): wardline-local fallback — the owning crate's real name + - the mechanical path route from the crate dir (no ``src/`` to strip). Loomweave's + ``build.rs``, ...): ``{crate}.#out.{}``. Loomweave's ``emittable_scope`` emits NOTHING for these files, so this qualname carries no - cross-tool conformance claim and cannot collide with a loomweave locator — - wardline scans them anyway (coverage is never narrowed to the entity surface). - 3. **Under no crate root** (a bare no-Cargo tree — the whole pre-SP2 preview - population): the pre-SP2 mechanical route, byte-unchanged — ``crate`` = scan-root - name, ``src_root`` = the scan root (no ``src/`` strip). Same no-conformance-claim - disclaimer as class 2; the preview scan population is unchanged. + cross-tool conformance claim; the reserved ``#out`` segment is structurally + impossible in loomweave's locator grammar (``#`` appears only inside + ``impl#<...>`` discriminators), so a class-2 route can never collide with a + class-1/loomweave locator (e.g. ``/tests/integration.rs`` vs + ``/src/tests/integration.rs`` -> ``rust_app.#out.tests.integration`` + vs ``rust_app.tests.integration``). Wardline scans these files anyway — + coverage is never narrowed to the entity surface. + 3. **Under no crate root** (a bare no-Cargo tree): the crate segment is the + CONSTANT ``"crate"`` (cargo forbids the keyword ``crate`` as a package name, + so it cannot collide with a class-1 crate) — route = + ``crate.#out.{}``. + Relpath-pure and scan-root-name-INDEPENDENT: renaming the scan-root + directory does not rekey fingerprints (e.g. ``bin/app.rs`` -> + ``crate.#out.bin.app`` whatever the root is called). Same + no-conformance-claim disclaimer as class 2. """ resolved = file.resolve() crate_dir = roots.crate_dir_for(resolved) crate_name = roots.crate_name_for(resolved) if crate_dir is not None and crate_name is not None: src_root = crate_dir / "src" - base = src_root if resolved.is_relative_to(src_root) else crate_dir - return q.rust_module_route(crate=crate_name, src_root=str(base), file=str(resolved)) - crate = resolved_root.name or "crate" + if resolved.is_relative_to(src_root): + return q.rust_module_route(crate=crate_name, src_root=str(src_root), file=str(resolved)) + return _out_route(crate_name, crate_dir, resolved) # class 2 try: - return q.rust_module_route(crate=crate, src_root=str(resolved_root), file=str(resolved)) + return _out_route("crate", resolved_root, resolved) # class 3 except ValueError: # file outside root (should not happen — discover confines to root); degrade to crate. - return crate + return "crate" + + +def _out_route(crate: str, base: Path, file: Path) -> str: + """The class-2/3 non-conformance route: ``{crate}.#out.{relpath stems}``. + + Mechanical and relpath-pure: every path segment from ``base`` contributes its + LITERAL stem (only the final ``.rs`` is stripped — ``main``/``lib``/``mod`` are + NOT collapsed, unlike the ADR-049 in-src route, because there is no module tree + to mirror out here and literal stems keep distinct files distinct). The ``#out`` + segment brands the route as outside loomweave's emittable scope. + """ + rel = file.relative_to(base) + segments = [*rel.parts[:-1], file.stem] + return ".".join([crate, "#out", *segments]) def _parse_error_finding(relpath: str, detail: str) -> Finding: diff --git a/src/wardline/rust/context.py b/src/wardline/rust/context.py index 0bbd5f56..7e35dc95 100644 --- a/src/wardline/rust/context.py +++ b/src/wardline/rust/context.py @@ -16,6 +16,7 @@ from collections.abc import Mapping, Sequence from wardline.core.finding import Finding + from wardline.core.node_id import NodeId from wardline.core.taints import TaintState from wardline.rust.dataflow import CommandTrigger from wardline.rust.index import RustEntity @@ -25,12 +26,20 @@ @dataclass(frozen=True, slots=True) class RustTriggerContext: - """One command trigger bound to its containing function's identity and trust tier.""" + """One command trigger bound to its containing function's identity and trust tier. + + ``entity_line_start`` / ``entity_node_id`` are the containing fn's own anchors + (``RustEntity.location.line_start`` / ``RustEntity.node_id``). The rules fold the + trigger's position into the fingerprint as DELTAS against them (wlfp2 + move-stability: an edit ABOVE the entity shifts every absolute line and pre-order + NodeId below it, but the entity-relative offsets are invariant).""" trigger: CommandTrigger qualname: str # the containing fn (finding qualname + fingerprint key) tier: TaintState # the containing fn's body taint — modulates rule severity path: str # the source file path (Location + fingerprint key) + entity_line_start: int # the containing fn's first line (entity-relative line anchor) + entity_node_id: NodeId # the containing fn's pre-order NodeId (entity-relative NodeId anchor) @dataclass(frozen=True, slots=True) diff --git a/src/wardline/rust/crate_roots.py b/src/wardline/rust/crate_roots.py index 27fd133a..df08363d 100644 --- a/src/wardline/rust/crate_roots.py +++ b/src/wardline/rust/crate_roots.py @@ -25,9 +25,13 @@ and files under no crate root — it emits no federation entity for them. That is its *entity surface*, not a scan filter: wardline keeps scanning ALL discovered ``.rs`` files. Files outside any crate's ``src/`` tree get a wardline-local -FALLBACK module route (see ``analyzer._module_for``) whose qualnames carry **no -cross-tool conformance claim** — loomweave emits nothing there, so no locator -collision is possible. +``#out``-branded module route (see ``analyzer._module_for``: class 2 = +``{crate}.#out.{...}``, class 3 = ``crate.#out.{...}`` with the constant crate +segment) whose qualnames carry **no cross-tool conformance claim**. The reserved +``#out`` segment is structurally impossible in loomweave's locator grammar (``#`` +appears only inside ``impl#<...>`` discriminators) and cargo forbids the keyword +``crate`` as a package name, so neither route can collide with a class-1 / +loomweave locator. """ from __future__ import annotations diff --git a/src/wardline/rust/rules.py b/src/wardline/rust/rules.py index 0e438003..220796d0 100644 --- a/src/wardline/rust/rules.py +++ b/src/wardline/rust/rules.py @@ -99,7 +99,11 @@ def _program_finding(tc: RustTriggerContext, severity: Severity) -> Finding: f"(constructed at line {trig.constructor_line}, run at line {trig.trigger_line}): " f"an attacker controls which executable runs (CWE-78)." ) - return _finding(RustProgramInjectionRule.rule_id, tc, severity, taint_path, message) + # Fingerprint discriminator (NOT the display path): source-derived + entity-relative + # only (wlfp2 — see _fp_discriminant). The constructor's relative line is folded so + # two stepwise builders terminating on one line stay distinct by construction site. + fp_disc = f"Command::new(program)@L+{trig.constructor_line - tc.entity_line_start}" + return _finding(RustProgramInjectionRule.rule_id, tc, severity, taint_path, fp_disc, message) def _shell_finding(tc: RustTriggerContext, severity: Severity, worst: TaintState) -> Finding: @@ -110,10 +114,40 @@ def _shell_finding(tc: RustTriggerContext, severity: Severity, worst: TaintState f"('{trig.program_literal} -c ...', run at line {trig.trigger_line}): " f"an attacker can inject shell syntax (CWE-78)." ) - return _finding(RustShellInjectionRule.rule_id, tc, severity, taint_path, message) + # The program literal is a source token (the spelling as written), so it is a legal + # wlfp2 discriminator component; the resolved `worst` tier is NOT (display-only). + fp_disc = f"arg->'{trig.program_literal} -c'" + return _finding(RustShellInjectionRule.rule_id, tc, severity, taint_path, fp_disc, message) + + +def _fp_discriminant(tc: RustTriggerContext, fp_disc: str) -> str: + """The wlfp2 ``taint_path`` fingerprint component for one trigger. + + Every folded position is ENTITY-RELATIVE (wlfp2 move-stability, the + rust-sp2-2026-06-10 keystone rekey — see core/finding.py:170): the trigger line + folds as ``line - entity_line_start`` and the trigger NodeId as + ``trigger_node_id - entity_node_id``. Both anchors are the containing fn's own + (its first line, its pre-order index), so an edit ABOVE the entity — a comment + at the top of the file, a sibling fn inserted above — shifts absolute lines and + pre-order indices in lockstep and leaves both deltas invariant. An edit INSIDE + the fn above the trigger moves the deltas and rekeys (accepted: that is the + same entity-relative limitation the Python sink rules carry). + + The relative-NodeId fold is the SOLE same-line discriminant: two DISTINCT + triggers on one physical line (identical rule/path/qualname/relative-line) get + distinct fingerprints — the no-collision invariant. Resolved taint tiers never + appear here (they drift across builds: weft-4a9d0f863c); the human-readable + ``properties["taint_path"]`` keeps the absolute-line display form. + """ + trig = tc.trigger + rel_line = trig.trigger_line - tc.entity_line_start + rel_node = int(trig.trigger_node_id) - int(tc.entity_node_id) + return f"{fp_disc}->exec@L+{rel_line}@node+{rel_node}" -def _finding(rule_id: str, tc: RustTriggerContext, severity: Severity, taint_path: str, message: str) -> Finding: +def _finding( + rule_id: str, tc: RustTriggerContext, severity: Severity, taint_path: str, fp_disc: str, message: str +) -> Finding: trig = tc.trigger return Finding( rule_id=rule_id, @@ -125,13 +159,7 @@ def _finding(rule_id: str, tc: RustTriggerContext, severity: Severity, taint_pat rule_id=rule_id, path=tc.path, qualname=tc.qualname, - # Fold the trigger's NodeId so two DISTINCT commands on the SAME line (identical - # taint_path) get distinct fingerprints — the no-collision invariant. line_start - # was dropped from the fingerprint by the move-stability rekey (wlfp2), so the - # NodeId fold is now the SOLE same-line discriminant. The NodeId is the - # reproducible pre-order index, so this stays deterministic across runs. The - # stored properties["taint_path"] keeps the clean, human-readable form. - taint_path=f"{taint_path}@node{trig.trigger_node_id}", + taint_path=_fp_discriminant(tc, fp_disc), ), qualname=tc.qualname, properties={ diff --git a/tests/docs/test_glossary_vocabulary.py b/tests/docs/test_glossary_vocabulary.py index 8cde3c98..744e80c1 100644 --- a/tests/docs/test_glossary_vocabulary.py +++ b/tests/docs/test_glossary_vocabulary.py @@ -41,9 +41,9 @@ ("src/wardline/core/run.py", 320, "active=sum"), ("src/wardline/core/run.py", 371, "honors_suppressions"), # src/wardline/cli/scan.py — CLI summary line + gate stderr - ("src/wardline/cli/scan.py", 403, "suppressed"), - ("src/wardline/cli/scan.py", 404, "{s.active} active"), - ("src/wardline/cli/scan.py", 438, "gate: FAILED"), + ("src/wardline/cli/scan.py", 406, "suppressed"), + ("src/wardline/cli/scan.py", 407, "{s.active} active"), + ("src/wardline/cli/scan.py", 441, "gate: FAILED"), # src/wardline/mcp/server.py — MCP scan summary + gate block ("src/wardline/mcp/server.py", 330, '"total": result.summary.total'), ("src/wardline/mcp/server.py", 331, '"active": result.summary.active'), diff --git a/tests/e2e/test_rust_live.py b/tests/e2e/test_rust_live.py index c0a0403c..24908138 100644 --- a/tests/e2e/test_rust_live.py +++ b/tests/e2e/test_rust_live.py @@ -48,7 +48,9 @@ def test_live_scan_rust_corpus_trips_gate_and_emits_findings(tmp_path) -> None: proc = _scan([str(work), "--lang", "rust", "--fail-on", "ERROR", "--output", str(out)]) assert proc.returncode == 1, proc.stderr # gate tripped on the RS-WL-108 ERRORs - assert "preview" in (proc.stdout + proc.stderr).lower() + # The graduated posture banner (the old "(preview)" banner is gone): coverage is + # the command-injection slice; severity-override gap still surfaced. + assert "command-injection slice" in (proc.stdout + proc.stderr) rule_ids = [json.loads(line)["rule_id"] for line in out.read_text().splitlines() if line.strip()] assert rule_ids.count("RS-WL-108") == 6 assert rule_ids.count("RS-WL-112") == 3 diff --git a/tests/golden/identity/rust/corpus/META.json b/tests/golden/identity/rust/corpus/META.json index bd2694c5..37ed99b0 100644 --- a/tests/golden/identity/rust/corpus/META.json +++ b/tests/golden/identity/rust/corpus/META.json @@ -1,5 +1,5 @@ { - "corpus_version": 2, + "corpus_version": 3, "fingerprint_scheme": "wlfp2", - "reason": "graduation (rust-sp2-2026-06-10 Task 7): drop provisional_identity property (rules.py no longer emits it; RS-WL-* baseline-eligible)" + "reason": "rekey (rust-sp2-2026-06-10 keystone): entity-relative fingerprint discriminators (wlfp2 move-stability) + #out non-conformance route branding + relpath-pure class-3 crate segment + class-2 fixture pinned" } diff --git a/tests/golden/identity/rust/corpus/rustapp.json b/tests/golden/identity/rust/corpus/rustapp.json index a16a8f86..774a94ff 100644 --- a/tests/golden/identity/rust/corpus/rustapp.json +++ b/tests/golden/identity/rust/corpus/rustapp.json @@ -193,12 +193,36 @@ }, "parent": "rust_app", "qualname": "rust_app.main" + }, + { + "kind": "module", + "location": { + "col_end": 0, + "col_start": 0, + "line_end": 14, + "line_start": 1, + "path": "tests/integration.rs" + }, + "parent": null, + "qualname": "rust_app.#out.tests.integration" + }, + { + "kind": "function", + "location": { + "col_end": 1, + "col_start": 0, + "line_end": 13, + "line_start": 9, + "path": "tests/integration.rs" + }, + "parent": "rust_app.#out.tests.integration", + "qualname": "rust_app.#out.tests.integration.shell_smoke" } ], "findings": [ { "confidence": null, - "fingerprint": "8eb7f8775b3756ecb5eb5047ee29433a282f169aef4caecb493596e238f167fa", + "fingerprint": "3b602642e9cdac085dfcaa3fccc454e7a668f58fb1d16254d33e73ce10429f4c", "kind": "defect", "location": { "col_end": null, @@ -224,7 +248,7 @@ }, { "confidence": null, - "fingerprint": "b34c868714b59c4aa6be57cbef8d1e29662ba853d92dd92cd9e87067e6ac2239", + "fingerprint": "d27ddae0f7699a677570dde97dd36ff2b581730e91acf19faa3f3fe0af3e99ef", "kind": "defect", "location": { "col_end": null, @@ -247,6 +271,32 @@ "suggestion": null, "suppression_reason": null, "suppression_state": "active" + }, + { + "confidence": null, + "fingerprint": "c0f302edd3c4cb61f0acf2f5e7f447268270e4d2ea04742f6634432f103a44cf", + "kind": "defect", + "location": { + "col_end": null, + "col_start": null, + "line_end": 12, + "line_start": 12, + "path": "tests/integration.rs" + }, + "maturity": "stable", + "message": "Untrusted data reaches a shell command line ('sh -c ...', run at line 12): an attacker can inject shell syntax (CWE-78).", + "properties": { + "constructor_line": 12, + "taint_path": "EXTERNAL_RAW->arg->'sh -c'->exec@L12", + "trigger_node_id": 74 + }, + "qualname": "rust_app.#out.tests.integration.shell_smoke", + "related_entities": [], + "rule_id": "RS-WL-112", + "severity": "WARN", + "suggestion": null, + "suppression_reason": null, + "suppression_state": "active" } ] } diff --git a/tests/golden/identity/rust/fixtures/rustapp/tests/integration.rs b/tests/golden/identity/rust/fixtures/rustapp/tests/integration.rs new file mode 100644 index 00000000..e834d793 --- /dev/null +++ b/tests/golden/identity/rust/fixtures/rustapp/tests/integration.rs @@ -0,0 +1,13 @@ +//! Class-2 fixture (under the crate root, OUTSIDE src/): frozen under the +//! reserved `#out` route segment — `rust_app.#out.tests.integration` — so a +//! class-2 FINDING with the non-conformance branding is pinned in the corpus. +//! No path-typed generic args (reserved-colon constraint, see the README). + +use std::process::Command; + +/// @trusted(level=ASSURED) +fn shell_smoke() { + let tool = std::env::var("RUSTAPP_TOOL").unwrap(); + // RS-WL-112: untrusted data reaches a `sh -c` shell command line. + Command::new("sh").arg("-c").arg(tool).output(); +} diff --git a/tests/golden/identity/rust/regen.py b/tests/golden/identity/rust/regen.py index 420a7a74..0a3e445d 100644 --- a/tests/golden/identity/rust/regen.py +++ b/tests/golden/identity/rust/regen.py @@ -25,7 +25,12 @@ # 1: initial freeze (SP2 completion gate) — crate-prefixed RS-WL-* identity. # 2: graduation — drop the provisional_identity property (rules.py no longer emits it; # RS-WL-* baseline-eligible). Fingerprints unchanged (the property was never folded in). -CORPUS_VERSION = 2 +# 3: rekey (rust-sp2-2026-06-10 keystone) — entity-relative fingerprint discriminators +# (wlfp2 move-stability: lines/NodeIds fold as deltas against the containing fn; +# resolved taint tiers dropped from the join key), `#out` non-conformance route +# branding (class 2 = {crate}.#out.{...}), relpath-pure constant-"crate" class-3 +# segment, and the class-2 fixture (tests/integration.rs) pinned. +CORPUS_VERSION = 3 def main() -> None: diff --git a/tests/golden/identity/rust/test_rust_identity_parity.py b/tests/golden/identity/rust/test_rust_identity_parity.py index 99aba801..a5c29b1b 100644 --- a/tests/golden/identity/rust/test_rust_identity_parity.py +++ b/tests/golden/identity/rust/test_rust_identity_parity.py @@ -41,6 +41,15 @@ def test_corpus_meta_has_engine_scheme() -> None: assert meta["fingerprint_scheme"] == FINGERPRINT_SCHEME +def test_corpus_meta_version_matches_regen() -> None: + # The keystone panel's discipline gap: bumping regen.CORPUS_VERSION without + # actually regenerating (or vice versa) must fail loudly, not drift silently. + from golden.identity.rust import regen # type: ignore[import-not-found] + + meta = json.loads((_HERE / "corpus" / "META.json").read_text(encoding="utf-8")) + assert meta["corpus_version"] == regen.CORPUS_VERSION + + @pytest.mark.parametrize("name", sorted(_INPUTS)) def test_rust_identity_corpus_is_byte_identical(name: str, request: pytest.FixtureRequest) -> None: root = _INPUTS[name] @@ -76,6 +85,17 @@ def test_corpus_freezes_a_finding_per_rule_with_real_fingerprints() -> None: ) +def test_corpus_freezes_a_class2_out_route_finding() -> None: + # The class-2 fixture (tests/integration.rs, outside src/) must freeze a real + # FINDING whose qualname carries the reserved `.#out.` non-conformance branding — + # the route shape the rekey introduced cannot silently fall out of the corpus. + findings = _corpus("rustapp")["findings"] + out_rows = [f for f in findings if ".#out." in f["qualname"]] + assert out_rows, f"no `.#out.` qualname frozen (qualnames={[f['qualname'] for f in findings]})" + assert any(f["rule_id"] == "RS-WL-112" for f in out_rows), "class-2 RS-WL-112 finding not frozen" + assert any(f["qualname"].startswith("rust_app.#out.tests.integration") for f in out_rows) + + def test_corpus_freezes_the_impl_entity_surface() -> None: entities = _corpus("rustapp")["entities"] assert any(e["kind"] == "impl" for e in entities), "no impl entity row frozen" diff --git a/tests/unit/core/test_rekey_population.py b/tests/unit/core/test_rekey_population.py index db39dc36..69f75b7c 100644 --- a/tests/unit/core/test_rekey_population.py +++ b/tests/unit/core/test_rekey_population.py @@ -116,10 +116,10 @@ def test_rekey_policy_config_rule_id_matches_scanner() -> None: assert _POLICY_CONFIG_RULE_ID == SCANNER_PCID -def test_rs_wl_excluded_from_migration_pending_p5() -> None: - # P5-REVISIT (named decision, not an accident): RS-WL-* (Rust) DEFECTs never appear on - # rc4 (the plugin is in a worktree), so they are excluded from the migration population. - # When the worktree rebases, decide whether RS-WL DEFECTs enter the stores and migrate. +def test_rs_wl_included_in_the_migration_population() -> None: + # P5-REVISIT decided 2026-06-10 (identity keystone): Rust identity graduated — + # RS-WL-* DEFECTs are baseline-eligible, so a stored RS-WL verdict must migrate + # like any other DEFECT (the former exclusion became a live orphaning path). rs = Finding( rule_id="RS-WL-108", message="rust cmd injection", @@ -128,4 +128,4 @@ def test_rs_wl_excluded_from_migration_pending_p5() -> None: location=Location(path="m.rs", line_start=4), fingerprint="r" * 64, ) - assert not is_join_population(rs) + assert is_join_population(rs) diff --git a/tests/unit/rust/test_analyzer_protocol.py b/tests/unit/rust/test_analyzer_protocol.py index 868d4aac..0ddce22d 100644 --- a/tests/unit/rust/test_analyzer_protocol.py +++ b/tests/unit/rust/test_analyzer_protocol.py @@ -42,10 +42,10 @@ def test_analyze_over_files_finds_injection_with_repo_relative_path(tmp_path) -> # repo-relative POSIX path (the Filigree/Location anchor), not the absolute fs path. assert rs[0].location.path == "src/m.rs" # No Cargo.toml and no src/lib|main.rs anywhere -> no crate root registers, so the - # SP2 router falls back to the pre-SP2 mechanical route (crate=root.name, src/ NOT - # stripped) — byte-unchanged for no-Cargo trees. Real crate-prefixed routes are - # pinned in tests/unit/rust/test_crate_roots.py. - assert rs[0].qualname == f"{tmp_path.name}.src.m.run" + # SP2 router uses the class-3 route: constant "crate" segment + #out branding + + # literal relpath stems (relpath-pure — scan-root-name-independent). Real + # crate-prefixed routes are pinned in tests/unit/rust/test_crate_roots.py. + assert rs[0].qualname == "crate.#out.src.m.run" def test_last_context_is_none_but_rust_context_is_retained(tmp_path) -> None: @@ -141,7 +141,7 @@ def test_fn_struct_same_name_keeps_both_entities_and_counts_one_callable(tmp_pat assert [f.rule_id for f in rs] == ["RS-WL-108"] # the fn IS exercised ctx = analyzer.last_rust_context assert ctx is not None - qual = f"{tmp_path.name}.m.S" + qual = "crate.#out.m.S" # class-3 route: constant crate segment + #out branding # BOTH entities survive, addressable by their kind-disambiguated entity-id keys. assert f"rust:struct:{qual}" in ctx.entities assert f"rust:function:{qual}" in ctx.entities diff --git a/tests/unit/rust/test_crate_roots.py b/tests/unit/rust/test_crate_roots.py index b4136363..b4e5f096 100644 --- a/tests/unit/rust/test_crate_roots.py +++ b/tests/unit/rust/test_crate_roots.py @@ -152,10 +152,13 @@ def test_workspace_member_files_route_to_their_own_crates(tmp_path: Path) -> Non assert route(m2 / "src" / "main.rs", tmp_path, roots) == "m_two" -def test_out_of_src_files_get_the_crate_named_fallback_route(tmp_path: Path) -> None: +def test_out_of_src_files_get_the_out_branded_crate_route(tmp_path: Path) -> None: # Class 2: under a crate root but OUTSIDE its src/ — loomweave's emittable_scope # emits NOTHING here; wardline routes mechanically from the crate dir under the - # real crate name (documented: no cross-tool conformance claim, no collision risk). + # real crate name + the reserved `#out` segment (no cross-tool conformance claim; + # `#` appears only inside loomweave's `impl#<...>` discriminators, so the route + # can never collide with a class-1/loomweave locator). ALL stems are literal — + # no main/lib/mod collapsing. analyzer = _analyzer_module() crate = _write_crate(tmp_path, "c", '[package]\nname = "c-app"\n') (crate / "tests").mkdir() @@ -164,14 +167,35 @@ def test_out_of_src_files_get_the_crate_named_fallback_route(tmp_path: Path) -> roots = discover_crate_roots(tmp_path) route = analyzer._module_for # noqa: SLF001 - assert route(crate / "build.rs", tmp_path, roots) == "c_app.build" - assert route(crate / "tests" / "integration.rs", tmp_path, roots) == "c_app.tests.integration" + assert route(crate / "build.rs", tmp_path, roots) == "c_app.#out.build" + assert route(crate / "tests" / "integration.rs", tmp_path, roots) == "c_app.#out.tests.integration" -def test_no_crate_files_keep_the_pre_sp2_mechanical_route(tmp_path: Path) -> None: - # Class 3: no owning crate root anywhere — today's entire preview population. - # The route is BYTE-IDENTICAL to the pre-SP2 behavior (crate = scan-root name, - # src_root = scan root, no src/ strip) so the preview scan population is unchanged. +def test_class2_route_cannot_collide_with_an_in_src_twin(tmp_path: Path) -> None: + # The keystone panel's collision repro, inverted: /tests/integration.rs + # (class 2) and /src/tests/integration.rs (class 1) used to mint the SAME + # qualname prefix (`c_app.tests.integration`). The `#out` branding separates them. + analyzer = _analyzer_module() + crate = _write_crate(tmp_path, "c", '[package]\nname = "c-app"\n') + (crate / "src" / "tests").mkdir() + (crate / "src" / "tests" / "integration.rs").write_text("", encoding="utf-8") + (crate / "tests").mkdir() + (crate / "tests" / "integration.rs").write_text("", encoding="utf-8") + roots = discover_crate_roots(tmp_path) + + route = analyzer._module_for # noqa: SLF001 + in_src = route(crate / "src" / "tests" / "integration.rs", tmp_path, roots) + out_of_src = route(crate / "tests" / "integration.rs", tmp_path, roots) + assert in_src == "c_app.tests.integration" # class 1: the conformance-bearing route + assert out_of_src == "c_app.#out.tests.integration" # class 2: #out-branded + assert in_src != out_of_src + + +def test_no_crate_files_get_the_relpath_pure_constant_crate_route(tmp_path: Path) -> None: + # Class 3: no owning crate root anywhere. The crate segment is the CONSTANT + # "crate" (cargo forbids the keyword as a package name, so it cannot collide + # with class 1) + the `#out` branding + literal relpath stems — relpath-pure, + # scan-root-name-INDEPENDENT (renaming the root directory does not rekey). analyzer = _analyzer_module() (tmp_path / "src").mkdir() (tmp_path / "src" / "m.rs").write_text("", encoding="utf-8") @@ -179,7 +203,7 @@ def test_no_crate_files_keep_the_pre_sp2_mechanical_route(tmp_path: Path) -> Non assert roots.crate_dir_for(tmp_path / "src" / "m.rs") is None # no lib.rs/main.rs -> no root got = analyzer._module_for(tmp_path / "src" / "m.rs", tmp_path, roots) # noqa: SLF001 - assert got == f"{tmp_path.name}.src.m" + assert got == "crate.#out.src.m" # NOT f"{tmp_path.name}..." — root-name-independent def test_scan_coverage_is_not_narrowed_to_emittable_scope(tmp_path: Path) -> None: @@ -210,6 +234,6 @@ def test_scan_coverage_is_not_narrowed_to_emittable_scope(tmp_path: Path) -> Non ] by_path = {f.location.path: f.qualname for f in rs} assert by_path["c/src/cmd.rs"] == "c_app.cmd.run" # class 1: the oracle route - assert by_path["c/build.rs"] == "c_app.build.run" # class 2: crate-named fallback - assert by_path["c/tests/integration.rs"] == "c_app.tests.integration.run" - assert by_path["bare/m.rs"] == f"{tmp_path.name}.bare.m.run" # class 3: pre-SP2 route + assert by_path["c/build.rs"] == "c_app.#out.build.run" # class 2: #out-branded crate route + assert by_path["c/tests/integration.rs"] == "c_app.#out.tests.integration.run" + assert by_path["bare/m.rs"] == "crate.#out.bare.m.run" # class 3: relpath-pure constant crate diff --git a/tests/unit/rust/test_fingerprint_stability.py b/tests/unit/rust/test_fingerprint_stability.py new file mode 100644 index 00000000..ee841a56 --- /dev/null +++ b/tests/unit/rust/test_fingerprint_stability.py @@ -0,0 +1,140 @@ +"""Rust fingerprint stability — the wlfp2 move-stability contract for RS-WL-*. + +The Rust analogue of ``tests/unit/core/test_fingerprint_stability.py`` (identity +keystone panel, rust-sp2-2026-06-10). The fingerprint inputs are +``(rule_id, path, qualname, taint_path)`` — ``line_start`` is NOT hashed (wlfp2, +wardline-8654423823), so the RS-WL-* discriminator must be ENTITY-RELATIVE: + + * **Whole-entity moves** — a comment at the top of the file, a comment directly + above the function, an unrelated sibling ``fn`` added ABOVE — shift every + absolute line AND every pre-order ``NodeId`` below them, but keep the + fingerprint, because both the line and the NodeId fold as deltas against the + containing entity's own anchor (``line - entity_line_start``, + ``trigger_node_id - entity_node_id``). + * **In-entity edits** — a statement inserted INSIDE the function ABOVE the + trigger — DO change the fingerprint (the trigger's offset relative to its own + fn moved). Entity-relative, not move-stable in the strong sense: the contract + is identical-source -> identical-fingerprint, and that edit is not identical + source. Asserted ``!=`` here to document the accepted boundary. + * **Same-line twins** — two distinct triggers on one physical line keep DISTINCT + fingerprints (the relative-NodeId fold is the sole same-line discriminant; see + also test_rules.test_two_commands_on_one_line_get_distinct_fingerprints). + +Plus the FIX-4 class-3 route repro (the panel's exp_e4e): a manifest-less tree's +identity is relpath-pure — renaming the scan-root DIRECTORY must not rekey. +""" + +from __future__ import annotations + +from pathlib import Path + +import pytest + +pytest.importorskip("tree_sitter", reason="wardline[rust] extra not installed") + +from wardline.core.config import WardlineConfig # noqa: E402 +from wardline.rust.analyzer import RustAnalyzer # noqa: E402 + +_TRUSTED = "/// @trusted(level=ASSURED)\n" + +# One RS-WL-108 (tainted program) + one RS-WL-112 (tainted arg on a `sh -c` line), +# in separate fns so each rule's fingerprint is tracked independently. +_BASE = ( + _TRUSTED + + "fn prog() {\n" + + ' let t = std::env::var("X").unwrap();\n' + + " Command::new(t).output();\n" + + "}\n" + + _TRUSTED + + "fn shell() {\n" + + ' let t = std::env::var("X").unwrap();\n' + + ' Command::new("sh").arg("-c").arg(t).output();\n' + + "}\n" +) + + +def _fingerprints(source: str) -> dict[str, str]: + findings = RustAnalyzer().analyze_source(source, module="demo.m", path="src/m.rs") + out = {f.rule_id: f.fingerprint for f in findings if f.rule_id.startswith("RS-WL-")} + assert set(out) == {"RS-WL-108", "RS-WL-112"}, f"fixture must fire both rules, got {sorted(out)}" + return out + + +def test_comment_at_top_of_file_keeps_both_fingerprints() -> None: + # Shifts every absolute line AND every pre-order NodeId in the file; both fold + # entity-relative, so the join key must not churn (the panel's critical repro). + base = _fingerprints(_BASE) + shifted = _fingerprints("// release notes header\n\n" + _BASE) + assert base == shifted + + +def test_comment_directly_above_the_function_keeps_both_fingerprints() -> None: + base = _fingerprints(_BASE) + shifted = _fingerprints(_BASE.replace(_TRUSTED + "fn prog", "// reviewed 2026-06-10\n" + _TRUSTED + "fn prog")) + assert base == shifted + + +def test_unrelated_sibling_fn_added_above_keeps_both_fingerprints() -> None: + # A whole sibling item above shifts NodeIds by MANY (not one) — the delta, not + # the absolute index, is what must be stable. + base = _fingerprints(_BASE) + shifted = _fingerprints("fn helper() -> i32 {\n 1 + 2\n}\n" + _BASE) + assert base == shifted + + +def test_statement_inserted_inside_the_fn_above_the_trigger_changes_fingerprint() -> None: + # The accepted entity-relative limitation (mirrors the Python contract): an edit + # INSIDE the entity above the trigger moves its relative offset -> rekey. + base = _fingerprints(_BASE) + mutated = _fingerprints(_BASE.replace("fn prog() {\n", "fn prog() {\n let _pad = 0;\n")) + assert base["RS-WL-108"] != mutated["RS-WL-108"] + + +def test_two_same_line_triggers_keep_distinct_fingerprints() -> None: + # The discriminator's original purpose survives the rekey: identical + # (rule, path, qualname, relative line) twins split on the relative NodeId. + src = ( + _TRUSTED + + "fn f() {\n" + + ' let t = std::env::var("X").unwrap();\n' + + " Command::new(t).output(); Command::new(t).spawn();\n" + + "}\n" + ) + findings = RustAnalyzer().analyze_source(src, module="demo.m", path="src/m.rs") + fps = [f.fingerprint for f in findings if f.rule_id == "RS-WL-108"] + assert len(fps) == 2 and len(set(fps)) == 2 + + +def test_display_taint_path_keeps_absolute_lines() -> None: + # properties["taint_path"] is the DISPLAY surface — human-readable, absolute lines + # exactly as before the rekey. Only the fingerprint discriminator went relative. + findings = RustAnalyzer().analyze_source(_BASE, module="demo.m", path="src/m.rs") + by_rule = {f.rule_id: f for f in findings if f.rule_id.startswith("RS-WL-")} + assert by_rule["RS-WL-108"].properties["taint_path"] == "EXTERNAL_RAW->Command::new(program)@L4->exec@L4" + assert by_rule["RS-WL-112"].properties["taint_path"] == "EXTERNAL_RAW->arg->'sh -c'->exec@L9" + + +# --- FIX 4: class-3 (no crate root) identity is relpath-pure ----------------- + + +def _scan_fingerprints(root: Path) -> dict[str, tuple[str, str]]: + files = sorted(root.rglob("*.rs")) + findings = RustAnalyzer().analyze(files, WardlineConfig(), root=root) + return {f.rule_id: (f.qualname or "", f.fingerprint) for f in findings if f.rule_id.startswith("RS-WL-")} + + +def test_class3_fingerprints_survive_a_scan_root_directory_rename(tmp_path: Path) -> None: + # The panel's exp_e4e repro: same content, two scan-root directory NAMES. A + # manifest-less (class-3) tree's crate segment is the CONSTANT "crate", so the + # qualname/fingerprint must be byte-identical across the rename. + src = _TRUSTED + 'fn run() {\n let t = std::env::var("X").unwrap();\n Command::new(t).output();\n}\n' + for name in ("alpha", "beta"): + d = tmp_path / name / "bin" + d.mkdir(parents=True) + (d / "app.rs").write_text(src, encoding="utf-8") + + alpha = _scan_fingerprints(tmp_path / "alpha") + beta = _scan_fingerprints(tmp_path / "beta") + assert alpha["RS-WL-108"] == beta["RS-WL-108"] + # And the route is the decided class-3 shape: constant crate segment + #out branding. + assert alpha["RS-WL-108"][0] == "crate.#out.bin.app.run" diff --git a/tests/unit/rust/test_rust_identity_graduated.py b/tests/unit/rust/test_rust_identity_graduated.py index 3822b680..0ce9d865 100644 --- a/tests/unit/rust/test_rust_identity_graduated.py +++ b/tests/unit/rust/test_rust_identity_graduated.py @@ -13,14 +13,15 @@ from __future__ import annotations -from datetime import date +from datetime import UTC, date, datetime import pytest from wardline.core.baseline import Baseline, build_baseline_document from wardline.core.finding import Finding, Kind, Location, Severity, SuppressionState -from wardline.core.suppression import apply_suppressions -from wardline.core.waivers import WaiverSet +from wardline.core.judged import JudgedFP, JudgedSet +from wardline.core.suppression import apply_suppressions, gate_trips +from wardline.core.waivers import Waiver, WaiverSet def _rs_finding(*, stray_provisional_prop: bool = False) -> Finding: @@ -51,6 +52,44 @@ def test_rs_finding_is_captured_in_a_generated_baseline() -> None: assert [e["fingerprint"] for e in doc["entries"]] == ["a" * 64] +def test_rs_finding_is_suppressed_by_a_matching_waiver() -> None: + # Graduation covers the WHOLE suppression machinery, not just the baseline leg: + # a hand-authored waiver keyed to the RS-WL fingerprint must resolve WAIVED + # (waiver > judged > baseline precedence lives in resolve_identity). + f = _rs_finding() + waivers = WaiverSet([Waiver(fingerprint="a" * 64, reason="accepted: sandboxed CLI", expires=None)]) + (out,) = apply_suppressions([f], Baseline(frozenset()), waivers, today=date(2026, 6, 10)) + assert out.suppressed is SuppressionState.WAIVED + assert out.suppression_reason # the waiver's reason travels with the verdict + + +def test_rs_finding_is_suppressed_by_a_judged_false_positive() -> None: + f = _rs_finding() + judged = JudgedSet( + [ + JudgedFP( + fingerprint="a" * 64, + rule_id="RS-WL-108", + path="src/m.rs", + message="program injection", + rationale="the program is a vetted constant at runtime", + model_id="test-judge", + confidence=0.97, + recorded_at=datetime(2026, 6, 10, tzinfo=UTC), + policy_hash="p" * 8, + ) + ] + ) + (out,) = apply_suppressions([f], Baseline(frozenset()), WaiverSet([]), today=date(2026, 6, 10), judged=judged) + assert out.suppressed is SuppressionState.JUDGED + + +def test_active_rs_error_trips_the_gate() -> None: + # An ACTIVE (unsuppressed) RS-WL-108 ERROR participates in --fail-on like any + # Python defect — graduated findings are gate citizens, not advisory. + assert gate_trips([_rs_finding()], Severity.ERROR) is True + + def test_producer_no_longer_emits_the_provisional_property() -> None: # The real RS-WL-* producer (rules.py) emits no retired identity flag. pytest.importorskip("tree_sitter", reason="wardline[rust] extra not installed") From aac2379c9242057c1ece1d25b418bd23a31564fb Mon Sep 17 00:00:00 2001 From: John Morrissey Date: Wed, 10 Jun 2026 15:12:16 +1000 Subject: [PATCH 090/186] docs(integration): record owner sign-off on the ADR-049 amendment-request letter ADR-049 amended same day (loomweave rc4 65be609, amendment 4). Corpus rows + both-producer implementation are next-sprint work; wardline-be5ee9cc34 and the const-spacing half of wardline-e8f7c0508f stay blocked on the rows, not the decision. Co-Authored-By: Claude Opus 4.8 (1M context) --- ...06-10-wardline-loomweave-rust-qualname-amendment-requests.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/integration/2026-06-10-wardline-loomweave-rust-qualname-amendment-requests.md b/docs/integration/2026-06-10-wardline-loomweave-rust-qualname-amendment-requests.md index c8a5e746..cec7f7ce 100644 --- a/docs/integration/2026-06-10-wardline-loomweave-rust-qualname-amendment-requests.md +++ b/docs/integration/2026-06-10-wardline-loomweave-rust-qualname-amendment-requests.md @@ -4,7 +4,7 @@ **To:** Loomweave maintainers (Rust plugin) **Date:** 2026-06-10 **Re:** Two un-decided dialect gaps both producers share — (1) reserved-colon path-typed generic args, (2) const-generic-arg spacing — plus the record of the amend-3 corpus tail your `rc4` now carries. -**Status:** **DRAFT — awaiting Wardline-owner sign-off before sending; proposal only, neither producer implements until ADR-049 is amended and corpus rows land.** +**Status:** **SIGNED OFF — Wardline owner (john@foundryside.dev), 2026-06-10. ADR-049 amended same day (loomweave `rc4`, amendment 4: one shared `escape_reserved(strip_ws(arg))` pipeline for every concrete generic argument, type or const). Implementation + the three corpus rows land next sprint; both producers conform in lockstep once the rows exist (`wardline-be5ee9cc34`, const-spacing half of `wardline-e8f7c0508f`, blocker `wardline-e3e9e109ba`).** --- From cb47c8a307a523bd2ee041ba77828d6637e55fa0 Mon Sep 17 00:00:00 2001 From: John Morrissey Date: Thu, 11 Jun 2026 04:42:07 +1000 Subject: [PATCH 091/186] =?UTF-8?q?feat(python)!:=20full-plugin=20hardenin?= =?UTF-8?q?g=20=E2=80=94=20engine=20soundness,=20boundary=20partition,=206?= =?UTF-8?q?=20sink-family=20expansions,=206=20new=20rules,=20fail-closed?= =?UTF-8?q?=20analyzer?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Empirical multi-agent evaluation + fix program over src/wardline/scanner (3 confirmed tracker bugs, 29 verified review findings, 8 expansion tickets, 22 panel findings). Engine soundness (variable_level.py): - Attribute writes recorded DURING the main L2 walk against per-statement var_taints (wardline-b369c7d06c: final-state launder + branch-unaware receiver types closed); post-hoc collect_attribute_writes deleted; receiver class tracking is set-valued and branch-aware (strong update straight-line, union at joins, zero-trip loop arm) - Typed-receiver RAW guard extended to the var_types Type.method dispatch (wardline-03c8805449) - ast.Compare/Raise/Assert/Delete expression resolution (closed a PY-WL-105 FN class and the phantom flow-insensitive fallbacks); receiver-type invalidation on for/with/walrus/ unpack rebinds; container builtins + str.format_map propagate worst-arg; nested-tuple unpack keeps taint; except-handler seeded from worst try-prefix; lambda **kw spread + omitted-param default seeding; unresolved bare-name calls propagate worst(seed, args) (trusted-seed launder closed); io.StringIO/BytesIO ctors propagate arg taint - Module-level channels: module bindings (callable aliases, constructed clients) resolve in sink rules; module-global raw seeds flow read+write direction (wardline-66b2c91470) Boundary-integrity partition (wardline-718048a518): exactly one of 102/111/113/119 per defective boundary — 113 enforces its rejection-exists premise; 119 wins bare return-p; one-hop same-module raising helpers, raising conversions (int/Enum/subscript), and IfExp falsy returns count as rejection (5 verified ERROR-FP classes closed); 113 requires the rejection to be swallowable by the matching handler. Sink families: 118 executescript + receiver heuristic + constant sqlalchemy text() exemption (parameterized-query FP closed); 117 construct-then-method (httpx/requests Session/aiohttp) + URL-position ArgSpec (tainted timeout no longer fires); 106 Unpickler/shelve + curated third-party table (dill/jsonpickle/joblib/torch/numpy allow_pickle); 115 runpy/importlib.util; 108 program-exec family (os.exec*/spawn*, posix_spawn, pty.spawn) + shlex.quote fragments GUARDED; 112 callable-alias bindings; 116 mutation/Path-method/archive-extraction sinks (Zip Slip; tarfile filter='data' exempt). PY-WL-108/112 base severity WARN -> ERROR (same exploit class as SQLi). New PREVIEW rules: PY-WL-121 XXE, 122 SSTI, 123 reflection injection, 124 native-library load, 125 log injection, 126 mail injection. Six parallel sink-check loops consolidated into one binding-aware TaintedSinkRule.check with SINK_SPECS/severity tables. Analyzer robustness (fail-closed): parse failures + per-file analysis aborts are now gate-eligible ERROR DEFECTs (BREAKING: previously silent exit 0); MemoryError escapes isolation; L2 fixpoint memo key is O(per-function) (was O(project) per function — the whole-scan O(n^2) hotspot) + callgraph known_fqns hoist; flow-insensitive fallback is a one-per-scan FACT finding instead of a UserWarning; WLN-CONFIG-SANITISER-SINK-COLLISION diagnostic; PY-WL-103/104 nested-def tier fixed; PY-WL-110/114 unified on the engine marker recognizer; PY-WL-120 receiver-aware (StringIO FP) + enablement-aware delegation. Verification: property/differential harness around the _resolve_expr/_resolve_call chokepoint (lattice monotonicity, idempotence, seed monotonicity, receiver guard — wardline-369f54b83b) + CHOKEPOINT_NOTES.md contextvar map; FALSE_POSITIVE corpus sentinels (wardline-2561197999); full 26-rule positive/negative/tier-suppression matrix; 26-rule vocabulary pin; output-determinism golden; goldens regenerated (severity calibration + corpus reshape). Docs: rules.md rewritten for 26 rules + CHANGELOG breaking notes; judge prompt catalogue updated to the full rule surface. Suite: 3641 passed (was 3133); ruff/mypy/mkdocs-strict clean; self-scan gate clean. Co-Authored-By: Claude Fable 5 --- CHANGELOG.md | 105 +++ docs/concepts/model.md | 7 +- docs/concepts/rules.md | 665 +++++++++++++-- docs/concepts/taint-algebra.md | 6 +- docs/concepts/trust-vocabulary-convergence.md | 2 +- src/wardline/core/judge.py | 73 +- src/wardline/core/run.py | 8 +- src/wardline/scanner/analyzer.py | 497 +++++++++-- src/wardline/scanner/context.py | 26 + src/wardline/scanner/grammar.py | 49 +- src/wardline/scanner/pipeline.py | 45 +- src/wardline/scanner/rules/__init__.py | 13 + src/wardline/scanner/rules/_ast_helpers.py | 216 ++++- src/wardline/scanner/rules/_sink_helpers.py | 703 +++++++++++++-- .../scanner/rules/assert_only_boundary.py | 23 +- .../rules/boundary_without_rejection.py | 40 +- src/wardline/scanner/rules/broad_exception.py | 9 +- .../scanner/rules/degenerate_boundary.py | 33 +- .../scanner/rules/failopen_boundary.py | 57 +- .../scanner/rules/invalid_decorator_level.py | 43 +- src/wardline/scanner/rules/none_leak.py | 11 +- src/wardline/scanner/rules/path_traversal.py | 173 +++- .../scanner/rules/silent_exception.py | 9 +- src/wardline/scanner/rules/sql_injection.py | 238 +++++- src/wardline/scanner/rules/ssrf.py | 104 ++- src/wardline/scanner/rules/stored_taint.py | 112 ++- .../scanner/rules/untrusted_to_command.py | 158 +++- .../rules/untrusted_to_deserialization.py | 146 +++- .../scanner/rules/untrusted_to_exec.py | 32 +- .../scanner/rules/untrusted_to_import.py | 29 +- .../scanner/rules/untrusted_to_log.py | 74 ++ .../scanner/rules/untrusted_to_mail.py | 70 ++ .../scanner/rules/untrusted_to_native.py | 63 ++ .../scanner/rules/untrusted_to_reflection.py | 66 ++ .../rules/untrusted_to_shell_subprocess.py | 11 +- .../scanner/rules/untrusted_to_template.py | 66 ++ .../scanner/rules/untrusted_to_xml.py | 87 ++ src/wardline/scanner/taint/callgraph.py | 8 +- .../scanner/taint/module_summariser.py | 101 ++- src/wardline/scanner/taint/variable_level.py | 803 +++++++++++++----- tests/corpus/MANIFEST.yaml | 23 +- tests/corpus/fixtures/validators.py | 10 +- tests/corpus/harness.py | 60 +- .../sentinels/clean_boundary_rejection.py | 10 + tests/corpus/sentinels/clean_command_const.py | 21 + tests/corpus/sentinels/clean_exec_const.py | 8 + .../sentinels/clean_sql_parameterized.py | 18 + tests/corpus/test_fp_rate.py | 45 +- tests/golden/identity/corpus/META.json | 2 +- tests/golden/identity/corpus/sampleapp.json | 4 +- tests/golden/identity/corpus/sinks.json | 6 +- tests/golden/identity/corpus/stress.json | 2 +- tests/grammar/golden/builtin_findings.jsonl | 10 +- tests/grammar/test_analyzer_wiring.py | 6 + tests/grammar/test_grammar_model.py | 6 + tests/grammar/test_output_determinism.py | 56 ++ .../rules/test_assert_only_boundary.py | 43 +- tests/unit/scanner/rules/test_ast_helpers.py | 161 ++++ .../scanner/rules/test_boundary_partition.py | 213 +++++ .../rules/test_boundary_without_rejection.py | 145 +++- .../scanner/rules/test_broad_exception.py | 77 ++ .../scanner/rules/test_command_expansion.py | 334 ++++++++ .../scanner/rules/test_default_registry.py | 6 + .../rules/test_deserialization_expansion.py | 366 ++++++++ .../scanner/rules/test_discriminator_shape.py | 8 +- .../unit/scanner/rules/test_exec_expansion.py | 158 ++++ .../scanner/rules/test_failopen_boundary.py | 130 +++ ...test_invalid_decorator_level_recognizer.py | 148 ++++ tests/unit/scanner/rules/test_none_leak.py | 16 + .../rules/test_path_traversal_expansion.py | 387 +++++++++ .../scanner/rules/test_review_fixups_rules.py | 287 +++++++ .../scanner/rules/test_silent_exception.py | 77 ++ .../unit/scanner/rules/test_sink_machinery.py | 484 +++++++++++ tests/unit/scanner/rules/test_sink_rules.py | 31 +- .../rules/test_sql_injection_expansion.py | 394 +++++++++ .../unit/scanner/rules/test_ssrf_expansion.py | 296 +++++++ .../rules/test_stored_taint_expansion.py | 274 ++++++ .../rules/test_taint_closures_locked.py | 23 +- .../scanner/rules/test_tier_gate_negative.py | 4 +- .../scanner/rules/test_untrusted_to_import.py | 128 ++- .../scanner/rules/test_untrusted_to_log.py | 130 +++ .../scanner/rules/test_untrusted_to_mail.py | 145 ++++ .../scanner/rules/test_untrusted_to_native.py | 100 +++ .../rules/test_untrusted_to_reflection.py | 117 +++ .../rules/test_untrusted_to_template.py | 135 +++ .../scanner/rules/test_untrusted_to_xml.py | 174 ++++ .../rules/test_vocabulary_shape_pin.py | 52 ++ .../rules/test_wave2_engine_precision.py | 30 +- .../rules/test_wave3_deferred_rules.py | 21 + tests/unit/scanner/taint/CHOKEPOINT_NOTES.md | 161 ++++ .../taint/test_attribute_write_flow.py | 433 ++++++++++ .../scanner/taint/test_decorator_provider.py | 6 +- .../scanner/taint/test_engine_precision.py | 610 +++++++++++++ .../scanner/taint/test_module_global_taint.py | 170 ++++ tests/unit/scanner/taint/test_propagation.py | 35 +- .../scanner/taint/test_property_chokepoint.py | 441 ++++++++++ .../taint/test_review_fixups_engine.py | 371 ++++++++ .../unit/scanner/taint/test_variable_level.py | 11 +- tests/unit/scanner/test_analyzer.py | 77 +- tests/unit/scanner/test_analyzer_isolation.py | 296 +++++++ .../test_grammar_sanitiser_collision.py | 37 + tests/unit/scanner/test_module_bindings.py | 149 ++++ tests/unit/scanner/test_pipeline.py | 74 ++ 103 files changed, 12469 insertions(+), 764 deletions(-) create mode 100644 src/wardline/scanner/rules/untrusted_to_log.py create mode 100644 src/wardline/scanner/rules/untrusted_to_mail.py create mode 100644 src/wardline/scanner/rules/untrusted_to_native.py create mode 100644 src/wardline/scanner/rules/untrusted_to_reflection.py create mode 100644 src/wardline/scanner/rules/untrusted_to_template.py create mode 100644 src/wardline/scanner/rules/untrusted_to_xml.py create mode 100644 tests/corpus/sentinels/clean_boundary_rejection.py create mode 100644 tests/corpus/sentinels/clean_command_const.py create mode 100644 tests/corpus/sentinels/clean_exec_const.py create mode 100644 tests/corpus/sentinels/clean_sql_parameterized.py create mode 100644 tests/grammar/test_output_determinism.py create mode 100644 tests/unit/scanner/rules/test_boundary_partition.py create mode 100644 tests/unit/scanner/rules/test_command_expansion.py create mode 100644 tests/unit/scanner/rules/test_deserialization_expansion.py create mode 100644 tests/unit/scanner/rules/test_exec_expansion.py create mode 100644 tests/unit/scanner/rules/test_invalid_decorator_level_recognizer.py create mode 100644 tests/unit/scanner/rules/test_path_traversal_expansion.py create mode 100644 tests/unit/scanner/rules/test_review_fixups_rules.py create mode 100644 tests/unit/scanner/rules/test_sink_machinery.py create mode 100644 tests/unit/scanner/rules/test_sql_injection_expansion.py create mode 100644 tests/unit/scanner/rules/test_ssrf_expansion.py create mode 100644 tests/unit/scanner/rules/test_stored_taint_expansion.py create mode 100644 tests/unit/scanner/rules/test_untrusted_to_log.py create mode 100644 tests/unit/scanner/rules/test_untrusted_to_mail.py create mode 100644 tests/unit/scanner/rules/test_untrusted_to_native.py create mode 100644 tests/unit/scanner/rules/test_untrusted_to_reflection.py create mode 100644 tests/unit/scanner/rules/test_untrusted_to_template.py create mode 100644 tests/unit/scanner/rules/test_untrusted_to_xml.py create mode 100644 tests/unit/scanner/taint/CHOKEPOINT_NOTES.md create mode 100644 tests/unit/scanner/taint/test_attribute_write_flow.py create mode 100644 tests/unit/scanner/taint/test_engine_precision.py create mode 100644 tests/unit/scanner/taint/test_module_global_taint.py create mode 100644 tests/unit/scanner/taint/test_property_chokepoint.py create mode 100644 tests/unit/scanner/taint/test_review_fixups_engine.py create mode 100644 tests/unit/scanner/test_analyzer_isolation.py create mode 100644 tests/unit/scanner/test_grammar_sanitiser_collision.py create mode 100644 tests/unit/scanner/test_module_bindings.py diff --git a/CHANGELOG.md b/CHANGELOG.md index 2eed29f0..39f96260 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -8,6 +8,75 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] ### Added +- **Six new PREVIEW sink rules, `PY-WL-121`–`PY-WL-126`** (the 2026-06-10 + coverage-gap families; all tier-modulated, argument-slot precise, and + construct-then-method / callable-alias aware): + - `PY-WL-121` — untrusted data reaches an **XML parsing** sink (CWE-611); + per-sink severity calibrated to default parser posture (`lxml.etree.*` at + ERROR — entity-resolving by default; stdlib etree/minidom/sax at WARN — + billion-laughs DoS only since CPython 3.7.1). Only the document slot fires. + - `PY-WL-122` — untrusted data **compiled into a server-side template** + (jinja2 `Template`/`Environment.from_string`, mako `Template`; SSTI, + CWE-1336, ERROR). A tainted *render variable* is the safe idiom and never fires. + - `PY-WL-123` — untrusted **attribute NAME** in `setattr`/`getattr` + (dynamic attribute injection / mass assignment, CWE-915, WARN). Fixed-name + writes of untrusted *values* stay silent. + - `PY-WL-124` — untrusted path reaches a **native-library load** + (`ctypes.CDLL`/`WinDLL`/`OleDLL`/`PyDLL`, `ctypes.cdll.LoadLibrary`; + CWE-114, ERROR). + - `PY-WL-125` — untrusted data as the **log MESSAGE format string** + (`logging.*` module-level and Logger-method forms; CWE-117, INFO — visible + to an explicit `--fail-on INFO` without tripping the default gate). The + lazy `%`-args parameterization (`logging.info('u=%s', raw)`) never fires. + - `PY-WL-126` — untrusted **recipient/message** in `smtplib` + `SMTP`/`SMTP_SSL` `.sendmail` (mail/CRLF header injection, CWE-93, WARN). +- **Per-file isolation: `WLN-ENGINE-FILE-FAILED`.** An unexpected exception + while analyzing one file no longer aborts the whole scan (losing every other + file's findings) — and is not a silent skip either: the scan continues and the + failed file is named by a gate-eligible `WLN-ENGINE-FILE-FAILED` ERROR defect, + counted toward `ScanSummary.unanalyzed` (the Rust frontend's per-file contract, + now on the Python path). +- **New config diagnostic `WLN-CONFIG-SANITISER-SINK-COLLISION`.** A configured + sanitiser that collides with a built-in serialisation sink of the same name + (e.g. declaring `pickle.loads` a sanitiser) can never take effect — the + conservative sink classification wins — yet it previously also suppressed + `WLN-CONFIG-UNUSED-SANITISER`, making the dead declaration a silent no-op. One + FACT per colliding sanitiser now names the collision so the operator learns + their suppression attempt was overridden, not honoured. +- **Sink-family expansions across the existing sink rules** (each fires on more + real-world shapes; shared machinery in `_sink_helpers` — construct-then-method + receiver resolution, callable-alias bindings, `ArgSpec` argument-position + matching, and a fail-closed per-argument taint resolver): + - `PY-WL-118` (SQLi) adds **`executescript`** (sqlite3 cursor AND connection — + multi-statement, no parameter binding at all), a fail-closed **receiver + heuristic** (DB-driver binding/name evidence fires, executor/pool evidence + suppresses, unknown receivers fire), and the **constant `text()` clause + exemption** for the canonical SQLAlchemy parameterized pattern. + - `PY-WL-117` (SSRF) now resolves **constructed client/session instance + methods** (`httpx.Client()`, `requests.Session()`, `aiohttp.ClientSession`, + chained and `with`-bound forms) plus client `base_url=`, and is **URL-slot + precise** — a tainted `timeout=`/`headers=` with a clean URL no longer fires. + - `PY-WL-116` (path traversal) adds the **filesystem-mutation** APIs + (`os.remove`/`rename`/`makedirs`/…, `shutil.rmtree`/`copy*`/`move`), + **`pathlib.Path` methods** on a tainted-constructed `Path`, and **archive + extraction** (`tarfile`/`zipfile` `extract`/`extractall` — Zip Slip), with a + literal **`filter="data"` exemption** (tarfile's safe extraction filter). + - `PY-WL-106` (deserialization) adds the **OO streaming-unpickle API** + (`pickle.Unpickler(stream).load()`), **`shelve.open`** (path slot only), and + a curated **third-party CWE-502 table** (`dill`, `jsonpickle.decode`, + `joblib.load`, `torch.load`, `numpy.load`) with two literal-keyword gates: + `numpy.load` fires only with a literal `allow_pickle=True`; `torch.load` is + suppressed by a literal `weights_only=True`. + - `PY-WL-115` (dynamic import) adds **`runpy.run_path`/`run_module`** and + **`importlib.util.spec_from_file_location`** (import-and-execute class). + - `PY-WL-108` (command execution) adds the **argv-style program-execution + family** (`os.exec*`/`os.spawn*`/`os.posix_spawn*`/`pty.spawn`) and decides + **`shlex.quote` GUARDED semantics**: a quote call as a fragment of a + constant-shaped concatenation/f-string guards the always-shell sinks + (`os.system("echo " + shlex.quote(raw))` is clean); a bare whole-command + quote still fires, and the guard never applies to the argv sinks. + - `PY-WL-107` (eval/exec/compile) adds the `builtins.` and `__builtins__.` + spellings. - **Rust support.** A new `--lang rust` frontend (behind the `wardline[rust]` extra: tree-sitter, no base dependency) sweeps `*.rs` and flags command-injection trust-boundary defects — `RS-WL-108` (program injection, ERROR: @@ -89,6 +158,42 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 through cleanly to the legacy/off rungs (emit stays soft-fail) (weft-23574069a1). ### Changed +- **BREAKING (gate): parse failures are now gate-eligible.** + `WLN-ENGINE-PARSE-ERROR` (a discovered file that could not be read/parsed) is + promoted from a NONE FACT to an **ERROR DEFECT**: its sinks were never + analyzed, so a default `--fail-on ERROR` reading green over it was a fail-open + (e.g. a latin-1 coding cookie CPython runs but the UTF-8 reader rejects hid + live code from the scan). Baseline/waiver still annotate it but cannot clear + the secure gate; `--trust-suppressions` can (an explicit operator trust + decision). A tree with unparseable files now trips `--fail-on ERROR` until the + files are fixed or the suppression is explicitly trusted. (Python frontend; + the Rust frontend's parse-error surfacing is unchanged — still a FACT.) +- **BREAKING (severity): `PY-WL-108` and `PY-WL-112` base severity WARN → ERROR.** + Tainted command/program execution (always-shell APIs, argv exec/spawn, + literal-`shell=True` subprocess) is the same blast-radius exploit class as + SQLi (CWE-78 ≅ CWE-89), so both calibrate with `PY-WL-118`'s ERROR. Findings + from these rules now trip a default `--fail-on ERROR` gate in fully-trusted + functions; override per project via `rules.severity` if needed. +- **BREAKING (fingerprints): the boundary-integrity family now partitions + four ways — bare `return p` boundaries re-key from `PY-WL-102` to `PY-WL-119`.** + At most one of {102, 111, 113, 119} fires per boundary: 119 wins the bare + degenerate shape (`return `), 102 keeps every other no-rejection shape, + 111 keeps assert-only (including an assert inside a substituting `try` — + documented precedence over 113), 113 fires only when a real rejection exists + and a fail-open handler can swallow it. Because `rule_id` is part of the + fingerprint, **baselined/waived/judged `PY-WL-102` entries for bare `return p` + boundaries go stale** (the finding now carries `PY-WL-119`) — re-baseline / + re-waive once after upgrade. The same boundary is no longer double-counted at + ERROR in the gate population. +- **`PY-WL-102`/`PY-WL-111` recognise more genuine rejection shapes (fewer + FPs on real validators):** a ONE-HOP same-module call to a raising helper + (a factored-out validator, raising staticmethod, or delegation to another + raising boundary — the helper must have a production-surviving rejection; + assert-only helpers never count), curated **raising conversions** + (`return int(p)` / `float`/`complex`/`Decimal`/`Fraction`/`UUID` over a + non-constant argument, and non-constant subscript lookups `Color[p]` / + `ALLOWED[p]` — validate-by-construction), and **conditional-expression falsy + returns** (`return m.group(0) if m else None`). - **BREAKING (wire): per-finding `suppressed` key renamed to `suppression_state`.** The JSONL stream, the Filigree `metadata.wardline.*` subtree, and the signed **legis scan artifact** now emit `suppression_state` (values unchanged: diff --git a/docs/concepts/model.md b/docs/concepts/model.md index cc205d29..8eec6b49 100644 --- a/docs/concepts/model.md +++ b/docs/concepts/model.md @@ -135,8 +135,11 @@ return is the higher trust level you declare. Boundaries matter because they are where trust is *earned*. A function that raises its declared trust but cannot actually reject anything — no `raise`, no early failing return — is not validating; it is just relabelling untrusted data -as trusted. Wardline flags exactly that case (rule [PY-WL-102](rules.md)), so a -boundary that claims to validate is held to actually being able to say "no." +as trusted. Wardline flags exactly that case with its boundary-integrity family +(rules [PY-WL-102 / 111 / 113 / 119](rules.md) — no rejection path, assert-only +rejection, fail-open handler, and the bare no-op `return p` shape, exactly one +of which fires per boundary), so a boundary that claims to validate is held to +actually being able to say "no." Get your boundaries right and the trust propagation does the rest: everything downstream of a real boundary is trusted, everything upstream is raw, and diff --git a/docs/concepts/rules.md b/docs/concepts/rules.md index ba3d4a52..417632f1 100644 --- a/docs/concepts/rules.md +++ b/docs/concepts/rules.md @@ -1,9 +1,10 @@ # Rules -Wardline ships eleven policy rules. They consume the [taint & trust -model](model.md): each one looks for a place where a function's *declared* trust -and its *actual* trust disagree, where untrusted data reaches a dangerous sink, -or where a trusted-tier function handles errors or declarations carelessly. +Wardline ships twenty-six Python policy rules, `PY-WL-101` through `PY-WL-126`. +They consume the [taint & trust model](model.md): each one looks for a place +where a function's *declared* trust and its *actual* trust disagree, where +untrusted data reaches a dangerous sink, or where a trusted-tier function +handles errors or declarations carelessly. All of them are off by default for undecorated code — they only fire where you have opted in by declaring trust. To turn a rule off entirely, or to change its @@ -11,29 +12,53 @@ severity, see [Configuration](../guides/configuration.md). ## The rules -| Rule | What it flags | Default severity | -|---|---|---| -| `PY-WL-101` | A trust-anchored function returns data less trusted than the level it declares — untrusted data reaches a trusted producer with no validation. | `ERROR` | -| `PY-WL-102` | A trust boundary (a function that raises declared trust on its return) has no rejection path — no raise, no falsy-constant return — so it cannot validate. | `ERROR` | -| `PY-WL-103` | A broad exception handler (bare except / Exception / BaseException) in a trusted-tier function. | `WARN` | -| `PY-WL-104` | An exception handler that silently swallows the error (body is only pass/.../continue/break or a bare constant expression, e.g. a docstring-like string) in a trusted-tier function. | `WARN` | -| `PY-WL-105` | Untrusted data is passed as an argument to a trusted producer at a call site. | `ERROR` | -| `PY-WL-106` | Untrusted data reaches a deserialization sink (pickle/marshal/yaml.load) in a trusted-tier function. | `WARN` | -| `PY-WL-107` | Untrusted data reaches a dynamic-code-execution sink (eval/exec/compile) in a trusted-tier function. | `WARN` | -| `PY-WL-108` | Untrusted data reaches an always-shell OS-command sink (os.system/os.popen/subprocess.getoutput). | `WARN` | -| `PY-WL-109` | A trusted producer has both a value-bearing return and a None-yielding return — None leaks from a function declaring trusted output. | `WARN` | -| `PY-WL-110` | An entity carries two or more distinct trust markers (e.g. `@trusted` + `@external_boundary`) — a contradictory declaration the engine resolves silently. | `WARN` | -| `PY-WL-111` | A trust boundary's only rejection path is `assert`, which `python -O` strips — the validation silently vanishes in production (CWE-617). | `ERROR` | +| Rule | What it flags | Base severity | Maturity | +|---|---|---|---| +| `PY-WL-101` | A trust-anchored function returns data less trusted than the level it declares — untrusted data reaches a trusted producer. | `ERROR` | stable | +| `PY-WL-102` | A trust boundary has no rejection path of any recognised shape — it cannot validate. (The bare `return p` shape is `PY-WL-119`'s.) | `ERROR` | stable | +| `PY-WL-103` | A broad exception handler (bare except / Exception / BaseException) in a trusted-tier function. | `WARN` | stable | +| `PY-WL-104` | An exception handler that silently swallows the error in a trusted-tier function. | `WARN` | stable | +| `PY-WL-105` | Untrusted data is passed as an argument to a trusted producer at a call site. | `ERROR` | stable | +| `PY-WL-106` | Untrusted data reaches a deserialization sink (pickle/marshal/yaml, `pickle.Unpickler`, `shelve.open`, and a curated third-party table) (CWE-502). | `WARN` | stable | +| `PY-WL-107` | Untrusted data reaches a dynamic-code-execution sink (`eval`/`exec`/`compile`, including the `builtins.`/`__builtins__.` spellings) (CWE-95). | `WARN` | stable | +| `PY-WL-108` | Untrusted data reaches a command/program-execution sink — the always-shell string APIs plus `os.exec*`/`os.spawn*`/`os.posix_spawn*`/`pty.spawn` (CWE-78). | `ERROR` | stable | +| `PY-WL-109` | A trusted producer has both a value-bearing return and a None-yielding return — None leaks from a function declaring trusted output. | `WARN` | stable | +| `PY-WL-110` | An entity carries two or more distinct trust markers — a contradictory declaration the engine resolves silently. | `WARN` | stable | +| `PY-WL-111` | A trust boundary's only rejection path is `assert`, which `python -O` strips (CWE-617). | `ERROR` | stable | +| `PY-WL-112` | Untrusted data reaches a `subprocess` call with a literal `shell=True` (CWE-78). | `ERROR` | stable | +| `PY-WL-113` | A trust boundary fails open — an exception handler swallows the failure and substitutes a value, so the boundary can be bypassed by triggering the exception (CWE-636). | `ERROR` | stable | +| `PY-WL-114` | A builtin trust decorator's level argument is statically readable but invalid or out-of-range — the engine would silently drop the declaration. | `ERROR` | stable | +| `PY-WL-115` | Untrusted data reaches a dynamic code/module-load sink (`importlib.import_module`, `__import__`, `runpy.run_path`/`run_module`, `importlib.util.spec_from_file_location`) (CWE-829/CWE-94). | `WARN` | stable | +| `PY-WL-116` | Untrusted data reaches a path/filesystem-traversal sink — open/join/`pathlib.Path`, filesystem mutation, `Path` methods, archive extraction (Zip Slip) (CWE-22). | `WARN` | preview | +| `PY-WL-117` | Untrusted data reaches the URL slot of an HTTP client sink (SSRF: requests/httpx/aiohttp/urllib, including constructed client/session methods) (CWE-918). | `WARN` | preview | +| `PY-WL-118` | Untrusted data reaches a SQL execution sink (`execute`/`executemany`/`executescript`) in the SQL-string position (CWE-89). | `ERROR` | preview | +| `PY-WL-119` | A degenerate (no-op) trust boundary — the body is a bare `return ` with no validation at all. | `ERROR` | preview | +| `PY-WL-120` | Stored/persisted taint (file reads, DB cursor fetches) reaches trusted state without validation. | `ERROR` | preview | +| `PY-WL-121` | Untrusted data reaches an XML parsing sink (XXE / billion-laughs) (CWE-611). | `ERROR` (stdlib parsers `WARN`) | preview | +| `PY-WL-122` | Untrusted data is compiled into a server-side template (jinja2/mako) — SSTI (CWE-1336). | `ERROR` | preview | +| `PY-WL-123` | Untrusted data is the attribute NAME in `setattr`/`getattr` — dynamic attribute injection / mass assignment (CWE-915). | `WARN` | preview | +| `PY-WL-124` | Untrusted data reaches a native-library load sink (`ctypes.CDLL` family) (CWE-114). | `ERROR` | preview | +| `PY-WL-125` | Untrusted data is the log MESSAGE format string of `logging` calls — log injection (CWE-117). | `INFO` | preview | +| `PY-WL-126` | Untrusted data reaches the recipient/message of `smtplib` `SMTP.sendmail` — mail/header injection (CWE-93). | `WARN` | preview | !!! info "Declaration-gated vs. tier-modulated severity" - `PY-WL-101`, `PY-WL-102`, `PY-WL-105`, `PY-WL-109`, `PY-WL-110`, and - `PY-WL-111` are **declaration-gated** — the decorator itself is the opt-in, - so they always fire at their base severity. `PY-WL-103`, `PY-WL-104`, and the - sink rules `PY-WL-106`/`107`/`108` are **tier-modulated**: their severity - scales with the function's own trust tier. They report at the base severity - in fully trusted functions (`INTEGRAL`/`ASSURED`), downgrade one step in - partially-trusted functions, and are suppressed entirely on undecorated code. - The `WARN` above is the trusted-tier value. + `PY-WL-101`, `PY-WL-102`, `PY-WL-105`, `PY-WL-109`, `PY-WL-110`, + `PY-WL-111`, `PY-WL-113`, `PY-WL-114`, and `PY-WL-119` are + **declaration-gated** — the decorator itself is the opt-in, so they always + fire at their base severity. `PY-WL-103`, `PY-WL-104`, and the sink rules + (`PY-WL-106`/`107`/`108`/`112`/`115`/`116`/`117`/`118`/`120`/`121`–`126`) + are **tier-modulated**: their severity scales with the function's own trust + tier. They report at the base severity in fully trusted functions + (`INTEGRAL`/`ASSURED`), downgrade one step in partially-trusted functions, + and are suppressed entirely on undecorated code. The base severity above is + the trusted-tier value. + +!!! note "Preview maturity" + Rules marked **preview** carry `maturity: preview` in the + [vocabulary descriptor](../reference/vocabulary.md): their charter and + sink sets are still being calibrated, and their predicates may sharpen + between releases. They participate in the gate, baseline, waivers, and + judge exactly like stable rules. --- @@ -61,23 +86,68 @@ EXTERNAL_RAW (less trusted) — untrusted data reaches a trusted producer The fix is to validate before returning — for example by routing the raw value through a `@trust_boundary` first. +## The boundary-integrity family (102 / 111 / 113 / 119) + +A `@trust_boundary` declares that it *raises* trust: its body sees raw data and +its return is the higher level it declares. Four rules police whether such a +boundary can actually do that job, and they **partition four ways — at most one +of them fires per boundary**: + +- **`PY-WL-119`** — the bare **degenerate** shape: the body is (modulo + docstrings/`pass`) a single `return `. The more-specific rule wins, so + 102 suppresses itself on this shape. The suppression is structural — keyed on + the shape, not on whether 119 is enabled. +- **`PY-WL-102`** — every *other* shape with **no rejection path** of any + recognised kind: the boundary cannot reject at all. +- **`PY-WL-111`** — the only rejection is **`assert`**, which `python -O` + strips: the validation silently vanishes in production (CWE-617). This + includes an assert inside a `try` whose handler substitutes — the rejection + is still assert-only, so 111 wins over 113. +- **`PY-WL-113`** — a **real rejection exists but a fail-open handler defeats + it**: an `except` swallows the failure and substitutes a value-bearing + result. + +### What counts as a rejection path + +A boundary is silent under 102/111 when it has any of these recognised +rejection shapes: + +- an own-scope **`raise`** (or an `assert` — but assert-*only* is 111's case); +- a **rejection-shaped `return`** — a falsy constant (`None`/`False`/`0`/`""`, + or an empty literal container), a **conditional expression with a rejecting + branch** (`return m.group(0) if m else None` is the ternary form of + `if not m: return None`), or a curated **raising conversion** — a + validate-by-construction expression that raises on bad input: `int(p)`, + `float(p)`, `complex(p)`, `Decimal(p)`, `Fraction(p)`, `UUID(p)`, or a + subscript lookup with a non-constant key (`Color[p]` raises `KeyError`; + `ALLOWED[p]` likewise). A *constant* argument or index (`int("3")`, + `parts[0]`) validates nothing and does not count; +- a **one-hop, same-module call to a raising helper** — a factored-out + validator whose own body has a real rejection (`_require_nonempty(p)`, a + raising staticmethod, or wholesale delegation to another raising boundary). + The helper's body is inspected exactly one hop deep, same module only, and it + must have a *real* (production-surviving) rejection: a helper that cannot + raise never counts, and an assert-only helper never counts (its assert + vanishes under `python -O` exactly like an inline one). + ### PY-WL-102 — trust boundary with no rejection path -Fires on a `@trust_boundary` validator that cannot actually reject anything. The -function declares it raises trust (its return is more trusted than its raw body), -but it contains no `raise` and no falsy-constant `return` — so it has no way to -say "no" to bad input. A validator that cannot reject is not validating. +Fires on a `@trust_boundary` validator that cannot actually reject anything — +no recognised rejection shape at all. A validator that cannot reject is not +validating. ```python @trust_boundary(to_level="ASSURED") -def validate(p): - return p # no raise, no falsy return — cannot reject +def v(p): + x = p + return x # no raise, no falsy return — cannot reject ``` -Wardline reports: +(The single-statement `return p` form of this defect is reported as +`PY-WL-119` instead.) Wardline reports: ``` -demo.validate declares a trust boundary (EXTERNAL_RAW -> ASSURED) but has no +demo.v declares a trust boundary (EXTERNAL_RAW -> ASSURED) but has no rejection path (no raise / no falsy return) — it cannot validate ``` @@ -85,12 +155,89 @@ The fix is to add a real rejection path: ```python @trust_boundary(to_level="ASSURED") -def validate(p): +def v(p): if not p: raise ValueError return p ``` +### PY-WL-111 — trust boundary whose only rejection is `assert` + +Fires on a `@trust_boundary` whose *only* rejection path is an `assert`. The +validation works in development but is stripped under `python -O`, so the +boundary silently stops rejecting in production (CWE-617). + +```python +@trust_boundary(to_level="ASSURED") +def validate(p): + assert p # stripped under python -O — validation vanishes + return p +``` + +A boundary with a real `raise` or rejection-shaped `return` — or a one-hop +same-module raising helper call (the helper's `raise` survives `-O`) — trips +neither 102 nor 111, even if it also has an `assert`: + +```python +@trust_boundary(to_level="ASSURED") +def validate(p): + assert isinstance(p, str) # an internal invariant, not the gate + if not p: + raise ValueError # the real, -O-safe rejection + return p +``` + +### PY-WL-113 — trust boundary that fails open + +Fires on a `@trust_boundary` where a real rejection *exists* but an `except` +handler swallows the failure and **substitutes** a value-bearing result instead +of re-raising — either by returning it directly or by assigning it to a name +the function returns by fall-through. Such a boundary can be bypassed by +*triggering* the exception. The most insidious shape is the self-catch, where +the handler catches the very exception the boundary's own rejection raises: + +```python +@trust_boundary(to_level="ASSURED") +def v(p): + try: + if bad(p): + raise ValueError # the rejection ... + return p + except ValueError: + return p # ... immediately caught and bypassed +``` + +The rule enforces its premise: a real, production-surviving rejection must +exist (no rejection at all is 102's domain; assert-only is 111's), and the +rejection must be lexically *swallowable* by the matching handler. A rejection +wholly outside the `try` cannot be defeated by the handler — the boundary fails +**closed** and the rule stays silent. A handler that re-raises, or that returns +a falsy/empty constant (signalling rejection, not substitution), never matches. + +### PY-WL-119 — degenerate (no-op) trust boundary + +Fires on the bare degenerate boundary: the body is a single `return ` — +no checks, no asserts, no validation of any kind. It is a strict structural +subset of "no rejection path", carved out of `PY-WL-102`'s domain so the family +partitions cleanly: 119 wins on this shape and the same boundary is never +counted twice at `ERROR` in the gate population. + +```python +@trust_boundary(to_level="ASSURED") +def validate(x): + return x # PY-WL-119: a no-op validator +``` + +```python +@trust_boundary(to_level="ASSURED") +def validate(x): + if not x: + raise ValueError + return x # clean +``` + +--- + ### PY-WL-103 — broad exception handler in a trusted-tier function Fires on a `bare except`, `except Exception`, or `except BaseException` inside a @@ -139,17 +286,59 @@ argument to a `@trusted`-style callee whose body operates on trusted data. Where 101 polices a function's *own* return, 105 polices the *arguments* a trusted callee is handed. Declaration-gated on the callee. -### PY-WL-106 / 107 / 108 — untrusted data reaches a dangerous sink +## The sink rules + +The sink rules fire when raw-zone data reaches a named dangerous call inside a +trusted-tier function. They are tier-modulated: they speak only where trust is +declared and are silent in the developer-freedom zone. Sink matching is +import-alias aware, and most families also resolve the **construct-then-method +form** (`client = httpx.Client(); client.get(url)`, `with smtplib.SMTP(h) as +s`, the chained `Ctor().method(raw)`) and **function-local callable aliases** +(`runner = subprocess.run; runner(...)`). Where only specific argument slots +are dangerous, the rules match by **argument position/keyword** so taint in a +harmless slot (a `timeout=`, a `parser=`, a logging `%`-args parameter) does +not fire. + +### PY-WL-106 — untrusted data reaches a deserialization sink (CWE-502) + +Deserializing untrusted bytes is a classic remote-code-execution vector. The +sink set covers four families: + +- **stdlib direct loaders** — `pickle.load`/`loads`, `marshal.load`/`loads`, + `yaml.load`/`load_all`/`unsafe_load`/`full_load` (the `safe_*` loaders and + the *dump* direction are deliberately not sinks; `json.loads` is excluded — + it does not execute); +- **the OO streaming-unpickle API** — `pickle.Unpickler(stream).load()`, + chained or stored-instance; the dangerous data is the stream handed to the + *constructor*, so the taint is read there; +- **`shelve.open`** — pickle-backed: opening a shelf at an attacker-controlled + *path* then reading keys unpickles attacker bytes (only the path slot is + dangerous); +- **a curated third-party table** — `dill.load`/`loads`, `jsonpickle.decode`, + `joblib.load`, `torch.load`, `numpy.load`. Two literal-keyword gates: + `numpy.load` fires **only** with a literal `allow_pickle=True` (the default + has been the safe `False` since numpy 1.16.3, so absent/`False`/dynamic + stays silent); `torch.load` is suppressed by a literal `weights_only=True` + (the restricted unpickler) and fires otherwise. + +Every entry is RCE-equivalent, so all carry the family base severity (`WARN`, +tier-modulated). + +```python +@trusted(level="ASSURED") +def f(req): + return pickle.loads(read_request(req)) # fires -The three sink rules fire when raw-zone data reaches a named dangerous call -inside a trusted-tier function: +@trusted(level="ASSURED") +def g(path): + return numpy.load("model.npy") # clean: no allow_pickle=True +``` -- **`PY-WL-106`** — a deserialization sink (`pickle.loads`, `marshal.loads`, - `yaml.load`): arbitrary-object construction from untrusted bytes. -- **`PY-WL-107`** — a dynamic-code-execution sink (`eval`, `exec`, `compile`): - arbitrary code execution (CWE-95). -- **`PY-WL-108`** — an always-shell OS-command sink (`os.system`, `os.popen`, - `subprocess.getoutput`): shell command injection. +### PY-WL-107 — untrusted data reaches a dynamic-code-execution sink (CWE-95) + +`eval` / `exec` / `compile` on untrusted input is arbitrary code execution. +Matches the bare builtins (`eval(x)`), the `builtins.eval` forms, and the +`__builtins__.eval` spelling. ```python @trusted(level="ASSURED") @@ -157,10 +346,331 @@ def run(req): eval(read_request(req)) # read_request is @external_boundary -> EXTERNAL_RAW ``` -These are tier-modulated: they speak only where trust is declared and are silent -in the developer-freedom zone. They match curated, importable sink symbols -(framework-specific sinks whose receiver is a runtime object — `cursor.execute`, -`Template().render` — belong in opt-in trust-grammar packs, not the builtin set). +### PY-WL-108 — untrusted data reaches a command/program-execution sink (CWE-78) + +Covers two sink shapes, both stdlib: + +- **always-shell string APIs** — `os.system`, `os.popen`, + `subprocess.getoutput`, `subprocess.getstatusoutput`: these take a shell + *string*, so an untrusted argument is directly injectable; +- **argv-style program execution** — the `os.exec*` and `os.spawn*` families, + `os.posix_spawn`/`os.posix_spawnp`, and `pty.spawn`: no shell mediates, but + an attacker-controlled program path or argv element *is* arbitrary-program + execution. + +Base severity is `ERROR`, calibrated with `PY-WL-118` (SQLi): tainted +command/program execution is the same blast-radius exploit class. + +The `subprocess.run`/`call`/`Popen`/`check_*` family is intentionally **not** +in this sink set — with the default `shell=False` an argv-list is safe, and the +one condition that makes it injectable (`shell=True`) is policed by +`PY-WL-112`. + +**`shlex.quote` semantics (GUARDED, concatenation context only).** +`shlex.quote(x)` neutralizes shell-string taint for the always-shell sinks +*only as a fragment of a constant-shaped command*: a `+` chain or f-string with +at least one constant fragment in which every non-constant leaf is a +`shlex.quote(...)` call is treated as guarded — +`os.system("echo " + shlex.quote(raw))` is clean. A **bare whole-command quote +still fires** (`os.system(shlex.quote(raw))` — a fully-quoted single token +handed to a shell executes that token as the program name, so the attacker +still picks what runs), and the guard never applies to the argv +program-execution sinks (no shell parses the value, so quoting protects +nothing). The guard is inline-syntactic only: a quote result routed through a +variable still fires. + +```python +@trusted(level="ASSURED") +def f(p): + os.execv(read_raw(p), ["prog"]) # fires (program execution) + +@trusted(level="ASSURED") +def g(p): + os.system("echo " + shlex.quote(read_raw(p))) # clean (guarded fragment) +``` + +### PY-WL-112 — untrusted data reaches a `shell=True` subprocess call (CWE-78) + +The completion of `PY-WL-108`'s deliberate exclusion: the +`subprocess.run`/`call`/`check_call`/`check_output`/`Popen` family fires only +when **both** a literal `shell=True` keyword is statically visible **and** +untrusted data reaches the call. A `**kwargs` spread, a non-constant +`shell=flag`, or `shell=1` is not matched (a bounded false negative, chosen +over flooding argv-list false positives); a fully-literal +`subprocess.run('ls -la', shell=True)` does not fire either — the rule keys on +untrusted *data*, not on `shell=True` alone. Base severity `ERROR`, calibrated +with 108/118. + +```python +@trusted(level="ASSURED") +def f(p): + subprocess.run(read_raw(p), shell=True) # fires + +@trusted(level="ASSURED") +def g(p): + subprocess.run(["ls", "-la"]) # clean: argv list, no shell +``` + +### PY-WL-115 — untrusted data reaches a dynamic code/module-load sink (CWE-829/CWE-94) + +The import-and-execute class: `importlib.import_module` and `__import__` +(attacker-chosen module name), `runpy.run_path` / `runpy.run_module` +(import-and-*execute* an attacker-controlled file path / module — blast radius +equivalent to `exec`), and `importlib.util.spec_from_file_location` (a tainted +file-path argument builds a loader for attacker-chosen code). + +```python +@trusted(level="ASSURED") +def f(p): + runpy.run_path(read_raw(p)) # fires + +@trusted(level="ASSURED") +def g(p): + importlib.import_module("sys") # clean +``` + +### PY-WL-116 — untrusted data reaches a path/filesystem-traversal sink (CWE-22) + +Three sink families: + +- **direct dotted calls** with a tainted path argument — `open`, `os.open`, + `os.path.join`, `pathlib.Path`, plus the filesystem-**mutation** APIs + (`os.remove`/`unlink`/`rmdir`/`mkdir`/`makedirs`/`rename`/`renames`/`replace`, + `shutil.rmtree`/`copy`/`copy2`/`copyfile`/`copytree`/`move`), where a tainted + path is a destructive traversal; +- **`Path` methods** (`read_text`/`read_bytes`/`write_text`/`write_bytes`/ + `open`/`unlink`/`rmdir`/`mkdir`) on a `pathlib.Path` **constructed from + tainted input** — the dangerous data is the constructor's argument + (`q = Path(raw); q.read_text()`); +- **archive extraction** (Zip Slip / tarbomb): `extractall`/`extract` on a + `tarfile.open`/`tarfile.TarFile`/`zipfile.ZipFile` instance whose *archive + source* is tainted — a malicious archive escapes the target directory via + `../` member names. **Exemption:** an extraction call passing tarfile's safe + filter as the literal `filter="data"` (blocks absolute paths, traversal, and + device members since Python 3.12) does not fire; any other filter value + (including `"fully_trusted"` or a dynamic expression) still fires. + +```python +@trusted(level="ASSURED") +def f(p): + tf = tarfile.open(read_raw(p)) + tf.extractall("/dst") # fires (Zip Slip) + +@trusted(level="ASSURED") +def g(p): + tf = tarfile.open(read_raw(p)) + tf.extractall("/dst", filter="data") # clean: safe extraction filter +``` + +### PY-WL-117 — untrusted data reaches an HTTP client sink (SSRF, CWE-918) + +Covers `requests`, `httpx`, `aiohttp`, and `urllib` — both the module-level +calls and **constructed client/session instance methods** +(`client = httpx.Client(); client.get(url)`, `async with httpx.AsyncClient() +as c`, `requests.Session().get(url)`, `aiohttp.ClientSession`), plus a client +constructor's `base_url=`. Matching is **URL-slot precise**: only the +request-target argument is an SSRF vector, so a tainted +`timeout=`/`verify=`/`headers=` with a clean literal URL does not fire. + +```python +@trusted(level="ASSURED") +def f(p): + client = httpx.Client() + client.get(read_raw(p)) # fires + +@trusted(level="ASSURED") +def g(p): + requests.get("https://example.com", timeout=read_raw(p)) # clean: not the URL slot +``` + +### PY-WL-118 — untrusted data reaches a SQL execution sink (CWE-89) + +Fires when untrusted data reaches the **SQL-string position** of +`cursor.execute`, `cursor.executemany`, or `executescript` (sqlite3 cursor +*and* connection — `executescript` runs a multi-statement script with **no +parameter binding at all**, so it is strictly more dangerous than `execute`). + +Three precision guards: + +- **operation-slot only** — SQLi is a property of the SQL *string*; untrusted + data passed as a *bound parameter* (the OWASP-canonical mitigation) cannot + alter query structure and does not fire. Splatted/`**`-unpacked arguments + that could supply the operation fail closed (fire); +- **receiver heuristic (fail-closed)** — `.execute` is matched by method name, + so binding evidence (a constructor from a known DB-driver module fires; one + from a known executor module suppresses) and exact name-token evidence + (`cursor`/`conn`/`db`/... fires and wins over mixed names; `pool`/`executor`/ + `worker`/... alone suppresses) keep task pools from firing a CWE-89 `ERROR`. + Unknown receivers **fire** — when unsure, a missed finding is worse than a + false positive; +- **constant `text()` exemption** — the canonical SQLAlchemy parameterized + pattern `conn.execute(text("... :id"), {"id": uid})` wraps a compile-time + constant in a recognized text-clause constructor and is treated as clean; + `text(tainted)` / `text(f"...")` still fire (`text()` is not a sanitiser). + +```python +@trusted(level="ASSURED") +def f(p, cursor): + cursor.executescript(read_raw(p)) # fires + +@trusted(level="ASSURED") +def g(p, cursor): + cursor.execute("SELECT * FROM t WHERE id = ?", (read_raw(p),)) # clean: bound parameter +``` + +### PY-WL-120 — stored/persisted taint reaches trusted state + +Fires when raw data loaded from persistent storage — file reads via +`open`/`read_text` or database cursor fetches (`fetchone`/`fetchall`/ +`fetchmany`) — reaches a trusted state (returned by a `@trusted` function or +passed to a `@trusted` callee) without being validated. The storage-read +matcher is **receiver-aware**: an `io.StringIO`/`io.BytesIO` receiver is an +in-memory buffer, never persistent storage, so its `.read()` is exempt. On the +return arm the rule de-conflicts with `PY-WL-101`: where 101 already reports +the trust-claim violation with unresolved provenance, 120 suppresses; where the +storage provenance is substantiated, the pair stands deliberately (101 reports +the trust claim, 120 adds the storage-provenance annotation). + +```python +@trusted(level="ASSURED") +def get_config(): + data = open("config.txt").read() + return data # fires +``` + +## The preview sink expansions (121–126) + +Six PREVIEW rules added in the 2026-06-10 coverage-gap pass. All are +tier-modulated, argument-slot precise, and resolve the construct-then-method +form and callable aliases. + +### PY-WL-121 — untrusted data reaches an XML parsing sink (CWE-611) + +A tainted document/stream reaching an XML parser. Only the DOCUMENT slot +(position 0 / its keyword spelling) is dangerous — taint in a `parser=` or +handler slot is not XXE. Severity is **per-sink**, calibrated to each parser's +*default* posture: `lxml.etree.*` is `ERROR` (resolves external entities by +default, so tainted XML is genuine XXE — local file disclosure / SSRF); the +stdlib `xml.etree.ElementTree` / `xml.dom.minidom` / `xml.sax` parsers are +`WARN` (external general entities have been disabled by default since CPython +3.7.1; the residual default-on risk is the billion-laughs internal-entity +expansion DoS). `defusedxml` is the blessed remediation and is deliberately +not a sink. An operator `rules.severity` override re-bases the whole rule. + +```python +from lxml import etree + +@trusted(level="ASSURED") +def f(p): + return etree.fromstring(read_raw(p)) # fires at ERROR + +@trusted(level="ASSURED") +def g(): + ET.fromstring("") # clean: constant document +``` + +### PY-WL-122 — untrusted data compiled into a server-side template (SSTI, CWE-1336) + +A tainted string reaching a template **compilation** sink — `jinja2.Template`, +`jinja2.Environment.from_string` (including the construct-then-method form), +`mako.template.Template`. Only the template SOURCE slot is dangerous: tainted +data passed as a *render variable* is the safe idiom and does not fire, and +loading a template *by name* (`env.get_template(raw)`) is not SSTI. Severity +`ERROR`: SSTI in Jinja2/Mako is RCE-adjacent. + +```python +@trusted(level="ASSURED") +def f(p): + return jinja2.Template(read_raw(p)).render() # fires + +@trusted(level="ASSURED") +def g(p): + jinja2.Template("Hello {{ name }}").render(name=read_raw(p)) # clean: render variable +``` + +### PY-WL-123 — tainted attribute NAME reaches `setattr`/`getattr` (CWE-915) + +Dynamic attribute injection — an untrusted NAME argument (position 1) to the +builtin `setattr`/`getattr` lets an attacker pick which attribute is +written/read (mass assignment). Only the NAME slot is dangerous: an untrusted +VALUE assigned to a fixed attribute, a tainted `getattr` default, or a tainted +receiver are ordinary data flow and stay silent. Severity `WARN` — a +mass-assignment *vector*, not direct code execution. + +```python +@trusted(level="ASSURED") +def f(p, obj): + setattr(obj, read_raw(p), 1) # fires: attacker picks the attribute + +@trusted(level="ASSURED") +def g(p, obj): + setattr(obj, "name", read_raw(p)) # clean: fixed attribute name +``` + +### PY-WL-124 — untrusted path reaches a native-library load sink (CWE-114) + +A tainted library path/name reaching `ctypes.CDLL` / `WinDLL` / `OleDLL` / +`PyDLL` or `ctypes.cdll.LoadLibrary`. Loading an attacker-controlled shared +object is arbitrary **native** code execution — the same blast radius as the +command-execution family, so the same `ERROR` base. + +```python +@trusted(level="ASSURED") +def f(p): + return ctypes.CDLL(read_raw(p)) # fires + +@trusted(level="ASSURED") +def g(): + ctypes.CDLL("libm.so.6") # clean +``` + +### PY-WL-125 — untrusted data as the log MESSAGE format string (CWE-117) + +Log injection / log forging — a tainted value used as the message FORMAT string +of `logging.debug/info/warning/error/critical/exception` (module-level +functions or the Logger-method form, `logger = logging.getLogger(...); +logger.info(raw)`). Newline-spoofed entries forge audit lines and seed +log-viewer XSS downstream. Only the message slot is dangerous: tainted data in +the lazy `%`-args parameters (`logging.info('user=%s', raw)`) is logging's own +parameterization — the canonical safe idiom — and never fires. Severity +`INFO`: the class is real but high-noise by nature, and its blast radius is +forgery/foothold, not execution — visible to agents and to an explicit +`--fail-on INFO` gate without ever tripping the default gate. + +```python +@trusted(level="ASSURED") +def f(p): + logging.info(read_raw(p)) # fires: tainted format string + +@trusted(level="ASSURED") +def g(p): + logging.info("user input = %s", read_raw(p)) # clean: lazy %-parameterization +``` + +### PY-WL-126 — untrusted recipient/message reaches `SMTP.sendmail` (CWE-93) + +Mail (CRLF/header) injection — tainted data in the `to_addrs` (position 1) or +`msg` (position 2) argument of `smtplib.SMTP.sendmail` / +`smtplib.SMTP_SSL.sendmail`, with the receiver matched through the +construct-then-method machinery. Newlines in a recipient or message inject +spoofed headers / BCC recipients. The envelope sender (`from_addr`) is +deliberately not a dangerous slot in v1, and `send_message` is out of scope +(its header serialization already rejects bare newlines). Severity `WARN` — +real injection, but bounded blast radius (spam/spoofing, not code execution). + +```python +@trusted(level="ASSURED") +def f(p): + s = smtplib.SMTP("localhost") + s.sendmail("from@example.com", "to@example.com", read_raw(p)) # fires + +@trusted(level="ASSURED") +def g(): + s = smtplib.SMTP("localhost") + s.sendmail("from@example.com", "to@example.com", "body") # clean +``` + +--- ### PY-WL-109 — None leaks from a trusted producer @@ -176,33 +686,40 @@ together with `@external_boundary`). The combination is contradictory; the engin resolves it fail-closed but the conflicting intent is declaration hygiene worth surfacing. Declaration-gated. -### PY-WL-111 — trust boundary whose only rejection is `assert` - -A `PY-WL-102`-adjacent refinement. Fires on a `@trust_boundary` whose *only* -rejection path is an `assert`. The validation works in development but is -stripped under `python -O`, so the boundary silently stops rejecting in -production (CWE-617). - -```python -@trust_boundary(to_level="ASSURED") -def validate(p): - assert p # stripped under python -O — validation vanishes - return p -``` - -The two rules partition the space: `PY-WL-102` fires when a boundary cannot -reject *at all*; `PY-WL-111` fires when it *appears* to reject but only via a -guard that disappears in production. A boundary with a real `raise` or a -falsy-constant `return` trips neither — even if it also has an `assert`. - -```python -@trust_boundary(to_level="ASSURED") -def validate(p): - assert isinstance(p, str) # an internal invariant, not the gate - if not p: - raise ValueError # the real, -O-safe rejection - return p -``` +### PY-WL-114 — invalid level on a builtin trust decorator + +Fires on any entity carrying a builtin trust decorator (`@trusted` or +`@trust_boundary`) whose level argument is statically readable but not a valid +trust level, or not within the decorator's allowed set. This is a critical +safety defect: a typo (e.g. `level='ASURED'`) causes the engine to silently +drop the decorator, disabling all taint gates on that function. Recognition +resolves the decorator to the builtin FQN (import-alias aware), so an aliased +builtin with a typo still fires while a foreign decorator that merely happens +to be spelled `trusted` does not. A dynamic level (`level=cfg.LEVEL`) is not +statically readable and stays silent. + +## Engine diagnostics and the gate + +Alongside the policy rules, the engine emits `WLN-ENGINE-*` / `WLN-CONFIG-*` +diagnostics about the scan itself. Two of them are **gate-eligible `ERROR` +defects** (fail-closed — their absence of analysis must not read as green): + +- **`WLN-ENGINE-PARSE-ERROR`** — a discovered file could not be read or parsed. + Its sinks were never analyzed, so a default `--fail-on ERROR` reading green + over it would be a fail-open. Baseline/waiver still *annotate* it but cannot + clear the secure gate; `--trust-suppressions` can (an explicit operator trust + decision). +- **`WLN-ENGINE-FILE-FAILED`** — an unexpected exception while analyzing one + file. The scan continues (per-file isolation — other files' findings are + kept) and the failed file is named, counted toward the scan's unanalyzed + population. + +A configuration that silently weakens analysis is also surfaced: +`WLN-CONFIG-SANITISER-SINK-COLLISION` (a fact, not a defect) reports a +configured sanitiser that collides with a built-in serialisation sink of the +same name — the conservative sink classification takes precedence, so the +sanitiser declaration has no effect, and the diagnostic says so instead of +letting the suppression attempt pass silently. ## Configuring rules diff --git a/docs/concepts/taint-algebra.md b/docs/concepts/taint-algebra.md index f6eb60c9..839fbed8 100644 --- a/docs/concepts/taint-algebra.md +++ b/docs/concepts/taint-algebra.md @@ -154,8 +154,10 @@ reads the validator's **declared** output tier (`effective_return`, the annotation as the contract. This is sound for the statically-decidable property. A **broken** validator with -*no rejection path at all* is caught by `PY-WL-102` (it can never raise, so it -cannot validate). +*no rejection path at all* is caught by the boundary-integrity family — `PY-WL-119` +for the bare `return p` shape, `PY-WL-102` for every other no-rejection shape +(it can never raise, so it cannot validate), with `PY-WL-111` (assert-only) and +`PY-WL-113` (fail-open handler) covering the defeated-rejection variants. The **residual** — accepted, out of static reach — is a validator that **has** a rejection path but checks the **wrong predicate** (e.g. it validates length when diff --git a/docs/concepts/trust-vocabulary-convergence.md b/docs/concepts/trust-vocabulary-convergence.md index 2450a176..6b89aa6d 100644 --- a/docs/concepts/trust-vocabulary-convergence.md +++ b/docs/concepts/trust-vocabulary-convergence.md @@ -29,7 +29,7 @@ the Weft mechanism that delivers it (or the reason it is declined). | Effect / idea | Verdict | Weft mechanism (or reason) | |---|---|---| -| **Fabrication test** — a trust boundary must be able to say *no* (reject), or it isn't a boundary | **Covered** | **PY-WL-102** (`boundary_without_rejection`): a `@trust_boundary` with no rejection path is flagged — it cannot say no, so it cannot be trusted to raise trust | +| **Fabrication test** — a trust boundary must be able to say *no* (reject), or it isn't a boundary | **Covered** | the **boundary-integrity family PY-WL-102/111/113/119** (anchored by `boundary_without_rejection`): a `@trust_boundary` with no rejection path — or only an `-O`-stripped `assert`, a fail-open handler, or a bare `return p` — is flagged; it cannot say no, so it cannot be trusted to raise trust | | **Custody / provenance** — trust is earned and tracked, never assumed | **Covered** | the trust **lattice** (a value is only as trusted as its least-trusted contributor; `least_trusted` weakest-link meet) + `taint_provenance` (source + contributing callee), carried on every finding and in the dossier | | **Fail-closed boundaries** — what cannot be proven is not trusted | **Covered** | the `UNKNOWN_*` states + observable `WLN-ENGINE-*` FACTs; a custom boundary the engine cannot prove seeds `UNKNOWN_RAW` and emits `WLN-ENGINE-UNPROVABLE-BOUNDARY` (T2.4) — the extension plane inherits the no-false-green guarantee | | **Tiered boundary** — a validated boundary *raises* trust to a named tier | **Covered** | `@trust_boundary(to_level=GUARDED\|ASSURED)` — named Weft levels rather than integer tiers, the same "raise trust at a validated boundary" effect expressed in the lattice | diff --git a/src/wardline/core/judge.py b/src/wardline/core/judge.py index 2cc5c424..e7fbc9c7 100644 --- a/src/wardline/core/judge.py +++ b/src/wardline/core/judge.py @@ -104,31 +104,54 @@ class JudgeResponse: @trust_boundary(to_level=L) -> raw input in, trusted level L out (a validator). @trusted(level=L) -> the function is asserted to operate at level L. -The four rules you will see: - PY-WL-101 untrusted-reaches-trusted: a @trusted(level=L) PRODUCER whose ACTUAL - returned taint is strictly less-trusted than its declared level L. Note: - trust-RAISING validators (@trust_boundary, where the body is less-trusted - than the declared return) are EXEMPT from 101 and handled by 102 instead — - so 101 fires on @trusted/@external producers, NOT on @trust_boundary - validators. TRUE positive: a @trusted(level=ASSURED) function that actually - returns raw / MIXED_RAW data. FALSE positive: the engine could not narrow - taint through a guard or helper it cannot model, so the body looks raw though - it is in fact validated. - PY-WL-102 boundary-without-rejection: a trust-raising @trust_boundary that lacks - any raise / falsy-return rejection path. TRUE: a validator that declares it - raises input to GUARDED but never rejects (returns raw input unchanged). - FALSE: rejection happens via a helper the engine did not resolve. - PY-WL-103 broad-except: a broad `except Exception` / bare except at a trusted - tier. TRUE: swallowing errors at a trust boundary. FALSE: re-raised or - handled deliberately in a way the tier modulation over-weighted. - PY-WL-104 silent-except: an except handler that suppresses the error with no - re-raise / log / handling — tier-modulated like 103. TRUE: swallowing an - error at a trusted tier hides a real failure. FALSE: a deliberate, documented - swallow the tier modulation over-weighted. - -Why undecorated code is silent: 101/102 fire ONLY on explicitly anchored / -declared functions (the @trusted / @trust_boundary / @external decorators); -103/104 are silenced on undecorated code by tier modulation in the UNKNOWN_RAW +The rule families you will see (rule_id is in the finding): + Producer integrity — PY-WL-101 untrusted-reaches-trusted: a @trusted(level=L) + PRODUCER whose ACTUAL returned taint is strictly less-trusted than its + declared level L. Note: trust-RAISING validators (@trust_boundary, where the + body is less-trusted than the declared return) are EXEMPT from 101 and + handled by the boundary-integrity family instead — so 101 fires on + @trusted/@external producers, NOT on @trust_boundary validators. TRUE + positive: a @trusted(level=ASSURED) function that actually returns raw / + MIXED_RAW data. FALSE positive: the engine could not narrow taint through a + guard or helper it cannot model, so the body looks raw though it is in fact + validated. + Boundary integrity — a FOUR-WAY PARTITION (exactly one fires per defective + boundary): PY-WL-119 a bare degenerate `return ` boundary; + PY-WL-102 any other trust-raising @trust_boundary with no rejection path + (no raise — including one-hop same-module raising helpers and raising + conversions like int()/Enum lookup — and no falsy return); PY-WL-111 the + ONLY rejection is `assert` (vanishes under python -O); PY-WL-113 a real + rejection exists but a fail-open except handler swallows it and substitutes + a value. TRUE: the boundary really cannot reject (or its rejection is + defeated). FALSE: rejection happens via a helper/path the engine did not + resolve (cross-module helpers are NOT resolved). + Exception handling (tier-modulated) — PY-WL-103 broad `except Exception` / + bare except at a trusted tier; PY-WL-104 a handler that suppresses the + error with no re-raise / log / handling. TRUE: swallowing errors at a trust + boundary. FALSE: re-raised or handled deliberately in a way the tier + modulation over-weighted. + Tainted-sink rules — untrusted data reaching a dangerous sink inside a + trust-declared function: PY-WL-105 (trusted callee), 106 (deserialization: + pickle/yaml/marshal incl. Unpickler/shelve and curated third-party loaders), + 107 (eval/exec/compile), 108 (command/program execution: os.system family, + os.exec*/spawn*, pty.spawn; shlex.quote'd fragments are treated GUARDED), + 112 (subprocess shell=True), 115 (dynamic import/code load incl. runpy), + 116 (path traversal: open/mutation/Path-method/archive-extraction sinks), + 117 (SSRF — URL-position-aware, incl. instance methods on constructed + clients/sessions), 118 (SQL execution; parameterized queries and constant + sqlalchemy text() do NOT fire), 121 XML/XXE, 122 template injection (SSTI), + 123 setattr/getattr with a tainted NAME, 124 native-library load (ctypes), + 125 log injection (message-position only), 126 mail injection (smtplib). + TRUE: attacker-influenceable data reaches the sink. FALSE: the value is + provably constrained by validation the engine could not model, or the taint + is an UNKNOWN_RAW pessimism artefact (unresolved helper), not a real flow. + Declaration hygiene — PY-WL-109 (None leak from a trusted producer), + 110/114 (contradictory or spoofed trust markers), 120 (stored/persisted + data read at a trusted tier without re-validation). + +Why undecorated code is silent: anchored rules fire ONLY on explicitly declared +functions (the @trusted / @trust_boundary / @external decorators); the +tier-modulated rules are silenced on undecorated code in the UNKNOWN_RAW freedom zone. So a finding only exists where a trust boundary was declared. ================================================================ diff --git a/src/wardline/core/run.py b/src/wardline/core/run.py index 9a422c74..3669ce74 100644 --- a/src/wardline/core/run.py +++ b/src/wardline/core/run.py @@ -62,10 +62,10 @@ class ScanSummary: # Files DISCOVERED but NEVER analysed despite being analysable — a genuine # under-scan (parse errors, too-deep skips, missing source roots). Benign # no-module skips (WLN-ENGINE-NO-MODULE) are EXCLUDED — see UNANALYZED_RULE_IDS. - # These are Severity.NONE FACTs that never trip the severity gate, so they are - # counted separately to surface a silent under-scan / false-green. This is an - # OVERLAY (a subset of ``informational``), NOT a partition member — it is not added - # into the sum-to-total identity. + # PARSE-ERROR/FILE-FAILED are gate-eligible ERROR DEFECTs (fail-closed: unscanned + # code must not read GREEN); FILE-SKIPPED/SOURCE-ROOT-MISSING stay non-gating FACTs. + # This is an OVERLAY counted by rule_id across both buckets, NOT a partition + # member — it is not added into the sum-to-total identity. unanalyzed: int = 0 diff --git a/src/wardline/scanner/analyzer.py b/src/wardline/scanner/analyzer.py index f6e4eeb2..f27e4768 100644 --- a/src/wardline/scanner/analyzer.py +++ b/src/wardline/scanner/analyzer.py @@ -17,7 +17,7 @@ from typing import TYPE_CHECKING from wardline.core.finding import ENGINE_PATH, Finding, Kind, Location, Severity -from wardline.core.taints import TaintState, combine +from wardline.core.taints import RAW_ZONE, TaintState, combine from wardline.scanner.context import AnalysisContext, RuleRegistry from wardline.scanner.diagnostics import ( build_collision_findings, @@ -25,17 +25,20 @@ build_metric_finding, build_unknown_import_findings, ) -from wardline.scanner.grammar import TrustGrammar, default_grammar +from wardline.scanner.grammar import TrustGrammar, build_sanitiser_collision_findings, default_grammar from wardline.scanner.index import Entity from wardline.scanner.pipeline import L2FunctionInput, ParseProjectInput, run_l2_function_stage, run_parse_project_stage from wardline.scanner.rules import build_default_registry +from wardline.scanner.rules._sink_helpers import SinkBindings, collect_sink_bindings from wardline.scanner.taint.call_taint_map import build_call_taint_map from wardline.scanner.taint.decorator_provider import ( DecoratorTaintSourceProvider, vocabulary_star_exports, ) +from wardline.scanner.taint.module_summariser import collect_module_global_raw_seeds, own_scope_global_names from wardline.scanner.taint.project_resolver import resolve_project_taints from wardline.scanner.taint.provider import TaintSourceProvider +from wardline.scanner.taint.variable_level import attribute_write_recording, project_attribute_writes if TYPE_CHECKING: from collections.abc import Sequence @@ -52,9 +55,14 @@ def _fp(*parts: str) -> str: _L2Record = tuple[Entity, TaintState, dict[str, TaintState], str, dict[str, str], str, bool] +# The L2 fixed-point memo key. ``seed`` and ``method_tm`` are FIXED per entity across +# iterations (computed once in pass 1), so the key carries only the iteration-VARYING +# inputs: the class-attribute overlay and the parameter meets — both O(per-function). +# Keying on the full sorted taint map was O(project) per function per iteration, +# the whole-scan O(n^2) hotspot (resource-exhaustion on large/adversarial trees). type _L2InputKey = tuple[ - TaintState, - tuple[tuple[str, TaintState], ...], + tuple[tuple[str, TaintState], ...] | None, + tuple[tuple[str, TaintState], ...] | None, tuple[tuple[str, TaintState], ...] | None, ] type _L2Result = tuple[ @@ -66,6 +74,143 @@ def _fp(*parts: str) -> str: dict[str, dict[str, TaintState]], ] +# Above this many candidate-key probes, fall back to the full (unpruned) per-function +# taint map — a sound, slower path for a single pathologically token-dense function. +_CANDIDATE_KEY_BUDGET = 50_000 + + +def _pruned_method_taint_map( + node: ast.AST, + alias_map: dict[str, str], + module_prefix: str, + call_tm: dict[str, TaintState], + project_return_taints: dict[str, TaintState], +) -> dict[str, TaintState]: + """Restrict the per-function taint map to keys the function can actually look up. + + Folding the whole project's return-taint map into EVERY function's taint map made + each per-function L2 run O(project): the map copy in ``analyze_function_variables``, + the per-call ``frozenset(taint_map.keys())``, and the fixed-point memo key all scale + with the map — an O(n^2) whole-scan blowup. The L2 resolver only ever looks a key up + by a form DERIVED FROM SOURCE TOKENS in the function body (variable_level.py): + + * a bare name / literal dotted chain as written (``foo`` / ``mod.fn`` / ``self.x``), + * an alias-resolved chain (``resolve_call_fqn`` / ``_resolve_expr_fqn``: + ``alias_map[root] + rest``), + * a module-local candidate (``{module_prefix}.{name}``), and + * a tracked-receiver-type method key (``{class_fqn}.{attr}`` where the class FQN + is itself one of the resolved forms above and ``attr`` is an attribute token). + + So the restriction of the merged map to those candidate forms is lookup-equivalent + to the full map: every key the body can derive is present with the same value + (project return taints win over ``call_tm`` on conflict, matching the previous + ``dict(call_tm); update(project_return_taints)`` precedence), and never-derivable + keys cannot be consulted. Extra candidates that happen to exist in the merged map + are included harmlessly (the full map contained them too). + """ + chains: set[str] = set() + attrs: set[str] = set() + for n in ast.walk(node): + if isinstance(n, ast.Name): + chains.add(n.id) + elif isinstance(n, ast.Attribute): + attrs.add(n.attr) + parts: list[str] = [] + cur: ast.expr = n + while isinstance(cur, ast.Attribute): + parts.append(cur.attr) + cur = cur.value + if isinstance(cur, ast.Name): + parts.append(cur.id) + chains.add(".".join(reversed(parts))) + forms: set[str] = set(chains) + for chain in chains: + root, _, rest = chain.partition(".") + target = alias_map.get(root) + if target is not None: + forms.add(f"{target}.{rest}" if rest else target) + if module_prefix: + forms.add(f"{module_prefix}.{chain}") + if len(forms) * (len(attrs) + 1) > _CANDIDATE_KEY_BUDGET: + merged = dict(call_tm) + merged.update(project_return_taints) + return merged + + tm: dict[str, TaintState] = {} + + def _take(key: str) -> None: + value = project_return_taints.get(key) + if value is None: + value = call_tm.get(key) + if value is not None: + tm[key] = value + + for form in forms: + _take(form) + for attr in attrs: + _take(f"{form}.{attr}") + return tm + + +def _with_module_global_params( + node: ast.FunctionDef | ast.AsyncFunctionDef, + param_meets: dict[str, TaintState] | None, + global_seeds: dict[str, TaintState] | None, +) -> tuple[ast.FunctionDef | ast.AsyncFunctionDef, dict[str, TaintState] | None]: + """Present raw MODULE GLOBALS to one function's L2 walk as implicit parameters. + + The L2 walk resolves a bare name as ``var_taints.get(name, function_taint)`` — + so an unassigned module-global read would otherwise inherit the trusted caller + seed (laundering the module-level taint, wardline-66b2c91470). Rather than + threading a new seed channel through the engine, the raw globals the body + references are appended as SYNTHETIC keyword-only parameters on a shallow + wrapper node — sharing the ORIGINAL body/decorator statement objects, so the + ``id()``-keyed call-site maps stay valid for every downstream consumer — and + their taints are delivered through the existing ``param_meets`` channel. + Semantically, module globals enter the function exactly like implicit + parameters carrying the module-level taint: a function-local assignment then + shadows the seed flow-sensitively, like any reassigned parameter. Names + already bound as real parameters are skipped (a parameter shadows the global + for the whole scope), as are globals the body never mentions (no dead + ``var_taints`` entries). + """ + if not global_seeds: + return node, param_meets + args = node.args + existing = {a.arg for a in (*args.posonlyargs, *args.args, *args.kwonlyargs)} + if args.vararg is not None: + existing.add(args.vararg.arg) + if args.kwarg is not None: + existing.add(args.kwarg.arg) + referenced = {n.id for n in ast.walk(node) if isinstance(n, ast.Name)} + names = sorted(name for name in global_seeds if name in referenced and name not in existing) + if not names: + return node, param_meets + synthetic = [ast.copy_location(ast.arg(arg=name), node) for name in names] + new_args = ast.arguments( + posonlyargs=list(args.posonlyargs), + args=list(args.args), + vararg=args.vararg, + kwonlyargs=[*args.kwonlyargs, *synthetic], + kw_defaults=[*args.kw_defaults, *([None] * len(synthetic))], + kwarg=args.kwarg, + defaults=list(args.defaults), + ) + wrapper = type(node)( + name=node.name, + args=new_args, + body=node.body, + decorator_list=node.decorator_list, + returns=node.returns, + type_comment=node.type_comment, + type_params=list(getattr(node, "type_params", [])), + ) + ast.copy_location(wrapper, node) + merged = dict(param_meets or {}) + for name in names: + merged[name] = combine(merged[name], global_seeds[name]) if name in merged else global_seeds[name] + return wrapper, merged + class WardlineAnalyzer: """SP1 analyzer implementing core.protocols.Analyzer.""" @@ -128,6 +273,11 @@ def _analyze_inner(self, files: Sequence[Path], config: WardlineConfig, *, root: file_meta = parse_stage.files parse_findings = list(parse_stage.parse_findings) dirty_modules = set(parse_stage.dirty_modules) + # Entity-qualname seeds applied in the parse stage ARE the directive taking + # effect — count them as matched, or a working untrusted_sources entry that + # names a project function is misreported as WLN-CONFIG-UNUSED-SOURCE (only + # the import/alias path in build_call_taint_map recorded matches before). + matched_sources.update(parse_stage.matched_config_sources) # Use the SHADOW-AWARE provider fingerprint computed during the parse stage # for BOTH the dirty-detection key (above, inside the parse stage) AND the @@ -162,6 +312,27 @@ def _analyze_inner(self, files: Sequence[Path], config: WardlineConfig, *, root: # @trust_boundary validator) body != return, and using body here would # mis-read validated output as raw (over-taint -> PY-WL-101 false positive). project_return_taints = dict(result.return_taint_map) + + # Nested-def RETURN taints, bucketed by their enclosing entity — injected + # into the enclosing function's per-entity taint map under the helper's + # BARE name. The L2 bare-name resolution otherwise treats an already- + # analyzed local helper (``m.f..clean``) as an unknown callee and + # applies the worst-arg conservatism, turning every local validate/parse + # helper into a PY-WL-101 ERROR FP (2026-06-10 review; the launder-closing + # conservatism of wardline-93d608c997 is preserved for genuinely-unknown + # bare names). This pass-1 seed uses the L1 return tiers; the fixed-point + # loop below overlays the PRECISE L2 actual-return taints once they exist + # (an undecorated ``def clean(x): return 1`` is UNKNOWN_RAW at L1 but + # INTEGRAL at L2). Built once per scan — a per-entity scan of the project + # map would reintroduce the O(n^2) hotspot the pruned taint map removed. + # Lexically sound: a nested def shadows any module-level/imported callable + # of the same name for the whole enclosing scope. + nested_def_returns: dict[str, dict[str, TaintState]] = {} + for nested_qn, nested_taint in project_return_taints.items(): + enclosing_qn, sep, bare = nested_qn.rpartition("..") + if sep and "." not in bare: + nested_def_returns.setdefault(enclosing_qn, {})[bare] = nested_taint + project_by_module: dict[str, dict[str, TaintState]] = {} for parsed in file_meta: module = parsed.module @@ -332,6 +503,7 @@ def _run_l2( alias_map: dict[str, str], param_meets: dict[str, TaintState] | None = None, module_prefix: str | None = None, + global_seeds: dict[str, TaintState] | None = None, ) -> tuple[ dict[int, dict[str, TaintState]], dict[int, dict[int | str | None, TaintState]], @@ -339,6 +511,8 @@ def _run_l2( TaintState | None, str | None, ]: + # Module-global taint channel: raw module globals enter as implicit params. + node, param_meets = _with_module_global_params(node, param_meets, global_seeds) result = run_l2_function_stage( L2FunctionInput( node=node, @@ -374,8 +548,58 @@ def _store( function_return_taints.pop(qn, None) function_return_callee[qn] = ret_callee + # ── Module-scope channels (wardline-66b2c91470 / wardline-13cfdd7b31) ── + # (a) module-level name bindings (callable aliases / constructed instances) for + # the sink rules' binding-aware resolution, exposed on the context; + # (b) module-global RAW seeds — module-level names assigned from a raw source at + # import time, presented to each function's L2 walk as implicit raw parameters. + module_sink_bindings: dict[str, SinkBindings] = {} + module_global_taints: dict[str, dict[str, TaintState]] = {} + for parsed in file_meta: + module_sink_bindings[parsed.module] = collect_sink_bindings(parsed.tree, parsed.alias_map, parsed.module) + global_raw_seeds = collect_module_global_raw_seeds( + parsed.tree, + module=parsed.module, + alias_map=parsed.alias_map, + return_taints=project_return_taints, + local_fqns=frozenset(ent.qualname for ent in parsed.entities), + untrusted_sources=frozenset(config.untrusted_sources), + ) + if global_raw_seeds: + module_global_taints[parsed.module] = global_raw_seeds + # ── L2 pass 1 — per-method var/return taints + per-class attribute summary ── all_classes = frozenset(c for parsed in file_meta for c in parsed.class_qualnames) + failed_paths: set[str] = set() + + def _record_file_failure(relpath: str, ent: Entity, exc: Exception) -> None: + # Per-file isolation, mirroring the Rust frontend's WLN-ENGINE-FILE-FAILED: + # an unexpected engine exception on ONE file's analysis must not abort the + # whole scan (losing every other file's findings) — and must not silently + # skip either. Fail closed: a gate-eligible ERROR DEFECT names the file + # (rule id already in UNANALYZED_RULE_IDS, so it counts toward + # ScanSummary.unanalyzed). One finding per file — the fingerprint is keyed + # on the relpath (the Rust contract), so a second failing entity in the + # same file must not mint a colliding distinct finding. ``line_start`` + # falls back to the entity line so the lineless-DEFECT downgrade + # (suppression.py) never demotes it back out of the gate. + l2_failed.add(ent.qualname) + if relpath in failed_paths: + return + failed_paths.add(relpath) + func_skip_findings.append( + Finding( + rule_id="WLN-ENGINE-FILE-FAILED", + message=f"{relpath}: analysis failed at {ent.qualname} ({type(exc).__name__}: {exc})", + severity=Severity.ERROR, + kind=Kind.DEFECT, + location=Location(path=relpath, line_start=ent.location.line_start or 1), + fingerprint=_fp("WLN-ENGINE-FILE-FAILED", relpath), + qualname=ent.qualname, + properties={"reason": "analysis_exception", "exception": type(exc).__name__}, + ) + ) + for parsed in file_meta: module = parsed.module entities = parsed.entities @@ -392,28 +616,53 @@ def _store( matched_sources=matched_sources, matched_sanitisers=matched_sanitisers, ) + # Per-class sibling RETURN-taint entries, built ONCE per module (the + # previous per-method rescan of all module entities was O(entities) per + # method). A caller observes a sibling's RETURN taint via self./cls. + sibling_tm: dict[str, dict[str, TaintState]] = {} + for ent in entities: + enclosing = ent.qualname.rsplit(".", 1)[0] + if enclosing in classes: + sib_name = ent.qualname[len(enclosing) + 1 :] + sib_taint = project_return_taints.get(ent.qualname, TaintState.UNKNOWN_RAW) + bucket = sibling_tm.setdefault(enclosing, {}) + bucket[f"self.{sib_name}"] = sib_taint + bucket[f"cls.{sib_name}"] = sib_taint for ent in entities: entity_index[ent.qualname] = ent seed = project_taints.get(ent.qualname, TaintState.UNKNOWN_RAW) - method_tm = dict(call_tm) - method_tm.update(project_return_taints) enclosing_class = ent.qualname.rsplit(".", 1)[0] is_method = enclosing_class in classes - if is_method: - sib_prefix = enclosing_class + "." - for sib in entities: - if sib.qualname.startswith(sib_prefix) and "." not in sib.qualname[len(sib_prefix) :]: - sib_name = sib.qualname[len(sib_prefix) :] - sib_taint = project_return_taints.get(sib.qualname, TaintState.UNKNOWN_RAW) - method_tm[f"self.{sib_name}"] = sib_taint - method_tm[f"cls.{sib_name}"] = sib_taint + # Attribute writes are recorded DURING the L2 walk (per-statement + # var_taints, branch-aware receiver types) — a post-hoc second walk + # against the FINAL var_taints laundered reassigned-after-write RHS + # variables and branch-rebound receivers (wardline-b369c7d06c). + recorded_writes: dict[str, dict[str, TaintState]] = {} + writes: dict[str, dict[str, TaintState]] = {} + method_tm: dict[str, TaintState] = {} try: - call_sites, call_args, var_taints, ret_taint, ret_callee = _run_l2( - ent.node, seed, method_tm, alias_map, module_prefix=module + method_tm = _pruned_method_taint_map(ent.node, alias_map, module, call_tm, project_return_taints) + if is_method: + method_tm.update(sibling_tm.get(enclosing_class, {})) + # Own nested defs shadow same-named module/imported callables + # for the whole scope, so they layer LAST (see nested_def_returns). + method_tm.update(nested_def_returns.get(ent.qualname, {})) + with attribute_write_recording(recorded_writes): + call_sites, call_args, var_taints, ret_taint, ret_callee = _run_l2( + ent.node, + seed, + method_tm, + alias_map, + module_prefix=module, + global_seeds=module_global_taints.get(module), + ) + writes = project_attribute_writes( + recorded_writes, all_classes, enclosing_class if is_method else None ) except RecursionError: l2_failed.add(ent.qualname) call_sites, call_args, var_taints, ret_taint, ret_callee = {}, {}, {}, None, None + writes = {} func_skip_findings.append( Finding( rule_id="WLN-ENGINE-FUNCTION-SKIPPED", @@ -426,28 +675,44 @@ def _store( properties={"reason": "recursion_limit"}, ) ) + except MemoryError: + raise # exhaustion is not a per-file condition — isolating it would thrash + except Exception as exc: # noqa: BLE001 — per-file isolation, see _record_file_failure + call_sites, call_args, var_taints, ret_taint, ret_callee = {}, {}, {}, None, None + writes = {} + _record_file_failure(parsed.relpath, ent, exc) _store(ent.qualname, call_sites, call_args, var_taints, ret_taint, ret_callee) project_call_site_arg_taints.update(call_args) l2_records.append((ent, seed, method_tm, enclosing_class, alias_map, module, is_method)) - if ent.qualname not in l2_failed: - from wardline.scanner.taint.variable_level import collect_attribute_writes - - writes = collect_attribute_writes( - ent.node, - seed, - dict(method_tm), - dict(var_taints), - all_classes, - alias_map, - module, - enclosing_class=enclosing_class if is_method else None, - ) - for target_class, cls_writes in writes.items(): - summary = class_attr_taints.setdefault(target_class, {}) - for attr_name, attr_taint in cls_writes.items(): - summary[attr_name] = ( - combine(summary[attr_name], attr_taint) if attr_name in summary else attr_taint - ) + for target_class, cls_writes in writes.items(): + summary = class_attr_taints.setdefault(target_class, {}) + for attr_name, attr_taint in cls_writes.items(): + summary[attr_name] = ( + combine(summary[attr_name], attr_taint) if attr_name in summary else attr_taint + ) + + # Module-global taint channel, WRITE direction: a function assigning raw to a + # declared ``global g`` marks the module global; the fixed-point loop below + # re-runs every function with the merged seeds, so OTHER functions reading + # ``g`` inherit. ONE merge hop only (documented approximation): the write + # taints are read from pass 1 and stay FIXED through the loop — a raw value + # routed global→function→second global needs a second hop and is a bounded + # FN. The write taint is the function's FINAL (exit-state) L2 taint for the + # name; only RAW_ZONE writes are recorded (the channel propagates raw, it + # never upgrades a module-level raw seed to clean), and seeds combine + # least-trusted-wins with the import-time seeds. + for ent, _seed, _tm, _enclosing_class, _alias_map, module, _is_method in l2_records: + if ent.qualname in l2_failed: + continue + global_names = own_scope_global_names(ent.node) + if not global_names: + continue + final_vars = function_var_taints.get(ent.qualname, {}) + for name in global_names: + write_taint = final_vars.get(name) + if write_taint is not None and write_taint in RAW_ZONE: + bucket = module_global_taints.setdefault(module, {}) + bucket[name] = combine(bucket[name], write_taint) if name in bucket else write_taint # Compute initial project-wide parameter meets from pass-1 call sites project_param_meets: dict[str, dict[str, TaintState]] = {} @@ -468,6 +733,19 @@ def _store( else: callee_meets[param] = taint + def _l2_nested_def_overlay() -> dict[str, dict[str, TaintState]]: + """Per-enclosing-entity nested-def bare-name map from the PRECISE L2 + actual-return taints (``function_return_taints``); see the pass-1 + ``nested_def_returns`` seed for the rationale. Recomputed per fixed- + point iteration so a helper chain converges; participates in the memo + key and the convergence check below.""" + overlay: dict[str, dict[str, TaintState]] = {} + for nested_qn, nested_taint in function_return_taints.items(): + enclosing_qn, sep, bare = nested_qn.rpartition("..") + if sep and "." not in bare: + overlay.setdefault(enclosing_qn, {})[bare] = nested_taint + return overlay + # ── Iterative Fixed-point L2 Loop to converge parameters and attributes ── l2_iteration_bound = _l2_iteration_bound(l2_records) l2_converged = False @@ -476,6 +754,7 @@ def _store( for _iteration in range(l2_iteration_bound): old_class_attr_taints = {k: dict(v) for k, v in class_attr_taints.items()} old_project_param_meets = {k: dict(v) for k, v in project_param_meets.items()} + nested_def_overlay = _l2_nested_def_overlay() # Run L2 pass on all functions with current class_attr_taints and project_param_meets class_attr_taints = {} @@ -483,40 +762,54 @@ def _store( for ent, seed, method_tm, enclosing_class, alias_map, module, is_method in l2_records: if ent.qualname in l2_failed: continue - tm_iter = dict(method_tm) attr_summary = old_class_attr_taints.get(enclosing_class) - if attr_summary: - for attr_name, attr_taint in attr_summary.items(): - tm_iter[f"self.{attr_name}"] = attr_taint - tm_iter[f"cls.{attr_name}"] = attr_taint param_meets = old_project_param_meets.get(ent.qualname) + nested_map = nested_def_overlay.get(ent.qualname) + # ``seed``/``method_tm`` are fixed per entity, so the memo key carries + # only the iteration-varying inputs (see ``_L2InputKey``) — and on a + # hit the O(per-function) ``tm_iter`` copy is skipped entirely. inputs_key = ( - seed, - tuple(sorted(tm_iter.items())), + tuple(sorted(attr_summary.items())) if attr_summary else None, tuple(sorted(param_meets.items())) if param_meets else None, + tuple(sorted(nested_map.items())) if nested_map else None, ) if last_l2_inputs.get(ent.qualname) == inputs_key: call_sites, call_args, var_taints, ret_taint, ret_callee, writes = last_l2_results[ent.qualname] else: + tm_iter = dict(method_tm) + if attr_summary: + for attr_name, attr_taint in attr_summary.items(): + tm_iter[f"self.{attr_name}"] = attr_taint + tm_iter[f"cls.{attr_name}"] = attr_taint + if nested_map: + # Own nested defs shadow same-named module/imported callables + # (and the pass-1 L1 seed) for the whole scope — layered last. + tm_iter.update(nested_map) + recorded_writes = {} try: - call_sites, call_args, var_taints, ret_taint, ret_callee = _run_l2( - ent.node, seed, tm_iter, alias_map, param_meets=param_meets, module_prefix=module - ) - from wardline.scanner.taint.variable_level import collect_attribute_writes - - writes = collect_attribute_writes( - ent.node, - seed, - dict(tm_iter), - dict(var_taints), - all_classes, - alias_map, - module, - enclosing_class=enclosing_class if is_method else None, + with attribute_write_recording(recorded_writes): + # ``module_global_taints`` is FIXED during this loop (merged + # once after pass 1), so the ``inputs_key`` memo stays valid. + call_sites, call_args, var_taints, ret_taint, ret_callee = _run_l2( + ent.node, + seed, + tm_iter, + alias_map, + param_meets=param_meets, + module_prefix=module, + global_seeds=module_global_taints.get(module), + ) + writes = project_attribute_writes( + recorded_writes, all_classes, enclosing_class if is_method else None ) except RecursionError: continue + except MemoryError: + raise # exhaustion is not a per-file condition — isolating it would thrash + except Exception as exc: # noqa: BLE001 — per-file isolation, see _record_file_failure + _record_file_failure(ent.location.path, ent, exc) + continue last_l2_inputs[ent.qualname] = inputs_key last_l2_results[ent.qualname] = (call_sites, call_args, var_taints, ret_taint, ret_callee, writes) @@ -549,8 +842,14 @@ def _store( else: callee_meets[param] = taint - # Break if class_attr_taints and project_param_meets did not change - if class_attr_taints == old_class_attr_taints and project_param_meets == old_project_param_meets: + # Break if class_attr_taints, project_param_meets, and the nested-def + # return overlay did not change (the overlay feeds tm_iter, so a still- + # moving helper chain must keep iterating). + if ( + class_attr_taints == old_class_attr_taints + and project_param_meets == old_project_param_meets + and _l2_nested_def_overlay() == nested_def_overlay + ): l2_converged = True break @@ -581,6 +880,22 @@ def _store( ) ) + # The registry is built BEFORE the context so the selected rule-id set can + # ride on it (PY-WL-120's suppress-and-delegate consults enablement; a + # duck-typed registry seam without a ``rules`` property yields None — + # "unknown", the historical assume-enabled posture). + registry = ( + self._registry + if self._registry is not None + else build_default_registry(config, rules=(self._grammar.rules if self._grammar is not None else None)) + ) + registry_rules = getattr(registry, "rules", None) + enabled_rule_ids = ( + frozenset(str(getattr(rule, "rule_id", type(rule).__name__)) for rule in registry_rules) + if registry_rules is not None + else None + ) + context = AnalysisContext( project_taints=project_taints, project_return_taints=project_return_taints, @@ -598,6 +913,8 @@ def _store( project_edges=result.project_edges, call_site_implicit_receivers=result.call_site_implicit_receivers, alias_maps={m.module_path: m.alias_map for m in modules}, + module_bindings=module_sink_bindings, + enabled_rule_ids=enabled_rule_ids, ) self.last_context = context @@ -646,13 +963,65 @@ def _store( properties={"sanitiser": san}, ) ) + # A sanitiser colliding with a modelled serialisation sink would otherwise be + # dropped silently; the collision FACT speaks instead of UNUSED-SANITISER. + findings.extend(build_sanitiser_collision_findings(config.sanitisers)) - registry = ( - self._registry - if self._registry is not None - else build_default_registry(config, rules=(self._grammar.rules if self._grammar is not None else None)) - ) - findings.extend(registry.run(context)) + # Per-rule isolation: one crashing rule must not abort the whole scan and + # silently lose every other rule's findings. Each rule runs in its own + # single-rule registry (reusing RuleRegistry.run's maturity stamping); a + # raise becomes a gate-eligible ERROR DEFECT at ENGINE_PATH (the lineless + # downgrade exempts ENGINE_PATH, so it trips --fail-on ERROR — fail-closed, + # same posture as WLN-ENGINE-FINGERPRINT-COLLISION). Duck-typed registry + # seams without a ``rules`` property keep the undelegated single call. + if registry_rules is None: + findings.extend(registry.run(context)) + else: + for rule in registry_rules: + solo = RuleRegistry() + solo.register(rule) + try: + findings.extend(solo.run(context)) + except Exception as exc: # noqa: BLE001 — per-rule isolation, see above + rid = str(getattr(rule, "rule_id", type(rule).__name__)) + findings.append( + Finding( + rule_id="WLN-ENGINE-RULE-FAILED", + message=( + f"rule {rid} aborted ({type(exc).__name__}: {exc}) — " + "its findings are missing from this scan" + ), + severity=Severity.ERROR, + kind=Kind.DEFECT, + location=Location(path=ENGINE_PATH), + fingerprint=_fp("WLN-ENGINE-RULE-FAILED", rid), + properties={"rule": rid, "exception": type(exc).__name__}, + ) + ) + # Sink-argument resolution degraded to the pessimistic flow-INSENSITIVE + # fallback somewhere (an L2-skipped function): surface the recorded set as + # ONE NONE/FACT finding per scan, mirroring WLN-ENGINE-FUNCTION-SKIPPED. A + # finding, not a UserWarning — MCP/library consumers see the degradation, + # and a warnings-as-error embedder cannot turn the diagnostic into a + # rule-aborting raise (review 2026-06-10). + if context.flow_insensitive_fallbacks: + findings.append( + Finding( + rule_id="WLN-ENGINE-FLOW-INSENSITIVE-FALLBACK", + message=( + "sink-argument taint resolution fell back to the pessimistic " + f"flow-insensitive map for {len(context.flow_insensitive_fallbacks)} " + "function(s) — their sink findings assume UNKNOWN_RAW arguments" + ), + severity=Severity.NONE, + kind=Kind.FACT, + location=Location(path=ENGINE_PATH), + # Keyed on the engine path only (never the affected qualnames) so the + # one-per-scan FACT keeps a stable identity as the set churns. + fingerprint=_fp("WLN-ENGINE-FLOW-INSENSITIVE-FALLBACK", ENGINE_PATH), + properties={"qualnames": sorted(context.flow_insensitive_fallbacks)}, + ) + ) # Proactive no-collision guard (wardline-8fb773a7af): every fingerprint # consumer joins on Finding.fingerprint as a unique key, so two DISTINCT # findings sharing one is a silent false-negative. Run last, over the full diff --git a/src/wardline/scanner/context.py b/src/wardline/scanner/context.py index 201c036f..70c086f1 100644 --- a/src/wardline/scanner/context.py +++ b/src/wardline/scanner/context.py @@ -23,6 +23,7 @@ from wardline.core.finding import Finding, Severity from wardline.core.taints import TaintState from wardline.scanner.index import Entity + from wardline.scanner.rules._sink_helpers import SinkBindings from wardline.scanner.taint.propagation import TaintProvenance @@ -98,6 +99,29 @@ class AnalysisContext: project_edges: Mapping[str, frozenset[str]] = field(default_factory=dict) # Import alias maps per module: ``{module: {alias: target_fqn}}``. alias_maps: Mapping[str, Mapping[str, str]] = field(default_factory=dict) + # MODULE-SCOPE name bindings per module: ``{module: SinkBindings}`` — module-level + # callable aliases (``runner = subprocess.run``) and constructed instances + # (``client = httpx.Client()``), collected from each module's top-level scope by the + # analyzer. The sink machinery layers a function's own bindings OVER these + # (``resolved_sink_calls(..., module_bindings=...)``), closing the documented + # module-level false negatives (wardline-13cfdd7b31 / wardline-66b2c91470). + # Defaulted so direct constructions (tests) need not supply it; absence degrades to + # function-scope-only binding resolution. + module_bindings: Mapping[str, SinkBindings] = field(default_factory=dict) + # Rule ids selected for THIS run (``rules.enable``), or ``None`` when unknown + # (direct constructions / duck-typed registry seams without a ``rules`` + # property). A rule that suppresses-and-delegates to a sibling (PY-WL-120 → + # PY-WL-101) consults this so it never delegates to a rule that will not run; + # ``None`` preserves the historical assume-enabled behavior. + enabled_rule_ids: frozenset[str] | None = None + # Per-scan degradation channel: qualnames whose sink-argument resolution fell + # back to the pessimistic flow-INSENSITIVE map (no L2 snapshot — an L2-skipped + # function). ``resolved_arg_taints`` records here instead of warning from + # inside rule ``check()`` calls; the analyzer surfaces the collected set as ONE + # ``WLN-ENGINE-FLOW-INSENSITIVE-FALLBACK`` NONE/FACT finding per scan + # (mirroring WLN-ENGINE-FUNCTION-SKIPPED). Deliberately a mutable set on a + # frozen context: it is a diagnostics side channel, not engine output. + flow_insensitive_fallbacks: set[str] = field(default_factory=set) def __post_init__(self) -> None: object.__setattr__(self, "project_taints", _freeze_mapping(self.project_taints)) @@ -135,6 +159,8 @@ def __post_init__(self) -> None: "alias_maps", _freeze_mapping(self.alias_maps), ) + # SinkBindings values are frozen dataclasses — only the outer map needs the proxy. + object.__setattr__(self, "module_bindings", _freeze_mapping(self.module_bindings)) class _RuleClass(Protocol): diff --git a/src/wardline/scanner/grammar.py b/src/wardline/scanner/grammar.py index 821e847a..b0962cf3 100644 --- a/src/wardline/scanner/grammar.py +++ b/src/wardline/scanner/grammar.py @@ -17,7 +17,7 @@ from __future__ import annotations -from collections.abc import Callable, Mapping +from collections.abc import Callable, Iterable, Mapping from dataclasses import dataclass from typing import TYPE_CHECKING @@ -28,6 +28,7 @@ if TYPE_CHECKING: # Annotation-only (lazy under `from __future__ import annotations`); kept out of # the runtime import surface so the zero-dep contract above stays literally true. + from wardline.core.finding import Finding from wardline.scanner.context import _RuleClass _VOCAB_PREFIX = "wardline.decorators" @@ -173,6 +174,52 @@ def extend( ) +def build_sanitiser_collision_findings(sanitisers: Iterable[str]) -> list[Finding]: + """WLN-CONFIG-* diagnostic: configured sanitisers shadowed by a serialisation sink. + + A config sanitiser whose dotted name IS a built-in serialisation sink (e.g. + ``json.loads``) can never take effect: the sink override is inserted into the + call-taint map before the config pass (which uses ``setdefault``), and + ``_resolve_call`` consults the sink set ahead of the taint map. Yet the + sanitiser still generates map keys and so counts as "matched", which + suppresses ``WLN-CONFIG-UNUSED-SANITISER`` — the declaration becomes a silent + no-op. This emits one ``WLN-CONFIG-SANITISER-SINK-COLLISION`` FACT per + colliding sanitiser (sorted, deterministic), naming the collision so the user + learns their suppression attempt was overridden, not honoured. + + Pure function of the config value (no scan state); the analyzer appends the + result alongside the other ``WLN-CONFIG-*`` diagnostics. + """ + # Local imports: keep this module's runtime import surface exactly the grammar + # meta-model (same pattern as default_grammar's rules import). + from wardline.core.finding import Finding, Kind, Location, Severity, compute_finding_fingerprint + from wardline.scanner.taint.variable_level import _SERIALISATION_SINKS + + findings: list[Finding] = [] + for san in sorted(set(sanitisers) & _SERIALISATION_SINKS): + findings.append( + Finding( + rule_id="WLN-CONFIG-SANITISER-SINK-COLLISION", + message=( + f"Configuration error: sanitiser '{san}' collides with the built-in " + "serialisation sink of the same name; the conservative sink " + "classification (UNKNOWN_RAW) takes precedence, so this sanitiser " + "declaration has no effect" + ), + severity=Severity.NONE, + kind=Kind.FACT, + location=Location(path="weft.toml"), + fingerprint=compute_finding_fingerprint( + rule_id="WLN-CONFIG-SANITISER-SINK-COLLISION", + path="weft.toml", + taint_path=san, + ), + properties={"sanitiser": san}, + ) + ) + return findings + + def default_grammar() -> TrustGrammar: """The builtin grammar: the 3 boundary types + the 4 rule classes, in today's exact order. The byte-identity oracle (design spec §5) pins this == today.""" diff --git a/src/wardline/scanner/pipeline.py b/src/wardline/scanner/pipeline.py index b7344fb2..da8dc855 100644 --- a/src/wardline/scanner/pipeline.py +++ b/src/wardline/scanner/pipeline.py @@ -60,6 +60,11 @@ class ParseProjectOutput: parse_findings: list[Finding] dirty_modules: frozenset[str] provider_fingerprint: str + # config.untrusted_sources entries that matched a PROJECT ENTITY QUALNAME and + # were applied as seeds here. The analyzer unions these into its matched-source + # bookkeeping — without this channel a WORKING directive was misreported as + # WLN-CONFIG-UNUSED-SOURCE (only the import/alias path recorded matches). + matched_config_sources: frozenset[str] = frozenset() def _provider_fingerprint_for_project(provider: TaintSourceProvider, project_modules: frozenset[str]) -> str: @@ -84,6 +89,7 @@ def run_parse_project_stage(stage_input: ParseProjectInput) -> ParseProjectOutpu parsed_files: list[ParsedFile] = [] parse_findings: list[Finding] = [] dirty_modules: set[str] = set() + matched_config_sources: set[str] = set() root = stage_input.root.resolve() # The set of dotted module names in the scan. Used to fail closed for builtin @@ -157,6 +163,9 @@ def run_parse_project_stage(stage_input: ParseProjectInput) -> ParseProjectOutpu if ent.qualname in stage_input.config.untrusted_sources: from wardline.scanner.taint.function_level import FunctionSeed + # The seed below IS the directive taking effect — record the match + # so the analyzer's unused-source diagnostic does not contradict it. + matched_config_sources.add(ent.qualname) seeds[ent.qualname] = FunctionSeed( qualname=ent.qualname, body_taint=TaintState.EXTERNAL_RAW, @@ -165,15 +174,26 @@ def run_parse_project_stage(stage_input: ParseProjectInput) -> ParseProjectOutpu unprovable_boundaries=(), ) except (SyntaxError, UnicodeDecodeError, OSError) as exc: + # A discovered-but-unparseable file is a GATE-ELIGIBLE ERROR DEFECT, not a + # NONE FACT: its sinks were never analyzed, so a default `--fail-on ERROR` + # reading GREEN over it is a fail-open (e.g. a latin-1 coding cookie that + # CPython runs but this UTF-8 reader rejects hides live code from the scan). + # Severity ERROR so the documented agent loop (`scan . --fail-on ERROR`) + # trips — the secure-by-default posture (same as the suppression gate and + # WLN-ENGINE-FINGERPRINT-COLLISION). Repository baseline/waiver still + # ANNOTATE it but cannot clear the secure gate; `--trust-suppressions` can + # (an explicit operator trust decision). ``line_start`` falls back to 1 so + # the lineless-DEFECT downgrade (suppression.py) never demotes it back out + # of the gate for read/encoding errors that carry no line. msg = getattr(exc, "msg", None) or str(exc) lineno = exc.lineno if isinstance(exc, SyntaxError) else None parse_findings.append( Finding( rule_id="WLN-ENGINE-PARSE-ERROR", message=f"{relpath}: could not read/parse ({msg})", - severity=Severity.NONE, - kind=Kind.FACT, - location=Location(path=relpath, line_start=lineno), + severity=Severity.ERROR, + kind=Kind.DEFECT, + location=Location(path=relpath, line_start=lineno or 1), fingerprint=_fp("WLN-ENGINE-PARSE-ERROR", relpath), properties={"module": module}, ) @@ -192,6 +212,24 @@ def run_parse_project_stage(stage_input: ParseProjectInput) -> ParseProjectOutpu ) ) continue + except Exception as exc: # noqa: BLE001 — per-file isolation, mirrors the Rust frontend + # An UNEXPECTED exception while indexing/seeding one file must not abort + # the whole scan (losing every other file's findings) — and must not be a + # silent skip either. Fail closed: a WLN-ENGINE-FILE-FAILED ERROR DEFECT + # (gate-eligible, counted toward ScanSummary.unanalyzed) names the file, + # and the scan continues — the Rust frontend's per-file contract. + parse_findings.append( + Finding( + rule_id="WLN-ENGINE-FILE-FAILED", + message=f"{relpath}: analysis failed ({type(exc).__name__}: {exc})", + severity=Severity.ERROR, + kind=Kind.DEFECT, + location=Location(path=relpath, line_start=1), + fingerprint=_fp("WLN-ENGINE-FILE-FAILED", relpath), + properties={"module": module, "reason": "analysis_exception", "exception": type(exc).__name__}, + ) + ) + continue modules.append( ModuleInput( @@ -240,6 +278,7 @@ def run_parse_project_stage(stage_input: ParseProjectInput) -> ParseProjectOutpu parse_findings=parse_findings, dirty_modules=frozenset(dirty_modules), provider_fingerprint=provider_fingerprint, + matched_config_sources=frozenset(matched_config_sources), ) diff --git a/src/wardline/scanner/rules/__init__.py b/src/wardline/scanner/rules/__init__.py index e351b9d2..aac657bb 100644 --- a/src/wardline/scanner/rules/__init__.py +++ b/src/wardline/scanner/rules/__init__.py @@ -35,8 +35,14 @@ from wardline.scanner.rules.untrusted_to_deserialization import UntrustedToDeserialization from wardline.scanner.rules.untrusted_to_exec import UntrustedToExec from wardline.scanner.rules.untrusted_to_import import UntrustedToImport +from wardline.scanner.rules.untrusted_to_log import UntrustedToLog +from wardline.scanner.rules.untrusted_to_mail import UntrustedToMail +from wardline.scanner.rules.untrusted_to_native import UntrustedToNative +from wardline.scanner.rules.untrusted_to_reflection import UntrustedToReflection from wardline.scanner.rules.untrusted_to_shell_subprocess import UntrustedToShellSubprocess +from wardline.scanner.rules.untrusted_to_template import UntrustedToTemplate from wardline.scanner.rules.untrusted_to_trusted_callee import UntrustedReachesTrustedCallee +from wardline.scanner.rules.untrusted_to_xml import UntrustedToXML if TYPE_CHECKING: from wardline.core.config import WardlineConfig @@ -64,6 +70,13 @@ SQLInjection, DegenerateBoundary, StoredTaint, + # PY-WL-121…126 — the 2026-06-10 coverage-gap sink families (all PREVIEW). + UntrustedToXML, + UntrustedToTemplate, + UntrustedToReflection, + UntrustedToNative, + UntrustedToLog, + UntrustedToMail, ) diff --git a/src/wardline/scanner/rules/_ast_helpers.py b/src/wardline/scanner/rules/_ast_helpers.py index c1605d59..dceb65ca 100644 --- a/src/wardline/scanner/rules/_ast_helpers.py +++ b/src/wardline/scanner/rules/_ast_helpers.py @@ -4,7 +4,10 @@ All helpers operate on a single function's *own* scope — they never descend into nested ``FunctionDef``/``AsyncFunctionDef``/``ClassDef`` bodies, so a finding is attributed to the function that lexically owns the construct (nested functions -are separate entities and are analysed in their own right). +are separate entities and are analysed in their own right). The single sanctioned +exception is :func:`rejecting_helper_calls`, a ONE-HOP, SAME-MODULE inspection of +a called helper's body — bounded interprocedural sight so factored-out validators +(``_require_nonempty(p)``) are not misread as "no rejection path". """ from __future__ import annotations @@ -13,10 +16,21 @@ from typing import TYPE_CHECKING if TYPE_CHECKING: - from collections.abc import Iterator + from collections.abc import Iterator, Mapping + + from wardline.scanner.index import Entity _BROAD_NAMES: frozenset[str] = frozenset({"Exception", "BaseException"}) +# CURATED raising-conversion callables: constructors that raise (ValueError / +# decimal.InvalidOperation / ...) on EVERY invalid input and return a value of +# guaranteed shape — the canonical validate-by-construction idiom +# (``@trust_boundary def to_port(p): return int(p)``). Deliberately a small +# allowlist matched by (possibly dotted) final name: treating ARBITRARY calls as +# rejections would be an FN hole (any helper call would silence PY-WL-102). +# ``str``/``bool``/``repr`` are absent on purpose — they accept everything. +_RAISING_CONVERSION_NAMES: frozenset[str] = frozenset({"int", "float", "complex", "Decimal", "Fraction", "UUID"}) + def _own_statements(node: ast.AST) -> Iterator[ast.stmt]: """Yield every statement in *node*'s own scope, not descending into nested @@ -86,46 +100,208 @@ def _is_falsy_constant_return(value: ast.expr | None) -> bool: return False +def _is_raising_conversion(value: ast.expr) -> bool: + """True for a CURATED raising-expression: a :data:`_RAISING_CONVERSION_NAMES` + constructor applied to at least one non-constant argument (``int(p)`` — a + constant argument validates nothing), or a Subscript lookup with a + non-constant key (``Color[p]`` raises ``KeyError``; ``ALLOWED[p]`` likewise — + a CONSTANT index like ``parts[0]`` is positional access, not a validating + lookup of the input).""" + if isinstance(value, ast.Call): + func = value.func + if isinstance(func, ast.Name): + name = func.id + elif isinstance(func, ast.Attribute): + name = func.attr + else: + return False + return name in _RAISING_CONVERSION_NAMES and any(not isinstance(arg, ast.Constant) for arg in value.args) + if isinstance(value, ast.Subscript): + return isinstance(value.value, (ast.Name, ast.Attribute)) and not isinstance(value.slice, ast.Constant) + return False + + +def _is_rejection_return(value: ast.expr | None) -> bool: + """A ``return`` value that constitutes a rejection path: a falsy constant, a + conditional expression with a rejecting branch (``return m.group(0) if m else + None`` is the ternary form of ``if not m: return None``), or a curated + raising-conversion (``return int(p)`` rejects-by-construction).""" + if _is_falsy_constant_return(value): + return True + if isinstance(value, ast.IfExp): + return _is_rejection_return(value.body) or _is_rejection_return(value.orelse) + return value is not None and _is_raising_conversion(value) + + +def _stmt_is_real_rejection(stmt: ast.stmt) -> bool: + """A statement that rejects IN PRODUCTION: a ``raise`` or a rejection-shaped + ``return``. Excludes ``assert`` (stripped under ``python -O`` — PY-WL-111's + hazard, never a *real* rejection).""" + if isinstance(stmt, ast.Raise): + return True + return isinstance(stmt, ast.Return) and _is_rejection_return(stmt.value) + + +def has_real_rejection(node: ast.FunctionDef | ast.AsyncFunctionDef) -> bool: + """True when *node*'s own scope contains a production-surviving rejection — + a ``raise`` or a rejection-shaped ``return`` — i.e. NOT counting ``assert``. + This is PY-WL-113's premise half: a rejection must EXIST (and survive ``-O``) + before a fail-open handler can be said to defeat it.""" + return any(_stmt_is_real_rejection(stmt) for stmt in _own_statements(node)) + + def has_rejection_path(node: ast.FunctionDef | ast.AsyncFunctionDef) -> bool: - """True when *node* can reject: any ``raise``, any falsy-constant ``return``, - or any ``assert`` in its own scope. Deliberately generous — PY-WL-102 is - always-on, so we err toward SEEING a rejection path (risk a missed finding) - over firing on a real validator. + """True when *node* can reject: any ``raise``, any rejection-shaped ``return`` + (falsy constant, rejecting ternary branch, curated raising-conversion), or any + ``assert`` in its own scope. Deliberately generous — PY-WL-102 is always-on, + so we err toward SEEING a rejection path (risk a missed finding) over firing + on a real validator. ``assert`` counts as a rejection here so PY-WL-102 does NOT fire on a boundary whose only reject is an assert — that boundary DOES reject at runtime. The distinct hazard (asserts are stripped under ``python -O``, so the validation silently vanishes in production) is PY-WL-111's job, via - :func:`asserts_are_sole_rejection`. The two rules partition the space cleanly: - 102 fires on "no rejection of any shape", 111 on "the only rejection is an - assert" — never both on one boundary.""" - for stmt in _own_statements(node): - if isinstance(stmt, (ast.Raise, ast.Assert)): - return True - if isinstance(stmt, ast.Return) and _is_falsy_constant_return(stmt.value): - return True - return False + :func:`asserts_are_sole_rejection`. + + **The boundary-integrity family partitions FOUR ways** (wardline-718048a518), + at most one of which fires per boundary: + - PY-WL-119 — the bare degenerate shape (:func:`is_degenerate_boundary`); + - PY-WL-102 — any other shape with no rejection path of any kind; + - PY-WL-111 — the only rejection is ``assert``; + - PY-WL-113 — a real rejection exists but a fail-open handler defeats it. + """ + return any(isinstance(stmt, ast.Assert) or _stmt_is_real_rejection(stmt) for stmt in _own_statements(node)) def asserts_are_sole_rejection(node: ast.FunctionDef | ast.AsyncFunctionDef) -> bool: """True when *node*'s ONLY rejection mechanism is ``assert`` — at least one - ``assert`` in its own scope, and NO ``raise`` and NO falsy-constant ``return``. + ``assert`` in its own scope, and NO real rejection (``raise`` / + rejection-shaped ``return``). This is PY-WL-111's predicate: such a boundary validates in development but is stripped under ``python -O`` (CWE-617), so the rejection silently vanishes in production. Mutually exclusive with PY-WL-102 (which fires only when - :func:`has_rejection_path` is False, i.e. NO assert either).""" + :func:`has_rejection_path` is False, i.e. NO assert either). Callers that can + see the project context additionally consult :func:`rejecting_helper_calls` — + a raising same-module helper survives ``-O`` and rescues the boundary.""" has_assert = False for stmt in _own_statements(node): - if isinstance(stmt, ast.Raise): - return False - if isinstance(stmt, ast.Return) and _is_falsy_constant_return(stmt.value): + if _stmt_is_real_rejection(stmt): return False if isinstance(stmt, ast.Assert): has_assert = True return has_assert +def is_degenerate_boundary(node: ast.FunctionDef | ast.AsyncFunctionDef) -> bool: + """True for the bare degenerate boundary: the body is (modulo docstrings / + ``pass``) a single ``return ``. PY-WL-119's shape — a strict subset of + "no rejection path", carved out of PY-WL-102's domain so the family partitions + cleanly (119 wins on this shape; 102 owns every other no-rejection shape).""" + param_names = {arg.arg for arg in (*node.args.posonlyargs, *node.args.args, *node.args.kwonlyargs)} + if node.args.vararg: + param_names.add(node.args.vararg.arg) + if node.args.kwarg: + param_names.add(node.args.kwarg.arg) + + non_trivial_stmts = [ + stmt for stmt in node.body if not isinstance(stmt, ast.Pass) and not _is_ellipsis_or_constant(stmt) + ] + if len(non_trivial_stmts) == 1 and isinstance(non_trivial_stmts[0], ast.Return): + ret_val = non_trivial_stmts[0].value + if isinstance(ret_val, ast.Name) and ret_val.id in param_names: + return True + return False + + +def _resolve_one_hop_callee( + call: ast.Call, + entity: Entity, + entities: Mapping[str, Entity], + call_site_callees: Mapping[int, str], +) -> Entity | None: + """Resolve *call* to a SAME-MODULE project entity, or None. + + The engine's resolved target (``call_site_callees``) wins when present; a + resolved CROSS-module callee is deliberately discarded (one-hop stays + same-module — cheap and conservative). Otherwise a lexical fallback maps a + bare ``helper(...)`` or single-attribute ``Cls.method(...)`` callee onto the + boundary's enclosing qualname prefixes, shortest first (module scope is what + a bare name actually resolves to; class scopes are tried later only as a + static/class-method courtesy).""" + resolved = call_site_callees.get(id(call)) + if resolved is not None: + callee = entities.get(resolved) + if callee is not None and callee.location.path == entity.location.path and callee.qualname != entity.qualname: + return callee + return None + func = call.func + if isinstance(func, ast.Name): + suffix = func.id + elif isinstance(func, ast.Attribute) and isinstance(func.value, ast.Name): + suffix = f"{func.value.id}.{func.attr}" + else: + return None + parts = entity.qualname.split(".") + for i in range(1, len(parts)): + candidate = entities.get(".".join((*parts[:i], suffix))) + if ( + candidate is not None + and candidate.location.path == entity.location.path + and candidate.qualname != entity.qualname + ): + return candidate + return None + + +def rejecting_helper_calls( + entity: Entity, + entities: Mapping[str, Entity], + call_site_callees: Mapping[int, str], +) -> frozenset[int]: + """The ``id()``s of own-scope ``Call`` nodes in *entity* that resolve (one hop, + same module) to a callee whose OWN body has a real rejection — a factored-out + validator (``_require_nonempty(p)``), a raising staticmethod helper, or + wholesale delegation to another raising boundary (``return inner(p)``). + + Bounded interprocedural sight for the boundary-integrity family: such a call + IS the boundary's rejection path, so PY-WL-102/111 must stay silent and + PY-WL-113 can locate the rejection inside a ``try``. Strictly ONE hop (the + callee's body is inspected with the own-scope :func:`has_real_rejection`, + never recursively) and strictly same-module (same ``location.path``). + + SOUNDNESS: the callee must have a REAL rejection — a helper that cannot raise + (logs and returns) never counts, and an assert-only helper never counts (its + assert vanishes under ``python -O`` exactly like an inline one, which would + falsely silence PY-WL-111).""" + ids: set[int] = set() + for n in own_nodes(entity.node): + if isinstance(n, ast.Call): + callee = _resolve_one_hop_callee(n, entity, entities, call_site_callees) + if callee is not None and has_real_rejection(callee.node): + ids.add(id(n)) + return frozenset(ids) + + +def block_has_real_rejection(stmts: list[ast.stmt], rejecting_call_ids: frozenset[int] = frozenset()) -> bool: + """True when the statement list *stmts* (a ``try`` body or handler body) + lexically contains a real rejection — a ``raise`` / rejection-shaped + ``return`` in its own scope, or a call whose ``id()`` is in + *rejecting_call_ids* (a one-hop rejecting helper, see + :func:`rejecting_helper_calls`). PY-WL-113's per-``try`` premise: a handler + can only swallow a rejection that lives inside its own ``try``.""" + for top in stmts: + for stmt in (top, *_own_statements(top)): + if _stmt_is_real_rejection(stmt): + return True + if rejecting_call_ids: + for top in stmts: + for n in own_nodes(top): + if isinstance(n, ast.Call) and id(n) in rejecting_call_ids: + return True + return False + + def _contains_reraise(handler: ast.ExceptHandler) -> bool: """True if *handler* re-raises anywhere in its own body (bare ``raise`` / ``raise X`` / ``raise X from e``). Does not descend into nested def/class.""" diff --git a/src/wardline/scanner/rules/_sink_helpers.py b/src/wardline/scanner/rules/_sink_helpers.py index 372c3b9c..7e97071a 100644 --- a/src/wardline/scanner/rules/_sink_helpers.py +++ b/src/wardline/scanner/rules/_sink_helpers.py @@ -1,10 +1,13 @@ # src/wardline/scanner/rules/_sink_helpers.py -"""Shared helpers for the dangerous-sink rules (PY-WL-106/107/108). +"""Shared machinery for the dangerous-sink rules (106/107/108/112/115/116/117/121-126). A "sink rule" fires when raw-zone data reaches a named dangerous call (a -deserialization / dynamic-exec / OS-command sink) inside a trusted-tier function. -These helpers find the sink calls in a function's own scope, canonicalize their -names through the module import alias map, and resolve the taint of their arguments. +deserialization / dynamic-exec / OS-command / SSRF / XML / template / native-load / +log / mail sink) inside a trusted-tier function. These helpers find the sink calls +in a function's own scope, canonicalize their names through the module import alias +map and the name-binding maps, and resolve the taint of their arguments; the +consolidated :class:`TaintedSinkRule` base (review 2026-06-10) is THE single check +loop the whole family runs on. **Argument-taint resolution is FLOW-SENSITIVE.** Each sink call reads the engine's per-call-site resolved argument taints (`function_call_site_arg_taints` — the taint of @@ -19,6 +22,7 @@ from __future__ import annotations import ast +from dataclasses import dataclass, field from typing import TYPE_CHECKING from wardline.core.finding import Finding, Kind, Location, Severity @@ -30,18 +34,30 @@ from collections.abc import Iterator, Mapping from wardline.scanner.context import AnalysisContext + from wardline.scanner.index import Entity from wardline.scanner.rules.metadata import RuleMetadata __all__ = [ "RAW_ZONE", + "ArgSpec", + "SinkBindings", "TaintedSinkRule", + "build_sink_finding", "canonical_call_name", + "collect_ctor_call_nodes", + "collect_sink_bindings", "dotted_name", "enclosing_declared_tier", + "module_alias_map", + "module_for_qualname", + "receiver_ctor_call", + "resolve_bound_call_fqn", "resolved_arg_taints", + "resolved_sink_calls", "sink_calls", "sink_method_calls", "worst_arg_taint", + "worst_dangerous_arg_taint", ] @@ -118,6 +134,26 @@ def _own_calls(node: ast.AST) -> Iterator[ast.Call]: yield from _own_calls(child) +def _direct_sink_fqn( + call: ast.Call, + sink_names: frozenset[str], + aliases: dict[str, str], + module_prefix: str, +) -> str | None: + """Canonical sink FQN of *call* when its func is a direct dotted spelling + (``eval`` / ``pickle.loads`` / an import-aliased form) in *sink_names*, else None.""" + from wardline.scanner.ast_primitives import resolve_call_fqn + + fqn = resolve_call_fqn(call, aliases, frozenset(), module_prefix) + if fqn is not None and fqn in sink_names: + return fqn + dotted = dotted_name(call.func) + if dotted is None: + return None + canonical = canonical_call_name(dotted, aliases) + return canonical if canonical in sink_names else None + + def sink_calls( func_node: ast.AST, sink_names: frozenset[str], @@ -126,20 +162,11 @@ def sink_calls( ) -> Iterator[tuple[ast.Call, str]]: """Yield ``(call, dotted_name)`` for own-scope calls whose func resolves to a canonical name in *sink_names* (matches both ``eval`` and ``pickle.loads`` forms).""" - from wardline.scanner.ast_primitives import resolve_call_fqn - aliases = dict(alias_map or {}) for call in _own_calls(func_node): - fqn = resolve_call_fqn(call, aliases, frozenset(), module_prefix) - if fqn is not None and fqn in sink_names: + fqn = _direct_sink_fqn(call, sink_names, aliases, module_prefix) + if fqn is not None: yield call, fqn - else: - dotted = dotted_name(call.func) - if dotted is None: - continue - canonical = canonical_call_name(dotted, aliases) - if canonical in sink_names: - yield call, canonical def sink_method_calls(func_node: ast.AST, method_names: frozenset[str]) -> Iterator[ast.Call]: @@ -156,7 +183,274 @@ def sink_method_calls(func_node: ast.AST, method_names: frozenset[str]) -> Itera yield call -def _module_for_qualname(qualname: str, context: AnalysisContext) -> str | None: +@dataclass(frozen=True) +class SinkBindings: + """Statically-known name bindings collected from one lexical scope. + + ``instance_classes`` maps a local var to the resolved FQN of the call that + constructed it (``c = httpx.Client()`` → ``{"c": "httpx.Client"}``); a method + call on the var then resolves to ``.``. The constructor FQN + is recorded WITHOUT verifying it names a class — a non-class FQN (``c = g()`` + → ``"m.g"``/``"g"``) yields method names like ``"g.get"`` that match no sink, + so the over-approximation is silent, never a false sink match. + + ``callable_aliases`` maps a local var assigned DIRECTLY from a resolvable + dotted callable (``runner = subprocess.run``) to that callable's FQN; a + bare-name call of the var then participates in sink matching under the FQN. + + A name lives in at most one of the two maps (binding one kind evicts the other). + """ + + instance_classes: Mapping[str, str] = field(default_factory=dict) + callable_aliases: Mapping[str, str] = field(default_factory=dict) + + +@dataclass(frozen=True) +class ArgSpec: + """Which argument slots of a sink are dangerous (taint-relevant). + + ``positions`` are 0-based positional indices; ``keywords`` are keyword names. + A sink with NO spec keeps the historical "worst of ALL args" behavior; a spec + that names no slots (empty tuples) means "no dangerous slots" and never + resolves a taint. Declare BOTH spellings of a slot that can be passed either + way (e.g. ``requests.get`` → ``positions=(0,), keywords=("url",)``). + """ + + positions: tuple[int, ...] = () + keywords: tuple[str, ...] = () + + +def _resolve_dotted(expr: ast.expr, aliases: Mapping[str, str]) -> str | None: + """Canonical dotted FQN of a pure Name/Attribute chain through the alias map, + else None (a dynamic expression — Call/Subscript receiver — never resolves).""" + dotted = dotted_name(expr) + return canonical_call_name(dotted, aliases) if dotted is not None else None + + +def collect_sink_bindings( + node: ast.AST, + alias_map: Mapping[str, str] | None = None, + module_prefix: str = "", # noqa: ARG001 — reserved: local-class constructor FQNs (not in v1) + parent: SinkBindings | None = None, +) -> SinkBindings: + """Collect *node*'s own-scope var→class and var→callable bindings. + + Works on a function OR a module node (nested def/class bodies are their own + scopes and are skipped). *parent* layers an outer scope's bindings underneath + (pass the module's bindings when collecting a function); a function-local + rebind shadows or invalidates the parent entry for that name. + + Binding forms: ``c = pkg.Cls()`` (direct construction), ``with pkg.Cls() as c`` + / ``async with`` (context targets), ``c: pkg.Cls`` (annotation, with or without + a value — the annotation wins over the value), ``def f(c: pkg.Cls)`` (a + function's OWN parameter annotations, seeded before the body so a body rebind + shadows/invalidates them — the most common spelling of an injected + logger/client receiver), ``r = pkg.fn`` (callable alias), and ``d = c`` (copy + of an existing binding). All resolve through the module's import alias map. + + NOT branch-aware (v1): the map is LAST-BINDING-WINS in source order across the + whole scope, so a var rebound to a different class resolves to the NEW class + everywhere — the stale class is never kept. Any rebind whose RHS does not + resolve (a non-dotted call, a constant, a tuple unpack, ``for`` targets, + augmented assignment, ``del``, a match capture) INVALIDATES the name. + """ + aliases = dict(alias_map or {}) + instances: dict[str, str] = dict(parent.instance_classes) if parent else {} + callables: dict[str, str] = dict(parent.callable_aliases) if parent else {} + + def invalidate(name: str) -> None: + instances.pop(name, None) + callables.pop(name, None) + + def invalidate_target(target: ast.expr) -> None: + for sub in ast.walk(target): + if isinstance(sub, ast.Name): + invalidate(sub.id) + + def bind_instance(name: str, fqn: str) -> None: + invalidate(name) + instances[name] = fqn + + def bind_callable(name: str, fqn: str) -> None: + invalidate(name) + callables[name] = fqn + + def bind_value(name: str, value: ast.expr) -> None: + if isinstance(value, ast.NamedExpr): # c = (d := Cls()) — classify the inner value + value = value.value + if isinstance(value, ast.Call): + fqn = _resolve_dotted(value.func, aliases) + bind_instance(name, fqn) if fqn is not None else invalidate(name) + elif isinstance(value, ast.Name) and value.id in instances: + bind_instance(name, instances[value.id]) # d = c — copy the binding + elif isinstance(value, ast.Name) and value.id in callables: + bind_callable(name, callables[value.id]) + elif isinstance(value, (ast.Name, ast.Attribute)): + fqn = _resolve_dotted(value, aliases) + bind_callable(name, fqn) if fqn is not None else invalidate(name) + else: + invalidate(name) + + def visit(stmts: list[ast.stmt]) -> None: + for stmt in stmts: + if isinstance(stmt, (ast.FunctionDef, ast.AsyncFunctionDef, ast.ClassDef)): + continue + # a walrus anywhere in the statement binds in THIS scope + for sub in ast.walk(stmt): + if isinstance(sub, ast.NamedExpr) and isinstance(sub.target, ast.Name): + bind_value(sub.target.id, sub.value) + if isinstance(stmt, ast.Assign): + if len(stmt.targets) == 1 and isinstance(stmt.targets[0], ast.Name): + bind_value(stmt.targets[0].id, stmt.value) + else: + for target in stmt.targets: + invalidate_target(target) + elif isinstance(stmt, ast.AnnAssign): + if isinstance(stmt.target, ast.Name): + ann_fqn = _resolve_dotted(stmt.annotation, aliases) + if ann_fqn is not None: + bind_instance(stmt.target.id, ann_fqn) + elif stmt.value is not None: + bind_value(stmt.target.id, stmt.value) + else: + invalidate(stmt.target.id) + else: + invalidate_target(stmt.target) + elif isinstance(stmt, ast.AugAssign): + invalidate_target(stmt.target) + elif isinstance(stmt, (ast.For, ast.AsyncFor)): + invalidate_target(stmt.target) + visit(stmt.body) + visit(stmt.orelse) + elif isinstance(stmt, (ast.With, ast.AsyncWith)): + for item in stmt.items: + if item.optional_vars is None: + continue + if isinstance(item.optional_vars, ast.Name): + if isinstance(item.context_expr, ast.Call): + fqn = _resolve_dotted(item.context_expr.func, aliases) + if fqn is not None: + bind_instance(item.optional_vars.id, fqn) + else: + invalidate(item.optional_vars.id) + else: + invalidate(item.optional_vars.id) + else: + invalidate_target(item.optional_vars) + visit(stmt.body) + elif isinstance(stmt, (ast.If, ast.While)): + visit(stmt.body) + visit(stmt.orelse) + elif isinstance(stmt, ast.Try): + visit(stmt.body) + for handler in stmt.handlers: + if handler.name is not None: + invalidate(handler.name) + visit(handler.body) + visit(stmt.orelse) + visit(stmt.finalbody) + elif isinstance(stmt, ast.Match): + for case in stmt.cases: + for sub in ast.walk(case.pattern): # capture patterns bind names + captured = getattr(sub, "name", None) or getattr(sub, "rest", None) + if isinstance(captured, str): + invalidate(captured) + visit(case.body) + elif isinstance(stmt, ast.Delete): + for target in stmt.targets: + invalidate_target(target) + + if isinstance(node, (ast.FunctionDef, ast.AsyncFunctionDef)): + # Parameter annotations seed instance bindings BEFORE the body walk, so a + # body rebind shadows/invalidates them like any other prior binding. + # ``*args``/``**kwargs`` are tuples/dicts of the annotated type, never the + # instance itself, so they are not seeded. An unresolvable annotation + # (subscripted generic, string literal) proves nothing and binds nothing. + for arg in (*node.args.posonlyargs, *node.args.args, *node.args.kwonlyargs): + if arg.annotation is not None: + ann_fqn = _resolve_dotted(arg.annotation, aliases) + if ann_fqn is not None: + bind_instance(arg.arg, ann_fqn) + body = getattr(node, "body", None) + if isinstance(body, list): + visit(body) + return SinkBindings(instance_classes=instances, callable_aliases=callables) + + +def resolve_bound_call_fqn( + call: ast.Call, + bindings: SinkBindings, + alias_map: Mapping[str, str] | None = None, + module_prefix: str = "", # noqa: ARG001 — reserved: local-class constructor FQNs (not in v1) +) -> str | None: + """Resolve *call* through name BINDINGS (not import spellings), or None. + + Three forms: + * bare-name callable alias — ``runner(...)`` where ``runner = subprocess.run`` + → ``"subprocess.run"``; + * method on a bound instance — ``c.get(...)`` where ``c``'s class is in + ``bindings.instance_classes`` → ``".get"``; + * chained construction — ``httpx.Client().get(...)`` → ``"httpx.Client.get"`` + (single constructor→method hop only; deeper chains like ``a().b().c()`` do + not resolve — the intermediate return class is unknown). + + Dynamic receivers (subscript / nested attribute paths like ``self.client``) + never resolve in v1. + """ + aliases = dict(alias_map or {}) + func = call.func + if isinstance(func, ast.Name): + return bindings.callable_aliases.get(func.id) + if isinstance(func, ast.Attribute): + receiver = func.value + if isinstance(receiver, ast.Name): + cls_fqn = bindings.instance_classes.get(receiver.id) + return f"{cls_fqn}.{func.attr}" if cls_fqn is not None else None + if isinstance(receiver, ast.Call): + ctor_fqn = _resolve_dotted(receiver.func, aliases) + return f"{ctor_fqn}.{func.attr}" if ctor_fqn is not None else None + return None + + +def resolved_sink_calls( + func_node: ast.AST, + sink_names: frozenset[str], + alias_map: Mapping[str, str] | None = None, + module_prefix: str = "", + *, + module_bindings: SinkBindings | None = None, +) -> Iterator[tuple[ast.Call, str]]: + """:func:`sink_calls` plus binding-aware resolution — a strict superset. + + Yields ``(call, canonical_fqn)`` for own-scope calls matching *sink_names* via + EITHER the direct dotted/import-aliased spelling (the :func:`sink_calls` path) + OR a name binding: construct-then-method (``c = httpx.Client(); c.get(u)`` / + ``with httpx.Client() as c`` / ``c: httpx.Client`` / chained + ``httpx.Client().get(u)``) and callable aliases (``runner = subprocess.run; + runner(...)``). Function-local bindings are collected from *func_node* and + layered over *module_bindings* (pass :func:`collect_sink_bindings` of the + module node to honor module-level assignments). + + Binding resolution is last-binding-wins (see :func:`collect_sink_bindings`), + so the yield for a rebound var reflects its FINAL statically-known class. + """ + aliases = dict(alias_map or {}) + bindings = collect_sink_bindings(func_node, aliases, module_prefix, parent=module_bindings) + for call in _own_calls(func_node): + fqn = _direct_sink_fqn(call, sink_names, aliases, module_prefix) + if fqn is None: + bound = resolve_bound_call_fqn(call, bindings, aliases, module_prefix) + if bound is not None and bound in sink_names: + fqn = bound + if fqn is not None: + yield call, fqn + + +def module_for_qualname(qualname: str, context: AnalysisContext) -> str | None: + """The longest module prefix of *qualname* known to ``context.alias_maps``. + + THE single longest-prefix module resolver for the rule layer (consolidation, + review 2026-06-10 — this used to exist as five per-rule private clones).""" modules = context.alias_maps.keys() for module in sorted(modules, key=len, reverse=True): if qualname == module or qualname.startswith(module + "."): @@ -164,6 +458,12 @@ def _module_for_qualname(qualname: str, context: AnalysisContext) -> str | None: return None +def module_alias_map(qualname: str, context: AnalysisContext) -> Mapping[str, str]: + """Import alias map of *qualname*'s module (longest module-prefix match), or empty.""" + module = module_for_qualname(qualname, context) + return context.alias_maps.get(module, {}) if module is not None else {} + + def resolved_arg_taints(call: ast.Call, qualname: str, context: AnalysisContext) -> dict[int | str | None, TaintState]: """Per-argument resolved taints for *call* — THE single fail-closed argument resolver. @@ -173,22 +473,26 @@ def resolved_arg_taints(call: ast.Call, qualname: str, context: AnalysisContext) (``**kwargs``), and a ``"*{i}"`` marker alongside the int key for a ``Starred`` positional. Empty dict when the call takes no arguments. - Fail-closed: when no L2 snapshot exists for *call* (an L2-skipped function), warns - ``WLN-ENGINE-FLOW-INSENSITIVE-FALLBACK`` and returns a pessimistic map marking every - syntactic argument ``UNKNOWN_RAW``. Each rule then SELECTS over this result on its own - terms (worst / any-provably-untrusted / by-position), so the fail-closed contract lives - in exactly one place and cannot drift between rules. The pessimism is correctly - direction-aware: a RAW_ZONE-gate (sink rules / a SQL-string position) still fires on the + Fail-closed: when no L2 snapshot exists for *call* (an L2-skipped function), the + qualname is recorded into ``context.flow_insensitive_fallbacks`` and a pessimistic + map marking every syntactic argument ``UNKNOWN_RAW`` is returned. The analyzer + surfaces the recorded set as ONE ``WLN-ENGINE-FLOW-INSENSITIVE-FALLBACK`` + NONE/FACT finding per scan — a finding, not a ``UserWarning``, so MCP/library + consumers see the degradation and a warnings-as-error embedder cannot turn the + diagnostic into a rule-aborting raise (review 2026-06-10). Each rule then + SELECTS over this result on its own terms (worst / any-provably-untrusted / + by-position), so the fail-closed contract lives in exactly one place and cannot + drift between rules. The pessimism is correctly direction-aware: a + RAW_ZONE-gate (sink rules / a SQL-string position) still fires on the UNKNOWN_RAW fallback, while a *provably*-untrusted gate (PY-WL-105, which excludes UNKNOWN_RAW) correctly stays silent — preserving its deliberate no-flood design.""" snapshot = context.function_call_site_arg_taints.get(qualname, {}).get(id(call)) if snapshot is not None: return dict(snapshot) - # Flow-sensitive snapshot is missing. Warn and build a pessimistic per-arg map. - import warnings - - warnings.warn(f"WLN-ENGINE-FLOW-INSENSITIVE-FALLBACK: {qualname}", stacklevel=2) + # Flow-sensitive snapshot is missing. Record the degradation (surfaced by the + # analyzer as one FACT finding per scan) and build a pessimistic per-arg map. + context.flow_insensitive_fallbacks.add(qualname) pessimistic: dict[int | str | None, TaintState] = {} for i, arg in enumerate(call.args): pessimistic[i] = TaintState.UNKNOWN_RAW @@ -213,20 +517,242 @@ def worst_arg_taint(call: ast.Call, qualname: str, context: AnalysisContext) -> return worst +def worst_dangerous_arg_taint( + call: ast.Call, + qualname: str, + context: AnalysisContext, + spec: ArgSpec | None = None, +) -> TaintState | None: + """The LEAST-trusted resolvable taint over ONLY the dangerous arg slots of *call*. + + ``spec is None`` → exactly :func:`worst_arg_taint` (worst of ALL args — the + backward-compatible default for sinks with no :class:`ArgSpec`). With a spec, + only the declared positional indices and keyword names are considered, so a + raw value in a non-dangerous slot (``requests.get(url, raw_body)`` with + ``positions=(0,)``) no longer drives the verdict. Returns None when no + dangerous slot resolves (including an empty spec — "no dangerous slots"). + + Fail-closed widening over the splat forms, where syntactic slots stop mapping + to runtime slots: a ``*args`` positional makes EVERY positional slot's taint + eligible when the spec names any position (the star may fill any of them); + a ``**kwargs`` taint is eligible when the spec names any keyword. The + underlying per-slot taints come from :func:`resolved_arg_taints`, so the + missing-snapshot pessimistic ``UNKNOWN_RAW`` fallback applies unchanged. + """ + if spec is None: + return worst_arg_taint(call, qualname, context) + taints = resolved_arg_taints(call, qualname, context) + worst: TaintState | None = None + for key in _eligible_slot_keys(call, taints, spec): + ts = taints[key] + if ts is not None and (worst is None or TRUST_RANK[ts] > TRUST_RANK[worst]): + worst = ts + return worst + + +def _eligible_slot_keys( + call: ast.Call, + taints: Mapping[int | str | None, TaintState], + spec: ArgSpec | None, +) -> set[int | str | None]: + """The slot keys of *taints* eligible for a sink's taint verdict under *spec*. + + ``spec is None`` → every key (the historical worst-of-ALL-args selector). + Otherwise the declared positional indices and keyword names, with the + fail-closed splat widening :func:`worst_dangerous_arg_taint` documents.""" + if spec is None: + return set(taints) + keys: set[int | str | None] = set() + keys.update(p for p in spec.positions if p in taints) + keys.update(kw for kw in spec.keywords if kw in taints) + if spec.positions and any(isinstance(arg, ast.Starred) for arg in call.args): + # positional indices are unreliable past a *args — widen to every positional slot + keys.update(k for k in taints if isinstance(k, int) or (isinstance(k, str) and k.startswith("*"))) + if spec.keywords and None in taints: # **kwargs may supply any keyword + keys.add(None) + return keys + + +def _slot_expr(call: ast.Call, key: int | str | None) -> ast.expr | None: + """The argument AST expression behind one :func:`resolved_arg_taints` slot key, + or None for the slots that cannot be syntactically attributed (``**kwargs`` → + ``None`` key, ``*args`` → ``"*{i}"`` marker).""" + if isinstance(key, int): + if 0 <= key < len(call.args) and not isinstance(call.args[key], ast.Starred): + return call.args[key] + return None + if isinstance(key, str) and not key.startswith("*"): + for kw in call.keywords: + if kw.arg == key: + return kw.value + return None + + +def collect_ctor_call_nodes( + node: ast.AST, + alias_map: Mapping[str, str] | None = None, + ctor_fqns: frozenset[str] | None = None, +) -> dict[str, ast.Call]: + """Own-scope var → LAST constructor :class:`ast.Call` bound to it (source order). + + Companion to :func:`collect_sink_bindings`: that map resolves the var's class + FQN (and is the GATE — it invalidates on every unresolvable rebind), while this + one supplies the constructor call NODE a receiver-anchored taint is read from. + Binding forms that carry a call are recorded (``v = Cls(...)``, ``with Cls(...) + as v``, walrus, ``v: T = Cls(...)``) plus the ``w = v`` copy of an existing + entry; a non-call, non-copy rebind drops the entry. A residual stale entry is + harmless: the bindings gate already refused to resolve the method call. + + *ctor_fqns* optionally restricts recording to constructors whose canonical + dotted FQN (through *alias_map*) is in the set — ``None`` records every call. + """ + aliases = dict(alias_map or {}) + ctors: dict[str, ast.Call] = {} + + def bind(name: str, value: ast.expr) -> None: + if isinstance(value, ast.NamedExpr): + value = value.value + if isinstance(value, ast.Call): + if ctor_fqns is None: + ctors[name] = value + return + fqn = _resolve_dotted(value.func, aliases) + if fqn is not None and fqn in ctor_fqns: + ctors[name] = value + else: + ctors.pop(name, None) + elif isinstance(value, ast.Name) and value.id in ctors: + ctors[name] = ctors[value.id] # w = v — copy the binding + else: + ctors.pop(name, None) + + def walk(parent: ast.AST) -> None: + for child in ast.iter_child_nodes(parent): + if isinstance(child, (ast.FunctionDef, ast.AsyncFunctionDef, ast.ClassDef)): + continue # nested scopes are their own entities + if isinstance(child, ast.Assign) and len(child.targets) == 1 and isinstance(child.targets[0], ast.Name): + bind(child.targets[0].id, child.value) + elif isinstance(child, ast.AnnAssign) and isinstance(child.target, ast.Name): + if child.value is not None: + bind(child.target.id, child.value) + elif isinstance(child, ast.NamedExpr) and isinstance(child.target, ast.Name): + bind(child.target.id, child.value) + elif isinstance(child, (ast.With, ast.AsyncWith)): + for item in child.items: + if isinstance(item.optional_vars, ast.Name): + bind(item.optional_vars.id, item.context_expr) + walk(child) + + walk(node) + return ctors + + +def receiver_ctor_call(call: ast.Call, ctor_nodes: Mapping[str, ast.Call]) -> ast.Call | None: + """The constructor call that built *call*'s receiver: the chained receiver itself + (``Cls(raw).method()``) or the bound var's recorded constructor, else None.""" + if not isinstance(call.func, ast.Attribute): + return None + receiver = call.func.value + if isinstance(receiver, ast.Call): + return receiver + if isinstance(receiver, ast.Name): + return ctor_nodes.get(receiver.id) + return None + + +def build_sink_finding( + *, + rule_id: str, + entity: Entity, + qualname: str, + call: ast.Call, + dotted: str, + severity: Severity, + tier: TaintState, + worst: TaintState, + sink_label: str, + message: str | None = None, +) -> Finding: + """THE shared sink-finding constructor — one message shape, one wlfp2 + discriminator, one properties dict for every call-site-anchored sink rule. + + Call-site-anchored: >1 finding per (rule, path, qualname) is possible (several + sinks in one function). Discriminate by SOURCE only — an ENTITY-RELATIVE line + offset (call line - the enclosing def's line, invariant to a comment ABOVE the + function: wlfp2/wardline-8654423823) plus the call's full lexical SPAN and the + sink dotted-name. The span (start:end), not the start column alone, separates + the outer/inner calls of a chain (``a.sink(x).sink(y)``), which share a start + column. Never the resolved arg taint (it drifts across builds: weft-4a9d0f863c). + + *message* overrides the standard message text only — fingerprint and + properties stay uniform (PathTraversal's receiver-anchored explanations). + """ + line = call.lineno + return Finding( + rule_id=rule_id, + message=message + or (f"{qualname}: {worst.value} (untrusted) data reaches the {sink_label} sink {dotted}() at line {line}"), + severity=severity, + kind=Kind.DEFECT, + location=Location(path=entity.location.path, line_start=line), + fingerprint=_fp( + rule_id=rule_id, + path=entity.location.path, + qualname=qualname, + taint_path=f"{line - (entity.location.line_start or 0)}:{call.col_offset}:{call.end_col_offset}:{dotted}", + ), + # OLD (wlfp1) taint_path, byte-exact, for `wardline rekey` (P4). + taint_path_v0=f"{dotted}@{call.col_offset}:{call.end_col_offset}", + qualname=qualname, + properties={"tier": tier.value, "sink": dotted, "arg_taint": worst.value}, + ) + + class TaintedSinkRule: - """Base for the dangerous-sink rules (106/107/108): raw-zone data reaching a named - sink inside a trusted-tier function. Tier-MODULATED — silent in the developer- - freedom zone (undecorated → UNKNOWN_RAW → modulate → NONE), speaking only where - trust is declared (the same opt-in/fail-closed discipline as 103/104). Subclasses - set ``rule_id``, ``metadata``, ``SINKS`` (the curated dotted sink names), and - ``sink_label`` (for the message). Plain (not ClassVar) annotations so the instances - satisfy the ``_Rule`` protocol (a settable instance ``rule_id``).""" + """THE base for the call-site-anchored dangerous-sink rules — raw-zone data + reaching a named sink inside a trusted-tier function. Tier-MODULATED — silent in + the developer-freedom zone (undecorated → UNKNOWN_RAW → modulate → NONE), + speaking only where trust is declared (the same opt-in/fail-closed discipline + as 103/104). + + Consolidated 2026-06-10 (review): this single ``check`` loop replaces the two + former mixins (108/112's ``BindingAwareSinkCheckMixin``, the 121–126 family's + ``SpecSinkCheckMixin``) and the SSRF/deserialization rule-local overrides. The + loop is binding-aware (:func:`resolved_sink_calls`, layered over + ``context.module_bindings`` — a strict superset of the old direct-spelling + match, so the historical base rules gain construct-then-method and + callable-alias resolution as NEW findings with zero fingerprint drift), and is + parameterized by: + + * ``SINK_SPECS`` — per-sink :class:`ArgSpec` slot precision (missing/``None`` + entry → the historical worst-of-all-args selector); + * ``SINK_SEVERITIES`` — per-sink base severity, applied BEFORE tier + modulation (PY-WL-121's lxml-vs-stdlib split). An operator + ``rules.severity`` override re-bases the WHOLE rule: the registry passes + ``base_severity`` only when the config carries an override, recorded as + ``self.severity_overridden`` — an explicit flag, never inferred by value + identity, so an override EQUAL to the metadata default still wins; + * hook :meth:`_accept_call` — extra per-call gate after the sink-name match + (112's literal ``shell=True``, 106's numpy/torch literal-keyword gates); + * hook :meth:`_arg_guarded` — per-slot syntactic neutralization (108's + ``shlex.quote`` concatenation guard); + * hook :meth:`_taint_anchor_call` — redirects the taint read to another call + (106's ``pickle.Unpickler(stream).load()`` reads the CONSTRUCTOR's stream + argument); ``None`` skips the call. + + Subclasses set ``rule_id``, ``metadata``, ``SINKS`` (the curated dotted sink + names), and ``sink_label`` (for the message). Plain (not ClassVar) annotations + so the instances satisfy the ``_Rule`` protocol (a settable instance + ``rule_id``).""" rule_id: str metadata: RuleMetadata SINKS: frozenset[str] sink_label: str + SINK_SPECS: Mapping[str, ArgSpec | None] = {} + SINK_SEVERITIES: Mapping[str, Severity] = {} + def __init_subclass__(cls, **kwargs: object) -> None: super().__init_subclass__(**kwargs) missing = [a for a in ("rule_id", "metadata", "SINKS", "sink_label") if not hasattr(cls, a)] @@ -234,58 +760,101 @@ def __init_subclass__(cls, **kwargs: object) -> None: raise TypeError(f"{cls.__name__} must define class attribute(s): {', '.join(missing)}") def __init__(self, base_severity: Severity | None = None) -> None: + # The registry passes base_severity ONLY for an operator rules.severity + # override (build_default_registry), so presence IS the override signal — + # value identity against the metadata default would silently ignore an + # explicit override equal to that default (review 2026-06-10). + self.severity_overridden = base_severity is not None self.base_severity = base_severity or self.metadata.base_severity - def _accept_call(self, call: ast.Call) -> bool: # noqa: PLR6301 + def _accept_call(self, call: ast.Call, fqn: str) -> bool: # noqa: ARG002, PLR6301 """Extra per-call gate after the SINK-name match. Default: accept.""" return True + def _arg_guarded(self, expr: ast.expr, fqn: str, alias_map: Mapping[str, str]) -> bool: # noqa: ARG002, PLR6301 + """Whether one argument slot of sink *fqn* is syntactically neutralized + (excluded from the worst-taint selection). Default: never.""" + return False + + def _taint_anchor_call( # noqa: PLR6301 + self, + call: ast.Call, + fqn: str, # noqa: ARG002 + entity_node: ast.AST, # noqa: ARG002 + alias_map: Mapping[str, str], # noqa: ARG002 + ) -> ast.Call | None: + """The call whose arguments carry the sink's dangerous data — *call* itself + by default. An override may redirect (a reader-method sink reads its + CONSTRUCTOR's arguments) or return ``None`` to skip (no resolvable anchor).""" + return call + + def _worst_sink_arg_taint( + self, + call: ast.Call, + fqn: str, + qualname: str, + context: AnalysisContext, + alias_map: Mapping[str, str], + ) -> TaintState | None: + """The LEAST-trusted taint over the sink's dangerous, unguarded arg slots. + + Slot selection honors the sink's :class:`ArgSpec` from ``SINK_SPECS`` + (missing/``None`` → every slot, the historical worst-of-all-args), then + :meth:`_arg_guarded` excludes syntactically neutralized slots. Slots with + no attributable expression (``*args`` / ``**kwargs``) are never guarded — + the fail-closed default.""" + taints = resolved_arg_taints(call, qualname, context) + worst: TaintState | None = None + for key in _eligible_slot_keys(call, taints, self.SINK_SPECS.get(fqn)): + ts = taints[key] + if ts is None: + continue + expr = _slot_expr(call, key) + if expr is not None and self._arg_guarded(expr, fqn, alias_map): + continue + if worst is None or TRUST_RANK[ts] > TRUST_RANK[worst]: + worst = ts + return worst + def check(self, context: AnalysisContext) -> list[Finding]: findings: list[Finding] = [] for qualname, entity in context.entities.items(): # Honor a nested def's OWN trust decorator, else inherit the nearest declared # enclosing scope's tier (wardline-bb8396f96e / wardline-9b88ec5419). tier = enclosing_declared_tier(qualname, context.project_taints, context.declared_qualnames) - severity = modulate(self.base_severity, tier) - if severity == Severity.NONE: - continue # freedom / fail-closed zone — suppressed - module = _module_for_qualname(qualname, context) + if modulate(self.base_severity, tier) == Severity.NONE: + continue # freedom / fail-closed zone — NONE for every base, so skip the entity + module = module_for_qualname(qualname, context) alias_map = context.alias_maps.get(module, {}) if module is not None else {} - for call, dotted in sink_calls(entity.node, self.SINKS, alias_map, module or ""): - if not self._accept_call(call): + module_bindings = context.module_bindings.get(module or "") + for call, dotted in resolved_sink_calls( + entity.node, self.SINKS, alias_map, module or "", module_bindings=module_bindings + ): + if not self._accept_call(call, dotted): continue - worst = worst_arg_taint(call, qualname, context) + base = ( + self.base_severity + if self.severity_overridden + else self.SINK_SEVERITIES.get(dotted, self.base_severity) + ) + severity = modulate(base, tier) + anchor = self._taint_anchor_call(call, dotted, entity.node, alias_map) + if anchor is None: + continue # no resolvable taint anchor — bounded FN, never a guess + worst = self._worst_sink_arg_taint(anchor, dotted, qualname, context, alias_map) if worst is None or worst not in RAW_ZONE: continue - line = call.lineno findings.append( - Finding( + build_sink_finding( rule_id=self.rule_id, - message=( - f"{qualname}: {worst.value} (untrusted) data reaches the {self.sink_label} " - f"sink {dotted}() at line {line}" - ), - severity=severity, - kind=Kind.DEFECT, - location=Location(path=entity.location.path, line_start=line), - fingerprint=_fp( - rule_id=self.rule_id, - path=entity.location.path, - qualname=qualname, - # Call-site-anchored: >1 finding per (rule, path, qualname) is possible - # (several sinks in one function). Discriminate by SOURCE only — an - # ENTITY-RELATIVE line offset (call line - the enclosing def's line, invariant - # to a comment ABOVE the function: wlfp2/wardline-8654423823) plus the call's - # full lexical SPAN and the sink dotted-name. The span (start:end), not the - # start column alone, separates the outer/inner calls of a chain - # (``a.sink(x).sink(y)``), which share a start column. Never the resolved arg - # taint (it drifts across builds: weft-4a9d0f863c). - taint_path=f"{line - (entity.location.line_start or 0)}:{call.col_offset}:{call.end_col_offset}:{dotted}", # noqa: E501 - ), - # OLD (wlfp1) taint_path, byte-exact, for `wardline rekey` (P4). - taint_path_v0=f"{dotted}@{call.col_offset}:{call.end_col_offset}", + entity=entity, qualname=qualname, - properties={"tier": tier.value, "sink": dotted, "arg_taint": worst.value}, + call=call, + dotted=dotted, + severity=severity, + tier=tier, + worst=worst, + sink_label=self.sink_label, ) ) return findings diff --git a/src/wardline/scanner/rules/assert_only_boundary.py b/src/wardline/scanner/rules/assert_only_boundary.py index 431e594a..2423ca18 100644 --- a/src/wardline/scanner/rules/assert_only_boundary.py +++ b/src/wardline/scanner/rules/assert_only_boundary.py @@ -10,9 +10,20 @@ A PY-WL-102-adjacent refinement: 102 fires when a boundary cannot reject *at all*; 111 fires when it *appears* to reject but only via a guard that disappears in -production. The two partition the space — a boundary with a real ``raise`` or a -falsy-constant ``return`` trips neither (see ``asserts_are_sole_rejection`` / -``has_rejection_path`` in ``_ast_helpers``). +production. A boundary with a real ``raise`` / rejection-shaped ``return`` +(see ``asserts_are_sole_rejection`` / ``has_rejection_path`` in ``_ast_helpers``) +— or a ONE-HOP same-module call to a helper that itself raises (the helper's +``raise`` survives ``python -O``, so the CWE-617 claim would be factually false) +— trips neither. + +**The boundary-integrity family partitions FOUR ways** (wardline-718048a518) — +at most one of {102, 111, 113, 119} fires per boundary: + - PY-WL-119 — the bare degenerate shape (single ``return ``); + - PY-WL-102 — every other shape with no rejection path; + - PY-WL-111 — rejection only via ``assert`` (this rule). This includes an + assert INSIDE a ``try`` whose handler substitutes: the rejection is still + assert-only, so 111 wins over 113 (documented precedence); + - PY-WL-113 — a real rejection exists but a fail-open handler defeats it. """ from __future__ import annotations @@ -22,7 +33,7 @@ from wardline.core.finding import Finding, Kind, Severity from wardline.core.finding import compute_finding_fingerprint as _fp from wardline.core.taints import TRUST_RANK -from wardline.scanner.rules._ast_helpers import asserts_are_sole_rejection +from wardline.scanner.rules._ast_helpers import asserts_are_sole_rejection, rejecting_helper_calls from wardline.scanner.rules.metadata import RuleMetadata if TYPE_CHECKING: @@ -66,6 +77,10 @@ def check(self, context: AnalysisContext) -> list[Finding]: continue if not asserts_are_sole_rejection(entity.node): continue + # One-hop: a same-module raising helper survives `python -O`, so the + # assert is NOT the sole rejection and the CWE-617 claim would be false. + if rejecting_helper_calls(entity, context.entities, context.call_site_callees): + continue findings.append( Finding( rule_id=self.rule_id, diff --git a/src/wardline/scanner/rules/boundary_without_rejection.py b/src/wardline/scanner/rules/boundary_without_rejection.py index 0135fdb5..fc98e269 100644 --- a/src/wardline/scanner/rules/boundary_without_rejection.py +++ b/src/wardline/scanner/rules/boundary_without_rejection.py @@ -3,9 +3,28 @@ A trust-RAISING transition (declared return strictly MORE trusted than body — the taint shape unique to ``@trust_boundary`` among the vocabulary) that contains -no ``raise`` and no falsy-constant ``return`` cannot actually reject bad input, -so it is not validating. Declaration-gated (the decorator is the opt-in), so it -emits at base severity (NOT tier-modulated). +no rejection path of any recognised shape cannot actually reject bad input, so it +is not validating. Declaration-gated (the decorator is the opt-in), so it emits +at base severity (NOT tier-modulated). + +Recognised rejection shapes (any one keeps the rule silent): + - an own-scope ``raise`` or ``assert`` (the assert-only case is PY-WL-111's); + - a rejection-shaped ``return`` — falsy constant, conditional expression with a + rejecting branch (``return m.group(0) if m else None``), or a curated + raising-conversion / lookup (``return int(p)`` / ``return Color[p]`` / + ``return ALLOWED[p]`` — validate-by-construction); + - a ONE-HOP, SAME-MODULE call to a helper whose own body has a real rejection + (``_require_nonempty(p)``, a raising staticmethod, or wholesale delegation to + another raising boundary). A helper that cannot raise never counts. + +**The boundary-integrity family partitions FOUR ways** (wardline-718048a518) — +at most one of {102, 111, 113, 119} fires per boundary: + - PY-WL-119 — the bare degenerate shape (single ``return ``): the + more-specific rule wins, so 102 SUPPRESSES itself there (the suppression is + structural — keyed on the shape, not on whether 119 is enabled); + - PY-WL-102 — every OTHER shape with no rejection path (cannot reject at all); + - PY-WL-111 — rejection only via ``assert`` (stripped under ``python -O``); + - PY-WL-113 — a real rejection exists but a fail-open handler defeats it. """ from __future__ import annotations @@ -15,7 +34,11 @@ from wardline.core.finding import Finding, Kind, Severity from wardline.core.finding import compute_finding_fingerprint as _fp from wardline.core.taints import TRUST_RANK -from wardline.scanner.rules._ast_helpers import has_rejection_path +from wardline.scanner.rules._ast_helpers import ( + has_rejection_path, + is_degenerate_boundary, + rejecting_helper_calls, +) from wardline.scanner.rules.metadata import RuleMetadata if TYPE_CHECKING: @@ -30,7 +53,8 @@ "has no rejection path — no raise, no falsy-constant return — so it cannot " "validate." ), - examples_violation=("@trust_boundary(to_level='ASSURED')\ndef v(p):\n return p",), + # NOT the bare `return p` shape — that is PY-WL-119's (the family partitions). + examples_violation=("@trust_boundary(to_level='ASSURED')\ndef v(p):\n x = p\n return x",), examples_clean=( "@trust_boundary(to_level='ASSURED')\ndef v(p):\n if not p:\n raise ValueError\n return p", ), @@ -59,6 +83,12 @@ def check(self, context: AnalysisContext) -> list[Finding]: continue if has_rejection_path(entity.node): continue + # The bare degenerate shape is PY-WL-119's domain (more-specific wins). + if is_degenerate_boundary(entity.node): + continue + # One-hop: a same-module raising helper IS this boundary's rejection path. + if rejecting_helper_calls(entity, context.entities, context.call_site_callees): + continue findings.append( Finding( rule_id=self.rule_id, diff --git a/src/wardline/scanner/rules/broad_exception.py b/src/wardline/scanner/rules/broad_exception.py index 11bada7d..a2b8255d 100644 --- a/src/wardline/scanner/rules/broad_exception.py +++ b/src/wardline/scanner/rules/broad_exception.py @@ -13,8 +13,8 @@ from wardline.core.finding import Finding, Kind, Location, Severity from wardline.core.finding import compute_finding_fingerprint as _fp -from wardline.core.taints import TaintState from wardline.scanner.rules._ast_helpers import is_broad_except, own_except_handlers +from wardline.scanner.rules._sink_helpers import enclosing_declared_tier from wardline.scanner.rules.metadata import RuleMetadata from wardline.scanner.rules.severity_model import modulate @@ -42,8 +42,11 @@ def __init__(self, base_severity: Severity | None = None) -> None: def check(self, context: AnalysisContext) -> list[Finding]: findings: list[Finding] = [] for qualname, entity in context.entities.items(): - lookup_name = qualname.split("..")[0] - tier = context.project_taints.get(lookup_name, TaintState.UNKNOWN_RAW) + # Nearest DECLARED enclosing scope governs a nested def (a nested def's own + # trust decorator wins; undeclared nested defs inherit) — the same + # enclosing_declared_tier semantics as the sink rule family, NOT the + # outermost-function strip (wardline-bb8396f96e / wardline-9b88ec5419). + tier = enclosing_declared_tier(qualname, context.project_taints, context.declared_qualnames) severity = modulate(self.base_severity, tier) if severity == Severity.NONE: continue # suppressed outside trusted/partial tiers diff --git a/src/wardline/scanner/rules/degenerate_boundary.py b/src/wardline/scanner/rules/degenerate_boundary.py index 768efd77..3f72b86e 100644 --- a/src/wardline/scanner/rules/degenerate_boundary.py +++ b/src/wardline/scanner/rules/degenerate_boundary.py @@ -4,44 +4,29 @@ A trust boundary (raises declared trust on return) that simply returns its input parameter directly without any conditional checks, assertions, or validations is a degenerate boundary. It does not perform any validation. + +**The boundary-integrity family partitions FOUR ways** (wardline-718048a518) — +at most one of {102, 111, 113, 119} fires per boundary. The degenerate shape is a +strict structural subset of PY-WL-102's "no rejection path", so 119 WINS on it +(more-specific rule) and 102 suppresses itself there — the same boundary is never +counted twice at ERROR in the gate population. The shared shape predicate is +``_ast_helpers.is_degenerate_boundary``. """ from __future__ import annotations -import ast from typing import TYPE_CHECKING from wardline.core.finding import Finding, Kind, Maturity, Severity from wardline.core.finding import compute_finding_fingerprint as _fp from wardline.core.taints import TRUST_RANK +from wardline.scanner.rules._ast_helpers import is_degenerate_boundary from wardline.scanner.rules.metadata import RuleMetadata if TYPE_CHECKING: from wardline.scanner.context import AnalysisContext -def _is_degenerate(node: ast.FunctionDef | ast.AsyncFunctionDef) -> bool: - param_names = {arg.arg for arg in (*node.args.posonlyargs, *node.args.args, *node.args.kwonlyargs)} - if node.args.vararg: - param_names.add(node.args.vararg.arg) - if node.args.kwarg: - param_names.add(node.args.kwarg.arg) - - non_trivial_stmts = [] - for stmt in node.body: - if isinstance(stmt, ast.Pass): - continue - if isinstance(stmt, ast.Expr) and isinstance(stmt.value, ast.Constant): - continue - non_trivial_stmts.append(stmt) - - if len(non_trivial_stmts) == 1 and isinstance(non_trivial_stmts[0], ast.Return): - ret_val = non_trivial_stmts[0].value - if isinstance(ret_val, ast.Name) and ret_val.id in param_names: - return True - return False - - METADATA = RuleMetadata( rule_id="PY-WL-119", base_severity=Severity.ERROR, @@ -75,7 +60,7 @@ def check(self, context: AnalysisContext) -> list[Finding]: # Trust-raising transition (== @trust_boundary): body less-trusted than return. if TRUST_RANK[body] <= TRUST_RANK[ret]: continue - if not _is_degenerate(entity.node): + if not is_degenerate_boundary(entity.node): continue findings.append( Finding( diff --git a/src/wardline/scanner/rules/failopen_boundary.py b/src/wardline/scanner/rules/failopen_boundary.py index 438fde43..dd28954f 100644 --- a/src/wardline/scanner/rules/failopen_boundary.py +++ b/src/wardline/scanner/rules/failopen_boundary.py @@ -25,10 +25,27 @@ def v(p): This is why the rule does NOT gate on a broad ``except`` — a *narrow* handler that names the rejection's own exception type is the worst case, not the mildest. -**The boundary-integrity trio partitions cleanly:** - - PY-WL-102 — boundary with NO rejection path (cannot reject at all); - - PY-WL-111 — rejection only via ``assert`` (stripped under ``python -O``); - - PY-WL-113 — rejection exists but is defeated by a fail-open handler. +**The boundary-integrity family partitions FOUR ways** (wardline-718048a518) — +at most one of {102, 111, 113, 119} fires per boundary: + - PY-WL-119 — the bare degenerate shape (single ``return ``); + - PY-WL-102 — every other shape with NO rejection path (cannot reject at all); + - PY-WL-111 — rejection only via ``assert`` (stripped under ``python -O``). + This precedence is deliberate: an assert inside a ``try`` caught by a + substituting handler is still 111's (the rejection is assert-only); + - PY-WL-113 — a REAL rejection exists but a fail-open handler defeats it. + +This rule enforces its premise in code, not just prose: + 1. a real (production-surviving) rejection must EXIST — an own-scope ``raise`` + / rejection-shaped ``return``, or a one-hop same-module raising helper call + (``rejecting_helper_calls``). No rejection → 102's domain; assert-only → + 111's domain; + 2. the rejection must be SWALLOWABLE by the matching handler — lexically inside + that handler's own ``try`` body (the self-catch), or inside the handler + itself (a handler that conditionally rejects but substitutes on the other + path). A rejection wholly OUTSIDE the ``try`` (validate-then-cache shapes) + cannot be defeated by the handler, so the boundary fails CLOSED and the rule + stays silent. + A handler that re-raises (even conditionally) is fail-closed and never matches (conservative, mirroring ``has_rejection_path``); a handler that returns a falsy/empty constant is signalling REJECTION, not substituting, and also never @@ -37,22 +54,26 @@ def v(p): Declaration-gated (the ``@trust_boundary`` decorator is the opt-in), so it emits at base severity — NOT tier-modulated — exactly like 102 and 111. -**Residual FP:** a declared boundary that legitimately catches an *unrelated* -exception and returns a default for a non-validation reason. Rare inside a thing -explicitly declared as a validator, and swallow-and-substitute in a validator is -itself the smell; tracked, not hidden. +**Residual FP:** a declared boundary whose ``try`` body holds its rejection AND an +unrelated exception source, with a narrow handler for only the unrelated type. +The rule does not match exception TYPES (the self-catch worst case is a *narrow* +handler), so it cannot tell that pair apart; tracked, not hidden. """ from __future__ import annotations +import ast from typing import TYPE_CHECKING from wardline.core.finding import Finding, Kind, Severity from wardline.core.finding import compute_finding_fingerprint as _fp from wardline.core.taints import TRUST_RANK from wardline.scanner.rules._ast_helpers import ( + _own_statements, + block_has_real_rejection, handler_substitutes_on_failure, - own_except_handlers, + has_real_rejection, + rejecting_helper_calls, returned_var_names, ) from wardline.scanner.rules.metadata import RuleMetadata @@ -102,8 +123,24 @@ def check(self, context: AnalysisContext) -> list[Finding]: # Trust-raising transition (== @trust_boundary): body less-trusted than return. if TRUST_RANK[body] <= TRUST_RANK[ret]: continue + # Premise 1 (the family partition): a REAL rejection must exist. None at + # all -> PY-WL-102's domain; assert-only -> PY-WL-111's domain. + rejecting_calls = rejecting_helper_calls(entity, context.entities, context.call_site_callees) + if not (has_real_rejection(entity.node) or rejecting_calls): + continue + # Premise 2: the rejection must be SWALLOWABLE by the substituting + # handler — inside its own try body, or inside the handler itself. returned = returned_var_names(entity.node) - if not any(handler_substitutes_on_failure(h, returned) for h in own_except_handlers(entity.node)): + if not any( + handler_substitutes_on_failure(handler, returned) + and ( + block_has_real_rejection(try_stmt.body, rejecting_calls) + or block_has_real_rejection(handler.body, rejecting_calls) + ) + for try_stmt in _own_statements(entity.node) + if isinstance(try_stmt, (ast.Try, ast.TryStar)) + for handler in try_stmt.handlers + ): continue findings.append( Finding( diff --git a/src/wardline/scanner/rules/invalid_decorator_level.py b/src/wardline/scanner/rules/invalid_decorator_level.py index d4218b6a..526fca71 100644 --- a/src/wardline/scanner/rules/invalid_decorator_level.py +++ b/src/wardline/scanner/rules/invalid_decorator_level.py @@ -17,6 +17,7 @@ from wardline.core.taints import TaintState from wardline.scanner.grammar import BUILTIN_BOUNDARY_TYPES from wardline.scanner.rules.metadata import RuleMetadata +from wardline.scanner.taint.decorator_provider import _is_builtin_decorator_fqn, _shadowed_builtin_roots if TYPE_CHECKING: from collections.abc import Mapping @@ -26,11 +27,10 @@ _BOUNDARY_LEVELS = frozenset({TaintState.GUARDED, TaintState.ASSURED}) _TRUSTED_LEVELS = frozenset({TaintState.INTEGRAL, TaintState.ASSURED}) -# The module prefixes a BUILTIN trust decorator may be imported from (``wardline.decorators`` -# and the renamed ``weft_markers`` shim), derived from the grammar so this rule cannot drift -# from the vocabulary the engine actually honours. PY-WL-114 polices the LEVEL-bearing builtin -# markers only — ``trusted`` (``level=``) and ``trust_boundary`` (``to_level=``). -_MARKER_MODULE_PREFIXES: frozenset[str] = frozenset(bt.module_prefix for bt in BUILTIN_BOUNDARY_TYPES) +# PY-WL-114 polices the LEVEL-bearing builtin markers only — ``trusted`` (``level=``) +# and ``trust_boundary`` (``to_level=``). Recognition uses the engine's OWN seeding +# predicate (``_is_builtin_decorator_fqn`` + shadowed-root fail-closed rejection), so +# the rule cannot recognise a marker the seeding rejects (wardline-09c09f14df). _LEVEL_MARKER_NAMES: frozenset[str] = frozenset({"trusted", "trust_boundary"}) METADATA = RuleMetadata( @@ -94,18 +94,29 @@ def _resolve_decorator_fqn(deco: ast.expr, alias_map: Mapping[str, str]) -> str return f"{head_fqn}.{rest}" if rest else head_fqn -def _builtin_level_marker(deco: ast.expr, alias_map: Mapping[str, str]) -> str | None: +def _builtin_level_marker(deco: ast.expr, alias_map: Mapping[str, str], shadowed_roots: frozenset[str]) -> str | None: """The canonical builtin marker name (``trusted`` / ``trust_boundary``) iff *deco* - resolves to a BUILTIN level-bearing trust decorator. Gating on the resolved FQN (not the - trailing identifier) fixes both the alias-blind FN — ``@t(level=...)`` where ``t`` aliases - the builtin — and the foreign-name FP — a non-wardline / locally-defined decorator that - merely happens to be spelled ``trusted`` (wardline-0267c31cd8).""" + resolves to a builtin level-bearing trust decorator THE ENGINE'S SEEDING WOULD HONOUR. + Gating on the resolved FQN (not the trailing identifier) fixes both the alias-blind FN — + ``@t(level=...)`` where ``t`` aliases the builtin — and the foreign-name FP — a + non-wardline / locally-defined decorator that merely happens to be spelled ``trusted`` + (wardline-0267c31cd8). Matching uses the seeding predicate itself: only the exact + exports ``P.`` / ``P.trust.`` count (an arbitrarily-nested path like + ``wardline.decorators.evil.trusted`` is seeded by NEITHER, so a bad level on it never + disabled any gate), and a marker whose root the scanned project shadows is rejected + fail-closed exactly as the provider rejects it. PY-WL-110 sidesteps shadows via its + anchored-provenance gate; this rule fires precisely where seeding FAILED, so it must + thread the shadow set explicitly.""" fqn = _resolve_decorator_fqn(deco, alias_map) if fqn is None: return None - last = fqn.rsplit(".", 1)[-1] - if last in _LEVEL_MARKER_NAMES and any(fqn.startswith(prefix + ".") for prefix in _MARKER_MODULE_PREFIXES): - return last + for bt in BUILTIN_BOUNDARY_TYPES: + if not bt.builtin or bt.canonical_name not in _LEVEL_MARKER_NAMES: + continue + if bt.module_prefix.split(".")[0] in shadowed_roots: + continue + if _is_builtin_decorator_fqn(fqn, bt.canonical_name, bt.module_prefix): + return bt.canonical_name return None @@ -119,6 +130,10 @@ def __init__(self, base_severity: Severity | None = None) -> None: def check(self, context: AnalysisContext) -> list[Finding]: findings: list[Finding] = [] modules = list(context.alias_maps.keys()) + # The scanned project's modules are the alias-map keys; the same shadow + # computation the provider runs (a project-local top-level ``wardline`` / + # ``weft_markers`` rejects every builtin marker under that root). + shadowed_roots = _shadowed_builtin_roots(frozenset(modules)) for qualname, entity in context.entities.items(): # The alias map for the module that owns this entity — needed to resolve an # aliased builtin decorator to its FQN (mirrors PY-WL-110). @@ -128,7 +143,7 @@ def check(self, context: AnalysisContext) -> list[Finding]: ) alias_map = (context.alias_maps.get(mod_name) if mod_name is not None else None) or {} for deco_ordinal, deco in enumerate(entity.node.decorator_list): - name = _builtin_level_marker(deco, alias_map) + name = _builtin_level_marker(deco, alias_map, shadowed_roots) if name is None: continue diff --git a/src/wardline/scanner/rules/none_leak.py b/src/wardline/scanner/rules/none_leak.py index 20a54dac..26961e38 100644 --- a/src/wardline/scanner/rules/none_leak.py +++ b/src/wardline/scanner/rules/none_leak.py @@ -28,6 +28,7 @@ from wardline.core.finding import compute_finding_fingerprint as _fp from wardline.core.taints import RAW_ZONE, TRUST_RANK from wardline.scanner.rules._ast_helpers import _own_statements +from wardline.scanner.rules._sink_helpers import module_for_qualname from wardline.scanner.rules.metadata import RuleMetadata if TYPE_CHECKING: @@ -50,14 +51,6 @@ ) -def _module_for_qualname(qualname: str, context: AnalysisContext) -> str | None: - modules = context.alias_maps.keys() - for module in sorted(modules, key=len, reverse=True): - if qualname == module or qualname.startswith(module + "."): - return module - return None - - def _is_none_return(stmt: ast.Return) -> bool: """A bare ``return`` (value is None) or an explicit ``return None``.""" return stmt.value is None or (isinstance(stmt.value, ast.Constant) and stmt.value.value is None) @@ -223,7 +216,7 @@ def check(self, context: AnalysisContext) -> list[Finding]: continue # trust-raising shape -> PY-WL-102's territory, not 109's if _is_generator(entity.node): continue - module = _module_for_qualname(qualname, context) + module = module_for_qualname(qualname, context) alias_map = context.alias_maps.get(module, {}) if module is not None else {} if not _promises_non_none(entity.node, alias_map): continue # no explicit non-None contract -> not a provable leak (FP guard) diff --git a/src/wardline/scanner/rules/path_traversal.py b/src/wardline/scanner/rules/path_traversal.py index 58399617..71496790 100644 --- a/src/wardline/scanner/rules/path_traversal.py +++ b/src/wardline/scanner/rules/path_traversal.py @@ -1,16 +1,60 @@ # src/wardline/scanner/rules/path_traversal.py """PY-WL-116 — untrusted data reaches a path/filesystem-traversal sink. -Passing untrusted data to filesystem APIs (``open``, ``os.open``, ``pathlib.Path``, -or helper functions like ``os.path.join``) can lead to path traversal (CWE-22). +Passing untrusted data to filesystem APIs can lead to path traversal (CWE-22). +Three sink families (wardline-04b65cf0be + the Zip Slip eval item): + +* **Direct dotted calls** with a tainted path argument — ``open``/``os.open``/ + ``os.path.join``/``pathlib.Path`` plus the filesystem-MUTATION APIs + (``os.remove``/``os.rename``/``shutil.rmtree``/``shutil.copy``/...), where a + tainted path is a destructive traversal (delete/move/copy outside the + intended directory). All argument slots are path-like, so the inherited + worst-of-all-args taint is the correct selector (a tainted destination is + traversal too) — no :class:`ArgSpec` needed. +* **Path METHODS** (``read_text``/``write_bytes``/``open``/``unlink``/...) on a + ``pathlib.Path`` CONSTRUCTED from tainted input, resolved through the shared + sink-binding machinery (``p = Path(raw); p.read_text()`` and the chained + ``pathlib.Path(raw).read_text()``). The dangerous data is the CONSTRUCTOR's + argument, so the taint is read from the constructor call, not the + (typically argument-less) method call. +* **Archive extraction** (Zip Slip / tarbomb): ``extractall``/``extract`` on a + ``tarfile.open``/``tarfile.TarFile``/``zipfile.ZipFile`` instance whose + ARCHIVE SOURCE is tainted — a malicious archive escapes the target directory + via ``../`` member names. Exemption: an extraction call passing tarfile's + safe filter as the literal ``filter="data"`` (blocks absolute paths, + traversal, and device members since 3.12) does not fire; any other filter + value (including ``"fully_trusted"`` or a dynamic expression) still fires. + +v1 method-sink scope: the constructor must be a function-local binding (an +assignment / ``with ... as`` / walrus in the same scope, or the chained +receiver). Module-level constructions and annotation-only bindings carry no +resolvable constructor call and stay silent. + Tier-modulated; fires only where trust is declared. """ from __future__ import annotations -from wardline.core.finding import Kind, Maturity, Severity -from wardline.scanner.rules._sink_helpers import TaintedSinkRule +import ast +from typing import TYPE_CHECKING + +from wardline.core.finding import Finding, Kind, Maturity, Severity +from wardline.scanner.rules._sink_helpers import ( + RAW_ZONE, + TaintedSinkRule, + build_sink_finding, + collect_ctor_call_nodes, + enclosing_declared_tier, + module_for_qualname, + receiver_ctor_call, + resolved_sink_calls, + worst_arg_taint, +) from wardline.scanner.rules.metadata import RuleMetadata +from wardline.scanner.rules.severity_model import modulate + +if TYPE_CHECKING: + from wardline.scanner.context import AnalysisContext _SINKS = frozenset( { @@ -19,29 +63,144 @@ "os.open", "os.path.join", "pathlib.Path", + # filesystem mutation — tainted paths here are destructive traversal + "os.remove", + "os.unlink", + "os.rmdir", + "os.makedirs", + "os.mkdir", + "os.rename", + "os.renames", + "os.replace", + "shutil.rmtree", + "shutil.copy", + "shutil.copy2", + "shutil.copyfile", + "shutil.copytree", + "shutil.move", } ) +_PATH_METHODS = ("read_text", "read_bytes", "write_text", "write_bytes", "open", "unlink", "rmdir", "mkdir") +_PATH_METHOD_SINKS = frozenset(f"pathlib.Path.{m}" for m in _PATH_METHODS) + +_ARCHIVE_CTORS = ("tarfile.open", "tarfile.TarFile", "zipfile.ZipFile") +_ARCHIVE_METHOD_SINKS = frozenset(f"{ctor}.{m}" for ctor in _ARCHIVE_CTORS for m in ("extractall", "extract")) + +_METHOD_SINKS = _PATH_METHOD_SINKS | _ARCHIVE_METHOD_SINKS + METADATA = RuleMetadata( rule_id="PY-WL-116", base_severity=Severity.WARN, kind=Kind.DEFECT, multi_emit=True, description=( - "Untrusted data reaches a path/filesystem-traversal sink (open/os.path.join/pathlib.Path) " - "in a trusted-tier function." + "Untrusted data reaches a path/filesystem-traversal sink (open/os.path.join/pathlib.Path, " + "filesystem mutation via os.remove/os.rename/shutil.*, Path methods on a tainted pathlib.Path, " + "or tarfile/zipfile archive extraction — Zip Slip) in a trusted-tier function." ), examples_violation=( "@external_boundary\ndef read_raw(p):\n return p\n" "@trusted(level='ASSURED')\ndef f(p):\n open(read_raw(p))", + "import shutil\n" + "@external_boundary\ndef read_raw(p):\n return p\n" + "@trusted(level='ASSURED')\ndef f(p):\n shutil.rmtree(read_raw(p))", + "import pathlib\n" + "@external_boundary\ndef read_raw(p):\n return p\n" + "@trusted(level='ASSURED')\ndef f(p):\n q = pathlib.Path(read_raw(p))\n return q.read_text()", + "import tarfile\n" + "@external_boundary\ndef read_raw(p):\n return p\n" + "@trusted(level='ASSURED')\ndef f(p):\n tf = tarfile.open(read_raw(p))\n tf.extractall('/dst')", + ), + examples_clean=( + "@trusted(level='ASSURED')\ndef f():\n open('safe_file.txt')", + "import tarfile\n" + "@external_boundary\ndef read_raw(p):\n return p\n" + "@trusted(level='ASSURED')\ndef f(p):\n tf = tarfile.open(read_raw(p))\n" + " tf.extractall('/dst', filter='data')", ), - examples_clean=("@trusted(level='ASSURED')\ndef f():\n open('safe_file.txt')",), maturity=Maturity.PREVIEW, ) +def _has_safe_extraction_filter(call: ast.Call) -> bool: + """True iff the extraction call passes tarfile's safe filter as the LITERAL + ``filter="data"``. Any other value — ``"fully_trusted"``, ``"tar"`` (which still + permits in-tree hardlink tricks pre-3.12.x fixes), or a dynamic expression — is + not accepted as a mitigation.""" + return any( + kw.arg == "filter" and isinstance(kw.value, ast.Constant) and kw.value.value == "data" for kw in call.keywords + ) + + class PathTraversal(TaintedSinkRule): rule_id = METADATA.rule_id metadata = METADATA SINKS = _SINKS sink_label = "path-traversal" + + def check(self, context: AnalysisContext) -> list[Finding]: + findings = super().check(context) # direct dotted sinks (worst-of-all-args) + findings.extend(self._method_sink_findings(context)) + return findings + + def _method_sink_findings(self, context: AnalysisContext) -> list[Finding]: + """Construct-then-method sinks: Path methods and archive extraction. + + The method call itself is usually argument-less — the dangerous data is what + the RECEIVER was constructed from, so the taint verdict is the constructor + call's worst argument taint (flow-sensitive, at the construction site). + """ + findings: list[Finding] = [] + for qualname, entity in context.entities.items(): + tier = enclosing_declared_tier(qualname, context.project_taints, context.declared_qualnames) + severity = modulate(self.base_severity, tier) + if severity == Severity.NONE: + continue # freedom / fail-closed zone — suppressed + module = module_for_qualname(qualname, context) + alias_map = context.alias_maps.get(module, {}) if module is not None else {} + ctor_nodes = collect_ctor_call_nodes(entity.node) + module_bindings = context.module_bindings.get(module or "") + for call, fqn in resolved_sink_calls( + entity.node, _METHOD_SINKS, alias_map, module or "", module_bindings=module_bindings + ): + is_archive = fqn in _ARCHIVE_METHOD_SINKS + if is_archive and _has_safe_extraction_filter(call): + continue + ctor = receiver_ctor_call(call, ctor_nodes) + if ctor is None: + continue # no resolvable construction site (v1 scope) + worst = worst_arg_taint(ctor, qualname, context) + if worst is None or worst not in RAW_ZONE: + continue + line = call.lineno + if is_archive: + message = ( + f"{qualname}: {worst.value} (untrusted) archive opened at line {ctor.lineno} " + f"reaches the archive extraction (Zip Slip) sink {fqn}() at line {line}" + ) + else: + message = ( + f"{qualname}: {worst.value} (untrusted) data reaches the path-traversal sink " + f"{fqn}() at line {line} via a Path constructed at line {ctor.lineno}" + ) + # Shared constructor — identical wlfp2 discriminator shape as the base + # loop, keyed on the METHOD call + the resolved sink FQN. The FQN + # differs from the constructor sink's ("pathlib.Path.read_text" vs + # "pathlib.Path"), so the chained form's co-located ctor and method + # findings never collide. + findings.append( + build_sink_finding( + rule_id=self.rule_id, + entity=entity, + qualname=qualname, + call=call, + dotted=fqn, + severity=severity, + tier=tier, + worst=worst, + sink_label=self.sink_label, + message=message, + ) + ) + return findings diff --git a/src/wardline/scanner/rules/silent_exception.py b/src/wardline/scanner/rules/silent_exception.py index b98a4f19..c1e89143 100644 --- a/src/wardline/scanner/rules/silent_exception.py +++ b/src/wardline/scanner/rules/silent_exception.py @@ -13,8 +13,8 @@ from wardline.core.finding import Finding, Kind, Location, Severity from wardline.core.finding import compute_finding_fingerprint as _fp -from wardline.core.taints import TaintState from wardline.scanner.rules._ast_helpers import is_silent_handler, own_except_handlers +from wardline.scanner.rules._sink_helpers import enclosing_declared_tier from wardline.scanner.rules.metadata import RuleMetadata from wardline.scanner.rules.severity_model import modulate @@ -45,8 +45,11 @@ def __init__(self, base_severity: Severity | None = None) -> None: def check(self, context: AnalysisContext) -> list[Finding]: findings: list[Finding] = [] for qualname, entity in context.entities.items(): - lookup_name = qualname.split("..")[0] - tier = context.project_taints.get(lookup_name, TaintState.UNKNOWN_RAW) + # Nearest DECLARED enclosing scope governs a nested def (a nested def's own + # trust decorator wins; undeclared nested defs inherit) — the same + # enclosing_declared_tier semantics as the sink rule family, NOT the + # outermost-function strip (wardline-bb8396f96e / wardline-9b88ec5419). + tier = enclosing_declared_tier(qualname, context.project_taints, context.declared_qualnames) severity = modulate(self.base_severity, tier) if severity == Severity.NONE: continue diff --git a/src/wardline/scanner/rules/sql_injection.py b/src/wardline/scanner/rules/sql_injection.py index c4422d99..60a09c97 100644 --- a/src/wardline/scanner/rules/sql_injection.py +++ b/src/wardline/scanner/rules/sql_injection.py @@ -2,21 +2,55 @@ """PY-WL-118 — untrusted data reaches SQL/database execution sinks. Passing untrusted data directly to database queries (``cursor.execute``, -``cursor.executemany``) can lead to SQL Injection (SQLi) (CWE-89). -Tier-modulated; fires only where trust is declared. +``cursor.executemany``, ``cursor.executescript``) can lead to SQL Injection +(SQLi) (CWE-89). ``executescript`` (sqlite3 cursor AND connection) runs a +multi-statement script with NO parameter binding at all, so it is strictly more +dangerous than ``execute``. Tier-modulated; fires only where trust is declared. + +**Receiver heuristic (FP guard, fail-closed).** ``.execute`` is matched by method +name, so a receiver gate keeps non-database executors (task pools, command +objects) from firing a CWE-89 ERROR. Evidence is consulted strongest-first: + +1. *Binding evidence* — the receiver was provably constructed in this function + (``pool = sqlite3.connect(...)`` / chained ``Cls().execute``): a constructor + from a known DB driver module fires regardless of the receiver's name; one + from a known executor module (``concurrent.futures`` etc.) is suppressed. +2. *Name evidence* — exact token match on the receiver's simple name (a ``Name`` + id or the LAST attribute segment, underscore/camelCase split): a DB token + (``cursor``/``conn``/``db``/``session``/``engine``/...) fires and WINS over a + non-DB token; a non-DB token (``pool``/``executor``/``worker``/...) alone + suppresses. +3. *No evidence* — UNKNOWN receivers (opaque names like ``c``/``s``, dynamic + expressions) FIRE: when unsure we keep the finding, because an FN here is + worse than an FP. Token matching is exact (never substring), so ``secure`` + can never match ``cur``. + +**Text-clause constant exemption (FP guard).** The canonical SQLAlchemy +parameterized pattern ``conn.execute(text("... :id"), {"id": uid})`` wraps a +compile-time CONSTANT in a recognized text-clause constructor — it cannot carry +attacker bytes, so the operation slot is treated as clean. The exemption needs +BOTH the recognized constructor FQN (import-alias aware) and all-constant +arguments: ``text(tainted)`` / ``text(f"...")`` still fire (``text()`` is not a +sanitiser). """ from __future__ import annotations import ast +import re from typing import TYPE_CHECKING -from wardline.core.finding import Finding, Kind, Location, Maturity, Severity -from wardline.core.finding import compute_finding_fingerprint as _fp +from wardline.core.finding import Finding, Kind, Maturity, Severity from wardline.core.taints import RAW_ZONE, TRUST_RANK, TaintState from wardline.scanner.rules._sink_helpers import ( + SinkBindings, TaintedSinkRule, + build_sink_finding, + canonical_call_name, + collect_sink_bindings, + dotted_name, enclosing_declared_tier, + module_alias_map, resolved_arg_taints, sink_method_calls, ) @@ -24,9 +58,11 @@ from wardline.scanner.rules.severity_model import modulate if TYPE_CHECKING: + from collections.abc import Mapping + from wardline.scanner.context import AnalysisContext -_SINKS = frozenset({"execute", "executemany"}) +_SINKS = frozenset({"execute", "executemany", "executescript"}) # The DB-API call shape is ``cursor.execute(operation[, parameters])`` / # ``cursor.executemany(operation, seq_of_params)``: the SQL string is the FIRST @@ -36,7 +72,108 @@ # the canonical OWASP mitigation), so taint in the parameters position is NOT SQLi and # must not fire (wardline-e0e44852e7). This is a deliberate behaviour decision, not just # a test fix: we gate on the operation-string position and ignore the parameter position. -_SQL_STRING_KEYS: frozenset[int | str] = frozenset({0, "operation", "sql", "query", "statement"}) +# ``sql_script`` is sqlite3's parameter name for the executescript slot (its only arg). +_SQL_STRING_KEYS: frozenset[int | str] = frozenset({0, "operation", "sql", "query", "statement", "sql_script"}) + +# Recognized text-clause constructors (canonical import-resolved FQNs). A call to one +# of these with ALL-constant arguments is a compile-time-fixed SQL string — not +# injectable, regardless of the engine's (unresolvable-third-party) UNKNOWN_RAW verdict. +_TEXT_CLAUSE_FQNS = frozenset( + { + "sqlalchemy.text", + "sqlalchemy.sql.text", + "sqlalchemy.sql.expression.text", + } +) + +# Receiver-heuristic vocabularies (see the module docstring). Tokens match EXACTLY +# against the underscore/camelCase-split words of the receiver's simple name. +_DB_RECEIVER_TOKENS = frozenset( + { + "cursor", "cur", "conn", "connection", "db", "database", "session", + "engine", "sql", "sqlite", "postgres", "psql", "mysql", "oracle", + "tx", "txn", "transaction", + } +) # fmt: skip +_NON_DB_RECEIVER_TOKENS = frozenset( + { + "pool", "executor", "thread", "threads", "scheduler", "worker", + "workers", "dispatcher", "queue", "task", "tasks", "job", "jobs", + } +) # fmt: skip +# Constructor-FQN module prefixes: provably-DB fires regardless of receiver name; +# provably-executor suppresses. Everything else falls through to the name heuristic. +_DB_MODULE_PREFIXES = ( + "sqlite3.", "aiosqlite.", "apsw.", "duckdb.", + "psycopg2.", "psycopg.", "asyncpg.", "pg8000.", + "pymysql.", "MySQLdb.", "mysql.", "mariadb.", + "cx_Oracle.", "oracledb.", "pyodbc.", "sqlalchemy.", +) # fmt: skip +_NON_DB_MODULE_PREFIXES = ("concurrent.", "multiprocessing.", "threading.", "asyncio.") + +_NAME_TOKEN_RE = re.compile(r"[a-z]+") +_CAMEL_SPLIT_RE = re.compile(r"(?<=[a-z0-9])(?=[A-Z])") + + +def _name_tokens(name: str) -> frozenset[str]: + """Exact-match word tokens of an identifier: underscore + camelCase split, + lowercased, digits dropped (``dbConn2`` → {"db", "conn"}).""" + return frozenset(_NAME_TOKEN_RE.findall(_CAMEL_SPLIT_RE.sub("_", name).lower())) + + +def _is_constant_text_clause(expr: ast.expr, alias_map: Mapping[str, str]) -> bool: + """Whether *expr* is a recognized text-clause constructor call whose every + argument is a compile-time constant (the SQL string itself a ``str``). + + Fail-closed everywhere else: a non-recognized wrapper, a non-constant argument + (Name / f-string / nested call), a ``*``/``**`` splat, or a zero-argument call + proves nothing and keeps the slot's engine taint.""" + if not isinstance(expr, ast.Call): + return False + dotted = dotted_name(expr.func) + if dotted is None or canonical_call_name(dotted, alias_map) not in _TEXT_CLAUSE_FQNS: + return False + if not (expr.args or expr.keywords): + return False + for arg in expr.args: + if not (isinstance(arg, ast.Constant) and isinstance(arg.value, str)): + return False + return all(kw.arg is not None and isinstance(kw.value, ast.Constant) for kw in expr.keywords) + + +def _receiver_is_non_db(call: ast.Call, bindings: SinkBindings, alias_map: Mapping[str, str]) -> bool: + """Whether the sink call's receiver is CLEARLY a non-database object. + + Returns True ONLY on positive non-DB evidence (binding to an executor-module + constructor, or a non-DB name token with no DB token); every unknown receiver + returns False so the rule keeps firing (fail-closed — FN is worse than FP here). + """ + func = call.func + assert isinstance(func, ast.Attribute) # guaranteed by sink_method_calls + receiver = func.value + ctor_fqn: str | None = None + if isinstance(receiver, ast.Name): + ctor_fqn = bindings.instance_classes.get(receiver.id) + elif isinstance(receiver, ast.Call): + dotted = dotted_name(receiver.func) + ctor_fqn = canonical_call_name(dotted, alias_map) if dotted is not None else None + if ctor_fqn is not None: + if ctor_fqn.startswith(_DB_MODULE_PREFIXES): + return False # provably a DB-driver object — definite sink + if ctor_fqn.startswith(_NON_DB_MODULE_PREFIXES): + return True # provably an executor/pool-family object + if isinstance(receiver, ast.Name): + hint = receiver.id + elif isinstance(receiver, ast.Attribute): + hint = receiver.attr # self.thread_pool → "thread_pool" + elif ctor_fqn is not None: + hint = ctor_fqn.rsplit(".", 1)[-1] # conn.cursor() → "cursor" + else: + return False # dynamic receiver, no evidence — fail closed, fire + tokens = _name_tokens(hint) + if tokens & _DB_RECEIVER_TOKENS: + return False # DB evidence wins over mixed names ("db_pool") + return bool(tokens & _NON_DB_RECEIVER_TOKENS) def _kwargs_may_target_sql_string(call: ast.Call) -> bool: @@ -66,12 +203,22 @@ def _kwargs_may_target_sql_string(call: ast.Call) -> bool: return False -def _sql_string_taint(call: ast.Call, qualname: str, context: AnalysisContext) -> TaintState | None: +def _sql_string_taint( + call: ast.Call, + qualname: str, + context: AnalysisContext, + alias_map: Mapping[str, str], +) -> TaintState | None: """The least-trusted taint reaching the SQL-STRING argument (the operation), ignoring bound-parameter arguments. Fail-closed on positions that cannot be isolated from the operation: a ``Starred`` first positional (``execute(*args)``) and a ``**`` unpack that could supply the ``operation`` keyword (``execute(**kwargs)`` / ``**{"operation": ...}``). + An SQL slot whose expression is a recognized text-clause constructor over constants + (:func:`_is_constant_text_clause`) is skipped — a compile-time-fixed operation string + is not injectable even when the engine's verdict for the unresolvable third-party + wrapper is ``UNKNOWN_RAW`` (the canonical SQLAlchemy parameterized-query FP). + The engine collapses a ``**`` unpack to a single ``None``-keyed taint (the worst across the unpacked dict's values), so per-key attribution is impossible at this layer: when a literal ``**`` dict carries an SQL-string key, the ``None`` taint is treated as reaching the @@ -79,6 +226,16 @@ def _sql_string_taint(call: ast.Call, qualname: str, context: AnalysisContext) - over-approximation (never an FN). See wardline-8c31463f9f / wardline-e0e44852e7.""" taints = resolved_arg_taints(call, qualname, context) kwargs_is_sql_slot = _kwargs_may_target_sql_string(call) + # Slot-key → source expression, so a slot's TAINT can be overridden by what the slot + # syntactically IS. Starred positionals keep only their "*i" fail-closed key; a ``**`` + # unpack (key None) has no single expression and is never exempted. + slot_exprs: dict[int | str | None, ast.expr] = {} + for i, arg in enumerate(call.args): + if not isinstance(arg, ast.Starred): + slot_exprs[i] = arg + for kw in call.keywords: + if kw.arg is not None: + slot_exprs[kw.arg] = kw.value worst: TaintState | None = None for key, ts in taints.items(): if ts is None: @@ -88,7 +245,12 @@ def _sql_string_taint(call: ast.Call, qualname: str, context: AnalysisContext) - or key == "*0" # Starred first positional — cannot isolate; fail closed or (key is None and kwargs_is_sql_slot) # ** unpack that may target the operation ) - if is_sql_slot and (worst is None or TRUST_RANK[ts] > TRUST_RANK[worst]): + if not is_sql_slot: + continue + expr = slot_exprs.get(key) + if expr is not None and _is_constant_text_clause(expr, alias_map): + continue # constant text() clause — compile-time-fixed, not injectable + if worst is None or TRUST_RANK[ts] > TRUST_RANK[worst]: worst = ts return worst @@ -99,11 +261,16 @@ def _sql_string_taint(call: ast.Call, qualname: str, context: AnalysisContext) - kind=Kind.DEFECT, multi_emit=True, description=( - "Untrusted data reaches a SQL/database execution sink (execute/executemany) in a trusted-tier function." + "Untrusted data reaches a SQL/database execution sink (execute/executemany/executescript) " + "in a trusted-tier function." ), examples_violation=( "@external_boundary\ndef read_raw(p):\n return p\n" "@trusted(level='ASSURED')\ndef f(p, cursor):\n cursor.execute(read_raw(p))", + # executescript runs a multi-statement script with NO parameter binding at all — + # the single most injection-prone DB-API method (wardline-1751b0fac6). + "@external_boundary\ndef read_raw(p):\n return p\n" + "@trusted(level='ASSURED')\ndef f(p, cursor):\n cursor.executescript(read_raw(p))", ), examples_clean=( "@trusted(level='ASSURED')\ndef f(cursor):\n cursor.execute('SELECT * FROM users')", @@ -112,6 +279,12 @@ def _sql_string_taint(call: ast.Call, qualname: str, context: AnalysisContext) - "@external_boundary\ndef read_raw(p):\n return p\n" "@trusted(level='ASSURED')\ndef g(p, cursor):\n" " cursor.execute('SELECT * FROM t WHERE id = ?', (read_raw(p),))", + # The canonical SQLAlchemy parameterized query: a compile-time-constant operation + # string wrapped in text(), with the untrusted value bound — not injectable. + "from sqlalchemy import text\n" + "@external_boundary\ndef read_raw(p):\n return p\n" + "@trusted(level='ASSURED')\ndef h(p, conn):\n" + " conn.execute(text('SELECT * FROM t WHERE id = :id'), {'id': read_raw(p)})", ), maturity=Maturity.PREVIEW, ) @@ -135,39 +308,34 @@ def check(self, context: AnalysisContext) -> list[Finding]: severity = modulate(self.base_severity, tier) if severity == Severity.NONE: continue # freedom / fail-closed zone — suppressed + alias_map = module_alias_map(qualname, context) + bindings: SinkBindings | None = None # collected lazily, once per entity for node in sink_method_calls(entity.node, self.SINKS): - worst = _sql_string_taint(node, qualname, context) + if bindings is None: + # Function-local bindings only: entities carry no module AST, so a + # module-level-constructed receiver falls to the name heuristic + # (which fails closed — fires — on anything not clearly non-DB). + bindings = collect_sink_bindings(entity.node, alias_map) + if _receiver_is_non_db(node, bindings, alias_map): + continue # clearly a task-pool/executor object, not a SQL surface + worst = _sql_string_taint(node, qualname, context, alias_map) if worst is None or worst not in RAW_ZONE: continue assert isinstance(node.func, ast.Attribute) # guaranteed by sink_method_calls - sink_name = node.func.attr - line = node.lineno + # Shared constructor — identical message shape and wlfp2 discriminator + # as the base loop, keyed on the METHOD NAME (execute/executemany/ + # executescript), never the resolved receiver (drifts). findings.append( - Finding( + build_sink_finding( rule_id=self.rule_id, - message=( - f"{qualname}: {worst.value} (untrusted) data reaches the {self.sink_label} " - f"sink {sink_name}() at line {line}" - ), - severity=severity, - kind=Kind.DEFECT, - location=Location(path=entity.location.path, line_start=line), - fingerprint=_fp( - rule_id=self.rule_id, - path=entity.location.path, - qualname=qualname, - # Call-site-anchored, >1 finding per (rule, path, qualname) possible (execute + - # executemany, or a chain ``cur.execute(a).execute(b)``). Discriminate SOURCE-only: - # an ENTITY-RELATIVE line offset (call line - def line, invariant to a comment - # ABOVE the function: wlfp2/wardline-8654423823) + the call's full lexical SPAN + - # the method name. The span (start:end), not the start column alone, separates a - # chain's outer/inner calls. Never the resolved arg taint (drifts). - taint_path=f"{line - (entity.location.line_start or 0)}:{node.col_offset}:{node.end_col_offset}:{sink_name}", # noqa: E501 - ), - # OLD (wlfp1) taint_path, byte-exact, for `wardline rekey` (P4). - taint_path_v0=f"{sink_name}@{node.col_offset}:{node.end_col_offset}", + entity=entity, qualname=qualname, - properties={"tier": tier.value, "sink": sink_name, "arg_taint": worst.value}, + call=node, + dotted=node.func.attr, + severity=severity, + tier=tier, + worst=worst, + sink_label=self.sink_label, ) ) return findings diff --git a/src/wardline/scanner/rules/ssrf.py b/src/wardline/scanner/rules/ssrf.py index 445425ce..5e23800d 100644 --- a/src/wardline/scanner/rules/ssrf.py +++ b/src/wardline/scanner/rules/ssrf.py @@ -1,36 +1,77 @@ # src/wardline/scanner/rules/ssrf.py """PY-WL-117 — untrusted data reaches an HTTP client sink. -Passing untrusted data to HTTP clients (``requests``, ``httpx``, ``urllib``) without -validation can lead to Server-Side Request Forgery (SSRF) (CWE-918). -Tier-modulated; fires only where trust is declared. +Passing untrusted data to HTTP clients (``requests``, ``httpx``, ``aiohttp``, +``urllib``) without validation can lead to Server-Side Request Forgery (SSRF) +(CWE-918). Tier-modulated; fires only where trust is declared. + +Two sharpenings over the generic :class:`TaintedSinkRule` machinery +(wardline-3002f63969 / wardline-66b2c91470): + +* **Construct-then-method** — production code overwhelmingly pools requests + through a constructed client (``client = httpx.Client(); client.get(url)``, + ``async with httpx.AsyncClient() as c``, ``requests.Session().get(url)``, + ``aiohttp.ClientSession``). Sink matching goes through + :func:`resolved_sink_calls`, which layers name-binding resolution over the + direct dotted/import-aliased spelling. + +* **URL-slot precision** — for an HTTP client, only the request-target slot is + an SSRF vector. Every sink carries an :class:`ArgSpec` naming its URL slot + (``base_url=`` for client constructors), so a tainted ``timeout=``/ + ``verify=``/``headers=`` with a clean literal URL no longer fires + (:func:`worst_dangerous_arg_taint` keeps the fail-closed splat widening). """ from __future__ import annotations +from typing import TYPE_CHECKING + from wardline.core.finding import Kind, Maturity, Severity -from wardline.scanner.rules._sink_helpers import TaintedSinkRule +from wardline.scanner.rules._sink_helpers import ArgSpec, TaintedSinkRule from wardline.scanner.rules.metadata import RuleMetadata -_SINKS = frozenset( - { - "requests.get", - "requests.post", - "requests.put", - "requests.patch", - "requests.delete", - "requests.request", - "httpx.get", - "httpx.post", - "httpx.put", - "httpx.patch", - "httpx.delete", - "httpx.request", - "httpx.Client", - "httpx.AsyncClient", - "urllib.request.urlopen", - } -) +if TYPE_CHECKING: + from collections.abc import Mapping + +# The URL is the first positional / the ``url`` keyword (requests.get, client.get, ...). +_URL_FIRST = ArgSpec(positions=(0,), keywords=("url",)) +# ``(method, url, ...)`` signatures: the URL is the SECOND slot; a tainted method verb +# cannot redirect the request target (requests.request, client.request, client.stream). +_URL_SECOND = ArgSpec(positions=(1,), keywords=("url",)) +# httpx client constructors: the first positional is NOT a URL; ``base_url`` is the +# only SSRF-relevant constructor argument (and is keyword-only). +_BASE_URL_ONLY = ArgSpec(keywords=("base_url",)) + +_HTTP_VERBS = ("get", "post", "put", "patch", "delete", "head", "options") +_CLIENT_CLASSES = ("httpx.Client", "httpx.AsyncClient", "requests.Session", "aiohttp.ClientSession") + + +def _build_sink_specs() -> dict[str, ArgSpec]: + specs: dict[str, ArgSpec] = {} + for mod in ("requests", "httpx"): + for verb in _HTTP_VERBS: + specs[f"{mod}.{verb}"] = _URL_FIRST + specs[f"{mod}.request"] = _URL_SECOND + specs["httpx.stream"] = _URL_SECOND + specs["urllib.request.urlopen"] = _URL_FIRST + specs["urllib.request.Request"] = _URL_FIRST # carries the tainted URL into urlopen + specs["httpx.Client"] = _BASE_URL_ONLY + specs["httpx.AsyncClient"] = _BASE_URL_ONLY + # aiohttp.ClientSession(base_url=...) — base_url IS the first positional there. + specs["aiohttp.ClientSession"] = ArgSpec(positions=(0,), keywords=("base_url",)) + # Instance methods on a constructed client/session (requests.Session() itself + # takes no arguments, so the constructor is not a sink — only its methods are). + for cls in _CLIENT_CLASSES: + for verb in _HTTP_VERBS: + specs[f"{cls}.{verb}"] = _URL_FIRST + specs[f"{cls}.request"] = _URL_SECOND + for cls in ("httpx.Client", "httpx.AsyncClient"): + specs[f"{cls}.stream"] = _URL_SECOND + return specs + + +_SINK_SPECS: Mapping[str, ArgSpec] = _build_sink_specs() +_SINKS = frozenset(_SINK_SPECS) METADATA = RuleMetadata( rule_id="PY-WL-117", @@ -38,19 +79,32 @@ kind=Kind.DEFECT, multi_emit=True, description=( - "Untrusted data reaches an HTTP client sink (SSRF, requests/httpx/urllib) in a trusted-tier function." + "Untrusted data reaches the URL slot of an HTTP client sink " + "(SSRF, requests/httpx/aiohttp/urllib — module-level calls, constructed " + "client/session methods, and client base_url=) in a trusted-tier function." ), examples_violation=( "@external_boundary\ndef read_raw(p):\n return p\n" "@trusted(level='ASSURED')\ndef f(p):\n requests.get(read_raw(p))", + "@external_boundary\ndef read_raw(p):\n return p\n" + "@trusted(level='ASSURED')\ndef f(p):\n" + " client = httpx.Client()\n client.get(read_raw(p))", + ), + examples_clean=( + "@trusted(level='ASSURED')\ndef f():\n requests.get('https://example.com')", + "@external_boundary\ndef read_raw(p):\n return p\n" + "@trusted(level='ASSURED')\ndef f(p):\n" + " requests.get('https://example.com', timeout=read_raw(p))", ), - examples_clean=("@trusted(level='ASSURED')\ndef f():\n requests.get('https://example.com')",), maturity=Maturity.PREVIEW, ) class SSRF(TaintedSinkRule): + # Attribute-only since the 2026-06-10 consolidation: the base check IS the + # binding-aware loop, and SINK_SPECS carries the URL-slot precision. rule_id = METADATA.rule_id metadata = METADATA SINKS = _SINKS + SINK_SPECS = _SINK_SPECS sink_label = "HTTP-client" diff --git a/src/wardline/scanner/rules/stored_taint.py b/src/wardline/scanner/rules/stored_taint.py index cd266591..1137f1b4 100644 --- a/src/wardline/scanner/rules/stored_taint.py +++ b/src/wardline/scanner/rules/stored_taint.py @@ -5,6 +5,26 @@ ``read_text`` or database cursor fetches) reaches a trusted state (returned by a ``@trusted`` function or passed to a ``@trusted`` callee) without being validated (e.g., through a ``@trust_boundary``). + +**Receiver-aware matching.** The storage-read matcher is binding-aware (the shared +``_sink_helpers`` class-tracking): an ``io.StringIO``/``io.BytesIO`` receiver is an +in-memory buffer whose ``.read()`` returns data the process itself put there — never +*persistent* storage — so it is exempt (wardline-66b2c91470: the receiver-blind +``.read()`` match mislabeled an in-memory constant as "stored/persisted data"). Any +taint flowing THROUGH such a buffer is still tracked by the engine (the buffer var +itself propagates via ``_collect_stored_vars``), and PY-WL-101 still polices the +producer's return claim. + +**PY-WL-101 de-confliction on the return arm (documented winner: 101).** A matched +return whose taint is ``UNKNOWN_RAW``/``MIXED_RAW`` has UNRESOLVED provenance — the +"stored/persisted" label rests solely on the AST name match, which cannot justify +it — so when PY-WL-101 fires on the same producer this rule SUPPRESSES its return +finding and delegates to 101 (the mature, gate-eligible trust-claim check; this rule +is PREVIEW). A return whose taint is ``EXTERNAL_RAW`` is SUBSTANTIATED storage +provenance (the open()/Path.read_*/fetch* seeds), and there the deliberate +complementary pair stands: 101 reports the trust-claim violation, 120 adds the +storage-provenance annotation (pinned by the frozen identity corpus and the wlfp1 +migration oracle). The call-argument arm is untouched — 101 never covers it. """ from __future__ import annotations @@ -14,24 +34,45 @@ from wardline.core.finding import Finding, Kind, Location, Maturity, Severity from wardline.core.finding import compute_finding_fingerprint as _fp -from wardline.core.taints import RAW_ZONE, TaintState +from wardline.core.taints import RAW_ZONE, TRUST_RANK, TaintState from wardline.scanner.rules._ast_helpers import own_nodes -from wardline.scanner.rules._sink_helpers import dotted_name, worst_arg_taint +from wardline.scanner.rules._sink_helpers import ( + SinkBindings, + collect_sink_bindings, + dotted_name, + module_alias_map, + resolve_bound_call_fqn, + worst_arg_taint, +) from wardline.scanner.rules.metadata import RuleMetadata from wardline.scanner.rules.severity_model import modulate if TYPE_CHECKING: + from collections.abc import Mapping + from wardline.scanner.context import AnalysisContext +_READ_ATTR_METHODS = frozenset({"read", "read_text", "read_bytes", "fetchone", "fetchall", "fetchmany"}) +# In-memory buffer classes: a ``.read()`` on one returns data the PROCESS itself put +# there — never persistent storage — so the storage-provenance label cannot apply. +# Canonical FQNs (the binding machinery resolves ``from io import StringIO`` / +# ``import io as x`` spellings through the module alias map before the lookup). +_IN_MEMORY_BUFFER_FQNS = frozenset({"io.StringIO", "io.BytesIO"}) + -def _is_storage_read_call(node: ast.AST) -> bool: +def _is_storage_read_call(node: ast.AST, bindings: SinkBindings, alias_map: Mapping[str, str]) -> bool: if isinstance(node, ast.Call): if isinstance(node.func, ast.Name): if node.func.id in ("open", "read"): return True elif isinstance(node.func, ast.Attribute): - if node.func.attr in ("read", "read_text", "read_bytes", "fetchone", "fetchall", "fetchmany"): - return True + if node.func.attr in _READ_ATTR_METHODS: + # Receiver-awareness (wardline-66b2c91470): a statically-known + # in-memory buffer receiver (bound var / with-target / chained + # ctor, via the shared binding machinery) is NOT a storage read. + # An unresolvable receiver stays conservatively matched. + bound = resolve_bound_call_fqn(node, bindings, alias_map) + return not (bound is not None and bound.rsplit(".", 1)[0] in _IN_MEMORY_BUFFER_FQNS) if ( isinstance(node.func.value, ast.Name) and node.func.value.id == "os" @@ -41,13 +82,13 @@ def _is_storage_read_call(node: ast.AST) -> bool: return False -def _collect_stored_vars(node: ast.AST) -> set[str]: +def _collect_stored_vars(node: ast.AST, bindings: SinkBindings, alias_map: Mapping[str, str]) -> set[str]: stored_vars: set[str] = set() for child in own_nodes(node): if isinstance(child, ast.Assign): is_storage = False for val_node in own_nodes(child.value): - if _is_storage_read_call(val_node): + if _is_storage_read_call(val_node, bindings, alias_map): is_storage = True break if not is_storage: @@ -66,7 +107,7 @@ def _collect_stored_vars(node: ast.AST) -> set[str]: elif isinstance(child, ast.AnnAssign) and child.value is not None: is_storage = False for val_node in own_nodes(child.value): - if _is_storage_read_call(val_node): + if _is_storage_read_call(val_node, bindings, alias_map): is_storage = True break if not is_storage: @@ -79,6 +120,34 @@ def _collect_stored_vars(node: ast.AST) -> set[str]: return stored_vars +def _return_delegated_to_101(qualname: str, context: AnalysisContext) -> bool: + """True when PY-WL-101 fires on *qualname*'s return (mirrors 101's gate). + + Deliberate coupling: this must stay in lockstep with + ``untrusted_reaches_trusted.UntrustedReachesTrusted.check`` — suppression is + only sound when the delegate actually picks the defect up (otherwise the + return finding must stand, e.g. a non-anchored tier 101 cannot police). + That includes ENABLEMENT: under ``rules.enable`` without PY-WL-101 the + delegate never runs, so suppressing here would drop the raw-storage-return + defect from the scan entirely (review 2026-06-10). ``None`` (a direct + construction / duck-typed registry seam) preserves the historical + assume-enabled behavior. + """ + if context.enabled_rule_ids is not None and "PY-WL-101" not in context.enabled_rule_ids: + return False + prov = context.taint_provenance.get(qualname) + if prov is None or prov.source != "anchored": + return False + declared = context.project_return_taints.get(qualname) + if declared is None or declared in RAW_ZONE: + return False # 101's trust-claim gate + body = context.project_taints.get(qualname) + if body is not None and TRUST_RANK[body] > TRUST_RANK[declared]: + return False # trust-raising shape — 101 delegates that to PY-WL-102 + actual = context.function_return_taints.get(qualname) + return actual is not None and TRUST_RANK[actual] > TRUST_RANK[declared] + + METADATA = RuleMetadata( rule_id="PY-WL-120", base_severity=Severity.ERROR, @@ -89,6 +158,9 @@ def _collect_stored_vars(node: ast.AST) -> set[str]: "@trusted(level='ASSURED')\ndef get_config():\n data = open('config.txt').read()\n return data", ), examples_clean=( + # validate must be a REAL @trust_boundary: an undefined bare name no longer + # launders to the caller's seed, so the example defines its own validator. + "@trust_boundary(to_level='ASSURED')\ndef validate(x):\n if not x:\n raise ValueError\n return x\n" "@trusted(level='ASSURED')\ndef get_config():\n data = validate(open('config.txt').read())\n return data", ), maturity=Maturity.PREVIEW, @@ -111,14 +183,16 @@ def check(self, context: AnalysisContext) -> list[Finding]: if severity == Severity.NONE: continue - stored_vars = _collect_stored_vars(entity.node) + alias_map = module_alias_map(qualname, context) + bindings = collect_sink_bindings(entity.node, alias_map) + stored_vars = _collect_stored_vars(entity.node, bindings, alias_map) if not stored_vars: # Check if there is a direct return of a storage read call has_direct_read = False for node in own_nodes(entity.node): if isinstance(node, ast.Return) and node.value is not None: for val_node in own_nodes(node.value): - if _is_storage_read_call(val_node): + if _is_storage_read_call(val_node, bindings, alias_map): has_direct_read = True break if not has_direct_read: @@ -128,20 +202,32 @@ def check(self, context: AnalysisContext) -> list[Finding]: for node in own_nodes(entity.node): if isinstance(node, ast.Return) and node.value is not None: is_stored_return = False - if _is_storage_read_call(node.value): + if _is_storage_read_call(node.value, bindings, alias_map): is_stored_return = True else: for val_node in own_nodes(node.value): if isinstance(val_node, ast.Name) and val_node.id in stored_vars: is_stored_return = True break - if _is_storage_read_call(val_node): + if _is_storage_read_call(val_node, bindings, alias_map): is_stored_return = True break if is_stored_return: # Check return taint ret_taint = context.function_return_taints.get(qualname) + if ( + ret_taint is not None + and ret_taint is not TaintState.EXTERNAL_RAW + and ret_taint in RAW_ZONE + and _return_delegated_to_101(qualname, context) + ): + # UNKNOWN/MIXED_RAW return: provenance UNRESOLVED, so the + # "stored/persisted" label is unsubstantiated — suppress and + # delegate the trust-claim violation to PY-WL-101 (101 wins; + # module docstring). EXTERNAL_RAW (a recognized storage seed) + # keeps the documented complementary 120+101 pair. + continue if ret_taint is not None and ret_taint in RAW_ZONE: findings.append( Finding( @@ -179,7 +265,7 @@ def check(self, context: AnalysisContext) -> list[Finding]: if isinstance(val_node, ast.Name) and val_node.id in stored_vars: has_stored_arg = True break - if _is_storage_read_call(val_node): + if _is_storage_read_call(val_node, bindings, alias_map): has_stored_arg = True break diff --git a/src/wardline/scanner/rules/untrusted_to_command.py b/src/wardline/scanner/rules/untrusted_to_command.py index cc70988d..3b47ee54 100644 --- a/src/wardline/scanner/rules/untrusted_to_command.py +++ b/src/wardline/scanner/rules/untrusted_to_command.py @@ -1,25 +1,68 @@ # src/wardline/scanner/rules/untrusted_to_command.py -"""PY-WL-108 — untrusted data reaches an OS-command sink. +"""PY-WL-108 — untrusted data reaches a command/program-execution sink. + +**Charter (expanded 2026-06-10, wardline-13cfdd7b31 / wardline-c83b40c73a):** the +rule covers two CWE-78 sink shapes, both stdlib: + +* **always-shell string APIs** — ``os.system``, ``os.popen``, + ``subprocess.getoutput`` / ``getstatusoutput``: these take a shell *string*, + so an untrusted argument is directly injectable; +* **argv-style program execution** — ``os.exec*`` / ``os.spawn*`` / + ``os.posix_spawn`` / ``os.posix_spawnp`` / ``pty.spawn``: no shell mediates, + but an attacker-controlled program path or argv element IS arbitrary-program + execution — neither always-shell nor ``shell=True``, so previously covered by + neither 108 nor 112. -Passing untrusted data to an **always-shell** OS-command API — ``os.system``, -``os.popen``, ``subprocess.getoutput`` / ``getstatusoutput`` — is OS command injection -(CWE-78): these take a shell *string*, so an untrusted argument is directly injectable. Tier-modulated; fires only where trust is declared. -**Scope (FP-safe):** the ``subprocess.run`` / ``call`` / ``Popen`` / ``check_*`` family -is intentionally NOT in the sink set — with the default ``shell=False`` an argv-LIST is -safe (no shell), so firing on them floods false positives; only ``shell=True`` makes them -injectable, and detecting that keyword reliably is policed separately by PY-WL-112. -Covering the always-shell APIs catches the unambiguous case without the argv-list FP. +**Scope (FP-safe):** the ``subprocess.run`` / ``call`` / ``Popen`` / ``check_*`` +family is intentionally NOT in the sink set — with the default ``shell=False`` an +argv-LIST is safe (no shell), so firing on them floods false positives; only +``shell=True`` makes them injectable, and detecting that keyword reliably is +policed separately by PY-WL-112. + +**shlex.quote semantics (decided, wardline-13cfdd7b31):** ``shlex.quote(x)`` +neutralizes shell-string taint for the ALWAYS-SHELL sinks **in concatenation +context only**. The command argument is GUARDED when it is a string +concatenation (``+`` chain or f-string) with at least one constant fragment in +which every non-constant leaf is a ``shlex.quote(...)`` call — the attacker +bytes then enter the shell line solely as a single quoted token of a +constant-shaped command (``os.system("echo " + shlex.quote(raw))`` is clean). +A BARE whole-command quote (``os.system(shlex.quote(raw))``) still fires: a +fully-quoted single token handed to a shell executes that token as the program +name, so the attacker still picks what runs. The guard NEVER applies to the +argv program-execution sinks — no shell parses the value, so quoting protects +nothing there. ``%``-formatting and ``str.format`` are not recognized as +concatenation (bounded: they keep firing). + +The guard is INLINE-syntactic only: a quote result routed through a variable +(``safe = shlex.quote(raw); os.system("echo " + safe)``) still fires. That is +deliberate — resolving the name through the (non-branch-aware, +last-binding-wins) binding collector would let a LATER ``x = shlex.quote(x)`` +launder an EARLIER raw use of ``x``; for a sink MATCH that over-approximation +is silent, but for a NEUTRALIZER it would be a false-negative hole. Clearing +the variable-mediated form soundly needs a flow-sensitive context-encoder +taint (engine-level), not a rule-side syntax check. """ from __future__ import annotations +import ast +from typing import TYPE_CHECKING + from wardline.core.finding import Kind, Severity -from wardline.scanner.rules._sink_helpers import TaintedSinkRule +from wardline.scanner.rules._sink_helpers import ( + TaintedSinkRule, + canonical_call_name, + dotted_name, +) from wardline.scanner.rules.metadata import RuleMetadata -_SINKS = frozenset( +if TYPE_CHECKING: + from collections.abc import Mapping + +# The always-shell string APIs — the only sinks the shlex.quote guard applies to. +_SHELL_STRING_SINKS = frozenset( { "os.system", "os.popen", @@ -28,17 +71,99 @@ } ) +# Argv-style program execution: the value is a program path / argv, not a shell +# string. shlex.quote does NOT protect these (no shell parses the value). +_PROGRAM_EXEC_SINKS = frozenset( + { + "os.execl", + "os.execle", + "os.execlp", + "os.execlpe", + "os.execv", + "os.execve", + "os.execvp", + "os.execvpe", + "os.spawnl", + "os.spawnle", + "os.spawnlp", + "os.spawnlpe", + "os.spawnv", + "os.spawnve", + "os.spawnvp", + "os.spawnvpe", + "os.posix_spawn", + "os.posix_spawnp", + "pty.spawn", + } +) + +_SINKS = _SHELL_STRING_SINKS | _PROGRAM_EXEC_SINKS + + +def _is_shlex_quote_call(expr: ast.expr, alias_map: Mapping[str, str]) -> bool: + """True iff *expr* is a call whose func canonicalizes to ``shlex.quote`` + (covers ``shlex.quote(x)``, ``from shlex import quote``, module aliases).""" + if not isinstance(expr, ast.Call): + return False + dotted = dotted_name(expr.func) + return dotted is not None and canonical_call_name(dotted, dict(alias_map)) == "shlex.quote" + + +def _quote_guarded_concat(expr: ast.expr, alias_map: Mapping[str, str]) -> bool: + """True iff *expr* is a string CONCATENATION (``+`` chain / f-string) with at + least one constant fragment whose every non-constant leaf is + ``shlex.quote(...)`` — the GUARDED shape for a shell-string command. + + The constant-fragment requirement is what excludes the bare whole-command + quote: ``shlex.quote(raw)`` alone is not a concatenation, and ``f"{...}"`` + of nothing but quote calls has no constant command around the token. + """ + leaves: list[ast.expr] = [] + + def flatten(e: ast.expr) -> None: + if isinstance(e, ast.BinOp) and isinstance(e.op, ast.Add): + flatten(e.left) + flatten(e.right) + elif isinstance(e, ast.JoinedStr): + for part in e.values: + if isinstance(part, ast.FormattedValue): + leaves.append(part.value) + else: + leaves.append(part) # the f-string's constant fragments + else: + leaves.append(e) + + flatten(expr) + has_const = any(isinstance(leaf, ast.Constant) for leaf in leaves) + has_quote = any(_is_shlex_quote_call(leaf, alias_map) for leaf in leaves) + if not (has_const and has_quote): + return False + return all(isinstance(leaf, ast.Constant) or _is_shlex_quote_call(leaf, alias_map) for leaf in leaves) + + METADATA = RuleMetadata( rule_id="PY-WL-108", - base_severity=Severity.WARN, + # Calibrated with PY-WL-118 (SQLi): tainted command/program execution is the + # same exploit class (CWE-78 ≅ CWE-89 in blast radius), so the same base. + base_severity=Severity.ERROR, kind=Kind.DEFECT, multi_emit=True, - description="Untrusted data reaches an always-shell OS-command sink (os.system/os.popen/subprocess.getoutput).", + description=( + "Untrusted data reaches a command/program-execution sink " + "(os.system/os.popen/subprocess.getoutput, os.exec*/os.spawn*/os.posix_spawn/pty.spawn)." + ), examples_violation=( "@external_boundary\ndef read_raw(p):\n return p\n" "@trusted(level='ASSURED')\ndef f(p):\n os.system(read_raw(p))", + "@external_boundary\ndef read_raw(p):\n return p\n" + "@trusted(level='ASSURED')\ndef f(p):\n os.execv(read_raw(p), ['prog'])", + ), + examples_clean=( + "@trusted(level='ASSURED')\ndef f():\n os.system('ls -la')", + # shlex.quote as a FRAGMENT of a constant command is the blessed remediation. + "@external_boundary\ndef read_raw(p):\n return p\n" + "@trusted(level='ASSURED')\ndef f(p):\n os.system('echo ' + shlex.quote(read_raw(p)))", ), - examples_clean=("@trusted(level='ASSURED')\ndef f():\n os.system('ls -la')",), ) @@ -47,3 +172,8 @@ class UntrustedToCommand(TaintedSinkRule): metadata = METADATA SINKS = _SINKS sink_label = "OS-command" + + def _arg_guarded(self, expr: ast.expr, fqn: str, alias_map: Mapping[str, str]) -> bool: # noqa: PLR6301 + # shlex.quote guards shell-STRING sinks only (see module docstring); a + # quoted value reaching an argv exec/spawn sink is still attacker-chosen. + return fqn in _SHELL_STRING_SINKS and _quote_guarded_concat(expr, alias_map) diff --git a/src/wardline/scanner/rules/untrusted_to_deserialization.py b/src/wardline/scanner/rules/untrusted_to_deserialization.py index 67c1830b..63cc4e5b 100644 --- a/src/wardline/scanner/rules/untrusted_to_deserialization.py +++ b/src/wardline/scanner/rules/untrusted_to_deserialization.py @@ -6,48 +6,166 @@ developer-freedom zone, fires where trust is declared. The ``safe_*`` loaders and the *dump* direction are intentionally NOT sinks here. ``json.loads`` is excluded (it does not execute) to avoid noise. + +Sink families (ticket wardline-4299f07bb4): + +* **Stdlib direct loaders** — pickle/marshal/yaml ``load``/``loads`` spellings. These + keep the historical worst-of-ALL-args taint test (no :class:`ArgSpec`). +* **OO streaming-unpickle API** — ``pickle.Unpickler(stream).load()``, chained or + stored-instance (``u = pickle.Unpickler(stream); u.load()``), resolved through the + shared sink-binding machinery. ``load()`` itself takes no arguments; the dangerous + data is the stream handed to the CONSTRUCTOR, so the taint is read from the + constructor call's first argument. An annotation-only binding (``u: + pickle.Unpickler`` with no constructor in scope) has no stream argument to read — + a documented bounded false negative. ``marshal`` has no reader-object API, so there + is no marshal analogue. +* **``shelve.open``** — pickle-backed: opening a shelf at an attacker-controlled PATH + then reading keys unpickles attacker bytes. The taint shape differs from the blob + loaders — it is on the path argument only (``positions=(0,)`` / + ``keywords=("filename",)``), so a tainted flag/protocol slot does not fire. +* **Curated third-party CWE-502 table** — ``dill.load``/``loads``, + ``jsonpickle.decode``, ``joblib.load``, ``torch.load``, ``numpy.load``. Matching is + by canonical dotted name at the AST level (through the module's import-alias map); + the analyzer never imports these packages. Two literal-keyword gates, same + statically-visible-literal discipline as PY-WL-112's ``shell=True``: + ``numpy.load`` fires ONLY with a literal ``allow_pickle=True`` (the default is + False — safe — since numpy 1.16.3, so absent/False/dynamic stays silent); + ``torch.load`` is suppressed by a literal ``weights_only=True`` (the restricted + unpickler — the modern safe spelling) and fires otherwise, because older torch + defaults to the full unpickler. + +**Severity decision:** every entry here is RCE-equivalent (arbitrary object-graph +execution), so all carry the rule family's base severity (WARN, tier-modulated) — +exactly the standing of ``pickle.loads``. No per-sink severity split. + +**Why WARN and not the 118/108/112/124 ERROR class — a deliberate FP-economics +call (severity lattice review 2026-06-10).** The ERROR family members buy their +base with strong per-finding evidence (slot-precise ArgSpecs, literal-keyword +gates, an SQL-string position test); the classic blob loaders here keep the +worst-of-all-args test, and deserializing data from internal stores/caches the +engine cannot vouch for is a pervasive legitimate idiom — exploitability hinges +on the SOURCE being attacker-reachable, which a static worst-arg test cannot +establish. One class lower balances that weaker evidence. Promote via +``rules.severity`` per project, or revisit alongside the frozen identity corpus. """ from __future__ import annotations +import ast +from typing import TYPE_CHECKING + from wardline.core.finding import Kind, Severity -from wardline.scanner.rules._sink_helpers import TaintedSinkRule +from wardline.scanner.rules._sink_helpers import ( + ArgSpec, + TaintedSinkRule, + collect_ctor_call_nodes, + receiver_ctor_call, +) from wardline.scanner.rules.metadata import RuleMetadata -_SINKS = frozenset( - { - "pickle.loads", - "pickle.load", - "marshal.loads", - "marshal.load", - "yaml.load", - "yaml.load_all", - "yaml.unsafe_load", - "yaml.full_load", - } -) +if TYPE_CHECKING: + from collections.abc import Mapping + +# Direct-call sinks: the taint test runs over the SINK CALL's own arguments. +# ``None`` keeps the historical worst-of-all-args behavior for the original stdlib +# loaders; an ArgSpec narrows the test to the dangerous slot(s) for sinks whose +# non-data slots (flags, mmap modes, map_location) are taint-irrelevant. +_SINK_SPECS: dict[str, ArgSpec | None] = { + "pickle.loads": None, + "pickle.load": None, + "marshal.loads": None, + "marshal.load": None, + "yaml.load": None, + "yaml.load_all": None, + "yaml.unsafe_load": None, + "yaml.full_load": None, + "shelve.open": ArgSpec(positions=(0,), keywords=("filename",)), + "dill.load": ArgSpec(positions=(0,), keywords=("file",)), + "dill.loads": ArgSpec(positions=(0,), keywords=("str",)), + "jsonpickle.decode": ArgSpec(positions=(0,), keywords=("string",)), + "joblib.load": ArgSpec(positions=(0,), keywords=("filename",)), + "torch.load": ArgSpec(positions=(0,), keywords=("f",)), + "numpy.load": ArgSpec(positions=(0,), keywords=("file",)), +} + +# Stream-reader sinks: the sink is a no-arg METHOD on a reader object; the taint test +# runs over the CONSTRUCTOR call's stream argument (the ArgSpec addresses the ctor). +_READER_CTORS = frozenset({"pickle.Unpickler"}) +_READER_SINK_SPECS: dict[str, ArgSpec] = { + "pickle.Unpickler.load": ArgSpec(positions=(0,), keywords=("file",)), +} + +_SINKS = frozenset(_SINK_SPECS) | frozenset(_READER_SINK_SPECS) + + +def _has_literal_true_kw(call: ast.Call, name: str) -> bool: + """True iff *call* passes ``=True`` as a literal keyword. ``**kwargs``, + a non-constant value, or any constant other than ``True`` is not matched — + only the unambiguous, statically-visible case (the PY-WL-112 discipline).""" + return any(kw.arg == name and isinstance(kw.value, ast.Constant) and kw.value.value is True for kw in call.keywords) + METADATA = RuleMetadata( rule_id="PY-WL-106", base_severity=Severity.WARN, kind=Kind.DEFECT, multi_emit=True, - description="Untrusted data reaches a deserialization sink (pickle/marshal/yaml.load) in a trusted-tier function.", + description=( + "Untrusted data reaches a deserialization sink (pickle/Unpickler/marshal/yaml.load/shelve " + "+ curated third-party: dill/jsonpickle/joblib/torch.load/numpy.load(allow_pickle=True)) " + "in a trusted-tier function." + ), examples_violation=( "@external_boundary\ndef read_raw(p):\n return p\n" "@trusted(level='ASSURED')\ndef f(p):\n pickle.loads(read_raw(p))", + "@external_boundary\ndef read_raw(p):\n return p\n" + "@trusted(level='ASSURED')\ndef f(p):\n return pickle.Unpickler(read_raw(p)).load()", ), examples_clean=( "@external_boundary\ndef read_raw(p):\n return p\n" "@trust_boundary(to_level='ASSURED')\ndef validate(x):\n if not x:\n raise ValueError\n return x\n" "@trusted(level='ASSURED')\ndef f(p):\n blob = validate(read_raw(p))\n" " obj = pickle.loads(blob)\n return blob", + # numpy.load without allow_pickle=True is safe by default (no object unpickling). + "@external_boundary\ndef read_raw(p):\n return p\n" + "@trusted(level='ASSURED')\ndef f(p):\n return numpy.load(read_raw(p))", ), ) class UntrustedToDeserialization(TaintedSinkRule): + """Attribute-only since the 2026-06-10 consolidation, plus two hooks: + + * :meth:`_accept_call` — the numpy/torch literal-keyword gates; + * :meth:`_taint_anchor_call` — a reader-method sink (``Unpickler.load``, + argument-less) reads its taint from the CONSTRUCTOR call's stream + argument (the merged ``SINK_SPECS`` entry addresses the ctor's slots). + """ + rule_id = METADATA.rule_id metadata = METADATA SINKS = _SINKS + SINK_SPECS: Mapping[str, ArgSpec | None] = {**_SINK_SPECS, **_READER_SINK_SPECS} sink_label = "deserialization" + + def _accept_call(self, call: ast.Call, fqn: str) -> bool: # noqa: PLR6301 + if fqn == "numpy.load" and not _has_literal_true_kw(call, "allow_pickle"): + return False # safe-by-default: only a literal allow_pickle=True unpickles + return not (fqn == "torch.load" and _has_literal_true_kw(call, "weights_only")) + # weights_only=True → restricted unpickler, the modern safe spelling + + def _taint_anchor_call( # noqa: PLR6301 + self, + call: ast.Call, + fqn: str, + entity_node: ast.AST, + alias_map: Mapping[str, str], + ) -> ast.Call | None: + if fqn not in _READER_SINK_SPECS: + return call + # Reader-method sink: the dangerous data is the stream handed to the + # CONSTRUCTOR. Resolve the chained ``pickle.Unpickler(s).load()`` receiver + # or the bound var's recorded constructor; ``None`` (an annotation-only + # binding with no constructor in scope) is a documented bounded FN. + ctors = collect_ctor_call_nodes(entity_node, alias_map, ctor_fqns=_READER_CTORS) + return receiver_ctor_call(call, ctors) diff --git a/src/wardline/scanner/rules/untrusted_to_exec.py b/src/wardline/scanner/rules/untrusted_to_exec.py index 745079d5..7180b253 100644 --- a/src/wardline/scanner/rules/untrusted_to_exec.py +++ b/src/wardline/scanner/rules/untrusted_to_exec.py @@ -3,7 +3,23 @@ ``eval`` / ``exec`` / ``compile`` on untrusted input is arbitrary code execution (CWE-95). Tier-modulated; fires only where trust is declared. Matches the bare -builtins (``eval(x)``) as well as ``builtins.eval`` forms. +builtins (``eval(x)``), the ``builtins.eval`` forms, and the ``__builtins__.eval`` +spelling. The ``__builtins__`` form is real but narrow: in ``__main__`` +``__builtins__`` is the builtins MODULE (so ``.eval`` executes), while in an +imported module it is a plain dict (attribute access fails) — it is matched +because where it does run, it is full arbitrary-code-execution +(wardline-c83b40c73a). + +**Severity: WARN — a deliberate FP-economics call, not an oversight (severity +lattice review 2026-06-10).** Blast-radius alone would argue ERROR alongside +108/112/118/124, but those rules buy their ERROR with strong per-finding +evidence: a slot-precise ArgSpec, a literal ``shell=True``, or an SQL-string +position gate. This rule's sinks take ONE polymorphic payload argument tested +worst-of-all-args, ``compile`` is pervasive in legitimate metaprogramming, and +``eval``/``exec`` payloads are routinely pre-validated in ways the engine cannot +prove — so its evidence per finding is one class weaker, and its base sits one +class lower. Promote via ``rules.severity`` per project, or revisit alongside +the frozen identity corpus when the rule gains slot/shape evidence. """ from __future__ import annotations @@ -12,7 +28,19 @@ from wardline.scanner.rules._sink_helpers import TaintedSinkRule from wardline.scanner.rules.metadata import RuleMetadata -_SINKS = frozenset({"eval", "exec", "compile", "builtins.eval", "builtins.exec", "builtins.compile"}) +_SINKS = frozenset( + { + "eval", + "exec", + "compile", + "builtins.eval", + "builtins.exec", + "builtins.compile", + "__builtins__.eval", + "__builtins__.exec", + "__builtins__.compile", + } +) METADATA = RuleMetadata( rule_id="PY-WL-107", diff --git a/src/wardline/scanner/rules/untrusted_to_import.py b/src/wardline/scanner/rules/untrusted_to_import.py index c469e07b..68da46db 100644 --- a/src/wardline/scanner/rules/untrusted_to_import.py +++ b/src/wardline/scanner/rules/untrusted_to_import.py @@ -1,8 +1,14 @@ # src/wardline/scanner/rules/untrusted_to_import.py -"""PY-WL-115 — untrusted data reaches a dynamic import sink in a trusted-tier function. +"""PY-WL-115 — untrusted data reaches a dynamic code/module-load sink in a trusted-tier function. -Fires when raw-zone data reaches a dynamic module load sink (importlib.import_module -or __import__) inside a trusted-tier function. +Fires when raw-zone data reaches a dynamic module-load or file-execution sink inside a +trusted-tier function. The sink family covers the import-and-execute class (CWE-829 / +CWE-94): ``importlib.import_module`` and ``__import__`` (attacker-chosen module name), +``runpy.run_path`` / ``runpy.run_module`` (import-and-EXECUTE an attacker-controlled +file path / module — blast radius equivalent to ``exec``), and +``importlib.util.spec_from_file_location`` (a tainted file-path arg builds a loader for +attacker-chosen code). Expanded from the original two-sink charter +(importlib.import_module / __import__) per wardline-c83b40c73a. """ from __future__ import annotations @@ -11,7 +17,15 @@ from wardline.scanner.rules._sink_helpers import TaintedSinkRule from wardline.scanner.rules.metadata import RuleMetadata -_SINKS = frozenset({"importlib.import_module", "__import__"}) +_SINKS = frozenset( + { + "importlib.import_module", + "__import__", + "runpy.run_path", + "runpy.run_module", + "importlib.util.spec_from_file_location", + } +) METADATA = RuleMetadata( rule_id="PY-WL-115", @@ -19,12 +33,15 @@ kind=Kind.DEFECT, multi_emit=True, description=( - "Untrusted data reaches a dynamic import sink (importlib.import_module / " - "__import__) in a trusted-tier function." + "Untrusted data reaches a dynamic code/module-load sink (importlib.import_module / " + "__import__ / runpy.run_path / runpy.run_module / " + "importlib.util.spec_from_file_location) in a trusted-tier function." ), examples_violation=( "@external_boundary\ndef read_raw(p):\n return p\n" "@trusted(level='ASSURED')\ndef f(p):\n importlib.import_module(read_raw(p))", + "@external_boundary\ndef read_raw(p):\n return p\n" + "@trusted(level='ASSURED')\ndef f(p):\n runpy.run_path(read_raw(p))", ), examples_clean=("@trusted(level='ASSURED')\ndef f(p):\n importlib.import_module('sys')",), ) diff --git a/src/wardline/scanner/rules/untrusted_to_log.py b/src/wardline/scanner/rules/untrusted_to_log.py new file mode 100644 index 00000000..f9825b91 --- /dev/null +++ b/src/wardline/scanner/rules/untrusted_to_log.py @@ -0,0 +1,74 @@ +# src/wardline/scanner/rules/untrusted_to_log.py +"""PY-WL-125 — untrusted data as the log MESSAGE format string (CWE-117). + +Charter: log injection / log forging — a tainted value used as the message +FORMAT string of ``logging.debug/info/warning/error/critical/exception`` (the +module-level functions or the Logger-method form via the construct-then-method +machinery: ``logger = logging.getLogger(...); logger.info(raw)``) inside a +trusted-tier function. Newline-spoofed entries forge audit lines and seed +log-viewer XSS downstream. + +Only the message slot (position 0 / ``msg``) is dangerous. Tainted data in the +lazy ``%``-args parameters (``logging.info('user=%s', raw)``) is logging's OWN +parameterization — the canonical safe idiom — and must NOT fire; flagging it +would be an FP factory. ``logging.log(level, msg)`` is deliberately out of +scope for v1 (its message sits at position 1; charter names the fixed-level +methods only). + +Severity calibration: INFO (below the task ceiling of WARN). CWE-117 is a +recognised weakness class but is high-noise by nature — almost every service +logs request-derived data somewhere — and its blast radius is forgery/foothold, +not execution. INFO keeps the finding visible to agents (and to an explicit +``--fail-on INFO`` gate) without ever tripping the default gate; peer tools +(bandit, CodeQL) rate the class LOW for the same reason. +""" + +from __future__ import annotations + +from wardline.core.finding import Kind, Maturity, Severity +from wardline.scanner.rules._sink_helpers import ArgSpec, TaintedSinkRule +from wardline.scanner.rules.metadata import RuleMetadata + +_METHODS = ("debug", "info", "warning", "error", "critical", "exception") +_MSG_SPEC = ArgSpec(positions=(0,), keywords=("msg",)) + +# Module-level functions + the Logger-method form. The binding machinery records +# a constructor FQN without verifying it names a class, so ``logging.getLogger.info`` +# is the canonical name a ``logger = logging.getLogger(...)`` method call resolves +# to; ``logging.Logger.info`` covers the ``log: logging.Logger`` annotation form. +_SINK_SPECS: dict[str, ArgSpec | None] = { + f"{prefix}.{method}": _MSG_SPEC + for prefix in ("logging", "logging.getLogger", "logging.Logger") + for method in _METHODS +} + +METADATA = RuleMetadata( + rule_id="PY-WL-125", + base_severity=Severity.INFO, + kind=Kind.DEFECT, + multi_emit=True, + description=( + "Untrusted data is used as the log MESSAGE format string " + "(logging.* / Logger methods) in a trusted-tier function (log injection)." + ), + examples_violation=( + "import logging\n" + "@external_boundary\ndef read_raw(p):\n return p\n" + "@trusted(level='ASSURED')\ndef f(p):\n logging.info(read_raw(p))\n return 1", + ), + examples_clean=( + # Lazy %-parameterization is logging's own safe idiom — never a finding. + "import logging\n" + "@external_boundary\ndef read_raw(p):\n return p\n" + "@trusted(level='ASSURED')\ndef f(p):\n logging.info('user input = %s', read_raw(p))\n return 1", + ), + maturity=Maturity.PREVIEW, +) + + +class UntrustedToLog(TaintedSinkRule): + rule_id = METADATA.rule_id + metadata = METADATA + SINKS = frozenset(_SINK_SPECS) + SINK_SPECS = _SINK_SPECS + sink_label = "log-message" diff --git a/src/wardline/scanner/rules/untrusted_to_mail.py b/src/wardline/scanner/rules/untrusted_to_mail.py new file mode 100644 index 00000000..926ec4b4 --- /dev/null +++ b/src/wardline/scanner/rules/untrusted_to_mail.py @@ -0,0 +1,70 @@ +# src/wardline/scanner/rules/untrusted_to_mail.py +"""PY-WL-126 — untrusted recipient/message reaches SMTP.sendmail (CWE-93). + +Charter: mail (CRLF/header) injection — tainted data in the ``to_addrs`` +(position 1) or ``msg`` (position 2) argument of ``smtplib.SMTP.sendmail`` / +``smtplib.SMTP_SSL.sendmail`` inside a trusted-tier function. The receiver is +matched through the construct-then-method machinery (``s = smtplib.SMTP(h)``, +``with smtplib.SMTP(h) as s``, ``s: smtplib.SMTP``, the chained form). +Newlines in a recipient or message inject spoofed headers / BCC recipients. + +Calibration: ``from_addr`` (position 0) is deliberately NOT a dangerous slot in +v1 — the recipient set and message body are the canonical CWE-93 injection +surfaces the gap report names; widening to the envelope sender can ride a later +calibration pass. ``send_message`` is out of scope (it takes an +``email.message.Message``, whose header serialization already rejects bare +newlines on supported Pythons). + +Severity: WARN. Real injection, but bounded blast radius (spam/spoofing, not +code execution) and gated on a constructed SMTP client — the mass-assignment +class, not the RCE class. +""" + +from __future__ import annotations + +from wardline.core.finding import Kind, Maturity, Severity +from wardline.scanner.rules._sink_helpers import ArgSpec, TaintedSinkRule +from wardline.scanner.rules.metadata import RuleMetadata + +# sendmail(from_addr, to_addrs, msg, ...) — recipient + message are the slots. +_SENDMAIL_SPEC = ArgSpec(positions=(1, 2), keywords=("to_addrs", "msg")) + +_SINK_SPECS: dict[str, ArgSpec | None] = { + "smtplib.SMTP.sendmail": _SENDMAIL_SPEC, + "smtplib.SMTP_SSL.sendmail": _SENDMAIL_SPEC, +} + +METADATA = RuleMetadata( + rule_id="PY-WL-126", + base_severity=Severity.WARN, + kind=Kind.DEFECT, + multi_emit=True, + description=( + "Untrusted data reaches the recipient/message of smtplib SMTP.sendmail " + "in a trusted-tier function (mail/header injection)." + ), + examples_violation=( + "import smtplib\n" + "@external_boundary\ndef read_raw(p):\n return p\n" + "@trusted(level='ASSURED')\ndef f(p):\n" + " s = smtplib.SMTP('localhost')\n" + " s.sendmail('from@example.com', 'to@example.com', read_raw(p))\n" + " return 1", + ), + examples_clean=( + "import smtplib\n" + "@trusted(level='ASSURED')\ndef f():\n" + " s = smtplib.SMTP('localhost')\n" + " s.sendmail('from@example.com', 'to@example.com', 'body')\n" + " return 1", + ), + maturity=Maturity.PREVIEW, +) + + +class UntrustedToMail(TaintedSinkRule): + rule_id = METADATA.rule_id + metadata = METADATA + SINKS = frozenset(_SINK_SPECS) + SINK_SPECS = _SINK_SPECS + sink_label = "SMTP-send" diff --git a/src/wardline/scanner/rules/untrusted_to_native.py b/src/wardline/scanner/rules/untrusted_to_native.py new file mode 100644 index 00000000..1746c42d --- /dev/null +++ b/src/wardline/scanner/rules/untrusted_to_native.py @@ -0,0 +1,63 @@ +# src/wardline/scanner/rules/untrusted_to_native.py +"""PY-WL-124 — untrusted path reaches a native-library load sink (CWE-114). + +Charter: a tainted library path/name reaching ``ctypes.CDLL`` / +``ctypes.WinDLL`` / ``ctypes.OleDLL`` / ``ctypes.PyDLL`` or +``ctypes.cdll.LoadLibrary`` inside a trusted-tier function. Loading an +attacker-controlled shared object is arbitrary NATIVE code execution +(CWE-114 process control / CWE-829 untrusted functionality). Tier-modulated; +fires only where trust is declared. + +Severity: ERROR. Same blast radius as the command/program-execution family +(PY-WL-108) and SQLi (PY-WL-118) — full process compromise with no further +preconditions — so the same ERROR base. + +Only the library NAME/PATH slot is dangerous (``ArgSpec`` slot precision, review +2026-06-10): a tainted ``mode=`` / ``use_errno=`` flag with a constant library +name is not a native-load injection and must not fire. +""" + +from __future__ import annotations + +from wardline.core.finding import Kind, Maturity, Severity +from wardline.scanner.rules._sink_helpers import ArgSpec, TaintedSinkRule +from wardline.scanner.rules.metadata import RuleMetadata + +# The library path/name is the first positional / the ``name`` keyword. +_NAME_SPEC = ArgSpec(positions=(0,), keywords=("name",)) + +_SINK_SPECS: dict[str, ArgSpec | None] = { + "ctypes.CDLL": _NAME_SPEC, + "ctypes.WinDLL": _NAME_SPEC, + "ctypes.OleDLL": _NAME_SPEC, + "ctypes.PyDLL": _NAME_SPEC, + "ctypes.cdll.LoadLibrary": ArgSpec(positions=(0,)), +} + +_SINKS = frozenset(_SINK_SPECS) + +METADATA = RuleMetadata( + rule_id="PY-WL-124", + base_severity=Severity.ERROR, + kind=Kind.DEFECT, + multi_emit=True, + description=( + "Untrusted data reaches a native-library load sink (ctypes.CDLL/WinDLL/OleDLL/PyDLL, " + "ctypes.cdll.LoadLibrary) in a trusted-tier function." + ), + examples_violation=( + "import ctypes\n" + "@external_boundary\ndef read_raw(p):\n return p\n" + "@trusted(level='ASSURED')\ndef f(p):\n return ctypes.CDLL(read_raw(p))", + ), + examples_clean=("import ctypes\n@trusted(level='ASSURED')\ndef f():\n ctypes.CDLL('libm.so.6')",), + maturity=Maturity.PREVIEW, +) + + +class UntrustedToNative(TaintedSinkRule): + rule_id = METADATA.rule_id + metadata = METADATA + SINKS = _SINKS + SINK_SPECS = _SINK_SPECS + sink_label = "native-library-load" diff --git a/src/wardline/scanner/rules/untrusted_to_reflection.py b/src/wardline/scanner/rules/untrusted_to_reflection.py new file mode 100644 index 00000000..28821a0b --- /dev/null +++ b/src/wardline/scanner/rules/untrusted_to_reflection.py @@ -0,0 +1,66 @@ +# src/wardline/scanner/rules/untrusted_to_reflection.py +"""PY-WL-123 — tainted attribute NAME reaches setattr/getattr (CWE-915). + +Charter: dynamic attribute injection — an untrusted NAME argument (position 1) +to the builtin ``setattr``/``getattr`` inside a trusted-tier function lets an +attacker pick which attribute is written/read (mass assignment, e.g. reaching +``__class__``-adjacent state). Tier-modulated; fires only where trust is +declared. + +Only the NAME slot is dangerous: an untrusted VALUE assigned to a fixed +attribute (``setattr(obj, 'name', raw)``), a tainted ``getattr`` default, or a +tainted receiver are ordinary data flow, not attribute injection — the +arg-position-aware :class:`ArgSpec` keeps them silent. + +Severity: WARN. Exploitation depends on the target object's shape (a +mass-assignment VECTOR, not direct code execution), so it sits below the +unconditional-RCE ERROR class (108/112/118/124). Note the WARN co-residents +106/107 are there for a DIFFERENT reason — they ARE direct-RCE sinks held at +WARN on FP economics (weaker worst-of-all-args evidence; see their module +docstrings) — so this rule does not cite them as a "non-RCE WARN convention". +""" + +from __future__ import annotations + +from wardline.core.finding import Kind, Maturity, Severity +from wardline.scanner.rules._sink_helpers import ArgSpec, TaintedSinkRule +from wardline.scanner.rules.metadata import RuleMetadata + +# setattr/getattr are positional-only builtins — the NAME is position 1, always. +_NAME_SPEC = ArgSpec(positions=(1,)) + +_SINK_SPECS: dict[str, ArgSpec | None] = { + "setattr": _NAME_SPEC, + "getattr": _NAME_SPEC, + "builtins.setattr": _NAME_SPEC, + "builtins.getattr": _NAME_SPEC, +} + +METADATA = RuleMetadata( + rule_id="PY-WL-123", + base_severity=Severity.WARN, + kind=Kind.DEFECT, + multi_emit=True, + description=( + "Untrusted data is used as the attribute NAME in setattr/getattr in a trusted-tier function " + "(dynamic attribute injection / mass assignment)." + ), + examples_violation=( + "@external_boundary\ndef read_raw(p):\n return p\n" + "@trusted(level='ASSURED')\ndef f(p, obj):\n setattr(obj, read_raw(p), 1)\n return 1", + ), + examples_clean=( + # Fixed attribute name: the untrusted VALUE slot is not the injection vector. + "@external_boundary\ndef read_raw(p):\n return p\n" + "@trusted(level='ASSURED')\ndef f(p, obj):\n setattr(obj, 'name', read_raw(p))\n return 1", + ), + maturity=Maturity.PREVIEW, +) + + +class UntrustedToReflection(TaintedSinkRule): + rule_id = METADATA.rule_id + metadata = METADATA + SINKS = frozenset(_SINK_SPECS) + SINK_SPECS = _SINK_SPECS + sink_label = "attribute-reflection" diff --git a/src/wardline/scanner/rules/untrusted_to_shell_subprocess.py b/src/wardline/scanner/rules/untrusted_to_shell_subprocess.py index c8ffd4fb..c20e5383 100644 --- a/src/wardline/scanner/rules/untrusted_to_shell_subprocess.py +++ b/src/wardline/scanner/rules/untrusted_to_shell_subprocess.py @@ -20,6 +20,11 @@ Tier-modulated and trusted-tier-gated exactly like 106/107/108 (silent in the undecorated developer-freedom zone, speaking only where trust is declared). + +Sink matching is BINDING-AWARE (the consolidated :class:`TaintedSinkRule` base): +a function-local callable alias — ``runner = subprocess.run; runner(raw, +shell=True)`` — resolves to the sink FQN and fires (wardline-13cfdd7b31). The +literal ``shell=True`` gate applies to binding-resolved calls identically. """ from __future__ import annotations @@ -55,7 +60,9 @@ def _has_literal_shell_true(call: ast.Call) -> bool: METADATA = RuleMetadata( rule_id="PY-WL-112", - base_severity=Severity.WARN, # matches the 108 OS-command family; ERROR is defensible + # Calibrated with 108 and PY-WL-118 (SQLi): tainted shell=True command + # execution is the same exploit class (CWE-78 ≅ CWE-89 in blast radius). + base_severity=Severity.ERROR, kind=Kind.DEFECT, multi_emit=True, description=( @@ -81,7 +88,7 @@ class UntrustedToShellSubprocess(TaintedSinkRule): SINKS = _SINKS sink_label = "shell=True subprocess" - def _accept_call(self, call: ast.Call) -> bool: # noqa: PLR6301 + def _accept_call(self, call: ast.Call, fqn: str) -> bool: # noqa: ARG002, PLR6301 """Extra per-call gate beyond the SINK-name match: require literal shell=True so the safe argv-list default (shell=False) never trips this family.""" return _has_literal_shell_true(call) diff --git a/src/wardline/scanner/rules/untrusted_to_template.py b/src/wardline/scanner/rules/untrusted_to_template.py new file mode 100644 index 00000000..e4cc8c14 --- /dev/null +++ b/src/wardline/scanner/rules/untrusted_to_template.py @@ -0,0 +1,66 @@ +# src/wardline/scanner/rules/untrusted_to_template.py +"""PY-WL-122 — untrusted data compiled into a server-side template (SSTI, CWE-1336). + +Charter: a tainted string reaching a template COMPILATION sink +(``jinja2.Template``, ``jinja2.Environment.from_string`` — including the +construct-then-method form — and ``mako.template.Template``) inside a +trusted-tier function. Tier-modulated; fires only where trust is declared. + +Only the template SOURCE slot is dangerous: tainted data passed as a render +variable (``Template('{{ x }}').render(x=raw)``) is the safe idiom and must not +fire — autoescaping/render-time substitution is exactly the mitigation. Loading +a template BY NAME (``env.get_template(raw)``) is not SSTI either. + +Severity: ERROR. SSTI in Jinja2/Mako is RCE-adjacent (``{{ ''.__class__ ... }}`` +sandbox escapes are a documented exploitation primitive), the same blast-radius +class as the PY-WL-118/108 ERROR family. +""" + +from __future__ import annotations + +from wardline.core.finding import Kind, Maturity, Severity +from wardline.scanner.rules._sink_helpers import ArgSpec, TaintedSinkRule +from wardline.scanner.rules.metadata import RuleMetadata + +# Only the template-source slot; render context / loader names are not SSTI. +_SINK_SPECS: dict[str, ArgSpec | None] = { + "jinja2.Template": ArgSpec(positions=(0,), keywords=("source",)), + "jinja2.Environment.from_string": ArgSpec(positions=(0,), keywords=("source",)), + "mako.template.Template": ArgSpec(positions=(0,), keywords=("text",)), +} + +METADATA = RuleMetadata( + rule_id="PY-WL-122", + base_severity=Severity.ERROR, + kind=Kind.DEFECT, + multi_emit=True, + description=( + "Untrusted data is compiled into a server-side template " + "(jinja2.Template/Environment.from_string, mako Template) in a trusted-tier function (SSTI)." + ), + examples_violation=( + "import jinja2\n" + "@external_boundary\ndef read_raw(p):\n return p\n" + "@trusted(level='ASSURED')\ndef f(p):\n return jinja2.Template(read_raw(p)).render()", + "import jinja2\n" + "@external_boundary\ndef read_raw(p):\n return p\n" + "@trusted(level='ASSURED')\ndef f(p):\n" + " env = jinja2.Environment()\n return env.from_string(read_raw(p))", + ), + examples_clean=( + # Tainted data as a RENDER variable is the safe idiom — only a tainted SOURCE is SSTI. + "import jinja2\n" + "@external_boundary\ndef read_raw(p):\n return p\n" + "@trusted(level='ASSURED')\ndef f(p):\n" + " jinja2.Template('Hello {{ name }}').render(name=read_raw(p))", + ), + maturity=Maturity.PREVIEW, +) + + +class UntrustedToTemplate(TaintedSinkRule): + rule_id = METADATA.rule_id + metadata = METADATA + SINKS = frozenset(_SINK_SPECS) + SINK_SPECS = _SINK_SPECS + sink_label = "template-compilation" diff --git a/src/wardline/scanner/rules/untrusted_to_xml.py b/src/wardline/scanner/rules/untrusted_to_xml.py new file mode 100644 index 00000000..2ff54b72 --- /dev/null +++ b/src/wardline/scanner/rules/untrusted_to_xml.py @@ -0,0 +1,87 @@ +# src/wardline/scanner/rules/untrusted_to_xml.py +"""PY-WL-121 — untrusted data reaches an XML parsing sink (XXE family, CWE-611). + +Charter: a tainted document/stream reaching an XML parser inside a trusted-tier +function. Tier-modulated; fires only where trust is declared. Only the DOCUMENT +slot (position 0 / its keyword spelling) is dangerous — taint in a ``parser=`` +or handler slot is not XXE. + +Severity is PER-SINK, calibrated to each parser's *default* posture (verified on +the project interpreter, 2026-06-10 scrub): + +* ``lxml.etree.*`` — **ERROR**: resolves external entities by default + (``resolve_entities=True``), so tainted XML is genuine XXE (local file + disclosure / SSRF). +* stdlib ``xml.etree.ElementTree`` / ``xml.dom.minidom`` / ``xml.sax`` — + **WARN**: external general entities have been disabled by default since + CPython 3.7.1 (``ET.fromstring`` raises on an external-entity payload), so + the default-on residual risk is the billion-laughs internal-entity-expansion + DoS shared by every pyexpat-based parser. NOT the ERROR class the gap report + originally claimed — the "stdlib resolves external entities by default" + premise was disproven by the verifier; all three stdlib families share the + same DoS-only default risk, hence the same WARN. + +``defusedxml`` is the blessed remediation and is deliberately not a sink. + +The 121…126 preview family's binding- and arg-slot-aware check machinery used to +live here as ``SpecSinkCheckMixin``; it graduated into the consolidated +:class:`TaintedSinkRule` base (``SINK_SPECS`` / ``SINK_SEVERITIES``, review +2026-06-10), so this module is now an attribute-only rule like its siblings. +""" + +from __future__ import annotations + +from wardline.core.finding import Kind, Maturity, Severity +from wardline.scanner.rules._sink_helpers import ArgSpec, TaintedSinkRule +from wardline.scanner.rules.metadata import RuleMetadata + +# Only the DOCUMENT slot is dangerous; ``parser=``/handler slots are not XXE. +_SINK_SPECS: dict[str, ArgSpec | None] = { + "xml.etree.ElementTree.fromstring": ArgSpec(positions=(0,), keywords=("text",)), + "xml.etree.ElementTree.parse": ArgSpec(positions=(0,), keywords=("source",)), + "xml.etree.ElementTree.XML": ArgSpec(positions=(0,), keywords=("text",)), + "xml.etree.ElementTree.iterparse": ArgSpec(positions=(0,), keywords=("source",)), + "xml.dom.minidom.parse": ArgSpec(positions=(0,), keywords=("file",)), + "xml.dom.minidom.parseString": ArgSpec(positions=(0,), keywords=("string",)), + "xml.sax.parse": ArgSpec(positions=(0,), keywords=("source",)), + "xml.sax.parseString": ArgSpec(positions=(0,), keywords=("string",)), + "lxml.etree.fromstring": ArgSpec(positions=(0,), keywords=("text",)), + "lxml.etree.parse": ArgSpec(positions=(0,), keywords=("source",)), + "lxml.etree.XML": ArgSpec(positions=(0,), keywords=("text",)), +} + +# Per-sink calibration (see module docstring): stdlib = billion-laughs DoS only +# since 3.7.1 → WARN; lxml = entity-resolving by default → the ERROR base. +_STDLIB_SEVERITIES: dict[str, Severity] = {fqn: Severity.WARN for fqn in _SINK_SPECS if fqn.startswith("xml.")} + +METADATA = RuleMetadata( + rule_id="PY-WL-121", + base_severity=Severity.ERROR, + kind=Kind.DEFECT, + multi_emit=True, + description=( + "Untrusted data reaches an XML parsing sink (XXE/billion-laughs: lxml.etree at ERROR, " + "stdlib etree/minidom/sax at WARN) in a trusted-tier function." + ), + examples_violation=( + "from lxml import etree\n" + "@external_boundary\ndef read_raw(p):\n return p\n" + "@trusted(level='ASSURED')\ndef f(p):\n return etree.fromstring(read_raw(p))", + "import xml.etree.ElementTree as ET\n" + "@external_boundary\ndef read_raw(p):\n return p\n" + "@trusted(level='ASSURED')\ndef f(p):\n return ET.fromstring(read_raw(p))", + ), + examples_clean=( + "import xml.etree.ElementTree as ET\n@trusted(level='ASSURED')\ndef f():\n ET.fromstring('')", + ), + maturity=Maturity.PREVIEW, +) + + +class UntrustedToXML(TaintedSinkRule): + rule_id = METADATA.rule_id + metadata = METADATA + SINKS = frozenset(_SINK_SPECS) + SINK_SPECS = _SINK_SPECS + SINK_SEVERITIES = _STDLIB_SEVERITIES + sink_label = "XML-parsing" diff --git a/src/wardline/scanner/taint/callgraph.py b/src/wardline/scanner/taint/callgraph.py index b13e2762..38c2908f 100644 --- a/src/wardline/scanner/taint/callgraph.py +++ b/src/wardline/scanner/taint/callgraph.py @@ -68,7 +68,7 @@ def _candidate_receiver_classes( alias_map: dict[str, str], module_prefix: str, class_qualnames: frozenset[str], - project_fqns: frozenset[str], + known_fqns: frozenset[str], ) -> dict[int, frozenset[str]]: """Flow-sensitive reaching-definitions pass over *func*'s own scope. @@ -81,7 +81,6 @@ def _candidate_receiver_classes( (wardline-499c22bbdd). Mirrors the merge discipline of ``variable_level``'s taint walk. """ candidates_at_call: dict[int, frozenset[str]] = {} - known_fqns = project_fqns | class_qualnames # resolve_call_fqn resolves constructors here def resolve_class(value: ast.expr | None, env: dict[str, set[str]]) -> set[str]: if isinstance(value, ast.NamedExpr): # ``(x := expr)`` evaluates to expr @@ -254,6 +253,9 @@ def build_call_edges( call_site_implicit_receivers: dict[int, str] = {} call_site_candidate_callees: dict[int, frozenset[str]] = {} entity_by_fqn = {entity.qualname: entity for entity in entities} + # Hoisted out of _candidate_receiver_classes: the union is O(project) and was + # rebuilt once PER FUNCTION, an O(n^2) whole-scan term on large trees. + known_fqns = project_fqns | class_qualnames # resolve_call_fqn resolves constructors here def _decorator_name(decorator: ast.expr) -> str | None: if isinstance(decorator, ast.Call): @@ -296,7 +298,7 @@ def _resolve_classmethod_call(call: ast.Call) -> str | None: alias_map=alias_map, module_prefix=module_prefix, class_qualnames=class_qualnames, - project_fqns=project_fqns, + known_fqns=known_fqns, ) callees: set[str] = set() diff --git a/src/wardline/scanner/taint/module_summariser.py b/src/wardline/scanner/taint/module_summariser.py index 5a4c6543..ae378f76 100644 --- a/src/wardline/scanner/taint/module_summariser.py +++ b/src/wardline/scanner/taint/module_summariser.py @@ -1,5 +1,5 @@ # src/wardline/scanner/taint/module_summariser.py -"""Per-module FunctionSummary emission. +"""Per-module FunctionSummary emission + module-global taint seeds. Maps each L1 ``FunctionSeed`` + the callgraph's unresolved-call count into a ``FunctionSummary``. The seed's 2-valued source (``provider``/``default``) maps @@ -8,12 +8,22 @@ provider expresses a module-wide default yet); SP2's richer provider populates it. The cache key is computed once and shared by all functions in the module (module-granular invalidation). + +Also hosts the MODULE-GLOBAL TAINT CHANNEL's two collection helpers +(wardline-66b2c91470): :func:`collect_module_global_raw_seeds` (the read +direction's import-time seeds) and :func:`own_scope_global_names` (the write +direction's ``global g`` declarations). The analyzer threads both into the L2 +walk — see ``analyzer._with_module_global_params`` for how the seeds enter a +function's variable map. """ from __future__ import annotations +import ast from collections.abc import Mapping +from wardline.core.taints import RAW_ZONE, TaintState +from wardline.scanner.ast_primitives import resolve_call_fqn from wardline.scanner.taint.function_level import FunctionSeed from wardline.scanner.taint.summary import ( SUMMARY_SCHEMA_VERSION, @@ -61,3 +71,92 @@ def summarise_module( ) ) return tuple(summaries) + + +def collect_module_global_raw_seeds( + tree: ast.Module, + *, + module: str, + alias_map: Mapping[str, str], + return_taints: Mapping[str, TaintState], + local_fqns: frozenset[str], + untrusted_sources: frozenset[str] = frozenset(), +) -> dict[str, TaintState]: + """Module-level simple names assigned RAW at import time → ``{name: taint}``. + + The READ direction of the module-global taint channel (wardline-66b2c91470): + a module-level ``RAW = read_raw(...)`` — a direct call whose FQN resolves + (through the import alias map or as a module-local function) to either + + * a ``config.untrusted_sources`` entry (seeded ``EXTERNAL_RAW``), or + * a project function whose RESOLVED RETURN TAINT is in ``RAW_ZONE`` + (an ``@external_boundary`` producer — its return taint, ``EXTERNAL_RAW``, + is the seed), + + marks ``RAW`` as a raw module global. The analyzer then presents these names + to every function's L2 walk as implicit raw parameters. + + Scope (documented v1 approximation): only DIRECT top-level statements are + considered (no descent into module-level ``if``/``try`` bodies), single-Name + ``=``/annotated-``=`` targets only, LAST-BINDING-WINS in source order — a + later rebind whose RHS is not a resolvable raw call CLEARS the name (the + same discipline as ``collect_sink_bindings``). This under-approximates + (bounded FN on conditional module init), never over-approximates a clean + module constant into a raw seed. + """ + aliases = dict(alias_map) + + def raw_taint_of(value: ast.expr) -> TaintState | None: + if not isinstance(value, ast.Call): + return None + fqn = resolve_call_fqn(value, aliases, local_fqns, module) + if fqn is None: + return None + if fqn in untrusted_sources: + return TaintState.EXTERNAL_RAW + taint = return_taints.get(fqn) + return taint if taint is not None and taint in RAW_ZONE else None + + seeds: dict[str, TaintState] = {} + for stmt in tree.body: + if isinstance(stmt, ast.Assign): + if len(stmt.targets) == 1 and isinstance(stmt.targets[0], ast.Name): + taint = raw_taint_of(stmt.value) + if taint is not None: + seeds[stmt.targets[0].id] = taint + else: + seeds.pop(stmt.targets[0].id, None) + else: # tuple/multi-target rebind — clear every touched name + for target in stmt.targets: + for sub in ast.walk(target): + if isinstance(sub, ast.Name): + seeds.pop(sub.id, None) + elif isinstance(stmt, ast.AnnAssign) and isinstance(stmt.target, ast.Name): + taint = raw_taint_of(stmt.value) if stmt.value is not None else None + if taint is not None: + seeds[stmt.target.id] = taint + else: + seeds.pop(stmt.target.id, None) + return seeds + + +def own_scope_global_names(func_node: ast.FunctionDef | ast.AsyncFunctionDef) -> frozenset[str]: + """Names a function declares ``global`` in its OWN scope. + + The WRITE direction of the module-global taint channel: the analyzer reads + each declared name's final L2 variable taint as the function's write to the + module global. Nested def/class bodies are skipped — they are their own + entities and carry their own ``global`` declarations. + """ + names: set[str] = set() + + def visit(node: ast.AST) -> None: + for child in ast.iter_child_nodes(node): + if isinstance(child, (ast.FunctionDef, ast.AsyncFunctionDef, ast.ClassDef)): + continue + if isinstance(child, ast.Global): + names.update(child.names) + visit(child) + + visit(func_node) + return frozenset(names) diff --git a/src/wardline/scanner/taint/variable_level.py b/src/wardline/scanner/taint/variable_level.py index 4e75cbb8..0b3e88c7 100644 --- a/src/wardline/scanner/taint/variable_level.py +++ b/src/wardline/scanner/taint/variable_level.py @@ -3,8 +3,11 @@ Given a function AST node and its Level 1 (function-level) taint, walks the body tracking taint per variable through assignments, control-flow joins, and call -sites. Pure (returns a new dict); conservative (unknown expressions inherit the -function's L1 taint). Both expression/value combiners (BinOp, IfExp, BoolOp, +sites. Pure (returns a new dict); conservative (an unknown NON-CALL expression +inherits the function's L1 taint; an unresolved bare-name CALL propagates the +worst of the caller seed and its argument taints, so a trusted seed cannot +launder a raw argument through an unmodeled callee). Both expression/value +combiners (BinOp, IfExp, BoolOp, containers, ``.get`` defaults, ``+=``, container writes) AND control-flow MERGES (if/else, loop back-edges, match arms, try/except handlers) use the rank-meet ``least_trusted`` (weakest-link): at a merge a variable holds the value of ONE @@ -23,6 +26,7 @@ import ast import contextvars +from contextlib import contextmanager from dataclasses import dataclass from typing import TYPE_CHECKING @@ -64,20 +68,67 @@ } ) -# Curated taint-PROPAGATING builtins. A small explicit table (NOT a general rule -# — bare unknown calls still fall back to function_taint, so len/int/validate stay -# unaffected and there is no false-positive explosion). These return the join of -# their argument taints: a string conversion / iterator advance carries whatever -# taint went in. ``next`` closes the ``next(genexp)`` shape (the iterator arg). -_PROPAGATING_BUILTINS: frozenset[str] = frozenset({"str", "repr", "ascii", "bytes", "bytearray", "format", "next"}) +# Curated taint-PROPAGATING builtins. These return the join of their argument +# taints (no args → INTEGRAL): a string/container conversion or iterator advance +# carries whatever taint went in. ``next`` closes the ``next(genexp)`` shape (the +# iterator arg); the container conversions (list/tuple/set/frozenset/dict/sorted/ +# reversed) return a container holding the SAME untrusted elements, so omitting +# them laundered ``subprocess.run(sorted(raw))`` — the idiomatic argv shape. +_PROPAGATING_BUILTINS: frozenset[str] = frozenset( + { + "str", + "repr", + "ascii", + "bytes", + "bytearray", + "format", + "next", + "list", + "tuple", + "set", + "frozenset", + "dict", + "sorted", + "reversed", + } +) + +# Curated NON-propagating builtins: validators/measurers whose result is derived +# FROM the data but does not carry it (a length, a parsed number, a predicate). +# These keep the caller-seed fallback; every OTHER unresolved bare-name call +# propagates the worst of (caller seed, argument taints) — an unknown callee +# cannot be assumed to clean a raw argument (see _resolve_call's bare-name +# fallback; matches the imported-but-unmodeled path's conservatism). +_NON_PROPAGATING_BUILTINS: frozenset[str] = frozenset( + { + "len", + "int", + "float", + "bool", + "complex", + "ord", + "hash", + "id", + "abs", + "round", + "isinstance", + "issubclass", + "callable", + "hasattr", + "any", + "all", + } +) -# Curated taint-PROPAGATING methods, keyed by attribute name. ``.format``/``.join`` -# combine the receiver with the arguments (``"sep".join(parts)`` carries both); +# Curated taint-PROPAGATING methods, keyed by attribute name. ``.format``/ +# ``.format_map``/``.join`` combine the receiver with the arguments +# (``"sep".join(parts)`` carries both; ``.format_map(mapping)`` embeds the +# mapping's values exactly like ``.format``); # ``.get``/``.pop``/``.setdefault`` carry the RECEIVER's taint — a container access # is the same shape as the ``Subscript`` read handler (``d['k']`` propagating while # ``d.get('k')`` did not was an inconsistency, not a feature). These do NOT add a # new SOURCE: ``.get`` returns the container's existing taint, nothing more. -_PROPAGATING_METHODS_WITH_ARGS: frozenset[str] = frozenset({"format", "join"}) +_PROPAGATING_METHODS_WITH_ARGS: frozenset[str] = frozenset({"format", "format_map", "join"}) _PROPAGATING_METHODS_RECEIVER: frozenset[str] = frozenset({"get", "pop", "setdefault"}) # Curated DB-API storage-read methods. A ``cursor.fetchone/fetchall/fetchmany()`` loads @@ -110,10 +161,40 @@ contextvars.ContextVar("_CURRENT_CALL_SITE_ARG_TAINTS", default=None) ) -_CURRENT_VAR_TYPES: contextvars.ContextVar[dict[str, str] | None] = contextvars.ContextVar( +# Maps a local name to the CANDIDATE SET of class/type FQNs it MAY hold. A single +# slot per name lost a receiver class rebound in one branch arm only: the write +# ``box.token = raw`` after ``box = Vault(); if flag: box = Ledger()`` attributed +# only to Ledger while on the no-flag path the receiver is still the Vault +# instance (wardline-b369c7d06c). Straight-line rebind is a STRONG update (the +# set is replaced); a branch join UNIONS the arms' sets. ``list`` not ``set`` for +# deterministic iteration order (mirrors _CURRENT_LAMBDA_BINDINGS). +_CURRENT_VAR_TYPES: contextvars.ContextVar[dict[str, list[str]] | None] = contextvars.ContextVar( "_CURRENT_VAR_TYPES", default=None ) +# Side channel for attribute-write recording DURING the main statement walk — +# ``{receiver_key: {attr: least_trusted write taint}}`` where receiver_key is a +# class FQN candidate (from _CURRENT_VAR_TYPES) or SELF_ATTRIBUTE_KEY for the +# implicit ``self``/``cls`` receiver. Recording at the write statement resolves +# the RHS against the CURRENT per-statement var_taints — a later reassignment of +# the RHS variable can no longer launder the recorded taint, which the post-hoc +# final-state second walk allowed (wardline-b369c7d06c). +_CURRENT_ATTR_WRITES: contextvars.ContextVar[dict[str, dict[str, TaintState]] | None] = contextvars.ContextVar( + "_CURRENT_ATTR_WRITES", default=None +) + +# Recording key for writes through the implicit instance/class receiver +# (``self.x = ...`` / ``cls.x = ...``). Not a valid dotted FQN, so it can never +# collide with a class-candidate key. +SELF_ATTRIBUTE_KEY = "" + +# Poison candidate marking a receiver-type set where SOME branch arm rebound the +# name to an untypeable value (see :func:`_merge_branch_types`). Not a valid +# dotted FQN, so it never resolves in any taint map — its presence makes the +# typed method dispatch's all-candidates completeness check fail, forcing the +# conservative generic resolution on the maybe-untyped path. +UNTYPED_ARM_CANDIDATE = "" + # Maps a local name to the CANDIDATE SET of lambda bodies it MAY hold. A single # slot per name would lose a sink-lambda bound in a non-last branch arm when a later # arm rebinds the same name (wardline-383f83fafe): only one survived the merge, so a @@ -140,6 +221,24 @@ } ) +# Curated WORST-ARG constructors: in-memory buffer ctors whose result holds +# exactly the data poured in. ``io.StringIO("const")`` is NOT external data — +# the unresolved-import UNKNOWN_RAW default was a documented PY-WL-101 FP on +# returning an in-memory constant read — while ``io.StringIO(raw)`` must keep +# carrying the raw content so a later ``.read()`` propagates it (the receiver +# RAW_ZONE path). No args → INTEGRAL (an empty buffer holds nothing). +_WORST_ARG_CONSTRUCTORS: frozenset[str] = frozenset({"io.StringIO", "io.BytesIO"}) + + +def _worst_arg_taint(arg_taints: list[TaintState]) -> TaintState: + """The weakest-link (least-trusted) of *arg_taints*; INTEGRAL when empty.""" + if not arg_taints: + return TaintState.INTEGRAL + result = arg_taints[0] + for at in arg_taints[1:]: + result = combine(result, at) + return result + @dataclass(frozen=True, slots=True) class VariableTaintContext: @@ -160,6 +259,71 @@ class VariableTaintResult: return_callee: str | None +@contextmanager +def attribute_write_recording(out: dict[str, dict[str, TaintState]]) -> Iterator[dict[str, dict[str, TaintState]]]: + """Record attribute writes into *out* for the duration of the block. + + Wrap around a :func:`compute_variable_taints` / L2-stage run. Every + ``. = ...`` (plain, annotated, or augmented) the statement + walk processes is recorded with its RHS resolved against the CURRENT + per-statement ``var_taints`` (branch-local inside arms), keyed by the + receiver's class-FQN candidates — or :data:`SELF_ATTRIBUTE_KEY` for + ``self``/``cls`` — and joined per attribute via :func:`combine` + (least-trusted wins). Map keys onto class qualnames with + :func:`project_attribute_writes`.""" + token = _CURRENT_ATTR_WRITES.set(out) + try: + yield out + finally: + _CURRENT_ATTR_WRITES.reset(token) + + +def project_attribute_writes( + recorded: dict[str, dict[str, TaintState]], + class_qualnames: frozenset[str], + enclosing_class: str | None, +) -> dict[str, dict[str, TaintState]]: + """Map a recording's receiver keys onto class qualnames. + + :data:`SELF_ATTRIBUTE_KEY` writes attribute to *enclosing_class* (dropped when + ``None`` — outside a method there is no instance to attribute to); candidate + keys are kept only when they name a known project class. Buckets landing on + the same class (a ``self`` write and a typed-receiver write) join per + attribute via :func:`combine`.""" + out: dict[str, dict[str, TaintState]] = {} + for key, attrs in recorded.items(): + target = enclosing_class if key == SELF_ATTRIBUTE_KEY else (key if key in class_qualnames else None) + if target is None: + continue + bucket = out.setdefault(target, {}) + for attr, taint in attrs.items(): + bucket[attr] = combine(bucket[attr], taint) if attr in bucket else taint + return out + + +def _record_attribute_write(target: ast.Attribute, rhs: TaintState) -> None: + """Record one attribute write into the active side channel (no-op when off). + + A ``self``/``cls`` receiver records under :data:`SELF_ATTRIBUTE_KEY`; a + tracked receiver records under EVERY class candidate it may hold — a write + through a branch-joined receiver is factually a write to whichever class the + taken path bound, so attributing to all candidates is the sound + over-approximation (wardline-b369c7d06c). Only a direct ``Name.attr`` target + is a class-attribute write (deeper chains would need container modelling).""" + out = _CURRENT_ATTR_WRITES.get() + if out is None or not isinstance(target.value, ast.Name): + return + receiver = target.value.id + keys: list[str] = [SELF_ATTRIBUTE_KEY] if receiver in ("self", "cls") else [] + var_types = _CURRENT_VAR_TYPES.get() + if var_types is not None: + keys.extend(var_types.get(receiver, ())) + attr = target.attr + for key in keys: + bucket = out.setdefault(key, {}) + bucket[attr] = combine(bucket[attr], rhs) if attr in bucket else rhs + + def analyze_function_variables( func_node: ast.FunctionDef | ast.AsyncFunctionDef, function_taint: TaintState, @@ -184,7 +348,18 @@ def analyze_function_variables( provenance_clash=context.provenance_clash, ) return_taint = compute_return_taint(func_node, function_taint, dict(taint_map), variable_taints) - return_callee = compute_return_callee(func_node, function_taint, dict(taint_map), dict(variable_taints)) + # compute_return_callee is PROVENANCE-ONLY: _assignment_callee re-resolves + # every direct-call assignment RHS against the FINAL var_taints, and letting + # that re-resolution record would COMBINE post-call taint into the at-call + # snapshot (a clean-at-the-call value re-read as raw → PY-WL-105/108 FPs; + # review 2026-06-10). Suppress recording for its duration — return-position + # sink calls already recorded during compute_return_taint above, which is + # the only recording they get. + token_no_record = _CURRENT_CALL_SITE_ARG_TAINTS.set(None) + try: + return_callee = compute_return_callee(func_node, function_taint, dict(taint_map), dict(variable_taints)) + finally: + _CURRENT_CALL_SITE_ARG_TAINTS.reset(token_no_record) return VariableTaintResult( call_site_taints=call_site_taints, call_site_arg_taints=call_site_arg_taints, @@ -285,23 +460,58 @@ def _resolve_lambda_body_at_call( """ scope = dict(var_taints) args = lam.args + # A ``**mapping`` spread arrives as a keyword with ``arg is None``. Its keys + # are unknown statically, so it MAY bind ANY named parameter not otherwise + # supplied (and lands in ``**kw``) — the unmatched-parameter fallback is the + # spread's taint when one is present, else the neutral seed + # (wardline-93d608c997: ``(lambda **kw: ...)(**raw)`` previously laundered). + spread_taint = kw_taints.get(None) + unmatched_seed = spread_taint if spread_taint is not None else function_taint positional_params = [*args.posonlyargs, *args.args] + # A parameter OMITTED at the call site evaluates its DEFAULT, so the default + # expression's taint seeds the param (``cb = lambda x=raw: os.system(x); cb()`` + # must fire — the default previously fell back to the neutral seed, a fail-open + # FN). Defaults resolve against the pristine enclosing scope (def-time + # semantics, before any param binding shadows the names). When a ``**spread`` + # is present the param MAY be bound by it instead — either/or, so the two + # alternatives combine (weakest-link). A param actually SUPPLIED at the call + # site never evaluates its default, so the matched paths below are untouched + # (no FP: ``cb("clean")`` / ``cb(x="clean")`` stay clean). + default_taints: dict[str, TaintState] = {} + defaulted_params = positional_params[len(positional_params) - len(args.defaults) :] + for param, default in zip(defaulted_params, args.defaults, strict=True): + default_taints[param.arg] = _resolve_expr(default, function_taint, taint_map, scope) + for param, kw_default in zip(args.kwonlyargs, args.kw_defaults, strict=True): + if kw_default is not None: + default_taints[param.arg] = _resolve_expr(kw_default, function_taint, taint_map, scope) + + def _omitted_seed(param_name: str) -> TaintState: + default = default_taints.get(param_name) + if default is None: + return unmatched_seed + return combine(default, spread_taint) if spread_taint is not None else default + for param, taint in zip(positional_params, pos_taints, strict=False): scope[param.arg] = taint for param in positional_params[len(pos_taints) :]: - scope[param.arg] = function_taint + # A param supplied by KEYWORD below keeps the neutral unmatched seed here + # (its default never evaluates); the keyword taint combines in afterwards. + scope[param.arg] = unmatched_seed if param.arg in kw_taints else _omitted_seed(param.arg) if args.vararg is not None: extra = pos_taints[len(positional_params) :] scope[args.vararg.arg] = extra[0] if extra else function_taint for taint in extra[1:]: scope[args.vararg.arg] = combine(scope[args.vararg.arg], taint) for param in args.kwonlyargs: - scope[param.arg] = kw_taints.get(param.arg, function_taint) + kw_hit = kw_taints.get(param.arg) + scope[param.arg] = kw_hit if kw_hit is not None else _omitted_seed(param.arg) for param in positional_params: if param.arg in kw_taints: scope[param.arg] = combine(scope.get(param.arg, function_taint), kw_taints[param.arg]) if args.kwarg is not None: - keyword_values = [taint for name, taint in kw_taints.items() if name is not None] + # Every keyword value — named (the over-approximation: non-param names + # land in ``**kw``) AND the ``**spread`` itself — feeds the kwarg dict. + keyword_values = list(kw_taints.values()) scope[args.kwarg.arg] = keyword_values[0] if keyword_values else function_taint for taint in keyword_values[1:]: scope[args.kwarg.arg] = combine(scope[args.kwarg.arg], taint) @@ -328,9 +538,12 @@ def compute_variable_taints( taint_map: call-resolution map keyed by the call-site name AS WRITTEN — bare (``"foo"``) for ``foo()``, dotted (``"mod.fn"``) for ``mod.fn()`` — mapping to that call's return taint. Bare calls whose - name is absent fall back to ``function_taint``; imported-but-unmodeled - calls resolve to ``UNKNOWN_RAW`` so external code cannot inherit a - trusted caller seed. + name is absent resolve to the WORST of ``function_taint`` and their + argument taints (an unknown callee cannot be assumed to clean a raw + argument — only the curated ``_NON_PROPAGATING_BUILTINS`` validators/ + measurers keep the plain ``function_taint`` fallback); + imported-but-unmodeled calls resolve to ``UNKNOWN_RAW`` so external + code cannot inherit a trusted caller seed. call_site_taints: optional out-dict for FLOW-SENSITIVE reads. When given, records ``{id(stmt): snapshot}`` — the per-variable taint map AS IT IS on entry to each statement (in branches, the branch-local copy). A sink @@ -426,7 +639,7 @@ def handle_arg(arg: ast.arg) -> None: if arg.annotation and var_types is not None: fqn = _resolve_expr_fqn(arg.annotation, alias_map) if fqn: - var_types[arg.arg] = fqn + var_types[arg.arg] = [fqn] for arg in (*args.posonlyargs, *args.args, *args.kwonlyargs): handle_arg(arg) @@ -504,7 +717,25 @@ def _resolve_expr( # (``(x := ...)``), so the False branch is unreachable by any parseable source. if isinstance(node.target, ast.Name): # pragma: no branch var_taints[node.target.id] = taint + # A walrus REBINDS the name in the enclosing scope, so the receiver-type + # candidate must follow: a ctor RHS is a typed strong update, anything + # else invalidates the stale candidate — leaving it let a clean @trusted + # method summary resolve on a rebound raw receiver (review 2026-06-10, + # the _assign_target/NamedExpr stale-type launder). + _update_var_type(node.target.id, node.value) return taint + if isinstance(node, ast.Compare): + # Resolve the left operand and every comparator so calls nested in a + # comparison position record their flow-sensitive arg-taint snapshots + # (``if store(read_raw(p)) == 1:`` previously never resolved, so + # PY-WL-105 missed it and constant-arg sink calls degraded to the + # pessimistic UNKNOWN_RAW fallback). The comparison RESULT is a bool — + # derived from the data but not carrying it — so it is INTEGRAL, the + # same posture as the curated validator/measurer builtins. + _resolve_expr(node.left, function_taint, taint_map, var_taints) + for comparator in node.comparators: + _resolve_expr(comparator, function_taint, taint_map, var_taints) + return TaintState.INTEGRAL if isinstance(node, ast.IfExp): # ``a if c else b`` evaluates to ONE of its arms — combine via the rank-meet # least_trusted (weakest-link), so two clean arms stay clean (no MIXED_RAW @@ -632,7 +863,9 @@ def _resolve_comprehension( local = dict(var_taints) for gen in node.generators: iter_t = _resolve_expr(gen.iter, function_taint, taint_map, local) - _assign_target(gen.target, iter_t, local) + # Comprehension targets bind a comprehension-LOCAL scope (Py3) — they must + # not invalidate the enclosing name's receiver-type candidate. + _assign_target(gen.target, iter_t, local, invalidate_types=False) for cond in gen.ifs: # Resolve conditions for walrus side-effects (PEP 572 → enclosing scope, # leaked back below). @@ -771,6 +1004,11 @@ def _resolve_call( return TaintState.UNKNOWN_RAW if imported_fqn in taint_map and not root_shadows_raw_local: return taint_map[imported_fqn] + if imported_fqn in _WORST_ARG_CONSTRUCTORS and not root_shadows_raw_local: + # In-memory buffer ctor: result = worst arg taint (INTEGRAL when + # empty). Gated on the shadow guard — a raw local shadowing ``io`` + # must not launder through the clean ctor model. + return _worst_arg_taint(arg_taints) if isinstance(node.func, ast.Attribute): dotted = _dotted_name(node.func) @@ -785,6 +1023,11 @@ def _resolve_call( # ``not root_shadows_raw_local``: a raw local/param shadowing the # module name must not inherit the module entry (see above). return taint_hit + if dotted in _WORST_ARG_CONSTRUCTORS and not root_shadows_raw_local: + # Dotted form without an alias-map resolution (degraded caller): + # same worst-arg in-memory buffer ctor model as the imported_fqn + # path above, behind the same shadow guard. + return _worst_arg_taint(arg_taints) if isinstance(node.func.value, ast.Name): var_types = _CURRENT_VAR_TYPES.get() if var_types is not None and node.func.value.id in var_types: @@ -795,16 +1038,34 @@ def _resolve_call( # typed object routinely has a RAW-ZONE value taint (an unmodeled # ``Type()`` constructor defaults to ``UNKNOWN_RAW``), and that object # must still resolve ``Type.method`` via its type — gating on the - # value taint false-positives the ``h = Helper(); h.get_assured()`` + # whole RAW_ZONE false-positives the ``h = Helper(); h.get_assured()`` # pattern (test_helper_method_returning_assured_does_not_false_positive). - # A config sanitiser CAN mint a ``Type.method`` key, so a raw-seeded - # *typed parameter* could in principle launder here; that narrow - # residual is accepted over the FP cost (wardline-f6a29ce23a). - type_prefix = var_types[node.func.value.id] - resolved_dotted = f"{type_prefix}.{node.func.attr}" - taint_hit = taint_map.get(resolved_dotted) - if taint_hit is not None: - return taint_hit + # A DECLARED-raw receiver (EXTERNAL_RAW/MIXED_RAW — a boundary-seeded + # typed parameter, not a constructor default) is the launder case the + # provenance distinguishes: its clean ``Type.method`` summary describes + # a trustworthy instance, not the attacker-controlled object actually + # flowing in, so the receiver's own taint wins (wardline-03c8805449; + # closes the residual previously accepted under wardline-f6a29ce23a). + # The receiver may hold SEVERAL class candidates (branch-joined set, + # wardline-b369c7d06c): dispatch returns the candidates' combined + # summaries (least-trusted wins) only when EVERY candidate resolves; + # a partial hit falls through to the conservative generic handling + # (an unresolved candidate has no summary to honour, and the + # fall-through never launders — FN-safe). + candidates = var_types[node.func.value.id] + hits = [ + taint_map[resolved] + for prefix in candidates + if (resolved := f"{prefix}.{node.func.attr}") in taint_map + ] + if hits and len(hits) == len(candidates): + receiver_value = var_taints.get(node.func.value.id) + if receiver_value in (TaintState.EXTERNAL_RAW, TaintState.MIXED_RAW): + return receiver_value + result = hits[0] + for hit in hits[1:]: + result = combine(result, hit) + return result attr = node.func.attr if attr in _STORAGE_READ_METHODS: # DB-cursor fetch loads stored/external data → seed EXTERNAL_RAW, like a file @@ -886,6 +1147,18 @@ def _resolve_call( # An imported call that was not modeled by project summaries, stdlib_taint, # config, sink handling, or propagation rules returns data we cannot prove. return TaintState.UNKNOWN_RAW + if isinstance(node.func, ast.Name) and node.func.id not in _NON_PROPAGATING_BUILTINS: + # Unresolved bare-name call (an unannotated parameter, a runtime-bound + # callable): the callee is unknown, so its result is the WORST of the + # caller seed and the argument taints — an unknown callee cannot be + # assumed to clean a raw argument (``os.system(transform(raw))`` with a + # bare-param ``transform`` previously inherited the trusted seed — + # wardline-93d608c997). Mirrors the imported-but-unmodeled conservatism; + # the curated validator/measurer builtins above keep the plain seed. + result = function_taint + for at in arg_taints: + result = combine(result, at) + return result return function_taint @@ -942,12 +1215,15 @@ def _process_stmt( lambda_bindings[stmt.target.id] = [value] else: lambda_bindings.pop(stmt.target.id, None) + _update_var_type(stmt.target.id, value) elif isinstance(stmt.target, (ast.Subscript, ast.Attribute)): # Annotated container/attribute write (``self.x: str = expr``, # ``d['k']: T = expr``) — same fail-open shape as the plain-Assign Part # B case: contaminate the base variable so a later read sees it. rhs = _resolve_expr(value, function_taint, taint_map, var_taints) _taint_container_base(stmt.target, rhs, function_taint, var_taints) + if isinstance(stmt.target, ast.Attribute): + _record_attribute_write(stmt.target, rhs) else: # pragma: no cover # Unreachable: an AnnAssign target is always Name/Attribute/Subscript by # the Python grammar, so the two branches above are exhaustive. Kept as a @@ -979,8 +1255,36 @@ def _process_stmt( elif isinstance(stmt, (ast.FunctionDef, ast.AsyncFunctionDef, ast.ClassDef)): pass # Nested function/class — don't descend (separate scope). + elif isinstance(stmt, ast.Raise): + # Resolve the full exception/cause expressions (not just walruses) so calls + # nested in a raise position record their flow-sensitive arg-taint snapshots + # (``raise ValueError(store(read_raw(p)))`` was invisible to PY-WL-105 and + # degraded sink rules to the pessimistic fallback — review 2026-06-10). + if stmt.exc is not None: + _resolve_expr(stmt.exc, function_taint, taint_map, var_taints) + if stmt.cause is not None: + _resolve_expr(stmt.cause, function_taint, taint_map, var_taints) + + elif isinstance(stmt, ast.Assert): + # Same snapshot rationale as Raise: the test and message are real + # expression positions whose calls must record. + _resolve_expr(stmt.test, function_taint, taint_map, var_taints) + if stmt.msg is not None: + _resolve_expr(stmt.msg, function_taint, taint_map, var_taints) + + elif isinstance(stmt, ast.Delete): + for target in stmt.targets: + # Resolve the target expressions (a ``del d[key_call()]`` slice is an + # expression position) and drop the deleted NAME's receiver-type + # candidate — after ``del v`` any later rebind starts untyped. + _resolve_expr(target, function_taint, taint_map, var_taints) + if isinstance(target, ast.Name): + var_types = _CURRENT_VAR_TYPES.get() + if var_types is not None: + var_types.pop(target.id, None) + else: - # Return, Raise, Import, Pass, Break, Continue, etc. + # Return, Import, Pass, Break, Continue, etc. _walk_exprs_for_walrus(stmt, function_taint, taint_map, var_taints) @@ -1006,6 +1310,9 @@ def _walk_exprs_for_walrus( # False branch is unreachable by any parseable source. if isinstance(child.target, ast.Name): # pragma: no branch var_taints[child.target.id] = taint + # Walrus rebind — receiver-type strong update / invalidation, the + # same discipline as _resolve_expr's NamedExpr branch. + _update_var_type(child.target.id, child.value) _walk_exprs_for_walrus(child, function_taint, taint_map, var_taints) @@ -1024,6 +1331,35 @@ def _resolve_expr_fqn(node: ast.expr, alias_map: dict[str, str] | None) -> str | return None +def _update_var_type(target_id: str, value: ast.expr) -> None: + """Strong-update the receiver-type candidate set for a straight-line assignment. + + ``x = Type()`` binds the single candidate ``[Type_fqn]``; ``x = y`` copies + ``y``'s candidate set. Reassignment to an RHS we cannot type precisely + (Subscript / BinOp / IfExp / f-string / comprehension / unresolvable Call / + typeless Name) INVALIDATES any prior recorded type: keeping the stale type let + a method call on a now-raw value resolve a clean @trusted summary, laundering + raw past the RAW_ZONE receiver guard (wardline-5ba7ce0f98). Dropping it falls + back to conservative resolution (more FPs at worst, never an FN). Multi-class + candidate sets arise only at branch joins (:func:`_merge_branch_types`).""" + var_types = _CURRENT_VAR_TYPES.get() + if var_types is None: + return + new_types: list[str] | None = None + if isinstance(value, ast.Call): + fqn = _resolve_expr_fqn(value.func, _CURRENT_ALIAS_MAP.get()) + if fqn: + new_types = [fqn] + elif isinstance(value, ast.Name): + prior = var_types.get(value.id) + if prior: + new_types = list(prior) + if new_types is not None: + var_types[target_id] = new_types + else: + var_types.pop(target_id, None) + + def _handle_assign( stmt: ast.Assign, function_taint: TaintState, @@ -1051,24 +1387,7 @@ def _handle_assign( else: lambda_bindings.pop(target.id, None) - var_types = _CURRENT_VAR_TYPES.get() - if var_types is not None: - alias_map = _CURRENT_ALIAS_MAP.get() - new_type: str | None = None - if isinstance(stmt.value, ast.Call): - new_type = _resolve_expr_fqn(stmt.value.func, alias_map) or None - elif isinstance(stmt.value, ast.Name): - new_type = var_types.get(stmt.value.id) - if new_type is not None: - var_types[target.id] = new_type - else: - # Reassignment to an RHS we cannot type precisely (Subscript / BinOp / - # IfExp / f-string / comprehension / unresolvable Call / typeless Name) - # INVALIDATES any prior recorded type. Keeping the stale type let a method - # call on a now-raw value resolve a clean @trusted summary, laundering raw - # past the RAW_ZONE receiver guard (wardline-5ba7ce0f98). Dropping it falls - # back to conservative resolution (more FPs at worst, never an FN). - var_types.pop(target.id, None) + _update_var_type(target.id, stmt.value) elif isinstance(target, (ast.Tuple, ast.List)): # Tuple unpacking: a, b = ... @@ -1091,6 +1410,10 @@ def _handle_assign( # read back at its creation taint — a fail-open under-taint. rhs = _resolve_expr(stmt.value, function_taint, taint_map, var_taints) _taint_container_base(target, rhs, function_taint, var_taints) + if isinstance(target, ast.Attribute): + # Class-attribute side channel: the RHS taint at THIS statement, + # before any later reassignment can launder it (wardline-b369c7d06c). + _record_attribute_write(target, rhs) def _handle_unpack( @@ -1110,6 +1433,9 @@ def _handle_unpack( return else: # RHS is not a matching literal tuple — all targets get RHS taint. + # _assign_target recurses into nested Tuple/List targets (``a, (b, c) = + # raw`` must taint b and c — skipping them was a fail-open under-taint) + # and contaminates the base container of a Subscript/Attribute target. rhs_taint = _resolve_expr( value, function_taint, @@ -1117,10 +1443,7 @@ def _handle_unpack( var_taints, ) for tgt in target.elts: - if isinstance(tgt, ast.Name): - var_taints[tgt.id] = rhs_taint - elif isinstance(tgt, ast.Starred) and isinstance(tgt.value, ast.Name): - var_taints[tgt.value.id] = rhs_taint + _assign_target(tgt, rhs_taint, var_taints) def _handle_literal_unpack( @@ -1135,16 +1458,12 @@ def _handle_literal_unpack( if len(value.elts) != len(target.elts): return False for tgt, val in zip(target.elts, value.elts, strict=False): - if isinstance(tgt, ast.Name): - taint = _resolve_expr( - val, - function_taint, - taint_map, - var_taints, - ) - var_taints[tgt.id] = taint - elif isinstance(tgt, (ast.Tuple, ast.List)): + if isinstance(tgt, (ast.Tuple, ast.List)): _handle_unpack(tgt, val, function_taint, taint_map, var_taints) + else: + # Name binds; a Subscript/Attribute target (``d['k'], _ = raw, 1``) + # contaminates its base container via _assign_target. + _assign_target(tgt, _resolve_expr(val, function_taint, taint_map, var_taints), var_taints) return True fixed_targets = len(target.elts) - 1 @@ -1153,10 +1472,10 @@ def _handle_literal_unpack( suffix_count = len(target.elts) - starred_idx - 1 for tgt, val in zip(target.elts[:starred_idx], value.elts[:starred_idx], strict=False): - if isinstance(tgt, ast.Name): - var_taints[tgt.id] = _resolve_expr(val, function_taint, taint_map, var_taints) - elif isinstance(tgt, (ast.Tuple, ast.List)): + if isinstance(tgt, (ast.Tuple, ast.List)): _handle_unpack(tgt, val, function_taint, taint_map, var_taints) + else: + _assign_target(tgt, _resolve_expr(val, function_taint, taint_map, var_taints), var_taints) slice_end = len(value.elts) - suffix_count if suffix_count else len(value.elts) captured = value.elts[starred_idx:slice_end] @@ -1173,10 +1492,10 @@ def _handle_literal_unpack( for offset in range(suffix_count): tgt = target.elts[starred_idx + 1 + offset] val = value.elts[len(value.elts) - suffix_count + offset] - if isinstance(tgt, ast.Name): - var_taints[tgt.id] = _resolve_expr(val, function_taint, taint_map, var_taints) - elif isinstance(tgt, (ast.Tuple, ast.List)): + if isinstance(tgt, (ast.Tuple, ast.List)): _handle_unpack(tgt, val, function_taint, taint_map, var_taints) + else: + _assign_target(tgt, _resolve_expr(val, function_taint, taint_map, var_taints), var_taints) return True @@ -1207,6 +1526,8 @@ def _handle_augassign( elif isinstance(stmt.target, (ast.Subscript, ast.Attribute)): # pragma: no branch # d[k] += expr / obj.x += expr — contaminate the base container. _taint_container_base(stmt.target, rhs_taint, function_taint, var_taints) + if isinstance(stmt.target, ast.Attribute): + _record_attribute_write(stmt.target, rhs_taint) def _container_base_name(target: ast.expr) -> str | None: @@ -1254,6 +1575,59 @@ def _branch_copy(parent: dict[str, list[ast.Lambda]] | None) -> dict[str, list[a return {name: list(lams) for name, lams in parent.items()} if parent is not None else None +def _types_branch_copy(parent: dict[str, list[str]] | None) -> dict[str, list[str]] | None: + """An arm-local copy of the receiver-type candidate map for one branch arm + (``None`` when types are not being tracked). Candidate LISTS are copied too, + so an arm's strong update cannot mutate the parent's or a sibling's set in + place — branch-local exactly like ``var_taints`` (wardline-b369c7d06c).""" + return {name: list(types) for name, types in parent.items()} if parent is not None else None + + +def _merge_branch_types( + parent: dict[str, list[str]] | None, + arms: list[dict[str, list[str]] | None], +) -> None: + """Merge mutually-exclusive branch arms' receiver-type candidates back into + *parent* in place. Post-branch a receiver MAY hold whichever class the taken + arm bound, so its candidate set is the UNION of every arm's set (dedup by + equality — FQNs are strings). An arm that never touched a name carries its + pre-branch set (each arm is a full copy of *parent*), so a class binding made + before the branch survives an arm that rebound the name — that surviving + candidate is what attributes a post-branch attribute write to the + not-rebound class too (wardline-b369c7d06c). A name EVERY arm rebound to an + untypeable value drops out (absent from all arms — sound: no path still + holds a tracked class). Mirrors :func:`_merge_branch_bindings`, including the + absent-or-non-empty invariant. + + A name SOME-but-not-all arms dropped (rebound to an untypeable value — a + for/with/unpack/handler rebind) keeps the surviving arms' candidates for the + attribute-write channel but gains the :data:`UNTYPED_ARM_CANDIDATE` poison + marker: on the dropped arm's path the receiver holds an object of UNKNOWN + type, so a method-summary READ must not resolve cleanly through the other + arm's class (the launder PY-WL-101 closed in the 2026-06-10 review). The + marker never resolves in any taint map, so the dispatch's all-candidates + completeness check fails and resolution falls through to the conservative + generic handling; the write recorder's bucket for it is dropped by + :func:`project_attribute_writes` (not a known class).""" + if parent is None: + return + merged: dict[str, list[str]] = {} + tracked_arms = [arm for arm in arms if arm is not None] + for arm in tracked_arms: + for name, types in arm.items(): + if not types: + continue # uphold the absent-or-non-empty invariant + bucket = merged.setdefault(name, []) + for fqn in types: + if fqn not in bucket: + bucket.append(fqn) + for name, bucket in merged.items(): + if UNTYPED_ARM_CANDIDATE not in bucket and any(not arm.get(name) for arm in tracked_arms): + bucket.append(UNTYPED_ARM_CANDIDATE) + parent.clear() + parent.update(merged) + + def _walk_branch_body( body: list[ast.stmt], function_taint: TaintState, @@ -1261,18 +1635,21 @@ def _walk_branch_body( var_taints: dict[str, TaintState], call_site_taints: dict[int, dict[str, TaintState]] | None, arm_bindings: dict[str, list[ast.Lambda]] | None, + arm_types: dict[str, list[str]] | None, ) -> None: - """Walk one branch arm's body with *arm_bindings* as the active (arm-local) - lambda-bindings map, so lambda assignments inside the arm mutate the copy, not the - shared parent. A plain ``_walk_body`` when bindings aren't tracked.""" - if arm_bindings is None: - _walk_body(body, function_taint, taint_map, var_taints, call_site_taints) - return - token = _CURRENT_LAMBDA_BINDINGS.set(arm_bindings) + """Walk one branch arm's body with *arm_bindings* / *arm_types* as the active + (arm-local) lambda-bindings and receiver-type maps, so lambda assignments and + class rebinds inside the arm mutate the copies, not the shared parent. A plain + ``_walk_body`` for whichever map isn't tracked.""" + token_bindings = _CURRENT_LAMBDA_BINDINGS.set(arm_bindings) if arm_bindings is not None else None + token_types = _CURRENT_VAR_TYPES.set(arm_types) if arm_types is not None else None try: _walk_body(body, function_taint, taint_map, var_taints, call_site_taints) finally: - _CURRENT_LAMBDA_BINDINGS.reset(token) + if token_types is not None: + _CURRENT_VAR_TYPES.reset(token_types) + if token_bindings is not None: + _CURRENT_LAMBDA_BINDINGS.reset(token_bindings) def _merge_branch_bindings( @@ -1339,24 +1716,32 @@ def _handle_if( # Snapshot before branches. pre_if = dict(var_taints) parent_lambdas = _CURRENT_LAMBDA_BINDINGS.get() + parent_types = _CURRENT_VAR_TYPES.get() - # Walk the if-body with an arm-local lambda-bindings copy — branch-local like - # var_taints, so a lambda bound here cannot leak into the else arm. + # Walk the if-body with arm-local lambda-bindings and receiver-type copies — + # branch-local like var_taints, so a lambda bound or class rebound here cannot + # leak into the else arm. if_taints = dict(var_taints) if_lambdas = _branch_copy(parent_lambdas) - _walk_branch_body(stmt.body, function_taint, taint_map, if_taints, call_site_taints, if_lambdas) + if_types = _types_branch_copy(parent_types) + _walk_branch_body(stmt.body, function_taint, taint_map, if_taints, call_site_taints, if_lambdas, if_types) if stmt.orelse: - # Walk the else-body on its own arm-local bindings copy. + # Walk the else-body on its own arm-local copies. else_taints = dict(var_taints) else_lambdas = _branch_copy(parent_lambdas) - _walk_branch_body(stmt.orelse, function_taint, taint_map, else_taints, call_site_taints, else_lambdas) + else_types = _types_branch_copy(parent_types) + _walk_branch_body( + stmt.orelse, function_taint, taint_map, else_taints, call_site_taints, else_lambdas, else_types + ) else: - # No else — the "else" branch is the pre-if state with bindings unchanged. + # No else — the "else" branch is the pre-if state with bindings/types unchanged. else_taints = pre_if else_lambdas = _branch_copy(parent_lambdas) + else_types = _types_branch_copy(parent_types) _merge_branch_bindings(parent_lambdas, [if_lambdas, else_lambdas]) + _merge_branch_types(parent_types, [if_types, else_types]) # Merge: for each variable, combine the two branch values. The var holds ONE # branch's value (an alternative), so combine via the rank-meet least_trusted @@ -1410,6 +1795,10 @@ def _handle_for( # post-loop call through the name misses the sink (the FN this closes, # wardline-d6af917bde). pre_loop_lambdas = _branch_copy(_CURRENT_LAMBDA_BINDINGS.get()) + # Same zero-trip arm for receiver-type candidates: a class rebound inside the + # body MAY not have run, so the pre-loop candidates are unioned back below + # (wardline-b369c7d06c). + pre_loop_types = _types_branch_copy(_CURRENT_VAR_TYPES.get()) # Iterate the walk until var_taints genuinely converges (WLN-MED-09). The bound is # num_vars × lattice_height, NOT lattice_height (8) alone: 8 caps a SINGLE variable's @@ -1445,6 +1834,10 @@ def _handle_for( if parent_lambdas is not None: post_body_arm = _branch_copy(parent_lambdas) _merge_branch_bindings(parent_lambdas, [post_body_arm, pre_loop_lambdas]) + parent_types = _CURRENT_VAR_TYPES.get() + if parent_types is not None: + post_body_types = _types_branch_copy(parent_types) + _merge_branch_types(parent_types, [post_body_types, pre_loop_types]) # Walk orelse (runs after normal loop completion). if stmt.orelse: @@ -1466,6 +1859,7 @@ def _handle_while( # the fixpoint — the no-`else` ``if`` mirror that keeps a pre-loop sink-lambda visible # on the zero-trip path (wardline-d6af917bde; see _handle_for for the full rationale). pre_loop_lambdas = _branch_copy(_CURRENT_LAMBDA_BINDINGS.get()) + pre_loop_types = _types_branch_copy(_CURRENT_VAR_TYPES.get()) # Iterate to genuine convergence with a num_vars × lattice_height backstop — see # _handle_for: a range(8) cap was unsound for loop-carried chains > 8 links @@ -1488,6 +1882,10 @@ def _handle_while( if parent_lambdas is not None: post_body_arm = _branch_copy(parent_lambdas) _merge_branch_bindings(parent_lambdas, [post_body_arm, pre_loop_lambdas]) + parent_types = _CURRENT_VAR_TYPES.get() + if parent_types is not None: + post_body_types = _types_branch_copy(parent_types) + _merge_branch_types(parent_types, [post_body_types, pre_loop_types]) if stmt.orelse: _walk_body(stmt.orelse, function_taint, taint_map, var_taints, call_site_taints) @@ -1524,28 +1922,67 @@ def _handle_try( """Handle try/except/else/finally — snapshot-branch-join pattern.""" pre_try = dict(var_taints) parent_lambdas = _CURRENT_LAMBDA_BINDINGS.get() + parent_types = _CURRENT_VAR_TYPES.get() - # Walk try body on a copy (arm-local lambda bindings — branch-local like var_taints). + # Walk try body on a copy (arm-local lambda bindings/receiver types — + # branch-local like var_taints). + pre_try_snapshot_ids = set(call_site_taints) if call_site_taints is not None else None try_taints = dict(pre_try) try_lambdas = _branch_copy(parent_lambdas) - _walk_branch_body(stmt.body, function_taint, taint_map, try_taints, call_site_taints, try_lambdas) + try_types = _types_branch_copy(parent_types) + _walk_branch_body(stmt.body, function_taint, taint_map, try_taints, call_site_taints, try_lambdas, try_types) + + # A handler runs after an exception raised at ANY point in the try body, so + # it can observe pre-try state OR any try-body prefix's state — handler arms + # are NOT mutually exclusive with the try body's assignments. Seed each arm + # with the per-variable WORST over the pre-try snapshot, every per-statement + # snapshot the try-body walk recorded (each is a reachable prefix state, so a + # value assigned raw and cleaned mid-body stays visible), and the body's + # post-walk state (the degraded no-snapshot fallback). Seeding from + # ``dict(pre_try)`` alone dropped taint assigned before a raising call — a + # fail-open under-taint. Lambda-binding/receiver-type arms keep their + # pre-try copies: those channels resolve call shapes, not data values, and + # extra candidates only over-approximate (the var-taint seed is the + # soundness fix). + handler_seed = dict(pre_try) + prefix_states: list[dict[str, TaintState]] = [try_taints] + if call_site_taints is not None and pre_try_snapshot_ids is not None: + prefix_states.extend( + snapshot for stmt_id, snapshot in call_site_taints.items() if stmt_id not in pre_try_snapshot_ids + ) + for state in prefix_states: + for name, taint in state.items(): + current = handler_seed.get(name) + if current is None or TRUST_RANK[taint] > TRUST_RANK[current]: + handler_seed[name] = taint - # Walk each handler on separate copies (mutually exclusive with try body). + # Walk each handler on separate copies. handler_branches: list[dict[str, TaintState]] = [try_taints] # try-success is one branch arm_bindings: list[dict[str, list[ast.Lambda]] | None] = [try_lambdas] + arm_types: list[dict[str, list[str]] | None] = [try_types] for handler in stmt.handlers: - handler_taints = dict(pre_try) + handler_taints = dict(handler_seed) + handler_lambdas = _branch_copy(parent_lambdas) + handler_types = _types_branch_copy(parent_types) if handler.name: handler_taints[handler.name] = function_taint - handler_lambdas = _branch_copy(parent_lambdas) - _walk_branch_body(handler.body, function_taint, taint_map, handler_taints, call_site_taints, handler_lambdas) + # ``except E as name`` REBINDS name to the exception instance — any + # receiver-type candidate the name carried describes an object it no + # longer holds, so the handler arm's copy drops it (review 2026-06-10; + # mirrors collect_sink_bindings' handler-name invalidation). + if handler_types is not None: + handler_types.pop(handler.name, None) + _walk_branch_body( + handler.body, function_taint, taint_map, handler_taints, call_site_taints, handler_lambdas, handler_types + ) handler_branches.append(handler_taints) arm_bindings.append(handler_lambdas) + arm_types.append(handler_types) # Walk orelse on the try-success branch (runs only if no exception) — continue the - # try arm's bindings, not a fresh copy. + # try arm's bindings/types, not fresh copies. if stmt.orelse: - _walk_branch_body(stmt.orelse, function_taint, taint_map, try_taints, call_site_taints, try_lambdas) + _walk_branch_body(stmt.orelse, function_taint, taint_map, try_taints, call_site_taints, try_lambdas, try_types) # Merge all branches. all_vars: set[str] = set() @@ -1572,9 +2009,11 @@ def _handle_try( except KeyError: _taint_val = None # var absent from pre-try state — leave unset - # Lambda bindings: union the mutually-exclusive arms (try-success + each handler) - # back into the parent, mirroring the var_taints join above. + # Lambda bindings / receiver types: union the mutually-exclusive arms + # (try-success + each handler) back into the parent, mirroring the var_taints + # join above. _merge_branch_bindings(parent_lambdas, arm_bindings) + _merge_branch_types(parent_types, arm_types) # finalbody runs unconditionally after merge — with the merged bindings (in place, # the active contextvar dict, since the function body continues into it). @@ -1605,16 +2044,21 @@ def _handle_match( pre_match = dict(var_taints) parent_lambdas = _CURRENT_LAMBDA_BINDINGS.get() + parent_types = _CURRENT_VAR_TYPES.get() branches: list[dict[str, TaintState]] = [] arm_bindings: list[dict[str, list[ast.Lambda]] | None] = [] + arm_types: list[dict[str, list[str]] | None] = [] for case in stmt.cases: case_taints = dict(pre_match) for name in _collect_pattern_targets(case.pattern): case_taints[name] = subject_taint - # Arm-local lambda bindings (guard + body share the arm), branch-local like - # var_taints so a lambda bound in one case cannot leak into a sibling case. + # Arm-local lambda bindings / receiver types (guard + body share the arm), + # branch-local like var_taints so a lambda bound or class rebound in one + # case cannot leak into a sibling case. case_lambdas = _branch_copy(parent_lambdas) + case_types = _types_branch_copy(parent_types) token = _CURRENT_LAMBDA_BINDINGS.set(case_lambdas) if case_lambdas is not None else None + token_types = _CURRENT_VAR_TYPES.set(case_types) if case_types is not None else None try: if case.guard is not None: # The guard is tested with the arm's captures in scope; resolve it for @@ -1622,14 +2066,18 @@ def _handle_match( _resolve_expr(case.guard, function_taint, taint_map, case_taints) _walk_body(case.body, function_taint, taint_map, case_taints, call_site_taints) finally: + if token_types is not None: + _CURRENT_VAR_TYPES.reset(token_types) if token is not None: _CURRENT_LAMBDA_BINDINGS.reset(token) branches.append(case_taints) arm_bindings.append(case_lambdas) + arm_types.append(case_types) - # The implicit "no arm matched" path keeps the pre-match state and bindings. + # The implicit "no arm matched" path keeps the pre-match state, bindings, and types. branches.append(pre_match) arm_bindings.append(_branch_copy(parent_lambdas)) + arm_types.append(_types_branch_copy(parent_types)) all_vars: set[str] = set() for branch in branches: @@ -1641,8 +2089,10 @@ def _handle_match( merged = combine(merged, v) var_taints[var] = merged - # Lambda bindings: union the mutually-exclusive case arms (+ no-match) into parent. + # Lambda bindings / receiver types: union the mutually-exclusive case arms + # (+ no-match) into parent. _merge_branch_bindings(parent_lambdas, arm_bindings) + _merge_branch_types(parent_types, arm_types) # ── Helpers ────────────────────────────────────────────────────── @@ -1688,135 +2138,48 @@ def _assign_target( target: ast.expr, taint: TaintState, var_taints: dict[str, TaintState], + *, + invalidate_types: bool = True, ) -> None: - """Assign taint to a target node (Name, Tuple, or List).""" + """Assign taint to a target node (Name, Tuple/List, Starred, Subscript, Attribute). + + Nested Tuple/List targets recurse so every leaf Name inherits the taint; a + Subscript/Attribute target (``for d['k'] in raws`` / ``d['k'], _ = raw``) + is a container WRITE, so its root Name absorbs the taint via the + weakest-link combine — silently skipping it left the container clean, a + fail-open under-taint (wardline-93d608c997). An Attribute target ALSO + records into the attribute-write side channel: ``self.x, y = raw, 1`` is + the same class-attribute write as ``self.x = raw``, and the recorder only + saw direct ``ast.Attribute`` Assign targets — the unpack form silently + skipped the cross-method summary (a fail-open under-taint). + + Every Name leaf bound here is a REBIND through a form that carries no + statically-typeable RHS (a for/with target, a tuple-unpack leaf, a starred + slice), so its receiver-type candidate is INVALIDATED — keeping the stale + class let a clean ``@trusted`` method summary launder a rebound raw + receiver past PY-WL-101 (review 2026-06-10; mirrors the invalidation + discipline ``collect_sink_bindings`` applies to the same binding forms). + ``invalidate_types=False`` is for COMPREHENSION generator targets, which + bind a comprehension-local scope and must not drop the enclosing name's + candidate. + """ if isinstance(target, ast.Name): var_taints[target.id] = taint + if invalidate_types: + var_types = _CURRENT_VAR_TYPES.get() + if var_types is not None: + var_types.pop(target.id, None) elif isinstance(target, (ast.Tuple, ast.List)): for elt in target.elts: - _assign_target(elt, taint, var_taints) - elif isinstance(target, ast.Starred) and isinstance(target.value, ast.Name): - var_taints[target.value.id] = taint - - -def _self_attr_name(target: ast.expr) -> str | None: - """The attribute name of a ``self.`` / ``cls.`` write target, else None. - - Only a direct attribute of the instance/class receiver (``self.x = ...``) — a - subscript-into-attribute (``self.cache[k] = ...``) or deeper chain is NOT a direct - attribute write and is deliberately excluded (it would need container modelling).""" - if isinstance(target, ast.Attribute) and isinstance(target.value, ast.Name) and target.value.id in ("self", "cls"): - return target.attr - return None - - -def collect_attribute_writes( - func_node: ast.FunctionDef | ast.AsyncFunctionDef, - function_taint: TaintState, - taint_map: dict[str, TaintState], - var_taints: dict[str, TaintState], - class_qualnames: frozenset[str], - alias_map: dict[str, str], - module_prefix: str, - enclosing_class: str | None = None, -) -> dict[str, dict[str, TaintState]]: - """Return ``{class_qualname: {attr_name: least_trusted RHS taint}}`` for every - instance attribute assignment in *func_node*'s body. - - Handles both internal ``self.x = ...`` writes (enclosing class) and external - ``obj.x = ...`` writes where ``obj`` was instantiated via a class constructor - call. - """ - from wardline.scanner.ast_primitives import resolve_call_fqn - - out: dict[str, dict[str, TaintState]] = {} - var_types: dict[str, str] = {} - - def _walk(node: ast.AST) -> None: - for child in ast.iter_child_nodes(node): - if isinstance(child, (ast.FunctionDef, ast.AsyncFunctionDef, ast.ClassDef, ast.Lambda)): - continue - - # 1. Track variable types assigned to constructors or copied - targets: list[ast.expr] = [] - value: ast.expr | None = None - if isinstance(child, ast.Assign): - targets = child.targets - value = child.value - elif isinstance(child, ast.AnnAssign): - targets = [child.target] - value = child.value - - class_fqn = None - if value is not None: - if isinstance(value, ast.Call): - fqn = resolve_call_fqn(value, alias_map, class_qualnames, module_prefix) - if fqn in class_qualnames: - class_fqn = fqn - elif isinstance(value, ast.Name): - class_fqn = var_types.get(value.id) - - if class_fqn is not None: - for tgt in targets: - if isinstance(tgt, ast.Name): - var_types[tgt.id] = class_fqn - - # 2. Record attribute writes - targets_to_check: list[ast.expr] = [] - if isinstance(child, ast.Assign): - targets_to_check = child.targets - value = child.value - elif isinstance(child, (ast.AnnAssign, ast.AugAssign)): - targets_to_check = [child.target] - value = child.value - - for tgt in targets_to_check: - if isinstance(tgt, ast.Attribute) and isinstance(tgt.value, ast.Name): - var_name = tgt.value.id - attr_name = tgt.attr - target_class = None - if var_name in ("self", "cls") and enclosing_class: - target_class = enclosing_class - elif var_name in var_types: - target_class = var_types[var_name] - - if target_class is not None and value is not None: - rhs_taint = _resolve_expr(value, function_taint, taint_map, var_taints) - cls_writes = out.setdefault(target_class, {}) - cls_writes[attr_name] = ( - combine(cls_writes[attr_name], rhs_taint) if attr_name in cls_writes else rhs_taint - ) - - _walk(child) - - token_types = _CURRENT_VAR_TYPES.set(var_types) - token_alias = _CURRENT_ALIAS_MAP.set(alias_map) - try: - _walk(func_node) - finally: - _CURRENT_VAR_TYPES.reset(token_types) - _CURRENT_ALIAS_MAP.reset(token_alias) - return out - - -def collect_self_attr_writes( - func_node: ast.FunctionDef | ast.AsyncFunctionDef, - function_taint: TaintState, - taint_map: dict[str, TaintState], - var_taints: dict[str, TaintState], -) -> dict[str, TaintState]: - """Compatibility wrapper for internal-only writes.""" - writes = collect_attribute_writes( - func_node, - function_taint, - taint_map, - var_taints, - class_qualnames=frozenset(), - alias_map={}, - module_prefix="", - enclosing_class="dummy", - ) - return writes.get("dummy", {}) + _assign_target(elt, taint, var_taints, invalidate_types=invalidate_types) + elif isinstance(target, ast.Starred): + _assign_target(target.value, taint, var_taints, invalidate_types=invalidate_types) + elif isinstance(target, (ast.Subscript, ast.Attribute)): + base = _container_base_name(target) + if base is not None: + var_taints[base] = combine(var_taints.get(base, taint), taint) + if isinstance(target, ast.Attribute): + _record_attribute_write(target, taint) def compute_return_taint( diff --git a/tests/corpus/MANIFEST.yaml b/tests/corpus/MANIFEST.yaml index b88c8142..4a169225 100644 --- a/tests/corpus/MANIFEST.yaml +++ b/tests/corpus/MANIFEST.yaml @@ -4,8 +4,13 @@ # FP rate = FALSE_POSITIVE / total active DEFECTs, gated <= 5%. # Every active DEFECT the engine produces over the fixtures MUST have an entry here # (an unaccounted finding fails the gate — that is how clean-shape regressions surface). -# Per the 2026-05-31 taint-combination audit the engine has 0 live FP, so every entry -# below is TRUE_POSITIVE today; the FALSE_POSITIVE label exists to capture a regression. +# `sentinels:` entries (files under tests/corpus/sentinels, a sibling scan root) are +# clean-shape FALSE_POSITIVE sentinels: the engine must NOT fire on them. A silent +# sentinel is passing (not stale); a fired sentinel is a live FP counted against the +# budget. Sentinels live OUTSIDE fixtures/ so the Track 2 byte-identity golden +# (tests/grammar, frozen over fixtures/ only) keeps its substrate. Per the 2026-05-31 +# taint-combination audit the engine has 0 live FP, so every sentinel is expected +# to stay silent. fixtures: contradictory.py: - {rule_id: PY-WL-110, qualname: "contradictory.conflicting", label: TRUE_POSITIVE, note: "@trusted + @external_boundary — two distinct trust markers"} @@ -32,8 +37,8 @@ fixtures: match_arms.py: - {rule_id: PY-WL-101, qualname: "match_arms.match_arm_leaks", label: TRUE_POSITIVE, note: "one match arm binds raw, then returned"} validators.py: - - {rule_id: PY-WL-102, qualname: "validators.no_rejection", label: TRUE_POSITIVE, note: "trust_boundary(ASSURED) with no rejection path"} - - {rule_id: PY-WL-102, qualname: "validators.no_rejection_guarded", label: TRUE_POSITIVE, note: "trust_boundary(GUARDED) with no rejection path"} + - {rule_id: PY-WL-102, qualname: "validators.no_rejection", label: TRUE_POSITIVE, note: "trust_boundary(ASSURED) with no rejection path (laundered shape; bare return-p belongs to PY-WL-119)"} + - {rule_id: PY-WL-102, qualname: "validators.no_rejection_guarded", label: TRUE_POSITIVE, note: "trust_boundary(GUARDED) with no rejection path (laundered shape)"} exceptions.py: - {rule_id: PY-WL-101, qualname: "exceptions.broad_handler", label: TRUE_POSITIVE, note: "returns undecorated work() (UNKNOWN_RAW) while declaring INTEGRAL"} - {rule_id: PY-WL-101, qualname: "exceptions.silent_handler", label: TRUE_POSITIVE, note: "returns undecorated work()/None while declaring INTEGRAL"} @@ -50,3 +55,13 @@ fixtures: - {rule_id: PY-WL-101, qualname: "more_shapes.augassign_raw", label: TRUE_POSITIVE, note: "augmented assignment merges raw in"} - {rule_id: PY-WL-101, qualname: "more_shapes.launders_through_broken_boundary", label: TRUE_POSITIVE, note: "declared ASSURED but body re-derives raw"} - {rule_id: PY-WL-102, qualname: "more_shapes.passthrough_no_check", label: TRUE_POSITIVE, note: "boundary that cannot reject"} +sentinels: + clean_sql_parameterized.py: + - {rule_id: PY-WL-118, qualname: "clean_sql_parameterized.parameterized_query", label: FALSE_POSITIVE, note: "sentinel: placeholder SQL with the raw value confined to the parameter tuple — must stay silent"} + clean_boundary_rejection.py: + - {rule_id: PY-WL-102, qualname: "clean_boundary_rejection.validate_token", label: FALSE_POSITIVE, note: "sentinel: boundary with a real raise-on-invalid rejection path — must stay silent"} + clean_exec_const.py: + - {rule_id: PY-WL-107, qualname: "clean_exec_const.const_eval", label: FALSE_POSITIVE, note: "sentinel: eval over a constant-only argument — must stay silent"} + clean_command_const.py: + - {rule_id: PY-WL-108, qualname: "clean_command_const.const_system", label: FALSE_POSITIVE, note: "sentinel: os.system over a literal command — must stay silent"} + - {rule_id: PY-WL-108, qualname: "clean_command_const.quoted_const_run", label: FALSE_POSITIVE, note: "sentinel: shlex-quoted constant through subprocess.run(shell=True) — must stay silent"} diff --git a/tests/corpus/fixtures/validators.py b/tests/corpus/fixtures/validators.py index fe26dbd5..e798d111 100644 --- a/tests/corpus/fixtures/validators.py +++ b/tests/corpus/fixtures/validators.py @@ -5,13 +5,15 @@ @trust_boundary(to_level="ASSURED") -def no_rejection(p): # TP: cannot reject → PY-WL-102 - return p +def no_rejection(p): # TP: cannot reject → PY-WL-102 (laundered shape; bare return-p is PY-WL-119's) + cleaned = p + return cleaned @trust_boundary(to_level="GUARDED") -def no_rejection_guarded(p): # TP: cannot reject → PY-WL-102 - return p +def no_rejection_guarded(p): # TP: cannot reject → PY-WL-102 (laundered shape; bare return-p is PY-WL-119's) + cleaned = p + return cleaned @trust_boundary(to_level="ASSURED") diff --git a/tests/corpus/harness.py b/tests/corpus/harness.py index 017bab83..114b3a5f 100644 --- a/tests/corpus/harness.py +++ b/tests/corpus/harness.py @@ -1,10 +1,18 @@ -"""Labeled-corpus harness (T1.4): run the engine over tests/corpus/fixtures and -reconcile active DEFECT findings against MANIFEST.yaml ground truth. +"""Labeled-corpus harness (T1.4): run the engine over tests/corpus/fixtures plus +tests/corpus/sentinels and reconcile active DEFECT findings against MANIFEST.yaml +ground truth. -Matching key = (path relative to fixtures, rule_id, qualname). Line numbers are +Matching key = (path relative to the scan root, rule_id, qualname). Line numbers are deliberately NOT part of the key so line edits can't break the corpus. The FP rate is measured over *active DEFECT* findings only (the policy surface, PY-WL-*) — engine FACTs/metrics are not findings a user triages. + +Two scan roots: `fixtures/` carries the TRUE_POSITIVE defect shapes and is the +frozen substrate of the Track 2 byte-identity golden (tests/grammar) — it must not +grow casually. `sentinels/` carries the clean-shape FALSE_POSITIVE sentinels the +engine must stay silent on; it is reconciled into the same DEFECT pool but is +invisible to the golden, so sentinels can be added freely. File names must be +unique across both roots (the matching key is root-relative). """ from __future__ import annotations @@ -18,6 +26,7 @@ from wardline.core.run import run_scan CORPUS_ROOT = Path(__file__).parent / "fixtures" +SENTINEL_ROOT = Path(__file__).parent / "sentinels" MANIFEST_PATH = Path(__file__).parent / "MANIFEST.yaml" TRUE_POSITIVE = "TRUE_POSITIVE" @@ -39,7 +48,7 @@ class Reconciliation: active_defects: int false_positives: int unaccounted: list[tuple[str, str, str]] # (path, rule_id, qualname) findings with no manifest entry - stale: list[Expectation] # manifest entries that matched no finding + stale: list[Expectation] # TRUE_POSITIVE entries that matched no finding (silent FP sentinels are passing) @property def fp_rate(self) -> float: @@ -49,32 +58,39 @@ def fp_rate(self) -> float: def load_manifest() -> list[Expectation]: raw = yaml.safe_load(MANIFEST_PATH.read_text()) or {} out: list[Expectation] = [] - for path, entries in (raw.get("fixtures") or {}).items(): - for entry in entries or []: - label = entry["label"] - if label not in _LABELS: - raise ValueError(f"{path}: bad label {label!r} (want one of {sorted(_LABELS)})") - out.append( - Expectation( - path=path, - rule_id=entry["rule_id"], - qualname=entry["qualname"], - label=label, - note=entry.get("note", ""), + seen_paths: set[str] = set() + for section in ("fixtures", "sentinels"): + for path, entries in (raw.get(section) or {}).items(): + if path in seen_paths: + raise ValueError(f"{path}: file name reused across scan roots — the matching key is root-relative") + seen_paths.add(path) + for entry in entries or []: + label = entry["label"] + if label not in _LABELS: + raise ValueError(f"{path}: bad label {label!r} (want one of {sorted(_LABELS)})") + if section == "sentinels" and label != FALSE_POSITIVE: + raise ValueError(f"{path}: sentinels/ holds clean shapes only — label must be FALSE_POSITIVE") + out.append( + Expectation( + path=path, + rule_id=entry["rule_id"], + qualname=entry["qualname"], + label=label, + note=entry.get("note", ""), + ) ) - ) return out def reconcile() -> Reconciliation: - result = run_scan(CORPUS_ROOT) + findings = [f for root in (CORPUS_ROOT, SENTINEL_ROOT) for f in run_scan(root).findings] expectations = load_manifest() by_key: dict[tuple[str, str, str], Expectation] = {(e.path, e.rule_id, e.qualname): e for e in expectations} matched_keys: set[tuple[str, str, str]] = set() active_defects = 0 false_positives = 0 unaccounted: list[tuple[str, str, str]] = [] - for finding in result.findings: + for finding in findings: if finding.kind is not Kind.DEFECT or finding.suppressed is not SuppressionState.ACTIVE: continue if finding.maturity is Maturity.PREVIEW: @@ -88,7 +104,11 @@ def reconcile() -> Reconciliation: matched_keys.add(key) if expectation.label == FALSE_POSITIVE: false_positives += 1 - stale = [e for e in expectations if (e.path, e.rule_id, e.qualname) not in matched_keys] + # Staleness only applies to TRUE_POSITIVE entries: a FALSE_POSITIVE sentinel the + # engine stays silent on is the engine behaving correctly, not a dead manifest row. + stale = [ + e for e in expectations if e.label == TRUE_POSITIVE and (e.path, e.rule_id, e.qualname) not in matched_keys + ] return Reconciliation( active_defects=active_defects, false_positives=false_positives, diff --git a/tests/corpus/sentinels/clean_boundary_rejection.py b/tests/corpus/sentinels/clean_boundary_rejection.py new file mode 100644 index 00000000..7624f064 --- /dev/null +++ b/tests/corpus/sentinels/clean_boundary_rejection.py @@ -0,0 +1,10 @@ +# FP sentinel for PY-WL-102: a trust boundary with a real raise-on-invalid rejection +# path CAN say "no", so the degenerate-boundary rule must stay silent. +from wardline.decorators import trust_boundary + + +@trust_boundary(to_level="ASSURED") +def validate_token(p): # FP sentinel: raise path = real rejection + if not isinstance(p, str) or not p.isalnum(): + raise ValueError("reject") + return p diff --git a/tests/corpus/sentinels/clean_command_const.py b/tests/corpus/sentinels/clean_command_const.py new file mode 100644 index 00000000..c6ca39a9 --- /dev/null +++ b/tests/corpus/sentinels/clean_command_const.py @@ -0,0 +1,21 @@ +# FP sentinels for PY-WL-108: constant-only command arguments — os.system over a +# literal, and a shlex-quoted constant through subprocess.run. No untrusted value +# reaches either command sink, so the engine must stay silent on both. +import os +import shlex +import subprocess + +from wardline.decorators import trusted + + +@trusted(level="ASSURED") +def const_system(p): # FP sentinel: literal command string + os.system("ls -l /tmp") + return 1 + + +@trusted(level="ASSURED") +def quoted_const_run(p): # FP sentinel: shlex-quoted constant command + cmd = shlex.quote("/usr/bin/true") + subprocess.run(cmd, shell=True) + return 1 diff --git a/tests/corpus/sentinels/clean_exec_const.py b/tests/corpus/sentinels/clean_exec_const.py new file mode 100644 index 00000000..b24cc156 --- /dev/null +++ b/tests/corpus/sentinels/clean_exec_const.py @@ -0,0 +1,8 @@ +# FP sentinel for PY-WL-107: eval over a constant-only argument — no untrusted value +# can reach the dynamic-exec sink, so the engine must stay silent. +from wardline.decorators import trusted + + +@trusted(level="ASSURED") +def const_eval(p): # FP sentinel: literal source text, nothing tainted flows in + return eval("1 + 1") diff --git a/tests/corpus/sentinels/clean_sql_parameterized.py b/tests/corpus/sentinels/clean_sql_parameterized.py new file mode 100644 index 00000000..0e24aca2 --- /dev/null +++ b/tests/corpus/sentinels/clean_sql_parameterized.py @@ -0,0 +1,18 @@ +# FP sentinel for PY-WL-118: a properly parameterized sqlite query — the raw value +# travels in the parameter tuple, never the SQL text, so the engine must stay silent. +import sqlite3 + +from wardline.decorators import external_boundary, trusted + + +@external_boundary +def read_raw(p): + return p + + +@trusted(level="ASSURED") +def parameterized_query(p): # FP sentinel: placeholder SQL, taint confined to params + name = read_raw(p) + conn = sqlite3.connect(":memory:") + conn.execute("SELECT * FROM users WHERE name = ?", (name,)) + return 1 diff --git a/tests/corpus/test_fp_rate.py b/tests/corpus/test_fp_rate.py index 8aad110b..cf40bfc7 100644 --- a/tests/corpus/test_fp_rate.py +++ b/tests/corpus/test_fp_rate.py @@ -2,11 +2,15 @@ FP rate = active DEFECTs labeled FALSE_POSITIVE / total active DEFECTs, gated <= 5%. The corpus is sized so a single mislabel cannot trivially breach the budget. + +FALSE_POSITIVE entries are clean-shape sentinels: the engine must NOT fire on them. +Silent sentinel = passing (not stale); fired sentinel = a live FP counted against the +budget. The corpus must carry sentinels so the gate exercises real reconciliation. """ from __future__ import annotations -from corpus.harness import reconcile # type: ignore[import-not-found] +from corpus.harness import FALSE_POSITIVE, load_manifest, reconcile # type: ignore[import-not-found] def test_fp_rate_within_budget(): @@ -24,6 +28,45 @@ def test_fp_rate_within_budget(): assert rec.fp_rate <= 0.05, f"FP rate {rec.fp_rate:.1%} exceeds the 5% budget" +def test_corpus_carries_false_positive_sentinels(): + # The gate only exercises real reconciliation if the corpus mixes labels: clean-shape + # sentinels (FALSE_POSITIVE) alongside the TRUE_POSITIVE defects. A silent sentinel is + # the engine behaving correctly, so it must NOT be reported stale. + expectations = load_manifest() + sentinels = [e for e in expectations if e.label == FALSE_POSITIVE] + assert len(sentinels) >= 3, ( + f"corpus has {len(sentinels)} FALSE_POSITIVE sentinels (need >= 3 so the FP-rate " + "gate computes over a mixed corpus, not a vacuous all-TP one)" + ) + rec = reconcile() + silent_stale = [e for e in rec.stale if e.label == FALSE_POSITIVE] + assert not silent_stale, ( + "silent FALSE_POSITIVE sentinels were reported stale — a sentinel the engine " + "does not fire on is PASSING, not stale: " + f"{[(e.path, e.rule_id, e.qualname) for e in silent_stale]}" + ) + + +def test_fired_sentinel_counts_against_budget(monkeypatch, tmp_path): + # End-to-end fired-sentinel path: relabel a known-firing fixture entry as + # FALSE_POSITIVE in a scratch manifest — the fired finding must be counted as a + # live FP (against the budget), not reported stale or unaccounted. + import corpus.harness as harness # type: ignore[import-not-found] + + original = harness.MANIFEST_PATH.read_text() + relabel_target = 'qualname: "deser_sink.loads_untrusted", label: TRUE_POSITIVE' + assert relabel_target in original, "relabel target drifted — pick another known-firing entry" + scratch = tmp_path / "MANIFEST.yaml" + scratch.write_text(original.replace(relabel_target, relabel_target.replace("TRUE_POSITIVE", "FALSE_POSITIVE"))) + monkeypatch.setattr(harness, "MANIFEST_PATH", scratch) + + rec = reconcile() + assert rec.false_positives == 1 + assert rec.fp_rate == 1 / rec.active_defects + assert not rec.unaccounted + assert "deser_sink.loads_untrusted" not in {e.qualname for e in rec.stale} + + def test_reconciliation_fp_rate_arithmetic(): # Directly exercise the FP-rate computation on the FALSE_POSITIVE path, which the # live corpus (all TRUE_POSITIVE today) never hits. Guards the gate's own math. diff --git a/tests/golden/identity/corpus/META.json b/tests/golden/identity/corpus/META.json index 0697150e..b3da5a9b 100644 --- a/tests/golden/identity/corpus/META.json +++ b/tests/golden/identity/corpus/META.json @@ -1,5 +1,5 @@ { "corpus_version": 4, "fingerprint_scheme": "wlfp2", - "reason": "rekey (wlfp2/wardline-8654423823): drop line_start + entity-relative move-stable discriminators" + "reason": "2026-06-10/11 python-plugin hardening (two regens, scheme wlfp2 unchanged): (1) PY-WL-108/112 base severity WARN->ERROR (command injection calibrated to the SQLi exploit class); (2) Compare-position resolution + nested-def return summaries: actual_return precision UNKNOWN_RAW->INTEGRAL for comparison returns and local helpers. Prior rekey provenance: wlfp2/wardline-8654423823 (drop line_start + entity-relative move-stable discriminators)." } diff --git a/tests/golden/identity/corpus/sampleapp.json b/tests/golden/identity/corpus/sampleapp.json index 569be10e..2fea242e 100644 --- a/tests/golden/identity/corpus/sampleapp.json +++ b/tests/golden/identity/corpus/sampleapp.json @@ -981,7 +981,7 @@ "qualname": "models.Money.__lt__", "schema_version": "wardline-taint-1", "taint": { - "actual_return": "UNKNOWN_RAW", + "actual_return": "INTEGRAL", "contributing_callee_qualname": null, "declared_return": "UNKNOWN_RAW", "resolved_call_count": 0, @@ -1149,7 +1149,7 @@ "qualname": "repository.Repository.__contains__", "schema_version": "wardline-taint-1", "taint": { - "actual_return": "UNKNOWN_RAW", + "actual_return": "INTEGRAL", "contributing_callee_qualname": null, "declared_return": "UNKNOWN_RAW", "resolved_call_count": 0, diff --git a/tests/golden/identity/corpus/sinks.json b/tests/golden/identity/corpus/sinks.json index b6772c52..f733fdef 100644 --- a/tests/golden/identity/corpus/sinks.json +++ b/tests/golden/identity/corpus/sinks.json @@ -714,7 +714,7 @@ "qualname": "wardline_sinks.run_export", "related_entities": [], "rule_id": "PY-WL-112", - "severity": "WARN", + "severity": "ERROR", "suggestion": null, "suppression_reason": null, "suppression_state": "active" @@ -1512,7 +1512,7 @@ ] } ], - "level": "warning", + "level": "error", "locations": [ { "physicalLocation": { @@ -1532,7 +1532,7 @@ "wardlineFingerprint/v2": "13c25150c1e15425e6ae5f9d99f67ec09dd5ffb4b60ac49e20367a0ad6240c6f" }, "properties": { - "internalSeverity": "WARN", + "internalSeverity": "ERROR", "kind": "defect", "qualname": "wardline_sinks.run_export", "wardlineProperties": { diff --git a/tests/golden/identity/corpus/stress.json b/tests/golden/identity/corpus/stress.json index e5e0c245..4afb9095 100644 --- a/tests/golden/identity/corpus/stress.json +++ b/tests/golden/identity/corpus/stress.json @@ -342,7 +342,7 @@ "qualname": "identity_stress.outer", "schema_version": "wardline-taint-1", "taint": { - "actual_return": "UNKNOWN_RAW", + "actual_return": "INTEGRAL", "contributing_callee_qualname": null, "declared_return": "UNKNOWN_RAW", "resolved_call_count": 0, diff --git a/tests/grammar/golden/builtin_findings.jsonl b/tests/grammar/golden/builtin_findings.jsonl index 43ba80f4..3167e5fd 100644 --- a/tests/grammar/golden/builtin_findings.jsonl +++ b/tests/grammar/golden/builtin_findings.jsonl @@ -22,8 +22,8 @@ {"confidence": null, "fingerprint": "feb63a22babf3e084187866a7d54170c61d22baa44470a347fa93a990a7ff778", "kind": "defect", "location": {"col_end": 14, "col_start": 0, "line_end": 57, "line_start": 54, "path": "tests/corpus/fixtures/more_shapes.py"}, "maturity": "stable", "message": "tests.corpus.fixtures.more_shapes.augassign_raw declares return trust ASSURED but actually returns UNKNOWN_RAW (less trusted) — untrusted data reaches a trusted producer", "properties": {"actual_return": "UNKNOWN_RAW", "declared_return": "ASSURED"}, "qualname": "tests.corpus.fixtures.more_shapes.augassign_raw", "related_entities": [], "rule_id": "PY-WL-101", "severity": "ERROR", "suggestion": null, "suppression_reason": null, "suppression_state": "active"} {"confidence": null, "fingerprint": "48db6ff8bfe3be3a2ffe31918ae202123b7f25f7ed1554b3c5897a9a3bf777ac", "kind": "defect", "location": {"col_end": 22, "col_start": 0, "line_end": 65, "line_start": 61, "path": "tests/corpus/fixtures/more_shapes.py"}, "maturity": "stable", "message": "tests.corpus.fixtures.more_shapes.launders_through_broken_boundary declares return trust ASSURED but actually returns UNKNOWN_RAW (less trusted) — untrusted data reaches a trusted producer", "properties": {"actual_return": "UNKNOWN_RAW", "declared_return": "ASSURED"}, "qualname": "tests.corpus.fixtures.more_shapes.launders_through_broken_boundary", "related_entities": [], "rule_id": "PY-WL-101", "severity": "ERROR", "suggestion": null, "suppression_reason": null, "suppression_state": "active"} {"confidence": null, "fingerprint": "988c22d25a99f03ed8e75f487e0743fdc5a9faf2d32d785cb2992ad9977dc28c", "kind": "defect", "location": {"col_end": 18, "col_start": 0, "line_end": 71, "line_start": 69, "path": "tests/corpus/fixtures/more_shapes.py"}, "maturity": "stable", "message": "tests.corpus.fixtures.more_shapes.passthrough_no_check declares a trust boundary (EXTERNAL_RAW -> ASSURED) but has no rejection path (no raise / no falsy return) — it cannot validate", "properties": {"body_taint": "EXTERNAL_RAW", "return_taint": "ASSURED"}, "qualname": "tests.corpus.fixtures.more_shapes.passthrough_no_check", "related_entities": [], "rule_id": "PY-WL-102", "severity": "ERROR", "suggestion": null, "suppression_reason": null, "suppression_state": "active"} -{"confidence": null, "fingerprint": "b780ea20891c98e4a01af2d3e4dd22c9977d2a1f266efeca818d3c9e820d46ba", "kind": "defect", "location": {"col_end": 12, "col_start": 0, "line_end": 9, "line_start": 8, "path": "tests/corpus/fixtures/validators.py"}, "maturity": "stable", "message": "tests.corpus.fixtures.validators.no_rejection declares a trust boundary (EXTERNAL_RAW -> ASSURED) but has no rejection path (no raise / no falsy return) — it cannot validate", "properties": {"body_taint": "EXTERNAL_RAW", "return_taint": "ASSURED"}, "qualname": "tests.corpus.fixtures.validators.no_rejection", "related_entities": [], "rule_id": "PY-WL-102", "severity": "ERROR", "suggestion": null, "suppression_reason": null, "suppression_state": "active"} -{"confidence": null, "fingerprint": "c0c5caa3685eef1f73f8e2f97605e9806a0ad444170317d5c5137c3f8469e8ee", "kind": "defect", "location": {"col_end": 12, "col_start": 0, "line_end": 14, "line_start": 13, "path": "tests/corpus/fixtures/validators.py"}, "maturity": "stable", "message": "tests.corpus.fixtures.validators.no_rejection_guarded declares a trust boundary (EXTERNAL_RAW -> GUARDED) but has no rejection path (no raise / no falsy return) — it cannot validate", "properties": {"body_taint": "EXTERNAL_RAW", "return_taint": "GUARDED"}, "qualname": "tests.corpus.fixtures.validators.no_rejection_guarded", "related_entities": [], "rule_id": "PY-WL-102", "severity": "ERROR", "suggestion": null, "suppression_reason": null, "suppression_state": "active"} +{"confidence": null, "fingerprint": "b780ea20891c98e4a01af2d3e4dd22c9977d2a1f266efeca818d3c9e820d46ba", "kind": "defect", "location": {"col_end": 18, "col_start": 0, "line_end": 10, "line_start": 8, "path": "tests/corpus/fixtures/validators.py"}, "maturity": "stable", "message": "tests.corpus.fixtures.validators.no_rejection declares a trust boundary (EXTERNAL_RAW -> ASSURED) but has no rejection path (no raise / no falsy return) — it cannot validate", "properties": {"body_taint": "EXTERNAL_RAW", "return_taint": "ASSURED"}, "qualname": "tests.corpus.fixtures.validators.no_rejection", "related_entities": [], "rule_id": "PY-WL-102", "severity": "ERROR", "suggestion": null, "suppression_reason": null, "suppression_state": "active"} +{"confidence": null, "fingerprint": "c0c5caa3685eef1f73f8e2f97605e9806a0ad444170317d5c5137c3f8469e8ee", "kind": "defect", "location": {"col_end": 18, "col_start": 0, "line_end": 16, "line_start": 14, "path": "tests/corpus/fixtures/validators.py"}, "maturity": "stable", "message": "tests.corpus.fixtures.validators.no_rejection_guarded declares a trust boundary (EXTERNAL_RAW -> GUARDED) but has no rejection path (no raise / no falsy return) — it cannot validate", "properties": {"body_taint": "EXTERNAL_RAW", "return_taint": "GUARDED"}, "qualname": "tests.corpus.fixtures.validators.no_rejection_guarded", "related_entities": [], "rule_id": "PY-WL-102", "severity": "ERROR", "suggestion": null, "suppression_reason": null, "suppression_state": "active"} {"confidence": null, "fingerprint": "c965eff4deb1719dd123d7a179cba25ae6590e324b0cb31637cf04f540cd8816", "kind": "defect", "location": {"col_end": null, "col_start": null, "line_end": null, "line_start": 16, "path": "tests/corpus/fixtures/exceptions.py"}, "maturity": "stable", "message": "tests.corpus.fixtures.exceptions.broad_handler: broad exception handler at line 16", "properties": {"tier": "INTEGRAL"}, "qualname": "tests.corpus.fixtures.exceptions.broad_handler", "related_entities": [], "rule_id": "PY-WL-103", "severity": "WARN", "suggestion": null, "suppression_reason": null, "suppression_state": "active"} {"confidence": null, "fingerprint": "c50617814cd7cac9b17ce5e07647a1bf3429d2ffd9d50e98481a128cb0037fad", "kind": "defect", "location": {"col_end": null, "col_start": null, "line_end": null, "line_start": 24, "path": "tests/corpus/fixtures/exceptions.py"}, "maturity": "stable", "message": "tests.corpus.fixtures.exceptions.silent_handler: exception silently swallowed at line 24", "properties": {"tier": "INTEGRAL"}, "qualname": "tests.corpus.fixtures.exceptions.silent_handler", "related_entities": [], "rule_id": "PY-WL-104", "severity": "WARN", "suggestion": null, "suppression_reason": null, "suppression_state": "active"} {"confidence": null, "fingerprint": "6c2e8485d5076b983edf80ad5081109f3f58e204e5bd5e8fbd50a109e54da101", "kind": "defect", "location": {"col_end": 12, "col_start": 0, "line_end": 8, "line_start": 7, "path": "tests/corpus/fixtures/contradictory.py"}, "maturity": "stable", "message": "tests.corpus.fixtures.contradictory.conflicting carries contradictory trust markers (external_boundary+trusted); the engine resolves the clash to the least-trusted seed, silently ignoring the rest", "properties": {"markers": "external_boundary+trusted"}, "qualname": "tests.corpus.fixtures.contradictory.conflicting", "related_entities": [], "rule_id": "PY-WL-110", "severity": "WARN", "suggestion": null, "suppression_reason": null, "suppression_state": "active"} @@ -31,6 +31,6 @@ {"confidence": null, "fingerprint": "b480326767381cab9c1995241e94f6cb9dc6308281f2842f10a9f036d1e0760e", "kind": "defect", "location": {"col_end": null, "col_start": null, "line_end": null, "line_start": 16, "path": "tests/corpus/fixtures/trusted_callee.py"}, "maturity": "stable", "message": "tests.corpus.fixtures.trusted_callee.handler: EXTERNAL_RAW (untrusted) data passed to trusted producer tests.corpus.fixtures.trusted_callee.store() at line 16", "properties": {"arg_taint": "EXTERNAL_RAW", "callee": "tests.corpus.fixtures.trusted_callee.store", "callee_body": "ASSURED"}, "qualname": "tests.corpus.fixtures.trusted_callee.handler", "related_entities": [], "rule_id": "PY-WL-105", "severity": "ERROR", "suggestion": null, "suppression_reason": null, "suppression_state": "active"} {"confidence": null, "fingerprint": "d666c4d2616986f1f3aeb1d743ed5badb0d67848b94b51b028322f714d684148", "kind": "defect", "location": {"col_end": null, "col_start": null, "line_end": null, "line_start": 15, "path": "tests/corpus/fixtures/deser_sink.py"}, "maturity": "stable", "message": "tests.corpus.fixtures.deser_sink.loads_untrusted: EXTERNAL_RAW (untrusted) data reaches the deserialization sink pickle.loads() at line 15", "properties": {"arg_taint": "EXTERNAL_RAW", "sink": "pickle.loads", "tier": "ASSURED"}, "qualname": "tests.corpus.fixtures.deser_sink.loads_untrusted", "related_entities": [], "rule_id": "PY-WL-106", "severity": "WARN", "suggestion": null, "suppression_reason": null, "suppression_state": "active"} {"confidence": null, "fingerprint": "090abfc382d13eda6255b2948426fca197c6d5fbe177845860a9368a50a6178e", "kind": "defect", "location": {"col_end": null, "col_start": null, "line_end": null, "line_start": 13, "path": "tests/corpus/fixtures/exec_sink.py"}, "maturity": "stable", "message": "tests.corpus.fixtures.exec_sink.evals_untrusted: EXTERNAL_RAW (untrusted) data reaches the dynamic-code-execution sink eval() at line 13", "properties": {"arg_taint": "EXTERNAL_RAW", "sink": "eval", "tier": "ASSURED"}, "qualname": "tests.corpus.fixtures.exec_sink.evals_untrusted", "related_entities": [], "rule_id": "PY-WL-107", "severity": "WARN", "suggestion": null, "suppression_reason": null, "suppression_state": "active"} -{"confidence": null, "fingerprint": "89495235b260639eb40e0aef10c4c40dee8d0fd3ab3767fde1f65ff7cc0f00ac", "kind": "defect", "location": {"col_end": null, "col_start": null, "line_end": null, "line_start": 15, "path": "tests/corpus/fixtures/command_sink.py"}, "maturity": "stable", "message": "tests.corpus.fixtures.command_sink.runs_untrusted: EXTERNAL_RAW (untrusted) data reaches the OS-command sink os.system() at line 15", "properties": {"arg_taint": "EXTERNAL_RAW", "sink": "os.system", "tier": "ASSURED"}, "qualname": "tests.corpus.fixtures.command_sink.runs_untrusted", "related_entities": [], "rule_id": "PY-WL-108", "severity": "WARN", "suggestion": null, "suppression_reason": null, "suppression_state": "active"} -{"confidence": null, "fingerprint": "d0431eaf7203c0e40e57e855c8e9820062f75146a50e371f04021b34be50489d", "kind": "defect", "location": {"col_end": null, "col_start": null, "line_end": null, "line_start": 24, "path": "tests/corpus/fixtures/shadow_launder.py"}, "maturity": "stable", "message": "tests.corpus.fixtures.shadow_launder.shadowed_sink: EXTERNAL_RAW (untrusted) data reaches the OS-command sink os.system() at line 24", "properties": {"arg_taint": "EXTERNAL_RAW", "sink": "os.system", "tier": "ASSURED"}, "qualname": "tests.corpus.fixtures.shadow_launder.shadowed_sink", "related_entities": [], "rule_id": "PY-WL-108", "severity": "WARN", "suggestion": null, "suppression_reason": null, "suppression_state": "active"} -{"confidence": null, "fingerprint": "db0b0dc75b64818b2d48960f83a2c457f2d433d40277fc0e724bc806ad7c84a6", "kind": "defect", "location": {"col_end": null, "col_start": null, "line_end": null, "line_start": 21, "path": "tests/corpus/fixtures/shadow_launder_bare.py"}, "maturity": "stable", "message": "tests.corpus.fixtures.shadow_launder_bare.bare_shadow_sink: EXTERNAL_RAW (untrusted) data reaches the OS-command sink os.system() at line 21", "properties": {"arg_taint": "EXTERNAL_RAW", "sink": "os.system", "tier": "ASSURED"}, "qualname": "tests.corpus.fixtures.shadow_launder_bare.bare_shadow_sink", "related_entities": [], "rule_id": "PY-WL-108", "severity": "WARN", "suggestion": null, "suppression_reason": null, "suppression_state": "active"} \ No newline at end of file +{"confidence": null, "fingerprint": "89495235b260639eb40e0aef10c4c40dee8d0fd3ab3767fde1f65ff7cc0f00ac", "kind": "defect", "location": {"col_end": null, "col_start": null, "line_end": null, "line_start": 15, "path": "tests/corpus/fixtures/command_sink.py"}, "maturity": "stable", "message": "tests.corpus.fixtures.command_sink.runs_untrusted: EXTERNAL_RAW (untrusted) data reaches the OS-command sink os.system() at line 15", "properties": {"arg_taint": "EXTERNAL_RAW", "sink": "os.system", "tier": "ASSURED"}, "qualname": "tests.corpus.fixtures.command_sink.runs_untrusted", "related_entities": [], "rule_id": "PY-WL-108", "severity": "ERROR", "suggestion": null, "suppression_reason": null, "suppression_state": "active"} +{"confidence": null, "fingerprint": "d0431eaf7203c0e40e57e855c8e9820062f75146a50e371f04021b34be50489d", "kind": "defect", "location": {"col_end": null, "col_start": null, "line_end": null, "line_start": 24, "path": "tests/corpus/fixtures/shadow_launder.py"}, "maturity": "stable", "message": "tests.corpus.fixtures.shadow_launder.shadowed_sink: EXTERNAL_RAW (untrusted) data reaches the OS-command sink os.system() at line 24", "properties": {"arg_taint": "EXTERNAL_RAW", "sink": "os.system", "tier": "ASSURED"}, "qualname": "tests.corpus.fixtures.shadow_launder.shadowed_sink", "related_entities": [], "rule_id": "PY-WL-108", "severity": "ERROR", "suggestion": null, "suppression_reason": null, "suppression_state": "active"} +{"confidence": null, "fingerprint": "db0b0dc75b64818b2d48960f83a2c457f2d433d40277fc0e724bc806ad7c84a6", "kind": "defect", "location": {"col_end": null, "col_start": null, "line_end": null, "line_start": 21, "path": "tests/corpus/fixtures/shadow_launder_bare.py"}, "maturity": "stable", "message": "tests.corpus.fixtures.shadow_launder_bare.bare_shadow_sink: EXTERNAL_RAW (untrusted) data reaches the OS-command sink os.system() at line 21", "properties": {"arg_taint": "EXTERNAL_RAW", "sink": "os.system", "tier": "ASSURED"}, "qualname": "tests.corpus.fixtures.shadow_launder_bare.bare_shadow_sink", "related_entities": [], "rule_id": "PY-WL-108", "severity": "ERROR", "suggestion": null, "suppression_reason": null, "suppression_state": "active"} \ No newline at end of file diff --git a/tests/grammar/test_analyzer_wiring.py b/tests/grammar/test_analyzer_wiring.py index cac3fbc5..967c8719 100644 --- a/tests/grammar/test_analyzer_wiring.py +++ b/tests/grammar/test_analyzer_wiring.py @@ -33,6 +33,12 @@ "PY-WL-118", "PY-WL-119", "PY-WL-120", + "PY-WL-121", + "PY-WL-122", + "PY-WL-123", + "PY-WL-124", + "PY-WL-125", + "PY-WL-126", ] diff --git a/tests/grammar/test_grammar_model.py b/tests/grammar/test_grammar_model.py index c5e513d9..904d129c 100644 --- a/tests/grammar/test_grammar_model.py +++ b/tests/grammar/test_grammar_model.py @@ -56,6 +56,12 @@ def test_default_grammar_has_builtin_marker_namespaces_and_all_rules() -> None: "PY-WL-118", "PY-WL-119", "PY-WL-120", + "PY-WL-121", + "PY-WL-122", + "PY-WL-123", + "PY-WL-124", + "PY-WL-125", + "PY-WL-126", ] diff --git a/tests/grammar/test_output_determinism.py b/tests/grammar/test_output_determinism.py new file mode 100644 index 00000000..54c55ee7 --- /dev/null +++ b/tests/grammar/test_output_determinism.py @@ -0,0 +1,56 @@ +"""Run-to-run output determinism guard (wardline-e159060db7). + +The product promises a stable finding stream (non-flaky gate, byte-stable +baselines/attestations). That property is currently held by convention scattered +across ~10 engine sites (sorted() discovery, Tarjan node/neighbor sorting, +least_trusted commutativity over unsorted callee sets, ...). The golden oracle +(``test_golden_oracle.py``) pins ONE run against a frozen golden, and only for +STABLE-maturity findings — it would catch drift, but a nondeterministic PREVIEW +rule or per-run engine state would slip past it. + +This is the single guard at the output boundary: two INDEPENDENT analyzer runs +over the fixed corpus must produce byte-identical full streams (every maturity, +every kind), in identical order. +""" + +from __future__ import annotations + +from pathlib import Path + +from wardline.core.config import WardlineConfig +from wardline.core.taints import TRUST_RANK, TaintState +from wardline.scanner.analyzer import WardlineAnalyzer + +REPO_ROOT = Path(__file__).resolve().parents[2] +_CORPUS = REPO_ROOT / "tests" / "corpus" / "fixtures" + + +def _full_stream() -> str: + """One complete, ordered, serialized analyzer run over the labeled corpus. + + Unlike ``golden_harness.produce_stream`` this does NOT filter to STABLE + maturity: PREVIEW rules (PY-WL-116..126) must be order-deterministic too — + they feed baselines and the delta gate even before graduation. + """ + files = sorted(_CORPUS.rglob("*.py")) + analyzer = WardlineAnalyzer() + findings = analyzer.analyze(files, WardlineConfig(), root=REPO_ROOT) + return "\n".join(f.to_jsonl() for f in findings) + + +def test_analyzer_output_is_byte_identical_across_runs() -> None: + first = _full_stream() + second = _full_stream() + assert first, "corpus produced an empty stream — fixture path broken" + assert first == second, "analyzer output differs between identical runs" + + +def test_trust_rank_is_injective() -> None: + # Load-bearing for order stability: callee sets are aggregated with the + # rank-meet ``least_trusted`` via ``reduce`` over an UNSORTED set + # (propagation.py); that is only order-independent (commutative + + # associative) while TRUST_RANK assigns every TaintState a DISTINCT rank. + # Two states sharing a rank would make the reduce pick whichever the set + # yields first — silent per-run nondeterminism. + ranks = [TRUST_RANK[t] for t in TaintState] + assert len(set(ranks)) == len(list(TaintState)) diff --git a/tests/unit/scanner/rules/test_assert_only_boundary.py b/tests/unit/scanner/rules/test_assert_only_boundary.py index 37ad1ec4..4fdd751c 100644 --- a/tests/unit/scanner/rules/test_assert_only_boundary.py +++ b/tests/unit/scanner/rules/test_assert_only_boundary.py @@ -36,7 +36,8 @@ def test_assert_only_boundary_fires_111_not_102(tmp_path) -> None: def test_no_rejection_at_all_is_102_not_111(tmp_path) -> None: - ctx = _analyze(tmp_path, _BOUNDARY + " return p\n") + # Laundered pass-through (NOT the bare degenerate shape, which is PY-WL-119's). + ctx = _analyze(tmp_path, _BOUNDARY + " x = p\n return x\n") assert AssertOnlyBoundary().check(ctx) == [] assert _ids(ctx) == {"PY-WL-102"} @@ -72,3 +73,43 @@ def test_trusted_producer_is_not_a_boundary(tmp_path) -> None: def test_undecorated_assert_is_silent(tmp_path) -> None: ctx = _analyze(tmp_path, "def v(p):\n assert p\n return p\n") assert AssertOnlyBoundary().check(ctx) == [] + + +def test_assert_plus_raising_helper_is_clean(tmp_path) -> None: + # boundary.json FP 2: a same-module helper that raises survives `python -O`, so + # the assert is NOT the sole rejection — 111 (and 102) must stay silent. + ctx = _analyze( + tmp_path, + "from wardline.decorators import trust_boundary\n" + "def _require_nonempty(p):\n if not p:\n raise ValueError('empty')\n" + "@trust_boundary(to_level='ASSURED')\n" + "def v(p):\n assert isinstance(p, str)\n _require_nonempty(p)\n return p\n", + ) + assert AssertOnlyBoundary().check(ctx) == [] + assert _ids(ctx) == set() + + +def test_assert_plus_non_raising_helper_still_fires_111(tmp_path) -> None: + # SOUNDNESS GUARD: a helper that cannot raise (logs and returns) does not rescue + # the boundary — the assert remains the sole rejection. + ctx = _analyze( + tmp_path, + "from wardline.decorators import trust_boundary\n" + "def _log(p):\n print(p)\n return p\n" + "@trust_boundary(to_level='ASSURED')\n" + "def v(p):\n assert p\n _log(p)\n return p\n", + ) + assert [(f.rule_id, f.qualname) for f in AssertOnlyBoundary().check(ctx)] == [("PY-WL-111", "m.v")] + + +def test_assert_plus_assert_only_helper_still_fires_111(tmp_path) -> None: + # A helper whose only rejection is itself an assert vanishes under -O too — + # it must not count as a real rejection. + ctx = _analyze( + tmp_path, + "from wardline.decorators import trust_boundary\n" + "def _check(p):\n assert p\n" + "@trust_boundary(to_level='ASSURED')\n" + "def v(p):\n assert isinstance(p, str)\n _check(p)\n return p\n", + ) + assert [(f.rule_id, f.qualname) for f in AssertOnlyBoundary().check(ctx)] == [("PY-WL-111", "m.v")] diff --git a/tests/unit/scanner/rules/test_ast_helpers.py b/tests/unit/scanner/rules/test_ast_helpers.py index 40fc7524..8e6e5d90 100644 --- a/tests/unit/scanner/rules/test_ast_helpers.py +++ b/tests/unit/scanner/rules/test_ast_helpers.py @@ -3,12 +3,17 @@ import ast import textwrap +from wardline.scanner.index import discover_file_entities from wardline.scanner.rules._ast_helpers import ( asserts_are_sole_rejection, + block_has_real_rejection, + has_real_rejection, has_rejection_path, is_broad_except, + is_degenerate_boundary, is_silent_handler, own_except_handlers, + rejecting_helper_calls, ) @@ -41,6 +46,162 @@ def test_asserts_are_sole_rejection() -> None: assert not asserts_are_sole_rejection(_fn("def f(p):\n return p\n")) +def test_has_rejection_path_sees_conditional_expression_falsy_branch() -> None: + # `return m.group(0) if m else None` is the ternary form of the recognised + # `if not m: return None` rejection — semantically identical, must count. + assert has_rejection_path(_fn("def f(p):\n m = check(p)\n return m.group(0) if m else None\n")) + assert has_rejection_path(_fn("def f(p):\n return p if ok(p) else False\n")) + # nested ternary: a falsy branch anywhere in the conditional tree counts + assert has_rejection_path(_fn("def f(p):\n return (a if x else None) if c else b\n")) + # a ternary with NO falsy branch is not a rejection + assert not has_rejection_path(_fn("def f(p):\n return a if c else b\n")) + + +def test_has_rejection_path_curated_raising_conversion_returns() -> None: + # Curated validate-by-construction shapes: the conversion/lookup raises on + # every invalid input (ValueError / KeyError), so the boundary CAN reject. + assert has_rejection_path(_fn("def f(p):\n return int(p)\n")) + assert has_rejection_path(_fn("def f(p):\n return float(p)\n")) + assert has_rejection_path(_fn("def f(p):\n return Decimal(p)\n")) + assert has_rejection_path(_fn("def f(p):\n return decimal.Decimal(p)\n")) + assert has_rejection_path(_fn("def f(p):\n return uuid.UUID(p)\n")) + # Enum subscript and mapping/allowlist subscript lookup raise KeyError + assert has_rejection_path(_fn("def f(p):\n return Color[p]\n")) + assert has_rejection_path(_fn("def f(p):\n return ALLOWED[p]\n")) + + +def test_raising_conversion_set_is_curated_not_arbitrary() -> None: + # SOUNDNESS: an arbitrary call is NOT a rejection (that would be an FN hole). + assert not has_rejection_path(_fn("def f(p):\n return frobnicate(p)\n")) + # str()/bool() never reject; not in the curated set + assert not has_rejection_path(_fn("def f(p):\n return str(p)\n")) + # a conversion of nothing / of a constant validates nothing + assert not has_rejection_path(_fn("def f(p):\n return int()\n")) + assert not has_rejection_path(_fn("def f(p):\n return int('5')\n")) + # a constant subscript is positional access, not a validating lookup + assert not has_rejection_path(_fn("def f(p):\n return parts[0]\n")) + + +def test_asserts_are_sole_rejection_sees_extended_rejection_returns() -> None: + # A raising-conversion or ternary-falsy return is a REAL (non-assert) rejection, + # so the assert is not the sole rejection -> PY-WL-111 stays silent. + assert not asserts_are_sole_rejection(_fn("def f(p):\n assert p\n return int(p)\n")) + assert not asserts_are_sole_rejection(_fn("def f(p):\n assert p\n m = c(p)\n return m.g() if m else None\n")) + + +def test_has_real_rejection_excludes_assert() -> None: + # has_real_rejection is the production-surviving predicate: assert alone is NOT real. + assert not has_real_rejection(_fn("def f(p):\n assert p\n return p\n")) + assert has_real_rejection(_fn("def f(p):\n if not p:\n raise ValueError\n return p\n")) + assert has_real_rejection(_fn("def f(p):\n if not p:\n return None\n return p\n")) + assert not has_real_rejection(_fn("def f(p):\n return p\n")) + + +def _entities(src: str): + tree = ast.parse(textwrap.dedent(src)) + ents = discover_file_entities(tree, module="m", path="m.py") + return {e.qualname: e for e in ents} + + +def test_rejecting_helper_calls_one_hop_lexical_fallback() -> None: + ents = _entities( + """ + def _require_nonempty(p): + if not p: + raise ValueError("empty") + def v(p): + _require_nonempty(p) + return p + """ + ) + calls = rejecting_helper_calls(ents["m.v"], ents, {}) + assert len(calls) == 1 + + +def test_rejecting_helper_calls_staticmethod_helper() -> None: + ents = _entities( + """ + class Validators: + @staticmethod + def require(p): + if not p: + raise ValueError("empty") + def v(p): + Validators.require(p) + return p + """ + ) + assert len(rejecting_helper_calls(ents["m.v"], ents, {})) == 1 + + +def test_rejecting_helper_calls_rejects_non_raising_helper() -> None: + # SOUNDNESS GUARD: a helper that cannot raise (logs and returns) is NOT a rejection. + ents = _entities( + """ + def _log(p): + print(p) + return p + def v(p): + _log(p) + return p + """ + ) + assert rejecting_helper_calls(ents["m.v"], ents, {}) == frozenset() + + +def test_rejecting_helper_calls_assert_only_helper_does_not_count() -> None: + # A helper whose only rejection is assert vanishes under -O just like an inline + # assert; it is not a REAL one-hop rejection (keeps the 102/111 partition honest). + ents = _entities( + """ + def _check(p): + assert p + def v(p): + _check(p) + return p + """ + ) + assert rejecting_helper_calls(ents["m.v"], ents, {}) == frozenset() + + +def test_rejecting_helper_calls_is_same_module_only() -> None: + # One-hop SAME-MODULE only: a resolved cross-module callee does not count. + ents_m = _entities("def v(p):\n helper(p)\n return p\n") + other = _entities("def helper(p):\n if not p:\n raise ValueError\n") + # graft the foreign entity (different path) into the lookup table under the + # qualname the resolver would report + tree = ast.parse("def helper(p):\n if not p:\n raise ValueError\n") + foreign = discover_file_entities(tree, module="n", path="n.py")[0] + entities = {**ents_m, foreign.qualname: foreign} + call = next(n for n in ast.walk(ents_m["m.v"].node) if isinstance(n, ast.Call)) + assert rejecting_helper_calls(ents_m["m.v"], entities, {id(call): "n.helper"}) == frozenset() + assert other # silence unused warning + + +def test_block_has_real_rejection_scans_statement_lists() -> None: + fn = _fn( + """ + def f(p): + try: + if not p: + raise ValueError + except ValueError: + return p + """ + ) + try_stmt = fn.body[0] + assert isinstance(try_stmt, ast.Try) + assert block_has_real_rejection(try_stmt.body) + assert not block_has_real_rejection(try_stmt.handlers[0].body) + + +def test_is_degenerate_boundary_shapes() -> None: + assert is_degenerate_boundary(_fn("def f(p):\n return p\n")) + assert is_degenerate_boundary(_fn("def f(p):\n 'doc'\n return p\n")) + assert not is_degenerate_boundary(_fn("def f(p):\n x = p\n return x\n")) + assert not is_degenerate_boundary(_fn("def f(p):\n return g(p)\n")) + + def test_own_except_handlers_skips_nested_functions() -> None: fn = _fn( "def f():\n" diff --git a/tests/unit/scanner/rules/test_boundary_partition.py b/tests/unit/scanner/rules/test_boundary_partition.py new file mode 100644 index 00000000..d61380ac --- /dev/null +++ b/tests/unit/scanner/rules/test_boundary_partition.py @@ -0,0 +1,213 @@ +"""Boundary-integrity family partition oracle (wardline-718048a518). + +The four rules partition the declared-boundary defect space — AT MOST ONE of +{PY-WL-102, PY-WL-111, PY-WL-113, PY-WL-119} fires per boundary: + + - PY-WL-119 — the bare degenerate shape (single ``return ``); + - PY-WL-102 — every other shape with NO rejection path; + - PY-WL-111 — the only rejection is ``assert`` (vanishes under ``python -O``); + - PY-WL-113 — a real rejection exists but a fail-open handler defeats it. + +This file is the executable form of that contract: each canonical shape is run +through the full analyzer and pinned to EXACTLY its owning rule. The partition +regressed silently before because no test asserted exactly-one-of; these do. +""" + +from __future__ import annotations + +import textwrap +from pathlib import Path + +import pytest + +from wardline.core.config import WardlineConfig +from wardline.scanner.analyzer import WardlineAnalyzer + +FAMILY = frozenset({"PY-WL-102", "PY-WL-111", "PY-WL-113", "PY-WL-119"}) + +_HEADER = "from wardline.decorators import trust_boundary\n" + + +def _family_ids(tmp_path: Path, src: str) -> set[str]: + p = tmp_path / "m.py" + p.write_text(_HEADER + textwrap.dedent(src), encoding="utf-8") + findings = WardlineAnalyzer().analyze([p], WardlineConfig(), root=tmp_path) + return {f.rule_id for f in findings if f.rule_id in FAMILY} + + +_CASES = [ + pytest.param( + """ + @trust_boundary(to_level='ASSURED') + def v(p): + return p + """, + {"PY-WL-119"}, + id="degenerate-bare-passthrough-is-119-only", + ), + pytest.param( + """ + @trust_boundary(to_level='ASSURED') + def v(p): + x = p + return x + """, + {"PY-WL-102"}, + id="laundered-passthrough-is-102-only", + ), + pytest.param( + """ + @trust_boundary(to_level='ASSURED') + def v(p): + assert p + return p + """, + {"PY-WL-111"}, + id="assert-only-is-111-only", + ), + pytest.param( + # canonical self-catch fail-open + """ + @trust_boundary(to_level='ASSURED') + def v(p): + try: + if not p: + raise ValueError + return p + except ValueError: + return p + """, + {"PY-WL-113"}, + id="self-catch-failopen-is-113-only", + ), + pytest.param( + # wardline-718048a518 repro A: no rejection anywhere + substituting handler + """ + def compute(p): + return p + @trust_boundary(to_level='ASSURED') + def v(p): + try: + x = compute(p) + except Exception: + return p + return x + """, + {"PY-WL-102"}, + id="repro-a-no-rejection-substituting-handler-is-102-only", + ), + pytest.param( + # wardline-718048a518 repro B: assert-only rejection + substituting handler + """ + @trust_boundary(to_level='ASSURED') + def v(p): + assert p + try: + return p + except Exception: + return "x" + """, + {"PY-WL-111"}, + id="repro-b-assert-only-substituting-handler-is-111-only", + ), + pytest.param( + # docstring precedence: the assert IS inside the try and caught by a + # substituting handler — the rejection is still assert-only, so 111 wins. + """ + @trust_boundary(to_level='ASSURED') + def v(p): + try: + assert p + return p + except AssertionError: + return "x" + """, + {"PY-WL-111"}, + id="assert-inside-try-swallowed-is-111-not-113", + ), + pytest.param( + # clean validator: real raise, fail-closed + """ + @trust_boundary(to_level='ASSURED') + def v(p): + if not p: + raise ValueError + return p + """, + set(), + id="raising-validator-is-clean", + ), + pytest.param( + # boundary.json FP 5: rejection outside the try; cache-miss fallback handler + """ + _cache = {} + @trust_boundary(to_level='ASSURED') + def v(p): + if not p.isdigit(): + raise ValueError + try: + cached = _cache[p] + return cached + except KeyError: + result = int(p) + _cache[p] = result + return result + """, + set(), + id="fail-closed-cache-fallback-is-clean", + ), + pytest.param( + # one-hop helper rejection: clean for 102/111, and fail-closed (no try) + """ + def _require_nonempty(p): + if not p: + raise ValueError("empty") + @trust_boundary(to_level='ASSURED') + def v(p): + _require_nonempty(p) + return p + """, + set(), + id="helper-rejecting-validator-is-clean", + ), + pytest.param( + # delegation to a raising boundary: single Return of a CALL — not degenerate + """ + @trust_boundary(to_level='ASSURED') + def inner(p): + if not p: + raise ValueError + return p + @trust_boundary(to_level='ASSURED') + def v(p): + return inner(p) + """, + set(), + id="delegating-validator-is-clean", + ), + pytest.param( + # helper rejection inside the try, swallowed by a substituting handler: + # the rejection exists (not 102) and is not assert-only (not 111) — 113 owns it. + """ + def _require_nonempty(p): + if not p: + raise ValueError("empty") + @trust_boundary(to_level='ASSURED') + def v(p): + try: + _require_nonempty(p) + return p + except ValueError: + return p + """, + {"PY-WL-113"}, + id="helper-rejection-swallowed-is-113-only", + ), +] + + +@pytest.mark.parametrize(("src", "expected"), _CASES) +def test_family_partition_exactly_one_owner(tmp_path: Path, src: str, expected: set[str]) -> None: + ids = _family_ids(tmp_path, src) + assert ids == expected + assert len(ids) <= 1, f"family partition violated — co-fired: {sorted(ids)}" diff --git a/tests/unit/scanner/rules/test_boundary_without_rejection.py b/tests/unit/scanner/rules/test_boundary_without_rejection.py index 9308a07f..d7d9e3d7 100644 --- a/tests/unit/scanner/rules/test_boundary_without_rejection.py +++ b/tests/unit/scanner/rules/test_boundary_without_rejection.py @@ -27,12 +27,14 @@ def _run(ctx): def test_boundary_without_rejection_fires(tmp_path) -> None: - # @trust_boundary that just returns its input — cannot reject -> DEFECT. + # @trust_boundary that launders its input through a local — cannot reject -> DEFECT. + # (The bare single-statement `return p` shape is PY-WL-119's domain now; 102 owns + # every OTHER no-rejection shape.) ctx, _ = _analyze( tmp_path, { "m.py": "from wardline.decorators import trust_boundary\n" - "@trust_boundary(to_level='ASSURED')\ndef v(p):\n return p\n", + "@trust_boundary(to_level='ASSURED')\ndef v(p):\n x = p\n return x\n", }, ) findings = _run(ctx) @@ -40,6 +42,20 @@ def test_boundary_without_rejection_fires(tmp_path) -> None: assert findings[0].kind == Kind.DEFECT +def test_degenerate_shape_is_119s_domain_102_suppressed(tmp_path) -> None: + # Suppress-and-delegate: the bare degenerate boundary (`return p`) is PY-WL-119's + # finding; 102 must NOT double-fire on it (wardline-718048a518 four-way partition). + ctx, findings = _analyze( + tmp_path, + { + "m.py": "from wardline.decorators import trust_boundary\n" + "@trust_boundary(to_level='ASSURED')\ndef v(p):\n return p\n", + }, + ) + assert _run(ctx) == [] + assert [(f.rule_id, f.qualname) for f in findings if f.rule_id == "PY-WL-119"] == [("PY-WL-119", "m.v")] + + def test_boundary_with_raise_is_clean(tmp_path) -> None: ctx, _ = _analyze( tmp_path, @@ -81,3 +97,128 @@ def test_non_boundary_decorators_are_ignored(tmp_path) -> None: def test_undecorated_is_silent(tmp_path) -> None: ctx, _ = _analyze(tmp_path, {"m.py": "def v(p):\n return p\n"}) assert _run(ctx) == [] + + +# ── One-hop same-module helper rejection (boundary.json FP 1) ──────────────── + + +def test_boundary_rejecting_via_same_module_raising_helper_is_clean(tmp_path) -> None: + ctx, _ = _analyze( + tmp_path, + { + "m.py": "from wardline.decorators import trust_boundary\n" + "def _require_nonempty(p):\n if not p:\n raise ValueError('empty')\n" + "@trust_boundary(to_level='ASSURED')\ndef v(p):\n _require_nonempty(p)\n return p\n", + }, + ) + assert _run(ctx) == [] + + +def test_boundary_rejecting_via_staticmethod_helper_is_clean(tmp_path) -> None: + ctx, _ = _analyze( + tmp_path, + { + "m.py": "from wardline.decorators import trust_boundary\n" + "class Validators:\n" + " @staticmethod\n" + " def require(p):\n if not p:\n raise ValueError('empty')\n" + "@trust_boundary(to_level='ASSURED')\ndef v(p):\n Validators.require(p)\n return p\n", + }, + ) + assert _run(ctx) == [] + + +def test_boundary_delegating_to_raising_boundary_is_clean(tmp_path) -> None: + # Wholesale delegation to another declared boundary that itself raises. + ctx, _ = _analyze( + tmp_path, + { + "m.py": "from wardline.decorators import trust_boundary\n" + "@trust_boundary(to_level='ASSURED')\n" + "def inner(p):\n if not p:\n raise ValueError\n return p\n" + "@trust_boundary(to_level='ASSURED')\ndef v(p):\n return inner(p)\n", + }, + ) + assert _run(ctx) == [] + + +def test_non_raising_helper_does_not_count_as_rejection(tmp_path) -> None: + # SOUNDNESS GUARD: a helper that logs and returns cannot reject — 102 still fires. + ctx, _ = _analyze( + tmp_path, + { + "m.py": "from wardline.decorators import trust_boundary\n" + "def _log(p):\n print(p)\n return p\n" + "@trust_boundary(to_level='ASSURED')\ndef v(p):\n _log(p)\n return p\n", + }, + ) + assert [(f.rule_id, f.qualname) for f in _run(ctx)] == [("PY-WL-102", "m.v")] + + +def test_cross_module_raising_helper_does_not_count(tmp_path) -> None: + # One-hop is SAME-MODULE only (cheap + conservative): a cross-module raising + # helper does not silence the rule. + ctx, _ = _analyze( + tmp_path, + { + "helpers.py": "def require(p):\n if not p:\n raise ValueError\n", + "m.py": "from wardline.decorators import trust_boundary\n" + "from helpers import require\n" + "@trust_boundary(to_level='ASSURED')\ndef v(p):\n require(p)\n return p\n", + }, + ) + assert [(f.rule_id, f.qualname) for f in _run(ctx)] == [("PY-WL-102", "m.v")] + + +def test_unresolvable_call_does_not_count_as_rejection(tmp_path) -> None: + ctx, _ = _analyze( + tmp_path, + { + "m.py": "from wardline.decorators import trust_boundary\n" + "@trust_boundary(to_level='ASSURED')\ndef v(p):\n frobnicate(p)\n return p\n", + }, + ) + assert [(f.rule_id, f.qualname) for f in _run(ctx)] == [("PY-WL-102", "m.v")] + + +# ── Conditional-expression and raising-conversion returns (FPs 3 + 4) ─────── + + +def test_ternary_falsy_return_is_clean(tmp_path) -> None: + ctx, _ = _analyze( + tmp_path, + { + "m.py": "import re\nfrom wardline.decorators import trust_boundary\n" + "@trust_boundary(to_level='ASSURED')\ndef v(p):\n" + " m = re.fullmatch(r'[a-z]+', p)\n return m.group(0) if m else None\n", + }, + ) + assert _run(ctx) == [] + + +def test_raising_conversion_returns_are_clean(tmp_path) -> None: + ctx, _ = _analyze( + tmp_path, + { + "m.py": "import enum\nfrom wardline.decorators import trust_boundary\n" + "class Color(enum.Enum):\n RED = 'red'\n" + "ALLOWED = {'a': 1}\n" + "@trust_boundary(to_level='ASSURED')\ndef to_port(p):\n return int(p)\n" + "@trust_boundary(to_level='ASSURED')\ndef to_color(p):\n return Color[p]\n" + "@trust_boundary(to_level='ASSURED')\ndef to_allowed(p):\n return ALLOWED[p]\n", + }, + ) + assert _run(ctx) == [] + + +def test_arbitrary_call_return_still_fires(tmp_path) -> None: + # SOUNDNESS GUARD: the raising-conversion set is curated — `return helper_obj(p)` + # for an unknown callee is NOT a rejection. + ctx, _ = _analyze( + tmp_path, + { + "m.py": "from wardline.decorators import trust_boundary\n" + "@trust_boundary(to_level='ASSURED')\ndef v(p):\n x = str(p)\n return str(x)\n", + }, + ) + assert [(f.rule_id, f.qualname) for f in _run(ctx)] == [("PY-WL-102", "m.v")] diff --git a/tests/unit/scanner/rules/test_broad_exception.py b/tests/unit/scanner/rules/test_broad_exception.py index f1cdb51c..2d1e13cc 100644 --- a/tests/unit/scanner/rules/test_broad_exception.py +++ b/tests/unit/scanner/rules/test_broad_exception.py @@ -131,3 +131,80 @@ def test_multiple_handlers_one_function_are_distinguished_only_by_line_start(tmp assert [f.qualname for f in findings] == ["m.f", "m.f"], "multi-emit: one finding per handler" fps = {f.fingerprint for f in findings} assert len(fps) == 2, "distinct today (via line_start); collides iff line_start leaves the key" + + +def test_nested_trusted_def_uses_its_own_tier(tmp_path) -> None: + # A nested def carrying its OWN trust declaration is governed by that tier, not + # the undecorated outer's UNKNOWN_RAW (which would wrongly suppress the rule). + # Aligns PY-WL-103 with the sink family's enclosing_declared_tier semantics + # (wardline-bb8396f96e). + ctx, _ = _analyze( + tmp_path, + { + "m.py": """\ + from wardline.decorators import trusted + + def outer(p): + @trusted(level="ASSURED") + def inner(): + try: + g() + except Exception: + pass + return inner + """, + }, + ) + findings = _run(ctx) + assert [(f.rule_id, f.qualname) for f in findings] == [("PY-WL-103", "m.outer..inner")] + assert findings[0].severity == Severity.WARN + assert findings[0].properties["tier"] == "ASSURED" + + +def test_nested_external_boundary_def_inside_trusted_is_suppressed(tmp_path) -> None: + # A nested @external_boundary def is explicitly in the raw zone; the @trusted + # OUTER's tier must not leak onto it (the FP direction of the .. strip). + ctx, _ = _analyze( + tmp_path, + { + "m.py": """\ + from wardline.decorators import external_boundary, trusted + + @trusted(level="ASSURED") + def outer(p): + @external_boundary + def inner(): + try: + g() + except Exception: + pass + return inner + """, + }, + ) + assert _run(ctx) == [] + + +def test_undeclared_nested_def_inherits_enclosing_declared_tier(tmp_path) -> None: + # A genuinely undeclared nested def inherits the nearest DECLARED enclosing + # scope's tier (wardline-9b88ec5419) — pins the inheritance half of the walk. + ctx, _ = _analyze( + tmp_path, + { + "m.py": """\ + from wardline.decorators import trusted + + @trusted(level="ASSURED") + def outer(p): + def inner(): + try: + g() + except Exception: + pass + return inner + """, + }, + ) + findings = _run(ctx) + assert [(f.rule_id, f.qualname) for f in findings] == [("PY-WL-103", "m.outer..inner")] + assert findings[0].properties["tier"] == "ASSURED" diff --git a/tests/unit/scanner/rules/test_command_expansion.py b/tests/unit/scanner/rules/test_command_expansion.py new file mode 100644 index 00000000..67520827 --- /dev/null +++ b/tests/unit/scanner/rules/test_command_expansion.py @@ -0,0 +1,334 @@ +"""PY-WL-108/112 command-family expansion + calibration (wardline-13cfdd7b31). + +Covers the four decided behavior changes plus the eval-flagged test gaps: + +* 108's charter is now **command/program-execution** — the always-shell string + APIs PLUS the argv-style program-execution family (``os.exec*`` / ``os.spawn*`` + / ``os.posix_spawn*`` / ``pty.spawn``), all CWE-78. +* ``shlex.quote`` neutralizes shell-string taint for 108 **in concatenation + context only**: a quoted fragment inside a larger constant command is GUARDED + (clean); a bare whole-command quote still fires (a fully-quoted single token + IS the attacker-chosen program name); the guard never applies to the argv + program-execution sinks (no shell — quoting protects nothing). +* Variable-binding aliases resolve: ``runner = subprocess.run; runner(raw, + shell=True)`` fires PY-WL-112 (and the same for 108's sinks). +* Severity calibration: 108/112 are base ERROR — same exploit class as SQLi + (PY-WL-118). +""" + +from __future__ import annotations + +import textwrap +from pathlib import Path + +import pytest + +from wardline.core.config import WardlineConfig +from wardline.core.finding import Kind, Severity +from wardline.scanner.analyzer import WardlineAnalyzer +from wardline.scanner.rules.untrusted_to_command import UntrustedToCommand +from wardline.scanner.rules.untrusted_to_shell_subprocess import UntrustedToShellSubprocess + +_HEADER = ( + "import os, pty, shlex, subprocess\n" + "from wardline.decorators import external_boundary, trusted\n" + "@external_boundary\n" + "def read_raw(p):\n" + " return p\n" +) + + +def _analyze(tmp_path: Path, src: str): + p = tmp_path / "m.py" + p.write_text(_HEADER + textwrap.dedent(src), encoding="utf-8") + analyzer = WardlineAnalyzer() + analyzer.analyze([p], WardlineConfig(), root=tmp_path) + assert analyzer.last_context is not None + return analyzer.last_context + + +# --------------------------------------------------------------------------- +# (1) Program-execution charter expansion: os.exec* / os.spawn* / posix_spawn / +# pty.spawn are PY-WL-108 sinks (attacker-controlled program path/argv). +# --------------------------------------------------------------------------- + +_PROGRAM_EXEC_SINKS = [ + "os.execl", + "os.execle", + "os.execlp", + "os.execlpe", + "os.execv", + "os.execve", + "os.execvp", + "os.execvpe", + "os.spawnl", + "os.spawnle", + "os.spawnlp", + "os.spawnlpe", + "os.spawnv", + "os.spawnve", + "os.spawnvp", + "os.spawnvpe", + "os.posix_spawn", + "os.posix_spawnp", + "pty.spawn", +] + + +@pytest.mark.parametrize("sink", _PROGRAM_EXEC_SINKS) +def test_108_raw_reaches_program_execution_sink(tmp_path, sink: str) -> None: + ctx = _analyze( + tmp_path, + f""" + @trusted(level='ASSURED') + def f(p): + {sink}(read_raw(p)) + """, + ) + findings = UntrustedToCommand().check(ctx) + assert [(x.rule_id, x.qualname, x.properties["sink"]) for x in findings] == [("PY-WL-108", "m.f", sink)] + assert findings[0].kind is Kind.DEFECT + assert findings[0].severity is Severity.ERROR + + +def test_108_spawn_with_constant_mode_and_raw_path_fires(tmp_path) -> None: + # The realistic spawn shape: a clean mode slot (os.P_WAIT) must not mask the + # raw program path in slot 1. + ctx = _analyze( + tmp_path, + """ + @trusted(level='ASSURED') + def f(p): + os.spawnl(os.P_WAIT, read_raw(p), 'x') + """, + ) + assert [(x.rule_id, x.qualname) for x in UntrustedToCommand().check(ctx)] == [("PY-WL-108", "m.f")] + + +def test_108_spawn_all_constant_args_does_not_fire(tmp_path) -> None: + ctx = _analyze( + tmp_path, + """ + @trusted(level='ASSURED') + def f(): + os.spawnl(os.P_WAIT, '/bin/ls', 'ls') + """, + ) + assert UntrustedToCommand().check(ctx) == [] + + +# --------------------------------------------------------------------------- +# (2) shlex.quote semantics: GUARDED as a fragment of a constant concatenation; +# NOT guarded as the whole command; NEVER guarding the argv exec sinks. +# --------------------------------------------------------------------------- + + +def test_108_shlex_quoted_fragment_concat_is_clean(tmp_path) -> None: + ctx = _analyze( + tmp_path, + """ + @trusted(level='ASSURED') + def f(p): + raw = read_raw(p) + os.system("echo " + shlex.quote(raw)) + """, + ) + assert UntrustedToCommand().check(ctx) == [] + + +def test_108_shlex_quoted_fragment_fstring_is_clean(tmp_path) -> None: + ctx = _analyze( + tmp_path, + """ + @trusted(level='ASSURED') + def f(p): + raw = read_raw(p) + os.system(f"echo {shlex.quote(raw)}") + """, + ) + assert UntrustedToCommand().check(ctx) == [] + + +def test_108_from_import_quote_alias_is_clean(tmp_path) -> None: + ctx = _analyze( + tmp_path, + """ + from shlex import quote + @trusted(level='ASSURED') + def f(p): + os.system("echo " + quote(read_raw(p))) + """, + ) + assert UntrustedToCommand().check(ctx) == [] + + +def test_108_unquoted_concat_of_raw_still_fires(tmp_path) -> None: + ctx = _analyze( + tmp_path, + """ + @trusted(level='ASSURED') + def f(p): + raw = read_raw(p) + os.system("echo " + raw) + """, + ) + findings = UntrustedToCommand().check(ctx) + assert [(x.rule_id, x.qualname, x.severity) for x in findings] == [("PY-WL-108", "m.f", Severity.ERROR)] + + +def test_108_whole_command_quote_still_fires(tmp_path) -> None: + # A fully-quoted single token passed to a shell EXECUTES that token as the + # program name — quoting the whole command is not a remediation. + ctx = _analyze( + tmp_path, + """ + @trusted(level='ASSURED') + def f(p): + raw = read_raw(p) + os.system(shlex.quote(raw)) + """, + ) + assert [(x.rule_id, x.qualname) for x in UntrustedToCommand().check(ctx)] == [("PY-WL-108", "m.f")] + + +def test_108_mixed_concat_with_unquoted_raw_leaf_fires(tmp_path) -> None: + ctx = _analyze( + tmp_path, + """ + @trusted(level='ASSURED') + def f(p): + raw = read_raw(p) + os.system("echo " + shlex.quote(raw) + raw) + """, + ) + assert [x.rule_id for x in UntrustedToCommand().check(ctx)] == ["PY-WL-108"] + + +def test_108_variable_mediated_quote_still_fires(tmp_path) -> None: + # The guard is INLINE-syntactic only (see the rule docstring): resolving a + # NAME leaf through the last-binding-wins binding collector would let a + # later ``x = shlex.quote(x)`` launder an earlier raw use — so the + # variable-mediated form deliberately stays a (conservative) positive. + ctx = _analyze( + tmp_path, + """ + @trusted(level='ASSURED') + def f(p): + safe = shlex.quote(read_raw(p)) + os.system("echo " + safe) + """, + ) + assert [x.rule_id for x in UntrustedToCommand().check(ctx)] == ["PY-WL-108"] + + +def test_108_quote_guard_does_not_apply_to_program_execution_sinks(tmp_path) -> None: + # No shell mediates os.execv — shlex.quote protects nothing; the program + # path is still attacker-controlled. + ctx = _analyze( + tmp_path, + """ + @trusted(level='ASSURED') + def f(p): + raw = read_raw(p) + os.execv("/bin/" + shlex.quote(raw), ["x"]) + """, + ) + assert [(x.rule_id, x.properties["sink"]) for x in UntrustedToCommand().check(ctx)] == [("PY-WL-108", "os.execv")] + + +# --------------------------------------------------------------------------- +# (3) Variable-binding aliases (function-local) resolve to the sink FQN. +# --------------------------------------------------------------------------- + + +def test_112_local_binding_alias_fires(tmp_path) -> None: + ctx = _analyze( + tmp_path, + """ + @trusted(level='ASSURED') + def f(p): + runner = subprocess.run + runner(read_raw(p), shell=True) + """, + ) + findings = UntrustedToShellSubprocess().check(ctx) + assert [(x.rule_id, x.qualname, x.properties["sink"]) for x in findings] == [("PY-WL-112", "m.f", "subprocess.run")] + assert findings[0].severity is Severity.ERROR + + +def test_112_local_binding_alias_without_shell_true_does_not_fire(tmp_path) -> None: + # The literal shell=True gate applies to binding-resolved calls too. + ctx = _analyze( + tmp_path, + """ + @trusted(level='ASSURED') + def f(p): + runner = subprocess.run + runner(read_raw(p)) + """, + ) + assert UntrustedToShellSubprocess().check(ctx) == [] + + +def test_108_local_binding_alias_fires(tmp_path) -> None: + ctx = _analyze( + tmp_path, + """ + @trusted(level='ASSURED') + def f(p): + run_cmd = os.system + run_cmd(read_raw(p)) + """, + ) + assert [(x.rule_id, x.properties["sink"]) for x in UntrustedToCommand().check(ctx)] == [("PY-WL-108", "os.system")] + + +# --------------------------------------------------------------------------- +# (4) Severity calibration: 108/112 base ERROR (same exploit class as PY-WL-118). +# --------------------------------------------------------------------------- + + +def test_108_and_112_base_severity_is_error() -> None: + # Tier MODULATION is unchanged machinery (severity_model tests); the + # calibration decision is the BASE severity, pinned here and exercised at + # ASSURED tier (base kept) by every positive test in this module. + assert UntrustedToCommand.metadata.base_severity is Severity.ERROR + assert UntrustedToShellSubprocess.metadata.base_severity is Severity.ERROR + + +# --------------------------------------------------------------------------- +# (5) Eval-flagged test gaps: per-sink positives for the pre-existing tables. +# --------------------------------------------------------------------------- + + +@pytest.mark.parametrize("sink", ["os.popen", "subprocess.getoutput", "subprocess.getstatusoutput"]) +def test_108_raw_reaches_shell_string_sink(tmp_path, sink: str) -> None: + ctx = _analyze( + tmp_path, + f""" + @trusted(level='ASSURED') + def f(p): + {sink}(read_raw(p)) + """, + ) + findings = UntrustedToCommand().check(ctx) + assert [(x.rule_id, x.qualname, x.properties["sink"]) for x in findings] == [("PY-WL-108", "m.f", sink)] + assert findings[0].severity is Severity.ERROR + + +@pytest.mark.parametrize( + "sink", + ["subprocess.run", "subprocess.call", "subprocess.check_call", "subprocess.check_output", "subprocess.Popen"], +) +def test_112_raw_reaches_family_member_shell_true(tmp_path, sink: str) -> None: + ctx = _analyze( + tmp_path, + f""" + @trusted(level='ASSURED') + def f(p): + {sink}(read_raw(p), shell=True) + """, + ) + findings = UntrustedToShellSubprocess().check(ctx) + assert [(x.rule_id, x.qualname, x.properties["sink"]) for x in findings] == [("PY-WL-112", "m.f", sink)] + assert findings[0].severity is Severity.ERROR diff --git a/tests/unit/scanner/rules/test_default_registry.py b/tests/unit/scanner/rules/test_default_registry.py index 3b325916..7331eda3 100644 --- a/tests/unit/scanner/rules/test_default_registry.py +++ b/tests/unit/scanner/rules/test_default_registry.py @@ -59,6 +59,12 @@ def test_default_registry_has_all_builtin_rules() -> None: "PY-WL-118", "PY-WL-119", "PY-WL-120", + "PY-WL-121", + "PY-WL-122", + "PY-WL-123", + "PY-WL-124", + "PY-WL-125", + "PY-WL-126", } diff --git a/tests/unit/scanner/rules/test_deserialization_expansion.py b/tests/unit/scanner/rules/test_deserialization_expansion.py new file mode 100644 index 00000000..dfbc0cc2 --- /dev/null +++ b/tests/unit/scanner/rules/test_deserialization_expansion.py @@ -0,0 +1,366 @@ +"""PY-WL-106 deserialization sink-family expansion (ticket wardline-4299f07bb4). + +Covers the three expansion axes plus the declared-sink test gap: + * OO streaming-unpickle API: ``pickle.Unpickler(raw).load()`` — chained AND + stored-instance form, resolved through the shared sink-binding machinery; + the dangerous data is the stream handed to the CONSTRUCTOR, so taint is + read from the constructor call's arguments. + * ``shelve.open`` — pickle-backed; the taint is on the PATH argument + (ArgSpec ``positions=(0,)`` / ``keywords=("filename",)``), so a tainted + non-path slot does not fire. + * Curated third-party CWE-502 table (name-matched at AST level — the modules + are never imported by the analyzer): dill.load/loads, jsonpickle.decode, + joblib.load, torch.load, numpy.load. numpy.load fires ONLY with a literal + ``allow_pickle=True`` (safe-by-default since numpy 1.16.3); torch.load is + suppressed by a literal ``weights_only=True`` (the modern safe spelling). + * Every entry in the rule's ``_SINKS`` gets at least one positive test + (closes the yaml/marshal/pickle.load mutation-survival gap), with a + completeness pin so a future sink addition forces a test. +""" + +from __future__ import annotations + +import textwrap +from pathlib import Path + +import pytest + +from wardline.core.config import WardlineConfig +from wardline.core.finding import Kind, Severity +from wardline.scanner.analyzer import WardlineAnalyzer +from wardline.scanner.rules.untrusted_to_deserialization import _SINKS, UntrustedToDeserialization + +# The analyzed module is parsed, never executed — third-party imports need not be installed. +_HEADER = ( + "import pickle, marshal, shelve, yaml\n" + "import dill, jsonpickle, joblib, torch, numpy\n" + "from wardline.decorators import external_boundary, trusted\n" + "@external_boundary\n" + "def read_raw(p):\n" + " return p\n" +) + + +def _analyze(tmp_path: Path, src: str): + p = tmp_path / "m.py" + p.write_text(_HEADER + textwrap.dedent(src), encoding="utf-8") + analyzer = WardlineAnalyzer() + analyzer.analyze([p], WardlineConfig(), root=tmp_path) + assert analyzer.last_context is not None + return analyzer.last_context + + +def _findings(tmp_path: Path, src: str): + return UntrustedToDeserialization().check(_analyze(tmp_path, src)) + + +# ── every declared sink has a positive (mutation-survival gap closure) ────────────── + +# (canonical sink fqn, the tainted sink expression using raw var ``b``) +_POSITIVE_SINK_CALLS = [ + ("pickle.loads", "pickle.loads(b)"), + ("pickle.load", "pickle.load(b)"), + ("marshal.loads", "marshal.loads(b)"), + ("marshal.load", "marshal.load(b)"), + ("yaml.load", "yaml.load(b)"), + ("yaml.load_all", "yaml.load_all(b)"), + ("yaml.unsafe_load", "yaml.unsafe_load(b)"), + ("yaml.full_load", "yaml.full_load(b)"), + ("pickle.Unpickler.load", "pickle.Unpickler(b).load()"), + ("shelve.open", "shelve.open(b)"), + ("dill.load", "dill.load(b)"), + ("dill.loads", "dill.loads(b)"), + ("jsonpickle.decode", "jsonpickle.decode(b)"), + ("joblib.load", "joblib.load(b)"), + ("torch.load", "torch.load(b)"), + ("numpy.load", "numpy.load(b, allow_pickle=True)"), +] + + +@pytest.mark.parametrize(("sink", "call"), _POSITIVE_SINK_CALLS, ids=[s for s, _ in _POSITIVE_SINK_CALLS]) +def test_every_declared_sink_fires_on_raw(tmp_path: Path, sink: str, call: str) -> None: + findings = _findings( + tmp_path, + f""" + @trusted(level='ASSURED') + def f(p): + b = read_raw(p) + {call} + """, + ) + assert [(x.rule_id, x.qualname, x.properties["sink"]) for x in findings] == [("PY-WL-106", "m.f", sink)] + # RCE-equivalent sinks carry the rule family's base severity (tier-modulated). + assert findings[0].severity == Severity.WARN + assert findings[0].kind == Kind.DEFECT + + +def test_positive_table_covers_every_declared_sink() -> None: + # Completeness pin: adding a sink to _SINKS without a positive test fails here. + assert {sink for sink, _ in _POSITIVE_SINK_CALLS} == set(_SINKS) + + +# ── OO streaming-unpickle API (pickle.Unpickler) ───────────────────────────────────── + + +def test_unpickler_stored_instance_fires(tmp_path: Path) -> None: + findings = _findings( + tmp_path, + """ + @trusted(level='ASSURED') + def f(p): + b = read_raw(p) + u = pickle.Unpickler(b) + return u.load() + """, + ) + assert [(x.rule_id, x.qualname, x.properties["sink"]) for x in findings] == [ + ("PY-WL-106", "m.f", "pickle.Unpickler.load") + ] + assert findings[0].properties["arg_taint"] == "EXTERNAL_RAW" + + +def test_unpickler_finding_anchors_on_the_load_call(tmp_path: Path) -> None: + findings = _findings( + tmp_path, + """ + @trusted(level='ASSURED') + def f(p): + b = read_raw(p) + u = pickle.Unpickler(b) + return u.load() + """, + ) + # _HEADER is 6 lines + the snippet's leading blank line → u.load() is line 12, + # NOT the constructor's line 11: the finding anchors on the sink METHOD call. + assert [x.location.line_start for x in findings] == [12] + + +def test_unpickler_clean_literal_stream_is_silent(tmp_path: Path) -> None: + findings = _findings( + tmp_path, + """ + @trusted(level='ASSURED') + def f(p): + return pickle.Unpickler('model.bin').load() + """, + ) + assert findings == [] + + +def test_unpickler_import_alias_resolves(tmp_path: Path) -> None: + findings = _findings( + tmp_path, + """ + import pickle as pk + @trusted(level='ASSURED') + def f(p): + return pk.Unpickler(read_raw(p)).load() + """, + ) + assert [x.properties["sink"] for x in findings] == ["pickle.Unpickler.load"] + + +def test_unpickler_annotation_only_binding_is_a_bounded_fn(tmp_path: Path) -> None: + # An annotation binds the class but carries no constructor call, so there is no + # stream argument to read taint from — documented bounded false negative. + findings = _findings( + tmp_path, + """ + @trusted(level='ASSURED') + def f(u: pickle.Unpickler): + x: pickle.Unpickler = u + return x.load() + """, + ) + assert findings == [] + + +# ── shelve.open (taint on the path argument) ───────────────────────────────────────── + + +def test_shelve_open_tainted_path_fires(tmp_path: Path) -> None: + findings = _findings( + tmp_path, + """ + @trusted(level='ASSURED') + def f(x): + raw = read_raw(x) + shelve.open(raw) + """, + ) + assert [(x.rule_id, x.properties["sink"]) for x in findings] == [("PY-WL-106", "shelve.open")] + + +def test_shelve_open_tainted_filename_keyword_fires(tmp_path: Path) -> None: + findings = _findings( + tmp_path, + """ + @trusted(level='ASSURED') + def f(x): + shelve.open(filename=read_raw(x)) + """, + ) + assert [x.properties["sink"] for x in findings] == ["shelve.open"] + + +def test_shelve_open_tainted_non_path_slot_is_silent(tmp_path: Path) -> None: + # ArgSpec precision: only the path slot is dangerous — a tainted flag is not. + findings = _findings( + tmp_path, + """ + @trusted(level='ASSURED') + def f(x): + shelve.open('app.db', read_raw(x)) + """, + ) + assert findings == [] + + +def test_shelve_open_as_context_manager_fires(tmp_path: Path) -> None: + findings = _findings( + tmp_path, + """ + @trusted(level='ASSURED') + def f(x): + with shelve.open(read_raw(x)) as db: + return db['k'] + """, + ) + assert [x.properties["sink"] for x in findings] == ["shelve.open"] + + +# ── curated third-party table ──────────────────────────────────────────────────────── + + +def test_numpy_load_without_allow_pickle_is_silent(tmp_path: Path) -> None: + # allow_pickle defaults to False (safe) in modern numpy — absent means safe. + findings = _findings( + tmp_path, + """ + @trusted(level='ASSURED') + def f(x): + numpy.load(read_raw(x)) + """, + ) + assert findings == [] + + +def test_numpy_load_allow_pickle_false_is_silent(tmp_path: Path) -> None: + findings = _findings( + tmp_path, + """ + @trusted(level='ASSURED') + def f(x): + numpy.load(read_raw(x), allow_pickle=False) + """, + ) + assert findings == [] + + +def test_numpy_load_dynamic_allow_pickle_is_silent(tmp_path: Path) -> None: + # Only the statically-visible literal True fires — a dynamic flag is a bounded FN. + findings = _findings( + tmp_path, + """ + @trusted(level='ASSURED') + def f(x, flag): + numpy.load(read_raw(x), allow_pickle=flag) + """, + ) + assert findings == [] + + +def test_numpy_load_alias_with_allow_pickle_fires(tmp_path: Path) -> None: + findings = _findings( + tmp_path, + """ + import numpy as np + @trusted(level='ASSURED') + def f(x): + np.load(read_raw(x), allow_pickle=True) + """, + ) + assert [x.properties["sink"] for x in findings] == ["numpy.load"] + + +def test_torch_load_literal_weights_only_true_is_silent(tmp_path: Path) -> None: + # The modern safe spelling: weights_only=True restricts the unpickler. + findings = _findings( + tmp_path, + """ + @trusted(level='ASSURED') + def f(x): + torch.load(read_raw(x), weights_only=True) + """, + ) + assert findings == [] + + +def test_torch_load_weights_only_false_fires(tmp_path: Path) -> None: + findings = _findings( + tmp_path, + """ + @trusted(level='ASSURED') + def f(x): + torch.load(read_raw(x), weights_only=False) + """, + ) + assert [x.properties["sink"] for x in findings] == ["torch.load"] + + +def test_third_party_from_import_alias_resolves(tmp_path: Path) -> None: + findings = _findings( + tmp_path, + """ + from dill import loads as dloads + @trusted(level='ASSURED') + def f(x): + dloads(read_raw(x)) + """, + ) + assert [x.properties["sink"] for x in findings] == ["dill.loads"] + + +def test_third_party_tainted_non_dangerous_slot_is_silent(tmp_path: Path) -> None: + # ArgSpec precision: torch.load's map_location is not the dangerous slot. + findings = _findings( + tmp_path, + """ + @trusted(level='ASSURED') + def f(x): + torch.load('model.pt', map_location=read_raw(x)) + """, + ) + assert findings == [] + + +# ── tier discipline + multi-emit identity ──────────────────────────────────────────── + + +def test_new_sinks_stay_silent_in_freedom_zone(tmp_path: Path) -> None: + # Undecorated → UNKNOWN_RAW tier → modulate → NONE (opt-in preserved). + findings = _findings( + tmp_path, + """ + def f(x): + b = read_raw(x) + torch.load(b) + shelve.open(b) + pickle.Unpickler(b).load() + """, + ) + assert findings == [] + + +def test_co_located_sinks_get_distinct_fingerprints(tmp_path: Path) -> None: + findings = _findings( + tmp_path, + """ + @trusted(level='ASSURED') + def f(x): + b = read_raw(x) + shelve.open(b); dill.loads(b) + """, + ) + assert sorted(x.properties["sink"] for x in findings) == ["dill.loads", "shelve.open"] + assert len({x.fingerprint for x in findings}) == 2 diff --git a/tests/unit/scanner/rules/test_discriminator_shape.py b/tests/unit/scanner/rules/test_discriminator_shape.py index fb53b1b9..8ec3d755 100644 --- a/tests/unit/scanner/rules/test_discriminator_shape.py +++ b/tests/unit/scanner/rules/test_discriminator_shape.py @@ -37,9 +37,11 @@ def _taint_path_none_flags(source_file: str) -> list[bool]: def test_taintedsinkrule_base_carries_a_discriminator() -> None: - # The 7 sink subclasses (106/107/108/112/115/116/117) have NO per-module _fp call — - # the single call lives in the shared base. It must carry a (non-None) span so every - # subclass is covered. + # The attribute-only sink subclasses (106/107/108/112/115/117/121-126) have NO + # per-module _fp call — the single call lives in the shared base's + # build_sink_finding (the 2026-06-10 consolidation restored this single-call + # property; the former mixins satisfied it only transitively). It must carry a + # (non-None) span so every subclass is covered. flags = _taint_path_none_flags(inspect.getfile(TaintedSinkRule)) assert flags, "expected the TaintedSinkRule base to build a fingerprint" assert not any(flags), "the shared sink-rule fingerprint must carry a discriminator (never taint_path=None)" diff --git a/tests/unit/scanner/rules/test_exec_expansion.py b/tests/unit/scanner/rules/test_exec_expansion.py new file mode 100644 index 00000000..51d99b38 --- /dev/null +++ b/tests/unit/scanner/rules/test_exec_expansion.py @@ -0,0 +1,158 @@ +# tests/unit/scanner/rules/test_exec_expansion.py +"""PY-WL-107 coverage expansion: exec()/compile() positives + the __builtins__ spelling. + +The pre-existing suite only had an eval() positive for PY-WL-107; exec() and +compile() were sinks with no positive regression cover, and the +``__builtins__.eval`` / ``__builtins__.exec`` spelling was unmatched entirely +(ticket wardline-c83b40c73a). +""" + +from __future__ import annotations + +import textwrap +from pathlib import Path + +from wardline.core.config import WardlineConfig +from wardline.core.finding import Kind, Severity +from wardline.scanner.analyzer import WardlineAnalyzer +from wardline.scanner.rules.untrusted_to_exec import UntrustedToExec + +_HEADER = ( + "from wardline.decorators import external_boundary, trusted\n@external_boundary\ndef read_raw(p):\n return p\n" +) + + +def _analyze(tmp_path: Path, src: str) -> tuple[WardlineAnalyzer, object]: + p = tmp_path / "m.py" + p.write_text(_HEADER + textwrap.dedent(src), encoding="utf-8") + analyzer = WardlineAnalyzer() + analyzer.analyze([p], WardlineConfig(), root=tmp_path) + assert analyzer.last_context is not None + return analyzer, analyzer.last_context + + +def test_107_raw_reaches_exec(tmp_path) -> None: + _, ctx = _analyze( + tmp_path, + """ + @trusted(level='ASSURED') + def f(p): + src = read_raw(p) + exec(src) + return 1 + """, + ) + findings = UntrustedToExec().check(ctx) + assert [(f.rule_id, f.qualname, f.kind, f.severity) for f in findings] == [ + ("PY-WL-107", "m.f", Kind.DEFECT, Severity.WARN) + ] + assert findings[0].properties["sink"] == "exec" + + +def test_107_raw_reaches_compile(tmp_path) -> None: + _, ctx = _analyze( + tmp_path, + """ + @trusted(level='ASSURED') + def f(p): + compile(read_raw(p), '', 'exec') + return 1 + """, + ) + findings = UntrustedToExec().check(ctx) + assert [(f.rule_id, f.qualname, f.kind, f.severity) for f in findings] == [ + ("PY-WL-107", "m.f", Kind.DEFECT, Severity.WARN) + ] + assert findings[0].properties["sink"] == "compile" + + +def test_107_raw_reaches_dunder_builtins_eval(tmp_path) -> None: + _, ctx = _analyze( + tmp_path, + """ + @trusted(level='ASSURED') + def f(p): + src = read_raw(p) + __builtins__.eval(src) + return 1 + """, + ) + findings = UntrustedToExec().check(ctx) + assert [(f.rule_id, f.qualname, f.kind, f.severity) for f in findings] == [ + ("PY-WL-107", "m.f", Kind.DEFECT, Severity.WARN) + ] + assert findings[0].properties["sink"] == "__builtins__.eval" + + +def test_107_raw_reaches_dunder_builtins_exec(tmp_path) -> None: + _, ctx = _analyze( + tmp_path, + """ + @trusted(level='ASSURED') + def f(p): + __builtins__.exec(read_raw(p)) + return 1 + """, + ) + findings = UntrustedToExec().check(ctx) + assert [(f.rule_id, f.qualname) for f in findings] == [("PY-WL-107", "m.f")] + assert findings[0].properties["sink"] == "__builtins__.exec" + + +def test_107_raw_reaches_dunder_builtins_compile(tmp_path) -> None: + # Same spelling class as __builtins__.eval/.exec — covered for parity with + # the builtins.compile form the sink table already carries. + _, ctx = _analyze( + tmp_path, + """ + @trusted(level='ASSURED') + def f(p): + __builtins__.compile(read_raw(p), '', 'exec') + return 1 + """, + ) + findings = UntrustedToExec().check(ctx) + assert [(f.rule_id, f.qualname) for f in findings] == [("PY-WL-107", "m.f")] + assert findings[0].properties["sink"] == "__builtins__.compile" + + +def test_107_exec_clean_constant(tmp_path) -> None: + _, ctx = _analyze( + tmp_path, + """ + @trusted(level='ASSURED') + def f(p): + exec('x = 1') + return 1 + """, + ) + findings = UntrustedToExec().check(ctx) + assert findings == [] + + +def test_107_compile_clean_constant(tmp_path) -> None: + _, ctx = _analyze( + tmp_path, + """ + @trusted(level='ASSURED') + def f(p): + compile('x = 1', '', 'exec') + return 1 + """, + ) + findings = UntrustedToExec().check(ctx) + assert findings == [] + + +def test_107_dunder_builtins_eval_undecorated_suppressed(tmp_path) -> None: + # New spelling obeys the same tier gate as every other sink form. + _, ctx = _analyze( + tmp_path, + """ + def f(p): + __builtins__.eval(read_raw(p)) + return 1 + """, + ) + findings = UntrustedToExec().check(ctx) + assert findings == [] diff --git a/tests/unit/scanner/rules/test_failopen_boundary.py b/tests/unit/scanner/rules/test_failopen_boundary.py index 19920732..f57842c9 100644 --- a/tests/unit/scanner/rules/test_failopen_boundary.py +++ b/tests/unit/scanner/rules/test_failopen_boundary.py @@ -242,6 +242,136 @@ def v(p): assert FailOpenBoundary().check(ctx) == [] +def test_failopen_silent_when_no_rejection_exists_102s_domain(tmp_path) -> None: + # wardline-718048a518 repro A: no rejection of any shape exists, so the boundary + # is PY-WL-102's ("cannot reject at all") — 113's premise (a rejection exists and + # is defeated) does not hold; 113 must stay silent. + ctx = _analyze( + tmp_path, + """ + def compute(p): + return p + @trust_boundary(to_level='ASSURED') + def v(p): + try: + x = compute(p) + except Exception: + return p + return x + """, + ) + assert FailOpenBoundary().check(ctx) == [] + + +def test_failopen_silent_when_assert_is_sole_rejection_111s_domain(tmp_path) -> None: + # wardline-718048a518 repro B: the only rejection is an assert — PY-WL-111's + # exclusive domain; 113 must stay silent. + ctx = _analyze( + tmp_path, + """ + @trust_boundary(to_level='ASSURED') + def v(p): + assert p + try: + return p + except Exception: + return "x" + """, + ) + assert FailOpenBoundary().check(ctx) == [] + + +def test_failopen_silent_on_unrelated_cache_miss_fallback(tmp_path) -> None: + # boundary.json FP 5: the boundary's rejection (the raise) is lexically OUTSIDE + # the try, so the handler cannot swallow it — fail-CLOSED, must stay silent. + ctx = _analyze( + tmp_path, + """ + _cache = {} + @trust_boundary(to_level='ASSURED') + def v(p): + if not p.isdigit(): + raise ValueError + try: + cached = _cache[p] + return cached + except KeyError: + result = int(p) + _cache[p] = result + return result + """, + ) + assert FailOpenBoundary().check(ctx) == [] + + +def test_failopen_fires_when_rejection_inside_try_with_unrelated_handler_type(tmp_path) -> None: + # TP preserved: an unrelated exception TYPE whose raise IS inside the same try + # can preempt the validation — still fires (type-blindness is documented). + ctx = _analyze( + tmp_path, + """ + _cache = {} + @trust_boundary(to_level='ASSURED') + def v(p): + try: + if not p.isdigit(): + raise ValueError + cached = _cache[p] + return cached + except KeyError: + return p + """, + ) + assert [(f.rule_id, f.qualname) for f in FailOpenBoundary().check(ctx)] == [("PY-WL-113", "m.v")] + + +def test_failopen_fires_when_helper_rejection_inside_try_is_swallowed(tmp_path) -> None: + # One-hop coherence: the rejection delegated to a same-module raising helper + # (which silences 102) is INSIDE the try and swallowed by a substituting + # handler — a real fail-open; 113 must own it. + ctx = _analyze( + tmp_path, + """ + def _require_nonempty(p): + if not p: + raise ValueError("empty") + @trust_boundary(to_level='ASSURED') + def v(p): + try: + _require_nonempty(p) + return p + except ValueError: + return p + """, + ) + assert [(f.rule_id, f.qualname) for f in FailOpenBoundary().check(ctx)] == [("PY-WL-113", "m.v")] + + +def test_failopen_silent_when_helper_rejection_outside_try(tmp_path) -> None: + # The helper-delegated rejection runs BEFORE the try — the cache-miss handler + # cannot swallow it; fail-closed, silent. + ctx = _analyze( + tmp_path, + """ + _cache = {} + def _require_digit(p): + if not p.isdigit(): + raise ValueError + @trust_boundary(to_level='ASSURED') + def v(p): + _require_digit(p) + try: + cached = _cache[p] + return cached + except KeyError: + result = int(p) + _cache[p] = result + return result + """, + ) + assert FailOpenBoundary().check(ctx) == [] + + def test_failopen_boundary_fires_on_conditional_return_then_fallthrough_assign(tmp_path) -> None: # panel-2 FN (wardline-c314a7140b): the handler returns only CONDITIONALLY (nested if) # and otherwise FALLS THROUGH with a value-substituting assignment. The fall-through diff --git a/tests/unit/scanner/rules/test_invalid_decorator_level_recognizer.py b/tests/unit/scanner/rules/test_invalid_decorator_level_recognizer.py new file mode 100644 index 00000000..c76dec28 --- /dev/null +++ b/tests/unit/scanner/rules/test_invalid_decorator_level_recognizer.py @@ -0,0 +1,148 @@ +# tests/unit/scanner/rules/test_invalid_decorator_level_recognizer.py +"""PY-WL-114 marker recognition must match the engine's seeding predicate. + +The rule may only recognise a builtin level-bearing marker the engine's seeding +would honour (``_is_builtin_decorator_fqn`` exact exports + shadowed-root +fail-closed rejection). Recognising a marker the seeding rejects produces a +false "a typo disables all taint gates" finding for a gate that never existed +(the same drift PY-WL-110 closed under wardline-09c09f14df). +""" + +from __future__ import annotations + +import textwrap +from pathlib import Path + +from wardline.core.config import WardlineConfig +from wardline.core.finding import Finding +from wardline.scanner.analyzer import WardlineAnalyzer +from wardline.scanner.rules.invalid_decorator_level import InvalidDecoratorLevel + + +def _findings_114(tmp_path: Path, files: dict[str, str]) -> list[Finding]: + paths = [] + for rel, src in files.items(): + p = tmp_path / rel + p.parent.mkdir(parents=True, exist_ok=True) + p.write_text(textwrap.dedent(src), encoding="utf-8") + paths.append(p) + analyzer = WardlineAnalyzer() + analyzer.analyze(sorted(paths), WardlineConfig(), root=tmp_path) + assert analyzer.last_context is not None + return InvalidDecoratorLevel().check(analyzer.last_context) + + +def test_nested_path_under_builtin_prefix_does_not_fire(tmp_path) -> None: + # The engine seeds ONLY the exact exports P. / P.trust.; + # an arbitrarily-nested path like wardline.decorators.evil.trusted is + # rejected by seeding, so no gate exists for a bad level to disable. + findings = _findings_114( + tmp_path, + { + "m.py": """\ + import wardline.decorators.evil + + @wardline.decorators.evil.trusted(level="BOGUS") + def f(p): + return p + """, + }, + ) + assert findings == [] + + +def test_shadowed_marker_root_does_not_fire(tmp_path) -> None: + # A project shipping its OWN top-level weft_markers module shadows the + # builtin marker root: seeding fails closed for every marker under it, so + # the decorator is a project-local foreign decorator, not the builtin. + findings = _findings_114( + tmp_path, + { + "weft_markers.py": """\ + def trusted(level=None): + def deco(fn): + return fn + return deco + """, + "m.py": """\ + from weft_markers import trusted + + @trusted(level="BOGUS") + def f(p): + return p + """, + }, + ) + assert findings == [] + + +def test_shadowed_wardline_root_does_not_fire(tmp_path) -> None: + # Same fail-closed rejection for a project-local top-level ``wardline`` module. + findings = _findings_114( + tmp_path, + { + "wardline.py": "X = 1\n", + "m.py": """\ + from wardline.decorators import trusted + + @trusted(level="BOGUS") + def f(p): + return p + """, + }, + ) + assert findings == [] + + +def test_exact_export_still_fires(tmp_path) -> None: + # Control: the real builtin export keeps firing after the recognizer tightening. + findings = _findings_114( + tmp_path, + { + "m.py": """\ + from wardline.decorators import trusted + + @trusted(level="BOGUS") + def f(p): + return p + """, + }, + ) + assert [(f.rule_id, f.qualname) for f in findings] == [("PY-WL-114", "m.f")] + + +def test_trust_submodule_export_still_fires(tmp_path) -> None: + # The implementation-module export P.trust. is also a seeded spelling. + findings = _findings_114( + tmp_path, + { + "m.py": """\ + from wardline.decorators.trust import trust_boundary + + @trust_boundary(to_level="INTEGRAL") + def g(p): + if not p: + raise ValueError + return p + """, + }, + ) + assert [(f.rule_id, f.qualname) for f in findings] == [("PY-WL-114", "m.g")] + + +def test_unshadowed_weft_markers_still_fires(tmp_path) -> None: + # Control: the weft_markers shim is a builtin root; with no project-local + # shadow it is seeded, so an invalid level on it is a real PY-WL-114. + findings = _findings_114( + tmp_path, + { + "m.py": """\ + from weft_markers import trusted + + @trusted(level="BOGUS") + def f(p): + return p + """, + }, + ) + assert [(f.rule_id, f.qualname) for f in findings] == [("PY-WL-114", "m.f")] diff --git a/tests/unit/scanner/rules/test_none_leak.py b/tests/unit/scanner/rules/test_none_leak.py index 83fa73a1..43828f68 100644 --- a/tests/unit/scanner/rules/test_none_leak.py +++ b/tests/unit/scanner/rules/test_none_leak.py @@ -133,6 +133,22 @@ def maybe(flag): assert _ids(ctx) == [] +def test_undecorated_does_not_fire(tmp_path) -> None: + # Tier-suppression matrix slot (wardline-e159060db7): PY-WL-109 is gated on + # an ANCHORED trusted producer. The identical annotated mixed-return shape + # without a trust marker makes no trusted-output claim -> silent. + ctx = _analyze( + tmp_path, + """ + def maybe(flag) -> int: + if flag: + return 1 + return + """, + ) + assert _ids(ctx) == [] + + def test_all_value_returns_do_not_fire(tmp_path) -> None: ctx = _analyze( tmp_path, diff --git a/tests/unit/scanner/rules/test_path_traversal_expansion.py b/tests/unit/scanner/rules/test_path_traversal_expansion.py new file mode 100644 index 00000000..4864c0ca --- /dev/null +++ b/tests/unit/scanner/rules/test_path_traversal_expansion.py @@ -0,0 +1,387 @@ +"""PY-WL-116 expansion (wardline-04b65cf0be + Zip Slip eval item) — empirical coverage. + +Three sink families joining the path-traversal rule: + +1. Filesystem-MUTATION sinks (``os.remove``/``shutil.rmtree``/...) — direct dotted + calls with a tainted path argument (destructive traversal: delete/move/copy + outside the intended directory). +2. Path-METHOD sinks — ``read_text``/``write_bytes``/... on a ``pathlib.Path`` + CONSTRUCTED from tainted input, both bound (``p = Path(raw); p.read_text()``) + and chained (``pathlib.Path(raw).read_text()``), via the shared sink-binding + machinery. The taint is read from the CONSTRUCTOR call's arguments. +3. Archive extraction (Zip Slip / tarbomb, CWE-22) — ``extractall``/``extract`` on + a ``tarfile.open``/``tarfile.TarFile``/``zipfile.ZipFile`` instance whose + ARCHIVE SOURCE is tainted; exempt when the call passes the tarfile safe filter + ``filter="data"``. + +Precision pins: constant paths (``os.remove('/var/log/x')``, +``Path('static.txt').read_text()``) stay silent; the freedom zone stays silent. +""" + +from __future__ import annotations + +import textwrap +from pathlib import Path + +import pytest + +from wardline.core.config import WardlineConfig +from wardline.core.finding import Severity +from wardline.scanner.analyzer import WardlineAnalyzer + +_HEADER = ( + "import os, os.path, shutil, pathlib, tarfile, zipfile\n" + "from pathlib import Path\n" + "from wardline.decorators import external_boundary, trusted\n" + "@external_boundary\n" + "def read_raw(p):\n" + " return p\n" +) + + +def _pt_findings(tmp_path: Path, src: str): + p = tmp_path / "m.py" + p.write_text(_HEADER + textwrap.dedent(src), encoding="utf-8") + analyzer = WardlineAnalyzer() + findings = analyzer.analyze([p], WardlineConfig(), root=tmp_path) + return [f for f in findings if f.rule_id == "PY-WL-116"] + + +def _sinks(findings) -> set[str]: + return {f.properties["sink"] for f in findings if f.properties} + + +# ── 1. Filesystem-mutation sinks (direct dotted calls) ───────────────────── + + +@pytest.mark.parametrize( + "call", + [ + "os.remove(read_raw(p))", + "os.unlink(read_raw(p))", + "os.rmdir(read_raw(p))", + "os.makedirs(read_raw(p))", + "os.mkdir(read_raw(p))", + "os.rename(read_raw(p), '/dst')", + "os.renames(read_raw(p), '/dst')", + "os.replace(read_raw(p), '/dst')", + "shutil.rmtree(read_raw(p))", + "shutil.copy(read_raw(p), '/dst')", + "shutil.copy2(read_raw(p), '/dst')", + "shutil.copyfile(read_raw(p), '/dst')", + "shutil.copytree(read_raw(p), '/dst')", + "shutil.move(read_raw(p), '/dst')", + ], +) +def test_fs_mutation_sink_fires_on_tainted_path(tmp_path: Path, call: str) -> None: + findings = _pt_findings( + tmp_path, + f""" + @trusted(level='ASSURED') + def f(p): + {call} + """, + ) + assert len(findings) == 1 + assert findings[0].qualname == "m.f" + assert findings[0].severity == Severity.WARN + + +def test_fs_mutation_tainted_destination_also_fires(tmp_path: Path) -> None: + # Worst-of-all-args: a tainted DESTINATION is traversal too (write outside the dir). + findings = _pt_findings( + tmp_path, + """ + @trusted(level='ASSURED') + def f(p): + shutil.copy('/etc/app.conf', read_raw(p)) + """, + ) + assert _sinks(findings) == {"shutil.copy"} + + +def test_fs_mutation_constant_path_is_silent(tmp_path: Path) -> None: + findings = _pt_findings( + tmp_path, + """ + @trusted(level='ASSURED') + def f(): + os.remove('/var/log/app.log') + shutil.rmtree('/tmp/scratch') + """, + ) + assert findings == [] + + +def test_fs_mutation_freedom_zone_is_silent(tmp_path: Path) -> None: + findings = _pt_findings( + tmp_path, + """ + def f(p): + os.remove(read_raw(p)) + """, + ) + assert findings == [] + + +def test_os_open_tainted_path_fires(tmp_path: Path) -> None: + # os.open is a declared _SINKS member with no dedicated positive + # (wardline-e159060db7) — a silent drop from the table would go undetected. + findings = _pt_findings( + tmp_path, + """ + @trusted(level='ASSURED') + def f(p): + os.open(read_raw(p), os.O_RDONLY) + """, + ) + assert [(x.rule_id, x.qualname, x.severity) for x in findings] == [("PY-WL-116", "m.f", Severity.WARN)] + + +def test_pathlib_path_constructor_tainted_fires(tmp_path: Path) -> None: + # The bare pathlib.Path(...) CONSTRUCTOR is itself a declared sink (distinct + # from the construct-then-method shapes below) — pin it individually. + findings = _pt_findings( + tmp_path, + """ + @trusted(level='ASSURED') + def f(p): + pathlib.Path(read_raw(p)) + """, + ) + assert [(x.rule_id, x.qualname, x.severity) for x in findings] == [("PY-WL-116", "m.f", Severity.WARN)] + + +# ── 2. Path-method sinks (construct-then-method via the binding machinery) ── + + +def test_bound_path_read_text_fires(tmp_path: Path) -> None: + findings = _pt_findings( + tmp_path, + """ + @trusted(level='ASSURED') + def f(p): + q = pathlib.Path(read_raw(p)) + return q.read_text() + """, + ) + # The tainted CONSTRUCTOR still fires (existing sink) and the method adds its own. + assert _sinks(findings) == {"pathlib.Path", "pathlib.Path.read_text"} + method = next(f for f in findings if f.properties["sink"] == "pathlib.Path.read_text") + assert method.severity == Severity.WARN + assert method.qualname == "m.f" + + +def test_chained_path_read_bytes_fires(tmp_path: Path) -> None: + findings = _pt_findings( + tmp_path, + """ + @trusted(level='ASSURED') + def f(p): + return pathlib.Path(read_raw(p)).read_bytes() + """, + ) + assert "pathlib.Path.read_bytes" in _sinks(findings) + + +def test_from_import_alias_path_write_text_fires(tmp_path: Path) -> None: + findings = _pt_findings( + tmp_path, + """ + @trusted(level='ASSURED') + def f(p): + q = Path(read_raw(p)) + q.write_text('payload') + """, + ) + assert "pathlib.Path.write_text" in _sinks(findings) + + +@pytest.mark.parametrize("method", ["write_bytes", "open", "unlink", "rmdir", "mkdir"]) +def test_bound_path_method_family_fires(tmp_path: Path, method: str) -> None: + findings = _pt_findings( + tmp_path, + f""" + @trusted(level='ASSURED') + def f(p): + q = Path(read_raw(p)) + q.{method}() + """, + ) + assert f"pathlib.Path.{method}" in _sinks(findings) + + +def test_static_path_methods_are_silent(tmp_path: Path) -> None: + findings = _pt_findings( + tmp_path, + """ + @trusted(level='ASSURED') + def f(): + q = Path('static.txt') + q.read_text() + Path('static.txt').read_bytes() + """, + ) + assert findings == [] + + +def test_method_on_non_path_instance_is_silent(tmp_path: Path) -> None: + # A var bound to some other constructor must not match the Path method sinks. + findings = _pt_findings( + tmp_path, + """ + @trusted(level='ASSURED') + def f(p): + c = compute(read_raw(p)) + return c.read_text() + """, + ) + assert findings == [] + + +def test_rebound_path_resolves_at_final_binding(tmp_path: Path) -> None: + # Last-binding-wins (machinery contract): tainted Path rebound to a constant + # Path before the method call — the method must not fire on the stale ctor. + findings = _pt_findings( + tmp_path, + """ + @trusted(level='ASSURED') + def f(p): + q = Path(read_raw(p)) + q = Path('static.txt') + return q.read_text() + """, + ) + assert _sinks(findings) == {"pathlib.Path"} # only the tainted ctor itself + + +# ── 3. Archive extraction (Zip Slip / tarbomb) ────────────────────────────── + + +def test_tarfile_bound_extractall_fires_and_names_archive_extraction(tmp_path: Path) -> None: + findings = _pt_findings( + tmp_path, + """ + @trusted(level='ASSURED') + def f(p): + tf = tarfile.open(read_raw(p)) + tf.extractall('/dst') + """, + ) + assert _sinks(findings) == {"tarfile.open.extractall"} + assert "archive extraction" in findings[0].message + assert findings[0].severity == Severity.WARN + + +def test_tarfile_with_binding_extract_fires(tmp_path: Path) -> None: + findings = _pt_findings( + tmp_path, + """ + @trusted(level='ASSURED') + def f(p, member): + with tarfile.open(read_raw(p)) as tf: + tf.extract(member, '/dst') + """, + ) + assert "tarfile.open.extract" in _sinks(findings) + + +def test_zipfile_chained_extractall_fires(tmp_path: Path) -> None: + findings = _pt_findings( + tmp_path, + """ + @trusted(level='ASSURED') + def f(p): + zipfile.ZipFile(read_raw(p)).extractall() + """, + ) + assert _sinks(findings) == {"zipfile.ZipFile.extractall"} + + +def test_zipfile_bound_extract_fires(tmp_path: Path) -> None: + findings = _pt_findings( + tmp_path, + """ + @trusted(level='ASSURED') + def f(p): + zf = zipfile.ZipFile(read_raw(p)) + zf.extract('member.txt') + """, + ) + assert "zipfile.ZipFile.extract" in _sinks(findings) + + +def test_tarfile_class_constructor_extractall_fires(tmp_path: Path) -> None: + findings = _pt_findings( + tmp_path, + """ + @trusted(level='ASSURED') + def f(p): + tf = tarfile.TarFile(read_raw(p)) + tf.extractall('/dst') + """, + ) + assert "tarfile.TarFile.extractall" in _sinks(findings) + + +def test_data_filter_exempts_extraction(tmp_path: Path) -> None: + # filter="data" is tarfile's safe extraction filter (blocks absolute paths, + # ../ traversal, devices) — the documented mitigation, so no finding. + findings = _pt_findings( + tmp_path, + """ + @trusted(level='ASSURED') + def f(p): + tf = tarfile.open(read_raw(p)) + tf.extractall('/dst', filter='data') + """, + ) + assert findings == [] + + +def test_non_data_filter_still_fires(tmp_path: Path) -> None: + # Only the "data" filter is the documented exemption; "fully_trusted" is unsafe. + findings = _pt_findings( + tmp_path, + """ + @trusted(level='ASSURED') + def f(p): + tf = tarfile.open(read_raw(p)) + tf.extractall('/dst', filter='fully_trusted') + """, + ) + assert _sinks(findings) == {"tarfile.open.extractall"} + + +def test_constant_archive_source_is_silent(tmp_path: Path) -> None: + # v1 scope decision: the rule keys on the ARCHIVE SOURCE; a tainted + # destination with a trusted archive does not fire this family. + findings = _pt_findings( + tmp_path, + """ + @trusted(level='ASSURED') + def f(p): + tf = tarfile.open('bundled.tar') + tf.extractall('/dst') + """, + ) + assert findings == [] + + +# ── Fingerprint discipline (multi_emit discriminator) ─────────────────────── + + +def test_co_located_method_and_ctor_findings_have_distinct_fingerprints(tmp_path: Path) -> None: + # The chained form emits both the ctor and the method finding on ONE line — + # the taint_path discriminator (span + sink name) must separate them. + findings = _pt_findings( + tmp_path, + """ + @trusted(level='ASSURED') + def f(p): + return pathlib.Path(read_raw(p)).read_text() + """, + ) + assert _sinks(findings) == {"pathlib.Path", "pathlib.Path.read_text"} + fps = [f.fingerprint for f in findings] + assert len(fps) == len(set(fps)) diff --git a/tests/unit/scanner/rules/test_review_fixups_rules.py b/tests/unit/scanner/rules/test_review_fixups_rules.py new file mode 100644 index 00000000..26d4f7af --- /dev/null +++ b/tests/unit/scanner/rules/test_review_fixups_rules.py @@ -0,0 +1,287 @@ +# tests/unit/scanner/rules/test_review_fixups_rules.py +"""Regression tests for the 2026-06-10 review-panel RULE-side fixups. + +Covers the confirmed rule-design / quality defects: + +1. **PY-WL-120 suppress-and-delegate must honor rule enablement** — with + ``rules.enable = ("PY-WL-120",)`` the delegate (PY-WL-101) never runs, so + the suppression dropped the raw-storage-return defect entirely. + +2. **PY-WL-124 slot precision** — the only 121–126 family member built on + worst-of-all-args fired ERROR on a tainted ``mode=``/``use_errno=`` flag + with a CONSTANT library name. + +3. **PY-WL-125/126 parameter-annotation receivers** — the docstrings claimed + the ``log: logging.Logger`` / ``s: smtplib.SMTP`` parameter forms bind, but + only body AnnAssign seeded ``instance_classes``. + +4. **Operator severity override equal to the metadata default** — the per-sink + severity table detected an override by value identity, so an explicit + ``rules.severity = {PY-WL-121: ERROR}`` (equal to the default) was ignored. + +5. **WLN-ENGINE-FLOW-INSENSITIVE-FALLBACK** is an engine FACT finding now, not + a per-qualname ``UserWarning`` from inside rule ``check()`` calls. + +6. **Sink-loop consolidation** — the base :class:`TaintedSinkRule` is + binding-aware, so the historical base rules (106/107/115/124) resolve + callable aliases / construct-then-method forms (new findings only). +""" + +from __future__ import annotations + +import textwrap +import warnings +from typing import TYPE_CHECKING + +from wardline.core.config import WardlineConfig +from wardline.core.finding import Kind, Severity +from wardline.scanner.analyzer import WardlineAnalyzer + +if TYPE_CHECKING: + from collections.abc import Sequence + from pathlib import Path + + from wardline.core.finding import Finding + +_PREAMBLE = ( + "import os\n" + "from wardline.decorators import external_boundary, trusted\n" + "@external_boundary\n" + "def read_raw(p):\n" + " return p\n" +) + + +def _scan( + tmp_path: Path, + body: str, + config: WardlineConfig | None = None, + preamble: str = _PREAMBLE, +) -> Sequence[Finding]: + p = tmp_path / "m.py" + p.write_text(preamble + textwrap.dedent(body), encoding="utf-8") + with warnings.catch_warnings(): + warnings.simplefilter("error") # the engine must not warn from rule checks + return WardlineAnalyzer().analyze([p], config or WardlineConfig(), root=tmp_path) + + +def _rule_hits(findings: Sequence[Finding], rule_id: str) -> list[Finding]: + return [f for f in findings if f.rule_id == rule_id] + + +_STORAGE_RETURN = """ +@trusted(level='ASSURED') +def f(fd): + data = os.read(fd, 10) + return data +""" + + +# ── 1. PY-WL-120 suppression gated on PY-WL-101 enablement ─────────────────── + + +def test_120_fires_when_101_is_not_enabled(tmp_path: Path) -> None: + findings = _scan(tmp_path, _STORAGE_RETURN, WardlineConfig(rules_enable=("PY-WL-120",))) + assert _rule_hits(findings, "PY-WL-101") == [] # the delegate never ran + hits = _rule_hits(findings, "PY-WL-120") + assert len(hits) == 1 # ...so 120 must NOT suppress its return finding + assert hits[0].qualname == "m.f" + + +def test_120_still_delegates_to_101_under_the_default_config(tmp_path: Path) -> None: + findings = _scan(tmp_path, _STORAGE_RETURN) + assert len(_rule_hits(findings, "PY-WL-101")) == 1 + assert _rule_hits(findings, "PY-WL-120") == [] # delegate fired — suppression stands + + +# ── 2. PY-WL-124 ArgSpec slot precision ────────────────────────────────────── + + +def test_124_tainted_mode_flag_with_constant_name_is_clean(tmp_path: Path) -> None: + findings = _scan( + tmp_path, + """ + import ctypes + @trusted(level='ASSURED') + def f(p): + a = ctypes.CDLL('libm.so.6', mode=read_raw(p)) + b = ctypes.CDLL('libm.so.6', use_errno=read_raw(p)) + return 1 + """, + ) + assert _rule_hits(findings, "PY-WL-124") == [] + + +def test_124_tainted_library_name_still_fires(tmp_path: Path) -> None: + findings = _scan( + tmp_path, + """ + import ctypes + @trusted(level='ASSURED') + def f(p): + a = ctypes.CDLL(read_raw(p)) + b = ctypes.CDLL(name=read_raw(p)) + c = ctypes.cdll.LoadLibrary(read_raw(p)) + return 1 + """, + ) + assert len(_rule_hits(findings, "PY-WL-124")) == 3 + + +# ── 3. PY-WL-125/126 parameter-annotation receivers ────────────────────────── + + +def test_125_param_annotated_logger_fires(tmp_path: Path) -> None: + findings = _scan( + tmp_path, + """ + import logging + @trusted(level='ASSURED') + def f(p, log: logging.Logger): + log.info(read_raw(p)) + return 1 + """, + ) + hits = _rule_hits(findings, "PY-WL-125") + assert len(hits) == 1 + assert hits[0].properties["sink"] == "logging.Logger.info" + + +def test_126_param_annotated_smtp_client_fires(tmp_path: Path) -> None: + findings = _scan( + tmp_path, + """ + import smtplib + @trusted(level='ASSURED') + def f(p, s: smtplib.SMTP): + s.sendmail('f@x.com', 't@x.com', read_raw(p)) + return 1 + """, + ) + assert len(_rule_hits(findings, "PY-WL-126")) == 1 + + +def test_param_annotation_is_shadowed_by_a_body_rebind(tmp_path: Path) -> None: + # A body rebind to an unresolvable RHS invalidates the parameter seed — + # the stale Logger class must not keep matching. + findings = _scan( + tmp_path, + """ + import logging + @trusted(level='ASSURED') + def f(p, log: logging.Logger): + log = make_thing() + log.info(read_raw(p)) + return 1 + """, + ) + assert _rule_hits(findings, "PY-WL-125") == [] + + +# ── 4. Operator severity override equal to the metadata default ────────────── + +_STDLIB_XML = """ +import xml.etree.ElementTree as ET +@trusted(level='ASSURED') +def f(p): + return ET.fromstring(read_raw(p)) +""" + + +def test_121_stdlib_sink_keeps_per_sink_warn_without_override(tmp_path: Path) -> None: + hits = _rule_hits(_scan(tmp_path, _STDLIB_XML), "PY-WL-121") + assert [f.severity for f in hits] == [Severity.WARN] + + +def test_121_explicit_override_equal_to_default_rebases_per_sink_severity(tmp_path: Path) -> None: + findings = _scan(tmp_path, _STDLIB_XML, WardlineConfig(rules_severity={"PY-WL-121": "ERROR"})) + hits = _rule_hits(findings, "PY-WL-121") + assert [f.severity for f in hits] == [Severity.ERROR] + + +def test_121_explicit_lower_override_still_rebases(tmp_path: Path) -> None: + findings = _scan(tmp_path, _STDLIB_XML, WardlineConfig(rules_severity={"PY-WL-121": "INFO"})) + hits = _rule_hits(findings, "PY-WL-121") + assert [f.severity for f in hits] == [Severity.INFO] + + +# ── 5. Flow-insensitive fallback is a FACT finding, not a UserWarning ──────── + + +def test_flow_insensitive_fallback_emits_one_fact_finding(tmp_path: Path, monkeypatch) -> None: + import ast + + import wardline.scanner.analyzer as analyzer_mod + + real = analyzer_mod.run_l2_function_stage + + def _boom(stage_input): # noqa: ANN001, ANN202 + if any(isinstance(n, ast.Name) and n.id == "boom" for n in ast.walk(stage_input.node)): + raise RecursionError("simulated deep L2") + return real(stage_input) + + monkeypatch.setattr(analyzer_mod, "run_l2_function_stage", _boom) + + findings = _scan( + tmp_path, + """ + @trusted(level='ASSURED') + def f(p): + boom = read_raw(p) + return eval(boom) + """, + ) + # The L2-skipped function degrades the sink rules to the pessimistic + # fallback. That degradation is one NONE/FACT finding per scan — never a + # warning (the _scan harness runs with warnings-as-error to prove it). + facts = _rule_hits(findings, "WLN-ENGINE-FLOW-INSENSITIVE-FALLBACK") + assert len(facts) == 1 + assert facts[0].kind == Kind.FACT + assert facts[0].severity == Severity.NONE + assert facts[0].properties["qualnames"] == ["m.f"] + # The pessimistic fallback itself still fails closed: the sink fires. + assert len(_rule_hits(findings, "PY-WL-107")) == 1 + + +def test_no_fallback_fact_on_a_clean_scan(tmp_path: Path) -> None: + findings = _scan( + tmp_path, + """ + @trusted(level='ASSURED') + def f(p): + return eval(read_raw(p)) + """, + ) + assert _rule_hits(findings, "WLN-ENGINE-FLOW-INSENSITIVE-FALLBACK") == [] + + +# ── 6. Consolidated base: historical sink rules gain binding-awareness ─────── + + +def test_106_callable_alias_resolves_after_consolidation(tmp_path: Path) -> None: + findings = _scan( + tmp_path, + """ + import pickle + @trusted(level='ASSURED') + def f(p): + loader = pickle.loads + return loader(read_raw(p)) + """, + ) + hits = _rule_hits(findings, "PY-WL-106") + assert len(hits) == 1 + assert hits[0].properties["sink"] == "pickle.loads" + + +def test_107_callable_alias_resolves_after_consolidation(tmp_path: Path) -> None: + findings = _scan( + tmp_path, + """ + @trusted(level='ASSURED') + def f(p): + e = eval + return e(read_raw(p)) + """, + ) + assert len(_rule_hits(findings, "PY-WL-107")) == 1 diff --git a/tests/unit/scanner/rules/test_silent_exception.py b/tests/unit/scanner/rules/test_silent_exception.py index 5eaa246a..5f064d63 100644 --- a/tests/unit/scanner/rules/test_silent_exception.py +++ b/tests/unit/scanner/rules/test_silent_exception.py @@ -80,3 +80,80 @@ def test_reraise_is_clean(tmp_path) -> None: }, ) assert _run(ctx) == [] + + +def test_nested_trusted_def_uses_its_own_tier(tmp_path) -> None: + # A nested def carrying its OWN trust declaration is governed by that tier, not + # the undecorated outer's UNKNOWN_RAW (which would wrongly suppress the rule). + # Aligns PY-WL-104 with the sink family's enclosing_declared_tier semantics + # (wardline-bb8396f96e). + ctx, _ = _analyze( + tmp_path, + { + "m.py": """\ + from wardline.decorators import trusted + + def outer(p): + @trusted(level="ASSURED") + def inner(): + try: + g() + except ValueError: + pass + return inner + """, + }, + ) + findings = _run(ctx) + assert [(f.rule_id, f.qualname) for f in findings] == [("PY-WL-104", "m.outer..inner")] + assert findings[0].severity == Severity.WARN + assert findings[0].properties["tier"] == "ASSURED" + + +def test_nested_external_boundary_def_inside_trusted_is_suppressed(tmp_path) -> None: + # A nested @external_boundary def is explicitly in the raw zone; the @trusted + # OUTER's tier must not leak onto it (the FP direction of the .. strip). + ctx, _ = _analyze( + tmp_path, + { + "m.py": """\ + from wardline.decorators import external_boundary, trusted + + @trusted(level="ASSURED") + def outer(p): + @external_boundary + def inner(): + try: + g() + except ValueError: + pass + return inner + """, + }, + ) + assert _run(ctx) == [] + + +def test_undeclared_nested_def_inherits_enclosing_declared_tier(tmp_path) -> None: + # A genuinely undeclared nested def inherits the nearest DECLARED enclosing + # scope's tier (wardline-9b88ec5419) — pins the inheritance half of the walk. + ctx, _ = _analyze( + tmp_path, + { + "m.py": """\ + from wardline.decorators import trusted + + @trusted(level="ASSURED") + def outer(p): + def inner(): + try: + g() + except ValueError: + pass + return inner + """, + }, + ) + findings = _run(ctx) + assert [(f.rule_id, f.qualname) for f in findings] == [("PY-WL-104", "m.outer..inner")] + assert findings[0].properties["tier"] == "ASSURED" diff --git a/tests/unit/scanner/rules/test_sink_machinery.py b/tests/unit/scanner/rules/test_sink_machinery.py new file mode 100644 index 00000000..b9afb352 --- /dev/null +++ b/tests/unit/scanner/rules/test_sink_machinery.py @@ -0,0 +1,484 @@ +# tests/unit/scanner/rules/test_sink_machinery.py +"""Unit tests for the shared sink-resolution machinery in ``_sink_helpers``. + +Covers the three additive capabilities (consumers wire in separately): + 1. CONSTRUCT-THEN-METHOD: ``obj.method(...)`` resolves to ``.method`` + when the receiver's class is statically known in the function (direct + construction, chained construction, with/async-with targets, ann-assign). + 2. VARIABLE-BINDING ALIAS: ``runner = subprocess.run; runner(...)`` participates + in sink matching under the resolved FQN (module- or function-level). + 3. ARG-POSITION-AWARE matching: an :class:`ArgSpec` restricts taint resolution + to the declared dangerous argument slots; no spec keeps "worst of all args". +""" + +from __future__ import annotations + +import ast +import textwrap + +from wardline.core.taints import TaintState +from wardline.scanner.context import AnalysisContext +from wardline.scanner.rules._sink_helpers import ( + ArgSpec, + SinkBindings, + collect_sink_bindings, + resolve_bound_call_fqn, + resolved_sink_calls, + worst_dangerous_arg_taint, +) + + +def _last_def(src: str) -> ast.FunctionDef | ast.AsyncFunctionDef: + node = ast.parse(textwrap.dedent(src)).body[-1] + assert isinstance(node, (ast.FunctionDef, ast.AsyncFunctionDef)) + return node + + +def _module(src: str) -> ast.Module: + return ast.parse(textwrap.dedent(src)) + + +def _sinks( + func: ast.AST, + sink_names: set[str], + alias_map: dict[str, str] | None = None, + module_bindings: SinkBindings | None = None, +) -> list[str]: + return [ + fqn + for _call, fqn in resolved_sink_calls( + func, frozenset(sink_names), alias_map or {}, "m", module_bindings=module_bindings + ) + ] + + +# --------------------------------------------------------------------------- +# Capability 1: construct-then-method +# --------------------------------------------------------------------------- + + +def test_direct_construction_resolves_method_to_class_fqn() -> None: + func = _last_def( + """ + def f(u): + c = httpx.Client() + return c.get(u) + """ + ) + assert _sinks(func, {"httpx.Client.get"}, {"httpx": "httpx"}) == ["httpx.Client.get"] + + +def test_chained_construction_resolves() -> None: + func = _last_def( + """ + def f(u): + a = httpx.Client().get(u) + b = requests.Session().get(u) + return a, b + """ + ) + assert _sinks( + func, + {"httpx.Client.get", "requests.Session.get"}, + {"httpx": "httpx", "requests": "requests"}, + ) == ["httpx.Client.get", "requests.Session.get"] + + +def test_with_target_resolves() -> None: + func = _last_def( + """ + def f(u): + with httpx.Client() as c: + return c.get(u) + """ + ) + assert _sinks(func, {"httpx.Client.get"}, {"httpx": "httpx"}) == ["httpx.Client.get"] + + +def test_async_with_target_resolves() -> None: + func = _last_def( + """ + async def f(u): + async with httpx.AsyncClient() as client: + return await client.get(u) + """ + ) + assert _sinks(func, {"httpx.AsyncClient.get"}, {"httpx": "httpx"}) == ["httpx.AsyncClient.get"] + + +def test_ann_assign_annotation_resolves() -> None: + # A bare annotation (no value) is a declaration of the var's class. + func = _last_def( + """ + def f(u, factory): + c: httpx.Client + c = factory() + return c.get(u) + """ + ) + # The later factory() rebind is unresolvable and invalidates — annotation + # alone (without an intervening unresolvable rebind) must resolve: + func2 = _last_def( + """ + def f(u): + c: httpx.Client = make_client() + return c.get(u) + """ + ) + assert _sinks(func, {"httpx.Client.get"}, {"httpx": "httpx"}) == [] + assert _sinks(func2, {"httpx.Client.get"}, {"httpx": "httpx"}) == ["httpx.Client.get"] + + +def test_construction_honors_import_alias() -> None: + func = _last_def( + """ + def f(u): + c = hx.Client() + return c.get(u) + """ + ) + assert _sinks(func, {"httpx.Client.get"}, {"hx": "httpx"}) == ["httpx.Client.get"] + + +def test_rebind_to_different_class_uses_new_class_never_stale() -> None: + func = _last_def( + """ + def f(u): + c = httpx.Client() + c = requests.Session() + return c.get(u) + """ + ) + alias = {"httpx": "httpx", "requests": "requests"} + # last-binding-wins: the stale httpx class must NOT match + assert _sinks(func, {"httpx.Client.get"}, alias) == [] + assert _sinks(func, {"requests.Session.get"}, alias) == ["requests.Session.get"] + + +def test_rebind_to_unresolvable_invalidates() -> None: + # rebind via a non-dotted callable (subscript) — class no longer known + func = _last_def( + """ + def f(u, factories): + c = httpx.Client() + c = factories[0]() + return c.get(u) + """ + ) + assert _sinks(func, {"httpx.Client.get"}, {"httpx": "httpx"}) == [] + + +def test_rebind_to_constant_invalidates() -> None: + func = _last_def( + """ + def f(u): + c = httpx.Client() + c = None + return c.get(u) + """ + ) + assert _sinks(func, {"httpx.Client.get"}, {"httpx": "httpx"}) == [] + + +def test_tuple_unpack_rebind_invalidates() -> None: + func = _last_def( + """ + def f(u, pair): + c = httpx.Client() + c, d = pair + return c.get(u) + """ + ) + assert _sinks(func, {"httpx.Client.get"}, {"httpx": "httpx"}) == [] + + +def test_unknown_class_method_never_matches() -> None: + func = _last_def( + """ + def f(u): + c = mystery.Thing() + return c.get(u) + """ + ) + assert _sinks(func, {"httpx.Client.get"}, {"httpx": "httpx"}) == [] + + +def test_method_on_unbound_name_never_matches() -> None: + func = _last_def( + """ + def f(u, c): + return c.get(u) + """ + ) + assert _sinks(func, {"httpx.Client.get"}, {"httpx": "httpx"}) == [] + + +def test_nested_def_bindings_do_not_leak_into_outer_scope() -> None: + func = _last_def( + """ + def f(u): + def g(): + c = httpx.Client() + return c + c = g() + return c.get(u) + """ + ) + assert _sinks(func, {"httpx.Client.get"}, {"httpx": "httpx"}) == [] + + +# --------------------------------------------------------------------------- +# Capability 2: variable-binding alias of a sink +# --------------------------------------------------------------------------- + + +def test_function_level_callable_alias_matches() -> None: + func = _last_def( + """ + def f(raw): + runner = subprocess.run + runner(raw, shell=True) + """ + ) + assert _sinks(func, {"subprocess.run"}, {"subprocess": "subprocess"}) == ["subprocess.run"] + + +def test_from_import_alias_callable_binding() -> None: + func = _last_def( + """ + def f(raw): + runner = run + runner(raw) + """ + ) + # ``from subprocess import run`` puts {"run": "subprocess.run"} in the alias map + assert _sinks(func, {"subprocess.run"}, {"run": "subprocess.run"}) == ["subprocess.run"] + + +def test_builtin_callable_alias() -> None: + func = _last_def( + """ + def f(raw): + do = eval + do(raw) + """ + ) + assert _sinks(func, {"eval"}) == ["eval"] + + +def test_module_level_callable_alias_via_module_bindings() -> None: + mod = _module( + """ + runner = subprocess.run + + def f(raw): + runner(raw, shell=True) + """ + ) + module_bindings = collect_sink_bindings(mod, {"subprocess": "subprocess"}, "m") + func = mod.body[-1] + assert _sinks(func, {"subprocess.run"}, {"subprocess": "subprocess"}, module_bindings) == ["subprocess.run"] + + +def test_module_level_instance_binding_via_module_bindings() -> None: + mod = _module( + """ + client = httpx.Client() + + def f(u): + return client.get(u) + """ + ) + module_bindings = collect_sink_bindings(mod, {"httpx": "httpx"}, "m") + func = mod.body[-1] + assert _sinks(func, {"httpx.Client.get"}, {"httpx": "httpx"}, module_bindings) == ["httpx.Client.get"] + + +def test_function_rebind_shadows_module_alias() -> None: + mod = _module( + """ + runner = subprocess.run + + def f(raw, other): + runner = other + runner(raw) + """ + ) + module_bindings = collect_sink_bindings(mod, {"subprocess": "subprocess"}, "m") + func = mod.body[-1] + assert _sinks(func, {"subprocess.run"}, {"subprocess": "subprocess"}, module_bindings) == [] + + +def test_alias_of_non_sink_never_matches() -> None: + func = _last_def( + """ + def f(raw): + runner = os.path.join + runner(raw) + """ + ) + assert _sinks(func, {"subprocess.run"}, {"os": "os", "subprocess": "subprocess"}) == [] + + +def test_alias_rebind_last_binding_wins() -> None: + func = _last_def( + """ + def f(raw): + runner = subprocess.run + runner = print + runner(raw) + """ + ) + assert _sinks(func, {"subprocess.run"}, {"subprocess": "subprocess"}) == [] + + +def test_direct_sink_calls_still_match_alongside_bindings() -> None: + # the extended iterator is a superset of sink_calls: direct dotted spellings keep working + func = _last_def( + """ + def f(raw): + runner = subprocess.run + runner(raw) + subprocess.run(raw, shell=True) + """ + ) + assert _sinks(func, {"subprocess.run"}, {"subprocess": "subprocess"}) == [ + "subprocess.run", + "subprocess.run", + ] + + +def test_collect_sink_bindings_separates_kinds() -> None: + func = _last_def( + """ + def f(): + c = httpx.Client() + runner = subprocess.run + """ + ) + bindings = collect_sink_bindings(func, {"httpx": "httpx", "subprocess": "subprocess"}, "m") + assert bindings.instance_classes == {"c": "httpx.Client"} + assert bindings.callable_aliases == {"runner": "subprocess.run"} + + +def test_resolve_bound_call_fqn_negative_paths() -> None: + bindings = SinkBindings(instance_classes={"c": "httpx.Client"}, callable_aliases={}) + # dynamic receiver (subscript) resolves to nothing + call = ast.parse("x[0].get(u)").body[0].value + assert isinstance(call, ast.Call) + assert resolve_bound_call_fqn(call, bindings, {}, "m") is None + # plain name call with no alias binding resolves to nothing + call2 = ast.parse("runner(u)").body[0].value + assert isinstance(call2, ast.Call) + assert resolve_bound_call_fqn(call2, bindings, {}, "m") is None + + +# --------------------------------------------------------------------------- +# Capability 3: arg-position-aware matching +# --------------------------------------------------------------------------- + + +def _call(src: str) -> ast.Call: + call = ast.parse(src).body[0].value # type: ignore[attr-defined] + assert isinstance(call, ast.Call) + return call + + +def _ctx(call: ast.Call, taints: dict[int | str | None, TaintState]) -> AnalysisContext: + return AnalysisContext( + project_taints={}, + project_return_taints={}, + function_var_taints={}, + function_return_taints={}, + function_return_callee={}, + entities={}, + taint_provenance={}, + function_call_site_arg_taints={"m.f": {id(call): taints}}, + ) + + +def test_no_spec_keeps_worst_of_all_args() -> None: + call = _call("requests.get(a, b)") + ctx = _ctx(call, {0: TaintState.ASSURED, 1: TaintState.EXTERNAL_RAW}) + assert worst_dangerous_arg_taint(call, "m.f", ctx, None) == TaintState.EXTERNAL_RAW + + +def test_spec_restricts_to_dangerous_positions() -> None: + # position 0 (the URL) is trusted; the raw arg sits in a non-dangerous slot + call = _call("requests.get(a, b)") + ctx = _ctx(call, {0: TaintState.ASSURED, 1: TaintState.EXTERNAL_RAW}) + spec = ArgSpec(positions=(0,)) + assert worst_dangerous_arg_taint(call, "m.f", ctx, spec) == TaintState.ASSURED + + +def test_spec_matches_dangerous_keyword() -> None: + call = _call("requests.get(timeout=t, url=u)") + ctx = _ctx(call, {"timeout": TaintState.ASSURED, "url": TaintState.EXTERNAL_RAW}) + spec = ArgSpec(positions=(0,), keywords=("url",)) + assert worst_dangerous_arg_taint(call, "m.f", ctx, spec) == TaintState.EXTERNAL_RAW + + +def test_spec_with_no_dangerous_args_present_returns_none() -> None: + call = _call("requests.get(timeout=t)") + ctx = _ctx(call, {"timeout": TaintState.EXTERNAL_RAW}) + spec = ArgSpec(positions=(0,), keywords=("url",)) + assert worst_dangerous_arg_taint(call, "m.f", ctx, spec) is None + + +def test_starred_positional_is_included_fail_closed() -> None: + # *parts may land in ANY positional slot — fail closed when positions are dangerous + call = _call("requests.get(*parts, t)") + ctx = _ctx( + call, + {0: TaintState.EXTERNAL_RAW, "*0": TaintState.EXTERNAL_RAW, 1: TaintState.ASSURED}, + ) + spec = ArgSpec(positions=(0,)) + assert worst_dangerous_arg_taint(call, "m.f", ctx, spec) == TaintState.EXTERNAL_RAW + + +def test_starred_not_included_when_only_keywords_dangerous() -> None: + call = _call("requests.get(*parts, url=u)") + ctx = _ctx( + call, + {0: TaintState.EXTERNAL_RAW, "*0": TaintState.EXTERNAL_RAW, "url": TaintState.ASSURED}, + ) + spec = ArgSpec(keywords=("url",)) + assert worst_dangerous_arg_taint(call, "m.f", ctx, spec) == TaintState.ASSURED + + +def test_double_star_kwargs_included_for_dangerous_keywords_fail_closed() -> None: + # **kw may supply any keyword — fail closed when keywords are dangerous + call = _call("requests.get(**kw)") + ctx = _ctx(call, {None: TaintState.EXTERNAL_RAW}) + spec = ArgSpec(keywords=("url",)) + assert worst_dangerous_arg_taint(call, "m.f", ctx, spec) == TaintState.EXTERNAL_RAW + + +def test_double_star_kwargs_not_included_when_only_positions_dangerous() -> None: + call = _call("requests.get(a, **kw)") + ctx = _ctx(call, {0: TaintState.ASSURED, None: TaintState.EXTERNAL_RAW}) + spec = ArgSpec(positions=(0,)) + assert worst_dangerous_arg_taint(call, "m.f", ctx, spec) == TaintState.ASSURED + + +def test_missing_snapshot_falls_back_pessimistic_for_dangerous_slot() -> None: + import warnings + + call = _call("requests.get(u)") + ctx = AnalysisContext( + project_taints={}, + project_return_taints={}, + function_var_taints={}, + function_return_taints={}, + function_return_callee={}, + entities={}, + taint_provenance={}, + ) + spec = ArgSpec(positions=(0,)) + # The degradation is RECORDED on the context (surfaced by the analyzer as one + # WLN-ENGINE-FLOW-INSENSITIVE-FALLBACK FACT per scan), never warned — a + # warnings-as-error embedder must not lose a whole rule to the diagnostic. + with warnings.catch_warnings(): + warnings.simplefilter("error") + assert worst_dangerous_arg_taint(call, "m.f", ctx, spec) == TaintState.UNKNOWN_RAW + assert ctx.flow_insensitive_fallbacks == {"m.f"} diff --git a/tests/unit/scanner/rules/test_sink_rules.py b/tests/unit/scanner/rules/test_sink_rules.py index 18811c07..a61f0dce 100644 --- a/tests/unit/scanner/rules/test_sink_rules.py +++ b/tests/unit/scanner/rules/test_sink_rules.py @@ -538,6 +538,21 @@ def f(): assert UntrustedToCommand().check(ctx) == [] +def test_108_undecorated_is_suppressed(tmp_path) -> None: + # Matrix slot (wardline-e159060db7): the suite pinned 106/112 undecorated + # suppression but not 108's — an undecorated (UNKNOWN_RAW freedom-zone) + # function must stay silent even with raw data at the always-shell sink. + ctx = _analyze( + tmp_path, + """ + def f(p): + os.system(read_raw(p)) + return 1 + """, + ) + assert UntrustedToCommand().check(ctx) == [] + + def test_107_self_method_call_arg_fires(tmp_path) -> None: # Sibling method call (self.get_raw) should resolve to EXTERNAL_RAW and trigger PY-WL-107. ctx = _analyze( @@ -767,8 +782,11 @@ def via_from_import_alias(p): ] -def test_fallback_flow_insensitive_warnings() -> None: - # If flow-sensitive map is missing, we warn and pessimistically assume UNKNOWN_RAW. +def test_fallback_flow_insensitive_records_on_the_context() -> None: + # If the flow-sensitive map is missing, the resolver records the degradation on + # the context (one WLN-ENGINE-FLOW-INSENSITIVE-FALLBACK FACT per scan, emitted + # by the analyzer) and pessimistically assumes UNKNOWN_RAW. Never a warning — + # a warnings-as-error embedder must not lose a whole rule to the diagnostic. import ast import warnings @@ -788,9 +806,8 @@ def test_fallback_flow_insensitive_warnings() -> None: taint_provenance={}, ) # Empty context has no flow-sensitive mappings - with warnings.catch_warnings(record=True) as w: - warnings.simplefilter("always") + with warnings.catch_warnings(): + warnings.simplefilter("error") res = worst_arg_taint(call, "m.f", context) - assert len(w) == 1 - assert "WLN-ENGINE-FLOW-INSENSITIVE-FALLBACK" in str(w[0].message) - assert res == TaintState.UNKNOWN_RAW + assert res == TaintState.UNKNOWN_RAW + assert context.flow_insensitive_fallbacks == {"m.f"} diff --git a/tests/unit/scanner/rules/test_sql_injection_expansion.py b/tests/unit/scanner/rules/test_sql_injection_expansion.py new file mode 100644 index 00000000..4a6f9f21 --- /dev/null +++ b/tests/unit/scanner/rules/test_sql_injection_expansion.py @@ -0,0 +1,394 @@ +"""PY-WL-118 precision + coverage expansion (wardline-1751b0fac6 + 2026-06-10 eval FPs). + +Three behaviour groups: + +1. Text-clause constant exemption — ``conn.execute(text("... :id"), {"id": uid})`` + is THE canonical safe SQLAlchemy parameterized pattern: the operation string is a + compile-time constant wrapped in a recognized text-clause constructor, so it cannot + carry attacker bytes. ``text(tainted)`` must still fire (the exemption keys on the + constructor FQN AND all-constant arguments, never on the wrapper alone). + +2. Receiver heuristic — ``.execute``/``.executemany`` matching was receiver-blind, so + ``pool.execute(task)`` on a thread-pool object fired a CWE-89 ERROR. Clearly-non-DB + receivers (pool/executor/worker names, or instances constructed from executor + modules) stop firing; DB-ish and UNKNOWN receivers keep firing (fail-closed — an FN + here is worse than an FP). + +3. Sink coverage — ``executescript`` (sqlite3 cursor/connection; non-parameterizable, + strictly more dangerous than ``execute``) joins the sink set, plus the previously + untested ``executemany`` positive direction. +""" + +from __future__ import annotations + +import textwrap +from collections.abc import Sequence +from pathlib import Path + +from wardline.core.config import WardlineConfig +from wardline.core.finding import Finding, Kind, Severity +from wardline.scanner.analyzer import WardlineAnalyzer + + +def _analyze_files(tmp_path: Path, files: dict[str, str]) -> Sequence[Finding]: + for name, content in files.items(): + p = tmp_path / name + p.parent.mkdir(parents=True, exist_ok=True) + header = ( + "from wardline.decorators import external_boundary, trust_boundary, trusted\n" + "@external_boundary\ndef read_raw(p):\n return p\n" + ) + p.write_text(header + textwrap.dedent(content), encoding="utf-8") + + analyzer = WardlineAnalyzer() + file_paths = [tmp_path / name for name in files] + return analyzer.analyze(file_paths, WardlineConfig(), root=tmp_path) + + +def _sqli(findings: Sequence[Finding]) -> list[Finding]: + return [f for f in findings if f.kind is Kind.DEFECT and f.rule_id == "PY-WL-118"] + + +# ── 1. text-clause constant exemption (canonical SQLAlchemy parameterized query) ── + + +def test_sqlalchemy_text_constant_operation_does_not_fire(tmp_path: Path) -> None: + # The canonical safe pattern: constant SQL wrapped in text(), untrusted value bound. + findings = _analyze_files( + tmp_path, + { + "m.py": """ + from sqlalchemy import text + + @trusted(level='ASSURED') + def f(p, conn): + uid = read_raw(p) + conn.execute(text("SELECT * FROM t WHERE id = :id"), {"id": uid}) + """ + }, + ) + assert _sqli(findings) == [] + + +def test_sqlalchemy_module_alias_text_constant_does_not_fire(tmp_path: Path) -> None: + # Import-alias awareness: ``import sqlalchemy as sa; sa.text(...)`` and the + # ``sqlalchemy.sql.text`` spelling both resolve to a recognized text-clause FQN. + findings = _analyze_files( + tmp_path, + { + "m.py": """ + import sqlalchemy as sa + from sqlalchemy.sql import text as mktext + + @trusted(level='ASSURED') + def f(p, conn): + uid = read_raw(p) + conn.execute(sa.text("SELECT * FROM t WHERE id = :id"), {"id": uid}) + conn.execute(mktext("SELECT * FROM u WHERE id = :id"), {"id": uid}) + """ + }, + ) + assert _sqli(findings) == [] + + +def test_sqlalchemy_text_constant_via_operation_keyword_does_not_fire(tmp_path: Path) -> None: + findings = _analyze_files( + tmp_path, + { + "m.py": """ + from sqlalchemy import text + + @trusted(level='ASSURED') + def f(p, conn): + conn.execute(statement=text("SELECT 1"), parameters={"id": read_raw(p)}) + """ + }, + ) + assert _sqli(findings) == [] + + +def test_sqlalchemy_text_with_tainted_argument_still_fires(tmp_path: Path) -> None: + # text() is NOT a sanitiser: a tainted string inside the wrapper is still SQLi. + findings = _analyze_files( + tmp_path, + { + "m.py": """ + from sqlalchemy import text + + @trusted(level='ASSURED') + def f(p, conn): + conn.execute(text(read_raw(p))) + """ + }, + ) + assert len(_sqli(findings)) == 1 + + +def test_sqlalchemy_text_with_fstring_argument_still_fires(tmp_path: Path) -> None: + # A JoinedStr is not a Constant — interpolated taint inside text() keeps firing. + findings = _analyze_files( + tmp_path, + { + "m.py": """ + from sqlalchemy import text + + @trusted(level='ASSURED') + def f(p, conn): + conn.execute(text(f"SELECT * FROM {read_raw(p)}")) + """ + }, + ) + assert len(_sqli(findings)) == 1 + + +def test_unrecognized_constant_arg_wrapper_still_fires(tmp_path: Path) -> None: + # The exemption is scoped to RECOGNIZED text-clause constructors — an arbitrary + # third-party wrapper around a constant stays fail-closed (UNKNOWN_RAW fires). + findings = _analyze_files( + tmp_path, + { + "m.py": """ + import mylib + + @trusted(level='ASSURED') + def f(p, conn): + conn.execute(mylib.textish("SELECT 1")) + """ + }, + ) + assert len(_sqli(findings)) == 1 + + +def test_sqlalchemy_text_zero_args_stays_fail_closed(tmp_path: Path) -> None: + # text() with no argument proves nothing constant; the slot keeps its engine taint. + findings = _analyze_files( + tmp_path, + { + "m.py": """ + from sqlalchemy import text + + @trusted(level='ASSURED') + def f(p, conn): + conn.execute(text()) + """ + }, + ) + assert len(_sqli(findings)) == 1 + + +# ── 2. receiver heuristic ────────────────────────────────────────────────── + + +def test_non_db_receiver_pool_does_not_fire(tmp_path: Path) -> None: + # The eval FP repro: a thread-pool/command object is not a SQL sink. + findings = _analyze_files( + tmp_path, + { + "m.py": """ + @trusted(level='ASSURED') + def f(p, pool): + task = read_raw(p) + pool.execute(task) + """ + }, + ) + assert _sqli(findings) == [] + + +def test_non_db_receiver_thread_pool_attribute_does_not_fire(tmp_path: Path) -> None: + # Non-DB token match on the LAST attribute segment (self.thread_pool → "thread_pool"). + findings = _analyze_files( + tmp_path, + { + "m.py": """ + @trusted(level='ASSURED') + def f(self, p): + self.thread_pool.execute(read_raw(p)) + """ + }, + ) + assert _sqli(findings) == [] + + +def test_non_db_constructed_executor_does_not_fire(tmp_path: Path) -> None: + # Construct-then-method resolution: an instance provably built from an executor + # module is non-DB even when its NAME looks DB-ish. + findings = _analyze_files( + tmp_path, + { + "m.py": """ + import concurrent.futures + + @trusted(level='ASSURED') + def f(p): + runner = concurrent.futures.ThreadPoolExecutor() + runner.execute(read_raw(p)) + """ + }, + ) + assert _sqli(findings) == [] + + +def test_db_constructed_instance_overrides_non_db_name(tmp_path: Path) -> None: + # Binding evidence beats the name heuristic: a var NAMED "pool" that is provably a + # sqlite3 connection is a real SQL sink and must fire. + findings = _analyze_files( + tmp_path, + { + "m.py": """ + import sqlite3 + + @trusted(level='ASSURED') + def f(p): + pool = sqlite3.connect(":memory:") + pool.execute(read_raw(p)) + """ + }, + ) + assert len(_sqli(findings)) == 1 + + +def test_unknown_receiver_fails_closed_and_fires(tmp_path: Path) -> None: + # An opaque single-letter receiver gives no evidence either way — fail closed, fire + # (FN is worse than FP; covers the SQLAlchemy ``s.execute`` / DB-API ``c.execute`` style). + findings = _analyze_files( + tmp_path, + { + "m.py": """ + @trusted(level='ASSURED') + def f(p, c): + c.execute(read_raw(p)) + """ + }, + ) + assert len(_sqli(findings)) == 1 + + +def test_db_token_receiver_keeps_firing(tmp_path: Path) -> None: + findings = _analyze_files( + tmp_path, + { + "m.py": """ + @trusted(level='ASSURED') + def f(p, db_session): + db_session.execute(read_raw(p)) + """ + }, + ) + assert len(_sqli(findings)) == 1 + + +def test_db_token_wins_over_non_db_token(tmp_path: Path) -> None: + # Mixed evidence ("db_pool") resolves toward firing — conservative by design. + findings = _analyze_files( + tmp_path, + { + "m.py": """ + @trusted(level='ASSURED') + def f(p, db_pool): + db_pool.execute(read_raw(p)) + """ + }, + ) + assert len(_sqli(findings)) == 1 + + +# ── 3. sink coverage: executescript + executemany ────────────────────────── + + +def test_executescript_tainted_fires_at_base_severity(tmp_path: Path) -> None: + findings = _analyze_files( + tmp_path, + { + "m.py": """ + @trusted(level='ASSURED') + def f(p, cursor): + cursor.executescript(read_raw(p)) + """ + }, + ) + sqli = _sqli(findings) + assert len(sqli) == 1 + assert sqli[0].severity is Severity.ERROR + assert sqli[0].properties["sink"] == "executescript" + + +def test_executescript_on_connection_fires(tmp_path: Path) -> None: + # sqlite3 exposes executescript on the CONNECTION as well as the cursor. + findings = _analyze_files( + tmp_path, + { + "m.py": """ + import sqlite3 + + @trusted(level='ASSURED') + def f(p): + conn = sqlite3.connect(":memory:") + conn.executescript(read_raw(p)) + """ + }, + ) + assert len(_sqli(findings)) == 1 + + +def test_executescript_sql_script_keyword_fires(tmp_path: Path) -> None: + # The sqlite3 parameter name spelling of the script slot. + findings = _analyze_files( + tmp_path, + { + "m.py": """ + @trusted(level='ASSURED') + def f(p, cursor): + cursor.executescript(sql_script=read_raw(p)) + """ + }, + ) + assert len(_sqli(findings)) == 1 + + +def test_executescript_constant_script_does_not_fire(tmp_path: Path) -> None: + findings = _analyze_files( + tmp_path, + { + "m.py": """ + @trusted(level='ASSURED') + def f(cursor): + cursor.executescript("CREATE TABLE t (id INTEGER); CREATE INDEX i ON t (id);") + """ + }, + ) + assert _sqli(findings) == [] + + +def test_executemany_tainted_operation_fires(tmp_path: Path) -> None: + # executemany is in _SINKS but had no positive test (wardline-1751b0fac6). + findings = _analyze_files( + tmp_path, + { + "m.py": """ + @trusted(level='ASSURED') + def f(p, cursor): + cursor.executemany(read_raw(p), []) + """ + }, + ) + sqli = _sqli(findings) + assert len(sqli) == 1 + assert sqli[0].properties["sink"] == "executemany" + assert sqli[0].severity is Severity.ERROR + + +def test_118_undecorated_is_suppressed(tmp_path: Path) -> None: + # Matrix slot (wardline-e159060db7): SQLInjection overrides check(), so its + # tier-gate branch needs its own undecorated (UNKNOWN_RAW freedom-zone) + # negative — raw data at the execute sink must stay silent without a trust claim. + findings = _analyze_files( + tmp_path, + { + "m.py": """ + def f(p, cursor): + cursor.execute(read_raw(p)) + """ + }, + ) + assert _sqli(findings) == [] diff --git a/tests/unit/scanner/rules/test_ssrf_expansion.py b/tests/unit/scanner/rules/test_ssrf_expansion.py new file mode 100644 index 00000000..9300acd9 --- /dev/null +++ b/tests/unit/scanner/rules/test_ssrf_expansion.py @@ -0,0 +1,296 @@ +# tests/unit/scanner/rules/test_ssrf_expansion.py +"""PY-WL-117 SSRF expansion — wardline-3002f63969 / wardline-66b2c91470. + +Pins three directions: + 1. Construct-then-method sinks (the dominant real-world shape): methods on a + constructed ``httpx.Client``/``AsyncClient`` (incl. ``async with``), + ``requests.Session()``, and ``aiohttp.ClientSession``. + 2. Module-level gaps: ``requests.head``/``requests.options`` and + ``urllib.request.Request`` (the constructor carries the tainted URL). + 3. Arg-position precision: only the URL slot (and ``base_url=`` on client + constructors) drives the verdict — a tainted ``timeout=``/``headers=``/ + ``verify=`` with a clean literal URL is NOT an SSRF vector. +""" + +from __future__ import annotations + +import textwrap +from pathlib import Path + +from wardline.core.config import WardlineConfig +from wardline.core.finding import Finding, Kind, Severity +from wardline.scanner.analyzer import WardlineAnalyzer + +_HEADER = ( + "import requests\n" + "import httpx\n" + "import aiohttp\n" + "import urllib.request\n" + "from wardline.decorators import external_boundary, trusted\n" + "@external_boundary\n" + "def read_raw(p):\n" + " return p\n" +) + + +def _ssrf(tmp_path: Path, src: str) -> list[Finding]: + p = tmp_path / "m.py" + p.write_text(_HEADER + textwrap.dedent(src), encoding="utf-8") + findings = WardlineAnalyzer().analyze([p], WardlineConfig(), root=tmp_path) + return [f for f in findings if f.rule_id == "PY-WL-117" and f.kind is Kind.DEFECT] + + +# ── 1. construct-then-method ─────────────────────────────────────────────── + + +def test_117_httpx_client_construct_then_get_fires(tmp_path: Path) -> None: + findings = _ssrf( + tmp_path, + """ + @trusted(level='ASSURED') + def f(p): + client = httpx.Client() + client.get(read_raw(p)) + """, + ) + assert [(f.qualname, f.properties["sink"]) for f in findings] == [("m.f", "httpx.Client.get")] + assert findings[0].severity is Severity.WARN + + +def test_117_httpx_async_with_client_fires(tmp_path: Path) -> None: + findings = _ssrf( + tmp_path, + """ + @trusted(level='ASSURED') + async def fetch(p): + async with httpx.AsyncClient() as client: + await client.get(read_raw(p)) + """, + ) + assert [f.properties["sink"] for f in findings] == ["httpx.AsyncClient.get"] + + +def test_117_requests_session_chained_fires(tmp_path: Path) -> None: + findings = _ssrf( + tmp_path, + """ + @trusted(level='ASSURED') + def f(p): + requests.Session().get(read_raw(p)) + """, + ) + assert [f.properties["sink"] for f in findings] == ["requests.Session.get"] + + +def test_117_requests_session_var_post_fires(tmp_path: Path) -> None: + findings = _ssrf( + tmp_path, + """ + @trusted(level='ASSURED') + def f(p): + s = requests.Session() + s.post(read_raw(p)) + """, + ) + assert [f.properties["sink"] for f in findings] == ["requests.Session.post"] + + +def test_117_aiohttp_session_method_fires(tmp_path: Path) -> None: + findings = _ssrf( + tmp_path, + """ + @trusted(level='ASSURED') + async def fetch(p): + async with aiohttp.ClientSession() as session: + await session.get(read_raw(p)) + """, + ) + assert [f.properties["sink"] for f in findings] == ["aiohttp.ClientSession.get"] + + +def test_117_client_request_tainted_url_position_fires(tmp_path: Path) -> None: + # client.request(method, url): the URL is the SECOND slot. + findings = _ssrf( + tmp_path, + """ + @trusted(level='ASSURED') + def f(p): + client = httpx.Client() + client.request("GET", read_raw(p)) + """, + ) + assert [f.properties["sink"] for f in findings] == ["httpx.Client.request"] + + +def test_117_client_request_tainted_method_verb_does_not_fire(tmp_path: Path) -> None: + # A tainted method VERB with a clean literal URL cannot redirect the request target. + findings = _ssrf( + tmp_path, + """ + @trusted(level='ASSURED') + def f(p): + client = httpx.Client() + client.request(read_raw(p), "https://safe.example") + """, + ) + assert findings == [] + + +def test_117_client_stream_fires(tmp_path: Path) -> None: + findings = _ssrf( + tmp_path, + """ + @trusted(level='ASSURED') + def f(p): + with httpx.Client() as client: + client.stream("GET", read_raw(p)) + """, + ) + assert [f.properties["sink"] for f in findings] == ["httpx.Client.stream"] + + +# ── 2. module-level gaps ─────────────────────────────────────────────────── + + +def test_117_requests_head_and_options_fire(tmp_path: Path) -> None: + findings = _ssrf( + tmp_path, + """ + @trusted(level='ASSURED') + def f(p): + requests.head(read_raw(p)) + requests.options(read_raw(p)) + """, + ) + assert [f.properties["sink"] for f in findings] == ["requests.head", "requests.options"] + + +def test_117_urllib_request_request_constructor_fires(tmp_path: Path) -> None: + findings = _ssrf( + tmp_path, + """ + @trusted(level='ASSURED') + def f(p): + req = urllib.request.Request(read_raw(p)) + return req + """, + ) + assert [f.properties["sink"] for f in findings] == ["urllib.request.Request"] + + +def test_117_module_level_requests_get_stays_green(tmp_path: Path) -> None: + findings = _ssrf( + tmp_path, + """ + @trusted(level='ASSURED') + def f(p): + requests.get(read_raw(p)) + """, + ) + assert [f.properties["sink"] for f in findings] == ["requests.get"] + + +# ── 3. arg-position precision ────────────────────────────────────────────── + + +def test_117_tainted_non_url_kwargs_clean_url_do_not_fire(tmp_path: Path) -> None: + findings = _ssrf( + tmp_path, + """ + @trusted(level='ASSURED') + def f(p): + requests.get('https://safe.example', timeout=read_raw(p)) + requests.get('https://safe.example', verify=read_raw(p)) + requests.get('https://safe.example', headers=read_raw(p)) + """, + ) + assert findings == [] + + +def test_117_tainted_url_keyword_fires(tmp_path: Path) -> None: + findings = _ssrf( + tmp_path, + """ + @trusted(level='ASSURED') + def f(p): + requests.get(url=read_raw(p), timeout=5) + """, + ) + assert [f.properties["sink"] for f in findings] == ["requests.get"] + + +def test_117_httpx_client_ctor_tainted_timeout_does_not_fire(tmp_path: Path) -> None: + findings = _ssrf( + tmp_path, + """ + @trusted(level='ASSURED') + def f(p): + httpx.Client(timeout=read_raw(p)) + """, + ) + assert findings == [] + + +def test_117_httpx_client_ctor_tainted_first_positional_does_not_fire(tmp_path: Path) -> None: + # httpx.Client()'s first positional is not a URL; base_url is keyword-only. + findings = _ssrf( + tmp_path, + """ + @trusted(level='ASSURED') + def f(p): + httpx.Client(read_raw(p)) + """, + ) + assert findings == [] + + +def test_117_httpx_client_ctor_tainted_base_url_fires(tmp_path: Path) -> None: + findings = _ssrf( + tmp_path, + """ + @trusted(level='ASSURED') + def f(p): + httpx.Client(base_url=read_raw(p)) + """, + ) + assert [f.properties["sink"] for f in findings] == ["httpx.Client"] + + +def test_117_instance_method_tainted_headers_clean_url_does_not_fire(tmp_path: Path) -> None: + findings = _ssrf( + tmp_path, + """ + @trusted(level='ASSURED') + def f(p): + client = httpx.Client() + client.get('https://safe.example', headers=read_raw(p)) + """, + ) + assert findings == [] + + +def test_117_star_args_widening_fires(tmp_path: Path) -> None: + # *args makes positional slots unmappable: fail-closed widening keeps the verdict. + findings = _ssrf( + tmp_path, + """ + @trusted(level='ASSURED') + def f(p): + parts = read_raw(p) + requests.get(*parts) + """, + ) + assert [f.properties["sink"] for f in findings] == ["requests.get"] + + +def test_117_undecorated_construct_then_method_is_suppressed(tmp_path: Path) -> None: + # Freedom zone: the tier gate applies to the new shapes exactly as to the old ones. + findings = _ssrf( + tmp_path, + """ + def f(p): + client = httpx.Client() + client.get(read_raw(p)) + """, + ) + assert findings == [] diff --git a/tests/unit/scanner/rules/test_stored_taint_expansion.py b/tests/unit/scanner/rules/test_stored_taint_expansion.py new file mode 100644 index 00000000..41eec37c --- /dev/null +++ b/tests/unit/scanner/rules/test_stored_taint_expansion.py @@ -0,0 +1,274 @@ +"""PY-WL-120 precision + stored-source coverage (wardline-66b2c91470 / stored.json FP). + +Three behaviors under test: + +1. **Receiver-aware storage matching.** ``io.StringIO`` / ``io.BytesIO`` are + in-memory buffers — a ``.read()`` on one returns data the process itself put + there, never *persistent storage* — so they are exempt from PY-WL-120's + storage-read matcher (the old matcher accepted ANY ``.read()`` receiver-blind + and mislabeled an in-memory constant as "stored/persisted data"). + +2. **101 delegation on unsubstantiated provenance.** When a matched return's + taint is ``UNKNOWN_RAW``/``MIXED_RAW`` the engine could not resolve where the + value came from, so the "stored/persisted" label rests solely on the AST name + match — unsubstantiated. PY-WL-120 suppresses its return finding and delegates + to PY-WL-101 (which polices the same trust-claim violation, gate-eligibly). + A substantiated ``EXTERNAL_RAW`` storage return keeps the documented + complementary 120+101 pair (both pinned by the frozen identity corpus). + +3. **DB-cursor seeding is real.** ``cursor.fetchone/fetchall/fetchmany()`` seed + ``EXTERNAL_RAW`` (wardline-e7c7cda31a), so PY-WL-120's advertised DB-cursor + coverage fires on both the trusted-callee arm and the return arm. +""" + +from __future__ import annotations + +import textwrap +from collections.abc import Sequence +from pathlib import Path + +from wardline.core.config import WardlineConfig +from wardline.core.finding import Finding, Kind +from wardline.core.taints import TaintState +from wardline.scanner.analyzer import WardlineAnalyzer + + +def _analyze(tmp_path: Path, source: str) -> Sequence[Finding]: + header = ( + "from wardline.decorators import external_boundary, trust_boundary, trusted\n" + "import io\n" + "@trust_boundary(to_level='ASSURED')\n" + "def validate(x):\n" + " if not x:\n raise ValueError\n return x\n" + ) + p = tmp_path / "m.py" + p.write_text(header + textwrap.dedent(source), encoding="utf-8") + analyzer = WardlineAnalyzer() + return analyzer.analyze([p], WardlineConfig(), root=tmp_path) + + +def _rule(findings: Sequence[Finding], rule_id: str) -> list[Finding]: + return [f for f in findings if f.kind is Kind.DEFECT and f.rule_id == rule_id] + + +# ── 1. In-memory buffer receivers are exempt (the verified FP) ─────────────── + + +def test_stringio_constant_read_is_not_stored_data(tmp_path: Path) -> None: + # stored.json FP: an io.StringIO of an internal constant involves zero + # persistent storage — the receiver-blind ``.read()`` match mislabeled it. + findings = _analyze( + tmp_path, + """ + @trusted(level='ASSURED') + def f(): + buf = io.StringIO("internal constant") + data = buf.read() + return data + """, + ) + assert _rule(findings, "PY-WL-120") == [] + + +def test_chained_stringio_read_is_not_stored_data(tmp_path: Path) -> None: + # Chained constructor→method form: io.StringIO("x").read(). + findings = _analyze( + tmp_path, + """ + @trusted(level='ASSURED') + def f(): + return io.StringIO("x").read() + """, + ) + assert _rule(findings, "PY-WL-120") == [] + + +def test_from_import_stringio_read_is_not_stored_data(tmp_path: Path) -> None: + # The exemption resolves through the module import alias map. + findings = _analyze( + tmp_path, + """ + from io import StringIO + + @trusted(level='ASSURED') + def f(): + buf = StringIO("x") + data = buf.read() + return data + """, + ) + assert _rule(findings, "PY-WL-120") == [] + + +def test_bytesio_with_block_read_is_not_stored_data(tmp_path: Path) -> None: + # ``with io.BytesIO(...) as buf`` binds via the context-manager target form. + findings = _analyze( + tmp_path, + """ + @trusted(level='ASSURED') + def f(): + with io.BytesIO(b"x") as buf: + data = buf.read() + return data + """, + ) + assert _rule(findings, "PY-WL-120") == [] + + +def test_open_file_read_still_fires(tmp_path: Path) -> None: + # CONTROL: a genuine file read (open() receiver, EXTERNAL_RAW provenance) + # keeps firing — the exemption is buffer-class-specific, not a .read() FN. + findings = _analyze( + tmp_path, + """ + @trusted(level='ASSURED') + def get_data(): + f = open('data.txt') + content = f.read() + return content + """, + ) + st = _rule(findings, "PY-WL-120") + assert [f.qualname for f in st] == ["m.get_data"] + + +def test_stringio_rebound_to_open_file_still_fires(tmp_path: Path) -> None: + # Last-binding-wins: a name rebound from a buffer to a real file is NOT exempt. + findings = _analyze( + tmp_path, + """ + @trusted(level='ASSURED') + def f(): + h = io.StringIO("x") + h = open('data.txt') + data = h.read() + return data + """, + ) + st = _rule(findings, "PY-WL-120") + assert [f.qualname for f in st] == ["m.f"] + + +# ── 2. Return-arm delegation to PY-WL-101 on unsubstantiated provenance ────── + + +def test_unresolved_receiver_read_return_delegates_to_101(tmp_path: Path) -> None: + # ``legacy.store.read()``: an imported-but-unmodeled dotted call resolves to + # UNKNOWN_RAW — provenance unresolved, so the "stored/persisted" label rests + # solely on the AST ``.read`` name match. 120 suppresses and delegates to 101, + # which polices the same trust-claim violation (ONE finding on the return, + # not two). (A bare PARAM receiver like ``handle.read()`` propagates the + # trusted seed instead — neither rule fires there; documented + # over-approximation, see variable_level.py.) + findings = _analyze( + tmp_path, + """ + import legacy + + @trusted(level='ASSURED') + def f(): + data = legacy.store.read() + return data + """, + ) + assert _rule(findings, "PY-WL-120") == [] + p101 = _rule(findings, "PY-WL-101") + assert [f.qualname for f in p101] == ["m.f"] + assert p101[0].properties["actual_return"] == TaintState.UNKNOWN_RAW.value + + +def test_external_raw_storage_return_keeps_the_documented_pair(tmp_path: Path) -> None: + # A SUBSTANTIATED storage return (EXTERNAL_RAW from the open() seed) keeps the + # deliberate complementary pair: 101 = trust-claim violation (gate-eligible), + # 120 = storage-provenance annotation (PREVIEW). Pinned by the frozen identity + # corpus (sinks fixture: open_catalog_file / lookup_member). + findings = _analyze( + tmp_path, + """ + @trusted(level='ASSURED') + def get_data(): + content = open('data.txt').read() + return content + """, + ) + assert len(_rule(findings, "PY-WL-120")) == 1 + p101 = _rule(findings, "PY-WL-101") + assert len(p101) == 1 + assert p101[0].properties["actual_return"] == TaintState.EXTERNAL_RAW.value + + +# ── 3. DB-cursor fetch coverage (advertised → real; wardline-e7c7cda31a) ───── + + +def test_cursor_fetchone_return_fires_with_external_raw_seed(tmp_path: Path) -> None: + # Acceptance: cursor.fetchone() flowing to a return in a @trusted fn produces + # the documented finding — the fetch* EXTERNAL_RAW seed substantiates the + # storage provenance, so the 120 return arm fires (alongside 101's claim check). + findings = _analyze( + tmp_path, + """ + @trusted(level='ASSURED') + def load_record(cursor): + row = cursor.fetchone() + return row + """, + ) + st = _rule(findings, "PY-WL-120") + assert [f.qualname for f in st] == ["m.load_record"] + assert st[0].properties["return_taint"] == TaintState.EXTERNAL_RAW.value + p101 = _rule(findings, "PY-WL-101") + assert len(p101) == 1 + assert p101[0].properties["actual_return"] == TaintState.EXTERNAL_RAW.value + + +def test_cursor_fetchone_to_trusted_callee_fires(tmp_path: Path) -> None: + # Acceptance: cursor.fetchone() flowing to a trusted-callee sink fires the + # 120 call arm (the arm PY-WL-101 does not cover). + findings = _analyze( + tmp_path, + """ + @trusted(level='ASSURED') + def store(val): + return 1 + + @trusted(level='ASSURED') + def load_record(cursor): + row = cursor.fetchone() + store(row) + """, + ) + st = _rule(findings, "PY-WL-120") + assert [f.qualname for f in st] == ["m.load_record"] + assert st[0].properties["callee"] == "m.store" + assert st[0].properties["arg_taint"] == TaintState.EXTERNAL_RAW.value + + +def test_cursor_fetchmany_return_fires(tmp_path: Path) -> None: + # fetchmany completes the advertised fetch{one,all,many} trio (fetchall is + # pinned in test_wave3_deferred_rules.py). + findings = _analyze( + tmp_path, + """ + @trusted(level='ASSURED') + def load_records(cursor): + rows = cursor.fetchmany(5) + return rows + """, + ) + st = _rule(findings, "PY-WL-120") + assert [f.qualname for f in st] == ["m.load_records"] + assert st[0].properties["return_taint"] == TaintState.EXTERNAL_RAW.value + + +def test_cursor_fetch_validated_stays_silent(tmp_path: Path) -> None: + # CONTROL: laundering through a @trust_boundary clears both arms. + findings = _analyze( + tmp_path, + """ + @trusted(level='ASSURED') + def load_record(cursor): + row = cursor.fetchone() + return validate(row) + """, + ) + assert _rule(findings, "PY-WL-120") == [] diff --git a/tests/unit/scanner/rules/test_taint_closures_locked.py b/tests/unit/scanner/rules/test_taint_closures_locked.py index 65d596c5..138f39ae 100644 --- a/tests/unit/scanner/rules/test_taint_closures_locked.py +++ b/tests/unit/scanner/rules/test_taint_closures_locked.py @@ -28,6 +28,9 @@ def _defects(tmp_path: Path, body: str) -> set[str]: + # Assertions below pin the EXACT defect set (wardline-e159060db7): a + # membership-only check would let a spurious co-firing (e.g. a boundary + # heuristic regression adding PY-WL-102) ship invisibly. p = tmp_path / "m.py" p.write_text(_HEADER + textwrap.dedent(body), encoding="utf-8") analyzer = WardlineAnalyzer() @@ -39,35 +42,35 @@ def _defects(tmp_path: Path, body: str) -> set[str]: def test_comprehension_propagates_raw_to_trusted_return(tmp_path: Path) -> None: - assert "PY-WL-101" in _defects( + assert {"PY-WL-101"} == _defects( tmp_path, "@trusted(level='ASSURED')\ndef f(p):\n return [x for x in read_raw(p)]", ) def test_walrus_propagates_raw_to_trusted_return(tmp_path: Path) -> None: - assert "PY-WL-101" in _defects( + assert {"PY-WL-101"} == _defects( tmp_path, "@trusted(level='ASSURED')\ndef f(p):\n (y := read_raw(p))\n return y", ) def test_starred_unpack_raw_slice_propagates_to_trusted_return(tmp_path: Path) -> None: - assert "PY-WL-101" in _defects( + assert {"PY-WL-101"} == _defects( tmp_path, "@trusted(level='ASSURED')\ndef f(p):\n (a, *rest, c) = (1, read_raw(p), 2)\n return rest", ) def test_existing_comprehension_walrus_target_propagates_to_trusted_return(tmp_path: Path) -> None: - assert "PY-WL-101" in _defects( + assert {"PY-WL-101"} == _defects( tmp_path, "@trusted(level='ASSURED')\ndef f(p):\n x = 1\n [(x := read_raw(p)) for _ in items]\n return x", ) def test_async_for_body_propagates_to_trusted_return(tmp_path: Path) -> None: - assert "PY-WL-101" in _defects( + assert {"PY-WL-101"} == _defects( tmp_path, "@trusted(level='ASSURED')\n" "async def f(p):\n" @@ -78,7 +81,7 @@ def test_async_for_body_propagates_to_trusted_return(tmp_path: Path) -> None: def test_trystar_handler_propagates_to_trusted_return(tmp_path: Path) -> None: - assert "PY-WL-101" in _defects( + assert {"PY-WL-101"} == _defects( tmp_path, "@trusted(level='ASSURED')\ndef f(p):\n" " x = 1\n" @@ -96,7 +99,7 @@ def test_trystar_handler_propagates_to_trusted_return(tmp_path: Path) -> None: @pytest.mark.parametrize("sink_arg", ["*args", "**kwargs"]) def test_starred_raw_into_sink_fires(tmp_path: Path, sink_arg: str) -> None: binder = "args = read_raw(p)" if sink_arg == "*args" else "kwargs = read_raw(p)" - assert "PY-WL-107" in _defects( + assert {"PY-WL-107"} == _defects( tmp_path, f"@trusted(level='ASSURED')\ndef f(p):\n {binder}\n eval({sink_arg})", ) @@ -105,7 +108,7 @@ def test_starred_raw_into_sink_fires(tmp_path: Path, sink_arg: str) -> None: def test_trusted_function_own_varargs_do_not_fire(tmp_path: Path) -> None: # *args/**kwargs of a @trusted function ARE that function's own params at its # declared tier — nothing untrusted entered, so returning them is clean. - assert "PY-WL-101" not in _defects( + assert set() == _defects( tmp_path, "@trusted(level='ASSURED')\ndef f(*args, **kwargs):\n return kwargs", ) @@ -115,7 +118,7 @@ def test_trusted_function_own_varargs_do_not_fire(tmp_path: Path) -> None: def test_call_through_wrapped_callee_propagates(tmp_path: Path) -> None: - assert "PY-WL-101" in _defects( + assert {"PY-WL-101"} == _defects( tmp_path, "def deco(fn):\n" " @functools.wraps(fn)\n def w(*a, **k):\n return fn(*a, **k)\n return w\n" @@ -125,7 +128,7 @@ def test_call_through_wrapped_callee_propagates(tmp_path: Path) -> None: def test_wraps_decorator_stacked_on_producer_propagates(tmp_path: Path) -> None: - assert "PY-WL-101" in _defects( + assert {"PY-WL-101"} == _defects( tmp_path, "def deco(fn):\n" " @functools.wraps(fn)\n def w(*a, **k):\n return fn(*a, **k)\n return w\n" diff --git a/tests/unit/scanner/rules/test_tier_gate_negative.py b/tests/unit/scanner/rules/test_tier_gate_negative.py index 616f7984..52b2bdcc 100644 --- a/tests/unit/scanner/rules/test_tier_gate_negative.py +++ b/tests/unit/scanner/rules/test_tier_gate_negative.py @@ -223,11 +223,13 @@ def test_102_below_anchor_and_non_raising_gate_is_silent_but_trust_boundary_fire # stay silent: (a) an undecorated boundary-shaped function (not anchored), and # (b) a @trusted producer (anchored but body == declared, so not trust-raising). # The positive control is a @trust_boundary with no raise -> fires at base ERROR. + # (Laundered shape: the bare `return p` single-statement body is PY-WL-119's in the + # four-way partition, so 102's control routes through a local.) undecorated = _analyze(tmp_path, "def v(p):\n return p\n") assert BoundaryWithoutRejection().check(undecorated) == [] trusted_not_raising = _analyze(tmp_path, "@trusted(level='ASSURED')\ndef v(p):\n return p\n") assert BoundaryWithoutRejection().check(trusted_not_raising) == [] - fires = _analyze(tmp_path, "@trust_boundary(to_level='ASSURED')\ndef v(p):\n return p\n") + fires = _analyze(tmp_path, "@trust_boundary(to_level='ASSURED')\ndef v(p):\n cleaned = p\n return cleaned\n") findings = BoundaryWithoutRejection().check(fires) assert [(x.rule_id, x.qualname) for x in findings] == [("PY-WL-102", "m.v")] assert findings[0].severity is Severity.ERROR diff --git a/tests/unit/scanner/rules/test_untrusted_to_import.py b/tests/unit/scanner/rules/test_untrusted_to_import.py index 01ff164d..0b725932 100644 --- a/tests/unit/scanner/rules/test_untrusted_to_import.py +++ b/tests/unit/scanner/rules/test_untrusted_to_import.py @@ -1,5 +1,5 @@ # tests/unit/scanner/rules/test_untrusted_to_import.py -"""Tests for PY-WL-115: untrusted module loaded via dynamic import sinks.""" +"""Tests for PY-WL-115: untrusted module loaded via dynamic code/module-load sinks.""" from __future__ import annotations @@ -18,6 +18,7 @@ "def read_raw(p):\n" " return p\n" ) +_HEADER_LINES = _HEADER.count("\n") def _analyze(tmp_path: Path, src: str) -> tuple[WardlineAnalyzer, object]: @@ -57,7 +58,45 @@ def g(p): """, ) findings = UntrustedToImport().check(ctx) - assert [f.rule_id for f in findings] == ["PY-WL-115"] + assert [(f.rule_id, f.qualname, f.kind, f.severity) for f in findings] == [ + ("PY-WL-115", "m.g", Kind.DEFECT, Severity.WARN) + ] + # snippet: blank line, @trusted, def g, sink — the sink sits 4 lines into the snippet + assert findings[0].location.line_start == _HEADER_LINES + 4 + + +def test_115_raw_reaches_aliased_importlib(tmp_path) -> None: + # `import importlib as il` exercises the shared alias-resolution path + # (canonical_call_name), which the direct-spelling tests above do not. + _, ctx = _analyze( + tmp_path, + """ + import importlib as il + + @trusted(level='ASSURED') + def f(p): + il.import_module(read_raw(p)) + return 1 + """, + ) + findings = UntrustedToImport().check(ctx) + assert [(f.rule_id, f.qualname) for f in findings] == [("PY-WL-115", "m.f")] + assert findings[0].severity == Severity.WARN + + +def test_115_undecorated_function_suppressed(tmp_path) -> None: + # No trust declaration → developer-freedom zone → the rule stays silent + # even on a provably raw flow. + _, ctx = _analyze( + tmp_path, + """ + def g(p): + importlib.import_module(read_raw(p)) + return 1 + """, + ) + findings = UntrustedToImport().check(ctx) + assert findings == [] def test_115_clean_import(tmp_path) -> None: @@ -72,3 +111,88 @@ def h(p): ) findings = UntrustedToImport().check(ctx) assert len(findings) == 0 + + +def test_115_dunder_import_clean(tmp_path) -> None: + _, ctx = _analyze( + tmp_path, + """ + @trusted(level='ASSURED') + def h(p): + __import__('os') + return 1 + """, + ) + findings = UntrustedToImport().check(ctx) + assert findings == [] + + +def test_115_raw_reaches_runpy_run_path(tmp_path) -> None: + _, ctx = _analyze( + tmp_path, + """ + import runpy + + @trusted(level='ASSURED') + def f(p): + path = read_raw(p) + runpy.run_path(path) + return 1 + """, + ) + findings = UntrustedToImport().check(ctx) + assert [(f.rule_id, f.qualname, f.kind, f.severity) for f in findings] == [ + ("PY-WL-115", "m.f", Kind.DEFECT, Severity.WARN) + ] + assert findings[0].properties["sink"] == "runpy.run_path" + + +def test_115_raw_reaches_runpy_run_module(tmp_path) -> None: + _, ctx = _analyze( + tmp_path, + """ + import runpy + + @trusted(level='ASSURED') + def f(p): + runpy.run_module(read_raw(p)) + return 1 + """, + ) + findings = UntrustedToImport().check(ctx) + assert [(f.rule_id, f.qualname) for f in findings] == [("PY-WL-115", "m.f")] + assert findings[0].properties["sink"] == "runpy.run_module" + + +def test_115_raw_reaches_spec_from_file_location(tmp_path) -> None: + # The tainted FILE PATH arg is what makes the loader dangerous. + _, ctx = _analyze( + tmp_path, + """ + import importlib.util + + @trusted(level='ASSURED') + def f(p): + importlib.util.spec_from_file_location('m', read_raw(p)) + return 1 + """, + ) + findings = UntrustedToImport().check(ctx) + assert [(f.rule_id, f.qualname) for f in findings] == [("PY-WL-115", "m.f")] + assert findings[0].properties["sink"] == "importlib.util.spec_from_file_location" + + +def test_115_runpy_clean_constant_path(tmp_path) -> None: + _, ctx = _analyze( + tmp_path, + """ + import runpy + + @trusted(level='ASSURED') + def f(p): + runpy.run_path('scripts/fixed.py') + return 1 + """, + ) + findings = UntrustedToImport().check(ctx) + assert findings == [] diff --git a/tests/unit/scanner/rules/test_untrusted_to_log.py b/tests/unit/scanner/rules/test_untrusted_to_log.py new file mode 100644 index 00000000..16c64387 --- /dev/null +++ b/tests/unit/scanner/rules/test_untrusted_to_log.py @@ -0,0 +1,130 @@ +"""PY-WL-125 — untrusted data as a log MESSAGE (log injection, CWE-117).""" + +from __future__ import annotations + +import textwrap +from pathlib import Path + +from wardline.core.config import WardlineConfig +from wardline.core.finding import Kind, Severity +from wardline.scanner.analyzer import WardlineAnalyzer +from wardline.scanner.rules.untrusted_to_log import UntrustedToLog + +_HEADER = ( + "import logging\n" + "from wardline.decorators import external_boundary, trusted\n" + "@external_boundary\n" + "def read_raw(p):\n" + " return p\n" +) + + +def _analyze(tmp_path: Path, src: str): + p = tmp_path / "m.py" + p.write_text(_HEADER + textwrap.dedent(src), encoding="utf-8") + analyzer = WardlineAnalyzer() + findings = analyzer.analyze([p], WardlineConfig(), root=tmp_path) + assert analyzer.last_context is not None + return analyzer.last_context, list(findings) + + +def test_125_module_level_logging_tainted_message_fires_info(tmp_path) -> None: + ctx, _ = _analyze( + tmp_path, + """ + @trusted(level='ASSURED') + def f(p): + logging.info(read_raw(p)) + return 1 + """, + ) + findings = UntrustedToLog().check(ctx) + assert [(x.rule_id, x.qualname) for x in findings] == [("PY-WL-125", "m.f")] + assert findings[0].kind == Kind.DEFECT + # Calibrated INFO: log forging is real but noisy by nature — advisory, never gate-tripping. + assert findings[0].severity == Severity.INFO + + +def test_125_all_module_level_methods_fire(tmp_path) -> None: + ctx, _ = _analyze( + tmp_path, + """ + @trusted(level='ASSURED') + def f(p): + logging.debug(read_raw(p)) + logging.warning(read_raw(p)) + logging.error(read_raw(p)) + logging.critical(read_raw(p)) + logging.exception(read_raw(p)) + return 1 + """, + ) + assert len(UntrustedToLog().check(ctx)) == 5 + + +def test_125_logger_instance_method_fires(tmp_path) -> None: + # construct-then-method: the getLogger(...) binding resolves logger.warning. + ctx, _ = _analyze( + tmp_path, + """ + @trusted(level='ASSURED') + def f(p): + logger = logging.getLogger('app') + logger.warning(read_raw(p)) + return 1 + """, + ) + assert [x.properties["sink"] for x in UntrustedToLog().check(ctx)] == ["logging.getLogger.warning"] + + +def test_125_parameterized_logging_is_the_clean_idiom(tmp_path) -> None: + # Tainted data in the lazy %-args position is logging's SAFE parameterization — + # firing on it would be an FP factory. Only the message FORMAT string counts. + ctx, _ = _analyze( + tmp_path, + """ + @trusted(level='ASSURED') + def f(p): + logging.info('user input = %s', read_raw(p)) + return 1 + """, + ) + assert UntrustedToLog().check(ctx) == [] + + +def test_125_literal_message_is_clean(tmp_path) -> None: + ctx, _ = _analyze( + tmp_path, + """ + @trusted(level='ASSURED') + def f(): + logging.error('fixed message') + return 1 + """, + ) + assert UntrustedToLog().check(ctx) == [] + + +def test_125_undecorated_is_suppressed(tmp_path) -> None: + ctx, _ = _analyze( + tmp_path, + """ + def f(p): + logging.info(read_raw(p)) + return 1 + """, + ) + assert UntrustedToLog().check(ctx) == [] + + +def test_125_registered_end_to_end(tmp_path) -> None: + _, findings = _analyze( + tmp_path, + """ + @trusted(level='ASSURED') + def f(p): + logging.info(read_raw(p)) + return 1 + """, + ) + assert "PY-WL-125" in {f.rule_id for f in findings} diff --git a/tests/unit/scanner/rules/test_untrusted_to_mail.py b/tests/unit/scanner/rules/test_untrusted_to_mail.py new file mode 100644 index 00000000..c68f3d97 --- /dev/null +++ b/tests/unit/scanner/rules/test_untrusted_to_mail.py @@ -0,0 +1,145 @@ +"""PY-WL-126 — untrusted recipient/message reaches SMTP.sendmail (mail injection, CWE-93).""" + +from __future__ import annotations + +import textwrap +from pathlib import Path + +from wardline.core.config import WardlineConfig +from wardline.core.finding import Kind, Severity +from wardline.scanner.analyzer import WardlineAnalyzer +from wardline.scanner.rules.untrusted_to_mail import UntrustedToMail + +_HEADER = ( + "import smtplib\n" + "from wardline.decorators import external_boundary, trusted\n" + "@external_boundary\n" + "def read_raw(p):\n" + " return p\n" +) + + +def _analyze(tmp_path: Path, src: str): + p = tmp_path / "m.py" + p.write_text(_HEADER + textwrap.dedent(src), encoding="utf-8") + analyzer = WardlineAnalyzer() + findings = analyzer.analyze([p], WardlineConfig(), root=tmp_path) + assert analyzer.last_context is not None + return analyzer.last_context, list(findings) + + +def test_126_tainted_message_construct_then_method_fires_warn(tmp_path) -> None: + ctx, _ = _analyze( + tmp_path, + """ + @trusted(level='ASSURED') + def f(p): + s = smtplib.SMTP('localhost') + s.sendmail('from@example.com', 'to@example.com', read_raw(p)) + return 1 + """, + ) + findings = UntrustedToMail().check(ctx) + assert [(x.rule_id, x.qualname) for x in findings] == [("PY-WL-126", "m.f")] + assert findings[0].kind == Kind.DEFECT + assert findings[0].severity == Severity.WARN + assert findings[0].properties["sink"] == "smtplib.SMTP.sendmail" + + +def test_126_tainted_recipient_fires(tmp_path) -> None: + ctx, _ = _analyze( + tmp_path, + """ + @trusted(level='ASSURED') + def f(p): + s = smtplib.SMTP('localhost') + s.sendmail('from@example.com', read_raw(p), 'body') + return 1 + """, + ) + assert [x.rule_id for x in UntrustedToMail().check(ctx)] == ["PY-WL-126"] + + +def test_126_with_statement_binding_fires(tmp_path) -> None: + ctx, _ = _analyze( + tmp_path, + """ + @trusted(level='ASSURED') + def f(p): + with smtplib.SMTP('localhost') as s: + s.sendmail('from@example.com', 'to@example.com', read_raw(p)) + return 1 + """, + ) + assert [x.rule_id for x in UntrustedToMail().check(ctx)] == ["PY-WL-126"] + + +def test_126_smtp_ssl_fires(tmp_path) -> None: + ctx, _ = _analyze( + tmp_path, + """ + @trusted(level='ASSURED') + def f(p): + s = smtplib.SMTP_SSL('localhost') + s.sendmail('from@example.com', 'to@example.com', read_raw(p)) + return 1 + """, + ) + assert [x.properties["sink"] for x in UntrustedToMail().check(ctx)] == ["smtplib.SMTP_SSL.sendmail"] + + +def test_126_literal_recipient_and_message_are_clean(tmp_path) -> None: + ctx, _ = _analyze( + tmp_path, + """ + @trusted(level='ASSURED') + def f(): + s = smtplib.SMTP('localhost') + s.sendmail('from@example.com', 'to@example.com', 'body') + return 1 + """, + ) + assert UntrustedToMail().check(ctx) == [] + + +def test_126_tainted_from_addr_only_is_clean(tmp_path) -> None: + # Charter: recipient (to_addrs) + message are the CWE-93 injection slots; + # from_addr alone is out of scope (documented calibration). + ctx, _ = _analyze( + tmp_path, + """ + @trusted(level='ASSURED') + def f(p): + s = smtplib.SMTP('localhost') + s.sendmail(read_raw(p), 'to@example.com', 'body') + return 1 + """, + ) + assert UntrustedToMail().check(ctx) == [] + + +def test_126_undecorated_is_suppressed(tmp_path) -> None: + ctx, _ = _analyze( + tmp_path, + """ + def f(p): + s = smtplib.SMTP('localhost') + s.sendmail('from@example.com', 'to@example.com', read_raw(p)) + return 1 + """, + ) + assert UntrustedToMail().check(ctx) == [] + + +def test_126_registered_end_to_end(tmp_path) -> None: + _, findings = _analyze( + tmp_path, + """ + @trusted(level='ASSURED') + def f(p): + s = smtplib.SMTP('localhost') + s.sendmail('from@example.com', 'to@example.com', read_raw(p)) + return 1 + """, + ) + assert "PY-WL-126" in {f.rule_id for f in findings} diff --git a/tests/unit/scanner/rules/test_untrusted_to_native.py b/tests/unit/scanner/rules/test_untrusted_to_native.py new file mode 100644 index 00000000..0fbc32a5 --- /dev/null +++ b/tests/unit/scanner/rules/test_untrusted_to_native.py @@ -0,0 +1,100 @@ +"""PY-WL-124 — untrusted path reaches a native-library load sink (ctypes, CWE-114).""" + +from __future__ import annotations + +import textwrap +from pathlib import Path + +from wardline.core.config import WardlineConfig +from wardline.core.finding import Kind, Severity +from wardline.scanner.analyzer import WardlineAnalyzer +from wardline.scanner.rules.untrusted_to_native import UntrustedToNative + +_HEADER = ( + "import ctypes\n" + "from wardline.decorators import external_boundary, trusted\n" + "@external_boundary\n" + "def read_raw(p):\n" + " return p\n" +) + + +def _analyze(tmp_path: Path, src: str): + p = tmp_path / "m.py" + p.write_text(_HEADER + textwrap.dedent(src), encoding="utf-8") + analyzer = WardlineAnalyzer() + findings = analyzer.analyze([p], WardlineConfig(), root=tmp_path) + assert analyzer.last_context is not None + return analyzer.last_context, list(findings) + + +def test_124_ctypes_cdll_fires_error(tmp_path) -> None: + ctx, _ = _analyze( + tmp_path, + """ + @trusted(level='ASSURED') + def f(p): + return ctypes.CDLL(read_raw(p)) + """, + ) + findings = UntrustedToNative().check(ctx) + assert [(x.rule_id, x.qualname) for x in findings] == [("PY-WL-124", "m.f")] + assert findings[0].kind == Kind.DEFECT + assert findings[0].severity == Severity.ERROR # arbitrary native-code execution + + +def test_124_dll_variants_and_loadlibrary_fire(tmp_path) -> None: + ctx, _ = _analyze( + tmp_path, + """ + @trusted(level='ASSURED') + def f(p): + ctypes.WinDLL(read_raw(p)) + ctypes.OleDLL(read_raw(p)) + ctypes.PyDLL(read_raw(p)) + ctypes.cdll.LoadLibrary(read_raw(p)) + return 1 + """, + ) + sinks = sorted(x.properties["sink"] for x in UntrustedToNative().check(ctx)) + assert sinks == [ + "ctypes.OleDLL", + "ctypes.PyDLL", + "ctypes.WinDLL", + "ctypes.cdll.LoadLibrary", + ] + + +def test_124_literal_library_path_is_clean(tmp_path) -> None: + ctx, _ = _analyze( + tmp_path, + """ + @trusted(level='ASSURED') + def f(): + return ctypes.CDLL('libm.so.6') + """, + ) + assert UntrustedToNative().check(ctx) == [] + + +def test_124_undecorated_is_suppressed(tmp_path) -> None: + ctx, _ = _analyze( + tmp_path, + """ + def f(p): + return ctypes.CDLL(read_raw(p)) + """, + ) + assert UntrustedToNative().check(ctx) == [] + + +def test_124_registered_end_to_end(tmp_path) -> None: + _, findings = _analyze( + tmp_path, + """ + @trusted(level='ASSURED') + def f(p): + return ctypes.cdll.LoadLibrary(read_raw(p)) + """, + ) + assert "PY-WL-124" in {f.rule_id for f in findings} diff --git a/tests/unit/scanner/rules/test_untrusted_to_reflection.py b/tests/unit/scanner/rules/test_untrusted_to_reflection.py new file mode 100644 index 00000000..401fa45e --- /dev/null +++ b/tests/unit/scanner/rules/test_untrusted_to_reflection.py @@ -0,0 +1,117 @@ +"""PY-WL-123 — tainted attribute NAME reaches setattr/getattr (CWE-915 mass assignment).""" + +from __future__ import annotations + +import textwrap +from pathlib import Path + +from wardline.core.config import WardlineConfig +from wardline.core.finding import Kind, Severity +from wardline.scanner.analyzer import WardlineAnalyzer +from wardline.scanner.rules.untrusted_to_reflection import UntrustedToReflection + +_HEADER = ( + "from wardline.decorators import external_boundary, trusted\n@external_boundary\ndef read_raw(p):\n return p\n" +) + + +def _analyze(tmp_path: Path, src: str): + p = tmp_path / "m.py" + p.write_text(_HEADER + textwrap.dedent(src), encoding="utf-8") + analyzer = WardlineAnalyzer() + findings = analyzer.analyze([p], WardlineConfig(), root=tmp_path) + assert analyzer.last_context is not None + return analyzer.last_context, list(findings) + + +def test_123_setattr_tainted_name_fires_warn(tmp_path) -> None: + ctx, _ = _analyze( + tmp_path, + """ + @trusted(level='ASSURED') + def f(p, obj): + setattr(obj, read_raw(p), 1) + return 1 + """, + ) + findings = UntrustedToReflection().check(ctx) + assert [(x.rule_id, x.qualname) for x in findings] == [("PY-WL-123", "m.f")] + assert findings[0].kind == Kind.DEFECT + assert findings[0].severity == Severity.WARN + + +def test_123_getattr_tainted_name_fires(tmp_path) -> None: + ctx, _ = _analyze( + tmp_path, + """ + @trusted(level='ASSURED') + def f(p, obj): + return getattr(obj, read_raw(p)) + """, + ) + assert [x.properties["sink"] for x in UntrustedToReflection().check(ctx)] == ["getattr"] + + +def test_123_tainted_value_with_literal_name_is_clean(tmp_path) -> None: + # Only the attribute NAME slot (position 1) is the mass-assignment vector; + # an untrusted VALUE assigned to a fixed attribute is ordinary data flow. + ctx, _ = _analyze( + tmp_path, + """ + @trusted(level='ASSURED') + def f(p, obj): + setattr(obj, 'name', read_raw(p)) + return 1 + """, + ) + assert UntrustedToReflection().check(ctx) == [] + + +def test_123_tainted_getattr_default_is_clean(tmp_path) -> None: + ctx, _ = _analyze( + tmp_path, + """ + @trusted(level='ASSURED') + def f(p, obj): + return getattr(obj, 'name', read_raw(p)) + """, + ) + assert UntrustedToReflection().check(ctx) == [] + + +def test_123_tainted_receiver_is_clean(tmp_path) -> None: + # A tainted RECEIVER with a literal name is not attribute injection. + ctx, _ = _analyze( + tmp_path, + """ + @trusted(level='ASSURED') + def f(p): + return getattr(read_raw(p), 'name') + """, + ) + assert UntrustedToReflection().check(ctx) == [] + + +def test_123_undecorated_is_suppressed(tmp_path) -> None: + ctx, _ = _analyze( + tmp_path, + """ + def f(p, obj): + setattr(obj, read_raw(p), 1) + return 1 + """, + ) + assert UntrustedToReflection().check(ctx) == [] + + +def test_123_registered_end_to_end(tmp_path) -> None: + _, findings = _analyze( + tmp_path, + """ + @trusted(level='ASSURED') + def f(p, obj): + setattr(obj, read_raw(p), 1) + return 1 + """, + ) + assert "PY-WL-123" in {f.rule_id for f in findings} diff --git a/tests/unit/scanner/rules/test_untrusted_to_template.py b/tests/unit/scanner/rules/test_untrusted_to_template.py new file mode 100644 index 00000000..ae3239fe --- /dev/null +++ b/tests/unit/scanner/rules/test_untrusted_to_template.py @@ -0,0 +1,135 @@ +"""PY-WL-122 — untrusted data compiled into a server-side template (SSTI, CWE-1336).""" + +from __future__ import annotations + +import textwrap +from pathlib import Path + +from wardline.core.config import WardlineConfig +from wardline.core.finding import Kind, Severity +from wardline.scanner.analyzer import WardlineAnalyzer +from wardline.scanner.rules.untrusted_to_template import UntrustedToTemplate + +_HEADER = ( + "from wardline.decorators import external_boundary, trusted\n@external_boundary\ndef read_raw(p):\n return p\n" +) + + +def _analyze(tmp_path: Path, src: str): + p = tmp_path / "m.py" + p.write_text(_HEADER + textwrap.dedent(src), encoding="utf-8") + analyzer = WardlineAnalyzer() + findings = analyzer.analyze([p], WardlineConfig(), root=tmp_path) + assert analyzer.last_context is not None + return analyzer.last_context, list(findings) + + +def test_122_jinja2_template_fires_error(tmp_path) -> None: + ctx, _ = _analyze( + tmp_path, + """ + import jinja2 + @trusted(level='ASSURED') + def f(p): + return jinja2.Template(read_raw(p)).render() + """, + ) + findings = UntrustedToTemplate().check(ctx) + assert [(x.rule_id, x.qualname) for x in findings] == [("PY-WL-122", "m.f")] + assert findings[0].kind == Kind.DEFECT + assert findings[0].severity == Severity.ERROR # SSTI is RCE-adjacent + + +def test_122_environment_from_string_construct_then_method(tmp_path) -> None: + ctx, _ = _analyze( + tmp_path, + """ + import jinja2 + @trusted(level='ASSURED') + def f(p): + env = jinja2.Environment() + return env.from_string(read_raw(p)) + """, + ) + findings = UntrustedToTemplate().check(ctx) + assert [x.properties["sink"] for x in findings] == ["jinja2.Environment.from_string"] + + +def test_122_chained_environment_from_string(tmp_path) -> None: + ctx, _ = _analyze( + tmp_path, + """ + import jinja2 + @trusted(level='ASSURED') + def f(p): + return jinja2.Environment().from_string(read_raw(p)) + """, + ) + assert [x.rule_id for x in UntrustedToTemplate().check(ctx)] == ["PY-WL-122"] + + +def test_122_mako_template_fires(tmp_path) -> None: + ctx, _ = _analyze( + tmp_path, + """ + import mako.template + @trusted(level='ASSURED') + def f(p): + return mako.template.Template(read_raw(p)) + """, + ) + assert [x.properties["sink"] for x in UntrustedToTemplate().check(ctx)] == ["mako.template.Template"] + + +def test_122_literal_template_with_tainted_render_context_is_clean(tmp_path) -> None: + # Tainted data as a RENDER variable is the safe idiom — only the template + # SOURCE being tainted is SSTI. + ctx, _ = _analyze( + tmp_path, + """ + import jinja2 + @trusted(level='ASSURED') + def f(p): + return jinja2.Template('Hello {{ name }}').render(name=read_raw(p)) + """, + ) + assert UntrustedToTemplate().check(ctx) == [] + + +def test_122_get_template_by_name_is_not_a_sink(tmp_path) -> None: + ctx, _ = _analyze( + tmp_path, + """ + import jinja2 + @trusted(level='ASSURED') + def f(p): + env = jinja2.Environment() + return env.get_template(read_raw(p)) + """, + ) + assert UntrustedToTemplate().check(ctx) == [] + + +def test_122_undecorated_is_suppressed(tmp_path) -> None: + ctx, _ = _analyze( + tmp_path, + """ + import jinja2 + def f(p): + return jinja2.Template(read_raw(p)) + """, + ) + assert UntrustedToTemplate().check(ctx) == [] + + +def test_122_registered_end_to_end(tmp_path) -> None: + _, findings = _analyze( + tmp_path, + """ + import jinja2 + @trusted(level='ASSURED') + def f(p): + return jinja2.Template(read_raw(p)) + """, + ) + assert "PY-WL-122" in {f.rule_id for f in findings} diff --git a/tests/unit/scanner/rules/test_untrusted_to_xml.py b/tests/unit/scanner/rules/test_untrusted_to_xml.py new file mode 100644 index 00000000..bea5ee34 --- /dev/null +++ b/tests/unit/scanner/rules/test_untrusted_to_xml.py @@ -0,0 +1,174 @@ +"""PY-WL-121 — untrusted data reaches an XML parsing sink (XXE / billion-laughs, CWE-611).""" + +from __future__ import annotations + +import textwrap +from pathlib import Path + +from wardline.core.config import WardlineConfig +from wardline.core.finding import Kind, Severity +from wardline.scanner.analyzer import WardlineAnalyzer +from wardline.scanner.rules.untrusted_to_xml import UntrustedToXML + +_HEADER = ( + "from wardline.decorators import external_boundary, trusted\n@external_boundary\ndef read_raw(p):\n return p\n" +) + + +def _analyze(tmp_path: Path, src: str): + p = tmp_path / "m.py" + p.write_text(_HEADER + textwrap.dedent(src), encoding="utf-8") + analyzer = WardlineAnalyzer() + findings = analyzer.analyze([p], WardlineConfig(), root=tmp_path) + assert analyzer.last_context is not None + return analyzer.last_context, list(findings) + + +def test_121_stdlib_etree_fromstring_fires_warn(tmp_path) -> None: + ctx, _ = _analyze( + tmp_path, + """ + import xml.etree.ElementTree as ET + @trusted(level='ASSURED') + def f(p): + return ET.fromstring(read_raw(p)) + """, + ) + findings = UntrustedToXML().check(ctx) + assert [(x.rule_id, x.qualname) for x in findings] == [("PY-WL-121", "m.f")] + assert findings[0].kind == Kind.DEFECT + # stdlib etree is entity-safe since 3.7.1 — billion-laughs DoS only -> WARN + assert findings[0].severity == Severity.WARN + + +def test_121_stdlib_etree_parse_and_iterparse_fire(tmp_path) -> None: + ctx, _ = _analyze( + tmp_path, + """ + import xml.etree.ElementTree as ET + @trusted(level='ASSURED') + def f(p): + ET.parse(read_raw(p)) + ET.iterparse(read_raw(p)) + return 1 + """, + ) + sinks = sorted(x.properties["sink"] for x in UntrustedToXML().check(ctx)) + assert sinks == ["xml.etree.ElementTree.iterparse", "xml.etree.ElementTree.parse"] + + +def test_121_minidom_and_sax_fire_warn(tmp_path) -> None: + ctx, _ = _analyze( + tmp_path, + """ + import xml.dom.minidom + import xml.sax + @trusted(level='ASSURED') + def f(p, h): + xml.dom.minidom.parseString(read_raw(p)) + xml.sax.parseString(read_raw(p), h) + return 1 + """, + ) + findings = UntrustedToXML().check(ctx) + assert sorted(x.properties["sink"] for x in findings) == [ + "xml.dom.minidom.parseString", + "xml.sax.parseString", + ] + # All pyexpat-based stdlib parsers share the same default-on risk class (DoS). + assert {x.severity for x in findings} == {Severity.WARN} + + +def test_121_lxml_fires_error(tmp_path) -> None: + ctx, _ = _analyze( + tmp_path, + """ + from lxml import etree + @trusted(level='ASSURED') + def f(p): + return etree.fromstring(read_raw(p)) + """, + ) + findings = UntrustedToXML().check(ctx) + assert [x.properties["sink"] for x in findings] == ["lxml.etree.fromstring"] + # lxml resolves external entities by default — genuine XXE -> ERROR + assert findings[0].severity == Severity.ERROR + + +def test_121_keyword_spelling_fires(tmp_path) -> None: + ctx, _ = _analyze( + tmp_path, + """ + import xml.etree.ElementTree as ET + @trusted(level='ASSURED') + def f(p): + return ET.fromstring(text=read_raw(p)) + """, + ) + assert [x.rule_id for x in UntrustedToXML().check(ctx)] == ["PY-WL-121"] + + +def test_121_literal_xml_is_clean(tmp_path) -> None: + ctx, _ = _analyze( + tmp_path, + """ + import xml.etree.ElementTree as ET + @trusted(level='ASSURED') + def f(): + return ET.fromstring('') + """, + ) + assert UntrustedToXML().check(ctx) == [] + + +def test_121_taint_in_non_dangerous_slot_is_clean(tmp_path) -> None: + # The parser keyword is not the XML document slot — taint there is not XXE. + ctx, _ = _analyze( + tmp_path, + """ + import xml.etree.ElementTree as ET + @trusted(level='ASSURED') + def f(p): + return ET.fromstring('', parser=read_raw(p)) + """, + ) + assert UntrustedToXML().check(ctx) == [] + + +def test_121_defusedxml_is_not_a_sink(tmp_path) -> None: + ctx, _ = _analyze( + tmp_path, + """ + import defusedxml.ElementTree + @trusted(level='ASSURED') + def f(p): + return defusedxml.ElementTree.fromstring(read_raw(p)) + """, + ) + assert UntrustedToXML().check(ctx) == [] + + +def test_121_undecorated_is_suppressed(tmp_path) -> None: + # Freedom zone -> modulate -> NONE -> no finding (opt-in preserved). + ctx, _ = _analyze( + tmp_path, + """ + import xml.etree.ElementTree as ET + def f(p): + return ET.fromstring(read_raw(p)) + """, + ) + assert UntrustedToXML().check(ctx) == [] + + +def test_121_registered_end_to_end(tmp_path) -> None: + _, findings = _analyze( + tmp_path, + """ + from lxml import etree + @trusted(level='ASSURED') + def f(p): + return etree.parse(read_raw(p)) + """, + ) + assert "PY-WL-121" in {f.rule_id for f in findings} diff --git a/tests/unit/scanner/rules/test_vocabulary_shape_pin.py b/tests/unit/scanner/rules/test_vocabulary_shape_pin.py index c7eee0bc..cf71b329 100644 --- a/tests/unit/scanner/rules/test_vocabulary_shape_pin.py +++ b/tests/unit/scanner/rules/test_vocabulary_shape_pin.py @@ -4,9 +4,11 @@ from pathlib import Path from wardline.core.config import WardlineConfig +from wardline.core.finding import Kind, Maturity, Severity from wardline.core.taints import TRUST_RANK from wardline.core.taints import TaintState as T from wardline.scanner.analyzer import WardlineAnalyzer +from wardline.scanner.rules import build_default_registry def _analyze(tmp_path: Path, files: dict[str, str]): @@ -40,3 +42,53 @@ def test_decorator_taint_shapes_are_distinct_and_stable(tmp_path) -> None: assert TRUST_RANK[body["m.tb"]] > TRUST_RANK[ret["m.tb"]] # @trusted: body == return, both trusted (NOT a transition) assert body["m.tr"] == ret["m.tr"] == T.ASSURED + + +# The full rule-metadata shape pin: every builtin PY-WL rule's (severity, kind, +# maturity) triple is part of the product vocabulary (gate behavior, baseline +# eligibility, doc tables). Asserting dict EQUALITY pins all three drift axes at +# once: a renamed/removed/added rule id changes the key set; a recalibrated +# severity (e.g. the 108/112 WARN->ERROR calibration), a kind change, or a +# PREVIEW->STABLE graduation changes a value. Any such drift must be a +# deliberate edit HERE, not an accident. +_EXPECTED_RULE_SHAPE = { + "PY-WL-101": (Severity.ERROR, Kind.DEFECT, Maturity.STABLE), + "PY-WL-102": (Severity.ERROR, Kind.DEFECT, Maturity.STABLE), + "PY-WL-103": (Severity.WARN, Kind.DEFECT, Maturity.STABLE), + "PY-WL-104": (Severity.WARN, Kind.DEFECT, Maturity.STABLE), + "PY-WL-105": (Severity.ERROR, Kind.DEFECT, Maturity.STABLE), + "PY-WL-106": (Severity.WARN, Kind.DEFECT, Maturity.STABLE), + "PY-WL-107": (Severity.WARN, Kind.DEFECT, Maturity.STABLE), + "PY-WL-108": (Severity.ERROR, Kind.DEFECT, Maturity.STABLE), + "PY-WL-109": (Severity.WARN, Kind.DEFECT, Maturity.STABLE), + "PY-WL-110": (Severity.WARN, Kind.DEFECT, Maturity.STABLE), + "PY-WL-111": (Severity.ERROR, Kind.DEFECT, Maturity.STABLE), + "PY-WL-112": (Severity.ERROR, Kind.DEFECT, Maturity.STABLE), + "PY-WL-113": (Severity.ERROR, Kind.DEFECT, Maturity.STABLE), + "PY-WL-114": (Severity.ERROR, Kind.DEFECT, Maturity.STABLE), + "PY-WL-115": (Severity.WARN, Kind.DEFECT, Maturity.STABLE), + "PY-WL-116": (Severity.WARN, Kind.DEFECT, Maturity.PREVIEW), + "PY-WL-117": (Severity.WARN, Kind.DEFECT, Maturity.PREVIEW), + "PY-WL-118": (Severity.ERROR, Kind.DEFECT, Maturity.PREVIEW), + "PY-WL-119": (Severity.ERROR, Kind.DEFECT, Maturity.PREVIEW), + "PY-WL-120": (Severity.ERROR, Kind.DEFECT, Maturity.PREVIEW), + "PY-WL-121": (Severity.ERROR, Kind.DEFECT, Maturity.PREVIEW), + "PY-WL-122": (Severity.ERROR, Kind.DEFECT, Maturity.PREVIEW), + "PY-WL-123": (Severity.WARN, Kind.DEFECT, Maturity.PREVIEW), + "PY-WL-124": (Severity.ERROR, Kind.DEFECT, Maturity.PREVIEW), + "PY-WL-125": (Severity.INFO, Kind.DEFECT, Maturity.PREVIEW), + "PY-WL-126": (Severity.WARN, Kind.DEFECT, Maturity.PREVIEW), +} + + +def test_builtin_rule_metadata_shape_is_pinned() -> None: + reg = build_default_registry(WardlineConfig()) + actual = {r.metadata.rule_id: (r.metadata.base_severity, r.metadata.kind, r.metadata.maturity) for r in reg.rules} + assert actual == _EXPECTED_RULE_SHAPE + + +def test_rule_metadata_id_matches_class_rule_id() -> None: + # The registry keys findings by `rule.rule_id` while docs/vocab read + # `rule.metadata.rule_id`; a divergence would mislabel every finding of that rule. + reg = build_default_registry(WardlineConfig()) + assert [(r.rule_id, r.metadata.rule_id) for r in reg.rules if r.rule_id != r.metadata.rule_id] == [] diff --git a/tests/unit/scanner/rules/test_wave2_engine_precision.py b/tests/unit/scanner/rules/test_wave2_engine_precision.py index 80b40462..ee4000b2 100644 --- a/tests/unit/scanner/rules/test_wave2_engine_precision.py +++ b/tests/unit/scanner/rules/test_wave2_engine_precision.py @@ -20,19 +20,24 @@ from wardline.core.finding import Finding, Kind from wardline.scanner.analyzer import WardlineAnalyzer +# Shared file preamble. Line-number assertions below are expressed relative to +# _HEADER_LINES (wardline-e159060db7): adding a line here used to silently shift +# four absolute line-number assertions with no engine change. +_HEADER = ( + "from wardline.decorators import external_boundary, trust_boundary, trusted\n" + "@external_boundary\ndef read_raw(p):\n return p\n" + "@trust_boundary(to_level='ASSURED')\n" + "def validate(x):\n" + " if not x:\n raise ValueError\n return x\n" +) +_HEADER_LINES = _HEADER.count("\n") + def _analyze_files(tmp_path: Path, files: dict[str, str]) -> Sequence[Finding]: for name, content in files.items(): p = tmp_path / name p.parent.mkdir(parents=True, exist_ok=True) - header = ( - "from wardline.decorators import external_boundary, trust_boundary, trusted\n" - "@external_boundary\ndef read_raw(p):\n return p\n" - "@trust_boundary(to_level='ASSURED')\n" - "def validate(x):\n" - " if not x:\n raise ValueError\n return x\n" - ) - p.write_text(header + textwrap.dedent(content), encoding="utf-8") + p.write_text(_HEADER + textwrap.dedent(content), encoding="utf-8") analyzer = WardlineAnalyzer() file_paths = [tmp_path / name for name in files] @@ -64,7 +69,8 @@ def test_flow(p): # Check that PY-WL-107 is raised, but only for the first eval. py_wl_107_findings = [f for f in defects if f.rule_id == "PY-WL-107"] assert len(py_wl_107_findings) == 1 - assert py_wl_107_findings[0].location.line_start == 14 + # Snippet line 5 (leading blank, @trusted, def, x=, eval) after the header. + assert py_wl_107_findings[0].location.line_start == _HEADER_LINES + 5 def test_starred_args_and_kwargs_resolution_flow_sensitive(tmp_path: Path) -> None: @@ -93,7 +99,8 @@ def test_starred(p): assert len(py_wl_107_findings) == 1 assert all(f.qualname == "m.target" for f in py_wl_107_findings) lines = sorted((f.location.line_start or 0) for f in py_wl_107_findings) - assert lines == [13] + # eval(a) is snippet line 4 (leading blank, @trusted, def, eval) after the header. + assert lines == [_HEADER_LINES + 4] def test_kwargs_unpack_only_taints_unfilled_keyword_capable_parameters(tmp_path: Path) -> None: @@ -122,7 +129,8 @@ def test_kwargs(p): defects = [f for f in findings if f.kind is Kind.DEFECT] py_wl_107_findings = [f for f in defects if f.rule_id == "PY-WL-107" and f.qualname == "m.target"] lines = sorted((f.location.line_start or 0) for f in py_wl_107_findings) - assert lines == [18, 20] + # eval(c) / eval(extra) are snippet lines 9 and 11 after the header. + assert lines == [_HEADER_LINES + 9, _HEADER_LINES + 11] # ── WP8: Cross-Module Call-Argument Taint ────────────────────────────────── diff --git a/tests/unit/scanner/rules/test_wave3_deferred_rules.py b/tests/unit/scanner/rules/test_wave3_deferred_rules.py index c6849a23..f8c9dc57 100644 --- a/tests/unit/scanner/rules/test_wave3_deferred_rules.py +++ b/tests/unit/scanner/rules/test_wave3_deferred_rules.py @@ -395,6 +395,27 @@ def safe(x): assert len(db_findings) == 0 +def test_degenerate_shape_without_boundary_claim_is_suppressed(tmp_path: Path) -> None: + # Tier-suppression matrix slot (wardline-e159060db7): PY-WL-119 is gated on an + # ANCHORED trust-RAISING declaration. The byte-identical bare-passthrough body + # must stay silent both undecorated (not anchored) and under @trusted + # (anchored, but body == return -> not a trust-raising boundary). + findings = _analyze_files( + tmp_path, + { + "m.py": """ + def plain(x): + return x + + @trusted(level='ASSURED') + def producer(x): + return x + """ + }, + ) + assert [f for f in findings if f.rule_id == "PY-WL-119"] == [] + + # ── PY-WL-120: Stored Taint ──────────────────────────────────────────────── diff --git a/tests/unit/scanner/taint/CHOKEPOINT_NOTES.md b/tests/unit/scanner/taint/CHOKEPOINT_NOTES.md new file mode 100644 index 00000000..8f8a2e58 --- /dev/null +++ b/tests/unit/scanner/taint/CHOKEPOINT_NOTES.md @@ -0,0 +1,161 @@ +# Chokepoint contextvar coupling notes (`scanner/taint/variable_level.py`) + +Companion to `test_property_chokepoint.py` (wardline-369f54b83b). The L2 walk +threads SEVEN pieces of ambient state through `contextvars.ContextVar` slots +instead of parameters. This invisible coupling is where new handlers go wrong: +a handler that forgets to branch-copy / merge / reset one of these maps either +leaks state across mutually-exclusive branch arms (FP/FN) or across functions +(stale-state launder). This file documents, per contextvar, who SETS the slot +(rebinds it, with token/reset discipline), who READS it, and who MUTATES the +dict it points to (a mutation is visible to every reader of the same slot +binding — strictly stronger coupling than a read). + +Line numbers reference the file as of 2026-06-11 (2,178 lines); they drift, +the handler names do not. + +## The seven contextvars + +| ContextVar | Type | Purpose | +|---|---|---| +| `_CURRENT_ALIAS_MAP` | `dict[str, str] \| None` | import alias -> FQN for call/annotation resolution | +| `_CURRENT_CALL_SITE_ARG_TAINTS` | `dict[int, dict[int\|str\|None, TaintState]] \| None` | out-channel: resolved arg taints keyed by `id(call_node)` (sink-rule input) | +| `_CURRENT_VAR_TYPES` | `dict[str, list[str]] \| None` | local name -> CANDIDATE SET of class FQNs it may hold (typed-receiver dispatch) | +| `_CURRENT_ATTR_WRITES` | `dict[str, dict[str, TaintState]] \| None` | out-channel: attribute writes recorded during the walk (class-attribute taint) | +| `_CURRENT_LAMBDA_BINDINGS` | `dict[str, list[ast.Lambda]] \| None` | local name -> CANDIDATE SET of lambda bodies it may hold | +| `_CURRENT_MODULE_PREFIX` | `str \| None` | module dotted prefix for FQN minting | +| `_PROVENANCE_CLASH` | `bool` (lives in `core/taints.py`) | switches `combine()` between `least_trusted` (default) and `taint_join` | + +## Who sets each slot (token + `finally: reset` discipline) + +| Setter | ALIAS_MAP | ARG_TAINTS | VAR_TYPES | ATTR_WRITES | LAMBDA_BINDINGS | MODULE_PREFIX | PROVENANCE_CLASH | +|---|---|---|---|---|---|---|---| +| `analyze_function_variables` (:326) | set | set | — | — | — | set | — | +| `compute_variable_taints` (:547) | set (if arg given) | set (if arg given) | set fresh `{}` | — | set fresh `{}` | — | set (if arg given) | +| `attribute_write_recording` CM (:264) | — | — | — | set | — | — | — | +| `_walk_branch_body` (:1559) | — | — | set arm copy | — | set arm copy | — | — | +| `_handle_match` per-case (:1969) | — | — | set arm copy | — | set arm copy | — | — | + +Key consequences: + +- **`compute_variable_taints` is the lifecycle owner** of `VAR_TYPES` and + `LAMBDA_BINDINGS`: both are ALWAYS fresh per function and ALWAYS reset in its + `finally`. Any code that runs AFTER it returns sees `None` for both. +- **`compute_return_taint` / `compute_return_callee` run with `VAR_TYPES` and + `LAMBDA_BINDINGS` unset** (they are called by `analyze_function_variables` + *after* `compute_variable_taints` has reset them). Re-resolving a `return + h.method()` therefore takes the GENERIC raw-receiver path (`_resolve_call` + :1102 receiver-taint check), never the typed-receiver dispatch — the + declared-raw guard is only exercisable through an in-body assignment. The + property tests bind `x = h.method()` in the body for exactly this reason. +- `ATTR_WRITES` is opt-in: it defaults to `None` and is only live inside the + analyzer's `attribute_write_recording` block. Every recorder is a no-op + when it is `None`. +- Branch arms get arm-local COPIES of `VAR_TYPES` / `LAMBDA_BINDINGS` (deep + enough: the candidate lists are copied too), then the arms are UNION-merged + back into the parent dict in place (`_merge_branch_types` / + `_merge_branch_bindings`). `var_taints` itself is NOT a contextvar — it is a + positional parameter copied/merged by the same handlers; keep the three in + lockstep when adding a branching construct. + +## Who reads / mutates each slot + +### `_CURRENT_ALIAS_MAP` +- `_seed_parameters` (:591) — read; feeds annotation FQN resolution. +- `_resolve_call` (:948) — read; feeds `resolve_call_fqn` (context encoders, + serialisation sinks, imported-FQN `taint_map` hits). +- `_update_var_type` (:1278) — read; types `x = Type()` constructor results. +- Never mutated by the walk; treated as read-only input. + +### `_CURRENT_CALL_SITE_ARG_TAINTS` +- `_resolve_call` (:893) — read slot, **mutates dict**: records/merges + `resolved_args` under `id(call_node)`. Also written through the lambda-body + resolution paths (`_resolve_lambda_bodies`, `_resolve_lambda_body_at_call`), + which re-enter `_resolve_expr` with the same slot live. +- Pure out-channel: nothing in this module reads entries back; sink rules + consume it after the walk. A call node resolved more than once (loop + fixpoint iterations, lambda candidates) MERGES via `combine` rather than + overwriting — re-resolution is monotone, never clean-overwriting. + +### `_CURRENT_VAR_TYPES` +- `_seed_parameters` (:590) — read slot, **mutates dict**: annotation types. +- `_resolve_call` (:991) — read-only: typed-receiver method dispatch + (the declared-raw receiver guard lives here, wardline-03c8805449). +- `_update_var_type` (:1273) — read slot, **mutates dict**: strong update on + straight-line assignment; INVALIDATES (pops) on untypeable RHS + (wardline-5ba7ce0f98 stale-type launder fix). +- `_record_attribute_write` (:308) — read-only: receiver class candidates. +- All branching handlers (`_handle_if` :1633, `_handle_for` :1716/:1752, + `_handle_while` :1777/:1800, `_handle_try` :1840, `_handle_match` :1956) — + read parent for arm copies; `_merge_branch_types` **mutates the parent dict + in place** (clear + union of arms). +- Loops snapshot the pre-loop map as a zero-trip arm and union it back after + the fixpoint (wardline-b369c7d06c / wardline-d6af917bde mirror). +- Invariant: a name is ABSENT or maps to a NON-EMPTY list. + +### `_CURRENT_ATTR_WRITES` +- `_record_attribute_write` (:303) — read slot, **mutates dict**: joins the + RHS taint per (receiver-key, attr) via `combine`. Called from + `_handle_assign`, the AnnAssign branch of `_process_stmt`, and + `_handle_augassign` — i.e. recording happens DURING the statement walk at + the write site, against the current per-statement `var_taints` + (wardline-b369c7d06c: a later reassignment cannot launder the recording). +- Receiver keys: class-FQN candidates from `VAR_TYPES`, or + `SELF_ATTRIBUTE_KEY` for `self`/`cls`; projected onto class qualnames later + by `project_attribute_writes` (pure function, no contextvar). + +### `_CURRENT_LAMBDA_BINDINGS` +- `_handle_assign` (:1309) and the AnnAssign branch of `_process_stmt` + (:1169) — read slot, **mutates dict**: linear rebind stores `[lam]` + (replace, never append); non-lambda RHS pops the name. +- `_resolve_call` (:902) — read-only: resolves a direct `cb(...)` against + EVERY candidate body. +- All branching handlers — arm copies + in-place union merge, exactly like + `VAR_TYPES` (wardline-36016d26f3 arm leak; wardline-383f83fafe single-slot + FN; wardline-d6af917bde zero-trip union). +- Invariant: ABSENT or NON-EMPTY; candidate lists deduped by node IDENTITY + (ast nodes are id-hashed — a set would destabilise golden corpora). + +### `_CURRENT_MODULE_PREFIX` +- `_resolve_call` (:953) — read; feeds `resolve_call_fqn`. +- `_resolve_expr_fqn` (:1251) — read; prefixes un-aliased bare names. +- Set only by `analyze_function_variables`; read-only everywhere else. NOTE: + `compute_variable_taints` does NOT set it — a direct call (as the unit tests + do) runs with whatever the ambient context holds (`None` in tests), so + annotation FQNs resolve to bare class names there but to + `pkg.mod.ClassName` under the analyzer. Test `taint_map` keys must match + the bare spelling; analyzer-built maps use the prefixed one. + +### `_PROVENANCE_CLASH` (in `core/taints.py`) +- `combine` — read on EVERY taint combination anywhere in the walk; flips + between `least_trusted` (default, rank-meet) and `taint_join` + (provenance-clash semantics). +- Set only via `compute_variable_taints(provenance_clash=...)`; default + `False` across the live pipeline. + +## Hazards for new handlers (the recurring bug shapes) + +1. **Forgetting the arm copy**: walking a conditionally-executed body against + the parent `LAMBDA_BINDINGS`/`VAR_TYPES` leaks bindings into sibling arms + (wardline-36016d26f3 class of bug). Use `_branch_copy` / + `_types_branch_copy` + `_walk_branch_body`, then merge. +2. **Forgetting the implicit arm**: no-`else` `if`, zero-trip loops, and + no-match `match` all keep the PRE-state alive; an arm set that omits the + pre-state copy drops it (wardline-d6af917bde). +3. **Overwrite instead of merge** on the out-channels: `ARG_TAINTS` entries + must `combine`-merge on re-resolution (loop fixpoints re-visit call nodes). +4. **Post-reset re-resolution**: anything that re-enters `_resolve_expr` after + `compute_variable_taints` returned (return-taint/callee computation, + sink-rule re-resolution) runs WITHOUT `VAR_TYPES`/`LAMBDA_BINDINGS` — do + not rely on typed dispatch or lambda candidates there. +5. **Empty-list candidates**: storing `[]` in `LAMBDA_BINDINGS`/`VAR_TYPES` + makes membership checks pass while candidate loops do nothing — a silent + FN. Writers must store non-empty or pop. + +## Future direction (from the ticket) + +The reviewers' suggestion stands: a single threaded `_WalkContext` dataclass +(alias_map, module_prefix, var_types, lambda_bindings, arg_taints out-channel, +attr_writes out-channel) passed positionally would make the coupling explicit +and let mypy police it. The property harness in `test_property_chokepoint.py` +is the safety net for that refactor: lattice monotonicity, idempotence, seed +monotonicity, and the receiver guard must all survive it unchanged. diff --git a/tests/unit/scanner/taint/test_attribute_write_flow.py b/tests/unit/scanner/taint/test_attribute_write_flow.py new file mode 100644 index 00000000..ac34609f --- /dev/null +++ b/tests/unit/scanner/taint/test_attribute_write_flow.py @@ -0,0 +1,433 @@ +"""Attribute-write flow soundness (wardline-b369c7d06c). + +The per-class attribute summary (``class_attr_taints``) used to be built by a +POST-HOC second walk (``collect_attribute_writes``) that resolved each write's +RHS against the function's FINAL ``var_taints`` and tracked receiver classes in +a branch-unaware, single-slot ``var_types`` map. Two laundering mechanisms +followed — both silent false negatives in the exact summaries the +secure-by-default gate's @trusted-method sink rules rely on: + +1. FINAL-STATE LAUNDER: ``v = read_raw(p); self.x = v; v = "safe"`` resolved + ``v`` post-reassignment, recording ``Store.x`` as INTEGRAL. +2. BRANCH-UNAWARE RECEIVER: ``box = Vault(); if flag: box = Ledger(); + box.token = read_raw(p)`` kept only the last class binding, attributing the + raw write solely to ``Ledger`` while on the no-flag path the receiver is + still the ``Vault`` instance. + +Attribute writes are now recorded DURING the main L2 walk (per-statement +``var_taints``, full branch handling) into a side channel +(:func:`attribute_write_recording`), and receiver class tracking is set-valued +and branch-aware: a straight-line class rebind is a strong update, a branch +join unions the arms, and a write through a multi-candidate receiver attributes +to EVERY candidate class. +""" + +from __future__ import annotations + +import ast +import textwrap +from pathlib import Path + +from wardline.core.config import WardlineConfig +from wardline.core.finding import Kind +from wardline.core.taints import TaintState +from wardline.scanner.analyzer import WardlineAnalyzer +from wardline.scanner.taint.variable_level import ( + SELF_ATTRIBUTE_KEY, + attribute_write_recording, + compute_variable_taints, + project_attribute_writes, +) + +T = TaintState + +_HEADER = ( + "import pickle\n" + "from wardline.decorators import external_boundary, trusted, trust_boundary\n" + "@external_boundary\ndef read_raw(p):\n return p\n" +) + + +def _analyze(tmp_path: Path, body: str) -> WardlineAnalyzer: + p = tmp_path / "m.py" + p.write_text(_HEADER + textwrap.dedent(body), encoding="utf-8") + analyzer = WardlineAnalyzer() + analyzer.analyze([p], WardlineConfig(), root=tmp_path) + return analyzer + + +def _defect_ids(tmp_path: Path, body: str) -> set[str]: + p = tmp_path / "m.py" + p.write_text(_HEADER + textwrap.dedent(body), encoding="utf-8") + analyzer = WardlineAnalyzer() + findings = analyzer.analyze([p], WardlineConfig(), root=tmp_path) + return {f.rule_id for f in findings if f.kind is Kind.DEFECT} + + +def _attr_taints(analyzer: WardlineAnalyzer) -> dict[str, dict[str, TaintState]]: + ctx = analyzer.last_context + assert ctx is not None + return ctx.class_attr_taints + + +def _record( + src: str, + taint_map: dict[str, TaintState] | None = None, + function_taint: TaintState = T.UNKNOWN_RAW, +) -> dict[str, dict[str, TaintState]]: + func = ast.parse(textwrap.dedent(src)).body[0] + assert isinstance(func, ast.FunctionDef | ast.AsyncFunctionDef) + out: dict[str, dict[str, TaintState]] = {} + with attribute_write_recording(out): + compute_variable_taints(func, function_taint, taint_map or {}, alias_map={}) + return out + + +# ── s1/s2 — per-statement RHS taint (final-state launder) ───────────────────── + +_STORE_CONTROL = """ +class Store: + def put(self, p): + v = read_raw(p) + self.x = v + + @trusted(level='ASSURED') + def use(self): + return pickle.loads(self.x) +""" + +_STORE_FINAL_STATE_LAUNDER = """ +class Store: + def put(self, p): + v = read_raw(p) + self.x = v + v = "safe" + + @trusted(level='ASSURED') + def use(self): + return pickle.loads(self.x) +""" + + +def test_s1_control_raw_attr_fires_sink_in_trusted_method(tmp_path: Path) -> None: + analyzer = _analyze(tmp_path, _STORE_CONTROL) + assert _attr_taints(analyzer)["m.Store"]["x"] == T.EXTERNAL_RAW + assert "PY-WL-106" in _defect_ids(tmp_path, _STORE_CONTROL) + + +def test_s2_reassignment_after_attr_write_does_not_launder(tmp_path: Path) -> None: + # ``v`` is RAW at the ``self.x = v`` statement; the trailing ``v = "safe"`` + # must not retroactively clean the recorded attribute write. + analyzer = _analyze(tmp_path, _STORE_FINAL_STATE_LAUNDER) + assert _attr_taints(analyzer)["m.Store"]["x"] == T.EXTERNAL_RAW + assert "PY-WL-106" in _defect_ids(tmp_path, _STORE_FINAL_STATE_LAUNDER) + + +def test_s4_branch_local_reassignment_after_write_stays_raw(tmp_path: Path) -> None: + # Already-sound pin: ``self.x = v`` then ``if flag: v = "safe"`` — the write + # happened while v was raw; the branch merge must not launder it either. + body = """ + class Store: + def put(self, p, flag): + v = read_raw(p) + self.x = v + if flag: + v = "safe" + + @trusted(level='ASSURED') + def use(self): + return pickle.loads(self.x) + """ + analyzer = _analyze(tmp_path, body) + assert _attr_taints(analyzer)["m.Store"]["x"] == T.EXTERNAL_RAW + assert "PY-WL-106" in _defect_ids(tmp_path, body) + + +def test_s3_raw_in_one_branch_arm_stays_raw(tmp_path: Path) -> None: + # Already-sound pin: a raw write in one arm joins (least-trusted) with the + # clean arm's write — the summary keeps the raw rank. + body = """ + class Store: + def put(self, p, flag): + if flag: + self.x = read_raw(p) + else: + self.x = "safe" + + @trusted(level='ASSURED') + def use(self): + return pickle.loads(self.x) + """ + analyzer = _analyze(tmp_path, body) + assert _attr_taints(analyzer)["m.Store"]["x"] == T.EXTERNAL_RAW + assert "PY-WL-106" in _defect_ids(tmp_path, body) + + +def test_augassign_attr_write_uses_per_statement_taint(tmp_path: Path) -> None: + body = """ + class Store: + def put(self, p): + v = read_raw(p) + self.x = "" + self.x += v + v = "safe" + + @trusted(level='ASSURED') + def use(self): + return pickle.loads(self.x) + """ + analyzer = _analyze(tmp_path, body) + assert _attr_taints(analyzer)["m.Store"]["x"] == T.EXTERNAL_RAW + + +def test_annassign_attr_write_uses_per_statement_taint(tmp_path: Path) -> None: + body = """ + class Store: + def put(self, p): + v = read_raw(p) + self.x: str = v + v = "safe" + + @trusted(level='ASSURED') + def use(self): + return pickle.loads(self.x) + """ + analyzer = _analyze(tmp_path, body) + assert _attr_taints(analyzer)["m.Store"]["x"] == T.EXTERNAL_RAW + + +# ── s5/s6/s7 — branch-aware, set-valued receiver class tracking ────────────── + +_VAULT_LEDGER = """ +class Vault: + def __init__(self): + self.token = "init" + + @trusted(level='ASSURED') + def use(self): + return pickle.loads(self.token) + +class Ledger: + def __init__(self): + self.token = "init" +""" + + +def test_s5_control_external_write_attributes_to_receiver_class(tmp_path: Path) -> None: + body = ( + _VAULT_LEDGER + + """ +def route(p, flag): + box = Vault() + box.token = read_raw(p) +""" + ) + analyzer = _analyze(tmp_path, body) + taints = _attr_taints(analyzer) + assert taints["m.Vault"]["token"] == T.EXTERNAL_RAW + assert taints["m.Ledger"]["token"] == T.INTEGRAL + assert "PY-WL-106" in _defect_ids(tmp_path, body) + + +def test_s6_branch_rebound_receiver_attributes_to_all_candidate_classes(tmp_path: Path) -> None: + # On the no-flag arm ``box`` is still the Vault instance, so the raw write + # must attribute to BOTH Vault and Ledger (union of arms), not just the + # last-bound class. + body = ( + _VAULT_LEDGER + + """ +def route(p, flag): + box = Vault() + if flag: + box = Ledger() + box.token = read_raw(p) +""" + ) + analyzer = _analyze(tmp_path, body) + taints = _attr_taints(analyzer) + assert taints["m.Vault"]["token"] == T.EXTERNAL_RAW + assert taints["m.Ledger"]["token"] == T.EXTERNAL_RAW + assert "PY-WL-106" in _defect_ids(tmp_path, body) + + +def test_s7_receiver_rebound_to_nonclass_in_one_arm_keeps_class_candidate(tmp_path: Path) -> None: + # Already-sound pin: one arm rebinds ``box`` to an untypeable raw value; the + # fall-through arm still holds the Vault instance, so Vault.token records raw. + body = ( + _VAULT_LEDGER + + """ +def route(p, flag): + box = Vault() + if flag: + box = read_raw(p) + box.token = read_raw(p) +""" + ) + analyzer = _analyze(tmp_path, body) + assert _attr_taints(analyzer)["m.Vault"]["token"] == T.EXTERNAL_RAW + assert "PY-WL-106" in _defect_ids(tmp_path, body) + + +def test_straight_line_class_rebind_is_a_strong_update(tmp_path: Path) -> None: + # Straight-line rebind: after ``box = Ledger()`` the receiver is a Ledger on + # EVERY path, so the raw write attributes only to Ledger — Vault stays clean. + body = ( + _VAULT_LEDGER + + """ +def route(p): + box = Vault() + box = Ledger() + box.token = read_raw(p) +""" + ) + analyzer = _analyze(tmp_path, body) + taints = _attr_taints(analyzer) + assert taints["m.Vault"]["token"] == T.INTEGRAL + assert taints["m.Ledger"]["token"] == T.EXTERNAL_RAW + assert "PY-WL-106" not in _defect_ids(tmp_path, body) + + +def test_try_except_rebound_receiver_attributes_to_both_arms(tmp_path: Path) -> None: + body = ( + _VAULT_LEDGER + + """ +def route(p): + box = Vault() + try: + box = Ledger() + except ValueError: + pass + box.token = read_raw(p) +""" + ) + analyzer = _analyze(tmp_path, body) + taints = _attr_taints(analyzer) + assert taints["m.Vault"]["token"] == T.EXTERNAL_RAW + assert taints["m.Ledger"]["token"] == T.EXTERNAL_RAW + + +def test_match_rebound_receiver_attributes_to_both_arms(tmp_path: Path) -> None: + body = ( + _VAULT_LEDGER + + """ +def route(p, flag): + box = Vault() + match flag: + case 1: + box = Ledger() + box.token = read_raw(p) +""" + ) + analyzer = _analyze(tmp_path, body) + taints = _attr_taints(analyzer) + assert taints["m.Vault"]["token"] == T.EXTERNAL_RAW + assert taints["m.Ledger"]["token"] == T.EXTERNAL_RAW + + +def test_zero_trip_loop_receiver_rebind_keeps_pre_loop_candidate(tmp_path: Path) -> None: + # A loop body is a conditionally-executed arm: on the zero-trip path the + # receiver is still the pre-loop Vault instance. + body = ( + _VAULT_LEDGER + + """ +def route(p, items): + box = Vault() + for _ in items: + box = Ledger() + box.token = read_raw(p) +""" + ) + analyzer = _analyze(tmp_path, body) + taints = _attr_taints(analyzer) + assert taints["m.Vault"]["token"] == T.EXTERNAL_RAW + assert taints["m.Ledger"]["token"] == T.EXTERNAL_RAW + + +# ── recording side channel (unit level) ────────────────────────────────────── + + +def test_recording_self_write_uses_per_statement_var_taints() -> None: + out = _record( + """ + def put(self, p): + v = read_raw(p) + self.x = v + v = "safe" + """, + taint_map={"read_raw": T.EXTERNAL_RAW}, + function_taint=T.INTEGRAL, + ) + assert out[SELF_ATTRIBUTE_KEY]["x"] == T.EXTERNAL_RAW + + +def test_recording_cls_write_maps_to_self_key() -> None: + out = _record( + """ + def put(cls, p): + cls.x = read_raw(p) + """, + taint_map={"read_raw": T.EXTERNAL_RAW}, + function_taint=T.INTEGRAL, + ) + assert out[SELF_ATTRIBUTE_KEY]["x"] == T.EXTERNAL_RAW + + +def test_recording_joins_multiple_writes_least_trusted() -> None: + out = _record( + """ + def put(self, p): + self.x = "clean" + self.x = read_raw(p) + """, + taint_map={"read_raw": T.EXTERNAL_RAW}, + function_taint=T.INTEGRAL, + ) + assert out[SELF_ATTRIBUTE_KEY]["x"] == T.EXTERNAL_RAW + + +def test_recording_typed_receiver_records_under_candidate_fqns() -> None: + out = _record( + """ + def route(p, flag): + box = Vault() + if flag: + box = Ledger() + box.token = read_raw(p) + """, + taint_map={"read_raw": T.EXTERNAL_RAW}, + function_taint=T.INTEGRAL, + ) + assert out["Vault"]["token"] == T.EXTERNAL_RAW + assert out["Ledger"]["token"] == T.EXTERNAL_RAW + + +def test_recording_disabled_by_default() -> None: + func = ast.parse("def put(self, p):\n self.x = p\n").body[0] + assert isinstance(func, ast.FunctionDef) + # No recording context — must not raise, and the walk stays pure. + compute_variable_taints(func, T.UNKNOWN_RAW, {}) + + +def test_project_attribute_writes_filters_and_maps_self() -> None: + recorded = { + SELF_ATTRIBUTE_KEY: {"x": T.EXTERNAL_RAW}, + "m.Vault": {"token": T.EXTERNAL_RAW}, + "m.not_a_class": {"y": T.EXTERNAL_RAW}, + } + projected = project_attribute_writes(recorded, frozenset({"m.Vault", "m.Store"}), "m.Store") + assert projected == { + "m.Store": {"x": T.EXTERNAL_RAW}, + "m.Vault": {"token": T.EXTERNAL_RAW}, + } + # Outside a method the self/cls writes have no class to attribute to. + projected_fn = project_attribute_writes(recorded, frozenset({"m.Vault", "m.Store"}), None) + assert projected_fn == {"m.Vault": {"token": T.EXTERNAL_RAW}} + + +def test_project_attribute_writes_joins_self_and_typed_keys_for_same_class() -> None: + recorded = { + SELF_ATTRIBUTE_KEY: {"x": T.INTEGRAL}, + "m.Store": {"x": T.EXTERNAL_RAW}, + } + projected = project_attribute_writes(recorded, frozenset({"m.Store"}), "m.Store") + assert projected == {"m.Store": {"x": T.EXTERNAL_RAW}} diff --git a/tests/unit/scanner/taint/test_decorator_provider.py b/tests/unit/scanner/taint/test_decorator_provider.py index 64f1c720..dcb85a90 100644 --- a/tests/unit/scanner/taint/test_decorator_provider.py +++ b/tests/unit/scanner/taint/test_decorator_provider.py @@ -241,7 +241,11 @@ def test_star_imported_trust_boundary_fires_end_to_end(tmp_path) -> None: ) result = run_scan(pkg) active = {f.rule_id for f in result.findings if f.suppressed.value == "active"} - assert "PY-WL-102" in active, "star-imported @trust_boundary was not seeded" + # The boundary-integrity family partitions (wardline-718048a518): the bare + # ``return p`` shape is PY-WL-119's domain, every other no-rejection shape is + # PY-WL-102's. Either member firing proves the star-imported decorator was + # seeded — the FN this test guards is the EMPTY intersection. + assert {"PY-WL-102", "PY-WL-119"} & active, "star-imported @trust_boundary was not seeded" # ── Coverage: fail-closed arms reached only by unusual / malformed decorator diff --git a/tests/unit/scanner/taint/test_engine_precision.py b/tests/unit/scanner/taint/test_engine_precision.py new file mode 100644 index 00000000..6026718d --- /dev/null +++ b/tests/unit/scanner/taint/test_engine_precision.py @@ -0,0 +1,610 @@ +# tests/unit/scanner/taint/test_engine_precision.py +"""Engine-precision regression suite (2026-06-10 eval batch). + +Covers the confirmed taint-engine FNs plus the engine-precision expansion +ticket (wardline-93d608c997) and the typed-dispatch launder +(wardline-03c8805449): + +1. Container-conversion builtins (list/tuple/set/dict/sorted/frozenset/ + reversed) must propagate the worst argument taint, not launder to the + caller seed. +2. ``str.format_map`` is a sibling of ``str.format`` — receiver+args combine. +3. Non-literal nested-tuple unpack ``a, (b, c) = raw`` must taint b and c. +4. A try handler can observe ANY prefix of the try body (an exception may be + raised mid-body), so its seed is the worst of the pre-try snapshot and the + try-body states — not the pre-try snapshot alone. +5. A TYPED parameter seeded declared-raw (EXTERNAL_RAW/MIXED_RAW) must not be + laundered by its class method's clean @trusted summary. +6. Expansion: ``**raw`` spread into a lambda's ``**kw`` param; Subscript + unpack/for targets contaminate their base container; unresolved bare-name + calls propagate the worst of (caller seed, arg taints). +""" + +from __future__ import annotations + +import ast +import textwrap +from pathlib import Path + +import pytest + +from wardline.core.config import WardlineConfig +from wardline.core.finding import Kind +from wardline.core.taints import TaintState +from wardline.scanner.analyzer import WardlineAnalyzer +from wardline.scanner.taint.variable_level import ( + SELF_ATTRIBUTE_KEY, + attribute_write_recording, + compute_variable_taints, +) + +T = TaintState + +_RAW_TM = {"read_raw": T.UNKNOWN_RAW} + +_HEADER = ( + "import os, subprocess\n" + "from wardline.decorators import external_boundary, trusted\n" + "@external_boundary\n" + "def read_raw(p):\n" + " return p\n" +) + + +def _vt( + src: str, + function_taint: TaintState = T.UNKNOWN_RAW, + taint_map: dict[str, TaintState] | None = None, + alias_map: dict[str, str] | None = None, + param_meets: dict[str, TaintState] | None = None, +) -> dict[str, TaintState]: + func = ast.parse(src).body[0] + assert isinstance(func, ast.FunctionDef | ast.AsyncFunctionDef) + return compute_variable_taints(func, function_taint, taint_map or {}, alias_map=alias_map, param_meets=param_meets) + + +def _defects(tmp_path: Path, src: str) -> list[tuple[str, int | None]]: + p = tmp_path / "m.py" + p.write_text(_HEADER + textwrap.dedent(src), encoding="utf-8") + findings = WardlineAnalyzer().analyze([p], WardlineConfig(), root=tmp_path) + return [(f.rule_id, f.location.line_start) for f in findings if f.kind is Kind.DEFECT] + + +# ── (1) container-conversion builtins propagate argument taint ────────────── + + +@pytest.mark.parametrize("builtin", ["list", "tuple", "set", "frozenset", "dict", "sorted", "reversed"]) +def test_container_builtin_propagates_arg_taint(builtin: str) -> None: + out = _vt( + f"def f(p):\n x = {builtin}(read_raw(p))\n", + function_taint=T.INTEGRAL, + taint_map=_RAW_TM, + ) + assert out["x"] == T.UNKNOWN_RAW + + +@pytest.mark.parametrize("builtin", ["list", "tuple", "set", "frozenset", "dict", "sorted"]) +def test_container_builtin_no_args_is_integral(builtin: str) -> None: + out = _vt(f"def f():\n x = {builtin}()\n", function_taint=T.UNKNOWN_RAW) + assert out["x"] == T.INTEGRAL + + +def test_sorted_raw_argv_fires_subprocess_sink(tmp_path: Path) -> None: + rules = _defects( + tmp_path, + """ + @trusted(level='ASSURED') + def f(p): + raw = read_raw(p) + args = sorted(raw) + subprocess.run(args, shell=True) + """, + ) + assert ("PY-WL-112", 11) in rules + + +# ── (2) str.format_map combines receiver and mapping argument ─────────────── + + +def test_format_map_method_combines_receiver_and_args() -> None: + out = _vt( + "def f(p):\n x = 'echo {x}'.format_map(read_raw(p))\n", + function_taint=T.INTEGRAL, + taint_map=_RAW_TM, + ) + assert out["x"] == T.UNKNOWN_RAW + + +def test_format_map_validated_arg_stays_clean() -> None: + # FP guard: validated mapping in an ASSURED producer stays clean + # (least_trusted(INTEGRAL receiver, ASSURED) = ASSURED, no MIXED_RAW clash). + out = _vt( + "def f(p):\n x = '{x}'.format_map(validate(p))\n", + function_taint=T.ASSURED, + taint_map={"validate": T.ASSURED}, + ) + assert out["x"] == T.ASSURED + + +def test_format_map_raw_mapping_fires_command_sink(tmp_path: Path) -> None: + rules = _defects( + tmp_path, + """ + @trusted(level='ASSURED') + def f(p): + raw = read_raw(p) + cmd = "echo {x}".format_map(raw) + os.system(cmd) + """, + ) + assert ("PY-WL-108", 11) in rules + + +# ── (3) non-literal nested-tuple unpack keeps RHS taint on every leaf ─────── + + +def test_nested_tuple_unpack_of_call_rhs_taints_leaves() -> None: + out = _vt( + "def f(p):\n a, (b, c) = read_raw(p)\n", + function_taint=T.INTEGRAL, + taint_map=_RAW_TM, + ) + assert out["a"] == T.UNKNOWN_RAW + assert out["b"] == T.UNKNOWN_RAW + assert out["c"] == T.UNKNOWN_RAW + + +def test_nested_tuple_unpack_leaf_reaches_exec_sink(tmp_path: Path) -> None: + rules = _defects( + tmp_path, + """ + @trusted(level='ASSURED') + def f(p): + a, (b, c) = read_raw(p) + eval(b) + return 1 + """, + ) + assert ("PY-WL-107", 10) in rules + + +# ── (4) try handler sees the worst of any try-body prefix ─────────────────── + + +def test_handler_sees_try_body_assignment_post_state() -> None: + # x is reassigned raw in the try body BEFORE a possibly-raising call; the + # handler may observe it. Seeding the handler from dict(pre_try) alone + # dropped the raw value (fail-open under-taint). + out = _vt( + "def f(p):\n" + " x = 'safe'\n" + " try:\n" + " x = read_raw(p)\n" + " risky()\n" + " except ValueError:\n" + " y = x\n", + function_taint=T.INTEGRAL, + taint_map=_RAW_TM, + ) + assert out["y"] == T.UNKNOWN_RAW + + +def test_handler_sink_on_try_assigned_raw_fires(tmp_path: Path) -> None: + rules = _defects( + tmp_path, + """ + @trusted(level='ASSURED') + def f(p): + x = "safe" + try: + x = read_raw(p) + risky() + except ValueError: + eval(x) + return 1 + """, + ) + assert any(r == "PY-WL-107" for r, _line in rules) + + +def test_handler_sink_sees_mid_try_prefix_worst(tmp_path: Path) -> None: + # x is raw after the FIRST try statement and cleaned by the second; an + # exception between them still hands the handler the raw value, so the + # handler seed is the worst over every try-body prefix, not the post-state. + rules = _defects( + tmp_path, + """ + @trusted(level='ASSURED') + def f(p): + x = "safe" + try: + x = read_raw(p) + x = "clean" + risky() + except ValueError: + eval(x) + return 1 + """, + ) + assert any(r == "PY-WL-107" for r, _line in rules) + + +def test_post_try_clean_reassign_in_both_arms_stays_clean() -> None: + # FP guard: when the try body ends clean AND the handler reassigns clean, + # the post-merge state stays clean (handler seeding must not leak into the + # merge for a var both arms finished clean). + out = _vt( + "def f(p):\n try:\n x = a()\n except Exception:\n x = b()\n", + function_taint=T.INTEGRAL, + taint_map={"a": T.ASSURED, "b": T.GUARDED}, + ) + assert out["x"] == T.GUARDED + + +# ── (5) declared-raw typed receiver is not laundered by a clean summary ───── + + +def test_declared_raw_typed_param_not_laundered_by_clean_summary() -> None: + out = _vt( + "def f(h: mymod.Schema):\n x = h.validate()\n", + function_taint=T.ASSURED, + taint_map={"mymod.Schema.validate": T.ASSURED}, + alias_map={"mymod": "mymod"}, + param_meets={"h": T.EXTERNAL_RAW}, + ) + assert out["x"] == T.EXTERNAL_RAW + + +def test_unknown_raw_typed_receiver_still_resolves_clean_summary() -> None: + # FP guard (wardline-f6a29ce23a): an unmodeled ``Type()`` constructor seeds + # the receiver UNKNOWN_RAW; the typed dispatch must STILL resolve the clean + # summary — only DECLARED-raw (EXTERNAL_RAW/MIXED_RAW) receivers are gated. + out = _vt( + "def f(obj: mymod.Schema):\n x = obj.validate()\n", + function_taint=T.UNKNOWN_RAW, + taint_map={"mymod.Schema.validate": T.ASSURED}, + alias_map={"mymod": "mymod"}, + ) + assert out["x"] == T.ASSURED + + +def test_typed_param_seeded_raw_interprocedurally_fires_sink(tmp_path: Path) -> None: + # Mirror of the wardline-03c8805449 var_typed repro: only the annotation + # differs from the untyped control, which already fires. + src = """ + class Helper: + @trusted(level='ASSURED') + def get_cmd(self): + return "noop()" + + @trusted(level='ASSURED') + def f(h{ann}): + eval(h.get_cmd()) + + @trusted(level='ASSURED') + def caller(p): + f(read_raw(p)) + """ + untyped = _defects(tmp_path, src.format(ann="")) + assert any(r == "PY-WL-107" for r, _line in untyped) # control + typed = _defects(tmp_path, src.format(ann=": Helper")) + assert any(r == "PY-WL-107" for r, _line in typed) + + +# ── (6a) **spread taint reaches a lambda's **kw param at the call site ─────── + + +def test_double_star_spread_seeds_lambda_kwarg_param(tmp_path: Path) -> None: + rules = _defects( + tmp_path, + """ + @trusted(level='ASSURED') + def process(p): + raw_kwargs = read_raw(p) + (lambda **kw: os.system(kw.get('cmd', '')))(**raw_kwargs) + """, + ) + assert any(r == "PY-WL-108" for r, _line in rules) + + +def test_double_star_spread_seeds_lambda_named_param(tmp_path: Path) -> None: + # A **spread can also bind a NAMED lambda parameter by key. + rules = _defects( + tmp_path, + """ + @trusted(level='ASSURED') + def process(p): + raw_kwargs = read_raw(p) + (lambda cmd='': os.system(cmd))(**raw_kwargs) + """, + ) + assert any(r == "PY-WL-108" for r, _line in rules) + + +# ── (6b) Subscript unpack / for targets contaminate the base container ────── + + +def test_unpack_subscript_target_contaminates_base_literal_rhs() -> None: + out = _vt( + "def f(p):\n d = {}\n d['cmd'], y = read_raw(p), 1\n", + function_taint=T.INTEGRAL, + taint_map=_RAW_TM, + ) + assert out["d"] == T.UNKNOWN_RAW + + +def test_unpack_subscript_target_contaminates_base_call_rhs() -> None: + out = _vt( + "def f(p):\n d = {}\n d['cmd'], y = read_raw(p)\n", + function_taint=T.INTEGRAL, + taint_map=_RAW_TM, + ) + assert out["d"] == T.UNKNOWN_RAW + assert out["y"] == T.UNKNOWN_RAW + + +def test_for_subscript_target_contaminates_base() -> None: + out = _vt( + "def f(p):\n d = {}\n for d['k'] in [read_raw(p)]:\n pass\n", + function_taint=T.INTEGRAL, + taint_map=_RAW_TM, + ) + assert out["d"] == T.UNKNOWN_RAW + + +def test_for_subscript_target_reaches_command_sink(tmp_path: Path) -> None: + rules = _defects( + tmp_path, + """ + @trusted(level='ASSURED') + def process(p): + d = {} + raw = read_raw(p) + for d['cmd'] in [raw]: + pass + os.system(d['cmd']) + """, + ) + assert any(r == "PY-WL-108" for r, _line in rules) + + +# ── (6c) unresolved bare-name calls propagate worst(seed, args) ────────────── + + +def test_unresolved_bare_call_propagates_worst_arg_taint() -> None: + # ``transform`` is absent from the taint_map (a bare parameter / runtime + # callable): an unknown callee cannot be assumed to clean its raw argument, + # matching the imported-unmodeled path's conservatism. + out = _vt( + "def f(p, transform):\n x = transform(read_raw(p))\n", + function_taint=T.GUARDED, + taint_map=_RAW_TM, + ) + assert out["x"] == T.UNKNOWN_RAW + + +def test_unresolved_bare_call_clean_args_keeps_function_taint() -> None: + # FP guard: clean arguments do not demote — the result stays at the worst + # of the caller seed and the args (here the seed). + out = _vt( + "def f(p, transform):\n x = transform('const')\n", + function_taint=T.GUARDED, + taint_map=_RAW_TM, + ) + assert out["x"] == T.GUARDED + + +def test_measuring_builtins_still_fall_back_to_function_taint() -> None: + # len/int validate/measure, they do not carry the data through — the + # curated non-propagating carve-out keeps them at the caller seed. + assert _vt("def f(p):\n x = len(read_raw(p))\n", function_taint=T.GUARDED, taint_map=_RAW_TM)["x"] == T.GUARDED + assert _vt("def f(p):\n x = int(read_raw(p))\n", function_taint=T.GUARDED, taint_map=_RAW_TM)["x"] == T.GUARDED + + +def test_bare_param_transform_call_fires_command_sink(tmp_path: Path) -> None: + rules = _defects( + tmp_path, + """ + @trusted(level='ASSURED') + def f(p, transform): + raw = read_raw(p) + os.system(transform(raw)) + """, + ) + assert any(r == "PY-WL-108" for r, _line in rules) + + +# ── (7) io.StringIO/io.BytesIO constructors carry the WORST ARG taint ──────── + + +@pytest.mark.parametrize("ctor", ["io.StringIO", "io.BytesIO"]) +def test_io_buffer_ctor_const_arg_is_integral(ctor: str) -> None: + # An in-memory buffer over a constant is NOT external data — the ctor + # result carries the worst argument taint, not the unresolved-import + # UNKNOWN_RAW default. + out = _vt( + f"def f():\n x = {ctor}('const')\n", + function_taint=T.INTEGRAL, + taint_map=_RAW_TM, + alias_map={"io": "io"}, + ) + assert out["x"] == T.INTEGRAL + + +@pytest.mark.parametrize("ctor", ["io.StringIO", "io.BytesIO"]) +def test_io_buffer_ctor_raw_arg_stays_raw(ctor: str) -> None: + out = _vt( + f"def f(p):\n x = {ctor}(read_raw(p))\n", + function_taint=T.INTEGRAL, + taint_map=_RAW_TM, + alias_map={"io": "io"}, + ) + assert out["x"] == T.UNKNOWN_RAW + + +def test_io_buffer_ctor_no_args_is_integral() -> None: + out = _vt( + "def f():\n x = io.StringIO()\n", + function_taint=T.INTEGRAL, + taint_map=_RAW_TM, + alias_map={"io": "io"}, + ) + assert out["x"] == T.INTEGRAL + + +def test_io_buffer_from_import_alias_resolves() -> None: + # ``from io import StringIO`` resolves through the alias map to the same + # canonical FQN — the bare-name form must not regress to UNKNOWN_RAW. + out = _vt( + "def f():\n x = StringIO('const')\n", + function_taint=T.INTEGRAL, + taint_map=_RAW_TM, + alias_map={"StringIO": "io.StringIO"}, + ) + assert out["x"] == T.INTEGRAL + + +def test_io_buffer_shadowed_by_raw_local_not_laundered() -> None: + # A raw local shadowing the ``io`` module name must not inherit the + # clean worst-arg ctor model (the wardline-f6a29ce23a shadow guard). + out = _vt( + "def f(p):\n io = read_raw(p)\n x = io.StringIO('const')\n", + function_taint=T.INTEGRAL, + taint_map=_RAW_TM, + alias_map={"io": "io"}, + ) + assert out["x"] == T.UNKNOWN_RAW + + +def test_stringio_const_read_no_longer_fires_return_rule(tmp_path: Path) -> None: + # End-to-end: returning the .read() of a CLEAN in-memory buffer is not a + # raw return — the PY-WL-101 FP this residual tracked. + rules = _defects( + tmp_path, + """ + import io + @trusted(level='ASSURED') + def render(): + buf = io.StringIO("constant template") + return buf.read() + """, + ) + assert not any(r == "PY-WL-101" for r, _line in rules) + + +def test_stringio_raw_read_still_fires_return_rule(tmp_path: Path) -> None: + # Soundness control: raw content poured into the buffer still propagates + # through .read() and fires the raw-return rule. + rules = _defects( + tmp_path, + """ + import io + @trusted(level='ASSURED') + def render(p): + buf = io.StringIO(read_raw(p)) + return buf.read() + """, + ) + assert any(r == "PY-WL-101" for r, _line in rules) + + +# ── (8) tuple-unpack attribute targets record into the attr-write channel ──── + + +def test_tuple_unpack_attribute_target_records_self_write() -> None: + func = ast.parse("def put(self, p):\n self.x, y = read_raw(p), 1\n").body[0] + assert isinstance(func, ast.FunctionDef) + out: dict[str, dict[str, TaintState]] = {} + with attribute_write_recording(out): + compute_variable_taints(func, T.INTEGRAL, dict(_RAW_TM), alias_map={}) + assert out[SELF_ATTRIBUTE_KEY]["x"] == T.UNKNOWN_RAW + + +def test_tuple_unpack_attribute_pair_from_call_records_both() -> None: + # ``(self.x, self.y) = pair`` with a non-literal RHS: both elements get + # the RHS taint and BOTH record into the channel. + func = ast.parse("def put(self, p):\n (self.x, self.y) = read_raw(p)\n").body[0] + assert isinstance(func, ast.FunctionDef) + out: dict[str, dict[str, TaintState]] = {} + with attribute_write_recording(out): + compute_variable_taints(func, T.INTEGRAL, dict(_RAW_TM), alias_map={}) + assert out[SELF_ATTRIBUTE_KEY]["x"] == T.UNKNOWN_RAW + assert out[SELF_ATTRIBUTE_KEY]["y"] == T.UNKNOWN_RAW + + +def test_tuple_unpack_attribute_write_fires_cross_method_sink(tmp_path: Path) -> None: + # End-to-end: the unpack-written attribute feeds the cross-method summary + # exactly like the plain ``self.x = raw`` path (which already fires). + rules = _defects( + tmp_path, + """ + class Store: + def put(self, p): + self.x, y = read_raw(p), 1 + @trusted(level='ASSURED') + def use(self): + os.system(self.x) + """, + ) + assert any(r == "PY-WL-108" for r, _line in rules) + + +# ── (9) lambda DEFAULT taint seeds the param when omitted at the call site ─── + + +def test_lambda_raw_default_omitted_at_call_fires_sink(tmp_path: Path) -> None: + rules = _defects( + tmp_path, + """ + @trusted(level='ASSURED') + def process(p): + raw = read_raw(p) + cb = lambda x=raw: os.system(x) + cb() + """, + ) + assert any(r == "PY-WL-108" for r, _line in rules) + + +def test_lambda_raw_kwonly_default_omitted_at_call_fires_sink(tmp_path: Path) -> None: + rules = _defects( + tmp_path, + """ + @trusted(level='ASSURED') + def process(p): + raw = read_raw(p) + cb = lambda *, x=raw: os.system(x) + cb() + """, + ) + assert any(r == "PY-WL-108" for r, _line in rules) + + +def test_lambda_raw_default_overridden_clean_does_not_fire(tmp_path: Path) -> None: + # FP guard: a clean argument supplied at the call site replaces the raw + # default — the default expression never evaluates on this path. + rules = _defects( + tmp_path, + """ + @trusted(level='ASSURED') + def process(p): + raw = read_raw(p) + cb = lambda x=raw: os.system(x) + cb("echo ok") + """, + ) + assert not any(r == "PY-WL-108" for r, _line in rules) + + +def test_lambda_raw_default_overridden_by_keyword_does_not_fire(tmp_path: Path) -> None: + rules = _defects( + tmp_path, + """ + @trusted(level='ASSURED') + def process(p): + raw = read_raw(p) + cb = lambda x=raw: os.system(x) + cb(x="echo ok") + """, + ) + assert not any(r == "PY-WL-108" for r, _line in rules) diff --git a/tests/unit/scanner/taint/test_module_global_taint.py b/tests/unit/scanner/taint/test_module_global_taint.py new file mode 100644 index 00000000..b6cb0ac3 --- /dev/null +++ b/tests/unit/scanner/taint/test_module_global_taint.py @@ -0,0 +1,170 @@ +"""Module-global taint channel (wardline-66b2c91470). + +Read direction: a module-level variable assigned from a raw source at import +time (``RAW = read_raw(...)`` where ``read_raw`` is an ``@external_boundary`` +function, or a call matching ``config.untrusted_sources``) carries its taint +into every function that reads it — the L2 walk seeds the global like an +implicit parameter, so a local reassignment shadows it flow-sensitively. + +Write direction: a function assigning raw to a declared ``global g`` marks the +module global, and OTHER functions reading ``g`` inherit (one merge hop — +see the analyzer's documented approximation). +""" + +from __future__ import annotations + +import textwrap +from typing import TYPE_CHECKING + +from wardline.core.config import WardlineConfig +from wardline.core.taints import RAW_ZONE +from wardline.scanner.analyzer import WardlineAnalyzer + +if TYPE_CHECKING: + from pathlib import Path + +_HEADER = ( + "import os\n" + "from wardline.decorators import external_boundary, trusted\n" + "@external_boundary\n" + "def read_raw(p):\n" + " return p\n" +) + + +def _scan(tmp_path: Path, src: str, config: WardlineConfig | None = None, header: str = _HEADER): + p = tmp_path / "m.py" + p.write_text(header + textwrap.dedent(src), encoding="utf-8") + analyzer = WardlineAnalyzer() + findings = analyzer.analyze([p], config or WardlineConfig(), root=tmp_path) + assert analyzer.last_context is not None + return findings, analyzer.last_context + + +def _hits(findings, rule_id: str) -> list[tuple[str, str | None]]: + return [(f.rule_id, f.qualname) for f in findings if f.rule_id == rule_id] + + +def test_module_level_raw_assignment_taints_reader(tmp_path) -> None: + findings, _ = _scan( + tmp_path, + """ + RAW = read_raw('seed') + + @trusted(level='ASSURED') + def f(): + os.system(RAW) + """, + ) + assert _hits(findings, "PY-WL-108") == [("PY-WL-108", "m.f")] + + +def test_module_level_raw_via_config_untrusted_sources(tmp_path) -> None: + findings, _ = _scan( + tmp_path, + """ + RAW = fetchlib.fetch() + + @trusted(level='ASSURED') + def f(): + os.system(RAW) + """, + config=WardlineConfig(untrusted_sources=("fetchlib.fetch",)), + header=("import os\nimport fetchlib\nfrom wardline.decorators import external_boundary, trusted\n"), + ) + assert _hits(findings, "PY-WL-108") == [("PY-WL-108", "m.f")] + + +def test_local_reassignment_shadows_module_global(tmp_path) -> None: + # Flow-sensitive shadowing: the function overwrites the global name with a + # literal BEFORE the sink — no finding. + findings, _ = _scan( + tmp_path, + """ + RAW = read_raw('seed') + + @trusted(level='ASSURED') + def f(): + RAW = 'ls -la' + os.system(RAW) + """, + ) + assert _hits(findings, "PY-WL-108") == [] + + +def test_clean_module_global_stays_clean(tmp_path) -> None: + findings, _ = _scan( + tmp_path, + """ + CMD = 'ls -la' + + @trusted(level='ASSURED') + def f(): + os.system(CMD) + """, + ) + assert _hits(findings, "PY-WL-108") == [] + + +def test_module_level_rebind_to_safe_clears_seed(tmp_path) -> None: + # Last-binding-wins at module scope (same discipline as collect_sink_bindings). + findings, _ = _scan( + tmp_path, + """ + RAW = read_raw('seed') + RAW = 'ls -la' + + @trusted(level='ASSURED') + def f(): + os.system(RAW) + """, + ) + assert _hits(findings, "PY-WL-108") == [] + + +def test_global_write_propagates_to_other_readers(tmp_path) -> None: + findings, _ = _scan( + tmp_path, + """ + G = 'init' + + def poison(p): + global G + G = read_raw(p) + + @trusted(level='ASSURED') + def use(): + os.system(G) + """, + ) + assert _hits(findings, "PY-WL-108") == [("PY-WL-108", "m.use")] + + +def test_global_clean_write_does_not_poison(tmp_path) -> None: + findings, _ = _scan( + tmp_path, + """ + def set_default(): + global G + G = 'ls -la' + + @trusted(level='ASSURED') + def use(): + os.system(G) + """, + ) + assert _hits(findings, "PY-WL-108") == [] + + +def test_raw_module_global_propagates_to_return_taint(tmp_path) -> None: + _, ctx = _scan( + tmp_path, + """ + RAW = read_raw('seed') + + @trusted(level='ASSURED') + def f(): + return RAW + """, + ) + assert ctx.function_return_taints["m.f"] in RAW_ZONE diff --git a/tests/unit/scanner/taint/test_propagation.py b/tests/unit/scanner/taint/test_propagation.py index 0da6ae19..ac3681b5 100644 --- a/tests/unit/scanner/taint/test_propagation.py +++ b/tests/unit/scanner/taint/test_propagation.py @@ -92,8 +92,15 @@ def test_two_anchored_callees_aggregate_via_least_trusted() -> None: edges = {"A": {"B", "C"}, "B": set(), "C": set()} tm = {"A": T.UNKNOWN_RAW, "B": T.GUARDED, "C": T.EXTERNAL_RAW} src = {"A": "fallback", "B": "anchored", "C": "anchored"} - refined, _prov, _diags, _it = _run(edges, tm, src) - assert refined["A"] != T.MIXED_RAW + refined, _prov, diags, _it = _run(edges, tm, src) + # Exact value (wardline-e159060db7): the fallback floor must HOLD A at + # UNKNOWN_RAW — `!= MIXED_RAW` also passed under a floor-drop mutation + # (combined = EXTERNAL_RAW satisfies it). And the floor must hold at the + # combination site itself, not via the monotonicity commit backstop: under + # the floor-drop mutation the value survives only because the guard rejects + # the move and emits L3_MONOTONICITY_VIOLATION — so no diagnostics allowed. + assert refined["A"] == T.UNKNOWN_RAW + assert diags == [] def test_clean_different_family_callees_stay_clean() -> None: @@ -165,8 +172,30 @@ def test_fallback_caller_clean_callees_floor_holds_not_mixed() -> None: edges = {"A": {"B", "C"}, "B": set(), "C": set()} tm = {"A": T.UNKNOWN_RAW, "B": T.ASSURED, "C": T.INTEGRAL} src = {"A": "fallback", "B": "anchored", "C": "anchored"} - refined, _prov, _diags, _it = _run(edges, tm, src) + refined, _prov, diags, _it = _run(edges, tm, src) assert refined["A"] == T.UNKNOWN_RAW # floor, NOT MIXED_RAW + # The floor must hold at the combination site, not via the monotonicity + # commit guard (which would emit L3_MONOTONICITY_VIOLATION while preserving + # the value) — see test_two_anchored_callees_aggregate_via_least_trusted. + assert diags == [] + + +def test_scc_round_floor_holds_inside_cycle_without_phase1_seed() -> None: + # Exercises the `_compute_scc_round` floor in ISOLATION from Phase-1 + # initialization (wardline-e159060db7): A and B form a cycle, so B is inside + # A's SCC and Phase 1's external-callee pass never touches A — the Phase-2 + # floor (current[A] = UNKNOWN_RAW) is the ONLY thing pinning A against its + # anchored-INTEGRAL cycle partner. Mutation-probed: with the floor dropped + # (`new_taint = combined`) the round proposes INTEGRAL for A and only the + # monotonicity commit guard saves the VALUE while emitting + # L3_MONOTONICITY_VIOLATION — hence both assertions are load-bearing. + edges = {"A": {"B"}, "B": {"A"}} + tm = {"A": T.UNKNOWN_RAW, "B": T.INTEGRAL} + src = {"A": "fallback", "B": "anchored"} + refined, _prov, diags, _it = _run(edges, tm, src) + assert refined["A"] == T.UNKNOWN_RAW # floor holds — fallback can't prove trust + assert refined["B"] == T.INTEGRAL # anchored partner unaffected + assert diags == [] def test_long_chain_converges_without_bound_diagnostic() -> None: diff --git a/tests/unit/scanner/taint/test_property_chokepoint.py b/tests/unit/scanner/taint/test_property_chokepoint.py new file mode 100644 index 00000000..e20f6e0b --- /dev/null +++ b/tests/unit/scanner/taint/test_property_chokepoint.py @@ -0,0 +1,441 @@ +# tests/unit/scanner/taint/test_property_chokepoint.py +"""Property/differential harness around the L2 taint chokepoint (wardline-369f54b83b). + +Every L2 statement handler routes through the shared recursive core +``_resolve_expr`` / ``_resolve_call`` in ``scanner/taint/variable_level.py`` — +the engine's historical soundness-bug magnet (fail-open launder, stale +``var_types``, lambda single-slot FN, branch-locality FN, raw-receiver bypass). +This module pins four PROPERTIES over generated program corpora, so a future +engine edit that re-opens any of those bug families trips a property here +instead of shipping as a silent FN: + +1. **Lattice monotonicity** (``TestLatticeMonotonicity``): a program whose one + EXTERNAL_RAW input syntactically flows to ``return`` through propagating + constructs never resolves a return taint outside the engine's ``RAW_ZONE`` + (no clean-direction launder). +2. **Idempotence** (``TestIdempotence``): analyzing the same file twice — a + fresh analyzer each time, and the same analyzer re-run — yields + byte-identical findings streams. +3. **Monotonic seeds** (``TestMonotonicSeeds``): weakening an input seed + (trusted literal -> EXTERNAL_RAW boundary read) never REMOVES a + taint-driven finding. +4. **Receiver-guard invariant** (``TestReceiverGuardInvariant``): a typed + receiver whose value is DECLARED raw (EXTERNAL_RAW / MIXED_RAW) never + resolves a trusted method-call result through its clean ``Type.method`` + summary (pins wardline-03c8805449 as a property). + +The corpus generators are deterministic (``random.Random(seed)`` — Mersenne +Twister, stable across CPython versions) and hand-rolled rather than +hypothesis-driven: a frozen, replayable corpus keeps CI runtime flat and makes +a property failure a stable repro (re-run the seed) instead of a shrink hunt. +The grammar deliberately contains NO sanctioning construct (no decorated/ +``taint_map``-mapped callee, no ``_CONTEXT_ENCODERS`` member, no +``_NON_PROPAGATING_BUILTINS`` validator), so every generated flow is one the +lattice must keep raw — templates that legitimately clean are excluded from +the property set by construction, per the ticket. + +Contextvar coupling notes for the handlers under test live in +``CHOKEPOINT_NOTES.md`` (same directory). +""" + +from __future__ import annotations + +import ast +import random +from typing import TYPE_CHECKING + +from wardline.core.config import WardlineConfig +from wardline.core.taints import RAW_ZONE, TRUST_RANK, TaintState +from wardline.scanner.analyzer import WardlineAnalyzer +from wardline.scanner.taint.variable_level import compute_return_taint, compute_variable_taints + +if TYPE_CHECKING: + from collections.abc import Callable, Sequence + from pathlib import Path + + from wardline.core.finding import Finding + +T = TaintState + +# ── Deterministic program generator ────────────────────────────────────────── +# +# A grammar of statement templates, each a taint-PROPAGATING construct: given +# the current tainted variable ``cur`` it emits statements that flow the taint +# into a fresh variable ``nxt``. Chains of 1–4 templates compose into a small +# function whose single EXTERNAL_RAW input (``src``) syntactically reaches the +# final ``return``. Templates cover the ``_resolve_expr`` dispatch surface +# (Name/BinOp/JoinedStr/IfExp/containers/Subscript/BoolOp/NamedExpr), the +# ``_resolve_call`` curated ops (propagating builtins, ``.join``, ``.get``, +# raw-receiver method calls), and the statement-layer merges (if/else, for, +# try/except, match, unpacking, augmented assignment). + +_Template = tuple[str, "Callable[[str, str], list[str]]"] + +_TEMPLATES: tuple[_Template, ...] = ( + ("alias", lambda c, n: [f"{n} = {c}"]), + ("binop", lambda c, n: [f'{n} = {c} + "lit"']), + ("fstring", lambda c, n: [f'{n} = f"v={{{c}}}!"']), + ("ifexp", lambda c, n: [f'{n} = "d" if flag else {c}']), + ("list_subscript", lambda c, n: [f'{n} = [{c}, "a"][0]']), + ("dict_get", lambda c, n: [f'_t_{n} = {{"k": {c}}}', f'{n} = _t_{n}.get("k", "d")']), + ("if_else_merge", lambda c, n: ["if flag:", f" {n} = {c}", "else:", f' {n} = "d"']), + ("augassign", lambda c, n: [f'{n} = ""', f"{n} += {c}"]), + ("tuple_unpack", lambda c, n: [f'{n}, _u_{n} = {c}, "d"']), + ("str_builtin", lambda c, n: [f"{n} = str({c})"]), + ("join_method", lambda c, n: [f'{n} = ",".join([{c}])']), + ("raw_receiver_method", lambda c, n: [f"{n} = {c}.strip()"]), + ("boolop", lambda c, n: [f'{n} = {c} or "d"']), + ("walrus", lambda c, n: [f"_w_{n} = ({n} := {c})"]), + ("for_loop", lambda c, n: [f"for _i_{n} in [{c}]:", f" {n} = _i_{n}"]), + ("try_except", lambda c, n: ["try:", f" {n} = {c}", "except Exception:", f" {n} = {c}"]), + ("match_capture", lambda c, n: [f"match {c}:", f" case _v_{n}:", f" {n} = _v_{n}"]), + ("starred_unpack", lambda c, n: [f'_a_{n}, *{n} = "d", {c}']), +) + +_P1_CASES = 200 + + +def _generate_chain(seed: int) -> tuple[str, list[str]]: + """One generated program: ``def f(src, flag)`` + 1–4 chained templates + + ``return ``. Returns ``(source, template_names_used)``.""" + rng = random.Random(seed) + n_stmts = rng.randint(1, 4) + lines = ["def f(src, flag):"] + cur = "src" + used: list[str] = [] + for i in range(n_stmts): + name, template = rng.choice(_TEMPLATES) + used.append(name) + nxt = f"v{i}" + lines.extend(" " + line for line in template(cur, nxt)) + cur = nxt + lines.append(f" return {cur}") + return "\n".join(lines) + "\n", used + + +def _resolve_program(src: str, *, param_meets: dict[str, TaintState], taint_map: dict[str, TaintState]) -> TaintState: + """Run a generated program through the chokepoint and return its return taint. + + ``function_taint=GUARDED`` is the neutral trusted-zone seed: every fallback + path (unknown name, unmodelled expression) lands at rank 2, so the ONLY way + a result reaches ``RAW_ZONE`` is genuine propagation of the raw seed — and + the only way a raw flow LEAVES ``RAW_ZONE`` is a launder, which is exactly + what the property must catch. + """ + func = ast.parse(src).body[0] + assert isinstance(func, ast.FunctionDef | ast.AsyncFunctionDef) + var_taints = compute_variable_taints(func, T.GUARDED, dict(taint_map), param_meets=param_meets) + result = compute_return_taint(func, T.GUARDED, dict(taint_map), var_taints) + assert result is not None, f"generated program must have a value-bearing return:\n{src}" + return result + + +class TestGeneratorMeta: + """The corpus itself is part of the contract: deterministic and covering.""" + + def test_corpus_is_deterministic(self) -> None: + first = [_generate_chain(seed)[0] for seed in range(_P1_CASES)] + second = [_generate_chain(seed)[0] for seed in range(_P1_CASES)] + assert first == second + + def test_corpus_parses_and_exercises_every_template(self) -> None: + seen: set[str] = set() + for seed in range(_P1_CASES): + src, used = _generate_chain(seed) + ast.parse(src) # every generated program is valid Python + seen.update(used) + missing = {name for name, _ in _TEMPLATES} - seen + assert not missing, f"templates never exercised by the corpus: {sorted(missing)}" + + +class TestLatticeMonotonicity: + """Property 1 — no clean-direction launder through the chokepoint. + + One input (``src``) is seeded EXTERNAL_RAW and flows syntactically to the + return through propagating constructs only; the resolved return taint must + stay in the engine's RAW_ZONE for every program in the corpus. + """ + + def test_raw_seed_never_resolves_to_trusted_zone(self) -> None: + failures: list[str] = [] + for seed in range(_P1_CASES): + src, used = _generate_chain(seed) + result = _resolve_program(src, param_meets={"src": T.EXTERNAL_RAW}, taint_map={}) + if result not in RAW_ZONE: + failures.append(f"seed={seed} return={result} (rank {TRUST_RANK[result]}) templates={used}\n{src}") + assert not failures, "raw input laundered to the trusted zone:\n\n" + "\n".join(failures) + + def test_property_actually_discriminates(self) -> None: + # Negative control: the SAME corpus with a GUARDED (trusted-zone) seed + # must NOT trip the raw-zone check everywhere — i.e. the property's + # signal comes from the seed, not from the templates being raw-by- + # construction (a corpus that returned RAW_ZONE regardless of seed + # would make property 1 vacuous). + trusted_zone_results = 0 + for seed in range(_P1_CASES): + src, _used = _generate_chain(seed) + result = _resolve_program(src, param_meets={"src": T.GUARDED}, taint_map={}) + if result not in RAW_ZONE: + trusted_zone_results += 1 + assert trusted_zone_results == _P1_CASES, ( + "with a trusted seed every chain should resolve in the trusted zone " + f"(got {trusted_zone_results}/{_P1_CASES} — some template injects raw on its own)" + ) + + +# ── Analyzer-level corpus (properties 2 and 3) ─────────────────────────────── +# +# File-level programs: an @external_boundary reader seeds EXTERNAL_RAW, a +# @trusted function carries it through a propagation chain into a sink. The +# @trusted decoration matters — undecorated functions sit in the freedom zone +# and the severity model suppresses their sink findings. + +_MODULE_HEADER = ( + "import os, pickle, subprocess\n" + "from wardline.decorators import external_boundary, trusted\n" + "@external_boundary\n" + "def read_raw(p):\n" + " return p\n" +) + +# Single-line propagation templates only: the raw and trusted module variants +# must be line-for-line aligned apart from the one seed line, so findings can +# be compared by (rule_id, qualname, line_start). +_FLAT_TEMPLATES: tuple[Callable[[str, str], list[str]], ...] = ( + lambda c, n: [f"{n} = {c}"], + lambda c, n: [f'{n} = {c} + "lit"'], + lambda c, n: [f'{n} = f"v={{{c}}}!"'], + lambda c, n: ["if flag:", f" {n} = {c}", "else:", f' {n} = "d"'], + lambda c, n: [f"{n} = str({c})"], + lambda c, n: [f'{n} = [{c}, "a"][0]'], +) + +_SINKS: tuple[Callable[[str], str], ...] = ( + lambda v: f"os.system({v})", + lambda v: f"pickle.loads({v})", + lambda v: f"eval({v})", + lambda v: f"subprocess.run({v}, shell=True)", +) + +# Rules whose firing decision is driven by a resolved taint reaching a sink / +# return (the population property 3 quantifies over). Structural rules +# (PY-WL-102/103/104/110/111/113/114/119) are excluded: they gate on shape, not +# taint, so a seed change can legitimately add or remove them in either +# direction (e.g. a boundary-partition verdict that depends on what the body +# returns). +_TAINT_DRIVEN_RULES = frozenset( + { + "PY-WL-101", + "PY-WL-105", + "PY-WL-106", + "PY-WL-107", + "PY-WL-108", + "PY-WL-109", + "PY-WL-112", + "PY-WL-115", + "PY-WL-116", + "PY-WL-117", + "PY-WL-118", + "PY-WL-120", + "PY-WL-121", + "PY-WL-122", + "PY-WL-123", + "PY-WL-124", + "PY-WL-125", + "PY-WL-126", + } +) + +_P3_CASES = 40 + + +def _generate_module(seed: int, *, raw_seed: bool) -> str: + """A sink-bearing module; ``raw_seed`` picks the one differing line + (boundary read vs trusted literal).""" + rng = random.Random(seed) + n_stmts = rng.randint(0, 2) + seed_expr = "read_raw(p)" if raw_seed else '"safe"' + lines = ["@trusted(level='ASSURED')", f"def fn{seed}(p, flag):", f" data = {seed_expr}"] + cur = "data" + for i in range(n_stmts): + template = rng.choice(_FLAT_TEMPLATES) + nxt = f"v{i}" + lines.extend(" " + line for line in template(cur, nxt)) + cur = nxt + lines.append(" " + rng.choice(_SINKS)(cur)) + lines.append(f" return {cur}") + return _MODULE_HEADER + "\n".join(lines) + "\n" + + +def _analyze_module(tmp_path: Path, name: str, src: str) -> Sequence[Finding]: + p = tmp_path / name + p.write_text(src, encoding="utf-8") + return WardlineAnalyzer().analyze([p], WardlineConfig(), root=tmp_path) + + +def _findings_stream(findings: Sequence[Finding]) -> str: + """The findings stream as bytes-comparable text — order-preserving, full + serialized payload (``to_jsonl`` covers every wire-visible field).""" + return "\n".join(f.to_jsonl() for f in findings) + + +class TestIdempotence: + """Property 2 — same file in, identical findings stream out. + + Both repetition shapes: a FRESH analyzer per run (no shared state at all) + and the SAME analyzer re-run (per-run state must be fully reset between + ``analyze`` calls — a stale cache / leaked contextvar shows up here). + """ + + # A construct-dense module on top of the generated ones: branch-bound + # lambdas, container mutators, typed receivers, try/except, match, loops — + # the id()-keyed and contextvar-coupled paths most likely to destabilise. + _RICH_MODULE = _MODULE_HEADER + ( + "@trusted(level='ASSURED')\n" + "def rich(p, flag):\n" + " data = read_raw(p)\n" + " box = []\n" + " box.append(data)\n" + " if flag:\n" + " cb = lambda v: os.system(v)\n" + " else:\n" + " cb = lambda v: eval(v)\n" + " cb(data)\n" + " try:\n" + " out = box[0]\n" + " except Exception:\n" + " out = 'd'\n" + " match out:\n" + " case v:\n" + " acc = ''\n" + " for item in box:\n" + " acc += item\n" + " subprocess.run(acc, shell=True)\n" + " return acc\n" + ) + + def _corpus(self) -> list[str]: + return [_generate_module(seed, raw_seed=True) for seed in (3, 7, 11)] + [self._RICH_MODULE] + + def test_fresh_analyzer_twice_is_byte_identical(self, tmp_path: Path) -> None: + for idx, src in enumerate(self._corpus()): + name = f"m_fresh_{idx}.py" + first = _findings_stream(_analyze_module(tmp_path, name, src)) + second = _findings_stream(_analyze_module(tmp_path, name, src)) + assert first == second, f"fresh-analyzer re-analysis diverged for module {idx}:\n{src}" + + def test_same_analyzer_rerun_is_byte_identical(self, tmp_path: Path) -> None: + for idx, src in enumerate(self._corpus()): + p = tmp_path / f"m_same_{idx}.py" + p.write_text(src, encoding="utf-8") + analyzer = WardlineAnalyzer() + first = _findings_stream(analyzer.analyze([p], WardlineConfig(), root=tmp_path)) + second = _findings_stream(analyzer.analyze([p], WardlineConfig(), root=tmp_path)) + assert first == second, f"same-analyzer re-run diverged for module {idx}:\n{src}" + + def test_idempotence_corpus_produces_findings(self, tmp_path: Path) -> None: + # Anti-vacuity: the corpus must actually exercise the finding pipeline + # (byte-equality of two empty streams proves nothing). + for idx, src in enumerate(self._corpus()): + findings = _analyze_module(tmp_path, f"m_av_{idx}.py", src) + assert any(f.rule_id in _TAINT_DRIVEN_RULES for f in findings), ( + f"idempotence module {idx} produced no taint-driven findings:\n{src}" + ) + + +class TestMonotonicSeeds: + """Property 3 — weakening a seed (trusted -> raw) never removes a + taint-driven finding: findings(raw) ⊇ findings(trusted) restricted to + ``_TAINT_DRIVEN_RULES``, compared on (rule_id, qualname, line_start). + """ + + @staticmethod + def _taint_keys(findings: Sequence[Finding]) -> set[tuple[str, str | None, int | None]]: + return {(f.rule_id, f.qualname, f.location.line_start) for f in findings if f.rule_id in _TAINT_DRIVEN_RULES} + + def test_variants_differ_in_exactly_the_seed_line(self) -> None: + # Meta: the (rule_id, qualname, line_start) comparison is only valid if + # the two variants are line-aligned apart from the seed expression. + for seed in range(_P3_CASES): + raw_lines = _generate_module(seed, raw_seed=True).splitlines() + trusted_lines = _generate_module(seed, raw_seed=False).splitlines() + assert len(raw_lines) == len(trusted_lines) + diff = [i for i, (a, b) in enumerate(zip(raw_lines, trusted_lines, strict=True)) if a != b] + assert len(diff) == 1, f"seed={seed}: variants must differ on exactly one line, got {diff}" + + def test_raw_seed_findings_superset_of_trusted_seed(self, tmp_path: Path) -> None: + failures: list[str] = [] + nonempty_raw_cases = 0 + for seed in range(_P3_CASES): + raw_keys = self._taint_keys( + _analyze_module(tmp_path, f"m_{seed}_raw.py", _generate_module(seed, raw_seed=True)) + ) + trusted_keys = self._taint_keys( + _analyze_module(tmp_path, f"m_{seed}_tr.py", _generate_module(seed, raw_seed=False)) + ) + if raw_keys: + nonempty_raw_cases += 1 + lost = trusted_keys - raw_keys + if lost: + failures.append( + f"seed={seed}: weakening the seed REMOVED findings {sorted(lost)}\n" + f"{_generate_module(seed, raw_seed=True)}" + ) + assert not failures, "seed monotonicity violated:\n\n" + "\n".join(failures) + # Anti-vacuity: the raw variants must actually fire taint-driven rules. + assert nonempty_raw_cases == _P3_CASES, ( + f"only {nonempty_raw_cases}/{_P3_CASES} raw-seed programs produced taint-driven findings" + ) + + +class TestReceiverGuardInvariant: + """Property 4 — the declared-raw typed-receiver guard (wardline-03c8805449). + + A receiver with a tracked type AND a DECLARED-raw value (EXTERNAL_RAW / + MIXED_RAW — a boundary-seeded parameter, not a constructor default) must + never resolve a trusted result through its clean ``Type.method`` summary: + the summary describes a trustworthy instance, not the attacker-controlled + object actually flowing in. + + UNKNOWN_RAW receivers are DELIBERATELY out of scope: an unmodelled + ``Type()`` constructor defaults to UNKNOWN_RAW, and gating those would + false-positive the ``h = Helper(); h.get_assured()`` pattern — the engine + resolves the clean summary there by design (pinned as the negative control + below, so a future edit that silently widens or narrows the guard trips + one of the two tests). + """ + + _SHAPES = ( + # direct assignment from the typed-receiver method call + "def f(h: {cls}):\n x = h.{meth}()\n return x\n", + # with an argument, then an alias hop + "def f(h: {cls}):\n y = h.{meth}('a')\n x = y\n return x\n", + ) + + def test_declared_raw_receiver_never_resolves_trusted(self) -> None: + failures: list[str] = [] + for cls, meth in (("Helper", "get_assured"), ("Vault", "load")): + for summary in (T.INTEGRAL, T.ASSURED, T.GUARDED): + for receiver in (T.EXTERNAL_RAW, T.MIXED_RAW): + for shape in self._SHAPES: + src = shape.format(cls=cls, meth=meth) + func = ast.parse(src).body[0] + assert isinstance(func, ast.FunctionDef) + taint_map = {f"{cls}.{meth}": summary} + var_taints = compute_variable_taints(func, T.GUARDED, taint_map, param_meets={"h": receiver}) + if var_taints["x"] not in RAW_ZONE: + failures.append( + f"summary={summary} receiver={receiver} resolved x={var_taints['x']}\n{src}" + ) + assert not failures, "declared-raw receiver laundered through a clean summary:\n\n" + "\n".join(failures) + + def test_unknown_raw_receiver_still_resolves_summary(self) -> None: + # Negative control / guard-boundary pin: an UNKNOWN_RAW-valued typed + # receiver (the constructor-default provenance) DOES resolve the clean + # summary — the FP-avoidance side of wardline-03c8805449. + src = "def f(h: Helper):\n x = h.get_assured()\n return x\n" + func = ast.parse(src).body[0] + assert isinstance(func, ast.FunctionDef) + taint_map = {"Helper.get_assured": T.ASSURED} + var_taints = compute_variable_taints(func, T.GUARDED, taint_map, param_meets={"h": T.UNKNOWN_RAW}) + assert var_taints["x"] == T.ASSURED diff --git a/tests/unit/scanner/taint/test_review_fixups_engine.py b/tests/unit/scanner/taint/test_review_fixups_engine.py new file mode 100644 index 00000000..692fe92e --- /dev/null +++ b/tests/unit/scanner/taint/test_review_fixups_engine.py @@ -0,0 +1,371 @@ +# tests/unit/scanner/taint/test_review_fixups_engine.py +"""Regression tests for the 2026-06-10 review-panel engine fixups. + +Covers four confirmed soundness defects in the L2 walk +(``wardline.scanner.taint.variable_level``): + +1. **Compare/Raise/Assert/Delete expression positions** — a call nested in an + ``ast.Compare`` operand (or a Raise/Assert/Delete statement) was never + resolved by ``_resolve_expr``/``_process_stmt``, so its per-call arg-taint + snapshot was missing: PY-WL-105's provably-untrusted gate stayed silent (FN) + and a CONSTANT-arg sink call degraded to the pessimistic UNKNOWN_RAW + fallback (a gate-tripping PY-WL-108 FP after the WARN→ERROR recalibration). + +2. **Stale receiver-type candidates across non-Assign rebinds** — for/with/ + walrus/tuple-unpack/except-handler rebinds updated ``var_taints`` but not + ``_CURRENT_VAR_TYPES``, so a stale class candidate resolved a clean + ``@trusted`` method summary onto a rebound raw receiver (PY-WL-101 FN). + +3. **Provenance-only re-resolution polluting the flow-sensitive snapshot** — + ``compute_return_callee``'s ``_assignment_callee`` re-resolved direct-call + assignment RHS against the FINAL var_taints with the arg-taint recorder + still active, combining post-call taint into the at-call snapshot + (PY-WL-105/108 FPs on values that were clean AT the call). + +4. **Nested local helper calls** — the bare-name worst-arg conservatism + (wardline-93d608c997) hit nested defs the engine had already analyzed + (``m.f..helper``), marking validated values raw (PY-WL-101 FP). +""" + +from __future__ import annotations + +import textwrap +import warnings +from typing import TYPE_CHECKING + +from wardline.core.config import WardlineConfig +from wardline.scanner.analyzer import WardlineAnalyzer + +if TYPE_CHECKING: + from collections.abc import Sequence + from pathlib import Path + + from wardline.core.finding import Finding + +_PREAMBLE = ( + "import os\n" + "from wardline.decorators import external_boundary, trusted\n" + "@external_boundary\n" + "def read_raw(p):\n" + " return p\n" + "@trusted(level='ASSURED')\n" + "def store(x):\n" + " return 1\n" +) + + +def _scan(tmp_path: Path, body: str, name: str = "m.py", preamble: str = _PREAMBLE) -> Sequence[Finding]: + p = tmp_path / name + p.write_text(preamble + textwrap.dedent(body), encoding="utf-8") + with warnings.catch_warnings(): + warnings.simplefilter("error") # the engine must not warn from rule checks + return WardlineAnalyzer().analyze([p], WardlineConfig(), root=tmp_path) + + +def _rule_hits(findings: Sequence[Finding], rule_id: str) -> list[Finding]: + return [f for f in findings if f.rule_id == rule_id] + + +# ── 1. Compare / Raise / Assert / Delete positions ─────────────────────────── + + +def test_105_fires_on_call_inside_compare(tmp_path: Path) -> None: + findings = _scan( + tmp_path, + """ + def h(p): + if store(read_raw(p)) == 1: + return 1 + return 0 + """, + ) + hits = _rule_hits(findings, "PY-WL-105") + assert len(hits) == 1 + assert hits[0].qualname == "m.h" + + +def test_105_fires_on_call_inside_raise(tmp_path: Path) -> None: + findings = _scan( + tmp_path, + """ + def h(p): + raise ValueError(store(read_raw(p))) + """, + ) + assert len(_rule_hits(findings, "PY-WL-105")) == 1 + + +def test_105_fires_on_call_inside_assert(tmp_path: Path) -> None: + findings = _scan( + tmp_path, + """ + def h(p): + assert store(read_raw(p)) + """, + ) + assert len(_rule_hits(findings, "PY-WL-105")) == 1 + + +def test_105_fires_on_call_inside_assert_msg_and_raise_cause(tmp_path: Path) -> None: + findings = _scan( + tmp_path, + """ + def h_msg(p): + assert p, store(read_raw(p)) + + def h_cause(p): + raise ValueError("x") from store(read_raw(p)) + """, + ) + assert {f.qualname for f in _rule_hits(findings, "PY-WL-105")} == {"m.h_msg", "m.h_cause"} + + +def test_108_constant_command_inside_compare_is_clean(tmp_path: Path) -> None: + """FP-corpus sentinel: ``if os.system(CONST) == 0:`` is THE idiomatic + command-success check — the missing-Compare pessimistic fallback must not + turn the constant into a gate-tripping ERROR.""" + findings = _scan( + tmp_path, + """ + @trusted(level='ASSURED') + def const_cmd(): + if os.system("ls") == 0: + return 1 + return 0 + """, + ) + assert _rule_hits(findings, "PY-WL-108") == [] + + +def test_del_subscript_resolves_walrus_and_call(tmp_path: Path) -> None: + """A Delete statement's target expressions are resolved (slice calls record).""" + findings = _scan( + tmp_path, + """ + def h(p, d): + del d[store(read_raw(p))] + """, + ) + assert len(_rule_hits(findings, "PY-WL-105")) == 1 + + +# ── 2. Receiver-type invalidation on non-Assign rebinds ────────────────────── + +_VAULT_PREAMBLE = ( + "import junklib\n" + "from wardline.decorators import trusted\n" + "class Vault:\n" + " @trusted(level='ASSURED')\n" + " def get(self):\n" + " return 'safe'\n" +) + + +def _vault_101(tmp_path: Path, body: str) -> list[Finding]: + findings = _scan(tmp_path, body, name="vault.py", preamble=_VAULT_PREAMBLE) + return _rule_hits(findings, "PY-WL-101") + + +def test_101_fires_after_for_target_rebind(tmp_path: Path) -> None: + hits = _vault_101( + tmp_path, + """ + @trusted(level='ASSURED', to_level='ASSURED') + def f(): + v = Vault() + for v in junklib.items(): + pass + x = v.get() + return x + """, + ) + assert len(hits) == 1 + + +def test_101_fires_after_with_target_rebind(tmp_path: Path) -> None: + hits = _vault_101( + tmp_path, + """ + @trusted(level='ASSURED', to_level='ASSURED') + def f(): + v = Vault() + with junklib.ctx() as v: + pass + x = v.get() + return x + """, + ) + assert len(hits) == 1 + + +def test_101_fires_after_walrus_rebind(tmp_path: Path) -> None: + hits = _vault_101( + tmp_path, + """ + @trusted(level='ASSURED', to_level='ASSURED') + def f(): + v = Vault() + if (v := junklib.one()): + pass + x = v.get() + return x + """, + ) + assert len(hits) == 1 + + +def test_101_fires_after_tuple_unpack_rebind(tmp_path: Path) -> None: + hits = _vault_101( + tmp_path, + """ + @trusted(level='ASSURED', to_level='ASSURED') + def f(): + v = Vault() + v, w = junklib.pair() + x = v.get() + return x + """, + ) + assert len(hits) == 1 + + +def test_except_handler_rebind_invalidates_receiver_type() -> None: + """``except E as v`` rebinds ``v`` to the exception instance — the stale + [Vault] candidate must not resolve ``Vault.fetch -> ASSURED`` on the handler + path (the launder is observable at the L2 unit level: with the stale type, + ``x`` came back ASSURED; with the invalidation it stays UNKNOWN_RAW). + + Note the END-TO-END except shape (rebind in the handler, method call after + the try) stays silent for a DIFFERENT, pre-existing reason: the handler + binds the exception name to the enclosing function's trusted seed (a value- + channel conservatism, not the receiver-type launder this fix closes). + """ + import ast + + from wardline.core.taints import TaintState + from wardline.scanner.taint.variable_level import ( + VariableTaintContext, + analyze_function_variables, + ) + + src = textwrap.dedent( + """ + def f(): + v = Vault() + try: + go() + except Exception as v: + x = v.fetch() + return x + """ + ) + fn = ast.parse(src).body[0] + assert isinstance(fn, ast.FunctionDef) + result = analyze_function_variables( + fn, + TaintState.UNKNOWN_RAW, + {"vault.Vault.fetch": TaintState.ASSURED}, + VariableTaintContext(alias_map={}, module_prefix="vault"), + ) + assert result.variable_taints["x"] is TaintState.UNKNOWN_RAW + + +def test_walrus_constructor_rebind_keeps_type_precision(tmp_path: Path) -> None: + """A walrus that REBINDS to a constructor is a typed strong update, not an + invalidation: ``(v := Vault())`` then ``v.get()`` must stay clean.""" + hits = _vault_101( + tmp_path, + """ + @trusted(level='ASSURED', to_level='ASSURED') + def f(): + if (v := Vault()): + pass + x = v.get() + return x + """, + ) + assert hits == [] + + +# ── 3. Provenance re-resolution must not pollute the at-call snapshot ──────── + + +def test_no_105_when_arg_becomes_raw_only_after_the_call(tmp_path: Path) -> None: + findings = _scan( + tmp_path, + """ + def f(p): + x = "const" + v = store(x) + x = read_raw(p) + return v + """, + ) + assert _rule_hits(findings, "PY-WL-105") == [] + + +def test_no_108_when_command_becomes_raw_only_after_the_call(tmp_path: Path) -> None: + findings = _scan( + tmp_path, + """ + @trusted(level='ASSURED') + def f(p): + cmd = "ls" + v = os.system(cmd) + cmd = read_raw(p) + return v + """, + ) + assert _rule_hits(findings, "PY-WL-108") == [] + + +# ── 4. Nested local helper calls resolve via the engine's own summary ──────── + + +def test_nested_local_helper_validating_raw_is_not_a_101_fp(tmp_path: Path) -> None: + findings = _scan( + tmp_path, + """ + @trusted(level='ASSURED', to_level='ASSURED') + def f(p): + def clean(x): + return 1 + raw = read_raw(p) + v = clean(raw) + return v + """, + ) + assert _rule_hits(findings, "PY-WL-101") == [] + + +def test_nested_local_helper_returning_raw_still_fires_101(tmp_path: Path) -> None: + """The nested-def lookup must use the helper's REAL summary — a helper that + passes raw through keeps firing (no launder through the bare-name hop).""" + findings = _scan( + tmp_path, + """ + @trusted(level='ASSURED', to_level='ASSURED') + def f(p): + def passthrough(x): + return x + raw = read_raw(p) + v = passthrough(raw) + return v + """, + ) + assert len(_rule_hits(findings, "PY-WL-101")) == 1 + + +def test_bare_param_callee_still_pessimistic(tmp_path: Path) -> None: + """wardline-93d608c997 stays closed: an UNKNOWN bare-name callee (a param) + cannot be assumed to clean a raw argument.""" + findings = _scan( + tmp_path, + """ + @trusted(level='ASSURED') + def launder_check(transform, p): + raw = read_raw(p) + os.system(transform(raw)) + """, + ) + assert len(_rule_hits(findings, "PY-WL-108")) == 1 diff --git a/tests/unit/scanner/taint/test_variable_level.py b/tests/unit/scanner/taint/test_variable_level.py index 4c8d927a..4dbbcf29 100644 --- a/tests/unit/scanner/taint/test_variable_level.py +++ b/tests/unit/scanner/taint/test_variable_level.py @@ -1066,12 +1066,17 @@ def test_dict_pop_joins_tainted_default_arg() -> None: def test_unknown_builtin_call_still_falls_back_to_function_taint() -> None: - # HARD BOUNDARY: len()/int()/validate() are NOT curated — they must STILL fall - # back to function_taint, or we explode in false positives. + # HARD BOUNDARY: len()/int() are curated NON-propagators (validators/ + # measurers) — they keep the function_taint fallback. An unknown bare call + # like validate() (absent from the taint_map) is the OPPOSITE: it now + # propagates the worst of (caller seed, args) — an unmodeled callee cannot + # be assumed to clean a raw argument (wardline-93d608c997; the unresolved + # bare-name launder closure). See test_engine_precision.py for the family. assert _vt("def f(p):\n x = len(read_raw(p))\n", function_taint=T.GUARDED, taint_map=_RAW_TM)["x"] == T.GUARDED assert _vt("def f(p):\n x = int(read_raw(p))\n", function_taint=T.GUARDED, taint_map=_RAW_TM)["x"] == T.GUARDED assert ( - _vt("def f(p):\n x = validate(read_raw(p))\n", function_taint=T.GUARDED, taint_map=_RAW_TM)["x"] == T.GUARDED + _vt("def f(p):\n x = validate(read_raw(p))\n", function_taint=T.GUARDED, taint_map=_RAW_TM)["x"] + == T.UNKNOWN_RAW ) diff --git a/tests/unit/scanner/test_analyzer.py b/tests/unit/scanner/test_analyzer.py index 706a90dc..0ba7e860 100644 --- a/tests/unit/scanner/test_analyzer.py +++ b/tests/unit/scanner/test_analyzer.py @@ -193,12 +193,23 @@ def test_analyzer_seeded_taints_drive_transitive_propagation(tmp_path) -> None: assert ctx.project_taints["m.mix"] == T.UNKNOWN_RAW # raw, floored — NOT MIXED_RAW -def test_analyzer_skips_unparseable_file_with_fact(tmp_path) -> None: +def test_analyzer_skips_unparseable_file_with_gating_defect(tmp_path) -> None: + # A discovered-but-unparseable file is a GATE-ELIGIBLE ERROR DEFECT (fail-closed: + # its sinks were never analyzed, so the default --fail-on ERROR loop must not read + # GREEN over it) — was a non-gating NONE FACT before the secure-gate change. + from wardline.core.finding import Severity + _write(tmp_path, "bad.py", "def f(:\n") # syntax error _write(tmp_path, "good.py", "def g(): return 1\n") analyzer = WardlineAnalyzer() findings = analyzer.analyze([tmp_path / "bad.py", tmp_path / "good.py"], WardlineConfig(), root=tmp_path) - assert any(f.rule_id == "WLN-ENGINE-PARSE-ERROR" and f.kind == Kind.FACT for f in findings) + parse_errors = [f for f in findings if f.rule_id == "WLN-ENGINE-PARSE-ERROR"] + assert len(parse_errors) == 1 + assert parse_errors[0].kind == Kind.DEFECT + assert parse_errors[0].severity == Severity.ERROR + # line_start is always set, so the lineless-DEFECT downgrade never demotes it. + assert parse_errors[0].location.line_start is not None + # The unparseable file is isolated; the clean sibling is still analysed. assert analyzer.last_context is not None assert "good.g" in analyzer.last_context.project_taints @@ -257,3 +268,65 @@ def test_analyzer_exposes_return_taints_and_resolves_validators(tmp_path) -> Non # actual returned-value taint per function assert ctx.function_return_taints["service.safe"] == T.ASSURED # validated -> clean assert ctx.function_return_taints["service.leaky"] == T.EXTERNAL_RAW # leaks raw + + +def test_config_source_matching_entity_qualname_is_not_reported_unused(tmp_path) -> None: + # An untrusted_sources entry naming a PROJECT ENTITY QUALNAME seeds that entity + # EXTERNAL_RAW (the directive takes effect) — it must therefore be recorded as + # matched, never reported WLN-CONFIG-UNUSED-SOURCE. Before the fix only the + # import/alias path recorded matches, so a WORKING directive was misreported as + # a "configuration error" — pushing an agent to delete a load-bearing entry. + _write( + tmp_path, + "m.py", + "from wardline.decorators import trusted\n" + "def get_input():\n return 'x'\n" + "@trusted(level='ASSURED')\ndef f():\n return get_input()\n", + ) + config = WardlineConfig(untrusted_sources=("m.get_input",)) + analyzer = WardlineAnalyzer() + findings = analyzer.analyze([tmp_path / "m.py"], config, root=tmp_path) + # The seed demonstrably took effect: f leaks the EXTERNAL_RAW source. + ctx = analyzer.last_context + assert ctx is not None + assert ctx.project_return_taints["m.get_input"] == T.EXTERNAL_RAW + # ...so the diagnostic must not contradict it. + assert not any(f.rule_id == "WLN-CONFIG-UNUSED-SOURCE" for f in findings) + + +def test_config_source_matching_nothing_is_still_reported_unused(tmp_path) -> None: + # The diagnostic itself stays live: a source matching neither an import nor a + # project entity is still a (probable) configuration error worth surfacing. + _write(tmp_path, "m.py", "def f():\n return 1\n") + config = WardlineConfig(untrusted_sources=("nowhere.get_input",)) + analyzer = WardlineAnalyzer() + findings = analyzer.analyze([tmp_path / "m.py"], config, root=tmp_path) + unused = [f for f in findings if f.rule_id == "WLN-CONFIG-UNUSED-SOURCE"] + assert [f.properties["source"] for f in unused] == ["nowhere.get_input"] + + +def test_l2_fixed_point_scales_linearly_with_function_count(tmp_path) -> None: + # Regression pin for the O(n^2) blowup: the L2 fixed-point folded the whole + # project's return-taint map into EVERY function's taint map AND memo key, so + # each of N functions paid O(N) per iteration (N=2000 took ~3.3s, N=4000 + # ~13.2s pre-fix — a clean 4x per doubling). With per-function key pruning the + # scan must scale ~linearly: a 4x corpus may not cost anywhere near 16x. + import time + + def _scan_n(n: int, name: str) -> float: + src = "".join(f"def f{i}(x):\n return x\n" for i in range(n)) + root = tmp_path / name + root.mkdir() + (root / "m.py").write_text(src, encoding="utf-8") + analyzer = WardlineAnalyzer() + start = time.perf_counter() + analyzer.analyze([root / "m.py"], WardlineConfig(), root=root) + return time.perf_counter() - start + + small = _scan_n(800, "small") + large = _scan_n(3200, "large") + # Linear scaling gives ~4x; the pre-fix quadratic gave ~16x. 10x is the alarm + # threshold with generous headroom for timer noise on a loaded machine. + assert large < small * 10, f"L2 scan no longer scales linearly: 800→{small:.3f}s, 3200→{large:.3f}s" + # Absolute sanity: pre-fix 3200 functions took ~8s+; post-fix it is well under. + assert large < 4.0, f"3200-function scan took {large:.2f}s — quadratic regression?" diff --git a/tests/unit/scanner/test_analyzer_isolation.py b/tests/unit/scanner/test_analyzer_isolation.py new file mode 100644 index 00000000..368c2fd0 --- /dev/null +++ b/tests/unit/scanner/test_analyzer_isolation.py @@ -0,0 +1,296 @@ +# tests/unit/scanner/test_analyzer_isolation.py +"""Per-file / per-rule exception isolation (analyzer robustness). + +Before this suite existed, any non-RecursionError raised by the L2 walk or by a +single rule aborted the ENTIRE scan (exit 2, zero findings) — one pathological +file lost every other file's findings, asymmetric with the Rust frontend's +WLN-ENGINE-FILE-FAILED per-file fence. The Python contract pinned here: + + * an unexpected exception during one file's analysis becomes a GATE-ELIGIBLE + WLN-ENGINE-FILE-FAILED ERROR DEFECT naming the file (fail-closed — engine + bugs surface, never silently skip), and the scan continues; + * a crashing rule becomes a WLN-ENGINE-RULE-FAILED ERROR DEFECT at ENGINE_PATH + and every other rule still contributes its findings; + * parse failures (syntax/encoding/null bytes) are gate-eligible ERROR DEFECTs + so the default ``--fail-on ERROR`` loop cannot read GREEN over unscanned code. +""" + +from __future__ import annotations + +import ast +import textwrap +from pathlib import Path + +from wardline.core.config import WardlineConfig +from wardline.core.finding import ENGINE_PATH, UNANALYZED_RULE_IDS, Kind, Severity +from wardline.core.run import gate_decision, run_scan +from wardline.scanner.analyzer import WardlineAnalyzer +from wardline.scanner.context import RuleRegistry + + +def _write(root: Path, rel: str, src: str) -> Path: + p = root / rel + p.parent.mkdir(parents=True, exist_ok=True) + p.write_text(textwrap.dedent(src), encoding="utf-8") + return p + + +# --------------------------------------------------------------------------- +# Per-file isolation: an L2 engine exception fails ONE file, not the scan. +# --------------------------------------------------------------------------- + + +def _boom_on_marker(monkeypatch, exc: Exception) -> None: + """Make the L2 stage raise ``exc`` for any function whose body names ``boom``.""" + import wardline.scanner.analyzer as analyzer_mod + + real = analyzer_mod.run_l2_function_stage + + def _boom(stage_input): # noqa: ANN001, ANN202 + if any(isinstance(n, ast.Name) and n.id == "boom" for n in ast.walk(stage_input.node)): + raise exc + return real(stage_input) + + monkeypatch.setattr(analyzer_mod, "run_l2_function_stage", _boom) + + +def test_l2_engine_exception_fails_file_not_scan(tmp_path, monkeypatch) -> None: + _boom_on_marker(monkeypatch, ValueError("synthetic engine defect")) + _write(tmp_path, "broken.py", "def a():\n boom = 1\n return boom\n") + _write(tmp_path, "clean.py", "def ok():\n return 1\n") + analyzer = WardlineAnalyzer() + findings = analyzer.analyze([tmp_path / "broken.py", tmp_path / "clean.py"], WardlineConfig(), root=tmp_path) + + failed = [f for f in findings if f.rule_id == "WLN-ENGINE-FILE-FAILED"] + assert len(failed) == 1 + # Fail-closed contract: gate-eligible ERROR DEFECT naming the file, with a + # line anchor so the lineless-DEFECT downgrade cannot demote it out of the gate. + assert failed[0].kind == Kind.DEFECT + assert failed[0].severity == Severity.ERROR + assert failed[0].location.path == "broken.py" + assert failed[0].location.line_start is not None + assert "ValueError" in failed[0].message + # Counted toward ScanSummary.unanalyzed (single source of truth in core.finding). + assert "WLN-ENGINE-FILE-FAILED" in UNANALYZED_RULE_IDS + # The scan CONTINUED: the clean sibling is fully analysed. + ctx = analyzer.last_context + assert ctx is not None + assert "clean.ok" in ctx.project_taints + + +def test_l2_engine_exception_emits_one_finding_per_file(tmp_path, monkeypatch) -> None: + # The fingerprint is keyed on the relpath (the Rust frontend contract), so two + # failing entities in ONE file must collapse to one finding — never two distinct + # findings sharing a fingerprint (that would trip the collision guard). + _boom_on_marker(monkeypatch, ValueError("synthetic engine defect")) + _write( + tmp_path, + "broken.py", + "def a():\n boom = 1\n return boom\ndef b():\n boom = 2\n return boom\n", + ) + analyzer = WardlineAnalyzer() + findings = analyzer.analyze([tmp_path / "broken.py"], WardlineConfig(), root=tmp_path) + failed = [f for f in findings if f.rule_id == "WLN-ENGINE-FILE-FAILED"] + assert len(failed) == 1 + assert not any(f.rule_id == "WLN-ENGINE-FINGERPRINT-COLLISION" for f in findings) + + +def test_recursion_error_still_yields_function_skip_fact(tmp_path, monkeypatch) -> None: + # The broad fence must NOT swallow the dedicated RecursionError boundary — + # too-deep functions keep their non-gating WLN-ENGINE-FUNCTION-SKIPPED FACT. + _boom_on_marker(monkeypatch, RecursionError("simulated deep L2")) + _write(tmp_path, "deep.py", "def a():\n boom = 1\n return boom\n") + analyzer = WardlineAnalyzer() + findings = analyzer.analyze([tmp_path / "deep.py"], WardlineConfig(), root=tmp_path) + assert any(f.rule_id == "WLN-ENGINE-FUNCTION-SKIPPED" and f.kind == Kind.FACT for f in findings) + assert not any(f.rule_id == "WLN-ENGINE-FILE-FAILED" for f in findings) + + +def test_parse_stage_engine_exception_fails_file_not_scan(tmp_path, monkeypatch) -> None: + # An unexpected exception while indexing/seeding ONE file (parse stage) is also + # fenced per-file: WLN-ENGINE-FILE-FAILED for the named file, sibling analysed. + import wardline.scanner.pipeline as pipeline_mod + + real = pipeline_mod.seed_function_taints + + def _boom(entities, *, ctx, provider): # noqa: ANN001, ANN202 + if ctx.module == "broken": + raise KeyError("synthetic seeding defect") + return real(entities, ctx=ctx, provider=provider) + + monkeypatch.setattr(pipeline_mod, "seed_function_taints", _boom) + _write(tmp_path, "broken.py", "def a():\n return 1\n") + _write(tmp_path, "clean.py", "def ok():\n return 1\n") + analyzer = WardlineAnalyzer() + findings = analyzer.analyze([tmp_path / "broken.py", tmp_path / "clean.py"], WardlineConfig(), root=tmp_path) + failed = [f for f in findings if f.rule_id == "WLN-ENGINE-FILE-FAILED"] + assert len(failed) == 1 + assert failed[0].location.path == "broken.py" + assert failed[0].kind == Kind.DEFECT + assert failed[0].severity == Severity.ERROR + ctx = analyzer.last_context + assert ctx is not None + assert "clean.ok" in ctx.project_taints + + +# --------------------------------------------------------------------------- +# Per-rule isolation: one crashing rule loses its own findings only. +# --------------------------------------------------------------------------- + + +class _RaisingRule: + rule_id = "PY-WL-TEST-BOOM" + + def check(self, context): # noqa: ANN001, ANN202 + raise RuntimeError("synthetic rule defect") + + +def test_crashing_rule_is_isolated_and_fails_closed(tmp_path) -> None: + # Build the default registry PLUS a raising rule: the default rules' findings + # must survive, and the crash surfaces as a gate-eligible ERROR DEFECT. + from wardline.scanner.rules import build_default_registry + + registry = build_default_registry(WardlineConfig()) + registry.register(_RaisingRule()) + _write( + tmp_path, + "svc.py", + "import subprocess\n" + "from wardline.decorators import external_boundary, trusted\n" + "@external_boundary\ndef read_raw(p):\n return p\n" + "@trusted(level='ASSURED')\n" + "def run(p):\n subprocess.run(read_raw(p), shell=True)\n return 1\n", + ) + analyzer = WardlineAnalyzer(registry=registry) + findings = analyzer.analyze([tmp_path / "svc.py"], WardlineConfig(), root=tmp_path) + + rule_failed = [f for f in findings if f.rule_id == "WLN-ENGINE-RULE-FAILED"] + assert len(rule_failed) == 1 + assert rule_failed[0].kind == Kind.DEFECT + assert rule_failed[0].severity == Severity.ERROR + # ENGINE_PATH location: exempt from the lineless-DEFECT downgrade, so it gates. + assert rule_failed[0].location.path == ENGINE_PATH + assert rule_failed[0].properties["rule"] == "PY-WL-TEST-BOOM" + # The other rules still ran — the real sink finding is NOT lost. + assert any(f.rule_id.startswith("PY-WL-") and f.kind is Kind.DEFECT for f in findings) + + +def test_duck_typed_registry_seam_still_runs(tmp_path) -> None: + # A test-seam registry exposing only run() (no .rules) keeps the undelegated call. + class _Custom: + def run(self, context): # noqa: ANN001, ANN202 + return [] + + _write(tmp_path, "m.py", "def f():\n return 1\n") + analyzer = WardlineAnalyzer(registry=_Custom()) + findings = analyzer.analyze([tmp_path / "m.py"], WardlineConfig(), root=tmp_path) + assert any(f.rule_id == "WLN-ENGINE-METRICS" for f in findings) + + +def test_registry_rules_maturity_stamp_survives_per_rule_dispatch(tmp_path) -> None: + # The per-rule dispatch reuses RuleRegistry.run, so a PREVIEW rule's findings + # still carry their maturity stamp (the stamping must not be lost to isolation). + from dataclasses import dataclass + + from wardline.core.finding import Finding, Location, Maturity + + @dataclass(frozen=True) + class _Meta: + maturity: Maturity = Maturity.PREVIEW + + class _PreviewRule: + rule_id = "PY-WL-TEST-PREVIEW" + metadata = _Meta() + + def check(self, context): # noqa: ANN001, ANN202 + return [ + Finding( + rule_id="PY-WL-TEST-PREVIEW", + message="preview finding", + severity=Severity.WARN, + kind=Kind.DEFECT, + location=Location(path="m.py", line_start=1), + fingerprint="ab" * 32, + ) + ] + + registry = RuleRegistry() + registry.register(_PreviewRule()) + _write(tmp_path, "m.py", "def f():\n return 1\n") + analyzer = WardlineAnalyzer(registry=registry) + findings = analyzer.analyze([tmp_path / "m.py"], WardlineConfig(), root=tmp_path) + preview = [f for f in findings if f.rule_id == "PY-WL-TEST-PREVIEW"] + assert len(preview) == 1 + assert preview[0].maturity is Maturity.PREVIEW + + +# --------------------------------------------------------------------------- +# Gate eligibility: unscanned code must not pass `--fail-on ERROR` (fail-open fix). +# --------------------------------------------------------------------------- + + +def test_unparseable_file_trips_default_fail_on_error_gate(tmp_path) -> None: + # The exact fail-open from the robustness review: a syntax error wrapping a real + # command sink used to read PASSED/exit 0 under the documented agent loop. + proj = tmp_path / "proj" + proj.mkdir() + (proj / "bad.py").write_text("import subprocess\ndef f(\n subprocess.run(evil)\n", encoding="utf-8") + result = run_scan(proj) + decision = gate_decision(result, Severity.ERROR) + assert decision.tripped is True + assert decision.verdict == "FAILED" + assert result.summary.unanalyzed == 1 + + +def test_runnable_but_undecodable_encoding_trips_gate(tmp_path) -> None: + # The strengthened repro: a latin-1 coding cookie CPython tokenizes and RUNS, + # but the scanner's UTF-8 read rejects — live code invisible to analysis must + # not read GREEN. The encoding error carries no line; the defect must still + # gate (line_start falls back, dodging the lineless-DEFECT downgrade). + proj = tmp_path / "proj" + proj.mkdir() + (proj / "enc.py").write_bytes(b'# -*- coding: latin-1 -*-\nimport os\nos.system("ls \xe9")\n') + result = run_scan(proj) + parse_errors = [f for f in result.findings if f.rule_id == "WLN-ENGINE-PARSE-ERROR"] + assert len(parse_errors) == 1 + assert parse_errors[0].kind is Kind.DEFECT + assert parse_errors[0].location.line_start is not None + decision = gate_decision(result, Severity.ERROR) + assert decision.tripped is True + + +def test_null_byte_file_is_gate_eligible(tmp_path) -> None: + # Null bytes raise SyntaxError on current CPython and ValueError on some others; + # either way the file must surface as a gate-eligible under-scan defect. + proj = tmp_path / "proj" + proj.mkdir() + (proj / "nul.py").write_bytes(b"import os\x00\nos.system('x')\n") + result = run_scan(proj) + under_scan = [f for f in result.findings if f.rule_id in {"WLN-ENGINE-PARSE-ERROR", "WLN-ENGINE-FILE-FAILED"}] + assert len(under_scan) == 1 + assert under_scan[0].kind is Kind.DEFECT + assert under_scan[0].severity is Severity.ERROR + assert gate_decision(result, Severity.ERROR).tripped is True + + +def test_parse_error_gates_in_secure_population_but_baseline_annotates(tmp_path) -> None: + # Secure-by-default precedent: a committed baseline ANNOTATES the parse-error + # defect (suppressed in the emitted findings) but cannot clear the secure gate; + # --trust-suppressions (trusted checkout) honors it. + from wardline.core.finding import FINGERPRINT_SCHEME + + proj = tmp_path / "proj" + proj.mkdir() + (proj / "bad.py").write_text("def f(:\n", encoding="utf-8") + fp = next(f.fingerprint for f in run_scan(proj).findings if f.rule_id == "WLN-ENGINE-PARSE-ERROR") + bl = proj / ".weft" / "wardline" / "baseline.yaml" + bl.parent.mkdir(parents=True, exist_ok=True) + bl.write_text( + f"fingerprint_scheme: {FINGERPRINT_SCHEME}\nversion: 1\nentries:\n" + f" - fingerprint: {fp}\n rule_id: WLN-ENGINE-PARSE-ERROR\n path: bad.py\n message: m\n", + encoding="utf-8", + ) + secure = run_scan(proj) + assert gate_decision(secure, Severity.ERROR).tripped is True # baseline cannot clear the gate + trusted = run_scan(proj, trust_suppressions=True) + assert gate_decision(trusted, Severity.ERROR).tripped is False # explicit operator trust diff --git a/tests/unit/scanner/test_grammar_sanitiser_collision.py b/tests/unit/scanner/test_grammar_sanitiser_collision.py new file mode 100644 index 00000000..60b2ecc0 --- /dev/null +++ b/tests/unit/scanner/test_grammar_sanitiser_collision.py @@ -0,0 +1,37 @@ +# tests/unit/scanner/test_grammar_sanitiser_collision.py +"""WLN-CONFIG-SANITISER-SINK-COLLISION — a config sanitiser naming a built-in +serialisation sink can never take effect (the conservative sink override wins), +yet it still counts as "matched", suppressing WLN-CONFIG-UNUSED-SANITISER. The +collision must surface as an explicit config-diagnostic FACT, not a silent no-op. +""" + +from __future__ import annotations + +from wardline.core.finding import Kind, Severity +from wardline.scanner.grammar import build_sanitiser_collision_findings + + +def test_colliding_sanitiser_emits_diagnostic() -> None: + findings = build_sanitiser_collision_findings(("json.loads",)) + assert [f.rule_id for f in findings] == ["WLN-CONFIG-SANITISER-SINK-COLLISION"] + f = findings[0] + assert f.kind is Kind.FACT + assert f.severity is Severity.NONE # diagnostic, not a gate-able defect + assert f.location.path == "weft.toml" # config diagnostics point at the config surface + assert "json.loads" in f.message + assert "serialisation sink" in f.message + assert f.properties["sanitiser"] == "json.loads" + + +def test_non_colliding_sanitiser_is_silent() -> None: + assert build_sanitiser_collision_findings(("mylib.clean",)) == [] + + +def test_empty_config_is_silent() -> None: + assert build_sanitiser_collision_findings(()) == [] + + +def test_multiple_collisions_sorted_with_distinct_fingerprints() -> None: + findings = build_sanitiser_collision_findings(("pickle.loads", "mylib.clean", "json.loads")) + assert [f.properties["sanitiser"] for f in findings] == ["json.loads", "pickle.loads"] + assert len({f.fingerprint for f in findings}) == 2 diff --git a/tests/unit/scanner/test_module_bindings.py b/tests/unit/scanner/test_module_bindings.py new file mode 100644 index 00000000..949b60ca --- /dev/null +++ b/tests/unit/scanner/test_module_bindings.py @@ -0,0 +1,149 @@ +"""Module-level binding channel (wardline-13cfdd7b31 / wardline-66b2c91470). + +Module-scope simple bindings — ``runner = subprocess.run`` (callable alias), +``client = httpx.Client()`` (constructed instance) — are collected per module +onto ``AnalysisContext.module_bindings`` and layered UNDER each function's own +bindings by the sink machinery (:func:`resolved_sink_calls`), closing the +documented v1 module-level false negatives: a module-level callable alias used +in a function now fires PY-WL-112/108, a module-level constructed client fires +PY-WL-117. +""" + +from __future__ import annotations + +import textwrap +from typing import TYPE_CHECKING + +from wardline.core.config import WardlineConfig +from wardline.scanner.analyzer import WardlineAnalyzer + +if TYPE_CHECKING: + from pathlib import Path + +_HEADER = ( + "import os, pickle, subprocess\n" + "import httpx\n" + "from wardline.decorators import external_boundary, trusted\n" + "@external_boundary\n" + "def read_raw(p):\n" + " return p\n" +) + + +def _scan(tmp_path: Path, src: str): + p = tmp_path / "m.py" + p.write_text(_HEADER + textwrap.dedent(src), encoding="utf-8") + analyzer = WardlineAnalyzer() + findings = analyzer.analyze([p], WardlineConfig(), root=tmp_path) + assert analyzer.last_context is not None + return findings, analyzer.last_context + + +def _hits(findings, rule_id: str) -> list[tuple[str, str | None]]: + return [(f.rule_id, f.qualname) for f in findings if f.rule_id == rule_id] + + +def test_context_module_bindings_collects_module_scope_bindings(tmp_path) -> None: + _, ctx = _scan( + tmp_path, + """ + runner = subprocess.run + client = httpx.Client() + + @trusted(level='ASSURED') + def f(p): + return 1 + """, + ) + bindings = ctx.module_bindings["m"] + assert bindings.callable_aliases["runner"] == "subprocess.run" + assert bindings.instance_classes["client"] == "httpx.Client" + + +def test_112_module_level_callable_alias_fires(tmp_path) -> None: + # The exact wardline-13cfdd7b31 repro: module-scope ``runner = subprocess.run`` + # used inside a trusted function with shell=True. + findings, _ = _scan( + tmp_path, + """ + runner = subprocess.run + + @trusted(level='ASSURED') + def f(p): + runner(read_raw(p), shell=True) + """, + ) + assert _hits(findings, "PY-WL-112") == [("PY-WL-112", "m.f")] + + +def test_117_module_level_client_construction_fires(tmp_path) -> None: + findings, _ = _scan( + tmp_path, + """ + client = httpx.Client() + + @trusted(level='ASSURED') + def f(p): + client.get(read_raw(p)) + """, + ) + assert _hits(findings, "PY-WL-117") == [("PY-WL-117", "m.f")] + + +def test_108_module_level_callable_alias_fires(tmp_path) -> None: + findings, _ = _scan( + tmp_path, + """ + sh = os.system + + @trusted(level='ASSURED') + def f(p): + sh(read_raw(p)) + """, + ) + assert _hits(findings, "PY-WL-108") == [("PY-WL-108", "m.f")] + + +def test_106_module_level_callable_alias_fires(tmp_path) -> None: + findings, _ = _scan( + tmp_path, + """ + loader = pickle.loads + + @trusted(level='ASSURED') + def f(p): + return loader(read_raw(p)) + """, + ) + assert _hits(findings, "PY-WL-106") == [("PY-WL-106", "m.f")] + + +def test_function_local_rebind_shadows_module_binding(tmp_path) -> None: + # A function-local rebind to an unresolvable/non-sink value must shadow the + # module-level binding — no false positive on the local ``client``. + findings, _ = _scan( + tmp_path, + """ + client = httpx.Client() + + @trusted(level='ASSURED') + def f(p): + client = object() + client.get(read_raw(p)) + """, + ) + assert _hits(findings, "PY-WL-117") == [] + + +def test_module_binding_clean_argument_does_not_fire(tmp_path) -> None: + findings, _ = _scan( + tmp_path, + """ + client = httpx.Client() + + @trusted(level='ASSURED') + def f(): + client.get('https://example.com') + """, + ) + assert _hits(findings, "PY-WL-117") == [] diff --git a/tests/unit/scanner/test_pipeline.py b/tests/unit/scanner/test_pipeline.py index 79c77eb2..30206ab7 100644 --- a/tests/unit/scanner/test_pipeline.py +++ b/tests/unit/scanner/test_pipeline.py @@ -138,3 +138,77 @@ def test_parse_project_stage_unshadowed_fingerprint_is_bare(tmp_path) -> None: assert result.provider_fingerprint == DecoratorTaintSourceProvider().fingerprint() seed = result.modules[0].seeds["m.f"] assert seed.body_taint == T.INTEGRAL + + +def test_parse_project_stage_records_entity_qualname_config_source_match(tmp_path) -> None: + # An untrusted_sources entry naming a project entity qualname is APPLIED here + # (the seed override below) — the match must be reported back to the analyzer + # so the directive is never misreported as WLN-CONFIG-UNUSED-SOURCE. + path = tmp_path / "m.py" + path.write_text("def get_input():\n return 'x'\n", encoding="utf-8") + result = run_parse_project_stage( + ParseProjectInput( + files=(path,), + root=tmp_path, + provider=DecoratorTaintSourceProvider(), + config=WardlineConfig(untrusted_sources=("m.get_input", "elsewhere.unmatched")), + star_exports=vocabulary_star_exports(), + ) + ) + seed = result.modules[0].seeds["m.get_input"] + assert seed.body_taint == T.EXTERNAL_RAW # the directive took effect... + assert result.matched_config_sources == frozenset({"m.get_input"}) # ...and is recorded + # The unmatched entry is NOT recorded — the unused-source diagnostic stays live. + + +def test_parse_project_stage_parse_failure_is_gating_error_defect(tmp_path) -> None: + # A discovered-but-unparseable file is a gate-eligible ERROR DEFECT (fail-closed: + # unscanned code must not pass the default --fail-on ERROR loop), never a NONE + # FACT. line_start is ALWAYS set (fallback 1) so the lineless-DEFECT downgrade + # in suppression.py cannot demote a no-line encoding failure out of the gate. + from wardline.core.finding import Kind, Severity + + (tmp_path / "syntax.py").write_text("def f(:\n", encoding="utf-8") + (tmp_path / "enc.py").write_bytes(b'# -*- coding: latin-1 -*-\nx = "\xe9"\n') + result = run_parse_project_stage( + ParseProjectInput( + files=(tmp_path / "syntax.py", tmp_path / "enc.py"), + root=tmp_path, + provider=DecoratorTaintSourceProvider(), + config=WardlineConfig(), + star_exports=vocabulary_star_exports(), + ) + ) + by_path = {f.location.path: f for f in result.parse_findings} + assert set(by_path) == {"syntax.py", "enc.py"} + for finding in by_path.values(): + assert finding.rule_id == "WLN-ENGINE-PARSE-ERROR" + assert finding.kind is Kind.DEFECT + assert finding.severity is Severity.ERROR + assert finding.location.line_start is not None + # The syntax error keeps its real line; the encoding error falls back to 1. + assert by_path["syntax.py"].location.line_start == 1 + assert by_path["enc.py"].location.line_start == 1 + + +def test_parse_project_stage_recursion_skip_stays_nongating_fact(tmp_path) -> None: + # The fail-closed change is scoped to PARSE failures: the recursion-limit + # file skip keeps its released non-gating FACT contract (it mirrors + # WLN-ENGINE-FUNCTION-SKIPPED, surfaced via summary.unanalyzed instead). + from wardline.core.finding import Kind, Severity + + expr = "p" + " + p" * 3000 + (tmp_path / "deep.py").write_text(f"def deep(p):\n x = {expr}\n return x\n", encoding="utf-8") + result = run_parse_project_stage( + ParseProjectInput( + files=(tmp_path / "deep.py",), + root=tmp_path, + provider=DecoratorTaintSourceProvider(), + config=WardlineConfig(), + star_exports=vocabulary_star_exports(), + ) + ) + skips = [f for f in result.parse_findings if f.rule_id == "WLN-ENGINE-FILE-SKIPPED"] + assert len(skips) == 1 + assert skips[0].kind is Kind.FACT + assert skips[0].severity is Severity.NONE From 1d8183920bf81c0b45fc00b7d76ffd27e73f4a52 Mon Sep 17 00:00:00 2001 From: John Morrissey Date: Thu, 11 Jun 2026 04:55:36 +1000 Subject: [PATCH 092/186] =?UTF-8?q?feat(rust):=20ADR-049=20Amendments=204+?= =?UTF-8?q?5=20lockstep=20=E2=80=94=20reserved-char=20escape=20on=20generi?= =?UTF-8?q?c=20args=20+=20method-level=20@cfg=20twins;=20re-vendor=2035-ro?= =?UTF-8?q?w=20corpus?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Consumes the Loomweave handoff (docs/federation/2026-06-11-rust-qualname-amendment-4-5 -changeset.md, loomweave side already landed; corpus blob md5 bf8d0996..., git blob 5f3a2fa5...). Amendment 4: every concrete generic arg (type + const, trait fragment + self-type prefix) renders through escape_reserved(strip_ws(arg)) — '%'->'%25' then ':'->'%3A' — and the non-Type::Path self-type fallback (&mut fmt::Formatter) escapes the same way. impl From for Foo -> Foo.impl[From]. The const-spacing gap closed oracle-side (strip_ws now canonical); both KNOWN GAP docstring blocks retired. entity_id's ':' rejection stays strict (escape happens in the producer). Amendment 5: cfg-gated twin METHODS carry their own @cfg() suffix after the method name, keyed on the FINAL impl qualname (post impl-level cfg) + method name and counted across all merged blocks — impl-level cfg-twins get no redundant suffix; method-twins inside a cfg-twin block compose (impl#<>@cfg(unix).go@cfg(a)). Parity gate exercises all 7 new rows (35 total) byte-for-byte; blob pin updated in the same commit per the re-vendor procedure. Suite 3648 green; frozen RS-WL identity corpus unchanged (no colliding/dropped shapes in it, as the letter guarantees). Co-Authored-By: Claude Fable 5 --- src/wardline/rust/index.py | 75 +++++++++++---- src/wardline/rust/qualname.py | 49 +++++----- tests/conformance/qualnames_rust.json | 93 +++++++++++++++++++ .../test_loomweave_rust_qualname_parity.py | 2 +- 4 files changed, 175 insertions(+), 44 deletions(-) diff --git a/src/wardline/rust/index.py b/src/wardline/rust/index.py index b9554004..685355f4 100644 --- a/src/wardline/rust/index.py +++ b/src/wardline/rust/index.py @@ -172,6 +172,24 @@ def _walk_scope( impl_segments[node.id] = seg impl_twin_counts[seg] += 1 + # Method-level cfg-twin counts (ADR-049 Amendment 5): keyed on the FINAL impl + # qualname (post impl-level cfg) + method name, counted across ALL merged blocks — + # so an impl-level cfg-twin (already split into distinct impl entities) gets no + # redundant method suffix, while methods merging across same-key blocks do. + final_impl_quals: dict[int, str] = {} + method_twin_counts: Counter[tuple[str, str]] = Counter() + for node, cfgs in items: + if node.type == "impl_item": + seg = impl_segments.get(node.id) + if seg is None: + continue + if cfgs and impl_twin_counts[seg] > 1: + seg += q.cfg_discriminant(cfgs) + final_qual = f"{module}.{seg}" + final_impl_quals[node.id] = final_qual + for method, _mcfgs in _impl_methods_with_cfgs(node): + method_twin_counts[(final_qual, _name(method))] += 1 + # First block with a given (cfg-augmented) impl qualname emits the ONE merged impl # entity; later same-key blocks only append methods (extract.rs `seen_impl_ids`). # The set is PER-INVOCATION (each nested scope gets a fresh one); that is sound @@ -193,16 +211,13 @@ def _walk_scope( entities.append(_entity(nested, "module", node, nmap, path, parent=module)) _walk_scope(body.children, nested, nmap, entities, path) elif node.type == "impl_item": - seg = impl_segments.get(node.id) - if seg is None: + impl_qualname = final_impl_quals.get(node.id) + if impl_qualname is None: continue - if cfgs and impl_twin_counts[seg] > 1: - seg += q.cfg_discriminant(cfgs) - impl_qualname = f"{module}.{seg}" if impl_qualname not in seen_impl_quals: seen_impl_quals.add(impl_qualname) entities.append(_entity(impl_qualname, "impl", node, nmap, path, parent=module)) - _emit_impl_methods(node, impl_qualname, nmap, entities, path) + _emit_impl_methods(node, impl_qualname, nmap, entities, path, method_twin_counts) else: kind = _LEAF_KINDS[node.type] name = _name(node) @@ -241,19 +256,47 @@ def _impl_segment(impl_node: Node) -> str | None: return f"{self_type}.impl#<{q.render_positional_generics(impl_node)}>" -def _emit_impl_methods( - impl_node: Node, impl_qualname: str, nmap: NodeIdMap, entities: list[RustEntity], path: str -) -> None: +def _impl_methods_with_cfgs(impl_node: Node) -> list[tuple[Node, list[str]]]: + """``(function_item, raw cfg predicates)`` pairs of an impl body, with the SAME + pending-cfg accumulation discipline as ``_walk_scope`` (comments transparent, + attributes accumulate, any other node resets).""" body = impl_node.child_by_field_name("body") if body is None: - return - for child in body.named_children: + return [] + out: list[tuple[Node, list[str]]] = [] + pending_cfgs: list[str] = [] + for child in body.children: + if child.type in _COMMENT_TYPES: + continue + if child.type == "attribute_item": + pred = q.cfg_predicate_of(child) + if pred is not None: + pending_cfgs.append(pred) + continue if child.type == "function_item": - # Methods re-parent onto the impl ENTITY (module -> impl -> method), and the - # method qualname builds from the cfg-AUGMENTED impl qualname (extract.rs). - entities.append( - _entity(f"{impl_qualname}.{_name(child)}", "method", child, nmap, path, parent=impl_qualname) - ) + out.append((child, pending_cfgs)) + pending_cfgs = [] + return out + + +def _emit_impl_methods( + impl_node: Node, + impl_qualname: str, + nmap: NodeIdMap, + entities: list[RustEntity], + path: str, + method_twin_counts: Counter[tuple[str, str]], +) -> None: + for child, mcfgs in _impl_methods_with_cfgs(impl_node): + # Methods re-parent onto the impl ENTITY (module -> impl -> method), and the + # method qualname builds from the cfg-AUGMENTED impl qualname (extract.rs). + # A cfg-gated TWIN method (same final impl qualname + name, counted across + # merged blocks) carries its own @cfg suffix (ADR-049 Amendment 5). + name = _name(child) + qualname = f"{impl_qualname}.{name}" + if mcfgs and method_twin_counts[(impl_qualname, name)] > 1: + qualname += q.cfg_discriminant(mcfgs) + entities.append(_entity(qualname, "method", child, nmap, path, parent=impl_qualname)) def _name(node: Node) -> str: diff --git a/src/wardline/rust/qualname.py b/src/wardline/rust/qualname.py index 9b461923..ee83c7b0 100644 --- a/src/wardline/rust/qualname.py +++ b/src/wardline/rust/qualname.py @@ -5,30 +5,17 @@ (``tests/conformance/qualnames_rust.json``) is the byte-for-byte oracle. ``:`` is the reserved separator and ``[ ] # < > @ $`` are legal segments of the dialect. -KNOWN GAP (path-typed generic args): a trait OR self-type concrete generic arg that is itself -a ``::``-path renders a segment containing ``:`` — both in the trait fragment -(``impl From`` -> ``...impl[From].from``) and, since the -self-type-args amendment (ADR-049 §2), in the self-type prefix (``impl Foo`` --> ``...Foo.impl#<>...``). Loomweave renders the BYTE-IDENTICAL segment (its -``trait_generic_args`` / ``self_ty_locator`` keep ``::`` via ``strip_ws`` — the cfg-only -``escape_reserved`` does NOT cover generic args), then REJECTS the assembled locator at -``entity_id`` construction (``validate_no_colon``) and degrades the whole file. Wardline is -faithful in *rendering* but lacks that validate-and-degrade gate, so it currently emits a -``:``-bearing locator. The correct fix is an ADR-049 amendment defining a colon-free canonical -form for path-typed generic args, adopted by both producers in lockstep — a Wardline-only -normalization would itself diverge from the (still-unreleased) oracle. The frozen Rust -identity corpus (``tests/golden/identity/rust/``) deliberately contains no path-typed -generic args, so graduation does not pre-empt the pending cross-tool decision. -Tracked: see the ``rust-bug-hunt-2026-06-09`` reserved-colon issue and the 2026-06-10 -ADR-049 amendment-request letter. - -KNOWN GAP (const-generic-arg spacing): a multi-token *const* generic arg (``Foo<{N + 1}>``, -``Foo<-1>``) is rendered by the oracle via ``to_token_stream().to_string()`` — proc-macro2 -CANONICAL spacing (``{ N + 1 }``), NOT whitespace-stripped — whereas Wardline ``_strip_ws``-es -every arg (``{N+1}``). Same impossibility class as the reserved-colon gap: matching would mean -reimplementing proc-macro2 token spacing, so the right fix is a lockstep ADR-049 amendment -(cleanest: the oracle ``strip_ws``-es const args too). Out-of-corpus, Tier-B; plain const args -(``Foo<3>``, ``Foo``, a bare const-param ident) already match byte-for-byte. +ADR-049 AMENDMENT 4 (2026-06-11, implemented): every CONCRETE generic arg — type or const, +in both the trait fragment and the self-type prefix — renders through +``escape_reserved(strip_ws(arg))``: whitespace-stripped first (the oracle now strips const +args too, closing the proc-macro2-spacing gap), then the injective reserved-char escape +(``%`` -> ``%25``, then ``:`` -> ``%3A``). ``impl From for Foo`` -> +``Foo.impl[From]``; ``impl Foo<{ 1 + 2 }>`` -> ``Foo<{1+2}>.impl#<>``. +The same escape covers the NON-``Type::Path`` self-type fallback (reference/tuple/slice/ptr: +``impl Serializer for &mut fmt::Formatter`` -> ``&mutfmt%3A%3AFormatter``). The escape happens +in the producer BEFORE the id is assembled — ``entity_id``'s ``:`` rejection stays strict. +Corpus rows: ``path_typed_generic_arg_trait``/``_inherent``, ``const_generic_arg_spacing``, +``reference_self_type_path_escape``. This module holds the pure string/CST-node renderers; ``index.py`` walks the tree and assembles full qualnames from them. tree-sitter types appear only under ``TYPE_CHECKING`` @@ -282,7 +269,10 @@ def render_self_type(type_node: Node, type_params: Sequence[str]) -> str: return f"{base}<{','.join(rendered)}>" if rendered else base if type_node.type in ("type_identifier", "scoped_type_identifier"): return _last_path_segment(type_node) - return _strip_ws(type_node) + # Non-Type::Path fallback (reference/tuple/slice/raw-pointer self types): may carry a + # ``::``-path (`&mut fmt::Formatter`), so it escapes like a concrete generic arg + # (ADR-049 Amendment 4 self-type-fallback completion). A `:`-free fallback is unchanged. + return _escape_reserved(_strip_ws(type_node)) def _self_type_arg(arg_node: Node, type_params: Sequence[str]) -> str: @@ -295,7 +285,7 @@ def _self_type_arg(arg_node: Node, type_params: Sequence[str]) -> str: name = _text(arg_node) if name in type_params: return f"${type_params.index(name)}" - return _strip_ws(arg_node) + return _escape_reserved(_strip_ws(arg_node)) def render_trait_segment(trait_node: Node) -> str: @@ -312,7 +302,12 @@ def render_trait_segment(trait_node: Node) -> str: args = ( # Drop dropped-kinds AND empty-rendering args (a malformed `From<>` error-recovers # to a blank arg; valid Rust never yields one) -> bare trait name, no empty `<>`. - [s for c in targs.named_children if c.type not in _DROPPED_GENERIC_ARGS if (s := _strip_ws(c))] + [ + _escape_reserved(s) + for c in targs.named_children + if c.type not in _DROPPED_GENERIC_ARGS + if (s := _strip_ws(c)) + ] if targs is not None else [] ) diff --git a/tests/conformance/qualnames_rust.json b/tests/conformance/qualnames_rust.json index ed436c82..5f3a2fa5 100644 --- a/tests/conformance/qualnames_rust.json +++ b/tests/conformance/qualnames_rust.json @@ -395,6 +395,99 @@ {"qualname": "demo.m.g@cfg(any(unix,windows))", "kind": "function"}, {"qualname": "demo.m.g@cfg(target_os=\"macos\")", "kind": "function"} ] + }, + { + "name": "path_typed_generic_arg_trait", + "crate": "demo", "module_path": "demo.m", "rel_path": "src/m.rs", + "reproducibility": "slice-1", + "note": "ADR-049 Amendment 4 (clarion-8245039f6b): a concrete trait generic arg that is itself a ::-path (impl From) carries the reserved ':' separator. Without escaping, build_entity_id rejects the id and degraded_aware collapses the WHOLE cleanly-parsed file to one syntax_error module. Every concrete generic arg renders through escape_reserved(strip_ws(arg)) — the same injective escape the cfg path pins — so std::io::Error renders std%3A%3Aio%3A%3AError in the trait fragment. Pins that a Wardline trait-impl extractor escapes path-typed generic args byte-for-byte.", + "source": "struct Foo;\nimpl From for Foo { fn from(_: std::io::Error) -> Self { Foo } }\n", + "expected": [ + {"qualname": "demo.m", "kind": "module"}, + {"qualname": "demo.m.Foo", "kind": "struct"}, + {"qualname": "demo.m.Foo.impl[From]", "kind": "impl"}, + {"qualname": "demo.m.Foo.impl[From].from", "kind": "function"} + ] + }, + { + "name": "path_typed_generic_arg_inherent", + "crate": "demo", "module_path": "demo.m", "rel_path": "src/m.rs", + "reproducibility": "slice-1", + "note": "ADR-049 Amendment 4 (clarion-8245039f6b): the path-typed concrete generic arg in the SELF-TYPE prefix (impl Foo) is escaped through the same escape_reserved(strip_ws(arg)) pipeline, so Foo renders Foo rather than dropping the file. Distinct paths stay distinct (injective).", + "source": "struct Foo(T);\nimpl Foo { fn get(&self) {} }\n", + "expected": [ + {"qualname": "demo.m", "kind": "module"}, + {"qualname": "demo.m.Foo", "kind": "struct"}, + {"qualname": "demo.m.Foo.impl#<>", "kind": "impl"}, + {"qualname": "demo.m.Foo.impl#<>.get", "kind": "function"} + ] + }, + { + "name": "const_generic_arg_spacing", + "crate": "demo", "module_path": "demo.m", "rel_path": "src/m.rs", + "reproducibility": "slice-1", + "note": "ADR-049 Amendment 4 (clarion-8245039f6b): GenericArgument::Const was the dialect's one whitespace-bearing render (proc-macro2 token spacing: `{ 1 + 2 }`), unreproducible from a tree-sitter frontend. Amendment 4 routes const args through the SAME strip_ws as type args, yielding {1+2}. Pins that a second producer renders const generic args whitespace-free, not proc-macro2-spaced.", + "source": "struct Foo;\nimpl Foo<{ 1 + 2 }> { fn get(&self) {} }\n", + "expected": [ + {"qualname": "demo.m", "kind": "module"}, + {"qualname": "demo.m.Foo", "kind": "struct"}, + {"qualname": "demo.m.Foo<{1+2}>.impl#<>", "kind": "impl"}, + {"qualname": "demo.m.Foo<{1+2}>.impl#<>.get", "kind": "function"} + ] + }, + { + "name": "method_cfg_twin_inherent", + "crate": "demo", "module_path": "demo.m", "rel_path": "src/m.rs", + "reproducibility": "slice-1", + "note": "ADR-049 Amendment 5 (clarion-dfeb905f46): two cfg-gated twin METHODS inside ONE inherent impl block. The impl-level @cfg cannot split them (one impl key, demo.m.Foo.impl#<>), so each method carries its OWN @cfg(...) suffix AFTER the method name (.go@cfg(unix)), mirroring the free-item rule. Without it both `go` render demo.m.Foo.impl#<>.go and the writer's ON CONFLICT silently keeps one. Pins that a second producer applies a method-level cfg suffix, not only an impl-level one.", + "source": "struct Foo;\nimpl Foo { #[cfg(unix)] fn go(&self){} #[cfg(windows)] fn go(&self){} }\n", + "expected": [ + {"qualname": "demo.m", "kind": "module"}, + {"qualname": "demo.m.Foo", "kind": "struct"}, + {"qualname": "demo.m.Foo.impl#<>", "kind": "impl"}, + {"qualname": "demo.m.Foo.impl#<>.go@cfg(unix)", "kind": "function"}, + {"qualname": "demo.m.Foo.impl#<>.go@cfg(windows)", "kind": "function"} + ] + }, + { + "name": "method_cfg_twin_trait", + "crate": "demo", "module_path": "demo.m", "rel_path": "src/m.rs", + "reproducibility": "slice-1", + "note": "ADR-049 Amendment 5 (clarion-dfeb905f46): cfg-gated twin methods inside ONE trait impl block. The method-level @cfg suffix lands after the method name on the trait-impl key (demo.m.Foo.impl[Tr].go@cfg(unix)). Pins that the method-level cfg discriminant applies to trait impls too, not only inherent ones.", + "source": "struct Foo;\nimpl Tr for Foo { #[cfg(unix)] fn go(&self){} #[cfg(windows)] fn go(&self){} }\n", + "expected": [ + {"qualname": "demo.m", "kind": "module"}, + {"qualname": "demo.m.Foo", "kind": "struct"}, + {"qualname": "demo.m.Foo.impl[Tr]", "kind": "impl"}, + {"qualname": "demo.m.Foo.impl[Tr].go@cfg(unix)", "kind": "function"}, + {"qualname": "demo.m.Foo.impl[Tr].go@cfg(windows)", "kind": "function"} + ] + }, + { + "name": "method_cfg_twin_cross_merged_block", + "crate": "demo", "module_path": "demo.m", "rel_path": "src/m.rs", + "reproducibility": "slice-1", + "note": "ADR-049 Amendment 5 (clarion-dfeb905f46), cross-merged-block variant: two SEPARATE inherent impl blocks on the same (type, sig, no-cfg) MERGE into one impl entity (Option b), each contributing a `go`. The merged `go` methods would collide unless the method-twin count spans ALL blocks sharing the final impl key (demo.m.Foo.impl#<>), not just one block. Pins that the method cfg discriminant is computed across merged blocks.", + "source": "struct Foo;\nimpl Foo { #[cfg(unix)] fn go(&self){} }\nimpl Foo { #[cfg(windows)] fn go(&self){} }\n", + "expected": [ + {"qualname": "demo.m", "kind": "module"}, + {"qualname": "demo.m.Foo", "kind": "struct"}, + {"qualname": "demo.m.Foo.impl#<>", "kind": "impl"}, + {"qualname": "demo.m.Foo.impl#<>.go@cfg(unix)", "kind": "function"}, + {"qualname": "demo.m.Foo.impl#<>.go@cfg(windows)", "kind": "function"} + ] + }, + { + "name": "reference_self_type_path_escape", + "crate": "demo", "module_path": "demo.m", "rel_path": "src/m.rs", + "reproducibility": "slice-1", + "note": "ADR-049 Amendment 4 (self-type fallback completion, clarion-8245039f6b): a non-Type::Path self type (here a reference) that carries a ::-path falls through self_ty_locator's type_textual fallback. Without escaping it leaks a raw ':' and build_entity_id rejects the id, degrading the whole file (the serde ser/fmt.rs `impl Serializer for &mut fmt::Formatter` drop). The fallback now routes through the same escape_reserved(strip_ws(..)) pipeline as concrete generic args, so &core::cell::Cell renders &core%3A%3Acell%3A%3ACell. A ':'-free reference (&Foo) is unchanged (escape is a no-op).", + "source": "impl Tr for &core::cell::Cell { fn ok(&self) {} }\n", + "expected": [ + {"qualname": "demo.m", "kind": "module"}, + {"qualname": "demo.m.&core%3A%3Acell%3A%3ACell.impl[Tr]", "kind": "impl"}, + {"qualname": "demo.m.&core%3A%3Acell%3A%3ACell.impl[Tr].ok", "kind": "function"} + ] } ] } diff --git a/tests/conformance/test_loomweave_rust_qualname_parity.py b/tests/conformance/test_loomweave_rust_qualname_parity.py index 7b764e83..8d1bea55 100644 --- a/tests/conformance/test_loomweave_rust_qualname_parity.py +++ b/tests/conformance/test_loomweave_rust_qualname_parity.py @@ -100,7 +100,7 @@ # The git blob hash of the vendored corpus as committed upstream (loomweave rc4 # @ cab95a1695a45f875933d8c4ac0e800e793c9305). Re-vendors update this constant in # the SAME commit as the new bytes — see the RE-VENDOR PROCEDURE in the header. -UPSTREAM_BLOB_SHA = "ed436c825861ad2b9e313f9211f5a55583b80c7c" +UPSTREAM_BLOB_SHA = "5f3a2fa52aa43ddbc7adbb4f84c9282ee1ca1cf8" _KNOWN_TIERS = {"slice-1", "sp2"} # The a209fc7 corpus carries the FULL ten-kind ADR-049 surface (leaf_item_kinds / From 71866d6420c1c3f676816f915f977b371abb8a21 Mon Sep 17 00:00:00 2001 From: John Morrissey Date: Thu, 11 Jun 2026 05:00:51 +1000 Subject: [PATCH 093/186] =?UTF-8?q?feat(ci):=20enhance=20deployment=20proc?= =?UTF-8?q?ess=20=E2=80=94=20assemble=20combined=20site=20tree=20for=20gh-?= =?UTF-8?q?pages=20and=20update=20.gitignore=20for=20CI=20artifacts?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- .github/workflows/ci.yml | 35 +++++++- .gitignore | 3 + .../05-quality-assessment.md | 83 +++++++++++++++++++ docs/index.md | 13 +-- mkdocs.yml | 2 +- 5 files changed, 127 insertions(+), 9 deletions(-) create mode 100644 docs/arch-analysis-2026-06-10/05-quality-assessment.md diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 965ac16a..1448af89 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -189,5 +189,36 @@ jobs: enable-cache: true python-version: "3.13" - run: uv sync --extra docs - - name: Deploy - run: uv run mkdocs gh-deploy --force + # The www/ front door owns the site root (and the CNAME); the MkDocs + # reference docs are served from the /docs/ subpath. `mkdocs gh-deploy` + # can only publish the bare MkDocs site, so assemble the combined tree + # by hand and force-push it to gh-pages (keeps the Pages source as the + # gh-pages branch — no repo-settings change). Dependency-free, matching + # Wardline's zero-dep ethos. + - name: Assemble site (www root + docs/ subpath) + run: | + set -euo pipefail + rm -rf publish + cp -r www/. publish/ + # site_url is pinned to .../docs/ in mkdocs.yml, so internal links + # and the canonical resolve under the subpath. --strict matches the + # docs-build gate so the deployed build can't silently differ. + uv run mkdocs build --strict -d "$PWD/publish/docs" + # CNAME has a single source of truth: www/CNAME (copied above to + # publish/CNAME). Fail loudly if it didn't make it to the root. + test -f publish/CNAME + touch publish/.nojekyll + - name: Deploy to gh-pages + env: + GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} + run: | + set -euo pipefail + cd publish + git init -q + git checkout -q -b gh-pages + git add -A + git -c user.email=ci@foundryside.dev -c user.name="wardline-ci" \ + commit -qm "deploy: www root + /docs/ (${GITHUB_SHA})" + git push -f \ + "https://x-access-token:${GITHUB_TOKEN}@github.com/${GITHUB_REPOSITORY}.git" \ + gh-pages diff --git a/.gitignore b/.gitignore index 886f2ea6..9b38f3be 100644 --- a/.gitignore +++ b/.gitignore @@ -56,3 +56,6 @@ wardline.yaml # git worktrees (isolated feature workspaces) .worktrees/ + +# CI-only assembled Pages tree (www root + docs/) +publish/ diff --git a/docs/arch-analysis-2026-06-10/05-quality-assessment.md b/docs/arch-analysis-2026-06-10/05-quality-assessment.md new file mode 100644 index 00000000..a287a4d7 --- /dev/null +++ b/docs/arch-analysis-2026-06-10/05-quality-assessment.md @@ -0,0 +1,83 @@ +# Architecture Quality Assessment — High-Risk Areas + +**Source:** Four-lens parallel specialist review (security / engine soundness / architecture / verification), each grounded in direct file reads of the working tree at `rc5` @ `21aeffaa`. +**Assessed:** 2026-06-10 +**Assessor:** architecture-critic synthesis (panel: threat-analyst, python-code-reviewer, architecture-critic, coverage-gap-analyst) + +## Executive Summary + +The codebase's security architecture is strong — THREAT-001 path confinement holds across every new MCP tool, the secure-by-default gate population is structurally enforced, and rekey cannot launder verdicts. The highest risks are elsewhere: **the CI self-hosting scan has no gate** (wardline does not fail its own build on its own findings), **active false-negative soundness gaps in the taint engine's chokepoint** (`_resolve_expr`/`_resolve_call` and loop merging), and **a structural drift seam** (un-layered `core/` held together by 158 deferred imports; the Rust frontend bolted alongside Python with no frontend abstraction). 2 Critical, 8 High findings. + +## Findings by Severity + +### Critical + +1. **CI self-hosting scan is SARIF-upload only — no gate** — verified: `.github/workflows/ci.yml:76` runs `wardline scan src/ --format sarif` with no `--fail-on`; the step succeeds regardless of findings. `tests/test_self_hosting.py` is additionally vacuous by construction (no decorators in own source → tier-gated rules never fire). + - **Impact:** wardline can ship a trust-boundary violation in itself with a green build — direct credibility risk for a trust-boundary gate product. + - **Fix:** add `--fail-on ERROR` to the CI step; add an annotated fixture module so self-scan exercises the real pipeline. Effort: S. + +2. **All live wire-contract oracles are weekly/manual only** — verified: `pyproject.toml:121` addopts excludes `loomweave_e2e`/`legis_e2e`/`filigree_e2e`/`rust_e2e`; the live job is `schedule`/`workflow_dispatch`-gated. Hermetic pins (legis key-set freeze, loomweave HMAC golden vector) do run in PR CI, but a sibling-side wire change surfaces only at the next weekly run. Known ticket `wardline-79ba05f464` (G6) covers the SEI-oracle half. + - **Impact:** wire-breaking drift across the federation seams merges green and fails up to a week later. + - **Fix:** fail-closed weekly job + a required `WARDLINE_LIVE_ORACLE_REQUIRED=1` escape hatch; land G6. Effort: M. + +### High — Engine soundness (the product's core value) + +3. **Zero-trip loop FN is real and the merge structure confirms it** (`scanner/taint/variable_level.py:1350–1394`): the post-loop state never re-merges `pre_loop` as a fallthrough arm, so a loop that *cleans* a RAW variable hides the zero-iteration path where it stayed RAW. Matches open bug `wardline-d6af917bde` (currently P3 — under-prioritized for a soundness FN). Fix is localized: combine post-loop state with `pre_loop`. Effort: S. + +4. **Raw-receiver `taint_map` bypass** (`variable_level.py:740–828`): for attribute calls, the `taint_map` hit returns early *before* the RAW_ZONE receiver guard, so `raw_obj.trusted_method()` can launder taint when the receiver's type isn't tracked. Same family as the scrub's PY-WL-105/stale-var_types cluster. Fix: apply/combine the receiver guard before the `taint_map` lookup. Effort: S–M. *(Confidence: Moderate — trigger needs an untyped raw receiver; confirm with a unit test first.)* + +5. **`collect_attribute_writes` is flow-insensitive with its own uncoordinated `var_types`** (`variable_level.py:1617–1703`): resolves RHS taint against final-state `var_taints`, not per-statement snapshots; branch-stale types can dispatch to `@trusted` summaries for now-raw receivers. *(Confidence: Moderate — the L2 fixed-point re-run in `analyzer.py:476` may partially compensate; verify empirically.)* + +### High — Structure + +6. **`core/` is un-layered: real import cycles + an engine→policy inversion, masked by 158 function-local deferred imports.** Verified cycles include `core.run → scanner.analyzer → … → core.attest → core.assure → core.run`; the inversion is `scanner/taint/project_resolver.py:143` importing `core.attest.ruleset_hash` (taint engine depending on the attestation layer). Every refactor near these modules risks surfacing masked breakage. Fix: `import-linter` contracts in report-only mode, then move `ruleset_hash` down a tier. Effort: M. + +7. **Rust frontend is a parallel vertical, not a plugged-in frontend.** Only `Finding`/`TaintState` are shared (`core/protocols.py:17`); context, rules, vocabulary, dataflow, qualname, and Finding-assembly are all reimplemented under `rust/`. A third language costs a third full vertical, and rule/severity/identity semantics will drift between the two rule trees. Fix: lift a `LanguageFrontend` interface before any third language. Effort: L. + +8. **MCP-vs-CLI surface drift seam is structural.** The `run_scan` spine is genuinely shared (strength), but the federation-status envelope is duplicated (`mcp/server.py:73,94` vs `cli/scan.py:450,480`) — exactly where the two dogfood drift incidents occurred — and `_register_tools` (`server.py:822–1271`) is a 450-line change-magnet. Fix: one shared status projector + per-tool schema declarations. Effort: M. + +### High — Verification gaps in trust-critical paths + +9. **Rekey adversarial scenarios untested**: mixed-scheme partial-migration state (one leg done, source changed before resume) and multi-store rollback (only single-store restore is tested, `tests/unit/core/test_rekey_rollback.py` has 2 tests). Rekey moves user trust decisions; rollback is the recovery path. Effort: S–M. + +10. **Tier-suppression negative tests absent**: no unit tests assert rules do NOT fire below the tier gate (e.g. `UNKNOWN_RAW` context); a silently loosened gate would only be caught if the pattern happens to exist in the labeled corpus. Effort: S. + +### Medium + +- **Symlinked `.env`/federation-token reads bypass `safe_project_file`** (`filigree/config.py:49,68`, `core/judge_run.py:67` — vs `attest_key.py:28`/`legis.py:143` which do it right): an attacker-authored repo can symlink `.env` out of root and wardline sends the first matching line as a bearer token to the configured sibling URL. The one security finding warranting a code change. Effort: S. +- **Judge prompt-injection surface is structural but contained**: attacker-authored source reaches the LLM; an injected FALSE_POSITIVE verdict feeds `judged.yaml` — but the secure-default gate ignores judged without `--trust-suppressions`, so no silent gate clear. Keep that invariant; document advisory status. +- **Rust qualname corpus has no drift alarm**: vendored byte-pinned at a loomweave blob; ADR-049 has already moved three times, each needing a manual re-vendor with no CI cross-check. Plus the known reserved-colon locator bug (`wardline-be5ee9cc34`) emits invalid locators with no degrade gate → fingerprint churn on the eventual fix. +- **Rust `write!`/`writeln!`/`format_args!` not modelled** in `rust/dataflow.py:202–214` (`format!` only) — tainted format strings through writers don't fire. +- **Three federation clients hand-roll the urllib transport independently** (`core/filigree_emit.py:229`, `filigree/dossier_client.py:50`, `loomweave/client.py`); only `read_response_text` is shared. Auth ladders should stay separate; the transport should not. +- **Corpus FP-rate gate has zero FALSE_POSITIVE-labeled entries** — the 5% budget math is never exercised against real corpus data. + +### Low (selected) + +- `resolve_under_root` is escape-rejecting but not symlink-refusing (unlike `safe_project_file`) — fine for today's read-only consumers; document the contract before any write path uses it. +- `attest`/`verify_attestation` is sound (timing-safe compare, schema/key_id binding); missing-`schema`-key and non-dict-payload edges untested. +- L2 loop-convergence backstop truncates silently (`variable_level.py:1362,1393`) with no `WLN-ENGINE-*` diagnostic, unlike the L3 bound. +- `time.sleep(0.1)` polling in `tests/e2e/test_loomweave_live.py:108,121`. + +## Cross-Cutting Concerns + +**Security:** Strong posture. Path confinement uniform across all MCP tools (including args never opened); secure-by-default gate population architecturally enforced (`run.py:87,298–301`); rekey carry keyed on finding-derived fingerprints (no laundering primitive); `install/block.py` injector hardened; secrets never read from `weft.toml`. The residual real item is the symlink token-read inconsistency. + +**Correctness:** The lattice discipline (`combine` vs `taint_join`) is rigorous and the L3 kernel monotone-guarded, but FN risk concentrates in `_resolve_expr`/`_resolve_call` (`variable_level.py:449,647`) — the shared chokepoint where the historical bug record also clusters. That chokepoint, not the file's 1,885-line length, is the real change-magnet; don't split the file for size's sake, invest in differential/property tests around the chokepoint. + +**Maintainability:** `core/paths.py` is a genuinely clean single source of truth for stores/config; the zero-dep constraint is well-contained. The debts are the un-layered `core/`, the duplicated Rust vertical, and the duplicated surface envelopes. + +## Priority Recommendations + +1. **Gate the CI self-scan** (`--fail-on ERROR` + non-vacuous fixture) — Critical, Effort S. A trust-gate product that doesn't gate itself is the cheapest, highest-credibility fix available. +2. **Fix the zero-trip loop FN** (re-merge `pre_loop`) and **confirm/fix the raw-receiver `taint_map` bypass** — Critical-class soundness in the core engine, Effort S each. Re-prioritize `wardline-d6af917bde` above P3. +3. **Close the symlinked token-read gap** (route filigree/judge `.env`+token reads through `safe_project_file` + regression test) — Medium severity, Effort S, restores the codebase's own established discipline. +4. **Land `import-linter` contracts (report-only) + fix the `project_resolver → core.attest` inversion** — High, Effort M; unblocks all future structural work. +5. **Rekey adversarial tests** (mixed-scheme partial, multi-store rollback) + **tier-suppression negative tests** — High, Effort S–M; protects user trust decisions. +6. **Before a third language: `LanguageFrontend` interface** — High, Effort L; the one item that caps the product ceiling. + +## Limitations + +- High-level risk review, not an exhaustive audit. Not reviewed: `core/triage.py`, `core/source_excerpt.py`, full `install/*`, `loomweave/client.py` send-side internals, LSP beyond delegation check. +- Engine findings 4–5 are structurally verified but not empirically reproduced — write the confirming unit tests before fixing (`wardline-d6af917bde`'s pattern: naive fixes here have regressed before). +- Loomweave index was empty (`never_analyzed`); the import-cycle graph was hand-built via AST and should be cross-checked after `loomweave analyze .`. +- No tests were executed; severity ratings assume the documented threat model (attacker authors repo content scanned by wardline). diff --git a/docs/index.md b/docs/index.md index 2d2e83f7..ca2c78c8 100644 --- a/docs/index.md +++ b/docs/index.md @@ -1,9 +1,10 @@ ---- -template: home.html -hide: - - navigation - - toc ---- +# Wardline + +Generic, lightweight semantic-tainting static analyzer for Python — track +untrusted data across your codebase and gate trust-boundary violations, with +zero runtime dependencies. The product front door lives at +[wardline.foundryside.dev](https://wardline.foundryside.dev/); these are the +reference docs. ## Install diff --git a/mkdocs.yml b/mkdocs.yml index 063d97c2..0270fb3d 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -1,6 +1,6 @@ site_name: Wardline site_description: Generic semantic-tainting static analyzer for Python -site_url: https://wardline.foundryside.dev +site_url: https://wardline.foundryside.dev/docs/ repo_url: https://github.com/foundryside-dev/wardline repo_name: foundryside-dev/wardline edit_uri: edit/main/docs/ From 5d6b534de9c3e647e524a018483a3dddf91af83e Mon Sep 17 00:00:00 2001 From: John Morrissey Date: Thu, 11 Jun 2026 08:10:07 +1000 Subject: [PATCH 094/186] =?UTF-8?q?feat(rust):=20ADR-049=20Amendments=206-?= =?UTF-8?q?9=20lockstep=20=E2=80=94=20residual-collision=20ladder=20+=20#[?= =?UTF-8?q?path]=20mount=20overlay=20+=20const=20=5F=20skip;=20batched=204?= =?UTF-8?q?-9=20re-vendor=20(63-row=20corpus)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Consumes the Loomweave gold-v2 handoff (loomweave docs/federation/ 2026-06-11-rust-qualname-amendment-6-9-changeset.md @ 113c2e2; corpus md5 a784a2f9..., git blob d81fb975... — ONE re-vendor covering Amendments 4-9: 49 entity rows + 6 module_route rows + the NEW module_mounts section, 8 rows). Closes all four Sprint-4 gold-blocker collision families on the Wardline side. Amendments 6+7 (closes clarion-8ff7f233fa / clarion-fa8bcf8731 families): impl qualnames now decided by the residual-collision LADDER — @cfg (unchanged bare-key counting) -> stage S (post-cfg groups with >=2 distinct self-type written-path witnesses re-render qself-free Type::Path bases as the escaped written path; impl T for a::X / b::X -> a%3A%3AX.impl[T] / b%3A%3AX.impl[T]) -> stage T (post-S groups with >=2 distinct trait written paths qualify EVERY member's impl[...] fragment; Compat<$0>.impl[a%3A%3AAsyncRead]) -> method-@cfg (Amendment 5, re-keyed on the final post-S/T qualname). Twin-gated end to end: only already-colliding ids change; frozen RS-WL identity corpus byte-identical. New qualname primitives: written_path_of / self_type_witness / trait_written_path / render_trait_segment_qualified / render_self_type_parts; qself-bearing self types now take the A4 fallback per the Amendment-6 text. Amendment 8 (closes clarion-bdb1eccf48 family): new wardline.rust.mounts — literal '#[path = ...] mod name;' mount discovery (rustc relative-path rule incl. inline-mod nesting), memoized fixed point (chains; cycles -> filesystem fallback; doubly-claimed targets first-wins by sorted (declaring-file, offset), the R5 pin), cross-form @cfg twin rule + cfg-twin-inline-mod prefix composition; macro-wrapped and cfg_attr-delivered mounts invisible by dialect rule. analyzer.analyze builds one overlay per crate from scanned in-src sources (files read once); rust_module_route stays the pure-filesystem default (path_attr_known_gap de-gapped to a FALLBACK pin). Identity capture mirrors the overlay. Amendment 9 (closes clarion-83870dc534 family): const _ is not an entity — skipped unconditionally at the free-item arm (the ordered-equality parity gate self-enforces via the ABSENT expected rows). Also: docs/integration/2026-06-11-wardline-resolve-plugin-hint-proposal.md — the §7 ResolveRequest plugin-hint shape agreement (optional batch-scoped 'plugin' field; loomweave lands first under deny_unknown_fields; wardline client slice tracked as wardline-7eccab28ee). Parity: all 49 entity + 6 route + 8 mount rows byte-green; UPSTREAM_BLOB_SHA bumped in the same commit per the re-vendor procedure. Suite 3678 green, ruff/format/mypy clean, self-scan gate exit 0, mkdocs --strict green. Filigree: wardline-803ff6b567 (A6), wardline-5d1485f381 (A7), wardline-da43ecb631 (A8), wardline-a16f283feb (A9), wardline-7eccab28ee (plugin hint, open). Co-Authored-By: Claude Fable 5 --- CHANGELOG.md | 23 ++ ...1-wardline-resolve-plugin-hint-proposal.md | 72 ++++ src/wardline/rust/analyzer.py | 60 +++- src/wardline/rust/index.py | 170 ++++++--- src/wardline/rust/mounts.py | 268 ++++++++++++++ src/wardline/rust/qualname.py | 145 ++++++-- tests/conformance/qualnames_rust.json | 327 +++++++++++++++++- .../test_loomweave_rust_qualname_parity.py | 54 ++- tests/golden/identity/rust/_capture.py | 10 +- tests/unit/rust/test_crate_roots.py | 11 +- tests/unit/rust/test_index.py | 32 ++ tests/unit/rust/test_mounts.py | 92 +++++ 12 files changed, 1171 insertions(+), 93 deletions(-) create mode 100644 docs/integration/2026-06-11-wardline-resolve-plugin-hint-proposal.md create mode 100644 src/wardline/rust/mounts.py create mode 100644 tests/unit/rust/test_mounts.py diff --git a/CHANGELOG.md b/CHANGELOG.md index 39f96260..83df87fd 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -158,6 +158,29 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 through cleanly to the legacy/off rungs (emit stays soft-fail) (weft-23574069a1). ### Changed +- **Rust qualname dialect: ADR-049 Amendments 6–9 (Loomweave lockstep, one + batched re-vendor covering 4–9).** Closes the four Sprint-4 gold-blocker + collision families at the dialect level, mirroring the authoritative + Loomweave producer byte-for-byte (49-entity corpus + the new `module_mounts` + section, blob `d81fb975…`): + - **Amendments 6+7 — the residual-collision ladder** (`@cfg` → stage S + self-type written path → stage T trait written path → method-`@cfg`): + same-key impl twins split on their *written* self-type path + (`impl T for a::X` / `b::X` → `a%3A%3AX.impl[T]` / `b%3A%3AX.impl[T]`) and + then on their written trait path (`Compat<$0>.impl[a%3A%3AAsyncRead]`). + Twin-gated end to end: only already-colliding ids change; a lone impl and + every un-fired group render byte-identically to before (the frozen RS-WL + identity corpus is unchanged). + - **Amendment 8 — `#[path]` mount overlay** (`wardline.rust.mounts`): literal + `#[path = "…"] mod name;` declarations now route mounted files/subtrees to + their *logical* module path (rustc's relative-path rule; chains, cfg-twin + `@cfg` composition, R5 first-wins determinism; macro-wrapped and + `cfg_attr`-delivered mounts stay invisible by dialect rule). Mounted files + re-key from their filesystem route — `rust_module_route` itself is + unchanged and remains the no-mount default. + - **Amendment 9 — `const _` skip-emission:** an unnamed `const _` is no + longer an entity (unconditional — nothing can ever name it; findings inside + one attribute to the enclosing module). - **BREAKING (gate): parse failures are now gate-eligible.** `WLN-ENGINE-PARSE-ERROR` (a discovered file that could not be read/parsed) is promoted from a NONE FACT to an **ERROR DEFECT**: its sinks were never diff --git a/docs/integration/2026-06-11-wardline-resolve-plugin-hint-proposal.md b/docs/integration/2026-06-11-wardline-resolve-plugin-hint-proposal.md new file mode 100644 index 00000000..40c0318b --- /dev/null +++ b/docs/integration/2026-06-11-wardline-resolve-plugin-hint-proposal.md @@ -0,0 +1,72 @@ +# Wardline → Loomweave: `ResolveRequest` plugin-hint — proposed contract shape + +**From:** Wardline engineering +**To:** Loomweave maintainers (federation resolver, ADR-036) +**Date:** 2026-06-11 +**Re:** §7 of the Amendments 6–9 changeset letter +(`loomweave docs/federation/2026-06-11-rust-qualname-amendment-6-9-changeset.md`): +the resolver is now plugin-aware (`clarion-69db8b2739`, ADR-036 amended +2026-06-11) and a qualname owned by more than one plugin resolves `Ambiguous`, +which the federation wire degrades to `unresolved`. The agreed disambiguator is +a **plugin-hint field on `ResolveRequest`**. Loomweave asked Wardline to agree +the shape in the same escalation-gated exchange as the 4–9 re-vendor; this is +that agreement, from the consumer side. Loomweave owns the normative resolver +semantics (ADR-036) — where this proposal and the implemented endpoint diverge, +the endpoint + its conformance fixtures are the contract. + +## Why a hint, not inference + +A Rust free-function locator (`crate.module.func`) is byte-ambiguous with a +Python qualname — no locator-dialect sniffing can attribute it. But Wardline +always knows which frontend minted a finding (`--lang` selects the analyzer; +findings carry `lang`), and both Wardline call sites of +`POST /api/wardline/resolve` — dossier source resolution and the +Filigree entity-association bridge — resolve qualnames taken from a finding. +The producer knows; the wire should carry it. + +## Proposed shape + +```json +POST /api/wardline/resolve +{ + "project": "…", + "qualnames": ["demo.m.func", "…"], + "plugin": "rust" +} +``` + +- **`plugin`** — OPTIONAL string; values are ADR-049 plugin ids exactly as they + appear in entity ids (`python`, `rust`). **Batch-scoped** (one hint per + request, not per qualname): every Wardline resolve batch is minted by exactly + one frontend, so a per-row field would only invite mixed batches no producer + sends. A consumer with mixed-language qualnames sends one request per plugin. +- **Semantics with the hint:** resolution is restricted to the named plugin's + namespace. Unique match → resolved; no match in that plugin → `unresolved` + (even if another plugin owns the qualname — the hint is a constraint, never a + preference order); ambiguous *within* one plugin → `unresolved` (fail-closed; + post-gold this is `duplicate_ids() = 0` territory, but the wire must not + guess if it recurs). +- **Semantics without the hint:** exactly today's behavior — cross-plugin + lookup, `Ambiguous` degrades to `unresolved`. Omission stays legal forever + (a consumer that genuinely does not know the language must not be forced to + fabricate a hint). +- **Unknown plugin value:** `unresolved` for the whole batch (or a 400 naming + the field — Loomweave's call; Wardline only ever sends ids it produces). + +## Rollout (deliberate, coordinated — `deny_unknown_fields` forces ordering) + +1. **Loomweave first:** accept + honor `plugin` on `ResolveRequest` (the struct + is `#[serde(deny_unknown_fields)]`, so Wardline cannot send a byte before + this lands). Reject-with-a-message that NAMES the field if it ever 400s, for + cross-version diagnosability. +2. **Wardline second (tracked, see ticket below):** thread the producing + plugin from the finding's `lang` through `LoomweaveClient.resolve()` at both + call sites, and treat a resolve 4xx as fail-soft `unresolved` (an old server + must degrade the dossier/association, not crash it). +3. **Conformance:** three fixture rows ride the resolver's conformance surface + and Wardline's vendored oracle when the field ships — hinted-hit, + hinted-miss (qualname owned by the *other* plugin only), and + unhinted-ambiguous → `unresolved`. + +No `ontology_version` interaction; no change to `resolved`/`unresolved` +response shape; SEIs stay opaque to Wardline throughout. diff --git a/src/wardline/rust/analyzer.py b/src/wardline/rust/analyzer.py index cc64eb6f..4f1a3a1c 100644 --- a/src/wardline/rust/analyzer.py +++ b/src/wardline/rust/analyzer.py @@ -29,6 +29,7 @@ from wardline.rust.crate_roots import CrateRoots, discover_crate_roots from wardline.rust.dataflow import analyze_command_dataflow from wardline.rust.index import index_entities +from wardline.rust.mounts import MountOverlay, build_mount_overlay from wardline.rust.nodeid import mint_node_ids from wardline.rust.parse import has_errors, parse_rust from wardline.rust.provider import RustTrustProvider @@ -91,22 +92,32 @@ def analyze(self, files: Sequence[Path], config: WardlineConfig, *, root: Path) # SP2 whole-tree pass: discover Cargo crate roots ONCE per scan; every file's # module route resolves against this map (longest-prefix, symlink-safe walk). crate_roots = discover_crate_roots(resolved_root) + # ADR-049 Amendment 8 pre-pass: read every file ONCE, then build each crate's + # #[path] mount overlay from its scanned in-src sources — class-1 module routes + # resolve mount-first (logical_module_path), filesystem-fallback otherwise. + sources: dict[Path, str] = {} + read_errors: dict[Path, str] = {} + for file in files: + try: + sources[file] = file.read_text(encoding="utf-8") + except (OSError, UnicodeDecodeError) as exc: + read_errors[file] = str(exc) + overlays = _build_overlays(sources, resolved_root, crate_roots) findings: list[Finding] = [] functions_total = 0 functions_declared = 0 files_analyzed = 0 for file in files: relpath = _relpath(file, resolved_root) - try: - source = file.read_text(encoding="utf-8") - except (OSError, UnicodeDecodeError) as exc: - findings.append(_parse_error_finding(relpath, str(exc))) + if file in read_errors: + findings.append(_parse_error_finding(relpath, read_errors[file])) continue + source = sources[file] tree = parse_rust(source) if has_errors(tree): findings.append(_parse_error_finding(relpath, "tree-sitter recovered from a syntax error")) continue - module = _module_for(file, resolved_root, crate_roots) + module = _module_for(file, resolved_root, crate_roots, overlays) try: file_findings, context, file_callables = self._analyze_tree(tree, module=module, path=relpath) except Exception as exc: # noqa: BLE001 — per-file isolation, see below @@ -207,10 +218,41 @@ def _relpath(file: Path, resolved_root: Path) -> str: return resolved.as_posix() -def _module_for(file: Path, resolved_root: Path, roots: CrateRoots) -> str: +def _build_overlays(sources: dict[Path, str], resolved_root: Path, roots: CrateRoots) -> dict[Path, MountOverlay]: + """One ``#[path]`` mount overlay per crate (ADR-049 Amendment 8), discovered over + the scanned IN-SRC sources of that crate (class-2/3 files keep their ``#out`` + non-conformance routes — a mount declared outside ``src/`` is outside loomweave's + emittable scope and never overlays a class-1 route). Paths are project-root-relative + posix, matching the R5 sort rule ("declaring-file path relative to the project + root"). A mount declared in a file outside the scan list is invisible — the overlay + is the view of the scanned tree.""" + per_crate: dict[Path, tuple[str, dict[str, str]]] = {} + for file, source in sources.items(): + resolved = file.resolve() + crate_dir = roots.crate_dir_for(resolved) + crate_name = roots.crate_name_for(resolved) + if crate_dir is None or crate_name is None or not resolved.is_relative_to(crate_dir / "src"): + continue + if not resolved.is_relative_to(resolved_root): + continue # defensive: discover confines to root + per_crate.setdefault(crate_dir, (crate_name, {}))[1][resolved.relative_to(resolved_root).as_posix()] = source + return { + crate_dir: build_mount_overlay( + crate_sources, + crate=crate_name, + src_root=(crate_dir / "src").relative_to(resolved_root).as_posix(), + ) + for crate_dir, (crate_name, crate_sources) in per_crate.items() + } + + +def _module_for(file: Path, resolved_root: Path, roots: CrateRoots, overlays: dict[Path, MountOverlay]) -> str: """The SP2 module route. Three file classes: - 1. **In-src** (under a crate root's ``src/``): the ADR-049 oracle route — + 1. **In-src** (under a crate root's ``src/``): the ADR-049 oracle route — the + crate's ``#[path]`` mount overlay first (Amendment 8, + ``MountOverlay.logical_module_path``), whose default for an un-mounted file is + the unchanged pure-filesystem ``rust_module_route(crate=, src_root=/src, file)``. Conformance-bearing: byte-identical to loomweave's emission for the same file. 2. **Under a crate root but OUTSIDE its src/** (``tests/``, ``benches/``, @@ -239,6 +281,10 @@ def _module_for(file: Path, resolved_root: Path, roots: CrateRoots) -> str: if crate_dir is not None and crate_name is not None: src_root = crate_dir / "src" if resolved.is_relative_to(src_root): + overlay = overlays.get(crate_dir) + if overlay is not None and resolved.is_relative_to(resolved_root): + return overlay.logical_module_path(resolved.relative_to(resolved_root).as_posix()) + # No overlay built for this crate (defensive): the filesystem default. return q.rust_module_route(crate=crate_name, src_root=str(src_root), file=str(resolved)) return _out_route(crate_name, crate_dir, resolved) # class 2 try: diff --git a/src/wardline/rust/index.py b/src/wardline/rust/index.py index 685355f4..0fb230cf 100644 --- a/src/wardline/rust/index.py +++ b/src/wardline/rust/index.py @@ -13,14 +13,19 @@ descends a ``function_item`` body — a finding inside one attributes to the enclosing named fn via ``line_start``), trait-body items (extract.rs deliberately never walks trait bodies — a trait definition is only its ``trait`` entity), bare -macro INVOCATIONS, external ``mod foo;`` declarations, and ``union`` items -(outside the ten-kind set, the oracle's ``_ => None`` arm). +macro INVOCATIONS, external ``mod foo;`` declarations, ``union`` items (outside +the ten-kind set, the oracle's ``_ => None`` arm), and unnamed ``const _`` items +(ADR-049 Amendment 9 — unconditionally skipped on ``ident == "_"``: nothing can +ever name the item, so no discriminant can rescue it). cfg twins are counted per-(kind, name) over the nine named item kinds (extract.rs ``twin_counts``): ``fn S`` and ``struct S`` never interfere — the entity id's kind segment already separates them — and the ``@cfg(...)`` suffix is applied only on a -within-kind collision. Impl blocks have their own pre-cfg twin counter keyed on -the full impl segment. +within-kind collision. Impl qualnames are decided by the RESIDUAL-COLLISION LADDER +(ADR-049 Amendment 6, spanning Amendments 1/5/6/7): per scope, four stages each +keyed on the previous stage's output — ``@cfg`` (pre-cfg impl-segment twin counter) +-> stage S (self-type written-path qualification) -> stage T (trait written-path +qualification) -> method-``@cfg`` (keyed on the FINAL post-S/T impl qualname). This is the single-file, file-module-rooted view: the caller supplies ``module`` (the SP2 whole-tree pass — ``Cargo.toml`` crate roots + cross-file routes — lives @@ -159,34 +164,61 @@ def _walk_scope( if key is not None: twin_counts[key] += 1 - # Pre-cfg impl-segment twin counts (mirrors extract.rs `impl_twin_counts`): two - # impls of the same (type, generic-sig) — incl. cfg-twins — share one key and would - # collide; a cfg-gated member of such a group is split by an `@cfg(...)` suffix on - # the impl key (now the ONLY distinguisher for inherent twins, the ordinal is gone). - impl_segments: dict[int, str] = {} - impl_twin_counts: Counter[str] = Counter() - for node, _cfg in items: - if node.type == "impl_item": - seg = _impl_segment(node) - if seg is not None: - impl_segments[node.id] = seg - impl_twin_counts[seg] += 1 - - # Method-level cfg-twin counts (ADR-049 Amendment 5): keyed on the FINAL impl - # qualname (post impl-level cfg) + method name, counted across ALL merged blocks — - # so an impl-level cfg-twin (already split into distinct impl entities) gets no - # redundant method suffix, while methods merging across same-key blocks do. - final_impl_quals: dict[int, str] = {} - method_twin_counts: Counter[tuple[str, str]] = Counter() + # ---- impl qualnames: the ADR-049 residual-collision LADDER (Amendments 1/5/6/7), + # decided per scope in four stages, each keyed on the previous stage's output: + # (1) @cfg -> (2) stage S (self-type written path) -> (3) stage T (trait written + # path) -> (4) method-@cfg. + # Twin-gated end to end: a lone impl never qualifies, un-fired groups change nothing, + # and cross-path cfg-twins (split at stage 1) leave S cold. Per-scope grouping IS the + # per-extraction-unit grouping: a qualname embeds the full module path, so groups can + # never span scopes. + + # Stage 1 (@cfg): pre-cfg impl-segment twin counts on the BARE keys, exactly as + # before the ladder existed (mirrors extract.rs `impl_twin_counts`) — already-@cfg- + # split twins keep their current ids byte-for-byte. + impl_keys: dict[int, _ImplKey] = {} + bare_counts: Counter[str] = Counter() for node, cfgs in items: if node.type == "impl_item": - seg = impl_segments.get(node.id) - if seg is None: - continue - if cfgs and impl_twin_counts[seg] > 1: - seg += q.cfg_discriminant(cfgs) - final_qual = f"{module}.{seg}" - final_impl_quals[node.id] = final_qual + ikey = _impl_key(node, cfgs) + if ikey is not None: + impl_keys[node.id] = ikey + bare_counts[ikey.key] += 1 + for k in impl_keys.values(): + if k.cfgs and bare_counts[k.key] > 1: + k.cfg_suffix = q.cfg_discriminant(k.cfgs) + + # Stage S (Amendment 6): a post-cfg group with >= 2 distinct self-type written-path + # witnesses re-renders every qself-free Type::Path member's base as the escaped + # written path; an A4-fallback member keeps its single-escaped render (its witness + # still counts toward distinctness). Identical-witness coherence-illegal twins do + # not fire — no witness can split them (`duplicate_ids()` is the alarm upstream). + for group in _impl_groups(impl_keys): + if len({m.self_witness for m in group}) > 1: + for m in group: + if m.self_is_path: + m.base = m.self_witness + + # Stage T (Amendment 7): a post-S group with >= 2 distinct trait written paths + # switches EVERY member's impl[...] fragment to the qualified rendering (a single- + # segment path renders byte-identically; inherent impls never fire — their #<> keys + # never group with [...] keys). Running T after S yields minimal qualification: a + # pair already split by S leaves T cold. + for group in _impl_groups(impl_keys): + if len({m.trait_witness for m in group if m.trait_witness is not None}) > 1: + for m in group: + if m.trait_node is not None: + m.fragment = f"impl[{q.render_trait_segment_qualified(m.trait_node)}]" + + # Stage 4 — method-level cfg-twin counts (ADR-049 Amendment 5): keyed on the FINAL + # (post-S/T) impl qualname + method name, counted across ALL merged blocks — so an + # impl-level cfg-twin (already split into distinct impl entities) gets no redundant + # method suffix, while methods merging across same-key blocks do. + final_impl_quals: dict[int, str] = {nid: f"{module}.{k.key}" for nid, k in impl_keys.items()} + method_twin_counts: Counter[tuple[str, str]] = Counter() + for node, _cfgs in items: + if node.type == "impl_item" and node.id in final_impl_quals: + final_qual = final_impl_quals[node.id] for method, _mcfgs in _impl_methods_with_cfgs(node): method_twin_counts[(final_qual, _name(method))] += 1 @@ -221,6 +253,13 @@ def _walk_scope( else: kind = _LEAF_KINDS[node.type] name = _name(node) + if kind == "const" and name == "_": + # ADR-049 Amendment 9: `const _` is NOT an entity — skipped + # UNCONDITIONALLY on `ident == "_"` (skip-only-when-twinned would make + # the emitted set sibling-dependent and churn SEI; nothing can ever name + # the item, so no discriminant can rescue it). No entity, no containment; + # a finding inside one attributes to the enclosing module by line. + continue qualname = f"{module}.{name}" if cfgs and twin_counts[(kind, name)] > 1: qualname += q.cfg_discriminant(cfgs) @@ -238,22 +277,71 @@ def _named_item_key(node: Node) -> tuple[str, str] | None: kind = _LEAF_KINDS.get(node.type) if kind is None: return None - return (kind, _name(node)) - - -def _impl_segment(impl_node: Node) -> str | None: - """The pre-cfg ``.impl[...]`` / ``.impl#<...>`` segment, or - ``None`` if the impl has no self type. The self type carries its concrete generic args - (ADR-049 §2 self-type-args amendment — ``Foo`` vs ``Foo`` are distinct keys, - the impl's own params positional); no ordinal (ADR-049 amend Option b).""" + name = _name(node) + if kind == "const" and name == "_": + return None # ADR-049 Amendment 9: never emitted, so never counted + return (kind, name) + + +@dataclass(slots=True) +class _ImplKey: + """One impl block's decomposed segment parts, MUTATED through the residual-collision + ladder (stage 1 sets ``cfg_suffix``; a fired stage S rewrites ``base``; a fired stage + T rewrites ``fragment``). ``key`` is the current ``.impl…@cfg`` segment — + before stage 1 it IS the bare pre-cfg key the @cfg twin counter runs on.""" + + cfgs: list[str] + base: str # self-type base render (last path segment / A4 fallback) + self_args: str # "<...>" args suffix, "" when none survive (stage-invariant) + fragment: str # "impl[...]" (bare) / "impl#<...>" + cfg_suffix: str # "" until stage 1 fires + self_witness: str # stage-S witness (escaped written path / A4 fallback render) + self_is_path: bool # qself-free Type::Path -> base re-renders on a fired S group + trait_node: Node | None + trait_witness: str | None # stage-T witness (written trait path), None for inherent + + @property + def key(self) -> str: + return f"{self.base}{self.self_args}.{self.fragment}{self.cfg_suffix}" + + +def _impl_key(impl_node: Node, cfgs: list[str]) -> _ImplKey | None: + """The decomposed pre-cfg ``.impl[...]`` / ``.impl#<...>`` parts, + or ``None`` if the impl has no self type. The self type carries its concrete generic + args (ADR-049 §2 self-type-args amendment — ``Foo`` vs ``Foo`` are distinct + keys, the impl's own params positional); no ordinal (ADR-049 amend Option b).""" type_node = impl_node.child_by_field_name("type") if type_node is None: return None - self_type = q.render_self_type(type_node, q.impl_type_param_names(impl_node)) + base, self_args = q.render_self_type_parts(type_node, q.impl_type_param_names(impl_node)) + self_witness, self_is_path = q.self_type_witness(type_node) trait_node = impl_node.child_by_field_name("trait") if trait_node is not None: - return f"{self_type}.impl[{q.render_trait_segment(trait_node)}]" - return f"{self_type}.impl#<{q.render_positional_generics(impl_node)}>" + fragment = f"impl[{q.render_trait_segment(trait_node)}]" + trait_witness = q.trait_written_path(trait_node) + else: + fragment = f"impl#<{q.render_positional_generics(impl_node)}>" + trait_witness = None + return _ImplKey( + cfgs=cfgs, + base=base, + self_args=self_args, + fragment=fragment, + cfg_suffix="", + self_witness=self_witness, + self_is_path=self_is_path, + trait_node=trait_node, + trait_witness=trait_witness, + ) + + +def _impl_groups(impl_keys: dict[int, _ImplKey]) -> list[list[_ImplKey]]: + """The current collision groups: impls sharing a ``key``, singletons dropped (the + ladder is twin-gated — a lone impl never qualifies).""" + by_key: dict[str, list[_ImplKey]] = {} + for k in impl_keys.values(): + by_key.setdefault(k.key, []).append(k) + return [g for g in by_key.values() if len(g) > 1] def _impl_methods_with_cfgs(impl_node: Node) -> list[tuple[Node, list[str]]]: diff --git a/src/wardline/rust/mounts.py b/src/wardline/rust/mounts.py new file mode 100644 index 00000000..8405e8f0 --- /dev/null +++ b/src/wardline/rust/mounts.py @@ -0,0 +1,268 @@ +"""ADR-049 Amendment 8: the ``#[path]`` mount overlay — logical module routing. + +Two producers minting one module id was the clarion-bdb1eccf48 family: the file walk +routed a mounted file by filesystem path while the AST walk emitted the inline facade at +the same dotted path. The fix is a targeted mount overlay WITH a filesystem default: +every literal ``#[path = "…"] mod name;`` declaration is collected (rustc's +relative-path rule), resolved through a memoized fixed point (mounts chain; cycles drop +to the filesystem fallback; a doubly-claimed target resolves first-by-sorted- +(declaring-file, byte offset) — the R5 determinism pin), and ``logical_module_path`` +then routes every file: exact mount hit, else longest mounted-subtree prefix, else the +unchanged pure-filesystem ``qualname.rust_module_route``. + +Invisible BY DIALECT RULE (never resolved): a macro-wrapped mount (inside an unexpanded +macro invocation) and a ``#[cfg_attr(pred, path = "…")]``-delivered mount are NOT +mounts — only a literal ``#[path]`` attribute is. Their targets route by filesystem +fallback. No producer expands macros or evaluates cfg predicates. Both fall out of the +walk structurally: a token tree never surfaces ``attribute_item``/``mod_item`` siblings +at a walked item list, and ``_path_attr_of`` matches only the attribute literally named +``path``. A ``#[path]`` on an INLINE ``mod name { … }`` is likewise not a mount (the +normative rule covers the decl form only); the inline body still nests by name. + +Twin discipline (ADR-049 §3, extended): a mount's own segment appends the normalised +``@cfg(...)`` discriminant only when its module name is TWINNED in the declaring item +list — counted across BOTH inline-``mod`` and decl-``mod`` forms. A mount declared +INSIDE a cfg-twin inline mod composes that mod's ``@cfg``-suffixed segment into its +logical PREFIX — the SAME twin counting + rendering the AST walk applies to the inline +mod's own entity path (counted over inline-with-body mods only, matching +``index._named_item_key``), so file-walk and AST-walk agree byte-for-byte. The +``#[path]`` target itself always resolves against the BARE would-be directory, which +carries no cfg. + +This is SP2 scope (mount discovery needs the parent chain — a declaring file's +directory anchors the relative target); ``module_route`` rows keep driving +``rust_module_route`` directly, pinned as the no-mount-context FALLBACK. The corpus +``module_mounts`` section pins the mounted routes end-to-end. A file tree-sitter cannot +fully parse contributes NO mounts (fail-closed: no routing derived from a file we +refuse to analyze). + +All paths are project-root-relative POSIX strings (the R5 sort rule is "declaring-file +path relative to the project root"); ``build_mount_overlay`` is the pure corpus-facing +entry, ``analyzer.analyze`` builds one overlay per crate from the scanned in-src +sources. tree-sitter types appear only under ``TYPE_CHECKING`` so importing this module +never pulls the ``wardline[rust]`` extra. +""" + +from __future__ import annotations + +import posixpath +from collections import Counter +from dataclasses import dataclass +from typing import TYPE_CHECKING + +from wardline.rust import qualname as q +from wardline.rust.parse import has_errors, parse_rust + +if TYPE_CHECKING: + from collections.abc import Iterable, Mapping + + from tree_sitter import Node + +__all__ = ["MountOverlay", "build_mount_overlay"] + +# Files whose own stem contributes no module segment; for nesting they anchor the +# would-be directory at their OWN directory (a non-root file anchors at dir/stem). +_ROOT_BASENAMES = frozenset({"lib.rs", "main.rs", "mod.rs"}) + +# Token-stream-invisible to the oracle — never resets the pending-attribute run +# (mirrors index._COMMENT_TYPES; the corpus pins the discipline for cfg attrs). +_COMMENT_TYPES = frozenset({"line_comment", "block_comment"}) + + +@dataclass(frozen=True, slots=True) +class _Mount: + """One literal ``#[path = "…"] mod name;`` declaration, resolved to its target.""" + + declaring_file: str # project-root-relative posix path (R5 sort key 1) + offset: int # byte offset of the `mod name;` item (R5 sort key 2) + segments: tuple[str, ...] # inline-mod prefix segments (cfg-composed) + own segment + target: str # normalised posix path of the mounted file + + +class MountOverlay: + """The per-crate routing table: mounted files/subtrees overlay the filesystem route. + + ``crate``/``src_root`` parameterise the filesystem DEFAULT (``rust_module_route``); + every path handed to ``logical_module_path`` must use the same base the mounts were + discovered under (project-root-relative posix).""" + + def __init__(self, mounts: Iterable[_Mount], *, crate: str, src_root: str) -> None: + self._crate = crate + self._src_root = src_root + exact: dict[str, _Mount] = {} + prefixes: dict[str, _Mount] = {} + # R5 determinism: first by sorted (declaring-file, byte offset) wins a + # doubly-claimed target file (and likewise a doubly-claimed subtree prefix). + for mount in sorted(mounts, key=lambda m: (m.declaring_file, m.offset)): + exact.setdefault(mount.target, mount) + target_dir, basename = posixpath.split(mount.target) + if basename == "mod.rs": + # A /mod.rs target registers / as a logical subtree prefix. + prefixes.setdefault(target_dir, mount) + elif basename.endswith(".rs"): + # An x.rs target registers /x/ for its child directory + # (rustc's non-mod-rs child rule). + prefixes.setdefault(posixpath.join(target_dir, basename[: -len(".rs")]), mount) + self._exact = exact + self._prefixes = prefixes + self._memo: dict[str, str] = {} + + def logical_module_path(self, file: str) -> str: + """Route ``file``: exact mount hit, else longest mounted-subtree prefix, else + the unchanged pure-filesystem ``rust_module_route``.""" + file = posixpath.normpath(file) + if file not in self._memo: + self._memo[file] = self._resolve(file, frozenset()) + return self._memo[file] + + def _resolve(self, file: str, resolving: frozenset[str]) -> str: + if file in resolving: + # Mount cycle: drop this link to the filesystem fallback (deterministic; + # the corpus does not pin cycles — unit-tested as Wardline behavior). + return self._fs_route(file) + mount = self._exact.get(file) + if mount is not None: + return self._mount_logical(mount, resolving | {file}) + hit = self._longest_prefix(file) + if hit is not None: + prefix_dir, mount = hit + base = self._mount_logical(mount, resolving | {file}) + # Children rewrite under the mount with the same stem discipline as the + # filesystem route (trailing `mod` stem collapses): reuse it verbatim, + # the mount's logical path standing in as the "crate" prefix. + return q.rust_module_route(crate=base, src_root=prefix_dir, file=file) + return self._fs_route(file) + + def _mount_logical(self, mount: _Mount, resolving: frozenset[str]) -> str: + # The mount's own logical path: the DECLARING file's logical path (which may + # itself route through a mount — the chained fixed point) + its segments. + return ".".join([self._resolve(mount.declaring_file, resolving), *mount.segments]) + + def _longest_prefix(self, file: str) -> tuple[str, _Mount] | None: + best: tuple[str, _Mount] | None = None + for prefix_dir, mount in self._prefixes.items(): + if file.startswith(prefix_dir + "/") and (best is None or len(prefix_dir) > len(best[0])): + best = (prefix_dir, mount) + return best + + def _fs_route(self, file: str) -> str: + return q.rust_module_route(crate=self._crate, src_root=self._src_root, file=file) + + +def build_mount_overlay(sources: Mapping[str, str], *, crate: str, src_root: str) -> MountOverlay: + """Discover every literal ``#[path]`` mount across ``sources`` (path -> source text, + paths project-root-relative posix) and build the crate's routing overlay.""" + mounts: list[_Mount] = [] + for file in sorted(sources): + if not file.endswith(".rs"): + continue + tree = parse_rust(sources[file]) + if has_errors(tree): + continue # fail-closed: no routing derived from a file we refuse to analyze + file_dir = posixpath.dirname(file) + # rustc's relative-path rule: a top-level #[path] resolves against the declaring + # FILE's directory; one declared inside inline mods resolves against the would-be + # directory of the nesting — anchored at the file's own dir for a mod-rs file + # (lib.rs/main.rs/mod.rs), at the file's stem directory otherwise. + stem_base = file_dir if posixpath.basename(file) in _ROOT_BASENAMES else file[: -len(".rs")] + _collect_mounts(tree.root_node.children, file, attr_dir=file_dir, nest_base=stem_base, prefix=(), out=mounts) + return MountOverlay(mounts, crate=crate, src_root=src_root) + + +def _collect_mounts( + children: Iterable[Node], + file: str, + *, + attr_dir: str, + nest_base: str, + prefix: tuple[str, ...], + out: list[_Mount], +) -> None: + items = _mod_items_with_attrs(children) + # The mount's OWN segment twin-gates its @cfg across BOTH forms per declaring item + # list; the inline-mod PREFIX segment twin-gates over inline-with-body mods only + # (the AST walk's _named_item_key rule) — so prefix composition matches the inline + # mod's own entity path byte-for-byte. + cross_form_counts: Counter[str] = Counter() + inline_counts: Counter[str] = Counter() + for node, _cfgs, _target in items: + name = _mod_name(node) + cross_form_counts[name] += 1 + if node.child_by_field_name("body") is not None: + inline_counts[name] += 1 + for node, cfgs, target in items: + name = _mod_name(node) + body = node.child_by_field_name("body") + if body is None: + if target is None: + continue # plain `mod foo;` — filesystem-routed, not a mount + segment = name + if cfgs and cross_form_counts[name] > 1: + segment += q.cfg_discriminant(cfgs) + resolved = posixpath.normpath(posixpath.join(attr_dir, target)) + out.append(_Mount(file, node.start_byte, (*prefix, segment), resolved)) + else: + segment = name + if cfgs and inline_counts[name] > 1: + segment += q.cfg_discriminant(cfgs) + # The #[path] target inside an inline mod resolves against the BARE + # would-be directory (no cfg ever reaches the filesystem side). + child_dir = posixpath.join(nest_base, name) + _collect_mounts( + body.children, file, attr_dir=child_dir, nest_base=child_dir, prefix=(*prefix, segment), out=out + ) + + +def _mod_items_with_attrs(children: Iterable[Node]) -> list[tuple[Node, list[str], str | None]]: + """``(mod_item, raw cfg predicates, #[path] target | None)`` triples of one item + list, with the SAME pending-attribute discipline as ``index._walk_scope`` (comments + transparent, attributes accumulate, any other node resets).""" + out: list[tuple[Node, list[str], str | None]] = [] + pending_cfgs: list[str] = [] + pending_path: str | None = None + for child in children: + if child.type in _COMMENT_TYPES: + continue + if child.type == "attribute_item": + pred = q.cfg_predicate_of(child) + if pred is not None: + pending_cfgs.append(pred) + else: + target = _path_attr_of(child) + if target is not None: + pending_path = target + continue + if child.type == "mod_item": + out.append((child, pending_cfgs, pending_path)) + pending_cfgs = [] + pending_path = None + return out + + +def _path_attr_of(attribute_item: Node) -> str | None: + """The literal target of a ``#[path = "…"]`` attribute, or ``None``. + + Only the literal form is a mount: a ``cfg_attr``-delivered ``path`` never matches + (its attribute path is ``cfg_attr``), and a path attribute without a string value + is not a mount either.""" + if not attribute_item.named_children: + return None + attribute = attribute_item.named_children[0] + if attribute.type != "attribute" or not attribute.named_children: + return None + if attribute.named_children[0].text != b"path": + return None + value = attribute.child_by_field_name("value") + if value is None or value.type != "string_literal": + return None + content = next((c for c in value.named_children if c.type == "string_content"), None) + if content is None or content.text is None: + return None + return content.text.decode("utf-8") + + +def _mod_name(node: Node) -> str: + name = node.child_by_field_name("name") + if name is not None and name.text is not None: + return name.text.decode("utf-8") + return "" diff --git a/src/wardline/rust/qualname.py b/src/wardline/rust/qualname.py index ee83c7b0..437ef263 100644 --- a/src/wardline/rust/qualname.py +++ b/src/wardline/rust/qualname.py @@ -17,6 +17,24 @@ Corpus rows: ``path_typed_generic_arg_trait``/``_inherent``, ``const_generic_arg_spacing``, ``reference_self_type_path_escape``. +ADR-049 AMENDMENTS 6+7 (2026-06-11, implemented): the residual-collision LADDER for impl +qualnames — ``@cfg -> stage S (self-type written path) -> stage T (trait written path) -> +method-@cfg`` — lives in ``index._walk_scope``; this module supplies the witness/render +primitives. ``written_path_of`` is the qself-free Type::Path detector (segment idents +joined ``::``, a leading ``::`` preserved, generic args ignored); ``self_type_witness`` +is the stage-S witness (escaped written path, or the Amendment-4 textual render for a +qself-bearing/non-path self type); ``trait_written_path`` + ``render_trait_segment_qualified`` +are the stage-T witness and fired-group rendering (escaped written trait path, final +segment keeping the existing generic-args rendering — a single-segment path renders +byte-identically to the bare fragment). All witnesses are SINGLE-FILE computable: written +paths as-typed, no name resolution (the second-producer constraint). + +ADR-049 AMENDMENT 8 (2026-06-11, implemented): ``rust_module_route`` stays the +PURE-FILESYSTEM route by design — ``#[path]``-aware routing is the SEPARATE entry point +``mounts.MountOverlay.logical_module_path`` (mount overlay first, this route as the +default), pinned by the corpus ``module_mounts`` section. Amendment 9 (``const _`` +skip-emission) lives in ``index._walk_scope``. + This module holds the pure string/CST-node renderers; ``index.py`` walks the tree and assembles full qualnames from them. tree-sitter types appear only under ``TYPE_CHECKING`` so importing this module never pulls the ``wardline[rust]`` extra. @@ -41,8 +59,13 @@ "normalize_cfg_predicate", "render_positional_generics", "render_self_type", + "render_self_type_parts", "render_trait_segment", + "render_trait_segment_qualified", "rust_module_route", + "self_type_witness", + "trait_written_path", + "written_path_of", ] _ROOT_STEMS = frozenset({"lib", "main", "mod"}) @@ -239,8 +262,17 @@ def render_self_type(type_node: Node, type_params: Sequence[str]) -> str: """The locator-relevant name of an impl's self type INCLUDING its concrete generic args (mirrors qualname.rs ``self_ty_locator``, ADR-049 §2 self-type-args amendment). - The bare last path segment (``Foo`` in ``crate::m::Foo``) plus its surviving generic - args in ``<...>`` — comma-joined, ``<>`` omitted ENTIRELY when none survive. An arg + ``base + args`` of ``render_self_type_parts`` — see there for the full rule.""" + base, args = render_self_type_parts(type_node, type_params) + return base + args + + +def render_self_type_parts(type_node: Node, type_params: Sequence[str]) -> tuple[str, str]: + """``(base, args)`` decomposition of the self-type render — the Amendment-6 ladder + re-renders the BASE alone (stage S) while the ``<...>`` args suffix is stage-invariant. + + The base is the bare last path segment (``Foo`` in ``crate::m::Foo``); ``args`` is the + surviving generic args in ``<...>`` — comma-joined, ``""`` when none survive. An arg that is exactly one of the impl's OWN declared params (``type_params``) renders positionally (``$N``, rename-stable: ``impl Foo`` -> ``Foo<$0>``); a CONCRETE arg (``i32``, ``Vec``, ``&T``) renders literally whitespace-free. Substitution is @@ -248,14 +280,19 @@ def render_self_type(type_node: Node, type_params: Sequence[str]) -> str: NOT ``Foo>``) keeps its literal text (F2 nested-param rule; a nested-param corpus row is owed by Loomweave, so the unit guard in test_qualname.py is the only check against accidental recursive substitution). Lifetimes/bindings dropped; a - non-generic self type renders the bare name; an exotic self type (tuple/array) renders - whitespace-free (out-of-corpus, Tier-B).""" + non-generic self type renders the bare name. A non-``Type::Path`` OR qself-bearing + self type (``&mut fmt::Formatter``, ``::C`` — ``written_path_of`` is None) + renders the Amendment-4 textual fallback as the base, args empty (Amendment 6 pins + qself with the fallback: the witness/render must not double-escape).""" if type_node.type == "generic_type": base_node = type_node.child_by_field_name("type") - base = _last_path_segment(base_node) if base_node is not None else "" + if base_node is None or written_path_of(base_node) is None: + # qself-bearing generic path (`::C`): whole-node A4 fallback. + return _escape_reserved(_strip_ws(type_node)), "" + base = _last_path_segment(base_node) targs = type_node.child_by_field_name("type_arguments") if targs is None: - return base + return base, "" # Drop dropped-kinds AND any arg that renders empty: a malformed empty turbofish # `Foo<>` error-recovers to a blank `type_identifier` (valid Rust never yields one, # so this never touches a real arg). Keeps the pure producer from emitting `Foo<>`; @@ -266,13 +303,60 @@ def render_self_type(type_node: Node, type_params: Sequence[str]) -> str: if c.type not in _DROPPED_GENERIC_ARGS if (r := _self_type_arg(c, type_params)) ] - return f"{base}<{','.join(rendered)}>" if rendered else base - if type_node.type in ("type_identifier", "scoped_type_identifier"): - return _last_path_segment(type_node) + return (base, f"<{','.join(rendered)}>") if rendered else (base, "") + if type_node.type in ("type_identifier", "scoped_type_identifier") and written_path_of(type_node) is not None: + return _last_path_segment(type_node), "" # Non-Type::Path fallback (reference/tuple/slice/raw-pointer self types): may carry a # ``::``-path (`&mut fmt::Formatter`), so it escapes like a concrete generic arg # (ADR-049 Amendment 4 self-type-fallback completion). A `:`-free fallback is unchanged. - return _escape_reserved(_strip_ws(type_node)) + return _escape_reserved(_strip_ws(type_node)), "" + + +def written_path_of(node: Node) -> str | None: + """The WRITTEN path of a qself-free Type::Path node — segment idents joined ``::``, + a leading ``::`` contributing a leading separator (``impl Tr for ::a::X`` and + ``impl Tr for a::X`` carry distinct witnesses), generic args never included — or + ``None`` for a non-path / qself-bearing node (``::C`` routes its + ``bracketed_type`` head here and falls out). Mirrors the Amendment-6 witness rule: + as-written, single-file, no name resolution.""" + if node.type in ("type_identifier", "identifier", "crate", "super", "self", "metavariable"): + return _text(node) + if node.type in ("scoped_type_identifier", "scoped_identifier"): + name = node.child_by_field_name("name") + if name is None: + return None + path = node.child_by_field_name("path") + if path is None: + # `::X` — a leading-:: anchor with no prefix path before the separator. + return f"::{_text(name)}" if any(c.type == "::" for c in node.children) else _text(name) + prefix = written_path_of(path) + return None if prefix is None else f"{prefix}::{_text(name)}" + return None + + +def self_type_witness(type_node: Node) -> tuple[str, bool]: + """The stage-S witness of an impl's self type (ADR-049 Amendment 6) as + ``(witness, is_path)``. + + For a qself-free ``Type::Path`` (a ``generic_type`` contributes its base — args are + already normalized and MUST NOT affect the witness): ``escape_reserved`` over the + written path, ``is_path=True`` — a fired group re-renders this member's base to the + witness verbatim. Otherwise the Amendment-4 textual render (already escaped) with + ``is_path=False`` — the member participates in distinctness but its rendering is + never touched (re-applying the escape would double-escape).""" + base = type_node.child_by_field_name("type") if type_node.type == "generic_type" else type_node + written = written_path_of(base) if base is not None else None + if written is not None: + return _escape_reserved(written), True + return _escape_reserved(_strip_ws(type_node)), False + + +def trait_written_path(trait_node: Node) -> str | None: + """The stage-T witness (ADR-049 Amendment 7): the trait path AS WRITTEN — segment + idents joined ``::``, leading ``::`` preserved, generic args ignored — or ``None`` + for a non-path trait node (out-of-corpus; such a member never drives a fire).""" + base = trait_node.child_by_field_name("type") if trait_node.type == "generic_type" else trait_node + return written_path_of(base) if base is not None else None def _self_type_arg(arg_node: Node, type_params: Sequence[str]) -> str: @@ -298,23 +382,38 @@ def render_trait_segment(trait_node: Node) -> str: if trait_node.type == "generic_type": base = trait_node.child_by_field_name("type") trait_name = _last_path_segment(base) if base is not None else "" - targs = trait_node.child_by_field_name("type_arguments") - args = ( - # Drop dropped-kinds AND empty-rendering args (a malformed `From<>` error-recovers - # to a blank arg; valid Rust never yields one) -> bare trait name, no empty `<>`. - [ - _escape_reserved(s) - for c in targs.named_children - if c.type not in _DROPPED_GENERIC_ARGS - if (s := _strip_ws(c)) - ] - if targs is not None - else [] - ) + args = _trait_generic_args(trait_node) return f"{trait_name}<{','.join(args)}>" if args else trait_name return _last_path_segment(trait_node) +def render_trait_segment_qualified(trait_node: Node) -> str: + """The Amendment-7 FIRED-group trait fragment: ``escape_reserved`` over the + ``::``-joined written trait path (leading ``::`` -> leading ``%3A%3A``), the final + segment keeping the existing ``trait_generic_args`` rendering. A single-segment + written path renders byte-identically to ``render_trait_segment``. Applied to EVERY + member of a fired post-S group — never outside one (the corpus-pinned bare fragment + is untouched for un-fired groups).""" + written = trait_written_path(trait_node) + if written is None: # non-path trait node (out-of-corpus): bare rendering, defensively + return render_trait_segment(trait_node) + qualified = _escape_reserved(written) + args = _trait_generic_args(trait_node) if trait_node.type == "generic_type" else [] + return f"{qualified}<{','.join(args)}>" if args else qualified + + +def _trait_generic_args(trait_node: Node) -> list[str]: + """The surviving (concrete type/const) generic args of a ``generic_type`` trait node, + each whitespace-stripped + escaped. Drops dropped-kinds AND empty-rendering args (a + malformed ``From<>`` error-recovers to a blank arg; valid Rust never yields one).""" + targs = trait_node.child_by_field_name("type_arguments") + if targs is None: + return [] + return [ + _escape_reserved(s) for c in targs.named_children if c.type not in _DROPPED_GENERIC_ARGS if (s := _strip_ws(c)) + ] + + def render_positional_generics(impl_node: Node) -> str: """The impl's TYPE params rendered positionally (De Bruijn): ``impl`` -> ``$0``; ``impl`` / ``impl<'a>`` / ``impl`` -> ``""``. Only ``type_parameter``s diff --git a/tests/conformance/qualnames_rust.json b/tests/conformance/qualnames_rust.json index 5f3a2fa5..d81fb975 100644 --- a/tests/conformance/qualnames_rust.json +++ b/tests/conformance/qualnames_rust.json @@ -1,9 +1,9 @@ { - "_doc": "Shared RUST qualname conformance corpus (Loomweave <-> Wardline). Mirrors tests/conformance/qualnames.json (the Python corpus) but for the Rust dialect. For Rust, LOOMWEAVE IS AUTHORITATIVE: the dialect is fixed by ADR-049 (docs/loomweave/adr/ADR-049-rust-qualname-canonicalization.md) and these 'expected' values were GENERATED by running the loomweave-plugin-rust extractor (extract_file / module_path_for), never hand-authored. Loomweave hosts this file at fixtures/qualnames_rust.json; Wardline vendors a pinned copy to tests/conformance/qualnames_rust.json and must reproduce 'expected' BYTE-FOR-BYTE from its tree-sitter frontend. Inverts the Python arrangement (Wardline seeded qualnames.json; here Loomweave seeds) because Loomweave's whole-tree extractor is the oracle that defines the Rust locator. The parity test crates/loomweave-plugin-rust/tests/qualname_conformance.rs fails loudly on any divergence: fix the extractor or this fixture, never silently. Nothing imports this file at runtime.", + "_doc": "Shared RUST qualname conformance corpus (Loomweave <-> Wardline). Mirrors tests/conformance/qualnames.json (the Python corpus) but for the Rust dialect. For Rust, LOOMWEAVE IS AUTHORITATIVE: the dialect is fixed by ADR-049 (docs/loomweave/adr/ADR-049-rust-qualname-canonicalization.md) and these 'expected' values were GENERATED by running the loomweave-plugin-rust extractor (extract_file / module_path_for), never hand-authored. Loomweave hosts this file at fixtures/qualnames_rust.json; Wardline vendors a pinned copy to tests/conformance/qualnames_rust.json and must reproduce 'expected' BYTE-FOR-BYTE from its tree-sitter frontend. Inverts the Python arrangement (Wardline seeded qualnames.json; here Loomweave seeds) because Loomweave's whole-tree extractor is the oracle that defines the Rust locator. The parity test crates/loomweave-plugin-rust/tests/qualname_conformance.rs fails loudly on any divergence: fix the extractor or this fixture, never silently. Nothing imports this file at runtime. SECTIONS: `module_route` rows drive the pure-filesystem module_path_for(crate, src_root, file) directly (no real files, no declaring-file context); `module_mounts` rows (ADR-049 Amendment 8) pin #[path]-mounted routing — the harness materialises each row's `files` into a tempdir (synthesising a Cargo.toml from `crate`), runs discover_crate_roots + discover_mounts, and asserts logical_module_path(crate, src/, file, mounts) == `expect[file]` byte-for-byte; `entities` rows drive extract_file.", "_dialect_summary": { "delimiter": ".", "crate_rooted": "first segment is the crate name, '-' normalised to '_' (loomweave-cli -> loomweave_cli); discovered from Cargo.toml [package].name read as TEXT, longest-path-prefix match (crate_roots.rs). Falls back to the dir holding src/lib.rs|src/main.rs.", - "module_route": "crate root + mod tree to the item; lib.rs/main.rs/mod.rs contribute NO segment; inline `mod foo {}` nests. #[path] is NOT yet honoured (KNOWN-GAP, sp2).", + "module_route": "crate root + mod tree to the item; lib.rs/main.rs/mod.rs contribute NO segment; inline `mod foo {}` nests. #[path]-mounted modules route to their MOUNTED logical path (ADR-049 Amendment 8): a targeted mount overlay (exact file + mounted-subtree prefixes, twin mounts split by @cfg) with the filesystem route as the default — see the module_mounts section. Macro-wrapped mounts are invisible (filesystem fallback); out-of-src targets are ignored.", "free_items": ".. for EVERY named free-item kind: function, struct, enum, trait, type_alias, const, static, macro (the macro_rules! definition). Trait BODIES are deliberately not walked: a trait definition emits only its own `trait` entity, never its method items.", "self_type_generic_args": " in every impl locator carries the self type's CONCRETE generic arguments (ADR-049 §2 amendment): impl Foo -> Foo, impl Foo -> Foo, so the two impls (and their like-named methods) get distinct ids instead of colliding on a bare `Foo` key and being spuriously merged (silent data loss at the writer's ON CONFLICT(id) DO UPDATE). A self-type arg that is the impl's OWN declared generic param renders POSITIONALLY ($0,$1,... De Bruijn) — impl Foo -> Foo<$0> — so a param rename does not churn; concrete args (i32, String, const, nested generics) render whitespace-free; lifetimes/associated-type bindings dropped. A non-generic self type (impl Foo) renders bare `Foo`. Composition order: .|impl[Trait]>[@cfg(...)].", "trait_impl_method": ".impl[]. e.g. Foo.impl[Display].fmt, Foo.impl[From].from, Foo.impl[Display].fmt. Lifetimes dropped; concrete type/const generic args KEPT in BOTH the self-type prefix and the trait fragment (they are part of the key).", @@ -23,7 +23,85 @@ {"name": "flat_file", "crate": "demo", "src_root": "/p/src", "file": "/p/src/config.rs", "expected_module": "demo.config", "reproducibility": "slice-1"}, {"name": "nested_file", "crate": "demo", "src_root": "/p/src", "file": "/p/src/plugin/host.rs", "expected_module": "demo.plugin.host", "reproducibility": "slice-1"}, {"name": "mod_rs_collapse", "crate": "demo", "src_root": "/p/src", "file": "/p/src/plugin/mod.rs", "expected_module": "demo.plugin", "reproducibility": "slice-1", "note": "mod.rs contributes no segment; collapses to its directory."}, - {"name": "path_attr_known_gap", "crate": "demo", "src_root": "/p/src", "file": "/p/src/renamed.rs", "expected_module": "demo.renamed", "reproducibility": "sp2", "known_gap": true, "note": "If `renamed.rs` is mounted via `#[path=\"renamed.rs\"] mod logical;` its logical module is `demo.logical`, but module_path_for routes purely by file path and emits `demo.renamed`. #[path] is NOT yet honoured (ADR-049 lists it; module_path.rs does not implement it). This row pins what the extractor ACTUALLY emits today; both engines must match the emitted form, and #[path]-correct routing is a shared sp2 task."} + {"name": "path_attr_known_gap", "crate": "demo", "src_root": "/p/src", "file": "/p/src/renamed.rs", "expected_module": "demo.renamed", "reproducibility": "sp2", "note": "FALLBACK pin (ADR-049 Amendment 8). module_route rows drive module_path_for directly — a bare (crate, src_root, file) call with NO declaring-file context — and module_path_for stays the PURE-FILESYSTEM route by design: it emits demo.renamed even when some parent file carries `#[path=\"renamed.rs\"] mod logical;`. #[path]-aware routing is the SEPARATE entry point logical_module_path (mount overlay first, module_path_for default), pinned by the module_mounts section below; a route with no mount covering the file MUST be byte-identical to this row. Historical: this row was the pre-Amendment-8 known_gap pin of the then-wrong emission."} + ], + "module_mounts": [ + {"name": "path_mount_dir_module", "crate": "demo", "reproducibility": "sp2", + "files": { + "src/lib.rs": "#[cfg(unix)]\n#[path = \"unix/mod.rs\"]\nmod imp;\n\n#[cfg(windows)]\n#[path = \"windows/mod.rs\"]\nmod imp;\n\n#[cfg(unix)]\npub(crate) mod unix {\n pub(crate) use super::imp::*;\n}\n", + "src/unix/mod.rs": "pub(crate) fn spawn() {}\n", + "src/windows/mod.rs": "pub(crate) fn spawn() {}\n"}, + "expect": { + "src/lib.rs": "demo", + "src/unix/mod.rs": "demo.imp@cfg(unix)", + "src/windows/mod.rs": "demo.imp@cfg(windows)"}, + "note": "The tokio src/process shape (clarion-bdb1eccf48): cfg-twin `mod imp;` mounts plus an inline facade. The TWIN rule counts module-name occurrences in the declaring item list across BOTH inline `mod n { }` and decl `mod n;` forms; a twin mount appends the normalised @cfg(...) discriminant, so the two mounted files split. The inline facade `mod unix` is a unique name and emits demo.unix as an ENTITY (entity emission, not file routing — covered by the extractor, not this section)."}, + {"name": "path_mount_child_prefix", "crate": "demo", "reproducibility": "sp2", + "files": { + "src/lib.rs": "#[path = \"engine_v2/mod.rs\"]\nmod engine;\n#[path = \"compat_v3.rs\"]\nmod compat;\n", + "src/engine_v2/mod.rs": "pub mod worker;\n", + "src/engine_v2/worker.rs": "pub fn run() {}\n", + "src/compat_v3.rs": "pub mod extra;\n", + "src/compat_v3/extra.rs": "pub fn e() {}\n"}, + "expect": { + "src/engine_v2/mod.rs": "demo.engine", + "src/engine_v2/worker.rs": "demo.engine.worker", + "src/compat_v3.rs": "demo.compat", + "src/compat_v3/extra.rs": "demo.compat.extra"}, + "note": "Mounted SUBTREES: a /mod.rs target registers / as a logical prefix (children rewrite under the mount, trailing `mod` stem collapses); an x.rs target registers the exact file PLUS /x/ for its child directory (rustc's non-mod-rs child rule)."}, + {"name": "path_mount_chain", "crate": "demo", "reproducibility": "sp2", + "files": { + "src/lib.rs": "#[path = \"stage_a.rs\"]\nmod first;\n", + "src/stage_a.rs": "#[path = \"stage_b.rs\"]\nmod second;\npub fn a() {}\n", + "src/stage_b.rs": "pub fn b() {}\n"}, + "expect": { + "src/stage_a.rs": "demo.first", + "src/stage_b.rs": "demo.first.second"}, + "note": "Chained mounts resolve through the fixed point: stage_b.rs's logical parent is stage_a.rs's MOUNTED path (demo.first), not its filesystem path (demo.stage_a). A file-level #[path] target is relative to the DECLARING file's directory."}, + {"name": "path_mount_macro_invisible_fallback", "crate": "demo", "reproducibility": "sp2", + "files": { + "src/lib.rs": "macro_rules! mount {\n () => {\n #[path = \"hidden_impl.rs\"]\n mod hidden;\n };\n}\nmount!();\n", + "src/hidden_impl.rs": "pub fn h() {}\n"}, + "expect": { + "src/hidden_impl.rs": "demo.hidden_impl"}, + "note": "A #[path] inside an unexpanded macro invocation is INVISIBLE by dialect rule (no producer expands macros; the route must be reproducible from one file) — the target routes by filesystem. Load-bearing: do NOT attempt macro expansion."}, + {"name": "path_mount_inline_nested_decl", "crate": "demo", "reproducibility": "sp2", + "files": { + "src/lib.rs": "mod platform;\nmod host {\n #[path = \"generic.rs\"]\n mod imp;\n}\n", + "src/platform.rs": "mod backend {\n #[path = \"special.rs\"]\n mod imp;\n}\n", + "src/host/generic.rs": "pub fn g() {}\n", + "src/platform/backend/special.rs": "pub fn s() {}\n"}, + "expect": { + "src/platform.rs": "demo.platform", + "src/host/generic.rs": "demo.host.imp", + "src/platform/backend/special.rs": "demo.platform.backend.imp"}, + "note": "rustc's relative-path rule for #[path] INSIDE inline mod blocks: the target is relative to the would-be directory of the inline-module nesting — for a mod-rs file (lib.rs/main.rs/mod.rs) that starts at the file's own directory (src/ + host/ here); for a non-mod-rs file it starts at the file's stem directory (src/platform/ + backend/ here)."}, + {"name": "path_mount_inside_cfg_twin_inline_mod", "crate": "demo", "reproducibility": "sp2", + "files": { + "src/lib.rs": "#[cfg(feature = \"a\")]\nmod outer {\n #[path = \"x.rs\"]\n mod m;\n}\n\n#[cfg(not(feature = \"a\"))]\nmod outer {\n #[path = \"y.rs\"]\n mod m;\n}\n", + "src/outer/x.rs": "pub fn fx() {}\n", + "src/outer/y.rs": "pub fn fy() {}\n"}, + "expect": { + "src/outer/x.rs": "demo.outer@cfg(feature=\"a\").m", + "src/outer/y.rs": "demo.outer@cfg(not(feature=\"a\")).m"}, + "note": "The cfg-twin-inline-mod composition rule (ADR-049 Amendment 8, remediation): a mount declared INSIDE a cfg-twin inline mod composes the inline mod's @cfg-suffixed segment into its logical prefix — the SAME twin counting + cfg_discriminant rendering the AST walk applies to the inline mod's own entity path, so file-walk and AST-walk agree byte-for-byte and the two mounted files split. Composing the BARE inline name instead routes BOTH x.rs and y.rs to demo.outer.m (a duplicate-id family). The #[path] target still resolves against the BARE would-be directory (src/outer/), which carries no cfg."}, + {"name": "path_mount_lone_cfg_no_suffix", "crate": "demo", "reproducibility": "sp2", + "files": { + "src/lib.rs": "#[cfg(unix)]\n#[path = \"unix/mod.rs\"]\nmod imp;\n", + "src/unix/mod.rs": "pub(crate) fn spawn() {}\n"}, + "expect": { + "src/unix/mod.rs": "demo.imp"}, + "note": "The @cfg discriminant on a mount is TWIN-GATED (ADR-049 §3 discipline): a lone #[cfg(unix)] #[path] mod imp; with no same-name sibling in the declaring item list keeps the bare mounted path — NO @cfg suffix. Gate-off pin for the cross-producer corpus."}, + {"name": "path_mount_one_file_two_mounts_first_wins", "crate": "demo", "reproducibility": "sp2", + "files": { + "src/lib.rs": "#[path = \"shared_impl.rs\"]\nmod alpha;\n#[path = \"shared_impl.rs\"]\nmod beta;\nmod zeta;\n", + "src/zeta.rs": "#[path = \"shared2_impl.rs\"]\nmod gamma;\n", + "src/shared_impl.rs": "pub fn s() {}\n", + "src/shared2_impl.rs": "pub fn s2() {}\n"}, + "expect": { + "src/shared_impl.rs": "demo.alpha", + "src/shared2_impl.rs": "demo.zeta.gamma"}, + "note": "R5 determinism pin: one target file claimed by several mounts resolves to the FIRST by sorted (declaring-file path relative to the project root, byte offset of the declaration) — shared_impl.rs takes `alpha` (lower byte offset in lib.rs), never `beta`. shared2_impl.rs pins that an unconflicted mount from a later-sorting declaring file is untouched by the rule."} ], "entities": [ { @@ -488,6 +566,249 @@ {"qualname": "demo.m.&core%3A%3Acell%3A%3ACell.impl[Tr]", "kind": "impl"}, {"qualname": "demo.m.&core%3A%3Acell%3A%3ACell.impl[Tr].ok", "kind": "function"} ] + }, + { + "name": "unnamed_const_skip", + "crate": "demo", "module_path": "demo.m", "rel_path": "src/m.rs", + "reproducibility": "slice-1", + "note": "ADR-049 Amendment 9 (clarion-83870dc534): an unnamed `const _` is NOT an entity \u2014 `_` is non-identifying (no cfg/ordinal/content discriminant can rescue a repeated `_` without churning SEI) and un-nameable (nothing can ever target it), so the extractor emits neither the entity, nor its contains edge, nor any references sites from its declared type/initializer. The skip is UNCONDITIONAL on ident == \"_\" (a lone anonymous const is skipped too \u2014 twin-gating would make the emitted set sibling-dependent). Pinned as ABSENT expected rows, precedent nested_fn_is_not_an_entity: no demo.m._ row may ever appear. A finding inside a `const _` body attributes to the enclosing module, the same convention as nested fns/closures. The named LIMIT sibling pins the skip as name-targeted, not kind-targeted.", + "source": "pub const LIMIT: u32 = 10;\nconst _: () = assert!(true);\nconst _: () = assert!(true);\n", + "expected": [ + {"qualname": "demo.m", "kind": "module"}, + {"qualname": "demo.m.LIMIT", "kind": "const"} + ] + }, + { + "name": "unnamed_const_cfg_twin_skip", + "crate": "demo", "module_path": "demo.m", "rel_path": "src/m.rs", + "reproducibility": "slice-1", + "note": "ADR-049 Amendment 9, the hard repro (all 15 rust-analyzer residuals): two `const _` items gated on the SAME cfg predicate. cfg-twin discrimination is useless here \u2014 byte-identical attrs hand both twins the IDENTICAL @cfg suffix \u2014 so the unconditional skip is the only duplicate-free rendering. Both items are skipped; only the module emits.", + "source": "#[cfg(target_pointer_width = \"64\")]\nconst _: () = ();\n#[cfg(target_pointer_width = \"64\")]\nconst _: () = ();\n", + "expected": [ + {"qualname": "demo.m", "kind": "module"} + ] + }, + { + "name": "self_type_path_twin_trait", + "crate": "demo", "module_path": "demo.m", "rel_path": "src/m.rs", + "reproducibility": "slice-1", + "note": "ADR-049 Amendment 6 (clarion-8ff7f233fa), the Semaphore shape (tokio chan.rs bounded::Semaphore vs unbounded::Semaphore): two impls of the SAME trait for same-NAMED types from different modules. Pre-amendment both rendered demo.m.X.impl[T] (last-segment self-type base), seen_impl_ids spuriously merged them, and the like-named `go` methods collapsed at the writer's ON CONFLICT. Stage S of the residual-collision ladder (@cfg on bare keys -> S on post-cfg groups -> T on post-S groups -> method-@cfg on final keys) fires on the >=2 distinct written self-type-path witnesses and qualifies every member's base with its escaped written path (a%3A%3AX / b%3A%3AX). The witness carries NO generic args, so Amendment-3 positional normalization never falsely fires the gate. Pins that a second producer qualifies by the path AS WRITTEN, not a resolved/canonical path.", + "source": "pub trait T { fn go(&self); }\nmod a { pub struct X; }\nmod b { pub struct X; }\nimpl T for a::X { fn go(&self){} }\nimpl T for b::X { fn go(&self){} }\n", + "expected": [ + {"qualname": "demo.m", "kind": "module"}, + {"qualname": "demo.m.T", "kind": "trait"}, + {"qualname": "demo.m.a", "kind": "module"}, + {"qualname": "demo.m.a.X", "kind": "struct"}, + {"qualname": "demo.m.b", "kind": "module"}, + {"qualname": "demo.m.b.X", "kind": "struct"}, + {"qualname": "demo.m.a%3A%3AX.impl[T]", "kind": "impl"}, + {"qualname": "demo.m.a%3A%3AX.impl[T].go", "kind": "function"}, + {"qualname": "demo.m.b%3A%3AX.impl[T]", "kind": "impl"}, + {"qualname": "demo.m.b%3A%3AX.impl[T].go", "kind": "function"} + ] + }, + { + "name": "self_type_path_twin_inherent", + "crate": "demo", "module_path": "demo.m", "rel_path": "src/m.rs", + "reproducibility": "slice-1", + "note": "ADR-049 Amendment 6, the inherent variant: stage S keys on the post-cfg impl qualname, which for inherent impls is the impl# form -- the gate is impl-kind-agnostic. Two inherent blocks on same-named types from different modules share demo.m.X.impl#<> pre-amendment and (worse than a trait pair) MERGE under Option (b), chimera-parenting both methods under one entity. Distinct witnesses (a::X / b::X) fire S; both bases qualify.", + "source": "mod a { pub struct X; }\nmod b { pub struct X; }\nimpl a::X { pub fn f(&self){} }\nimpl b::X { pub fn g(&self){} }\n", + "expected": [ + {"qualname": "demo.m", "kind": "module"}, + {"qualname": "demo.m.a", "kind": "module"}, + {"qualname": "demo.m.a.X", "kind": "struct"}, + {"qualname": "demo.m.b", "kind": "module"}, + {"qualname": "demo.m.b.X", "kind": "struct"}, + {"qualname": "demo.m.a%3A%3AX.impl#<>", "kind": "impl"}, + {"qualname": "demo.m.a%3A%3AX.impl#<>.f", "kind": "function"}, + {"qualname": "demo.m.b%3A%3AX.impl#<>", "kind": "impl"}, + {"qualname": "demo.m.b%3A%3AX.impl#<>.g", "kind": "function"} + ] + }, + { + "name": "self_type_path_lone_unqualified_negative", + "crate": "demo", "module_path": "demo.m", "rel_path": "src/m.rs", + "reproducibility": "slice-1", + "note": "ADR-049 Amendment 6 NEGATIVE control (the S gate off): a LONE path-qualified self-type impl -- by far the common case (e.g. loomweave-cli guidance.rs `impl StorageResultExt for loomweave_storage::Result`) -- keeps the bare last-segment base demo.m.X.impl[T], byte-identical to pre-amendment. Qualification is paid ONLY inside a fired group; a lone impl never moves. Rejects the always-qualify alternative (mass SEI churn, ~2% of impls + their methods).", + "source": "pub trait T {}\nmod a { pub struct X; }\nimpl T for a::X {}\n", + "expected": [ + {"qualname": "demo.m", "kind": "module"}, + {"qualname": "demo.m.T", "kind": "trait"}, + {"qualname": "demo.m.a", "kind": "module"}, + {"qualname": "demo.m.a.X", "kind": "struct"}, + {"qualname": "demo.m.X.impl[T]", "kind": "impl"} + ] + }, + { + "name": "self_type_path_mixed_bare_and_qualified", + "crate": "demo", "module_path": "demo.m", "rel_path": "src/m.rs", + "reproducibility": "slice-1", + "note": "ADR-049 Amendment 6, mixed bare/qualified group: `impl T for X` + `impl T for b::X` share the bare key, distinct witnesses (X / b::X) fire S -- but a single-segment witness renders BYTE-IDENTICALLY to the bare base (escape_reserved is a no-op on an ident), so the bare member's id demo.m.X.impl[T] is unchanged and only the multi-segment member moves to b%3A%3AX. Within a fired group, minimal movement.", + "source": "pub trait T {}\nstruct X;\nmod b { pub struct X; }\nimpl T for X {}\nimpl T for b::X {}\n", + "expected": [ + {"qualname": "demo.m", "kind": "module"}, + {"qualname": "demo.m.T", "kind": "trait"}, + {"qualname": "demo.m.X", "kind": "struct"}, + {"qualname": "demo.m.b", "kind": "module"}, + {"qualname": "demo.m.b.X", "kind": "struct"}, + {"qualname": "demo.m.X.impl[T]", "kind": "impl"}, + {"qualname": "demo.m.b%3A%3AX.impl[T]", "kind": "impl"} + ] + }, + { + "name": "trait_path_twin", + "crate": "demo", "module_path": "demo.m", "rel_path": "src/m.rs", + "reproducibility": "slice-1", + "note": "ADR-049 Amendment 7 (clarion-fa8bcf8731), the Compat shape (tokio_util compat.rs `impl tokio::io::AsyncRead for Compat` vs `impl futures_io::AsyncRead for Compat`): two same-NAMED traits from different paths implemented for the SAME self type. Pre-amendment both rendered demo.m.Compat<$0>.impl[AsyncRead] (last-segment trait fragment) and both poll_read methods collapsed. Stage S is COLD (identical self-type witnesses); stage T groups by post-S qualname and fires on >=2 distinct qualified trait renderings, switching every member's impl[...] fragment to the full written trait path, escape_reserved over the ::-joined segment idents (impl[a%3A%3AAsyncRead] / impl[b%3A%3AAsyncRead]). Final-segment trait generic args keep the existing Amendment-4 rendering.", + "source": "struct Compat(T);\nmod a { pub trait AsyncRead { fn poll_read(&self); } }\nmod b { pub trait AsyncRead { fn poll_read(&self); } }\nimpl a::AsyncRead for Compat { fn poll_read(&self){} }\nimpl b::AsyncRead for Compat { fn poll_read(&self){} }\n", + "expected": [ + {"qualname": "demo.m", "kind": "module"}, + {"qualname": "demo.m.Compat", "kind": "struct"}, + {"qualname": "demo.m.a", "kind": "module"}, + {"qualname": "demo.m.a.AsyncRead", "kind": "trait"}, + {"qualname": "demo.m.b", "kind": "module"}, + {"qualname": "demo.m.b.AsyncRead", "kind": "trait"}, + {"qualname": "demo.m.Compat<$0>.impl[a%3A%3AAsyncRead]", "kind": "impl"}, + {"qualname": "demo.m.Compat<$0>.impl[a%3A%3AAsyncRead].poll_read", "kind": "function"}, + {"qualname": "demo.m.Compat<$0>.impl[b%3A%3AAsyncRead]", "kind": "impl"}, + {"qualname": "demo.m.Compat<$0>.impl[b%3A%3AAsyncRead].poll_read", "kind": "function"} + ] + }, + { + "name": "trait_path_twin_single_vs_multi", + "crate": "demo", "module_path": "demo.m", "rel_path": "src/m.rs", + "reproducibility": "slice-1", + "note": "ADR-049 Amendment 7, single-vs-multi-segment fired pair: `impl Tr for Foo` + `impl other::Tr for Foo`. T fires (two distinct renderings), but the single-segment written path renders byte-identically to the bare fragment, so the bare member keeps demo.m.Foo.impl[Tr] and only the multi-segment member moves to impl[other%3A%3ATr]. The as-written spelling is the witness -- a second producer must NOT resolve `Tr` to its defining path.", + "source": "struct Foo;\npub trait Tr { fn go(&self); }\nmod other { pub trait Tr { fn go(&self); } }\nimpl Tr for Foo { fn go(&self){} }\nimpl other::Tr for Foo { fn go(&self){} }\n", + "expected": [ + {"qualname": "demo.m", "kind": "module"}, + {"qualname": "demo.m.Foo", "kind": "struct"}, + {"qualname": "demo.m.Tr", "kind": "trait"}, + {"qualname": "demo.m.other", "kind": "module"}, + {"qualname": "demo.m.other.Tr", "kind": "trait"}, + {"qualname": "demo.m.Foo.impl[Tr]", "kind": "impl"}, + {"qualname": "demo.m.Foo.impl[Tr].go", "kind": "function"}, + {"qualname": "demo.m.Foo.impl[other%3A%3ATr]", "kind": "impl"}, + {"qualname": "demo.m.Foo.impl[other%3A%3ATr].go", "kind": "function"} + ] + }, + { + "name": "trait_path_lone_multi_segment", + "crate": "demo", "module_path": "demo.m", "rel_path": "src/m.rs", + "reproducibility": "slice-1", + "note": "ADR-049 Amendment 7 NEGATIVE control (the T gate off): a LONE multi-segment trait path keeps the bare last-segment fragment demo.m.Foo.impl[Read] -- byte-identical to pre-amendment, reinforcing the trait_method row (impl std::fmt::Display -> impl[Display]). Rejects the always-qualify alternative (6-41% of trait impls churn).", + "source": "mod deep { pub mod io { pub trait Read { fn r(&self); } } }\nstruct Foo;\nimpl deep::io::Read for Foo { fn r(&self){} }\n", + "expected": [ + {"qualname": "demo.m", "kind": "module"}, + {"qualname": "demo.m.deep", "kind": "module"}, + {"qualname": "demo.m.deep.io", "kind": "module"}, + {"qualname": "demo.m.deep.io.Read", "kind": "trait"}, + {"qualname": "demo.m.Foo", "kind": "struct"}, + {"qualname": "demo.m.Foo.impl[Read]", "kind": "impl"}, + {"qualname": "demo.m.Foo.impl[Read].r", "kind": "function"} + ] + }, + { + "name": "trait_path_cfg_twin_unqualified", + "crate": "demo", "module_path": "demo.m", "rel_path": "src/m.rs", + "reproducibility": "slice-1", + "note": "ADR-049 Amendments 6/7 LADDER-ORDERING pin: cfg-gated twins whose trait paths ALSO differ (#[cfg(unix)] impl a::Tr / #[cfg(windows)] impl b::Tr). Stage 1 (@cfg, computed on the BARE pre-cfg keys exactly as before Amendment 6) splits them first; the post-cfg groups are singletons, so S and T stay COLD and the ids are byte-identical to the pre-amendment @cfg form -- demo.m.Foo.impl[Tr]@cfg(unix), with NO path qualification. This no-churn invariant is WHY the ladder runs @cfg before S/T; an independent-gates design would have qualified (and churned) these.", + "source": "struct Foo;\nmod a { pub trait Tr { fn go(&self); } }\nmod b { pub trait Tr { fn go(&self); } }\n#[cfg(unix)] impl a::Tr for Foo { fn go(&self){} }\n#[cfg(windows)] impl b::Tr for Foo { fn go(&self){} }\n", + "expected": [ + {"qualname": "demo.m", "kind": "module"}, + {"qualname": "demo.m.Foo", "kind": "struct"}, + {"qualname": "demo.m.a", "kind": "module"}, + {"qualname": "demo.m.a.Tr", "kind": "trait"}, + {"qualname": "demo.m.b", "kind": "module"}, + {"qualname": "demo.m.b.Tr", "kind": "trait"}, + {"qualname": "demo.m.Foo.impl[Tr]@cfg(unix)", "kind": "impl"}, + {"qualname": "demo.m.Foo.impl[Tr]@cfg(unix).go", "kind": "function"}, + {"qualname": "demo.m.Foo.impl[Tr]@cfg(windows)", "kind": "impl"}, + {"qualname": "demo.m.Foo.impl[Tr]@cfg(windows).go", "kind": "function"} + ] + }, + { + "name": "impl_ladder_self_splits_before_trait", + "crate": "demo", "module_path": "demo.m", "rel_path": "src/m.rs", + "reproducibility": "slice-1", + "note": "ADR-049 Amendments 6/7 composition row: the MIXED pair (`impl a::Tr for c::X` + `impl b::Tr for d::X`) differs on BOTH the self-type path and the trait path. Stage S fires first (distinct self-type witnesses c::X / d::X) and already splits the group; the post-S qualnames are distinct singletons, so stage T stays COLD and the trait fragment keeps its bare last segment -- demo.m.c%3A%3AX.impl[Tr], NOT c%3A%3AX.impl[a%3A%3ATr]. Minimal qualification: each stage only fires on a residual collision left by the previous one.", + "source": "mod a { pub trait Tr { fn go(&self); } }\nmod b { pub trait Tr { fn go(&self); } }\nmod c { pub struct X; }\nmod d { pub struct X; }\nimpl a::Tr for c::X { fn go(&self){} }\nimpl b::Tr for d::X { fn go(&self){} }\n", + "expected": [ + {"qualname": "demo.m", "kind": "module"}, + {"qualname": "demo.m.a", "kind": "module"}, + {"qualname": "demo.m.a.Tr", "kind": "trait"}, + {"qualname": "demo.m.b", "kind": "module"}, + {"qualname": "demo.m.b.Tr", "kind": "trait"}, + {"qualname": "demo.m.c", "kind": "module"}, + {"qualname": "demo.m.c.X", "kind": "struct"}, + {"qualname": "demo.m.d", "kind": "module"}, + {"qualname": "demo.m.d.X", "kind": "struct"}, + {"qualname": "demo.m.c%3A%3AX.impl[Tr]", "kind": "impl"}, + {"qualname": "demo.m.c%3A%3AX.impl[Tr].go", "kind": "function"}, + {"qualname": "demo.m.d%3A%3AX.impl[Tr]", "kind": "impl"}, + {"qualname": "demo.m.d%3A%3AX.impl[Tr].go", "kind": "function"} + ] + }, + { + "name": "self_type_path_leading_colon_twin", + "crate": "demo", "module_path": "demo.m", "rel_path": "src/m.rs", + "reproducibility": "slice-1", + "note": "ADR-049 Amendment 6 witness symmetry (remediation): `impl T for a::X` + `impl T for ::a::X` is valid Rust when a local `mod a` shadows an extern crate `a`. The leading `::` is PART of the written self-type-path witness (symmetric with the Amendment-7 trait-path rule), so the two witnesses differ, stage S fires, and the qualified bases render a%3A%3AX vs %3A%3Aa%3A%3AX (the leading `::` contributes a leading %3A%3A). Without the leading-colon contribution both witnesses read a::X, the gate stays cold, and the two impls + their `go` methods collide.", + "source": "pub trait T { fn go(&self); }\nmod a { pub struct X; }\nimpl T for a::X { fn go(&self){} }\nimpl T for ::a::X { fn go(&self){} }\n", + "expected": [ + {"qualname": "demo.m", "kind": "module"}, + {"qualname": "demo.m.T", "kind": "trait"}, + {"qualname": "demo.m.a", "kind": "module"}, + {"qualname": "demo.m.a.X", "kind": "struct"}, + {"qualname": "demo.m.a%3A%3AX.impl[T]", "kind": "impl"}, + {"qualname": "demo.m.a%3A%3AX.impl[T].go", "kind": "function"}, + {"qualname": "demo.m.%3A%3Aa%3A%3AX.impl[T]", "kind": "impl"}, + {"qualname": "demo.m.%3A%3Aa%3A%3AX.impl[T].go", "kind": "function"} + ] + }, + { + "name": "impl_ladder_self_then_trait_residual", + "crate": "demo", "module_path": "demo.m", "rel_path": "src/m.rs", + "reproducibility": "slice-1", + "note": "ADR-049 Amendments 6/7 S-then-T residual row: three impls share the bare key demo.m.X.impl[Tr]. Stage S fires (witnesses c::X / d::X) and qualifies every base — but the two c::X members STILL collide post-S (demo.m.c%3A%3AX.impl[Tr] twice), so stage T fires on that post-S group and switches BOTH members' fragments to the qualified trait rendering (impl[a%3A%3ATr] / impl[b%3A%3ATr]). The d::X member's post-S group is a SINGLETON, so T stays cold for it and it keeps the bare impl[Tr] fragment — minimal qualification, each stage fires only on the residual the previous one left.", + "source": "mod a { pub trait Tr { fn go(&self); } }\nmod b { pub trait Tr { fn go(&self); } }\nmod c { pub struct X; }\nmod d { pub struct X; }\nimpl a::Tr for c::X { fn go(&self){} }\nimpl b::Tr for c::X { fn go(&self){} }\nimpl a::Tr for d::X { fn go(&self){} }\n", + "expected": [ + {"qualname": "demo.m", "kind": "module"}, + {"qualname": "demo.m.a", "kind": "module"}, + {"qualname": "demo.m.a.Tr", "kind": "trait"}, + {"qualname": "demo.m.b", "kind": "module"}, + {"qualname": "demo.m.b.Tr", "kind": "trait"}, + {"qualname": "demo.m.c", "kind": "module"}, + {"qualname": "demo.m.c.X", "kind": "struct"}, + {"qualname": "demo.m.d", "kind": "module"}, + {"qualname": "demo.m.d.X", "kind": "struct"}, + {"qualname": "demo.m.c%3A%3AX.impl[a%3A%3ATr]", "kind": "impl"}, + {"qualname": "demo.m.c%3A%3AX.impl[a%3A%3ATr].go", "kind": "function"}, + {"qualname": "demo.m.c%3A%3AX.impl[b%3A%3ATr]", "kind": "impl"}, + {"qualname": "demo.m.c%3A%3AX.impl[b%3A%3ATr].go", "kind": "function"}, + {"qualname": "demo.m.d%3A%3AX.impl[Tr]", "kind": "impl"}, + {"qualname": "demo.m.d%3A%3AX.impl[Tr].go", "kind": "function"} + ] + }, + { + "name": "method_cfg_twin_in_s_fired_merged_blocks", + "crate": "demo", "module_path": "demo.m", "rel_path": "src/m.rs", + "reproducibility": "slice-1", + "note": "ADR-049 Amendment 5 composed with Amendment 6 (remediation): the method-@cfg twin counter keys on the FINAL post-S/T impl qualname. An S-fired group whose a::X member is TWO merged same-witness blocks carrying cfg-twin `go` methods: both blocks land on the single S-qualified impl entity demo.m.a%3A%3AX.impl[T], so the two `go` methods are twins on the FINAL key and each carries its own @cfg suffix on the %3A%3A-qualified method id. The b::X member's lone `go` collects no suffix.", + "source": "pub trait T { fn go(&self); }\nmod a { pub struct X; }\nmod b { pub struct X; }\nimpl T for a::X { #[cfg(unix)] fn go(&self){} }\nimpl T for a::X { #[cfg(windows)] fn go(&self){} }\nimpl T for b::X { fn go(&self){} }\n", + "expected": [ + {"qualname": "demo.m", "kind": "module"}, + {"qualname": "demo.m.T", "kind": "trait"}, + {"qualname": "demo.m.a", "kind": "module"}, + {"qualname": "demo.m.a.X", "kind": "struct"}, + {"qualname": "demo.m.b", "kind": "module"}, + {"qualname": "demo.m.b.X", "kind": "struct"}, + {"qualname": "demo.m.a%3A%3AX.impl[T]", "kind": "impl"}, + {"qualname": "demo.m.a%3A%3AX.impl[T].go@cfg(unix)", "kind": "function"}, + {"qualname": "demo.m.a%3A%3AX.impl[T].go@cfg(windows)", "kind": "function"}, + {"qualname": "demo.m.b%3A%3AX.impl[T]", "kind": "impl"}, + {"qualname": "demo.m.b%3A%3AX.impl[T].go", "kind": "function"} + ] } ] } diff --git a/tests/conformance/test_loomweave_rust_qualname_parity.py b/tests/conformance/test_loomweave_rust_qualname_parity.py index 8d1bea55..0d6e1d74 100644 --- a/tests/conformance/test_loomweave_rust_qualname_parity.py +++ b/tests/conformance/test_loomweave_rust_qualname_parity.py @@ -8,11 +8,14 @@ the *second producer*, it MINTS the same locator string and never parses it. Provenance — re-vendor when Loomweave bumps the corpus: - source: loomweave rc4 @ cab95a1695a45f875933d8c4ac0e800e793c9305 - (fixtures/qualnames_rust.json, blob ed436c825861ad2b9e313f9211f5a55583b80c7c, + source: loomweave rc4 @ 113c2e2217131fe67f7edb9ea42a2f9eeb48642b + (fixtures/qualnames_rust.json, blob d81fb97544ed1c26b50198556022662e5387a130, extractor-generated, locked by crates/loomweave-plugin-rust/tests/qualname_conformance.rs) - vendored byte-identical to tests/conformance/qualnames_rust.json (2026-06-10). + vendored byte-identical to tests/conformance/qualnames_rust.json (2026-06-11, + the Amendments 4-9 batch re-vendor — one blob covers the 4-5 AND 6-9 changeset + letters; 49 entity rows + 6 module_route rows + the NEW module_mounts section, + 8 rows). Drift alarm (two layers — wardline-868908944b): 1. Byte-pin (default suite): ``UPSTREAM_BLOB_SHA`` below pins the vendored file's @@ -65,6 +68,19 @@ change-set (docs/integration/2026-06-09-loomweave-rust-qualname-phase1b-changeset.md, amended for amendment 3), which is the authoritative description of this corpus. Where any doc and the live extractor + this corpus diverge, the extractor + corpus are the oracle. + The 113c2e2 re-vendor (ADR-049 Amendments 4-9, ONE batch covering both the 4-5 and + 6-9 changeset letters) adds 14 entity rows + the module_mounts section and de-gaps + `path_attr_known_gap` (now a FALLBACK pin of the unchanged pure-filesystem route): + **Amendments 4+5** — generic-arg escape pipeline (escape_reserved(strip_ws(arg)) at + every concrete-arg + non-Type::Path self-type render site) and method-level @cfg on + cfg-twin impl fns keyed on the FINAL impl qualname. **Amendments 6+7** — the + residual-collision LADDER (@cfg -> stage S self-type written path -> stage T trait + written path -> method-@cfg), twin-gated end to end (rows self_type_path_* / + trait_path_* / impl_ladder_* / method_cfg_twin_in_s_fired_merged_blocks). + **Amendment 8** — the #[path] mount overlay (module_mounts section, mounted routes + end-to-end incl. chains, macro-invisibility, cfg-twin composition, R5 first-wins). + **Amendment 9** — `const _` skip-emission (unnamed_const_skip pins the skip as an + ABSENT expected row; the ordered-equality gate self-enforces it). Status: the Rust frontend (``wardline.rust.*``) now EXISTS (slice-1 WP2 landed), so the *producer-parity* tests below run live (``_rust_producer`` resolves @@ -98,9 +114,9 @@ _CORPUS: dict[str, Any] = json.loads(_CORPUS_PATH.read_text("utf-8")) # The git blob hash of the vendored corpus as committed upstream (loomweave rc4 -# @ cab95a1695a45f875933d8c4ac0e800e793c9305). Re-vendors update this constant in +# @ 113c2e2217131fe67f7edb9ea42a2f9eeb48642b). Re-vendors update this constant in # the SAME commit as the new bytes — see the RE-VENDOR PROCEDURE in the header. -UPSTREAM_BLOB_SHA = "5f3a2fa52aa43ddbc7adbb4f84c9282ee1ca1cf8" +UPSTREAM_BLOB_SHA = "d81fb97544ed1c26b50198556022662e5387a130" _KNOWN_TIERS = {"slice-1", "sp2"} # The a209fc7 corpus carries the FULL ten-kind ADR-049 surface (leaf_item_kinds / @@ -136,10 +152,11 @@ def _expected_all_pairs(case: dict[str, Any]) -> list[tuple[str, str]]: def test_corpus_shape() -> None: - for key in ("_doc", "_dialect_summary", "_consumer_comparison", "module_route", "entities"): + for key in ("_doc", "_dialect_summary", "_consumer_comparison", "module_route", "module_mounts", "entities"): assert key in _CORPUS, f"vendored corpus is missing the '{key}' section" assert _CORPUS["entities"], "corpus has no entity cases" assert _CORPUS["module_route"], "corpus has no module_route cases" + assert _CORPUS["module_mounts"], "corpus has no module_mounts cases (ADR-049 Amendment 8)" assert _CORPUS["_consumer_comparison"].strip(), "the comparison contract must travel with the data" @@ -148,6 +165,7 @@ def test_reproducibility_tiers_are_known() -> None: # slice-1-runs / sp2-xfails gating below. tiers = {c["reproducibility"] for c in _CORPUS["entities"]} tiers |= {r["reproducibility"] for r in _CORPUS["module_route"]} + tiers |= {r["reproducibility"] for r in _CORPUS["module_mounts"]} assert tiers <= _KNOWN_TIERS, f"unhandled reproducibility tiers: {tiers - _KNOWN_TIERS}" @@ -252,9 +270,25 @@ def test_entity_qualnames(case: dict[str, Any]) -> None: @pytest.mark.parametrize("route", _CORPUS["module_route"], ids=lambda r: r["name"]) def test_module_route(route: dict[str, Any]) -> None: _, rust_qualname = _rust_producer() - # SP2 landed: the sp2 `path_attr_known_gap` row asserts for real — #[path] stays - # un-honoured by BOTH producers (the shared known gap), so the mechanical route IS - # the expected byte form and passes without special-casing. - # WP2 contract: route a file to its dotted module given the crate + src_root. + # module_route rows drive the PURE-FILESYSTEM route directly — a bare + # (crate, src_root, file) call with NO declaring-file context. `path_attr_known_gap` + # is the Amendment-8 FALLBACK pin: #[path]-aware routing is the SEPARATE + # logical_module_path entry point (test_module_mounts below); a route with no mount + # covering the file MUST be byte-identical to this one. got = rust_qualname.rust_module_route(crate=route["crate"], src_root=route["src_root"], file=route["file"]) assert got == route["expected_module"] + + +@pytest.mark.parametrize("case", _CORPUS["module_mounts"], ids=lambda c: c["name"]) +def test_module_mounts(case: dict[str, Any]) -> None: + """ADR-049 Amendment 8: the #[path] mount overlay routes end-to-end — mount + discovery over the case's file map (rustc's relative-path rule, cross-form @cfg + twins, cfg-twin inline-mod prefix composition), fixed-point resolution (chains, + R5 first-wins), and the filesystem fallback for macro-invisible mounts.""" + _rust_producer() # tree-sitter gate (skips when the wardline[rust] extra is absent) + rust_mounts: Any = importlib.import_module("wardline.rust.mounts") + # Every module_mounts case lays its files under `src/` at the project root — + # src_root is the constant "src" by corpus construction. + overlay = rust_mounts.build_mount_overlay(case["files"], crate=case["crate"], src_root="src") + for file, expected in case["expect"].items(): + assert overlay.logical_module_path(file) == expected, f"{case['name']}: {file}" diff --git a/tests/golden/identity/rust/_capture.py b/tests/golden/identity/rust/_capture.py index 5f0a11a9..8ecffabb 100644 --- a/tests/golden/identity/rust/_capture.py +++ b/tests/golden/identity/rust/_capture.py @@ -39,7 +39,7 @@ from wardline.core.paths import weft_config_path from wardline.core.run import run_scan from wardline.rust import qualname as q -from wardline.rust.analyzer import _module_for +from wardline.rust.analyzer import _build_overlays, _module_for from wardline.rust.crate_roots import discover_crate_roots from wardline.rust.edges import RustParsedFile, discover_rust_edges, index_rust_file @@ -73,12 +73,14 @@ def _parsed_files(root: Path) -> list[RustParsedFile]: cfg = config_mod.load(weft_config_path(resolved_root), explicit=False) files = discover(resolved_root, cfg, confine_to_root=True, suffixes=frozenset({".rs"})) crate_roots = discover_crate_roots(resolved_root) + sources = {file: file.read_text(encoding="utf-8") for file in files} + # Same Amendment-8 pre-pass as the analyzer: per-crate #[path] mount overlays. + overlays = _build_overlays(sources, resolved_root, crate_roots) parsed: list[RustParsedFile] = [] for file in files: - source = file.read_text(encoding="utf-8") - module = _module_for(file, resolved_root, crate_roots) + module = _module_for(file, resolved_root, crate_roots, overlays) relpath = file.resolve().relative_to(resolved_root).as_posix() - parsed.append(index_rust_file(source, module=module, path=relpath)) + parsed.append(index_rust_file(source=sources[file], module=module, path=relpath)) return parsed diff --git a/tests/unit/rust/test_crate_roots.py b/tests/unit/rust/test_crate_roots.py index b4e5f096..bdbe09a3 100644 --- a/tests/unit/rust/test_crate_roots.py +++ b/tests/unit/rust/test_crate_roots.py @@ -20,6 +20,7 @@ from __future__ import annotations import os +from functools import partial from pathlib import Path import pytest @@ -134,7 +135,7 @@ def test_in_src_files_get_the_oracle_crate_prefixed_route(tmp_path: Path) -> Non (crate / "src" / "a" / "mod.rs").write_text("", encoding="utf-8") roots = discover_crate_roots(tmp_path) - route = analyzer._module_for # noqa: SLF001 - the unit under test + route = partial(analyzer._module_for, overlays={}) # noqa: SLF001 - the unit under test (overlay-free: the filesystem default) assert route(crate / "src" / "lib.rs", tmp_path, roots) == "my_app" assert route(crate / "src" / "a" / "b.rs", tmp_path, roots) == "my_app.a.b" assert route(crate / "src" / "a" / "mod.rs", tmp_path, roots) == "my_app.a" @@ -147,7 +148,7 @@ def test_workspace_member_files_route_to_their_own_crates(tmp_path: Path) -> Non m2 = _write_crate(tmp_path, "m2", '[package]\nname = "m_two"\n', lib=False) roots = discover_crate_roots(tmp_path) - route = analyzer._module_for # noqa: SLF001 + route = partial(analyzer._module_for, overlays={}) # noqa: SLF001 assert route(m1 / "src" / "lib.rs", tmp_path, roots) == "m_one" assert route(m2 / "src" / "main.rs", tmp_path, roots) == "m_two" @@ -166,7 +167,7 @@ def test_out_of_src_files_get_the_out_branded_crate_route(tmp_path: Path) -> Non (crate / "build.rs").write_text("", encoding="utf-8") roots = discover_crate_roots(tmp_path) - route = analyzer._module_for # noqa: SLF001 + route = partial(analyzer._module_for, overlays={}) # noqa: SLF001 assert route(crate / "build.rs", tmp_path, roots) == "c_app.#out.build" assert route(crate / "tests" / "integration.rs", tmp_path, roots) == "c_app.#out.tests.integration" @@ -183,7 +184,7 @@ def test_class2_route_cannot_collide_with_an_in_src_twin(tmp_path: Path) -> None (crate / "tests" / "integration.rs").write_text("", encoding="utf-8") roots = discover_crate_roots(tmp_path) - route = analyzer._module_for # noqa: SLF001 + route = partial(analyzer._module_for, overlays={}) # noqa: SLF001 in_src = route(crate / "src" / "tests" / "integration.rs", tmp_path, roots) out_of_src = route(crate / "tests" / "integration.rs", tmp_path, roots) assert in_src == "c_app.tests.integration" # class 1: the conformance-bearing route @@ -202,7 +203,7 @@ def test_no_crate_files_get_the_relpath_pure_constant_crate_route(tmp_path: Path roots = discover_crate_roots(tmp_path) assert roots.crate_dir_for(tmp_path / "src" / "m.rs") is None # no lib.rs/main.rs -> no root - got = analyzer._module_for(tmp_path / "src" / "m.rs", tmp_path, roots) # noqa: SLF001 + got = analyzer._module_for(tmp_path / "src" / "m.rs", tmp_path, roots, {}) # noqa: SLF001 assert got == "crate.#out.src.m" # NOT f"{tmp_path.name}..." — root-name-independent diff --git a/tests/unit/rust/test_index.py b/tests/unit/rust/test_index.py index 1b9cf52c..cc465ae3 100644 --- a/tests/unit/rust/test_index.py +++ b/tests/unit/rust/test_index.py @@ -276,3 +276,35 @@ def run() -> list[tuple[str, str, str | None, int, int]]: ] assert run() == run() + + +def test_identical_witness_twins_do_not_fire_stage_s() -> None: + # ADR-049 Amendment 6, known residual BY DESIGN: coherence-illegal twins with the + # SAME written self-type path carry identical witnesses — no witness can split + # them, stage S stays cold, and the two blocks still MERGE onto one bare key + # (upstream, `duplicate_ids()` is the alarm; un-corpused, pinned here). + src = ( + "pub trait T { fn go(&self); }\n" + "mod a { pub struct X; }\n" + "impl T for a::X { fn go(&self){} }\n" + "impl T for a::X { fn other(&self){} }\n" + ) + rows = [(e.qualname, e.kind) for e in discover_rust_entities(src, module="demo.m")] + assert rows == [ + ("demo.m", "module"), + ("demo.m.T", "trait"), + ("demo.m.a", "module"), + ("demo.m.a.X", "struct"), + ("demo.m.X.impl[T]", "impl"), + ("demo.m.X.impl[T].go", "method"), + ("demo.m.X.impl[T].other", "method"), + ] + + +def test_unnamed_const_skip_leaves_named_siblings_untouched() -> None: + # ADR-049 Amendment 9 unit guard (the corpus pins the skip via ordered-equality; + # this pins the SEMANTIC kind view too): `const _` emits nothing — no entity, no + # twin-count participation — while named consts are unaffected. + src = "pub const LIMIT: u32 = 10;\nconst _: () = assert!(true);\nconst _: () = assert!(true);\n" + rows = [(e.qualname, e.kind) for e in discover_rust_entities(src, module="demo.m")] + assert rows == [("demo.m", "module"), ("demo.m.LIMIT", "const")] diff --git a/tests/unit/rust/test_mounts.py b/tests/unit/rust/test_mounts.py new file mode 100644 index 00000000..a8d7d74e --- /dev/null +++ b/tests/unit/rust/test_mounts.py @@ -0,0 +1,92 @@ +# tests/unit/rust/test_mounts.py +"""Unit guards for the #[path] mount overlay (ADR-049 Amendment 8) — the behaviors the +vendored ``module_mounts`` corpus does NOT pin (mounted routes end-to-end live in +tests/conformance/test_loomweave_rust_qualname_parity.py::test_module_mounts): + +* a mount CYCLE drops to the filesystem fallback (normative in the amendment text but + un-corpused — these pin Wardline's deterministic resolution); +* a file tree-sitter cannot fully parse contributes NO mounts (fail-closed, matching + the analyzer's refuse-to-half-analyze posture); +* a ``cfg_attr``-delivered ``path`` and a ``#[path]`` on an INLINE mod are not mounts; +* an un-mounted file routes byte-identically to ``rust_module_route`` (the overlay's + default IS the fallback pin). +""" + +from __future__ import annotations + +import pytest + +pytest.importorskip("tree_sitter", reason="wardline[rust] extra not installed") + +from wardline.rust.mounts import build_mount_overlay # noqa: E402 +from wardline.rust.qualname import rust_module_route # noqa: E402 + + +def test_mount_cycle_drops_to_filesystem_fallback() -> None: + # a.rs mounts b.rs; b.rs mounts a.rs — resolving either re-enters itself. The + # amendment says cycles drop to the filesystem fallback; the corpus does not pin + # WHICH link falls back, so this pins Wardline's resolution: the re-entered file + # resolves by filesystem, and the chain stays deterministic from there. + files = { + "src/a.rs": '#[path = "b.rs"]\nmod from_a;\n', + "src/b.rs": '#[path = "a.rs"]\nmod from_b;\n', + } + overlay = build_mount_overlay(files, crate="demo", src_root="src") + # Resolving b walks b -> (declared in a) -> a -> (declared in b) -> b: the + # RE-ENTERED file (the one being resolved) drops to its filesystem route, so each + # file's chain terminates at itself. Order-independent: _resolve never consults + # the memo mid-chain. + assert overlay.logical_module_path("src/b.rs") == "demo.b.from_b.from_a" + assert overlay.logical_module_path("src/a.rs") == "demo.a.from_a.from_b" + + +def test_self_mount_terminates() -> None: + # Degenerate self-mount: a file mounting itself must terminate via the cycle rule. + files = {"src/loop.rs": '#[path = "loop.rs"]\nmod me;\n'} + overlay = build_mount_overlay(files, crate="demo", src_root="src") + assert overlay.logical_module_path("src/loop.rs") == "demo.loop.me" + + +def test_unparseable_file_contributes_no_mounts() -> None: + # Fail-closed: tree-sitter error-recovery in the declaring file drops its mounts — + # no routing is derived from a file the analyzer refuses to analyze. + files = { + "src/lib.rs": '#[path = "impl_a.rs"]\nmod good; fn broken( {\n', + "src/impl_a.rs": "pub fn a() {}\n", + } + overlay = build_mount_overlay(files, crate="demo", src_root="src") + assert overlay.logical_module_path("src/impl_a.rs") == "demo.impl_a" + + +def test_cfg_attr_delivered_path_is_not_a_mount() -> None: + # ADR-049 Amendment 8: only a LITERAL #[path] attribute is a mount — a + # #[cfg_attr(pred, path = "…")] is invisible by dialect rule (no producer + # evaluates cfg predicates); the target routes by filesystem. + files = { + "src/lib.rs": '#[cfg_attr(unix, path = "unix_impl.rs")]\nmod imp;\n', + "src/unix_impl.rs": "pub fn u() {}\n", + } + overlay = build_mount_overlay(files, crate="demo", src_root="src") + assert overlay.logical_module_path("src/unix_impl.rs") == "demo.unix_impl" + + +def test_path_on_inline_mod_is_not_a_mount() -> None: + # The normative rule covers the DECL form (`mod name;`) only: a #[path] on an + # inline `mod name { … }` mounts nothing, and the inline body still nests by name. + files = { + "src/lib.rs": '#[path = "elsewhere"]\nmod host { #[path = "real.rs"] mod imp; }\n', + "src/host/real.rs": "pub fn r() {}\n", + } + overlay = build_mount_overlay(files, crate="demo", src_root="src") + # The inner decl-mod mount still resolves against the BARE would-be directory + # (src/host/), unaffected by the inline mod's ignored #[path]. + assert overlay.logical_module_path("src/host/real.rs") == "demo.host.imp" + + +def test_unmounted_file_matches_pure_filesystem_route() -> None: + # The overlay's no-mount default MUST be byte-identical to rust_module_route + # (the de-gapped `path_attr_known_gap` discipline). + files = {"src/lib.rs": "pub fn f() {}\n"} + overlay = build_mount_overlay(files, crate="demo", src_root="src") + for file in ("src/lib.rs", "src/config.rs", "src/plugin/host.rs", "src/plugin/mod.rs"): + assert overlay.logical_module_path(file) == rust_module_route(crate="demo", src_root="src", file=file) From 005eebaa27fdbc55e5e1d3dbbb5bf8d4ad5928ae Mon Sep 17 00:00:00 2001 From: John Morrissey Date: Thu, 11 Jun 2026 09:12:06 +1000 Subject: [PATCH 095/186] =?UTF-8?q?feat(loomweave):=20ADR-036=20ResolveReq?= =?UTF-8?q?uest=20plugin=20hint=20=E2=80=94=20client=20slice=20(closes=20w?= =?UTF-8?q?ardline-7eccab28ee)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Loomweave landed the optional batch-scoped 'plugin' field on POST /api/wardline/resolve (their rc4 0c68ddd, per the agreed shape in docs/integration/2026-06-11-wardline-resolve-plugin-hint-proposal.md). This is the Wardline side: - LoomweaveClient.resolve(qualnames, *, plugin=None): sends the hint verbatim when given, omits the field when unhinted (legal forever). Hinted 4xx downgrades the chunk to fail-soft unresolved (an older deny_unknown_fields server 400s on the field; identity enrichment degrades, never crashes); UNHINTED 4xx stays loud (cannot be hint-field skew — a real request bug like INVALID_PATH must stay diagnosable; the pre-existing pin holds). - Threading at every producer-knowing call site: attest boundary enrichment + decorator coverage send plugin='python' (Python-surface by construction); the Filigree identity-attach path derives the producer from the finding's rule family via new plugin_for_finding() (RS-WL-* -> rust), so a Rust finding now mints rust:function: locators and rust:function entity associations instead of the hardcoded python:function. Dossier entities (user-supplied, language unknown) stay unhinted by design. - Protocols widened (_ResolveClient, _LoomweaveClient); test fakes record the hint; new pins: hint-on-wire / omission / hinted-4xx-soft / unhinted-4xx-loud (client), rust threading end-to-end (locator + resolve hint + entity_kind), attest sends python on every hop. Live e2e against the new server rides the deployed-binary rebuild on the loomweave side. Suite 3684 green, ruff/mypy clean, self-scan gate exit 0. Co-Authored-By: Claude Fable 5 --- CHANGELOG.md | 13 ++++ src/wardline/core/attest.py | 4 +- src/wardline/core/filigree_issue.py | 33 +++++++--- src/wardline/core/scan_file_workflow.py | 2 + src/wardline/loomweave/client.py | 24 +++++++- src/wardline/loomweave/dossier_sources.py | 16 +++-- src/wardline/weft_decorator_coverage.py | 4 +- src/wardline/weft_dossier.py | 2 +- tests/unit/core/test_attest.py | 9 ++- tests/unit/core/test_filigree_issue.py | 65 +++++++++++++++++++- tests/unit/core/test_weft_dossier.py | 3 +- tests/unit/loomweave/test_client.py | 38 ++++++++++++ tests/unit/loomweave/test_dossier_sources.py | 3 +- tests/unit/mcp/test_server_dossier.py | 2 +- 14 files changed, 194 insertions(+), 24 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 83df87fd..d7ca1aeb 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -158,6 +158,19 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 through cleanly to the legacy/off rungs (emit stays soft-fail) (weft-23574069a1). ### Changed +- **Loomweave resolve client: ADR-036 plugin hint + hinted fail-soft.** + `LoomweaveClient.resolve()` accepts an optional batch-scoped `plugin` hint + (contract: `docs/integration/2026-06-11-wardline-resolve-plugin-hint-proposal.md`) + and threads it from every call site that knows the producer: attest boundary + enrichment and decorator coverage send `python` (Python-surface by + construction); the Filigree identity-attach path derives the producer from + the finding's rule family (`RS-WL-*` → `rust`), so a Rust finding now mints a + `rust:function:` locator and a `rust:function` entity association instead of + the previously hardcoded `python:function`. A 4xx on a *hinted* request + degrades fail-soft to `unresolved` (an older Loomweave under + `deny_unknown_fields` rejects the new field; identity enrichment must + degrade, not crash); an unhinted 4xx stays loud. User-supplied dossier + entities stay unhinted — the contract never fabricates a hint. - **Rust qualname dialect: ADR-049 Amendments 6–9 (Loomweave lockstep, one batched re-vendor covering 4–9).** Closes the four Sprint-4 gold-blocker collision families at the dialect level, mirroring the authoritative diff --git a/src/wardline/core/attest.py b/src/wardline/core/attest.py index 6475c0eb..e9900fb6 100644 --- a/src/wardline/core/attest.py +++ b/src/wardline/core/attest.py @@ -147,7 +147,9 @@ def _enrich_seis(boundaries: list[dict[str, Any]], loomweave_client: Any) -> str resolved_any = False for boundary in boundaries: try: - binding = resolve_entity_binding(loomweave_client, resolver, boundary["qualname"]) + # Boundaries come from the Python AnalysisContext (declared_qualnames), + # so the producer is known — send the ADR-036 plugin hint. + binding = resolve_entity_binding(loomweave_client, resolver, boundary["qualname"], plugin="python") except Exception: # noqa: BLE001 — per-boundary fail-soft, see docstring continue if binding is not None and binding.sei: diff --git a/src/wardline/core/filigree_issue.py b/src/wardline/core/filigree_issue.py index 44d498f6..b07c2bf3 100644 --- a/src/wardline/core/filigree_issue.py +++ b/src/wardline/core/filigree_issue.py @@ -249,8 +249,19 @@ def identity_attach_result_to_json(result: IdentityAttachResult) -> dict[str, An } -def _locator_for_finding_qualname(qualname: str) -> str: - return f"python:function:{qualname}" +def plugin_for_finding(finding: Any) -> str: + """The ADR-049 plugin id that minted ``finding`` — the resolve-hint discriminator + (ADR-036). Rust rules are the ``RS-WL-`` family; everything else with a qualname is + the Python frontend. Derived from the rule id (findings carry no ``lang`` field; + the rule family IS the producer).""" + rule_id = getattr(finding, "rule_id", "") or "" + return "rust" if rule_id.startswith("RS-WL-") else "python" + + +def _locator_for_finding_qualname(qualname: str, plugin: str) -> str: + # Both frontends' callable findings carry function-kind qualnames (Wardline's + # semantic `method` maps to the id-kind `function` in both dialects). + return f"{plugin}:function:{qualname}" def _finding_for_fingerprint(fingerprint: str, root: Path, config_path: Path | None) -> Any | None: @@ -288,6 +299,7 @@ def attach_loomweave_identity_for_finding( issue_id=issue_id, filer=filer, loomweave_client=loomweave_client, + plugin=plugin_for_finding(finding), ) @@ -297,12 +309,13 @@ def attach_loomweave_identity_for_qualname( issue_id: str | None, filer: FiligreeIssueFiler, loomweave_client: Any, + plugin: str = "python", ) -> IdentityAttachResult: if not issue_id: return IdentityAttachResult.not_attempted("no issue_id from Filigree promote") if loomweave_client is None: return IdentityAttachResult.not_attempted("no Loomweave URL configured") - locator = _locator_for_finding_qualname(qualname) + locator = _locator_for_finding_qualname(qualname, plugin) try: resolver = SeiResolver.detect(loomweave_client) binding = resolver.resolve_locator(locator) @@ -314,25 +327,29 @@ def attach_loomweave_identity_for_qualname( issue_id=issue_id, entity_id=binding.sei, content_hash=binding.content_hash, - entity_kind="python:function", + entity_kind=f"{plugin}:function", ) - legacy = _legacy_locator_binding(loomweave_client, qualname, fallback_locator=binding.locator or locator) + legacy = _legacy_locator_binding( + loomweave_client, qualname, fallback_locator=binding.locator or locator, plugin=plugin + ) if legacy.entity_id and legacy.content_hash: return filer.attach_entity_association( issue_id=issue_id, entity_id=legacy.entity_id, content_hash=legacy.content_hash, - entity_kind="python:function", + entity_kind=f"{plugin}:function", ) return legacy -def _legacy_locator_binding(loomweave_client: Any, qualname: str, *, fallback_locator: str) -> IdentityAttachResult: +def _legacy_locator_binding( + loomweave_client: Any, qualname: str, *, fallback_locator: str, plugin: str = "python" +) -> IdentityAttachResult: entity_id: str | None = fallback_locator content_hash: str | None = None try: - resolved = loomweave_client.resolve([qualname]) + resolved = loomweave_client.resolve([qualname], plugin=plugin) except Exception as exc: return IdentityAttachResult.skipped( f"Loomweave legacy locator resolve failed: {exc}", diff --git a/src/wardline/core/scan_file_workflow.py b/src/wardline/core/scan_file_workflow.py index ef1e6e56..50743301 100644 --- a/src/wardline/core/scan_file_workflow.py +++ b/src/wardline/core/scan_file_workflow.py @@ -13,6 +13,7 @@ IdentityAttachResult, attach_loomweave_identity_for_qualname, identity_attach_result_to_json, + plugin_for_finding, ) from wardline.core.finding import Finding, Kind, Severity, SuppressionState from wardline.core.run import gate_decision, run_scan @@ -174,6 +175,7 @@ def scan_file_findings( issue_id=file_result.issue_id, filer=filigree_filer, loomweave_client=loomweave_client, + plugin=plugin_for_finding(finding), ) else: identity_result = IdentityAttachResult.skipped("finding has no qualname") diff --git a/src/wardline/loomweave/client.py b/src/wardline/loomweave/client.py index 35805caa..cc5ff60d 100644 --- a/src/wardline/loomweave/client.py +++ b/src/wardline/loomweave/client.py @@ -191,14 +191,34 @@ def _require_ok(self, resp: Response, path: str) -> dict[str, Any]: # an empty envelope rather than raising (mirrors filigree_emit's defensiveness). return parsed if isinstance(parsed, dict) else {} - def resolve(self, qualnames: list[str]) -> ResolveResult | None: + def resolve(self, qualnames: list[str], *, plugin: str | None = None) -> ResolveResult | None: + """POST /api/wardline/resolve — qualname -> locator, batched. + + ``plugin`` is the OPTIONAL batch-scoped producer hint (ADR-036 plugin-aware + resolution; shape agreed in docs/integration/2026-06-11-wardline-resolve- + plugin-hint-proposal.md): a CONSTRAINT, never a preference — resolution is + restricted to that plugin's namespace, so a hinted qualname owned only by + another plugin stays unresolved. Omitted = the unhinted cross-plugin behavior + (ambiguous degrades to unresolved server-side), legal forever. + + Fail-soft posture: a 4xx on a HINTED request downgrades the chunk to + unresolved — an older Loomweave whose ``ResolveRequest`` is + ``deny_unknown_fields`` 400s on the hint field, and identity enrichment must + degrade, not crash. An UNHINTED 4xx stays loud (it cannot be a version skew on + this field — it is a real request bug, e.g. ``INVALID_PATH``, and silence + would hide it). Outage/5xx stays ``None`` ("unreachable", ``_send``).""" resolved: dict[str, str] = {} unresolved: list[str] = [] for chunk in _chunks(qualnames, self._batch_max): - payload = {"project": self._project, "qualnames": list(chunk)} + payload: dict[str, Any] = {"project": self._project, "qualnames": list(chunk)} + if plugin is not None: + payload["plugin"] = plugin resp = self._send("POST", "/api/wardline/resolve", payload) if resp is None: return None + if plugin is not None and 400 <= resp.status < 500: + unresolved.extend(str(q) for q in chunk) + continue data = self._require_ok(resp, "/api/wardline/resolve") r = data.get("resolved") if isinstance(r, dict): diff --git a/src/wardline/loomweave/dossier_sources.py b/src/wardline/loomweave/dossier_sources.py index f2118fec..9df4ecf0 100644 --- a/src/wardline/loomweave/dossier_sources.py +++ b/src/wardline/loomweave/dossier_sources.py @@ -33,7 +33,7 @@ def get_callees(self, entity_id: str, *, limit: int = ...) -> LinkageResult | No class _ResolveClient(Protocol): - def resolve(self, qualnames: list[str]) -> ResolveResult | None: ... + def resolve(self, qualnames: list[str], *, plugin: str | None = None) -> ResolveResult | None: ... class _Resolver(Protocol): @@ -79,14 +79,22 @@ def linkages(self, binding: EntityBinding) -> LinkagesSection: ) -def resolve_entity_binding(client: _ResolveClient, resolver: _Resolver, qualname: str) -> EntityBinding | None: +def resolve_entity_binding( + client: _ResolveClient, resolver: _Resolver, qualname: str, *, plugin: str | None = None +) -> EntityBinding | None: """Resolve a Wardline qualname to its opaque SEI :class:`EntityBinding`. Two hops, both via existing Track-3 surfaces: ``resolve`` maps the qualname to its Loomweave locator, then the ``SeiResolver`` maps the locator to its SEI binding (the identity axis). Returns None when the qualname cannot be resolved to a locator (the - caller degrades to a no-binding, honest-unavailable dossier — never a guessed key).""" - rr = client.resolve([qualname]) + caller degrades to a no-binding, honest-unavailable dossier — never a guessed key). + + ``plugin`` is the batch-scoped producer hint (ADR-036): pass it when the caller + KNOWS which frontend minted the qualname (attest boundaries and decorator coverage + are Python-surface; a finding's producer derives from its rule id); omit it when + the language is genuinely unknown (a user-supplied dossier entity) — the contract + never fabricates a hint.""" + rr = client.resolve([qualname], plugin=plugin) locator = rr.resolved.get(qualname) if rr is not None else None if not locator: return None diff --git a/src/wardline/weft_decorator_coverage.py b/src/wardline/weft_decorator_coverage.py index e33448af..de61cd45 100644 --- a/src/wardline/weft_decorator_coverage.py +++ b/src/wardline/weft_decorator_coverage.py @@ -21,7 +21,9 @@ def __init__(self, loomweave_client: Any) -> None: self._resolver = SeiResolver(loomweave_client, SeiCapability.from_capabilities(capabilities)) def binding_for(self, qualname: str) -> EntityBinding | None: - return resolve_entity_binding(self._client, self._resolver, qualname) + # Decorator coverage is a Python-surface report (@trust_boundary/@trusted + # decorators), so the producer is known — send the ADR-036 plugin hint. + return resolve_entity_binding(self._client, self._resolver, qualname, plugin="python") def build_weft_decorator_coverage( diff --git a/src/wardline/weft_dossier.py b/src/wardline/weft_dossier.py index a8302a9a..118a3f48 100644 --- a/src/wardline/weft_dossier.py +++ b/src/wardline/weft_dossier.py @@ -42,7 +42,7 @@ class _LoomweaveClient(Protocol): composes with the narrower provider/resolver Protocols downstream).""" def capabilities(self) -> dict[str, Any] | None: ... - def resolve(self, qualnames: list[str]) -> Any: ... + def resolve(self, qualnames: list[str], *, plugin: str | None = None) -> Any: ... def resolve_identity(self, locator: str) -> dict[str, Any] | None: ... def resolve_sei(self, sei: str) -> dict[str, Any] | None: ... def get_callers(self, entity_id: str, *, limit: int = ...) -> LinkageResult | None: ... diff --git a/tests/unit/core/test_attest.py b/tests/unit/core/test_attest.py index 32a0f955..e2baaaf5 100644 --- a/tests/unit/core/test_attest.py +++ b/tests/unit/core/test_attest.py @@ -455,7 +455,8 @@ def __init__(self, *, hit: str = "m.leak", sei: str = _SEI) -> None: def capabilities(self) -> dict[str, object]: return {"sei": {"supported": True, "version": 1}} - def resolve(self, qualnames: list[str]) -> ResolveResult: + def resolve(self, qualnames: list[str], *, plugin: str | None = None) -> ResolveResult: + self.plugin_hints = [*getattr(self, "plugin_hints", []), plugin] resolved = {q: f"python:function:{q}" for q in qualnames if q == self._hit} unresolved = [q for q in qualnames if q != self._hit] return ResolveResult(resolved=resolved, unresolved=unresolved) @@ -476,10 +477,14 @@ def capabilities(self) -> dict[str, object]: def test_sei_keyed_bundle_fills_resolved_boundary_only(tmp_path: Path) -> None: tree = _annotated_tree(tmp_path) - bundle = build_attestation(tree, _KEY, loomweave_client=_FakeLoomweave(hit="m.leak"), today=_PINNED) + client = _FakeLoomweave(hit="m.leak") + bundle = build_attestation(tree, _KEY, loomweave_client=client, today=_PINNED) payload = bundle["payload"] assert payload["sei_source"] == "loomweave" + # Boundaries come from the Python AnalysisContext, so every resolve hop carries + # the ADR-036 plugin hint (constraint, never fabricated). + assert set(client.plugin_hints) == {"python"} by_qn = {b["qualname"]: b for b in payload["boundaries"]} assert by_qn["m.leak"]["sei"] == _SEI # the one resolvable qualname is keyed assert by_qn["m.clean"]["sei"] is None # unresolved → honestly None diff --git a/tests/unit/core/test_filigree_issue.py b/tests/unit/core/test_filigree_issue.py index 6e2cf061..848b4811 100644 --- a/tests/unit/core/test_filigree_issue.py +++ b/tests/unit/core/test_filigree_issue.py @@ -204,7 +204,8 @@ class DownLoomweave: def capabilities(self): return None - def resolve(self, qualnames): + def resolve(self, qualnames, *, plugin=None): + self.plugin_hints = [*getattr(self, "plugin_hints", []), plugin] return None @@ -212,7 +213,8 @@ class LegacyLoomweave: def capabilities(self): return None - def resolve(self, qualnames): + def resolve(self, qualnames, *, plugin=None): + self.plugin_hints = [*getattr(self, "plugin_hints", []), plugin] return SimpleNamespace(resolved={qualnames[0]: "python:function:pkg.mod.leaky"}, unresolved=[]) def get_taint_fact(self, qualname): @@ -334,3 +336,62 @@ def test_attach_loomweave_identity_reports_association_failure(monkeypatch, tmp_ assert res.attached is False assert res.entity_id == "loomweave:eid:abc" assert res.reason == "filigree association returned HTTP 500" + + +class RustFakeFinding: + def __init__(self, qualname): + self.qualname = qualname + self.rule_id = "RS-WL-101" + + +def test_plugin_for_finding_discriminates_by_rule_family(): + from wardline.core.filigree_issue import plugin_for_finding + + assert plugin_for_finding(RustFakeFinding("demo.m.f")) == "rust" + assert plugin_for_finding(FakeFinding("pkg.mod.leaky")) == "python" # no rule_id attr -> python + assert plugin_for_finding(SimpleNamespace(rule_id="PY-WL-101")) == "python" + + +def test_attach_threads_rust_plugin_through_locator_resolve_and_entity_kind(monkeypatch, tmp_path): + # ADR-036 plugin hint, end to end on the Wardline side: a Rust finding mints a + # rust: locator for the SEI hop, sends plugin="rust" on the legacy resolve hop, + # and stamps the association entity_kind as rust:function. + from wardline.core import filigree_issue as mod + + monkeypatch.setattr(mod, "_finding_for_fingerprint", lambda fp, root, cfg: RustFakeFinding("demo.m.leaky")) + + class RustLegacyLoomweave: + def __init__(self): + self.identity_locators = [] + self.plugin_hints = [] + + def capabilities(self): + return None # pre-SEI -> the legacy locator path + + def resolve(self, qualnames, *, plugin=None): + self.plugin_hints.append(plugin) + return SimpleNamespace(resolved={qualnames[0]: "rust:function:demo.m.leaky"}, unresolved=[]) + + def resolve_identity(self, locator): + self.identity_locators.append(locator) + return None + + def get_taint_fact(self, qualname): + return SimpleNamespace(current_content_hash="rust-hash") + + client = RustLegacyLoomweave() + transport = RecordingTransport() + filer = FiligreeIssueFiler("http://f/api/weft/scan-results", transport=transport) + + res = attach_loomweave_identity_for_finding( + fingerprint="fp1", + issue_id="wardline-1", + root=tmp_path, + filer=filer, + loomweave_client=client, + ) + + assert client.plugin_hints == ["rust"] + assert res.attached is True + assert transport.calls[0]["body"]["entity_id"] == "rust:function:demo.m.leaky" + assert transport.calls[0]["body"]["entity_kind"] == "rust:function" diff --git a/tests/unit/core/test_weft_dossier.py b/tests/unit/core/test_weft_dossier.py index 43c6e648..4fc88d54 100644 --- a/tests/unit/core/test_weft_dossier.py +++ b/tests/unit/core/test_weft_dossier.py @@ -47,7 +47,8 @@ def capabilities(self): "sei": {"supported": self._sei_supported, "version": 1}, } - def resolve(self, qualnames): + def resolve(self, qualnames, *, plugin=None): + self.plugin_hints = [*getattr(self, "plugin_hints", []), plugin] return ResolveResult(resolved={q: f"python:function:{q}" for q in qualnames}, unresolved=[]) def resolve_identity(self, locator): diff --git a/tests/unit/loomweave/test_client.py b/tests/unit/loomweave/test_client.py index 073814dd..877c827e 100644 --- a/tests/unit/loomweave/test_client.py +++ b/tests/unit/loomweave/test_client.py @@ -125,3 +125,41 @@ def request(self, *a, **k): raise OSError("connection refused") assert _client(Boom()).batch_get(["a"]) is None + + +def test_resolve_sends_batch_scoped_plugin_hint(): + # ADR-036 plugin-aware resolution: the OPTIONAL batch-scoped hint rides the + # request verbatim (docs/integration/2026-06-11-wardline-resolve-plugin-hint- + # proposal.md). One hint per request — never per qualname. + t = FakeTransport([Response(status=200, body='{"resolved":{},"unresolved":["m.f"]}')]) + _client(t).resolve(["m.f"], plugin="rust") + assert json.loads(t.calls[0][2])["plugin"] == "rust" + + +def test_resolve_omits_plugin_field_when_unhinted(): + # Omission is today's behavior FOREVER (the contract never fabricates a hint) — + # and an absent field is what keeps unhinted requests valid against any server + # version under deny_unknown_fields. + t = FakeTransport([Response(status=200, body='{"resolved":{},"unresolved":["m.f"]}')]) + _client(t).resolve(["m.f"]) + assert "plugin" not in json.loads(t.calls[0][2]) + + +def test_resolve_hinted_4xx_downgrades_chunk_to_unresolved(): + # Fail-soft: an older Loomweave whose ResolveRequest is deny_unknown_fields 400s + # on the hint field — identity enrichment must degrade to unresolved, not crash + # the dossier/attach path. + t = FakeTransport([Response(status=400, body='{"error":"unknown field `plugin`"}')]) + result = _client(t).resolve(["m.f", "m.g"], plugin="rust") + assert result is not None + assert result.resolved == {} + assert result.unresolved == ["m.f", "m.g"] + + +def test_resolve_unhinted_4xx_stays_loud(): + # An unhinted 4xx cannot be hint-field version skew — it is a real request bug + # and must stay diagnosable (the pre-existing INVALID_PATH pin, re-asserted + # against the hint-conditional soft band). + t = FakeTransport([Response(status=400, body='{"code":"INVALID_PATH"}')]) + with pytest.raises(LoomweaveError, match="INVALID_PATH"): + _client(t).resolve(["m.f"]) diff --git a/tests/unit/loomweave/test_dossier_sources.py b/tests/unit/loomweave/test_dossier_sources.py index 23f51866..658ec294 100644 --- a/tests/unit/loomweave/test_dossier_sources.py +++ b/tests/unit/loomweave/test_dossier_sources.py @@ -27,7 +27,8 @@ def get_callers(self, entity_id, *, limit=50): def get_callees(self, entity_id, *, limit=50): return self._callees - def resolve(self, qualnames): + def resolve(self, qualnames, *, plugin=None): + self.plugin_hints = [*getattr(self, "plugin_hints", []), plugin] from wardline.loomweave.client import ResolveResult resolved = {q: self._resolved[q] for q in qualnames if q in self._resolved} diff --git a/tests/unit/mcp/test_server_dossier.py b/tests/unit/mcp/test_server_dossier.py index c4085406..511d5bcf 100644 --- a/tests/unit/mcp/test_server_dossier.py +++ b/tests/unit/mcp/test_server_dossier.py @@ -94,7 +94,7 @@ def __init__(self, *a, **k): def capabilities(self): return None - def resolve(self, qualnames): + def resolve(self, qualnames, *, plugin=None): return None monkeypatch.setattr("wardline.loomweave.client.LoomweaveClient", _FakeClient) From 808fad2512c366e11586bfdcc0f0e08bc3263bdc Mon Sep 17 00:00:00 2001 From: John Morrissey Date: Thu, 11 Jun 2026 14:32:25 +1000 Subject: [PATCH 096/186] =?UTF-8?q?fix(scan):=20N-3=20scan-root=20guard=20?= =?UTF-8?q?=E2=80=94=20nested-scan-root=20FACT=20+=20loud=20CLI=20warning?= =?UTF-8?q?=20+=20help=20note;=20N-8=20dossier=20resolve-against-scan-root?= =?UTF-8?q?=20error=20(closes=20wardline-8669de3576)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The scan root governs finding identity (qualnames/fingerprints minted relative to it, .weft/wardline state read beneath it), and nothing detected a root nested inside an enclosing weft project — a subdirectory scan silently minted qualnames no federated tool matches, skipped the project baseline, and wrote output into the subdir (senior dogfood 2026-06-11, hub weft-074cda672c). - core/paths.enclosing_project_root: nearest strict ancestor carrying weft.toml or .weft/wardline/ (sibling-only .weft deliberately not a marker; fresh unmarked trees never warn) - core/run.run_scan: WLN-ENGINE-NESTED-SCAN-ROOT FACT (severity NONE, non-gating, not unanalyzed) naming the qualname prefix, project root, and all three hazards — reaches CLI, MCP, and findings.jsonl alike; empty prefix handled for src-layout roots (P/src mints the same qualnames as P) - cli/scan: stderr warning reusing the FACT message verbatim; --help documents the scan-root/qualname coupling - N-8 (same root cause, folded in): DossierError for entity-not-found now teaches the coupling, names the scan-relative form that DOES match, and gives the exact project-root rerun — no silent auto-rerooting (that would change suppression state and mint unasked-for identities); dossier --help notes the coupling - glossary file:line anchors rebased over the run.py/scan.py insertions Co-Authored-By: Claude Fable 5 --- .../reference/finding-lifecycle-vocabulary.md | 26 ++++---- src/wardline/cli/dossier.py | 8 ++- src/wardline/cli/scan.py | 17 ++++- src/wardline/core/dossier.py | 38 +++++++++++- src/wardline/core/paths.py | 35 +++++++++++ src/wardline/core/run.py | 43 ++++++++++++- tests/docs/test_glossary_vocabulary.py | 14 ++--- tests/unit/cli/test_cli.py | 45 ++++++++++++++ tests/unit/core/test_dossier_assembler.py | 46 ++++++++++++++ tests/unit/core/test_paths.py | 48 ++++++++++++++ tests/unit/core/test_run.py | 62 +++++++++++++++++++ 11 files changed, 358 insertions(+), 24 deletions(-) diff --git a/docs/reference/finding-lifecycle-vocabulary.md b/docs/reference/finding-lifecycle-vocabulary.md index ce724fff..f73e6d7b 100644 --- a/docs/reference/finding-lifecycle-vocabulary.md +++ b/docs/reference/finding-lifecycle-vocabulary.md @@ -61,7 +61,7 @@ The Filigree metadata only carries the key when the state is not `active` **"suppressed"** survives only as the umbrella *word* for "any state other than `active`": `baselined` + `waived` + `judged`. The CLI prints this sum as the -`suppressed` count (`src/wardline/cli/scan.py:406`). +`suppressed` count (`src/wardline/cli/scan.py:415`). ## `active` is the one word for "non-suppressed defect" @@ -71,8 +71,8 @@ consistently, on every surface: | Surface | Where | Term | | --- | --- | --- | | Enum | `src/wardline/core/finding.py:72` | `SuppressionState.ACTIVE = "active"` | -| Summary field | `src/wardline/core/run.py:51`, built at `src/wardline/core/run.py:320` | `ScanSummary.active` | -| CLI summary line | `src/wardline/cli/scan.py:407` | `… {s.active} active` | +| Summary field | `src/wardline/core/run.py:51`, built at `src/wardline/core/run.py:361` | `ScanSummary.active` | +| CLI summary line | `src/wardline/cli/scan.py:416` | `… {s.active} active` | | MCP scan response | `src/wardline/mcp/server.py:331` | `summary.active` | | Agent-summary JSON | `src/wardline/core/agent_summary.py:135` | `summary.active_defects` | | `wardline:loop` prompt | `src/wardline/mcp/prompts.py:13` | "Read `summary.active`" | @@ -111,19 +111,19 @@ There are **two distinct populations** of defects in one scan, and they can differ on purpose: 1. **Emitted-active** — `summary.active` counts `active` defects in the - **emitted** (post-annotation) findings (built at `src/wardline/core/run.py:320`). + **emitted** (post-annotation) findings (built at `src/wardline/core/run.py:361`). Baseline / waiver / judged annotate these findings in place; a suppressed defect is still emitted, just not counted as `active`. 2. **Gate population** — the `--fail-on` gate evaluates a **separate** `ScanResult.gate_findings` list: the *unsuppressed* population - (`src/wardline/core/run.py:286`). By default, repository-controlled + (`src/wardline/core/run.py:327`). By default, repository-controlled baseline / waiver / judged entries **annotate** the emitted findings but do **not** clear the gate — so a malicious PR cannot green the gate by committing a suppression keyed to its own new defect. `gate_decision` evaluates `gate_findings` when present, else falls back to `findings` (the trusted `--trust-suppressions` / directly-constructed path), selected at - `src/wardline/core/run.py:371` (`honors_suppressions`). + `src/wardline/core/run.py:412` (`honors_suppressions`). This is why **`summary.active: 0` can co-exist with `gate.tripped: true`**: every defect was suppressed by a committed baseline (so emitted-active is 0), but those @@ -151,11 +151,11 @@ The MCP `scan` gate block exposes `gate.tripped` (`src/wardline/mcp/server.py:34 `src/wardline/core/agent_summary.py:153` (`verdict`). The CLI prints `gate: FAILED (--fail-on …) — ` then `gate: evaluated <…>`, or a `gate: NOT_EVALUATED — …` line for a bare scan -(`src/wardline/cli/scan.py:441`). +(`src/wardline/cli/scan.py:456`). `--new-since` scopes **both** populations identically: any `active` defect outside the delta is re-marked `baselined` in both the emitted and gate lists -(`src/wardline/core/run.py:296`, `def apply_delta_scope`). +(`src/wardline/core/run.py:337`, `def apply_delta_scope`). ## The three meanings of "new" @@ -166,8 +166,8 @@ still legitimately means three different things depending on the surface: | "new" on this surface | Means | Owner / anchor | | --- | --- | --- | | Filigree store | An **unseen fingerprint** — first time this finding identity is seen for a `(file, scan_source)`. | **Filigree-owned** lifecycle (`src/wardline/core/filigree_emit.py:68-76`) | -| `wardline scan --new-since ` | **Delta-scope**: the gate fires only on defects in files/entities changed since a git ref; everything else is re-marked `baselined`. | `src/wardline/core/run.py:296`; help text `src/wardline/cli/scan.py` (`--new-since`) | -| (historical) CLI summary | Formerly relabelled the `active` count as "N new". **Corrected to "N active"**. | `src/wardline/cli/scan.py:406` | +| `wardline scan --new-since ` | **Delta-scope**: the gate fires only on defects in files/entities changed since a git ref; everything else is re-marked `baselined`. | `src/wardline/core/run.py:337`; help text `src/wardline/cli/scan.py` (`--new-since`) | +| (historical) CLI summary | Formerly relabelled the `active` count as "N new". **Corrected to "N active"**. | `src/wardline/cli/scan.py:415` | The first-seen Filigree sense and the delta-scope `--new-since` sense are genuinely distinct concepts; neither is "active". @@ -179,8 +179,8 @@ How each concept appears on each surface: | Concept | CLI summary text | `ScanSummary` field | MCP `summary` key | Agent-summary key | Filigree store | | --- | --- | --- | --- | --- | --- | | every finding | `N finding(s)` | `total` (`run.py:50`) | `total` (`server.py:330`) | `total_findings` (`agent_summary.py:134`) | one finding per wire entry | -| live defect | `N active` (`scan.py:407`) | `active` (`run.py:51,320`) | `active` (`server.py:331`) | `active_defects` (`agent_summary.py:135`) | no `suppression_state` key (`finding.py:285`) | -| suppressed (sum) | `N suppressed` (`scan.py:406`) | `baselined+waived+judged` | the three keys | `suppressed_findings` (`agent_summary.py:136`) | `metadata.wardline.suppression_state` (`finding.py:285`) | +| live defect | `N active` (`scan.py:416`) | `active` (`run.py:51,361`) | `active` (`server.py:331`) | `active_defects` (`agent_summary.py:135`) | no `suppression_state` key (`finding.py:285`) | +| suppressed (sum) | `N suppressed` (`scan.py:415`) | `baselined+waived+judged` | the three keys | `suppressed_findings` (`agent_summary.py:136`) | `metadata.wardline.suppression_state` (`finding.py:285`) | | baselined | `N baseline` | `baselined` (`run.py:53`) | `baselined` (`server.py:332`) | `baselined` (`agent_summary.py:138`) | `suppression_state: "baselined"` | | waived | `N waiver` | `waived` (`run.py:54`) | `waived` (`server.py:333`) | `waived` (`agent_summary.py:139`) | `suppression_state: "waived"` | | judged | `N judged` | `judged` (`run.py:55`) | `judged` (`server.py:334`) | `judged` (`agent_summary.py:140`) | `suppression_state: "judged"` | @@ -190,7 +190,7 @@ How each concept appears on each surface: | gate verdict | exit code + `--fail-on` | (`gate_findings`, `run.py:88`; `GateDecision`, `run.py:98`, `verdict` `run.py:107`) | `gate.tripped` (`server.py:346`), `gate.verdict` (`server.py:349`) | `gate.tripped` (`agent_summary.py:150`), `gate.verdict` (`agent_summary.py:153`) | not emitted to Filigree | The unsuppressed gate population is built from `Baseline(frozenset())` -(`src/wardline/core/run.py:286`). +(`src/wardline/core/run.py:327`). ## For the suite diff --git a/src/wardline/cli/dossier.py b/src/wardline/cli/dossier.py index 9ab2d8d2..c17341cb 100644 --- a/src/wardline/cli/dossier.py +++ b/src/wardline/cli/dossier.py @@ -47,7 +47,13 @@ def dossier( loomweave_url: str | None, filigree_url: str | None, ) -> None: - """Assemble the one-call dossier for ENTITY (a function qualname) under PATH.""" + """Assemble the one-call dossier for ENTITY (a function qualname) under PATH. + + PATH is the scan root and governs qualnames: ENTITY must be qualified + relative to it. Run against the project root (the directory holding + weft.toml / .weft/wardline/) for the package-qualified form other Weft + tools — and the MCP dossier — use. + """ from wardline.weft_dossier import build_weft_dossier try: diff --git a/src/wardline/cli/scan.py b/src/wardline/cli/scan.py index fd8fd0a6..51f62854 100644 --- a/src/wardline/cli/scan.py +++ b/src/wardline/cli/scan.py @@ -160,7 +160,16 @@ def scan( trust_suppressions: bool, allow_dirty: bool, ) -> None: - """Scan PATH for findings.""" + """Scan PATH for findings. + + PATH is the scan root and GOVERNS finding identity: qualnames and + fingerprints are minted relative to it, and baseline/waiver/judged + suppression state is read from PATH's .weft/wardline/. Scan the project + root — a subdirectory scan mints qualnames other Weft tools + (Loomweave/Filigree/dossier) will not match, misses the project's + suppression state, and writes output into the subdirectory (wardline + warns when it detects this). + """ if lang == "rust": # Posture banner: RS-WL-* identity is graduated (frozen, baseline-eligible) but # rule coverage is the command-injection slice and weft.toml severity overrides @@ -407,6 +416,12 @@ def confirm_cb(rel_path: str, orig: str, replacement: str, f: Finding) -> bool: f"({s.baselined} baseline / {s.waived} waiver / {s.judged} judged), {s.active} active" f"{unanalyzed_segment} -> {output}" ) + # N-3: a scan rooted in a subdirectory of a weft project mints qualnames no + # federated tool matches and skips the project's suppression state. The FACT + # carries the full explanation — reuse it verbatim so CLI and MCP say the same. + nested = next((f for f in result.findings if f.rule_id == "WLN-ENGINE-NESTED-SCAN-ROOT"), None) + if nested is not None: + click.echo(f"warning: {nested.message}", err=True) # A discovered-but-not-analysed file is a silent under-scan; never hide it. if s.unanalyzed: click.echo( diff --git a/src/wardline/core/dossier.py b/src/wardline/core/dossier.py index 3e4307a8..13baa8d0 100644 --- a/src/wardline/core/dossier.py +++ b/src/wardline/core/dossier.py @@ -39,6 +39,7 @@ from wardline.core.errors import DossierError from wardline.core.finding import UNANALYZED_RULE_IDS, Kind, SuppressionState from wardline.core.identity import ContentStatus, EntityBinding, IdentityStatus +from wardline.core.paths import enclosing_project_root from wardline.core.run import run_scan from wardline.core.taints import TaintState @@ -629,6 +630,41 @@ def _synthesize(identity: IdentitySection, trust: TrustSection, linkages: Linkag return " ".join(bits) +def _entity_not_found_message(entity: str, root: Path, context: AnalysisContext | None) -> str: + """The entity-not-found error, with the resolve-against-scan-root remedy. + + N-8 (folded into wardline-8669de3576): qualnames are minted relative to the + scan root, so a dossier rooted in a SUBDIRECTORY of a weft project rejects the + package-qualified qualname the project-rooted call (the MCP server's shape) + accepts. When that is detectable, teach the coupling and name both remedies — + the scan-relative form that DOES match under this root, and the project root + to rerun against. We deliberately do NOT auto-resolve: silently scanning a + different root would change suppression state and mint identities the caller + did not ask for (the exact silent divergence N-3 is about). + """ + base = f"entity not found in scanned set: {entity}" + enclosing = enclosing_project_root(root) + if enclosing is None: + return base + rel = root.resolve().relative_to(enclosing) + parts = rel.parts[1:] if rel.parts and rel.parts[0] == "src" else rel.parts + prefix = ".".join(parts) + coupling = ( + f"{base}. Qualnames are minted relative to the scan root, and {root.resolve()} is a " + f"subdirectory of the weft project at {enclosing}." + ) + stripped = entity[len(prefix) + 1 :] if prefix and entity.startswith(prefix + ".") else None + if stripped and context is not None and stripped in context.entities: + return ( + f"{coupling} '{stripped}' matches under this scan root; for the package-qualified " + f"form rerun against the project root: wardline dossier {entity} {enclosing}" + ) + return ( + f"{coupling} Rerun against the project root for package-qualified qualnames: " + f"wardline dossier {entity} {enclosing}" + ) + + def build_dossier( entity: str, *, @@ -654,7 +690,7 @@ def build_dossier( result = run_scan(root, config_path=config_path, confine_to_root=confine_to_root) context = result.context if context is None or entity not in context.entities: - raise DossierError(f"entity not found in scanned set: {entity}") + raise DossierError(_entity_not_found_message(entity, root, context)) target = context.entities[entity] identity = _build_identity(target, binding) diff --git a/src/wardline/core/paths.py b/src/wardline/core/paths.py index a53a93dd..774fba36 100644 --- a/src/wardline/core/paths.py +++ b/src/wardline/core/paths.py @@ -91,6 +91,41 @@ def migration_journal_path(root: Path) -> Path: return weft_state_dir(root) / "migration_journal.yaml" +def _has_project_markers(candidate: Path) -> bool: + """Whether *candidate* looks like a weft project root for wardline's purposes. + + Markers are exactly the two surfaces that change wardline's behaviour: the + operator-authored ``weft.toml`` (config, severity overrides, federation URLs) + and the default ``.weft/wardline/`` state subtree (baseline/waivers/judged). + A ``.weft/`` holding only SIBLING members (filigree/loomweave) deliberately + does NOT count — there is no wardline config or suppression state to miss + there, and counting it would make wardline's own repo (``.weft/filigree``) + warn on every in-repo fixture scan. A non-default ``store_dir`` override can + only be set in ``weft.toml``, so the config marker already covers it. + """ + return (candidate / WEFT_CONFIG_FILE).is_file() or (candidate / _WEFT_DIR / WEFT_MEMBER).is_dir() + + +def enclosing_project_root(root: Path) -> Path | None: + """The nearest STRICT ancestor of ``root`` that is a weft project root, or None. + + The scan root governs finding identity: qualnames are minted relative to it + and baseline/waiver/judged state is read beneath it (N-3, wardline-8669de3576). + A non-None result therefore means ``root`` is a *subdirectory* of a weft + project — the scan would mint qualnames no federated tool matches and skip the + project's suppression state. Returns None when ``root`` itself carries project + markers (it IS a root, including a vendored project inside another) or when no + ancestor does (a fresh, unfederated tree — warning there would be pure noise). + """ + root = root.resolve() + if _has_project_markers(root): + return None + for ancestor in root.parents: + if _has_project_markers(ancestor): + return ancestor + return None + + def sibling_state_dir(root: Path, sibling: str) -> Path: """Preferred location of a sibling member's runtime subtree.""" return root / _WEFT_DIR / sibling diff --git a/src/wardline/core/run.py b/src/wardline/core/run.py index 3669ce74..57e93471 100644 --- a/src/wardline/core/run.py +++ b/src/wardline/core/run.py @@ -30,7 +30,7 @@ ) from wardline.core.frontends import FRONTENDS from wardline.core.judged import load_judged -from wardline.core.paths import baseline_path, judged_path, weft_config_path +from wardline.core.paths import baseline_path, enclosing_project_root, judged_path, weft_config_path from wardline.core.protocols import Analyzer from wardline.core.suppression import SEVERITY_ORDER, apply_suppressions, gate_trips, severity_gates from wardline.core.waivers import WaiverSet, load_project_waivers @@ -255,6 +255,47 @@ def run_scan( properties={"source_root": src}, ) ) + # N-3 (wardline-8669de3576): the scan root GOVERNS finding identity — qualnames + # are minted relative to it, suppression state is read beneath it, and output + # defaults under it. A scan rooted in a SUBDIRECTORY of a weft project silently + # mints qualnames no federated tool (Loomweave/Filigree/dossier) matches and + # skips the project baseline. Surface it as a FACT so it reaches the CLI + # warning, the MCP result, and findings.jsonl alike. Not an under-scan (every + # discovered file WAS analysed), so it never counts toward unanalyzed. + enclosing = enclosing_project_root(root) + if enclosing is not None: + rel = root.resolve().relative_to(enclosing) + prefix_parts = rel.parts[1:] if rel.parts and rel.parts[0] == "src" else rel.parts + # module_dotted_name strips one leading src/ component, so scanning P/src + # mints the SAME qualnames as scanning P — the prefix is empty there and + # the message must not claim a phantom 'src.' prefix. + qualname_prefix = ".".join(prefix_parts) + qualname_clause = ( + f"qualnames are minted relative to the scan root (missing the '{qualname_prefix}.' " + "package prefix other Weft tools expect), " + if qualname_prefix + else "qualnames are minted relative to the scan root, " + ) + raw.append( + Finding( + rule_id="WLN-ENGINE-NESTED-SCAN-ROOT", + message=( + f"scan root '{rel.as_posix()}' is a subdirectory of the weft project at " + f"{enclosing}: {qualname_clause}the project's baseline/waivers/judged state " + "is not loaded, and output defaults under the subdirectory. Scan the project " + "root for federation-stable results." + ), + severity=Severity.NONE, + kind=Kind.FACT, + location=Location(path=rel.as_posix()), + fingerprint=_fp("WLN-ENGINE-NESTED-SCAN-ROOT", rel.as_posix()), + properties={ + "scan_root": str(root.resolve()), + "project_root": str(enclosing), + "qualname_prefix": qualname_prefix, + }, + ) + ) if cache is not None: cache.save() today = date.today() diff --git a/tests/docs/test_glossary_vocabulary.py b/tests/docs/test_glossary_vocabulary.py index 744e80c1..8657e4aa 100644 --- a/tests/docs/test_glossary_vocabulary.py +++ b/tests/docs/test_glossary_vocabulary.py @@ -36,14 +36,14 @@ ("src/wardline/core/run.py", 88, "gate_findings:"), ("src/wardline/core/run.py", 98, "class GateDecision"), ("src/wardline/core/run.py", 107, "verdict: str"), - ("src/wardline/core/run.py", 286, "Baseline(frozenset())"), - ("src/wardline/core/run.py", 296, "def apply_delta_scope"), - ("src/wardline/core/run.py", 320, "active=sum"), - ("src/wardline/core/run.py", 371, "honors_suppressions"), + ("src/wardline/core/run.py", 327, "Baseline(frozenset())"), + ("src/wardline/core/run.py", 337, "def apply_delta_scope"), + ("src/wardline/core/run.py", 361, "active=sum"), + ("src/wardline/core/run.py", 412, "honors_suppressions"), # src/wardline/cli/scan.py — CLI summary line + gate stderr - ("src/wardline/cli/scan.py", 406, "suppressed"), - ("src/wardline/cli/scan.py", 407, "{s.active} active"), - ("src/wardline/cli/scan.py", 441, "gate: FAILED"), + ("src/wardline/cli/scan.py", 415, "suppressed"), + ("src/wardline/cli/scan.py", 416, "{s.active} active"), + ("src/wardline/cli/scan.py", 456, "gate: FAILED"), # src/wardline/mcp/server.py — MCP scan summary + gate block ("src/wardline/mcp/server.py", 330, '"total": result.summary.total'), ("src/wardline/mcp/server.py", 331, '"active": result.summary.active'), diff --git a/tests/unit/cli/test_cli.py b/tests/unit/cli/test_cli.py index bca673bf..8b510d4d 100644 --- a/tests/unit/cli/test_cli.py +++ b/tests/unit/cli/test_cli.py @@ -1441,3 +1441,48 @@ def test_scan_loomweave_with_unresolved_qualnames(tmp_path, monkeypatch) -> None assert result.exit_code == 0, result.output assert "wrote 1 taint fact(s)" in result.output assert "unresolved" in result.output + + +# --- N-3 (wardline-8669de3576): subdirectory scans warn loudly --------------- + + +def test_scan_subdirectory_of_weft_project_warns(tmp_path: Path) -> None: + # Scanning a subdirectory of a weft project mints scan-relative qualnames, + # skips the project baseline, and writes output into the subdir. The CLI must + # be LOUD about it (stderr warning sourced from the WLN-ENGINE-NESTED-SCAN-ROOT + # fact) while the scan itself still succeeds. + proj = tmp_path / "proj" + (proj / ".weft" / "wardline").mkdir(parents=True) + sub = proj / "specimen" + sub.mkdir() + (sub / "m.py").write_text("def f(): return 1\n", encoding="utf-8") + result = CliRunner().invoke(cli, ["scan", str(sub)]) + assert result.exit_code == 0, result.output + assert "warning:" in result.stderr + assert "qualname" in result.stderr + assert str(proj.resolve()) in result.stderr + + +def test_scan_project_root_does_not_warn_nested(tmp_path: Path) -> None: + proj = tmp_path / "proj" + (proj / ".weft" / "wardline").mkdir(parents=True) + (proj / "m.py").write_text("def f(): return 1\n", encoding="utf-8") + result = CliRunner().invoke(cli, ["scan", str(proj)]) + assert result.exit_code == 0, result.output + assert "WLN-ENGINE-NESTED-SCAN-ROOT" not in result.stderr + assert "subdirectory" not in result.stderr + + +def test_scan_help_documents_scan_root_qualname_coupling() -> None: + result = CliRunner().invoke(cli, ["scan", "--help"]) + assert result.exit_code == 0 + helptext = result.output.lower() + assert "qualname" in helptext + assert "scan root" in helptext + + +def test_dossier_help_documents_scan_root_qualname_coupling() -> None: + result = CliRunner().invoke(cli, ["dossier", "--help"]) + assert result.exit_code == 0 + helptext = result.output.lower() + assert "scan root" in helptext or "project root" in helptext diff --git a/tests/unit/core/test_dossier_assembler.py b/tests/unit/core/test_dossier_assembler.py index bc215772..a1193003 100644 --- a/tests/unit/core/test_dossier_assembler.py +++ b/tests/unit/core/test_dossier_assembler.py @@ -372,3 +372,49 @@ def test_synthesis_is_present_and_degrades_without_optional_sources(tmp_path: Pa # best-effort: mentions the live defect; never asserts a join it could not compute assert d.synthesis is not None assert "PY-WL-101" in d.synthesis + + +# --- N-8 (folded into wardline-8669de3576): resolve-against-scan-root remedy -- + + +def test_unknown_entity_error_names_nested_scan_root_remedy(tmp_path: Path) -> None: + # The CLI dossier run against a subdirectory rejects the package-qualified + # qualname the MCP dossier (rooted at the project) accepts — same root cause as + # N-3: qualnames are minted relative to the scan root. The not-found error must + # carry the resolve-against-scan-root remedy: name the scan-relative form that + # DOES match under this root and the project root to rerun against. + proj = tmp_path / "proj" + (proj / ".weft" / "wardline").mkdir(parents=True) + sub = proj / "specimen" + sub.mkdir() + (sub / "svc.py").write_text(_LEAKY, encoding="utf-8") + with pytest.raises(DossierError) as exc: + build_dossier("specimen.svc.leaky", root=sub) + msg = str(exc.value) + assert "specimen.svc.leaky" in msg + assert "'svc.leaky'" in msg # the scan-relative form that matches under this root + assert str(proj.resolve()) in msg # the project root to rerun against + + +def test_unknown_entity_error_nested_without_match_still_points_at_root(tmp_path: Path) -> None: + # Nested root but the entity genuinely doesn't exist under either form: the + # error still teaches the scan-root coupling and points at the project root. + proj = tmp_path / "proj" + (proj / ".weft" / "wardline").mkdir(parents=True) + sub = proj / "specimen" + sub.mkdir() + (sub / "svc.py").write_text(_LEAKY, encoding="utf-8") + with pytest.raises(DossierError) as exc: + build_dossier("specimen.svc.nonexistent", root=sub) + msg = str(exc.value) + assert str(proj.resolve()) in msg + assert "scan root" in msg + + +def test_unknown_entity_error_stays_plain_when_not_nested(tmp_path: Path) -> None: + # No enclosing project: the established terse error is unchanged. + with pytest.raises(DossierError) as exc: + build_dossier("svc.does_not_exist", root=_proj(tmp_path)) + msg = str(exc.value) + assert "entity not found in scanned set: svc.does_not_exist" in msg + assert "scan root" not in msg diff --git a/tests/unit/core/test_paths.py b/tests/unit/core/test_paths.py index 88b41cb1..40560562 100644 --- a/tests/unit/core/test_paths.py +++ b/tests/unit/core/test_paths.py @@ -60,3 +60,51 @@ def test_store_dir_absolute_outside_root_falls_back_to_default(tmp_path): def test_store_dir_relative_escape_falls_back_to_default(tmp_path): (tmp_path / "weft.toml").write_text('[wardline]\nstore_dir = "../escape"\n', encoding="utf-8") assert paths.weft_state_dir(tmp_path) == tmp_path / ".weft" / "wardline" + + +# --- enclosing_project_root (N-3: the scan root governs qualnames) ----------- + + +def test_enclosing_project_root_none_when_root_has_weft_toml(tmp_path): + (tmp_path / "weft.toml").write_text("[wardline]\n", encoding="utf-8") + assert paths.enclosing_project_root(tmp_path) is None + + +def test_enclosing_project_root_none_when_root_has_state_dir(tmp_path): + (tmp_path / ".weft" / "wardline").mkdir(parents=True) + assert paths.enclosing_project_root(tmp_path) is None + + +def test_enclosing_project_root_finds_weft_toml_ancestor(tmp_path): + (tmp_path / "weft.toml").write_text("[wardline]\n", encoding="utf-8") + sub = tmp_path / "specimen" + sub.mkdir() + assert paths.enclosing_project_root(sub) == tmp_path.resolve() + + +def test_enclosing_project_root_finds_state_dir_ancestor_deep(tmp_path): + (tmp_path / ".weft" / "wardline").mkdir(parents=True) + sub = tmp_path / "a" / "b" + sub.mkdir(parents=True) + assert paths.enclosing_project_root(sub) == tmp_path.resolve() + + +def test_enclosing_project_root_nested_project_is_its_own_root(tmp_path): + # A subdirectory that is its OWN weft project root (vendored tree) is not + # "nested" — its markers win and no enclosing root is reported. + (tmp_path / "weft.toml").write_text("[wardline]\n", encoding="utf-8") + sub = tmp_path / "vendored" + sub.mkdir() + (sub / "weft.toml").write_text("[wardline]\n", encoding="utf-8") + assert paths.enclosing_project_root(sub) is None + + +def test_enclosing_project_root_ignores_sibling_only_weft_dir(tmp_path): + # A .weft/ holding only SIBLING members (filigree/loomweave) marks nothing for + # wardline: neither operator config (weft.toml) nor wardline state exists, so + # there is no baseline to miss and no [wardline] config to skip. (This also keeps + # wardline's own repo — .weft/filigree only — from warning on in-repo fixture scans.) + (tmp_path / ".weft" / "filigree").mkdir(parents=True) + sub = tmp_path / "specimen" + sub.mkdir() + assert paths.enclosing_project_root(sub) is None diff --git a/tests/unit/core/test_run.py b/tests/unit/core/test_run.py index 33ae53fd..6b00db65 100644 --- a/tests/unit/core/test_run.py +++ b/tests/unit/core/test_run.py @@ -624,3 +624,65 @@ def test_run_scan_out_of_root_symlink_yields_finding(tmp_path: Path) -> None: assert len(skipped) == 1 assert skipped[0].location.path == "src/evil.py" assert skipped[0].properties.get("reason") == "out_of_root_symlink" + + +# --- N-3 (wardline-8669de3576): nested scan root is surfaced, never silent --- + + +def test_run_scan_nested_scan_root_yields_fact(tmp_path: Path) -> None: + # A subdirectory scan of a weft project silently mints scan-relative qualnames, + # skips the project baseline, and drops output into the subdir. run_scan must + # surface the nested root as a structured FACT (reaching both the CLI warning + # and the MCP result). + proj = tmp_path / "proj" + (proj / ".weft" / "wardline").mkdir(parents=True) + sub = proj / "specimen" + sub.mkdir() + (sub / "svc.py").write_text(_LEAKY, encoding="utf-8") + result = run_scan(sub) + facts = [f for f in result.findings if f.rule_id == "WLN-ENGINE-NESTED-SCAN-ROOT"] + assert len(facts) == 1 + fact = facts[0] + assert fact.kind is Kind.FACT and fact.severity is Severity.NONE + assert fact.properties["project_root"] == str(proj.resolve()) + assert fact.properties["qualname_prefix"] == "specimen" + # the qualname hazard and the remedy root are named in the message (the + # agent-actionable signal — the CLI warning reuses this verbatim) + assert "qualname" in fact.message and str(proj.resolve()) in fact.message + # a scope hazard, not an under-scan — never counted as unanalyzed + assert result.summary.unanalyzed == 0 + # the PY-WL-101 defect still fires, with the scan-relative qualname the fact warns about + leak = next(f for f in result.findings if f.rule_id == "PY-WL-101") + assert leak.qualname == "svc.leaky" + + +def test_run_scan_project_root_scan_has_no_nested_fact(tmp_path: Path) -> None: + proj, _ = _leaky_proj(tmp_path) + (proj / ".weft" / "wardline").mkdir(parents=True, exist_ok=True) + result = run_scan(proj) + assert not [f for f in result.findings if f.rule_id == "WLN-ENGINE-NESTED-SCAN-ROOT"] + + +def test_run_scan_fresh_tree_subdir_has_no_nested_fact(tmp_path: Path) -> None: + # No weft markers anywhere above: a fresh unfederated tree must not warn — + # warning every first-time user would dilute the signal into habitual noise. + sub = tmp_path / "plain" / "pkg" + sub.mkdir(parents=True) + (sub / "m.py").write_text("def f(): return 1\n", encoding="utf-8") + result = run_scan(sub) + assert not [f for f in result.findings if f.rule_id == "WLN-ENGINE-NESTED-SCAN-ROOT"] + + +def test_run_scan_nested_src_root_has_empty_qualname_prefix(tmp_path: Path) -> None: + # Scanning P/src of a src-layout project mints the SAME qualnames as scanning P + # (module_dotted_name strips one leading src/ component) — the baseline/output + # hazards remain so the FACT still fires, but the prefix must be empty so the + # message never claims a phantom 'src.' qualname prefix. + proj = tmp_path / "proj" + (proj / ".weft" / "wardline").mkdir(parents=True) + src = proj / "src" + src.mkdir() + (src / "m.py").write_text("def f(): return 1\n", encoding="utf-8") + result = run_scan(src) + fact = next(f for f in result.findings if f.rule_id == "WLN-ENGINE-NESTED-SCAN-ROOT") + assert fact.properties["qualname_prefix"] == "" From 6e257bfa1502d7c8354ca85933003387295b9b6d Mon Sep 17 00:00:00 2001 From: John Morrissey Date: Thu, 11 Jun 2026 14:58:27 +1000 Subject: [PATCH 097/186] feat(cli): N-2 wardline explain-taint CLI twin + N-5 case-insensitive closed-vocab filters & flat flags (closes wardline-0be02bf8e6, wardline-dc6f44707d) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit N-2 (dogfood 2026-06-11): a CLI-only agent dead-ended at step 2 of the scan -> explain -> fix -> rescan loop — explain_taint was MCP-only. - core/explain: explanation_to_dict/remediation_to_dict moved here from mcp/tooling (pure projections of TaintExplanation, they belong with the dataclass) + new explain_taint_result, the FULL result-dict builder (explanation + remediation + optional Loomweave chain walk) - mcp/server._explain_taint now delegates to it (arg parsing + path confinement + ToolError stay at the MCP layer) - new cli/explain_taint.py: wardline explain-taint FINGERPRINT [PATH] (--sink-qualname/--chain/--max-hops/--loomweave-url/--config) — identical JSON by construction, pinned by a CLI==MCP parity test; stale fingerprint exits 2 with the same re-scan message the MCP tool returns - wardline-gate skill + docs/reference/cli.md updated (incl. regenerating the stale top-level command list; new explain-taint + findings sections) N-5 (weft-a828583ef7): severity in --where was uppercase-only and a wrong-case or out-of-domain value (filigree's lowercase/medium habits) silently returned empty — undiagnosable. - core/finding_query: closed-vocabulary keys (severity/suppression/kind) normalize case-insensitively to canonical casing; out-of-domain values raise ValueError naming the allowed vocabulary (shared by MCP scan(where=) and the CLI). Open keys (rule_id/qualname/sink/tier) unchanged — packs extend tiers. - cli/findings: first-class --rule-id/--severity/--sink flat flags (X-5 filter shape); a filter given both as flag and --where key is rejected, never silently overridden - scan/scan-file-findings --fail-on: case_sensitive=False (canonical uppercase echoed back) Glossary file:line anchors recomputed token-first after the server.py refactor shifted them. Co-Authored-By: Claude Fable 5 --- CHANGELOG.md | 23 ++++ docs/reference/cli.md | 88 +++++++++++-- .../reference/finding-lifecycle-vocabulary.md | 28 ++--- src/wardline/cli/explain_taint.py | 103 +++++++++++++++ src/wardline/cli/findings.py | 28 ++++- src/wardline/cli/main.py | 2 + src/wardline/cli/scan.py | 2 +- src/wardline/cli/scan_file_findings.py | 2 +- src/wardline/core/explain.py | 117 ++++++++++++++++++ src/wardline/core/finding_query.py | 30 ++++- src/wardline/mcp/server.py | 35 ++---- src/wardline/mcp/tooling.py | 62 +--------- src/wardline/skills/wardline-gate/SKILL.md | 13 +- tests/docs/test_glossary_vocabulary.py | 20 +-- tests/unit/cli/test_cli.py | 12 ++ tests/unit/cli/test_explain_taint_cmd.py | 74 +++++++++++ tests/unit/cli/test_findings_cmd.py | 43 +++++++ tests/unit/core/test_finding_query.py | 39 ++++++ tests/unit/mcp/test_tooling_explanation.py | 3 +- 19 files changed, 592 insertions(+), 132 deletions(-) create mode 100644 src/wardline/cli/explain_taint.py create mode 100644 tests/unit/cli/test_explain_taint_cmd.py diff --git a/CHANGELOG.md b/CHANGELOG.md index d7ca1aeb..0c2f6d2e 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -8,6 +8,29 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] ### Added +- **`wardline explain-taint [PATH]`** — the CLI twin of the MCP + `explain_taint` tool (same core builder, identical JSON: provenance slice, + remediation hint, optional `--chain` walk via a Loomweave store), so a + CLI-only agent no longer dead-ends at step 2 of the scan → explain → fix → + rescan loop (dogfood N-2). +- **`wardline findings` flat filter flags** `--rule-id` / `--severity` / + `--sink` alongside the JSON `--where` blob; a filter given via both is + rejected, never silently overridden (dogfood N-5/X-5). +- **Nested-scan-root guard** (dogfood N-3, `wardline-8669de3576`): scanning a + SUBDIRECTORY of a weft project (an ancestor carries `weft.toml` or + `.weft/wardline/`) now emits a `WLN-ENGINE-NESTED-SCAN-ROOT` FACT and a loud + CLI warning — qualnames are minted relative to the scan root, the project's + baseline/waivers are not loaded, and output lands in the subdirectory. + `scan --help`/`dossier --help` document the scan-root/qualname coupling, and + the dossier's entity-not-found error now names the scan-relative form that + WOULD match plus the project root to rerun against (dogfood N-8). + +### Changed +- **Closed-vocabulary query values match case-insensitively** (`severity`, + `suppression`, `kind`) in `wardline findings --where` and the MCP + `scan(where=)`; an out-of-domain value (e.g. filigree's `medium`) now errors + loudly naming the allowed vocabulary instead of silently returning empty + (dogfood N-5). `--fail-on` accepts any casing (canonical uppercase echoed). - **Six new PREVIEW sink rules, `PY-WL-121`–`PY-WL-126`** (the 2026-06-10 coverage-gap families; all tier-modulated, argument-slot precise, and construct-then-method / callable-alias aware): diff --git a/docs/reference/cli.md b/docs/reference/cli.md index d7048170..1a61bc66 100644 --- a/docs/reference/cli.md +++ b/docs/reference/cli.md @@ -42,14 +42,24 @@ Options: --help Show this message and exit. Commands: - baseline Manage the finding baseline (.weft/wardline/baseline.yaml). - decorator-coverage - List every Wardline trust-decorated entity under PATH. - file-finding - File the finding identified by FINGERPRINT into a tracked... - judge Triage active DEFECTs with the opt-in LLM judge. - scan Scan PATH for findings. - vocab Emit the NG-25 trust-vocabulary descriptor as YAML... + assure Report the trust-surface coverage posture for PATH. + attest Build a signed evidence bundle for PATH (or verify... + baseline Manage the finding baseline... + decorator-coverage List every Wardline trust-decorated entity under PATH. + doctor Check Wardline agent install artifacts and sibling... + dossier Assemble the one-call dossier for ENTITY (a... + explain-taint Explain ONE finding's taint provenance by... + file-finding File the finding identified by FINGERPRINT into a... + findings Scan PATH and print filtered findings as JSONL... + fix Scan PATH and apply autofixes interactively. + install Install wardline's agent-facing guidance and... + judge Triage active DEFECTs with the opt-in LLM judge. + lsp Run the Wardline LSP diagnostics server over stdio... + mcp Run the Wardline MCP server over stdio (JSON-RPC 2.0). + rekey Re-key baseline/waiver/judge verdicts across a... + scan Scan PATH for findings. + scan-file-findings Run the agent workflow from scan to optionally... + vocab Emit the NG-25 trust-vocabulary descriptor as YAML... ``` Check the installed version: @@ -122,6 +132,68 @@ $ wardline scan src/ --format agent-summary --output findings.agent-summary.json See the [getting-started guide](../getting-started.md) for a first end-to-end scan and how to read the findings. +## `wardline explain-taint` + +**Purpose:** explain ONE finding's taint provenance — the immediate tainted +callee, the originating boundary, the trust tiers at the sink, and a +remediation hint. The CLI twin of the MCP `explain_taint` tool (same core +builder, identical JSON), so a CLI-only agent can run the full +scan → explain → fix-at-the-boundary → rescan loop. + +```text +Usage: wardline explain-taint [OPTIONS] FINGERPRINT [PATH] +``` + +| Option | What it does | +|---|---| +| `--sink-qualname TEXT` | The finding's `qualname`: with a configured Loomweave store this serves the explanation from the store with no re-scan. | +| `--chain` | Also walk the full taint chain to the originating boundary (needs a Loomweave store; degrades to the single-hop explanation without one). | +| `--max-hops INTEGER` | Chain-walk hop budget (default 20). | +| `--loomweave-url TEXT` | Loomweave taint-store URL (opt-in; also resolved from env/published port). | +| `--config FILE` | Explicit config file. | + +Call it right after a scan and before editing: a fingerprint from a stale scan +errors with exit 2 and asks for a re-scan. `PATH` is the scan root and must +match the scan that minted the fingerprint (qualnames and fingerprints are +minted relative to it). + +```text +$ wardline scan . --fail-on ERROR +$ wardline explain-taint 40dd3530…54619e . +{ + "fingerprint": "40dd3530…54619e", + "rule_id": "PY-WL-101", + "sink_qualname": "specimen.trust_flow.leaks_untrusted", + "location": {"path": "specimen/trust_flow.py", "line": 13}, + "tier_in": "UNKNOWN_RAW", + "tier_out": "ASSURED", + "immediate_tainted_callee": "read_raw", + "source_boundary_qualname": "specimen.trust_flow.read_raw", + "remediation": {"kind": "boundary_placement", "summary": "Validate or normalize data from …"} +} +``` + +## `wardline findings` + +**Purpose:** read-only filtered query — scan PATH and print matching findings +as JSONL. The CLI counterpart of the MCP `scan(where=)` filter; no file +output, no Filigree/Loomweave emission. + +```text +Usage: wardline findings [OPTIONS] [PATH] +``` + +| Option | What it does | +|---|---| +| `--rule-id TEXT` | Filter by rule id, e.g. `PY-WL-101`. | +| `--severity TEXT` | Filter by severity, case-insensitive: `CRITICAL`/`ERROR`/`WARN`/`INFO`/`NONE`. An out-of-vocabulary value (e.g. `medium`) errors loudly with the allowed list — never a silent empty result. | +| `--sink TEXT` | Filter by the finding's `sink` property, e.g. `subprocess.run`. | +| `--where TEXT` | JSON filter object for the full predicate set (`rule_id`, `qualname`, `severity`, `suppression`, `kind`, `path_glob`, `sink`, `tier`), conjunctive. Closed-vocabulary values (`severity`, `suppression`, `kind`) match case-insensitively. | +| `--config FILE` | Explicit config file. | + +A filter given both as a flag and inside `--where` is rejected (exit 2) rather +than silently preferring one. + ## `wardline file-finding` **Purpose:** promote one already-emitted finding, keyed by fingerprint, into a diff --git a/docs/reference/finding-lifecycle-vocabulary.md b/docs/reference/finding-lifecycle-vocabulary.md index f73e6d7b..a34ae4d4 100644 --- a/docs/reference/finding-lifecycle-vocabulary.md +++ b/docs/reference/finding-lifecycle-vocabulary.md @@ -73,7 +73,7 @@ consistently, on every surface: | Enum | `src/wardline/core/finding.py:72` | `SuppressionState.ACTIVE = "active"` | | Summary field | `src/wardline/core/run.py:51`, built at `src/wardline/core/run.py:361` | `ScanSummary.active` | | CLI summary line | `src/wardline/cli/scan.py:416` | `… {s.active} active` | -| MCP scan response | `src/wardline/mcp/server.py:331` | `summary.active` | +| MCP scan response | `src/wardline/mcp/server.py:330` | `summary.active` | | Agent-summary JSON | `src/wardline/core/agent_summary.py:135` | `summary.active_defects` | | `wardline:loop` prompt | `src/wardline/mcp/prompts.py:13` | "Read `summary.active`" | @@ -101,8 +101,8 @@ So `active + baselined + waived + judged + informational == total` (`src/wardline/core/run.py:50` for `total: int`). `unanalyzed` (`src/wardline/core/run.py:69`) is an **overlay** — a subset of `informational` that surfaces a silent under-scan — and is deliberately **not** a partition member. -The MCP `summary` block exposes `informational` (`src/wardline/mcp/server.py:339`) -and `unanalyzed` (`src/wardline/mcp/server.py:343`); the agent-summary block mirrors +The MCP `summary` block exposes `informational` (`src/wardline/mcp/server.py:338`) +and `unanalyzed` (`src/wardline/mcp/server.py:342`); the agent-summary block mirrors both (`src/wardline/core/agent_summary.py:146`, `src/wardline/core/agent_summary.py:147`). ## Emitted-active vs the gate population @@ -143,9 +143,9 @@ judged. The `verdict` is one of: - **`PASSED`** — a threshold ran and nothing tripped. - **`FAILED`** — a threshold ran and tripped. -The MCP `scan` gate block exposes `gate.tripped` (`src/wardline/mcp/server.py:346`), -`gate.verdict` (`src/wardline/mcp/server.py:349`), `would_trip_at`, `reason`, -`evaluated`, and `migration_hint`, opened at `src/wardline/mcp/server.py:345` +The MCP `scan` gate block exposes `gate.tripped` (`src/wardline/mcp/server.py:345`), +`gate.verdict` (`src/wardline/mcp/server.py:348`), `would_trip_at`, `reason`, +`evaluated`, and `migration_hint`, opened at `src/wardline/mcp/server.py:344` (`"gate": {`); the agent-summary mirrors them at `src/wardline/core/agent_summary.py:150` (`tripped`) and `src/wardline/core/agent_summary.py:153` (`verdict`). The CLI prints @@ -178,16 +178,16 @@ How each concept appears on each surface: | Concept | CLI summary text | `ScanSummary` field | MCP `summary` key | Agent-summary key | Filigree store | | --- | --- | --- | --- | --- | --- | -| every finding | `N finding(s)` | `total` (`run.py:50`) | `total` (`server.py:330`) | `total_findings` (`agent_summary.py:134`) | one finding per wire entry | -| live defect | `N active` (`scan.py:416`) | `active` (`run.py:51,361`) | `active` (`server.py:331`) | `active_defects` (`agent_summary.py:135`) | no `suppression_state` key (`finding.py:285`) | +| every finding | `N finding(s)` | `total` (`run.py:50`) | `total` (`server.py:329`) | `total_findings` (`agent_summary.py:134`) | one finding per wire entry | +| live defect | `N active` (`scan.py:416`) | `active` (`run.py:51,361`) | `active` (`server.py:330`) | `active_defects` (`agent_summary.py:135`) | no `suppression_state` key (`finding.py:285`) | | suppressed (sum) | `N suppressed` (`scan.py:415`) | `baselined+waived+judged` | the three keys | `suppressed_findings` (`agent_summary.py:136`) | `metadata.wardline.suppression_state` (`finding.py:285`) | -| baselined | `N baseline` | `baselined` (`run.py:53`) | `baselined` (`server.py:332`) | `baselined` (`agent_summary.py:138`) | `suppression_state: "baselined"` | -| waived | `N waiver` | `waived` (`run.py:54`) | `waived` (`server.py:333`) | `waived` (`agent_summary.py:139`) | `suppression_state: "waived"` | -| judged | `N judged` | `judged` (`run.py:55`) | `judged` (`server.py:334`) | `judged` (`agent_summary.py:140`) | `suppression_state: "judged"` | -| informational (summary) | (the remainder of `total`) | `informational` (`run.py:61`) | `informational` (`server.py:339`) | `informational` (`agent_summary.py:146`) | facts/metrics | +| baselined | `N baseline` | `baselined` (`run.py:53`) | `baselined` (`server.py:331`) | `baselined` (`agent_summary.py:138`) | `suppression_state: "baselined"` | +| waived | `N waiver` | `waived` (`run.py:54`) | `waived` (`server.py:332`) | `waived` (`agent_summary.py:139`) | `suppression_state: "waived"` | +| judged | `N judged` | `judged` (`run.py:55`) | `judged` (`server.py:333`) | `judged` (`agent_summary.py:140`) | `suppression_state: "judged"` | +| informational (summary) | (the remainder of `total`) | `informational` (`run.py:61`) | `informational` (`server.py:338`) | `informational` (`agent_summary.py:146`) | facts/metrics | | informational (display) | n/a | n/a | n/a | `informational` display array (`agent_summary.py:171`) — non-defect, non-engine-fact findings (metrics, classifications, suggestions, non-engine facts); excludes `engine_facts` which has its own display slot | facts/metrics | -| under-scan | `N file(s) could not be analyzed` | `unanalyzed` (`run.py:69`) | `unanalyzed` (`server.py:343`) | `unanalyzed` (`agent_summary.py:147`) | `WLN-ENGINE-*` facts | -| gate verdict | exit code + `--fail-on` | (`gate_findings`, `run.py:88`; `GateDecision`, `run.py:98`, `verdict` `run.py:107`) | `gate.tripped` (`server.py:346`), `gate.verdict` (`server.py:349`) | `gate.tripped` (`agent_summary.py:150`), `gate.verdict` (`agent_summary.py:153`) | not emitted to Filigree | +| under-scan | `N file(s) could not be analyzed` | `unanalyzed` (`run.py:69`) | `unanalyzed` (`server.py:342`) | `unanalyzed` (`agent_summary.py:147`) | `WLN-ENGINE-*` facts | +| gate verdict | exit code + `--fail-on` | (`gate_findings`, `run.py:88`; `GateDecision`, `run.py:98`, `verdict` `run.py:107`) | `gate.tripped` (`server.py:345`), `gate.verdict` (`server.py:348`) | `gate.tripped` (`agent_summary.py:150`), `gate.verdict` (`agent_summary.py:153`) | not emitted to Filigree | The unsuppressed gate population is built from `Baseline(frozenset())` (`src/wardline/core/run.py:327`). diff --git a/src/wardline/cli/explain_taint.py b/src/wardline/cli/explain_taint.py new file mode 100644 index 00000000..78bdc4dd --- /dev/null +++ b/src/wardline/cli/explain_taint.py @@ -0,0 +1,103 @@ +# src/wardline/cli/explain_taint.py +"""`wardline explain-taint` — the CLI twin of the MCP `explain_taint` tool (N-2). + +Thin delegator to ``core.explain.explain_taint_result`` (the same builder the +MCP handler calls — CLI and MCP identical by construction), so a CLI-only agent +can run the full scan -> explain -> fix-at-the-boundary -> rescan loop without +an MCP server.""" + +from __future__ import annotations + +import json +from pathlib import Path + +import click + +from wardline.core.config import resolve_loomweave_url +from wardline.core.errors import WardlineError + + +@click.command("explain-taint") +@click.argument("fingerprint", type=str) +@click.argument("path", type=click.Path(exists=True, file_okay=False, path_type=Path), default=".") +@click.option( + "--config", + "config_path", + type=click.Path(exists=True, file_okay=True, dir_okay=False, path_type=Path), + default=None, +) +@click.option( + "--sink-qualname", + default=None, + help=( + "The finding's qualname: with a configured Loomweave store this serves " + "the explanation from the store (no re-scan)." + ), +) +@click.option( + "--chain", + is_flag=True, + default=False, + help=( + "Also walk the full taint chain to the originating boundary (needs a " + "Loomweave store; degrades to single-hop without one)." + ), +) +@click.option("--max-hops", type=int, default=20, show_default=True, help="Chain-walk hop budget.") +@click.option( + "--loomweave-url", + "loomweave_url", + default=None, + help="Loomweave taint-store URL (opt-in; also resolved from env/published port).", +) +def explain_taint( + fingerprint: str, + path: Path, + config_path: Path | None, + sink_qualname: str | None, + chain: bool, + max_hops: int, + loomweave_url: str | None, +) -> None: + """Explain ONE finding's taint provenance by FINGERPRINT under PATH. + + Prints the immediate tainted callee, the originating boundary, the trust + tiers at the sink, and a remediation hint — the same JSON the MCP + `explain_taint` tool returns. Call right after a scan and before editing: + a fingerprint from a stale scan errors (exit 2) and asks for a re-scan. + PATH is the scan root and must match the scan that minted the fingerprint. + """ + try: + loomweave_url = resolve_loomweave_url(loomweave_url, path, config_path) + loomweave = None + if loomweave_url is not None: + from wardline.loomweave.client import LoomweaveClient + from wardline.loomweave.config import load_loomweave_token, resolve_project_name + + loomweave = LoomweaveClient( + loomweave_url, + secret=load_loomweave_token(path), + project=resolve_project_name(path), + ) + from wardline.core.explain import explain_taint_result + + result = explain_taint_result( + path, + fingerprint=fingerprint, + config_path=config_path, + confine_to_root=True, + loomweave=loomweave, + sink_qualname=sink_qualname, + chain=chain, + max_hops=max_hops, + ) + except WardlineError as exc: + click.echo(f"error: {exc}", err=True) + raise SystemExit(2) from exc + if result is None: + click.echo( + "error: fingerprint not in current scan; your code changed since the scan that produced it — re-scan.", + err=True, + ) + raise SystemExit(2) + click.echo(json.dumps(result, indent=2, ensure_ascii=False)) diff --git a/src/wardline/cli/findings.py b/src/wardline/cli/findings.py index 6f2db077..3240bba0 100644 --- a/src/wardline/cli/findings.py +++ b/src/wardline/cli/findings.py @@ -27,7 +27,21 @@ default=None, ) @click.option("--where", "where_json", default=None, help='JSON filter object, e.g. \'{"rule_id":"PY-WL-106"}\'.') -def findings(path: Path, config_path: Path | None, where_json: str | None) -> None: +# N-5 / X-5 (wardline-dc6f44707d): the common filters as first-class flat flags so +# an agent does not need the JSON --where blob (filigree-style flag shape). +@click.option( + "--rule-id", "rule_id", default=None, help='Filter by rule id, e.g. PY-WL-101 (same as --where {"rule_id":...}).' +) +@click.option("--severity", default=None, help="Filter by severity (case-insensitive): CRITICAL/ERROR/WARN/INFO/NONE.") +@click.option("--sink", default=None, help="Filter by the finding's sink property, e.g. subprocess.run.") +def findings( + path: Path, + config_path: Path | None, + where_json: str | None, + rule_id: str | None, + severity: str | None, + sink: str | None, +) -> None: """Scan PATH and print filtered findings as JSONL (read-only).""" where = None if where_json is not None: @@ -39,6 +53,18 @@ def findings(path: Path, config_path: Path | None, where_json: str | None) -> No if not isinstance(where, dict): click.echo("error: --where must be a JSON object", err=True) raise SystemExit(2) + flat = {k: v for k, v in {"rule_id": rule_id, "severity": severity, "sink": sink}.items() if v is not None} + if flat: + # A flag and a --where key naming the same filter is ambiguous — refuse + # rather than silently prefer one (the silent-override anti-pattern). + overlap = sorted(set(flat) & set(where or {})) + if overlap: + click.echo( + f"error: {', '.join(overlap)} given both as a flag and inside --where; pass each filter once", + err=True, + ) + raise SystemExit(2) + where = {**(where or {}), **flat} result = run_scan(path, config_path=config_path) try: resolved_where = resolve_query_filters(where, path, config_path) diff --git a/src/wardline/cli/main.py b/src/wardline/cli/main.py index e508ed3a..8502031f 100644 --- a/src/wardline/cli/main.py +++ b/src/wardline/cli/main.py @@ -14,6 +14,7 @@ from wardline.cli.decorator_coverage import decorator_coverage from wardline.cli.doctor import doctor from wardline.cli.dossier import dossier +from wardline.cli.explain_taint import explain_taint from wardline.cli.file_finding import file_finding from wardline.cli.findings import findings from wardline.cli.fix import fix @@ -47,6 +48,7 @@ def cli() -> None: cli.add_command(install) cli.add_command(doctor) cli.add_command(dossier) +cli.add_command(explain_taint) cli.add_command(findings) cli.add_command(file_finding) cli.add_command(assure) diff --git a/src/wardline/cli/scan.py b/src/wardline/cli/scan.py index 51f62854..38ed4191 100644 --- a/src/wardline/cli/scan.py +++ b/src/wardline/cli/scan.py @@ -48,7 +48,7 @@ ) @click.option("--output", type=click.Path(path_type=Path), default=None) # exit 1 if any non-suppressed DEFECT has severity >= this threshold (SP3b) -@click.option("--fail-on", type=click.Choice(["CRITICAL", "ERROR", "WARN", "INFO"]), default=None) +@click.option("--fail-on", type=click.Choice(["CRITICAL", "ERROR", "WARN", "INFO"], case_sensitive=False), default=None) # Opt-in CI enforcement: exit 1 when any file was discovered but not analysed # (parse error / too-deep / missing source root — NOT benign no-module skips). # Default FALSE preserves the released exit-code behaviour; the count is ALWAYS diff --git a/src/wardline/cli/scan_file_findings.py b/src/wardline/cli/scan_file_findings.py index 368bf600..03a5d719 100644 --- a/src/wardline/cli/scan_file_findings.py +++ b/src/wardline/cli/scan_file_findings.py @@ -20,7 +20,7 @@ @click.command(name="scan-file-findings") @click.argument("path", type=click.Path(exists=True, file_okay=False, path_type=Path), default=".") @click.option("--config", "config_path", type=click.Path(exists=True, file_okay=True, dir_okay=False, path_type=Path)) -@click.option("--fail-on", type=click.Choice(["CRITICAL", "ERROR", "WARN", "INFO"]), default=None) +@click.option("--fail-on", type=click.Choice(["CRITICAL", "ERROR", "WARN", "INFO"], case_sensitive=False), default=None) @click.option("--cache-dir", type=click.Path(path_type=Path), default=None) @click.option("--filigree-url", "filigree_url", default=None, help="Filigree Weft URL (else flag/env).") @click.option("--loomweave-url", "loomweave_url", default=None, help="Loomweave URL for optional identity attachment.") diff --git a/src/wardline/core/explain.py b/src/wardline/core/explain.py index 0d9bbe95..605a058b 100644 --- a/src/wardline/core/explain.py +++ b/src/wardline/core/explain.py @@ -335,3 +335,120 @@ def explain_finding( config_path=config_path, confine_to_root=confine_to_root, ) + + +def explanation_to_dict(exp: TaintExplanation) -> dict[str, Any]: + """The serialized explanation slice + remediation hint. Single source for the + MCP ``explain_taint`` result and the CLI ``wardline explain-taint`` output + (identical by construction — the N-2 dead-end was the CLI lacking this).""" + return { + "tier_in": exp.tier_in, + "tier_out": exp.tier_out, + "immediate_tainted_callee": exp.immediate_tainted_callee, + "source_boundary_qualname": exp.source_boundary_qualname, + "resolved_call_count": exp.resolved_call_count, + "unresolved_call_count": exp.unresolved_call_count, + "remediation": remediation_to_dict(exp), + } + + +def remediation_to_dict(exp: TaintExplanation) -> dict[str, Any]: + if exp.rule_id != "PY-WL-101": + return { + "kind": "review_required", + "rule_id": exp.rule_id, + "summary": ( + "Review the finding and apply the rule-specific fix; no automated remediation hint is available." + ), + "sink_qualname": exp.sink_qualname, + "source_qualname": exp.source_boundary_qualname, + "caveat": "This hint is advisory and does not replace the factual taint explanation.", + } + + source = exp.source_boundary_qualname or exp.immediate_tainted_callee + sink = exp.sink_qualname + if source and sink: + summary = ( + f"Validate or normalize data from {source} before it reaches trusted producer {sink}. " + "Add or repair a @trust_boundary only on the function that actually rejects invalid data." + ) + elif sink: + summary = ( + f"Validate or normalize the raw input before it reaches trusted producer {sink}; " + "the taint source is unresolved in this explanation. Add or repair a @trust_boundary only where " + "the code actually rejects invalid data." + ) + else: + summary = ( + "Validate or normalize the raw input before it reaches the trusted producer; the taint source is " + "unresolved in this explanation. Add or repair a @trust_boundary only where the code actually " + "rejects invalid data." + ) + return { + "kind": "boundary_placement", + "rule_id": exp.rule_id, + "summary": summary, + "sink_qualname": sink, + "source_qualname": source, + "caveat": ( + "Do not use blind decorator insertion; mark a trust boundary only on code that validates " + "and rejects invalid data." + ), + } + + +def explain_taint_result( + root: Path, + *, + fingerprint: str | None = None, + path: str | None = None, + line: int | None = None, + config_path: Path | None = None, + confine_to_root: bool = True, + loomweave: Any | None = None, + sink_qualname: str | None = None, + chain: bool = False, + max_hops: int = 20, +) -> dict[str, Any] | None: + """The full ``explain_taint`` result dict shared by the MCP handler and the + CLI command. None means the fingerprint/location is not in the current scan + (the caller maps that to its own error channel — ToolError or exit 2). + + ``chain=True`` additionally walks the full taint chain when a Loomweave + store is configured; without one it degrades silently to the single-hop + explanation (no ``chain`` block), exactly as the MCP tool documents. + """ + exp = explain_finding( + root, + fingerprint=fingerprint, + path=path, + line=line, + config_path=config_path, + confine_to_root=confine_to_root, + loomweave=loomweave, + sink_qualname=sink_qualname, + ) + if exp is None: + return None + result: dict[str, Any] = { + "fingerprint": exp.fingerprint, + "rule_id": exp.rule_id, + "sink_qualname": exp.sink_qualname, + "location": {"path": exp.path, "line": exp.line}, + **explanation_to_dict(exp), + } + if chain and loomweave is not None and exp.sink_qualname: + ch = explain_chain(root, sink_qualname=exp.sink_qualname, loomweave=loomweave, max_hops=max_hops) + result["chain"] = { + "hops": [ + { + "qualname": h.qualname, + "tier_in": h.tier_in, + "tier_out": h.tier_out, + "contributing_callee_qualname": h.contributing_callee_qualname, + } + for h in ch.hops + ], + "truncated_at": ch.truncated_at, + } + return result diff --git a/src/wardline/core/finding_query.py b/src/wardline/core/finding_query.py index e018cb39..1c7a2cff 100644 --- a/src/wardline/core/finding_query.py +++ b/src/wardline/core/finding_query.py @@ -10,7 +10,7 @@ from fnmatch import fnmatch from typing import Any -from wardline.core.finding import Finding +from wardline.core.finding import Finding, Kind, Severity, SuppressionState # Property keys that carry a trust-tier value across the rule set: 101/109 -> # actual_return/declared_return; 106/107/108 -> tier/arg_taint; 104/105 -> @@ -20,6 +20,33 @@ _ALLOWED = frozenset({"rule_id", "qualname", "severity", "suppression", "kind", "path_glob", "sink", "tier"}) +# Closed-vocabulary predicate keys (N-5, wardline-dc6f44707d): their values come +# from an enum, so a wrong-case or out-of-domain value can NEVER match — a silent +# empty result there is a bad-error an agent cannot diagnose (filigree's lowercase +# severity habit was the live trip). Normalize case-insensitively to the canonical +# casing; reject anything outside the domain loudly, naming the vocabulary. +# rule_id/qualname/sink/tier stay OPEN (packs can extend tiers; rule ids are data). +_CLOSED_VOCAB: dict[str, tuple[str, ...]] = { + "severity": tuple(s.value for s in Severity), + "suppression": tuple(s.value for s in SuppressionState), + "kind": tuple(k.value for k in Kind), +} + + +def _normalize_closed_vocab(where: Mapping[str, Any]) -> dict[str, Any]: + normalized = dict(where) + for key, allowed in _CLOSED_VOCAB.items(): + value = normalized.get(key) + if value is None: + continue + if not isinstance(value, str): + raise ValueError(f"filter {key!r} must be a string; allowed (case-insensitive): {list(allowed)}") + canonical = next((a for a in allowed if a.lower() == value.lower()), None) + if canonical is None: + raise ValueError(f"unknown {key} {value!r}; allowed (case-insensitive): {list(allowed)}") + normalized[key] = canonical + return normalized + def _matches(f: Finding, where: Mapping[str, Any]) -> bool: if (v := where.get("rule_id")) is not None and f.rule_id != v: @@ -47,4 +74,5 @@ def filter_findings(findings: Sequence[Finding], where: Mapping[str, Any] | None unknown = set(where) - _ALLOWED if unknown: raise ValueError(f"unknown filter key(s): {sorted(unknown)}; allowed: {sorted(_ALLOWED)}") + where = _normalize_closed_vocab(where) return [f for f in findings if _matches(f, where)] diff --git a/src/wardline/mcp/server.py b/src/wardline/mcp/server.py index 860e768d..caeb6c89 100644 --- a/src/wardline/mcp/server.py +++ b/src/wardline/mcp/server.py @@ -20,7 +20,7 @@ from wardline.core.attest_key import load_attest_key from wardline.core.baseline import generate_baseline, load_baseline from wardline.core.errors import WardlineError -from wardline.core.explain import explain_chain, explain_finding, explanation_from_context +from wardline.core.explain import explain_taint_result, explanation_from_context, explanation_to_dict from wardline.core.filigree_emit import FiligreeEmitter, filigree_destination, filigree_disabled_reason from wardline.core.finding import Finding, Severity from wardline.core.finding_query import filter_findings @@ -35,7 +35,6 @@ from wardline.mcp.resources import list_resources, read_resource from wardline.mcp.tooling import Tool, ToolCapability, ToolError, ToolPolicy from wardline.mcp.tooling import cfg as _cfg -from wardline.mcp.tooling import explanation_to_dict as _explanation_to_dict from wardline.mcp.tooling import require as _require from wardline.mcp.tooling import resolve_under_root as _resolve_under_root @@ -318,7 +317,7 @@ def _scan( if f is None or f.qualname is None: continue if attached < explain_cap: - entry["explanation"] = _explanation_to_dict(explanation_from_context(f, result.context)) + entry["explanation"] = explanation_to_dict(explanation_from_context(f, result.context)) attached += 1 else: explanations_truncated = True @@ -454,7 +453,8 @@ def _explain_taint(args: dict[str, Any], root: Path, loomweave: Any = None) -> d match_path = args.get("path") if args.get("line") is not None else None if match_path is not None: _resolve_under_root(root, match_path) # reject escapes; result discarded - exp = explain_finding( + max_hops_raw = args.get("max_hops") + result_dict = explain_taint_result( root, fingerprint=args.get("fingerprint"), path=match_path, @@ -463,34 +463,13 @@ def _explain_taint(args: dict[str, Any], root: Path, loomweave: Any = None) -> d confine_to_root=True, loomweave=loomweave, sink_qualname=args.get("sink_qualname"), + chain=bool(args.get("chain")), + max_hops=int(max_hops_raw) if max_hops_raw is not None else 20, ) - if exp is None: + if result_dict is None: raise ToolError( "fingerprint not in current scan; your code changed since the scan that produced it — re-scan.", ) - result_dict: dict[str, Any] = { - "fingerprint": exp.fingerprint, - "rule_id": exp.rule_id, - "sink_qualname": exp.sink_qualname, - "location": {"path": exp.path, "line": exp.line}, - **_explanation_to_dict(exp), - } - if args.get("chain") and loomweave is not None and exp.sink_qualname: - max_hops_raw = args.get("max_hops") - max_hops = int(max_hops_raw) if max_hops_raw is not None else 20 - ch = explain_chain(root, sink_qualname=exp.sink_qualname, loomweave=loomweave, max_hops=max_hops) - result_dict["chain"] = { - "hops": [ - { - "qualname": h.qualname, - "tier_in": h.tier_in, - "tier_out": h.tier_out, - "contributing_callee_qualname": h.contributing_callee_qualname, - } - for h in ch.hops - ], - "truncated_at": ch.truncated_at, - } return result_dict diff --git a/src/wardline/mcp/tooling.py b/src/wardline/mcp/tooling.py index 784bd3dd..afd7ea96 100644 --- a/src/wardline/mcp/tooling.py +++ b/src/wardline/mcp/tooling.py @@ -7,13 +7,10 @@ from dataclasses import dataclass from enum import StrEnum from pathlib import Path -from typing import TYPE_CHECKING, Any +from typing import Any from wardline.core.finding import Finding -if TYPE_CHECKING: - from wardline.core.explain import TaintExplanation - class ToolError(Exception): """Tool-execution error returned to the MCP client as an isError result.""" @@ -67,63 +64,6 @@ def finding_to_dict(finding: Finding) -> dict[str, Any]: return parsed -def explanation_to_dict(exp: TaintExplanation) -> dict[str, Any]: - return { - "tier_in": exp.tier_in, - "tier_out": exp.tier_out, - "immediate_tainted_callee": exp.immediate_tainted_callee, - "source_boundary_qualname": exp.source_boundary_qualname, - "resolved_call_count": exp.resolved_call_count, - "unresolved_call_count": exp.unresolved_call_count, - "remediation": remediation_to_dict(exp), - } - - -def remediation_to_dict(exp: TaintExplanation) -> dict[str, Any]: - if exp.rule_id != "PY-WL-101": - return { - "kind": "review_required", - "rule_id": exp.rule_id, - "summary": ( - "Review the finding and apply the rule-specific fix; no automated remediation hint is available." - ), - "sink_qualname": exp.sink_qualname, - "source_qualname": exp.source_boundary_qualname, - "caveat": "This hint is advisory and does not replace the factual taint explanation.", - } - - source = exp.source_boundary_qualname or exp.immediate_tainted_callee - sink = exp.sink_qualname - if source and sink: - summary = ( - f"Validate or normalize data from {source} before it reaches trusted producer {sink}. " - "Add or repair a @trust_boundary only on the function that actually rejects invalid data." - ) - elif sink: - summary = ( - f"Validate or normalize the raw input before it reaches trusted producer {sink}; " - "the taint source is unresolved in this explanation. Add or repair a @trust_boundary only where " - "the code actually rejects invalid data." - ) - else: - summary = ( - "Validate or normalize the raw input before it reaches the trusted producer; the taint source is " - "unresolved in this explanation. Add or repair a @trust_boundary only where the code actually " - "rejects invalid data." - ) - return { - "kind": "boundary_placement", - "rule_id": exp.rule_id, - "summary": summary, - "sink_qualname": sink, - "source_qualname": source, - "caveat": ( - "Do not use blind decorator insertion; mark a trust boundary only on code that validates " - "and rejects invalid data." - ), - } - - def resolve_under_root(root: Path, arg: str) -> Path: """Resolve a caller-supplied path/config arg against root and refuse escapes.""" candidate = (root / arg).resolve() diff --git a/src/wardline/skills/wardline-gate/SKILL.md b/src/wardline/skills/wardline-gate/SKILL.md index 87ee134a..7a3d4aa8 100644 --- a/src/wardline/skills/wardline-gate/SKILL.md +++ b/src/wardline/skills/wardline-gate/SKILL.md @@ -20,11 +20,12 @@ receive validated data). When untrusted data reaches a trusted producer it raise 1. **Scan.** Run `wardline scan . --fail-on ERROR` (or call the `scan` MCP tool). Read the gate verdict and the active (non-suppressed) findings — `active` is the population the gate enforces on. -2. **Explain.** For each active defect, call `explain_taint` with the finding's - `fingerprint`, `path`+`line`, and its `qualname` as `sink_qualname`. Do this +2. **Explain.** For each active defect, call `explain_taint` (MCP) or run + `wardline explain-taint [PATH]` (CLI) with the finding's + `fingerprint`, and its `qualname` as `sink_qualname`. Do this right after the scan and before editing — a stale fingerprint returns an error. - With a Loomweave store configured, pass `chain: true` to walk the full taint - chain back to the originating boundary. + With a Loomweave store configured, pass `chain: true` (`--chain` on the CLI) + to walk the full taint chain back to the originating boundary. 3. **Fix at the BOUNDARY, not the sink.** Add validation or rejection at the hop where untrusted data should have been checked — not a band-aid at the sink. 4. **Re-scan.** Confirm the finding is gone. @@ -56,7 +57,9 @@ non-issue, always with a reason: ## CLI vs MCP -- **CLI:** `wardline scan`, `wardline judge`, `wardline baseline create/update`. +- **CLI:** `wardline scan`, `wardline explain-taint`, `wardline findings` + (read-only filtered query: `--rule-id` / `--severity` / `--sink` or a JSON + `--where`), `wardline judge`, `wardline baseline create/update`. Branch on the exit code; read the findings file it writes. - **MCP:** `wardline mcp` exposes `scan`, `explain_taint`, `fix`, `judge` (network), `baseline`, `waiver_add`; resources diff --git a/tests/docs/test_glossary_vocabulary.py b/tests/docs/test_glossary_vocabulary.py index 8657e4aa..881cd5c6 100644 --- a/tests/docs/test_glossary_vocabulary.py +++ b/tests/docs/test_glossary_vocabulary.py @@ -45,16 +45,16 @@ ("src/wardline/cli/scan.py", 416, "{s.active} active"), ("src/wardline/cli/scan.py", 456, "gate: FAILED"), # src/wardline/mcp/server.py — MCP scan summary + gate block - ("src/wardline/mcp/server.py", 330, '"total": result.summary.total'), - ("src/wardline/mcp/server.py", 331, '"active": result.summary.active'), - ("src/wardline/mcp/server.py", 332, '"baselined": result.summary.baselined'), - ("src/wardline/mcp/server.py", 333, '"waived": result.summary.waived'), - ("src/wardline/mcp/server.py", 334, '"judged": result.summary.judged'), - ("src/wardline/mcp/server.py", 339, '"informational": result.summary.informational'), - ("src/wardline/mcp/server.py", 343, '"unanalyzed": result.summary.unanalyzed'), - ("src/wardline/mcp/server.py", 345, '"gate": {'), - ("src/wardline/mcp/server.py", 346, '"tripped": decision.tripped'), - ("src/wardline/mcp/server.py", 349, '"verdict": decision.verdict'), + ("src/wardline/mcp/server.py", 329, '"total": result.summary.total'), + ("src/wardline/mcp/server.py", 330, '"active": result.summary.active'), + ("src/wardline/mcp/server.py", 331, '"baselined": result.summary.baselined'), + ("src/wardline/mcp/server.py", 332, '"waived": result.summary.waived'), + ("src/wardline/mcp/server.py", 333, '"judged": result.summary.judged'), + ("src/wardline/mcp/server.py", 338, '"informational": result.summary.informational'), + ("src/wardline/mcp/server.py", 342, '"unanalyzed": result.summary.unanalyzed'), + ("src/wardline/mcp/server.py", 344, '"gate": {'), + ("src/wardline/mcp/server.py", 345, '"tripped": decision.tripped'), + ("src/wardline/mcp/server.py", 348, '"verdict": decision.verdict'), # src/wardline/core/agent_summary.py — agent-summary JSON keys ("src/wardline/core/agent_summary.py", 134, '"total_findings"'), ("src/wardline/core/agent_summary.py", 135, '"active_defects"'), diff --git a/tests/unit/cli/test_cli.py b/tests/unit/cli/test_cli.py index 8b510d4d..1bc9b314 100644 --- a/tests/unit/cli/test_cli.py +++ b/tests/unit/cli/test_cli.py @@ -1486,3 +1486,15 @@ def test_dossier_help_documents_scan_root_qualname_coupling() -> None: assert result.exit_code == 0 helptext = result.output.lower() assert "scan root" in helptext or "project root" in helptext + + +def test_scan_fail_on_accepts_lowercase(tmp_path: Path) -> None: + # N-5 (wardline-dc6f44707d): --fail-on was uppercase-only; an agent carrying + # filigree's lowercase habit got a usage error. Case-insensitive now; the + # canonical (uppercase) form is what the gate output echoes back. + proj = tmp_path / "proj" + proj.mkdir() + (proj / "svc.py").write_text(_LEAKY, encoding="utf-8") + result = CliRunner().invoke(cli, ["scan", str(proj), "--fail-on", "error"]) + assert result.exit_code == 1, result.output + assert "--fail-on ERROR" in result.stderr diff --git a/tests/unit/cli/test_explain_taint_cmd.py b/tests/unit/cli/test_explain_taint_cmd.py new file mode 100644 index 00000000..04ad289f --- /dev/null +++ b/tests/unit/cli/test_explain_taint_cmd.py @@ -0,0 +1,74 @@ +# tests/unit/cli/test_explain_taint_cmd.py +"""N-2 (wardline-0be02bf8e6): `wardline explain-taint` — the CLI twin of the MCP +`explain_taint` tool, so a CLI-only agent does not dead-end at step 2 of the +scan -> explain -> fix -> rescan loop. Thin wrapper over the same core builder +(`core.explain.explain_taint_result`) the MCP handler uses — identical by +construction.""" + +from __future__ import annotations + +import json +from pathlib import Path + +from click.testing import CliRunner + +from wardline.cli.main import cli +from wardline.core.run import run_scan + +_LEAKY = ( + "from wardline.decorators import external_boundary, trusted\n" + "@external_boundary\ndef read_raw(p):\n return p\n" + "@trusted\ndef leaky(p):\n return read_raw(p)\n" +) + + +def _leaky_proj(tmp_path: Path) -> tuple[Path, str]: + proj = tmp_path / "proj" + proj.mkdir() + (proj / "svc.py").write_text(_LEAKY, encoding="utf-8") + fp = next(f for f in run_scan(proj).findings if f.rule_id == "PY-WL-101").fingerprint + return proj, fp + + +def test_explain_taint_by_fingerprint(tmp_path: Path) -> None: + proj, fp = _leaky_proj(tmp_path) + res = CliRunner().invoke(cli, ["explain-taint", fp, str(proj)]) + assert res.exit_code == 0, res.output + payload = json.loads(res.output) + assert payload["fingerprint"] == fp + assert payload["rule_id"] == "PY-WL-101" + assert payload["sink_qualname"] == "svc.leaky" + assert payload["immediate_tainted_callee"] == "read_raw" + # the remediation hint rides along, exactly as the MCP result carries it + assert payload["remediation"]["kind"] == "boundary_placement" + + +def test_explain_taint_matches_mcp_result_shape(tmp_path: Path) -> None: + # Identical-by-construction pin: the CLI JSON equals the MCP handler's result + # dict for the same finding (no chain, no loomweave). + from wardline.mcp.server import _explain_taint + + proj, fp = _leaky_proj(tmp_path) + mcp_result = _explain_taint({"fingerprint": fp}, proj) + res = CliRunner().invoke(cli, ["explain-taint", fp, str(proj)]) + assert res.exit_code == 0, res.output + assert json.loads(res.output) == mcp_result + + +def test_explain_taint_stale_fingerprint_exits_2(tmp_path: Path) -> None: + proj, _ = _leaky_proj(tmp_path) + res = CliRunner().invoke(cli, ["explain-taint", "0" * 64, str(proj)]) + assert res.exit_code == 2 + err = res.output + res.stderr + assert "re-scan" in err # same actionable message the MCP tool returns + + +def test_explain_taint_listed_in_cli_help() -> None: + res = CliRunner().invoke(cli, ["--help"]) + assert "explain-taint" in res.output + + +def test_explain_taint_help_names_the_loop() -> None: + res = CliRunner().invoke(cli, ["explain-taint", "--help"]) + assert res.exit_code == 0 + assert "fingerprint" in res.output.lower() diff --git a/tests/unit/cli/test_findings_cmd.py b/tests/unit/cli/test_findings_cmd.py index 47922e97..bcbcad45 100644 --- a/tests/unit/cli/test_findings_cmd.py +++ b/tests/unit/cli/test_findings_cmd.py @@ -41,3 +41,46 @@ def test_findings_non_object_where_exits_2(tmp_path): res = CliRunner().invoke(cli, ["findings", str(tmp_path), "--where", "[1,2]"]) assert res.exit_code == 2 assert "JSON object" in res.output + + +# --- N-5 (wardline-dc6f44707d): case-insensitive severity + flat flags -------- + + +def test_findings_lowercase_severity_matches(tmp_path): + (tmp_path / "svc.py").write_text(_SRC, encoding="utf-8") + res = CliRunner().invoke(cli, ["findings", str(tmp_path), "--where", json.dumps({"severity": "error"})]) + assert res.exit_code == 0 + lines = [json.loads(line) for line in res.output.splitlines() if line.strip()] + assert lines and all(d["severity"] == "ERROR" for d in lines) + + +def test_findings_out_of_domain_severity_exits_2_with_vocab(tmp_path): + (tmp_path / "svc.py").write_text(_SRC, encoding="utf-8") + res = CliRunner().invoke(cli, ["findings", str(tmp_path), "--where", json.dumps({"severity": "medium"})]) + assert res.exit_code == 2 + err = res.output + res.stderr + assert "medium" in err and "ERROR" in err # names the offender and the vocabulary + + +def test_findings_flat_flags_filter(tmp_path): + (tmp_path / "svc.py").write_text(_SRC, encoding="utf-8") + res = CliRunner().invoke(cli, ["findings", str(tmp_path), "--rule-id", "PY-WL-101", "--severity", "error"]) + assert res.exit_code == 0, res.output + lines = [json.loads(line) for line in res.output.splitlines() if line.strip()] + assert lines and all(d["rule_id"] == "PY-WL-101" and d["severity"] == "ERROR" for d in lines) + + +def test_findings_flat_flag_conflicts_with_where_key(tmp_path): + (tmp_path / "svc.py").write_text(_SRC, encoding="utf-8") + res = CliRunner().invoke( + cli, + ["findings", str(tmp_path), "--rule-id", "PY-WL-101", "--where", json.dumps({"rule_id": "PY-WL-106"})], + ) + assert res.exit_code == 2 + assert "rule_id" in (res.output + res.stderr) + + +def test_findings_sink_flat_flag_accepted(tmp_path): + (tmp_path / "svc.py").write_text(_SRC, encoding="utf-8") + res = CliRunner().invoke(cli, ["findings", str(tmp_path), "--sink", "subprocess.run"]) + assert res.exit_code == 0, res.output # no sink-family finding here: empty, not an error diff --git a/tests/unit/core/test_finding_query.py b/tests/unit/core/test_finding_query.py index 6f538ac1..ad3d2e44 100644 --- a/tests/unit/core/test_finding_query.py +++ b/tests/unit/core/test_finding_query.py @@ -81,3 +81,42 @@ def test_conjunction_all_must_match(): def test_unknown_key_raises_valueerror(): with pytest.raises(ValueError, match="unknown filter key"): filter_findings([_f()], {"bogus": "x"}) + + +# --- N-5 (wardline-dc6f44707d): closed-vocab values normalize, never silent-empty --- + + +def test_severity_matches_case_insensitively(): + a = _f(severity=Severity.WARN) + assert filter_findings([a], {"severity": "warn"}) == [a] + assert filter_findings([a], {"severity": "WARN"}) == [a] + assert filter_findings([a], {"severity": "Warn"}) == [a] + + +def test_kind_and_suppression_match_case_insensitively(): + a = _f(kind=Kind.FACT, suppressed=SuppressionState.BASELINED, severity=Severity.NONE) + assert filter_findings([a], {"kind": "FACT"}) == [a] + assert filter_findings([a], {"suppression": "Baselined"}) == [a] + + +def test_severity_out_of_domain_raises_with_allowed_values(): + # A value that can NEVER match (e.g. filigree's 'medium' scale) must error + # loudly with the allowed vocabulary — a silent empty result is the N-5 + # bad-error an agent cannot diagnose. + with pytest.raises(ValueError, match="medium"): + filter_findings([_f()], {"severity": "medium"}) + with pytest.raises(ValueError, match="WARN"): + filter_findings([_f()], {"severity": "medium"}) + + +def test_suppression_and_kind_out_of_domain_raise(): + with pytest.raises(ValueError, match="suppression"): + filter_findings([_f()], {"suppression": "suppressed"}) + with pytest.raises(ValueError, match="kind"): + filter_findings([_f()], {"kind": "bug"}) + + +def test_open_keys_stay_exact_and_silent(): + # rule_id/qualname/sink/tier are open vocabularies (packs can extend tiers) — + # no normalization, no domain error; unmatched simply filters to empty. + assert filter_findings([_f()], {"rule_id": "py-wl-101"}) == [] diff --git a/tests/unit/mcp/test_tooling_explanation.py b/tests/unit/mcp/test_tooling_explanation.py index 20b662b6..053b532c 100644 --- a/tests/unit/mcp/test_tooling_explanation.py +++ b/tests/unit/mcp/test_tooling_explanation.py @@ -1,5 +1,4 @@ -from wardline.core.explain import TaintExplanation -from wardline.mcp.tooling import explanation_to_dict +from wardline.core.explain import TaintExplanation, explanation_to_dict def test_explanation_remediation_degrades_when_source_is_unknown() -> None: From a6340065ad7065ca9bc62f3403235589bda91836 Mon Sep 17 00:00:00 2001 From: John Morrissey Date: Thu, 11 Jun 2026 15:46:33 +1000 Subject: [PATCH 098/186] =?UTF-8?q?feat(mcp):=20A1=20=E2=80=94=20scan=20to?= =?UTF-8?q?ol=20lang=20arg,=20Rust=20frontend=20reachable=20over=20MCP=20(?= =?UTF-8?q?closes=20wardline-2ee1bbda82)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The MCP scan tool now takes lang (enum python|rust, default python) and plumbs it to run_scan's existing frontend selector — the same selector CLI --lang exposes — so the Rust command-injection slice (RS-WL-108/112) is reachable over the primary surface. An unknown value is rejected loudly: jsonschema enum at the protocol layer, run_scan's ConfigError (naming the valid set) behind it. CLI==MCP Rust parity pinned by a real-Click-vs-_scan differential alongside the python one; default-lang behaviour byte-identical (pinned). Docs drop the now-false "frontend is CLI-only" claim; glossary file:line anchors rebased over the server.py shift (two-way lock held). Checked the other run_scan callers (explain_taint/scan_file_findings/ decorator_coverage/assure/attest/baseline/judge): none take lang in core and none expose it on the CLI either, so cross-surface parity holds; the cross-surface gap (RS findings invisible to baseline/judge/promotion generation paths) is filed separately. Co-Authored-By: Claude Fable 5 --- CHANGELOG.md | 6 ++ docs/guides/agents.md | 4 +- docs/guides/rust-preview.md | 17 +++++- .../reference/finding-lifecycle-vocabulary.md | 28 ++++----- src/wardline/mcp/server.py | 14 +++++ tests/docs/test_glossary_vocabulary.py | 20 +++---- tests/unit/core/test_cli_mcp_parity.py | 42 ++++++++++++++ tests/unit/mcp/test_server_scan_rust.py | 57 +++++++++++++++++++ 8 files changed, 161 insertions(+), 27 deletions(-) create mode 100644 tests/unit/mcp/test_server_scan_rust.py diff --git a/CHANGELOG.md b/CHANGELOG.md index 0c2f6d2e..54de5123 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -8,6 +8,12 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] ### Added +- **MCP `scan` tool `lang` argument** (`python` | `rust`, default `python`) — + the same frontend selector as CLI `--lang`, so the Rust command-injection + slice (`RS-WL-108`/`RS-WL-112`) is reachable over the MCP surface; CLI and + MCP Rust scans return identical findings (pinned by a parity test). An + unknown value is rejected loudly naming the valid set + (`wardline-2ee1bbda82`, MCP-primary A1). - **`wardline explain-taint [PATH]`** — the CLI twin of the MCP `explain_taint` tool (same core builder, identical JSON: provenance slice, remediation hint, optional `--chain` walk via a Loomweave store), so a diff --git a/docs/guides/agents.md b/docs/guides/agents.md index cd16bdf8..3db8c777 100644 --- a/docs/guides/agents.md +++ b/docs/guides/agents.md @@ -287,7 +287,9 @@ facts, and Loomweave/Filigree write status when configured. ## Scanning Rust -For a Rust codebase, add `--lang rust` (install the `wardline[rust]` extra first). +For a Rust codebase, add `--lang rust` (install the `wardline[rust]` extra first); +over MCP, pass `lang: "rust"` to the `scan` tool — the two surfaces share the +engine and return identical findings. It sweeps `*.rs` and flags command-injection defects (`RS-WL-108` program injection / `RS-WL-112` shell injection) through the same gate, formats, and emission paths as the Python frontend. Rust finding identity is frozen and diff --git a/docs/guides/rust-preview.md b/docs/guides/rust-preview.md index e5f58175..6f540d60 100644 --- a/docs/guides/rust-preview.md +++ b/docs/guides/rust-preview.md @@ -26,8 +26,10 @@ manifest-less form — always scan from the crate root. Rule coverage is the **command-injection slice** (`RS-WL-108` / `RS-WL-112`) — a Rust scan says nothing about other defect families. `weft.toml` severity overrides do **not** apply to Rust rules yet (they carry hardcoded base - severities), and the frontend is **CLI-only** — the MCP `scan` tool analyzes - Python. + severities). The frontend is reachable from both surfaces — CLI `--lang rust` + and the MCP `scan` tool's `lang: "rust"` argument — but only on `scan`: + `baseline`, `judge`, `explain_taint`, `assure`, and the other verbs still + analyze Python only on both surfaces. ## Running it @@ -49,6 +51,17 @@ else about the scan is unchanged: `--fail-on`, `--format {jsonl,sarif,agent-summ `--output`, `--new-since`, Filigree/Loomweave emission, and the exit-code gate all work exactly as they do for Python. The default (`--lang python`) is untouched. +Over MCP, the `scan` tool takes the same selector as a `lang` argument: + +```json +{"name": "scan", "arguments": {"lang": "rust", "fail_on": "ERROR"}} +``` + +The two surfaces share `run_scan`, so an MCP Rust scan returns the same findings +and gate verdict as the CLI (pinned by a parity test). The CLI's stderr posture +banners have no MCP twin — over MCP, read the `WLN-RUST-COVERAGE` fact in the +scan payload before trusting `0 active` on a tree with no `@trusted` markers. + ## What it finds | Rule | Severity | What it catches | diff --git a/docs/reference/finding-lifecycle-vocabulary.md b/docs/reference/finding-lifecycle-vocabulary.md index a34ae4d4..f3b7efb8 100644 --- a/docs/reference/finding-lifecycle-vocabulary.md +++ b/docs/reference/finding-lifecycle-vocabulary.md @@ -73,7 +73,7 @@ consistently, on every surface: | Enum | `src/wardline/core/finding.py:72` | `SuppressionState.ACTIVE = "active"` | | Summary field | `src/wardline/core/run.py:51`, built at `src/wardline/core/run.py:361` | `ScanSummary.active` | | CLI summary line | `src/wardline/cli/scan.py:416` | `… {s.active} active` | -| MCP scan response | `src/wardline/mcp/server.py:330` | `summary.active` | +| MCP scan response | `src/wardline/mcp/server.py:334` | `summary.active` | | Agent-summary JSON | `src/wardline/core/agent_summary.py:135` | `summary.active_defects` | | `wardline:loop` prompt | `src/wardline/mcp/prompts.py:13` | "Read `summary.active`" | @@ -101,8 +101,8 @@ So `active + baselined + waived + judged + informational == total` (`src/wardline/core/run.py:50` for `total: int`). `unanalyzed` (`src/wardline/core/run.py:69`) is an **overlay** — a subset of `informational` that surfaces a silent under-scan — and is deliberately **not** a partition member. -The MCP `summary` block exposes `informational` (`src/wardline/mcp/server.py:338`) -and `unanalyzed` (`src/wardline/mcp/server.py:342`); the agent-summary block mirrors +The MCP `summary` block exposes `informational` (`src/wardline/mcp/server.py:342`) +and `unanalyzed` (`src/wardline/mcp/server.py:346`); the agent-summary block mirrors both (`src/wardline/core/agent_summary.py:146`, `src/wardline/core/agent_summary.py:147`). ## Emitted-active vs the gate population @@ -143,9 +143,9 @@ judged. The `verdict` is one of: - **`PASSED`** — a threshold ran and nothing tripped. - **`FAILED`** — a threshold ran and tripped. -The MCP `scan` gate block exposes `gate.tripped` (`src/wardline/mcp/server.py:345`), -`gate.verdict` (`src/wardline/mcp/server.py:348`), `would_trip_at`, `reason`, -`evaluated`, and `migration_hint`, opened at `src/wardline/mcp/server.py:344` +The MCP `scan` gate block exposes `gate.tripped` (`src/wardline/mcp/server.py:349`), +`gate.verdict` (`src/wardline/mcp/server.py:352`), `would_trip_at`, `reason`, +`evaluated`, and `migration_hint`, opened at `src/wardline/mcp/server.py:348` (`"gate": {`); the agent-summary mirrors them at `src/wardline/core/agent_summary.py:150` (`tripped`) and `src/wardline/core/agent_summary.py:153` (`verdict`). The CLI prints @@ -178,16 +178,16 @@ How each concept appears on each surface: | Concept | CLI summary text | `ScanSummary` field | MCP `summary` key | Agent-summary key | Filigree store | | --- | --- | --- | --- | --- | --- | -| every finding | `N finding(s)` | `total` (`run.py:50`) | `total` (`server.py:329`) | `total_findings` (`agent_summary.py:134`) | one finding per wire entry | -| live defect | `N active` (`scan.py:416`) | `active` (`run.py:51,361`) | `active` (`server.py:330`) | `active_defects` (`agent_summary.py:135`) | no `suppression_state` key (`finding.py:285`) | +| every finding | `N finding(s)` | `total` (`run.py:50`) | `total` (`server.py:333`) | `total_findings` (`agent_summary.py:134`) | one finding per wire entry | +| live defect | `N active` (`scan.py:416`) | `active` (`run.py:51,361`) | `active` (`server.py:334`) | `active_defects` (`agent_summary.py:135`) | no `suppression_state` key (`finding.py:285`) | | suppressed (sum) | `N suppressed` (`scan.py:415`) | `baselined+waived+judged` | the three keys | `suppressed_findings` (`agent_summary.py:136`) | `metadata.wardline.suppression_state` (`finding.py:285`) | -| baselined | `N baseline` | `baselined` (`run.py:53`) | `baselined` (`server.py:331`) | `baselined` (`agent_summary.py:138`) | `suppression_state: "baselined"` | -| waived | `N waiver` | `waived` (`run.py:54`) | `waived` (`server.py:332`) | `waived` (`agent_summary.py:139`) | `suppression_state: "waived"` | -| judged | `N judged` | `judged` (`run.py:55`) | `judged` (`server.py:333`) | `judged` (`agent_summary.py:140`) | `suppression_state: "judged"` | -| informational (summary) | (the remainder of `total`) | `informational` (`run.py:61`) | `informational` (`server.py:338`) | `informational` (`agent_summary.py:146`) | facts/metrics | +| baselined | `N baseline` | `baselined` (`run.py:53`) | `baselined` (`server.py:335`) | `baselined` (`agent_summary.py:138`) | `suppression_state: "baselined"` | +| waived | `N waiver` | `waived` (`run.py:54`) | `waived` (`server.py:336`) | `waived` (`agent_summary.py:139`) | `suppression_state: "waived"` | +| judged | `N judged` | `judged` (`run.py:55`) | `judged` (`server.py:337`) | `judged` (`agent_summary.py:140`) | `suppression_state: "judged"` | +| informational (summary) | (the remainder of `total`) | `informational` (`run.py:61`) | `informational` (`server.py:342`) | `informational` (`agent_summary.py:146`) | facts/metrics | | informational (display) | n/a | n/a | n/a | `informational` display array (`agent_summary.py:171`) — non-defect, non-engine-fact findings (metrics, classifications, suggestions, non-engine facts); excludes `engine_facts` which has its own display slot | facts/metrics | -| under-scan | `N file(s) could not be analyzed` | `unanalyzed` (`run.py:69`) | `unanalyzed` (`server.py:342`) | `unanalyzed` (`agent_summary.py:147`) | `WLN-ENGINE-*` facts | -| gate verdict | exit code + `--fail-on` | (`gate_findings`, `run.py:88`; `GateDecision`, `run.py:98`, `verdict` `run.py:107`) | `gate.tripped` (`server.py:345`), `gate.verdict` (`server.py:348`) | `gate.tripped` (`agent_summary.py:150`), `gate.verdict` (`agent_summary.py:153`) | not emitted to Filigree | +| under-scan | `N file(s) could not be analyzed` | `unanalyzed` (`run.py:69`) | `unanalyzed` (`server.py:346`) | `unanalyzed` (`agent_summary.py:147`) | `WLN-ENGINE-*` facts | +| gate verdict | exit code + `--fail-on` | (`gate_findings`, `run.py:88`; `GateDecision`, `run.py:98`, `verdict` `run.py:107`) | `gate.tripped` (`server.py:349`), `gate.verdict` (`server.py:352`) | `gate.tripped` (`agent_summary.py:150`), `gate.verdict` (`agent_summary.py:153`) | not emitted to Filigree | The unsuppressed gate population is built from `Baseline(frozenset())` (`src/wardline/core/run.py:327`). diff --git a/src/wardline/mcp/server.py b/src/wardline/mcp/server.py index caeb6c89..36a3d5c5 100644 --- a/src/wardline/mcp/server.py +++ b/src/wardline/mcp/server.py @@ -216,6 +216,9 @@ def _scan( trusted_packs = _trusted_packs_arg(args) cache_dir = _cache_dir_arg(args, root) trust_suppressions = bool(args.get("trust_suppressions") or False) + # A1 (wardline-2ee1bbda82): the same frontend selector the CLI's --lang exposes. + # A bad value is run_scan's ConfigError (names the valid set) -> isError result. + lang = args.get("lang") or "python" result = run_scan( path, config_path=_cfg(args, root), @@ -226,6 +229,7 @@ def _scan( trusted_packs=trusted_packs, strict_defaults=strict_defaults, trust_suppressions=trust_suppressions, + lang=lang, ) # Fail-soft Loomweave write: only when a client was injected (server has a URL). # An outage/403 yields a not-reachable WriteResult; never raises here. @@ -819,6 +823,16 @@ def _register_tools(self) -> None: "path": {"type": "string", "description": "subdir relative to project root"}, "fail_on": {"type": "string", "enum": _SEVERITY_ENUM}, "config": {"type": "string"}, + "lang": { + "type": "string", + "enum": ["python", "rust"], + "description": "Language frontend (default python). 'rust' sweeps .rs files " + "for the command-injection slice (RS-WL-108 program injection / RS-WL-112 " + "shell injection; frozen identity, baseline-eligible). Preview posture: " + "weft.toml severity overrides do not yet apply to Rust findings, and a tree " + "with no /// @trusted markers is vacuously green — read the WLN-RUST-COVERAGE " + "fact before trusting '0 active'. Requires the wardline[rust] extra.", + }, "where": { "type": "object", "description": "Filter the returned findings (conjunctive). Keys: " diff --git a/tests/docs/test_glossary_vocabulary.py b/tests/docs/test_glossary_vocabulary.py index 881cd5c6..ac092cae 100644 --- a/tests/docs/test_glossary_vocabulary.py +++ b/tests/docs/test_glossary_vocabulary.py @@ -45,16 +45,16 @@ ("src/wardline/cli/scan.py", 416, "{s.active} active"), ("src/wardline/cli/scan.py", 456, "gate: FAILED"), # src/wardline/mcp/server.py — MCP scan summary + gate block - ("src/wardline/mcp/server.py", 329, '"total": result.summary.total'), - ("src/wardline/mcp/server.py", 330, '"active": result.summary.active'), - ("src/wardline/mcp/server.py", 331, '"baselined": result.summary.baselined'), - ("src/wardline/mcp/server.py", 332, '"waived": result.summary.waived'), - ("src/wardline/mcp/server.py", 333, '"judged": result.summary.judged'), - ("src/wardline/mcp/server.py", 338, '"informational": result.summary.informational'), - ("src/wardline/mcp/server.py", 342, '"unanalyzed": result.summary.unanalyzed'), - ("src/wardline/mcp/server.py", 344, '"gate": {'), - ("src/wardline/mcp/server.py", 345, '"tripped": decision.tripped'), - ("src/wardline/mcp/server.py", 348, '"verdict": decision.verdict'), + ("src/wardline/mcp/server.py", 333, '"total": result.summary.total'), + ("src/wardline/mcp/server.py", 334, '"active": result.summary.active'), + ("src/wardline/mcp/server.py", 335, '"baselined": result.summary.baselined'), + ("src/wardline/mcp/server.py", 336, '"waived": result.summary.waived'), + ("src/wardline/mcp/server.py", 337, '"judged": result.summary.judged'), + ("src/wardline/mcp/server.py", 342, '"informational": result.summary.informational'), + ("src/wardline/mcp/server.py", 346, '"unanalyzed": result.summary.unanalyzed'), + ("src/wardline/mcp/server.py", 348, '"gate": {'), + ("src/wardline/mcp/server.py", 349, '"tripped": decision.tripped'), + ("src/wardline/mcp/server.py", 352, '"verdict": decision.verdict'), # src/wardline/core/agent_summary.py — agent-summary JSON keys ("src/wardline/core/agent_summary.py", 134, '"total_findings"'), ("src/wardline/core/agent_summary.py", 135, '"active_defects"'), diff --git a/tests/unit/core/test_cli_mcp_parity.py b/tests/unit/core/test_cli_mcp_parity.py index 08f32fe0..45e2cf95 100644 --- a/tests/unit/core/test_cli_mcp_parity.py +++ b/tests/unit/core/test_cli_mcp_parity.py @@ -58,6 +58,48 @@ def test_cli_and_mcp_scan_agree_on_findings_and_gate() -> None: assert any(e["kind"] == "defect" for e in cli_ag["active_defects"]) +def test_cli_and_mcp_scan_agree_on_rust_findings(tmp_path: Path) -> None: + """A1 (wardline-2ee1bbda82): the Rust frontend must yield the SAME findings over + both surfaces. The CLI side is the REAL Click command (`scan --lang rust`) writing + jsonl; the MCP side is `_scan({"lang": "rust", ...})`. Every agent_summary bucket + entry carries a fingerprint, so the union must equal the CLI's emitted finding set, + and the gate verdicts must agree.""" + import json + + import pytest + + pytest.importorskip("tree_sitter", reason="wardline[rust] extra not installed") + from click.testing import CliRunner + + from wardline.cli.scan import scan as scan_cmd + + trusted = "/// @trusted(level=ASSURED)\n" + (tmp_path / "hot.rs").write_text( + trusted + 'fn run() {\n let t = std::env::var("X").unwrap();\n Command::new(t).output();\n}\n', + encoding="utf-8", + ) + (tmp_path / "clean.rs").write_text(trusted + 'fn ok() {\n Command::new("ls").output();\n}\n', encoding="utf-8") + + out = tmp_path / "findings.jsonl" + cli = CliRunner().invoke(scan_cmd, [str(tmp_path), "--lang", "rust", "--fail-on", "ERROR", "--output", str(out)]) + assert cli.exit_code == 1 # the RS-WL-108 injection trips the gate + cli_fps = sorted(json.loads(line)["fingerprint"] for line in out.read_text().splitlines() if line.strip()) + + mcp = _scan({"lang": "rust", "fail_on": "ERROR", "full": True}, root=tmp_path) + ag = mcp["agent_summary"] + mcp_fps = sorted( + e["fingerprint"] + for key in ("active_defects", "suppressed_findings", "engine_facts", "informational") + for e in ag[key] + ) + assert mcp_fps == cli_fps + assert mcp["gate"]["tripped"] is True + assert mcp["gate"]["verdict"] == "FAILED" + # Engine-level parity too: the shared run_scan path under lang="rust" is what both drove. + engine = run_scan(tmp_path, lang="rust") + assert sorted(f.fingerprint for f in engine.findings) == cli_fps + + def test_cli_and_mcp_emit_identical_filigree_body() -> None: """The Filigree emission set must be identical across surfaces. The CLI passes `result.findings` and `result.scanned_paths` to FiligreeEmitter.emit. The MCP diff --git a/tests/unit/mcp/test_server_scan_rust.py b/tests/unit/mcp/test_server_scan_rust.py new file mode 100644 index 00000000..713802c1 --- /dev/null +++ b/tests/unit/mcp/test_server_scan_rust.py @@ -0,0 +1,57 @@ +"""A1 (wardline-2ee1bbda82): the MCP ``scan`` tool's ``lang`` arg. + +CLI ``wardline scan --lang rust`` selects the Rust frontend; the MCP scan tool +must expose the same selector or the Rust line is unreachable over the primary +surface. The schema declares the enum, the handler plumbs it to ``run_scan``, +and a bad value surfaces as the agent-actionable ``ConfigError`` (isError +result), never a silent python-default scan. +""" + +from __future__ import annotations + +import pytest + +from wardline.core.errors import ConfigError +from wardline.mcp.server import WardlineMCPServer, _scan + +_TRUSTED = "/// @trusted(level=ASSURED)\n" +_INJECTION = _TRUSTED + 'fn run() {\n let t = std::env::var("X").unwrap();\n Command::new(t).output();\n}\n' + + +def test_scan_lang_rust_finds_injection_and_trips_gate(tmp_path) -> None: + pytest.importorskip("tree_sitter", reason="wardline[rust] extra not installed") + (tmp_path / "m.rs").write_text(_INJECTION, encoding="utf-8") + response = _scan({"lang": "rust", "fail_on": "ERROR", "full": True}, root=tmp_path) + rule_ids = {e["rule_id"] for e in response["agent_summary"]["active_defects"]} + assert "RS-WL-108" in rule_ids + assert response["gate"]["tripped"] is True + assert response["gate"]["verdict"] == "FAILED" + + +def test_scan_default_lang_is_python_and_ignores_rs(tmp_path) -> None: + # Absent lang must stay byte-identical to the released behaviour: .rs files + # are not swept, the gate stays green. + (tmp_path / "m.rs").write_text(_INJECTION, encoding="utf-8") + response = _scan({"fail_on": "ERROR"}, root=tmp_path) + assert response["files_scanned"] == 0 + assert response["gate"]["tripped"] is False + + +def test_scan_unknown_lang_is_agent_actionable_config_error(tmp_path) -> None: + # ConfigError is a WardlineError -> the MCP loop maps it to an isError result + # the agent can read; the message names the valid set. + with pytest.raises(ConfigError, match="unknown language 'go'.*'python'.*'rust'"): + _scan({"lang": "go"}, root=tmp_path) + + +def test_scan_tool_schema_declares_lang_enum(tmp_path) -> None: + server = WardlineMCPServer(root=tmp_path) + tools = server.rpc.dispatch({"jsonrpc": "2.0", "id": 1, "method": "tools/list", "params": {}}) + scan_tool = next(t for t in tools["result"]["tools"] if t["name"] == "scan") + lang_schema = scan_tool["inputSchema"]["properties"]["lang"] + assert lang_schema["enum"] == ["python", "rust"] + assert lang_schema["type"] == "string" + # The description must carry the preview posture the CLI banner carries + # (slice coverage; no severity-override parity) — MCP agents read tool docs, + # not stderr. + assert "RS-WL-108" in lang_schema["description"] From 81e8d04ea8515e382a86d6ad1984c246249f1c2f Mon Sep 17 00:00:00 2001 From: John Morrissey Date: Thu, 11 Jun 2026 15:59:17 +1000 Subject: [PATCH 099/186] =?UTF-8?q?feat(mcp):=20A2=20=E2=80=94=20doctor=20?= =?UTF-8?q?MCP=20twin=20+=20server-freshness=20self-identification=20(clos?= =?UTF-8?q?es=20wardline-4c5165e896)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit New `doctor` MCP tool serving the CLI `doctor --fix` machine-readable envelope (same machine_readable_doctor builder — CLI==MCP by construction) plus the running server's self-identification: package version, pid, project root, start time, and a source-freshness verdict. fresh=false means a long-lived server's loaded code predates the on-disk wardline package (mtime vs started_at) — the 2026-06-06 stale-server incident class — surfaced as a uniform server.freshness check that flips ok and lands a restart instruction in next_actions. Read-only by default; repair:true is the explicit WRITE opt-in (mirrors CLI --fix), NETWORK added only when a filigree probe URL resolves (probe stays loopback-only). Live-verified over JSON-RPC including a mid-session staleness flip. Glossary anchors rebased (+1, `import time`). Co-Authored-By: Claude Fable 5 --- CHANGELOG.md | 8 + docs/guides/agents.md | 8 + .../reference/finding-lifecycle-vocabulary.md | 28 ++-- src/wardline/mcp/freshness.py | 84 ++++++++++ src/wardline/mcp/server.py | 72 +++++++++ tests/conformance/test_mcp_handshake.py | 1 + tests/docs/test_glossary_vocabulary.py | 20 +-- tests/unit/mcp/test_server_doctor.py | 146 ++++++++++++++++++ tests/unit/mcp/test_server_structure.py | 1 + 9 files changed, 344 insertions(+), 24 deletions(-) create mode 100644 src/wardline/mcp/freshness.py create mode 100644 tests/unit/mcp/test_server_doctor.py diff --git a/CHANGELOG.md b/CHANGELOG.md index 54de5123..4f1600d8 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -8,6 +8,14 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] ### Added +- **MCP `doctor` tool** — the CLI `doctor --fix` health envelope over MCP + (install artifacts, MCP registration, config parseability, sibling URLs, + Filigree emit-auth probe; read-only by default, `repair: true` is the + explicit write-gated opt-in) **plus server self-identification**: package + version, pid, project root, start time, and a source-freshness verdict + (`server.fresh` / `server.freshness` check) that detects a long-lived MCP + server serving code older than the on-disk tree — the 2026-06-06 + stale-server incident class (`wardline-4c5165e896`, MCP-primary A2). - **MCP `scan` tool `lang` argument** (`python` | `rust`, default `python`) — the same frontend selector as CLI `--lang`, so the Rust command-injection slice (`RS-WL-108`/`RS-WL-112`) is reachable over the MCP surface; CLI and diff --git a/docs/guides/agents.md b/docs/guides/agents.md index 3db8c777..706a213c 100644 --- a/docs/guides/agents.md +++ b/docs/guides/agents.md @@ -77,6 +77,14 @@ dashboard, or changing sibling tool config. It refreshes the instruction blocks, skills, and MCP entries, and re-detects siblings using the same discovery rules as `wardline install` — it never writes `weft.toml` or a sibling binding. +Over MCP, the `doctor` tool returns the same machine-readable envelope +(read-only by default; pass `repair: true` for the write-gated repair) **plus +the running server's self-identification**: package version, pid, start time, +and a source-freshness verdict. If `server.fresh` is `false`, the long-lived +MCP server process predates the on-disk wardline code — every result it serves +is stale; restart the server. Call it whenever federation writes fail or after +upgrading/editing wardline itself. + ## Gate the agent's work with `wardline scan` Wardline marks trust boundaries with marker decorators from `weft_markers`: diff --git a/docs/reference/finding-lifecycle-vocabulary.md b/docs/reference/finding-lifecycle-vocabulary.md index f3b7efb8..d0fe67ee 100644 --- a/docs/reference/finding-lifecycle-vocabulary.md +++ b/docs/reference/finding-lifecycle-vocabulary.md @@ -73,7 +73,7 @@ consistently, on every surface: | Enum | `src/wardline/core/finding.py:72` | `SuppressionState.ACTIVE = "active"` | | Summary field | `src/wardline/core/run.py:51`, built at `src/wardline/core/run.py:361` | `ScanSummary.active` | | CLI summary line | `src/wardline/cli/scan.py:416` | `… {s.active} active` | -| MCP scan response | `src/wardline/mcp/server.py:334` | `summary.active` | +| MCP scan response | `src/wardline/mcp/server.py:335` | `summary.active` | | Agent-summary JSON | `src/wardline/core/agent_summary.py:135` | `summary.active_defects` | | `wardline:loop` prompt | `src/wardline/mcp/prompts.py:13` | "Read `summary.active`" | @@ -101,8 +101,8 @@ So `active + baselined + waived + judged + informational == total` (`src/wardline/core/run.py:50` for `total: int`). `unanalyzed` (`src/wardline/core/run.py:69`) is an **overlay** — a subset of `informational` that surfaces a silent under-scan — and is deliberately **not** a partition member. -The MCP `summary` block exposes `informational` (`src/wardline/mcp/server.py:342`) -and `unanalyzed` (`src/wardline/mcp/server.py:346`); the agent-summary block mirrors +The MCP `summary` block exposes `informational` (`src/wardline/mcp/server.py:343`) +and `unanalyzed` (`src/wardline/mcp/server.py:347`); the agent-summary block mirrors both (`src/wardline/core/agent_summary.py:146`, `src/wardline/core/agent_summary.py:147`). ## Emitted-active vs the gate population @@ -143,9 +143,9 @@ judged. The `verdict` is one of: - **`PASSED`** — a threshold ran and nothing tripped. - **`FAILED`** — a threshold ran and tripped. -The MCP `scan` gate block exposes `gate.tripped` (`src/wardline/mcp/server.py:349`), -`gate.verdict` (`src/wardline/mcp/server.py:352`), `would_trip_at`, `reason`, -`evaluated`, and `migration_hint`, opened at `src/wardline/mcp/server.py:348` +The MCP `scan` gate block exposes `gate.tripped` (`src/wardline/mcp/server.py:350`), +`gate.verdict` (`src/wardline/mcp/server.py:353`), `would_trip_at`, `reason`, +`evaluated`, and `migration_hint`, opened at `src/wardline/mcp/server.py:349` (`"gate": {`); the agent-summary mirrors them at `src/wardline/core/agent_summary.py:150` (`tripped`) and `src/wardline/core/agent_summary.py:153` (`verdict`). The CLI prints @@ -178,16 +178,16 @@ How each concept appears on each surface: | Concept | CLI summary text | `ScanSummary` field | MCP `summary` key | Agent-summary key | Filigree store | | --- | --- | --- | --- | --- | --- | -| every finding | `N finding(s)` | `total` (`run.py:50`) | `total` (`server.py:333`) | `total_findings` (`agent_summary.py:134`) | one finding per wire entry | -| live defect | `N active` (`scan.py:416`) | `active` (`run.py:51,361`) | `active` (`server.py:334`) | `active_defects` (`agent_summary.py:135`) | no `suppression_state` key (`finding.py:285`) | +| every finding | `N finding(s)` | `total` (`run.py:50`) | `total` (`server.py:334`) | `total_findings` (`agent_summary.py:134`) | one finding per wire entry | +| live defect | `N active` (`scan.py:416`) | `active` (`run.py:51,361`) | `active` (`server.py:335`) | `active_defects` (`agent_summary.py:135`) | no `suppression_state` key (`finding.py:285`) | | suppressed (sum) | `N suppressed` (`scan.py:415`) | `baselined+waived+judged` | the three keys | `suppressed_findings` (`agent_summary.py:136`) | `metadata.wardline.suppression_state` (`finding.py:285`) | -| baselined | `N baseline` | `baselined` (`run.py:53`) | `baselined` (`server.py:335`) | `baselined` (`agent_summary.py:138`) | `suppression_state: "baselined"` | -| waived | `N waiver` | `waived` (`run.py:54`) | `waived` (`server.py:336`) | `waived` (`agent_summary.py:139`) | `suppression_state: "waived"` | -| judged | `N judged` | `judged` (`run.py:55`) | `judged` (`server.py:337`) | `judged` (`agent_summary.py:140`) | `suppression_state: "judged"` | -| informational (summary) | (the remainder of `total`) | `informational` (`run.py:61`) | `informational` (`server.py:342`) | `informational` (`agent_summary.py:146`) | facts/metrics | +| baselined | `N baseline` | `baselined` (`run.py:53`) | `baselined` (`server.py:336`) | `baselined` (`agent_summary.py:138`) | `suppression_state: "baselined"` | +| waived | `N waiver` | `waived` (`run.py:54`) | `waived` (`server.py:337`) | `waived` (`agent_summary.py:139`) | `suppression_state: "waived"` | +| judged | `N judged` | `judged` (`run.py:55`) | `judged` (`server.py:338`) | `judged` (`agent_summary.py:140`) | `suppression_state: "judged"` | +| informational (summary) | (the remainder of `total`) | `informational` (`run.py:61`) | `informational` (`server.py:343`) | `informational` (`agent_summary.py:146`) | facts/metrics | | informational (display) | n/a | n/a | n/a | `informational` display array (`agent_summary.py:171`) — non-defect, non-engine-fact findings (metrics, classifications, suggestions, non-engine facts); excludes `engine_facts` which has its own display slot | facts/metrics | -| under-scan | `N file(s) could not be analyzed` | `unanalyzed` (`run.py:69`) | `unanalyzed` (`server.py:346`) | `unanalyzed` (`agent_summary.py:147`) | `WLN-ENGINE-*` facts | -| gate verdict | exit code + `--fail-on` | (`gate_findings`, `run.py:88`; `GateDecision`, `run.py:98`, `verdict` `run.py:107`) | `gate.tripped` (`server.py:349`), `gate.verdict` (`server.py:352`) | `gate.tripped` (`agent_summary.py:150`), `gate.verdict` (`agent_summary.py:153`) | not emitted to Filigree | +| under-scan | `N file(s) could not be analyzed` | `unanalyzed` (`run.py:69`) | `unanalyzed` (`server.py:347`) | `unanalyzed` (`agent_summary.py:147`) | `WLN-ENGINE-*` facts | +| gate verdict | exit code + `--fail-on` | (`gate_findings`, `run.py:88`; `GateDecision`, `run.py:98`, `verdict` `run.py:107`) | `gate.tripped` (`server.py:350`), `gate.verdict` (`server.py:353`) | `gate.tripped` (`agent_summary.py:150`), `gate.verdict` (`agent_summary.py:153`) | not emitted to Filigree | The unsuppressed gate population is built from `Baseline(frozenset())` (`src/wardline/core/run.py:327`). diff --git a/src/wardline/mcp/freshness.py b/src/wardline/mcp/freshness.py new file mode 100644 index 00000000..6b37da05 --- /dev/null +++ b/src/wardline/mcp/freshness.py @@ -0,0 +1,84 @@ +"""Server self-identification + source-freshness verdict for the MCP `doctor` tool. + +The 2026-06-06 stale-server incident: long-lived `wardline mcp` processes (editable +install) kept serving code that predated the tree being edited, and nothing on the +wire said so — `initialize` reports only name+version, which does not change on an +editable re-install. The freshness verdict here is the mtime test that DOES catch it: +if any file under the imported ``wardline`` package is newer than this process's +start time, the running server is serving old code and must be restarted. +""" + +from __future__ import annotations + +import os +from datetime import UTC, datetime +from pathlib import Path +from typing import Any + +from wardline._version import __version__ + +__all__ = ["attach_server_identity", "server_identity"] + + +def _iso(ts: float) -> str: + return datetime.fromtimestamp(ts, tz=UTC).isoformat() + + +def _latest_source_mtime() -> tuple[float | None, str | None]: + """(mtime, relative path) of the newest ``*.py`` under the imported wardline + package, or (None, None) when nothing is statable. Walking the package the + PROCESS imported (not the project root) is the point: that is the code this + server is actually serving, wherever it is installed.""" + import wardline + + pkg_dir = Path(wardline.__file__).resolve().parent + latest: float | None = None + latest_path: str | None = None + for p in pkg_dir.rglob("*.py"): + try: + mtime = p.stat().st_mtime + except OSError: + continue + if latest is None or mtime > latest: + latest = mtime + latest_path = str(p.relative_to(pkg_dir)) + return latest, latest_path + + +def server_identity(*, root: Path, started_at: float) -> dict[str, Any]: + """The running server's self-identification block. ``fresh`` is False when the + on-disk package source changed after this process started.""" + latest, latest_path = _latest_source_mtime() + fresh = latest is None or latest <= started_at + identity: dict[str, Any] = { + "package_version": __version__, + "pid": os.getpid(), + "project_root": str(root), + "started_at": _iso(started_at), + "source_latest_mtime": _iso(latest) if latest is not None else None, + "source_latest_path": latest_path, + "fresh": fresh, + } + return identity + + +def attach_server_identity(payload: dict[str, Any], *, root: Path, started_at: float) -> dict[str, Any]: + """Merge the server block + a ``server.freshness`` check into a + ``machine_readable_doctor`` envelope (same check shape, so the agent reads one + uniform list). A stale server flips ``ok`` and lands in ``next_actions`` — it is + a health failure: every other verdict in the payload came from old code.""" + identity = server_identity(root=root, started_at=started_at) + payload["server"] = identity + if identity["fresh"]: + payload["checks"].append({"id": "server.freshness", "status": "ok", "fixed": False}) + return payload + message = ( + f"wardline source changed after this MCP server started " + f"({identity['source_latest_path']} at {identity['source_latest_mtime']}, " + f"server started {identity['started_at']}) — this server is serving OLD code; " + f"restart the wardline MCP server" + ) + payload["checks"].append({"id": "server.freshness", "status": "error", "fixed": False, "message": message}) + payload["ok"] = False + payload["next_actions"].append(f"server.freshness: {message}") + return payload diff --git a/src/wardline/mcp/server.py b/src/wardline/mcp/server.py index 36a3d5c5..bee8dc5d 100644 --- a/src/wardline/mcp/server.py +++ b/src/wardline/mcp/server.py @@ -8,6 +8,7 @@ from __future__ import annotations import json +import time from dataclasses import replace from datetime import date from pathlib import Path @@ -677,6 +678,29 @@ def _waiver_add(args: dict[str, Any], root: Path) -> dict[str, Any]: } +def _doctor( + args: dict[str, Any], + root: Path, + *, + started_at: float, + filigree_url: str | None = None, +) -> dict[str, Any]: + """The CLI `doctor --fix` envelope over MCP (A2, wardline-2ee1bbda82's sibling): + install/federation health checks via the SAME machine_readable_doctor builder, + plus the running server's self-identification + source-freshness verdict so an + agent can detect a stale long-lived server without shelling out. Read-only by + default; `repair: true` is the explicit WRITE opt-in (mirrors CLI --fix).""" + from wardline.install.doctor import machine_readable_doctor + from wardline.mcp.freshness import attach_server_identity + + repair = _bool_arg(args, "repair", False) + flag = args.get("filigree_url") + if flag is not None and not isinstance(flag, str): + raise ToolError("filigree_url must be a string") + payload = machine_readable_doctor(root, fix=repair, filigree_url=flag or filigree_url) + return attach_server_identity(payload, root=root, started_at=started_at) + + def _fix(args: dict[str, Any], root: Path) -> dict[str, Any]: """Scan the path and apply mechanical autofixes to findings in-place.""" path = _resolve_under_root(root, args["path"]) if args.get("path") else root @@ -733,6 +757,9 @@ def __init__( self.root = Path(root) self.loomweave_url = loomweave_url self.filigree_url = filigree_url + # Recorded once at construction: the doctor tool's freshness verdict compares + # on-disk source mtimes against this to expose a stale long-lived server. + self.started_at = time.time() self._tool_policy = ToolPolicy(allow_write=allow_write, allow_network=allow_network) self.rpc = JsonRpcServer(server_name="wardline", server_version=__version__) self._tools: dict[str, Tool] = {} @@ -1261,6 +1288,41 @@ def _register_tools(self) -> None: handler=_fix, ) ) + self.add_tool( + Tool( + name="doctor", + description="Health-check the wardline install and federation wiring " + "(same checks as CLI `doctor --fix`: instruction blocks, skills, MCP " + "registration, config parseability, sibling URLs, Filigree emit auth) " + "PLUS this server's self-identification: package version, pid, start " + "time, and a source-FRESHNESS verdict. If `server.fresh` is false this " + "long-lived server predates the on-disk wardline code — its results are " + "stale; restart the MCP server. Read-only by default; `repair: true` " + "(write-gated) repairs install artifacts and re-pins a rejected " + "federation token.", + input_schema={ + "type": "object", + "properties": { + "repair": { + "type": "boolean", + "description": "Default false (pure probe, writes nothing). true repairs " + "install artifacts (CLAUDE.md/AGENTS.md blocks, .claude/.agents skills, " + ".mcp.json + Codex registration, .weft state dir) and, when Filigree " + "rejected the emit token, re-pins the accepted local mint in .env.", + }, + "filigree_url": { + "type": "string", + "description": "Filigree URL to probe for emit auth (default: the server's " + "configured URL, then WARDLINE_FILIGREE_URL, then the .mcp.json arg). " + "Only loopback origins are ever probed with a token.", + }, + }, + }, + handler=lambda args, root: _doctor( + args, root, started_at=self.started_at, filigree_url=self.filigree_url + ), + ) + ) def add_tool(self, tool: Tool) -> None: schema = dict(tool.input_schema) @@ -1328,6 +1390,16 @@ def _effective_tool_capabilities(self, tool: Tool, arguments: dict[str, Any]) -> capabilities.add(ToolCapability.NETWORK) if tool.name == "judge" and bool(arguments.get("write", False)): capabilities.add(ToolCapability.WRITE) + if tool.name == "doctor": + if bool(arguments.get("repair", False)): + capabilities.add(ToolCapability.WRITE) + from wardline.install.doctor import _resolve_probe_url + + flag = arguments.get("filigree_url") + probe_url = flag if isinstance(flag, str) and flag else None + if _resolve_probe_url(self.root, probe_url or self.filigree_url) is not None: + # The filigree-auth probe will touch the (loopback-only) network. + capabilities.add(ToolCapability.NETWORK) return frozenset(capabilities) def _tools_call(self, params: dict[str, Any]) -> dict[str, Any]: diff --git a/tests/conformance/test_mcp_handshake.py b/tests/conformance/test_mcp_handshake.py index 81c027ae..a44e5cd7 100644 --- a/tests/conformance/test_mcp_handshake.py +++ b/tests/conformance/test_mcp_handshake.py @@ -69,6 +69,7 @@ def test_full_client_handshake_and_every_surface() -> None: "baseline", "fix", "waiver_add", + "doctor", } # resources/list: the four stable URIs resource_uris = {r["uri"] for r in by_id[3]["result"]["resources"]} diff --git a/tests/docs/test_glossary_vocabulary.py b/tests/docs/test_glossary_vocabulary.py index ac092cae..7390d73d 100644 --- a/tests/docs/test_glossary_vocabulary.py +++ b/tests/docs/test_glossary_vocabulary.py @@ -45,16 +45,16 @@ ("src/wardline/cli/scan.py", 416, "{s.active} active"), ("src/wardline/cli/scan.py", 456, "gate: FAILED"), # src/wardline/mcp/server.py — MCP scan summary + gate block - ("src/wardline/mcp/server.py", 333, '"total": result.summary.total'), - ("src/wardline/mcp/server.py", 334, '"active": result.summary.active'), - ("src/wardline/mcp/server.py", 335, '"baselined": result.summary.baselined'), - ("src/wardline/mcp/server.py", 336, '"waived": result.summary.waived'), - ("src/wardline/mcp/server.py", 337, '"judged": result.summary.judged'), - ("src/wardline/mcp/server.py", 342, '"informational": result.summary.informational'), - ("src/wardline/mcp/server.py", 346, '"unanalyzed": result.summary.unanalyzed'), - ("src/wardline/mcp/server.py", 348, '"gate": {'), - ("src/wardline/mcp/server.py", 349, '"tripped": decision.tripped'), - ("src/wardline/mcp/server.py", 352, '"verdict": decision.verdict'), + ("src/wardline/mcp/server.py", 334, '"total": result.summary.total'), + ("src/wardline/mcp/server.py", 335, '"active": result.summary.active'), + ("src/wardline/mcp/server.py", 336, '"baselined": result.summary.baselined'), + ("src/wardline/mcp/server.py", 337, '"waived": result.summary.waived'), + ("src/wardline/mcp/server.py", 338, '"judged": result.summary.judged'), + ("src/wardline/mcp/server.py", 343, '"informational": result.summary.informational'), + ("src/wardline/mcp/server.py", 347, '"unanalyzed": result.summary.unanalyzed'), + ("src/wardline/mcp/server.py", 349, '"gate": {'), + ("src/wardline/mcp/server.py", 350, '"tripped": decision.tripped'), + ("src/wardline/mcp/server.py", 353, '"verdict": decision.verdict'), # src/wardline/core/agent_summary.py — agent-summary JSON keys ("src/wardline/core/agent_summary.py", 134, '"total_findings"'), ("src/wardline/core/agent_summary.py", 135, '"active_defects"'), diff --git a/tests/unit/mcp/test_server_doctor.py b/tests/unit/mcp/test_server_doctor.py new file mode 100644 index 00000000..9fe5f9a4 --- /dev/null +++ b/tests/unit/mcp/test_server_doctor.py @@ -0,0 +1,146 @@ +"""A2 (wardline-4c5165e896): the `doctor` MCP twin + server-freshness self-identification. + +The CLI `doctor --fix` envelope (machine_readable_doctor) is served read-only over MCP, +plus a `server` self-identification block (package version, pid, start time, source +freshness) so an agent can detect the 2026-06-06 stale-server class — a long-lived +`wardline mcp` process serving code older than the tree it scans — without shelling out. +Repair is behind an explicit `repair: true` (WRITE-gated); the default call writes nothing. +""" + +from __future__ import annotations + +import os +import time +from pathlib import Path +from typing import Any + +from wardline._version import __version__ +from wardline.install.doctor import machine_readable_doctor +from wardline.mcp.server import WardlineMCPServer, _doctor + + +def _isolate(tmp_path: Path, monkeypatch) -> Path: + home = tmp_path / "home" + monkeypatch.delenv("WARDLINE_LOOMWEAVE_URL", raising=False) + monkeypatch.delenv("WARDLINE_FILIGREE_URL", raising=False) + monkeypatch.setattr("wardline.install.mcp_json.Path.home", lambda: home) + monkeypatch.setattr("wardline.install.mcp_json._find_wardline_command", lambda: "/bin/wardline") + monkeypatch.setattr("wardline.install.detect.shutil.which", lambda _: None) + return home + + +def test_doctor_matches_cli_machine_readable_envelope(tmp_path: Path, monkeypatch) -> None: + """CLI==MCP by construction: the MCP doctor's check set IS machine_readable_doctor's + (fix=False), with exactly one extra MCP-only check appended (server.freshness).""" + _isolate(tmp_path, monkeypatch) + cli = machine_readable_doctor(tmp_path, fix=False) + mcp = _doctor({}, tmp_path, started_at=time.time()) + assert mcp["checks"][:-1] == cli["checks"] + assert mcp["checks"][-1]["id"] == "server.freshness" + # A fresh server adds no failure: ok parity holds. + assert mcp["ok"] == cli["ok"] + + +def test_doctor_reports_server_identity(tmp_path: Path, monkeypatch) -> None: + _isolate(tmp_path, monkeypatch) + now = time.time() + payload = _doctor({}, tmp_path, started_at=now) + server = payload["server"] + assert server["package_version"] == __version__ + assert server["pid"] == os.getpid() + assert server["project_root"] == str(tmp_path) + assert server["started_at"].startswith("20") # ISO timestamp + assert server["fresh"] is True + assert payload["checks"][-1] == {"id": "server.freshness", "status": "ok", "fixed": False} + + +def test_doctor_detects_stale_server(tmp_path: Path, monkeypatch) -> None: + """A server started before the on-disk wardline source last changed is STALE — + the exact dogfood-2026-06-06 failure class. The verdict must flip ok and land in + next_actions with the restart instruction.""" + _isolate(tmp_path, monkeypatch) + payload = _doctor({}, tmp_path, started_at=1.0) # 1970 — everything on disk is newer + server = payload["server"] + assert server["fresh"] is False + freshness = payload["checks"][-1] + assert freshness["id"] == "server.freshness" + assert freshness["status"] == "error" + assert "restart" in freshness["message"].lower() + assert payload["ok"] is False + assert any("server.freshness" in action and "restart" in action.lower() for action in payload["next_actions"]) + + +def test_doctor_default_is_read_only(tmp_path: Path, monkeypatch) -> None: + home = _isolate(tmp_path, monkeypatch) + _doctor({}, tmp_path, started_at=time.time()) + assert not (tmp_path / "CLAUDE.md").exists() + assert not (tmp_path / ".mcp.json").exists() + assert not (home / ".codex" / "config.toml").exists() + + +def test_doctor_repair_true_repairs_install_artifacts(tmp_path: Path, monkeypatch) -> None: + home = _isolate(tmp_path, monkeypatch) + payload = _doctor({"repair": True}, tmp_path, started_at=time.time()) + assert "wardline:instructions:" in (tmp_path / "CLAUDE.md").read_text(encoding="utf-8") + assert (tmp_path / ".mcp.json").is_file() + assert (home / ".codex" / "config.toml").is_file() + assert (tmp_path / ".weft" / "wardline").is_dir() + by_id = {c["id"]: c for c in payload["checks"]} + assert by_id["mcp.registration"]["status"] == "ok" + + +def _tool_call(server: WardlineMCPServer, name: str, arguments: dict[str, Any] | None = None) -> dict[str, Any]: + resp = server.rpc.dispatch( + { + "jsonrpc": "2.0", + "id": 1, + "method": "tools/call", + "params": {"name": name, "arguments": arguments or {}}, + } + ) + assert "error" not in resp, resp + return resp["result"] + + +def test_doctor_repair_is_denied_by_no_write_policy(tmp_path: Path, monkeypatch) -> None: + _isolate(tmp_path, monkeypatch) + server = WardlineMCPServer(root=tmp_path, allow_write=False) + result = _tool_call(server, "doctor", {"repair": True}) + assert result["isError"] is True + assert "write" in result["content"][0]["text"].lower() + # The read-only probe stays allowed under the same policy. + ok = _tool_call(server, "doctor") + assert "isError" not in ok + + +def test_doctor_with_probe_url_is_denied_by_no_network_policy(tmp_path: Path, monkeypatch) -> None: + _isolate(tmp_path, monkeypatch) + server = WardlineMCPServer(root=tmp_path, filigree_url="http://127.0.0.1:9/weft", allow_network=False) + result = _tool_call(server, "doctor") + assert result["isError"] is True + assert "network" in result["content"][0]["text"].lower() + + +def test_doctor_registered_with_served_schema(tmp_path: Path, monkeypatch) -> None: + _isolate(tmp_path, monkeypatch) + server = WardlineMCPServer(root=tmp_path) + resp = server.rpc.dispatch({"jsonrpc": "2.0", "id": 1, "method": "tools/list", "params": {}}) + doctor = next(t for t in resp["result"]["tools"] if t["name"] == "doctor") + props = doctor["inputSchema"]["properties"] + assert props["repair"]["type"] == "boolean" + assert props["filigree_url"]["type"] == "string" + # The anti-stale-server contract is advertised where agents read it. + assert "stale" in doctor["description"].lower() or "fresh" in doctor["description"].lower() + + +def test_doctor_over_rpc_serves_identity(tmp_path: Path, monkeypatch) -> None: + """End-to-end through the dispatch loop: the server's own started_at feeds the + freshness verdict, and a just-started server is fresh.""" + import json + + _isolate(tmp_path, monkeypatch) + server = WardlineMCPServer(root=tmp_path) + result = _tool_call(server, "doctor") + payload = json.loads(result["content"][0]["text"]) + assert payload["server"]["fresh"] is True + assert payload["server"]["package_version"] == __version__ diff --git a/tests/unit/mcp/test_server_structure.py b/tests/unit/mcp/test_server_structure.py index fed60425..00d1d851 100644 --- a/tests/unit/mcp/test_server_structure.py +++ b/tests/unit/mcp/test_server_structure.py @@ -37,6 +37,7 @@ def test_mcp_advertisement_snapshot() -> None: "baseline", "waiver_add", "fix", + "doctor", ] assert [resource["uri"] for resource in resources["result"]["resources"]] == [ "wardline://vocab", From 9e5feb4f67822cb4672202c97d1e4a5020f27351 Mon Sep 17 00:00:00 2001 From: John Morrissey Date: Thu, 11 Jun 2026 16:05:33 +1000 Subject: [PATCH 100/186] =?UTF-8?q?feat(mcp):=20A3=20=E2=80=94=20rekey=20M?= =?UTF-8?q?CP=20twin,=20probe-by-default=20(closes=20wardline-d8cc650ab9)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit New `rekey` MCP tool over the same core.rekey the CLI drives — no second migration path. Default is the read-only probe (match/orphans-with-cause/ collisions/per_store, writes nothing); apply/resume/rollback are explicit, mutually exclusive, WRITE-gated via _effective_tool_capabilities; apply adds NETWORK only when a Filigree URL resolves (the re-emit leg). Journal payload is lean: carried as counts, orphans verbatim (a dropped verdict is never silent), collisions with messages, next_pending_leg + re-run hint when incomplete. ORPHAN_CAUSE moved to core.rekey so CLI and MCP share one string. Live-verified over JSON-RPC: probe → apply (real wlfp1 verdict carried) → re-apply refused loudly → rollback byte-identical. Co-Authored-By: Claude Fable 5 --- CHANGELOG.md | 7 + src/wardline/cli/rekey.py | 4 +- src/wardline/core/rekey.py | 3 + src/wardline/mcp/server.py | 151 +++++++++++++++++++++ tests/conformance/test_mcp_handshake.py | 1 + tests/unit/mcp/test_server_rekey.py | 170 ++++++++++++++++++++++++ tests/unit/mcp/test_server_structure.py | 1 + 7 files changed, 334 insertions(+), 3 deletions(-) create mode 100644 tests/unit/mcp/test_server_rekey.py diff --git a/CHANGELOG.md b/CHANGELOG.md index 4f1600d8..9fe5c8ad 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -8,6 +8,13 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] ### Added +- **MCP `rekey` tool** — fingerprint-scheme migration over MCP via the same + `core.rekey` the CLI drives (no second migration path). Probe-by-default + (read-only match/orphan/collision report, writes nothing); `apply` / + `resume` / `rollback` are explicit, mutually exclusive, write-gated args; + `apply` re-emits to a configured Filigree (network-gated) like the CLI's + `--filigree-url` leg. Orphaned verdicts are listed verbatim with the shared + orphan-cause explanation (`wardline-d8cc650ab9`, MCP-primary A3). - **MCP `doctor` tool** — the CLI `doctor --fix` health envelope over MCP (install artifacts, MCP registration, config parseability, sibling URLs, Filigree emit-auth probe; read-only by default, `repair: true` is the diff --git a/src/wardline/cli/rekey.py b/src/wardline/cli/rekey.py index 09f7fa27..422c3b71 100644 --- a/src/wardline/cli/rekey.py +++ b/src/wardline/cli/rekey.py @@ -15,12 +15,10 @@ from wardline.core.config import resolve_filigree_url from wardline.core.errors import WardlineError from wardline.core.filigree_emit import FiligreeEmitter +from wardline.core.rekey import ORPHAN_CAUSE as _ORPHAN_CAUSE from wardline.core.rekey import Journal, ProbeReport, probe, resume_rekey, rollback, run_rekey from wardline.core.run import run_scan -# Why a verdict can orphan (NOT only a source move) — shared by --probe and --resume output. -_ORPHAN_CAUSE = "source moved/deleted, or a custom multi-emit rule not surfacing taint_path_v0" - def _print_prescheme_caution() -> None: click.echo( diff --git a/src/wardline/core/rekey.py b/src/wardline/core/rekey.py index bca5c58e..f0386e60 100644 --- a/src/wardline/core/rekey.py +++ b/src/wardline/core/rekey.py @@ -30,6 +30,9 @@ from wardline.core.waivers import WAIVERS_VERSION SNAPSHOT_DIR_NAME = ".rekey_snapshot" +# Why a verdict can orphan (NOT only a source move) — the one explanation both +# surfaces (CLI rekey output, MCP rekey payload) attach to every dropped verdict. +ORPHAN_CAUSE = "source moved/deleted, or a custom multi-emit rule not surfacing taint_path_v0" # (store filename, list-key inside the YAML doc, version constant) — the three YAML # legs, in gate-criticality order (baseline first restores the local --fail-on gate). _STORES: tuple[tuple[str, str, int], ...] = ( diff --git a/src/wardline/mcp/server.py b/src/wardline/mcp/server.py index bee8dc5d..3ec75fde 100644 --- a/src/wardline/mcp/server.py +++ b/src/wardline/mcp/server.py @@ -701,6 +701,93 @@ def _doctor( return attach_server_identity(payload, root=root, started_at=started_at) +def _rekey(args: dict[str, Any], root: Path, filigree: Any = None) -> dict[str, Any]: + """Fingerprint-scheme migration over MCP (A3): the same core.rekey the CLI drives — + no second migration path. Probe-by-default (read-only: report match/orphans/ + collisions, write NOTHING); `apply`/`resume`/`rollback` are explicit, mutually + exclusive, WRITE-gated. The injected Filigree emitter (apply only) re-emits under + the new fingerprints, best-effort like the CLI's --filigree-url leg.""" + from wardline.core.rekey import ORPHAN_CAUSE, Journal, probe, resume_rekey, rollback, run_rekey + + apply_ = _bool_arg(args, "apply", False) + do_resume = _bool_arg(args, "resume", False) + do_rollback = _bool_arg(args, "rollback", False) + if sum((apply_, do_resume, do_rollback)) > 1: + raise ToolError("apply, resume and rollback are mutually exclusive") + path = _resolve_under_root(root, args["path"]) if args.get("path") else root + + def journal_block(journal: Journal) -> dict[str, Any]: + # Lean payload: carried as a COUNT (can be the whole store), orphans VERBATIM + # (a dropped verdict must never be silent — mirrors the CLI's per-orphan lines). + block: dict[str, Any] = { + "complete": journal.complete, + "fingerprint_scheme_from": journal.fingerprint_scheme_from, + "fingerprint_scheme_to": journal.fingerprint_scheme_to, + "snapshot_prescheme": journal.snapshot_prescheme, + "orphan_cause": ORPHAN_CAUSE, + "collisions": [ + {"new_fp": c.new_fp, "old_fps": list(c.old_fps), "message": c.message} for c in journal.collisions + ], + "legs": [ + { + "name": leg.name, + "done": leg.done, + "carried_count": len(leg.carried), + "orphaned": list(leg.orphaned), + "debt": leg.debt, + } + for leg in journal.legs + ], + "next_pending_leg": journal.next_pending_leg(), + } + if not journal.complete: + block["next_action"] = "re-run the rekey tool to finish pending leg(s)" + return block + + if do_rollback: + rolled = rollback(path) + return { + "mode": "rollback", + "restored": list(rolled.restored), + "note": "Filigree associations from the forward run are NOT reversed " + "(no remap endpoint); reconcile manually if needed.", + } + if do_resume: + # Resume NEVER re-scans — YAML legs re-carry from the snapshot; a pending + # Filigree leg is deferred as debt (re-run with apply to retry it). + return {"mode": "resume", **journal_block(resume_rekey(path))} + + # Probe (the default) and apply both need the suppression-free scan: migration + # scans WITHOUT loading the stores it is about to rekey (they are still + # old-scheme and would SCHEME_MISMATCH). + result = run_scan( + path, + config_path=_cfg(args, root), + cache_dir=_cache_dir_arg(args, root), + confine_to_root=True, + trust_local_packs=bool(args.get("trust_local_packs", False)), + trusted_packs=_trusted_packs_arg(args), + strict_defaults=bool(args.get("strict_defaults", False)), + skip_suppression=True, + ) + if not apply_: + report = probe(path, result.findings) + return { + "mode": "probe", + "scanned_findings": report.scanned_findings, + "matched": report.matched, + "orphaned": list(report.orphaned), + "orphan_cause": ORPHAN_CAUSE, + "collisions": [ + {"new_fp": c.new_fp, "old_fps": list(c.old_fps), "message": c.message} for c in report.collisions + ], + "per_store": dict(report.per_store), + "prescheme": report.prescheme, + "clean": report.clean, + } + return {"mode": "apply", **journal_block(run_rekey(path, result.findings, filigree=filigree))} + + def _fix(args: dict[str, Any], root: Path) -> dict[str, Any]: """Scan the path and apply mechanical autofixes to findings in-place.""" path = _resolve_under_root(root, args["path"]) if args.get("path") else root @@ -1323,6 +1410,64 @@ def _register_tools(self) -> None: ), ) ) + self.add_tool( + Tool( + name="rekey", + description="Re-key baseline/waiver/judged verdicts across a fingerprint-scheme " + "change (after the engine's fingerprint formula migrates, NOT after ordinary " + "refactors — fingerprints are line-insensitive). DEFAULT is a read-only PROBE: " + "reports how many stored verdicts will carry, which orphan and why, and any " + "collisions — writes nothing. Pass `apply: true` to migrate (snapshots first, " + "resumable journal), `resume: true` to finish an interrupted migration without " + "re-scanning, `rollback: true` to restore the pre-migration stores. The three " + "are mutually exclusive and write-gated.", + input_schema={ + "type": "object", + "properties": { + "path": {"type": "string", "description": "subdir relative to project root"}, + "config": {"type": "string"}, + "cache_dir": { + "type": "string", + "description": "subdir relative to project root for summary cache", + }, + "trust_packs": {"type": "array", "items": {"type": "string"}}, + "trust_local_packs": { + "type": "boolean", + "description": "Allow loading custom trust-grammar packs from the local project directory", + }, + "strict_defaults": { + "type": "boolean", + "description": "Ignore repository-supplied custom configuration overrides (weft.toml)", + }, + "apply": { + "type": "boolean", + "description": "Default false (read-only probe). true RUNS the migration: " + "snapshot stores, write the resumable journal, carry verdicts to the " + "new fingerprints, best-effort re-emit to a configured Filigree.", + }, + "resume": { + "type": "boolean", + "description": "Finish an interrupted migration from the journal WITHOUT " + "re-scanning (YAML legs re-carry from the snapshot).", + }, + "rollback": { + "type": "boolean", + "description": "Restore the pre-migration stores byte-identical from the " + "snapshot and remove the journal. Filigree associations are NOT reversed.", + }, + }, + }, + handler=lambda args, root: _rekey( + args, + root, + # The Filigree leg only runs under apply; building the emitter is + # cheap and returns None when no URL resolves. + self._filigree_emitter(_cfg(args, root), strict_defaults=bool(args.get("strict_defaults") or False)) + if args.get("apply") + else None, + ), + ) + ) def add_tool(self, tool: Tool) -> None: schema = dict(tool.input_schema) @@ -1400,6 +1545,12 @@ def _effective_tool_capabilities(self, tool: Tool, arguments: dict[str, Any]) -> if _resolve_probe_url(self.root, probe_url or self.filigree_url) is not None: # The filigree-auth probe will touch the (loopback-only) network. capabilities.add(ToolCapability.NETWORK) + if tool.name == "rekey": + if any(bool(arguments.get(k, False)) for k in ("apply", "resume", "rollback")): + capabilities.add(ToolCapability.WRITE) + if bool(arguments.get("apply", False)) and self._resolved_filigree_url_for_policy(arguments) is not None: + # apply's last leg re-emits the rekeyed findings to Filigree. + capabilities.add(ToolCapability.NETWORK) return frozenset(capabilities) def _tools_call(self, params: dict[str, Any]) -> dict[str, Any]: diff --git a/tests/conformance/test_mcp_handshake.py b/tests/conformance/test_mcp_handshake.py index a44e5cd7..db4469aa 100644 --- a/tests/conformance/test_mcp_handshake.py +++ b/tests/conformance/test_mcp_handshake.py @@ -70,6 +70,7 @@ def test_full_client_handshake_and_every_surface() -> None: "fix", "waiver_add", "doctor", + "rekey", } # resources/list: the four stable URIs resource_uris = {r["uri"] for r in by_id[3]["result"]["resources"]} diff --git a/tests/unit/mcp/test_server_rekey.py b/tests/unit/mcp/test_server_rekey.py new file mode 100644 index 00000000..35cf8e47 --- /dev/null +++ b/tests/unit/mcp/test_server_rekey.py @@ -0,0 +1,170 @@ +"""A3 (wardline-d8cc650ab9): the `rekey` MCP twin. + +Probe-by-default (read-only: report match/orphans/collisions, write NOTHING); +`apply` / `resume` / `rollback` are explicit, mutually exclusive, WRITE-gated args. +Shares the CLI's core implementation (core.rekey) — no second migration path. +""" + +from __future__ import annotations + +from pathlib import Path +from typing import Any + +import pytest + +yaml = pytest.importorskip("yaml") +pytest.importorskip("blake3", reason="run_scan needs wardline[loomweave]") + +from wardline.core import paths # noqa: E402 +from wardline.core.baseline import load_baseline # noqa: E402 +from wardline.core.fingerprint_v0 import compute_finding_fingerprint_v0 # noqa: E402 +from wardline.core.rekey import load_journal, snapshot_dir, write_journal # noqa: E402 +from wardline.core.run import run_scan # noqa: E402 +from wardline.mcp.server import WardlineMCPServer, _rekey # noqa: E402 +from wardline.mcp.tooling import ToolError # noqa: E402 + +_LEAKY = ( + "from wardline.decorators import external_boundary, trusted\n" + "@external_boundary\ndef raw(p):\n return p\n" + "@trusted\ndef leaky(p):\n return raw(p)\n" +) + + +def _project(tmp_path: Path) -> Path: + project = tmp_path / "proj" + project.mkdir() + (project / "svc.py").write_text(_LEAKY, encoding="utf-8") + return project + + +def _seed_wlfp1_baseline(project: Path, *, extra_fps: tuple[str, ...] = ()): + leak = next(f for f in run_scan(project).findings if f.rule_id == "PY-WL-101") + old_fp = compute_finding_fingerprint_v0( + rule_id=leak.rule_id, + path=leak.location.path, + line_start=leak.location.line_start, + qualname=leak.qualname, + taint_path=leak.taint_path_v0, + ) + entries = [{"fingerprint": old_fp, "rule_id": leak.rule_id, "path": leak.location.path, "message": leak.message}] + entries += [{"fingerprint": fp, "rule_id": "PY-WL-101", "path": "gone.py", "message": "x"} for fp in extra_fps] + bp = paths.baseline_path(project) + bp.parent.mkdir(parents=True, exist_ok=True) + bp.write_text( + yaml.safe_dump({"fingerprint_scheme": "wlfp1", "version": 1, "entries": entries}), + encoding="utf-8", + ) + return leak, old_fp + + +def test_rekey_defaults_to_read_only_probe(tmp_path: Path) -> None: + project = _project(tmp_path) + _seed_wlfp1_baseline(project) + result = _rekey({}, project) + assert result["mode"] == "probe" + assert result["matched"] == 1 + assert result["orphaned"] == [] + assert result["clean"] is True + # Writes NOTHING: no snapshot, no journal, store untouched (still wlfp1). + assert not paths.migration_journal_path(project).exists() + assert not snapshot_dir(project).exists() + assert "wlfp1" in paths.baseline_path(project).read_text(encoding="utf-8") + + +def test_rekey_probe_reports_orphans_with_cause(tmp_path: Path) -> None: + project = _project(tmp_path) + _seed_wlfp1_baseline(project, extra_fps=("deadbeef" * 8,)) + result = _rekey({}, project) + assert result["clean"] is False + assert result["orphaned"] == ["deadbeef" * 8] + assert result["per_store"] == {"baseline.yaml": 1} + assert "moved" in result["orphan_cause"] + + +def test_rekey_apply_migrates_and_reports_journal(tmp_path: Path) -> None: + project = _project(tmp_path) + leak, old_fp = _seed_wlfp1_baseline(project) + assert leak.fingerprint != old_fp + result = _rekey({"apply": True}, project) + assert result["mode"] == "apply" + assert result["complete"] is True + legs = {leg["name"]: leg for leg in result["legs"]} + assert legs["baseline"]["done"] is True + assert legs["baseline"]["carried_count"] == 1 + assert load_baseline(paths.baseline_path(project)).fingerprints == frozenset({leak.fingerprint}) + assert (snapshot_dir(project) / "baseline.yaml").is_file() + + +def test_rekey_resume_finishes_without_rescan(tmp_path: Path) -> None: + project = _project(tmp_path) + leak, _old = _seed_wlfp1_baseline(project) + _rekey({"apply": True}, project) + # Revert the baseline leg to pending, corrupt the live store, delete the source — + # resume must re-carry from the snapshot and NEVER re-scan. + jpath = paths.migration_journal_path(project) + journal = load_journal(jpath) + journal.leg("baseline").done = False + write_journal(jpath, journal, root=project) + paths.baseline_path(project).write_text( + yaml.safe_dump({"fingerprint_scheme": "wlfp2", "version": 1, "entries": []}), encoding="utf-8" + ) + (project / "svc.py").unlink() + result = _rekey({"resume": True}, project) + assert result["mode"] == "resume" + assert result["complete"] is True + assert load_baseline(paths.baseline_path(project)).fingerprints == frozenset({leak.fingerprint}) + + +def test_rekey_rollback_restores_stores(tmp_path: Path) -> None: + project = _project(tmp_path) + _leak, old_fp = _seed_wlfp1_baseline(project) + before = paths.baseline_path(project).read_bytes() + _rekey({"apply": True}, project) + result = _rekey({"rollback": True}, project) + assert result["mode"] == "rollback" + assert "baseline.yaml" in result["restored"] + assert "not reversed" in result["note"].lower() + assert paths.baseline_path(project).read_bytes() == before + assert not paths.migration_journal_path(project).exists() + + +def test_rekey_modes_are_mutually_exclusive(tmp_path: Path) -> None: + project = _project(tmp_path) + with pytest.raises(ToolError, match="mutually exclusive"): + _rekey({"apply": True, "rollback": True}, project) + + +def _tool_call(server: WardlineMCPServer, name: str, arguments: dict[str, Any] | None = None) -> dict[str, Any]: + resp = server.rpc.dispatch( + { + "jsonrpc": "2.0", + "id": 1, + "method": "tools/call", + "params": {"name": name, "arguments": arguments or {}}, + } + ) + assert "error" not in resp, resp + return resp["result"] + + +def test_rekey_apply_denied_by_no_write_policy(tmp_path: Path) -> None: + project = _project(tmp_path) + _seed_wlfp1_baseline(project) + server = WardlineMCPServer(root=project, allow_write=False) + for arg in ("apply", "resume", "rollback"): + result = _tool_call(server, "rekey", {arg: True}) + assert result["isError"] is True, arg + assert "write" in result["content"][0]["text"].lower() + # The default read-only probe stays allowed under the same policy. + ok = _tool_call(server, "rekey") + assert "isError" not in ok + + +def test_rekey_apply_with_filigree_url_denied_by_no_network_policy(tmp_path: Path, monkeypatch) -> None: + monkeypatch.delenv("WARDLINE_FILIGREE_URL", raising=False) + project = _project(tmp_path) + _seed_wlfp1_baseline(project) + server = WardlineMCPServer(root=project, filigree_url="http://127.0.0.1:9/weft", allow_network=False) + result = _tool_call(server, "rekey", {"apply": True}) + assert result["isError"] is True + assert "network" in result["content"][0]["text"].lower() diff --git a/tests/unit/mcp/test_server_structure.py b/tests/unit/mcp/test_server_structure.py index 00d1d851..af6c29b7 100644 --- a/tests/unit/mcp/test_server_structure.py +++ b/tests/unit/mcp/test_server_structure.py @@ -38,6 +38,7 @@ def test_mcp_advertisement_snapshot() -> None: "waiver_add", "fix", "doctor", + "rekey", ] assert [resource["uri"] for resource in resources["result"]["resources"]] == [ "wardline://vocab", From a67f22fe0e505f1196f86bbda3b050af6a05ac5d Mon Sep 17 00:00:00 2001 From: John Morrissey Date: Thu, 11 Jun 2026 16:24:44 +1000 Subject: [PATCH 101/186] =?UTF-8?q?feat(mcp):=20A4=20=E2=80=94=20fail=5Fon?= =?UTF-8?q?=5Funanalyzed=20arg=20on=20MCP=20scan;=20unanalyzed=20gate=20fo?= =?UTF-8?q?lded=20into=20gate=5Fdecision=20(closes=20wardline-7fd0f3a82c)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The CLI's --fail-on-unanalyzed knob existed only as an exit-code OR at the CLI layer, so the MCP surface (no exit code, gate block only) could report summary.unanalyzed but never gate on it. Root-cause fix: the unanalyzed sub-gate now lives inside gate_decision itself — one shared implementation both surfaces serialise, so they cannot drift. - GateDecision: fail_on_unanalyzed knob + severity_tripped/unanalyzed_tripped sub-gate attribution, guarded by invariants (an overall trip no sub-gate explains is unconstructible, same discipline as the dogfood-#2 guards); verdict is NOT_EVALUATED iff NEITHER knob is set. - MCP scan: fail_on_unanalyzed boolean arg (default off, same as CLI); gate block exposes the knob + both sub-trip flags. - CLI: delegates entirely to the shared decision; an unanalyzed-only trip now prints an honest "gate: FAILED (--fail-on-unanalyzed)" instead of "gate: NOT_EVALUATED" + exit 1, and the FAILED line names only the knob(s) that actually tripped. - next_actions: an unanalyzed trip points at fixing what blocked analysis (parse errors / skips / source roots) — never at trust_suppressions / new_since, which cannot clear an under-scan. - Acceptance pinned: CLI and MCP verdicts agree on an unanalyzed fixture under both knob settings (test_cli_mcp_parity); live-verified over JSON-RPC against a fresh wardline mcp (tool count stays 15). Co-Authored-By: Claude Fable 5 --- CHANGELOG.md | 13 ++ .../reference/finding-lifecycle-vocabulary.md | 66 +++++---- src/wardline/cli/scan.py | 25 +++- src/wardline/core/agent_summary.py | 79 +++++++---- src/wardline/core/run.py | 68 ++++++++-- src/wardline/mcp/server.py | 17 ++- tests/docs/test_glossary_vocabulary.py | 30 ++-- tests/unit/core/test_cli_mcp_parity.py | 35 +++++ tests/unit/core/test_run.py | 128 +++++++++++++++++- 9 files changed, 371 insertions(+), 90 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 9fe5c8ad..0d368a72 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -8,6 +8,19 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] ### Added +- **MCP `scan` tool `fail_on_unanalyzed` argument** — the CLI's + `--fail-on-unanalyzed` knob over the MCP surface (default off, same as the + CLI), so gate semantics are fully controllable on the primary surface. The + unanalyzed gate now lives **inside** `gate_decision` (one shared + implementation; the CLI exit-code OR is gone), and the gate block/decision + gained sub-gate attribution: `gate.fail_on_unanalyzed`, + `gate.severity_tripped`, `gate.unanalyzed_tripped` — so an agent can tell a + severity trip from an under-scan trip without parsing `reason`. An + unanalyzed trip's `next_actions` point at fixing what blocked analysis + (suppressions cannot clear it). Side fix: CLI `--fail-on-unanalyzed` alone + no longer prints `gate: NOT_EVALUATED` while exiting 1 — the verdict is now + an honest `FAILED`/`PASSED` whenever either sub-gate ran + (`wardline-7fd0f3a82c`, MCP-primary A4). - **MCP `rekey` tool** — fingerprint-scheme migration over MCP via the same `core.rekey` the CLI drives (no second migration path). Probe-by-default (read-only match/orphan/collision report, writes nothing); `apply` / diff --git a/docs/reference/finding-lifecycle-vocabulary.md b/docs/reference/finding-lifecycle-vocabulary.md index d0fe67ee..e45133e1 100644 --- a/docs/reference/finding-lifecycle-vocabulary.md +++ b/docs/reference/finding-lifecycle-vocabulary.md @@ -71,9 +71,9 @@ consistently, on every surface: | Surface | Where | Term | | --- | --- | --- | | Enum | `src/wardline/core/finding.py:72` | `SuppressionState.ACTIVE = "active"` | -| Summary field | `src/wardline/core/run.py:51`, built at `src/wardline/core/run.py:361` | `ScanSummary.active` | +| Summary field | `src/wardline/core/run.py:51`, built at `src/wardline/core/run.py:378` | `ScanSummary.active` | | CLI summary line | `src/wardline/cli/scan.py:416` | `… {s.active} active` | -| MCP scan response | `src/wardline/mcp/server.py:335` | `summary.active` | +| MCP scan response | `src/wardline/mcp/server.py:337` | `summary.active` | | Agent-summary JSON | `src/wardline/core/agent_summary.py:135` | `summary.active_defects` | | `wardline:loop` prompt | `src/wardline/mcp/prompts.py:13` | "Read `summary.active`" | @@ -101,8 +101,8 @@ So `active + baselined + waived + judged + informational == total` (`src/wardline/core/run.py:50` for `total: int`). `unanalyzed` (`src/wardline/core/run.py:69`) is an **overlay** — a subset of `informational` that surfaces a silent under-scan — and is deliberately **not** a partition member. -The MCP `summary` block exposes `informational` (`src/wardline/mcp/server.py:343`) -and `unanalyzed` (`src/wardline/mcp/server.py:347`); the agent-summary block mirrors +The MCP `summary` block exposes `informational` (`src/wardline/mcp/server.py:345`) +and `unanalyzed` (`src/wardline/mcp/server.py:349`); the agent-summary block mirrors both (`src/wardline/core/agent_summary.py:146`, `src/wardline/core/agent_summary.py:147`). ## Emitted-active vs the gate population @@ -111,19 +111,19 @@ There are **two distinct populations** of defects in one scan, and they can differ on purpose: 1. **Emitted-active** — `summary.active` counts `active` defects in the - **emitted** (post-annotation) findings (built at `src/wardline/core/run.py:361`). + **emitted** (post-annotation) findings (built at `src/wardline/core/run.py:378`). Baseline / waiver / judged annotate these findings in place; a suppressed defect is still emitted, just not counted as `active`. 2. **Gate population** — the `--fail-on` gate evaluates a **separate** `ScanResult.gate_findings` list: the *unsuppressed* population - (`src/wardline/core/run.py:327`). By default, repository-controlled + (`src/wardline/core/run.py:326`). By default, repository-controlled baseline / waiver / judged entries **annotate** the emitted findings but do **not** clear the gate — so a malicious PR cannot green the gate by committing a suppression keyed to its own new defect. `gate_decision` evaluates `gate_findings` when present, else falls back to `findings` (the trusted `--trust-suppressions` / directly-constructed path), selected at - `src/wardline/core/run.py:412` (`honors_suppressions`). + `src/wardline/core/run.py:439` (`honors_suppressions`). This is why **`summary.active: 0` can co-exist with `gate.tripped: true`**: every defect was suppressed by a committed baseline (so emitted-active is 0), but those @@ -137,25 +137,33 @@ bug. `would_trip_at`, alongside a human `reason` and the `evaluated` population it judged. The `verdict` is one of: -- **`NOT_EVALUATED`** — no `--fail-on` threshold was set, so the gate never judged. - A bare scan is **not** a clean pass; `would_trip_at` names the highest severity - that *would* trip so the agent's first call is not a false green (weft-b937e53854). -- **`PASSED`** — a threshold ran and nothing tripped. -- **`FAILED`** — a threshold ran and tripped. - -The MCP `scan` gate block exposes `gate.tripped` (`src/wardline/mcp/server.py:350`), -`gate.verdict` (`src/wardline/mcp/server.py:353`), `would_trip_at`, `reason`, -`evaluated`, and `migration_hint`, opened at `src/wardline/mcp/server.py:349` +- **`NOT_EVALUATED`** — neither gate knob (`--fail-on` / `--fail-on-unanalyzed`) + was set, so the gate never judged. A bare scan is **not** a clean pass; + `would_trip_at` names the highest severity that *would* trip so the agent's + first call is not a false green (weft-b937e53854). +- **`PASSED`** — at least one knob ran and nothing tripped. +- **`FAILED`** — a knob ran and tripped. + +The decision composes two independent sub-gates: the severity gate (`fail_on`) +and the opt-in unanalyzed gate (`fail_on_unanalyzed`, MCP-primary A4 — +trips when any file was discovered but never analysed; benign no-module skips +excluded). `severity_tripped` / `unanalyzed_tripped` attribute an overall +`tripped` to its sub-gate(s) so no consumer has to parse `reason`. + +The MCP `scan` gate block exposes `gate.tripped` (`src/wardline/mcp/server.py:352`), +`gate.fail_on_unanalyzed`, `gate.verdict` (`src/wardline/mcp/server.py:356`), +`gate.severity_tripped`, `gate.unanalyzed_tripped`, `would_trip_at`, `reason`, +`evaluated`, and `migration_hint`, opened at `src/wardline/mcp/server.py:351` (`"gate": {`); the agent-summary mirrors them at `src/wardline/core/agent_summary.py:150` (`tripped`) and `src/wardline/core/agent_summary.py:153` (`verdict`). The CLI prints -`gate: FAILED (--fail-on …) — ` then `gate: evaluated <…>`, or a +`gate: FAILED () — ` then `gate: evaluated <…>`, or a `gate: NOT_EVALUATED — …` line for a bare scan -(`src/wardline/cli/scan.py:456`). +(`src/wardline/cli/scan.py:468`). `--new-since` scopes **both** populations identically: any `active` defect outside the delta is re-marked `baselined` in both the emitted and gate lists -(`src/wardline/core/run.py:337`, `def apply_delta_scope`). +(`src/wardline/core/run.py:354`, `def apply_delta_scope`). ## The three meanings of "new" @@ -166,7 +174,7 @@ still legitimately means three different things depending on the surface: | "new" on this surface | Means | Owner / anchor | | --- | --- | --- | | Filigree store | An **unseen fingerprint** — first time this finding identity is seen for a `(file, scan_source)`. | **Filigree-owned** lifecycle (`src/wardline/core/filigree_emit.py:68-76`) | -| `wardline scan --new-since ` | **Delta-scope**: the gate fires only on defects in files/entities changed since a git ref; everything else is re-marked `baselined`. | `src/wardline/core/run.py:337`; help text `src/wardline/cli/scan.py` (`--new-since`) | +| `wardline scan --new-since ` | **Delta-scope**: the gate fires only on defects in files/entities changed since a git ref; everything else is re-marked `baselined`. | `src/wardline/core/run.py:354`; help text `src/wardline/cli/scan.py` (`--new-since`) | | (historical) CLI summary | Formerly relabelled the `active` count as "N new". **Corrected to "N active"**. | `src/wardline/cli/scan.py:415` | The first-seen Filigree sense and the delta-scope `--new-since` sense are @@ -178,19 +186,19 @@ How each concept appears on each surface: | Concept | CLI summary text | `ScanSummary` field | MCP `summary` key | Agent-summary key | Filigree store | | --- | --- | --- | --- | --- | --- | -| every finding | `N finding(s)` | `total` (`run.py:50`) | `total` (`server.py:334`) | `total_findings` (`agent_summary.py:134`) | one finding per wire entry | -| live defect | `N active` (`scan.py:416`) | `active` (`run.py:51,361`) | `active` (`server.py:335`) | `active_defects` (`agent_summary.py:135`) | no `suppression_state` key (`finding.py:285`) | +| every finding | `N finding(s)` | `total` (`run.py:50`) | `total` (`server.py:336`) | `total_findings` (`agent_summary.py:134`) | one finding per wire entry | +| live defect | `N active` (`scan.py:416`) | `active` (`run.py:51,361`) | `active` (`server.py:337`) | `active_defects` (`agent_summary.py:135`) | no `suppression_state` key (`finding.py:285`) | | suppressed (sum) | `N suppressed` (`scan.py:415`) | `baselined+waived+judged` | the three keys | `suppressed_findings` (`agent_summary.py:136`) | `metadata.wardline.suppression_state` (`finding.py:285`) | -| baselined | `N baseline` | `baselined` (`run.py:53`) | `baselined` (`server.py:336`) | `baselined` (`agent_summary.py:138`) | `suppression_state: "baselined"` | -| waived | `N waiver` | `waived` (`run.py:54`) | `waived` (`server.py:337`) | `waived` (`agent_summary.py:139`) | `suppression_state: "waived"` | -| judged | `N judged` | `judged` (`run.py:55`) | `judged` (`server.py:338`) | `judged` (`agent_summary.py:140`) | `suppression_state: "judged"` | -| informational (summary) | (the remainder of `total`) | `informational` (`run.py:61`) | `informational` (`server.py:343`) | `informational` (`agent_summary.py:146`) | facts/metrics | +| baselined | `N baseline` | `baselined` (`run.py:53`) | `baselined` (`server.py:338`) | `baselined` (`agent_summary.py:138`) | `suppression_state: "baselined"` | +| waived | `N waiver` | `waived` (`run.py:54`) | `waived` (`server.py:339`) | `waived` (`agent_summary.py:139`) | `suppression_state: "waived"` | +| judged | `N judged` | `judged` (`run.py:55`) | `judged` (`server.py:340`) | `judged` (`agent_summary.py:140`) | `suppression_state: "judged"` | +| informational (summary) | (the remainder of `total`) | `informational` (`run.py:61`) | `informational` (`server.py:345`) | `informational` (`agent_summary.py:146`) | facts/metrics | | informational (display) | n/a | n/a | n/a | `informational` display array (`agent_summary.py:171`) — non-defect, non-engine-fact findings (metrics, classifications, suggestions, non-engine facts); excludes `engine_facts` which has its own display slot | facts/metrics | -| under-scan | `N file(s) could not be analyzed` | `unanalyzed` (`run.py:69`) | `unanalyzed` (`server.py:347`) | `unanalyzed` (`agent_summary.py:147`) | `WLN-ENGINE-*` facts | -| gate verdict | exit code + `--fail-on` | (`gate_findings`, `run.py:88`; `GateDecision`, `run.py:98`, `verdict` `run.py:107`) | `gate.tripped` (`server.py:350`), `gate.verdict` (`server.py:353`) | `gate.tripped` (`agent_summary.py:150`), `gate.verdict` (`agent_summary.py:153`) | not emitted to Filigree | +| under-scan | `N file(s) could not be analyzed` | `unanalyzed` (`run.py:69`) | `unanalyzed` (`server.py:349`) | `unanalyzed` (`agent_summary.py:147`) | `WLN-ENGINE-*` facts | +| gate verdict | exit code + `--fail-on` | (`gate_findings`, `run.py:88`; `GateDecision`, `run.py:98`, `verdict` `run.py:107`) | `gate.tripped` (`server.py:352`), `gate.verdict` (`server.py:356`) | `gate.tripped` (`agent_summary.py:150`), `gate.verdict` (`agent_summary.py:153`) | not emitted to Filigree | The unsuppressed gate population is built from `Baseline(frozenset())` -(`src/wardline/core/run.py:327`). +(`src/wardline/core/run.py:326`). ## For the suite diff --git a/src/wardline/cli/scan.py b/src/wardline/cli/scan.py index 38ed4191..a590f246 100644 --- a/src/wardline/cli/scan.py +++ b/src/wardline/cli/scan.py @@ -445,23 +445,38 @@ def confirm_cb(rel_path: str, orig: str, replacement: str, f: Finding) -> bool: "Add /// @trusted(level=ASSURED) markers to your boundary functions.", err=True, ) - gate_dec = gate_decision(result, Severity(fail_on)) if fail_on is not None else gate_decision(result, None) - gate_tripped = gate_dec.tripped + # Both sub-gates (severity + opt-in unanalyzed) live in the ONE shared decision — + # the same one the MCP scan tool serialises — so the surfaces cannot drift (A4). + gate_dec = gate_decision( + result, + Severity(fail_on) if fail_on is not None else None, + fail_on_unanalyzed=fail_on_unanalyzed, + ) if gate_dec.verdict == "NOT_EVALUATED": # A bare scan never ran the gate — say so explicitly so a clean-looking exit is not # mistaken for a PASS (weft-b937e53854). Carries would_trip_at via the reason. click.echo(f"gate: NOT_EVALUATED — {gate_dec.reason}", err=True) elif gate_dec.tripped: # Never let "0 active + gate FAILED" read as a bug: say why and which population. - click.echo(f"gate: FAILED (--fail-on {gate_dec.fail_on}) — {gate_dec.reason}", err=True) + # Name only the knob(s) that actually tripped — an unanalyzed-only trip must not + # print "--fail-on None". + knobs = [] + if gate_dec.severity_tripped: + knobs.append(f"--fail-on {gate_dec.fail_on}") + if gate_dec.unanalyzed_tripped: + knobs.append("--fail-on-unanalyzed") + click.echo(f"gate: FAILED ({', '.join(knobs)}) — {gate_dec.reason}", err=True) click.echo(f"gate: evaluated {gate_dec.evaluated}", err=True) # The secure-gate-default rollout signal: a committed baseline that used to clear # the gate now re-enters it. Loud + separable from the generic reason above. hint = baseline_migration_hint(result, gate_dec, root=path, new_since=new_since) if hint is not None: click.echo(hint, err=True) - # Independent of the severity gate: opt-in enforcement of "everything analysed". - if gate_tripped or (fail_on_unanalyzed and s.unanalyzed): + elif gate_dec.fail_on is None: + # Only the unanalyzed gate ran and it passed — keep the no-vacuous-severity-green + # signal a NOT_EVALUATED verdict used to carry here. + click.echo(f"gate: PASSED (--fail-on-unanalyzed only) — {gate_dec.reason}", err=True) + if gate_dec.tripped: raise SystemExit(1) diff --git a/src/wardline/core/agent_summary.py b/src/wardline/core/agent_summary.py index 7efa3233..13dc01c0 100644 --- a/src/wardline/core/agent_summary.py +++ b/src/wardline/core/agent_summary.py @@ -278,6 +278,20 @@ def _finding_entry(finding: Finding, *, include_next: bool) -> dict[str, Any]: return entry +def _unanalyzed_trip_action(gate: GateDecision) -> dict[str, Any]: + # An unanalyzed trip is an under-scan, not a finding — suppression escape hatches + # (trust_suppressions / new_since / baseline edits) cannot clear it; only fixing what + # blocked analysis (or dropping the knob) can. + return { + "tool": "scan", + "reason": ( + f"gate FAILED on unanalyzed files — {gate.reason}. Fix what blocked analysis " + "(parse errors / too-deep skips / missing source roots — see the WLN-ENGINE-* " + "facts), or drop fail_on_unanalyzed; suppressions cannot clear this trip." + ), + } + + def _next_actions_for(active_count: int, gate: GateDecision) -> list[dict[str, Any]]: if active_count > 0: actions = [ @@ -285,29 +299,59 @@ def _next_actions_for(active_count: int, gate: GateDecision) -> list[dict[str, A {"tool": "file_finding", "reason": "promote confirmed true positives after Filigree emission"}, {"tool": "scan", "reason": "rescan after fixes to verify closure"}, ] - if gate.verdict == "NOT_EVALUATED": - # Active defects AND no threshold ran — name the enforcement step so a green-looking - # exit is not mistaken for a pass (weft-b937e53854). + if gate.fail_on is None: + # Active defects AND no severity threshold ran (the gate may still have evaluated + # via fail_on_unanalyzed) — name the enforcement step so a green-looking exit is + # not mistaken for a pass (weft-b937e53854). actions.append( { "tool": "scan", "reason": ( - f"gate NOT_EVALUATED (no --fail-on ran); pass --fail-on " + f"severity gate did not run (no --fail-on); pass --fail-on " f"{gate.would_trip_at or 'ERROR'} to enforce" ), } ) + if gate.unanalyzed_tripped: + actions.append(_unanalyzed_trip_action(gate)) return actions - if gate.verdict == "NOT_EVALUATED": - # 0 active defects but the gate never ran. If would_trip_at is set, suppressed/baselined - # defects would re-enter an unsuppressed gate (the dogfood-#2 "worse" case); never let - # this read as a clean pass. + if gate.tripped: + # 0 active defects but the gate FAILED. Attribute each tripping sub-gate — and never + # point an unanalyzed trip at the suppression escape hatches (they cannot clear it). + actions = [] + if gate.unanalyzed_tripped: + actions.append(_unanalyzed_trip_action(gate)) + if gate.severity_tripped: + # The severity gate tripped on suppressed/baselined findings. Do NOT say + # "rescan after edits" (which reads as passed); point at the gate verdict. + detail = gate.reason or "the gate tripped on suppressed (baselined/waived/judged) findings" + actions.append( + { + "tool": "scan", + "reason": ( + f"gate FAILED with 0 active defects — {detail}. To clear: pass " + "trust_suppressions (trusted checkout) or new_since (PR), or remove the " + "baseline/waiver/judged entries; see gate.reason / gate.migration_hint." + ), + } + ) + return actions + if gate.fail_on is None: + # 0 active defects but the severity gate never ran (NOT_EVALUATED, or PASSED on the + # unanalyzed knob alone). If would_trip_at is set, suppressed/baselined defects would + # re-enter an unsuppressed gate (the dogfood-#2 "worse" case); never let this read as + # a clean pass. + label = ( + "gate NOT_EVALUATED (no --fail-on ran)" + if gate.verdict == "NOT_EVALUATED" + else "severity gate did not run (no --fail-on)" + ) if gate.would_trip_at is not None: return [ { "tool": "scan", "reason": ( - f"gate NOT_EVALUATED (no --fail-on ran); 0 active defects but suppressed/baselined " + f"{label}; 0 active defects but suppressed/baselined " f"{gate.would_trip_at}+ finding(s) would trip an unsuppressed gate — pass --fail-on " f"{gate.would_trip_at} to enforce, or trust_suppressions / new_since to scope" ), @@ -316,22 +360,7 @@ def _next_actions_for(active_count: int, gate: GateDecision) -> list[dict[str, A return [ { "tool": "scan", - "reason": "gate NOT_EVALUATED (no --fail-on ran); no defect would trip at any threshold — " - "pass --fail-on ERROR to lock it in", - } - ] - if gate.tripped: - # 0 active defects but the gate FAILED — it tripped on suppressed/baselined findings. - # Do NOT say "rescan after edits" (which reads as passed); point at the gate verdict. - detail = gate.reason or "the gate tripped on suppressed (baselined/waived/judged) findings" - return [ - { - "tool": "scan", - "reason": ( - f"gate FAILED with 0 active defects — {detail}. To clear: pass " - "trust_suppressions (trusted checkout) or new_since (PR), or remove the " - "baseline/waiver/judged entries; see gate.reason / gate.migration_hint." - ), + "reason": f"{label}; no defect would trip at any threshold — pass --fail-on ERROR to lock it in", } ] return [{"tool": "scan", "reason": "no active defects; rescan after edits"}] diff --git a/src/wardline/core/run.py b/src/wardline/core/run.py index 57e93471..0265b35d 100644 --- a/src/wardline/core/run.py +++ b/src/wardline/core/run.py @@ -114,6 +114,14 @@ class GateDecision: reason: str | None = None evaluated: str | None = None would_trip_at: str | None = None + # The unanalyzed sub-gate (A4, wardline-7fd0f3a82c). ``fail_on_unanalyzed`` is the knob + # state; the two ``*_tripped`` flags decompose ``tripped`` so consumers (CLI echo, MCP + # next_actions) can attribute a trip to its sub-gate without parsing ``reason``. The + # decomposition lives IN the decision — not as a surface-level exit-code OR — so the + # MCP gate block (which has no exit code) can express the unanalyzed gate at all. + fail_on_unanalyzed: bool = False + severity_tripped: bool = False + unanalyzed_tripped: bool = False def __post_init__(self) -> None: # Enforce the invariants the ``gate_decision`` factory upholds so a *second* @@ -125,9 +133,10 @@ def __post_init__(self) -> None: if self.verdict not in _VERDICT_VALUES: raise ValueError(f"verdict {self.verdict!r} is not one of {sorted(_VERDICT_VALUES)}") # The verdict is keyed to the gate state — these guards are what stop a tripped gate - # from ever serialising as a pass (the dogfood #2 regression). - if (self.verdict == "NOT_EVALUATED") != (self.fail_on is None): - raise ValueError("verdict NOT_EVALUATED iff no --fail-on threshold is set") + # from ever serialising as a pass (the dogfood #2 regression). The gate is evaluated + # when EITHER sub-gate is configured (a severity threshold or the unanalyzed knob). + if (self.verdict == "NOT_EVALUATED") != (self.fail_on is None and not self.fail_on_unanalyzed): + raise ValueError("verdict NOT_EVALUATED iff neither --fail-on nor --fail-on-unanalyzed is set") if (self.verdict == "FAILED") != self.tripped: raise ValueError("verdict FAILED iff the gate tripped") # Every decision carries its reason now — including NOT_EVALUATED (what would trip). @@ -139,6 +148,14 @@ def __post_init__(self) -> None: raise ValueError(f"fail_on {self.fail_on!r} is not a valid Severity value") if self.would_trip_at is not None and self.would_trip_at not in _SEVERITY_VALUES: raise ValueError(f"would_trip_at {self.would_trip_at!r} is not a valid Severity value") + # The sub-trip decomposition must EXPLAIN the overall trip — an overall trip no + # sub-gate accounts for (or a sub-trip without its knob) is an illegal state. + if self.tripped != (self.severity_tripped or self.unanalyzed_tripped): + raise ValueError("tripped must equal (severity_tripped or unanalyzed_tripped)") + if self.severity_tripped and self.fail_on is None: + raise ValueError("severity_tripped requires a fail_on threshold") + if self.unanalyzed_tripped and not self.fail_on_unanalyzed: + raise ValueError("unanalyzed_tripped requires fail_on_unanalyzed") def run_scan( @@ -392,8 +409,8 @@ def _would_trip_at(gate_population: list[Finding]) -> str | None: return None -def _not_evaluated_reason(would_trip_at: str | None, evaluated: str) -> str: - base = "no --fail-on threshold set; gate did not evaluate" +def _not_evaluated_reason(would_trip_at: str | None, evaluated: str, *, gate: str = "gate") -> str: + base = f"no --fail-on threshold set; {gate} did not evaluate" if would_trip_at is None: return f"{base}. No active defect would trip at any threshold; evaluated {evaluated}" return ( @@ -402,8 +419,18 @@ def _not_evaluated_reason(would_trip_at: str | None, evaluated: str) -> str: ) -def gate_decision(result: ScanResult, fail_on: Severity | None) -> GateDecision: - """Translate a scan into a pass/fail verdict. A trip is data, not an error.""" +def gate_decision( + result: ScanResult, fail_on: Severity | None, *, fail_on_unanalyzed: bool = False +) -> GateDecision: + """Translate a scan into a pass/fail verdict. A trip is data, not an error. + + Two independent sub-gates compose into one decision: the severity gate (``fail_on``) + and the unanalyzed gate (``fail_on_unanalyzed`` — trips when any file was discovered + but never analysed; benign no-module skips excluded, see ``ScanSummary.unanalyzed``). + Folding the unanalyzed gate in HERE (A4, wardline-7fd0f3a82c) is what lets both + surfaces share it: the CLI exits on ``tripped`` and the MCP gate block serialises the + same decision, so neither can drift. + """ # None SENTINEL: evaluate the unsuppressed gate population when present (secure # default), else the suppressed ``findings`` (trusted ``--trust-suppressions`` / # a directly-constructed ScanResult with no gate_findings). Population selection is @@ -418,7 +445,7 @@ def gate_decision(result: ScanResult, fail_on: Severity | None) -> GateDecision: if honors_suppressions else "unsuppressed (repository baseline/waiver/judged ignored)" ) - if fail_on is None: + if fail_on is None and not fail_on_unanalyzed: # NOT a clean pass — the gate never ran. The verdict says so; would_trip_at names the # worst severity present so the agent's first bare scan is not a false green. return GateDecision( @@ -430,17 +457,34 @@ def gate_decision(result: ScanResult, fail_on: Severity | None) -> GateDecision: evaluated=evaluated, would_trip_at=would_trip_at, ) - tripped = gate_trips(gate_population, fail_on) - sev = fail_on.value - reason = _gate_reason(result, fail_on, tripped=tripped, honors_suppressions=honors_suppressions) + severity_tripped = fail_on is not None and gate_trips(gate_population, fail_on) + unanalyzed_tripped = bool(fail_on_unanalyzed and result.summary.unanalyzed) + tripped = severity_tripped or unanalyzed_tripped + if fail_on is not None: + reason = _gate_reason(result, fail_on, tripped=severity_tripped, honors_suppressions=honors_suppressions) + else: + # Unanalyzed-only gate: the unanalyzed sub-gate evaluated but the severity gate + # never ran — the reason must say so, or a PASSED here is a vacuous severity green. + reason = _not_evaluated_reason(would_trip_at, evaluated, gate="the severity gate") + if fail_on_unanalyzed: + n = result.summary.unanalyzed + prefix = ( + f"{n} file(s) discovered but not analyzed (fail_on_unanalyzed tripped)" + if unanalyzed_tripped + else "0 files unanalyzed (fail_on_unanalyzed passed)" + ) + reason = f"{prefix}; {reason}" return GateDecision( tripped=tripped, - fail_on=sev, + fail_on=fail_on.value if fail_on is not None else None, exit_class=1 if tripped else 0, verdict="FAILED" if tripped else "PASSED", reason=reason, evaluated=evaluated, would_trip_at=would_trip_at, + fail_on_unanalyzed=fail_on_unanalyzed, + severity_tripped=severity_tripped, + unanalyzed_tripped=unanalyzed_tripped, ) diff --git a/src/wardline/mcp/server.py b/src/wardline/mcp/server.py index 3ec75fde..4624cdaf 100644 --- a/src/wardline/mcp/server.py +++ b/src/wardline/mcp/server.py @@ -213,6 +213,8 @@ def _scan( # A bad enum value is agent-actionable — give it the valid set rather than # letting it surface as an opaque generic JSON-RPC -32603. raise ToolError("fail_on must be one of CRITICAL/ERROR/WARN/INFO") from exc + # A4 (wardline-7fd0f3a82c): the CLI's --fail-on-unanalyzed knob, same default (off). + fail_on_unanalyzed = _bool_arg(args, "fail_on_unanalyzed", False) new_since = args.get("new_since") trusted_packs = _trusted_packs_arg(args) cache_dir = _cache_dir_arg(args, root) @@ -253,7 +255,7 @@ def _scan( "unresolved_qualnames": list(wr.unresolved_qualnames), "disabled_reason": wr.disabled_reason, } - decision = gate_decision(result, threshold) + decision = gate_decision(result, threshold, fail_on_unanalyzed=fail_on_unanalyzed) migration_hint = baseline_migration_hint(result, decision, root=path, new_since=new_since) filigree_block = _emit_filigree(result.findings, filigree, scanned_paths=result.scanned_paths) filigree_status = _filigree_emit_status(filigree_block) @@ -349,8 +351,13 @@ def _scan( "gate": { "tripped": decision.tripped, "fail_on": decision.fail_on, + "fail_on_unanalyzed": decision.fail_on_unanalyzed, "exit_class": decision.exit_class, "verdict": decision.verdict, + # Sub-gate attribution: which knob(s) the overall trip came from, so an agent + # never has to parse `reason` to tell a severity trip from an unanalyzed one. + "severity_tripped": decision.severity_tripped, + "unanalyzed_tripped": decision.unanalyzed_tripped, "would_trip_at": decision.would_trip_at, "reason": decision.reason, "evaluated": decision.evaluated, @@ -936,6 +943,14 @@ def _register_tools(self) -> None: "properties": { "path": {"type": "string", "description": "subdir relative to project root"}, "fail_on": {"type": "string", "enum": _SEVERITY_ENUM}, + "fail_on_unanalyzed": { + "type": "boolean", + "description": "Also trip the gate when any file was discovered but could " + "not be analyzed (parse error / too-deep skip / missing source root; benign " + "no-module skips excluded). Default false — same default as the CLI's " + "--fail-on-unanalyzed; summary.unanalyzed always reports the count either " + "way, and gate.unanalyzed_tripped attributes a trip to this knob.", + }, "config": {"type": "string"}, "lang": { "type": "string", diff --git a/tests/docs/test_glossary_vocabulary.py b/tests/docs/test_glossary_vocabulary.py index 7390d73d..71046f01 100644 --- a/tests/docs/test_glossary_vocabulary.py +++ b/tests/docs/test_glossary_vocabulary.py @@ -36,25 +36,25 @@ ("src/wardline/core/run.py", 88, "gate_findings:"), ("src/wardline/core/run.py", 98, "class GateDecision"), ("src/wardline/core/run.py", 107, "verdict: str"), - ("src/wardline/core/run.py", 327, "Baseline(frozenset())"), - ("src/wardline/core/run.py", 337, "def apply_delta_scope"), - ("src/wardline/core/run.py", 361, "active=sum"), - ("src/wardline/core/run.py", 412, "honors_suppressions"), + ("src/wardline/core/run.py", 326, "Baseline(frozenset())"), + ("src/wardline/core/run.py", 354, "def apply_delta_scope"), + ("src/wardline/core/run.py", 378, "active=sum"), + ("src/wardline/core/run.py", 439, "honors_suppressions"), # src/wardline/cli/scan.py — CLI summary line + gate stderr ("src/wardline/cli/scan.py", 415, "suppressed"), ("src/wardline/cli/scan.py", 416, "{s.active} active"), - ("src/wardline/cli/scan.py", 456, "gate: FAILED"), + ("src/wardline/cli/scan.py", 468, "gate: FAILED"), # src/wardline/mcp/server.py — MCP scan summary + gate block - ("src/wardline/mcp/server.py", 334, '"total": result.summary.total'), - ("src/wardline/mcp/server.py", 335, '"active": result.summary.active'), - ("src/wardline/mcp/server.py", 336, '"baselined": result.summary.baselined'), - ("src/wardline/mcp/server.py", 337, '"waived": result.summary.waived'), - ("src/wardline/mcp/server.py", 338, '"judged": result.summary.judged'), - ("src/wardline/mcp/server.py", 343, '"informational": result.summary.informational'), - ("src/wardline/mcp/server.py", 347, '"unanalyzed": result.summary.unanalyzed'), - ("src/wardline/mcp/server.py", 349, '"gate": {'), - ("src/wardline/mcp/server.py", 350, '"tripped": decision.tripped'), - ("src/wardline/mcp/server.py", 353, '"verdict": decision.verdict'), + ("src/wardline/mcp/server.py", 336, '"total": result.summary.total'), + ("src/wardline/mcp/server.py", 337, '"active": result.summary.active'), + ("src/wardline/mcp/server.py", 338, '"baselined": result.summary.baselined'), + ("src/wardline/mcp/server.py", 339, '"waived": result.summary.waived'), + ("src/wardline/mcp/server.py", 340, '"judged": result.summary.judged'), + ("src/wardline/mcp/server.py", 345, '"informational": result.summary.informational'), + ("src/wardline/mcp/server.py", 349, '"unanalyzed": result.summary.unanalyzed'), + ("src/wardline/mcp/server.py", 351, '"gate": {'), + ("src/wardline/mcp/server.py", 352, '"tripped": decision.tripped'), + ("src/wardline/mcp/server.py", 356, '"verdict": decision.verdict'), # src/wardline/core/agent_summary.py — agent-summary JSON keys ("src/wardline/core/agent_summary.py", 134, '"total_findings"'), ("src/wardline/core/agent_summary.py", 135, '"active_defects"'), diff --git a/tests/unit/core/test_cli_mcp_parity.py b/tests/unit/core/test_cli_mcp_parity.py index 45e2cf95..05e664b6 100644 --- a/tests/unit/core/test_cli_mcp_parity.py +++ b/tests/unit/core/test_cli_mcp_parity.py @@ -44,8 +44,11 @@ def test_cli_and_mcp_scan_agree_on_findings_and_gate() -> None: assert mcp["gate"] == { "tripped": cli_gate.tripped, "fail_on": cli_gate.fail_on, + "fail_on_unanalyzed": cli_gate.fail_on_unanalyzed, "exit_class": cli_gate.exit_class, "verdict": cli_gate.verdict, + "severity_tripped": cli_gate.severity_tripped, + "unanalyzed_tripped": cli_gate.unanalyzed_tripped, "would_trip_at": cli_gate.would_trip_at, "reason": cli_gate.reason, "evaluated": cli_gate.evaluated, @@ -127,3 +130,35 @@ def emit(self, findings, *, scanned_paths=()): # active-only) silently diverging the two surfaces' Filigree emission. assert [f.fingerprint for f in cap.seen] == [f.fingerprint for f in cli_result.findings] assert cap.scanned_paths == cli_result.scanned_paths + + +def test_cli_and_mcp_agree_on_fail_on_unanalyzed_gate(tmp_path: Path) -> None: + """A4 (wardline-7fd0f3a82c): the unanalyzed gate must be controllable over BOTH + surfaces and yield the SAME verdict. The CLI side is the REAL Click command; the MCP + side is `_scan` with the new `fail_on_unanalyzed` arg. Fixture: an unparseable file + (discovered but never analysed) and no severity threshold.""" + from click.testing import CliRunner + + from wardline.cli.scan import scan as scan_cmd + + proj = tmp_path / "proj" + proj.mkdir() + (proj / "bad.py").write_text("def f(:\n", encoding="utf-8") # syntax error -> unanalyzed + + for knob, expect_trip in ((True, True), (False, False)): + cli_args = [str(proj), "--output", str(tmp_path / "f.jsonl")] + cli_args.append("--fail-on-unanalyzed" if knob else "--no-fail-on-unanalyzed") + cli = CliRunner().invoke(scan_cmd, cli_args) + mcp = _scan({"fail_on_unanalyzed": knob}, root=proj) + assert cli.exit_code == (1 if expect_trip else 0), cli.output + assert mcp["gate"]["tripped"] is expect_trip + assert mcp["gate"]["exit_class"] == cli.exit_code + assert mcp["gate"]["fail_on_unanalyzed"] is knob + assert mcp["gate"]["unanalyzed_tripped"] is expect_trip + assert mcp["summary"]["unanalyzed"] >= 1 + if expect_trip: + assert mcp["gate"]["verdict"] == "FAILED" + assert "not analyzed" in (mcp["gate"]["reason"] or "") + else: + # Knob off + no threshold: the gate never ran — released behaviour. + assert mcp["gate"]["verdict"] == "NOT_EVALUATED" diff --git a/tests/unit/core/test_run.py b/tests/unit/core/test_run.py index 6b00db65..b943323d 100644 --- a/tests/unit/core/test_run.py +++ b/tests/unit/core/test_run.py @@ -270,7 +270,13 @@ def test_gate_decision_rejects_contradictory_construction() -> None: tripped=False, fail_on="ERROR", exit_class=0, verdict="PASSED", reason="clean", evaluated="unsuppressed" ) GateDecision( - tripped=True, fail_on="ERROR", exit_class=1, verdict="FAILED", reason="1 active", evaluated="unsuppressed" + tripped=True, + fail_on="ERROR", + exit_class=1, + verdict="FAILED", + reason="1 active", + evaluated="unsuppressed", + severity_tripped=True, ) @@ -294,6 +300,108 @@ def test_gate_decision_no_threshold_is_not_evaluated() -> None: assert decision.evaluated is not None +def _unanalyzed_result(unanalyzed: int) -> ScanResult: + # The unanalyzed overlay counts non-gating engine FACTs (file-skipped / missing source + # root), so the severity gate population can be empty while unanalyzed > 0. + return ScanResult( + findings=[], + summary=ScanSummary(unanalyzed, 0, 0, 0, 0, informational=unanalyzed, unanalyzed=unanalyzed), + files_scanned=1, + context=None, + ) + + +def test_gate_decision_fail_on_unanalyzed_trips_without_threshold() -> None: + # A4 (wardline-7fd0f3a82c): the unanalyzed gate is part of the decision itself, not a + # CLI-only exit-code OR — so the MCP surface (no exit code) can express it too. + decision = gate_decision(_unanalyzed_result(2), None, fail_on_unanalyzed=True) + assert decision.tripped is True and decision.exit_class == 1 + assert decision.verdict == "FAILED" + assert decision.fail_on is None + assert decision.fail_on_unanalyzed is True + assert decision.unanalyzed_tripped is True and decision.severity_tripped is False + assert decision.reason is not None and "2" in decision.reason and "not analyzed" in decision.reason + # The severity gate still never ran — the reason must say so (no vacuous severity green). + assert "no --fail-on threshold set" in decision.reason + + +def test_gate_decision_fail_on_unanalyzed_clean_passes_without_threshold() -> None: + # The knob alone makes the gate EVALUATED (it judged the unanalyzed count), but the + # reason must still flag that no severity threshold ran. + decision = gate_decision(_unanalyzed_result(0), None, fail_on_unanalyzed=True) + assert decision.tripped is False and decision.exit_class == 0 + assert decision.verdict == "PASSED" + assert decision.fail_on is None and decision.fail_on_unanalyzed is True + assert decision.unanalyzed_tripped is False and decision.severity_tripped is False + assert decision.reason is not None and "no --fail-on threshold set" in decision.reason + + +def test_gate_decision_fail_on_unanalyzed_composes_with_severity_gate(tmp_path: Path) -> None: + # Both sub-gates configured: a severity-clean tree with an unanalyzed file still FAILS, + # and the reason names BOTH outcomes. + proj, _fp_ = _leaky_proj(tmp_path) + result = run_scan(proj) + decision = gate_decision(result, Severity.ERROR, fail_on_unanalyzed=True) + # The leaky fixture has no unanalyzed files: severity trips, unanalyzed does not. + assert decision.severity_tripped is True and decision.unanalyzed_tripped is False + assert decision.tripped is True and decision.verdict == "FAILED" + clean = gate_decision(_unanalyzed_result(1), Severity.ERROR, fail_on_unanalyzed=True) + assert clean.severity_tripped is False and clean.unanalyzed_tripped is True + assert clean.tripped is True and clean.verdict == "FAILED" + assert clean.fail_on == "ERROR" + assert clean.reason is not None and "not analyzed" in clean.reason + + +def test_gate_decision_default_ignores_unanalyzed() -> None: + # Default (knob off) preserves released behaviour on BOTH surfaces: unanalyzed is + # reported, never gated. + decision = gate_decision(_unanalyzed_result(3), None) + assert decision.tripped is False and decision.verdict == "NOT_EVALUATED" + assert decision.fail_on_unanalyzed is False and decision.unanalyzed_tripped is False + + +def test_gate_decision_unanalyzed_invariants_unconstructible() -> None: + # The sub-trip decomposition is guarded the same way dogfood #2 is: an overall trip + # that no sub-gate explains (or vice versa) must not construct. + with pytest.raises(ValueError, match="severity_tripped|unanalyzed_tripped|tripped"): + # tripped with NEITHER sub-gate tripping. + GateDecision(tripped=True, fail_on="ERROR", exit_class=1, verdict="FAILED", reason="x", evaluated="y") + with pytest.raises(ValueError, match="unanalyzed_tripped"): + # unanalyzed trip without the knob on. + GateDecision( + tripped=True, + fail_on="ERROR", + exit_class=1, + verdict="FAILED", + reason="x", + evaluated="y", + unanalyzed_tripped=True, + ) + with pytest.raises(ValueError, match="severity_tripped"): + # severity trip without a threshold. + GateDecision( + tripped=True, + fail_on=None, + exit_class=1, + verdict="FAILED", + reason="x", + evaluated="y", + fail_on_unanalyzed=True, + severity_tripped=True, + ) + with pytest.raises(ValueError, match="NOT_EVALUATED"): + # The knob alone makes the gate evaluated — NOT_EVALUATED is no longer its shape. + GateDecision( + tripped=False, + fail_on=None, + exit_class=0, + verdict="NOT_EVALUATED", + reason="x", + evaluated="y", + fail_on_unanalyzed=True, + ) + + def _hint(proj: Path, *, new_since=None, trust=False): result = run_scan(proj, new_since=new_since, trust_suppressions=trust) decision = gate_decision(result, Severity.ERROR) @@ -539,12 +647,26 @@ def test_gate_decision_rejects_unknown_fail_on() -> None: # fail_on is always a Severity value; an arbitrary string is an illegal state the # other guards would otherwise let through (it satisfies "reason iff fail_on"). with pytest.raises(ValueError, match="fail_on"): - GateDecision(tripped=True, fail_on="banana", exit_class=1, verdict="FAILED", reason="x", evaluated="y") + GateDecision( + tripped=True, + fail_on="banana", + exit_class=1, + verdict="FAILED", + reason="x", + evaluated="y", + severity_tripped=True, + ) def test_gate_decision_accepts_valid_severity_value() -> None: dec = GateDecision( - tripped=True, fail_on=Severity.ERROR.value, exit_class=1, verdict="FAILED", reason="x", evaluated="y" + tripped=True, + fail_on=Severity.ERROR.value, + exit_class=1, + verdict="FAILED", + reason="x", + evaluated="y", + severity_tripped=True, ) assert dec.fail_on == "ERROR" From 146ea19031e67b5eaadceb5980b35518f3f21afc Mon Sep 17 00:00:00 2001 From: John Morrissey Date: Thu, 11 Jun 2026 22:06:11 +1000 Subject: [PATCH 102/186] =?UTF-8?q?feat(mcp):=20B1+B2=20=E2=80=94=20output?= =?UTF-8?q?Schema/structuredContent,=20annotations+title=20on=20all=2015?= =?UTF-8?q?=20tools;=20protocol=20negotiation=20(closes=20wardline-47ff226?= =?UTF-8?q?ebe,=20wardline-e63204176b)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit B1 (wardline-47ff226ebe): every tool declares outputSchema in tools/list and dual-emits structuredContent + a byte-identical text block on tools/call. isError results never carry structuredContent. Output schemas were derived by tracing every payload branch to its leaves and are pinned by a new conformance suite that validates each tool's live payload against its declared schema. B2 (wardline-e63204176b): standard ToolAnnotations (readOnlyHint / destructiveHint / idempotentHint / openWorldHint) + human title per tool, mapped from the existing capability model; the homegrown capabilities key is still emitted. Hints describe the integration-free baseline posture; ToolPolicy + _effective_tool_capabilities stay the enforcement authority. Also: - MCP protocol version negotiation: 2025-06-18 / 2025-03-26 / 2024-11-05 (structured output and annotations only exist in the newer revisions; the server previously hard-pinned 2024-11-05). - Per-tool declarations colocated next to their handlers; _register_tools shrinks to binding declarations to injected-client lambdas (the schema half of wardline-80e457bc41; the federation-status projector half stays open). - Adversarial-review fixes: Filigree promote issue_id type-narrowed at the wire boundary (file_finding/scan_file_findings schema violation); store-blob tier/callee fields guarded in explain_taint; scan_file_findings emit disabled_reason now uses the shared 401/403-vs-5xx ladder and the no-filer branch names the real cause. Suite: 3806 passed; ruff + mypy clean; live stdio smoke against a fresh subprocess green (negotiation, 15-tool surface, dual emission, schema validation, isError discipline). Co-Authored-By: Claude Fable 5 --- CHANGELOG.md | 33 + .../reference/finding-lifecycle-vocabulary.md | 30 +- src/wardline/core/explain.py | 22 +- src/wardline/core/filigree_issue.py | 6 +- src/wardline/core/run.py | 4 +- src/wardline/core/scan_file_workflow.py | 16 +- src/wardline/mcp/protocol.py | 11 +- src/wardline/mcp/server.py | 3385 ++++++++++++++--- src/wardline/mcp/tooling.py | 13 + tests/conformance/test_mcp_handshake.py | 6 +- .../conformance/test_mcp_structured_output.py | 384 ++ tests/docs/test_glossary_vocabulary.py | 22 +- tests/golden/identity/rust/_capture.py | 2 +- tests/unit/core/test_filigree_issue.py | 9 + tests/unit/core/test_scan_file_workflow.py | 32 + tests/unit/mcp/test_protocol.py | 33 +- tests/unit/mcp/test_server_loomweave_write.py | 40 + tests/unit/mcp/test_server_structure.py | 17 + tests/unit/rust/test_index.py | 2 +- .../taint/test_attribute_write_flow.py | 3 +- 20 files changed, 3576 insertions(+), 494 deletions(-) create mode 100644 tests/conformance/test_mcp_structured_output.py diff --git a/CHANGELOG.md b/CHANGELOG.md index 0d368a72..454bf333 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -8,6 +8,39 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] ### Added +- **MCP structured tool output on all 15 tools** — every tool now declares an + `outputSchema` in `tools/list` and returns `structuredContent` alongside the + (byte-identical) text block on `tools/call`, so MCP-spec clients consume and + validate results without parsing JSON out of a text blob. Tool-execution + errors stay `isError` results and never carry `structuredContent`. The + server now negotiates the MCP protocol revision (`2025-06-18` / + `2025-03-26` / `2024-11-05`; previously hard-pinned to `2024-11-05`). + Per-tool declarations (schemas, annotations, capabilities) moved out of + `_register_tools` to module level next to their handlers + (`wardline-47ff226ebe`, MCP-primary B1; declaration colocation from + `wardline-80e457bc41`). +- **Standard MCP tool `annotations` + `title`** — each tool's `tools/list` + entry now carries a human `title` and the standard `ToolAnnotations` hints + (`readOnlyHint`, `destructiveHint`, `idempotentHint`, `openWorldHint`) + derived from the existing capability model, so standard MCP clients see the + read-only/destructive signal wardline already computes. The homegrown + `capabilities` key is still emitted for existing consumers; hints describe + the integration-free baseline posture and `ToolPolicy` remains the + enforcement authority (`wardline-e63204176b`, MCP-primary B2). + +### Fixed +- **Non-string `issue_id` from Filigree promote is normalized to `null`** — + the promote response is type-narrowed at the wire boundary, so a skewed + 2xx body can no longer leak a non-string `issue_id` into `file_finding` / + `scan_file_findings` payloads (violating their published output schemas). +- **Type-skewed Loomweave store blobs coerce to `null` in `explain_taint`** — + `tier_in`/`tier_out`/callee qualnames read from a store blob now get the + same isinstance guards as the adjacent fingerprint/path fields. +- **`scan_file_findings` federation honesty** — the emit block's + `disabled_reason` now uses the shared 401/403-vs-5xx-vs-transport ladder + instead of a flat `filigree unreachable`, and the no-Filigree-URL branch no + longer misattributes the identity-attach skip to a promote that never ran. + - **MCP `scan` tool `fail_on_unanalyzed` argument** — the CLI's `--fail-on-unanalyzed` knob over the MCP surface (default off, same as the CLI), so gate semantics are fully controllable on the primary surface. The diff --git a/docs/reference/finding-lifecycle-vocabulary.md b/docs/reference/finding-lifecycle-vocabulary.md index e45133e1..72430d5c 100644 --- a/docs/reference/finding-lifecycle-vocabulary.md +++ b/docs/reference/finding-lifecycle-vocabulary.md @@ -73,7 +73,7 @@ consistently, on every surface: | Enum | `src/wardline/core/finding.py:72` | `SuppressionState.ACTIVE = "active"` | | Summary field | `src/wardline/core/run.py:51`, built at `src/wardline/core/run.py:378` | `ScanSummary.active` | | CLI summary line | `src/wardline/cli/scan.py:416` | `… {s.active} active` | -| MCP scan response | `src/wardline/mcp/server.py:337` | `summary.active` | +| MCP scan response | `src/wardline/mcp/server.py:775` | `summary.active` | | Agent-summary JSON | `src/wardline/core/agent_summary.py:135` | `summary.active_defects` | | `wardline:loop` prompt | `src/wardline/mcp/prompts.py:13` | "Read `summary.active`" | @@ -101,8 +101,8 @@ So `active + baselined + waived + judged + informational == total` (`src/wardline/core/run.py:50` for `total: int`). `unanalyzed` (`src/wardline/core/run.py:69`) is an **overlay** — a subset of `informational` that surfaces a silent under-scan — and is deliberately **not** a partition member. -The MCP `summary` block exposes `informational` (`src/wardline/mcp/server.py:345`) -and `unanalyzed` (`src/wardline/mcp/server.py:349`); the agent-summary block mirrors +The MCP `summary` block exposes `informational` (`src/wardline/mcp/server.py:783`) +and `unanalyzed` (`src/wardline/mcp/server.py:787`); the agent-summary block mirrors both (`src/wardline/core/agent_summary.py:146`, `src/wardline/core/agent_summary.py:147`). ## Emitted-active vs the gate population @@ -123,7 +123,7 @@ differ on purpose: a suppression keyed to its own new defect. `gate_decision` evaluates `gate_findings` when present, else falls back to `findings` (the trusted `--trust-suppressions` / directly-constructed path), selected at - `src/wardline/core/run.py:439` (`honors_suppressions`). + `src/wardline/core/run.py:437` (`honors_suppressions`). This is why **`summary.active: 0` can co-exist with `gate.tripped: true`**: every defect was suppressed by a committed baseline (so emitted-active is 0), but those @@ -150,10 +150,10 @@ trips when any file was discovered but never analysed; benign no-module skips excluded). `severity_tripped` / `unanalyzed_tripped` attribute an overall `tripped` to its sub-gate(s) so no consumer has to parse `reason`. -The MCP `scan` gate block exposes `gate.tripped` (`src/wardline/mcp/server.py:352`), -`gate.fail_on_unanalyzed`, `gate.verdict` (`src/wardline/mcp/server.py:356`), +The MCP `scan` gate block exposes `gate.tripped` (`src/wardline/mcp/server.py:790`), +`gate.fail_on_unanalyzed`, `gate.verdict` (`src/wardline/mcp/server.py:794`), `gate.severity_tripped`, `gate.unanalyzed_tripped`, `would_trip_at`, `reason`, -`evaluated`, and `migration_hint`, opened at `src/wardline/mcp/server.py:351` +`evaluated`, and `migration_hint`, opened at `src/wardline/mcp/server.py:789` (`"gate": {`); the agent-summary mirrors them at `src/wardline/core/agent_summary.py:150` (`tripped`) and `src/wardline/core/agent_summary.py:153` (`verdict`). The CLI prints @@ -186,16 +186,16 @@ How each concept appears on each surface: | Concept | CLI summary text | `ScanSummary` field | MCP `summary` key | Agent-summary key | Filigree store | | --- | --- | --- | --- | --- | --- | -| every finding | `N finding(s)` | `total` (`run.py:50`) | `total` (`server.py:336`) | `total_findings` (`agent_summary.py:134`) | one finding per wire entry | -| live defect | `N active` (`scan.py:416`) | `active` (`run.py:51,361`) | `active` (`server.py:337`) | `active_defects` (`agent_summary.py:135`) | no `suppression_state` key (`finding.py:285`) | +| every finding | `N finding(s)` | `total` (`run.py:50`) | `total` (`server.py:774`) | `total_findings` (`agent_summary.py:134`) | one finding per wire entry | +| live defect | `N active` (`scan.py:416`) | `active` (`run.py:51,361`) | `active` (`server.py:775`) | `active_defects` (`agent_summary.py:135`) | no `suppression_state` key (`finding.py:285`) | | suppressed (sum) | `N suppressed` (`scan.py:415`) | `baselined+waived+judged` | the three keys | `suppressed_findings` (`agent_summary.py:136`) | `metadata.wardline.suppression_state` (`finding.py:285`) | -| baselined | `N baseline` | `baselined` (`run.py:53`) | `baselined` (`server.py:338`) | `baselined` (`agent_summary.py:138`) | `suppression_state: "baselined"` | -| waived | `N waiver` | `waived` (`run.py:54`) | `waived` (`server.py:339`) | `waived` (`agent_summary.py:139`) | `suppression_state: "waived"` | -| judged | `N judged` | `judged` (`run.py:55`) | `judged` (`server.py:340`) | `judged` (`agent_summary.py:140`) | `suppression_state: "judged"` | -| informational (summary) | (the remainder of `total`) | `informational` (`run.py:61`) | `informational` (`server.py:345`) | `informational` (`agent_summary.py:146`) | facts/metrics | +| baselined | `N baseline` | `baselined` (`run.py:53`) | `baselined` (`server.py:776`) | `baselined` (`agent_summary.py:138`) | `suppression_state: "baselined"` | +| waived | `N waiver` | `waived` (`run.py:54`) | `waived` (`server.py:777`) | `waived` (`agent_summary.py:139`) | `suppression_state: "waived"` | +| judged | `N judged` | `judged` (`run.py:55`) | `judged` (`server.py:778`) | `judged` (`agent_summary.py:140`) | `suppression_state: "judged"` | +| informational (summary) | (the remainder of `total`) | `informational` (`run.py:61`) | `informational` (`server.py:783`) | `informational` (`agent_summary.py:146`) | facts/metrics | | informational (display) | n/a | n/a | n/a | `informational` display array (`agent_summary.py:171`) — non-defect, non-engine-fact findings (metrics, classifications, suggestions, non-engine facts); excludes `engine_facts` which has its own display slot | facts/metrics | -| under-scan | `N file(s) could not be analyzed` | `unanalyzed` (`run.py:69`) | `unanalyzed` (`server.py:349`) | `unanalyzed` (`agent_summary.py:147`) | `WLN-ENGINE-*` facts | -| gate verdict | exit code + `--fail-on` | (`gate_findings`, `run.py:88`; `GateDecision`, `run.py:98`, `verdict` `run.py:107`) | `gate.tripped` (`server.py:352`), `gate.verdict` (`server.py:356`) | `gate.tripped` (`agent_summary.py:150`), `gate.verdict` (`agent_summary.py:153`) | not emitted to Filigree | +| under-scan | `N file(s) could not be analyzed` | `unanalyzed` (`run.py:69`) | `unanalyzed` (`server.py:787`) | `unanalyzed` (`agent_summary.py:147`) | `WLN-ENGINE-*` facts | +| gate verdict | exit code + `--fail-on` | (`gate_findings`, `run.py:88`; `GateDecision`, `run.py:98`, `verdict` `run.py:107`) | `gate.tripped` (`server.py:790`), `gate.verdict` (`server.py:794`) | `gate.tripped` (`agent_summary.py:150`), `gate.verdict` (`agent_summary.py:153`) | not emitted to Filigree | The unsuppressed gate population is built from `Baseline(frozenset())` (`src/wardline/core/run.py:326`). diff --git a/src/wardline/core/explain.py b/src/wardline/core/explain.py index 605a058b..df751453 100644 --- a/src/wardline/core/explain.py +++ b/src/wardline/core/explain.py @@ -142,6 +142,16 @@ def _is_fresh(view: Any) -> bool: return stamped is not None and stamped == view.current_content_hash +def _opt_str(value: Any) -> str | None: + """Type-narrow an untrusted store-blob field to string-or-None. + + The blob is external input (hand-editable, version-skewable); the adjacent + fingerprint/rule_id/path/line fields are already isinstance-guarded — taint tiers + and callee qualnames get the same treatment so a type-skewed blob cannot put a + non-string into a payload published as string|null in the MCP outputSchema.""" + return value if isinstance(value, str) else None + + def _callee_leaf(callee_qualname: str | None) -> str | None: """The blob stores the resolved callee QUALNAME; SP8's immediate_tainted_callee is the bare trailing name. Project back for surface parity with the SP8 shape.""" @@ -208,7 +218,7 @@ def _explanation_from_blob( first = _select_blob_finding(finding_rows, fingerprint=fingerprint, path=path, line=line, rule_id=rule_id) if first is None: return None - callee_q = taint.get("contributing_callee_qualname") + callee_q = _opt_str(taint.get("contributing_callee_qualname")) stored_fingerprint = first.get("fingerprint") stored_rule_id = first.get("rule_id") stored_path = first.get("path") @@ -220,8 +230,8 @@ def _explanation_from_blob( sink_qualname=qualname if isinstance(qualname, str) else None, path=stored_path if isinstance(stored_path, str) else "", line=stored_line if isinstance(stored_line, int) else None, - tier_in=taint.get("actual_return"), - tier_out=taint.get("declared_return"), + tier_in=_opt_str(taint.get("actual_return")), + tier_out=_opt_str(taint.get("declared_return")), immediate_tainted_callee=_callee_leaf(callee_q), source_boundary_qualname=callee_q, resolved_call_count=int(taint.get("resolved_call_count", 0) or 0), @@ -275,12 +285,12 @@ def explain_chain( return TaintChain(hops=hops, truncated_at=current) blob = view.wardline_json or {} taint = blob.get("taint", {}) - next_q = taint.get("contributing_callee_qualname") + next_q = _opt_str(taint.get("contributing_callee_qualname")) hops.append( ChainHop( qualname=current, - tier_in=taint.get("actual_return"), - tier_out=taint.get("declared_return"), + tier_in=_opt_str(taint.get("actual_return")), + tier_out=_opt_str(taint.get("declared_return")), contributing_callee_qualname=next_q, ) ) diff --git a/src/wardline/core/filigree_issue.py b/src/wardline/core/filigree_issue.py index b07c2bf3..dd091e0d 100644 --- a/src/wardline/core/filigree_issue.py +++ b/src/wardline/core/filigree_issue.py @@ -193,7 +193,11 @@ def file( payload = json.loads(resp.body) if resp.body else {} except json.JSONDecodeError: payload = {} - issue_id = payload.get("issue_id") if isinstance(payload, dict) else None + # Type-narrow at the wire boundary like the emit path does (_safe_int): a 2xx + # body carrying a non-string issue_id must not flow verbatim into tool payloads + # that publish issue_id as string|null in their MCP outputSchema. + raw_issue_id = payload.get("issue_id") if isinstance(payload, dict) else None + issue_id = raw_issue_id if isinstance(raw_issue_id, str) else None created = bool(payload.get("created")) if isinstance(payload, dict) else False return FileResult(reachable=True, issue_id=issue_id, created=created) diff --git a/src/wardline/core/run.py b/src/wardline/core/run.py index 0265b35d..d24e9092 100644 --- a/src/wardline/core/run.py +++ b/src/wardline/core/run.py @@ -419,9 +419,7 @@ def _not_evaluated_reason(would_trip_at: str | None, evaluated: str, *, gate: st ) -def gate_decision( - result: ScanResult, fail_on: Severity | None, *, fail_on_unanalyzed: bool = False -) -> GateDecision: +def gate_decision(result: ScanResult, fail_on: Severity | None, *, fail_on_unanalyzed: bool = False) -> GateDecision: """Translate a scan into a pass/fail verdict. A trip is data, not an error. Two independent sub-gates compose into one decision: the severity gate (``fail_on``) diff --git a/src/wardline/core/scan_file_workflow.py b/src/wardline/core/scan_file_workflow.py index 50743301..100e7624 100644 --- a/src/wardline/core/scan_file_workflow.py +++ b/src/wardline/core/scan_file_workflow.py @@ -7,7 +7,7 @@ from wardline.core.errors import WardlineError from wardline.core.explain import TaintExplanation, explanation_from_context -from wardline.core.filigree_emit import EmitResult +from wardline.core.filigree_emit import EmitResult, filigree_disabled_reason from wardline.core.filigree_issue import ( FileResult, IdentityAttachResult, @@ -63,7 +63,14 @@ def _emit_to_dict(result: EmitResult | None, *, configured: bool) -> dict[str, A "updated": result.updated, "failed": result.failed, "warnings": list(result.warnings), - "disabled_reason": None if result.reachable else "filigree unreachable", + # Delegate to the shared 401/403-vs-5xx-vs-transport ladder (dogfood #5) instead + # of flattening every soft failure to "filigree unreachable". + "disabled_reason": filigree_disabled_reason( + reachable=result.reachable, + status=result.status, + token_sent=result.token_sent, + url=result.url, + ), } @@ -180,7 +187,10 @@ def scan_file_findings( else: identity_result = IdentityAttachResult.skipped("finding has no qualname") else: - identity_result = IdentityAttachResult.not_attempted("no issue_id from Filigree promote") + # No promote was ever attempted on this branch — name the actual cause, + # matching _file_to_dict's configured=False wording (the old "no issue_id + # from Filigree promote" misattributed the failure). + identity_result = IdentityAttachResult.not_attempted("no Filigree URL configured") item = _finding_base(finding, explanation) item["promotion"] = _file_to_dict(file_result, selected=selected_here, configured=filigree_filer is not None) item["identity_attach"] = identity_attach_result_to_json(identity_result) diff --git a/src/wardline/mcp/protocol.py b/src/wardline/mcp/protocol.py index 065180ea..fba3cc26 100644 --- a/src/wardline/mcp/protocol.py +++ b/src/wardline/mcp/protocol.py @@ -11,7 +11,10 @@ from collections.abc import Callable from typing import Any, TextIO -PROTOCOL_VERSION = "2024-11-05" # MCP protocol revision this server speaks +# MCP protocol revisions this server speaks, newest first. structuredContent/outputSchema +# exist only in 2025-06-18; tool annotations/title arrived in 2025-03-26. +SUPPORTED_PROTOCOL_VERSIONS = ("2025-06-18", "2025-03-26", "2024-11-05") +PROTOCOL_VERSION = "2025-06-18" # the latest MCP protocol revision this server speaks Handler = Callable[[dict[str, Any]], Any] @@ -46,8 +49,12 @@ def register(self, method: str, handler: Handler) -> None: self._handlers[method] = handler def _initialize(self, params: dict[str, Any]) -> dict[str, Any]: + # Spec negotiation: echo the client's requested revision when we support it, + # otherwise answer with the latest revision we speak. + requested = params.get("protocolVersion") + version = requested if requested in SUPPORTED_PROTOCOL_VERSIONS else PROTOCOL_VERSION return { - "protocolVersion": PROTOCOL_VERSION, + "protocolVersion": version, "capabilities": self.capabilities, "serverInfo": {"name": self._name, "version": self._version}, } diff --git a/src/wardline/mcp/server.py b/src/wardline/mcp/server.py index 4624cdaf..a0806947 100644 --- a/src/wardline/mcp/server.py +++ b/src/wardline/mcp/server.py @@ -39,6 +39,20 @@ from wardline.mcp.tooling import require as _require from wardline.mcp.tooling import resolve_under_root as _resolve_under_root +# Gate thresholds are the four defect severities. Severity also defines NONE +# (the "facts carry no defect severity" sentinel), deliberately excluded here: +# fail_on=NONE is not a meaningful gate threshold. +_SEVERITY_ENUM = ["CRITICAL", "ERROR", "WARN", "INFO"] + +# Default ceiling on the number of active-defect provenances inlined by `explain: true` +# on the MCP `scan`. Bounds the one-shot payload (the dogfood report hit 56,820 chars on +# one line over a whole repo); an explicit `max_findings` tightens it further. +_EXPLAIN_DEFAULT_CAP = 10 +# The bounded-default page size for `scan` (weft-439d09fc8d). A bare scan returns at most +# this many finding bodies so an agent's first natural call cannot overflow its context; +# full=true lifts the cap and offset pages through the rest. +_DEFAULT_MAX_FINDINGS = 25 + def _emit_filigree( findings: list[Finding], filigree: Any, *, scanned_paths: tuple[str, ...] = () @@ -137,6 +151,117 @@ def _file_finding(args: dict[str, Any], root: Path, filer: Any, loomweave: Any = return payload +_FILE_FINDING_OUTPUT_SCHEMA: dict[str, Any] = { + "type": "object", + "description": "Success payload of the file_finding tool: the outcome of promoting ONE finding (by fingerprint) " + "into a tracked Filigree issue, fail-soft on reachability.", + "properties": { + "reachable": { + "type": "boolean", + "description": "Whether Filigree's promote route was reachable. False on transport failure, 5xx outage, " + "or 401/403 auth refusal (all soft).", + }, + "issue_id": { + "type": ["string", "null"], + "description": "The Filigree issue id the fingerprint was promoted into; null when unreachable or the " + "fingerprint was not found.", + }, + "created": { + "type": "boolean", + "description": "True when the promote created a NEW issue (vs returning an existing one).", + }, + "not_found": { + "type": "boolean", + "description": "True when Filigree was reachable but the fingerprint is unknown to it (404 — emit " + "findings to Filigree first).", + }, + "fingerprint": {"type": "string", "description": "The fingerprint that was filed (echoed from the request)."}, + "disabled_reason": { + "type": ["string", "null"], + "description": "Why enrichment was unavailable (e.g. 'filigree unreachable', 'filigree 503'); null on " + "success.", + }, + "identity_attach": { + "type": "object", + "description": "Present only when attach_loomweave_identity=true was requested: the outcome of binding " + "the finding's Loomweave entity identity to the filed issue.", + "properties": { + "attempted": { + "type": "boolean", + "description": "Whether an identity attach was attempted at all (false when there is no issue_id " + "or no Loomweave URL configured).", + }, + "attached": { + "type": "boolean", + "description": "Whether the entity association was successfully attached to the Filigree issue.", + }, + "entity_id": { + "type": ["string", "null"], + "description": "The entity identifier used for the binding — a 'loomweave:eid:...' SEI or a " + "legacy '{plugin}:function:{qualname}' locator.", + }, + "content_hash": { + "type": ["string", "null"], + "description": "The entity content hash captured at attach time (drift-detection anchor); null " + "when unresolved.", + }, + "binding_kind": { + "type": ["string", "null"], + "enum": ["sei", "locator", None], + "description": "Whether the binding used a rename-stable SEI or a legacy locator; null when no " + "binding was attempted.", + }, + "reason": { + "type": ["string", "null"], + "description": "Human-readable reason when not attempted or skipped; null on success.", + }, + }, + "required": ["attempted", "attached", "entity_id", "content_hash", "binding_kind", "reason"], + "additionalProperties": False, + }, + }, + "required": ["reachable", "issue_id", "created", "not_found", "fingerprint", "disabled_reason"], + "additionalProperties": False, +} + + +_FILE_FINDING_TOOL: dict[str, Any] = { + "name": "file_finding", + "title": "File finding as Filigree issue", + "description": "File ONE finding (by `fingerprint`) into a tracked Filigree issue and " + "return its `issue_id`. Idempotent (re-filing returns the same issue). Emit findings " + "to Filigree first (scan with a configured Filigree URL) so the fingerprint is known; " + "a `not_found: true` result means it isn't. Reconciliation (close-on-fixed / " + "reopen-on-regress) happens automatically on later scans. Fail-soft.", + "input_schema": { + "type": "object", + "required": ["fingerprint"], + "properties": { + "fingerprint": {"type": "string"}, + "priority": {"type": "string", "description": "Filigree priority, e.g. P2"}, + "labels": {"type": "array", "items": {"type": "string"}}, + "attach_loomweave_identity": { + "type": "boolean", + "description": ( + "Opt in to resolving the finding qualname through Loomweave and attaching " + "a Filigree entity association." + ), + }, + "config": {"type": "string"}, + }, + }, + "output_schema": _FILE_FINDING_OUTPUT_SCHEMA, + "annotations": { + "title": "File finding as Filigree issue", + "readOnlyHint": False, + "destructiveHint": False, + "idempotentHint": True, + "openWorldHint": True, + }, + "capabilities": frozenset({ToolCapability.READ, ToolCapability.WRITE, ToolCapability.NETWORK}), +} + + def _scan_file_findings( args: dict[str, Any], root: Path, @@ -173,6 +298,319 @@ def _scan_file_findings( ) +_SCAN_FILE_FINDINGS_OUTPUT_SCHEMA: dict[str, Any] = { + "type": "object", + "description": "Success payload of the scan_file_findings tool: one-shot scan -> (optionally) emit findings to " + "Filigree -> (optionally) promote selected active defects into tracked issues.", + "properties": { + "mode": { + "type": "string", + "enum": ["dry_run", "all_active", "fingerprints"], + "description": "Selection mode that ran: dry_run (nothing promoted), all_active (every active defect " + "selected), or fingerprints (explicit selection).", + }, + "files_scanned": {"type": "integer", "description": "Number of files the scan analyzed."}, + "summary": { + "type": "object", + "description": "Finding counts by suppression class for the whole scan.", + "properties": { + "total": {"type": "integer", "description": "Every finding (defects + facts/metrics)."}, + "active": {"type": "integer", "description": "Non-suppressed defects."}, + "baselined": {"type": "integer", "description": "Defects suppressed by the baseline."}, + "waived": {"type": "integer", "description": "Defects suppressed by waivers."}, + "judged": {"type": "integer", "description": "Defects suppressed by judge FALSE_POSITIVE records."}, + "informational": {"type": "integer", "description": "Informational (non-gating) findings."}, + "unanalyzed": { + "type": "integer", + "description": "Files discovered but never analyzed (benign no-module skips excluded).", + }, + }, + "required": ["total", "active", "baselined", "waived", "judged", "informational", "unanalyzed"], + "additionalProperties": False, + }, + "gate": { + "type": "object", + "description": "The pass/fail gate decision for this scan (a trip is data, not an error).", + "properties": { + "tripped": {"type": "boolean", "description": "Whether the gate tripped."}, + "fail_on": { + "type": ["string", "null"], + "description": "The severity threshold the gate evaluated (e.g. 'ERROR'); null when no threshold " + "was given.", + }, + "exit_class": { + "type": "integer", + "description": "CLI-equivalent exit class: 0 clean, 1 gate tripped (2 is reserved for tool errors " + "and never appears here).", + }, + "verdict": { + "type": "string", + "enum": ["NOT_EVALUATED", "PASSED", "FAILED"], + "description": "NOT_EVALUATED = no threshold ran; PASSED/FAILED = a threshold ran. Never reads a " + "bare scan as a clean pass.", + }, + "would_trip_at": { + "type": ["string", "null"], + "description": "Highest severity at which the gate WOULD trip on the evaluated population; null " + "when nothing would trip.", + }, + }, + "required": ["tripped", "fail_on", "exit_class", "verdict", "would_trip_at"], + "additionalProperties": False, + }, + "filigree_emit": { + "type": "object", + "description": "Outcome of bulk-emitting scan findings to Filigree (runs only when findings were selected " + "and an emitter is configured).", + "properties": { + "configured": { + "type": "boolean", + "description": "Whether a Filigree emitter is configured for this server.", + }, + "reachable": { + "type": ["boolean", "null"], + "description": "Whether Filigree was reachable for the emit; null when no emit was attempted.", + }, + "created": {"type": "integer", "description": "Findings newly created in Filigree."}, + "updated": {"type": "integer", "description": "Findings updated in Filigree."}, + "failed": {"type": "integer", "description": "Findings Filigree rejected."}, + "warnings": {"type": "array", "items": {"type": "string"}, "description": "Non-fatal emit warnings."}, + "disabled_reason": { + "type": ["string", "null"], + "description": "Why the emit failed soft — the discriminated 401/403-vs-5xx-vs-transport " + "ladder ('not configured', 'filigree rejected the token (401)...', 'filigree unreachable'). " + "null means success OR no emit was attempted (dry-run / nothing selected) — read `reachable` " + "to tell them apart (null = no attempt).", + }, + }, + "required": ["configured", "reachable", "created", "updated", "failed", "warnings", "disabled_reason"], + "additionalProperties": False, + }, + "active_defects": { + "type": "array", + "description": "Every active (non-suppressed) defect in the scan, each with its per-finding promotion and " + "identity-attach outcome.", + "items": { + "type": "object", + "properties": { + "fingerprint": { + "type": "string", + "description": "Stable finding fingerprint (the promotion join key).", + }, + "rule_id": {"type": "string", "description": "Rule that produced the finding (e.g. PY-WL-101)."}, + "severity": { + "type": "string", + "enum": ["CRITICAL", "ERROR", "WARN", "INFO", "NONE"], + "description": "Finding severity.", + }, + "message": {"type": "string", "description": "Human-readable finding message."}, + "qualname": { + "type": ["string", "null"], + "description": "Dotted module-qualified name of the enclosing callable; null when the finding " + "has no callable anchor.", + }, + "path": {"type": "string", "description": "Repo-relative file path of the finding."}, + "line": {"type": ["integer", "null"], "description": "1-based start line; null when unknown."}, + "explanation": { + "type": "object", + "description": "One-hop taint provenance slice. Present only when the finding has a qualname " + "AND the scan produced an analysis context.", + "properties": { + "tier_in": { + "type": ["string", "null"], + "description": "Actual (untrusted) trust tier arriving at the sink.", + }, + "tier_out": { + "type": ["string", "null"], + "description": "Trust tier the sink declares it returns.", + }, + "immediate_tainted_callee": { + "type": ["string", "null"], + "description": "The directly-called function that contributed the taint, if resolved.", + }, + "source_boundary_qualname": { + "type": ["string", "null"], + "description": "Qualname of the boundary function the taint originated from (one hop " + "only).", + }, + "resolved_call_count": { + "type": "integer", + "description": "Calls inside the function the analyzer resolved.", + }, + "unresolved_call_count": { + "type": "integer", + "description": "Calls the analyzer could not resolve.", + }, + }, + "required": [ + "tier_in", + "tier_out", + "immediate_tainted_callee", + "source_boundary_qualname", + "resolved_call_count", + "unresolved_call_count", + ], + "additionalProperties": False, + }, + "promotion": { + "type": "object", + "description": "Per-finding Filigree promote outcome for this defect.", + "properties": { + "selected": { + "type": "boolean", + "description": "Whether this finding was in the selection set.", + }, + "attempted": { + "type": "boolean", + "description": "Whether a promote was actually attempted (false in dry_run, when " + "unselected, or when no filer is configured).", + }, + "reachable": { + "type": ["boolean", "null"], + "description": "Whether Filigree was reachable for the promote; null when not " + "attempted.", + }, + "issue_id": { + "type": ["string", "null"], + "description": "The Filigree issue id; null when not attempted, unreachable, or not " + "found.", + }, + "created": {"type": "boolean", "description": "True when the promote created a NEW issue."}, + "not_found": { + "type": "boolean", + "description": "True when Filigree was reachable but the fingerprint is unknown to it " + "(404).", + }, + "disabled_reason": { + "type": ["string", "null"], + "description": "Why the promote did not happen or failed soft; null on success.", + }, + }, + "required": [ + "selected", + "attempted", + "reachable", + "issue_id", + "created", + "not_found", + "disabled_reason", + ], + "additionalProperties": False, + }, + "identity_attach": { + "type": "object", + "description": "Outcome of binding the finding's Loomweave entity identity to the promoted " + "issue.", + "properties": { + "attempted": { + "type": "boolean", + "description": "Whether an identity attach was attempted at all.", + }, + "attached": { + "type": "boolean", + "description": "Whether the entity association was successfully attached.", + }, + "entity_id": { + "type": ["string", "null"], + "description": "The entity identifier used — a 'loomweave:eid:...' SEI or a legacy " + "locator.", + }, + "content_hash": { + "type": ["string", "null"], + "description": "Entity content hash captured at attach time; null when unresolved.", + }, + "binding_kind": { + "type": ["string", "null"], + "enum": ["sei", "locator", None], + "description": "Whether the binding used a rename-stable SEI or a legacy locator; " + "null when no binding was attempted.", + }, + "reason": { + "type": ["string", "null"], + "description": "Why the attach was not attempted or was skipped; null on success.", + }, + }, + "required": ["attempted", "attached", "entity_id", "content_hash", "binding_kind", "reason"], + "additionalProperties": False, + }, + }, + "required": [ + "fingerprint", + "rule_id", + "severity", + "message", + "qualname", + "path", + "line", + "promotion", + "identity_attach", + ], + "additionalProperties": False, + }, + }, + "selected_count": { + "type": "integer", + "description": "How many selected fingerprints matched known active defects.", + }, + "unknown_fingerprints": { + "type": "array", + "items": {"type": "string"}, + "description": "Explicitly-requested fingerprints that are not among the scan's active defects.", + }, + }, + "required": [ + "mode", + "files_scanned", + "summary", + "gate", + "filigree_emit", + "active_defects", + "selected_count", + "unknown_fingerprints", + ], + "additionalProperties": False, +} + + +_SCAN_FILE_FINDINGS_TOOL: dict[str, Any] = { + "name": "scan_file_findings", + "title": "Scan and file findings", + "description": "One-shot agent workflow: run a scan, list active defects first with " + "inline explanation summaries, optionally emit to Filigree, promote selected " + "fingerprints or all active defects, and attach Loomweave identity when available. " + "Defaults to dry-run unless fingerprints or all_active are supplied.", + "input_schema": { + "type": "object", + "properties": { + "path": {"type": "string", "description": "subdir relative to project root"}, + "fail_on": {"type": "string", "enum": _SEVERITY_ENUM}, + "config": {"type": "string"}, + "cache_dir": { + "type": "string", + "description": "subdir relative to project root for summary cache", + }, + "fingerprints": {"type": "array", "items": {"type": "string"}}, + "all_active": {"type": "boolean"}, + "dry_run": {"type": "boolean"}, + "priority": {"type": "string", "description": "Filigree priority, e.g. P2"}, + "labels": {"type": "array", "items": {"type": "string"}}, + "trust_packs": {"type": "array", "items": {"type": "string"}}, + "trust_local_packs": {"type": "boolean"}, + "strict_defaults": {"type": "boolean"}, + }, + }, + "output_schema": _SCAN_FILE_FINDINGS_OUTPUT_SCHEMA, + "annotations": { + "title": "Scan and file findings", + "readOnlyHint": False, + "destructiveHint": False, + "idempotentHint": True, + "openWorldHint": True, + }, + "capabilities": frozenset({ToolCapability.READ, ToolCapability.WRITE, ToolCapability.NETWORK}), +} + + def _trusted_packs_arg(args: dict[str, Any]) -> tuple[str, ...]: trusted_packs_raw = args.get("trust_packs") or [] if not isinstance(trusted_packs_raw, list) or not all(isinstance(p, str) for p in trusted_packs_raw): @@ -381,6 +819,801 @@ def _scan( return response +_SCAN_OUTPUT_SCHEMA: dict[str, Any] = { + "type": "object", + "description": "Success payload of the wardline MCP `scan` tool (the dict _scan returns, served verbatim as " + "structuredContent).", + "properties": { + "files_scanned": {"type": "integer", "description": "Number of files discovered and handed to the analyzer."}, + "summary": { + "type": "object", + "description": "Whole-project finding counts. active+baselined+waived+judged+informational == total; " + "unanalyzed is an overlay, not a partition member.", + "properties": { + "total": {"type": "integer", "description": "Every finding (defects + facts/metrics)."}, + "active": {"type": "integer", "description": "Non-suppressed defects."}, + "baselined": {"type": "integer"}, + "waived": {"type": "integer"}, + "judged": {"type": "integer"}, + "informational": { + "type": "integer", + "description": "All non-defect findings (facts, metrics, classifications).", + }, + "unanalyzed": { + "type": "integer", + "description": "Files discovered but never analysed (parse errors / too-deep / missing source " + "roots); overlay count.", + }, + }, + "required": ["total", "active", "baselined", "waived", "judged", "informational", "unanalyzed"], + "additionalProperties": False, + }, + "gate": { + "type": "object", + "description": "The pass/fail gate decision (a trip is data, not an error).", + "properties": { + "tripped": {"type": "boolean"}, + "fail_on": { + "description": "The severity threshold evaluated, or null when no --fail-on ran.", + "enum": ["CRITICAL", "ERROR", "WARN", "INFO", "NONE", None], + }, + "fail_on_unanalyzed": { + "type": "boolean", + "description": "Whether the unanalyzed sub-gate knob was set.", + }, + "exit_class": { + "type": "integer", + "enum": [0, 1], + "description": "0 clean, 1 gate tripped (mirrors tripped).", + }, + "verdict": { + "type": "string", + "enum": ["NOT_EVALUATED", "PASSED", "FAILED"], + "description": "NOT_EVALUATED when neither sub-gate was configured; FAILED iff tripped.", + }, + "severity_tripped": { + "type": "boolean", + "description": "Sub-gate attribution: the severity threshold tripped.", + }, + "unanalyzed_tripped": { + "type": "boolean", + "description": "Sub-gate attribution: the unanalyzed gate tripped.", + }, + "would_trip_at": { + "description": "Highest severity at which the gate WOULD trip on the evaluated population, or " + "null if nothing would.", + "enum": ["CRITICAL", "ERROR", "WARN", "INFO", None], + }, + "reason": { + "type": "string", + "description": "Human-readable verdict naming the count/class of defects that decided it.", + }, + "evaluated": { + "type": ["string", "null"], + "description": "Which population the gate judged (unsuppressed default vs suppression-honoring).", + }, + "migration_hint": { + "type": ["string", "null"], + "description": "Secure-gate-default rollout hint, or null.", + }, + }, + "required": [ + "tripped", + "fail_on", + "fail_on_unanalyzed", + "exit_class", + "verdict", + "severity_tripped", + "unanalyzed_tripped", + "would_trip_at", + "reason", + "evaluated", + "migration_hint", + ], + "additionalProperties": False, + }, + "loomweave": { + "description": "Raw Loomweave taint-fact write result; null when no Loomweave client is configured.", + "oneOf": [ + {"type": "null"}, + { + "type": "object", + "properties": { + "reachable": {"type": "boolean"}, + "written": {"type": "integer", "description": "Entity taint blobs written."}, + "unresolved_qualnames": {"type": "array", "items": {"type": "string"}}, + "disabled_reason": {"type": ["string", "null"]}, + }, + "required": ["reachable", "written", "unresolved_qualnames", "disabled_reason"], + "additionalProperties": False, + }, + ], + }, + "filigree": { + "description": "Raw Filigree emit result; null when no emitter is configured.", + "oneOf": [ + {"type": "null"}, + { + "type": "object", + "properties": { + "reachable": {"type": "boolean"}, + "created": {"type": "integer"}, + "updated": {"type": "integer"}, + "failed": {"type": "integer"}, + "warnings": {"type": "array", "items": {"type": "string"}}, + "status": { + "type": ["integer", "null"], + "description": "HTTP error status (401/403/5xx) for soft failures; null on success or " + "transport failure.", + }, + "auth_rejected": { + "type": "boolean", + "description": "True when the emit was refused with 401/403.", + }, + "token_sent": { + "type": "boolean", + "description": "Whether a bearer token was actually sent (splits a 401 into absent vs " + "rejected).", + }, + "url": {"type": ["string", "null"], "description": "The endpoint attempted."}, + "destination": {"$ref": "#/$defs/filigree_destination"}, + }, + "required": [ + "reachable", + "created", + "updated", + "failed", + "warnings", + "status", + "auth_rejected", + "token_sent", + "url", + "destination", + ], + "additionalProperties": False, + }, + ], + }, + "loomweave_write": {"$ref": "#/$defs/loomweave_write_status"}, + "filigree_emit": {"$ref": "#/$defs/filigree_emit_status"}, + "agent_summary": { + "type": "object", + "description": "The stable agent-oriented handoff block (schema wardline-agent-summary-1): active defects " + "first, suppressed debt visible, integration status explicit, suggested next tool calls.", + "properties": { + "schema": {"type": "string", "enum": ["wardline-agent-summary-1"]}, + "summary": { + "type": "object", + "description": "Whole-project counts (never affected by where/pagination filters).", + "properties": { + "files_scanned": {"type": "integer"}, + "total_findings": {"type": "integer"}, + "active_defects": {"type": "integer"}, + "suppressed_findings": {"type": "integer"}, + "engine_facts": { + "type": "integer", + "description": "Kind.FACT findings with a WLN-ENGINE-* rule_id.", + }, + "baselined": {"type": "integer"}, + "waived": {"type": "integer"}, + "judged": {"type": "integer"}, + "informational": { + "type": "integer", + "description": "ALL non-defect findings (engine facts included; the display array below " + "excludes them).", + }, + "unanalyzed": {"type": "integer"}, + }, + "required": [ + "files_scanned", + "total_findings", + "active_defects", + "suppressed_findings", + "engine_facts", + "baselined", + "waived", + "judged", + "informational", + "unanalyzed", + ], + "additionalProperties": False, + }, + "gate": { + "type": "object", + "description": "Gate echo inside the agent summary (no sub-gate attribution flags here; see the " + "top-level gate block for those).", + "properties": { + "tripped": {"type": "boolean"}, + "fail_on": {"enum": ["CRITICAL", "ERROR", "WARN", "INFO", "NONE", None]}, + "exit_class": {"type": "integer", "enum": [0, 1]}, + "verdict": {"type": "string", "enum": ["NOT_EVALUATED", "PASSED", "FAILED"]}, + "would_trip_at": {"enum": ["CRITICAL", "ERROR", "WARN", "INFO", None]}, + "reason": {"type": "string"}, + "evaluated": {"type": ["string", "null"]}, + "migration_hint": {"type": ["string", "null"]}, + }, + "required": [ + "tripped", + "fail_on", + "exit_class", + "verdict", + "would_trip_at", + "reason", + "evaluated", + "migration_hint", + ], + "additionalProperties": False, + }, + "integrations": { + "type": "object", + "properties": { + "filigree_emit": {"$ref": "#/$defs/filigree_emit_status"}, + "loomweave_write": {"$ref": "#/$defs/loomweave_write_status"}, + }, + "required": ["filigree_emit", "loomweave_write"], + "additionalProperties": False, + }, + "active_defects": { + "type": "array", + "description": "Non-suppressed defects in the displayed page (severity-sorted). Each entry " + "carries explain/next_tool_calls hints; with explain:true an inlined explanation (capped — see " + "truncation.explanations_truncated).", + "items": {"$ref": "#/$defs/active_defect_entry"}, + }, + "suppressed_findings": { + "type": "array", + "description": "Suppressed (baselined/waived/judged) defects in the displayed page.", + "items": {"$ref": "#/$defs/finding_entry"}, + }, + "engine_facts": { + "type": "array", + "description": "Engine diagnostic facts (WLN-ENGINE-*) in the displayed page.", + "items": {"$ref": "#/$defs/finding_entry"}, + }, + "informational": { + "type": "array", + "description": "Non-defect, non-engine-fact findings (metrics, classifications, suggestions) in " + "the displayed page.", + "items": {"$ref": "#/$defs/finding_entry"}, + }, + "truncation": { + "type": "object", + "description": "Single pagination descriptor for the four display arrays (one ordered union: " + "active, suppressed, engine facts, informational).", + "properties": { + "summary_only": {"type": "boolean"}, + "include_suppressed": {"type": "boolean"}, + "max_findings": { + "type": ["integer", "null"], + "description": "Effective page size; null means uncapped (full:true).", + }, + "offset": {"type": "integer"}, + "findings_total": { + "type": "integer", + "description": "Size of the displayed union before paging.", + }, + "findings_returned": {"type": "integer"}, + "next_offset": { + "type": ["integer", "null"], + "description": "Pass as offset to fetch the next page; null when complete.", + }, + "findings_truncated": {"type": "boolean"}, + "explanations_truncated": { + "type": "boolean", + "description": "True when explain:true hit the inlining cap before covering every shown " + "active defect.", + }, + }, + "required": [ + "summary_only", + "include_suppressed", + "max_findings", + "offset", + "findings_total", + "findings_returned", + "next_offset", + "findings_truncated", + "explanations_truncated", + ], + "additionalProperties": False, + }, + "next_actions": { + "type": "array", + "description": "Gate-aware suggested next tool calls, driven by the whole-project active count " + "(not the displayed slice).", + "items": { + "type": "object", + "properties": { + "tool": {"type": "string", "enum": ["explain_taint", "file_finding", "scan"]}, + "reason": {"type": "string"}, + }, + "required": ["tool", "reason"], + "additionalProperties": False, + }, + }, + }, + "required": [ + "schema", + "summary", + "gate", + "integrations", + "active_defects", + "suppressed_findings", + "engine_facts", + "informational", + "truncation", + "next_actions", + ], + "additionalProperties": False, + }, + "legis_artifact": { + "type": "object", + "description": "OPTIONAL: the verbatim-postable signed scan object for legis POST /wardline/scan-results. " + "Present only when a WARDLINE_LEGIS_ARTIFACT_KEY is provisioned or legis_artifact:true was passed, AND " + "building it did not fail (a signing refusal omits it).", + "properties": { + "scanner_identity": {"type": "string", "description": "wardline@."}, + "rule_set_version": {"type": "string", "description": "Hash of the effective ruleset."}, + "fingerprint_scheme": {"type": "string"}, + "findings": { + "type": "array", + "description": "The gate population projected onto legis's accepted vocabulary.", + "items": { + "type": "object", + "properties": { + "rule_id": {"type": "string"}, + "message": {"type": "string"}, + "severity": {"type": "string", "enum": ["CRITICAL", "ERROR", "WARN", "INFO", "NONE"]}, + "kind": { + "type": "string", + "enum": ["defect", "fact", "classification", "metric", "suggestion"], + }, + "fingerprint": {"type": "string"}, + "qualname": {"type": ["string", "null"]}, + "properties": { + "type": "object", + "description": "Trust-tier-valued properties only (plus suppression_reason proof on " + "non-active defects); diagnostics dropped.", + "additionalProperties": {"type": "string"}, + }, + "suppression_state": {"type": "string", "enum": ["active", "waived", "suppressed"]}, + }, + "required": [ + "rule_id", + "message", + "severity", + "kind", + "fingerprint", + "qualname", + "properties", + "suppression_state", + ], + "additionalProperties": False, + }, + }, + "commit_sha": { + "type": "string", + "description": "Present when provenance was readable (always on the signed path).", + }, + "tree_sha": { + "type": "string", + "description": "Committed tree SHA; present on the signed path and best-effort otherwise.", + }, + "artifact_signature": { + "type": "string", + "description": "hmac-sha256:v2: over the canonical scan-minus-signature; present only on the " + "signed (key + clean tree) path.", + }, + "dirty": { + "type": "boolean", + "description": "Present (true) only when the working tree was dirty (unsigned dev artifact).", + }, + }, + "required": ["scanner_identity", "rule_set_version", "fingerprint_scheme", "findings"], + "additionalProperties": False, + }, + "legis_artifact_status": { + "type": "object", + "description": "OPTIONAL: signed/dirty status of the legis artifact attempt. Present whenever the legis " + "block was activated (key provisioned or legis_artifact:true), including when signing was refused and " + "legis_artifact itself is absent.", + "properties": { + "configured": {"type": "boolean", "enum": [True]}, + "signed": {"type": "boolean"}, + "key_id": { + "type": ["string", "null"], + "description": "Non-secret short id of the HMAC key (first 8 hex of sha256), or null when unkeyed.", + }, + "reason": { + "type": ["string", "null"], + "description": "Refusal/unverified reason (e.g. dirty-tree refusal), or null.", + }, + "dirty": { + "type": "boolean", + "description": "Present only when the artifact was actually built (absent on a build refusal).", + }, + }, + "required": ["configured", "signed", "key_id", "reason"], + "additionalProperties": False, + }, + }, + "required": [ + "files_scanned", + "summary", + "gate", + "loomweave", + "filigree", + "loomweave_write", + "filigree_emit", + "agent_summary", + ], + "additionalProperties": False, + "$defs": { + "filigree_destination": { + "type": "object", + "description": "Where findings were (or would be) sent, so a wrong-project write is visible.", + "properties": { + "url": {"type": ["string", "null"]}, + "project": { + "type": ["string", "null"], + "description": "The project pinned in the URL, or null when Filigree resolves it server-side.", + }, + "project_pinned": {"type": "boolean"}, + }, + "required": ["url", "project", "project_pinned"], + "additionalProperties": False, + }, + "filigree_emit_status": { + "type": "object", + "description": "Normalized Filigree emit status (always an object; configured:false when no emitter).", + "properties": { + "configured": {"type": "boolean"}, + "reachable": {"type": ["boolean", "null"], "description": "null when not configured."}, + "created": {"type": "integer"}, + "updated": {"type": "integer"}, + "failed": {"type": "integer"}, + "warnings": {"type": "array", "items": {"type": "string"}}, + "disabled_reason": { + "type": ["string", "null"], + "description": "Actionable reason (auth-rejected vs server error vs unreachable vs not " + "configured), or null when reached.", + }, + "destination": {"$ref": "#/$defs/filigree_destination"}, + "status": { + "type": ["integer", "null"], + "description": "HTTP error status for soft failures; absent when not configured.", + }, + "auth_rejected": {"type": "boolean", "description": "Absent when not configured."}, + "token_sent": {"type": "boolean", "description": "Absent when not configured."}, + "url": {"type": ["string", "null"], "description": "Absent when not configured."}, + }, + "required": [ + "configured", + "reachable", + "created", + "updated", + "failed", + "warnings", + "disabled_reason", + "destination", + ], + "additionalProperties": False, + }, + "loomweave_write_status": { + "type": "object", + "description": "Normalized Loomweave taint-fact write status (always an object; configured:false when no " + "client).", + "properties": { + "configured": {"type": "boolean"}, + "reachable": {"type": ["boolean", "null"], "description": "null when not configured."}, + "written": {"type": "integer"}, + "unresolved_qualnames": {"type": "array", "items": {"type": "string"}}, + "disabled_reason": {"type": ["string", "null"]}, + }, + "required": ["configured", "reachable", "written", "unresolved_qualnames", "disabled_reason"], + "additionalProperties": False, + }, + "location": { + "type": "object", + "properties": { + "path": {"type": "string", "description": "Repo-relative POSIX path."}, + "line_start": {"type": ["integer", "null"]}, + "line_end": {"type": ["integer", "null"]}, + }, + "required": ["path", "line_start", "line_end"], + "additionalProperties": False, + }, + "finding_entry": { + "type": "object", + "description": "One finding (suppressed / engine-fact / informational display arrays).", + "properties": { + "fingerprint": {"type": "string"}, + "rule_id": {"type": "string"}, + "severity": {"type": "string", "enum": ["CRITICAL", "ERROR", "WARN", "INFO", "NONE"]}, + "kind": {"type": "string", "enum": ["defect", "fact", "classification", "metric", "suggestion"]}, + "qualname": {"type": ["string", "null"]}, + "location": {"$ref": "#/$defs/location"}, + "message": {"type": "string"}, + "suppression_state": {"type": "string", "enum": ["active", "baselined", "waived", "judged"]}, + "suppression_reason": {"type": ["string", "null"]}, + }, + "required": [ + "fingerprint", + "rule_id", + "severity", + "kind", + "qualname", + "location", + "message", + "suppression_state", + "suppression_reason", + ], + "additionalProperties": False, + }, + "active_defect_entry": { + "type": "object", + "description": "One active defect, with explain availability and suggested next tool calls; explanation " + "is inlined only under scan(explain:true) for findings with a qualname, up to the cap.", + "properties": { + "fingerprint": {"type": "string"}, + "rule_id": {"type": "string"}, + "severity": {"type": "string", "enum": ["CRITICAL", "ERROR", "WARN", "INFO", "NONE"]}, + "kind": {"type": "string", "enum": ["defect", "fact", "classification", "metric", "suggestion"]}, + "qualname": {"type": ["string", "null"]}, + "location": {"$ref": "#/$defs/location"}, + "message": {"type": "string"}, + "suppression_state": {"type": "string", "enum": ["active", "baselined", "waived", "judged"]}, + "suppression_reason": {"type": ["string", "null"]}, + "explain": { + "type": "object", + "properties": { + "available": {"type": "boolean"}, + "reason": { + "type": ["string", "null"], + "description": "Why explain is unavailable (no qualname), or null.", + }, + "suggested_call": { + "oneOf": [ + {"type": "null"}, + { + "type": "object", + "properties": { + "tool": {"type": "string", "enum": ["explain_taint"]}, + "arguments": { + "type": "object", + "properties": {"fingerprint": {"type": "string"}}, + "required": ["fingerprint"], + "additionalProperties": False, + }, + }, + "required": ["tool", "arguments"], + "additionalProperties": False, + }, + ], + }, + }, + "required": ["available", "reason", "suggested_call"], + "additionalProperties": False, + }, + "next_tool_calls": { + "type": "array", + "items": { + "type": "object", + "properties": { + "tool": {"type": "string", "enum": ["explain_taint", "file_finding"]}, + "arguments": { + "type": "object", + "properties": {"fingerprint": {"type": "string"}}, + "required": ["fingerprint"], + "additionalProperties": False, + }, + }, + "required": ["tool", "arguments"], + "additionalProperties": False, + }, + }, + "explanation": {"$ref": "#/$defs/explanation"}, + }, + "required": [ + "fingerprint", + "rule_id", + "severity", + "kind", + "qualname", + "location", + "message", + "suppression_state", + "suppression_reason", + "explain", + "next_tool_calls", + ], + "additionalProperties": False, + }, + "explanation": { + "type": "object", + "description": "Inlined taint provenance (same shape as the explanation slice of explain_taint).", + "properties": { + "tier_in": {"type": ["string", "null"], "description": "Actual (untrusted) tier arriving at the sink."}, + "tier_out": {"type": ["string", "null"], "description": "Tier the sink declares it returns."}, + "immediate_tainted_callee": {"type": ["string", "null"]}, + "source_boundary_qualname": {"type": ["string", "null"]}, + "resolved_call_count": {"type": "integer"}, + "unresolved_call_count": {"type": "integer"}, + "remediation": { + "type": "object", + "properties": { + "kind": {"type": "string", "enum": ["boundary_placement", "review_required"]}, + "rule_id": {"type": "string"}, + "summary": {"type": "string"}, + "sink_qualname": {"type": ["string", "null"]}, + "source_qualname": {"type": ["string", "null"]}, + "caveat": {"type": "string"}, + }, + "required": ["kind", "rule_id", "summary", "sink_qualname", "source_qualname", "caveat"], + "additionalProperties": False, + }, + }, + "required": [ + "tier_in", + "tier_out", + "immediate_tainted_callee", + "source_boundary_qualname", + "resolved_call_count", + "unresolved_call_count", + "remediation", + ], + "additionalProperties": False, + }, + }, +} + + +_SCAN_TOOL: dict[str, Any] = { + "name": "scan", + "title": "Trust-boundary scan", + "description": "Whole-program taint scan of the project. Returns structured " + "findings, the suppression summary (active = unsuppressed defects; " + "by default the --fail-on gate evaluates the UNSUPPRESSED population so " + "repo-controlled baseline/waiver/judged annotate but do not clear it — " + "pass `trust_suppressions: true` for the trusted-local behaviour), " + "and the gate verdict. Pass `where` to filter the returned findings " + "(conjunctive; summary/gate stay whole-project) and `explain: true` to inline " + "each active defect's taint provenance — one call, no per-finding explain_taint. " + "When a Filigree URL is configured, also POSTs the " + "findings to Filigree (fail-soft: an unreachable sibling or rejected payload " + "is reported in the `filigree` block, never fails the scan).", + "input_schema": { + "type": "object", + "properties": { + "path": {"type": "string", "description": "subdir relative to project root"}, + "fail_on": {"type": "string", "enum": _SEVERITY_ENUM}, + "fail_on_unanalyzed": { + "type": "boolean", + "description": "Also trip the gate when any file was discovered but could " + "not be analyzed (parse error / too-deep skip / missing source root; benign " + "no-module skips excluded). Default false — same default as the CLI's " + "--fail-on-unanalyzed; summary.unanalyzed always reports the count either " + "way, and gate.unanalyzed_tripped attributes a trip to this knob.", + }, + "config": {"type": "string"}, + "lang": { + "type": "string", + "enum": ["python", "rust"], + "description": "Language frontend (default python). 'rust' sweeps .rs files " + "for the command-injection slice (RS-WL-108 program injection / RS-WL-112 " + "shell injection; frozen identity, baseline-eligible). Preview posture: " + "weft.toml severity overrides do not yet apply to Rust findings, and a tree " + "with no /// @trusted markers is vacuously green — read the WLN-RUST-COVERAGE " + "fact before trusting '0 active'. Requires the wardline[rust] extra.", + }, + "where": { + "type": "object", + "description": "Filter the returned findings (conjunctive). Keys: " + "rule_id, qualname, severity, suppression, kind, path_glob, sink, tier. " + "summary/gate still describe the whole project.", + "properties": { + "rule_id": {"type": "string"}, + "qualname": {"type": "string"}, + "severity": {"type": "string", "enum": _SEVERITY_ENUM}, + "suppression": {"type": "string", "enum": ["active", "baselined", "waived", "judged"]}, + "kind": { + "type": "string", + "enum": ["defect", "fact", "classification", "metric", "suggestion"], + }, + "path_glob": {"type": "string"}, + "sink": {"type": "string"}, + "tier": {"type": "string"}, + }, + }, + "explain": { + "type": "boolean", + "description": "Inline each active defect's taint provenance " + "(immediate tainted callee, source boundary, trust tiers, resolution " + "counts) — one call instead of an explain_taint per finding. Inlining is " + "capped at 10 provenances by default (raise/lower with max_findings); the cut " + "is reported at agent_summary.truncation.explanations_truncated.", + }, + "summary_only": { + "type": "boolean", + "description": "Return counts + gate only, no finding bodies — the smallest " + "'did the gate pass?' payload. summary/gate still describe the whole project.", + }, + "full": { + "type": "boolean", + "description": "Default false. The default scan is BOUNDED (≤25 finding bodies) so " + "it cannot overflow your context; set full=true to return ALL bodies in one call " + "(or page with offset). summary/gate counts are always whole-project.", + }, + "max_findings": { + "type": "integer", + "minimum": 0, + "description": "Override the page size for returned finding bodies (and the inlined-" + "explanation cap). Default 25; full=true ignores it. Must be a non-negative integer. " + "The cut + next page are reported in agent_summary.truncation; counts stay whole.", + }, + "offset": { + "type": "integer", + "minimum": 0, + "description": "Pagination cursor into the ordered finding union (active → suppressed " + "→ engine_facts → informational). Pass agent_summary.truncation.next_offset from " + "the previous call to fetch the next page. Default 0.", + }, + "include_suppressed": { + "type": "boolean", + "description": "Default true. Set false to drop suppressed (baselined/waived/" + "judged) finding bodies from the response; the suppression counts stay in " + "summary.", + }, + "new_since": { + "type": "string", + "description": "PR-scoped 'new findings only' gate: only gate on findings in " + "files/entities changed since this git ref", + }, + "cache_dir": { + "type": "string", + "description": "subdir relative to project root for summary cache", + }, + "trust_packs": {"type": "array", "items": {"type": "string"}}, + "trust_local_packs": { + "type": "boolean", + "description": "Allow loading custom trust-grammar packs from the local project directory", + }, + "strict_defaults": { + "type": "boolean", + "description": "Ignore repository-supplied custom configuration overrides (weft.toml)", + }, + "trust_suppressions": { + "type": "boolean", + "description": "Let repository-controlled baseline/waiver/judged clear the gate " + "(they always annotate findings regardless). Default false — the gate " + "evaluates the unsuppressed population so a PR cannot self-suppress its " + "own defect. Use only on a trusted checkout; in CI prefer new_since.", + }, + "legis_artifact": { + "type": "boolean", + "description": "Attach the verbatim-postable legis scan-artifact " + "(`legis_artifact` block) even when no signing key is provisioned " + "(unsigned, for legis's optional-verify posture).", + }, + "allow_dirty": { + "type": "boolean", + "description": "For the legis artifact only: on a dirty tree emit an UNSIGNED, " + "clearly-marked (dirty: true) dev artifact instead of refusing to sign. " + "Signing stays clean-tree-only; legis records it unverified.", + }, + }, + }, + "output_schema": _SCAN_OUTPUT_SCHEMA, + "annotations": { + "title": "Trust-boundary scan", + "readOnlyHint": True, + "destructiveHint": False, + "idempotentHint": True, + "openWorldHint": False, + }, +} + + def _attach_legis_artifact( response: dict[str, Any], result: Any, @@ -485,6 +1718,209 @@ def _explain_taint(args: dict[str, Any], root: Path, loomweave: Any = None) -> d return result_dict +_EXPLAIN_TAINT_OUTPUT_SCHEMA: dict[str, Any] = { + "type": "object", + "description": "Success payload of the explain_taint tool: the taint provenance slice for one finding (single " + "source: core/explain.explain_taint_result, shared with the CLI `wardline explain-taint`). Served either from a " + "fresh Loomweave store fact (no re-scan) or from an SP8 re-run; both paths produce this same key set. With " + "chain=true and a configured Loomweave store, a `chain` block is additionally attached.", + "properties": { + "fingerprint": { + "type": "string", + "description": "Stable finding fingerprint. May be the empty string on the store-served path when the " + "entity blob carries no per-finding rows (the entity is known, the specific finding is not).", + }, + "rule_id": { + "type": "string", + "description": "Rule that produced the finding (e.g. PY-WL-101). May be the empty string on the " + "store-served path when no per-finding row matched.", + }, + "sink_qualname": { + "type": ["string", "null"], + "description": "Qualified name of the sink function the tainted value reaches (null when the engine has " + "no qualname for the finding).", + }, + "location": { + "type": "object", + "description": "Source location of the finding. path may be empty and line null on the store-served path " + "when the blob has no per-finding rows.", + "properties": { + "path": { + "type": "string", + "description": "Root-relative posix path of the finding's file (empty string when unknown on the " + "store-served path).", + }, + "line": { + "type": ["integer", "null"], + "description": "1-based start line of the finding (null when unknown).", + }, + }, + "required": ["path", "line"], + "additionalProperties": False, + }, + "tier_in": { + "type": ["string", "null"], + "description": "Actual (untrusted) trust tier arriving at the sink, e.g. EXTERNAL_RAW (null when the " + "engine recorded none).", + }, + "tier_out": { + "type": ["string", "null"], + "description": "Trust tier the sink declares it returns, e.g. INTEGRAL (null when the engine recorded " + "none).", + }, + "immediate_tainted_callee": { + "type": ["string", "null"], + "description": "Bare trailing name of the call that introduced the untrusted return into the sink (null " + "when unresolved).", + }, + "source_boundary_qualname": { + "type": ["string", "null"], + "description": "Originating boundary resolved one hop from the sink: qualified name of the boundary " + "function the taint came from (null when not resolvable in one hop). On the store-served path this is the " + "blob's contributing_callee_qualname.", + }, + "resolved_call_count": { + "type": "integer", + "description": "Number of calls inside the sink the engine resolved during taint computation.", + }, + "unresolved_call_count": { + "type": "integer", + "description": "Number of calls inside the sink the engine could NOT resolve (residual uncertainty in the " + "explanation).", + }, + "remediation": { + "type": "object", + "description": "Advisory fix-at-the-boundary hint derived from the explanation; never replaces the " + "factual taint fields above.", + "properties": { + "kind": { + "type": "string", + "enum": ["boundary_placement", "review_required"], + "description": "boundary_placement for PY-WL-101 (place/repair a @trust_boundary at the " + "validating function); review_required for every other rule (no automated hint).", + }, + "rule_id": { + "type": "string", + "description": "Rule the hint applies to (echoes the top-level rule_id).", + }, + "summary": {"type": "string", "description": "Human/agent-readable remediation guidance sentence."}, + "sink_qualname": { + "type": ["string", "null"], + "description": "Sink the hint refers to (null when the finding has no qualname).", + }, + "source_qualname": { + "type": ["string", "null"], + "description": "Taint source the hint refers to: the resolved boundary, else the immediate " + "tainted callee, else null when unresolved.", + }, + "caveat": { + "type": "string", + "description": "Standing warning against blind decorator insertion / over-trusting the hint.", + }, + }, + "required": ["kind", "rule_id", "summary", "sink_qualname", "source_qualname", "caveat"], + "additionalProperties": False, + }, + "chain": { + "type": "object", + "description": "Full N-hop taint chain from the sink to the originating boundary, walked from the " + "Loomweave store. PRESENT only when the call passed chain=true AND a Loomweave store is configured AND " + "the finding has a sink qualname; otherwise absent (silent degradation to the single-hop explanation).", + "properties": { + "hops": { + "type": "array", + "description": "Ordered hops from the sink toward the boundary leaf. The walk stops cleanly at a " + "boundary (contributing_callee_qualname null on the last hop) or truncates explicitly (see " + "truncated_at).", + "items": { + "type": "object", + "properties": { + "qualname": { + "type": "string", + "description": "Qualified name of the function at this hop.", + }, + "tier_in": { + "type": ["string", "null"], + "description": "Actual trust tier arriving at this hop (from the stored fact; null " + "when absent).", + }, + "tier_out": { + "type": ["string", "null"], + "description": "Trust tier this hop declares it returns (from the stored fact; null " + "when absent).", + }, + "contributing_callee_qualname": { + "type": ["string", "null"], + "description": "Next hop toward the boundary; null at the boundary leaf (clean " + "finish).", + }, + }, + "required": ["qualname", "tier_in", "tier_out", "contributing_callee_qualname"], + "additionalProperties": False, + }, + }, + "truncated_at": { + "type": ["string", "null"], + "description": "Qualified name of the next hop the walk could NOT take (stale/absent fact, read " + "error, cycle, or max_hops reached) — truncation is always explicit; null means the chain reached " + "the boundary cleanly.", + }, + }, + "required": ["hops", "truncated_at"], + "additionalProperties": False, + }, + }, + "required": [ + "fingerprint", + "rule_id", + "sink_qualname", + "location", + "tier_in", + "tier_out", + "immediate_tainted_callee", + "source_boundary_qualname", + "resolved_call_count", + "unresolved_call_count", + "remediation", + ], + "additionalProperties": False, +} + + +_EXPLAIN_TAINT_TOOL: dict[str, Any] = { + "name": "explain_taint", + "title": "Explain finding taint", + "description": "Explain ONE finding's taint: the immediate tainted callee, the " + "originating boundary, and the trust tiers at the sink. Call right " + "after scan and before editing — a stale fingerprint returns an error. " + "Pass the finding's `qualname` as `sink_qualname`: when a Loomweave store " + "is configured this serves the explanation from the store instead of " + "re-scanning. Pass `chain: true` (needs a configured Loomweave store) to " + "also walk the full taint chain from the sink to the originating boundary; " + "without a store it degrades to the single-hop explanation (no `chain` block).", + "input_schema": { + "type": "object", + "properties": { + "fingerprint": {"type": "string"}, + "path": {"type": "string"}, + "line": {"type": "integer"}, + "sink_qualname": {"type": "string"}, + "chain": {"type": "boolean"}, + "max_hops": {"type": "integer"}, + "config": {"type": "string"}, + }, + }, + "output_schema": _EXPLAIN_TAINT_OUTPUT_SCHEMA, + "annotations": { + "title": "Explain finding taint", + "readOnlyHint": True, + "destructiveHint": False, + "idempotentHint": True, + "openWorldHint": False, + }, +} + + def _dossier( args: dict[str, Any], root: Path, loomweave: Any = None, filigree_url: str | None = None ) -> dict[str, Any]: @@ -505,6 +1941,284 @@ def _dossier( return dossier.to_dict() +_DOSSIER_OUTPUT_SCHEMA: dict[str, Any] = { + "type": "object", + "description": "One-call entity dossier envelope: Wardline's own trust posture (always re-derived fresh) plus " + "Loomweave linkages and Filigree open work, each cross-tool section degrading to an honest unavailable shape " + "(available=false + reason) when its source is absent. Token-bounded with an explicit truncation marker.", + "properties": { + "identity": { + "type": "object", + "description": "Who the entity is plus its two-axis freshness (identity axis / content axis). Never " + "trimmed by the budgeter.", + "properties": { + "qualname": { + "type": "string", + "description": "The entity's qualified name, minted relative to the scan root.", + }, + "kind": {"type": ["string", "null"], "description": "Entity kind (e.g. function); null when unknown."}, + "path": {"type": ["string", "null"], "description": "Source file path of the entity."}, + "line_start": {"type": ["integer", "null"]}, + "line_end": {"type": ["integer", "null"]}, + "sei": { + "type": ["string", "null"], + "description": "Opaque stable entity identifier (the cross-tool binding key); null when no " + "Loomweave binding was resolved.", + }, + "keyed_on_sei": { + "type": "boolean", + "description": "True when the cross-tool sections were keyed on the SEI rather than a locator.", + }, + "identity_status": { + "type": "string", + "enum": ["alive", "orphaned", "unavailable"], + "description": "Identity axis: is this the same entity? Never inferred from content.", + }, + "content_status": { + "type": "string", + "enum": ["fresh", "stale", "unknown"], + "description": "Content axis: has the entity's code changed? Never inferred from identity.", + }, + "content_hash": { + "type": ["string", "null"], + "description": "Current content hash from the binding, when available.", + }, + }, + "required": [ + "qualname", + "kind", + "path", + "line_start", + "line_end", + "sei", + "keyed_on_sei", + "identity_status", + "content_status", + "content_hash", + ], + "additionalProperties": False, + }, + "shape": { + "type": "object", + "description": "Signature and decorators as declared in source.", + "properties": { + "signature": {"type": ["string", "null"], "description": 'Rendered signature, e.g. "(p) -> str".'}, + "decorators": { + "type": "array", + "items": {"type": "string"}, + "description": "Decorators as declared, each prefixed with '@'.", + }, + }, + "required": ["signature", "decorators"], + "additionalProperties": False, + }, + "trust": { + "type": "object", + "description": "Wardline's OWN trust posture, re-derived from a live scan (fresh by construction).", + "properties": { + "declared_return": { + "type": ["string", "null"], + "description": "Declared return trust tier; null when undeclared.", + }, + "actual_return": { + "type": ["string", "null"], + "description": "Engine-computed actual return taint; null when not computed.", + }, + "gate_verdict": { + "type": "string", + "enum": ["defect", "clean", "unknown"], + "description": "Three-valued, fail-closed verdict: defect (active findings), clean (declared " + "posture that conforms), unknown (undeclared/unprovable/under-scanned).", + }, + "active_findings": { + "type": "array", + "description": "Active (non-suppressed) defect findings on the entity. May be trimmed by the " + "token budgeter (see truncation.elided).", + "items": { + "type": "object", + "properties": { + "rule_id": {"type": "string"}, + "severity": {"type": "string", "enum": ["CRITICAL", "ERROR", "WARN", "INFO", "NONE"]}, + "message": {"type": "string"}, + "line": {"type": ["integer", "null"]}, + }, + "required": ["rule_id", "severity", "message", "line"], + "additionalProperties": False, + }, + }, + "suppressed_findings": { + "type": "integer", + "description": "Count of accepted (baselined/waived/judged) defects — known debt a clean verdict " + "must not hide.", + }, + "unanalyzed_reason": { + "type": ["string", "null"], + "description": "Engine under-scan fact (parse error / recursion skip) when the body was not " + "analysed; else null.", + }, + "freshness": { + "type": "string", + "enum": ["fresh_by_construction"], + "description": "Constant: the trust section is re-derived on demand, never stale.", + }, + }, + "required": [ + "declared_return", + "actual_return", + "gate_verdict", + "active_findings", + "suppressed_findings", + "unanalyzed_reason", + "freshness", + ], + "additionalProperties": False, + }, + "linkages": { + "type": "object", + "description": "Call-graph neighbourhood from Loomweave. available=false with a reason when Loomweave is " + "not configured / unreachable / serves no HTTP linkages.", + "properties": { + "available": {"type": "boolean"}, + "callers": { + "type": "array", + "items": {"type": "string"}, + "description": "Caller entity locators; empty when unavailable. May be trimmed by the token " + "budgeter.", + }, + "callees": { + "type": "array", + "items": {"type": "string"}, + "description": "Callee entity locators; empty when unavailable. May be trimmed by the token " + "budgeter.", + }, + "scc_peers": { + "type": "array", + "items": {"type": "string"}, + "description": "Strongly-connected-component peers (currently always empty: SCC membership is not " + "served over HTTP).", + }, + "identity_status": {"type": "string", "enum": ["alive", "orphaned", "unavailable"]}, + "content_status": {"type": "string", "enum": ["fresh", "stale", "unknown"]}, + "reason": { + "type": ["string", "null"], + "description": "Why the section is unavailable or degraded (e.g. one-sided linkage failure); null " + "when fully available.", + }, + }, + "required": ["available", "callers", "callees", "scc_peers", "identity_status", "content_status", "reason"], + "additionalProperties": False, + }, + "work": { + "type": "object", + "description": "Open work from Filigree, keyed on the SEI. available=false with a reason when Filigree is " + "not configured / unreachable / there is no binding.", + "properties": { + "available": {"type": "boolean"}, + "tickets": { + "type": "array", + "description": "Filigree issues bound to the entity. May be trimmed by the token budgeter.", + "items": { + "type": "object", + "properties": { + "issue_id": {"type": "string"}, + "status": {"type": ["string", "null"]}, + "priority": {"type": ["string", "null"]}, + "title": {"type": ["string", "null"]}, + "drift": { + "type": "boolean", + "description": "True when the issue was bound to a PRIOR version of the entity " + "(content hash at attach no longer matches).", + }, + }, + "required": ["issue_id", "status", "priority", "title", "drift"], + "additionalProperties": False, + }, + }, + "identity_status": {"type": "string", "enum": ["alive", "orphaned", "unavailable"]}, + "content_status": { + "type": "string", + "enum": ["fresh", "stale", "unknown"], + "description": "stale when any ticket binding drifted; unknown when a compare was impossible.", + }, + "reason": { + "type": ["string", "null"], + "description": "Why the section is unavailable; null when available.", + }, + }, + "required": ["available", "tickets", "identity_status", "content_status", "reason"], + "additionalProperties": False, + }, + "synthesis": { + "type": ["string", "null"], + "description": "Best-effort one-paragraph actionable join of all sections; null when dropped to fit the " + "token budget.", + }, + "truncation": { + "type": "object", + "description": "Elision-honest truncation marker: truncated=false on a complete envelope; when true, " + "elided names every trimmed list with shown-of-total counts.", + "properties": { + "truncated": {"type": "boolean"}, + "elided": { + "type": "array", + "items": { + "type": "object", + "properties": { + "section": { + "type": "string", + "description": 'Dotted list path that was trimmed, e.g. "linkages.callers" or ' + '"trust.active_findings".', + }, + "shown": {"type": "integer"}, + "total": {"type": "integer"}, + }, + "required": ["section", "shown", "total"], + "additionalProperties": False, + }, + }, + "note": { + "type": ["string", "null"], + "description": "Human-readable trim summary; null when not truncated.", + }, + }, + "required": ["truncated", "elided", "note"], + "additionalProperties": False, + }, + }, + "required": ["identity", "shape", "trust", "linkages", "work", "synthesis", "truncation"], + "additionalProperties": False, +} + + +_DOSSIER_TOOL: dict[str, Any] = { + "name": "dossier", + "title": "Entity trust dossier", + "description": "One-call entity dossier for a function `entity` (a qualname): its " + "trust posture (declared vs actual taint, gate verdict, active findings — always " + "computed fresh), plus Loomweave call-graph linkages and Filigree open work joined on " + "the entity's opaque SEI. Every cross-tool section is freshness-stamped on BOTH axes " + "(identity alive/orphaned/unavailable + content fresh/stale/unknown) and degrades to " + "an honest `unavailable` when its source is absent. Token-bounded (~2k) with an " + "explicit truncation marker. Read the whole context without opening the source.", + "input_schema": { + "type": "object", + "required": ["entity"], + "properties": { + "entity": {"type": "string", "description": "the function qualname, e.g. pkg.mod.func"}, + "config": {"type": "string"}, + }, + }, + "output_schema": _DOSSIER_OUTPUT_SCHEMA, + "annotations": { + "title": "Entity trust dossier", + "readOnlyHint": True, + "destructiveHint": False, + "idempotentHint": True, + "openWorldHint": False, + }, +} + + def _assure(args: dict[str, Any], root: Path) -> dict[str, Any]: """Trust-surface COVERAGE posture — the pre-trust-decision read. How many declared trust boundaries the engine reached a definite verdict on vs. how many are honestly @@ -515,6 +2229,128 @@ def _assure(args: dict[str, Any], root: Path) -> dict[str, Any]: return posture.to_dict() +_ASSURE_OUTPUT_SCHEMA: dict[str, Any] = { + "type": "object", + "description": "Trust-surface coverage posture: how many declared trust boundaries got a definite verdict vs. how " + "many are honestly unknown, plus waiver debt. Identical to the CLI `assure` JSON.", + "properties": { + "boundaries_total": { + "type": "integer", + "description": "Denominator: count of anchored (trust-declared) entities. proven + defect_total + " + "len(unknown) == boundaries_total.", + }, + "proven": {"type": "integer", "description": "Boundaries with a definite clean verdict."}, + "defect_total": { + "type": "integer", + "description": "Boundaries with a definite defect verdict (a defect counts as COVERED — the engine " + "reached a verdict).", + }, + "unknown": { + "type": "array", + "description": "The honesty gap: anchored entities whose trust could not be proven either way, sorted by " + "qualname.", + "items": { + "type": "object", + "properties": { + "qualname": {"type": "string", "description": "Qualified name of the anchored entity."}, + "tier": {"type": ["string", "null"], "description": "Declared trust tier, or null if undeclared."}, + "location": { + "type": "object", + "description": "Where the entity is declared; both fields null when no location is known.", + "properties": {"path": {"type": ["string", "null"]}, "line": {"type": ["integer", "null"]}}, + "required": ["path", "line"], + "additionalProperties": False, + }, + "reason": { + "type": ["string", "null"], + "description": "Engine under-scan FACT message when the body was not analysed " + "(parse/recursion skip), else null (undeclared / unprovable).", + }, + }, + "required": ["qualname", "tier", "location", "reason"], + "additionalProperties": False, + }, + }, + "engine_limited": { + "type": "integer", + "description": "Sub-count of `unknown` that are unknown specifically because the engine under-scanned " + "them (reason != null).", + }, + "coverage_pct": { + "type": ["number", "null"], + "description": "Share of boundaries with a definite verdict, rounded to 1 decimal. null when " + "boundaries_total == 0 (no trust surface — never a vacuous 100%).", + }, + "unanalyzed_rule_ids": { + "type": "array", + "description": "Distinct under-scan rule ids present in the scan findings, sorted lexicographically.", + "items": {"type": "string"}, + }, + "waiver_debt": { + "type": "array", + "description": "Configured waivers with days-to-expiry, sorted by fingerprint. Populated even when " + "nothing was analysable.", + "items": { + "type": "object", + "properties": { + "fingerprint": {"type": "string", "description": "Finding fingerprint the waiver covers."}, + "expires": { + "type": ["string", "null"], + "description": "ISO date the waiver expires, or null for no expiry.", + }, + "days_left": { + "type": ["integer", "null"], + "description": "Days until expiry; may be NEGATIVE for a lapsed waiver; null for no expiry.", + }, + "reason": {"type": "string", "description": "The waiver's recorded justification."}, + }, + "required": ["fingerprint", "expires", "days_left", "reason"], + "additionalProperties": False, + }, + }, + "baselined_total": {"type": "integer", "description": "Findings suppressed by the baseline in this scan."}, + "judged_total": {"type": "integer", "description": "Findings suppressed by judge verdicts in this scan."}, + }, + "required": [ + "boundaries_total", + "proven", + "defect_total", + "unknown", + "engine_limited", + "coverage_pct", + "unanalyzed_rule_ids", + "waiver_debt", + "baselined_total", + "judged_total", + ], + "additionalProperties": False, +} + + +_ASSURE_TOOL: dict[str, Any] = { + "name": "assure", + "title": "Trust-surface coverage posture", + "description": "Trust-surface COVERAGE posture: how many declared trust boundaries the " + "engine reached a definite verdict on vs. how many are honestly unknown, plus " + "waiver-debt. Consult before deciding to trust a module.", + "input_schema": { + "type": "object", + "properties": { + "path": {"type": "string"}, + "config": {"type": "string"}, + }, + }, + "output_schema": _ASSURE_OUTPUT_SCHEMA, + "annotations": { + "title": "Trust-surface coverage posture", + "readOnlyHint": True, + "destructiveHint": False, + "idempotentHint": True, + "openWorldHint": False, + }, +} + + def _decorator_coverage( args: dict[str, Any], root: Path, @@ -535,6 +2371,217 @@ def _decorator_coverage( return report.to_dict() +_DECORATOR_COVERAGE_OUTPUT_SCHEMA: dict[str, Any] = { + "type": "object", + "description": "Row-level inventory of every trust-decorated entity under the project (the row-level sibling of " + "assure): each declared trust-surface entity with its current verdict, findings, identity binding, and open-work " + "state, plus a rollup summary.", + "properties": { + "summary": { + "type": "object", + "description": "Rollup counts over rows by finding_state.", + "properties": { + "total": {"type": "integer", "description": "Total declared trust-surface entities."}, + "clean": {"type": "integer", "description": "Rows with finding_state == clean."}, + "defect": {"type": "integer", "description": "Rows with finding_state == defect."}, + "unknown": {"type": "integer", "description": "Rows with finding_state == unknown."}, + "suppressed": {"type": "integer", "description": "Rows with finding_state == suppressed."}, + }, + "required": ["total", "clean", "defect", "unknown", "suppressed"], + "additionalProperties": False, + }, + "rows": { + "type": "array", + "description": "One row per declared trust-decorated entity, sorted by qualname. Empty when the scan " + "produced no analysis context.", + "items": { + "type": "object", + "properties": { + "qualname": { + "type": "string", + "description": "The entity's qualified name, minted relative to the scan root.", + }, + "path": { + "type": ["string", "null"], + "description": "Source file path; null when the declared qualname has no scanned entity.", + }, + "line": { + "type": ["integer", "null"], + "description": "Entity start line; null when the declared qualname has no scanned entity.", + }, + "decorators": { + "type": "array", + "items": {"type": "string"}, + "description": "Decorators as declared, each prefixed with '@'.", + }, + "declared_tier": { + "type": ["string", "null"], + "description": "Declared return trust tier; null when undeclared.", + }, + "actual_tier": { + "type": ["string", "null"], + "description": "Engine-computed actual return taint; null when not computed.", + }, + "verdict": { + "type": "string", + "enum": ["defect", "clean", "unknown"], + "description": "Three-valued, fail-closed trust verdict for the entity.", + }, + "finding_state": { + "type": "string", + "enum": ["defect", "suppressed", "unknown", "clean"], + "description": "defect when active findings exist; suppressed when only accepted findings " + "exist; unknown when the verdict is unknown; else clean.", + }, + "active_finding_fingerprints": { + "type": "array", + "items": {"type": "string"}, + "description": "Sorted fingerprints of active (non-suppressed) defect findings on the entity.", + }, + "suppressed_finding_fingerprints": { + "type": "array", + "items": {"type": "string"}, + "description": "Sorted fingerprints of accepted (baselined/waived/judged) defect findings.", + }, + "identity": { + "type": "object", + "description": "Cross-tool identity binding for the row. Degrades to available=false with a " + "reason when Loomweave is not configured / unreachable / resolved no SEI.", + "properties": { + "available": { + "type": "boolean", + "description": "True only when an SEI was resolved for the entity.", + }, + "locator": { + "type": "string", + "description": "The Loomweave-style locator, e.g. python:function:pkg.mod.func.", + }, + "sei": { + "type": ["string", "null"], + "description": "Opaque stable entity identifier; null when not resolved.", + }, + "identity_status": { + "type": "string", + "enum": ["alive", "orphaned", "unavailable"], + "description": "Identity axis: is this the same entity?", + }, + "content_status": { + "type": "string", + "enum": ["fresh", "stale", "unknown"], + "description": "Content axis: has the entity's code changed?", + }, + "content_hash": { + "type": ["string", "null"], + "description": "Current content hash from the binding, when available.", + }, + "reason": { + "type": ["string", "null"], + "description": "Why identity is unavailable (e.g. 'loomweave not configured', 'no SEI " + "resolved'); null when an SEI was resolved.", + }, + }, + "required": [ + "available", + "locator", + "sei", + "identity_status", + "content_status", + "content_hash", + "reason", + ], + "additionalProperties": False, + }, + "work": { + "type": "object", + "description": "Open work from Filigree, keyed on the SEI. available=false with a reason when " + "Filigree is not configured / unreachable / there is no SEI binding.", + "properties": { + "available": {"type": "boolean"}, + "tickets": { + "type": "array", + "description": "Filigree issues bound to the entity.", + "items": { + "type": "object", + "properties": { + "issue_id": {"type": "string"}, + "status": {"type": ["string", "null"]}, + "priority": {"type": ["string", "null"]}, + "title": {"type": ["string", "null"]}, + "drift": { + "type": "boolean", + "description": "True when the issue was bound to a PRIOR version of the " + "entity (content hash at attach no longer matches).", + }, + }, + "required": ["issue_id", "status", "priority", "title", "drift"], + "additionalProperties": False, + }, + }, + "identity_status": {"type": "string", "enum": ["alive", "orphaned", "unavailable"]}, + "content_status": { + "type": "string", + "enum": ["fresh", "stale", "unknown"], + "description": "stale when any ticket binding drifted; unknown when a compare was " + "impossible.", + }, + "reason": { + "type": ["string", "null"], + "description": "Why the section is unavailable; null when available.", + }, + }, + "required": ["available", "tickets", "identity_status", "content_status", "reason"], + "additionalProperties": False, + }, + }, + "required": [ + "qualname", + "path", + "line", + "decorators", + "declared_tier", + "actual_tier", + "verdict", + "finding_state", + "active_finding_fingerprints", + "suppressed_finding_fingerprints", + "identity", + "work", + ], + "additionalProperties": False, + }, + }, + }, + "required": ["summary", "rows"], + "additionalProperties": False, +} + + +_DECORATOR_COVERAGE_TOOL: dict[str, Any] = { + "name": "decorator_coverage", + "title": "Trust-decorator inventory", + "description": "Stable JSON inventory of every Wardline trust-decorated entity: " + "qualname, path/line, decorators, declared/actual tier, gate verdict, " + "active/suppressed finding fingerprints, optional SEI/content status, and " + "optional Filigree linked work status. Optional sources degrade explicitly.", + "input_schema": { + "type": "object", + "properties": { + "path": {"type": "string"}, + "config": {"type": "string"}, + }, + }, + "output_schema": _DECORATOR_COVERAGE_OUTPUT_SCHEMA, + "annotations": { + "title": "Trust-decorator inventory", + "readOnlyHint": True, + "destructiveHint": False, + "idempotentHint": True, + "openWorldHint": True, + }, + "capabilities": frozenset({ToolCapability.READ, ToolCapability.NETWORK}), +} + + def _attest(args: dict[str, Any], root: Path, loomweave: Any = None) -> dict[str, Any]: """Build a SIGNED, reproducible evidence bundle for the project — identical to the CLI `attest` by construction (both call ``build_attestation``). Path/config confined @@ -561,6 +2608,207 @@ def _attest(args: dict[str, Any], root: Path, loomweave: Any = None) -> dict[str ) +_ATTEST_OUTPUT_SCHEMA: dict[str, Any] = { + "type": "object", + "description": "Signed, reproducible evidence bundle: a deterministic payload plus an HMAC-SHA256 signature under " + "the shared project key (tamper-evidence within the key-holding trust domain, not asymmetric proof).", + "properties": { + "schema": { + "type": "string", + "enum": ["wardline-attest-1"], + "description": "Wire-contract tag; bound into the HMAC so a relabel cannot verify.", + }, + "payload": { + "type": "object", + "description": "The signed, deterministic attestation payload (canonical compact key-sorted JSON is the " + "reproducibility target).", + "properties": { + "wardline_version": {"type": "string", "description": "Wardline version that produced the bundle."}, + "attested_at": { + "type": "string", + "description": "ISO date the bundle was built; re-derivation on verify uses this as `today`.", + }, + "commit": { + "type": ["string", "null"], + "description": "`git rev-parse HEAD` of the tree, or null for a non-git tree / missing git.", + }, + "dirty": { + "type": "boolean", + "description": "True iff the working tree had uncommitted changes (MCP refuses dirty trees unless " + "allow_dirty=true).", + }, + "ruleset_hash": { + "type": "string", + "description": "Deterministic 'sha256:' over the effective scan policy.", + }, + "posture": { + "type": "object", + "description": "The trust-surface coverage posture at attestation time (same shape as the " + "`assure` tool result).", + "properties": { + "boundaries_total": {"type": "integer"}, + "proven": {"type": "integer"}, + "defect_total": {"type": "integer"}, + "unknown": { + "type": "array", + "description": "Anchored entities with no definite verdict, sorted by qualname.", + "items": { + "type": "object", + "properties": { + "qualname": {"type": "string"}, + "tier": {"type": ["string", "null"]}, + "location": { + "type": "object", + "properties": { + "path": {"type": ["string", "null"]}, + "line": {"type": ["integer", "null"]}, + }, + "required": ["path", "line"], + "additionalProperties": False, + }, + "reason": {"type": ["string", "null"]}, + }, + "required": ["qualname", "tier", "location", "reason"], + "additionalProperties": False, + }, + }, + "engine_limited": {"type": "integer"}, + "coverage_pct": {"type": ["number", "null"], "description": "null when boundaries_total == 0."}, + "unanalyzed_rule_ids": {"type": "array", "items": {"type": "string"}}, + "waiver_debt": { + "type": "array", + "description": "Configured waivers with days-to-expiry (the only date-sensitive payload " + "field), sorted by fingerprint.", + "items": { + "type": "object", + "properties": { + "fingerprint": {"type": "string"}, + "expires": {"type": ["string", "null"]}, + "days_left": {"type": ["integer", "null"]}, + "reason": {"type": "string"}, + }, + "required": ["fingerprint", "expires", "days_left", "reason"], + "additionalProperties": False, + }, + }, + "baselined_total": {"type": "integer"}, + "judged_total": {"type": "integer"}, + }, + "required": [ + "boundaries_total", + "proven", + "defect_total", + "unknown", + "engine_limited", + "coverage_pct", + "unanalyzed_rule_ids", + "waiver_debt", + "baselined_total", + "judged_total", + ], + "additionalProperties": False, + }, + "boundaries": { + "type": "array", + "description": "Per-boundary trust verdicts, sorted by qualname; empty when nothing was " + "analysable.", + "items": { + "type": "object", + "properties": { + "qualname": {"type": "string", "description": "Qualified name of the anchored entity."}, + "sei": { + "type": ["string", "null"], + "description": "Loomweave SEI resolved at build time; null without a Loomweave client " + "or when unresolvable.", + }, + "verdict": { + "type": "string", + "enum": ["clean", "defect", "unknown"], + "description": "The entity's trust verdict from the single source of truth " + "(classify_entity_trust).", + }, + "tier": { + "type": ["string", "null"], + "description": "Declared trust tier, or null if undeclared.", + }, + }, + "required": ["qualname", "sei", "verdict", "tier"], + "additionalProperties": False, + }, + }, + "sei_source": { + "type": "string", + "enum": ["loomweave", "unavailable"], + "description": "'loomweave' iff a client was supplied AND at least one SEI resolved; else " + "'unavailable'.", + }, + }, + "required": [ + "wardline_version", + "attested_at", + "commit", + "dirty", + "ruleset_hash", + "posture", + "boundaries", + "sei_source", + ], + "additionalProperties": False, + }, + "signature": { + "type": "object", + "description": "HMAC-SHA256 over the canonical {schema, payload} envelope bytes.", + "properties": { + "alg": {"type": "string", "enum": ["HMAC-SHA256"]}, + "value": {"type": "string", "description": "Hex HMAC digest."}, + "key_id": { + "type": "string", + "description": "Non-secret 8-hex short id of the signing key (distinguishes keys without " + "revealing them).", + }, + }, + "required": ["alg", "value", "key_id"], + "additionalProperties": False, + }, + }, + "required": ["schema", "payload", "signature"], + "additionalProperties": False, +} + + +_ATTEST_TOOL: dict[str, Any] = { + "name": "attest", + "title": "Build signed attestation", + "description": "Build a SIGNED, reproducible evidence bundle (commit, ruleset hash, " + "trust-surface posture, boundaries) for the project. HMAC-signed with the " + "install-minted project key. Refuses a dirty working tree unless allow_dirty=true. " + "SEI-keyed when a Loomweave store is configured.", + "input_schema": { + "type": "object", + "properties": { + "path": {"type": "string"}, + "config": {"type": "string"}, + "allow_dirty": {"type": "boolean"}, + "cache_dir": { + "type": "string", + "description": "subdir relative to project root for summary cache", + }, + "trust_packs": {"type": "array", "items": {"type": "string"}}, + "trust_local_packs": {"type": "boolean"}, + "strict_defaults": {"type": "boolean"}, + }, + }, + "output_schema": _ATTEST_OUTPUT_SCHEMA, + "annotations": { + "title": "Build signed attestation", + "readOnlyHint": True, + "destructiveHint": False, + "idempotentHint": True, + "openWorldHint": False, + }, +} + + def _verify_attestation(args: dict[str, Any], root: Path, loomweave: Any = None) -> dict[str, Any]: """Verify an attestation bundle's signature (offline, needs the project key) and optionally re-derive it at the current tree (`reproduce=true`). Identical to the CLI @@ -588,6 +2836,73 @@ def _verify_attestation(args: dict[str, Any], root: Path, loomweave: Any = None) ) +_VERIFY_ATTESTATION_OUTPUT_SCHEMA: dict[str, Any] = { + "type": "object", + "description": "Result of verifying an attestation bundle: signature check (always, offline) plus optional " + "reproducibility re-derivation against the current tree.", + "properties": { + "signature_valid": { + "type": "boolean", + "description": "True iff the recomputed HMAC over the recorded payload matches the stored signature " + "(schema tag, alg, and key_id must all match too).", + }, + "reproduced": { + "type": ["boolean", "null"], + "description": "null when reproduce was not requested (or no root); true when the re-derived payload's " + "canonical bytes equal the recorded payload's; false otherwise. A false may mean the tree moved on, not " + "tamper.", + }, + "mismatches": { + "type": "array", + "description": "Top-level payload keys that differ between the recorded and re-derived payloads. Empty " + "unless reproduced is false.", + "items": {"type": "string"}, + }, + "note": { + "type": "string", + "description": "Fixed caveat: reproducibility holds against the RECORDED commit; a mismatch may mean the " + "tree moved, not tamper.", + }, + }, + "required": ["signature_valid", "reproduced", "mismatches", "note"], + "additionalProperties": False, +} + + +_VERIFY_ATTESTATION_TOOL: dict[str, Any] = { + "name": "verify_attestation", + "title": "Verify attestation bundle", + "description": "Verify an attestation bundle's signature (offline, needs the project " + "key) and optionally its reproducibility (reproduce=true re-derives at the current " + "tree). Returns {signature_valid, reproduced, mismatches, note}.", + "input_schema": { + "type": "object", + "required": ["bundle"], + "properties": { + "bundle": {"type": "object"}, + "reproduce": {"type": "boolean"}, + "config": {"type": "string"}, + "path": {"type": "string"}, + "cache_dir": { + "type": "string", + "description": "subdir relative to project root for summary cache", + }, + "trust_packs": {"type": "array", "items": {"type": "string"}}, + "trust_local_packs": {"type": "boolean"}, + "strict_defaults": {"type": "boolean"}, + }, + }, + "output_schema": _VERIFY_ATTESTATION_OUTPUT_SCHEMA, + "annotations": { + "title": "Verify attestation bundle", + "readOnlyHint": True, + "destructiveHint": False, + "idempotentHint": True, + "openWorldHint": False, + }, +} + + def _judge(args: dict[str, Any], root: Path) -> dict[str, Any]: # No key/.env → run_judge's default caller raises JudgeConfigurationError (a # WardlineError) naming WARDLINE_OPENROUTER_API_KEY; _tools_call turns that into @@ -626,6 +2941,87 @@ def _judge(args: dict[str, Any], root: Path) -> dict[str, Any]: } +_JUDGE_OUTPUT_SCHEMA: dict[str, Any] = { + "type": "object", + "description": "Success payload of the judge tool: per-finding LLM triage verdicts plus the persistence outcome " + "of FALSE_POSITIVE records.", + "properties": { + "verdicts": { + "type": "array", + "description": "One flattened verdict per triaged finding.", + "items": { + "type": "object", + "properties": { + "fingerprint": {"type": "string", "description": "Stable fingerprint of the judged finding."}, + "rule_id": {"type": "string", "description": "Rule that produced the finding."}, + "path": {"type": "string", "description": "Repo-relative file path of the finding."}, + "line": {"type": ["integer", "null"], "description": "1-based start line; null when unknown."}, + "label": { + "type": "string", + "enum": ["TRUE_POSITIVE", "FALSE_POSITIVE"], + "description": "The judge's verdict: a real defect vs an analyzer over-approximation.", + }, + "confidence": { + "type": "number", + "description": "The judge's calibrated confidence in the verdict, 0.0 to 1.0.", + }, + "rationale": { + "type": "string", + "description": "The model's verbatim reasoning (the audit primitive); always a non-empty " + "string.", + }, + }, + "required": ["fingerprint", "rule_id", "path", "line", "label", "confidence", "rationale"], + "additionalProperties": False, + }, + }, + "wrote": { + "type": "integer", + "description": "FALSE_POSITIVE verdicts persisted to .wardline/judged.yaml (0 unless write=true).", + }, + "held_back": { + "type": "integer", + "description": "FALSE_POSITIVE verdicts NOT persisted because their confidence fell below the write floor.", + }, + }, + "required": ["verdicts", "wrote", "held_back"], + "additionalProperties": False, +} + + +_JUDGE_TOOL: dict[str, Any] = { + "name": "judge", + "title": "LLM triage of findings", + "description": "NETWORK: opt-in LLM triage of active defects via OpenRouter " + "(needs WARDLINE_OPENROUTER_API_KEY). Labels each TRUE/FALSE positive. " + "Never run automatically; never folded into scan.", + "input_schema": { + "type": "object", + "properties": { + "config": {"type": "string"}, + "model": {"type": "string"}, + "max_findings": {"type": "integer"}, + "write": {"type": "boolean", "description": "append above-floor FPs to judged.yaml"}, + "trust_judge_config": {"type": "boolean"}, + "trust_judge_policy": {"type": "boolean"}, + "trust_packs": {"type": "array", "items": {"type": "string"}}, + "trust_local_packs": {"type": "boolean"}, + "strict_defaults": {"type": "boolean"}, + "context_lines": {"type": "integer"}, + }, + }, + "output_schema": _JUDGE_OUTPUT_SCHEMA, + "annotations": { + "title": "LLM triage of findings", + "readOnlyHint": False, + "destructiveHint": False, + "idempotentHint": False, + "openWorldHint": True, + }, + "capabilities": frozenset({ToolCapability.READ, ToolCapability.NETWORK}), +} + + def _baseline(args: dict[str, Any], root: Path) -> dict[str, Any]: reason = args.get("reason") baseline_path = baseline_file(root) @@ -657,6 +3053,68 @@ def _baseline(args: dict[str, Any], root: Path) -> dict[str, Any]: return payload +_BASELINE_OUTPUT_SCHEMA: dict[str, Any] = { + "type": "object", + "description": "Success payload of the wardline MCP `baseline` tool: records the result of generating (or finding " + "an existing) suppression baseline for the project.", + "properties": { + "baselined_count": { + "type": "integer", + "description": "Number of finding fingerprints in the baseline. On a fresh generation this is the count " + "just written; when the baseline already existed (already_exists=true) it is the count in the existing " + "file.", + }, + "path": {"type": "string", "description": "Absolute path of the baseline file."}, + "reason": { + "type": ["string", "null"], + "description": "Caller-supplied reason for creating the baseline; null when not provided.", + }, + "already_exists": { + "type": "boolean", + "description": "Only present when overwrite was not requested: true if a baseline file already existed " + "(nothing was written; counts reflect the existing file), false if a new baseline was created. Absent " + "when overwrite=true succeeds.", + }, + }, + "required": ["baselined_count", "path", "reason"], + "additionalProperties": False, +} + + +_BASELINE_TOOL: dict[str, Any] = { + "name": "baseline", + "title": "Snapshot findings baseline", + "description": "Snapshot current defects as the baseline so only NEW findings surface. " + "Default overwrite=false refuses to clobber and returns already_exists=true. " + "Set overwrite=true to re-derive and overwrite the baseline. " + "Prefer FIXING a finding over baselining it. Optional reason.", + "input_schema": { + "type": "object", + "properties": { + "reason": {"type": "string"}, + "overwrite": {"type": "boolean"}, + "config": {"type": "string"}, + "cache_dir": { + "type": "string", + "description": "subdir relative to project root for summary cache", + }, + "trust_packs": {"type": "array", "items": {"type": "string"}}, + "trust_local_packs": {"type": "boolean"}, + "strict_defaults": {"type": "boolean"}, + }, + }, + "output_schema": _BASELINE_OUTPUT_SCHEMA, + "annotations": { + "title": "Snapshot findings baseline", + "readOnlyHint": False, + "destructiveHint": False, + "idempotentHint": True, + "openWorldHint": False, + }, + "capabilities": frozenset({ToolCapability.READ, ToolCapability.WRITE}), +} + + def _waiver_add(args: dict[str, Any], root: Path) -> dict[str, Any]: fp = _require(args, "fingerprint") reason = _require(args, "reason") @@ -685,6 +3143,59 @@ def _waiver_add(args: dict[str, Any], root: Path) -> dict[str, Any]: } +_WAIVER_ADD_OUTPUT_SCHEMA: dict[str, Any] = { + "type": "object", + "description": "Success payload of the wardline MCP `waiver_add` tool: the waiver that now covers the fingerprint " + "(newly added, or the pre-existing one when a waiver for the fingerprint was already present).", + "properties": { + "fingerprint": {"type": "string", "description": "The finding fingerprint the waiver covers."}, + "reason": { + "type": "string", + "description": "The waiver's reason. When already_exists=true this is the EXISTING waiver's reason, which " + "may differ from the one passed in this call.", + }, + "expires": { + "type": ["string", "null"], + "description": "Waiver expiry as an ISO date (YYYY-MM-DD). Null only when a pre-existing waiver loaded " + "from the waivers file carries no expiry (the tool itself requires expires on new waivers).", + }, + "already_exists": { + "type": "boolean", + "description": "True when a waiver for this fingerprint already existed and the returned fields describe " + "that existing waiver; false when this call added a new waiver.", + }, + }, + "required": ["fingerprint", "reason", "expires", "already_exists"], + "additionalProperties": False, +} + + +_WAIVER_ADD_TOOL: dict[str, Any] = { + "name": "waiver_add", + "title": "Add time-boxed waiver", + "description": "Waive ONE finding by fingerprint with a mandatory reason and expiry. " + "Prefer fixing; a waiver is an audited, time-boxed exception.", + "input_schema": { + "type": "object", + "required": ["fingerprint", "reason", "expires"], + "properties": { + "fingerprint": {"type": "string"}, + "reason": {"type": "string"}, + "expires": {"type": "string", "description": "YYYY-MM-DD"}, + }, + }, + "output_schema": _WAIVER_ADD_OUTPUT_SCHEMA, + "annotations": { + "title": "Add time-boxed waiver", + "readOnlyHint": False, + "destructiveHint": False, + "idempotentHint": True, + "openWorldHint": False, + }, + "capabilities": frozenset({ToolCapability.READ, ToolCapability.WRITE}), +} + + def _doctor( args: dict[str, Any], root: Path, @@ -708,6 +3219,131 @@ def _doctor( return attach_server_identity(payload, root=root, started_at=started_at) +_DOCTOR_OUTPUT_SCHEMA: dict[str, Any] = { + "type": "object", + "description": "Doctor success payload: the CLI `doctor --fix` machine-readable envelope (install/federation " + "health checks) plus the running MCP server's self-identification block and a `server.freshness` check appended " + "to the check list.", + "properties": { + "ok": { + "type": "boolean", + "description": "True only when every check (including the appended server.freshness check) passed.", + }, + "checks": { + "type": "array", + "description": "Uniform health-check verdicts: wardline.config, mcp.registration, marker_package, " + "loomweave.url, filigree.url, decorator_grammar, scan.output_path, auth.token, filigree.auth, then " + "server.freshness last.", + "items": { + "type": "object", + "properties": { + "id": { + "type": "string", + "description": "Stable check identifier, e.g. 'wardline.config' or 'server.freshness'.", + }, + "status": {"type": "string", "enum": ["ok", "error"], "description": "Check verdict."}, + "fixed": { + "type": "boolean", + "description": "True when this run's repair (repair: true) corrected the underlying condition.", + }, + "message": { + "type": "string", + "description": "Human/agent-readable detail. Present only when the check produced a non-empty " + "message (always on errors; sometimes on informational ok results).", + }, + }, + "required": ["id", "status", "fixed"], + "additionalProperties": False, + }, + }, + "next_actions": { + "type": "array", + "description": "One ': ' action line per failed check that carries a message; empty " + "when everything is healthy.", + "items": {"type": "string"}, + }, + "server": { + "type": "object", + "description": "The running MCP server's self-identification: detects a stale long-lived server (source " + "on disk newer than process start).", + "properties": { + "package_version": {"type": "string", "description": "Installed wardline package version."}, + "pid": {"type": "integer", "description": "Server process id."}, + "project_root": {"type": "string", "description": "Absolute project root this server is confined to."}, + "started_at": {"type": "string", "description": "ISO-8601 UTC timestamp of server process start."}, + "source_latest_mtime": { + "type": ["string", "null"], + "description": "ISO-8601 UTC mtime of the newest *.py file under the imported wardline package, " + "or null when nothing was statable.", + }, + "source_latest_path": { + "type": ["string", "null"], + "description": "Package-relative path of that newest source file, or null.", + }, + "fresh": { + "type": "boolean", + "description": "False when on-disk package source changed after this process started — the server " + "is serving OLD code and must be restarted.", + }, + }, + "required": [ + "package_version", + "pid", + "project_root", + "started_at", + "source_latest_mtime", + "source_latest_path", + "fresh", + ], + "additionalProperties": False, + }, + }, + "required": ["ok", "checks", "next_actions", "server"], + "additionalProperties": False, +} + + +_DOCTOR_TOOL: dict[str, Any] = { + "name": "doctor", + "title": "Install and federation health check", + "description": "Health-check the wardline install and federation wiring " + "(same checks as CLI `doctor --fix`: instruction blocks, skills, MCP " + "registration, config parseability, sibling URLs, Filigree emit auth) " + "PLUS this server's self-identification: package version, pid, start " + "time, and a source-FRESHNESS verdict. If `server.fresh` is false this " + "long-lived server predates the on-disk wardline code — its results are " + "stale; restart the MCP server. Read-only by default; `repair: true` " + "(write-gated) repairs install artifacts and re-pins a rejected " + "federation token.", + "input_schema": { + "type": "object", + "properties": { + "repair": { + "type": "boolean", + "description": "Default false (pure probe, writes nothing). true repairs " + "install artifacts (CLAUDE.md/AGENTS.md blocks, .claude/.agents skills, " + ".mcp.json + Codex registration, .weft state dir) and, when Filigree " + "rejected the emit token, re-pins the accepted local mint in .env.", + }, + "filigree_url": { + "type": "string", + "description": "Filigree URL to probe for emit auth (default: the server's " + "configured URL, then WARDLINE_FILIGREE_URL, then the .mcp.json arg). " + "Only loopback origins are ever probed with a token.", + }, + }, + }, + "output_schema": _DOCTOR_OUTPUT_SCHEMA, + "annotations": { + "title": "Install and federation health check", + "readOnlyHint": False, + "destructiveHint": False, + "idempotentHint": True, + "openWorldHint": False, + }, +} + + def _rekey(args: dict[str, Any], root: Path, filigree: Any = None) -> dict[str, Any]: """Fingerprint-scheme migration over MCP (A3): the same core.rekey the CLI drives — no second migration path. Probe-by-default (read-only: report match/orphans/ @@ -795,6 +3431,212 @@ def journal_block(journal: Journal) -> dict[str, Any]: return {"mode": "apply", **journal_block(run_rekey(path, result.findings, filigree=filigree))} +_REKEY_OUTPUT_SCHEMA: dict[str, Any] = { + "type": "object", + "description": "Rekey success payload. Four shapes share the 'mode' discriminator: 'probe' (default, read-only " + "dry run), 'apply' and 'resume' (journal block reporting the migration's per-leg state), and 'rollback' ({mode, " + "restored, note}). All non-'mode' fields are mode-specific.", + "properties": { + "mode": { + "type": "string", + "enum": ["probe", "apply", "resume", "rollback"], + "description": "Which rekey operation ran. 'probe' is the read-only default; 'apply'/'resume'/'rollback' " + "are explicit, mutually exclusive write modes.", + }, + "scanned_findings": { + "type": "integer", + "description": "probe only: number of findings the suppression-free scan produced (each carries an " + "old/new fingerprint pair).", + }, + "matched": { + "type": "integer", + "description": "probe only: count of distinct stored old-scheme fingerprints that map to a current " + "finding (will carry).", + }, + "orphaned": { + "type": "array", + "items": {"type": "string"}, + "description": "probe only: sorted stored fingerprints with NO current finding — these verdicts would be " + "dropped by an apply.", + }, + "orphan_cause": { + "type": "string", + "description": "probe/apply/resume: fixed explanation string for why a stored fingerprint can orphan " + "(source moved/deleted, or a custom multi-emit rule not surfacing taint_path_v0).", + }, + "collisions": { + "type": "array", + "description": "probe/apply/resume: pre-rekey fingerprints that collapse to the same new fingerprint " + "under the new scheme; all involved old fingerprints are orphaned, not carried.", + "items": { + "type": "object", + "properties": { + "new_fp": { + "type": "string", + "description": "The new-scheme fingerprint that more than one old fingerprint maps onto.", + }, + "old_fps": { + "type": "array", + "items": {"type": "string"}, + "description": "The colliding pre-rekey fingerprints (sorted).", + }, + "message": { + "type": "string", + "description": "WLN-ENGINE-FINGERPRINT-COLLISION diagnostic explaining that the colliding " + "verdicts are orphaned.", + }, + }, + "required": ["new_fp", "old_fps", "message"], + "additionalProperties": False, + }, + }, + "per_store": { + "type": "object", + "description": "probe only: open map of store file name (e.g. 'baseline.yaml') -> count of its stored " + "fingerprints with no current finding. Only stores with orphans appear.", + "additionalProperties": {"type": "integer"}, + }, + "prescheme": { + "type": "boolean", + "description": "probe only: true when a live store predates the fingerprint-scheme stamp, so orphans MAY " + "be a fingerprint-formula change rather than source churn.", + }, + "clean": { + "type": "boolean", + "description": "probe only: true when there are no orphans and no collisions — an apply would carry every " + "stored verdict.", + }, + "complete": {"type": "boolean", "description": "apply/resume: true when every migration leg is done."}, + "fingerprint_scheme_from": { + "type": "string", + "description": "apply/resume: the fingerprint scheme being migrated from (e.g. 'wlfp1').", + }, + "fingerprint_scheme_to": { + "type": "string", + "description": "apply/resume: the fingerprint scheme being migrated to (e.g. 'wlfp2').", + }, + "snapshot_prescheme": { + "type": "boolean", + "description": "apply/resume: true when the snapshotted stores carried no scheme stamp (pre-scheme), so " + "orphans may be a formula change — surfaced as a caution.", + }, + "legs": { + "type": "array", + "description": "apply/resume: per-leg migration state, in application order (baseline, judged, waivers, " + "filigree).", + "items": { + "type": "object", + "properties": { + "name": { + "type": "string", + "description": "Leg name: one of 'baseline', 'judged', 'waivers', 'filigree' in a journal " + "this code wrote.", + }, + "done": {"type": "boolean", "description": "True when this leg has been applied and persisted."}, + "carried_count": { + "type": "integer", + "description": "Number of stored verdicts re-keyed and carried forward by this leg (count " + "only; can be the whole store).", + }, + "orphaned": { + "type": "array", + "items": {"type": "string"}, + "description": "Verbatim old fingerprints this leg dropped — a dropped verdict is never " + "silent.", + }, + "debt": { + "type": ["string", "null"], + "description": "Filigree leg only: recorded reconciliation debt when the leg soft-failed " + "(re-run with apply to retry); null otherwise.", + }, + }, + "required": ["name", "done", "carried_count", "orphaned", "debt"], + "additionalProperties": False, + }, + }, + "next_pending_leg": { + "type": ["string", "null"], + "description": "apply/resume: name of the first not-done leg, or null when the migration is complete.", + }, + "next_action": { + "type": "string", + "description": "apply/resume, only when the migration is INCOMPLETE: instruction to re-run the rekey tool " + "to finish pending leg(s).", + }, + "restored": { + "type": "array", + "items": {"type": "string"}, + "description": "rollback only: store file names restored from the pre-migration snapshot.", + }, + "note": { + "type": "string", + "description": "rollback only: caveat that Filigree associations from the forward run are NOT reversed " + "(no remap endpoint); reconcile manually if needed.", + }, + }, + "required": ["mode"], + "additionalProperties": False, +} + + +_REKEY_TOOL: dict[str, Any] = { + "name": "rekey", + "title": "Migrate fingerprint scheme", + "description": "Re-key baseline/waiver/judged verdicts across a fingerprint-scheme " + "change (after the engine's fingerprint formula migrates, NOT after ordinary " + "refactors — fingerprints are line-insensitive). DEFAULT is a read-only PROBE: " + "reports how many stored verdicts will carry, which orphan and why, and any " + "collisions — writes nothing. Pass `apply: true` to migrate (snapshots first, " + "resumable journal), `resume: true` to finish an interrupted migration without " + "re-scanning, `rollback: true` to restore the pre-migration stores. The three " + "are mutually exclusive and write-gated.", + "input_schema": { + "type": "object", + "properties": { + "path": {"type": "string", "description": "subdir relative to project root"}, + "config": {"type": "string"}, + "cache_dir": { + "type": "string", + "description": "subdir relative to project root for summary cache", + }, + "trust_packs": {"type": "array", "items": {"type": "string"}}, + "trust_local_packs": { + "type": "boolean", + "description": "Allow loading custom trust-grammar packs from the local project directory", + }, + "strict_defaults": { + "type": "boolean", + "description": "Ignore repository-supplied custom configuration overrides (weft.toml)", + }, + "apply": { + "type": "boolean", + "description": "Default false (read-only probe). true RUNS the migration: " + "snapshot stores, write the resumable journal, carry verdicts to the " + "new fingerprints, best-effort re-emit to a configured Filigree.", + }, + "resume": { + "type": "boolean", + "description": "Finish an interrupted migration from the journal WITHOUT " + "re-scanning (YAML legs re-carry from the snapshot).", + }, + "rollback": { + "type": "boolean", + "description": "Restore the pre-migration stores byte-identical from the " + "snapshot and remove the journal. Filigree associations are NOT reversed.", + }, + }, + }, + "output_schema": _REKEY_OUTPUT_SCHEMA, + "annotations": { + "title": "Migrate fingerprint scheme", + "readOnlyHint": False, + "destructiveHint": True, + "idempotentHint": False, + "openWorldHint": False, + }, +} + + def _fix(args: dict[str, Any], root: Path) -> dict[str, Any]: """Scan the path and apply mechanical autofixes to findings in-place.""" path = _resolve_under_root(root, args["path"]) if args.get("path") else root @@ -823,19 +3665,60 @@ def _fix(args: dict[str, Any], root: Path) -> dict[str, Any]: } -# Gate thresholds are the four defect severities. Severity also defines NONE -# (the "facts carry no defect severity" sentinel), deliberately excluded here: -# fail_on=NONE is not a meaningful gate threshold. -_SEVERITY_ENUM = ["CRITICAL", "ERROR", "WARN", "INFO"] +_FIX_OUTPUT_SCHEMA: dict[str, Any] = { + "type": "object", + "description": "Success payload of the wardline MCP `fix` tool: mechanical autofix results (PY-WL-111 " + "assert-at-boundary rewrites), either previewed (default / dry_run) or applied in-place (apply=true).", + "properties": { + "fixed": { + "type": "object", + "description": "Open map of file path (relative to the scanned `path` argument, NOT the project root) " + "-> list of human-readable descriptions of the fixes previewed/applied in that file. Empty object when " + "no fixable findings were found or no fixes could be produced.", + "additionalProperties": { + "type": "array", + "items": {"type": "string", "description": "Description of one fix in this file."}, + }, + }, + "applied": { + "type": "boolean", + "description": "True when fixes were written to disk (apply=true and not dry_run), false when this was a " + "preview. Absent on the early-return path where the scan found no fixable findings.", + }, + "message": { + "type": "string", + "description": "Human-readable summary, e.g. 'No fixable findings found.', 'Previewed fixes for N files.' " + "or 'Applied fixes for N files.'", + }, + }, + "required": ["fixed", "message"], + "additionalProperties": False, +} -# Default ceiling on the number of active-defect provenances inlined by `explain: true` -# on the MCP `scan`. Bounds the one-shot payload (the dogfood report hit 56,820 chars on -# one line over a whole repo); an explicit `max_findings` tightens it further. -_EXPLAIN_DEFAULT_CAP = 10 -# The bounded-default page size for `scan` (weft-439d09fc8d). A bare scan returns at most -# this many finding bodies so an agent's first natural call cannot overflow its context; -# full=true lifts the cap and offset pages through the rest. -_DEFAULT_MAX_FINDINGS = 25 + +_FIX_TOOL: dict[str, Any] = { + "name": "fix", + "title": "Apply mechanical autofixes", + "description": "Scan and apply mechanical autofixes to findings (currently only PY-WL-111 is supported).", + "input_schema": { + "type": "object", + "properties": { + "path": {"type": "string", "description": "subdir relative to project root to scan and fix"}, + "config": {"type": "string"}, + "dry_run": {"type": "boolean", "description": "preview changes without modifying files"}, + "apply": {"type": "boolean", "description": "must be true to modify files"}, + }, + }, + "output_schema": _FIX_OUTPUT_SCHEMA, + "annotations": { + "title": "Apply mechanical autofixes", + "readOnlyHint": False, + "destructiveHint": True, + "idempotentHint": True, + "openWorldHint": False, + }, + "capabilities": frozenset({ToolCapability.READ, ToolCapability.WRITE}), +} class WardlineMCPServer: @@ -924,141 +3807,12 @@ def _filigree_filer( return FiligreeIssueFiler(url, token=load_filigree_token(self.root)) def _register_tools(self) -> None: + # Each tool's full declaration (schemas, annotations, capabilities) lives + # module-level next to its handler; registration just binds it to the + # injected-client lambda. Order is the published surface order. self.add_tool( Tool( - name="scan", - description="Whole-program taint scan of the project. Returns structured " - "findings, the suppression summary (active = unsuppressed defects; " - "by default the --fail-on gate evaluates the UNSUPPRESSED population so " - "repo-controlled baseline/waiver/judged annotate but do not clear it — " - "pass `trust_suppressions: true` for the trusted-local behaviour), " - "and the gate verdict. Pass `where` to filter the returned findings " - "(conjunctive; summary/gate stay whole-project) and `explain: true` to inline " - "each active defect's taint provenance — one call, no per-finding explain_taint. " - "When a Filigree URL is configured, also POSTs the " - "findings to Filigree (fail-soft: an unreachable sibling or rejected payload " - "is reported in the `filigree` block, never fails the scan).", - input_schema={ - "type": "object", - "properties": { - "path": {"type": "string", "description": "subdir relative to project root"}, - "fail_on": {"type": "string", "enum": _SEVERITY_ENUM}, - "fail_on_unanalyzed": { - "type": "boolean", - "description": "Also trip the gate when any file was discovered but could " - "not be analyzed (parse error / too-deep skip / missing source root; benign " - "no-module skips excluded). Default false — same default as the CLI's " - "--fail-on-unanalyzed; summary.unanalyzed always reports the count either " - "way, and gate.unanalyzed_tripped attributes a trip to this knob.", - }, - "config": {"type": "string"}, - "lang": { - "type": "string", - "enum": ["python", "rust"], - "description": "Language frontend (default python). 'rust' sweeps .rs files " - "for the command-injection slice (RS-WL-108 program injection / RS-WL-112 " - "shell injection; frozen identity, baseline-eligible). Preview posture: " - "weft.toml severity overrides do not yet apply to Rust findings, and a tree " - "with no /// @trusted markers is vacuously green — read the WLN-RUST-COVERAGE " - "fact before trusting '0 active'. Requires the wardline[rust] extra.", - }, - "where": { - "type": "object", - "description": "Filter the returned findings (conjunctive). Keys: " - "rule_id, qualname, severity, suppression, kind, path_glob, sink, tier. " - "summary/gate still describe the whole project.", - "properties": { - "rule_id": {"type": "string"}, - "qualname": {"type": "string"}, - "severity": {"type": "string", "enum": _SEVERITY_ENUM}, - "suppression": {"type": "string", "enum": ["active", "baselined", "waived", "judged"]}, - "kind": { - "type": "string", - "enum": ["defect", "fact", "classification", "metric", "suggestion"], - }, - "path_glob": {"type": "string"}, - "sink": {"type": "string"}, - "tier": {"type": "string"}, - }, - }, - "explain": { - "type": "boolean", - "description": "Inline each active defect's taint provenance " - "(immediate tainted callee, source boundary, trust tiers, resolution " - "counts) — one call instead of an explain_taint per finding. Inlining is " - "capped at 10 provenances by default (raise/lower with max_findings); the cut " - "is reported at agent_summary.truncation.explanations_truncated.", - }, - "summary_only": { - "type": "boolean", - "description": "Return counts + gate only, no finding bodies — the smallest " - "'did the gate pass?' payload. summary/gate still describe the whole project.", - }, - "full": { - "type": "boolean", - "description": "Default false. The default scan is BOUNDED (≤25 finding bodies) so " - "it cannot overflow your context; set full=true to return ALL bodies in one call " - "(or page with offset). summary/gate counts are always whole-project.", - }, - "max_findings": { - "type": "integer", - "minimum": 0, - "description": "Override the page size for returned finding bodies (and the inlined-" - "explanation cap). Default 25; full=true ignores it. Must be a non-negative integer. " - "The cut + next page are reported in agent_summary.truncation; counts stay whole.", - }, - "offset": { - "type": "integer", - "minimum": 0, - "description": "Pagination cursor into the ordered finding union (active → suppressed " - "→ engine_facts → informational). Pass agent_summary.truncation.next_offset from " - "the previous call to fetch the next page. Default 0.", - }, - "include_suppressed": { - "type": "boolean", - "description": "Default true. Set false to drop suppressed (baselined/waived/" - "judged) finding bodies from the response; the suppression counts stay in " - "summary.", - }, - "new_since": { - "type": "string", - "description": "PR-scoped 'new findings only' gate: only gate on findings in " - "files/entities changed since this git ref", - }, - "cache_dir": { - "type": "string", - "description": "subdir relative to project root for summary cache", - }, - "trust_packs": {"type": "array", "items": {"type": "string"}}, - "trust_local_packs": { - "type": "boolean", - "description": "Allow loading custom trust-grammar packs from the local project directory", - }, - "strict_defaults": { - "type": "boolean", - "description": "Ignore repository-supplied custom configuration overrides (weft.toml)", - }, - "trust_suppressions": { - "type": "boolean", - "description": "Let repository-controlled baseline/waiver/judged clear the gate " - "(they always annotate findings regardless). Default false — the gate " - "evaluates the unsuppressed population so a PR cannot self-suppress its " - "own defect. Use only on a trusted checkout; in CI prefer new_since.", - }, - "legis_artifact": { - "type": "boolean", - "description": "Attach the verbatim-postable legis scan-artifact " - "(`legis_artifact` block) even when no signing key is provisioned " - "(unsigned, for legis's optional-verify posture).", - }, - "allow_dirty": { - "type": "boolean", - "description": "For the legis artifact only: on a dirty tree emit an UNSIGNED, " - "clearly-marked (dirty: true) dev artifact instead of refusing to sign. " - "Signing stays clean-tree-only; legis records it unverified.", - }, - }, - }, + **_SCAN_TOOL, handler=lambda args, root: _scan( args, root, @@ -1075,48 +3829,13 @@ def _register_tools(self) -> None: ) self.add_tool( Tool( - name="explain_taint", - description="Explain ONE finding's taint: the immediate tainted callee, the " - "originating boundary, and the trust tiers at the sink. Call right " - "after scan and before editing — a stale fingerprint returns an error. " - "Pass the finding's `qualname` as `sink_qualname`: when a Loomweave store " - "is configured this serves the explanation from the store instead of " - "re-scanning. Pass `chain: true` (needs a configured Loomweave store) to " - "also walk the full taint chain from the sink to the originating boundary; " - "without a store it degrades to the single-hop explanation (no `chain` block).", - input_schema={ - "type": "object", - "properties": { - "fingerprint": {"type": "string"}, - "path": {"type": "string"}, - "line": {"type": "integer"}, - "sink_qualname": {"type": "string"}, - "chain": {"type": "boolean"}, - "max_hops": {"type": "integer"}, - "config": {"type": "string"}, - }, - }, + **_EXPLAIN_TAINT_TOOL, handler=lambda args, root: _explain_taint(args, root, self._loomweave_client(_cfg(args, root))), ) ) self.add_tool( Tool( - name="dossier", - description="One-call entity dossier for a function `entity` (a qualname): its " - "trust posture (declared vs actual taint, gate verdict, active findings — always " - "computed fresh), plus Loomweave call-graph linkages and Filigree open work joined on " - "the entity's opaque SEI. Every cross-tool section is freshness-stamped on BOTH axes " - "(identity alive/orphaned/unavailable + content fresh/stale/unknown) and degrades to " - "an honest `unavailable` when its source is absent. Token-bounded (~2k) with an " - "explicit truncation marker. Read the whole context without opening the source.", - input_schema={ - "type": "object", - "required": ["entity"], - "properties": { - "entity": {"type": "string", "description": "the function qualname, e.g. pkg.mod.func"}, - "config": {"type": "string"}, - }, - }, + **_DOSSIER_TOOL, handler=lambda args, root: _dossier( args, root, @@ -1127,35 +3846,13 @@ def _register_tools(self) -> None: ) self.add_tool( Tool( - name="assure", - description="Trust-surface COVERAGE posture: how many declared trust boundaries the " - "engine reached a definite verdict on vs. how many are honestly unknown, plus " - "waiver-debt. Consult before deciding to trust a module.", - input_schema={ - "type": "object", - "properties": { - "path": {"type": "string"}, - "config": {"type": "string"}, - }, - }, + **_ASSURE_TOOL, handler=lambda args, root: _assure(args, root), ) ) self.add_tool( Tool( - name="decorator_coverage", - capabilities=frozenset({ToolCapability.READ, ToolCapability.NETWORK}), - description="Stable JSON inventory of every Wardline trust-decorated entity: " - "qualname, path/line, decorators, declared/actual tier, gate verdict, " - "active/suppressed finding fingerprints, optional SEI/content status, and " - "optional Filigree linked work status. Optional sources degrade explicitly.", - input_schema={ - "type": "object", - "properties": { - "path": {"type": "string"}, - "config": {"type": "string"}, - }, - }, + **_DECORATOR_COVERAGE_TOOL, handler=lambda args, root: _decorator_coverage( args, root, @@ -1166,26 +3863,7 @@ def _register_tools(self) -> None: ) self.add_tool( Tool( - name="attest", - description="Build a SIGNED, reproducible evidence bundle (commit, ruleset hash, " - "trust-surface posture, boundaries) for the project. HMAC-signed with the " - "install-minted project key. Refuses a dirty working tree unless allow_dirty=true. " - "SEI-keyed when a Loomweave store is configured.", - input_schema={ - "type": "object", - "properties": { - "path": {"type": "string"}, - "config": {"type": "string"}, - "allow_dirty": {"type": "boolean"}, - "cache_dir": { - "type": "string", - "description": "subdir relative to project root for summary cache", - }, - "trust_packs": {"type": "array", "items": {"type": "string"}}, - "trust_local_packs": {"type": "boolean"}, - "strict_defaults": {"type": "boolean"}, - }, - }, + **_ATTEST_TOOL, handler=lambda args, root: _attest( args, root, @@ -1197,27 +3875,7 @@ def _register_tools(self) -> None: ) self.add_tool( Tool( - name="verify_attestation", - description="Verify an attestation bundle's signature (offline, needs the project " - "key) and optionally its reproducibility (reproduce=true re-derives at the current " - "tree). Returns {signature_valid, reproduced, mismatches, note}.", - input_schema={ - "type": "object", - "required": ["bundle"], - "properties": { - "bundle": {"type": "object"}, - "reproduce": {"type": "boolean"}, - "config": {"type": "string"}, - "path": {"type": "string"}, - "cache_dir": { - "type": "string", - "description": "subdir relative to project root for summary cache", - }, - "trust_packs": {"type": "array", "items": {"type": "string"}}, - "trust_local_packs": {"type": "boolean"}, - "strict_defaults": {"type": "boolean"}, - }, - }, + **_VERIFY_ATTESTATION_TOOL, handler=lambda args, root: _verify_attestation( args, root, @@ -1229,30 +3887,7 @@ def _register_tools(self) -> None: ) self.add_tool( Tool( - name="file_finding", - capabilities=frozenset({ToolCapability.READ, ToolCapability.WRITE, ToolCapability.NETWORK}), - description="File ONE finding (by `fingerprint`) into a tracked Filigree issue and " - "return its `issue_id`. Idempotent (re-filing returns the same issue). Emit findings " - "to Filigree first (scan with a configured Filigree URL) so the fingerprint is known; " - "a `not_found: true` result means it isn't. Reconciliation (close-on-fixed / " - "reopen-on-regress) happens automatically on later scans. Fail-soft.", - input_schema={ - "type": "object", - "required": ["fingerprint"], - "properties": { - "fingerprint": {"type": "string"}, - "priority": {"type": "string", "description": "Filigree priority, e.g. P2"}, - "labels": {"type": "array", "items": {"type": "string"}}, - "attach_loomweave_identity": { - "type": "boolean", - "description": ( - "Opt in to resolving the finding qualname through Loomweave and attaching " - "a Filigree entity association." - ), - }, - "config": {"type": "string"}, - }, - }, + **_FILE_FINDING_TOOL, handler=lambda args, root: _file_finding( args, root, @@ -1265,32 +3900,7 @@ def _register_tools(self) -> None: ) self.add_tool( Tool( - name="scan_file_findings", - capabilities=frozenset({ToolCapability.READ, ToolCapability.WRITE, ToolCapability.NETWORK}), - description="One-shot agent workflow: run a scan, list active defects first with " - "inline explanation summaries, optionally emit to Filigree, promote selected " - "fingerprints or all active defects, and attach Loomweave identity when available. " - "Defaults to dry-run unless fingerprints or all_active are supplied.", - input_schema={ - "type": "object", - "properties": { - "path": {"type": "string", "description": "subdir relative to project root"}, - "fail_on": {"type": "string", "enum": _SEVERITY_ENUM}, - "config": {"type": "string"}, - "cache_dir": { - "type": "string", - "description": "subdir relative to project root for summary cache", - }, - "fingerprints": {"type": "array", "items": {"type": "string"}}, - "all_active": {"type": "boolean"}, - "dry_run": {"type": "boolean"}, - "priority": {"type": "string", "description": "Filigree priority, e.g. P2"}, - "labels": {"type": "array", "items": {"type": "string"}}, - "trust_packs": {"type": "array", "items": {"type": "string"}}, - "trust_local_packs": {"type": "boolean"}, - "strict_defaults": {"type": "boolean"}, - }, - }, + **_SCAN_FILE_FINDINGS_TOOL, handler=lambda args, root: _scan_file_findings( args, root, @@ -1306,120 +3916,31 @@ def _register_tools(self) -> None: ) self.add_tool( Tool( - name="judge", - capabilities=frozenset({ToolCapability.READ, ToolCapability.NETWORK}), - description="NETWORK: opt-in LLM triage of active defects via OpenRouter " - "(needs WARDLINE_OPENROUTER_API_KEY). Labels each TRUE/FALSE positive. " - "Never run automatically; never folded into scan.", - input_schema={ - "type": "object", - "properties": { - "config": {"type": "string"}, - "model": {"type": "string"}, - "max_findings": {"type": "integer"}, - "write": {"type": "boolean", "description": "append above-floor FPs to judged.yaml"}, - "trust_judge_config": {"type": "boolean"}, - "trust_judge_policy": {"type": "boolean"}, - "trust_packs": {"type": "array", "items": {"type": "string"}}, - "trust_local_packs": {"type": "boolean"}, - "strict_defaults": {"type": "boolean"}, - "context_lines": {"type": "integer"}, - }, - }, + **_JUDGE_TOOL, handler=_judge, ) ) self.add_tool( Tool( - name="baseline", - capabilities=frozenset({ToolCapability.READ, ToolCapability.WRITE}), - description="Snapshot current defects as the baseline so only NEW findings surface. " - "Default overwrite=false refuses to clobber and returns already_exists=true. " - "Set overwrite=true to re-derive and overwrite the baseline. " - "Prefer FIXING a finding over baselining it. Optional reason.", - input_schema={ - "type": "object", - "properties": { - "reason": {"type": "string"}, - "overwrite": {"type": "boolean"}, - "config": {"type": "string"}, - "cache_dir": { - "type": "string", - "description": "subdir relative to project root for summary cache", - }, - "trust_packs": {"type": "array", "items": {"type": "string"}}, - "trust_local_packs": {"type": "boolean"}, - "strict_defaults": {"type": "boolean"}, - }, - }, + **_BASELINE_TOOL, handler=_baseline, ) ) self.add_tool( Tool( - name="waiver_add", - capabilities=frozenset({ToolCapability.READ, ToolCapability.WRITE}), - description="Waive ONE finding by fingerprint with a mandatory reason and expiry. " - "Prefer fixing; a waiver is an audited, time-boxed exception.", - input_schema={ - "type": "object", - "required": ["fingerprint", "reason", "expires"], - "properties": { - "fingerprint": {"type": "string"}, - "reason": {"type": "string"}, - "expires": {"type": "string", "description": "YYYY-MM-DD"}, - }, - }, + **_WAIVER_ADD_TOOL, handler=_waiver_add, ) ) self.add_tool( Tool( - name="fix", - capabilities=frozenset({ToolCapability.READ, ToolCapability.WRITE}), - description="Scan and apply mechanical autofixes to findings (currently only PY-WL-111 is supported).", - input_schema={ - "type": "object", - "properties": { - "path": {"type": "string", "description": "subdir relative to project root to scan and fix"}, - "config": {"type": "string"}, - "dry_run": {"type": "boolean", "description": "preview changes without modifying files"}, - "apply": {"type": "boolean", "description": "must be true to modify files"}, - }, - }, + **_FIX_TOOL, handler=_fix, ) ) self.add_tool( Tool( - name="doctor", - description="Health-check the wardline install and federation wiring " - "(same checks as CLI `doctor --fix`: instruction blocks, skills, MCP " - "registration, config parseability, sibling URLs, Filigree emit auth) " - "PLUS this server's self-identification: package version, pid, start " - "time, and a source-FRESHNESS verdict. If `server.fresh` is false this " - "long-lived server predates the on-disk wardline code — its results are " - "stale; restart the MCP server. Read-only by default; `repair: true` " - "(write-gated) repairs install artifacts and re-pins a rejected " - "federation token.", - input_schema={ - "type": "object", - "properties": { - "repair": { - "type": "boolean", - "description": "Default false (pure probe, writes nothing). true repairs " - "install artifacts (CLAUDE.md/AGENTS.md blocks, .claude/.agents skills, " - ".mcp.json + Codex registration, .weft state dir) and, when Filigree " - "rejected the emit token, re-pins the accepted local mint in .env.", - }, - "filigree_url": { - "type": "string", - "description": "Filigree URL to probe for emit auth (default: the server's " - "configured URL, then WARDLINE_FILIGREE_URL, then the .mcp.json arg). " - "Only loopback origins are ever probed with a token.", - }, - }, - }, + **_DOCTOR_TOOL, handler=lambda args, root: _doctor( args, root, started_at=self.started_at, filigree_url=self.filigree_url ), @@ -1427,51 +3948,7 @@ def _register_tools(self) -> None: ) self.add_tool( Tool( - name="rekey", - description="Re-key baseline/waiver/judged verdicts across a fingerprint-scheme " - "change (after the engine's fingerprint formula migrates, NOT after ordinary " - "refactors — fingerprints are line-insensitive). DEFAULT is a read-only PROBE: " - "reports how many stored verdicts will carry, which orphan and why, and any " - "collisions — writes nothing. Pass `apply: true` to migrate (snapshots first, " - "resumable journal), `resume: true` to finish an interrupted migration without " - "re-scanning, `rollback: true` to restore the pre-migration stores. The three " - "are mutually exclusive and write-gated.", - input_schema={ - "type": "object", - "properties": { - "path": {"type": "string", "description": "subdir relative to project root"}, - "config": {"type": "string"}, - "cache_dir": { - "type": "string", - "description": "subdir relative to project root for summary cache", - }, - "trust_packs": {"type": "array", "items": {"type": "string"}}, - "trust_local_packs": { - "type": "boolean", - "description": "Allow loading custom trust-grammar packs from the local project directory", - }, - "strict_defaults": { - "type": "boolean", - "description": "Ignore repository-supplied custom configuration overrides (weft.toml)", - }, - "apply": { - "type": "boolean", - "description": "Default false (read-only probe). true RUNS the migration: " - "snapshot stores, write the resumable journal, carry verdicts to the " - "new fingerprints, best-effort re-emit to a configured Filigree.", - }, - "resume": { - "type": "boolean", - "description": "Finish an interrupted migration from the journal WITHOUT " - "re-scanning (YAML legs re-carry from the snapshot).", - }, - "rollback": { - "type": "boolean", - "description": "Restore the pre-migration stores byte-identical from the " - "snapshot and remove the journal. Filigree associations are NOT reversed.", - }, - }, - }, + **_REKEY_TOOL, handler=lambda args, root: _rekey( args, root, @@ -1503,17 +3980,23 @@ def _wire(self) -> None: self.rpc.register("prompts/get", self._prompts_get) def _tools_list(self, params: dict[str, Any]) -> dict[str, Any]: - return { - "tools": [ - { - "name": t.name, - "description": t.description, - "inputSchema": t.input_schema, - "capabilities": [cap.value for cap in sorted(t.capabilities, key=lambda c: c.value)], - } - for t in self._tools.values() - ] - } + # B1/B2: standard MCP metadata (title / outputSchema / annotations, with + # annotations.title for 2025-03-26 clients) ALONGSIDE the homegrown + # "capabilities" key — mapped, not replaced, so existing consumers keep working. + tools: list[dict[str, Any]] = [] + for t in self._tools.values(): + entry: dict[str, Any] = {"name": t.name} + if t.title is not None: + entry["title"] = t.title + entry["description"] = t.description + entry["inputSchema"] = t.input_schema + if t.output_schema is not None: + entry["outputSchema"] = t.output_schema + if t.annotations is not None: + entry["annotations"] = t.annotations + entry["capabilities"] = [cap.value for cap in sorted(t.capabilities, key=lambda c: c.value)] + tools.append(entry) + return {"tools": tools} def _resolved_loomweave_url_for_policy(self, arguments: dict[str, Any]) -> str | None: return config_mod.resolve_loomweave_url( @@ -1631,7 +4114,13 @@ def _tools_call(self, params: dict[str, Any]) -> dict[str, Any]: traceback.print_exc(file=sys.stderr) return self._is_error(f"wardline internal error: {exc}") - return {"content": [{"type": "text", "text": json.dumps(payload, ensure_ascii=False)}]} + # Dual emission (B1): the text block stays byte-identical to the pre-B1 shape so + # existing clients parsing it keep working; structuredContent carries the same + # payload for 2025-06-18 clients. isError results never carry structuredContent. + return { + "content": [{"type": "text", "text": json.dumps(payload, ensure_ascii=False)}], + "structuredContent": payload, + } def _resources_list(self, params: dict[str, Any]) -> dict[str, Any]: return {"resources": list_resources()} diff --git a/src/wardline/mcp/tooling.py b/src/wardline/mcp/tooling.py index afd7ea96..cf87b57b 100644 --- a/src/wardline/mcp/tooling.py +++ b/src/wardline/mcp/tooling.py @@ -51,6 +51,19 @@ class Tool: handler: Callable[[dict[str, Any], Path], Any] network: bool = False capabilities: frozenset[ToolCapability] = frozenset({ToolCapability.READ}) + # B1/B2 (wardline-47ff226ebe / wardline-e63204176b): MCP rev 2025-06-18 structured + # output + 2025-03-26 display metadata. ``annotations`` is the standard MCP + # ToolAnnotations object (title, readOnlyHint, destructiveHint, idempotentHint, + # openWorldHint). CONVENTION: the hints describe the tool's integration-free + # baseline posture — readOnlyHint mirrors the DECLARED capability set, and + # openWorldHint mirrors the declared NETWORK capability. Opt-in federation reach + # (a configured Filigree/Loomweave URL widening scan/dossier/attest at runtime) + # is deliberately NOT reflected here: hints are static, untrusted UX advisories, + # and ToolPolicy + _effective_tool_capabilities remain the enforcement authority + # over what a call may actually do. + title: str | None = None + output_schema: dict[str, Any] | None = None + annotations: dict[str, Any] | None = None def __post_init__(self) -> None: capabilities = set(self.capabilities) or {ToolCapability.READ} diff --git a/tests/conformance/test_mcp_handshake.py b/tests/conformance/test_mcp_handshake.py index db4469aa..2ee5f2bc 100644 --- a/tests/conformance/test_mcp_handshake.py +++ b/tests/conformance/test_mcp_handshake.py @@ -53,7 +53,7 @@ def test_full_client_handshake_and_every_surface() -> None: assert init["protocolVersion"] == PROTOCOL_VERSION assert init["serverInfo"]["name"] == "wardline" assert {"tools", "resources", "prompts"} <= set(init["capabilities"]) - # tools/list: the eleven documented tools, no more no less + # tools/list: the fifteen documented tools, no more no less tool_names = {t["name"] for t in by_id[2]["result"]["tools"]} assert tool_names == { "scan", @@ -83,6 +83,8 @@ def test_full_client_handshake_and_every_surface() -> None: assert call["content"][0]["type"] == "text" payload = json.loads(call["content"][0]["text"]) assert {"agent_summary", "summary", "gate"} <= set(payload) + # B1 dual emission: structuredContent carries the SAME payload as the text block + assert call["structuredContent"] == payload # resources/read: the vocab resource round-trips non-empty text through the loop read = by_id[6]["result"] assert read["contents"][0]["uri"] == "wardline://vocab" @@ -137,6 +139,8 @@ def test_tool_execution_error_is_iserror_result_not_jsonrpc_error() -> None: assert "error" not in resp # NOT a JSON-RPC error assert resp["result"]["isError"] is True # IS an isError result assert "re-scan" in resp["result"]["content"][0]["text"].lower() + # B1: isError results never carry structuredContent + assert "structuredContent" not in resp["result"] def test_unknown_method_is_a_jsonrpc_error() -> None: diff --git a/tests/conformance/test_mcp_structured_output.py b/tests/conformance/test_mcp_structured_output.py new file mode 100644 index 00000000..2fbfd642 --- /dev/null +++ b/tests/conformance/test_mcp_structured_output.py @@ -0,0 +1,384 @@ +"""B1+B2 conformance: MCP structured output + display metadata across all 15 tools. + +Three layers, each pinned for EVERY registered tool: + +1. ADVERTISEMENT — every tools/list entry carries a non-empty ``title``, a complete + standard ``annotations`` object (2025-03-26), an object-typed ``outputSchema`` + (2025-06-18), AND the legacy homegrown ``capabilities`` key (mapped, not replaced). + The annotation hints must be consistent with the declared capability sets. +2. EXECUTION — a representative SUCCESSFUL tools/call per tool: the returned + ``structuredContent`` validates against that tool's own declared outputSchema and + equals ``json.loads(content[0].text)`` (dual emission, byte-compatible text block). + isError results carry NO structuredContent. +3. NEGOTIATION — initialize echoes any supported protocolVersion verbatim and answers + the latest revision for an unknown one. +""" + +from __future__ import annotations + +import json +from datetime import UTC, datetime +from pathlib import Path +from typing import Any + +import jsonschema +import pytest + +from wardline.core.judge import JudgeResponse, JudgeVerdict +from wardline.mcp.protocol import PROTOCOL_VERSION, SUPPORTED_PROTOCOL_VERSIONS +from wardline.mcp.server import WardlineMCPServer +from wardline.mcp.tooling import ToolCapability + +FIXTURE = Path("tests/fixtures/sample_project") + +# The published 15-tool surface, in advertisement order. +EXPECTED_TOOLS = ( + "scan", + "explain_taint", + "dossier", + "assure", + "decorator_coverage", + "attest", + "verify_attestation", + "file_finding", + "scan_file_findings", + "judge", + "baseline", + "waiver_add", + "fix", + "doctor", + "rekey", +) + +# B2 acceptance: the pure-read surface advertises readOnlyHint: true. +READ_ONLY_TOOLS = frozenset( + {"scan", "explain_taint", "dossier", "assure", "decorator_coverage", "attest", "verify_attestation"} +) + +_ANNOTATION_KEYS = {"title", "readOnlyHint", "destructiveHint", "idempotentHint", "openWorldHint"} +_HINT_KEYS = ("readOnlyHint", "destructiveHint", "idempotentHint", "openWorldHint") + +_LEAKY = ( + "from wardline.decorators import external_boundary, trusted\n" + "@external_boundary\ndef read_raw(p):\n return p\n" + "@trusted\ndef leaky(p):\n return read_raw(p)\n" +) + + +def _leaky_project(tmp_path: Path) -> Path: + proj = tmp_path / "proj" + proj.mkdir() + (proj / "svc.py").write_text(_LEAKY, encoding="utf-8") + return proj + + +def _entries(server: WardlineMCPServer) -> dict[str, dict[str, Any]]: + resp = server.rpc.dispatch({"jsonrpc": "2.0", "id": 1, "method": "tools/list", "params": {}}) + assert "error" not in resp, resp + return {t["name"]: t for t in resp["result"]["tools"]} + + +def _call(server: WardlineMCPServer, name: str, arguments: dict[str, Any]) -> dict[str, Any]: + resp = server.rpc.dispatch( + {"jsonrpc": "2.0", "id": 1, "method": "tools/call", "params": {"name": name, "arguments": arguments}} + ) + assert "error" not in resp, resp + result: dict[str, Any] = resp["result"] + return result + + +def _validated(server: WardlineMCPServer, name: str, arguments: dict[str, Any]) -> dict[str, Any]: + """tools/call success → structuredContent validates against the tool's OWN declared + outputSchema AND equals the parsed text block (dual emission).""" + result = _call(server, name, arguments) + assert result.get("isError") is not True, result + schema = _entries(server)[name]["outputSchema"] + assert "structuredContent" in result, f"{name}: success result missing structuredContent" + structured: dict[str, Any] = result["structuredContent"] + jsonschema.validate(structured, schema) + assert json.loads(result["content"][0]["text"]) == structured, f"{name}: dual emission diverged" + return structured + + +@pytest.fixture(scope="module") +def fixture_server() -> WardlineMCPServer: + return WardlineMCPServer(root=FIXTURE) + + +# --------------------------------------------------------------------------- +# 1. Advertisement conformance +# --------------------------------------------------------------------------- + + +def test_advertises_exactly_the_published_surface(fixture_server: WardlineMCPServer) -> None: + assert tuple(_entries(fixture_server)) == EXPECTED_TOOLS + + +@pytest.mark.parametrize("name", EXPECTED_TOOLS) +def test_tools_list_entry_carries_b1_b2_metadata(fixture_server: WardlineMCPServer, name: str) -> None: + entry = _entries(fixture_server)[name] + # title: present and non-empty + assert isinstance(entry["title"], str) and entry["title"] + # annotations: the COMPLETE standard ToolAnnotations object, nothing else + ann = entry["annotations"] + assert set(ann) == _ANNOTATION_KEYS + for hint in _HINT_KEYS: + assert isinstance(ann[hint], bool), f"{name}.{hint} must be a bool" + assert ann["title"] == entry["title"] + # outputSchema: an object schema (2025-06-18 structured output contract) + out = entry["outputSchema"] + assert isinstance(out, dict) + assert out["type"] == "object" + # the legacy homegrown capabilities key is mapped, NOT replaced + assert isinstance(entry["capabilities"], list) and entry["capabilities"] + + +@pytest.mark.parametrize("name", EXPECTED_TOOLS) +def test_annotations_consistent_with_declared_capabilities(fixture_server: WardlineMCPServer, name: str) -> None: + """The standard hints must never CONTRADICT the homegrown capability declaration.""" + tool = fixture_server._tools[name] + ann = _entries(fixture_server)[name]["annotations"] + if ToolCapability.WRITE in tool.capabilities: + assert ann["readOnlyHint"] is False, f"{name}: WRITE capability with readOnlyHint true" + if ToolCapability.NETWORK in tool.capabilities: + assert ann["openWorldHint"] is True, f"{name}: NETWORK capability with openWorldHint false" + if ann["readOnlyHint"]: + assert ann["destructiveHint"] is False, f"{name}: read-only tools cannot be destructive" + + +def test_pure_read_surface_advertises_read_only(fixture_server: WardlineMCPServer) -> None: + entries = _entries(fixture_server) + for name in EXPECTED_TOOLS: + expected = name in READ_ONLY_TOOLS + assert entries[name]["annotations"]["readOnlyHint"] is expected, name + + +# --------------------------------------------------------------------------- +# 2. Execution conformance — one representative SUCCESS per tool +# --------------------------------------------------------------------------- + + +def test_scan_structured_output(tmp_path: Path) -> None: + server = WardlineMCPServer(root=_leaky_project(tmp_path)) + out = _validated(server, "scan", {"fail_on": "ERROR"}) + assert out["gate"]["tripped"] is True + + +def test_explain_taint_structured_output(tmp_path: Path) -> None: + server = WardlineMCPServer(root=_leaky_project(tmp_path)) + scan_out = _validated(server, "scan", {}) + fp = next(e["fingerprint"] for e in scan_out["agent_summary"]["active_defects"] if e["rule_id"] == "PY-WL-101") + out = _validated(server, "explain_taint", {"fingerprint": fp}) + assert out["fingerprint"] == fp + + +def test_dossier_structured_output(tmp_path: Path) -> None: + server = WardlineMCPServer(root=_leaky_project(tmp_path)) + out = _validated(server, "dossier", {"entity": "svc.leaky"}) + assert out["identity"]["qualname"] == "svc.leaky" + + +def test_assure_structured_output(tmp_path: Path) -> None: + server = WardlineMCPServer(root=_leaky_project(tmp_path)) + out = _validated(server, "assure", {}) + assert out["boundaries_total"] >= 1 + + +def test_decorator_coverage_structured_output(tmp_path: Path) -> None: + server = WardlineMCPServer(root=_leaky_project(tmp_path)) + out = _validated(server, "decorator_coverage", {}) + assert out["summary"]["total"] >= 1 + + +def test_attest_and_verify_attestation_structured_output(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None: + from wardline.core.attest_key import WARDLINE_ATTEST_KEY_ENV, mint_attest_key + + monkeypatch.delenv(WARDLINE_ATTEST_KEY_ENV, raising=False) + proj = _leaky_project(tmp_path) + mint_attest_key(proj) # minted into proj/.env (non-git tree → dirty=False, strict default still builds) + server = WardlineMCPServer(root=proj) + + bundle = _validated(server, "attest", {}) + assert bundle["signature"]["key_id"] + + verified = _validated(server, "verify_attestation", {"bundle": bundle}) + assert verified["signature_valid"] is True + + +def test_file_finding_structured_output(tmp_path: Path) -> None: + from wardline.core.filigree_issue import FileResult + + class FakeFiler: + def file(self, fingerprint: str, *, scan_source: str = "wardline", priority=None, labels=None) -> FileResult: + return FileResult(reachable=True, issue_id="wardline-abc", created=True) + + server = WardlineMCPServer(root=_leaky_project(tmp_path)) + # Inject the fake filer through the same seam the registered lambda resolves it. + server._filigree_filer = lambda *a, **k: FakeFiler() # type: ignore[method-assign] + out = _validated(server, "file_finding", {"fingerprint": "f" * 64}) + assert out["issue_id"] == "wardline-abc" + + +def test_scan_file_findings_structured_output(tmp_path: Path) -> None: + # Default invocation is a REAL dry-run scan (no Filigree configured): the + # fail-soft emit/file blocks report 'not configured'. + server = WardlineMCPServer(root=_leaky_project(tmp_path)) + out = _validated(server, "scan_file_findings", {}) + assert out["mode"] == "dry_run" + + +def test_judge_structured_output(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None: + # Network-fenced: patch the caller the default run_judge path imports. + fake = JudgeResponse( + verdict=JudgeVerdict.TRUE_POSITIVE, + rationale="genuinely reaches a trusted sink", + confidence=0.91, + model_id="fake/model", + recorded_at=datetime.now(UTC), + prompt_tokens_total=128, + prompt_tokens_cached=None, + policy_hash="deadbeef", + ) + monkeypatch.setattr("wardline.core.judge_run.call_judge", lambda *a, **k: fake) + monkeypatch.setenv("WARDLINE_OPENROUTER_API_KEY", "sk-or-test") + server = WardlineMCPServer(root=_leaky_project(tmp_path)) + out = _validated(server, "judge", {}) + assert out["verdicts"], out + + +def test_baseline_structured_output(tmp_path: Path) -> None: + server = WardlineMCPServer(root=_leaky_project(tmp_path)) + out = _validated(server, "baseline", {"reason": "accept current debt"}) + assert out["baselined_count"] >= 1 + + +def test_waiver_add_structured_output(tmp_path: Path) -> None: + server = WardlineMCPServer(root=_leaky_project(tmp_path)) + out = _validated( + server, "waiver_add", {"fingerprint": "a" * 64, "reason": "validated upstream", "expires": "2026-12-31"} + ) + assert out["fingerprint"] == "a" * 64 + + +def test_fix_structured_output(tmp_path: Path) -> None: + proj = tmp_path / "proj" + proj.mkdir() + (proj / "weft.toml").write_text('[wardline.autofix]\nboundary_exception = "ValueError"\n', encoding="utf-8") + (proj / "svc.py").write_text( + "from wardline.decorators import trust_boundary\n" + "@trust_boundary(to_level='ASSURED')\n" + "def check(val):\n" + " assert val is not None\n" + " return val\n", + encoding="utf-8", + ) + server = WardlineMCPServer(root=proj) + out = _validated(server, "fix", {"dry_run": True}) + assert "svc.py" in out["fixed"] + assert out["applied"] is False + + +def test_doctor_structured_output(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None: + # Hermetic: never read the real HOME or probe a real federation endpoint. + monkeypatch.delenv("WARDLINE_LOOMWEAVE_URL", raising=False) + monkeypatch.delenv("WARDLINE_FILIGREE_URL", raising=False) + monkeypatch.setattr("wardline.install.mcp_json.Path.home", lambda: tmp_path / "home") + monkeypatch.setattr("wardline.install.mcp_json._find_wardline_command", lambda: "/bin/wardline") + monkeypatch.setattr("wardline.install.detect.shutil.which", lambda _: None) + proj = tmp_path / "proj" + proj.mkdir() + server = WardlineMCPServer(root=proj) + out = _validated(server, "doctor", {}) + assert out["checks"][-1]["id"] == "server.freshness" + assert out["server"]["fresh"] is True + + +def test_rekey_structured_output(tmp_path: Path) -> None: + # Probe-by-default: read-only report on a store-less project — writes nothing. + server = WardlineMCPServer(root=_leaky_project(tmp_path)) + out = _validated(server, "rekey", {}) + assert out["mode"] == "probe" + assert out["clean"] is True + + +def test_execution_conformance_covers_every_advertised_tool(fixture_server: WardlineMCPServer) -> None: + """Tripwire: a 16th tool must add an execution-conformance case above.""" + assert set(_entries(fixture_server)) == set(EXPECTED_TOOLS) + + +def test_doctor_freshness_check_appears_in_structured_content(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None: + # A STALE server (started before the on-disk source changed) must still produce a + # schema-valid payload: the error branch of the freshness check is schema-pinned too. + monkeypatch.delenv("WARDLINE_LOOMWEAVE_URL", raising=False) + monkeypatch.delenv("WARDLINE_FILIGREE_URL", raising=False) + monkeypatch.setattr("wardline.install.mcp_json.Path.home", lambda: tmp_path / "home") + monkeypatch.setattr("wardline.install.mcp_json._find_wardline_command", lambda: "/bin/wardline") + monkeypatch.setattr("wardline.install.detect.shutil.which", lambda _: None) + proj = tmp_path / "proj" + proj.mkdir() + server = WardlineMCPServer(root=proj) + server.started_at = 1.0 # 1970 — everything on disk is newer + out = _validated(server, "doctor", {}) + assert out["server"]["fresh"] is False + assert out["ok"] is False + + +# --------------------------------------------------------------------------- +# isError results never carry structuredContent +# --------------------------------------------------------------------------- + + +def test_tool_execution_error_has_no_structured_content() -> None: + # Stale/unknown fingerprint → isError result, text-only. + server = WardlineMCPServer(root=FIXTURE) + result = _call(server, "explain_taint", {"fingerprint": "0" * 64}) + assert result["isError"] is True + assert "structuredContent" not in result + + +def test_invalid_arguments_error_has_no_structured_content(tmp_path: Path) -> None: + # jsonschema argument rejection → isError result, text-only. + server = WardlineMCPServer(root=tmp_path) + result = _call( + server, + "waiver_add", + {"fingerprint": "a" * 64, "reason": "ok", "expires": "2026-12-31", "apply": True}, + ) + assert result["isError"] is True + assert "additional properties" in result["content"][0]["text"].lower() + assert "structuredContent" not in result + + +# --------------------------------------------------------------------------- +# 3. Protocol version negotiation +# --------------------------------------------------------------------------- + + +def test_supported_protocol_versions_are_pinned() -> None: + assert SUPPORTED_PROTOCOL_VERSIONS == ("2025-06-18", "2025-03-26", "2024-11-05") + assert PROTOCOL_VERSION == "2025-06-18" + + +def _initialize(server: WardlineMCPServer, requested: str) -> str: + resp = server.rpc.dispatch( + { + "jsonrpc": "2.0", + "id": 1, + "method": "initialize", + "params": {"protocolVersion": requested, "capabilities": {}}, + } + ) + assert "error" not in resp, resp + version: str = resp["result"]["protocolVersion"] + return version + + +@pytest.mark.parametrize("requested", SUPPORTED_PROTOCOL_VERSIONS) +def test_initialize_echoes_each_supported_protocol_version(requested: str) -> None: + server = WardlineMCPServer(root=FIXTURE) + assert _initialize(server, requested) == requested + + +def test_initialize_answers_latest_for_unknown_protocol_version() -> None: + server = WardlineMCPServer(root=FIXTURE) + assert _initialize(server, "1999-01-01") == "2025-06-18" diff --git a/tests/docs/test_glossary_vocabulary.py b/tests/docs/test_glossary_vocabulary.py index 71046f01..6bdc8de8 100644 --- a/tests/docs/test_glossary_vocabulary.py +++ b/tests/docs/test_glossary_vocabulary.py @@ -39,22 +39,22 @@ ("src/wardline/core/run.py", 326, "Baseline(frozenset())"), ("src/wardline/core/run.py", 354, "def apply_delta_scope"), ("src/wardline/core/run.py", 378, "active=sum"), - ("src/wardline/core/run.py", 439, "honors_suppressions"), + ("src/wardline/core/run.py", 437, "honors_suppressions"), # src/wardline/cli/scan.py — CLI summary line + gate stderr ("src/wardline/cli/scan.py", 415, "suppressed"), ("src/wardline/cli/scan.py", 416, "{s.active} active"), ("src/wardline/cli/scan.py", 468, "gate: FAILED"), # src/wardline/mcp/server.py — MCP scan summary + gate block - ("src/wardline/mcp/server.py", 336, '"total": result.summary.total'), - ("src/wardline/mcp/server.py", 337, '"active": result.summary.active'), - ("src/wardline/mcp/server.py", 338, '"baselined": result.summary.baselined'), - ("src/wardline/mcp/server.py", 339, '"waived": result.summary.waived'), - ("src/wardline/mcp/server.py", 340, '"judged": result.summary.judged'), - ("src/wardline/mcp/server.py", 345, '"informational": result.summary.informational'), - ("src/wardline/mcp/server.py", 349, '"unanalyzed": result.summary.unanalyzed'), - ("src/wardline/mcp/server.py", 351, '"gate": {'), - ("src/wardline/mcp/server.py", 352, '"tripped": decision.tripped'), - ("src/wardline/mcp/server.py", 356, '"verdict": decision.verdict'), + ("src/wardline/mcp/server.py", 774, '"total": result.summary.total'), + ("src/wardline/mcp/server.py", 775, '"active": result.summary.active'), + ("src/wardline/mcp/server.py", 776, '"baselined": result.summary.baselined'), + ("src/wardline/mcp/server.py", 777, '"waived": result.summary.waived'), + ("src/wardline/mcp/server.py", 778, '"judged": result.summary.judged'), + ("src/wardline/mcp/server.py", 783, '"informational": result.summary.informational'), + ("src/wardline/mcp/server.py", 787, '"unanalyzed": result.summary.unanalyzed'), + ("src/wardline/mcp/server.py", 789, '"gate": {'), + ("src/wardline/mcp/server.py", 790, '"tripped": decision.tripped'), + ("src/wardline/mcp/server.py", 794, '"verdict": decision.verdict'), # src/wardline/core/agent_summary.py — agent-summary JSON keys ("src/wardline/core/agent_summary.py", 134, '"total_findings"'), ("src/wardline/core/agent_summary.py", 135, '"active_defects"'), diff --git a/tests/golden/identity/rust/_capture.py b/tests/golden/identity/rust/_capture.py index 8ecffabb..72172a61 100644 --- a/tests/golden/identity/rust/_capture.py +++ b/tests/golden/identity/rust/_capture.py @@ -32,7 +32,7 @@ import json from typing import TYPE_CHECKING, Any -from golden.identity._capture import _finding_sort_key, to_json +from golden.identity._capture import _finding_sort_key, to_json # type: ignore[import-not-found] from wardline.core import config as config_mod from wardline.core.discovery import discover from wardline.core.finding import Finding, Kind diff --git a/tests/unit/core/test_filigree_issue.py b/tests/unit/core/test_filigree_issue.py index 848b4811..055702ff 100644 --- a/tests/unit/core/test_filigree_issue.py +++ b/tests/unit/core/test_filigree_issue.py @@ -395,3 +395,12 @@ def get_taint_fact(self, qualname): assert res.attached is True assert transport.calls[0]["body"]["entity_id"] == "rust:function:demo.m.leaky" assert transport.calls[0]["body"]["entity_kind"] == "rust:function" + + +def test_file_2xx_non_string_issue_id_is_normalized_to_none(): + # Filigree's promote response is external input: a non-string issue_id (e.g. an + # integer) must not flow verbatim into tool payloads that publish issue_id as + # string|null in their MCP outputSchema — type-narrow at the wire boundary. + t = FakeTransport(200, '{"issue_id": 123, "created": true}') + res = FiligreeIssueFiler("http://h/api/weft/scan-results", transport=t).file("fp") + assert res.reachable is True and res.issue_id is None and res.created is True diff --git a/tests/unit/core/test_scan_file_workflow.py b/tests/unit/core/test_scan_file_workflow.py index cd841189..210134e8 100644 --- a/tests/unit/core/test_scan_file_workflow.py +++ b/tests/unit/core/test_scan_file_workflow.py @@ -116,3 +116,35 @@ def test_scan_file_findings_surfaces_partial_failures(tmp_path): assert finding["promotion"]["disabled_reason"] == "filigree unreachable" assert finding["identity_attach"]["attempted"] is False assert "no issue_id" in finding["identity_attach"]["reason"] + + +def test_scan_file_findings_no_filer_reason_names_missing_url(tmp_path): + # When no Filigree filer is configured, identity_attach must name the actual + # cause (no URL), matching promotion.disabled_reason — not claim a promote + # happened and returned no issue_id. + root = _project(tmp_path) + dry = scan_file_findings(root) + fp = dry["active_defects"][0]["fingerprint"] + + out = scan_file_findings(root, fingerprints=(fp,), dry_run=False) + + finding = out["active_defects"][0] + assert finding["promotion"]["disabled_reason"] == "no Filigree URL configured" + assert finding["identity_attach"]["attempted"] is False + assert finding["identity_attach"]["reason"] == "no Filigree URL configured" + + +def test_scan_file_findings_emit_disabled_reason_uses_discriminated_ladder(tmp_path): + # A 401-with-token soft emit failure must surface the discriminated ladder + # (dogfood #5), not the flat "filigree unreachable". + root = _project(tmp_path) + dry = scan_file_findings(root) + fp = dry["active_defects"][0]["fingerprint"] + emitter = FakeEmitter(EmitResult(reachable=False, status=401, token_sent=True, url="http://filigree.local")) + + out = scan_file_findings(root, fingerprints=(fp,), filigree_emitter=emitter, dry_run=False) + + reason = out["filigree_emit"]["disabled_reason"] + assert "401" in reason + assert "token" in reason + assert "http://filigree.local" in reason diff --git a/tests/unit/mcp/test_protocol.py b/tests/unit/mcp/test_protocol.py index 3955d8c1..1878a6cb 100644 --- a/tests/unit/mcp/test_protocol.py +++ b/tests/unit/mcp/test_protocol.py @@ -1,7 +1,7 @@ import io import json -from wardline.mcp.protocol import PROTOCOL_VERSION, JsonRpcServer, McpError +from wardline.mcp.protocol import PROTOCOL_VERSION, SUPPORTED_PROTOCOL_VERSIONS, JsonRpcServer, McpError def _server() -> JsonRpcServer: @@ -27,6 +27,37 @@ def test_initialize_returns_capabilities_and_protocol_version() -> None: assert resp["result"]["serverInfo"]["name"] == "wardline" +def test_initialize_negotiates_each_supported_protocol_version() -> None: + # Spec negotiation: a supported requested revision is echoed VERBATIM, so an + # older client (e.g. pinned to 2024-11-05) keeps its own revision. + srv = _server() + for requested in SUPPORTED_PROTOCOL_VERSIONS: + resp = srv.dispatch( + { + "jsonrpc": "2.0", + "id": 1, + "method": "initialize", + "params": {"protocolVersion": requested, "capabilities": {}}, + } + ) + assert resp["result"]["protocolVersion"] == requested + + +def test_initialize_unknown_protocol_version_answers_latest() -> None: + # An unsupported revision gets the newest revision this server speaks. + srv = _server() + resp = srv.dispatch( + { + "jsonrpc": "2.0", + "id": 1, + "method": "initialize", + "params": {"protocolVersion": "1999-01-01", "capabilities": {}}, + } + ) + assert resp["result"]["protocolVersion"] == PROTOCOL_VERSION + assert SUPPORTED_PROTOCOL_VERSIONS[0] == PROTOCOL_VERSION # newest first + + def test_notification_initialized_returns_none() -> None: srv = _server() # notifications (no id) must not produce a response diff --git a/tests/unit/mcp/test_server_loomweave_write.py b/tests/unit/mcp/test_server_loomweave_write.py index e1630595..b6ce3551 100644 --- a/tests/unit/mcp/test_server_loomweave_write.py +++ b/tests/unit/mcp/test_server_loomweave_write.py @@ -103,3 +103,43 @@ def test_explain_taint_chain_block_with_store(tmp_path): assert "chain" in out assert [h["qualname"] for h in out["chain"]["hops"]] == ["svc.leaky", "svc.read_raw"] assert out["chain"]["truncated_at"] is None + + +def _type_skewed_view(proj, qualname): + # A hand-edited / version-skewed blob: tiers and callee carry non-string types. + h = blake3.blake3((proj / "svc.py").read_bytes()).hexdigest() + blob = { + "schema_version": "wardline-taint-1", + "qualname": qualname, + "content_hash_at_compute": h, + "taint": { + "declared_return": 7, + "actual_return": ["EXTERNAL_RAW"], + "source": "anchored", + "contributing_callee_qualname": 99, + "resolved_call_count": 1, + "unresolved_call_count": 0, + }, + "findings": [], + } + return TaintFactView(qualname=qualname, exists=True, wardline_json=blob, current_content_hash=h) + + +def test_explain_taint_type_skewed_blob_fields_coerce_to_none(tmp_path): + # The store blob is external input: non-string tiers/callee must coerce to None + # (the fields are string|null in the published outputSchema), matching the + # adjacent fingerprint/rule_id/path guards — never flow through verbatim. + (tmp_path / "svc.py").write_text(_LEAKY, encoding="utf-8") + client = MapClient({"svc.leaky": _type_skewed_view(tmp_path, "svc.leaky")}) + + out = _explain_taint({"sink_qualname": "svc.leaky", "chain": True}, tmp_path, client) + + assert out["tier_in"] is None + assert out["tier_out"] is None + assert out["immediate_tainted_callee"] is None + assert out["source_boundary_qualname"] is None + hop = out["chain"]["hops"][0] + assert hop["tier_in"] is None + assert hop["tier_out"] is None + assert hop["contributing_callee_qualname"] is None + assert out["chain"]["truncated_at"] is None diff --git a/tests/unit/mcp/test_server_structure.py b/tests/unit/mcp/test_server_structure.py index af6c29b7..0e4bb4f3 100644 --- a/tests/unit/mcp/test_server_structure.py +++ b/tests/unit/mcp/test_server_structure.py @@ -47,3 +47,20 @@ def test_mcp_advertisement_snapshot() -> None: "wardline://config-schema", ] assert [prompt["name"] for prompt in prompts["result"]["prompts"]] == ["wardline:loop"] + + # B1/B2: every advertised tool carries the standard MCP metadata surface — + # title (2025-03-26), a complete annotations object whose title mirrors the + # tool title, an object-typed outputSchema (2025-06-18) — alongside the + # homegrown capabilities key (mapped, not replaced). + for tool in tools["result"]["tools"]: + assert isinstance(tool["title"], str) and tool["title"], tool["name"] + assert set(tool["annotations"]) == { + "title", + "readOnlyHint", + "destructiveHint", + "idempotentHint", + "openWorldHint", + }, tool["name"] + assert tool["annotations"]["title"] == tool["title"], tool["name"] + assert tool["outputSchema"]["type"] == "object", tool["name"] + assert isinstance(tool["capabilities"], list) and tool["capabilities"], tool["name"] diff --git a/tests/unit/rust/test_index.py b/tests/unit/rust/test_index.py index cc465ae3..3eed6cb8 100644 --- a/tests/unit/rust/test_index.py +++ b/tests/unit/rust/test_index.py @@ -269,7 +269,7 @@ def test_emission_is_deterministic() -> None: "#[cfg(unix)]\nfn f() {}\n#[cfg(windows)]\nfn f() {}\n" ) - def run() -> list[tuple[str, str, str | None, int, int]]: + def run() -> list[tuple[str, str, str | None, int | None, int | None]]: return [ (e.qualname, e.kind, e.parent, e.location.line_start, e.location.line_end) for e in discover_rust_entities(src, module="demo.m", path="m.rs") diff --git a/tests/unit/scanner/taint/test_attribute_write_flow.py b/tests/unit/scanner/taint/test_attribute_write_flow.py index ac34609f..bb70a390 100644 --- a/tests/unit/scanner/taint/test_attribute_write_flow.py +++ b/tests/unit/scanner/taint/test_attribute_write_flow.py @@ -26,6 +26,7 @@ import ast import textwrap +from collections.abc import Mapping from pathlib import Path from wardline.core.config import WardlineConfig @@ -64,7 +65,7 @@ def _defect_ids(tmp_path: Path, body: str) -> set[str]: return {f.rule_id for f in findings if f.kind is Kind.DEFECT} -def _attr_taints(analyzer: WardlineAnalyzer) -> dict[str, dict[str, TaintState]]: +def _attr_taints(analyzer: WardlineAnalyzer) -> Mapping[str, Mapping[str, TaintState]]: ctx = analyzer.last_context assert ctx is not None return ctx.class_attr_taints From de50b7af90cd082b13d5de7bec31f03328f44b12 Mon Sep 17 00:00:00 2001 From: John Morrissey Date: Fri, 12 Jun 2026 05:45:47 +1000 Subject: [PATCH 103/186] fix(scanner): guard recursive lambda taint resolution --- src/wardline/scanner/taint/variable_level.py | 31 +++++++++++++++++++ .../taint/test_review_fixups_engine.py | 25 +++++++++++++++ 2 files changed, 56 insertions(+) diff --git a/src/wardline/scanner/taint/variable_level.py b/src/wardline/scanner/taint/variable_level.py index 0b3e88c7..b4e05d8f 100644 --- a/src/wardline/scanner/taint/variable_level.py +++ b/src/wardline/scanner/taint/variable_level.py @@ -208,6 +208,10 @@ "_CURRENT_LAMBDA_BINDINGS", default=None ) +_CURRENT_ACTIVE_LAMBDAS: contextvars.ContextVar[set[int] | None] = contextvars.ContextVar( + "_CURRENT_ACTIVE_LAMBDAS", default=None +) + _CURRENT_MODULE_PREFIX: contextvars.ContextVar[str | None] = contextvars.ContextVar( "_CURRENT_MODULE_PREFIX", default=None ) @@ -458,6 +462,33 @@ def _resolve_lambda_body_at_call( Recording the body here prevents a later clean assignment from laundering a raw value that was captured when the direct call actually ran. """ + active = _CURRENT_ACTIVE_LAMBDAS.get() + token_active = None + if active is None: + active = set() + token_active = _CURRENT_ACTIVE_LAMBDAS.set(active) + lam_id = id(lam) + if lam_id in active: + if token_active is not None: + _CURRENT_ACTIVE_LAMBDAS.reset(token_active) + return + active.add(lam_id) + try: + _resolve_lambda_body_at_call_inner(lam, function_taint, taint_map, var_taints, pos_taints, kw_taints) + finally: + active.discard(lam_id) + if token_active is not None: + _CURRENT_ACTIVE_LAMBDAS.reset(token_active) + + +def _resolve_lambda_body_at_call_inner( + lam: ast.Lambda, + function_taint: TaintState, + taint_map: dict[str, TaintState], + var_taints: dict[str, TaintState], + pos_taints: list[TaintState], + kw_taints: dict[str | None, TaintState], +) -> None: scope = dict(var_taints) args = lam.args # A ``**mapping`` spread arrives as a keyword with ``arg is None``. Its keys diff --git a/tests/unit/scanner/taint/test_review_fixups_engine.py b/tests/unit/scanner/taint/test_review_fixups_engine.py index 692fe92e..b5590eb8 100644 --- a/tests/unit/scanner/taint/test_review_fixups_engine.py +++ b/tests/unit/scanner/taint/test_review_fixups_engine.py @@ -25,6 +25,10 @@ 4. **Nested local helper calls** — the bare-name worst-arg conservatism (wardline-93d608c997) hit nested defs the engine had already analyzed (``m.f..helper``), marking validated values raw (PY-WL-101 FP). + +5. **Recursive lambda bindings** — call-time lambda body resolution must not + re-enter the same bound body indefinitely and skip the whole function before + later trust-boundary rules can run. """ from __future__ import annotations @@ -369,3 +373,24 @@ def launder_check(transform, p): """, ) assert len(_rule_hits(findings, "PY-WL-108")) == 1 + + +# ── 5. Recursive lambda call-time resolution must not skip the function ────── + + +def test_recursive_lambda_binding_does_not_skip_later_trust_boundary_check(tmp_path: Path) -> None: + findings = _scan( + tmp_path, + """ + @trusted(level='ASSURED', to_level='ASSURED') + def f(p): + cb = lambda x: cb(x) + if False: + cb(read_raw(p)) + return read_raw(p) + """, + ) + assert _rule_hits(findings, "WLN-ENGINE-FUNCTION-SKIPPED") == [] + hits = _rule_hits(findings, "PY-WL-101") + assert len(hits) == 1 + assert hits[0].qualname == "m.f" From 508e48d349ab461b776929a5727ac00c1c6a9b0e Mon Sep 17 00:00:00 2001 From: John Morrissey Date: Fri, 12 Jun 2026 05:48:47 +1000 Subject: [PATCH 104/186] fix(scanner): resolve returns at their statement snapshots --- src/wardline/scanner/taint/variable_level.py | 46 ++++++++++++++----- .../rules/test_untrusted_reaches_trusted.py | 22 +++++++++ 2 files changed, 57 insertions(+), 11 deletions(-) diff --git a/src/wardline/scanner/taint/variable_level.py b/src/wardline/scanner/taint/variable_level.py index b4e05d8f..056c083c 100644 --- a/src/wardline/scanner/taint/variable_level.py +++ b/src/wardline/scanner/taint/variable_level.py @@ -351,7 +351,13 @@ def analyze_function_variables( param_meets=context.param_meets, provenance_clash=context.provenance_clash, ) - return_taint = compute_return_taint(func_node, function_taint, dict(taint_map), variable_taints) + return_taint = compute_return_taint( + func_node, + function_taint, + dict(taint_map), + variable_taints, + call_site_taints, + ) # compute_return_callee is PROVENANCE-ONLY: _assignment_callee re-resolves # every direct-call assignment RHS against the FINAL var_taints, and letting # that re-resolution record would COMBINE post-call taint into the at-call @@ -361,7 +367,13 @@ def analyze_function_variables( # the only recording they get. token_no_record = _CURRENT_CALL_SITE_ARG_TAINTS.set(None) try: - return_callee = compute_return_callee(func_node, function_taint, dict(taint_map), dict(variable_taints)) + return_callee = compute_return_callee( + func_node, + function_taint, + dict(taint_map), + dict(variable_taints), + call_site_taints, + ) finally: _CURRENT_CALL_SITE_ARG_TAINTS.reset(token_no_record) return VariableTaintResult( @@ -2218,21 +2230,23 @@ def compute_return_taint( function_taint: TaintState, taint_map: dict[str, TaintState], var_taints: dict[str, TaintState], + return_snapshots: dict[int, dict[str, TaintState]] | None = None, ) -> TaintState | None: """Compute the *actual* taint a function returns (least-trusted of all paths). Resolves every value-bearing ``return`` statement in *func_node*'s own scope - (nested functions/lambdas excluded) against the already-computed ``var_taints`` - and the call-resolution ``taint_map``, and joins them with :func:`least_trusted` - — the worst (least-trusted) value any path can return. Returns ``None`` when the - function has no value-bearing ``return`` (implicit ``None`` / bare ``return`` / - pure side-effect): there is no returned data to police. + (nested functions/lambdas excluded) against the statement snapshot for that + return when available, falling back to the already-computed final + ``var_taints``. It then joins paths with :func:`least_trusted` — the worst + (least-trusted) value any path can return. Returns ``None`` when the function + has no value-bearing ``return`` (implicit ``None`` / bare ``return`` / pure + side-effect): there is no returned data to police. This is the precise input PY-WL-101 needs — distinct from ``project_taints`` (the function's anchored *body* taint, pinned to its declaration). """ returns: list[tuple[TaintState, str | None, ast.expr]] = [] - _collect_return_paths(list(func_node.body), function_taint, taint_map, var_taints, returns) + _collect_return_paths(list(func_node.body), function_taint, taint_map, var_taints, returns, return_snapshots) if not returns: return None result = returns[0][0] @@ -2246,6 +2260,7 @@ def compute_return_callee( function_taint: TaintState, taint_map: dict[str, TaintState], var_taints: dict[str, TaintState], + return_snapshots: dict[int, dict[str, TaintState]] | None = None, ) -> str | None: """Identify the callee that contributes a function's *actual* (least-trusted) return taint — the property ``explain_finding`` reports for a PY-WL-101 sink. @@ -2271,7 +2286,7 @@ def compute_return_callee( beyond one hop stay ``None`` (the N-hop walk lives in the Loomweave stored-fact path). """ returns: list[tuple[TaintState, str | None, ast.expr]] = [] - _collect_return_paths(list(func_node.body), function_taint, taint_map, var_taints, returns) + _collect_return_paths(list(func_node.body), function_taint, taint_map, var_taints, returns, return_snapshots) if not returns: return None worst = returns[0][0] @@ -2349,6 +2364,7 @@ def _collect_return_paths( taint_map: dict[str, TaintState], var_taints: dict[str, TaintState], out: list[tuple[TaintState, str | None, ast.expr]], + return_snapshots: dict[int, dict[str, TaintState]] | None = None, ) -> None: """Recurse the AST collecting ``(taint, callee_or_None, value_node)`` for each value-bearing return, descending into ALL children EXCEPT nested ``FunctionDef``/ @@ -2370,6 +2386,14 @@ def _collect_return_paths( if isinstance(node, (ast.FunctionDef, ast.AsyncFunctionDef, ast.ClassDef, ast.Lambda)): continue if isinstance(node, (ast.Return, ast.Yield, ast.YieldFrom)) and node.value is not None: - taint = _resolve_expr(node.value, function_taint, taint_map, var_taints) + snapshot = return_snapshots.get(id(node)) if return_snapshots is not None else None + taint = _resolve_expr(node.value, function_taint, taint_map, dict(snapshot or var_taints)) out.append((taint, _return_callee(node.value), node.value)) - _collect_return_paths(list(ast.iter_child_nodes(node)), function_taint, taint_map, var_taints, out) + _collect_return_paths( + list(ast.iter_child_nodes(node)), + function_taint, + taint_map, + var_taints, + out, + return_snapshots, + ) diff --git a/tests/unit/scanner/rules/test_untrusted_reaches_trusted.py b/tests/unit/scanner/rules/test_untrusted_reaches_trusted.py index ea546f1b..bc6b874f 100644 --- a/tests/unit/scanner/rules/test_untrusted_reaches_trusted.py +++ b/tests/unit/scanner/rules/test_untrusted_reaches_trusted.py @@ -44,6 +44,28 @@ def test_trusted_returning_raw_fires(tmp_path) -> None: assert all(f.kind == Kind.DEFECT for f in findings) +def test_trusted_early_returning_raw_before_later_clean_reassignment_fires(tmp_path) -> None: + ctx, _ = _analyze( + tmp_path, + { + "io.py": "from wardline.decorators import external_boundary\n" + "@external_boundary\ndef read_raw(p):\n return p\n", + "svc.py": "from wardline.decorators import trusted\n" + "from io import read_raw\n" + "@trusted(level='ASSURED', to_level='ASSURED')\n" + "def leaky(p, cond):\n" + " x = read_raw(p)\n" + " if cond:\n" + " return x\n" + " x = 1\n" + " return x\n", + }, + ) + assert ctx.function_return_taints["svc.leaky"] == TaintState.EXTERNAL_RAW + findings = _run(ctx) + assert ("PY-WL-101", "svc.leaky") in {(f.rule_id, f.qualname) for f in findings} + + def test_duplicate_trusted_function_uses_second_definition_for_py_wl_101(tmp_path) -> None: ctx, _ = _analyze( tmp_path, From 84f3d4e859c511d7293cc291b74eb85cd52d71a4 Mon Sep 17 00:00:00 2001 From: John Morrissey Date: Fri, 12 Jun 2026 05:52:59 +1000 Subject: [PATCH 105/186] fix(config): refuse symlinked published port files --- src/wardline/core/config.py | 7 ++++--- src/wardline/core/safe_paths.py | 24 ++++++++++++++++++++++++ src/wardline/install/detect.py | 7 ++++--- tests/unit/core/test_config.py | 20 ++++++++++++++++++++ tests/unit/install/test_detect.py | 21 ++++++++++++++++++++- 5 files changed, 72 insertions(+), 7 deletions(-) diff --git a/src/wardline/core/config.py b/src/wardline/core/config.py index 0726f126..c6dfd2e7 100644 --- a/src/wardline/core/config.py +++ b/src/wardline/core/config.py @@ -20,6 +20,7 @@ legacy_sibling_dir, sibling_state_dir, ) +from wardline.core.safe_paths import safe_read_text_if_regular def validate_boundary_exception_name(value: str) -> str: @@ -267,10 +268,10 @@ def _read_published_port(root: Path, sibling: str) -> int | None: during the federation transition window. Returns a valid port or ``None`` (missing / unreadable / malformed / out-of-range) — fail-soft.""" for base in (sibling_state_dir(root, sibling), legacy_sibling_dir(root, sibling)): - try: - raw = (base / "ephemeral.port").read_text(encoding="ascii").strip() - except (OSError, UnicodeDecodeError): + text = safe_read_text_if_regular(root, base / "ephemeral.port", label="ephemeral.port", encoding="ascii") + if text is None: continue + raw = text.strip() # Guard int(): isdigit() is a superset of what int() parses, so an # all-digit payload over CPython's 4300-digit cap raises ValueError (the # ascii read above already excludes Unicode digits). Catch it so a planted diff --git a/src/wardline/core/safe_paths.py b/src/wardline/core/safe_paths.py index 4b15216e..e8e50573 100644 --- a/src/wardline/core/safe_paths.py +++ b/src/wardline/core/safe_paths.py @@ -2,6 +2,7 @@ from __future__ import annotations +import stat from pathlib import Path from wardline.core.errors import WardlineError @@ -32,3 +33,26 @@ def safe_write_text(root: Path, target: Path, content: str, *, label: str | None """Safely write ``content`` to ``target`` under ``root``, resolving symlinks and boundary checks.""" safe_path = safe_project_file(root, target, label=label) safe_path.write_text(content, encoding="utf-8") + + +def safe_read_text_if_regular( + root: Path, + target: Path, + *, + label: str | None = None, + encoding: str = "utf-8", +) -> str | None: + """Read an optional fixed project file only when it is a regular in-root file. + + Returns ``None`` for missing, symlinked, non-regular, escaping, unreadable, or + undecodable paths. This is for fail-soft discovery/config rungs such as + sibling ``ephemeral.port`` files, where an attacker-controlled repository must + not be able to make Wardline block on a FIFO/device or follow a symlink. + """ + try: + safe_path = safe_project_file(root, target, label=label) + if not stat.S_ISREG(safe_path.stat(follow_symlinks=False).st_mode): + return None + return safe_path.read_text(encoding=encoding) + except (OSError, UnicodeDecodeError, WardlineError): + return None diff --git a/src/wardline/install/detect.py b/src/wardline/install/detect.py index 40e30b9c..6f791d31 100644 --- a/src/wardline/install/detect.py +++ b/src/wardline/install/detect.py @@ -18,6 +18,7 @@ from wardline.core.config import filigree_server_scoped_url from wardline.core.paths import legacy_sibling_dir, sibling_state_dir +from wardline.core.safe_paths import safe_read_text_if_regular def _strip_scalar(value: str) -> str: @@ -81,10 +82,10 @@ def _filigree_url_from_project(root: Path) -> str | None: # ascii read, mirroring core/config._read_published_port: ephemeral.port is # an ASCII integer by protocol, so non-ASCII bytes (incl. Unicode "digit" # chars that pass isdigit() but raise in int()) are rejected at decode time. - try: - text = (base / "ephemeral.port").read_text(encoding="ascii").strip() - except (OSError, UnicodeDecodeError): + port_text = safe_read_text_if_regular(root, base / "ephemeral.port", label="ephemeral.port", encoding="ascii") + if port_text is None: continue + text = port_text.strip() # Guard int(): isdigit() is a superset of what int() parses, so an all-digit # payload over CPython's 4300-digit cap raises ValueError (the ascii read # above already excludes Unicode digits). Catch it so a planted ephemeral.port diff --git a/tests/unit/core/test_config.py b/tests/unit/core/test_config.py index d45b87cb..8f478591 100644 --- a/tests/unit/core/test_config.py +++ b/tests/unit/core/test_config.py @@ -465,6 +465,26 @@ def test_filigree_published_port_malformed_returns_none(tmp_path: Path, monkeypa assert resolve_filigree_url(None, tmp_path, None) is None +@pytest.mark.skipif(not hasattr(Path, "symlink_to"), reason="symlink support unavailable") +def test_filigree_published_port_symlink_is_not_opened(tmp_path: Path, monkeypatch) -> None: + monkeypatch.delenv("WARDLINE_FILIGREE_URL", raising=False) + port_dir = tmp_path / ".weft" / "filigree" + port_dir.mkdir(parents=True) + port_path = port_dir / "ephemeral.port" + port_path.symlink_to(Path("/dev/zero")) + + real_read_text = Path.read_text + + def _read_text(self: Path, *args: object, **kwargs: object) -> str: + if self == port_path: + raise AssertionError("symlinked ephemeral.port must not be opened") + return real_read_text(self, *args, **kwargs) + + monkeypatch.setattr(Path, "read_text", _read_text) + + assert resolve_filigree_url(None, tmp_path, None) is None + + def test_filigree_published_port_boundaries_accepted(tmp_path: Path, monkeypatch) -> None: monkeypatch.delenv("WARDLINE_FILIGREE_URL", raising=False) _publish_filigree_port(tmp_path, "1") diff --git a/tests/unit/install/test_detect.py b/tests/unit/install/test_detect.py index 24a7ed21..929ba98f 100644 --- a/tests/unit/install/test_detect.py +++ b/tests/unit/install/test_detect.py @@ -2,7 +2,7 @@ import pytest -from wardline.install.detect import detect_siblings +from wardline.install.detect import _filigree_url_from_project, detect_siblings def _assert_no_config_written(root: Path) -> None: @@ -62,6 +62,25 @@ def test_filigree_hostile_port_payload_is_soft(tmp_path: Path, monkeypatch, payl _assert_no_config_written(tmp_path) +@pytest.mark.skipif(not hasattr(Path, "symlink_to"), reason="symlink support unavailable") +def test_filigree_published_port_symlink_is_not_opened(tmp_path: Path, monkeypatch) -> None: + port_dir = tmp_path / ".weft" / "filigree" + port_dir.mkdir(parents=True) + port_path = port_dir / "ephemeral.port" + port_path.symlink_to(Path("/dev/zero")) + + real_read_text = Path.read_text + + def _read_text(self: Path, *args: object, **kwargs: object) -> str: + if self == port_path: + raise AssertionError("symlinked ephemeral.port must not be opened") + return real_read_text(self, *args, **kwargs) + + monkeypatch.setattr(Path, "read_text", _read_text) + + assert _filigree_url_from_project(tmp_path) is None + + def test_filigree_legacy_dot_dir_port_is_detected(tmp_path: Path, monkeypatch) -> None: # The legacy .filigree/ dot-dir is tolerated during the federation transition. monkeypatch.delenv("WARDLINE_FILIGREE_URL", raising=False) From 56aa0adeed7dbfec723e5e147fd70e9d76439172 Mon Sep 17 00:00:00 2001 From: John Morrissey Date: Fri, 12 Jun 2026 05:57:19 +1000 Subject: [PATCH 106/186] fix(filigree): prioritize env token aliases over dotenv --- src/wardline/filigree/config.py | 48 ++++++++++++++++++------------ tests/unit/filigree/test_config.py | 24 ++++++++------- 2 files changed, 43 insertions(+), 29 deletions(-) diff --git a/src/wardline/filigree/config.py b/src/wardline/filigree/config.py index 610a85ad..3d9c21ae 100644 --- a/src/wardline/filigree/config.py +++ b/src/wardline/filigree/config.py @@ -9,17 +9,18 @@ ceremony on a same-host install. Resolution order (highest precedence first): 1. env ``WEFT_FEDERATION_TOKEN`` — operator override / cross-host - 2. ``root/.env`` ``WEFT_FEDERATION_TOKEN`` — wardline's portable convention - 3. ``/.weft/filigree/federation_token`` — filigree's auto-minted 0600 file - 4. legacy ``WARDLINE_FILIGREE_TOKEN`` (env then .env) — deprecated transition fallback - 5. None — auth stays off + 2. legacy env ``WARDLINE_FILIGREE_TOKEN`` — deprecated operator fallback + 3. ``root/.env`` ``WEFT_FEDERATION_TOKEN`` — wardline's portable convention + 4. ``/.weft/filigree/federation_token`` — filigree's auto-minted 0600 file + 5. legacy ``root/.env`` ``WARDLINE_FILIGREE_TOKEN`` — deprecated file fallback + 6. None — auth stays off Rung 3 is the same project-store file filigree mints and validates against (the C-9e same-host cross-member read; conventions.md C-3 + conflict-register §A-15): reading it makes the client token match the per-project daemon with no env/.env/.mcp.json config. This token is loopback deconfliction/identification plumbing, **not** a secret — the -0600 mode is filigree's to set; wardline only reads. The env-sourced tokens (rungs 1/4) -otherwise come from env / ``.env`` ONLY, never from weft.toml — the same discipline as +0600 mode is filigree's to set; wardline only reads. The configured tokens otherwise +come from env / ``.env`` ONLY, never from weft.toml — the same discipline as the Loomweave secret and the OpenRouter judge key. """ @@ -41,13 +42,14 @@ _FILIGREE_MINT_RELPATH = (".weft", "filigree", "federation_token") -def _read_token(name: str, root: Path) -> str | None: - """Return the value of ``name`` from the environment, or from a single - ``KEY=VALUE`` line in ``root/.env``, or None. An already-set environment value - always wins over the file.""" +def _read_env_token(name: str) -> str | None: + """Return a non-empty process environment token, or None.""" value = os.environ.get(name) - if value: - return value + return value or None + + +def _read_dotenv_token(name: str, root: Path) -> str | None: + """Return ``name`` from a single ``KEY=VALUE`` line in ``root/.env``, or None.""" env_path = safe_project_file(root, root / ".env", label=".env") if not env_path.is_file(): return None @@ -77,18 +79,26 @@ def _read_filigree_mint(root: Path) -> str | None: def load_filigree_token(root: Path) -> str | None: """Resolve the outbound Filigree bearer token (see the module docstring for the - five-rung order), or None when federation auth is off.""" - # Rungs 1-2: the canonical federation name, resolved fully (env then .env). - value = _read_token(WEFT_FEDERATION_TOKEN_ENV, root) + six-rung order), or None when federation auth is off.""" + # Rungs 1-2: process environment is operator-controlled and outranks every + # project-local token file, including root/.env entries with newer names. + value = _read_env_token(WEFT_FEDERATION_TOKEN_ENV) + if value: + return value + value = _read_env_token(WARDLINE_FILIGREE_TOKEN_ENV) + if value: + return value + # Rung 3: the canonical federation name in root/.env. + value = _read_dotenv_token(WEFT_FEDERATION_TOKEN_ENV, root) if value: return value - # Rung 3: filigree's auto-minted project-store token (same-host, zero-ceremony). + # Rung 4: filigree's auto-minted project-store token (same-host, zero-ceremony). value = _read_filigree_mint(root) if value: return value - # Rung 4: the deprecated legacy name (env then .env) for un-migrated deployments. - value = _read_token(WARDLINE_FILIGREE_TOKEN_ENV, root) + # Rung 5: the deprecated legacy name in root/.env for un-migrated deployments. + value = _read_dotenv_token(WARDLINE_FILIGREE_TOKEN_ENV, root) if value: return value - # Rung 5: off. + # Rung 6: off. return None diff --git a/tests/unit/filigree/test_config.py b/tests/unit/filigree/test_config.py index c5c317a2..dd0d5fc8 100644 --- a/tests/unit/filigree/test_config.py +++ b/tests/unit/filigree/test_config.py @@ -68,15 +68,13 @@ def test_new_name_in_dot_env_wins_over_legacy_in_dot_env(tmp_path: Path) -> None assert load_filigree_token(tmp_path) == "new-file" -def test_new_name_in_dot_env_wins_over_legacy_in_environment(monkeypatch, tmp_path: Path) -> None: - # The migration-relevant cross-tier rung: the new name is resolved FULLY (env then - # .env) before the legacy name is consulted at all, so the new name in .env (rung 2) - # beats the legacy name in the actual environment (rung 3) — a file entry outranks an - # env var across the name boundary. This pins "new-name-first" where it could silently - # regress to legacy-first (relevant while lacuna still exports WARDLINE_FILIGREE_TOKEN). +def test_legacy_environment_wins_over_new_name_in_dot_env(monkeypatch, tmp_path: Path) -> None: + # Process environment is operator-controlled; root/.env may come from the scanned + # repository. All environment aliases must outrank all project .env aliases, even + # across the canonical/legacy name boundary. monkeypatch.setenv(WARDLINE_FILIGREE_TOKEN_ENV, "legacy-env") (tmp_path / ".env").write_text(f"{WEFT_FEDERATION_TOKEN_ENV}=new-file\n", encoding="utf-8") - assert load_filigree_token(tmp_path) == "new-file" + assert load_filigree_token(tmp_path) == "legacy-env" # --------------------------------------------------------------------------- @@ -114,10 +112,16 @@ def test_dot_env_overrides_mint_file(tmp_path: Path) -> None: assert load_filigree_token(tmp_path) == "from-file" -def test_mint_file_wins_over_legacy(monkeypatch, tmp_path: Path) -> None: - # Rung 3 (mint file) outranks rung 4 (legacy name), so a fresh same-host install - # picks the daemon's minted token over a stale legacy env/.env value. +def test_legacy_env_overrides_mint_file(monkeypatch, tmp_path: Path) -> None: + # Process environment is operator-controlled and outranks every repo-local token + # source, including the same-host mint file. monkeypatch.setenv(WARDLINE_FILIGREE_TOKEN_ENV, "legacy-env") + _mint_filigree_token(tmp_path, "minted-tok") + assert load_filigree_token(tmp_path) == "legacy-env" + + +def test_mint_file_wins_over_legacy_dot_env(tmp_path: Path) -> None: + # The same-host mint file still outranks a deprecated repo-local .env fallback. (tmp_path / ".env").write_text(f"{WARDLINE_FILIGREE_TOKEN_ENV}=legacy-file\n", encoding="utf-8") _mint_filigree_token(tmp_path, "minted-tok") assert load_filigree_token(tmp_path) == "minted-tok" From 49ef1f4aa42a1ac645ad9d8bd2035532dd40c5f8 Mon Sep 17 00:00:00 2001 From: John Morrissey Date: Fri, 12 Jun 2026 06:04:20 +1000 Subject: [PATCH 107/186] fix(legis): sign scan scope in artifacts --- docs/guides/legis-handoff.md | 17 ++++++ docs/reference/cli.md | 2 +- src/wardline/core/legis.py | 61 +++++++++++++++++++ src/wardline/mcp/server.py | 39 +++++++++++- tests/conformance/legis_scan_wire.golden.json | 16 ++++- .../test_legis_artifact_contract_freeze.py | 37 ++++++++++- .../test_legis_scan_wire_golden.py | 2 + 7 files changed, 170 insertions(+), 4 deletions(-) diff --git a/docs/guides/legis-handoff.md b/docs/guides/legis-handoff.md index 72820e22..5a690ea3 100644 --- a/docs/guides/legis-handoff.md +++ b/docs/guides/legis-handoff.md @@ -25,6 +25,14 @@ signature: "rule_set_version": "sha256:9f86d0…", "commit_sha": "0a4a00e…", "tree_sha": "4b825dc…", + "scan_scope": { + "schema": "wardline-legis-scan-scope-1", + "scan_root": ".", + "is_git_root": true, + "source_roots": ["."], + "resolved_source_roots": ["."], + "scanned_paths": ["src/service.py"] + }, "findings": [ … ], "artifact_signature": "hmac-sha256:v2:73eb9f0c…" } @@ -38,6 +46,9 @@ The agent posts it verbatim as the `scan` field: one [`attest`](attestation.md) signs. Two artifacts with the same value were produced under the same rules. * **`commit_sha` / `tree_sha`** — the committed revision and its tree object SHA. +* **`scan_scope`** — the signed scope binding: scan root, whether that root is the + git toplevel, configured and resolved `source_roots`, and the files actually + scanned. A signed CI artifact must have `"is_git_root": true`. * **`artifact_signature`** — `hmac-sha256:v2:` over the canonical JSON of every other field (see [Signing](#signing)). @@ -61,6 +72,12 @@ as `unverified` — the trust-the-agent posture before a key is set). `tree_sha` that does not match the scanned content is false provenance, so it is refused rather than emitted. +!!! warning "Signed artifacts are repository-root scans" + When `WARDLINE_LEGIS_ARTIFACT_KEY` is provisioned, Wardline signs only when `PATH` + is the containing git repository root. A subdirectory scan still emits an unsigned + dev artifact when requested without a key, but it cannot be presented as verified + evidence for the repository commit/tree. + !!! tip "Dev/tour loop on a dirty tree: `--allow-dirty`" Signing is clean-tree-only, but you do not need a commit to exercise the Wardline→legis handshake. Pass `--allow-dirty` (CLI) / `allow_dirty: true` (MCP diff --git a/docs/reference/cli.md b/docs/reference/cli.md index 1a61bc66..2e1219d5 100644 --- a/docs/reference/cli.md +++ b/docs/reference/cli.md @@ -103,7 +103,7 @@ it at a package root, not a single file. | Option | Effect | | --- | --- | | `--config FILE` | Path to a `weft.toml` config file; Wardline reads its `[wardline]` table for rule enable/severity and judge settings (defaults to `weft.toml` in the scan path). | -| `--format [jsonl\|sarif\|agent-summary\|legis]` | Output shape. `jsonl` is one finding per line; `sarif` is SARIF 2.1.0 for GitHub code-scanning and other generic SARIF consumers; `agent-summary` is stable versioned JSON for agents (`schema: wardline-agent-summary-1`) with active defects first, suppressed findings, engine facts, integration status, and suggested next tool calls; `legis` is the signed, verbatim-postable `scan` for legis's `POST /wardline/scan-results` (signed when `WARDLINE_LEGIS_ARTIFACT_KEY` is provisioned — write it **outside** the working tree, see the [legis handoff guide](../guides/legis-handoff.md)). SARIF carries Wardline identity in `partialFingerprints["wardlineFingerprint/v2"]`; downstream Filigree lifecycle quality depends on importers preserving that field. | +| `--format [jsonl\|sarif\|agent-summary\|legis]` | Output shape. `jsonl` is one finding per line; `sarif` is SARIF 2.1.0 for GitHub code-scanning and other generic SARIF consumers; `agent-summary` is stable versioned JSON for agents (`schema: wardline-agent-summary-1`) with active defects first, suppressed findings, engine facts, integration status, and suggested next tool calls; `legis` is the signed, verbatim-postable `scan` for legis's `POST /wardline/scan-results` (signed when `WARDLINE_LEGIS_ARTIFACT_KEY` is provisioned and `PATH` is the git repository root — write it **outside** the working tree, see the [legis handoff guide](../guides/legis-handoff.md)). SARIF carries Wardline identity in `partialFingerprints["wardlineFingerprint/v2"]`; downstream Filigree lifecycle quality depends on importers preserving that field. | | `--lang [python\|rust]` | Language frontend (default `python`). `rust` sweeps `*.rs` and covers the **command-injection slice** (`RS-WL-108`/`RS-WL-112`); needs the `wardline[rust]` extra. Finding identity is frozen and crate-prefixed (baseline-eligible); config severity overrides do not yet apply to Rust findings — see the [Rust support guide](../guides/rust-preview.md). | | `--output PATH` | Write findings to a file instead of stdout. | | `--fail-on [CRITICAL\|ERROR\|WARN\|INFO]` | Exit non-zero when any finding at or above this severity survives the baseline. Use this as your CI gate. | diff --git a/src/wardline/core/legis.py b/src/wardline/core/legis.py index c47ea196..db6d5d69 100644 --- a/src/wardline/core/legis.py +++ b/src/wardline/core/legis.py @@ -70,6 +70,8 @@ FINDINGS_FIELD = "findings" DIRTY_FIELD = "dirty" FINGERPRINT_SCHEME_FIELD = "fingerprint_scheme" +SCAN_SCOPE_FIELD = "scan_scope" +SCAN_SCOPE_SCHEMA = "wardline-legis-scan-scope-1" # The one shared vocabulary — legis carries these 8 tiers verbatim (TRUST_TIERS in # legis ingest.py). Sourced from the lattice so the two can never drift. @@ -209,6 +211,55 @@ def _git_tree_sha(root: Path) -> str | None: return rev.stdout.strip() or None +def _git_repo_root(root: Path) -> Path | None: + """The containing git repository root, or None when unavailable.""" + try: + rev = subprocess.run( + ["git", "rev-parse", "--show-toplevel"], + cwd=root, + capture_output=True, + text=True, + ) + except OSError: + return None + if rev.returncode != 0: + return None + value = rev.stdout.strip() + if not value: + return None + try: + return Path(value).resolve() + except OSError: + return None + + +def _relative_posix(path: Path, base: Path) -> str: + """Return *path* relative to *base* in stable POSIX form, allowing ``..``.""" + try: + rel = path.relative_to(base) + except ValueError: + rel = Path(os.path.relpath(path, base)) + return rel.as_posix() if rel.parts else "." + + +def _scan_scope(result: ScanResult, *, root: Path, config: WardlineConfig) -> dict[str, Any]: + """Signed description of the exact scan scope carried by the artifact.""" + resolved_root = root.resolve() + repo_root = _git_repo_root(root) + scope_base = repo_root if repo_root is not None else resolved_root + resolved_source_roots = [ + _relative_posix((resolved_root / source_root).resolve(), scope_base) for source_root in config.source_roots + ] + return { + "schema": SCAN_SCOPE_SCHEMA, + "scan_root": _relative_posix(resolved_root, scope_base), + "is_git_root": repo_root is not None and resolved_root == repo_root, + "source_roots": list(config.source_roots), + "resolved_source_roots": resolved_source_roots, + "scanned_paths": list(result.scanned_paths), + } + + def build_legis_artifact( result: ScanResult, *, @@ -265,8 +316,13 @@ def build_legis_artifact( # fingerprints stay BARE — legis reads them from to_jsonl (SARIF-style value). FINGERPRINT_SCHEME_FIELD: FINGERPRINT_SCHEME, FINDINGS_FIELD: findings, + # Bind the artifact to the requested and realized scope. This prevents a + # signed subdirectory or narrowed-source-root scan from being indistinguishable + # from a full-repository scan carrying the same commit/tree provenance. + SCAN_SCOPE_FIELD: _scan_scope(result, root=root, config=config), } commit, dirty = git_state(root) + repo_root = _git_repo_root(root) # Signing is CLEAN-TREE-ONLY. A key + clean tree produces the signed, verified # artifact. A key + dirty tree is refused loudly UNLESS ``allow_dirty`` — and even @@ -281,6 +337,11 @@ def build_legis_artifact( raise LegisArtifactError( "cannot sign legis artifact: not a git repository, so commit/tree provenance is unavailable" ) + if repo_root is None or root.resolve() != repo_root: + raise LegisArtifactError( + "cannot sign legis artifact: scan root is not the git repository root; " + "scan the repository root so commit/tree provenance and scan scope match" + ) tree = _git_tree_sha(root) if tree is None: raise LegisArtifactError("cannot sign legis artifact: tree SHA unavailable") diff --git a/src/wardline/mcp/server.py b/src/wardline/mcp/server.py index a0806947..911d728b 100644 --- a/src/wardline/mcp/server.py +++ b/src/wardline/mcp/server.py @@ -1191,6 +1191,43 @@ def _scan( "additionalProperties": False, }, }, + "scan_scope": { + "type": "object", + "description": ( + "Signed scope binding: scan root, configured/resolved source roots, and realized files." + ), + "properties": { + "schema": {"type": "string", "enum": ["wardline-legis-scan-scope-1"]}, + "scan_root": { + "type": "string", + "description": "Scan root relative to the git repository root when in git; otherwise '.'.", + }, + "is_git_root": { + "type": "boolean", + "description": "True only when the scan root is the containing git repository root.", + }, + "source_roots": {"type": "array", "items": {"type": "string"}}, + "resolved_source_roots": { + "type": "array", + "items": {"type": "string"}, + "description": "Configured source roots resolved relative to the signed scope base.", + }, + "scanned_paths": { + "type": "array", + "items": {"type": "string"}, + "description": "Files actually discovered and analyzed, relative to the scan root.", + }, + }, + "required": [ + "schema", + "scan_root", + "is_git_root", + "source_roots", + "resolved_source_roots", + "scanned_paths", + ], + "additionalProperties": False, + }, "commit_sha": { "type": "string", "description": "Present when provenance was readable (always on the signed path).", @@ -1209,7 +1246,7 @@ def _scan( "description": "Present (true) only when the working tree was dirty (unsigned dev artifact).", }, }, - "required": ["scanner_identity", "rule_set_version", "fingerprint_scheme", "findings"], + "required": ["scanner_identity", "rule_set_version", "fingerprint_scheme", "findings", "scan_scope"], "additionalProperties": False, }, "legis_artifact_status": { diff --git a/tests/conformance/legis_scan_wire.golden.json b/tests/conformance/legis_scan_wire.golden.json index 3b1c8fb0..a418fee4 100644 --- a/tests/conformance/legis_scan_wire.golden.json +++ b/tests/conformance/legis_scan_wire.golden.json @@ -17,7 +17,21 @@ "suppression_state": "active" } ], + "scan_scope": { + "schema": "wardline-legis-scan-scope-1", + "scan_root": ".", + "is_git_root": true, + "source_roots": [ + "." + ], + "resolved_source_roots": [ + "." + ], + "scanned_paths": [ + "svc.py" + ] + }, "commit_sha": "0000000000000000000000000000000000000000", "tree_sha": "1111111111111111111111111111111111111111", - "artifact_signature": "hmac-sha256:v2:05e268b2e90dd8ab446c148a110d7367cb47854551b695f13cee3274ad2a2c86" + "artifact_signature": "hmac-sha256:v2:15ffe67f96e2298d608d901cfd72328b77ee7235b9418ff0c190a168f0e429b5" } diff --git a/tests/conformance/test_legis_artifact_contract_freeze.py b/tests/conformance/test_legis_artifact_contract_freeze.py index 4b1d442b..21b3d91f 100644 --- a/tests/conformance/test_legis_artifact_contract_freeze.py +++ b/tests/conformance/test_legis_artifact_contract_freeze.py @@ -52,6 +52,7 @@ import pytest from wardline.core.config import load as load_config +from wardline.core.errors import LegisArtifactError from wardline.core.finding import Finding, Kind, Location, Severity, SuppressionState from wardline.core.legis import build_legis_artifact, project_finding from wardline.core.run import run_scan @@ -66,7 +67,7 @@ # --- The frozen contract ----------------------------------------------------- # Always present, in every mode. -_BASE = frozenset({"scanner_identity", "rule_set_version", "fingerprint_scheme", "findings"}) +_BASE = frozenset({"scanner_identity", "rule_set_version", "fingerprint_scheme", "findings", "scan_scope"}) # No git repo: no commit/tree provenance to read. _NON_REPO = _BASE # A clean committed repo, no signing key: best-effort committed provenance. @@ -80,6 +81,9 @@ _FINDING_KEYS = frozenset( {"rule_id", "message", "severity", "kind", "fingerprint", "qualname", "properties", "suppression_state"} ) +_SCOPE_KEYS = frozenset( + {"schema", "scan_root", "is_git_root", "source_roots", "resolved_source_roots", "scanned_paths"} +) def _proj(tmp_path: Path) -> Path: @@ -152,6 +156,37 @@ def test_projected_finding_key_set_is_frozen(tmp_path: Path) -> None: assert set(finding) == _FINDING_KEYS +def test_scan_scope_key_set_and_values_are_frozen(tmp_path: Path) -> None: + root = tmp_path / "proj" + (root / "src").mkdir(parents=True) + (root / "src" / "svc.py").write_text(_LEAKY, encoding="utf-8") + (root / "weft.toml").write_text('[wardline]\nsource_roots = ["src"]\n', encoding="utf-8") + + scan = _build(root, key=None) + scope = scan["scan_scope"] + + assert set(scope) == _SCOPE_KEYS + assert scope["schema"] == "wardline-legis-scan-scope-1" + assert scope["scan_root"] == "." + assert scope["is_git_root"] is False + assert scope["source_roots"] == ["src"] + assert scope["resolved_source_roots"] == ["src"] + assert scope["scanned_paths"] == ["src/svc.py"] + + +def test_signed_artifact_refuses_subdirectory_scan_root(tmp_path: Path) -> None: + root = _proj(tmp_path) + subdir = root / "safe" + subdir.mkdir() + (subdir / "svc.py").write_text(_LEAKY, encoding="utf-8") + _git_commit(root) + result = run_scan(subdir) + cfg = load_config(subdir / "weft.toml") + + with pytest.raises(LegisArtifactError, match="git repository root"): + build_legis_artifact(result, root=subdir, config=cfg, key=b"shared-secret") + + def test_findings_is_a_list(tmp_path: Path) -> None: # Guard the container TYPE, not just the key name: legis iterates findings; a # key whose value silently became a dict would route nothing the same way a diff --git a/tests/conformance/test_legis_scan_wire_golden.py b/tests/conformance/test_legis_scan_wire_golden.py index de4411d2..75f38a55 100644 --- a/tests/conformance/test_legis_scan_wire_golden.py +++ b/tests/conformance/test_legis_scan_wire_golden.py @@ -33,6 +33,7 @@ DIRTY_FIELD, FINDINGS_FIELD, FINGERPRINT_SCHEME_FIELD, + SCAN_SCOPE_FIELD, build_legis_artifact, sign_artifact, ) @@ -86,6 +87,7 @@ def test_golden_vector_keys_are_the_named_constants() -> None: vector = _vector() assert FINDINGS_FIELD in vector assert FINGERPRINT_SCHEME_FIELD in vector + assert SCAN_SCOPE_FIELD in vector assert DIRTY_FIELD not in vector # clean signed artifact carries no dirty marker assert isinstance(vector[FINDINGS_FIELD], list) and vector[FINDINGS_FIELD] From 687345c3c1aac4cfe00d6325e954464d5513bef0 Mon Sep 17 00:00:00 2001 From: John Morrissey Date: Fri, 12 Jun 2026 06:06:15 +1000 Subject: [PATCH 108/186] fix(cli): guard default agent-summary output --- src/wardline/cli/scan.py | 11 ++++++++--- tests/unit/cli/test_agent_summary_cmd.py | 13 +++++++++++++ 2 files changed, 21 insertions(+), 3 deletions(-) diff --git a/src/wardline/cli/scan.py b/src/wardline/cli/scan.py index a590f246..5ea5b298 100644 --- a/src/wardline/cli/scan.py +++ b/src/wardline/cli/scan.py @@ -21,6 +21,7 @@ from wardline.core.finding import Severity from wardline.core.paths import weft_config_path from wardline.core.run import baseline_migration_hint, gate_decision, run_scan +from wardline.core.safe_paths import safe_write_text from wardline.core.sarif import SarifSink @@ -188,6 +189,7 @@ def scan( default_name = "scan.legis.json" else: default_name = "findings.jsonl" + output_is_default = output is None output = output if output is not None else (path / default_name) emit_result: EmitResult | None = None loomweave_result = None @@ -314,7 +316,7 @@ def confirm_cb(rel_path: str, orig: str, replacement: str, f: Finding) -> bool: from wardline.core.agent_summary import build_agent_summary decision = gate_decision(result, Severity(fail_on)) if fail_on is not None else gate_decision(result, None) - output.write_text( + agent_summary_json = ( json.dumps( build_agent_summary( result, @@ -325,9 +327,12 @@ def confirm_cb(rel_path: str, orig: str, replacement: str, f: Finding) -> bool: ).to_dict(), sort_keys=True, ) - + "\n", - encoding="utf-8", + + "\n" ) + if output_is_default: + safe_write_text(path, output, agent_summary_json, label=default_name) + else: + output.write_text(agent_summary_json, encoding="utf-8") except WardlineError as exc: click.echo(f"error: {exc}", err=True) raise SystemExit(2) from exc diff --git a/tests/unit/cli/test_agent_summary_cmd.py b/tests/unit/cli/test_agent_summary_cmd.py index bf024fdd..55eef7b1 100644 --- a/tests/unit/cli/test_agent_summary_cmd.py +++ b/tests/unit/cli/test_agent_summary_cmd.py @@ -29,3 +29,16 @@ def test_scan_agent_summary_format_writes_stable_json(tmp_path: Path) -> None: assert data["schema"] == "wardline-agent-summary-1" assert data["active_defects"][0]["rule_id"] == "PY-WL-101" assert "summary.json" in result.output + + +def test_scan_agent_summary_default_output_refuses_repo_symlink(tmp_path: Path) -> None: + (tmp_path / "svc.py").write_text(_LEAKY, encoding="utf-8") + outside = tmp_path.parent / f"{tmp_path.name}-outside.json" + outside.write_text("keep-me\n", encoding="utf-8") + (tmp_path / "findings.agent-summary.json").symlink_to(outside) + + result = CliRunner().invoke(cli, ["scan", str(tmp_path), "--format", "agent-summary"]) + + assert result.exit_code == 2 + assert "findings.agent-summary.json: refusing to write through a symlink" in result.stderr + assert outside.read_text(encoding="utf-8") == "keep-me\n" From 0546dee9cf828b25147826b52982cebb3d5964e4 Mon Sep 17 00:00:00 2001 From: John Morrissey Date: Fri, 12 Jun 2026 06:07:46 +1000 Subject: [PATCH 109/186] fix(cli): honor scan-file-findings gate exit --- src/wardline/cli/scan_file_findings.py | 3 +++ tests/unit/cli/test_scan_file_findings_cmd.py | 21 +++++++++++++++++++ 2 files changed, 24 insertions(+) diff --git a/src/wardline/cli/scan_file_findings.py b/src/wardline/cli/scan_file_findings.py index 03a5d719..e8263fd5 100644 --- a/src/wardline/cli/scan_file_findings.py +++ b/src/wardline/cli/scan_file_findings.py @@ -90,3 +90,6 @@ def scan_file_findings( click.echo(f"error: {exc}", err=True) raise SystemExit(2) from exc click.echo(json.dumps(result)) + exit_class = result.get("gate", {}).get("exit_class", 0) + if exit_class: + raise SystemExit(exit_class) diff --git a/tests/unit/cli/test_scan_file_findings_cmd.py b/tests/unit/cli/test_scan_file_findings_cmd.py index 6325f85e..0a6bc89d 100644 --- a/tests/unit/cli/test_scan_file_findings_cmd.py +++ b/tests/unit/cli/test_scan_file_findings_cmd.py @@ -8,6 +8,16 @@ from wardline.cli.main import cli +_LEAKY = ( + "from wardline.decorators import external_boundary, trusted\n" + "@external_boundary\n" + "def read_raw(p):\n" + " return p\n" + "@trusted\n" + "def leaky(p):\n" + " return read_raw(p)\n" +) + def test_scan_file_findings_cli_defaults_to_dry_run(tmp_path, monkeypatch): from wardline.cli import scan_file_findings as mod @@ -56,3 +66,14 @@ def __init__(self, url, *, secret, project): assert seen["dry_run"] is False assert seen["filigree_emitter"] == ("emitter", "http://f/api/weft/scan-results") assert isinstance(seen["loomweave_client"], FakeLoomweave) + + +def test_scan_file_findings_cli_fail_on_uses_gate_exit_class(tmp_path): + (tmp_path / "svc.py").write_text(_LEAKY, encoding="utf-8") + + res = CliRunner().invoke(cli, ["scan-file-findings", str(tmp_path), "--fail-on", "ERROR"]) + + assert res.exit_code == 1 + payload = json.loads(res.output) + assert payload["gate"]["tripped"] is True + assert payload["gate"]["exit_class"] == 1 From 668e2cc2be799d1fb7e0d7272f62217d5b755488 Mon Sep 17 00:00:00 2001 From: John Morrissey Date: Fri, 12 Jun 2026 06:09:33 +1000 Subject: [PATCH 110/186] fix(mcp): preserve strict defaults for SEI filters --- src/wardline/core/sei_resolution.py | 4 +++- src/wardline/mcp/server.py | 8 +++++++- tests/unit/mcp/test_server_query_explain.py | 19 +++++++++++++++++++ 3 files changed, 29 insertions(+), 2 deletions(-) diff --git a/src/wardline/core/sei_resolution.py b/src/wardline/core/sei_resolution.py index 4600cd8c..5c7031a7 100644 --- a/src/wardline/core/sei_resolution.py +++ b/src/wardline/core/sei_resolution.py @@ -26,6 +26,8 @@ def resolve_query_filters( root: Path, config_path: Path | None, loomweave_client: Any = None, + *, + strict_defaults: bool = False, ) -> dict[str, Any] | None: """Resolve a `qualname` filter starting with `sei:` in findings queries to its resolved qualname.""" if not where or "qualname" not in where: @@ -38,7 +40,7 @@ def resolve_query_filters( if loomweave_client is None: from wardline.core.config import resolve_loomweave_url - loomweave_url = resolve_loomweave_url(None, root, config_path) + loomweave_url = resolve_loomweave_url(None, root, config_path, strict_defaults=strict_defaults) if loomweave_url is not None: from wardline.loomweave.client import LoomweaveClient from wardline.loomweave.config import load_loomweave_token, resolve_project_name diff --git a/src/wardline/mcp/server.py b/src/wardline/mcp/server.py index 911d728b..ba89ca22 100644 --- a/src/wardline/mcp/server.py +++ b/src/wardline/mcp/server.py @@ -700,7 +700,13 @@ def _scan( loomweave_status = _loomweave_write_status(loomweave_block) where = args.get("where") try: - resolved_where = resolve_query_filters(where, root, _cfg(args, root), loomweave) + resolved_where = resolve_query_filters( + where, + root, + _cfg(args, root), + loomweave, + strict_defaults=strict_defaults, + ) selected = filter_findings(result.findings, resolved_where) except (ValueError, WardlineError) as exc: # An unknown filter key or SEI resolution failure is agent-actionable -> isError result. diff --git a/tests/unit/mcp/test_server_query_explain.py b/tests/unit/mcp/test_server_query_explain.py index 0e855b9f..d27abc0f 100644 --- a/tests/unit/mcp/test_server_query_explain.py +++ b/tests/unit/mcp/test_server_query_explain.py @@ -60,6 +60,25 @@ def test_where_unknown_key_is_toolerror(tmp_path): _scan({"where": {"bogus": "x"}}, tmp_path) +def test_where_sei_filter_honors_strict_defaults(tmp_path, monkeypatch): + from wardline.core import config as config_mod + + (tmp_path / "svc.py").write_text(_SRC, encoding="utf-8") + seen: dict[str, bool] = {} + + def fake_resolve_loomweave_url(flag, root, config_path, *, strict_defaults=False): + seen["strict_defaults"] = strict_defaults + if not strict_defaults: + raise AssertionError("strict_defaults was not propagated to SEI filter resolution") + return None + + monkeypatch.setattr(config_mod, "resolve_loomweave_url", fake_resolve_loomweave_url) + + with pytest.raises(ToolError, match="no Loomweave URL configured"): + _scan({"where": {"qualname": "sei:python:function:svc.leak_a"}}, tmp_path, strict_defaults=True) + assert seen["strict_defaults"] is True + + def test_explain_inlines_provenance_on_active_defects(tmp_path): (tmp_path / "svc.py").write_text(_SRC, encoding="utf-8") out = _scan({"explain": True}, tmp_path) From aae950a0e8a18781ecefb077f15a5a484c1260ad Mon Sep 17 00:00:00 2001 From: John Morrissey Date: Fri, 12 Jun 2026 06:12:58 +1000 Subject: [PATCH 111/186] fix(cli): fail stale attestation reproduction --- src/wardline/cli/attest.py | 4 +++- tests/unit/cli/test_attest_cmd.py | 23 +++++++++++++++++++++++ 2 files changed, 26 insertions(+), 1 deletion(-) diff --git a/src/wardline/cli/attest.py b/src/wardline/cli/attest.py index 04b597e2..76d531f1 100644 --- a/src/wardline/cli/attest.py +++ b/src/wardline/cli/attest.py @@ -138,7 +138,9 @@ def attest( click.echo(f"error: invalid attestation bundle: {exc}", err=True) raise SystemExit(2) from exc click.echo(json.dumps(result)) - raise SystemExit(0 if result["signature_valid"] else 1) + if not result["signature_valid"] or (reproduce and result.get("reproduced") is not True): + raise SystemExit(1) + raise SystemExit(0) from wardline.core.attest import build_attestation diff --git a/tests/unit/cli/test_attest_cmd.py b/tests/unit/cli/test_attest_cmd.py index bf667313..6fbc608e 100644 --- a/tests/unit/cli/test_attest_cmd.py +++ b/tests/unit/cli/test_attest_cmd.py @@ -187,6 +187,29 @@ def test_verify_mode_valid_and_tampered(tmp_path: Path, monkeypatch: pytest.Monk assert json.loads(bad.output)["signature_valid"] is False +def test_verify_reproduce_stale_bundle_exits_1(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None: + monkeypatch.delenv("WARDLINE_ATTEST_KEY", raising=False) + _make_clean_repo(tmp_path) + + runner = CliRunner() + bundle_path = tmp_path / "attest.json" + built = runner.invoke(cli, ["attest", str(tmp_path), "--out", str(bundle_path)]) + assert built.exit_code == 0, built.output + + (tmp_path / "m.py").write_text( + _MODULE + "\n@trusted(level='INTEGRAL')\ndef extra():\n return 2\n", + encoding="utf-8", + ) + + stale = runner.invoke(cli, ["attest", str(tmp_path), "--verify", str(bundle_path), "--reproduce"]) + + assert stale.exit_code == 1 + result = json.loads(stale.output) + assert result["signature_valid"] is True + assert result["reproduced"] is False + assert "boundaries" in result["mismatches"] + + def test_verify_reproduce_refuses_escaping_source_roots_by_default( tmp_path: Path, monkeypatch: pytest.MonkeyPatch ) -> None: From faf67c2b0712784934fa528338a96750e1782a52 Mon Sep 17 00:00:00 2001 From: John Morrissey Date: Fri, 12 Jun 2026 06:15:38 +1000 Subject: [PATCH 112/186] fix(attest): disable repo fsmonitor during git state --- src/wardline/core/attest.py | 6 +++-- tests/unit/core/test_attest.py | 40 ++++++++++++++++++++++++++++++++++ 2 files changed, 44 insertions(+), 2 deletions(-) diff --git a/src/wardline/core/attest.py b/src/wardline/core/attest.py index e9900fb6..5c7236f0 100644 --- a/src/wardline/core/attest.py +++ b/src/wardline/core/attest.py @@ -60,6 +60,8 @@ from wardline.core.waivers import load_project_waivers ATTEST_SCHEMA = "wardline-attest-1" +# `git status` must not execute repo-local helpers while attesting untrusted trees. +_SAFE_GIT_CONFIG = ("-c", "core.fsmonitor=false") def git_state(root: Path) -> tuple[str | None, bool]: @@ -74,7 +76,7 @@ def git_state(root: Path) -> tuple[str | None, bool]: """ try: rev = subprocess.run( - ["git", "rev-parse", "HEAD"], + ["git", *_SAFE_GIT_CONFIG, "rev-parse", "HEAD"], cwd=root, capture_output=True, text=True, @@ -89,7 +91,7 @@ def git_state(root: Path) -> tuple[str | None, bool]: try: status = subprocess.run( - ["git", "status", "--porcelain"], + ["git", *_SAFE_GIT_CONFIG, "status", "--porcelain"], cwd=root, capture_output=True, text=True, diff --git a/tests/unit/core/test_attest.py b/tests/unit/core/test_attest.py index e2baaaf5..0d6bb9b6 100644 --- a/tests/unit/core/test_attest.py +++ b/tests/unit/core/test_attest.py @@ -14,6 +14,7 @@ from __future__ import annotations +import shlex import subprocess from datetime import date from pathlib import Path @@ -231,6 +232,45 @@ def test_git_state_repo_clean_dirty_and_non_git(tmp_path: Path) -> None: assert dirty2 is True +def test_git_state_disables_repo_configured_fsmonitor(tmp_path: Path) -> None: + repo = tmp_path / "repo" + repo.mkdir() + (repo / "f.py").write_text("x = 1\n", encoding="utf-8") + _git(["init"], repo) + _git(["add", "-A"], repo) + _git( + [ + "-c", + "user.email=t@example.com", + "-c", + "user.name=Test", + "commit", + "-m", + "init", + ], + repo, + ) + + log_path = tmp_path / "fsmonitor.log" + hook_path = tmp_path / "fsmonitor.sh" + hook_path.write_text( + f"#!/bin/sh\nprintf 'fsmonitor ran\\n' >> {shlex.quote(str(log_path))}\nprintf 'hook-token\\n'\n", + encoding="utf-8", + ) + hook_path.chmod(0o755) + _git(["config", "core.fsmonitor", str(hook_path)], repo) + + commit, dirty = git_state(repo) + + assert commit is not None + assert dirty is False + assert not log_path.exists() + + (repo / "f.py").write_text("x = 2\n", encoding="utf-8") + assert git_state(repo) == (commit, True) + assert not log_path.exists() + + # --------------------------------------------------------------------------- # # 3. build_attestation + signature round-trip / wrong-key / tamper # --------------------------------------------------------------------------- # From 81b457f77146403a507b65a19e6364cda5f3f8ff Mon Sep 17 00:00:00 2001 From: John Morrissey Date: Fri, 12 Jun 2026 06:18:47 +1000 Subject: [PATCH 113/186] fix(install): refuse attest key mint into tracked env --- docs/guides/attestation.md | 7 ++++-- src/wardline/core/attest_key.py | 28 +++++++++++++++++++++-- tests/unit/cli/test_install_attest_key.py | 25 ++++++++++++++++++++ tests/unit/core/test_attest_key.py | 22 ++++++++++++++++++ 4 files changed, 78 insertions(+), 4 deletions(-) diff --git a/docs/guides/attestation.md b/docs/guides/attestation.md index 0d09e63e..88ac1821 100644 --- a/docs/guides/attestation.md +++ b/docs/guides/attestation.md @@ -25,8 +25,11 @@ system, a governance plugin like legis, or an agent making a deploy decision. `wardline install` mints a 64-hex signing key and appends it to `.env` at the project root as `WARDLINE_ATTEST_KEY=""`. It also ensures `.env` is listed -in `.gitignore` — the key must never be committed. This is the only setup step. -Pass `--no-attest-key` to `wardline install` if you want to skip minting. +in `.gitignore` — the key must never be committed. If `.env` is already tracked +by Git, install refuses to mint into it; untrack `.env` first or pass +`--no-attest-key` and provide the key through the environment. This is the only +setup step. Pass `--no-attest-key` to `wardline install` if you want to skip +minting. The key lookup order at run time: environment variable `WARDLINE_ATTEST_KEY` → `root/.env` line `WARDLINE_ATTEST_KEY=`. An already-set environment value diff --git a/src/wardline/core/attest_key.py b/src/wardline/core/attest_key.py index 63357afc..1ccbae43 100644 --- a/src/wardline/core/attest_key.py +++ b/src/wardline/core/attest_key.py @@ -9,12 +9,30 @@ import hashlib import os import secrets +import subprocess from contextlib import suppress from pathlib import Path +from wardline.core.errors import WardlineError from wardline.core.safe_paths import safe_project_file WARDLINE_ATTEST_KEY_ENV = "WARDLINE_ATTEST_KEY" +_SAFE_GIT_CONFIG = ("-c", "core.fsmonitor=false") + + +def _git_tracks_path(root: Path, target: Path) -> bool: + try: + root_resolved = root.resolve() + relpath = target.resolve(strict=False).relative_to(root_resolved).as_posix() + result = subprocess.run( + ["git", *_SAFE_GIT_CONFIG, "ls-files", "--error-unmatch", "--", relpath], + cwd=root_resolved, + capture_output=True, + text=True, + ) + except (OSError, ValueError): + return False + return result.returncode == 0 def load_attest_key(root: Path) -> str | None: @@ -52,10 +70,16 @@ def mint_attest_key(root: Path) -> tuple[str, str]: if existing: return existing, "present" - key = secrets.token_hex(32) - # --- write to .env -------------------------------------------------- env_path = safe_project_file(root, root / ".env", label=".env") + if _git_tracks_path(root, env_path): + raise WardlineError( + "refusing to mint WARDLINE_ATTEST_KEY into tracked .env; " + "untrack .env or pass --no-attest-key and provide WARDLINE_ATTEST_KEY from the environment" + ) + + key = secrets.token_hex(32) + if env_path.exists(): text = env_path.read_text(encoding="utf-8") if not text.endswith("\n"): diff --git a/tests/unit/cli/test_install_attest_key.py b/tests/unit/cli/test_install_attest_key.py index f85537b4..01eef804 100644 --- a/tests/unit/cli/test_install_attest_key.py +++ b/tests/unit/cli/test_install_attest_key.py @@ -3,6 +3,7 @@ from __future__ import annotations +import subprocess from pathlib import Path from click.testing import CliRunner @@ -20,6 +21,10 @@ ] +def _git(args: list[str], cwd: Path) -> None: + subprocess.run(["git", *args], cwd=cwd, check=True, capture_output=True, text=True) + + def test_install_mints_attest_key(tmp_path: Path, monkeypatch) -> None: """First run: key is minted, .env created, .gitignore contains .env.""" monkeypatch.delenv("WARDLINE_ATTEST_KEY", raising=False) @@ -69,3 +74,23 @@ def test_install_attest_key_idempotent(tmp_path: Path, monkeypatch) -> None: second = runner.invoke(cli, args) assert second.exit_code == 0, second.output assert "attest key: present" in second.output + + +def test_install_refuses_to_mint_into_tracked_dotenv(tmp_path: Path, monkeypatch) -> None: + monkeypatch.delenv("WARDLINE_ATTEST_KEY", raising=False) + _git(["init"], tmp_path) + _git(["config", "user.email", "test@example.com"], tmp_path) + _git(["config", "user.name", "Test"], tmp_path) + dotenv = tmp_path / ".env" + dotenv.write_text("EXISTING=1\n", encoding="utf-8") + _git(["add", ".env"], tmp_path) + _git(["commit", "-m", "track env"], tmp_path) + + result = CliRunner().invoke( + cli, + ["install", "--root", str(tmp_path), *_SKIP_ALL_OTHER], + ) + + assert result.exit_code == 2 + assert "tracked .env" in result.stderr + assert dotenv.read_text(encoding="utf-8") == "EXISTING=1\n" diff --git a/tests/unit/core/test_attest_key.py b/tests/unit/core/test_attest_key.py index b58f07ea..a670f721 100644 --- a/tests/unit/core/test_attest_key.py +++ b/tests/unit/core/test_attest_key.py @@ -4,6 +4,7 @@ from __future__ import annotations import re +import subprocess from pathlib import Path import pytest @@ -16,6 +17,11 @@ ) from wardline.core.errors import WardlineError + +def _git(args: list[str], cwd: Path) -> None: + subprocess.run(["git", *args], cwd=cwd, check=True, capture_output=True, text=True) + + # --------------------------------------------------------------------------- # Test 1: load_attest_key — env wins # --------------------------------------------------------------------------- @@ -91,6 +97,22 @@ def test_mint_rejects_symlinked_dotenv(tmp_path: Path, monkeypatch: pytest.Monke assert outside.read_text(encoding="utf-8") == "" +def test_mint_refuses_tracked_dotenv(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None: + monkeypatch.delenv(WARDLINE_ATTEST_KEY_ENV, raising=False) + _git(["init"], tmp_path) + _git(["config", "user.email", "test@example.com"], tmp_path) + _git(["config", "user.name", "Test"], tmp_path) + dotenv = tmp_path / ".env" + dotenv.write_text("EXISTING=1\n", encoding="utf-8") + _git(["add", ".env"], tmp_path) + _git(["commit", "-m", "track env"], tmp_path) + + with pytest.raises(WardlineError, match="tracked \\.env"): + mint_attest_key(tmp_path) + + assert dotenv.read_text(encoding="utf-8") == "EXISTING=1\n" + + def test_mint_key_loadable_after_mint(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None: """load_attest_key returns the minted key immediately after minting.""" monkeypatch.delenv(WARDLINE_ATTEST_KEY_ENV, raising=False) From 8b51b4cc993b2e9c464df30beecd7c43fc6d2959 Mon Sep 17 00:00:00 2001 From: John Morrissey Date: Fri, 12 Jun 2026 06:23:14 +1000 Subject: [PATCH 114/186] fix(assure): count unanalyzed files in coverage --- docs/guides/assurance-posture.md | 28 +++++++++++++++++------- src/wardline/cli/assure.py | 16 +++++++++----- src/wardline/core/assure.py | 36 ++++++++++++++++++++++--------- src/wardline/mcp/server.py | 20 ++++++++++++----- tests/unit/cli/test_assure_cmd.py | 15 +++++++++++++ tests/unit/core/test_assure.py | 24 +++++++++++++++++++++ 6 files changed, 111 insertions(+), 28 deletions(-) diff --git a/docs/guides/assurance-posture.md b/docs/guides/assurance-posture.md index 532b081f..e751d196 100644 --- a/docs/guides/assurance-posture.md +++ b/docs/guides/assurance-posture.md @@ -28,7 +28,9 @@ fires, and it never enters the coverage denominator. `assure` is not about ## Coverage: definite verdict vs. the honesty gap ``` -coverage_pct = 100 × (boundaries_total − unknown_count) / boundaries_total +coverage_pct = + 100 × (boundaries_total − unknown_count) + / (boundaries_total + unanalyzed_total) ``` A **definite verdict** is either: @@ -40,11 +42,15 @@ A **definite verdict** is either: The **honesty gap** is `unknown` — entities whose trust the engine could not determine. Wardline records these explicitly rather than silently passing them. +`unanalyzed_total` counts source files discovered but never analyzed; each counts +as at least one uncovered surface item because the engine could not know whether +the skipped file contained trust declarations. -When `boundaries_total == 0` (no trust annotations in the scanned path), -`coverage_pct` is `null` — no trust surface to cover means coverage is null, -never a vacuous `100.0` that reads as a false-green to an agent using a numeric -gate. The human format prints "nothing to assure" to make this explicit. +When `boundaries_total == 0` and `unanalyzed_total == 0` (no trust annotations +and no skipped source in the scanned path), `coverage_pct` is `null` — no trust +surface to cover means coverage is null, never a vacuous `100.0` that reads as a +false-green to an agent using a numeric gate. The human format prints "nothing +to assure" to make this explicit. ## The structured posture object @@ -73,6 +79,7 @@ return the same object (identical by construction — both call the same ], "engine_limited": 1, "coverage_pct": 83.3, + "unanalyzed_total": 0, "unanalyzed_rule_ids": ["WLN-ENGINE-PARSE-ERROR"], "waiver_debt": [ { @@ -91,12 +98,13 @@ return the same object (identical by construction — both call the same | Field | Type | Meaning | |---|---|---| -| `boundaries_total` | int | Count of anchored (trust-declared) entities — the denominator | +| `boundaries_total` | int | Count of known anchored (trust-declared) entities | | `proven` | int | Entities with a clean verdict (no active defect) | | `defect_total` | int | Entities with an active defect (covered — a definite negative verdict) | | `unknown` | list | Entities with no definite verdict — the honesty gap | -| `engine_limited` | int | Subset of `unknown` caused by engine under-scan (parse/recursion skip → `WLN-ENGINE-*` FACT) | -| `coverage_pct` | float \| null | `100 × (boundaries_total − unknown_count) / boundaries_total`; `null` when `boundaries_total == 0` (no trust surface → not a false-green 100%) | +| `engine_limited` | int | Unknown known entities plus unanalyzed files caused by engine under-scan (parse/recursion skip → `WLN-ENGINE-*` FACT) | +| `coverage_pct` | float \| null | `100 × (boundaries_total − unknown_count) / (boundaries_total + unanalyzed_total)`; `null` when both counts are zero (no trust surface → not a false-green 100%) | +| `unanalyzed_total` | int | Files discovered but never analyzed; each counts as at least one uncovered surface item | | `unanalyzed_rule_ids` | list[str] | Distinct `WLN-ENGINE-*` rule ids seen in findings — indicates *why* engine-limited unknowns occurred | | `waiver_debt` | list | Every waiver from `.weft/wardline/waivers.yaml`, with days-to-expiry | | `baselined_total` | int | Findings suppressed via the accepted baseline | @@ -148,6 +156,7 @@ When the scanned path contains no trust-annotated functions: "unknown": [], "engine_limited": 0, "coverage_pct": null, + "unanalyzed_total": 0, "unanalyzed_rule_ids": [], "waiver_debt": [], "baselined_total": 0, @@ -203,6 +212,9 @@ if posture["unknown"]: # May be a missing decorator or a complex data-flow pattern. report_unprovable_boundaries(unprovable) +if posture["unanalyzed_total"]: + report_parse_failures([], posture["unanalyzed_rule_ids"]) + lapsed = [w for w in posture["waiver_debt"] if w["days_left"] is not None and w["days_left"] < 0] if lapsed: # Accepted-debt waivers have expired — require re-review before merge. diff --git a/src/wardline/cli/assure.py b/src/wardline/cli/assure.py index ed1f2557..b7b866a8 100644 --- a/src/wardline/cli/assure.py +++ b/src/wardline/cli/assure.py @@ -54,11 +54,13 @@ def _render_human(posture: object) -> None: assert isinstance(posture, AssurancePosture) d = posture.to_dict() - total: int = d["boundaries_total"] + boundary_total: int = d["boundaries_total"] + unanalyzed_total: int = d["unanalyzed_total"] + coverage_total = boundary_total + unanalyzed_total unknown_list: list[dict[str, Any]] = d["unknown"] waiver_debt: list[dict[str, Any]] = d["waiver_debt"] - if total == 0: + if coverage_total == 0: click.echo("No trust surface declared (0 trust-annotated boundaries) — nothing to assure.") else: pct: float = d["coverage_pct"] @@ -67,12 +69,16 @@ def _render_human(posture: object) -> None: unknown_count = len(unknown_list) engine_limited: int = d["engine_limited"] - definite = total - unknown_count - click.echo(f"Trust-surface coverage: {pct}% ({definite}/{total} boundaries reached a definite verdict)") + definite = boundary_total - unknown_count + click.echo( + f"Trust-surface coverage: {pct}% ({definite}/{coverage_total} surface item(s) reached a definite verdict)" + ) click.echo(f" proven: {proven}") click.echo(f" defect: {defect}") engine_note = f" ({engine_limited} engine-limited)" if engine_limited else "" - click.echo(f" unknown: {unknown_count}{engine_note}") + click.echo(f" unknown: {unknown_count + unanalyzed_total}{engine_note}") + if unanalyzed_total: + click.echo(f" unanalyzed files: {unanalyzed_total}") if unknown_list: click.echo(" Unknown boundaries:") diff --git a/src/wardline/core/assure.py b/src/wardline/core/assure.py index 424ca60f..7b1538d5 100644 --- a/src/wardline/core/assure.py +++ b/src/wardline/core/assure.py @@ -10,9 +10,11 @@ Coverage measures "verdict reached EITHER WAY". A defect is COVERED: the engine reached a definite (negative) verdict. The honesty gap is the ``unknown`` set — entities whose trust could not be proven (no computed return taint, an undeclared / -``UNKNOWN_*`` tier, or an engine under-scan). ``engine_limited`` is the sub-count of -those that are unknown specifically because the engine under-scanned them (a parse / -recursion skip), as distinct from a developer simply not declaring trust. +``UNKNOWN_*`` tier, or an engine under-scan). ``unanalyzed_total`` counts files the +engine discovered but never analyzed; coverage treats each as at least one uncovered +surface item because trust declarations in that file are unknown. ``engine_limited`` +is the total under-scan pressure: entity-level unknowns with a skip reason plus +``unanalyzed_total``. Per-entity verdicts are delegated WHOLESALE to :func:`wardline.core.dossier.classify_entity_trust` — the single source of truth — @@ -90,13 +92,20 @@ def to_dict(self) -> dict[str, Any]: @dataclass(frozen=True, slots=True) class AssurancePosture: """The trust-surface coverage rollup. ``boundaries_total`` is the denominator - (anchored entities only); ``proven`` + ``defect_total`` + ``len(unknown)`` == + for known anchored entities; ``proven`` + ``defect_total`` + ``len(unknown)`` == ``boundaries_total``. ``coverage_pct`` is the share with a definite verdict - (defects counted as covered); ``unknown`` is the honesty gap. + (defects counted as covered) over known boundaries plus ``unanalyzed_total``; + ``unknown`` is the known-entity honesty gap. - ``coverage_pct`` is ``None`` when ``boundaries_total == 0`` — no trust surface to - cover means coverage is null, never a vacuous 100% that reads as a false-green to - an agent using a numeric gate (the project's #1 forbidden failure mode).""" + ``unanalyzed_total`` is discovered source the engine never analysed. Because + those files may contain trust declarations the parser could not recover, + coverage treats each as at least one uncovered surface item instead of silently + shrinking the denominator. + + ``coverage_pct`` is ``None`` when there are no known boundaries and no + unanalyzed files — no trust surface to cover means coverage is null, never a + vacuous 100% that reads as a false-green to an agent using a numeric gate (the + project's #1 forbidden failure mode).""" boundaries_total: int proven: int @@ -104,6 +113,7 @@ class AssurancePosture: unknown: list[UnknownBoundary] engine_limited: int coverage_pct: float | None + unanalyzed_total: int unanalyzed_rule_ids: list[str] waiver_debt: list[WaiverDebtEntry] baselined_total: int @@ -117,6 +127,7 @@ def to_dict(self) -> dict[str, Any]: "unknown": [u.to_dict() for u in self.unknown], "engine_limited": self.engine_limited, "coverage_pct": self.coverage_pct, + "unanalyzed_total": self.unanalyzed_total, "unanalyzed_rule_ids": list(self.unanalyzed_rule_ids), "waiver_debt": [w.to_dict() for w in self.waiver_debt], "baselined_total": self.baselined_total, @@ -141,6 +152,7 @@ def _empty_posture(waivers: tuple[Waiver, ...], today: date) -> AssurancePosture unknown=[], engine_limited=0, coverage_pct=None, + unanalyzed_total=0, unanalyzed_rule_ids=[], waiver_debt=_waiver_debt(waivers, today), baselined_total=0, @@ -208,10 +220,13 @@ def posture_from_scan( unknown.sort(key=lambda u: u.qualname) - if boundaries_total == 0: + unanalyzed_total = result.summary.unanalyzed + engine_limited += unanalyzed_total + coverage_denominator = boundaries_total + unanalyzed_total + if coverage_denominator == 0: coverage_pct: float | None = None else: - coverage_pct = round(100 * (boundaries_total - len(unknown)) / boundaries_total, 1) + coverage_pct = round(100 * (boundaries_total - len(unknown)) / coverage_denominator, 1) unanalyzed_rule_ids = sorted({f.rule_id for f in result.findings if f.rule_id in UNDER_SCAN_RULE_IDS}) @@ -222,6 +237,7 @@ def posture_from_scan( unknown=unknown, engine_limited=engine_limited, coverage_pct=coverage_pct, + unanalyzed_total=unanalyzed_total, unanalyzed_rule_ids=unanalyzed_rule_ids, waiver_debt=_waiver_debt(waivers, today), baselined_total=result.summary.baselined, diff --git a/src/wardline/mcp/server.py b/src/wardline/mcp/server.py index ba89ca22..5155347b 100644 --- a/src/wardline/mcp/server.py +++ b/src/wardline/mcp/server.py @@ -2316,13 +2316,17 @@ def _assure(args: dict[str, Any], root: Path) -> dict[str, Any]: }, "engine_limited": { "type": "integer", - "description": "Sub-count of `unknown` that are unknown specifically because the engine under-scanned " - "them (reason != null).", + "description": "Under-scan pressure: entity-level unknowns with an engine reason plus unanalyzed files.", }, "coverage_pct": { "type": ["number", "null"], - "description": "Share of boundaries with a definite verdict, rounded to 1 decimal. null when " - "boundaries_total == 0 (no trust surface — never a vacuous 100%).", + "description": "Share of known boundaries with a definite verdict over known boundaries plus " + "unanalyzed files, rounded to 1 decimal. null when there are no boundaries and no unanalyzed files.", + }, + "unanalyzed_total": { + "type": "integer", + "description": "Files discovered but never analyzed. Each counts as at least one uncovered surface " + "item in coverage_pct.", }, "unanalyzed_rule_ids": { "type": "array", @@ -2361,6 +2365,7 @@ def _assure(args: dict[str, Any], root: Path) -> dict[str, Any]: "unknown", "engine_limited", "coverage_pct", + "unanalyzed_total", "unanalyzed_rule_ids", "waiver_debt", "baselined_total", @@ -2716,7 +2721,11 @@ def _attest(args: dict[str, Any], root: Path, loomweave: Any = None) -> dict[str }, }, "engine_limited": {"type": "integer"}, - "coverage_pct": {"type": ["number", "null"], "description": "null when boundaries_total == 0."}, + "coverage_pct": { + "type": ["number", "null"], + "description": "null when there are no boundaries and no unanalyzed files.", + }, + "unanalyzed_total": {"type": "integer"}, "unanalyzed_rule_ids": {"type": "array", "items": {"type": "string"}}, "waiver_debt": { "type": "array", @@ -2744,6 +2753,7 @@ def _attest(args: dict[str, Any], root: Path, loomweave: Any = None) -> dict[str "unknown", "engine_limited", "coverage_pct", + "unanalyzed_total", "unanalyzed_rule_ids", "waiver_debt", "baselined_total", diff --git a/tests/unit/cli/test_assure_cmd.py b/tests/unit/cli/test_assure_cmd.py index ce15d13e..2d9db2c3 100644 --- a/tests/unit/cli/test_assure_cmd.py +++ b/tests/unit/cli/test_assure_cmd.py @@ -40,6 +40,9 @@ ) _PLAIN = "def f():\n return 1\n" +_BAD_DECORATED = ( + "from wardline.decorators.trust import trusted\n\n@trusted(level='INTEGRAL')\ndef broken(:\n return 1\n" +) def test_json_output_equals_core(tmp_path: Path) -> None: @@ -74,6 +77,18 @@ def test_human_format_empty_surface(tmp_path: Path) -> None: assert "100% coverage" not in result.output +def test_human_format_unanalyzed_files_are_not_full_coverage(tmp_path: Path) -> None: + (tmp_path / "good.py").write_text(_MODULE, encoding="utf-8") + (tmp_path / "bad.py").write_text(_BAD_DECORATED, encoding="utf-8") + + result = CliRunner().invoke(cli, ["assure", str(tmp_path), "--format", "human"]) + + assert result.exit_code == 0, result.output + assert "100.0%" not in result.output + assert "75.0%" in result.output + assert "unanalyzed files: 1" in result.output + + def test_human_lapsed_waiver_wording(tmp_path: Path) -> None: """A lapsed waiver must say "expired N day(s) ago", not "-N day(s) until earliest expiry".""" # Write a decorated module so the posture is non-empty, then invoke via JSON diff --git a/tests/unit/core/test_assure.py b/tests/unit/core/test_assure.py index 16762e3b..1c124fd4 100644 --- a/tests/unit/core/test_assure.py +++ b/tests/unit/core/test_assure.py @@ -51,6 +51,10 @@ " return src()\n" ) +_BAD_DECORATED_MODULE = ( + "from wardline.decorators.trust import trusted\n\n@trusted(level='INTEGRAL')\ndef broken(:\n return 1\n" +) + def test_coverage_denominator_end_to_end(tmp_path: Path) -> None: (tmp_path / "m.py").write_text(_MODULE, encoding="utf-8") @@ -76,6 +80,7 @@ def test_coverage_denominator_end_to_end(tmp_path: Path) -> None: assert got["engine_limited"] == 0 # 3 of 3 reached a definite verdict — the defect counts as COVERED. assert got["coverage_pct"] == 100.0 + assert got["unanalyzed_total"] == 0 assert got["unanalyzed_rule_ids"] == [] # 2026-07-01 − 2026-06-03 = 28 days. assert got["waiver_debt"] == [ @@ -153,6 +158,7 @@ def test_unknown_and_engine_limited_branch() -> None: assert got["proven"] == 1 assert len(got["unknown"]) == 2 assert got["engine_limited"] == 1 + assert got["unanalyzed_total"] == 0 assert got["coverage_pct"] == round(100 * (3 - 2) / 3, 1) == 33.3 # `unknown` is sorted by qualname: m.b before m.c. assert [u["qualname"] for u in got["unknown"]] == ["m.b", "m.c"] @@ -180,6 +186,7 @@ def test_empty_surface_coverage_is_null(tmp_path: Path) -> None: got = posture.to_dict() assert got["boundaries_total"] == 0 + assert got["unanalyzed_total"] == 0 assert got["coverage_pct"] is None, ( f"expected None but got {got['coverage_pct']!r}; " "a vacuous 100.0 reads as 'fully assured' to a numeric gate — false-green" @@ -216,3 +223,20 @@ def test_empty_surface_coverage_is_null(tmp_path: Path) -> None: pure_posture = posture_from_scan(empty_result, empty_ctx, waivers=(), today=date(2026, 6, 3)) assert pure_posture.coverage_pct is None assert pure_posture.to_dict()["coverage_pct"] is None + + +def test_unanalyzed_files_count_against_assurance_coverage(tmp_path: Path) -> None: + (tmp_path / "good.py").write_text(_MODULE, encoding="utf-8") + (tmp_path / "bad.py").write_text(_BAD_DECORATED_MODULE, encoding="utf-8") + + posture = build_posture(tmp_path, today=date(2026, 6, 3)) + got = posture.to_dict() + + assert got["boundaries_total"] == 3 + assert got["proven"] == 2 + assert got["defect_total"] == 1 + assert got["unknown"] == [] + assert got["unanalyzed_total"] == 1 + assert got["engine_limited"] == 1 + assert got["coverage_pct"] == round(100 * 3 / 4, 1) == 75.0 + assert got["unanalyzed_rule_ids"] == ["WLN-ENGINE-PARSE-ERROR"] From db14c3eae0bdfe72165a21a8b97c5d6de3c79fef Mon Sep 17 00:00:00 2001 From: John Morrissey Date: Fri, 12 Jun 2026 06:26:13 +1000 Subject: [PATCH 115/186] fix(filigree): disable mark_unseen on unanalyzed scans --- docs/reference/finding-lifecycle-vocabulary.md | 7 +++++-- src/wardline/core/filigree_emit.py | 10 ++++++++-- tests/unit/core/test_filigree_emit.py | 16 ++++++++++++++++ 3 files changed, 29 insertions(+), 4 deletions(-) diff --git a/docs/reference/finding-lifecycle-vocabulary.md b/docs/reference/finding-lifecycle-vocabulary.md index 72430d5c..607dbece 100644 --- a/docs/reference/finding-lifecycle-vocabulary.md +++ b/docs/reference/finding-lifecycle-vocabulary.md @@ -207,8 +207,11 @@ owned by sibling tools and are recorded here as coordination context: - **Filigree's "new" / `seen_count` lifecycle is Filigree-owned.** Filigree decides first-seen vs returning purely from fingerprint presence across scans - (`mark_unseen`, `src/wardline/core/filigree_emit.py:68-76`). Wardline emits the - fingerprint and `scanned_paths`; it does not rename Filigree's first-seen concept. + (`mark_unseen`, `src/wardline/core/filigree_emit.py`). Wardline emits the + fingerprint and `scanned_paths`; it does not rename Filigree's first-seen + concept. If a scan contains under-analysis findings (`WLN-ENGINE-*` unanalyzed + rule ids), Wardline disables `mark_unseen` for that batch so an absent fingerprint + cannot be treated as fixed when the source was not actually analyzed. - **legis receives the gate population, keyed by `suppression_state`.** The legis scan artifact projects the *whole scan*, mapping `baselined` / `judged` onto diff --git a/src/wardline/core/filigree_emit.py b/src/wardline/core/filigree_emit.py index f4d16e11..d89136ca 100644 --- a/src/wardline/core/filigree_emit.py +++ b/src/wardline/core/filigree_emit.py @@ -22,6 +22,7 @@ from wardline.core.errors import FiligreeEmitError from wardline.core.finding import ( FINGERPRINT_SCHEME, + UNANALYZED_RULE_IDS, Finding, format_fingerprint, severity_to_filigree, @@ -74,13 +75,18 @@ def build_scan_results_body( ``mark_unseen`` opts into Filigree's per-(file, scan_source) absent-fingerprint sweep: a fingerprint seen before but absent now in a scanned file enters ``unseen_in_latest``. Clean files are represented by ``scanned_paths`` so - close-on-fixed can reconcile a file whose last finding disappeared.""" + close-on-fixed can reconcile a file whose last finding disappeared. + + If any file was discovered but not analyzed, do not run the absent-fingerprint + sweep: a parse/file failure means missing findings are not proof of a fix. + """ findings_wire = [_finding_to_wire(f) for f in findings] scanned = list(dict.fromkeys(p for p in scanned_paths if p)) + has_unanalyzed = any(f.rule_id in UNANALYZED_RULE_IDS for f in findings) body = { "scan_source": scan_source, "fingerprint_scheme": FINGERPRINT_SCHEME, - "mark_unseen": bool(findings_wire or scanned), + "mark_unseen": bool(findings_wire or scanned) and not has_unanalyzed, "findings": findings_wire, } if scanned: diff --git a/tests/unit/core/test_filigree_emit.py b/tests/unit/core/test_filigree_emit.py index 1e56d093..804ea689 100644 --- a/tests/unit/core/test_filigree_emit.py +++ b/tests/unit/core/test_filigree_emit.py @@ -67,6 +67,22 @@ def test_scan_results_body_can_reconcile_clean_scanned_files() -> None: assert body["findings"] == [] +def test_scan_results_body_disables_mark_unseen_when_scan_is_unanalyzed() -> None: + parse_error = _f( + rule_id="WLN-ENGINE-PARSE-ERROR", + severity=Severity.ERROR, + kind=Kind.DEFECT, + location=Location(path="src/m.py", line_start=1), + fingerprint="b" * 64, + ) + + body = build_scan_results_body([parse_error], scanned_paths=("src/m.py",)) + + assert body["mark_unseen"] is False + assert body["scanned_paths"] == ["src/m.py"] + assert body["findings"][0]["rule_id"] == "WLN-ENGINE-PARSE-ERROR" + + def test_finding_uses_path_not_file_path() -> None: wire = build_scan_results_body([_f()])["findings"][0] assert wire["path"] == "src/m.py" From cfc7489e6568424fe02ad94ee04e0c06a98e93e1 Mon Sep 17 00:00:00 2001 From: John Morrissey Date: Fri, 12 Jun 2026 06:29:39 +1000 Subject: [PATCH 116/186] fix(taint): include seed dependencies in grammar fingerprint --- .../scanner/taint/decorator_provider.py | 75 ++++++++++++++++--- .../scanner/taint/test_decorator_provider.py | 34 +++++++++ 2 files changed, 100 insertions(+), 9 deletions(-) diff --git a/src/wardline/scanner/taint/decorator_provider.py b/src/wardline/scanner/taint/decorator_provider.py index c56380ad..192c8caa 100644 --- a/src/wardline/scanner/taint/decorator_provider.py +++ b/src/wardline/scanner/taint/decorator_provider.py @@ -174,20 +174,77 @@ def _read_level( return default +def _seed_value_identity(value: object) -> str: + if value is None or isinstance(value, (str, int, float, bool)): + return repr(value) + if isinstance(value, TaintState): + return f"TaintState:{value.value}" + if isinstance(value, FunctionTaint): + return ( + "FunctionTaint(" + f"body={_seed_value_identity(value.body_taint)}," + f"return={_seed_value_identity(value.return_taint)}" + ")" + ) + if isinstance(value, (tuple, list)): + return type(value).__name__ + "(" + ",".join(_seed_value_identity(v) for v in value) + ")" + if isinstance(value, dict): + parts = sorted((_seed_value_identity(k), _seed_value_identity(v)) for k, v in value.items()) + return "dict(" + ",".join(f"{k}:{v}" for k, v in parts) + ")" + + module = getattr(value, "__module__", None) + qualname = getattr(value, "__qualname__", None) + if isinstance(module, str) and isinstance(qualname, str): + return f"{module}.{qualname}" + name = getattr(value, "__name__", None) + if isinstance(module, str) and isinstance(name, str): + return f"{module}.{name}" + return repr(value) + + +def _closure_identity(seed: object) -> tuple[str, ...]: + items: list[str] = [] + for cell in getattr(seed, "__closure__", None) or (): + try: + items.append(_seed_value_identity(cell.cell_contents)) + except ValueError: + items.append("") + return tuple(items) + + def _seed_identity(seed: object) -> str: """A stable identity string for a boundary type's seed callable. - For a Python function/lambda, keys on the bytecode + constants - (``__code__.co_code`` + ``co_consts``) — so two DISTINCT lambda bodies that share - ``__qualname__ == ""`` get DISTINCT identities (closing the cache - cross-contamination false-green: two grammars differing only in a lambda seed - body must not share cached summaries). For a non-function callable (no - ``__code__``), falls back to ``__qualname__`` / ``repr``. This only ever - OVER-invalidates the summary cache (a changed seed body → a different identity → - a cold re-scan), never wrongly reuses — strictly safe.""" + For a Python function/lambda, keys on bytecode, constants, referenced names, + defaults, closures, and the stable identities of referenced globals. Bytecode + alone is not enough: ``return SAFE_SEED`` and ``return RAW_SEED`` can share + ``co_code``/``co_consts`` while differing only by ``co_names`` or the value bound + to that name. For a non-function callable (no ``__code__``), falls back to + ``__qualname__`` / ``repr``. This only ever OVER-invalidates the summary cache (a + changed seed body/dependency → a different identity → a cold re-scan), never + wrongly reuses — strictly safe.""" code = getattr(seed, "__code__", None) if code is not None: - return f"{code.co_code.hex()}|{code.co_consts!r}" + globals_map = getattr(seed, "__globals__", {}) + global_parts = [] + if isinstance(globals_map, dict): + for name in code.co_names: + global_parts.append(f"{name}={_seed_value_identity(globals_map.get(name, ''))}") + return "|".join( + ( + str(getattr(seed, "__module__", "")), + str(getattr(seed, "__qualname__", getattr(seed, "__name__", ""))), + code.co_code.hex(), + repr(code.co_consts), + repr(code.co_names), + repr(code.co_freevars), + repr(code.co_cellvars), + repr(getattr(seed, "__defaults__", None)), + _seed_value_identity(getattr(seed, "__kwdefaults__", None)), + repr(_closure_identity(seed)), + repr(tuple(global_parts)), + ) + ) return str(getattr(seed, "__qualname__", repr(seed))) diff --git a/tests/unit/scanner/taint/test_decorator_provider.py b/tests/unit/scanner/taint/test_decorator_provider.py index dcb85a90..9dbc0a44 100644 --- a/tests/unit/scanner/taint/test_decorator_provider.py +++ b/tests/unit/scanner/taint/test_decorator_provider.py @@ -2,10 +2,12 @@ from __future__ import annotations import ast +import types from wardline.core.registry import REGISTRY_VERSION from wardline.core.taints import TaintState as T from wardline.scanner.ast_primitives import build_import_alias_map +from wardline.scanner.grammar import BoundaryType from wardline.scanner.index import discover_file_entities from wardline.scanner.taint.decorator_provider import DecoratorTaintSourceProvider from wardline.scanner.taint.provider import FunctionTaint, SeedContext @@ -28,6 +30,38 @@ def _seed( return {e.qualname: provider.taint_for(e, ctx).taint for e in entities} +def _custom_provider_fingerprint(seed: object) -> str: + boundary = BoundaryType("custom_boundary", "custom_pack", 1, (), seed) + return DecoratorTaintSourceProvider(boundary_types=(boundary,)).fingerprint() + + +def test_custom_seed_fingerprint_includes_referenced_global_names() -> None: + seed_a = eval("lambda levels: safe_seed") # noqa: S307 - local test fixture, no user input + seed_b = eval("lambda levels: raw_seed") # noqa: S307 - local test fixture, no user input + + assert seed_a.__code__.co_code == seed_b.__code__.co_code + assert seed_a.__code__.co_consts == seed_b.__code__.co_consts + assert _custom_provider_fingerprint(seed_a) != _custom_provider_fingerprint(seed_b) + + +def test_custom_seed_fingerprint_includes_referenced_global_values() -> None: + safe_seed = FunctionTaint(T.INTEGRAL, T.INTEGRAL) + raw_seed = FunctionTaint(T.EXTERNAL_RAW, T.EXTERNAL_RAW) + + def template(levels): # noqa: ANN001, ANN202 + return SEED # noqa: F821 + + seed_a = types.FunctionType(template.__code__, {"SEED": safe_seed}, name="seed") + seed_b = types.FunctionType(template.__code__, {"SEED": raw_seed}, name="seed") + seed_a.__module__ = seed_b.__module__ = "custom_pack" + seed_a.__qualname__ = seed_b.__qualname__ = "seed" + + assert seed_a.__code__.co_code == seed_b.__code__.co_code + assert seed_a.__code__.co_consts == seed_b.__code__.co_consts + assert seed_a.__code__.co_names == seed_b.__code__.co_names + assert _custom_provider_fingerprint(seed_a) != _custom_provider_fingerprint(seed_b) + + def test_external_boundary_from_import() -> None: out = _seed("from wardline.decorators import external_boundary\n@external_boundary\ndef read(p):\n return p\n") assert out["m.read"] == FunctionTaint(T.EXTERNAL_RAW, T.EXTERNAL_RAW) From 13936d6d65cdb13690a653dcc2c4b8cdda6923cf Mon Sep 17 00:00:00 2001 From: John Morrissey Date: Fri, 12 Jun 2026 06:31:34 +1000 Subject: [PATCH 117/186] fix(ci): pin setup-uv action --- .github/workflows/ci.yml | 30 ++++++++++++++++++++++-------- 1 file changed, 22 insertions(+), 8 deletions(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 1448af89..421d4ddf 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -23,7 +23,9 @@ jobs: if: github.event_name != 'schedule' steps: - uses: actions/checkout@v4 - - uses: astral-sh/setup-uv@v7 + with: + persist-credentials: false + - uses: astral-sh/setup-uv@37802adc94f370d6bfd71619e3f0bf239e1f3b78 # v7 with: enable-cache: true python-version: "3.13" @@ -42,7 +44,9 @@ jobs: if: github.event_name != 'schedule' steps: - uses: actions/checkout@v4 - - uses: astral-sh/setup-uv@v7 + with: + persist-credentials: false + - uses: astral-sh/setup-uv@37802adc94f370d6bfd71619e3f0bf239e1f3b78 # v7 with: enable-cache: true python-version: "3.13" @@ -58,7 +62,9 @@ jobs: python-version: ["3.12", "3.13"] steps: - uses: actions/checkout@v4 - - uses: astral-sh/setup-uv@v7 + with: + persist-credentials: false + - uses: astral-sh/setup-uv@37802adc94f370d6bfd71619e3f0bf239e1f3b78 # v7 with: enable-cache: true python-version: ${{ matrix.python-version }} @@ -72,7 +78,9 @@ jobs: if: github.event_name != 'schedule' steps: - uses: actions/checkout@v4 - - uses: astral-sh/setup-uv@v7 + with: + persist-credentials: false + - uses: astral-sh/setup-uv@37802adc94f370d6bfd71619e3f0bf239e1f3b78 # v7 with: enable-cache: true python-version: "3.13" @@ -95,7 +103,9 @@ jobs: if: github.event_name == 'schedule' steps: - uses: actions/checkout@v4 - - uses: astral-sh/setup-uv@v7 + with: + persist-credentials: false + - uses: astral-sh/setup-uv@37802adc94f370d6bfd71619e3f0bf239e1f3b78 # v7 with: enable-cache: true python-version: "3.13" @@ -134,7 +144,9 @@ jobs: marker: filigree_e2e steps: - uses: actions/checkout@v4 - - uses: astral-sh/setup-uv@v7 + with: + persist-credentials: false + - uses: astral-sh/setup-uv@37802adc94f370d6bfd71619e3f0bf239e1f3b78 # v7 with: enable-cache: true python-version: "3.13" @@ -167,7 +179,7 @@ jobs: - uses: actions/checkout@v4 with: persist-credentials: false - - uses: astral-sh/setup-uv@v7 + - uses: astral-sh/setup-uv@37802adc94f370d6bfd71619e3f0bf239e1f3b78 # v7 with: enable-cache: true python-version: "3.13" @@ -184,7 +196,9 @@ jobs: contents: write steps: - uses: actions/checkout@v4 - - uses: astral-sh/setup-uv@v7 + with: + persist-credentials: false + - uses: astral-sh/setup-uv@37802adc94f370d6bfd71619e3f0bf239e1f3b78 # v7 with: enable-cache: true python-version: "3.13" From 284479752a8c986a59a6be2ca925fe3a48f5cbc3 Mon Sep 17 00:00:00 2001 From: John Morrissey Date: Fri, 12 Jun 2026 06:35:04 +1000 Subject: [PATCH 118/186] fix(make): refuse symlinked clean targets --- Makefile | 13 +++++- tests/unit/test_makefile_clean.py | 78 +++++++++++++++++++++++++++++++ 2 files changed, 90 insertions(+), 1 deletion(-) create mode 100644 tests/unit/test_makefile_clean.py diff --git a/Makefile b/Makefile index 909e9798..6968ab16 100644 --- a/Makefile +++ b/Makefile @@ -34,7 +34,18 @@ build: ## Build sdist + wheel uv build clean: ## Remove build + cache artifacts - rm -rf dist/ build/ *.egg-info .mypy_cache .ruff_cache .pytest_cache .coverage coverage.json + @set -eu; \ + for path in dist build *.egg-info .mypy_cache .ruff_cache .pytest_cache; do \ + if [ ! -e "$$path" ] && [ ! -L "$$path" ]; then \ + continue; \ + fi; \ + if [ -L "$$path" ]; then \ + echo "refusing to remove symlink $$path" >&2; \ + exit 1; \ + fi; \ + rm -rf -- "$$path"; \ + done; \ + rm -f -- .coverage coverage.json find . -type d -name __pycache__ -exec rm -rf {} + 2>/dev/null || true ci: lint typecheck test-cov ## Run the full local CI gate diff --git a/tests/unit/test_makefile_clean.py b/tests/unit/test_makefile_clean.py new file mode 100644 index 00000000..c6837ed4 --- /dev/null +++ b/tests/unit/test_makefile_clean.py @@ -0,0 +1,78 @@ +from __future__ import annotations + +import shutil +import subprocess +from pathlib import Path + +import pytest + +PROJECT_ROOT = Path(__file__).resolve().parents[2] +MAKE = shutil.which("make") + + +pytestmark = pytest.mark.skipif(MAKE is None, reason="make is not installed") + + +def run_make_clean(workdir: Path) -> subprocess.CompletedProcess[str]: + return subprocess.run( + [MAKE, "-f", str(PROJECT_ROOT / "Makefile"), "clean"], + cwd=workdir, + check=False, + text=True, + capture_output=True, + ) + + +def test_make_clean_refuses_symlinked_recursive_targets(tmp_path: Path) -> None: + outside = tmp_path / "outside" + outside.mkdir() + keep = outside / "keep.txt" + keep.write_text("keep", encoding="utf-8") + + workdir = tmp_path / "workdir" + workdir.mkdir() + (workdir / "dist").symlink_to(outside, target_is_directory=True) + + result = run_make_clean(workdir) + + assert result.returncode != 0 + assert "refusing to remove symlink dist" in result.stderr + assert keep.read_text(encoding="utf-8") == "keep" + assert (workdir / "dist").is_symlink() + + +def test_make_clean_removes_expected_non_symlink_artifacts(tmp_path: Path) -> None: + workdir = tmp_path / "workdir" + workdir.mkdir() + for dirname in ( + "dist", + "build", + "example.egg-info", + ".mypy_cache", + ".ruff_cache", + ".pytest_cache", + ): + artifact_dir = workdir / dirname + artifact_dir.mkdir() + (artifact_dir / "artifact.txt").write_text("artifact", encoding="utf-8") + for filename in (".coverage", "coverage.json"): + (workdir / filename).write_text("artifact", encoding="utf-8") + pycache = workdir / "pkg" / "__pycache__" + pycache.mkdir(parents=True) + (pycache / "module.pyc").write_text("artifact", encoding="utf-8") + + result = run_make_clean(workdir) + + assert result.returncode == 0, result.stderr + for name in ( + "dist", + "build", + "example.egg-info", + ".mypy_cache", + ".ruff_cache", + ".pytest_cache", + ".coverage", + "coverage.json", + ): + assert not (workdir / name).exists() + assert not pycache.exists() From 855f171d12e930f03664b97278b9c9834a24e274 Mon Sep 17 00:00:00 2001 From: John Morrissey Date: Fri, 12 Jun 2026 06:36:27 +1000 Subject: [PATCH 119/186] docs: mark install unsafe for untrusted ci --- docs/guides/agents.md | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-) diff --git a/docs/guides/agents.md b/docs/guides/agents.md index 706a213c..5df20f0c 100644 --- a/docs/guides/agents.md +++ b/docs/guides/agents.md @@ -54,10 +54,14 @@ wardline install: runtime markers: install `weft-markers` and import from `weft_markers` ``` -It is idempotent (re-run to refresh after upgrading wardline) and non-interactive -(safe in CI). Opt out of any piece with `--no-claude-md`, `--no-agents-md`, -`--no-skill`, `--no-mcp`, or `--no-bindings`. There is no SessionStart hook — -freshness is enforced only when you re-run `wardline install`. +It is idempotent (re-run to refresh after upgrading wardline) and +non-interactive, but it writes project-local agent and MCP files. Run it only +on a trusted checkout or as an operator-controlled bootstrap step. For +untrusted pull-request CI, use `wardline scan ... --fail-on ERROR`; do not run +`wardline install` against attacker-controlled working-tree contents. Opt out +of any piece with `--no-claude-md`, `--no-agents-md`, `--no-skill`, `--no-mcp`, +or `--no-bindings`. There is no SessionStart hook — freshness is enforced only +when you re-run `wardline install`. Once installed, the MCP server resolves a Loomweave/Filigree URL at runtime from the flag, env var, or published `.weft//ephemeral.port` rung — not from From 13032e263335343595e76c91ecf1ecd7cce16e70 Mon Sep 17 00:00:00 2001 From: John Morrissey Date: Fri, 12 Jun 2026 06:40:11 +1000 Subject: [PATCH 120/186] fix(install): confine skill directory writes --- src/wardline/core/safe_paths.py | 7 ++++++- src/wardline/install/skill.py | 11 +++++++++- tests/unit/cli/test_install.py | 22 ++++++++++++++++++++ tests/unit/install/test_skill.py | 35 ++++++++++++++++++++++++++++++++ 4 files changed, 73 insertions(+), 2 deletions(-) diff --git a/src/wardline/core/safe_paths.py b/src/wardline/core/safe_paths.py index e8e50573..6d084303 100644 --- a/src/wardline/core/safe_paths.py +++ b/src/wardline/core/safe_paths.py @@ -8,7 +8,7 @@ from wardline.core.errors import WardlineError -def safe_project_file(root: Path, target: Path, *, label: str | None = None) -> Path: +def safe_project_path(root: Path, target: Path, *, label: str | None = None) -> Path: """Return ``target`` only if writes to it stay under ``root``. Fixed install/write targets must not follow a final symlink out of a project. @@ -29,6 +29,11 @@ def safe_project_file(root: Path, target: Path, *, label: str | None = None) -> return candidate +def safe_project_file(root: Path, target: Path, *, label: str | None = None) -> Path: + """Return ``target`` only if file writes to it stay under ``root``.""" + return safe_project_path(root, target, label=label) + + def safe_write_text(root: Path, target: Path, content: str, *, label: str | None = None) -> None: """Safely write ``content`` to ``target`` under ``root``, resolving symlinks and boundary checks.""" safe_path = safe_project_file(root, target, label=label) diff --git a/src/wardline/install/skill.py b/src/wardline/install/skill.py index e7c1d74f..37b72e4a 100644 --- a/src/wardline/install/skill.py +++ b/src/wardline/install/skill.py @@ -5,6 +5,9 @@ import shutil from pathlib import Path +from wardline.core.errors import WardlineError +from wardline.core.safe_paths import safe_project_path + def _skill_source() -> Path: # src/wardline/install/skill.py -> src/wardline/skills/wardline-gate @@ -19,11 +22,17 @@ def install_skill(root: Path) -> dict[str, str]: src = _skill_source() results: dict[str, str] = {} for base in (".claude", ".agents"): - dest = root / base / "skills" / "wardline-gate" + label = f"{base}/skills/wardline-gate" + dest = safe_project_path(root, root / base / "skills" / "wardline-gate", label=label) existed = dest.exists() or dest.is_symlink() if existed: + if dest.is_symlink(): + raise WardlineError(f"{label}: refusing to overwrite a symlink") + if not dest.is_dir(): + raise WardlineError(f"{label}: expected a directory") shutil.rmtree(dest) dest.parent.mkdir(parents=True, exist_ok=True) + dest = safe_project_path(root, dest, label=label) shutil.copytree(src, dest) results[base] = "overwritten" if existed else "created" return results diff --git a/tests/unit/cli/test_install.py b/tests/unit/cli/test_install.py index 4c76a999..d39947c6 100644 --- a/tests/unit/cli/test_install.py +++ b/tests/unit/cli/test_install.py @@ -130,6 +130,28 @@ def test_install_summary_includes_binding_lines(tmp_path: Path, monkeypatch) -> assert "filigree:" in result.output +def test_install_refuses_symlinked_skill_parent_escape(tmp_path: Path, monkeypatch) -> None: + root = tmp_path / "repo" + root.mkdir() + outside = tmp_path / "outside" + outside_skill = outside / "skills" / "wardline-gate" + outside_skill.mkdir(parents=True) + sentinel = outside_skill / "keep.txt" + sentinel.write_text("keep", encoding="utf-8") + (root / ".claude").symlink_to(outside, target_is_directory=True) + + monkeypatch.setattr("wardline.install.detect.shutil.which", lambda _: None) + monkeypatch.setattr("wardline.install.mcp_json.Path.home", lambda: tmp_path / "home") + monkeypatch.setattr("wardline.install.mcp_json._find_wardline_command", lambda: "/bin/wardline") + + result = CliRunner().invoke(cli, ["install", "--root", str(root)]) + + assert result.exit_code == 2 + assert "escapes project root" in result.output + assert sentinel.read_text(encoding="utf-8") == "keep" + assert not (outside_skill / "SKILL.md").exists() + + def test_install_detects_filigree_from_ephemeral_port(tmp_path: Path, monkeypatch) -> None: # The "wire config" feature was removed: install DETECTS the sibling from its # published port and REPORTS it, writing no config file. diff --git a/tests/unit/install/test_skill.py b/tests/unit/install/test_skill.py index 77d7b269..569f5eb5 100644 --- a/tests/unit/install/test_skill.py +++ b/tests/unit/install/test_skill.py @@ -1,5 +1,8 @@ from pathlib import Path +import pytest + +from wardline.core.errors import WardlineError from wardline.install.skill import install_skill @@ -20,3 +23,35 @@ def test_reinstall_overwrites(tmp_path: Path) -> None: assert results[".claude"] == "overwritten" assert results[".agents"] == "overwritten" assert "name: wardline-gate" in stale.read_text(encoding="utf-8") + + +def test_install_skill_rejects_symlinked_parent_escape(tmp_path: Path) -> None: + root = tmp_path / "repo" + root.mkdir() + outside = tmp_path / "outside" + outside_skill = outside / "skills" / "wardline-gate" + outside_skill.mkdir(parents=True) + sentinel = outside_skill / "keep.txt" + sentinel.write_text("keep", encoding="utf-8") + (root / ".claude").symlink_to(outside, target_is_directory=True) + + with pytest.raises(WardlineError, match="escapes project root"): + install_skill(root) + + assert sentinel.read_text(encoding="utf-8") == "keep" + assert not (outside_skill / "SKILL.md").exists() + + +def test_install_skill_rejects_symlinked_skill_target(tmp_path: Path) -> None: + outside = tmp_path / "outside-skill" + outside.mkdir() + sentinel = outside / "keep.txt" + sentinel.write_text("keep", encoding="utf-8") + skills = tmp_path / ".claude" / "skills" + skills.mkdir(parents=True) + (skills / "wardline-gate").symlink_to(outside, target_is_directory=True) + + with pytest.raises(WardlineError, match="symlink"): + install_skill(tmp_path) + + assert sentinel.read_text(encoding="utf-8") == "keep" From 68fa667bdca7f4ebd1dd3236e3d7b35261f2f536 Mon Sep 17 00:00:00 2001 From: John Morrissey Date: Fri, 12 Jun 2026 06:45:18 +1000 Subject: [PATCH 121/186] fix(explain): verify taint fact freshness locally --- src/wardline/core/explain.py | 73 +++++++++++++++++---- tests/unit/core/test_explain_loomweave.py | 79 +++++++++++++++++++++++ 2 files changed, 138 insertions(+), 14 deletions(-) diff --git a/src/wardline/core/explain.py b/src/wardline/core/explain.py index df751453..2cbe038f 100644 --- a/src/wardline/core/explain.py +++ b/src/wardline/core/explain.py @@ -125,20 +125,42 @@ def _explain_local( return explanation_from_context(finding, result.context) -def _is_fresh(view: Any) -> bool: - """Fresh iff: exists, a live current_content_hash is present, the blob is a - structurally sound dict (with a dict ``taint``), and the in-blob - content_hash_at_compute equals that live hash. Wardline decides freshness by - comparing the stamp IT wrote against the hash Loomweave read live; Loomweave never - asserts a verdict. Missing hash (file deleted/unreadable), exists:false, or a - malformed/skewed blob (wrong types after the HTTP round-trip) ⇒ stale, so the - caller falls through to the SP8 re-run rather than serving a broken fact.""" - if not view.exists or view.current_content_hash is None: +def _local_content_hash(root: Path, path: str) -> str | None: + """Return the local whole-file blake3 for an in-project relative path.""" + try: + from wardline.loomweave import require_blake3 + + blake3 = require_blake3() + root_resolved = root.resolve() + candidate = Path(path) + local = candidate if candidate.is_absolute() else root / candidate + resolved = local.resolve(strict=True) + if not resolved.is_file() or not resolved.is_relative_to(root_resolved): + return None + return str(blake3.blake3(resolved.read_bytes()).hexdigest()) + except (OSError, ValueError, LoomweaveError): + return None + + +def _is_fresh(view: Any, *, root: Path | None = None, path: str | None = None) -> bool: + """Fresh iff the stored Wardline stamp matches current file bytes. + + When a local ``root`` and selected finding ``path`` are available, freshness is + checked against the local file hash. The remote ``current_content_hash`` is kept + only for legacy chain reads that lack a path; it is not authoritative for + ``explain_finding`` blob serving. + """ + if not view.exists: return False blob = view.wardline_json if not isinstance(blob, dict) or not isinstance(blob.get("taint"), dict): return False stamped = blob.get("content_hash_at_compute") + if root is not None and path is not None: + local_hash = _local_content_hash(root, path) + return isinstance(stamped, str) and local_hash is not None and stamped == local_hash + if view.current_content_hash is None: + return False return stamped is not None and stamped == view.current_content_hash @@ -158,6 +180,12 @@ def _callee_leaf(callee_qualname: str | None) -> str | None: return None if callee_qualname is None else callee_qualname.rsplit(".", 1)[-1] +def _opt_count(value: Any) -> int | None: + if isinstance(value, int) and not isinstance(value, bool) and value >= 0: + return value + return None + + def _blob_finding_matches( finding: dict[str, Any], *, @@ -193,12 +221,14 @@ def _select_blob_finding( ), None, ) - return findings[0] if findings else {} + return findings[0] if findings else None def _explanation_from_blob( view: Any, *, + root: Path | None = None, + sink_qualname: str | None = None, fingerprint: str | None = None, path: str | None = None, line: int | None = None, @@ -219,11 +249,19 @@ def _explanation_from_blob( if first is None: return None callee_q = _opt_str(taint.get("contributing_callee_qualname")) + resolved_call_count = _opt_count(taint.get("resolved_call_count", 0)) + unresolved_call_count = _opt_count(taint.get("unresolved_call_count", 0)) + if resolved_call_count is None or unresolved_call_count is None: + return None stored_fingerprint = first.get("fingerprint") stored_rule_id = first.get("rule_id") stored_path = first.get("path") stored_line = first.get("line_start") qualname = blob.get("qualname") + if sink_qualname is not None and qualname != sink_qualname: + return None + if root is not None and (not isinstance(stored_path, str) or not _is_fresh(view, root=root, path=stored_path)): + return None return TaintExplanation( fingerprint=stored_fingerprint if isinstance(stored_fingerprint, str) else "", rule_id=stored_rule_id if isinstance(stored_rule_id, str) else "", @@ -234,8 +272,8 @@ def _explanation_from_blob( tier_out=_opt_str(taint.get("declared_return")), immediate_tainted_callee=_callee_leaf(callee_q), source_boundary_qualname=callee_q, - resolved_call_count=int(taint.get("resolved_call_count", 0) or 0), - unresolved_call_count=int(taint.get("unresolved_call_count", 0) or 0), + resolved_call_count=resolved_call_count, + unresolved_call_count=unresolved_call_count, ) @@ -332,8 +370,15 @@ def explain_finding( # load-bearing for explain: degrade to the SP8 re-run, never raise. (The # store read is optional enrichment; only the WRITE path surfaces a 4xx.) views = None - if views and views[0].qualname == sink_qualname and _is_fresh(views[0]): - served = _explanation_from_blob(views[0], fingerprint=fingerprint, path=path, line=line) + if views and views[0].qualname == sink_qualname: + served = _explanation_from_blob( + views[0], + root=root, + sink_qualname=sink_qualname, + fingerprint=fingerprint, + path=path, + line=line, + ) if served is not None: return served # miss/stale/outage → fall through to the re-run diff --git a/tests/unit/core/test_explain_loomweave.py b/tests/unit/core/test_explain_loomweave.py index d824fdd2..6acdb85f 100644 --- a/tests/unit/core/test_explain_loomweave.py +++ b/tests/unit/core/test_explain_loomweave.py @@ -134,6 +134,85 @@ def counting(*a, **k): assert calls["n"] == 1 +def test_spoofed_remote_hash_falls_back_to_reanalysis(tmp_path, monkeypatch): + proj = _proj(tmp_path) + local_finding = next( + f + for f in run_scan(proj).findings + if f.kind is Kind.DEFECT and f.suppressed is SuppressionState.ACTIVE and f.rule_id == "PY-WL-101" + ) + blob, _h = _fresh_blob(proj, "svc.leaky") + spoofed = "f" * 64 + blob["content_hash_at_compute"] = spoofed + blob["findings"] = [{"rule_id": "PY-WL-101", "fingerprint": "fp-forged", "path": "svc.py", "line_start": 6}] + view = TaintFactView(qualname="svc.leaky", exists=True, wardline_json=blob, current_content_hash=spoofed) + + import wardline.core.explain as explain_mod + + calls = {"n": 0} + real = explain_mod.run_scan + + def counting(*a, **k): + calls["n"] += 1 + return real(*a, **k) + + monkeypatch.setattr(explain_mod, "run_scan", counting) + + exp = explain_finding(proj, path="svc.py", line=6, loomweave=SpyClient([view]), sink_qualname="svc.leaky") + + assert exp is not None + assert exp.fingerprint == local_finding.fingerprint + assert exp.fingerprint != "fp-forged" + assert calls["n"] == 1 + + +def test_blob_qualname_mismatch_falls_back_to_reanalysis(tmp_path, monkeypatch): + proj = _proj(tmp_path) + blob, h = _fresh_blob(proj, "svc.other") + view = TaintFactView(qualname="svc.leaky", exists=True, wardline_json=blob, current_content_hash=h) + + import wardline.core.explain as explain_mod + + calls = {"n": 0} + real = explain_mod.run_scan + + def counting(*a, **k): + calls["n"] += 1 + return real(*a, **k) + + monkeypatch.setattr(explain_mod, "run_scan", counting) + + exp = explain_finding(proj, path="svc.py", line=6, loomweave=SpyClient([view]), sink_qualname="svc.leaky") + + assert exp is not None + assert exp.sink_qualname == "svc.leaky" + assert calls["n"] == 1 + + +def test_malformed_blob_counts_fall_back_to_reanalysis(tmp_path, monkeypatch): + proj = _proj(tmp_path) + blob, h = _fresh_blob(proj, "svc.leaky") + blob["taint"]["resolved_call_count"] = "not-an-int" + view = TaintFactView(qualname="svc.leaky", exists=True, wardline_json=blob, current_content_hash=h) + + import wardline.core.explain as explain_mod + + calls = {"n": 0} + real = explain_mod.run_scan + + def counting(*a, **k): + calls["n"] += 1 + return real(*a, **k) + + monkeypatch.setattr(explain_mod, "run_scan", counting) + + exp = explain_finding(proj, path="svc.py", line=6, loomweave=SpyClient([view]), sink_qualname="svc.leaky") + + assert exp is not None + assert exp.resolved_call_count == 1 + assert calls["n"] == 1 + + def test_stale_hash_falls_back_to_reanalysis(tmp_path): proj = _proj(tmp_path) blob, h = _fresh_blob(proj, "svc.leaky") From 72411e7c6e410717c8806ea82577c783bb01622f Mon Sep 17 00:00:00 2001 From: John Morrissey Date: Fri, 12 Jun 2026 06:48:39 +1000 Subject: [PATCH 122/186] fix(loomweave): cap client request bodies --- src/wardline/loomweave/client.py | 75 ++++++++++++++++++++++++----- tests/unit/loomweave/test_client.py | 48 ++++++++++++++++++ 2 files changed, 110 insertions(+), 13 deletions(-) diff --git a/src/wardline/loomweave/client.py b/src/wardline/loomweave/client.py index cc5ff60d..bc71ed54 100644 --- a/src/wardline/loomweave/client.py +++ b/src/wardline/loomweave/client.py @@ -19,7 +19,7 @@ import urllib.error import urllib.parse import urllib.request -from collections.abc import Iterator, Mapping, Sequence +from collections.abc import Callable, Iterator, Mapping, Sequence from dataclasses import dataclass from typing import Any, Protocol @@ -31,6 +31,8 @@ _ALLOWED_SCHEMES = ("http", "https") +MAX_REQUEST_BODY_BYTES = 4 * 1024 * 1024 +_JSON_SEPARATORS = (",", ":") @dataclass(frozen=True, slots=True) @@ -124,9 +126,8 @@ def from_wire(cls, obj: Mapping[str, Any]) -> TaintFactBySeiView: ) -def _chunks(seq: Sequence[Any], size: int) -> Iterator[Sequence[Any]]: - for i in range(0, len(seq), size): - yield seq[i : i + size] +def _json_body(payload: dict[str, Any]) -> bytes: + return json.dumps(payload, separators=_JSON_SEPARATORS).encode("utf-8") def _error_code(body: str) -> str | None: @@ -146,18 +147,49 @@ def __init__( project: str, transport: Transport | None = None, batch_max: int = 2000, + max_body_bytes: int = MAX_REQUEST_BODY_BYTES, ) -> None: self._base = base_url.rstrip("/") self._secret = secret self._project = project self._transport: Transport = transport if transport is not None else UrllibTransport() - self._batch_max = batch_max + self._batch_max = max(1, batch_max) + self._max_body_bytes = max(1, max_body_bytes) + + def _payload_chunks( + self, + items: Sequence[Any], + make_payload: Callable[[list[Any]], dict[str, Any]], + ) -> Iterator[list[Any]]: + chunk: list[Any] = [] + for item in items: + candidate = [*chunk, item] + over_count = len(candidate) > self._batch_max + over_bytes = len(_json_body(make_payload(candidate))) > self._max_body_bytes + if chunk and (over_count or over_bytes): + yield chunk + chunk = [item] + continue + chunk = candidate + if len(chunk) == 1 and len(_json_body(make_payload(chunk))) > self._max_body_bytes: + yield chunk + chunk = [] + if chunk: + yield chunk def _send(self, method: str, path_and_query: str, payload: dict[str, Any] | None) -> Response | None: """Sign + send. Returns the Response, or None on a SOFT failure (outage/5xx).""" import time - body = json.dumps(payload).encode("utf-8") if payload is not None else b"" + body = _json_body(payload) if payload is not None else b"" + if len(body) > self._max_body_bytes: + logger.warning( + "Loomweave request body for %s is %d bytes, over the %d-byte cap", + path_and_query, + len(body), + self._max_body_bytes, + ) + return None headers: dict[str, str] = {} if payload is not None: headers["Content-Type"] = "application/json" @@ -209,10 +241,15 @@ def resolve(self, qualnames: list[str], *, plugin: str | None = None) -> Resolve would hide it). Outage/5xx stays ``None`` ("unreachable", ``_send``).""" resolved: dict[str, str] = {} unresolved: list[str] = [] - for chunk in _chunks(qualnames, self._batch_max): + + def make_payload(chunk: list[Any]) -> dict[str, Any]: payload: dict[str, Any] = {"project": self._project, "qualnames": list(chunk)} if plugin is not None: payload["plugin"] = plugin + return payload + + for chunk in self._payload_chunks(qualnames, make_payload): + payload = make_payload(chunk) resp = self._send("POST", "/api/wardline/resolve", payload) if resp is None: return None @@ -237,8 +274,12 @@ def write_taint_facts(self, facts: list[dict[str, Any]]) -> WriteResult: first failure — the caller's remedy is to re-run the whole scan/write.""" written = 0 unresolved: list[str] = [] - for chunk in _chunks(facts, self._batch_max): - payload = {"project": self._project, "facts": list(chunk)} + + def make_payload(chunk: list[Any]) -> dict[str, Any]: + return {"project": self._project, "facts": list(chunk)} + + for chunk in self._payload_chunks(facts, make_payload): + payload = make_payload(chunk) resp = self._send("POST", "/api/wardline/taint-facts", payload) if resp is None: return WriteResult(reachable=False) # soft outage @@ -264,8 +305,12 @@ def get_taint_fact(self, qualname: str) -> TaintFactView | None: def batch_get(self, qualnames: list[str]) -> list[TaintFactView] | None: views: list[TaintFactView] = [] - for chunk in _chunks(qualnames, self._batch_max): - payload = {"project": self._project, "qualnames": list(chunk)} + + def make_payload(chunk: list[Any]) -> dict[str, Any]: + return {"project": self._project, "qualnames": list(chunk)} + + for chunk in self._payload_chunks(qualnames, make_payload): + payload = make_payload(chunk) resp = self._send("POST", "/api/wardline/taint-facts:batch-get", payload) if resp is None: return None @@ -290,8 +335,12 @@ def batch_get_by_sei(self, seis: list[str]) -> list[TaintFactBySeiView] | None: Gate on :meth:`wardline.loomweave.identity.TaintStoreCapability` before calling — the route is absent on a pre-0006 Loomweave.""" views: list[TaintFactBySeiView] = [] - for chunk in _chunks(seis, self._batch_max): - payload = {"project": self._project, "seis": list(chunk)} + + def make_payload(chunk: list[Any]) -> dict[str, Any]: + return {"project": self._project, "seis": list(chunk)} + + for chunk in self._payload_chunks(seis, make_payload): + payload = make_payload(chunk) resp = self._send("POST", "/api/wardline/taint-facts/by-sei", payload) if resp is None: return None diff --git a/tests/unit/loomweave/test_client.py b/tests/unit/loomweave/test_client.py index 877c827e..cfc2a91a 100644 --- a/tests/unit/loomweave/test_client.py +++ b/tests/unit/loomweave/test_client.py @@ -68,6 +68,27 @@ def test_write_chunks_against_batch_max(): assert result.written == 6 +def test_write_chunks_against_serialized_body_size(): + t = FakeTransport([Response(status=200, body='{"written":1,"unresolved_qualnames":[]}')] * 3) + facts = [{"qualname": f"m.f{i}", "wardline_json": {"payload": "x" * 30}} for i in range(3)] + + result = _client(t, batch_max=100, max_body_bytes=180).write_taint_facts(facts) + + assert result.reachable is True + assert len(t.calls) > 1 + assert all(len(body) <= 180 for _method, _url, body, _headers in t.calls) + + +def test_write_oversized_single_fact_is_fail_soft_without_sending(): + t = FakeTransport() + fact = {"qualname": "m.big", "wardline_json": {"payload": "x" * 300}} + + result = _client(t, max_body_bytes=120).write_taint_facts([fact]) + + assert result.reachable is False + assert t.calls == [] + + def test_batch_get_chunks_and_preserves_input_order(): r1 = json.dumps([{"qualname": "a", "exists": False}, {"qualname": "b", "exists": False}]) r2 = json.dumps([{"qualname": "c", "exists": True, "wardline_json": {"x": 1}, "current_content_hash": "deadbeef"}]) @@ -119,6 +140,33 @@ def _raise(req, timeout=None): # noqa: ARG001 assert resp.body.endswith("[truncated]") +def test_urllib_transport_bounds_success_body(monkeypatch) -> None: + import io + + from wardline.core.http import MAX_RESPONSE_BODY_BYTES + from wardline.loomweave.client import UrllibTransport + + class HugeResponse(io.BytesIO): + status = 200 + + def __enter__(self): + return self + + def __exit__(self, *args): + return None + + monkeypatch.setattr( + urllib.request, + "urlopen", + lambda req, timeout=None: HugeResponse(b"x" * (MAX_RESPONSE_BODY_BYTES + 9)), # noqa: ARG005 + ) + + resp = UrllibTransport().request("POST", "http://loomweave.example/api/wardline/resolve", b"{}", {}) + + assert len(resp.body) < MAX_RESPONSE_BODY_BYTES + 128 + assert resp.body.endswith("[truncated]") + + def test_connection_error_is_soft(): class Boom: def request(self, *a, **k): From f9fa72cd4503deb9d67e260e11c17c13b910b77a Mon Sep 17 00:00:00 2001 From: John Morrissey Date: Fri, 12 Jun 2026 06:50:45 +1000 Subject: [PATCH 123/186] fix(mcp): expose cli policy gates --- src/wardline/cli/mcp.py | 18 ++++++++-- tests/unit/cli/test_mcp_cli.py | 61 ++++++++++++++++++++++++++++++++++ 2 files changed, 77 insertions(+), 2 deletions(-) diff --git a/src/wardline/cli/mcp.py b/src/wardline/cli/mcp.py index 2440d0b8..f2e629df 100644 --- a/src/wardline/cli/mcp.py +++ b/src/wardline/cli/mcp.py @@ -32,7 +32,15 @@ "`dossier` reads entity-associations (open work) from it." ), ) -def mcp(root: Path, loomweave_url: str | None, filigree_url: str | None) -> None: +@click.option("--read-only", is_flag=True, help="Disable MCP tools that require write capability.") +@click.option("--no-network", is_flag=True, help="Disable MCP tools that require network capability.") +def mcp( + root: Path, + loomweave_url: str | None, + filigree_url: str | None, + read_only: bool, + no_network: bool, +) -> None: """Run the Wardline MCP server over stdio (JSON-RPC 2.0).""" from wardline.core.config import resolve_filigree_url, resolve_loomweave_url @@ -42,4 +50,10 @@ def mcp(root: Path, loomweave_url: str | None, filigree_url: str | None) -> None # point thread the real path here too for parity. See resolve_loomweave_url's docstring. loomweave_url = resolve_loomweave_url(loomweave_url, root, None) filigree_url = resolve_filigree_url(filigree_url, root, None) - WardlineMCPServer(root=root, loomweave_url=loomweave_url, filigree_url=filigree_url).rpc.run_stdio() + WardlineMCPServer( + root=root, + loomweave_url=loomweave_url, + filigree_url=filigree_url, + allow_write=not read_only, + allow_network=not no_network, + ).rpc.run_stdio() diff --git a/tests/unit/cli/test_mcp_cli.py b/tests/unit/cli/test_mcp_cli.py index 382449dc..c2ce6989 100644 --- a/tests/unit/cli/test_mcp_cli.py +++ b/tests/unit/cli/test_mcp_cli.py @@ -38,6 +38,67 @@ def test_mcp_command_is_registered() -> None: result = CliRunner().invoke(cli, ["mcp", "--help"]) assert result.exit_code == 0 assert "stdio" in result.output.lower() or "mcp" in result.output.lower() + assert "--read-only" in result.output + assert "--no-network" in result.output + + +def test_mcp_command_passes_policy_flags(tmp_path, monkeypatch) -> None: + from click.testing import CliRunner + + import wardline.cli.mcp as mcp_cli + from wardline.cli.main import cli + + captured = {} + + class FakeRpc: + def run_stdio(self) -> None: + captured["ran"] = True + + class FakeServer: + def __init__( + self, + *, + root, + loomweave_url=None, + filigree_url=None, + allow_write=True, + allow_network=True, + ) -> None: + captured.update( + { + "root": root, + "loomweave_url": loomweave_url, + "filigree_url": filigree_url, + "allow_write": allow_write, + "allow_network": allow_network, + } + ) + self.rpc = FakeRpc() + + monkeypatch.setattr(mcp_cli, "WardlineMCPServer", FakeServer) + + result = CliRunner().invoke( + cli, + [ + "mcp", + "--root", + str(tmp_path), + "--loomweave-url", + "http://localhost:9100", + "--filigree-url", + "http://localhost:8628/api/weft/scan-results", + "--read-only", + "--no-network", + ], + ) + + assert result.exit_code == 0, result.output + assert captured["root"] == tmp_path + assert captured["loomweave_url"] == "http://localhost:9100" + assert captured["filigree_url"] == "http://localhost:8628/api/weft/scan-results" + assert captured["allow_write"] is False + assert captured["allow_network"] is False + assert captured["ran"] is True def test_mcp_command_runs_stdio_end_to_end(tmp_path) -> None: From a2828915ce29d69fda13f42d213343353c957f2b Mon Sep 17 00:00:00 2001 From: John Morrissey Date: Fri, 12 Jun 2026 06:52:24 +1000 Subject: [PATCH 124/186] docs: move scan cache outside checkout --- docs/reference/cli.md | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/docs/reference/cli.md b/docs/reference/cli.md index 2e1219d5..c1b9fc01 100644 --- a/docs/reference/cli.md +++ b/docs/reference/cli.md @@ -107,7 +107,7 @@ it at a package root, not a single file. | `--lang [python\|rust]` | Language frontend (default `python`). `rust` sweeps `*.rs` and covers the **command-injection slice** (`RS-WL-108`/`RS-WL-112`); needs the `wardline[rust]` extra. Finding identity is frozen and crate-prefixed (baseline-eligible); config severity overrides do not yet apply to Rust findings — see the [Rust support guide](../guides/rust-preview.md). | | `--output PATH` | Write findings to a file instead of stdout. | | `--fail-on [CRITICAL\|ERROR\|WARN\|INFO]` | Exit non-zero when any finding at or above this severity survives the baseline. Use this as your CI gate. | -| `--cache-dir PATH` | Persist the L3 inter-procedural summary cache here so the next scan reuses unchanged summaries. | +| `--cache-dir PATH` | Persist the L3 inter-procedural summary cache here so the next scan reuses unchanged summaries. Use an operator-owned directory outside untrusted checkouts; do not put the cache in a path that pull-request content can commit or modify. | | `--filigree-url TEXT` | Opt-in: POST findings to a Filigree Weft scan-results endpoint as well as emitting them locally. Prefer this native path when agents need Filigree promotion, deduplication, or close/reopen lifecycle state. | Realistic invocation — scan the source tree, emit SARIF to a file, and fail the @@ -120,9 +120,14 @@ $ wardline scan src/ --format sarif --output wardline.sarif --fail-on ERROR Incremental local run reusing a warm cache: ```text -$ wardline scan src/ --cache-dir .weft/wardline/cache +$ wardline scan src/ --cache-dir ~/.cache/wardline/my-project ``` +Only reuse a disk summary cache that is outside the scanned repository or is +otherwise protected from repository content. A cache directory inside an +untrusted checkout can be pre-populated by a pull request and must not be used as +CI gate input. + Agent handoff summary: ```text From 7bd4d56ac2961bbc434112a24fec13713349ecc0 Mon Sep 17 00:00:00 2001 From: John Morrissey Date: Fri, 12 Jun 2026 06:57:07 +1000 Subject: [PATCH 125/186] fix(release): pin publish workflow inputs --- .github/workflows/release.yml | 20 +++++++++++++++----- pyproject.toml | 2 +- 2 files changed, 16 insertions(+), 6 deletions(-) diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index 19faeb4c..38c9bc34 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -12,14 +12,20 @@ jobs: name: Build distributions runs-on: ubuntu-latest steps: - - uses: actions/checkout@v4 - - uses: astral-sh/setup-uv@v7 + - uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4.3.1 + with: + persist-credentials: false + - uses: astral-sh/setup-uv@37802adc94f370d6bfd71619e3f0bf239e1f3b78 # v7 with: enable-cache: true python-version: "3.13" - name: Build run: uv build - - uses: actions/upload-artifact@v4 + - name: Record artifact hashes + run: | + cd dist + sha256sum * > SHA256SUMS + - uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2 with: name: dist path: dist/ @@ -32,8 +38,12 @@ jobs: permissions: id-token: write steps: - - uses: actions/download-artifact@v8 + - uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1 with: name: dist path: dist/ - - uses: pypa/gh-action-pypi-publish@release/v1 + - name: Verify artifact hashes + run: | + cd dist + sha256sum --check SHA256SUMS + - uses: pypa/gh-action-pypi-publish@ed0c53931b1dc9bd32cbe73a98c7f6766f8a527e # v1.13.0 diff --git a/pyproject.toml b/pyproject.toml index 21ee4738..d06d34a5 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -1,5 +1,5 @@ [build-system] -requires = ["hatchling"] +requires = ["hatchling==1.30.1"] build-backend = "hatchling.build" [project] From e7ed990d3a1c6c769848816449d4c4bb10fc7fe0 Mon Sep 17 00:00:00 2001 From: John Morrissey Date: Fri, 12 Jun 2026 06:59:06 +1000 Subject: [PATCH 126/186] fix(ci): isolate sarif upload permission --- .github/workflows/ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 421d4ddf..d77c8b1d 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -14,7 +14,6 @@ concurrency: permissions: contents: read - security-events: write jobs: lint: @@ -90,9 +89,28 @@ jobs: # stays green, but a genuinely-introduced ERROR trust-boundary finding now goes red # instead of being silently uploaded. (wardline-751a9ae71b) run: uv run wardline scan src/ --format sarif --output results.sarif --fail-on ERROR + - name: Preserve SARIF for trusted upload + if: always() && github.event_name == 'push' && github.ref == 'refs/heads/main' + uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2 + with: + name: self-hosting-sarif + path: results.sarif + + self-hosting-sarif-upload: + name: Upload Self-Hosting SARIF + runs-on: ubuntu-latest + needs: self-hosting-scan + if: always() && github.event_name == 'push' && github.ref == 'refs/heads/main' + permissions: + actions: read + security-events: write + steps: + - uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1 + with: + name: self-hosting-sarif + path: . - name: Upload SARIF - if: always() - uses: github/codeql-action/upload-sarif@v3 + uses: github/codeql-action/upload-sarif@b0c4fd77f6c559021d78430ec4d0d169ae74a4eb # v3 with: sarif_file: results.sarif category: wardline-self-hosting From 37ab36bbc48743a1d960e24830e8b4d20a5ac29e Mon Sep 17 00:00:00 2001 From: John Morrissey Date: Fri, 12 Jun 2026 07:01:50 +1000 Subject: [PATCH 127/186] fix(sarif): refuse symlinked scan outputs --- src/wardline/cli/scan.py | 2 +- src/wardline/core/safe_paths.py | 20 ++++++++++++++++++-- src/wardline/core/sarif.py | 7 ++++--- tests/unit/cli/test_cli.py | 16 ++++++++++++++++ tests/unit/core/test_sarif.py | 15 +++++++++++++++ 5 files changed, 54 insertions(+), 6 deletions(-) diff --git a/src/wardline/cli/scan.py b/src/wardline/cli/scan.py index 5ea5b298..d131a13a 100644 --- a/src/wardline/cli/scan.py +++ b/src/wardline/cli/scan.py @@ -248,7 +248,7 @@ def confirm_cb(rel_path: str, orig: str, replacement: str, f: Finding) -> bool: ) findings = result.findings if fmt == "sarif": - sarif_sink = SarifSink(output) + sarif_sink = SarifSink(output, root=path if output_is_default else None) sarif_sink.write(findings, result.context) elif fmt == "jsonl": jsonl_sink = JsonlSink(output) diff --git a/src/wardline/core/safe_paths.py b/src/wardline/core/safe_paths.py index 6d084303..13c6a842 100644 --- a/src/wardline/core/safe_paths.py +++ b/src/wardline/core/safe_paths.py @@ -2,6 +2,7 @@ from __future__ import annotations +import os import stat from pathlib import Path @@ -18,7 +19,7 @@ def safe_project_path(root: Path, target: Path, *, label: str | None = None) -> root_resolved = root.resolve() candidate = target if target.is_absolute() else root_resolved / target name = label or candidate.name - if candidate.exists() and candidate.is_symlink(): + if candidate.is_symlink(): raise WardlineError(f"{name}: refusing to write through a symlink") resolved = candidate.resolve(strict=False) if not (resolved == root_resolved or resolved.is_relative_to(root_resolved)): @@ -37,7 +38,22 @@ def safe_project_file(root: Path, target: Path, *, label: str | None = None) -> def safe_write_text(root: Path, target: Path, content: str, *, label: str | None = None) -> None: """Safely write ``content`` to ``target`` under ``root``, resolving symlinks and boundary checks.""" safe_path = safe_project_file(root, target, label=label) - safe_path.write_text(content, encoding="utf-8") + safe_path.parent.mkdir(parents=True, exist_ok=True) + _write_text_no_follow(safe_path, content, label=label or safe_path.name) + + +def _write_text_no_follow(path: Path, content: str, *, label: str) -> None: + flags = os.O_WRONLY | os.O_CREAT | os.O_TRUNC + if hasattr(os, "O_NOFOLLOW"): + flags |= os.O_NOFOLLOW + try: + fd = os.open(path, flags, 0o666) + except OSError as exc: + if path.is_symlink(): + raise WardlineError(f"{label}: refusing to write through a symlink") from exc + raise + with os.fdopen(fd, "w", encoding="utf-8") as handle: + handle.write(content) def safe_read_text_if_regular( diff --git a/src/wardline/core/sarif.py b/src/wardline/core/sarif.py index e481de9a..083670ff 100644 --- a/src/wardline/core/sarif.py +++ b/src/wardline/core/sarif.py @@ -15,6 +15,7 @@ from wardline import __version__ from wardline.core.finding import Finding, Kind, Location, Severity, SuppressionState +from wardline.core.safe_paths import safe_write_text from wardline.scanner.flow_trace import build_finding_code_flow if TYPE_CHECKING: @@ -160,10 +161,10 @@ def build_sarif(findings: Sequence[Finding], context: AnalysisContext | None = N class SarifSink: - def __init__(self, path: Path) -> None: + def __init__(self, path: Path, *, root: Path | None = None) -> None: self._path = path + self._root = root def write(self, findings: Sequence[Finding], context: AnalysisContext | None = None) -> None: - self._path.parent.mkdir(parents=True, exist_ok=True) content = json.dumps(build_sarif(findings, context), indent=2, ensure_ascii=False) - self._path.write_text(content, encoding="utf-8") + safe_write_text(self._root or self._path.parent, self._path, content, label=self._path.name) diff --git a/tests/unit/cli/test_cli.py b/tests/unit/cli/test_cli.py index 1bc9b314..18236378 100644 --- a/tests/unit/cli/test_cli.py +++ b/tests/unit/cli/test_cli.py @@ -51,6 +51,22 @@ def test_scan_format_sarif_default_output_path(tmp_path: Path) -> None: assert (project / "findings.sarif").exists() +def test_scan_format_sarif_default_refuses_symlinked_output(tmp_path: Path) -> None: + import shutil + + project = tmp_path / "proj" + shutil.copytree(FIXTURE, project) + outside = tmp_path / "outside.txt" + outside.write_text("keep\n", encoding="utf-8") + (project / "findings.sarif").symlink_to(outside) + + result = CliRunner().invoke(cli, ["scan", str(project), "--format", "sarif"]) + + assert result.exit_code == 2 + assert "refusing to write through a symlink" in result.output + assert outside.read_text(encoding="utf-8") == "keep\n" + + def test_scan_format_sarif_still_gates(tmp_path: Path) -> None: proj = tmp_path / "proj" proj.mkdir() diff --git a/tests/unit/core/test_sarif.py b/tests/unit/core/test_sarif.py index ae13b285..2b851216 100644 --- a/tests/unit/core/test_sarif.py +++ b/tests/unit/core/test_sarif.py @@ -4,6 +4,9 @@ import json from pathlib import Path +import pytest + +from wardline.core.errors import WardlineError from wardline.core.finding import Finding, Kind, Location, Severity, SuppressionState from wardline.core.sarif import SarifSink, build_sarif from wardline.core.taints import TaintState @@ -115,6 +118,18 @@ def test_sink_writes_valid_json(tmp_path: Path) -> None: assert loaded["version"] == "2.1.0" +def test_sink_refuses_symlink_target(tmp_path: Path) -> None: + outside = tmp_path / "outside.txt" + outside.write_text("keep\n", encoding="utf-8") + out = tmp_path / "findings.sarif" + out.symlink_to(outside) + + with pytest.raises(WardlineError, match="refusing to write through a symlink"): + SarifSink(out).write([_f()]) + + assert outside.read_text(encoding="utf-8") == "keep\n" + + def test_metric_findings_excluded_from_sarif() -> None: """Kind.METRIC findings (engine telemetry) must not appear in SARIF output.""" metric = _f(rule_id="WLN-L3-LOW-RESOLUTION", sev=Severity.INFO, kind=Kind.METRIC) From 2ae437bd1624359734e641a0a5c77a1d03707158 Mon Sep 17 00:00:00 2001 From: John Morrissey Date: Fri, 12 Jun 2026 07:03:26 +1000 Subject: [PATCH 128/186] test(taint): pin match subject walrus propagation --- .../unit/scanner/rules/test_taint_closures_locked.py | 12 ++++++++++++ tests/unit/scanner/taint/test_variable_level.py | 6 ++++++ 2 files changed, 18 insertions(+) diff --git a/tests/unit/scanner/rules/test_taint_closures_locked.py b/tests/unit/scanner/rules/test_taint_closures_locked.py index 138f39ae..601e0679 100644 --- a/tests/unit/scanner/rules/test_taint_closures_locked.py +++ b/tests/unit/scanner/rules/test_taint_closures_locked.py @@ -55,6 +55,18 @@ def test_walrus_propagates_raw_to_trusted_return(tmp_path: Path) -> None: ) +def test_match_subject_nested_walrus_propagates_raw_to_trusted_return(tmp_path: Path) -> None: + assert {"PY-WL-101"} == _defects( + tmp_path, + "@trusted(level='ASSURED')\n" + "def f(p):\n" + " match (s := read_raw(p))[0]:\n" + " case _:\n" + " pass\n" + " return s", + ) + + def test_starred_unpack_raw_slice_propagates_to_trusted_return(tmp_path: Path) -> None: assert {"PY-WL-101"} == _defects( tmp_path, diff --git a/tests/unit/scanner/taint/test_variable_level.py b/tests/unit/scanner/taint/test_variable_level.py index 4dbbcf29..2afd0973 100644 --- a/tests/unit/scanner/taint/test_variable_level.py +++ b/tests/unit/scanner/taint/test_variable_level.py @@ -554,6 +554,12 @@ def test_match_subject_walrus_is_captured() -> None: assert out["s"] == T.EXTERNAL_RAW +def test_match_subject_nested_walrus_is_captured() -> None: + src = "def f(p):\n match (s := tainted())[0]:\n case _:\n pass\n" + out = _vt(src, function_taint=T.ASSURED, taint_map={"tainted": T.EXTERNAL_RAW}) + assert out["s"] == T.EXTERNAL_RAW + + def test_match_does_not_descend_into_nested_function() -> None: src = ( "def f(p):\n" From f051635ca28a4d765f6bafa3b6aef61fc7bd31fe Mon Sep 17 00:00:00 2001 From: John Morrissey Date: Fri, 12 Jun 2026 07:08:06 +1000 Subject: [PATCH 129/186] fix(rules): ignore unreachable boundary rejections --- src/wardline/scanner/rules/_ast_helpers.py | 114 +++++++++++++++--- tests/unit/scanner/rules/test_ast_helpers.py | 21 ++++ .../rules/test_boundary_without_rejection.py | 25 ++++ 3 files changed, 146 insertions(+), 14 deletions(-) diff --git a/src/wardline/scanner/rules/_ast_helpers.py b/src/wardline/scanner/rules/_ast_helpers.py index dceb65ca..2cc5579e 100644 --- a/src/wardline/scanner/rules/_ast_helpers.py +++ b/src/wardline/scanner/rules/_ast_helpers.py @@ -43,6 +43,86 @@ def _own_statements(node: ast.AST) -> Iterator[ast.stmt]: yield from _own_statements(child) +def _own_reachable_statements(node: ast.FunctionDef | ast.AsyncFunctionDef) -> Iterator[ast.stmt]: + yield from _reachable_statements_in_block(node.body) + + +def _own_reachable_nodes(node: ast.FunctionDef | ast.AsyncFunctionDef) -> Iterator[ast.AST]: + for stmt in _own_reachable_statements(node): + yield from _own_nodes_in_reachable_stmt(stmt) + + +def _own_nodes_in_reachable_stmt(stmt: ast.stmt) -> Iterator[ast.AST]: + yield stmt + if isinstance(stmt, (ast.FunctionDef, ast.AsyncFunctionDef, ast.ClassDef)): + return + yield from _walk_own_non_stmt_children(stmt) + + +def _walk_own_non_stmt_children(node: ast.AST) -> Iterator[ast.AST]: + for child in ast.iter_child_nodes(node): + if isinstance(child, (ast.FunctionDef, ast.AsyncFunctionDef, ast.ClassDef, ast.Lambda)): + yield child + elif isinstance(child, ast.stmt): + continue + else: + yield child + yield from _walk_own_non_stmt_children(child) + + +def _reachable_statements_in_block(stmts: list[ast.stmt]) -> Iterator[ast.stmt]: + for stmt in stmts: + yield stmt + if not isinstance(stmt, (ast.FunctionDef, ast.AsyncFunctionDef, ast.ClassDef)): + for block in _child_statement_blocks(stmt): + yield from _reachable_statements_in_block(block) + if _stmt_always_terminates(stmt): + break + + +def _child_statement_blocks(stmt: ast.stmt) -> Iterator[list[ast.stmt]]: + if isinstance(stmt, (ast.If, ast.For, ast.AsyncFor, ast.While)): + yield stmt.body + yield stmt.orelse + elif isinstance(stmt, (ast.With, ast.AsyncWith)): + yield stmt.body + elif isinstance(stmt, (ast.Try, ast.TryStar)): + yield stmt.body + yield stmt.orelse + yield stmt.finalbody + for handler in stmt.handlers: + yield handler.body + elif isinstance(stmt, ast.Match): + for case in stmt.cases: + yield case.body + + +def _block_always_terminates(stmts: list[ast.stmt]) -> bool: + return any(_stmt_always_terminates(stmt) for stmt in stmts) + + +def _match_has_irrefutable_case(stmt: ast.Match) -> bool: + return any( + isinstance(case.pattern, ast.MatchAs) and case.pattern.pattern is None and case.guard is None + for case in stmt.cases + ) + + +def _stmt_always_terminates(stmt: ast.stmt) -> bool: + if isinstance(stmt, (ast.Return, ast.Raise)): + return True + if isinstance(stmt, ast.If): + return ( + bool(stmt.body) + and bool(stmt.orelse) + and _block_always_terminates(stmt.body) + and _block_always_terminates(stmt.orelse) + ) + if isinstance(stmt, ast.Match): + return _match_has_irrefutable_case(stmt) and all(_block_always_terminates(case.body) for case in stmt.cases) + return False + + def own_except_handlers(node: ast.FunctionDef | ast.AsyncFunctionDef) -> Iterator[ast.ExceptHandler]: """Yield the ``except`` handlers in *node*'s own scope (excludes nested defs).""" for stmt in _own_statements(node): @@ -147,7 +227,7 @@ def has_real_rejection(node: ast.FunctionDef | ast.AsyncFunctionDef) -> bool: a ``raise`` or a rejection-shaped ``return`` — i.e. NOT counting ``assert``. This is PY-WL-113's premise half: a rejection must EXIST (and survive ``-O``) before a fail-open handler can be said to defeat it.""" - return any(_stmt_is_real_rejection(stmt) for stmt in _own_statements(node)) + return any(_stmt_is_real_rejection(stmt) for stmt in _own_reachable_statements(node)) def has_rejection_path(node: ast.FunctionDef | ast.AsyncFunctionDef) -> bool: @@ -170,7 +250,10 @@ def has_rejection_path(node: ast.FunctionDef | ast.AsyncFunctionDef) -> bool: - PY-WL-111 — the only rejection is ``assert``; - PY-WL-113 — a real rejection exists but a fail-open handler defeats it. """ - return any(isinstance(stmt, ast.Assert) or _stmt_is_real_rejection(stmt) for stmt in _own_statements(node)) + return any( + isinstance(stmt, ast.Assert) or _stmt_is_real_rejection(stmt) + for stmt in _own_reachable_statements(node) + ) def asserts_are_sole_rejection(node: ast.FunctionDef | ast.AsyncFunctionDef) -> bool: @@ -185,7 +268,7 @@ def asserts_are_sole_rejection(node: ast.FunctionDef | ast.AsyncFunctionDef) -> see the project context additionally consult :func:`rejecting_helper_calls` — a raising same-module helper survives ``-O`` and rescues the boundary.""" has_assert = False - for stmt in _own_statements(node): + for stmt in _own_reachable_statements(node): if _stmt_is_real_rejection(stmt): return False if isinstance(stmt, ast.Assert): @@ -275,7 +358,7 @@ def rejecting_helper_calls( assert vanishes under ``python -O`` exactly like an inline one, which would falsely silence PY-WL-111).""" ids: set[int] = set() - for n in own_nodes(entity.node): + for n in _own_reachable_nodes(entity.node): if isinstance(n, ast.Call): callee = _resolve_one_hop_callee(n, entity, entities, call_site_callees) if callee is not None and has_real_rejection(callee.node): @@ -283,22 +366,25 @@ def rejecting_helper_calls( return frozenset(ids) +def _own_reachable_nodes_in_blocks(stmts: list[ast.stmt]) -> Iterator[ast.AST]: + for stmt in _reachable_statements_in_block(stmts): + yield from _own_nodes_in_reachable_stmt(stmt) + + def block_has_real_rejection(stmts: list[ast.stmt], rejecting_call_ids: frozenset[int] = frozenset()) -> bool: """True when the statement list *stmts* (a ``try`` body or handler body) - lexically contains a real rejection — a ``raise`` / rejection-shaped - ``return`` in its own scope, or a call whose ``id()`` is in + lexically contains a reachable real rejection — a ``raise`` / rejection-shaped + ``return`` in its own scope, or a reachable call whose ``id()`` is in *rejecting_call_ids* (a one-hop rejecting helper, see :func:`rejecting_helper_calls`). PY-WL-113's per-``try`` premise: a handler can only swallow a rejection that lives inside its own ``try``.""" - for top in stmts: - for stmt in (top, *_own_statements(top)): - if _stmt_is_real_rejection(stmt): - return True + for stmt in _reachable_statements_in_block(stmts): + if _stmt_is_real_rejection(stmt): + return True if rejecting_call_ids: - for top in stmts: - for n in own_nodes(top): - if isinstance(n, ast.Call) and id(n) in rejecting_call_ids: - return True + for n in _own_reachable_nodes_in_blocks(stmts): + if isinstance(n, ast.Call) and id(n) in rejecting_call_ids: + return True return False diff --git a/tests/unit/scanner/rules/test_ast_helpers.py b/tests/unit/scanner/rules/test_ast_helpers.py index 8e6e5d90..c7471400 100644 --- a/tests/unit/scanner/rules/test_ast_helpers.py +++ b/tests/unit/scanner/rules/test_ast_helpers.py @@ -35,6 +35,13 @@ def test_has_rejection_path_detects_raise_and_falsy_returns() -> None: assert not has_rejection_path(_fn("def f(p):\n x = p\n return x\n")) +def test_unreachable_rejections_do_not_count() -> None: + assert not has_rejection_path(_fn("def f(p):\n return p\n raise ValueError\n")) + assert not has_real_rejection(_fn("def f(p):\n return p\n raise ValueError\n")) + assert not has_rejection_path(_fn("def f(p):\n if c:\n return p\n else:\n return p\n raise ValueError\n")) + assert not asserts_are_sole_rejection(_fn("def f(p):\n return p\n assert p\n")) + + def test_asserts_are_sole_rejection() -> None: # only an assert -> True (PY-WL-111's case) assert asserts_are_sole_rejection(_fn("def f(p):\n assert p\n return p\n")) @@ -118,6 +125,20 @@ def v(p): assert len(calls) == 1 +def test_rejecting_helper_call_after_return_does_not_count() -> None: + ents = _entities( + """ + def _require_nonempty(p): + if not p: + raise ValueError("empty") + def v(p): + return p + _require_nonempty(p) + """ + ) + assert rejecting_helper_calls(ents["m.v"], ents, {}) == frozenset() + + def test_rejecting_helper_calls_staticmethod_helper() -> None: ents = _entities( """ diff --git a/tests/unit/scanner/rules/test_boundary_without_rejection.py b/tests/unit/scanner/rules/test_boundary_without_rejection.py index d7d9e3d7..9e2aa7c6 100644 --- a/tests/unit/scanner/rules/test_boundary_without_rejection.py +++ b/tests/unit/scanner/rules/test_boundary_without_rejection.py @@ -68,6 +68,18 @@ def test_boundary_with_raise_is_clean(tmp_path) -> None: assert _run(ctx) == [] +def test_unreachable_raise_does_not_rescue_boundary(tmp_path) -> None: + ctx, _ = _analyze( + tmp_path, + { + "m.py": "from wardline.decorators import trust_boundary\n" + "@trust_boundary(to_level='ASSURED')\n" + "def v(p):\n x = p\n return x\n raise ValueError\n", + }, + ) + assert [(f.rule_id, f.qualname) for f in _run(ctx)] == [("PY-WL-102", "m.v")] + + def test_boundary_with_falsy_return_is_clean(tmp_path) -> None: ctx, _ = _analyze( tmp_path, @@ -114,6 +126,19 @@ def test_boundary_rejecting_via_same_module_raising_helper_is_clean(tmp_path) -> assert _run(ctx) == [] +def test_unreachable_rejecting_helper_does_not_rescue_boundary(tmp_path) -> None: + ctx, _ = _analyze( + tmp_path, + { + "m.py": "from wardline.decorators import trust_boundary\n" + "def _require_nonempty(p):\n if not p:\n raise ValueError('empty')\n" + "@trust_boundary(to_level='ASSURED')\n" + "def v(p):\n x = p\n return x\n _require_nonempty(p)\n", + }, + ) + assert [(f.rule_id, f.qualname) for f in _run(ctx)] == [("PY-WL-102", "m.v")] + + def test_boundary_rejecting_via_staticmethod_helper_is_clean(tmp_path) -> None: ctx, _ = _analyze( tmp_path, From 5bbe6035716462844af61f370ff863ef3f1e2b9f Mon Sep 17 00:00:00 2001 From: John Morrissey Date: Fri, 12 Jun 2026 07:10:11 +1000 Subject: [PATCH 130/186] fix(taint): preserve project submodule import taint --- src/wardline/scanner/taint/call_taint_map.py | 9 ++++++- .../rules/test_untrusted_reaches_trusted.py | 16 +++++++++++++ .../unit/scanner/taint/test_call_taint_map.py | 24 +++++++++++++++++++ 3 files changed, 48 insertions(+), 1 deletion(-) diff --git a/src/wardline/scanner/taint/call_taint_map.py b/src/wardline/scanner/taint/call_taint_map.py index cbb9d014..698c3f26 100644 --- a/src/wardline/scanner/taint/call_taint_map.py +++ b/src/wardline/scanner/taint/call_taint_map.py @@ -91,7 +91,14 @@ def build_call_taint_map( # module import: dotted ``local.func`` calls for func_name, taint in bucket.items(): tm[f"{local}.{func_name}"] = taint - continue + for module, module_bucket in project_by_module.items(): + if module.startswith(target + "."): + # ``import pkg.sub`` collapses the alias to ``pkg``; the call is + # written ``local..fn`` just like multi-component + # stdlib imports. + remainder = module[len(target) + 1 :] + for func_name, taint in module_bucket.items(): + tm[f"{local}.{remainder}.{func_name}"] = taint # from-import of a project function: target == "module.func_name" mod, _, leaf = target.rpartition(".") mod_bucket = project_by_module.get(mod) diff --git a/tests/unit/scanner/rules/test_untrusted_reaches_trusted.py b/tests/unit/scanner/rules/test_untrusted_reaches_trusted.py index bc6b874f..b2e0fd85 100644 --- a/tests/unit/scanner/rules/test_untrusted_reaches_trusted.py +++ b/tests/unit/scanner/rules/test_untrusted_reaches_trusted.py @@ -44,6 +44,22 @@ def test_trusted_returning_raw_fires(tmp_path) -> None: assert all(f.kind == Kind.DEFECT for f in findings) +def test_trusted_returning_project_submodule_imported_raw_fires(tmp_path) -> None: + ctx, _ = _analyze( + tmp_path, + { + "pkg/sources.py": "from wardline.decorators import external_boundary\n" + "@external_boundary\ndef read(p):\n return p\n", + "svc.py": "import pkg.sources\n" + "from wardline.decorators import trusted\n" + "@trusted(level='ASSURED')\n" + "def leaky(p):\n return pkg.sources.read(p)\n", + }, + ) + assert ctx.function_return_taints["svc.leaky"] == TaintState.EXTERNAL_RAW + assert ("PY-WL-101", "svc.leaky") in {(f.rule_id, f.qualname) for f in _run(ctx)} + + def test_trusted_early_returning_raw_before_later_clean_reassignment_fires(tmp_path) -> None: ctx, _ = _analyze( tmp_path, diff --git a/tests/unit/scanner/taint/test_call_taint_map.py b/tests/unit/scanner/taint/test_call_taint_map.py index 60c8d74d..5b401f40 100644 --- a/tests/unit/scanner/taint/test_call_taint_map.py +++ b/tests/unit/scanner/taint/test_call_taint_map.py @@ -43,6 +43,16 @@ def test_dotted_module_project_call_keyed_dotted() -> None: assert tm["other.fn"] == T.MIXED_RAW +def test_multicomponent_project_plain_import_keyed_fully() -> None: + aliases = _aliases("import pkg.sources\n", "m") + tm = build_call_taint_map( + module_path="m", + alias_map=aliases, + project_by_module={"pkg.sources": {"read": T.EXTERNAL_RAW}}, + ) + assert tm["pkg.sources.read"] == T.EXTERNAL_RAW + + def test_stdlib_external_dotted_taint_carries() -> None: # Positive external-dotted channel: a non-sink stdlib entry, aliased. aliases = _aliases("import subprocess as sp\n", "m") @@ -133,6 +143,20 @@ def test_l2_urllib_plain_import_carries_external_raw_end_to_end() -> None: assert out["x"] == T.EXTERNAL_RAW # the curated network-source taint, not dropped +def test_l2_project_plain_submodule_import_carries_return_taint_end_to_end() -> None: + src = "import pkg.sources\ndef f(p):\n x = pkg.sources.read(p)\n" + func = ast.parse(src).body[1] + assert isinstance(func, ast.FunctionDef) + aliases = build_import_alias_map(ast.parse(src), module_path="m") + tm = build_call_taint_map( + module_path="m", + alias_map=aliases, + project_by_module={"pkg.sources": {"read": T.EXTERNAL_RAW}}, + ) + out = compute_variable_taints(func, T.ASSURED, dict(tm)) + assert out["x"] == T.EXTERNAL_RAW + + # ── PART D: aliased serialisation sinks NOT in stdlib_taint resolve to UNKNOWN_RAW ── # # json.dumps/json.dump are in _SERIALISATION_SINKS but ABSENT from stdlib_taint From 12e3bd587e1b56560220ef7f68d4f08e95ec1431 Mon Sep 17 00:00:00 2001 From: John Morrissey Date: Fri, 12 Jun 2026 07:18:08 +1000 Subject: [PATCH 131/186] fix(taint): authenticate summary cache files --- docs/reference/cli.md | 16 +- src/wardline/core/run.py | 4 +- src/wardline/scanner/taint/summary_cache.py | 113 ++++++++++++-- tests/unit/cli/test_cli.py | 5 +- .../scanner/taint/test_provenance_clash.py | 16 +- .../unit/scanner/taint/test_summary_cache.py | 143 +++++++++++++++--- 6 files changed, 244 insertions(+), 53 deletions(-) diff --git a/docs/reference/cli.md b/docs/reference/cli.md index c1b9fc01..1f384c43 100644 --- a/docs/reference/cli.md +++ b/docs/reference/cli.md @@ -90,8 +90,9 @@ Options: .rs files for command-injection findings. --output PATH --fail-on [CRITICAL|ERROR|WARN|INFO] - --cache-dir PATH Persist L3 summary cache here for faster - incremental scans. + --cache-dir PATH Store authenticated L3 summary-cache entries + here for faster incremental scans when + WARDLINE_SUMMARY_CACHE_KEY is set. --filigree-url TEXT POST findings to this Filigree Weft scan- results URL (opt-in). --help Show this message and exit. @@ -107,7 +108,7 @@ it at a package root, not a single file. | `--lang [python\|rust]` | Language frontend (default `python`). `rust` sweeps `*.rs` and covers the **command-injection slice** (`RS-WL-108`/`RS-WL-112`); needs the `wardline[rust]` extra. Finding identity is frozen and crate-prefixed (baseline-eligible); config severity overrides do not yet apply to Rust findings — see the [Rust support guide](../guides/rust-preview.md). | | `--output PATH` | Write findings to a file instead of stdout. | | `--fail-on [CRITICAL\|ERROR\|WARN\|INFO]` | Exit non-zero when any finding at or above this severity survives the baseline. Use this as your CI gate. | -| `--cache-dir PATH` | Persist the L3 inter-procedural summary cache here so the next scan reuses unchanged summaries. Use an operator-owned directory outside untrusted checkouts; do not put the cache in a path that pull-request content can commit or modify. | +| `--cache-dir PATH` | Store the L3 inter-procedural summary cache here so the next scan can reuse unchanged summaries when `WARDLINE_SUMMARY_CACHE_KEY` is set in the process environment. Unsigned or incorrectly signed files are ignored and the scan falls back to recomputing summaries. Use an operator-owned directory outside untrusted checkouts; do not put the cache in a path that pull-request content can commit or modify. | | `--filigree-url TEXT` | Opt-in: POST findings to a Filigree Weft scan-results endpoint as well as emitting them locally. Prefer this native path when agents need Filigree promotion, deduplication, or close/reopen lifecycle state. | Realistic invocation — scan the source tree, emit SARIF to a file, and fail the @@ -120,13 +121,14 @@ $ wardline scan src/ --format sarif --output wardline.sarif --fail-on ERROR Incremental local run reusing a warm cache: ```text +$ export WARDLINE_SUMMARY_CACHE_KEY="$(openssl rand -hex 32)" $ wardline scan src/ --cache-dir ~/.cache/wardline/my-project ``` -Only reuse a disk summary cache that is outside the scanned repository or is -otherwise protected from repository content. A cache directory inside an -untrusted checkout can be pre-populated by a pull request and must not be used as -CI gate input. +Only reuse a disk summary cache when `WARDLINE_SUMMARY_CACHE_KEY` is provisioned +from trusted process environment. Unsigned cache files are never loaded, and a +cache directory inside an untrusted checkout is still a poor CI choice because +repository content can force cold-cache behavior by deleting or replacing files. Agent handoff summary: diff --git a/src/wardline/core/run.py b/src/wardline/core/run.py index d24e9092..89a39147 100644 --- a/src/wardline/core/run.py +++ b/src/wardline/core/run.py @@ -200,7 +200,7 @@ def run_scan( raise ConfigError(f"unknown language {lang!r}; expected one of {known}") frontend = FRONTENDS[lang] suffixes = frontend.suffixes - from wardline.scanner.taint.summary_cache import SummaryCache + from wardline.scanner.taint.summary_cache import SummaryCache, summary_cache_auth_secret_from_env # An EXPLICIT --config path must NOT silently fall back to default policy # (dropping the operator's severity overrides/excludes) whether it is missing @@ -217,7 +217,7 @@ def run_scan( ) cache = None if cache_dir is not None: - cache = SummaryCache(cache_dir=cache_dir) + cache = SummaryCache(cache_dir=cache_dir, cache_auth_secret=summary_cache_auth_secret_from_env()) from wardline.core.taints import _PROVENANCE_CLASH token_clash = _PROVENANCE_CLASH.set(cfg.provenance_clash) diff --git a/src/wardline/scanner/taint/summary_cache.py b/src/wardline/scanner/taint/summary_cache.py index 4bc2e7e8..518eccd6 100644 --- a/src/wardline/scanner/taint/summary_cache.py +++ b/src/wardline/scanner/taint/summary_cache.py @@ -1,17 +1,20 @@ # src/wardline/scanner/taint/summary_cache.py -"""In-memory (+ optional disk) summary cache for L3 project-scope transitive taint. +"""In-memory (+ authenticated optional disk) summary cache for L3 project-scope transitive taint. Keyed on the SP1d module ``cache_key`` (every FunctionSummary in a module shares one key), so the store maps ``cache_key -> tuple[FunctionSummary, ...]``. -Disk persistence: construct with ``cache_dir=Path(...)`` then call ``load()`` -before analysis and ``save()`` after. Malformed / stale-schema / non-hex-stem -files are silently dropped on load (cold-cache fallback). Atomic write-then- -replace so a crash leaves the prior file or none — never a partial file. +Disk persistence: construct with ``cache_dir=Path(...)`` and a process-controlled +``cache_auth_secret`` then call ``load()`` before analysis and ``save()`` after. +Malformed / stale-schema / non-hex-stem / unauthenticated files are silently +dropped on load (cold-cache fallback). Atomic write-then-replace so a crash +leaves the prior file or none — never a partial file. -No governance: ``.old``'s CI-attestation / GOVERNANCE-CACHE-UNATTESTED path is -discarded outright. The 64-char hex key validation is retained: it is cheap and -guards the persistent-cache file path against traversal. +No repository governance: ``.old``'s CI-attestation / GOVERNANCE-CACHE-UNATTESTED +path is discarded outright. The 64-char hex key validation is retained: it is +cheap and guards the persistent-cache file path against traversal. Disk cache +files are integrity-checked with an operator-held HMAC key before they can +rehydrate summaries, so repository-controlled JSON cannot become analyzer truth. Correctness note: this cache memoizes only the source-determined taint contract (body/return/source), which ``cache_key`` fully captures. The resolver always @@ -22,11 +25,14 @@ from __future__ import annotations import contextlib +import hashlib +import hmac import json import logging import os import re import tempfile +from collections.abc import Mapping from pathlib import Path from typing import TYPE_CHECKING, ClassVar, cast @@ -41,6 +47,11 @@ _logger = logging.getLogger(__name__) +SUMMARY_CACHE_KEY_ENV = "WARDLINE_SUMMARY_CACHE_KEY" +"""Process environment key used to authenticate disk summary-cache entries.""" + +_CACHE_FILE_SCHEMA_VERSION = 1 + # The full reachable taint set for an analyzed function's cached body/return # taint. Unlike the stdlib table, a cached summary CAN be INTEGRAL (a @trusted # function produces INTEGRAL), so INTEGRAL is legal here. What must never be @@ -93,16 +104,21 @@ class SummaryCache: # its directory via a crafted key. _CACHE_KEY_PATTERN: ClassVar[re.Pattern[str]] = re.compile(r"^[0-9a-f]{64}$") - def __init__(self, *, cache_dir: Path | None = None) -> None: + def __init__(self, *, cache_dir: Path | None = None, cache_auth_secret: bytes | None = None) -> None: self._entries: dict[str, tuple[FunctionSummary, ...]] = {} self._hits: int = 0 self._misses: int = 0 self._cache_dir: Path | None = cache_dir + self._cache_auth_secret: bytes | None = cache_auth_secret @property def cache_dir(self) -> Path | None: return self._cache_dir + @property + def has_disk_auth(self) -> bool: + return self._cache_auth_secret is not None + @property def schema_version(self) -> int: return SUMMARY_SCHEMA_VERSION @@ -191,14 +207,19 @@ def save(self) -> None: """Atomically write every in-memory entry to ``/.json``. Write-temp-then-os.replace, so a crash leaves the prior file or none — - never a partial file. Raises ValueError if no cache_dir was set. + never a partial file. Raises ValueError if no cache_dir was set. Without + a ``cache_auth_secret``, disk persistence is disabled and save is a no-op + after ensuring the directory exists. """ if self._cache_dir is None: raise ValueError("SummaryCache.save() requires cache_dir") self._cache_dir.mkdir(parents=True, exist_ok=True) + if self._cache_auth_secret is None: + return for cache_key, summaries in self._entries.items(): target = self._cache_dir / f"{cache_key}.json" - payload = [_serialise_summary(s) for s in summaries] + payload = _cache_payload(cache_key, summaries) + payload["mac"] = _cache_payload_mac(self._cache_auth_secret, payload) # Opened outside a `with` so temp_path is known for cleanup-on-failure; # the `with tf:` below is the actual context manager. (SIM115) tf = tempfile.NamedTemporaryFile( # noqa: SIM115 @@ -211,7 +232,7 @@ def save(self) -> None: temp_path = Path(tf.name) try: with tf: - json.dump(payload, tf) + json.dump(payload, tf, sort_keys=True, separators=(",", ":")) os.replace(temp_path, target) except (OSError, ValueError, TypeError): # Clean up the temp file on ANY failure (dump or replace) so the @@ -221,12 +242,18 @@ def save(self) -> None: raise def load(self) -> None: - """Populate the store from ``/*.json``. Malformed / stale / - non-hex-stem files are silently dropped (cold-cache fallback). Raises - ValueError if no cache_dir was set.""" + """Populate the store from ``/*.json``. + + Malformed / stale / non-hex-stem / unauthenticated files are silently + dropped (cold-cache fallback). Raises ValueError if no cache_dir was set. + Without a ``cache_auth_secret``, disk loading is disabled and load is a + no-op after ensuring the directory exists. + """ if self._cache_dir is None: raise ValueError("SummaryCache.load() requires cache_dir") self._cache_dir.mkdir(parents=True, exist_ok=True) + if self._cache_auth_secret is None: + return for path in sorted(self._cache_dir.iterdir()): if path.suffix != ".json": continue @@ -235,7 +262,11 @@ def load(self) -> None: continue try: payload = json.loads(path.read_text(encoding="utf-8")) - summaries = tuple(_deserialise_summary(d) for d in payload) + summaries = _deserialise_cache_payload( + payload, + expected_cache_key=cache_key, + cache_auth_secret=self._cache_auth_secret, + ) _validate_loaded_entry(cache_key, summaries) except (json.JSONDecodeError, ValueError, KeyError, TypeError) as exc: _logger.warning("SummaryCache.load: dropping malformed entry %s: %s", path, exc) @@ -250,6 +281,13 @@ def load(self) -> None: self._entries[cache_key] = summaries +def summary_cache_auth_secret_from_env() -> bytes | None: + value = os.environ.get(SUMMARY_CACHE_KEY_ENV) + if value is None or value == "": + return None + return value.encode("utf-8") + + def _validate_loaded_entry(cache_key: str, summaries: tuple[FunctionSummary, ...]) -> None: mismatched = [s.fqn for s in summaries if s.cache_key != cache_key] if mismatched: @@ -283,3 +321,46 @@ def _deserialise_summary(d: dict[str, object]) -> FunctionSummary: schema_version=int(cast("int", d["schema_version"])), cache_key=str(d["cache_key"]), ) + + +def _cache_payload(cache_key: str, summaries: tuple[FunctionSummary, ...]) -> dict[str, object]: + return { + "schema_version": _CACHE_FILE_SCHEMA_VERSION, + "cache_key": cache_key, + "summaries": [_serialise_summary(s) for s in summaries], + } + + +def _cache_payload_mac(cache_auth_secret: bytes, payload: Mapping[str, object]) -> str: + payload_without_mac = {key: value for key, value in payload.items() if key != "mac"} + encoded = json.dumps(payload_without_mac, sort_keys=True, separators=(",", ":"), ensure_ascii=True).encode("utf-8") + return hmac.new(cache_auth_secret, encoded, hashlib.sha256).hexdigest() + + +def _deserialise_cache_payload( + payload: object, + *, + expected_cache_key: str, + cache_auth_secret: bytes, +) -> tuple[FunctionSummary, ...]: + if not isinstance(payload, dict): + raise ValueError("cache entry is not an authenticated envelope") + mac = payload.get("mac") + if not isinstance(mac, str): + raise ValueError("cache entry missing mac") + expected_mac = _cache_payload_mac(cache_auth_secret, cast("Mapping[str, object]", payload)) + if not hmac.compare_digest(mac, expected_mac): + raise ValueError("cache entry mac mismatch") + if payload.get("schema_version") != _CACHE_FILE_SCHEMA_VERSION: + raise ValueError( + f"cache file schema_version={payload.get('schema_version')!r} " + f"!= {_CACHE_FILE_SCHEMA_VERSION}" + ) + if payload.get("cache_key") != expected_cache_key: + raise ValueError( + f"envelope cache_key={payload.get('cache_key')!r} does not match filename key {expected_cache_key!r}" + ) + raw_summaries = payload.get("summaries") + if not isinstance(raw_summaries, list): + raise ValueError("cache entry summaries must be a list") + return tuple(_deserialise_summary(cast("dict[str, object]", item)) for item in raw_summaries) diff --git a/tests/unit/cli/test_cli.py b/tests/unit/cli/test_cli.py index 18236378..3edd1200 100644 --- a/tests/unit/cli/test_cli.py +++ b/tests/unit/cli/test_cli.py @@ -401,10 +401,11 @@ def test_scan_cache_dir_persists_warm_taints_equal_cold(tmp_path) -> None: out1 = tmp_path / "f1.jsonl" out2 = tmp_path / "f2.jsonl" runner = CliRunner() - r1 = runner.invoke(scan, [str(proj), "--cache-dir", str(cache), "--output", str(out1)]) + env = {"WARDLINE_SUMMARY_CACHE_KEY": "unit-test-summary-cache-key"} + r1 = runner.invoke(scan, [str(proj), "--cache-dir", str(cache), "--output", str(out1)], env=env) assert r1.exit_code == 0, r1.output assert cache.exists() and any(cache.iterdir()) # cache written to disk - r2 = runner.invoke(scan, [str(proj), "--cache-dir", str(cache), "--output", str(out2)]) + r2 = runner.invoke(scan, [str(proj), "--cache-dir", str(cache), "--output", str(out2)], env=env) assert r2.exit_code == 0, r2.output def _parse(p: Path) -> list[dict]: diff --git a/tests/unit/scanner/taint/test_provenance_clash.py b/tests/unit/scanner/taint/test_provenance_clash.py index 5ecbff17..c5539d52 100644 --- a/tests/unit/scanner/taint/test_provenance_clash.py +++ b/tests/unit/scanner/taint/test_provenance_clash.py @@ -9,7 +9,7 @@ from wardline.core.taints import TaintState as T from wardline.scanner.taint.propagation import propagate_callgraph_taints from wardline.scanner.taint.summary import SUMMARY_SCHEMA_VERSION -from wardline.scanner.taint.summary_cache import _deserialise_summary, _serialise_summary +from wardline.scanner.taint.summary_cache import SummaryCache, _deserialise_summary, _serialise_summary from wardline.scanner.taint.variable_level import compute_variable_taints @@ -169,7 +169,7 @@ def test_summary_cache_mixed_raw_clash() -> None: _PROVENANCE_CLASH.reset(token) -def test_run_scan_provenance_clash_with_cache(tmp_path) -> None: +def test_run_scan_provenance_clash_with_cache(tmp_path, monkeypatch) -> None: from wardline.core.run import run_scan proj = tmp_path / "proj" @@ -196,6 +196,7 @@ def clashing(cond, p): (proj / "m.py").write_text(source, encoding="utf-8") cache = tmp_path / "cache" + monkeypatch.setenv("WARDLINE_SUMMARY_CACHE_KEY", "unit-test-summary-cache-key") run_scan(proj, cache_dir=cache) assert cache.exists() @@ -206,15 +207,12 @@ def clashing(cond, p): assert hit_rate > 0.0, "Cache was not hit! MIXED_RAW cache entry might have been dropped." -def test_run_scan_provenance_clash_loads_mixed_raw_cache(tmp_path) -> None: - import json - +def test_run_scan_provenance_clash_loads_signed_mixed_raw_cache(tmp_path, monkeypatch) -> None: from wardline.core.run import run_scan from wardline.core.taints import TaintState as T from wardline.scanner.taint.decorator_provider import DecoratorTaintSourceProvider from wardline.scanner.taint.project_resolver import _RESOLVER_VERSION, compute_cache_key from wardline.scanner.taint.summary import SUMMARY_SCHEMA_VERSION, FunctionSummary - from wardline.scanner.taint.summary_cache import _serialise_summary proj = tmp_path / "proj" proj.mkdir() @@ -252,7 +250,11 @@ def test_run_scan_provenance_clash_loads_mixed_raw_cache(tmp_path) -> None: cache_key=key, ) - (cache_dir / f"{key}.json").write_text(json.dumps([_serialise_summary(s)]), encoding="utf-8") + secret = b"unit-test-summary-cache-key" + cache = SummaryCache(cache_dir=cache_dir, cache_auth_secret=secret) + cache.put(key, (s,)) + cache.save() + monkeypatch.setenv("WARDLINE_SUMMARY_CACHE_KEY", secret.decode("utf-8")) res = run_scan(proj, cache_dir=cache_dir) diff --git a/tests/unit/scanner/taint/test_summary_cache.py b/tests/unit/scanner/taint/test_summary_cache.py index af54e8f2..27de6ca4 100644 --- a/tests/unit/scanner/taint/test_summary_cache.py +++ b/tests/unit/scanner/taint/test_summary_cache.py @@ -8,12 +8,15 @@ from wardline.scanner.taint.summary import SUMMARY_SCHEMA_VERSION, FunctionSummary from wardline.scanner.taint.summary_cache import ( SummaryCache, + _cache_payload, + _cache_payload_mac, _deserialise_summary, _serialise_summary, ) _KEY = "a" * 64 _KEY2 = "b" * 64 +_SECRET = b"unit-test-summary-cache-key" def _summary(fqn: str, *, schema: int = SUMMARY_SCHEMA_VERSION, key: str = _KEY) -> FunctionSummary: @@ -121,15 +124,49 @@ def test_schema_version_property() -> None: def test_save_and_load_roundtrip(tmp_path) -> None: - c = SummaryCache(cache_dir=tmp_path) + c = SummaryCache(cache_dir=tmp_path, cache_auth_secret=_SECRET) summaries = (_summary("m.a"), _summary("m.b")) c.put(_KEY, summaries) c.save() - c2 = SummaryCache(cache_dir=tmp_path) + c2 = SummaryCache(cache_dir=tmp_path, cache_auth_secret=_SECRET) c2.load() assert c2.get(_KEY) == summaries +def test_save_without_auth_secret_does_not_persist_entries(tmp_path) -> None: + c = SummaryCache(cache_dir=tmp_path) + c.put(_KEY, (_summary("m.a"),)) + c.save() + assert list(tmp_path.iterdir()) == [] + + +def test_load_drops_unsigned_cache_file_with_matching_key(tmp_path) -> None: + import json + + payload = [_serialise_summary(_summary("m.a"))] + (tmp_path / f"{_KEY}.json").write_text(json.dumps(payload), encoding="utf-8") + + c = SummaryCache(cache_dir=tmp_path, cache_auth_secret=_SECRET) + c.load() + + assert len(c) == 0 + assert c.get(_KEY) is None + + +def test_load_drops_bad_mac_cache_file(tmp_path) -> None: + import json + + payload = _cache_payload(_KEY, (_summary("m.a"),)) + payload["mac"] = "0" * 64 + (tmp_path / f"{_KEY}.json").write_text(json.dumps(payload), encoding="utf-8") + + c = SummaryCache(cache_dir=tmp_path, cache_auth_secret=_SECRET) + c.load() + + assert len(c) == 0 + assert c.get(_KEY) is None + + def test_warm_cache_honours_untrusted_sources_policy_change(tmp_path) -> None: # A warm run whose config newly names a function as an untrusted source must produce the # SAME defects as a cold run with that config — the cache key binds the effective-scan- @@ -164,13 +201,70 @@ def defects(az: WardlineAnalyzer, cfg: WardlineConfig) -> list[str]: assert defects(az, cfg_src) == cold # must NOT serve the stale-clean summary +def test_run_scan_ignores_unsigned_forged_summary_cache(tmp_path) -> None: + import json + + from wardline.core.config import WardlineConfig + from wardline.core.ruleset import ruleset_hash + from wardline.core.run import run_scan + from wardline.scanner.taint.decorator_provider import DecoratorTaintSourceProvider + from wardline.scanner.taint.project_resolver import _RESOLVER_VERSION + from wardline.scanner.taint.summary import compute_cache_key + + proj = tmp_path / "proj" + proj.mkdir() + source = ( + "from wardline.decorators import external_boundary, trusted\n" + "@external_boundary\n" + "def read_raw(p):\n" + " return p\n" + "@trusted(level='ASSURED')\n" + "def sink(x):\n" + " return x\n" + "def f(p):\n" + " return sink(read_raw(p))\n" + ) + (proj / "m.py").write_text(source, encoding="utf-8") + + cache_dir = tmp_path / "cache" + cache_dir.mkdir() + key = compute_cache_key( + module_path="m", + source_bytes=source.encode("utf-8"), + schema_version=SUMMARY_SCHEMA_VERSION, + resolver_version=_RESOLVER_VERSION, + provider_fingerprint=DecoratorTaintSourceProvider().fingerprint(), + scan_policy_hash=ruleset_hash(WardlineConfig()), + ) + forged = tuple( + FunctionSummary( + fqn=fqn, + body_taint=T.INTEGRAL, + return_taint=T.INTEGRAL, + taint_source="anchored", + unresolved_calls=0, + schema_version=SUMMARY_SCHEMA_VERSION, + cache_key=key, + ) + for fqn in ("m.read_raw", "m.sink", "m.f") + ) + (cache_dir / f"{key}.json").write_text(json.dumps([_serialise_summary(s) for s in forged]), encoding="utf-8") + + result = run_scan(proj, cache_dir=cache_dir) + metrics = next(f for f in result.findings if f.rule_id == "WLN-ENGINE-METRICS") + + assert metrics.properties["cache_hit_rate"] == 0.0 + assert any(f.rule_id == "PY-WL-101" for f in result.findings) + + def test_load_drops_file_when_internal_cache_key_mismatches_filename(tmp_path) -> None: import json - payload = [_serialise_summary(_summary("m.a", key=_KEY2))] + payload = _cache_payload(_KEY, (_summary("m.a", key=_KEY2),)) + payload["mac"] = _cache_payload_mac(_SECRET, payload) (tmp_path / f"{_KEY}.json").write_text(json.dumps(payload), encoding="utf-8") - c = SummaryCache(cache_dir=tmp_path) + c = SummaryCache(cache_dir=tmp_path, cache_auth_secret=_SECRET) c.load() assert len(c) == 0 @@ -180,13 +274,17 @@ def test_load_drops_file_when_internal_cache_key_mismatches_filename(tmp_path) - def test_load_drops_file_when_records_mix_cache_keys(tmp_path) -> None: import json - payload = [ - _serialise_summary(_summary("m.a", key=_KEY)), - _serialise_summary(_summary("m.b", key=_KEY2)), - ] + payload = _cache_payload( + _KEY, + ( + _summary("m.a", key=_KEY), + _summary("m.b", key=_KEY2), + ), + ) + payload["mac"] = _cache_payload_mac(_SECRET, payload) (tmp_path / f"{_KEY}.json").write_text(json.dumps(payload), encoding="utf-8") - c = SummaryCache(cache_dir=tmp_path) + c = SummaryCache(cache_dir=tmp_path, cache_auth_secret=_SECRET) c.load() assert len(c) == 0 @@ -195,14 +293,14 @@ def test_load_drops_file_when_records_mix_cache_keys(tmp_path) -> None: def test_load_drops_malformed_json(tmp_path) -> None: (tmp_path / f"{_KEY}.json").write_text("{not json", encoding="utf-8") - c = SummaryCache(cache_dir=tmp_path) + c = SummaryCache(cache_dir=tmp_path, cache_auth_secret=_SECRET) c.load() # must not raise assert len(c) == 0 def test_load_ignores_non_hex_stem_files(tmp_path) -> None: (tmp_path / "notes.json").write_text("[]", encoding="utf-8") - c = SummaryCache(cache_dir=tmp_path) + c = SummaryCache(cache_dir=tmp_path, cache_auth_secret=_SECRET) c.load() assert len(c) == 0 @@ -273,7 +371,7 @@ def test_deserialise_integral_roundtrips() -> None: def test_integral_survives_full_save_load_cycle(tmp_path) -> None: # End-to-end: a poisoned-but-valid trio state would be dropped by load(), # but a legitimate INTEGRAL summary must survive the disk round-trip. - c = SummaryCache(cache_dir=tmp_path) + c = SummaryCache(cache_dir=tmp_path, cache_auth_secret=_SECRET) s = FunctionSummary( fqn="m.f", body_taint=T.INTEGRAL, @@ -285,7 +383,7 @@ def test_integral_survives_full_save_load_cycle(tmp_path) -> None: ) c.put(_KEY, (s,)) c.save() - c2 = SummaryCache(cache_dir=tmp_path) + c2 = SummaryCache(cache_dir=tmp_path, cache_auth_secret=_SECRET) c2.load() assert c2.get(_KEY) == (s,) @@ -295,8 +393,14 @@ def test_load_drops_poisoned_trio_cache_file(tmp_path, caplog) -> None: # dropped (cold-cache fallback), not injected — load() catches the ValueError. import json - (tmp_path / f"{_KEY}.json").write_text(json.dumps([_summary_dict("MIXED_RAW", "MIXED_RAW")]), encoding="utf-8") - c = SummaryCache(cache_dir=tmp_path) + payload: dict[str, object] = { + "schema_version": 1, + "cache_key": _KEY, + "summaries": [_summary_dict("MIXED_RAW", "MIXED_RAW")], + } + payload["mac"] = _cache_payload_mac(_SECRET, payload) + (tmp_path / f"{_KEY}.json").write_text(json.dumps(payload), encoding="utf-8") + c = SummaryCache(cache_dir=tmp_path, cache_auth_secret=_SECRET) c.load() # must not raise assert len(c) == 0 @@ -309,7 +413,7 @@ def test_save_cleans_up_temp_file_when_replace_fails(tmp_path, monkeypatch) -> N # and the error re-raised — the except cleanup arm. import os - c = SummaryCache(cache_dir=tmp_path) + c = SummaryCache(cache_dir=tmp_path, cache_auth_secret=_SECRET) c.put(_KEY, (_summary("m.a"),)) def boom(_src, _dst): @@ -325,7 +429,7 @@ def boom(_src, _dst): def test_load_skips_non_json_files(tmp_path) -> None: # A non-.json file in the cache dir must be skipped (the suffix guard), not parsed. (tmp_path / f"{_KEY}.txt").write_text("garbage", encoding="utf-8") - c = SummaryCache(cache_dir=tmp_path) + c = SummaryCache(cache_dir=tmp_path, cache_auth_secret=_SECRET) c.load() # must not raise assert len(c) == 0 @@ -339,9 +443,10 @@ def test_load_drops_stale_schema_entry_from_disk(tmp_path) -> None: stale = _summary("m.a") object.__setattr__(stale, "schema_version", SUMMARY_SCHEMA_VERSION + 1) - payload = [_serialise_summary(stale)] + payload = _cache_payload(_KEY, (stale,)) + payload["mac"] = _cache_payload_mac(_SECRET, payload) (tmp_path / f"{_KEY}.json").write_text(json.dumps(payload), encoding="utf-8") - c = SummaryCache(cache_dir=tmp_path) + c = SummaryCache(cache_dir=tmp_path, cache_auth_secret=_SECRET) c.load() assert len(c) == 0 # stale-schema file dropped (not rehydrated) From 29a7c64ae19dab59942b339eff2ccf8f97845d46 Mon Sep 17 00:00:00 2001 From: John Morrissey Date: Fri, 12 Jun 2026 07:19:40 +1000 Subject: [PATCH 132/186] test(cli): reject forged summary cache entries --- tests/unit/cli/test_cli.py | 64 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 64 insertions(+) diff --git a/tests/unit/cli/test_cli.py b/tests/unit/cli/test_cli.py index 3edd1200..0e6958d7 100644 --- a/tests/unit/cli/test_cli.py +++ b/tests/unit/cli/test_cli.py @@ -426,6 +426,70 @@ def _hit_rate(fs: list[dict]) -> float: assert _hit_rate(f2) > 0.0 +def test_scan_cache_dir_ignores_unsigned_forged_cache(tmp_path: Path) -> None: + import json + + from wardline.core.config import WardlineConfig + from wardline.core.ruleset import ruleset_hash + from wardline.core.taints import TaintState as T + from wardline.scanner.taint.decorator_provider import DecoratorTaintSourceProvider + from wardline.scanner.taint.project_resolver import _RESOLVER_VERSION + from wardline.scanner.taint.summary import SUMMARY_SCHEMA_VERSION, FunctionSummary, compute_cache_key + from wardline.scanner.taint.summary_cache import _serialise_summary + + proj = tmp_path / "proj" + proj.mkdir() + source = ( + "from wardline.decorators import external_boundary, trusted\n" + "@external_boundary\n" + "def read_raw(p):\n" + " return p\n" + "@trusted(level='ASSURED')\n" + "def sink(x):\n" + " return x\n" + "def f(p):\n" + " return sink(read_raw(p))\n" + ) + (proj / "m.py").write_text(source, encoding="utf-8") + + cache = tmp_path / "repo-controlled-cache" + cache.mkdir() + key = compute_cache_key( + module_path="m", + source_bytes=source.encode("utf-8"), + schema_version=SUMMARY_SCHEMA_VERSION, + resolver_version=_RESOLVER_VERSION, + provider_fingerprint=DecoratorTaintSourceProvider().fingerprint(), + scan_policy_hash=ruleset_hash(WardlineConfig()), + ) + forged = tuple( + FunctionSummary( + fqn=fqn, + body_taint=T.INTEGRAL, + return_taint=T.INTEGRAL, + taint_source="anchored", + unresolved_calls=0, + schema_version=SUMMARY_SCHEMA_VERSION, + cache_key=key, + ) + for fqn in ("m.read_raw", "m.sink", "m.f") + ) + (cache / f"{key}.json").write_text(json.dumps([_serialise_summary(s) for s in forged]), encoding="utf-8") + + out = tmp_path / "findings.jsonl" + result = CliRunner().invoke( + scan, + [str(proj), "--cache-dir", str(cache), "--output", str(out)], + env={"WARDLINE_SUMMARY_CACHE_KEY": "unit-test-summary-cache-key"}, + ) + + assert result.exit_code == 0, result.output + findings = [_json.loads(line) for line in out.read_text().splitlines() if line.strip()] + metrics = next(f for f in findings if f["rule_id"] == "WLN-ENGINE-METRICS") + assert metrics["properties"]["cache_hit_rate"] == 0.0 + assert any(f["rule_id"] == "PY-WL-101" for f in findings) + + def _write(project, name, src): p = project / name p.write_text(src, encoding="utf-8") From e8e1515d304c43c99c03412d8bcc48f6150eb92f Mon Sep 17 00:00:00 2001 From: John Morrissey Date: Fri, 12 Jun 2026 07:21:50 +1000 Subject: [PATCH 133/186] fix(cli): refuse symlinked jsonl output --- src/wardline/cli/scan.py | 2 +- src/wardline/core/emit.py | 11 +++++------ tests/unit/cli/test_cli.py | 17 +++++++++++++++++ tests/unit/core/test_emit.py | 28 ++++++++++++++++++++++++++++ 4 files changed, 51 insertions(+), 7 deletions(-) diff --git a/src/wardline/cli/scan.py b/src/wardline/cli/scan.py index d131a13a..14ceaac6 100644 --- a/src/wardline/cli/scan.py +++ b/src/wardline/cli/scan.py @@ -251,7 +251,7 @@ def confirm_cb(rel_path: str, orig: str, replacement: str, f: Finding) -> bool: sarif_sink = SarifSink(output, root=path if output_is_default else None) sarif_sink.write(findings, result.context) elif fmt == "jsonl": - jsonl_sink = JsonlSink(output) + jsonl_sink = JsonlSink(output, root=path if output_is_default else None) jsonl_sink.write(findings) elif fmt == "legis": # The signed, verbatim-postable scan for legis's POST /wardline/scan-results. diff --git a/src/wardline/core/emit.py b/src/wardline/core/emit.py index d6a8a709..3ec14a02 100644 --- a/src/wardline/core/emit.py +++ b/src/wardline/core/emit.py @@ -8,6 +8,7 @@ from typing import Protocol from wardline.core.finding import Finding +from wardline.core.safe_paths import safe_write_text class Sink(Protocol): @@ -15,12 +16,10 @@ def write(self, findings: Sequence[Finding]) -> None: ... class JsonlSink: - def __init__(self, path: Path) -> None: + def __init__(self, path: Path, *, root: Path | None = None) -> None: self._path = path + self._root = root def write(self, findings: Sequence[Finding]) -> None: - self._path.parent.mkdir(parents=True, exist_ok=True) - with self._path.open("w", encoding="utf-8") as handle: - for finding in findings: - handle.write(finding.to_jsonl()) - handle.write("\n") + content = "".join(f"{finding.to_jsonl()}\n" for finding in findings) + safe_write_text(self._root or self._path.parent, self._path, content, label=self._path.name) diff --git a/tests/unit/cli/test_cli.py b/tests/unit/cli/test_cli.py index 0e6958d7..5aba7588 100644 --- a/tests/unit/cli/test_cli.py +++ b/tests/unit/cli/test_cli.py @@ -110,6 +110,23 @@ def test_scan_default_output_lands_in_scanned_path(tmp_path: Path) -> None: assert (project / "findings.jsonl").exists() +def test_scan_jsonl_default_refuses_symlinked_output(tmp_path: Path) -> None: + import shutil + + project = tmp_path / "proj" + shutil.copytree(FIXTURE, project) + outside = tmp_path / "outside.jsonl" + outside.write_text("keep\n", encoding="utf-8") + (project / "findings.jsonl").unlink(missing_ok=True) + (project / "findings.jsonl").symlink_to(outside) + + result = CliRunner().invoke(cli, ["scan", str(project)]) + + assert result.exit_code == 2 + assert "findings.jsonl: refusing to write through a symlink" in result.output + assert outside.read_text(encoding="utf-8") == "keep\n" + + def _git(repo: Path, *args: str) -> None: import subprocess diff --git a/tests/unit/core/test_emit.py b/tests/unit/core/test_emit.py index 358f3923..7bbbf43d 100644 --- a/tests/unit/core/test_emit.py +++ b/tests/unit/core/test_emit.py @@ -1,7 +1,10 @@ import json from pathlib import Path +import pytest + from wardline.core.emit import JsonlSink +from wardline.core.errors import WardlineError from wardline.core.finding import Finding, Kind, Location, Severity @@ -29,3 +32,28 @@ def test_jsonl_sink_writes_empty_file_for_no_findings(tmp_path: Path) -> None: JsonlSink(out).write([]) assert out.exists() assert out.read_text(encoding="utf-8") == "" + + +def test_jsonl_sink_refuses_symlink_target(tmp_path: Path) -> None: + outside = tmp_path / "outside.jsonl" + outside.write_text("keep\n", encoding="utf-8") + out = tmp_path / "findings.jsonl" + out.symlink_to(outside) + + with pytest.raises(WardlineError, match="refusing to write through a symlink"): + JsonlSink(out).write([_finding()]) + + assert outside.read_text(encoding="utf-8") == "keep\n" + + +def test_jsonl_sink_with_root_refuses_parent_symlink_escape(tmp_path: Path) -> None: + root = tmp_path / "project" + root.mkdir() + outside = tmp_path / "outside" + outside.mkdir() + (root / "reports").symlink_to(outside, target_is_directory=True) + + with pytest.raises(WardlineError, match="escapes project root"): + JsonlSink(root / "reports" / "findings.jsonl", root=root).write([_finding()]) + + assert not (outside / "findings.jsonl").exists() From 955a916d8011f9f3720a0a2205b08fa2b1a75978 Mon Sep 17 00:00:00 2001 From: John Morrissey Date: Fri, 12 Jun 2026 07:23:14 +1000 Subject: [PATCH 134/186] fix(scanner): remove fail-open noop analyzer --- src/wardline/scanner/__init__.py | 17 ++--------------- tests/unit/scanner/test_noop.py | 13 ++++++------- 2 files changed, 8 insertions(+), 22 deletions(-) diff --git a/src/wardline/scanner/__init__.py b/src/wardline/scanner/__init__.py index 7ca0f96a..b6c10d7b 100644 --- a/src/wardline/scanner/__init__.py +++ b/src/wardline/scanner/__init__.py @@ -1,20 +1,7 @@ -"""Wardline analysis engine. SP0 ships a no-op; SP1 replaces it.""" +"""Wardline analysis engine public API.""" from __future__ import annotations -from collections.abc import Sequence -from pathlib import Path - -from wardline.core.config import WardlineConfig -from wardline.core.finding import Finding from wardline.scanner.analyzer import WardlineAnalyzer - -class NoOpAnalyzer: - """Placeholder analyzer that performs no analysis (SP0).""" - - def analyze(self, files: Sequence[Path], config: WardlineConfig, *, root: Path) -> Sequence[Finding]: - return [] - - -__all__ = ["NoOpAnalyzer", "WardlineAnalyzer"] +__all__ = ["WardlineAnalyzer"] diff --git a/tests/unit/scanner/test_noop.py b/tests/unit/scanner/test_noop.py index 5c0b8762..1a79acf3 100644 --- a/tests/unit/scanner/test_noop.py +++ b/tests/unit/scanner/test_noop.py @@ -1,9 +1,8 @@ -from pathlib import Path +import wardline.scanner as scanner +from wardline.scanner.analyzer import WardlineAnalyzer -from wardline.core.config import WardlineConfig -from wardline.scanner import NoOpAnalyzer - -def test_noop_analyzer_returns_no_findings() -> None: - result = NoOpAnalyzer().analyze([Path("a.py")], WardlineConfig(), root=Path(".")) - assert list(result) == [] +def test_noop_analyzer_is_not_exported() -> None: + assert scanner.__all__ == ["WardlineAnalyzer"] + assert not hasattr(scanner, "NoOpAnalyzer") + assert scanner.WardlineAnalyzer is WardlineAnalyzer From 687e498e6a3c83b7ab0689489db1fd1f0efaa903 Mon Sep 17 00:00:00 2001 From: John Morrissey Date: Fri, 12 Jun 2026 07:24:34 +1000 Subject: [PATCH 135/186] fix(gitignore): retain legacy clarion runtime ignore --- .gitignore | 1 + 1 file changed, 1 insertion(+) diff --git a/.gitignore b/.gitignore index 9b38f3be..624761d1 100644 --- a/.gitignore +++ b/.gitignore @@ -19,6 +19,7 @@ findings.jsonl # Sibling legacy locations (transition window — .weft// is preferred but # the old dot-dirs may still be present until siblings finish migrating). .loomweave +.clarion # NOTE: do NOT ignore .weft/ — wardline's own .weft/wardline/{baseline,judged,waivers}.yaml # are deliberately committed; weft.toml is operator-authored and tracked. From 4c40c0acc456dc01cb83d3b3a14ff4e196a2ae40 Mon Sep 17 00:00:00 2001 From: John Morrissey Date: Fri, 12 Jun 2026 07:26:06 +1000 Subject: [PATCH 136/186] fix(coverage): guard decorator unparse recursion --- src/wardline/core/decorator_coverage.py | 8 +++++++- tests/unit/core/test_decorator_coverage.py | 20 +++++++++++++++++++- 2 files changed, 26 insertions(+), 2 deletions(-) diff --git a/src/wardline/core/decorator_coverage.py b/src/wardline/core/decorator_coverage.py index eded7f75..ce4d7744 100644 --- a/src/wardline/core/decorator_coverage.py +++ b/src/wardline/core/decorator_coverage.py @@ -139,7 +139,13 @@ def _locator_for(qualname: str) -> str: def _decorators_of(entity: Entity) -> list[str]: - return [f"@{ast.unparse(dec)}" for dec in entity.node.decorator_list] + decorators: list[str] = [] + for dec in entity.node.decorator_list: + try: + decorators.append(f"@{ast.unparse(dec)}") + except RecursionError: + decorators.append(f"@") + return decorators def _identity_for(provider: BindingProvider | None, qualname: str) -> tuple[IdentityCoverage, EntityBinding | None]: diff --git a/tests/unit/core/test_decorator_coverage.py b/tests/unit/core/test_decorator_coverage.py index bacee14a..92adf15c 100644 --- a/tests/unit/core/test_decorator_coverage.py +++ b/tests/unit/core/test_decorator_coverage.py @@ -3,7 +3,7 @@ from pathlib import Path from wardline.core.baseline import write_baseline -from wardline.core.decorator_coverage import build_decorator_coverage +from wardline.core.decorator_coverage import build_decorator_coverage, decorator_coverage_from_scan from wardline.core.dossier import TicketRef, WorkSection from wardline.core.identity import ContentStatus, EntityBinding, IdentityStatus from wardline.core.paths import baseline_path @@ -113,3 +113,21 @@ def test_decorator_coverage_surfaces_suppressed_defects(tmp_path: Path) -> None: assert rows["svc.leaky"].finding_state == "suppressed" assert rows["svc.leaky"].active_finding_fingerprints == [] assert rows["svc.leaky"].suppressed_finding_fingerprints == [leak.fingerprint] + + +def test_decorator_coverage_degrades_when_decorator_unparse_recurses( + tmp_path: Path, + monkeypatch, +) -> None: + root = _project(tmp_path) + result = run_scan(root) + assert result.context is not None + + def raise_recursion(_node) -> str: + raise RecursionError("simulated decorator depth") + + monkeypatch.setattr("wardline.core.decorator_coverage.ast.unparse", raise_recursion) + + rows = {row.qualname: row for row in decorator_coverage_from_scan(result, result.context).rows} + + assert rows["svc.clean"].decorators == ["@"] From f5b270ca16bac5918abb268108a9ca7897f45866 Mon Sep 17 00:00:00 2001 From: John Morrissey Date: Fri, 12 Jun 2026 07:28:00 +1000 Subject: [PATCH 137/186] fix(cli): redact taint-store urls in logs --- src/wardline/cli/scan.py | 26 ++++++++++++++++++++++++-- tests/unit/cli/test_cli.py | 22 ++++++++++++++++++++++ 2 files changed, 46 insertions(+), 2 deletions(-) diff --git a/src/wardline/cli/scan.py b/src/wardline/cli/scan.py index 14ceaac6..346f6ea0 100644 --- a/src/wardline/cli/scan.py +++ b/src/wardline/cli/scan.py @@ -4,6 +4,7 @@ from __future__ import annotations import json +import urllib.parse from pathlib import Path import click @@ -395,14 +396,15 @@ def confirm_cb(rel_path: str, orig: str, replacement: str, f: Finding) -> bool: line += f"; {len(emit_result.warnings)} warning(s): " + "; ".join(emit_result.warnings) click.echo(line) if loomweave_result is not None: + logged_loomweave_url = _redact_url_for_log(loomweave_url) if not loomweave_result.reachable: reason = loomweave_result.disabled_reason or "unreachable" click.echo( - f"warning: Loomweave taint store not written at {loomweave_url} ({reason}); scan unaffected.", + f"warning: Loomweave taint store not written at {logged_loomweave_url} ({reason}); scan unaffected.", err=True, ) else: - line = f"wrote {loomweave_result.written} taint fact(s) to {loomweave_url}" + line = f"wrote {loomweave_result.written} taint fact(s) to {logged_loomweave_url}" if loomweave_result.unresolved_qualnames: line += ( f"; {len(loomweave_result.unresolved_qualnames)} qualname(s) unresolved (not indexed by Loomweave)" @@ -531,3 +533,23 @@ def _loomweave_status(result: object | None) -> dict[str, object]: "unresolved_qualnames": list(getattr(result, "unresolved_qualnames", ())), "disabled_reason": getattr(result, "disabled_reason", None), } + + +def _redact_url_for_log(url: str | None) -> str: + if url is None: + return "" + parts = urllib.parse.urlsplit(url) + if not parts.scheme or not parts.netloc: + return url.split("?", 1)[0].split("#", 1)[0] + try: + host = parts.hostname or "" + port = parts.port + except ValueError: + return f"{parts.scheme}://" + if ":" in host and not host.startswith("["): + host = f"[{host}]" + if port is not None: + host = f"{host}:{port}" + if parts.username is not None or parts.password is not None: + host = f"@{host}" + return urllib.parse.urlunsplit((parts.scheme, host, parts.path, "", "")) diff --git a/tests/unit/cli/test_cli.py b/tests/unit/cli/test_cli.py index 5aba7588..0ace9e56 100644 --- a/tests/unit/cli/test_cli.py +++ b/tests/unit/cli/test_cli.py @@ -999,6 +999,28 @@ def test_scan_loomweave_soft_outage_does_not_change_exit(tmp_path, monkeypatch) assert "scan unaffected" in result.output +def test_scan_loomweave_soft_outage_redacts_url_secrets(tmp_path, monkeypatch) -> None: + from wardline.loomweave.client import WriteResult + + proj = tmp_path / "proj" + proj.mkdir() + _write(proj, "svc.py", _LEAKY) + monkeypatch.setattr( + "wardline.loomweave.write.write_facts_to_loomweave", + lambda *a, **k: WriteResult(reachable=False, disabled_reason="connection refused"), + ) + out = tmp_path / "f.jsonl" + secret_url = "https://user:secret@loomweave.example/api/taint?token=abc#frag" + + result = CliRunner().invoke(scan, [str(proj), "--output", str(out), "--loomweave-url", secret_url]) + + assert result.exit_code == 0, result.output + assert "https://@loomweave.example/api/taint" in result.output + assert "user:secret" not in result.output + assert "token=abc" not in result.output + assert "#frag" not in result.output + + def test_scan_reports_filigree_success_and_loomweave_unreachable_independently(tmp_path, monkeypatch) -> None: from wardline.loomweave.client import WriteResult From 9a62efdd89e23d90b5a4ca50a0500ddc1f44defa Mon Sep 17 00:00:00 2001 From: John Morrissey Date: Fri, 12 Jun 2026 07:29:52 +1000 Subject: [PATCH 138/186] fix(cli): escape assure human output --- src/wardline/cli/assure.py | 13 +++++++---- tests/unit/cli/test_assure_cmd.py | 36 ++++++++++++++++++++++++++++++- 2 files changed, 44 insertions(+), 5 deletions(-) diff --git a/src/wardline/cli/assure.py b/src/wardline/cli/assure.py index b7b866a8..2e76b4b6 100644 --- a/src/wardline/cli/assure.py +++ b/src/wardline/cli/assure.py @@ -84,14 +84,19 @@ def _render_human(posture: object) -> None: click.echo(" Unknown boundaries:") for u in unknown_list: loc = u.get("location") or {} - loc_str = f" {loc.get('path', '?')}:{loc.get('line', '?')}" - reason_str = f" [{u['reason']}]" if u.get("reason") else "" - tier_str = f" (tier: {u['tier']})" if u.get("tier") else "" - click.echo(f" {u['qualname']}{tier_str}{loc_str}{reason_str}") + loc_str = f" {_human_text(loc.get('path', '?'))}:{_human_text(loc.get('line', '?'))}" + reason_str = f" [{_human_text(u['reason'])}]" if u.get("reason") else "" + tier_str = f" (tier: {_human_text(u['tier'])})" if u.get("tier") else "" + click.echo(f" {_human_text(u['qualname'])}{tier_str}{loc_str}{reason_str}") _render_waiver_debt(waiver_debt) +def _human_text(value: object) -> str: + """Escape control characters before writing repository-derived text to terminals.""" + return ascii(str(value))[1:-1] + + def _render_waiver_debt(waiver_debt: list[dict[str, Any]]) -> None: """Render the waiver-debt summary line.""" if not waiver_debt: diff --git a/tests/unit/cli/test_assure_cmd.py b/tests/unit/cli/test_assure_cmd.py index 2d9db2c3..39ba7855 100644 --- a/tests/unit/cli/test_assure_cmd.py +++ b/tests/unit/cli/test_assure_cmd.py @@ -15,8 +15,9 @@ from click.testing import CliRunner +from wardline.cli.assure import _render_human from wardline.cli.main import cli -from wardline.core.assure import build_posture +from wardline.core.assure import AssurancePosture, UnknownBoundary, build_posture # Identical decorated module used by test_assure.py so the engine produces # boundaries_total >= 1 (two @trusted producers + one @external_boundary = 3). @@ -89,6 +90,39 @@ def test_human_format_unanalyzed_files_are_not_full_coverage(tmp_path: Path) -> assert "unanalyzed files: 1" in result.output +def test_human_format_escapes_repository_control_chars(capsys) -> None: + posture = AssurancePosture( + boundaries_total=1, + proven=0, + defect_total=0, + unknown=[ + UnknownBoundary( + qualname="svc.\x1b]52;clipboard", + tier="ASSURED", + path="evil\nname.py", + line=7, + reason="\x1b[31mrecursion", + ) + ], + engine_limited=1, + coverage_pct=0.0, + unanalyzed_total=0, + unanalyzed_rule_ids=[], + waiver_debt=[], + baselined_total=0, + judged_total=0, + ) + + _render_human(posture) + out = capsys.readouterr().out + + assert "\x1b" not in out + assert "evil\nname.py" not in out + assert r"svc.\x1b]52;clipboard" in out + assert r"evil\nname.py" in out + assert r"\x1b[31mrecursion" in out + + def test_human_lapsed_waiver_wording(tmp_path: Path) -> None: """A lapsed waiver must say "expired N day(s) ago", not "-N day(s) until earliest expiry".""" # Write a decorated module so the posture is non-empty, then invoke via JSON From 060ae2300170aa60b733ff548581c617831219a3 Mon Sep 17 00:00:00 2001 From: John Morrissey Date: Fri, 12 Jun 2026 07:34:09 +1000 Subject: [PATCH 139/186] fix(loomweave): harden linkage total parsing --- src/wardline/loomweave/client.py | 16 +++++++-- tests/unit/cli/test_mcp_cli.py | 2 +- tests/unit/loomweave/test_client_linkages.py | 34 ++++++++++++++++++++ tests/unit/test_makefile_clean.py | 1 + 4 files changed, 50 insertions(+), 3 deletions(-) diff --git a/src/wardline/loomweave/client.py b/src/wardline/loomweave/client.py index bc71ed54..0f3cb10f 100644 --- a/src/wardline/loomweave/client.py +++ b/src/wardline/loomweave/client.py @@ -15,6 +15,7 @@ import json import logging +import math import secrets import urllib.error import urllib.parse @@ -138,6 +139,17 @@ def _error_code(body: str) -> str | None: return parsed.get("code") if isinstance(parsed, dict) else None +def _linkage_total(raw: object, fallback: int) -> int: + if isinstance(raw, bool): + return fallback + if isinstance(raw, int): + return raw if raw >= 0 else fallback + if isinstance(raw, float) and math.isfinite(raw): + total = int(raw) + return total if total >= 0 else fallback + return fallback + + class LoomweaveClient: def __init__( self, @@ -426,8 +438,8 @@ def _get_linkages(self, entity_id: str, direction: str, limit: int) -> LinkageRe total = data.get("total") return LinkageResult( neighbours=neighbours, - # accept int or a JSON float; any other shape → fall back to the count read - total=int(total) if isinstance(total, (int, float)) and not isinstance(total, bool) else len(neighbours), + # accept finite JSON numbers; malformed totals fall back to the count read + total=_linkage_total(total, len(neighbours)), truncated=bool(data.get("truncated", False)), ) diff --git a/tests/unit/cli/test_mcp_cli.py b/tests/unit/cli/test_mcp_cli.py index c2ce6989..04fbf50e 100644 --- a/tests/unit/cli/test_mcp_cli.py +++ b/tests/unit/cli/test_mcp_cli.py @@ -48,7 +48,7 @@ def test_mcp_command_passes_policy_flags(tmp_path, monkeypatch) -> None: import wardline.cli.mcp as mcp_cli from wardline.cli.main import cli - captured = {} + captured: dict[str, object] = {} class FakeRpc: def run_stdio(self) -> None: diff --git a/tests/unit/loomweave/test_client_linkages.py b/tests/unit/loomweave/test_client_linkages.py index b7bdeb80..311d645e 100644 --- a/tests/unit/loomweave/test_client_linkages.py +++ b/tests/unit/loomweave/test_client_linkages.py @@ -72,6 +72,40 @@ def test_get_callees_reads_the_callees_field(): assert res.truncated is True +def test_linkage_non_finite_large_float_total_falls_back_to_neighbour_count(): + body = ( + '{"entity_id":"x","callers":[' + '{"entity_id":"python:function:svc.caller","confidence":"resolved","call_site_count":1}' + '],"total":1e999999,"truncated":false}' + ) + res = _client(FakeTransport([Response(status=200, body=body)])).get_callers("x") + assert res is not None + assert res.neighbours == ("python:function:svc.caller",) + assert res.total == 1 + + +def test_linkage_nan_total_falls_back_to_neighbour_count(): + body = '{"entity_id":"x","callees":[],"total":NaN,"truncated":false}' + res = _client(FakeTransport([Response(status=200, body=body)])).get_callees("x") + assert res is not None + assert res.neighbours == () + assert res.total == 0 + + +def test_linkage_negative_total_falls_back_to_neighbour_count(): + body = json.dumps( + { + "entity_id": "x", + "callers": [{"entity_id": "python:function:svc.caller"}], + "total": -4, + "truncated": False, + } + ) + res = _client(FakeTransport([Response(status=200, body=body)])).get_callers("x") + assert res is not None + assert res.total == 1 + + def test_linkage_404_entity_unknown_is_soft_none(): # entity not known to Loomweave → 404 → honest None (caller degrades), never a crash t = FakeTransport([Response(status=404, body='{"code":"NOT_FOUND","error":"unknown"}')]) diff --git a/tests/unit/test_makefile_clean.py b/tests/unit/test_makefile_clean.py index c6837ed4..7e728483 100644 --- a/tests/unit/test_makefile_clean.py +++ b/tests/unit/test_makefile_clean.py @@ -14,6 +14,7 @@ def run_make_clean(workdir: Path) -> subprocess.CompletedProcess[str]: + assert MAKE is not None return subprocess.run( [MAKE, "-f", str(PROJECT_ROOT / "Makefile"), "clean"], cwd=workdir, From b19ff41c4c282098f4bfe825ac43dcc236fa3c9f Mon Sep 17 00:00:00 2001 From: John Morrissey Date: Fri, 12 Jun 2026 07:38:50 +1000 Subject: [PATCH 140/186] fix(dossier): bound source-derived shape text --- src/wardline/core/dossier.py | 115 +++++++++++++++++++--- tests/unit/core/test_dossier_assembler.py | 27 +++++ tests/unit/core/test_dossier_envelope.py | 14 +-- 3 files changed, 138 insertions(+), 18 deletions(-) diff --git a/src/wardline/core/dossier.py b/src/wardline/core/dossier.py index 13baa8d0..fe8cf598 100644 --- a/src/wardline/core/dossier.py +++ b/src/wardline/core/dossier.py @@ -21,8 +21,9 @@ * **SEI is opaque.** It is carried verbatim as the binding key, never parsed. * **No false-green.** An absent/unreachable source yields an *honest partial* section (``available=False`` + reason), never fabricated data and never a crash. - Over-budget content is trimmed with an EXPLICIT, elision-honest truncation marker - (shown-of-total) — a silent cap reads as "covered everything" when it did not. + Over-budget content is trimmed or compacted with an EXPLICIT, elision-honest + truncation marker (shown-of-total for lists, a visible marker for scalar text) — + a silent cap reads as "covered everything" when it did not. * **Zero-dependency base.** Stdlib only; no tokenizer, no blake3, no extras. """ @@ -351,8 +352,8 @@ def estimated_tokens(self) -> int: # --- token budgeting -------------------------------------------------------- # Lists are trimmed in this order (lowest value first) until the envelope fits. -# Identity/shape/trust-verdict are never trimmed — only the variable-length lists -# and finally the synthesis prose. +# Scalar identity/shape/trust-verdict fields are not dropped, but oversized strings +# are compacted after lower-value lists and synthesis have been trimmed. _LIST_TRIM_ORDER = ( "linkages.scc_peers", "linkages.callers", @@ -360,6 +361,8 @@ def estimated_tokens(self) -> int: "work.tickets", "trust.active_findings", ) +_CORE_TEXT_TRIM_STEPS = (512, 256, 128, 64) +_TRUNCATED_TEXT_MARKER = "......" def _list_for(dossier: EntityDossier, key: str) -> list[Any]: @@ -373,18 +376,80 @@ def _with_list(dossier: EntityDossier, key: str, items: list[Any]) -> EntityDoss return dataclasses.replace(dossier, **{section: new_section}) +def _truncate_text(value: str | None, max_chars: int) -> tuple[str | None, bool]: + if value is None or len(value) <= max_chars: + return value, False + if max_chars <= len(_TRUNCATED_TEXT_MARKER): + return _TRUNCATED_TEXT_MARKER[:max_chars], True + keep = max_chars - len(_TRUNCATED_TEXT_MARKER) + head = (keep + 1) // 2 + tail = keep // 2 + suffix = value[-tail:] if tail else "" + return f"{value[:head]}{_TRUNCATED_TEXT_MARKER}{suffix}", True + + +def _compact_core_text(dossier: EntityDossier, *, max_chars: int) -> tuple[EntityDossier, set[str]]: + changed: set[str] = set() + qualname, qualname_truncated = _truncate_text(dossier.identity.qualname, max_chars) + if qualname_truncated: + changed.add("identity.qualname") + kind, kind_truncated = _truncate_text(dossier.identity.kind, max_chars) + if kind_truncated: + changed.add("identity.kind") + path, path_truncated = _truncate_text(dossier.identity.path, max_chars) + if path_truncated: + changed.add("identity.path") + sei, sei_truncated = _truncate_text(dossier.identity.sei, max_chars) + if sei_truncated: + changed.add("identity.sei") + content_hash, content_hash_truncated = _truncate_text(dossier.identity.content_hash, max_chars) + if content_hash_truncated: + changed.add("identity.content_hash") + + signature, signature_truncated = _truncate_text(dossier.shape.signature, max_chars) + if signature_truncated: + changed.add("shape.signature") + + decorators: list[str] = [] + decorators_changed = False + for decorator in dossier.shape.decorators: + truncated, was_truncated = _truncate_text(decorator, max_chars) + decorators.append(truncated or "") + decorators_changed = decorators_changed or was_truncated + if decorators_changed: + changed.add("shape.decorators") + + identity = dataclasses.replace( + dossier.identity, + qualname=qualname or "", + kind=kind, + path=path, + sei=sei, + content_hash=content_hash, + ) + shape = dataclasses.replace( + dossier.shape, + signature=signature, + decorators=decorators if decorators_changed else dossier.shape.decorators, + ) + return dataclasses.replace(dossier, identity=identity, shape=shape), changed + + def bound_to_budget(dossier: EntityDossier, *, budget: int = DOSSIER_TOKEN_BUDGET) -> EntityDossier: """Return a copy of *dossier* whose estimated token size fits ``budget``, trimming variable-length lists (and finally the synthesis prose) in priority order and recording every elision honestly in :class:`Truncation`. - Identity and shape are never trimmed — they are the load-bearing minimum. If a - list is trimmed, ``Truncation.elided`` reports its shown-of-total counts so a - caller always knows something was dropped (no silent cap / false-green).""" + Identity and shape are never dropped — they are the load-bearing minimum — but + attacker-controlled scalar source text is compacted when it alone would exceed + the budget. If a list is trimmed, ``Truncation.elided`` reports its shown-of-total + counts so a caller always knows something was dropped (no silent cap / + false-green).""" if dossier.estimated_tokens() <= budget: return dataclasses.replace(dossier, truncation=Truncation.none()) totals = {key: len(_list_for(dossier, key)) for key in _LIST_TRIM_ORDER} + shape_decorator_total = len(dossier.shape.decorators) current = dossier # Pass 1: shrink lists, cheapest-value first, halving until the envelope fits. @@ -399,23 +464,51 @@ def bound_to_budget(dossier: EntityDossier, *, budget: int = DOSSIER_TOKEN_BUDGE current = dataclasses.replace(current, synthesis=None) synthesis_dropped = True + # Pass 3: decorators are shape, so keep them for synthesis-only overflow; trim + # them only when the remaining shape itself is still responsible for the excess. + while current.estimated_tokens() > budget and current.shape.decorators: + current = dataclasses.replace( + current, + shape=dataclasses.replace( + current.shape, + decorators=current.shape.decorators[: len(current.shape.decorators) // 2], + ), + ) + + # Pass 4: if the remaining load-bearing core is still too large, compact scalar + # text fields rather than returning an unbounded source-controlled envelope. + scalar_compacted: set[str] = set() + if current.estimated_tokens() > budget: + for max_chars in _CORE_TEXT_TRIM_STEPS: + current, compacted = _compact_core_text(current, max_chars=max_chars) + scalar_compacted.update(compacted) + if current.estimated_tokens() <= budget: + break + elided = [ ElidedSection(section=key, shown=len(_list_for(current, key)), total=totals[key]) for key in _LIST_TRIM_ORDER if len(_list_for(current, key)) < totals[key] ] + if len(current.shape.decorators) < shape_decorator_total: + elided.append( + ElidedSection(section="shape.decorators", shown=len(current.shape.decorators), total=shape_decorator_total) + ) note_bits = [] if elided: note_bits.append("lists trimmed to fit token budget") if synthesis_dropped: note_bits.append("synthesis dropped to fit token budget") - # Honest about a fit we did NOT achieve: identity/shape are never trimmed, so a - # pathologically long qualname/path can leave the core over budget. Say so rather - # than claiming "trimmed to fit" — a dishonest marker is itself a false-green. + if scalar_compacted: + fields = ", ".join(sorted(scalar_compacted)) + note_bits.append(f"core scalar fields compacted to fit token budget: {fields}") + # Honest about a fit we did NOT achieve after all trimmable content has been + # removed/compacted. This should be rare with the default budget, but the marker + # must not claim success if a caller supplies a smaller-than-schema budget. remaining = current.estimated_tokens() if remaining > budget: note = ( - f"EXCEEDS token budget: untrimmable identity/shape ~{remaining} of {budget} tokens; " + f"EXCEEDS token budget after all trimmable content: ~{remaining} of {budget} tokens; " "nothing further is trimmable" ) else: diff --git a/tests/unit/core/test_dossier_assembler.py b/tests/unit/core/test_dossier_assembler.py index a1193003..2006ffdc 100644 --- a/tests/unit/core/test_dossier_assembler.py +++ b/tests/unit/core/test_dossier_assembler.py @@ -364,6 +364,33 @@ def test_assembled_envelope_is_token_bounded(tmp_path: Path) -> None: assert estimate_tokens(text) <= DOSSIER_TOKEN_BUDGET +def test_shape_source_text_cannot_bypass_dossier_budget(tmp_path: Path) -> None: + proj = tmp_path / "shape" + proj.mkdir() + long_default = repr("d" * 10_000) + long_annotation = "tuple[" + ", ".join(["str"] * 2_000) + "]" + long_decorator = repr("x" * 10_000) + (proj / "m.py").write_text( + "def noisy(*args, **kwargs):\n" + " def wrap(fn):\n" + " return fn\n" + " return wrap\n" + f"@noisy({long_decorator})\n" + f"def target(arg: {long_annotation} = {long_default}):\n" + " return arg\n", + encoding="utf-8", + ) + + d = build_dossier("m.target", root=proj) + text = json.dumps(d.to_dict(), sort_keys=True) + + assert estimate_tokens(text) <= DOSSIER_TOKEN_BUDGET + assert d.truncation.truncated is True + assert "shape" in (d.truncation.note or "") + assert d.shape.signature is not None + assert len(d.shape.signature) < 1_000 + + # --- synthesis is best-effort and degrades with its inputs ------------------ diff --git a/tests/unit/core/test_dossier_envelope.py b/tests/unit/core/test_dossier_envelope.py index 20720bb8..fa10d998 100644 --- a/tests/unit/core/test_dossier_envelope.py +++ b/tests/unit/core/test_dossier_envelope.py @@ -219,16 +219,16 @@ def test_oversize_from_synthesis_only_drops_synthesis_with_empty_elision() -> No assert estimate_tokens(json.dumps(bounded.to_dict(), sort_keys=True)) <= DOSSIER_TOKEN_BUDGET -def test_untrimmable_core_over_budget_is_marked_honestly() -> None: - # identity/shape are never trimmed, so a pathologically long qualname can leave - # the core over budget. The marker must NOT claim "trimmed to fit" — it must say - # the budget was not achievable (a dishonest marker is itself a false-green). +def test_oversized_core_scalars_are_compacted_and_marked() -> None: + # Source-controlled identity/shape text is load-bearing but not unlimited: a + # pathologically long qualname must be compacted rather than bypassing the budget. huge = _minimal_dossier(identity=_identity(qualname="m." + "x" * 12_000)) bounded = bound_to_budget(huge) assert bounded.truncation.truncated is True - assert estimate_tokens(json.dumps(bounded.to_dict(), sort_keys=True)) > DOSSIER_TOKEN_BUDGET - assert "EXCEEDS token budget" in (bounded.truncation.note or "") - # identity is preserved even when un-fittable + assert estimate_tokens(json.dumps(bounded.to_dict(), sort_keys=True)) <= DOSSIER_TOKEN_BUDGET + assert "core scalar fields compacted" in (bounded.truncation.note or "") + assert "identity.qualname" in (bounded.truncation.note or "") + assert "......" in bounded.identity.qualname assert bounded.identity.qualname.startswith("m.x") From dd9f54308e4ae4b4951c39bf9c8961ddce8611f2 Mon Sep 17 00:00:00 2001 From: John Morrissey Date: Fri, 12 Jun 2026 07:40:16 +1000 Subject: [PATCH 141/186] test(cli): accept mcp policy args in install test --- tests/unit/cli/test_install.py | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-) diff --git a/tests/unit/cli/test_install.py b/tests/unit/cli/test_install.py index d39947c6..64644cfd 100644 --- a/tests/unit/cli/test_install.py +++ b/tests/unit/cli/test_install.py @@ -40,9 +40,19 @@ def test_mcp_resolves_loomweave_url_from_env(tmp_path: Path, monkeypatch) -> Non captured: dict[str, object] = {} class _FakeServer: - def __init__(self, *, root: Path, loomweave_url: str | None = None, filigree_url: str | None = None) -> None: + def __init__( + self, + *, + root: Path, + loomweave_url: str | None = None, + filigree_url: str | None = None, + allow_write: bool = True, + allow_network: bool = True, + ) -> None: captured["loomweave_url"] = loomweave_url captured["filigree_url"] = filigree_url + captured["allow_write"] = allow_write + captured["allow_network"] = allow_network self.rpc = self def run_stdio(self) -> None: From e8cd1dfd993412d9b6b8c543b1291a787298de23 Mon Sep 17 00:00:00 2001 From: John Morrissey Date: Fri, 12 Jun 2026 07:42:32 +1000 Subject: [PATCH 142/186] fix(explain): fail soft on malformed chain blobs --- src/wardline/core/explain.py | 2 ++ tests/unit/core/test_explain_chain.py | 11 +++++++++++ tests/unit/mcp/test_server_loomweave_write.py | 18 ++++++++++++++++-- 3 files changed, 29 insertions(+), 2 deletions(-) diff --git a/src/wardline/core/explain.py b/src/wardline/core/explain.py index 2cbe038f..407ab05d 100644 --- a/src/wardline/core/explain.py +++ b/src/wardline/core/explain.py @@ -382,6 +382,8 @@ def explain_finding( if served is not None: return served # miss/stale/outage → fall through to the re-run + if fingerprint is None and path is None and line is None: + return None return _explain_local( root, fingerprint=fingerprint, diff --git a/tests/unit/core/test_explain_chain.py b/tests/unit/core/test_explain_chain.py index 763035ad..5c92ae8b 100644 --- a/tests/unit/core/test_explain_chain.py +++ b/tests/unit/core/test_explain_chain.py @@ -103,3 +103,14 @@ def batch_get(self, qualnames): chain = explain_chain(proj, sink_qualname="svc.leaky", loomweave=RaisingClient(), max_hops=10) assert chain.hops == [] assert chain.truncated_at == "svc.leaky" + + +def test_chain_treats_malformed_callee_qualname_as_boundary_leaf(tmp_path): + proj = _proj(tmp_path) + client = MapClient({"svc.leaky": _fresh_view(proj, "svc.leaky", {"not": "a qualname"})}) + + chain = explain_chain(proj, sink_qualname="svc.leaky", loomweave=client, max_hops=10) + + assert [hop.qualname for hop in chain.hops] == ["svc.leaky"] + assert chain.hops[0].contributing_callee_qualname is None + assert chain.truncated_at is None diff --git a/tests/unit/mcp/test_server_loomweave_write.py b/tests/unit/mcp/test_server_loomweave_write.py index b6ce3551..4aa6c762 100644 --- a/tests/unit/mcp/test_server_loomweave_write.py +++ b/tests/unit/mcp/test_server_loomweave_write.py @@ -76,7 +76,14 @@ def _fresh_view(proj, qualname, callee_qualname): "resolved_call_count": 1, "unresolved_call_count": 0, }, - "findings": [], + "findings": [ + { + "rule_id": "PY-WL-101", + "fingerprint": "f" * 64, + "path": "svc.py", + "line_start": 5, + } + ], } return TaintFactView(qualname=qualname, exists=True, wardline_json=blob, current_content_hash=h) @@ -120,7 +127,14 @@ def _type_skewed_view(proj, qualname): "resolved_call_count": 1, "unresolved_call_count": 0, }, - "findings": [], + "findings": [ + { + "rule_id": "PY-WL-101", + "fingerprint": "f" * 64, + "path": "svc.py", + "line_start": 5, + } + ], } return TaintFactView(qualname=qualname, exists=True, wardline_json=blob, current_content_hash=h) From 9fa20c42d3b32bcdb2c9944c9b47575ed4a51489 Mon Sep 17 00:00:00 2001 From: John Morrissey Date: Fri, 12 Jun 2026 07:45:37 +1000 Subject: [PATCH 143/186] fix(loomweave): verify facts against analyzed bytes --- src/wardline/loomweave/facts.py | 42 ++++++++++++++++++------------ src/wardline/scanner/analyzer.py | 1 + src/wardline/scanner/context.py | 5 ++++ src/wardline/scanner/pipeline.py | 3 +++ tests/unit/loomweave/test_facts.py | 9 +++++++ 5 files changed, 43 insertions(+), 17 deletions(-) diff --git a/src/wardline/loomweave/facts.py b/src/wardline/loomweave/facts.py index 590e0742..d0ace9aa 100644 --- a/src/wardline/loomweave/facts.py +++ b/src/wardline/loomweave/facts.py @@ -21,6 +21,7 @@ from __future__ import annotations +import hashlib from pathlib import Path from typing import TYPE_CHECKING, Any @@ -57,7 +58,6 @@ def build_taint_facts(result: ScanResult, root: Path) -> list[dict[str, Any]]: context = result.context if context is None: return [] - blake3 = require_blake3() hash_cache: dict[str, str] = {} findings_by_qualname: dict[str, list[dict[str, Any]]] = {} @@ -76,22 +76,9 @@ def build_taint_facts(result: ScanResult, root: Path) -> list[dict[str, Any]]: facts: list[dict[str, Any]] = [] for qualname, entity in context.entities.items(): rel_path = entity.location.path - if rel_path not in hash_cache: - hash_cache[rel_path] = blake3.blake3(_read_bytes(root / rel_path)).hexdigest() - # RESIDUAL (builtin-marker shadow false-green): ``content_hash_at_compute`` - # is whole-file raw-byte blake3 ONLY — it cannot observe the shadow state of - # a builtin marker root. So identical file bytes scanned once UNSHADOWED then - # under a project that shadows ``wardline``/``weft_markers`` could serve a - # stale TRUSTED fact via the MCP explain_taint / Loomweave read path. We do NOT - # fold the shadow bit / provider fingerprint into this hash: it is a - # CROSS-TOOL contract value — Loomweave's read path INDEPENDENTLY recomputes - # the whole-file blake3 (loomweave_storage::current_file_hash) and compares it - # against the in-blob copy. Mixing in a Wardline-private bit would make every - # comparison mismatch and break fact reconciliation entirely; there is no - # separate Wardline-owned compute-key the freshness gate consults. Closing - # this fully needs a Loomweave read-path contract change. Lower impact: this - # path is opt-in (--loomweave-url) and not the scan gate. See CHANGELOG. - content_hash = hash_cache[rel_path] + content_hash = _content_hash_for_analyzed_file(root, rel_path, context, hash_cache) + if content_hash is None: + continue declared = context.project_return_taints.get(qualname) actual = context.function_return_taints.get(qualname) @@ -132,3 +119,24 @@ def _dead_code_root_blob(is_declared: bool) -> dict[str, Any]: "tags": ["entry-point"], "reason": _ROOT_REASON, } + + +def _content_hash_for_analyzed_file( + root: Path, + rel_path: str, + context: AnalysisContext, + hash_cache: dict[str, str], +) -> str | None: + if rel_path in hash_cache: + return hash_cache[rel_path] + try: + current_bytes = _read_bytes(root / rel_path) + except OSError: + return None + analyzed_sha256 = context.analyzed_source_sha256.get(rel_path) + if analyzed_sha256 is not None and hashlib.sha256(current_bytes).hexdigest() != analyzed_sha256: + return None + blake3 = require_blake3() + content_hash = str(blake3.blake3(current_bytes).hexdigest()) + hash_cache[rel_path] = content_hash + return content_hash diff --git a/src/wardline/scanner/analyzer.py b/src/wardline/scanner/analyzer.py index f27e4768..ee89cfa1 100644 --- a/src/wardline/scanner/analyzer.py +++ b/src/wardline/scanner/analyzer.py @@ -913,6 +913,7 @@ def _l2_nested_def_overlay() -> dict[str, dict[str, TaintState]]: project_edges=result.project_edges, call_site_implicit_receivers=result.call_site_implicit_receivers, alias_maps={m.module_path: m.alias_map for m in modules}, + analyzed_source_sha256={parsed.relpath: parsed.source_sha256 for parsed in file_meta}, module_bindings=module_sink_bindings, enabled_rule_ids=enabled_rule_ids, ) diff --git a/src/wardline/scanner/context.py b/src/wardline/scanner/context.py index 70c086f1..2853fb2c 100644 --- a/src/wardline/scanner/context.py +++ b/src/wardline/scanner/context.py @@ -99,6 +99,10 @@ class AnalysisContext: project_edges: Mapping[str, frozenset[str]] = field(default_factory=dict) # Import alias maps per module: ``{module: {alias: target_fqn}}``. alias_maps: Mapping[str, Mapping[str, str]] = field(default_factory=dict) + # SHA-256 of the exact source bytes parsed for each project-relative file. Optional + # consumers that stamp external facts use this to refuse post-scan disk races without + # making the scanner depend on optional BLAKE3. + analyzed_source_sha256: Mapping[str, str] = field(default_factory=dict) # MODULE-SCOPE name bindings per module: ``{module: SinkBindings}`` — module-level # callable aliases (``runner = subprocess.run``) and constructed instances # (``client = httpx.Client()``), collected from each module's top-level scope by the @@ -159,6 +163,7 @@ def __post_init__(self) -> None: "alias_maps", _freeze_mapping(self.alias_maps), ) + object.__setattr__(self, "analyzed_source_sha256", _freeze_mapping(self.analyzed_source_sha256)) # SinkBindings values are frozen dataclasses — only the outer map needs the proxy. object.__setattr__(self, "module_bindings", _freeze_mapping(self.module_bindings)) diff --git a/src/wardline/scanner/pipeline.py b/src/wardline/scanner/pipeline.py index da8dc855..f2849742 100644 --- a/src/wardline/scanner/pipeline.py +++ b/src/wardline/scanner/pipeline.py @@ -41,6 +41,7 @@ class ParsedFile: entities: tuple[Entity, ...] alias_map: dict[str, str] class_qualnames: frozenset[str] + source_sha256: str @dataclass(frozen=True, slots=True) @@ -128,6 +129,7 @@ def run_parse_project_stage(stage_input: ParseProjectInput) -> ParseProjectOutpu try: source = path.read_text(encoding="utf-8") source_bytes = source.encode("utf-8") + source_sha256 = hashlib.sha256(source_bytes).hexdigest() from wardline.core.ruleset import ruleset_hash from wardline.scanner.taint.project_resolver import _RESOLVER_VERSION @@ -249,6 +251,7 @@ def run_parse_project_stage(stage_input: ParseProjectInput) -> ParseProjectOutpu entities=entities, alias_map=alias_map, class_qualnames=classes, + source_sha256=source_sha256, ) ) for ent in entities: diff --git a/tests/unit/loomweave/test_facts.py b/tests/unit/loomweave/test_facts.py index 99d27b6a..ffb55f97 100644 --- a/tests/unit/loomweave/test_facts.py +++ b/tests/unit/loomweave/test_facts.py @@ -53,6 +53,15 @@ def test_content_hash_is_blake3_whole_file_and_top_level_and_in_blob(tmp_path): assert len(expected) == 64 +def test_fact_emission_refuses_files_changed_after_scan(tmp_path): + proj, result = _scan_leaky(tmp_path) + (proj / "svc.py").write_text("def replacement():\n return 1\n", encoding="utf-8") + + facts = build_taint_facts(result, proj) + + assert facts == [] + + def test_per_file_hash_is_memoized(tmp_path, monkeypatch): proj, result = _scan_leaky(tmp_path) import wardline.loomweave.facts as facts_mod From 1b547dc1fa9209077b9900bb6e55ba174acbda27 Mon Sep 17 00:00:00 2001 From: John Morrissey Date: Fri, 12 Jun 2026 07:47:59 +1000 Subject: [PATCH 144/186] fix(waivers): refuse symlinked waiver writes --- src/wardline/core/safe_paths.py | 6 ++++++ src/wardline/core/waivers.py | 12 ++++++------ tests/unit/core/test_waiver_add.py | 27 ++++++++++++++++++++++++++- 3 files changed, 38 insertions(+), 7 deletions(-) diff --git a/src/wardline/core/safe_paths.py b/src/wardline/core/safe_paths.py index 13c6a842..e4d5bf51 100644 --- a/src/wardline/core/safe_paths.py +++ b/src/wardline/core/safe_paths.py @@ -42,6 +42,12 @@ def safe_write_text(root: Path, target: Path, content: str, *, label: str | None _write_text_no_follow(safe_path, content, label=label or safe_path.name) +def write_text_no_follow(target: Path, content: str, *, label: str | None = None) -> None: + """Write ``content`` without following a final-component symlink.""" + target.parent.mkdir(parents=True, exist_ok=True) + _write_text_no_follow(target, content, label=label or target.name) + + def _write_text_no_follow(path: Path, content: str, *, label: str) -> None: flags = os.O_WRONLY | os.O_CREAT | os.O_TRUNC if hasattr(os, "O_NOFOLLOW"): diff --git a/src/wardline/core/waivers.py b/src/wardline/core/waivers.py index 111203d8..e58a704e 100644 --- a/src/wardline/core/waivers.py +++ b/src/wardline/core/waivers.py @@ -22,7 +22,7 @@ from wardline.core.finding import FINGERPRINT_SCHEME, require_fingerprint_scheme from wardline.core.optional_deps import require_yaml from wardline.core.paths import waivers_path -from wardline.core.safe_paths import safe_project_file +from wardline.core.safe_paths import safe_project_file, safe_write_text, write_text_no_follow WAIVERS_VERSION: int = 1 """Bumped on a format change; validated on load (mirrors BASELINE_VERSION).""" @@ -170,11 +170,11 @@ def add_waiver( waivers.append(entry) # Re-stamp the scheme header (idempotent) and place it first for readability. document = {"fingerprint_scheme": FINGERPRINT_SCHEME, "version": WAIVERS_VERSION, "waivers": waivers} - config_path.parent.mkdir(parents=True, exist_ok=True) - config_path.write_text( - yaml.safe_dump(document, sort_keys=False, default_flow_style=False, allow_unicode=True), - encoding="utf-8", - ) + content = yaml.safe_dump(document, sort_keys=False, default_flow_style=False, allow_unicode=True) + if root is not None: + safe_write_text(root, config_path, content, label=config_path.name) + else: + write_text_no_follow(config_path, content, label=config_path.name) return waiver diff --git a/tests/unit/core/test_waiver_add.py b/tests/unit/core/test_waiver_add.py index f71f4681..38e6fb7f 100644 --- a/tests/unit/core/test_waiver_add.py +++ b/tests/unit/core/test_waiver_add.py @@ -3,7 +3,7 @@ import pytest -from wardline.core.errors import ConfigError +from wardline.core.errors import ConfigError, WardlineError from wardline.core.paths import waivers_path from wardline.core.waivers import add_waiver, load_project_waivers @@ -80,3 +80,28 @@ def test_add_waiver_no_expiry_omits_field(tmp_path: Path) -> None: assert w.expires is None waivers = load_project_waivers(tmp_path) assert waivers[0].expires is None + + +def test_add_waiver_refuses_direct_symlink_target(tmp_path: Path) -> None: + outside = tmp_path / "outside.yaml" + outside.write_text("", encoding="utf-8") + link = tmp_path / "waivers.yaml" + link.symlink_to(outside) + + with pytest.raises(WardlineError, match="symlink"): + add_waiver(link, fingerprint=FP, reason="ok", expires=None) + + assert outside.read_text(encoding="utf-8") == "" + + +def test_add_waiver_refuses_rooted_symlink_target(tmp_path: Path) -> None: + outside = tmp_path / "outside.yaml" + outside.write_text("", encoding="utf-8") + wp = waivers_path(tmp_path) + wp.parent.mkdir(parents=True) + wp.symlink_to(outside) + + with pytest.raises(WardlineError, match="symlink"): + add_waiver(wp, fingerprint=FP, reason="ok", expires=None, root=tmp_path) + + assert outside.read_text(encoding="utf-8") == "" From 41e381c3c8a81b12bc71913dca8900e2c7d089b7 Mon Sep 17 00:00:00 2001 From: John Morrissey Date: Fri, 12 Jun 2026 07:49:36 +1000 Subject: [PATCH 145/186] docs: avoid fixed tmp pre-commit output --- docs/guides/agents.md | 15 +++++++++------ 1 file changed, 9 insertions(+), 6 deletions(-) diff --git a/docs/guides/agents.md b/docs/guides/agents.md index 5df20f0c..66f2aeff 100644 --- a/docs/guides/agents.md +++ b/docs/guides/agents.md @@ -158,15 +158,18 @@ To make the gate run on every commit, drop a `.git/hooks/pre-commit` script ```bash #!/usr/bin/env sh # Block a commit if Wardline finds a new ERROR-or-worse defect. -# Write the findings file outside the working tree so the commit stays clean. -wardline scan . --fail-on ERROR --output /tmp/wardline-findings.jsonl +# Use mktemp for the findings file; never write to a predictable shared /tmp path. +out="$(mktemp "${TMPDIR:-/tmp}/wardline-findings.XXXXXX.jsonl")" || exit 2 +trap 'rm -f "$out"' EXIT +wardline scan . --fail-on ERROR --output "$out" ``` A `scan` always writes a findings file (default `findings.jsonl` in the scan -path), so point `--output` outside the tree — as above — or at a git-ignored -path; otherwise the hook litters every commit. The script's exit code becomes -the hook's exit code: a clean tree commits, a new defect aborts the commit with -the finding already on screen for the agent to act on. +path), so point `--output` at a per-run temporary file — as above — or at a +git-ignored path inside the repository; otherwise the hook litters every commit. +Avoid predictable filenames in shared directories such as `/tmp`. The script's +exit code becomes the hook's exit code: a clean tree commits, a new defect +aborts the commit with the finding already on screen for the agent to act on. ## Let the agent triage with `wardline judge` From fd8cc6ca798a47d0467e1f385f9b237d52bac0a2 Mon Sep 17 00:00:00 2001 From: John Morrissey Date: Fri, 12 Jun 2026 07:52:37 +1000 Subject: [PATCH 146/186] fix(baseline): refuse symlinked baseline writes --- src/wardline/core/baseline.py | 10 +++++----- tests/unit/core/test_baseline.py | 28 +++++++++++++++++++++++++++- 2 files changed, 32 insertions(+), 6 deletions(-) diff --git a/src/wardline/core/baseline.py b/src/wardline/core/baseline.py index 86f97474..617c92b3 100644 --- a/src/wardline/core/baseline.py +++ b/src/wardline/core/baseline.py @@ -25,7 +25,7 @@ ) from wardline.core.optional_deps import require_yaml from wardline.core.paths import baseline_path as baseline_file -from wardline.core.safe_paths import safe_project_file +from wardline.core.safe_paths import safe_write_text, write_text_no_follow BASELINE_VERSION: int = 1 """Bumped on a format change; validated on load (mirrors STDLIB_TAINT_VERSION).""" @@ -70,13 +70,13 @@ def build_baseline_document(findings: Iterable[Finding]) -> dict[str, Any]: def write_baseline(path: Path, findings: Iterable[Finding], root: Path | None = None) -> None: yaml = require_yaml("writing baseline.yaml") - if root is not None: - path = safe_project_file(root, path, label=path.name) - path.parent.mkdir(parents=True, exist_ok=True) text = yaml.safe_dump( build_baseline_document(findings), sort_keys=False, default_flow_style=False, allow_unicode=True ) - path.write_text(text, encoding="utf-8") + if root is not None: + safe_write_text(root, path, text, label=path.name) + else: + write_text_no_follow(path, text, label=path.name) def collect_and_write_baseline( diff --git a/tests/unit/core/test_baseline.py b/tests/unit/core/test_baseline.py index ac1caa72..6348fcea 100644 --- a/tests/unit/core/test_baseline.py +++ b/tests/unit/core/test_baseline.py @@ -11,8 +11,9 @@ load_baseline, write_baseline, ) -from wardline.core.errors import ConfigError, SchemeMismatchError +from wardline.core.errors import ConfigError, SchemeMismatchError, WardlineError from wardline.core.finding import FINGERPRINT_SCHEME, Finding, Kind, Location, Severity +from wardline.core.paths import baseline_path _FP_A = "a" * 64 _FP_B = "b" * 64 @@ -104,6 +105,31 @@ def test_write_then_load_round_trips(tmp_path: Path) -> None: assert bl.contains(_FP_A) and not bl.contains("c" * 64) +def test_write_baseline_refuses_direct_symlink_target(tmp_path: Path) -> None: + outside = tmp_path / "outside.yaml" + outside.write_text("", encoding="utf-8") + link = tmp_path / "baseline.yaml" + link.symlink_to(outside) + + with pytest.raises(WardlineError, match="symlink"): + write_baseline(link, [_finding(_FP_A)]) + + assert outside.read_text(encoding="utf-8") == "" + + +def test_write_baseline_refuses_rooted_symlink_target(tmp_path: Path) -> None: + outside = tmp_path / "outside.yaml" + outside.write_text("", encoding="utf-8") + bp = baseline_path(tmp_path) + bp.parent.mkdir(parents=True) + bp.symlink_to(outside) + + with pytest.raises(WardlineError, match="symlink"): + write_baseline(bp, [_finding(_FP_A)], root=tmp_path) + + assert outside.read_text(encoding="utf-8") == "" + + def test_missing_file_is_empty_baseline(tmp_path: Path) -> None: assert load_baseline(tmp_path / "nope.yaml").fingerprints == frozenset() From 82fa50e143a42445be52529c172e7b5218315bfe Mon Sep 17 00:00:00 2001 From: John Morrissey Date: Fri, 12 Jun 2026 07:55:36 +1000 Subject: [PATCH 147/186] fix(taint): fail closed on dynamic trusted levels --- .../scanner/taint/decorator_provider.py | 43 +++++++++++++------ .../scanner/taint/test_decorator_provider.py | 34 +++++++++++++-- 2 files changed, 61 insertions(+), 16 deletions(-) diff --git a/src/wardline/scanner/taint/decorator_provider.py b/src/wardline/scanner/taint/decorator_provider.py index 192c8caa..8498c099 100644 --- a/src/wardline/scanner/taint/decorator_provider.py +++ b/src/wardline/scanner/taint/decorator_provider.py @@ -155,23 +155,42 @@ def _read_level( Returns ``default`` when the decorator is not called or the arg is absent; ``None`` (fail-closed) when the arg is present but unreadable, an invalid - state, or outside ``allowed``. Positional args are intentionally ignored — - the real decorators are keyword-only, so a positional form is malformed - source and reads as ``default`` (fail-closed for required-arg decorators). + state, outside ``allowed``, duplicated, or mixed with malformed decorator + call syntax. The real level-bearing decorators are keyword-only; positional + args and unexpected keywords are not trusted as the default level. """ if not isinstance(deco, ast.Call): return default + if deco.args: + return None + values: list[ast.expr] = [] for kw in deco.keywords: - if kw.arg == arg: - token = _level_token(kw.value, alias_map) - if token is None: - return None - try: - level = TaintState(token) - except ValueError: + if kw.arg is None: + if not isinstance(kw.value, ast.Dict): return None - return level if level in allowed else None - return default + for key, value in zip(kw.value.keys, kw.value.values, strict=True): + if not isinstance(key, ast.Constant) or not isinstance(key.value, str): + return None + if key.value != arg: + return None + values.append(value) + continue + if kw.arg == arg: + values.append(kw.value) + continue + return None + if not values: + return default + if len(values) != 1: + return None + token = _level_token(values[0], alias_map) + if token is None: + return None + try: + level = TaintState(token) + except ValueError: + return None + return level if level in allowed else None def _seed_value_identity(value: object) -> str: diff --git a/tests/unit/scanner/taint/test_decorator_provider.py b/tests/unit/scanner/taint/test_decorator_provider.py index 9dbc0a44..6f0388d4 100644 --- a/tests/unit/scanner/taint/test_decorator_provider.py +++ b/tests/unit/scanner/taint/test_decorator_provider.py @@ -161,6 +161,32 @@ def test_trusted_level_assured() -> None: assert out["m.f"] == FunctionTaint(T.ASSURED, T.ASSURED) +def test_trusted_level_static_kwargs_assured() -> None: + out = _seed( + "from wardline.decorators import trusted\n@trusted(**{'level': 'ASSURED'})\ndef f():\n return 1\n" + ) + assert out["m.f"] == FunctionTaint(T.ASSURED, T.ASSURED) + + +def test_trusted_dynamic_kwargs_is_no_opinion() -> None: + out = _seed( + "from wardline.decorators import trusted\n" + "KW = {'level': 'ASSURED'}\n" + "@trusted(**KW)\ndef f():\n return 1\n" + ) + assert out["m.f"] is None + + +def test_trusted_positional_arg_is_no_opinion() -> None: + out = _seed("from wardline.decorators import trusted\n@trusted('ASSURED')\ndef f():\n return 1\n") + assert out["m.f"] is None + + +def test_trusted_unexpected_keyword_is_no_opinion() -> None: + out = _seed("from wardline.decorators import trusted\n@trusted(other=1)\ndef f():\n return 1\n") + assert out["m.f"] is None + + def test_trusted_disallowed_level_is_no_opinion() -> None: out = _seed("from wardline.decorators import trusted\n@trusted(level='GUARDED')\ndef f():\n return 1\n") assert out["m.f"] is None @@ -293,14 +319,14 @@ def test_subscript_decorator_is_no_opinion() -> None: assert out["m.f"] is None -def test_non_matching_keyword_is_skipped_before_level_arg() -> None: - # A decorator with an UNRELATED keyword before to_level: the kw loop must skip the - # non-matching keyword (118->117) and still read to_level correctly. +def test_unexpected_keyword_fails_closed_even_with_valid_level_arg() -> None: + # Builtin level-bearing decorators are keyword-only with a fixed signature. A + # malformed call must not be seeded from the valid-looking level beside it. out = _seed( "from wardline.decorators import trust_boundary\n" "@trust_boundary(other=1, to_level='ASSURED')\ndef v(x):\n return x\n" ) - assert out["m.v"] == FunctionTaint(T.EXTERNAL_RAW, T.ASSURED) + assert out["m.v"] is None def test_invalid_taintstate_token_is_no_opinion() -> None: From ed2bf38d9c1008c50b31a211ec1feb4378c5fdac Mon Sep 17 00:00:00 2001 From: John Morrissey Date: Fri, 12 Jun 2026 08:00:45 +1000 Subject: [PATCH 148/186] fix(cli): write explicit relative outputs in place --- src/wardline/core/emit.py | 7 +++++-- src/wardline/core/sarif.py | 7 +++++-- 2 files changed, 10 insertions(+), 4 deletions(-) diff --git a/src/wardline/core/emit.py b/src/wardline/core/emit.py index 3ec14a02..3174ce1d 100644 --- a/src/wardline/core/emit.py +++ b/src/wardline/core/emit.py @@ -8,7 +8,7 @@ from typing import Protocol from wardline.core.finding import Finding -from wardline.core.safe_paths import safe_write_text +from wardline.core.safe_paths import safe_write_text, write_text_no_follow class Sink(Protocol): @@ -22,4 +22,7 @@ def __init__(self, path: Path, *, root: Path | None = None) -> None: def write(self, findings: Sequence[Finding]) -> None: content = "".join(f"{finding.to_jsonl()}\n" for finding in findings) - safe_write_text(self._root or self._path.parent, self._path, content, label=self._path.name) + if self._root is not None: + safe_write_text(self._root, self._path, content, label=self._path.name) + else: + write_text_no_follow(self._path, content, label=self._path.name) diff --git a/src/wardline/core/sarif.py b/src/wardline/core/sarif.py index 083670ff..9e92a2cb 100644 --- a/src/wardline/core/sarif.py +++ b/src/wardline/core/sarif.py @@ -15,7 +15,7 @@ from wardline import __version__ from wardline.core.finding import Finding, Kind, Location, Severity, SuppressionState -from wardline.core.safe_paths import safe_write_text +from wardline.core.safe_paths import safe_write_text, write_text_no_follow from wardline.scanner.flow_trace import build_finding_code_flow if TYPE_CHECKING: @@ -167,4 +167,7 @@ def __init__(self, path: Path, *, root: Path | None = None) -> None: def write(self, findings: Sequence[Finding], context: AnalysisContext | None = None) -> None: content = json.dumps(build_sarif(findings, context), indent=2, ensure_ascii=False) - safe_write_text(self._root or self._path.parent, self._path, content, label=self._path.name) + if self._root is not None: + safe_write_text(self._root, self._path, content, label=self._path.name) + else: + write_text_no_follow(self._path, content, label=self._path.name) From 207e1214ef6af18293967f15dbcfadb6dfaf0a7c Mon Sep 17 00:00:00 2001 From: John Morrissey Date: Fri, 12 Jun 2026 08:01:23 +1000 Subject: [PATCH 149/186] fix(analyzer): fail closed on unanalyzable code --- src/wardline/core/finding.py | 6 +-- src/wardline/scanner/analyzer.py | 51 +++++++++++++------ src/wardline/scanner/context.py | 4 +- src/wardline/scanner/pipeline.py | 6 +-- src/wardline/scanner/rules/_sink_helpers.py | 7 +-- tests/unit/scanner/test_analyzer.py | 8 +-- tests/unit/scanner/test_analyzer_isolation.py | 7 ++- tests/unit/scanner/test_pipeline.py | 12 ++--- 8 files changed, 63 insertions(+), 38 deletions(-) diff --git a/src/wardline/core/finding.py b/src/wardline/core/finding.py index 56332a42..6bb76a62 100644 --- a/src/wardline/core/finding.py +++ b/src/wardline/core/finding.py @@ -28,9 +28,9 @@ # Rule ids for files where analysis was ATTEMPTED OR EXPECTED but FAILED / never # happened — a genuine under-scan: parse/read failures, files too deep to walk -# (recursion), and missing source roots. These are Severity.NONE FACTs that never -# trip the severity gate, so callers count them separately (ScanSummary.unanalyzed) -# to surface the silent under-scan and (opt-in) gate on it. +# (recursion), and missing source roots. Some are gate-eligible DEFECTs and some +# are non-gating FACTs, so callers also count them separately +# (ScanSummary.unanalyzed) to expose the under-scan independent of severity. # # WLN-ENGINE-NO-MODULE is DELIBERATELY EXCLUDED: a file that maps to no module # (e.g. a top-level / src/__init__.py) is a benign layout artifact with nothing to diff --git a/src/wardline/scanner/analyzer.py b/src/wardline/scanner/analyzer.py index ee89cfa1..79b9e5cc 100644 --- a/src/wardline/scanner/analyzer.py +++ b/src/wardline/scanner/analyzer.py @@ -571,6 +571,7 @@ def _store( # ── L2 pass 1 — per-method var/return taints + per-class attribute summary ── all_classes = frozenset(c for parsed in file_meta for c in parsed.class_qualnames) failed_paths: set[str] = set() + function_skip_recorded: set[str] = set() def _record_file_failure(relpath: str, ent: Entity, exc: Exception) -> None: # Per-file isolation, mirroring the Rust frontend's WLN-ENGINE-FILE-FAILED: @@ -600,6 +601,24 @@ def _record_file_failure(relpath: str, ent: Entity, exc: Exception) -> None: ) ) + def _record_l2_recursion(ent: Entity) -> None: + l2_failed.add(ent.qualname) + if ent.qualname in function_skip_recorded: + return + function_skip_recorded.add(ent.qualname) + func_skip_findings.append( + Finding( + rule_id="WLN-ENGINE-FUNCTION-SKIPPED", + message=f"{ent.qualname}: skipped L2 — expression too deep to analyze safely", + severity=Severity.ERROR, + kind=Kind.DEFECT, + location=ent.location, + fingerprint=_fp("WLN-ENGINE-FUNCTION-SKIPPED", ent.qualname), + qualname=ent.qualname, + properties={"reason": "recursion_limit"}, + ) + ) + for parsed in file_meta: module = parsed.module entities = parsed.entities @@ -660,21 +679,15 @@ def _record_file_failure(relpath: str, ent: Entity, exc: Exception) -> None: recorded_writes, all_classes, enclosing_class if is_method else None ) except RecursionError: - l2_failed.add(ent.qualname) - call_sites, call_args, var_taints, ret_taint, ret_callee = {}, {}, {}, None, None - writes = {} - func_skip_findings.append( - Finding( - rule_id="WLN-ENGINE-FUNCTION-SKIPPED", - message=f"{ent.qualname}: skipped L2 — expression too deep to analyze safely", - severity=Severity.NONE, - kind=Kind.FACT, - location=ent.location, - fingerprint=_fp("WLN-ENGINE-FUNCTION-SKIPPED", ent.qualname), - qualname=ent.qualname, - properties={"reason": "recursion_limit"}, - ) + _record_l2_recursion(ent) + call_sites, call_args, var_taints, ret_taint, ret_callee = ( + {}, + {}, + {}, + TaintState.UNKNOWN_RAW, + None, ) + writes = {} except MemoryError: raise # exhaustion is not a per-file condition — isolating it would thrash except Exception as exc: # noqa: BLE001 — per-file isolation, see _record_file_failure @@ -804,7 +817,15 @@ def _l2_nested_def_overlay() -> dict[str, dict[str, TaintState]]: recorded_writes, all_classes, enclosing_class if is_method else None ) except RecursionError: - continue + _record_l2_recursion(ent) + call_sites, call_args, var_taints, ret_taint, ret_callee, writes = ( + {}, + {}, + {}, + TaintState.UNKNOWN_RAW, + None, + {}, + ) except MemoryError: raise # exhaustion is not a per-file condition — isolating it would thrash except Exception as exc: # noqa: BLE001 — per-file isolation, see _record_file_failure diff --git a/src/wardline/scanner/context.py b/src/wardline/scanner/context.py index 2853fb2c..bbb1a5c6 100644 --- a/src/wardline/scanner/context.py +++ b/src/wardline/scanner/context.py @@ -122,8 +122,8 @@ class AnalysisContext: # back to the pessimistic flow-INSENSITIVE map (no L2 snapshot — an L2-skipped # function). ``resolved_arg_taints`` records here instead of warning from # inside rule ``check()`` calls; the analyzer surfaces the collected set as ONE - # ``WLN-ENGINE-FLOW-INSENSITIVE-FALLBACK`` NONE/FACT finding per scan - # (mirroring WLN-ENGINE-FUNCTION-SKIPPED). Deliberately a mutable set on a + # ``WLN-ENGINE-FLOW-INSENSITIVE-FALLBACK`` NONE/FACT finding per scan. + # Deliberately a mutable set on a # frozen context: it is a diagnostics side channel, not engine output. flow_insensitive_fallbacks: set[str] = field(default_factory=set) diff --git a/src/wardline/scanner/pipeline.py b/src/wardline/scanner/pipeline.py index f2849742..ed8ac58d 100644 --- a/src/wardline/scanner/pipeline.py +++ b/src/wardline/scanner/pipeline.py @@ -206,9 +206,9 @@ def run_parse_project_stage(stage_input: ParseProjectInput) -> ParseProjectOutpu Finding( rule_id="WLN-ENGINE-FILE-SKIPPED", message=f"{relpath}: skipped — expression too deep to analyze safely", - severity=Severity.NONE, - kind=Kind.FACT, - location=Location(path=relpath), + severity=Severity.ERROR, + kind=Kind.DEFECT, + location=Location(path=relpath, line_start=1), fingerprint=_fp("WLN-ENGINE-FILE-SKIPPED", relpath), properties={"module": module, "reason": "recursion_limit"}, ) diff --git a/src/wardline/scanner/rules/_sink_helpers.py b/src/wardline/scanner/rules/_sink_helpers.py index 7e97071a..8b5076a8 100644 --- a/src/wardline/scanner/rules/_sink_helpers.py +++ b/src/wardline/scanner/rules/_sink_helpers.py @@ -477,9 +477,10 @@ def resolved_arg_taints(call: ast.Call, qualname: str, context: AnalysisContext) qualname is recorded into ``context.flow_insensitive_fallbacks`` and a pessimistic map marking every syntactic argument ``UNKNOWN_RAW`` is returned. The analyzer surfaces the recorded set as ONE ``WLN-ENGINE-FLOW-INSENSITIVE-FALLBACK`` - NONE/FACT finding per scan — a finding, not a ``UserWarning``, so MCP/library - consumers see the degradation and a warnings-as-error embedder cannot turn the - diagnostic into a rule-aborting raise (review 2026-06-10). Each rule then + NONE/FACT finding per scan in addition to the gate-eligible function skip — a + finding, not a ``UserWarning``, so MCP/library consumers see the degradation and + a warnings-as-error embedder cannot turn the diagnostic into a rule-aborting raise + (review 2026-06-10). Each rule then SELECTS over this result on its own terms (worst / any-provably-untrusted / by-position), so the fail-closed contract lives in exactly one place and cannot drift between rules. The pessimism is correctly direction-aware: a diff --git a/tests/unit/scanner/test_analyzer.py b/tests/unit/scanner/test_analyzer.py index 0ba7e860..cf473842 100644 --- a/tests/unit/scanner/test_analyzer.py +++ b/tests/unit/scanner/test_analyzer.py @@ -132,12 +132,12 @@ def _boom(stage_input): # noqa: ANN001, ANN202 assert ctx is not None assert ctx.function_var_taints["m.a"] == {} # L2 contained assert "b" in ctx.function_var_taints["m.b"] or ctx.function_var_taints["m.b"] == {} - # The contained function is NOT silently dropped — a FACT records the skip - # so its absent return taint is observable, not an invisible under-taint. + # The contained function is NOT silently dropped: the skip is gate-eligible + # and its actual return is pessimistic rather than absent. skips = [f for f in findings if f.rule_id == "WLN-ENGINE-FUNCTION-SKIPPED"] assert [f.qualname for f in skips] == ["m.a"] - assert all(f.kind == Kind.FACT for f in skips) - assert "m.a" not in ctx.function_return_taints + assert all(f.kind == Kind.DEFECT for f in skips) + assert ctx.function_return_taints["m.a"] == T.UNKNOWN_RAW def test_analyzer_default_provider_seeds_from_decorators(tmp_path) -> None: diff --git a/tests/unit/scanner/test_analyzer_isolation.py b/tests/unit/scanner/test_analyzer_isolation.py index 368c2fd0..682a5e57 100644 --- a/tests/unit/scanner/test_analyzer_isolation.py +++ b/tests/unit/scanner/test_analyzer_isolation.py @@ -97,12 +97,15 @@ def test_l2_engine_exception_emits_one_finding_per_file(tmp_path, monkeypatch) - def test_recursion_error_still_yields_function_skip_fact(tmp_path, monkeypatch) -> None: # The broad fence must NOT swallow the dedicated RecursionError boundary — - # too-deep functions keep their non-gating WLN-ENGINE-FUNCTION-SKIPPED FACT. + # too-deep functions keep their dedicated WLN-ENGINE-FUNCTION-SKIPPED finding. _boom_on_marker(monkeypatch, RecursionError("simulated deep L2")) _write(tmp_path, "deep.py", "def a():\n boom = 1\n return boom\n") analyzer = WardlineAnalyzer() findings = analyzer.analyze([tmp_path / "deep.py"], WardlineConfig(), root=tmp_path) - assert any(f.rule_id == "WLN-ENGINE-FUNCTION-SKIPPED" and f.kind == Kind.FACT for f in findings) + skipped = [f for f in findings if f.rule_id == "WLN-ENGINE-FUNCTION-SKIPPED"] + assert len(skipped) == 1 + assert skipped[0].kind == Kind.DEFECT + assert skipped[0].severity == Severity.ERROR assert not any(f.rule_id == "WLN-ENGINE-FILE-FAILED" for f in findings) diff --git a/tests/unit/scanner/test_pipeline.py b/tests/unit/scanner/test_pipeline.py index 30206ab7..65099393 100644 --- a/tests/unit/scanner/test_pipeline.py +++ b/tests/unit/scanner/test_pipeline.py @@ -191,10 +191,9 @@ def test_parse_project_stage_parse_failure_is_gating_error_defect(tmp_path) -> N assert by_path["enc.py"].location.line_start == 1 -def test_parse_project_stage_recursion_skip_stays_nongating_fact(tmp_path) -> None: - # The fail-closed change is scoped to PARSE failures: the recursion-limit - # file skip keeps its released non-gating FACT contract (it mirrors - # WLN-ENGINE-FUNCTION-SKIPPED, surfaced via summary.unanalyzed instead). +def test_parse_project_stage_recursion_skip_is_gate_eligible(tmp_path) -> None: + # A recursion-limit file skip means policy rules never ran for the file. It + # must be a gate-eligible under-scan defect, not a green severity result. from wardline.core.finding import Kind, Severity expr = "p" + " + p" * 3000 @@ -210,5 +209,6 @@ def test_parse_project_stage_recursion_skip_stays_nongating_fact(tmp_path) -> No ) skips = [f for f in result.parse_findings if f.rule_id == "WLN-ENGINE-FILE-SKIPPED"] assert len(skips) == 1 - assert skips[0].kind is Kind.FACT - assert skips[0].severity is Severity.NONE + assert skips[0].kind is Kind.DEFECT + assert skips[0].severity is Severity.ERROR + assert skips[0].location.line_start == 1 From 3826e246d5e43fa9edeb9e29a0ea7417e8e85953 Mon Sep 17 00:00:00 2001 From: John Morrissey Date: Fri, 12 Jun 2026 08:03:29 +1000 Subject: [PATCH 150/186] fix(diagnostics): narrow native import allowlist --- src/wardline/scanner/diagnostics.py | 43 ++++++++++---------------- tests/unit/scanner/test_diagnostics.py | 33 ++++++++++++++++---- 2 files changed, 43 insertions(+), 33 deletions(-) diff --git a/src/wardline/scanner/diagnostics.py b/src/wardline/scanner/diagnostics.py index dca5fb6f..c5ec986c 100644 --- a/src/wardline/scanner/diagnostics.py +++ b/src/wardline/scanner/diagnostics.py @@ -28,26 +28,20 @@ "weft_markers": frozenset({"external_boundary", "trust_boundary", "trusted"}), } -# Declarative native / first-party module prefixes. An import whose dotted module -# equals one of these or sits under it resolves cleanly EVEN WHEN it has no Python -# AST in the scanned tree — e.g. a compiled ``wardline.core`` extension after the -# Rust migration, which is definitionally unresolvable to an AST import analyzer. -# This is the SEAM the Rust migration extends: add the compiled submodule's dotted -# prefix here. Distinct from _BUILTIN_MARKER_IMPORTS (alias-specific, for the -# statically-modelled marker decorators); this is "any import from this prefix is -# first-party, resolve it". Matching is on dotted-component boundaries, so -# ``wardline.core`` does NOT swallow an unrelated ``wardline.core_helpers``. -_NATIVE_FIRST_PARTY_PREFIXES: frozenset[str] = frozenset( - { - "wardline.core", - "wardline.decorators", - } -) - - -def _is_native_first_party(mod: str) -> bool: - """True if ``mod`` is, or is under, a declared native/first-party prefix.""" - return any(mod == prefix or mod.startswith(prefix + ".") for prefix in _NATIVE_FIRST_PARTY_PREFIXES) +# Declarative native / first-party imports. A compiled module may have no Python +# AST in the scanned tree, so exact known exports can resolve cleanly even when +# absent from ``project_modules``. This is alias-specific on purpose: a broad +# ``wardline.core`` or ``wardline.decorators`` prefix would hide misspelled exports +# and bogus submodules, which are real diagnostic coverage gaps. +_NATIVE_FIRST_PARTY_IMPORTS: dict[str, frozenset[str]] = { + "wardline.core.registry": frozenset({"REGISTRY", "REGISTRY_VERSION", "RegistryEntry"}), +} + + +def _is_native_first_party_import(mod: str, alias: str) -> bool: + """True for exact declared native/first-party imports.""" + names = _NATIVE_FIRST_PARTY_IMPORTS.get(mod) + return names is not None and alias in names # code -> (rule_id, severity, kind) @@ -347,13 +341,6 @@ def diagnose_unknown_imports( continue if mod in project_modules: continue - # Skip declared native / first-party modules. A compiled wardline.core - # extension has no Python AST so it is absent from project_modules, but it - # is first-party, not an external precision gap — resolve it via the - # declarative allowlist. (Only DECLARED prefixes; a genuine unknown - # third-party import still falls through to a FACT below.) - if _is_native_first_party(mod): - continue # Skip Python stdlib modules — any import whose top-level name # appears in ``sys.stdlib_module_names`` is resolvable at runtime # by definition and is not a precision gap. @@ -384,6 +371,8 @@ def diagnose_unknown_imports( for alias in node.names: if _is_builtin_marker_import(mod, alias.name): continue + if _is_native_first_party_import(mod, alias.name): + continue if (mod, alias.name) in stdlib_keys: continue unresolved_aliases.append(alias.name) diff --git a/tests/unit/scanner/test_diagnostics.py b/tests/unit/scanner/test_diagnostics.py index 5d73360a..368d5bb0 100644 --- a/tests/unit/scanner/test_diagnostics.py +++ b/tests/unit/scanner/test_diagnostics.py @@ -222,10 +222,10 @@ def test_diagnose_unresolved_star_module_still_emits_fact() -> None: # --- Native / first-party module resolution (Task C) ------------------------- # When wardline.core becomes a compiled (PyO3) module it has NO Python AST in the # scanned tree, so it drops out of project_modules and would fire UNKNOWN-IMPORT. -# The declarative native-prefix allowlist resolves it. These tests SIMULATE the -# native case by passing an empty project_modules (the obvious "scan self" test -# is green today because the .py files are still present, so it would gate -# nothing). +# The declarative native import allowlist resolves exact known exports. These +# tests SIMULATE the native case by passing an empty project_modules (the obvious +# "scan self" test is green today because the .py files are still present, so it +# would gate nothing). def test_native_first_party_core_import_resolves_without_project_module() -> None: @@ -255,14 +255,35 @@ def test_native_allowlist_does_not_suppress_genuine_third_party() -> None: def test_native_allowlist_does_not_suppress_undeclared_wardline_submodule() -> None: - # Precision: only DECLARED native prefixes resolve. A wardline.* module that is - # neither a project module nor a declared native prefix must still report, so + # Precision: only DECLARED native imports resolve. A wardline.* module that is + # neither a project module nor a declared native import must still report, so # the allowlist can't silently swallow a real gap. tree = ast.parse("from wardline.experimental.zzz import q\n") out = diagnose_unknown_imports(tree=tree, module_path="x", project_modules=frozenset(), stdlib_keys=frozenset()) assert len(out) == 1 +def test_native_allowlist_does_not_suppress_unknown_decorator_export() -> None: + tree = ast.parse("from wardline.decorators import nonexistent\n") + out = diagnose_unknown_imports(tree=tree, module_path="x", project_modules=frozenset(), stdlib_keys=frozenset()) + assert len(out) == 1 + assert "nonexistent" in out[0][2] + + +def test_native_allowlist_does_not_suppress_nested_decorator_spoof() -> None: + tree = ast.parse("from wardline.decorators.evil import trusted\n") + out = diagnose_unknown_imports(tree=tree, module_path="x", project_modules=frozenset(), stdlib_keys=frozenset()) + assert len(out) == 1 + assert "wardline.decorators.evil" in out[0][2] + + +def test_native_allowlist_does_not_suppress_unknown_core_submodule() -> None: + tree = ast.parse("from wardline.core.evil import x\n") + out = diagnose_unknown_imports(tree=tree, module_path="x", project_modules=frozenset(), stdlib_keys=frozenset()) + assert len(out) == 1 + assert "wardline.core.evil" in out[0][2] + + def test_native_allowlist_prefix_boundary_is_dotted() -> None: # An adjacent prefix that shares a string-prefix but is NOT under the package # (wardline.core_helpers vs wardline.core) must NOT be suppressed — guards the From 78f015a2d7462ca8cdcb686c873749d7a6e7f2dc Mon Sep 17 00:00:00 2001 From: John Morrissey Date: Fri, 12 Jun 2026 08:05:33 +1000 Subject: [PATCH 151/186] fix(diagnostics): report unresolved plain imports --- src/wardline/scanner/diagnostics.py | 28 +++++++++++++++++++++++++- tests/unit/scanner/test_diagnostics.py | 28 +++++++++++++++++++++++++- 2 files changed, 54 insertions(+), 2 deletions(-) diff --git a/src/wardline/scanner/diagnostics.py b/src/wardline/scanner/diagnostics.py index c5ec986c..4cc6b40d 100644 --- a/src/wardline/scanner/diagnostics.py +++ b/src/wardline/scanner/diagnostics.py @@ -215,7 +215,7 @@ def build_unknown_import_findings( stdlib_keys=stdlib_keys, resolvable_star_modules=resolvable_star_modules, ): - package = detail.split()[1] if detail.startswith("from ") else detail + package = detail.split()[1] if detail.startswith(("from ", "import ")) else detail findings.append( Finding( rule_id="WLN-ENGINE-UNKNOWN-IMPORT", @@ -303,6 +303,8 @@ def diagnose_unknown_imports( import, de-duplicated by ``(source_module, target_package)``. Triggers: + * ``import X`` where ``X`` is not a project module, not a Python stdlib + module, and no stdlib_taint entry has X as its package. * ``from X import *`` where ``X`` is not a project module, not a Python stdlib module, and no stdlib_taint entry has X as its package. @@ -321,6 +323,30 @@ def diagnose_unknown_imports( findings: list[tuple[str, str, str]] = [] seen: set[tuple[str, str]] = set() for node in ast.walk(tree): + if isinstance(node, ast.Import): + if _is_type_checking_guarded(node, tree): + continue + for alias in node.names: + mod = alias.name + if not mod: + continue + if mod in project_modules: + continue + if _is_stdlib_module(mod): + continue + if any(key[0] == mod for key in stdlib_keys): + continue + key = (module_path, mod) + if key not in seen: + seen.add(key) + findings.append( + ( + module_path, + f"import {mod}", + f"external import {mod!r} cannot be resolved", + ) + ) + continue if not isinstance(node, ast.ImportFrom): continue # Skip relative imports entirely — they resolve inside the diff --git a/tests/unit/scanner/test_diagnostics.py b/tests/unit/scanner/test_diagnostics.py index 368d5bb0..51f3b8ef 100644 --- a/tests/unit/scanner/test_diagnostics.py +++ b/tests/unit/scanner/test_diagnostics.py @@ -148,6 +148,31 @@ def test_diagnose_unknown_imports_flags_external_named_import() -> None: assert "external_pkg" in out[0][2] +def test_diagnose_unknown_imports_flags_external_plain_import() -> None: + tree = ast.parse("import external_pkg\n") + out = diagnose_unknown_imports( + tree=tree, + module_path="m", + project_modules=frozenset({"m"}), + stdlib_keys=frozenset(), + ) + assert len(out) == 1 + assert out[0][1] == "import external_pkg" + assert "external_pkg" in out[0][2] + + +def test_diagnose_unknown_imports_flags_external_aliased_plain_import() -> None: + tree = ast.parse("import external_pkg as e\n") + out = diagnose_unknown_imports( + tree=tree, + module_path="m", + project_modules=frozenset({"m"}), + stdlib_keys=frozenset(), + ) + assert len(out) == 1 + assert out[0][1] == "import external_pkg" + + def test_diagnose_unknown_imports_skips_stdlib_and_project_and_relative() -> None: tree = ast.parse( "import os\n" @@ -165,7 +190,7 @@ def test_diagnose_unknown_imports_skips_stdlib_and_project_and_relative() -> Non def test_unknown_import_findings_are_facts() -> None: - tree = ast.parse("from external_pkg import thing\n") + tree = ast.parse("import external_pkg\n") findings = build_unknown_import_findings( [("pkg/mod.py", "pkg.mod", tree)], project_modules=frozenset({"pkg.mod"}), @@ -173,6 +198,7 @@ def test_unknown_import_findings_are_facts() -> None: assert len(findings) == 1 assert findings[0].kind == Kind.FACT assert findings[0].rule_id == "WLN-ENGINE-UNKNOWN-IMPORT" + assert findings[0].properties["package"] == "external_pkg" # Fingerprint stable from (module, package) — not message text. again = build_unknown_import_findings([("pkg/mod.py", "pkg.mod", tree)], project_modules=frozenset({"pkg.mod"})) assert findings[0].fingerprint == again[0].fingerprint From e426527c3c8625402a7cbbdd12b4274ed902de8e Mon Sep 17 00:00:00 2001 From: John Morrissey Date: Fri, 12 Jun 2026 08:10:55 +1000 Subject: [PATCH 152/186] fix(resolver): fail closed on duplicate FQNs --- src/wardline/scanner/diagnostics.py | 1 + .../scanner/taint/project_resolver.py | 83 +++++++++++++++++-- .../scanner/taint/test_project_resolver.py | 56 ++++++++++++- tests/unit/scanner/test_analyzer_isolation.py | 19 +++++ tests/unit/scanner/test_diagnostics.py | 3 + 5 files changed, 153 insertions(+), 9 deletions(-) diff --git a/src/wardline/scanner/diagnostics.py b/src/wardline/scanner/diagnostics.py index 4cc6b40d..6e126f84 100644 --- a/src/wardline/scanner/diagnostics.py +++ b/src/wardline/scanner/diagnostics.py @@ -49,6 +49,7 @@ def _is_native_first_party_import(mod: str, alias: str) -> bool: "L3_CONVERGENCE_BOUND": ("WLN-L3-CONVERGENCE-BOUND", Severity.WARN, Kind.METRIC), "L3_MONOTONICITY_VIOLATION": ("WLN-L3-MONOTONICITY-VIOLATION", Severity.ERROR, Kind.DEFECT), "L3_LOW_RESOLUTION": ("WLN-L3-LOW-RESOLUTION", Severity.INFO, Kind.METRIC), + "DUPLICATE_FQN": ("WLN-ENGINE-DUPLICATE-FQN", Severity.ERROR, Kind.DEFECT), } diff --git a/src/wardline/scanner/taint/project_resolver.py b/src/wardline/scanner/taint/project_resolver.py index 025e762c..e5e95262 100644 --- a/src/wardline/scanner/taint/project_resolver.py +++ b/src/wardline/scanner/taint/project_resolver.py @@ -26,6 +26,7 @@ from wardline.core.config import WardlineConfig from wardline.core.ruleset import ruleset_hash +from wardline.core.taints import combine from wardline.scanner.taint.callgraph import build_call_edges from wardline.scanner.taint.module_summariser import summarise_module from wardline.scanner.taint.propagation import propagate_callgraph_taints @@ -74,6 +75,70 @@ def _cached_summaries_match_module( return frozenset(actual) == expected +def _duplicate_fqn_diagnostics(modules: Sequence[ModuleInput]) -> tuple[tuple[str, str], ...]: + locations_by_fqn: dict[str, list[str]] = {} + for module in modules: + for entity in module.entities: + line = entity.location.line_start if entity.location.line_start is not None else 1 + locations_by_fqn.setdefault(entity.qualname, []).append( + f"{module.module_path} ({entity.location.path}:{line})" + ) + + diagnostics: list[tuple[str, str]] = [] + for fqn in sorted(locations_by_fqn): + locations = locations_by_fqn[fqn] + if len(locations) < 2: + continue + preview = ", ".join(locations[:5]) + if len(locations) > 5: + preview += f", ... ({len(locations)} total)" + diagnostics.append( + ( + "DUPLICATE_FQN", + ( + f"Duplicate function qualname {fqn!r} across {len(locations)} entities: {preview}; " + "project taint summaries are keyed by qualname and cannot disambiguate these definitions" + ), + ) + ) + return tuple(diagnostics) + + +def _merge_edges(target: dict[str, frozenset[str]], new: dict[str, frozenset[str]]) -> None: + for fqn, callees in new.items(): + existing = target.get(fqn) + target[fqn] = callees if existing is None else frozenset((*existing, *callees)) + + +def _add_counts(target: dict[str, int], new: dict[str, int]) -> None: + for fqn, count in new.items(): + target[fqn] = target.get(fqn, 0) + count + + +def _merge_taint_source(existing: TaintSourceClass, incoming: TaintSourceClass) -> TaintSourceClass: + if existing == incoming: + return existing + return "fallback" + + +def _merge_summary_maps( + summaries: Sequence[FunctionSummary], +) -> tuple[dict[str, TaintState], dict[str, TaintState], dict[str, TaintSourceClass]]: + taint_map: dict[str, TaintState] = {} + return_taint_map: dict[str, TaintState] = {} + taint_sources: dict[str, TaintSourceClass] = {} + for summary in summaries: + if summary.fqn in taint_map: + taint_map[summary.fqn] = combine(taint_map[summary.fqn], summary.body_taint) + return_taint_map[summary.fqn] = combine(return_taint_map[summary.fqn], summary.return_taint) + taint_sources[summary.fqn] = _merge_taint_source(taint_sources[summary.fqn], summary.taint_source) + continue + taint_map[summary.fqn] = summary.body_taint + return_taint_map[summary.fqn] = summary.return_taint + taint_sources[summary.fqn] = summary.taint_source + return taint_map, return_taint_map, taint_sources + + def resolve_project_taints( *, modules: Sequence[ModuleInput], @@ -96,6 +161,7 @@ def resolve_project_taints( "dirty_modules=frozenset() explicitly if nothing has changed" ) + resolver_diagnostics = list(_duplicate_fqn_diagnostics(modules)) project_fqns = frozenset(e.qualname for m in modules for e in m.entities) all_classes = frozenset(c for m in modules for c in m.class_qualnames) @@ -120,9 +186,9 @@ def resolve_project_taints( module_prefix=m.module_path, project_fqns=project_fqns, ) - edges.update(m_edges) - resolved_counts.update(m_resolved) - unresolved_counts.update(m_unresolved) + _merge_edges(edges, m_edges) + _add_counts(resolved_counts, m_resolved) + _add_counts(unresolved_counts, m_unresolved) call_site_callees.update(m_callees) call_site_implicit_receivers.update(m_implicit_receivers) call_site_candidate_callees.update(m_candidate_callees) @@ -130,7 +196,10 @@ def resolve_project_taints( # Transitive dirty frontier (performance over-approximation — bounds which # clean modules skip provider re-invocation; NOT a correctness gate). if summary_cache is not None and dirty_modules is not None: - fqn_to_module = {e.qualname: m.module_path for m in modules for e in m.entities} + fqn_to_module: dict[str, str] = {} + for m in modules: + for e in m.entities: + fqn_to_module.setdefault(e.qualname, m.module_path) frontier = ReverseModuleIndex.from_forward_edges( {k: set(v) for k, v in edges.items()}, fqn_to_module=fqn_to_module, @@ -177,9 +246,7 @@ def resolve_project_taints( if summary_cache is not None: summary_cache.put(cache_key, fresh) - taint_map: dict[str, TaintState] = {s.fqn: s.body_taint for s in summaries} - return_taint_map: dict[str, TaintState] = {s.fqn: s.return_taint for s in summaries} - taint_sources: dict[str, TaintSourceClass] = {s.fqn: s.taint_source for s in summaries} + taint_map, return_taint_map, taint_sources = _merge_summary_maps(summaries) refined, provenance, diagnostics, scc_iteration_counts = propagate_callgraph_taints( edges={k: set(v) for k, v in edges.items()}, @@ -225,6 +292,6 @@ def resolve_project_taints( call_site_implicit_receivers=MappingProxyType(call_site_implicit_receivers), call_site_candidate_callees=MappingProxyType(call_site_candidate_callees), taint_provenance=MappingProxyType(dict(provenance)), - diagnostics=tuple(diagnostics), + diagnostics=(*resolver_diagnostics, *diagnostics), metadata=metadata, ) diff --git a/tests/unit/scanner/taint/test_project_resolver.py b/tests/unit/scanner/taint/test_project_resolver.py index 003df036..fc07ffe2 100644 --- a/tests/unit/scanner/taint/test_project_resolver.py +++ b/tests/unit/scanner/taint/test_project_resolver.py @@ -8,7 +8,7 @@ from wardline.core.taints import TaintState as T from wardline.scanner.ast_primitives import build_import_alias_map from wardline.scanner.index import discover_class_qualnames, discover_file_entities -from wardline.scanner.taint.function_level import seed_function_taints +from wardline.scanner.taint.function_level import FunctionSeed, seed_function_taints from wardline.scanner.taint.project_resolver import _RESOLVER_VERSION, ModuleInput, resolve_project_taints from wardline.scanner.taint.provider import ( DefaultTaintSourceProvider, @@ -59,6 +59,28 @@ def _module_input(module: str, src: str, provider) -> ModuleInput: ) +def _module_input_with_seed(module: str, path: str, src: str, *, body_taint: T, return_taint: T) -> ModuleInput: + tree = ast.parse(src) + entities = tuple(discover_file_entities(tree, module=module, path=path)) + seeds = { + entity.qualname: FunctionSeed( + qualname=entity.qualname, + body_taint=body_taint, + return_taint=return_taint, + source="provider", + ) + for entity in entities + } + return ModuleInput( + module_path=module, + entities=entities, + class_qualnames=discover_class_qualnames(tree, module=module), + alias_map=build_import_alias_map(tree, module_path=module), + seeds=seeds, + source_bytes=src.encode("utf-8"), + ) + + def test_transitive_raw_flows_across_modules_and_self_method() -> None: provider = _RawLeafProvider() inputs = [ @@ -266,6 +288,38 @@ def fingerprint(self) -> str: assert result.return_taint_map["m.plain"] == result.taint_map["m.plain"] +def test_duplicate_project_fqns_emit_diagnostic_and_do_not_overwrite_taints() -> None: + modules = [ + _module_input_with_seed( + "pkg.foo", + "pkg/foo.py", + "def f(p):\n return p\n", + body_taint=T.EXTERNAL_RAW, + return_taint=T.EXTERNAL_RAW, + ), + _module_input_with_seed( + "pkg.foo", + "src/pkg/foo.py", + "def f(p):\n return 'safe'\n", + body_taint=T.INTEGRAL, + return_taint=T.INTEGRAL, + ), + ] + + result = resolve_project_taints(modules=modules, provider_fingerprint="dup-test-v1") + + assert result.diagnostics == ( + ( + "DUPLICATE_FQN", + "Duplicate function qualname 'pkg.foo.f' across 2 entities: " + "pkg.foo (pkg/foo.py:1), pkg.foo (src/pkg/foo.py:1); project taint summaries are keyed by " + "qualname and cannot disambiguate these definitions", + ), + ) + assert result.taint_map["pkg.foo.f"] == T.EXTERNAL_RAW + assert result.return_taint_map["pkg.foo.f"] == T.EXTERNAL_RAW + + def test_cache_miss_on_changed_source_recomputes() -> None: # A module whose source changes gets a different cache_key -> miss -> # fresh summary, even if the caller forgets to mark it dirty. diff --git a/tests/unit/scanner/test_analyzer_isolation.py b/tests/unit/scanner/test_analyzer_isolation.py index 682a5e57..6a328e3d 100644 --- a/tests/unit/scanner/test_analyzer_isolation.py +++ b/tests/unit/scanner/test_analyzer_isolation.py @@ -276,6 +276,25 @@ def test_null_byte_file_is_gate_eligible(tmp_path) -> None: assert gate_decision(result, Severity.ERROR).tripped is True +def test_duplicate_project_fqns_trip_default_fail_on_error_gate(tmp_path) -> None: + # A default repository-root scan maps both pkg/foo.py and src/pkg/foo.py to + # pkg.foo. Duplicate function qualnames must fail loud before a later module + # can hide an unsafe summary under the same project-wide key. + proj = tmp_path / "proj" + proj.mkdir() + _write(proj, "pkg/foo.py", "def f(p):\n return p\n") + _write(proj, "src/pkg/foo.py", "def f(p):\n return 'safe'\n") + + result = run_scan(proj) + + duplicates = [f for f in result.findings if f.rule_id == "WLN-ENGINE-DUPLICATE-FQN"] + assert len(duplicates) == 1 + assert duplicates[0].kind is Kind.DEFECT + assert duplicates[0].severity is Severity.ERROR + assert "pkg.foo.f" in duplicates[0].message + assert gate_decision(result, Severity.ERROR).tripped is True + + def test_parse_error_gates_in_secure_population_but_baseline_annotates(tmp_path) -> None: # Secure-by-default precedent: a committed baseline ANNOTATES the parse-error # defect (suppressed in the emitted findings) but cannot clear the secure gate; diff --git a/tests/unit/scanner/test_diagnostics.py b/tests/unit/scanner/test_diagnostics.py index 51f3b8ef..6007a748 100644 --- a/tests/unit/scanner/test_diagnostics.py +++ b/tests/unit/scanner/test_diagnostics.py @@ -44,6 +44,7 @@ def test_l3_diagnostic_findings_map_code_to_severity() -> None: ("L3_MONOTONICITY_VIOLATION", "func x moved up"), ("L3_CONVERGENCE_BOUND", "SCC of size 3 hit bound"), ("L3_LOW_RESOLUTION", "Function m.f has 80% unresolved (4/5)"), + ("DUPLICATE_FQN", "duplicate m.f"), ] out = {f.rule_id: f for f in build_diagnostic_findings(diags)} assert out["WLN-L3-MONOTONICITY-VIOLATION"].severity == Severity.ERROR @@ -51,6 +52,8 @@ def test_l3_diagnostic_findings_map_code_to_severity() -> None: assert out["WLN-L3-CONVERGENCE-BOUND"].severity == Severity.WARN assert out["WLN-L3-LOW-RESOLUTION"].severity == Severity.INFO assert out["WLN-L3-LOW-RESOLUTION"].kind == Kind.METRIC + assert out["WLN-ENGINE-DUPLICATE-FQN"].severity == Severity.ERROR + assert out["WLN-ENGINE-DUPLICATE-FQN"].kind == Kind.DEFECT def test_unknown_diagnostic_code_is_error_not_silent() -> None: From be430be71487d4b5f6dafad994d6cf8a967e38a9 Mon Sep 17 00:00:00 2001 From: John Morrissey Date: Fri, 12 Jun 2026 20:20:26 +1000 Subject: [PATCH 153/186] fix(filigree-routes): one dialect-aware parser for every route derived from --filigree-url (dogfood-4 A3/A4, weft-d0df42c739) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The configured value stays the scan-results endpoint (canonical per federation-topology.md), but route derivation moves to a single shared parser, filigree_api_base_url, that understands every accepted form: bare origin, /api base, classic /api/weft/scan-results, project-scoped /api/p//weft/scan-results, and ?project= query. Fixes, all from one root cause (classic-only marker parsing): - scan_file_findings rejected the project-scoped endpoint the installer itself writes (A3) - dossier/decorator_coverage work-joins appended /api to the scoped endpoint and 404'd on every row (A4) - promote_url_from_weft silently DROPPED a ?project= query, landing promotes in the default project (latent misroute, found in review) When a project is pinned, derived routes always use the path-scoped /api/p// dialect: filigree dual-mounts all routes under it, while ?project= is honored only on weft-scoped paths. Verified live against filigree :8749 — the scoped entity-associations route answers 200 where the old derivation was dead. Co-Authored-By: Claude Fable 5 --- src/wardline/core/filigree_emit.py | 22 ++++++++++++++ src/wardline/core/filigree_issue.py | 24 +++++++-------- src/wardline/filigree/dossier_client.py | 17 ++++------- tests/unit/core/test_filigree_issue.py | 34 ++++++++++++++++++++-- tests/unit/filigree/test_dossier_client.py | 10 +++++++ 5 files changed, 79 insertions(+), 28 deletions(-) diff --git a/src/wardline/core/filigree_emit.py b/src/wardline/core/filigree_emit.py index d89136ca..6cbcb7ac 100644 --- a/src/wardline/core/filigree_emit.py +++ b/src/wardline/core/filigree_emit.py @@ -124,6 +124,28 @@ def filigree_destination(url: str | None) -> dict[str, Any]: return {"url": url, "project": project, "project_pinned": project is not None} +def filigree_api_base_url(url: str) -> str: + """Normalize any accepted Filigree URL form — bare origin, ``/api`` base, or a + classic / project-scoped ``…/weft/scan-results`` endpoint, with or without + ``?project=`` — to the API base every sibling route derives from. + + When the input pins a project (either dialect), the base is the path-scoped + ``…/api/p/`` form: Filigree dual-mounts every route under it, whereas the + ``?project=`` query is honored only on weft-scoped paths — so the path dialect is + the only one that also scopes classic routes (entity-associations, the dossier + work-join). The single parser exists so the emit echo, the promote route, and the + work-join can never disagree about what one configured URL means (dogfood-4 A3/A4).""" + parts = urllib.parse.urlsplit(url) + if parts.scheme.lower() not in _ALLOWED_SCHEMES: + raise FiligreeEmitError(f"filigree URL must use http or https; got scheme {parts.scheme!r} in {url!r}") + segments = [s for s in parts.path.split("/") if s] + base = segments[: segments.index("api") + 1] if "api" in segments else [*segments, "api"] + project = filigree_url_project(url) + if project is not None and base[-2:] != ["p", project]: + base += ["p", project] + return urllib.parse.urlunsplit((parts.scheme, parts.netloc, "/" + "/".join(base), "", "")) + + # --- transport + emitter ----------------------------------------------------- diff --git a/src/wardline/core/filigree_issue.py b/src/wardline/core/filigree_issue.py index dd091e0d..c700c78e 100644 --- a/src/wardline/core/filigree_issue.py +++ b/src/wardline/core/filigree_issue.py @@ -21,31 +21,27 @@ from typing import Any, Protocol from wardline.core.errors import FiligreeEmitError +from wardline.core.filigree_emit import filigree_api_base_url from wardline.core.finding import FINGERPRINT_SCHEME, format_fingerprint from wardline.core.http import read_response_text from wardline.loomweave.identity import SeiResolver _ALLOWED_SCHEMES = ("http", "https") -_WEFT_MARKER = "/api/weft/" def promote_url_from_weft(weft_url: str) -> str: - """Derive the promote route from the configured Weft scan-results URL — both - live under /api/weft/. Reject a URL that isn't a Weft endpoint (a clear config - error rather than a 404 against a wrong host).""" - idx = weft_url.find(_WEFT_MARKER) - if idx == -1: - raise FiligreeEmitError(f"filigree URL must be a Weft endpoint containing {_WEFT_MARKER!r}: {weft_url!r}") - base = weft_url[: idx + len(_WEFT_MARKER)] - return base + "findings/promote" + """Derive the promote route from the configured Filigree URL. Accepts every form + ``filigree_api_base_url`` does; a pinned project (``/api/p//…`` path or + ``?project=`` query) is preserved as the path-scoped dialect, so a scoped emit + config can no longer promote into the default project (dogfood-4 A3).""" + return api_base_url_from_weft(weft_url) + "/weft/findings/promote" def api_base_url_from_weft(weft_url: str) -> str: - """Normalize a Weft scan-results URL to Filigree's ``/api`` base.""" - idx = weft_url.find(_WEFT_MARKER) - if idx == -1: - raise FiligreeEmitError(f"filigree URL must be a Weft endpoint containing {_WEFT_MARKER!r}: {weft_url!r}") - return weft_url[:idx].rstrip("/") + "/api" + """Normalize the configured Filigree URL to its API base — project-scoped when the + input pins a project. Thin alias over the shared dialect parser in + ``core/filigree_emit.py`` so every derived route agrees with the emit destination.""" + return filigree_api_base_url(weft_url) def build_promote_body( diff --git a/src/wardline/filigree/dossier_client.py b/src/wardline/filigree/dossier_client.py index 5f62599b..9e43ccbb 100644 --- a/src/wardline/filigree/dossier_client.py +++ b/src/wardline/filigree/dossier_client.py @@ -26,6 +26,7 @@ from wardline.core.dossier import TicketRef, WorkSection from wardline.core.errors import FiligreeEmitError +from wardline.core.filigree_emit import filigree_api_base_url from wardline.core.http import read_response_text from wardline.core.identity import ContentStatus, EntityBinding, content_status @@ -80,17 +81,11 @@ def _rows_of(parsed: Any) -> list[dict[str, Any]]: def _api_base_url(url: str) -> str: - """Normalize an origin/API/scan-results URL to the Filigree API base.""" - parsed = urllib.parse.urlsplit(url.rstrip("/")) - scheme = parsed.scheme.lower() - if scheme not in _ALLOWED_SCHEMES: - raise FiligreeEmitError(f"filigree dossier URL must use http or https; got scheme {scheme!r} in {url!r}") - path = parsed.path.rstrip("/") - if path.endswith("/api/weft/scan-results"): - path = path[: -len("/weft/scan-results")] - elif not path.endswith("/api"): - path = f"{path}/api" if path else "/api" - return urllib.parse.urlunsplit((parsed.scheme, parsed.netloc, path, "", "")).rstrip("/") + """Normalize an origin/API/scan-results URL (classic or project-scoped, either + dialect) to the Filigree API base, via the shared parser in + ``core/filigree_emit.py``. Dogfood-4 A4 was this function appending ``/api`` to a + project-scoped endpoint, 404ing every work-join on the wired-up repo.""" + return filigree_api_base_url(url) class FiligreeWorkProvider: diff --git a/tests/unit/core/test_filigree_issue.py b/tests/unit/core/test_filigree_issue.py index 055702ff..6d5402e5 100644 --- a/tests/unit/core/test_filigree_issue.py +++ b/tests/unit/core/test_filigree_issue.py @@ -70,9 +70,37 @@ def test_api_base_url_derived_from_scan_results_url(): assert api_base_url_from_weft("http://h:8628/api/weft/scan-results") == "http://h:8628/api" -def test_promote_url_rejects_non_weft_url(): - with pytest.raises(FiligreeEmitError, match="/api/weft/"): - promote_url_from_weft("http://h/api/something/else") +def test_promote_url_preserves_project_scoped_path_dialect(): + # Dogfood-4 A3: the installer writes the /api/p// scoped endpoint; the promote + # route must stay inside that scope or the promote lands in the default project. + assert ( + promote_url_from_weft("http://127.0.0.1:8749/api/p/lacuna/weft/scan-results") + == "http://127.0.0.1:8749/api/p/lacuna/weft/findings/promote" + ) + + +def test_api_base_url_preserves_project_scoped_path_dialect(): + assert ( + api_base_url_from_weft("http://127.0.0.1:8749/api/p/lacuna/weft/scan-results") + == "http://127.0.0.1:8749/api/p/lacuna" + ) + + +def test_query_project_dialect_converts_to_path_scope(): + # ?project= is honored only on weft-scoped routes server-side; derived routes must + # carry the scope as the path dialect, which Filigree dual-mounts for ALL routes. + assert ( + api_base_url_from_weft("http://h:8749/api/weft/scan-results?project=lacuna") == "http://h:8749/api/p/lacuna" + ) + assert ( + promote_url_from_weft("http://h:8749/api/weft/scan-results?project=lacuna") + == "http://h:8749/api/p/lacuna/weft/findings/promote" + ) + + +def test_promote_url_rejects_non_http_scheme(): + with pytest.raises(FiligreeEmitError, match="http or https"): + promote_url_from_weft("file:///etc/passwd") def test_file_returns_issue_id_on_200(): diff --git a/tests/unit/filigree/test_dossier_client.py b/tests/unit/filigree/test_dossier_client.py index e015791c..803720d8 100644 --- a/tests/unit/filigree/test_dossier_client.py +++ b/tests/unit/filigree/test_dossier_client.py @@ -67,6 +67,16 @@ def test_scan_results_url_is_normalized_to_api_origin_for_associations() -> None assert t.calls[0][0].startswith("http://filigree.example/api/entity-associations?") +def test_project_scoped_scan_results_url_keeps_scope_on_associations() -> None: + # Dogfood-4 A4: the lacuna config value. The old normalizer appended /api to the + # scoped endpoint, 404ing the work-join on every dossier/decorator_coverage row. + t = FakeTransport(Response(status=200, body=_rows())) + + FiligreeWorkProvider("http://127.0.0.1:8749/api/p/lacuna/weft/scan-results", transport=t).work(_BINDING) + + assert t.calls[0][0].startswith("http://127.0.0.1:8749/api/p/lacuna/entity-associations?") + + @pytest.mark.parametrize( "url", [ From 64847ce9decd619cf3f5c685652b8d5043fb1837 Mon Sep 17 00:00:00 2001 From: John Morrissey Date: Fri, 12 Jun 2026 20:38:33 +1000 Subject: [PATCH 154/186] =?UTF-8?q?fix(autofix):=20resolve=20root=20before?= =?UTF-8?q?=20relativizing=20=E2=80=94=20fix=20crashed=20on=20any=20invoca?= =?UTF-8?q?tion=20with=20--root=20.=20(dogfood-4=20A1,=20weft-6090d9f76d)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit run_autofix resolved each candidate file but relativized against the caller's unresolved root; with the MCP server's literal --root . that raised ValueError ('is not in the subpath of .') before doing anything, dry-run included. Resolve root once at entry. Regression test runs the relative-root dry-run end to end. Co-Authored-By: Claude Fable 5 --- src/wardline/core/autofix.py | 8 +++++++- tests/unit/core/test_autofix.py | 30 ++++++++++++++++++++++++++++++ 2 files changed, 37 insertions(+), 1 deletion(-) diff --git a/src/wardline/core/autofix.py b/src/wardline/core/autofix.py index b4939fd0..f8660bd3 100644 --- a/src/wardline/core/autofix.py +++ b/src/wardline/core/autofix.py @@ -98,12 +98,18 @@ def run_autofix( Returns a mapping of relative_path -> list of description strings of applied fixes. """ applied: dict[str, list[str]] = defaultdict(list) + # Resolve once up front: the MCP server passes the literal `--root .`, and + # relativizing a resolved file path against the UNRESOLVED root raises + # ValueError ("is not in the subpath of '.'") — dogfood-4 A1, the crash that + # made the only autofix verb unusable. Every comparison below works on the + # resolved root. + root = root.resolve() # Group findings by file path (resolved relative to root) by_file: dict[Path, list[Finding]] = defaultdict(list) for f in findings: if f.rule_id == "PY-WL-111" and f.location.path: full_path = (root / f.location.path).resolve() - if full_path.is_relative_to(root.resolve()): + if full_path.is_relative_to(root): by_file[full_path].append(f) exception_name = config.boundary_exception diff --git a/tests/unit/core/test_autofix.py b/tests/unit/core/test_autofix.py index 9952eafc..2639abb8 100644 --- a/tests/unit/core/test_autofix.py +++ b/tests/unit/core/test_autofix.py @@ -44,6 +44,36 @@ def test_autofix_basic_assert(tmp_path: Path) -> None: assert new_content == expected +def test_autofix_relative_root_dry_run(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None: + # Dogfood-4 A1: the MCP server launches with the literal `--root .`; the + # resolved file path relativized against the unresolved root raised + # ValueError ("is not in the subpath of '.'") on ANY invocation, dry-run + # included. The relative root must work end to end. + content = """def my_func(x): + assert x > 0 + return x +""" + (tmp_path / "test_file.py").write_text(content, encoding="utf-8") + findings = [ + Finding( + rule_id="PY-WL-111", + message="assert-only boundary check", + severity=Severity.ERROR, + kind=Kind.DEFECT, + location=Location(path="test_file.py", line_start=1), + fingerprint="test_fp", + ) + ] + config = WardlineConfig(autofix={"boundary_exception": "ValueError"}) + monkeypatch.chdir(tmp_path) + + result = run_autofix(findings, config, Path("."), dry_run=True) + + assert "test_file.py" in result + # dry-run: reported but not written + assert (tmp_path / "test_file.py").read_text(encoding="utf-8") == content + + def test_autofix_assert_with_msg(tmp_path: Path) -> None: content = """def my_func(x): assert x > 0, "x must be positive" From a24d3ff2346994c32087c1ec8d2c05f6f7afcbde Mon Sep 17 00:00:00 2001 From: John Morrissey Date: Fri, 12 Jun 2026 20:41:05 +1000 Subject: [PATCH 155/186] fix(mcp-scan): suppress legis_artifact under summary_only unless explicitly requested (dogfood-4 B6, weft-74200b0acf) summary_only is documented as the smallest 'did the gate pass?' payload, but a provisioned WARDLINE_LEGIS_ARTIFACT_KEY auto-attached the ~56KB verbatim signed artifact into it, blowing the MCP token cap. Under summary_only:true the artifact now requires the explicit legis_artifact:true opt-in; asking for both honors both. Co-Authored-By: Claude Fable 5 --- src/wardline/mcp/server.py | 12 +++++++++-- tests/unit/mcp/test_server_legis_artifact.py | 21 ++++++++++++++++++++ 2 files changed, 31 insertions(+), 2 deletions(-) diff --git a/src/wardline/mcp/server.py b/src/wardline/mcp/server.py index 5155347b..3622db35 100644 --- a/src/wardline/mcp/server.py +++ b/src/wardline/mcp/server.py @@ -1156,7 +1156,8 @@ def _scan( "type": "object", "description": "OPTIONAL: the verbatim-postable signed scan object for legis POST /wardline/scan-results. " "Present only when a WARDLINE_LEGIS_ARTIFACT_KEY is provisioned or legis_artifact:true was passed, AND " - "building it did not fail (a signing refusal omits it).", + "building it did not fail (a signing refusal omits it). Suppressed under summary_only:true unless " + "legis_artifact:true is passed explicitly — summary_only promises the smallest gate payload.", "properties": { "scanner_identity": {"type": "string", "description": "wardline@."}, "rule_set_version": {"type": "string", "description": "Hash of the effective ruleset."}, @@ -1688,8 +1689,15 @@ def _attach_legis_artifact( ) key_str = load_legis_artifact_key(path) - if key_str is None and not bool(args.get("legis_artifact")): + explicit = bool(args.get("legis_artifact")) + if key_str is None and not explicit: return # not requested — default response unchanged + if _bool_arg(args, "summary_only", False) and not explicit: + # summary_only promises the smallest "did the gate pass?" payload; a + # provisioned key must not auto-attach a ~56KB verbatim artifact into it + # (dogfood-4 B6 blew the MCP token cap exactly this way). An explicit + # legis_artifact:true still wins when the caller asks for both. + return cfg = config_mod.load( _cfg(args, path) or weft_config_path(path), diff --git a/tests/unit/mcp/test_server_legis_artifact.py b/tests/unit/mcp/test_server_legis_artifact.py index 22da75fa..238db12a 100644 --- a/tests/unit/mcp/test_server_legis_artifact.py +++ b/tests/unit/mcp/test_server_legis_artifact.py @@ -60,6 +60,27 @@ def test_legis_artifact_unsigned_when_no_key(tmp_path, monkeypatch) -> None: assert "artifact_signature" not in out["legis_artifact"] +def test_legis_suppressed_under_summary_only_even_with_key(tmp_path, monkeypatch) -> None: + # Dogfood-4 B6: summary_only promises the smallest gate payload, but a + # provisioned key auto-attached a ~56KB artifact into it (blew the MCP token + # cap). With a key and summary_only:true the artifact must stay off. + monkeypatch.setenv(LEGIS_ARTIFACT_KEY_ENV, "testsecret") + repo = _committed_repo(tmp_path) + out = _scan({"summary_only": True}, repo, None, None) + assert "legis_artifact" not in out + assert "legis_artifact_status" not in out + + +def test_legis_explicit_opt_in_wins_over_summary_only(tmp_path, monkeypatch) -> None: + # The caller who asks for both gets both: explicit legis_artifact:true still + # attaches under summary_only. + monkeypatch.setenv(LEGIS_ARTIFACT_KEY_ENV, "testsecret") + repo = _committed_repo(tmp_path) + out = _scan({"summary_only": True, "legis_artifact": True}, repo, None, None) + assert "legis_artifact" in out + assert out["legis_artifact_status"]["signed"] is True + + def test_legis_clean_tree_with_key_is_signed(tmp_path, monkeypatch) -> None: # The positive arm of `signed = key and not dirty`: a key present on a CLEAN tree signs. monkeypatch.setenv(LEGIS_ARTIFACT_KEY_ENV, "testsecret") From 5f20f08cf118989d2597024a092b0d0529541287 Mon Sep 17 00:00:00 2001 From: John Morrissey Date: Fri, 12 Jun 2026 20:45:05 +1000 Subject: [PATCH 156/186] fix(doctor): url checks report the answering process's effective config with provenance (dogfood-4 B8, weft-dc7b805dc4) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit doctor reported loomweave.url/filigree.url 'not configured' while the same server was launched with both flags and using them successfully — it only ever read the env vars. The MCP server now threads both launch flags into machine_readable_doctor; each url verdict names its source (launch flag / env / honest absence) so the status surface can never silently disagree with the runtime wiring it vouches for. Co-Authored-By: Claude Fable 5 --- src/wardline/install/doctor.py | 28 ++++++++++++++++++---------- src/wardline/mcp/server.py | 16 +++++++++++++--- tests/unit/mcp/test_server_doctor.py | 21 +++++++++++++++++++++ 3 files changed, 52 insertions(+), 13 deletions(-) diff --git a/src/wardline/install/doctor.py b/src/wardline/install/doctor.py index 31d7cebd..6861a3d0 100644 --- a/src/wardline/install/doctor.py +++ b/src/wardline/install/doctor.py @@ -172,18 +172,25 @@ def _valid_http_url(url: str) -> bool: return parsed.scheme.lower() in {"http", "https"} and bool(parsed.netloc) -def _check_url(root: Path, key: str, *, fixed: bool) -> DoctorCheck: - # Sibling-endpoint config keys were retired (pending the hub shared-endpoint - # schema); a fixed endpoint comes only from the env var now, so that is what we - # validate. Live local discovery (.weft//ephemeral.port) is dynamic and - # not a doctor concern. +def _check_url(root: Path, key: str, *, fixed: bool, effective_url: str | None = None) -> DoctorCheck: + # Doctor must vouch for the EFFECTIVE config of the process answering it + # (dogfood-4 B8: it said "not configured" while the same server was launched + # with --loomweave-url/--filigree-url and using them successfully). Precedence + # mirrors runtime resolution: the launch flag the caller threads in, then the + # env var. Each verdict names its provenance so two surfaces can never + # silently disagree about WHICH config they describe. Live local discovery + # (.weft//ephemeral.port) is dynamic and not a doctor concern. env_key = "WARDLINE_LOOMWEAVE_URL" if key == "loomweave" else "WARDLINE_FILIGREE_URL" - url = os.environ.get(env_key) check_id = f"{key}.url" + if effective_url: + if _valid_http_url(effective_url): + return DoctorCheck(check_id, "ok", fixed=fixed, message=f"from --{key}-url launch flag") + return DoctorCheck(check_id, "error", fixed=False, message=f"invalid URL (launch flag): {effective_url!r}") + url = os.environ.get(env_key) if not url: - return DoctorCheck(check_id, "ok", fixed=fixed, message="not configured") + return DoctorCheck(check_id, "ok", fixed=fixed, message="not configured (no launch flag, no env)") if _valid_http_url(url): - return DoctorCheck(check_id, "ok", fixed=fixed) + return DoctorCheck(check_id, "ok", fixed=fixed, message=f"from env {env_key}") return DoctorCheck(check_id, "error", fixed=False, message=f"invalid URL: {url!r}") @@ -389,6 +396,7 @@ def machine_readable_doctor( *, fix: bool = False, filigree_url: str | None = None, + loomweave_url: str | None = None, transport: Transport | None = None, ) -> dict[str, Any]: """Return the shared machine-readable doctor shape, optionally repairing install bindings.""" @@ -408,8 +416,8 @@ def machine_readable_doctor( checks.append(_check_config(root, fixed=fix and not weft_config_path(root).exists())) checks.append(_check_mcp_registration(root, before=before)) checks.append(_check_marker_package()) - checks.append(_check_url(root, "loomweave", fixed=bindings_fixed)) - checks.append(_check_url(root, "filigree", fixed=bindings_fixed)) + checks.append(_check_url(root, "loomweave", fixed=bindings_fixed, effective_url=loomweave_url)) + checks.append(_check_url(root, "filigree", fixed=bindings_fixed, effective_url=filigree_url)) checks.append(_check_decorator_grammar()) checks.append(_check_scan_output_path(root)) checks.append(_check_auth_token(root)) diff --git a/src/wardline/mcp/server.py b/src/wardline/mcp/server.py index 3622db35..e2acedd5 100644 --- a/src/wardline/mcp/server.py +++ b/src/wardline/mcp/server.py @@ -3263,12 +3263,16 @@ def _doctor( *, started_at: float, filigree_url: str | None = None, + loomweave_url: str | None = None, ) -> dict[str, Any]: """The CLI `doctor --fix` envelope over MCP (A2, wardline-2ee1bbda82's sibling): install/federation health checks via the SAME machine_readable_doctor builder, plus the running server's self-identification + source-freshness verdict so an agent can detect a stale long-lived server without shelling out. Read-only by - default; `repair: true` is the explicit WRITE opt-in (mirrors CLI --fix).""" + default; `repair: true` is the explicit WRITE opt-in (mirrors CLI --fix). + + Both launch-flag URLs are threaded in so the url checks describe the EFFECTIVE + config of THIS server process, with provenance — not just env (dogfood-4 B8).""" from wardline.install.doctor import machine_readable_doctor from wardline.mcp.freshness import attach_server_identity @@ -3276,7 +3280,9 @@ def _doctor( flag = args.get("filigree_url") if flag is not None and not isinstance(flag, str): raise ToolError("filigree_url must be a string") - payload = machine_readable_doctor(root, fix=repair, filigree_url=flag or filigree_url) + payload = machine_readable_doctor( + root, fix=repair, filigree_url=flag or filigree_url, loomweave_url=loomweave_url + ) return attach_server_identity(payload, root=root, started_at=started_at) @@ -4003,7 +4009,11 @@ def _register_tools(self) -> None: Tool( **_DOCTOR_TOOL, handler=lambda args, root: _doctor( - args, root, started_at=self.started_at, filigree_url=self.filigree_url + args, + root, + started_at=self.started_at, + filigree_url=self.filigree_url, + loomweave_url=self.loomweave_url, ), ) ) diff --git a/tests/unit/mcp/test_server_doctor.py b/tests/unit/mcp/test_server_doctor.py index 9fe5f9a4..11d9bf07 100644 --- a/tests/unit/mcp/test_server_doctor.py +++ b/tests/unit/mcp/test_server_doctor.py @@ -41,6 +41,27 @@ def test_doctor_matches_cli_machine_readable_envelope(tmp_path: Path, monkeypatc assert mcp["ok"] == cli["ok"] +def test_doctor_url_checks_report_launch_flags_with_provenance(tmp_path: Path, monkeypatch) -> None: + """Dogfood-4 B8: doctor said loomweave.url/filigree.url 'not configured' while + the answering server was launched with both flags and using them. The url + checks must describe THIS process's effective config and name the source.""" + _isolate(tmp_path, monkeypatch) + payload = _doctor( + {}, + tmp_path, + started_at=time.time(), + filigree_url="http://127.0.0.1:8749/api/p/lacuna/weft/scan-results", + loomweave_url="http://127.0.0.1:9730", + ) + by_id = {c["id"]: c for c in payload["checks"]} + assert by_id["loomweave.url"]["message"] == "from --loomweave-url launch flag" + assert by_id["filigree.url"]["message"] == "from --filigree-url launch flag" + # And honest absence names what was checked, not a bare "not configured". + bare = _doctor({}, tmp_path, started_at=time.time()) + by_id = {c["id"]: c for c in bare["checks"]} + assert by_id["loomweave.url"]["message"] == "not configured (no launch flag, no env)" + + def test_doctor_reports_server_identity(tmp_path: Path, monkeypatch) -> None: _isolate(tmp_path, monkeypatch) now = time.time() From 3e20f6cb4e6104bc09a6cc8ba722722cd6461aa8 Mon Sep 17 00:00:00 2001 From: John Morrissey Date: Sat, 13 Jun 2026 06:14:26 +1000 Subject: [PATCH 157/186] fix(rekey): probe judges each store against its own scheme; bounded orphan output (dogfood A7, weft-dda1a6d8dd) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The read-only rekey probe compared every store's fingerprints against the wlfp1-reconstructed remap keys only, so a store ALREADY at the live wlfp2 scheme — a demonstrably healthy baseline whose 44/44 entries match and suppress findings in the same scan — read as 100% orphaned ("source moved/deleted…"). An agent consulting the probe before a migration would conclude the entire baseline is dead. Root cause + fix (core/rekey.py): - _store_fingerprints now reads each store's fingerprint_scheme header; probe() matches current-scheme stores against the CURRENT scan fingerprints (a rekey is a no-op for them) and only wlfp1/pre-scheme stores against the old remap keys. Healthy baseline now reports matched=N, orphaned=0, clean=true, no_op=true. - Current-scheme entries with no current finding are classified STALE (baseline drift) with their own cause string, never migration orphans. - run_rekey refuses when every populated store is already at the live scheme (the destructive twin: applying would push wlfp2 keys through the wlfp1 remap and orphan every healthy verdict), and _carry_store identity-carries an already-current snapshot store (mixed-scheme leg). - The rekey scan now runs EVERY frontend (python + rust): the stores hold RS-WL verdicts too, and a python-only scan misread each healthy Rust verdict as orphaned/stale. Token noise (same issue): probe/journal surfaces no longer dump every raw fingerprint — counts + a bounded sample (ORPHAN_SAMPLE_LIMIT=10) with an explicit remainder marker; the full list still lands verbatim in the on-disk migration journal. MCP payload keys: orphaned -> orphaned_count/ orphaned_sample (+ stale_*, current_scheme_stores, no_op), legs.orphaned -> orphaned_count/orphaned_sample; output schema updated to match. Live-verified against ~/lacuna's 44-entry baseline: before 0 carry / 44 orphaned / clean=false; after 44 match / 0 orphaned / clean=true / "no fingerprint migration pending". Co-Authored-By: Claude Fable 5 --- src/wardline/cli/rekey.py | 73 ++++++++++++----- src/wardline/core/rekey.py | 77 +++++++++++++++--- src/wardline/mcp/server.py | 121 +++++++++++++++++++++------- tests/unit/core/test_rekey_probe.py | 100 +++++++++++++++++++++++ tests/unit/mcp/test_server_rekey.py | 41 +++++++++- 5 files changed, 349 insertions(+), 63 deletions(-) diff --git a/src/wardline/cli/rekey.py b/src/wardline/cli/rekey.py index 422c3b71..bda6a50b 100644 --- a/src/wardline/cli/rekey.py +++ b/src/wardline/cli/rekey.py @@ -16,6 +16,8 @@ from wardline.core.errors import WardlineError from wardline.core.filigree_emit import FiligreeEmitter from wardline.core.rekey import ORPHAN_CAUSE as _ORPHAN_CAUSE +from wardline.core.rekey import ORPHAN_SAMPLE_LIMIT as _SAMPLE_LIMIT +from wardline.core.rekey import STALE_CAUSE as _STALE_CAUSE from wardline.core.rekey import Journal, ProbeReport, probe, resume_rekey, rollback, run_rekey from wardline.core.run import run_scan @@ -30,18 +32,38 @@ def _print_prescheme_caution() -> None: ) +def _echo_bounded( + fingerprints: tuple[str, ...], *, indent: str = " ", err: bool = True, remainder_hint: str = "" +) -> None: + """Counts + a bounded sample, never the whole list (bounded-by-default; the cut is + explicit so a sample never reads as the full set).""" + for of in fingerprints[:_SAMPLE_LIMIT]: + click.echo(f"{indent}{of}", err=err) + if len(fingerprints) > _SAMPLE_LIMIT: + more = len(fingerprints) - _SAMPLE_LIMIT + click.echo(f"{indent}… and {more} more (sample bounded{remainder_hint})", err=err) + + def _print_probe(report: ProbeReport) -> None: - click.echo(f"probe: {report.scanned_findings} finding(s) scanned; {report.matched} verdict(s) will carry.") + verb = "match current findings" if report.no_op else "will carry" + click.echo(f"probe: {report.scanned_findings} finding(s) scanned; {report.matched} verdict(s) {verb}.") + if report.no_op: + stores = ", ".join(report.current_scheme_stores) or "(no populated stores)" + click.echo(f"probe: no fingerprint migration pending — store(s) already at the current scheme: {stores}.") if report.orphaned: click.echo(f" {len(report.orphaned)} orphaned ({_ORPHAN_CAUSE}) — verdict will NOT carry:", err=True) - for of in report.orphaned: - click.echo(f" {of}", err=True) + _echo_bounded(report.orphaned) + if report.stale: + click.echo(f" {len(report.stale)} stale ({_STALE_CAUSE}):", err=True) + _echo_bounded(report.stale) for c in report.collisions: click.echo(f" COLLISION: {c.message}", err=True) if report.prescheme: _print_prescheme_caution() if report.clean: - click.echo("probe: clean — every stored verdict will carry.") + click.echo( + "probe: clean — nothing to migrate." if report.no_op else "probe: clean — every stored verdict will carry." + ) def _print_journal(journal: Journal) -> None: @@ -54,10 +76,12 @@ def _print_journal(journal: Journal) -> None: continue status = "done" if leg.done else "PENDING" click.echo(f" {leg.name}: {status} ({len(leg.carried)} carried, {len(leg.orphaned)} orphaned)") - # Surface the orphaned fingerprints, not just the count — a dropped verdict must - # never be silent (the original is recoverable from .rekey_snapshot/ until rollback). - for of in leg.orphaned: - click.echo(f" orphaned ({_ORPHAN_CAUSE}) — verdict NOT carried: {of}", err=True) + # Surface dropped verdicts loudly but BOUNDED (count + sample): the FULL orphan + # list is recorded verbatim in the migration journal, and the original verdicts + # stay recoverable from .rekey_snapshot/ until rollback. + if leg.orphaned: + click.echo(f" orphaned ({_ORPHAN_CAUSE}) — verdict NOT carried:", err=True) + _echo_bounded(tuple(leg.orphaned), indent=" ") for c in journal.collisions: click.echo(f" COLLISION: {c.message}", err=True) if journal.snapshot_prescheme: @@ -139,19 +163,26 @@ def rekey( raise SystemExit(0 if journal.complete else 1) resolved_url = resolve_filigree_url(filigree_url, path, config_path, strict_defaults=strict_defaults) - result = run_scan( - path, - config_path=config_path, - cache_dir=cache_dir, - trust_local_packs=trust_local_packs, - trusted_packs=trusted_packs, - strict_defaults=strict_defaults, - confine_to_root=True, - # Migration scans the project WITHOUT loading the stores it is about to - # rekey — they are still old-scheme and would SCHEME_MISMATCH. - skip_suppression=True, - ) - findings = result.findings + # The stores hold verdicts from EVERY frontend (RS-WL graduated to + # baseline-eligible — see core.rekey.is_join_population), so the probe/apply + # scan must cover every frontend too: a python-only scan misreads each healthy + # Rust verdict as orphaned/stale (A7, weft-dda1a6d8dd). + findings = [] + for lang in ("python", "rust"): + result = run_scan( + path, + config_path=config_path, + cache_dir=cache_dir, + trust_local_packs=trust_local_packs, + trusted_packs=trusted_packs, + strict_defaults=strict_defaults, + confine_to_root=True, + # Migration scans the project WITHOUT loading the stores it is about to + # rekey — they are still old-scheme and would SCHEME_MISMATCH. + skip_suppression=True, + lang=lang, + ) + findings.extend(result.findings) if probe_only: report = probe(path, findings) diff --git a/src/wardline/core/rekey.py b/src/wardline/core/rekey.py index f0386e60..5c2ee939 100644 --- a/src/wardline/core/rekey.py +++ b/src/wardline/core/rekey.py @@ -33,6 +33,16 @@ # Why a verdict can orphan (NOT only a source move) — the one explanation both # surfaces (CLI rekey output, MCP rekey payload) attach to every dropped verdict. ORPHAN_CAUSE = "source moved/deleted, or a custom multi-emit rule not surfacing taint_path_v0" +# Why a CURRENT-scheme entry can fail to match (NOT a migration orphan): the store is +# already at the live scheme, so a rekey would not touch it — a non-matching entry is +# baseline drift (the source changed since it was recorded), surfaced separately so a +# healthy-but-drifted store is never misread as a dead one (A7, weft-dda1a6d8dd). +STALE_CAUSE = "already at the current scheme but matches no current finding — baseline drift, not a rekey orphan" +# Bounded-by-default display: surfaces emit COUNTS plus at most this many example +# fingerprints with an explicit remainder marker (a bounded page never reads as the +# full set — agent_summary's convention). The full orphan list still lands verbatim +# in the migration journal on apply; the probe is advisory. +ORPHAN_SAMPLE_LIMIT = 10 # (store filename, list-key inside the YAML doc, version constant) — the three YAML # legs, in gate-criticality order (baseline first restores the local --fail-on gate). _STORES: tuple[tuple[str, str, int], ...] = ( @@ -250,6 +260,10 @@ def _carry_store(snapshot_path: Path, list_key: str, version: int, old_to_new: d Deterministic order: (rule_id, path, new fingerprint).""" loaded = _read_old_store(snapshot_path) raw_entries = loaded.get(list_key) or [] + # A snapshot store ALREADY at the live scheme needs no remap: its fingerprints are + # wlfp2 keys, and pushing them through the wlfp1->wlfp2 map would orphan every one + # (the mixed-scheme leg of A7, weft-dda1a6d8dd). Identity-carry it untouched. + already_current = loaded.get("fingerprint_scheme") == FINGERPRINT_SCHEME carried: list[str] = [] orphaned: list[str] = [] new_entries: list[dict[str, Any]] = [] @@ -257,7 +271,7 @@ def _carry_store(snapshot_path: Path, list_key: str, version: int, old_to_new: d old_fp = entry.get("fingerprint") if isinstance(entry, dict) else None if not isinstance(old_fp, str): continue # not a valid entry — nothing to carry or orphan - new_fp = old_to_new.get(old_fp) + new_fp = old_fp if already_current else old_to_new.get(old_fp) if new_fp is None: orphaned.append(old_fp) continue @@ -494,23 +508,27 @@ def _apply_filigree_leg(leg: Leg, findings: Sequence[Finding] | None, filigree: # --- S9: --probe (read-only cross-check; writes NOTHING) -------------------------- -def _store_fingerprints(root: Path) -> dict[str, set[str]]: - """The old_fps each live store currently records, read RAW (the stores are still - old-scheme until the migration runs). Read-only.""" - out: dict[str, set[str]] = {} +def _store_fingerprints(root: Path) -> dict[str, tuple[str | None, set[str]]]: + """Per live store: its ``fingerprint_scheme`` header (None when pre-scheme) and the + fingerprints it records, read RAW (a pre-migration store would SCHEME_MISMATCH the + enforcing loaders). The scheme is load-bearing: a store ALREADY at the live scheme + holds wlfp2 fingerprints, and judging it against the wlfp1-reconstructed remap keys + misreads every healthy entry as orphaned (A7, weft-dda1a6d8dd). Read-only.""" + out: dict[str, tuple[str | None, set[str]]] = {} state = paths.weft_state_dir(root) for name, key, _ver in _STORES: p = state / name if not p.is_file(): continue loaded = _read_old_store(p) + scheme = loaded.get("fingerprint_scheme") fps = { e["fingerprint"] for e in (loaded.get(key) or []) if isinstance(e, dict) and isinstance(e.get("fingerprint"), str) } if fps: - out[name] = fps + out[name] = (scheme if isinstance(scheme, str) else None, fps) return out @@ -539,6 +557,16 @@ class ProbeReport: collisions: tuple[RekeyCollision, ...] per_store: dict[str, int] # store name -> count of its old_fps with no current finding prescheme: bool = False # a live store predates the scheme stamp (possible formula drift) + # Stores ALREADY stamped with the live scheme (sorted). A rekey is a no-op for + # them; their entries are matched against the CURRENT fingerprints, never the + # wlfp1 remap keys (A7, weft-dda1a6d8dd). + current_scheme_stores: tuple[str, ...] = () + # Current-scheme entries with no current finding — baseline drift (STALE_CAUSE), + # not migration orphans; they do not dirty the probe. + stale: tuple[str, ...] = () + # True when every populated store already carries the live scheme (vacuously when + # none holds fingerprints): no fingerprint migration is pending. + no_op: bool = False @property def clean(self) -> bool: @@ -547,15 +575,28 @@ def clean(self) -> bool: def probe(root: Path, findings: Sequence[Finding]) -> ProbeReport: """Read-only dry run: which stored verdicts will carry, which orphan, any collisions. - Writes nothing — no snapshot, no journal, no store rewrite.""" + Each store is judged against ITS OWN scheme: a store still at wlfp1 (or pre-scheme) + against the reconstructed old-fingerprint remap keys, a store already at the live + scheme against the current scan's fingerprints (a rekey would not touch it, so a + healthy wlfp2 baseline reports matched=N / orphaned=0 / clean — A7, + weft-dda1a6d8dd). Writes nothing — no snapshot, no journal, no store rewrite.""" remaps = compute_old_new_fingerprints(findings) result = build_remap(remaps) keys = set(result.old_to_new) - store_fps = _store_fingerprints(root) + new_fps = {r.new_fp for r in remaps} matched: set[str] = set() orphaned: set[str] = set() + stale: set[str] = set() per_store: dict[str, int] = {} - for name, fps in store_fps.items(): + current_scheme_stores: list[str] = [] + migration_pending = False + for name, (scheme, fps) in sorted(_store_fingerprints(root).items()): + if scheme == FINGERPRINT_SCHEME: + current_scheme_stores.append(name) + matched |= fps & new_fps + stale |= fps - new_fps + continue + migration_pending = True store_orphans = fps - keys matched |= fps & keys orphaned |= store_orphans @@ -565,9 +606,16 @@ def probe(root: Path, findings: Sequence[Finding]) -> ProbeReport: scanned_findings=len(remaps), matched=len(matched), orphaned=tuple(sorted(orphaned)), + # Collisions stay LOUD even when no migration is pending: >1 old_fp collapsing + # to one new_fp means two CURRENT findings share a fingerprint — a discriminator + # bug (WLN-ENGINE-FINGERPRINT-COLLISION), not a migration artifact. A healthy + # baseline has none, so this never dirties the A7 clean-no-op verdict. collisions=result.collisions, per_store=per_store, prescheme=_dir_has_prescheme_store(paths.weft_state_dir(root)), + current_scheme_stores=tuple(current_scheme_stores), + stale=tuple(sorted(stale)), + no_op=not migration_pending, ) @@ -589,6 +637,17 @@ def run_rekey(root: Path, findings: Sequence[Finding], *, filigree: Any = None) "use `wardline rekey --rollback` to undo it, or delete " f"{snapshot_dir(root)} + {jpath} to migrate afresh." ) + # Refuse a rekey when every populated store ALREADY carries the live scheme: there + # is nothing to migrate, and re-keying wlfp2 entries through the wlfp1 remap would + # orphan every healthy verdict (the destructive twin of the A7 probe misread, + # weft-dda1a6d8dd). Checked BEFORE the snapshot — a refused run writes nothing. + populated_schemes = [scheme for scheme, _fps in _store_fingerprints(root).values()] + if populated_schemes and all(s == FINGERPRINT_SCHEME for s in populated_schemes): + raise WardlineError( + f"every store is already at the {FINGERPRINT_SCHEME} fingerprint scheme — " + "no fingerprint migration is pending; a rekey would only orphan healthy " + "verdicts. Nothing to do (run `wardline rekey --probe` for the per-store view)." + ) snapshot_stores(root) # must precede any store write journal = new_journal(compute_old_new_fingerprints(findings)) # Detect from the immutable snapshot (byte-identical to the pre-migration live stores) diff --git a/src/wardline/mcp/server.py b/src/wardline/mcp/server.py index e2acedd5..81636518 100644 --- a/src/wardline/mcp/server.py +++ b/src/wardline/mcp/server.py @@ -3280,9 +3280,7 @@ def _doctor( flag = args.get("filigree_url") if flag is not None and not isinstance(flag, str): raise ToolError("filigree_url must be a string") - payload = machine_readable_doctor( - root, fix=repair, filigree_url=flag or filigree_url, loomweave_url=loomweave_url - ) + payload = machine_readable_doctor(root, fix=repair, filigree_url=flag or filigree_url, loomweave_url=loomweave_url) return attach_server_identity(payload, root=root, started_at=started_at) @@ -3417,7 +3415,16 @@ def _rekey(args: dict[str, Any], root: Path, filigree: Any = None) -> dict[str, collisions, write NOTHING); `apply`/`resume`/`rollback` are explicit, mutually exclusive, WRITE-gated. The injected Filigree emitter (apply only) re-emits under the new fingerprints, best-effort like the CLI's --filigree-url leg.""" - from wardline.core.rekey import ORPHAN_CAUSE, Journal, probe, resume_rekey, rollback, run_rekey + from wardline.core.rekey import ( + ORPHAN_CAUSE, + ORPHAN_SAMPLE_LIMIT, + STALE_CAUSE, + Journal, + probe, + resume_rekey, + rollback, + run_rekey, + ) apply_ = _bool_arg(args, "apply", False) do_resume = _bool_arg(args, "resume", False) @@ -3427,8 +3434,9 @@ def _rekey(args: dict[str, Any], root: Path, filigree: Any = None) -> dict[str, path = _resolve_under_root(root, args["path"]) if args.get("path") else root def journal_block(journal: Journal) -> dict[str, Any]: - # Lean payload: carried as a COUNT (can be the whole store), orphans VERBATIM - # (a dropped verdict must never be silent — mirrors the CLI's per-orphan lines). + # Lean payload: carried as a COUNT (can be the whole store), orphans as a count + # plus a BOUNDED sample (bounded-by-default; never silent — the FULL list is + # recorded verbatim in the migration journal on disk). block: dict[str, Any] = { "complete": journal.complete, "fingerprint_scheme_from": journal.fingerprint_scheme_from, @@ -3443,7 +3451,8 @@ def journal_block(journal: Journal) -> dict[str, Any]: "name": leg.name, "done": leg.done, "carried_count": len(leg.carried), - "orphaned": list(leg.orphaned), + "orphaned_count": len(leg.orphaned), + "orphaned_sample": list(leg.orphaned[:ORPHAN_SAMPLE_LIMIT]), "debt": leg.debt, } for leg in journal.legs @@ -3469,33 +3478,47 @@ def journal_block(journal: Journal) -> dict[str, Any]: # Probe (the default) and apply both need the suppression-free scan: migration # scans WITHOUT loading the stores it is about to rekey (they are still - # old-scheme and would SCHEME_MISMATCH). - result = run_scan( - path, - config_path=_cfg(args, root), - cache_dir=_cache_dir_arg(args, root), - confine_to_root=True, - trust_local_packs=bool(args.get("trust_local_packs", False)), - trusted_packs=_trusted_packs_arg(args), - strict_defaults=bool(args.get("strict_defaults", False)), - skip_suppression=True, - ) + # old-scheme and would SCHEME_MISMATCH). EVERY frontend participates — the + # stores hold RS-WL verdicts too, and a python-only scan misreads each healthy + # Rust verdict as orphaned/stale (A7, weft-dda1a6d8dd). + findings: list[Any] = [] + for lang in ("python", "rust"): + result = run_scan( + path, + config_path=_cfg(args, root), + cache_dir=_cache_dir_arg(args, root), + confine_to_root=True, + trust_local_packs=bool(args.get("trust_local_packs", False)), + trusted_packs=_trusted_packs_arg(args), + strict_defaults=bool(args.get("strict_defaults", False)), + skip_suppression=True, + lang=lang, + ) + findings.extend(result.findings) if not apply_: - report = probe(path, result.findings) + report = probe(path, findings) return { "mode": "probe", "scanned_findings": report.scanned_findings, "matched": report.matched, - "orphaned": list(report.orphaned), + # Counts + a bounded sample (A7, weft-dda1a6d8dd): never the full + # fingerprint dump — the cut is explicit via the count. + "orphaned_count": len(report.orphaned), + "orphaned_sample": list(report.orphaned[:ORPHAN_SAMPLE_LIMIT]), "orphan_cause": ORPHAN_CAUSE, + "stale_count": len(report.stale), + "stale_sample": list(report.stale[:ORPHAN_SAMPLE_LIMIT]), + "stale_cause": STALE_CAUSE, "collisions": [ {"new_fp": c.new_fp, "old_fps": list(c.old_fps), "message": c.message} for c in report.collisions ], "per_store": dict(report.per_store), "prescheme": report.prescheme, + "current_scheme_stores": list(report.current_scheme_stores), + "no_op": report.no_op, "clean": report.clean, } - return {"mode": "apply", **journal_block(run_rekey(path, result.findings, filigree=filigree))} + return {"mode": "apply", **journal_block(run_rekey(path, findings, filigree=filigree))} _REKEY_OUTPUT_SCHEMA: dict[str, Any] = { @@ -3517,20 +3540,40 @@ def journal_block(journal: Journal) -> dict[str, Any]: }, "matched": { "type": "integer", - "description": "probe only: count of distinct stored old-scheme fingerprints that map to a current " - "finding (will carry).", + "description": "probe only: count of distinct stored fingerprints that map to a current finding — each " + "store is judged against ITS OWN scheme (a wlfp1 store via the migration remap, a store already at the " + "live scheme via the current fingerprints).", + }, + "orphaned_count": { + "type": "integer", + "description": "probe only: number of stored OLD-SCHEME fingerprints with no current finding — these " + "verdicts would be dropped by an apply. Stores already at the live scheme never orphan (see stale_*).", }, - "orphaned": { + "orphaned_sample": { "type": "array", "items": {"type": "string"}, - "description": "probe only: sorted stored fingerprints with NO current finding — these verdicts would be " - "dropped by an apply.", + "description": "probe only: bounded sorted sample of the orphaned fingerprints (counts are authoritative; " + "an apply records the full list verbatim in the migration journal).", }, "orphan_cause": { "type": "string", "description": "probe/apply/resume: fixed explanation string for why a stored fingerprint can orphan " "(source moved/deleted, or a custom multi-emit rule not surfacing taint_path_v0).", }, + "stale_count": { + "type": "integer", + "description": "probe only: number of CURRENT-scheme store entries matching no current finding — baseline " + "drift, NOT migration orphans; a rekey would not touch them and they do not dirty the probe.", + }, + "stale_sample": { + "type": "array", + "items": {"type": "string"}, + "description": "probe only: bounded sorted sample of the stale fingerprints.", + }, + "stale_cause": { + "type": "string", + "description": "probe only: fixed explanation string for why a current-scheme entry can be stale.", + }, "collisions": { "type": "array", "description": "probe/apply/resume: pre-rekey fingerprints that collapse to the same new fingerprint " @@ -3568,10 +3611,21 @@ def journal_block(journal: Journal) -> dict[str, Any]: "description": "probe only: true when a live store predates the fingerprint-scheme stamp, so orphans MAY " "be a fingerprint-formula change rather than source churn.", }, + "current_scheme_stores": { + "type": "array", + "items": {"type": "string"}, + "description": "probe only: store file names ALREADY at the live fingerprint scheme — a rekey is a no-op " + "for them; their entries were matched against the current scan's fingerprints.", + }, + "no_op": { + "type": "boolean", + "description": "probe only: true when every populated store already carries the live scheme — no " + "fingerprint migration is pending and an apply would be refused.", + }, "clean": { "type": "boolean", "description": "probe only: true when there are no orphans and no collisions — an apply would carry every " - "stored verdict.", + "stored verdict (or, when no_op, there is simply nothing to migrate).", }, "complete": {"type": "boolean", "description": "apply/resume: true when every migration leg is done."}, "fingerprint_scheme_from": { @@ -3605,11 +3659,16 @@ def journal_block(journal: Journal) -> dict[str, Any]: "description": "Number of stored verdicts re-keyed and carried forward by this leg (count " "only; can be the whole store).", }, - "orphaned": { + "orphaned_count": { + "type": "integer", + "description": "Number of stored verdicts this leg dropped — never silent; the full verbatim " + "list is recorded in the on-disk migration journal.", + }, + "orphaned_sample": { "type": "array", "items": {"type": "string"}, - "description": "Verbatim old fingerprints this leg dropped — a dropped verdict is never " - "silent.", + "description": "Bounded sample of the old fingerprints this leg dropped (counts are " + "authoritative; full list in the migration journal).", }, "debt": { "type": ["string", "null"], @@ -3617,7 +3676,7 @@ def journal_block(journal: Journal) -> dict[str, Any]: "(re-run with apply to retry); null otherwise.", }, }, - "required": ["name", "done", "carried_count", "orphaned", "debt"], + "required": ["name", "done", "carried_count", "orphaned_count", "orphaned_sample", "debt"], "additionalProperties": False, }, }, diff --git a/tests/unit/core/test_rekey_probe.py b/tests/unit/core/test_rekey_probe.py index b316af02..90e1502a 100644 --- a/tests/unit/core/test_rekey_probe.py +++ b/tests/unit/core/test_rekey_probe.py @@ -93,6 +93,106 @@ def test_probe_does_not_flag_an_empty_project(tmp_path: Path) -> None: assert probe(tmp_path, [_finding()]).prescheme is False +def _write_store(state: Path, name: str, *, scheme: str | None, fingerprints: list[str]) -> None: + doc: dict = { + "version": 1, + "entries": [{"fingerprint": fp, "rule_id": "PY-WL-108", "path": "m.py", "message": "x"} for fp in fingerprints], + } + if scheme is not None: + doc["fingerprint_scheme"] = scheme + (state / name).write_text(yaml.safe_dump(doc), encoding="utf-8") + + +def test_probe_reports_a_healthy_current_scheme_baseline_as_clean_noop(tmp_path: Path) -> None: + # A7 (weft-dda1a6d8dd): a store ALREADY stamped with the live scheme whose entries + # all match the current scan is a healthy baseline with NO migration pending. The + # probe must report matched=N, orphaned=0, clean — never 100% orphaned (which it + # did when it compared every store against the wlfp1-reconstructed keys only). + root = tmp_path + f = _finding() + state = paths.weft_state_dir(root) + state.mkdir(parents=True) + _write_store(state, "baseline.yaml", scheme="wlfp2", fingerprints=[f.fingerprint]) + + report = probe(root, [f]) + assert report.matched == 1 + assert report.orphaned == () + assert report.stale == () + assert report.clean + assert report.no_op + assert report.current_scheme_stores == ("baseline.yaml",) + # Read-only, as ever. + assert not paths.migration_journal_path(root).exists() + assert not snapshot_dir(root).exists() + + +def test_probe_current_scheme_entry_without_a_finding_is_stale_not_orphaned(tmp_path: Path) -> None: + # An entry already at the live scheme that matches no current finding is baseline + # DRIFT (the source changed since baselining) — a rekey would not touch it, so it + # must not be reported as a migration orphan with the source-moved cause. + root = tmp_path + f = _finding() + state = paths.weft_state_dir(root) + state.mkdir(parents=True) + _write_store(state, "baseline.yaml", scheme="wlfp2", fingerprints=[f.fingerprint, "e" * 64]) + + report = probe(root, [f]) + assert report.matched == 1 + assert report.orphaned == () + assert report.stale == ("e" * 64,) + assert report.no_op + assert report.clean # no migration pending — stale entries are hygiene, not rekey risk + + +def test_probe_mixed_schemes_judges_each_store_against_its_own_scheme(tmp_path: Path) -> None: + # One store still wlfp1 (migration pending for it), one already wlfp2: the wlfp1 + # store is judged against the reconstructed old keys, the wlfp2 store against the + # live fingerprints — neither contaminates the other's verdict. + root = tmp_path + f = _finding() + old_fp = compute_finding_fingerprint_v0( + rule_id="PY-WL-108", path="m.py", line_start=10, qualname="m.f", taint_path="os.system@4:20" + ) + state = paths.weft_state_dir(root) + state.mkdir(parents=True) + _write_store(state, "baseline.yaml", scheme="wlfp1", fingerprints=[old_fp, ORPHAN]) + doc = { + "fingerprint_scheme": "wlfp2", + "version": 1, + "waivers": [{"fingerprint": f.fingerprint, "rule_id": "PY-WL-108", "path": "m.py", "message": "x"}], + } + (state / "waivers.yaml").write_text(yaml.safe_dump(doc), encoding="utf-8") + + report = probe(root, [f]) + assert report.matched == 2 # old_fp via the remap keys, f.fingerprint via the live set + assert report.orphaned == (ORPHAN,) + assert report.per_store == {"baseline.yaml": 1} + assert report.current_scheme_stores == ("waivers.yaml",) + assert not report.no_op # a wlfp1 store still pends migration + assert not report.clean + + +def test_run_rekey_refuses_when_no_migration_is_pending(tmp_path: Path) -> None: + # Companion guard to the probe fix: applying a rekey over stores ALREADY at the + # live scheme would re-key wlfp2 entries through the wlfp1 remap and orphan every + # verdict (the destructive twin of the A7 probe misread). Refuse before writing. + from wardline.core.errors import WardlineError + from wardline.core.rekey import run_rekey + + root = tmp_path + f = _finding() + state = paths.weft_state_dir(root) + state.mkdir(parents=True) + _write_store(state, "baseline.yaml", scheme="wlfp2", fingerprints=[f.fingerprint]) + before = (state / "baseline.yaml").read_bytes() + + with pytest.raises(WardlineError, match="no fingerprint migration is pending"): + run_rekey(root, [f]) + assert (state / "baseline.yaml").read_bytes() == before + assert not paths.migration_journal_path(root).exists() + assert not snapshot_dir(root).exists() + + def test_probe_flags_a_scheme_less_prescheme_store(tmp_path: Path) -> None: # A POPULATED store with NO fingerprint_scheme header predates P1's stamp. Its # fingerprints may also predate 705acfe (resolved-taint) -> v0 can't reconstruct them diff --git a/tests/unit/mcp/test_server_rekey.py b/tests/unit/mcp/test_server_rekey.py index 35cf8e47..c5fd9b95 100644 --- a/tests/unit/mcp/test_server_rekey.py +++ b/tests/unit/mcp/test_server_rekey.py @@ -63,7 +63,9 @@ def test_rekey_defaults_to_read_only_probe(tmp_path: Path) -> None: result = _rekey({}, project) assert result["mode"] == "probe" assert result["matched"] == 1 - assert result["orphaned"] == [] + assert result["orphaned_count"] == 0 + assert result["orphaned_sample"] == [] + assert result["no_op"] is False # a wlfp1 store pends migration assert result["clean"] is True # Writes NOTHING: no snapshot, no journal, store untouched (still wlfp1). assert not paths.migration_journal_path(project).exists() @@ -76,11 +78,46 @@ def test_rekey_probe_reports_orphans_with_cause(tmp_path: Path) -> None: _seed_wlfp1_baseline(project, extra_fps=("deadbeef" * 8,)) result = _rekey({}, project) assert result["clean"] is False - assert result["orphaned"] == ["deadbeef" * 8] + assert result["orphaned_count"] == 1 + assert result["orphaned_sample"] == ["deadbeef" * 8] assert result["per_store"] == {"baseline.yaml": 1} assert "moved" in result["orphan_cause"] +def test_rekey_probe_reports_a_healthy_current_scheme_baseline_as_clean_noop(tmp_path: Path) -> None: + # A7 (weft-dda1a6d8dd): the live-lacuna shape — a wlfp2 baseline whose entries all + # match the current scan must read matched=N / orphaned=0 / clean, never 100% orphaned. + project = _project(tmp_path) + leak = next(f for f in run_scan(project).findings if f.rule_id == "PY-WL-101") + bp = paths.baseline_path(project) + bp.parent.mkdir(parents=True, exist_ok=True) + bp.write_text( + yaml.safe_dump( + { + "fingerprint_scheme": "wlfp2", + "version": 1, + "entries": [ + { + "fingerprint": leak.fingerprint, + "rule_id": leak.rule_id, + "path": leak.location.path, + "message": leak.message, + } + ], + } + ), + encoding="utf-8", + ) + result = _rekey({}, project) + assert result["mode"] == "probe" + assert result["matched"] == 1 + assert result["orphaned_count"] == 0 + assert result["stale_count"] == 0 + assert result["no_op"] is True + assert result["current_scheme_stores"] == ["baseline.yaml"] + assert result["clean"] is True + + def test_rekey_apply_migrates_and_reports_journal(tmp_path: Path) -> None: project = _project(tmp_path) leak, old_fp = _seed_wlfp1_baseline(project) From 235e6b49e863cef4e812934f229c79882beca59c Mon Sep 17 00:00:00 2001 From: John Morrissey Date: Sat, 13 Jun 2026 06:25:44 +1000 Subject: [PATCH 158/186] fix(explain): name the taint source on sink findings; honest-degrade markers per C-10(c) (dogfood B7, weft-0d24cf9152) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit explain_taint on the flagship sentinel (PY-WL-125, specimen.preview_sinks.log_export_request) returned immediate_tainted_callee: null, source_boundary_qualname: null, null tiers, and generic "review the finding" remediation — the actual source (read_report_field, 3 lines above the sink) was never named. Root cause: explanation_from_context only consulted function_return_callee (the call contributing a function's RETURN taint). A sink finding's tainted value reaches the sink ARGUMENT, never the return, so the return-callee map is structurally empty for the whole dangerous-sink family; the tier fields read actual_return/declared_return, which sink findings don't carry either. (a) Name the source from wardline's own analysis (core/explain.py): - _sink_taint_source derives the sink-argument source from the retained context: a raw-returning in-project call inline in the sink's arguments, or the last raw-returning call assigned to a name the sink's arguments use (call_site_callees + function_return_taints/RAW_ZONE). - Sink findings project tier_in from arg_taint and tier_out from tier. - The Loomweave fast path no longer serves a cached fact that names no source when a local re-run is possible — correctness over the cached read. (b) Honest degrade, never complete-but-empty nulls (C-10(c)): - New source_resolution block on every explain_taint result: status resolved/unresolved; when unresolved it names WHY and, when no Loomweave store is configured, missing_capability=loomweave_taint_store plus the enablement path. - chain=true without a store (or without a sink qualname) now returns an explicit chain {status: unavailable, missing_capability, enablement} instead of silently omitting the block. Remediation is rule-specific where the rule is known: new sink_hygiene kind with per-rule fix guidance for the dangerous-sink family (PY-WL-106..126), naming the resolved source and sink (PY-WL-125: lazy %-parameterization). MCP output schemas (explain_taint + scan-inline explanation) updated. Live-verified against lacuna's PY-WL-125: before all-null; after immediate_tainted_callee=read_report_field, source_boundary_qualname= specimen.preview_sinks.read_report_field, EXTERNAL_RAW->ASSURED, and the rule-specific log-injection remediation. Co-Authored-By: Claude Fable 5 --- src/wardline/core/explain.py | 245 +++++++++++++++++++++++++++++++- src/wardline/mcp/server.py | 82 +++++++++-- tests/unit/core/test_explain.py | 133 +++++++++++++++++ 3 files changed, 445 insertions(+), 15 deletions(-) diff --git a/src/wardline/core/explain.py b/src/wardline/core/explain.py index 407ab05d..dd139c4b 100644 --- a/src/wardline/core/explain.py +++ b/src/wardline/core/explain.py @@ -11,17 +11,30 @@ from __future__ import annotations +import ast from dataclasses import dataclass from pathlib import Path from typing import TYPE_CHECKING, Any from wardline.core.errors import LoomweaveError from wardline.core.run import run_scan +from wardline.core.taints import RAW_ZONE if TYPE_CHECKING: from wardline.core.finding import Finding from wardline.scanner.context import AnalysisContext +# The C-10(c) honest-degrade marker: when the taint source is not derivable from +# wardline's own single-scan analysis, the response names the capability that could +# resolve it further and how to enable it — never nulls that read as a +# complete-but-empty answer (dogfood-5 B7, weft-0d24cf9152). +LOOMWEAVE_CAPABILITY = "loomweave_taint_store" +LOOMWEAVE_ENABLEMENT = ( + "configure a Loomweave store (--loomweave-url / WARDLINE_LOOMWEAVE_URL / " + "weft.toml [loomweave].url) and call explain_taint with chain=true to walk " + "the stored cross-entity taint chain" +) + @dataclass(frozen=True, slots=True) class TaintExplanation: @@ -36,6 +49,81 @@ class TaintExplanation: source_boundary_qualname: str | None resolved_call_count: int unresolved_call_count: int + # Dotted name of the dangerous sink CALLABLE for call-site-anchored sink findings + # (properties["sink"], e.g. "logging.getLogger.info"); None for return-tier rules + # and on the store-served path (the blob carries no sink property). + sink: str | None = None + + +def _untrusted_project_callee(call: ast.Call, context: AnalysisContext) -> str | None: + """The resolved in-project callee of *call* when its RETURN taint is in the raw + zone (an untrusted source), else None.""" + callee = context.call_site_callees.get(id(call)) + if callee is None: + return None + taint = context.function_return_taints.get(callee, context.project_return_taints.get(callee)) + return callee if taint in RAW_ZONE else None + + +def _sink_arg_exprs(call: ast.Call) -> list[ast.expr]: + return [*call.args, *[kw.value for kw in call.keywords]] + + +def _sink_taint_source(finding: Finding, context: AnalysisContext) -> str | None: + """For a call-site-anchored sink finding, the in-project call that introduced the + untrusted value into the sink's arguments — derived from wardline's OWN analysis + (no store needed): either a raw-returning call INLINE in the sink's argument list + (``logger.info(read_raw(p))``) or the LAST raw-returning call assigned to a name + the sink's arguments use (``msg = read_raw(p); logger.info(msg)``). Returns the + callee's qualname, or None when the source is genuinely not derivable from the + single scan (imported/dynamic sources — the Loomweave chain walk's territory).""" + qualname, line = finding.qualname, finding.location.line_start + if qualname is None or line is None: + return None + entity = context.entities.get(qualname) + if entity is None: + return None + sink_calls = [n for n in ast.walk(entity.node) if isinstance(n, ast.Call) and getattr(n, "lineno", None) == line] + if not sink_calls: + return None + # (a) a raw-returning call nested directly in the sink's own arguments. + for sink in sink_calls: + for arg in _sink_arg_exprs(sink): + for sub in ast.walk(arg): + if isinstance(sub, ast.Call): + callee = _untrusted_project_callee(sub, context) + if callee is not None: + return callee + # (b) a Name argument assigned from a raw-returning call earlier in the function; + # the LAST such assignment before the sink line wins (mirrors L2's forward walk). + arg_names = { + sub.id + for sink in sink_calls + for arg in _sink_arg_exprs(sink) + for sub in ast.walk(arg) + if isinstance(sub, ast.Name) + } + if not arg_names: + return None + best: tuple[int, str] | None = None + for node in ast.walk(entity.node): + targets: list[ast.expr] + value: ast.expr | None + if isinstance(node, (ast.Assign, ast.AnnAssign)): + targets = list(node.targets) if isinstance(node, ast.Assign) else [node.target] + value = node.value + elif isinstance(node, ast.NamedExpr): + targets, value = [node.target], node.value + else: + continue + if value is None or not isinstance(value, ast.Call) or node.lineno > line: + continue + if not any(isinstance(t, ast.Name) and t.id in arg_names for t in targets): + continue + callee = _untrusted_project_callee(value, context) + if callee is not None and (best is None or node.lineno > best[0]): + best = (node.lineno, callee) + return best[1] if best else None def explanation_from_context(finding: Finding, context: AnalysisContext) -> TaintExplanation: @@ -57,6 +145,23 @@ def explanation_from_context(finding: Finding, context: AnalysisContext) -> Tain candidate = f"{module}.{immediate_tainted_callee}" if candidate in context.entities and context.function_return_callee.get(candidate) is None: source_boundary_qualname = candidate + sink = finding.properties.get("sink") + if immediate_tainted_callee is None and isinstance(sink, str): + # A sink finding's tainted value reaches the sink ARGUMENT, never the return, + # so the return-callee map cannot explain it (B7, weft-0d24cf9152): derive the + # source from the sink's argument flow instead. + source = _sink_taint_source(finding, context) + if source is not None: + immediate_tainted_callee = source.rsplit(".", 1)[-1] + if source in context.entities and context.function_return_callee.get(source) is None: + source_boundary_qualname = source # a leaf source — the 1-hop boundary + tier_in = finding.properties.get("actual_return") + tier_out = finding.properties.get("declared_return") + if isinstance(sink, str): + # Sink findings carry their tier facts as arg_taint (what arrives at the sink) + # and tier (the enclosing declared tier) — project them, not nulls. + tier_in = tier_in or finding.properties.get("arg_taint") + tier_out = tier_out or finding.properties.get("tier") prov = context.taint_provenance.get(qualname) if qualname is not None else None return TaintExplanation( fingerprint=finding.fingerprint, @@ -64,12 +169,13 @@ def explanation_from_context(finding: Finding, context: AnalysisContext) -> Tain sink_qualname=qualname, path=finding.location.path, line=finding.location.line_start, - tier_in=finding.properties.get("actual_return"), - tier_out=finding.properties.get("declared_return"), + tier_in=tier_in, + tier_out=tier_out, immediate_tainted_callee=immediate_tainted_callee, source_boundary_qualname=source_boundary_qualname, resolved_call_count=prov.resolved_call_count if prov is not None else 0, unresolved_call_count=prov.unresolved_call_count if prov is not None else 0, + sink=sink if isinstance(sink, str) else None, ) @@ -379,9 +485,20 @@ def explain_finding( path=path, line=line, ) - if served is not None: + # Serve the cached fact only when it actually NAMES a source, OR when no + # re-run is possible (a pure-store read keyed on sink_qualname alone has + # no local match key). A blob whose contributing callee is null answers + # nothing for a sink-argument finding (the store records return-taint + # facts), while the SP8 re-run can derive the source from the sink's + # argument flow (B7, weft-0d24cf9152) — correctness over the cached read. + can_rerun = fingerprint is not None or (path is not None and line is not None) + if served is not None and ( + served.immediate_tainted_callee is not None + or served.source_boundary_qualname is not None + or not can_rerun + ): return served - # miss/stale/outage → fall through to the re-run + # miss/stale/outage/sourceless → fall through to the re-run if fingerprint is None and path is None and line is None: return None return _explain_local( @@ -394,7 +511,35 @@ def explain_finding( ) -def explanation_to_dict(exp: TaintExplanation) -> dict[str, Any]: +def source_resolution_to_dict(exp: TaintExplanation, *, loomweave_configured: bool = False) -> dict[str, Any]: + """The C-10(c) honesty block: is the taint source named, and if not, WHY and what + capability would resolve it further. Fixed key set so an unresolved source is an + explicit degrade marker, never nulls that read as a complete-but-empty answer + (B7, weft-0d24cf9152).""" + if exp.immediate_tainted_callee is not None or exp.source_boundary_qualname is not None: + return {"status": "resolved", "reason": None, "missing_capability": None, "enablement": None} + where = f" in {exp.sink_qualname}" if exp.sink_qualname else "" + reason = ( + "wardline's single-scan analysis could not attribute the tainted value" + f"{where} to a resolvable in-project call (imported or dynamic sources are " + "beyond its single-scan reach)" + ) + if loomweave_configured: + return { + "status": "unresolved", + "reason": reason + "; the configured Loomweave store's facts also carried no contributing callee", + "missing_capability": None, + "enablement": None, + } + return { + "status": "unresolved", + "reason": reason, + "missing_capability": LOOMWEAVE_CAPABILITY, + "enablement": LOOMWEAVE_ENABLEMENT, + } + + +def explanation_to_dict(exp: TaintExplanation, *, loomweave_configured: bool = False) -> dict[str, Any]: """The serialized explanation slice + remediation hint. Single source for the MCP ``explain_taint`` result and the CLI ``wardline explain-taint`` output (identical by construction — the N-2 dead-end was the CLI lacking this).""" @@ -403,13 +548,80 @@ def explanation_to_dict(exp: TaintExplanation) -> dict[str, Any]: "tier_out": exp.tier_out, "immediate_tainted_callee": exp.immediate_tainted_callee, "source_boundary_qualname": exp.source_boundary_qualname, + "source_resolution": source_resolution_to_dict(exp, loomweave_configured=loomweave_configured), "resolved_call_count": exp.resolved_call_count, "unresolved_call_count": exp.unresolved_call_count, "remediation": remediation_to_dict(exp), } +# Rule-specific fix guidance for the call-site-anchored sink family — generic "review +# the finding" text on a SPECIFIC finding is part of the B7 defect. One sentence per +# rule, composed with the named source/sink below. +_SINK_FIX_HINTS: dict[str, str] = { + "PY-WL-106": ( + "Deserialize only trusted bytes: verify provenance or switch to a " + "schema-validated format (e.g. json.loads + validation) before the sink." + ), + "PY-WL-107": ( + "Never pass untrusted text to dynamic execution; replace it with an explicit " + "dispatch table or parser, or strictly allowlist the input first." + ), + "PY-WL-108": ( + "Run a fixed program path and pass untrusted values only as list-form " + "arguments (never through a shell); validate or allowlist them first." + ), + "PY-WL-112": ( + "Avoid shell=True with untrusted input: use list-form arguments without a " + "shell, or quote via shlex.quote after validating." + ), + "PY-WL-115": ( + "Do not import modules named by untrusted input; resolve the name through an " + "explicit allowlist of importable modules." + ), + "PY-WL-116": ( + "Resolve untrusted paths against a fixed base directory and reject escapes " + "(e.g. Path.resolve() + is_relative_to) before opening." + ), + "PY-WL-118": ("Use parameterized queries (placeholders) — never interpolate untrusted values into SQL text."), + "PY-WL-117": ( + "Validate the URL against an allowlist of schemes and hosts before fetching; never fetch a raw caller URL." + ), + "PY-WL-121": ("Parse untrusted XML with entity resolution disabled (e.g. defusedxml) instead of the raw parser."), + "PY-WL-122": ( + "Never compile untrusted text as a template; render untrusted values only as template DATA (context variables)." + ), + "PY-WL-123": ( + "Do not let untrusted input choose attribute names; map allowed names " + "explicitly instead of reflecting raw input." + ), + "PY-WL-124": ( + "Load native libraries only from fixed, vetted paths — never from a path derived from untrusted input." + ), + "PY-WL-125": ( + "Do not use untrusted data as the log message format string: use logging's " + "lazy parameterization (logger.info('value=%s', value)) or strip " + "newline/control characters first." + ), + "PY-WL-126": ("Validate and normalize recipient addresses and message bodies (reject CR/LF) before the mail API."), +} + + def remediation_to_dict(exp: TaintExplanation) -> dict[str, Any]: + source = exp.source_boundary_qualname or exp.immediate_tainted_callee + hint = _SINK_FIX_HINTS.get(exp.rule_id) + if exp.rule_id != "PY-WL-101" and hint is not None: + sink_call = f"the {exp.sink} sink" if exp.sink else "the sink" + where = f" in {exp.sink_qualname}" if exp.sink_qualname else "" + origin = f"from {source} " if source else "" + return { + "kind": "sink_hygiene", + "rule_id": exp.rule_id, + "summary": f"Untrusted data {origin}reaches {sink_call}{where}. {hint}", + "sink_qualname": exp.sink_qualname, + "source_qualname": source, + "caveat": "This hint is advisory and does not replace the factual taint explanation.", + } if exp.rule_id != "PY-WL-101": return { "kind": "review_required", @@ -418,7 +630,7 @@ def remediation_to_dict(exp: TaintExplanation) -> dict[str, Any]: "Review the finding and apply the rule-specific fix; no automated remediation hint is available." ), "sink_qualname": exp.sink_qualname, - "source_qualname": exp.source_boundary_qualname, + "source_qualname": source, "caveat": "This hint is advisory and does not replace the factual taint explanation.", } @@ -492,11 +704,12 @@ def explain_taint_result( "rule_id": exp.rule_id, "sink_qualname": exp.sink_qualname, "location": {"path": exp.path, "line": exp.line}, - **explanation_to_dict(exp), + **explanation_to_dict(exp, loomweave_configured=loomweave is not None), } if chain and loomweave is not None and exp.sink_qualname: ch = explain_chain(root, sink_qualname=exp.sink_qualname, loomweave=loomweave, max_hops=max_hops) result["chain"] = { + "status": "walked", "hops": [ { "qualname": h.qualname, @@ -507,5 +720,23 @@ def explain_taint_result( for h in ch.hops ], "truncated_at": ch.truncated_at, + "missing_capability": None, + "enablement": None, + } + elif chain: + # chain=true but the walk cannot run: say so EXPLICITLY (C-10(c)) — an absent + # block was a silent degrade an agent read as "no chain exists". + missing = LOOMWEAVE_CAPABILITY if loomweave is None else "sink_qualname" + enablement = ( + LOOMWEAVE_ENABLEMENT + if loomweave is None + else "this finding carries no qualname for the engine to anchor the walk on; re-scan and use a finding with one" # noqa: E501 + ) + result["chain"] = { + "status": "unavailable", + "hops": [], + "truncated_at": None, + "missing_capability": missing, + "enablement": enablement, } return result diff --git a/src/wardline/mcp/server.py b/src/wardline/mcp/server.py index 81636518..e0a7711a 100644 --- a/src/wardline/mcp/server.py +++ b/src/wardline/mcp/server.py @@ -768,7 +768,9 @@ def _scan( if f is None or f.qualname is None: continue if attached < explain_cap: - entry["explanation"] = explanation_to_dict(explanation_from_context(f, result.context)) + entry["explanation"] = explanation_to_dict( + explanation_from_context(f, result.context), loomweave_configured=loomweave is not None + ) attached += 1 else: explanations_truncated = True @@ -1481,12 +1483,25 @@ def _scan( "tier_out": {"type": ["string", "null"], "description": "Tier the sink declares it returns."}, "immediate_tainted_callee": {"type": ["string", "null"]}, "source_boundary_qualname": {"type": ["string", "null"]}, + "source_resolution": { + "type": "object", + "description": "C-10(c) honesty block: explicit resolved/unresolved verdict on the taint source, " + "with reason + missing capability + enablement when unresolved.", + "properties": { + "status": {"type": "string", "enum": ["resolved", "unresolved"]}, + "reason": {"type": ["string", "null"]}, + "missing_capability": {"type": ["string", "null"]}, + "enablement": {"type": ["string", "null"]}, + }, + "required": ["status", "reason", "missing_capability", "enablement"], + "additionalProperties": False, + }, "resolved_call_count": {"type": "integer"}, "unresolved_call_count": {"type": "integer"}, "remediation": { "type": "object", "properties": { - "kind": {"type": "string", "enum": ["boundary_placement", "review_required"]}, + "kind": {"type": "string", "enum": ["boundary_placement", "sink_hygiene", "review_required"]}, "rule_id": {"type": "string"}, "summary": {"type": "string"}, "sink_qualname": {"type": ["string", "null"]}, @@ -1502,6 +1517,7 @@ def _scan( "tier_out", "immediate_tainted_callee", "source_boundary_qualname", + "source_resolution", "resolved_call_count", "unresolved_call_count", "remediation", @@ -1830,6 +1846,36 @@ def _explain_taint(args: dict[str, Any], root: Path, loomweave: Any = None) -> d "function the taint came from (null when not resolvable in one hop). On the store-served path this is the " "blob's contributing_callee_qualname.", }, + "source_resolution": { + "type": "object", + "description": "C-10(c) honesty block: whether the taint source is named above, and when it is NOT, why " + "and what capability would resolve it further — an explicit degrade marker, never nulls that read as a " + "complete-but-empty answer.", + "properties": { + "status": { + "type": "string", + "enum": ["resolved", "unresolved"], + "description": "resolved when immediate_tainted_callee or source_boundary_qualname is named; " + "unresolved otherwise.", + }, + "reason": { + "type": ["string", "null"], + "description": "unresolved only: why wardline's own single-scan analysis could not name the " + "source; null when resolved.", + }, + "missing_capability": { + "type": ["string", "null"], + "description": "unresolved only: capability that could resolve further — 'loomweave_taint_store' " + "when no store is configured; null when resolved or when nothing more would help.", + }, + "enablement": { + "type": ["string", "null"], + "description": "unresolved only: how to enable the missing capability; null otherwise.", + }, + }, + "required": ["status", "reason", "missing_capability", "enablement"], + "additionalProperties": False, + }, "resolved_call_count": { "type": "integer", "description": "Number of calls inside the sink the engine resolved during taint computation.", @@ -1846,9 +1892,10 @@ def _explain_taint(args: dict[str, Any], root: Path, loomweave: Any = None) -> d "properties": { "kind": { "type": "string", - "enum": ["boundary_placement", "review_required"], + "enum": ["boundary_placement", "sink_hygiene", "review_required"], "description": "boundary_placement for PY-WL-101 (place/repair a @trust_boundary at the " - "validating function); review_required for every other rule (no automated hint).", + "validating function); sink_hygiene for the dangerous-sink family (rule-specific fix guidance " + "naming the source and sink); review_required for rules with no automated hint.", }, "rule_id": { "type": "string", @@ -1875,9 +1922,26 @@ def _explain_taint(args: dict[str, Any], root: Path, loomweave: Any = None) -> d "chain": { "type": "object", "description": "Full N-hop taint chain from the sink to the originating boundary, walked from the " - "Loomweave store. PRESENT only when the call passed chain=true AND a Loomweave store is configured AND " - "the finding has a sink qualname; otherwise absent (silent degradation to the single-hop explanation).", + "Loomweave store. Present whenever the call passed chain=true: status 'walked' carries the hops; status " + "'unavailable' is the explicit C-10(c) degrade marker (no Loomweave store configured, or no sink " + "qualname to anchor on) naming the missing capability and its enablement path — the walk never degrades " + "silently.", "properties": { + "status": { + "type": "string", + "enum": ["walked", "unavailable"], + "description": "walked: the store walk ran (hops below). unavailable: the walk could not run; " + "see missing_capability/enablement.", + }, + "missing_capability": { + "type": ["string", "null"], + "description": "unavailable only: what the walk lacked ('loomweave_taint_store' or " + "'sink_qualname'); null when walked.", + }, + "enablement": { + "type": ["string", "null"], + "description": "unavailable only: how to enable the missing capability; null when walked.", + }, "hops": { "type": "array", "description": "Ordered hops from the sink toward the boundary leaf. The walk stops cleanly at a " @@ -1917,7 +1981,7 @@ def _explain_taint(args: dict[str, Any], root: Path, loomweave: Any = None) -> d "the boundary cleanly.", }, }, - "required": ["hops", "truncated_at"], + "required": ["status", "hops", "truncated_at", "missing_capability", "enablement"], "additionalProperties": False, }, }, @@ -1930,6 +1994,7 @@ def _explain_taint(args: dict[str, Any], root: Path, loomweave: Any = None) -> d "tier_out", "immediate_tainted_callee", "source_boundary_qualname", + "source_resolution", "resolved_call_count", "unresolved_call_count", "remediation", @@ -1948,7 +2013,8 @@ def _explain_taint(args: dict[str, Any], root: Path, loomweave: Any = None) -> d "is configured this serves the explanation from the store instead of " "re-scanning. Pass `chain: true` (needs a configured Loomweave store) to " "also walk the full taint chain from the sink to the originating boundary; " - "without a store it degrades to the single-hop explanation (no `chain` block).", + "without a store the `chain` block is an explicit `status: unavailable` marker " + "naming the missing capability and its enablement path.", "input_schema": { "type": "object", "properties": { diff --git a/tests/unit/core/test_explain.py b/tests/unit/core/test_explain.py index 947da0c3..273fab8f 100644 --- a/tests/unit/core/test_explain.py +++ b/tests/unit/core/test_explain.py @@ -96,3 +96,136 @@ def test_explain_finding_still_projects_provenance(tmp_path: Path) -> None: assert exp.sink_qualname == "svc.leaky" assert exp.immediate_tainted_callee == "read_raw" assert exp.source_boundary_qualname == "svc.read_raw" + + +# --- B7 (weft-0d24cf9152): sink-rule findings must name their taint source --------- + +# A PY-WL-125-style sink finding: the tainted value reaches the sink ARGUMENT (never +# the return), so the return-callee path cannot explain it — the sink-argument +# derivation must. +_SINK_LEAKY = ( + "import logging\n" + "from wardline.decorators import external_boundary, trusted\n" + "logger = logging.getLogger(__name__)\n" + "@external_boundary\ndef read_raw(p):\n return p\n" + "@trusted\ndef log_it(p):\n msg = read_raw(p)\n logger.info(msg)\n" +) + +# Same sink, the source call INLINE in the sink's argument list. +_SINK_LEAKY_INLINE = ( + "import logging\n" + "from wardline.decorators import external_boundary, trusted\n" + "logger = logging.getLogger(__name__)\n" + "@external_boundary\ndef read_raw(p):\n return p\n" + "@trusted\ndef log_it(p):\n logger.info(read_raw(p))\n" +) + +# The tainted argument comes from an IMPORTED callee wardline cannot resolve — the +# source is genuinely underivable from the single-scan analysis. +_SINK_UNDERIVABLE = ( + "import logging\n" + "from somewhere_else import fetch_text\n" + "from wardline.decorators import trusted\n" + "logger = logging.getLogger(__name__)\n" + "@trusted\ndef log_it(p):\n msg = fetch_text(p)\n logger.info(msg)\n" +) + + +def _sink_project(tmp_path: Path, source: str) -> Path: + proj = tmp_path / "proj" + proj.mkdir() + (proj / "svc.py").write_text(source, encoding="utf-8") + return proj + + +def _finding_125(root: Path): + result = run_scan(root) + for f in result.findings: + if f.rule_id == "PY-WL-125": + return f + raise AssertionError("sink project has no PY-WL-125 finding") + + +def test_explain_sink_finding_names_the_taint_source(tmp_path: Path) -> None: + root = _sink_project(tmp_path, _SINK_LEAKY) + f = _finding_125(root) + exp = explain_finding(root, fingerprint=f.fingerprint) + assert exp is not None + # The actual source — read_raw, assigned to msg three lines above the sink — + # is named, with its one-hop boundary resolution. + assert exp.immediate_tainted_callee == "read_raw" + assert exp.source_boundary_qualname == "svc.read_raw" + # Sink findings carry tier facts in tier/arg_taint properties, not the + # return-mismatch keys — the explanation must project them, not null out. + assert exp.tier_in == "EXTERNAL_RAW" + assert exp.tier_out == "INTEGRAL" + + +def test_explain_sink_finding_names_an_inline_call_source(tmp_path: Path) -> None: + root = _sink_project(tmp_path, _SINK_LEAKY_INLINE) + f = _finding_125(root) + exp = explain_finding(root, fingerprint=f.fingerprint) + assert exp is not None + assert exp.immediate_tainted_callee == "read_raw" + assert exp.source_boundary_qualname == "svc.read_raw" + + +def test_explain_taint_result_marks_source_resolution_resolved(tmp_path: Path) -> None: + from wardline.core.explain import explain_taint_result + + root = _sink_project(tmp_path, _SINK_LEAKY) + f = _finding_125(root) + result = explain_taint_result(root, fingerprint=f.fingerprint) + assert result is not None + res = result["source_resolution"] + assert res["status"] == "resolved" + assert res["missing_capability"] is None + + +def test_explain_taint_result_degrades_honestly_when_source_underivable(tmp_path: Path) -> None: + # C-10(c): an unresolved source must SAY what is missing and how to enable it — + # never nulls that read as a complete-but-empty answer. + from wardline.core.explain import explain_taint_result + + root = _sink_project(tmp_path, _SINK_UNDERIVABLE) + f = _finding_125(root) + result = explain_taint_result(root, fingerprint=f.fingerprint, loomweave=None) + assert result is not None + assert result["immediate_tainted_callee"] is None + res = result["source_resolution"] + assert res["status"] == "unresolved" + assert res["reason"] # names WHY, not just null + assert res["missing_capability"] == "loomweave_taint_store" + assert "loomweave" in res["enablement"].lower() + + +def test_explain_taint_result_chain_unavailable_is_explicit(tmp_path: Path) -> None: + # chain=true without a Loomweave store used to degrade SILENTLY (no chain block); + # C-10(c) requires an explicit marker naming the capability and enablement path. + from wardline.core.explain import explain_taint_result + + root = _sink_project(tmp_path, _SINK_LEAKY) + f = _finding_125(root) + result = explain_taint_result(root, fingerprint=f.fingerprint, chain=True, loomweave=None) + assert result is not None + chain = result["chain"] + assert chain["status"] == "unavailable" + assert chain["hops"] == [] + assert chain["missing_capability"] == "loomweave_taint_store" + assert "loomweave" in chain["enablement"].lower() + + +def test_remediation_is_rule_specific_for_known_sink_rules(tmp_path: Path) -> None: + # B7: generic "review the finding" text on a SPECIFIC finding is part of the + # defect — PY-WL-125 has a canonical fix (lazy %-parameterization). + from wardline.core.explain import explain_taint_result + + root = _sink_project(tmp_path, _SINK_LEAKY) + f = _finding_125(root) + result = explain_taint_result(root, fingerprint=f.fingerprint) + assert result is not None + rem = result["remediation"] + assert rem["kind"] == "sink_hygiene" + assert rem["source_qualname"] == "svc.read_raw" + assert "parameteriz" in rem["summary"] or "%s" in rem["summary"] + assert "Review the finding and apply the rule-specific fix" not in rem["summary"] From c330bb95607dfe9817de91976fc4276d1300c5e2 Mon Sep 17 00:00:00 2001 From: John Morrissey Date: Sat, 13 Jun 2026 06:27:38 +1000 Subject: [PATCH 159/186] =?UTF-8?q?test(conformance):=20C-13=20hostile-inp?= =?UTF-8?q?ut=20fixture=20=E2=80=94=20fail-degraded,=20never=20fail-dead?= =?UTF-8?q?=20(weft-b181c75e39,=20partial)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Federation convention C-13 (ratified b0eee6e) requires every source-walking member to skip + flag + continue on hostile input, with an in-band marker in the result envelope, and adoption includes a hostile-input fixture in the member's own suite. Wardline's cell was "unknown": behaviour untested. Adds the nesting_bomb-class corpus (generated in-test, mirroring lacuna's specimen/nesting_bomb.py): (a) 600-deep nested if blocks, (b) the 20,000-term BinOp chain — module-level and inside a function body — that killed legis's policy-boundary-check (dogfood-4 A2). The test asserts the scan COMPLETES, every hostile file carries an in-band gate-eligible ERROR DEFECT marker (WLN-ENGINE-PARSE-ERROR / FILE-SKIPPED / FUNCTION-SKIPPED / FILE-FAILED), the healthy neighbour's PY-WL-101 finding is still reported, and the degraded scope is visible in the envelope (summary.unanalyzed). Verdict: wardline ALREADY conforms — the pipeline's per-file isolation (parse-stage RecursionError -> WLN-ENGINE-FILE-SKIPPED, per-entity L2 RecursionError -> WLN-ENGINE-FUNCTION-SKIPPED, blanket per-file FILE-FAILED) holds against both bomb classes; no engine change was needed. This locks the behaviour as a conformance contract. Reference implementations: loomweave plugins/python extractor (too_complex, d5baac5); legis policy/boundary_scan.py POLICY_BOUNDARY_FILE_TOO_COMPLEX. Co-Authored-By: Claude Fable 5 --- .../conformance/test_hostile_input_degrade.py | 105 ++++++++++++++++++ 1 file changed, 105 insertions(+) create mode 100644 tests/conformance/test_hostile_input_degrade.py diff --git a/tests/conformance/test_hostile_input_degrade.py b/tests/conformance/test_hostile_input_degrade.py new file mode 100644 index 00000000..77fa9d4b --- /dev/null +++ b/tests/conformance/test_hostile_input_degrade.py @@ -0,0 +1,105 @@ +"""C-13 — fail-degraded, never fail-dead: one hostile file never kills a run. + +Federation convention C-13 (hub `conventions.md`, ratified `b0eee6e`; hub issue +weft-b181c75e39): a member tool that walks or parses project source MUST NOT +hard-fail the whole run on one hostile input. Per file: skip + flag + continue — +the skip is an IN-BAND marker in the result envelope (a finding, never only a log +line), and the run completes over the remaining files. + +This is wardline's adoption fixture: the ``nesting_bomb``-class corpus that killed +legis's ``policy-boundary-check`` with an uncaught RecursionError (dogfood-4 A2) +and that loomweave's python extractor degrades to ``too_complex`` (the C-13 +reference, `d5baac5`). Reference implementations: loomweave +``plugins/python/.../extractor.py``; legis ``policy/boundary_scan.py`` 58-80 +(``POLICY_BOUNDARY_FILE_TOO_COMPLEX``). + +The bombs are GENERATED here (a 20,000-term line has no business in git history); +``lacuna/specimen/nesting_bomb.py`` is the live twin this corpus mirrors. +""" + +from __future__ import annotations + +from pathlib import Path + +from wardline.core.finding import Kind, Severity +from wardline.core.run import run_scan + +# In-band per-file degrade markers the engine may legitimately emit for a hostile +# file, in any stage: parse (PARSE-ERROR for parser-level rejections, +# FILE-SKIPPED for RecursionError), per-entity L2 (FUNCTION-SKIPPED), or the +# unexpected-exception isolation net (FILE-FAILED). +_DEGRADE_RULES = frozenset( + { + "WLN-ENGINE-PARSE-ERROR", + "WLN-ENGINE-FILE-SKIPPED", + "WLN-ENGINE-FUNCTION-SKIPPED", + "WLN-ENGINE-FILE-FAILED", + } +) + +# A healthy violating neighbour: its PY-WL-101 finding proves the scan kept going +# and kept ANALYZING after meeting the hostile files (alphabetically the bombs +# sort both before and after it). +_LEAKY = ( + "from wardline.decorators import external_boundary, trusted\n" + "@external_boundary\ndef read_raw(p):\n return p\n" + "@trusted\ndef leaky(p):\n return read_raw(p)\n" +) + +_BINOP_TERMS = 20_000 # parses fine; blows naive recursive AST visitors (killed legis) +_NESTING_DEPTH = 600 # deeply nested if blocks; rejected at the parser layer + + +def _hostile_corpus(tmp_path: Path) -> Path: + proj = tmp_path / "proj" + proj.mkdir() + (proj / "middle_clean.py").write_text(_LEAKY, encoding="utf-8") + # (a) deeply nested if blocks — hostile to the parser/compiler layer. + nested = ["x = 1"] + nested += [" " * i + f"if x > {i}:" for i in range(_NESTING_DEPTH)] + nested += [" " * _NESTING_DEPTH + "pass"] + (proj / "aaa_nesting_bomb.py").write_text("\n".join(nested) + "\n", encoding="utf-8") + # (b) the 20,000-term BinOp chain, module-level (lacuna's specimen/nesting_bomb.py + # shape) and inside a function body (so the per-entity walks meet it too). + chain = "+".join(["1"] * _BINOP_TERMS) + (proj / "aab_binop_bomb_module.py").write_text(f"BOMB = {chain}\n", encoding="utf-8") + (proj / "zzz_binop_bomb_func.py").write_text(f"def f():\n return {chain}\n", encoding="utf-8") + return proj + + +_HOSTILE_FILES = ("aaa_nesting_bomb.py", "aab_binop_bomb_module.py", "zzz_binop_bomb_func.py") + + +def test_hostile_corpus_scan_completes_flags_and_continues(tmp_path: Path) -> None: + proj = _hostile_corpus(tmp_path) + + # COMPLETES: no RecursionError (or anything else) escapes the scan. + result = run_scan(proj, confine_to_root=True) + + # FLAGS: every hostile file carries an in-band degrade marker in the result + # envelope — a finding the consumer can see, never only a log line. + by_path = {p: [f for f in result.findings if f.location.path == p] for p in _HOSTILE_FILES} + for path, findings in by_path.items(): + markers = [f for f in findings if f.rule_id in _DEGRADE_RULES] + assert markers, f"{path}: hostile file skipped with NO in-band marker (C-13 violation)" + # Fail-closed posture: unscanned code must not read GREEN — the marker is a + # gate-eligible ERROR DEFECT, not an ignorable FACT. + assert all(m.kind is Kind.DEFECT and m.severity is Severity.ERROR for m in markers), path + + # CONTINUES: the healthy neighbour's findings are still reported. + leaky = [f for f in result.findings if f.rule_id == "PY-WL-101" and f.location.path == "middle_clean.py"] + assert leaky, "scan lost the healthy file's findings after meeting the hostile ones" + + # The degraded scope is visible in the envelope (count), per C-10(a). + assert result.summary.unanalyzed >= len(_HOSTILE_FILES) - 1 # FUNCTION-SKIPPED counts per entity, not per file + assert result.files_scanned == 4 # all four files were DISCOVERED, none aborted the run + + +def test_hostile_corpus_gate_trips_rather_than_reads_green(tmp_path: Path) -> None: + # C-13 + the fail-closed gate: a degraded scan must be able to TRIP an ERROR + # gate (the skipped files were never analyzed), never silently pass as green. + proj = _hostile_corpus(tmp_path) + result = run_scan(proj, confine_to_root=True) + degrade_markers = [f for f in result.findings if f.rule_id in _DEGRADE_RULES] + assert degrade_markers + assert {f.severity for f in degrade_markers} == {Severity.ERROR} From 7c97eaae0f9dcb5d5e24f38c81d7184c8f821f14 Mon Sep 17 00:00:00 2001 From: John Morrissey Date: Sat, 13 Jun 2026 06:29:41 +1000 Subject: [PATCH 160/186] test(golden): regen identity corpus for the additive TaintExplanation.sink field (weft-0d24cf9152) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Sanctioned regen (golden.identity.regen --reason): the B7 explain fix adds the 'sink' field to the explanation projection, so the three input corpora gain one additive 'sink: null' line each — every fingerprint VALUE is byte-identical (verified by diff: the only changed lines are the new key). assure.json deliberately NOT regenerated: its drift (unanalyzed_total) is pre-existing on rc5 and not adjudicated by this change — absorbing it here would hide a separate intent-or-regression question. test_assure_posture_is_frozen stays red exactly as at baseline. Co-Authored-By: Claude Fable 5 --- tests/golden/identity/corpus/META.json | 2 +- tests/golden/identity/corpus/sampleapp.json | 1 + tests/golden/identity/corpus/sinks.json | 1 + tests/golden/identity/corpus/stress.json | 1 + 4 files changed, 4 insertions(+), 1 deletion(-) diff --git a/tests/golden/identity/corpus/META.json b/tests/golden/identity/corpus/META.json index b3da5a9b..e42457bc 100644 --- a/tests/golden/identity/corpus/META.json +++ b/tests/golden/identity/corpus/META.json @@ -1,5 +1,5 @@ { "corpus_version": 4, "fingerprint_scheme": "wlfp2", - "reason": "2026-06-10/11 python-plugin hardening (two regens, scheme wlfp2 unchanged): (1) PY-WL-108/112 base severity WARN->ERROR (command injection calibrated to the SQLi exploit class); (2) Compare-position resolution + nested-def return summaries: actual_return precision UNKNOWN_RAW->INTEGRAL for comparison returns and local helpers. Prior rekey provenance: wlfp2/wardline-8654423823 (drop line_start + entity-relative move-stable discriminators)." + "reason": "explain_taint B7 fix (weft-0d24cf9152): TaintExplanation gains the additive 'sink' field (dotted sink callable for call-site-anchored sink findings); fingerprint values unchanged" } diff --git a/tests/golden/identity/corpus/sampleapp.json b/tests/golden/identity/corpus/sampleapp.json index 2fea242e..7b2673b5 100644 --- a/tests/golden/identity/corpus/sampleapp.json +++ b/tests/golden/identity/corpus/sampleapp.json @@ -549,6 +549,7 @@ "path": "trust_flow.py", "resolved_call_count": 1, "rule_id": "PY-WL-101", + "sink": null, "sink_qualname": "trust_flow.unsafe_account_key", "source_boundary_qualname": "trust_flow.read_raw_username", "tier_in": "EXTERNAL_RAW", diff --git a/tests/golden/identity/corpus/sinks.json b/tests/golden/identity/corpus/sinks.json index f733fdef..43e3bbb1 100644 --- a/tests/golden/identity/corpus/sinks.json +++ b/tests/golden/identity/corpus/sinks.json @@ -129,6 +129,7 @@ "path": "wardline_sinks.py", "resolved_call_count": 1, "rule_id": "PY-WL-101", + "sink": null, "sink_qualname": "wardline_sinks.import_catalog_blob", "source_boundary_qualname": null, "tier_in": "UNKNOWN_RAW", diff --git a/tests/golden/identity/corpus/stress.json b/tests/golden/identity/corpus/stress.json index 4afb9095..9c467e9b 100644 --- a/tests/golden/identity/corpus/stress.json +++ b/tests/golden/identity/corpus/stress.json @@ -139,6 +139,7 @@ "path": "identity_stress.py", "resolved_call_count": 0, "rule_id": "PY-WL-111", + "sink": null, "sink_qualname": "identity_stress.assert_only_boundary", "source_boundary_qualname": null, "tier_in": null, From 7aa98af187f818bb01d56f3aa5f0ad2f1ccb40c6 Mon Sep 17 00:00:00 2001 From: John Morrissey Date: Sat, 13 Jun 2026 13:12:59 +1000 Subject: [PATCH 161/186] fix(taint): PY-WL-101 reads declared body taint; @trusted tolerates legacy to_level Two trust-surface correctness fixes on the producer rule and its provider: - decorator_provider now threads ignored_args={'to_level'} through _read_level for @trusted, so @trusted(level=, to_level=) reads as a statically-known trusted producer instead of failing closed on the legacy compatibility kwarg. Genuinely unknown kwargs still fail closed (P0 PY-WL-101 taint fix). - PY-WL-101's trust-RAISING delegation now consults a new AnalysisContext.declared_body_taints snapshot (the provider's original body tier, captured in the analyzer before L3 callgraph refinement) instead of context.project_taints. L3 propagation can make a leaky @trusted producer's body raw; reading the propagated map there wrongly delegated a real leak to PY-WL-102. The declaration snapshot stays stable, so a leaky producer is policed by 101 as intended while genuine @trust_boundary validators remain delegated. Co-Authored-By: Claude Opus 4.8 (1M context) --- src/wardline/scanner/analyzer.py | 3 +++ src/wardline/scanner/context.py | 23 +++++++++++++------ .../rules/untrusted_reaches_trusted.py | 20 +++++++++------- .../scanner/taint/decorator_provider.py | 19 ++++++++++++++- .../scanner/taint/test_decorator_provider.py | 10 ++++++++ 5 files changed, 59 insertions(+), 16 deletions(-) diff --git a/src/wardline/scanner/analyzer.py b/src/wardline/scanner/analyzer.py index 79b9e5cc..acc99cd1 100644 --- a/src/wardline/scanner/analyzer.py +++ b/src/wardline/scanner/analyzer.py @@ -931,6 +931,9 @@ def _l2_nested_def_overlay() -> dict[str, dict[str, TaintState]]: entities=entity_index, taint_provenance=dict(result.taint_provenance), declared_qualnames=frozenset(q for m in modules for q, s in m.seeds.items() if s.source == "provider"), + declared_body_taints={ + q: s.body_taint for m in modules for q, s in m.seeds.items() if s.source == "provider" + }, project_edges=result.project_edges, call_site_implicit_receivers=result.call_site_implicit_receivers, alias_maps={m.module_path: m.alias_map for m in modules}, diff --git a/src/wardline/scanner/context.py b/src/wardline/scanner/context.py index bbb1a5c6..d0b23270 100644 --- a/src/wardline/scanner/context.py +++ b/src/wardline/scanner/context.py @@ -48,13 +48,17 @@ class AnalysisContext: to a source dict) cannot mutate engine output. ``project_return_taints`` is the effective return tier per function (anchored: - declared; non-anchored: refined body). ``function_return_taints`` is the actual - least-trusted returned-value taint per function, computed from L2 variable - analysis — the precise input for PY-WL-101. ``function_return_callee`` is the - callee that contributed each function's actual (least-trusted) return taint, or - ``None`` when that worst return path is not a direct call (``return p`` / - ``return some_var`` — chain resolution is deferred to SP9). This is the property - ``explain_finding`` reports as the immediate tainted callee. + declared; non-anchored: refined body). ``declared_body_taints`` is the provider's + original body tier for declared functions only, before L3 propagation refines the + body map. Rules that need declaration shape (for example, distinguishing + ``@trust_boundary`` from a leaky ``@trusted`` producer) must read this snapshot, + not ``project_taints``. ``function_return_taints`` is the actual least-trusted + returned-value taint per function, computed from L2 variable analysis — the + precise input for PY-WL-101. ``function_return_callee`` is the callee that + contributed each function's actual (least-trusted) return taint, or ``None`` when + that worst return path is not a direct call (``return p`` / ``return some_var`` — + chain resolution is deferred to SP9). This is the property ``explain_finding`` + reports as the immediate tainted callee. """ project_taints: Mapping[str, TaintState] @@ -94,6 +98,10 @@ class AnalysisContext: # denominator. Additive + defaulted so direct constructions/tests need not # supply it; frozenset is already immutable so no proxy wrap is needed. declared_qualnames: frozenset[str] = frozenset() + # Provider-declared body tier for trust-surface functions, before L3 callgraph + # refinement. ``project_taints`` is the propagated body tier and may become raw + # for a leaky ``@trusted`` producer; this declaration snapshot stays stable. + declared_body_taints: Mapping[str, TaintState] = field(default_factory=dict) # Inter-module call edges: ``{caller: frozenset({callees})}``. Defaulted for # direct constructions; absence means no project edges available. project_edges: Mapping[str, frozenset[str]] = field(default_factory=dict) @@ -153,6 +161,7 @@ def __post_init__(self) -> None: _freeze_mapping(self.call_site_candidate_callees), ) object.__setattr__(self, "class_attr_taints", _freeze_mapping(self.class_attr_taints)) + object.__setattr__(self, "declared_body_taints", _freeze_mapping(self.declared_body_taints)) object.__setattr__( self, "project_edges", diff --git a/src/wardline/scanner/rules/untrusted_reaches_trusted.py b/src/wardline/scanner/rules/untrusted_reaches_trusted.py index 36ea1a99..1478c07c 100644 --- a/src/wardline/scanner/rules/untrusted_reaches_trusted.py +++ b/src/wardline/scanner/rules/untrusted_reaches_trusted.py @@ -9,15 +9,16 @@ is what makes the strict rank comparison safe. Declaration-gated, so it emits at base severity (NOT tier-modulated). -**Trust-boundary delegation.** A trust-RAISING transition — a function whose body -taint is strictly less-trusted than its declared return (the taint shape unique -to ``@trust_boundary``) — is EXEMPT from this rule and delegated to PY-WL-102. +**Trust-boundary delegation.** A declared trust-RAISING transition — a function +whose provider-declared body taint is strictly less-trusted than its declared +return (the taint shape unique to ``@trust_boundary``) — is EXEMPT from this rule +and delegated to PY-WL-102. Reason: such a validator's parameters seed at the raw body taint, and the engine does not narrow taint after a ``raise`` guard, so the L2 actual return is always the raw body taint — meaning *every* ``@trust_boundary`` (correct or not) would fire here. That is noise, not signal: the statically-decidable property for a validator is "can it reject at all", which is exactly PY-WL-102's check. So -PY-WL-101 polices ``@trusted`` producers (body == declared) only. +PY-WL-101 polices ``@trusted`` producers (declared body == declared return) only. """ from __future__ import annotations @@ -87,13 +88,16 @@ def check(self, context: AnalysisContext) -> list[Finding]: prov = context.taint_provenance.get(qualname) if prov is None or prov.source != "anchored": continue - body = context.project_taints.get(qualname) declared = context.project_return_taints.get(qualname) if declared is None or declared in RAW_ZONE: continue # trust-claim gate - # Trust-RAISING transition (body less trusted than return) is - # @trust_boundary's shape — covered by PY-WL-102, not PY-WL-101. - if body is not None and TRUST_RANK[body] > TRUST_RANK[declared]: + declared_body = context.declared_body_taints.get(qualname) + # Declared trust-RAISING transition (body less trusted than return) is + # @trust_boundary's shape — covered by PY-WL-102, not PY-WL-101. Do + # not read context.project_taints here: L3 propagation can make a + # leaky @trusted producer's body raw, but that is evidence for 101, + # not a declaration that turns the producer into a validator. + if declared_body is not None and TRUST_RANK[declared_body] > TRUST_RANK[declared]: continue actual = context.function_return_taints.get(qualname) if actual is None: diff --git a/src/wardline/scanner/taint/decorator_provider.py b/src/wardline/scanner/taint/decorator_provider.py index 8498c099..f55acf4f 100644 --- a/src/wardline/scanner/taint/decorator_provider.py +++ b/src/wardline/scanner/taint/decorator_provider.py @@ -150,6 +150,7 @@ def _read_level( allowed: frozenset[TaintState], default: TaintState | None, alias_map: Mapping[str, str], + ignored_args: frozenset[str] = frozenset(), ) -> TaintState | None: """Read a level keyword arg from a decorator, normalised + allow-checked. @@ -171,6 +172,8 @@ def _read_level( for key, value in zip(kw.value.keys, kw.value.values, strict=True): if not isinstance(key, ast.Constant) or not isinstance(key.value, str): return None + if key.value in ignored_args: + continue if key.value != arg: return None values.append(value) @@ -178,6 +181,8 @@ def _read_level( if kw.arg == arg: values.append(kw.value) continue + if kw.arg in ignored_args: + continue return None if not values: return default @@ -392,7 +397,19 @@ def _match( levels: dict[str, TaintState] = {} unreadable = False for la in bt.level_args: - lvl = _read_level(deco, la.arg_name, allowed=la.allowed, default=la.default, alias_map=alias_map) + # Legacy review fixtures and older sample code sometimes supplied + # ``to_level`` on ``@trusted``. Treat it as inert compatibility + # only when the real ``level`` argument remains statically readable; + # genuinely unknown kwargs still fail closed. + ignored = frozenset({"to_level"}) if bt.canonical_name == "trusted" else frozenset() + lvl = _read_level( + deco, + la.arg_name, + allowed=la.allowed, + default=la.default, + alias_map=alias_map, + ignored_args=ignored, + ) if lvl is None: unreadable = True break diff --git a/tests/unit/scanner/taint/test_decorator_provider.py b/tests/unit/scanner/taint/test_decorator_provider.py index 6f0388d4..92a861c1 100644 --- a/tests/unit/scanner/taint/test_decorator_provider.py +++ b/tests/unit/scanner/taint/test_decorator_provider.py @@ -161,6 +161,16 @@ def test_trusted_level_assured() -> None: assert out["m.f"] == FunctionTaint(T.ASSURED, T.ASSURED) +def test_trusted_level_tolerates_legacy_to_level_keyword() -> None: + out = _seed( + "from wardline.decorators import trusted\n" + "@trusted(level='ASSURED', to_level='ASSURED')\n" + "def f():\n" + " return 1\n" + ) + assert out["m.f"] == FunctionTaint(T.ASSURED, T.ASSURED) + + def test_trusted_level_static_kwargs_assured() -> None: out = _seed( "from wardline.decorators import trusted\n@trusted(**{'level': 'ASSURED'})\ndef f():\n return 1\n" From 9e530c8e942cd5d1b241028133bad9c98fbb8248 Mon Sep 17 00:00:00 2001 From: John Morrissey Date: Sat, 13 Jun 2026 13:13:43 +1000 Subject: [PATCH 162/186] feat(scan): pollable file-backed scan jobs + Filigree-emit hardening MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Land the RC5 scan-execution WIP as one cohesive unit: - Pollable scan jobs (core/scan_jobs.py, cli/scan_job.py, cli/scan_job_worker.py, cli scan-job command, MCP scan_job_start/status/cancel tools). A file-backed, daemon-free job writes status JSON under .weft/wardline/jobs/ while a worker subprocess advances through scan -> artifact -> optional enrichment -> gate. Status reads refresh liveness (dead-worker / stale-heartbeat) so a hung job is never ambiguous. run_scan gains a progress_callback the worker threads through. The MCP surface grows from 15 to 18 tools; handshake / structured-output conformance and the glossary code-anchor line locks are updated to match. - Filigree emit hardening: max_findings_per_request capping (CLI --filigree-max-findings-per-request + env, MCP arg, threaded into jobs), --local-only / --no-emit to fence sibling emission, and a fail-soft protocol_errors_loud=False mode so a Filigree reject is reported as an upload failure rather than a pre-gate exit 2. ENRICH-ONLY preserved: a sibling's absence never breaks the core scan/gate. - finding.to_filigree_metadata maps property-accessor qualnames (:setter/:deleter) to the bare wire qualname for cross-tool reconciliation. - assure posture gains an additive unanalyzed_total coverage field; the frozen identity corpus is regenerated (additive field only — fingerprints, spans and SARIF are byte-unchanged). Full suite green (3948 passed); wardline self-scan gate clean (0 active). Co-Authored-By: Claude Opus 4.8 (1M context) --- .../reference/finding-lifecycle-vocabulary.md | 72 +-- src/wardline/cli/doctor.py | 11 +- src/wardline/cli/main.py | 2 + src/wardline/cli/scan.py | 49 +- src/wardline/cli/scan_job.py | 119 +++++ src/wardline/cli/scan_job_worker.py | 24 + src/wardline/core/discovery.py | 25 +- src/wardline/core/filigree_emit.py | 314 +++++++++-- src/wardline/core/finding.py | 12 +- src/wardline/core/run.py | 17 +- src/wardline/core/scan_jobs.py | 497 ++++++++++++++++++ src/wardline/install/block.py | 3 +- src/wardline/install/doctor.py | 58 +- src/wardline/mcp/server.py | 233 ++++++++ src/wardline/rust/analyzer.py | 26 +- .../filigree_suppression_filter_contract.json | 20 + tests/conformance/fixtures/PROVENANCE.md | 10 + .../fixtures/sei-conformance-oracle.json | 85 +++ .../legis_dirty_scan_wire.golden.json | 43 ++ ...st_filigree_suppression_filter_contract.py | 36 ++ .../test_legis_scan_wire_golden.py | 44 ++ tests/conformance/test_mcp_handshake.py | 5 +- .../conformance/test_mcp_structured_output.py | 66 ++- tests/conformance/test_sei_oracle.py | 156 ++++++ tests/docs/test_glossary_vocabulary.py | 56 +- tests/golden/identity/corpus/META.json | 2 +- tests/golden/identity/corpus/assure.json | 3 + tests/unit/cli/test_cli.py | 80 ++- tests/unit/cli/test_doctor.py | 50 +- tests/unit/cli/test_install.py | 8 +- tests/unit/cli/test_scan_job.py | 204 +++++++ tests/unit/core/test_discovery.py | 12 + tests/unit/core/test_filigree_emit.py | 115 ++++ tests/unit/core/test_finding.py | 9 + tests/unit/core/test_run.py | 13 + tests/unit/install/test_block.py | 1 + tests/unit/mcp/test_server_scan_jobs.py | 147 ++++++ tests/unit/mcp/test_server_structure.py | 3 + tests/unit/rust/test_analyzer_protocol.py | 21 +- 39 files changed, 2490 insertions(+), 161 deletions(-) create mode 100644 src/wardline/cli/scan_job.py create mode 100644 src/wardline/cli/scan_job_worker.py create mode 100644 src/wardline/core/scan_jobs.py create mode 100644 tests/conformance/filigree_suppression_filter_contract.json create mode 100644 tests/conformance/fixtures/PROVENANCE.md create mode 100644 tests/conformance/fixtures/sei-conformance-oracle.json create mode 100644 tests/conformance/legis_dirty_scan_wire.golden.json create mode 100644 tests/conformance/test_filigree_suppression_filter_contract.py create mode 100644 tests/conformance/test_sei_oracle.py create mode 100644 tests/unit/cli/test_scan_job.py create mode 100644 tests/unit/mcp/test_server_scan_jobs.py diff --git a/docs/reference/finding-lifecycle-vocabulary.md b/docs/reference/finding-lifecycle-vocabulary.md index 607dbece..d528dddb 100644 --- a/docs/reference/finding-lifecycle-vocabulary.md +++ b/docs/reference/finding-lifecycle-vocabulary.md @@ -50,18 +50,18 @@ onto the state (`src/wardline/core/suppression.py:78-87`). ### The per-finding key is `suppression_state` (not `suppressed`) Each serialized finding carries its state under the key **`suppression_state`** -(`src/wardline/core/finding.py:140` in `to_jsonl`; `src/wardline/core/finding.py:285` +(`src/wardline/core/finding.py:140` in `to_jsonl`; `src/wardline/core/finding.py:295` in the Filigree `metadata.wardline.*` subtree; the agent-summary entries and the legis artifact use the same key). The key was renamed from `suppressed` → `suppression_state` (weft-f506e5f845) so the per-finding **state** never reads as the opposite of the summary's `active` **count**: `suppression_state: "active"` clearly names a state, while `summary.active` is a count of unsuppressed defects. The Filigree metadata only carries the key when the state is not `active` -(`src/wardline/core/finding.py:285`). +(`src/wardline/core/finding.py:295`). **"suppressed"** survives only as the umbrella *word* for "any state other than `active`": `baselined` + `waived` + `judged`. The CLI prints this sum as the -`suppressed` count (`src/wardline/cli/scan.py:415`). +`suppressed` count (`src/wardline/cli/scan.py:457`). ## `active` is the one word for "non-suppressed defect" @@ -71,9 +71,9 @@ consistently, on every surface: | Surface | Where | Term | | --- | --- | --- | | Enum | `src/wardline/core/finding.py:72` | `SuppressionState.ACTIVE = "active"` | -| Summary field | `src/wardline/core/run.py:51`, built at `src/wardline/core/run.py:378` | `ScanSummary.active` | -| CLI summary line | `src/wardline/cli/scan.py:416` | `… {s.active} active` | -| MCP scan response | `src/wardline/mcp/server.py:775` | `summary.active` | +| Summary field | `src/wardline/core/run.py:52`, built at `src/wardline/core/run.py:393` | `ScanSummary.active` | +| CLI summary line | `src/wardline/cli/scan.py:458` | `… {s.active} active` | +| MCP scan response | `src/wardline/mcp/server.py:842` | `summary.active` | | Agent-summary JSON | `src/wardline/core/agent_summary.py:135` | `summary.active_defects` | | `wardline:loop` prompt | `src/wardline/mcp/prompts.py:13` | "Read `summary.active`" | @@ -88,21 +88,21 @@ surfaces. ## The summary buckets partition the total -`ScanSummary` (`src/wardline/core/run.py:48-69`) counts split the whole scan into +`ScanSummary` (`src/wardline/core/run.py:50-70`) counts split the whole scan into buckets that **sum to `total`** exactly (weft-f506e5f845): - the defect buckets partition the `DEFECT`s by state — - `active` (`src/wardline/core/run.py:51`) + `baselined` (`src/wardline/core/run.py:53`) - + `waived` (`src/wardline/core/run.py:54`) + `judged` (`src/wardline/core/run.py:55`); -- `informational` (`src/wardline/core/run.py:61`) is **every non-defect finding** + `active` (`src/wardline/core/run.py:52`) + `baselined` (`src/wardline/core/run.py:54`) + + `waived` (`src/wardline/core/run.py:55`) + `judged` (`src/wardline/core/run.py:56`); +- `informational` (`src/wardline/core/run.py:62`) is **every non-defect finding** (facts, metrics, classifications) — the rest of `total`. So `active + baselined + waived + judged + informational == total` -(`src/wardline/core/run.py:50` for `total: int`). `unanalyzed` -(`src/wardline/core/run.py:69`) is an **overlay** — a subset of `informational` +(`src/wardline/core/run.py:51` for `total: int`). `unanalyzed` +(`src/wardline/core/run.py:70`) is an **overlay** — a subset of `informational` that surfaces a silent under-scan — and is deliberately **not** a partition member. -The MCP `summary` block exposes `informational` (`src/wardline/mcp/server.py:783`) -and `unanalyzed` (`src/wardline/mcp/server.py:787`); the agent-summary block mirrors +The MCP `summary` block exposes `informational` (`src/wardline/mcp/server.py:850`) +and `unanalyzed` (`src/wardline/mcp/server.py:854`); the agent-summary block mirrors both (`src/wardline/core/agent_summary.py:146`, `src/wardline/core/agent_summary.py:147`). ## Emitted-active vs the gate population @@ -111,19 +111,19 @@ There are **two distinct populations** of defects in one scan, and they can differ on purpose: 1. **Emitted-active** — `summary.active` counts `active` defects in the - **emitted** (post-annotation) findings (built at `src/wardline/core/run.py:378`). + **emitted** (post-annotation) findings (built at `src/wardline/core/run.py:393`). Baseline / waiver / judged annotate these findings in place; a suppressed defect is still emitted, just not counted as `active`. 2. **Gate population** — the `--fail-on` gate evaluates a **separate** `ScanResult.gate_findings` list: the *unsuppressed* population - (`src/wardline/core/run.py:326`). By default, repository-controlled + (`src/wardline/core/run.py:341`). By default, repository-controlled baseline / waiver / judged entries **annotate** the emitted findings but do **not** clear the gate — so a malicious PR cannot green the gate by committing a suppression keyed to its own new defect. `gate_decision` evaluates `gate_findings` when present, else falls back to `findings` (the trusted `--trust-suppressions` / directly-constructed path), selected at - `src/wardline/core/run.py:437` (`honors_suppressions`). + `src/wardline/core/run.py:452` (`honors_suppressions`). This is why **`summary.active: 0` can co-exist with `gate.tripped: true`**: every defect was suppressed by a committed baseline (so emitted-active is 0), but those @@ -132,8 +132,8 @@ bug. ### The gate verdict is explicit (never a vacuous green) -`GateDecision` (`src/wardline/core/run.py:98`) carries `tripped` / `fail_on` / -`exit_class` **plus** an explicit `verdict` (`src/wardline/core/run.py:107`) and a +`GateDecision` (`src/wardline/core/run.py:99`) carries `tripped` / `fail_on` / +`exit_class` **plus** an explicit `verdict` (`src/wardline/core/run.py:108`) and a `would_trip_at`, alongside a human `reason` and the `evaluated` population it judged. The `verdict` is one of: @@ -150,20 +150,20 @@ trips when any file was discovered but never analysed; benign no-module skips excluded). `severity_tripped` / `unanalyzed_tripped` attribute an overall `tripped` to its sub-gate(s) so no consumer has to parse `reason`. -The MCP `scan` gate block exposes `gate.tripped` (`src/wardline/mcp/server.py:790`), -`gate.fail_on_unanalyzed`, `gate.verdict` (`src/wardline/mcp/server.py:794`), +The MCP `scan` gate block exposes `gate.tripped` (`src/wardline/mcp/server.py:857`), +`gate.fail_on_unanalyzed`, `gate.verdict` (`src/wardline/mcp/server.py:861`), `gate.severity_tripped`, `gate.unanalyzed_tripped`, `would_trip_at`, `reason`, -`evaluated`, and `migration_hint`, opened at `src/wardline/mcp/server.py:789` +`evaluated`, and `migration_hint`, opened at `src/wardline/mcp/server.py:856` (`"gate": {`); the agent-summary mirrors them at `src/wardline/core/agent_summary.py:150` (`tripped`) and `src/wardline/core/agent_summary.py:153` (`verdict`). The CLI prints `gate: FAILED () — ` then `gate: evaluated <…>`, or a `gate: NOT_EVALUATED — …` line for a bare scan -(`src/wardline/cli/scan.py:468`). +(`src/wardline/cli/scan.py:510`). `--new-since` scopes **both** populations identically: any `active` defect outside the delta is re-marked `baselined` in both the emitted and gate lists -(`src/wardline/core/run.py:354`, `def apply_delta_scope`). +(`src/wardline/core/run.py:369`, `def apply_delta_scope`). ## The three meanings of "new" @@ -174,8 +174,8 @@ still legitimately means three different things depending on the surface: | "new" on this surface | Means | Owner / anchor | | --- | --- | --- | | Filigree store | An **unseen fingerprint** — first time this finding identity is seen for a `(file, scan_source)`. | **Filigree-owned** lifecycle (`src/wardline/core/filigree_emit.py:68-76`) | -| `wardline scan --new-since ` | **Delta-scope**: the gate fires only on defects in files/entities changed since a git ref; everything else is re-marked `baselined`. | `src/wardline/core/run.py:354`; help text `src/wardline/cli/scan.py` (`--new-since`) | -| (historical) CLI summary | Formerly relabelled the `active` count as "N new". **Corrected to "N active"**. | `src/wardline/cli/scan.py:415` | +| `wardline scan --new-since ` | **Delta-scope**: the gate fires only on defects in files/entities changed since a git ref; everything else is re-marked `baselined`. | `src/wardline/core/run.py:369`; help text `src/wardline/cli/scan.py` (`--new-since`) | +| (historical) CLI summary | Formerly relabelled the `active` count as "N new". **Corrected to "N active"**. | `src/wardline/cli/scan.py:457` | The first-seen Filigree sense and the delta-scope `--new-since` sense are genuinely distinct concepts; neither is "active". @@ -186,19 +186,19 @@ How each concept appears on each surface: | Concept | CLI summary text | `ScanSummary` field | MCP `summary` key | Agent-summary key | Filigree store | | --- | --- | --- | --- | --- | --- | -| every finding | `N finding(s)` | `total` (`run.py:50`) | `total` (`server.py:774`) | `total_findings` (`agent_summary.py:134`) | one finding per wire entry | -| live defect | `N active` (`scan.py:416`) | `active` (`run.py:51,361`) | `active` (`server.py:775`) | `active_defects` (`agent_summary.py:135`) | no `suppression_state` key (`finding.py:285`) | -| suppressed (sum) | `N suppressed` (`scan.py:415`) | `baselined+waived+judged` | the three keys | `suppressed_findings` (`agent_summary.py:136`) | `metadata.wardline.suppression_state` (`finding.py:285`) | -| baselined | `N baseline` | `baselined` (`run.py:53`) | `baselined` (`server.py:776`) | `baselined` (`agent_summary.py:138`) | `suppression_state: "baselined"` | -| waived | `N waiver` | `waived` (`run.py:54`) | `waived` (`server.py:777`) | `waived` (`agent_summary.py:139`) | `suppression_state: "waived"` | -| judged | `N judged` | `judged` (`run.py:55`) | `judged` (`server.py:778`) | `judged` (`agent_summary.py:140`) | `suppression_state: "judged"` | -| informational (summary) | (the remainder of `total`) | `informational` (`run.py:61`) | `informational` (`server.py:783`) | `informational` (`agent_summary.py:146`) | facts/metrics | +| every finding | `N finding(s)` | `total` (`run.py:51`) | `total` (`server.py:841`) | `total_findings` (`agent_summary.py:134`) | one finding per wire entry | +| live defect | `N active` (`scan.py:458`) | `active` (`run.py:52,393`) | `active` (`server.py:842`) | `active_defects` (`agent_summary.py:135`) | no `suppression_state` key (`finding.py:295`) | +| suppressed (sum) | `N suppressed` (`scan.py:457`) | `baselined+waived+judged` | the three keys | `suppressed_findings` (`agent_summary.py:136`) | `metadata.wardline.suppression_state` (`finding.py:295`) | +| baselined | `N baseline` | `baselined` (`run.py:54`) | `baselined` (`server.py:843`) | `baselined` (`agent_summary.py:138`) | `suppression_state: "baselined"` | +| waived | `N waiver` | `waived` (`run.py:55`) | `waived` (`server.py:844`) | `waived` (`agent_summary.py:139`) | `suppression_state: "waived"` | +| judged | `N judged` | `judged` (`run.py:56`) | `judged` (`server.py:845`) | `judged` (`agent_summary.py:140`) | `suppression_state: "judged"` | +| informational (summary) | (the remainder of `total`) | `informational` (`run.py:62`) | `informational` (`server.py:850`) | `informational` (`agent_summary.py:146`) | facts/metrics | | informational (display) | n/a | n/a | n/a | `informational` display array (`agent_summary.py:171`) — non-defect, non-engine-fact findings (metrics, classifications, suggestions, non-engine facts); excludes `engine_facts` which has its own display slot | facts/metrics | -| under-scan | `N file(s) could not be analyzed` | `unanalyzed` (`run.py:69`) | `unanalyzed` (`server.py:787`) | `unanalyzed` (`agent_summary.py:147`) | `WLN-ENGINE-*` facts | -| gate verdict | exit code + `--fail-on` | (`gate_findings`, `run.py:88`; `GateDecision`, `run.py:98`, `verdict` `run.py:107`) | `gate.tripped` (`server.py:790`), `gate.verdict` (`server.py:794`) | `gate.tripped` (`agent_summary.py:150`), `gate.verdict` (`agent_summary.py:153`) | not emitted to Filigree | +| under-scan | `N file(s) could not be analyzed` | `unanalyzed` (`run.py:70`) | `unanalyzed` (`server.py:854`) | `unanalyzed` (`agent_summary.py:147`) | `WLN-ENGINE-*` facts | +| gate verdict | exit code + `--fail-on` | (`gate_findings`, `run.py:89`; `GateDecision`, `run.py:99`, `verdict` `run.py:108`) | `gate.tripped` (`server.py:857`), `gate.verdict` (`server.py:861`) | `gate.tripped` (`agent_summary.py:150`), `gate.verdict` (`agent_summary.py:153`) | not emitted to Filigree | The unsuppressed gate population is built from `Baseline(frozenset())` -(`src/wardline/core/run.py:326`). +(`src/wardline/core/run.py:341`). ## For the suite diff --git a/src/wardline/cli/doctor.py b/src/wardline/cli/doctor.py index 43e8a3c8..a7442657 100644 --- a/src/wardline/cli/doctor.py +++ b/src/wardline/cli/doctor.py @@ -9,6 +9,7 @@ from wardline.core.errors import WardlineError from wardline.install.doctor import ( + _check_config, _check_filigree_auth, _resolve_probe_url, check_install, @@ -53,24 +54,30 @@ def doctor(root: Path, repair: bool, fix_json: bool, filigree_url: str | None) - click.echo(f"error: {exc}", err=True) raise SystemExit(2) from exc after = check_install(root) + config_check = _check_config(root, fixed=statuses.get("weft.toml") == "created") click.echo("wardline doctor:") for check in after: status = statuses.get(check.name, "checked") if check.ok else f"failed ({check.message})" click.echo(f" {check.name}: {status}") + config_status = statuses.get("weft.toml", "checked") if config_check.ok else f"failed ({config_check.message})" + click.echo(f" weft.toml: {config_status}") fcheck = _check_filigree_auth(root, repair=True, filigree_url=probe_url) fstatus = ("fixed" if fcheck.fixed else fcheck.message) if fcheck.ok else f"failed ({fcheck.message})" click.echo(f" filigree.auth: {fstatus}") - if not all(check.ok for check in after) or not fcheck.ok: + if not all(check.ok for check in after) or not config_check.ok or not fcheck.ok: raise SystemExit(1) return checks = check_install(root) + config_check = _check_config(root, fixed=False) fcheck = _check_filigree_auth(root, repair=False, filigree_url=filigree_url) - ok = all(check.ok for check in checks) and fcheck.ok + ok = all(check.ok for check in checks) and config_check.ok and fcheck.ok click.echo("wardline doctor: ok" if ok else "wardline doctor:") for check in checks: if not check.ok: click.echo(f" {check.name}: {check.message}") + if not config_check.ok: + click.echo(f" weft.toml: {config_check.message}") fmsg = fcheck.message or ("ok" if fcheck.ok else "error") click.echo(f" filigree.auth: {fmsg}") if not ok: diff --git a/src/wardline/cli/main.py b/src/wardline/cli/main.py index 8502031f..d24423f3 100644 --- a/src/wardline/cli/main.py +++ b/src/wardline/cli/main.py @@ -25,6 +25,7 @@ from wardline.cli.rekey import rekey from wardline.cli.scan import scan from wardline.cli.scan_file_findings import scan_file_findings +from wardline.cli.scan_job import scan_job from wardline.core.baseline import collect_and_write_baseline from wardline.core.descriptor import descriptor_to_yaml from wardline.core.errors import WardlineError @@ -39,6 +40,7 @@ def cli() -> None: cli.add_command(scan) +cli.add_command(scan_job) cli.add_command(scan_file_findings) cli.add_command(rekey) cli.add_command(judge_command) diff --git a/src/wardline/cli/scan.py b/src/wardline/cli/scan.py index 346f6ea0..be6e1138 100644 --- a/src/wardline/cli/scan.py +++ b/src/wardline/cli/scan.py @@ -72,6 +72,25 @@ default=None, help="POST findings to this Filigree Weft scan-results URL (opt-in).", ) +@click.option( + "--local-only", + "--no-emit", + "local_only", + is_flag=True, + default=False, + help=( + "Disable sibling emission even when Filigree or Loomweave URLs resolve from flags, env, or local install state." + ), +) +@click.option( + "--filigree-max-findings-per-request", + type=click.IntRange(min=1), + default=None, + help=( + "Maximum Wardline findings per Filigree scan-results POST " + "(default 1000; also configurable with WARDLINE_FILIGREE_MAX_FINDINGS_PER_REQUEST)." + ), +) @click.option( "--loomweave-url", "loomweave_url", @@ -151,6 +170,8 @@ def scan( fail_on_unanalyzed: bool, cache_dir: Path | None, filigree_url: str | None, + local_only: bool, + filigree_max_findings_per_request: int | None, loomweave_url: str | None, new_since: str | None, trusted_packs: tuple[str, ...], @@ -195,8 +216,19 @@ def scan( emit_result: EmitResult | None = None loomweave_result = None try: + if config_path is None and not strict_defaults and not weft_config_path(path).is_file(): + click.echo( + "warning: no weft.toml found; using built-in source_roots=['.'], which can make " + "project-root scans broad and slow. Run `wardline doctor --repair --root " + f"{path}` to create a bounded default policy, or `wardline scan-job start {path}` " + "for a pollable long-running scan.", + err=True, + ) filigree_url = resolve_filigree_url(filigree_url, path, config_path, strict_defaults=strict_defaults) loomweave_url = resolve_loomweave_url(loomweave_url, path, config_path, strict_defaults=strict_defaults) + if local_only: + filigree_url = None + loomweave_url = None result = run_scan( path, config_path=config_path, @@ -291,14 +323,17 @@ def confirm_cb(rel_path: str, orig: str, replacement: str, f: Finding) -> bool: "(dirty: true, legis records it unverified). Commit for a signed artifact.", err=True, ) - # Weft emission is additive: a FiligreeEmitError (HTTP >= 400) is a Wardline - # payload bug -> caught below -> exit 2; an unreachable sibling warns + continues. + # Weft emission is additive: scan uses the emitter's fail-soft protocol mode so + # a Filigree reject is reported as upload failure, not as a pre-gate exit 2. if filigree_url is not None: from wardline.filigree.config import load_filigree_token - emit_result = FiligreeEmitter(filigree_url, token=load_filigree_token(path)).emit( - findings, scanned_paths=result.scanned_paths - ) + emit_result = FiligreeEmitter( + filigree_url, + token=load_filigree_token(path), + max_findings_per_request=filigree_max_findings_per_request, + protocol_errors_loud=False, + ).emit(findings, scanned_paths=result.scanned_paths) # Loomweave taint-store write is fail-soft: an outage/403 returns a not-reachable # WriteResult (reported below); a LoomweaveError (missing extra, 4xx, bad scheme) # is a WardlineError → caught here → exit 2, exactly as Filigree errors do. @@ -384,7 +419,7 @@ def confirm_cb(rel_path: str, orig: str, replacement: str, f: Finding) -> bool: where = ( f"project {dest_project!r}" if dest_project - else "server-default project (URL pins none — add ?project= to make it explicit)" + else "unscoped endpoint (URL pins no project; add ?project= to make routing explicit)" ) line = ( f"emitted {len(findings)} finding(s) to {filigree_url} [{where}] — " @@ -433,7 +468,7 @@ def confirm_cb(rel_path: str, orig: str, replacement: str, f: Finding) -> bool: if s.unanalyzed: click.echo( f"warning: {s.unanalyzed} file(s) were discovered but could not be analyzed " - f"(see WLN-ENGINE-* facts in {output}).", + f"(see WLN-ENGINE-* diagnostics in {output}).", err=True, ) if lang == "rust": diff --git a/src/wardline/cli/scan_job.py b/src/wardline/cli/scan_job.py new file mode 100644 index 00000000..f113b3e4 --- /dev/null +++ b/src/wardline/cli/scan_job.py @@ -0,0 +1,119 @@ +"""`wardline scan-job` — file-backed start/status surface for long scans.""" + +from __future__ import annotations + +import json +from pathlib import Path + +import click + +from wardline.core.errors import WardlineError +from wardline.core.scan_jobs import ( + DEFAULT_SCAN_JOB_TIMEOUT_SECONDS, + cancel_scan_job, + read_scan_job_status, + start_scan_job, +) + + +@click.group(name="scan-job") +def scan_job() -> None: + """Start and poll file-backed Wardline scan jobs.""" + + +@scan_job.command("start") +@click.argument("path", type=click.Path(exists=True, file_okay=False, path_type=Path), default=".") +@click.option("--config", "config_path", type=click.Path(exists=True, file_okay=True, dir_okay=False, path_type=Path)) +@click.option("--format", "fmt", type=click.Choice(["jsonl", "sarif", "agent-summary"]), default="jsonl") +@click.option("--output", type=click.Path(path_type=Path), default=None) +@click.option("--fail-on", type=click.Choice(["CRITICAL", "ERROR", "WARN", "INFO"], case_sensitive=False), default=None) +@click.option("--fail-on-unanalyzed/--no-fail-on-unanalyzed", default=False) +@click.option("--cache-dir", type=click.Path(path_type=Path), default=None) +@click.option("--filigree-url", "filigree_url", default=None) +@click.option("--local-only", "--no-emit", "local_only", is_flag=True, default=False) +@click.option("--filigree-max-findings-per-request", type=click.IntRange(min=1), default=None) +@click.option( + "--timeout", + "timeout_seconds", + type=click.FloatRange(min=0.0), + default=None, + help=f"Fail the job after SECONDS; defaults to {DEFAULT_SCAN_JOB_TIMEOUT_SECONDS}. Use 0 to disable.", +) +@click.option("--lang", type=click.Choice(["python", "rust"]), default="python") +@click.option("--new-since", type=str, default=None) +@click.option("--trust-pack", "trusted_packs", multiple=True) +@click.option("--allow-custom-packs", "trust_local_packs", is_flag=True, default=False) +@click.option("--strict-defaults", is_flag=True, default=False) +@click.option("--trust-suppressions", is_flag=True, default=False) +@click.option("--foreground", is_flag=True, hidden=True, default=False) +def start( + path: Path, + config_path: Path | None, + fmt: str, + output: Path | None, + fail_on: str | None, + fail_on_unanalyzed: bool, + cache_dir: Path | None, + filigree_url: str | None, + local_only: bool, + filigree_max_findings_per_request: int | None, + timeout_seconds: float | None, + lang: str, + new_since: str | None, + trusted_packs: tuple[str, ...], + trust_local_packs: bool, + strict_defaults: bool, + trust_suppressions: bool, + foreground: bool, +) -> None: + """Start a scan job and print its status JSON.""" + request = { + "config": str(config_path) if config_path else None, + "format": fmt, + "output": str(output) if output else None, + "fail_on": fail_on.upper() if fail_on else None, + "fail_on_unanalyzed": fail_on_unanalyzed, + "cache_dir": str(cache_dir) if cache_dir else None, + "filigree_url": filigree_url, + "local_only": local_only, + "filigree_max_findings_per_request": filigree_max_findings_per_request, + "timeout_seconds": timeout_seconds, + "lang": lang, + "new_since": new_since, + "trusted_packs": list(trusted_packs), + "trust_local_packs": trust_local_packs, + "strict_defaults": strict_defaults, + "trust_suppressions": trust_suppressions, + } + try: + status = start_scan_job(path, request, foreground=foreground) + except WardlineError as exc: + click.echo(f"error: {exc}", err=True) + raise SystemExit(2) from exc + click.echo(json.dumps(status, sort_keys=True)) + + +@scan_job.command("status") +@click.argument("job_id") +@click.option("--path", "path", type=click.Path(exists=True, file_okay=False, path_type=Path), default=".") +def status(job_id: str, path: Path) -> None: + """Print status JSON for a scan job.""" + try: + payload = read_scan_job_status(path, job_id) + except WardlineError as exc: + click.echo(f"error: {exc}", err=True) + raise SystemExit(2) from exc + click.echo(json.dumps(payload, sort_keys=True)) + + +@scan_job.command("cancel") +@click.argument("job_id") +@click.option("--path", "path", type=click.Path(exists=True, file_okay=False, path_type=Path), default=".") +def cancel(job_id: str, path: Path) -> None: + """Cancel a running scan job and print its terminal status JSON.""" + try: + payload = cancel_scan_job(path, job_id) + except WardlineError as exc: + click.echo(f"error: {exc}", err=True) + raise SystemExit(2) from exc + click.echo(json.dumps(payload, sort_keys=True)) diff --git a/src/wardline/cli/scan_job_worker.py b/src/wardline/cli/scan_job_worker.py new file mode 100644 index 00000000..1e7a42b6 --- /dev/null +++ b/src/wardline/cli/scan_job_worker.py @@ -0,0 +1,24 @@ +"""Subprocess entrypoint for file-backed scan jobs.""" + +from __future__ import annotations + +import sys +from pathlib import Path + +from wardline.core.errors import WardlineError +from wardline.core.scan_jobs import run_scan_job_worker + + +def main() -> None: + if len(sys.argv) != 3: + print("usage: python -m wardline.cli.scan_job_worker ROOT JOB_ID", file=sys.stderr) + raise SystemExit(2) + try: + run_scan_job_worker(Path(sys.argv[1]), sys.argv[2]) + except WardlineError as exc: + print(f"error: {exc}", file=sys.stderr) + raise SystemExit(2) from exc + + +if __name__ == "__main__": + main() diff --git a/src/wardline/core/discovery.py b/src/wardline/core/discovery.py index 03c91822..1bd3692c 100644 --- a/src/wardline/core/discovery.py +++ b/src/wardline/core/discovery.py @@ -4,6 +4,7 @@ from __future__ import annotations import fnmatch +import os import warnings from collections.abc import Iterable from pathlib import Path @@ -11,7 +12,21 @@ from wardline.core.config import WardlineConfig from wardline.core.errors import ConfigError -_ALWAYS_SKIP = frozenset({"__pycache__", ".venv", "venv", ".git", ".mypy_cache"}) +_ALWAYS_SKIP = frozenset( + { + "__pycache__", + ".venv", + "venv", + ".git", + ".mypy_cache", + ".uv-cache", + ".pytest_cache", + ".ruff_cache", + ".tox", + ".nox", + "node_modules", + } +) def discover( @@ -47,7 +62,13 @@ def discover( if not base.exists(): warnings.warn(f"source root does not exist: {base}", stacklevel=2) continue - candidates = sorted({path for suffix in suffixes for path in base.rglob(f"*{suffix}")}) + candidates: list[Path] = [] + for dirpath, dirnames, filenames in os.walk(base): + dirnames[:] = sorted(dirname for dirname in dirnames if dirname not in skip_dirs) + current = Path(dirpath) + for filename in sorted(filenames): + if any(filename.endswith(suffix) for suffix in suffixes): + candidates.append(current / filename) for path in candidates: rel_parts = path.relative_to(base).parts if path.is_relative_to(base) else path.parts if any(part in skip_dirs for part in rel_parts): diff --git a/src/wardline/core/filigree_emit.py b/src/wardline/core/filigree_emit.py index 6cbcb7ac..34b8abea 100644 --- a/src/wardline/core/filigree_emit.py +++ b/src/wardline/core/filigree_emit.py @@ -5,13 +5,15 @@ HTTP emitter (``FiligreeEmitter``). stdlib-only; no runtime dependency on Filigree. Federation discipline: a *sibling-absent* network failure warns and continues; a 5xx outage and a 401/403 (Filigree present but refusing bearer auth) are likewise treated -as *enrichment unavailable* (warn + continue). A 400 (Wardline built a bad payload) is -the one loud band — that is our own bug, not the sibling's posture. +as *enrichment unavailable* (warn + continue). Client/protocol errors default loud +for callers that need strict reconciliation, but `wardline scan` opts into fail-soft +enrichment so an upload reject cannot preempt the local gate verdict. """ from __future__ import annotations import json +import os import urllib.error import urllib.parse import urllib.request @@ -32,6 +34,8 @@ _SUGGESTION_LIMIT = 10000 _ALLOWED_SCHEMES = ("http", "https") +_DEFAULT_MAX_FINDINGS_PER_REQUEST = 1000 +_MAX_FINDINGS_ENV = "WARDLINE_FILIGREE_MAX_FINDINGS_PER_REQUEST" def _safe_int(value: Any) -> int: @@ -70,6 +74,7 @@ def build_scan_results_body( *, scan_source: str = "wardline", scanned_paths: Sequence[str] = (), + mark_unseen: bool | None = None, ) -> dict[str, Any]: """Build the ``POST /api/weft/scan-results`` request body. Emits ALL finding kinds. ``mark_unseen`` opts into Filigree's per-(file, scan_source) absent-fingerprint sweep: @@ -83,10 +88,12 @@ def build_scan_results_body( findings_wire = [_finding_to_wire(f) for f in findings] scanned = list(dict.fromkeys(p for p in scanned_paths if p)) has_unanalyzed = any(f.rule_id in UNANALYZED_RULE_IDS for f in findings) + if mark_unseen is None: + mark_unseen = bool(findings_wire or scanned) and not has_unanalyzed body = { "scan_source": scan_source, "fingerprint_scheme": FINGERPRINT_SCHEME, - "mark_unseen": bool(findings_wire or scanned) and not has_unanalyzed, + "mark_unseen": bool(mark_unseen) and not has_unanalyzed, "findings": findings_wire, } if scanned: @@ -206,6 +213,112 @@ class ProbeResult: status: int | None = None +@dataclass(frozen=True, slots=True) +class _ScanResultChunk: + findings: tuple[Finding, ...] + scanned_paths: tuple[str, ...] + mark_unseen: bool + + +def _scan_result_chunks( + findings: Sequence[Finding], + scanned_paths: Sequence[str], + max_findings_per_request: int, +) -> tuple[_ScanResultChunk, ...]: + """Split large emits without corrupting Filigree's per-file unseen sweep. + + Filigree's ``mark_unseen`` reconciliation is scoped to scanned file paths, so a + chunk must carry a complete set of findings for every path it names. Most large + scans can be chunked by whole-file groups. If one file alone exceeds the cap, the + file has to be split and reconciliation is disabled only for those chunks. + """ + if max_findings_per_request < 1: + raise ValueError("max_findings_per_request must be at least 1") + + deduped_scanned_paths = tuple(dict.fromkeys(p for p in scanned_paths if p)) + can_mark_unseen = not any(f.rule_id in UNANALYZED_RULE_IDS for f in findings) + if len(findings) <= max_findings_per_request: + return ( + _ScanResultChunk( + tuple(findings), + deduped_scanned_paths, + bool(findings or deduped_scanned_paths) and can_mark_unseen, + ), + ) + + by_path: dict[str, list[Finding]] = {} + path_order: list[str] = [] + for finding in findings: + path = finding.location.path or "" + if path not in by_path: + by_path[path] = [] + path_order.append(path) + by_path[path].append(finding) + + chunks: list[_ScanResultChunk] = [] + current_findings: list[Finding] = [] + current_paths: list[str] = [] + paths_with_findings = set(path_order) + clean_scanned_paths = [p for p in deduped_scanned_paths if p not in paths_with_findings] + + def flush_current() -> None: + nonlocal current_findings, current_paths + if current_findings or current_paths: + chunks.append( + _ScanResultChunk( + tuple(current_findings), + tuple(dict.fromkeys(current_paths)), + bool(current_findings or current_paths) and can_mark_unseen, + ) + ) + current_findings = [] + current_paths = [] + + for path in path_order: + group = by_path[path] + if len(group) > max_findings_per_request: + flush_current() + for start in range(0, len(group), max_findings_per_request): + # Complete-file reconciliation is impossible for this path under the cap. + chunks.append(_ScanResultChunk(tuple(group[start : start + max_findings_per_request]), (path,), False)) + continue + if current_findings and len(current_findings) + len(group) > max_findings_per_request: + flush_current() + current_findings.extend(group) + if path: + current_paths.append(path) + + current_paths.extend(clean_scanned_paths) + flush_current() + return tuple(chunks) + + +def _parse_success_response(resp: Response) -> EmitResult: + # 2xx success. Parse defensively: a 2xx with an unreadable body means the POST was + # accepted but the report is unparseable — surface a warning, never crash (charter). + warnings: list[str] = [] + try: + parsed = json.loads(resp.body) if resp.body else {} + except json.JSONDecodeError: + parsed = None + payload: dict[str, Any] = parsed if isinstance(parsed, dict) else {} + if not isinstance(parsed, dict): + warnings.append(f"Filigree returned {resp.status} with a non-JSON-object body; stats unavailable.") + raw_stats = payload.get("stats") + stats: dict[str, Any] = raw_stats if isinstance(raw_stats, dict) else {} + raw_warnings = payload.get("warnings") + if isinstance(raw_warnings, list): + warnings.extend(str(w) for w in raw_warnings) + failed = payload.get("failed") or [] + return EmitResult( + reachable=True, + created=_safe_int(stats.get("findings_created")), + updated=_safe_int(stats.get("findings_updated")), + failed=len(failed) if isinstance(failed, list) else 0, + warnings=tuple(warnings), + ) + + def filigree_disabled_reason( *, reachable: bool, status: int | None, token_sent: bool = False, url: str | None = None ) -> str | None: @@ -268,63 +381,186 @@ def post(self, url: str, body: bytes, headers: Mapping[str, str]) -> Response: with exc: return Response(status=exc.code, body=read_response_text(exc)) + def get(self, url: str, headers: Mapping[str, str]) -> Response: + scheme = urllib.parse.urlsplit(url).scheme.lower() + if scheme not in _ALLOWED_SCHEMES: + raise FiligreeEmitError(f"filigree URL must use http or https; got scheme {scheme!r} in {url!r}") + request = urllib.request.Request(url, headers=dict(headers), method="GET") + try: + with urllib.request.urlopen(request, timeout=self._timeout) as resp: # noqa: S310 + return Response(status=resp.status, body=read_response_text(resp)) + except urllib.error.HTTPError as exc: + with exc: + return Response(status=exc.code, body=read_response_text(exc)) + + +def _resolve_operator_max_findings_per_request(explicit: int | None) -> int | None: + if explicit is not None: + if explicit < 1: + raise ValueError("max_findings_per_request must be at least 1") + return explicit + raw = os.environ.get(_MAX_FINDINGS_ENV) + if raw: + try: + value = int(raw) + except ValueError as exc: + raise ValueError(f"{_MAX_FINDINGS_ENV} must be an integer") from exc + if value < 1: + raise ValueError(f"{_MAX_FINDINGS_ENV} must be at least 1") + return value + return None + + +def _limit_from_mapping(node: Mapping[str, Any]) -> int | None: + for key in ( + "max_findings_per_request", + "max_findings", + "findings_per_request", + "findings_per_request_limit", + "finding_limit", + "findings_limit", + ): + value = node.get(key) + if isinstance(value, int) and not isinstance(value, bool) and value > 0: + return value + for nested_key in ("limits", "request_limits", "chunking_guidance", "scan_results"): + nested = node.get(nested_key) + if isinstance(nested, Mapping): + found = _limit_from_mapping(nested) + if found is not None: + return found + return None + + +def _is_scan_results_node(key: str, node: Mapping[str, Any]) -> bool: + haystacks = [key] + for field in ("path", "url", "endpoint", "route", "name"): + value = node.get(field) + if isinstance(value, str): + haystacks.append(value) + return any("scan-results" in item or "scan_results" in item for item in haystacks) + + +def _extract_scan_results_limit(schema: Mapping[str, Any]) -> int | None: + for key in ("scan_results", "scan-results", "scan_results_limits", "scan-results-limits"): + value = schema.get(key) + if isinstance(value, Mapping): + found = _limit_from_mapping(value) + if found is not None: + return found + + endpoints = schema.get("endpoints") + if isinstance(endpoints, Mapping): + for key, value in endpoints.items(): + if isinstance(value, Mapping) and _is_scan_results_node(str(key), value): + found = _limit_from_mapping(value) + if found is not None: + return found + elif isinstance(endpoints, list): + for value in endpoints: + if isinstance(value, Mapping) and _is_scan_results_node("", value): + found = _limit_from_mapping(value) + if found is not None: + return found + return None + + +def _scan_results_schema_url(url: str) -> str: + return f"{filigree_api_base_url(url).rstrip('/')}/files/_schema" + + +def _fetch_scan_results_limit(url: str, transport: Transport, headers: Mapping[str, str]) -> int | None: + get = getattr(transport, "get", None) + if get is None: + return None + try: + resp = get(_scan_results_schema_url(url), headers) + except (FiligreeEmitError, urllib.error.URLError, OSError, json.JSONDecodeError): + return None + if not 200 <= resp.status < 300: + return None + try: + parsed = json.loads(resp.body) if resp.body else {} + except json.JSONDecodeError: + return None + return _extract_scan_results_limit(parsed) if isinstance(parsed, Mapping) else None + class FiligreeEmitter: """POST findings to a Filigree Weft scan-results URL with an injectable transport.""" - def __init__(self, url: str, *, transport: Transport | None = None, token: str | None = None) -> None: + def __init__( + self, + url: str, + *, + transport: Transport | None = None, + token: str | None = None, + max_findings_per_request: int | None = None, + protocol_errors_loud: bool = True, + ) -> None: self._url = url self._transport: Transport = transport if transport is not None else UrllibTransport() self._token = token + self._operator_max_findings_per_request = _resolve_operator_max_findings_per_request(max_findings_per_request) + self._protocol_errors_loud = protocol_errors_loud def emit(self, findings: Sequence[Finding], *, scanned_paths: Sequence[str] = ()) -> EmitResult: - body = json.dumps(build_scan_results_body(findings, scanned_paths=scanned_paths)).encode("utf-8") headers = {"Content-Type": "application/json"} token_sent = bool(self._token) if token_sent: headers["Authorization"] = f"Bearer {self._token}" + max_findings_per_request = ( + self._operator_max_findings_per_request + or _fetch_scan_results_limit(self._url, self._transport, headers) + or _DEFAULT_MAX_FINDINGS_PER_REQUEST + ) + chunks = list(_scan_result_chunks(findings, scanned_paths, max_findings_per_request)) + created = 0 + updated = 0 + failed = 0 + warnings: list[str] = [] try: - resp = self._transport.post(self._url, body, headers) + for chunk_index, chunk in enumerate(chunks, start=1): + body = json.dumps( + build_scan_results_body( + chunk.findings, + scanned_paths=chunk.scanned_paths, + mark_unseen=chunk.mark_unseen, + ) + ).encode("utf-8") + resp = self._transport.post(self._url, body, headers) + if resp.status in (401, 403): + # Filigree is present but its opt-in bearer auth is on and refusing us. + # Stays SOFT (enrichment unavailable, never exit-2) — but distinguished + # as auth so the caller can say the actionable thing. + return EmitResult(reachable=False, status=resp.status, token_sent=token_sent, url=self._url) + if resp.status >= 500: + # Server-side outage (5xx) — the sibling is degraded, not a Wardline + # payload bug. Treat like absent (warn + continue), carrying the status. + return EmitResult(reachable=False, status=resp.status, token_sent=token_sent, url=self._url) + if not 200 <= resp.status < 300: + message = f"Filigree rejected scan-results ({resp.status}) at {self._url}: {resp.body}" + if self._protocol_errors_loud: + raise FiligreeEmitError(message) + remaining_findings = sum(len(c.findings) for c in chunks[chunk_index - 1 :]) + failed += remaining_findings + warnings.append(message) + break + chunk_result = _parse_success_response(resp) + created += chunk_result.created + updated += chunk_result.updated + failed += chunk_result.failed + warnings.extend(chunk_result.warnings) except (urllib.error.URLError, OSError): # Connection refused / DNS / timeout — sibling absent. Enrichment is # non-load-bearing: warn (at the CLI) and continue. No status reached us, so # this is the genuine "could not reach" case (status=None). return EmitResult(reachable=False, token_sent=token_sent, url=self._url) - if resp.status in (401, 403): - # Filigree is present but its opt-in bearer auth is on and refusing us. Stays - # SOFT (enrichment unavailable, never exit-2) — but distinguished as auth (and by - # token_sent: no-token vs token-rejected) so the caller can say the actionable - # thing instead of "could not reach". - return EmitResult(reachable=False, status=resp.status, token_sent=token_sent, url=self._url) - if resp.status >= 500: - # Server-side outage (5xx) — the sibling is degraded, not a Wardline payload bug. - # Treat like absent (warn + continue), carrying the status for an honest message. - return EmitResult(reachable=False, status=resp.status, token_sent=token_sent, url=self._url) - if not 200 <= resp.status < 300: - # 3xx (a redirect reached the client) or any remaining 4xx (notably 400): Wardline - # sent a request the server would not accept — bad payload / wrong endpoint. Loud. - raise FiligreeEmitError(f"Filigree rejected scan-results ({resp.status}) at {self._url}: {resp.body}") - # 2xx success. Parse defensively: a 2xx with an unreadable body means the POST was - # accepted but the report is unparseable — surface a warning, never crash (charter). - warnings: list[str] = [] - try: - parsed = json.loads(resp.body) if resp.body else {} - except json.JSONDecodeError: - parsed = None - payload: dict[str, Any] = parsed if isinstance(parsed, dict) else {} - if not isinstance(parsed, dict): - warnings.append(f"Filigree returned {resp.status} with a non-JSON-object body; stats unavailable.") - raw_stats = payload.get("stats") - stats: dict[str, Any] = raw_stats if isinstance(raw_stats, dict) else {} - raw_warnings = payload.get("warnings") - if isinstance(raw_warnings, list): - warnings.extend(str(w) for w in raw_warnings) - failed = payload.get("failed") or [] return EmitResult( reachable=True, - created=_safe_int(stats.get("findings_created")), - updated=_safe_int(stats.get("findings_updated")), - failed=len(failed) if isinstance(failed, list) else 0, + created=created, + updated=updated, + failed=failed, warnings=tuple(warnings), token_sent=token_sent, url=self._url, diff --git a/src/wardline/core/finding.py b/src/wardline/core/finding.py index 6bb76a62..a57664de 100644 --- a/src/wardline/core/finding.py +++ b/src/wardline/core/finding.py @@ -260,12 +260,22 @@ def require_fingerprint_scheme(document: Mapping[str, Any], *, store_name: str) Severity.NONE: "info", } +_PROPERTY_ACCESSOR_QUALNAME_SUFFIXES = (":setter", ":deleter") + def severity_to_filigree(severity: Severity) -> str: """Map Wardline's 4-level (+NONE) vocabulary to Filigree's 5-level set.""" return _SEVERITY_TO_FILIGREE[severity] +def _to_wire_qualname(qualname: str) -> str: + """Return the cross-tool reconciliation qualname for Wardline metadata.""" + for suffix in _PROPERTY_ACCESSOR_QUALNAME_SUFFIXES: + if qualname.endswith(suffix): + return qualname.removesuffix(suffix) + return qualname + + def to_filigree_metadata(finding: Finding) -> dict[str, Any]: """Build the ``metadata.wardline.*`` subtree (semantic JSON, not byte-stable).""" wardline: dict[str, Any] = { @@ -274,7 +284,7 @@ def to_filigree_metadata(finding: Finding) -> dict[str, Any]: "kind": finding.kind.value, } if finding.qualname is not None: - wardline["qualname"] = finding.qualname + wardline["qualname"] = _to_wire_qualname(finding.qualname) if finding.confidence is not None: wardline["confidence"] = finding.confidence if finding.related_entities: diff --git a/src/wardline/core/run.py b/src/wardline/core/run.py index 89a39147..9f618ad8 100644 --- a/src/wardline/core/run.py +++ b/src/wardline/core/run.py @@ -9,10 +9,11 @@ from __future__ import annotations import hashlib +from collections.abc import Callable from dataclasses import dataclass, replace from datetime import date from pathlib import Path -from typing import TYPE_CHECKING +from typing import TYPE_CHECKING, Any from wardline.core import config as config_mod from wardline.core.baseline import Baseline, load_baseline @@ -171,6 +172,7 @@ def run_scan( trust_suppressions: bool = False, skip_suppression: bool = False, lang: str = "python", + progress_callback: Callable[[dict[str, Any]], None] | None = None, ) -> ScanResult: """Discover → analyze → apply suppressions. Pure function of (disk + config). @@ -231,6 +233,8 @@ def run_scan( warnings.simplefilter("always") files = discover(root, cfg, confine_to_root=confine_to_root, suffixes=suffixes) captured_warnings = list(w) + if progress_callback is not None: + progress_callback({"phase": "discovered", "files_discovered": len(files)}) for warn in captured_warnings: msg = str(warn.message) if not msg.startswith("WLN-ENGINE-FILE-SKIPPED: "): @@ -241,7 +245,18 @@ def run_scan( warn.lineno, ) analyzer: Analyzer = frontend.build_analyzer(config=cfg, summary_cache=cache) + if progress_callback is not None: + progress_callback({"phase": "analyzing", "files_discovered": len(files)}) raw = list(analyzer.analyze(files, cfg, root=root)) + if progress_callback is not None: + progress_callback( + { + "phase": "analyzed", + "files_discovered": len(files), + "files_analyzed": len(files), + "findings": len(raw), + } + ) for warn in captured_warnings: msg = str(warn.message) if msg.startswith("WLN-ENGINE-FILE-SKIPPED: "): diff --git a/src/wardline/core/scan_jobs.py b/src/wardline/core/scan_jobs.py new file mode 100644 index 00000000..846815ce --- /dev/null +++ b/src/wardline/core/scan_jobs.py @@ -0,0 +1,497 @@ +"""File-backed Wardline scan jobs. + +The job surface is intentionally local and daemon-free: ``start`` writes a stable +handle under ``.weft/wardline/jobs/`` and a worker process updates status JSON as it +moves through scan, artifact write, optional enrichment, and gate evaluation. +""" + +from __future__ import annotations + +import json +import os +import re +import signal +import subprocess +import sys +import threading +import uuid +from datetime import UTC, datetime +from pathlib import Path +from typing import Any + +from wardline.core.agent_summary import build_agent_summary +from wardline.core.emit import JsonlSink +from wardline.core.errors import WardlineError +from wardline.core.filigree_emit import ( + EmitResult, + FiligreeEmitter, + filigree_destination, + filigree_disabled_reason, +) +from wardline.core.finding import Severity +from wardline.core.run import baseline_migration_hint, gate_decision, run_scan +from wardline.core.safe_paths import safe_project_path, safe_write_text +from wardline.core.sarif import SarifSink + +_JOB_ID_RE = re.compile(r"^[0-9a-f]{32}$") +_JOB_STEPS_TOTAL = 4 +_HEARTBEAT_INTERVAL_SECONDS = 5.0 +_STALE_AFTER_SECONDS = 30.0 +DEFAULT_SCAN_JOB_TIMEOUT_SECONDS = 30 * 60 +_TERMINAL_STATUSES = {"completed", "completed_with_enrichment_failure", "failed", "cancelled"} + + +class _ScanJobTimeout(WardlineError): + """Internal terminal error for scan-job timeouts.""" + + +def _now() -> str: + return datetime.now(UTC).isoformat().replace("+00:00", "Z") + + +def jobs_dir(root: Path) -> Path: + return safe_project_path(root, root / ".weft" / "wardline" / "jobs", label="wardline jobs") + + +def job_dir(root: Path, job_id: str) -> Path: + if not _JOB_ID_RE.fullmatch(job_id): + raise WardlineError(f"invalid scan job id: {job_id!r}") + return safe_project_path(root, jobs_dir(root) / job_id, label="wardline scan job") + + +def status_path(root: Path, job_id: str) -> Path: + return job_dir(root, job_id) / "status.json" + + +def request_path(root: Path, job_id: str) -> Path: + return job_dir(root, job_id) / "request.json" + + +def read_scan_job_status(root: Path, job_id: str) -> dict[str, Any]: + root = root.resolve() + path = status_path(root, job_id) + try: + parsed = json.loads(path.read_text(encoding="utf-8")) + except FileNotFoundError as exc: + raise WardlineError(f"scan job {job_id} not found under {root}") from exc + if not isinstance(parsed, dict): + raise WardlineError(f"scan job {job_id} status is malformed") + return _refresh_liveness(root, job_id, parsed) + + +def cancel_scan_job(root: Path, job_id: str) -> dict[str, Any]: + """Cancel a non-terminal scan job and persist the terminal status.""" + root = root.resolve() + status = read_scan_job_status(root, job_id) + if str(status.get("status")) in _TERMINAL_STATUSES: + return status + pid = _status_pid(status) + if pid is not None and _pid_alive(pid): + try: + os.killpg(pid, signal.SIGTERM) + except ProcessLookupError: + pass + except PermissionError as exc: + raise WardlineError(f"scan job {job_id} worker could not be cancelled: permission denied") from exc + status.update( + { + "status": "cancelled", + "phase": "cancelled", + "failure_kind": "cancelled", + "error": "cancelled by operator", + } + ) + return _write_status(root, job_id, status) + + +def _write_status(root: Path, job_id: str, status: dict[str, Any]) -> dict[str, Any]: + timestamp = _now() + status.setdefault("created_at", timestamp) + status["updated_at"] = timestamp + status["heartbeat"] = timestamp + safe_write_text(root, status_path(root, job_id), json.dumps(status, indent=2, sort_keys=True) + "\n") + return status + + +def _parse_timestamp(value: object) -> datetime | None: + if not isinstance(value, str) or not value: + return None + try: + return datetime.fromisoformat(value.replace("Z", "+00:00")) + except ValueError: + return None + + +def _status_pid(status: dict[str, Any]) -> int | None: + pid = status.get("pid") + if isinstance(pid, int) and pid > 0: + return pid + return None + + +def _pid_alive(pid: int) -> bool: + try: + os.kill(pid, 0) + except ProcessLookupError: + return False + except PermissionError: + return True + return True + + +def _refresh_liveness(root: Path, job_id: str, status: dict[str, Any]) -> dict[str, Any]: + if str(status.get("status")) in _TERMINAL_STATUSES: + return status + pid = _status_pid(status) + if pid is not None and not _pid_alive(pid): + status.update( + { + "status": "failed", + "phase": "failed", + "failure_kind": "stale_worker", + "error": f"scan job worker pid {pid} is no longer running", + } + ) + return _write_status(root, job_id, status) + heartbeat = _parse_timestamp(status.get("heartbeat")) + if str(status.get("status")) in {"running", "running_stale"} and heartbeat is not None: + stale_for = (datetime.now(UTC) - heartbeat).total_seconds() + if stale_for > _STALE_AFTER_SECONDS: + status["status"] = "running_stale" + status["stale_for_seconds"] = int(stale_for) + status.setdefault("warning", "scan job heartbeat is stale; worker may still be running") + return status + status.pop("stale_for_seconds", None) + if status.get("status") == "running_stale": + status["status"] = "running" + return status + + +def _start_heartbeat( + root: Path, + job_id: str, + status: dict[str, Any], + lock: threading.Lock, + stop: threading.Event, +) -> threading.Thread: + def beat() -> None: + while not stop.wait(_HEARTBEAT_INTERVAL_SECONDS): + with lock: + if str(status.get("status")) in _TERMINAL_STATUSES: + return + progress = dict(status.get("progress") if isinstance(status.get("progress"), dict) else {}) + progress.setdefault("message", "scan still running") + status["progress"] = progress + try: + _write_status(root, job_id, status) + except OSError: + return + + thread = threading.Thread(target=beat, name=f"wardline-scan-job-{job_id[:8]}-heartbeat", daemon=True) + thread.start() + return thread + + +def _progress(step: int, **extra: object) -> dict[str, object]: + data: dict[str, object] = {"steps_completed": step, "steps_total": _JOB_STEPS_TOTAL} + data.update(extra) + return data + + +def _base_status(job_id: str, request: dict[str, Any]) -> dict[str, Any]: + timestamp = _now() + return { + "job_id": job_id, + "status": "queued", + "phase": "queued", + "progress": _progress(0), + "created_at": timestamp, + "updated_at": timestamp, + "heartbeat": timestamp, + "request": request, + "artifacts": {}, + "failure_kind": None, + "error": None, + } + + +def _normalize_request(request: dict[str, Any]) -> dict[str, Any]: + normalized = dict(request) + timeout_seconds = normalized.get("timeout_seconds") + if timeout_seconds is None: + normalized["timeout_seconds"] = DEFAULT_SCAN_JOB_TIMEOUT_SECONDS + return normalized + + +def _filigree_status(result: EmitResult | None) -> dict[str, object]: + if result is None: + return { + "configured": False, + "reachable": None, + "created": 0, + "updated": 0, + "failed": 0, + "warnings": [], + "disabled_reason": "not configured", + "destination": filigree_destination(None), + } + return { + "configured": True, + "reachable": result.reachable, + "created": result.created, + "updated": result.updated, + "failed": result.failed, + "warnings": list(result.warnings), + "disabled_reason": filigree_disabled_reason( + reachable=result.reachable, + status=result.status, + token_sent=result.token_sent, + url=result.url, + ), + "destination": filigree_destination(result.url), + } + + +def _write_scan_artifact(root: Path, output: Path, fmt: str, result: Any, fail_on: str | None) -> None: + sink_root = root if output.is_relative_to(root.resolve()) else None + if fmt == "sarif": + SarifSink(output, root=sink_root).write(result.findings, result.context) + return + if fmt == "agent-summary": + decision = gate_decision(result, Severity(fail_on)) if fail_on is not None else gate_decision(result, None) + output.parent.mkdir(parents=True, exist_ok=True) + output.write_text(json.dumps(build_agent_summary(result, decision).to_dict(), sort_keys=True) + "\n") + return + JsonlSink(output, root=sink_root).write(result.findings) + + +def start_scan_job(root: Path, request: dict[str, Any], *, foreground: bool = False) -> dict[str, Any]: + root = root.resolve() + request = _normalize_request(request) + job_id = uuid.uuid4().hex + directory = job_dir(root, job_id) + directory.mkdir(parents=True, exist_ok=True) + status = _base_status(job_id, request) + _write_status(root, job_id, status) + safe_write_text(root, request_path(root, job_id), json.dumps(request, indent=2, sort_keys=True) + "\n") + if foreground: + run_scan_job_worker(root, job_id) + return read_scan_job_status(root, job_id) + + stdout_path = directory / "stdout.log" + stderr_path = directory / "stderr.log" + with stdout_path.open("ab") as stdout, stderr_path.open("ab") as stderr: + proc = subprocess.Popen( # noqa: S603 + [sys.executable, "-m", "wardline.cli.scan_job_worker", str(root), job_id], + cwd=root, + stdout=stdout, + stderr=stderr, + start_new_session=True, + ) + status["status"] = "running" + status["phase"] = "starting" + status["pid"] = proc.pid + status["artifacts"] = {"stdout": str(stdout_path), "stderr": str(stderr_path)} + return _write_status(root, job_id, status) + + +def run_scan_job_worker(root: Path, job_id: str) -> None: + root = root.resolve() + request_file = request_path(root, job_id) + request = json.loads(request_file.read_text(encoding="utf-8")) + if not isinstance(request, dict): + raise WardlineError(f"scan job {job_id} request is malformed") + status = read_scan_job_status(root, job_id) + status.setdefault("pid", os.getpid()) + default_output = job_dir(root, job_id) / _default_output_name(str(request.get("format", "jsonl"))) + output = Path(str(request.get("output") or default_output)) + if not output.is_absolute(): + output = root / output + artifacts = dict(status.get("artifacts") if isinstance(status.get("artifacts"), dict) else {}) + artifacts["findings"] = str(output) + lock = threading.Lock() + heartbeat_stop = threading.Event() + heartbeat_thread: threading.Thread | None = None + previous_alarm: tuple[int, Any] | None = None + + def progress_update(event: dict[str, Any]) -> None: + with lock: + phase = event.get("phase") + if isinstance(phase, str): + status["phase"] = phase + progress = dict(status.get("progress") if isinstance(status.get("progress"), dict) else {}) + progress.update(event) + progress.setdefault("steps_completed", 1) + progress.setdefault("steps_total", _JOB_STEPS_TOTAL) + status["progress"] = progress + _write_status(root, job_id, status) + + def timeout_handler(signum: int, frame: object) -> None: + timeout_seconds = request.get("timeout_seconds") + raise _ScanJobTimeout(f"scan job timed out after {timeout_seconds} seconds") + + try: + status.update({"status": "running", "phase": "scanning", "progress": _progress(1), "artifacts": artifacts}) + with lock: + _write_status(root, job_id, status) + heartbeat_thread = _start_heartbeat(root, job_id, status, lock, heartbeat_stop) + timeout_seconds = request.get("timeout_seconds") + if isinstance(timeout_seconds, int | float) and timeout_seconds > 0: + previous_alarm = (signal.SIGALRM, signal.getsignal(signal.SIGALRM)) + signal.signal(signal.SIGALRM, timeout_handler) + signal.setitimer(signal.ITIMER_REAL, float(timeout_seconds)) + result = run_scan( + root, + config_path=Path(str(request["config"])) if request.get("config") else None, + cache_dir=Path(str(request["cache_dir"])) if request.get("cache_dir") else None, + new_since=str(request["new_since"]) if request.get("new_since") else None, + trust_local_packs=bool(request.get("trust_local_packs", False)), + trusted_packs=tuple(str(p) for p in request.get("trusted_packs", ())), + strict_defaults=bool(request.get("strict_defaults", False)), + trust_suppressions=bool(request.get("trust_suppressions", False)), + lang=str(request.get("lang", "python")), + progress_callback=progress_update, + ) + if previous_alarm is not None: + signal.setitimer(signal.ITIMER_REAL, 0.0) + signal.signal(previous_alarm[0], previous_alarm[1]) + previous_alarm = None + heartbeat_stop.set() + if heartbeat_thread is not None: + heartbeat_thread.join(timeout=1.0) + status.update( + { + "phase": "writing_artifact", + "progress": _progress(2, files_scanned=result.files_scanned, findings=result.summary.total), + } + ) + with lock: + _write_status(root, job_id, status) + fmt = str(request.get("format", "jsonl")) + _write_scan_artifact(root, output, fmt, result, str(request["fail_on"]) if request.get("fail_on") else None) + + emit_result: EmitResult | None = None + if request.get("filigree_url") and not request.get("local_only"): + from wardline.filigree.config import load_filigree_token + + status.update({"phase": "emitting_filigree", "progress": _progress(3)}) + with lock: + _write_status(root, job_id, status) + explicit_cap = request.get("filigree_max_findings_per_request") + max_findings = int(explicit_cap) if explicit_cap is not None else None + emit_result = FiligreeEmitter( + str(request["filigree_url"]), + token=load_filigree_token(root), + max_findings_per_request=max_findings, + protocol_errors_loud=False, + ).emit(result.findings, scanned_paths=result.scanned_paths) + + fail_on = str(request["fail_on"]) if request.get("fail_on") else None + decision = gate_decision( + result, + Severity(fail_on) if fail_on is not None else None, + fail_on_unanalyzed=bool(request.get("fail_on_unanalyzed", False)), + ) + filigree_block = _filigree_status(emit_result) + enrichment_failed = emit_result is not None and (not emit_result.reachable or emit_result.failed > 0) + terminal = "completed" + failure_kind = None + error = None + if decision.tripped: + terminal = "failed" + failure_kind = "gate" + error = decision.reason + elif enrichment_failed: + terminal = "completed_with_enrichment_failure" + failure_kind = "enrichment" + error = filigree_block["disabled_reason"] or f"{emit_result.failed} Filigree finding(s) failed" + status.update( + { + "status": terminal, + "phase": "complete", + "progress": _progress(4, files_scanned=result.files_scanned, findings=result.summary.total), + "failure_kind": failure_kind, + "error": error, + "artifacts": artifacts, + "summary": { + "total": result.summary.total, + "active": result.summary.active, + "baselined": result.summary.baselined, + "waived": result.summary.waived, + "judged": result.summary.judged, + "informational": result.summary.informational, + "unanalyzed": result.summary.unanalyzed, + }, + "gate": { + "tripped": decision.tripped, + "fail_on": decision.fail_on, + "fail_on_unanalyzed": decision.fail_on_unanalyzed, + "exit_class": decision.exit_class, + "verdict": decision.verdict, + "reason": decision.reason, + "evaluated": decision.evaluated, + "migration_hint": baseline_migration_hint( + result, + decision, + root=root, + new_since=request.get("new_since"), + ), + }, + "filigree_emit": filigree_block, + } + ) + with lock: + _write_status(root, job_id, status) + except _ScanJobTimeout as exc: + heartbeat_stop.set() + if previous_alarm is not None: + signal.setitimer(signal.ITIMER_REAL, 0.0) + signal.signal(previous_alarm[0], previous_alarm[1]) + previous_alarm = None + if heartbeat_thread is not None: + heartbeat_thread.join(timeout=1.0) + status.update( + { + "status": "failed", + "phase": "failed", + "progress": _progress(4), + "failure_kind": "timeout", + "error": str(exc), + "artifacts": artifacts, + } + ) + with lock: + _write_status(root, job_id, status) + except WardlineError as exc: + heartbeat_stop.set() + if previous_alarm is not None: + signal.setitimer(signal.ITIMER_REAL, 0.0) + signal.signal(previous_alarm[0], previous_alarm[1]) + previous_alarm = None + if heartbeat_thread is not None: + heartbeat_thread.join(timeout=1.0) + status.update( + { + "status": "failed", + "phase": "failed", + "progress": _progress(4), + "failure_kind": "scan", + "error": str(exc), + "artifacts": artifacts, + } + ) + with lock: + _write_status(root, job_id, status) + finally: + heartbeat_stop.set() + if previous_alarm is not None: + signal.setitimer(signal.ITIMER_REAL, 0.0) + signal.signal(previous_alarm[0], previous_alarm[1]) + + +def _default_output_name(fmt: str) -> str: + if fmt == "sarif": + return "findings.sarif" + if fmt == "agent-summary": + return "findings.agent-summary.json" + return "findings.jsonl" diff --git a/src/wardline/install/block.py b/src/wardline/install/block.py index 7d117ef2..293ac8e1 100644 --- a/src/wardline/install/block.py +++ b/src/wardline/install/block.py @@ -39,6 +39,7 @@ _OWN_NS = "wardline" _END_MARKER = f"" +_WRITER_MARKER = f"" # A complete, well-formed wardline block (open .. close). Own-namespace only, so # it can only ever match wardline's own spans (C-4 (b) own-namespace mutation). @@ -65,7 +66,7 @@ def _body_hash() -> str: def render_block() -> str: - return f"\n{_BODY}\n{_END_MARKER}" + return f"\n{_WRITER_MARKER}\n{_BODY}\n{_END_MARKER}" def _first_real_foreign_block_pos(content: str, search_from: int) -> int: diff --git a/src/wardline/install/doctor.py b/src/wardline/install/doctor.py index 6861a3d0..074ae71c 100644 --- a/src/wardline/install/doctor.py +++ b/src/wardline/install/doctor.py @@ -16,6 +16,7 @@ from wardline.core.errors import ConfigError from wardline.core.filigree_emit import FiligreeEmitter, Transport, UrllibTransport from wardline.core.paths import weft_config_path, weft_state_dir +from wardline.core.safe_paths import safe_write_text from wardline.filigree.config import load_filigree_token from wardline.install.block import inject_block from wardline.install.detect import ( @@ -59,6 +60,50 @@ def to_dict(self) -> dict[str, Any]: return data +_DEFAULT_CONFIG_EXCLUDES = ( + ".git/**", + ".venv/**", + "venv/**", + ".uv-cache/**", + ".mypy_cache/**", + ".pytest_cache/**", + ".ruff_cache/**", + ".tox/**", + ".nox/**", + "node_modules/**", + "telemetry/**", + "data/**", +) + + +def _format_toml_array(values: tuple[str, ...]) -> str: + return "[" + ", ".join(json.dumps(value) for value in values) + "]" + + +def _default_source_roots(root: Path) -> tuple[str, ...]: + return ("src",) if (root / "src").is_dir() else (".",) + + +def _default_weft_config(root: Path) -> str: + source_roots = _default_source_roots(root) + return ( + "# Created by `wardline doctor --repair`.\n" + "# Keep the scan rooted at the project root for stable identity; bound the\n" + "# analyzed source here so agent gates do not traverse caches or run artifacts.\n" + "[wardline]\n" + f"source_roots = {_format_toml_array(source_roots)}\n" + f"exclude = {_format_toml_array(_DEFAULT_CONFIG_EXCLUDES)}\n" + ) + + +def _ensure_weft_config(root: Path) -> bool: + cfg_path = weft_config_path(root) + if cfg_path.exists(): + return False + safe_write_text(root, cfg_path, _default_weft_config(root), label="weft.toml") + return True + + def _has_instruction_block(path: Path) -> bool: if not path.is_file(): return False @@ -124,6 +169,13 @@ def _check_config(root: Path, *, fixed: bool) -> DoctorCheck: # weft.toml (a sibling's section may be broken). doctor restores the operator # signal by distinguishing ABSENT (ok — defaults are intentional) from # PRESENT-BUT-BROKEN (error — your policy is silently not applying). + if not cfg_path.exists(): + return DoctorCheck( + "wardline.config", + "error", + fixed=False, + message="missing weft.toml; run `wardline doctor --repair` to create a bounded default policy", + ) if cfg_path.is_file(): try: parsed = tomllib.loads(cfg_path.read_text(encoding="utf-8")) @@ -401,6 +453,7 @@ def machine_readable_doctor( ) -> dict[str, Any]: """Return the shared machine-readable doctor shape, optionally repairing install bindings.""" before = {check.name: check for check in check_install(root)} + config_missing_before = not weft_config_path(root).exists() bindings_fixed = False if fix: repair_install(root) @@ -413,7 +466,7 @@ def machine_readable_doctor( probe_url = _resolve_probe_url(root, filigree_url) checks: list[DoctorCheck] = [] - checks.append(_check_config(root, fixed=fix and not weft_config_path(root).exists())) + checks.append(_check_config(root, fixed=fix and config_missing_before and weft_config_path(root).exists())) checks.append(_check_mcp_registration(root, before=before)) checks.append(_check_marker_package()) checks.append(_check_url(root, "loomweave", fixed=bindings_fixed, effective_url=loomweave_url)) @@ -467,7 +520,8 @@ def repair_install(root: Path) -> dict[str, str]: statuses["Codex MCP"] = "repaired" detect_siblings(root) statuses["bindings"] = "detected" - # doctor MAY create its OWN state subtree (never weft.toml, never a sibling's). + statuses["weft.toml"] = "created" if _ensure_weft_config(root) else "checked" + # doctor MAY create its OWN state subtree (never a sibling's). weft_state_dir(root).mkdir(parents=True, exist_ok=True) statuses["state_dir"] = "ensured" return statuses diff --git a/src/wardline/mcp/server.py b/src/wardline/mcp/server.py index e0a7711a..11604d9a 100644 --- a/src/wardline/mcp/server.py +++ b/src/wardline/mcp/server.py @@ -29,6 +29,7 @@ from wardline.core.paths import baseline_path as baseline_file from wardline.core.paths import waivers_path, weft_config_path from wardline.core.run import baseline_migration_hint, gate_decision, run_scan +from wardline.core.scan_jobs import cancel_scan_job, read_scan_job_status, start_scan_job from wardline.core.sei_resolution import resolve_query_filters from wardline.core.waivers import add_waiver, load_project_waivers from wardline.mcp.prompts import get_prompt, list_prompts @@ -622,6 +623,10 @@ def _cache_dir_arg(args: dict[str, Any], root: Path) -> Path | None: return _resolve_under_root(root, args["cache_dir"]) if args.get("cache_dir") else None +def _path_arg(args: dict[str, Any], root: Path) -> Path: + return _resolve_under_root(root, args["path"]) if args.get("path") else root + + def _bool_arg(args: dict[str, Any], name: str, default: bool) -> bool: # Reject non-bool values loudly rather than ``bool(...)``-coercing them: a JSON string # like "false" would otherwise coerce to True, silently inverting intent. Matches the @@ -634,6 +639,60 @@ def _bool_arg(args: dict[str, Any], name: str, default: bool) -> bool: return val +def _scan_job_request(args: dict[str, Any], root: Path, filigree_url: str | None) -> dict[str, Any]: + fail_on = args.get("fail_on") + if fail_on is not None: + try: + Severity(str(fail_on)) + except ValueError as exc: + raise ToolError("fail_on must be one of CRITICAL/ERROR/WARN/INFO") from exc + lang = str(args.get("lang") or "python") + if lang not in {"python", "rust"}: + raise ToolError("lang must be one of python/rust") + fmt = str(args.get("format") or "jsonl") + if fmt not in {"jsonl", "sarif", "agent-summary"}: + raise ToolError("format must be one of jsonl/sarif/agent-summary") + local_only = _bool_arg(args, "local_only", False) + trusted_packs = _trusted_packs_arg(args) + output = _resolve_under_root(root, args["output"]) if args.get("output") else None + config = _cfg(args, root) + cache_dir = _cache_dir_arg(args, root) + return { + "config": str(config) if config is not None else None, + "format": fmt, + "output": str(output) if output is not None else None, + "fail_on": str(fail_on).upper() if fail_on else None, + "fail_on_unanalyzed": _bool_arg(args, "fail_on_unanalyzed", False), + "cache_dir": str(cache_dir) if cache_dir is not None else None, + "filigree_url": None if local_only else filigree_url, + "local_only": local_only, + "filigree_max_findings_per_request": args.get("filigree_max_findings_per_request"), + "timeout_seconds": args.get("timeout_seconds"), + "lang": lang, + "new_since": args.get("new_since"), + "trusted_packs": list(trusted_packs), + "trust_local_packs": _bool_arg(args, "trust_local_packs", False), + "strict_defaults": _bool_arg(args, "strict_defaults", False), + "trust_suppressions": _bool_arg(args, "trust_suppressions", False), + } + + +def _scan_job_start(args: dict[str, Any], root: Path, filigree_url: str | None = None) -> dict[str, Any]: + path = _path_arg(args, root) + request = _scan_job_request(args, root, filigree_url) + return start_scan_job(path, request) + + +def _scan_job_status(args: dict[str, Any], root: Path) -> dict[str, Any]: + job_id = str(_require(args, "job_id")) + return read_scan_job_status(_path_arg(args, root), job_id) + + +def _scan_job_cancel(args: dict[str, Any], root: Path) -> dict[str, Any]: + job_id = str(_require(args, "job_id")) + return cancel_scan_job(_path_arg(args, root), job_id) + + def _scan( args: dict[str, Any], root: Path, @@ -1674,6 +1733,152 @@ def _scan( } +_SCAN_JOB_STATUS_OUTPUT_SCHEMA: dict[str, Any] = { + "type": "object", + "description": "File-backed scan-job status. Non-terminal jobs include heartbeat/pid/progress; terminal jobs " + "include summary/gate/artifacts when the worker reached them.", + "properties": { + "job_id": {"type": "string"}, + "status": { + "type": "string", + "enum": [ + "queued", + "running", + "running_stale", + "completed", + "completed_with_enrichment_failure", + "failed", + "cancelled", + ], + }, + "phase": {"type": "string"}, + "progress": {"type": "object"}, + "heartbeat": {"type": "string"}, + "pid": {"type": "integer"}, + "artifacts": {"type": "object"}, + "failure_kind": {"type": ["string", "null"]}, + "error": {"type": ["string", "null"]}, + "request": {"type": "object"}, + }, + "required": ["job_id", "status", "phase", "progress", "heartbeat", "artifacts", "failure_kind", "error", "request"], + "additionalProperties": True, +} + + +_SCAN_JOB_START_INPUT_PROPERTIES: dict[str, Any] = { + "path": {"type": "string", "description": "scan root subdir relative to the MCP server project root"}, + "config": {"type": "string", "description": "config file relative to project root"}, + "format": {"type": "string", "enum": ["jsonl", "sarif", "agent-summary"], "description": "artifact format"}, + "output": {"type": "string", "description": "artifact output path relative to project root"}, + "fail_on": {"type": "string", "enum": _SEVERITY_ENUM}, + "fail_on_unanalyzed": { + "type": "boolean", + "description": "Trip the gate when any file was discovered but could not be analyzed.", + }, + "cache_dir": {"type": "string", "description": "summary-cache directory relative to project root"}, + "local_only": { + "type": "boolean", + "description": "Disable sibling emission even when a Filigree URL resolves from launch/env/install state.", + }, + "filigree_max_findings_per_request": {"type": "integer", "minimum": 1}, + "timeout_seconds": { + "type": "number", + "minimum": 0, + "description": "Fail the background scan job after this many seconds. Defaults to 1800; use 0 to disable.", + }, + "lang": {"type": "string", "enum": ["python", "rust"]}, + "new_since": { + "type": "string", + "description": "PR-scoped 'new findings only' gate: only gate on findings in files/entities changed " + "since this git ref.", + }, + "trust_packs": {"type": "array", "items": {"type": "string"}}, + "trust_local_packs": { + "type": "boolean", + "description": "Allow loading custom trust-grammar packs from the local project directory.", + }, + "strict_defaults": { + "type": "boolean", + "description": "Ignore repository-supplied custom configuration overrides (weft.toml).", + }, + "trust_suppressions": { + "type": "boolean", + "description": "Let repository-controlled baseline/waiver/judged files clear the gate.", + }, +} + + +_SCAN_JOB_START_TOOL: dict[str, Any] = { + "name": "scan_job_start", + "title": "Start scan job", + "description": "Start a file-backed Wardline scan job and return its stable job id plus initial status. " + "Use scan_job_status to poll heartbeat/progress and scan_job_cancel to stop it. This is the MCP-safe " + "surface for long scans; prefer it over synchronous scan when the project may take more than a short call.", + "input_schema": { + "type": "object", + "properties": _SCAN_JOB_START_INPUT_PROPERTIES, + }, + "output_schema": _SCAN_JOB_STATUS_OUTPUT_SCHEMA, + "annotations": { + "title": "Start scan job", + "readOnlyHint": False, + "destructiveHint": False, + "idempotentHint": False, + "openWorldHint": False, + }, + "capabilities": frozenset({ToolCapability.READ, ToolCapability.WRITE}), +} + + +_SCAN_JOB_STATUS_TOOL: dict[str, Any] = { + "name": "scan_job_status", + "title": "Read scan-job status", + "description": "Read the current status JSON for a file-backed Wardline scan job. Reports stale heartbeat " + "or dead-worker terminal failure instead of leaving an apparently hung job ambiguous.", + "input_schema": { + "type": "object", + "required": ["job_id"], + "properties": { + "job_id": {"type": "string", "pattern": "^[0-9a-f]{32}$"}, + "path": {"type": "string", "description": "scan root subdir relative to the MCP server project root"}, + }, + }, + "output_schema": _SCAN_JOB_STATUS_OUTPUT_SCHEMA, + "annotations": { + "title": "Read scan-job status", + "readOnlyHint": True, + "destructiveHint": False, + "idempotentHint": True, + "openWorldHint": False, + }, + "capabilities": frozenset({ToolCapability.READ}), +} + + +_SCAN_JOB_CANCEL_TOOL: dict[str, Any] = { + "name": "scan_job_cancel", + "title": "Cancel scan job", + "description": "Cancel a non-terminal file-backed Wardline scan job and return the persisted terminal status.", + "input_schema": { + "type": "object", + "required": ["job_id"], + "properties": { + "job_id": {"type": "string", "pattern": "^[0-9a-f]{32}$"}, + "path": {"type": "string", "description": "scan root subdir relative to the MCP server project root"}, + }, + }, + "output_schema": _SCAN_JOB_STATUS_OUTPUT_SCHEMA, + "annotations": { + "title": "Cancel scan job", + "readOnlyHint": False, + "destructiveHint": True, + "idempotentHint": True, + "openWorldHint": False, + }, + "capabilities": frozenset({ToolCapability.READ, ToolCapability.WRITE}), +} + + def _attach_legis_artifact( response: dict[str, Any], result: Any, @@ -4019,6 +4224,28 @@ def _register_tools(self) -> None: ), ) ) + self.add_tool( + Tool( + **_SCAN_JOB_START_TOOL, + handler=lambda args, root: _scan_job_start( + args, + root, + None if bool(args.get("local_only") or False) else self._resolved_filigree_url_for_policy(args), + ), + ) + ) + self.add_tool( + Tool( + **_SCAN_JOB_STATUS_TOOL, + handler=_scan_job_status, + ) + ) + self.add_tool( + Tool( + **_SCAN_JOB_CANCEL_TOOL, + handler=_scan_job_cancel, + ) + ) self.add_tool( Tool( **_EXPLAIN_TAINT_TOOL, @@ -4217,6 +4444,12 @@ def _effective_tool_capabilities(self, tool: Tool, arguments: dict[str, Any]) -> or self._resolved_filigree_url_for_policy(arguments) is not None ): capabilities.update({ToolCapability.NETWORK, ToolCapability.WRITE}) + if ( + tool.name == "scan_job_start" + and not bool(arguments.get("local_only", False)) + and self._resolved_filigree_url_for_policy(arguments) is not None + ): + capabilities.add(ToolCapability.NETWORK) if ( tool.name in {"explain_taint", "attest", "verify_attestation"} and self._resolved_loomweave_url_for_policy(arguments) is not None diff --git a/src/wardline/rust/analyzer.py b/src/wardline/rust/analyzer.py index 4f1a3a1c..3490ef14 100644 --- a/src/wardline/rust/analyzer.py +++ b/src/wardline/rust/analyzer.py @@ -9,8 +9,9 @@ ``run_scan`` drives under ``--lang rust``. It discovers the tree's Cargo crate roots ONCE (SP2, ``wardline.rust.crate_roots`` — the loomweave-oracle-mirroring whole-tree pass), routes each ``.rs`` file to its real crate-prefixed module (``_module_for``), runs the -per-file pipeline, and surfaces a ``WLN-ENGINE-PARSE-ERROR`` FACT for any file -tree-sitter cannot fully parse (then contributes no findings for it — never half-analyze). +per-file pipeline, and surfaces a gate-eligible ``WLN-ENGINE-PARSE-ERROR`` defect for +any file tree-sitter cannot fully parse (then contributes no findings for it — never +half-analyze). ``last_context`` is the engine-shaped ``AnalysisContext | None`` (None in slice-1: the Rust-native context is incompatible with the delta/SARIF consumers). The Rust-native @@ -86,7 +87,7 @@ def analyze(self, files: Sequence[Path], config: WardlineConfig, *, root: Path) ``config`` is accepted for protocol parity but unused in slice-1 (the Rust rules carry hardcoded base severities; ``weft.toml`` severity overrides are a preview gap, surfaced in the docs). A file that does not fully parse yields a - ``WLN-ENGINE-PARSE-ERROR`` FACT and no ``RS-WL-*`` findings. + gate-eligible ``WLN-ENGINE-PARSE-ERROR`` defect and no ``RS-WL-*`` findings. """ resolved_root = root.resolve() # SP2 whole-tree pass: discover Cargo crate roots ONCE per scan; every file's @@ -124,7 +125,7 @@ def analyze(self, files: Sequence[Path], config: WardlineConfig, *, root: Path) # One pathological file (e.g. a RecursionError on a deeply-nested expression) # must not abort the whole scan and lose every other file's findings. Mirror # the Python engine's per-function isolation: degrade to a counted diagnostic - # FACT (WLN-ENGINE-FILE-FAILED ∈ UNANALYZED_RULE_IDS) and keep scanning. + # DEFECT (WLN-ENGINE-FILE-FAILED ∈ UNANALYZED_RULE_IDS) and keep scanning. findings.append(_file_failed_finding(relpath, f"{type(exc).__name__}: {exc}")) continue self._last_rust_context = context @@ -310,13 +311,13 @@ def _out_route(crate: str, base: Path, file: Path) -> str: def _parse_error_finding(relpath: str, detail: str) -> Finding: # Reuse the engine's parse-error rule id so it counts toward ScanSummary.unanalyzed - # and the CLI "see WLN-ENGINE-* facts" line works for free (UNANALYZED_RULE_IDS). - return _engine_fact("WLN-ENGINE-PARSE-ERROR", f"{relpath}: could not parse Rust source ({detail})", relpath) + # while also tripping the default ERROR gate: unscanned code must not read green. + return _engine_defect("WLN-ENGINE-PARSE-ERROR", f"{relpath}: could not parse Rust source ({detail})", relpath) def _file_failed_finding(relpath: str, detail: str) -> Finding: # Analysis raised AFTER a clean parse — a per-file under-scan, counted toward unanalyzed. - return _engine_fact("WLN-ENGINE-FILE-FAILED", f"{relpath}: Rust analysis failed ({detail})", relpath) + return _engine_defect("WLN-ENGINE-FILE-FAILED", f"{relpath}: Rust analysis failed ({detail})", relpath) def _coverage_finding(functions_total: int, functions_declared: int, files_analyzed: int) -> Finding: @@ -349,13 +350,16 @@ def _coverage_finding(functions_total: int, functions_declared: int, files_analy ) -def _engine_fact(rule_id: str, message: str, relpath: str) -> Finding: +def _engine_defect(rule_id: str, message: str, relpath: str) -> Finding: return Finding( rule_id=rule_id, message=message, - severity=Severity.NONE, - kind=Kind.FACT, - location=Location(path=relpath), + severity=Severity.ERROR, + kind=Kind.DEFECT, + # File-level under-scan defects need a concrete source anchor. A lineless, + # non-ENGINE_PATH DEFECT is intentionally downgraded before gate evaluation + # to avoid unsafe fingerprint joins, so use the stable file start. + location=Location(path=relpath, line_start=1, line_end=1), fingerprint=_fp(rule_id, relpath), properties={"lang": "rust"}, ) diff --git a/tests/conformance/filigree_suppression_filter_contract.json b/tests/conformance/filigree_suppression_filter_contract.json new file mode 100644 index 00000000..7bcb6993 --- /dev/null +++ b/tests/conformance/filigree_suppression_filter_contract.json @@ -0,0 +1,20 @@ +{ + "contract": "weft/wardline-filigree-suppression-filter", + "version": 1, + "owner": "wardline", + "description": "Wardline-owned suppression_state vocabulary consumed by Filigree's finding-list suppression filter. Filigree adds only the local 'all' no-filter sentinel; every other accepted filter value must come from Wardline's SuppressionState enum.", + "suppression_states": [ + "active", + "baselined", + "waived", + "judged" + ], + "filigree_filter_sentinel": "all", + "filigree_filter_values": [ + "active", + "all", + "baselined", + "judged", + "waived" + ] +} diff --git a/tests/conformance/fixtures/PROVENANCE.md b/tests/conformance/fixtures/PROVENANCE.md new file mode 100644 index 00000000..0c9f7b25 --- /dev/null +++ b/tests/conformance/fixtures/PROVENANCE.md @@ -0,0 +1,10 @@ +# Vendored SEI conformance oracle fixture + +`sei-conformance-oracle.json` is a verbatim copy of the shared, normative +fixture from: + + /home/john/loomweave/docs/federation/fixtures/sei-conformance-oracle.json + +It is vendored so Wardline CI can run the SEI consumer oracle without requiring +the Loomweave checkout. `tests/conformance/test_sei_oracle.py` compares this +copy against the sibling authority fixture when the checkout is present. diff --git a/tests/conformance/fixtures/sei-conformance-oracle.json b/tests/conformance/fixtures/sei-conformance-oracle.json new file mode 100644 index 00000000..0ea57702 --- /dev/null +++ b/tests/conformance/fixtures/sei-conformance-oracle.json @@ -0,0 +1,85 @@ +{ + "_meta": { + "contract": "weft-sei-conformance-oracle", + "standard": "Weft Stable Entity Identity (SEI) conformance standard §8", + "authority": "Loomweave ADR-038 (token/signature/persistence/reserved-namespace); SEI standard (suite-wide)", + "fixture_version": 1, + "stability": "normative", + "token_format_agnostic": true, + "verification": "cargo test -p loomweave-storage --test sei_conformance_oracle", + "updated": "2026-06-02", + "description": "The six shared SEI conformance scenarios every Weft tool runs against a reference Loomweave. Asserts BEHAVIOUR and OPACITY, never the SEI's internal form. A subsystem is SEI-conformant only when it passes all six (no grandfathering)." + }, + "invariants": [ + "SEI is opaque: a consumer never parses it. It carries the reserved `loomweave:eid:` prefix and is NOT a locator.", + "Fail-closed: when sameness cannot be PROVEN, mint a new SEI and orphan the old one — never silently re-point.", + "Lineage is append-only: born / locator_changed / moved / orphaned / superseded.", + "Identity is carried (never re-minted) for an unchanged locator; SEI values are not part of the byte-identical-run guarantee, but carry/mint decisions are deterministic given the same bindings + source." + ], + "scenarios": [ + { + "id": "identity_round_trip_and_opacity", + "given": "A function entity is analyzed for the first time.", + "when": "Mint an SEI; resolve(locator) → sei; resolve_sei(sei) → locator.", + "expect": { + "resolve_locator": { "sei": "", "current_locator": "", "content_hash": "", "alive": true }, + "resolve_sei": { "current_locator": "", "alive": true }, + "opacity": "the returned `sei` begins with `loomweave:eid:` and is treated as an opaque string by the consumer (never parsed); it is not equal to the locator" + } + }, + { + "id": "rename", + "given": "An entity with an alive SEI; its file/module is renamed so the locator prefix changes; the body is byte-identical; a git-rename signal maps old_locator → new_locator.", + "when": "Re-index.", + "expect": { + "carry": true, + "sei": "unchanged (same token as before)", + "current_locator": "the new locator", + "lineage_appends": "locator_changed", + "resolve_locator(old)": { "alive": false }, + "resolve_locator(new)": { "alive": true, "sei": "" } + } + }, + { + "id": "move", + "given": "An entity with an alive SEI is moved to a new module; body hash AND signature are identical at the new locator; exactly one vanished candidate matches; no git signal required.", + "when": "Re-index.", + "expect": { + "carry": true, + "sei": "unchanged", + "lineage_appends": "moved" + } + }, + { + "id": "ambiguous", + "given": "An entity is renamed WITH a body edit (the body hash changes), even if a git-rename signal is present.", + "when": "Re-index.", + "expect": { + "carry": false, + "new_entity": "minted a fresh SEI (born)", + "old_binding": "orphaned (resolve_sei → alive:false with an `orphaned` lineage event)", + "rationale": "the matcher cannot PROVE sameness → fail closed; a governance attestation on the old SEI is never silently carried across an unproven match" + } + }, + { + "id": "delete", + "given": "An entity present in a prior run is absent from the current run and was not rematched by a rename/move.", + "when": "Re-index.", + "expect": { + "old_binding": "orphaned", + "resolve_locator(old)": { "alive": false }, + "resolve_sei(old_sei)": { "alive": false, "lineage": "includes an `orphaned` event" } + } + }, + { + "id": "capability_absent", + "given": "A Loomweave instance that has not populated SEI (pre-SEI DB, or `_capabilities.sei.supported` false / absent).", + "when": "A consumer probes `_capabilities` and/or resolves.", + "expect": { + "consumer": "detects the absent capability and DEGRADES gracefully — keeps working on locators, no crash, honest 'identity unavailable'", + "resolve_locator(any)": { "alive": false }, + "resolve_sei(unknown)": { "alive": false, "lineage": [] } + } + } + ] +} diff --git a/tests/conformance/legis_dirty_scan_wire.golden.json b/tests/conformance/legis_dirty_scan_wire.golden.json new file mode 100644 index 00000000..ddf57d02 --- /dev/null +++ b/tests/conformance/legis_dirty_scan_wire.golden.json @@ -0,0 +1,43 @@ +{ + "contract": "weft/wardline-dirty-scan-artifact", + "version": 1, + "description": "Shared Weft conformance vector for the unsigned Wardline->legis dirty dev-artifact path. Wardline emits this shape when scan --format legis --allow-dirty is used on a dirty working tree; Legis consumes the same dirty key to distinguish keyless dev, CI skip, and explicit dev-mode governance. The artifact is intentionally unsigned because signing dirty working-tree content would assert false tree provenance.", + "dirty_key": "dirty", + "signature_key": "artifact_signature", + "signing": { + "key_utf8": "test-shared-secret-key", + "policy": "dirty dev artifacts are never signed; a configured consumer key without dev-mode must return SKIPPED_DIRTY_TREE" + }, + "valid": [ + { + "name": "dirty_unsigned_dev_artifact", + "description": "Unsigned dirty-tree dev artifact: dirty must be strict boolean true, artifact_signature must be absent, and findings remains present even when empty.", + "artifact": { + "scanner_identity": "wardline@CONFORMANCE", + "rule_set_version": "sha256:0000000000000000000000000000000000000000000000000000000000000000", + "fingerprint_scheme": "wlfp2", + "findings": [], + "scan_scope": { + "schema": "wardline-legis-scan-scope-1", + "scan_root": ".", + "is_git_root": true, + "source_roots": [ + "." + ], + "resolved_source_roots": [ + "." + ], + "scanned_paths": [ + "svc.py" + ] + }, + "commit_sha": "cccccccccccccccccccccccccccccccccccccccc", + "tree_sha": "dddddddddddddddddddddddddddddddddddddddd", + "dirty": true + }, + "expected_keyless_artifact_status": "dirty", + "expected_ci_allow_dirty_artifact_status": "dirty", + "expected_ci_reject_reason": "SKIPPED_DIRTY_TREE" + } + ] +} diff --git a/tests/conformance/test_filigree_suppression_filter_contract.py b/tests/conformance/test_filigree_suppression_filter_contract.py new file mode 100644 index 00000000..7451ed74 --- /dev/null +++ b/tests/conformance/test_filigree_suppression_filter_contract.py @@ -0,0 +1,36 @@ +"""Wardline-owned suppression-state vocabulary consumed by Filigree. + +Filigree's finding-list ``suppression`` filter accepts Wardline's +``SuppressionState`` values plus its own local ``all`` no-filter sentinel. This +contract file is the producer-side anchor: if Wardline adds a suppression state, +the shared vector must change in the same commit and Filigree's consumer test +will fail until its filter grammar follows. +""" + +from __future__ import annotations + +import json +from pathlib import Path + +from wardline.core.finding import SuppressionState + +VECTOR_PATH = Path(__file__).parent / "filigree_suppression_filter_contract.json" + + +def _vector() -> dict: + return json.loads(VECTOR_PATH.read_text(encoding="utf-8")) + + +def test_vector_matches_suppression_state_enum() -> None: + vector = _vector() + + assert vector["contract"] == "weft/wardline-filigree-suppression-filter" + assert set(vector["suppression_states"]) == {state.value for state in SuppressionState} + + +def test_filigree_filter_values_are_enum_plus_all_sentinel() -> None: + vector = _vector() + expected = {state.value for state in SuppressionState} | {vector["filigree_filter_sentinel"]} + + assert vector["filigree_filter_sentinel"] == "all" + assert set(vector["filigree_filter_values"]) == expected diff --git a/tests/conformance/test_legis_scan_wire_golden.py b/tests/conformance/test_legis_scan_wire_golden.py index 75f38a55..92693848 100644 --- a/tests/conformance/test_legis_scan_wire_golden.py +++ b/tests/conformance/test_legis_scan_wire_golden.py @@ -30,6 +30,7 @@ from wardline.core.config import load as load_config from wardline.core.legis import ( + ARTIFACT_SIGNATURE_FIELD, DIRTY_FIELD, FINDINGS_FIELD, FINGERPRINT_SCHEME_FIELD, @@ -44,6 +45,7 @@ GOLDEN_KEY = b"weft-shared-conformance-key" _VECTOR_PATH = Path(__file__).parent / "legis_scan_wire.golden.json" +_DIRTY_VECTOR_PATH = Path(__file__).parent / "legis_dirty_scan_wire.golden.json" # Same leaky boundary→sink fixture the freeze test uses: yields one real PY-WL-101 # defect carrying every per-finding wire key. @@ -58,6 +60,10 @@ def _vector() -> dict: return json.loads(_VECTOR_PATH.read_text(encoding="utf-8")) +def _dirty_vector() -> dict: + return json.loads(_DIRTY_VECTOR_PATH.read_text(encoding="utf-8")) + + def _signed_clean_artifact(tmp_path: Path) -> dict: proj = tmp_path / "proj" proj.mkdir() @@ -75,6 +81,24 @@ def _signed_clean_artifact(tmp_path: Path) -> dict: return build_legis_artifact(result, root=proj, config=cfg, key=GOLDEN_KEY) +def _dirty_unsigned_artifact(tmp_path: Path) -> dict: + proj = tmp_path / "proj" + proj.mkdir() + (proj / "svc.py").write_text(_LEAKY, encoding="utf-8") + for cmd in ( + ["git", "init", "-q"], + ["git", "config", "user.email", "t@example.com"], + ["git", "config", "user.name", "t"], + ["git", "add", "-A"], + ["git", "commit", "-qm", "init"], + ): + subprocess.run(cmd, cwd=proj, check=True, capture_output=True) + (proj / "svc.py").write_text(_LEAKY + "\n# uncommitted edit\n", encoding="utf-8") + result = run_scan(proj) + cfg = load_config(proj / "weft.toml") + return build_legis_artifact(result, root=proj, config=cfg, key=GOLDEN_KEY, allow_dirty=True) + + def test_golden_vector_is_a_valid_signed_artifact() -> None: # The consumer verifies the vector offline; prove it round-trips under GOLDEN_KEY. vector = _vector() @@ -116,3 +140,23 @@ def test_vector_defect_routes_as_active(tmp_path: Path) -> None: defect = vector[FINDINGS_FIELD][0] assert defect["kind"] == "defect" assert defect["suppression_state"] == "active" + + +def test_dirty_vector_declares_the_named_dirty_contract() -> None: + vector = _dirty_vector() + assert vector["contract"] == "weft/wardline-dirty-scan-artifact" + assert vector["dirty_key"] == DIRTY_FIELD + assert vector["signature_key"] == ARTIFACT_SIGNATURE_FIELD + + +def test_live_dirty_emit_top_level_keys_match_the_dirty_vector(tmp_path: Path) -> None: + case = _dirty_vector()["valid"][0] + live = _dirty_unsigned_artifact(tmp_path) + expected = case["artifact"] + + assert set(live) == set(expected) + assert live[DIRTY_FIELD] is True + assert ARTIFACT_SIGNATURE_FIELD not in live + assert isinstance(live[FINDINGS_FIELD], list) + assert live[FINGERPRINT_SCHEME_FIELD] == expected[FINGERPRINT_SCHEME_FIELD] + assert set(live[SCAN_SCOPE_FIELD]) == set(expected[SCAN_SCOPE_FIELD]) diff --git a/tests/conformance/test_mcp_handshake.py b/tests/conformance/test_mcp_handshake.py index 2ee5f2bc..4cd2ede2 100644 --- a/tests/conformance/test_mcp_handshake.py +++ b/tests/conformance/test_mcp_handshake.py @@ -53,10 +53,13 @@ def test_full_client_handshake_and_every_surface() -> None: assert init["protocolVersion"] == PROTOCOL_VERSION assert init["serverInfo"]["name"] == "wardline" assert {"tools", "resources", "prompts"} <= set(init["capabilities"]) - # tools/list: the fifteen documented tools, no more no less + # tools/list: the eighteen documented tools, no more no less tool_names = {t["name"] for t in by_id[2]["result"]["tools"]} assert tool_names == { "scan", + "scan_job_start", + "scan_job_status", + "scan_job_cancel", "explain_taint", "dossier", "assure", diff --git a/tests/conformance/test_mcp_structured_output.py b/tests/conformance/test_mcp_structured_output.py index 2fbfd642..bbbb1f17 100644 --- a/tests/conformance/test_mcp_structured_output.py +++ b/tests/conformance/test_mcp_structured_output.py @@ -1,4 +1,4 @@ -"""B1+B2 conformance: MCP structured output + display metadata across all 15 tools. +"""B1+B2 conformance: MCP structured output + display metadata across all 18 tools. Three layers, each pinned for EVERY registered tool: @@ -31,9 +31,12 @@ FIXTURE = Path("tests/fixtures/sample_project") -# The published 15-tool surface, in advertisement order. +# The published 18-tool surface, in advertisement order. EXPECTED_TOOLS = ( "scan", + "scan_job_start", + "scan_job_status", + "scan_job_cancel", "explain_taint", "dossier", "assure", @@ -52,7 +55,16 @@ # B2 acceptance: the pure-read surface advertises readOnlyHint: true. READ_ONLY_TOOLS = frozenset( - {"scan", "explain_taint", "dossier", "assure", "decorator_coverage", "attest", "verify_attestation"} + { + "scan", + "scan_job_status", + "explain_taint", + "dossier", + "assure", + "decorator_coverage", + "attest", + "verify_attestation", + } ) _ANNOTATION_KEYS = {"title", "readOnlyHint", "destructiveHint", "idempotentHint", "openWorldHint"} @@ -164,6 +176,52 @@ def test_scan_structured_output(tmp_path: Path) -> None: assert out["gate"]["tripped"] is True +def _job_status_stub(job_id: str = "a" * 32, status: str = "running") -> dict[str, Any]: + """A schema-valid scan-job status payload (the core's _base_status shape).""" + return { + "job_id": job_id, + "status": status, + "phase": "scanning", + "progress": {"steps_completed": 1, "steps_total": 4}, + "heartbeat": "2026-06-13T00:00:00Z", + "artifacts": {}, + "failure_kind": None, + "error": None, + "request": {}, + } + + +def test_scan_job_start_structured_output(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None: + # Hermetic: never spawn a worker subprocess — pin the core start to a + # schema-valid status. local_only keeps the network-fenced default happy. + import wardline.mcp.server as server_mod + + monkeypatch.setattr(server_mod, "start_scan_job", lambda root, request, **kw: _job_status_stub()) + server = WardlineMCPServer(root=_leaky_project(tmp_path)) + out = _validated(server, "scan_job_start", {"local_only": True}) + assert out["status"] == "running" + + +def test_scan_job_status_structured_output(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None: + import wardline.mcp.server as server_mod + + monkeypatch.setattr(server_mod, "read_scan_job_status", lambda root, job_id: _job_status_stub(job_id=job_id)) + server = WardlineMCPServer(root=_leaky_project(tmp_path)) + out = _validated(server, "scan_job_status", {"job_id": "b" * 32}) + assert out["job_id"] == "b" * 32 + + +def test_scan_job_cancel_structured_output(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None: + import wardline.mcp.server as server_mod + + monkeypatch.setattr( + server_mod, "cancel_scan_job", lambda root, job_id: _job_status_stub(job_id=job_id, status="cancelled") + ) + server = WardlineMCPServer(root=_leaky_project(tmp_path)) + out = _validated(server, "scan_job_cancel", {"job_id": "c" * 32}) + assert out["status"] == "cancelled" + + def test_explain_taint_structured_output(tmp_path: Path) -> None: server = WardlineMCPServer(root=_leaky_project(tmp_path)) scan_out = _validated(server, "scan", {}) @@ -302,7 +360,7 @@ def test_rekey_structured_output(tmp_path: Path) -> None: def test_execution_conformance_covers_every_advertised_tool(fixture_server: WardlineMCPServer) -> None: - """Tripwire: a 16th tool must add an execution-conformance case above.""" + """Tripwire: a 19th tool must add an execution-conformance case above.""" assert set(_entries(fixture_server)) == set(EXPECTED_TOOLS) diff --git a/tests/conformance/test_sei_oracle.py b/tests/conformance/test_sei_oracle.py new file mode 100644 index 00000000..6db99d42 --- /dev/null +++ b/tests/conformance/test_sei_oracle.py @@ -0,0 +1,156 @@ +"""Weft SEI §8 conformance oracle — Wardline as consumer. + +The scenario list is loaded from the vendored ``sei-conformance-oracle.json`` +fixture, copied from Loomweave's authoritative fixture. Each scenario id is +claimed by one consumer assertion so a fixture change fails CI until Wardline +updates the corresponding behavior check. +""" + +from __future__ import annotations + +import json +import os +from pathlib import Path +from typing import Any + +import pytest + +from wardline.loomweave.identity import IdentityStatus, SeiCapability, SeiResolver + +ORACLE_PATH = Path(__file__).parent / "fixtures" / "sei-conformance-oracle.json" + + +def _load_oracle() -> dict[str, Any]: + return json.loads(ORACLE_PATH.read_text(encoding="utf-8")) + + +def _scenario(scenario_id: str) -> dict[str, Any]: + for item in _load_oracle()["scenarios"]: + if item["id"] == scenario_id: + return item + raise AssertionError(f"missing SEI oracle scenario {scenario_id!r}") + + +def _loomweave_oracle_source() -> Path | None: + candidates: list[Path] = [] + if env := os.environ.get("LOOMWEAVE_REPO"): + candidates.append(Path(env) / "docs" / "federation" / "fixtures" / "sei-conformance-oracle.json") + candidates.append( + Path(__file__).resolve().parents[3] + / "loomweave" + / "docs" + / "federation" + / "fixtures" + / "sei-conformance-oracle.json" + ) + return next((path for path in candidates if path.exists()), None) + + +COVERED_SCENARIOS = { + "identity_round_trip_and_opacity", + "rename", + "move", + "ambiguous", + "delete", + "capability_absent", +} + + +class FakeClient: + def __init__( + self, + *, + caps: dict[str, Any] | None = None, + resolve: dict[str, Any] | None = None, + resolve_sei: dict[str, Any] | None = None, + ) -> None: + self._caps = caps + self._resolve = resolve + self._resolve_sei = resolve_sei + self.resolve_calls: list[str] = [] + + def capabilities(self) -> dict[str, Any] | None: + return self._caps + + def resolve_identity(self, locator: str) -> dict[str, Any] | None: + self.resolve_calls.append(locator) + return self._resolve + + def resolve_sei(self, sei: str) -> dict[str, Any] | None: + return self._resolve_sei + + +def test_vendored_oracle_matches_loomweave_source() -> None: + source = _loomweave_oracle_source() + if source is None: + pytest.skip("Loomweave repo not found; set LOOMWEAVE_REPO to enable drift check") + assert _load_oracle() == json.loads(source.read_text(encoding="utf-8")) + + +def test_every_oracle_scenario_is_covered() -> None: + fixture_ids = {item["id"] for item in _load_oracle()["scenarios"]} + assert fixture_ids == COVERED_SCENARIOS + + +def test_identity_round_trip_and_opacity() -> None: + scenario = _scenario("identity_round_trip_and_opacity") + locator = "python:function:m.f" + sei = "loomweave:eid:0123456789abcdef0123456789abcdef" + client = FakeClient( + caps={"sei": {"supported": True, "version": 1}}, + resolve={"sei": sei, "current_locator": locator, "content_hash": "h", "alive": True}, + resolve_sei={"current_locator": locator, "content_hash": "h", "alive": True}, + ) + resolver = SeiResolver.detect(client) + + binding = resolver.resolve_locator(locator) + + assert scenario["expect"]["resolve_locator"]["alive"] is True + assert binding.identity is IdentityStatus.ALIVE + assert binding.sei == sei + assert binding.binding_key == sei + assert binding.keyed_on_sei is True + assert binding.sei.startswith("loomweave:eid:") + assert binding.sei != locator + assert resolver.resolve_identity_status(sei) is IdentityStatus.ALIVE + + +@pytest.mark.parametrize("scenario_id", ["rename", "move"]) +def test_carried_sei_remains_alive_for_rename_and_move(scenario_id: str) -> None: + scenario = _scenario(scenario_id) + sei = "loomweave:eid:carried" + client = FakeClient(caps={"sei": {"supported": True, "version": 1}}, resolve_sei={"sei": sei, "alive": True}) + resolver = SeiResolver.detect(client) + + assert scenario["expect"]["carry"] is True + assert resolver.resolve_identity_status(sei) is IdentityStatus.ALIVE + + +@pytest.mark.parametrize("scenario_id", ["ambiguous", "delete"]) +def test_orphaned_sei_surfaces_as_orphaned_for_ambiguous_and_delete(scenario_id: str) -> None: + scenario = _scenario(scenario_id) + sei = "loomweave:eid:orphaned" + client = FakeClient( + caps={"sei": {"supported": True, "version": 1}}, + resolve_sei={"sei": sei, "alive": False, "lineage": [{"event": "orphaned"}]}, + ) + resolver = SeiResolver.detect(client) + + assert "orphaned" in json.dumps(scenario["expect"]) + assert resolver.resolve_identity_status(sei) is IdentityStatus.ORPHANED + + +def test_capability_absent_degrades_gracefully() -> None: + scenario = _scenario("capability_absent") + locator = "python:function:any" + client = FakeClient(caps={"linkages": {"http": True}}) + resolver = SeiResolver.detect(client) + + binding = resolver.resolve_locator(locator) + + assert scenario["expect"]["resolve_locator(any)"]["alive"] is False + assert resolver.capability == SeiCapability(supported=False) + assert binding.identity is IdentityStatus.UNAVAILABLE + assert binding.sei is None + assert binding.binding_key == locator + assert client.resolve_calls == [] diff --git a/tests/docs/test_glossary_vocabulary.py b/tests/docs/test_glossary_vocabulary.py index 6bdc8de8..88dce876 100644 --- a/tests/docs/test_glossary_vocabulary.py +++ b/tests/docs/test_glossary_vocabulary.py @@ -26,35 +26,35 @@ # ``(repo-relative path, 1-based line, substring required on that line)``. _ANCHORS: tuple[tuple[str, int, str], ...] = ( # src/wardline/core/run.py — ScanSummary fields, gate population, delta-scope, gate_decision - ("src/wardline/core/run.py", 50, "total: int"), - ("src/wardline/core/run.py", 51, "active: int"), - ("src/wardline/core/run.py", 53, "baselined: int"), - ("src/wardline/core/run.py", 54, "waived: int"), - ("src/wardline/core/run.py", 55, "judged: int"), - ("src/wardline/core/run.py", 61, "informational: int"), - ("src/wardline/core/run.py", 69, "unanalyzed: int"), - ("src/wardline/core/run.py", 88, "gate_findings:"), - ("src/wardline/core/run.py", 98, "class GateDecision"), - ("src/wardline/core/run.py", 107, "verdict: str"), - ("src/wardline/core/run.py", 326, "Baseline(frozenset())"), - ("src/wardline/core/run.py", 354, "def apply_delta_scope"), - ("src/wardline/core/run.py", 378, "active=sum"), - ("src/wardline/core/run.py", 437, "honors_suppressions"), + ("src/wardline/core/run.py", 51, "total: int"), + ("src/wardline/core/run.py", 52, "active: int"), + ("src/wardline/core/run.py", 54, "baselined: int"), + ("src/wardline/core/run.py", 55, "waived: int"), + ("src/wardline/core/run.py", 56, "judged: int"), + ("src/wardline/core/run.py", 62, "informational: int"), + ("src/wardline/core/run.py", 70, "unanalyzed: int"), + ("src/wardline/core/run.py", 89, "gate_findings:"), + ("src/wardline/core/run.py", 99, "class GateDecision"), + ("src/wardline/core/run.py", 108, "verdict: str"), + ("src/wardline/core/run.py", 341, "Baseline(frozenset())"), + ("src/wardline/core/run.py", 369, "def apply_delta_scope"), + ("src/wardline/core/run.py", 393, "active=sum"), + ("src/wardline/core/run.py", 452, "honors_suppressions"), # src/wardline/cli/scan.py — CLI summary line + gate stderr - ("src/wardline/cli/scan.py", 415, "suppressed"), - ("src/wardline/cli/scan.py", 416, "{s.active} active"), - ("src/wardline/cli/scan.py", 468, "gate: FAILED"), + ("src/wardline/cli/scan.py", 457, "suppressed"), + ("src/wardline/cli/scan.py", 458, "{s.active} active"), + ("src/wardline/cli/scan.py", 510, "gate: FAILED"), # src/wardline/mcp/server.py — MCP scan summary + gate block - ("src/wardline/mcp/server.py", 774, '"total": result.summary.total'), - ("src/wardline/mcp/server.py", 775, '"active": result.summary.active'), - ("src/wardline/mcp/server.py", 776, '"baselined": result.summary.baselined'), - ("src/wardline/mcp/server.py", 777, '"waived": result.summary.waived'), - ("src/wardline/mcp/server.py", 778, '"judged": result.summary.judged'), - ("src/wardline/mcp/server.py", 783, '"informational": result.summary.informational'), - ("src/wardline/mcp/server.py", 787, '"unanalyzed": result.summary.unanalyzed'), - ("src/wardline/mcp/server.py", 789, '"gate": {'), - ("src/wardline/mcp/server.py", 790, '"tripped": decision.tripped'), - ("src/wardline/mcp/server.py", 794, '"verdict": decision.verdict'), + ("src/wardline/mcp/server.py", 841, '"total": result.summary.total'), + ("src/wardline/mcp/server.py", 842, '"active": result.summary.active'), + ("src/wardline/mcp/server.py", 843, '"baselined": result.summary.baselined'), + ("src/wardline/mcp/server.py", 844, '"waived": result.summary.waived'), + ("src/wardline/mcp/server.py", 845, '"judged": result.summary.judged'), + ("src/wardline/mcp/server.py", 850, '"informational": result.summary.informational'), + ("src/wardline/mcp/server.py", 854, '"unanalyzed": result.summary.unanalyzed'), + ("src/wardline/mcp/server.py", 856, '"gate": {'), + ("src/wardline/mcp/server.py", 857, '"tripped": decision.tripped'), + ("src/wardline/mcp/server.py", 861, '"verdict": decision.verdict'), # src/wardline/core/agent_summary.py — agent-summary JSON keys ("src/wardline/core/agent_summary.py", 134, '"total_findings"'), ("src/wardline/core/agent_summary.py", 135, '"active_defects"'), @@ -70,7 +70,7 @@ ("src/wardline/core/agent_summary.py", 171, '"informational": informational'), # per-finding suppression_state output key (renamed from `suppressed`, weft-f506e5f845) ("src/wardline/core/finding.py", 140, '"suppression_state"'), - ("src/wardline/core/finding.py", 285, 'wardline["suppression_state"]'), + ("src/wardline/core/finding.py", 295, 'wardline["suppression_state"]'), # stable-file anchors (lower churn, but locked for free) ("src/wardline/core/finding.py", 72, 'ACTIVE = "active"'), ("src/wardline/core/suppression.py", 24, "SuppressionState.BASELINED"), diff --git a/tests/golden/identity/corpus/META.json b/tests/golden/identity/corpus/META.json index e42457bc..82fbc731 100644 --- a/tests/golden/identity/corpus/META.json +++ b/tests/golden/identity/corpus/META.json @@ -1,5 +1,5 @@ { "corpus_version": 4, "fingerprint_scheme": "wlfp2", - "reason": "explain_taint B7 fix (weft-0d24cf9152): TaintExplanation gains the additive 'sink' field (dotted sink callable for call-site-anchored sink findings); fingerprint values unchanged" + "reason": "assure posture gains additive unanalyzed_total coverage field (pollable scan-job + coverage WIP)" } diff --git a/tests/golden/identity/corpus/assure.json b/tests/golden/identity/corpus/assure.json index 53aa21f3..11643539 100644 --- a/tests/golden/identity/corpus/assure.json +++ b/tests/golden/identity/corpus/assure.json @@ -8,6 +8,7 @@ "judged_total": 0, "proven": 3, "unanalyzed_rule_ids": [], + "unanalyzed_total": 0, "unknown": [], "waiver_debt": [] }, @@ -20,6 +21,7 @@ "judged_total": 0, "proven": 1, "unanalyzed_rule_ids": [], + "unanalyzed_total": 0, "unknown": [], "waiver_debt": [] }, @@ -32,6 +34,7 @@ "judged_total": 0, "proven": 3, "unanalyzed_rule_ids": [], + "unanalyzed_total": 0, "unknown": [], "waiver_debt": [] } diff --git a/tests/unit/cli/test_cli.py b/tests/unit/cli/test_cli.py index 0ace9e56..08bfbf98 100644 --- a/tests/unit/cli/test_cli.py +++ b/tests/unit/cli/test_cli.py @@ -32,6 +32,29 @@ def test_scan_writes_findings_and_exits_zero(tmp_path: Path) -> None: assert any(_json.loads(ln)["rule_id"] == "WLN-ENGINE-METRICS" for ln in lines) +def test_scan_warns_when_weft_toml_is_missing(tmp_path: Path) -> None: + (tmp_path / "app.py").write_text("def ok():\n return 1\n", encoding="utf-8") + out = tmp_path / "findings.jsonl" + + result = CliRunner().invoke(cli, ["scan", str(tmp_path), "--output", str(out)]) + + assert result.exit_code == 0, result.output + assert "warning: no weft.toml found" in result.output + assert "wardline doctor --repair" in result.output + assert "wardline scan-job start" in result.output + + +def test_scan_does_not_warn_when_weft_toml_exists(tmp_path: Path) -> None: + (tmp_path / "weft.toml").write_text('[wardline]\nsource_roots = ["."]\n', encoding="utf-8") + (tmp_path / "app.py").write_text("def ok():\n return 1\n", encoding="utf-8") + out = tmp_path / "findings.jsonl" + + result = CliRunner().invoke(cli, ["scan", str(tmp_path), "--output", str(out)]) + + assert result.exit_code == 0, result.output + assert "warning: no weft.toml found" not in result.output + + def test_scan_format_sarif_writes_sarif_file(tmp_path: Path) -> None: out = tmp_path / "out.sarif" result = CliRunner().invoke(cli, ["scan", str(FIXTURE), "--format", "sarif", "--output", str(out)]) @@ -632,6 +655,7 @@ def test_scan_benign_no_module_is_quiet(tmp_path) -> None: proj = tmp_path / "proj" proj.mkdir() + (proj / "weft.toml").write_text('[wardline]\nsource_roots = ["."]\n', encoding="utf-8") _write(proj, "__init__.py", "VERSION = 1\n") _write(proj, "mod.py", "def g(): return 1\n") out = tmp_path / "f.jsonl" @@ -815,6 +839,7 @@ def test_scan_relative_root_emits_relative_path_and_qualname(tmp_path) -> None: def test_scan_filigree_emit_success(tmp_path, monkeypatch) -> None: proj = tmp_path / "proj" proj.mkdir() + (proj / "weft.toml").write_text('[wardline]\nsource_roots = ["."]\n', encoding="utf-8") _write(proj, "svc.py", _LEAKY) captured: dict[str, object] = {} @@ -838,26 +863,63 @@ def emit(self, findings, *, scanned_paths=()): assert captured["url"] == "http://x/api/weft/scan-results" assert captured["scanned_paths"] == ("svc.py",) assert "emitted" in result.output and "warning" not in result.output # stats surfaced, no warning + assert "unscoped endpoint" in result.output + assert "server-default project" not in result.output -def test_scan_filigree_protocol_error_exits_2(tmp_path, monkeypatch) -> None: +def test_scan_filigree_protocol_error_does_not_preempt_gate(tmp_path, monkeypatch) -> None: proj = tmp_path / "proj" proj.mkdir() _write(proj, "svc.py", _LEAKY) - from wardline.core.errors import FiligreeEmitError + captured: dict[str, object] = {} - class _BadEmitter: + class _RejectedEmitter: def __init__(self, url, **kw): - pass + captured["url"] = url + captured["kwargs"] = kw def emit(self, findings, *, scanned_paths=()): - raise FiligreeEmitError("Filigree rejected (400): bad path") + from wardline.core.filigree_emit import EmitResult - monkeypatch.setattr("wardline.cli.scan.FiligreeEmitter", _BadEmitter) + return EmitResult( + reachable=True, + failed=len(findings), + warnings=("Filigree rejected scan-results (400): payload too large",), + ) + + monkeypatch.setattr("wardline.cli.scan.FiligreeEmitter", _RejectedEmitter) out = tmp_path / "f.jsonl" - result = CliRunner().invoke(scan, [str(proj), "--output", str(out), "--filigree-url", "http://x"]) - assert result.exit_code == 2, result.output - assert "bad path" in result.output + result = CliRunner().invoke( + scan, + [str(proj), "--output", str(out), "--filigree-url", "http://x", "--fail-on", "ERROR"], + ) + assert result.exit_code == 1, result.output + assert captured["url"] == "http://x" + kwargs = captured["kwargs"] + assert isinstance(kwargs, dict) + assert kwargs["protocol_errors_loud"] is False + assert "Filigree rejected scan-results (400)" in result.output + assert "gate: FAILED" in result.output + + +def test_scan_local_only_suppresses_resolved_emit_urls(tmp_path, monkeypatch) -> None: + proj = tmp_path / "proj" + proj.mkdir() + _write(proj, "svc.py", _LEAKY) + monkeypatch.setenv("WARDLINE_FILIGREE_URL", "http://x/api/weft/scan-results") + monkeypatch.setenv("WARDLINE_LOOMWEAVE_URL", "http://loom/api/wardline/taint-facts") + + class _UnexpectedEmitter: + def __init__(self, *args, **kwargs): + raise AssertionError("local-only must not construct FiligreeEmitter") + + monkeypatch.setattr("wardline.cli.scan.FiligreeEmitter", _UnexpectedEmitter) + out = tmp_path / "f.jsonl" + result = CliRunner().invoke(scan, [str(proj), "--output", str(out), "--local-only"]) + + assert result.exit_code == 0, result.output + assert "emitted" not in result.output + assert "Filigree" not in result.output def test_scan_filigree_absent_continues(tmp_path, monkeypatch) -> None: diff --git a/tests/unit/cli/test_doctor.py b/tests/unit/cli/test_doctor.py index be5f5ed2..f927ae06 100644 --- a/tests/unit/cli/test_doctor.py +++ b/tests/unit/cli/test_doctor.py @@ -38,13 +38,17 @@ def test_doctor_repair_installs_artifacts_and_discovers_bindings(tmp_path: Path, assert "CLAUDE.md: repaired" in result.output assert ".mcp.json: repaired" in result.output assert "Codex MCP: repaired" in result.output + assert "weft.toml: created" in result.output # Bindings are no longer wired into config — repair only DETECTS siblings and - # ensures the .weft/wardline/ state dir exists. No config file is written. + # ensures the .weft/wardline/ state dir exists. Doctor now creates Wardline's + # bounded local policy when it is missing. assert "bindings: detected" in result.output assert (tmp_path / ".mcp.json").is_file() assert (home / ".codex" / "config.toml").is_file() assert (tmp_path / ".weft" / "wardline").is_dir() - assert not (tmp_path / "weft.toml").exists() + cfg = (tmp_path / "weft.toml").read_text(encoding="utf-8") + assert 'source_roots = ["."]' in cfg + assert '".uv-cache/**"' in cfg def test_doctor_passes_after_repair(tmp_path: Path, monkeypatch) -> None: @@ -97,6 +101,10 @@ def test_doctor_fix_emits_shared_machine_readable_shape(tmp_path: Path, monkeypa assert checks[check_id]["status"] == "ok" assert isinstance(checks[check_id]["fixed"], bool) assert checks["mcp.registration"]["fixed"] is True + assert checks["wardline.config"]["fixed"] is True + cfg = (tmp_path / "weft.toml").read_text(encoding="utf-8") + assert 'source_roots = ["."]' in cfg + assert '"telemetry/**"' in cfg def test_doctor_reports_present_but_broken_weft_toml_as_error(tmp_path: Path, monkeypatch) -> None: @@ -125,7 +133,7 @@ def test_doctor_fix_reports_filigree_url_ok_from_env(tmp_path: Path, monkeypatch # The "upgrade commented binding when a port appears" feature was removed: doctor # no longer writes config and the filigree.url check is now ENV-ONLY (a published # port is a scan-time discovery concern, not a doctor concern). When the env var - # is set to a valid URL, the check is ok; doctor writes no config file. + # is set to a valid URL, the check is ok. home = tmp_path / "home" monkeypatch.delenv("WARDLINE_LOOMWEAVE_URL", raising=False) monkeypatch.delenv("WARDLINE_LOOMWEAVE_TOKEN", raising=False) @@ -141,7 +149,41 @@ def test_doctor_fix_reports_filigree_url_ok_from_env(tmp_path: Path, monkeypatch payload = json.loads(result.output) checks = {check["id"]: check for check in payload["checks"]} assert checks["filigree.url"]["status"] == "ok" - assert not (tmp_path / "weft.toml").exists() + assert (tmp_path / "weft.toml").exists() + + +def test_doctor_repair_creates_src_bounded_weft_toml(tmp_path: Path, monkeypatch) -> None: + home = tmp_path / "home" + monkeypatch.delenv("WARDLINE_LOOMWEAVE_URL", raising=False) + monkeypatch.delenv("WARDLINE_FILIGREE_URL", raising=False) + monkeypatch.setattr("wardline.install.mcp_json.Path.home", lambda: home) + monkeypatch.setattr("wardline.install.mcp_json._find_wardline_command", lambda: "/bin/wardline") + monkeypatch.setattr("wardline.install.detect.shutil.which", lambda _: None) + (tmp_path / "src").mkdir() + + result = CliRunner().invoke(cli, ["doctor", "--root", str(tmp_path), "--repair"]) + + assert result.exit_code == 0, result.output + cfg = (tmp_path / "weft.toml").read_text(encoding="utf-8") + assert 'source_roots = ["src"]' in cfg + assert '"telemetry/**"' in cfg + + +def test_doctor_reports_missing_weft_toml_without_repair(tmp_path: Path, monkeypatch) -> None: + home = tmp_path / "home" + monkeypatch.delenv("WARDLINE_LOOMWEAVE_URL", raising=False) + monkeypatch.delenv("WARDLINE_FILIGREE_URL", raising=False) + monkeypatch.setattr("wardline.install.mcp_json.Path.home", lambda: home) + monkeypatch.setattr("wardline.install.mcp_json._find_wardline_command", lambda: "/bin/wardline") + monkeypatch.setattr("wardline.install.detect.shutil.which", lambda _: None) + repair = CliRunner().invoke(cli, ["doctor", "--root", str(tmp_path), "--repair"]) + assert repair.exit_code == 0, repair.output + (tmp_path / "weft.toml").unlink() + + result = CliRunner().invoke(cli, ["doctor", "--root", str(tmp_path)]) + + assert result.exit_code == 1 + assert "weft.toml: missing weft.toml" in result.output def test_doctor_accepts_filigree_url_flag_and_reports_not_configured(tmp_path: Path, monkeypatch) -> None: diff --git a/tests/unit/cli/test_install.py b/tests/unit/cli/test_install.py index 64644cfd..5480c80c 100644 --- a/tests/unit/cli/test_install.py +++ b/tests/unit/cli/test_install.py @@ -17,7 +17,7 @@ def test_scan_resolves_filigree_url_from_published_port(tmp_path: Path, monkeypa captured: dict[str, object] = {} class _FakeEmitter: - def __init__(self, url: str, *, token: str | None = None) -> None: + def __init__(self, url: str, *, token: str | None = None, **_kwargs) -> None: captured["url"] = url def emit(self, findings, *, scanned_paths=()): # noqa: ANN001 @@ -211,7 +211,7 @@ def test_install_rerun_detects_filigree_when_port_appears_after_initial_install( captured: dict[str, object] = {} class _FakeEmitter: - def __init__(self, url: str, *, token: str | None = None) -> None: + def __init__(self, url: str, *, token: str | None = None, **_kwargs) -> None: captured["url"] = url def emit(self, findings, *, scanned_paths=()): # noqa: ANN001 @@ -238,7 +238,7 @@ def test_scan_threads_filigree_bearer_token_from_env(tmp_path: Path, monkeypatch captured: dict[str, object] = {} class _FakeEmitter: - def __init__(self, url: str, *, token: str | None = None) -> None: + def __init__(self, url: str, *, token: str | None = None, **_kwargs) -> None: captured["url"] = url captured["token"] = token @@ -266,7 +266,7 @@ def test_scan_threads_filigree_bearer_token_from_deprecated_env(tmp_path: Path, captured: dict[str, object] = {} class _FakeEmitter: - def __init__(self, url: str, *, token: str | None = None) -> None: + def __init__(self, url: str, *, token: str | None = None, **_kwargs) -> None: captured["url"] = url captured["token"] = token diff --git a/tests/unit/cli/test_scan_job.py b/tests/unit/cli/test_scan_job.py new file mode 100644 index 00000000..805bcef9 --- /dev/null +++ b/tests/unit/cli/test_scan_job.py @@ -0,0 +1,204 @@ +import json +import signal +import time +from datetime import UTC, datetime +from pathlib import Path + +from click.testing import CliRunner + +from wardline.cli.main import cli +from wardline.core.filigree_emit import EmitResult +from wardline.core.scan_jobs import DEFAULT_SCAN_JOB_TIMEOUT_SECONDS + +_LEAKY = ( + "from wardline.decorators import external_boundary, trusted\n" + "@external_boundary\ndef read_raw(p):\n return p\n" + "@trusted\ndef leaky(p):\n return read_raw(p)\n" +) + + +def _write(project: Path, name: str, src: str) -> Path: + path = project / name + path.write_text(src, encoding="utf-8") + return path + + +def _write_job_status(project: Path, job_id: str, payload: dict[str, object]) -> None: + job_dir = project / ".weft" / "wardline" / "jobs" / job_id + job_dir.mkdir(parents=True) + (job_dir / "status.json").write_text(json.dumps(payload), encoding="utf-8") + + +def _base_job(job_id: str) -> dict[str, object]: + now = datetime.now(UTC).isoformat().replace("+00:00", "Z") + return { + "job_id": job_id, + "status": "running", + "phase": "scanning", + "pid": 12345, + "progress": {"steps_completed": 1, "steps_total": 4}, + "created_at": now, + "updated_at": now, + "heartbeat": now, + "request": {}, + "artifacts": {}, + "failure_kind": None, + "error": None, + } + + +def test_scan_job_start_foreground_completes_and_status_is_pollable(tmp_path: Path) -> None: + project = tmp_path / "proj" + project.mkdir() + _write(project, "svc.py", "def ok():\n return 1\n") + + result = CliRunner().invoke(cli, ["scan-job", "start", str(project), "--foreground"]) + + assert result.exit_code == 0, result.output + payload = json.loads(result.output) + assert payload["status"] == "completed" + assert payload["phase"] == "complete" + assert payload["progress"]["steps_completed"] == payload["progress"]["steps_total"] == 4 + assert payload["heartbeat"] + assert Path(payload["artifacts"]["findings"]).exists() + + status = CliRunner().invoke(cli, ["scan-job", "status", payload["job_id"], "--path", str(project)]) + assert status.exit_code == 0, status.output + assert json.loads(status.output)["job_id"] == payload["job_id"] + + +def test_scan_job_gate_failure_is_terminal_gate_status(tmp_path: Path) -> None: + project = tmp_path / "proj" + project.mkdir() + _write(project, "svc.py", _LEAKY) + + result = CliRunner().invoke(cli, ["scan-job", "start", str(project), "--fail-on", "ERROR", "--foreground"]) + + assert result.exit_code == 0, result.output + payload = json.loads(result.output) + assert payload["status"] == "failed" + assert payload["failure_kind"] == "gate" + assert payload["gate"]["verdict"] == "FAILED" + assert Path(payload["artifacts"]["findings"]).exists() + + +def test_scan_job_enrichment_failure_keeps_scan_artifact(tmp_path: Path, monkeypatch) -> None: + project = tmp_path / "proj" + project.mkdir() + _write(project, "svc.py", "def ok():\n return 1\n") + + class _UnavailableEmitter: + def __init__(self, *args: object, **kwargs: object) -> None: + pass + + def emit(self, findings: object, *, scanned_paths: object = ()) -> EmitResult: + return EmitResult(reachable=False, url="http://x/api/weft/scan-results") + + monkeypatch.setattr("wardline.core.scan_jobs.FiligreeEmitter", _UnavailableEmitter) + + result = CliRunner().invoke( + cli, + [ + "scan-job", + "start", + str(project), + "--filigree-url", + "http://x/api/weft/scan-results", + "--foreground", + ], + ) + + assert result.exit_code == 0, result.output + payload = json.loads(result.output) + assert payload["status"] == "completed_with_enrichment_failure" + assert payload["failure_kind"] == "enrichment" + assert payload["filigree_emit"]["disabled_reason"] == "filigree unreachable at http://x/api/weft/scan-results" + assert Path(payload["artifacts"]["findings"]).exists() + + +def test_scan_job_status_marks_dead_worker_failed(tmp_path: Path, monkeypatch) -> None: + job_id = "a" * 32 + project = tmp_path / "proj" + project.mkdir() + _write_job_status(project, job_id, _base_job(job_id)) + monkeypatch.setattr("wardline.core.scan_jobs._pid_alive", lambda pid: False) + + result = CliRunner().invoke(cli, ["scan-job", "status", job_id, "--path", str(project)]) + + assert result.exit_code == 0, result.output + payload = json.loads(result.output) + assert payload["status"] == "failed" + assert payload["failure_kind"] == "stale_worker" + assert "12345" in payload["error"] + + persisted = json.loads((project / ".weft" / "wardline" / "jobs" / job_id / "status.json").read_text()) + assert persisted["status"] == "failed" + assert persisted["failure_kind"] == "stale_worker" + + +def test_scan_job_cancel_signals_process_group_and_persists_terminal_status(tmp_path: Path, monkeypatch) -> None: + job_id = "b" * 32 + project = tmp_path / "proj" + project.mkdir() + _write_job_status(project, job_id, _base_job(job_id)) + sent: list[tuple[int, int]] = [] + monkeypatch.setattr("wardline.core.scan_jobs._pid_alive", lambda pid: True) + monkeypatch.setattr("wardline.core.scan_jobs.os.killpg", lambda pid, sig: sent.append((pid, sig))) + + result = CliRunner().invoke(cli, ["scan-job", "cancel", job_id, "--path", str(project)]) + + assert result.exit_code == 0, result.output + payload = json.loads(result.output) + assert payload["status"] == "cancelled" + assert payload["phase"] == "cancelled" + assert payload["failure_kind"] == "cancelled" + assert sent == [(12345, signal.SIGTERM)] + + persisted = json.loads((project / ".weft" / "wardline" / "jobs" / job_id / "status.json").read_text()) + assert persisted["status"] == "cancelled" + + +def test_scan_job_timeout_is_terminal_timeout_status(tmp_path: Path, monkeypatch) -> None: + project = tmp_path / "proj" + project.mkdir() + _write(project, "svc.py", "def ok():\n return 1\n") + + def slow_scan(*args: object, **kwargs: object) -> object: + time.sleep(0.2) + raise AssertionError("timeout should interrupt run_scan before it returns") + + monkeypatch.setattr("wardline.core.scan_jobs.run_scan", slow_scan) + + result = CliRunner().invoke(cli, ["scan-job", "start", str(project), "--timeout", "0.01", "--foreground"]) + + assert result.exit_code == 0, result.output + payload = json.loads(result.output) + assert payload["status"] == "failed" + assert payload["failure_kind"] == "timeout" + assert "timed out" in payload["error"] + + +def test_scan_job_start_applies_default_timeout(tmp_path: Path) -> None: + project = tmp_path / "proj" + project.mkdir() + _write(project, "svc.py", "def ok():\n return 1\n") + + result = CliRunner().invoke(cli, ["scan-job", "start", str(project), "--foreground"]) + + assert result.exit_code == 0, result.output + payload = json.loads(result.output) + assert payload["status"] == "completed" + assert payload["request"]["timeout_seconds"] == DEFAULT_SCAN_JOB_TIMEOUT_SECONDS + + +def test_scan_job_start_allows_timeout_opt_out(tmp_path: Path) -> None: + project = tmp_path / "proj" + project.mkdir() + _write(project, "svc.py", "def ok():\n return 1\n") + + result = CliRunner().invoke(cli, ["scan-job", "start", str(project), "--timeout", "0", "--foreground"]) + + assert result.exit_code == 0, result.output + payload = json.loads(result.output) + assert payload["status"] == "completed" + assert payload["request"]["timeout_seconds"] == 0.0 diff --git a/tests/unit/core/test_discovery.py b/tests/unit/core/test_discovery.py index 97cf7aa0..75aac4a6 100644 --- a/tests/unit/core/test_discovery.py +++ b/tests/unit/core/test_discovery.py @@ -19,6 +19,18 @@ def test_respects_exclude_globs() -> None: assert all(p.name != "mod.py" for p in files) +def test_prunes_tool_cache_directories_before_discovery(tmp_path: Path) -> None: + (tmp_path / "src").mkdir() + (tmp_path / "src" / "app.py").write_text("x = 1\n", encoding="utf-8") + cache = tmp_path / ".uv-cache" / "archive" / "pkg" + cache.mkdir(parents=True) + (cache / "cached.py").write_text("y = 2\n", encoding="utf-8") + + files = discover(tmp_path, WardlineConfig(source_roots=(".",))) + + assert [p.relative_to(tmp_path).as_posix() for p in files] == ["src/app.py"] + + def test_skip_dirs_are_relative_to_source_root_not_absolute_parents(tmp_path: Path) -> None: root = tmp_path / ".venv" / "proj" root.mkdir(parents=True) diff --git a/tests/unit/core/test_filigree_emit.py b/tests/unit/core/test_filigree_emit.py index 804ea689..3cfb3e94 100644 --- a/tests/unit/core/test_filigree_emit.py +++ b/tests/unit/core/test_filigree_emit.py @@ -173,6 +173,121 @@ def test_http_400_raises_filigree_emit_error() -> None: assert '{"error":"bad path key"}' in str(exc.value) # verbatim response body echoed +def test_http_400_can_degrade_to_warning_result() -> None: + t = _FakeTransport(response=Response(status=400, body='{"error":"payload too large"}')) + res = FiligreeEmitter("http://x", transport=t, protocol_errors_loud=False).emit([_f()]) + + assert res.reachable is True + assert res.failed == 1 + assert res.warnings + assert "payload too large" in res.warnings[0] + + +def test_large_emit_chunks_by_finding_cap() -> None: + class _ChunkTransport: + def __init__(self) -> None: + self.calls: list[dict[str, object]] = [] + + def post(self, url: str, body: bytes, headers: dict[str, str]) -> Response: # noqa: ARG002 + self.calls.append(json.loads(body.decode("utf-8"))) + return Response(status=200, body=_ok_body()) + + t = _ChunkTransport() + findings = [_f(location=Location(path=f"src/{name}.py", line_start=1)) for name in ("a", "b", "c")] + + res = FiligreeEmitter("http://x", transport=t, max_findings_per_request=2).emit( + findings, + scanned_paths=("src/a.py", "src/b.py", "src/c.py", "src/clean.py"), + ) + + assert res.reachable is True + assert len(t.calls) == 2 + assert [len(call["findings"]) for call in t.calls] == [2, 1] + assert all(call["mark_unseen"] is True for call in t.calls) + assert t.calls[-1]["scanned_paths"] == ["src/c.py", "src/clean.py"] + + +def test_schema_advertised_limit_drives_chunking_when_no_override() -> None: + class _SchemaTransport: + def __init__(self) -> None: + self.gets: list[tuple[str, dict[str, str]]] = [] + self.posts: list[dict[str, object]] = [] + + def get(self, url: str, headers: dict[str, str]) -> Response: + self.gets.append((url, dict(headers))) + return Response( + status=200, + body=json.dumps( + { + "endpoints": { + "POST /api/scan-results": { + "limits": {"max_findings_per_request": 2}, + } + } + } + ), + ) + + def post(self, url: str, body: bytes, headers: dict[str, str]) -> Response: # noqa: ARG002 + self.posts.append(json.loads(body.decode("utf-8"))) + return Response(status=200, body=_ok_body()) + + t = _SchemaTransport() + findings = [_f(location=Location(path=f"src/{name}.py", line_start=1)) for name in ("a", "b", "c")] + + FiligreeEmitter("http://x/api/p/demo/weft/scan-results", transport=t).emit(findings) + + assert t.gets == [("http://x/api/p/demo/files/_schema", {"Content-Type": "application/json"})] + assert [len(call["findings"]) for call in t.posts] == [2, 1] + + +def test_explicit_limit_takes_precedence_over_schema_limit() -> None: + class _SchemaTransport: + def __init__(self) -> None: + self.get_called = False + self.posts: list[dict[str, object]] = [] + + def get(self, url: str, headers: dict[str, str]) -> Response: # noqa: ARG002 + self.get_called = True + return Response(status=200, body=json.dumps({"scan_results": {"max_findings_per_request": 1}})) + + def post(self, url: str, body: bytes, headers: dict[str, str]) -> Response: # noqa: ARG002 + self.posts.append(json.loads(body.decode("utf-8"))) + return Response(status=200, body=_ok_body()) + + t = _SchemaTransport() + findings = [_f(location=Location(path=f"src/{name}.py", line_start=1)) for name in ("a", "b", "c")] + + FiligreeEmitter("http://x/api/weft/scan-results", transport=t, max_findings_per_request=3).emit(findings) + + assert t.get_called is False + assert [len(call["findings"]) for call in t.posts] == [3] + + +def test_oversize_single_file_chunk_disables_mark_unseen() -> None: + class _ChunkTransport: + def __init__(self) -> None: + self.calls: list[dict[str, object]] = [] + + def post(self, url: str, body: bytes, headers: dict[str, str]) -> Response: # noqa: ARG002 + self.calls.append(json.loads(body.decode("utf-8"))) + return Response(status=200, body=_ok_body()) + + t = _ChunkTransport() + findings = [ + _f(location=Location(path="src/a.py", line_start=1), fingerprint="b" * 64), + _f(location=Location(path="src/a.py", line_start=2), fingerprint="c" * 64), + ] + + FiligreeEmitter("http://x", transport=t, max_findings_per_request=1).emit( + findings, + scanned_paths=("src/a.py",), + ) + + assert len(t.calls) == 2 + assert all(call["mark_unseen"] is False for call in t.calls) + + def test_http_5xx_is_sibling_degraded_not_loud() -> None: # A server outage (503) is the sibling's fault, not a Wardline payload bug: # warn + continue (reachable=False), never exit-2. (Charter: non-load-bearing.) diff --git a/tests/unit/core/test_finding.py b/tests/unit/core/test_finding.py index cbb9c695..1f54e1d3 100644 --- a/tests/unit/core/test_finding.py +++ b/tests/unit/core/test_finding.py @@ -7,6 +7,7 @@ Location, Severity, compute_finding_fingerprint, + to_filigree_metadata, ) @@ -110,3 +111,11 @@ def test_filigree_metadata_includes_suppression_only_when_suppressed() -> None: waived = to_filigree_metadata(_finding(suppressed=SuppressionState.WAIVED, suppression_reason="ok"))["wardline"] assert waived["suppression_state"] == "waived" assert waived["suppression_reason"] == "ok" + + +def test_filigree_metadata_strips_property_accessor_suffix_from_wire_qualname() -> None: + setter = to_filigree_metadata(_finding(qualname="pkg.mod.C.value:setter"))["wardline"] + deleter = to_filigree_metadata(_finding(qualname="pkg.mod.C.value:deleter"))["wardline"] + + assert setter["qualname"] == "pkg.mod.C.value" + assert deleter["qualname"] == "pkg.mod.C.value" diff --git a/tests/unit/core/test_run.py b/tests/unit/core/test_run.py index b943323d..55d2f107 100644 --- a/tests/unit/core/test_run.py +++ b/tests/unit/core/test_run.py @@ -49,6 +49,19 @@ def test_run_scan_returns_findings_summary_and_context() -> None: assert result.context is not None +def test_run_scan_reports_discovery_and_analysis_progress(tmp_path: Path) -> None: + (tmp_path / "svc.py").write_text("def ok():\n return 1\n", encoding="utf-8") + events: list[dict[str, object]] = [] + + result = run_scan(tmp_path, progress_callback=events.append) + + assert result.files_scanned == 1 + assert [event["phase"] for event in events] == ["discovered", "analyzing", "analyzed"] + assert events[0]["files_discovered"] == 1 + assert events[-1]["files_analyzed"] == 1 + assert "findings" in events[-1] + + def test_gate_decision_trips_on_active_error(tmp_path: Path) -> None: proj = tmp_path / "proj" proj.mkdir() diff --git a/tests/unit/install/test_block.py b/tests/unit/install/test_block.py index 925be143..a4b8706d 100644 --- a/tests/unit/install/test_block.py +++ b/tests/unit/install/test_block.py @@ -18,6 +18,7 @@ def test_render_block_is_fenced_and_mentions_the_gate() -> None: block = render_block() assert block.startswith("") + assert "" in block.splitlines()[1] assert "wardline scan" in block assert "wardline-gate" in block diff --git a/tests/unit/mcp/test_server_scan_jobs.py b/tests/unit/mcp/test_server_scan_jobs.py new file mode 100644 index 00000000..3427cd11 --- /dev/null +++ b/tests/unit/mcp/test_server_scan_jobs.py @@ -0,0 +1,147 @@ +import json +from pathlib import Path +from typing import Any + +import wardline.mcp.server as server_mod +from wardline.mcp.server import WardlineMCPServer + + +def _tool_call(server: WardlineMCPServer, name: str, arguments: dict[str, Any] | None = None) -> dict[str, Any]: + resp = server.rpc.dispatch( + { + "jsonrpc": "2.0", + "id": 1, + "method": "tools/call", + "params": {"name": name, "arguments": arguments or {}}, + } + ) + assert "error" not in resp, resp + result = resp["result"] + if result.get("isError"): + return result + return json.loads(result["content"][0]["text"]) + + +def _status(job_id: str = "a" * 32, status: str = "running") -> dict[str, Any]: + return { + "job_id": job_id, + "status": status, + "phase": "scanning", + "progress": {"steps_completed": 1, "steps_total": 4}, + "heartbeat": "2026-06-13T00:00:00Z", + "request": {}, + "artifacts": {}, + "failure_kind": None, + "error": None, + } + + +def test_scan_job_tools_are_advertised_with_capabilities(tmp_path: Path) -> None: + server = WardlineMCPServer(root=tmp_path) + resp = server.rpc.dispatch({"jsonrpc": "2.0", "id": 1, "method": "tools/list", "params": {}}) + tools = {tool["name"]: tool for tool in resp["result"]["tools"]} + + assert {"scan_job_start", "scan_job_status", "scan_job_cancel"} <= set(tools) + assert {"read", "write"} <= set(tools["scan_job_start"]["capabilities"]) + assert tools["scan_job_status"]["capabilities"] == ["read"] + assert {"read", "write"} <= set(tools["scan_job_cancel"]["capabilities"]) + assert tools["scan_job_start"]["outputSchema"]["type"] == "object" + + +def test_scan_job_start_threads_request_to_core(tmp_path: Path, monkeypatch) -> None: + calls: list[tuple[Path, dict[str, Any], bool]] = [] + + def fake_start(root: Path, request: dict[str, Any], *, foreground: bool = False) -> dict[str, Any]: + calls.append((root, request, foreground)) + return _status() + + monkeypatch.setattr(server_mod, "start_scan_job", fake_start) + server = WardlineMCPServer(root=tmp_path, filigree_url="http://filigree.local/api/weft/scan-results") + + out = _tool_call( + server, + "scan_job_start", + { + "format": "agent-summary", + "fail_on": "ERROR", + "fail_on_unanalyzed": True, + "timeout_seconds": 12.5, + "lang": "python", + "trust_packs": ["org.pack"], + }, + ) + + assert out["job_id"] == "a" * 32 + assert calls == [ + ( + tmp_path, + { + "config": None, + "format": "agent-summary", + "output": None, + "fail_on": "ERROR", + "fail_on_unanalyzed": True, + "cache_dir": None, + "filigree_url": "http://filigree.local/api/weft/scan-results", + "local_only": False, + "filigree_max_findings_per_request": None, + "timeout_seconds": 12.5, + "lang": "python", + "new_since": None, + "trusted_packs": ["org.pack"], + "trust_local_packs": False, + "strict_defaults": False, + "trust_suppressions": False, + }, + False, + ) + ] + + +def test_scan_job_status_and_cancel_call_core(tmp_path: Path, monkeypatch) -> None: + seen: list[tuple[str, Path, str]] = [] + + def fake_status(root: Path, job_id: str) -> dict[str, Any]: + seen.append(("status", root, job_id)) + return _status(job_id=job_id) + + def fake_cancel(root: Path, job_id: str) -> dict[str, Any]: + seen.append(("cancel", root, job_id)) + return _status(job_id=job_id, status="cancelled") + + monkeypatch.setattr(server_mod, "read_scan_job_status", fake_status) + monkeypatch.setattr(server_mod, "cancel_scan_job", fake_cancel) + server = WardlineMCPServer(root=tmp_path) + + status = _tool_call(server, "scan_job_status", {"job_id": "b" * 32}) + cancel = _tool_call(server, "scan_job_cancel", {"job_id": "b" * 32}) + + assert status["status"] == "running" + assert cancel["status"] == "cancelled" + assert seen == [("status", tmp_path, "b" * 32), ("cancel", tmp_path, "b" * 32)] + + +def test_scan_job_start_respects_write_and_network_policy(tmp_path: Path, monkeypatch) -> None: + monkeypatch.setenv("WARDLINE_FILIGREE_URL", "http://filigree.local/api/weft/scan-results") + called = False + + def fake_start(root: Path, request: dict[str, Any], *, foreground: bool = False) -> dict[str, Any]: + nonlocal called + called = True + return _status() + + monkeypatch.setattr(server_mod, "start_scan_job", fake_start) + + no_write = WardlineMCPServer(root=tmp_path, allow_write=False) + write_denied = _tool_call(no_write, "scan_job_start") + assert write_denied["isError"] is True + assert "write" in write_denied["content"][0]["text"].lower() + + no_network = WardlineMCPServer(root=tmp_path, allow_network=False) + network_denied = _tool_call(no_network, "scan_job_start") + assert network_denied["isError"] is True + assert "network" in network_denied["content"][0]["text"].lower() + + local_only = _tool_call(no_network, "scan_job_start", {"local_only": True}) + assert local_only["status"] == "running" + assert called is True diff --git a/tests/unit/mcp/test_server_structure.py b/tests/unit/mcp/test_server_structure.py index 0e4bb4f3..a5a0cc7e 100644 --- a/tests/unit/mcp/test_server_structure.py +++ b/tests/unit/mcp/test_server_structure.py @@ -25,6 +25,9 @@ def test_mcp_advertisement_snapshot() -> None: assert [tool["name"] for tool in tools["result"]["tools"]] == [ "scan", + "scan_job_start", + "scan_job_status", + "scan_job_cancel", "explain_taint", "dossier", "assure", diff --git a/tests/unit/rust/test_analyzer_protocol.py b/tests/unit/rust/test_analyzer_protocol.py index 0ddce22d..5faaf09e 100644 --- a/tests/unit/rust/test_analyzer_protocol.py +++ b/tests/unit/rust/test_analyzer_protocol.py @@ -7,9 +7,10 @@ * ``last_context`` returns the *Python-shaped* ``AnalysisContext | None`` (None in slice-1 — the Rust-native context is incompatible and would crash the delta/SARIF consumers; it lives on the separate ``last_rust_context`` accessor); and -* a file tree-sitter cannot fully parse yields a ``WLN-ENGINE-PARSE-ERROR`` FACT and - contributes NO ``RS-WL-*`` findings (never half-analyze a file), mirroring the - Python pipeline's parse-error policy so it counts toward ``ScanSummary.unanalyzed``. +* a file tree-sitter cannot fully parse yields a gate-eligible + ``WLN-ENGINE-PARSE-ERROR`` defect and contributes NO ``RS-WL-*`` findings (never + half-analyze a file), mirroring the Python pipeline's parse-error policy so it + counts toward ``ScanSummary.unanalyzed``. """ from __future__ import annotations @@ -62,7 +63,7 @@ def test_last_context_is_none_but_rust_context_is_retained(tmp_path) -> None: assert analyzer.last_rust_context.triggers -def test_unparseable_file_emits_parse_error_fact_and_no_rs_findings(tmp_path) -> None: +def test_unparseable_file_emits_parse_error_defect_and_no_rs_findings(tmp_path) -> None: # A truncated fn: tree-sitter recovers a partial tree (root_node.has_error). We must # surface the diagnostic and NOT report findings over a half-parsed file. (tmp_path / "broken.rs").write_text("fn f( {\n let t = std::env::var(\n", encoding="utf-8") @@ -71,7 +72,9 @@ def test_unparseable_file_emits_parse_error_fact_and_no_rs_findings(tmp_path) -> parse_errors = [f for f in findings if f.rule_id == "WLN-ENGINE-PARSE-ERROR"] assert len(parse_errors) == 1 assert parse_errors[0].location.path == "broken.rs" - assert parse_errors[0].severity.value == "NONE" + assert parse_errors[0].location.line_start == 1 + assert parse_errors[0].severity.value == "ERROR" + assert parse_errors[0].kind.value == "defect" assert all(not f.rule_id.startswith("RS-WL-") for f in findings) @@ -170,8 +173,8 @@ def test_multiple_files_accumulate_findings(tmp_path) -> None: def test_one_crashing_file_is_isolated_and_does_not_lose_other_findings(tmp_path) -> None: # A clean-parsing but pathologically deep expression overflows the recursive dataflow # walk (RecursionError). Per-file isolation must degrade THAT file to a counted - # WLN-ENGINE-FILE-FAILED FACT and still emit the OTHER file's real RS-WL-108 — never - # abort the whole scan (the engine's per-function isolation, mirrored per-file). + # WLN-ENGINE-FILE-FAILED defect and still emit the OTHER file's real RS-WL-108 — + # never abort the whole scan (the engine's per-function isolation, mirrored per-file). deep_expr = "+".join(["x"] * 6000) # nested binary_expression depth >> default recursionlimit deep = tmp_path / "deep.rs" deep.write_text( @@ -186,7 +189,9 @@ def test_one_crashing_file_is_isolated_and_does_not_lose_other_findings(tmp_path file_failed = [f for f in findings if f.rule_id == "WLN-ENGINE-FILE-FAILED"] assert len(file_failed) == 1 and file_failed[0].location.path == "deep.rs" - assert file_failed[0].severity.value == "NONE" + assert file_failed[0].location.line_start == 1 + assert file_failed[0].severity.value == "ERROR" + assert file_failed[0].kind.value == "defect" # The other file's real finding survived the neighbour's crash. survivors = [f for f in findings if f.rule_id == "RS-WL-108"] assert len(survivors) == 1 and survivors[0].location.path == "inject.rs" From 5781794e818583172e9024042955203cf5f87382 Mon Sep 17 00:00:00 2001 From: John Morrissey Date: Sat, 13 Jun 2026 13:14:30 +1000 Subject: [PATCH 163/186] docs: refresh README/docs/site for the RC5 surface; changelog scan-jobs + emit cap - README, docs index, CLI reference, Rust/Weft/assurance guides and the mkdocs theme refreshed against the 2026-05-29..2026-06-12 release-candidate surface. - www landing page + styles updated to the current product shape. - CHANGELOG: record pollable scan jobs (18-tool MCP surface), --filigree-max-findings-per-request capping and --local-only/--no-emit fencing, alongside the install/attestation/scan-correctness/federation hardening entries. - finding-lifecycle glossary code-anchor citations re-pinned to current line numbers (kept in step with the test_glossary_vocabulary lock). Co-Authored-By: Claude Opus 4.8 (1M context) --- CHANGELOG.md | 38 ++++++ README.md | 88 ++++++++---- docs/guides/rust-preview.md | 20 ++- docs/guides/weft.md | 17 ++- docs/index.md | 35 +++-- docs/reference/cli.md | 31 +++++ docs/stylesheets/extra.css | 134 +++++++++--------- docs/stylesheets/weft-mkdocs.css | 226 ++++++++++++++++--------------- mkdocs.yml | 2 +- www/index.html | 141 ++++++++++++++++--- www/styles.css | 155 +++++++++++++++++++-- 11 files changed, 631 insertions(+), 256 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 454bf333..89267284 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -8,6 +8,25 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] ### Added +- **Pollable file-backed scan jobs.** `wardline scan-job start|status|cancel` + (CLI) and the matching `scan_job_start` / `scan_job_status` / `scan_job_cancel` + MCP tools run a long scan in a daemon-free worker subprocess that persists status + JSON under `.weft/wardline/jobs/`, so an agent can start a slow scan and poll its + heartbeat/progress instead of blocking a single MCP call. Status reads refresh + liveness (dead-worker / stale-heartbeat) so a hung job is never ambiguous. The MCP + surface is now 18 tools. +- **Filigree-emit capping and local-only fencing.** `scan` gains + `--filigree-max-findings-per-request` (env `WARDLINE_FILIGREE_MAX_FINDINGS_PER_REQUEST`, + default 1000) to bound per-POST payloads, and `--local-only`/`--no-emit` to disable + sibling emission even when URLs resolve from flags/env/install state. Emission stays + ENRICH-ONLY — a sibling's absence never breaks the core scan or gate. +- **Top-level documentation refreshed against the 2026-05-29..2026-06-12 + release-candidate surface.** The README now describes the current product shape: + Python remains the full taint-analysis frontend; Rust is a command-injection + preview behind `wardline[rust]`; configuration/state live in `weft.toml` and + `.weft/wardline/`; the agent/MCP surface includes doctor, rekey, assurance, + attestation, dossier, and finding-lifecycle tools; and the docs index points to + the Rust, Weft, and assurance guides. - **MCP structured tool output on all 15 tools** — every tool now declares an `outputSchema` in `tools/list` and returns `structuredContent` alongside the (byte-identical) text block on `tools/call`, so MCP-spec clients consume and @@ -29,6 +48,25 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 enforcement authority (`wardline-e63204176b`, MCP-primary B2). ### Fixed +- **Install, attestation, and local automation hardening from the 2026-06-12 + security pass.** `wardline install`/doctor paths now refuse unsafe writes through + symlinked published-port files, refuse to mint attestation keys into tracked + environments, preserve loopback-scope handling without needless churn, and keep + operator-pinned sibling URLs intact. Attestation capture disables repository + `fsmonitor` while reading git state and fails stale reproduction instead of + accepting mismatched evidence. The repo `make clean` target refuses symlinked + cleanup targets, and CI pins the `setup-uv` action. +- **Scan/reporting correctness hardening from the latest review pass.** Grammar + fingerprints now include seed dependencies, return-taint resolution uses the + statement snapshot for the return being analyzed, recursive lambda taint + resolution is guarded, `assure` counts unanalyzed files in coverage, and + `scan-file-findings` honors the gate exit status. Default agent-summary output + is guarded from oversized or misleading responses. +- **Filigree/Loomweave federation safety fixes.** Filigree token resolution now + prioritizes env aliases over dotenv fallback, mark-unseen lifecycle emission is + disabled for unanalyzed scans, strict MCP SEI filters preserve their defaults, + and signed scan artifacts include scan scope so a receiver can distinguish + evidence from different roots. - **Non-string `issue_id` from Filigree promote is normalized to `null`** — the promote response is type-narrowed at the wire boundary, so a skewed 2xx body can no longer leak a non-string `issue_id` into `file_finding` / diff --git a/README.md b/README.md index 375911c2..96cd4d10 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,8 @@ # Wardline -Generic, lightweight semantic-tainting static analyzer for Python — track untrusted data across your codebase and gate trust-boundary violations, with zero runtime dependencies. +Generic, lightweight semantic-tainting static analyzer for trust boundaries. Wardline's full analyzer targets +Python; its Rust preview catches command-injection defects over crate-aware identity. The base package has zero +runtime dependencies, and scanner/front-end functionality stays behind opt-in extras. [![CI](https://github.com/foundryside-dev/wardline/actions/workflows/ci.yml/badge.svg)](https://github.com/foundryside-dev/wardline/actions/workflows/ci.yml) [![PyPI](https://img.shields.io/pypi/v/wardline)](https://pypi.org/project/wardline/) @@ -39,11 +41,13 @@ engine facts): ## What is Wardline? -Wardline reads your Python statically — it never runs your code — and asks one -question of every trust-annotated function: **is the data this function works -with as trusted as it claims?** It tracks a *taint* (a trust level) for every -value and propagates it across the whole project, flagging the places where -untrusted data reaches a trusted producer with no validation in between. +Wardline reads your code statically — it never runs it — and asks one question +of every trust-annotated boundary: **is the data this function works with as +trusted as it claims?** For Python, it tracks a *taint* (a trust level) for +values through function bodies and the project call graph, flagging places where +untrusted data reaches a trusted producer with no validation in between. For +Rust, the preview frontend currently focuses on command-injection sinks around +`std::process::Command`. Wardline is part of **Weft** — an agent-first suite of small, local-first developer tools, each driven by a coding agent as much as a person, giving small @@ -59,21 +63,34 @@ lets it scan a large untouched codebase (including its own) with zero noise. ## Key Features -- **Deterministic whole-program taint** — function-, variable-, and project-level - analysis over an inter-module call graph; no runtime instrumentation. -- **Opt-in trust model** — three decorators (`@external_boundary`, - `@trust_boundary`, `@trusted`) mark your boundaries; the engine infers the rest. -- **Four policy rules** — untrusted-reaches-trusted, non-rejecting boundary, - broad exception handler, and silently-swallowed exception. -- **Zero-dependency base** — `pip install wardline` pulls nothing; functionality - lives behind small extras. -- **Structured output** — JSONL, SARIF (generic interchange/GitHub - code-scanning), and native Filigree emit for finding lifecycle work. -- **Agent-native** — `wardline mcp` is a dependency-free MCP-over-stdio server; - `wardline install` wires Wardline into your coding agent in one command. +- **Deterministic Python taint analysis** — function-, variable-, and + project-level analysis over an inter-module call graph; no runtime + instrumentation. +- **Rust command-injection preview** — `wardline scan --lang rust` finds + `RS-WL-108`/`RS-WL-112` over `.rs` trees with crate-prefixed, baseline-eligible + finding identity. +- **Opt-in trust model** — Python decorators (`@external_boundary`, + `@trust_boundary`, `@trusted`) and Rust doc-comment markers declare the + boundary surface; undecorated code stays quiet. +- **Trust-boundary and sink rules** — boundary-integrity rules, exception-flow + rules, and expanded sink families for command execution, dynamic code/imports, + deserialization, path traversal, SSRF, SQL injection, XML parsing, templates, + native library loads, logging format strings, and SMTP sends. +- **Zero-dependency base** — `pip install wardline` pulls nothing; scanner, + Loomweave, Rust, and docs functionality live behind small extras. +- **Structured output** — JSONL, SARIF, agent-summary JSON, signed legis scan + artifacts, and native Filigree emission for finding lifecycle work. +- **MCP-primary agent surface** — `wardline mcp` is a dependency-free + MCP-over-stdio server with structured tool output, schema declarations, and + tools for scan, explain, fix, judge, doctor, rekey, assurance, attestation, + dossier, and finding lifecycle work. +- **Reproducible evidence and migrations** — `assure`, `attest`, and `rekey` + report trust-surface coverage, sign reproducible posture bundles, and migrate + fingerprint-keyed stores across scheme changes. - **Opt-in LLM triage** — `wardline judge` labels findings TRUE/FALSE positive (dependency-free; never runs automatically). -- **Light-touch suppression** — baselines and time-boxed, reasoned waivers. +- **Light-touch suppression** — baselines, time-boxed waivers, and judged + findings with explicit gate semantics. - **Loomweave integration** — persist per-entity taint facts to a Loomweave store. ## Quick Start @@ -107,6 +124,7 @@ Fix findings at the **boundary** (validate before returning), not at the sink. pip install weft-markers # tiny runtime marker package for application code pip install wardline # zero-dependency base (library + decorators) pip install 'wardline[scanner]' # the scan/judge/baseline CLI + MCP server (quote for zsh) +pip install 'wardline[rust]' # Rust command-injection preview frontend ``` Prefer `weft_markers` in application code. Wardline still recognizes @@ -117,6 +135,7 @@ Prefer `weft_markers` in application code. Wardline still recognizes |-------|-------|---------| | `scanner` | pyyaml, jsonschema, click | the `wardline` CLI and `wardline mcp` server | | `loomweave` | blake3 | persisting taint facts to a Loomweave store | +| `rust` | scanner extra, tree-sitter, tree-sitter-rust | `wardline scan --lang rust` | | `docs` | mkdocs, mkdocs-material | building the documentation site | The LLM triage judge (`wardline judge`) is dependency-free (stdlib `urllib` → @@ -130,12 +149,11 @@ wardline install This injects a hash-fenced instruction block into `CLAUDE.md`/`AGENTS.md`, installs the `wardline-gate` skill, merges a `wardline` entry into `.mcp.json`, -writes Codex's `~/.codex/config.toml` MCP entry, and records Loomweave/Filigree -bindings if present. When local sibling config exposes a URL, `install` wires it -directly; otherwise it leaves a commented stanza to fill in. Agents then run the -scan → explain → fix-at-boundary → rescan loop natively. The `wardline mcp` -server exposes `scan`, `explain_taint`, `fix`, `judge`, baseline, and waiver -tools over JSON-RPC with no SDK. +and writes Codex's `~/.codex/config.toml` MCP entry. Agents then run the scan → +explain → fix-at-boundary → rescan loop natively. The `wardline mcp` server +exposes the primary tool surface over JSON-RPC with no SDK, including scan, +filtered findings, explain-taint, fix, judge, baseline/waiver, doctor, rekey, +assure, attest, dossier, and Filigree filing tools. `wardline install` also reminds application projects to install `weft-markers` and import from `weft_markers` when they want runtime-importable trust markers @@ -145,6 +163,17 @@ Run `wardline doctor` to check those artifacts later, or `wardline doctor --repair` to refresh stale/missing wiring after moving tools or starting a Filigree dashboard. +## Configuration and state + +Wardline reads operator configuration from the `[wardline]` table in +`weft.toml`. Machine-written state lives under `.weft/wardline/`: baselines, +waivers, judged findings, and cache data stay out of the authored config file. + +Sibling URLs are resolved at runtime from flags, environment variables, or +published local Weft port files. `wardline install` and `wardline doctor` detect +sibling tools such as Filigree and Loomweave, but they do not persist endpoint +bindings into project config. + ## Where Wardline fits Use Wardline when you want a deterministic, opt-in trust-boundary gate you can @@ -158,9 +187,11 @@ It is **not** the right tool when you need: - **A broad SAST suite.** Wardline checks trust boundaries and a small set of exception-handling rules; it is not a replacement for a general-purpose scanner that covers dozens of vulnerability classes. -- **Non-Python code.** Wardline analyzes Python ≥3.12 only. +- **Full non-Python coverage.** Wardline's Rust frontend is a preview for + command-injection findings only; it is not a general Rust SAST engine. - **Zero-config coverage.** Wardline is silent until you declare trust — that is - the point, but it means it finds nothing on an un-annotated codebase. + the point, but it means it finds nothing meaningful on an un-annotated + codebase. ## Documentation @@ -174,6 +205,9 @@ Full documentation lives at ****. | [Configuration](https://foundryside-dev.github.io/wardline/guides/configuration/) | `weft.toml` `[wardline]`: rules, severity, excludes | | [Suppression](https://foundryside-dev.github.io/wardline/guides/suppression/) | Baselines and waivers | | [LLM Triage Judge](https://foundryside-dev.github.io/wardline/guides/judge/) | Opt-in TRUE/FALSE-positive labelling | +| [Rust Support](https://foundryside-dev.github.io/wardline/guides/rust-preview/) | Preview Rust command-injection frontend | +| [Weft Integration](https://foundryside-dev.github.io/wardline/guides/weft/) | SARIF, Filigree, Loomweave, and sibling URL resolution | +| [Assurance Posture](https://foundryside-dev.github.io/wardline/guides/assurance-posture/) | Coverage posture, attestations, and trust-surface evidence | | [Loomweave Taint Store](https://foundryside-dev.github.io/wardline/guides/loomweave-taint-store/) | Persisting taint facts | | [CLI Reference](https://foundryside-dev.github.io/wardline/reference/cli/) | Every command and flag | | [Trust Vocabulary](https://foundryside-dev.github.io/wardline/reference/vocabulary/) | The decorators and their arguments | diff --git a/docs/guides/rust-preview.md b/docs/guides/rust-preview.md index 6f540d60..f4c1557c 100644 --- a/docs/guides/rust-preview.md +++ b/docs/guides/rust-preview.md @@ -12,6 +12,14 @@ data reaching the program or shell command line of `std::process::Command`. suppression machinery — they match committed baseline/waiver/judged entries and are captured by `wardline baseline` like any Python finding. +!!! note "Weft freeze-set posture" + The Rust qualname dialect is governed by Loomweave ADR-049 and the vendored + `qualnames_rust.json` corpus. It is **out of the clean-break core API freeze + set** and versions with the Rust feature, even though Weft PDR-0014 gates the + launch cutover on Rust reaching gold. Wardline must adopt ADR-049 amendments + in lockstep with Loomweave by re-vendoring the shared corpus; it must not + independently reinterpret the dialect. + Finding identity is **keyed to the crate name**: the qualname every fingerprint hashes starts with the crate read from `Cargo.toml` (`[package].name`, underscored). Adding or removing a `Cargo.toml`, or renaming the crate in the @@ -140,9 +148,9 @@ slice, documented so you do not mistake silence for safety: mechanically by file path — a known gap shared with the Loomweave extractor. A `.rs` file that tree-sitter cannot fully parse is **not** half-analyzed: it is -surfaced as a `WLN-ENGINE-PARSE-ERROR` fact, counts toward the "could not be -analyzed" total, and gates under `--fail-on-unanalyzed` — never reported as a -clean result. Likewise, a single file whose analysis fails (for example a -pathologically deep expression that overflows the dataflow walk) is isolated to a -`WLN-ENGINE-FILE-FAILED` fact and counted as under-scanned; it never aborts the -run or loses the other files' findings. +surfaced as a gate-eligible `WLN-ENGINE-PARSE-ERROR` ERROR defect, counts toward +the "could not be analyzed" total, and trips the default `--fail-on ERROR` gate — +never reported as a clean result. Likewise, a single file whose analysis fails +(for example a pathologically deep expression that overflows the dataflow walk) +is isolated to a gate-eligible `WLN-ENGINE-FILE-FAILED` ERROR defect and counted +as under-scanned; it never aborts the run or loses the other files' findings. diff --git a/docs/guides/weft.md b/docs/guides/weft.md index 2fae8cc1..bf761c87 100644 --- a/docs/guides/weft.md +++ b/docs/guides/weft.md @@ -88,20 +88,31 @@ $ wardline scan . --filigree-url http://localhost:8377/api/weft/scan-results This is layered on top of the normal local output — Wardline still writes `findings.jsonl` (or your `--output`) and runs the gate; emission is additive. +Use `--local-only` (alias `--no-emit`) when a scan must stay local even though a +Filigree or Loomweave endpoint is discoverable from the environment or project +install state. The emitter is stdlib `urllib` only (no new dependency). Findings of **all** kinds are sent; each goes on the wire with `path`, `rule_id`, `message`, mapped lowercase `severity`, line range, a **top-level `fingerprint`** (Filigree's cross-run identity key), and a `metadata.wardline.*` namespace carrying qualname, kind, internal severity, and per-rule properties. +Large finding sets are chunked before upload. Wardline uses the explicit +`--filigree-max-findings-per-request` value first, then +`WARDLINE_FILIGREE_MAX_FINDINGS_PER_REQUEST`, then Filigree's advertised +scan-results limit from `/api/files/_schema` when reachable, and finally a safe +default of 1000. Chunking keeps complete file groups together whenever possible +so Filigree's `mark_unseen` reconciliation does not treat a later chunk as a fix. + The outcome split is **load-bearing for the charter guarantee**: - **Sibling absent / outage** — connection refused, timeout, or any 5xx: warn and continue. The scan proceeds to its gate; the exit code is unaffected. A Filigree outage must never make Wardline's gate load-bearing. -- **Client/protocol error** — a 4xx (or stray 3xx): loud failure. Wardline sent a - request the server rejected — a payload/config bug — so the response body is - echoed and the command exits `2`, even if findings are otherwise clean. +- **Client/protocol error** — a 4xx (or stray 3xx): warn and continue for + `wardline scan`, after the local findings and gate remain authoritative. + Wardline reports the rejected upload as failed enrichment instead of exiting + `2` before the gate verdict. - **Success** — a one-line summary reports created/updated counts plus any server-side `warnings` (Filigree reports severity coercions and line clamps there) and partial-ingest failures. diff --git a/docs/index.md b/docs/index.md index ca2c78c8..a91800ba 100644 --- a/docs/index.md +++ b/docs/index.md @@ -1,10 +1,13 @@ # Wardline -Generic, lightweight semantic-tainting static analyzer for Python — track -untrusted data across your codebase and gate trust-boundary violations, with -zero runtime dependencies. The product front door lives at -[wardline.foundryside.dev](https://wardline.foundryside.dev/); these are the -reference docs. +Wardline is a lightweight semantic-tainting static analyzer for trust +boundaries. It scans Python source, includes a Rust command-injection preview, +and gives agents and CI a deterministic gate for untrusted data reaching trusted +code. + +The product front door lives at +[wardline.foundryside.dev](https://wardline.foundryside.dev/). These pages are +the reference docs. ## Install @@ -18,12 +21,13 @@ Wardline ships in layers, so you only pull what you use: | --- | --- | --- | | `wardline` (base) | nothing | the analysis engine as a zero-dependency library | | `wardline[scanner]` | pyyaml, jsonschema, click | the `wardline scan` command-line tool | +| `wardline[rust]` | scanner extra, tree-sitter, tree-sitter-rust | the Rust command-injection preview frontend | The `wardline scan` CLI lives in the `scanner` extra, so install `wardline[scanner]` to run the examples below. Everything in the -[Weft integration](guides/weft.md) guide — SARIF output, the Filigree emitter, -Loomweave conformance — also ships in `scanner` (the Filigree emitter uses only -the standard library), so no further extra is required. +[Weft integration](guides/weft.md) guide — SARIF output, agent-summary output, +signed governance artifacts, native Filigree emission, and Loomweave +conformance — composes with the normal scanner path. ## 30-second example @@ -50,8 +54,23 @@ actually returns raw, untrusted data — a trust-boundary leak. The [Getting Started](getting-started.md) guide walks through this finding field by field. +## Product workflow + +1. Mark the boundary with `@external_boundary`, `@trust_boundary`, or + `@trusted`. +2. Run `wardline scan . --fail-on ERROR` locally or in CI. +3. Ask `wardline explain-taint` or the MCP `explain_taint` tool why the gate + tripped. +4. Fix the validation or normalization at the boundary and rescan. + +Agents can run the same loop through `wardline mcp` without scraping terminal +output. Use `wardline install` to add the agent guidance and MCP registration to +an application project. + ## Next steps - [Getting Started](getting-started.md) — install, run a first scan, and read a finding. - [The model](concepts/model.md) — trust tiers, boundaries, and how taint flows. +- [Rust support](guides/rust-preview.md) — command-injection preview for Rust code. +- [Weft integration](guides/weft.md) — SARIF, Filigree, Loomweave, and signed handoff paths. - [Arming agents](guides/agents.md) — using Wardline to give coding agents a trust-boundary check. diff --git a/docs/reference/cli.md b/docs/reference/cli.md index 1f384c43..079c891a 100644 --- a/docs/reference/cli.md +++ b/docs/reference/cli.md @@ -110,6 +110,8 @@ it at a package root, not a single file. | `--fail-on [CRITICAL\|ERROR\|WARN\|INFO]` | Exit non-zero when any finding at or above this severity survives the baseline. Use this as your CI gate. | | `--cache-dir PATH` | Store the L3 inter-procedural summary cache here so the next scan can reuse unchanged summaries when `WARDLINE_SUMMARY_CACHE_KEY` is set in the process environment. Unsigned or incorrectly signed files are ignored and the scan falls back to recomputing summaries. Use an operator-owned directory outside untrusted checkouts; do not put the cache in a path that pull-request content can commit or modify. | | `--filigree-url TEXT` | Opt-in: POST findings to a Filigree Weft scan-results endpoint as well as emitting them locally. Prefer this native path when agents need Filigree promotion, deduplication, or close/reopen lifecycle state. | +| `--local-only`, `--no-emit` | Disable sibling emission for this scan, even if Filigree or Loomweave URLs resolve from flags, environment, MCP install state, or local published ports. The scan still writes local output and evaluates the gate. | +| `--filigree-max-findings-per-request INTEGER` | Cap findings per Filigree scan-results POST. Precedence is explicit CLI value, then `WARDLINE_FILIGREE_MAX_FINDINGS_PER_REQUEST`, then Filigree's advertised scan-results limit from `/api/files/_schema` when reachable, then Wardline's safe default (`1000`). Wardline chunks by complete file groups when possible so Filigree reconciliation does not mark later chunks as fixed. | Realistic invocation — scan the source tree, emit SARIF to a file, and fail the build on any `ERROR`-or-worse finding: @@ -139,6 +141,35 @@ $ wardline scan src/ --format agent-summary --output findings.agent-summary.json See the [getting-started guide](../getting-started.md) for a first end-to-end scan and how to read the findings. +## `wardline scan-job` + +**Purpose:** start a file-backed scan job and poll status without requiring a +daemon. Job state lives under `.weft/wardline/jobs//status.json`; the +default findings artifact lives beside it. + +```text +$ wardline scan-job start . --fail-on ERROR --timeout 600 +{"job_id":"...","status":"running","phase":"starting",...} + +$ wardline scan-job status --path . +{"job_id":"...","status":"completed","phase":"complete",...} + +$ wardline scan-job cancel --path . +{"job_id":"...","status":"cancelled","phase":"cancelled",...} +``` + +Status JSON includes `phase`, `progress`, `heartbeat`, `artifacts.findings`, +`gate`, `filigree_emit`, and a terminal `failure_kind`. Scan/tool errors report +`failure_kind: "scan"`, gate trips report `"gate"`, and successful scans with +non-load-bearing upload failure report `status: +"completed_with_enrichment_failure"` plus `failure_kind: "enrichment"`. Use +scan jobs default to a 30-minute timeout so a stuck analyzer becomes a terminal +`failure_kind: "timeout"` status instead of an ambiguous hang. Use +`--timeout SECONDS` to choose a tighter or looser bound, or `--timeout 0` for an +intentionally unbounded local run. Polling `status` also reports stale/dead +workers, and `cancel` persists a terminal `cancelled` status for scans that are +no longer useful. + ## `wardline explain-taint` **Purpose:** explain ONE finding's taint provenance — the immediate tainted diff --git a/docs/stylesheets/extra.css b/docs/stylesheets/extra.css index 1ee3a29a..2e58d9e9 100644 --- a/docs/stylesheets/extra.css +++ b/docs/stylesheets/extra.css @@ -7,15 +7,15 @@ headings, links, admonitions, tables, radii. We do NOT re-declare any of that. What stays here is purely Wardline's bespoke landing-page identity: - - The Wardline thread accent: CORAL (#F4845F), the federation's trust-boundary + - The Wardline thread accent: CORAL (#F0875E), the federation's trust-boundary strand — a deliberate change from the previous teal. It repaints the hero CTA, eyebrow, links inside the landing, decorator names, section eyebrows, - and the "you are here" badge. (In the LIGHT scheme it deepens to #C2410C so + and the "you are here" badge. (In the LIGHT scheme it deepens to #CF5630 so coral-as-text clears WCAG AA on white — mirroring how weft-mkdocs deepens its sky accent for light.) - The .wl-* landing components (hero, finding panel, feature cards, the eight-state trust lattice, the Weft suite grid), recolored onto the Weft - dark-teal surface ramp + 3/6/8 radii. + warm espresso / paper surface ramp + 3/6/8 radii. - The eight-state lattice swatches are SEMANTIC, not brand: they ride the design-system semantic ramp — most-trusted (INTEGRAL) = --ready emerald, least-trusted (MIXED_RAW) = --stale red, midpoints through --aging amber. @@ -60,9 +60,9 @@ the --ready emerald, least-trusted end the --stale red, gradient through the --aging amber. Pinned here (not per-scheme) so the ramp reads identically in both light and dark — it is a fixed semantic legend, not a chrome surface. */ - --ds-ready: #10B981; /* emerald — most trusted */ - --ds-aging: #F59E0B; /* amber — intermediate */ - --ds-stale: #EF4444; /* red — least trusted */ + --ds-ready: #5FB98E; /* emerald — most trusted */ + --ds-aging: #E9B04A; /* amber — intermediate */ + --ds-stale: #E2604E; /* red — least trusted */ } /* ========================================================================== @@ -79,71 +79,71 @@ /* Repaint Material's interactive accent to the Wardline coral thread so the chrome weft-mkdocs styles (links, nav active rail, TOC marker, headerlinks) read in Wardline's identity colour rather than the suite's default sky. */ - --md-accent-fg-color: var(--thread-wardline); /* #F4845F coral */ + --md-accent-fg-color: var(--thread-wardline); /* #F0875E coral */ --md-typeset-a-color: var(--thread-wardline); /* Weft dark surface ramp (design-system: base / raised / overlay / hover) */ - --wl-bg: #0B1215; /* surface-base */ - --wl-bg-subtle: #131E24; /* surface-raised */ - --wl-bg-card: #1A2B34; /* surface-overlay */ - --wl-bg-hover: #243A45; /* surface-hover */ + --wl-bg: #14110D; /* surface-base */ + --wl-bg-subtle: #1E1A13; /* surface-raised */ + --wl-bg-card: #2A2319; /* surface-overlay */ + --wl-bg-hover: #39301F; /* surface-hover */ /* Code surfaces stay dark in both schemes (editor-screenshot motif) */ - --wl-bg-code: #0B1215; - --wl-fg-code: #E2EEF2; + --wl-bg-code: #14110D; + --wl-fg-code: #F2E9D8; /* Text ramp */ - --wl-text: #E2EEF2; /* text-primary */ - --wl-text-muted: #8FAAB8; /* text-secondary */ + --wl-text: #F2E9D8; /* text-primary */ + --wl-text-muted: #B6A78E; /* text-secondary */ /* Borders — Weft hairlines */ - --wl-border: #1E3340; /* border-default */ - --wl-border-strong: #2A4454; /* border-strong */ + --wl-border: #332A1F; /* border-default */ + --wl-border-strong: #4A3C2A; /* border-strong */ /* Coral accent (the Wardline thread) + a soft tint for chips/pills */ - --wl-accent: var(--thread-wardline); /* #F4845F */ - --wl-accent-soft: rgba(244, 132, 95, 0.14); - --wl-on-accent: #0B1215; /* text on coral fill */ + --wl-accent: var(--thread-wardline); /* #F0875E */ + --wl-accent-soft: rgba(240, 135, 94, 0.14); + --wl-on-accent: #14110D; /* text on coral fill */ /* Semantic signal colours (verdict ✓/✗) from the design-system ramp */ - --wl-danger: var(--ds-stale); /* #EF4444 */ - --wl-danger-soft: rgba(239, 68, 68, 0.14); - --wl-ok: var(--ds-ready); /* #10B981 */ - --wl-ok-soft: rgba(16, 185, 129, 0.14); + --wl-danger: var(--ds-stale); /* #E2604E */ + --wl-danger-soft: rgba(226, 96, 78, 0.14); + --wl-ok: var(--ds-ready); /* #5FB98E */ + --wl-ok-soft: rgba(95, 185, 142, 0.14); } /* ---- LIGHT (default) scheme — the documented alternate ---- */ [data-md-color-scheme="default"] { - /* Coral-as-text fails AA on white; deepen to #C2410C (the same darker coral + /* Coral-as-text fails AA on white; deepen to #CF5630 (the same darker coral weft-mkdocs uses for the light-scheme code-constant colour) so links and the eyebrow clear 4.5:1. Brand reads coral; contrast holds. */ - --md-accent-fg-color: #C2410C; - --md-typeset-a-color: #C2410C; + --md-accent-fg-color: #CF5630; + --md-typeset-a-color: #CF5630; /* Weft light surface ramp */ - --wl-bg: #F0F6F8; /* surface-base */ - --wl-bg-subtle: #E8F1F4; /* surface-overlay */ - --wl-bg-card: #FFFFFF; /* surface-raised */ - --wl-bg-hover: #DCE9EE; /* surface-hover */ + --wl-bg: #ECE3D1; /* surface-base */ + --wl-bg-subtle: #E2D7BF; /* surface-overlay */ + --wl-bg-card: #F8F1E3; /* surface-raised */ + --wl-bg-hover: #D8CBB1; /* surface-hover */ /* Code surfaces deliberately stay dark in both schemes */ - --wl-bg-code: #0B1215; - --wl-fg-code: #E2EEF2; + --wl-bg-code: #14110D; + --wl-fg-code: #F2E9D8; - --wl-text: #0F2027; /* text-primary (light) */ - --wl-text-muted: #3D6070; /* text-secondary */ + --wl-text: #241E13; /* text-primary (light) */ + --wl-text-muted: #5C5238; /* text-secondary */ - --wl-border: #C5D8E0; /* border-default (light) */ - --wl-border-strong: #9BBBC8; /* border-strong */ + --wl-border: #CDBE9F; /* border-default (light) */ + --wl-border-strong: #A8966F; /* border-strong */ - --wl-accent: #C2410C; /* deepened coral */ - --wl-accent-soft: rgba(194, 65, 12, 0.10); - --wl-on-accent: #FFFFFF; /* text on coral fill */ + --wl-accent: #CF5630; /* deepened coral */ + --wl-accent-soft: rgba(207, 86, 48, 0.10); + --wl-on-accent: #F8F1E3; /* text on coral fill */ - --wl-danger: #B91C1C; /* deepened red for AA on light */ - --wl-danger-soft: rgba(239, 68, 68, 0.10); - --wl-ok: #047857; /* deepened emerald for AA on light */ - --wl-ok-soft: rgba(16, 185, 129, 0.12); + --wl-danger: #B23A28; /* deepened red for AA on light */ + --wl-danger-soft: rgba(226, 96, 78, 0.10); + --wl-ok: #2E7D52; /* deepened emerald for AA on light */ + --wl-ok-soft: rgba(95, 185, 142, 0.12); } /* ========================================================================== @@ -317,9 +317,9 @@ align-items: center; gap: var(--wl-space-2); padding: var(--wl-space-3) var(--wl-space-4); - background: #131E24; /* surface-raised — one step up from base */ + background: #1E1A13; /* surface-raised — one step up from base */ border-bottom: 1px solid var(--wl-border); - color: #8FAAB8; + color: #B6A78E; font-size: 0.72rem; } .wl-finding__dot { @@ -345,7 +345,7 @@ display: inline-block; width: 1.4rem; text-align: right; - color: #5A7D8C; /* text-muted */ + color: #7F6F58; /* text-muted */ user-select: none; margin-right: var(--wl-space-3); } @@ -358,12 +358,12 @@ /* Syntax tokens — terminal-grade, tuned to the design-system code-highlight palette (sky / violet / aqua / gold / coral). */ -.wl-tok-kw { color: #38BDF8; } /* keyword — sky */ -.wl-tok-dec { color: #E3B341; } /* decorator — gold */ -.wl-tok-fn { color: #4ECDC4; } /* function — aqua */ -.wl-tok-str { color: #10B981; } /* string — emerald */ -.wl-tok-cmt { color: #5A7D8C; font-style: italic; } /* comment — muted */ -.wl-tok-raw { color: #EF4444; font-weight: 600; } /* the raw leak — red */ +.wl-tok-kw { color: #56B7E2; } /* keyword — sky */ +.wl-tok-dec { color: #E9B04A; } /* decorator — gold */ +.wl-tok-fn { color: #52C9B8; } /* function — aqua */ +.wl-tok-str { color: #5FB98E; } /* string — emerald */ +.wl-tok-cmt { color: #7F6F58; font-style: italic; } /* comment — muted */ +.wl-tok-raw { color: #E2604E; font-weight: 600; } /* the raw leak — red */ /* The verdict strip under the code */ .wl-verdict { @@ -372,12 +372,12 @@ gap: var(--wl-space-3); padding: var(--wl-space-3) var(--wl-space-4); border-top: 1px solid var(--wl-border); - background: rgba(239, 68, 68, 0.12); /* stale-red wash, on the dark panel */ + background: rgba(226, 96, 78, 0.12); /* stale-red wash, on the dark panel */ } .wl-verdict__mark { flex: none; font-weight: 700; - color: #EF4444; + color: #E2604E; font-size: 1rem; line-height: 1.5; } @@ -385,17 +385,17 @@ font-family: var(--wl-font-mono); font-size: 0.74rem; line-height: 1.5; - color: #E2EEF2; /* always on the dark panel */ + color: #F2E9D8; /* always on the dark panel */ text-align: left; } .wl-verdict__rule { - color: #EF4444; + color: #E2604E; font-weight: 700; } /* INTEGRAL (claimed, trusted) reads green; EXTERNAL_RAW (actual, untrusted) reads red — the same most→least-trusted semantics as the lattice. */ -.wl-verdict__body .wl-tier-bad { color: #EF4444; font-weight: 600; } -.wl-verdict__body .wl-tier-good { color: #10B981; font-weight: 600; } +.wl-verdict__body .wl-tier-bad { color: #E2604E; font-weight: 600; } +.wl-verdict__body .wl-tier-good { color: #5FB98E; font-weight: 600; } /* ---- Trust-flow motif: EXTERNAL_RAW flowing into a trusted producer ---- */ .wl-flow { @@ -414,10 +414,10 @@ border: 1px solid currentColor; white-space: nowrap; } -.wl-flow__node--raw { color: #EF4444; } /* untrusted — stale red */ -.wl-flow__node--claim { color: #10B981; } /* claimed trusted — green */ +.wl-flow__node--raw { color: #E2604E; } /* untrusted — stale red */ +.wl-flow__node--claim { color: #5FB98E; } /* claimed trusted — green */ .wl-flow__arrow { - color: #5A7D8C; + color: #7F6F58; flex: none; } @@ -610,16 +610,16 @@ .wl-tier__tag--inf { color: var(--wl-text-muted); } /* Swatch hues — SEMANTIC ramp most→least trusted, anchored on the design-system - triplet: --ready emerald (#10B981) → --aging amber (#F59E0B) → --stale red - (#EF4444). The endpoints are the exact tokens; the interior steps interpolate + triplet: --ready emerald (#5FB98E) → --aging amber (#E9B04A) → --stale red + (#E2604E). The endpoints are the exact tokens; the interior steps interpolate in hue so the eight ordered states stay distinguishable. */ -.wl-sw-1 { background: #10B981; } /* INTEGRAL — --ready emerald */ +.wl-sw-1 { background: #5FB98E; } /* INTEGRAL — --ready emerald */ .wl-sw-2 { background: #34C77E; } /* ASSURED */ .wl-sw-3 { background: #74C66A; } /* GUARDED */ .wl-sw-4 { background: #C2BE4A; } /* UNKNOWN_ASSURED — nearing --aging */ -.wl-sw-5 { background: #F59E0B; } /* UNKNOWN_GUARDED — --aging amber */ +.wl-sw-5 { background: #E9B04A; } /* UNKNOWN_GUARDED — --aging amber */ .wl-sw-6 { background: #F4762A; } /* EXTERNAL_RAW */ -.wl-sw-7 { background: #EF4444; } /* UNKNOWN_RAW — --stale red */ +.wl-sw-7 { background: #E2604E; } /* UNKNOWN_RAW — --stale red */ .wl-sw-8 { background: #C0303A; } /* MIXED_RAW — deepest (most stale) */ /* ========================================================================== diff --git a/docs/stylesheets/weft-mkdocs.css b/docs/stylesheets/weft-mkdocs.css index 04f58d98..bc2a510f 100644 --- a/docs/stylesheets/weft-mkdocs.css +++ b/docs/stylesheets/weft-mkdocs.css @@ -1,5 +1,5 @@ /* ============================================================================ - weft-mkdocs.css · v1 · 2026-06-05 + weft-mkdocs.css · v2 · 2026-06-07 ---------------------------------------------------------------------------- The Weft Federation shared docs theme for MkDocs Material. @@ -16,8 +16,10 @@ light-dark()/[data-theme]. We map the handoff tokens onto Material's own scheme selectors: [data-md-color-scheme="slate"] = dark (CANONICAL), [data-md-color-scheme="default"] = light. - - Token VALUES are lifted verbatim from the design system (terminal-grade - dark-teal surfaces, sky accent, the per-member thread palette, radii). + - Token VALUES are lifted verbatim from the design system. This is the "Loom" + revision: the dark canonical scheme is a warm espresso ground with a dyed- + amber accent and the warmed per-member thread palette; the light scheme is + "Specimen" — a warm-paper field-ledger with oxblood-red ink. - Fonts: JetBrains Mono = product/body face; Space Grotesk = brand/headings. Bundled locally (OFL) so the site works fully offline; mkdocs.yml sets `font: false` so Material pulls no Google Fonts. @@ -58,14 +60,14 @@ --weft-radius: 6px; --weft-radius-lg: 8px; - /* The Weft thread palette — one accent per member (legible on dark base). */ - --thread-loomweave: #4ECDC4; - --thread-filigree: #38BDF8; - --thread-wardline: #F4845F; - --thread-legis: #A78BFA; - --thread-charter: #E3B341; - --thread-shuttle: #6B7C8C; - --lacuna-accent: #CE7AAE; + /* The Weft thread palette — one accent per member (warmed for "Loom"). */ + --thread-loomweave: #52C9B8; + --thread-filigree: #56B7E2; + --thread-wardline: #F0875E; + --thread-legis: #B79BF2; + --thread-charter: #E9B04A; + --thread-shuttle: #8C7C68; + --lacuna-accent: #C77FA6; /* Wire Material's own font slots to the Weft faces (font:false in mkdocs.yml means Material emits --md-text/code-font-family but loads no webfont). */ @@ -74,115 +76,117 @@ } /* =========================================================================== - DARK / slate scheme — the CANONICAL DEFAULT + DARK / slate scheme — the CANONICAL DEFAULT ("Loom") Surfaces, text ramp, accent: design-system dark tokens. =========================================================================== */ [data-md-color-scheme="slate"] { - /* hue used by Material to derive its own slate ramp; keep it teal-leaning */ - --md-hue: 200; + /* hue used by Material to derive its own slate ramp; keep it warm/amber */ + --md-hue: 35; /* Surfaces (design-system: base/raised/overlay/hover) */ - --md-default-bg-color: #0B1215; /* surface-base */ - --md-default-bg-color--light: #131E24; /* surface-raised */ - --md-default-bg-color--lighter: #1A2B34; /* surface-overlay */ - --md-default-bg-color--lightest:#243A45; /* surface-hover */ + --md-default-bg-color: #14110D; /* surface-base */ + --md-default-bg-color--light: #1E1A13; /* surface-raised */ + --md-default-bg-color--lighter: #2A2319; /* surface-overlay */ + --md-default-bg-color--lightest:#39301F; /* surface-hover */ /* Text ramp (primary / secondary / muted) */ - --md-default-fg-color: #E2EEF2; /* text-primary */ - --md-default-fg-color--light: #8FAAB8; /* text-secondary*/ - --md-default-fg-color--lighter: #5A7D8C; /* text-muted */ - --md-default-fg-color--lightest:#1E3340; /* hairline tint */ - - /* Primary chrome (header/nav) = raised surface, sky accent for marks/links */ - --md-primary-fg-color: #131E24; - --md-primary-fg-color--light: #1A2B34; - --md-primary-fg-color--dark: #0B1215; - --md-primary-bg-color: #E2EEF2; - --md-primary-bg-color--light: #8FAAB8; - - /* Accent — sky (the suite's interactive blue) */ - --md-accent-fg-color: #38BDF8; - --md-accent-fg-color--transparent: rgba(56, 189, 248, 0.10); - --md-accent-bg-color: #0B1215; - --md-accent-bg-color--light: #131E24; + --md-default-fg-color: #F2E9D8; /* text-primary */ + --md-default-fg-color--light: #B6A78E; /* text-secondary*/ + --md-default-fg-color--lighter: #7F6F58; /* text-muted */ + --md-default-fg-color--lightest:#332A1F; /* hairline tint (border-default) */ + + /* Primary chrome (header/nav) = raised surface, amber accent for marks/links */ + --md-primary-fg-color: #1E1A13; + --md-primary-fg-color--light: #2A2319; + --md-primary-fg-color--dark: #14110D; + --md-primary-bg-color: #F2E9D8; + --md-primary-bg-color--light: #B6A78E; + + /* Accent — dyed amber (the suite's interactive thread) */ + --md-accent-fg-color: #E9B04A; + --md-accent-fg-color--transparent: rgba(233, 176, 74, 0.10); + --md-accent-bg-color: #14110D; + --md-accent-bg-color--light: #1E1A13; /* Links resolve through the typeset link var below */ - --md-typeset-a-color: #38BDF8; + --md-typeset-a-color: #E9B04A; /* Code surfaces */ - --md-code-bg-color: #131E24; - --md-code-fg-color: #E2EEF2; - --md-code-hl-color: rgba(56, 189, 248, 0.15); - - /* Syntax highlight (Pygments) — dark, terminal-grade */ - --md-code-hl-comment-color: #5A7D8C; - --md-code-hl-keyword-color: #A78BFA; /* violet */ - --md-code-hl-string-color: #4ECDC4; /* aqua */ - --md-code-hl-number-color: #E3B341; /* gold */ - --md-code-hl-name-color: #E2EEF2; - --md-code-hl-function-color: #38BDF8; /* sky */ - --md-code-hl-constant-color: #F4845F; /* coral */ - --md-code-hl-operator-color: #8FAAB8; - --md-code-hl-punctuation-color:#8FAAB8; - --md-code-hl-variable-color: #E2EEF2; - --md-code-hl-special-color: #CE7AAE; + --md-code-bg-color: #1E1A13; + --md-code-fg-color: #F2E9D8; + --md-code-hl-color: rgba(233, 176, 74, 0.15); + + /* Syntax highlight (Pygments) — warm, terminal-grade */ + --md-code-hl-comment-color: #7F6F58; + --md-code-hl-keyword-color: #B79BF2; /* violet */ + --md-code-hl-string-color: #52C9B8; /* aqua */ + --md-code-hl-number-color: #E9B04A; /* gold */ + --md-code-hl-name-color: #F2E9D8; + --md-code-hl-function-color: #56B7E2; /* sky */ + --md-code-hl-constant-color: #F0875E; /* coral */ + --md-code-hl-operator-color: #B6A78E; + --md-code-hl-punctuation-color:#B6A78E; + --md-code-hl-variable-color: #F2E9D8; + --md-code-hl-special-color: #C77FA6; /* Selection / footer */ - --md-typeset-mark-color: rgba(227, 179, 65, 0.30); - --md-footer-bg-color: #0B1215; - --md-footer-bg-color--dark: #070D0F; - --md-footer-fg-color: #E2EEF2; - --md-footer-fg-color--light: #8FAAB8; - --md-footer-fg-color--lighter:#5A7D8C; + --md-typeset-mark-color: rgba(233, 176, 74, 0.30); + --md-footer-bg-color: #14110D; + --md-footer-bg-color--dark: #0C0A07; + --md-footer-fg-color: #F2E9D8; + --md-footer-fg-color--light: #B6A78E; + --md-footer-fg-color--lighter:#7F6F58; } /* =========================================================================== - LIGHT / default scheme — the design system's documented alternate + LIGHT / default scheme — the design system's documented alternate ("Specimen") + Warm-paper field-ledger with oxblood-red ink. =========================================================================== */ [data-md-color-scheme="default"] { - --md-default-bg-color: #F0F6F8; /* surface-base (light) */ - --md-default-bg-color--light: #FFFFFF; /* surface-raised */ - --md-default-bg-color--lighter: #E8F1F4; /* surface-overlay*/ - --md-default-bg-color--lightest:#DCE9EE; /* surface-hover */ - - --md-default-fg-color: #0F2027; /* text-primary */ - --md-default-fg-color--light: #3D6070; /* text-secondary*/ - --md-default-fg-color--lighter: #6B8D9C; /* text-muted */ - --md-default-fg-color--lightest:#C5D8E0; /* hairline tint */ - - --md-primary-fg-color: #FFFFFF; - --md-primary-fg-color--light: #E8F1F4; - --md-primary-fg-color--dark: #DCE9EE; - --md-primary-bg-color: #0F2027; - --md-primary-bg-color--light: #3D6070; - - /* Accent — deeper sky for AA contrast on white */ - --md-accent-fg-color: #0284C7; - --md-accent-fg-color--transparent: rgba(2, 132, 199, 0.10); - --md-accent-bg-color: #FFFFFF; - --md-accent-bg-color--light: #F0F6F8; - - --md-typeset-a-color: #0284C7; - - --md-code-bg-color: #E8F1F4; - --md-code-fg-color: #0F2027; - --md-code-hl-color: rgba(2, 132, 199, 0.12); - - --md-code-hl-comment-color: #6B8D9C; - --md-code-hl-keyword-color: #7C3AED; - --md-code-hl-string-color: #0F766E; - --md-code-hl-number-color: #B45309; - --md-code-hl-function-color: #0284C7; - --md-code-hl-constant-color: #C2410C; - --md-code-hl-operator-color: #3D6070; - --md-code-hl-punctuation-color:#3D6070; - - --md-typeset-mark-color: rgba(2, 132, 199, 0.18); - --md-footer-bg-color: #0F2027; - --md-footer-bg-color--dark: #0B1215; - --md-footer-fg-color: #E2EEF2; - --md-footer-fg-color--light: #8FAAB8; - --md-footer-fg-color--lighter:#5A7D8C; + --md-default-bg-color: #ECE3D1; /* surface-base (paper) */ + --md-default-bg-color--light: #F8F1E3; /* surface-raised (stock) */ + --md-default-bg-color--lighter: #E2D7BF; /* surface-overlay*/ + --md-default-bg-color--lightest:#D8CBB1; /* surface-hover */ + + --md-default-fg-color: #241E13; /* text-primary */ + --md-default-fg-color--light: #5C5238; /* text-secondary*/ + --md-default-fg-color--lighter: #897B5F; /* text-muted */ + --md-default-fg-color--lightest:#CDBE9F; /* hairline tint (border-default) */ + + --md-primary-fg-color: #F8F1E3; + --md-primary-fg-color--light: #E2D7BF; + --md-primary-fg-color--dark: #D8CBB1; + --md-primary-bg-color: #241E13; + --md-primary-bg-color--light: #5C5238; + + /* Accent — oxblood / madder (the ruling-red ink) */ + --md-accent-fg-color: #A33B2C; + --md-accent-fg-color--transparent: rgba(163, 59, 44, 0.10); + --md-accent-bg-color: #F8F1E3; + --md-accent-bg-color--light: #ECE3D1; + + --md-typeset-a-color: #A33B2C; + + --md-code-bg-color: #E2D7BF; + --md-code-fg-color: #241E13; + --md-code-hl-color: rgba(163, 59, 44, 0.12); + + /* Syntax highlight (Pygments) — embroidery-floss threads on paper */ + --md-code-hl-comment-color: #897B5F; + --md-code-hl-keyword-color: #6E4FC0; /* violet */ + --md-code-hl-string-color: #118C7E; /* aqua */ + --md-code-hl-number-color: #A9791F; /* gold */ + --md-code-hl-function-color: #1E7AB0; /* sky */ + --md-code-hl-constant-color: #CF5630; /* coral */ + --md-code-hl-operator-color: #5C5238; + --md-code-hl-punctuation-color:#5C5238; + + --md-typeset-mark-color: rgba(163, 59, 44, 0.18); + --md-footer-bg-color: #241E13; + --md-footer-bg-color--dark: #14110D; + --md-footer-fg-color: #F2E9D8; + --md-footer-fg-color--light: #B6A78E; + --md-footer-fg-color--lighter:#7F6F58; } /* =========================================================================== @@ -225,29 +229,29 @@ border-left: 4px solid var(--md-accent-fg-color); } -/* note/info → sky */ +/* note/info → sky (status-wip, the cool pop) */ .md-typeset .admonition.note, -.md-typeset .admonition.info { border-left-color: #38BDF8; } +.md-typeset .admonition.info { border-left-color: #56B7E2; } .md-typeset .note > .admonition-title, -.md-typeset .info > .admonition-title { background-color: rgba(56,189,248,0.12); } +.md-typeset .info > .admonition-title { background-color: rgba(86,183,226,0.12); } /* tip/success → emerald (ready) */ .md-typeset .admonition.tip, -.md-typeset .admonition.success { border-left-color: #10B981; } +.md-typeset .admonition.success { border-left-color: #5FB98E; } .md-typeset .tip > .admonition-title, -.md-typeset .success > .admonition-title { background-color: rgba(16,185,129,0.12); } +.md-typeset .success > .admonition-title { background-color: rgba(95,185,142,0.12); } /* warning → amber (aging) */ -.md-typeset .admonition.warning { border-left-color: #F59E0B; } -.md-typeset .warning > .admonition-title { background-color: rgba(245,158,11,0.12); } +.md-typeset .admonition.warning { border-left-color: #E9B04A; } +.md-typeset .warning > .admonition-title { background-color: rgba(233,176,74,0.12); } /* danger/bug/failure → red (stale) */ .md-typeset .admonition.danger, .md-typeset .admonition.bug, -.md-typeset .admonition.failure { border-left-color: #EF4444; } +.md-typeset .admonition.failure { border-left-color: #E2604E; } .md-typeset .danger > .admonition-title, .md-typeset .bug > .admonition-title, -.md-typeset .failure > .admonition-title { background-color: rgba(239,68,68,0.12); } +.md-typeset .failure > .admonition-title { background-color: rgba(226,96,78,0.12); } /* =========================================================================== Radii — apply the 3/6/8 system to the surfaces Material rounds. diff --git a/mkdocs.yml b/mkdocs.yml index 0270fb3d..d20b5dbf 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -1,5 +1,5 @@ site_name: Wardline -site_description: Generic semantic-tainting static analyzer for Python +site_description: Lightweight semantic-tainting static analyzer for trust boundaries site_url: https://wardline.foundryside.dev/docs/ repo_url: https://github.com/foundryside-dev/wardline repo_name: foundryside-dev/wardline diff --git a/www/index.html b/www/index.html index 0d5fb346..efddb08d 100644 --- a/www/index.html +++ b/www/index.html @@ -3,8 +3,8 @@ -Wardline — trust-boundary analysis - +Wardline — trust-boundary analysis for agents and CI + @@ -32,7 +32,9 @@ ~/wardline