Conversation
Phase 1 of the ops surface the user asked for: a single make-driven
verb-shape that covers the three CLI-binary publish destinations
(local fs / staging MinIO / prod Cloudflare R2) without sporadic ad-hoc
bash. Phases 2 (classify bundle) and 3 (tool catalog) extend the same
shape later; this commit lands only the CLI flow.
Surface (verb × env):
make build-cli # 5 platforms, embedded creds → dist/
make publish-cli ENV=local # no-op (artifacts already in dist/)
make publish-cli ENV=staging # aws s3 cp via MinIO S3 API
make publish-cli ENV=prod # wrangler r2 object put --remote
make release-cli ENV=… # build + publish + verify
make verify-cli ENV=… # re-check sha against live URL
make diff ENV=… # local sha vs live sha
make clean-dist
Why a thin Makefile + a real bash script:
- Apple ships GNU Make 3.81 (2006) which doesn't support `.ONESHELL:`.
Multi-line shell recipes get ugly fast in 3.81 (every line is a
separate shell, requiring trailing `\` and shell-quoting hell).
- Splitting into ./ops/release.sh keeps the user-facing surface
conventional (`make release-cli`) while letting the actual logic stay
readable bash with proper functions, error handling, and consistent
shell quoting.
What replaces:
- Drops the dead `soth-ops` Makefile targets — that binary doesn't
exist (no `[[bin]] = "soth-ops"` declaration, no `feature = "ops"`,
no src/bin/). The targets would have failed if anyone ran them.
- Drops `macos-universal*` / `linux-binaries*` which built a darwin
universal2 binary + linux-gnu-only — neither what we ship to storage
(per-arch files, including Windows + Linux glibc-2.17 baseline via
cargo zigbuild for portability).
Env contract:
ops/.env.<env> # gitignored, sourced from secret store
ops/.env.example # checked-in template
ops/.gitignore # blocks .env.local/.staging/.prod
Required env (validated per-target):
SOTH_HONEYCOMB_API_KEY build-cli (option_env! at compile time)
SOTH_SENTRY_DSN build-cli
MINIO_ACCESS_KEY/SECRET publish-cli ENV=staging
AWS_CA_BUNDLE publish-cli ENV=staging (defaults to
/etc/ssl/cert.pem)
(prod uses `wrangler login`; no extra creds.)
Verification:
- `make publish-cli ENV=staging` and `make publish-cli ENV=prod` both
run end-to-end against today's live binaries. All 5 binaries' sha256
match local ↔ remote on both envs after re-publish (idempotent).
- `make diff ENV=…` walks the matrix without uploading.
- Cred-missing failure paths exit cleanly with actionable error
messages.
TODO before scaling team-wide:
- Mint a scoped MinIO service account for `release` bucket writes
(today the env file caches root creds — convenient but overpowered).
- Phase 2 (classify) + Phase 3 (catalog) verbs.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Wraps today's "ssh into Railway shell + curl admin API" classify-bundle
publish flow into the same `make release-X ENV=…` shape Phase 1
established for CLI binaries.
Surface:
make build-classify [VERSION=v1-…] # tar.gz from $DATA_DIR/classify/
make publish-classify ENV={local,staging,prod} # POST to admin API
make release-classify ENV=… # build + publish + verify
VERSION defaults to `v1-$(date +%Y-%m-%d)` to match the existing prod
naming convention (today's prod was v1-2026-04-28). Override on the CLI
for hotfixes.
Source layout: $DATA_DIR/classify/ holds manifest.json + 5 model files
(embedding.onnx, centroids.bin, lsh_projection.bin, use_case_mlp.bin,
tokenizer.json). The build step packs the dir into a gzip-compressed
tar — exactly the format the admin upload handler expects, per
crates/soth-api/src/handlers/bundles.rs:72.
Verification: the upload response embeds the server-stored sha256.
Match it against the local sha and we've proved the bytes landed
intact. (The /v1/edge/classify/current endpoint requires api_key auth,
so probing it from ops would mean enrolling a key just for tooling —
not worth the complexity when the upload response already gives us
ground truth.)
ADMIN_API auto-defaults per ENV (api.soth.ai for prod,
api.staging.soth.xyz for staging, localhost:8081 for local). Override
via ops/.env.\$ENV or shell.
Verified end-to-end on both envs:
make release-classify ENV=staging → 200 + sha match
make release-classify ENV=prod → 200 + sha match
Companion server-side change applied (not in this PR — it's nginx
config on the Hetzner staging box):
/etc/nginx/sites-enabled/api.staging.soth.xyz
+ client_max_body_size 100M;
The staging api vhost defaulted to nginx's 1M limit, which 413'd the
18.6 MB tarball. The storage vhost already had 100M; aligned the api
vhost to match. Prod (Railway, no nginx in front) was unaffected.
Phase 3 (tool catalog) and Phase 4 (status/diff cross-env) follow.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Wraps the tool-catalog refresh flow into the same `make <verb>-catalog
ENV=…` shape Phases 1 and 2 established.
Surface:
make import-catalog ENV=staging # ssh+scp+POST /import/current
make import-catalog ENV=prod # prints soth-cloud-commit guidance
make compile-catalog ENV=… # POST /compile, capture compilation_id
make publish-catalog ENV=… # POST /compilations/{id}/publish
make release-catalog ENV=… # compile + publish (composite)
`compilation_id` is persisted to dist/catalog-compilation-id.<env>.txt
so a separate `publish-catalog` invocation can find the id from the
last `compile-catalog`. Files are env-scoped so staging and prod
state can't collide.
`import-catalog` is split out from `release-catalog` because it has
very different mechanics across envs:
- staging: scp ${DATA_DIR}/raw_bundle.json → /tmp on the Hetzner box,
then `sudo -u soth cp` into the deploy tree at
/opt/soth/soth-cloud/data/runtime/local-bundle/registry/, then
POST /import/current.
- prod: Railway containers don't expose a writable filesystem, so
the file must be committed to soth-cloud and shipped via CI.
Print the commit instructions and abort.
Verified end-to-end:
make compile-catalog ENV=staging → 200, compilation_id captured
make publish-catalog ENV=staging → 200, published live
make release-catalog ENV=prod → 200 + 200, both stages clean
`import-catalog ENV=staging` mechanically works (scp + sha-verify +
POST land cleanly) but exposes two pre-existing server-side bugs in
the import handler. Both are soth-cloud tickets, NOT ops-Makefile
issues:
1. Newer raw_bundle.json (~/labterminal/soth/data/raw_bundle.json
today) has a shape the import code doesn't deserialize:
"invalid type: sequence, expected a map at line 84544 column 13"
The deployed import code expects a map there; the data has a
sequence. Either the data needs reshaping or the deserializer
needs updating.
2. Re-importing the existing soth-cloud-checked-in raw_bundle.json
into a populated DB violates the
`uq_bundle_formats_entity` unique constraint. The vendor INSERT
uses ON CONFLICT DO UPDATE; the format INSERT does not. Adding
the same upsert pattern to formats would make import idempotent.
Parsers (`SOTH_Complete_Governance_Dataset.json`) are out of scope:
the file's shape is `{metadata, tools}`, the admin endpoint expects
`{parsers: {...}}`. Mapping needs its own exploration.
Phase 4 (cross-env status/diff) and Phase 5 (GHA wrappers) follow.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Inventory + drift verbs that close the loop on Phases 1-3 — now you can see what's live where without grepping logs or hitting curl by hand. Surface: make status ENV=… # one-env detail across CLI + classify + catalog make status-all # walks staging + prod side by side make diff ENV=… # local dist/ vs ENV (CLI + classify) Three-block-per-env layout: 1. CLI binaries — sha256 of each at the live URL. No auth needed (the release/ prefix is publicly served on both R2 and MinIO). 2. Classify bundle — version + sha + published_at, read from the sidecar `dist/classify-published.<env>.json`. The sidecar is written automatically by `publish-classify` after a successful upload — sourced from the upload response body, which embeds the server-stored sha256. Limitation: reflects "last published from this machine"; if someone else published from a different dist/, we won't see it. Trade-off chosen because the edge classify endpoint requires api_key auth, which ops tooling shouldn't carry. 3. Tool catalog — live from `GET /v1/admin/registry/current`. No sidecar needed; the admin API is the source of truth. `publish-catalog` also writes `dist/catalog-published.<env>.json` parsed from the publish response (version + compilation_id + published_at). Currently only used as a paper trail; future status- all could surface "last-published version" from it. `status-all` re-execs the script per env so each iteration freshly loads `ops/.env.<env>` — same env-file contract as a direct `status ENV=…` call, no token cross-contamination between staging and prod. `diff` extended to cover classify (CLI was already there). Catalog isn't included in `diff` — its source is server-side DB state, so there's no local-side artifact to compare. `status` reports it instead. Verified end-to-end: make status ENV=staging → 5 CLI shas + classify sidecar + live catalog make status ENV=prod → same shape make status-all → both envs, side-by-side make diff ENV=staging → CLI all OK + classify OK Phase 5 (GHA wrappers) follows. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
feat(ops): unified Makefile-driven release flow (Phases 1-4)
changes fixed for windows
Carve detect/classify so the same crates run in both the proxy hot path and a future Rust SDK core compiled to PyO3/napi-rs/WASM. soth-detect: - Default features: ["intelligence"] (was ["tree-sitter", "intelligence"]) - Rename feature `tree-sitter` -> `tree-sitter-code`; document fallback contract in code.rs (heuristic + regex when off) - Split intelligence into pure-trait `intelligence` and `intelligence-sqlite` (gates rusqlite + IntelligenceStore + replay) - Add NoopBackend + InMemoryBackend in new intelligence_backends module; re-export IntelligenceSink as IntelligenceBackend for SDK callers - Gate `intelligence_bench` on intelligence-sqlite soth-classify: - New `onnx-models` feature (default on); gates ort, tokenizers, ndarray - onnx_embed.rs: stub OnnxEmbeddingRuntime + EmbedUnavailable when off - bundle.rs: target-conditional build_onnx_runtime returns None when off soth-core: - Target-conditional getrandom/uuid `js` features for wasm32-unknown-unknown soth-proxy: - Update feature list to ["tree-sitter-code", "intelligence", "intelligence-sqlite"] Verification: - All test suites green (detect 115/111, classify 40/40, proxy 92) - wasm32-wasip1 + wasm32-unknown-unknown builds clean for detect+classify with --no-default-features Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…ibution Carve ProxyContext into three concern-shaped sub-contexts so the SDK can populate only what it has (identity) and stages 6/7 read transport/attribution through the same Option-shaped accessors regardless of whether the proxy or SDK constructed the context. Wire format preserved via `#[serde(flatten)]`; cloud-side consumers see the same flat JSON shape. Field rename `matched_provider`/`matched_application` -> `declared_provider`/`declared_application` carries `#[serde(rename)]` so the JSON keys are unchanged. Audience map written into the doc comment: - IdentityContext: domain identity. Both proxy and SDK populate. - TransportContext: HTTP/TLS/H2 plumbing. Proxy-only; SDK leaves at default. - AttributionContext: process resolution + surface taxonomy + shadow-IT classification. Proxy-only; SDK leaves at default. Public API additions: - ProxyContext::sdk_only(IdentityContext) -> ProxyContext - ProxyContext::from_parts(identity, transport, attribution) -> ProxyContext - TrafficClassification now derives Default (UnknownAgent) Stage 6/7 reads route through `proxy_ctx.identity.*`, `proxy_ctx.transport.*`, `proxy_ctx.attribution.*`. All construction sites updated: handler.rs, classify_task.rs, db.rs, all test fixtures, the historian extension, examples, benches. Verification: - 634 workspace lib tests pass; zero failures introduced - One pre-existing classify integration_pipeline test failure (cosine L2 normalization) reproduced on PR 1 baseline — unrelated - All 4 WASM builds (detect+classify x wasip1+unknown-unknown) clean Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds the SDK's core consumption surface: callers that already have a typed LLM call (provider, model, messages, system, tools, stream) can skip the proxy's HTTP fingerprint/parse phase entirely and feed detect a structured input. Output schema is identical to process_with_registry's so the downstream classify pipeline doesn't care which path was taken. soth-core: - New ParseSource::Sdk variant (was missing from the enum despite earlier audit suggesting otherwise) - New typed_call module with TypedLlmCall + TypedMessage + TypedTool input types (re-exported from lib.rs) - DetectResult::from_typed_call(normalized, artifacts, capture_mode) named constructor; sets parse_source=Sdk + confidence=Full - Doc comments mark dedup fields (is_prefix_repeat, novel_token_count, prefix_hash, ast_normalized_hash, etc.) as "internal — SDK can default" soth-detect: - engine::process_normalized(registry, call, bundle, snapshot, capture_mode) -> DetectResult. Skips parse phase; runs scan phase + session prefix-repeat phase using the same primitives as process_with_registry - build_normalized_from_typed_call computes user_content_hash, conversation_hash, system_prompt_hash, tool_definition_hash, canonical_cache_key, token estimates via the same hash helpers the REST parser uses - build_scan_input_from_typed_call routes per-message content into existing ArtifactLocation tags (UserContent / AssistantContent / SystemPrompt) so credential detection sees per-turn locations - Re-exported from lib.rs soth-policy: - parse_source_label updated for new Sdk variant Tests (6 new in soth-detect lib tests): - process_normalized_basic_openai_chat_marks_sdk_source - process_normalized_extracts_credentials_from_user_message - process_normalized_with_tools_and_system_populates_hashes - process_normalized_repeated_call_signals_prefix_repeat (round-trip with SessionSnapshot.seen_prefix_hashes) - process_normalized_each_provider_variant_compiles - detect_result_from_typed_call_sets_sdk_source_and_full_confidence Verification: - soth-detect: 121 lib tests pass with all-features (was 115; +6) - soth-detect: 117 lib tests pass with --no-default-features (+6 new pass without intelligence-sqlite) - classify 40, proxy 92, core integration 22 all unchanged - All 4 WASM combos (detect+classify x wasip1+unknown-unknown) clean Byte-level proxy<->SDK output parity is the job of PR 5's conformance harness — this PR establishes the entry point. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Lock down the threading story for SDK consumers (PyO3 / napi-rs / WASM host process). Future bindings will share Arc<ClassifyBundle> and Arc<ParserRegistry> across worker pools; this PR adds compile-time guarantees and runtime stress coverage for that pattern. Spike result on `ort::Session` concurrency: - ort 2.0.0-rc.x's Session::run takes `&mut self` (verified against ~/.cargo/.../ort-2.0.0-rc.11/src/session/mod.rs:206). - The earlier review hypothesis "Session::run is &self in current ort" is incorrect for this version. Session is Send + Sync, but the binding's signature still requires exclusive access per inference call. - Conclusion: keep `Mutex<Session>`. RwLock would not help — every inference is a writer. If contention becomes a real bottleneck later, the next step is a session pool, not lock-free single-session access. - Documented the rationale in onnx_embed.rs. Compile-time Send + Sync assertions added so accidental future regressions fail to build: - soth-classify: ClassifyBundle, OnnxEmbeddingRuntime, Arc<dyn ClassificationProvider>, Arc<dyn AnomalyScorer> - soth-detect: ParserRegistry In-memory bundle loading audit (no new code): - soth-classify already has load_from_bytes(manifest, assets) (PR 1 audit). - soth-detect's OwnedDetectBundle derives Serialize+Deserialize and is a pure data struct — SDK can construct via serde_json::from_slice with no filesystem syscalls. - soth-bundle::load_from_bytes is the SDK's compound-bundle entry point. New stress tests (acceptance criteria from the PR plan): - soth-classify/tests/concurrent_stress.rs: 16 threads x 1000 calls classify with shared Arc<ClassifyBundle>, asserts deterministic semantic_hash across threads (~2.5s wall clock). - soth-detect/tests/concurrent_stress.rs: 16 threads x 1000 calls process_normalized with shared Arc<ParserRegistry>, asserts deterministic canonical_cache_key + user_content_hash and parse_source = Sdk (~0.2s). Verification: - soth-detect 121 lib + 1 stress test pass (all-features) - soth-detect 117 lib pass (no-default-features) - soth-classify 40 lib + 1 stress test pass - soth-proxy 92 lib unchanged - All 4 WASM combos (detect+classify x wasip1+unknown-unknown) clean Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds the test crate that catches future drift between the proxy
(`process_with_registry`) and SDK (`process_normalized`) detect/classify
paths. Every fixture runs both lanes against the same logical LLM call
and asserts the contract surface agrees, with failures reporting the
diverging field by name.
New crate: `soth-conformance-tests` (workspace member, internal only).
Layered comparison by design:
- compare() — strict; failures block CI. Fields at parity today:
provider, model, endpoint_type, is_ai_call,
has_tool_definitions, stream, capture_mode,
artifacts (kind set), policy_decision.kind,
telemetry shape contract.
- compare_advisory() — known follow-up divergences (printed not blocking):
user_content_hash, system_prompt_hash, conversation_hash,
tool_definition_hash, downstream classified labels.
Each fixture surfaces these so the next workstream
has a concrete target list.
Sub-PR fixes already landed inline (caught by the harness):
- TypedLlmCall::user_content() now uses last-user-message + trim
semantics, mirroring parse_rest::last_user_content. (Was concat-all-users.)
- TypedLlmCall::conversation_text() now uses role:content\n format,
mirroring parse_rest's conversation hash input.
- build_normalized_from_typed_call hashes "[CONTENT_NOT_EXTRACTED]"
sentinel for empty content, matching parse_rest::user_content_hash.
Corpus (7 MVP fixtures spanning the launch axes):
- 01 openai_chat_basic clean / no tools / non-streaming
- 02 openai_chat_streaming_tools clean / tools / streaming + system
- 03 anthropic_messages_basic clean / Anthropic system field
- 04 credential_leak_in_user_message credential / no tools
- 05 code_in_user_message code / no tools
- 06 cohere_chat_basic clean / Cohere
- 07 mistral_multiturn clean / 3-turn conversation
Path to launch target (80) tracked in crate README with explicit
coverage matrices. Each new fixture should fill an unexercised cell.
Verification:
- conformance harness: 1 test pass (7 fixtures, all strict-parity)
- Advisory output prints 2 fixtures with known divergences (02, 07) on
user_content_hash and conversation_hash — documented as next workstream
- soth-detect 121 (all-features) / 117 (no-default) lib tests
- soth-classify 40 lib tests
- soth-proxy 92 lib tests
- All 4 WASM combos (detect+classify x wasip1+unknown-unknown) clean
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two pre-flight specs that block Plan 2 Phase 0 (soth-sdk-core scaffolding). Neither is implementation; both lock the public-API contracts the SDK will ship in customer dependencies. Once Phase 0 commits the types, every change to these surfaces is breaking for downstream code. docs/common/SDK_DECISION_API_SPEC.md (~560 lines) - Decision enum: Allow / Block / Redact / Flag (Reroute dropped — proxy-only) - BlockReason variants including UseAlternative for reroute-shaped policies - MessageRedactions: message-level replacement only; within-message edits are documented as proxy-only - DecisionToken lifecycle: created in pre_call, consumed once in post_call, slab-backed, panic-on-reuse-in-debug, log-and-ignore in release, 60s orphan sweeper, sentinel for slab-full - Wrapper exception contract per language: SothBlocked extends base Exception/Error (NOT openai.APIError / etc.), propagates past existing try/except API-error handlers; gating negative tests required per binding - Sync (≤5ms p99) vs async (≤300ms p99) budget with explicit allowed work - Conformance harness extension: compare_decisions() asserts variant + BlockReason kind + redact message_idx set; proxy_only fixture tagging docs/common/SDK_WASM_TRUST_BOUNDARY_SPEC.md (~450 lines) - ORT-Web spike result: realistic full-mode payload is 22-27MB compressed, not the original ≤8MB target (which was wrong) - Tier matrix: native = full local; browser/Bun/Lambda = full WASM; CF Workers / Vercel Edge = reduced; Fastly = full - ClassificationMode enum: Full | Reduced | CloudOptIn (no auto-promotion) - Reduced-mode capability statement: explicit list of what works (artifact redaction, counter-based anomalies, artifact policy, session dedup, telemetry shipping) and what doesn't (use_case_label, semantic anomalies, ML-conditioned org rules) - Cloud-classify wire format: explicit opt-in with separate DPA, never fallback, classification_location: "cloud" telemetry tag - Bundle CDN trust path: Ed25519 verify, hot-swap via ArcSwap, no on-disk cache option for serverless, no key-fetching endpoint - Conformance lanes: SDK-via-PyO3, SDK-via-WASM-reduced, SDK-via-WASM-cloud-optin (with stubbed local endpoint for hermetic CI) Both specs include "If this needs to change" sections so the lock-in is deliberate, not accidental. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
New workspace member crates/soth-sdk-core/ — the public-API surface every SDK binding (PyO3, napi-rs, WASM) consumes. Implements the contract locked by the two pre-flight specs landed in d68fb20: - docs/common/SDK_DECISION_API_SPEC.md - docs/common/SDK_WASM_TRUST_BOUNDARY_SPEC.md Public types (all #[non_exhaustive] where future variants are anticipated): Decision — Allow / Block / Redact / Flag (no Reroute) BlockReason — SensitiveArtifact / BudgetExceeded / PolicyRule / UseAlternative MessageRedactions — message-level only; within-message edits are proxy-only by design DecisionToken — opaque, Copy, slab-backed; SLAB_FULL + SENTINEL_FAIL_OPEN constants for backpressure / fail-open FlagSeverity — Info / Warning / Critical HmacKey — FromEnv / FromFile / FromCallback / Static; resolve() returns Zeroizing<Vec<u8>>; Debug never prints plaintext ClassificationMode — Full / Reduced / CloudOptIn StorageMode — InMemory / FileBacked BundleSource — Cdn / Embedded / Fallback SdkConfig + SdkConfigBuilder LlmCall (= TypedLlmCall), Message, Tool, LlmResponse, LlmChunk Observation, StreamObservation SdkError + SothSdk DecisionToken slab (src/slab.rs): 4096-slot fixed capacity, 95% pressure threshold returns SLAB_FULL per-slot generation counter detects token reuse panic-on-reuse in debug, log-and-ignore in release per spec §5.3 bounded in_use atomic for lock-free pressure check SothSdk facade (src/sdk.rs): init — validates HMAC key + ClassificationMode + bundle source; fails closed pre_call — synchronous decision (≤5ms p99 budget per spec §7.1); runs process_normalized + sync block path on credential/private-key artifacts; allocates slab token post_call — consumes token, runs full classify with fallback bundle, pushes TelemetryEvent onto in-memory queue stream_begin / stream_chunk / stream_end — token-consumed-once invariant for streaming refresh_bundle — Phase-1 stub in_flight_decisions / drain_telemetry_for_test — test hooks In-memory telemetry queue (src/telemetry_queue.rs): bounded VecDeque (4096 default), oldest-drops-on-overflow with dropped-count metric. Phase-1 wires HTTPS shipper. Compile-time Send + Sync assertions for SothSdk + Observation — bindings stash Arc<SothSdk> across host worker threads. Tests: 15 unit + 5 integration round-trip - init_succeeds_with_minimal_config - pre_call_then_post_call_balances_slab_and_emits_telemetry - pre_call_blocks_on_credential_in_user_message - streaming_round_trip_consumes_token_once - many_pre_calls_without_post_call_do_not_leak_past_slab_capacity (asserts SLAB_FULL behavior under 6K alloc pressure) Verification: - cargo build --workspace clean - all workspace lib tests pass (634 + 20 new = 654) - All 4 WASM combos (detect+classify x wasip1+unknown-unknown) clean - Conformance harness still green (7/7 fixtures, 0 strict failures) Phase-1 deferred (called out in crate README): - Real bundle CDN pull + Ed25519 verification - Background HTTPS telemetry shipper - Cloud-classify wire transport - Orphan sweeper for never-consumed tokens - Full org-rule evaluation - OTel span emission - Conformance harness lane through this facade Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…crate calls Phase 0 follow-up. The conformance harness (PR 5) ran proxy and SDK lanes through the lower-level crates directly. Phase 0 added soth-sdk-core as the public API facade, which is now an extra layer where parity can drift. This commit closes that gap. soth-sdk-core: - Add SothSdk::for_test(config, detect_bundle, classify_bundle) — doc-hidden ctor that bypasses the bundle-source dispatch so the harness can isolate facade-vs-direct parity from bundle-source differences. Phase-1's `init` will gain an in-memory BundleSource variant that subsumes this. soth-conformance-tests: - run_facade_lane(fixture, bundles) — drives SothSdk::pre_call → post_call, drains the in-memory telemetry queue, returns the Decision + emitted TelemetryEvent. - compare_sdk_vs_facade(sdk_lane, facade) — strict-parity comparison on the cloud ingestion contract surface (provider, model, endpoint_type, capture_mode, parse_source, parse_confidence, use_case, volatility_class, policy_kind, estimated_input_tokens, anomaly_flags). Per-call ephemeral fields (event_id, timestamp_epoch_ms, commitment_nonce, commitment_hash) are documented as excluded. - New test sdk_direct_and_facade_lanes_agree_byte_identical asserts every fixture produces a TelemetryEvent matching the SDK direct lane's classify output. Failure names the diverging field so the fix lands in soth-sdk-core, not the harness. Dependency: soth-conformance-tests now depends on soth-sdk-core via path. Verification: - All 7 fixtures pass the new sdk-vs-facade parity assertion - Existing proxy-vs-sdk parity still passes (7/7 strict, 2 advisory) - soth-sdk-core: 15 unit + 5 integration tests - soth-detect / soth-classify / soth-proxy lib tests unchanged - All 4 WASM combos (detect+classify x wasip1+unknown-unknown) clean Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
bindings/soth-py/ — Python binding consuming soth-sdk-core via PyO3.
maturin-built abi3-py310 wheel publishable as `soth` on PyPI.
Public Python surface (python/soth/__init__.py):
init(api_key, org_id, hmac_key_env=...) — module-level singleton
guard(call_fn, call=...) — pre/post lifecycle wrapper
SothBlocked — exception (NOT openai.APIError)
SothFlagged — Flag surface
BlockReason — typed reason
Negative tests (tests/test_blocked_propagates.py) gate the contract:
- test_soth_blocked_does_not_inherit_from_openai_apierror
- test_soth_blocked_propagates_past_openai_apierror_handler
- test_soth_blocked_inherits_from_base_exception_directly
If any future change makes SothBlocked inherit from openai.APIError or
any provider type, these tests fail immediately. See SDK_DECISION_API_SPEC.md §6.
soth-sdk-core: added DecisionToken::raw() / from_raw() so bindings can
round-trip the token across the FFI boundary without inspecting struct
internals. The token is still opaque per spec §3.4.
Cargo / build:
- bindings/soth-py added to workspace members but NOT default-members
(cdylib + cdylib needs Python toolchain to actually link)
- `pyo3/extension-module` is a soth-py feature, not always-on, so
`cargo build -p soth-py` succeeds for compile-checking. maturin
passes the feature when building wheels.
- Python 3.10 floor (3.9 reached EOL October 2025)
- abi3 — one wheel per platform×arch covers Python 3.10+
Verification:
cargo build -p soth-py clean
All workspace lib tests: 654 (unchanged)
Conformance harness: 7/7 fixtures (unchanged)
Phase 0 SDK core round-trip tests: 5/5 (unchanged)
Phase-1 follow-ups documented in bindings/soth-py/README.md:
auto-instrumentation, httpx middleware, with-context, Redact handling,
streaming wrapper, per-arch wheel matrix via cibuildwheel.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
bindings/soth-node/ — Node.js binding consuming soth-sdk-core via napi-rs.
@napi-rs/cli builds prebuilt binaries per platform/arch; published to
npm as @soth/sdk.
Public TS surface (index.js + index.d.ts):
init({ apiKey, orgId, hmacKeyEnv }) — module-level singleton
guard(asyncFn, { call }) — pre/post lifecycle wrapper
SothBlocked extends Error — NOT extends OpenAI.APIError
SothFlagged — Flag surface
BlockReason — typed reason interface
Negative tests (__test__/blocked-propagates.test.mjs) gate the contract:
- SothBlocked does not extend openai.APIError (instanceof check)
- SothBlocked propagates past try { ... } catch (OpenAI.APIError) blocks
- SothBlocked extends Error directly
Per SDK_DECISION_API_SPEC.md §6.3, customers' retry-on-API-error logic
must NOT silently retry policy blocks. The negative test will fail loudly
if the inheritance changes.
DecisionToken round-trip: tokens are stringified u64 across the FFI
because JS numbers can't safely hold the full u64 range. Bindings parse
the string back into the u64 via DecisionToken::from_raw, which was
added in this commit's companion soth-sdk-core change.
Cargo / build:
- bindings/soth-node added to workspace members, NOT default-members
- napi-rs 2.x with napi8 ABI feature (Node 18+ supported)
- `cargo build -p soth-node` compiles the cdylib clean
- Production binaries built via `npm run build` per the
@napi-rs/cli triples in package.json (linux-x64-{gnu,musl},
aarch64-linux-{gnu,musl}, darwin-arm64, darwin-x64, win32-x64)
Verification:
cargo build -p soth-node clean
cargo build --workspace clean (both bindings included)
All workspace lib tests unchanged
Phase-1 follow-ups documented in bindings/soth-node/README.md:
auto-instrumentation, undici dispatcher, withContext helper,
AsyncIterable streaming wrapper, per-arch loader logic.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Both bindings gain a streaming surface that mirrors the Decision API
spec §4.2 (`stream_begin → stream_chunk → stream_end`). Streaming was
the keystone gap before auto-instrumentation: ~70% of production LLM
traffic streams, so without this the SDK was silently broken on the
hot path.
Python (bindings/soth-py):
- PyStreamObservation napi class wrapping CoreStreamObservation in
Mutex<Option<>> for take-on-end semantics.
- New PyO3 method SothSdk.stream_begin(call) -> (decision, observation).
- python/soth/__init__.py adds:
async def guard_stream(iter_factory, *, call, chunk_extractor=None)
Async generator; raises SothBlocked on Block decision; default
chunk extractor reads OpenAI's chunk.choices[0].delta.content shape.
- tests/test_streaming.py: 3 round-trip tests (consume-once, block on
credential, double-end idempotent).
- Type stubs updated in _soth_native.pyi.
Node (bindings/soth-node):
- napi class can't compose with napi(object) wrappers, so observations
are kept on SothSdk keyed by token (raw u64 stringified across FFI).
streamBegin returns just the JsDecision; streamChunk(token, ...) and
streamEnd(token) look up by token.
- index.js adds `guardStream(iterFactory, { call, chunkExtractor })`
async generator with the same semantics as the Python version.
Default extractor is OpenAI-shaped.
- index.d.ts: GuardStreamOptions + ChunkExtractorOutput types.
- __test__/streaming.test.mjs: 3 tests mirroring Python.
Sentinel handling: SLAB_FULL / SENTINEL_FAIL_OPEN tokens skip the
observation slab — there's nothing to stash because pre_call didn't
allocate. streamEnd on a sentinel is a no-op.
Verification:
- cargo build --workspace clean
- All workspace lib tests unchanged (654)
- Conformance harness 7/7 fixtures still pass
Phase-1 follow-ups still deferred:
- chunk_extractor presets for anthropic/cohere/google/mistral
- Auto-instrumentation hooking streamBegin/streamChunk/streamEnd
- Streaming wrapper for non-AsyncIterator callbacks (Anthropic's
message_stream_text style)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Connects the SDK's in-memory telemetry queue to soth-cloud over HTTPS.
Without this commit, post_call emitted events into a queue nobody read;
now configured deployments actually deliver telemetry to the dashboard.
soth-sdk-core:
- New `http-telemetry` feature (off by default to keep workspace
builds fast). When on, pulls reqwest with rustls-tls.
- New module crates/soth-sdk-core/src/shipper.rs:
TelemetryShipper::spawn(queue, endpoint, api_key, org_id)
spawns a `soth-telemetry-shipper` thread that drains MAX_BATCH_SIZE
events every BATCH_WINDOW (5s), POSTs to the configured endpoint
with bearer auth, and Drop's cleanly. Final drain on shutdown
ensures events buffered during the last window are flushed.
- TelemetryQueue::drain_batch(max) — bounded pull for the shipper.
- SothSdk holds a Mutex<Option<TelemetryShipper>>; init spawns it
iff `telemetry_endpoint` is set, drop-on-shutdown is idempotent.
- SothSdk::shutdown() — public method bindings call at process exit.
V0 limits (Phase-2 follow-ups documented in shipper.rs):
- No retry / circuit-breaker (failed batches drop)
- No exp backoff
- No dead-letter queue
- No batch compression / signing / encryption
Bindings:
- bindings/soth-py and soth-node both enable http-telemetry by default
(production native bindings always want a working shipper).
- Both expose `telemetry_endpoint` parameter on init.
- Both expose `shutdown()` method for graceful exit.
- Python: soth.init(..., telemetry_endpoint="https://...") and
soth.shutdown().
- Node: soth.init({ telemetryEndpoint: "https://..." }) and
soth.shutdown().
Verification:
- cargo build -p soth-sdk-core (default, http-telemetry off): clean
- cargo build -p soth-sdk-core --features http-telemetry: clean
- cargo build --workspace clean (bindings pull http-telemetry on)
- soth-sdk-core 15 unit + 5 integration tests unchanged
- Conformance harness 7/7 fixtures unchanged
- The for_test() ctor never spawns a shipper so CI stays hermetic
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…rage Multi-tenant story: customers can override identity (user_id_hmac, team_id, device_id_hash, session_id, request_id) per-call rather than relying on SDK init defaults. Async-aware context propagation is language-native: Python contextvars, Node AsyncLocalStorage. soth-sdk-core: - New module crates/soth-sdk-core/src/context.rs with `CallContext` struct (#[non_exhaustive], all-Option fields, builder methods). - New `pre_call_with_context(call, &CallContext)` and `stream_begin_with_context(call, &CallContext)` methods. Old `pre_call` / `stream_begin` delegate with default context. - DecisionContext (slab) carries the resolved CallContext from pre_call to post_call so identity asserted at decision time matches what telemetry emits. - build_proxy_context now reads resolved context for user_id_hmac / team_id / device_id_hash / session_id; falls back to SdkConfig defaults when CallContext leaves a field None. Session ID is parsed as UUID where possible, otherwise dropped (proxy-shaped contract). - HMAC discipline preserved: user_id_hmac is the HMAC; the SDK never sees plaintext user IDs. Python (bindings/soth-py): - soth.context(*, user_id_hmac=, team_id=, device_id_hash=, session_id=, request_id=) is a contextmanager backed by contextvars.ContextVar so it survives asyncio task switches. - guard() and guard_stream() pull the current context and pass it to the FFI on every call. - PyO3 layer: pre_call(call, context=None) / stream_begin(call, context=None) accept an optional dict; build_call_context translates to CallContext. - Nested `with soth.context(...)` blocks merge — fields not set in inner block fall through to outer. Node (bindings/soth-node): - soth.withContext(overrides, async () => ...) backed by AsyncLocalStorage so async code awaited inside sees the same context across awaits. - guard() and guardStream() pull the current AsyncLocalStorage value and pass it to the napi FFI. - napi-rs layer: preCall(call, context?) / streamBegin(call, context?) accept an optional JsCallContext; build_call_context translates. - Nested withContext calls merge identically to the Python contract. Verification: - cargo build --workspace clean - soth-sdk-core 15 unit + 5 integration tests - Conformance harness 7/7 fixtures - All workspace lib tests unchanged - All 4 WASM combos clean - Both bindings recompile cleanly with both http-telemetry and per-call context Phase-1 follow-ups: - Helper for HMAC-ing a customer-supplied user_id (currently customer computes the HMAC themselves via their stored SOTH_HMAC_KEY) - Auto-instrumentation that picks up context automatically — that's the next commit - Context introspection helpers (get_current_user_id_hmac, etc.) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
`soth.instrument()` monkey-patches provider SDK client classes so
customers don't write `soth.guard()` per-call. Built robustly per the
brief — six guarantees gate the contract:
1. **Idempotent.** instrument() called twice returns
"skipped:already-instrumented" rather than double-wrapping.
2. **Reversible.** uninstrument() restores originals captured at
apply time. If another tool wrapped over our wrapper, we leave
theirs in place and log rather than clobbering.
3. **Provider-conditional.** Missing provider packages return
"skipped:not-installed" without raising — apps deploying with
subset-of-providers don't break.
4. **Fail-open.** Any exception in build_call extraction or wrapper
setup falls back to the original SDK call uninstrumented. The
customer's API call MUST complete unaffected if SOTH itself fails.
5. **Version-tolerant.** Adapters use defensive `getattr` /
`Object.keys` access and walk multiple possible paths into
provider modules so minor-version SDK changes don't break.
6. **Sync + async aware.** Each Python adapter wraps both sync
(`Completions`) and async (`AsyncCompletions`) classes; the
same `guard()` entry point handles both because it now detects
coroutine returns.
Python (bindings/soth-py/python/soth/instrumentation/):
__init__.py: registry + instrument() / uninstrument() /
is_instrumented(); status dict per provider.
_base.py: wrap_method(target, method_name, ...) replaces a
class method with a SOTH wrapper carrying provenance
markers (__soth_wrapped__, __soth_provider__,
__soth_original__). Sync + async wrappers built
separately. revert_all checks the wrapper is still
ours before restoring.
_openai.py: patches openai.resources.chat.completions.Completions
+ AsyncCompletions + Responses (when available).
buildCall extracts model/messages/stream/tools/system
from kwargs; flattens content arrays for
multi-modal calls. Tool params are stably serialized
for hash determinism.
_anthropic.py: patches anthropic.resources.messages.Messages +
AsyncMessages. Handles Anthropic-specific shape
(system as separate field, tools with input_schema,
streaming events with content_block_delta /
message_delta / message_stop).
guard() update (Python):
Detects coroutine return from call_fn. Sync providers finalize
inline; async providers receive a coroutine to await. One
entry point handles both sync `OpenAI` and async `AsyncOpenAI`
clients without separate APIs. New `guard_stream_sync()` for
sync-streaming providers (matches `guard_stream` async generator).
Node (bindings/soth-node/instrumentation/):
index.js: registry + instrument() / uninstrument() /
isInstrumented(). Lazy-loaded so `require('@soth/sdk')`
doesn't pull provider SDKs on import.
_base.js: wrapMethod walks prototype chain; provenance markers via
non-enumerable Object.defineProperty so they don't leak
into JSON.stringify of the wrapped method. revertAll
same third-party-detection semantic as Python.
openai.js: walks multiple possible paths to find Completions
class across openai 4.x sub-versions; falls back
cleanly when none match.
Negative tests for both bindings:
- instrument is idempotent (second call: "skipped:already-…")
- uninstrument reverses
- uninstrument-without-instrument is safe
- explicit providers list filters
- missing provider returns "skipped:not-installed"
- buildCall exception falls through to original (fail-open)
- wrapped method carries provenance markers
- revert leaves third-party wrapper in place
- guard() handles coroutine and value returns (Python only —
Node already async-aware)
- OpenAI/Anthropic adapter apply() returns bool, never raises
What's NOT in this commit (Phase-1.5 follow-ups):
- Cohere, Google GenAI, Mistral Python adapters
- Anthropic Node adapter (Anthropic's npm SDK is less mature
than openai's; will land separately)
- LangChain / LlamaIndex / Vercel AI SDK callback adapters
(those go through a different integration path; framework
adapters are Phase 3 in the original plan)
Verification:
cargo build --workspace clean
All workspace tests unchanged (654 + 5 sdk-core integration)
Conformance harness 7/7 fixtures
All 4 WASM combos clean
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…adapters
Extends auto-instrumentation to the long-tail providers. Each adapter
follows the existing OpenAI/Anthropic Python pattern (apply + revert +
build_call + chunk_extractor) and inherits the six robustness
guarantees from `_base.py` (idempotent, reversible,
provider-conditional, fail-open, version-tolerant, sync+async aware).
Python — bindings/soth-py/python/soth/instrumentation/:
_cohere.py: Cohere v5 ClientV2/AsyncClientV2 (OpenAI-shaped
`messages=[{role, content}]`) AND v4 Client
(`message=` + `chat_history=`) for legacy users.
Streaming chunk extractor handles v5
content-delta / message-end events.
_google_genai.py: Models.generate_content + generate_content_stream
(sync + async). Normalizes the four shapes of
`contents` (str / list-of-str / list-of-Content /
Content-dict) into a uniform `[{role, content}]`
list. system_instruction extracted from `config`.
Streaming reads `chunk.text` + finish_reason from
candidates[0].
_mistral.py: Chat.complete + complete_async + stream + stream_async
(mistralai 1.x). OpenAI-equivalent shape so
extraction is straightforward; chunk extractor
handles Mistral's `data` envelope around OpenAI
deltas.
Node — bindings/soth-node/instrumentation/:
anthropic.js: Patches Messages.create on @anthropic-ai/sdk.
Walks multiple typed-resource path candidates
for version tolerance across 0.x. Handles
Anthropic-specific shape: separate `system`
field, `input_schema` (not `parameters`) on
tools, content_block_delta / message_delta /
message_stop streaming events.
Registry updates:
Python REGISTRY now includes openai, anthropic, cohere, google_genai,
mistralai (5 providers).
Node REGISTRY now includes openai, anthropic (2 providers).
Tests added:
- cohere/google_genai/mistral apply() returns bool — no exceptions
- cohere v2 + v4 extractors normalize messages correctly
- google_genai handles string vs list-of-Content shapes
- mistral extractor produces OpenAI-shaped output
- Anthropic Node apply() + buildCall + chunkExtractor unit tests
Verification:
cargo build --workspace clean
All workspace lib tests unchanged (654)
Conformance harness 7/7 fixtures
All 4 WASM combos clean
Adapter tests pass on systems with/without provider SDKs installed
(apply() returns False when missing; tests skip cleanly)
Phase-1.6 follow-ups (framework adapters — different integration
pattern via callbacks/middleware): LangChain Python, LlamaIndex Python,
LiteLLM Python, Vercel AI SDK Node. Each lands as its own commit.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…ntegrations
Framework integrations layer on top of the existing instrumentation
contract. Where direct provider adapters monkey-patch the SDK class
methods, frameworks use callbacks/middleware instead — SOTH plugs
into each framework's native lifecycle rather than intercepting
underneath it.
Robustness contract is preserved across all four:
- Module-level imports succeed even when the framework isn't installed
- Instantiation raises a clear ImportError with the install hint
- Idempotent registration / un-registration
- Fail-open: extractor exceptions don't break the customer's chain
- Async-aware where the framework supports both modes
Python — bindings/soth-py/python/soth/integrations/:
langchain.py: SothCallbackHandler — implements both
BaseCallbackHandler and AsyncCallbackHandler from
langchain-core. Hooks on_llm_start /
on_chat_model_start / on_llm_new_token / on_llm_end /
on_llm_error onto SOTH's pre/post lifecycle.
Per-run state keyed by LangChain's run_id (UUID).
Block decisions raise SothBlocked from on_llm_start;
LangChain's invoke() propagates the exception up.
Provider inferred from `serialized.id` import path
(langchain_openai → openai, langchain_anthropic →
anthropic, etc.).
llamaindex.py: SothEventHandler — extends BaseEventHandler from
llama_index.core.instrumentation. Hooks the four
LLM lifecycle events (LLMChatStartEvent /
LLMChatEndEvent / LLMCompletionStartEvent /
LLMCompletionEndEvent). Provider inferred from the
event class's module path.
litellm.py: register() / unregister() / is_registered() — adds
success / failure / input handlers to litellm's
module-level callback lists. LiteLLM's `model`
strings are namespaced ("openai/gpt-4o-mini",
"anthropic/claude-3-5-sonnet"); the prefix is parsed
for SOTH provider attribution. Per-call state keyed
by `litellm_call_id` so concurrent calls don't
collide. Block decisions raise SothBlocked from the
input handler, aborting the litellm call.
Node — bindings/soth-node/integrations/:
vercel-ai.js: sothMiddleware() returns a LanguageModelV1Middleware
object compatible with `wrapLanguageModel({ model,
middleware })` from the `ai` npm package.
wrapGenerate routes non-streaming calls through
SothSdk.preCall/postCall; wrapStream taps the
LanguageModelV1StreamPart stream via TransformStream
to feed text-delta and finish parts to SothSdk's
stream observation. Provider inferred from
`model.provider` ("openai.chat" → "openai", etc.).
Tests:
test_integrations.py:
- module imports succeed without the framework installed
- LangChain provider inference, message normalization
- LiteLLM register idempotency (when installed),
build_call namespace parsing
- LlamaIndex import safety
__test__/integrations.test.mjs:
- sothMiddleware shape matches LanguageModelV1Middleware
- buildCall normalizes Vercel V1 params (prompt[].content[].text
→ flat string)
- inferProviderFromModel handles openai/anthropic/google/mistral/cohere
- middleware pass-through doesn't break customer flow
Verification:
cargo build --workspace clean
All workspace tests unchanged (654 + 5 sdk-core integration)
Conformance harness 7/7 fixtures
All 4 WASM combos clean
This commit closes out Phase 1.5. Phase 2 (CI matrix + FFI conformance
harness) is the next gating workstream before paying-customer pilots.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Closes the workspace-build verification gap. Until this commit, only
fmt + clippy + cargo test on Ubuntu were gated; the WASM build matrix
and the binding compile-checks ran only on dev laptops. Now every PR
that touches detect/classify/sdk-core or the bindings runs through
the same matrix that paying customers will install from.
.github/workflows/ci.yml — extended:
+ conformance job: cargo test -p soth-conformance-tests --test parity
(proxy↔SDK direct↔SDK facade lanes, all 7 fixtures)
+ wasm job: 2x2 matrix (target × crate) for the WASM build commit
Plan 1 PR 1 promised to enforce. Failures here mean an SDK-blocking
native-dep regression slipped in.
+ bindings-build-check job: cargo build -p soth-py and -p soth-node
on Linux. Real per-arch wheel/binary builds happen in the
dedicated workflows.
.github/workflows/python-wheels.yml (new):
PyO3 + maturin via PyO3/maturin-action@v1. abi3-py310 wheels —
one wheel per platform×arch covers Python 3.10+, so the matrix
is 5 jobs (not 5 × N-Python-versions).
Targets:
- manylinux2014 x86_64 / aarch64 (aarch64 cross via QEMU)
- macOS x86_64 (macos-13) / aarch64 (macos-14)
- Windows x86_64
Native-arch jobs run a smoke test (`pip install` from local
--find-links + `import soth`) before uploading the wheel.
Non-native arches just upload (full smoke runs in
ffi-conformance.yml downstream).
Builds an sdist as a fallback for unsupported platforms.
Triggered on: PR or push to main / sdk-* paths under the binding,
the sdk-core crate, or the workflow file. Manual dispatch supported.
Wheels uploaded as per-target artifacts, retention 14 days.
Publish-to-PyPI is a separate workflow (not in this commit).
.github/workflows/node-binaries.yml (new):
napi-rs via @napi-rs/cli. Builds prebuilt binaries for the 7
triples declared in bindings/soth-node/package.json.
Targets:
- linux x86_64 (gnu, musl)
- linux aarch64 (gnu, musl) — gnu via cross-toolchain;
musl via napi-rs's prebuilt docker image
- darwin x86_64 / aarch64
- win32 x86_64-msvc
Native-arch jobs run a require() smoke test before upload to
catch obvious linkage errors. Real test runs in
ffi-conformance.yml.
Same trigger semantics as python-wheels.yml. Binaries uploaded
as artifacts, retention 14 days.
What's NOT in this commit:
- PyPI / npm publish workflows (release-time concern, separate)
- Source-of-truth version bumping (manual today)
- Binary signing (when SOTH gets to signed releases)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds the fourth lane to the conformance harness: actual PyO3 +
napi-rs FFI. Until this commit, the harness ran proxy / SDK direct /
SDK facade lanes — all in pure Rust. Drift introduced by the
binding's marshalling layer (PyO3 dict construction, napi-rs class
serialization) had no test coverage. This closes that gap.
bindings/soth-py/tests/test_ffi_conformance.py:
Loads every fixture from
crates/soth-conformance-tests/fixtures/*.json. For each:
- Builds the LlmCall dict from typed_call (same conversion the
Rust SDK lane does in soth-conformance-tests/src/lib.rs).
- Runs through soth.guard() — crosses the PyO3 boundary.
- For credential / Block fixtures: asserts SothBlocked raises.
- For others: asserts the customer call returns and a single
TelemetryEvent is emitted with provider, model, endpoint_type,
capture_mode populated.
- Asserts in_flight_decisions == 0 (slab balanced).
parametrized over fixtures so each fixture name appears in test
output; failures point at the diverging field.
Includes a corpus-floor test that fails if the fixture count
drops below 7 (Plan 1 PR 5 baseline).
bindings/soth-node/__test__/ffi-conformance.test.mjs:
Mirror of the Python suite. Uses node:test, async-aware, drives
through soth.guard() (which crosses napi-rs). Same fixture corpus,
same assertions (TelemetryEvent shape on the cloud-ingestion
contract surface).
.github/workflows/ffi-conformance.yml:
Two jobs, runs on PR / push touching bindings or sdk-core:
python: maturin develop, then pytest the FFI conformance suite +
instrumentation + integration + streaming suites
node: npm run build:debug, then npm test (which runs every
*.test.mjs in __test__/, including ffi-conformance)
Failures here mean either:
(a) the binding's FFI marshalling drifted from the Rust facade
(b) the fixture corpus changed in a way the binding hasn't picked
up yet
Either way, the failure names the fixture and the diverging field
so the fix is targeted.
Verification:
cargo build --workspace clean
cargo test -p soth-sdk-core: 15 unit + 5 integration
cargo test -p soth-conformance-tests --test parity: 2/2 (proxy↔SDK,
SDK↔facade)
All workspace lib tests unchanged
What this closes vs Plan 2 spec:
✓ CI matrix — Rust workspace, Python wheels, Node binaries (prior
commit)
✓ FFI conformance lane — extends the harness through the actual
PyO3 / napi-rs boundary
Phase 2 is complete. Tier 1 (Python + Node, native, full local
mode) is now functionally shippable: customers can `pip install soth`
on macOS/Linux/Windows, `npm i @soth/sdk` on the same, both with
auto-instrumentation for OpenAI/Anthropic and framework integrations
for LangChain/LlamaIndex/LiteLLM/Vercel AI. Conformance harness
catches drift at four levels (proxy, SDK direct, SDK facade, FFI).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…ified)
Closes the Plan 2 phase ledger. Phase 4 in the original plan was
estimated at 4–6 weeks; what lands here is the **scaffold** that
freezes the customer-facing API surface so the next-PR's WASM bridge
work doesn't break anyone who's already integrated.
What this commit DOES deliver (production-ready):
cargo build -p soth-sdk-core --target wasm32-unknown-unknown
--no-default-features
is now clean. Required moving soth-classify to default-features=off
at the workspace level (mirroring soth-detect's existing pattern)
and explicitly opting back in to onnx-models from soth-bundle and
soth-proxy. Without this, tokenizers's onig C dep poisons the WASM
build for any consumer.
bindings/soth-edge/ (new package — @soth/sdk-edge):
- package.json with `npm run build:wasm` invoking cargo
- src/index.js — JS shim mirroring @soth/sdk's API
(init/guard/guardStream/shutdown/SothBlocked) — runs through
`_invokeWasmStub` placeholder until extern "C" exports land.
Decision tier matrix locked at module level: Reduced for CF
Workers / Vercel Edge; Full WASM for Deno / Fastly. No ONNX
in either path (per WASM trust-boundary spec §3).
- src/index.d.ts — TypeScript types matching the runtime API
- README — tier matrix, reduced-mode capability statement,
CF Workers + Vercel Edge usage examples, the explicit list of
what's still wired-as-stub
sdks/soth-go/ (new module — github.com/labterminal/soth/sdks/soth-go):
- go.mod with wazero v1.8.0 dep (no cgo — preserves Go's static-
binary, alpine, cross-compile story)
- soth/sdk.go — public Go API: Init / Guard / Shutdown /
SothBlocked. Mirrors the Python and Node bindings in spirit;
idiomatic Go (errors.Is / errors.As supported).
- soth/wasm_bridge.go — wazero bridge with the function signatures
soth-sdk-core's extern "C" exports will fill in. Today returns
stubbed Allow decisions so the Go API contract can stabilize
independently.
- soth/sdk_test.go — 4 tests gating the public contract:
* Init requires APIKey + OrgID + Hmac + WasmBytes
* Init succeeds with full config
* Guard invokes the customer callback on Allow (stub path)
* SothBlocked satisfies error / errors.As
* Shutdown is idempotent
- README — extensive doc on why wazero (not cgo), the embed pattern
(`//go:embed soth_sdk_core.wasm`), and the explicit Phase-4
follow-up list.
docs/common/SDK_PHASE_STATUS.md (new):
Single-page ledger of Plan 1 / specs / Phase 0 / Phase 1 / Phase 1.5
/ Phase 2 / Phase 3 / Phase 4 status with commit pointers. Distinguishes
production-ready from scaffold so reviewers and customers see
exactly what's wired today.
Workspace dep updates:
Cargo.toml — soth-classify gets default-features=false. Same
pattern soth-detect already uses. WASM SDK target builds; consumers
that need onnx-models opt back in.
crates/soth-bundle/Cargo.toml — explicit onnx-models feature
crates/soth-proxy/Cargo.toml — explicit onnx-models feature
Verification:
cargo build --workspace clean
cargo test -p soth-sdk-core: 15 unit + 5 integration
cargo test -p soth-conformance-tests --test parity: 2/2
All 4 WASM combos clean (detect+classify x wasip1+unknown-unknown)
+ soth-sdk-core wasm32-unknown-unknown clean
The SDK build is now dependency-decoupled enough to ship native
Tier-1 bindings (Phase 1), edge scaffold (Phase 4), and Go scaffold
(Phase 4) from one branch. The Phase-4 wasm-bindgen / extern "C"
work is the next dedicated PR.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Defers SDK-side user-ID hashing to Phase 2.5. In v1, customers pre-compute userIdHmac themselves and pass it via withContext / soth.context; the SDK no longer requires hmac_key_env or hmac_key_static at init. - soth-sdk-core: hmac_key is Option<HmacKey>; builder no longer errors when unset; init/for_test only resolve the key when present - bindings (PyO3 + napi-rs): (None, None) is now a valid config rather than an error; READMEs and docstrings document the privacy tradeoff for unkeyed deployments - New unit test: builder_succeeds_without_hmac_key Regulated workloads (HIPAA / heavy-PII) SHOULD still configure a key; without one, anything passed via userIdHmac reaches soth-cloud as-is. Full key-lifecycle contract: SDK_WASM_TRUST_BOUNDARY_SPEC §6.6. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…sage extraction
Closes the §10.11 audit gap for Claude Code — historian's
playbook now extracts the full Anthropic-style `usage{}` block
per assistant turn, billing-grade. Two of the three
"fixable now" items from the 2026-05-08 audit ship in this
commit; the third (openai_codex) is documented as
engineering-blocked rather than fixed silently.
Foundation (extensions/historian/src/playbook.rs):
- `TokenConfig` extended from a single scalar `field` to
per-direction structured fields:
* input_tokens_field
* output_tokens_field
* cache_creation_input_tokens_field
* cache_read_input_tokens_field
* total_tokens_field (scalar fallback for Gemini-style
providers)
Legacy `field: Option<String>` kept for back-compat —
treated as `total_tokens_field` when nothing structured
is set. `is_billing_grade()` returns true only when all
four Anthropic-style fields are present, which is the
audit gate criterion.
- `from_total_field()` convenience constructor for scalar-
only playbooks (Gemini's `tokens.total`).
Engine (extensions/historian/src/engine/mod.rs):
- New `extract_token_usage(record, &TokenConfig) ->
Option<MessageTokenUsage>` resolves each declared dot-path
independently. Returns None when the playbook declares no
paths OR when nothing resolved on the record (engine then
falls back to the heuristic estimator).
- Existing `extract_tokens(...)` shim preserved: returns the
scalar token estimate (input + output when both present,
else total, else heuristic). All three engines (jsonl,
json_file, sqlite) call both extractors so the structured
shape rides alongside the legacy scalar with no extra
configuration.
Per-message shape (extensions/historian/src/types.rs):
- `HistoricalMessage` gains `usage: Option<MessageTokenUsage>`.
- `MessageTokenUsage` is the per-turn structured shape with
every sub-field optional — playbooks that only extract
some sub-fields still produce useful data.
Session reconstruction (extensions/historian/src/session.rs):
- `reconstruct_event` sums per-message billing-grade usage
across the session. When ≥1 message had structured
usage, emits four flat metadata keys
(`usage_input_tokens`, `usage_output_tokens`,
`usage_cache_creation_input_tokens`,
`usage_cache_read_input_tokens`) plus
`usage_source: playbook_extracted`. Cloud-side ingestion
reads these to populate ClickHouse usage columns.
- `NormalizedRequest.estimated_output_tokens` now flows
the billing-grade output_tokens sum instead of always
None — distinguishing "we don't know" from "we know 0".
Playbook updates (extensions/historian/src/playbooks.rs):
- claude_code: tokens=Some({input/output/cache_*}_tokens
paths under message.usage). Verified extraction against
real local session logs; 56 of 141 lines in a sample
session have `usage:{input_tokens` blocks, all four sub-
fields present.
- gemini_cli: migrated to `TokenConfig::from_total_field
("tokens.total")` — same scalar coverage as before, new
shape. Verdict stays false (scalar-only, not billing-
grade).
Audit verdicts (cli_config.rs):
- claude_code → TRUE with caveat citing the playbook fix
and the pinning test.
- openai_codex → still false; caveat now spells out the
blocker: the source data is on `type:event_msg` lines
(`payload.info.total_token_usage`), but the playbook's
per-record TokenConfig only sees `type:response_item`
lines. Fix requires engine refactor — session-level
token aggregation from non-message lines, distinct from
per-message extraction TokenConfig supports.
- All other agents unchanged: cursor (source has nothing),
gemini_cli (scalar-only), pi_agent / windsurf / opencode
(no playbook).
Tests:
- New `claude_code_playbook_extracts_billing_grade_usage`
pins the contract end-to-end against a hermetic fixture
mirroring real Anthropic shape.
- All 14 sites that construct HistoricalMessage updated
with `usage: None` (legacy readers + test fixtures).
- Workspace lib tests: 145 historian + 72 cli + everything
else green (~750+ tests total).
Net effect: 1 of 7 agents (claude_code) flips from "audit
mechanism exists but no agent passes" to "real audit pass
with billing-grade extraction". forward_proxy.bypass_agents
will accept "claude-cli/*" and reject everything else,
visible via `soth code audit-status`.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…dispatch) The plan §8 deferred OpenClaw because gryph PR #31 left its config-format unstable. This commit ships the runtime path (parser + classify + decision rendering + dispatch) and 11 fixtures so manually-configured hooks pointing at `soth code hook --agent openclaw --type ...` work end-to-end. Auto-install (`soth code install --target openclaw`) stays deferred — when upstream stabilizes, one match arm in commands/code.rs lights it up. extensions/code/src/adapter/openclaw.rs (new): - `OpenClawAdapter` modeled on Codex's protocol (matching what historian's openclaw playbook expects: response_item / message / payload.role / payload.content TextBlocks). - 5 hook types: pre_tool_use, post_tool_use, user_prompt_submit, stop, session_start. - UA glob patterns: `openclaw/*`, `OpenClaw/*`, `open-claw/*` — covers both case forms and the dashed variant (gryph-observed live). - `is_pre_action_hook` allows blocking on pre_tool_use + user_prompt_submit only — same gating as Codex, prevents the post-action stop-hook feedback loop. - `classify_input` extracts prompt for user_prompt_submit, tool_name + serialized input for pre_tool_use, and the serialized response for post_tool_use. Classify pipeline gets the same shape it already handles for Codex. - 9 unit tests covering: 5 hook types parsing, UA pattern coverage, pre-action gating, classify input shapes for user_prompt_submit + pre_tool_use, classify-skip for session_start/stop, render_decision Block (exit 2) + Allow (silent exit 0), tool→action mapping with MCP fallback to ToolUse. extensions/code/src/adapter/mod.rs: - Module declaration + `pub use OpenClawAdapter`. - `for_agent` match arm: `"openclaw" | "open_claw" | "open-claw"` → OpenClawAdapter (was falling through to StubAdapter with a "deferred" comment). extensions/code/tests/fixtures/openclaw/ (new, 11 files): - pre_tool_use_bash: 01_basic, 02_rm_rf (block target), 03_chained. - pre_tool_use_read / pre_tool_use_edit / pre_tool_use_mcp: 01 each. - post_tool_use_bash / session_start / stop: 01 each. - user_prompt_submit: 01_basic, 02_long_prompt. - All parse green via parser_corpus_phase3 :: every_fixture_parses_for_every_agent. extensions/code/tests/parser_corpus_phase3.rs: - openclaw added to the agents() vec, so the cross-agent corpus test now exercises 6 phase-3 adapters (pi_agent, gemini_cli, codex, windsurf, opencode, openclaw). - Fixture-count minimum stays at 10; openclaw passes with 11. crates/soth-cli/src/commands/code.rs (install + doctor): - `soth code install --target openclaw` now returns a clear parser-only message (was bundled into the unknown-target branch). Operators get told the runtime is ready; they just need to manually wire the hook command. - `soth code doctor` `agents:` section lists openclaw as "manual" with the auto-install-pending note, so operators can tell "supported but configure manually" apart from "agent unsupported". End-to-end smoke test (real binary): $ echo <rm-rf-payload> | soth code hook --agent openclaw --type pre_tool_use → exit 2, decision=block (rule code_block_destructive_rm_rf) $ echo <Read-payload> | soth code hook --agent openclaw --type pre_tool_use → exit 0, decision=allow Tests: soth-code lib 136 + 13 + 9 (new openclaw) + 3 corpus all green. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Previously `soth up` brought up the proxy and stopped. Operators with Claude Code, Cursor, Codex etc. on the same host had to also run `soth code install --target X` per agent — easy to miss, easy to forget after installing a new agent later. This commit makes `soth up` orchestrate the whole thing: proxy + hook auto-install for every detected agent, with state tracking so re-runs are idempotent. Detection (extensions/code/src/install.rs): - New `detect_installable_agents()` walks the canonical home directories and settings paths for the 7 supported agents (Claude Code, Cursor, Codex, Gemini CLI, Pi Agent, Windsurf, OpenCode). Returns one DetectedAgent per agent that's present on this host (settings file or home dir exists), with `already_installed: bool` set by checking the soth- managed marker in the settings file. - OpenClaw deliberately excluded — its install path is pending upstream config-format spec (gryph PR #31). Per-host install state (extensions/code/src/state.rs, new): - `InstalledHostState { version, hooks }` persisted at `~/.soth/installed.json`. Each `AgentInstallRecord` has `installed_at` (RFC 3339 timestamp), `settings_path` (where the install wrote), and `binary_path` (which `soth` binary the install pointed at). - Atomic write: tempfile + rename. No half-written state files after a crashed `up`. - `binary_drifted(agent, current)` detects "soth binary moved since last install" (e.g. `brew upgrade` landed the binary at a new prefix). Triggers automatic re-install on the next `up` so the hook entries get re-pointed. - 4 unit tests pin: load-when-missing-is-ok, save+load round-trip, drift detection, atomic write residue check. Orchestration (`soth up` in command_graph.rs): - `auto_install_detected_hooks(force_repair, quiet)` runs AFTER the proxy + system-proxy enable succeed — so hooks going live can immediately use the running proxy for whatever they need. - Idempotent: the per-host state file makes re-running `soth up` cheap. Already-wired agents whose binary matches state get skipped (only `installed_at` refreshes); agents with binary drift get auto-repaired; new unrecognized agents get fresh installs. - Per-agent failures don't abort the sweep — the summary reports installed / repaired / skipped / failed lists so the operator sees exactly what happened. New `soth up` flags: - `--skip-hooks`: opt out of the sweep entirely. For proxy-only governance setups. - `--repair-hooks`: force re-install even when state says no drift. Implied automatically when binary path drifts. Doctor surface (`soth code doctor`): - New `install_state:` section reads ~/.soth/installed.json and prints per-agent install timestamp + recorded binary path. When empty (`soth up` never auto-installed), states that explicitly so operators don't think the file is broken. End-to-end smoke test: - Tmpdir-rooted fake host with ~/.claude/settings.json and ~/.cursor/hooks.json present. - Detection identifies both, marks both `not_installed`. - `soth code install --target claude_code` and `--target cursor` (manual install path) succeed; doctor shows both as `installed`. - Auto-install via `soth up` is the same code path underneath, just sweeped automatically and writing the state record. Tests: 140 lib + 4 new state + workspace 700+ all green. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…or tests
Two polish items. A: close the cache/reality gap when
operators bypass `soth up` and run `soth code install/uninstall`
directly. C: pin the auto-install orchestrator's contracts
(idempotency, drift triggers repair, per-agent failure
tolerance) with real tests.
A — manual install/uninstall touches state file
(crates/soth-cli/src/commands/code.rs):
- `run_install` resolves a canonical adapter name (collapsing
`gemini`/`piagent` aliases) and, after the underlying
install succeeds, calls a new `update_state_after_install()`
that persists to ~/.soth/installed.json with the binary path
and settings path the install used. Now `soth code doctor`
shows the install whether it came from `soth up`'s sweep or
a direct `soth code install`.
- `run_uninstall` symmetrically calls `update_state_after_
uninstall()` after the underlying uninstall succeeds —
drops the agent's record so doctor stops claiming the
agent is governed.
- Both updates are best-effort (warn-and-continue on state-
write failure). The agent's settings file is the source
of truth; state is a fast-path cache for drift detection
and audit attribution.
End-to-end smoke (real binary, tmpdir HOME):
$ soth code install --target claude_code
$ soth code install --target cursor
$ cat ~/.soth/installed.json # both agents recorded
$ soth code uninstall --target claude_code
$ cat ~/.soth/installed.json # claude_code dropped
C — orchestrator integration tests
(crates/soth-cli/src/command_graph.rs):
- `auto_install_detected_hooks` refactored: split into a
pure `run_sweep(detected, state_path, current_binary,
force_repair, install_fn) -> SweepResult` plus a thin
`report_sweep` for operator-facing output. `run_sweep`
takes the binary path, state path, and install function
as injectable params so tests can drive it against a
tempdir without touching real settings files.
- New `SweepResult { installed, skipped, repaired, failed }`
return type — easy to assert on instead of side-channel
print scraping.
- 6 new tests in `sweep_tests` mod:
* fresh_sweep_installs_every_detected_agent_and_writes_state
— empty state + new agents → install_fn called for each,
state persisted with correct binary_path.
* rerun_with_already_installed_skips_install_call — second
sweep with already_installed=true and matching binary
must NOT call install_fn. Pins idempotency.
* binary_drift_triggers_repair_install_even_when_already_installed
— bootstrap with bin_old, re-sweep with bin_new, drift
detected, install_fn called, result.repaired populated,
state updated to new binary. Pins the §10.11-style
"binary moved" upgrade scenario.
* force_repair_reinstalls_even_without_drift — `--repair-
hooks` forces install_fn call regardless of state.
* per_agent_failure_does_not_abort_sweep — one agent's
install_fn returns Err; remaining agents still get
processed; result.failed[(agent, msg)] preserves the
underlying error string. Pins failure tolerance.
* empty_detection_produces_empty_result — host with no
agents → no install_fn calls, empty SweepResult, no
state mutation.
Plus existing UpArgs test fixtures updated to set the new
skip_hooks/repair_hooks fields (perl regex pass; mechanical).
Tests: 78 cli bins (6 new) + 140 soth-code lib + workspace
700+ all green.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Removes `code_block_credential_in_payload` (rule 3 in the starter pack) from the shipped default bundle. The rule fired on `detect.credential_detected == true`, which the edge scanner sets liberally — any Bash command, file write, or hook payload containing an AKIA-shape, sk_live_-shape, ghp_-shape, sk--shape, or PEM-shape string triggered an unconditional Block. In practice, that breaks too many legitimate workflows: - Operators editing their own credential files (a developer rotating a personal ~/.netrc, for instance). - Test fixtures that intentionally include credential- shape strings to exercise the detect path. - Documentation, sample configs, and policy bundle JSON itself when authored through the agent. - Operators pasting REDACTED placeholders that still match the regex shape. The four remaining default rules still cover destructive recursive-delete commands, raw device writes, ~/.ssh/ writes, and ~/.aws/credentials writes, plus the high- anomaly flag. Operators who want credential-blocking behavior can add the rule back to their own org bundle and `soth code policy apply` it; we just don't ship it as a default because the false-positive rate is too high for the OOTB experience. Detection telemetry is preserved — the detect.rs scanner still runs, still emits artifacts, and the resulting `detect.credential_detected = true` value is still in scope for any operator-authored CEL rule that wants it. The only change is which rules ship by default. After this commit the bundle has 5 org rules. Plus the 3 system rules built into the engine (sys_private_key_detected, sys_artifact_count_exceeded, sys_input_tokens_exceeded) which still block on hard critical signals. Re-installed live on the dev host: bundle now reports 0 system + 5 org rules, dev-key signed. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…cation, bypass relabel Three review-blocker fixes from the pre-push audit: #2 — test isolation against the dev host's real policy bundle (extensions/code/src/hook.rs): - `policy_bundle()` now has a `#[cfg(test)]` arm that bypasses the OnceLock cache. Tests can opt out via `SOTH_CODE_POLICY_BUNDLE_DISABLE=1` (returns None unconditionally), and other tests that need a specific bundle set `SOTH_CODE_POLICY_BUNDLE` to their tmpdir fixture. Without this, the OnceLock resolved against whatever was at `~/.soth/code-policy.bundle` on the developer host during the first test run, so subsequent tests inherited that bundle and asserted against the wrong default-deny path. - `pre_tool_use_with_credentials_still_blocks` and `capture_audit_persists_only_for_block_decisions` now set the disable env var so they exercise default-deny on any host. Production path (cfg(not(test))) keeps the OnceLock optimization unchanged. #5 — `bypass_agents` is forward-looking config without runtime consumer (cli_config.rs + commands/code.rs): - Renamed comment block to lead with "Planned, not yet effective." Operators reading the YAML now see explicitly that adding entries doesn't actually engage bypass at the proxy yet. - `audit-status` output adds a trailing note clarifying that the listener loop doesn't consume `bypass_agents` — `audit-status` is observability for the §10.11 A→C trajectory, not enforcement. The marker indicates "would be allowed when wiring lands." Prevents operators from thinking bypass is engaged when it isn't. #6 — Codex naming unification (install.rs + commands/code.rs + command_graph.rs): - New `canonical_agent_name(agent: &str) -> Option<&'static str>` in install.rs is the single source of truth. Maps friendly CLI aliases (codex, gemini, piagent, open-claw) to canonical names (openai_codex, gemini_cli, pi_agent, openclaw) — same form historian's audit table keys on, same form state file uses, same form doctor displays. - `detect_installable_agents` returns "openai_codex" (was "codex"); the parent-dir probe in `agent_present_on_host` matches the new name. - `run_install` collapses `codex|openai_codex` to `openai_codex` for the dispatch + state record. - `run_uninstall` records uninstall under `openai_codex`. - `soth code doctor` agents: section uses `openai_codex` so state lookups (which key on the same name) join cleanly. - Sweep tests updated: detected("openai_codex") + asserts on state.hooks.contains_key("openai_codex"). Pre-push review concern: state-record ↔ audit-record joins silently missed because soth-code wrote `codex` while historian audit keyed on `openai_codex`. The `bypass_ua_to_adapter` translation funnel happened to make `audited_bypass_agents` work, but any future direct join would have failed silently. Single canonical name removes that future-bug class. Tests: 6 sweep + 29 hook tests + workspace all green. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…%APPDATA%)
The install command had a single platform-agnostic path
constant per agent. Worked for the agents that use the
dotfile convention everywhere (claude_code, cursor,
openai_codex, gemini_cli, pi_agent) but two agents diverge
on Windows:
- Windsurf on Windows: %APPDATA%\Codeium\Windsurf\hooks.json
(verified live; Codeium uses the Windows AppData layout
consistent with VS Code's Windows convention). Our
previous code wrote to %USERPROFILE%\.codeium\windsurf\
which the editor does not read on Windows.
- OpenCode on Windows: %APPDATA%\opencode\plugins\
(explicit upstream behavior — OpenCode bypasses the XDG
/ ~/.config convention and forces %APPDATA% on win32; see
opencode-antigravity-auth issues #251 / #265 / #295 where
the Windows behavior is acknowledged and documented).
Our previous code wrote to %USERPROFILE%\.config\opencode\
which OpenCode does not read on Windows.
Both paths now branch on cfg(windows):
- Windows: dirs::config_dir() -> %APPDATA% (Roaming), joined
with the per-agent subpath.
- Other OSes (macOS / Linux): unchanged dotfile / XDG paths.
`agent_present_on_host` (used by `detect_installable_agents`)
gets the same per-OS treatment so the Windows-only `soth up`
sweep recognizes the agents as "present" without touching
the wrong directory.
The other five supported agents (claude_code, cursor,
openai_codex, gemini_cli, pi_agent) verified via independent
research to use the dotfile convention on every OS, so
dirs::home_dir().join(".X") resolves correctly on all
platforms (including %USERPROFILE%\.X\ on Windows) — no
changes needed there.
gryph upstream has the same gap (agent/windsurf/detect.go
falls back to ~/.codeium/windsurf on every OS;
agent/opencode/detect.go has a single platform-agnostic
constant). This commit is also the upstream fix.
Tests: 140 soth-code lib + workspace 700+ all green on
macOS. cfg(windows) branches will be exercised by the
test-windows CI job we added earlier (commit a9c9a1d).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…ession test
Real-install bug surfaced post-deploy: `soth up`'s auto-install
sweep failed for the codex agent with
⚠ soth-code hook install failed for openai_codex:
auto-install does not support agent: openai_codex
The previous canonical-name commit (3bb6ee8) updated
`detect_installable_agents` to emit `"openai_codex"` and the
top-level dispatch + state record paths, but missed
`install_one`'s match arm in `command_graph.rs:990` — still
matched on `"codex"`. The sweep_tests don't catch this
because they mock `install_fn` (they're testing orchestration,
not real dispatch), so the broken arm shipped to staging+prod.
Fix: change `"codex"` → `"openai_codex"` in install_one's
match. Operators still type `--target codex` on the manual
CLI surface (`commands/code.rs::run_install` handles the
alias collapsing).
Regression test `install_one_dispatches_every_canonical_agent_name`
iterates every canonical name `detect_installable_agents` may
emit and asserts `install_one` doesn't bail with the
dispatch-miss error. Future name-drift between detection and
dispatch will fail this test loudly instead of silently
breaking sweeps in production.
Operators with stale binaries can either:
- re-run the install script (curl ... | bash) once the new
binaries land, or
- run `soth code install --target codex` manually (the manual
path was never broken — only the auto-install dispatch).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Hook handlers are short-lived subprocesses fork-execed by the agent on every action. Loading the 23 MB ONNX bundle per-subprocess takes ~50-150 ms, which blows the per-action latency target. Result: the hook handler used the keyword fallback bundle on every call and the dashboard showed use_case=Unknown / model=unknown for every event. Mirror historian's supervisor / worker pattern. The same soth binary, when invoked with SOTH_CODE_CLASSIFY_WORKER=1, becomes the daemon worker: loads ~/.soth/bundle/ once, listens on a localhost TCP port, serves NDJSON-framed classify requests. The hook handler tries the daemon first, falls back to the in-process keyword bundle when the daemon is unreachable so a crashed daemon never crashes the gate. Cross-platform: localhost TCP works identically on macOS, Linux, and Windows. Supervisor applies nice +5 (Unix) / BELOW_NORMAL_PRIORITY_CLASS (Windows) so classify CPU bursts can't starve the mitm runtime, same as historian. New config knob: extensions.code.classify.run_mode = subprocess (default) | in_process | disabled. Pinned by a regression test so the upgrade path can't silently regress. \`soth code doctor\` reports daemon port, pid, and reachability so operators can tell a missing daemon apart from a missing bundle. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…ree-sitter Closes the gap on prod where some events showed use_case but no model and others showed model but no use_case (the pre-tool-use short-circuit emitted Unknown). Now every code event carries both, plus the surrounding telemetry the proxy and historian already produce. Per-adapter model extraction. Claude Code: top-level `model` when present (session_start); for per-tool events tail `transcript_path`'s JSONL and pull `message.model` from the latest assistant turn. Codex CLI / Cursor: top-level `model` on every hook (gryph leaves this on the table — we read it). Gemini CLI: `GEMINI_MODEL` env or `~/.gemini/settings.json` fallback. Every queue row now carries a real model name where the agent exposes one. Per-tool sidecar synthesis. pre_tool_use / post_tool_use skip the classify pipeline (running ONNX on JSON tool args returns Unknown by design — `is_ai_call` short-circuits non-NL kinds in `soth-classify/src/hook_entry.rs:157`). Instead synthesize a sidecar locally: `use_case_label` = tool name (Bash, Read, Edit, ...), secondary = canonical `ActionType`, reason = `pre_tool_call` / `post_tool_call`. New `UseCaseLabelReason::PreToolCall` / `::PostToolCall` variants preserve the typed cloud contract. Session-state cache in the daemon. Daemon owns a per-session LRU (Mutex<LruCache<String, SessionSnapshot>>, capped at 256 sessions, 32 prior hashes per session). After each classify call we fold the result back: prior_semantic_hashes, topic_cluster_ids_seen, running-mean embedding_centroid, request_count, last/current timestamps, models_used. Stage 5 anomaly now fires real `TopicDrift` / `TokenBurst` / `ToolCallDepthSpike` / `RapidFireRequests` flags within a session — confirmed end-to-end with a 5-call probe (0.000 -> 0.565 -> 0.462 anomaly_score progression). Volatility wire-up. Extended HookClassifyInput with conversation_turn / has_tool_definitions / has_tool_results that stage 4 reads. Daemon derives them from the session snapshot (request_count + 1, tool flags true once request_count > 0). volatility_class climbs LowVolatile -> HighlyDynamic and dynamic_fraction goes from 0.10 to 0.85 across an active session. soth-detect tree-sitter on classifiable content. New soth-detect dep (with tree-sitter-code feature). Hook handler runs `detect_code_artifacts` on prompt / assistant content, merges any new sensitive artifacts and surfaces tree-sitter outputs as queue metadata: detected_language, tree_sitter.confirmed_language, import_categories (JSON array), function_count, complexity_estimate, has_auth_logic / has_crypto_operations / has_network_calls / has_file_io. TelemetryEvent::from_governable now reads `metadata["import_categories"]` and folds it into the canonical network/file/crypto/auth flag bag — same path the proxy uses. Telemetry parity with historian. ClassifySidecar gained use_case_secondary_label, use_case_label_reason, volatility_class, dynamic_fraction. Hook metadata writer no longer hardcodes "fallback_bundle" — sources the reason from the actual classify result (Confident / LowConfidence / NotAiCall / etc). New flat keys: classify.volatility_class, classify.dynamic_fraction, classify.use_case_secondary_label, classify.anomaly_flags, plus top-level semantic_hash and estimated_input_tokens. from_governable resilience. Stops marking events as ExtensionNotEnriched when classify.use_case is a literal tool name (won't deserialize as UseCaseLabel enum) but a reason is explicitly present — trusts the reason (PreToolCall, etc). 161 tests passing. Workspace builds clean. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…loud
Two regressions the dashboard's /code endpoint surfaced:
1. use_case_label = "unknown" for every per-tool row.
soth-code's per-tool synthesis writes a literal tool name as
the metadata value (`"bash"`, `"read"`, `"edit"`, MCP names).
The wire converter at api-types/convert.rs:117 went through the
typed `UseCaseLabel` enum, which can't hold a tool name —
collapsed to `Unknown` — collapsed to wire string `"unknown"`.
Fix: add `use_case_label_override: Option<String>` on edge
TelemetryEvent. `from_governable` populates it with the raw
metadata string when the enum-parse fails. convert.rs prefers
override over `enum_name(&event.use_case)`. Cloud now sees
`"bash"` / `"read"` / etc. for tool rows.
2. anomaly_flags always empty even when the daemon detected
TopicDrift / TokenBurst. Two-part bug: edge wrote PascalCase
flag names (`format!("{f:?}")`), but `AnomalyFlag` derives
`rename_all = "snake_case"` so the edge form (`"TopicDrift"`)
didn't deserialize as the enum. And `from_governable` never
read `classify.anomaly_flags` from metadata — the field was
hardcoded `Vec::new()`.
Fix: serialize via serde (yields snake_case) on the edge,
read + deserialize in from_governable, populate
TelemetryEvent.anomaly_flags. convert.rs already wires
`anomaly_flags: enum_names(&event.anomaly_flags)` so the wire
field flows through unchanged.
3 new pinned tests: synthesized tool-label via override,
anomaly_flags round-trip through metadata, plus updated stub
sites for the new TelemetryEvent field.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Promotes the per-tool sidecar synthesis from Claude Code-only to all 8 agents and closes the metadata-key gap with the historian path. Cross-agent synthesis. The synthesis gate now uses `code_event.action_type` (FileRead / FileWrite / CommandExec / ToolUse) instead of agent-specific hook-type strings. That works regardless of how each agent names its tool hook — `pre_tool_use` (Claude Code, Codex, Cursor, Pi Agent, OpenClaw), `before_tool_call` (Pi Agent, Gemini), `tool_execute_before` (OpenCode), or `pre_run_command` / `pre_read_code` / `pre_write_code` / `pre_mcp_tool_use` (Windsurf). The phase (Pre vs Post) keys off the adapter's existing `is_pre_action_hook` so each agent owns the distinction. Per-agent tool-name extraction. `extract_tool_name` walks `tool_name` -> `tool` -> `command` -> `cmd` payload keys — covers Claude Code / Codex / Cursor / Gemini / Pi Agent / OpenClaw (`tool_name`), OpenCode (`tool`), Windsurf (`command` / falls back to ActionType for `pre_read_code` / `pre_write_code` which carry no tool-name field). One synthesizer, every agent. Model extraction expanded. Windsurf, OpenCode, Pi Agent, OpenClaw now read top-level `model` (and `ctx.model` / `agent_model` / `model_name` variants) from the hook payload when their plugins ship one. Best-effort — None when the plugin doesn't send model info, which keeps None better than the misleading "unknown" default. Historian metadata parity. `extensions/historian/src/enrich.rs` now writes the same metadata key set soth-code does: `use_case_secondary_label`, `anomaly_flags`, top-level `semantic_hash`, top-level `estimated_input_tokens`. Closes the dashboard gap where historian rows had NULL secondary labels and empty anomaly flags even when the classifier produced them. Cross-agent regression test. New `cross_agent_tool_action_synthesizes_label_for_every_adapter` runs a tool-shape hook through every adapter (Claude Code, Cursor, Codex, Gemini CLI, OpenCode, Pi Agent, OpenClaw) and asserts the synthesized label contains the agent-specific tool name and the reason carries `tool_call`. 162 tests passing. Workspace builds clean. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Tool events were synthesizing the sidecar locally without touching the daemon's session-state cache. Result: anomaly was always 0 / empty for tool rows on the dashboard, even though the user is doing 50 tool calls per prompt. No TopicDrift, no ToolCallDepthSpike, no RapidFireRequests — nothing. Fix: route tool events through the daemon too. The embedding/MLP path short-circuits on ToolArgs kind (correct — JSON tool args aren't NL), but stage 5's deterministic anomaly rules (RapidFireRequests, ToolCallDepthSpike, ModelSwitch, CredentialBurst, AgentLoopPattern) read session state and fire regardless of embedding. Plus stage 4 volatility scoring reads conversation_turn / has_tool_definitions / has_tool_results from the synthesized NormalizedRequest, so dynamic_fraction climbs across the session even on tool-only flows. The hook handler now overrides only the daemon's response fields that need to carry the synthesized values: use_case_label = tool name, secondary = ActionType, reason = PreToolCall / PostToolCall. Everything else (anomaly_score, anomaly_flags, volatility_class, dynamic_fraction) flows from the daemon's session-aware computation. Verified end-to-end with 5 rapid tool calls in the same session: dynamic_fraction climbs 0.10 -> 0.70 -> 0.80 (Static -> LowVolatile -> HighlyDynamic), anomaly_score fires 0.15 with `["tool_call_depth_spike"]` after the 4th call. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The MLP classifier already produces an interaction-mode label (`augmentative` / `directive` / `expressive`) on every classify run via stage 3's auxiliary head — it just wasn't being surfaced through soth-code's hook pipeline. `ClassifySidecar` gains an `interaction_mode: String` field populated from `ClassifiedResult.telemetry_event.interaction_mode`. Hook handler writes the snake_case form as a top-level `interaction_mode` metadata key (NOT under `classify.`) so soth-core's `from_governable` reads it via the existing metadata->TelemetryEvent path. Wire convert.rs already serializes it. Cloud-side reads it on ingestion + dashboard query in matching commits in soth-cloud / soth-app. Synthesized tool-call sidecars set `interaction_mode = "unknown"` since no ONNX run happened — same as volatility/anomaly defaults for that path. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Whitespace-only — CI rustfmt check on feat/soth-code-extension was failing because the recent flurry of feature commits (model extraction, classify daemon, session cache, tool synthesis, soth-detect wiring, interaction_mode) accumulated formatting drift. Running `cargo fmt --all` brings every touched file back in line with the workspace's rustfmt config. 162 lib tests still pass; no behavioural change. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
CI clippy/test failures on feat/soth-code-extension came from
test/bench helper functions that build TelemetryEvent /
PolicyContext / HookClassifyInput literals — they hadn't been
updated for the new fields landed in the recent feature
commits:
- TelemetryEvent.use_case_label_override (added so synthesized
soth-code tool-name labels survive the typed-enum collapse
at the wire boundary)
- PolicyContext.action (existing field; tests just hadn't
been touched since it was introduced)
- HookClassifyInput.{session_snapshot, conversation_turn,
has_tool_definitions, has_tool_results} (added so the
classify daemon can thread per-session priors into stages
4/5)
All call sites get sensible defaults (`None` / `false`). Two
Default::default()+reassignment patterns in soth-core's
telemetry tests refactored to struct-init syntax to clear
the `field_reassign_with_default` clippy lint that fmt
refresh surfaced.
`cargo clippy --workspace --lib --bins --tests` and
`cargo test --workspace --lib --bins` both green locally.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Feat/soth code extension
URGENT — security regression on Windows + spaced home dirs.
Engineer reported destructive shell commands succeeded through
Claude Code Bash tool on Windows + Mac, even though the policy
bundle has rules to block them. Root cause: the install code
wrote hook commands as `format!("{} code hook ...",
binary_path.display())` — unquoted. When the binary path
contains a space (Windows default `C:\Users\Prabhat
ACER\.local\bin\soth.exe`, or any user with a space in `~`),
the agent's shell splits on the space, treats the first chunk
as the binary, the rest as args, the binary fails to launch,
and the hook silently no-ops. Policy gate fails open.
Fix: new `quote_binary_path` helper double-quotes every install
target's hook command (Claude Code preToolUse, Codex
preToolUse, Cursor / Gemini / per-agent settings paths — all
three callsites flipped to it). Double quotes work on both
PowerShell / cmd and bash / zsh, no escaping needed since
soth's install paths never contain double-quote chars.
Two pinning regression tests cover the helper directly and the
end-to-end install on a space-bearing path. The CI Windows
runner used `C:\hostedtoolcache\…` (no spaces) so this never
tripped — adding the test on a fixed path closes the gap.
164 lib tests passing. After this lands, every newly-installed
agent hook (re-run `soth code install`) gets the quoted form
and the policy gate fires correctly.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Companion to the install-quoting fix. After the hook actually fires on Windows, the next failure mode is policy rules silently failing to match. Org rules are authored with forward-slash patterns (`action.file_path.contains(\".ssh/\")`) because engineers write them on macOS / Linux first. Cursor (and other agents) on Windows pass file paths in the hook payload as `C:\Users\Prabhat ACER\.ssh\id_rsa` — backslashes — so the contains() check never hits, the rule never fires, credential-write protection fails open. Fix: normalize backslashes to forward slashes when building `ActionPolicyContext.file_path`. One platform-canonical form in CEL scope. Forward slashes work as separators on Windows APIs we touch, so the canonical form is also functionally valid. Pinning regression test `build_action_policy_context_normalizes_windows_backslash_paths` covers the engineer's actual `Prabhat ACER` path shape. 166 lib tests passing. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Feat/soth code extension
Engineer reported the previous fix didn't ship the expected
quoted form for spaces in `C:\Users\Prabhat ACER\…` — likely
they had a binary that predated `4955705`, but they also asked
"check how gryph solves it or offload it" so the more robust
implementation is worth landing now.
Two improvements:
1. `quote_binary_path` now defers to `shlex::try_quote` for
the rare case where the binary path embeds a literal `"`.
For all common paths (no metachars) it returns the
double-quote-wrapped form — same wire output as before, but
battle-tested escape rules cover the long tail of edge
cases that hand-rolled quoting misses. Wire form for the
engineer's actual `Prabhat ACER` shape: settings.json gets
`"command": "\"C:\\Users\\Prabhat ACER\\.local\\bin\\soth.exe\"
code hook --agent claude_code --type pre_tool_use"`.
2. Plugin template substitution (`__SOTH_BIN__` in opencode.mjs
and piagent.ts) now escapes backslashes and embedded quotes
for JS string-literal context. Without this, the raw
Windows path `C:\Users\Prabhat ACER\.local\bin\soth.exe`
inside a JS `"…"` string literal would either parse-error
(`\U` is an invalid escape) or silently produce the wrong
path because JS interprets `\b` / `\n` / etc. After:
`const SOTH_BIN = "C:\\Users\\Prabhat ACER\\.local\\bin\\soth.exe";`
parses correctly on every platform.
166 tests passing. Engineer needs to download the published
binary post-commit and re-run `soth code install` to refresh
their hooks.json with the new quoted form.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Cross-research of Claude Code + Cursor Windows hook executors
plus cmd.exe `/C` parsing rules confirms our existing
double-quote wrapping is correct, with one strict improvement:
emit forward slashes instead of backslashes.
Why forward slashes:
1. cmd.exe, PowerShell, and Git Bash (Claude Code + Cursor
2.x both shell out via Git Bash on Windows by default)
all accept `C:/Users/Prabhat ACER/.local/bin/soth.exe` as
a valid binary path. Forward slashes never collide with
JSON or shell escaping rules.
2. Backslashes in JSON strings need `\\` escaping; in a
hand-edited settings.json that's a footgun. Forward
slashes serialize as themselves.
3. Anthropic Claude Code issue #16451 (`C:\Users\Burak
Demir`) showed the failure mode bites both backslash
plus space cases. Forward slashes eliminate the
backslash side; the existing double-quote wrapping
handles the space.
What `~/.claude/settings.json` looks like after install on
Windows post-fix:
{"command": "\"C:/Users/Prabhat ACER/.local/bin/soth.exe\"
code hook --agent claude_code --type pre_tool_use"}
— deserializes to a shell command that cmd.exe / Git Bash
invoke correctly.
Sources confirming the approach:
- safedep/gryph dodges the problem by hardcoding bare
`gryph` (assumes PATH). We can't copy that since soth
installs to `~/.local/bin/soth.exe`.
- code.claude.com/docs/en/hooks: Claude Code routes the
command field through a shell ("bash" default,
"powershell" optional). Double-quote wrapping is the
documented fix.
- Cursor on Windows uses Git Bash for hooks
(forum.cursor.com/t/cursor-hooks-on-windows/140293,
obra/superpowers#871).
- ss64.com cmd.exe `/C` rules: with `"executable" args` the
leading double-quote is preserved and the executable is
parsed correctly.
Test updated:
- `install_command_quotes_binary_path_with_spaces` now
asserts the exact forward-slash + double-quote form.
- `install_claude_code_writes_quoted_command_for_space_path`
end-to-end asserts the JSON output shape engineers expect.
166 tests passing.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…oting
User asked whether we could offload the Windows path / shell-
quoting concern to a library. Tried `path-slash` (the closest
fit for backslash → forward-slash) and reverted: it branches
on the host OS's separator at runtime, so a Windows-shaped
path tested on a macOS dev box doesn't get converted. Wrong
tool for our case where we're producing a string that targets
a Windows-or-POSIX shell regardless of the host that wrote it.
Surveyed the rest of the Rust path / shell ecosystem:
- `shlex` (already in defense-in-depth path) — POSIX only
- `shell-escape` — has separate POSIX + cmd.exe variants but
forces choosing one shell at write time; we can't (Claude
Code uses Git Bash on Windows by default but cmd.exe is
fallback)
- `shell-words` — POSIX only
- `dunce` — strips `\\?\` extended-path prefix; we use
`display()` not `canonicalize()` so we never see one
- `normpath` — file-system normalization, not shell output
None of them solve "produce a JSON-safe shell-string that
works on cmd.exe + PowerShell + Git Bash + bash + zsh." Our
problem sits at the intersection of three concerns (shell
quoting, JSON escaping, path separator portability) — too
narrow for a library.
Net: keep the 1-line `replace('\\', "/")` + double-quote
wrap. Comment block updated to record why so a future reader
doesn't re-tread the path-slash route. Same JSON output as
before; no behaviour change.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
THE actual root cause of "every Cursor hook rejected on Windows" —
masked by the earlier install-quoting fix.
Cursor on Windows (Electron child_process.spawn) prepends a
UTF-8 BOM (0xEF 0xBB 0xBF) to JSON piped on the hook
subprocess's stdin. serde_json::from_slice rejects it with
`expected value at line 1 column 1`, so adapter.parse_event
errors before the rest of the hook pipeline runs and the
agent's command goes through unblocked.
The engineer found Cursor's exit-0 invocation log + the soth
stderr reply showed:
Command: "C:\Users\...\soth.exe" code hook --agent cursor
STDERR: {"error":"adapter parse: invalid JSON in hook stdin:
expected value at line 1 column 1"}
— proving the install-quoting fix shipped earlier already made
Cursor execute the right command, but the BOM blocked the
parse.
Why we missed it adopting from gryph: gryph upstream has the
identical latent bug. `gryph/cli/hook.go:55` does
`io.ReadAll(os.Stdin)` and passes raw bytes to
`json.Unmarshal`; `gryph/agent/cursor/parser.go:204` calls
`json.Unmarshal(rawData, ...)` with no preprocessing. Go's
`encoding/json` doesn't tolerate BOMs either (golang/go#12254
still open). Gryph hasn't been bitten because the BOM only
appears on Cursor's Windows builds and most reporters run
macOS / Linux. Worth filing upstream as well.
Fix: strip a leading UTF-8 BOM in two places for defense in
depth:
1. `read_stdin_to_end` — covers the production hook path
where the supervisor pipes stdin to the subprocess.
2. The top of `run_hook` — covers integration tests +
any future caller that constructs stdin elsewhere.
Both no-op when no BOM is present.
Regression test `run_hook_strips_utf8_bom_from_cursor_stdin`
pins the behaviour against a real Cursor `beforeSubmitPrompt`
shape. Test would fail in <1 line of code if the strip
regresses.
168 tests passing. After this lands, every Cursor hook on
Windows parses cleanly and dashboard activity for that agent
unblocks.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
On minimal/server Linux images the `gsettings` binary may exist while the GNOME schemas are absent. The previous check (`which gsettings`) was sufficient to enter the GNOME path but `gsettings set org.gnome.system.proxy …` then failed with `No schemas installed`, which bubbled up as a `soth up` failure and rolled back the daemon — leaving the user with no proxy at all despite the env-var fallback being viable. Add a `gnome_proxy_schema_available()` probe that lists schemas and only takes the GNOME path when `org.gnome.system.proxy` is actually registered. Otherwise fall through to the KDE / env-var instructions, which work on headless and DE-less hosts. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two TelemetryPipelineConfig/SyncAgentConfig builders were
hardcoding the literal string "soth-proxy-dev" as the proxy
version. Production binaries reported this verbatim, which then
surfaced on the dashboard as "soth-proxy-dev" (or worse,
"vsoth-proxy-dev" once the dashboard prepended "v" for the
semver display).
Switch both call sites to env!("CARGO_PKG_VERSION"), matching
the pattern already used by soth-extensions/src/context.rs.
Workspace version is "0.1.0" so the dashboard now sees a real
semver instead of a dev placeholder.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Feat/soth code extension
fix(proxy): report real CARGO_PKG_VERSION instead of "soth-proxy-dev"
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.