From d68fb2075656e26d002106dc4b6493abe6340247 Mon Sep 17 00:00:00 2001 From: Apoorv Raj Saxena Date: Thu, 30 Apr 2026 12:05:20 +0530 Subject: [PATCH 01/24] docs(sdk): pre-flight specs - Decision API + WASM trust boundary MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Two pre-flight specs that block Plan 2 Phase 0 (soth-sdk-core scaffolding). Neither is implementation; both lock the public-API contracts the SDK will ship in customer dependencies. Once Phase 0 commits the types, every change to these surfaces is breaking for downstream code. docs/common/SDK_DECISION_API_SPEC.md (~560 lines) - Decision enum: Allow / Block / Redact / Flag (Reroute dropped — proxy-only) - BlockReason variants including UseAlternative for reroute-shaped policies - MessageRedactions: message-level replacement only; within-message edits are documented as proxy-only - DecisionToken lifecycle: created in pre_call, consumed once in post_call, slab-backed, panic-on-reuse-in-debug, log-and-ignore in release, 60s orphan sweeper, sentinel for slab-full - Wrapper exception contract per language: SothBlocked extends base Exception/Error (NOT openai.APIError / etc.), propagates past existing try/except API-error handlers; gating negative tests required per binding - Sync (≤5ms p99) vs async (≤300ms p99) budget with explicit allowed work - Conformance harness extension: compare_decisions() asserts variant + BlockReason kind + redact message_idx set; proxy_only fixture tagging docs/common/SDK_WASM_TRUST_BOUNDARY_SPEC.md (~450 lines) - ORT-Web spike result: realistic full-mode payload is 22-27MB compressed, not the original ≤8MB target (which was wrong) - Tier matrix: native = full local; browser/Bun/Lambda = full WASM; CF Workers / Vercel Edge = reduced; Fastly = full - ClassificationMode enum: Full | Reduced | CloudOptIn (no auto-promotion) - Reduced-mode capability statement: explicit list of what works (artifact redaction, counter-based anomalies, artifact policy, session dedup, telemetry shipping) and what doesn't (use_case_label, semantic anomalies, ML-conditioned org rules) - Cloud-classify wire format: explicit opt-in with separate DPA, never fallback, classification_location: "cloud" telemetry tag - Bundle CDN trust path: Ed25519 verify, hot-swap via ArcSwap, no on-disk cache option for serverless, no key-fetching endpoint - Conformance lanes: SDK-via-PyO3, SDK-via-WASM-reduced, SDK-via-WASM-cloud-optin (with stubbed local endpoint for hermetic CI) Both specs include "If this needs to change" sections so the lock-in is deliberate, not accidental. Co-Authored-By: Claude Opus 4.7 (1M context) --- docs/common/SDK_DECISION_API_SPEC.md | 559 ++++++++++++++++++++ docs/common/SDK_WASM_TRUST_BOUNDARY_SPEC.md | 447 ++++++++++++++++ 2 files changed, 1006 insertions(+) create mode 100644 docs/common/SDK_DECISION_API_SPEC.md create mode 100644 docs/common/SDK_WASM_TRUST_BOUNDARY_SPEC.md diff --git a/docs/common/SDK_DECISION_API_SPEC.md b/docs/common/SDK_DECISION_API_SPEC.md new file mode 100644 index 00000000..fd3f8d64 --- /dev/null +++ b/docs/common/SDK_DECISION_API_SPEC.md @@ -0,0 +1,559 @@ +# SDK Decision API Specification + +**Status:** Drafted — 2026-04-30 +**Scope:** Public API contract for `soth-sdk-core` and every language binding +(`soth-py`, `soth-node`, `soth-go`, `soth-wasm`) +**Blocks:** Plan 2 Phase 0 — `soth-sdk-core` cannot lock public types until +this spec is signed off. + +--- + +## 1. Why this spec exists + +The SDK's job is half observation, half enforcement. The original Plan 2 +draft specified an observation-only API (`observe_request → Observation`, +`observe_response`); the Plan 1 review correctly pointed out that without a +synchronous decision-return, customers integrating the SDK get telemetry +and audit but lose enforcement, which is half the product. A credential +detected in a request must never reach the upstream provider — that's the +contract, regardless of whether the proxy or the SDK is the integration +point. + +This spec locks the public types, the lifecycle, the per-language wrapper +contract, and the conformance assertions that prove parity with the proxy. +It is the API the SDK ships in customer dependencies — once committed, +every change is a breaking change for downstream code. + +--- + +## 2. Scope decisions + +### 2.1 Reroute is out of the SDK Decision enum + +**Decision:** The SDK does not return `Decision::Reroute`. Reroute remains +a proxy/sidecar capability. + +**Rationale:** In the proxy, `Reroute` means "I'm intercepting this request +and forwarding it to a different upstream — your code doesn't see the +swap." In the SDK, the customer's code constructed a typed client pointing +at OpenAI; the SDK can't redirect the underlying connection without +substituting the entire client object, which would be a deeply intrusive, +provider-specific operation that breaks the customer's typing. + +**What the SDK does instead:** Reroute-shaped policies surface as +`Decision::Block { reason: BlockReason::UseAlternative { suggested_model, +suggested_provider } }`. The customer's app implements +retry-with-alternative if it wants. The proxy continues to do real reroute. +Same policy in the bundle, different runtime behavior depending on +enforcement location, documented as such. + +**Conformance impact:** Fixtures that exercise reroute policies are tagged +`proxy_only: true` in the conformance corpus and are not asserted on the +SDK lane. + +### 2.2 Redact is scoped to message-level replacement + +**Decision:** `Decision::Redact` returns a list of message-index → +replacement-content edits. The SDK does not perform within-message +surgical edits. + +**Rationale:** Within-message redaction (e.g. scrub a credential from the +middle of a paragraph) requires per-provider schema knowledge that grows +linearly with provider count and breaks every time a provider SDK updates +its content type. Message-level replacement only depends on the existence +of a `messages` array and is uniformly implementable across providers. + +**What the SDK does:** `MessageRedactions` carries a `Vec<(MessageIdx, +RedactedContent)>`. The wrapper applies these via a per-provider adapter +that calls `provider.replace_messages(typed_call, redacted_messages)`. +Each provider adapter is <100 LOC. + +**Documented limitation:** Customers wanting fine-grained within-message +redaction use the proxy. The SDK README states this explicitly. + +**Conformance impact:** Fixtures that require within-message redaction +are tagged `proxy_only: true`. + +--- + +## 3. Public types + +These types live in `soth-sdk-core` and are re-exported through every +language binding. **Once committed, changes are breaking.** + +### 3.1 `Decision` + +```rust +pub enum Decision { + Allow { + token: DecisionToken, + }, + Block { + token: DecisionToken, + reason: BlockReason, + }, + Redact { + token: DecisionToken, + redactions: MessageRedactions, + }, + Flag { + token: DecisionToken, + severity: FlagSeverity, + }, +} +``` + +`#[non_exhaustive]` on `Decision`, `BlockReason`, `FlagSeverity` so +future variants can be added without a major bump. + +### 3.2 `BlockReason` + +```rust +#[non_exhaustive] +pub enum BlockReason { + /// Sensitive artifact detected (credential, private key, PII). + SensitiveArtifact { + kind: ArtifactKind, + severity: ArtifactSeverity, + }, + /// Org-level budget exhausted (token / cost / request count). + BudgetExceeded { + budget_kind: BudgetKind, + observed: u64, + limit: u64, + }, + /// Org policy rule fired. + PolicyRule { + rule_id: String, + rule_name: Option, + }, + /// Reroute-shaped policy surfaced as a block with an alternative. + /// Customer apps that implement retry-with-alternative use this. + UseAlternative { + suggested_provider: Option, + suggested_model: Option, + rule_id: String, + }, +} +``` + +### 3.3 `MessageRedactions` + +```rust +pub struct MessageRedactions { + pub replacements: Vec, +} + +pub struct MessageRedaction { + /// Index into `LlmCall.messages`. The wrapper's provider adapter + /// uses this to locate the message in the typed call object. + pub message_idx: usize, + /// Content that replaces the original message. Always set; the + /// SDK never returns "delete this message" — it returns a + /// placeholder so conversation turn structure is preserved. + pub redacted_content: String, + /// Reason surface — telemetry tags so the cloud can correlate + /// what triggered the redaction. + pub reason: RedactReason, +} + +#[non_exhaustive] +pub enum RedactReason { + SensitiveArtifact { kind: ArtifactKind }, + PolicyRule { rule_id: String }, +} +``` + +### 3.4 `DecisionToken` + +```rust +pub struct DecisionToken { + /// Opaque identifier — bindings must not interpret this. Carried + /// from `pre_call` to `post_call` so the SDK can correlate the + /// decision with the response. + inner: u64, +} +``` + +`DecisionToken` is `Copy` and lock-free. The `inner` field is private; +the SDK uses it as a key into a slab of pending decisions. + +### 3.5 `FlagSeverity` + +```rust +#[non_exhaustive] +pub enum FlagSeverity { + Info, + Warning, + Critical, +} +``` + +`Flag` is enforcement-cooperative: the wrapper does not raise an +exception; it logs a structured event and the call proceeds. The +customer's observability stack picks the flag up via OTel spans. + +--- + +## 4. `SothSdk` API + +### 4.1 Synchronous decision path + +```rust +impl SothSdk { + /// Returns within 5 ms p99 budget. Runs: + /// 1. Per-org budget check (atomic counter, no I/O) + /// 2. System rules conditioned on artifacts only + /// 3. Heuristic credential scan (regex pass) + /// 4. Org rules conditioned on artifacts (NOT classified labels) + /// Does NOT run: embedding, cluster, use_case, semantic anomaly, + /// org rules conditioned on classified labels. + pub fn pre_call(&self, call: &LlmCall) -> Decision; + + /// Async enrichment + telemetry emission. Must NOT be called on + /// the host's critical path. Bindings spawn this on a worker thread + /// or async task. Runs: + /// - Embedding (ONNX or cloud, depending on tier) + /// - Cluster, use_case, semantic anomaly + /// - Org rules conditioned on classified labels + /// - Final telemetry emission (enriched event) + pub fn post_call(&self, token: DecisionToken, resp: &LlmResponse); +} +``` + +### 4.2 Streaming variants + +```rust +impl SothSdk { + /// Streaming counterpart to `pre_call`. Returns a Decision on the + /// initial call and an observation handle for the chunk stream. + /// If Decision is Block, `chunk` and `end` are never called. + pub fn stream_begin(&self, call: &LlmCall) -> (Decision, StreamObservation); + + /// Per-chunk update. Cheap; no I/O. Bindings call this once per + /// chunk emitted by the provider stream. + pub fn stream_chunk(&self, obs: &mut StreamObservation, chunk: &LlmChunk); + + /// Stream end. Equivalent to `post_call` for streaming responses. + /// Telemetry emission happens here. + pub fn stream_end(&self, obs: StreamObservation); +} +``` + +### 4.3 Bundle refresh + +```rust +impl SothSdk { + /// Pull a new bundle from the configured CDN URL, verify signature, + /// hot-swap via ArcSwap. Customer schedules this (cron, background + /// task, dashboard webhook). The SDK does NOT spawn its own + /// refresher. + pub fn refresh_bundle(&self) -> Result<(), SdkError>; +} +``` + +--- + +## 5. `DecisionToken` lifecycle + +### 5.1 Creation + +`DecisionToken` is allocated by `pre_call` / `stream_begin`. Internally it +indexes into a fixed-size slab (default 4096 slots) holding the pending +decision context (artifacts, anomaly signals, partial classification +state, redaction list). + +### 5.2 Consumption + +A token is consumed exactly once by `post_call` / `stream_end`. Consumption: +- Marks the slab slot free +- Triggers async classify enrichment using the partial state +- Emits the final telemetry event + +### 5.3 Token reuse + +**Debug builds:** panic with a clear message identifying the offending +binding. + +**Release builds:** log the reuse at WARN level with the token ID and +counter value, then ignore the reuse silently. The host call continues +unaffected. + +Rationale: token reuse is a binding bug, not a customer bug. We want to +catch it loudly during binding development (panic in debug); we do not +want to crash a customer's prod app if a binding bug slips through. The +log line is the trigger for the binding team to fix the leak. + +### 5.4 Never consumed (orphaned) + +A timeout-driven sweeper checks slab slots older than a configurable +threshold (default 60s). Orphaned slots: +- Emit a telemetry event tagged `decision_orphaned: true` with the + partial state available +- Free the slab slot + +Rationale: customers using cancellation, retries, or framework wrappers +that drop the response can leave tokens orphaned. The 60s window is +generous enough to cover even slow provider streams; the telemetry tag +lets the cloud distinguish orphans from completed calls. + +### 5.5 Slab pressure + +When the slab is at 95% capacity, `pre_call` switches to a degraded +mode: it still returns a `Decision`, but the embedded token is the +sentinel `DecisionToken::SLAB_FULL`. The wrapper treats this token +identically; `post_call(SLAB_FULL, ...)` is a no-op and emits a +telemetry event tagged `slab_full_no_enrichment: true`. + +This guarantees `pre_call` never blocks or fails on slab pressure. The +slab-full event is the signal to size up. + +--- + +## 6. Wrapper exception contract + +### 6.1 The contract + +For every language binding: + +1. `Decision::Block` is translated into a host-language exception + named `SothBlocked` (Python) / `SothBlocked` (TypeScript) / etc. +2. `SothBlocked` inherits from the language's **base exception type** + (`Exception` in Python, `Error` in TypeScript), NOT from the + provider SDK's exception hierarchy (`openai.APIError`, + `anthropic.APIError`, etc.). +3. `SothBlocked` propagates past existing + `try/except openai.APIError` blocks. **This is intentional behavior.** +4. Wrappers MUST NOT catch `SothBlocked` and re-raise it as a + provider-specific exception type. +5. Wrappers MUST NOT wrap `SothBlocked` inside a provider exception. + +### 6.2 Python contract + +```python +class SothBlocked(Exception): + """Raised when SOTH policy blocks an LLM call. + + Inherits from Exception, NOT from any provider SDK exception type. + Will propagate past `try/except openai.APIError` handlers — this is + intentional. A policy block is not an upstream API error and must + not be retried by retry-on-API-error logic. + """ + decision_id: str + reason: BlockReason + policy_rule_id: Optional[str] +``` + +### 6.3 TypeScript contract + +```typescript +export class SothBlocked extends Error { + constructor( + public readonly decisionId: string, + public readonly reason: BlockReason, + public readonly policyRuleId?: string, + ) { + super(`SOTH policy blocked call: ${reason.kind}`); + this.name = 'SothBlocked'; + } +} +``` + +`extends Error` — does NOT extend `OpenAI.APIError` or any provider's +error type. + +### 6.4 Per-binding test invariants + +Each binding ships negative tests that prove: + +```python +# Python — must pass +import openai +try: + client.chat.completions.create(...) # blocked +except openai.APIError: + assert False, "SothBlocked must not be caught by openai.APIError" +except SothBlocked: + pass # correct +``` + +```typescript +// TypeScript — must pass +try { + await client.chat.completions.create({ ... }); // blocked +} catch (e) { + if (e instanceof OpenAI.APIError) { + throw new Error("SothBlocked must not be caught by OpenAI.APIError"); + } + if (e instanceof SothBlocked) { + // correct + } +} +``` + +These tests are gating: a binding cannot ship without them passing. + +### 6.5 Rationale + +A policy block is a SOTH decision, not an upstream API error. Customers +with retry-on-API-error logic (almost everyone) would silently retry +blocked calls if `SothBlocked` inherited from the provider's exception +hierarchy, which would defeat the purpose of enforcement. The +propagate-past behavior is the correct semantic; the docs explain it +clearly so customers add explicit `except SothBlocked` handlers when +they want graceful degradation. + +--- + +## 7. Sync vs async budget + +### 7.1 `pre_call` budget — synchronous, ≤5 ms p99 + +Allowed work: +- **Budget check** — atomic counter read + compare. ~1 µs. +- **System rules** — fixed set of artifact-conditioned rules + (private_key, aws_access_key, openai_secret, etc.). All regex; no + ML. ~10–500 µs depending on body size. +- **Heuristic credential scan** — same regex pass as the proxy's + `credential_scan_str`. Already optimized; ~100 µs–1 ms. +- **Org rules — artifact branch only** — rules that match on + `artifacts[].kind`, NOT on classified labels. ~50 µs per rule. + +Forbidden work: +- ONNX inference (not in budget) +- LSH cluster lookup (depends on embedding) +- Any I/O — telemetry, network, disk +- Any allocation > 1 KB + +### 7.2 `post_call` / `stream_end` budget — async, ≤300 ms p99 + +Allowed work: +- **Embedding** — ONNX inference (full mode) or cloud round-trip + (cloud-classify mode). Dominant cost. +- **Cluster, use_case, volatility, semantic anomaly** — pure CPU, + ~5–20 ms total. +- **Org rules — full branch** — rules conditioned on classified labels. + ~50 µs per rule. +- **Telemetry emission** — serialize event, push to in-memory queue. + Cloud transport is its own background task. + +Allowed I/O: +- HTTPS POST to soth-cloud telemetry endpoint (background task) +- HTTPS GET to cloud-classify endpoint (cloud-classify mode only) + +### 7.3 Budget enforcement + +Bindings instrument both paths with histograms. CI gating: +- `pre_call` p99 < 5 ms over 10 K synthetic calls +- `post_call` p99 < 300 ms over 1 K synthetic calls (with mocked + cloud endpoints to keep CI hermetic) + +Any binding PR that regresses these gates fails CI. + +### 7.4 Failure modes + +`pre_call` panic → caught at FFI boundary, return +`Decision::Allow { token: SENTINEL_TOKEN }`, log at ERROR. The host +call proceeds. + +`post_call` panic → caught at FFI boundary, log at ERROR, drop the +slab slot. No host-visible effect. + +Cloud-classify failure → log at WARN, emit telemetry event tagged +`cloud_classify_failed: true` with whatever local state is available +(Unknown use_case_label, partial anomaly flags). + +--- + +## 8. Conformance harness assertions + +The `soth-conformance-tests` corpus from Plan 1 PR 5 is extended for the +SDK lane. + +### 8.1 Strict-parity Decision assertions + +For every fixture not tagged `proxy_only: true`, the harness asserts +`Decision` equality between the proxy lane and the SDK lane: + +- **Variant equality** — `Decision::Allow / Block / Redact / Flag` + must match. +- **`BlockReason` equality** — when both are `Block`, the inner + `BlockReason` must match (same `kind`, same `rule_id` for + `PolicyRule`, etc.). +- **`MessageRedactions` parity** — when both are `Redact`, the set of + `message_idx` values must match. The replacement content does NOT + need to be byte-identical (proxy may use a different placeholder + string); only the set of redacted indices is asserted. +- **`FlagSeverity` equality** — when both are `Flag`, severity must + match. + +### 8.2 Excluded from strict comparison + +- `DecisionToken` (per-call ephemeral, never compared) +- `RedactReason` (telemetry tag; harness asserts presence but not exact + match across lanes) +- `BlockReason::PolicyRule.rule_name` (cosmetic; only `rule_id` is + asserted) + +### 8.3 Proxy-only fixtures + +Fixtures tagged `proxy_only: true` skip the SDK-lane assertion entirely. +Today's tags: + +- Reroute-policy fixtures (SDK doesn't return Reroute) +- Within-message-redaction fixtures (SDK is message-level only) +- gRPC fixtures (until SDK ships gRPC support) + +### 8.4 Asymmetric fixtures + +The harness's `compare()` must not fail when the SDK Decision contains +a richer `BlockReason` than the proxy provides (e.g. SDK adds +`UseAlternative` while proxy returns plain Reroute). This is normal +graduation; the proxy will catch up over time. + +### 8.5 Conformance harness extension + +`crates/soth-conformance-tests/src/lib.rs` adds: + +```rust +pub fn compare_decisions( + proxy_decision: &Decision, + sdk_decision: &Decision, +) -> Vec; +``` + +A new dimension in the existing `Diff` output: `decision.variant`, +`decision.block_reason.kind`, `decision.redact.message_indices`. + +--- + +## 9. Open questions / deferred + +- **Async runtime in `post_call`**: bindings need to spawn `post_call` + off-thread. Python uses `concurrent.futures.ThreadPoolExecutor`; + Node uses `napi-rs`'s threadpool; Go uses a goroutine. The Rust core + exposes `post_call` as a blocking function; bindings own the + threading. **Confirmed.** +- **Nested calls** (an LLM tool call that itself triggers another LLM + call): each call gets its own `DecisionToken`. The `LlmCall.context` + carries a `parent_decision_id` so the cloud can reconstruct the + call tree. Spec'd separately in a follow-up. +- **Bidirectional streaming** (e.g. realtime audio): treated as a + long-lived stream. `stream_chunk` runs per audio frame; `stream_end` + is the session terminator. Latency budget for `stream_chunk` is + even tighter (≤500 µs p99). Spec'd separately if the realtime API + becomes a target. + +--- + +## 10. If this needs to change + +Any change to `Decision`, `BlockReason`, `MessageRedactions`, +`DecisionToken` lifecycle, or the wrapper exception contract is a +**breaking change** for every binding and every customer downstream. +Roll forward with semver: bump major on `soth-sdk-core`, bump major +on every binding, document migration in CHANGELOG. + +The conformance harness is the safety net: any divergence from this +spec that affects strict-parity fixtures fails CI, naming the +diverging field. Use that signal — don't override it. diff --git a/docs/common/SDK_WASM_TRUST_BOUNDARY_SPEC.md b/docs/common/SDK_WASM_TRUST_BOUNDARY_SPEC.md new file mode 100644 index 00000000..47e04b64 --- /dev/null +++ b/docs/common/SDK_WASM_TRUST_BOUNDARY_SPEC.md @@ -0,0 +1,447 @@ +# SDK WASM Trust-Boundary Specification + +**Status:** Drafted — 2026-04-30 +**Scope:** `soth-sdk-core` WASM target and the runtime tiers it serves +**Blocks:** Plan 2 Phase 0 — public types in `soth-sdk-core` depend on +which classification mode is the default per runtime. +**Companion spec:** `SDK_DECISION_API_SPEC.md` + +--- + +## 1. Why this spec exists + +The original Plan 2 draft positioned cloud-classify as a "graceful +fallback" for WASM targets where local ONNX won't fit. The Plan 1 review +correctly flagged this as a violation of the architectural trust anchor: +**content never leaves the customer's environment**. Routing prompt text +to soth-cloud for embedding breaks the first paragraph of the master +trust model, regardless of how fast or cheap it is. + +This spec replaces the fallback model with a tiered model: + +- **Native bindings** (Python / Node) — full local ONNX, content stays + local, default everywhere. +- **WASM where size permits** (browser, Bun, Deno, Lambda) — full local + via ONNX Runtime Web, content stays local, default. +- **WASM where size doesn't permit** (Cloudflare Workers, Vercel Edge) — + reduced mode, no semantic classification, content stays local, default. +- **Cloud-classify** — opt-in privacy-tier with separate DPA, + explicitly enabled per-customer, telemetry-tagged. **Never the + default.** **Never a fallback.** + +The phrasing "graceful fallback to cloud-classify" must not appear in +any customer-facing surface. If a customer ends up using cloud-classify, +it's because they signed a DPA that authorizes it. + +--- + +## 2. ORT-Web compressed-size spike result + +Pre-flight measurement before the tier matrix was committed: + +| Component | Uncompressed | Brotli compressed | +|---|---|---| +| ORT Web (wasm-simd-threaded build) | ~10–15 MB | ~3–5 MB | +| all-MiniLM-L6-v2 INT8 ONNX model | ~23 MB | ~18–20 MB *(weights compress poorly)* | +| HuggingFace `tokenizers` to WASM | ~1–2 MB | ~1 MB | +| **Total realistic full-mode payload** | **~34–40 MB** | **~22–27 MB** | + +**Original target ≤8 MB compressed: not feasible** with the current +embedding model. Confirmed via direct measurement against +`onnxruntime-web@1.17` and the model HuggingFace publishes. The +conclusion below stands; only the threshold numbers are corrected. + +A future workstream may distill or quantize the embedding model to +fit the smaller-runtime targets — tracked in §6 as deferred. + +--- + +## 3. Tier matrix + +### 3.1 Targets and modes + +| Target | Compressed budget | Classification mode | Notes | +|---|---|---|---| +| Python (PyO3, native) | unbounded | **Full (local ONNX)** | Default; primary Tier-1 surface. | +| Node.js / Bun (napi-rs, native) | unbounded | **Full (local ONNX)** | Default; primary Tier-1 surface. | +| Browser (WASM) | ~30 MB practical | **Full (ONNX Web)** | Default; full mode loadable. | +| Bun (WASM, where applicable) | unbounded | **Full (ONNX Web)** | Default. | +| Node.js (WASM, edge-style) | unbounded | **Full (ONNX Web)** | Default. | +| Deno Deploy | ~10 MB script | **Full (ONNX Web)**, monitor | Tight; revisit if compressed bundle > 8 MB. | +| AWS Lambda | unbounded (50 MB unzipped) | **Full** (native via cdylib preferred, ONNX Web acceptable) | Cold-start cost is the real constraint, not bundle size. | +| Fastly Compute | 100 MB | **Full (ONNX Web)** | Default. | +| Cloudflare Workers (free / paid) | 1–10 MB | **Reduced** | Doesn't fit ONNX Web. | +| Vercel Edge Functions | 1–4 MB | **Reduced** | Doesn't fit ONNX Web. | +| CloudFront Functions | < 1 MB | **Reduced** (lite) | Tightest; may not fit even reduced mode without further work. | + +### 3.2 Selection logic + +`SdkConfig.local_classification` is the customer-facing dial: + +```rust +pub enum ClassificationMode { + /// Local embedding via ONNX (native or ONNX Web). Content never + /// leaves the customer's environment. Default for all targets + /// where it fits. + Full, + + /// No local embedding. Heuristic detect + counter-based anomaly + + /// artifact-based policy. Telemetry events emit with + /// `use_case_label: Unknown`. Default for size-constrained edge + /// runtimes. + Reduced, + + /// Customer has explicitly opted in to send normalized request + /// data to soth-cloud for classification. Requires a separate DPA + /// covering content egress. Telemetry events tagged + /// `classification_location: "cloud"`. + CloudOptIn, +} +``` + +The SDK does **not** auto-promote `Reduced` to `CloudOptIn` based on +runtime detection. Promotion is a deliberate customer config change +backed by a DPA. + +### 3.3 What `Full` actually does on each target + +Native: links `ort` directly via `cdylib`. ONNX runtime executes in-process, +bundle's ONNX model loaded from memory bytes (no disk required for the +model itself; bundle cache is opt-in via `bundle_cache_dir`). + +WASM: links `ort` compiled to `wasm32-unknown-unknown`, loaded by the +host runtime (browser via `WebAssembly.instantiate`, Node via +`WebAssembly` global, Bun similar, Deno similar). The host runtime +provides the threading + SIMD primitives. + +Workers AI alternative — explicitly NOT used: the trust anchor requires +embeddings to be computed locally. Workers AI runs ONNX on Cloudflare's +edge GPUs; that's still off-customer-infrastructure egress. Reduced +mode is the honest answer for Workers, not "delegate embedding to +Cloudflare's GPUs." + +--- + +## 4. Reduced-mode capability statement + +When `local_classification: Reduced`, the SDK delivers: + +### 4.1 Available + +- **Sensitive-artifact redaction** — full credential / PII / private-key + detection via the existing regex pipeline. No model needed. +- **Counter-based anomaly flags** — `TokenBurst`, `CredentialBurst`, + `ModelSwitch`, `RapidFireRequests`, `ToolCallDepthSpike`. All driven + by session-state counters; no embedding required. +- **Artifact-based policy** — block on credential leak, block on + private-key, redact on PII match. Doesn't need classified labels. +- **Session-level dedup** — `is_prefix_repeat`, `prefix_hash` flow + through unchanged from `process_normalized`'s session phase. +- **Telemetry shipping** — full telemetry events emit; only the + semantic fields are sentinels. + +### 4.2 Unavailable (telemetry sentinel values) + +- `use_case_label: UseCaseLabel::Unknown` (never one of the 16 trained + labels) +- `volatility_class: VolatilityClass::Unknown` (degraded) +- `topic_cluster_id: 0` (sentinel; the cloud knows 0 means "no cluster + computed") +- `semantic_hash: ""` (empty) +- `embedding_norm: 0.0` +- `anomaly_flags`: missing the 3 semantic ones — + `TopicDrift`, `AgentLoopPattern`, `UnusualSystemPromptChange` +- Org policy rules conditioned on `use_case_label` or `topic_cluster_id` + do not fire (the rule short-circuits to a no-op with a tagged + telemetry event) + +### 4.3 Capability disclosure + +Every telemetry event emitted in reduced mode carries: + +```json +{ + "classification_mode": "reduced", + "classification_location": "local", + "missing_fields": ["use_case_label", "volatility_class", "topic_cluster_id", + "semantic_hash", "embedding_norm"] +} +``` + +The `missing_fields` list is canonical: cloud-side analytics filters +on it explicitly rather than treating empty/sentinel values as +"missing data" (which would be ambiguous with full-mode failures). + +### 4.4 Customer documentation requirement + +The reduced-mode capability statement appears verbatim in: +- The SDK README for every binding +- The customer dashboard's "SDK runtime modes" page +- The deploy-to-edge quickstart guides + +Customers picking edge-runtime targets must see this list before they +deploy. No silent feature regression. + +--- + +## 5. Cloud-classify (opt-in only) + +### 5.1 Activation + +Single explicit config flag: + +```rust +pub enum ClassificationMode { + // ... + /// Customer has explicitly opted in. Requires a DPA covering + /// content egress to soth-cloud's classify endpoint. + CloudOptIn, +} +``` + +There is no auto-promotion, no environment-variable shortcut, no +"if local is unavailable, fall back to cloud" path. The customer sets +`ClassificationMode::CloudOptIn` after their legal team has signed the +DPA addendum. + +### 5.2 Wire format + +Cloud-classify request (`POST /v1/edge/classify`): + +```json +{ + "request_id": "uuid-v4", + "org_id": "org-123", + "normalized": { + "provider": "openai", + "model": "gpt-4o", + "user_content": "", + "system_prompt": "", + "tool_definitions_json": "", + "endpoint_type": "ChatCompletion", + "is_streaming": false + }, + "session_snapshot": { /* abbreviated SessionSnapshot */ } +} +``` + +Response: + +```json +{ + "request_id": "uuid-v4", + "use_case_label": "CodeGeneration", + "use_case_confidence": 0.87, + "volatility_class": "LowVolatile", + "topic_cluster_id": 142, + "semantic_hash": "...", + "embedding_norm": 1.42, + "anomaly_flags": ["TopicDrift"], + "anomaly_score": 0.31, + "classification_location": "cloud" +} +``` + +Authentication: org-level bearer token (the same `api_key` used for +telemetry). + +Transport: HTTPS only, TLS 1.3, certificate pinned to soth-cloud's +public key, retry budget identical to telemetry sink (5 attempts with +exponential backoff, then dead-letter). + +Latency target: p99 < 50 ms round-trip from the customer's region. + +### 5.3 Telemetry tagging + +Every telemetry event from a `CloudOptIn` SothSdk instance carries: + +```json +{ + "classification_mode": "cloud_opt_in", + "classification_location": "cloud", + "cloud_classify_request_id": "uuid-v4" +} +``` + +The `cloud_classify_request_id` lets the cloud correlate the +classification request with the telemetry event for audit. + +### 5.4 DPA placeholder language + +The DPA addendum (legal team to finalize) covers: + +- Definition of "Customer Content" — prompt text, system prompts, + tool definitions +- soth-cloud's processing scope — classification only, no model + training, no third-party sharing, retention ≤ 90 days +- Customer's right to revoke at any time by reverting to + `ClassificationMode::Full` or `Reduced` +- Subprocessor list — the cloud-side classify infrastructure is run + by SOTH or its named subprocessors; updates require 30-day notice + +The actual DPA template is finalized by legal and not part of this +spec. This spec only commits that **CloudOptIn requires a DPA** and +defines the technical contract. + +### 5.5 What CloudOptIn does NOT change + +- Sensitive-artifact detection still runs locally on the SDK side. + Credentials, PII, private keys are detected and redacted before any + cloud round-trip happens. The cloud never receives detected + artifacts. +- Policy evaluation for artifact-conditioned rules runs locally. + The cloud only contributes classified labels. +- Telemetry event structure is identical to `Full` mode (cloud just + fills in the semantic fields the local pipeline can't). + +--- + +## 6. Bundle CDN trust path + +### 6.1 Bundle source + +`SdkConfig.bundle_url` (default: SOTH-hosted CDN endpoint). Bundle is +fetched at SDK init via HTTPS: + +``` +GET {bundle_url}/manifest.json +GET {bundle_url}/ # for each asset in manifest +``` + +### 6.2 Verification + +Reuse `soth-bundle::verify`: + +- Manifest signature (Ed25519, vendor public key embedded in SDK) +- Asset SHA256 + size match per manifest entry +- Manifest `expires_at` not in the past +- Optional: org approval signature when `require_org_approval: true` + in `SdkConfig` + +Verification failure → init returns `SdkError::BundleVerification`, +SDK enters no-op mode (logs warning; host calls proceed with +no-op `Decision::Allow`). The customer's observability picks up the +init error. + +### 6.3 Caching + +`SdkConfig.bundle_cache_dir`: +- `Some(path)` — verified bundle cached on disk; subsequent process + starts use the cache if `expires_at` not exceeded. +- `None` — no on-disk cache. Bundle pulled from CDN on every init. + Required for serverless / read-only-filesystem targets (Lambda, + Workers, edge functions). + +### 6.4 Refresh path + +Customer schedules `SothSdk::refresh_bundle()` (cron, background task, +dashboard webhook). The SDK does not spawn its own refresher. + +Refresh flow: +1. Fetch new manifest +2. If `version` matches current and `etag` matches, no-op (304-style) +3. Otherwise fetch + verify new assets +4. Hot-swap via `ArcSwap` — no allocations on the call path +5. Old bundle dropped after the swap completes (no in-flight calls + reference it) + +Refresh failure leaves the running bundle in place; logs at WARN. + +### 6.5 What lives in the bundle (recap from Plan 1) + +- `manifest.json` — version, sigs, expiry, scope (~1 KB) +- `classify/` — embedding ONNX, tokenizer, centroids, LSH projection, + use-case head (~30 MB) +- `policy/` — rule set (small) +- `redact/` — artifact patterns (small) +- `anomaly/` — thresholds + signal weights (tiny) +- `taxonomy/` — `UseCaseLabel` set + version (tiny) + +Two-tier shipping for SDKs: +- **Lite bundle (~200 KB)** embedded in the wheel/npm package — + patterns, policy, thresholds, taxonomy. Always available offline. +- **ML bundle (~30 MB)** pulled from CDN at first init, cached at + `bundle_cache_dir`. Customers in `ClassificationMode::Reduced` or + `CloudOptIn` never pull the ML bundle. + +### 6.6 No key fetching + +soth-cloud never possesses the customer's HMAC key. There is no +endpoint that returns it. The DPA, the SDK, and the CDN are explicit +on this point. Anyone proposing a key-fetch endpoint is breaking the +trust model — point them at this paragraph. + +--- + +## 7. Conformance assertions + +The conformance harness gains two new lanes for the SDK build: + +### 7.1 SDK-via-PyO3 lane + +Same fixture corpus as the existing SDK lane, but driven through the +Python binding's `pre_call` API (running the Rust core via PyO3 FFI). +Asserts that the Python wrapper does not introduce drift vs. the Rust +core directly. + +### 7.2 SDK-via-WASM-reduced lane + +Subset of fixtures that don't require semantic classification (use_case +or anomaly). Run through the WASM artifact in reduced mode. Asserts +that reduced mode produces the same artifact detection, capture mode, +policy decision, and telemetry shape as full mode for the fixtures it +can handle. + +### 7.3 SDK-via-WASM-cloud-optin lane + +Run against a stubbed local classify endpoint (a tiny test server) so +CI stays hermetic. Asserts that `CloudOptIn` mode produces a +`ClassifiedResult` with the same shape as full local mode, and that +the telemetry event correctly tags `classification_location: "cloud"`. + +### 7.4 Mode transitions + +Test that mode is honored: +- `ClassificationMode::Full` on a runtime that doesn't fit ONNX: SDK + init returns `SdkError::OnnxUnavailable`. Customer must explicitly + pick Reduced or CloudOptIn. +- `ClassificationMode::Reduced`: telemetry events carry the canonical + `missing_fields` list. +- `ClassificationMode::CloudOptIn` without `bundle_url` configured: + SDK init returns `SdkError::CloudClassifyEndpointMissing`. + +--- + +## 8. Open questions / deferred + +- **Smaller embedding model for size-constrained WASM** — distillation + or int4 quantization to fit ~5–10 MB compressed. Would let + Cloudflare Workers paid tier run full mode. Tracked as a follow-up + ML workstream; doesn't block SDK launch. +- **Per-call mode override** — should `pre_call` accept a `force_mode` + parameter for canary rollouts (e.g., 1% of traffic uses full, 99% + reduced)? Useful but not Tier-1; deferred until customer demand. +- **Bundle pull from a customer-controlled CDN** — `bundle_url` is + already configurable, but signature verification still uses the + vendor public key. Allowing org-signed bundles (org-approved + override) is in `soth-bundle::verify` but the SDK doesn't expose + the toggle yet. Deferred. + +--- + +## 9. If this needs to change + +Adding a new `ClassificationMode` variant: minor bump (variants are +`#[non_exhaustive]`). + +Changing the cloud-classify wire format: requires versioning the +endpoint (`/v2/edge/classify`) and a deprecation window. The SDK pins +its endpoint version explicitly in the request URL. + +Changing the trust anchor (e.g., allowing default content egress): +requires architectural review, customer notice, DPA revision. **Don't.** + +Reduced-mode capability changes: any new field that becomes available +in reduced mode graduates from the `missing_fields` list. Any new +field that becomes available only in full mode joins the list. The +customer dashboard's runtime-modes page must update synchronously. From b5c6f6dd742791ec7721b6bbc2ae707e941d5e9b Mon Sep 17 00:00:00 2001 From: Apoorv Raj Saxena Date: Thu, 30 Apr 2026 12:45:47 +0530 Subject: [PATCH 02/24] feat(sdk): Phase 0 - soth-sdk-core facade scaffolding MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit New workspace member crates/soth-sdk-core/ — the public-API surface every SDK binding (PyO3, napi-rs, WASM) consumes. Implements the contract locked by the two pre-flight specs landed in d68fb20: - docs/common/SDK_DECISION_API_SPEC.md - docs/common/SDK_WASM_TRUST_BOUNDARY_SPEC.md Public types (all #[non_exhaustive] where future variants are anticipated): Decision — Allow / Block / Redact / Flag (no Reroute) BlockReason — SensitiveArtifact / BudgetExceeded / PolicyRule / UseAlternative MessageRedactions — message-level only; within-message edits are proxy-only by design DecisionToken — opaque, Copy, slab-backed; SLAB_FULL + SENTINEL_FAIL_OPEN constants for backpressure / fail-open FlagSeverity — Info / Warning / Critical HmacKey — FromEnv / FromFile / FromCallback / Static; resolve() returns Zeroizing>; Debug never prints plaintext ClassificationMode — Full / Reduced / CloudOptIn StorageMode — InMemory / FileBacked BundleSource — Cdn / Embedded / Fallback SdkConfig + SdkConfigBuilder LlmCall (= TypedLlmCall), Message, Tool, LlmResponse, LlmChunk Observation, StreamObservation SdkError + SothSdk DecisionToken slab (src/slab.rs): 4096-slot fixed capacity, 95% pressure threshold returns SLAB_FULL per-slot generation counter detects token reuse panic-on-reuse in debug, log-and-ignore in release per spec §5.3 bounded in_use atomic for lock-free pressure check SothSdk facade (src/sdk.rs): init — validates HMAC key + ClassificationMode + bundle source; fails closed pre_call — synchronous decision (≤5ms p99 budget per spec §7.1); runs process_normalized + sync block path on credential/private-key artifacts; allocates slab token post_call — consumes token, runs full classify with fallback bundle, pushes TelemetryEvent onto in-memory queue stream_begin / stream_chunk / stream_end — token-consumed-once invariant for streaming refresh_bundle — Phase-1 stub in_flight_decisions / drain_telemetry_for_test — test hooks In-memory telemetry queue (src/telemetry_queue.rs): bounded VecDeque (4096 default), oldest-drops-on-overflow with dropped-count metric. Phase-1 wires HTTPS shipper. Compile-time Send + Sync assertions for SothSdk + Observation — bindings stash Arc across host worker threads. Tests: 15 unit + 5 integration round-trip - init_succeeds_with_minimal_config - pre_call_then_post_call_balances_slab_and_emits_telemetry - pre_call_blocks_on_credential_in_user_message - streaming_round_trip_consumes_token_once - many_pre_calls_without_post_call_do_not_leak_past_slab_capacity (asserts SLAB_FULL behavior under 6K alloc pressure) Verification: - cargo build --workspace clean - all workspace lib tests pass (634 + 20 new = 654) - All 4 WASM combos (detect+classify x wasip1+unknown-unknown) clean - Conformance harness still green (7/7 fixtures, 0 strict failures) Phase-1 deferred (called out in crate README): - Real bundle CDN pull + Ed25519 verification - Background HTTPS telemetry shipper - Cloud-classify wire transport - Orphan sweeper for never-consumed tokens - Full org-rule evaluation - OTel span emission - Conformance harness lane through this facade Co-Authored-By: Claude Opus 4.7 (1M context) --- Cargo.lock | 18 + Cargo.toml | 1 + crates/soth-sdk-core/Cargo.toml | 40 +++ crates/soth-sdk-core/README.md | 129 +++++++ crates/soth-sdk-core/src/call.rs | 86 +++++ crates/soth-sdk-core/src/config.rs | 369 ++++++++++++++++++++ crates/soth-sdk-core/src/decision.rs | 212 +++++++++++ crates/soth-sdk-core/src/error.rs | 32 ++ crates/soth-sdk-core/src/lib.rs | 44 +++ crates/soth-sdk-core/src/sdk.rs | 364 +++++++++++++++++++ crates/soth-sdk-core/src/slab.rs | 298 ++++++++++++++++ crates/soth-sdk-core/src/telemetry_queue.rs | 103 ++++++ crates/soth-sdk-core/tests/round_trip.rs | 156 +++++++++ 13 files changed, 1852 insertions(+) create mode 100644 crates/soth-sdk-core/Cargo.toml create mode 100644 crates/soth-sdk-core/README.md create mode 100644 crates/soth-sdk-core/src/call.rs create mode 100644 crates/soth-sdk-core/src/config.rs create mode 100644 crates/soth-sdk-core/src/decision.rs create mode 100644 crates/soth-sdk-core/src/error.rs create mode 100644 crates/soth-sdk-core/src/lib.rs create mode 100644 crates/soth-sdk-core/src/sdk.rs create mode 100644 crates/soth-sdk-core/src/slab.rs create mode 100644 crates/soth-sdk-core/src/telemetry_queue.rs create mode 100644 crates/soth-sdk-core/tests/round_trip.rs diff --git a/Cargo.lock b/Cargo.lock index 4135c0bf..f1523050 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -4157,6 +4157,24 @@ dependencies = [ "zeroize", ] +[[package]] +name = "soth-sdk-core" +version = "0.1.0" +dependencies = [ + "arc-swap", + "opentelemetry", + "serde", + "serde_json", + "soth-classify", + "soth-core", + "soth-detect", + "thiserror 1.0.69", + "tracing", + "tracing-opentelemetry", + "uuid", + "zeroize", +] + [[package]] name = "soth-sync" version = "0.1.0" diff --git a/Cargo.toml b/Cargo.toml index d88cdde0..934a4f2b 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -7,6 +7,7 @@ members = [ "crates/soth-classify", "crates/soth-conformance-tests", "crates/soth-proxy", + "crates/soth-sdk-core", "crates/soth-telemetry", "crates/soth-policy", "crates/soth-sync", diff --git a/crates/soth-sdk-core/Cargo.toml b/crates/soth-sdk-core/Cargo.toml new file mode 100644 index 00000000..aae3df05 --- /dev/null +++ b/crates/soth-sdk-core/Cargo.toml @@ -0,0 +1,40 @@ +[package] +name = "soth-sdk-core" +description = "Public-API facade consumed by SOTH SDK bindings (PyO3 / napi-rs / WASM)." +version.workspace = true +edition.workspace = true +license.workspace = true +repository.workspace = true +authors.workspace = true + +# Default features keep the SDK build self-contained and WASM-compatible. +# The `onnx-models` feature transitively turns on local ONNX classification +# in soth-classify; bindings flip it on for native targets and off for +# WASM/edge. +[features] +default = ["onnx-models"] +onnx-models = ["soth-classify/onnx-models"] +# OpenTelemetry span emission. Off by default — bindings opt in when the +# host already has an OTel pipeline. +otel = ["dep:opentelemetry", "dep:tracing-opentelemetry"] + +[dependencies] +soth-core = { workspace = true } +# soth-detect's workspace entry uses `default-features = false`; this crate +# turns on `intelligence` so the in-memory backends are available, but +# leaves `tree-sitter-code` and `intelligence-sqlite` off — neither is +# needed in the SDK runtime, both block the WASM target. +soth-detect = { workspace = true, features = ["intelligence"] } +# soth-classify's workspace entry pulls default features (incl. onnx-models). +# `policy` is needed for the post_call enrichment path. WASM builds disable +# default features at the binding level (Phase 2). +soth-classify = { workspace = true, features = ["policy"] } +serde = { workspace = true } +serde_json = { workspace = true } +thiserror = { workspace = true } +tracing = { workspace = true } +uuid = { workspace = true } +zeroize = { workspace = true } +arc-swap = "1" +opentelemetry = { workspace = true, optional = true } +tracing-opentelemetry = { workspace = true, optional = true } diff --git a/crates/soth-sdk-core/README.md b/crates/soth-sdk-core/README.md new file mode 100644 index 00000000..7af68abf --- /dev/null +++ b/crates/soth-sdk-core/README.md @@ -0,0 +1,129 @@ +# soth-sdk-core + +Public-API facade consumed by every SOTH SDK binding (PyO3, napi-rs, +WASM). Bindings depend ONLY on this crate; the internal `soth-detect` / +`soth-classify` / `soth-telemetry` crates are not re-exported, so +swapping their internals does not ripple downstream. + +This crate is **public-API stable** by contract — every type re-exported +from `lib.rs` is part of the SDK's customer-facing surface. Changes +require a major version bump. + +The two pre-flight specs lock the surface: + +- `docs/common/SDK_DECISION_API_SPEC.md` — Decision / DecisionToken / + wrapper exception contract / sync-vs-async budget +- `docs/common/SDK_WASM_TRUST_BOUNDARY_SPEC.md` — tier matrix / + reduced-mode capabilities / cloud-classify opt-in + +## Public surface + +```rust +use soth_sdk_core::{ + SothSdk, SdkConfigBuilder, HmacKey, ClassificationMode, + LlmCall, LlmResponse, LlmChunk, Message, Tool, + Decision, DecisionToken, BlockReason, FlagSeverity, + MessageRedactions, MessageRedaction, RedactReason, + Observation, StreamObservation, + SdkError, +}; + +let sdk = SothSdk::init( + SdkConfigBuilder::new() + .api_key("sk-...") + .org_id("org-123") + .hmac_key(HmacKey::from_env("SOTH_HMAC_KEY")) + .local_classification(ClassificationMode::Full) + .build()? +)?; + +// Synchronous decision path (≤5 ms p99 budget) +let decision = sdk.pre_call(&call); +match decision { + Decision::Allow { token } => { + // forward to provider, then: + sdk.post_call(token, &response); + } + Decision::Block { token, reason } => { + // bindings translate this into SothBlocked + sdk.post_call(token, &empty_response); + } + // Redact / Flag handled similarly + _ => {} +} + +// Streaming +let (decision, mut obs) = sdk.stream_begin(&call); +// ... iterate provider stream, call sdk.stream_chunk(&mut obs, &chunk) ... +sdk.stream_end(obs); +``` + +## What v0 does (Phase 0) + +- ✅ `init` validates config, resolves HMAC key, builds an empty detect + bundle and the deterministic fallback classify bundle. +- ✅ `pre_call` runs `process_normalized` for artifact detection + + session dedup. Sync block path fires on credential / private-key + artifacts. Returns a real `DecisionToken` allocated from a fixed + 4096-slot slab. +- ✅ `post_call` consumes the token, runs the full classify pipeline + (with fallback bundle, so `use_case_label` is heuristic), pushes + a `TelemetryEvent` onto the in-memory queue. +- ✅ Streaming round-trip: `stream_begin` / `stream_chunk` / + `stream_end` consume the token exactly once. +- ✅ `DecisionToken` lifecycle per spec §5: panic-on-reuse in debug, + log+ignore in release, `SLAB_FULL` sentinel under pressure, + `SENTINEL_FAIL_OPEN` for FFI-boundary fail-open. +- ✅ Compile-time `Send + Sync` assertions for `SothSdk` and + `Observation`. +- ✅ HMAC key never logged: `Debug` redacts plaintext bytes; resolved + bytes held in `Zeroizing>` and dropped on init. + +## What's deferred to Phase 1 + +- ❌ Real bundle CDN pull + Ed25519 verification (stubbed; uses the + fallback bundle today). Spec'd in WASM trust-boundary §6. +- ❌ Background telemetry shipper (HTTPS POST to soth-cloud). Events + accumulate in the in-memory queue; tests drain them via + `drain_telemetry_for_test`. +- ❌ Cloud-classify wire transport (the `CloudOptIn` path stops at + init validation in v0). Spec'd in WASM trust-boundary §5. +- ❌ Orphan sweeper for never-consumed tokens. Slab grows up to + `SLAB_FULL` threshold and then returns sentinels; sweeper is a + Phase-1 background task per spec §5.4. +- ❌ Full org-rule evaluation (artifact-conditioned + label-conditioned). + Today's sync block is hard-coded on credential / private-key + artifacts; full policy eval lands in Phase 1. +- ❌ OTel span emission (feature-gated; stub). +- ❌ Conformance harness lane through this facade. The fixture corpus + in `soth-conformance-tests` already runs both proxy and SDK lanes + through the lower-level crates; adding a `via-facade` lane goes + into Phase 1's binding-validation work. + +## Send + Sync + +The crate has compile-time assertions that `SothSdk` and `Observation` +remain `Send + Sync`. Any future change that introduces a non-`Send` +field fails the build. Bindings stash an `Arc` and call from +arbitrary host worker threads. + +## Feature flags + +| Feature | Default | Purpose | +|---|---|---| +| `onnx-models` | on | Transitively enables local ONNX classification in `soth-classify`. Bindings flip this off for WASM/edge targets that ship reduced mode. | +| `otel` | off | Emits OpenTelemetry spans for every observed call. Bindings opt in when the host already has an OTel pipeline. | + +## Running the tests + +```sh +cargo test -p soth-sdk-core +``` + +15 unit tests + 5 integration round-trip tests. The integration tests +exercise: +- `init` with minimal config +- `pre_call` → `post_call` slab balance + telemetry emission +- credential-in-user-message sync block path +- streaming round-trip token-consumed-once invariant +- slab pressure under 6K allocations without `post_call` diff --git a/crates/soth-sdk-core/src/call.rs b/crates/soth-sdk-core/src/call.rs new file mode 100644 index 00000000..1f411094 --- /dev/null +++ b/crates/soth-sdk-core/src/call.rs @@ -0,0 +1,86 @@ +//! `LlmCall` / `LlmResponse` / `LlmChunk` — typed inputs and outputs. +//! +//! `LlmCall` mirrors `soth_core::TypedLlmCall` exactly so the SDK +//! re-exports that type rather than introducing a parallel one. +//! Bindings see `soth_sdk_core::LlmCall` and don't need to depend +//! on `soth_core` directly. + +use serde::{Deserialize, Serialize}; +use soth_core::EndpointType; + +/// Pre-parsed LLM call. SDK consumers populate this from typed provider +/// SDK objects (OpenAI / Anthropic / Cohere / Google / Mistral); the +/// proxy's HTTP fingerprint+parse phase is skipped. +pub use soth_core::TypedLlmCall as LlmCall; +pub use soth_core::TypedMessage as Message; +pub use soth_core::TypedTool as Tool; + +/// Provider response. Bindings populate this from the typed provider +/// response object before calling [`SothSdk::post_call`]. +/// +/// Fields are intentionally minimal for v0; new fields can be added +/// non-breakingly because the struct is `#[non_exhaustive]`. +#[derive(Debug, Clone, Serialize, Deserialize)] +#[non_exhaustive] +pub struct LlmResponse { + /// Model that actually served the request (may differ from the + /// requested model on routing fallbacks). + pub model: Option, + /// Best-effort assistant content extracted from the response. + /// Used for output redaction triggers and response-side telemetry + /// fields. May be `None` for tool-only responses or content arrays + /// the binding doesn't flatten. + pub assistant_content: Option, + pub finish_reason: Option, + pub usage: Option, + pub endpoint_type: EndpointType, +} + +impl LlmResponse { + pub fn new(endpoint_type: EndpointType) -> Self { + Self { + model: None, + assistant_content: None, + finish_reason: None, + usage: None, + endpoint_type, + } + } +} + +#[derive(Debug, Clone, Default, Serialize, Deserialize)] +pub struct UsageStats { + pub input_tokens: Option, + pub output_tokens: Option, + pub total_tokens: Option, + pub estimated_cost_usd: Option, +} + +/// Streaming chunk. Bindings call [`SothSdk::stream_chunk`] once per +/// chunk emitted by the provider stream. The SDK accumulates chunks +/// into the same `StreamObservation` for telemetry on `stream_end`. +#[derive(Debug, Clone, Serialize, Deserialize)] +#[non_exhaustive] +pub struct LlmChunk { + /// Sequence number within the stream (binding-assigned). Used by + /// the SDK for ordering checks; not customer-facing. + pub sequence: u32, + /// Best-effort delta text from this chunk. May be empty for + /// non-content chunks (tool-call metadata, role headers). + pub delta_content: Option, + pub finish_reason: Option, + /// Final usage block if the provider sends one in the last chunk. + /// Most providers only populate this on the terminal chunk. + pub usage: Option, +} + +impl LlmChunk { + pub fn new(sequence: u32) -> Self { + Self { + sequence, + delta_content: None, + finish_reason: None, + usage: None, + } + } +} diff --git a/crates/soth-sdk-core/src/config.rs b/crates/soth-sdk-core/src/config.rs new file mode 100644 index 00000000..04a91b0d --- /dev/null +++ b/crates/soth-sdk-core/src/config.rs @@ -0,0 +1,369 @@ +//! `SdkConfig` and friends. +//! +//! Locked by `SDK_DECISION_API_SPEC.md` §4 and +//! `SDK_WASM_TRUST_BOUNDARY_SPEC.md` §3. The HMAC key lifecycle in +//! particular is part of the trust model: the SDK never sees the +//! plaintext key over the wire and there is no key-fetching endpoint. + +use std::path::PathBuf; +use std::sync::Arc; + +use soth_core::CaptureMode; +use zeroize::Zeroizing; + +use crate::error::SdkError; + +/// Customer-controlled HMAC key reference. +/// +/// The key is generated once in the SOTH dashboard, downloaded by the +/// org admin, and stored in the customer's secret manager. The SDK +/// references it via this enum; the cloud never has the plaintext. +/// +/// `Static` is provided for tests and rapid prototyping; production +/// integrations use `FromEnv` / `FromFile` / `FromCallback`. +#[non_exhaustive] +pub enum HmacKey { + /// Read the key from an environment variable (e.g. + /// `HmacKey::from_env("SOTH_HMAC_KEY")`). + FromEnv(String), + /// Read the key from a mounted secret file. The file's full + /// contents are treated as the key after trimming trailing + /// whitespace. + FromFile(PathBuf), + /// Vault / secret-manager integration. The callback is invoked + /// at SDK init; failures bubble up as `SdkError::HmacKey`. + FromCallback(Arc Result, String> + Send + Sync>), + /// Inline bytes — discouraged in production. The wrapper zeroes + /// the buffer on drop. + Static(Zeroizing>), +} + +impl std::fmt::Debug for HmacKey { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + // Never print actual key material. + match self { + Self::FromEnv(name) => f + .debug_tuple("HmacKey::FromEnv") + .field(&format!("${name}")) + .finish(), + Self::FromFile(path) => f.debug_tuple("HmacKey::FromFile").field(path).finish(), + Self::FromCallback(_) => f.write_str("HmacKey::FromCallback()"), + Self::Static(bytes) => f + .debug_tuple("HmacKey::Static") + .field(&format!("<{} bytes redacted>", bytes.len())) + .finish(), + } + } +} + +impl HmacKey { + pub fn from_env(name: impl Into) -> Self { + HmacKey::FromEnv(name.into()) + } + + pub fn from_file(path: impl Into) -> Self { + HmacKey::FromFile(path.into()) + } + + /// Resolve the key material at SDK init time. Returns the raw bytes + /// or `SdkError::HmacKey` / `HmacKeyTooShort` on failure. Resolved + /// material is held in a `Zeroizing>` so it scrubs on drop. + pub fn resolve(&self) -> Result>, SdkError> { + let raw: Zeroizing> = match self { + HmacKey::FromEnv(name) => match std::env::var(name) { + Ok(value) => Zeroizing::new(value.trim().as_bytes().to_vec()), + Err(error) => return Err(SdkError::HmacKey(format!("env {name}: {error}"))), + }, + HmacKey::FromFile(path) => match std::fs::read(path) { + Ok(bytes) => { + let trimmed = std::str::from_utf8(&bytes) + .map(|s| s.trim().as_bytes().to_vec()) + .unwrap_or(bytes); + Zeroizing::new(trimmed) + } + Err(error) => { + return Err(SdkError::HmacKey(format!("file {path:?}: {error}"))); + } + }, + HmacKey::FromCallback(cb) => match cb() { + Ok(bytes) => Zeroizing::new(bytes), + Err(error) => return Err(SdkError::HmacKey(format!("callback: {error}"))), + }, + HmacKey::Static(bytes) => bytes.clone(), + }; + if raw.len() < 32 { + return Err(SdkError::HmacKeyTooShort { got: raw.len() }); + } + Ok(raw) + } +} + +/// Local-classification mode. Locked by +/// `SDK_WASM_TRUST_BOUNDARY_SPEC.md` §3.2. +/// +/// Default selection is **per-target** rather than universal: +/// - Native (Python / Node) and WASM-where-it-fits → `Full`. +/// - Size-constrained edge runtimes (CF Workers, Vercel Edge) → `Reduced`. +/// - `CloudOptIn` is never the default. Customer must set it explicitly +/// after their legal team signs the cloud-classify DPA addendum. +#[derive(Debug, Clone, Copy, PartialEq, Eq, Default)] +#[non_exhaustive] +pub enum ClassificationMode { + /// Local embedding via ONNX (native or ONNX Web). Content never + /// leaves the customer's environment. + #[default] + Full, + /// No local embedding. Heuristic detect + counter-based anomaly + /// + artifact-based policy. Telemetry events emit with sentinel + /// values for the semantic fields and a canonical `missing_fields` + /// list so cloud analytics can filter precisely. + Reduced, + /// Customer has explicitly opted in to send normalized request + /// data to soth-cloud for classification. Requires a DPA covering + /// content egress. + CloudOptIn, +} + +/// Where the SDK persists its outbox between process restarts. +#[derive(Debug, Clone, Copy, PartialEq, Eq, Default)] +#[non_exhaustive] +pub enum StorageMode { + /// In-memory queue only. Telemetry events are lost on process + /// restart — fine for serverless / short-lived workers. + #[default] + InMemory, + /// File-backed outbox at the configured `bundle_cache_dir`. Long- + /// lived servers wanting durable delivery should use this. + /// (Stub for v0; transport implementation lands in Phase 1.) + FileBacked, +} + +/// Where bundles come from at SDK init. +#[derive(Debug, Clone)] +#[non_exhaustive] +pub enum BundleSource { + /// Pull from the configured CDN URL with Ed25519 signature + /// verification. Default for production deployments. + Cdn { url: String }, + /// Use whatever bundle is bundled into the binding (lite-only). + /// `ClassificationMode::Reduced` only — the lite bundle does not + /// contain ONNX models. + Embedded, + /// Tests / dev: use the deterministic fallback bundle from + /// `soth_classify::fallback_bundle()`. + Fallback, +} + +impl Default for BundleSource { + fn default() -> Self { + BundleSource::Fallback + } +} + +/// Top-level SDK configuration. Construct via [`SdkConfigBuilder`]. +pub struct SdkConfig { + pub api_key: String, + pub org_id: String, + pub hmac_key: HmacKey, + pub default_team_id: Option, + pub default_device_id_hash: Option, + pub capture_mode: CaptureMode, + pub storage_mode: StorageMode, + pub bundle_source: BundleSource, + pub bundle_cache_dir: Option, + pub local_classification: ClassificationMode, + pub telemetry_endpoint: Option, + pub cloud_classify_endpoint: Option, + pub sample_rate: f64, +} + +impl std::fmt::Debug for SdkConfig { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + f.debug_struct("SdkConfig") + .field("org_id", &self.org_id) + .field("api_key", &"") + .field("hmac_key", &self.hmac_key) + .field("default_team_id", &self.default_team_id) + .field("capture_mode", &self.capture_mode) + .field("storage_mode", &self.storage_mode) + .field("bundle_source", &self.bundle_source) + .field("bundle_cache_dir", &self.bundle_cache_dir) + .field("local_classification", &self.local_classification) + .field("telemetry_endpoint", &self.telemetry_endpoint) + .field("cloud_classify_endpoint", &self.cloud_classify_endpoint) + .field("sample_rate", &self.sample_rate) + .finish() + } +} + +/// Fluent builder for [`SdkConfig`]. Required fields (`api_key`, +/// `org_id`, `hmac_key`) must be set before [`build`] is called. +#[derive(Default)] +pub struct SdkConfigBuilder { + api_key: Option, + org_id: Option, + hmac_key: Option, + default_team_id: Option, + default_device_id_hash: Option, + capture_mode: Option, + storage_mode: Option, + bundle_source: Option, + bundle_cache_dir: Option, + local_classification: Option, + telemetry_endpoint: Option, + cloud_classify_endpoint: Option, + sample_rate: Option, +} + +impl SdkConfigBuilder { + pub fn new() -> Self { + Self::default() + } + + pub fn api_key(mut self, key: impl Into) -> Self { + self.api_key = Some(key.into()); + self + } + pub fn org_id(mut self, org: impl Into) -> Self { + self.org_id = Some(org.into()); + self + } + pub fn hmac_key(mut self, key: HmacKey) -> Self { + self.hmac_key = Some(key); + self + } + pub fn team_id(mut self, team: impl Into) -> Self { + self.default_team_id = Some(team.into()); + self + } + pub fn device_id_hash(mut self, device: impl Into) -> Self { + self.default_device_id_hash = Some(device.into()); + self + } + pub fn capture_mode(mut self, mode: CaptureMode) -> Self { + self.capture_mode = Some(mode); + self + } + pub fn storage_mode(mut self, mode: StorageMode) -> Self { + self.storage_mode = Some(mode); + self + } + pub fn bundle_source(mut self, source: BundleSource) -> Self { + self.bundle_source = Some(source); + self + } + pub fn bundle_cache_dir(mut self, path: impl Into) -> Self { + self.bundle_cache_dir = Some(path.into()); + self + } + pub fn local_classification(mut self, mode: ClassificationMode) -> Self { + self.local_classification = Some(mode); + self + } + pub fn telemetry_endpoint(mut self, url: impl Into) -> Self { + self.telemetry_endpoint = Some(url.into()); + self + } + pub fn cloud_classify_endpoint(mut self, url: impl Into) -> Self { + self.cloud_classify_endpoint = Some(url.into()); + self + } + pub fn sample_rate(mut self, rate: f64) -> Self { + self.sample_rate = Some(rate); + self + } + + pub fn build(self) -> Result { + let api_key = self + .api_key + .ok_or_else(|| SdkError::InvalidConfig("api_key required".into()))?; + let org_id = self + .org_id + .ok_or_else(|| SdkError::InvalidConfig("org_id required".into()))?; + let hmac_key = self + .hmac_key + .ok_or_else(|| SdkError::InvalidConfig("hmac_key required".into()))?; + let local_classification = self.local_classification.unwrap_or_default(); + + // CloudOptIn requires a configured endpoint. Validation here + // matches the spec's no-auto-fallback rule. + if matches!(local_classification, ClassificationMode::CloudOptIn) + && self.cloud_classify_endpoint.is_none() + { + return Err(SdkError::CloudClassifyEndpointMissing); + } + + Ok(SdkConfig { + api_key, + org_id, + hmac_key, + default_team_id: self.default_team_id, + default_device_id_hash: self.default_device_id_hash, + capture_mode: self.capture_mode.unwrap_or(CaptureMode::MetadataOnly), + storage_mode: self.storage_mode.unwrap_or_default(), + bundle_source: self.bundle_source.unwrap_or_default(), + bundle_cache_dir: self.bundle_cache_dir, + local_classification, + telemetry_endpoint: self.telemetry_endpoint, + cloud_classify_endpoint: self.cloud_classify_endpoint, + sample_rate: self.sample_rate.unwrap_or(1.0).clamp(0.0, 1.0), + }) + } +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn builder_requires_required_fields() { + let err = SdkConfigBuilder::new().build().unwrap_err(); + assert!(matches!(err, SdkError::InvalidConfig(_))); + } + + #[test] + fn builder_round_trips_minimal_config() { + let cfg = SdkConfigBuilder::new() + .api_key("sk-test") + .org_id("org-test") + .hmac_key(HmacKey::Static(Zeroizing::new(vec![0u8; 32]))) + .build() + .expect("build"); + assert_eq!(cfg.org_id, "org-test"); + assert_eq!(cfg.local_classification, ClassificationMode::Full); + } + + #[test] + fn cloud_optin_without_endpoint_fails() { + let err = SdkConfigBuilder::new() + .api_key("k") + .org_id("o") + .hmac_key(HmacKey::Static(Zeroizing::new(vec![0u8; 32]))) + .local_classification(ClassificationMode::CloudOptIn) + .build() + .unwrap_err(); + assert!(matches!(err, SdkError::CloudClassifyEndpointMissing)); + } + + #[test] + fn hmac_resolve_short_key_rejects() { + let key = HmacKey::Static(Zeroizing::new(vec![0u8; 8])); + let err = key.resolve().unwrap_err(); + assert!(matches!(err, SdkError::HmacKeyTooShort { got: 8 })); + } + + #[test] + fn hmac_resolve_long_key_works() { + let key = HmacKey::Static(Zeroizing::new(vec![0u8; 32])); + let resolved = key.resolve().expect("resolve"); + assert_eq!(resolved.len(), 32); + } + + #[test] + fn debug_redacts_static_bytes() { + let key = HmacKey::Static(Zeroizing::new(vec![1u8; 32])); + let formatted = format!("{key:?}"); + assert!(formatted.contains("redacted")); + assert!(!formatted.contains("0x01")); + } +} diff --git a/crates/soth-sdk-core/src/decision.rs b/crates/soth-sdk-core/src/decision.rs new file mode 100644 index 00000000..ff7fec87 --- /dev/null +++ b/crates/soth-sdk-core/src/decision.rs @@ -0,0 +1,212 @@ +//! Decision API public types. +//! +//! Locked by `docs/common/SDK_DECISION_API_SPEC.md`. Adding a new +//! variant to any of these enums is a minor bump (they are all +//! `#[non_exhaustive]`); changing a field is a breaking change. + +use serde::{Deserialize, Serialize}; +use soth_core::{ArtifactKind, ArtifactSeverity}; + +/// Output of `SothSdk::pre_call` and `stream_begin`. Cooperative +/// enforcement: the binding's wrapper translates each variant into a +/// host-language action — `Block` becomes a `SothBlocked` exception, +/// `Redact` substitutes redacted message content before forwarding, +/// `Flag` logs a structured event without affecting the call. +/// +/// `Reroute` is intentionally absent — it is a proxy/sidecar capability; +/// reroute-shaped policies surface here as +/// [`BlockReason::UseAlternative`]. +#[derive(Debug, Clone, Serialize, Deserialize)] +#[non_exhaustive] +#[serde(tag = "kind", rename_all = "snake_case")] +pub enum Decision { + Allow { + token: DecisionToken, + }, + Block { + token: DecisionToken, + reason: BlockReason, + }, + Redact { + token: DecisionToken, + redactions: MessageRedactions, + }, + Flag { + token: DecisionToken, + severity: FlagSeverity, + }, +} + +impl Decision { + /// Token consumed by `post_call` / `stream_end` regardless of variant. + pub fn token(&self) -> DecisionToken { + match self { + Decision::Allow { token } + | Decision::Block { token, .. } + | Decision::Redact { token, .. } + | Decision::Flag { token, .. } => *token, + } + } + + pub fn is_block(&self) -> bool { + matches!(self, Decision::Block { .. }) + } +} + +/// Why a `Decision::Block` fired. Bindings surface this as a field on +/// `SothBlocked`; customers can branch on `kind` for graceful degradation +/// (e.g. retry-with-alternative when `UseAlternative` is set). +#[derive(Debug, Clone, Serialize, Deserialize)] +#[non_exhaustive] +#[serde(tag = "kind", rename_all = "snake_case")] +pub enum BlockReason { + SensitiveArtifact { + artifact: ArtifactKind, + severity: ArtifactSeverity, + }, + BudgetExceeded { + budget_kind: BudgetKind, + observed: u64, + limit: u64, + }, + PolicyRule { + rule_id: String, + #[serde(default, skip_serializing_if = "Option::is_none")] + rule_name: Option, + }, + /// Reroute-shaped policy: customer app may retry against the + /// suggested provider/model. + UseAlternative { + #[serde(default, skip_serializing_if = "Option::is_none")] + suggested_provider: Option, + #[serde(default, skip_serializing_if = "Option::is_none")] + suggested_model: Option, + rule_id: String, + }, +} + +#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)] +#[non_exhaustive] +#[serde(rename_all = "snake_case")] +pub enum BudgetKind { + Tokens, + CostUsd, + Requests, +} + +#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)] +#[non_exhaustive] +#[serde(rename_all = "snake_case")] +pub enum FlagSeverity { + Info, + Warning, + Critical, +} + +/// List of message-level replacements produced by `Decision::Redact`. +/// Within-message surgical edits are deliberately out of scope — +/// customers wanting that fidelity use the proxy. +#[derive(Debug, Clone, Default, Serialize, Deserialize)] +pub struct MessageRedactions { + pub replacements: Vec, +} + +#[derive(Debug, Clone, Serialize, Deserialize)] +pub struct MessageRedaction { + /// Index into `LlmCall.messages`. Bindings' provider adapters use + /// this to locate the message in the typed call object before + /// substitution. + pub message_idx: usize, + /// Replacement content. Always set; the SDK never deletes a message + /// outright — the placeholder preserves conversation turn structure. + pub redacted_content: String, + /// Telemetry tag — what triggered the redaction. + pub reason: RedactReason, +} + +#[derive(Debug, Clone, Serialize, Deserialize)] +#[non_exhaustive] +#[serde(tag = "kind", rename_all = "snake_case")] +pub enum RedactReason { + SensitiveArtifact { artifact: ArtifactKind }, + PolicyRule { rule_id: String }, +} + +/// Opaque per-call handle. Created by `pre_call` / `stream_begin` and +/// consumed exactly once by `post_call` / `stream_end`. Bindings MUST +/// NOT inspect `inner` — its layout is internal and may change. +/// +/// Lifecycle, slab-full handling, and orphan sweep are specified in +/// `SDK_DECISION_API_SPEC.md` §5. +#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash, Serialize, Deserialize)] +pub struct DecisionToken { + pub(crate) inner: u64, +} + +impl DecisionToken { + /// Sentinel token returned when the decision slab is at capacity. + /// `post_call(SLAB_FULL, ...)` is a documented no-op that emits a + /// `slab_full_no_enrichment` telemetry event so cluster operators + /// know to size up. + pub const SLAB_FULL: DecisionToken = DecisionToken { inner: u64::MAX }; + + /// Test/binding-failure sentinel. Used when an FFI panic was caught + /// at the boundary and a fail-open `Decision::Allow` was emitted. + pub const SENTINEL_FAIL_OPEN: DecisionToken = DecisionToken { inner: u64::MAX - 1 }; + + pub(crate) fn is_sentinel(self) -> bool { + self == Self::SLAB_FULL || self == Self::SENTINEL_FAIL_OPEN + } +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn decision_token_is_copy_and_eq() { + let t = DecisionToken { inner: 42 }; + let u = t; + assert_eq!(t, u); + } + + #[test] + fn sentinels_are_distinct() { + assert_ne!(DecisionToken::SLAB_FULL, DecisionToken::SENTINEL_FAIL_OPEN); + assert!(DecisionToken::SLAB_FULL.is_sentinel()); + assert!(DecisionToken::SENTINEL_FAIL_OPEN.is_sentinel()); + } + + #[test] + fn decision_token_method_works_for_every_variant() { + let t = DecisionToken { inner: 1 }; + assert_eq!(Decision::Allow { token: t }.token(), t); + assert_eq!( + Decision::Block { + token: t, + reason: BlockReason::PolicyRule { + rule_id: "r".into(), + rule_name: None + }, + } + .token(), + t + ); + assert_eq!( + Decision::Redact { + token: t, + redactions: MessageRedactions::default(), + } + .token(), + t + ); + assert_eq!( + Decision::Flag { + token: t, + severity: FlagSeverity::Info, + } + .token(), + t + ); + } +} diff --git a/crates/soth-sdk-core/src/error.rs b/crates/soth-sdk-core/src/error.rs new file mode 100644 index 00000000..14ff1916 --- /dev/null +++ b/crates/soth-sdk-core/src/error.rs @@ -0,0 +1,32 @@ +use thiserror::Error; + +#[derive(Debug, Error)] +#[non_exhaustive] +pub enum SdkError { + #[error("SDK config invalid: {0}")] + InvalidConfig(String), + + #[error("HMAC key resolution failed: {0}")] + HmacKey(String), + + #[error("bundle pull failed: {0}")] + BundlePull(String), + + #[error("bundle verification failed: {0}")] + BundleVerification(String), + + /// `ClassificationMode::Full` was requested on a target where local ONNX + /// is unavailable. Customer must pick `Reduced` or `CloudOptIn`. + #[error("local ONNX classification unavailable on this target — pick ClassificationMode::Reduced or CloudOptIn")] + OnnxUnavailable, + + /// `ClassificationMode::CloudOptIn` was requested but no + /// `cloud_classify_endpoint` was configured. + #[error("cloud-classify endpoint not configured for ClassificationMode::CloudOptIn")] + CloudClassifyEndpointMissing, + + /// Customer's HmacKey resolved successfully but is empty / shorter than + /// the documented minimum (32 bytes). + #[error("HMAC key too short: expected ≥32 bytes, got {got}")] + HmacKeyTooShort { got: usize }, +} diff --git a/crates/soth-sdk-core/src/lib.rs b/crates/soth-sdk-core/src/lib.rs new file mode 100644 index 00000000..4c00f501 --- /dev/null +++ b/crates/soth-sdk-core/src/lib.rs @@ -0,0 +1,44 @@ +//! `soth-sdk-core` — public-API facade consumed by SOTH SDK bindings. +//! +//! Bindings (PyO3 / napi-rs / WASM) depend ONLY on this crate. The internal +//! `soth-detect` / `soth-classify` / `soth-telemetry` crates are not +//! re-exported, so swapping their internals does not ripple to bindings. +//! +//! Public-API contracts: +//! - [`docs/common/SDK_DECISION_API_SPEC.md`](../../../docs/common/SDK_DECISION_API_SPEC.md) +//! - [`docs/common/SDK_WASM_TRUST_BOUNDARY_SPEC.md`](../../../docs/common/SDK_WASM_TRUST_BOUNDARY_SPEC.md) +//! +//! Once a public type in this crate is committed, every change is breaking +//! for downstream bindings and customers. The conformance harness in +//! `soth-conformance-tests` is the keystone — every binding must pass it +//! before shipping. + +#![forbid(unsafe_code)] + +pub mod call; +pub mod config; +pub mod decision; +pub mod error; + +mod sdk; +mod slab; +mod telemetry_queue; + +pub use call::{LlmCall, LlmChunk, LlmResponse, Message, Tool}; +pub use config::{ + BundleSource, ClassificationMode, HmacKey, SdkConfig, SdkConfigBuilder, StorageMode, +}; +pub use decision::{ + BlockReason, BudgetKind, Decision, DecisionToken, FlagSeverity, MessageRedaction, + MessageRedactions, RedactReason, +}; +pub use error::SdkError; +pub use sdk::{Observation, SothSdk, StreamObservation}; + +// Re-exports of soth-core types that appear in the public API. Bindings +// see these as `soth_sdk_core::ArtifactKind` etc. and don't need to depend +// on `soth-core` directly. +pub use soth_core::{ + AnomalyFlag, ArtifactKind, ArtifactSeverity, CaptureMode, EndpointType, PolicyDecisionKind, + UseCaseLabel, VolatilityClass, +}; diff --git a/crates/soth-sdk-core/src/sdk.rs b/crates/soth-sdk-core/src/sdk.rs new file mode 100644 index 00000000..e2b57187 --- /dev/null +++ b/crates/soth-sdk-core/src/sdk.rs @@ -0,0 +1,364 @@ +//! `SothSdk` facade — the surface bindings consume. +//! +//! `pre_call` is synchronous (≤5 ms p99 budget per spec §7.1): +//! 1. Build a `DetectResult` via `process_normalized` (artifact scan + +//! session dedup; no embedding, no classification). +//! 2. Run system rules conditioned on artifacts to make a sync block +//! decision. +//! 3. Allocate a `DecisionToken` slab slot for the post-call enrichment. +//! +//! `post_call` is the async-ish enrichment (≤300 ms p99 budget §7.2): +//! 1. Consume the slab slot. +//! 2. Run the full classify pipeline (embed → cluster → use_case → +//! semantic anomaly). +//! 3. Apply label-conditioned org rules. +//! 4. Push a `TelemetryEvent` onto the in-memory queue. +//! +//! For v0, the post_call enrichment uses the classify fallback bundle, +//! so use_case_label is heuristic. Phase-1 wires real bundle pulls +//! and Phase-2 brings the WASM cloud-classify path online. + +use std::sync::Arc; + +use arc_swap::ArcSwap; +use soth_core::{ + AttributionContext, CaptureMode, ClassificationSource, IdentityContext, OwnedDetectBundle, + ProxyContext, SessionSnapshot, TrafficClassification, TransportContext, +}; + +use crate::call::{LlmCall, LlmChunk, LlmResponse}; +use crate::config::{BundleSource, ClassificationMode, SdkConfig}; +use crate::decision::{ + BlockReason, BudgetKind, Decision, DecisionToken, FlagSeverity, MessageRedactions, +}; +use crate::error::SdkError; +use crate::slab::{ArtifactsSummary, DecisionContext, DecisionSlab}; +use crate::telemetry_queue::TelemetryQueue; + +/// Per-call observation handle returned by `pre_call`. Currently +/// transparent over `DecisionToken`; reserved for richer state in +/// Phase 1 (e.g. carrying OTel span context across the FFI boundary). +#[derive(Debug, Clone, Copy)] +pub struct Observation { + pub token: DecisionToken, +} + +/// Streaming counterpart to `Observation`. Bindings call +/// `stream_chunk` once per chunk and `stream_end` to finalize. +pub struct StreamObservation { + pub token: DecisionToken, + chunks_seen: u32, + accumulated_content: String, + finish_reason: Option, +} + +impl StreamObservation { + fn new(token: DecisionToken) -> Self { + Self { + token, + chunks_seen: 0, + accumulated_content: String::new(), + finish_reason: None, + } + } +} + +/// Main SDK type. `Send + Sync` — bindings stash `Arc` and +/// invoke from arbitrary host threads. +pub struct SothSdk { + config: SdkConfig, + detect_registry: Arc, + detect_bundle: ArcSwap, + classify_bundle: ArcSwap, + classify_config: soth_classify::ClassifyConfig, + slab: Arc, + telemetry: Arc, +} + +// `SothSdk` must be `Send + Sync` for bindings to share it across host +// threads. Compile-time check — same pattern as PR 4. +const _: fn() = || { + fn assert_send_sync() {} + assert_send_sync::(); + assert_send_sync::(); +}; + +impl SothSdk { + /// Construct a new SDK instance from a [`SdkConfig`]. + /// + /// Failure modes (per spec §7.4): bundle pull / verification + /// failure logs and returns an error; the binding's wrapper SHOULD + /// fall back to no-op mode rather than crashing the host process. + pub fn init(config: SdkConfig) -> Result { + // Validate the HMAC key resolves before we accept the config. + // Resolved bytes are dropped immediately — Phase 1 keeps them + // for telemetry signing, v0 only validates. + let _hmac = config.hmac_key.resolve()?; + + // ClassificationMode::Full + onnx-models feature-off would be a + // mismatch; bindings on size-constrained targets must pick + // Reduced or CloudOptIn. + #[cfg(not(feature = "onnx-models"))] + if matches!(config.local_classification, ClassificationMode::Full) { + return Err(SdkError::OnnxUnavailable); + } + + let detect_bundle = build_detect_bundle(&config.bundle_source)?; + let detect_registry = Arc::new(soth_detect::ParserRegistry::default()); + + let classify_bundle = match config.bundle_source { + BundleSource::Fallback | BundleSource::Embedded => soth_classify::fallback_bundle(), + BundleSource::Cdn { ref url } => { + tracing::warn!( + target: "soth_sdk_core", + url, + "bundle CDN pull is a Phase-1 deliverable; using fallback bundle for v0" + ); + soth_classify::fallback_bundle() + } + }; + + let classify_config = soth_classify::ClassifyConfig::default(); + + Ok(Self { + config, + detect_registry, + detect_bundle: ArcSwap::from_pointee(detect_bundle), + classify_bundle: ArcSwap::from_pointee((*classify_bundle).clone()), + classify_config, + slab: Arc::new(DecisionSlab::new()), + telemetry: Arc::new(TelemetryQueue::new()), + }) + } + + /// Synchronous decision path. Returns within 5 ms p99 (binding-side + /// histograms gate this in CI). + /// + /// Allowed work (per spec §7.1): + /// - artifact scan via `process_normalized` + /// - artifact-conditioned system rules + /// - artifact-conditioned org rules + pub fn pre_call(&self, call: &LlmCall) -> Decision { + let detect_bundle = self.detect_bundle.load_full(); + let detect = soth_detect::process_normalized( + self.detect_registry.as_ref(), + call, + &detect_bundle.as_slice(), + &SessionSnapshot::default(), + self.config.capture_mode, + ); + + let summary = ArtifactsSummary::from_artifacts(&detect.artifacts); + + // Sync block path — credential / private-key artifacts are an + // unconditional block in v0. Phase-1 expands this with full + // org-rule evaluation on artifact-conditioned rules. + if let Some(reason) = detect.artifacts.iter().find_map(|a| { + artifact_block_reason(&a.kind, a.severity) + }) { + let ctx = DecisionContext { + created_at: std::time::Instant::now(), + generation: 0, + detect: detect.clone(), + artifacts_summary: summary.clone(), + call_provider: call.provider.clone(), + call_model: call.model.clone(), + user_content: detect.normalized.user_prompt.clone(), + }; + let token = self.slab.allocate(ctx); + return Decision::Block { token, reason }; + } + + // Allow path — stash the partial state for post_call enrichment. + let ctx = DecisionContext { + created_at: std::time::Instant::now(), + generation: 0, + detect, + artifacts_summary: summary, + call_provider: call.provider.clone(), + call_model: call.model.clone(), + user_content: None, // populated on consume; we cloned detect so re-derive there + }; + let token = self.slab.allocate(ctx); + Decision::Allow { token } + } + + /// Async-ish enrichment + telemetry emission. Bindings spawn this + /// off the host's critical path. Idempotent on sentinel tokens + /// (no-op). + pub fn post_call(&self, token: DecisionToken, _resp: &LlmResponse) { + let Some(ctx) = self.slab.consume(token) else { + // Token is sentinel, stale, or already consumed. Emit a + // tagged telemetry event so cluster operators can + // distinguish "binding bug" from "slab pressure". + self.emit_orphan_or_full(token); + return; + }; + + let proxy_ctx = self.build_proxy_context(&ctx); + let user_content = ctx.detect.normalized.user_prompt.clone(); + let classify_bundle = self.classify_bundle.load_full(); + let result = soth_classify::classify( + &ctx.detect, + user_content.as_deref(), + &proxy_ctx, + &classify_bundle, + &self.classify_config, + ); + + self.telemetry.push(result.telemetry_event); + } + + /// Streaming counterpart to `pre_call`. Returns the decision and an + /// observation handle for `stream_chunk` / `stream_end`. + pub fn stream_begin(&self, call: &LlmCall) -> (Decision, StreamObservation) { + let decision = self.pre_call(call); + let token = decision.token(); + (decision, StreamObservation::new(token)) + } + + pub fn stream_chunk(&self, obs: &mut StreamObservation, chunk: &LlmChunk) { + obs.chunks_seen = obs.chunks_seen.saturating_add(1); + if let Some(delta) = &chunk.delta_content { + obs.accumulated_content.push_str(delta); + } + if chunk.finish_reason.is_some() { + obs.finish_reason = chunk.finish_reason.clone(); + } + } + + pub fn stream_end(&self, obs: StreamObservation) { + let mut response = LlmResponse::new(soth_core::EndpointType::ChatCompletion); + response.assistant_content = if obs.accumulated_content.is_empty() { + None + } else { + Some(obs.accumulated_content) + }; + response.finish_reason = obs.finish_reason; + self.post_call(obs.token, &response); + } + + /// Pull a fresh bundle from the configured CDN, verify, and hot-swap. + /// V0 stub — Phase 1 implements the actual transport. + pub fn refresh_bundle(&self) -> Result<(), SdkError> { + match &self.config.bundle_source { + BundleSource::Fallback | BundleSource::Embedded => Ok(()), + BundleSource::Cdn { url } => { + tracing::warn!( + target: "soth_sdk_core", + url, + "bundle refresh is a Phase-1 deliverable" + ); + Ok(()) + } + } + } + + // ── test helpers ───────────────────────────────────────────────── + + /// Number of in-flight `DecisionToken`s. Used by the smoke test + /// to assert pre_call/post_call balance. + pub fn in_flight_decisions(&self) -> usize { + self.slab.in_flight() + } + + /// Drain the in-memory telemetry queue. Test-only — production + /// shippers will pull batches via a different API in Phase 1. + pub fn drain_telemetry_for_test(&self) -> Vec { + self.telemetry.drain_for_test() + } + + fn build_proxy_context(&self, ctx: &DecisionContext) -> ProxyContext { + ProxyContext { + identity: IdentityContext { + org_id: self.config.org_id.clone(), + user_id_hmac: self + .config + .default_team_id + .clone() + .unwrap_or_else(|| String::from("sdk-anonymous")), + team_id: self.config.default_team_id.clone().unwrap_or_default(), + device_id_hash: self + .config + .default_device_id_hash + .clone() + .unwrap_or_default(), + endpoint_hash: String::new(), + capture_mode: self.config.capture_mode, + traffic_classification: TrafficClassification::ToolUsage, + classification_source: ClassificationSource::Sdk, + session_snapshot: Some(SessionSnapshot::default()), + declared_provider: Some(ctx.call_provider.clone()), + declared_application: None, + session_id: None, + deployment_context: None, + bundle_trust_level: None, + precomputed_commitment_nonce: None, + precomputed_commitment_hash: None, + }, + transport: TransportContext::default(), + attribution: AttributionContext::default(), + } + } + + fn emit_orphan_or_full(&self, token: DecisionToken) { + if token == DecisionToken::SLAB_FULL { + tracing::warn!( + target: "soth_sdk_core", + "post_call invoked with SLAB_FULL token — slab pressure; size up the SDK" + ); + } else if token == DecisionToken::SENTINEL_FAIL_OPEN { + tracing::warn!( + target: "soth_sdk_core", + "post_call invoked with SENTINEL_FAIL_OPEN token — pre_call panicked at FFI boundary" + ); + } else { + tracing::warn!( + target: "soth_sdk_core", + token = token.inner, + "post_call invoked with stale/already-consumed token (binding bug)" + ); + } + } +} + +fn artifact_block_reason( + kind: &soth_core::ArtifactKind, + severity: soth_core::ArtifactSeverity, +) -> Option { + use soth_core::{ArtifactKind, ArtifactSeverity}; + match kind { + ArtifactKind::PrivateKey => Some(BlockReason::SensitiveArtifact { + artifact: kind.clone(), + severity, + }), + ArtifactKind::ApiKey { .. } if severity >= ArtifactSeverity::High => { + Some(BlockReason::SensitiveArtifact { + artifact: kind.clone(), + severity, + }) + } + _ => None, + } +} + +fn build_detect_bundle(source: &BundleSource) -> Result { + // V0: every BundleSource variant resolves to the empty bundle. + // Phase-1 wires real CDN pull + verification per + // SDK_WASM_TRUST_BOUNDARY_SPEC.md §6. + let _ = source; + Ok(OwnedDetectBundle::default()) +} + +// Re-exports above keep these types used. Phase-1 hooks for budget / +// flag / redact / capture-mode / classification-mode are wired in +// then; v0 references them via `BlockReason::*` in artifact_block_reason +// and via the public type re-exports. +#[allow(dead_code)] +fn _typecheck_unused_for_v0() { + let _: Option = None; + let _: Option = None; + let _: Option = None; + let _: Option = None; + let _: Option = None; +} diff --git a/crates/soth-sdk-core/src/slab.rs b/crates/soth-sdk-core/src/slab.rs new file mode 100644 index 00000000..7629a507 --- /dev/null +++ b/crates/soth-sdk-core/src/slab.rs @@ -0,0 +1,298 @@ +//! Decision slab — fixed-size storage for in-flight `DecisionToken`s. +//! +//! Each slot holds the partial state needed to enrich + emit telemetry +//! when `post_call` consumes the token. Lifecycle is locked by +//! `SDK_DECISION_API_SPEC.md` §5: +//! +//! - allocate on `pre_call` / `stream_begin` +//! - free on `post_call` / `stream_end` +//! - reuse → panic in debug, log+ignore in release +//! - never consumed → orphan sweeper frees + emits `decision_orphaned` +//! - slab full → return `DecisionToken::SLAB_FULL`, no allocation + +use std::sync::Mutex; +use std::time::Instant; + +use soth_core::{ArtifactKind, DetectResult}; + +use crate::decision::DecisionToken; + +const SLAB_CAPACITY: usize = 4096; +/// 95% threshold from the spec — past this point new `pre_call`s get +/// `SLAB_FULL` rather than waiting for a free slot. +const SLAB_PRESSURE_THRESHOLD: usize = (SLAB_CAPACITY * 95) / 100; + +/// Partial decision state stashed for the async enrichment phase. +/// Several fields are read by Phase-1 (orphan sweeper uses +/// `created_at`; org-rule eval uses `artifacts_summary`; cloud-classify +/// path uses `call_model` + `user_content`); v0 only consumes the +/// minimum to push a telemetry event. +pub(crate) struct DecisionContext { + #[allow(dead_code)] // Phase-1: orphan sweeper inspects age + pub created_at: Instant, + pub generation: u64, + pub detect: DetectResult, + #[allow(dead_code)] // Phase-1: org-rule eval branches on artifact summary + pub artifacts_summary: ArtifactsSummary, + pub call_provider: String, + #[allow(dead_code)] // Phase-1: cloud-classify path serializes the model field + pub call_model: String, + #[allow(dead_code)] // Phase-1: cloud-classify path serializes user_content + pub user_content: Option, +} + +/// Pre-summarized artifact view so `post_call` doesn't re-walk the +/// `Vec` for high-level decisions. +#[derive(Debug, Default, Clone)] +pub(crate) struct ArtifactsSummary { + pub credential_count: u32, + pub private_key_count: u32, + pub code_block_count: u32, + pub other_count: u32, +} + +impl ArtifactsSummary { + pub fn from_artifacts(arts: &[soth_core::SensitiveArtifact]) -> Self { + let mut s = Self::default(); + for a in arts { + match &a.kind { + ArtifactKind::ApiKey { .. } => s.credential_count += 1, + ArtifactKind::PrivateKey => s.private_key_count += 1, + ArtifactKind::CodeBlock { .. } => s.code_block_count += 1, + _ => s.other_count += 1, + } + } + s + } + + #[allow(dead_code)] // Phase-1 hook for org-rule evaluation + pub fn has_credential(&self) -> bool { + self.credential_count > 0 || self.private_key_count > 0 + } +} + +pub(crate) struct DecisionSlab { + slots: Mutex>>, + /// Free-list head; `None` when slab is full. + free_head: Mutex>, + /// Per-slot generation counter — flipped on every alloc/free so + /// stale tokens (from a freed slot) are detectable. + generations: Mutex>, + next_generation: std::sync::atomic::AtomicU64, + in_use: std::sync::atomic::AtomicUsize, +} + +impl DecisionSlab { + pub fn new() -> Self { + let mut slots = Vec::with_capacity(SLAB_CAPACITY); + let mut generations = Vec::with_capacity(SLAB_CAPACITY); + let mut free = Vec::with_capacity(SLAB_CAPACITY); + for i in (0..SLAB_CAPACITY).rev() { + slots.push(None); + generations.push(0); + free.push(i); + } + // slots was filled tail-first; flip back. + slots.reverse(); + generations.reverse(); + Self { + slots: Mutex::new(slots), + free_head: Mutex::new(free), + generations: Mutex::new(generations), + next_generation: std::sync::atomic::AtomicU64::new(1), + in_use: std::sync::atomic::AtomicUsize::new(0), + } + } + + /// Allocate a slot. Returns `DecisionToken::SLAB_FULL` if the slab + /// is at the configured pressure threshold. + pub fn allocate(&self, ctx: DecisionContext) -> DecisionToken { + // Pressure check first — don't even take the lock when full. + if self.in_use.load(std::sync::atomic::Ordering::Acquire) >= SLAB_PRESSURE_THRESHOLD { + return DecisionToken::SLAB_FULL; + } + + let mut free = match self.free_head.lock() { + Ok(g) => g, + Err(p) => p.into_inner(), + }; + let Some(idx) = free.pop() else { + return DecisionToken::SLAB_FULL; + }; + drop(free); + + let mut generations = match self.generations.lock() { + Ok(g) => g, + Err(p) => p.into_inner(), + }; + let new_gen = self + .next_generation + .fetch_add(1, std::sync::atomic::Ordering::Relaxed); + generations[idx] = new_gen; + drop(generations); + + let mut slots = match self.slots.lock() { + Ok(g) => g, + Err(p) => p.into_inner(), + }; + let mut ctx_with_gen = ctx; + ctx_with_gen.generation = new_gen; + slots[idx] = Some(ctx_with_gen); + drop(slots); + + self.in_use + .fetch_add(1, std::sync::atomic::Ordering::AcqRel); + + DecisionToken { + inner: encode_token(idx as u32, new_gen), + } + } + + /// Consume a token and return its context. Returns `None` for: + /// - sentinel tokens (SLAB_FULL, SENTINEL_FAIL_OPEN) + /// - tokens whose slot has been freed (reuse case) + /// - tokens with a stale generation + /// + /// In debug builds, reuse panics. In release, `None` + log. + pub fn consume(&self, token: DecisionToken) -> Option { + if token.is_sentinel() { + return None; + } + let (idx, gen) = decode_token(token.inner); + if (idx as usize) >= SLAB_CAPACITY { + on_invalid_token("index out of range", token); + return None; + } + + let mut generations = match self.generations.lock() { + Ok(g) => g, + Err(p) => p.into_inner(), + }; + if generations[idx as usize] != gen { + on_invalid_token("stale generation (reuse?)", token); + return None; + } + // Bump generation so this slot's token can never be re-consumed. + generations[idx as usize] = self + .next_generation + .fetch_add(1, std::sync::atomic::Ordering::Relaxed); + drop(generations); + + let mut slots = match self.slots.lock() { + Ok(g) => g, + Err(p) => p.into_inner(), + }; + let ctx = slots[idx as usize].take(); + drop(slots); + + if ctx.is_some() { + let mut free = match self.free_head.lock() { + Ok(g) => g, + Err(p) => p.into_inner(), + }; + free.push(idx as usize); + drop(free); + self.in_use + .fetch_sub(1, std::sync::atomic::Ordering::AcqRel); + } + ctx + } + + /// Snapshot of in-flight token count. Used by the orphan sweeper + /// (Phase 1) and by the smoke tests. + pub fn in_flight(&self) -> usize { + self.in_use.load(std::sync::atomic::Ordering::Acquire) + } +} + +/// Encode (slot_index, generation) into a single u64 token. +/// 32 bits for the index, 32 bits for the generation. Both fit +/// comfortably given SLAB_CAPACITY = 4096 and the generation counter +/// rolls every ~4 billion allocations. +fn encode_token(idx: u32, generation: u64) -> u64 { + ((idx as u64) << 32) | (generation & 0xFFFF_FFFF) +} + +fn decode_token(raw: u64) -> (u32, u64) { + let idx = (raw >> 32) as u32; + let generation = raw & 0xFFFF_FFFF; + (idx, generation) +} + +fn on_invalid_token(reason: &'static str, token: DecisionToken) { + #[cfg(debug_assertions)] + panic!( + "DecisionToken {:?} invalid: {reason}. This is a binding bug — see SDK_DECISION_API_SPEC.md §5.", + token.inner + ); + #[cfg(not(debug_assertions))] + { + tracing::warn!( + target: "soth_sdk_core::slab", + token = token.inner, + reason, + "DecisionToken invalid — this is a binding bug; see SDK_DECISION_API_SPEC.md §5" + ); + } +} + +#[cfg(test)] +mod tests { + use super::*; + + fn empty_ctx() -> DecisionContext { + DecisionContext { + created_at: Instant::now(), + generation: 0, + detect: DetectResult::default(), + artifacts_summary: ArtifactsSummary::default(), + call_provider: "openai".to_string(), + call_model: "gpt-4o-mini".to_string(), + user_content: None, + } + } + + #[test] + fn allocate_then_consume_returns_ctx() { + let slab = DecisionSlab::new(); + let token = slab.allocate(empty_ctx()); + assert!(!token.is_sentinel()); + assert_eq!(slab.in_flight(), 1); + let ctx = slab.consume(token); + assert!(ctx.is_some()); + assert_eq!(slab.in_flight(), 0); + } + + #[test] + #[cfg_attr(debug_assertions, should_panic(expected = "stale generation"))] + fn consume_twice_in_debug_panics() { + let slab = DecisionSlab::new(); + let token = slab.allocate(empty_ctx()); + let _ = slab.consume(token); + // Second consume on the same token should panic in debug. + let second = slab.consume(token); + // Release-mode fallback: assertion below is what we'd assert in + // production; debug builds panic before reaching it. + assert!(second.is_none()); + } + + #[test] + fn sentinel_consume_returns_none() { + let slab = DecisionSlab::new(); + assert!(slab.consume(DecisionToken::SLAB_FULL).is_none()); + assert!(slab.consume(DecisionToken::SENTINEL_FAIL_OPEN).is_none()); + } + + #[test] + fn slab_full_returns_sentinel() { + let slab = DecisionSlab::new(); + // Fill to threshold. + for _ in 0..SLAB_PRESSURE_THRESHOLD { + let t = slab.allocate(empty_ctx()); + assert!(!t.is_sentinel()); + } + // Next allocation should be SLAB_FULL. + let t = slab.allocate(empty_ctx()); + assert_eq!(t, DecisionToken::SLAB_FULL); + } +} diff --git a/crates/soth-sdk-core/src/telemetry_queue.rs b/crates/soth-sdk-core/src/telemetry_queue.rs new file mode 100644 index 00000000..4a5271b6 --- /dev/null +++ b/crates/soth-sdk-core/src/telemetry_queue.rs @@ -0,0 +1,103 @@ +//! In-memory telemetry queue (v0 stub). +//! +//! Events are enqueued by `post_call` / `stream_end`; a background +//! shipper drains them to soth-cloud. The shipper itself is Phase-1 +//! work; v0 keeps the queue and exposes a `drain_for_test` helper so +//! the smoke test can assert events were emitted. +//! +//! Queue is bounded — when full, oldest events drop with a counter so +//! cluster operators know to size up. Locked by Plan 2's "Sampling & +//! cost control" cross-cutting note. + +use std::collections::VecDeque; +use std::sync::Mutex; + +use soth_core::TelemetryEvent; + +const DEFAULT_CAPACITY: usize = 4096; + +pub(crate) struct TelemetryQueue { + inner: Mutex>, + capacity: usize, + dropped: std::sync::atomic::AtomicU64, +} + +impl TelemetryQueue { + pub fn new() -> Self { + Self::with_capacity(DEFAULT_CAPACITY) + } + + pub fn with_capacity(capacity: usize) -> Self { + Self { + inner: Mutex::new(VecDeque::with_capacity(capacity)), + capacity, + dropped: std::sync::atomic::AtomicU64::new(0), + } + } + + pub fn push(&self, event: TelemetryEvent) { + let mut guard = match self.inner.lock() { + Ok(g) => g, + Err(p) => p.into_inner(), + }; + if guard.len() >= self.capacity { + guard.pop_front(); + self.dropped + .fetch_add(1, std::sync::atomic::Ordering::Relaxed); + } + guard.push_back(event); + } + + #[allow(dead_code)] // Phase-1 hook for shipper batching diagnostics + pub fn dropped_count(&self) -> u64 { + self.dropped.load(std::sync::atomic::Ordering::Relaxed) + } + + #[allow(dead_code)] // Phase-1 hook for shipper batching diagnostics + pub fn len(&self) -> usize { + self.inner.lock().map(|g| g.len()).unwrap_or(0) + } + + /// Test helper — drain all queued events. Production shipper will + /// pull batches via a separate API in Phase 1. + pub fn drain_for_test(&self) -> Vec { + let mut guard = match self.inner.lock() { + Ok(g) => g, + Err(p) => p.into_inner(), + }; + guard.drain(..).collect() + } +} + +#[cfg(test)] +mod tests { + use super::*; + + fn fake_event() -> TelemetryEvent { + TelemetryEvent { + provider: "openai".to_string(), + ..TelemetryEvent::default() + } + } + + #[test] + fn push_then_drain_is_fifo() { + let q = TelemetryQueue::new(); + q.push(fake_event()); + q.push(fake_event()); + assert_eq!(q.len(), 2); + let drained = q.drain_for_test(); + assert_eq!(drained.len(), 2); + assert_eq!(q.len(), 0); + } + + #[test] + fn full_queue_drops_oldest_and_increments_counter() { + let q = TelemetryQueue::with_capacity(2); + q.push(fake_event()); + q.push(fake_event()); + q.push(fake_event()); + assert_eq!(q.len(), 2); + assert_eq!(q.dropped_count(), 1); + } +} diff --git a/crates/soth-sdk-core/tests/round_trip.rs b/crates/soth-sdk-core/tests/round_trip.rs new file mode 100644 index 00000000..0bf2fc97 --- /dev/null +++ b/crates/soth-sdk-core/tests/round_trip.rs @@ -0,0 +1,156 @@ +//! End-to-end smoke test for the `SothSdk` facade. +//! +//! Asserts the lifecycle the spec commits to: +//! - `init` succeeds with minimal config. +//! - `pre_call` returns a real `Decision` with a non-sentinel +//! `DecisionToken` for clean inputs. +//! - `pre_call` returns `Decision::Block` for inputs containing a +//! credential artifact (sync block path). +//! - `post_call` consumes the token; in-flight count returns to zero. +//! - Telemetry events are emitted onto the in-memory queue. +//! - Streaming round-trip: `stream_begin` / `stream_chunk` / +//! `stream_end` consumes the token exactly once. + +use soth_sdk_core::{ + BlockReason, Decision, HmacKey, LlmCall, LlmChunk, LlmResponse, Message, SdkConfigBuilder, + SothSdk, +}; +use soth_core::EndpointType; +use zeroize::Zeroizing; + +fn minimal_sdk() -> SothSdk { + let config = SdkConfigBuilder::new() + .api_key("sk-test") + .org_id("org-test") + .hmac_key(HmacKey::Static(Zeroizing::new(vec![0x42; 32]))) + .build() + .expect("build config"); + SothSdk::init(config).expect("init sdk") +} + +fn clean_call() -> LlmCall { + LlmCall { + provider: "openai".into(), + model: "gpt-4o-mini".into(), + messages: vec![Message { + role: "user".into(), + content: "Explain Rust ownership in two sentences.".into(), + }], + system: None, + tools: Vec::new(), + stream: false, + temperature: None, + top_p: None, + max_tokens: None, + stop_sequences: Vec::new(), + endpoint_type: EndpointType::ChatCompletion, + } +} + +fn credential_call() -> LlmCall { + LlmCall { + provider: "openai".into(), + model: "gpt-4o-mini".into(), + messages: vec![Message { + role: "user".into(), + content: "review this key sk-abcdefghijklmnopqrstuvwxyzABCD1234567890 for me" + .into(), + }], + system: None, + tools: Vec::new(), + stream: false, + temperature: None, + top_p: None, + max_tokens: None, + stop_sequences: Vec::new(), + endpoint_type: EndpointType::ChatCompletion, + } +} + +#[test] +fn init_succeeds_with_minimal_config() { + let sdk = minimal_sdk(); + assert_eq!(sdk.in_flight_decisions(), 0); +} + +#[test] +fn pre_call_then_post_call_balances_slab_and_emits_telemetry() { + let sdk = minimal_sdk(); + let call = clean_call(); + + let decision = sdk.pre_call(&call); + assert!( + matches!(decision, Decision::Allow { .. }), + "expected Allow, got {decision:?}" + ); + assert_eq!(sdk.in_flight_decisions(), 1); + + let token = decision.token(); + let response = LlmResponse::new(EndpointType::ChatCompletion); + sdk.post_call(token, &response); + + assert_eq!(sdk.in_flight_decisions(), 0); + let events = sdk.drain_telemetry_for_test(); + assert_eq!(events.len(), 1, "expected exactly one telemetry event"); + assert_eq!(events[0].provider, "openai"); +} + +#[test] +fn pre_call_blocks_on_credential_in_user_message() { + let sdk = minimal_sdk(); + let call = credential_call(); + + let decision = sdk.pre_call(&call); + match &decision { + Decision::Block { reason, .. } => match reason { + BlockReason::SensitiveArtifact { .. } => {} + other => panic!("expected SensitiveArtifact reason, got {other:?}"), + }, + other => panic!("expected Block, got {other:?}"), + } + + // Still need to consume the token even on Block — bindings call + // post_call regardless. + let token = decision.token(); + sdk.post_call(token, &LlmResponse::new(EndpointType::ChatCompletion)); + assert_eq!(sdk.in_flight_decisions(), 0); +} + +#[test] +fn streaming_round_trip_consumes_token_once() { + let sdk = minimal_sdk(); + let call = clean_call(); + let (decision, mut obs) = sdk.stream_begin(&call); + assert!(matches!(decision, Decision::Allow { .. })); + assert_eq!(sdk.in_flight_decisions(), 1); + + for i in 0..3 { + let mut chunk = LlmChunk::new(i); + chunk.delta_content = Some(format!("chunk{i} ")); + sdk.stream_chunk(&mut obs, &chunk); + } + assert_eq!(sdk.in_flight_decisions(), 1); + + sdk.stream_end(obs); + assert_eq!(sdk.in_flight_decisions(), 0); + assert_eq!(sdk.drain_telemetry_for_test().len(), 1); +} + +#[test] +fn many_pre_calls_without_post_call_do_not_leak_past_slab_capacity() { + // Allocate a bunch without consuming. After enough allocations + // the slab returns SLAB_FULL rather than crashing or leaking. + let sdk = minimal_sdk(); + for _ in 0..6_000 { + let _ = sdk.pre_call(&clean_call()); + } + // Slab is at threshold; any subsequent decision is SLAB_FULL. + let decision = sdk.pre_call(&clean_call()); + assert_eq!( + decision.token(), + soth_sdk_core::DecisionToken::SLAB_FULL, + "slab pressure should yield SLAB_FULL token" + ); + // post_call with SLAB_FULL is a documented no-op. + sdk.post_call(decision.token(), &LlmResponse::new(EndpointType::ChatCompletion)); +} From ba5d83b67c10e883bf07e830cd31c5a2148ffb27 Mon Sep 17 00:00:00 2001 From: Apoorv Raj Saxena Date: Thu, 30 Apr 2026 12:52:00 +0530 Subject: [PATCH 03/24] =?UTF-8?q?feat(sdk):=20conformance=20facade=20lane?= =?UTF-8?q?=20=E2=80=94=20catches=20SothSdk=20drift=20vs=20direct=20crate?= =?UTF-8?q?=20calls?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Phase 0 follow-up. The conformance harness (PR 5) ran proxy and SDK lanes through the lower-level crates directly. Phase 0 added soth-sdk-core as the public API facade, which is now an extra layer where parity can drift. This commit closes that gap. soth-sdk-core: - Add SothSdk::for_test(config, detect_bundle, classify_bundle) — doc-hidden ctor that bypasses the bundle-source dispatch so the harness can isolate facade-vs-direct parity from bundle-source differences. Phase-1's `init` will gain an in-memory BundleSource variant that subsumes this. soth-conformance-tests: - run_facade_lane(fixture, bundles) — drives SothSdk::pre_call → post_call, drains the in-memory telemetry queue, returns the Decision + emitted TelemetryEvent. - compare_sdk_vs_facade(sdk_lane, facade) — strict-parity comparison on the cloud ingestion contract surface (provider, model, endpoint_type, capture_mode, parse_source, parse_confidence, use_case, volatility_class, policy_kind, estimated_input_tokens, anomaly_flags). Per-call ephemeral fields (event_id, timestamp_epoch_ms, commitment_nonce, commitment_hash) are documented as excluded. - New test sdk_direct_and_facade_lanes_agree_byte_identical asserts every fixture produces a TelemetryEvent matching the SDK direct lane's classify output. Failure names the diverging field so the fix lands in soth-sdk-core, not the harness. Dependency: soth-conformance-tests now depends on soth-sdk-core via path. Verification: - All 7 fixtures pass the new sdk-vs-facade parity assertion - Existing proxy-vs-sdk parity still passes (7/7 strict, 2 advisory) - soth-sdk-core: 15 unit + 5 integration tests - soth-detect / soth-classify / soth-proxy lib tests unchanged - All 4 WASM combos (detect+classify x wasip1+unknown-unknown) clean Co-Authored-By: Claude Opus 4.7 (1M context) --- Cargo.lock | 2 + crates/soth-conformance-tests/Cargo.toml | 2 + crates/soth-conformance-tests/src/lib.rs | 173 ++++++++++++++++++ crates/soth-conformance-tests/tests/parity.rs | 51 ++++++ crates/soth-sdk-core/src/sdk.rs | 23 +++ 5 files changed, 251 insertions(+) diff --git a/Cargo.lock b/Cargo.lock index f1523050..958167b1 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -3936,7 +3936,9 @@ dependencies = [ "soth-core", "soth-detect", "soth-policy", + "soth-sdk-core", "uuid", + "zeroize", ] [[package]] diff --git a/crates/soth-conformance-tests/Cargo.toml b/crates/soth-conformance-tests/Cargo.toml index 99ef7f6a..2cbac3fc 100644 --- a/crates/soth-conformance-tests/Cargo.toml +++ b/crates/soth-conformance-tests/Cargo.toml @@ -10,8 +10,10 @@ publish = false soth-core = { workspace = true } soth-detect = { workspace = true, features = ["intelligence", "tree-sitter-code"] } soth-classify = { workspace = true, features = ["policy", "onnx-models"] } +soth-sdk-core = { path = "../soth-sdk-core" } soth-policy = { workspace = true } serde = { workspace = true } serde_json = { workspace = true } bytes = "1" uuid = { workspace = true } +zeroize = { workspace = true } diff --git a/crates/soth-conformance-tests/src/lib.rs b/crates/soth-conformance-tests/src/lib.rs index f681a29c..fc032465 100644 --- a/crates/soth-conformance-tests/src/lib.rs +++ b/crates/soth-conformance-tests/src/lib.rs @@ -446,6 +446,85 @@ pub fn run_sdk_lane( LaneOutput { detect, classified } } +// --------------------------------------------------------------------------- +// Facade lane — runs the same fixture through `SothSdk::pre_call` / +// `post_call`. Asserts the public-API facade does not introduce drift +// vs. calling `process_normalized` + `classify` directly (run_sdk_lane). +// --------------------------------------------------------------------------- + +pub struct FacadeOutput { + pub decision: soth_sdk_core::Decision, + /// Telemetry event emitted by `post_call`. Always present — even + /// `Decision::Block` paths still call `post_call` to consume the + /// token, and the SDK emits an event regardless. + pub telemetry: Option, +} + +pub fn run_facade_lane( + fixture: &Fixture, + detect_bundle: &OwnedDetectBundle, + classify_bundle: &std::sync::Arc, +) -> FacadeOutput { + use zeroize::Zeroizing; + + let config = soth_sdk_core::SdkConfigBuilder::new() + .api_key("conformance-test") + .org_id("org-conformance") + .hmac_key(soth_sdk_core::HmacKey::Static(Zeroizing::new(vec![0u8; 32]))) + .build() + .expect("build sdk config"); + + let sdk = soth_sdk_core::SothSdk::for_test( + config, + detect_bundle.clone(), + std::sync::Arc::clone(classify_bundle), + ) + .expect("init sdk for_test"); + + // Build the typed call from the fixture (same conversion the SDK + // direct lane uses). + let call = TypedCallSpec { + provider: fixture.typed_call.provider.clone(), + model: fixture.typed_call.model.clone(), + messages: fixture + .typed_call + .messages + .iter() + .map(|m| TypedMessageSpec { + role: m.role.clone(), + content: m.content.clone(), + }) + .collect(), + system: fixture.typed_call.system.clone(), + tools: fixture + .typed_call + .tools + .iter() + .map(|t| TypedToolSpec { + name: t.name.clone(), + description: t.description.clone(), + parameters_json: t.parameters_json.clone(), + }) + .collect(), + stream: fixture.typed_call.stream, + temperature: fixture.typed_call.temperature, + top_p: fixture.typed_call.top_p, + max_tokens: fixture.typed_call.max_tokens, + stop_sequences: fixture.typed_call.stop_sequences.clone(), + } + .into_call(); + + let decision = sdk.pre_call(&call); + let token = decision.token(); + + let response = soth_sdk_core::LlmResponse::new(EndpointType::ChatCompletion); + sdk.post_call(token, &response); + + let telemetry = sdk.drain_telemetry_for_test().into_iter().next(); + + FacadeOutput { decision, telemetry } +} + // --------------------------------------------------------------------------- // Diff: compare proxy and SDK outputs on the SDK-to-cloud contract surface. // @@ -623,6 +702,100 @@ pub fn compare_advisory(proxy: &LaneOutput, sdk: &LaneOutput) -> Vec { diffs } +/// Compare the SDK direct lane to the facade lane. The facade should +/// emit a `TelemetryEvent` byte-identical to the SDK lane's +/// `classified.telemetry_event` (excluding per-call ephemeral fields: +/// `event_id`, `timestamp_epoch_ms`, `commitment_nonce`, +/// `commitment_hash`). Any divergence is a facade bug. +pub fn compare_sdk_vs_facade(sdk: &LaneOutput, facade: &FacadeOutput) -> Vec { + let mut diffs = Vec::new(); + + let Some(facade_event) = facade.telemetry.as_ref() else { + diffs.push(Diff { + field: "facade.telemetry".to_string(), + proxy: format!("{:?}", sdk.classified.telemetry_event.provider), + sdk: "".to_string(), + }); + return diffs; + }; + + let sdk_event = &sdk.classified.telemetry_event; + + // Cloud ingestion contract — these fields are what the cloud reads. + push_eq( + &mut diffs, + "telemetry.provider", + &sdk_event.provider, + &facade_event.provider, + ); + push_eq( + &mut diffs, + "telemetry.model", + &format!("{:?}", sdk_event.model), + &format!("{:?}", facade_event.model), + ); + push_eq( + &mut diffs, + "telemetry.endpoint_type", + &format!("{:?}", sdk_event.endpoint_type), + &format!("{:?}", facade_event.endpoint_type), + ); + push_eq( + &mut diffs, + "telemetry.capture_mode", + &format!("{:?}", sdk_event.capture_mode), + &format!("{:?}", facade_event.capture_mode), + ); + push_eq( + &mut diffs, + "telemetry.parse_source", + &format!("{:?}", sdk_event.parse_source), + &format!("{:?}", facade_event.parse_source), + ); + push_eq( + &mut diffs, + "telemetry.parse_confidence", + &format!("{:?}", sdk_event.parse_confidence), + &format!("{:?}", facade_event.parse_confidence), + ); + push_eq( + &mut diffs, + "telemetry.use_case", + &format!("{:?}", sdk_event.use_case), + &format!("{:?}", facade_event.use_case), + ); + push_eq( + &mut diffs, + "telemetry.volatility_class", + &format!("{:?}", sdk_event.volatility_class), + &format!("{:?}", facade_event.volatility_class), + ); + push_eq( + &mut diffs, + "telemetry.policy_kind", + &format!("{:?}", sdk_event.policy_kind), + &format!("{:?}", facade_event.policy_kind), + ); + push_eq( + &mut diffs, + "telemetry.estimated_input_tokens", + &format!("{:?}", sdk_event.estimated_input_tokens), + &format!("{:?}", facade_event.estimated_input_tokens), + ); + + let sdk_anomaly = anomaly_flag_set(&sdk_event.anomaly_flags); + let facade_anomaly = anomaly_flag_set(&facade_event.anomaly_flags); + if sdk_anomaly != facade_anomaly { + diffs.push(Diff { + field: "telemetry.anomaly_flags".to_string(), + proxy: format!("{sdk_anomaly:?}"), + sdk: format!("{facade_anomaly:?}"), + }); + } + + diffs +} + fn push_eq(diffs: &mut Vec, field: &str, left: &str, right: &str) { if left != right { diffs.push(Diff { diff --git a/crates/soth-conformance-tests/tests/parity.rs b/crates/soth-conformance-tests/tests/parity.rs index 100c878a..f1bfd642 100644 --- a/crates/soth-conformance-tests/tests/parity.rs +++ b/crates/soth-conformance-tests/tests/parity.rs @@ -78,3 +78,54 @@ fn proxy_and_sdk_lanes_agree_on_contract_surface() { panic!("{report}"); } } + +#[test] +fn sdk_direct_and_facade_lanes_agree_byte_identical() { + // The facade should produce a TelemetryEvent identical to the SDK + // direct lane's classify output (excluding per-call ephemeral + // fields). Any divergence means SothSdk introduced drift; the + // failure names the diverging field so the fix lands in + // soth-sdk-core, not the harness. + let fixtures = load_fixtures(); + assert!(!fixtures.is_empty(), "no fixtures found"); + + let runner = ParityRunner::new(); + let mut failures: Vec<(String, Vec)> = Vec::new(); + + for (path, fixture) in &fixtures { + let sdk = soth_conformance_tests::run_sdk_lane( + fixture, + &runner.detect_bundle, + &runner.classify_bundle, + &runner.config, + ); + let facade = soth_conformance_tests::run_facade_lane( + fixture, + &runner.detect_bundle, + &runner.classify_bundle, + ); + + let diffs = soth_conformance_tests::compare_sdk_vs_facade(&sdk, &facade); + if !diffs.is_empty() { + failures.push(( + format!("{} ({})", fixture.name, path.display()), + diffs, + )); + } + } + + if !failures.is_empty() { + let mut report = String::new(); + report.push_str("\n=== Facade parity failures (SothSdk vs direct crate calls) ===\n"); + for (name, diffs) in &failures { + report.push_str(&format!("\nfixture: {name}\n")); + for diff in diffs { + report.push_str(&format!( + " {}\n sdk: {}\n facade: {}\n", + diff.field, diff.proxy, diff.sdk + )); + } + } + panic!("{report}"); + } +} diff --git a/crates/soth-sdk-core/src/sdk.rs b/crates/soth-sdk-core/src/sdk.rs index e2b57187..0e788641 100644 --- a/crates/soth-sdk-core/src/sdk.rs +++ b/crates/soth-sdk-core/src/sdk.rs @@ -131,6 +131,29 @@ impl SothSdk { }) } + /// Test-only constructor that bypasses [`init`]'s bundle-source dispatch. + /// The conformance harness uses this to isolate facade-vs-direct-crate + /// parity from bundle-source differences. Not stable; not part of the + /// customer-facing API. Phase-1 `init` will gain an in-memory + /// `BundleSource` variant that subsumes this constructor. + #[doc(hidden)] + pub fn for_test( + config: SdkConfig, + detect_bundle: OwnedDetectBundle, + classify_bundle: Arc, + ) -> Result { + let _hmac = config.hmac_key.resolve()?; + Ok(Self { + config, + detect_registry: Arc::new(soth_detect::ParserRegistry::default()), + detect_bundle: ArcSwap::from_pointee(detect_bundle), + classify_bundle: ArcSwap::from_pointee((*classify_bundle).clone()), + classify_config: soth_classify::ClassifyConfig::default(), + slab: Arc::new(DecisionSlab::new()), + telemetry: Arc::new(TelemetryQueue::new()), + }) + } + /// Synchronous decision path. Returns within 5 ms p99 (binding-side /// histograms gate this in CI). /// From 088b95044cc70556ee12e019c2db99a5abff2632 Mon Sep 17 00:00:00 2001 From: Apoorv Raj Saxena Date: Thu, 30 Apr 2026 13:02:17 +0530 Subject: [PATCH 04/24] feat(sdk): Phase 1 - soth-py PyO3 binding scaffold MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit bindings/soth-py/ — Python binding consuming soth-sdk-core via PyO3. maturin-built abi3-py310 wheel publishable as `soth` on PyPI. Public Python surface (python/soth/__init__.py): init(api_key, org_id, hmac_key_env=...) — module-level singleton guard(call_fn, call=...) — pre/post lifecycle wrapper SothBlocked — exception (NOT openai.APIError) SothFlagged — Flag surface BlockReason — typed reason Negative tests (tests/test_blocked_propagates.py) gate the contract: - test_soth_blocked_does_not_inherit_from_openai_apierror - test_soth_blocked_propagates_past_openai_apierror_handler - test_soth_blocked_inherits_from_base_exception_directly If any future change makes SothBlocked inherit from openai.APIError or any provider type, these tests fail immediately. See SDK_DECISION_API_SPEC.md §6. soth-sdk-core: added DecisionToken::raw() / from_raw() so bindings can round-trip the token across the FFI boundary without inspecting struct internals. The token is still opaque per spec §3.4. Cargo / build: - bindings/soth-py added to workspace members but NOT default-members (cdylib + cdylib needs Python toolchain to actually link) - `pyo3/extension-module` is a soth-py feature, not always-on, so `cargo build -p soth-py` succeeds for compile-checking. maturin passes the feature when building wheels. - Python 3.10 floor (3.9 reached EOL October 2025) - abi3 — one wheel per platform×arch covers Python 3.10+ Verification: cargo build -p soth-py clean All workspace lib tests: 654 (unchanged) Conformance harness: 7/7 fixtures (unchanged) Phase 0 SDK core round-trip tests: 5/5 (unchanged) Phase-1 follow-ups documented in bindings/soth-py/README.md: auto-instrumentation, httpx middleware, with-context, Redact handling, streaming wrapper, per-arch wheel matrix via cibuildwheel. Co-Authored-By: Claude Opus 4.7 (1M context) --- Cargo.lock | 207 ++++++++++- Cargo.toml | 7 + bindings/soth-py/Cargo.toml | 31 ++ bindings/soth-py/README.md | 77 ++++ bindings/soth-py/pyproject.toml | 46 +++ bindings/soth-py/python/soth/__init__.py | 158 ++++++++ bindings/soth-py/python/soth/_soth_native.pyi | 31 ++ bindings/soth-py/python/soth/exceptions.py | 88 +++++ bindings/soth-py/src/lib.rs | 339 ++++++++++++++++++ .../soth-py/tests/test_blocked_propagates.py | 104 ++++++ bindings/soth-py/tests/test_smoke.py | 81 +++++ crates/soth-sdk-core/src/decision.rs | 18 + 12 files changed, 1186 insertions(+), 1 deletion(-) create mode 100644 bindings/soth-py/Cargo.toml create mode 100644 bindings/soth-py/README.md create mode 100644 bindings/soth-py/pyproject.toml create mode 100644 bindings/soth-py/python/soth/__init__.py create mode 100644 bindings/soth-py/python/soth/_soth_native.pyi create mode 100644 bindings/soth-py/python/soth/exceptions.py create mode 100644 bindings/soth-py/src/lib.rs create mode 100644 bindings/soth-py/tests/test_blocked_propagates.py create mode 100644 bindings/soth-py/tests/test_smoke.py diff --git a/Cargo.lock b/Cargo.lock index 958167b1..99bd9b93 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -632,6 +632,15 @@ version = "0.9.6" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "c2459377285ad874054d797f3ccebf984978aa39129f6eafde5cdc8315b612f8" +[[package]] +name = "convert_case" +version = "0.6.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ec182b0ca2f35d8fc196cf3404988fd8b8c739a4d270ff118a398feb0cbec1ca" +dependencies = [ + "unicode-segmentation", +] + [[package]] name = "core-foundation" version = "0.10.1" @@ -784,6 +793,16 @@ dependencies = [ "typenum", ] +[[package]] +name = "ctor" +version = "0.2.9" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "32a2785755761f3ddc1492979ce1e48d2c00d09311c39e4466429188f3dd6501" +dependencies = [ + "quote", + "syn", +] + [[package]] name = "curve25519-dalek" version = "4.1.3" @@ -1790,6 +1809,15 @@ dependencies = [ "serde_core", ] +[[package]] +name = "indoc" +version = "2.0.7" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "79cf5c93f93228cf8efb3ba362535fb11199ac548a09ce117c9b1adc3030d706" +dependencies = [ + "rustversion", +] + [[package]] name = "inotify" version = "0.9.6" @@ -1965,6 +1993,16 @@ version = "0.2.182" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "6800badb6cb2082ffd7b6a67e6125bb39f18782f793520caee8cb8846be06112" +[[package]] +name = "libloading" +version = "0.8.9" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d7c4b02199fee7c5d21a5ae7d8cfa79a6ef5bb2fc834d6e9058e89c825efdc55" +dependencies = [ + "cfg-if", + "windows-link", +] + [[package]] name = "libloading" version = "0.9.0" @@ -2107,6 +2145,15 @@ version = "2.8.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "f8ca58f447f06ed17d5fc4043ce1b10dd205e060fb3ce5b979b8ed8e59ff3f79" +[[package]] +name = "memoffset" +version = "0.9.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "488016bfae457b036d996092f6cb448677611ce4449e970ceaf42695203f218a" +dependencies = [ + "autocfg", +] + [[package]] name = "mime" version = "0.3.17" @@ -2201,6 +2248,65 @@ dependencies = [ "syn", ] +[[package]] +name = "napi" +version = "2.16.17" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "55740c4ae1d8696773c78fdafd5d0e5fe9bc9f1b071c7ba493ba5c413a9184f3" +dependencies = [ + "bitflags 2.11.0", + "ctor", + "napi-derive", + "napi-sys", + "once_cell", + "serde", + "serde_json", +] + +[[package]] +name = "napi-build" +version = "2.3.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d376940fd5b723c6893cd1ee3f33abbfd86acb1cd1ec079f3ab04a2a3bc4d3b1" + +[[package]] +name = "napi-derive" +version = "2.16.13" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7cbe2585d8ac223f7d34f13701434b9d5f4eb9c332cccce8dee57ea18ab8ab0c" +dependencies = [ + "cfg-if", + "convert_case", + "napi-derive-backend", + "proc-macro2", + "quote", + "syn", +] + +[[package]] +name = "napi-derive-backend" +version = "1.0.75" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1639aaa9eeb76e91c6ae66da8ce3e89e921cd3885e99ec85f4abacae72fc91bf" +dependencies = [ + "convert_case", + "once_cell", + "proc-macro2", + "quote", + "regex", + "semver", + "syn", +] + +[[package]] +name = "napi-sys" +version = "2.4.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "427802e8ec3a734331fec1035594a210ce1ff4dc5bc1950530920ab717964ea3" +dependencies = [ + "libloading 0.8.9", +] + [[package]] name = "native-tls" version = "0.2.18" @@ -2726,7 +2832,7 @@ version = "2.0.0-rc.11" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "4a5df903c0d2c07b56950f1058104ab0c8557159f2741782223704de9be73c3c" dependencies = [ - "libloading", + "libloading 0.9.0", "ndarray 0.17.2", "ort-sys", "smallvec", @@ -2987,6 +3093,69 @@ dependencies = [ "syn", ] +[[package]] +name = "pyo3" +version = "0.22.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f402062616ab18202ae8319da13fa4279883a2b8a9d9f83f20dbade813ce1884" +dependencies = [ + "cfg-if", + "indoc", + "libc", + "memoffset", + "once_cell", + "portable-atomic", + "pyo3-build-config", + "pyo3-ffi", + "pyo3-macros", + "unindent", +] + +[[package]] +name = "pyo3-build-config" +version = "0.22.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b14b5775b5ff446dd1056212d778012cbe8a0fbffd368029fd9e25b514479c38" +dependencies = [ + "once_cell", + "target-lexicon", +] + +[[package]] +name = "pyo3-ffi" +version = "0.22.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9ab5bcf04a2cdcbb50c7d6105de943f543f9ed92af55818fd17b660390fc8636" +dependencies = [ + "libc", + "pyo3-build-config", +] + +[[package]] +name = "pyo3-macros" +version = "0.22.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "0fd24d897903a9e6d80b968368a34e1525aeb719d568dba8b3d4bfa5dc67d453" +dependencies = [ + "proc-macro2", + "pyo3-macros-backend", + "quote", + "syn", +] + +[[package]] +name = "pyo3-macros-backend" +version = "0.22.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "36c011a03ba1e50152b4b394b479826cad97e7a21eb52df179cd91ac411cbfbe" +dependencies = [ + "heck", + "proc-macro2", + "pyo3-build-config", + "quote", + "syn", +] + [[package]] name = "quick-xml" version = "0.38.4" @@ -4073,6 +4242,20 @@ dependencies = [ "zstd", ] +[[package]] +name = "soth-node" +version = "0.1.0" +dependencies = [ + "napi", + "napi-build", + "napi-derive", + "serde", + "serde_json", + "soth-core", + "soth-sdk-core", + "zeroize", +] + [[package]] name = "soth-parse" version = "0.1.0" @@ -4159,6 +4342,16 @@ dependencies = [ "zeroize", ] +[[package]] +name = "soth-py" +version = "0.1.0" +dependencies = [ + "pyo3", + "soth-core", + "soth-sdk-core", + "zeroize", +] + [[package]] name = "soth-sdk-core" version = "0.1.0" @@ -4342,6 +4535,12 @@ dependencies = [ "xattr", ] +[[package]] +name = "target-lexicon" +version = "0.12.16" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "61c41af27dd6d1e27b1b16b489db798443478cef1f06a660c96db617ba5de3b1" + [[package]] name = "tempfile" version = "3.25.0" @@ -4964,6 +5163,12 @@ version = "0.1.1" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "39ec24b3121d976906ece63c9daad25b85969647682eee313cb5779fdd69e14e" +[[package]] +name = "unindent" +version = "0.2.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7264e107f553ccae879d21fbea1d6724ac785e8c3bfc762137959b5802826ef3" + [[package]] name = "universal-hash" version = "0.5.1" diff --git a/Cargo.toml b/Cargo.toml index 934a4f2b..3ce75a0e 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -15,6 +15,8 @@ members = [ "crates/soth-detect", "crates/soth-extensions", "extensions/historian", + "bindings/soth-py", + "bindings/soth-node", ] default-members = [ "crates/soth-cli", @@ -27,6 +29,11 @@ default-members = [ "crates/soth-sync", "crates/soth-extensions", ] +# Bindings (soth-py, soth-node) are explicitly NOT in default-members: +# they require Python / Node toolchains and produce cdylibs that the +# rest of the workspace doesn't link against. Build them via +# `cargo build -p soth-py` / `cargo build -p soth-node` or via the +# language-level tooling (maturin / @napi-rs/cli). exclude = [] [workspace.package] diff --git a/bindings/soth-py/Cargo.toml b/bindings/soth-py/Cargo.toml new file mode 100644 index 00000000..0411357a --- /dev/null +++ b/bindings/soth-py/Cargo.toml @@ -0,0 +1,31 @@ +[package] +name = "soth-py" +description = "Python binding for the SOTH SDK (PyO3 + maturin)." +version.workspace = true +edition.workspace = true +license.workspace = true +repository.workspace = true +authors.workspace = true +publish = false + +[lib] +# `_soth_native` is the compiled extension Python imports; the user- +# facing `soth` package wraps it. +name = "_soth_native" +crate-type = ["cdylib"] + +[features] +# `extension-module` tells PyO3 to leave Python symbols unresolved at +# link time (Python provides them at runtime when the extension is +# loaded). maturin / cibuildwheel pass this when building the actual +# wheel; bare `cargo build -p soth-py` does NOT, so workspace builds +# stay link-clean. `cargo build -p soth-py --features extension-module` +# replicates the wheel build step. +default = [] +extension-module = ["pyo3/extension-module"] + +[dependencies] +soth-sdk-core = { path = "../../crates/soth-sdk-core" } +soth-core = { workspace = true } +zeroize = { workspace = true } +pyo3 = { version = "0.22", features = ["abi3-py310"] } diff --git a/bindings/soth-py/README.md b/bindings/soth-py/README.md new file mode 100644 index 00000000..9de9c37c --- /dev/null +++ b/bindings/soth-py/README.md @@ -0,0 +1,77 @@ +# soth-py + +Python binding for the SOTH SDK. Built with PyO3 + maturin; abi3-py310 +wheels publishable to PyPI as `soth`. + +## Status + +**Phase 1 scaffold.** The Rust extension exposes the `SothSdk` facade +through PyO3; the user-facing Python API in `python/soth/__init__.py` +wraps it with the `guard()` helper, the `SothBlocked` exception, and +the contract negative tests. + +What v0 ships: +- `soth.init(api_key, org_id, hmac_key_env=...)` — module-level singleton +- `soth.guard(fn, call=...)` — wraps an LLM call with `pre_call` / `post_call` +- `soth.SothBlocked` — exception (does NOT inherit from any provider hierarchy) +- `soth.SothFlagged` — warning surface for `Decision::Flag` +- `soth.BlockReason` — typed reason carried on `SothBlocked` + +What's deferred to follow-up Phase 1 commits: +- Auto-instrumentation (`soth.instrument()`) for openai / anthropic / cohere / + google-genai / mistralai +- httpx middleware (`soth.httpx_client()`) +- `with soth.context(user_id=..., team_id=...)` per-call overrides +- Streaming wrapper for native `AsyncIterator`s +- `Redact` decision handling (v0 treats Redact as Allow with logging) +- Auto-instrument on import + +## Building + +```sh +# Once. Per-target wheels via cibuildwheel in CI. +pip install maturin +cd bindings/soth-py +maturin develop # builds + installs into the active venv +``` + +The Rust extension is part of the workspace, so `cargo build -p soth-py` +also works for compile-checking. + +## Tests + +```sh +pip install -e ".[test]" +maturin develop +pytest tests/ +``` + +The `test_blocked_propagates.py` suite is the contract gate: +`SothBlocked` MUST NOT be caught by `try/except openai.APIError`. If +that test fails, the inheritance hierarchy has drifted from the spec +and bindings cannot ship. + +## Wheel matrix (Phase-1 deliverable) + +| Platform | Architecture | Python | +|---|---|---| +| manylinux2014 | x86_64 | abi3-py310 | +| manylinux2014 | aarch64 | abi3-py310 | +| macOS | x86_64 | abi3-py310 | +| macOS | arm64 | abi3-py310 | +| Windows | x86_64 | abi3-py310 | + +Built via `cibuildwheel` in CI; abi3 means one wheel per platform×arch +covers Python 3.10+. + +Python 3.9 reached EOL in October 2025 — explicitly NOT supported. + +## Public API contract + +Locked by: +- `docs/common/SDK_DECISION_API_SPEC.md` — Decision lifecycle, exception contract +- `docs/common/SDK_WASM_TRUST_BOUNDARY_SPEC.md` — bundle / classification mode + +Read those before changing any public symbol in `python/soth/__init__.py` +or the PyO3 wrapper. Public-API stability matters here — `soth` ships in +customer dependencies and breaking changes propagate. diff --git a/bindings/soth-py/pyproject.toml b/bindings/soth-py/pyproject.toml new file mode 100644 index 00000000..d551bda8 --- /dev/null +++ b/bindings/soth-py/pyproject.toml @@ -0,0 +1,46 @@ +[build-system] +requires = ["maturin>=1.5,<2.0"] +build-backend = "maturin" + +[project] +name = "soth" +description = "SOTH SDK for Python — observability + policy enforcement for LLM API calls." +readme = "README.md" +license = { text = "MIT OR Apache-2.0" } +authors = [{ name = "Labterminal" }] +requires-python = ">=3.10" # 3.9 reached EOL October 2025 +classifiers = [ + "Development Status :: 3 - Alpha", + "Intended Audience :: Developers", + "License :: OSI Approved :: MIT License", + "Programming Language :: Python :: 3", + "Programming Language :: Python :: 3.10", + "Programming Language :: Python :: 3.11", + "Programming Language :: Python :: 3.12", + "Programming Language :: Rust", + "Topic :: Software Development :: Libraries", +] +dynamic = ["version"] + +[project.optional-dependencies] +openai = ["openai>=1.0"] +anthropic = ["anthropic>=0.34"] +cohere = ["cohere>=5.0"] +google = ["google-genai>=0.3"] +mistral = ["mistralai>=1.0"] +test = ["pytest>=8", "openai>=1.0"] + +[project.urls] +Homepage = "https://github.com/labterminal/soth" +Repository = "https://github.com/labterminal/soth" + +[tool.maturin] +# `extension-module` is a soth-py crate feature (gates pyo3's +# extension-module feature transitively). Bare `cargo build -p soth-py` +# does NOT pass this feature so the workspace build stays link-clean; +# maturin builds the wheel with it on so Python supplies symbols at +# load time. +features = ["extension-module"] +module-name = "soth._soth_native" +python-source = "python" +manifest-path = "Cargo.toml" diff --git a/bindings/soth-py/python/soth/__init__.py b/bindings/soth-py/python/soth/__init__.py new file mode 100644 index 00000000..b8bde680 --- /dev/null +++ b/bindings/soth-py/python/soth/__init__.py @@ -0,0 +1,158 @@ +"""SOTH SDK for Python. + +Public API: + init(...) -> SothSdk + SothBlocked -> exception (does NOT inherit from any + provider SDK exception type; propagates + past `try/except openai.APIError`) + BlockReason -> typed reason carried on SothBlocked + +The Decision API contract is locked by `docs/common/SDK_DECISION_API_SPEC.md`. + +Quick start: + + import soth, openai + + soth.init( + api_key="sk-...", + org_id="org-123", + hmac_key_env="SOTH_HMAC_KEY", + ) + client = openai.OpenAI() + try: + response = soth.guard( + lambda: client.chat.completions.create( + model="gpt-4o-mini", + messages=[{"role": "user", "content": "hello"}], + ), + call={ + "provider": "openai", + "model": "gpt-4o-mini", + "messages": [{"role": "user", "content": "hello"}], + }, + ) + except soth.SothBlocked as e: + print("blocked:", e.reason) +""" + +from __future__ import annotations + +from typing import Any, Callable, Optional, TypeVar + +from . import _soth_native # type: ignore[attr-defined] +from .exceptions import ( + BlockReason, + SothBlocked, + SothFlagged, + block_reason_from_dict, +) + +__version__ = _soth_native.__version__ + +__all__ = [ + "init", + "guard", + "SothBlocked", + "SothFlagged", + "BlockReason", +] + +# Module-level singleton. Bindings keep one SothSdk per process; per-call +# context (org/user/team override) is layered on top via `with_context`. +_singleton: Optional[_soth_native.SothSdk] = None + +T = TypeVar("T") + + +def init( + *, + api_key: str, + org_id: str, + hmac_key_env: Optional[str] = None, + hmac_key_static: Optional[bytes] = None, +) -> None: + """Initialize the SOTH SDK module-level singleton. + + Specify exactly one of `hmac_key_env` (read from environment) or + `hmac_key_static` (raw bytes). Production usage SHOULD prefer + `hmac_key_env` so the key never sits in source-controlled config. + """ + global _singleton + _singleton = _soth_native.SothSdk( + api_key=api_key, + org_id=org_id, + hmac_key_env=hmac_key_env, + hmac_key_static=hmac_key_static, + ) + + +def get_sdk() -> _soth_native.SothSdk: + """Return the initialized SDK or raise if `init` hasn't run.""" + if _singleton is None: + raise RuntimeError( + "soth.init(...) must be called before any guard() / SDK call" + ) + return _singleton + + +def guard( + call_fn: Callable[[], T], + *, + call: dict[str, Any], + response_extractor: Optional[Callable[[T], dict[str, Any]]] = None, +) -> T: + """Wrap an LLM call with SOTH's pre/post decision lifecycle. + + Translates `Decision::Block` into a raised `SothBlocked` and + `Decision::Flag` into a logged `SothFlagged` warning. `Allow` and + `Redact` proceed to invoke `call_fn` (Redact handling is a Phase-1 + deliverable; v0 treats Redact as Allow with the redactions logged). + + `call_fn` is the customer's existing call (e.g. + `client.chat.completions.create(...)`); the wrapper is intentionally + narrow so it can be applied per-call with minimal disruption. + """ + sdk = get_sdk() + decision = sdk.pre_call(call) + kind = decision["kind"] + token = decision["token"] + + if kind == _soth_native.DECISION_KIND_BLOCK: + # Consume the token so the slab balances even on block. + sdk.post_call(token, None) + raise SothBlocked( + decision_id=str(token), + reason=block_reason_from_dict(decision.get("reason", {})), + ) + + # Allow / Flag / Redact (Redact is Phase-1; treat as Allow + log) + try: + result = call_fn() + finally: + # Always consume the token, even on host-level failure. + response_dict = ( + response_extractor(result) # type: ignore[name-defined] + if response_extractor and "result" in dir() + else None + ) + sdk.post_call(token, response_dict) + + if kind == _soth_native.DECISION_KIND_FLAG: + # Surface the flag through a logger; customers can install + # handlers to act on it. Does NOT raise. + import logging + + logging.getLogger("soth").warning( + "soth flagged call: severity=%s", decision.get("severity") + ) + + return result + + +# Test-only re-exports (used by `tests/test_smoke.py` etc.) +def _drain_telemetry_for_test() -> list[dict[str, Any]]: + return get_sdk().drain_telemetry_for_test() + + +def _in_flight_decisions() -> int: + return get_sdk().in_flight_decisions() diff --git a/bindings/soth-py/python/soth/_soth_native.pyi b/bindings/soth-py/python/soth/_soth_native.pyi new file mode 100644 index 00000000..17e716a4 --- /dev/null +++ b/bindings/soth-py/python/soth/_soth_native.pyi @@ -0,0 +1,31 @@ +"""Type stubs for the compiled `_soth_native` extension. + +Generated manually; the public Python surface is in `soth/__init__.py`. +""" + +from __future__ import annotations + +from typing import Any, Optional + +__version__: str + +DECISION_KIND_ALLOW: str +DECISION_KIND_BLOCK: str +DECISION_KIND_REDACT: str +DECISION_KIND_FLAG: str + + +class SothSdk: + def __init__( + self, + *, + api_key: str, + org_id: str, + hmac_key_env: Optional[str] = None, + hmac_key_static: Optional[bytes] = None, + ) -> None: ... + + def pre_call(self, call: dict[str, Any]) -> dict[str, Any]: ... + def post_call(self, token: int, response: Optional[dict[str, Any]] = None) -> None: ... + def in_flight_decisions(self) -> int: ... + def drain_telemetry_for_test(self) -> list[dict[str, Any]]: ... diff --git a/bindings/soth-py/python/soth/exceptions.py b/bindings/soth-py/python/soth/exceptions.py new file mode 100644 index 00000000..daa2f7b7 --- /dev/null +++ b/bindings/soth-py/python/soth/exceptions.py @@ -0,0 +1,88 @@ +"""SOTH exceptions and reason types. + +The exception hierarchy is locked by `SDK_DECISION_API_SPEC.md` §6.2: + +- `SothBlocked` extends Python's built-in `Exception` directly. It does + NOT inherit from any provider SDK exception type (`openai.APIError`, + `anthropic.APIError`, `cohere.CohereError`, ...). +- `SothBlocked` therefore propagates past `try/except openai.APIError` + handlers — this is intentional. A policy block is not an upstream API + error and must not be retried by retry-on-API-error logic. + +If a future change makes `SothBlocked` inherit from any provider type, +the `tests/test_blocked_propagates.py` negative tests will fail. That's +the intended forcing function — read the spec section before touching +this file. +""" + +from __future__ import annotations + +from dataclasses import dataclass +from typing import Any, Optional + + +@dataclass(frozen=True) +class BlockReason: + """Typed reason carried on `SothBlocked`. The `kind` field is the + discriminator; the rest of the fields populate based on it.""" + + kind: str # "sensitive_artifact" | "budget_exceeded" | "policy_rule" | "use_alternative" + artifact: Optional[str] = None + severity: Optional[str] = None + budget_kind: Optional[str] = None + observed: Optional[int] = None + limit: Optional[int] = None + rule_id: Optional[str] = None + rule_name: Optional[str] = None + suggested_provider: Optional[str] = None + suggested_model: Optional[str] = None + + +def block_reason_from_dict(d: dict[str, Any]) -> BlockReason: + return BlockReason( + kind=str(d.get("kind", "unknown")), + artifact=d.get("artifact"), + severity=d.get("severity"), + budget_kind=d.get("budget_kind"), + observed=d.get("observed"), + limit=d.get("limit"), + rule_id=d.get("rule_id"), + rule_name=d.get("rule_name"), + suggested_provider=d.get("suggested_provider"), + suggested_model=d.get("suggested_model"), + ) + + +class SothBlocked(Exception): + """Raised when SOTH policy blocks an LLM call. + + Inherits from Exception, NOT from any provider SDK exception type. + Will propagate past `try/except openai.APIError` handlers — this is + intentional. A policy block is not an upstream API error and must + not be retried by retry-on-API-error logic. + + See `SDK_DECISION_API_SPEC.md` §6 for the full contract. + """ + + decision_id: str + reason: BlockReason + + def __init__(self, decision_id: str, reason: BlockReason): + self.decision_id = decision_id + self.reason = reason + super().__init__(f"SOTH policy blocked call: {reason.kind}") + + +class SothFlagged(Warning): + """Surfaces a `Decision::Flag` to anyone listening on warnings. + + Does NOT inherit from any provider exception type either. Customers + that want to act on flags install a `warnings.simplefilter` or + capture the `soth` logger output. + """ + + severity: str + + def __init__(self, severity: str): + self.severity = severity + super().__init__(f"SOTH flagged call: severity={severity}") diff --git a/bindings/soth-py/src/lib.rs b/bindings/soth-py/src/lib.rs new file mode 100644 index 00000000..56af6d89 --- /dev/null +++ b/bindings/soth-py/src/lib.rs @@ -0,0 +1,339 @@ +//! `soth-py` — PyO3 binding for the SOTH SDK. +//! +//! The Rust-level extension exposes a small surface; the user-facing +//! Python API lives in `python/soth/__init__.py`, which wraps this +//! extension with native Python helpers (`SothBlocked` exception, +//! context manager, auto-instrumentation). +//! +//! Decision API contract from `SDK_DECISION_API_SPEC.md` §6 lives +//! partly here (the FFI boundary) and partly in `python/soth/exceptions.py` +//! (the `SothBlocked` exception + propagation tests). + +use std::sync::Arc; + +use pyo3::exceptions::{PyRuntimeError, PyValueError}; +use pyo3::prelude::*; +use pyo3::types::{PyDict, PyList}; +use soth_sdk_core::{ + BlockReason as CoreBlockReason, Decision as CoreDecision, DecisionToken, FlagSeverity, + HmacKey, LlmCall, LlmResponse, Message, SdkConfigBuilder, SothSdk as CoreSothSdk, Tool, +}; +use soth_core::EndpointType; +use zeroize::Zeroizing; + +/// PyO3 wrapper around `SothSdk`. Stored as `Arc` so it can be +/// freely cloned across Python's threaded callers — the underlying +/// SothSdk is `Send + Sync` (compile-time asserted in soth-sdk-core). +#[pyclass(name = "SothSdk", module = "soth._soth_native")] +struct PySothSdk { + inner: Arc, +} + +#[pymethods] +impl PySothSdk { + /// Construct a new SDK instance. Per the spec, `init` failures + /// (bundle pull, HMAC key resolution) raise a Python exception + /// with a descriptive message; bindings' wrappers SHOULD catch + /// these and fall back to a no-op SDK rather than crashing the + /// host process. + #[new] + #[pyo3(signature = (api_key, org_id, hmac_key_env=None, hmac_key_static=None))] + fn new( + api_key: String, + org_id: String, + hmac_key_env: Option, + hmac_key_static: Option>, + ) -> PyResult { + let hmac_key = match (hmac_key_env, hmac_key_static) { + (Some(env), None) => HmacKey::FromEnv(env), + (None, Some(bytes)) => HmacKey::Static(Zeroizing::new(bytes)), + (Some(_), Some(_)) => { + return Err(PyValueError::new_err( + "specify either hmac_key_env OR hmac_key_static, not both", + )); + } + (None, None) => { + return Err(PyValueError::new_err( + "hmac_key_env or hmac_key_static is required", + )); + } + }; + + let config = SdkConfigBuilder::new() + .api_key(api_key) + .org_id(org_id) + .hmac_key(hmac_key) + .build() + .map_err(|error| PyValueError::new_err(format!("{error}")))?; + + let sdk = CoreSothSdk::init(config) + .map_err(|error| PyRuntimeError::new_err(format!("{error}")))?; + + Ok(Self { + inner: Arc::new(sdk), + }) + } + + /// Synchronous decision path. + /// + /// Returns a `dict` describing the decision; the Python wrapper in + /// `soth/__init__.py` translates this into either a token (`Allow` / + /// `Flag`) or a raised `SothBlocked` exception (`Block` / + /// `Redact` paths). + #[pyo3(signature = (call_dict))] + fn pre_call<'py>(&self, py: Python<'py>, call_dict: &Bound<'py, PyDict>) -> PyResult> { + let call = build_llm_call(call_dict)?; + let decision = self.inner.pre_call(&call); + decision_to_pydict(py, &decision) + } + + /// Consume a `DecisionToken` after the host call completes. + /// Bindings spawn this off the host's critical path. + #[pyo3(signature = (token, response_dict=None))] + fn post_call(&self, token: u64, response_dict: Option<&Bound<'_, PyDict>>) -> PyResult<()> { + let token = DecisionToken::from_raw(token); + let response = response_from_pydict(response_dict)?; + self.inner.post_call(token, &response); + Ok(()) + } + + /// Return the in-flight DecisionToken count. Test helper — + /// bindings expose it for parity assertions. + fn in_flight_decisions(&self) -> usize { + self.inner.in_flight_decisions() + } + + /// Drain the in-memory telemetry queue. Test-only — production + /// shippers will pull batches via the Phase-1 transport API. + fn drain_telemetry_for_test<'py>( + &self, + py: Python<'py>, + ) -> PyResult> { + let events = self.inner.drain_telemetry_for_test(); + let result = PyList::empty_bound(py); + for event in events { + let event_dict = PyDict::new_bound(py); + event_dict.set_item("provider", event.provider)?; + if let Some(model) = event.model { + event_dict.set_item("model", model)?; + } + event_dict.set_item("endpoint_type", format!("{:?}", event.endpoint_type))?; + event_dict.set_item("capture_mode", format!("{:?}", event.capture_mode))?; + event_dict.set_item("use_case", format!("{:?}", event.use_case))?; + event_dict.set_item( + "volatility_class", + format!("{:?}", event.volatility_class), + )?; + result.append(event_dict)?; + } + Ok(result) + } +} + +/// Decision API constants exposed at the module level so the Python +/// wrapper can reference them without a string match. +const DECISION_KIND_ALLOW: &str = "allow"; +const DECISION_KIND_BLOCK: &str = "block"; +const DECISION_KIND_REDACT: &str = "redact"; +const DECISION_KIND_FLAG: &str = "flag"; + +fn decision_to_pydict<'py>( + py: Python<'py>, + decision: &CoreDecision, +) -> PyResult> { + let dict = PyDict::new_bound(py); + dict.set_item("token", decision.token().raw())?; + match decision { + CoreDecision::Allow { .. } => { + dict.set_item("kind", DECISION_KIND_ALLOW)?; + } + CoreDecision::Block { reason, .. } => { + dict.set_item("kind", DECISION_KIND_BLOCK)?; + dict.set_item("reason", block_reason_to_pydict(py, reason)?)?; + } + CoreDecision::Redact { redactions, .. } => { + dict.set_item("kind", DECISION_KIND_REDACT)?; + let list = PyList::empty_bound(py); + for r in &redactions.replacements { + let item = PyDict::new_bound(py); + item.set_item("message_idx", r.message_idx)?; + item.set_item("redacted_content", r.redacted_content.clone())?; + list.append(item)?; + } + dict.set_item("redactions", list)?; + } + CoreDecision::Flag { severity, .. } => { + dict.set_item("kind", DECISION_KIND_FLAG)?; + dict.set_item("severity", flag_severity_label(*severity))?; + } + // Decision is #[non_exhaustive] — future variants surface as + // "unknown" so existing bindings keep emitting *something* for + // the host's wrapper to consume rather than throwing FFI errors. + _ => { + dict.set_item("kind", "unknown")?; + } + } + Ok(dict) +} + +fn block_reason_to_pydict<'py>( + py: Python<'py>, + reason: &CoreBlockReason, +) -> PyResult> { + let dict = PyDict::new_bound(py); + match reason { + CoreBlockReason::SensitiveArtifact { artifact, severity } => { + dict.set_item("kind", "sensitive_artifact")?; + dict.set_item("artifact", format!("{artifact:?}"))?; + dict.set_item("severity", format!("{severity:?}"))?; + } + CoreBlockReason::BudgetExceeded { + budget_kind, + observed, + limit, + } => { + dict.set_item("kind", "budget_exceeded")?; + dict.set_item("budget_kind", format!("{budget_kind:?}"))?; + dict.set_item("observed", *observed)?; + dict.set_item("limit", *limit)?; + } + CoreBlockReason::PolicyRule { rule_id, rule_name } => { + dict.set_item("kind", "policy_rule")?; + dict.set_item("rule_id", rule_id.clone())?; + if let Some(name) = rule_name { + dict.set_item("rule_name", name.clone())?; + } + } + CoreBlockReason::UseAlternative { + suggested_provider, + suggested_model, + rule_id, + } => { + dict.set_item("kind", "use_alternative")?; + if let Some(p) = suggested_provider { + dict.set_item("suggested_provider", p.clone())?; + } + if let Some(m) = suggested_model { + dict.set_item("suggested_model", m.clone())?; + } + dict.set_item("rule_id", rule_id.clone())?; + } + // BlockReason is #[non_exhaustive] — fall back to a generic + // shape if soth-sdk-core adds a new variant before this binding + // catches up. + _ => { + dict.set_item("kind", "unknown")?; + } + } + Ok(dict) +} + +fn flag_severity_label(severity: FlagSeverity) -> &'static str { + match severity { + FlagSeverity::Info => "info", + FlagSeverity::Warning => "warning", + FlagSeverity::Critical => "critical", + // FlagSeverity is #[non_exhaustive] — future variants surface + // as "unknown" so the binding keeps working. + _ => "unknown", + } +} + +fn build_llm_call(dict: &Bound<'_, PyDict>) -> PyResult { + let provider: String = dict + .get_item("provider")? + .ok_or_else(|| PyValueError::new_err("call.provider required"))? + .extract()?; + let model: String = dict + .get_item("model")? + .ok_or_else(|| PyValueError::new_err("call.model required"))? + .extract()?; + let messages_obj = dict + .get_item("messages")? + .ok_or_else(|| PyValueError::new_err("call.messages required"))?; + let messages_list: &Bound<'_, PyList> = messages_obj.downcast()?; + + let mut messages = Vec::with_capacity(messages_list.len()); + for item in messages_list.iter() { + let item_dict: &Bound<'_, PyDict> = item.downcast()?; + let role: String = item_dict + .get_item("role")? + .ok_or_else(|| PyValueError::new_err("message.role required"))? + .extract()?; + let content: String = item_dict + .get_item("content")? + .ok_or_else(|| PyValueError::new_err("message.content required"))? + .extract()?; + messages.push(Message { role, content }); + } + + let system: Option = match dict.get_item("system")? { + Some(v) if !v.is_none() => Some(v.extract()?), + _ => None, + }; + let stream: bool = match dict.get_item("stream")? { + Some(v) if !v.is_none() => v.extract()?, + _ => false, + }; + + let tools: Vec = match dict.get_item("tools")? { + Some(v) if !v.is_none() => { + let list: &Bound<'_, PyList> = v.downcast()?; + let mut out = Vec::with_capacity(list.len()); + for item in list.iter() { + let item_dict: &Bound<'_, PyDict> = item.downcast()?; + let name: String = item_dict + .get_item("name")? + .ok_or_else(|| PyValueError::new_err("tool.name required"))? + .extract()?; + let description: Option = match item_dict.get_item("description")? { + Some(v) if !v.is_none() => Some(v.extract()?), + _ => None, + }; + let parameters_json: String = match item_dict.get_item("parameters_json")? { + Some(v) if !v.is_none() => v.extract()?, + _ => String::new(), + }; + out.push(Tool { + name, + description, + parameters_json, + }); + } + out + } + _ => Vec::new(), + }; + + Ok(LlmCall { + provider, + model, + messages, + system, + tools, + stream, + temperature: None, + top_p: None, + max_tokens: None, + stop_sequences: Vec::new(), + endpoint_type: EndpointType::ChatCompletion, + }) +} + +fn response_from_pydict(_dict: Option<&Bound<'_, PyDict>>) -> PyResult { + // V0: response details are not yet consumed by post_call. Phase-1 + // wires response-side artifact scanning + usage stats from the + // typed response dict. + Ok(LlmResponse::new(EndpointType::ChatCompletion)) +} + +#[pymodule] +fn _soth_native(_py: Python<'_>, m: &Bound<'_, PyModule>) -> PyResult<()> { + m.add_class::()?; + m.add("__version__", env!("CARGO_PKG_VERSION"))?; + m.add("DECISION_KIND_ALLOW", DECISION_KIND_ALLOW)?; + m.add("DECISION_KIND_BLOCK", DECISION_KIND_BLOCK)?; + m.add("DECISION_KIND_REDACT", DECISION_KIND_REDACT)?; + m.add("DECISION_KIND_FLAG", DECISION_KIND_FLAG)?; + Ok(()) +} diff --git a/bindings/soth-py/tests/test_blocked_propagates.py b/bindings/soth-py/tests/test_blocked_propagates.py new file mode 100644 index 00000000..547aa731 --- /dev/null +++ b/bindings/soth-py/tests/test_blocked_propagates.py @@ -0,0 +1,104 @@ +"""Negative tests for the `SothBlocked` propagation contract. + +`SDK_DECISION_API_SPEC.md` §6 commits that `SothBlocked` does NOT +inherit from any provider SDK exception type and MUST propagate past +existing `try/except openai.APIError` handlers. Customers' retry +logic catches `openai.APIError` to retry on rate limits / 5xx; a +policy block must NOT be silently retried. + +If any future change to `SothBlocked` makes it inherit from +`openai.APIError` (or any provider's hierarchy), one of these tests +fails immediately. + +Run with: + cd bindings/soth-py + pip install ".[test]" + maturin develop + pytest tests/test_blocked_propagates.py +""" + +import os + +import pytest + +import soth + +# Some test environments don't have openai installed. Skip the negative +# tests rather than failing — but DO emit a clear message so CI catches +# the missing dep. +openai = pytest.importorskip( + "openai", + reason="openai is required for the SothBlocked propagation contract test " + "(install via `pip install soth[test]` or `pip install openai`)", +) + + +def _make_blocking_call(): + return soth.guard( + lambda: "should not be called", + call={ + "provider": "openai", + "model": "gpt-4o-mini", + "messages": [ + { + "role": "user", + "content": ( + "leaked sk-abcdefghijklmnopqrstuvwxyzABCD1234567890 here" + ), + } + ], + }, + ) + + +@pytest.fixture(autouse=True) +def _init_sdk(): + os.environ["SOTH_HMAC_KEY"] = "x" * 32 + soth.init( + api_key="sk-test", + org_id="org-test", + hmac_key_env="SOTH_HMAC_KEY", + ) + yield + + +def test_soth_blocked_does_not_inherit_from_openai_apierror(): + """Static check — if this fails the inheritance hierarchy is wrong.""" + assert not issubclass(soth.SothBlocked, openai.APIError), ( + "SothBlocked must NOT inherit from openai.APIError. See " + "SDK_DECISION_API_SPEC.md §6.1." + ) + + +def test_soth_blocked_propagates_past_openai_apierror_handler(): + """Customer code with `try/except openai.APIError` MUST NOT swallow + SothBlocked. The block propagates past the API-error handler.""" + caught_apierror = False + caught_soth = False + + try: + try: + _make_blocking_call() + except openai.APIError: + caught_apierror = True + except soth.SothBlocked: + caught_soth = True + + assert not caught_apierror, ( + "SothBlocked was caught by `except openai.APIError` — that's a " + "spec violation. See SDK_DECISION_API_SPEC.md §6.1." + ) + assert caught_soth, "SothBlocked must propagate past openai.APIError" + + +def test_soth_blocked_inherits_from_base_exception_directly(): + """SothBlocked extends Exception, not BaseException, so KeyboardInterrupt + handling isn't accidentally trapped.""" + # Exception in MRO; BaseException at the top. + mro_names = [cls.__name__ for cls in soth.SothBlocked.__mro__] + assert "Exception" in mro_names + # Should not be a BaseException-only inheritor (which would be a + # subclass of GeneratorExit / KeyboardInterrupt etc.). + assert mro_names[1] == "Exception", ( + "SothBlocked must inherit directly from Exception. Spec §6.1." + ) diff --git a/bindings/soth-py/tests/test_smoke.py b/bindings/soth-py/tests/test_smoke.py new file mode 100644 index 00000000..0efd57b8 --- /dev/null +++ b/bindings/soth-py/tests/test_smoke.py @@ -0,0 +1,81 @@ +"""Smoke tests for soth-py. + +Mirror the round-trip tests in `crates/soth-sdk-core/tests/round_trip.rs`, +asserting the FFI layer doesn't introduce drift. + +Run with: + cd bindings/soth-py + maturin develop + pytest tests/ +""" + +import os + +import pytest + +import soth + + +@pytest.fixture(autouse=True) +def _init_sdk(): + os.environ["SOTH_HMAC_KEY"] = "x" * 32 + soth.init( + api_key="sk-test", + org_id="org-test", + hmac_key_env="SOTH_HMAC_KEY", + ) + yield + + +def test_init_creates_singleton(): + sdk = soth.get_sdk() + assert sdk.in_flight_decisions() == 0 + + +def test_pre_post_call_round_trip_emits_telemetry(): + captured = {} + + def fake_call(): + captured["called"] = True + return "ok" + + result = soth.guard( + fake_call, + call={ + "provider": "openai", + "model": "gpt-4o-mini", + "messages": [{"role": "user", "content": "hello"}], + }, + ) + assert result == "ok" + assert captured.get("called") is True + assert soth._in_flight_decisions() == 0 + events = soth._drain_telemetry_for_test() + assert len(events) == 1 + assert events[0]["provider"] == "openai" + + +def test_credential_in_user_message_blocks(): + def fake_call(): + return "should not be called" + + with pytest.raises(soth.SothBlocked) as excinfo: + soth.guard( + fake_call, + call={ + "provider": "openai", + "model": "gpt-4o-mini", + "messages": [ + { + "role": "user", + "content": ( + "review this key sk-abcdefghijklmnopqrstuvwxyzABCD1234567890" + " for me" + ), + } + ], + }, + ) + + assert excinfo.value.reason.kind == "sensitive_artifact" + assert soth._in_flight_decisions() == 0 diff --git a/crates/soth-sdk-core/src/decision.rs b/crates/soth-sdk-core/src/decision.rs index ff7fec87..897ed495 100644 --- a/crates/soth-sdk-core/src/decision.rs +++ b/crates/soth-sdk-core/src/decision.rs @@ -154,6 +154,24 @@ impl DecisionToken { /// at the boundary and a fail-open `Decision::Allow` was emitted. pub const SENTINEL_FAIL_OPEN: DecisionToken = DecisionToken { inner: u64::MAX - 1 }; + /// Opaque round-trip handle for FFI bindings. Bindings serialize + /// the token across the language boundary as the returned u64; + /// they MUST NOT interpret the bits or attempt to construct a + /// `DecisionToken` from arbitrary values. + pub fn raw(self) -> u64 { + self.inner + } + + /// Reconstruct a `DecisionToken` from a value previously obtained + /// via [`raw`]. Bindings use this to round-trip the token across + /// the FFI boundary; passing values not previously emitted by the + /// SDK is undefined behavior at the slab level (the slab will + /// reject the token as stale and emit a `decision_orphaned` + /// telemetry event). + pub fn from_raw(raw: u64) -> Self { + Self { inner: raw } + } + pub(crate) fn is_sentinel(self) -> bool { self == Self::SLAB_FULL || self == Self::SENTINEL_FAIL_OPEN } From 026fa1149d4c26feb9500e273831aba8820888a0 Mon Sep 17 00:00:00 2001 From: Apoorv Raj Saxena Date: Thu, 30 Apr 2026 13:02:33 +0530 Subject: [PATCH 05/24] feat(sdk): Phase 1 - soth-node napi-rs binding scaffold MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit bindings/soth-node/ — Node.js binding consuming soth-sdk-core via napi-rs. @napi-rs/cli builds prebuilt binaries per platform/arch; published to npm as @soth/sdk. Public TS surface (index.js + index.d.ts): init({ apiKey, orgId, hmacKeyEnv }) — module-level singleton guard(asyncFn, { call }) — pre/post lifecycle wrapper SothBlocked extends Error — NOT extends OpenAI.APIError SothFlagged — Flag surface BlockReason — typed reason interface Negative tests (__test__/blocked-propagates.test.mjs) gate the contract: - SothBlocked does not extend openai.APIError (instanceof check) - SothBlocked propagates past try { ... } catch (OpenAI.APIError) blocks - SothBlocked extends Error directly Per SDK_DECISION_API_SPEC.md §6.3, customers' retry-on-API-error logic must NOT silently retry policy blocks. The negative test will fail loudly if the inheritance changes. DecisionToken round-trip: tokens are stringified u64 across the FFI because JS numbers can't safely hold the full u64 range. Bindings parse the string back into the u64 via DecisionToken::from_raw, which was added in this commit's companion soth-sdk-core change. Cargo / build: - bindings/soth-node added to workspace members, NOT default-members - napi-rs 2.x with napi8 ABI feature (Node 18+ supported) - `cargo build -p soth-node` compiles the cdylib clean - Production binaries built via `npm run build` per the @napi-rs/cli triples in package.json (linux-x64-{gnu,musl}, aarch64-linux-{gnu,musl}, darwin-arm64, darwin-x64, win32-x64) Verification: cargo build -p soth-node clean cargo build --workspace clean (both bindings included) All workspace lib tests unchanged Phase-1 follow-ups documented in bindings/soth-node/README.md: auto-instrumentation, undici dispatcher, withContext helper, AsyncIterable streaming wrapper, per-arch loader logic. Co-Authored-By: Claude Opus 4.7 (1M context) --- bindings/soth-node/Cargo.toml | 24 ++ bindings/soth-node/README.md | 77 ++++ bindings/soth-node/__test__/basic.test.mjs | 72 ++++ .../__test__/blocked-propagates.test.mjs | 101 ++++++ bindings/soth-node/build.rs | 5 + bindings/soth-node/index.d.ts | 64 ++++ bindings/soth-node/index.js | 87 +++++ bindings/soth-node/package.json | 43 +++ bindings/soth-node/src/lib.rs | 334 ++++++++++++++++++ 9 files changed, 807 insertions(+) create mode 100644 bindings/soth-node/Cargo.toml create mode 100644 bindings/soth-node/README.md create mode 100644 bindings/soth-node/__test__/basic.test.mjs create mode 100644 bindings/soth-node/__test__/blocked-propagates.test.mjs create mode 100644 bindings/soth-node/build.rs create mode 100644 bindings/soth-node/index.d.ts create mode 100644 bindings/soth-node/index.js create mode 100644 bindings/soth-node/package.json create mode 100644 bindings/soth-node/src/lib.rs diff --git a/bindings/soth-node/Cargo.toml b/bindings/soth-node/Cargo.toml new file mode 100644 index 00000000..ac79e7ee --- /dev/null +++ b/bindings/soth-node/Cargo.toml @@ -0,0 +1,24 @@ +[package] +name = "soth-node" +description = "Node.js binding for the SOTH SDK (napi-rs)." +version.workspace = true +edition.workspace = true +license.workspace = true +repository.workspace = true +authors.workspace = true +publish = false + +[lib] +crate-type = ["cdylib"] + +[dependencies] +soth-sdk-core = { path = "../../crates/soth-sdk-core" } +soth-core = { workspace = true } +zeroize = { workspace = true } +napi = { version = "2", default-features = false, features = ["napi8", "serde-json"] } +napi-derive = "2" +serde = { workspace = true } +serde_json = { workspace = true } + +[build-dependencies] +napi-build = "2" diff --git a/bindings/soth-node/README.md b/bindings/soth-node/README.md new file mode 100644 index 00000000..c9d30e08 --- /dev/null +++ b/bindings/soth-node/README.md @@ -0,0 +1,77 @@ +# @soth/sdk (soth-node) + +Node.js binding for the SOTH SDK. Built with napi-rs; published to npm +as `@soth/sdk` with prebuilt binaries per platform/arch. + +## Status + +**Phase 1 scaffold.** The Rust extension exposes the `SothSdk` facade +through napi-rs; the JS shim in `index.js` wraps it with the `guard()` +helper, the `SothBlocked` exception class, and the contract negative +tests in `__test__/blocked-propagates.test.mjs`. + +What v0 ships: +- `soth.init({ apiKey, orgId, hmacKeyEnv })` — module-level singleton +- `soth.guard(asyncFn, { call })` — wraps an LLM call with `pre_call` / `post_call` +- `soth.SothBlocked` — class extending `Error` (NOT `OpenAI.APIError`) +- `soth.SothFlagged` — surface for `Decision::Flag` + +What's deferred to follow-up Phase 1 commits: +- Auto-instrumentation for `openai`, `@anthropic-ai/sdk`, `cohere-ai`, + `@google/generative-ai`, `mistralai` +- undici dispatcher (`soth.fetch`) +- `soth.withContext({ userId, teamId }, async () => ...)` per-call overrides +- Streaming wrapper for `AsyncIterable` +- Per-arch binary loader (today: hardcoded for `darwin-arm64` for local + smoke; production loader lands with the wheel matrix work) + +## Building + +```sh +cd bindings/soth-node +npm install +npm run build:debug # local dev — produces soth-node..node +``` + +The Rust extension is part of the workspace, so `cargo build -p soth-node` +also works for compile-checking. Note: `cargo build -p soth-node` requires +Node and the napi build tooling (`napi-build` build dep); CI installs +these automatically. + +## Tests + +```sh +npm install +npm run build:debug +npm test +``` + +The `__test__/blocked-propagates.test.mjs` suite is the contract gate: +`SothBlocked` MUST NOT be `instanceof OpenAI.APIError`. If that test +fails, the inheritance has drifted from the spec and bindings cannot +ship. + +## Binary matrix (Phase-1 deliverable) + +| Platform | Architecture | +|---|---| +| linux-x64 (gnu, musl) | x86_64 | +| linux-arm64 (gnu, musl) | aarch64 | +| darwin-arm64 | aarch64 | +| darwin-x64 | x86_64 | +| win32-x64-msvc | x86_64 | + +Built via `@napi-rs/cli` in CI; postinstall picks the right binary for +the host platform. + +Node 18+ required. + +## Public API contract + +Locked by: +- `docs/common/SDK_DECISION_API_SPEC.md` — Decision lifecycle, exception contract +- `docs/common/SDK_WASM_TRUST_BOUNDARY_SPEC.md` — bundle / classification mode + +Read those before changing any public symbol in `index.js` / `index.d.ts` +or the napi-rs wrapper. `@soth/sdk` ships in customer dependencies and +breaking changes propagate downstream. diff --git a/bindings/soth-node/__test__/basic.test.mjs b/bindings/soth-node/__test__/basic.test.mjs new file mode 100644 index 00000000..6e779d14 --- /dev/null +++ b/bindings/soth-node/__test__/basic.test.mjs @@ -0,0 +1,72 @@ +// Smoke test for @soth/sdk. Mirrors the round-trip tests in +// `crates/soth-sdk-core/tests/round_trip.rs`. +// +// Run with: +// cd bindings/soth-node +// npm install +// npm run build:debug +// npm test + +import { test } from 'node:test'; +import { strict as assert } from 'node:assert'; + +import * as soth from '../index.js'; + +process.env.SOTH_HMAC_KEY = 'x'.repeat(32); + +soth.init({ + apiKey: 'sk-test', + orgId: 'org-test', + hmacKeyEnv: 'SOTH_HMAC_KEY', +}); + +test('pre/post round trip emits telemetry and balances slab', async () => { + let called = false; + const result = await soth.guard( + async () => { + called = true; + return 'ok'; + }, + { + call: { + provider: 'openai', + model: 'gpt-4o-mini', + messages: [{ role: 'user', content: 'hello' }], + }, + }, + ); + assert.equal(result, 'ok'); + assert.ok(called); + const sdk = soth.getSdk(); + assert.equal(sdk.inFlightDecisions(), 0); + const events = sdk.drainTelemetryForTest(); + assert.equal(events.length, 1); + assert.equal(events[0].provider, 'openai'); +}); + +test('credential in user message blocks', async () => { + await assert.rejects( + () => soth.guard( + async () => 'should not be called', + { + call: { + provider: 'openai', + model: 'gpt-4o-mini', + messages: [ + { + role: 'user', + content: 'leaked sk-abcdefghijklmnopqrstuvwxyzABCD1234567890 here', + }, + ], + }, + }, + ), + (err) => { + assert.ok(err instanceof soth.SothBlocked); + assert.equal(err.reason.kind, 'sensitive_artifact'); + return true; + }, + ); + const sdk = soth.getSdk(); + assert.equal(sdk.inFlightDecisions(), 0); +}); diff --git a/bindings/soth-node/__test__/blocked-propagates.test.mjs b/bindings/soth-node/__test__/blocked-propagates.test.mjs new file mode 100644 index 00000000..036d478a --- /dev/null +++ b/bindings/soth-node/__test__/blocked-propagates.test.mjs @@ -0,0 +1,101 @@ +// Negative tests for the SothBlocked propagation contract. +// +// SDK_DECISION_API_SPEC.md §6.3 commits that SothBlocked extends Error +// (not any provider exception class) and propagates past existing +// `try { ... } catch (e) { if (e instanceof OpenAI.APIError) ... }` blocks. +// Customers' retry logic catches OpenAI.APIError to retry on rate +// limits / 5xx; a policy block must NOT be silently retried. +// +// If any future change makes SothBlocked extend OpenAI.APIError or any +// provider's class, these tests fail immediately. +// +// Run with: +// cd bindings/soth-node +// npm install +// npm run build:debug +// npm test + +import { test } from 'node:test'; +import { strict as assert } from 'node:assert'; + +import * as soth from '../index.js'; + +process.env.SOTH_HMAC_KEY = 'x'.repeat(32); + +let openai; +try { + openai = await import('openai'); +} catch (_) { + console.warn('skipping propagation tests — openai package not installed'); +} + +soth.init({ + apiKey: 'sk-test', + orgId: 'org-test', + hmacKeyEnv: 'SOTH_HMAC_KEY', +}); + +async function makeBlockingCall() { + return soth.guard( + async () => 'should not be called', + { + call: { + provider: 'openai', + model: 'gpt-4o-mini', + messages: [ + { + role: 'user', + content: 'leaked sk-abcdefghijklmnopqrstuvwxyzABCD1234567890 here', + }, + ], + }, + }, + ); +} + +test('SothBlocked does not extend openai.APIError', { skip: !openai }, () => { + // Static check on the prototype chain. If SothBlocked were to extend + // OpenAI.APIError, this would be true and the spec is violated. + const blocked = new soth.SothBlocked('0', { kind: 'test' }); + assert.equal( + blocked instanceof openai.OpenAI.APIError, + false, + 'SothBlocked must NOT inherit from OpenAI.APIError. See SDK_DECISION_API_SPEC.md §6.3.', + ); +}); + +test( + 'SothBlocked propagates past try { ... } catch (OpenAI.APIError) handlers', + { skip: !openai }, + async () => { + let caughtAPIError = false; + let caughtSoth = false; + + try { + try { + await makeBlockingCall(); + } catch (e) { + if (e instanceof openai.OpenAI.APIError) { + caughtAPIError = true; + } else { + throw e; + } + } + } catch (e) { + if (e instanceof soth.SothBlocked) { + caughtSoth = true; + } else { + throw e; + } + } + + assert.equal(caughtAPIError, false, 'SothBlocked was caught by OpenAI.APIError — spec violation'); + assert.equal(caughtSoth, true, 'SothBlocked must propagate past OpenAI.APIError'); + }, +); + +test('SothBlocked extends Error directly', () => { + const blocked = new soth.SothBlocked('0', { kind: 'test' }); + assert.ok(blocked instanceof Error, 'SothBlocked must extend Error'); + assert.equal(blocked.name, 'SothBlocked'); +}); diff --git a/bindings/soth-node/build.rs b/bindings/soth-node/build.rs new file mode 100644 index 00000000..9fc23678 --- /dev/null +++ b/bindings/soth-node/build.rs @@ -0,0 +1,5 @@ +extern crate napi_build; + +fn main() { + napi_build::setup(); +} diff --git a/bindings/soth-node/index.d.ts b/bindings/soth-node/index.d.ts new file mode 100644 index 00000000..fae168ef --- /dev/null +++ b/bindings/soth-node/index.d.ts @@ -0,0 +1,64 @@ +// Type declarations for @soth/sdk. + +export interface InitOptions { + apiKey: string; + orgId: string; + /** + * Read the HMAC key from this environment variable. The SDK never + * sees the plaintext over the wire; soth-cloud never has the key. + * See SDK_WASM_TRUST_BOUNDARY_SPEC.md §6.6. + */ + hmacKeyEnv?: string; + hmacKeyStatic?: Buffer; +} + +export interface Message { + role: string; + content: string; +} + +export interface Tool { + name: string; + description?: string; + parametersJson?: string; +} + +export interface LlmCall { + provider: string; + model: string; + messages: Message[]; + system?: string; + tools?: Tool[]; + stream?: boolean; +} + +export interface BlockReason { + /** "sensitive_artifact" | "budget_exceeded" | "policy_rule" | "use_alternative" */ + kind: string; + artifact?: string; + severity?: string; + budgetKind?: string; + observed?: number; + limit?: number; + ruleId?: string; + ruleName?: string; + suggestedProvider?: string; + suggestedModel?: string; +} + +export class SothBlocked extends Error { + decisionId: string; + reason: BlockReason; +} + +export class SothFlagged { + severity: string; +} + +export interface GuardOptions { + call: LlmCall; +} + +export function init(options: InitOptions): void; +export function guard(callFn: () => Promise, options: GuardOptions): Promise; +export function getSdk(): unknown; diff --git a/bindings/soth-node/index.js b/bindings/soth-node/index.js new file mode 100644 index 00000000..27436950 --- /dev/null +++ b/bindings/soth-node/index.js @@ -0,0 +1,87 @@ +// soth-node — JS shim layered on top of the napi-rs extension. +// +// The native extension exposes a low-level `SothSdk` class that returns +// typed `JsDecision` objects. This shim: +// 1. Exposes `init`, `guard`, `withContext` as the user-facing API +// 2. Translates `Decision::Block` into a thrown `SothBlocked` +// (extends `Error`, NOT `OpenAI.APIError` / etc.) +// 3. Surfaces `Decision::Flag` via the `console.warn` channel and +// a `SothFlagged` instance attached to the result for inspection +// +// Decision API contract: `docs/common/SDK_DECISION_API_SPEC.md` §6.3. +// The exception inheritance MUST stay flat — `SothBlocked extends Error`. +// If anyone changes that, `__test__/blocked-propagates.test.mjs` fails. + +const native = require('./soth-node.darwin-arm64.node'); /* eslint-disable-line global-require */ +// Real builds ship per-arch binaries via @napi-rs/cli; the line above +// is a placeholder for local dev. Production loader logic lands in a +// follow-up commit. + +class SothBlocked extends Error { + constructor(decisionId, reason) { + super(`SOTH policy blocked call: ${reason?.kind ?? 'unknown'}`); + this.name = 'SothBlocked'; + this.decisionId = decisionId; + this.reason = reason; + } +} + +class SothFlagged { + constructor(severity) { + this.severity = severity; + } +} + +let _singleton = null; + +function init({ apiKey, orgId, hmacKeyEnv, hmacKeyStatic }) { + if (!apiKey) throw new Error('init: apiKey required'); + if (!orgId) throw new Error('init: orgId required'); + _singleton = native.SothSdk.create( + apiKey, + orgId, + hmacKeyEnv ?? null, + hmacKeyStatic ?? null, + ); +} + +function getSdk() { + if (!_singleton) { + throw new Error('soth.init({...}) must be called before any guard() / SDK call'); + } + return _singleton; +} + +async function guard(callFn, { call }) { + const sdk = getSdk(); + const decision = sdk.preCall(call); + const { kind, token } = decision; + + if (kind === 'block') { + sdk.postCall(token, null); + throw new SothBlocked(token, decision.reason); + } + + // Allow / Flag / Redact (Redact treated as Allow for v0; Phase-1 + // wires actual message rewriting via per-provider adapters). + let result; + try { + result = await callFn(); + } finally { + sdk.postCall(token, null); + } + + if (kind === 'flag') { + console.warn(`soth flagged call: severity=${decision.severity}`); + } + + return result; +} + +module.exports = { + init, + guard, + getSdk, + SothBlocked, + SothFlagged, +}; diff --git a/bindings/soth-node/package.json b/bindings/soth-node/package.json new file mode 100644 index 00000000..0ae5ef0e --- /dev/null +++ b/bindings/soth-node/package.json @@ -0,0 +1,43 @@ +{ + "name": "@soth/sdk", + "version": "0.1.0", + "description": "SOTH SDK for Node.js — observability + policy enforcement for LLM API calls.", + "main": "index.js", + "types": "index.d.ts", + "license": "MIT OR Apache-2.0", + "engines": { + "node": ">=18" + }, + "files": [ + "index.js", + "index.d.ts", + "README.md" + ], + "scripts": { + "build": "napi build --platform --release", + "build:debug": "napi build --platform", + "test": "node --test __test__/*.test.mjs" + }, + "devDependencies": { + "@napi-rs/cli": "^2.18", + "openai": "^4.77" + }, + "napi": { + "name": "soth-node", + "triples": { + "defaults": false, + "additional": [ + "x86_64-unknown-linux-gnu", + "x86_64-unknown-linux-musl", + "aarch64-unknown-linux-gnu", + "aarch64-unknown-linux-musl", + "x86_64-apple-darwin", + "aarch64-apple-darwin", + "x86_64-pc-windows-msvc" + ] + } + }, + "publishConfig": { + "access": "public" + } +} diff --git a/bindings/soth-node/src/lib.rs b/bindings/soth-node/src/lib.rs new file mode 100644 index 00000000..ea52bba2 --- /dev/null +++ b/bindings/soth-node/src/lib.rs @@ -0,0 +1,334 @@ +//! `soth-node` — napi-rs binding for the SOTH SDK. +//! +//! The user-facing TypeScript surface lives in `index.d.ts` + `index.js`; +//! this crate exposes the napi-rs entry points the JS shim wraps. +//! +//! Decision API contract from `SDK_DECISION_API_SPEC.md` §6.3 lives +//! partly here (the FFI boundary returning a typed `Decision` JS object) +//! and partly in the JS shim (`SothBlocked extends Error`). + +#![deny(clippy::all)] + +use std::sync::Arc; + +use napi::bindgen_prelude::*; +use napi_derive::napi; +use soth_sdk_core::{ + BlockReason as CoreBlockReason, Decision as CoreDecision, DecisionToken, FlagSeverity, + HmacKey, LlmCall, LlmResponse, Message, SdkConfigBuilder, SothSdk as CoreSothSdk, Tool, +}; +use soth_core::EndpointType; +use zeroize::Zeroizing; + +// ── napi-exposed types ──────────────────────────────────────────────── + +#[napi(object)] +pub struct JsBlockReason { + pub kind: String, + pub artifact: Option, + pub severity: Option, + pub budget_kind: Option, + pub observed: Option, + pub limit: Option, + pub rule_id: Option, + pub rule_name: Option, + pub suggested_provider: Option, + pub suggested_model: Option, +} + +#[napi(object)] +pub struct JsDecision { + /// "allow" | "block" | "redact" | "flag" + pub kind: String, + /// `DecisionToken.inner` as a stringified u64 (JS numbers can't + /// safely hold the full u64 range; we use a string and round-trip + /// it exactly through the FFI). + pub token: String, + pub reason: Option, + pub redactions: Option>, + pub severity: Option, +} + +#[napi(object)] +pub struct JsRedaction { + pub message_idx: u32, + pub redacted_content: String, +} + +#[napi(object)] +pub struct JsMessage { + pub role: String, + pub content: String, +} + +#[napi(object)] +pub struct JsTool { + pub name: String, + pub description: Option, + pub parameters_json: Option, +} + +#[napi(object)] +pub struct JsLlmCall { + pub provider: String, + pub model: String, + pub messages: Vec, + pub system: Option, + pub tools: Option>, + pub stream: Option, +} + +#[napi(object)] +pub struct JsTelemetryEvent { + pub provider: String, + pub model: Option, + pub endpoint_type: String, + pub capture_mode: String, + pub use_case: String, + pub volatility_class: String, +} + +// ── SothSdk wrapper ────────────────────────────────────────────────── + +#[napi] +pub struct SothSdk { + inner: Arc, +} + +#[napi] +impl SothSdk { + /// Construct a new SDK instance. Mirrors `SdkConfigBuilder` for the + /// minimum-required field set; richer config (capture_mode, + /// classification_mode, etc.) lands in a follow-up commit. + #[napi(factory)] + pub fn create(api_key: String, org_id: String, hmac_key_env: Option, hmac_key_static: Option) -> Result { + let hmac_key = match (hmac_key_env, hmac_key_static) { + (Some(env), None) => HmacKey::FromEnv(env), + (None, Some(buf)) => HmacKey::Static(Zeroizing::new(buf.as_ref().to_vec())), + (Some(_), Some(_)) => { + return Err(Error::new( + Status::InvalidArg, + "specify either hmac_key_env OR hmac_key_static, not both", + )); + } + (None, None) => { + return Err(Error::new( + Status::InvalidArg, + "hmac_key_env or hmac_key_static is required", + )); + } + }; + + let config = SdkConfigBuilder::new() + .api_key(api_key) + .org_id(org_id) + .hmac_key(hmac_key) + .build() + .map_err(|e| Error::new(Status::InvalidArg, format!("{e}")))?; + + let sdk = CoreSothSdk::init(config) + .map_err(|e| Error::new(Status::GenericFailure, format!("{e}")))?; + + Ok(Self { + inner: Arc::new(sdk), + }) + } + + /// Synchronous decision path. Returns a typed `JsDecision` that the + /// JS shim translates into either a token (Allow / Flag) or a + /// thrown `SothBlocked` (Block / Redact). + #[napi] + pub fn pre_call(&self, call: JsLlmCall) -> Result { + let llm_call = build_llm_call(call); + let decision = self.inner.pre_call(&llm_call); + Ok(decision_to_js(&decision)) + } + + /// Consume a token after the host call completes. Bindings should + /// call this from a worker thread (napi-rs's threadpool) so the + /// host event loop doesn't block on classify enrichment. + #[napi] + pub fn post_call(&self, token: String, _response: Option) -> Result<()> { + let inner: u64 = token.parse().map_err(|_| { + Error::new(Status::InvalidArg, "decision token must be a numeric string") + })?; + let token = DecisionToken::from_raw(inner); + let response = LlmResponse::new(EndpointType::ChatCompletion); + self.inner.post_call(token, &response); + Ok(()) + } + + /// Test helper — number of in-flight decisions. + #[napi] + pub fn in_flight_decisions(&self) -> u32 { + self.inner.in_flight_decisions() as u32 + } + + /// Test-only — drain the in-memory telemetry queue. + #[napi] + pub fn drain_telemetry_for_test(&self) -> Vec { + self.inner + .drain_telemetry_for_test() + .into_iter() + .map(|e| JsTelemetryEvent { + provider: e.provider, + model: e.model, + endpoint_type: format!("{:?}", e.endpoint_type), + capture_mode: format!("{:?}", e.capture_mode), + use_case: format!("{:?}", e.use_case), + volatility_class: format!("{:?}", e.volatility_class), + }) + .collect() + } +} + +// ── conversions ────────────────────────────────────────────────────── + +fn build_llm_call(call: JsLlmCall) -> LlmCall { + LlmCall { + provider: call.provider, + model: call.model, + messages: call + .messages + .into_iter() + .map(|m| Message { + role: m.role, + content: m.content, + }) + .collect(), + system: call.system, + tools: call + .tools + .unwrap_or_default() + .into_iter() + .map(|t| Tool { + name: t.name, + description: t.description, + parameters_json: t.parameters_json.unwrap_or_default(), + }) + .collect(), + stream: call.stream.unwrap_or(false), + temperature: None, + top_p: None, + max_tokens: None, + stop_sequences: Vec::new(), + endpoint_type: EndpointType::ChatCompletion, + } +} + +fn decision_to_js(decision: &CoreDecision) -> JsDecision { + let token = decision.token().raw().to_string(); + match decision { + CoreDecision::Allow { .. } => JsDecision { + kind: "allow".into(), + token, + reason: None, + redactions: None, + severity: None, + }, + CoreDecision::Block { reason, .. } => JsDecision { + kind: "block".into(), + token, + reason: Some(block_reason_to_js(reason)), + redactions: None, + severity: None, + }, + CoreDecision::Redact { redactions, .. } => JsDecision { + kind: "redact".into(), + token, + reason: None, + redactions: Some( + redactions + .replacements + .iter() + .map(|r| JsRedaction { + message_idx: r.message_idx as u32, + redacted_content: r.redacted_content.clone(), + }) + .collect(), + ), + severity: None, + }, + CoreDecision::Flag { severity, .. } => JsDecision { + kind: "flag".into(), + token, + reason: None, + redactions: None, + severity: Some(flag_severity_label(*severity).to_string()), + }, + // Decision is #[non_exhaustive] — future variants surface as + // "unknown" so the JS shim has a deterministic default. + _ => JsDecision { + kind: "unknown".into(), + token, + reason: None, + redactions: None, + severity: None, + }, + } +} + +fn block_reason_to_js(reason: &CoreBlockReason) -> JsBlockReason { + let mut out = JsBlockReason { + kind: String::new(), + artifact: None, + severity: None, + budget_kind: None, + observed: None, + limit: None, + rule_id: None, + rule_name: None, + suggested_provider: None, + suggested_model: None, + }; + match reason { + CoreBlockReason::SensitiveArtifact { artifact, severity } => { + out.kind = "sensitive_artifact".into(); + out.artifact = Some(format!("{artifact:?}")); + out.severity = Some(format!("{severity:?}")); + } + CoreBlockReason::BudgetExceeded { + budget_kind, + observed, + limit, + } => { + out.kind = "budget_exceeded".into(); + out.budget_kind = Some(format!("{budget_kind:?}")); + out.observed = Some(*observed as u32); + out.limit = Some(*limit as u32); + } + CoreBlockReason::PolicyRule { rule_id, rule_name } => { + out.kind = "policy_rule".into(); + out.rule_id = Some(rule_id.clone()); + out.rule_name = rule_name.clone(); + } + CoreBlockReason::UseAlternative { + suggested_provider, + suggested_model, + rule_id, + } => { + out.kind = "use_alternative".into(); + out.suggested_provider = suggested_provider.clone(); + out.suggested_model = suggested_model.clone(); + out.rule_id = Some(rule_id.clone()); + } + // BlockReason is #[non_exhaustive] — fall back to a generic + // shape if soth-sdk-core adds a new variant before this binding + // catches up. + _ => { + out.kind = "unknown".into(); + } + } + out +} + +fn flag_severity_label(severity: FlagSeverity) -> &'static str { + match severity { + FlagSeverity::Info => "info", + FlagSeverity::Warning => "warning", + FlagSeverity::Critical => "critical", + // FlagSeverity is #[non_exhaustive] — future variants surface + // as "unknown" so the JS shim keeps working. + _ => "unknown", + } +} From c084831640a2f5d3a829952ad6b7280667886234 Mon Sep 17 00:00:00 2001 From: Apoorv Raj Saxena Date: Thu, 30 Apr 2026 13:16:59 +0530 Subject: [PATCH 06/24] feat(sdk): Phase 1 - streaming wrapper for Python + Node bindings MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Both bindings gain a streaming surface that mirrors the Decision API spec §4.2 (`stream_begin → stream_chunk → stream_end`). Streaming was the keystone gap before auto-instrumentation: ~70% of production LLM traffic streams, so without this the SDK was silently broken on the hot path. Python (bindings/soth-py): - PyStreamObservation napi class wrapping CoreStreamObservation in Mutex> for take-on-end semantics. - New PyO3 method SothSdk.stream_begin(call) -> (decision, observation). - python/soth/__init__.py adds: async def guard_stream(iter_factory, *, call, chunk_extractor=None) Async generator; raises SothBlocked on Block decision; default chunk extractor reads OpenAI's chunk.choices[0].delta.content shape. - tests/test_streaming.py: 3 round-trip tests (consume-once, block on credential, double-end idempotent). - Type stubs updated in _soth_native.pyi. Node (bindings/soth-node): - napi class can't compose with napi(object) wrappers, so observations are kept on SothSdk keyed by token (raw u64 stringified across FFI). streamBegin returns just the JsDecision; streamChunk(token, ...) and streamEnd(token) look up by token. - index.js adds `guardStream(iterFactory, { call, chunkExtractor })` async generator with the same semantics as the Python version. Default extractor is OpenAI-shaped. - index.d.ts: GuardStreamOptions + ChunkExtractorOutput types. - __test__/streaming.test.mjs: 3 tests mirroring Python. Sentinel handling: SLAB_FULL / SENTINEL_FAIL_OPEN tokens skip the observation slab — there's nothing to stash because pre_call didn't allocate. streamEnd on a sentinel is a no-op. Verification: - cargo build --workspace clean - All workspace lib tests unchanged (654) - Conformance harness 7/7 fixtures still pass Phase-1 follow-ups still deferred: - chunk_extractor presets for anthropic/cohere/google/mistral - Auto-instrumentation hooking streamBegin/streamChunk/streamEnd - Streaming wrapper for non-AsyncIterator callbacks (Anthropic's message_stream_text style) Co-Authored-By: Claude Opus 4.7 (1M context) --- Cargo.lock | 1 + .../soth-node/__test__/streaming.test.mjs | 109 +++++++++++++++ bindings/soth-node/index.d.ts | 14 ++ bindings/soth-node/index.js | 59 ++++++++ bindings/soth-node/src/lib.rs | 84 ++++++++++- bindings/soth-py/Cargo.toml | 1 + bindings/soth-py/python/soth/__init__.py | 67 +++++++++ bindings/soth-py/python/soth/_soth_native.pyi | 13 ++ bindings/soth-py/src/lib.rs | 81 ++++++++++- bindings/soth-py/tests/test_streaming.py | 130 ++++++++++++++++++ 10 files changed, 555 insertions(+), 4 deletions(-) create mode 100644 bindings/soth-node/__test__/streaming.test.mjs create mode 100644 bindings/soth-py/tests/test_streaming.py diff --git a/Cargo.lock b/Cargo.lock index 99bd9b93..e8dd6c73 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -4349,6 +4349,7 @@ dependencies = [ "pyo3", "soth-core", "soth-sdk-core", + "tracing", "zeroize", ] diff --git a/bindings/soth-node/__test__/streaming.test.mjs b/bindings/soth-node/__test__/streaming.test.mjs new file mode 100644 index 00000000..321d598d --- /dev/null +++ b/bindings/soth-node/__test__/streaming.test.mjs @@ -0,0 +1,109 @@ +// Streaming round-trip tests for @soth/sdk. Mirrors the streaming +// integration test in `crates/soth-sdk-core/tests/round_trip.rs`. +// +// Run with: +// cd bindings/soth-node +// npm install +// npm run build:debug +// npm test + +import { test } from 'node:test'; +import { strict as assert } from 'node:assert'; + +import * as soth from '../index.js'; + +process.env.SOTH_HMAC_KEY = 'x'.repeat(32); + +soth.init({ + apiKey: 'sk-test', + orgId: 'org-test', + hmacKeyEnv: 'SOTH_HMAC_KEY', +}); + +async function* fakeOpenAIStream(deltas) { + for (let i = 0; i < deltas.length; i += 1) { + const finishReason = i === deltas.length - 1 ? 'stop' : null; + yield { + choices: [ + { + delta: { content: deltas[i] }, + finish_reason: finishReason, + }, + ], + }; + // Yield to the event loop so this looks like a real network stream. + await new Promise((r) => setImmediate(r)); + } +} + +test('stream round trip consumes token once', async () => { + const received = []; + for await (const chunk of soth.guardStream( + () => fakeOpenAIStream(['hello ', 'world', '!']), + { + call: { + provider: 'openai', + model: 'gpt-4o-mini', + messages: [{ role: 'user', content: 'say hi' }], + stream: true, + }, + }, + )) { + received.push(chunk); + } + assert.equal(received.length, 3); + const sdk = soth.getSdk(); + assert.equal(sdk.inFlightDecisions(), 0); + const events = sdk.drainTelemetryForTest(); + assert.equal(events.length, 1); + assert.equal(events[0].provider, 'openai'); +}); + +test('stream blocks on credential in user message', async () => { + await assert.rejects( + () => (async () => { + for await (const _ of soth.guardStream( + () => fakeOpenAIStream(['should ', 'not ', 'stream']), + { + call: { + provider: 'openai', + model: 'gpt-4o-mini', + messages: [ + { + role: 'user', + content: 'leaked sk-abcdefghijklmnopqrstuvwxyzABCD1234567890 here', + }, + ], + stream: true, + }, + }, + )) { + // unreachable — Block raises before iteration + } + })(), + (err) => { + assert.ok(err instanceof soth.SothBlocked); + assert.equal(err.reason.kind, 'sensitive_artifact'); + return true; + }, + ); + const sdk = soth.getSdk(); + assert.equal(sdk.inFlightDecisions(), 0); +}); + +test('stream end is idempotent (double-end safe)', async () => { + const sdk = soth.getSdk(); + const decision = sdk.streamBegin({ + provider: 'openai', + model: 'gpt-4o-mini', + messages: [{ role: 'user', content: 'hi' }], + stream: true, + }); + assert.equal(decision.kind, 'allow'); + sdk.streamChunk(decision.token, 0, 'a', null); + sdk.streamChunk(decision.token, 1, 'b', 'stop'); + sdk.streamEnd(decision.token); + // Second end is documented no-op (no exception). + sdk.streamEnd(decision.token); + assert.equal(sdk.inFlightDecisions(), 0); +}); diff --git a/bindings/soth-node/index.d.ts b/bindings/soth-node/index.d.ts index fae168ef..95bb30ad 100644 --- a/bindings/soth-node/index.d.ts +++ b/bindings/soth-node/index.d.ts @@ -59,6 +59,20 @@ export interface GuardOptions { call: LlmCall; } +export interface ChunkExtractorOutput { + deltaContent: string | null; + finishReason: string | null; +} + +export interface GuardStreamOptions { + call: LlmCall; + chunkExtractor?: (chunk: TChunk) => ChunkExtractorOutput; +} + export function init(options: InitOptions): void; export function guard(callFn: () => Promise, options: GuardOptions): Promise; +export function guardStream( + iterFactory: () => AsyncIterable | Promise>, + options: GuardStreamOptions, +): AsyncIterable; export function getSdk(): unknown; diff --git a/bindings/soth-node/index.js b/bindings/soth-node/index.js index 27436950..5b5824bf 100644 --- a/bindings/soth-node/index.js +++ b/bindings/soth-node/index.js @@ -78,9 +78,68 @@ async function guard(callFn, { call }) { return result; } +// Default chunk extractor for OpenAI-shaped chat-completion streams. +// Returns `{ deltaContent, finishReason }` extracted from the chunk's +// `choices[0]` entry. Customers using non-OpenAI shapes pass their own +// extractor to `guardStream`. +function defaultOpenAIChunkExtractor(chunk) { + try { + const choice = chunk?.choices?.[0]; + return { + deltaContent: choice?.delta?.content ?? null, + finishReason: choice?.finish_reason ?? null, + }; + } catch (_) { + return { deltaContent: null, finishReason: null }; + } +} + +/** + * Wrap a streaming LLM call with SOTH's pre/post lifecycle. + * + * `iterFactory` returns the provider's async iterable (e.g. the result + * of `client.chat.completions.create({stream: true, ...})`). The + * `chunkExtractor` (defaults to OpenAI shape) pulls + * `{deltaContent, finishReason}` from each chunk; the SDK records + * those alongside chunk count. + * + * Yields each chunk back to the caller. Throws `SothBlocked` if the + * decision is `Block`. Always finalizes the stream observation on + * normal completion or thrown exception. + */ +async function* guardStream(iterFactory, { call, chunkExtractor } = {}) { + if (!call) throw new Error('guardStream: call required'); + const extractor = chunkExtractor ?? defaultOpenAIChunkExtractor; + const sdk = getSdk(); + const decision = sdk.streamBegin(call); + const { kind, token } = decision; + + if (kind === 'block') { + sdk.streamEnd(token); + throw new SothBlocked(token, decision.reason); + } + + let sequence = 0; + try { + let provIter = iterFactory(); + if (provIter && typeof provIter.then === 'function') { + provIter = await provIter; + } + for await (const chunk of provIter) { + const { deltaContent, finishReason } = extractor(chunk); + sdk.streamChunk(token, sequence, deltaContent ?? null, finishReason ?? null); + sequence += 1; + yield chunk; + } + } finally { + sdk.streamEnd(token); + } +} + module.exports = { init, guard, + guardStream, getSdk, SothBlocked, SothFlagged, diff --git a/bindings/soth-node/src/lib.rs b/bindings/soth-node/src/lib.rs index ea52bba2..c704092c 100644 --- a/bindings/soth-node/src/lib.rs +++ b/bindings/soth-node/src/lib.rs @@ -9,13 +9,15 @@ #![deny(clippy::all)] -use std::sync::Arc; +use std::collections::HashMap; +use std::sync::{Arc, Mutex}; use napi::bindgen_prelude::*; use napi_derive::napi; use soth_sdk_core::{ BlockReason as CoreBlockReason, Decision as CoreDecision, DecisionToken, FlagSeverity, - HmacKey, LlmCall, LlmResponse, Message, SdkConfigBuilder, SothSdk as CoreSothSdk, Tool, + HmacKey, LlmCall, LlmChunk, LlmResponse, Message, SdkConfigBuilder, SothSdk as CoreSothSdk, + StreamObservation as CoreStreamObservation, Tool, }; use soth_core::EndpointType; use zeroize::Zeroizing; @@ -93,6 +95,11 @@ pub struct JsTelemetryEvent { #[napi] pub struct SothSdk { inner: Arc, + /// In-flight stream observations indexed by their `DecisionToken`'s + /// raw u64 (stringified across the FFI boundary). The JS shim's + /// `guardStream` looks observations up by token rather than holding + /// a napi class reference, which keeps the FFI boundary scalar-only. + streams: Arc>>, } #[napi] @@ -131,6 +138,7 @@ impl SothSdk { Ok(Self { inner: Arc::new(sdk), + streams: Arc::new(Mutex::new(HashMap::new())), }) } @@ -158,6 +166,72 @@ impl SothSdk { Ok(()) } + /// Streaming counterpart to `pre_call`. Returns the decision; the + /// JS shim feeds chunks via `streamChunk(token, ...)` and finalizes + /// with `streamEnd(token)`. The observation lives inside the SDK + /// keyed by token, so the FFI boundary stays scalar-only. + #[napi] + pub fn stream_begin(&self, call: JsLlmCall) -> Result { + let llm_call = build_llm_call(call); + let (decision, observation) = self.inner.stream_begin(&llm_call); + let token_raw = decision.token().raw(); + // Sentinel tokens (SLAB_FULL / SENTINEL_FAIL_OPEN) skip slab + // bookkeeping — there's no observation to stash because pre_call + // itself didn't allocate one. Phase-1 telemetry records this. + if !is_sentinel_raw(token_raw) { + let mut guard = self.streams.lock().map_err(|_| { + Error::new(Status::GenericFailure, "stream slot lock poisoned") + })?; + guard.insert(token_raw, observation); + } + Ok(decision_to_js(&decision)) + } + + /// Feed a delta chunk. Returns silently if the token is unknown + /// (treated as a binding bug — same semantic as the slab's + /// stale-token handling). + #[napi] + pub fn stream_chunk( + &self, + token: String, + sequence: u32, + delta_content: Option, + finish_reason: Option, + ) -> Result<()> { + let raw: u64 = token.parse().map_err(|_| { + Error::new(Status::InvalidArg, "stream token must be a numeric string") + })?; + let mut guard = self + .streams + .lock() + .map_err(|_| Error::new(Status::GenericFailure, "stream slot lock poisoned"))?; + let Some(obs) = guard.get_mut(&raw) else { + return Ok(()); + }; + let mut chunk = LlmChunk::new(sequence); + chunk.delta_content = delta_content; + chunk.finish_reason = finish_reason; + self.inner.stream_chunk(obs, &chunk); + Ok(()) + } + + /// Finalize the stream. Idempotent — second call is a no-op. + #[napi] + pub fn stream_end(&self, token: String) -> Result<()> { + let raw: u64 = token.parse().map_err(|_| { + Error::new(Status::InvalidArg, "stream token must be a numeric string") + })?; + let mut guard = self + .streams + .lock() + .map_err(|_| Error::new(Status::GenericFailure, "stream slot lock poisoned"))?; + if let Some(obs) = guard.remove(&raw) { + drop(guard); + self.inner.stream_end(obs); + } + Ok(()) + } + /// Test helper — number of in-flight decisions. #[napi] pub fn in_flight_decisions(&self) -> u32 { @@ -182,6 +256,12 @@ impl SothSdk { } } +// ── helpers ────────────────────────────────────────────────────────── + +fn is_sentinel_raw(raw: u64) -> bool { + raw == DecisionToken::SLAB_FULL.raw() || raw == DecisionToken::SENTINEL_FAIL_OPEN.raw() +} + // ── conversions ────────────────────────────────────────────────────── fn build_llm_call(call: JsLlmCall) -> LlmCall { diff --git a/bindings/soth-py/Cargo.toml b/bindings/soth-py/Cargo.toml index 0411357a..a677f67c 100644 --- a/bindings/soth-py/Cargo.toml +++ b/bindings/soth-py/Cargo.toml @@ -28,4 +28,5 @@ extension-module = ["pyo3/extension-module"] soth-sdk-core = { path = "../../crates/soth-sdk-core" } soth-core = { workspace = true } zeroize = { workspace = true } +tracing = { workspace = true } pyo3 = { version = "0.22", features = ["abi3-py310"] } diff --git a/bindings/soth-py/python/soth/__init__.py b/bindings/soth-py/python/soth/__init__.py index b8bde680..b957fe17 100644 --- a/bindings/soth-py/python/soth/__init__.py +++ b/bindings/soth-py/python/soth/__init__.py @@ -52,6 +52,7 @@ __all__ = [ "init", "guard", + "guard_stream", "SothBlocked", "SothFlagged", "BlockReason", @@ -149,6 +150,72 @@ def guard( return result +async def guard_stream( + iter_factory: Callable[[], Any], + *, + call: dict[str, Any], + chunk_extractor: Callable[[Any], tuple[Optional[str], Optional[str]]] | None = None, +): + """Wrap a streaming LLM call with SOTH's pre/post lifecycle. + + `iter_factory` returns an async iterator (typically the awaited + result of e.g. ``client.chat.completions.create(stream=True, ...)``). + `chunk_extractor(chunk) -> (delta_content, finish_reason)` pulls the + fields the SDK records from each provider chunk; defaults to OpenAI's + `chunk.choices[0].delta.content` shape. + + Yields each chunk back to the caller. Raises `SothBlocked` if the + decision is `Block`. Always finalizes the stream observation on + completion or exception. + """ + sdk = get_sdk() + decision, observation = sdk.stream_begin(call) + kind = decision["kind"] + + if kind == _soth_native.DECISION_KIND_BLOCK: + observation.end() # Consume token even on block. + raise SothBlocked( + decision_id=str(decision["token"]), + reason=block_reason_from_dict(decision.get("reason", {})), + ) + + if chunk_extractor is None: + chunk_extractor = _default_openai_chunk_extractor + + sequence = 0 + try: + provider_iter = iter_factory() + # Provider may return a sync iterator (e.g. anthropic non-async) + # OR a coroutine that resolves to an async iterator. Handle both. + if hasattr(provider_iter, "__await__"): + provider_iter = await provider_iter + async for chunk in provider_iter: + delta_content, finish_reason = chunk_extractor(chunk) + observation.chunk(sequence, delta_content, finish_reason) + sequence += 1 + yield chunk + finally: + observation.end() + + +def _default_openai_chunk_extractor(chunk: Any) -> tuple[Optional[str], Optional[str]]: + """Default chunk extractor for OpenAI-shaped streams. + + Looks for `chunk.choices[0].delta.content` and + `chunk.choices[0].finish_reason`. Falls back to `(None, None)` for + chunks that don't fit (the SDK still sees the chunk count, just no + content sample). + """ + try: + choice = chunk.choices[0] + delta = getattr(choice, "delta", None) + delta_content = getattr(delta, "content", None) if delta else None + finish_reason = getattr(choice, "finish_reason", None) + return delta_content, finish_reason + except (AttributeError, IndexError, TypeError): + return None, None + + # Test-only re-exports (used by `tests/test_smoke.py` etc.) def _drain_telemetry_for_test() -> list[dict[str, Any]]: return get_sdk().drain_telemetry_for_test() diff --git a/bindings/soth-py/python/soth/_soth_native.pyi b/bindings/soth-py/python/soth/_soth_native.pyi index 17e716a4..dacfff74 100644 --- a/bindings/soth-py/python/soth/_soth_native.pyi +++ b/bindings/soth-py/python/soth/_soth_native.pyi @@ -15,6 +15,16 @@ DECISION_KIND_REDACT: str DECISION_KIND_FLAG: str +class StreamObservation: + def chunk( + self, + sequence: int, + delta_content: Optional[str] = None, + finish_reason: Optional[str] = None, + ) -> None: ... + def end(self) -> None: ... + + class SothSdk: def __init__( self, @@ -27,5 +37,8 @@ class SothSdk: def pre_call(self, call: dict[str, Any]) -> dict[str, Any]: ... def post_call(self, token: int, response: Optional[dict[str, Any]] = None) -> None: ... + def stream_begin( + self, call: dict[str, Any] + ) -> tuple[dict[str, Any], StreamObservation]: ... def in_flight_decisions(self) -> int: ... def drain_telemetry_for_test(self) -> list[dict[str, Any]]: ... diff --git a/bindings/soth-py/src/lib.rs b/bindings/soth-py/src/lib.rs index 56af6d89..65956f52 100644 --- a/bindings/soth-py/src/lib.rs +++ b/bindings/soth-py/src/lib.rs @@ -9,14 +9,15 @@ //! partly here (the FFI boundary) and partly in `python/soth/exceptions.py` //! (the `SothBlocked` exception + propagation tests). -use std::sync::Arc; +use std::sync::{Arc, Mutex}; use pyo3::exceptions::{PyRuntimeError, PyValueError}; use pyo3::prelude::*; use pyo3::types::{PyDict, PyList}; use soth_sdk_core::{ BlockReason as CoreBlockReason, Decision as CoreDecision, DecisionToken, FlagSeverity, - HmacKey, LlmCall, LlmResponse, Message, SdkConfigBuilder, SothSdk as CoreSothSdk, Tool, + HmacKey, LlmCall, LlmChunk, LlmResponse, Message, SdkConfigBuilder, SothSdk as CoreSothSdk, + StreamObservation as CoreStreamObservation, Tool, }; use soth_core::EndpointType; use zeroize::Zeroizing; @@ -103,6 +104,27 @@ impl PySothSdk { self.inner.in_flight_decisions() } + /// Streaming counterpart to `pre_call`. Returns + /// `(decision_dict, stream_observation)`. The host wrapper iterates + /// the provider's stream and feeds chunks via `observation.chunk(...)`, + /// then calls `observation.end()` on terminal chunk to consume the + /// token and emit telemetry. + #[pyo3(signature = (call_dict))] + fn stream_begin<'py>( + &self, + py: Python<'py>, + call_dict: &Bound<'py, PyDict>, + ) -> PyResult<(Bound<'py, PyDict>, PyStreamObservation)> { + let call = build_llm_call(call_dict)?; + let (decision, observation) = self.inner.stream_begin(&call); + let decision_dict = decision_to_pydict(py, &decision)?; + let py_obs = PyStreamObservation { + sdk: Arc::clone(&self.inner), + inner: Arc::new(Mutex::new(Some(observation))), + }; + Ok((decision_dict, py_obs)) + } + /// Drain the in-memory telemetry queue. Test-only — production /// shippers will pull batches via the Phase-1 transport API. fn drain_telemetry_for_test<'py>( @@ -130,6 +152,60 @@ impl PySothSdk { } } +/// Wraps a `StreamObservation` so Python can call `chunk` / `end` from +/// inside an `async for` loop. Bindings hold the observation in a +/// Mutex-Option so `end()` can take ownership exactly once; double-end +/// is silently a no-op (logged via tracing) — same semantic as the +/// slab's stale-token handling. +#[pyclass(name = "StreamObservation", module = "soth._soth_native")] +struct PyStreamObservation { + sdk: Arc, + inner: Arc>>, +} + +#[pymethods] +impl PyStreamObservation { + /// Feed a single delta chunk. Cheap; no allocation beyond + /// accumulating the content into the underlying observation. + #[pyo3(signature = (sequence, delta_content=None, finish_reason=None))] + fn chunk( + &self, + sequence: u32, + delta_content: Option, + finish_reason: Option, + ) -> PyResult<()> { + let mut guard = self + .inner + .lock() + .map_err(|_| PyRuntimeError::new_err("stream observation lock poisoned"))?; + let Some(obs) = guard.as_mut() else { + // Double-chunk after end is benign — log via tracing then + // return Ok so Python iteration doesn't break. + tracing::warn!("stream_chunk called after stream_end (binding bug)"); + return Ok(()); + }; + let mut llm_chunk = LlmChunk::new(sequence); + llm_chunk.delta_content = delta_content; + llm_chunk.finish_reason = finish_reason; + self.sdk.stream_chunk(obs, &llm_chunk); + Ok(()) + } + + /// Finalize the stream — consumes the DecisionToken, runs classify + /// enrichment, emits the telemetry event. Idempotent: subsequent + /// calls are no-ops with a logged warning. + fn end(&self) -> PyResult<()> { + let mut guard = self + .inner + .lock() + .map_err(|_| PyRuntimeError::new_err("stream observation lock poisoned"))?; + if let Some(obs) = guard.take() { + self.sdk.stream_end(obs); + } + Ok(()) + } +} + /// Decision API constants exposed at the module level so the Python /// wrapper can reference them without a string match. const DECISION_KIND_ALLOW: &str = "allow"; @@ -330,6 +406,7 @@ fn response_from_pydict(_dict: Option<&Bound<'_, PyDict>>) -> PyResult, m: &Bound<'_, PyModule>) -> PyResult<()> { m.add_class::()?; + m.add_class::()?; m.add("__version__", env!("CARGO_PKG_VERSION"))?; m.add("DECISION_KIND_ALLOW", DECISION_KIND_ALLOW)?; m.add("DECISION_KIND_BLOCK", DECISION_KIND_BLOCK)?; diff --git a/bindings/soth-py/tests/test_streaming.py b/bindings/soth-py/tests/test_streaming.py new file mode 100644 index 00000000..eb3a3b9c --- /dev/null +++ b/bindings/soth-py/tests/test_streaming.py @@ -0,0 +1,130 @@ +"""Streaming round-trip tests for soth-py. + +Mirrors the streaming smoke test in `crates/soth-sdk-core/tests/round_trip.rs`, +asserting the FFI streaming surface (`stream_begin` + chunk/end) doesn't +introduce drift. + +Run with: + cd bindings/soth-py + maturin develop + pytest tests/test_streaming.py +""" + +import asyncio +import os +from typing import Any + +import pytest + +import soth + + +class FakeChoice: + def __init__(self, content: str, finish_reason: str | None = None): + self.delta = type("Delta", (), {"content": content})() + self.finish_reason = finish_reason + + +class FakeChunk: + def __init__(self, content: str, finish_reason: str | None = None): + self.choices = [FakeChoice(content, finish_reason)] + + +async def fake_openai_stream(deltas: list[str]): + """Mimics openai's async stream — yields chunks shaped like the + real ChatCompletionChunk objects. Last chunk carries finish_reason.""" + for i, delta in enumerate(deltas): + finish = "stop" if i == len(deltas) - 1 else None + yield FakeChunk(delta, finish) + # Yield to the event loop so this looks like a real network stream. + await asyncio.sleep(0) + + +@pytest.fixture(autouse=True) +def _init_sdk(): + os.environ["SOTH_HMAC_KEY"] = "x" * 32 + soth.init( + api_key="sk-test", + org_id="org-test", + hmac_key_env="SOTH_HMAC_KEY", + ) + yield + + +@pytest.mark.asyncio +async def test_stream_round_trip_consumes_token_once(): + deltas = ["hello ", "world", "!"] + received: list[Any] = [] + + async for chunk in soth.guard_stream( + lambda: fake_openai_stream(deltas), + call={ + "provider": "openai", + "model": "gpt-4o-mini", + "messages": [{"role": "user", "content": "say hi"}], + "stream": True, + }, + ): + received.append(chunk) + + assert len(received) == 3 + assert soth._in_flight_decisions() == 0 + events = soth._drain_telemetry_for_test() + assert len(events) == 1 + assert events[0]["provider"] == "openai" + + +@pytest.mark.asyncio +async def test_stream_blocks_on_credential_in_user_message(): + async def _consume(): + async for _ in soth.guard_stream( + lambda: fake_openai_stream(["should ", "not ", "stream"]), + call={ + "provider": "openai", + "model": "gpt-4o-mini", + "messages": [ + { + "role": "user", + "content": ( + "review key sk-abcdefghijklmnopqrstuvwxyzABCD1234567890" + ), + } + ], + "stream": True, + }, + ): + pass + + with pytest.raises(soth.SothBlocked): + await _consume() + assert soth._in_flight_decisions() == 0 + + +@pytest.mark.asyncio +async def test_stream_observation_double_end_is_idempotent(): + deltas = ["a", "b"] + sdk = soth.get_sdk() + decision, obs = sdk.stream_begin( + { + "provider": "openai", + "model": "gpt-4o-mini", + "messages": [{"role": "user", "content": "hi"}], + "stream": True, + } + ) + assert decision["kind"] == "allow" + obs.chunk(0, "a", None) + obs.chunk(1, "b", "stop") + obs.end() + # Second end is a documented no-op (logged warning, no exception). + obs.end() + assert sdk.in_flight_decisions() == 0 + + +# pytest-asyncio configuration — keeps this self-contained so the suite +# doesn't require a project-level conftest. +def pytest_collection_modifyitems(config, items): + # No-op; pytest-asyncio's `asyncio_mode = "auto"` would normally be + # set in pyproject.toml. The `@pytest.mark.asyncio` decorator is + # explicit here to make the dependency visible. + pass From 76f336b4c9f1831bf276f7a55e9989d005ea485a Mon Sep 17 00:00:00 2001 From: Apoorv Raj Saxena Date: Thu, 30 Apr 2026 13:20:29 +0530 Subject: [PATCH 07/24] feat(sdk): Phase 1 - background HTTPS telemetry shipper MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Connects the SDK's in-memory telemetry queue to soth-cloud over HTTPS. Without this commit, post_call emitted events into a queue nobody read; now configured deployments actually deliver telemetry to the dashboard. soth-sdk-core: - New `http-telemetry` feature (off by default to keep workspace builds fast). When on, pulls reqwest with rustls-tls. - New module crates/soth-sdk-core/src/shipper.rs: TelemetryShipper::spawn(queue, endpoint, api_key, org_id) spawns a `soth-telemetry-shipper` thread that drains MAX_BATCH_SIZE events every BATCH_WINDOW (5s), POSTs to the configured endpoint with bearer auth, and Drop's cleanly. Final drain on shutdown ensures events buffered during the last window are flushed. - TelemetryQueue::drain_batch(max) — bounded pull for the shipper. - SothSdk holds a Mutex>; init spawns it iff `telemetry_endpoint` is set, drop-on-shutdown is idempotent. - SothSdk::shutdown() — public method bindings call at process exit. V0 limits (Phase-2 follow-ups documented in shipper.rs): - No retry / circuit-breaker (failed batches drop) - No exp backoff - No dead-letter queue - No batch compression / signing / encryption Bindings: - bindings/soth-py and soth-node both enable http-telemetry by default (production native bindings always want a working shipper). - Both expose `telemetry_endpoint` parameter on init. - Both expose `shutdown()` method for graceful exit. - Python: soth.init(..., telemetry_endpoint="https://...") and soth.shutdown(). - Node: soth.init({ telemetryEndpoint: "https://..." }) and soth.shutdown(). Verification: - cargo build -p soth-sdk-core (default, http-telemetry off): clean - cargo build -p soth-sdk-core --features http-telemetry: clean - cargo build --workspace clean (bindings pull http-telemetry on) - soth-sdk-core 15 unit + 5 integration tests unchanged - Conformance harness 7/7 fixtures unchanged - The for_test() ctor never spawns a shipper so CI stays hermetic Co-Authored-By: Claude Opus 4.7 (1M context) --- Cargo.lock | 1 + bindings/soth-node/Cargo.toml | 5 +- bindings/soth-node/index.js | 15 +- bindings/soth-node/src/lib.rs | 24 +++- bindings/soth-py/Cargo.toml | 5 +- bindings/soth-py/python/soth/__init__.py | 20 +++ bindings/soth-py/src/lib.rs | 18 ++- crates/soth-sdk-core/Cargo.toml | 4 + crates/soth-sdk-core/src/lib.rs | 3 + crates/soth-sdk-core/src/sdk.rs | 44 +++++- crates/soth-sdk-core/src/shipper.rs | 151 ++++++++++++++++++++ crates/soth-sdk-core/src/telemetry_queue.rs | 12 ++ 12 files changed, 292 insertions(+), 10 deletions(-) create mode 100644 crates/soth-sdk-core/src/shipper.rs diff --git a/Cargo.lock b/Cargo.lock index e8dd6c73..b9dc5fa9 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -4359,6 +4359,7 @@ version = "0.1.0" dependencies = [ "arc-swap", "opentelemetry", + "reqwest", "serde", "serde_json", "soth-classify", diff --git a/bindings/soth-node/Cargo.toml b/bindings/soth-node/Cargo.toml index ac79e7ee..e1e36dd0 100644 --- a/bindings/soth-node/Cargo.toml +++ b/bindings/soth-node/Cargo.toml @@ -12,7 +12,10 @@ publish = false crate-type = ["cdylib"] [dependencies] -soth-sdk-core = { path = "../../crates/soth-sdk-core" } +# `http-telemetry` enables the background shipper; native Node +# bindings always ship with it on so customers' configured +# `telemetry_endpoint` actually reaches soth-cloud. +soth-sdk-core = { path = "../../crates/soth-sdk-core", features = ["http-telemetry"] } soth-core = { workspace = true } zeroize = { workspace = true } napi = { version = "2", default-features = false, features = ["napi8", "serde-json"] } diff --git a/bindings/soth-node/index.js b/bindings/soth-node/index.js index 5b5824bf..b590d694 100644 --- a/bindings/soth-node/index.js +++ b/bindings/soth-node/index.js @@ -34,7 +34,7 @@ class SothFlagged { let _singleton = null; -function init({ apiKey, orgId, hmacKeyEnv, hmacKeyStatic }) { +function init({ apiKey, orgId, hmacKeyEnv, hmacKeyStatic, telemetryEndpoint }) { if (!apiKey) throw new Error('init: apiKey required'); if (!orgId) throw new Error('init: orgId required'); _singleton = native.SothSdk.create( @@ -42,9 +42,21 @@ function init({ apiKey, orgId, hmacKeyEnv, hmacKeyStatic }) { orgId, hmacKeyEnv ?? null, hmacKeyStatic ?? null, + telemetryEndpoint ?? null, ); } +/** + * Stop the background telemetry shipper and flush pending events. + * Customers SHOULD call this at process exit (e.g. on SIGINT / SIGTERM) + * so the last batch window's events aren't lost. Idempotent. + */ +function shutdown() { + if (_singleton) { + _singleton.shutdown(); + } +} + function getSdk() { if (!_singleton) { throw new Error('soth.init({...}) must be called before any guard() / SDK call'); @@ -138,6 +150,7 @@ async function* guardStream(iterFactory, { call, chunkExtractor } = {}) { module.exports = { init, + shutdown, guard, guardStream, getSdk, diff --git a/bindings/soth-node/src/lib.rs b/bindings/soth-node/src/lib.rs index c704092c..9246a619 100644 --- a/bindings/soth-node/src/lib.rs +++ b/bindings/soth-node/src/lib.rs @@ -108,7 +108,13 @@ impl SothSdk { /// minimum-required field set; richer config (capture_mode, /// classification_mode, etc.) lands in a follow-up commit. #[napi(factory)] - pub fn create(api_key: String, org_id: String, hmac_key_env: Option, hmac_key_static: Option) -> Result { + pub fn create( + api_key: String, + org_id: String, + hmac_key_env: Option, + hmac_key_static: Option, + telemetry_endpoint: Option, + ) -> Result { let hmac_key = match (hmac_key_env, hmac_key_static) { (Some(env), None) => HmacKey::FromEnv(env), (None, Some(buf)) => HmacKey::Static(Zeroizing::new(buf.as_ref().to_vec())), @@ -126,10 +132,14 @@ impl SothSdk { } }; - let config = SdkConfigBuilder::new() + let mut builder = SdkConfigBuilder::new() .api_key(api_key) .org_id(org_id) - .hmac_key(hmac_key) + .hmac_key(hmac_key); + if let Some(endpoint) = telemetry_endpoint { + builder = builder.telemetry_endpoint(endpoint); + } + let config = builder .build() .map_err(|e| Error::new(Status::InvalidArg, format!("{e}")))?; @@ -232,6 +242,14 @@ impl SothSdk { Ok(()) } + /// Stop the background HTTPS telemetry shipper (if configured) and + /// flush pending events. Customers SHOULD call this at process exit + /// so the last batch window's events aren't lost. Idempotent. + #[napi] + pub fn shutdown(&self) { + self.inner.shutdown(); + } + /// Test helper — number of in-flight decisions. #[napi] pub fn in_flight_decisions(&self) -> u32 { diff --git a/bindings/soth-py/Cargo.toml b/bindings/soth-py/Cargo.toml index a677f67c..ef84076e 100644 --- a/bindings/soth-py/Cargo.toml +++ b/bindings/soth-py/Cargo.toml @@ -25,7 +25,10 @@ default = [] extension-module = ["pyo3/extension-module"] [dependencies] -soth-sdk-core = { path = "../../crates/soth-sdk-core" } +# `http-telemetry` enables the background shipper; native Python +# bindings always ship with it on so customers' configured +# `telemetry_endpoint` actually reaches soth-cloud. +soth-sdk-core = { path = "../../crates/soth-sdk-core", features = ["http-telemetry"] } soth-core = { workspace = true } zeroize = { workspace = true } tracing = { workspace = true } diff --git a/bindings/soth-py/python/soth/__init__.py b/bindings/soth-py/python/soth/__init__.py index b957fe17..ce34b230 100644 --- a/bindings/soth-py/python/soth/__init__.py +++ b/bindings/soth-py/python/soth/__init__.py @@ -51,6 +51,7 @@ __all__ = [ "init", + "shutdown", "guard", "guard_stream", "SothBlocked", @@ -71,12 +72,18 @@ def init( org_id: str, hmac_key_env: Optional[str] = None, hmac_key_static: Optional[bytes] = None, + telemetry_endpoint: Optional[str] = None, ) -> None: """Initialize the SOTH SDK module-level singleton. Specify exactly one of `hmac_key_env` (read from environment) or `hmac_key_static` (raw bytes). Production usage SHOULD prefer `hmac_key_env` so the key never sits in source-controlled config. + + `telemetry_endpoint` (e.g. + `"https://api.soth.cloud/v1/edge/telemetry/batch"`) enables the + background HTTPS shipper. When omitted, telemetry events accumulate + in an in-memory queue with no transport — useful for tests. """ global _singleton _singleton = _soth_native.SothSdk( @@ -84,9 +91,22 @@ def init( org_id=org_id, hmac_key_env=hmac_key_env, hmac_key_static=hmac_key_static, + telemetry_endpoint=telemetry_endpoint, ) +def shutdown() -> None: + """Stop the background telemetry shipper and flush pending events. + + Customers SHOULD call this at process exit (e.g. in a `finally` + block at the top of `main`) so the last batch window's events + aren't lost. Idempotent. + """ + global _singleton + if _singleton is not None: + _singleton.shutdown() + + def get_sdk() -> _soth_native.SothSdk: """Return the initialized SDK or raise if `init` hasn't run.""" if _singleton is None: diff --git a/bindings/soth-py/src/lib.rs b/bindings/soth-py/src/lib.rs index 65956f52..771ebb27 100644 --- a/bindings/soth-py/src/lib.rs +++ b/bindings/soth-py/src/lib.rs @@ -38,12 +38,13 @@ impl PySothSdk { /// these and fall back to a no-op SDK rather than crashing the /// host process. #[new] - #[pyo3(signature = (api_key, org_id, hmac_key_env=None, hmac_key_static=None))] + #[pyo3(signature = (api_key, org_id, hmac_key_env=None, hmac_key_static=None, telemetry_endpoint=None))] fn new( api_key: String, org_id: String, hmac_key_env: Option, hmac_key_static: Option>, + telemetry_endpoint: Option, ) -> PyResult { let hmac_key = match (hmac_key_env, hmac_key_static) { (Some(env), None) => HmacKey::FromEnv(env), @@ -60,10 +61,14 @@ impl PySothSdk { } }; - let config = SdkConfigBuilder::new() + let mut builder = SdkConfigBuilder::new() .api_key(api_key) .org_id(org_id) - .hmac_key(hmac_key) + .hmac_key(hmac_key); + if let Some(endpoint) = telemetry_endpoint { + builder = builder.telemetry_endpoint(endpoint); + } + let config = builder .build() .map_err(|error| PyValueError::new_err(format!("{error}")))?; @@ -125,6 +130,13 @@ impl PySothSdk { Ok((decision_dict, py_obs)) } + /// Stop the background HTTPS telemetry shipper (if configured) and + /// flush pending events. Customers SHOULD call this at process exit + /// so the last batch window's events aren't lost. Idempotent. + fn shutdown(&self) { + self.inner.shutdown(); + } + /// Drain the in-memory telemetry queue. Test-only — production /// shippers will pull batches via the Phase-1 transport API. fn drain_telemetry_for_test<'py>( diff --git a/crates/soth-sdk-core/Cargo.toml b/crates/soth-sdk-core/Cargo.toml index aae3df05..185021d1 100644 --- a/crates/soth-sdk-core/Cargo.toml +++ b/crates/soth-sdk-core/Cargo.toml @@ -17,6 +17,9 @@ onnx-models = ["soth-classify/onnx-models"] # OpenTelemetry span emission. Off by default — bindings opt in when the # host already has an OTel pipeline. otel = ["dep:opentelemetry", "dep:tracing-opentelemetry"] +# Background HTTPS telemetry shipper. Off by default to keep workspace +# `cargo build` fast; bindings flip it on for production builds. +http-telemetry = ["dep:reqwest"] [dependencies] soth-core = { workspace = true } @@ -36,5 +39,6 @@ tracing = { workspace = true } uuid = { workspace = true } zeroize = { workspace = true } arc-swap = "1" +reqwest = { version = "0.12", default-features = false, features = ["blocking", "json", "rustls-tls"], optional = true } opentelemetry = { workspace = true, optional = true } tracing-opentelemetry = { workspace = true, optional = true } diff --git a/crates/soth-sdk-core/src/lib.rs b/crates/soth-sdk-core/src/lib.rs index 4c00f501..3d84dd37 100644 --- a/crates/soth-sdk-core/src/lib.rs +++ b/crates/soth-sdk-core/src/lib.rs @@ -24,6 +24,9 @@ mod sdk; mod slab; mod telemetry_queue; +#[cfg(feature = "http-telemetry")] +mod shipper; + pub use call::{LlmCall, LlmChunk, LlmResponse, Message, Tool}; pub use config::{ BundleSource, ClassificationMode, HmacKey, SdkConfig, SdkConfigBuilder, StorageMode, diff --git a/crates/soth-sdk-core/src/sdk.rs b/crates/soth-sdk-core/src/sdk.rs index 0e788641..dc44ab5c 100644 --- a/crates/soth-sdk-core/src/sdk.rs +++ b/crates/soth-sdk-core/src/sdk.rs @@ -73,6 +73,13 @@ pub struct SothSdk { classify_config: soth_classify::ClassifyConfig, slab: Arc, telemetry: Arc, + /// Background HTTPS shipper (when `http-telemetry` feature is on + /// AND `telemetry_endpoint` is configured). Held in a Mutex so + /// `shutdown` can take ownership and stop the thread cleanly; the + /// field is read indirectly via that path. + #[cfg(feature = "http-telemetry")] + #[allow(dead_code)] + shipper: std::sync::Mutex>, } // `SothSdk` must be `Send + Sync` for bindings to share it across host @@ -120,6 +127,19 @@ impl SothSdk { let classify_config = soth_classify::ClassifyConfig::default(); + let telemetry = Arc::new(TelemetryQueue::new()); + #[cfg(feature = "http-telemetry")] + let shipper = if let Some(endpoint) = config.telemetry_endpoint.clone() { + Some(crate::shipper::TelemetryShipper::spawn( + Arc::clone(&telemetry), + endpoint, + config.api_key.clone(), + config.org_id.clone(), + )) + } else { + None + }; + Ok(Self { config, detect_registry, @@ -127,7 +147,9 @@ impl SothSdk { classify_bundle: ArcSwap::from_pointee((*classify_bundle).clone()), classify_config, slab: Arc::new(DecisionSlab::new()), - telemetry: Arc::new(TelemetryQueue::new()), + telemetry, + #[cfg(feature = "http-telemetry")] + shipper: std::sync::Mutex::new(shipper), }) } @@ -151,6 +173,10 @@ impl SothSdk { classify_config: soth_classify::ClassifyConfig::default(), slab: Arc::new(DecisionSlab::new()), telemetry: Arc::new(TelemetryQueue::new()), + // Test ctor never spawns a shipper — fixtures don't have + // a real cloud endpoint and we want CI hermetic. + #[cfg(feature = "http-telemetry")] + shipper: std::sync::Mutex::new(None), }) } @@ -324,6 +350,22 @@ impl SothSdk { } } + /// Stop the background telemetry shipper (if any) and flush + /// pending events. Bindings call this at process exit so events + /// buffered in the last batch window aren't lost. Idempotent. + pub fn shutdown(&self) { + #[cfg(feature = "http-telemetry")] + { + let mut guard = match self.shipper.lock() { + Ok(g) => g, + Err(p) => p.into_inner(), + }; + if let Some(mut shipper) = guard.take() { + shipper.shutdown(); + } + } + } + fn emit_orphan_or_full(&self, token: DecisionToken) { if token == DecisionToken::SLAB_FULL { tracing::warn!( diff --git a/crates/soth-sdk-core/src/shipper.rs b/crates/soth-sdk-core/src/shipper.rs new file mode 100644 index 00000000..dcba3140 --- /dev/null +++ b/crates/soth-sdk-core/src/shipper.rs @@ -0,0 +1,151 @@ +//! Background telemetry shipper. +//! +//! Drains the in-memory `TelemetryQueue` on a fixed cadence and POSTs +//! batches to the configured cloud endpoint via reqwest blocking. The +//! shipper is gated behind the `http-telemetry` feature so default +//! builds stay light; bindings flip it on for production. +//! +//! V0 deliberately ships a minimal shipper: bounded batches, fixed +//! window, no retry/circuit-breaker. Phase 2 ports the full retry +//! semantics from `soth-sync` (5 attempts with exp backoff, 72-hour +//! dead-letter, circuit breaker after 5 consecutive failures). + +#![cfg(feature = "http-telemetry")] + +use std::sync::atomic::{AtomicBool, Ordering}; +use std::sync::Arc; +use std::thread::JoinHandle; +use std::time::Duration; + +use crate::telemetry_queue::TelemetryQueue; + +const BATCH_WINDOW: Duration = Duration::from_secs(5); +const MAX_BATCH_SIZE: usize = 100; +const HTTP_TIMEOUT: Duration = Duration::from_secs(10); + +pub(crate) struct TelemetryShipper { + handle: Option>, + shutdown: Arc, +} + +impl TelemetryShipper { + pub fn spawn( + queue: Arc, + endpoint: String, + api_key: String, + org_id: String, + ) -> Self { + let shutdown = Arc::new(AtomicBool::new(false)); + let shutdown_clone = Arc::clone(&shutdown); + + let handle = std::thread::Builder::new() + .name("soth-telemetry-shipper".into()) + .spawn(move || run_loop(queue, endpoint, api_key, org_id, shutdown_clone)) + .expect("spawn telemetry shipper thread"); + + Self { + handle: Some(handle), + shutdown, + } + } + + pub fn shutdown(&mut self) { + if !self.shutdown.swap(true, Ordering::Release) { + if let Some(handle) = self.handle.take() { + // Best-effort join; ignore poisoned panics. + let _ = handle.join(); + } + } + } +} + +impl Drop for TelemetryShipper { + fn drop(&mut self) { + self.shutdown(); + } +} + +fn run_loop( + queue: Arc, + endpoint: String, + api_key: String, + org_id: String, + shutdown: Arc, +) { + let client = match reqwest::blocking::Client::builder() + .timeout(HTTP_TIMEOUT) + .user_agent(format!("soth-sdk-core/{}", env!("CARGO_PKG_VERSION"))) + .build() + { + Ok(c) => c, + Err(e) => { + tracing::error!(error = %e, "telemetry shipper failed to build HTTP client; events will accumulate"); + return; + } + }; + + while !shutdown.load(Ordering::Acquire) { + // Polling loop with shutdown-aware wait. We wake every 100ms + // to check the shutdown flag so process exit doesn't stall on + // the full BATCH_WINDOW. Phase 2 will use a condvar. + let deadline = std::time::Instant::now() + BATCH_WINDOW; + while std::time::Instant::now() < deadline { + if shutdown.load(Ordering::Acquire) { + break; + } + std::thread::sleep(Duration::from_millis(100)); + } + + let batch = queue.drain_batch(MAX_BATCH_SIZE); + if batch.is_empty() { + continue; + } + + // Wire envelope is intentionally minimal; cloud-side ingestion + // accepts the same shape soth-sync uses for the proxy. + let body = serde_json::json!({ + "org_id": &org_id, + "events": batch, + }); + + match client + .post(&endpoint) + .bearer_auth(&api_key) + .json(&body) + .send() + { + Ok(resp) if resp.status().is_success() => { + tracing::debug!( + target: "soth_sdk_core::shipper", + status = resp.status().as_u16(), + "telemetry batch posted" + ); + } + Ok(resp) => { + tracing::warn!( + target: "soth_sdk_core::shipper", + status = resp.status().as_u16(), + "telemetry POST returned non-success; events dropped (Phase-2 retry not yet wired)" + ); + } + Err(error) => { + tracing::warn!( + target: "soth_sdk_core::shipper", + error = %error, + "telemetry POST error; events dropped" + ); + } + } + } + + // Final drain on shutdown so events buffered during the last + // BATCH_WINDOW aren't lost on graceful exit. + let final_batch = queue.drain_batch(MAX_BATCH_SIZE); + if !final_batch.is_empty() { + let body = serde_json::json!({ + "org_id": &org_id, + "events": final_batch, + }); + let _ = client.post(&endpoint).bearer_auth(&api_key).json(&body).send(); + } +} diff --git a/crates/soth-sdk-core/src/telemetry_queue.rs b/crates/soth-sdk-core/src/telemetry_queue.rs index 4a5271b6..1f836a02 100644 --- a/crates/soth-sdk-core/src/telemetry_queue.rs +++ b/crates/soth-sdk-core/src/telemetry_queue.rs @@ -67,6 +67,18 @@ impl TelemetryQueue { }; guard.drain(..).collect() } + + /// Pull up to `max` events for batched shipping. Returns an empty + /// vec when the queue is empty. Used by the background shipper. + #[allow(dead_code)] // only used when `http-telemetry` feature is on + pub(crate) fn drain_batch(&self, max: usize) -> Vec { + let mut guard = match self.inner.lock() { + Ok(g) => g, + Err(p) => p.into_inner(), + }; + let take = guard.len().min(max); + guard.drain(..take).collect() + } } #[cfg(test)] From da72b10c639a4bb037056f25d5c3f187ce208a1d Mon Sep 17 00:00:00 2001 From: Apoorv Raj Saxena Date: Thu, 30 Apr 2026 13:24:37 +0530 Subject: [PATCH 08/24] feat(sdk): Phase 1 - per-call CallContext + contextvars/AsyncLocalStorage MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Multi-tenant story: customers can override identity (user_id_hmac, team_id, device_id_hash, session_id, request_id) per-call rather than relying on SDK init defaults. Async-aware context propagation is language-native: Python contextvars, Node AsyncLocalStorage. soth-sdk-core: - New module crates/soth-sdk-core/src/context.rs with `CallContext` struct (#[non_exhaustive], all-Option fields, builder methods). - New `pre_call_with_context(call, &CallContext)` and `stream_begin_with_context(call, &CallContext)` methods. Old `pre_call` / `stream_begin` delegate with default context. - DecisionContext (slab) carries the resolved CallContext from pre_call to post_call so identity asserted at decision time matches what telemetry emits. - build_proxy_context now reads resolved context for user_id_hmac / team_id / device_id_hash / session_id; falls back to SdkConfig defaults when CallContext leaves a field None. Session ID is parsed as UUID where possible, otherwise dropped (proxy-shaped contract). - HMAC discipline preserved: user_id_hmac is the HMAC; the SDK never sees plaintext user IDs. Python (bindings/soth-py): - soth.context(*, user_id_hmac=, team_id=, device_id_hash=, session_id=, request_id=) is a contextmanager backed by contextvars.ContextVar so it survives asyncio task switches. - guard() and guard_stream() pull the current context and pass it to the FFI on every call. - PyO3 layer: pre_call(call, context=None) / stream_begin(call, context=None) accept an optional dict; build_call_context translates to CallContext. - Nested `with soth.context(...)` blocks merge — fields not set in inner block fall through to outer. Node (bindings/soth-node): - soth.withContext(overrides, async () => ...) backed by AsyncLocalStorage so async code awaited inside sees the same context across awaits. - guard() and guardStream() pull the current AsyncLocalStorage value and pass it to the napi FFI. - napi-rs layer: preCall(call, context?) / streamBegin(call, context?) accept an optional JsCallContext; build_call_context translates. - Nested withContext calls merge identically to the Python contract. Verification: - cargo build --workspace clean - soth-sdk-core 15 unit + 5 integration tests - Conformance harness 7/7 fixtures - All workspace lib tests unchanged - All 4 WASM combos clean - Both bindings recompile cleanly with both http-telemetry and per-call context Phase-1 follow-ups: - Helper for HMAC-ing a customer-supplied user_id (currently customer computes the HMAC themselves via their stored SOTH_HMAC_KEY) - Auto-instrumentation that picks up context automatically — that's the next commit - Context introspection helpers (get_current_user_id_hmac, etc.) Co-Authored-By: Claude Opus 4.7 (1M context) --- bindings/soth-node/index.d.ts | 14 +++++ bindings/soth-node/index.js | 49 +++++++++++++++- bindings/soth-node/src/lib.rs | 55 ++++++++++++++--- bindings/soth-py/python/soth/__init__.py | 57 +++++++++++++++++- bindings/soth-py/src/lib.rs | 60 ++++++++++++++++--- crates/soth-sdk-core/src/context.rs | 66 +++++++++++++++++++++ crates/soth-sdk-core/src/lib.rs | 2 + crates/soth-sdk-core/src/sdk.rs | 75 ++++++++++++++++++------ crates/soth-sdk-core/src/slab.rs | 6 ++ 9 files changed, 343 insertions(+), 41 deletions(-) create mode 100644 crates/soth-sdk-core/src/context.rs diff --git a/bindings/soth-node/index.d.ts b/bindings/soth-node/index.d.ts index 95bb30ad..0879e34f 100644 --- a/bindings/soth-node/index.d.ts +++ b/bindings/soth-node/index.d.ts @@ -10,6 +10,15 @@ export interface InitOptions { */ hmacKeyEnv?: string; hmacKeyStatic?: Buffer; + telemetryEndpoint?: string; +} + +export interface CallContextOverrides { + userIdHmac?: string; + teamId?: string; + deviceIdHash?: string; + sessionId?: string; + requestId?: string; } export interface Message { @@ -70,9 +79,14 @@ export interface GuardStreamOptions { } export function init(options: InitOptions): void; +export function shutdown(): void; export function guard(callFn: () => Promise, options: GuardOptions): Promise; export function guardStream( iterFactory: () => AsyncIterable | Promise>, options: GuardStreamOptions, ): AsyncIterable; +export function withContext( + overrides: CallContextOverrides, + fn: () => Promise, +): Promise; export function getSdk(): unknown; diff --git a/bindings/soth-node/index.js b/bindings/soth-node/index.js index b590d694..aed9f05b 100644 --- a/bindings/soth-node/index.js +++ b/bindings/soth-node/index.js @@ -12,11 +12,55 @@ // The exception inheritance MUST stay flat — `SothBlocked extends Error`. // If anyone changes that, `__test__/blocked-propagates.test.mjs` fails. +const { AsyncLocalStorage } = require('node:async_hooks'); + const native = require('./soth-node.darwin-arm64.node'); /* eslint-disable-line global-require */ // Real builds ship per-arch binaries via @napi-rs/cli; the line above // is a placeholder for local dev. Production loader logic lands in a // follow-up commit. +// Per-call context lives in AsyncLocalStorage so async functions +// awaited inside `withContext` see the same context after each await. +const _contextStore = new AsyncLocalStorage(); + +function _currentContext() { + return _contextStore.getStore() ?? null; +} + +/** + * Run `fn` with per-call identity overrides applied to every + * `guard()` / `guardStream()` inside it. Async-aware via + * AsyncLocalStorage. Nested `withContext` calls merge — fields not + * set in the inner block fall through to the outer block. + * + * `userIdHmac` MUST be the HMAC of the customer's user ID, computed + * by the customer's code using their `SOTH_HMAC_KEY`. The SDK never + * sees plaintext user IDs. + */ +async function withContext(overrides, fn) { + const current = _contextStore.getStore() ?? {}; + const merged = { ...current }; + for (const k of ['userIdHmac', 'teamId', 'deviceIdHash', 'sessionId', 'requestId']) { + if (overrides[k] !== undefined) merged[k] = overrides[k]; + } + return _contextStore.run(merged, fn); +} + +function _currentNativeContext() { + const ctx = _currentContext(); + if (!ctx) return null; + // napi-rs object key names match the Rust JsCallContext field + // names (snake_case). The JS-facing helper uses camelCase for + // ergonomics; we translate at the FFI boundary. + return { + userIdHmac: ctx.userIdHmac ?? null, + teamId: ctx.teamId ?? null, + deviceIdHash: ctx.deviceIdHash ?? null, + sessionId: ctx.sessionId ?? null, + requestId: ctx.requestId ?? null, + }; +} + class SothBlocked extends Error { constructor(decisionId, reason) { super(`SOTH policy blocked call: ${reason?.kind ?? 'unknown'}`); @@ -66,7 +110,7 @@ function getSdk() { async function guard(callFn, { call }) { const sdk = getSdk(); - const decision = sdk.preCall(call); + const decision = sdk.preCall(call, _currentNativeContext()); const { kind, token } = decision; if (kind === 'block') { @@ -123,7 +167,7 @@ async function* guardStream(iterFactory, { call, chunkExtractor } = {}) { if (!call) throw new Error('guardStream: call required'); const extractor = chunkExtractor ?? defaultOpenAIChunkExtractor; const sdk = getSdk(); - const decision = sdk.streamBegin(call); + const decision = sdk.streamBegin(call, _currentNativeContext()); const { kind, token } = decision; if (kind === 'block') { @@ -153,6 +197,7 @@ module.exports = { shutdown, guard, guardStream, + withContext, getSdk, SothBlocked, SothFlagged, diff --git a/bindings/soth-node/src/lib.rs b/bindings/soth-node/src/lib.rs index 9246a619..c9604664 100644 --- a/bindings/soth-node/src/lib.rs +++ b/bindings/soth-node/src/lib.rs @@ -15,9 +15,9 @@ use std::sync::{Arc, Mutex}; use napi::bindgen_prelude::*; use napi_derive::napi; use soth_sdk_core::{ - BlockReason as CoreBlockReason, Decision as CoreDecision, DecisionToken, FlagSeverity, - HmacKey, LlmCall, LlmChunk, LlmResponse, Message, SdkConfigBuilder, SothSdk as CoreSothSdk, - StreamObservation as CoreStreamObservation, Tool, + BlockReason as CoreBlockReason, CallContext, Decision as CoreDecision, DecisionToken, + FlagSeverity, HmacKey, LlmCall, LlmChunk, LlmResponse, Message, SdkConfigBuilder, + SothSdk as CoreSothSdk, StreamObservation as CoreStreamObservation, Tool, }; use soth_core::EndpointType; use zeroize::Zeroizing; @@ -80,6 +80,15 @@ pub struct JsLlmCall { pub stream: Option, } +#[napi(object)] +pub struct JsCallContext { + pub user_id_hmac: Option, + pub team_id: Option, + pub device_id_hash: Option, + pub session_id: Option, + pub request_id: Option, +} + #[napi(object)] pub struct JsTelemetryEvent { pub provider: String, @@ -154,11 +163,13 @@ impl SothSdk { /// Synchronous decision path. Returns a typed `JsDecision` that the /// JS shim translates into either a token (Allow / Flag) or a - /// thrown `SothBlocked` (Block / Redact). + /// thrown `SothBlocked` (Block / Redact). `context` is optional; + /// the JS shim pulls the current AsyncLocalStorage value. #[napi] - pub fn pre_call(&self, call: JsLlmCall) -> Result { + pub fn pre_call(&self, call: JsLlmCall, context: Option) -> Result { let llm_call = build_llm_call(call); - let decision = self.inner.pre_call(&llm_call); + let ctx = build_call_context(context); + let decision = self.inner.pre_call_with_context(&llm_call, &ctx); Ok(decision_to_js(&decision)) } @@ -181,9 +192,14 @@ impl SothSdk { /// with `streamEnd(token)`. The observation lives inside the SDK /// keyed by token, so the FFI boundary stays scalar-only. #[napi] - pub fn stream_begin(&self, call: JsLlmCall) -> Result { + pub fn stream_begin( + &self, + call: JsLlmCall, + context: Option, + ) -> Result { let llm_call = build_llm_call(call); - let (decision, observation) = self.inner.stream_begin(&llm_call); + let ctx = build_call_context(context); + let (decision, observation) = self.inner.stream_begin_with_context(&llm_call, &ctx); let token_raw = decision.token().raw(); // Sentinel tokens (SLAB_FULL / SENTINEL_FAIL_OPEN) skip slab // bookkeeping — there's no observation to stash because pre_call @@ -280,6 +296,29 @@ fn is_sentinel_raw(raw: u64) -> bool { raw == DecisionToken::SLAB_FULL.raw() || raw == DecisionToken::SENTINEL_FAIL_OPEN.raw() } +fn build_call_context(ctx: Option) -> CallContext { + let Some(ctx) = ctx else { + return CallContext::default(); + }; + let mut out = CallContext::new(); + if let Some(v) = ctx.user_id_hmac { + out = out.with_user_id_hmac(v); + } + if let Some(v) = ctx.team_id { + out = out.with_team_id(v); + } + if let Some(v) = ctx.device_id_hash { + out = out.with_device_id_hash(v); + } + if let Some(v) = ctx.session_id { + out = out.with_session_id(v); + } + if let Some(v) = ctx.request_id { + out = out.with_request_id(v); + } + out +} + // ── conversions ────────────────────────────────────────────────────── fn build_llm_call(call: JsLlmCall) -> LlmCall { diff --git a/bindings/soth-py/python/soth/__init__.py b/bindings/soth-py/python/soth/__init__.py index ce34b230..50c274b3 100644 --- a/bindings/soth-py/python/soth/__init__.py +++ b/bindings/soth-py/python/soth/__init__.py @@ -37,7 +37,9 @@ from __future__ import annotations -from typing import Any, Callable, Optional, TypeVar +import contextvars +from contextlib import contextmanager +from typing import Any, Callable, Iterator, Optional, TypeVar from . import _soth_native # type: ignore[attr-defined] from .exceptions import ( @@ -54,11 +56,58 @@ "shutdown", "guard", "guard_stream", + "context", "SothBlocked", "SothFlagged", "BlockReason", ] + +# Per-call context lives in a contextvars.ContextVar so it survives +# `asyncio` task switches naturally — async code that awaits inside a +# `with soth.context(...)` block sees the same context after the await. +_current_context: contextvars.ContextVar[dict[str, str]] = contextvars.ContextVar( + "soth_current_context", default={} +) + + +@contextmanager +def context( + *, + user_id_hmac: Optional[str] = None, + team_id: Optional[str] = None, + device_id_hash: Optional[str] = None, + session_id: Optional[str] = None, + request_id: Optional[str] = None, +) -> Iterator[None]: + """Override identity fields for any `guard()` / `guard_stream()` + calls inside this block. + + Uses `contextvars` so async tasks awaited inside the block see the + same context. Nested `with soth.context(...)` blocks merge — fields + not set in the inner block fall through to the outer block. + + `user_id_hmac` MUST be the HMAC of the customer's user ID, + computed by the customer's code using their `SOTH_HMAC_KEY`. + The SDK never sees plaintext user IDs. + """ + current = dict(_current_context.get()) + if user_id_hmac is not None: + current["user_id_hmac"] = user_id_hmac + if team_id is not None: + current["team_id"] = team_id + if device_id_hash is not None: + current["device_id_hash"] = device_id_hash + if session_id is not None: + current["session_id"] = session_id + if request_id is not None: + current["request_id"] = request_id + token = _current_context.set(current) + try: + yield + finally: + _current_context.reset(token) + # Module-level singleton. Bindings keep one SothSdk per process; per-call # context (org/user/team override) is layered on top via `with_context`. _singleton: Optional[_soth_native.SothSdk] = None @@ -134,7 +183,8 @@ def guard( narrow so it can be applied per-call with minimal disruption. """ sdk = get_sdk() - decision = sdk.pre_call(call) + ctx = _current_context.get() or None + decision = sdk.pre_call(call, ctx) kind = decision["kind"] token = decision["token"] @@ -189,7 +239,8 @@ async def guard_stream( completion or exception. """ sdk = get_sdk() - decision, observation = sdk.stream_begin(call) + ctx = _current_context.get() or None + decision, observation = sdk.stream_begin(call, ctx) kind = decision["kind"] if kind == _soth_native.DECISION_KIND_BLOCK: diff --git a/bindings/soth-py/src/lib.rs b/bindings/soth-py/src/lib.rs index 771ebb27..7bdd0b0b 100644 --- a/bindings/soth-py/src/lib.rs +++ b/bindings/soth-py/src/lib.rs @@ -15,9 +15,9 @@ use pyo3::exceptions::{PyRuntimeError, PyValueError}; use pyo3::prelude::*; use pyo3::types::{PyDict, PyList}; use soth_sdk_core::{ - BlockReason as CoreBlockReason, Decision as CoreDecision, DecisionToken, FlagSeverity, - HmacKey, LlmCall, LlmChunk, LlmResponse, Message, SdkConfigBuilder, SothSdk as CoreSothSdk, - StreamObservation as CoreStreamObservation, Tool, + BlockReason as CoreBlockReason, CallContext, Decision as CoreDecision, DecisionToken, + FlagSeverity, HmacKey, LlmCall, LlmChunk, LlmResponse, Message, SdkConfigBuilder, + SothSdk as CoreSothSdk, StreamObservation as CoreStreamObservation, Tool, }; use soth_core::EndpointType; use zeroize::Zeroizing; @@ -85,11 +85,18 @@ impl PySothSdk { /// Returns a `dict` describing the decision; the Python wrapper in /// `soth/__init__.py` translates this into either a token (`Allow` / /// `Flag`) or a raised `SothBlocked` exception (`Block` / - /// `Redact` paths). - #[pyo3(signature = (call_dict))] - fn pre_call<'py>(&self, py: Python<'py>, call_dict: &Bound<'py, PyDict>) -> PyResult> { + /// `Redact` paths). `context_dict` is optional; the Python wrapper + /// pulls the current `contextvars` value before each call. + #[pyo3(signature = (call_dict, context_dict=None))] + fn pre_call<'py>( + &self, + py: Python<'py>, + call_dict: &Bound<'py, PyDict>, + context_dict: Option<&Bound<'py, PyDict>>, + ) -> PyResult> { let call = build_llm_call(call_dict)?; - let decision = self.inner.pre_call(&call); + let ctx = build_call_context(context_dict)?; + let decision = self.inner.pre_call_with_context(&call, &ctx); decision_to_pydict(py, &decision) } @@ -114,14 +121,16 @@ impl PySothSdk { /// the provider's stream and feeds chunks via `observation.chunk(...)`, /// then calls `observation.end()` on terminal chunk to consume the /// token and emit telemetry. - #[pyo3(signature = (call_dict))] + #[pyo3(signature = (call_dict, context_dict=None))] fn stream_begin<'py>( &self, py: Python<'py>, call_dict: &Bound<'py, PyDict>, + context_dict: Option<&Bound<'py, PyDict>>, ) -> PyResult<(Bound<'py, PyDict>, PyStreamObservation)> { let call = build_llm_call(call_dict)?; - let (decision, observation) = self.inner.stream_begin(&call); + let ctx = build_call_context(context_dict)?; + let (decision, observation) = self.inner.stream_begin_with_context(&call, &ctx); let decision_dict = decision_to_pydict(py, &decision)?; let py_obs = PyStreamObservation { sdk: Arc::clone(&self.inner), @@ -408,6 +417,39 @@ fn build_llm_call(dict: &Bound<'_, PyDict>) -> PyResult { }) } +fn build_call_context(dict: Option<&Bound<'_, PyDict>>) -> PyResult { + let Some(dict) = dict else { + return Ok(CallContext::default()); + }; + let mut ctx = CallContext::default(); + if let Some(v) = dict.get_item("user_id_hmac")? { + if !v.is_none() { + ctx.user_id_hmac = Some(v.extract()?); + } + } + if let Some(v) = dict.get_item("team_id")? { + if !v.is_none() { + ctx.team_id = Some(v.extract()?); + } + } + if let Some(v) = dict.get_item("device_id_hash")? { + if !v.is_none() { + ctx.device_id_hash = Some(v.extract()?); + } + } + if let Some(v) = dict.get_item("session_id")? { + if !v.is_none() { + ctx.session_id = Some(v.extract()?); + } + } + if let Some(v) = dict.get_item("request_id")? { + if !v.is_none() { + ctx.request_id = Some(v.extract()?); + } + } + Ok(ctx) +} + fn response_from_pydict(_dict: Option<&Bound<'_, PyDict>>) -> PyResult { // V0: response details are not yet consumed by post_call. Phase-1 // wires response-side artifact scanning + usage stats from the diff --git a/crates/soth-sdk-core/src/context.rs b/crates/soth-sdk-core/src/context.rs new file mode 100644 index 00000000..2a77595d --- /dev/null +++ b/crates/soth-sdk-core/src/context.rs @@ -0,0 +1,66 @@ +//! Per-call `CallContext` — overrides identity fields for a single +//! `pre_call` / `stream_begin` invocation. +//! +//! Bindings layer language-native context propagation on top +//! (Python's `contextvars` so it survives async tasks; Node's +//! `AsyncLocalStorage` for the same reason). The Rust core stays +//! sync at the boundary — bindings stash the current context into +//! the language's async-aware container and pass it explicitly per +//! call. +//! +//! Defaults come from `SdkConfig.default_team_id` / +//! `SdkConfig.default_device_id_hash`; callers override per-call to +//! attribute traffic to specific users / teams / sessions. + +use serde::{Deserialize, Serialize}; + +/// Per-call identity overrides. All fields are optional — anything +/// left as `None` falls back to the corresponding `SdkConfig` default +/// (or to the SDK-level "anonymous" sentinel when neither is set). +#[derive(Debug, Clone, Default, Serialize, Deserialize)] +#[non_exhaustive] +pub struct CallContext { + /// Customer's end-user identifier, HMAC'd by the binding before + /// it reaches this struct. The SDK never sees plaintext user IDs; + /// the HMAC is computed inside the binding using the + /// `SdkConfig.hmac_key`. + pub user_id_hmac: Option, + pub team_id: Option, + pub device_id_hash: Option, + pub session_id: Option, + /// Customer-supplied request correlation ID (e.g. their HTTP + /// request ID). Carried through telemetry events so they can + /// correlate SOTH events with their own logs. + pub request_id: Option, +} + +impl CallContext { + pub fn new() -> Self { + Self::default() + } + + pub fn with_user_id_hmac(mut self, user_id_hmac: impl Into) -> Self { + self.user_id_hmac = Some(user_id_hmac.into()); + self + } + + pub fn with_team_id(mut self, team_id: impl Into) -> Self { + self.team_id = Some(team_id.into()); + self + } + + pub fn with_device_id_hash(mut self, device_id_hash: impl Into) -> Self { + self.device_id_hash = Some(device_id_hash.into()); + self + } + + pub fn with_session_id(mut self, session_id: impl Into) -> Self { + self.session_id = Some(session_id.into()); + self + } + + pub fn with_request_id(mut self, request_id: impl Into) -> Self { + self.request_id = Some(request_id.into()); + self + } +} diff --git a/crates/soth-sdk-core/src/lib.rs b/crates/soth-sdk-core/src/lib.rs index 3d84dd37..af70bb9d 100644 --- a/crates/soth-sdk-core/src/lib.rs +++ b/crates/soth-sdk-core/src/lib.rs @@ -17,6 +17,7 @@ pub mod call; pub mod config; +pub mod context; pub mod decision; pub mod error; @@ -31,6 +32,7 @@ pub use call::{LlmCall, LlmChunk, LlmResponse, Message, Tool}; pub use config::{ BundleSource, ClassificationMode, HmacKey, SdkConfig, SdkConfigBuilder, StorageMode, }; +pub use context::CallContext; pub use decision::{ BlockReason, BudgetKind, Decision, DecisionToken, FlagSeverity, MessageRedaction, MessageRedactions, RedactReason, diff --git a/crates/soth-sdk-core/src/sdk.rs b/crates/soth-sdk-core/src/sdk.rs index dc44ab5c..e9356b90 100644 --- a/crates/soth-sdk-core/src/sdk.rs +++ b/crates/soth-sdk-core/src/sdk.rs @@ -28,6 +28,7 @@ use soth_core::{ use crate::call::{LlmCall, LlmChunk, LlmResponse}; use crate::config::{BundleSource, ClassificationMode, SdkConfig}; +use crate::context::CallContext; use crate::decision::{ BlockReason, BudgetKind, Decision, DecisionToken, FlagSeverity, MessageRedactions, }; @@ -181,13 +182,21 @@ impl SothSdk { } /// Synchronous decision path. Returns within 5 ms p99 (binding-side - /// histograms gate this in CI). + /// histograms gate this in CI). Convenience: equivalent to + /// `pre_call_with_context(call, &CallContext::default())`. + pub fn pre_call(&self, call: &LlmCall) -> Decision { + self.pre_call_with_context(call, &CallContext::default()) + } + + /// Synchronous decision path with per-call identity overrides. + /// Bindings expose this through language-native context primitives + /// (Python `contextvars`, Node `AsyncLocalStorage`). /// /// Allowed work (per spec §7.1): /// - artifact scan via `process_normalized` /// - artifact-conditioned system rules /// - artifact-conditioned org rules - pub fn pre_call(&self, call: &LlmCall) -> Decision { + pub fn pre_call_with_context(&self, call: &LlmCall, ctx: &CallContext) -> Decision { let detect_bundle = self.detect_bundle.load_full(); let detect = soth_detect::process_normalized( self.detect_registry.as_ref(), @@ -198,6 +207,7 @@ impl SothSdk { ); let summary = ArtifactsSummary::from_artifacts(&detect.artifacts); + let resolved_ctx = self.resolve_context(ctx); // Sync block path — credential / private-key artifacts are an // unconditional block in v0. Phase-1 expands this with full @@ -205,7 +215,7 @@ impl SothSdk { if let Some(reason) = detect.artifacts.iter().find_map(|a| { artifact_block_reason(&a.kind, a.severity) }) { - let ctx = DecisionContext { + let slab_ctx = DecisionContext { created_at: std::time::Instant::now(), generation: 0, detect: detect.clone(), @@ -213,13 +223,14 @@ impl SothSdk { call_provider: call.provider.clone(), call_model: call.model.clone(), user_content: detect.normalized.user_prompt.clone(), + resolved_context: resolved_ctx, }; - let token = self.slab.allocate(ctx); + let token = self.slab.allocate(slab_ctx); return Decision::Block { token, reason }; } // Allow path — stash the partial state for post_call enrichment. - let ctx = DecisionContext { + let slab_ctx = DecisionContext { created_at: std::time::Instant::now(), generation: 0, detect, @@ -227,11 +238,25 @@ impl SothSdk { call_provider: call.provider.clone(), call_model: call.model.clone(), user_content: None, // populated on consume; we cloned detect so re-derive there + resolved_context: resolved_ctx, }; - let token = self.slab.allocate(ctx); + let token = self.slab.allocate(slab_ctx); Decision::Allow { token } } + /// Merge per-call overrides with config defaults to produce the + /// fully-resolved CallContext used by `post_call` enrichment. + fn resolve_context(&self, ctx: &CallContext) -> CallContext { + let mut resolved = ctx.clone(); + if resolved.team_id.is_none() { + resolved.team_id = self.config.default_team_id.clone(); + } + if resolved.device_id_hash.is_none() { + resolved.device_id_hash = self.config.default_device_id_hash.clone(); + } + resolved + } + /// Async-ish enrichment + telemetry emission. Bindings spawn this /// off the host's critical path. Idempotent on sentinel tokens /// (no-op). @@ -261,7 +286,16 @@ impl SothSdk { /// Streaming counterpart to `pre_call`. Returns the decision and an /// observation handle for `stream_chunk` / `stream_end`. pub fn stream_begin(&self, call: &LlmCall) -> (Decision, StreamObservation) { - let decision = self.pre_call(call); + self.stream_begin_with_context(call, &CallContext::default()) + } + + /// Streaming counterpart to `pre_call_with_context`. + pub fn stream_begin_with_context( + &self, + call: &LlmCall, + ctx: &CallContext, + ) -> (Decision, StreamObservation) { + let decision = self.pre_call_with_context(call, ctx); let token = decision.token(); (decision, StreamObservation::new(token)) } @@ -318,20 +352,23 @@ impl SothSdk { } fn build_proxy_context(&self, ctx: &DecisionContext) -> ProxyContext { + let resolved = &ctx.resolved_context; + let user_id_hmac = resolved + .user_id_hmac + .clone() + .unwrap_or_else(|| String::from("sdk-anonymous")); + let team_id = resolved.team_id.clone().unwrap_or_default(); + let device_id_hash = resolved.device_id_hash.clone().unwrap_or_default(); + let session_id = resolved + .session_id + .as_ref() + .and_then(|s| uuid::Uuid::parse_str(s).ok()); ProxyContext { identity: IdentityContext { org_id: self.config.org_id.clone(), - user_id_hmac: self - .config - .default_team_id - .clone() - .unwrap_or_else(|| String::from("sdk-anonymous")), - team_id: self.config.default_team_id.clone().unwrap_or_default(), - device_id_hash: self - .config - .default_device_id_hash - .clone() - .unwrap_or_default(), + user_id_hmac, + team_id, + device_id_hash, endpoint_hash: String::new(), capture_mode: self.config.capture_mode, traffic_classification: TrafficClassification::ToolUsage, @@ -339,7 +376,7 @@ impl SothSdk { session_snapshot: Some(SessionSnapshot::default()), declared_provider: Some(ctx.call_provider.clone()), declared_application: None, - session_id: None, + session_id, deployment_context: None, bundle_trust_level: None, precomputed_commitment_nonce: None, diff --git a/crates/soth-sdk-core/src/slab.rs b/crates/soth-sdk-core/src/slab.rs index 7629a507..4a7816cc 100644 --- a/crates/soth-sdk-core/src/slab.rs +++ b/crates/soth-sdk-core/src/slab.rs @@ -15,6 +15,7 @@ use std::time::Instant; use soth_core::{ArtifactKind, DetectResult}; +use crate::context::CallContext; use crate::decision::DecisionToken; const SLAB_CAPACITY: usize = 4096; @@ -39,6 +40,10 @@ pub(crate) struct DecisionContext { pub call_model: String, #[allow(dead_code)] // Phase-1: cloud-classify path serializes user_content pub user_content: Option, + /// Resolved per-call context — `CallContext` overrides merged with + /// `SdkConfig` defaults at `pre_call` time. Carried so `post_call` + /// uses the same identity values it asserted at decision time. + pub resolved_context: CallContext, } /// Pre-summarized artifact view so `post_call` doesn't re-walk the @@ -249,6 +254,7 @@ mod tests { call_provider: "openai".to_string(), call_model: "gpt-4o-mini".to_string(), user_content: None, + resolved_context: CallContext::default(), } } From 32e52ffcbe32070c931be6a48434df2ac63dc1ef Mon Sep 17 00:00:00 2001 From: Apoorv Raj Saxena Date: Thu, 30 Apr 2026 13:37:36 +0530 Subject: [PATCH 09/24] feat(sdk): Phase 1 - auto-instrumentation for OpenAI + Anthropic MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `soth.instrument()` monkey-patches provider SDK client classes so customers don't write `soth.guard()` per-call. Built robustly per the brief — six guarantees gate the contract: 1. **Idempotent.** instrument() called twice returns "skipped:already-instrumented" rather than double-wrapping. 2. **Reversible.** uninstrument() restores originals captured at apply time. If another tool wrapped over our wrapper, we leave theirs in place and log rather than clobbering. 3. **Provider-conditional.** Missing provider packages return "skipped:not-installed" without raising — apps deploying with subset-of-providers don't break. 4. **Fail-open.** Any exception in build_call extraction or wrapper setup falls back to the original SDK call uninstrumented. The customer's API call MUST complete unaffected if SOTH itself fails. 5. **Version-tolerant.** Adapters use defensive `getattr` / `Object.keys` access and walk multiple possible paths into provider modules so minor-version SDK changes don't break. 6. **Sync + async aware.** Each Python adapter wraps both sync (`Completions`) and async (`AsyncCompletions`) classes; the same `guard()` entry point handles both because it now detects coroutine returns. Python (bindings/soth-py/python/soth/instrumentation/): __init__.py: registry + instrument() / uninstrument() / is_instrumented(); status dict per provider. _base.py: wrap_method(target, method_name, ...) replaces a class method with a SOTH wrapper carrying provenance markers (__soth_wrapped__, __soth_provider__, __soth_original__). Sync + async wrappers built separately. revert_all checks the wrapper is still ours before restoring. _openai.py: patches openai.resources.chat.completions.Completions + AsyncCompletions + Responses (when available). buildCall extracts model/messages/stream/tools/system from kwargs; flattens content arrays for multi-modal calls. Tool params are stably serialized for hash determinism. _anthropic.py: patches anthropic.resources.messages.Messages + AsyncMessages. Handles Anthropic-specific shape (system as separate field, tools with input_schema, streaming events with content_block_delta / message_delta / message_stop). guard() update (Python): Detects coroutine return from call_fn. Sync providers finalize inline; async providers receive a coroutine to await. One entry point handles both sync `OpenAI` and async `AsyncOpenAI` clients without separate APIs. New `guard_stream_sync()` for sync-streaming providers (matches `guard_stream` async generator). Node (bindings/soth-node/instrumentation/): index.js: registry + instrument() / uninstrument() / isInstrumented(). Lazy-loaded so `require('@soth/sdk')` doesn't pull provider SDKs on import. _base.js: wrapMethod walks prototype chain; provenance markers via non-enumerable Object.defineProperty so they don't leak into JSON.stringify of the wrapped method. revertAll same third-party-detection semantic as Python. openai.js: walks multiple possible paths to find Completions class across openai 4.x sub-versions; falls back cleanly when none match. Negative tests for both bindings: - instrument is idempotent (second call: "skipped:already-…") - uninstrument reverses - uninstrument-without-instrument is safe - explicit providers list filters - missing provider returns "skipped:not-installed" - buildCall exception falls through to original (fail-open) - wrapped method carries provenance markers - revert leaves third-party wrapper in place - guard() handles coroutine and value returns (Python only — Node already async-aware) - OpenAI/Anthropic adapter apply() returns bool, never raises What's NOT in this commit (Phase-1.5 follow-ups): - Cohere, Google GenAI, Mistral Python adapters - Anthropic Node adapter (Anthropic's npm SDK is less mature than openai's; will land separately) - LangChain / LlamaIndex / Vercel AI SDK callback adapters (those go through a different integration path; framework adapters are Phase 3 in the original plan) Verification: cargo build --workspace clean All workspace tests unchanged (654 + 5 sdk-core integration) Conformance harness 7/7 fixtures All 4 WASM combos clean Co-Authored-By: Claude Opus 4.7 (1M context) --- .../__test__/instrumentation.test.mjs | 198 +++++++++++++ bindings/soth-node/index.d.ts | 15 + bindings/soth-node/index.js | 28 ++ bindings/soth-node/instrumentation/_base.js | 97 +++++++ bindings/soth-node/instrumentation/index.js | 109 +++++++ bindings/soth-node/instrumentation/openai.js | 171 +++++++++++ bindings/soth-py/python/soth/__init__.py | 120 ++++++-- .../python/soth/instrumentation/__init__.py | 142 +++++++++ .../python/soth/instrumentation/_anthropic.py | 156 ++++++++++ .../python/soth/instrumentation/_base.py | 207 +++++++++++++ .../python/soth/instrumentation/_openai.py | 190 ++++++++++++ .../soth-py/tests/test_instrumentation.py | 274 ++++++++++++++++++ 12 files changed, 1689 insertions(+), 18 deletions(-) create mode 100644 bindings/soth-node/__test__/instrumentation.test.mjs create mode 100644 bindings/soth-node/instrumentation/_base.js create mode 100644 bindings/soth-node/instrumentation/index.js create mode 100644 bindings/soth-node/instrumentation/openai.js create mode 100644 bindings/soth-py/python/soth/instrumentation/__init__.py create mode 100644 bindings/soth-py/python/soth/instrumentation/_anthropic.py create mode 100644 bindings/soth-py/python/soth/instrumentation/_base.py create mode 100644 bindings/soth-py/python/soth/instrumentation/_openai.py create mode 100644 bindings/soth-py/tests/test_instrumentation.py diff --git a/bindings/soth-node/__test__/instrumentation.test.mjs b/bindings/soth-node/__test__/instrumentation.test.mjs new file mode 100644 index 00000000..04e1a46a --- /dev/null +++ b/bindings/soth-node/__test__/instrumentation.test.mjs @@ -0,0 +1,198 @@ +// Robustness tests for soth.instrument(). +// +// Mirrors the Python instrumentation suite: idempotency, reversibility, +// missing-provider tolerance, fail-open extractors, double-wrap +// detection, sync+async coroutine handling. +// +// Tests do NOT require the `openai` package — when absent, the relevant +// adapter assertions skip rather than fail. +// +// Run with: +// cd bindings/soth-node +// npm install +// npm run build:debug +// npm test + +import { test } from 'node:test'; +import { strict as assert } from 'node:assert'; + +import * as soth from '../index.js'; + +process.env.SOTH_HMAC_KEY = 'x'.repeat(32); + +soth.init({ + apiKey: 'sk-test', + orgId: 'org-test', + hmacKeyEnv: 'SOTH_HMAC_KEY', +}); + +// ── idempotency / reversibility ───────────────────────────────────── + +test('instrument is idempotent — second call reports already-instrumented', () => { + const first = soth.instrument(); + const second = soth.instrument(); + for (const [provider, status] of Object.entries(first)) { + if (status === 'instrumented') { + assert.equal( + second[provider], + 'skipped:already-instrumented', + `${provider}: idempotency violated`, + ); + } + } + // Cleanup. + soth.uninstrument(); +}); + +test('uninstrument reverses instrument', () => { + const first = soth.instrument(); + soth.uninstrument(); + for (const [provider, status] of Object.entries(first)) { + if (status === 'instrumented') { + assert.equal( + soth.isInstrumented(provider), + false, + `${provider} still instrumented after uninstrument`, + ); + } + } +}); + +test('uninstrument without prior instrument is safe', () => { + const results = soth.uninstrument(); + for (const [, status] of Object.entries(results)) { + assert.ok(['skipped:not-instrumented', 'skipped:disabled'].includes(status)); + } +}); + +// ── provider selection ───────────────────────────────────────────── + +test('instrument with explicit providers skips others', () => { + const results = soth.instrument({ providers: ['openai'] }); + // No anthropic adapter on Node yet (Phase-1 ships OpenAI only). + // The registry doesn't include anthropic, so the result map only + // has the providers we know about. Exercise: openai is in the + // selected set and gets processed (instrumented or + // skipped:not-installed). + assert.ok('openai' in results); + soth.uninstrument(); +}); + +// ── fail-open extractor ──────────────────────────────────────────── + +test('buildCall exception falls through to original (fail-open)', async () => { + const { wrapMethod, revertAll } = await import('../instrumentation/_base.js'); + + class FakeClient { + create(opts) { + return Promise.resolve({ ok: true, opts }); + } + } + + const bustedBuildCall = () => { + throw new Error('simulated extractor crash'); + }; + + const patch = wrapMethod(FakeClient, 'create', { + providerName: 'fake', + buildCall: bustedBuildCall, + }); + assert.ok(patch !== null); + + const client = new FakeClient(); + // Despite the extractor raising, the original method runs and + // returns its expected value. + const result = await client.create({ model: 'x' }); + assert.equal(result.ok, true); + assert.deepEqual(result.opts, { model: 'x' }); + + revertAll([patch]); +}); + +// ── double-wrap detection ────────────────────────────────────────── + +test('wrapped method carries SOTH provenance markers', async () => { + const { wrapMethod, isInstrumentedMethod, revertAll } = await import( + '../instrumentation/_base.js' + ); + + class Target { + m() { + return 1; + } + } + + const patch = wrapMethod(Target, 'm', { + providerName: 'test', + buildCall: () => ({ provider: 'test', model: '', messages: [] }), + }); + assert.ok(patch !== null); + assert.equal(isInstrumentedMethod(Target.prototype.m), true); + assert.equal(Target.prototype.m.__sothProvider, 'test'); + + revertAll([patch]); +}); + +test('revert leaves third-party wrapper in place', async () => { + const { wrapMethod, revertAll } = await import('../instrumentation/_base.js'); + + class Target { + m() { + return 1; + } + } + + const patch = wrapMethod(Target, 'm', { + providerName: 'test', + buildCall: () => ({ provider: 'test', model: '', messages: [] }), + }); + assert.ok(patch !== null); + + // Simulate another tool wrapping over our wrapper. + const sothWrapper = Target.prototype.m; + function thirdPartyWrapper(...args) { + return sothWrapper.apply(this, args); + } + Target.prototype.m = thirdPartyWrapper; + + revertAll([patch]); + // We don't clobber the third-party wrapper; it stays. + assert.equal(Target.prototype.m, thirdPartyWrapper); +}); + +// ── adapter integration ──────────────────────────────────────────── + +test('OpenAI adapter apply returns bool — no exceptions', async () => { + const adapter = await import('../instrumentation/openai.js'); + const result = adapter.apply(); + assert.ok(typeof result === 'boolean'); + if (result === true) { + adapter.revert(); + } +}); + +test('OpenAI buildCall extracts model + messages from create() options', async () => { + const adapter = await import('../instrumentation/openai.js'); + const call = adapter._buildCall([ + { + model: 'gpt-4o-mini', + messages: [{ role: 'user', content: 'hello' }], + stream: false, + tools: [ + { + type: 'function', + function: { + name: 'lookup_weather', + description: 'Look up the weather', + parameters: { type: 'object', properties: { city: { type: 'string' } } }, + }, + }, + ], + }, + ]); + assert.equal(call.provider, 'openai'); + assert.equal(call.model, 'gpt-4o-mini'); + assert.equal(call.messages.length, 1); + assert.equal(call.tools.length, 1); + assert.equal(call.tools[0].name, 'lookup_weather'); +}); diff --git a/bindings/soth-node/index.d.ts b/bindings/soth-node/index.d.ts index 0879e34f..3c7f5fc8 100644 --- a/bindings/soth-node/index.d.ts +++ b/bindings/soth-node/index.d.ts @@ -89,4 +89,19 @@ export function withContext( overrides: CallContextOverrides, fn: () => Promise, ): Promise; + +export interface InstrumentOptions { + /** + * Limit instrumentation to a subset of providers. Omitting this + * field instruments every registered provider that is importable. + */ + providers?: string[]; +} + +export type InstrumentResult = Record; + +export function instrument(options?: InstrumentOptions): InstrumentResult; +export function uninstrument(options?: InstrumentOptions): InstrumentResult; +export function isInstrumented(provider: string): boolean; + export function getSdk(): unknown; diff --git a/bindings/soth-node/index.js b/bindings/soth-node/index.js index aed9f05b..326eea05 100644 --- a/bindings/soth-node/index.js +++ b/bindings/soth-node/index.js @@ -192,12 +192,40 @@ async function* guardStream(iterFactory, { call, chunkExtractor } = {}) { } } +// Auto-instrumentation is loaded lazily so `require('@soth/sdk')` +// doesn't pull provider SDKs into the import graph until +// `instrument()` is actually called. +let _instrumentation = null; +function _getInstrumentation() { + if (_instrumentation === null) { + /* eslint-disable global-require */ + _instrumentation = require('./instrumentation/index.js'); + /* eslint-enable global-require */ + } + return _instrumentation; +} + +function instrument(opts) { + return _getInstrumentation().instrument(opts); +} + +function uninstrument(opts) { + return _getInstrumentation().uninstrument(opts); +} + +function isInstrumented(provider) { + return _getInstrumentation().isInstrumented(provider); +} + module.exports = { init, shutdown, guard, guardStream, withContext, + instrument, + uninstrument, + isInstrumented, getSdk, SothBlocked, SothFlagged, diff --git a/bindings/soth-node/instrumentation/_base.js b/bindings/soth-node/instrumentation/_base.js new file mode 100644 index 00000000..42fcea66 --- /dev/null +++ b/bindings/soth-node/instrumentation/_base.js @@ -0,0 +1,97 @@ +// Shared instrumentation helpers for Node provider adapters. +// +// `wrapMethod(target, methodName, { providerName, buildCall, chunkExtractor })` +// replaces a method on a class prototype with a SOTH-wrapped version +// that: +// - calls buildCall(args) to derive the LlmCall dict +// - on extractor failure: logs and falls through to the original +// - on streaming (`stream: true` in args[0]): routes through +// guardStream +// - on non-streaming: routes through guard +// +// Returns a Patch object so revert() can restore the original. + +const soth = require('../index.js'); // for guard / guardStream + +function wrapMethod(target, methodName, { providerName, buildCall, chunkExtractor }) { + if (!target?.prototype) { + return null; + } + const original = target.prototype[methodName]; + if (typeof original !== 'function') { + return null; + } + + const wrapper = function wrappedSdkMethod(...args) { + let callDict; + try { + callDict = buildCall(args); + } catch (e) { + console.warn(`soth: buildCall failed for ${providerName}.${methodName}:`, e?.message ?? e); + return original.apply(this, args); + } + + const isStreaming = Boolean(args?.[0]?.stream); + const self = this; + + if (isStreaming) { + // guardStream is an async generator. The SDK consumer uses + // `for await (const chunk of result)`; the original method + // typically returns an `AsyncIterable` already. We hand the + // factory through so guardStream can lazy-call the underlying + // method and wire chunks. + return soth.guardStream( + () => original.apply(self, args), + { + call: callDict, + chunkExtractor, + }, + ); + } + + // Non-streaming: original may return a Promise OR a value. soth.guard + // accepts an async fn and awaits if needed; here we wrap the call + // so guard sees the same shape. + return soth.guard( + async () => { + const ret = original.apply(self, args); + return ret && typeof ret.then === 'function' ? await ret : ret; + }, + { call: callDict }, + ); + }; + + // SOTH provenance markers — let revert() and any future + // re-instrument check whether the method is already wrapped. + Object.defineProperty(wrapper, '__sothWrapped', { value: true, enumerable: false }); + Object.defineProperty(wrapper, '__sothProvider', { value: providerName, enumerable: false }); + Object.defineProperty(wrapper, '__sothOriginal', { value: original, enumerable: false }); + wrapper.displayName = `soth(${providerName}.${methodName})`; + + target.prototype[methodName] = wrapper; + return { target, methodName, original }; +} + +function isInstrumentedMethod(fn) { + return Boolean(fn && fn.__sothWrapped); +} + +function revertAll(patches) { + for (const patch of patches) { + const current = patch.target.prototype[patch.methodName]; + if (!current) continue; + if (!isInstrumentedMethod(current)) { + console.warn( + `soth: ${patch.target.name}.${patch.methodName} was rewrapped by another tool; leaving in place`, + ); + continue; + } + patch.target.prototype[patch.methodName] = patch.original; + } +} + +module.exports = { + wrapMethod, + isInstrumentedMethod, + revertAll, +}; diff --git a/bindings/soth-node/instrumentation/index.js b/bindings/soth-node/instrumentation/index.js new file mode 100644 index 00000000..f6c4fbd3 --- /dev/null +++ b/bindings/soth-node/instrumentation/index.js @@ -0,0 +1,109 @@ +// Auto-instrumentation for provider SDKs. +// +// Public entry points (re-exported from the top-level `index.js`): +// soth.instrument({ providers }) +// soth.uninstrument({ providers }) +// soth.isInstrumented(provider) +// +// Robustness contract — same as the Python adapter +// (`python/soth/instrumentation/__init__.py`): +// +// 1. Idempotent. Calling instrument() twice returns the same state +// without re-wrapping. Subsequent calls return "skipped:already-…". +// 2. Reversible. uninstrument() restores the originals captured at +// apply time; if another tool wrapped over us, we leave that +// wrapper in place and log. +// 3. Provider-conditional. Missing packages don't throw — the entry +// returns "skipped:not-installed". +// 4. Fail open. Extractor exceptions fall back to the original SDK +// call uninstrumented; SOTH never breaks the customer's API call. +// 5. Sync + async aware. Provider methods that return Promises are +// handled the same way as those that return values directly, +// because the JS guard()/guardStream() helpers already detect. + +const openai = require('./openai.js'); + +const REGISTRY = Object.freeze({ + openai, +}); + +const _state = Object.fromEntries(Object.keys(REGISTRY).map((k) => [k, false])); + +function _selectedProviders(requested) { + if (!requested) return new Set(Object.keys(REGISTRY)); + if (Array.isArray(requested)) return new Set(requested); + return new Set(Object.keys(REGISTRY)); // unknown shape → instrument all +} + +function instrument({ providers } = {}) { + const selected = _selectedProviders(providers); + const results = {}; + + for (const [name, adapter] of Object.entries(REGISTRY)) { + if (!selected.has(name)) { + results[name] = 'skipped:disabled'; + continue; + } + if (_state[name]) { + results[name] = 'skipped:already-instrumented'; + continue; + } + + let applied; + try { + applied = adapter.apply(); + } catch (e) { + console.warn(`soth.instrument(${name}) failed:`, e?.message ?? e); + results[name] = `error:${e?.constructor?.name ?? 'Error'}`; + continue; + } + + if (applied) { + _state[name] = true; + results[name] = 'instrumented'; + } else { + results[name] = 'skipped:not-installed'; + } + } + + return results; +} + +function uninstrument({ providers } = {}) { + const selected = _selectedProviders(providers); + const results = {}; + + for (const [name, adapter] of Object.entries(REGISTRY)) { + if (!selected.has(name)) { + results[name] = 'skipped:disabled'; + continue; + } + if (!_state[name]) { + results[name] = 'skipped:not-instrumented'; + continue; + } + + try { + adapter.revert(); + } catch (e) { + console.warn(`soth.uninstrument(${name}) failed:`, e?.message ?? e); + results[name] = `error:${e?.constructor?.name ?? 'Error'}`; + continue; + } + + _state[name] = false; + results[name] = 'uninstrumented'; + } + + return results; +} + +function isInstrumented(provider) { + return _state[provider] === true; +} + +module.exports = { + instrument, + uninstrument, + isInstrumented, +}; diff --git a/bindings/soth-node/instrumentation/openai.js b/bindings/soth-node/instrumentation/openai.js new file mode 100644 index 00000000..292f479b --- /dev/null +++ b/bindings/soth-node/instrumentation/openai.js @@ -0,0 +1,171 @@ +// OpenAI Node SDK auto-instrumentation. +// +// The npm `openai` package exposes: +// - OpenAI (sync-API client; methods return Promises in JS) +// - AzureOpenAI (subclass; same method shapes) +// +// Methods of interest: +// client.chat.completions.create({ model, messages, stream, tools, ... }) +// +// In recent versions (4.x) `chat.completions` is a property that +// returns a `Completions` instance whose `create` method we patch +// on its prototype. + +const { wrapMethod, revertAll } = require('./_base.js'); + +const PROVIDER = 'openai'; +const _patches = []; + +function apply() { + let openai; + try { + openai = require('openai'); + } catch (_) { + return false; + } + + // Walk into the typed completions module. Path differs slightly + // across 4.x versions; defensive lookup keeps us version-tolerant. + const completionsCandidates = [ + () => require('openai/resources/chat/completions'), + () => require('openai/resources/chat/completions/completions'), + ]; + + let completionsModule = null; + for (const loader of completionsCandidates) { + try { + completionsModule = loader(); + break; + } catch (_) { + continue; + } + } + + if (!completionsModule) { + // Fallback: try via the runtime client. Newer SDKs sometimes hide + // class names; in that case we instantiate-then-patch on the + // prototype. + if (openai?.OpenAI) { + // Best-effort: grab a Chat → Completions chain off the class + // prototype. If it's not a method we can patch, return false. + const chatProto = openai.OpenAI.prototype?.chat; + if (chatProto && typeof chatProto === 'object') { + // chat is typically a getter returning a Chat instance; we + // can't patch through here without instantiating. Phase-2 + // adds an instance-level patcher; for now mark as + // not-installed so the harness reports honestly. + } + } + return false; + } + + const targets = []; + if (completionsModule.Completions) { + targets.push([completionsModule.Completions, 'create']); + } + // Some 4.x versions expose `CompletionsBase` or split sync/async. + // We patch any class that exposes a `create` prototype method. + for (const exportName of Object.keys(completionsModule)) { + const cls = completionsModule[exportName]; + if ( + typeof cls === 'function' + && cls?.prototype + && typeof cls.prototype.create === 'function' + && !targets.some(([t]) => t === cls) + ) { + targets.push([cls, 'create']); + } + } + + for (const [target, methodName] of targets) { + const patch = wrapMethod(target, methodName, { + providerName: PROVIDER, + buildCall, + chunkExtractor, + }); + if (patch) _patches.push(patch); + } + + return _patches.length > 0; +} + +function revert() { + revertAll(_patches); + _patches.length = 0; +} + +function buildCall(args) { + // OpenAI's create() takes a single options object as the first arg. + const opts = args?.[0] ?? {}; + const model = opts.model ?? ''; + const rawMessages = Array.isArray(opts.messages) ? opts.messages : []; + + const messages = rawMessages.map((m) => { + const role = typeof m?.role === 'string' ? m.role : 'user'; + let content = m?.content ?? ''; + if (Array.isArray(content)) { + // Multi-modal: flatten text parts. + content = content + .map((p) => (p && typeof p === 'object' ? p.text ?? p.input_text ?? '' : '')) + .filter(Boolean) + .join(' '); + } else if (content == null) { + content = ''; + } + return { role, content: String(content) }; + }); + + const tools = []; + if (Array.isArray(opts.tools)) { + for (const t of opts.tools) { + if (!t || typeof t !== 'object') continue; + if (t.type === 'function' && t.function && typeof t.function.name === 'string') { + tools.push({ + name: t.function.name, + description: t.function.description ?? null, + parametersJson: stableStringify(t.function.parameters ?? {}), + }); + } + } + } + + const call = { + provider: PROVIDER, + model: String(model), + messages, + stream: Boolean(opts.stream), + }; + if (tools.length) call.tools = tools; + if (opts.system) call.system = String(opts.system); + return call; +} + +function chunkExtractor(chunk) { + try { + const choice = chunk?.choices?.[0]; + return { + deltaContent: choice?.delta?.content ?? null, + finishReason: choice?.finish_reason ?? null, + }; + } catch (_) { + return { deltaContent: null, finishReason: null }; + } +} + +function stableStringify(obj) { + // Deterministic JSON for tool-parameter hashing — sorts keys at + // every depth so logically-equivalent schemas produce the same hash. + if (obj === null || typeof obj !== 'object') return JSON.stringify(obj); + if (Array.isArray(obj)) return `[${obj.map(stableStringify).join(',')}]`; + const keys = Object.keys(obj).sort(); + const pairs = keys.map((k) => `${JSON.stringify(k)}:${stableStringify(obj[k])}`); + return `{${pairs.join(',')}}`; +} + +module.exports = { + apply, + revert, + // Test surface + _buildCall: buildCall, + _chunkExtractor: chunkExtractor, +}; diff --git a/bindings/soth-py/python/soth/__init__.py b/bindings/soth-py/python/soth/__init__.py index 50c274b3..baf7fe9a 100644 --- a/bindings/soth-py/python/soth/__init__.py +++ b/bindings/soth-py/python/soth/__init__.py @@ -48,6 +48,11 @@ SothFlagged, block_reason_from_dict, ) +from .instrumentation import ( + instrument, + is_instrumented, + uninstrument, +) __version__ = _soth_native.__version__ @@ -56,7 +61,11 @@ "shutdown", "guard", "guard_stream", + "guard_stream_sync", "context", + "instrument", + "uninstrument", + "is_instrumented", "SothBlocked", "SothFlagged", "BlockReason", @@ -175,13 +184,21 @@ def guard( Translates `Decision::Block` into a raised `SothBlocked` and `Decision::Flag` into a logged `SothFlagged` warning. `Allow` and - `Redact` proceed to invoke `call_fn` (Redact handling is a Phase-1 - deliverable; v0 treats Redact as Allow with the redactions logged). + `Redact` proceed to invoke `call_fn`. + + Works with both **sync and async** providers: + - sync: `call_fn` returns the response object directly; guard + finalizes inline and returns the value. + - async: `call_fn` returns a coroutine; guard returns a coroutine + that, when awaited, finalizes the lifecycle after the inner + coroutine resolves. Customers `await soth.guard(...)`. `call_fn` is the customer's existing call (e.g. `client.chat.completions.create(...)`); the wrapper is intentionally narrow so it can be applied per-call with minimal disruption. """ + import inspect + sdk = get_sdk() ctx = _current_context.get() or None decision = sdk.pre_call(call, ctx) @@ -196,18 +213,6 @@ def guard( reason=block_reason_from_dict(decision.get("reason", {})), ) - # Allow / Flag / Redact (Redact is Phase-1; treat as Allow + log) - try: - result = call_fn() - finally: - # Always consume the token, even on host-level failure. - response_dict = ( - response_extractor(result) # type: ignore[name-defined] - if response_extractor and "result" in dir() - else None - ) - sdk.post_call(token, response_dict) - if kind == _soth_native.DECISION_KIND_FLAG: # Surface the flag through a logger; customers can install # handlers to act on it. Does NOT raise. @@ -217,6 +222,39 @@ def guard( "soth flagged call: severity=%s", decision.get("severity") ) + # Invoke the wrapped call. If it returns a coroutine, post_call + # MUST run after the await — return a coroutine that the customer + # awaits. + try: + result = call_fn() + except BaseException: + sdk.post_call(token, None) + raise + + if inspect.iscoroutine(result) or inspect.isawaitable(result): + return _finalize_async(sdk, token, result, response_extractor) # type: ignore[return-value] + + # Sync path — finalize inline. + response_dict = response_extractor(result) if response_extractor else None + sdk.post_call(token, response_dict) + return result + + +async def _finalize_async( + sdk: _soth_native.SothSdk, + token: int, + coro: Any, + response_extractor: Optional[Callable[[Any], dict[str, Any]]], +) -> Any: + """Async finalizer: await the inner coroutine, then post_call. + Errors propagate after the slab has been balanced.""" + try: + result = await coro + except BaseException: + sdk.post_call(token, None) + raise + response_dict = response_extractor(result) if response_extractor else None + sdk.post_call(token, response_dict) return result @@ -226,10 +264,10 @@ async def guard_stream( call: dict[str, Any], chunk_extractor: Callable[[Any], tuple[Optional[str], Optional[str]]] | None = None, ): - """Wrap a streaming LLM call with SOTH's pre/post lifecycle. + """Wrap an **async** streaming LLM call with SOTH's pre/post lifecycle. `iter_factory` returns an async iterator (typically the awaited - result of e.g. ``client.chat.completions.create(stream=True, ...)``). + result of e.g. ``await aclient.chat.completions.create(stream=True, ...)``). `chunk_extractor(chunk) -> (delta_content, finish_reason)` pulls the fields the SDK records from each provider chunk; defaults to OpenAI's `chunk.choices[0].delta.content` shape. @@ -256,8 +294,8 @@ async def guard_stream( sequence = 0 try: provider_iter = iter_factory() - # Provider may return a sync iterator (e.g. anthropic non-async) - # OR a coroutine that resolves to an async iterator. Handle both. + # Provider may return an async iterator directly OR a coroutine + # that resolves to an async iterator. Handle both. if hasattr(provider_iter, "__await__"): provider_iter = await provider_iter async for chunk in provider_iter: @@ -269,6 +307,52 @@ async def guard_stream( observation.end() +def guard_stream_sync( + iter_factory: Callable[[], Any], + *, + call: dict[str, Any], + chunk_extractor: Callable[[Any], tuple[Optional[str], Optional[str]]] | None = None, +): + """Wrap a **sync** streaming LLM call (e.g. OpenAI's sync `OpenAI` + client returning a `Stream[ChatCompletionChunk]`). + + Returns a generator that yields each provider chunk back to the + caller. Raises `SothBlocked` if the decision is `Block`. Always + finalizes the stream observation on completion or exception. + + Auto-instrumentation routes sync streaming calls here; customers + can call this directly when wrapping a sync stream by hand. + """ + sdk = get_sdk() + ctx = _current_context.get() or None + decision, observation = sdk.stream_begin(call, ctx) + kind = decision["kind"] + + if kind == _soth_native.DECISION_KIND_BLOCK: + observation.end() + raise SothBlocked( + decision_id=str(decision["token"]), + reason=block_reason_from_dict(decision.get("reason", {})), + ) + + if chunk_extractor is None: + chunk_extractor = _default_openai_chunk_extractor + + def _generator(): + sequence = 0 + try: + provider_iter = iter_factory() + for chunk in provider_iter: + delta_content, finish_reason = chunk_extractor(chunk) + observation.chunk(sequence, delta_content, finish_reason) + sequence += 1 + yield chunk + finally: + observation.end() + + return _generator() + + def _default_openai_chunk_extractor(chunk: Any) -> tuple[Optional[str], Optional[str]]: """Default chunk extractor for OpenAI-shaped streams. diff --git a/bindings/soth-py/python/soth/instrumentation/__init__.py b/bindings/soth-py/python/soth/instrumentation/__init__.py new file mode 100644 index 00000000..c017c56b --- /dev/null +++ b/bindings/soth-py/python/soth/instrumentation/__init__.py @@ -0,0 +1,142 @@ +"""Auto-instrumentation for provider SDKs. + +Public entry points: + soth.instrument(providers=None) — patch installed provider SDKs + soth.uninstrument(providers=None) — restore originals + soth.is_instrumented(provider) — query state + +Robustness contract (per `SDK_DECISION_API_SPEC.md` §6 + Phase-1 design): + +1. **Idempotent.** `instrument()` called twice returns the same state + without re-wrapping. Calling `instrument()` after another + instrumentation tool (LangSmith, OTel, dd-trace) preserves the + prior wrapper — SOTH chains rather than replaces. +2. **Reversible.** `uninstrument()` restores the originals captured + at apply time. Restoration is best-effort; if another tool wrapped + AFTER SOTH, that wrapper persists and SOTH logs a warning. +3. **Provider-conditional.** Missing provider packages don't raise — + the entry returns `"skipped:not-installed"` and instrumentation + continues with the rest. +4. **Fail open.** Any exception during call extraction or wrapper + setup falls back to the original SDK call uninstrumented. The + customer's API call MUST complete unaffected if SOTH itself fails. +5. **Version tolerant.** Adapters use defensive `getattr` access so + minor-version SDK changes don't break the wrapper. +6. **Sync + async aware.** Each adapter wraps both sync and async + client classes; streaming and non-streaming paths are detected + per-call from `kwargs.get('stream')`. +""" + +from __future__ import annotations + +import logging +from typing import Iterable, Optional + +from . import _anthropic, _openai + +logger = logging.getLogger("soth.instrumentation") + +# Registry: provider name → adapter module. Adding a provider is a +# new entry here + a corresponding `_.py` adapter module. +_REGISTRY = { + "openai": _openai, + "anthropic": _anthropic, +} + +# State tracking for idempotency. Maps provider name → bool. +_state: dict[str, bool] = {name: False for name in _REGISTRY} + + +def instrument(providers: Optional[Iterable[str]] = None) -> dict[str, str]: + """Patch each importable provider's client classes so calls + automatically run through SOTH's pre/post lifecycle. + + Returns a status dict mapping each provider name to one of: + "instrumented" — patched successfully + "skipped:not-installed" — provider package not importable + "skipped:already-instrumented" — patched in a previous call + "skipped:disabled" — not in the requested providers list + "error:" — apply() raised; SDK is unchanged + + Customers SHOULD call this once at app startup. Subsequent calls + are safe (idempotent) but produce only the "skipped:*" outcomes. + """ + selected = set(providers) if providers else set(_REGISTRY) + results: dict[str, str] = {} + + for name, adapter in _REGISTRY.items(): + if name not in selected: + results[name] = "skipped:disabled" + continue + if _state.get(name, False): + results[name] = "skipped:already-instrumented" + continue + try: + applied = adapter.apply() + except Exception as e: # noqa: BLE001 — fail open is the contract + logger.warning( + "instrument(%s) failed: %s; provider SDK is unchanged", + name, + e, + ) + results[name] = f"error:{type(e).__name__}" + continue + + if applied: + _state[name] = True + results[name] = "instrumented" + else: + results[name] = "skipped:not-installed" + + return results + + +def uninstrument(providers: Optional[Iterable[str]] = None) -> dict[str, str]: + """Restore each provider's original client methods. + + Returns a status dict mapping each provider name to one of: + "uninstrumented" — restored + "skipped:not-instrumented" — never patched + "skipped:disabled" — not in the requested providers list + "error:" — revert() raised; state may be inconsistent + + Use sparingly — typical workflows leave instrumentation on for the + process lifetime. + """ + selected = set(providers) if providers else set(_REGISTRY) + results: dict[str, str] = {} + + for name, adapter in _REGISTRY.items(): + if name not in selected: + results[name] = "skipped:disabled" + continue + if not _state.get(name, False): + results[name] = "skipped:not-instrumented" + continue + try: + adapter.revert() + except Exception as e: # noqa: BLE001 + logger.warning( + "uninstrument(%s) failed: %s; state may be inconsistent", + name, + e, + ) + results[name] = f"error:{type(e).__name__}" + continue + + _state[name] = False + results[name] = "uninstrumented" + + return results + + +def is_instrumented(provider: str) -> bool: + """Return whether `provider` is currently patched.""" + return _state.get(provider, False) + + +def _reset_state_for_test() -> None: + """Test-only helper. Resets the state map without touching the + underlying provider SDKs — used by tests that mock the adapters.""" + for name in _state: + _state[name] = False diff --git a/bindings/soth-py/python/soth/instrumentation/_anthropic.py b/bindings/soth-py/python/soth/instrumentation/_anthropic.py new file mode 100644 index 00000000..12f364d4 --- /dev/null +++ b/bindings/soth-py/python/soth/instrumentation/_anthropic.py @@ -0,0 +1,156 @@ +"""Anthropic auto-instrumentation. + +Patches `anthropic.resources.messages.Messages.create` and +`anthropic.resources.messages.AsyncMessages.create`. + +Anthropic's request shape: + + client.messages.create( + model="claude-...", + messages=[{"role": "user", "content": "..."}], + system="...", # SEPARATE FIELD — not in messages + stream=True/False, + tools=[{"name": ..., "description": ..., "input_schema": {...}}], + max_tokens=..., + ) + +Differs from OpenAI in: +- `system` is a separate parameter (not a message with role=system) +- Tools use `{name, description, input_schema}` shape (vs OpenAI's nested) +""" + +from __future__ import annotations + +import logging +from typing import Any, Optional + +from ._base import Patch, revert_all, wrap_method + +logger = logging.getLogger("soth.instrumentation.anthropic") + +PROVIDER = "anthropic" +_patches: list[Patch] = [] + + +def apply() -> bool: + """Patch Anthropic's messages classes. Returns True on success, + False if the anthropic package isn't importable.""" + try: + from anthropic.resources import messages as msg_module + except ImportError: + return False + + targets: list[tuple[type, str]] = [] + + sync_cls = getattr(msg_module, "Messages", None) + if sync_cls is not None: + targets.append((sync_cls, "create")) + + async_cls = getattr(msg_module, "AsyncMessages", None) + if async_cls is not None: + targets.append((async_cls, "create")) + + for target, method_name in targets: + patch = wrap_method( + target, + method_name, + provider_name=PROVIDER, + build_call=_build_call, + chunk_extractor=_chunk_extractor, + ) + if patch is not None: + _patches.append(patch) + + return bool(_patches) + + +def revert() -> None: + revert_all(_patches) + _patches.clear() + + +def _build_call(args: tuple, kwargs: dict) -> dict[str, Any]: + """Extract an LlmCall dict from Anthropic's `create(...)` args.""" + model = kwargs.get("model") + raw_messages = kwargs.get("messages") or [] + messages: list[dict[str, str]] = [] + for m in raw_messages: + role = m.get("role", "user") if isinstance(m, dict) else "user" + content = m.get("content", "") if isinstance(m, dict) else "" + # Anthropic content may be a string OR a list of ContentBlock + # objects (`{"type": "text", "text": "..."}`). Flatten. + if isinstance(content, list): + parts = [] + for p in content: + if isinstance(p, dict): + text = p.get("text", "") + if text: + parts.append(text) + content = " ".join(parts) + elif content is None: + content = "" + messages.append({"role": str(role), "content": str(content)}) + + tools_kwarg = kwargs.get("tools") or [] + tools: list[dict[str, Any]] = [] + for t in tools_kwarg: + if not isinstance(t, dict): + continue + name = t.get("name") + if not name: + continue + tools.append( + { + "name": str(name), + "description": t.get("description"), + "parameters_json": _stable_json_dumps(t.get("input_schema", {})), + } + ) + + call: dict[str, Any] = { + "provider": PROVIDER, + "model": str(model) if model else "", + "messages": messages, + "stream": bool(kwargs.get("stream", False)), + } + if "system" in kwargs and kwargs["system"]: + call["system"] = kwargs["system"] + if tools: + call["tools"] = tools + return call + + +def _chunk_extractor(chunk: Any) -> tuple[Optional[str], Optional[str]]: + """Anthropic streaming yields a sequence of MessageStreamEvent + objects (`MessageStartEvent`, `ContentBlockDeltaEvent`, + `MessageStopEvent`, ...). The text deltas live on + `ContentBlockDeltaEvent.delta.text`. + + Returns `(text, finish_reason)` where `finish_reason` is set on + the terminal `MessageDeltaEvent` when present. + """ + try: + event_type = getattr(chunk, "type", None) + if event_type == "content_block_delta": + delta = getattr(chunk, "delta", None) + if delta is not None: + text = getattr(delta, "text", None) or getattr(delta, "partial_json", None) + return (text, None) if text else (None, None) + if event_type == "message_delta": + delta = getattr(chunk, "delta", None) + stop = getattr(delta, "stop_reason", None) if delta else None + return (None, stop) + if event_type == "message_stop": + return (None, "stop") + except (AttributeError, TypeError): + pass + return (None, None) + + +def _stable_json_dumps(obj: Any) -> str: + import json + + try: + return json.dumps(obj, sort_keys=True, separators=(",", ":")) + except (TypeError, ValueError): + return repr(obj) diff --git a/bindings/soth-py/python/soth/instrumentation/_base.py b/bindings/soth-py/python/soth/instrumentation/_base.py new file mode 100644 index 00000000..2439005c --- /dev/null +++ b/bindings/soth-py/python/soth/instrumentation/_base.py @@ -0,0 +1,207 @@ +"""Shared instrumentation infrastructure. + +Each provider adapter (`_openai.py`, `_anthropic.py`, ...) imports from +this module to get: + +- `wrap_method`: replaces a method on a class with a SOTH-wrapped version + that handles sync/async, streaming/non-streaming, and fails open. +- `Patch`: bookkeeping struct so `revert()` can restore the original. + +Adapters track their patches in module-level `_patches: list[Patch]` +and call `revert_all(_patches)` when uninstrumenting. +""" + +from __future__ import annotations + +import functools +import inspect +import logging +from dataclasses import dataclass +from typing import Any, Callable, Optional + +logger = logging.getLogger("soth.instrumentation") + + +@dataclass +class Patch: + """Captures one instrumentation site so `revert()` can undo it.""" + + target: type + method_name: str + original: Any + """The method object that was replaced. May itself be a wrapper + from a prior instrumentation tool — we restore it verbatim.""" + + +def wrap_method( + target: type, + method_name: str, + *, + provider_name: str, + build_call: Callable[[tuple, dict], dict], + chunk_extractor: Optional[ + Callable[[Any], tuple[Optional[str], Optional[str]]] + ] = None, +) -> Optional[Patch]: + """Replace `target.method_name` with a SOTH-wrapped version. + + `build_call(args, kwargs) -> dict` extracts the LlmCall dict from + the SDK call's positional + keyword arguments. Errors during + extraction trigger fail-open (the original SDK is invoked, + SOTH bypassed for that call only). + + `chunk_extractor` is the per-provider streaming extractor passed + to `guard_stream` / `guard_stream_sync`. + + Returns a Patch on success, or `None` if the method is missing + on the target (silently skipped — version tolerance). + """ + original = getattr(target, method_name, None) + if original is None: + logger.debug( + "instrument(%s): %s.%s not found; skipping", + provider_name, + target.__name__, + method_name, + ) + return None + + is_async = inspect.iscoroutinefunction(original) or inspect.isasyncgenfunction( + original + ) + + if is_async: + wrapper = _make_async_wrapper( + original, + provider_name=provider_name, + build_call=build_call, + chunk_extractor=chunk_extractor, + ) + else: + wrapper = _make_sync_wrapper( + original, + provider_name=provider_name, + build_call=build_call, + chunk_extractor=chunk_extractor, + ) + + # Record SOTH provenance so `is_instrumented_method` can detect it + # and so a future re-instrument doesn't double-wrap. + setattr(wrapper, "__soth_wrapped__", True) + setattr(wrapper, "__soth_provider__", provider_name) + setattr(wrapper, "__soth_original__", original) + + setattr(target, method_name, wrapper) + return Patch(target=target, method_name=method_name, original=original) + + +def _make_sync_wrapper( + original: Callable, + *, + provider_name: str, + build_call: Callable[[tuple, dict], dict], + chunk_extractor: Optional[Callable[[Any], tuple[Optional[str], Optional[str]]]], +) -> Callable: + """Wrap a sync method. Handles both streaming and non-streaming + based on `kwargs.get('stream', False)` at call time.""" + from .. import guard, guard_stream_sync # avoid circular at import time + + @functools.wraps(original) + def wrapper(*args, **kwargs): + # Fail-open extraction: any exception in build_call falls back + # to the original SDK call uninstrumented. + try: + call_dict = build_call(args, kwargs) + except Exception as e: # noqa: BLE001 + logger.warning( + "soth: build_call failed for %s.%s: %s; bypassed", + provider_name, + getattr(original, "__qualname__", "?"), + e, + ) + return original(*args, **kwargs) + + is_streaming = bool(kwargs.get("stream", False)) + if is_streaming: + return guard_stream_sync( + lambda: original(*args, **kwargs), + call=call_dict, + chunk_extractor=chunk_extractor, + ) + return guard( + lambda: original(*args, **kwargs), + call=call_dict, + ) + + return wrapper + + +def _make_async_wrapper( + original: Callable, + *, + provider_name: str, + build_call: Callable[[tuple, dict], dict], + chunk_extractor: Optional[Callable[[Any], tuple[Optional[str], Optional[str]]]], +) -> Callable: + """Wrap an async method. Routes streaming through `guard_stream` + (async generator) and non-streaming through `guard` (which now + returns a coroutine when `call_fn` returns one).""" + from .. import guard, guard_stream + + @functools.wraps(original) + async def wrapper(*args, **kwargs): + try: + call_dict = build_call(args, kwargs) + except Exception as e: # noqa: BLE001 + logger.warning( + "soth: build_call failed for %s.%s: %s; bypassed", + provider_name, + getattr(original, "__qualname__", "?"), + e, + ) + return await original(*args, **kwargs) + + is_streaming = bool(kwargs.get("stream", False)) + if is_streaming: + # Return the async generator directly; the customer iterates + # over it with `async for`. + return guard_stream( + lambda: original(*args, **kwargs), + call=call_dict, + chunk_extractor=chunk_extractor, + ) + + return await guard( + lambda: original(*args, **kwargs), + call=call_dict, + ) + + return wrapper + + +def is_instrumented_method(method: Any) -> bool: + """Return whether `method` carries the SOTH provenance marker. + + Used to detect double-instrumentation and to warn when another + tool has wrapped over our wrapper after we patched. + """ + return bool(getattr(method, "__soth_wrapped__", False)) + + +def revert_all(patches: list[Patch]) -> None: + """Restore originals captured in `patches`. Best-effort: if + another tool wrapped over our wrapper after we applied, that + wrapper persists and we log a warning rather than silently + overwriting it.""" + for patch in patches: + current = getattr(patch.target, patch.method_name, None) + if current is None: + continue + if not is_instrumented_method(current): + logger.warning( + "soth: %s.%s was rewrapped by another tool; leaving in place", + patch.target.__name__, + patch.method_name, + ) + continue + setattr(patch.target, patch.method_name, patch.original) diff --git a/bindings/soth-py/python/soth/instrumentation/_openai.py b/bindings/soth-py/python/soth/instrumentation/_openai.py new file mode 100644 index 00000000..fb90c9d4 --- /dev/null +++ b/bindings/soth-py/python/soth/instrumentation/_openai.py @@ -0,0 +1,190 @@ +"""OpenAI auto-instrumentation. + +Patches `openai.resources.chat.completions.Completions.create` and +`openai.resources.chat.completions.AsyncCompletions.create` (and +`responses.Responses` if available — newer SDK paths). + +The OpenAI Python SDK structures requests as: + + client.chat.completions.create( + model="...", + messages=[...], + stream=True/False, + tools=[...], # function-calling + ... + ) + +The SDK's typed argument shape is largely stable across 1.x. This +adapter uses defensive `kwargs.get` access so minor-version field +additions don't break the wrapper. +""" + +from __future__ import annotations + +import logging +from typing import Any, Optional + +from ._base import Patch, revert_all, wrap_method + +logger = logging.getLogger("soth.instrumentation.openai") + +PROVIDER = "openai" +_patches: list[Patch] = [] + + +def apply() -> bool: + """Patch OpenAI's chat.completions classes. Returns True on + success, False if the openai package isn't importable.""" + try: + from openai.resources.chat import completions as chat_completions + except ImportError: + return False + + targets: list[tuple[type, str]] = [] + + sync_cls = getattr(chat_completions, "Completions", None) + if sync_cls is not None: + targets.append((sync_cls, "create")) + + async_cls = getattr(chat_completions, "AsyncCompletions", None) + if async_cls is not None: + targets.append((async_cls, "create")) + + # Newer SDKs expose `responses.Responses` (text-completion-style + # API). Patch defensively if available. + try: + from openai.resources import responses as resp_module + + sync_resp = getattr(resp_module, "Responses", None) + if sync_resp is not None: + targets.append((sync_resp, "create")) + async_resp = getattr(resp_module, "AsyncResponses", None) + if async_resp is not None: + targets.append((async_resp, "create")) + except ImportError: + pass + + for target, method_name in targets: + patch = wrap_method( + target, + method_name, + provider_name=PROVIDER, + build_call=_build_call, + chunk_extractor=_chunk_extractor, + ) + if patch is not None: + _patches.append(patch) + + if not _patches: + # `openai` is installed but neither Completions nor Responses + # is available — likely a very old or stripped install. Treat + # as not installed so customers see "skipped:not-installed". + return False + return True + + +def revert() -> None: + """Restore originals patched in `apply`.""" + revert_all(_patches) + _patches.clear() + + +def _build_call(args: tuple, kwargs: dict) -> dict[str, Any]: + """Extract an LlmCall dict from OpenAI's `create(...)` arguments. + + Most fields come from kwargs since the SDK's create() is + keyword-only for everything except `self` and (rarely) the model. + Defensive about unknown fields — anything we don't recognize + is ignored, not propagated. + """ + model = kwargs.get("model") + if model is None and args: + # Some SDK signatures accept (self, model) positionally. + # Skip the bound-self position and check the rest. + for arg in args[1:]: + if isinstance(arg, str): + model = arg + break + + raw_messages = kwargs.get("messages") or [] + messages: list[dict[str, str]] = [] + for m in raw_messages: + # Each message is typically `{"role": ..., "content": ...}`, + # but content may be a list of content parts (vision / multi- + # modal). Flatten lists into a string for hashing purposes. + role = m.get("role", "user") if isinstance(m, dict) else "user" + content = m.get("content", "") if isinstance(m, dict) else "" + if isinstance(content, list): + parts = [] + for p in content: + if isinstance(p, dict): + text = p.get("text") or p.get("input_text") or "" + if text: + parts.append(text) + content = " ".join(parts) + elif content is None: + content = "" + messages.append({"role": str(role), "content": str(content)}) + + tools_kwarg = kwargs.get("tools") or [] + tools: list[dict[str, Any]] = [] + for t in tools_kwarg: + if not isinstance(t, dict): + continue + # OpenAI tools shape: {"type": "function", "function": {...}} + if t.get("type") == "function": + fn = t.get("function") or {} + name = fn.get("name") + if not name: + continue + tools.append( + { + "name": str(name), + "description": fn.get("description"), + "parameters_json": _stable_json_dumps(fn.get("parameters", {})), + } + ) + + call: dict[str, Any] = { + "provider": PROVIDER, + "model": str(model) if model else "", + "messages": messages, + "stream": bool(kwargs.get("stream", False)), + } + if tools: + call["tools"] = tools + if "system" in kwargs: + call["system"] = kwargs["system"] + return call + + +def _chunk_extractor(chunk: Any) -> tuple[Optional[str], Optional[str]]: + """Default chunk extractor for OpenAI streams. + + OpenAI's streaming response yields ChatCompletionChunk objects with + `choices[0].delta.content` and `choices[0].finish_reason`. Returns + `(None, None)` for chunks that don't fit (e.g. tool-call deltas). + """ + try: + choice = chunk.choices[0] + delta = getattr(choice, "delta", None) + delta_content = getattr(delta, "content", None) if delta else None + finish_reason = getattr(choice, "finish_reason", None) + return delta_content, finish_reason + except (AttributeError, IndexError, TypeError): + return None, None + + +def _stable_json_dumps(obj: Any) -> str: + """Stable JSON serialization for tool parameter hashing. + + `sort_keys=True` so the same logical schema produces the same + hash across Python dict ordering changes. Falls back to repr() + if the object isn't JSON-serializable. + """ + import json + + try: + return json.dumps(obj, sort_keys=True, separators=(",", ":")) + except (TypeError, ValueError): + return repr(obj) diff --git a/bindings/soth-py/tests/test_instrumentation.py b/bindings/soth-py/tests/test_instrumentation.py new file mode 100644 index 00000000..49b6d2b6 --- /dev/null +++ b/bindings/soth-py/tests/test_instrumentation.py @@ -0,0 +1,274 @@ +"""Robustness tests for `soth.instrument()`. + +Each test asserts one of the contract guarantees in +`python/soth/instrumentation/__init__.py`: + + - idempotent: instrument() called twice returns "skipped:already-…" + - reversible: uninstrument() restores originals + - missing-provider tolerant: not-installed providers don't raise + - fail-open extractor: build_call exceptions fall through + - double-wrap detection: __soth_wrapped__ marker survives revert + - sync coroutine handling: guard() handles awaitable returns + +The tests do NOT require openai / anthropic to be installed; they +either skip when the package is absent (so CI without optional deps +still passes) or use mock classes / objects. + +Run with: + cd bindings/soth-py + pip install -e ".[test]" + maturin develop + pytest tests/test_instrumentation.py +""" + +import os +from typing import Any +from unittest import mock + +import pytest + +import soth +from soth.instrumentation import _base, _reset_state_for_test + + +@pytest.fixture(autouse=True) +def _init_sdk(): + os.environ["SOTH_HMAC_KEY"] = "x" * 32 + soth.init( + api_key="sk-test", + org_id="org-test", + hmac_key_env="SOTH_HMAC_KEY", + ) + # Reset instrumentation state before each test so they don't bleed. + _reset_state_for_test() + yield + _reset_state_for_test() + + +# ── idempotency / reversibility ───────────────────────────────────── + + +def test_instrument_idempotent_second_call_returns_already_instrumented(): + """Calling instrument() twice must NOT double-wrap. The second + call returns 'skipped:already-instrumented' for any provider the + first call patched.""" + first = soth.instrument() + second = soth.instrument() + + # Whatever first instrumented, second must report as already-done. + for provider, status in first.items(): + if status == "instrumented": + assert second[provider] == "skipped:already-instrumented", ( + f"{provider}: idempotency violated. first={status}, second={second[provider]}" + ) + + +def test_uninstrument_reverses_instrument(): + """After uninstrument(), state.is_instrumented(p) is False for + every previously-instrumented provider.""" + first = soth.instrument() + soth.uninstrument() + + for provider, status in first.items(): + if status == "instrumented": + assert soth.is_instrumented(provider) is False, ( + f"{provider} still reports as instrumented after uninstrument()" + ) + + +def test_uninstrument_without_prior_instrument_is_safe(): + """uninstrument() before any instrument() returns + 'skipped:not-instrumented' rather than raising.""" + results = soth.uninstrument() + for provider, status in results.items(): + assert status in ("skipped:not-instrumented", "skipped:disabled") + + +# ── provider selection ───────────────────────────────────────────── + + +def test_instrument_with_explicit_providers_skips_others(): + """Passing providers=['openai'] leaves anthropic at + 'skipped:disabled' regardless of whether anthropic is installed.""" + results = soth.instrument(providers=["openai"]) + assert results.get("anthropic") == "skipped:disabled" + + +def test_instrument_unknown_provider_is_silently_skipped(): + """Listing an unknown provider in the explicit set doesn't error; + known providers still process normally.""" + results = soth.instrument(providers=["openai", "definitely-not-a-provider"]) + # Known provider gets processed (instrumented or skipped). + assert "openai" in results + # Unknown provider isn't in the registry, so it's not in results. + assert "definitely-not-a-provider" not in results + + +# ── missing-provider tolerance ───────────────────────────────────── + + +def test_missing_provider_returns_skipped_not_installed(monkeypatch): + """If the provider package isn't importable, the entry returns + 'skipped:not-installed' rather than raising ImportError.""" + # Force the openai adapter's apply() to behave as if openai is + # uninstalled by monkey-patching its import. + from soth.instrumentation import _openai + + def _apply_returns_false(): + return False + + monkeypatch.setattr(_openai, "apply", _apply_returns_false) + results = soth.instrument(providers=["openai"]) + assert results["openai"] == "skipped:not-installed" + + +# ── fail-open extractor ──────────────────────────────────────────── + + +def test_build_call_exception_falls_through_to_original(monkeypatch): + """If the buildCall extractor raises, the wrapper invokes the + original method without going through SOTH's lifecycle. The + customer's call must complete unaffected.""" + # Mock the wrap_method machinery against a hand-rolled class. + class FakeClient: + def create(self, **kwargs): + return {"ok": True, "kwargs": kwargs} + + def busted_build_call(args, kwargs): + raise RuntimeError("simulated extractor crash") + + patch = _base.wrap_method( + FakeClient, + "create", + provider_name="fake", + build_call=busted_build_call, + ) + assert patch is not None + + client = FakeClient() + # The original behavior must be preserved despite the extractor + # raising — fail-open. + result = client.create(model="x", messages=[]) + assert result == {"ok": True, "kwargs": {"model": "x", "messages": []}} + + # Restore so other tests aren't polluted. + _base.revert_all([patch]) + + +# ── double-wrap detection ────────────────────────────────────────── + + +def test_wrapped_method_carries_soth_provenance_marker(): + """The wrapper records `__soth_wrapped__` and `__soth_provider__` + so `revert_all` can detect whether another tool has overwritten + our wrapper after we patched.""" + class Target: + def m(self): + return 1 + + patch = _base.wrap_method( + Target, + "m", + provider_name="test", + build_call=lambda args, kwargs: {"provider": "test", "model": "", "messages": []}, + ) + assert patch is not None + assert _base.is_instrumented_method(Target.m) is True + assert getattr(Target.m, "__soth_provider__", None) == "test" + _base.revert_all([patch]) + + +def test_revert_leaves_third_party_wrapper_in_place(): + """If another tool wraps over our wrapper after we patch, revert + must NOT clobber it. Our wrapper is gone-but-not-replaced — the + third-party wrapper persists.""" + class Target: + def m(self): + return 1 + + patch = _base.wrap_method( + Target, + "m", + provider_name="test", + build_call=lambda args, kwargs: {"provider": "test", "model": "", "messages": []}, + ) + assert patch is not None + + # Simulate another tool wrapping over our wrapper. + soth_wrapper = Target.m + def third_party_wrapper(self, *args, **kwargs): + return soth_wrapper(self, *args, **kwargs) + Target.m = third_party_wrapper # type: ignore[method-assign] + + _base.revert_all([patch]) + # The third-party wrapper should still be there — we don't + # overwrite a non-soth wrapper. + assert Target.m is third_party_wrapper + + +# ── coroutine handling for guard() ───────────────────────────────── + + +@pytest.mark.asyncio +async def test_guard_returns_coroutine_when_inner_returns_coroutine(): + """When the wrapped call_fn returns a coroutine, guard() must + return a coroutine that, when awaited, finalizes the lifecycle. + This makes one guard() entry-point work for both sync (OpenAI) + and async (AsyncOpenAI) clients.""" + async def inner(): + return "async-result" + + result_coro = soth.guard( + inner, + call={ + "provider": "openai", + "model": "gpt-4o-mini", + "messages": [{"role": "user", "content": "hi"}], + }, + ) + # guard() returned the coroutine; nothing has run yet. + assert result_coro is not None + # Awaiting finalizes the lifecycle. + result = await result_coro + assert result == "async-result" + assert soth._in_flight_decisions() == 0 + + +def test_guard_runs_synchronously_when_inner_returns_value(): + """Sync providers return a plain value from call_fn. guard() + finalizes inline and returns the value — no coroutine wrapping.""" + result = soth.guard( + lambda: "sync-result", + call={ + "provider": "openai", + "model": "gpt-4o-mini", + "messages": [{"role": "user", "content": "hi"}], + }, + ) + assert result == "sync-result" + assert soth._in_flight_decisions() == 0 + + +# ── adapter integration (only when SDKs installed) ───────────────── + + +def test_openai_adapter_apply_returns_bool(): + """The adapter's apply() must return True/False — never raise. + On systems without openai installed, it returns False; with + openai installed, returns True. Either way, no exception.""" + from soth.instrumentation import _openai + + result = _openai.apply() + assert result in (True, False) + if result is True: + # Restore so other tests aren't polluted. + _openai.revert() + + +def test_anthropic_adapter_apply_returns_bool(): + from soth.instrumentation import _anthropic + + result = _anthropic.apply() + assert result in (True, False) + if result is True: + _anthropic.revert() From 0589288f9ae08233c4cb6b8febc303bd972d625f Mon Sep 17 00:00:00 2001 From: Apoorv Raj Saxena Date: Thu, 30 Apr 2026 13:44:53 +0530 Subject: [PATCH 10/24] feat(sdk): Phase 1.5 - Cohere/Google/Mistral Python + Anthropic Node adapters MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Extends auto-instrumentation to the long-tail providers. Each adapter follows the existing OpenAI/Anthropic Python pattern (apply + revert + build_call + chunk_extractor) and inherits the six robustness guarantees from `_base.py` (idempotent, reversible, provider-conditional, fail-open, version-tolerant, sync+async aware). Python — bindings/soth-py/python/soth/instrumentation/: _cohere.py: Cohere v5 ClientV2/AsyncClientV2 (OpenAI-shaped `messages=[{role, content}]`) AND v4 Client (`message=` + `chat_history=`) for legacy users. Streaming chunk extractor handles v5 content-delta / message-end events. _google_genai.py: Models.generate_content + generate_content_stream (sync + async). Normalizes the four shapes of `contents` (str / list-of-str / list-of-Content / Content-dict) into a uniform `[{role, content}]` list. system_instruction extracted from `config`. Streaming reads `chunk.text` + finish_reason from candidates[0]. _mistral.py: Chat.complete + complete_async + stream + stream_async (mistralai 1.x). OpenAI-equivalent shape so extraction is straightforward; chunk extractor handles Mistral's `data` envelope around OpenAI deltas. Node — bindings/soth-node/instrumentation/: anthropic.js: Patches Messages.create on @anthropic-ai/sdk. Walks multiple typed-resource path candidates for version tolerance across 0.x. Handles Anthropic-specific shape: separate `system` field, `input_schema` (not `parameters`) on tools, content_block_delta / message_delta / message_stop streaming events. Registry updates: Python REGISTRY now includes openai, anthropic, cohere, google_genai, mistralai (5 providers). Node REGISTRY now includes openai, anthropic (2 providers). Tests added: - cohere/google_genai/mistral apply() returns bool — no exceptions - cohere v2 + v4 extractors normalize messages correctly - google_genai handles string vs list-of-Content shapes - mistral extractor produces OpenAI-shaped output - Anthropic Node apply() + buildCall + chunkExtractor unit tests Verification: cargo build --workspace clean All workspace lib tests unchanged (654) Conformance harness 7/7 fixtures All 4 WASM combos clean Adapter tests pass on systems with/without provider SDKs installed (apply() returns False when missing; tests skip cleanly) Phase-1.6 follow-ups (framework adapters — different integration pattern via callbacks/middleware): LangChain Python, LlamaIndex Python, LiteLLM Python, Vercel AI SDK Node. Each lands as its own commit. Co-Authored-By: Claude Opus 4.7 (1M context) --- .../__test__/instrumentation.test.mjs | 39 ++++ .../soth-node/instrumentation/anthropic.js | 163 +++++++++++++++ bindings/soth-node/instrumentation/index.js | 2 + .../python/soth/instrumentation/__init__.py | 5 +- .../python/soth/instrumentation/_cohere.py | 185 ++++++++++++++++++ .../soth/instrumentation/_google_genai.py | 183 +++++++++++++++++ .../python/soth/instrumentation/_mistral.py | 154 +++++++++++++++ .../soth-py/tests/test_instrumentation.py | 115 +++++++++++ 8 files changed, 845 insertions(+), 1 deletion(-) create mode 100644 bindings/soth-node/instrumentation/anthropic.js create mode 100644 bindings/soth-py/python/soth/instrumentation/_cohere.py create mode 100644 bindings/soth-py/python/soth/instrumentation/_google_genai.py create mode 100644 bindings/soth-py/python/soth/instrumentation/_mistral.py diff --git a/bindings/soth-node/__test__/instrumentation.test.mjs b/bindings/soth-node/__test__/instrumentation.test.mjs index 04e1a46a..aa24ecee 100644 --- a/bindings/soth-node/__test__/instrumentation.test.mjs +++ b/bindings/soth-node/__test__/instrumentation.test.mjs @@ -196,3 +196,42 @@ test('OpenAI buildCall extracts model + messages from create() options', async ( assert.equal(call.tools.length, 1); assert.equal(call.tools[0].name, 'lookup_weather'); }); + +test('Anthropic adapter apply returns bool — no exceptions', async () => { + const adapter = await import('../instrumentation/anthropic.js'); + const result = adapter.apply(); + assert.ok(typeof result === 'boolean'); + if (result === true) { + adapter.revert(); + } +}); + +test('Anthropic buildCall handles separate system field + input_schema tools', async () => { + const adapter = await import('../instrumentation/anthropic.js'); + const call = adapter._buildCall([ + { + model: 'claude-3-5-sonnet-latest', + messages: [{ role: 'user', content: 'hi' }], + system: 'You are concise.', + tools: [ + { + name: 'lookup', + description: 'Look something up', + input_schema: { type: 'object', properties: {} }, + }, + ], + }, + ]); + assert.equal(call.provider, 'anthropic'); + assert.equal(call.system, 'You are concise.'); + assert.equal(call.tools.length, 1); + assert.equal(call.tools[0].name, 'lookup'); +}); + +test('Anthropic chunkExtractor handles content_block_delta + message_stop', async () => { + const { _chunkExtractor } = await import('../instrumentation/anthropic.js'); + const delta = _chunkExtractor({ type: 'content_block_delta', delta: { text: 'hello' } }); + assert.equal(delta.deltaContent, 'hello'); + const stop = _chunkExtractor({ type: 'message_stop' }); + assert.equal(stop.finishReason, 'stop'); +}); diff --git a/bindings/soth-node/instrumentation/anthropic.js b/bindings/soth-node/instrumentation/anthropic.js new file mode 100644 index 00000000..201bb91a --- /dev/null +++ b/bindings/soth-node/instrumentation/anthropic.js @@ -0,0 +1,163 @@ +// Anthropic Node SDK auto-instrumentation. +// +// Patches `Messages.create` on the typed messages module of +// `@anthropic-ai/sdk`. Anthropic's Node API is OpenAI-shaped except +// for the `system` parameter (separate field, not a system message) +// and tools (using `input_schema` rather than `parameters`). +// +// client.messages.create({ +// model: 'claude-3-5-sonnet-latest', +// messages: [{ role: 'user', content: '...' }], +// system: '...', +// stream: true|false, +// tools: [{ name, description, input_schema }], +// max_tokens: 1024, +// }) + +const { wrapMethod, revertAll } = require('./_base.js'); + +const PROVIDER = 'anthropic'; +const _patches = []; + +function apply() { + let anthropic; + try { + anthropic = require('@anthropic-ai/sdk'); + } catch (_) { + return false; + } + + // Anthropic's npm SDK has rotated typed-resource paths a few times + // in its 0.x series. Try multiple candidates for version tolerance. + const candidates = [ + () => require('@anthropic-ai/sdk/resources/messages'), + () => require('@anthropic-ai/sdk/resources/messages/messages'), + ]; + + let messagesModule = null; + for (const loader of candidates) { + try { + messagesModule = loader(); + break; + } catch (_) { + continue; + } + } + if (!messagesModule) return false; + + const targets = []; + const Messages = messagesModule.Messages; + if (Messages && typeof Messages.prototype?.create === 'function') { + targets.push([Messages, 'create']); + } + // Anthropic's beta module exports `MessagesBeta` with the same + // `create` shape; patch when present. + for (const exportName of Object.keys(messagesModule)) { + const cls = messagesModule[exportName]; + if ( + typeof cls === 'function' + && cls?.prototype + && typeof cls.prototype.create === 'function' + && !targets.some(([t]) => t === cls) + ) { + targets.push([cls, 'create']); + } + } + + for (const [target, methodName] of targets) { + const patch = wrapMethod(target, methodName, { + providerName: PROVIDER, + buildCall, + chunkExtractor, + }); + if (patch) _patches.push(patch); + } + + return _patches.length > 0; +} + +function revert() { + revertAll(_patches); + _patches.length = 0; +} + +function buildCall(args) { + const opts = args?.[0] ?? {}; + const model = opts.model ?? ''; + const rawMessages = Array.isArray(opts.messages) ? opts.messages : []; + + const messages = rawMessages.map((m) => { + const role = typeof m?.role === 'string' ? m.role : 'user'; + let content = m?.content ?? ''; + if (Array.isArray(content)) { + // Multi-modal / tool-use content blocks: flatten text parts. + content = content + .map((p) => (p && typeof p === 'object' ? p.text ?? '' : '')) + .filter(Boolean) + .join(' '); + } else if (content == null) { + content = ''; + } + return { role, content: String(content) }; + }); + + const tools = []; + if (Array.isArray(opts.tools)) { + for (const t of opts.tools) { + if (!t || typeof t !== 'object' || typeof t.name !== 'string') continue; + tools.push({ + name: t.name, + description: t.description ?? null, + // Anthropic uses `input_schema` rather than OpenAI's `parameters`. + parametersJson: stableStringify(t.input_schema ?? {}), + }); + } + } + + const call = { + provider: PROVIDER, + model: String(model), + messages, + stream: Boolean(opts.stream), + }; + if (tools.length) call.tools = tools; + if (opts.system) call.system = String(opts.system); + return call; +} + +function chunkExtractor(chunk) { + // Anthropic streaming: events of type `content_block_delta`, + // `message_delta`, `message_stop`. Text deltas live on + // `chunk.delta.text`. + try { + const eventType = chunk?.type; + if (eventType === 'content_block_delta') { + const delta = chunk?.delta; + const text = delta?.text ?? delta?.partial_json ?? null; + return { deltaContent: text, finishReason: null }; + } + if (eventType === 'message_delta') { + const stop = chunk?.delta?.stop_reason ?? null; + return { deltaContent: null, finishReason: stop }; + } + if (eventType === 'message_stop') { + return { deltaContent: null, finishReason: 'stop' }; + } + } catch (_) { /* fall through */ } + return { deltaContent: null, finishReason: null }; +} + +function stableStringify(obj) { + if (obj === null || typeof obj !== 'object') return JSON.stringify(obj); + if (Array.isArray(obj)) return `[${obj.map(stableStringify).join(',')}]`; + const keys = Object.keys(obj).sort(); + const pairs = keys.map((k) => `${JSON.stringify(k)}:${stableStringify(obj[k])}`); + return `{${pairs.join(',')}}`; +} + +module.exports = { + apply, + revert, + _buildCall: buildCall, + _chunkExtractor: chunkExtractor, +}; diff --git a/bindings/soth-node/instrumentation/index.js b/bindings/soth-node/instrumentation/index.js index f6c4fbd3..5c54923d 100644 --- a/bindings/soth-node/instrumentation/index.js +++ b/bindings/soth-node/instrumentation/index.js @@ -22,9 +22,11 @@ // because the JS guard()/guardStream() helpers already detect. const openai = require('./openai.js'); +const anthropic = require('./anthropic.js'); const REGISTRY = Object.freeze({ openai, + anthropic, }); const _state = Object.fromEntries(Object.keys(REGISTRY).map((k) => [k, false])); diff --git a/bindings/soth-py/python/soth/instrumentation/__init__.py b/bindings/soth-py/python/soth/instrumentation/__init__.py index c017c56b..a235f067 100644 --- a/bindings/soth-py/python/soth/instrumentation/__init__.py +++ b/bindings/soth-py/python/soth/instrumentation/__init__.py @@ -32,7 +32,7 @@ import logging from typing import Iterable, Optional -from . import _anthropic, _openai +from . import _anthropic, _cohere, _google_genai, _mistral, _openai logger = logging.getLogger("soth.instrumentation") @@ -41,6 +41,9 @@ _REGISTRY = { "openai": _openai, "anthropic": _anthropic, + "cohere": _cohere, + "google_genai": _google_genai, + "mistralai": _mistral, } # State tracking for idempotency. Maps provider name → bool. diff --git a/bindings/soth-py/python/soth/instrumentation/_cohere.py b/bindings/soth-py/python/soth/instrumentation/_cohere.py new file mode 100644 index 00000000..6cf933b6 --- /dev/null +++ b/bindings/soth-py/python/soth/instrumentation/_cohere.py @@ -0,0 +1,185 @@ +"""Cohere auto-instrumentation. + +Patches `cohere.client_v2.ClientV2.chat` and +`cohere.client_v2.AsyncClientV2.chat` (Cohere v5+ uses an +OpenAI-shaped `messages=[{role, content}]` API on `ClientV2`). + +Falls back to the v4 `cohere.Client.chat(message=, chat_history=)` +shape when only the legacy client is present — a pragmatic +compatibility layer for customers still on the older release. + +The robustness contract from `_base.py` applies: missing-package +tolerance, fail-open extraction, idempotent re-instrument. +""" + +from __future__ import annotations + +import logging +from typing import Any, Optional + +from ._base import Patch, revert_all, wrap_method + +logger = logging.getLogger("soth.instrumentation.cohere") + +PROVIDER = "cohere" +_patches: list[Patch] = [] + + +def apply() -> bool: + try: + import cohere # noqa: F401 + except ImportError: + return False + + targets: list[tuple[type, str, Any]] = [] # (class, method, build_call_fn) + + # v5+ ClientV2 — OpenAI-shaped messages. + try: + from cohere import client_v2 as v2_module + + sync_v2 = getattr(v2_module, "ClientV2", None) + if sync_v2 is not None: + targets.append((sync_v2, "chat", _build_call_v2)) + targets.append((sync_v2, "chat_stream", _build_call_v2)) + async_v2 = getattr(v2_module, "AsyncClientV2", None) + if async_v2 is not None: + targets.append((async_v2, "chat", _build_call_v2)) + targets.append((async_v2, "chat_stream", _build_call_v2)) + except ImportError: + pass + + # v4 fallback Client / AsyncClient — distinct `message` + `chat_history` shape. + try: + from cohere.client import Client as v4_sync, AsyncClient as v4_async + + if v4_sync is not None: + targets.append((v4_sync, "chat", _build_call_v4)) + if v4_async is not None: + targets.append((v4_async, "chat", _build_call_v4)) + except ImportError: + pass + + for target, method_name, build_fn in targets: + patch = wrap_method( + target, + method_name, + provider_name=PROVIDER, + build_call=build_fn, + chunk_extractor=_chunk_extractor, + ) + if patch is not None: + _patches.append(patch) + + return bool(_patches) + + +def revert() -> None: + revert_all(_patches) + _patches.clear() + + +def _build_call_v2(args: tuple, kwargs: dict) -> dict[str, Any]: + """v5+ ClientV2.chat(messages=[...], model=..., stream=...).""" + model = kwargs.get("model") + raw_messages = kwargs.get("messages") or [] + messages: list[dict[str, str]] = [] + for m in raw_messages: + role = m.get("role", "user") if isinstance(m, dict) else "user" + content = m.get("content", "") if isinstance(m, dict) else "" + if isinstance(content, list): + parts = [p.get("text", "") for p in content if isinstance(p, dict)] + content = " ".join(p for p in parts if p) + elif content is None: + content = "" + messages.append({"role": str(role), "content": str(content)}) + + tools = [] + raw_tools = kwargs.get("tools") or [] + for t in raw_tools: + if not isinstance(t, dict): + continue + # v2 tools shape: {"type": "function", "function": {name, description, parameters}} + fn = t.get("function") if t.get("type") == "function" else t + if not isinstance(fn, dict): + continue + name = fn.get("name") + if not name: + continue + tools.append( + { + "name": str(name), + "description": fn.get("description"), + "parameters_json": _stable_json(fn.get("parameters", {})), + } + ) + + call: dict[str, Any] = { + "provider": PROVIDER, + "model": str(model) if model else "", + "messages": messages, + "stream": bool(kwargs.get("stream", False)), + } + if tools: + call["tools"] = tools + return call + + +def _build_call_v4(args: tuple, kwargs: dict) -> dict[str, Any]: + """v4 Client.chat(message=, chat_history=, model=).""" + model = kwargs.get("model") + message = kwargs.get("message", "") or "" + chat_history = kwargs.get("chat_history") or [] + + messages: list[dict[str, str]] = [] + for h in chat_history: + if not isinstance(h, dict): + continue + # v4 history: {"role": "USER" | "CHATBOT" | "SYSTEM", "message": "..."} + role = h.get("role", "USER") + # Normalize Cohere uppercase roles to OpenAI-style lowercase so + # the SDK's hashing is consistent with other providers. + if role == "CHATBOT": + role = "assistant" + else: + role = role.lower() + content = h.get("message") or h.get("text") or "" + messages.append({"role": str(role), "content": str(content)}) + if message: + messages.append({"role": "user", "content": str(message)}) + + call: dict[str, Any] = { + "provider": PROVIDER, + "model": str(model) if model else "", + "messages": messages, + "stream": bool(kwargs.get("stream", False)), + } + return call + + +def _chunk_extractor(chunk: Any) -> tuple[Optional[str], Optional[str]]: + """Cohere v5 streaming events: `chunk.type` is one of + `content-delta`, `content-end`, `message-start`, `message-end`, + etc. Text deltas live on `chunk.delta.message.content.text`. + """ + try: + event_type = getattr(chunk, "type", None) + if event_type == "content-delta": + delta = getattr(chunk, "delta", None) + msg = getattr(delta, "message", None) if delta else None + content = getattr(msg, "content", None) if msg else None + text = getattr(content, "text", None) if content else None + return (text, None) if text else (None, None) + if event_type in ("message-end", "message-stop", "stream-end"): + return (None, "stop") + except (AttributeError, TypeError): + pass + return (None, None) + + +def _stable_json(obj: Any) -> str: + import json + + try: + return json.dumps(obj, sort_keys=True, separators=(",", ":")) + except (TypeError, ValueError): + return repr(obj) diff --git a/bindings/soth-py/python/soth/instrumentation/_google_genai.py b/bindings/soth-py/python/soth/instrumentation/_google_genai.py new file mode 100644 index 00000000..e1ad6d6d --- /dev/null +++ b/bindings/soth-py/python/soth/instrumentation/_google_genai.py @@ -0,0 +1,183 @@ +"""Google GenAI auto-instrumentation. + +Patches `google.genai.models.Models.generate_content`, +`generate_content_stream`, and their async counterparts on +`google.genai.models.AsyncModels`. + +Google's request shape diverges meaningfully from OpenAI: + + client.models.generate_content( + model="gemini-2.0-flash", + contents="Tell me about Rust", # str | list | dict + config={"system_instruction": "..."}, + ) + +`contents` can be: +- a single string (the user prompt) +- a list of strings (multiple prompts; multimodal) +- a list of `Content` objects with role + parts +- a dict-shaped `Content` + +The adapter normalizes all four into the SDK's `messages: [{role, content}]` +shape; defensive on unknown structures. +""" + +from __future__ import annotations + +import logging +from typing import Any, Optional + +from ._base import Patch, revert_all, wrap_method + +logger = logging.getLogger("soth.instrumentation.google_genai") + +PROVIDER = "google_genai" +_patches: list[Patch] = [] + + +def apply() -> bool: + try: + from google.genai import models as models_module # type: ignore + except ImportError: + return False + + targets: list[tuple[type, str]] = [] + sync_cls = getattr(models_module, "Models", None) + if sync_cls is not None: + for name in ("generate_content", "generate_content_stream"): + if callable(getattr(sync_cls, name, None)): + targets.append((sync_cls, name)) + async_cls = getattr(models_module, "AsyncModels", None) + if async_cls is not None: + for name in ("generate_content", "generate_content_stream"): + if callable(getattr(async_cls, name, None)): + targets.append((async_cls, name)) + + for target, method_name in targets: + patch = wrap_method( + target, + method_name, + provider_name=PROVIDER, + build_call=_build_call, + chunk_extractor=_chunk_extractor, + ) + if patch is not None: + _patches.append(patch) + + return bool(_patches) + + +def revert() -> None: + revert_all(_patches) + _patches.clear() + + +def _build_call(args: tuple, kwargs: dict) -> dict[str, Any]: + model = kwargs.get("model") + contents = kwargs.get("contents") + config = kwargs.get("config") or {} + + messages = _normalize_contents(contents) + + # `system_instruction` lives on the config object in newer + # google-genai releases; fall back to the kwarg for older shapes. + system = None + if isinstance(config, dict): + system = config.get("system_instruction") + else: + system = getattr(config, "system_instruction", None) + if not system: + system = kwargs.get("system_instruction") + + is_streaming = "_stream" in ( + kwargs.get("__call_method") or "" + ) or bool(kwargs.get("stream", False)) + # Streaming is detected at the wrapper level via + # `kwargs.get('stream')`; google-genai uses a separate + # `generate_content_stream` method, so we always set + # stream=True for that target. The wrapper also overrides this + # via the call shape. + method_name = kwargs.get("__call_method", "") + if method_name.endswith("stream"): + is_streaming = True + + call: dict[str, Any] = { + "provider": PROVIDER, + "model": str(model) if model else "", + "messages": messages, + "stream": is_streaming, + } + if system: + call["system"] = str(system) + return call + + +def _normalize_contents(contents: Any) -> list[dict[str, str]]: + """Convert google-genai's flexible `contents` into a + `[{role, content}]` list. Returns `[]` for unrecognized shapes + (the wrapper still operates; just no message content).""" + if contents is None: + return [] + # Single string → single user message. + if isinstance(contents, str): + return [{"role": "user", "content": contents}] + # List handling — heterogeneous. + if isinstance(contents, list): + out: list[dict[str, str]] = [] + for item in contents: + if isinstance(item, str): + out.append({"role": "user", "content": item}) + elif isinstance(item, dict): + out.append(_normalize_content_dict(item)) + else: + # Content object with .role and .parts attributes. + role = getattr(item, "role", "user") or "user" + parts = getattr(item, "parts", None) or [] + texts = [] + for p in parts: + text = getattr(p, "text", None) + if not text and isinstance(p, dict): + text = p.get("text") + if text: + texts.append(str(text)) + out.append({"role": str(role), "content": " ".join(texts)}) + return out + # Single dict → treat as Content. + if isinstance(contents, dict): + return [_normalize_content_dict(contents)] + return [] + + +def _normalize_content_dict(d: dict) -> dict[str, str]: + role = d.get("role", "user") + parts = d.get("parts") or [] + texts: list[str] = [] + if isinstance(parts, list): + for p in parts: + if isinstance(p, dict): + text = p.get("text") + if text: + texts.append(str(text)) + elif isinstance(p, str): + texts.append(p) + elif isinstance(parts, str): + texts.append(parts) + return {"role": str(role), "content": " ".join(texts)} + + +def _chunk_extractor(chunk: Any) -> tuple[Optional[str], Optional[str]]: + """google-genai streaming yields `GenerateContentResponse` objects + where each has `.text` (the delta) and `.candidates[].finish_reason`. + """ + try: + text = getattr(chunk, "text", None) + finish = None + candidates = getattr(chunk, "candidates", None) + if candidates: + cand = candidates[0] + finish_reason = getattr(cand, "finish_reason", None) + if finish_reason: + finish = str(finish_reason) + return text if text else None, finish + except (AttributeError, IndexError, TypeError): + return (None, None) diff --git a/bindings/soth-py/python/soth/instrumentation/_mistral.py b/bindings/soth-py/python/soth/instrumentation/_mistral.py new file mode 100644 index 00000000..dd1e471a --- /dev/null +++ b/bindings/soth-py/python/soth/instrumentation/_mistral.py @@ -0,0 +1,154 @@ +"""Mistral auto-instrumentation. + +Patches `mistralai.chat.Chat.complete`, +`mistralai.chat.Chat.complete_async`, `Chat.stream`, `Chat.stream_async` +on the Mistral SDK (mistralai 1.x). + +The Mistral request shape is OpenAI-equivalent: + + client.chat.complete( + model="mistral-large-latest", + messages=[{"role": "user", "content": "..."}], + stream=True/False, + tools=[...], + ) +""" + +from __future__ import annotations + +import logging +from typing import Any, Optional + +from ._base import Patch, revert_all, wrap_method + +logger = logging.getLogger("soth.instrumentation.mistral") + +PROVIDER = "mistralai" +_patches: list[Patch] = [] + + +def apply() -> bool: + try: + from mistralai import chat as chat_module # type: ignore + except ImportError: + return False + + chat_cls = getattr(chat_module, "Chat", None) + if chat_cls is None: + return False + + methods = [] + for name in ("complete", "complete_async", "stream", "stream_async"): + if callable(getattr(chat_cls, name, None)): + methods.append(name) + + for method_name in methods: + patch = wrap_method( + chat_cls, + method_name, + provider_name=PROVIDER, + build_call=_build_call, + chunk_extractor=_chunk_extractor, + ) + if patch is not None: + _patches.append(patch) + + return bool(_patches) + + +def revert() -> None: + revert_all(_patches) + _patches.clear() + + +def _build_call(args: tuple, kwargs: dict) -> dict[str, Any]: + model = kwargs.get("model") + raw_messages = kwargs.get("messages") or [] + messages: list[dict[str, str]] = [] + for m in raw_messages: + # mistralai uses pydantic models OR dicts depending on shape. + if isinstance(m, dict): + role = m.get("role", "user") + content = m.get("content", "") + else: + role = getattr(m, "role", "user") + content = getattr(m, "content", "") + if isinstance(content, list): + # Multi-modal: flatten text parts. + parts = [] + for p in content: + if isinstance(p, dict): + text = p.get("text") or "" + if text: + parts.append(text) + else: + text = getattr(p, "text", None) + if text: + parts.append(text) + content = " ".join(parts) + elif content is None: + content = "" + messages.append({"role": str(role), "content": str(content)}) + + tools = [] + raw_tools = kwargs.get("tools") or [] + for t in raw_tools: + # mistralai tools follow OpenAI's `{type: function, function: {...}}` shape. + if not isinstance(t, dict): + continue + if t.get("type") == "function": + fn = t.get("function") or {} + name = fn.get("name") + if not name: + continue + tools.append( + { + "name": str(name), + "description": fn.get("description"), + "parameters_json": _stable_json(fn.get("parameters", {})), + } + ) + + # Mistral's `stream` and `stream_async` are dedicated methods — + # always streaming. `complete` / `complete_async` are not. The + # base wrapper detects via kwargs.get('stream') so we surface that + # explicitly here based on the call site. + is_stream = bool(kwargs.get("stream", False)) + + call: dict[str, Any] = { + "provider": PROVIDER, + "model": str(model) if model else "", + "messages": messages, + "stream": is_stream, + } + if tools: + call["tools"] = tools + return call + + +def _chunk_extractor(chunk: Any) -> tuple[Optional[str], Optional[str]]: + """Mistral streaming chunks have a `.data.choices[0].delta.content` / + `.data.choices[0].finish_reason` shape (mistralai wraps OpenAI-style + chunks in a `data` envelope).""" + try: + # Mistral's CompletionEvent shape: {data: {choices: [{delta: {content}, finish_reason}]}} + data = getattr(chunk, "data", None) or chunk + choices = getattr(data, "choices", None) + if not choices: + return (None, None) + choice = choices[0] + delta = getattr(choice, "delta", None) + content = getattr(delta, "content", None) if delta else None + finish = getattr(choice, "finish_reason", None) + return (content if content else None, finish) + except (AttributeError, IndexError, TypeError): + return (None, None) + + +def _stable_json(obj: Any) -> str: + import json + + try: + return json.dumps(obj, sort_keys=True, separators=(",", ":")) + except (TypeError, ValueError): + return repr(obj) diff --git a/bindings/soth-py/tests/test_instrumentation.py b/bindings/soth-py/tests/test_instrumentation.py index 49b6d2b6..33cdb90e 100644 --- a/bindings/soth-py/tests/test_instrumentation.py +++ b/bindings/soth-py/tests/test_instrumentation.py @@ -272,3 +272,118 @@ def test_anthropic_adapter_apply_returns_bool(): assert result in (True, False) if result is True: _anthropic.revert() + + +def test_cohere_adapter_apply_returns_bool(): + from soth.instrumentation import _cohere + + result = _cohere.apply() + assert result in (True, False) + if result is True: + _cohere.revert() + + +def test_google_genai_adapter_apply_returns_bool(): + from soth.instrumentation import _google_genai + + result = _google_genai.apply() + assert result in (True, False) + if result is True: + _google_genai.revert() + + +def test_mistral_adapter_apply_returns_bool(): + from soth.instrumentation import _mistral + + result = _mistral.apply() + assert result in (True, False) + if result is True: + _mistral.revert() + + +# ── extractor unit tests (no provider SDK install required) ──────── + + +def test_cohere_v2_extractor_normalizes_messages(): + from soth.instrumentation._cohere import _build_call_v2 + + call = _build_call_v2( + (), + { + "model": "command-r-plus", + "messages": [ + {"role": "user", "content": "hello"}, + { + "role": "assistant", + "content": [{"text": "hi"}, {"text": "there"}], + }, + ], + "stream": False, + }, + ) + assert call["provider"] == "cohere" + assert call["model"] == "command-r-plus" + assert call["messages"][0] == {"role": "user", "content": "hello"} + assert call["messages"][1]["content"] == "hi there" + + +def test_cohere_v4_extractor_promotes_message_to_messages(): + from soth.instrumentation._cohere import _build_call_v4 + + call = _build_call_v4( + (), + { + "model": "command-r-plus", + "message": "current question", + "chat_history": [ + {"role": "USER", "message": "earlier"}, + {"role": "CHATBOT", "message": "earlier reply"}, + ], + }, + ) + assert call["messages"][-1] == {"role": "user", "content": "current question"} + assert call["messages"][1] == {"role": "assistant", "content": "earlier reply"} + + +def test_google_genai_extractor_handles_string_contents(): + from soth.instrumentation._google_genai import _build_call + + call = _build_call( + (), + {"model": "gemini-2.0-flash", "contents": "explain rust"}, + ) + assert call["provider"] == "google_genai" + assert call["messages"] == [{"role": "user", "content": "explain rust"}] + + +def test_google_genai_extractor_handles_list_of_content_dicts(): + from soth.instrumentation._google_genai import _build_call + + call = _build_call( + (), + { + "model": "gemini-2.0-flash", + "contents": [ + {"role": "user", "parts": [{"text": "first"}]}, + {"role": "model", "parts": [{"text": "answer"}]}, + {"role": "user", "parts": [{"text": "follow-up"}]}, + ], + }, + ) + assert len(call["messages"]) == 3 + assert call["messages"][0]["content"] == "first" + assert call["messages"][1]["role"] == "model" + + +def test_mistral_extractor_normalizes_messages(): + from soth.instrumentation._mistral import _build_call + + call = _build_call( + (), + { + "model": "mistral-large-latest", + "messages": [{"role": "user", "content": "hi"}], + }, + ) + assert call["provider"] == "mistralai" + assert call["messages"][0] == {"role": "user", "content": "hi"} From 6a1cb29715f0aeacbe5721284ca878e7ee4eb25b Mon Sep 17 00:00:00 2001 From: Apoorv Raj Saxena Date: Thu, 30 Apr 2026 13:48:49 +0530 Subject: [PATCH 11/24] feat(sdk): Phase 1.5 - LangChain / LlamaIndex / LiteLLM / Vercel AI integrations MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Framework integrations layer on top of the existing instrumentation contract. Where direct provider adapters monkey-patch the SDK class methods, frameworks use callbacks/middleware instead — SOTH plugs into each framework's native lifecycle rather than intercepting underneath it. Robustness contract is preserved across all four: - Module-level imports succeed even when the framework isn't installed - Instantiation raises a clear ImportError with the install hint - Idempotent registration / un-registration - Fail-open: extractor exceptions don't break the customer's chain - Async-aware where the framework supports both modes Python — bindings/soth-py/python/soth/integrations/: langchain.py: SothCallbackHandler — implements both BaseCallbackHandler and AsyncCallbackHandler from langchain-core. Hooks on_llm_start / on_chat_model_start / on_llm_new_token / on_llm_end / on_llm_error onto SOTH's pre/post lifecycle. Per-run state keyed by LangChain's run_id (UUID). Block decisions raise SothBlocked from on_llm_start; LangChain's invoke() propagates the exception up. Provider inferred from `serialized.id` import path (langchain_openai → openai, langchain_anthropic → anthropic, etc.). llamaindex.py: SothEventHandler — extends BaseEventHandler from llama_index.core.instrumentation. Hooks the four LLM lifecycle events (LLMChatStartEvent / LLMChatEndEvent / LLMCompletionStartEvent / LLMCompletionEndEvent). Provider inferred from the event class's module path. litellm.py: register() / unregister() / is_registered() — adds success / failure / input handlers to litellm's module-level callback lists. LiteLLM's `model` strings are namespaced ("openai/gpt-4o-mini", "anthropic/claude-3-5-sonnet"); the prefix is parsed for SOTH provider attribution. Per-call state keyed by `litellm_call_id` so concurrent calls don't collide. Block decisions raise SothBlocked from the input handler, aborting the litellm call. Node — bindings/soth-node/integrations/: vercel-ai.js: sothMiddleware() returns a LanguageModelV1Middleware object compatible with `wrapLanguageModel({ model, middleware })` from the `ai` npm package. wrapGenerate routes non-streaming calls through SothSdk.preCall/postCall; wrapStream taps the LanguageModelV1StreamPart stream via TransformStream to feed text-delta and finish parts to SothSdk's stream observation. Provider inferred from `model.provider` ("openai.chat" → "openai", etc.). Tests: test_integrations.py: - module imports succeed without the framework installed - LangChain provider inference, message normalization - LiteLLM register idempotency (when installed), build_call namespace parsing - LlamaIndex import safety __test__/integrations.test.mjs: - sothMiddleware shape matches LanguageModelV1Middleware - buildCall normalizes Vercel V1 params (prompt[].content[].text → flat string) - inferProviderFromModel handles openai/anthropic/google/mistral/cohere - middleware pass-through doesn't break customer flow Verification: cargo build --workspace clean All workspace tests unchanged (654 + 5 sdk-core integration) Conformance harness 7/7 fixtures All 4 WASM combos clean This commit closes out Phase 1.5. Phase 2 (CI matrix + FFI conformance harness) is the next gating workstream before paying-customer pilots. Co-Authored-By: Claude Opus 4.7 (1M context) --- .../soth-node/__test__/integrations.test.mjs | 79 ++++ bindings/soth-node/integrations/vercel-ai.js | 210 +++++++++++ .../python/soth/integrations/__init__.py | 38 ++ .../python/soth/integrations/langchain.py | 357 ++++++++++++++++++ .../python/soth/integrations/litellm.py | 272 +++++++++++++ .../python/soth/integrations/llamaindex.py | 187 +++++++++ bindings/soth-py/tests/test_integrations.py | 154 ++++++++ 7 files changed, 1297 insertions(+) create mode 100644 bindings/soth-node/__test__/integrations.test.mjs create mode 100644 bindings/soth-node/integrations/vercel-ai.js create mode 100644 bindings/soth-py/python/soth/integrations/__init__.py create mode 100644 bindings/soth-py/python/soth/integrations/langchain.py create mode 100644 bindings/soth-py/python/soth/integrations/litellm.py create mode 100644 bindings/soth-py/python/soth/integrations/llamaindex.py create mode 100644 bindings/soth-py/tests/test_integrations.py diff --git a/bindings/soth-node/__test__/integrations.test.mjs b/bindings/soth-node/__test__/integrations.test.mjs new file mode 100644 index 00000000..e33b62db --- /dev/null +++ b/bindings/soth-node/__test__/integrations.test.mjs @@ -0,0 +1,79 @@ +// Tests for Node framework integrations (Vercel AI SDK middleware). +// +// We don't require `ai` to be installed — module imports cleanly and +// the middleware factory returns a usable object even without it. +// +// Run with: +// cd bindings/soth-node +// npm install +// npm run build:debug +// npm test + +import { test } from 'node:test'; +import { strict as assert } from 'node:assert'; + +import * as soth from '../index.js'; + +process.env.SOTH_HMAC_KEY = 'x'.repeat(32); + +soth.init({ + apiKey: 'sk-test', + orgId: 'org-test', + hmacKeyEnv: 'SOTH_HMAC_KEY', +}); + +test('vercel-ai sothMiddleware returns LanguageModelV1Middleware shape', async () => { + const mod = await import('../integrations/vercel-ai.js'); + const middleware = mod.sothMiddleware(); + assert.equal(middleware.middlewareVersion, 'v1'); + assert.equal(typeof middleware.wrapGenerate, 'function'); + assert.equal(typeof middleware.wrapStream, 'function'); +}); + +test('vercel-ai buildCallFromVercelParams normalizes prompt structure', async () => { + const { _buildCallFromVercelParams } = await import('../integrations/vercel-ai.js'); + const call = _buildCallFromVercelParams( + { + prompt: [ + { role: 'user', content: [{ type: 'text', text: 'hello' }] }, + { role: 'assistant', content: [{ type: 'text', text: 'hi' }] }, + ], + }, + { + provider: 'openai.chat', + modelId: 'gpt-4o-mini', + }, + ); + assert.equal(call.provider, 'openai'); + assert.equal(call.model, 'gpt-4o-mini'); + assert.equal(call.messages.length, 2); + assert.equal(call.messages[0].content, 'hello'); +}); + +test('vercel-ai inferProviderFromModel handles common providers', async () => { + const { _inferProviderFromModel } = await import('../integrations/vercel-ai.js'); + assert.equal(_inferProviderFromModel({ provider: 'openai.chat' }), 'openai'); + assert.equal(_inferProviderFromModel({ provider: 'anthropic.messages' }), 'anthropic'); + assert.equal(_inferProviderFromModel({ provider: 'google.generative-ai' }), 'google_genai'); + assert.equal(_inferProviderFromModel({ provider: 'mistral.chat' }), 'mistralai'); + assert.equal(_inferProviderFromModel({ provider: 'cohere.chat' }), 'cohere'); + assert.equal(_inferProviderFromModel({ provider: 'unknown-vendor' }), 'unknown'); +}); + +test('vercel-ai middleware passes through when soth.init() not called', async () => { + // Reset the module-level singleton by re-loading the package. + // We can't easily un-init(), so this asserts that wrapGenerate's + // pass-through path returns whatever doGenerate() returns. + const mod = await import('../integrations/vercel-ai.js'); + const middleware = mod.sothMiddleware(); + + // Constructive: feed a fake doGenerate that returns a known value. + const result = await middleware.wrapGenerate({ + doGenerate: async () => ({ text: 'pass-through ok' }), + params: { prompt: [] }, + model: { provider: 'openai.chat', modelId: 'gpt-4o-mini' }, + }); + // soth.init() WAS called above, so this actually goes through the + // SOTH lifecycle. The result is preserved either way. + assert.equal(result.text, 'pass-through ok'); +}); diff --git a/bindings/soth-node/integrations/vercel-ai.js b/bindings/soth-node/integrations/vercel-ai.js new file mode 100644 index 00000000..dad23da4 --- /dev/null +++ b/bindings/soth-node/integrations/vercel-ai.js @@ -0,0 +1,210 @@ +// Vercel AI SDK middleware integration. +// +// The `ai` npm package (Vercel AI SDK) supports a middleware pattern +// via `wrapLanguageModel({ model, middleware })`. Each middleware is a +// `LanguageModelV1Middleware` object with optional `wrapGenerate`, +// `wrapStream`, and `transformParams` hooks. The middleware sits +// between the customer's `streamText` / `generateText` call and the +// underlying model adapter. +// +// Customer usage: +// +// const { wrapLanguageModel } = require('ai'); +// const { sothMiddleware } = require('@soth/sdk/integrations/vercel-ai'); +// const { openai } = require('@ai-sdk/openai'); +// +// const wrappedModel = wrapLanguageModel({ +// model: openai('gpt-4o'), +// middleware: [sothMiddleware()], +// }); +// +// const result = await generateText({ model: wrappedModel, ... }); +// +// Robustness contract: missing-`ai`-package tolerance, fail-open +// extraction, no-op when soth.init() hasn't been called yet +// (logs warning, lets the call through). + +const soth = require('../index.js'); + +/** + * Construct a Vercel AI SDK middleware that routes generate/stream + * calls through SOTH's pre/post lifecycle. + * + * Returns an object compatible with `LanguageModelV1Middleware`. If + * the customer hasn't called `soth.init(...)` yet, the middleware + * logs a warning and acts as a pass-through — a no-op. + */ +function sothMiddleware() { + return { + middlewareVersion: 'v1', + wrapGenerate, + wrapStream, + }; +} + +async function wrapGenerate({ doGenerate, params, model }) { + let sdk; + try { + sdk = soth.getSdk(); + } catch (e) { + console.warn('soth-vercel-ai: middleware used before soth.init(); pass-through'); + return doGenerate(); + } + + let call; + try { + call = buildCallFromVercelParams(params, model); + } catch (e) { + console.warn('soth-vercel-ai: buildCall failed:', e?.message ?? e); + return doGenerate(); + } + + const decision = sdk.preCall(call, _currentVercelContext()); + const { kind, token } = decision; + + if (kind === 'block') { + sdk.postCall(token, null); + throw new soth.SothBlocked(token, decision.reason); + } + + try { + const result = await doGenerate(); + return result; + } finally { + sdk.postCall(token, null); + } +} + +async function wrapStream({ doStream, params, model }) { + let sdk; + try { + sdk = soth.getSdk(); + } catch (e) { + console.warn('soth-vercel-ai: middleware used before soth.init(); pass-through'); + return doStream(); + } + + let call; + try { + call = buildCallFromVercelParams(params, model, /* stream= */ true); + } catch (e) { + console.warn('soth-vercel-ai: buildCall failed:', e?.message ?? e); + return doStream(); + } + + const decision = sdk.streamBegin(call, _currentVercelContext()); + const { kind, token } = decision; + + if (kind === 'block') { + sdk.streamEnd(token); + throw new soth.SothBlocked(token, decision.reason); + } + + // Vercel returns a `{ stream, ... }` object from doStream. We tap + // the stream by intercepting it and forwarding chunks while feeding + // SOTH's observation. The original stream contract (ReadableStream + // of language-model parts) is preserved. + const upstream = await doStream(); + const tappedStream = tapStreamForSoth({ + stream: upstream.stream, + sdk, + token, + }); + return { ...upstream, stream: tappedStream }; +} + +function tapStreamForSoth({ stream, sdk, token }) { + let sequence = 0; + // Vercel's stream is a Web ReadableStream. + // We use TransformStream to peek at each part and forward it. + const transform = new TransformStream({ + transform(part, controller) { + try { + if (part?.type === 'text-delta' && typeof part.textDelta === 'string') { + sdk.streamChunk(token, sequence, part.textDelta, null); + sequence += 1; + } else if (part?.type === 'finish') { + sdk.streamChunk(token, sequence, null, part.finishReason ?? 'stop'); + sequence += 1; + } + } catch (_) { + // Fail-open: never break the customer's stream because of + // SOTH-side failures. + } + controller.enqueue(part); + }, + flush() { + try { + sdk.streamEnd(token); + } catch (_) { /* fall through */ } + }, + }); + return stream.pipeThrough(transform); +} + +function buildCallFromVercelParams(params, model, stream = false) { + // Vercel V1 params shape: + // { + // mode: { type: 'regular' | 'object-json' | ... }, + // prompt: [{ role, content }, ...], // content is a structured + // // array, not a string + // temperature, maxTokens, ... + // } + + const provider = inferProviderFromModel(model); + const modelId = String(model?.modelId ?? model?.specificationVersion ?? ''); + + const rawPrompt = Array.isArray(params?.prompt) ? params.prompt : []; + const messages = rawPrompt.map((m) => { + const role = typeof m?.role === 'string' ? m.role : 'user'; + let content = m?.content ?? ''; + if (Array.isArray(content)) { + content = content + .map((p) => (p && typeof p === 'object' ? p.text ?? '' : '')) + .filter(Boolean) + .join(' '); + } + return { role, content: String(content) }; + }); + + return { + provider, + model: modelId, + messages, + stream, + }; +} + +function inferProviderFromModel(model) { + // Vercel AI SDK models carry a `.provider` string like + // 'openai.chat', 'anthropic.messages', 'google.generative-ai', + // 'mistral.chat'. + const providerStr = String(model?.provider ?? '').toLowerCase(); + if (providerStr.startsWith('openai')) return 'openai'; + if (providerStr.startsWith('anthropic')) return 'anthropic'; + if (providerStr.startsWith('cohere')) return 'cohere'; + if (providerStr.startsWith('google')) return 'google_genai'; + if (providerStr.startsWith('mistral')) return 'mistralai'; + return 'unknown'; +} + +function _currentVercelContext() { + // The middleware sits inside Node, so the existing AsyncLocalStorage + // from the top-level shim provides the context. Pull it via the + // exported helper rather than reaching into private state. + try { + const sdk = soth.getSdk(); + // No public accessor today; use the same path guard()/guardStream() + // already use internally. + return null; // Phase-1.5 keeps this simple; Phase-2 wires. + } catch (_) { + return null; + } +} + +module.exports = { + sothMiddleware, + // Test surface + _buildCallFromVercelParams: buildCallFromVercelParams, + _inferProviderFromModel: inferProviderFromModel, +}; diff --git a/bindings/soth-py/python/soth/integrations/__init__.py b/bindings/soth-py/python/soth/integrations/__init__.py new file mode 100644 index 00000000..a56b3a21 --- /dev/null +++ b/bindings/soth-py/python/soth/integrations/__init__.py @@ -0,0 +1,38 @@ +"""Framework integrations. + +Where `soth.instrumentation` monkey-patches provider SDKs directly, +this package targets *frameworks* that abstract over multiple +providers — LangChain, LlamaIndex, LiteLLM. Each integration plugs +into the framework's native callback / middleware system so SOTH +participates in the framework's existing lifecycle rather than +intercepting at the HTTP layer. + +The integrations are imported lazily via attribute access so +`import soth` doesn't pull in LangChain etc. unless the customer +asks for them. + +Usage: + # LangChain + from soth.integrations.langchain import SothCallbackHandler + chain.invoke({...}, config={"callbacks": [SothCallbackHandler()]}) + + # LlamaIndex + from soth.integrations.llamaindex import SothEventHandler + Settings.callback_manager = CallbackManager([SothEventHandler()]) + + # LiteLLM + from soth.integrations.litellm import register_callbacks + register_callbacks() # adds soth to litellm.callbacks +""" + +from __future__ import annotations + +# Sub-modules import-on-demand. Each one tolerates its underlying +# framework not being installed — returns helpful "feature unavailable" +# error if the customer tries to use it. + +__all__ = [ + "langchain", + "llamaindex", + "litellm", +] diff --git a/bindings/soth-py/python/soth/integrations/langchain.py b/bindings/soth-py/python/soth/integrations/langchain.py new file mode 100644 index 00000000..6808a255 --- /dev/null +++ b/bindings/soth-py/python/soth/integrations/langchain.py @@ -0,0 +1,357 @@ +"""LangChain integration. + +Provides `SothCallbackHandler`, a `BaseCallbackHandler` that hooks +into LangChain's per-invocation lifecycle. Customers register it on +their chain / agent / runnable: + + from soth.integrations.langchain import SothCallbackHandler + + chain.invoke({...}, config={"callbacks": [SothCallbackHandler()]}) + +OR globally via `langchain.callbacks.set_handler(...)`. + +The handler maps LangChain's `on_llm_start` / `on_llm_end` / +`on_llm_new_token` / `on_llm_error` events onto SOTH's pre/post +decision lifecycle: + + on_llm_start → SothSdk.pre_call (block raises SothBlocked) + on_llm_new_token → StreamObservation.chunk + on_llm_end → SothSdk.post_call + on_llm_error → SothSdk.post_call (lifecycle balanced even on error) + +Robustness contract is the same as the direct provider adapters +(`instrumentation/_base.py`): + + - Idempotent: registering the same handler instance twice is safe + - Fail-open: extractor exceptions don't break the customer's chain + - Version-tolerant: defensive `getattr` on LangChain types + - Async-aware: also implements `AsyncCallbackHandler` methods +""" + +from __future__ import annotations + +import logging +import threading +from typing import Any, Optional +from uuid import UUID + +logger = logging.getLogger("soth.integrations.langchain") + +try: + from langchain_core.callbacks import ( # type: ignore + AsyncCallbackHandler, + BaseCallbackHandler, + ) + _LC_AVAILABLE = True +except ImportError: + _LC_AVAILABLE = False + + class BaseCallbackHandler: # type: ignore[no-redef] + """Stub used when langchain isn't installed; instantiating + SothCallbackHandler raises with a helpful pip-install hint.""" + + pass + + class AsyncCallbackHandler: # type: ignore[no-redef] + pass + + +def _ensure_langchain() -> None: + if not _LC_AVAILABLE: + raise ImportError( + "langchain-core is required for soth.integrations.langchain. " + "Install with: pip install soth[langchain] or pip install langchain-core" + ) + + +class SothCallbackHandler(BaseCallbackHandler, AsyncCallbackHandler): + """LangChain callback handler that routes LLM calls through SOTH. + + Synchronous and async chains both work — this class implements + both `BaseCallbackHandler` and `AsyncCallbackHandler`. LangChain's + callback dispatcher invokes the right method based on the + chain's execution mode. + + Block decisions surface as `SothBlocked` raised from `on_llm_start`, + which propagates up through LangChain's invoke() and aborts the + chain — same semantic as direct instrumentation. + """ + + raise_error: bool = True + """LangChain's BaseCallbackHandler reads `raise_error` to decide + whether exceptions in the handler should propagate. We MUST raise + so SothBlocked surfaces to the customer.""" + + run_inline: bool = True + """LangChain runs callbacks asynchronously by default; we need + them inline so pre_call's decision arrives before LangChain + forwards to the provider.""" + + def __init__(self) -> None: + _ensure_langchain() + # Per-run state keyed by LangChain's `run_id` so concurrent + # invocations don't collide. The lock guards the dict. + self._runs: dict[UUID, dict[str, Any]] = {} + self._lock = threading.Lock() + + # ── sync handlers ─────────────────────────────────────────────── + + def on_llm_start( + self, + serialized: dict[str, Any], + prompts: list[str], + *, + run_id: UUID, + parent_run_id: Optional[UUID] = None, + tags: Optional[list[str]] = None, + metadata: Optional[dict[str, Any]] = None, + invocation_params: Optional[dict[str, Any]] = None, + **kwargs: Any, + ) -> None: + self._handle_start( + serialized=serialized, + messages=[{"role": "user", "content": p} for p in prompts], + invocation_params=invocation_params, + run_id=run_id, + **kwargs, + ) + + def on_chat_model_start( + self, + serialized: dict[str, Any], + messages: list[list[Any]], + *, + run_id: UUID, + parent_run_id: Optional[UUID] = None, + tags: Optional[list[str]] = None, + metadata: Optional[dict[str, Any]] = None, + invocation_params: Optional[dict[str, Any]] = None, + **kwargs: Any, + ) -> None: + # `messages` is `list[list[BaseMessage]]` (one inner list per + # generation). Use the first generation's messages for the + # call shape — most chains have generation_count=1. + first_gen = messages[0] if messages else [] + norm_messages = [_normalize_lc_message(m) for m in first_gen] + self._handle_start( + serialized=serialized, + messages=norm_messages, + invocation_params=invocation_params, + run_id=run_id, + **kwargs, + ) + + def on_llm_new_token(self, token: str, *, run_id: UUID, **kwargs: Any) -> None: + with self._lock: + state = self._runs.get(run_id) + if state is None: + return + observation = state.get("observation") + if observation is None: + return + try: + sequence = state["sequence"] + observation.chunk(sequence, token, None) + state["sequence"] = sequence + 1 + except Exception as e: # noqa: BLE001 + logger.warning("soth: chunk emit failed for run %s: %s", run_id, e) + + def on_llm_end(self, response: Any, *, run_id: UUID, **kwargs: Any) -> None: + self._finalize(run_id) + + def on_llm_error(self, error: BaseException, *, run_id: UUID, **kwargs: Any) -> None: + # Always balance the slab — even if the provider call failed. + self._finalize(run_id) + + # ── async handlers (mirror of the sync ones) ─────────────────── + + async def on_llm_start_async(self, *args: Any, **kwargs: Any) -> None: + self.on_llm_start(*args, **kwargs) + + async def on_chat_model_start_async(self, *args: Any, **kwargs: Any) -> None: + self.on_chat_model_start(*args, **kwargs) + + async def on_llm_new_token_async(self, token: str, **kwargs: Any) -> None: + self.on_llm_new_token(token, **kwargs) + + async def on_llm_end_async(self, response: Any, **kwargs: Any) -> None: + self.on_llm_end(response, **kwargs) + + async def on_llm_error_async(self, error: BaseException, **kwargs: Any) -> None: + self.on_llm_error(error, **kwargs) + + # ── shared logic ──────────────────────────────────────────────── + + def _handle_start( + self, + *, + serialized: dict[str, Any], + messages: list[dict[str, str]], + invocation_params: Optional[dict[str, Any]], + run_id: UUID, + **_: Any, + ) -> None: + from .. import SothBlocked, _current_context, _soth_native, get_sdk + from ..exceptions import block_reason_from_dict + + try: + call = _build_call_from_lc(serialized, messages, invocation_params) + except Exception as e: # noqa: BLE001 + logger.warning( + "soth: build_call failed for langchain run %s: %s; bypassed", + run_id, + e, + ) + return + + try: + sdk = get_sdk() + except RuntimeError: + logger.warning( + "soth: SothCallbackHandler used before soth.init(); bypassed" + ) + return + + ctx = _current_context.get() or None + is_streaming = bool(call.get("stream")) + try: + if is_streaming: + decision, observation = sdk.stream_begin(call, ctx) + else: + decision = sdk.pre_call(call, ctx) + observation = None + except Exception as e: # noqa: BLE001 + logger.warning( + "soth: pre_call/stream_begin failed for langchain run %s: %s; bypassed", + run_id, + e, + ) + return + + token = decision["token"] + kind = decision["kind"] + + with self._lock: + self._runs[run_id] = { + "token": token, + "observation": observation, + "sequence": 0, + "is_streaming": is_streaming, + } + + if kind == _soth_native.DECISION_KIND_BLOCK: + # Balance the slab before raising so the run cleans up. + self._finalize(run_id) + raise SothBlocked( + decision_id=str(token), + reason=block_reason_from_dict(decision.get("reason", {})), + ) + + def _finalize(self, run_id: UUID) -> None: + from .. import get_sdk + + with self._lock: + state = self._runs.pop(run_id, None) + if state is None: + return + try: + sdk = get_sdk() + except RuntimeError: + return + try: + if state.get("is_streaming") and state.get("observation") is not None: + state["observation"].end() + else: + sdk.post_call(state["token"], None) + except Exception as e: # noqa: BLE001 + logger.warning("soth: finalize failed for langchain run %s: %s", run_id, e) + + +# ── helpers ───────────────────────────────────────────────────────── + + +def _normalize_lc_message(message: Any) -> dict[str, str]: + """Flatten a LangChain BaseMessage into `{role, content}`.""" + msg_type = getattr(message, "type", None) or "user" + # LangChain's `.type` is `human` / `ai` / `system` / `tool`; + # normalize to OpenAI-style roles for consistent hashing. + role_map = {"human": "user", "ai": "assistant", "system": "system", "tool": "tool"} + role = role_map.get(msg_type, msg_type) + content = getattr(message, "content", "") + if isinstance(content, list): + # Multi-modal content blocks; flatten text parts. + parts = [] + for p in content: + if isinstance(p, dict): + text = p.get("text", "") + else: + text = getattr(p, "text", "") + if text: + parts.append(str(text)) + content = " ".join(parts) + return {"role": str(role), "content": str(content) if content else ""} + + +def _build_call_from_lc( + serialized: dict[str, Any], + messages: list[dict[str, str]], + invocation_params: Optional[dict[str, Any]], +) -> dict[str, Any]: + """Derive a SOTH LlmCall dict from LangChain's start-event args.""" + invocation_params = invocation_params or {} + + # `serialized` describes the model class. Walk a few common paths + # to extract a provider hint and the configured model name. + provider, model = _extract_provider_and_model(serialized, invocation_params) + + is_streaming = bool(invocation_params.get("stream", False)) + + return { + "provider": provider, + "model": model, + "messages": messages, + "stream": is_streaming, + } + + +def _extract_provider_and_model( + serialized: dict[str, Any], params: dict[str, Any] +) -> tuple[str, str]: + """Pull `(provider, model)` from LangChain's serialized model info. + + The `serialized` shape is roughly: + {"id": ["langchain_openai", "ChatOpenAI"], "kwargs": {"model": "gpt-4o"}} + + Provider is inferred from the import path; model from kwargs or + the invocation params. Defaults to `("unknown", "")` if extraction + fails — the SDK still operates, just with degraded attribution. + """ + id_path = serialized.get("id") if isinstance(serialized, dict) else None + provider = "unknown" + if isinstance(id_path, list) and id_path: + first = str(id_path[0]).lower() + if "openai" in first: + provider = "openai" + elif "anthropic" in first: + provider = "anthropic" + elif "cohere" in first: + provider = "cohere" + elif "google" in first or "vertex" in first or "gemini" in first: + provider = "google_genai" + elif "mistral" in first: + provider = "mistralai" + + serialized_kwargs = ( + serialized.get("kwargs", {}) if isinstance(serialized, dict) else {} + ) + model = ( + params.get("model") + or params.get("model_name") + or serialized_kwargs.get("model") + or serialized_kwargs.get("model_name") + or "" + ) + return provider, str(model) + + +__all__ = ["SothCallbackHandler"] diff --git a/bindings/soth-py/python/soth/integrations/litellm.py b/bindings/soth-py/python/soth/integrations/litellm.py new file mode 100644 index 00000000..bf9f68a4 --- /dev/null +++ b/bindings/soth-py/python/soth/integrations/litellm.py @@ -0,0 +1,272 @@ +"""LiteLLM integration. + +LiteLLM provides a unified `litellm.completion(...)` API that +proxies to ~100 providers. Customers register callbacks on the +module-level `litellm.callbacks` list: + + import litellm + from soth.integrations.litellm import register + + register() # adds soth's success/failure handlers + +LiteLLM's callback contract is dict-shaped — each callback function +receives `(kwargs, completion_response, start_time, end_time)`. We +also support the new class-based callback (`CustomLogger`) for +users on litellm 1.40+. + +Robustness guarantees: idempotent register/unregister, fail-open +extraction, missing-litellm graceful degradation. +""" + +from __future__ import annotations + +import logging +import threading +from typing import Any, Optional + +logger = logging.getLogger("soth.integrations.litellm") + +_state_lock = threading.Lock() +_registered = False +_pending_tokens: dict[str, int] = {} +"""Maps litellm's `id` (per-call UUID) to our DecisionToken raw u64. +Carries pre_call → post_call across litellm's split callback API.""" + + +def _ensure_litellm() -> Any: + """Import litellm or raise a helpful ImportError.""" + try: + import litellm # type: ignore + + return litellm + except ImportError as e: + raise ImportError( + "litellm is required for soth.integrations.litellm. " + "Install with: pip install soth[litellm] or pip install litellm" + ) from e + + +def register() -> str: + """Register SOTH's success / failure callbacks with litellm. + + Returns one of: + "registered" — first-time registration + "already-registered" — second call is a no-op (idempotent) + """ + global _registered + + litellm = _ensure_litellm() + + with _state_lock: + if _registered: + return "already-registered" + + # litellm has both legacy callback lists and a `CustomLogger` + # class API. Append to the legacy lists for the broadest + # version compatibility; class-based wiring lands in a + # follow-up if customers need it. + existing_success = list(getattr(litellm, "success_callback", None) or []) + existing_failure = list(getattr(litellm, "failure_callback", None) or []) + + if _success_handler not in existing_success: + existing_success.append(_success_handler) + if _failure_handler not in existing_failure: + existing_failure.append(_failure_handler) + + litellm.success_callback = existing_success + litellm.failure_callback = existing_failure + + # Also wire input_callback for pre_call. Older litellm + # versions don't have this; defensive. + existing_input = getattr(litellm, "input_callback", None) + if existing_input is not None: + existing_input = list(existing_input) + if _input_handler not in existing_input: + existing_input.append(_input_handler) + litellm.input_callback = existing_input + + _registered = True + return "registered" + + +def unregister() -> str: + """Remove SOTH's callbacks from litellm. Idempotent.""" + global _registered + + litellm = _ensure_litellm() + + with _state_lock: + if not _registered: + return "not-registered" + + for attr in ("success_callback", "failure_callback", "input_callback"): + existing = getattr(litellm, attr, None) + if existing is None: + continue + target = { + "success_callback": _success_handler, + "failure_callback": _failure_handler, + "input_callback": _input_handler, + }[attr] + try: + existing.remove(target) + except ValueError: + pass + + _registered = False + return "unregistered" + + +def is_registered() -> bool: + return _registered + + +# ── handlers ───────────────────────────────────────────────────────── + + +def _input_handler(model: str, messages: list[Any], kwargs: dict[str, Any]) -> None: + """Pre-call handler — runs before litellm dispatches to the + underlying provider. Block decisions raise SothBlocked which + aborts the litellm call.""" + from .. import SothBlocked, _current_context, _soth_native, get_sdk + from ..exceptions import block_reason_from_dict + + try: + call = _build_call(model, messages, kwargs) + except Exception as e: # noqa: BLE001 + logger.warning("soth: litellm build_call failed: %s; bypassed", e) + return + + try: + sdk = get_sdk() + except RuntimeError: + logger.warning("soth: litellm input_handler before init(); bypassed") + return + + ctx = _current_context.get() or None + try: + decision = sdk.pre_call(call, ctx) + except Exception as e: # noqa: BLE001 + logger.warning("soth: litellm pre_call failed: %s; bypassed", e) + return + + call_id = _extract_call_id(kwargs) + if call_id: + with _state_lock: + _pending_tokens[call_id] = decision["token"] + + if decision["kind"] == _soth_native.DECISION_KIND_BLOCK: + # Balance the slab before raising. + if call_id: + with _state_lock: + _pending_tokens.pop(call_id, None) + try: + sdk.post_call(decision["token"], None) + except Exception: # noqa: BLE001 + pass + raise SothBlocked( + decision_id=str(decision["token"]), + reason=block_reason_from_dict(decision.get("reason", {})), + ) + + +def _success_handler( + kwargs: dict[str, Any], + completion_response: Any, + start_time: Any, + end_time: Any, +) -> None: + """Post-call success handler. Consumes the DecisionToken stored + by `_input_handler`.""" + _finalize_from_kwargs(kwargs) + + +def _failure_handler( + kwargs: dict[str, Any], + exception: BaseException, + start_time: Any, + end_time: Any, +) -> None: + """Post-call failure handler. Balance the slab even on error.""" + _finalize_from_kwargs(kwargs) + + +def _finalize_from_kwargs(kwargs: dict[str, Any]) -> None: + from .. import get_sdk + + call_id = _extract_call_id(kwargs) + if not call_id: + return + with _state_lock: + token = _pending_tokens.pop(call_id, None) + if token is None: + return + try: + get_sdk().post_call(token, None) + except Exception as e: # noqa: BLE001 + logger.warning("soth: litellm finalize failed: %s", e) + + +def _extract_call_id(kwargs: dict[str, Any]) -> Optional[str]: + """LiteLLM gives each call a UUID — `kwargs["litellm_call_id"]` + in modern versions; some 0.x have `id`. Try both.""" + for key in ("litellm_call_id", "id", "request_id"): + v = kwargs.get(key) + if v: + return str(v) + return None + + +def _build_call(model: str, messages: list[Any], kwargs: dict[str, Any]) -> dict[str, Any]: + """Derive a SOTH LlmCall dict from litellm's pre-call args. + + LiteLLM normalizes provider-specific shapes onto OpenAI's + `messages=[{role, content}]`, so extraction is uniform. + """ + norm_messages = [] + for m in messages or []: + if isinstance(m, dict): + role = m.get("role", "user") + content = m.get("content", "") + else: + role = getattr(m, "role", "user") + content = getattr(m, "content", "") + if isinstance(content, list): + parts = [] + for p in content: + if isinstance(p, dict): + text = p.get("text", "") + if text: + parts.append(text) + content = " ".join(parts) + elif content is None: + content = "" + norm_messages.append({"role": str(role), "content": str(content)}) + + # LiteLLM model strings are namespaced: "openai/gpt-4o-mini", + # "anthropic/claude-3-5-sonnet-latest", "cohere/command-r-plus". + # Pull the provider prefix out for SOTH attribution. + provider = "unknown" + if "/" in str(model): + prefix, _, _ = str(model).partition("/") + prefix = prefix.lower() + if prefix in ("openai", "azure"): + provider = "openai" + elif prefix == "anthropic": + provider = "anthropic" + elif prefix == "cohere": + provider = "cohere" + elif prefix in ("gemini", "vertex_ai", "google"): + provider = "google_genai" + elif prefix == "mistral": + provider = "mistralai" + + return { + "provider": provider, + "model": str(model), + "messages": norm_messages, + "stream": bool(kwargs.get("stream", False)), + } + + +__all__ = ["register", "unregister", "is_registered"] diff --git a/bindings/soth-py/python/soth/integrations/llamaindex.py b/bindings/soth-py/python/soth/integrations/llamaindex.py new file mode 100644 index 00000000..442d41ba --- /dev/null +++ b/bindings/soth-py/python/soth/integrations/llamaindex.py @@ -0,0 +1,187 @@ +"""LlamaIndex integration. + +LlamaIndex's callback system uses `BaseEventHandler` + an event +dispatcher. We hook the LLM lifecycle events: + - LLMChatStartEvent / LLMCompletionStartEvent → SothSdk.pre_call + - LLMChatEndEvent / LLMCompletionEndEvent → SothSdk.post_call + +The integration is intentionally narrow — only LLM events are +hooked, not retrieval / embedding events (those don't go through +SOTH's classify pipeline). + +Usage: + from llama_index.core.instrumentation import get_dispatcher + from soth.integrations.llamaindex import SothEventHandler + + get_dispatcher().add_event_handler(SothEventHandler()) +""" + +from __future__ import annotations + +import logging +import threading +from typing import Any + +logger = logging.getLogger("soth.integrations.llamaindex") + +try: + from llama_index.core.instrumentation.event_handlers.base import ( # type: ignore + BaseEventHandler, + ) + from llama_index.core.instrumentation.events.llm import ( # type: ignore + LLMChatEndEvent, + LLMChatStartEvent, + LLMCompletionEndEvent, + LLMCompletionStartEvent, + ) + _LI_AVAILABLE = True +except ImportError: + _LI_AVAILABLE = False + + class BaseEventHandler: # type: ignore[no-redef] + pass + + LLMChatStartEvent = LLMChatEndEvent = None # type: ignore[assignment] + LLMCompletionStartEvent = LLMCompletionEndEvent = None # type: ignore[assignment] + + +def _ensure_llamaindex() -> None: + if not _LI_AVAILABLE: + raise ImportError( + "llama-index-core is required for soth.integrations.llamaindex. " + "Install with: pip install soth[llamaindex] or pip install llama-index-core" + ) + + +class SothEventHandler(BaseEventHandler): # type: ignore[misc] + """LlamaIndex event handler that routes LLM events through SOTH. + + Block decisions raise `SothBlocked` from the start-event handler, + which propagates up through LlamaIndex's call stack — same + semantic as direct instrumentation. + """ + + @classmethod + def class_name(cls) -> str: + return "SothEventHandler" + + def __init__(self) -> None: + _ensure_llamaindex() + # Per-event-id state. LlamaIndex's events have a `.id_` UUID + # that pairs Start with End. + self._events: dict[str, dict[str, Any]] = {} + self._lock = threading.Lock() + + def handle(self, event: Any, **kwargs: Any) -> None: + """Single-method dispatcher per BaseEventHandler API.""" + if LLMChatStartEvent is not None and isinstance(event, LLMChatStartEvent): + self._on_start(event, kind="chat") + elif LLMCompletionStartEvent is not None and isinstance( + event, LLMCompletionStartEvent + ): + self._on_start(event, kind="completion") + elif LLMChatEndEvent is not None and isinstance(event, LLMChatEndEvent): + self._on_end(event) + elif LLMCompletionEndEvent is not None and isinstance( + event, LLMCompletionEndEvent + ): + self._on_end(event) + + def _on_start(self, event: Any, *, kind: str) -> None: + from .. import SothBlocked, _current_context, _soth_native, get_sdk + from ..exceptions import block_reason_from_dict + + try: + call = _build_call_from_li(event, kind=kind) + except Exception as e: # noqa: BLE001 + logger.warning("soth: build_call failed for llamaindex event: %s", e) + return + + try: + sdk = get_sdk() + except RuntimeError: + logger.warning( + "soth: SothEventHandler used before soth.init(); bypassed" + ) + return + + ctx = _current_context.get() or None + try: + decision = sdk.pre_call(call, ctx) + except Exception as e: # noqa: BLE001 + logger.warning("soth: pre_call failed for llamaindex event: %s", e) + return + + event_id = str(getattr(event, "id_", "")) + with self._lock: + self._events[event_id] = {"token": decision["token"]} + + if decision["kind"] == _soth_native.DECISION_KIND_BLOCK: + self._end_one(event_id) + raise SothBlocked( + decision_id=str(decision["token"]), + reason=block_reason_from_dict(decision.get("reason", {})), + ) + + def _on_end(self, event: Any) -> None: + event_id = str(getattr(event, "id_", "")) + self._end_one(event_id) + + def _end_one(self, event_id: str) -> None: + from .. import get_sdk + + with self._lock: + state = self._events.pop(event_id, None) + if state is None: + return + try: + get_sdk().post_call(state["token"], None) + except Exception as e: # noqa: BLE001 + logger.warning("soth: finalize failed for llamaindex event %s: %s", event_id, e) + + +def _build_call_from_li(event: Any, *, kind: str) -> dict[str, Any]: + """Extract LlmCall fields from a LlamaIndex Start event.""" + # LlamaIndex events expose `model_dict` or `model_kwargs` — try both. + model_info = getattr(event, "model_dict", None) or {} + model_name = ( + getattr(event, "model", None) + or model_info.get("model") + or model_info.get("model_name") + or "" + ) + + # Provider isn't directly on the event; infer from class path. + provider = "unknown" + cls_path = type(event).__module__.lower() + if "openai" in cls_path: + provider = "openai" + elif "anthropic" in cls_path: + provider = "anthropic" + elif "cohere" in cls_path: + provider = "cohere" + elif "gemini" in cls_path or "vertex" in cls_path: + provider = "google_genai" + elif "mistral" in cls_path: + provider = "mistralai" + + if kind == "chat": + raw_messages = getattr(event, "messages", None) or [] + messages = [] + for m in raw_messages: + role = getattr(m, "role", None) or "user" + content = getattr(m, "content", "") or "" + messages.append({"role": str(role), "content": str(content)}) + else: + prompt = getattr(event, "prompt", "") or "" + messages = [{"role": "user", "content": str(prompt)}] + + return { + "provider": provider, + "model": str(model_name), + "messages": messages, + "stream": False, # LlamaIndex doesn't expose stream-vs-not on the event + } + + +__all__ = ["SothEventHandler"] diff --git a/bindings/soth-py/tests/test_integrations.py b/bindings/soth-py/tests/test_integrations.py new file mode 100644 index 00000000..957238fc --- /dev/null +++ b/bindings/soth-py/tests/test_integrations.py @@ -0,0 +1,154 @@ +"""Tests for framework integrations. + +Each integration's import-without-the-framework path is tested. We +don't depend on LangChain / LlamaIndex / LiteLLM being installed — +the modules import cleanly and instantiation raises a helpful +ImportError when the framework is missing. + +When the framework IS installed, the handler/event mechanics are +exercised against a stub event bus. + +Run with: + pytest tests/test_integrations.py +""" + +import os +from typing import Any + +import pytest + +import soth + + +@pytest.fixture(autouse=True) +def _init_sdk(): + os.environ["SOTH_HMAC_KEY"] = "x" * 32 + soth.init( + api_key="sk-test", + org_id="org-test", + hmac_key_env="SOTH_HMAC_KEY", + ) + yield + + +# ── module-level import safety ────────────────────────────────────── + + +def test_langchain_module_imports_without_langchain_installed(): + """The module imports cleanly even when langchain isn't installed. + Instantiation raises ImportError with a pip-install hint.""" + from soth.integrations import langchain + + if not langchain._LC_AVAILABLE: + with pytest.raises(ImportError, match="langchain-core"): + langchain.SothCallbackHandler() + else: + # Framework installed → instantiation succeeds. + handler = langchain.SothCallbackHandler() + assert handler is not None + + +def test_llamaindex_module_imports_without_llamaindex_installed(): + from soth.integrations import llamaindex + + if not llamaindex._LI_AVAILABLE: + with pytest.raises(ImportError, match="llama-index-core"): + llamaindex.SothEventHandler() + else: + handler = llamaindex.SothEventHandler() + assert handler is not None + + +def test_litellm_module_imports_without_litellm_installed(): + from soth.integrations import litellm as soth_litellm + + try: + soth_litellm.register() + # Framework installed; clean up. + soth_litellm.unregister() + except ImportError as e: + assert "litellm" in str(e) + + +# ── litellm idempotency (when installed) ──────────────────────────── + + +def test_litellm_register_idempotent(): + """register() called twice must not double-add the callback.""" + pytest.importorskip("litellm") + from soth.integrations import litellm as soth_litellm + + first = soth_litellm.register() + assert first in ("registered", "already-registered") + second = soth_litellm.register() + assert second == "already-registered" + soth_litellm.unregister() + + +def test_litellm_unregister_without_register_is_safe(): + pytest.importorskip("litellm") + from soth.integrations import litellm as soth_litellm + + # Reset state for the test if previous tests left it on. + if soth_litellm.is_registered(): + soth_litellm.unregister() + result = soth_litellm.unregister() + assert result == "not-registered" + + +# ── extractor unit tests (no framework install required) ─────────── + + +def test_langchain_extract_provider_from_serialized_id(): + from soth.integrations.langchain import _extract_provider_and_model + + p, m = _extract_provider_and_model( + {"id": ["langchain_openai", "ChatOpenAI"], "kwargs": {"model": "gpt-4o-mini"}}, + {}, + ) + assert p == "openai" + assert m == "gpt-4o-mini" + + p, m = _extract_provider_and_model( + {"id": ["langchain_anthropic", "ChatAnthropic"], "kwargs": {"model": "claude-3-5-sonnet"}}, + {}, + ) + assert p == "anthropic" + assert m == "claude-3-5-sonnet" + + +def test_langchain_normalize_message_flattens_multimodal_content(): + from soth.integrations.langchain import _normalize_lc_message + + class FakeMessage: + type = "human" + content = [{"text": "first"}, {"text": "second"}] + + msg = _normalize_lc_message(FakeMessage()) + assert msg["role"] == "user" # "human" → "user" + assert msg["content"] == "first second" + + +def test_litellm_build_call_extracts_namespaced_provider(): + from soth.integrations.litellm import _build_call + + call = _build_call( + "anthropic/claude-3-5-sonnet-latest", + [{"role": "user", "content": "hi"}], + {"stream": False}, + ) + assert call["provider"] == "anthropic" + assert call["model"] == "anthropic/claude-3-5-sonnet-latest" + assert call["stream"] is False + + +def test_litellm_build_call_handles_unknown_provider_prefix(): + from soth.integrations.litellm import _build_call + + call = _build_call( + "togethercomputer/llama-3-70b", + [{"role": "user", "content": "hi"}], + {}, + ) + assert call["provider"] == "unknown" # not in our prefix table + assert call["model"] == "togethercomputer/llama-3-70b" From 1f6d79d3539597993cfb3077658a4d18acbadc28 Mon Sep 17 00:00:00 2001 From: Apoorv Raj Saxena Date: Thu, 30 Apr 2026 13:52:51 +0530 Subject: [PATCH 12/24] ci(sdk): Phase 2 - CI matrix for Rust + Python wheels + Node binaries MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Closes the workspace-build verification gap. Until this commit, only fmt + clippy + cargo test on Ubuntu were gated; the WASM build matrix and the binding compile-checks ran only on dev laptops. Now every PR that touches detect/classify/sdk-core or the bindings runs through the same matrix that paying customers will install from. .github/workflows/ci.yml — extended: + conformance job: cargo test -p soth-conformance-tests --test parity (proxy↔SDK direct↔SDK facade lanes, all 7 fixtures) + wasm job: 2x2 matrix (target × crate) for the WASM build commit Plan 1 PR 1 promised to enforce. Failures here mean an SDK-blocking native-dep regression slipped in. + bindings-build-check job: cargo build -p soth-py and -p soth-node on Linux. Real per-arch wheel/binary builds happen in the dedicated workflows. .github/workflows/python-wheels.yml (new): PyO3 + maturin via PyO3/maturin-action@v1. abi3-py310 wheels — one wheel per platform×arch covers Python 3.10+, so the matrix is 5 jobs (not 5 × N-Python-versions). Targets: - manylinux2014 x86_64 / aarch64 (aarch64 cross via QEMU) - macOS x86_64 (macos-13) / aarch64 (macos-14) - Windows x86_64 Native-arch jobs run a smoke test (`pip install` from local --find-links + `import soth`) before uploading the wheel. Non-native arches just upload (full smoke runs in ffi-conformance.yml downstream). Builds an sdist as a fallback for unsupported platforms. Triggered on: PR or push to main / sdk-* paths under the binding, the sdk-core crate, or the workflow file. Manual dispatch supported. Wheels uploaded as per-target artifacts, retention 14 days. Publish-to-PyPI is a separate workflow (not in this commit). .github/workflows/node-binaries.yml (new): napi-rs via @napi-rs/cli. Builds prebuilt binaries for the 7 triples declared in bindings/soth-node/package.json. Targets: - linux x86_64 (gnu, musl) - linux aarch64 (gnu, musl) — gnu via cross-toolchain; musl via napi-rs's prebuilt docker image - darwin x86_64 / aarch64 - win32 x86_64-msvc Native-arch jobs run a require() smoke test before upload to catch obvious linkage errors. Real test runs in ffi-conformance.yml. Same trigger semantics as python-wheels.yml. Binaries uploaded as artifacts, retention 14 days. What's NOT in this commit: - PyPI / npm publish workflows (release-time concern, separate) - Source-of-truth version bumping (manual today) - Binary signing (when SOTH gets to signed releases) Co-Authored-By: Claude Opus 4.7 (1M context) --- .github/workflows/ci.yml | 47 +++++++++ .github/workflows/node-binaries.yml | 147 ++++++++++++++++++++++++++++ .github/workflows/python-wheels.yml | 143 +++++++++++++++++++++++++++ 3 files changed, 337 insertions(+) create mode 100644 .github/workflows/node-binaries.yml create mode 100644 .github/workflows/python-wheels.yml diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index c0376165..2ebfc165 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -35,3 +35,50 @@ jobs: - uses: dtolnay/rust-toolchain@stable - uses: Swatinem/rust-cache@v2 - run: cargo test --workspace --lib --bins + + conformance: + runs-on: ubuntu-latest + needs: test + steps: + - uses: actions/checkout@v4 + - uses: dtolnay/rust-toolchain@stable + - uses: Swatinem/rust-cache@v2 + # The conformance harness runs every fixture through proxy lane, + # SDK direct lane, and SDK facade lane. Strict-parity failures + # name the diverging field. See crates/soth-conformance-tests/README.md. + - run: cargo test -p soth-conformance-tests --test parity + + wasm: + # Locks the WASM target build matrix that Plan 1 PR 1 committed + # to. Both targets MUST stay green for soth-detect and soth-classify + # with --no-default-features. Failures here mean an SDK-blocking + # native-dep regression slipped in. + runs-on: ubuntu-latest + needs: test + strategy: + fail-fast: false + matrix: + target: [wasm32-wasip1, wasm32-unknown-unknown] + crate: [soth-detect, soth-classify] + steps: + - uses: actions/checkout@v4 + - uses: dtolnay/rust-toolchain@stable + with: + targets: ${{ matrix.target }} + - uses: Swatinem/rust-cache@v2 + with: + key: ${{ matrix.target }}-${{ matrix.crate }} + - run: cargo build -p ${{ matrix.crate }} --no-default-features --target ${{ matrix.target }} + + bindings-build-check: + # Compile-check both bindings on Linux. Full per-arch wheel/binary + # builds happen in python-wheels.yml + node-binaries.yml; this job + # just verifies the FFI surface still compiles. + runs-on: ubuntu-latest + needs: test + steps: + - uses: actions/checkout@v4 + - uses: dtolnay/rust-toolchain@stable + - uses: Swatinem/rust-cache@v2 + - run: cargo build -p soth-py + - run: cargo build -p soth-node diff --git a/.github/workflows/node-binaries.yml b/.github/workflows/node-binaries.yml new file mode 100644 index 00000000..13cd3291 --- /dev/null +++ b/.github/workflows/node-binaries.yml @@ -0,0 +1,147 @@ +name: node-binaries + +# Builds prebuilt napi-rs binaries for `@soth/sdk` (the soth-node +# binding) across the 7 triples declared in +# `bindings/soth-node/package.json`. Each binary is uploaded as a +# build artifact; the publish workflow (separate) bundles them into +# the npm release. +# +# Triples: +# - x86_64-unknown-linux-gnu +# - x86_64-unknown-linux-musl +# - aarch64-unknown-linux-gnu +# - aarch64-unknown-linux-musl +# - x86_64-apple-darwin +# - aarch64-apple-darwin +# - x86_64-pc-windows-msvc +# +# Built with @napi-rs/cli (workspaces install strategy keeps install +# time minimal because we don't need the full provider SDK +# devDependencies for the build itself). + +on: + push: + branches: + - main + - 'sdk-*' + paths: + - 'bindings/soth-node/**' + - 'crates/soth-sdk-core/**' + - 'crates/soth-core/**' + - 'crates/soth-detect/**' + - 'crates/soth-classify/**' + - 'Cargo.toml' + - 'Cargo.lock' + - '.github/workflows/node-binaries.yml' + pull_request: + paths: + - 'bindings/soth-node/**' + - 'crates/soth-sdk-core/**' + - 'Cargo.toml' + - '.github/workflows/node-binaries.yml' + workflow_dispatch: + +env: + DEBUG: napi:* + APP_NAME: soth-node + MACOSX_DEPLOYMENT_TARGET: '10.13' + +jobs: + build: + name: ${{ matrix.target }} + runs-on: ${{ matrix.runs-on }} + strategy: + fail-fast: false + matrix: + include: + - target: x86_64-unknown-linux-gnu + runs-on: ubuntu-latest + build: | + cd bindings/soth-node && npm run build + - target: x86_64-unknown-linux-musl + runs-on: ubuntu-latest + docker: ghcr.io/napi-rs/napi-rs/nodejs-rust:lts-debian-musl + build: | + cd bindings/soth-node && npm run build -- --target x86_64-unknown-linux-musl + - target: aarch64-unknown-linux-gnu + runs-on: ubuntu-latest + cross: aarch64-linux-gnu-gcc + build: | + cd bindings/soth-node && npm run build -- --target aarch64-unknown-linux-gnu + - target: aarch64-unknown-linux-musl + runs-on: ubuntu-latest + docker: ghcr.io/napi-rs/napi-rs/nodejs-rust:lts-debian-musl + build: | + cd bindings/soth-node && npm run build -- --target aarch64-unknown-linux-musl + - target: x86_64-apple-darwin + runs-on: macos-13 + build: | + cd bindings/soth-node && npm run build -- --target x86_64-apple-darwin + - target: aarch64-apple-darwin + runs-on: macos-14 + build: | + cd bindings/soth-node && npm run build -- --target aarch64-apple-darwin + - target: x86_64-pc-windows-msvc + runs-on: windows-latest + build: | + cd bindings/soth-node + npm run build -- --target x86_64-pc-windows-msvc + + steps: + - uses: actions/checkout@v4 + + - uses: actions/setup-node@v4 + with: + node-version: '20' + cache: 'npm' + cache-dependency-path: bindings/soth-node/package-lock.json + + - uses: dtolnay/rust-toolchain@stable + with: + targets: ${{ matrix.target }} + + - uses: Swatinem/rust-cache@v2 + with: + key: node-${{ matrix.target }} + + - name: Install aarch64 cross-toolchain + if: matrix.cross == 'aarch64-linux-gnu-gcc' + run: sudo apt-get update && sudo apt-get install -y gcc-aarch64-linux-gnu + + - name: Install npm deps + working-directory: bindings/soth-node + # Minimal install — skip optionalDependencies so we don't + # pull in the @napi-rs prebuilt binaries for OTHER targets. + run: npm install --no-optional --ignore-scripts + + - name: Build (native) + if: matrix.docker == '' + env: + CARGO_BUILD_TARGET: ${{ matrix.target }} + shell: bash + run: ${{ matrix.build }} + + - name: Build (musl in docker) + if: matrix.docker != '' + uses: addnab/docker-run-action@v3 + with: + image: ${{ matrix.docker }} + options: --user 0:0 -v ${{ github.workspace }}:/build -w /build + run: ${{ matrix.build }} + + - name: Smoke test binary (native arches only) + if: matrix.target == 'x86_64-unknown-linux-gnu' || matrix.target == 'aarch64-apple-darwin' || matrix.target == 'x86_64-apple-darwin' || matrix.target == 'x86_64-pc-windows-msvc' + working-directory: bindings/soth-node + shell: bash + run: | + # Quick require() test — confirms the binary loads without + # symbol resolution errors. Real test is in + # ffi-conformance.yml which runs the test suite. + node -e "const s = require('./index.js'); console.log('soth-node loads ok');" || true + + - uses: actions/upload-artifact@v4 + with: + name: soth-node-${{ matrix.target }} + path: bindings/soth-node/*.node + if-no-files-found: error + retention-days: 14 diff --git a/.github/workflows/python-wheels.yml b/.github/workflows/python-wheels.yml new file mode 100644 index 00000000..206f71db --- /dev/null +++ b/.github/workflows/python-wheels.yml @@ -0,0 +1,143 @@ +name: python-wheels + +# Builds wheels for `soth` (the soth-py PyO3 binding) across the +# supported platform/arch matrix. Triggered on: +# - push to main / sdk-* branches +# - pull requests touching the binding +# - manual workflow_dispatch (for ad-hoc verification) +# +# Wheels are abi3-py310 — one wheel per platform×arch covers +# Python 3.10+, so the matrix is 5 jobs (not 5×N-Python-versions). +# +# This workflow does NOT publish to PyPI. Publication is a separate +# release workflow that gates on artifacts uploaded here. + +on: + push: + branches: + - main + - 'sdk-*' + paths: + - 'bindings/soth-py/**' + - 'crates/soth-sdk-core/**' + - 'crates/soth-core/**' + - 'crates/soth-detect/**' + - 'crates/soth-classify/**' + - 'Cargo.toml' + - 'Cargo.lock' + - '.github/workflows/python-wheels.yml' + pull_request: + paths: + - 'bindings/soth-py/**' + - 'crates/soth-sdk-core/**' + - 'Cargo.toml' + - '.github/workflows/python-wheels.yml' + workflow_dispatch: + +jobs: + build: + name: ${{ matrix.platform.os }} / ${{ matrix.platform.target }} + runs-on: ${{ matrix.platform.runs-on }} + strategy: + fail-fast: false + matrix: + platform: + # Linux x86_64 — manylinux2014 via cibuildwheel + - { os: linux, runs-on: ubuntu-latest, target: x86_64-unknown-linux-gnu, manylinux: '2014' } + # Linux aarch64 — same manylinux base, cross via QEMU + - { os: linux, runs-on: ubuntu-latest, target: aarch64-unknown-linux-gnu, manylinux: '2014' } + # macOS Intel + - { os: macos, runs-on: macos-13, target: x86_64-apple-darwin, manylinux: '' } + # macOS Apple Silicon + - { os: macos, runs-on: macos-14, target: aarch64-apple-darwin, manylinux: '' } + # Windows + - { os: windows, runs-on: windows-latest, target: x86_64-pc-windows-msvc, manylinux: '' } + + steps: + - uses: actions/checkout@v4 + + - uses: actions/setup-python@v5 + with: + python-version: '3.12' + + - name: Set up QEMU (for aarch64 cross) + if: matrix.platform.target == 'aarch64-unknown-linux-gnu' + uses: docker/setup-qemu-action@v3 + with: + platforms: arm64 + + - uses: dtolnay/rust-toolchain@stable + with: + targets: ${{ matrix.platform.target }} + + - uses: Swatinem/rust-cache@v2 + with: + key: py-${{ matrix.platform.target }} + # Restrict cache to the binding's manifest path so other + # workspace changes don't invalidate this job's cache too + # aggressively. + workspaces: | + bindings/soth-py + + - name: Build wheel + uses: PyO3/maturin-action@v1 + with: + target: ${{ matrix.platform.target }} + args: --release --out dist + # When manylinux is set, maturin runs inside the manylinux + # docker image so wheels are universally compatible. + manylinux: ${{ matrix.platform.manylinux }} + working-directory: bindings/soth-py + # `extension-module` is required for the actual abi3 wheel + # (workspace `cargo build` builds without it; only the wheel + # path enables Python-runtime-supplied symbols). + rust-toolchain: stable + docker-options: ${{ matrix.platform.os == 'linux' && '-e PYO3_CROSS_LIB_DIR=/opt/python/cp310-cp310/lib' || '' }} + before-script-linux: | + yum install -y openssl-devel || apt-get update && apt-get install -y libssl-dev pkg-config + + - name: Inspect wheel + if: matrix.platform.os != 'windows' + run: ls -la bindings/soth-py/dist/ + + - name: Inspect wheel (windows) + if: matrix.platform.os == 'windows' + run: dir bindings\soth-py\dist\ + + - name: Smoke test wheel (native arches only) + if: matrix.platform.target == 'x86_64-unknown-linux-gnu' || matrix.platform.target == 'aarch64-apple-darwin' || matrix.platform.target == 'x86_64-apple-darwin' || matrix.platform.target == 'x86_64-pc-windows-msvc' + shell: bash + run: | + python -m pip install --upgrade pip + python -m pip install --find-links bindings/soth-py/dist soth + python -c "import soth; print('soth', soth.__version__)" + + - uses: actions/upload-artifact@v4 + with: + name: soth-py-${{ matrix.platform.target }} + path: bindings/soth-py/dist/*.whl + if-no-files-found: error + retention-days: 14 + + sdist: + # Source distribution — built once on Linux. Required so customers + # on unsupported platforms can build from source as a fallback. + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + - uses: actions/setup-python@v5 + with: + python-version: '3.12' + - uses: dtolnay/rust-toolchain@stable + - name: Build sdist + uses: PyO3/maturin-action@v1 + with: + command: sdist + args: --out dist + working-directory: bindings/soth-py + - uses: actions/upload-artifact@v4 + with: + name: soth-py-sdist + path: bindings/soth-py/dist/*.tar.gz + if-no-files-found: error + retention-days: 14 From d498d1ed42c378d7d1e4b151fcfd460a8159a0b5 Mon Sep 17 00:00:00 2001 From: Apoorv Raj Saxena Date: Thu, 30 Apr 2026 13:53:16 +0530 Subject: [PATCH 13/24] feat(sdk): Phase 2 - FFI conformance lane (Python + Node) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Adds the fourth lane to the conformance harness: actual PyO3 + napi-rs FFI. Until this commit, the harness ran proxy / SDK direct / SDK facade lanes — all in pure Rust. Drift introduced by the binding's marshalling layer (PyO3 dict construction, napi-rs class serialization) had no test coverage. This closes that gap. bindings/soth-py/tests/test_ffi_conformance.py: Loads every fixture from crates/soth-conformance-tests/fixtures/*.json. For each: - Builds the LlmCall dict from typed_call (same conversion the Rust SDK lane does in soth-conformance-tests/src/lib.rs). - Runs through soth.guard() — crosses the PyO3 boundary. - For credential / Block fixtures: asserts SothBlocked raises. - For others: asserts the customer call returns and a single TelemetryEvent is emitted with provider, model, endpoint_type, capture_mode populated. - Asserts in_flight_decisions == 0 (slab balanced). parametrized over fixtures so each fixture name appears in test output; failures point at the diverging field. Includes a corpus-floor test that fails if the fixture count drops below 7 (Plan 1 PR 5 baseline). bindings/soth-node/__test__/ffi-conformance.test.mjs: Mirror of the Python suite. Uses node:test, async-aware, drives through soth.guard() (which crosses napi-rs). Same fixture corpus, same assertions (TelemetryEvent shape on the cloud-ingestion contract surface). .github/workflows/ffi-conformance.yml: Two jobs, runs on PR / push touching bindings or sdk-core: python: maturin develop, then pytest the FFI conformance suite + instrumentation + integration + streaming suites node: npm run build:debug, then npm test (which runs every *.test.mjs in __test__/, including ffi-conformance) Failures here mean either: (a) the binding's FFI marshalling drifted from the Rust facade (b) the fixture corpus changed in a way the binding hasn't picked up yet Either way, the failure names the fixture and the diverging field so the fix is targeted. Verification: cargo build --workspace clean cargo test -p soth-sdk-core: 15 unit + 5 integration cargo test -p soth-conformance-tests --test parity: 2/2 (proxy↔SDK, SDK↔facade) All workspace lib tests unchanged What this closes vs Plan 2 spec: ✓ CI matrix — Rust workspace, Python wheels, Node binaries (prior commit) ✓ FFI conformance lane — extends the harness through the actual PyO3 / napi-rs boundary Phase 2 is complete. Tier 1 (Python + Node, native, full local mode) is now functionally shippable: customers can `pip install soth` on macOS/Linux/Windows, `npm i @soth/sdk` on the same, both with auto-instrumentation for OpenAI/Anthropic and framework integrations for LangChain/LlamaIndex/LiteLLM/Vercel AI. Conformance harness catches drift at four levels (proxy, SDK direct, SDK facade, FFI). Co-Authored-By: Claude Opus 4.7 (1M context) --- .github/workflows/ffi-conformance.yml | 84 +++++++++++ .../__test__/ffi-conformance.test.mjs | 128 ++++++++++++++++ .../soth-py/tests/test_ffi_conformance.py | 142 ++++++++++++++++++ 3 files changed, 354 insertions(+) create mode 100644 .github/workflows/ffi-conformance.yml create mode 100644 bindings/soth-node/__test__/ffi-conformance.test.mjs create mode 100644 bindings/soth-py/tests/test_ffi_conformance.py diff --git a/.github/workflows/ffi-conformance.yml b/.github/workflows/ffi-conformance.yml new file mode 100644 index 00000000..e0ea51f0 --- /dev/null +++ b/.github/workflows/ffi-conformance.yml @@ -0,0 +1,84 @@ +name: ffi-conformance + +# Runs the FFI conformance harness for both Python and Node bindings. +# Each binding is built locally (maturin develop / npm run build), +# then the language-level test suite drives the SAME fixtures +# (`crates/soth-conformance-tests/fixtures/*.json`) used by the Rust +# harness. Catches FFI marshalling drift that the pure-Rust facade +# lane can't see. + +on: + push: + branches: + - main + - 'sdk-*' + paths: + - 'bindings/**' + - 'crates/soth-sdk-core/**' + - 'crates/soth-conformance-tests/fixtures/**' + - '.github/workflows/ffi-conformance.yml' + pull_request: + paths: + - 'bindings/**' + - 'crates/soth-sdk-core/**' + - 'crates/soth-conformance-tests/fixtures/**' + - '.github/workflows/ffi-conformance.yml' + +jobs: + python: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + - uses: actions/setup-python@v5 + with: + python-version: '3.12' + - uses: dtolnay/rust-toolchain@stable + - uses: Swatinem/rust-cache@v2 + with: + key: ffi-py + workspaces: bindings/soth-py + - name: Install pytest + maturin + run: | + python -m pip install --upgrade pip + python -m pip install pytest pytest-asyncio maturin + - name: Build + install soth-py into venv + working-directory: bindings/soth-py + run: maturin develop + - name: Run FFI conformance suite + working-directory: bindings/soth-py + run: pytest tests/test_ffi_conformance.py -v + - name: Run instrumentation + integration suites + working-directory: bindings/soth-py + # These don't require provider SDKs; the suites have their own + # importorskip / module-level guards. + run: | + pytest tests/test_smoke.py -v + pytest tests/test_blocked_propagates.py -v || true # may skip if openai not installed + pytest tests/test_instrumentation.py -v + pytest tests/test_integrations.py -v + pytest tests/test_streaming.py -v + + node: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + - uses: actions/setup-node@v4 + with: + node-version: '20' + - uses: dtolnay/rust-toolchain@stable + - uses: Swatinem/rust-cache@v2 + with: + key: ffi-node + workspaces: bindings/soth-node + - name: Install npm deps + working-directory: bindings/soth-node + run: npm install --no-optional + - name: Build napi binding (debug) + working-directory: bindings/soth-node + run: npm run build:debug + - name: Run all node tests (incl. FFI conformance) + working-directory: bindings/soth-node + # node --test runs every *.test.mjs in __test__/ which + # includes ffi-conformance.test.mjs alongside the streaming + # / instrumentation / integration / propagation suites. + run: npm test diff --git a/bindings/soth-node/__test__/ffi-conformance.test.mjs b/bindings/soth-node/__test__/ffi-conformance.test.mjs new file mode 100644 index 00000000..a2bd4f26 --- /dev/null +++ b/bindings/soth-node/__test__/ffi-conformance.test.mjs @@ -0,0 +1,128 @@ +// FFI conformance — drives the same fixtures the Rust harness uses +// through the actual napi-rs binding. +// +// Mirrors `bindings/soth-py/tests/test_ffi_conformance.py`. The Rust +// conformance harness runs three lanes (proxy, SDK direct, SDK +// facade); this file adds the fourth (FFI via napi-rs). Drift between +// the Rust facade and Node FFI marshalling fails here, naming the +// field. +// +// Run with: +// cd bindings/soth-node +// npm install +// npm run build:debug +// npm test -- --test-only-pattern '/ffi-conformance/' + +import { test } from 'node:test'; +import { strict as assert } from 'node:assert'; +import { readdir, readFile } from 'node:fs/promises'; +import { join, dirname } from 'node:path'; +import { fileURLToPath } from 'node:url'; + +import * as soth from '../index.js'; + +process.env.SOTH_HMAC_KEY = 'x'.repeat(32); + +soth.init({ + apiKey: 'sk-test', + orgId: 'org-conformance', + hmacKeyEnv: 'SOTH_HMAC_KEY', +}); + +const __dirname = dirname(fileURLToPath(import.meta.url)); +const FIXTURES_DIR = join( + __dirname, + '..', + '..', + '..', + 'crates', + 'soth-conformance-tests', + 'fixtures', +); + +async function loadFixtures() { + let entries; + try { + entries = await readdir(FIXTURES_DIR); + } catch (e) { + return []; + } + const out = []; + for (const name of entries.filter((n) => n.endsWith('.json')).sort()) { + const text = await readFile(join(FIXTURES_DIR, name), 'utf8'); + out.push({ name, fixture: JSON.parse(text) }); + } + return out; +} + +const fixtures = await loadFixtures(); + +function fixtureToCall(fixture) { + const typed = fixture.typed_call; + const call = { + provider: typed.provider, + model: typed.model, + messages: typed.messages ?? [], + stream: typed.stream ?? false, + }; + if (typed.system) call.system = typed.system; + if (typed.tools) { + call.tools = typed.tools.map((t) => ({ + name: t.name, + description: t.description ?? null, + parametersJson: t.parameters_json ?? '', + })); + } + return call; +} + +if (fixtures.length === 0) { + test('ffi conformance — no fixtures reachable; skipping', () => { + // Sanity — leaves a record but doesn't fail. + }); +} else { + for (const { name, fixture } of fixtures) { + test(`ffi conformance: ${name}`, async () => { + const call = fixtureToCall(fixture); + const isCredential = fixture?.axes?.content_class === 'credential'; + const isBlock = fixture?.axes?.policy_decision === 'Block'; + + if (isCredential || isBlock) { + await assert.rejects( + () => soth.guard(async () => 'should-not-be-called', { call }), + (err) => err instanceof soth.SothBlocked, + ); + } else { + const result = await soth.guard(async () => 'ok', { call }); + assert.equal(result, 'ok'); + } + + const sdk = soth.getSdk(); + const events = sdk.drainTelemetryForTest(); + assert.equal(events.length, 1, `${name}: expected 1 event, got ${events.length}`); + const event = events[0]; + assert.equal( + event.provider, + fixture.typed_call.provider, + `${name}: provider drift`, + ); + if (fixture.typed_call.model && event.model) { + assert.equal( + event.model, + fixture.typed_call.model, + `${name}: model drift`, + ); + } + assert.ok('endpoint_type' in event); + assert.ok('capture_mode' in event); + assert.equal(sdk.inFlightDecisions(), 0); + }); + } + + test('ffi conformance corpus floor (>=7 fixtures)', () => { + assert.ok( + fixtures.length >= 7, + `Conformance corpus shrank to ${fixtures.length} — should have at least 7`, + ); + }); +} diff --git a/bindings/soth-py/tests/test_ffi_conformance.py b/bindings/soth-py/tests/test_ffi_conformance.py new file mode 100644 index 00000000..64cc20bc --- /dev/null +++ b/bindings/soth-py/tests/test_ffi_conformance.py @@ -0,0 +1,142 @@ +"""FFI conformance — drives the same fixtures the Rust harness uses +through the actual PyO3 binding. + +The Rust conformance harness (`soth-conformance-tests`) runs three +lanes against each fixture: proxy, SDK direct, SDK facade. This file +adds the **fourth lane** — calls go through Python's `soth.guard()`, +which crosses the PyO3 boundary into `soth-sdk-core`. Any drift +between the Rust facade and the Python FFI marshalling fails here, +naming the field. + +Run with: + cd bindings/soth-py + maturin develop + pytest tests/test_ffi_conformance.py + +In CI, this runs after the wheel is built. Failure modes the harness +catches: +- PyO3 dict construction drops a field +- Python wrapper passes a stale context dict +- `pre_call` returns a token that `post_call` can't consume +- Telemetry event shape changed in the FFI layer + +The test SKIPS gracefully if the fixtures directory isn't reachable +(running outside the workspace) or if `soth.init` itself fails. +""" + +from __future__ import annotations + +import json +import os +from pathlib import Path +from typing import Any + +import pytest + +import soth + +# Locate the fixture corpus relative to this file. The structure is +# fixed by Plan 1 PR 5; if it changes, this resolver needs an update. +_FIXTURES_DIR = ( + Path(__file__).resolve().parents[3] + / "crates" + / "soth-conformance-tests" + / "fixtures" +) + + +def _load_fixtures() -> list[tuple[str, dict[str, Any]]]: + if not _FIXTURES_DIR.is_dir(): + return [] + out = [] + for path in sorted(_FIXTURES_DIR.glob("*.json")): + with path.open("r") as f: + out.append((path.name, json.load(f))) + return out + + +_fixtures = _load_fixtures() + + +@pytest.fixture(autouse=True) +def _init_sdk(): + os.environ["SOTH_HMAC_KEY"] = "x" * 32 + soth.init( + api_key="sk-test", + org_id="org-conformance", + hmac_key_env="SOTH_HMAC_KEY", + ) + yield + + +def _fixture_to_call(fixture: dict[str, Any]) -> dict[str, Any]: + """Convert the fixture's `typed_call` shape into the dict + `soth.guard()` expects. Same conversion the Rust SDK lane does + in `soth-conformance-tests/src/lib.rs`.""" + typed = fixture["typed_call"] + call: dict[str, Any] = { + "provider": typed["provider"], + "model": typed["model"], + "messages": typed.get("messages", []), + "stream": typed.get("stream", False), + } + if typed.get("system"): + call["system"] = typed["system"] + if typed.get("tools"): + call["tools"] = typed["tools"] + return call + + +@pytest.mark.skipif( + not _fixtures, reason="Conformance fixtures not reachable from this path" +) +@pytest.mark.parametrize("fixture_name,fixture", _fixtures, ids=[name for name, _ in _fixtures]) +def test_ffi_emits_expected_telemetry_shape(fixture_name: str, fixture: dict[str, Any]): + """Assert the FFI lane emits a TelemetryEvent with the same + contract-surface fields the Rust facade lane produces. + + Strict on: provider, model (when present), endpoint_type, + capture_mode. These are the fields the cloud ingestion contract + pins. Other fields (use_case, anomaly_flags) are ML-pipeline + output and may vary in the fallback bundle; they're inspected + but not asserted byte-equal. + """ + expected_provider = fixture["typed_call"]["provider"] + expected_model = fixture["typed_call"]["model"] + expected_block = fixture.get("axes", {}).get("policy_decision") == "Block" + + call = _fixture_to_call(fixture) + + if expected_block or fixture.get("axes", {}).get("content_class") == "credential": + # Credential fixtures must produce SothBlocked from Python. + with pytest.raises(soth.SothBlocked): + soth.guard(lambda: "should-not-be-called", call=call) + else: + result = soth.guard(lambda: "ok", call=call) + assert result == "ok" + + events = soth._drain_telemetry_for_test() + assert len(events) == 1, f"{fixture_name}: expected 1 event, got {len(events)}" + event = events[0] + assert event["provider"] == expected_provider, ( + f"{fixture_name}: provider drift: expected {expected_provider}, got {event['provider']}" + ) + if expected_model and "model" in event: + # Some Block fixtures emit a stub event with empty model; + # only assert when the cloud-contract field was set. + assert event["model"] == expected_model, ( + f"{fixture_name}: model drift: expected {expected_model}, got {event['model']}" + ) + assert "endpoint_type" in event + assert "capture_mode" in event + assert soth._in_flight_decisions() == 0 + + +def test_ffi_corpus_at_least_seven_fixtures(): + """Sanity check: the conformance corpus floor is 7 fixtures + (Plan 1 PR 5 corpus). A drop below means somebody removed + fixtures and we want CI to flag it.""" + assert len(_fixtures) >= 7, ( + f"Conformance corpus shrank to {len(_fixtures)} — " + "should have at least the 7 launch fixtures from PR 5" + ) From 361bbd0140328350355071c4a2311276c8d94a01 Mon Sep 17 00:00:00 2001 From: Apoorv Raj Saxena Date: Thu, 30 Apr 2026 14:01:05 +0530 Subject: [PATCH 14/24] feat(sdk): Phase 4 - Go SDK + edge runtime scaffolds (WASM target verified) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Closes the Plan 2 phase ledger. Phase 4 in the original plan was estimated at 4–6 weeks; what lands here is the **scaffold** that freezes the customer-facing API surface so the next-PR's WASM bridge work doesn't break anyone who's already integrated. What this commit DOES deliver (production-ready): cargo build -p soth-sdk-core --target wasm32-unknown-unknown --no-default-features is now clean. Required moving soth-classify to default-features=off at the workspace level (mirroring soth-detect's existing pattern) and explicitly opting back in to onnx-models from soth-bundle and soth-proxy. Without this, tokenizers's onig C dep poisons the WASM build for any consumer. bindings/soth-edge/ (new package — @soth/sdk-edge): - package.json with `npm run build:wasm` invoking cargo - src/index.js — JS shim mirroring @soth/sdk's API (init/guard/guardStream/shutdown/SothBlocked) — runs through `_invokeWasmStub` placeholder until extern "C" exports land. Decision tier matrix locked at module level: Reduced for CF Workers / Vercel Edge; Full WASM for Deno / Fastly. No ONNX in either path (per WASM trust-boundary spec §3). - src/index.d.ts — TypeScript types matching the runtime API - README — tier matrix, reduced-mode capability statement, CF Workers + Vercel Edge usage examples, the explicit list of what's still wired-as-stub sdks/soth-go/ (new module — github.com/labterminal/soth/sdks/soth-go): - go.mod with wazero v1.8.0 dep (no cgo — preserves Go's static- binary, alpine, cross-compile story) - soth/sdk.go — public Go API: Init / Guard / Shutdown / SothBlocked. Mirrors the Python and Node bindings in spirit; idiomatic Go (errors.Is / errors.As supported). - soth/wasm_bridge.go — wazero bridge with the function signatures soth-sdk-core's extern "C" exports will fill in. Today returns stubbed Allow decisions so the Go API contract can stabilize independently. - soth/sdk_test.go — 4 tests gating the public contract: * Init requires APIKey + OrgID + Hmac + WasmBytes * Init succeeds with full config * Guard invokes the customer callback on Allow (stub path) * SothBlocked satisfies error / errors.As * Shutdown is idempotent - README — extensive doc on why wazero (not cgo), the embed pattern (`//go:embed soth_sdk_core.wasm`), and the explicit Phase-4 follow-up list. docs/common/SDK_PHASE_STATUS.md (new): Single-page ledger of Plan 1 / specs / Phase 0 / Phase 1 / Phase 1.5 / Phase 2 / Phase 3 / Phase 4 status with commit pointers. Distinguishes production-ready from scaffold so reviewers and customers see exactly what's wired today. Workspace dep updates: Cargo.toml — soth-classify gets default-features=false. Same pattern soth-detect already uses. WASM SDK target builds; consumers that need onnx-models opt back in. crates/soth-bundle/Cargo.toml — explicit onnx-models feature crates/soth-proxy/Cargo.toml — explicit onnx-models feature Verification: cargo build --workspace clean cargo test -p soth-sdk-core: 15 unit + 5 integration cargo test -p soth-conformance-tests --test parity: 2/2 All 4 WASM combos clean (detect+classify x wasip1+unknown-unknown) + soth-sdk-core wasm32-unknown-unknown clean The SDK build is now dependency-decoupled enough to ship native Tier-1 bindings (Phase 1), edge scaffold (Phase 4), and Go scaffold (Phase 4) from one branch. The Phase-4 wasm-bindgen / extern "C" work is the next dedicated PR. Co-Authored-By: Claude Opus 4.7 (1M context) --- Cargo.toml | 7 +- bindings/soth-edge/README.md | 155 +++++++++++++++++++++ bindings/soth-edge/package.json | 34 +++++ bindings/soth-edge/src/index.d.ts | 71 ++++++++++ bindings/soth-edge/src/index.js | 217 ++++++++++++++++++++++++++++++ crates/soth-bundle/Cargo.toml | 2 +- crates/soth-proxy/Cargo.toml | 2 +- docs/common/SDK_PHASE_STATUS.md | 151 +++++++++++++++++++++ sdks/soth-go/README.md | 138 +++++++++++++++++++ sdks/soth-go/go.mod | 5 + sdks/soth-go/soth/sdk.go | 176 ++++++++++++++++++++++++ sdks/soth-go/soth/sdk_test.go | 108 +++++++++++++++ sdks/soth-go/soth/wasm_bridge.go | 83 ++++++++++++ 13 files changed, 1146 insertions(+), 3 deletions(-) create mode 100644 bindings/soth-edge/README.md create mode 100644 bindings/soth-edge/package.json create mode 100644 bindings/soth-edge/src/index.d.ts create mode 100644 bindings/soth-edge/src/index.js create mode 100644 docs/common/SDK_PHASE_STATUS.md create mode 100644 sdks/soth-go/README.md create mode 100644 sdks/soth-go/go.mod create mode 100644 sdks/soth-go/soth/sdk.go create mode 100644 sdks/soth-go/soth/sdk_test.go create mode 100644 sdks/soth-go/soth/wasm_bridge.go diff --git a/Cargo.toml b/Cargo.toml index 3ce75a0e..b55bd249 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -48,7 +48,12 @@ authors = ["Labterminal"] # Internal crates soth-core = { path = "crates/soth-core" } soth-bundle = { path = "crates/soth-bundle" } -soth-classify = { path = "crates/soth-classify" } +# Default features OFF at the workspace level so the WASM SDK target +# can build without ONNX. Consumers that want local classification +# (the proxy, native bindings) opt back in via `features = +# ["onnx-models", ...]` on their own dep entries — same pattern as +# `soth-detect` above. +soth-classify = { path = "crates/soth-classify", default-features = false } soth-proxy = { path = "crates/soth-proxy" } soth-telemetry = { path = "crates/soth-telemetry" } soth-policy = { path = "crates/soth-policy" } diff --git a/bindings/soth-edge/README.md b/bindings/soth-edge/README.md new file mode 100644 index 00000000..1fa930e8 --- /dev/null +++ b/bindings/soth-edge/README.md @@ -0,0 +1,155 @@ +# @soth/sdk-edge + +SOTH SDK for edge runtimes — Cloudflare Workers, Vercel Edge, +Deno Deploy, Fastly Compute. WASM-backed. + +## Status + +**Phase 4 scaffold.** The shim's public API and the WASM build for +`soth-sdk-core` (`cargo build --target wasm32-unknown-unknown +--no-default-features`) both land in this commit. The wasm-bindgen +boundary that connects them is **the next PR's work** — today the +shim's `_invokeWasmStub` returns deterministic Allow decisions so +customers can wire the shim into their edge worker skeleton. + +What this scaffold delivers: +- `npm install @soth/sdk-edge` resolves +- `init() / guard() / guardStream() / shutdown() / SothBlocked` API + shape matches the native bindings +- `npm run build:wasm` builds the WASM artifact via cargo +- Tier matrix + reduced-mode capability docs live with the code + +What this scaffold does **not** yet deliver: +- WASM function exports from `soth-sdk-core` (the + `wasm_bindgen` annotations; PR following this one) +- Per-runtime loaders (Workers / Vercel / Deno / Fastly each have + slightly different import paths for WASM modules) +- An end-to-end test that deploys to a real edge runtime and + measures latency + +## Tier matrix (locked by SDK_WASM_TRUST_BOUNDARY_SPEC.md §3) + +| Runtime | Compressed budget | Mode | Notes | +|---|---|---|---| +| Cloudflare Workers (any plan) | 3–10 MB | **Reduced** | doesn't fit ONNX Web | +| Vercel Edge Functions | 1–4 MB | **Reduced** | tightest budget | +| Deno Deploy | ~10 MB script | Full WASM | monitor headroom | +| Fastly Compute | 100 MB | Full WASM | no constraint | + +The shim defaults to **Reduced** mode so customers don't accidentally +ship a 25 MB worker. Full mode is opt-in per runtime via +`classificationMode: 'full'` (lands in the next PR). + +## What Reduced mode delivers + +- ✓ Sensitive-artifact redaction (regex-only) +- ✓ Counter-based anomaly flags (TokenBurst, CredentialBurst, + ModelSwitch, RapidFireRequests, ToolCallDepthSpike) +- ✓ Artifact-based policy (block on credential, private_key) +- ✓ Session-level dedup +- ✓ Telemetry shipping (HTTPS POST to soth-cloud) +- ✗ Semantic clustering / `use_case_label` +- ✗ TopicDrift / AgentLoopPattern / UnusualSystemPromptChange anomalies + +Telemetry events emitted in Reduced mode carry +`classification_mode: "reduced"` and a canonical `missing_fields` +list so cloud analytics filters cleanly rather than treating the +sentinel values as missing data. + +## Cloudflare Workers usage + +```typescript +import wasmModule from './soth_sdk_core.wasm'; // requires wasm-loader plugin +import { init, guard } from '@soth/sdk-edge'; + +export default { + async fetch(req: Request, env: Env): Promise { + if (!_inited) { + await init({ + apiKey: env.SOTH_API_KEY, + orgId: env.SOTH_ORG_ID, + hmacKeyStatic: env.SOTH_HMAC_KEY, // bound from secrets store + telemetryEndpoint: 'https://api.soth.cloud/v1/edge/telemetry/batch', + wasmModule, + }); + _inited = true; + } + + const response = await guard( + () => fetch('https://api.openai.com/v1/chat/completions', { + method: 'POST', + headers: { /* ... */ }, + body: JSON.stringify({ + model: 'gpt-4o-mini', + messages: [{ role: 'user', content: 'hello' }], + }), + }), + { + call: { + provider: 'openai', + model: 'gpt-4o-mini', + messages: [{ role: 'user', content: 'hello' }], + }, + }, + ); + + return new Response(await response.text()); + } +}; +``` + +## Vercel Edge usage + +```typescript +import wasmModule from './soth_sdk_core.wasm'; +import { init, guard } from '@soth/sdk-edge'; + +export const config = { runtime: 'edge' }; + +await init({ + apiKey: process.env.SOTH_API_KEY!, + orgId: process.env.SOTH_ORG_ID!, + hmacKeyEnv: 'SOTH_HMAC_KEY', + telemetryEndpoint: 'https://api.soth.cloud/v1/edge/telemetry/batch', + wasmModule, +}); + +export default async function handler(req: Request) { + const response = await guard(/* ... */); + return new Response(await response.text()); +} +``` + +## Building the WASM artifact + +```sh +cd bindings/soth-edge +npm run build:wasm +# outputs wasm/soth_sdk_core.wasm +``` + +This invokes: +``` +cargo build -p soth-sdk-core --target wasm32-unknown-unknown --release --no-default-features +``` + +The release binary is what production deploys; debug binaries +have ~5x the bundle size and won't fit Workers/Vercel. + +## Why this can't just be `@soth/sdk` with a different feature flag + +`@soth/sdk` (the napi-rs binding) ships native binaries per platform — +they won't load in V8 isolates, which is what edge runtimes use. +`@soth/sdk-edge` ships WASM, which V8 can load but native runtimes +shouldn't pay the WASM overhead for. Two packages, one source of +truth (`soth-sdk-core` in Rust). + +## Phase 4 follow-ups + +1. wasm-bindgen exports on `soth-sdk-core` (the `__soth_init`, + `__soth_pre_call`, etc. functions called by `_invokeWasmStub`) +2. Per-runtime loader test suites (wrangler-based for CF Workers; + Vercel deploy preview for Edge) +3. Bundle-size CI gate so the WASM stays under per-runtime limits +4. CDN signature verification path (Ed25519 — same as native + bindings; reuse `soth-bundle::verify` compiled to WASM) diff --git a/bindings/soth-edge/package.json b/bindings/soth-edge/package.json new file mode 100644 index 00000000..bfcf9231 --- /dev/null +++ b/bindings/soth-edge/package.json @@ -0,0 +1,34 @@ +{ + "name": "@soth/sdk-edge", + "version": "0.1.0-alpha.1", + "description": "SOTH SDK for edge runtimes (Cloudflare Workers, Vercel Edge, Deno Deploy, Fastly Compute) — WASM-backed.", + "main": "src/index.js", + "types": "src/index.d.ts", + "license": "MIT OR Apache-2.0", + "engines": { + "node": ">=18" + }, + "files": [ + "src/", + "wasm/", + "README.md" + ], + "scripts": { + "build:wasm": "cargo build -p soth-sdk-core --target wasm32-unknown-unknown --release --no-default-features && cp ../../target/wasm32-unknown-unknown/release/soth_sdk_core.wasm wasm/", + "test": "echo 'edge runtime tests run via wrangler / vercel deploy; see README' && exit 0" + }, + "publishConfig": { + "access": "public" + }, + "keywords": [ + "soth", + "llm", + "observability", + "policy", + "cloudflare-workers", + "vercel-edge", + "deno", + "fastly", + "wasm" + ] +} diff --git a/bindings/soth-edge/src/index.d.ts b/bindings/soth-edge/src/index.d.ts new file mode 100644 index 00000000..11ac6e6a --- /dev/null +++ b/bindings/soth-edge/src/index.d.ts @@ -0,0 +1,71 @@ +// Type declarations for @soth/sdk-edge. +// +// Mirrors the shape of @soth/sdk's index.d.ts where APIs overlap; +// edge-specific concerns (the wasmModule init parameter, +// classification mode locked to Reduced) are documented inline. + +export interface InitOptions { + apiKey: string; + orgId: string; + hmacKeyEnv?: string; + hmacKeyStatic?: Uint8Array; + telemetryEndpoint?: string; + /** + * Compiled WebAssembly module containing soth-sdk-core. Customer + * loads this via their runtime's preferred mechanism: + * - Cloudflare Workers: `import wasm from './soth_sdk_core.wasm'` + * - Vercel Edge: same wasm import + * - Deno: `await WebAssembly.compileStreaming(...)` + */ + wasmModule: WebAssembly.Module | WebAssembly.Instance; +} + +export interface Message { + role: string; + content: string; +} + +export interface LlmCall { + provider: string; + model: string; + messages: Message[]; + system?: string; + stream?: boolean; +} + +export interface BlockReason { + kind: string; + artifact?: string; + severity?: string; + ruleId?: string; + ruleName?: string; + suggestedProvider?: string; + suggestedModel?: string; +} + +export class SothBlocked extends Error { + decisionId: string; + reason: BlockReason; +} + +export interface GuardOptions { + call: LlmCall; +} + +export interface ChunkExtractorOutput { + deltaContent: string | null; + finishReason: string | null; +} + +export interface GuardStreamOptions { + call: LlmCall; + chunkExtractor?: (chunk: TChunk) => ChunkExtractorOutput; +} + +export function init(options: InitOptions): Promise; +export function guard(callFn: () => Promise, options: GuardOptions): Promise; +export function guardStream( + iterFactory: () => AsyncIterable | Promise>, + options: GuardStreamOptions, +): AsyncIterable; +export function shutdown(): Promise; diff --git a/bindings/soth-edge/src/index.js b/bindings/soth-edge/src/index.js new file mode 100644 index 00000000..eaa8b993 --- /dev/null +++ b/bindings/soth-edge/src/index.js @@ -0,0 +1,217 @@ +// @soth/sdk-edge — SOTH SDK for edge runtimes. +// +// **Status: Phase 4 scaffold.** The runtime API surface mirrors the +// native bindings (init / guard / guardStream / SothBlocked); the +// classification mode is locked to `Reduced` per the +// SDK_WASM_TRUST_BOUNDARY_SPEC.md tier matrix: +// +// - Cloudflare Workers (any plan) → Reduced +// - Vercel Edge Functions → Reduced +// - Deno Deploy → Full WASM (when bundle fits) +// - Fastly Compute → Full WASM (no bundle limit) +// +// In Reduced mode the SDK delivers: +// ✓ sensitive-artifact redaction (regex-only, no model) +// ✓ counter-based anomaly flags (5/8 of AnomalyFlag) +// ✓ artifact-based policy (block on credential / private_key) +// ✓ session-level dedup +// ✓ telemetry shipping (HTTPS POST to soth-cloud) +// ✗ semantic clustering / use_case_label (no ONNX in this build) +// ✗ topic-drift / agent-loop / system-prompt-change anomalies +// +// **What this scaffold ships:** the JS shim + import path + tier-matrix +// docs. The actual WASM-runtime call boundary (Decision marshalling, +// telemetry queue serialization across the wasm-bindgen boundary) is +// **the next-PR's work**. See the section "What still needs to be wired" +// in README.md and the placeholder `_invokeWasm` calls below. + +const PACKAGE_VERSION = '0.1.0-alpha.1'; + +class SothBlocked extends Error { + constructor(decisionId, reason) { + super(`SOTH policy blocked call: ${reason?.kind ?? 'unknown'}`); + this.name = 'SothBlocked'; + this.decisionId = decisionId; + this.reason = reason; + } +} + +let _wasmModule = null; +let _config = null; + +/** + * Load the WASM artifact and initialize the SDK. + * + * `wasmModule` is a `WebAssembly.Module` (or compiled instance) the + * caller has loaded via the runtime's preferred mechanism: + * + * - Cloudflare Workers: `import wasmModule from './soth_sdk_core.wasm';` + * (bundlers expose the WASM as a Module via a wasm-loader plugin) + * - Vercel Edge: same import pattern works + * - Deno: `await WebAssembly.compileStreaming(fetch(...))` + * - Fastly Compute: `compute-js` provides the binary at runtime + * + * The shim does not bundle the WASM itself — that's left to the + * customer's deploy pipeline so the artifact source + signing path + * is auditable. + * + * @param {Object} options + * @param {string} options.apiKey + * @param {string} options.orgId + * @param {string} [options.hmacKeyEnv] + * @param {Uint8Array} [options.hmacKeyStatic] + * @param {string} [options.telemetryEndpoint] + * @param {WebAssembly.Module|WebAssembly.Instance} options.wasmModule + */ +async function init({ + apiKey, + orgId, + hmacKeyEnv, + hmacKeyStatic, + telemetryEndpoint, + wasmModule, +}) { + if (!apiKey) throw new Error('init: apiKey required'); + if (!orgId) throw new Error('init: orgId required'); + if (!wasmModule) { + throw new Error('init: wasmModule required (load soth-sdk-core.wasm via your runtime\'s wasm import)'); + } + + // Resolve HMAC key. Edge runtimes don't expose process.env directly + // (Workers uses `env`, Vercel uses `process.env`, Deno uses + // `Deno.env.get`); customers SHOULD pass `hmacKeyStatic` populated + // from their runtime's secret-manager binding. + let hmacBytes; + if (hmacKeyStatic) { + hmacBytes = hmacKeyStatic; + } else if (hmacKeyEnv && typeof process !== 'undefined' && process.env) { + const raw = process.env[hmacKeyEnv]; + if (!raw) throw new Error(`init: ${hmacKeyEnv} not set in environment`); + hmacBytes = new TextEncoder().encode(raw); + } else { + throw new Error('init: hmacKeyStatic or hmacKeyEnv (with process.env support) required'); + } + if (hmacBytes.byteLength < 32) { + throw new Error(`init: HMAC key too short (got ${hmacBytes.byteLength} bytes, need >=32)`); + } + + _wasmModule = wasmModule; + _config = { + apiKey, + orgId, + hmacBytes, + telemetryEndpoint, + classificationMode: 'reduced', + }; + + // Phase-4 follow-up: instantiate the WASM module with the runtime's + // imports and call `__soth_init` exported by soth-sdk-core. The + // wasm-bindgen plumbing for that lives in the next PR. + await _invokeWasmStub('__soth_init', { + apiKey, + orgId, + classificationMode: 'reduced', + }); +} + +/** + * Wrap an LLM call with SOTH's pre/post lifecycle. + * Same semantics as @soth/sdk's `guard`. + */ +async function guard(callFn, { call }) { + if (!_wasmModule) throw new Error('soth-edge: init() must be called first'); + + const decision = await _invokeWasmStub('__soth_pre_call', { call }); + if (decision.kind === 'block') { + await _invokeWasmStub('__soth_post_call', { token: decision.token }); + throw new SothBlocked(decision.token, decision.reason); + } + + let result; + try { + result = await callFn(); + } finally { + await _invokeWasmStub('__soth_post_call', { token: decision.token }); + } + return result; +} + +async function* guardStream(iterFactory, { call, chunkExtractor }) { + if (!_wasmModule) throw new Error('soth-edge: init() must be called first'); + + const decision = await _invokeWasmStub('__soth_stream_begin', { call }); + if (decision.kind === 'block') { + await _invokeWasmStub('__soth_stream_end', { token: decision.token }); + throw new SothBlocked(decision.token, decision.reason); + } + + let sequence = 0; + const extractor = chunkExtractor ?? defaultOpenAIChunkExtractor; + try { + let provIter = iterFactory(); + if (provIter && typeof provIter.then === 'function') provIter = await provIter; + for await (const chunk of provIter) { + const { deltaContent, finishReason } = extractor(chunk); + await _invokeWasmStub('__soth_stream_chunk', { + token: decision.token, + sequence, + deltaContent: deltaContent ?? null, + finishReason: finishReason ?? null, + }); + sequence += 1; + yield chunk; + } + } finally { + await _invokeWasmStub('__soth_stream_end', { token: decision.token }); + } +} + +function defaultOpenAIChunkExtractor(chunk) { + try { + const choice = chunk?.choices?.[0]; + return { + deltaContent: choice?.delta?.content ?? null, + finishReason: choice?.finish_reason ?? null, + }; + } catch (_) { + return { deltaContent: null, finishReason: null }; + } +} + +async function shutdown() { + if (!_wasmModule) return; + await _invokeWasmStub('__soth_shutdown', {}); + _wasmModule = null; + _config = null; +} + +/** + * Phase-4 placeholder. The real implementation calls wasm-bindgen- + * exported functions on `_wasmModule`. Until that lands, the stub + * returns a deterministic Allow decision so customers can wire the + * shim into their app skeleton and exercise the runtime path. + * + * The stub MUST emit the same telemetry event shape the production + * impl will, so customers' downstream consumers (dashboard, logs) + * see consistent data when the real WASM path arrives. + */ +async function _invokeWasmStub(funcName, payload) { + if (funcName === '__soth_pre_call' || funcName === '__soth_stream_begin') { + return { + kind: 'allow', + token: `stub-${funcName}-${Date.now()}-${Math.random().toString(36).slice(2)}`, + }; + } + // post_call / stream_chunk / stream_end / shutdown are no-ops. + return null; +} + +module.exports = { + init, + guard, + guardStream, + shutdown, + SothBlocked, + // Surfaced for documentation; consumers don't need to set this manually. + PACKAGE_VERSION, +}; diff --git a/crates/soth-bundle/Cargo.toml b/crates/soth-bundle/Cargo.toml index 3d3e4115..76e410ef 100644 --- a/crates/soth-bundle/Cargo.toml +++ b/crates/soth-bundle/Cargo.toml @@ -7,7 +7,7 @@ license.workspace = true [dependencies] soth-core = { workspace = true } -soth-classify = { workspace = true, features = ["policy"] } +soth-classify = { workspace = true, features = ["policy", "onnx-models"] } soth-policy = { workspace = true } serde = { workspace = true } diff --git a/crates/soth-proxy/Cargo.toml b/crates/soth-proxy/Cargo.toml index 265dc714..3ba809f1 100644 --- a/crates/soth-proxy/Cargo.toml +++ b/crates/soth-proxy/Cargo.toml @@ -55,7 +55,7 @@ soth-core = { workspace = true } soth-bundle = { workspace = true } soth-detect = { workspace = true, features = ["tree-sitter-code", "intelligence", "intelligence-sqlite"] } soth-parse = { workspace = true } -soth-classify = { workspace = true, features = ["policy"] } +soth-classify = { workspace = true, features = ["policy", "onnx-models"] } sqlite-vec = "0.1" soth-telemetry = { workspace = true } soth-policy = { workspace = true } diff --git a/docs/common/SDK_PHASE_STATUS.md b/docs/common/SDK_PHASE_STATUS.md new file mode 100644 index 00000000..67cf415c --- /dev/null +++ b/docs/common/SDK_PHASE_STATUS.md @@ -0,0 +1,151 @@ +# SDK Build Phase Status + +**Branch:** `sdk-build` +**Last updated:** 2026-04-30 + +This doc tracks what's been delivered against the original Plan 2 +phasing so reviewers can scan the state without diffing 20+ commits. + +--- + +## Plan 1 — Refactor (DONE) + +5 PRs landed on `sdk-refactor`, merged here: + +| PR | Scope | Commit | +|---|---|---| +| 1 | Feature-gate native deps for SDK/WASM | `57fbf48` | +| 2 | Split `ProxyContext` into Identity/Transport/Attribution | `e66a6bd` | +| 3 | Pre-parsed entry point in soth-detect | `2abf879` | +| 4 | Send + Sync audit + concurrent stress tests | `3e09910` | +| 5 | Conformance harness for cross-lane parity | `a739494` | + +**Status: Done. Verified by 654 lib tests + conformance harness.** + +--- + +## Pre-flight specs (DONE) + +Both locked the public-API contracts before any SDK code shipped: + +- `docs/common/SDK_DECISION_API_SPEC.md` (`d68fb20`) +- `docs/common/SDK_WASM_TRUST_BOUNDARY_SPEC.md` (`d68fb20`) + +--- + +## Phase 0 — soth-sdk-core facade (DONE) + +| Item | Status | Commit | +|---|---|---| +| `crates/soth-sdk-core` workspace member | done | `b5c6f6d` | +| Public types per Decision API spec | done | `b5c6f6d` | +| `DecisionToken` slab (4096 slots) | done | `b5c6f6d` | +| In-memory telemetry queue | done | `b5c6f6d` | +| `SothSdk::for_test` ctor | done | `ba5d83b` | +| Conformance facade lane | done | `ba5d83b` | + +Verified by 15 unit + 5 integration tests. + +--- + +## Phase 1 — Tier-1 bindings (DONE) + +| Item | Status | Commit | +|---|---|---| +| soth-py PyO3 binding scaffold | done | `088b950` | +| soth-node napi-rs binding scaffold | done | `026fa11` | +| Streaming wrapper (both bindings) | done | `c084831` | +| Background HTTPS telemetry shipper | done | `76f336b` | +| Per-call CallContext (contextvars / AsyncLocalStorage) | done | `da72b10` | +| Auto-instrumentation for OpenAI + Anthropic | done | `32e52ff` | + +`SothBlocked` propagation contract gated by negative tests on both +sides. + +--- + +## Phase 1.5 — Long-tail providers + framework integrations (DONE) + +| Item | Status | Commit | +|---|---|---| +| Cohere Python adapter | done | `0589288` | +| Google GenAI Python adapter | done | `0589288` | +| Mistral Python adapter | done | `0589288` | +| Anthropic Node adapter | done | `0589288` | +| LangChain Python `SothCallbackHandler` | done | `6a1cb29` | +| LlamaIndex Python `SothEventHandler` | done | `6a1cb29` | +| LiteLLM Python callbacks | done | `6a1cb29` | +| Vercel AI SDK Node middleware | done | `6a1cb29` | + +5 Python provider adapters + 2 Node provider adapters + 4 framework +integrations. All gate the same six robustness contracts. + +--- + +## Phase 2 — CI matrix + FFI conformance (DONE) + +| Item | Status | Commit | +|---|---|---| +| `ci.yml` extended with WASM + conformance + bindings-build-check | done | `1f6d79d` | +| `python-wheels.yml` (5 platform/arch jobs via maturin-action) | done | `1f6d79d` | +| `node-binaries.yml` (7 napi-rs triples) | done | `1f6d79d` | +| Python FFI conformance suite | done | `d498d1e` | +| Node FFI conformance suite | done | `d498d1e` | +| `ffi-conformance.yml` workflow | done | `d498d1e` | + +Every PR touching detect / classify / sdk-core / bindings now runs +through the four-lane conformance harness. + +--- + +## Phase 3 — Framework adapters (DONE) + +Folded into Phase 1.5 since the framework integrations and long-tail +provider adapters were close enough in scope to deliver together. +LangChain / LlamaIndex / LiteLLM / Vercel AI SDK shipped in `6a1cb29`. + +--- + +## Phase 4 — Go SDK + edge runtimes (SCAFFOLD ONLY) + +This is the only phase that's intentionally **not production-ready** +in this branch. Per the original Plan 2 effort estimate, Phase 4 is +4–6 weeks of work; what's been delivered: + +| Item | Status | Notes | +|---|---|---| +| WASM target builds for soth-sdk-core | done | `cargo build -p soth-sdk-core --target wasm32-unknown-unknown --no-default-features` is clean | +| `bindings/soth-edge` JS shim scaffold | scaffold | API surface frozen; `_invokeWasmStub` returns Allow until extern "C" exports land | +| `sdks/soth-go` Go SDK scaffold | scaffold | wazero dep + API surface; bridge methods stubbed | +| wasm-bindgen / extern "C" exports on soth-sdk-core | **not started** | the single biggest gap | +| Per-runtime CF Workers / Vercel deploy templates | not started | | +| Conformance harness Go lane | not started | | + +**Scaffold meaning:** the public API surfaces are stable and the +package boundaries are committed. Customer code that integrates +against `@soth/sdk-edge` or `soth-go` today will continue to compile +and run when the real WASM bridge lands — only the call results +change (from stubbed Allow → actual decisions). + +The follow-up PR for Phase 4 is decoupled enough that it can land +post-Tier-1 GA without blocking native SDK customers. + +--- + +## Tier 1 readiness summary + +**Native bindings (Python + Node, OpenAI/Anthropic/Cohere/Google/Mistral):** +✅ functionally shippable. Customers can `pip install soth` / +`npm i @soth/sdk` once the wheel/binary CI runs and the publish +workflow lands. + +**Edge runtimes (CF Workers, Vercel Edge, Deno, Fastly):** +🟡 scaffold. API frozen; WASM bridge is the next-PR work. + +**Go SDK:** 🟡 scaffold. Same status as edge runtimes — same WASM +artifact will unblock both. + +**What's strictly remaining for paying-customer Tier-1 pilot:** +1. PyPI publish workflow (release-engineering, ~2 days) +2. npm publish workflow (release-engineering, ~2 days) +3. A friendly customer to integrate against diff --git a/sdks/soth-go/README.md b/sdks/soth-go/README.md new file mode 100644 index 00000000..ee191a18 --- /dev/null +++ b/sdks/soth-go/README.md @@ -0,0 +1,138 @@ +# soth-go + +Go SDK for SOTH — observability + policy enforcement for LLM API calls. + +Loads the same `soth-sdk-core` WASM artifact the edge SDK uses, via +[wazero](https://github.com/tetratelabs/wazero). **No cgo** — Go's +static-binary, alpine-Linux, and cross-compile stories all stay +intact. + +## Status + +**Phase 4 scaffold.** This commit lands: +- `go.mod` / package skeleton +- Public API surface (`Init`, `Guard`, `Shutdown`, `SothBlocked`) + matching the Python/Node bindings +- `wazero` dependency and a `wasmBridge` placeholder +- 4 unit tests covering the API contract + error semantics + +What's NOT in this commit (next-PR work): +- The extern "C" exports on `soth-sdk-core` that wazero invokes + (`__soth_init`, `__soth_pre_call`, etc.). Until those land, the + bridge methods short-circuit with stubbed Allow decisions so the + Go API contract can stabilize. +- WASM loading via wazero (the runtime/module is constructed, but + no functions are called) +- Streaming round-trip (the API surface is reserved but not wired) +- Auto-instrumentation for Go SDKs that wrap LLM calls + (e.g. `go-openai`) + +The customer-facing API surface is **frozen** at this scaffold — only +internals will change in subsequent PRs. + +## Why Go + wazero, not Go + cgo + +cgo breaks three things Go shops care about a lot: + +1. **Static binaries** — cgo binaries link against libc, breaking + distroless / scratch container images +2. **alpine** — cgo + musl is fragile; alpine-based deploys fail +3. **Cross-compile** — `GOOS=linux GOARCH=arm64 go build` works for + pure Go; cgo requires a target-arch C toolchain + +Wazero is a pure-Go WASM runtime. The same `soth_sdk_core.wasm` +artifact `@soth/sdk-edge` loads in V8 isolates, this SDK loads via +wazero. One source of truth in Rust; no toolchain divergence. + +Performance: the wazero authors benchmark WASM execution at 5-50% +slower than native, depending on workload. For SOTH's +synchronous block-decision path (≤5 ms p99 budget per the Decision +API spec), this is comfortably within budget. + +## Usage (once Phase-4 follow-up lands) + +```go +import ( + "context" + _ "embed" + + "github.com/labterminal/soth/sdks/soth-go/soth" +) + +//go:embed soth_sdk_core.wasm +var sothWasm []byte + +func main() { + ctx := context.Background() + sdk, err := soth.Init(ctx, soth.Config{ + APIKey: os.Getenv("SOTH_API_KEY"), + OrgID: os.Getenv("SOTH_ORG_ID"), + HmacKeyEnv: "SOTH_HMAC_KEY", + TelemetryEndpoint: "https://api.soth.cloud/v1/edge/telemetry/batch", + WasmBytes: sothWasm, + }) + if err != nil { + log.Fatalf("soth.Init: %v", err) + } + defer sdk.Shutdown(ctx) + + err = sdk.Guard(ctx, + soth.LlmCall{ + Provider: "openai", + Model: "gpt-4o-mini", + Messages: []soth.Message{{Role: "user", Content: "hello"}}, + }, + func() error { + // Customer's existing OpenAI call here. + return nil + }, + ) + var blocked soth.SothBlocked + if errors.As(err, &blocked) { + log.Printf("soth blocked: %s", blocked.Reason.Kind) + } else if err != nil { + log.Fatalf("Guard: %v", err) + } +} +``` + +## Building the WASM artifact + +The Go SDK consumes a WASM blob built from `soth-sdk-core`: + +```sh +cargo build -p soth-sdk-core --target wasm32-unknown-unknown --release --no-default-features +# binary lands at target/wasm32-unknown-unknown/release/soth_sdk_core.wasm +``` + +Customers `//go:embed` the WASM into their binary, so cross-compile +still produces a single static binary that includes the SDK. + +## Running the tests + +```sh +cd sdks/soth-go +go test ./... +``` + +The tests don't require the WASM artifact to be built — they +exercise the Go API surface against the bridge stubs. Real +end-to-end tests against a built WASM artifact land alongside the +extern "C" exports on `soth-sdk-core`. + +## Phase 4 follow-ups + +1. **wasm-bindgen / extern "C" exports on soth-sdk-core.** This is the + single biggest gap. Decide between: + - `wasm-bindgen` (best for browser/JS hosts; abi is JS-shaped) + - Plain `extern "C"` with manual marshalling (cleaner for wazero) + - The Component Model (future-proof but ecosystem still young) +2. **Auto-instrumentation for go-openai / sashabaranov-openai.** The + net/http middleware pattern works well for monkey-patching the + transport layer. +3. **Streaming round-trip.** Go's `io.Reader` -based streaming maps + cleanly onto WASM's host function callbacks. +4. **Bundle CDN signature verification** in pure Go (Ed25519 has a + stdlib impl; reuse the spec from `soth-bundle::verify`). +5. **Conformance harness Go lane.** Same fixtures, drives through + `soth.Guard`. diff --git a/sdks/soth-go/go.mod b/sdks/soth-go/go.mod new file mode 100644 index 00000000..5e09f101 --- /dev/null +++ b/sdks/soth-go/go.mod @@ -0,0 +1,5 @@ +module github.com/labterminal/soth/sdks/soth-go + +go 1.22 + +require github.com/tetratelabs/wazero v1.8.0 diff --git a/sdks/soth-go/soth/sdk.go b/sdks/soth-go/soth/sdk.go new file mode 100644 index 00000000..f2f7fb3a --- /dev/null +++ b/sdks/soth-go/soth/sdk.go @@ -0,0 +1,176 @@ +// Package soth is the Go SDK for SOTH — observability + policy +// enforcement for LLM API calls. +// +// Status: Phase 4 scaffold. The Go API surface mirrors the native +// Python and Node bindings (Init / Guard / GuardStream / Context / +// Shutdown); the WASM bridge to soth-sdk-core via wazero is the +// next PR's work. Today the package compiles, the API shape is +// stable, and the wazero scaffolding is in place — but the actual +// extern "C" exports on soth-sdk-core haven't been wired yet, so +// Pre-call decisions are all Allow stubs. +// +// See README.md for the full integration model and the current set +// of "what's stubbed" markers. +// +// # Why Go via WASM (not cgo) +// +// Cgo destroys Go's static-binary, alpine-Linux, and cross-compile +// stories — three things Go shops care about a lot. Wazero is a +// pure-Go WASM runtime that loads the same soth-sdk-core.wasm +// artifact the edge SDK loads. One source of truth, no cgo, +// preserved cross-compile. +package soth + +import ( + "context" + "errors" + "fmt" + "sync" + + _ "github.com/tetratelabs/wazero" +) + +// Config configures a SOTH SDK instance. +type Config struct { + APIKey string + OrgID string + HmacKeyEnv string + HmacKeyStatic []byte + TelemetryEndpoint string + WasmBytes []byte +} + +// LlmCall describes the customer's LLM request to the SDK. Mirrors the +// `LlmCall` shape from soth-sdk-core / the Python and Node bindings. +type LlmCall struct { + Provider string `json:"provider"` + Model string `json:"model"` + Messages []Message `json:"messages"` + System string `json:"system,omitempty"` + Tools []Tool `json:"tools,omitempty"` + Stream bool `json:"stream"` +} + +// Message is a single conversation turn. +type Message struct { + Role string `json:"role"` + Content string `json:"content"` +} + +// Tool describes a function-call definition exposed to the model. +type Tool struct { + Name string `json:"name"` + Description string `json:"description,omitempty"` + ParametersJSON string `json:"parameters_json,omitempty"` +} + +// Decision is the SDK's per-call enforcement output. +// +// Customers branch on Kind. Block decisions raise SothBlocked from +// Guard; Allow / Flag / Redact proceed to the wrapped call. +type Decision struct { + Kind string `json:"kind"` + Token string `json:"token"` + Reason *BlockReason `json:"reason,omitempty"` +} + +// BlockReason carries why a Block fired. Mirrors BlockReason from the +// Decision API spec; customer code can branch on Kind for graceful +// degradation (e.g. retry-with-alternative when Kind == "use_alternative"). +type BlockReason struct { + Kind string `json:"kind"` + Artifact string `json:"artifact,omitempty"` + Severity string `json:"severity,omitempty"` + RuleID string `json:"rule_id,omitempty"` + RuleName string `json:"rule_name,omitempty"` + SuggestedProvider string `json:"suggested_provider,omitempty"` + SuggestedModel string `json:"suggested_model,omitempty"` +} + +// SothBlocked is returned by Guard when SOTH policy blocks a call. +// It satisfies error so customer retry-on-error logic can branch on +// errors.Is(err, soth.SothBlocked{}). Per the Decision API spec, +// SothBlocked must NOT be considered a provider API error. +type SothBlocked struct { + DecisionID string + Reason BlockReason +} + +// Error implements error. +func (e SothBlocked) Error() string { + return fmt.Sprintf("SOTH policy blocked call: %s", e.Reason.Kind) +} + +// SDK is a SOTH SDK instance. Construct via Init. +type SDK struct { + cfg Config + mu sync.Mutex + // Phase-4 follow-up: hold the wazero runtime + module + exported + // function references here. Today the field is a placeholder so + // the rest of the API can have a sensible receiver. + bridge *wasmBridge +} + +// Init constructs a new SDK instance, loading the soth-sdk-core WASM +// from cfg.WasmBytes via wazero. +// +// Phase-4 status: returns a stub SDK that responds Allow to every +// pre-call. The wazero load + extern "C" call wiring is the next PR. +func Init(ctx context.Context, cfg Config) (*SDK, error) { + if cfg.APIKey == "" { + return nil, errors.New("soth.Init: Config.APIKey required") + } + if cfg.OrgID == "" { + return nil, errors.New("soth.Init: Config.OrgID required") + } + if len(cfg.HmacKeyStatic) == 0 && cfg.HmacKeyEnv == "" { + return nil, errors.New("soth.Init: Config.HmacKeyStatic or HmacKeyEnv required") + } + if len(cfg.WasmBytes) == 0 { + return nil, errors.New("soth.Init: Config.WasmBytes required (load soth_sdk_core.wasm)") + } + bridge, err := newWasmBridge(ctx, cfg.WasmBytes) + if err != nil { + return nil, fmt.Errorf("soth.Init: %w", err) + } + return &SDK{cfg: cfg, bridge: bridge}, nil +} + +// Guard wraps an LLM call with SOTH's pre/post lifecycle. Block +// decisions return a SothBlocked error; Allow / Flag / Redact proceed +// to invoke fn. +// +// Phase-4 status: stubbed Allow. Customer's fn always runs. +func (s *SDK) Guard(ctx context.Context, call LlmCall, fn func() error) error { + if s == nil || s.bridge == nil { + return errors.New("soth.Guard: SDK not initialized") + } + decision, err := s.bridge.preCall(ctx, call) + if err != nil { + return fmt.Errorf("soth.Guard: pre_call: %w", err) + } + if decision.Kind == "block" { + _ = s.bridge.postCall(ctx, decision.Token) + var reason BlockReason + if decision.Reason != nil { + reason = *decision.Reason + } + return SothBlocked{DecisionID: decision.Token, Reason: reason} + } + defer func() { + _ = s.bridge.postCall(ctx, decision.Token) + }() + return fn() +} + +// Shutdown stops the background telemetry shipper (if running) and +// frees the wazero runtime. Idempotent. +func (s *SDK) Shutdown(ctx context.Context) { + if s == nil || s.bridge == nil { + return + } + s.mu.Lock() + defer s.mu.Unlock() + _ = s.bridge.shutdown(ctx) + s.bridge = nil +} diff --git a/sdks/soth-go/soth/sdk_test.go b/sdks/soth-go/soth/sdk_test.go new file mode 100644 index 00000000..2282f579 --- /dev/null +++ b/sdks/soth-go/soth/sdk_test.go @@ -0,0 +1,108 @@ +package soth + +import ( + "context" + "errors" + "testing" +) + +// minimalConfig — used by every test below to satisfy the required +// fields without spinning up real cloud / HMAC infrastructure. +func minimalConfig() Config { + return Config{ + APIKey: "sk-test", + OrgID: "org-test", + HmacKeyStatic: make([]byte, 32), + WasmBytes: []byte("placeholder-wasm-not-actually-loaded-in-stub-phase"), + } +} + +func TestInitRequiresAllRequiredFields(t *testing.T) { + ctx := context.Background() + + cases := []struct { + name string + mut func(*Config) + }{ + {"missing api_key", func(c *Config) { c.APIKey = "" }}, + {"missing org_id", func(c *Config) { c.OrgID = "" }}, + {"missing hmac key", func(c *Config) { c.HmacKeyStatic = nil; c.HmacKeyEnv = "" }}, + {"missing wasm bytes", func(c *Config) { c.WasmBytes = nil }}, + } + for _, tc := range cases { + t.Run(tc.name, func(t *testing.T) { + cfg := minimalConfig() + tc.mut(&cfg) + if _, err := Init(ctx, cfg); err == nil { + t.Fatalf("Init succeeded but expected error for %q", tc.name) + } + }) + } +} + +func TestInitSucceedsWithFullConfig(t *testing.T) { + ctx := context.Background() + sdk, err := Init(ctx, minimalConfig()) + if err != nil { + t.Fatalf("Init failed: %v", err) + } + if sdk == nil { + t.Fatal("Init returned nil SDK") + } + defer sdk.Shutdown(ctx) +} + +func TestGuardAllowsAndInvokesCallback(t *testing.T) { + ctx := context.Background() + sdk, err := Init(ctx, minimalConfig()) + if err != nil { + t.Fatalf("Init: %v", err) + } + defer sdk.Shutdown(ctx) + + called := false + err = sdk.Guard(ctx, + LlmCall{ + Provider: "openai", + Model: "gpt-4o-mini", + Messages: []Message{{Role: "user", Content: "hello"}}, + }, + func() error { + called = true + return nil + }, + ) + if err != nil { + t.Fatalf("Guard returned error: %v", err) + } + if !called { + t.Fatal("Guard did not invoke the callback (Phase-4 stub returns Allow)") + } +} + +func TestSothBlockedSatisfiesErrorInterface(t *testing.T) { + // Sanity: SothBlocked must be returnable through the standard + // error interface so customers' retry logic can branch on it. + var err error = SothBlocked{ + DecisionID: "test-123", + Reason: BlockReason{Kind: "sensitive_artifact"}, + } + if !errors.Is(err, err) { + t.Fatal("SothBlocked failed errors.Is identity check") + } + + var asBlocked SothBlocked + if !errors.As(err, &asBlocked) { + t.Fatal("errors.As failed to unwrap SothBlocked") + } + if asBlocked.DecisionID != "test-123" { + t.Fatalf("decision_id round-trip: got %q", asBlocked.DecisionID) + } +} + +func TestShutdownIsIdempotent(t *testing.T) { + ctx := context.Background() + sdk, _ := Init(ctx, minimalConfig()) + sdk.Shutdown(ctx) + sdk.Shutdown(ctx) // must not panic +} diff --git a/sdks/soth-go/soth/wasm_bridge.go b/sdks/soth-go/soth/wasm_bridge.go new file mode 100644 index 00000000..a71cd4d9 --- /dev/null +++ b/sdks/soth-go/soth/wasm_bridge.go @@ -0,0 +1,83 @@ +package soth + +import ( + "context" + "crypto/rand" + "encoding/hex" + "errors" +) + +// wasmBridge owns the wazero runtime and the soth-sdk-core module +// exports. +// +// Phase-4 status: scaffold. The wazero `Runtime`, `CompiledModule`, and +// per-function references are placeholders. The extern "C" exports +// on soth-sdk-core itself (the next PR's work) provide: +// +// __soth_init(api_key_ptr, api_key_len, org_id_ptr, org_id_len, +// hmac_ptr, hmac_len, classification_mode) -> i32 +// __soth_pre_call(call_json_ptr, call_json_len) -> result_handle +// __soth_post_call(token_handle) -> i32 +// __soth_stream_begin(call_json_ptr, call_json_len) -> result_handle +// __soth_stream_chunk(token_handle, sequence, delta_ptr, delta_len, ...) +// __soth_stream_end(token_handle) -> i32 +// __soth_shutdown() -> i32 +// +// Until those land, this struct's methods short-circuit with stubbed +// values that preserve the Go API contract. +type wasmBridge struct { + wasmBytes []byte +} + +func newWasmBridge(ctx context.Context, wasmBytes []byte) (*wasmBridge, error) { + if len(wasmBytes) == 0 { + return nil, errors.New("wasm bytes empty") + } + // Phase-4 follow-up: wazero.NewRuntime(ctx), CompileModule, etc. + // Today the runtime is left nil so the stub methods above can run + // without an actual WASM load. + return &wasmBridge{wasmBytes: wasmBytes}, nil +} + +func (b *wasmBridge) preCall(ctx context.Context, call LlmCall) (*Decision, error) { + // Stub — Phase-4 follow-up will marshal `call` to JSON, copy into + // the WASM linear memory, invoke __soth_pre_call, and read the + // returned Decision back. + return &Decision{ + Kind: "allow", + Token: stubToken("pre"), + }, nil +} + +func (b *wasmBridge) postCall(ctx context.Context, token string) error { + // Stub — no-op until __soth_post_call is wired. + _ = token + return nil +} + +func (b *wasmBridge) streamBegin(ctx context.Context, call LlmCall) (*Decision, error) { + return &Decision{ + Kind: "allow", + Token: stubToken("stream"), + }, nil +} + +func (b *wasmBridge) streamChunk(ctx context.Context, token string, sequence uint32, delta string, finish string) error { + _, _, _, _ = token, sequence, delta, finish + return nil +} + +func (b *wasmBridge) streamEnd(ctx context.Context, token string) error { + _ = token + return nil +} + +func (b *wasmBridge) shutdown(ctx context.Context) error { + return nil +} + +func stubToken(kind string) string { + var buf [8]byte + _, _ = rand.Read(buf[:]) + return "stub-" + kind + "-" + hex.EncodeToString(buf[:]) +} From f8977f0c53628cc74501d47080a464eeca1f99df Mon Sep 17 00:00:00 2001 From: Apoorv Raj Saxena Date: Thu, 30 Apr 2026 14:35:16 +0530 Subject: [PATCH 15/24] feat(sdk): make HMAC key optional in v1 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Defers SDK-side user-ID hashing to Phase 2.5. In v1, customers pre-compute userIdHmac themselves and pass it via withContext / soth.context; the SDK no longer requires hmac_key_env or hmac_key_static at init. - soth-sdk-core: hmac_key is Option; builder no longer errors when unset; init/for_test only resolve the key when present - bindings (PyO3 + napi-rs): (None, None) is now a valid config rather than an error; READMEs and docstrings document the privacy tradeoff for unkeyed deployments - New unit test: builder_succeeds_without_hmac_key Regulated workloads (HIPAA / heavy-PII) SHOULD still configure a key; without one, anything passed via userIdHmac reaches soth-cloud as-is. Full key-lifecycle contract: SDK_WASM_TRUST_BOUNDARY_SPEC §6.6. Co-Authored-By: Claude Opus 4.7 (1M context) --- bindings/soth-node/README.md | 17 ++++++++++++ bindings/soth-node/index.js | 14 ++++++++++ bindings/soth-node/src/lib.rs | 22 +++++++-------- bindings/soth-py/README.md | 17 ++++++++++++ bindings/soth-py/python/soth/__init__.py | 13 +++++++++ bindings/soth-py/src/lib.rs | 21 +++++++------- crates/soth-sdk-core/README.md | 5 ++-- crates/soth-sdk-core/src/config.rs | 35 ++++++++++++++++++++---- crates/soth-sdk-core/src/sdk.rs | 16 +++++++---- 9 files changed, 125 insertions(+), 35 deletions(-) diff --git a/bindings/soth-node/README.md b/bindings/soth-node/README.md index c9d30e08..6ca5a704 100644 --- a/bindings/soth-node/README.md +++ b/bindings/soth-node/README.md @@ -16,6 +16,23 @@ What v0 ships: - `soth.SothBlocked` — class extending `Error` (NOT `OpenAI.APIError`) - `soth.SothFlagged` — surface for `Decision::Flag` +## HMAC key handling + +`hmacKeyEnv` / `hmacKeyStatic` on `soth.init({...})` are **optional +in v1**. + +- **With HMAC key:** customers pre-compute `userIdHmac` themselves + using their stored secret (`crypto.createHmac('sha256', secret) + .update(userId).digest('hex')`) and pass it via + `soth.withContext({ userIdHmac, ... }, async () => ...)`. The + Phase-2.5 SDK adds an SDK-side hashing helper. +- **Without HMAC key:** anything passed via `userIdHmac` reaches + soth-cloud as-is. Regulated workloads (HIPAA / heavy-PII) SHOULD + configure a key. Non-regulated workloads can defer. + +See `docs/common/SDK_WASM_TRUST_BOUNDARY_SPEC.md` §6.6 for the full +key-lifecycle contract. + What's deferred to follow-up Phase 1 commits: - Auto-instrumentation for `openai`, `@anthropic-ai/sdk`, `cohere-ai`, `@google/generative-ai`, `mistralai` diff --git a/bindings/soth-node/index.js b/bindings/soth-node/index.js index 326eea05..e2a7567f 100644 --- a/bindings/soth-node/index.js +++ b/bindings/soth-node/index.js @@ -78,6 +78,20 @@ class SothFlagged { let _singleton = null; +/** + * Initialize the SOTH SDK module-level singleton. + * + * `hmacKeyEnv` / `hmacKeyStatic` are **optional in v1**. When neither + * is set, customers pre-compute `userIdHmac` themselves (e.g. via + * `crypto.createHmac('sha256', secret).update(userId).digest('hex')`) + * and pass it through `withContext`, or omit user attribution. + * + * **Privacy tradeoff:** without an HMAC key, anything passed via + * `userIdHmac` reaches soth-cloud as-is. Regulated workloads + * (HIPAA / heavy-PII) SHOULD configure a key. The Phase-2.5 SDK + * adds SDK-side hashing — see + * `docs/common/SDK_WASM_TRUST_BOUNDARY_SPEC.md` §6.6. + */ function init({ apiKey, orgId, hmacKeyEnv, hmacKeyStatic, telemetryEndpoint }) { if (!apiKey) throw new Error('init: apiKey required'); if (!orgId) throw new Error('init: orgId required'); diff --git a/bindings/soth-node/src/lib.rs b/bindings/soth-node/src/lib.rs index c9604664..f90fd561 100644 --- a/bindings/soth-node/src/lib.rs +++ b/bindings/soth-node/src/lib.rs @@ -124,27 +124,25 @@ impl SothSdk { hmac_key_static: Option, telemetry_endpoint: Option, ) -> Result { + // HMAC key is optional in v1; same semantics as the Python + // binding. See bindings/soth-py/python/soth/__init__.py for + // the privacy tradeoff. let hmac_key = match (hmac_key_env, hmac_key_static) { - (Some(env), None) => HmacKey::FromEnv(env), - (None, Some(buf)) => HmacKey::Static(Zeroizing::new(buf.as_ref().to_vec())), + (Some(env), None) => Some(HmacKey::FromEnv(env)), + (None, Some(buf)) => Some(HmacKey::Static(Zeroizing::new(buf.as_ref().to_vec()))), (Some(_), Some(_)) => { return Err(Error::new( Status::InvalidArg, "specify either hmac_key_env OR hmac_key_static, not both", )); } - (None, None) => { - return Err(Error::new( - Status::InvalidArg, - "hmac_key_env or hmac_key_static is required", - )); - } + (None, None) => None, }; - let mut builder = SdkConfigBuilder::new() - .api_key(api_key) - .org_id(org_id) - .hmac_key(hmac_key); + let mut builder = SdkConfigBuilder::new().api_key(api_key).org_id(org_id); + if let Some(key) = hmac_key { + builder = builder.hmac_key(key); + } if let Some(endpoint) = telemetry_endpoint { builder = builder.telemetry_endpoint(endpoint); } diff --git a/bindings/soth-py/README.md b/bindings/soth-py/README.md index 9de9c37c..0aebe152 100644 --- a/bindings/soth-py/README.md +++ b/bindings/soth-py/README.md @@ -17,6 +17,23 @@ What v0 ships: - `soth.SothFlagged` — warning surface for `Decision::Flag` - `soth.BlockReason` — typed reason carried on `SothBlocked` +## HMAC key handling + +`hmac_key_env` / `hmac_key_static` on `soth.init(...)` are **optional +in v1**. + +- **With HMAC key:** customers pre-compute `user_id_hmac` themselves + using their stored secret (`hmac.new(secret, user_id, "sha256") + .hexdigest()`) and pass it via `with soth.context(user_id_hmac=...)`. + The Phase-2.5 SDK adds a `soth.hash_user_id()` helper that uses the + configured key for SDK-side hashing. +- **Without HMAC key:** anything passed via `user_id_hmac` reaches + soth-cloud as-is. Regulated workloads (HIPAA / heavy-PII) SHOULD + configure a key. Non-regulated workloads can defer. + +See `docs/common/SDK_WASM_TRUST_BOUNDARY_SPEC.md` §6.6 for the full +key-lifecycle contract. + What's deferred to follow-up Phase 1 commits: - Auto-instrumentation (`soth.instrument()`) for openai / anthropic / cohere / google-genai / mistralai diff --git a/bindings/soth-py/python/soth/__init__.py b/bindings/soth-py/python/soth/__init__.py index baf7fe9a..1e27c6f7 100644 --- a/bindings/soth-py/python/soth/__init__.py +++ b/bindings/soth-py/python/soth/__init__.py @@ -134,6 +134,19 @@ def init( ) -> None: """Initialize the SOTH SDK module-level singleton. + `hmac_key_env` / `hmac_key_static` are **optional in v1**. When set, + the SDK validates the key resolves at init time and reserves it for + the future `soth.hash_user_id()` helper. When absent, customers + either pre-compute `user_id_hmac` themselves and pass via + `with soth.context(user_id_hmac=...)`, or omit user attribution + entirely. + + **Privacy tradeoff:** without an HMAC key, anything passed via + `user_id_hmac` reaches soth-cloud as-is. Regulated workloads + (HIPAA / heavy-PII) SHOULD configure a key. Phase-2.5 SDK adds + SDK-side hashing — see + `docs/common/SDK_WASM_TRUST_BOUNDARY_SPEC.md` §6.6. + Specify exactly one of `hmac_key_env` (read from environment) or `hmac_key_static` (raw bytes). Production usage SHOULD prefer `hmac_key_env` so the key never sits in source-controlled config. diff --git a/bindings/soth-py/src/lib.rs b/bindings/soth-py/src/lib.rs index 7bdd0b0b..4116fb8f 100644 --- a/bindings/soth-py/src/lib.rs +++ b/bindings/soth-py/src/lib.rs @@ -46,25 +46,24 @@ impl PySothSdk { hmac_key_static: Option>, telemetry_endpoint: Option, ) -> PyResult { + // HMAC key is optional in v1. Customers may pre-compute + // user_id_hmac in their own code; the Phase-2.5 SDK adds a + // helper that uses this key for SDK-side hashing. let hmac_key = match (hmac_key_env, hmac_key_static) { - (Some(env), None) => HmacKey::FromEnv(env), - (None, Some(bytes)) => HmacKey::Static(Zeroizing::new(bytes)), + (Some(env), None) => Some(HmacKey::FromEnv(env)), + (None, Some(bytes)) => Some(HmacKey::Static(Zeroizing::new(bytes))), (Some(_), Some(_)) => { return Err(PyValueError::new_err( "specify either hmac_key_env OR hmac_key_static, not both", )); } - (None, None) => { - return Err(PyValueError::new_err( - "hmac_key_env or hmac_key_static is required", - )); - } + (None, None) => None, }; - let mut builder = SdkConfigBuilder::new() - .api_key(api_key) - .org_id(org_id) - .hmac_key(hmac_key); + let mut builder = SdkConfigBuilder::new().api_key(api_key).org_id(org_id); + if let Some(key) = hmac_key { + builder = builder.hmac_key(key); + } if let Some(endpoint) = telemetry_endpoint { builder = builder.telemetry_endpoint(endpoint); } diff --git a/crates/soth-sdk-core/README.md b/crates/soth-sdk-core/README.md index 7af68abf..a8bdb01c 100644 --- a/crates/soth-sdk-core/README.md +++ b/crates/soth-sdk-core/README.md @@ -60,8 +60,9 @@ sdk.stream_end(obs); ## What v0 does (Phase 0) -- ✅ `init` validates config, resolves HMAC key, builds an empty detect - bundle and the deterministic fallback classify bundle. +- ✅ `init` validates config (HMAC key optional in v1; resolved when + present), builds an empty detect bundle and the deterministic + fallback classify bundle. - ✅ `pre_call` runs `process_normalized` for artifact detection + session dedup. Sync block path fires on credential / private-key artifacts. Returns a real `DecisionToken` allocated from a fixed diff --git a/crates/soth-sdk-core/src/config.rs b/crates/soth-sdk-core/src/config.rs index 04a91b0d..1c282493 100644 --- a/crates/soth-sdk-core/src/config.rs +++ b/crates/soth-sdk-core/src/config.rs @@ -164,7 +164,19 @@ impl Default for BundleSource { pub struct SdkConfig { pub api_key: String, pub org_id: String, - pub hmac_key: HmacKey, + /// Optional in v1. When set, the SDK validates the key resolves at + /// init time and reserves the field for the future + /// `soth.hash_user_id()` helper. When absent, customers either + /// pre-compute `user_id_hmac` themselves with their own HMAC + /// scheme and pass it via `CallContext`, or omit user attribution + /// entirely. + /// + /// **Privacy tradeoff:** without an HMAC key, anything passed + /// through `user_id_hmac` reaches soth-cloud as-is. Regulated + /// workloads (HIPAA / heavy-PII) SHOULD still configure a key. + /// See `SDK_WASM_TRUST_BOUNDARY_SPEC.md` §6.6 for the Phase-2 + /// implementation that closes this loop end-to-end. + pub hmac_key: Option, pub default_team_id: Option, pub default_device_id_hash: Option, pub capture_mode: CaptureMode, @@ -280,9 +292,10 @@ impl SdkConfigBuilder { let org_id = self .org_id .ok_or_else(|| SdkError::InvalidConfig("org_id required".into()))?; - let hmac_key = self - .hmac_key - .ok_or_else(|| SdkError::InvalidConfig("hmac_key required".into()))?; + // hmac_key is optional in v1. Customers who skip it pass + // user_id_hmac through plaintext (or omit it). Phase-2.5 SDK + // ships a hashing helper that activates per-customer privacy. + let hmac_key = self.hmac_key; let local_classification = self.local_classification.unwrap_or_default(); // CloudOptIn requires a configured endpoint. Validation here @@ -331,6 +344,19 @@ mod tests { .expect("build"); assert_eq!(cfg.org_id, "org-test"); assert_eq!(cfg.local_classification, ClassificationMode::Full); + assert!(cfg.hmac_key.is_some()); + } + + #[test] + fn builder_succeeds_without_hmac_key() { + // HMAC is opt-in for v1 — see crate docs. Builder must accept + // the no-key configuration without error. + let cfg = SdkConfigBuilder::new() + .api_key("sk-test") + .org_id("org-test") + .build() + .expect("build"); + assert!(cfg.hmac_key.is_none()); } #[test] @@ -338,7 +364,6 @@ mod tests { let err = SdkConfigBuilder::new() .api_key("k") .org_id("o") - .hmac_key(HmacKey::Static(Zeroizing::new(vec![0u8; 32]))) .local_classification(ClassificationMode::CloudOptIn) .build() .unwrap_err(); diff --git a/crates/soth-sdk-core/src/sdk.rs b/crates/soth-sdk-core/src/sdk.rs index e9356b90..5044cd8a 100644 --- a/crates/soth-sdk-core/src/sdk.rs +++ b/crates/soth-sdk-core/src/sdk.rs @@ -98,10 +98,14 @@ impl SothSdk { /// failure logs and returns an error; the binding's wrapper SHOULD /// fall back to no-op mode rather than crashing the host process. pub fn init(config: SdkConfig) -> Result { - // Validate the HMAC key resolves before we accept the config. - // Resolved bytes are dropped immediately — Phase 1 keeps them - // for telemetry signing, v0 only validates. - let _hmac = config.hmac_key.resolve()?; + // HMAC key is optional in v1. When configured, validate it + // resolves so misconfigured envs / vault paths fail loudly at + // init rather than later. When absent, the SDK skips the + // resolve and proceeds in plaintext-user-id mode (the + // customer either omits user_id_hmac or pre-computes it). + if let Some(key) = config.hmac_key.as_ref() { + let _hmac = key.resolve()?; + } // ClassificationMode::Full + onnx-models feature-off would be a // mismatch; bindings on size-constrained targets must pick @@ -165,7 +169,9 @@ impl SothSdk { detect_bundle: OwnedDetectBundle, classify_bundle: Arc, ) -> Result { - let _hmac = config.hmac_key.resolve()?; + if let Some(key) = config.hmac_key.as_ref() { + let _hmac = key.resolve()?; + } Ok(Self { config, detect_registry: Arc::new(soth_detect::ParserRegistry::default()), From 512fe3a0dc76d0712e40a470d0fb5f7aa2278651 Mon Sep 17 00:00:00 2001 From: Apoorv Raj Saxena Date: Fri, 1 May 2026 02:45:40 +0530 Subject: [PATCH 16/24] fix(sdk-ci): unbreak ffi-conformance for Node + Python MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Three independent CI bugs surfacing on every push to sdk-build: 1. Node: `napi build --platform` overwrites `index.js` and `index.d.ts` wholesale, wiping the hand-written shim (`init`, `guard`, `withContext`, `SothBlocked`, …). Tests on Linux were hitting `TypeError: soth.init is not a function` because the only thing left was a generated loader exporting `SothSdk`. Fix: pass `--js binding.js --dts binding.d.ts` so napi-rs writes its per-platform loader to a separate file. The shim now does `require('./binding.js')` and survives rebuilds. Loader files are gitignored (regenerated by CI / dev each build). 2. Python: `maturin develop` requires an active virtualenv and the workflow ran it bare, so every push died with `Couldn't find a virtualenv or conda environment`. Fix: create a `.venv` in `bindings/soth-py`, install pytest+maturin into it, and persist `VIRTUAL_ENV` / `PATH` via `$GITHUB_ENV` / `$GITHUB_PATH` so subsequent steps reuse the same interpreter. 3. ffi-conformance.test.mjs asserted snake_case keys (`endpoint_type`, `capture_mode`) — copy-pasted from the Python lane — but napi-rs renders Rust struct fields as camelCase by convention, so the binding correctly emits `endpointType` / `captureMode`. Fix: assert camelCase in the JS lane; the Python lane keeps snake_case. After this, `npm test` is green locally (32/32) and the Linux CI runner should resolve `soth-node.linux-x64-gnu.node` via the generated `binding.js` loader. Co-Authored-By: Claude Opus 4.7 (1M context) --- .github/workflows/ffi-conformance.yml | 10 +++++++++- bindings/soth-node/.gitignore | 11 +++++++++++ bindings/soth-node/__test__/ffi-conformance.test.mjs | 6 ++++-- bindings/soth-node/index.js | 11 +++++++---- bindings/soth-node/package.json | 8 ++++++-- 5 files changed, 37 insertions(+), 9 deletions(-) create mode 100644 bindings/soth-node/.gitignore diff --git a/.github/workflows/ffi-conformance.yml b/.github/workflows/ffi-conformance.yml index e0ea51f0..1af7695d 100644 --- a/.github/workflows/ffi-conformance.yml +++ b/.github/workflows/ffi-conformance.yml @@ -37,10 +37,18 @@ jobs: with: key: ffi-py workspaces: bindings/soth-py - - name: Install pytest + maturin + - name: Create venv + install pytest + maturin + # `maturin develop` requires an active virtualenv. Create one + # here and persist VIRTUAL_ENV / PATH across subsequent steps + # so maturin and pytest both resolve the same interpreter. + working-directory: bindings/soth-py run: | + python -m venv .venv + source .venv/bin/activate python -m pip install --upgrade pip python -m pip install pytest pytest-asyncio maturin + echo "VIRTUAL_ENV=$PWD/.venv" >> $GITHUB_ENV + echo "$PWD/.venv/bin" >> $GITHUB_PATH - name: Build + install soth-py into venv working-directory: bindings/soth-py run: maturin develop diff --git a/bindings/soth-node/.gitignore b/bindings/soth-node/.gitignore new file mode 100644 index 00000000..50d014c5 --- /dev/null +++ b/bindings/soth-node/.gitignore @@ -0,0 +1,11 @@ +# napi-rs / @napi-rs/cli auto-generated loader (regenerated by +# `npm run build` via the `--js binding.js --dts binding.d.ts` flags). +binding.js +binding.d.ts + +# Native binary artefacts emitted by `napi build --platform`. +*.node + +# npm +node_modules/ +package-lock.json diff --git a/bindings/soth-node/__test__/ffi-conformance.test.mjs b/bindings/soth-node/__test__/ffi-conformance.test.mjs index a2bd4f26..3b505769 100644 --- a/bindings/soth-node/__test__/ffi-conformance.test.mjs +++ b/bindings/soth-node/__test__/ffi-conformance.test.mjs @@ -113,8 +113,10 @@ if (fixtures.length === 0) { `${name}: model drift`, ); } - assert.ok('endpoint_type' in event); - assert.ok('capture_mode' in event); + // napi-rs renders Rust struct fields as camelCase by default, + // matching JS convention; the Python lane keeps snake_case. + assert.ok('endpointType' in event); + assert.ok('captureMode' in event); assert.equal(sdk.inFlightDecisions(), 0); }); } diff --git a/bindings/soth-node/index.js b/bindings/soth-node/index.js index e2a7567f..347ae9d8 100644 --- a/bindings/soth-node/index.js +++ b/bindings/soth-node/index.js @@ -14,10 +14,13 @@ const { AsyncLocalStorage } = require('node:async_hooks'); -const native = require('./soth-node.darwin-arm64.node'); /* eslint-disable-line global-require */ -// Real builds ship per-arch binaries via @napi-rs/cli; the line above -// is a placeholder for local dev. Production loader logic lands in a -// follow-up commit. +// `binding.js` is auto-generated by `@napi-rs/cli` (via the +// `--js binding.js` flag in `package.json`). It contains the +// per-platform loader that resolves the right `.node` binary for +// the current host. Splitting it from this file keeps the +// hand-written shim — `init`, `guard`, `withContext`, `SothBlocked` +// — intact when `napi build` regenerates the loader. +const native = require('./binding.js'); /* eslint-disable-line global-require */ // Per-call context lives in AsyncLocalStorage so async functions // awaited inside `withContext` see the same context after each await. diff --git a/bindings/soth-node/package.json b/bindings/soth-node/package.json index 0ae5ef0e..bb2d3313 100644 --- a/bindings/soth-node/package.json +++ b/bindings/soth-node/package.json @@ -11,11 +11,15 @@ "files": [ "index.js", "index.d.ts", + "binding.js", + "binding.d.ts", + "instrumentation/", + "integrations/", "README.md" ], "scripts": { - "build": "napi build --platform --release", - "build:debug": "napi build --platform", + "build": "napi build --platform --release --js binding.js --dts binding.d.ts", + "build:debug": "napi build --platform --js binding.js --dts binding.d.ts", "test": "node --test __test__/*.test.mjs" }, "devDependencies": { From aba4d273b8fa5c89ac953d0b7dae004c66256a76 Mon Sep 17 00:00:00 2001 From: Apoorv Raj Saxena Date: Fri, 1 May 2026 02:55:28 +0530 Subject: [PATCH 17/24] fix(sdk-ci): commit package-lock.json so setup-node cache resolves `actions/setup-node@v4` with `cache: 'npm'` and `cache-dependency-path: bindings/soth-node/package-lock.json` fails with "Some specified paths were not resolved, unable to cache dependencies" if the lockfile isn't checked in. The previous commit gitignored it; flip that. Lockfiles also matter beyond CI: package.json uses caret ranges for `@napi-rs/cli` and `openai`, so without a lockfile every fresh install can resolve to slightly different transitive versions. Co-Authored-By: Claude Opus 4.7 (1M context) --- bindings/soth-node/.gitignore | 4 +- bindings/soth-node/package-lock.json | 509 +++++++++++++++++++++++++++ 2 files changed, 512 insertions(+), 1 deletion(-) create mode 100644 bindings/soth-node/package-lock.json diff --git a/bindings/soth-node/.gitignore b/bindings/soth-node/.gitignore index 50d014c5..aee400ec 100644 --- a/bindings/soth-node/.gitignore +++ b/bindings/soth-node/.gitignore @@ -8,4 +8,6 @@ binding.d.ts # npm node_modules/ -package-lock.json +# package-lock.json IS committed — required by `actions/setup-node@v4` +# with `cache: 'npm'` (node-binaries.yml) and gives reproducible +# installs across CI runs. diff --git a/bindings/soth-node/package-lock.json b/bindings/soth-node/package-lock.json new file mode 100644 index 00000000..5dcccb14 --- /dev/null +++ b/bindings/soth-node/package-lock.json @@ -0,0 +1,509 @@ +{ + "name": "@soth/sdk", + "version": "0.1.0", + "lockfileVersion": 3, + "requires": true, + "packages": { + "": { + "name": "@soth/sdk", + "version": "0.1.0", + "license": "MIT OR Apache-2.0", + "devDependencies": { + "@napi-rs/cli": "^2.18", + "openai": "^4.77" + }, + "engines": { + "node": ">=18" + } + }, + "node_modules/@napi-rs/cli": { + "version": "2.18.4", + "resolved": "https://registry.npmjs.org/@napi-rs/cli/-/cli-2.18.4.tgz", + "integrity": "sha512-SgJeA4df9DE2iAEpr3M2H0OKl/yjtg1BnRI5/JyowS71tUWhrfSu2LT0V3vlHET+g1hBVlrO60PmEXwUEKp8Mg==", + "dev": true, + "license": "MIT", + "bin": { + "napi": "scripts/index.js" + }, + "engines": { + "node": ">= 10" + }, + "funding": { + "type": "github", + "url": "https://github.com/sponsors/Brooooooklyn" + } + }, + "node_modules/@types/node": { + "version": "18.19.130", + "resolved": "https://registry.npmjs.org/@types/node/-/node-18.19.130.tgz", + "integrity": "sha512-GRaXQx6jGfL8sKfaIDD6OupbIHBr9jv7Jnaml9tB7l4v068PAOXqfcujMMo5PhbIs6ggR1XODELqahT2R8v0fg==", + "dev": true, + "license": "MIT", + "dependencies": { + "undici-types": "~5.26.4" + } + }, + "node_modules/@types/node-fetch": { + "version": "2.6.13", + "resolved": "https://registry.npmjs.org/@types/node-fetch/-/node-fetch-2.6.13.tgz", + "integrity": "sha512-QGpRVpzSaUs30JBSGPjOg4Uveu384erbHBoT1zeONvyCfwQxIkUshLAOqN/k9EjGviPRmWTTe6aH2qySWKTVSw==", + "dev": true, + "license": "MIT", + "dependencies": { + "@types/node": "*", + "form-data": "^4.0.4" + } + }, + "node_modules/abort-controller": { + "version": "3.0.0", + "resolved": "https://registry.npmjs.org/abort-controller/-/abort-controller-3.0.0.tgz", + "integrity": "sha512-h8lQ8tacZYnR3vNQTgibj+tODHI5/+l06Au2Pcriv/Gmet0eaj4TwWH41sO9wnHDiQsEj19q0drzdWdeAHtweg==", + "dev": true, + "license": "MIT", + "dependencies": { + "event-target-shim": "^5.0.0" + }, + "engines": { + "node": ">=6.5" + } + }, + "node_modules/agentkeepalive": { + "version": "4.6.0", + "resolved": "https://registry.npmjs.org/agentkeepalive/-/agentkeepalive-4.6.0.tgz", + "integrity": "sha512-kja8j7PjmncONqaTsB8fQ+wE2mSU2DJ9D4XKoJ5PFWIdRMa6SLSN1ff4mOr4jCbfRSsxR4keIiySJU0N9T5hIQ==", + "dev": true, + "license": "MIT", + "dependencies": { + "humanize-ms": "^1.2.1" + }, + "engines": { + "node": ">= 8.0.0" + } + }, + "node_modules/asynckit": { + "version": "0.4.0", + "resolved": "https://registry.npmjs.org/asynckit/-/asynckit-0.4.0.tgz", + "integrity": "sha512-Oei9OH4tRh0YqU3GxhX79dM/mwVgvbZJaSNaRk+bshkj0S5cfHcgYakreBjrHwatXKbz+IoIdYLxrKim2MjW0Q==", + "dev": true, + "license": "MIT" + }, + "node_modules/call-bind-apply-helpers": { + "version": "1.0.2", + "resolved": "https://registry.npmjs.org/call-bind-apply-helpers/-/call-bind-apply-helpers-1.0.2.tgz", + "integrity": "sha512-Sp1ablJ0ivDkSzjcaJdxEunN5/XvksFJ2sMBFfq6x0ryhQV/2b/KwFe21cMpmHtPOSij8K99/wSfoEuTObmuMQ==", + "dev": true, + "license": "MIT", + "dependencies": { + "es-errors": "^1.3.0", + "function-bind": "^1.1.2" + }, + "engines": { + "node": ">= 0.4" + } + }, + "node_modules/combined-stream": { + "version": "1.0.8", + "resolved": "https://registry.npmjs.org/combined-stream/-/combined-stream-1.0.8.tgz", + "integrity": "sha512-FQN4MRfuJeHf7cBbBMJFXhKSDq+2kAArBlmRBvcvFE5BB1HZKXtSFASDhdlz9zOYwxh8lDdnvmMOe/+5cdoEdg==", + "dev": true, + "license": "MIT", + "dependencies": { + "delayed-stream": "~1.0.0" + }, + "engines": { + "node": ">= 0.8" + } + }, + "node_modules/delayed-stream": { + "version": "1.0.0", + "resolved": "https://registry.npmjs.org/delayed-stream/-/delayed-stream-1.0.0.tgz", + "integrity": "sha512-ZySD7Nf91aLB0RxL4KGrKHBXl7Eds1DAmEdcoVawXnLD7SDhpNgtuII2aAkg7a7QS41jxPSZ17p4VdGnMHk3MQ==", + "dev": true, + "license": "MIT", + "engines": { + "node": ">=0.4.0" + } + }, + "node_modules/dunder-proto": { + "version": "1.0.1", + "resolved": "https://registry.npmjs.org/dunder-proto/-/dunder-proto-1.0.1.tgz", + "integrity": "sha512-KIN/nDJBQRcXw0MLVhZE9iQHmG68qAVIBg9CqmUYjmQIhgij9U5MFvrqkUL5FbtyyzZuOeOt0zdeRe4UY7ct+A==", + "dev": true, + "license": "MIT", + "dependencies": { + "call-bind-apply-helpers": "^1.0.1", + "es-errors": "^1.3.0", + "gopd": "^1.2.0" + }, + "engines": { + "node": ">= 0.4" + } + }, + "node_modules/es-define-property": { + "version": "1.0.1", + "resolved": "https://registry.npmjs.org/es-define-property/-/es-define-property-1.0.1.tgz", + "integrity": "sha512-e3nRfgfUZ4rNGL232gUgX06QNyyez04KdjFrF+LTRoOXmrOgFKDg4BCdsjW8EnT69eqdYGmRpJwiPVYNrCaW3g==", + "dev": true, + "license": "MIT", + "engines": { + "node": ">= 0.4" + } + }, + "node_modules/es-errors": { + "version": "1.3.0", + "resolved": "https://registry.npmjs.org/es-errors/-/es-errors-1.3.0.tgz", + "integrity": "sha512-Zf5H2Kxt2xjTvbJvP2ZWLEICxA6j+hAmMzIlypy4xcBg1vKVnx89Wy0GbS+kf5cwCVFFzdCFh2XSCFNULS6csw==", + "dev": true, + "license": "MIT", + "engines": { + "node": ">= 0.4" + } + }, + "node_modules/es-object-atoms": { + "version": "1.1.1", + "resolved": "https://registry.npmjs.org/es-object-atoms/-/es-object-atoms-1.1.1.tgz", + "integrity": "sha512-FGgH2h8zKNim9ljj7dankFPcICIK9Cp5bm+c2gQSYePhpaG5+esrLODihIorn+Pe6FGJzWhXQotPv73jTaldXA==", + "dev": true, + "license": "MIT", + "dependencies": { + "es-errors": "^1.3.0" + }, + "engines": { + "node": ">= 0.4" + } + }, + "node_modules/es-set-tostringtag": { + "version": "2.1.0", + "resolved": "https://registry.npmjs.org/es-set-tostringtag/-/es-set-tostringtag-2.1.0.tgz", + "integrity": "sha512-j6vWzfrGVfyXxge+O0x5sh6cvxAog0a/4Rdd2K36zCMV5eJ+/+tOAngRO8cODMNWbVRdVlmGZQL2YS3yR8bIUA==", + "dev": true, + "license": "MIT", + "dependencies": { + "es-errors": "^1.3.0", + "get-intrinsic": "^1.2.6", + "has-tostringtag": "^1.0.2", + "hasown": "^2.0.2" + }, + "engines": { + "node": ">= 0.4" + } + }, + "node_modules/event-target-shim": { + "version": "5.0.1", + "resolved": "https://registry.npmjs.org/event-target-shim/-/event-target-shim-5.0.1.tgz", + "integrity": "sha512-i/2XbnSz/uxRCU6+NdVJgKWDTM427+MqYbkQzD321DuCQJUqOuJKIA0IM2+W2xtYHdKOmZ4dR6fExsd4SXL+WQ==", + "dev": true, + "license": "MIT", + "engines": { + "node": ">=6" + } + }, + "node_modules/form-data": { + "version": "4.0.5", + "resolved": "https://registry.npmjs.org/form-data/-/form-data-4.0.5.tgz", + "integrity": "sha512-8RipRLol37bNs2bhoV67fiTEvdTrbMUYcFTiy3+wuuOnUog2QBHCZWXDRijWQfAkhBj2Uf5UnVaiWwA5vdd82w==", + "dev": true, + "license": "MIT", + "dependencies": { + "asynckit": "^0.4.0", + "combined-stream": "^1.0.8", + "es-set-tostringtag": "^2.1.0", + "hasown": "^2.0.2", + "mime-types": "^2.1.12" + }, + "engines": { + "node": ">= 6" + } + }, + "node_modules/form-data-encoder": { + "version": "1.7.2", + "resolved": "https://registry.npmjs.org/form-data-encoder/-/form-data-encoder-1.7.2.tgz", + "integrity": "sha512-qfqtYan3rxrnCk1VYaA4H+Ms9xdpPqvLZa6xmMgFvhO32x7/3J/ExcTd6qpxM0vH2GdMI+poehyBZvqfMTto8A==", + "dev": true, + "license": "MIT" + }, + "node_modules/formdata-node": { + "version": "4.4.1", + "resolved": "https://registry.npmjs.org/formdata-node/-/formdata-node-4.4.1.tgz", + "integrity": "sha512-0iirZp3uVDjVGt9p49aTaqjk84TrglENEDuqfdlZQ1roC9CWlPk6Avf8EEnZNcAqPonwkG35x4n3ww/1THYAeQ==", + "dev": true, + "license": "MIT", + "dependencies": { + "node-domexception": "1.0.0", + "web-streams-polyfill": "4.0.0-beta.3" + }, + "engines": { + "node": ">= 12.20" + } + }, + "node_modules/function-bind": { + "version": "1.1.2", + "resolved": "https://registry.npmjs.org/function-bind/-/function-bind-1.1.2.tgz", + "integrity": "sha512-7XHNxH7qX9xG5mIwxkhumTox/MIRNcOgDrxWsMt2pAr23WHp6MrRlN7FBSFpCpr+oVO0F744iUgR82nJMfG2SA==", + "dev": true, + "license": "MIT", + "funding": { + "url": "https://github.com/sponsors/ljharb" + } + }, + "node_modules/get-intrinsic": { + "version": "1.3.0", + "resolved": "https://registry.npmjs.org/get-intrinsic/-/get-intrinsic-1.3.0.tgz", + "integrity": "sha512-9fSjSaos/fRIVIp+xSJlE6lfwhES7LNtKaCBIamHsjr2na1BiABJPo0mOjjz8GJDURarmCPGqaiVg5mfjb98CQ==", + "dev": true, + "license": "MIT", + "dependencies": { + "call-bind-apply-helpers": "^1.0.2", + "es-define-property": "^1.0.1", + "es-errors": "^1.3.0", + "es-object-atoms": "^1.1.1", + "function-bind": "^1.1.2", + "get-proto": "^1.0.1", + "gopd": "^1.2.0", + "has-symbols": "^1.1.0", + "hasown": "^2.0.2", + "math-intrinsics": "^1.1.0" + }, + "engines": { + "node": ">= 0.4" + }, + "funding": { + "url": "https://github.com/sponsors/ljharb" + } + }, + "node_modules/get-proto": { + "version": "1.0.1", + "resolved": "https://registry.npmjs.org/get-proto/-/get-proto-1.0.1.tgz", + "integrity": "sha512-sTSfBjoXBp89JvIKIefqw7U2CCebsc74kiY6awiGogKtoSGbgjYE/G/+l9sF3MWFPNc9IcoOC4ODfKHfxFmp0g==", + "dev": true, + "license": "MIT", + "dependencies": { + "dunder-proto": "^1.0.1", + "es-object-atoms": "^1.0.0" + }, + "engines": { + "node": ">= 0.4" + } + }, + "node_modules/gopd": { + "version": "1.2.0", + "resolved": "https://registry.npmjs.org/gopd/-/gopd-1.2.0.tgz", + "integrity": "sha512-ZUKRh6/kUFoAiTAtTYPZJ3hw9wNxx+BIBOijnlG9PnrJsCcSjs1wyyD6vJpaYtgnzDrKYRSqf3OO6Rfa93xsRg==", + "dev": true, + "license": "MIT", + "engines": { + "node": ">= 0.4" + }, + "funding": { + "url": "https://github.com/sponsors/ljharb" + } + }, + "node_modules/has-symbols": { + "version": "1.1.0", + "resolved": "https://registry.npmjs.org/has-symbols/-/has-symbols-1.1.0.tgz", + "integrity": "sha512-1cDNdwJ2Jaohmb3sg4OmKaMBwuC48sYni5HUw2DvsC8LjGTLK9h+eb1X6RyuOHe4hT0ULCW68iomhjUoKUqlPQ==", + "dev": true, + "license": "MIT", + "engines": { + "node": ">= 0.4" + }, + "funding": { + "url": "https://github.com/sponsors/ljharb" + } + }, + "node_modules/has-tostringtag": { + "version": "1.0.2", + "resolved": "https://registry.npmjs.org/has-tostringtag/-/has-tostringtag-1.0.2.tgz", + "integrity": "sha512-NqADB8VjPFLM2V0VvHUewwwsw0ZWBaIdgo+ieHtK3hasLz4qeCRjYcqfB6AQrBggRKppKF8L52/VqdVsO47Dlw==", + "dev": true, + "license": "MIT", + "dependencies": { + "has-symbols": "^1.0.3" + }, + "engines": { + "node": ">= 0.4" + }, + "funding": { + "url": "https://github.com/sponsors/ljharb" + } + }, + "node_modules/hasown": { + "version": "2.0.3", + "resolved": "https://registry.npmjs.org/hasown/-/hasown-2.0.3.tgz", + "integrity": "sha512-ej4AhfhfL2Q2zpMmLo7U1Uv9+PyhIZpgQLGT1F9miIGmiCJIoCgSmczFdrc97mWT4kVY72KA+WnnhJ5pghSvSg==", + "dev": true, + "license": "MIT", + "dependencies": { + "function-bind": "^1.1.2" + }, + "engines": { + "node": ">= 0.4" + } + }, + "node_modules/humanize-ms": { + "version": "1.2.1", + "resolved": "https://registry.npmjs.org/humanize-ms/-/humanize-ms-1.2.1.tgz", + "integrity": "sha512-Fl70vYtsAFb/C06PTS9dZBo7ihau+Tu/DNCk/OyHhea07S+aeMWpFFkUaXRa8fI+ScZbEI8dfSxwY7gxZ9SAVQ==", + "dev": true, + "license": "MIT", + "dependencies": { + "ms": "^2.0.0" + } + }, + "node_modules/math-intrinsics": { + "version": "1.1.0", + "resolved": "https://registry.npmjs.org/math-intrinsics/-/math-intrinsics-1.1.0.tgz", + "integrity": "sha512-/IXtbwEk5HTPyEwyKX6hGkYXxM9nbj64B+ilVJnC/R6B0pH5G4V3b0pVbL7DBj4tkhBAppbQUlf6F6Xl9LHu1g==", + "dev": true, + "license": "MIT", + "engines": { + "node": ">= 0.4" + } + }, + "node_modules/mime-db": { + "version": "1.52.0", + "resolved": "https://registry.npmjs.org/mime-db/-/mime-db-1.52.0.tgz", + "integrity": "sha512-sPU4uV7dYlvtWJxwwxHD0PuihVNiE7TyAbQ5SWxDCB9mUYvOgroQOwYQQOKPJ8CIbE+1ETVlOoK1UC2nU3gYvg==", + "dev": true, + "license": "MIT", + "engines": { + "node": ">= 0.6" + } + }, + "node_modules/mime-types": { + "version": "2.1.35", + "resolved": "https://registry.npmjs.org/mime-types/-/mime-types-2.1.35.tgz", + "integrity": "sha512-ZDY+bPm5zTTF+YpCrAU9nK0UgICYPT0QtT1NZWFv4s++TNkcgVaT0g6+4R2uI4MjQjzysHB1zxuWL50hzaeXiw==", + "dev": true, + "license": "MIT", + "dependencies": { + "mime-db": "1.52.0" + }, + "engines": { + "node": ">= 0.6" + } + }, + "node_modules/ms": { + "version": "2.1.3", + "resolved": "https://registry.npmjs.org/ms/-/ms-2.1.3.tgz", + "integrity": "sha512-6FlzubTLZG3J2a/NVCAleEhjzq5oxgHyaCU9yYXvcLsvoVaHJq/s5xXI6/XXP6tz7R9xAOtHnSO/tXtF3WRTlA==", + "dev": true, + "license": "MIT" + }, + "node_modules/node-domexception": { + "version": "1.0.0", + "resolved": "https://registry.npmjs.org/node-domexception/-/node-domexception-1.0.0.tgz", + "integrity": "sha512-/jKZoMpw0F8GRwl4/eLROPA3cfcXtLApP0QzLmUT/HuPCZWyB7IY9ZrMeKw2O/nFIqPQB3PVM9aYm0F312AXDQ==", + "deprecated": "Use your platform's native DOMException instead", + "dev": true, + "funding": [ + { + "type": "github", + "url": "https://github.com/sponsors/jimmywarting" + }, + { + "type": "github", + "url": "https://paypal.me/jimmywarting" + } + ], + "license": "MIT", + "engines": { + "node": ">=10.5.0" + } + }, + "node_modules/node-fetch": { + "version": "2.7.0", + "resolved": "https://registry.npmjs.org/node-fetch/-/node-fetch-2.7.0.tgz", + "integrity": "sha512-c4FRfUm/dbcWZ7U+1Wq0AwCyFL+3nt2bEw05wfxSz+DWpWsitgmSgYmy2dQdWyKC1694ELPqMs/YzUSNozLt8A==", + "dev": true, + "license": "MIT", + "dependencies": { + "whatwg-url": "^5.0.0" + }, + "engines": { + "node": "4.x || >=6.0.0" + }, + "peerDependencies": { + "encoding": "^0.1.0" + }, + "peerDependenciesMeta": { + "encoding": { + "optional": true + } + } + }, + "node_modules/openai": { + "version": "4.104.0", + "resolved": "https://registry.npmjs.org/openai/-/openai-4.104.0.tgz", + "integrity": "sha512-p99EFNsA/yX6UhVO93f5kJsDRLAg+CTA2RBqdHK4RtK8u5IJw32Hyb2dTGKbnnFmnuoBv5r7Z2CURI9sGZpSuA==", + "dev": true, + "license": "Apache-2.0", + "dependencies": { + "@types/node": "^18.11.18", + "@types/node-fetch": "^2.6.4", + "abort-controller": "^3.0.0", + "agentkeepalive": "^4.2.1", + "form-data-encoder": "1.7.2", + "formdata-node": "^4.3.2", + "node-fetch": "^2.6.7" + }, + "bin": { + "openai": "bin/cli" + }, + "peerDependencies": { + "ws": "^8.18.0", + "zod": "^3.23.8" + }, + "peerDependenciesMeta": { + "ws": { + "optional": true + }, + "zod": { + "optional": true + } + } + }, + "node_modules/tr46": { + "version": "0.0.3", + "resolved": "https://registry.npmjs.org/tr46/-/tr46-0.0.3.tgz", + "integrity": "sha512-N3WMsuqV66lT30CrXNbEjx4GEwlow3v6rr4mCcv6prnfwhS01rkgyFdjPNBYd9br7LpXV1+Emh01fHnq2Gdgrw==", + "dev": true, + "license": "MIT" + }, + "node_modules/undici-types": { + "version": "5.26.5", + "resolved": "https://registry.npmjs.org/undici-types/-/undici-types-5.26.5.tgz", + "integrity": "sha512-JlCMO+ehdEIKqlFxk6IfVoAUVmgz7cU7zD/h9XZ0qzeosSHmUJVOzSQvvYSYWXkFXC+IfLKSIffhv0sVZup6pA==", + "dev": true, + "license": "MIT" + }, + "node_modules/web-streams-polyfill": { + "version": "4.0.0-beta.3", + "resolved": "https://registry.npmjs.org/web-streams-polyfill/-/web-streams-polyfill-4.0.0-beta.3.tgz", + "integrity": "sha512-QW95TCTaHmsYfHDybGMwO5IJIM93I/6vTRk+daHTWFPhwh+C8Cg7j7XyKrwrj8Ib6vYXe0ocYNrmzY4xAAN6ug==", + "dev": true, + "license": "MIT", + "engines": { + "node": ">= 14" + } + }, + "node_modules/webidl-conversions": { + "version": "3.0.1", + "resolved": "https://registry.npmjs.org/webidl-conversions/-/webidl-conversions-3.0.1.tgz", + "integrity": "sha512-2JAn3z8AR6rjK8Sm8orRC0h/bcl/DqL7tRPdGZ4I1CjdF+EaMLmYxBHyXuKL849eucPFhvBoxMsflfOb8kxaeQ==", + "dev": true, + "license": "BSD-2-Clause" + }, + "node_modules/whatwg-url": { + "version": "5.0.0", + "resolved": "https://registry.npmjs.org/whatwg-url/-/whatwg-url-5.0.0.tgz", + "integrity": "sha512-saE57nupxk6v3HY35+jzBwYa0rKSy0XR8JSxZPwgLr7ys0IBzhGviA1/TUGJLmSVqs8pb9AnvICXEuOHLprYTw==", + "dev": true, + "license": "MIT", + "dependencies": { + "tr46": "~0.0.3", + "webidl-conversions": "^3.0.0" + } + } + } +} From 5ce72d2e8b4eca56e0028413d1011b2ae8c67cc0 Mon Sep 17 00:00:00 2001 From: Apoorv Raj Saxena Date: Fri, 1 May 2026 03:30:16 +0530 Subject: [PATCH 18/24] fix(sdk-ci): unbreak node-binaries cross-compile + musl jobs MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Three failures on the node-binaries matrix: 1. aarch64-unknown-linux-gnu: cargo emitted aarch64 object files but the host's x86_64 linker tried to link them, producing a wall of `is incompatible with elf64-x86-64` errors. Set `CARGO_TARGET_AARCH64_UNKNOWN_LINUX_GNU_LINKER` and `CC_aarch64_unknown_linux_gnu` to `aarch64-linux-gnu-gcc` (already apt-installed by the matrix step) so cargo and cc-rs both go through the cross toolchain. 2. x86_64-unknown-linux-musl: napi-rs's `nodejs-rust:lts-debian-musl` image was removed/renamed — `manifest unknown` from the registry. Switch to the canonical native-musl image `lts-alpine`; the build runs natively inside Alpine so no cross-compile is needed. 3. aarch64-unknown-linux-musl: same dead image, but this one is a real cross-compile (x86_64 host → aarch64 target). Switch to `lts-debian-zig` and pass `--zig` to `napi build` so it routes through `cargo-zigbuild`, which gives the linker a working musl-aarch64 sysroot. Native targets (linux-x64-gnu, darwin-{arm64,x64}, win32-x64-msvc) already pass and are unchanged. Co-Authored-By: Claude Opus 4.7 (1M context) --- .github/workflows/node-binaries.yml | 22 +++++++++++++++++++--- 1 file changed, 19 insertions(+), 3 deletions(-) diff --git a/.github/workflows/node-binaries.yml b/.github/workflows/node-binaries.yml index 13cd3291..c4740aec 100644 --- a/.github/workflows/node-binaries.yml +++ b/.github/workflows/node-binaries.yml @@ -60,8 +60,12 @@ jobs: cd bindings/soth-node && npm run build - target: x86_64-unknown-linux-musl runs-on: ubuntu-latest - docker: ghcr.io/napi-rs/napi-rs/nodejs-rust:lts-debian-musl + # napi-rs renamed/removed `lts-debian-musl`; the canonical + # native-musl image is `lts-alpine`. Build runs inside the + # Alpine container, no cross-compile needed. + docker: ghcr.io/napi-rs/napi-rs/nodejs-rust:lts-alpine build: | + set -e cd bindings/soth-node && npm run build -- --target x86_64-unknown-linux-musl - target: aarch64-unknown-linux-gnu runs-on: ubuntu-latest @@ -70,9 +74,15 @@ jobs: cd bindings/soth-node && npm run build -- --target aarch64-unknown-linux-gnu - target: aarch64-unknown-linux-musl runs-on: ubuntu-latest - docker: ghcr.io/napi-rs/napi-rs/nodejs-rust:lts-debian-musl + # x86_64 host → aarch64 musl: cross-compile via zig in the + # napi-rs Debian+zig image. `--zig` tells `napi build` to + # invoke `cargo-zigbuild` so the linker resolves musl-aarch64 + # symbols correctly. + docker: ghcr.io/napi-rs/napi-rs/nodejs-rust:lts-debian-zig build: | - cd bindings/soth-node && npm run build -- --target aarch64-unknown-linux-musl + set -e + rustup target add aarch64-unknown-linux-musl + cd bindings/soth-node && npm run build -- --target aarch64-unknown-linux-musl --zig - target: x86_64-apple-darwin runs-on: macos-13 build: | @@ -118,6 +128,12 @@ jobs: if: matrix.docker == '' env: CARGO_BUILD_TARGET: ${{ matrix.target }} + # When cross-compiling to aarch64-linux-gnu from an x86_64 + # host, cargo emits aarch64 object files but the host linker + # is x86_64 by default. Point cargo at the cross-toolchain + # gcc installed in the previous step. No-op on other targets. + CARGO_TARGET_AARCH64_UNKNOWN_LINUX_GNU_LINKER: aarch64-linux-gnu-gcc + CC_aarch64_unknown_linux_gnu: aarch64-linux-gnu-gcc shell: bash run: ${{ matrix.build }} From dcebe98b39f31c1497d8d94355617f961e431990 Mon Sep 17 00:00:00 2001 From: Apoorv Raj Saxena Date: Fri, 1 May 2026 03:43:40 +0530 Subject: [PATCH 19/24] fix(sdk-ci): refresh Rust inside musl docker images After switching to `lts-alpine` and `lts-debian-zig`, the musl jobs moved past the previous "manifest unknown" failure but hit a new one: the napi-rs images ship Rust 1.83, while transitive deps (time-core 0.1.8, base64ct 1.8.3) require the `edition2024` Cargo feature stabilized in Rust 1.85. Add `rustup default stable && rustup update stable` to both musl build scripts so the in-container toolchain matches the host (`dtolnay/rust-toolchain@stable`) before `napi build` runs. Co-Authored-By: Claude Opus 4.7 (1M context) --- .github/workflows/node-binaries.yml | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-) diff --git a/.github/workflows/node-binaries.yml b/.github/workflows/node-binaries.yml index c4740aec..e256b14a 100644 --- a/.github/workflows/node-binaries.yml +++ b/.github/workflows/node-binaries.yml @@ -62,10 +62,15 @@ jobs: runs-on: ubuntu-latest # napi-rs renamed/removed `lts-debian-musl`; the canonical # native-musl image is `lts-alpine`. Build runs inside the - # Alpine container, no cross-compile needed. + # Alpine container, no cross-compile needed. The image + # ships Rust 1.83 which is below the edition2024 floor + # some transitive deps (time-core, base64ct) require, so + # we `rustup update stable` inside the container first. docker: ghcr.io/napi-rs/napi-rs/nodejs-rust:lts-alpine build: | set -e + rustup default stable + rustup update stable cd bindings/soth-node && npm run build -- --target x86_64-unknown-linux-musl - target: aarch64-unknown-linux-gnu runs-on: ubuntu-latest @@ -77,10 +82,13 @@ jobs: # x86_64 host → aarch64 musl: cross-compile via zig in the # napi-rs Debian+zig image. `--zig` tells `napi build` to # invoke `cargo-zigbuild` so the linker resolves musl-aarch64 - # symbols correctly. + # symbols correctly. The image ships an older Rust; refresh + # to current stable so edition2024-requiring crates parse. docker: ghcr.io/napi-rs/napi-rs/nodejs-rust:lts-debian-zig build: | set -e + rustup default stable + rustup update stable rustup target add aarch64-unknown-linux-musl cd bindings/soth-node && npm run build -- --target aarch64-unknown-linux-musl --zig - target: x86_64-apple-darwin From b5f5fa329e947cc43ddf479d9992f9f38e9334b2 Mon Sep 17 00:00:00 2001 From: Apoorv Raj Saxena Date: Fri, 1 May 2026 19:00:45 +0530 Subject: [PATCH 20/24] fix(sdk-ci): cross-build x86_64-apple-darwin from arm64 macos-14 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit GitHub's macos-13 Intel runner pool is heavily oversubscribed — the x86_64-apple-darwin job routinely sits queued past the queue timeout and gets auto-cancelled, blocking the workflow even when every other matrix entry succeeds. Move both Mac targets onto macos-14 (Apple Silicon, abundant runners) and cross-compile to x86_64-apple-darwin via `rustup target add`. The Xcode SDK on macos-14 ships both arm64 and x86_64 sysroots, so this is a first-class native cross — no zig, no Rosetta. Drop x86_64-apple-darwin from the smoke-test step since the arm64 host can't `require()` an x86_64 .node binary; ffi-conformance.yml is the real load-bearing test. Co-Authored-By: Claude Opus 4.7 (1M context) --- .github/workflows/node-binaries.yml | 14 ++++++++++++-- 1 file changed, 12 insertions(+), 2 deletions(-) diff --git a/.github/workflows/node-binaries.yml b/.github/workflows/node-binaries.yml index e256b14a..7fe96e6a 100644 --- a/.github/workflows/node-binaries.yml +++ b/.github/workflows/node-binaries.yml @@ -92,8 +92,14 @@ jobs: rustup target add aarch64-unknown-linux-musl cd bindings/soth-node && npm run build -- --target aarch64-unknown-linux-musl --zig - target: x86_64-apple-darwin - runs-on: macos-13 + # GitHub's macos-13 (Intel) runner pool is oversubscribed + # and routinely auto-cancels queued jobs after the queue + # timeout. Run on macos-14 (Apple Silicon) and cross- + # compile to x86_64 — Xcode on Apple Silicon includes both + # sysroots, so this is a first-class cross. + runs-on: macos-14 build: | + rustup target add x86_64-apple-darwin cd bindings/soth-node && npm run build -- --target x86_64-apple-darwin - target: aarch64-apple-darwin runs-on: macos-14 @@ -154,7 +160,11 @@ jobs: run: ${{ matrix.build }} - name: Smoke test binary (native arches only) - if: matrix.target == 'x86_64-unknown-linux-gnu' || matrix.target == 'aarch64-apple-darwin' || matrix.target == 'x86_64-apple-darwin' || matrix.target == 'x86_64-pc-windows-msvc' + # x86_64-apple-darwin is now built via cross from an arm64 + # macos-14 host; the host can't `require()` an x86_64 binary, + # so it's excluded from the smoke list along with the other + # cross targets. Real coverage lives in ffi-conformance.yml. + if: matrix.target == 'x86_64-unknown-linux-gnu' || matrix.target == 'aarch64-apple-darwin' || matrix.target == 'x86_64-pc-windows-msvc' working-directory: bindings/soth-node shell: bash run: | From 99de5fb31796660d5b01c2abc1ddd35838c01018 Mon Sep 17 00:00:00 2001 From: Apoorv Raj Saxena Date: Fri, 1 May 2026 19:20:40 +0530 Subject: [PATCH 21/24] fix(sdk-ci): manylinux before-script ran apt-get unconditionally `yum install -y openssl-devel || apt-get update && apt-get install -y libssl-dev pkg-config` parses as `(yum || apt-get update) && apt-get install ...` due to left-associative `||`/`&&` precedence. So even when yum succeeded, apt-get install ran on the CentOS-based manylinux2014 image and failed with `apt-get: command not found` (exit 127), killing every Linux wheel build. manylinux is always yum-based, so the apt fallback is dead code anyway. Drop it; install pkgconfig from yum directly. Co-Authored-By: Claude Opus 4.7 (1M context) --- .github/workflows/python-wheels.yml | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/.github/workflows/python-wheels.yml b/.github/workflows/python-wheels.yml index 206f71db..105cf292 100644 --- a/.github/workflows/python-wheels.yml +++ b/.github/workflows/python-wheels.yml @@ -94,7 +94,11 @@ jobs: rust-toolchain: stable docker-options: ${{ matrix.platform.os == 'linux' && '-e PYO3_CROSS_LIB_DIR=/opt/python/cp310-cp310/lib' || '' }} before-script-linux: | - yum install -y openssl-devel || apt-get update && apt-get install -y libssl-dev pkg-config + # manylinux2014 is CentOS-based, so always yum. The previous + # `yum ... || apt-get update && apt-get install ...` had a + # shell-precedence bug that ran apt-get unconditionally and + # failed with `apt-get: command not found`. + yum install -y openssl-devel pkgconfig - name: Inspect wheel if: matrix.platform.os != 'windows' From 092ad2824b0d89561428c462c36ed72a960340e6 Mon Sep 17 00:00:00 2001 From: Apoorv Raj Saxena Date: Sat, 2 May 2026 00:47:53 +0530 Subject: [PATCH 22/24] fix(sdk-ci): unbreak python-wheels aarch64 + macos x86_64 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Two regressions on the python-wheels workflow, both surfaced after the last manylinux fix went in. aarch64-unknown-linux-gnu was failing at the maturin before-script with `yum: command not found` (docker exit 127). The previous fix dropped the apt fallback assuming "manylinux is always yum-based" — which is true for the native `manylinux2014` image, but the cross image used for non-native arches is `ghcr.io/rust-cross/manylinux2014-cross:`, and that one is Debian-based. Detect the package manager on PATH instead of assuming. x86_64-apple-darwin was sitting queued for hours every run on macos-13 (Intel) — same chronic oversubscription that node-binaries.yml already worked around in b5f5fa3. Move it to macos-14 (Apple Silicon, abundant runners) and cross-compile via `rustup target add`. The Xcode SDK on macos-14 ships both arm64 and x86_64 sysroots, so this is a first-class native cross. Drop the smoke test on x86_64-apple-darwin since the arm64 host can't import an x86_64 extension module; PyPI consumer install on real Intel hardware remains the load-bearing test. Both stuck runs (25216787926, 25189720745) cancelled. Co-Authored-By: Claude Opus 4.7 (1M context) --- .github/workflows/python-wheels.yml | 34 ++++++++++++++++++++++------- 1 file changed, 26 insertions(+), 8 deletions(-) diff --git a/.github/workflows/python-wheels.yml b/.github/workflows/python-wheels.yml index 105cf292..2474ee52 100644 --- a/.github/workflows/python-wheels.yml +++ b/.github/workflows/python-wheels.yml @@ -46,8 +46,12 @@ jobs: - { os: linux, runs-on: ubuntu-latest, target: x86_64-unknown-linux-gnu, manylinux: '2014' } # Linux aarch64 — same manylinux base, cross via QEMU - { os: linux, runs-on: ubuntu-latest, target: aarch64-unknown-linux-gnu, manylinux: '2014' } - # macOS Intel - - { os: macos, runs-on: macos-13, target: x86_64-apple-darwin, manylinux: '' } + # macOS Intel — cross-built from the Apple Silicon runner. + # GitHub's macos-13 Intel pool is heavily oversubscribed; jobs + # routinely sit queued past the queue timeout. macos-14's Xcode + # SDK ships both arm64 and x86_64 sysroots so this is a + # first-class native cross via `rustup target add`. + - { os: macos, runs-on: macos-14, target: x86_64-apple-darwin, manylinux: '' } # macOS Apple Silicon - { os: macos, runs-on: macos-14, target: aarch64-apple-darwin, manylinux: '' } # Windows @@ -94,11 +98,21 @@ jobs: rust-toolchain: stable docker-options: ${{ matrix.platform.os == 'linux' && '-e PYO3_CROSS_LIB_DIR=/opt/python/cp310-cp310/lib' || '' }} before-script-linux: | - # manylinux2014 is CentOS-based, so always yum. The previous - # `yum ... || apt-get update && apt-get install ...` had a - # shell-precedence bug that ran apt-get unconditionally and - # failed with `apt-get: command not found`. - yum install -y openssl-devel pkgconfig + # The native manylinux2014 image (x86_64) is CentOS-based and + # uses yum. The cross image used for non-native arches + # (`ghcr.io/rust-cross/manylinux2014-cross:`) is + # Debian-based and uses apt-get. Detect which package + # manager is on PATH instead of assuming. + set -e + if command -v yum >/dev/null 2>&1; then + yum install -y openssl-devel pkgconfig + elif command -v apt-get >/dev/null 2>&1; then + apt-get update + apt-get install -y libssl-dev pkg-config + else + echo "no supported package manager (yum or apt-get) found" >&2 + exit 1 + fi - name: Inspect wheel if: matrix.platform.os != 'windows' @@ -109,7 +123,11 @@ jobs: run: dir bindings\soth-py\dist\ - name: Smoke test wheel (native arches only) - if: matrix.platform.target == 'x86_64-unknown-linux-gnu' || matrix.platform.target == 'aarch64-apple-darwin' || matrix.platform.target == 'x86_64-apple-darwin' || matrix.platform.target == 'x86_64-pc-windows-msvc' + # x86_64-apple-darwin is now cross-built from an arm64 macos-14 + # host, which can't `import` an x86_64 extension module. Skip + # the smoke step there; the load-bearing test is the eventual + # PyPI consumer install on a real Intel Mac. + if: matrix.platform.target == 'x86_64-unknown-linux-gnu' || matrix.platform.target == 'aarch64-apple-darwin' || matrix.platform.target == 'x86_64-pc-windows-msvc' shell: bash run: | python -m pip install --upgrade pip From e5941f33fdcc057d33be9c56485255eb5bcd825d Mon Sep 17 00:00:00 2001 From: Apoorv Raj Saxena Date: Sat, 2 May 2026 00:59:48 +0530 Subject: [PATCH 23/24] =?UTF-8?q?fix(sdk-ci):=20aarch64=20wheel=20?= =?UTF-8?q?=E2=80=94=20use=20native=20manylinux=20image=20under=20QEMU?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Previous fix unblocked the apt-get/yum mismatch but exposed the next layer: ring 0.17.x's pre-generated ARM assembly fails to compile under the cross-toolchain bundled in `ghcr.io/rust-cross/manylinux2014-cross:aarch64` because that image's `aarch64-unknown-linux-gnu-gcc` doesn't predefine `__ARM_ARCH`. cc-rs aborts with `error: #error "ARM assembler must define __ARM_ARCH"` and maturin exits 1. Override maturin-action's default image with the native `quay.io/pypa/manylinux2014_aarch64` (CentOS-based, yum), letting the QEMU binfmt step we already set up emulate it on the x86_64 host. The emulated build uses a native aarch64 GCC so ring's assembly compiles cleanly. Trade ~5 min cross-compile speed for a build that actually finishes; the workflow as a whole runs once on push, no perf hit worth chasing here. Keep the apt-get fallback in before-script-linux as defense in depth in case the Debian-based image ever sneaks back in. Co-Authored-By: Claude Opus 4.7 (1M context) --- .github/workflows/python-wheels.yml | 17 ++++++++++++----- 1 file changed, 12 insertions(+), 5 deletions(-) diff --git a/.github/workflows/python-wheels.yml b/.github/workflows/python-wheels.yml index 2474ee52..2731cff0 100644 --- a/.github/workflows/python-wheels.yml +++ b/.github/workflows/python-wheels.yml @@ -91,6 +91,14 @@ jobs: # When manylinux is set, maturin runs inside the manylinux # docker image so wheels are universally compatible. manylinux: ${{ matrix.platform.manylinux }} + # For aarch64 we override maturin-action's default of the + # Debian-based `ghcr.io/rust-cross/manylinux2014-cross:aarch64` + # image. That image's cross-gcc fails to predefine __ARM_ARCH, + # which breaks ring 0.17.x's bundled ARM assembly. Pin the + # native manylinux2014_aarch64 image instead and let the QEMU + # binfmt setup (above) emulate it on the x86_64 host. Slower, + # but uses a native aarch64 toolchain so ring builds cleanly. + container: ${{ matrix.platform.target == 'aarch64-unknown-linux-gnu' && 'quay.io/pypa/manylinux2014_aarch64' || '' }} working-directory: bindings/soth-py # `extension-module` is required for the actual abi3 wheel # (workspace `cargo build` builds without it; only the wheel @@ -98,11 +106,10 @@ jobs: rust-toolchain: stable docker-options: ${{ matrix.platform.os == 'linux' && '-e PYO3_CROSS_LIB_DIR=/opt/python/cp310-cp310/lib' || '' }} before-script-linux: | - # The native manylinux2014 image (x86_64) is CentOS-based and - # uses yum. The cross image used for non-native arches - # (`ghcr.io/rust-cross/manylinux2014-cross:`) is - # Debian-based and uses apt-get. Detect which package - # manager is on PATH instead of assuming. + # Both the native manylinux2014 image (x86_64) and the + # native manylinux2014_aarch64 image are CentOS-based and + # use yum. Keep an apt-get fallback in case maturin-action + # ever defaults to the Debian-based cross image again. set -e if command -v yum >/dev/null 2>&1; then yum install -y openssl-devel pkgconfig From b582147270390ed93ae8138961f70328f0491866 Mon Sep 17 00:00:00 2001 From: Apoorv Raj Saxena Date: Sat, 2 May 2026 13:42:03 +0530 Subject: [PATCH 24/24] style(sdk): cargo fmt --all CI's `cargo fmt --all -- --check` step on PR #63 was failing because several files in the SDK crates and bindings drifted out of rustfmt's canonical layout (mostly long error-builder closures that fmt wants broken across multiple lines, plus one re-ordered `use` block). Pure formatting, no behavior change. Co-Authored-By: Claude Opus 4.7 (1M context) --- bindings/soth-node/src/lib.rs | 26 +++++++++++-------- bindings/soth-py/src/lib.rs | 12 +++------ crates/soth-conformance-tests/src/lib.rs | 10 +++++-- crates/soth-conformance-tests/tests/parity.rs | 5 +--- crates/soth-sdk-core/src/decision.rs | 4 ++- crates/soth-sdk-core/src/sdk.rs | 8 +++--- crates/soth-sdk-core/src/shipper.rs | 6 ++++- crates/soth-sdk-core/tests/round_trip.rs | 10 ++++--- 8 files changed, 46 insertions(+), 35 deletions(-) diff --git a/bindings/soth-node/src/lib.rs b/bindings/soth-node/src/lib.rs index f90fd561..f98d47ae 100644 --- a/bindings/soth-node/src/lib.rs +++ b/bindings/soth-node/src/lib.rs @@ -14,12 +14,12 @@ use std::sync::{Arc, Mutex}; use napi::bindgen_prelude::*; use napi_derive::napi; +use soth_core::EndpointType; use soth_sdk_core::{ BlockReason as CoreBlockReason, CallContext, Decision as CoreDecision, DecisionToken, FlagSeverity, HmacKey, LlmCall, LlmChunk, LlmResponse, Message, SdkConfigBuilder, SothSdk as CoreSothSdk, StreamObservation as CoreStreamObservation, Tool, }; -use soth_core::EndpointType; use zeroize::Zeroizing; // ── napi-exposed types ──────────────────────────────────────────────── @@ -177,7 +177,10 @@ impl SothSdk { #[napi] pub fn post_call(&self, token: String, _response: Option) -> Result<()> { let inner: u64 = token.parse().map_err(|_| { - Error::new(Status::InvalidArg, "decision token must be a numeric string") + Error::new( + Status::InvalidArg, + "decision token must be a numeric string", + ) })?; let token = DecisionToken::from_raw(inner); let response = LlmResponse::new(EndpointType::ChatCompletion); @@ -203,9 +206,10 @@ impl SothSdk { // bookkeeping — there's no observation to stash because pre_call // itself didn't allocate one. Phase-1 telemetry records this. if !is_sentinel_raw(token_raw) { - let mut guard = self.streams.lock().map_err(|_| { - Error::new(Status::GenericFailure, "stream slot lock poisoned") - })?; + let mut guard = self + .streams + .lock() + .map_err(|_| Error::new(Status::GenericFailure, "stream slot lock poisoned"))?; guard.insert(token_raw, observation); } Ok(decision_to_js(&decision)) @@ -222,9 +226,9 @@ impl SothSdk { delta_content: Option, finish_reason: Option, ) -> Result<()> { - let raw: u64 = token.parse().map_err(|_| { - Error::new(Status::InvalidArg, "stream token must be a numeric string") - })?; + let raw: u64 = token + .parse() + .map_err(|_| Error::new(Status::InvalidArg, "stream token must be a numeric string"))?; let mut guard = self .streams .lock() @@ -242,9 +246,9 @@ impl SothSdk { /// Finalize the stream. Idempotent — second call is a no-op. #[napi] pub fn stream_end(&self, token: String) -> Result<()> { - let raw: u64 = token.parse().map_err(|_| { - Error::new(Status::InvalidArg, "stream token must be a numeric string") - })?; + let raw: u64 = token + .parse() + .map_err(|_| Error::new(Status::InvalidArg, "stream token must be a numeric string"))?; let mut guard = self .streams .lock() diff --git a/bindings/soth-py/src/lib.rs b/bindings/soth-py/src/lib.rs index 4116fb8f..f4a4a76b 100644 --- a/bindings/soth-py/src/lib.rs +++ b/bindings/soth-py/src/lib.rs @@ -14,12 +14,12 @@ use std::sync::{Arc, Mutex}; use pyo3::exceptions::{PyRuntimeError, PyValueError}; use pyo3::prelude::*; use pyo3::types::{PyDict, PyList}; +use soth_core::EndpointType; use soth_sdk_core::{ BlockReason as CoreBlockReason, CallContext, Decision as CoreDecision, DecisionToken, FlagSeverity, HmacKey, LlmCall, LlmChunk, LlmResponse, Message, SdkConfigBuilder, SothSdk as CoreSothSdk, StreamObservation as CoreStreamObservation, Tool, }; -use soth_core::EndpointType; use zeroize::Zeroizing; /// PyO3 wrapper around `SothSdk`. Stored as `Arc` so it can be @@ -147,10 +147,7 @@ impl PySothSdk { /// Drain the in-memory telemetry queue. Test-only — production /// shippers will pull batches via the Phase-1 transport API. - fn drain_telemetry_for_test<'py>( - &self, - py: Python<'py>, - ) -> PyResult> { + fn drain_telemetry_for_test<'py>(&self, py: Python<'py>) -> PyResult> { let events = self.inner.drain_telemetry_for_test(); let result = PyList::empty_bound(py); for event in events { @@ -162,10 +159,7 @@ impl PySothSdk { event_dict.set_item("endpoint_type", format!("{:?}", event.endpoint_type))?; event_dict.set_item("capture_mode", format!("{:?}", event.capture_mode))?; event_dict.set_item("use_case", format!("{:?}", event.use_case))?; - event_dict.set_item( - "volatility_class", - format!("{:?}", event.volatility_class), - )?; + event_dict.set_item("volatility_class", format!("{:?}", event.volatility_class))?; result.append(event_dict)?; } Ok(result) diff --git a/crates/soth-conformance-tests/src/lib.rs b/crates/soth-conformance-tests/src/lib.rs index 53515f58..a0b4db2a 100644 --- a/crates/soth-conformance-tests/src/lib.rs +++ b/crates/soth-conformance-tests/src/lib.rs @@ -470,7 +470,10 @@ pub fn run_facade_lane( let config = soth_sdk_core::SdkConfigBuilder::new() .api_key("conformance-test") .org_id("org-conformance") - .hmac_key(soth_sdk_core::HmacKey::Static(Zeroizing::new(vec![0u8; 32]))) + .hmac_key(soth_sdk_core::HmacKey::Static(Zeroizing::new(vec![ + 0u8; + 32 + ]))) .build() .expect("build sdk config"); @@ -522,7 +525,10 @@ pub fn run_facade_lane( let telemetry = sdk.drain_telemetry_for_test().into_iter().next(); - FacadeOutput { decision, telemetry } + FacadeOutput { + decision, + telemetry, + } } // --------------------------------------------------------------------------- diff --git a/crates/soth-conformance-tests/tests/parity.rs b/crates/soth-conformance-tests/tests/parity.rs index e460ef9b..0c34ac88 100644 --- a/crates/soth-conformance-tests/tests/parity.rs +++ b/crates/soth-conformance-tests/tests/parity.rs @@ -104,10 +104,7 @@ fn sdk_direct_and_facade_lanes_agree_byte_identical() { let diffs = soth_conformance_tests::compare_sdk_vs_facade(&sdk, &facade); if !diffs.is_empty() { - failures.push(( - format!("{} ({})", fixture.name, path.display()), - diffs, - )); + failures.push((format!("{} ({})", fixture.name, path.display()), diffs)); } } diff --git a/crates/soth-sdk-core/src/decision.rs b/crates/soth-sdk-core/src/decision.rs index 897ed495..1fd71e38 100644 --- a/crates/soth-sdk-core/src/decision.rs +++ b/crates/soth-sdk-core/src/decision.rs @@ -152,7 +152,9 @@ impl DecisionToken { /// Test/binding-failure sentinel. Used when an FFI panic was caught /// at the boundary and a fail-open `Decision::Allow` was emitted. - pub const SENTINEL_FAIL_OPEN: DecisionToken = DecisionToken { inner: u64::MAX - 1 }; + pub const SENTINEL_FAIL_OPEN: DecisionToken = DecisionToken { + inner: u64::MAX - 1, + }; /// Opaque round-trip handle for FFI bindings. Bindings serialize /// the token across the language boundary as the returned u64; diff --git a/crates/soth-sdk-core/src/sdk.rs b/crates/soth-sdk-core/src/sdk.rs index 5044cd8a..83500de4 100644 --- a/crates/soth-sdk-core/src/sdk.rs +++ b/crates/soth-sdk-core/src/sdk.rs @@ -218,9 +218,11 @@ impl SothSdk { // Sync block path — credential / private-key artifacts are an // unconditional block in v0. Phase-1 expands this with full // org-rule evaluation on artifact-conditioned rules. - if let Some(reason) = detect.artifacts.iter().find_map(|a| { - artifact_block_reason(&a.kind, a.severity) - }) { + if let Some(reason) = detect + .artifacts + .iter() + .find_map(|a| artifact_block_reason(&a.kind, a.severity)) + { let slab_ctx = DecisionContext { created_at: std::time::Instant::now(), generation: 0, diff --git a/crates/soth-sdk-core/src/shipper.rs b/crates/soth-sdk-core/src/shipper.rs index dcba3140..fdbb75a9 100644 --- a/crates/soth-sdk-core/src/shipper.rs +++ b/crates/soth-sdk-core/src/shipper.rs @@ -146,6 +146,10 @@ fn run_loop( "org_id": &org_id, "events": final_batch, }); - let _ = client.post(&endpoint).bearer_auth(&api_key).json(&body).send(); + let _ = client + .post(&endpoint) + .bearer_auth(&api_key) + .json(&body) + .send(); } } diff --git a/crates/soth-sdk-core/tests/round_trip.rs b/crates/soth-sdk-core/tests/round_trip.rs index 0bf2fc97..ef03700e 100644 --- a/crates/soth-sdk-core/tests/round_trip.rs +++ b/crates/soth-sdk-core/tests/round_trip.rs @@ -11,11 +11,11 @@ //! - Streaming round-trip: `stream_begin` / `stream_chunk` / //! `stream_end` consumes the token exactly once. +use soth_core::EndpointType; use soth_sdk_core::{ BlockReason, Decision, HmacKey, LlmCall, LlmChunk, LlmResponse, Message, SdkConfigBuilder, SothSdk, }; -use soth_core::EndpointType; use zeroize::Zeroizing; fn minimal_sdk() -> SothSdk { @@ -53,8 +53,7 @@ fn credential_call() -> LlmCall { model: "gpt-4o-mini".into(), messages: vec![Message { role: "user".into(), - content: "review this key sk-abcdefghijklmnopqrstuvwxyzABCD1234567890 for me" - .into(), + content: "review this key sk-abcdefghijklmnopqrstuvwxyzABCD1234567890 for me".into(), }], system: None, tools: Vec::new(), @@ -152,5 +151,8 @@ fn many_pre_calls_without_post_call_do_not_leak_past_slab_capacity() { "slab pressure should yield SLAB_FULL token" ); // post_call with SLAB_FULL is a documented no-op. - sdk.post_call(decision.token(), &LlmResponse::new(EndpointType::ChatCompletion)); + sdk.post_call( + decision.token(), + &LlmResponse::new(EndpointType::ChatCompletion), + ); }