Conversation
Two pre-flight specs that block Plan 2 Phase 0 (soth-sdk-core scaffolding). Neither is implementation; both lock the public-API contracts the SDK will ship in customer dependencies. Once Phase 0 commits the types, every change to these surfaces is breaking for downstream code. docs/common/SDK_DECISION_API_SPEC.md (~560 lines) - Decision enum: Allow / Block / Redact / Flag (Reroute dropped — proxy-only) - BlockReason variants including UseAlternative for reroute-shaped policies - MessageRedactions: message-level replacement only; within-message edits are documented as proxy-only - DecisionToken lifecycle: created in pre_call, consumed once in post_call, slab-backed, panic-on-reuse-in-debug, log-and-ignore in release, 60s orphan sweeper, sentinel for slab-full - Wrapper exception contract per language: SothBlocked extends base Exception/Error (NOT openai.APIError / etc.), propagates past existing try/except API-error handlers; gating negative tests required per binding - Sync (≤5ms p99) vs async (≤300ms p99) budget with explicit allowed work - Conformance harness extension: compare_decisions() asserts variant + BlockReason kind + redact message_idx set; proxy_only fixture tagging docs/common/SDK_WASM_TRUST_BOUNDARY_SPEC.md (~450 lines) - ORT-Web spike result: realistic full-mode payload is 22-27MB compressed, not the original ≤8MB target (which was wrong) - Tier matrix: native = full local; browser/Bun/Lambda = full WASM; CF Workers / Vercel Edge = reduced; Fastly = full - ClassificationMode enum: Full | Reduced | CloudOptIn (no auto-promotion) - Reduced-mode capability statement: explicit list of what works (artifact redaction, counter-based anomalies, artifact policy, session dedup, telemetry shipping) and what doesn't (use_case_label, semantic anomalies, ML-conditioned org rules) - Cloud-classify wire format: explicit opt-in with separate DPA, never fallback, classification_location: "cloud" telemetry tag - Bundle CDN trust path: Ed25519 verify, hot-swap via ArcSwap, no on-disk cache option for serverless, no key-fetching endpoint - Conformance lanes: SDK-via-PyO3, SDK-via-WASM-reduced, SDK-via-WASM-cloud-optin (with stubbed local endpoint for hermetic CI) Both specs include "If this needs to change" sections so the lock-in is deliberate, not accidental. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
New workspace member crates/soth-sdk-core/ — the public-API surface every SDK binding (PyO3, napi-rs, WASM) consumes. Implements the contract locked by the two pre-flight specs landed in d68fb20: - docs/common/SDK_DECISION_API_SPEC.md - docs/common/SDK_WASM_TRUST_BOUNDARY_SPEC.md Public types (all #[non_exhaustive] where future variants are anticipated): Decision — Allow / Block / Redact / Flag (no Reroute) BlockReason — SensitiveArtifact / BudgetExceeded / PolicyRule / UseAlternative MessageRedactions — message-level only; within-message edits are proxy-only by design DecisionToken — opaque, Copy, slab-backed; SLAB_FULL + SENTINEL_FAIL_OPEN constants for backpressure / fail-open FlagSeverity — Info / Warning / Critical HmacKey — FromEnv / FromFile / FromCallback / Static; resolve() returns Zeroizing<Vec<u8>>; Debug never prints plaintext ClassificationMode — Full / Reduced / CloudOptIn StorageMode — InMemory / FileBacked BundleSource — Cdn / Embedded / Fallback SdkConfig + SdkConfigBuilder LlmCall (= TypedLlmCall), Message, Tool, LlmResponse, LlmChunk Observation, StreamObservation SdkError + SothSdk DecisionToken slab (src/slab.rs): 4096-slot fixed capacity, 95% pressure threshold returns SLAB_FULL per-slot generation counter detects token reuse panic-on-reuse in debug, log-and-ignore in release per spec §5.3 bounded in_use atomic for lock-free pressure check SothSdk facade (src/sdk.rs): init — validates HMAC key + ClassificationMode + bundle source; fails closed pre_call — synchronous decision (≤5ms p99 budget per spec §7.1); runs process_normalized + sync block path on credential/private-key artifacts; allocates slab token post_call — consumes token, runs full classify with fallback bundle, pushes TelemetryEvent onto in-memory queue stream_begin / stream_chunk / stream_end — token-consumed-once invariant for streaming refresh_bundle — Phase-1 stub in_flight_decisions / drain_telemetry_for_test — test hooks In-memory telemetry queue (src/telemetry_queue.rs): bounded VecDeque (4096 default), oldest-drops-on-overflow with dropped-count metric. Phase-1 wires HTTPS shipper. Compile-time Send + Sync assertions for SothSdk + Observation — bindings stash Arc<SothSdk> across host worker threads. Tests: 15 unit + 5 integration round-trip - init_succeeds_with_minimal_config - pre_call_then_post_call_balances_slab_and_emits_telemetry - pre_call_blocks_on_credential_in_user_message - streaming_round_trip_consumes_token_once - many_pre_calls_without_post_call_do_not_leak_past_slab_capacity (asserts SLAB_FULL behavior under 6K alloc pressure) Verification: - cargo build --workspace clean - all workspace lib tests pass (634 + 20 new = 654) - All 4 WASM combos (detect+classify x wasip1+unknown-unknown) clean - Conformance harness still green (7/7 fixtures, 0 strict failures) Phase-1 deferred (called out in crate README): - Real bundle CDN pull + Ed25519 verification - Background HTTPS telemetry shipper - Cloud-classify wire transport - Orphan sweeper for never-consumed tokens - Full org-rule evaluation - OTel span emission - Conformance harness lane through this facade Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…crate calls Phase 0 follow-up. The conformance harness (PR 5) ran proxy and SDK lanes through the lower-level crates directly. Phase 0 added soth-sdk-core as the public API facade, which is now an extra layer where parity can drift. This commit closes that gap. soth-sdk-core: - Add SothSdk::for_test(config, detect_bundle, classify_bundle) — doc-hidden ctor that bypasses the bundle-source dispatch so the harness can isolate facade-vs-direct parity from bundle-source differences. Phase-1's `init` will gain an in-memory BundleSource variant that subsumes this. soth-conformance-tests: - run_facade_lane(fixture, bundles) — drives SothSdk::pre_call → post_call, drains the in-memory telemetry queue, returns the Decision + emitted TelemetryEvent. - compare_sdk_vs_facade(sdk_lane, facade) — strict-parity comparison on the cloud ingestion contract surface (provider, model, endpoint_type, capture_mode, parse_source, parse_confidence, use_case, volatility_class, policy_kind, estimated_input_tokens, anomaly_flags). Per-call ephemeral fields (event_id, timestamp_epoch_ms, commitment_nonce, commitment_hash) are documented as excluded. - New test sdk_direct_and_facade_lanes_agree_byte_identical asserts every fixture produces a TelemetryEvent matching the SDK direct lane's classify output. Failure names the diverging field so the fix lands in soth-sdk-core, not the harness. Dependency: soth-conformance-tests now depends on soth-sdk-core via path. Verification: - All 7 fixtures pass the new sdk-vs-facade parity assertion - Existing proxy-vs-sdk parity still passes (7/7 strict, 2 advisory) - soth-sdk-core: 15 unit + 5 integration tests - soth-detect / soth-classify / soth-proxy lib tests unchanged - All 4 WASM combos (detect+classify x wasip1+unknown-unknown) clean Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
bindings/soth-py/ — Python binding consuming soth-sdk-core via PyO3.
maturin-built abi3-py310 wheel publishable as `soth` on PyPI.
Public Python surface (python/soth/__init__.py):
init(api_key, org_id, hmac_key_env=...) — module-level singleton
guard(call_fn, call=...) — pre/post lifecycle wrapper
SothBlocked — exception (NOT openai.APIError)
SothFlagged — Flag surface
BlockReason — typed reason
Negative tests (tests/test_blocked_propagates.py) gate the contract:
- test_soth_blocked_does_not_inherit_from_openai_apierror
- test_soth_blocked_propagates_past_openai_apierror_handler
- test_soth_blocked_inherits_from_base_exception_directly
If any future change makes SothBlocked inherit from openai.APIError or
any provider type, these tests fail immediately. See SDK_DECISION_API_SPEC.md §6.
soth-sdk-core: added DecisionToken::raw() / from_raw() so bindings can
round-trip the token across the FFI boundary without inspecting struct
internals. The token is still opaque per spec §3.4.
Cargo / build:
- bindings/soth-py added to workspace members but NOT default-members
(cdylib + cdylib needs Python toolchain to actually link)
- `pyo3/extension-module` is a soth-py feature, not always-on, so
`cargo build -p soth-py` succeeds for compile-checking. maturin
passes the feature when building wheels.
- Python 3.10 floor (3.9 reached EOL October 2025)
- abi3 — one wheel per platform×arch covers Python 3.10+
Verification:
cargo build -p soth-py clean
All workspace lib tests: 654 (unchanged)
Conformance harness: 7/7 fixtures (unchanged)
Phase 0 SDK core round-trip tests: 5/5 (unchanged)
Phase-1 follow-ups documented in bindings/soth-py/README.md:
auto-instrumentation, httpx middleware, with-context, Redact handling,
streaming wrapper, per-arch wheel matrix via cibuildwheel.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
bindings/soth-node/ — Node.js binding consuming soth-sdk-core via napi-rs.
@napi-rs/cli builds prebuilt binaries per platform/arch; published to
npm as @soth/sdk.
Public TS surface (index.js + index.d.ts):
init({ apiKey, orgId, hmacKeyEnv }) — module-level singleton
guard(asyncFn, { call }) — pre/post lifecycle wrapper
SothBlocked extends Error — NOT extends OpenAI.APIError
SothFlagged — Flag surface
BlockReason — typed reason interface
Negative tests (__test__/blocked-propagates.test.mjs) gate the contract:
- SothBlocked does not extend openai.APIError (instanceof check)
- SothBlocked propagates past try { ... } catch (OpenAI.APIError) blocks
- SothBlocked extends Error directly
Per SDK_DECISION_API_SPEC.md §6.3, customers' retry-on-API-error logic
must NOT silently retry policy blocks. The negative test will fail loudly
if the inheritance changes.
DecisionToken round-trip: tokens are stringified u64 across the FFI
because JS numbers can't safely hold the full u64 range. Bindings parse
the string back into the u64 via DecisionToken::from_raw, which was
added in this commit's companion soth-sdk-core change.
Cargo / build:
- bindings/soth-node added to workspace members, NOT default-members
- napi-rs 2.x with napi8 ABI feature (Node 18+ supported)
- `cargo build -p soth-node` compiles the cdylib clean
- Production binaries built via `npm run build` per the
@napi-rs/cli triples in package.json (linux-x64-{gnu,musl},
aarch64-linux-{gnu,musl}, darwin-arm64, darwin-x64, win32-x64)
Verification:
cargo build -p soth-node clean
cargo build --workspace clean (both bindings included)
All workspace lib tests unchanged
Phase-1 follow-ups documented in bindings/soth-node/README.md:
auto-instrumentation, undici dispatcher, withContext helper,
AsyncIterable streaming wrapper, per-arch loader logic.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Both bindings gain a streaming surface that mirrors the Decision API
spec §4.2 (`stream_begin → stream_chunk → stream_end`). Streaming was
the keystone gap before auto-instrumentation: ~70% of production LLM
traffic streams, so without this the SDK was silently broken on the
hot path.
Python (bindings/soth-py):
- PyStreamObservation napi class wrapping CoreStreamObservation in
Mutex<Option<>> for take-on-end semantics.
- New PyO3 method SothSdk.stream_begin(call) -> (decision, observation).
- python/soth/__init__.py adds:
async def guard_stream(iter_factory, *, call, chunk_extractor=None)
Async generator; raises SothBlocked on Block decision; default
chunk extractor reads OpenAI's chunk.choices[0].delta.content shape.
- tests/test_streaming.py: 3 round-trip tests (consume-once, block on
credential, double-end idempotent).
- Type stubs updated in _soth_native.pyi.
Node (bindings/soth-node):
- napi class can't compose with napi(object) wrappers, so observations
are kept on SothSdk keyed by token (raw u64 stringified across FFI).
streamBegin returns just the JsDecision; streamChunk(token, ...) and
streamEnd(token) look up by token.
- index.js adds `guardStream(iterFactory, { call, chunkExtractor })`
async generator with the same semantics as the Python version.
Default extractor is OpenAI-shaped.
- index.d.ts: GuardStreamOptions + ChunkExtractorOutput types.
- __test__/streaming.test.mjs: 3 tests mirroring Python.
Sentinel handling: SLAB_FULL / SENTINEL_FAIL_OPEN tokens skip the
observation slab — there's nothing to stash because pre_call didn't
allocate. streamEnd on a sentinel is a no-op.
Verification:
- cargo build --workspace clean
- All workspace lib tests unchanged (654)
- Conformance harness 7/7 fixtures still pass
Phase-1 follow-ups still deferred:
- chunk_extractor presets for anthropic/cohere/google/mistral
- Auto-instrumentation hooking streamBegin/streamChunk/streamEnd
- Streaming wrapper for non-AsyncIterator callbacks (Anthropic's
message_stream_text style)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Connects the SDK's in-memory telemetry queue to soth-cloud over HTTPS.
Without this commit, post_call emitted events into a queue nobody read;
now configured deployments actually deliver telemetry to the dashboard.
soth-sdk-core:
- New `http-telemetry` feature (off by default to keep workspace
builds fast). When on, pulls reqwest with rustls-tls.
- New module crates/soth-sdk-core/src/shipper.rs:
TelemetryShipper::spawn(queue, endpoint, api_key, org_id)
spawns a `soth-telemetry-shipper` thread that drains MAX_BATCH_SIZE
events every BATCH_WINDOW (5s), POSTs to the configured endpoint
with bearer auth, and Drop's cleanly. Final drain on shutdown
ensures events buffered during the last window are flushed.
- TelemetryQueue::drain_batch(max) — bounded pull for the shipper.
- SothSdk holds a Mutex<Option<TelemetryShipper>>; init spawns it
iff `telemetry_endpoint` is set, drop-on-shutdown is idempotent.
- SothSdk::shutdown() — public method bindings call at process exit.
V0 limits (Phase-2 follow-ups documented in shipper.rs):
- No retry / circuit-breaker (failed batches drop)
- No exp backoff
- No dead-letter queue
- No batch compression / signing / encryption
Bindings:
- bindings/soth-py and soth-node both enable http-telemetry by default
(production native bindings always want a working shipper).
- Both expose `telemetry_endpoint` parameter on init.
- Both expose `shutdown()` method for graceful exit.
- Python: soth.init(..., telemetry_endpoint="https://...") and
soth.shutdown().
- Node: soth.init({ telemetryEndpoint: "https://..." }) and
soth.shutdown().
Verification:
- cargo build -p soth-sdk-core (default, http-telemetry off): clean
- cargo build -p soth-sdk-core --features http-telemetry: clean
- cargo build --workspace clean (bindings pull http-telemetry on)
- soth-sdk-core 15 unit + 5 integration tests unchanged
- Conformance harness 7/7 fixtures unchanged
- The for_test() ctor never spawns a shipper so CI stays hermetic
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…rage Multi-tenant story: customers can override identity (user_id_hmac, team_id, device_id_hash, session_id, request_id) per-call rather than relying on SDK init defaults. Async-aware context propagation is language-native: Python contextvars, Node AsyncLocalStorage. soth-sdk-core: - New module crates/soth-sdk-core/src/context.rs with `CallContext` struct (#[non_exhaustive], all-Option fields, builder methods). - New `pre_call_with_context(call, &CallContext)` and `stream_begin_with_context(call, &CallContext)` methods. Old `pre_call` / `stream_begin` delegate with default context. - DecisionContext (slab) carries the resolved CallContext from pre_call to post_call so identity asserted at decision time matches what telemetry emits. - build_proxy_context now reads resolved context for user_id_hmac / team_id / device_id_hash / session_id; falls back to SdkConfig defaults when CallContext leaves a field None. Session ID is parsed as UUID where possible, otherwise dropped (proxy-shaped contract). - HMAC discipline preserved: user_id_hmac is the HMAC; the SDK never sees plaintext user IDs. Python (bindings/soth-py): - soth.context(*, user_id_hmac=, team_id=, device_id_hash=, session_id=, request_id=) is a contextmanager backed by contextvars.ContextVar so it survives asyncio task switches. - guard() and guard_stream() pull the current context and pass it to the FFI on every call. - PyO3 layer: pre_call(call, context=None) / stream_begin(call, context=None) accept an optional dict; build_call_context translates to CallContext. - Nested `with soth.context(...)` blocks merge — fields not set in inner block fall through to outer. Node (bindings/soth-node): - soth.withContext(overrides, async () => ...) backed by AsyncLocalStorage so async code awaited inside sees the same context across awaits. - guard() and guardStream() pull the current AsyncLocalStorage value and pass it to the napi FFI. - napi-rs layer: preCall(call, context?) / streamBegin(call, context?) accept an optional JsCallContext; build_call_context translates. - Nested withContext calls merge identically to the Python contract. Verification: - cargo build --workspace clean - soth-sdk-core 15 unit + 5 integration tests - Conformance harness 7/7 fixtures - All workspace lib tests unchanged - All 4 WASM combos clean - Both bindings recompile cleanly with both http-telemetry and per-call context Phase-1 follow-ups: - Helper for HMAC-ing a customer-supplied user_id (currently customer computes the HMAC themselves via their stored SOTH_HMAC_KEY) - Auto-instrumentation that picks up context automatically — that's the next commit - Context introspection helpers (get_current_user_id_hmac, etc.) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
`soth.instrument()` monkey-patches provider SDK client classes so
customers don't write `soth.guard()` per-call. Built robustly per the
brief — six guarantees gate the contract:
1. **Idempotent.** instrument() called twice returns
"skipped:already-instrumented" rather than double-wrapping.
2. **Reversible.** uninstrument() restores originals captured at
apply time. If another tool wrapped over our wrapper, we leave
theirs in place and log rather than clobbering.
3. **Provider-conditional.** Missing provider packages return
"skipped:not-installed" without raising — apps deploying with
subset-of-providers don't break.
4. **Fail-open.** Any exception in build_call extraction or wrapper
setup falls back to the original SDK call uninstrumented. The
customer's API call MUST complete unaffected if SOTH itself fails.
5. **Version-tolerant.** Adapters use defensive `getattr` /
`Object.keys` access and walk multiple possible paths into
provider modules so minor-version SDK changes don't break.
6. **Sync + async aware.** Each Python adapter wraps both sync
(`Completions`) and async (`AsyncCompletions`) classes; the
same `guard()` entry point handles both because it now detects
coroutine returns.
Python (bindings/soth-py/python/soth/instrumentation/):
__init__.py: registry + instrument() / uninstrument() /
is_instrumented(); status dict per provider.
_base.py: wrap_method(target, method_name, ...) replaces a
class method with a SOTH wrapper carrying provenance
markers (__soth_wrapped__, __soth_provider__,
__soth_original__). Sync + async wrappers built
separately. revert_all checks the wrapper is still
ours before restoring.
_openai.py: patches openai.resources.chat.completions.Completions
+ AsyncCompletions + Responses (when available).
buildCall extracts model/messages/stream/tools/system
from kwargs; flattens content arrays for
multi-modal calls. Tool params are stably serialized
for hash determinism.
_anthropic.py: patches anthropic.resources.messages.Messages +
AsyncMessages. Handles Anthropic-specific shape
(system as separate field, tools with input_schema,
streaming events with content_block_delta /
message_delta / message_stop).
guard() update (Python):
Detects coroutine return from call_fn. Sync providers finalize
inline; async providers receive a coroutine to await. One
entry point handles both sync `OpenAI` and async `AsyncOpenAI`
clients without separate APIs. New `guard_stream_sync()` for
sync-streaming providers (matches `guard_stream` async generator).
Node (bindings/soth-node/instrumentation/):
index.js: registry + instrument() / uninstrument() /
isInstrumented(). Lazy-loaded so `require('@soth/sdk')`
doesn't pull provider SDKs on import.
_base.js: wrapMethod walks prototype chain; provenance markers via
non-enumerable Object.defineProperty so they don't leak
into JSON.stringify of the wrapped method. revertAll
same third-party-detection semantic as Python.
openai.js: walks multiple possible paths to find Completions
class across openai 4.x sub-versions; falls back
cleanly when none match.
Negative tests for both bindings:
- instrument is idempotent (second call: "skipped:already-…")
- uninstrument reverses
- uninstrument-without-instrument is safe
- explicit providers list filters
- missing provider returns "skipped:not-installed"
- buildCall exception falls through to original (fail-open)
- wrapped method carries provenance markers
- revert leaves third-party wrapper in place
- guard() handles coroutine and value returns (Python only —
Node already async-aware)
- OpenAI/Anthropic adapter apply() returns bool, never raises
What's NOT in this commit (Phase-1.5 follow-ups):
- Cohere, Google GenAI, Mistral Python adapters
- Anthropic Node adapter (Anthropic's npm SDK is less mature
than openai's; will land separately)
- LangChain / LlamaIndex / Vercel AI SDK callback adapters
(those go through a different integration path; framework
adapters are Phase 3 in the original plan)
Verification:
cargo build --workspace clean
All workspace tests unchanged (654 + 5 sdk-core integration)
Conformance harness 7/7 fixtures
All 4 WASM combos clean
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…adapters
Extends auto-instrumentation to the long-tail providers. Each adapter
follows the existing OpenAI/Anthropic Python pattern (apply + revert +
build_call + chunk_extractor) and inherits the six robustness
guarantees from `_base.py` (idempotent, reversible,
provider-conditional, fail-open, version-tolerant, sync+async aware).
Python — bindings/soth-py/python/soth/instrumentation/:
_cohere.py: Cohere v5 ClientV2/AsyncClientV2 (OpenAI-shaped
`messages=[{role, content}]`) AND v4 Client
(`message=` + `chat_history=`) for legacy users.
Streaming chunk extractor handles v5
content-delta / message-end events.
_google_genai.py: Models.generate_content + generate_content_stream
(sync + async). Normalizes the four shapes of
`contents` (str / list-of-str / list-of-Content /
Content-dict) into a uniform `[{role, content}]`
list. system_instruction extracted from `config`.
Streaming reads `chunk.text` + finish_reason from
candidates[0].
_mistral.py: Chat.complete + complete_async + stream + stream_async
(mistralai 1.x). OpenAI-equivalent shape so
extraction is straightforward; chunk extractor
handles Mistral's `data` envelope around OpenAI
deltas.
Node — bindings/soth-node/instrumentation/:
anthropic.js: Patches Messages.create on @anthropic-ai/sdk.
Walks multiple typed-resource path candidates
for version tolerance across 0.x. Handles
Anthropic-specific shape: separate `system`
field, `input_schema` (not `parameters`) on
tools, content_block_delta / message_delta /
message_stop streaming events.
Registry updates:
Python REGISTRY now includes openai, anthropic, cohere, google_genai,
mistralai (5 providers).
Node REGISTRY now includes openai, anthropic (2 providers).
Tests added:
- cohere/google_genai/mistral apply() returns bool — no exceptions
- cohere v2 + v4 extractors normalize messages correctly
- google_genai handles string vs list-of-Content shapes
- mistral extractor produces OpenAI-shaped output
- Anthropic Node apply() + buildCall + chunkExtractor unit tests
Verification:
cargo build --workspace clean
All workspace lib tests unchanged (654)
Conformance harness 7/7 fixtures
All 4 WASM combos clean
Adapter tests pass on systems with/without provider SDKs installed
(apply() returns False when missing; tests skip cleanly)
Phase-1.6 follow-ups (framework adapters — different integration
pattern via callbacks/middleware): LangChain Python, LlamaIndex Python,
LiteLLM Python, Vercel AI SDK Node. Each lands as its own commit.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…ntegrations
Framework integrations layer on top of the existing instrumentation
contract. Where direct provider adapters monkey-patch the SDK class
methods, frameworks use callbacks/middleware instead — SOTH plugs
into each framework's native lifecycle rather than intercepting
underneath it.
Robustness contract is preserved across all four:
- Module-level imports succeed even when the framework isn't installed
- Instantiation raises a clear ImportError with the install hint
- Idempotent registration / un-registration
- Fail-open: extractor exceptions don't break the customer's chain
- Async-aware where the framework supports both modes
Python — bindings/soth-py/python/soth/integrations/:
langchain.py: SothCallbackHandler — implements both
BaseCallbackHandler and AsyncCallbackHandler from
langchain-core. Hooks on_llm_start /
on_chat_model_start / on_llm_new_token / on_llm_end /
on_llm_error onto SOTH's pre/post lifecycle.
Per-run state keyed by LangChain's run_id (UUID).
Block decisions raise SothBlocked from on_llm_start;
LangChain's invoke() propagates the exception up.
Provider inferred from `serialized.id` import path
(langchain_openai → openai, langchain_anthropic →
anthropic, etc.).
llamaindex.py: SothEventHandler — extends BaseEventHandler from
llama_index.core.instrumentation. Hooks the four
LLM lifecycle events (LLMChatStartEvent /
LLMChatEndEvent / LLMCompletionStartEvent /
LLMCompletionEndEvent). Provider inferred from the
event class's module path.
litellm.py: register() / unregister() / is_registered() — adds
success / failure / input handlers to litellm's
module-level callback lists. LiteLLM's `model`
strings are namespaced ("openai/gpt-4o-mini",
"anthropic/claude-3-5-sonnet"); the prefix is parsed
for SOTH provider attribution. Per-call state keyed
by `litellm_call_id` so concurrent calls don't
collide. Block decisions raise SothBlocked from the
input handler, aborting the litellm call.
Node — bindings/soth-node/integrations/:
vercel-ai.js: sothMiddleware() returns a LanguageModelV1Middleware
object compatible with `wrapLanguageModel({ model,
middleware })` from the `ai` npm package.
wrapGenerate routes non-streaming calls through
SothSdk.preCall/postCall; wrapStream taps the
LanguageModelV1StreamPart stream via TransformStream
to feed text-delta and finish parts to SothSdk's
stream observation. Provider inferred from
`model.provider` ("openai.chat" → "openai", etc.).
Tests:
test_integrations.py:
- module imports succeed without the framework installed
- LangChain provider inference, message normalization
- LiteLLM register idempotency (when installed),
build_call namespace parsing
- LlamaIndex import safety
__test__/integrations.test.mjs:
- sothMiddleware shape matches LanguageModelV1Middleware
- buildCall normalizes Vercel V1 params (prompt[].content[].text
→ flat string)
- inferProviderFromModel handles openai/anthropic/google/mistral/cohere
- middleware pass-through doesn't break customer flow
Verification:
cargo build --workspace clean
All workspace tests unchanged (654 + 5 sdk-core integration)
Conformance harness 7/7 fixtures
All 4 WASM combos clean
This commit closes out Phase 1.5. Phase 2 (CI matrix + FFI conformance
harness) is the next gating workstream before paying-customer pilots.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Closes the workspace-build verification gap. Until this commit, only
fmt + clippy + cargo test on Ubuntu were gated; the WASM build matrix
and the binding compile-checks ran only on dev laptops. Now every PR
that touches detect/classify/sdk-core or the bindings runs through
the same matrix that paying customers will install from.
.github/workflows/ci.yml — extended:
+ conformance job: cargo test -p soth-conformance-tests --test parity
(proxy↔SDK direct↔SDK facade lanes, all 7 fixtures)
+ wasm job: 2x2 matrix (target × crate) for the WASM build commit
Plan 1 PR 1 promised to enforce. Failures here mean an SDK-blocking
native-dep regression slipped in.
+ bindings-build-check job: cargo build -p soth-py and -p soth-node
on Linux. Real per-arch wheel/binary builds happen in the
dedicated workflows.
.github/workflows/python-wheels.yml (new):
PyO3 + maturin via PyO3/maturin-action@v1. abi3-py310 wheels —
one wheel per platform×arch covers Python 3.10+, so the matrix
is 5 jobs (not 5 × N-Python-versions).
Targets:
- manylinux2014 x86_64 / aarch64 (aarch64 cross via QEMU)
- macOS x86_64 (macos-13) / aarch64 (macos-14)
- Windows x86_64
Native-arch jobs run a smoke test (`pip install` from local
--find-links + `import soth`) before uploading the wheel.
Non-native arches just upload (full smoke runs in
ffi-conformance.yml downstream).
Builds an sdist as a fallback for unsupported platforms.
Triggered on: PR or push to main / sdk-* paths under the binding,
the sdk-core crate, or the workflow file. Manual dispatch supported.
Wheels uploaded as per-target artifacts, retention 14 days.
Publish-to-PyPI is a separate workflow (not in this commit).
.github/workflows/node-binaries.yml (new):
napi-rs via @napi-rs/cli. Builds prebuilt binaries for the 7
triples declared in bindings/soth-node/package.json.
Targets:
- linux x86_64 (gnu, musl)
- linux aarch64 (gnu, musl) — gnu via cross-toolchain;
musl via napi-rs's prebuilt docker image
- darwin x86_64 / aarch64
- win32 x86_64-msvc
Native-arch jobs run a require() smoke test before upload to
catch obvious linkage errors. Real test runs in
ffi-conformance.yml.
Same trigger semantics as python-wheels.yml. Binaries uploaded
as artifacts, retention 14 days.
What's NOT in this commit:
- PyPI / npm publish workflows (release-time concern, separate)
- Source-of-truth version bumping (manual today)
- Binary signing (when SOTH gets to signed releases)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds the fourth lane to the conformance harness: actual PyO3 +
napi-rs FFI. Until this commit, the harness ran proxy / SDK direct /
SDK facade lanes — all in pure Rust. Drift introduced by the
binding's marshalling layer (PyO3 dict construction, napi-rs class
serialization) had no test coverage. This closes that gap.
bindings/soth-py/tests/test_ffi_conformance.py:
Loads every fixture from
crates/soth-conformance-tests/fixtures/*.json. For each:
- Builds the LlmCall dict from typed_call (same conversion the
Rust SDK lane does in soth-conformance-tests/src/lib.rs).
- Runs through soth.guard() — crosses the PyO3 boundary.
- For credential / Block fixtures: asserts SothBlocked raises.
- For others: asserts the customer call returns and a single
TelemetryEvent is emitted with provider, model, endpoint_type,
capture_mode populated.
- Asserts in_flight_decisions == 0 (slab balanced).
parametrized over fixtures so each fixture name appears in test
output; failures point at the diverging field.
Includes a corpus-floor test that fails if the fixture count
drops below 7 (Plan 1 PR 5 baseline).
bindings/soth-node/__test__/ffi-conformance.test.mjs:
Mirror of the Python suite. Uses node:test, async-aware, drives
through soth.guard() (which crosses napi-rs). Same fixture corpus,
same assertions (TelemetryEvent shape on the cloud-ingestion
contract surface).
.github/workflows/ffi-conformance.yml:
Two jobs, runs on PR / push touching bindings or sdk-core:
python: maturin develop, then pytest the FFI conformance suite +
instrumentation + integration + streaming suites
node: npm run build:debug, then npm test (which runs every
*.test.mjs in __test__/, including ffi-conformance)
Failures here mean either:
(a) the binding's FFI marshalling drifted from the Rust facade
(b) the fixture corpus changed in a way the binding hasn't picked
up yet
Either way, the failure names the fixture and the diverging field
so the fix is targeted.
Verification:
cargo build --workspace clean
cargo test -p soth-sdk-core: 15 unit + 5 integration
cargo test -p soth-conformance-tests --test parity: 2/2 (proxy↔SDK,
SDK↔facade)
All workspace lib tests unchanged
What this closes vs Plan 2 spec:
✓ CI matrix — Rust workspace, Python wheels, Node binaries (prior
commit)
✓ FFI conformance lane — extends the harness through the actual
PyO3 / napi-rs boundary
Phase 2 is complete. Tier 1 (Python + Node, native, full local
mode) is now functionally shippable: customers can `pip install soth`
on macOS/Linux/Windows, `npm i @soth/sdk` on the same, both with
auto-instrumentation for OpenAI/Anthropic and framework integrations
for LangChain/LlamaIndex/LiteLLM/Vercel AI. Conformance harness
catches drift at four levels (proxy, SDK direct, SDK facade, FFI).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…ified)
Closes the Plan 2 phase ledger. Phase 4 in the original plan was
estimated at 4–6 weeks; what lands here is the **scaffold** that
freezes the customer-facing API surface so the next-PR's WASM bridge
work doesn't break anyone who's already integrated.
What this commit DOES deliver (production-ready):
cargo build -p soth-sdk-core --target wasm32-unknown-unknown
--no-default-features
is now clean. Required moving soth-classify to default-features=off
at the workspace level (mirroring soth-detect's existing pattern)
and explicitly opting back in to onnx-models from soth-bundle and
soth-proxy. Without this, tokenizers's onig C dep poisons the WASM
build for any consumer.
bindings/soth-edge/ (new package — @soth/sdk-edge):
- package.json with `npm run build:wasm` invoking cargo
- src/index.js — JS shim mirroring @soth/sdk's API
(init/guard/guardStream/shutdown/SothBlocked) — runs through
`_invokeWasmStub` placeholder until extern "C" exports land.
Decision tier matrix locked at module level: Reduced for CF
Workers / Vercel Edge; Full WASM for Deno / Fastly. No ONNX
in either path (per WASM trust-boundary spec §3).
- src/index.d.ts — TypeScript types matching the runtime API
- README — tier matrix, reduced-mode capability statement,
CF Workers + Vercel Edge usage examples, the explicit list of
what's still wired-as-stub
sdks/soth-go/ (new module — github.com/labterminal/soth/sdks/soth-go):
- go.mod with wazero v1.8.0 dep (no cgo — preserves Go's static-
binary, alpine, cross-compile story)
- soth/sdk.go — public Go API: Init / Guard / Shutdown /
SothBlocked. Mirrors the Python and Node bindings in spirit;
idiomatic Go (errors.Is / errors.As supported).
- soth/wasm_bridge.go — wazero bridge with the function signatures
soth-sdk-core's extern "C" exports will fill in. Today returns
stubbed Allow decisions so the Go API contract can stabilize
independently.
- soth/sdk_test.go — 4 tests gating the public contract:
* Init requires APIKey + OrgID + Hmac + WasmBytes
* Init succeeds with full config
* Guard invokes the customer callback on Allow (stub path)
* SothBlocked satisfies error / errors.As
* Shutdown is idempotent
- README — extensive doc on why wazero (not cgo), the embed pattern
(`//go:embed soth_sdk_core.wasm`), and the explicit Phase-4
follow-up list.
docs/common/SDK_PHASE_STATUS.md (new):
Single-page ledger of Plan 1 / specs / Phase 0 / Phase 1 / Phase 1.5
/ Phase 2 / Phase 3 / Phase 4 status with commit pointers. Distinguishes
production-ready from scaffold so reviewers and customers see
exactly what's wired today.
Workspace dep updates:
Cargo.toml — soth-classify gets default-features=false. Same
pattern soth-detect already uses. WASM SDK target builds; consumers
that need onnx-models opt back in.
crates/soth-bundle/Cargo.toml — explicit onnx-models feature
crates/soth-proxy/Cargo.toml — explicit onnx-models feature
Verification:
cargo build --workspace clean
cargo test -p soth-sdk-core: 15 unit + 5 integration
cargo test -p soth-conformance-tests --test parity: 2/2
All 4 WASM combos clean (detect+classify x wasip1+unknown-unknown)
+ soth-sdk-core wasm32-unknown-unknown clean
The SDK build is now dependency-decoupled enough to ship native
Tier-1 bindings (Phase 1), edge scaffold (Phase 4), and Go scaffold
(Phase 4) from one branch. The Phase-4 wasm-bindgen / extern "C"
work is the next dedicated PR.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Defers SDK-side user-ID hashing to Phase 2.5. In v1, customers pre-compute userIdHmac themselves and pass it via withContext / soth.context; the SDK no longer requires hmac_key_env or hmac_key_static at init. - soth-sdk-core: hmac_key is Option<HmacKey>; builder no longer errors when unset; init/for_test only resolve the key when present - bindings (PyO3 + napi-rs): (None, None) is now a valid config rather than an error; READMEs and docstrings document the privacy tradeoff for unkeyed deployments - New unit test: builder_succeeds_without_hmac_key Regulated workloads (HIPAA / heavy-PII) SHOULD still configure a key; without one, anything passed via userIdHmac reaches soth-cloud as-is. Full key-lifecycle contract: SDK_WASM_TRUST_BOUNDARY_SPEC §6.6. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Three independent CI bugs surfacing on every push to sdk-build:
1. Node: `napi build --platform` overwrites `index.js` and `index.d.ts`
wholesale, wiping the hand-written shim (`init`, `guard`,
`withContext`, `SothBlocked`, …). Tests on Linux were hitting
`TypeError: soth.init is not a function` because the only thing
left was a generated loader exporting `SothSdk`. Fix: pass
`--js binding.js --dts binding.d.ts` so napi-rs writes its
per-platform loader to a separate file. The shim now does
`require('./binding.js')` and survives rebuilds. Loader files
are gitignored (regenerated by CI / dev each build).
2. Python: `maturin develop` requires an active virtualenv and the
workflow ran it bare, so every push died with
`Couldn't find a virtualenv or conda environment`. Fix: create a
`.venv` in `bindings/soth-py`, install pytest+maturin into it,
and persist `VIRTUAL_ENV` / `PATH` via `$GITHUB_ENV` /
`$GITHUB_PATH` so subsequent steps reuse the same interpreter.
3. ffi-conformance.test.mjs asserted snake_case keys
(`endpoint_type`, `capture_mode`) — copy-pasted from the Python
lane — but napi-rs renders Rust struct fields as camelCase by
convention, so the binding correctly emits `endpointType` /
`captureMode`. Fix: assert camelCase in the JS lane; the
Python lane keeps snake_case.
After this, `npm test` is green locally (32/32) and the Linux CI
runner should resolve `soth-node.linux-x64-gnu.node` via the
generated `binding.js` loader.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
`actions/setup-node@v4` with `cache: 'npm'` and `cache-dependency-path: bindings/soth-node/package-lock.json` fails with "Some specified paths were not resolved, unable to cache dependencies" if the lockfile isn't checked in. The previous commit gitignored it; flip that. Lockfiles also matter beyond CI: package.json uses caret ranges for `@napi-rs/cli` and `openai`, so without a lockfile every fresh install can resolve to slightly different transitive versions. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Three failures on the node-binaries matrix:
1. aarch64-unknown-linux-gnu: cargo emitted aarch64 object files but
the host's x86_64 linker tried to link them, producing a wall of
`is incompatible with elf64-x86-64` errors. Set
`CARGO_TARGET_AARCH64_UNKNOWN_LINUX_GNU_LINKER` and
`CC_aarch64_unknown_linux_gnu` to `aarch64-linux-gnu-gcc` (already
apt-installed by the matrix step) so cargo and cc-rs both go
through the cross toolchain.
2. x86_64-unknown-linux-musl: napi-rs's
`nodejs-rust:lts-debian-musl` image was removed/renamed —
`manifest unknown` from the registry. Switch to the canonical
native-musl image `lts-alpine`; the build runs natively inside
Alpine so no cross-compile is needed.
3. aarch64-unknown-linux-musl: same dead image, but this one is a
real cross-compile (x86_64 host → aarch64 target). Switch to
`lts-debian-zig` and pass `--zig` to `napi build` so it routes
through `cargo-zigbuild`, which gives the linker a working
musl-aarch64 sysroot.
Native targets (linux-x64-gnu, darwin-{arm64,x64}, win32-x64-msvc)
already pass and are unchanged.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
After switching to `lts-alpine` and `lts-debian-zig`, the musl jobs moved past the previous "manifest unknown" failure but hit a new one: the napi-rs images ship Rust 1.83, while transitive deps (time-core 0.1.8, base64ct 1.8.3) require the `edition2024` Cargo feature stabilized in Rust 1.85. Add `rustup default stable && rustup update stable` to both musl build scripts so the in-container toolchain matches the host (`dtolnay/rust-toolchain@stable`) before `napi build` runs. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
GitHub's macos-13 Intel runner pool is heavily oversubscribed — the x86_64-apple-darwin job routinely sits queued past the queue timeout and gets auto-cancelled, blocking the workflow even when every other matrix entry succeeds. Move both Mac targets onto macos-14 (Apple Silicon, abundant runners) and cross-compile to x86_64-apple-darwin via `rustup target add`. The Xcode SDK on macos-14 ships both arm64 and x86_64 sysroots, so this is a first-class native cross — no zig, no Rosetta. Drop x86_64-apple-darwin from the smoke-test step since the arm64 host can't `require()` an x86_64 .node binary; ffi-conformance.yml is the real load-bearing test. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
`yum install -y openssl-devel || apt-get update && apt-get install -y libssl-dev pkg-config` parses as `(yum || apt-get update) && apt-get install ...` due to left-associative `||`/`&&` precedence. So even when yum succeeded, apt-get install ran on the CentOS-based manylinux2014 image and failed with `apt-get: command not found` (exit 127), killing every Linux wheel build. manylinux is always yum-based, so the apt fallback is dead code anyway. Drop it; install pkgconfig from yum directly. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two regressions on the python-wheels workflow, both surfaced after the last manylinux fix went in. aarch64-unknown-linux-gnu was failing at the maturin before-script with `yum: command not found` (docker exit 127). The previous fix dropped the apt fallback assuming "manylinux is always yum-based" — which is true for the native `manylinux2014` image, but the cross image used for non-native arches is `ghcr.io/rust-cross/manylinux2014-cross:<arch>`, and that one is Debian-based. Detect the package manager on PATH instead of assuming. x86_64-apple-darwin was sitting queued for hours every run on macos-13 (Intel) — same chronic oversubscription that node-binaries.yml already worked around in b5f5fa3. Move it to macos-14 (Apple Silicon, abundant runners) and cross-compile via `rustup target add`. The Xcode SDK on macos-14 ships both arm64 and x86_64 sysroots, so this is a first-class native cross. Drop the smoke test on x86_64-apple-darwin since the arm64 host can't import an x86_64 extension module; PyPI consumer install on real Intel hardware remains the load-bearing test. Both stuck runs (25216787926, 25189720745) cancelled. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Previous fix unblocked the apt-get/yum mismatch but exposed the next layer: ring 0.17.x's pre-generated ARM assembly fails to compile under the cross-toolchain bundled in `ghcr.io/rust-cross/manylinux2014-cross:aarch64` because that image's `aarch64-unknown-linux-gnu-gcc` doesn't predefine `__ARM_ARCH`. cc-rs aborts with `error: #error "ARM assembler must define __ARM_ARCH"` and maturin exits 1. Override maturin-action's default image with the native `quay.io/pypa/manylinux2014_aarch64` (CentOS-based, yum), letting the QEMU binfmt step we already set up emulate it on the x86_64 host. The emulated build uses a native aarch64 GCC so ring's assembly compiles cleanly. Trade ~5 min cross-compile speed for a build that actually finishes; the workflow as a whole runs once on push, no perf hit worth chasing here. Keep the apt-get fallback in before-script-linux as defense in depth in case the Debian-based image ever sneaks back in. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
CI's `cargo fmt --all -- --check` step on PR #63 was failing because several files in the SDK crates and bindings drifted out of rustfmt's canonical layout (mostly long error-builder closures that fmt wants broken across multiple lines, plus one re-ordered `use` block). Pure formatting, no behavior change. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.