refactor(api-types): extract shared cloud wire contract from soth-sync#69
Merged
secxena merged 1 commit intoMay 4, 2026
Merged
Conversation
The SDK and the proxy were silently drifting on the cloud wire format:
the proxy converted in-process `soth_core::TelemetryEvent` -> wire
`TelemetryEvent` via a private `map_event` in soth-sync, while the SDK
shipper just `serde_json::json!`'d the raw in-process struct. That
worked at first but blew up on the cloud as a 422 (`topic_cluster_id`
expected string, got integer 0) — and any future field divergence
would silently drop SDK telemetry without affecting proxy telemetry.
Extract the cloud API types and the in-process -> wire conversion into
a new pure-types crate `soth-api-types`. Both `soth-sync` (proxy) and
`soth-sdk-core` (SDK) depend on it, so a contract change in either
direction now propagates to both at once.
Also fix the latent SDK shipper bug that produced the 422:
- send `X-Soth-Api-Version: 2026-02-01` (matches soth-sync)
- run each event through `soth_api_types::convert::map_event`
- ship a real `TelemetryBatchRequest` envelope (batch_id,
device_id_hash="sdk:<org>", proxy_signature="sdk-unsigned"),
matching the proxy envelope shape
`SOTH_SHIPPER_VERBOSE=1` opts into stderr lines with the POST status +
event count so SDK customers can verify batching cadence at runtime.
Verified against the production cloud endpoint:
20 anthropic calls -> 1 POST /v1/edge/telemetry/batch -> 200 (20 events)
…from both soth-py and soth-node, with `https_proxy` unset (SDK is
genuinely proxy-independent).
Build/test parity (CI mirror, locally green):
- cargo fmt --all -- --check
- cargo clippy --workspace --lib --bins --tests
- cargo test --workspace --lib --bins
- cargo test -p soth-conformance-tests --test parity (2/2)
- WASM matrix: soth-detect / soth-classify on wasm32-{wasip1,unknown-unknown}
- cargo build -p soth-py / soth-node
- pytest bindings/soth-py/tests (45 passed, 2 skipped)
- npm test in bindings/soth-node (32/32)
Notes:
* `observation_records` on `TelemetryBatchRequest` becomes
`Option<Vec<serde_json::Value>>` so soth-api-types stays free of
the `soth-telemetry` (rusqlite/tokio) transitive closure. The
proxy serializes its typed records to Value at the call site;
JSON wire shape is identical.
* `proxy_signature: "sdk-unsigned"` is a documented placeholder
for the SDK lane — full ed25519 signing of SDK telemetry remains
Phase-2 work, called out in shipper.rs.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The SDK and the proxy were silently drifting on the cloud wire format: the proxy converted in-process
soth_core::TelemetryEvent-> wireTelemetryEventvia a privatemap_eventin soth-sync, while the SDK shipper justserde_json::json!'d the raw in-process struct. That worked at first but blew up on the cloud as a 422 (topic_cluster_idexpected string, got integer 0) — and any future field divergence would silently drop SDK telemetry without affecting proxy telemetry.Extract the cloud API types and the in-process -> wire conversion into a new pure-types crate
soth-api-types. Bothsoth-sync(proxy) andsoth-sdk-core(SDK) depend on it, so a contract change in either direction now propagates to both at once.Also fix the latent SDK shipper bug that produced the 422:
X-Soth-Api-Version: 2026-02-01(matches soth-sync)soth_api_types::convert::map_eventTelemetryBatchRequestenvelope (batch_id, device_id_hash="sdk:", proxy_signature="sdk-unsigned"), matching the proxy envelope shapeSOTH_SHIPPER_VERBOSE=1opts into stderr lines with the POST status + event count so SDK customers can verify batching cadence at runtime.Verified against the production cloud endpoint:
20 anthropic calls -> 1 POST /v1/edge/telemetry/batch -> 200 (20 events)
…from both soth-py and soth-node, with
https_proxyunset (SDK is genuinely proxy-independent).Build/test parity (CI mirror, locally green):
Notes:
observation_recordsonTelemetryBatchRequestbecomesOption<Vec<serde_json::Value>>so soth-api-types stays free of thesoth-telemetry(rusqlite/tokio) transitive closure. The proxy serializes its typed records to Value at the call site; JSON wire shape is identical.proxy_signature: "sdk-unsigned"is a documented placeholder for the SDK lane — full ed25519 signing of SDK telemetry remains Phase-2 work, called out in shipper.rs.