Skip to content

fix: run Venice E2EE attestation before spending tokens#2

Merged
sh1ftred merged 16 commits into
mainfrom
feat/venice-integration
Jun 24, 2026
Merged

fix: run Venice E2EE attestation before spending tokens#2
sh1ftred merged 16 commits into
mainfrom
feat/venice-integration

Conversation

@sh1ftred

Copy link
Copy Markdown
Contributor

Previously attestation happened AFTER _spendToken(), meaning if attestation failed (e.g. unverified key, nonce mismatch, server error), the user had already lost their tokens with no refund path.

This moves E2EE attestation before any spending in both fetchAIResponse() and _prepareRoutedRequest():

  • Added _getAttestationAuth() helper that gets auth headers without the main spend:
    • apikeys mode: just looks up the stored API key (zero spend)
    • xcashu mode: does a minimal 1-sat spend for attestation auth; the returned token is reused for the actual request to avoid a double spend
  • If attestation throws, no tokens are lost — the error surfaces before _spendToken()
  • Updated VeniceImpl.md to document the new ordering guarantee

sh1ftred added 7 commits May 18, 2026 20:52
Previously attestation happened AFTER _spendToken(), meaning if
attestation failed the user lost their tokens with no refund.
Now attestation runs first — if it throws, no tokens are spent.

- Added _getAttestationAuth() helper that gets auth headers without
  the main spend (key lookup for apikeys, minimal spend for xcashu)
- xcashu mode reuses the attestation spend token for the actual
  request to avoid a double spend
- Both fetchAIResponse() and _prepareRoutedRequest() restructured
  to do E2EE attestation before _spendToken()
- Updated VeniceImpl.md to document the new ordering guarantee
@shroominic

Copy link
Copy Markdown

Verification: two fund-loss bugs + a credential leak — reproduced with a running harness

I built this PR (pull/2/head @ 341299e) and drove the real RoutstrClient for an e2ee-* model in xcashu mode, instrumenting cashuSpender.spend(). All findings below were reproduced by running code, not static review (stubbed fetch + spender; no live endpoints, no real funds).

1. Double-spend (1 sat + the full request amount)

One fetchAIResponse fires two spends: [1, 50].

  • _getAttestationAuth() spends 1 sat (RoutstrClient.ts:1838) and returns it as spendResult (:1853), but nothing consumes itfetchAIResponse (:584) and _prepareRoutedRequest (:381) both then call _spendToken({amount: requiredSats}) unconditionally.
  • The reuse-ternary that would prevent this exists only in VeniceImpl.md (lines 201, 205–206), never in code.

Observed: TOTAL spends: 2 | amounts: [1, 50].

2. The attestation token is spent before attestation is verified

Event order: spend(seq0) → attestation-fetch(seq1) → attestation-verified(seq2) → spend(seq3). The 1-sat spend at seq0 precedes verification (VeniceE2EE.ts:151), so a failed/forged attestation has already cost the user — contradicting this PR's stated goal ("attest BEFORE spending tokens").

3. _storeRequest writes live credentials to disk on every request

_storeRequest (RoutstrClient.ts:828, invoked at :746 from _makeRequest) serializes the full headers to reqs/req-*.json via JSON.stringify — including X-Cashu (spendable ecash) and Authorization: Bearer <api key>. reqs/ is not gitignored. Confirmed by running a request and reading the file back: the on-disk X-Cashu value is byte-for-byte the live spend token. This fires on every request, all models/modes — not just e2ee.

Reproduction

Harness: __tests__/pr2_doublespend.test.ts and __tests__/pr2_storeleak.test.tsvitest run prints the spend counts and the on-disk token.

Recommendation — please hold

Required before merge: (1) remove/redact _storeRequest (or gate it behind an explicit debug flag with auth headers stripped); (2) don't spend the request amount until attestation verifies; (3) resolve the attestation-payment design so a single payment covers the request (the e2eeSpendResult reuse described in VeniceImpl.md), or document why a separate paid attestation is intended.

A reworked version is on the way — I'll link it here.

@shroominic

Copy link
Copy Markdown

As promised, here's the reworked version: #14 (draft). It removes the _storeRequest credential-to-disk leak, locks attestation-before-spend with a regression test, and keeps a labeled characterization test for the double-spend so the remaining payment-design decision (reuse the attestation spend vs. a separate paid attestation — see the PR body) stays visible for you to resolve. It's branched off this PR's head, so it needs the same rebase onto main.

sh1ftred added 7 commits June 20, 2026 11:06
Route requests for models whose id starts with "tinfoil-" through Tinfoil's
SecureClient. SecureClient performs enclave attestation and encrypts request
bodies via EHBP (HPKE) so only the attested enclave can decrypt them; the
response stream is already decrypted before the SDK's SSE processing sees it.

Flow mirrors the existing Venice E2EE pattern (attest BEFORE spending tokens)
for both the initial request and the failover path, preserving the
attest-before-spend invariant so Cashu tokens are never minted for providers
whose enclaves cannot be attested.

New module client/TinfoilSecure.ts:
- isTinfoilModel() detects the tinfoil- prefix
- prepareTinfoilClient() lazily imports tinfoil, attests, and caches the
  SecureClient per resolved option set; failed attestations are evicted so
  the next call re-attests
- The tinfoil- prefix is NOT stripped; the caller-facing id is sent upstream

Review cleanups applied:
- Remove dead try/catch around getVerificationDocument(); the getter is a
  bare field return and cannot throw post-ready(), so the swallow-catch only
  masked impossible conditions and confused the type/log. Tightened
  TinfoilClientContext.verification to non-nullable VerificationDocument and
  dropped the optional-chaining and "unknown" fallbacks from the attestation
  log line.
- Drop redundant normalizeBaseUrl() call from cacheKey(); resolveOptions()
  already normalizes baseUrl, so the key was double-normalizing. Normalization
  now lives in one place.
- Remove imprecise "encrypted at fetch time" comment and reword the redacted
  body placeholder to "encrypted inside SecureClient.fetch" to accurately
  describe that HPKE sealing happens within the fetch() call.

Adds tinfoil@^1.1.6 as a runtime dependency.
The Routstr proxy's /tee/attestation endpoint is an unauthenticated free
GET that bypasses all auth/payment logic and is forwarded directly upstream.
No credential is needed to call it.

- Delete _getAttestationAuth() entirely
- Remove authHeaders param from fetchVeniceAttestation() and prepareE2EERequest()
- Call prepareE2EERequest() directly without any pre-attestation spend
- Add reqs/, *.sqlite, audit.log, scripts/events.db to .gitignore

Fixes double-spend in xcashu mode where a 1-sat token was burned just to
authenticate an endpoint that never checked it.
…odel header

The attested Tinfoil enclave expects the bare model id (e.g. "kimi-k2-6"),
not the caller-facing routstr id (e.g. "tinfoil-kimi-k2-6"). The SDK now
strips the tinfoil- prefix from the model id inside the EHBP-encrypted
request body via getTinfoilUpstreamModelId().

Since the body is HPKE-encrypted and opaque to the proxy, the full
caller-facing model id is sent in the X-Routstr-Model header so the
proxy can do model lookup, cost calculation, and routing without
parsing the encrypted body.

End-to-end model id flow:
  SDK → proxy (header):   X-Routstr-Model: tinfoil-kimi-k2-6
  Proxy → PPQ (header):   X-Private-Model: private/kimi-k2-6  (server-side)
  Enclave (body):         body.model = kimi-k2-6              (stripped)
…tream error logging, add custom EHBP fetch wrapper
@sh1ftred sh1ftred force-pushed the feat/venice-integration branch from f174123 to 0e706e8 Compare June 23, 2026 12:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants