docs(scenarios): the Scenarios section — 17 real-world API integration problems - #638
Open
rejifald wants to merge 30 commits into
Open
docs(scenarios): the Scenarios section — 17 real-world API integration problems#638rejifald wants to merge 30 commits into
rejifald wants to merge 30 commits into
Conversation
…uth2 refresh tokens A scenario is a real integration problem with no one-line answer anywhere. Each page states the problem, prices the solutions the ecosystem converged on, shows the StitchAPI shape next to them, and then says plainly what StitchAPI does NOT solve — so the remaining work is visible before anyone commits to it. First scenario: a single-use, rotating refresh token plus ordinary concurrency. The provider reads a replayed token as theft and revokes the whole token family, so the failure is "the integration lost the account", not "a request failed". Filed in the wild against openai/codex#10332, modelcontextprotocol/typescript-sdk#1760, oauth2-proxy#1992, and CVE-2026-53517 (better-auth). Verdict: achievable, but only with a user-written AuthStrategy — 79 lines for rotation + write-before-use durability + in-process single-flight, +42 more for cross-worker exclusion built on vault.increment(). Backed by 7 proof scripts run offline against the real runtime (docs/scenarios/proofs/), not by reading source: C1 20 concurrent cold callers -> 1 token request PASS C2 20 simultaneous 401s -> 1 refresh; staggered -> 10 C3 2 workers, shared store+key -> 2 token requests (store is a cache, not a lock) C4 oauth2() rotating grant -> family revoked FAIL C5 custom AuthStrategy -> 0 replays PASS C6 store-backed lock, 3 workers-> 1 redemption PASS C7 cookieSession as a seam -> {ok,status} only refuted C4 also surfaced a separate hazard: `params` can express a rotating grant that typechecks and succeeds on the FIRST redemption, then revokes the account — and the oauth2 guide names a custom grant_type as an intended use. Captured as an unfiled draft in docs/scenarios/issue-drafts/ for review. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…ot the status Scenario 2. Shopify's GraphQL Admin API meters a 1000-point bucket refilling at 50 points/sec, prices each query differently, and answers 200 OK with a THROTTLED entry in errors[] when you overspend. The wait is arithmetic the server hands you — (requested - available) / restoreRate — and the bucket belongs to the shop, so another app draining it moves your headroom between two of your own requests. Verdict: achievable, via a custom Surface — ~73 lines. `interpret` sees every body (detect + ledger) and SurfaceOutcome.after carries the computed deficit into the engine's own retry loop, so timeout.total, the circuit breaker, and retry events all keep working. Wrapping the adapter instead was measured going blind: 2 requests and a 6s sleep reported as attempts: 1 with zero retry events. Backed by 6 proof scripts, 125 checks, run offline (docs/scenarios/proofs/): C1 retry on 200-with-THROTTLED -> predicate gets 1 arg (status) FAIL C2 wait computed from the body -> backoff fn ignored; .after: 6000ms PASS via surface C3 extensions.cost reachable -> hooks.onResponse only PASS (one seam) C4 throttle.delegate on a 200 -> status-keyed, default [429] FAIL C5 throttle.rate as cost budget -> 18,000ms for an instant burst FAIL C6 assembled from public API -> 8/8 vs a draining neighbour PASS My own pre-verification framing was wrong and is corrected in the capture: C1-C5 all fail, but that does not total to "not achievable" — it missed the Surface seam (ADR 0022 Decision 5), which is public and exported. The real gap is that nothing points there: throttle's doc comment sends readers to `delegate`, which is status-keyed and cannot reach a body-reported quota. Three footguns captured as an unfiled draft, headline first: verdict.flag returns ok: true and hands the caller the THROTTLED envelope as data (absent path reads as "no signal"); .safe() drops RateLimitError.body; a cast-past backoff function is silently ignored rather than throwing. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…e request Scenario 3. A bulk endpoint (DynamoDB BatchWriteItem, Elasticsearch _bulk, SQS SendMessageBatch) answers HTTP 200 and reports inside the body that some items didn't land. Replaying the request re-applies the writes that already succeeded, so the correct behaviour is to rewrite the body to the failed subset, resend with growing backoff, and hand back whatever never landed. Verdict: achievable, ~50 lines on Surface.interpret + hooks.onRequest — the only seam in the library that can change a request between attempts. Measured against a capacity-limited table: 4 rounds at t=0/1000/3000/7000, abcdef -> cdef -> def -> f, zero duplicate writes, residue returned as data. Backed by 7 proof scripts, 183 checks, run offline (docs/scenarios/proofs/): C1 built-in retry -> 10 duplicate writes on a HEALTHY batch cannot C2 paginate as residue loop -> 0 duplicates, then 4/6 rows lost, ok:true trap C3 backoff between rounds -> six rounds at t=0; no field to declare one absent C4 can a Surface rewrite -> no; hooks.onRequest can key finding C5 retryable vs terminal -> 400 doc sent once, never written PASS C6 is the residue reachable -> no channel the engine owns carries it FAIL C7 assembled solution -> works, 50 lines vs 28 hand-rolled PASS Published honestly: this is MORE code than the hand-rolled while loop. The pitch is not brevity — it is that timeout.total, the circuit breaker, attempts, and retry events keep working, each measured against the hand-rolled version. My capture was wrong in both directions again: it nominated paginate (which is a trap here) and concluded no seam could rewrite a request (hooks.onRequest can). Escalated as the pass's strongest draft, and a bug rather than a footgun: paginate breaks on items.length === 0 BEFORE calling next (engine.ts:984), so a round that lands nothing — the ordinary response from a table out of capacity — ends the run with ok: true and the remainder unfetched. Hitting the pages cap returns ok: true too, so "finished" and "gave up" are indistinguishable, and a residue ledger built in next is stale by one round (reported cdef for a true residue of def). All three scenarios so far: achievable, but only off the documented path. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Scenario 4, and the first that spans several endpoints rather than one. A 202 with a Location header, a status endpoint that reports Failed at HTTP 200, a result URL that is single-use, a budget measured in hours, and a restart that orphans a job still running server-side. The tell that this has no easy answer: jsforce#298, go-salesforce#139 and salesforcer#13 are all the same request — the SDK's poll timeout is hardcoded, and the documented workaround is "turn the helper off and write the loop yourself". Verdict: achievable, ~110 lines across four seams — Surface.interpret (in-band state -> retry/fail/succeed, plus reading Retry-After), hooks.onResponse/onRequest (the Location hop, persisting the job id), linked + StitchInput.signal (one trace chain and the operation deadline), and per-stitch retry (poll patience vs a single-use download). Polling here is spelled as "retry, where the failure is 'not done yet'". Backed by 9 proof scripts run offline on an injected clock (docs/scenarios/proofs/): C1 Location -> next URL -> reachable via interpret/onResponse; nothing built-in follows it C2 poll loop as a Surface -> 5 polls at 30s virtual spacing; Failed stops at once C3 Retry-After on body path -> NOTHING honours it: asked 30s, measured 7ms C4 paginate -> refuted: it loops, but gaps 0,0,0 and it cannot fail C5 one deadline -> two routes, each losing something (see below) C6 linked trace chain -> 1 traceId, 3 spans, each parented to the last C7 single-use download -> default safe; per-stitch split gives 20 poll / 1 download C8 resumability -> entirely user-side; engine writes 0 store keys C9 assembled -> 110 lines vs 49 hand-rolled, byte-identical on the wire The finding worth carrying: this is a genuine design tension, not a gap. You can have ONE DEADLINE over the triangle, or PER-HOP RETRY POLICIES, but not both. Collapsing to one stitch buys timeout.total and loses the retry split (8 shared attempts burned on a dead link) plus concurrency safety; three stitches under linked keep both and replace the deadline with a caller-owned AbortSignal. The page leads with that trade rather than hiding it. Escalated two findings of the "the tool tells you something untrue" class: timeout.total is compared against wall-clock while its sleeps run on the injected clock, so under manualClock it never fires — 60 polls across 59 virtual seconds under a 10s budget, meaning a test of "give up after an hour" is guaranteed green and meaningless. And .inspect()/.report() re-issue the request: one safe + one inspect + one report submitted three jobs, two of them orphans. Fourth scenario, fourth time my pre-verification hypotheses were wrong. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Scenario 5, and the first to touch streaming. The 200 was spent on the first token, so every later failure arrives in-band or not at all: an SSE error frame, or a socket that simply stops. Retrying is not neutral — it re-runs the model, and duplicates content the consumer already accumulated. Split verdict, worth keeping: For a RESUMABLE feed (id: on every frame, Last-Event-ID honoured) this is one flag and it is correct — measured ABCDE, zero duplication, header moving (none) -> t2, server `retry: 9000` pacing honoured. First clean built-in win in five scenarios. For an LLM stream it needs ~62 lines across two seams: Surface.execute for connect-only retry, and the surface's `stream` hook to require [DONE] and reject in-band error frames. Notably the hand-rolled twin is LARGER (83 lines, it has to bring its own SSE parser) — the first scenario where StitchAPI is smaller. Backed by 9 proof scripts, 171 checks, run offline (docs/scenarios/proofs/): C1 drop mid-body -> .stream() and await disagree; clean close == success C2 does retry re-emit -> NO, refuted: retry never runs on a stream at all C3 reconnect, resumable -> ABCDE, zero duplication PASS C4 reconnect, id-less -> a COMPLETED stream replayed 4x BUG C5 in-band error frame -> interpret runs 0 times; only `output` works C6 missing [DONE] -> no built-in; 8 lines of surface `stream` hook C7 partial output -> .stream() only; every buffered accessor empty C8 connect vs body retry -> not expressible in config; Surface.execute does it C9 assembled -> 62 lines vs 83 hand-rolled PASS Escalated the pass's most severe finding, and it is in code shipped two commits ago (#622): `sse: { reconnect: true }` reopens a stream that COMPLETED CLEANLY. Measured 4 opens, 4 [DONE] sentinels, 24 deltas, the text ABCDEABCDEABCDEABCDE delivered to the consumer as one uninterrupted stream, ending done(ok: true). Two defects compose — `resumable` is decided from surface CAPABILITY before any frame is read (engine.ts:1288), so an id-less stream is classified resumable and every reopen requests the whole completion with no Last-Event-ID; and a clean close takes the same path as a drop (engine.ts:1466), so [DONE] terminates nothing. `sse: true` is the same flag. Smallest useful fix: reconnect.requireToken. The capture's C2 hypothesis was refuted — but the duplication hazard it was hunting for is real, just from sse.reconnect rather than retry, and it fires on streams that never failed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Scenario 6, and the first where the failure signal is a STATUS rather than something in-band: a 304 means "use what you have", carries no body, and is not a 2xx. Treat it as a failure and every unchanged poll errors; treat it as success and the caller gets undefined. The only correct move is substituting the cached body, which means the cache and the request path have to know about each other. Verdict: achievable, 87 lines on ONE seam — Surface.execute, the only position that owns a request and its own response in a single function, and the only one downstream of auth.apply that can key an ETag store by credential. It needs no custom interpret: the substituted body rides back on a still-304 response and classifyStatus only fails at >= 400, so .inspect().status honestly reports 304 while .data is the resource. Backed by 9 proof scripts, 170 checks, run offline (docs/scenarios/proofs/): C1 bare 304 -> ok:true, data:undefined; verdict.accept is a no-op C2 replay the validator -> 1 billed of 3; buildRequest vs onRequest differ C3 304 -> cached body -> PASS, and interpret DOES run on non-2xx C4 output schema -> breaks a bare poll; correct with substitution C5 built-in cache -> CANNOT revalidate; it is a value store C6 per-credential -> tenancy protects the built-in cache only C7 weak validators -> W/"v1" byte-exact both directions C8 the payoff -> 8/10 polls free with ZERO staleness C9 assembled -> 87 lines vs 79 hand-rolled The payoff table is the argument for the page: over 10 polls with one change, a TTL cache bills 1/10 and NEVER SEES THE CHANGE (5 of 10 polls serve a superseded version), while revalidation bills 2/10 and picks it up on the poll it happened. Cheaper and wrong, versus one extra request and correct. Published honestly: this is a wash on size — 87 lines against 79 for a feature-matched hand-rolled twin, identical behaviour on all four shapes. The 8 extra lines attribute exactly to two helpers that exist only because the engine hands a surface a shared, never-case-folded header record. Corrected a standing assumption: interpret runs for EVERY response including non-2xx on a buffered stitch (measured on [200, 304, 404]). Scenario 5's "interpret is dead code" is specific to runStreaming. The two paths disagree and nothing documents it. Escalated as a capability gap: cache is a value store so an ETag can never reach it (revalidateOnHit re-checks the SCHEMA, not the network), and a surface cannot see the bound principal — which is what makes a hand-written ETag store leak. That leak is measured: a store keyed on METHOD URL served bob alice's data, while the rate-limit metrics IMPROVED, because a 50% 304 rate is what working looks like. Second clock finding folded in: cache.ttl reads Date.now(), so a manualClock test of cache expiry passes vacuously — same shape as timeout.total in scenario 4. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Scenario 7, and the first about the REQUEST side. S3-style multipart is four steps, and the fourth — abort on failure — is the one no HTTP client models. Abandon an upload and every part already sent bills as storage indefinitely, while being invisible to `aws s3 ls`; AWS puts incomplete multipart uploads at up to 20% of an S3 bill. That is a COMPENSATING ACTION: a failure in step 2 obliges you to call a different endpoint. Verdict: achievable — 141 lines vs 163 hand-rolled — but this is the first scenario where the library contributes nothing to the requirement the scenario exists for. Retry-with-backoff, the concurrency pool, the retryable-status set and URL assembly became config, which is the 22 lines saved. The try/finally, the loud-cleanup rule, the per-part high-water progress map and the input-order assembly are byte-for-byte identical on both sides. Backed by 8 proof scripts, 201 checks, run offline (docs/scenarios/proofs/): C1 upload progress -> xhrAdapter 4 ticks; fetchAdapter 0; capability info event C2 ETag header, ordered -> stored [3,1,4,2], assembled [1,2,3,4]; settle order = 4 orphans C3 bounded concurrency -> all() bounds NOTHING (peak 8); pool:'host' gives 3 C4 compensation seam -> NONE EXISTS FAIL C5 retry granularity -> per-part fine; default retry.on excludes S3's 500 C6 cancelled siblings -> 2 parts stored, 0 nameable for the abort C7 progress aggregation -> naive sum 400 vs real 160; high-water is correct C8 assembled -> 0 orphans on every exit path C4 is a genuine FAIL and the reason for the issue draft. hooks.onError is not a failure hook — it is the catch around the transport, and fired 0 times on an HTTP 500. HookContext has no run-scoped slot to hold an UploadId, linked() has no finally, and an invented onFinally is accepted at runtime and never runs. Measured orphans with no cleanup: 3 parts / 15 MiB / 0 DELETEs. Via AbortSignal: 2 orphans. Via timeout: 3 orphans. Cancellation cancels in-flight work and forgets what landed. Two footguns that count double, both measured: `.safe()` on the abort cannot throw, so a correct-looking try/finally aimed at a wrong UploadId leaves 3 orphans and throws nothing; and cleanup inside Surface.execute runs AFTER the caller returns — 3 orphans and 0 DELETEs at the instant the caller saw the timeout, the DELETE landing several turns later, which in a lambda never happens at all. Also recorded in the ledger, verified and deliberately left unfixed to keep these commits scoped: run-identity.mdx:26,33 describe a `pipe()` combinator, but stitchapi/pipe exports exactly all, any, linked, race — the construct described is linked(). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…ally is
Scenario 8, and the first inbound one. The honest answer is that StitchAPI does
not receive webhooks — the-stitch.mdx:44 already says inbound webhooks stay the
application's job. This page makes that boundary concrete and measured rather than
asserted, and says what the library does own on the far side of it.
Verdict: SPLIT — receipt OUT OF SCOPE by design, reaction in scope and the whole
of it.
Backed by 7 proof scripts run offline against a real local node:http server
(docs/scenarios/proofs/webhook-receipt/):
C1 raw bytes at any path -> 404 at all 5 paths; body pre-JSON.parse'd;
INBOUND HEADERS DROPPED ENTIRELY; 162 signed bytes
vs 153 after a round-trip
C2 inbound signature prim -> none; 72 exports enumerated; aws-sigv4 key is
usages:['sign'] so it structurally cannot verify
C3 a seam on serve -> none; an UNSIGNED FORGED BODY ran the stitch, 200
C4 fetch-on-receipt -> payload order moot; WRITE order is not
C5 StitchStore as ledger -> PASS; get+set races [true,true,true],
increment gives [true,false,false]
C6 fast 2xx -> no; serve acked only after 10 virtual seconds
C7 the boundary as a number -> receipt 154 lines, ZERO stitchapi runtime imports;
reaction 63 lines, mostly config
71% of the code by line, and 100% by concern, is the half StitchAPI does not
participate in. That is by design, and the page leads with it.
Three things the capture missed, all worth knowing: the inbound headers are
dropped (not just the raw bytes), so there is nothing to verify against either
way; `serve` is unauthenticated, so mistaking it for a webhook endpoint is not a
404 but an open endpoint; and fetch-on-receipt fixes payload order without fixing
write order — concurrent handlers still need a version guard.
Escalated a finding that applies to ANY stitch, not just this scenario:
`void call(input)` — the idiomatic ack-then-continue spelling — made 0 HTTP calls
and raised 0 errors, because a stitch call is a lazy thenable that starts on
.then. In a webhook handler that means the work is dropped after the provider was
told 200, and it will not retry. `.safe()` in that position reports the failure
nowhere. Also: backoff.base is silently clamped by backoff.max (base: 30_000
measured a 10,000ms sleep), the third silent policy downgrade found in this pass.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Scenario 9, and the highest production impact of the pass. A SaaS calling one
vendor API on behalf of many customers: the unit of FAILURE is the tenant, but the
unit of PROTECTION is the dependency. When protection is broader than failure, one
customer's problem becomes everyone's.
Measured on a shared seam with circuit: { failures: 3, cooldown: '30s' }: one
customer with a revoked token failed 9 OF 9 HEALTHY CUSTOMERS with 503 circuit
open, and zero of their requests reached the vendor. Worse than predicted — it
does not self-heal: half-open admits one trial call and the broken tenant is the
one retrying hardest, so across four cooldown windows the healthy tenant measured
503,503,503,503. Recovery is a race, not a policy.
Verdict: achievable — roughly 3 strings per tenant — but the fix has to be KNOWN,
and nothing in the types, docs or runtime points at it.
Backed by 8 proof scripts, 152 checks, run offline (docs/scenarios/proofs/):
C1 shared breaker -> 9/9 healthy down, never self-heals
C2 partition breaker -> key STRING only; 10 stitches -> 1 key; 10 seams -> 1 key
C3 exclude a 401 -> accept:[401] alone SWALLOWS it; + flag:'ok' is correct
C4 noisy neighbour -> quiet tenant t=0 -> t=2000; with concurrency, LAST
C5 partition throttle -> two ways; pool:'host' collapses both AND re-keys circuit
C6 token isolation -> works, fails closed; but default tenancy is 'app'
C7 cost of isolation -> cheap (0 timers, 0 pools) but breaker keys are immortal
C8 four resources -> 2 isolated by principal, 2 by a hand-written string
The framing worth keeping: the split falls exactly on the auth/resilience line.
AuthContext.principal reaches the auth strategies and the cache-key builder and
nothing else — so the two resources whose isolation is a SECURITY property fail
closed, and the two whose isolation is an AVAILABILITY property fail open,
silently.
Three wrong hypotheses in the capture, and for the first time two were wrong
OPTIMISTICALLY: isolation is a property of the key string and never of the object
graph (10 separate seams still share one breaker), the rate bucket does partition
two ways, and 100 per-tenant seams cost under 40kb each with zero timers and zero
pools because a seam owns no transport.
Escalated as the pass's top triage item. The ask is small and sits on an axis the
codebase already has: add tenancy?: 'principal' | 'app' to ThrottleOptions and
CircuitOptions, the same axis CacheOptions and OAuth2Options carry, and thread the
principal into hostKey/createCircuit.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Scenario 10, and the last untested exports in stitchapi/pipe. Two techniques wear
similar clothes — failover (one call on the happy path) and hedging (two calls,
always) — and picking the wrong one is a bill, not a bug report.
Verdict: achievable, ~30 lines of routing on `linked`. Everything PER PROVIDER is
free and declarative: different origins, paths, auth strategies, retry policies,
breakers, timeouts and response shapes composed with zero glue, and neither
credential appeared on the other provider. The library carries ~74% of the
scenario and all of it is per-member; its contribution to the routing BETWEEN
members is zero.
Backed by 8 proof scripts, 165 checks, run offline (docs/scenarios/proofs/):
C1 does any() call both -> 20 requests for 10 answers on a healthy primary
C2 sequential fallback -> linked+try/catch: [10,0] and ONE traceId
C3 classification -> AggregateError drops status AND body
C4 one input per member -> per-call authorization reached the OTHER provider
C5 winner identity -> unrecoverable; group emits 0 events
C6 cancellation -> cancelled != free; two equal providers save 0
C7 hedge amplification -> race is 2.00x healthy AND degraded, no threshold
C8 assembled -> 30 lines vs 104 hand-rolled
The headline: any() is named and documented for failover ("a primary and a mirror,
two regions, two providers") and priced as a hedge. Ten calls where the primary
succeeded every time cost 20 provider requests. "The losers are auto-cancelled"
reads as "the losers are free" and is neither — the abort runs in a finally AFTER
the winner settles, so the request always arrives (10/10 completed, 0 aborted),
and a cancelled loser is still billed for the winner's latency. Two equally fast
providers: 80 units of work for one answer, 0 saved. And any() has no preferred
member — a healthy primary 10ms slower LOST, so it silently routes away from the
provider you chose.
The finding I would fix first is smaller and sharper: a per-call header is
broadcast to every member, so a `headers: { authorization }` written for the
primary arrived at the backup VERBATIM. One vendor handed another vendor's
credential, no type error, no warning.
Escalated both, plus a second sighting of scenario 9's breaker-key collision from
a different direction: two url-only stitches key on the literal string 'stitch',
so the primary's outage opened the BACKUP's breaker and fast-failed 3 of 5 healthy
calls. Setting `name` fixes it — which makes a diagnostic label load-bearing.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Scenario 11, and paginate's third appearance — deliberately from the data-
correctness angle rather than the mechanics. Offset pagination over a live
collection silently returns wrong lists, and every damaged run reports success:
ok: true, error: null, findings: [], status: 200, byte-identical to a clean run.
MY CAPTURE HAD THE CAUSATION BACKWARDS, and the proofs caught it against ground
truth. The correct version:
insert behind the cursor -> rows shift to HIGHER indices -> the next fixed
offset lands on a row already read -> DUPLICATE (measured ["r04"], 11 items
for 10 rows). An offset insert can never cause a skip.
delete behind the cursor -> rows shift to LOWER indices -> the next offset
jumps past one -> SKIP (measured ["r05"]).
This is the first capture error of FACT rather than prediction in eleven
scenarios, and it would have shipped a page teaching the wrong mechanism.
Verdict: achievable — keyset is 4 lines through next(prevBody) and correct on
every workload that broke offset. Detection, when only offset is on offer, is
yours to write; the safe seam is `output`, after the loop.
Backed by 8 proof scripts, 213 checks, run offline (docs/scenarios/proofs/):
C1 insert behind cursor -> duplicate, not skip REFUTED IN DIRECTION
C2 delete behind cursor -> skip, and total moves with it REFUTED, worse
C3 ties, ZERO writes -> skip AND duplicate, cancelling exactly
C4 keyset via next -> clean on all workloads; 4 lines
C5 detection seams -> deduping in items LOST 6 ROWS
C6 total reconciliation -> the standard check fired 0 of 4
C7 drift vs known edges -> empty page mid-run; pages:50 silently truncates
C8 assembled -> 0 false negatives, 84 lines vs 74 hand-rolled
Two findings worth the escalation. Deduping by id — the standard mitigation — put
in items/transform runs ABOVE the break at engine.ts:984, so an all-duplicate page
aggregates zero items, the loop reads that as the end, and the run finished ok
having skipped 6 rows. The fix causes the damage. And the check everyone writes
(length === total) fired 0 of 4 times: a delete removes one row from the result
and one from `total` simultaneously, so the arithmetic balances while a row is
gone. The signal that does carry it is that the DECLARED TOTAL MOVED.
That break at engine.ts:984 has now cost data in three separate scenarios (3, 4
and 11). It is one line, and it is the most expensive default found in this pass.
Published honestly: 84 lines vs 74 hand-rolled — the library version is longer,
because the detection is identical in both. What the 74 lines lack is the
resilience stack: one retry line recovered a page that 500'd mid-run.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Scenario 12, and the last untested headline feature. The existing recipes cover
catching a breaking change; this is the operational case — a canary rollout, or a
shape that varies by data, so drift is intermittent and only interpretable as a
rate.
Leveled drift tests WELL, and both of my hypotheses were refuted in the library's
favour — a first for this pass:
The default is SAFE. z.number() on "12345" fails the call; z.coerce.number() on
"abc" fails too because Zod rejects NaN. StitchAPI does not manufacture a $0
charge on its own. The capture feared it would.
Aggregation WORKS. The capture predicted it would be the gap, citing every
prior scenario's failure to find cross-call state. Wrong: TraceSink + ctx.spanId
is the one place in the library where cross-call state is the design rather than
a leak. Measured 5.0% -> 25.0% on a rolling window as the canary widened.
Precision is excellent too: over 100 calls with a geocoder returning null on 5,
drift fired on exactly calls [20,40,60,80,100], matching the vendor ledger, zero
false positives, each naming the field.
Backed by 8 proof scripts, 142 checks, run offline (docs/scenarios/proofs/):
C1 added field -> one info|undeclared; 51 values collapse to 2 findings
C2 removed field -> error|invalid + failed call, IF required
C3 $0-transaction -> default safe; but coerce+null -> 0 with no .catch()
C4 5% null -> exact precision; .nullable() -> nothing at all
C5 severity -> 3 soft kinds re-level; keyed by mechanism not change class
C6 aggregation -> works; 5.0% -> 25.0%
C7 actionability -> path always; await/.safe() carry nothing for a soft finding
C8 assembled -> zero $0 charges; the soft schema produced TEN
Two gaps survive. A `coerced` finding is kindOf(old) -> kindOf(new) with no
values, so "12345"->12345 (correct) and "abc"->0 (a $0 charge) emit BYTE-IDENTICAL
findings — only a spanId join in a sink separates them. And the fourth industry
change class, "nullable is a warning, value intact", has no spelling: required
fails the call and discards the four fields that were fine, .nullable() reports
nothing so the rollout is invisible, .catch('') fabricates a value. The
hand-rolled classifier beats DriftOptions on exactly that row.
The footgun worth knowing: z.coerce.number() maps null -> 0 with no .catch()
involved, because Number(null) === 0 — as do "", " ", false and []. That is the
$0 transaction arriving through the front door rather than the library's.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Scenario 13, and the first about MEMORY rather than correctness — nothing returns
wrong data, the process just dies. Backed by 8 proof scripts (97 checks), one
process per measurement, peak retained heap sampled after a forced GC.
The finding, and it is precise: the library ships a genuinely O(1) NDJSON decoder
and spends the win one line later.
ndjson decoder driven DIRECTLY: 0.8 MB retained for 1,000,000 rows / 214 MB
the same decoder THROUGH the engine: 3.5 -> 30.2 MB, linear
engine.ts:1443 pushes every delta onto a `chunks` array unconditionally so the
terminal `result` can mirror the spine. `.stream()` does not escape it — measured
30.2 MB iterating against 33.5 MB awaiting, the same number twice — and the
engine's own MEMORY NOTE recommends `.stream()` as the mitigation for exactly this,
which the measurement refutes.
Second defect, one branch: `decode: 'json'` over a single top-level array streams
the parse and buffers the text. Emission is CORRECT under every adversarial case
(commas in strings, escaped quotes, embedded newlines, 1-char chunk boundaries),
but heap tracks the whole array at 0.88x wire, time is quadratic, and it trips its
own 8,388,608-char default at ~37,000 rows with a message blaming the vendor for a
"malformed or never-closing value". On a 60,000-row array the consumer gets 37,312
rows then error/done(ok:false) — a silent truncation for any loop matching only
`delta`. Root cause: compact() floors on valueStart, which for a top-level array
is the opening `[`. Control: the same 100,000 records as CONCATENATED top-level
values run flat at 0.9 MB.
C1 buffered baseline -> 2.5x, matching bare JSON.parse to within 0.2 MB
C2 ndjson flat? -> decoder yes, engine no
C3 decode:'json' -> correct emission, unbounded memory, silent truncation
C4 does output rebuffer -> NO, capture refuted: per-delta, free
C5 pick/transform -> neither runs; transform called 0 times
C6 backpressure -> propagates; the cap throws; it is not a budget
C7 buffered guard -> none; exit 134, no catchable error, no finally
C8 assembled -> 1.3 MB flat vs 53.8 MB, a 40x cut, one seam, 75 lines
Fair to the library on the baseline: the 2.5x multiplier is JSON.parse's, not
StitchAPI's — overhead over a bare parse of the same bytes was 0.2 MB. It adds no
copy, and it removes none.
Also recorded: the buffered and streaming paths now disagree about six things
silently (retry, interpret, pick, transform, output-transform, stream({kind})),
which one "this slot does nothing on a stream" diagnostic would cover.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Scenario 14, and the FIRST in this pass to come out ACHIEVABLE outright for its
deciding claim. It also exercises @stitchapi/aws-sigv4, the one package the pass
had not touched.
The deciding question was whether a signature ages in a queue you control. It does
not, and the reason is structural: acquireWithin (engine.ts:629) sits ABOVE
cfg.auth.apply (:649) inside the attempt loop, and cloneReq gives each attempt
fresh headers off the UNSIGNED base.
4 calls behind rate:'1/2m', granted 0/2/4/6 virtual min
-> signature ages [0,0,0,0] ms, statuses [200,200,200,200]
the same calls PRE-SIGNED (the botocore#149 shape)
-> ages [0,2,4,6] min, and a 403 on the last
A StitchAPI throttle cannot expire a signature. Nor can a concurrency cap (0ms
after 6 virtual minutes), nor a retry backoff (3 attempts, 3 distinct signatures),
nor a 10-minute Retry-After park. The breaker does not even queue a signed request
— 3 blocked calls produced 0 signings.
Backed by 8 proof scripts, 105 checks, stable across 24 runs:
C1 signed per attempt -> 3 distinct signatures, ages [0,0,0]
C2 wait before signing -> BEFORE. The deciding claim, positive
C3 concurrency -> same
C4 circuit cooldown -> fast-fails before signing
C5 injected clock -> NO: 600 virtual seconds moved the stamp 0s
C6 skew 403 -> not retried (good); but opens the circuit
C7 skew correction -> reachable via shouldRefresh/refresh, 600000ms learned
C8 assembled -> 4 of 4 through 10-min drift AND a 6-min queue
Clock drift itself is still the user's: per-attempt signing faithfully re-mints
the same wrong time (4 attempts, 4 identical 600000ms skews). The AWS SDK
correction — learn the offset from the server's Date header, re-sign — is
reachable in 26 lines, and re-signs the SAME attempt so it costs no retry budget.
The footgun is the inverse of the finding: hooks.onRequest runs AFTER signing
(:652 vs :649), so a hand-rolled pacing gate there re-creates botocore#149 inside
a library that is otherwise immune to it — measured, a 6-minute wait in onRequest
aged the signature 6 minutes and got a 403. Pace with throttle, never with a sleep
in a hook.
Two patterns reached their third sighting and are now recorded in the ledger:
SigV4 joins timeout.total and cache.ttl in reading wall-clock while its neighbours
use the injected clock; and verdict.flag's absent-path rule produced a silent
success for the third time (scenarios 2, 11, 14), each fixed by ~6 lines of
Surface.interpret.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Scenario 15, and the highest-stakes of the pass because the measurement is
charges — counted against a ledger the fake vendor owns, not inferred.
A payment POST times out with no response. You cannot tell "never arrived" from
"processed, response lost". Retrying may double-charge; not retrying may lose the
charge.
The deciding finding: `idempotency: true` is per CALL, not per INTENT. The default
key is randomUUID() minted inside buildRequest (engine.ts:172, :257), so a queue
re-driving a job after a crash mints a new key.
re-driven job, `idempotency: true` + retry -> 2 keys, 2 CHARGES for 1 intended
the same job with `keyOf` -> 1 key, 1 charge
End to end over six workloads (crash-and-re-drive, lost response, TTL expiry, two
concurrent runs, a decline): the default produced 8 CHARGES FOR 6 INTENDED
PAYMENTS. A derived key plus a query-first recovery produced 5, the sixth
legitimately declined.
And the guard is narrower than readers will assume. stitch.ts:386 warns only when
there is a random key AND no retry — sound on its own terms, since a random key
does protect the retries inside one call. The effect is that following the
warning's own advice ("add retry") silences it, while leaving the restart case
open. The message's own wording, "only dedupes its own retries", is accurate and
is exactly the limitation, so the fix may be a re-wording rather than a behaviour
change.
Backed by 8 proof scripts, 120 checks (docs/scenarios/proofs/):
C1 same key per attempt -> 1 key, 1 charge; a lost response was RECOVERED
C2 restart-stable? -> NO. 2 charges for 1 intended FAIL
C3 derived key -> JSON.stringify(body) moved on KEY ORDER alone
C4 cached 500 -> not retried by default; capture REFUTED
C5 key/body mismatch -> 409 not retried, actionable body
C6 TTL expiry -> 2 charges, clean 200, no replay marker FAIL
C7 timeout ambiguity -> field-for-field identical errors FAIL
C8 assembled -> 5/6 vs the default's 8/6
Two findings neither side anticipated. `keyOf: (i) => JSON.stringify(i.body)` —
the obvious spelling — fails by CHARGING TWICE rather than erroring, because key
order alone moves the hash and the vendor never sees the same key twice (statuses
[200,200,200], no 409). And a STABLE key makes a recorded failure sticky for the
whole TTL — a declined card stayed declined — while the random key never reaches
the record and charges again. Both are defensible; nothing selects between them.
verdict.flag's absent-path rule swallowed an idempotency 409 — its FOURTH silent
success in the pass (scenarios 2, 11, 14, 15), now the single most-repeated
finding, and fixed by ~6 lines of Surface.interpret every time.
Also repaired: iteration 14's drafts-table row was never added to the ledger.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Scenario 16 — the most common composition shape in API integration, and the one
that produced the pass's strongest POSITIVE result.
cache.coalesce genuinely collapses IN-FLIGHT duplicates. 100 concurrent calls over
30 distinct ids made 30 REQUESTS — one per id, exactly the floor — with every call
in flight and not one response landed, from a single `cache: { ttl }` block and no
user code. 70 callers were served without a request of their own; coalesce: false
on the same cache made 100. Most clients do not have this, and the N+1 fan-out is
precisely the shape it fixes.
Three more things are configuration: throttle:{concurrency:8} held peak 8 exactly
against 100 unbounded; the retry default genuinely de-clusters a herd (100 calls
429'd in one instant retried across ~98 distinct milliseconds, where 'fixed' AND
'expo' both put all 100 into ONE millisecond); and `linked` gives one trace with
per-call inputs. 45 executable lines against 87 hand-rolled.
Where it loses is the failure path. A coalesced FAILURE is not shared — the leader
releases every joiner to re-run, so 100 concurrent calls for one 404ing id made
100 requests in two waves. Each joiner gets its own honest 404: the right error at
the wrong price. End to end, 44 requests against the hand-rolled 32, and a deleted
customer cost 4 against 1 — exactly the shape a dead foreign key takes.
Backed by 8 proof scripts, 180 checks (docs/scenarios/proofs/):
C1 combinators -> confirmed, WRONG REASON: the input broadcast, not length
C2 cache.coalesce -> 30 requests for 100 calls / 30 ids STRONGEST POSITIVE
C3 bounded concurrency -> peak 8 on one stitch; 100 on 100 stitches
C4 partial failure -> .safe() keeps 99; Promise.all keeps 0 and still pays
C5 thundering herd -> default works; Retry-After re-clusters it
C6 the trace -> one tree via linked, at depth 101 fan-out 1
C7 ordering -> positional and safe; ALIASING is the finding
C8 assembled -> 45 lines vs 87, but 44 requests vs 32
My capture was right that the combinators can't express this and wrong about why:
the runtime length is fine (all(ids.map(...)) compiles), the INPUT BROADCAST is the
wall — 100 members with one input made 100 requests for ONE distinct id. Third
scenario to land on that same broadcast (7, 10, 16), so a per-member input would
close all three.
Escalated two things beside the positive. A declared concurrency budget silently
multiplies across stitch objects (100 stitches x 8 = peak 100); pool:'host' repairs
it, and ADDING A STORE SILENTLY BREAKS THE REPAIR because the store-backed throttle
never reads pool — and the store is exactly what you add to make the RATE budget
cross-process. Plus: cache:{ttl:0} caches forever, and coalesced callers share one
object by reference (mutating row 0 changed row 5).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Scenario 17, run deliberately as a CONSOLIDATION test. Deprecation (RFC 9745) and
Sunset (RFC 8594) arrive on responses that SUCCEEDED, so nothing fails, nothing
retries and no status changes — every mechanism a client has for noticing trouble
points the wrong way.
The deliverable is a definitive accessor -> headers table, because three earlier
scenarios each rediscovered one row of it. Response headers ARE reachable on a
successful call, in exactly three places: the adapter, hooks.onResponse, and
Surface.interpret — the only seat where a header and the returned value are in
scope together. Eleven accessors carry none, including the ENTIRE EVENT SPINE
(4 events, 15 distinct keys, zero headers), which is why a TraceSink inherits the
same hole and can only aggregate what a Surface folded into the value.
A claimed correction that does NOT survive checking: the verification reported
that scenarios 6, 7 and 15 "generalised one step too far". Checked against what
actually shipped, they did not — the multipart page says "so WITHOUT A SURFACE the
ETag is unrecoverable", the unconfirmed-write page scopes it to StitchError, and
the 304 page's own solution uses Surface.execute. The drafts are scoped the same
way. Nothing is retracted; the table is a consolidation.
C1 where are headers -> exactly 3 places; 11 accessors carry none
C2 hooks.onResponse -> sees them, AND CAN REWRITE THE CALL
C3 a levelled finding-> one narrow door; carries neither value nor endpoint
C4 fleet aggregation -> works, unchanged at 200:1 traffic skew
C5 the tripwire -> deterministic to the ms; burns no retries, no breaker
C6 both formats -> no help reachable; Date.parse("@1735689600") is NaN
C7 noise -> 600 calls -> 360 lines / 3 facts
C8 assembled -> 132 lines vs 81 — first time the count goes against us
Escalated three things. hooks.onResponse can change what a stitch returns, and the
hooks guide says it cannot: ctx.res is the engine's live object, so mutating
res.status turned a vendor 200 into a thrown HTTP 503. A seam-level `kind` is a
COMPILE ERROR that works perfectly at runtime (TS2353, yet members inherited the
surface and folded correctly) — the opposite failure to scenario 13's
stream({kind}) silently DROPPING it, so the two are worth fixing together. And
parseRetryAfter is not merely similar to what a Sunset parser needs, it is
identical — and unexported, one turn worse than scenario 14 found it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The section shipped 18 pages and hand-written `meta.json` files but was never added to `content.manifest.ts`, so the IA drift guard (`test/content-manifest.spec.ts`) failed three ways at once: 18 undocumented pages, a `scenarios` folder owning a `meta.json` with no section entry, and a root `meta.json` listing a section the manifest did not know about. Adds the `sections` entry between Recipes and Concepts — the slot the committed root `meta.json` already expected — and all 18 pages, with `title` and `description` taken verbatim from each page's frontmatter so the manifest and the pages cannot disagree. `index` is `landing`, the 17 scenarios are `guide`. Page order matches the section's existing `meta.json` exactly, which is what `childrenOf` regenerates. Not `HAND_MAINTAINED_SECTIONS`: that hatch is for sections whose page set churns independently of this file (integrations, one page per shipped package). A curated 17-scenario set is not that, and routing it through the manifest is what puts each `description` into search and `llms.txt`. `pnpm gen:docs` is a no-op on the content — 0 stubs created, and no `meta.json` changed, confirming the committed files were already right and the manifest was the only gap. Docs suite goes from 3 failing to 59 passing. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
rejifald
force-pushed
the
claude/api-integration-scenarios-436a38
branch
from
August 5, 2026 13:06
7b17dad to
f5b6d54
Compare
Filed against rejifald/StitchAPI, self-contained (each carries its own minimal reproduction and source-line evidence, verified against main at filing time): #640 sse: { reconnect: true } replays a completed stream — a completed OpenAI-shaped stream reopened 4x, ABCDEABCDEABCDEABCDE delivered, done(ok: true). Two composing defects, engine.ts:1288 and :1466. [bug] #641 throttle and circuit have no tenancy axis — one tenant's revoked token opened the breaker for 9 of 9 healthy tenants, and it never self-heals. Fix is one option name on two interfaces, on an axis CacheOptions and OAuth2Options already carry. [enhancement] #642 idempotency: true is per call, not per intent — a re-driven job produced 8 charges for 6 intended payments, and adding `retry` (which the warning itself advises) silences the warning. [documentation, enhancement] One correction made while filing: the idempotency nudge is at stitch.ts:389 on current main, not :386 as the draft had it — main moved since the draft was written. Six of the other seven cited lines verified unchanged. Drafts and ledger updated so the filed three are not re-filed. Fourteen remain unfiled. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…d the paginate pair #643 Combinators broadcast one input to every member — a per-call `authorization` written for the primary arrived at the BACKUP verbatim. Root cause is runMember spreading one StitchInput (pipe.ts:75-86) with input headers merging OVER config headers (engine.ts:232), so each auth strategy only reclaims its own header name. Filed with the credential leak as the headline and any()'s hedge pricing as the related section, since both come off the same broadcast. [bug] #644 paginate ends the run ok: true when a page aggregates zero items, BEFORE asking next whether there was more (engine.ts:984 precedes :985). Four different endings share that break and one successful result. [bug] #645 Companion to #644: deduping in `items` — the textbook drift mitigation — empties a page and loses rows. The fix causes the damage. [bug] #644 and #645 cross-linked in both directions. All citations re-verified against current main before filing; engine.ts:984, engine.ts:232 and pipe.ts:75 all hold unchanged. Eleven drafts remain unfiled. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Scenario 18. The credential boundary held under the sharpest test available:
34 JSON-RPC exchanges, 30 payload scans, 14,529 bytes, five held credentials,
zero hits — including a vendor 401 with a credential-shaped body and the same
run over stdio, with controls confirming the wire carried them.
sanitizeAgentInput refuted our own starting hypothesis: six model-supplied
headers including authorization, cookie and host reached the vendor as zero.
The argument boundary is the operator's, and that is where the findings are.
A query parameter pinned in a configured path is only a default: the model
sent tenant=globex and the vendor returned the other tenant's data. Declaring
a schema does not close it, because validateInput discards the parsed value.
Two drafts: the unfiltered MCP error channel (a DNS failure under
apiKey({ in: 'query' }) put the key in the model's context, zero user code),
and input schemas that check but never filter — which is not MCP-specific.
8 claims, 181 checks, offline, re-run before writing.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Scenario 19. Resilience and streams test perfectly offline: attempt counts assertable three ways, the whole circuit closed/open/half-open/closed trace readable from callCount(), throttle spacing exact and self-reporting, and five repeat runs of a mid-stream failure producing one byte-identical outcome. The scenario's own direction is invisible offline. A vendor drifting while the fixture sits still measures as test ok=true with zero findings against production ok=false and five keys different. The comparison that catches it is 8 lines; what is missing is a place to put a call you may only make sometimes. Pattern 2b of this pass is now settled deliberately rather than by accident: manualClock drives 6 of 12 time-driven features. OAuth2 token expiry is a new sighting and is not in ADR 0010's documented list -- auth.ts has no clock plumbing at all. And the pass's own prediction got the timeout.total mechanism wrong: the budget resets each attempt, it is not ignored. Two bugs in the testing kit itself: stubStitch().safe() throws on a synchronous throw where the real stitch returns ok=false, and mockAdapter violates the library's own adapter abort rule on any route without a delay. 8 claims, 168 checks, offline, re-run before writing. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Corrects the ADR 0010 claim: it documents four of the six wall-clock features as deliberate, not three, and SigV4 is not among them. Two are undocumented -- SigV4 and OAuth2 token expiry. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Holds back two as security-sensitive: the MCP error channel credential leak, and the cross-principal cache leak in cache-cannot-revalidate section 2. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Scenario 20. JSON.parse turns 1234567890123456789 into
1234567890123456768 through the real default transport, and the spine is
four events with zero drift, zero error and zero info.
The capture's central prediction was wrong in a way that changes the
answer. I wrote that the Adapter is the only seam that can see bytes; it
is not. wire: { response: 'text' } is published config honoured at
http-adapter.ts:123, an else-if that returns before the JSON branch at
:135, so the repair is 16 lines on the stock transport. I reasoned from
AdapterResponse.body being pre-parsed and never checked whether config
could stop the parse happening at all.
Two more refutations: validation CAN help (a refine on
Number.isSafeInteger has provably zero false negatives), and trace sinks
survive BigInt because trace.ts ships a bigintSafe replacer on purpose.
Money also turns out to be a different bug -- 19.99 round-trips fine;
decimals fail in arithmetic, not in transport.
One draft: a bigint in params silently vanishes (util.ts:392 omits it
while stringifyLeaf includes it), so the same id is exact in query and
gone in params.
8 claims, 191 checks, offline, re-run before writing.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…ngrades `body-verdict-footguns` finding 2 measured `.safe()` handing back a bare StitchError with `body: undefined` when a delegate-backoff RateLimitError was the terminal, with the real instance reachable only via `.cause`. #662 fixes it, and not by either of the two asks (preserve the fields across `asStitchError`, or document that `delegate` needs try/catch): the coercion existed only because the two classes were siblings while `SafeResult.error` is typed `StitchError`. Making RateLimitError extend StitchError removes the need for it — `.safe()` returns the instance `await` throws. Rewrites the (b2) proof block to measure the new behaviour (verified 19/19 against #662's core) and marks the draft + LEDGER row accordingly. Findings 1 and 3 are untouched and still open. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Owner
Author
|
Finding 2 of the
Not pushed: this branch has 8 unpushed commits ahead of git -C .claude/worktrees/nifty-merkle-171edc push origin HEAD:claude/api-integration-scenarios-436a38Worth saying: the scenario pass found this from the consumer side at the same time reviewing #639 found it from the code side. Independent confirmation that the sibling-classes design was the actual defect, not the two-arm |
rejifald
added a commit
that referenced
this pull request
Aug 5, 2026
The two error classes were siblings, and CONTRACT.md P10 kept them in parity by having `RateLimitError` re-declare `status`/`attempts`/`body`/`url` by hand — a written rule enforcing exactly what `extends` gives for free. Root the taxonomy at `StitchError` instead. The duplication had a cost beyond tidiness. `SafeResult.error` is typed `StitchError`, so `.safe()` coerced a delegate-backoff `RateLimitError` into a bare one: the instance moved to `.cause`, the `instanceof` test stopped working, and `error.body` came back `undefined` — dropping the payload an outer gate reads to pace itself. The mode whose whole point is handing back-pressure outward lost its signal on the path the docs otherwise recommend. `.safe()` now returns the same instance `await` throws. (Independently found from the consumer side by the scenario pass in #638, `body-verdict-footguns.md` item 2.) Every dispatch site also carried a two-arm `instanceof StitchError` / `instanceof RateLimitError` check; #639 was adding a second copy of it inside one function. One test now covers both. The one hazard the subclassing introduces is arm ordering — a leading generic `StitchError` arm swallows the delegate signal. `engine.ts`'s `errEvt` already had the order right (a RateLimitError also carries `.response`); it is now commented as load-bearing, P10 requires it, and the docs and error catalog say so where they branch. Also fixes the test stub, which flattened a stubbed `RateLimitError` and never stamped `retryAfter` on the streamed `error` event the way the engine does. Breaking for consumers that branch on both classes. Pre-GA (1.0.0-rc.7). Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Scenario 21. The exposure is binary and therefore auditable: 13 destinations carry all seven PII sentinels, 11 carry zero, nothing is partially redacted. Exactly one event carries the body -- result, on data. An output allowlist is the answer and takes it to zero at every value-reading destination, at depth and inside array elements, without naming a single PII field. That works because of the asymmetry scenario 20 measured: the engine serves the validated value, so output filters where input merely checks (#648). Wrapped in drift() it also yields a value-free inventory of everything it stripped. Two refutations. My "credentials protected, PII not" split was too clean: the real line is credentials the library PLACES versus credentials that ride the payload -- a declarative bearer never enters the spine at all, while an access_token in a response body goes 3 of 3 into the JSONL. And the measurements contradict a shipped ADR. ADR 0018 section 4 says findings never leak a secret, justified by detailFor emitting kinds only. True of the three soft drift kinds; false of hard validation, where validationErrors copies the validator's message verbatim. The pre-registered suspicion from scenario 18 held exactly: sensitive: true gates the cache and nothing else -- one read, engine.ts:1022. The issue draft is HELD BACK as security-sensitive. 8 claims, 196 checks, offline, re-run before writing. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
| const ok = Object.is(actual, expected); | ||
| if (!ok) failures++; | ||
| console.log( | ||
| ` ${ok ? 'ok ' : 'FAIL'} ${label}: measured ${show(actual)}${ok ? '' : ` (expected ${show(expected)})`}`, |
| const ok = payload.includes(needle); | ||
| if (!ok) failures++; | ||
| console.log( | ||
| ` ${ok ? 'ok ' : 'FAIL'} ${where.padEnd(34)} -> ${ok ? 'DISCLOSES' : 'absent'} ${show(needle)}`, |
| const ok = JSON.stringify(actual) === JSON.stringify(expected); | ||
| if (!ok) failures++; | ||
| console.log( | ||
| ` ${ok ? 'ok ' : 'FAIL'} wire ${field.padEnd(29)} = ${show(actual)}${ok ? '' : ` (expected ${show(expected)})`}`, |
| const ok = Object.is(actual, expected); | ||
| if (!ok) failures++; | ||
| console.log( | ||
| ` ${ok ? 'ok ' : 'FAIL'} ${label}: measured ${String(actual)}${ok ? '' : ` (expected ${String(expected)})`}`, |
Scenario 22. Every shadow-traffic guide in the field assumes you own the gateway and both services. As the consumer of a third-party API you own neither endpoint, the shadow spends the vendor's meter, and you cannot shadow a write at all. Both pre-registered predictions confirmed. #643's one-input broadcast is exactly wrong for a dual-run: the shadow's URL was /v2/customers with no id. And #641 confirmed in 2 of 5 configs -- but refuted as I stated it. Sharing is not the default; it is a key collision on (store) x ('circuit:' + (circuit.key ?? name ?? path ?? 'stitch')). The two configs that break are both ordinary dual-run shapes, because v1 and v2 usually share a path and differ by base URL, and sit on one host. I predicted four isolation channels; there are five, and the one I missed is the worst: all() aborts the in-flight PRIMARY when the shadow settles first. Only the retry budget is safe by default. Also refuted: a thunk moves the whole path rather than just the origin, seams pool a throttle correctly by default, and two value-vs-value comparators do ship -- diff and classifyDiff -- they are simply not exported from any of the 17 subpaths. 8 claims, 142 checks, offline, re-run before writing. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…ifest `content-manifest.spec.ts` was failing two ways: three scenario pages existed as `.mdx` with no manifest entry, and `scenarios/meta.json` did not match what the manifest would generate. Two more pages have landed since, so five in total — agent-holds-the-tool, stale-fixture, precision-loss, pii-in-the-logs and dual-run-migration. `pnpm gen:docs` alone was the wrong fix: it regenerates `meta.json` FROM the manifest, so on its own it deleted all five from the nav rather than adding them — green test, five pages unreachable. The manifest is the source of truth, so the entries go there and the generator agrees for free (meta.json is byte-identical afterwards; title/description mirror each page's frontmatter). apps/docs: 60/60 green, prettier clean. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Owner
Author
|
Manifest fixed and pushed ( Worth flagging, because So the entries went into
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Adds the Scenarios section under
docs/scenarios— 17 self-contained write-ups, each taking one real integration problem from symptom to working shape, plus theapps/docswiring to render and navigate the section.The scenarios
Shape
docs/scenarios, 20 underapps/docs(section wiring / navigation).+56,895with no deletions outside the section's own iterations.Note for review
This branch is currently behind
main, and deliberately not rebased — it lives in a separate worktree that may be in active use, so rewriting its history wasn't something to do unprompted. Hit Update branch (or ask me to rebase) before merging. The PR diff is computed from the merge-base, so what's shown here is this branch's own work only.🤖 Generated with Claude Code