A truncated completion is a success on all eight observers, and `llmSurface` is `{ id: 'llm' }` so there is nothing to wrap

[rejifald]

Scenario: truncated-completion
Proofs: docs/scenarios/proofs/truncated-completion/ (8 scripts, 396 checks, real node:http impersonating both providers)

No disclosure content — correctness, API-surface and cost-observability findings about
stitchapi/llm. No real provider is contacted; every token count is the fixture's stated
convention and no number here is a pricing claim.

First, what works — and the reason this is a cheap fix

• The signal is already lifted. LlmResult (llm.ts:51-57) normalises text, model,
  usage and finishReason across providers, from stop_reason (:308) and
  finish_reason (:355-356). That is more than most clients expose.
• usage is reachable at every success vantage point — LlmResult.usage, .report().data,
  .inspect().raw, the result event, and a TraceSink twice (normalised and in the wire
  spelling inside raw).
• The compile guards explain themselves. NoRequestShapeOnLlm rejects method and wire.body
  with the reason in the diagnostic text; kind and a misspelled maxTokens are also TS2322.
• A body-aware retry arm exists and is legible: interpret returning { retry: true } produces
  a progress event whose detail is interpret: truncated (max_tokens) (engine.ts:808-813),
  distinguishable from a status retry's status 200.

1. Nothing acts on finishReason

makeLlmSurface's interpret (llm.ts:145-149) is verdictOf(res, cfg) and then
{ ok: true, data: provider.parse(res.body) }. So a 200 whose stop_reason is max_tokens is a
success. Measured across eight observers, all eight reach the reason and none flags it:

observer	what it says
await / .unwrap()	resolves
.safe()	ok: true, error: null
.report()	error: null, status: 200, attempts: 1, findings: []
.inspect()	0 findings
.stream()	start, progress, result, done; done.ok: true; 0 drift, 0 error
TraceSink	the literal "max_tokens" twice, with no level attached

Ask: because the field is already normalised, a one-line default — treat
max_tokens/length as a surface failure, or emit an info event naming it — is much cheaper here
than in the other places this shape has appeared. At minimum, a drift-level finding would make it
findable.

2. The token default is provider-specific, and nobody wrote it

llm.ts:283 is max_tokens: req.tokens ?? 1024 (Anthropic requires the field); llm.ts:333 is
max_tokens: req.tokens (OpenAI does not). Measured off the vendor's ledger with tokens unset and
the same 1500-token answer:

anthropic → {"model":"…","max_tokens":1024,…}  → stop_reason max_tokens, 1024 of 1500
openai    → {"model":"…","messages":[…]}        → finish_reason stop,     1500 of 1500

The max_tokens key is absent from the OpenAI body, not undefined. So the identical call
truncates on one provider and completes on the other, losing 476 tokens to a default the caller
never wrote.

Ask: document the asymmetry on LlmRequest.tokens, or emit an info event when the default is
applied. It is defensible — the API requires the field — and it is currently invisible.

3. llmSurface is { id: 'llm' }, so the correct fix cannot be a wrapper

llm.ts:117 is export const llmSurface: Surface = { id: 'llm' }; — one key, with buildRequest
and interpret both undefined, because the live surface is built per stitch by makeLlmSurface
(llm.ts:130), which is not exported.

So a caller who wants to add a truncation check to interpret cannot spread the shipped surface.
Every guard in this proof directory re-implements makeLlmSurface from the exported LlmProvider
and goes through stitch({ kind }) — which gives back up the NoRequestShapeOnLlm compile guards
(method: 'GET' compiles again, documented at types.ts:789-794), the maxTokens excess-property
check, and llm.bind(seam).

Ask: export makeLlmSurface (or accept an interpret override on LlmOptions). The same
scenario on the graphql surface was a one-line fix precisely because graphqlSurface carries a
real interpret that can be spread.

4. stream and sse compile and are silently ignored

LlmOptions = Partial<Omit<StitchConfig, 'kind'>> (llm.ts:155) omits only kind, so
stream: 'lines', stream: { decode: 'ndjson' }, sse: true and sse: { reconnect: true } all
type-check — verified out of band with tsc (5.9.3) against the package's own strict settings,
which produced diagnostics for method/wire.body/kind/maxTokens and none for stream or
sse.

At runtime all four are ignored: start, progress, result, done, 0 delta events, one buffered
result, one request. The module header already says token streaming is a follow-up
(llm.ts:9-11) — the gap is that the config accepts the spelling.

Ask: add stream/sse to the NoRequestShapeOnLlm family with the same self-describing
ConfigError text the other four already get. The machinery exists.

5. A surface rejection cannot carry the cost it just read

SurfaceOutcome's failure arm is { message, status? } — a string. So interpret, having parsed
usage one line earlier, can only hand-format the number into the message
("truncated after 12 output tokens (7 in)") while the vendor has already billed it.

Per-tenant attribution via hooks.onResponse costs 24 lines and sees the wire body, so an
Anthropic-spelled meter reads input_tokens correctly and 0 off an OpenAI body
(prompt_tokens/completion_tokens). The TraceSink alternative reads the normalised usage but
attributes only by ctx.name — a stitch per tenant — and its start event carries the whole prompt
into the log.

Ask: an optional structured slot on the failure arm (the same ask as the graphql draft, from a
different direction).

6. The status the provider tells you to retry is the one that is not retried

Default retry.on is [429, 502, 503, 504] (engine.ts:635). Measured: context-length 400 → 1
request (correct), rate limit 429 → 3 (correct), overloaded 529 → 1 (wrong), content filter 200
→ 1 (correct, and unreachable by any status rule). One predicate separates the three status arms
exactly; none can separate the two 200 arms.

Also worth a line somewhere: the error code is reachable via StitchError.body at a different
path per provider — error.type (Anthropic) versus error.code (OpenAI) — while the message is
only ever HTTP 400/429/529.

Ask: consider 529 in the default set, or name it in the retry guide as a provider-specific
addition.

7. Smaller, all measured

• A truncated tool call has no LlmResult slot: text: '' on both providers, arguments only via
  raw. On OpenAI the failure is a SyntaxError raised in caller code; on Anthropic nothing
  throws — input arrives as a valid object with 2 of 4 fields.
• A transform that parses is never retried (it runs at engine.ts:1220-1225, outside the
  attempt loop) and the consumers disagree: await re-throws the raw SyntaxError, .safe()
  flattens it.
• finishReason is lifted but not normalised: 0 of 5 arms share a spelling
  (max_tokens/end_turn/tool_use/refusal versus length/stop/tool_calls/content_filter).
• A parse handed the wrong provider's body returns quietly: text: '',
  finishReason: undefined, and a truthy but empty usage: {} — if (b.usage) is true and
  compact drops both undefined members.
• The system prompt relocates differently: identical caller code sends 1 message on
  Anthropic (folded into a top-level system, joined with \n\n) and 3 on OpenAI.

Reproduction

npx tsx docs/scenarios/proofs/truncated-completion/c1-nothing-says-cut-off.ts

Eight scripts run offline against one node:http server that serves both providers' wire shapes by
path; the stitches reach it by overriding url, so the shipped mappings run unmodified. The ledger
records the exact request body, so every max_tokens and token count is counted rather than
inferred.

Source references (verified against origin/main)

• llm.ts:51-57 — export interface LlmResult { text; model?; usage?; finishReason?; raw }
• llm.ts:117 — export const llmSurface: Surface = { id: 'llm' };
• llm.ts:130 — function makeLlmSurface(d: LlmDefaults): Surface<StitchInput, LlmResult> { (not exported)
• llm.ts:145-149 — interpret: (res, cfg) => { const failure = verdictOf(res, cfg); if (failure) return failure; return { ok: true, data: provider.parse(res.body) }; }
• llm.ts:155 — export type LlmOptions = Partial<Omit<StitchConfig, 'kind'>> & {
• llm.ts:283 — max_tokens: req.tokens ?? 1024, · llm.ts:333 — max_tokens: req.tokens,
• llm.ts:308 — if (b.stop_reason) result.finishReason = b.stop_reason;
• llm.ts:355-356 — const fr = b.choices?.[0]?.finish_reason; / if (fr) result.finishReason = fr;
• llm.ts:9-11 — the module header: token streaming is a follow-up; sse/stream do not compose onto llm yet
• engine.ts:635 — const retryMatch = acceptsStatus(cfg.retry?.on ?? [429, 502, 503, 504]);
• engine.ts:808-813 — the interpret retry arm and its interpret: ${outcome.message} detail
• engine.ts:1220-1225 — if (cfg.transform) value = await cfg.transform(value); (outside the attempt loop)
• types.ts:789-794 — stitch({ kind: llmSurface, method: 'PUT' }) keeps its PUT
• types.ts:820-825 — NoRequestShapeOnLlm, guarding method and wire.body only

---

Found by a scenario pass that researches a real-world API integration problem, captures it, and proves or refutes each claim with runnable offline code. Every source reference above was read on origin/main.