Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions .changeset/svelte-callback-propagation.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
---
'@tanstack/ai-svelte': patch
---

fix(ai-svelte): propagate `createChat` callback changes uniformly

`onResponse`, `onChunk`, and `onCustomEvent` were passed as direct references to the underlying `ChatClient`, while `onFinish` and `onError` were wrapped to read from `options.onX?.(...)` at call time. This meant callers who mutated the options object in-place (or invoked `client.updateOptions(...)`) would see their replacement propagate for the latter two but silently miss for the former three. All five user-supplied callbacks now go through the same indirection, matching the React / Preact / Vue / Solid sibling wrappers.
69 changes: 67 additions & 2 deletions docs/chat/streaming.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ import { chat } from "@tanstack/ai";
import { openaiText } from "@tanstack/ai-openai";

const stream = chat({
adapter: openaiText("gpt-5.2"),
adapter: openaiText("gpt-4o"),
messages,
});

Expand All @@ -46,7 +46,7 @@ export async function POST(request: Request) {
const { messages } = await request.json();

const stream = chat({
adapter: openaiText("gpt-5.2"),
adapter: openaiText("gpt-4o"),
messages,
});

Expand Down Expand Up @@ -87,6 +87,71 @@ TanStack AI implements the [AG-UI Protocol](https://docs.ag-ui.com/introduction)

> **Tip:** Some models expose their internal reasoning as thinking content that streams before the response. See [Thinking & Reasoning](./thinking-content).

### Type-Safe Tool Call Events

When you pass typed tools (defined with `toolDefinition()` and Zod schemas) to `chat()`, the stream chunks automatically carry type information for tool call events. The `toolName` field narrows to the union of your tool name literals, and the `input` field on `TOOL_CALL_END` events is typed as the union of your tool input schemas:

```typescript
import { chat, toolDefinition } from "@tanstack/ai";
import { openaiText } from "@tanstack/ai-openai";
import { z } from "zod";

const weatherTool = toolDefinition({
name: "get_weather",
description: "Get weather for a location",
inputSchema: z.object({
location: z.string(),
unit: z.enum(["celsius", "fahrenheit"]).optional(),
}),
});

const stream = chat({
adapter: openaiText("gpt-4o"),
messages,
tools: [weatherTool],
});

for await (const chunk of stream) {
if (chunk.type === "TOOL_CALL_END") {
chunk.toolName; // ✅ typed as "get_weather" (not string)
chunk.input; // ✅ typed as { location: string; unit?: "celsius" | "fahrenheit" } | undefined
}
}
```

Without typed tools, `toolName` defaults to `string` and `input` defaults to `unknown` — the same behavior as before. The type narrowing is automatic when you use `toolDefinition()` with Zod schemas.

When multiple tools are provided, tool call events form a **discriminated union** — checking `toolName` narrows `input` to that specific tool's type:

```typescript
const searchTool = toolDefinition({
name: "search",
description: "Search the web",
inputSchema: z.object({ query: z.string() }),
});

const stream = chat({
adapter: openaiText("gpt-4o"),
messages,
tools: [weatherTool, searchTool],
});

for await (const chunk of stream) {
if (chunk.type === "TOOL_CALL_END") {
if (chunk.toolName === "get_weather") {
// ✅ input is narrowed to { location: string; unit?: "celsius" | "fahrenheit" }
console.log(`Weather in ${chunk.input?.location}`);
}
if (chunk.toolName === "search") {
// ✅ input is narrowed to { query: string }
console.log(`Searched for: ${chunk.input?.query}`);
}
}
}
```

> **Tip:** The typed stream chunk type is exported as `TypedStreamChunk<TTools>` if you need to annotate variables or function parameters. When used without type arguments, `TypedStreamChunk` is equivalent to `StreamChunk`.

### Thinking Chunks

Thinking/reasoning is represented by AG-UI events `STEP_STARTED` and `STEP_FINISHED`. They stream separately from the final response text:
Expand Down
4 changes: 3 additions & 1 deletion docs/reference/type-aliases/StreamChunk.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,9 @@ title: StreamChunk
type StreamChunk = AGUIEvent;
```

Defined in: [packages/typescript/ai/src/types.ts:1364](https://github.com/TanStack/ai/blob/main/packages/typescript/ai/src/types.ts#L1364)
Defined in: [packages/typescript/ai/src/types.ts](https://github.com/TanStack/ai/blob/main/packages/typescript/ai/src/types.ts)

Chunk returned by the SDK during streaming chat completions.
Uses the AG-UI protocol event format.

For the tool-aware variant that narrows `TOOL_CALL_START`/`TOOL_CALL_END` events by tool name and `CUSTOM` events by tagged literal name, see [`TypedStreamChunk`](./TypedStreamChunk).
32 changes: 32 additions & 0 deletions docs/reference/type-aliases/TaggedCustomEvent.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
---
id: TaggedCustomEvent
title: TaggedCustomEvent
---

# Type Alias: TaggedCustomEvent\<T\>

```ts
type TaggedCustomEvent<T = unknown> =
| StructuredOutputStartEvent
| StructuredOutputCompleteEvent<T>
| ApprovalRequestedEvent
| ToolInputAvailableEvent;
```

Defined in: [packages/typescript/ai/src/types.ts](https://github.com/TanStack/ai/blob/main/packages/typescript/ai/src/types.ts)

Discriminated union of the orchestrator-tagged `CUSTOM` events. Each variant has a literal `name`, so a single narrow on `chunk.name` yields a typed `value` with no helper or cast:

```ts
if (chunk.type === 'CUSTOM' && chunk.name === 'approval-requested') {
chunk.value.toolCallId // typed as string
}
```

The `StructuredOutputCompleteEvent` value is parameterized by `T`, which the chat orchestrator narrows to the schema's inferred type after Standard Schema validation. Adapters always emit it with `T = unknown`.

`TaggedCustomEvent` is included in [`TypedStreamChunk`](./TypedStreamChunk)'s typed-tools branch so consumers iterating `chat()` streams get tagged narrowing alongside the per-tool `TOOL_CALL_START`/`TOOL_CALL_END` typing.

## Caveat: user-emitted custom events

Tools can emit arbitrary user-defined custom events via the `emitCustomEvent(name, value)` context API. Those flow through the stream at runtime but are intentionally absent from this union — including a bare `CustomEvent` (whose `value: any` would poison the union) would collapse `chunk.value` back to `any` after the narrow. If you rely on `emitCustomEvent`, branch on `CUSTOM` outside the literal-`name` narrows or cast the chunk to [`StreamChunk`](./StreamChunk) to recover the wider shape.
67 changes: 67 additions & 0 deletions docs/reference/type-aliases/TypedStreamChunk.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
---
id: TypedStreamChunk
title: TypedStreamChunk
---

# Type Alias: TypedStreamChunk\<TTools\>

```ts
type TypedStreamChunk<
TTools extends ReadonlyArray<Tool<any, any, any>> = ReadonlyArray<Tool<any, any, any>>,
> =
HasTypedTools<TTools> extends true
?
| Exclude<
StreamChunk,
| { type: 'TOOL_CALL_START' }
| { type: 'TOOL_CALL_END' }
| { type: 'CUSTOM' }
>
| DistributedToolCallStart<TTools>
| DistributedToolCallEnd<TTools>
| TaggedCustomEvent
: StreamChunk;
```

Defined in: [packages/typescript/ai/src/types.ts](https://github.com/TanStack/ai/blob/main/packages/typescript/ai/src/types.ts)

A variant of [`StreamChunk`](./StreamChunk) parameterized by the tools array. When specific tool types are provided (e.g. from `chat({ tools: [myTool] })`):

- `TOOL_CALL_START` and `TOOL_CALL_END` events form a **discriminated union** over tool names.
- Checking `toolName === 'x'` narrows `input` to that specific tool's input type.
- `TOOL_CALL_END` events have `input` typed per-tool via Standard Schema inference.
- `CUSTOM` events with literal tagged names (`structured-output.start`, `structured-output.complete`, `approval-requested`, `tool-input-available`) narrow `value` to the corresponding payload via the [`TaggedCustomEvent`](./TaggedCustomEvent) union.

When tools are untyped or absent, `TypedStreamChunk` falls back to plain `StreamChunk` so existing consumers that pass streams as `AsyncIterable<StreamChunk>` keep working.

This is the type returned by `chat()` when streaming is enabled (the default). You don't typically need to reference it directly unless annotating function parameters or return types.

```ts
import { chat, toolDefinition, type TypedStreamChunk } from "@tanstack/ai";
import { openaiText } from "@tanstack/ai-openai";
import { z } from "zod";

const weatherTool = toolDefinition({
name: "get_weather",
description: "Get weather for a location",
inputSchema: z.object({ location: z.string() }),
});

const searchTool = toolDefinition({
name: "search",
description: "Search the web",
inputSchema: z.object({ query: z.string() }),
});

// Inferred from `chat()` — typed tool call events plus tagged CUSTOM events
const stream = chat({
adapter: openaiText("gpt-4o"),
messages,
tools: [weatherTool, searchTool],
});

// Explicit annotation
type Chunk = TypedStreamChunk<[typeof weatherTool, typeof searchTool]>;
```

See [Streaming - Type-Safe Tool Call Events](../../chat/streaming) for a practical walkthrough.
2 changes: 2 additions & 0 deletions docs/tools/server-tools.md
Original file line number Diff line number Diff line change
Expand Up @@ -307,6 +307,8 @@ const getUserData = getUserDataDef.server(async (args) => {

> **Note:** JSON Schema tools skip runtime validation. Zod schemas are recommended for full type safety and validation.

> **Tip:** When you pass typed tools (server, client, or definition) to `chat()`, the returned stream is fully typed — `toolName` narrows to your tool name literals and `input` narrows per-tool when you check the name. See [Type-Safe Tool Call Events](../chat/streaming#type-safe-tool-call-events).

## Best Practices

1. **Keep tools focused** - Each tool should do one thing well
Expand Down
2 changes: 2 additions & 0 deletions docs/tools/tools.md
Original file line number Diff line number Diff line change
Expand Up @@ -91,6 +91,8 @@ const inputSchema: JSONSchema = {

> **Note:** When using JSON Schema, TypeScript will infer `any` for input/output types since JSON Schema cannot provide compile-time type information. Zod schemas are recommended for full type safety.

> **Tip:** Type safety from Zod schemas extends beyond tool execution — when you iterate over the stream returned by `chat()`, tool call events have typed `toolName` and `input` fields too. See [Type-Safe Tool Call Events](../chat/streaming#type-safe-tool-call-events).

## Tool Definition

Tools are defined using `toolDefinition()` from `@tanstack/ai`:
Expand Down
25 changes: 16 additions & 9 deletions packages/typescript/ai-svelte/src/create-chat.svelte.ts
Original file line number Diff line number Diff line change
Expand Up @@ -77,11 +77,18 @@ export function createChat<
type Final = InferSchemaType<NonNullable<TSchema>>

// Create ChatClient instance.
// Note: Svelte's createChat runs once per instance and `options` is captured
// by reference. Callbacks are therefore frozen to whatever the caller passed
// at creation — to swap them dynamically, mutate the options object
// in-place or call `client.updateOptions(...)` imperatively.
// Optional fields use conditional spread because the target
//
// Svelte's `createChat` runs once per instance, so `options` is captured by
// reference at creation time. Wrapping each user-supplied callback through
// `options.onX?.(...)` lets callers mutate the options object in place (or
// call `client.updateOptions(...)` imperatively) and have the next invocation
// pick up the new function — without this indirection, those five callbacks
// would be frozen to whatever was passed at `createChat(...)` time, which
// diverges from the React/Preact/Vue/Solid sibling wrappers. This is the
// same uniform treatment applied to `onFinish`/`onError`; the other three
// (`onResponse`, `onChunk`, `onCustomEvent`) used to be direct references.
//
// Non-callback optional fields use conditional spread because the target
// `ChatClientOptions` declares them as `field?: T` (absent vs. present)
// rather than `field?: T | undefined`. Under `exactOptionalPropertyTypes`,
// passing an explicit `undefined` for an absent-only optional is a type
Expand All @@ -96,7 +103,7 @@ export function createChat<
...(options.forwardedProps !== undefined && {
forwardedProps: options.forwardedProps,
}),
...(options.onResponse !== undefined && { onResponse: options.onResponse }),
onResponse: (response) => options.onResponse?.(response),
onChunk: (chunk: StreamChunk) => {
options.onChunk?.(chunk)
},
Expand All @@ -107,9 +114,9 @@ export function createChat<
options.onError?.(err)
},
tools: options.tools,
...(options.onCustomEvent !== undefined && {
onCustomEvent: options.onCustomEvent,
}),
onCustomEvent: (eventType, data, context) => {
options.onCustomEvent?.(eventType, data, context)
},
...(options.streamProcessor !== undefined && {
streamProcessor: options.streamProcessor,
}),
Expand Down
Loading
Loading