fix: align custom tool SSE lifecycle with OpenAI API compatibility

[maralbahari]

Summary

This PR fixes custom tool support across the gateway while preserving OpenAI’s public Responses API contract.

What was wrong

vLLM accepts a type: "custom" declaration, but exposes the resulting call as a regular function_call. Its streaming response emits:

• response.function_call_arguments.delta
• response.function_call_arguments.done

It does not emit OpenAI-compatible custom-tool events:

• response.custom_tool_call_input.delta
• response.custom_tool_call_input.done

Forwarding vLLM’s response directly therefore leaked the internal normalized function shape instead of returning a public custom_tool_call.

What changed

The gateway now:

• Normalizes custom tools into model-facing function tools with a required string input parameter.
• Keeps custom tools client-owned; the gateway does not execute them.
• Hides vLLM’s internal normalized function-call events.
• Restores blocking output as custom_tool_call.
• Emits the OpenAI-compatible streaming lifecycle:
  • response.output_item.added
  • response.custom_tool_call_input.delta
  • response.custom_tool_call_input.done
  • response.output_item.done
• Uses one stable ctc_ item ID throughout the lifecycle.
• Converts custom call/output history into the normalized function representation only when sending continuation context upstream.

Testing

Added OpenAI and gateway cassettes for streaming and non-streaming two-turn custom-tool flows. Tests verify:

• Gateway output matches OpenAI’s public custom-tool contract.
• Raw custom input is preserved.
• Internal function_call items do not leak.
• Streaming lifecycle IDs and sequence numbers remain consistent.
• custom_tool_call_output works correctly on the continuation turn.

[franciscojavierarceo]

left a few inline comments.

[franciscojavierarceo]

once a custom call appears, the rest of the upstream stream gets buffered until inference finishes. this turns the input delta into one full-input event, delays later message and reasoning events, and leaves us with an unbounded buffer. we should emit normalized custom deltas incrementally and bound any remaining deferred storage.

[maralbahari]

@franciscojavierarceo Fixed by adding a function SSE translator that runs immediately after ResponseAccumulator updates the authoritative in-flight function call. Normalized custom-tool argument deltas are now converted and emitted incrementally as custom_tool_call_input.delta, without delaying later stream events. Only unnamed calls are temporarily buffered, with a 256 KiB limit. The re-recorded OpenAI and gateway cassettes both emit the same six-delta lifecycle, and the integration test now compares the exact delta sequence, lifecycle ordering, stable item ID, and completed output.

[maralbahari]

@franciscojavierarceo
Function-call classification and wire conversion were moved out of executor/upstream.rs into a dedicated FunctionSseTranslator. After ResponseAccumulator updates the authoritative internal function call, the translator uses a request-scoped name → ToolType map to pass through normal functions, incrementally translate custom-tool events, and suppress gateway-owned MCP/web-search frames until execution emits their public lifecycle. This replaces the previous hide, buffer, flush, and function-call classification helpers scattered throughout executor/upstream.rs; only initially unnamed calls are buffered, with a 256 KiB limit.

these changes made the PR pushed out of the scope but it's cleaner and helps in up coming tool type support.

[ashwing]

Focused on the accumulator/normalizer seam since the client-contract and public-API surface is already well covered. Two things there — a coverage gap on the done-only path and the input-extraction heuristic.

[maralbahari]

@franciscojavierarceo @ashwing
Thanks for the review. it's ready to review again.
The changes after addressing the comments are as following:

• Custom-tool grammar handling: Preserved custom format constraints in the normalized function description so the model sees the original grammar or regex requirements instead of receiving an unconstrained string tool.

• Structured tool outputs: Added typed handling for text and content-array outputs, preserving OpenAI image/file content semantics and rejecting unsupported output shapes instead of stringifying them.

• Public Rust API compatibility: Restored UpstreamTool while retaining function-only upstream serialization, since custom tools are normalized to functions before being sent upstream.

• OpenAI tool-choice contract: Restored support for auto, none, required, forced function/custom choices, and typed allowed_tools entries.

• Streaming response metadata: Added a custom-tool name map, following the namespace restoration design, so response.created and response.in_progress expose the original custom tools and tool choices rather than internal normalized functions.

• Authoritative done events: Function calls now apply the complete output_item.done payload, preventing initially unnamed calls from leaking incomplete IDs, names, or arguments.

• Malformed custom envelopes: Added debug logging when normalized custom arguments cannot be unwrapped, making any raw-envelope fallback observable.

• Done-only streaming regression: Added coverage for custom calls that receive output_item.done without a preceding output_item.added, including correct ordering after reasoning items.

• Incremental custom-tool SSE: Added a generic function SSE translator that uses the accumulator’s authoritative in-flight state and converts normalized function deltas into incremental custom_tool_call_input.delta events without delaying subsequent stream events.

• Bounded buffering: Only unnamed function calls are temporarily buffered, with a 256 KiB limit; ordinary functions pass through and gateway-owned MCP/web-search calls retain their execution-owned lifecycle.

• OpenAI cassette verification: Re-recorded two-turn streaming and non-streaming cassettes for both OpenAI and the gateway. The streaming test now compares the exact six-delta sequence, lifecycle ordering, stable public item ID, completed item, and custom_tool_call_output continuation.

[maralbahari]

Function-call SSE translation refactor

• Detecting normalized function calls that represent gateway-owned tools.
• Hiding their internal function_call events.
• Buffering calls whose names were unavailable in output_item.added.
• Flushing ordinary function events once their ownership was known.
• Deferring events after a gateway-owned call to preserve output ordering.
• Synthesizing the public custom-tool lifecycle after inference completed.

This logic was spread across emit_upstream_stream_event, defer_after_gateway_call, record_first_hidden_gateway_output_index, defer_or_flush_function_event, should_hide_upstream_event, flush_pending_function_events, and drop_pending_function_events in executor/upstream.rs.

The refactor moves function-call classification and public wire translation into a dedicated FunctionSseTranslator. The translator receives only a request-scoped name → ToolType map, rather than the full ToolRegistry, and handles normalized function calls according to their declared public type:

• Ordinary functions and namespace members pass through unchanged.
• Custom tools are translated incrementally from function_call_arguments.* into custom_tool_call_input.*.
• Gateway-owned MCP and web-search function frames are suppressed until their execution-specific public lifecycle is emitted.
• Calls without an initial name are buffered only until their type can be resolved, with a 256 KiB safety limit.

ResponseAccumulator still processes the original upstream frame first. This keeps the normalized FunctionToolCall as the authoritative internal representation used for dispatch and model continuation. The translator then reads that accumulated call state to produce client-facing frames, avoiding a second argument accumulator or changes to internal output items.