Preserve and replay complete reasoning items across Responses continuations

[franciscojavierarceo]

Summary

Preserve complete Responses API reasoning items losslessly across previous_response_id and conversation continuations for both streaming and non-streaming responses.

The persistence unit must remain the complete typed reasoning item—id, every content entry, every summary entry, encrypted_content, and status. Do not add a reasoning-text table, column, or database migration.

Motivation

OpenAI reports materially better ARC-AGI-3 results when reasoning is retained across tool calls and turns, and its API guidance says applications should keep each reasoning item exactly as returned and send it back on the next turn:

• How enabling two settings tripled our ARC-AGI-3 scores
• Use reasoning.encrypted_content

For self-hosted vLLM, usable continuation state is normally plaintext content[].text; opaque encrypted reasoning cannot be decrypted or interpreted by the gateway.

Current behavior

The core typed storage and rehydration design is already suitable:

• ReasoningOutput carries the complete item.
• InOutItem stores typed input/output items as JSON.
• rehydrate_conversation restores stored items in history order.
• OutputItem::to_input_item retains reasoning items.

The primary defect is the streaming accumulator:

• reasoning summary events share the reasoning-text buffer;
• response.output_item.done.item is ignored for reasoning items;
• complete summary, encrypted_content, status, and final content can therefore be lost;
• a valid completed reasoning item is dropped when output_item.added is absent.

Required behavior

• Preserve the complete reasoning item as wire-equivalent structured data.
• Treat a successfully parsed reasoning item in response.output_item.done as authoritative.
• Reconcile by item ID without duplicates and retain original output_index ordering.
• Let explicitly supplied completed fields win, including empty arrays and null; use accumulated deltas only when a field is omitted.
• Keep reasoning_text events in content and reasoning_summary_text events in summary.
• Retain valid delta-derived state if a completed item is malformed.
• Persist reasoning beside messages and tool calls in original order, then replay it before the next turn's new input.
• Never expose the private _agentic_item_kind marker upstream.
• Preserve opaque encrypted_content in storage. For the vLLM-bound enriched-request copy, replay plaintext content without forwarding incompatible non-null encrypted state. Reject encrypted-only reasoning with a clear invalid-request error before any vLLM network call.
• Do not log reasoning text, encrypted content, or raw request/response bodies. Byte counts, item types, response IDs, and status remain safe.

Proposed work breakdown

Phase 1 / PR 1: lossless streaming accumulation

1. Split normalized reasoning-text and reasoning-summary payloads.
2. Give in-flight reasoning independent content and summary buffers.
3. Parse and reconcile reasoning from output_item.done.item, using raw key presence to distinguish omitted fields from explicit empty/null fields.
4. Track output_index when finalizing completed items so missing output_item.added events cannot disturb ordering.
5. Add focused accumulator/normalizer tests for:
   • text deltas followed by a complete item;
   • summary deltas remaining separate;
   • completed encrypted_content and status;
   • completed fields overriding incomplete delta state;
   • delta fallback for omitted content;
   • completed reasoning without output_item.added;
   • no duplicate when both delta and completed paths exist;
   • malformed completion retaining valid accumulated reasoning.
6. Add a full storage serialization/deserialization round-trip test for the complete typed reasoning item.

This phase is independently reviewable: streamed and non-streamed first-turn outputs become equivalent and persistence receives the same typed item.

Phase 2 / PR 2: continuation replay and vLLM compatibility

1. Validate/sanitize only the enriched vLLM request copy; do not mutate the stored reasoning item or the turn's persistence inputs.
2. Replay plaintext reasoning content.
3. Return a clear InvalidRequest before inference for encrypted-only reasoning; never use a summary as substitute continuation state.
4. Add streamed and non-streamed two-turn integration tests that inspect the recorded second upstream request.
5. Cover ordering across reasoning, assistant message, function call, function call output, and new user input.
6. Assert _agentic_item_kind is absent from upstream JSON.
7. Remove existing uncommitted raw-body/SSE trace statements from inference.rs while retaining safe byte-count diagnostics.

This phase depends on Phase 1 so the streamed first turn persists the same complete item as the non-streamed path.

Related issues

• Persist reliable Responses continuation checkpoints #71 tracks broader durable continuation/checkpoint correctness; this issue is the typed reasoning-item prerequisite and does not add checkpoint metadata.
• [SSE Completion - Issue 2] Prior-turn reasoning leaks <think> into the visible answer #120 covers reasoning leakage across gateway inference rounds. Its final fix should align with the eventual reasoning.context policy rather than globally dropping persisted reasoning items.
• GPT-oss streaming: message not captured when output_item.added is missing #62 is the completed precedent for recovering missing streaming items when providers omit output_item.added.

Acceptance criteria

• Streaming and non-streaming paths produce equivalent complete reasoning items.
• Storage contains the complete typed reasoning item, not collapsed text.
• Chained requests replay plaintext reasoning to vLLM in original item-history order.
• Summary and encrypted state survive persistence even when vLLM does not use them.
• Encrypted-only reasoning fails clearly before a vLLM call.
• No reasoning is duplicated, downgraded to a summary, silently dropped, or logged.
• No schema migration or separate reasoning-text field is introduced.
• All focused tests pass, followed by:

cargo test -p agentic-server-core
cargo clippy -p agentic-server-core --all-targets -- -D warnings
cargo fmt -- --check

Out of scope

• compaction or automatic context summarization;
• encryption-at-rest redesign;
• decrypting opaque reasoning;
• vLLM changes;
• full reasoning.context support.

A follow-up for reasoning.context should define current_turn as retaining reasoning across inference rounds in the active tool loop while excluding earlier completed turns, and all_turns as including historical reasoning.