Skip to content

feat: Anthropic eager input streaming for Code API tools#184

Open
danny-avila wants to merge 2 commits into
mainfrom
danny-avila/anthropic-codeapi-eager-input-streaming
Open

feat: Anthropic eager input streaming for Code API tools#184
danny-avila wants to merge 2 commits into
mainfrom
danny-avila/anthropic-codeapi-eager-input-streaming

Conversation

@danny-avila

Copy link
Copy Markdown
Owner

Summary

Adds Anthropic's current per-tool eager_input_streaming: true flag to the final Anthropic tool definition for CodeAPI/event-driven execution tools:

  • execute_code
  • bash_tool
  • run_tools_with_code
  • run_tools_with_bash
  • read_file

The flag is applied only in the Anthropic adapter after LangChain formats tools for Anthropic, because LangChain's generic tool extras schema does not currently pass this field through. Explicit eager_input_streaming: false is preserved, and non-CodeAPI tools are left unchanged.

Research Notes

Anthropic's current docs describe fine-grained tool input streaming as a per-tool eager_input_streaming setting rather than relying solely on the old beta header. Live testing showed input_json_delta events still occur without the field on the tested model, but the field changed chunk/timing behavior and gave earlier tool-input deltas in samples.

Live Validation

Model: claude-haiku-4-5-20251001

Raw Anthropic stream comparison:

Case First input delta Delta count Result
no eager field 1407ms 225 streamed input_json_delta
old beta header only 874ms 83 streamed input_json_delta
per-tool eager_input_streaming 452ms 50 streamed input_json_delta, earliest in sample

Run/eager event pipeline comparison with mocked host execution:

Case First ON_TOOL_EXECUTE First model end Eager before model end
field stripped 2264ms 2290ms yes
new field 1868ms 2031ms yes

Validity check at the streamed-tool seal point:

Case bash first delta read_file first delta / bash seal Eager bash args parseable Eager args == final args
field stripped 838ms 1049ms yes yes
eager_input_streaming: true 764ms 964ms yes yes

Tests

  • ANTHROPIC_API_KEY=test NODE_OPTIONS='--experimental-vm-modules' npx jest src/llm/anthropic/llm.spec.ts --runInBand -t 'Anthropic (CodeAPI tool|fine-grained tool input streaming)'
  • ANTHROPIC_API_KEY=test NODE_OPTIONS='--experimental-vm-modules' npx jest src/__tests__/stream.eagerEventExecution.test.ts --runInBand -t 'CodeAPI|eager|stale|redispatch|changed'
  • ANTHROPIC_API_KEY=test npx tsc -p tsconfig.json --noEmit
  • npx eslint src/llm/anthropic/index.ts
  • npx prettier --check src/llm/anthropic/index.ts src/llm/anthropic/llm.spec.ts
  • git diff --check

@danny-avila danny-avila changed the title [codex] Enable Anthropic eager input streaming for CodeAPI tools feat: Anthropic eager input streaming for Code API tools May 19, 2026
@danny-avila danny-avila marked this pull request as ready for review May 19, 2026 14:17

Copy link
Copy Markdown
Owner Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 5e89578523

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/llm/anthropic/index.ts
Comment thread src/llm/anthropic/index.ts

Copy link
Copy Markdown
Owner Author

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. What shall we delve into next?

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant