Skip to content

Make Executor ReAct loop append-only for prompt-prefix caching#187

Draft
yzx9 wants to merge 1 commit into
mainfrom
executor-append-only
Draft

Make Executor ReAct loop append-only for prompt-prefix caching#187
yzx9 wants to merge 1 commit into
mainfrom
executor-append-only

Conversation

@yzx9

@yzx9 yzx9 commented Jun 30, 2026

Copy link
Copy Markdown
Collaborator

Rewrite Executor.run to seed [system, user(task)] once and append one assistant turn (via the shared reconstruct_react_text) plus one user observation turn per step, instead of rebuilding a single growing user message each iteration. The static system prompt and all prior turns now form a byte-stable prefix, so provider-side prefix caching hits on every iteration. Previously only the system block cached; the whole rebuilt user message (Previous-Steps JSON + Iteration counter + last response) was a cache miss every step, with hit rate declining as the loop grew.

Folded correctness fixes (surfaced in plan-eng-review + /review):

  • _generate_final_summary: render the string action_summary directly instead of calling .get('name') on it (AttributeError on exhaustion).
  • tool_retry_counter: reset at run() start. Executors are reused across delegated tasks, so the counter leaked and could poison later runs.
  • max_syntax_errors=3 budget with reset-on-success: abort a model stuck emitting malformed ReAct instead of spamming up to 15 correction turns. The budget resets after any cleanly-parsed turn, so only 3 CONSECUTIVE bad turns abort (not 3 slips scattered across a long run).

Extract reconstruct_react_text into copilotj/multiagent/react_format.py (LeaderAgent and Executor both import it; leader_multiagent imports Executor, so it could not live there without a circular import). Document in _build_enhanced_system_prompt that the tool text is load-bearing: the ReAct wrapper strips the tools= API param before the provider call, so the system-prompt text is the model's only view of the tools.

Add copilotj/test/multiagent/test_executor.py (9 cases incl. a post-format prefix-stability cache-contract test that runs messages through the Anthropic formatter rather than asserting raw list-append). Full suite 206 passed; ruff clean.

Assisted-by: Claude-Code:GLM-5.2

@yzx9 yzx9 changed the base branch from main to dev June 30, 2026 06:15
Rewrite Executor.run to seed [system, user(task)] once and append one
assistant turn (via the shared reconstruct_react_text) plus one user
observation turn per step, instead of rebuilding a single growing user
message each iteration. The static system prompt and all prior turns now
form a byte-stable prefix, so provider-side prefix caching hits on every
iteration. Previously only the system block cached; the whole rebuilt user
message (Previous-Steps JSON + Iteration counter + last response) was a
cache miss every step, with hit rate declining as the loop grew.

Folded correctness fixes (surfaced in plan-eng-review + /review):
- _generate_final_summary: render the string action_summary directly
  instead of calling .get('name') on it (AttributeError on exhaustion).
- tool_retry_counter: reset at run() start. Executors are reused across
  delegated tasks, so the counter leaked and could poison later runs.
- max_syntax_errors=3 budget with reset-on-success: abort a model stuck
  emitting malformed ReAct instead of spamming up to 15 correction turns.
  The budget resets after any cleanly-parsed turn, so only 3 CONSECUTIVE
  bad turns abort (not 3 slips scattered across a long run).

Extract reconstruct_react_text into copilotj/multiagent/react_format.py
(LeaderAgent and Executor both import it; leader_multiagent imports
Executor, so it could not live there without a circular import). Document
in _build_enhanced_system_prompt that the tool text is load-bearing: the
ReAct wrapper strips the tools= API param before the provider call, so the
system-prompt text is the model's only view of the tools.

Document pre-existing dead branch in _suggest_tool_based_on_context and
deferred correction-path cache trade-off (codex P2 #2)

Add copilotj/test/multiagent/test_executor.py (9 cases incl. a post-format
prefix-stability cache-contract test that runs messages through the
Anthropic formatter rather than asserting raw list-append). Full suite
206 passed; ruff clean.
@yzx9 yzx9 force-pushed the executor-append-only branch from 45cf031 to a32d7a3 Compare June 30, 2026 13:36

@00JackLu 00JackLu left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The code looks good, but I haven’t tested it.

@yzx9 yzx9 changed the base branch from dev to main July 3, 2026 02:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants