Skip to content

[Story 4.2b] Core parallel dispatch in presentAssistantMessage #375

@edelauna

Description

@edelauna

Part of #359 (Epic 4: Parallel Tool Execution).

Depends on: Story 4.2a (DispatchState), #362 (Story 1.2 — TaskSemaphore), #363 (Story 1.3 — experiment flag).

Fixes #190 (missing tool_result blocks — root cause is the missing Promise.allSettled guarantee this story adds).
Relates to #131 (parallel read-only tools — our plan's parallel dispatch for read-only tools directly enables this; execute_command stays sequential per the plan).

Context

With DispatchState in place (Story 4.2a), the parallel dispatch path can be added safely. When PARALLEL_TOOL_EXECUTION is enabled and the stream is complete, tools are dispatched via Promise.allSettled rather than the sequential switch. Promise.allSettled (not Promise.all) is critical — a missing tool_result for any tool_use block causes provider API errors on the next call (all major providers require a result for every tool call), so sibling results must always be collected even when one tool rejects.

This story covers core dispatch logic: tool partitioning, parallel execution, rejection strategy, and DispatchState interlock. File-level write serialization and observability are in Story 4.2c.

Developer Notes

In src/core/assistant-message/presentAssistantMessage.ts:

  • Add import pLimit from 'p-limit' — dependency already at ^6.2.0 in src/package.json.
  • Add presentAssistantMessageParallel(cline: Task): Promise<void>:
    • Gate: if !cline.assistantMessageSavedToHistory or cline.dispatchState !== IDLE → set pendingWork = { mode: 'parallel' } and return.
    • Set dispatchState = PARALLEL.
    • Wait for didCompleteReadingStream = true.
    • Partition all tool_use blocks into three categories:
      • Sequential (always serial even with flag on): new_task, attempt_completion, ask_followup_question, execute_command, switch_mode. execute_command is kept sequential as a deliberate safety margin — same stance as OpenAI Codex (shell_command defaults supports_parallel_tool_calls() = false).
      • MCP tools: Default to sequential — unknown side effects. Future opt-in via server metadata.
      • Parallelizable: read_file, list_files, search_files, list_code_definition_names, write_to_file, apply_diff, insert_code_block. Write tools gain per-file serialization in Story 4.2c; treat as fully parallelizable here (safe since this story ships behind the flag and 4.2c lands before flag promotion to default-on).
      • Default: unclassified tools are sequential. Add a compile-time or test-time exhaustiveness check: every tool type in the switch must be in exactly one classification bucket.
    • Dispatch parallelizable tools with Promise.allSettled via pLimit(PARALLEL_TOOL_CONCURRENCY). Define PARALLEL_TOOL_CONCURRENCY = 8.
    • Dispatch sequential tools one at a time after the parallel batch.
    • Rejection handling (two-tier): When askApproval returns false → cancel all not-yet-dispatched tools (they receive tool_result with is_error: true and cancellation message). Already-executing tools finish normally.
    • pLimit cancellation caveat: pLimit has no API for cancelling queued work. Check a shared cancelled flag inside each wrapped function before executing tool logic.
    • All supported providers require a tool_result for every tool_use. Denied tools produce { is_error: true, content: "Tool use was denied by user." }. Cancelled tools produce { is_error: true, content: "Tool execution cancelled — a sibling tool was denied." }.
    • In finally: reset dispatchState = IDLE, drain pendingWork.

In src/core/task/Task.ts:

  • After didCompleteReadingStream = true, check experiments.isEnabled(stateExperiments, "parallelToolExecution") and dispatch to presentAssistantMessageParallel or presentAssistantMessage accordingly.

Files: src/core/assistant-message/presentAssistantMessage.ts, src/core/task/Task.ts

Tests (src/core/assistant-message/__tests__/presentAssistantMessageParallel.spec.ts):

Dispatch behavior:

  • Controlled promises for each tool — resolve in arbitrary order; assert all results appear in userMessageContent.
  • Result ordering: tool_result entries match tool_use block ordering (required by all providers).
  • new_task in same response → dispatched sequentially after parallel batch.
  • MCP tool in same response → dispatched sequentially.
  • Flag off → presentAssistantMessageParallel never called.

Rejection strategy:

  • Tool A rejected, Tool B already executing → B finishes; A has denial result, B has normal result.
  • Tool A rejected → Tool C (not yet started) gets cancellation result, never dispatched.
  • All tool_use blocks have a corresponding tool_result regardless of outcome.
  • pLimit(8) with 10 tools — 8 dispatched, 2 queued. Rejection cancels the 2 queued tools. Assert all 10 have results.

Tool classification:

  • Exhaustiveness check: Every tool type in the switch is present in exactly one bucket. Adding a new tool without updating classification must fail this test.

All mocked; completes in < 100ms.

Acceptance Criteria

  • Flag off: zero observable change — all existing presentAssistantMessage tests pass.
  • Flag on: multiple read_file / list_files calls in one response execute concurrently.
  • pushToolResultToUserContent's duplicate guard (lines 371–384) prevents double-writes.
  • No duplicate dispatch possible via the dirty-flag path during parallel execution.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions