Skip to content

Add end-of-turn batching layer for same-channel messages (#91)#95

Open
strix-tkellogg wants to merge 1 commit into
mainfrom
strix/91-batched-processing
Open

Add end-of-turn batching layer for same-channel messages (#91)#95
strix-tkellogg wants to merge 1 commit into
mainfrom
strix/91-batched-processing

Conversation

@strix-tkellogg
Copy link
Copy Markdown
Collaborator

Summary

  • End-of-turn drain: after _process_event runs for a discord_message, pop every queued same-channel discord_message and process them all in a single follow-up turn. A burst of N messages now costs 2 turns instead of N.
  • Newest drained event is the synthetic trigger; _render_prompt already pulls full recent-channel history, so every batched message lands in context.
  • Queue accounting preserved — the drain helper mutates the backing deque directly and leaves _unfinished_tasks untouched; caller balances with task_done per drained event.
  • Other-channel events and non-discord triggers are never absorbed (scheduler ticks, web messages, etc. stay in queue).

Telemetry

  • New batched_turn_start event: channel_id, trigger_event_type, batch_size.
  • turn_timing now carries a batched boolean — distinguishes the triggering turn from the follow-up batch leg so you can jq the events log for turns where batching actually saved work.

Closes #91. Part 1 (turn-time instrumentation) shipped in #94 so this can be evaluated against real numbers.

Test plan

  • tests/test_batched_processing.py::test_same_channel_messages_drain_into_one_batched_turn — 3 queued (2 same-channel + 1 other) → exactly 2 agent invocations, batched_turn_start{batch_size: 2}, one turn_timing{batched: true}, other-channel event survives.
  • tests/test_batched_processing.py::test_no_batched_turn_when_queue_has_no_same_channel_messages — unrelated queued event does not trigger batching.
  • tests/test_batched_processing.py::test_non_discord_trigger_does_not_batch — scheduler tick does not absorb a matching discord_message.
  • tests/test_batched_processing.py::test_drain_helper_preserves_order_of_kept_events — order-preserving filter across mixed-channel queue.
  • Full suite green via pre-commit: 246 passed, 1 skipped (was 242, added 4).

🤖 Generated with Claude Code

Part 2 of #91. After responding to a Discord message, drain any
same-channel discord_message events queued during the turn and process
them in a single follow-up turn so a rapid thread costs 2 turns instead
of N.

- `_drain_same_channel_discord_events` filters the queue's backing deque
  directly so `_unfinished_tasks` accounting stays intact; caller is
  responsible for `task_done` per drained event.
- New `batched_turn_start` event records channel_id and batch_size.
- `turn_timing` now carries a `batched` field distinguishing the
  triggering turn from the follow-up batch leg.
- Other-channel and non-discord events stay untouched in the queue.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Channel-only batched message processing + turn-time instrumentation

2 participants