Skip to content

feat(agent-gui): replace processing row with per-turn progress bar#1406

Open
devRickyyy wants to merge 1 commit into
mainfrom
feat/play-next-move-progress-bar
Open

feat(agent-gui): replace processing row with per-turn progress bar#1406
devRickyyy wants to merge 1 commit into
mainfrom
feat/play-next-move-progress-bar

Conversation

@devRickyyy

@devRickyyy devRickyyy commented Jul 19, 2026

Copy link
Copy Markdown
Contributor

Summary

Replaces the "Planning next moves" processing row in the AgentGUI transcript with a live per-turn progress bar that follows the active turn:

  • Model phase + per-phase timer: shows "Waiting for response" vs "Responding" with an elapsed timer anchored to real timestamps — response-start for the streaming phase, last message/tool completion for the awaiting phase (so the timer is not inflated by long tool executions). The whole row hides while a tool call runs, matching the previous suppression semantics.
  • Per-turn token counters: cumulative ↑ input / ↓ output tokens, shown only for providers with the tokenUsage capability (Claude Code, Codex/tutti-agent). Providers without an input/output split (ACP family) show phase + timer only.

Phase and timing are derived in the renderer from canonical turn state and in-flight message timestamps — no new wire phases. Token counters are accumulated daemon-side per turn and flushed through turn transitions at ~1 Hz into a new durable WorkspaceAgentTurn.tokenUsage field:

  • Claude Code: sidecar now also parses message_start usage (per-API-call input); daemon folds per-call cumulative output deltas into turn totals.
  • Codex: per-turn baseline diff of thread/tokenUsage/updated thread totals (keyed by threadId; reasoning output counted as output; stale/negative diffs clamped).
  • Transport follows the existing fileChanges precedent: turn.updated metadata → TurnTransition → new workspace_agent_turns.token_usage_json column (migration included) → OpenAPI/WS turn payload. Final flush always lands before the settle transition (settled turns reject later transitions).

Contract changes: WorkspaceAgentTurnTokenUsage schema in tuttid.v1.yaml, matching optional tokenUsage on the turn_update WS payload, regenerated clients, and the mirrored types in agent-activity-core.

Documentation: docs/architecture/agent-gui-node.md §3.2/§4.6 now record the tokenUsage ownership and the renderer-side progress-row derivation rule.

Verification

  • cd services/tuttid && go test ./... && go build ./...; same for packages/agent/daemon and packages/agent/store-sqlite — all green, incl. new counter, repository round-trip, migration idempotency, and projection-guard tests
  • golangci-lint run on the touched daemon packages — 0 issues
  • pnpm --filter @tutti-os/claude-sdk-sidecar test — 87/87
  • pnpm --filter @tutti-os/agent-gui test — 227 files / 1999 tests (new projection, component, and transcript cases)
  • pnpm --filter @tutti-os/agent-activity-core test — 208 pass; desktop bridge/adapter/activity-service suites green (39/39 on the reconcile-failure fixture)
  • pnpm check:api-generated, pnpm check:event-protocol-generated, pnpm check:i18n, pnpm lint:ts, pnpm typecheck (27/27), electron/renderer/ui boundary checks, agent-gui degradation check
  • Pre-push pnpm check:changed -- --push-ready — 32/32 lanes green

Fallback Three Questions

  • Source: token/usage signals arrive per message delta (Claude) or per app-server notification (Codex) — far above the rate the UI or the durable store should absorb.
  • Why can it not be fixed at that source: the providers own the emission cadence; a bounded (~1 Hz per turn) flush in the daemon is the transport-level choke point that already exists for fileChanges.
  • Removal condition: the throttle constant can go away if turn metrics ever move to a dedicated high-frequency event channel instead of riding turn transitions.

Checklist

  • I kept the change focused on one concern.
  • I updated documentation when behavior, setup, or contributor workflow changed.
  • I updated all README/CONTRIBUTING language variants when changing their English source.
  • I ran the lowest meaningful local checks for the changed surface.
  • My commits are signed off with DCO when required: git commit -s.

Open in Devin Review

Note

Medium Risk
Touches turn lifecycle ordering (token flush before settle), multi-provider token accounting, and live transcript projection; mistakes could show wrong timers/tokens or drop final usage on settled turns.

Overview
Replaces the generic "Planning next moves" transcript row with a per-turn progress bar on the active turn: Waiting for response vs Responding, a per-phase elapsed timer from canonical turn phase and message/tool timestamps (not provider wire phases), and ↑/↓ token counts when the session has the tokenUsage capability. The row is suppressed for non-live turn phases and when another turn-specific progress row (e.g. compact) is already shown; it still hides during in-flight tool work via existing presentation rules.

Daemon / persistence: Claude sidecar emits usage_updated on root message_start; Claude and Codex adapters accumulate per-turn input/output (Codex diffs thread totals with baseline/replay handling), throttle mid-turn flushes to ~1 Hz, and always flush before settle. Counters persist on WorkspaceAgentTurn.tokenUsage (token_usage_json) through the same turn.updated path as fileChanges, with types wired through activity-core, desktop adapter, and reconcile bridge coercion.

Supporting fixes: timeline merge prefers latest completedAtUnixMs for messages/tools; completion times flow through detail projection for accurate awaiting-phase anchors; Claude lifecycle logging moved to claude_sdk_diagnostics.go; interaction upsert/list helpers consolidated in store-sqlite.

Reviewed by Cursor Bugbot for commit a6bf856. Bugbot is set up for automated code reviews on this repo. Configure here.

@cursor

cursor Bot commented Jul 19, 2026

Copy link
Copy Markdown

Bugbot couldn't run - usage limit reached

Bugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit.

A user or team admin can review and increase usage limits in the Cursor dashboard.

(requestId: serverGenReqId_59ab4761-a27f-4d82-b205-47e3f0889074)

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no potential bugs to report.

View in Devin Review to see 3 additional findings.

Open in Devin Review

The "Planning next moves" row is now a live progress bar that follows
the active turn: model phase (waiting for response vs responding) with
a per-phase elapsed timer anchored to message/tool completion times,
plus cumulative per-turn token counters for providers with the
tokenUsage capability (Claude Code, Codex).

Phase and timing are derived in the renderer from canonical turn and
message timestamps. Token counters are accumulated daemon-side per
turn (Claude: message_start input plus delta output; Codex: thread
total baseline diff) and flushed through turn transitions into the
durable WorkspaceAgentTurn.tokenUsage field (new workspace_agent_turns
token_usage_json column).

Signed-off-by: devRickyyy <74520253+devRickyyy@users.noreply.github.com>
@jomeswang
jomeswang force-pushed the feat/play-next-move-progress-bar branch from a6bf856 to dedbd04 Compare July 19, 2026 04:15

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 1 new potential issue.

View 3 additional findings in Devin Review.

Open in Devin Review

event.Type != activityshared.EventRootProviderTurnCompleted {
patch.Turn = &agentsessionstore.WorkspaceAgentTurnPatch{
TurnID: turnID,
CapabilityRefs: activityCapabilityReferencesFromEvent(event),

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 Per-turn token counts never get saved, so the new usage display stays empty

The per-turn token counts are dropped from each turn update (statePatchFromSessionEvent at packages/agent/daemon/runtime/reporter_state.go:76-80) before the turn is saved, so the input/output token numbers never reach the transcript.
Impact: The new ↑/↓ token counters on the active-turn progress bar are always blank, and no turn ever persists its token usage, making the entire per-turn token feature inert.

Why the structured field is never populated

The daemon computes token counters and emits them as turn.updated events whose Payload.Metadata["tokenUsage"] map carries inputTokens/outputTokens (see turnTokenUsageEvent at packages/agent/daemon/runtime/turn_token_usage.go:27-33). Every downstream stage reads a structured field, not the metadata: cloneWorkspaceAgentStatePatch copies patch.Turn.TokenUsage (packages/agent/daemon/activity/compat.go:410), turnTransitionFromStateInput copies turn.TokenUsage (services/tuttid/service/agent/activity_projection_turns.go:120), and the store/OpenAPI/WS payloads read that transition field.

The only producer of patch.Turn is statePatchFromSessionEvent. Its if patch.Turn != nil block (packages/agent/daemon/runtime/reporter_state.go:76-80) extracts metadata["fileChanges"] into patch.Turn.FileChanges but has no equivalent line copying metadata["tokenUsage"] into patch.Turn.TokenUsage. Neither applyLifecycleSnapshotToPatch nor applyExplicitTurnLifecycleToPatch sets it either. As a result patch.Turn.TokenUsage is always nil, and every downstream copy is nil.

This also means the new test TestStatePatchFromSessionEventMapsTokenUsage (packages/agent/daemon/runtime/turn_token_usage_test.go:333), which asserts patch.Turn.TokenUsage.InputTokens == 1200, cannot pass against the current file.

Fix: mirror the fileChanges precedent, e.g. after line 79 add extraction of metadata["tokenUsage"] (inputTokens/outputTokens via payloadInt64, tolerating float64) into a &agentsessionstore.WorkspaceAgentTurnTokenUsage{...}.

Prompt for agents
statePatchFromSessionEvent in packages/agent/daemon/runtime/reporter_state.go is the sole producer of patch.Turn (a *agentsessionstore.WorkspaceAgentTurnPatch). The per-turn token feature emits turn.updated events with Payload.Metadata["tokenUsage"] = { inputTokens, outputTokens } (see turnTokenUsageEvent in turn_token_usage.go). Every downstream stage (compat.go cloneWorkspaceAgentStatePatch, tuttid turnTransitionFromStateInput, the store, and the OpenAPI/WS payload) consumes the structured patch.Turn.TokenUsage field, NOT the metadata. But the `if patch.Turn != nil` block (around lines 76-80) only copies metadata["fileChanges"] into patch.Turn.FileChanges; it never copies metadata["tokenUsage"] into patch.Turn.TokenUsage. Consequently the token counters are computed and emitted but never persisted or transported, and the test TestStatePatchFromSessionEventMapsTokenUsage would fail. Add a metadata extraction alongside fileChanges that reads inputTokens/outputTokens (JSON metadata decodes numbers as float64, so use payloadInt64 which already handles float64/json.Number) and assigns patch.Turn.TokenUsage = &agentsessionstore.WorkspaceAgentTurnTokenUsage{InputTokens: ..., OutputTokens: ...} when the tokenUsage map is present.
Open in Devin Review

Was this helpful? React with 👍 or 👎 to provide feedback.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant