Skip to content

Fix stalled orphan session queue adoption #327

Description

@JRedeker

Problem

After OpenCode restart, prior-session ADV workflows can remain routed to orphan per-session Temporal queues. Their query handlers become unreachable even though broad worker liveness reports healthy.

Observed impact

  • adv_change_show pinBuildkitImage and adv_change_show verifyStagingBuildkit both fail ServiceError: Failed to query Workflow.
  • adv_change_archive pinBuildkitImage dryRun:true cannot evaluate release obligations.
  • adv_doctor reports server_alive, worker_alive, queue serviceability, and search attributes healthy, but adoption remains stalled.
  • Orphan-queue adoption scan shows 17 prior session queues, scanInFlight:true, all attemptCount:0; queue count does not advance.
  • Query RPC evidence includes 1 CANCELLED: context canceled.

Root cause hypothesis

The current worker polls its own session queue while workflows from earlier sessions remain on their original queues. The adoption scanner should register those queues for polling but is stuck before any adoption attempt. Restarting creates another session queue and does not repair old query routing.

Acceptance criteria

  • A live worker detects and adopts eligible prior-session queues containing running workflows.
  • Adoption progress is observable: attempt count advances or typed refusal/error explains why it cannot.
  • Querying a workflow on an adopted prior-session queue succeeds after restart.
  • Failed adoption is bounded, retryable, and fail-closed; no workflow task-queue field mutation.
  • adv_doctor surfaces query-path/adoption health rather than reporting fully healthy when adoption is stalled.
  • Regression coverage includes multiple prior session queues and query recovery after restart.

Safety

No blind restarts, direct Temporal workflow rewrites, state-file edits, or archive/gate bypasses. Operator recovery must use a typed, auditable path.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions