regression(orchestrator): Blocked → Rework Kanban move remains scheduler-blocked during degraded status refresh

[michaelhvisser]

schema: 1
effort: xhigh

Problem

On Detent v0.48.0 (87d1438), an operator can successfully move cards from Blocked to Rework through the integrated Kanban, while the scheduler continues skipping those same Rework candidates as internally blocked.

This makes the board look correctly requeued but leaves the Rework slots idle until a later successful reconciliation or a process restart clears the stale runtime state.

Live evidence

Observed on 2026-07-22 in the Parable project:

1. Four cards were moved through POST /api/v1/kanban/move:
   • getparable/parable#1351
   • getparable/parable#1516
   • getparable/parable#1524
   • getparable/parable#1634
2. Every mutation returned HTTP 200: {"message":"Moved card to Rework.","ok":true}.
3. The live board immediately showed:
   • Blocked: 14 → 10
   • Rework: 0 → 4
4. At 2026-07-22T11:57:36Z, scheduler decisions 114028–114031 saw all four candidates in lane Rework, with project_state_available: 2 and project_state_used: 0, but skipped every one with:
   • result=skipped
   • reason=blocked
   • wait_reason=blocked
5. /api/v1/state still reported the old blocked count while the Kanban showed the cards in Rework.
6. The same refresh window had degraded GitHub hydration/status reads from the REST fan-out cap. After restarting Detent, the stale runtime state disappeared and fix(board): hide scheduled recovery pacing #1351/#1516 dispatched immediately into the two Rework slots.

Likely seam

The integrated Kanban mutation updates the tracker and web mutation overlay, but does not synchronously invalidate the matching orchestrator state.Blocked entry.

In internal/orchestrator/tick.go, a failed/degraded blocked-status transition refresh can merge previous.blockedStatusIssues back into the current set before dispatch. clearBlockedStatusIssue in internal/orchestrator/reconcile.go only clears through reconciliation, so a fresh candidate that is already visibly Rework can still be vetoed by stale project-status memory.

Related closed issue: #1378.

Expected behavior

A confirmed operator move away from Blocked must make that card dispatchable on the next scheduler decision, even if the broader observed-status refresh is temporarily degraded.

Acceptance criteria

• A successful integrated Kanban move from Blocked to an active state invalidates the matching project-status blocked entry without waiting for a full tracker refresh.
• Retry, claim, breaker, and blocked memory are reconciled only for the moved item; unrelated blockers are preserved.
• A regression test moves Blocked → Rework, injects a failed/degraded blocked-status refresh, and proves the candidate dispatches when lane capacity exists.
• Board, /api/v1/state, and scheduler-decision blocked counts converge without a process restart.
• Audit/telemetry records the operator move as the reason the runtime block was cleared.

[michaelhvisser]

@corylanou — filed from live Detent v0.48.0 evidence on macOS/launchd. The reproduction, related closed issues, and acceptance criteria are captured above.

[corylanou]

Codex Workpad

Plan

• Reproduce the Blocked → Rework stale runtime-block regression under degraded status refresh.
• Trace the integrated Kanban mutation boundary and item-scoped orchestrator reconciliation paths.
• Invalidate only the moved item's stale runtime memories after a confirmed move, with operator-move audit telemetry.
• Add regression coverage for dispatch, unrelated blocker preservation, and state/count convergence.
• Run focused tests and make check, then open a ready PR and verify current-head CI/reviews.

Acceptance criteria

• Successful Blocked → configured non-Blocked Kanban moves synchronously clear the matching project-status runtime block.
• Retry, claim, breaker, and blocked memory are reconciled only for the moved item.
• Degraded blocked-status refresh cannot restore the moved item's stale veto before the next dispatch.
• Board, API state, and scheduler blocked counts converge without restart.
• Audit/telemetry attributes the runtime unblock to the operator move.
• Focused tests and make check pass.

Implementation

• Added a synchronous event-loop reconciliation request after a confirmed integrated Kanban move away from Blocked.
• Updated the cached issue state and cleared only project-status block, claim, retry, repeated/instant failure, and matching breaker memory for the moved item.
• Preserved unrelated blockers and non-project-status blocks.
• Added operator_kanban_move_reconciled and runtime_block_cleared telemetry with reason operator_kanban_move.
• Reconciled configured active, observed, and terminal targets; unknown targets remain ignored.

Validation

• Focused orchestrator regressions passed, including Blocked → Rework dispatch during a degraded blocked-status transition refresh.
• go test ./internal/orchestrator ./internal/web ./internal/cli -count=1 passed.
• Post-review-fix make check passed: generation checks, build, golangci-lint, vet, nilaway, race tests, and coverage floors.
• Coverage: 77.5% repository; 84.9% orchestrator.
• PR fix(orchestrator): reconcile operator moves #1486 is open, ready, mergeable, and contains Fixes #1482 at head 885929ada014e42d56469638797fa8ad68dd02a9.
• The P2 configured-inactive-target review finding was fixed and its only review thread is resolved.
• Current-head CI is 11/11 green. The first GoReleaser attempt encountered an unrelated synthetic 429 from a connector test mock; the exact test passed 20 local repetitions and the isolated job rerun passed.

Gate telemetry

• Quiet-window wait: 0s.
• Local merge-gate duration: N/A; this is the pre-review gate, not the serialized Merging stage.
• PR CI duration: 15m08s from first check start through successful isolated rerun.
• Slow checks: Portability Verify (windows-latest) 7m41s; Browser Visual 6m57s; successful GoReleaser Snapshot rerun 6m36s.
• Post-merge main CI: not started; PR remains unmerged.

Blockers

None.

schema: 1
status: complete
blockers: []
human_action: null

[corylanou]

Auto-promoted this issue from In Progress to Merging.

• reason: ready
• pull request: fix(orchestrator): reconcile operator moves #1486
• mergeable_state: behind
• workpad_comment: regression(orchestrator): Blocked → Rework Kanban move remains scheduler-blocked during degraded status refresh #1482 (comment)