Skip to content

regression(shutdown): draining /health hangs and forced SIGTERM is reported as github_auth #1484

Description

@michaelhvisser
schema: 1
effort: xhigh

Problem

During an active-session shutdown on Detent v0.48.0 (87d1438), the drain banner appears correctly, but the HTTP listener stops answering /health. A second SIGTERM then reaches the force path, yet the process exits with a misleading github_auth error caused by canceled-context token resolution instead of the primary forced-shutdown result.

The initial wait itself was expected because two sessions were active. The regressions are loss of draining observability and shutdown-result misclassification.

Live evidence

Observed on 2026-07-22:

  1. First SIGTERM printed:
    shutdown requested; draining sessions; press Ctrl+C again to force quit immediately
  2. Two Codex sessions were still active, so drain mode was legitimate.
  3. The Detent process continued listening on 127.0.0.1:4000, but GET /health timed out after 3 seconds with no response.
  4. A second SIGTERM printed:
    force quit requested; interrupting sessions
  5. Instead of a forced-shutdown result, the final structured error was:
    • code: github_auth
    • detail: resolve github_token via gh auth token: context deadline exceeded
    • exit_code: 2

Related closed issues #383 and #1227 explicitly require /health and /api/v1/state to report draining and require deterministic second-signal semantics.

Expected behavior

Drain mode remains observable, and cancellation noise from connector/auth work cannot replace the shutdown state machine’s primary outcome.

Acceptance criteria

  • While draining, /health responds promptly with a draining status and active-session count.
  • /api/v1/state remains readable enough to show drain progress until teardown begins.
  • A second SIGINT/SIGTERM deterministically produces the forced-shutdown sentinel/result.
  • Context-canceled connector, token, metrics, and store work during teardown is logged as secondary cleanup noise and cannot be classified as a new auth failure.
  • Tests cover first signal during a slow connector refresh, health requests during drain, and second-signal error precedence.
  • The forced path terminates owned worker process groups and exits within the documented hard deadline.

Metadata

Metadata

Assignees

No one assigned

    Labels

    detent:doneDetent state: complete.

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions