Write structured crash reason file on fatal service exit

[geoffjay]

Summary

Each service should persist a structured crash reason to a well-known file on fatal exit, enabling diagnostic tools to quickly surface why a service is down without parsing raw logs.

Problem

When a service crashes (e.g., due to a migration mismatch), the only record is in the stderr log. Diagnostic tools and other services have no structured way to determine the cause of a crash. This is especially problematic when:

• Log files are rotated or truncated
• The crash happens during startup before logging is fully initialized
• Multiple restart attempts fill the log with repeated errors

Proposed Solution

• On fatal exit, each service writes a JSON file to a standard location, e.g.:
  ~/Library/Application Support/agentd-<service>/last-error.json

• Format:

  {
    "timestamp": "2026-04-17T19:15:49Z",
    "error": "Migration file of version 'm20260329_000012_create_projects_table' is missing",
    "category": "migration",
    "exit_code": 1
  }

• The file is overwritten on each crash, so it always reflects the most recent failure.

• On successful startup, the file is deleted (or a last-healthy.json is written alongside it).

Acceptance Criteria

• All services write last-error.json on fatal exit
• File is written to a standard, discoverable location
• File is cleared or replaced on successful startup
• MCP diagnostic tools read this file when checking service health

[geoffjay]

✅ Triage Complete

Field	Value
Type	enhancement
Complexity	medium
Milestone	v1.0.0 — Production Readiness (existing assignment retained)
Labels	enhancement, complexity:medium

Notes:

• No duplicates found. This issue is the producer in a three-issue cluster:
  • Standardize service log file locations #1155 — standardize log/data file locations (defines the path convention this issue uses)
  • Write structured crash reason file on fatal service exit #1154 (this issue) — write last-error.json on fatal service exit
  • Surface crash logs in MCP diagnostic tools for unreachable services #1153 — surface crash files in MCP diagnostic tools (consumer, reads what this issue writes)
    Logical sequencing: Standardize service log file locations #1155 → Write structured crash reason file on fatal service exit #1154 → Surface crash logs in MCP diagnostic tools for unreachable services #1153.
• enhancement retained — adding structured crash file persistence is new capability.
• Milestone v1.0.0 kept: crash diagnostics for running services is an operational concern. When a service enters a crash loop under KeepAlive, operators need structured data, not just raw logs. Fits "Operational concerns for running agentd in real environments" well.
• Complexity medium: the implementation work has two parts. First, a shared helper in agentd-common (~40–50 lines: CrashReason struct with JSON serialization, write_crash_reason() with atomic file write, clear_crash_reason() called on healthy startup). Second, per-service integration across 8+ service binaries — each needs to call clear_crash_reason() after successful startup and write a CrashReason on fatal error paths (~5–10 lines each). The pattern is mechanical but the file count makes this solidly medium.
• Implementation note (not a blocking question): Rust offers several approaches to capture fatal exits — std::panic::set_hook for panics, wrapping main() return for Result::Err, or a shared run_service() wrapper. The developer can choose; none require clarification.
• Acceptance criteria are concrete; no clarification needed.