Skip to content

Stabilize and alert on missing-embeddings backlog #591

Description

@EtanHey

Incident evidence

During the 2026-07-13 Tier-0 incident, the missing-embeddings backlog was hundreds of rows and changed while enrichment drained it (477 in an older health state, then 433 in a later bounded read-only check). The health-check query itself was fixed separately; this issue is about the underlying backlog behavior.

Follow-up

  • Establish backlog age, arrival rate, and drain-rate telemetry
  • Distinguish transient healthy catch-up from a stuck or growing backlog
  • Add an SLO/alert based on age and sustained growth, not one point-in-time count
  • Identify quota, provider, poison-row, and enrichment-worker causes in the diagnostic output
  • Document a safe recovery path that preserves write arbitration

Safety

Diagnosis should use read-only queries and existing telemetry. Do not bypass the single-writer policy or bulk-update the canonical DB.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions