From 087305ef77bcb41db12141ba39d87afea401cc9a Mon Sep 17 00:00:00 2001 From: Sarthak Agrawal Date: Sat, 15 Aug 2026 18:47:23 +0530 Subject: [PATCH] fix(ci): surface scheduled seed failures as a tracked issue MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The weekly `Seed popular repos` workflow is the only cron-driven job in the repository, and a failed scheduled run left no signal outside Actions history and a notification email — the exact way a scheduled job rots unnoticed. Scheduled runs now reconcile a single open tracking issue labelled `scheduled-job-failure`: a failure opens it (or comments on the existing one) with a link to the failing run, and the next successful scheduled run closes it. Manual dispatches are excluded because an operator is already watching, and cancelled or skipped runs are not health signals. Only the new `alert` job holds `issues: write`; the seed job stays `contents: read`. The Turso-era row-read circuit breaker that previously blocked this work no longer applies: D1 is authoritative and the recent scheduled seed runs are green. Closes #33 Co-Authored-By: Claude Opus 5 (1M context) --- .github/workflows/seed-popular.yml | 66 ++++++++++++++++++++++++++++++ PROJECT_STATUS.md | 10 +++++ docs/operations/jobs.md | 9 ++++ 3 files changed, 85 insertions(+) diff --git a/.github/workflows/seed-popular.yml b/.github/workflows/seed-popular.yml index b71daa4..bf95c07 100644 --- a/.github/workflows/seed-popular.yml +++ b/.github/workflows/seed-popular.yml @@ -112,3 +112,69 @@ jobs: TOOL_ENRICH_LIMIT: ${{ github.event.inputs.tool_enrich_limit || '250' }} TOOL_ENRICH_HARD_LIMIT: '750' TOOL_MIN_STARS: '10000' + + # A scheduled run has no human watching it, so a failure is otherwise only + # visible in Actions history or a notification email. Keep one open tracking + # issue that reflects the current health of the scheduled seed. + alert: + needs: seed + # Cancelled and skipped runs are not health signals, so they neither open + # nor close the tracking issue. + if: >- + always() && github.event_name == 'schedule' + && (needs.seed.result == 'success' || needs.seed.result == 'failure') + runs-on: ubuntu-latest + timeout-minutes: 5 + permissions: + contents: read + issues: write + env: + GH_TOKEN: ${{ github.token }} + GH_REPO: ${{ github.repository }} + ALERT_LABEL: scheduled-job-failure + RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }} + SEED_RESULT: ${{ needs.seed.result }} + steps: + - name: Ensure the alert label exists + run: | + gh label create "$ALERT_LABEL" \ + --color B60205 \ + --description 'A scheduled workflow run failed' >/dev/null 2>&1 || true + + - name: Reconcile the scheduled-seed health issue + run: | + set -euo pipefail + open_issue="$(gh issue list --label "$ALERT_LABEL" --state open \ + --limit 1 --json number --jq '.[0].number // empty')" + + if [[ "$SEED_RESULT" == 'success' ]]; then + if [[ -n "$open_issue" ]]; then + gh issue close "$open_issue" \ + --comment "Scheduled seed recovered: [run]($RUN_URL) completed successfully." + echo "Closed alert #$open_issue after recovery." + else + echo 'Scheduled seed succeeded; no open alert to close.' + fi + exit 0 + fi + + body="The scheduled \`Seed popular repos\` run finished with result \`$SEED_RESULT\`. + + - Failing run: $RUN_URL + - Job steps: migrations → \`db:seed-popular\` → Worker embedding → \`db:enrich-tools\` + + This issue closes automatically when the next scheduled run succeeds." + + if [[ -n "$open_issue" ]]; then + gh issue comment "$open_issue" --body "$body" + echo "Commented on existing alert #$open_issue." + else + created="$(gh issue create \ + --title 'Scheduled seed run failed' \ + --label "$ALERT_LABEL" \ + --body "$body")" + echo "Opened alert $created." + # Assignment is best-effort: an alert must never be lost because the + # owner handle is not assignable from this token. + gh issue edit "$created" --add-assignee sarthakagrawal927 >/dev/null 2>&1 || true + fi diff --git a/PROJECT_STATUS.md b/PROJECT_STATUS.md index 8342d21..17e37f3 100644 --- a/PROJECT_STATUS.md +++ b/PROJECT_STATUS.md @@ -72,6 +72,14 @@ provenance. The workflow is free and has no billing or entitlement gate. ## Timeline +- **2026-08-15 (scheduled seed failures are visible)** — The weekly seed is the + only cron-driven workflow, and a failed run previously left no signal outside + Actions history. Scheduled runs now reconcile one open tracking issue labelled + `scheduled-job-failure`: a failure opens or updates it with the failing run + link, and the next successful scheduled run closes it. The Turso-era row-read + circuit breaker no longer applies — D1 is authoritative, and the six most + recent scheduled seed runs completed successfully. + - **2026-08-13 (weekly catalog reconciliation implemented locally)** — Replaced the multi-day popular-repository cursor walk with one complete weekly GitHub identity reconciliation. Immutable creation-date partitions fit in single @@ -290,6 +298,8 @@ provenance. The workflow is free and has no billing or entitlement gate. 70% branches) on selected core logic; production-accurate Playwright journeys cover the Astro landing and public app at desktop and mobile widths. - Pre-push lint hook. +- Scheduled seed runs self-report health through a single auto-opened and + auto-closed `scheduled-job-failure` tracking issue. - TypeScript config and Astro landing tooling made self-contained for green Cloudflare builds. ## Work queue diff --git a/docs/operations/jobs.md b/docs/operations/jobs.md index ba286af..8504a82 100644 --- a/docs/operations/jobs.md +++ b/docs/operations/jobs.md @@ -48,6 +48,15 @@ annotates intent, inputs, and dependencies. enumeration is expected to use roughly 250 requests, below the workflow token's 1,000 requests/hour per-repository allowance. +- **Failure visibility:** a follow-on `alert` job runs after every *scheduled* + seed and keeps exactly one open tracking issue labelled + `scheduled-job-failure`. A failed run opens that issue (or comments on the + existing one) with a link to the failing run; the next successful scheduled + run closes it. Manual `workflow_dispatch` runs are skipped because an operator + is already watching, and cancelled or skipped runs neither open nor close the + issue. The `alert` job holds the only `issues: write` grant in the workflow; + `seed` itself stays `contents: read`. + ## embed-pending (`.github/workflows/embed-pending.yml`) - **Schedule:** `workflow_dispatch` only (manual).