Skip to content

fix(ci): surface scheduled seed failures as a tracked issue - #92

Merged
sarthakagrawal927 merged 1 commit into
mainfrom
fix/scheduled-actions-failure-alerting
Aug 15, 2026
Merged

fix(ci): surface scheduled seed failures as a tracked issue#92
sarthakagrawal927 merged 1 commit into
mainfrom
fix/scheduled-actions-failure-alerting

Conversation

@sarthakagrawal927

Copy link
Copy Markdown
Member

Problem

Seed popular repos is the only cron-driven workflow in the repository. When a
scheduled run failed, the only signal was Actions history and a notification
email — the exact way a scheduled job rots unnoticed. That is the remaining,
in-repo half of #33 ("keep seed/enrich/embed workflows green").

The other half of #33 was blocked on the Turso row-read circuit breaker. That
blocker no longer applies: Turso is retired, D1 is authoritative, and the six
most recent scheduled seed runs (2026-08-09 → 2026-08-14) all completed
successfully.

Change

A follow-on alert job runs after every scheduled seed and reconciles a
single open tracking issue labelled scheduled-job-failure:

  • failure → opens the issue, or comments on the existing one, with a link to
    the failing run and the job's step order;
  • success → closes the open issue with a recovery comment;
  • cancelled / skipped → no action, because neither is a health signal;
  • workflow_dispatch → no action, because an operator is already watching.

Assignment is best-effort so an alert is never lost to a non-assignable handle.
The alert job holds the only issues: write grant; seed stays
contents: read.

Verification

  • Extracted the job's shell script and exercised all four branches against a
    mocked gh (failure/no issue, failure/existing issue, success/existing issue,
    success/no issue) — each produced the intended call.
  • bash -n and shellcheck clean on the extracted script; the workflow parses
    as valid YAML with the expected if expression.
  • pnpm docs:check, pnpm lint, pnpm quality:hygiene pass. The one lint
    warning is pre-existing on main (verified by stashing).

No production deploy, migration, or schedule change is included. The alerting
path first exercises itself on the next scheduled Sunday run.

Closes #33

🤖 Generated with Claude Code

The weekly `Seed popular repos` workflow is the only cron-driven job in the
repository, and a failed scheduled run left no signal outside Actions history
and a notification email — the exact way a scheduled job rots unnoticed.

Scheduled runs now reconcile a single open tracking issue labelled
`scheduled-job-failure`: a failure opens it (or comments on the existing one)
with a link to the failing run, and the next successful scheduled run closes
it. Manual dispatches are excluded because an operator is already watching, and
cancelled or skipped runs are not health signals. Only the new `alert` job
holds `issues: write`; the seed job stays `contents: read`.

The Turso-era row-read circuit breaker that previously blocked this work no
longer applies: D1 is authoritative and the recent scheduled seed runs are
green.

Closes #33

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@sarthakagrawal927

Copy link
Copy Markdown
Member Author

Live verification (2026-08-15)

The alert path was executed in Actions under the real GITHUB_TOKEN, using a temporary push-triggered harness on a throwaway branch. The reconcile script embedded in the harness was extracted programmatically from this PR's alert job and asserted byte-identical (sha256 c39e4d5d…), so what ran is the code being merged.

Run: https://github.com/Codevetter/starboard/actions/runs/31887458085

Step Result
gh label create scheduled-job-failure ✅ created
SEED_RESULT=failure → open ✅ opened issue #93
SEED_RESULT=failure again → dedupe ✅ commented on #93, no duplicate
SEED_RESULT=success → recover ✅ closed #93 with recovery comment

Final state of #93 before cleanup: CLOSED, labelled scheduled-job-failure, assigned to sarthakagrawal927, exactly 2 comments (dedupe + recovery). Body markdown rendered correctly — the YAML block-scalar de-indentation leaves the bullets at column 0.

This also settles the one thing static review could not: the repository's default workflow token permission is read (default_workflow_permissions: "read"), and the job-level permissions: issues: write block does elevate past it.

Incidental finding, not a defect in this PR: gh issue list --label is briefly eventually-consistent. A capture step in the harness queried ~1.5s after creation and got an empty result, while the next step ~1s later found #93. In production, consecutive scheduled runs are a week apart, so this cannot cause a duplicate alert; the worst realistic case is a second alert issue if a failed run is re-run within a second or two. Not worth guarding against.

Cleanup: test issue #93 deleted, harness branch deleted, no harness file in this PR's diff. The scheduled-job-failure label is intentionally retained — the job needs it.

@sarthakagrawal927
sarthakagrawal927 merged commit db44c62 into main Aug 15, 2026
2 checks passed
@sarthakagrawal927
sarthakagrawal927 deleted the fix/scheduled-actions-failure-alerting branch August 15, 2026 13:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Scheduled Actions stability

1 participant