Skip to content

Add pipeline run-status observability and stale/catch-up UX #68

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
Copilot wants to merge 7 commits into
base: main
Choose a base branch
from
Open

Add pipeline run-status observability and stale/catch-up UX #68

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
4d693a6
Initial plan
Copilot May 8, 2026
0f1c9e1
feat: add run-status metadata and stale pipeline UX
Copilot May 8, 2026
d1242e4
fix: simplify pipeline badge class rendering
Copilot May 8, 2026
bd7ac42
fix: address review feedback on run-status updates
Copilot May 8, 2026
7fe2865
fix: tighten run-status parsing and stale threshold constants
Copilot May 8, 2026
82eb999
fix: preserve last successful run timestamp on degraded runs
Copilot May 8, 2026
20b56ff
style: auto-fix formatting
github-actions[bot] May 8, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .github/agents/wazzup-writer.agent.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,15 +6,18 @@ tools: [execute, edit]
user-invocable: true
argument-hint: "Path to prompt.json and requested output file"
---

You are the Wazzup briefing writer. Your job is to turn ranked source items into concise, source-grounded English news briefing JSON for a single technical reader.

## Boundaries

- Read only the input file named in the user prompt.
- Write only the output file named in the user prompt.
- Do not fetch web pages or add claims that are not present in the input item data.
- Do not include Markdown fences, commentary, analysis notes, or prose outside the requested JSON object.

## Writing Rules

- Preserve the input item order so the newest hourly articles stay at the top, except when merging related items into one synthesized bullet.
- Merge closely related input items into one synthesized bullet when they describe the same story, campaign, incident, vendor, product, or affected organization.
- Every bullet must include citations containing source item IDs from the input.
Expand All @@ -27,6 +30,7 @@ You are the Wazzup briefing writer. Your job is to turn ranked source items into
- Never mention scoring internals such as source weight, score, recency bonus, or duplicate group IDs.

## Output Contract

Write strict JSON with this shape:

```json
Expand Down
6 changes: 3 additions & 3 deletions .github/copilot-instructions.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -68,11 +68,11 @@ mise exec -- task install
The CI workflow ([.github/workflows/ci.yml](.github/workflows/ci.yml)) runs,
in order:

1. `task install` – installs `PyYAML`.
2. `task ci` – runs `format:check`, `lint`, `test`, `build`.
1. `task install` – installs `PyYAML`.
2. `task ci` – runs `format:check`, `lint`, `test`, `build`.
3. `task pipeline:generate:fixtures` – deterministic briefing from
`tests/fixtures` with `AI_PROVIDER=fake`.
4. `task validate:data` – schema/shape checks of `public/data`.
4. `task validate:data` – schema/shape checks of `public/data`.

Always reproduce that exact order locally before pushing. The individual
pieces (validated to work in this devcontainer) are:
Expand Down
3 changes: 2 additions & 1 deletion config/sources.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -525,7 +525,8 @@ sources:
- standings
fetch:
timeoutSeconds: 8
notes: "Official NBA news feed for league-wide headlines. This endpoint can be slow or unreachable from server-side runners, so it uses a shorter timeout."
notes: "Official NBA news feed for league-wide headlines. This endpoint can be slow or unreachable from server-side runners,
so it uses a shorter timeout."

- id: espn-nba
name: ESPN NBA
Expand Down
17 changes: 9 additions & 8 deletions docs/architecture.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -249,6 +249,7 @@ Example fields:
- `latestEveningBriefingUrl`
- `generatedAt`
- `health`
- `runStatus` (last attempted/successful run timestamps, provider, briefing kind, item/source counts, and status)

## Scheduling model

Expand Down Expand Up @@ -406,12 +407,12 @@ The future MCP server should depend on the domain contracts in [../src/wazzup](.

## Key risks and mitigations

| Risk | Mitigation |
| --------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------- |
| Risk | Mitigation |
| --------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------ |
| GitHub Pages exposes personal interests | Public output is accepted for the current deployment; keep source preferences and prompts minimal and support private/static alternatives later. |
| Repository bloat from generated data | Store rolling state in a GitHub Release asset and deploy Pages artifacts without committing generated YAML/JSON. |
| AI hallucinations | Require citations, validate structured output, keep source links visible. |
| AI provider cost spikes | Cap item count today; add summary caching and token/monthly accounting before relying on strict budget guarantees. |
| Scheduled workflows delayed | Treat schedules as best-effort and compute windows from timestamps. |
| Feed parsing failures | Isolate source failures and publish source health. |
| Copyright issues | Store metadata and summaries only; avoid republishing full content. |
| Repository bloat from generated data | Store rolling state in a GitHub Release asset and deploy Pages artifacts without committing generated YAML/JSON. |
| AI hallucinations | Require citations, validate structured output, keep source links visible. |
| AI provider cost spikes | Cap item count today; add summary caching and token/monthly accounting before relying on strict budget guarantees. |
| Scheduled workflows delayed | Treat schedules as best-effort and compute windows from timestamps. |
| Feed parsing failures | Isolate source failures and publish source health. |
| Copyright issues | Store metadata and summaries only; avoid republishing full content. |
47 changes: 30 additions & 17 deletions docs/github-actions.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,19 +2,19 @@

## Workflow overview

| Workflow | Trigger | Responsibility |
| ------------------- | ----------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------ |
| CI | Pull request and manual dispatch | Formatting, syntax linting, tests, compile checks, fixture generation, and generated-data validation. |
| Lint | Pull request and manual dispatch | Reusable organization lint workflow from `DevSecNinja/.github`. |
| Auto-fix formatting | Manual dispatch | Reusable organization formatting workflow that commits dprint/yamlfmt fixes back to the branch. |
| News hourly | Hourly schedule with local cadence gate and manual dispatch | Fetch feeds, generate a rolling briefing, validate data, persist release-backed state, and upload a short-lived `public` artifact for debugging. |
| Workflow | Trigger | Responsibility |
| ------------------- | -------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------ |
| CI | Pull request and manual dispatch | Formatting, syntax linting, tests, compile checks, fixture generation, and generated-data validation. |
| Lint | Pull request and manual dispatch | Reusable organization lint workflow from `DevSecNinja/.github`. |
| Auto-fix formatting | Manual dispatch | Reusable organization formatting workflow that commits dprint/yamlfmt fixes back to the branch. |
| News hourly | Hourly schedule with local cadence gate and manual dispatch | Fetch feeds, generate a rolling briefing, validate data, persist release-backed state, and upload a short-lived `public` artifact for debugging. |
| Pages | Successful `News hourly` workflow run, push to `main`, and manual dispatch | Deploy PWA and static YAML/JSON data to GitHub Pages through the reusable `DevSecNinja/.github` Pages workflow. |
| Config Sync | Weekly and manual dispatch | Open PRs when shared repo config from `DevSecNinja/.github` drifts. |
| Label Sync | Daily, manual dispatch, and label config changes | Sync repository labels from the org base labels plus repo-specific labels. |
| Labeler | Pull requests, issues, and manual dispatch | Apply area/type labels using shared labeler automation. |
| Live smoke | Not implemented yet | Optional real feed and AI provider canary checks with strict budgets. |
| Archive cleanup | Not implemented yet | Keep release-backed rolling state compact and optionally publish monthly recap archives. |
| Release automation | Not implemented yet | Future release-please workflow driven by Conventional Commits. |
| Config Sync | Weekly and manual dispatch | Open PRs when shared repo config from `DevSecNinja/.github` drifts. |
| Label Sync | Daily, manual dispatch, and label config changes | Sync repository labels from the org base labels plus repo-specific labels. |
| Labeler | Pull requests, issues, and manual dispatch | Apply area/type labels using shared labeler automation. |
| Live smoke | Not implemented yet | Optional real feed and AI provider canary checks with strict budgets. |
| Archive cleanup | Not implemented yet | Keep release-backed rolling state compact and optionally publish monthly recap archives. |
| Release automation | Not implemented yet | Future release-please workflow driven by Conventional Commits. |

## Recommended workflow boundaries

Expand Down Expand Up @@ -44,12 +44,12 @@ Release Please remains deferred until the app has an explicit first release/vers

The preferred current path is Copilot CLI because it is GitHub-native and can run directly inside a scheduled workflow. The pipeline should still expose a provider abstraction so the same request can be handled by Copilot CLI, an API provider, Ollama, or a fake test provider.

| Runner | When to use | Workflow implications |
| ------------- | --------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Runner | When to use | Workflow implications |
| ------------- | --------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Copilot CLI | Preferred first production runner. | Install `@github/copilot`, set `COPILOT_GITHUB_TOKEN` from a fine-grained PAT with Copilot Requests permission, run `copilot -p` with `--model`, `--agent`, `--no-ask-user`, and restricted `--allow-tool`. |
| API provider | Fallback or production runner when strict structured output, model selection, or accounting is easier through an API. | Store provider keys in Actions secrets and call through the pipeline adapter. |
| Ollama | Optional local-model experiment or privacy-focused smoke run. | Install/start Ollama, pull/cache a small model, expect slower CPU inference on GitHub-hosted runners. |
| Fake provider | CI and deterministic tests. | No secrets or network calls. |
| API provider | Fallback or production runner when strict structured output, model selection, or accounting is easier through an API. | Store provider keys in Actions secrets and call through the pipeline adapter. |
| Ollama | Optional local-model experiment or privacy-focused smoke run. | Install/start Ollama, pull/cache a small model, expect slower CPU inference on GitHub-hosted runners. |
| Fake provider | CI and deterministic tests. | No secrets or network calls. |

## Implemented CI workflow

Expand Down Expand Up @@ -159,6 +159,19 @@ jobs:

The workflow triggers hourly because GitHub cron is UTC-only and does not understand `Europe/Amsterdam` daylight-saving transitions. A first cadence step computes the local hour and continues every hour from 06:00 to 21:59, then only on even local hours from 22:00 to 05:59. Manual dispatch always runs.

### Manual catch-up for delayed or missed runs

When cron delivery is delayed, use the existing **News hourly** `workflow_dispatch` path:

1. Open **Actions → News hourly → Run workflow**.
2. Keep `forceBriefing=auto` for normal catch-up, or select `hourly`/`morning`/`evening` for an explicit run.
3. Keep `aiProvider=copilot-cli` unless you intentionally want deterministic fallback with `fake`.
4. Run once, then verify `public/data/latest.json` shows a fresh `runStatus.lastSuccessfulRunAt`.

### Lightweight stale-run alert path

The PWA now marks pipeline status as **Stale** when the last attempted run age exceeds the UI threshold. For an operational alert, add a small scheduled workflow that checks `public/data/latest.json` (`runStatus.lastAttemptedRunAt`) and opens an issue or sends a notification when stale for too long.

Operational learning: the first live News hourly run failed because Copilot CLI was requested but the token secret was empty. The workflow now selects an effective provider before installing Node/Copilot. If `copilot-cli` is requested without `COPILOT_REQUESTS_PAT` or `COPILOT_GITHUB_TOKEN`, it logs a warning and uses `AI_PROVIDER=fake` so the release state and Pages deployment path can still be validated end to end.

After enabling the Copilot PAT, one live run failed because Copilot CLI wrote JSON without the required `sections` array. The provider now treats invalid structured Copilot output as an AI-provider failure and falls back to the deterministic summary shape, recording `provider.type: copilot-cli-fallback` and the validation reason instead of failing the whole state/deploy pipeline.
Expand Down
Loading