Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion .github/workflows/pages.yml
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,7 @@ jobs:
artifact-path: public
install-command: |-
python3 -m pip install --break-system-packages -r requirements.txt
test-command: PYTHONPATH=src python3 scripts/check_format.py && python3 scripts/lint.py && PYTHONPATH=src python3 -m unittest discover -s tests && python3 -m compileall -q src scripts
test-command: |-
PYTHONPATH=src python3 scripts/check_format.py && python3 scripts/lint.py && PYTHONPATH=src python3 -m unittest discover -s tests && python3 -m compileall -q src scripts
build-command: PYTHONPATH=src python3 scripts/pages_build.py
cloudflare-preview: false
28 changes: 14 additions & 14 deletions docs/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,20 +37,20 @@ flowchart LR

## Runtime components

| Component | Responsibility | Current implementation |
| -------------------- | -------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------- |
| Source configuration | Defines feeds, short source tags, categories, weights, headers, and interest hints. | [../config/sources.yml](../config/sources.yml) and [../config/interests.yml](../config/interests.yml). |
| Fetcher | Retrieves RSS and Atom XML feeds. | `urllib.request` based Python fetcher in [../src/wazzup/feeds.py](../src/wazzup/feeds.py). |
| Normalizer | Converts source entries into `ContentItem` records. | Pure functions with fixtures. |
| Deduplicator | Groups duplicate or near-duplicate articles. | Canonical URL + raw ref/GUID + normalized title/day transitive groups. |
| Ranker | Scores items against interests, source quality, recency, and coverage. | Deterministic scoring plus optional AI reranking later. |
| Curator | Selects and orders the most relevant items from the ranked list for the briefing. | AI curation provider abstraction; `wazzup-curator` agent for Copilot CLI, deterministic passthrough for fake. |
| Summarizer | Generates article and briefing summaries from the curated item selection. | AI summary provider abstraction with prompt versioning; `wazzup-writer` agent for Copilot CLI. |
| Component | Responsibility | Current implementation |
| --------------------- | -------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------- |
| Source configuration | Defines feeds, short source tags, categories, weights, headers, and interest hints. | [../config/sources.yml](../config/sources.yml) and [../config/interests.yml](../config/interests.yml). |
| Fetcher | Retrieves RSS and Atom XML feeds. | `urllib.request` based Python fetcher in [../src/wazzup/feeds.py](../src/wazzup/feeds.py). |
| Normalizer | Converts source entries into `ContentItem` records. | Pure functions with fixtures. |
| Deduplicator | Groups duplicate or near-duplicate articles. | Canonical URL + raw ref/GUID + normalized title/day transitive groups. |
| Ranker | Scores items against interests, source quality, recency, and coverage. | Deterministic scoring plus optional AI reranking later. |
| Curator | Selects and orders the most relevant items from the ranked list for the briefing. | AI curation provider abstraction; `wazzup-curator` agent for Copilot CLI, deterministic passthrough for fake. |
| Summarizer | Generates article and briefing summaries from the curated item selection. | AI summary provider abstraction with prompt versioning; `wazzup-writer` agent for Copilot CLI. |
| Transparency reporter | Explains run inputs, source health, selection, and AI providers for auditability. | AI report provider abstraction; `wazzup-transparency-reporter` agent for Copilot CLI, deterministic fake report for tests. |
| Publisher | Writes canonical static JSON, source health, transparency reports, `latest`, and `manifest` files. | [../src/wazzup/publisher.py](../src/wazzup/publisher.py). |
| State store | Persists generated data across scheduled runs without commits. | `news-state` GitHub Release asset `wazzup-state.zip`. |
| Delivery adapters | Pushes selected briefings to external channels. | Not implemented yet. |
| Frontend | Displays latest briefing and source health. | Static vanilla PWA in [../public](../public). |
| Publisher | Writes canonical static JSON, source health, transparency reports, `latest`, and `manifest` files. | [../src/wazzup/publisher.py](../src/wazzup/publisher.py). |
| State store | Persists generated data across scheduled runs without commits. | `news-state` GitHub Release asset `wazzup-state.zip`. |
| Delivery adapters | Pushes selected briefings to external channels. | Not implemented yet. |
| Frontend | Displays latest briefing and source health. | Static vanilla PWA in [../public](../public). |

## Pipeline flow

Expand Down Expand Up @@ -420,7 +420,7 @@ The future MCP server should depend on the domain contracts in [../src/wazzup](.
| Risk | Mitigation |
| --------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------ |
| GitHub Pages exposes personal interests | Public output is accepted for the current deployment; keep source preferences and prompts minimal and support private/static alternatives later. |
| Repository bloat from generated data | Store rolling state in a GitHub Release asset and deploy Pages artifacts without committing generated JSON. |
| Repository bloat from generated data | Store rolling state in a GitHub Release asset and deploy Pages artifacts without committing generated JSON. |
| AI hallucinations | Require citations, validate structured output, keep source links visible. |
| AI provider cost spikes | Cap item count today; add summary caching and token/monthly accounting before relying on strict budget guarantees. |
| Scheduled workflows delayed | Treat schedules as best-effort and compute windows from timestamps. |
Expand Down
40 changes: 20 additions & 20 deletions docs/github-actions.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,19 +2,19 @@

## Workflow overview

| Workflow | Trigger | Responsibility |
| ------------------- | -------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------ |
| CI | Pull request and manual dispatch | Formatting, syntax linting, tests, compile checks, fixture generation, and generated-data validation. |
| Lint | Pull request and manual dispatch | Reusable organization lint workflow from `DevSecNinja/.github`. |
| Auto-fix formatting | Manual dispatch | Reusable organization formatting workflow that commits dprint/yamlfmt fixes back to the branch. |
| News | Hourly schedule with a local two-hour active-window cadence gate and manual dispatch | Fetch feeds, generate a rolling briefing, validate data, persist release-backed state, and upload a short-lived `public` artifact for debugging. |
| Pages | Explicit dispatch from `News` after state persists, push to `main`, and manual dispatch | Deploy PWA and static JSON data to GitHub Pages through the reusable `DevSecNinja/.github` Pages workflow. |
| Config Sync | Weekly and manual dispatch | Open PRs when shared repo config from `DevSecNinja/.github` drifts. |
| Label Sync | Daily, manual dispatch, and label config changes | Sync repository labels from the org base labels plus repo-specific labels. |
| Labeler | Pull requests, issues, and manual dispatch | Apply area/type labels using shared labeler automation. |
| Live smoke | Not implemented yet | Optional real feed and AI provider canary checks with strict budgets. |
| Archive cleanup | Not implemented yet | Keep release-backed rolling state compact and optionally publish monthly recap archives. |
| Release automation | Not implemented yet | Future release-please workflow driven by Conventional Commits. |
| Workflow | Trigger | Responsibility |
| ------------------- | --------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------ |
| CI | Pull request and manual dispatch | Formatting, syntax linting, tests, compile checks, fixture generation, and generated-data validation. |
| Lint | Pull request and manual dispatch | Reusable organization lint workflow from `DevSecNinja/.github`. |
| Auto-fix formatting | Manual dispatch | Reusable organization formatting workflow that commits dprint/yamlfmt fixes back to the branch. |
| News | Hourly schedule with a local two-hour active-window cadence gate and manual dispatch | Fetch feeds, generate a rolling briefing, validate data, persist release-backed state, and upload a short-lived `public` artifact for debugging. |
| Pages | Explicit dispatch from `News` after state persists, push to `main`, and manual dispatch | Deploy PWA and static JSON data to GitHub Pages through the reusable `DevSecNinja/.github` Pages workflow. |
| Config Sync | Weekly and manual dispatch | Open PRs when shared repo config from `DevSecNinja/.github` drifts. |
| Label Sync | Daily, manual dispatch, and label config changes | Sync repository labels from the org base labels plus repo-specific labels. |
| Labeler | Pull requests, issues, and manual dispatch | Apply area/type labels using shared labeler automation. |
| Live smoke | Not implemented yet | Optional real feed and AI provider canary checks with strict budgets. |
| Archive cleanup | Not implemented yet | Keep release-backed rolling state compact and optionally publish monthly recap archives. |
| Release automation | Not implemented yet | Future release-please workflow driven by Conventional Commits. |

## Recommended workflow boundaries

Expand Down Expand Up @@ -278,9 +278,9 @@ Expected secrets:
| Secret | Purpose |
| ---------------------------- | ----------------------------------------------------------------------------------------------------------- |
| `COPILOT_REQUESTS_PAT` | Preferred repository secret containing a fine-grained PAT for Copilot CLI with Copilot Requests permission. |
| `COPILOT_GITHUB_TOKEN` | Alternative secret name accepted by the News workflow. |
| `COPILOT_MODEL` | Optional Copilot CLI model override for curator and transparency; defaults to `claude-sonnet-4.6`. |
| `COPILOT_WRITER_MODEL` | Optional Copilot CLI model override for the briefing writer; defaults to `claude-opus-4.8`. |
| `COPILOT_GITHUB_TOKEN` | Alternative secret name accepted by the News workflow. |
| `COPILOT_MODEL` | Optional Copilot CLI model override for curator and transparency; defaults to `claude-sonnet-4.6`. |
| `COPILOT_WRITER_MODEL` | Optional Copilot CLI model override for the briefing writer; defaults to `claude-opus-4.8`. |
| `AZURE_OPENAI_ENDPOINT` | Azure OpenAI endpoint. |
| `AZURE_OPENAI_API_KEY` | Azure OpenAI API key. |
| `OPENAI_API_KEY` | Optional alternative provider. |
Expand Down Expand Up @@ -331,10 +331,10 @@ The scheduled `News` workflow is responsible for mutating release-backed state.

Observed failure modes and fixes:

| Failure | Cause | Fix |
| -------------------------------------------------------- | -------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------- |
| News run failed in Copilot CLI | `COPILOT_GITHUB_TOKEN` was empty while `AI_PROVIDER=copilot-cli`. | Select effective provider first and fall back to `fake` when no Copilot token secret exists. |
| Pages deploy failed validating `public/data/latest.json` | Reusable workflow received empty `GH_TOKEN`, state restore skipped, and `public/data` was missing. | Restore public release asset without a token in Pages builds and make missing retained state fatal for `pages:build`. |
| Failure | Cause | Fix |
| -------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------- |
| News run failed in Copilot CLI | `COPILOT_GITHUB_TOKEN` was empty while `AI_PROVIDER=copilot-cli`. | Select effective provider first and fall back to `fake` when no Copilot token secret exists. |
| Pages deploy failed validating `public/data/latest.json` | Reusable workflow received empty `GH_TOKEN`, state restore skipped, and `public/data` was missing. | Restore public release asset without a token in Pages builds and make missing retained state fatal for `pages:build`. |
| Pages deploy failed during mise install | Reusable workflow install command could not pass `github.token`, so mise made unauthenticated GitHub API calls and hit rate limits. | Avoid mise in Pages deploy; install Python deps directly and run `scripts/pages_build.py`. |

## Security hardening
Expand Down
Loading