Skip to content

docs: publish your run's badge.json so shields.io can render it — workflow snippet + embed markdown (slice of #63) #196

Description

@alphacrack

User story

As a repo owner who added the readme2demo Action to CI, I want a copy-paste workflow that publishes my run's badge.json at a stable URL, so that img.shields.io/endpoint?url=… renders a live "README verified" badge in my README without me reverse-engineering where the file went.

What

Documentation for step 2 of #63 — the hop between "every run writes badge.json" (shipped in #156) and "a badge renders in a README". Concretely, a new ## Verification badge section in docs/usage.md, immediately after ## Outputs (docs/usage.md:64-68), containing:

  1. Where the file is and what's in it. badge.json lands in the run dir like every other artifact; from a workflow it's at ${{ steps.<id>.outputs.run-dir }}/badge.json (the Action exposes run-diraction.yml:63-68). Show both documents verbatim (verified/green, unverified/red) and note the extra commit key is provenance for humans — shields ignores unknown keys (src/readme2demo/tutorial.py:302-304).

  2. Route A (recommended): a dedicated orphan branch. A complete, actionlint-clean workflow that copies badge.json onto a readme2demo-badge branch and commits it. Per Emit a shields.io-compatible verification badge endpoint per run #63's own suggestion ("commit to a readme2demo-badge branch or push to gh-pages"). Endpoint URL is then https://raw.githubusercontent.com/OWNER/REPO/readme2demo-badge/badge.json. Needs permissions: contents: write scoped to that job only — call that out.

  3. Route B: GitHub Pages — with the conflict warning. Publishing via actions/upload-pages-artifact + actions/deploy-pages is prior art in this repo (.github/workflows/docs.yml:38-40,49-50, with the Pages permission set at :11-15 and concurrency: group: pages at :17-20). But a repo has one Pages deployment source: a second Pages-deploying workflow replaces the whole site. If the consumer already publishes docs via Pages, the badge must be folded into the existing site build as an extra file, not deployed separately. Document this rather than handing someone a snippet that nukes their docs site.

  4. The three mechanics that make it correct (see Why — this is the substance):

    • if: always() on the publish step, because the Action exits 1 on unverified.
    • A guard for "no badge.json at all" (pipeline broke before the tutorial stage) that leaves the previous badge untouched rather than publishing anything.
    • Trigger on push to the default branch only — not pull_request (fork PRs have no secrets, and a PR shouldn't mint the repo's badge).
  5. The README markdown to copy, with OWNER/REPO placeholders and the badge linked to the workflow run page, matching the badge style already in README.md:3-5:

    [![README verified](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/OWNER/REPO/readme2demo-badge/badge.json)](https://github.com/OWNER/REPO/actions/workflows/readme-check.yml)
  6. A short caching caveat. Both shields.io and raw.githubusercontent.com cache; the badge lags a fresh run. State what you actually observe — do not invent a TTL.

Out of scope (do not bundle):

Why

#63 lists three deliverables. Item 1 (emit badge.json) landed in #156render_badge and write_badge_json at src/readme2demo/tutorial.py:277-316, called as the first statement of the tutorial stage at src/readme2demo/orchestrator.py:291-295. Items 2 and 3 (publish it, document the embed) are still open, and they are the half a user actually touches: today a run mints a badge into a directory on a CI runner that is deleted when the job ends. docs/whats-new-0.7.0.md:52-62 announces the file and closes with "Hosting it as a live badge endpoint is tracked in #63" — this issue is the docs half of that tracking.

The if: always() requirement is not a style nit, it's the run-dir-level expression of the repo invariant. #156's design goes out of its way to guarantee an unverified run gets a loud red badge and never a missing one — hence writing it before the LLM polish pass (orchestrator.py:292-295) and the docstring at tutorial.py:311-312. But the Action's gate step exits 1 when verified != true (action.yml:151-154), so a naive publish step never runs on exactly the runs that most need publishing: the badge silently freezes on its last green value while the README is broken. A stale green badge is the worst possible outcome for a project whose entire pitch is verified provenance. The docs must lead with this.

The complementary case matters too and behaves differently: a completed but unverified run does produce badge.json (the render stage is skipped when unverified — orchestrator.py:269-274 — but tutorial runs before render and writes the badge), whereas a run that dies in ingest/agent/distill never reaches the tutorial stage and produces no badge at all. "Red badge" and "no badge" are different signals; the snippet must not paper over the second by publishing something invented.

Pointers

All verified against main at 64a3cbd.

  • src/readme2demo/tutorial.py:277-305render_badge(manifest) -> dict. Pure; verdict from manifest.verified alone, date from stages["verify"].finished_at, falling back to today's UTC date. Line 303 comment: "Provenance extra; shields.io ignores unknown keys."
  • src/readme2demo/tutorial.py:308-316write_badge_json(run_dir, manifest) -> Path, writing run_dir / "badge.json" with indent=2. Docstring: "Always written: an unverified run gets a loud red badge, never a missing file."
  • src/readme2demo/orchestrator.py:291-295_stage_tutorial calls write_badge_json as its first statement, with the comment "Badge first: its verdict comes from manifest.verified alone … a TutorialError mid-stage must not be able to suppress it."
  • src/readme2demo/orchestrator.py:266-274_stage_render skips the video when not manifest.verified ("Tutorial still ships, loudly labeled UNVERIFIED"). This is why an unverified run still has a badge.json.
  • action.yml:63-68 — the run-dir output: "Absolute path (on the runner) to the runs/<run-id> directory … Empty if the run failed before a run directory was created." The empty case is the guard condition for the snippet.
  • action.yml:136-154 — "Fail the check when the run is unverified": reads readme2demo report --json, and exit 1 when verified != "true". The reason if: always() is mandatory.
  • action.yml:156-167 — "Upload run artifacts", if: ${{ always() && … }}, path list = tutorial.md, step_by_step.md, verify.log, demo.gif. badge.json is not in this list; the always() idiom here is the pattern to copy.
  • .github/workflows/docs.yml:11-15,17-20,38-40,49-50 — prior art for the Pages route: pages: write + id-token: write, concurrency: group: pages, actions/upload-pages-artifact@v5 (path site), actions/deploy-pages@v5 under environment: github-pages. Also the source of the one-Pages-source warning.
  • docs/usage.md:64-68 — the ## Outputs artifact list. It has not been updated since feat: emit shields.io endpoint badge.json per run #156: it names tutorial.md, step_by_step.md, troubleshooting.md, commands.sh, demo.tape, demo.mp4, demo.gif, howto.jsonld, manifest.json — no badge.json. Fix this one-line omission as part of the change; it's the anchor the new section hangs off.
  • docs/whats-new-0.7.0.md:52-62 — "Step 4 — every run now mints badge.json", ending "Hosting it as a live badge endpoint is tracked in Emit a shields.io-compatible verification badge endpoint per run #63." Link the new section from here.
  • README.md:3-5 — existing shields badge markdown; match the style.
  • mkdocs.yml:38-44 — nav. No change needed if the content goes in docs/usage.md (already listed); a new page would require a nav entry, since the site builds with --strict (docs.yml:35).

Correction to the brief that produced this issue: it suggested the deliverable includes "a workflow file". It should not add one under .github/workflows/ — that directory holds this repo's own CI, and a badge-publish workflow here would have nothing to publish. The snippet lives in the docs as a fenced block; mkdocs-material has content.code.copy enabled (mkdocs.yml:17), so the fenced block is the copy surface. This also matches how #149 handles its readme-check.yml snippet.

Acceptance criteria

  • docs/usage.md gains a ## Verification badge section after ## Outputs
  • docs/usage.md:66-68's artifact list adds badge.json
  • Both badge documents (verified/green, unverified/red) shown verbatim, matching what render_badge actually emits — copy from a real run dir or from render_badge, don't hand-write
  • Route A snippet is complete and self-contained: on: push to the default branch, the Action step, a publish step with if: always(), permissions: contents: write at job scope, and a guard for a missing/empty run-dir or absent badge.json
  • The if: always() requirement is stated with its reason (action.yml:151-154 exits 1 on unverified) — not left as an unexplained line of YAML
  • "Red badge" vs "no badge" is distinguished, and the snippet leaves the existing badge untouched in the no-badge case rather than publishing a substitute
  • Route B documents the one-Pages-source conflict and points at .github/workflows/docs.yml as the reference implementation
  • README embed markdown given with OWNER/REPO placeholders, linked to the workflow run page
  • Snippets are actionlint-clean (extract to a file, run actionlint <file>)
  • docs/whats-new-0.7.0.md:62 links to the new section
  • mkdocs build --strict passes
  • Cross-links to docs: github-action.md — consumer guide + copy-paste readme-check.yml for the README-breaker Action (docs slice of #62) #149's Action guide if it has landed; otherwise a plain sentence pointing at the Action, reconciled at review time
  • No src/ changes, no .github/workflows/ additions

Notes for contributors

  • Pure docs. No Docker, no API key, no LLM calls to write or verify this. pip install -e ".[docs]" then mkdocs serve to preview; mkdocs build --strict is the gate.
  • No new tests expected — this touches no Python. Still run pip install -e ".[dev]" && python -m pytest tests/ -q before opening the PR; it's pure Python with no Docker or network and finishes in about 1.5 seconds. The badge functions already have coverage in tests/test_tutorial.py:654-698 and tests/test_orchestrator.py:192-222; read those to see the guaranteed behavior you're documenting.
  • Honest prerequisite: exercising the full loop end-to-end (real run → published badge → shields renders it) needs a repo with the Action wired up, Docker, and a metered API key — that costs real money. You do not need to do that to land this. Deriving the two JSON documents from render_badge and reasoning from action.yml is sufficient and verifiable. If you do run it end-to-end, put the resulting badge URL in the PR description — that's a genuinely valuable artifact.
  • Grounding implication — read before writing. This issue touches no grounding-path module (distill.py, tutorial.py's LLM pass, normalize.py, engines/, prompts/, templates/); nothing here can put an unverified command into a published artifact. But the spirit applies directly: a badge that goes stale instead of turning red is the same class of failure as publishing an unverified command. If you find yourself writing a snippet where a broken README leaves a green badge up, the snippet is wrong.
  • Zero invented specifics. Don't state a shields.io cache TTL, a raw.githubusercontent.com TTL, or a content-type requirement you haven't confirmed. "Both layers cache, so the badge can lag a run" is true and sufficient; a made-up number is worse than no number.
  • Cheapest way to get a real badge.json to quote: python -c "import json, pathlib; from readme2demo.manifest import Manifest; from readme2demo import tutorial; print(json.dumps(tutorial.render_badge(Manifest.model_validate_json(pathlib.Path('examples/readme2demo/manifest.json').read_text())), indent=2))" — pure Python, no run needed. (examples/readme2demo/manifest.json has "verified": true, so flip the field in a scratch copy for the red document.)

Metadata

Metadata

Assignees

No one assigned

    Labels

    area:ciGitHub workflows, release, packagingarea:docsdocs/ site contentdocumentationDocs, README, docs-sitegood first issueSmall, self-contained, newcomer-friendlyroadmap:nextROADMAP 'Next' — hardening the core

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions