docs: examples gallery table — add Ecosystem/Demonstrates columns, fix two false facts, and guard every row against manifest.json drift

[alphacrack]

User story

As a developer evaluating readme2demo, I want the examples gallery to tell me at a glance what ecosystem each run covers and what it proves, and I want to trust that every fact in that table came from the run's own manifest.json, so that I can pick the example closest to my repo without opening five files — and so the table doesn't quietly rot as the gallery grows.

What

examples/README.md already has a gallery table (examples/README.md:18-21) with Repo / Result / Verified / Commit / Agent cost / Artifacts. This issue does not create it — it closes three gaps in it and adds a test so the next six rows can't drift.

1. Two new columns. The table has no ecosystem and no "what does this run prove" column; today that information lives only in the prose sections below (examples/README.md:26-36), which doesn't scale past a handful of rows.

• Ecosystem — the target's language/toolchain (Python, Go), taken from the target repo, not guessed.
• Demonstrates — one short clause on what the run exercises (e.g. self-run: install, tests, CLI / Go build + MCP server CLI).

2. Fix two statements that are factually wrong.

• examples/README.md:46 says demo.mp4 is "produced per run but kept out of git to stay lean". Both mp4s are committed — git ls-files examples/ lists examples/readme2demo/demo.mp4 (4.0 MB) and examples/toolhive/demo.mp4 (10.7 MB), and .gitignore only ignores runs/. Correct the sentence to describe what is actually committed.
• The Agent cost column header is wrong twice over. The values ($0.72, $0.69) are manifest["total_cost_usd"] — the sum across all stages — while the agent stage's own cost_usd is 0.0 in both manifests (claude-code subscription runs don't report agent cost). Rename the column to Run cost (or Total cost).

3. Add tests/test_examples_index.py — pure-Python, stdlib-only, reads files off disk. For every examples/*/ directory containing a manifest.json, assert:

• the directory has a row in the gallery table;
• the Commit cell equals manifest["commit_sha"][:7];
• the run-cost cell equals manifest["total_cost_usd"] rounded to 2dp;
• the Result cell says Verified iff manifest["verified"] is true;
• the Verified date cell equals the UTC date of manifest["stages"]["verify"]["finished_at"] (this issue also defines that column, which is currently undefined — both existing rows already satisfy it, see Pointers);
• every relative link in the row resolves to a file that exists.

Explicitly out of scope

• Adding new example runs — that is Examples gallery: verified runs across ecosystems (Go, Rust, Terraform, MCP servers, security scanners) #73, and it needs Docker plus real API budget. This issue is the table those rows land into.
• Any generator/codegen that writes the table from manifests. The deliverable is a check, not a build step; a hand-written table that a test polices is the intent.
• Deciding whether 14.7 MB of committed mp4 should stay, move to LFS, or be dropped. Only the sentence describing the status quo gets fixed here; the policy question deserves its own issue.
• Any grounding-path module (distill.py, tutorial.py, normalize.py, engines/, prompts/, templates/). Nothing under src/readme2demo/ is touched.

Why

Ordering against #73. #73 ("Examples gallery: verified runs across ecosystems") is open and assigned, and solicits up to six more rows — Go, Rust, Terraform, an MCP server, a security scanner, and a deliberately BLOCKED example. Every one of those rows will hand-copy commit SHA, cost, verified state and date out of a manifest.json into Markdown. Landing the columns and the drift test before those PRs arrive means each contributor gets a red test instead of a reviewer catching a typo'd SHA — and gives them a table shape to fill in rather than inventing one.

Precedent that this drifts. #43 is an open bug for exactly this failure mode elsewhere in the docs (README says 175 tests, the step-by-step guide says 163). The two false facts found while writing this issue — the mp4 sentence and the Agent cost mislabel — are that same rot, already present in examples/README.md.

Grounding implication (worth stating explicitly, though no grounding code changes). The Result column is the public face of the project's one non-negotiable invariant. A row reading "✅ Verified" whose manifest["verified"] is false would publish a verification claim nothing verified. orchestrator.summarize_markdown already encodes this rule for the CI summary — its docstring at src/readme2demo/orchestrator.py:396-398 says the badge derives from manifest.verified "and nothing else … must not soften, infer, or upgrade it." The gallery table deserves the same rule, enforced the same way: in code, not in a review convention.

Pointers

All verified by reading the files at 64a3cbd.

• examples/README.md:18 — the current header row: | Repo | Result | Verified | Commit | Agent cost | Artifacts |. Rows at :20 (readme2demo) and :21 (toolhive).

• examples/README.md:23-24 — already promises the manifest is the source of truth: "Runs are reproducible from the committed manifest.json in each folder — engine, base image, commit SHA, per-stage status, and total cost included." The test makes that promise enforceable.

• examples/README.md:38-46 — the "What's in each folder" list; line 46 holds the incorrect demo.mp4 claim.

• examples/README.md:48-55 — "Adding an example"; a line pointing contributors at the new test belongs here.

• src/readme2demo/manifest.py:41-55 — the pydantic v2 Manifest model. Every field the table needs exists and is present in both committed manifests: run_id, repo_url, commit_sha, engine, base_image, created_at, stages, verified, total_cost_usd. Note there is no language/ecosystem field — the Ecosystem column is human-supplied and deliberately not test-checked.

• src/readme2demo/manifest.py:32-38 — StageRecord: status, started_at, finished_at, error, cost_usd, meta. stages.verify.finished_at is what the Verified date column should mean.

• src/readme2demo/orchestrator.py:389-419 — summarize_markdown, the existing pure manifest→Markdown renderer, and the verified-badge rule at :396-398.

• src/readme2demo/cli.py:510-528 — readme2demo report <run_dir> --markdown; contributors can use it to read a run's facts rather than hand-parsing JSON.

• tests/test_version.py:15-36 — the closest existing pattern: a test that asserts one hand-maintained file agrees with another (pyproject.toml ↔ __version__). Model the new test on it.

• Current values, confirmed against the manifests — all four checked facts already pass, so the new test goes green on the first run:

  folder	commit_sha[:7]	total_cost_usd	verified	stages.verify.finished_at
  readme2demo	f677e21	0.722585 → $0.72	true	2026-07-05T19:35:58Z
  toolhive	a935334	0.686917 → $0.69	true	2026-07-03T12:51:09Z

Acceptance criteria

• examples/README.md gallery table gains an Ecosystem column and a Demonstrates column, filled in for both existing rows.
• The Agent cost column is renamed to reflect that it holds total_cost_usd (all stages), not the agent stage's cost_usd (which is 0.0 in both manifests).
• The demo.mp4 sentence at examples/README.md:46 is corrected — both mp4s are committed and tracked by git.
• tests/test_examples_index.py exists, is stdlib-only (json, re, pathlib), and discovers example folders by globbing examples/*/manifest.json — adding a folder must not require editing the test.
• For each discovered folder the test asserts: a table row exists; commit matches commit_sha[:7]; cost matches total_cost_usd to 2dp; the Result cell asserts Verified iff manifest["verified"]; the date matches stages.verify.finished_at; every relative link in the row points at a file that exists.
• At least one test carries a """Regression: ...""" docstring naming the drift it prevents (the mp4/cost-label errors found here are fair game).
• The test fails if a row's commit, cost, verified flag, or date is edited away from its manifest — demonstrate by temporarily editing a cell.
• python -m pytest tests/ -q is green; ruff check . is clean; new functions have type hints and docstrings.
• "Adding an example" (examples/README.md:48-55) tells contributors the table row is test-checked and which fields come from manifest.json.
• No file under src/readme2demo/ is modified.

Notes for contributors

Honest prerequisites: this one is genuinely self-contained.

• Pure Python, stdlib only. No Docker, no API key, no network, no LLM call, no pipeline run. Everything you need is already committed in examples/.
• Setup and test loop:

  pip install -e ".[dev]"
  python -m pytest tests/ -q

  Per CLAUDE.md the suite runs in under a second with no Docker, network, or API access. (Heads-up: tests/test_distill.py and friends need jinja2, which comes with the [dev] extra — without it you'll get collection errors, not test failures.)
• Run just yours while iterating: python -m pytest tests/test_examples_index.py -q.
• Read the manifests before writing the parser — examples/readme2demo/manifest.json and examples/toolhive/manifest.json are your only two fixtures, and both are real committed runs. python -m json.tool examples/toolhive/manifest.json is the fastest way in.
• Keep the Markdown parser boring. Split on |, strip cells, match the folder name out of the artifact links. Resist a general-purpose Markdown parser; a table-row regex anchored on the ## Gallery heading is enough, and a clear assertion message beats clever parsing when it fires on someone's PR.
• Don't test the Ecosystem or Demonstrates columns' contents. They have no manifest field behind them (see manifest.py:41-55) — asserting on prose would just be a second thing to keep in sync. Assert the cells are non-empty at most.
• Every bug fix in this repo carries a regression test with a """Regression: ...""" docstring — grep the test files for Regression to see the house pattern.
• If you'd rather add a new example repo than build the table, that's Examples gallery: verified runs across ecosystems (Go, Rust, Terraform, MCP servers, security scanners) #73 — but note it needs Docker and real spend, so it is not a first issue.