You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
As a readme2demo user whose run finished but came back UNVERIFIED, I want an issue template that asks me for exactly the artifacts a maintainer opens first, so that my report is debuggable on the first read instead of costing three round-trips of "please also paste X".
What
Add .github/ISSUE_TEMPLATE/run_not_verified.md — a third template alongside the two that exist today (bug_report.md, feature_request.md) — whose fields are lifted directly from CLAUDE.md's maintenance meta-workflow step 1.
In scope
New file.github/ISSUE_TEMPLATE/run_not_verified.md with front matter roughly:
name: Run did not verifyabout: The pipeline completed but the fresh-container replay failed (or the video/guide came out wrong)title: "[run] "labels: bug
Sections, in this order:
One-paste summary — ask for readme2demo report <run-dir> --markdown output first. This single paste already carries stage statuses, the Verified badge, per-stage cost, stage errors, and which artifacts exist (cli.py:511, orchestrator.py:389), so it front-loads most of what a maintainer needs.
Which exit code — 0 verified / 1 completed UNVERIFIED / 2 a stage failed, per _report_exit_code (cli.py:494). This is the field that routes the report; bug_report.md has no equivalent.
Artifacts checklist covering all four the meta-workflow names: manifest.json, tail of verify.log, command_log.json (what the agent actually ran), and tape_coverage.json (which guide steps were dropped from the video).
Environment — readme2demo --version output, OS + arch, docker version (and whether docker info succeeds), --engine, --llm-backend, --base-image if overridden.
What you already tried — a prompt for the readme2demo resume <run> --from-stage <s> invocation attempted, so maintainers don't re-suggest it.
Redaction note — transcript.ndjson and verify.log are raw agent/command output from the target repo; tell reporters to scrub secrets before pasting.
Two-line edit to CONTRIBUTING.md — the "Reporting a broken run" section (CONTRIBUTING.md:53-58) currently names only manifest.json and verify.log/transcript.ndjson; point it at the two templates and say which is which.
Explicitly out of scope
Converting any template to GitHub's YAML issue-forms schema. Keep all three as markdown for consistency; a forms migration is a separate, larger change.
Touching bug_report.md's existing fields beyond adding one "if your run merely failed to verify, use the other template" pointer line.
Any Python. No change to cli.py, report, or the run-dir layout — this issue is satisfied by markdown alone.
1. blank_issues_enabled: false (.github/ISSUE_TEMPLATE/config.yml:1) — reporters must pick one of the two existing templates. There is no third option.
2. The only fitting template assumes a stage failed.bug_report.md:24-26 asks:
But the most common bad outcome has no failed stage at all. _report_exit_code (cli.py:494) distinguishes exactly this:
2 — a stage failed (the run itself broke);
1 — no stage failed, but the fresh-container replay did not pass
(completed UNVERIFIED);
0 — verified.
and _drive prints ⚠ Completed UNVERIFIED. (cli.py:637) while exiting 0 — a run that produced every artifact, cost real money, and still isn't publishable. A reporter in that state has to leave the "which stage failed" field blank or guess.
The difference in one line:bug_report.md is for readme2demo misbehaved; this template is for readme2demo behaved as designed and my run still isn't verified — a target-repo/environment/coverage problem, which is a different debugging path.
bug_report.md was last corrected by #131 ("docs: fix bug report backend values", closing #99), which fixed the --llm-backend value list and split --engine into its own field (bug_report.md:45-46). That fix is good and this issue keeps it — the new template should copy those exact field labels rather than reinvent them, so the two stay in sync.
Adjacent but distinct: #12 (readme2demo doctor) would produce the environment half of this report as command output. The two compose — once #12 lands, the Environment section becomes "paste readme2demo doctor" — but neither blocks the other, and #12 does nothing for the artifact half.
Pointers
Verified by reading the files at 64a3cbd.
.github/ISSUE_TEMPLATE/ contains exactly three files today: bug_report.md, feature_request.md, config.yml. There is no template for a completed-but-unverified run.
.github/ISSUE_TEMPLATE/config.yml:1 — blank_issues_enabled: false; contact links route only to security advisories and Discussions.
.github/ISSUE_TEMPLATE/bug_report.md:24-26 — the "Which stage failed" section, which presumes a failure.
.github/ISSUE_TEMPLATE/bug_report.md:32-34 — current artifact checklist is manifest.json, tail of verify.logortranscript.ndjson, and commands.sh. command_log.json and tape_coverage.json are absent — the first two things CLAUDE.md's meta-workflow step 1 tells a maintainer to open.
src/readme2demo/cli.py:494-508 — _report_exit_code, quoted above; the 0/1/2 split the new "which exit code" field mirrors.
src/readme2demo/cli.py:511-551 — the report command; --json and --markdown are mutually exclusive, and Manifest.load(run_dir) is the only file it parses.
src/readme2demo/cli.py:482-491 — REPORT_ARTIFACTS; the comment states the summary uses "existence checks only, so the summary keeps working on partial and failed runs" — which is why report --markdown is safe to ask for on a broken run.
src/readme2demo/cli.py:637-638 — the ⚠ Completed UNVERIFIED. branch pointing at verify.log.
src/readme2demo/cli.py:409-415 — resume and its --from-stage option (choices from manifest.STAGES).
src/readme2demo/orchestrator.py:389-421 — summarize_markdown; its docstring is worth honoring in the template's framing: "The Verified badge derives from manifest.verified and nothing else — that flag is set only by the fresh-container replay in the verify stage, and this summary must not soften, infer, or upgrade it."
src/readme2demo/types.py:7-21 — authoritative run-dir layout (manifest.json, plan.json, transcript.ndjson, command_log.json, commands.sh, demo.tape, verify.log, …). Use these exact filenames.
src/readme2demo/distill.py:764-778 — build_tape_from_step_by_step writes tape_coverage.json as {"guide_steps": N, "tape_steps": M, "dropped": [...]} and warns ⚠ {n}/{N} guide steps excluded from the video (not proven in the agent run). This file is the direct answer to "why is my video short / missing steps", and no template asks for it.
CONTRIBUTING.md:53-58 — "Reporting a broken run" names only the repo URL, manifest.json, and the tail of verify.log/transcript.ndjson; update it to point at both templates.
docs/usage.md:48, docs/whats-new-0.7.0.md:40-46, README.md:93 — readme2demo report and report --markdown are already user-facing and documented, so asking for that paste is not asking users to learn something new.
Acceptance criteria
.github/ISSUE_TEMPLATE/run_not_verified.md exists with valid YAML front matter (name, about, title, labels) and renders as a third choice on the "New issue" chooser.
The template asks for readme2demo report <run-dir> --markdown output as its first field.
The template asks which exit code readme2demo report returned, with the 0/1/2 meanings spelled out to match _report_exit_code.
The artifact checklist names all four of manifest.json, verify.log (tail), command_log.json, tape_coverage.json, spelled exactly as in types.py:7-21.
The environment block asks for readme2demo --version, OS + arch, Docker version and whether docker info succeeds, --engine, and --llm-backend — reusing the post-docs: fix bug report backend values #131 value lists from bug_report.md:45-46 verbatim.
The template includes a "scrub secrets before pasting" note covering transcript.ndjson and verify.log.
bug_report.md gains one pointer line distinguishing the two templates; no other change to it.
CONTRIBUTING.md's "Reporting a broken run" section points at both templates and says which to pick.
No file under src/ or tests/ is modified.
Notes for contributors
Prerequisites: none beyond a text editor. This is markdown only — no Python, no Docker, no API key, no model credential, and no readme2demo run required. You do not need to reproduce a failing run to write the template; every field is sourced from the file:line references above.
Useful to skim first: CLAUDE.md's "maintenance meta-workflow" section (step 1 is the field list) and its known-failure-classes table — classes 3, 5, 6, 8, and 13 are all diagnosed from command_log.json or tape_coverage.json, which is why those two artifacts carry their weight.
If you do want to run the suite: pip install -e ".[dev]" && python -m pytest tests/ -q — pure Python, no Docker, no network, no API key, and it finishes in about a second (PR #131 reported 281 passed). This change touches no tests, so no new test is expected; the repo's regression-test rule applies to code fixes, not template additions.
Grounding implications: none. This change adds no code to the grounding path (distill.py, tutorial.py, normalize.py, engines/, prompts/, templates/) and cannot let an unverified command reach tutorial.md, step_by_step.md, commands.sh, or demo.tape. One thing to preserve in wording, though: the template must never imply a maintainer can mark a run verified by inspection. manifest.verified is set only by the fresh-container replay in the verify stage, and summarize_markdown's docstring says the summary "must not soften, infer, or upgrade it" — keep that framing in any prose you write.
To preview locally, push the branch and open https://github.com/<you>/readme2demo/issues/new/choose on your fork — GitHub reads templates from the default branch of the repo you're viewing, so you'll need the branch merged into your fork's default branch to see it in the chooser.
User story
What
Add
.github/ISSUE_TEMPLATE/run_not_verified.md— a third template alongside the two that exist today (bug_report.md,feature_request.md) — whose fields are lifted directly from CLAUDE.md's maintenance meta-workflow step 1.In scope
.github/ISSUE_TEMPLATE/run_not_verified.mdwith front matter roughly:readme2demo report <run-dir> --markdownoutput first. This single paste already carries stage statuses, the Verified badge, per-stage cost, stage errors, and which artifacts exist (cli.py:511,orchestrator.py:389), so it front-loads most of what a maintainer needs.0verified /1completed UNVERIFIED /2a stage failed, per_report_exit_code(cli.py:494). This is the field that routes the report;bug_report.mdhas no equivalent.manifest.json, tail ofverify.log,command_log.json(what the agent actually ran), andtape_coverage.json(which guide steps were dropped from the video).readme2demo --versionoutput, OS + arch,docker version(and whetherdocker infosucceeds),--engine,--llm-backend,--base-imageif overridden.readme2demo resume <run> --from-stage <s>invocation attempted, so maintainers don't re-suggest it.transcript.ndjsonandverify.logare raw agent/command output from the target repo; tell reporters to scrub secrets before pasting.CONTRIBUTING.md— the "Reporting a broken run" section (CONTRIBUTING.md:53-58) currently names onlymanifest.jsonandverify.log/transcript.ndjson; point it at the two templates and say which is which.Explicitly out of scope
bug_report.md's existing fields beyond adding one "if your run merely failed to verify, use the other template" pointer line.cli.py,report, or the run-dir layout — this issue is satisfied by markdown alone.readme2demo doctorcommand. That is Add areadme2demo doctorcommand to check local setup #12 and stays independent (see Why).Why
Two concrete gaps, both verified in-tree:
1.
blank_issues_enabled: false(.github/ISSUE_TEMPLATE/config.yml:1) — reporters must pick one of the two existing templates. There is no third option.2. The only fitting template assumes a stage failed.
bug_report.md:24-26asks:But the most common bad outcome has no failed stage at all.
_report_exit_code(cli.py:494) distinguishes exactly this:and
_driveprints⚠ Completed UNVERIFIED.(cli.py:637) while exiting 0 — a run that produced every artifact, cost real money, and still isn't publishable. A reporter in that state has to leave the "which stage failed" field blank or guess.The difference in one line:
bug_report.mdis for readme2demo misbehaved; this template is for readme2demo behaved as designed and my run still isn't verified — a target-repo/environment/coverage problem, which is a different debugging path.bug_report.mdwas last corrected by #131 ("docs: fix bug report backend values", closing #99), which fixed the--llm-backendvalue list and split--engineinto its own field (bug_report.md:45-46). That fix is good and this issue keeps it — the new template should copy those exact field labels rather than reinvent them, so the two stay in sync.Adjacent but distinct: #12 (
readme2demo doctor) would produce the environment half of this report as command output. The two compose — once #12 lands, the Environment section becomes "pastereadme2demo doctor" — but neither blocks the other, and #12 does nothing for the artifact half.Pointers
Verified by reading the files at
64a3cbd..github/ISSUE_TEMPLATE/contains exactly three files today:bug_report.md,feature_request.md,config.yml. There is no template for a completed-but-unverified run..github/ISSUE_TEMPLATE/config.yml:1—blank_issues_enabled: false; contact links route only to security advisories and Discussions..github/ISSUE_TEMPLATE/bug_report.md:24-26— the "Which stage failed" section, which presumes a failure..github/ISSUE_TEMPLATE/bug_report.md:32-34— current artifact checklist ismanifest.json, tail ofverify.logortranscript.ndjson, andcommands.sh.command_log.jsonandtape_coverage.jsonare absent — the first two things CLAUDE.md's meta-workflow step 1 tells a maintainer to open..github/ISSUE_TEMPLATE/bug_report.md:40-46— Environment block; copy these labels verbatim (post-docs: fix bug report backend values #131 wording).src/readme2demo/cli.py:494-508—_report_exit_code, quoted above; the 0/1/2 split the new "which exit code" field mirrors.src/readme2demo/cli.py:511-551— thereportcommand;--jsonand--markdownare mutually exclusive, andManifest.load(run_dir)is the only file it parses.src/readme2demo/cli.py:482-491—REPORT_ARTIFACTS; the comment states the summary uses "existence checks only, so the summary keeps working on partial and failed runs" — which is whyreport --markdownis safe to ask for on a broken run.src/readme2demo/cli.py:637-638— the⚠ Completed UNVERIFIED.branch pointing atverify.log.src/readme2demo/cli.py:409-415—resumeand its--from-stageoption (choices frommanifest.STAGES).src/readme2demo/orchestrator.py:389-421—summarize_markdown; its docstring is worth honoring in the template's framing: "The Verified badge derives frommanifest.verifiedand nothing else — that flag is set only by the fresh-container replay in the verify stage, and this summary must not soften, infer, or upgrade it."src/readme2demo/types.py:7-21— authoritative run-dir layout (manifest.json,plan.json,transcript.ndjson,command_log.json,commands.sh,demo.tape,verify.log, …). Use these exact filenames.src/readme2demo/normalize.py:20—COMMAND_LOG_FILENAME = "command_log.json".src/readme2demo/distill.py:764-778—build_tape_from_step_by_stepwritestape_coverage.jsonas{"guide_steps": N, "tape_steps": M, "dropped": [...]}and warns⚠ {n}/{N} guide steps excluded from the video (not proven in the agent run). This file is the direct answer to "why is my video short / missing steps", and no template asks for it.src/readme2demo/manifest.py:21—STAGES = ["ingest", "agent", "normalize", "distill", "verify", "tutorial", "render"].CONTRIBUTING.md:53-58— "Reporting a broken run" names only the repo URL,manifest.json, and the tail ofverify.log/transcript.ndjson; update it to point at both templates.docs/usage.md:48,docs/whats-new-0.7.0.md:40-46,README.md:93—readme2demo reportandreport --markdownare already user-facing and documented, so asking for that paste is not asking users to learn something new.Acceptance criteria
.github/ISSUE_TEMPLATE/run_not_verified.mdexists with valid YAML front matter (name,about,title,labels) and renders as a third choice on the "New issue" chooser.readme2demo report <run-dir> --markdownoutput as its first field.readme2demo reportreturned, with the 0/1/2 meanings spelled out to match_report_exit_code.manifest.json,verify.log(tail),command_log.json,tape_coverage.json, spelled exactly as intypes.py:7-21.readme2demo --version, OS + arch, Docker version and whetherdocker infosucceeds,--engine, and--llm-backend— reusing the post-docs: fix bug report backend values #131 value lists frombug_report.md:45-46verbatim.transcript.ndjsonandverify.log.bug_report.mdgains one pointer line distinguishing the two templates; no other change to it.CONTRIBUTING.md's "Reporting a broken run" section points at both templates and says which to pick.src/ortests/is modified.Notes for contributors
Prerequisites: none beyond a text editor. This is markdown only — no Python, no Docker, no API key, no model credential, and no readme2demo run required. You do not need to reproduce a failing run to write the template; every field is sourced from the file:line references above.
Useful to skim first: CLAUDE.md's "maintenance meta-workflow" section (step 1 is the field list) and its known-failure-classes table — classes 3, 5, 6, 8, and 13 are all diagnosed from
command_log.jsonortape_coverage.json, which is why those two artifacts carry their weight.If you do want to run the suite:
pip install -e ".[dev]" && python -m pytest tests/ -q— pure Python, no Docker, no network, no API key, and it finishes in about a second (PR #131 reported 281 passed). This change touches no tests, so no new test is expected; the repo's regression-test rule applies to code fixes, not template additions.Grounding implications: none. This change adds no code to the grounding path (
distill.py,tutorial.py,normalize.py,engines/,prompts/,templates/) and cannot let an unverified command reachtutorial.md,step_by_step.md,commands.sh, ordemo.tape. One thing to preserve in wording, though: the template must never imply a maintainer can mark a run verified by inspection.manifest.verifiedis set only by the fresh-container replay in the verify stage, andsummarize_markdown's docstring says the summary "must not soften, infer, or upgrade it" — keep that framing in any prose you write.To preview locally, push the branch and open
https://github.com/<you>/readme2demo/issues/new/chooseon your fork — GitHub reads templates from the default branch of the repo you're viewing, so you'll need the branch merged into your fork's default branch to see it in the chooser.