From c59d2c71a5421e199b58d6a9949bc627bf442226 Mon Sep 17 00:00:00 2001 From: Huiyan Wan Date: Thu, 6 Aug 2026 11:40:48 +0100 Subject: [PATCH] feat(overnight-multi-issue-implementation): what one 11-hour, 20-PR overnight run taught (1.3.0) Source: a single overnight run of 17 items -- 20 merged PRs, 4 orchestrated workflows, 91 subagents, ~11 hours. Every addition below is something that cost real work on that night, not a guess about what might go wrong. Seven additions to the SKILL body, all verified absent from 1.2.2 by grep before splicing: 1. Never truncate a findings payload. Three reviewers returned six critical findings; the merging agent was handed `JSON.stringify(reviews).slice(0, 9000)` and five arrived. The sixth was cut mid-object and was the worst of the six. Three reviewers dispatched, three verdicts returned, findings commented on the PR, PR merged -- nothing looked wrong. Rules: never slice a findings list; make the merging agent report received-vs-answered and treat a mismatch as blocking; the run journal holds each agent's true return value when the orchestrator's own view was truncated. Same family as a green suite that ran zero tests. 2. Coordinating with sessions you do not control. The existing collision audits assume the other work is a branch you can diff. Increasingly it is another live agent session whose files are not committed. Claim your intent -- id, state, plain label, and the file list -- on a committed shared board before starting. It worked: a parallel session read the board, saw two of its three tasks already claimed, and did only the third. Append, never replace (two sessions each wrote a one-element array and the second erased the first). Don't drop the claim while your PRs are open. Nothing expires it, so take it down at the end. 3. Amend a running orchestration through a file on disk, not the script. Guardrail text is embedded in every agent prompt, so editing the script changes every call signature and a resume re-runs completed work instead of replaying from cache -- on this run that would have discarded eight finished PRs. Point the script at a plan file and append dated addenda. Check which phase is running first: an addendum after the last agent started is a note to nobody. 4. A third kind of collision neither pre-flight audit can see: the same piece of work, in two places, under two names, in a live session's uncommitted tree and declared against a different issue. Gate rather than parallelise, then extract the piece into a tested module and prove the refactor changed no result by diffing both paths' outputs. 5. The baseline numbers in your own brief go stale mid-run. A parallel PR re-derived the published test counts while the run was in flight; the guardrail text said 269, the truth became 314, and a later item quoted its own brief. Measure the baseline on your own rebase; never carry a count from a document. An item pasted a stale count inside the PR that existed to fix stale counts. 6. What an autonomous run may and may not decide. An assumption is a default, not a ruling, and never overrides a standing decision or a pre-registration. For pre-registered questions the conservative action is DISCLOSE, not COMPUTE. An un-run unit is "no result", not "inconclusive". Production changes only in the reverting direction and only on unanimous reviewer authorisation, asked as a separate explicit field. Merging its own PRs is a separate grant from changing production -- neither is implied by the instruction to implement. 7. A verbatim line for every reviewer prompt: "Check whether this change ships a fresh instance of the defect it repairs." Five of the twenty merged PRs did -- a correction to a figure with no corpus named printed a figure with no corpus named. It was the most common review finding of the night, ahead of ordinary regressions, and every instance was caught by someone re-deriving a number, never by reading the diff. Plus: dropping an honest positive is drift too. Seams adjusted, not content rewritten: one connective sentence so the tier rubric still reads as "below"; the new section's merge/production paragraph now points at Phase C step 4 and the anti-patterns, which hold the ask-before-merge line; the "don't auto-deploy" anti-pattern names the one bounded exception the new section introduces, so the skill does not argue against itself; the blocked-reviewer sister plugin is now listed in References, since the body cites it. The draft's "a fourth collision" was wrong -- the document establishes two (ID-level and file-level), so it is a third. An independent reviewer read the result against CONTRIBUTING.md and found three things the splice had left broken. All three are fixed here: - **"The observed run" was never defined anywhere in the shipped file.** Five new passages rest their evidence on it, including "five of twenty merged PRs", while the only run the file described was the 2026-05-08 chatbox session -- 2 PRs, ~6 hours, ~38 dispatches. A reader following the phrase landed on numbers that contradicted it, and the run that actually backs the prose lived only in the README and this message, neither of which ships to an installed user. The Overview now says which run the phrase means, and a second worked example at the end gives it, with each lesson traced to what it cost. - **The production carve-out quietly removed the human.** The anti-pattern demands "human review for high-stakes changes"; the new exception permitted a change on "unanimous reviewer authorisation", and everywhere else in this file a reviewer is a subagent. As written the run could change production unattended but could not merge a PR without waking someone. Both ends now say it plainly: the reviewers are the run's own agents, no human is in the loop at 3am, that is why the direction bound comes first, and anything that is not a revert still waits for a person. - **The stale-baseline lesson collided with the recipe three lines above it.** That recipe defines the baseline as a failing-set snapshot taken once, before the chain; the new text said to measure it "on its own rebase", which would absorb an earlier PR's regression and then print IDENTICAL over it. The two are now distinguished: the thing you must never quote from a document is the suite's headline count, not the pre-chain snapshot. Two smaller ones from the same read: the companion-skill pointer had been left below the new third-collision paragraphs, where it looked like it covered them (it is branch-based, and the third kind is defined by having no branch) -- moved above, and it now says so; and `resumeFromRunId` was named without saying what provides it. Two pre-existing anonymization scars in files this PR already touches, fixed while here: "for the\nthe project" in SKILL.md:319 and "the project propensity project" in README.md:58 are both the visible seam of an old scrub, not identifiers. Versions, in every place this repo records one -- the failure mode of #20/#21/ #24 is a content change that never ships because a version did not move: SKILL.md frontmatter 1.2.0 -> 1.3.0 (had drifted; the 1.2.1 and 1.2.2 bumps were manifest-only) plugins/.../.claude-plugin/plugin.json 1.2.2 -> 1.3.0 .claude-plugin/marketplace.json 1.2.2 -> 1.3.0 VERSION (bundle) 1.2.1 -> 1.3.0 VERSION had also drifted a patch behind the plugins. release.yml's own comment says to bump it in the same PR as the version bump, so it moves here and cuts the v1.3.0 release. No description changed. The gate reports overnight-multi-issue-implementation at 1,463 chars with 73 to spare, identical to before this commit. SKILL.md 553 -> 742 lines. Gates: description-cap exit 0, leak exit 0, validate_plugins exit 0, listing budget fits at 1M context with 9,496 chars to spare. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_013JVMGELnoA2vM56Va4gLXE --- .claude-plugin/marketplace.json | 2 +- README.md | 5 +- VERSION | 2 +- .../.claude-plugin/plugin.json | 2 +- .../SKILL.md | 201 +++++++++++++++++- 5 files changed, 201 insertions(+), 11 deletions(-) diff --git a/.claude-plugin/marketplace.json b/.claude-plugin/marketplace.json index 772083d..da1ab3a 100644 --- a/.claude-plugin/marketplace.json +++ b/.claude-plugin/marketplace.json @@ -28,7 +28,7 @@ "name": "overnight-multi-issue-implementation", "source": "./plugins/overnight-multi-issue-implementation", "description": "Overnight autonomous workflow that takes a cluster of related GitHub issues (typically a P1 review-panel finding set) and ships them to merged stacked PRs by morning. Builds on subagent-driven-development with overnight-specific discipline: stacked PRs (so PR2 doesn't wait on a human PR1-merge mid-night), pre-flight tracker-id audit (concurrent sessions on main steal IDs), final PR-level code review before proposing merge, review findings preserved as PR comments before squash. Sister to overnight-review-client-delivery and overnight-insight-discovery.", - "version": "1.2.2" + "version": "1.3.0" }, { "name": "large-redesign-parallel-branch-collision-audit", diff --git a/README.md b/README.md index a4e98c1..f60f032 100644 --- a/README.md +++ b/README.md @@ -12,7 +12,7 @@ Sister [Claude Code](https://claude.com/claude-code) plugins for running **auton |---|---| | [**overnight-review-client-delivery**](plugins/overnight-review-client-delivery/) | You already have a client deliverable (slide deck, report, HTML, memo) that needs polishing + quality-gating before a morning hand-off. Runs Phase A (content work) + Phase B (8-agent review panel in parallel) + Phase C (morning synthesis). | | [**overnight-insight-discovery**](plugins/overnight-insight-discovery/) | You want to *generate* a client-facing insight brief from scratch — surfacing funnel leaks and surprise patterns from data. Runs two parallel tracks (B = LLM-autonomous creative exploration + C = hybrid deterministic-with-narration), consolidates, and reviews. Gated by BOTH a **novelty** check (not a known feature restated) and an **analytical validity** check (`references/observational_analysis_rigor.md` — composition / leak / anchor-timing / marker-vs-lever) so a surprising-but-wrong finding can't ship. | -| [**overnight-multi-issue-implementation**](plugins/overnight-multi-issue-implementation/) | You have a cluster of 6–15 related GitHub issues (typically a P1 review-panel finding set) and want them implemented + reviewed + opened as stacked PRs by morning. Runs Phase A (PR1 tasks via subagent-driven-development) + Phase B (PR2 tasks stacked on PR1) + Phase C (PR-level code review + morning hand-off). Also covers the **plan-driven variant** (independent PRs from a written plan rather than stacked PRs from issues) and **Phase 0** (stale-backlog triage + owner-ruling application, for issue clusters that aren't pre-validated). | +| [**overnight-multi-issue-implementation**](plugins/overnight-multi-issue-implementation/) | You have a cluster of 6–15 related GitHub issues (typically a P1 review-panel finding set) and want them implemented + reviewed + opened as stacked PRs by morning. Runs Phase A (PR1 tasks via subagent-driven-development) + Phase B (PR2 tasks stacked on PR1) + Phase C (PR-level code review + morning hand-off). Also covers the **plan-driven variant** (independent PRs from a written plan rather than stacked PRs from issues), **Phase 0** (stale-backlog triage + owner-ruling application, for issue clusters that aren't pre-validated), and the multi-session discipline a long run needs: never truncating a findings payload, claiming intent on a shared in-repo board so a parallel agent session doesn't duplicate your work, amending a running orchestration through a file on disk rather than the script, and an explicit line on what the run may and may not decide on its own. | ## Companion safety patterns @@ -55,7 +55,7 @@ Neither plugin trusts the author (or the track) to self-review. A panel of 4–8 **But verify every reviewer actually saw what it reviewed.** Many review/search subagents (`feature-dev:code-reviewer`, `voltagent-*`, `Explore`) ship **without a `Bash` tool**, so a reviewer told to `gh pr diff`/checkout a PR returns a **BLOCKED** report — or silently reviews the current checkout (often `main`, which predates the work) instead. In an unattended overnight run, **a BLOCKED reviewer reads as a CLEAN one**, and the bug it never looked at ships by morning. Pre-generate per-base diffs to files + materialize PR branches as worktrees + hand each reviewer explicit paths, and in the morning synthesis treat **BLOCKED as not-clean** (re-dispatch before counting the vote). See [`overnight-review-panel-blocked-reviewer-reads-as-clean`](plugins/overnight-review-panel-blocked-reviewer-reads-as-clean/). -> **Standing convention — review every non-trivial PR with the panel before merge.** Distinct from the *deliverable* panel above (Phase B audits a doc/deck): before squash-merging any **non-trivial code PR**, run the [`roundtable:agent-review-panel`](https://github.com/wan-huiyan/agent-review-panel) skill with **all panel agents set to `model: opus`** (the skill's enforced default) instead of (or in addition to) a single code-reviewer agent — multiple independent opus reviewers catch what one reviewer misses, gating client-facing / substantive changes. Fold/triage findings, fix, re-run if needed, THEN squash-merge. **Trivial / docs-only PRs may skip the full panel** (same non-trivial threshold). This `roundtable:`-invoked panel is the same `agent-review-panel` dependency named in §1 + [Dependencies](#dependencies). (Origin: the project propensity project, 2026-06-02.) +> **Standing convention — review every non-trivial PR with the panel before merge.** Distinct from the *deliverable* panel above (Phase B audits a doc/deck): before squash-merging any **non-trivial code PR**, run the [`roundtable:agent-review-panel`](https://github.com/wan-huiyan/agent-review-panel) skill with **all panel agents set to `model: opus`** (the skill's enforced default) instead of (or in addition to) a single code-reviewer agent — multiple independent opus reviewers catch what one reviewer misses, gating client-facing / substantive changes. Fold/triage findings, fix, re-run if needed, THEN squash-merge. **Trivial / docs-only PRs may skip the full panel** (same non-trivial threshold). This `roundtable:`-invoked panel is the same `agent-review-panel` dependency named in §1 + [Dependencies](#dependencies). (Origin: the admissions propensity project, 2026-06-02.) ### 2. Locked-file escape hatch @@ -164,6 +164,7 @@ All three plugins encode patterns from real overnight runs. `overnight-review-cl ## Version history +- **2026-08-06** — `overnight-multi-issue-implementation` → **v1.3.0** (SKILL + both manifests + bundle `VERSION`, which had drifted a patch behind): seven lessons from a single overnight run of 17 items — 20 merged PRs, 4 orchestrated workflows, 91 subagents, ~11 hours. **Never `.slice()` a findings payload** (three reviewers returned six critical findings; the merging agent received five, and every visible signal still said the gate had worked — make the actor count what it received against what it answered, and keep the run journal as the recovery path). **Coordinating with sessions you do not control**: claim your intent and your file list on a shared in-repo board before you start, append rather than replace, and don't take the claim down while your PRs are open. **Amend a running orchestration through a file on disk, not the script** — editing the script changes every agent prompt and a resume then re-runs completed work instead of replaying it from cache. A **third kind of collision** neither pre-flight audit can see (the same piece of work under two names, in a live session's uncommitted tree). **Baseline numbers in your own brief go stale mid-run** — measure, never quote. **What an autonomous run may and may not decide** (an assumption is a default, not a ruling; for pre-registered questions disclose rather than compute; an un-run unit is "no result", not "inconclusive"; production changes only in the reverting direction and only on unanimous authorisation from the run's own reviewers, with anything that is not a revert still waiting for a person; merging is a separate grant from changing production). And a **verbatim line for every reviewer prompt** — "Check whether this change ships a fresh instance of the defect it repairs." — which caught five of the twenty merged PRs, every one by re-deriving a number rather than by reading the diff. - **2026-07-17** — `overnight-multi-issue-implementation` → **v1.2.0** (SKILL + manifests, fixing a manifest-version drift): adds **Phase 0 — backlog triage + owner-ruling application** for unvalidated issue clusters (triage biased against dismissal with adversarial verification of dismissals only; owner cut-line ratification via an interactive review page; rulings baked as greppable issue comments before any build; decision-session / build-session split with a wave-ordered kickoff prompt; follow-up ruling rounds handled additively; successor-before-close sequencing). Also documents that the close-keyword issue trap fires from **docs-only planning PR bodies** ("then close #N" in a kickoff-prompt addendum closes the live tracker on merge). Extracted from a real large-backlog triage-and-rulings run. Cross-links the new [`interactive-feedback-report`](https://github.com/wan-huiyan/interactive-feedback-report) skill. - **2026-06-02** — Standing convention added: review every **non-trivial PR** with the `roundtable:agent-review-panel` skill (all agents `model: opus`) before squash-merge; trivial/docs-only PRs may skip the full panel. Reconciled with the per-PR tier rubric (the panel is the heavyweight tier; single-reviewer tiers remain for low-risk PRs). `overnight-multi-issue-implementation` SKILL → v1.1.1, `subagent-review-tier-calibration-for-overnight-pr-chains` SKILL → v1.0.1. - **v1.1.0** (2026-05-08) — Adds `overnight-multi-issue-implementation` for the engineering-side overnight pattern (issues → stacked PRs). README updated to reflect three plugins; install + compose sections expanded. diff --git a/VERSION b/VERSION index 6085e94..f0bb29e 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -1.2.1 +1.3.0 diff --git a/plugins/overnight-multi-issue-implementation/.claude-plugin/plugin.json b/plugins/overnight-multi-issue-implementation/.claude-plugin/plugin.json index cb80bbe..e101366 100644 --- a/plugins/overnight-multi-issue-implementation/.claude-plugin/plugin.json +++ b/plugins/overnight-multi-issue-implementation/.claude-plugin/plugin.json @@ -1,7 +1,7 @@ { "name": "overnight-multi-issue-implementation", "description": "Overnight autonomous workflow that takes a cluster of related GitHub issues (typically a P1 review-panel finding set) and ships them to merged stacked PRs by morning. Builds on subagent-driven-development with overnight-specific discipline: stacked PRs (so PR2 doesn't wait on a human PR1-merge mid-night), pre-flight tracker-id audit (concurrent sessions on main steal IDs), final PR-level code review before proposing merge, review findings preserved as PR comments before squash. Sister to overnight-review-client-delivery and overnight-insight-discovery. v1.2.0 adds Phase 0 for unvalidated backlogs: stale-issue triage biased against dismissal, owner cut-line ratification, rulings baked as issue comments, and a wave-ordered kickoff prompt.", - "version": "1.2.2", + "version": "1.3.0", "author": { "name": "wan-huiyan" }, diff --git a/plugins/overnight-multi-issue-implementation/SKILL.md b/plugins/overnight-multi-issue-implementation/SKILL.md index 4f61ffe..38bbd4e 100644 --- a/plugins/overnight-multi-issue-implementation/SKILL.md +++ b/plugins/overnight-multi-issue-implementation/SKILL.md @@ -16,7 +16,7 @@ description: | work (use plain `subagent-driven-development`), polishing an existing deliverable (use `overnight-review-client-delivery`), or generating insights from data (use `overnight-insight-discovery`). author: wan-huiyan + Claude Code -version: 1.2.0 +version: 1.3.0 date: 2026-05-29 --- @@ -35,6 +35,12 @@ Sister to `overnight-review-client-delivery` (polishes existing deliverables) and `overnight-insight-discovery` (generates insights from data). Different problem shape, same overnight-autonomous philosophy. +Two real runs back this skill, and both are written up at the end. Passages +below that say **"the observed run"** or "the observed night" all mean the +larger one: 2026-08-05, 17 items across 4 orchestrated workflows, 91 +subagents, ~11 hours, 20 PRs merged by morning. The 2026-05-08 chatbox +session is the smaller run the skill was first extracted from. + ## When to use All of these conditions: @@ -157,7 +163,36 @@ digraph overnight { Per task: **implementer subagent → spec-compliance reviewer → code-quality reviewer → mark complete**. Standard `subagent-driven-development` protocol. -For overnight throughput, calibrate review intensity **per-task** using the 3-tier rubric below (this generalizes the previous "two pragmatic deviations" version into a formal framework — see companion plugin `subagent-review-tier-calibration-for-overnight-pr-chains` for the standalone skill). +For overnight throughput, calibrate review intensity **per-task** using the 3-tier rubric below (this generalizes the previous "two pragmatic deviations" version into a formal framework — see companion plugin `subagent-review-tier-calibration-for-overnight-pr-chains` for the standalone skill). Whichever tier a task lands in, the reviewer's findings must reach the actor intact — the rule immediately below holds at every tier. + +### Never truncate a findings payload — the reviews happen, the fixes don't + +The reviewer's output is the only thing standing between a bad change and `main`. If your +orchestration hands it onward as a **sliced string**, findings die silently and every visible signal +still says the gate worked. + +Observed: three reviewers returned **six** critical findings; the merging agent received +`JSON.stringify(reviews).slice(0, 9000)` and **five arrived**. The sixth was cut mid-object. It was +the worst of the six — a keep-or-kill decision resting on a margin about five times finer than the +data could resolve, in a 553-line pre-registration with no power statement anywhere. Three +reviewers dispatched, three verdicts returned, findings commented on the PR, PR merged. Nothing +looked wrong. + +**Rules:** + +- **Never `.slice()` a findings list, verdict, or review payload.** Truncate self-reports and prose + if you must; never the artifact whose entire purpose is to block an action. +- **Make the actor count.** Require the merging agent to report how many findings it *received* + against how many it *answered*, and treat a mismatch as blocking. The only reason the loss above + was recoverable is that the agent **noticed the payload was cut and said so** instead of + proceeding quietly. Put that instruction in the prompt in those words. +- **The journal is the recovery path.** A workflow journal (`journal.jsonl`, one + `{"type":"result"}` line per agent) holds each agent's true return value even when the + orchestrator's own view was truncated. Parse it and diff against what was acted on. + +Same family as a green suite that executed zero tests: the machinery reports success over an empty +set. See sister skill `overnight-review-panel-blocked-reviewer-reads-as-clean` — that one is a +reviewer that could not see; this one is a reviewer that saw and could not be heard. ### Tier 1 — Full two-stage (strict `subagent-driven-development`) @@ -194,6 +229,18 @@ with P0/P1/P2 categorization. Return VERDICT: APPROVE | REQUEST_CHANGES | REJECT with categorized findings. ``` +**Put this line in every reviewer prompt, verbatim — Tier 1's review-panel prompts included:** +*"Check whether this change ships a fresh instance of the defect it repairs."* On the observed run +**five of twenty merged PRs did** — a correction to a figure with no corpus named printed a figure +with no corpus named; a document about uncited copied numbers contained an uncited copied number. +About one in four, on documents. It was the single most common review finding, ahead of ordinary +regressions, and **every instance was caught by someone re-deriving a number, never by reading the +diff**. Budget a round for it. + +**And tell reviewers that dropping an honest POSITIVE is drift too.** A summary that omits the +reassuring facts its source carries is not "conservative" — it is inaccurate in the direction nobody +audits, and it reads as more alarming than the truth. Check omissions in both directions. + ### Tier 3 — Bash-only verification (no reviewer subagent) Controller verifies inline via bash/grep on the PR diff, no subagent dispatch. @@ -226,6 +273,17 @@ pytest -q 2>&1 | grep -E '^FAILED' | sort > /tmp/now_fails.txt diff /tmp/baseline_fails.txt /tmp/now_fails.txt && echo "IDENTICAL — zero regressions" ``` +**And your own brief's baseline numbers go stale mid-run.** On the observed night the published test +counts were re-derived by a parallel PR while the run was in flight — the guardrail text said +`server 269`, the truth became `314`, and a later item read a number from its own brief. **The rule +survives; the numbers do not.** Instruct every agent to measure the baseline itself, on its own +rebase, immediately before judging its branch, and never to carry a count from a document. This is +about the *count a document quotes*, not the failing-set snapshot above — that one is still taken +once, before the chain, or a regression gets absorbed into a re-measured baseline and the set-diff +prints IDENTICAL over it. An item +that pastes a stale count *inside the PR that exists to fix stale counts* is not hypothetical — it +happened, in a document about uncited copied figures. + **For UI tasks, static checks are not verification.** `node --check` is syntax-only; a render test proves the template renders, not that it *looks right* or that interactive JS works. When the live flow is blocked (auth/seed bugs) or needs heavy state, verify the rendered template standalone: render to a file, inline the stylesheet, serve it (`python3 -m http.server` — `file://` is blocked in the Playwright MCP), then drive with Playwright and assert layout facts via `getBoundingClientRect` (e.g. "the dropdown's bottom extends past its clipping ancestor AND its last item is within the viewport" proves an `overflow:hidden` clip fix). Bounding-box assertions beat screenshots, which time out on external web-font loading. See `flask-webapp-browser-debug`. ### Decision rubric @@ -267,8 +325,8 @@ If your overnight run is the only writer, this is a no-op. If you're running in parallel with another session (common during P1 sweeps after a review panel), the concurrent session WILL take your reserved IDs by the time you reach Phase B's finalization. Resolve via the project's standard -PR-conflict skill (e.g., `pr-conflict-site-regen` for the -the project) — hand-union the generator + regenerate site. +PR-conflict skill (e.g., `pr-conflict-site-regen` for the project) — +hand-union the generator + regenerate site. ## Pre-flight: parallel-branch file-collision audit @@ -305,7 +363,67 @@ For each collision-risk branch, surface a 3-way decision to the user BEFORE plan The user owns this decision. Don't decide unilaterally — the cost asymmetry is large (10 min of audit pre-flight vs. hours of careful manual conflict resolution post-redesign). -For the full pattern, decision rubric, and worked example, see the companion skill `large-redesign-parallel-branch-collision-audit` (plugin in this bundle). +For the full pattern, decision rubric, and worked example, see the companion skill `large-redesign-parallel-branch-collision-audit` (plugin in this bundle). It covers the branch-level case only — the third kind below is outside its reach by construction, because there is no branch to diff. + +**A third kind of collision, and neither pre-flight audit can see it: the same PIECE OF WORK, in two +places, under two names.** A live session was building a box-to-region join inside its own analysis +script, declared against a *different* issue than the one that tracks the join. A file-level audit +shows no conflict — different paths, different issues — and two implementations get built. What +catches it is reading the live session's actual working tree, not its branch. + +**When you find it, gate rather than parallelise.** The correct sequencing is: let the session that +is already in it finish, then a follow-up item **extracts** the piece into a named, tested, +reusable module and closes the tracking issue — proving the refactor changed no result by running +both paths and diffing the outputs. That gate is worth writing even when you expect it to skip; +a plan that predicts a skip and skips is honest, and it fires when the blocker clears (it did). + +## Coordinating with sessions you do not control + +The collision audits above assume the other work is a *branch*. Increasingly it is another **agent +session**, running right now, whose files are not committed yet. A branch you can diff; a live +session you can only negotiate with. + +**Claim your intent on a shared board, in the repo, before you start.** One entry appended to a +committed file (this project uses `docs/site/assets/live.json`) carrying: an id, a state, a +plain-English label, and **the list of files this run intends to touch**. It works — during the +observed run a parallel session read the board, saw two of its three assigned tasks already claimed, +and correctly did only the third. That coordination cost one small merged PR. + +**Rules that make the board load-bearing rather than decorative:** + +- **APPEND to the array; never replace it.** Two sessions each wrote a single-element `running` + array on the same night, so whichever landed second erased the other's claim. Neither noticed. +- **Amend the claim when your scope grows.** Adding three items mid-run without updating the board + invites a parallel session to start one of them. Do it before the work, not at wrap-up. +- **A pause needs an honest state — and your validator may reject the one you invent.** `paused` was + refused by the project's own schema (`running | waiting | blocked`), which is the gate working. An + absent state often defaults to "running", so it must be set explicitly. +- **Do NOT delete your claim while your PRs are still open.** It is tempting on a pause: it frees the + files. It also invites another session to pick up half-reviewed PRs and land them. Keep the claim, + list the open PR numbers in the note, and say plainly what to do if the run never comes back. +- **Take it down at the end.** Nothing expires it. There is no heartbeat and no TTL. + +## Amend a running orchestration through a file on disk, not the script + +An overnight run will need correcting mid-flight — a PR merges and unblocks something, a +reservation lifts, a premise moves. **Editing the orchestration script is the wrong channel**: the +guardrail text is embedded in every agent prompt, so changing it changes every call signature, and a +resume then re-runs completed work instead of replaying it from cache. On the observed run that +would have discarded eight finished PRs. + +**So put the brief on disk and have every agent read it at start.** The script points at a plan +file; amendments are appended to that file as dated addenda. Agents that have already started keep +their instructions; agents that start later read the correction. No cache invalidation, no rebuild. + +Corollaries: + +- **Verify the amendment actually reaches someone.** An addendum appended after the last agent has + started is a note to nobody. Check which phase is running first. +- **State in the addendum which of the brief's own facts it supersedes**, by name. "The baseline + counts in your guardrails are stale — measure your own" beats silently changing a number. +- Resume by run id — whatever your orchestration harness calls it (`resumeFromRunId` in the one used + here) — so unchanged agents replay from cache. Same script plus same args equals a 100% hit; the + first edited call and everything after it runs live. ## Stacked-PR strategy (load-bearing for overnight) @@ -338,6 +456,45 @@ If `gh pr merge --delete-branch` fails locally with the worktree-checkout- trap, leave the branch undeleted overnight; user can clean up morning. See sister skill `gh-pr-merge-worktree-checkout-trap`. +## What an autonomous run may and may not decide + +Overnight autonomy is a spectrum, and the useful line is not "how risky" but "who owns the call". +Settle these before the run, in the brief, in these words. + +**"Proceed on a clearly-flagged assumption" never overrides a standing rule.** Letting the run +proceed rather than park is usually right — a parked item delivers nothing and the owner wakes to a +queue. But an assumption is a *default*, not a *ruling*. Anything a documented decision already +settles, or that a pre-registration exists to protect, is out of scope for an assumption no matter +how well flagged. Name those explicitly in the guardrails; do not rely on judgement. + +**For pre-registered questions the conservative assumption is DISCLOSE, not COMPUTE.** Computing a +registered criterion after the outcome is known is precisely the thing registration prevents. The +honest autonomous action is to record that it was never satisfied and leave the arithmetic to the +owner. + +**An un-run unit is not an inconclusive result.** When a gate correctly stops a round from running, +report "no result" — inconclusive is an outcome of something that *happened*, and reporting it +claims a measurement that does not exist. The observed run got this right unprompted and it is +worth making explicit. + +**Production changes: the reverting direction only, and only on unanimous authorisation.** If the +run can change a live system, bound it two ways. First, direction: it may return production to the +behaviour that ran before, never enable something new — a revert is cheap to undo and its failure +mode is known. Second, authorisation: require every independent reviewer to answer, as a separate +explicit field, *"is a production change authorised by what I personally verified?"* — anchored to a +gate that passed and a rule fixed before the data. Unanimity, or nothing changes. Be clear-eyed +about who those reviewers are: they are the run's own review agents, and no human is in the loop at +3am. That is exactly why the direction bound comes first — unanimity among agents is not a +substitute for a person, so the only change they may authorise is one whose failure mode is already +known and cheap to undo. + +**Merging its own PRs is a separate grant from changing production.** Do not infer one from the +other. Where the repo has no CI and no branch protection, say so in the brief: the run's own review +gates are the *only* safety net, which is the argument for tiering them rather than skipping them. +Neither grant is implied by the instruction to *implement* — Phase C step 4 and the anti-patterns +below hold that line; what this section adds is that the two grants are also independent of each +other. + ## Phase C: morning hand-off discipline Before proposing merge to user: @@ -493,7 +650,12 @@ By morning the user should have: - **Don't** auto-deploy after merge. The user is asleep; even if your project has auto-deploy-on-merge wired, the deploy preflight (e.g., `deploy-from-stale-worktree-silent-rollback`) needs human review for - high-stakes changes. + high-stakes changes. The only bounded exception is the one set out in + "What an autonomous run may and may not decide", and it is narrow: returning + production to the behaviour that ran before, never enabling anything new, + only where the evening's brief granted it, and only on unanimous + authorisation from the run's own reviewers. Anything that is not a revert + still waits for a person. - **Don't** put a close-keyword next to an issue `#N` you're only partially resolving — *even negated* (`does not close #N` still closes it). For partial-slice PRs in a chain, use a non-keyword verb and verify `#N` stays @@ -524,6 +686,10 @@ By morning the user should have: polishing an existing deliverable. - `overnight-insight-discovery` — sister overnight pattern for surfacing ah-ha findings from data. +- `overnight-review-panel-blocked-reviewer-reads-as-clean` — the other half + of the review-integrity pair: a reviewer that could not see the code reads + as a clean one (this skill's "never truncate a findings payload" covers the + reviewer that saw and could not be heard). - `gh-pr-merge-worktree-checkout-trap` — handles the "merge succeeded but local cleanup failed" gotcha. - `stacked-pr-base-branch-deletion-auto-closes-dependent` — handles the @@ -551,3 +717,26 @@ PR-level reviews = ~38 dispatches. Lessons fed back: 1 new global skill (`stacked-pr-base-branch-deletion- auto-closes-dependent`), 1 new project feedback (always run code-reviewer pass before merging PRs), this skill. + +## Worked example — 2026-08-05 overnight run ("the observed run") + +17 items across 4 orchestrated workflows, 91 subagents, ~11 hours, 20 PRs +merged by morning, on a repo with no CI and no branch protection. This is the +run every "observed run" above refers to. What it cost, in the order the +lessons appear: + +- One of six critical findings was lost to a `slice(0, 9000)` on the reviews + payload, and was recovered only because the merging agent said the payload + looked cut → "Never truncate a findings payload". +- Five of the twenty merged PRs shipped a fresh instance of the defect they + repaired; every one was caught by re-deriving a number → the verbatim + reviewer line in the tier rubric. +- The published test counts moved under the run (`server 269` → `314`) and an + item quoted its own brief → the stale-baseline rule. +- A parallel session read the shared board, found two of its three assigned + tasks already claimed, and correctly did only the third → "Coordinating with + sessions you do not control". +- A mid-flight correction went into the plan file rather than the + orchestration script, so the resume replayed eight finished PRs from cache + instead of rebuilding them → "Amend a running orchestration through a file + on disk".