feature: nightly live releases behind an activity gate (design)

[Jammy2211]

Overview

Move from ad-hoc manual releases to unattended nightly live PyPI releases behind two gates: an activity gate (skip entirely on quiet days, loudly) and the unchanged Heart GREEN release gate (stop and page on anything not GREEN). Decided 2026-07-09: once the next manual release succeeds, nightly runs ship for real with no per-release human approval — the human's role becomes a kill switch, paging response, and the morning digest. This is phase 1 of two: a design note reviewed by the human plus the dated PyAutoBrain/AUTONOMY.md doctrine edit that scopes the standing grant. Phase 2 (implementation) gets its own prompt once the design is approved.

Plan

• Write a design note settling the nightly-release architecture: orchestration shape, activity-gate definition, Heart-GREEN acquisition in CI, loud skip/stop notifications, kill switch, version scheme, rehearsal/live composition, and the arming checklist.
• Draft the dated PyAutoBrain/AUTONOMY.md edit: the scheduled-nightly path (activity-gated, Heart-GREEN-gated, kill-switchable) becomes a human-pre-authorised standing grant; manual and agent-initiated releases stay human-required.
• Ship both as docs-only PRs; the PyAutoBuild PR review is the phase-1 human design review.
• Phase 2 implements the reviewed design in release.yml (gate jobs, notifications, kill switch) as a follow-up prompt.

Detailed implementation plan

Affected Repositories

• PyAutoBuild (primary — design note)
• PyAutoBrain (AUTONOMY.md doctrine edit)

Branch Survey

Repository	Current Branch	Dirty?
./PyAutoBuild	main	clean
./PyAutoBrain	main	clean
./PyAutoHeart	main	clean (not edited in phase 1)

Suggested branch: feature/nightly-release-activity-gate

Grounding (what already exists)

• PyAutoBuild/.github/workflows/release.yml already has the cron (0 2 * * 1-5); scheduled runs rehearse (TestPyPI-only) until repo var RELEASE_MODE=live (resolve_mode job). Scheduled versions auto-derive YYYY.M.D.1; run_attempt suffixes re-runs; live releases no longer commit version stamps back to library repos (research: version-pinning design review — exact pins vs daily release #118/feat: release stamping slim — wheels+tags only, no daily bump commits (R3-core) #120 fix), so pipeline self-commits on main are limited to notebook-regen / Colab-bump / API-baseline commits, all authored GitHub Actions bot with Release <version>: prefixes.
• announce_release already posts live success/failure to Slack via PYAUTO_RELEASE_WEBHOOK_URL.
• PyAutoHeart/heart/readiness.py is the authoritative gate; releases require GREEN, which requires a fresh passing release-validation report matching current main — how a 2 AM CI run obtains that verdict is the central design question. Heart is observer-only (no foreign dispatch), so the gate cannot live in heart-health.yml.
• Known blocker to arming live: PyAutoBuild#126 (pipeline commits smoke-regenerated workspace datasets) is open.

Implementation Steps

1. PyAutoBuild/docs/nightly_release_design.md (new) — sections:
   • Orchestration shape — recommended: gate jobs (activity_gate, heart_gate) at the head of release.yml's scheduled path; rejected alternative (external Brain-driven dispatcher) documented with rationale.
   • Activity gate — merged-to-main commits in the last 24 h across the 11 release-relevant repos (5 libraries, autofit/autogalaxy/autolens workspaces, HowToFit/Galaxy/Lens) via GitHub API; exclude pipeline self-commits (bot author + Release *: prefix) so a release never triggers the next; quiet night → skip downstream, report "no activity, skipped".
   • Heart GREEN gate — verdict acquisition design (evaluate pyauto-heart readiness --json over Heart's published state snapshot with an explicit freshness rule, vs rehearsal→validation-first); non-GREEN → stop + page; covers known-red items like bug: release pipeline commits 15×15 smoke-regenerated workspace datasets (30/49 at 2026.7.6.649) #126.
   • Notifications — three scheduled-run outcomes through the existing webhook: shipped / skipped—no activity / stopped—not GREEN (paging). Silence always means the pipeline broke.
   • Kill switch — repo var NIGHTLY_RELEASES, independent of RELEASE_MODE.
   • Version scheme — date-derived YYYY.M.D.1 on schedule (already implemented), same-day re-run semantics, pre_build minor-version ask automated on this path only.
   • Rehearsal/live composition + arming checklist — next manual live release succeeds → bug: release pipeline commits 15×15 smoke-regenerated workspace datasets (30/49 at 2026.7.6.649) #126 fixed → RELEASE_MODE=live + NIGHTLY_RELEASES=true.
   • Phase 2 sketch — concrete release.yml job diffs so implementation is mechanical.
2. PyAutoBrain/AUTONOMY.md — dated edit to the caps table release row + hard invariants: the scheduled-nightly path is a scoped, human-pre-authorised standing grant; everything else stays human-required; cross-reference the design note.

Key Files

• PyAutoBuild/.github/workflows/release.yml — existing cron/rehearsal/announce plumbing the design extends (read-only in phase 1)
• PyAutoBuild/docs/nightly_release_design.md — new design note (the deliverable)
• PyAutoBrain/AUTONOMY.md — caps table + hard invariants doctrine edit
• PyAutoHeart/heart/readiness.py — verdict semantics the design consumes (read-only)

Verification

Docs-only: markdown renders, AUTONOMY.md caps table stays consistent with its consumers, cross-links resolve. No workflow/code behaviour changes in phase 1.

Original Prompt

Click to expand starting prompt

Nightly releases with an activity gate

Type: feature
Target: pyautobuild
Difficulty: large
Autonomy: human-required
Priority: high
Status: formalised

Nightly/routine releases with an activity gate: switch from ad-hoc manual
releases to a scheduled dispatch of the release pipeline (cron in
release.yml or a Heart/Brain-driven routine). The nightly run must first
judge whether any work merged in the past 24h (commits/merged PRs across the
release-relevant repos — libraries, workspaces, HowTo) and bypass the release
entirely on quiet days, reporting "no activity, skipped" rather than cutting
an identical version. Releases still require Heart GREEN, and a scheduled
release that would ship a known-red item should stop and page the human
instead. Design first, then implement behind human review.

Design constraints (from the 2026-07-09 session that conceived this)

• DECIDED endpoint (user, 2026-07-09): once the next manual release
  succeeds, nightly runs perform FULL LIVE PyPI RELEASES unattended — no
  per-release human approval. This is a deliberate, scoped exception to the
  autonomy contract's "release is always human-required" invariant, so phase 1
  must include the dated doctrine edit to PyAutoBrain/AUTONOMY.md: the
  scheduled-nightly path (activity-gated, Heart-GREEN-gated) is
  human-pre-authorised as a standing grant; manual and agent-initiated
  releases stay human-required. The human's role moves to: a kill switch
  (e.g. a NIGHTLY_RELEASES repo var to pause the schedule), paging on any
  red/anomaly, and reviewing the morning digest of what shipped. pre_build's
  human minor-version ask must be automated on this path (date scheme derives
  it).
• Activity gate: "work merged in the past 24h" needs a concrete definition
  — merged PRs / pushed commits to main across the 5 libraries, 3
  workspaces, 3 HowTo repos (the release-relevant set from repos.yaml).
  Version-stamp/notebook-regeneration commits made by the release pipeline
  itself must not count as activity (else every release triggers the next).
• Heart GREEN is the gate, unchanged — the nightly path must call the same
  pyauto-heart readiness release gate as a manual release; STALE/YELLOW do
  not pass for releases. A known-red item (e.g. an open release-blocking
  issue like PyAutoBuild#126) stops the run and notifies (reuse the morning
  Slack webhook plumbing for paging).
• Skip must be loud: a bypassed night reports "no activity, skipped"
  through the same notification channel, so silence always means something
  broke.
• Version scheme: nightly cadence fits the date-based YYYY.M.D.B scheme;
  the design should cover the build-number choice and what happens on
  same-day re-runs.
• Interactions to cover: rehearsal vs live (RELEASE_MODE, rehearsal
  input), TestPyPI rehearsal as the nightly's dry-run mode option, and how
  this composes with pre_build (which currently asks a human for the minor
  version).

Phase 1 = design note reviewed by the human (this prompt's deliverable);
phase 2 = implementation behind that reviewed design.

[Jammy2211]

Phase 1 is up for review — and the review of these PRs is the phase-1 design review:

• Design note: docs: nightly release design — activity gate + unattended live releases (phase 1) #128 — docs/nightly_release_design.md
• Doctrine edit: docs: AUTONOMY.md — dated standing grant for the scheduled-nightly release path PyAutoBrain#62 — the dated AUTONOMY.md standing grant

Two things worth flagging from the grounding pass:

1. The nightly already half-exists: release.yml has carried a 0 2 * * 1-5 cron since the version-pinning redesign, rehearsing until RELEASE_MODE=live. That's also a hazard — flipping the var today would make the cron release live with no activity or Heart gate. The design removes the cron from release.yml in phase 2 and moves scheduling authority to a Brain driver.
2. The plan's tentative orchestration lean flipped during design: the approved plan leaned toward gate jobs inside release.yml; the design instead recommends a scheduled Brain driver composing the existing M1–M4 release-validation machinery (release validate Stages 0–3 → readiness GREEN → dispatch live). Rationale in §2 of the note: Build is a pure executor, and release.yml can't run the validate choreography of itself — any honest gate needs the Brain orchestration to exist anyway.

Open questions for the reviewer are in §11 (schedule days, activity-window anchor, release-blocker label mechanism). Shipped through an acknowledged Heart YELLOW (reasons unrelated to this docs-only diff). Phase 2 gets its own prompt once the design is approved — nothing implemented yet.

[Jammy2211]

Phase 1 complete — design approved and merged (human-directed): #128 and PyAutoLabs/PyAutoBrain#62 are both in main. docs/nightly_release_design.md is now the reviewed design of record, and AUTONOMY.md carries the dated standing grant.

Phase-1 worktree and feature branches cleaned up. Phase 2 (implementation behind the reviewed design) is being filed as its own prompt now: Brain nightly driver (nightly.sh + nightly-release.yml), removal of the release.yml cron (closing the ungated-live hazard), the Heart release-gate CI evidence profile, the release-blocker label (applied to #126), and the morning-digest missing-outcome flag. This issue stays open as the tracker until phase 2 ships.

[Jammy2211]

Nightly release implementation — Phase 2 (build the reviewed design)

Context

Phase 1 merged 2026-07-09: PyAutoBuild/docs/nightly_release_design.md (PR #128) is the approved design of record; PyAutoBrain/AUTONOMY.md (PR #62) carries the dated standing grant. This phase implements design §10 exactly; any forced deviation updates the note in the same PR and is called out for review. Prompt: PyAutoMind/feature/pyautobuild/nightly_release_implementation.md (feature/pyautobuild, large, human-required). Tracker: reuse PyAutoBuild#127 (it holds the whole review trail) rather than filing a new issue — the approved plan gets posted there.

Branch feature/nightly-release-implementation; worktree ~/Code/PyAutoLabs-wt/nightly-release-implementation/ with PyAutoBrain + PyAutoBuild + PyAutoHeart. The small PyAutoMind digest edit rides a plain branch on the canonical checkout (Mind's main takes constant registry pushes; no worktree claim needed). All four repos surveyed clean on main, no conflicts.

Deliverables

1. PyAutoBrain — the nightly driver (the bulk)

• agents/conductors/release/nightly.sh (new): deterministic driver for design §3 steps 0–7:
  • Step 0 kill switch: workflow passes vars.NIGHTLY_RELEASES in as env; anything ≠ true → exit 0 silently.
  • Step 1 same-day guard: today's YYYY.M.D.* non-dev tag on PyAutoLens via gh api → notify already released today, skipped.
  • Step 2 activity gate: window-end anchor persisted as a repo Actions variable NIGHTLY_LAST_WINDOW_END (simpler and more durable than cache artifacts; written by the driver at each outcome); gh api /repos/{o}/{r}/commits?sha=main&since=<anchor> across the 11 release-relevant repos; exclusion filter per design §4 (^Release [0-9]{4}\. message OR pipeline bot identity). Filter logic lives in agents/conductors/release/activity_gate.py (pure functions: parse, filter, summarise) so it is unit-testable.
  • Step 3 known-red: open issues labelled release-blocker across the 11 repos + PyAutoBuild/PyAutoHeart → page and stop.
  • Step 4 validate: sequence the existing validate.sh Phase A → run the emitted Stage-2 dispatch plan (gh workflow run release.yml -f rehearsal=true, poll, download artifacts) → Phase B --stage3-plan → run the Stage-3 dispatch (workspace-validation.yml mode=release) → Phase C ingest (exit codes already defined: 0 green / 2 yellow / 3 red / 4 unknown). No logic duplicated from validate.sh.
  • Step 5 gate: pyauto-heart readiness --json --profile release-ci (new Heart profile, below) must be GREEN; anything else → page with reasons.
  • Step 6 live dispatch: gh workflow run release.yml -f rehearsal=false -f minor_version=1, poll to completion; dry_run=true (workflow input, default true) replaces this step with a log + notify line.
  • Step 7 + all stops: one Slack message per terminal outcome (shipped / skipped / page) via PYAUTO_RELEASE_WEBHOOK_URL — a small notify() helper, message texts per design §6.
• .github/workflows/nightly-release.yml (new): schedule: 0 2 * * * + workflow_dispatch(dry_run: default true); checks out Brain + Heart (for pyauto-heart); env: NIGHTLY_RELEASES, PAT_PYAUTOLABS, PYAUTO_RELEASE_WEBHOOK_URL; runs nightly.sh.
• bin/pyauto-brain: route release nightly → nightly.sh (same pattern as rehearse/validate in release.sh).
• PyAutoBrain/tests/test_activity_gate.py (new — Brain has no tests/ yet; minimal pytest layout): self-commit exclusion, window math, empty-window skip, summary rendering.

2. PyAutoBuild — remove the cron

• release.yml: delete the schedule: trigger and the schedule branch of resolve_mode (dispatch input passthrough stays); update the header comments. No other behaviour change. This closes the standing hazard that flipping RELEASE_MODE=live today releases ungated at 2 AM — after this PR, RELEASE_MODE governs nothing and is noted in the design as retired-in-cleanup.

3. PyAutoHeart — release-gate CI profile

• heart/readiness.py: add --profile release-ci — named required-evidence set (ci_status, open_prs, release-validation report, verify_install); dev-box-local checks (repo_state, worktree_drift, script_timing, local test_run) reported n/a-by-scope (listed in output, never silently green); unknown required evidence stays YELLOW. Default (no flag) behaviour byte-identical to today.
• Extend tests/test_readiness.py: profile GREEN with local checks absent; YELLOW on missing required evidence; default-path regression.

4. Process legs

• Create the release-blocker label (PyAutoBuild + the 11 release-relevant repos + PyAutoHeart); apply to PyAutoBuild#126.
• PyAutoMind/.github/workflows/morning_health.yml: it already checks release.yml ("Daily release (rehearsal)", 4-day window) — repoint that check at Brain's nightly-release.yml (expect one outcome per night; flag a missing outcome) and reword the release.yml line to dispatch-only. Own small branch + PR on Mind.

What is NOT done here

Nothing is armed: NIGHTLY_RELEASES stays unset, dry_run defaults true. Arming is the human checklist (design §9): next manual live release → #126 fixed → observed GREEN dry-run → arm.

Verification

• Heart: pytest tests/test_readiness.py (+ full Heart suite) in the worktree.
• Brain: pytest tests/test_activity_gate.py; bash -n / shellcheck on nightly.sh; local read-only run pyauto-brain release nightly --dry-run --no-dispatch exercising steps 0–3 against the real GitHub API (read-only) to show the gate verdicts on today's data.
• Build: YAML parses; schedule: absent; workflow_dispatch path unchanged by inspection of resolve_mode.
• End-to-end (post-merge, human-observable): manually dispatch nightly-release.yml with dry_run=true — this is arming-checklist step 3, not a merge blocker.
• Ship via ship_library: 4 PRs (Brain, Build, Heart, Mind), tests/smoke legs stated per repo; Heart gate re-consulted at ship; stop at PR-open (merge is yours).

[Jammy2211]

Phase 2 implemented — four PRs open, stopping at PR-open for your review:

• feat: the scheduled-nightly release driver (release nightly) PyAutoBrain#63 — the nightly driver (release nightly: kill switch → same-day guard → activity gate → release-blocker check → M1–M4 validate choreography → release-ci GREEN gate → live dispatch → one Slack outcome per path) + the 02:00 UTC scheduler workflow (dry_run defaults true)
• feat: remove release.yml's cron — scheduling moves to the Brain nightly driver #129 — release.yml cron removed (the ungated-live hazard is closed on merge); design note §4 updated to the implemented window anchor
• feat: release-ci readiness profile — the nightly gate's CI evidence scope PyAutoHeart#47 — the release-ci readiness profile (dev-box-local gaps → na_reasons, listed not gating; required evidence and adverse signals gate as before); 244 tests pass
• digest: watch the nightly release driver (missing-outcome watchdog) PyAutoMind#43 — morning digest watches the nightly driver at max-age 0d (the missing-outcome watchdog)

Also done outside the PRs: release-blocker label created on the 11 release-relevant repos + PyAutoBuild + PyAutoHeart, and applied to #126 — verified live: a gates-only driver run now stops at step 3 with exit 2 and the paging message naming #126. The activity gate was also verified on live data (24h window correctly quiet; 8-day window found activity in all 11 repos).

Nothing is armed. Scheduled runs exit at the kill switch (NIGHTLY_RELEASES unset) and dry_run defaults true. Arming remains the §9 human checklist: manual live release succeeds → #126 fixed/de-labelled → an observed GREEN dry-run night → set NIGHTLY_RELEASES=true + flip the dry_run default. Merge note: Brain + Build same day (details on the PRs).

[Jammy2211]

All four phase-2 PRs merged (human-directed): PyAutoLabs/PyAutoBrain#63, #129, PyAutoLabs/PyAutoHeart#47, PyAutoLabs/PyAutoMind#43. Worktree and feature branches cleaned up; task retired to Mind complete.md. The nightly system is fully in place and unarmed.

Arming checklist step 1 is now in flight: the manual live release 2026.7.9.1 was dispatched (human-directed, via this session) — release run 29041595906. This is the first live PyPI release since the version-pinning redesign (wheels/tags-only, no version-stamp commits). Outcome will be reported here; announce_release also posts it to Slack.

Remaining before arming: #126 fixed/de-labelled (it does not endanger this release — notebook generation is text conversion, no dataset execution — but it pages every scheduled night while open) → one observed GREEN dry_run=true nightly → set NIGHTLY_RELEASES=true + flip the workflow's dry_run default. Tonight's 02:00 cron will fire and exit at the kill switch — that run is the heartbeat the 06:00 digest now watches.

[Jammy2211]

Post-release actions (human-directed, 2026-07-09 evening):

• fix: assistant API-baseline script path in release.yml #130 merged — the assistant API-baseline path fix (work/ → autoassistant/); wiki_currency_check un-skips at the next release.
• The nightly is ARMED: feat: arm nightly live releases (dry_run default → false) PyAutoBrain#64 merged (dry_run default → false) + NIGHTLY_RELEASES=true set. The human judged the pipeline stable after 2026.7.9.1 and consciously waived the observed-dry-run step. All gates unchanged.
• What tonight (02:00 UTC) will actually do: if the 24h window is quiet (release self-commits are excluded), it reports 💤 skipped; if there is activity, it stops 🚨 at step 3 on bug: release pipeline commits 15×15 smoke-regenerated workspace datasets (30/49 at 2026.7.6.649) #126 (release-blocker). No live nightly release ships until bug: release pipeline commits 15×15 smoke-regenerated workspace datasets (30/49 at 2026.7.6.649) #126 is closed/de-labelled — and bug/autofit_workspace/searches_mcmc_corner_no_dynamic_range.md (filed in Mind) is a blocker in practice too, since the mcmc corner crash fails Stage-3 release-fidelity validation and keeps Heart off GREEN.
• Version-drift audit (post-release): no incoherent drift. PyPI = 2026.7.9.1; tags = 2026.7.9.1 on all 11 repos; Colab URLs (workspaces + library docs) = 2026.7.9.1. Library __version__ on main and workspace version.txt all read 2026.7.6.649 consistently — by design: wheels are stamped at build (never committed back), and version.txt is the compatibility floor (2026.7.9.1 ≥ floor passes), not a version claim. The one cosmetic oddity is autolens_assistant's floor at 2026.5.29.4, which Heart's legacy skew check mislabels "pinned BEHIND" — that's the queued version_skew-prompt redesign, not a repo fix.