From 84cdea42435ec319dddfa1727608f4cb4dc04098 Mon Sep 17 00:00:00 2001 From: Jammy2211 Date: Wed, 8 Jul 2026 11:22:16 +0100 Subject: [PATCH] Stub CLAUDE.md; move internals to docs/internals.md MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit CLAUDE.md is auto-injected into agent context on first file read in this repo — at ~230 lines it was the largest involuntary context load in the organism. It is now a 9-line stub pointing at AGENTS.md (matching PyAutoBrain/PyAutoHeart); the internals (autobuild CLI, pre-build steps, workspace structure, configs, release.yml) move to docs/internals.md, read on demand. The never-rewrite-history safety block moves to AGENTS.md so it stays in the always-read path. Co-Authored-By: Claude Fable 5 --- AGENTS.md | 26 ++++++- CLAUDE.md | 194 +++------------------------------------------- docs/internals.md | 169 ++++++++++++++++++++++++++++++++++++++++ 3 files changed, 202 insertions(+), 187 deletions(-) create mode 100644 docs/internals.md diff --git a/AGENTS.md b/AGENTS.md index 3b25e7e..d081d3d 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -37,5 +37,27 @@ tick|fix` are thin shims that delegate to `pyauto-heart`. Build keeps only the executor primitives (`pre_build`, `generate`, `run*`, `tag_and_merge`, `bump_colab_urls`, `release.yml`). -See [`CLAUDE.md`](CLAUDE.md) for the build pipeline, workspace folder structure, -config files, and `release.yml` details. +See [`docs/internals.md`](docs/internals.md) for the build pipeline, workspace +folder structure, config files, and `release.yml` details. Read it when +changing the pipeline itself, not by default. + +## Never rewrite history + +NEVER perform these operations on any repo with a remote: + +- `git init` in a directory already tracked by git +- `rm -rf .git && git init` +- Commit with subject "Initial commit", "Fresh start", "Start fresh", "Reset + for AI workflow", or any equivalent message on a branch with a remote +- `git push --force` to `main` (or any branch tracked as `origin/HEAD`) +- `git filter-repo` / `git filter-branch` on shared branches +- `git rebase -i` rewriting commits already pushed to a shared branch + +If the working tree needs a clean state, the **only** correct sequence is: + + git fetch origin + git reset --hard origin/main + git clean -fd + +This applies equally to humans, local Claude Code, cloud Claude agents, Codex, +and any other agent. diff --git a/CLAUDE.md b/CLAUDE.md index 42616ee..66cfd49 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -1,185 +1,9 @@ -# CLAUDE.md - -This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository. - -## What This Project Does - -PyAutoBuild is the **executor** of the PyAuto release ecosystem (PyAutoConf, PyAutoFit, PyAutoArray, PyAutoGalaxy, PyAutoLens) — it runs **no** release-readiness checks of its own (that is PyAutoHeart's job; see [`AGENTS.md`](AGENTS.md) for the canonical Build/Heart/Brain boundary and the `Brain → Heart → Build` call chain). It automates: -1. Building and releasing packages to TestPyPI, then PyPI -2. Running workspace Python scripts (integration tests) -3. Converting Python scripts to Jupyter notebooks and executing them -4. Committing generated notebooks to workspace `main` branches and tagging each workspace with a version matching the released library - -The pipeline is triggered via GitHub Actions (`release.yml`) and is manually dispatched with configurable options. Release-readiness gating happens upstream: the PyAutoBrain release agent calls `pyauto-heart readiness` and only dispatches `release.yml` on a green verdict. - -## Bash CLI - -Every operation in this repo is invokable from the shell via the `autobuild` dispatcher at `bin/autobuild`. List subcommands with `autobuild help`; print the docstring for one with `autobuild help ` (or `autobuild --help`). - -Recommended alias for `~/.bashrc`: - -```bash -alias autobuild-help='$HOME/Code/PyAutoLabs/PyAutoBuild/bin/autobuild help' -``` - -The dispatcher routes to the underlying bash script directly, or to the Python tool with `PYTHONPATH` already set so the internal `build_util` / `result_collector` / `env_config` imports resolve. The same operations remain callable as Claude skills (`/pre_build`, `/verify_install`, `/review_release`); use the skill when you want the validation + summary wrapper, the CLI when you just want to fire the underlying tool. - -## Pre-Build Steps - -Before triggering a build, run: - -```bash -bash $HOME/Code/PyAutoLabs/PyAutoBuild/bin/autobuild pre_build [minor_version] -# minor_version defaults to 1 -# (equivalent to: bash $HOME/Code/PyAutoLabs/PyAutoBuild/pre_build.sh [minor_version]) -``` - -This script does the following for each repo: - -| Repo | black | generate.py | commit & push | -|------|-------|-------------|---------------| -| `autofit_workspace` | yes | yes (`autofit`) | yes | -| `autogalaxy_workspace` | yes | yes (`autogalaxy`) | yes | -| `autolens_workspace` | yes | yes (`autolens`) | yes | -| `autofit_workspace_test` | yes | no | yes | -| `autogalaxy_workspace_test` | yes | no | yes | -| `autolens_workspace_test` | yes | no | yes | -| `euclid_strong_lens_modeling_pipeline` | yes | no | yes | -| `HowToGalaxy` | yes | yes (`howtogalaxy`) | yes | -| `HowToLens` | yes | yes (`howtolens`) | yes | -| `HowToFit` | yes | yes (`howtofit`) | yes | - -Before the per-repo loop, `pre_build.sh` invokes `admin_jammy/software/ensure_workspace_labels.sh` to assert the canonical `pending-release` label across every release-window repo (idempotent — a no-op when nothing has drifted). - -Release-readiness checking is **not** Build's job — PyAutoBuild is a pure executor. The version-skew check that used to live here (`verify_workspace_versions.sh`, a fail-fast guard against a workspace pinned ahead of its installed library, or a `config/general.yaml` ↔ `version.txt` disagreement) now lives in **PyAutoHeart** as the `version_skew` check feeding `pyauto-heart readiness`. The PyAutoBrain release agent gates on `pyauto-heart readiness` before invoking `pre_build`; a human running `pre_build` directly is trusted to have checked readiness first. See PyAutoHeart for the resolution precedence (`config/general.yaml:version.workspace_version`, then `version.txt`) — identical to `autoconf.workspace.check_version`, so workspace/library mismatches still surface on every script run via each library's `__init__.py`. - -`generate.py` is run from the workspace root with `PYTHONPATH` pointing at `PyAutoBuild/autobuild/`. Only specific safe directories are committed — never `output/`, `output_model/`, or run-generated artefacts. After all workspaces are done, PyAutoBuild itself is committed and pushed, then `gh workflow run release.yml` dispatches the GitHub Actions release. - -## Workspace Folder Structure - -Each workspace repo (`autofit_workspace`, `autogalaxy_workspace`, `autolens_workspace`, their `_test` variants, and the lecture repos `HowToGalaxy`/`HowToLens`) has the following expected structure. **Only these paths should ever be committed.** - -| Folder / file | autofit | autogalaxy | autolens | Notes | -|---|---|---|---|---| -| `config/` | yes | yes | yes | PyAutoConf config files | -| `dataset/` | yes | yes | yes | Input data; force-added with `git add -f` | -| `notebooks/` | yes | yes | yes | Generated from `scripts/` by `generate.py` | -| `scripts/` | yes | yes | yes | Source Python scripts | -| `slam_pipeline/` | no | no | yes | autolens only | -| `output/` | — | — | — | **Always empty** — kept under git with a `.gitignore` only | -| Root-level files | yes | yes | yes | `README.md`, `setup.py`, `pyproject.toml`, `requirements.txt`, `*.cfg`, `*.ini`, `*.yml`, `*.yaml`, `LICENSE*` | - -### Paths that must NEVER be committed - -- `output/` contents — run results; the folder itself exists only via `.gitignore` -- `output_model/` — model JSON/pickle artefacts written during script execution -- `path/to/model/` or any nested model JSON files written at runtime -- `.fits` files outside `dataset/` (e.g. `image.fits`, `dataset.fits` generated by simulators into `scripts/` or other subdirectories) - -## Running Tests - -```bash -# Run all tests -pytest - -# Run a single test -pytest tests/test_files_to_run.py::test_script_order -``` - -## Codex / sandboxed runs - -When running Python from Codex or any restricted environment, set writable cache directories so `numba` and `matplotlib` do not fail on unwritable home or source-tree paths: - -```bash -NUMBA_CACHE_DIR=/tmp/numba_cache MPLCONFIGDIR=/tmp/matplotlib pytest -``` - -This workspace is often imported from `/mnt/c/...` and Codex may not be able to write to module `__pycache__` directories or `/home/jammy/.cache`, which can cause import-time `numba` caching failures without this override. - -## Key Scripts - -All scripts in `autobuild/` are run from within a checked-out workspace directory (not from this repo root). They rely on `PYTHONPATH` including the PyAutoBuild directory. - -- **`run_python.py `** — Executes Python scripts in a workspace folder, skipping files listed in `config/no_run.yaml` -- **`run.py [--visualise]`** — Executes Jupyter notebooks in a workspace folder, skipping files in `config/no_run.yaml` -- **`generate.py `** — Converts Python scripts in `scripts/` to `.ipynb` notebooks in `notebooks/`, run from within the workspace root -- **`script_matrix.py [project2 ...]`** — Outputs a JSON matrix of `{name, directory}` pairs for GitHub Actions matrix strategy -- **`tag_and_merge.sh --version `** — Commits pending changes and tags library repos (PyAutoConf, PyAutoFit, PyAutoArray, PyAutoGalaxy, PyAutoLens) for release -- **`url_check`** — URL hygiene moved to PyAutoHeart (Heart owns all health checking). `autobuild url_check` is now a thin shim to `pyauto-heart url_check`; the ecosystem-wide sweep runs from PyAutoHeart's central `url-check.yml` workflow (replacing the old per-repo `url_check.yml` workflows). The runnable scripts live at `PyAutoHeart/heart/checks/url_check*.{sh,py}`. -- **`bump_colab_urls.sh `** — Rewrites every `colab.research.google.com/github/PyAutoLabs//blob//...` URL in cwd to use ``, where `` is one of `autofit_workspace`, `autogalaxy_workspace`, `autolens_workspace`, `HowToGalaxy`, `HowToLens`. Called by the `release_workspaces` and `bump_library_colab_urls` jobs in `release.yml` so README/docs Colab links always pin to the just-released tag. Idempotent; skips URLs not in canonical PyAutoLabs/date-tagged form. - -## Architecture - -### Script-to-Notebook Conversion Pipeline - -`generate.py` → `generate_autofit.py` + `build_util.py`: -1. `add_notebook_quotes.py` transforms triple-quoted docstrings into `# %%` cell markers in a temp `.py` file -2. `ipynb-py-convert` converts the temp file to `.ipynb` -3. `build_util.uncomment_jupyter_magic()` restores commented-out Jupyter magic commands (e.g. `# %matplotlib` → `%matplotlib`) -4. Generated notebooks are `git add -f`ed directly - -### Script Execution Order - -`build_util.find_scripts_in_folder()` enforces a specific ordering: -1. Scripts with "simulator" in the path (data must be generated first) -2. Scripts named `start_here.py` -3. All other scripts - -### Config Files - -Each workspace owns its own build config under `/config/build/`: - -- **`no_run.yaml`** — flat list of script/notebook patterns to skip during execution -- **`env_vars.yaml`** — defaults + per-pattern overrides for environment variables -- **`copy_files.yaml`** — flat list of script paths to copy as-is to `notebooks/` instead of converting -- **`visualise_notebooks.yaml`** — flat list of notebook stems to run when `--visualise` flag is used - -`autobuild/config/` retains keyed-dict copies of `no_run.yaml`, `copy_files.yaml`, and `visualise_notebooks.yaml` as fallbacks for legacy workspaces (HowTo*, BSc_Galaxies_Project) that have not been migrated yet. The 6 main workspaces (autofit/autogalaxy/autolens and their `_test` variants) own their own configs and do not consult these fallbacks. - -### Environment Variables - -- `BUILD_PYTHON_INTERPRETER` — Python interpreter to use for script execution (defaults to `python3`) -- `PYAUTO_TEST_MODE` — Set to `1` for workspace runs, `0` for `*_test` workspace runs -- `PYAUTO_SMALL_DATASETS` — Set to `1` for workspace runs (caps grids to 15x15), not set for `*_test` runs -- `PYAUTO_FAST_PLOTS` — Set to `1` for workspace runs (skips `tight_layout()` in subplots and critical curve/caustic overlays in plots), not set for `*_test` runs -- `JAX_ENABLE_X64` — Set to `True` during CI runs - -### GitHub Actions Workflow - -The workflow (`release.yml`) is manually dispatched with inputs: -- `minor_version` — appended to date-based version (format: `YYYY.M.D.minor`) -- `rehearsal` — the one mode switch (default `false` = full real release). When `true`, it is - TestPyPI-only rehearsal mode: build every package from source, publish to TestPyPI, emit the - resolved version as the `testpypi-rehearsal-version` artifact, then STOP (no PyPI upload, no git - tag, no notebook/version commits, no Colab bumps). This is the mode the Heart/Brain - release-validation gate dispatches so it can install and validate the built wheels. - -(The legacy `skip_scripts` / `skip_notebooks` / `skip_release` force-through knobs and the -`update_notebook_visualisations` path were removed with the Heart/Build split — Build is a pure -executor with no ad-hoc skip levers or inline notebook-visualisation job. "Build without -releasing" is now exactly what `rehearsal` mode is for.) - -`release.yml` is a **pure executor**: it builds, tests-the-install, publishes to PyPI, and commits generated notebooks + version pins to the workspaces. Workspace-integration validation (the old `find_scripts` / `generate_notebooks` / `run_scripts` / `run_notebooks` / `analyze_results` jobs) moved to **PyAutoHeart**'s `workspace-validation.yml`; release readiness is gated upstream by the PyAutoBrain release agent via `pyauto-heart readiness` before this workflow is dispatched. The `script_matrix.py` / `run_python.py` / `run.py` / `aggregate_results.py` primitives remain here and are checked out + reused by the Heart workflow. -## Never rewrite history - -NEVER perform these operations on any repo with a remote: - -- `git init` in a directory already tracked by git -- `rm -rf .git && git init` -- Commit with subject "Initial commit", "Fresh start", "Start fresh", "Reset - for AI workflow", or any equivalent message on a branch with a remote -- `git push --force` to `main` (or any branch tracked as `origin/HEAD`) -- `git filter-repo` / `git filter-branch` on shared branches -- `git rebase -i` rewriting commits already pushed to a shared branch - -If the working tree needs a clean state, the **only** correct sequence is: - - git fetch origin - git reset --hard origin/main - git clean -fd - -This applies equally to humans, local Claude Code, cloud Claude agents, Codex, -and any other agent. The "Initial commit — fresh start for AI workflow" pattern -that appeared independently on origin and local for three workspace repos is -exactly what this rule prevents — it costs ~40 commits of redundant local work -every time it happens. +# PyAutoBuild — Claude guidance + +Read [`AGENTS.md`](AGENTS.md) in this directory. It is the shared source of +truth for PyAutoBuild — the executor (Hands) of the PyAuto organism — and for +the Brain / Heart / Build boundary. + +Internals (the `autobuild` CLI, pre-build steps, workspace folder structure, +config files, `release.yml`) are in [`docs/internals.md`](docs/internals.md) — +read it when working on the build pipeline itself, not by default. diff --git a/docs/internals.md b/docs/internals.md new file mode 100644 index 0000000..d8eec32 --- /dev/null +++ b/docs/internals.md @@ -0,0 +1,169 @@ +# PyAutoBuild — internals + +Operational detail for working **inside** this repo: the `autobuild` CLI, the +pre-build steps, workspace folder structure, config files, and `release.yml`. +What PyAutoBuild *is* and the Brain/Heart/Build boundary live in +[`AGENTS.md`](../AGENTS.md) — read that first; read this only when changing the +build pipeline itself. + +## What the pipeline automates + +PyAutoBuild runs **no** release-readiness checks of its own (that is +PyAutoHeart's job). It automates: +1. Building and releasing packages to TestPyPI, then PyPI +2. Running workspace Python scripts (integration tests) +3. Converting Python scripts to Jupyter notebooks and executing them +4. Committing generated notebooks to workspace `main` branches and tagging each workspace with a version matching the released library + +The pipeline is triggered via GitHub Actions (`release.yml`) and is manually dispatched with configurable options. Release-readiness gating happens upstream: the PyAutoBrain release agent calls `pyauto-heart readiness` and only dispatches `release.yml` on a green verdict. + +## Bash CLI + +Every operation in this repo is invokable from the shell via the `autobuild` dispatcher at `bin/autobuild`. List subcommands with `autobuild help`; print the docstring for one with `autobuild help ` (or `autobuild --help`). + +Recommended alias for `~/.bashrc`: + +```bash +alias autobuild-help='$HOME/Code/PyAutoLabs/PyAutoBuild/bin/autobuild help' +``` + +The dispatcher routes to the underlying bash script directly, or to the Python tool with `PYTHONPATH` already set so the internal `build_util` / `result_collector` / `env_config` imports resolve. The same operations remain callable as Claude skills (`/pre_build`, `/verify_install`, `/review_release`); use the skill when you want the validation + summary wrapper, the CLI when you just want to fire the underlying tool. + +## Pre-Build Steps + +Before triggering a build, run: + +```bash +bash $HOME/Code/PyAutoLabs/PyAutoBuild/bin/autobuild pre_build [minor_version] +# minor_version defaults to 1 +# (equivalent to: bash $HOME/Code/PyAutoLabs/PyAutoBuild/pre_build.sh [minor_version]) +``` + +This script does the following for each repo: + +| Repo | black | generate.py | commit & push | +|------|-------|-------------|---------------| +| `autofit_workspace` | yes | yes (`autofit`) | yes | +| `autogalaxy_workspace` | yes | yes (`autogalaxy`) | yes | +| `autolens_workspace` | yes | yes (`autolens`) | yes | +| `autofit_workspace_test` | yes | no | yes | +| `autogalaxy_workspace_test` | yes | no | yes | +| `autolens_workspace_test` | yes | no | yes | +| `euclid_strong_lens_modeling_pipeline` | yes | no | yes | +| `HowToGalaxy` | yes | yes (`howtogalaxy`) | yes | +| `HowToLens` | yes | yes (`howtolens`) | yes | +| `HowToFit` | yes | yes (`howtofit`) | yes | + +Before the per-repo loop, `pre_build.sh` invokes `admin_jammy/software/ensure_workspace_labels.sh` to assert the canonical `pending-release` label across every release-window repo (idempotent — a no-op when nothing has drifted). + +Release-readiness checking is **not** Build's job — PyAutoBuild is a pure executor. The version-skew check that used to live here (`verify_workspace_versions.sh`, a fail-fast guard against a workspace pinned ahead of its installed library, or a `config/general.yaml` ↔ `version.txt` disagreement) now lives in **PyAutoHeart** as the `version_skew` check feeding `pyauto-heart readiness`. The PyAutoBrain release agent gates on `pyauto-heart readiness` before invoking `pre_build`; a human running `pre_build` directly is trusted to have checked readiness first. See PyAutoHeart for the resolution precedence (`config/general.yaml:version.workspace_version`, then `version.txt`) — identical to `autoconf.workspace.check_version`, so workspace/library mismatches still surface on every script run via each library's `__init__.py`. + +`generate.py` is run from the workspace root with `PYTHONPATH` pointing at `PyAutoBuild/autobuild/`. Only specific safe directories are committed — never `output/`, `output_model/`, or run-generated artefacts. After all workspaces are done, PyAutoBuild itself is committed and pushed, then `gh workflow run release.yml` dispatches the GitHub Actions release. + +## Workspace Folder Structure + +Each workspace repo (`autofit_workspace`, `autogalaxy_workspace`, `autolens_workspace`, their `_test` variants, and the lecture repos `HowToGalaxy`/`HowToLens`) has the following expected structure. **Only these paths should ever be committed.** + +| Folder / file | autofit | autogalaxy | autolens | Notes | +|---|---|---|---|---| +| `config/` | yes | yes | yes | PyAutoConf config files | +| `dataset/` | yes | yes | yes | Input data; force-added with `git add -f` | +| `notebooks/` | yes | yes | yes | Generated from `scripts/` by `generate.py` | +| `scripts/` | yes | yes | yes | Source Python scripts | +| `slam_pipeline/` | no | no | yes | autolens only | +| `output/` | — | — | — | **Always empty** — kept under git with a `.gitignore` only | +| Root-level files | yes | yes | yes | `README.md`, `setup.py`, `pyproject.toml`, `requirements.txt`, `*.cfg`, `*.ini`, `*.yml`, `*.yaml`, `LICENSE*` | + +### Paths that must NEVER be committed + +- `output/` contents — run results; the folder itself exists only via `.gitignore` +- `output_model/` — model JSON/pickle artefacts written during script execution +- `path/to/model/` or any nested model JSON files written at runtime +- `.fits` files outside `dataset/` (e.g. `image.fits`, `dataset.fits` generated by simulators into `scripts/` or other subdirectories) + +## Running Tests + +```bash +# Run all tests +pytest + +# Run a single test +pytest tests/test_files_to_run.py::test_script_order +``` + +## Codex / sandboxed runs + +When running Python from Codex or any restricted environment, set writable cache directories so `numba` and `matplotlib` do not fail on unwritable home or source-tree paths: + +```bash +NUMBA_CACHE_DIR=/tmp/numba_cache MPLCONFIGDIR=/tmp/matplotlib pytest +``` + +This workspace is often imported from `/mnt/c/...` and Codex may not be able to write to module `__pycache__` directories or `/home/jammy/.cache`, which can cause import-time `numba` caching failures without this override. + +## Key Scripts + +All scripts in `autobuild/` are run from within a checked-out workspace directory (not from this repo root). They rely on `PYTHONPATH` including the PyAutoBuild directory. + +- **`run_python.py `** — Executes Python scripts in a workspace folder, skipping files listed in `config/no_run.yaml` +- **`run.py [--visualise]`** — Executes Jupyter notebooks in a workspace folder, skipping files in `config/no_run.yaml` +- **`generate.py `** — Converts Python scripts in `scripts/` to `.ipynb` notebooks in `notebooks/`, run from within the workspace root +- **`script_matrix.py [project2 ...]`** — Outputs a JSON matrix of `{name, directory}` pairs for GitHub Actions matrix strategy +- **`tag_and_merge.sh --version `** — Commits pending changes and tags library repos (PyAutoConf, PyAutoFit, PyAutoArray, PyAutoGalaxy, PyAutoLens) for release +- **`url_check`** — URL hygiene moved to PyAutoHeart (Heart owns all health checking). `autobuild url_check` is now a thin shim to `pyauto-heart url_check`; the ecosystem-wide sweep runs from PyAutoHeart's central `url-check.yml` workflow (replacing the old per-repo `url_check.yml` workflows). The runnable scripts live at `PyAutoHeart/heart/checks/url_check*.{sh,py}`. +- **`bump_colab_urls.sh `** — Rewrites every `colab.research.google.com/github/PyAutoLabs//blob//...` URL in cwd to use ``, where `` is one of `autofit_workspace`, `autogalaxy_workspace`, `autolens_workspace`, `HowToGalaxy`, `HowToLens`. Called by the `release_workspaces` and `bump_library_colab_urls` jobs in `release.yml` so README/docs Colab links always pin to the just-released tag. Idempotent; skips URLs not in canonical PyAutoLabs/date-tagged form. + +## Architecture + +### Script-to-Notebook Conversion Pipeline + +`generate.py` → `generate_autofit.py` + `build_util.py`: +1. `add_notebook_quotes.py` transforms triple-quoted docstrings into `# %%` cell markers in a temp `.py` file +2. `ipynb-py-convert` converts the temp file to `.ipynb` +3. `build_util.uncomment_jupyter_magic()` restores commented-out Jupyter magic commands (e.g. `# %matplotlib` → `%matplotlib`) +4. Generated notebooks are `git add -f`ed directly + +### Script Execution Order + +`build_util.find_scripts_in_folder()` enforces a specific ordering: +1. Scripts with "simulator" in the path (data must be generated first) +2. Scripts named `start_here.py` +3. All other scripts + +### Config Files + +Each workspace owns its own build config under `/config/build/`: + +- **`no_run.yaml`** — flat list of script/notebook patterns to skip during execution +- **`env_vars.yaml`** — defaults + per-pattern overrides for environment variables +- **`copy_files.yaml`** — flat list of script paths to copy as-is to `notebooks/` instead of converting +- **`visualise_notebooks.yaml`** — flat list of notebook stems to run when `--visualise` flag is used + +`autobuild/config/` retains keyed-dict copies of `no_run.yaml`, `copy_files.yaml`, and `visualise_notebooks.yaml` as fallbacks for legacy workspaces (HowTo*, BSc_Galaxies_Project) that have not been migrated yet. The 6 main workspaces (autofit/autogalaxy/autolens and their `_test` variants) own their own configs and do not consult these fallbacks. + +### Environment Variables + +- `BUILD_PYTHON_INTERPRETER` — Python interpreter to use for script execution (defaults to `python3`) +- `PYAUTO_TEST_MODE` — Set to `1` for workspace runs, `0` for `*_test` workspace runs +- `PYAUTO_SMALL_DATASETS` — Set to `1` for workspace runs (caps grids to 15x15), not set for `*_test` runs +- `PYAUTO_FAST_PLOTS` — Set to `1` for workspace runs (skips `tight_layout()` in subplots and critical curve/caustic overlays in plots), not set for `*_test` runs +- `JAX_ENABLE_X64` — Set to `True` during CI runs + +### GitHub Actions Workflow + +The workflow (`release.yml`) is manually dispatched with inputs: +- `minor_version` — appended to date-based version (format: `YYYY.M.D.minor`) +- `rehearsal` — the one mode switch (default `false` = full real release). When `true`, it is + TestPyPI-only rehearsal mode: build every package from source, publish to TestPyPI, emit the + resolved version as the `testpypi-rehearsal-version` artifact, then STOP (no PyPI upload, no git + tag, no notebook/version commits, no Colab bumps). This is the mode the Heart/Brain + release-validation gate dispatches so it can install and validate the built wheels. + +(The legacy `skip_scripts` / `skip_notebooks` / `skip_release` force-through knobs and the +`update_notebook_visualisations` path were removed with the Heart/Build split — Build is a pure +executor with no ad-hoc skip levers or inline notebook-visualisation job. "Build without +releasing" is now exactly what `rehearsal` mode is for.) + +`release.yml` is a **pure executor**: it builds, tests-the-install, publishes to PyPI, and commits generated notebooks + version pins to the workspaces. Workspace-integration validation (the old `find_scripts` / `generate_notebooks` / `run_scripts` / `run_notebooks` / `analyze_results` jobs) moved to **PyAutoHeart**'s `workspace-validation.yml`; release readiness is gated upstream by the PyAutoBrain release agent via `pyauto-heart readiness` before this workflow is dispatched. The `script_matrix.py` / `run_python.py` / `run.py` / `aggregate_results.py` primitives remain here and are checked out + reused by the Heart workflow. +The never-rewrite-history rules live in [`AGENTS.md`](../AGENTS.md) and apply +here as everywhere.