feat: PYAUTO_SMALL_DATASETS support for weak lensing catalogues (weak series step 9)

[Jammy2211]

Overview

Step 9 of the weak-lensing series: extend the PYAUTO_SMALL_DATASETS=1 smoke-mode mechanism to catalogue-shaped weak-lensing data so workspace-test smoke runs of scripts/weak/ are fast. Pixel datasets are capped to 15×15 by autoarray/util/dataset_util.py; catalogues have no pixels, so the lever is the galaxy count.

Plan

• PyAutoArray: add SMALL_DATASETS_N_CATALOGUE = 25 and cap_catalogue_size_for_small_datasets(n) to autoarray/util/dataset_util.py (env-var logic stays centralised beside the pixel caps; reusable by other catalogue datasets, e.g. point).
• PyAutoLens: SimulatorShearYX.via_tracer_random_positions_from caps n_galaxies through the new helper (explicit-grid via_tracer_from deliberately uncapped — a user-provided grid is never mutated).
• Unit tests in both repos (env-var set/unset via monkeypatch).
• Verification (no workspace commit needed — should_simulate adoption already shipped in feat: tangential/cross shear profile + Kaiser-Squires map (weak series step 6) #581/Feature/remove settings pixelization #244): run scripts/weak/modeling.py under PYAUTO_SMALL_DATASETS=1 and confirm a 25-galaxy catalogue regenerates on disk and the fit completes fast.

Details

Branch: feature/weak-small-datasets; worktree ~/Code/PyAutoLabs-wt/weak-small-datasets/ (PyAutoArray + PyAutoLens). Merge autoarray first (upstream).
Gotchas carried from prior work: env-var mutations can bite inside decorators — the weak path uses Grid2DIrregular, untouched by the existing Grid2D.uniform/Mask2D.circular caps, so the simulator cap is the only site. Smoke env vars are wired via PyAutoBuild config/build/env_vars.yaml; no os.environ mutation in scripts.
Autonomy: --auto continuation 2026-07-09, effective supervised (header safe, feature cap) — plan-to-issue, park at ship sign-off.

[Jammy2211]

Ship sign-off requested (supervised checkpoint — run parked here)

What was done (committed locally on feature/weak-small-datasets, not pushed):

• PyAutoArray 2bf9ad37: SMALL_DATASETS_N_CATALOGUE = 25 + cap_catalogue_size_for_small_datasets(n) in dataset_util (+2 env-var tests). +22 lines, additive.
• PyAutoLens 6b29d3bf: via_tracer_random_positions_from routes n_galaxies through the new helper; explicit-grid via_tracer_from never mutated (+3 tests incl. that invariant). +6 lines in the simulator.

Validation. Full suites green: autoarray 869 passed, autolens 356 passed. End-to-end: scripts/weak/modeling.py under PYAUTO_SMALL_DATASETS=1 PYAUTO_TEST_MODE=1 deleted-and-regenerated a 25-galaxy catalogue via the already-shipped should_simulate path and completed the full Nautilus fit in 30 s (vs ~7 min unsmoked). Canonical dataset restored afterwards; no workspace commit needed.

Heart: YELLOW, same acknowledged reason set. Merge order: autoarray first (upstream), then autolens.

The question. Approve shipping? On approval: push both, open both pending-release PRs, merge autoarray-first on green CI, close this issue. This also makes scripts/weak/ smoke-able for workspace_test coverage (smoke-list additions stay curated/human per standing policy — none proposed here).

[Jammy2211]

Shipped ✅ — PyAutoArray#366 and PyAutoLens#584 merged upstream-first (the initial #584 CI red was the expected cross-repo ordering: the dependency-branch checkout didn't pick up the autoarray branch, and the re-run against merged autoarray main was green). scripts/weak/ is now smoke-able: modeling.py at 25 galaxies completes in ~30 s under PYAUTO_SMALL_DATASETS=1 + PYAUTO_TEST_MODE=1. Closing.