feat: CSV API stress-test + extensions; port Lenstool example onto canonical mass/light/point.csv

[Jammy2211]

Overview

User directive (2026-07-09): the flagship Lenstool example must teach the canonical PyAutoLens CSV API rather than bespoke CSVs, and the API itself should be deep-researched / stress-tested against realistic user requirements first — different mass models, time delays and fluxes in point data, member-catalogue geometry — extending it where gaps surface. Follows the review on autolens_workspace#249's thread: data side (point_datasets.csv) is solid; galaxy_table_from_csv silently drops geometry/mag columns; the named-galaxy model CSVs are strong but under-exercised (Lenstool example bypassed them; no dPIEMassLenstool doc row).

Autonomy: supervised --auto (in-session directive); heart-ack carried.

Plan

1. Stress-test (empirical, harness in scratch then distilled into unit tests): push through mass.csv/light.csv/point.csv + galaxy_table — dPIEMassLenstool rows (.par-as-a-table), NFWMCRLudlow (redshift args), elliptical profiles (ell_comps_0/1 and centre y/x tuple splitting), ExternalShear, multipole profiles, linear/operated light profiles (dispatch module coverage), point sources with fluxes, per-row redshifts, af.Model round-trips with priors on selected columns, write→read→write idempotency. Record every failure as a gap.
2. Extend what the stress-test demands (candidates: GalaxyTable optional ellipticity/angle_pos/mag columns replacing today's silent-ignore; dispatch-module coverage for light variants; anything the harness surfaces). Library changes + tests in PyAutoGalaxy, numpy-only.
3. Port the Lenstool example: data.py writes canonical mass.csv (dPIEMassLenstool rows = the .par file as a table) + extended scaling_galaxies.csv; modeling.py reconstruction via al.galaxies_from_csv_tables and refit via al.galaxy_af_models_from_csv_tables — no bespoke readers. Re-verify the 0.068" reconstruction parity after the port.
4. Docs: csv_api.py gains a Lenstool-parameterized row example + the extended table schema; point.csv time-delay/flux columns documented against the data-side round-trip.

Affected Repositories

• PyAutoGalaxy (primary: galaxy_table/galaxy_model_csv extensions + tests)
• autolens_workspace (lenstool example port + csv_api.py docs; parallel to open claims — scripts/cluster/ disjoint; catalogue-regen coordination continues)

Branch: feature/csv-api-lenstool · Worktree: ~/Code/PyAutoLabs-wt/csv-api-lenstool/

Original directive

verbatim

"yep, do it, making sure we focu on using the bespoke PyAutoLens csv API throughout, do a bit of a deep research making sure the autoens .csv API is fully developed for what we need, i like the mass/ligth/point.csv but youre right e may need to extend it to cover all uses (different mass models? time delays in poiunt)? stress test diferent possible user requirements"

[Jammy2211]

Shipped to PR-open: #491 (library extensions) + PyAutoLabs/autolens_workspace#252 (example port, stacked on #250 — merge order: #250 → #491 → #252). Both pending-release ✓.

Stress-test verdict (harness distilled into 12 unit tests): the named-galaxy CSV API held up well — .par-style dPIEMassLenstool rows, NFW redshift args, shear + PowerLaw sparse files, multipoles, PointFlux, and flux/time-delay data columns (via output_to_csv/list_from_csv) all pass unmodified. Six gaps found, five real, all fixed:

1. Linear/operated light variants rejected → qualified class names (linear.Sersic).
2. Typo'd parameter columns silently ignored (profile silently at default!) → loud ValueError.
3. Duplicate (galaxy, attr_name) rows silently overwrote → loud ValueError.
4. GalaxyTable dropped extra catalogue columns silently → properties (floats/strings).
5. dPIEMassLenstool hardcoded Planck15 (uncovered by the port — a Lenstool run's cosmology couldn't ride in a CSV) → flat H0/Om0 args, prior-configured.
   (6th was a wrong function name in the harness, not a gap.)

The payoff: the flagship example's data.py now writes the entire published model as one canonical mass.csv — 149 rows whose header is literally the .par vocabulary — and modeling.py reads it back with the same three calls every cluster script uses. Parity re-verified after the port: 0.0680″ median source-plane rms, bit-identical to pre-port (a first attempt in the wrong env measured 0.0727″ — which was itself a nice accidental demonstration that the H0/Om0 gap was physically real, ~0.3% in b0). PyAutoGalaxy suite 962 pass.

One decision batched for merge review: the loud-guard behaviour change (typo'd columns now error rather than silently defaulting).

[Jammy2211]

PRs #491 + autolens_workspace#252 merged (stack order honoured) — CSV API extensions live and the Lenstool example teaches the canonical interface end to end at 0.0680" parity. Closing.