Validation: slacs1430+4105 ACS reduction vs trusted legacy dataset

[Jammy2211]

Overview

Validate PyAutoReduce's HST/ACS path on a second SLACS lens: reduce slacs1430+4105 (J1430+4105) from archival ACS/WFC F814W data through the production pipeline and compare the modeling-ready product against the trusted legacy dataset at /mnt/c/Users/Jammy/Science/subhalo/dataset/slacs/slacs1430+4105 (collaborator-provided, used for years in subhalo work). Goes beyond the pixel-level parity of reduce_slacs0008.py (issue #2) by adding lens-model parity fits driven through autolens_assistant — the question is not just "do the pixels match" but "does the science answer change". Methodology settled here carries to the PJ011646 WFC3 follow-up task.

Plan

• Inspect the legacy slacs1430+4105 dataset (real-data gate: plot data/noise/PSF; it ships positions.json, extra_galaxies_centres.json, mask_extra_galaxies.fits — that extra-galaxy treatment is reused identically on both datasets).
• Query MAST at J1430+4105 (RA 217.5171°, Dec +41.0992°) for ACS/WFC F814W exposures; confirm the SLACS proposal ID(s) from the archive rather than assuming slacs0008's 10886.
• Write scripts/reduce_slacs1430.py adapted from scripts/reduce_slacs0008.py; run the production pipeline end-to-end (outputs to gitignored scripts/output/).
• Pixel-level parity: autoreduce.validation.registered_ratios (sub-pixel registration) + side-by-side data / noise / S-N / PSF-profile comparison plots.
• Lens-model parity: identical model, mask, positions and priors fit to both datasets; compare posteriors, residuals and evidence, and check against literature values (θ_E = 1.52", q = 0.68, PA = 111.7°, σ = 322 km/s).
• Verdict on this issue: is the PyAutoReduce product scientifically interchangeable with the legacy reduction? Pipeline defects found escalate as new PyAutoMind prompts (PyAutoReduce source is claimed by frame-products; no source edits here).

Detailed implementation plan

Work classification

Workspace (scripts + analysis; no library source edits). Autonomy: safe effective (--auto launch 2026-07-09; plan approved in-session by the human before launch). The Brain Feature Agent suggested re-homing the prompt as research/; the human explicitly classified it test at intake — kept, dissent recorded here.

Affected repositories

• PyAutoReduce (primary — new scripts/reduce_slacs1430.py + parity-fit script)
• autolens_assistant (driver only: skills, real-data gate, API grounding; no commits — it is a generic public template, scratch outputs go to its gitignored scripts/scratch/)

Branch survey (2026-07-09)

Repository	Current Branch	Dirty?
./PyAutoReduce	main	2 untracked (output/, prototypes/b1938_lens_fit.py — keck-ao task's parked analysis)
./autolens_assistant	main	1 untracked (james.bib)

Suggested branch: feature/slacs1430-acs-parity (created only at ship time)

Claim handling: PyAutoReduce's worktree is held by frame-products (library-dev). This task follows the keck-ao-acceptance-checks pattern: analysis runs on the PyAutoReduce main checkout with no worktree claim — reductions write only to gitignored scripts/output/; the script commit/PR is deferred until the claim releases (or ships via coordination). Registered in active.md with worktree: none.

Implementation steps

1. Phase 0 — orient. Read docs/design/hst_acs_pipeline.md (validation section). Plot the legacy dataset (data/noise/PSF + extra-galaxy mask overlay) via the assistant's al_prepare_imaging_data procedure; record pixel scale, shape, PSF stamp sizes. Post dataset.png + the extra-galaxy/artefact review to this issue (the real-data gate checkpoint — treatment is pinned by the legacy dataset's own mask files, flagged here for human review rather than blocking).
2. Phase 1 — reduce. MAST query at (217.517083, +41.099194) → SLACS proposal ID(s) → TargetSpec(name="slacs1430+4105", ra=..., dec=..., proposal_ids=(...,)) → reduce_target(...) with ~/venv/PyAuto/bin/python, mirroring reduce_slacs0008.py (cache under scripts/cache/, output under scripts/output/slacs1430+4105/). Write parity_summary.json (registered_ratios, correlated_noise_factor, PSF diagnostics).
3. Phase 2 — pixel comparison. Registered data/noise ratio maps, S/N histograms, PSF radial-profile overlay (legacy psf.fits/psf_full.fits vs new), grid/orientation/centering checks. Plots via assistant scratch; key figures attached to this issue.
4. Phase 3 — model parity. Same model both datasets: SIE + external shear, lens light (Sersic/MGE per assistant guidance), same source parameterisation, legacy positions.json, 3.5" circular mask (per legacy info.json), identical extra-galaxy handling. Nautilus, identical priors/settings. Compare θ_E, q, PA, source properties, residual maps, log-evidence; reference literature values above.
5. Phase 4 — report + ship. Verdict comment on this issue; reduce_slacs1430.py + parity-fit script committed to feature/slacs1430-acs-parity and PR opened behind the four-leg autonomous-ship gate once the PyAutoReduce claim allows (else parked uncommitted like keck-ao, with resume notes here). Calibration-log row appended at PR-open/park. Methodology notes → PJ011646 WFC3 follow-up prompt.

Key files

• PyAutoReduce/scripts/reduce_slacs0008.py — template: production reduction + registered_ratios parity vs a legacy dataset in the same /mnt/c/.../subhalo/dataset/slacs/ tree
• PyAutoReduce/autoreduce/validation.py — registered_ratios sub-pixel registration comparison
• PyAutoReduce/docs/design/hst_acs_pipeline.md — authoritative ACS stage design + validation bar
• /mnt/c/Users/Jammy/Science/subhalo/dataset/slacs/slacs1430+4105/ — legacy dataset (data, noise_map, psf, psf_full, positions.json, extra_galaxies_centres.json, mask_extra_galaxies.fits, info.json)
• autolens_assistant/skills/al_prepare_imaging_data.md, al_build_imaging_model.md — assistant procedures driving phases 0 and 3

Original Prompt

Click to expand starting prompt

PyAutoReduce validation: slacs1430+4105 ACS reduction vs trusted legacy dataset

Type: test
Target: workspaces
Repos:

• autolens_assistant
• PyAutoReduce
  Difficulty: medium
  Autonomy: safe
  Priority: normal
  Status: formalised

Validate PyAutoReduce against a trusted legacy reduction: reduce the SLACS lens slacs1430+4105 from archival HST/ACS data using PyAutoReduce, then compare the resulting modeling-ready dataset (data, noise map, PSF) to the long-used collaborator-provided dataset at /mnt/c/Users/Jammy/Science/subhalo/dataset/slacs/slacs1430+4105 (used for many years in subhalo work). Comparison is a science project driven through autolens_assistant: image/noise/PSF residuals plus lens-model parity fits on both datasets.

[Jammy2211]

Phase 0 complete — legacy dataset inspected (real-data gate).

Loaded the trusted dataset at /mnt/c/Users/Jammy/Science/subhalo/dataset/slacs/slacs1430+4105 (281×281 @ 0.05″/pix = 14.05″ FoV, peak S/N 319) and reviewed it before any fitting:

• The lens: textbook Einstein ring, θ_E ≈ 1.5″ consistent with the literature 1.52″, bright lens light, clumpy multi-knot source. Both positions.json points sit on the ring. No artefacts inside the 3.5″ fit mask.
• Extra objects: mask_extra_galaxies.fits has 5 components, all >4″ from centre (outside the fit mask): a saturated star at (y=+5.9″, x=−5.3″) with 14 zeroed core pixels, the compact elliptical companion at (+6.3″, −4.2″) — the single entry in extra_galaxies_centres.json — a faint galaxy W, two low-SB objects S, and an edge-on streak SE.
• Gate question for the human (non-blocking, treatment pinned by the legacy files themselves): I will reuse this exact extra-galaxy treatment on both reductions in the parity fits. Flag here if any of those outer objects (esp. the star) deserves different handling.
• One convention note: extra_galaxies_centres.json / positions.json are (y, x) with y=+up, x=+right (row 0 = bottom) — verified against the mask components, worth knowing when the PyAutoReduce cutout is compared for grid alignment in Phase 2.

Inspection plots (local, session scratch): autolens_assistant/scripts/scratch/slacs1430_parity/plots/legacy_dataset.png + legacy_data_annotated.png.

Baselines carried into Phase 1 from slacs0008-0004 production parity: data ratio 0.941, noise ratio 0.925 (R = 1.364 applied), tier-1 ePSF (29 stars). Next: MAST query at (217.517083, +41.099194) to confirm the SLACS ACS/F814W proposal ID(s), then the production reduction.

[Jammy2211]

Phase 1, first pass — three findings, one re-run in flight.

The pipeline ran end-to-end (MAST → CRDS → AstroDrizzle 0.05″ north-up → ePSF tier 1, 123 stars → package), but the out-of-the-box parity stats were garbage (data ratio 0.21, registration offset 55 px). Diagnosis, in order of discovery:

1. The pipeline's astrometry is correct — the lens centroid lands (0.44, 0.28) px from the cutout centre, i.e. the GAIA-aligned WCS + my SDSS-name coordinates agree to ~20 mas. Not the problem.
2. The legacy dataset is x-mirrored and rotated ~7.2° relative to north-up (measured from the lens→star vector in both frames; the star also shows a 1.4% apparent scale difference, likely its saturated-core centroid bias in the legacy frame). The legacy reduction was evidently not drizzled north-up/east-left, so the global cross-correlation registration (star-dominated) misplaces the lens and scrambles the annular flux ratios. After fliplr alone the data ratio recovers to 0.64; with the lens properly aligned, total flux within the 3.5″ aperture agrees to 2.6% (ratio 1.026) — photometric parity is basically there. Residual sky offset: new 0.0157 vs legacy 0.0090 e−/s (same sky-subtraction family of differences as slacs0008).
3. Genuine acquire bug: duplicate exposures. download_exposures globs every *_flc.fits after the MAST product filter, and MAST delivers each exposure in two product families — HAP (hst_10886_36_..._j9op36bf_flc.fits, GAIA-aligned) and standard cal (j9op36bfq_flc.fits). The run stacked 8 files = 4 unique exposures (EXPTIME 4256 s = 2×2128). Filed as PyAutoMind/bug/pyautoreduce/acquire_duplicate_exposure_families.md (no source edits in this task — repo claimed by frame-products). Workaround applied here: cache-manifest surgery keeping the HAP family only; clean 4-exposure re-reduction is running now.

Phase 2 comparison (with flip+rotation handled about the lens centre) follows once the re-run lands. The legacy fit (Phase 3, MGE + SIE + shear + MGE) is already sampling in Nautilus.

[Jammy2211]

Phase 2 complete — pixel-level parity is excellent once the frame orientation is understood.

The phase-1 "mirror + 7°" reading was wrong — a star-anchored two-point transform can't distinguish dihedral candidates. An exhaustive dihedral × rotation search over the star-free ring annulus settles it: the legacy dataset is rot270 of the pipeline's north-up frame (no mirror), with a −0.3° residual (normalized ring correlation 0.997). The legacy reduction was simply drizzled with north along a different axis.

Parity after rot270 + lens-centroid anchoring (clean 4-exposure reduction):

Quantity	slacs1430+4105	slacs0008-0004 baseline	Reading
lens centring	2.5 px anchor shift, lens 0.4 px from cutout centre	(0, 2) px	GAIA WCS + target coords agree to ~20 mas
data ratio (bright px, n=8506)	1.040 [1.00, 1.10]	0.941	e−/s confirmed; ~4% flux scale, halo-edge ring pattern → small additive sky-offset difference (new sky 0.016 vs legacy 0.009 e−/s)
noise ratio	1.306 [1.17, 1.35], histogram peak at 1.35	0.925 (R applied)	ratio map is uniform ≈ R = 1.364 → the legacy slacs1430 noise map lacks the correlated-noise correction — opposite of the slacs0008 legacy. The trusted legacy family is internally inconsistent; the pipeline standardises this.
PSF FWHM	0.129″ vs legacy 0.120″	tier-1 ePSF both	8% broader; 120 ePSF stars
S/N (bright px)	uniformly ≈ 0.77	—	direct consequence of the noise-R line above

Figures: autolens_assistant/scripts/scratch/slacs1430_parity/plots/pixel_comparison.png, psf_comparison.png, corner_check.png.

Implications for Phase 3 (both fits now running, identical model/mask/priors; autoreduce positions mapped through the anchored similarity transform): parameter medians should agree; posterior widths on the autoreduce dataset will be ~1.36× wider (its noise carries R; the legacy noise doesn't). θ_E vs the literature 1.52″ is the headline number.

[Jammy2211]

Parked for the night (user request). Both phase-3 Nautilus fits are checkpointed mid-sampling (the two could not run concurrently — ~5.5 GB each OOM'd the laptop; they will resume serially). Full resume state is in PyAutoMind active.md; short version: run prototypes/slacs1430_parity_fit.py autoreduce then legacy one at a time (auto-resume), compare fit_summary_*.json, verdict here, then the phase-4 ship once the frame-products claim releases.

[Jammy2211]

Verdict (close-out 2026-07-10, human-directed)

Pixel-level parity: PASS. The PyAutoReduce production reduction of slacs1430+4105 (program 10886, 4 exposures, 2128 s) reproduces the trusted legacy dataset up to fully-understood differences:

	Result
Astrometry	lens centred to 0.4 px by the GAIA WCS; legacy frame is rot270 of north-up (no mirror; anchor transform
Photometry	bright-pixel flux ratio 1.040 [1.00–1.10]; 3.5″-aperture total 1.026; small additive sky-offset ring at the halo edge (new 0.016 vs legacy 0.009 e−/s)
Noise	uniform ratio ≈ 1.35 = R → the legacy noise map lacks the correlated-noise correction, while slacs0008's legacy had it — the legacy family is internally inconsistent; the pipeline standardises this
PSF	tier-1 ePSF (120 stars), FWHM 0.129″ vs legacy 0.120″
Acquire bug	duplicate HAP+cal product families (8 files = 4 exposures, noise √2-low) — independently found and fixed on main by the #18 branch (is_direct_product); my analysis used a manual dedupe keeping the HAP family (disclosed on PR #23)

Model-level parity: NOT RUN — the design doc's science-invariance fits are checkpointed mid-sampling (~5.5 GB each; repeatedly OOM'd against other workloads). This issue stays open with that leg as the last checklist item:

• reduction + pixel parity + PSF comparison
• scripts shipped: PR test: slacs1430+4105 ACS validation scripts (parity study lens 2) #23
• model-parity fits: from PyAutoReduce root, run serially on a quiet machine: ~/venv/PyAuto/bin/python prototypes/slacs1430_parity_fit.py autoreduce then ... legacy (Nautilus auto-resumes from output/slacs1430_parity/<key>); compare prototypes/output/slacs1430_parity/fit_summary_*.json (θ_E vs lit 1.52″; expect autoreduce widths ~1.36× wider from the noise-R difference) and post the verdict here

Methodology notes for the queued PJ011646 WFC3 follow-up: confirm the legacy/collaborator frame orientation first (dihedral × rotation scan over source structure, never a star-anchored two-point fit); check whether the collaborator noise map carries R; PJ011646's dataset has no positions.json or extra-galaxy masks, so those must be created at inspection time.