sp_validation restructuring & cleanup

[cailmdaley]

Tracking issue (epic) for the sp_validation restructuring & cleanup — bringing the package to a clean, modular, multi-person layout now that the science has outgrown its original structure. PRs land and fall away; this issue is the durable home for the whole effort.

The goal

A sp_validation where the things you run live at the top, every module has an honest home, the validation workflow is modular enough for more than one person, and the style is consistent — so the package is something others can contribute to, not a single-author tangle.

Pieces

The restructuring is a stack of PRs toward develop — each is based on the previous, so #199 merges first and the rest rebase down.

Landed

• Restructuring: top-level layout + workflow/papers/scratch division #197 — top-level layout + workflow/ / papers/ / scratch/ split
• chore: deterministic dependency pinning (SHA-pin getdist, camb floor, scipy root-cause) #200 — deterministic dependency pinning

In flight (the stack, bottom → top)

• refactor(src): dissolve basic.py, give every module an honest home #199 — dissolve basic.py, give every module an honest home (behavior-preserving src/ reshape) — in review (Martin + Sasha)
• chore: ruff format + region-aware lint gate #206 — project-wide ruff pass: format + region-aware lint gate
• refactor(cosmo_val): god-class → tested, primitive-backed package #207 — cosmo_val god-class → tested, primitive-backed package: split the 3,700-line class into a mixin package (85 methods conserved), cleanup + cross-module dedup, thin the orchestrator into survey.py and a new top-level pseudo_cl.py of stateless primitives, and add the first numeric pins for the pseudo-Cℓ estimator. Three bugs fixed along the way — a non-parsing demo file, the aperture-mass DES calibration, and the pseudo-Cℓ noise-debiasing reproducibility.

Follow-ups surfaced (not blocking the stack)

• Five undefined-name bugs in papers/harmonic + papers/consistency analysis scripts — flagged for their authors.
• __all__ on the star-imported modules.

Not in this epic

• Tomography extension (tracked in sp_validation: extend to tomography #205) and the cosmology-inference work — feature work on top of the restructured base, not the cleanup itself.

— Claude on behalf of Cail

[LisaGoh]

Heyo, I know this issue has been closed, but it seemed like to most appropriate place to raise this (also sorry for not saying this earlier!), but I'll be happy to clean up the cosmo_inference folder. It has some obselete scripts and notebooks that have overlapping functions with what cosmo_val already does. I can also try to homogenise it with the Snakemake scripts and inference.smk.