Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -180,7 +180,8 @@ When **not** in maintainer mode, commit at natural checkpoints (a script + its
straight to the `aplt.*` plotting call (e.g. `aplt.subplot_imaging_dataset`,
`aplt.subplot_fit_imaging`) — there is no separate `aplt.Output`/`MatPlot2D` object.
Then `print(...)` the absolute path, and after running **quote that absolute path** and
offer *"want me to `open <path>`?"* — don't just say "plot saved". One offer per plot.
offer to open it (platform opener: `open` on macOS, `xdg-open` on Linux,
`explorer.exe`/`wslview` from WSL) — don't just say "plot saved". One offer per plot.

---

Expand Down
12 changes: 6 additions & 6 deletions skills/_style.md
Original file line number Diff line number Diff line change
Expand Up @@ -279,8 +279,8 @@ the user-facing content above should read like a conversation arc, not a recipe.
- Point at the wiki by relative path every time you teach a concept.
- For newcomers, surface the relevant HowToLens notebook before the code block, not
after. See "Newcomer mode" in Adaptive depth above.
- When a script produces plot files, quote the absolute path and offer to
`open <path>` (macOS). See "Plot output and path announcement" below.
- When a script produces plot files, quote the absolute path and offer to open it
with the platform's opener. See "Plot output and path announcement" below.

**Don't**

Expand Down Expand Up @@ -338,9 +338,9 @@ Skills that produce visualisations save them through the function-style
because each `aplt.*` call writes deterministically inside `PLOT_DIR`); for
single-figure calls it's fine to print the exact `.png` path instead.
3. **The agent quotes the path back** to the user after running the script
and offers *"want me to `open <path>`?"* — one offer per plot run, not
nagging. `open` on macOS opens a directory in Finder or a PNG in
Preview.
and offers to open it — one offer per plot run, not nagging. Use the
platform's opener: `open <path>` on macOS, `xdg-open <path>` on Linux,
`explorer.exe` (or `wslview`) from WSL.

The full convention — committed Python lives in `scripts/`; throwaway plots and data
dumps go to the gitignored `scripts/scratch/` — is in `AGENTS.md`
Expand Down Expand Up @@ -376,6 +376,6 @@ Rules:

## Iteration

This guide is the the workspace v1 writing guide. As patterns emerge, update this file in
This guide is the workspace v1 writing guide. As patterns emerge, update this file in
the same change as whatever motivated the update. Note the change at the top of that
PR description.
12 changes: 6 additions & 6 deletions skills/al_build_interferometer_model.md
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,9 @@ dataset = al.Interferometer.from_fits(
noise_map_path=dataset_path / "noise_map.fits",
uv_wavelengths_path=dataset_path / "uv_wavelengths.fits",
real_space_mask=real_space_mask,
# transformer_class defaults to al.TransformerNUFFT — the NUFFT that maps the
# model image to visibilities. al.TransformerDFT is exact but only viable for
# small visibility counts.
)

# Identical model composition to the imaging case.
Expand All @@ -64,12 +67,9 @@ source = af.Model(
)
model = af.Collection(galaxies=af.Collection(lens=lens, source=source))

# Difference from imaging: AnalysisInterferometer, plus a transformer class that
# converts the model image to visibilities via NUFFT.
analysis = al.AnalysisInterferometer(
dataset=dataset,
settings=al.Settings(),
)
# Difference from imaging: AnalysisInterferometer computes the likelihood in
# visibility space (the transformer was chosen on the dataset above).
analysis = al.AnalysisInterferometer(dataset=dataset)
```

Source citations:
Expand Down
12 changes: 8 additions & 4 deletions skills/al_chain_searches.md
Original file line number Diff line number Diff line change
Expand Up @@ -77,8 +77,8 @@ result_1 = search_1.fit(model=model_1, analysis=analysis)
"""
__Phase 2__

Swap in a pixelised source (`al.mesh.Delaunay` + `al.reg.ConstantSplit`) for maximum source
flexibility, inheriting the lens mass and shear from phase 1's posterior so the search
Swap in a pixelised source for maximum source flexibility,
inheriting the lens mass and shear from phase 1's posterior so the search
starts from a good region of parameter space rather than the prior. `result_1.model`
returns a new `af.Model` whose priors are the previous search's posterior, keeping the
parameters free to vary.
Expand All @@ -95,8 +95,12 @@ source_2 = af.Model(
redshift=1.0,
pixelization=af.Model(
al.Pixelization,
mesh=al.mesh.Delaunay,
regularization=al.reg.ConstantSplit,
# NOTE: Delaunay + ConstantSplit is the standard production choice, but a
# known regression (PyAutoArray #332, still open) crashes Delaunay inside
# FitImaging and breaks ConstantSplit on RectangularUniform. Until it is
# fixed, use the combination below.
mesh=al.mesh.RectangularUniform,
regularization=al.reg.Constant,
),
)
model_2 = af.Collection(galaxies=af.Collection(lens=lens_2, source=source_2))
Expand Down
13 changes: 7 additions & 6 deletions skills/al_configure_search.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
name: al_configure_search
description: Pick and tune a non-linear search for a lens-modelling fit. Defaults to Nautilus (nested sampling, the recommended choice); covers Dynesty, Emcee, Zeus, UltraNest, and the gradient/swarm options for completeness. Sets sampler-specific knobs (live points, walkers, tolerance) and the output `path_prefix` / `name` that determine where results land. Pairs with `al_run_search`, which actually calls `search.fit`.
description: Pick and tune a non-linear search for a lens-modelling fit. Defaults to Nautilus (nested sampling, the recommended choice); covers Dynesty, Emcee, Zeus, and the gradient/optimizer options for completeness. Sets sampler-specific knobs (live points, walkers, tolerance) and the output `path_prefix` / `name` that determine where results land. Pairs with `al_run_search`, which actually calls `search.fit`.
---

# Configuring a non-linear search
Expand All @@ -19,7 +19,7 @@ search (what nested sampling does, what MCMC does, what gradient descent does),

- *"How complex is the model?"* — number of free parameters and whether any are
expected to be multi-modal. Multi-modality strongly favours nested sampling
(Nautilus / Dynesty / UltraNest) over MCMC.
(Nautilus / Dynesty) over MCMC.
- *"How fast does the likelihood evaluate?"* — fast (<1s with JAX) → can afford more
live points or walkers. Slow → tighten the search.
- *"Is this a first exploratory fit, or a final production run?"* — exploratory fits
Expand All @@ -41,7 +41,7 @@ search = af.Nautilus(
name="modeling_sie_sersic",
unique_tag="<your_dataset>",
n_live=200, # 100 for exploration, 200 for production, 400+ for complex models
iterations_per_update=2500,
iterations_per_full_update=2500,
number_of_cores=4, # parallel likelihood evaluations
)
```
Expand All @@ -52,7 +52,8 @@ Knobs to know:
- `n_live` — more = more accurate posterior, slower. Start at 200; go to 400+ only if
the posterior looks multi-modal or thin.
- `number_of_cores` — set to your CPU core count for parallel likelihood eval.
- `iterations_per_update` — how often the search writes intermediate samples to disk.
- `iterations_per_full_update` / `iterations_per_quick_update` — how often the search
writes full output (samples, visualisation) vs quick intermediate updates to disk.

## Branch — Dynesty

Expand Down Expand Up @@ -88,8 +89,8 @@ Source: `PyAutoFit:autofit/non_linear/search/mcmc/emcee.py`.

## Branch — Other searches

Zeus (ensemble slice), UltraNest (nested sampling alternative), PySwarms (particle swarm),
BFGS (gradient descent for MLE), Drawer (random prior draws — debugging only). See
Zeus (ensemble slice MCMC), DynestyDynamic (dynamic nested sampling), BFGS / LBFGS
(gradient descent for MLE), Drawer (random prior draws — debugging only). See
[`wiki/core/api/searches.md`](../wiki/core/api/searches.md) for the comparison table.

## Output folder layout
Expand Down
89 changes: 54 additions & 35 deletions skills/al_custom_profile.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,18 +24,21 @@ Canonical reference: `autolens_workspace:scripts/guides/advanced/add_a_profile.p

## Branch — custom light profile

Subclass `LightProfile` (or `LightProfileLinear` for a profile whose intensity is
solved during the fit). Implement `image_2d_from(grid)`:
Subclass `ag.LightProfile` (or `LightProfileLinear` for a profile whose intensity is
solved during the fit). Implement `image_2d_from(grid)` — the base class provides the
geometry helpers (`elliptical_radii_grid_from` returns each coordinate's elliptical
radius from the profile centre; the returned object exposes the raw values as
`.array`). No serialisation mixin is needed: profiles serialise to `tracer.json`
automatically from their constructor arguments.

```python
# scripts/my_profile.py
import numpy as np
import autogalaxy as ag
from autoconf.dictable import Dictable


class TwistedSersic(ag.LightProfile, Dictable):
"""Sersic light profile with isophote position-angle twist with radius."""
class TruncatedSersic(ag.LightProfile):
"""Sersic light profile with an exponential outer truncation."""

def __init__(
self,
Expand All @@ -44,41 +47,47 @@ class TwistedSersic(ag.LightProfile, Dictable):
intensity=1.0,
effective_radius=1.0,
sersic_index=2.0,
twist_rate=0.0,
truncation_radius=5.0,
):
super().__init__(centre=centre, ell_comps=ell_comps)
self.intensity = intensity
self.effective_radius = effective_radius
self.sersic_index = sersic_index
self.twist_rate = twist_rate
self.truncation_radius = truncation_radius

@property
def sersic_constant(self):
# b_n via the Ciotti & Bertin (1999) expansion — the base class does not
# provide it for custom profiles, so state it explicitly.
n = self.sersic_index
return 2.0 * n - 1.0 / 3.0 + 4.0 / (405.0 * n) + 46.0 / (25515.0 * n**2)

def image_2d_from(self, grid, **kwargs):
# Apply the elliptical + twisted transform manually, then evaluate Sersic.
radii = self._radii_from_grid(grid) # helper method
intensity = self.intensity * np.exp(
-self._b_n() * ((radii / self.effective_radius) ** (1.0 / self.sersic_index) - 1.0)
radii = self.elliptical_radii_grid_from(grid)
sersic = self.intensity * np.exp(
-self.sersic_constant
* ((radii.array / self.effective_radius) ** (1.0 / self.sersic_index) - 1.0)
)
return intensity
return sersic * np.exp(-((radii.array / self.truncation_radius) ** 2))
```

Source citations:
- `PyAutoGalaxy:autogalaxy/profiles/light/abstract.py` — `LightProfile` base class
and required interface.
- `PyAutoGalaxy:autogalaxy/profiles/light/abstract.py` — `LightProfile` base class,
required interface, and the radii-grid helpers.
- `PyAutoGalaxy:autogalaxy/profiles/light/standard/sersic.py` — Sersic implementation
to copy structure from.
- `PyAutoConf:autoconf/dictable.py` — `Dictable` mixin so the profile serialises to
`tracer.json`.
to copy structure from (note how it reads `grid_radii.array`).

Use in a model:
Use in a model (both files live in `scripts/`, so a sibling import works when the
script is run as `python scripts/build_model.py`):

```python
import autofit as af
from work.my_profile import TwistedSersic
from my_profile import TruncatedSersic

lens = af.Model(
ag.Galaxy,
redshift=0.5,
bulge=af.Model(TwistedSersic), # new profile, slots into the af.Model API
bulge=af.Model(TruncatedSersic), # new profile, slots into the af.Model API
)
```

Expand All @@ -89,8 +98,8 @@ Subclass `MassProfile` and implement at least `convergence_2d_from(grid)` and
deflection has no closed form.

```python
class CustomNFW(ag.mp.MassProfile, Dictable):
"""NFW with a custom inner-slope tweak."""
class CustomNFW(ag.mp.MassProfile):
"""NFW with a custom inner-slope tweak (spherical)."""

def __init__(self, centre=(0.0, 0.0), kappa_s=1.0, scale_radius=1.0, inner_slope=1.0):
super().__init__(centre=centre, ell_comps=(0.0, 0.0))
Expand All @@ -99,7 +108,7 @@ class CustomNFW(ag.mp.MassProfile, Dictable):
self.inner_slope = inner_slope

def convergence_2d_from(self, grid, **kwargs):
r = self._radii_from_grid(grid) / self.scale_radius
r = self.radial_grid_from(grid).array / self.scale_radius
return self.kappa_s / (r ** self.inner_slope * (1.0 + r) ** (3.0 - self.inner_slope))

def deflections_yx_2d_from(self, grid, **kwargs):
Expand All @@ -123,7 +132,7 @@ class that interpolates on the grid:
```python
from scipy.interpolate import RegularGridInterpolator

class TabulatedConvergence(ag.mp.MassProfile, Dictable):
class TabulatedConvergence(ag.mp.MassProfile):
def __init__(self, centre, table_path):
super().__init__(centre=centre, ell_comps=(0.0, 0.0))
self._interp = self._load_table(table_path)
Expand All @@ -137,31 +146,41 @@ it numerically but at a runtime cost. Profile your fit if this is slow.

## Priors and defaults

For the new profile to play nicely in `af.Model`, give each parameter a sensible
default prior. Either set it in the model declaration:
A custom profile has **no default priors** — `af.Model(TruncatedSersic)` reports
`prior_count == 0` and cannot be fit until every free parameter is given one. Set them
explicitly in the model declaration:

```python
profile = af.Model(TwistedSersic)
profile.twist_rate = af.UniformPrior(lower_limit=-1.0, upper_limit=1.0)
profile = af.Model(TruncatedSersic)
profile.centre.centre_0 = af.GaussianPrior(mean=0.0, sigma=0.1)
profile.centre.centre_1 = af.GaussianPrior(mean=0.0, sigma=0.1)
profile.intensity = af.LogUniformPrior(lower_limit=1e-4, upper_limit=1e2)
profile.effective_radius = af.UniformPrior(lower_limit=0.0, upper_limit=5.0)
profile.sersic_index = af.UniformPrior(lower_limit=0.5, upper_limit=8.0)
profile.truncation_radius = af.UniformPrior(lower_limit=0.5, upper_limit=10.0)
```

Or — for repeated use — add a default-priors YAML under
`PyAutoGalaxy:autogalaxy/config/priors/` keyed by class name. The conf system reads
it automatically.
Or — for repeated use — add a default-priors YAML under **this workspace's**
`config/priors/` (mirror the schema of the existing files in `config/priors/light/`),
which the conf system reads automatically. Never write the YAML into the installed
PyAutoGalaxy's own config — that edits library source, which the source-edit boundary
forbids, and it is lost on the next upgrade.

## Verification

```python
# Sanity: image_2d_from returns the right shape and finite values.
import autoarray as aa
grid = aa.Grid2D.uniform(shape_native=(50, 50), pixel_scales=0.05)
prof = TwistedSersic(centre=(0.0, 0.0), intensity=1.0, effective_radius=0.5, sersic_index=2.0, twist_rate=0.1)
prof = TruncatedSersic(centre=(0.0, 0.0), intensity=1.0, effective_radius=0.5, sersic_index=2.0)
img = prof.image_2d_from(grid=grid)
assert img.shape == (2500,) and np.isfinite(img).all()
assert np.asarray(img).shape == (2500,) and np.isfinite(np.asarray(img)).all()
```

For mass profiles, also sanity-check `convergence_2d_from` against a known case (set
`twist_rate=0`, your custom Sersic should match the built-in Sersic).
Also sanity-check against a known case: with the custom behaviour switched off
(here, `truncation_radius` set very large), the profile should match the built-in it
extends (`ag.lp.Sersic` with the same parameters). For mass profiles do the same with
`convergence_2d_from` against the canonical implementation.

## Combine

Expand Down
23 changes: 16 additions & 7 deletions skills/al_debug_fit_failure.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,18 +35,27 @@ Read the search log: `output/<path>/.../search.log`. Common causes:

## Branch — search completed but residuals are huge

Load the result and look at the fit:
Load the result and look at the fit (same pattern as
[`al_load_results`](./al_load_results.md) — load the max-log-likelihood tracer from
JSON, reload the dataset, rebuild the fit):

```python
import autofit as af
from pathlib import Path
from autoconf.dictable import from_json
import autolens as al
import autolens.plot as aplt

agg = af.Aggregator(af.db.open_database("sqlite://"))
agg.add_directory("output/imaging/<your_lens>/<name>")
result = list(agg.values("samples"))[-1] # most recent run
out = Path("output/imaging/<your_lens>/<name>/<hash>")
tracer = from_json(file_path=out / "files" / "tracer.json")

# Reload the dataset exactly as the fit script did (path, pixel_scales, mask).
dataset = ... # see al_prepare_imaging_data

tracer = result.max_log_likelihood_tracer
fit = al.FitImaging(dataset=dataset, tracer=tracer)
aplt.subplot_fit_imaging(fit=fit)
aplt.subplot_fit_imaging(
fit=fit, output_path="scripts/scratch/debug/", output_filename="fit",
output_format="png",
)
```

Look at the residual map. Common signatures:
Expand Down
Loading
Loading