docs: complete wiki/core workspace coverage + llms.txt release audit

[Jammy2211]

Overview

autolens_assistant is heading to its first full public release. During development its documentation layer was deliberately kept partial. This task makes the wiki part of the layer that maps to autolens_workspace fully complete, and audits llms.txt — the front door that ChatGPT/Claude and other agents read — so it is release-ready. The 14 stub skills (Status: stub, honestly labeled in skills/README.md) are out of scope here: this task completes the wiki, not the skill recipes (see the scope question below).

Plan

• Audit wiki/core/ against the full autolens_workspace surface (llms-full.txt + scripts/ tree) and post a coverage table (covered / shallow / missing / stale) as a milestone comment.
• Write the missing wiki pages — confirmed candidates from the "catalogued, not yet stubbed" queue with real workspace scripts: over-sampling, multi-plane lensing, line-of-sight halos; plus whatever the audit surfaces (workflow outputs, Bayesian model comparison, shapelets/operated-profile/sky-background coverage — some may fold into existing pages).
• Refresh stale pages: every page is last_updated: 2026-05-22, predating the July weak-lensing rework (FitWeak) and the functional plot API — concepts/weak_lensing.md and api/plotting.md are known suspects.
• Verify each stub skill's companion wiki page is genuinely complete, and update wiki/core/index.md + wiki/core/external/skill_citation_map.md for any new pages.
• Audit llms.txt: every link resolves, routing and framing match the mature release state, hand-off to the workspace catalogue (autolens_workspace/llms.txt / llms-full.txt) is correct.
• Validate with autoassistant/audit_skill_apis.py (zero stale PyAuto* symbols) and ship via ship_workspace — run ends at PR-open per the autonomy contract.

Scope question (supervised checkpoint): the 14 stub skill recipes stay stubs for this release — the wiki is completed, the skills index labels stubs honestly, and recipe-filling is queued as follow-up prompts issued one at a time. Reply here if you want stub recipes pulled into release scope instead.

Detailed implementation plan

Affected Repositories

• autolens_assistant (primary, only repo)

Work Classification

Workspace (docs; develops in-place — no worktree, per repo precedent)

Branch Survey

Repository	Current Branch	Dirty?
./autolens_assistant	main	clean (1 untracked personal file james.bib — never commit; repo is a public template)

Suggested branch: feature/assistant-wiki-release

Worktree root: none — autolens_assistant develops in-place on a feature branch.

Implementation Steps

1. Coverage audit (read-only). Build the area map from autolens_workspace: llms-full.txt, workspace_index.json, and the scripts/ tree — imaging (features: extra_galaxies, linear_light_profiles, multi_gaussian_expansion, no_lens_light, pixelization, scaling_relation; advanced: double_einstein_ring, los_halos, mass_stellar_dark, operated_light_profile, shapelets, sky_background, subhalo), interferometer (+features/datacube), point_source, group, cluster, multi, weak, guides (advanced: multi_plane, over_sampling, over_sampling_chaining, custom_analysis, add_a_profile; results; plot; profiles; units; modeling). For each area, locate its wiki/core/ page(s) and classify: covered / shallow / missing / stale (frontmatter pinned_commit + last_updated vs current source). Post the table to this issue.
2. Write missing pages under wiki/core/concepts/ (or api/ where the content is API-shaped), following the existing page schema (frontmatter: title, sources with pinned_commit, last_updated). Judgment call per area: dedicated page vs a section in an existing page (e.g. shapelets → concepts/light_profiles.md). wiki/core/ is read-only except via the al_update_wiki maintainer workflow — this task is that workflow; follow skills/al_update_wiki.md.
3. Refresh stale pages. Known suspects: wiki/core/concepts/weak_lensing.md (July rework: FitWeak, NumPy-only, loaders default is_reduced=True) and wiki/core/api/plotting.md (functional aplt.* API; no MatPlot2D/Output objects). Re-pin sources: commits; bump last_updated.
4. Cross-layer consistency. For every new page: add to wiki/core/index.md; check wiki/core/external/skill_citation_map.md rows still resolve; confirm each stub skill's companion wiki page is complete enough to answer "what/why" questions the stub can't.
5. llms.txt audit (autolens_assistant/llms.txt): verify all relative links (AGENTS.md, skills/README.md, wiki/) and external URLs (workspace llms.txt, HowToLens) resolve; the paste-friendly opening prompt stays; claims match post-audit reality. Cross-check wiki/core/external/workspace.md still matches what the workspace ships at its root.
6. Validate. python autoassistant/audit_skill_apis.py full audit — zero stale symbols across skills/ and wiki/core/api+stack/ (regenerate api_audit_baseline.json if the audit workflow requires); link-check edited pages.
7. Ship via ship_workspace: commit on feature/assistant-wiki-release, PR to PyAutoLabs/autolens_assistant, four-leg gate (tests: n/a — no test dir, stated in PR body; smoke: n/a — doc-only change, stated; review faculty: CLEAN required; Heart: GREEN/STALE required, no launch-time YELLOW ack was given). Run ends at PR-open; merge is human.

Key Files

• wiki/core/index.md — wiki table of contents; every new page registers here
• wiki/core/concepts/*.md, wiki/core/api/*.md — the pages being completed/refreshed
• wiki/core/external/{workspace.md,skill_citation_map.md} — the workspace mapping layer
• llms.txt — the agent-facing front door under audit
• skills/README.md — stub labeling stays truthful if anything changes
• autoassistant/audit_skill_apis.py, api_audit_baseline.json — validation gate
• autolens_workspace/{llms.txt,llms-full.txt,workspace_index.json} — read-only source of the coverage map

Prose tier

Wiki pages are science-teaching prose → judgment tier (Fable/Opus), per the tutorial-prose split in PyAutoBrain/skills/WORKFLOW.md. Mechanical ship steps delegate to the execution tier.

Original Prompt

Click to expand starting prompt

the autolens_assistant was improved and made mature by Fable, but I think its skill layer which maps to autolens_workspace is incomplete (with the motivation being we were in development so didn't want to put everything in there whilst not mature). We are now doing a full release, so make sure the wiki part of this layer is fully complete. Also, make sure the llms.txt file, which interfaces with ChatGPT and others is also good to go

[Jammy2211]

Milestone 1 — coverage audit complete ✅

Mapped the full autolens_workspace surface (llms-full.txt + scripts/ tree, ~115 areas) against wiki/core/, and ran the assistant's API audit (autoassistant/audit_skill_apis.py).

Headline: the wiki is in much better shape than feared. The API audit is clean — 0 missing/broken symbols across 55 files, 116 symbols — so nothing cites removed API. And most topics I suspected were missing turned out to be covered as sections: over-sampling (grids_and_masks.md), multi-plane (tracer.md), linear light profiles / MGE / shapelets / operated profiles (light_profiles.md + light_profile_catalog.md), stellar+dark decomposition (mass_profiles.md), astrometric dataset offsets (multi_wavelength.md), Bayesian model comparison (substructure_and_subhalos.md), workflow outputs + database (aggregator.md). The plotting page already documents the functional aplt.* API.

Genuine gaps — workspace features with zero wiki treatment (Phase 2 scope):

Workspace feature	Fix
imaging/features/advanced/los_halos/	New section in concepts/substructure_and_subhalos.md
imaging+group/features/advanced/sky_background/	New section (placement decided from the script — likely extra_galaxies_and_noise_scaling.md or datasets.md)
point_source/features/deblending/	New section in concepts/point_source.md
guides/advanced/custom_analysis.py	New "custom analysis subclass" section in api/analysis_objects.md (companion for the al_custom_analysis stub skill)
imaging+group/features/advanced/double_einstein_ring/	Subsection/pointer in concepts/tracer.md multi-plane section
July weak-lensing series (FitWeak, weak/modeling.py, real_data/)	Enrich concepts/weak_lensing.md (API-current but predates the series)

Content will be derived by reading the workspace scripts + installed source (never from memory), per al_update_wiki.

Next: Phase 2 (write the pages) → llms.txt audit → validation → PR.

[Jammy2211]

Milestone 2 — implementation complete; parked at ship sign-off ⏸️

All phases are done and validated. The work is committed locally on feature/assistant-wiki-release (6adacdd, 36 files, +329/−156) but not pushed — under the supervised autonomy contract, ship sign-off is yours.

What changed since milestone 1

Beyond the planned coverage additions, the release audit turned up a second staleness class the API-symbol audit can't see — Project:path source citations pointing at pre-refactor file layouts — and fixed all of it:

• New wiki coverage: LOS-halo machinery (LOSSampler, negative-kappa sheets, He et al. 2022) in substructure_and_subhalos.md; sky background via DatasetModel in extra_galaxies_and_noise_scaling.md; deblending in point_source.md; custom Analysis subclasses in analysis_objects.md; double-Einstein-ring pointer in tracer.md.
• Weak lensing rewritten for the shipped July API: FitWeak + AnalysisWeak, reduced shear / is_reduced defaults, the A2744 real-data capstone, the strong+weak joint series.
• Removed Plane class purged: galaxy_and_plane.md, tracer.md, stack/{autogalaxy,autolens}.md, cosmology_and_units.md documented a class that no longer exists (tracer.planes returns Galaxies collections now).
• Citation moves re-grounded against installed source: sampler modules→packages (nest/nautilus/ etc., 8 paths × 6 files), over_sampling/, grid/sensitivity/, mass/abstract/, profiles/basis.py, CSV API (→ autogalaxy/galaxy/galaxy_model_csv.py), priors/visualize config layouts (no more mat_wrap_*, no autolens/config/priors/), slam_pipeline.md's fictional from autolens_workspace.slam import slam_main replaced with the inline-functions reality.
• index.md gained the missing operations/dataset, hpc_infrastructure and External entries; llms.txt reference bullets now point at the two wiki indexes (rest of the file audited good — links resolve, framing right).

Gate legs (autonomous-ship gate)

Leg	Status
Tests	n/a — no test dir (docs repo); analogue audit_skill_apis.py: 0 missing/broken, 117 symbols
Smoke	n/a — docs-only change, no downstream script surface; provenance check: 0 errors
Review	surface prepared (pyauto-brain review); mechanical checks clean (all Project:path citations + relative links resolve); verdict deferred to this sign-off
Heart	YELLOW, no launch-time ack: workspace validation not passing (3 failed); 58 stale parked scripts; autolens_assistant pinned BEHIND installed; PyAutoMind open PR 10d old (+2 stale)

The question

Approve ship? On approval I push the branch and open the PR (stopping at PR-open; merge stays yours). Approval here also covers the Heart YELLOW ack for the listed reasons. If you want changes, say what — the commit is local-only and can be amended cleanly.

Scope reminder from milestone 1: the 14 stub skill recipes stay stubs (wiki companions verified complete); recipe-filling queues as follow-ups.

[Jammy2211]

Shipped — 2026-07-09

PRs

• docs: complete wiki/core workspace coverage, llms.txt audit + --check-citations mode #41 (merged)

Summary

The wiki layer mapping to autolens_workspace is release-complete: real coverage gaps filled (LOS halos, sky background, deblending, custom Analysis, double Einstein ring, weak-lensing FitWeak rewrite), the removed-Plane-class documentation purged, ~35 stale Project:path citations re-grounded, llms.txt audited and its reference routing sharpened. The staleness class the audit couldn't see is now mechanically checkable via the new audit_skill_apis.py --check-citations mode (391 citations, 0 missing on main).

Session Notes

• Ship sign-off + merge were human-authorized in-conversation (covering the Heart YELLOW ack for the reasons listed at milestone 2).
• Scope call stands: the 14 stub skill recipes remain honestly-labeled stubs; their wiki companions are verified complete. Recipe-filling is queued as a Mind backlog prompt, to be issued one at a time.
• Follow-up idea for CI: run --check-citations alongside the symbol audit in whatever check gates assistant PRs.

[Jammy2211]

CI wiring shipped — 2026-07-09

PR #42 (merged): --check-citations now runs as the fifth wiki-currency leg on every PR and release-time workflow_call.

Getting it green took two design corrections, both caught by the PR's own CI runs:

1. Ground truth is the ref the docs pin, not the release wheel. The first run false-failed on post-release refactors (ultranest/, pyswarms/, weak/model/, quantity/) because CI installs the released stack while the wiki pins main. CI now shallow-clones the five libraries (+ sparse autolens_workspace) into sources/ for the citations leg; the release install remains ground truth for the symbol/version/idiom checks.
2. Real drift the local run couldn't see: grading against origin/main clones exposed that local sibling checkouts were behind — upstream has removed nest/ultranest/ and mle/pyswarms/ and archived autolens/quantity/ to autolens_workspace_developer/legacy. searches.md and stack/autolens.md no longer cite them (the samplers' prose already said "not public"; quantity is now a Derived-quantities note pointing at Tracer methods + workspace guides).

Final run: all five legs green — citations 388 checked, 0 missing, 0 skipped.