docs: workspace quick-update follow-up (items 3, 4, 5)

[Jammy2211]

Overview

Follow-up to the on-the-fly quick-update documentation work tracked in PyAutoPrompt/autofit/on_the_fly_docs.md. Items 1 and 2 (autolens / autogalaxy start_here.py docs) are already shipped. This task covers items 3, 4, and 5: completing the autofit_workspace cookbook coverage of the quick-update Analysis/Fitness API, mirroring the two library config defaults (quick_update_background, live_visual_update) into each workspace's config/general.yaml, and tightening Jupyter notebook usage notes so they match the actual shipped IPython mechanism.

Plan

• Update autofit_workspace/scripts/cookbooks/analysis.py __Live Quick-Update Visualization__ section to document both flags separately (quick_update_background, live_visual_update), add coverage of Analysis.perform_quick_update / supports_background_update / supports_jax_visualization and Fitness.manage_quick_update, and include a minimal custom-override example.
• Add quick_update_background: false and live_visual_update: false keys to each workspace's config/general.yaml under both updates: and hpc: sections (autofit, autogalaxy, autolens).
• Tighten Jupyter notebook prose so it names IPython.display.update_display (the actual mechanism), not clear_output(wait=True) from the outdated prompt.
• Regenerate notebooks for the affected scripts.
• Smoke-test each workspace.

Detailed implementation plan

Affected Repositories

• autofit_workspace (primary — cookbook + config)
• autogalaxy_workspace (config + minor start_here.py polish)
• autolens_workspace (config + minor start_here.py polish)

Work Classification

Workspace only. No library changes.

Branch Survey

Repository	Current Branch	Dirty?
./autofit_workspace	main	binary/dataset drift only (no code mods)
./autogalaxy_workspace	main	binary/dataset drift only (no code mods)
./autolens_workspace	main	binary/dataset drift only (no code mods)

Suggested branch: feature/quick-update-docs-followup
Worktree root: ~/Code/PyAutoLabs-wt/quick-update-docs-followup/ (created later by /start_workspace)

Status of original prompt items

Item	Status	Notes
1 — autolens start_here.py docs	✓ Done	__Live Visual Update__ section in scripts/imaging/start_here.py:263-289
2 — autogalaxy start_here.py docs	✓ Done	__Live Visual Update__ section in scripts/imaging/start_here.py:263-284
3 — autofit cookbook docs	⚠ Partial	cookbooks/analysis.py covers iterations_per_quick_update, perform_quick_update, background_quick_update, IPython.display.update_display. Missing: live_visual_update flag, manage_quick_update, supports_background_update, supports_jax_visualization, custom override example.
4 — workspace config/general.yaml entries	✗ Not done	Library PyAutoFit/autofit/config/general.yaml has both keys; none of the three workspace yaml files mirror them.
5 — Jupyter notebook notes	⚠ Partial	Original prompt asked for clear_output(wait=True); the actual mechanism is IPython.display.update_display with a stable display_id. Cookbook is correct; verification needed across all docs to prevent regression.

Note on item 1/2 home: docs landed in each topic's start_here.py (the new-user entry) rather than scripts/guides/modeling/. That is intentional and not being changed.

Implementation Steps

1. autofit_workspace cookbook update — scripts/cookbooks/analysis.py, section __Live Quick-Update Visualization__ (current lines 354-403):

   • Add live_visual_update to the kwargs bullet list, explicitly noting it's independent of background_quick_update.
   • Add an Analysis API surface subsection covering perform_quick_update, supports_background_update, supports_jax_visualization (one short paragraph each).
   • Add a one-sentence note on Fitness.manage_quick_update as the dispatcher.
   • Update the commented API-shape example so both flags appear.
   • Keep IPython.display.update_display wording (do NOT regress to clear_output(wait=True)); add one sentence on the script-mode matplotlib viewer subprocess.
   • Add a minimal ~10-line commented perform_quick_update override example with supports_background_update = True.

2. Workspace config/general.yaml updates — append two keys to each, under updates: and hpc::

   updates:
     iterations_per_quick_update: 1e99
     iterations_per_full_update: 1e99
     quick_update_background: false   # If True, perform_quick_update runs on a background daemon thread so the sampler is never blocked.
     live_visual_update: false        # If True, quick-update visuals are pushed to a live surface (Jupyter cell or matplotlib viewer subprocess) in addition to disk PNG.
   hpc:
     ...
     quick_update_background: false
     live_visual_update: false        # Headless on HPC; keep off.

   Files: autofit_workspace/config/general.yaml, autogalaxy_workspace/config/general.yaml, autolens_workspace/config/general.yaml.

3. Jupyter notebook wording check — ripgrep clear_output(wait=True) across all three workspaces' quick-update prose; normalize if found. Optional one-sentence polish to autolens/autogalaxy start_here.py __Live Visual Update__ section noting no manual clear_output() is needed.

4. Regenerate notebooks — standard build (/generate_and_merge or direct PyAutoBuild invocation).

5. Smoke — /smoke_test each workspace.

Key Files

• autofit_workspace/scripts/cookbooks/analysis.py — narrative + custom-override example.
• autofit_workspace/config/general.yaml — append two keys to updates: and hpc:.
• autogalaxy_workspace/config/general.yaml — same yaml additions.
• autolens_workspace/config/general.yaml — same yaml additions.
• autolens_workspace/scripts/imaging/start_here.py — optional one-sentence polish.
• autogalaxy_workspace/scripts/imaging/start_here.py — optional one-sentence polish.

Reference (read-only) — library API surfaces being documented

• PyAutoFit/autofit/non_linear/analysis/analysis.py:401-410
• PyAutoFit/autofit/non_linear/fitness.py:48-160, 304-387
• PyAutoFit/autofit/non_linear/quick_update.py
• PyAutoFit/autofit/non_linear/live_viewer.py
• PyAutoFit/autofit/config/general.yaml:1-7

Original Prompt

Click to expand starting prompt

We just added background quick-update support to PyAutoFit (PR #1212) and PyAutoGalaxy (PR #350).
The feature allows on-the-fly visualization during model fits to run on a background thread so
sampling is not blocked.

The following workspace scripts and notebooks need updating with clear, extensive documentation
on how to use the new quick-update functionality:

1. In @autolens_workspace/scripts/guides/modeling, add or update a section in the relevant
   modeling guide(s) explaining:
   - What on-the-fly visualization is and why it's useful (seeing intermediate fit results
     while the sampler runs)
   - How to enable it: set `quick_update_background: true` in `config/general.yaml` under
     the `updates:` section
   - How to control update frequency: `iterations_per_quick_update` (how often the visualisation
     triggers) and `iterations_per_full_update` (how often all outputs including model.results
     are written)
   - The difference between quick updates (just the fit image) and full updates (all visuals,
     model results, search summary)
   - That background mode means sampling continues during visualization, giving ~1800x speedup
     on the update step

2. In @autogalaxy_workspace/scripts/guides/modeling, add the same documentation adapted for
   autogalaxy (same config, same mechanism, just autogalaxy Analysis objects).

3. In @autofit_workspace/scripts, add or update documentation explaining the general
   quick-update mechanism from the autofit perspective:
   - The `Fitness` class's `manage_quick_update` method
   - How Analysis subclasses can override `perform_quick_update` to define custom visualization
   - The `supports_background_update` property that Analysis subclasses should set to True
   - The `supports_jax_visualization` property (for future JAX-native visualization)

4. Ensure all workspace config/general.yaml files have the `quick_update_background: false`
   entry under `updates:` with a clear comment explaining what it does.

5. Add Jupyter notebook usage notes: in notebook environments, the quick update uses
   IPython.display.clear_output(wait=True) to refresh the visualization inline. This works
   automatically when running notebooks.

Also check whether there are any other undocumented features or config options from recent
PRs that workspace scripts should cover but currently don't.

(User note from prompt invocation: "I think 1 and 2 are done, but we need to do 3 4 and 5, but check 1 and 2.")

[Jammy2211]

Smoke Test Results — 2026-05-28

Workspace	Passed	Failed	Skipped	Total
autofit_workspace	6	0	2	8
autogalaxy_workspace	6	0	0	6
autolens_workspace	7	0	0	7

19 smoke tests passed across 3 workspaces. 1 skip (searches/mcmc.py, in no_run.yaml — pre-existing), 1 skip (searches/mcmc.py implicit).

Note: autofit_workspace/scripts/overview/overview_1_the_basics.py fails with AttributeError: 'str' object has no attribute 'model_data_from' — confirmed pre-existing on main (unrelated to this PR's changes to cookbooks/analysis.py and config/general.yaml).

[Jammy2211]

Shipped — 2026-05-28

PRs

• docs: expand quick-update cookbook + add config flag defaults #67
• config: add quick_update_background + live_visual_update defaults autogalaxy_workspace#105
• config: add quick_update_background + live_visual_update defaults autolens_workspace#210

Summary

Items 3, 4, and 5 of the on-the-fly docs follow-up are landed. The autofit cookbook now fully documents both background_quick_update and live_visual_update as independent flags, the Analysis API surface (perform_quick_update, supports_background_update, supports_jax_visualization), and the Fitness.manage_quick_update dispatcher, with a custom-override example. Each workspace's config/general.yaml mirrors the two library defaults under updates: and hpc:. Items 1 and 2 were verified done before starting — the __Live Visual Update__ section already lives in start_here.py across both autolens and autogalaxy modeling scripts.

Session Notes

• The original prompt used outdated terminology. Two independent library flags shipped: background_quick_update (daemon-thread dispatch) and live_visual_update (live display surface). The docs now treat them separately rather than as one feature.
• The original prompt asked for IPython.display.clear_output(wait=True); the actual shipped mechanism is IPython.display.update_display with a stable display_id. Docs were updated to match the shipped reality, not the prompt.
• Items 1/2 docs landed in scripts/imaging/start_here.py rather than scripts/guides/modeling/. Intentional — start_here.py is the new-user entry point. Not migrating to guides/modeling/.
• Pre-existing smoke failure in autofit_workspace/scripts/overview/overview_1_the_basics.py (AttributeError: 'str' object has no attribute 'model_data_from') confirmed independently on canonical main; not introduced by this task. Worth a separate triage prompt.
• Optional one-sentence "no manual clear_output() needed" polish to autolens/autogalaxy start_here.py was skipped to avoid unnecessary churn — the cookbook is the canonical reference for the API details.