diff --git a/README.md b/README.md index 28af4c12..f815789c 100644 --- a/README.md +++ b/README.md @@ -174,10 +174,15 @@ That's the loop: **talk → plan → inspect.** With hardware connected (drop `- | Guide | Audience | What you'll learn | |-------|----------|-------------------| +| [Documentation Home](docs/index.md) | Everyone | Browse the generated-docs structure for Gently | +| [Full Stack Microscopy](docs/full-stack-microscopy.md) | Everyone | How intent, samples, hardware, perception, data, and operators fit together | | [Try Without Hardware](docs/guides/try-offline.md) | Everyone | Run the agent in 10 minutes — conversation, plan mode, perception | | [What Gently Can Do](docs/guides/capabilities.md) | Everyone | Perception, detection, plan mode, memory, mesh, safety | | [Build a Plugin](docs/guides/build-a-plugin.md) | Developers | Create organism and hardware plugins for other modalities | | [Hardware Setup](docs/guides/hardware-setup.md) | Labs | Connect a diSPIM, start the device layer, first acquisition | +| [Sample & Hardware Model](docs/architecture/sample-hardware-domains.md) | Developers | Generalize samples, hardware operations, and device-profile boundaries | +| [Sample Tracking Metrics](docs/architecture/sample-tracking-metrics.md) | Developers | Define reusable sample state, exposure, focus, and perception metrics | +| [Hardware Profile Template](docs/architecture/hardware-profile-template.md) | Developers | Checklist for adding or documenting a new microscope profile | ## Architecture diff --git a/docs/architecture/hardware-profile-template.md b/docs/architecture/hardware-profile-template.md new file mode 100644 index 00000000..bee826df --- /dev/null +++ b/docs/architecture/hardware-profile-template.md @@ -0,0 +1,97 @@ +# Hardware Profile Template + +Use this template when adding or documenting a new hardware profile. + +## Profile Identity + +- Profile name: +- Primary sample type: +- Primary acquisition modality: +- Device-layer entry point: +- Configuration file(s): + +## Standard Operations + +| Operation | Supported | Implementation | Notes | +| --- | --- | --- | --- | +| `sample_overview` | yes/no | | | +| `detect_samples` | yes/no | | | +| `move_to_sample` | yes/no | | | +| `focus_scan` | yes/no | | | +| `position_calibration` | yes/no | | | +| `acquire_snapshot` | yes/no | | | +| `acquire_volume` | yes/no | | | +| `set_illumination` | yes/no | | | +| `read_device_state` | yes/no | | | + +## Coordinate Frames + +List every coordinate frame exposed to the agent or UI. + +| Frame | Units | Origin | Axes | Conversion owner | +| --- | --- | --- | --- | --- | +| overview pixels | px | | | | +| stage | | | | | +| acquisition volume | voxels | | | | + +Required notes: + +- Which frame is stored in `position_coarse`. +- Which frame is stored in `position_fine`. +- Whether any axis is inverted relative to the overview image. +- Which code owns pixel-to-stage transforms. + +## Safety Limits + +| Device/axis | Lower | Upper | Enforcement layer | How verified | +| --- | --- | --- | --- | --- | +| | | | device `set()` | | +| | | | firmware | | + +Every motion source should have a lowest-layer safety limit. If joystick, +manual controls, or vendor UI can bypass Python checks, document the firmware or +operator procedure that closes that path. + +## State Reporting + +| State field | Source | Poll/callback | Frequency | Read-only | +| --- | --- | --- | --- | --- | +| stage position | | | | yes | +| focus actuator | | | | yes | +| illumination state | | | | yes | +| temperature | | | | yes | + +State reporting endpoints must not mutate hardware. + +## Data Products + +| Product | Format | Storage owner | Metadata required | +| --- | --- | --- | --- | +| overview image | | | sample id, position, pixel size | +| snapshot | | | sample id, channel, exposure | +| volume | | | sample id, voxel size, channel, exposure | + +Large arrays should be written by the device layer or persisted once, then +referenced by uid/path. Avoid repeated JSON/base64 transfer for production data. + +## Sample Metrics Populated + +Check the metrics this profile updates: + +- [ ] `position_coarse` +- [ ] `position_fine` +- [ ] `position_history` +- [ ] `exposure_count` +- [ ] `total_exposure_ms` +- [ ] `focus_history` +- [ ] `signal_intensity_history` +- [ ] `perception_runs` + +## Test Plan + +- [ ] Offline import smoke test +- [ ] Mock device unit tests +- [ ] Hardware availability diagnostic +- [ ] Safety limit rejection test +- [ ] Acquisition dry-run or simulated run +- [ ] Live acquisition test with explicit operator opt-in diff --git a/docs/architecture/sample-hardware-domains.md b/docs/architecture/sample-hardware-domains.md new file mode 100644 index 00000000..8b8ab4b6 --- /dev/null +++ b/docs/architecture/sample-hardware-domains.md @@ -0,0 +1,104 @@ +# Sample and Hardware Domain Model + +Gently's current production profile is a diSPIM imaging C. elegans embryos, but +the control model should be described in terms that also fit other microscopes +and sample types. This document defines that vocabulary. + +## Design Rule + +Application code should talk about samples, observations, acquisitions, and +calibration. Hardware modules translate those concepts to device-specific +plans, device names, and timing details. + +The current `Embryo` state is therefore an implementation of a more general +sample state. It should remain useful for C. elegans while keeping fields and +operations that other sample models can reuse. + +## Concept Mapping + +| Gently concept | Current diSPIM implementation | Other modality examples | +| --- | --- | --- | +| Sample | C. elegans embryo | well, organoid, cell colony, tissue region | +| Sample overview | Bottom-camera widefield image | low-mag confocal tile, plate overview, brightfield montage | +| Sample detection | Embryo detection/marking | well detection, cell segmentation, ROI picking | +| Position calibration | Pixel-to-stage and SPIM alignment | tile registration, well-to-stage map, objective-specific alignment | +| Focus scan | Piezo/galvo focus sweep | Z-stack focus curve, autofocus objective sweep | +| 3D acquisition | Lightsheet volume | confocal Z-stack, widefield deconvolution stack | +| Timepoint | One scheduled sample observation | one image, stack, or multimodal acquisition at a sample/time | +| Perception | VLM/classifier stage reasoning | phenotype call, quality control, event detection | + +## Standard Operation Names + +Hardware profiles should expose conceptually named operations even when their +device implementation is modality-specific. + +| Operation name | Meaning | diSPIM backing operation | +| --- | --- | --- | +| `sample_overview` | Capture a field that can locate samples | bottom camera capture | +| `detect_samples` | Produce sample candidates/ROIs | embryo detector or manual marking | +| `move_to_sample` | Move the instrument to a sample's resolved position | XY stage move | +| `focus_scan` | Search focus around a sample or plane | piezo/galvo focus sweep | +| `position_calibration` | Refine sample position/calibration state | center/verify and SPIM calibration | +| `acquire_volume` | Acquire a 3D observation | lightsheet volume scan | +| `acquire_snapshot` | Acquire a 2D observation | lightsheet snap or overview image | +| `set_illumination` | Configure light source state/power | laser, LED, room light controls | +| `read_device_state` | Report physical state without mutation | device-layer status endpoints | + +Names in tool descriptions, plan metadata, logs, and docs should prefer these +general concepts. The hardware package may still contain files named for the +real devices (`bottom_camera`, `galvo`, `piezo`) because those are implementation +details inside the diSPIM profile. + +## Layering + +1. `gently.core` defines storage, events, coordinates, and image utilities. +2. `gently.harness` defines reusable agent, session, tool, prompt, and planning + mechanics. +3. `gently.organisms` defines biological semantics, such as C. elegans stages. +4. `gently.hardware` defines device profiles and maps standard operations to + concrete devices/plans. +5. `gently.app` composes one organism and one hardware profile into the + microscopy agent. + +Domain state should move upward as structured sample records. Raw device +details should stay in the hardware profile unless the UI is explicitly showing +hardware diagnostics. + +## Plan Naming Convention + +Use `*_plan` for Bluesky/device-layer plans and include the conceptual operation +in the metadata when possible: + +```python +_md = { + "plan_name": "acquire_volume", + "operation": "acquire_volume", + "hardware_profile": "dispim", +} +``` + +Recommended naming: + +| Preferred | Avoid for new public API | Reason | +| --- | --- | --- | +| `sample_overview_plan` | `bottom_camera_plan` | overview generalizes beyond diSPIM | +| `focus_scan_plan` | `piezo_sweep_plan` | focus is the user-facing concept | +| `position_calibration_plan` | `center_embryo_plan` | calibration can apply to many samples | +| `acquire_volume_plan` | `lightsheet_only_plan` | volume acquisition is modality-neutral | + +Existing diSPIM-specific names do not need churn. New public tools, docs, and +metadata should use the conceptual names and point to the diSPIM implementation. + +## Extension Checklist + +A new hardware profile should document: + +- Which device or plan provides `sample_overview`. +- How sample coordinates map to stage coordinates. +- Which acquisition operations are supported: snapshot, volume, timelapse, + multichannel, burst. +- Which state readings are available without moving hardware. +- Which safety limits are enforced in hardware/device classes. +- Which sample tracking metrics are populated by the profile. + +Use `docs/architecture/hardware-profile-template.md` as the starting point. diff --git a/docs/architecture/sample-tracking-metrics.md b/docs/architecture/sample-tracking-metrics.md new file mode 100644 index 00000000..2421d44b --- /dev/null +++ b/docs/architecture/sample-tracking-metrics.md @@ -0,0 +1,122 @@ +# Sample Tracking Metrics + +Sample tracking metrics describe what happened to a sample over time. They are +kept separate from organism-specific interpretation so they can support embryos, +wells, cells, organoids, or tissue regions. + +## Metric Categories + +| Category | Purpose | Examples | +| --- | --- | --- | +| Position | Where the sample was observed | coarse XY, fine XY, position history | +| Exposure | How much light/acquisition burden it received | exposure count, total exposure ms | +| Focus | Whether optical focus is stable | focus history, drift rate | +| Signal | Quantitative image measurements | channel intensity, photobleaching curve | +| Perception | Semantic interpretations | developmental stage, health state, event flags | +| Provenance | How the record was produced | detector id, model version, confidence | + +## Universal Fields + +These fields are broadly useful across sample types and should remain stable +where possible. + +| Field | Type | Description | +| --- | --- | --- | +| `sample_id` | string | Stable id within a session. Current C. elegans code uses `embryo_id`. | +| `sample_uid` | string/null | Optional globally meaningful id. Current code uses `uid`/`embryo_uid`. | +| `role` | string | Experimental role such as test, calibration, control, unassigned. | +| `position_coarse` | object/null | Overview/manual position, usually XY stage coordinates. | +| `position_fine` | object/null | Refined acquisition position for the primary objective/modality. | +| `has_fine_position` | bool | True when fine position should override coarse for acquisition. | +| `position_history` | list | Optional time series of positions and sources. | +| `exposure_count` | integer | Number of acquisitions or exposure events. | +| `total_exposure_ms` | number | Integrated illumination/exposure time. | +| `last_imaged` | ISO datetime/null | Last successful observation time. | +| `focus_history` | list | Focus measurements keyed by position, modality, and score. | +| `signal_intensity_history` | object/list | Per-channel measurements over time. | +| `perception_runs` | list | Links to semantic classifications and reasoning traces. | + +## Domain-Specific Fields + +Domain fields are valuable, but should be clearly scoped to an organism/sample +plugin. + +| Field | Current meaning | Scope | +| --- | --- | --- | +| `developmental_stage` | C. elegans morphology stage | C. elegans organism profile | +| `hatching_status` | C. elegans hatch state | C. elegans organism profile | +| `morphology_history` | Stage/shape observations | organism-specific | +| `custom_classifications` | User-defined labels | experiment-specific | + +## Position Semantics + +Gently distinguishes coarse and fine positions: + +- `position_coarse` comes from an overview image, manual marking, or low-mag + sample map. +- `position_fine` comes from a refined alignment step for the acquisition + modality. +- `stage_position` is a compatibility/read convenience: fine if present, + otherwise coarse. + +Persistence and import/export code should preserve both positions. Updating the +coarse position should clear or invalidate fine position unless the update is +known not to affect fine alignment. + +## Exposure Semantics + +Exposure tracking should describe burden on the sample, not simply image count. + +Minimum record: + +```yaml +sample_id: embryo_1 +timepoint: 12 +modality: lightsheet_volume +frames: 50 +exposure_ms: 10.0 +channels: + 488nm: + laser_power_pct: 5.0 + total_ms: 500.0 +created_at: 2026-05-30T12:00:00 +``` + +For multimodal acquisitions, store one record per channel/modality or include a +structured `channels` map. Do not collapse illumination wavelength and camera +exposure into a single ambiguous number. + +## Focus Semantics + +Focus measurements should include: + +- acquisition modality or objective, +- stage position/context, +- focus actuator position, +- score algorithm, +- score value, +- timestamp, +- source: hardware autofocus, FFT, VLM, manual. + +This allows drift estimates such as micrometers/hour without tying the schema to +one microscope. + +## Perception Provenance + +Every semantic decision should be traceable: + +- model or detector id, +- input image/volume uid, +- prompt or detector config hash when applicable, +- confidence or score, +- reasoning trace path when available, +- timestamp and triggering event. + +This is the bridge between sample tracking and the evaluation/trajectory +debugging systems. + +## Compatibility Notes + +Current `EmbryoState` fields map directly onto this schema. Future sample types +can either implement their own typed state object or reuse a generic +`SampleState` shape, provided API responses keep the universal fields above. diff --git a/docs/full-stack-microscopy.md b/docs/full-stack-microscopy.md new file mode 100644 index 00000000..0188968a --- /dev/null +++ b/docs/full-stack-microscopy.md @@ -0,0 +1,55 @@ +# Full Stack Microscopy + +Gently should be documented as a microscopy system, not only as an agent or a +device controller. The useful view for a biologist or instrument developer is +the full path from experimental intent to stored evidence. + +## Stack Map + +| Stack | What it owns | Gently surface | +| --- | --- | --- | +| Experimental intent | Scientific question, hypothesis, controls, success criteria | plan mode, campaigns, plan items | +| Sample preparation | Organism, strain, treatment, mounting, perturbation | sample records, sample-tracking metrics | +| Hardware integration | Microscope devices, safety limits, device state, calibration | hardware profiles, device layer, profile templates | +| Acquisition | Snapshots, volumes, timepoints, illumination, temperature | acquisition tools, Bluesky plans, session metadata | +| Perception | Detection, classification, quality control, event recognition | perception traces, predictions, reasoning records | +| Closed-loop decisions | When to continue, stop, adapt, or ask the operator | agent tools, event logs, decision logs | +| Data and provenance | Raw data, derived data, logs, plans, exports | FileStore/GentlyStore, session directories, debug bundles | +| Operator experience | Setup, monitoring, intervention, recovery | web UI, chat, settings, docs/tutorials | + +## Documentation Shape + +Generated docs should include three kinds of material: + +- Tutorials: task-focused paths such as "run without hardware", "add a hardware + profile", and "start a safe timelapse". +- Concepts: the full-stack map, sample/hardware domain boundaries, and data + provenance expectations. +- References: API surfaces, command-line flags, storage layouts, hardware + profile checklists, and test markers. + +## Hardware as One Stack + +Hardware is a core stack, but it should not dominate the documentation model. +The device layer matters because it connects intent to physical state safely: +limits, calibration, timing, illumination, and temperature all shape what +scientific claims can be made from the data. + +Hardware docs should therefore connect each device profile to: + +- the sample state it can observe or change +- the safety boundaries it enforces +- the metadata it records +- the simulator or live-hardware tests that cover it +- the operator workflow for setup and recovery + +## Tutorial Roadmap + +Priority tutorials: + +- run Gently offline and create a plan +- connect a local diSPIM device layer +- add a new hardware profile +- add a new organism/sample type +- inspect a stored session and export a debug bundle +- write a hardware contract test and an opt-in live hardware test diff --git a/docs/index.md b/docs/index.md new file mode 100644 index 00000000..08b8b6a6 --- /dev/null +++ b/docs/index.md @@ -0,0 +1,31 @@ +# Gently Documentation + +Gently is a full-stack microscopy system: it joins sample preparation, +instrument control, perception, planning, storage, and operator workflows into +one inspectable loop. + +This documentation is organized around that full stack rather than around a +single API layer. + +## Start Here + +- [Full Stack Microscopy](full-stack-microscopy.md): the system-integration map. +- [Try Without Hardware](guides/try-offline.md): run Gently offline. +- [Hardware Setup](guides/hardware-setup.md): connect a microscope/device layer. +- [Build a Plugin](guides/build-a-plugin.md): add organisms or hardware profiles. + +## Architecture References + +- [Sample and Hardware Domains](architecture/sample-hardware-domains.md) +- [Sample Tracking Metrics](architecture/sample-tracking-metrics.md) +- [Hardware Profile Template](architecture/hardware-profile-template.md) + +## Build Locally + +If MkDocs is installed, run: + +```shell +mkdocs serve +``` + +The docs are plain Markdown, so they also remain readable directly in GitHub. diff --git a/mkdocs.yml b/mkdocs.yml new file mode 100644 index 00000000..b356873f --- /dev/null +++ b/mkdocs.yml @@ -0,0 +1,18 @@ +site_name: Gently +site_description: Full-stack microscopy documentation for Gently +docs_dir: docs +nav: + - Home: index.md + - Full Stack Microscopy: full-stack-microscopy.md + - Guides: + - Try Without Hardware: guides/try-offline.md + - What Gently Can Do: guides/capabilities.md + - Build a Plugin: guides/build-a-plugin.md + - Hardware Setup: guides/hardware-setup.md + - Architecture: + - Sample and Hardware Domains: architecture/sample-hardware-domains.md + - Sample Tracking Metrics: architecture/sample-tracking-metrics.md + - Hardware Profile Template: architecture/hardware-profile-template.md +markdown_extensions: + - admonition + - tables