diff --git a/.gitignore b/.gitignore index 7b10933..9785c95 100644 --- a/.gitignore +++ b/.gitignore @@ -36,5 +36,4 @@ data-cache/ *.nii.gz *.npy *.npz -!src/mouse_brain_planner/assets/vasculature/lambada_p60_606_major_vessels_v1.npz -!src/mouse_brain_planner/assets/vasculature/vessap_bl6j1_major_vessels_50um_v1.npz +*.pdf diff --git a/ATLAS_DATA.md b/ATLAS_DATA.md index 72d47f9..8f88672 100644 --- a/ATLAS_DATA.md +++ b/ATLAS_DATA.md @@ -179,40 +179,15 @@ clearance. The workflow remains archived in the backend and is absent from the p 50 µm prepared grid does not acquire 25 µm vascular resolution merely because it is composited over a 25 µm atlas image. -## Archived LAMBADA major-vessel evidence - -The repository retains a derivative of Renier, de Launoit, and Skriabine's -P60_606 graph, [Zenodo record 18876865](https://zenodo.org/records/18876865), DOI -`10.5281/zenodo.18876865`, CC BY 4.0. It is bound to the accepted Allen 25 µm array shape and -contains maximal consecutive in-bounds runs whose point radius is at least 15 µm (diameter at -least 30 µm). Those deterministic extraction facts do not qualify it as a display layer. - -AP and DV orientation evidence passed, but the source describes hemisphere specimens and the -exact graph has no persisted biological hemisphere/laterality binding. Whole-brain coverage and -ML polarity are unqualified, and the application does not mirror the source. The production -reference endpoints never load this derivative. Nothing from it is overlaid in slices, Dorsal, -or 3D. - -The source also omits pial and choroidal vessels, and the derivative omits smaller vessels. Full -identity, extraction, coordinate qualification, checks, counts, and limitations are in -[LAMBADA Major Vessels](docs/LAMBADA_MAJOR_VESSELS.md) and -[Third-Party Software and Data](THIRD_PARTY.md). - -## VesSAP display-only major-vessel reference - -The current visible reference is a bundled derivative of the public VesSAP `BL6J-no1` 3 µm -whole-brain skeleton/radius volumes. It retains nominal diameter ≥30 µm source centerlines, -preserves true source adjacency, applies the authors' Euler plus B-spline Allen transform with -the reviewed ML reflection, and reduces the paths on a 50 µm display grid. - -The adjacent manifest binds the exact source files, transform, atlas identity, extraction -statistics, NPZ digest, and mandatory limitations. The backend serves metadata and geometry only -after all checks pass; the same geometry is overlaid in all five modes. It is one fixed cleared -C57BL/6J specimen, omits smaller vessels, and has no subject-registration or tissue-distortion -error bound. `vessel.major.reference.analyze` therefore fails with -`VESSEL_ANALYSIS_UNAVAILABLE` before accessing a project or geometry. - -See [VesSAP Major Vessels](docs/VESSAP_MAJOR_VESSELS.md) for the exact data and transform record. +## Optional VesSAP major-vessel reference + +The VesSAP display layer is loaded only when its NPZ and manifest are installed under the +application data directory. The data files are not stored in this repository or application +bundle. The loader validates their source identity, transform, atlas binding, digest, arrays, +bounds, and extraction statistics before exposing the five-view geometry. + +See [VesSAP External Major-vessel Data](docs/VESSAP_MAJOR_VESSELS.md) for the file locations and +data contract. ## No official CCF bregma diff --git a/COORDINATE_SYSTEMS.md b/COORDINATE_SYSTEMS.md index a27ef29..beab722 100644 --- a/COORDINATE_SYSTEMS.md +++ b/COORDINATE_SYSTEMS.md @@ -376,11 +376,6 @@ An ontology identity does not guarantee voxel or mesh geometry. In the reviewed four 2D overlays contain zero selected pixels, and 3D reports that no reviewed geometry exists; no inferred surface is substituted. -The archived LAMBADA P60_606 derivative is not displayed. Its exact qualification found -supporting AP and DV orientation evidence but rejected biological ML/laterality and whole-brain -coverage because the source is a hemisphere specimen and its graph has no persisted hemisphere -binding. Numeric points on both sides of the atlas midpoint do not establish bilateral anatomy. -No mirroring or display-axis guess is permitted. VesSAP remains one fixed cleared reference, not the current animal. The shared coordinate frame permits an overlay but does not supply subject registration, tissue-distortion, or inter-animal diff --git a/DEVELOPMENT.md b/DEVELOPMENT.md index 271e304..5b6a970 100644 --- a/DEVELOPMENT.md +++ b/DEVELOPMENT.md @@ -1,140 +1,47 @@ # Development -Brain3D has one supported product path: a SwiftUI + SceneKit macOS application and a Python 3.12 -scientific service connected by typed NDJSON. Scientific transforms, atlas access, calibration, -probe/region analysis, vessel qualification, and persistence stay in Python; Swift owns -presentation, input, accessibility, native file handling, and schema-checked SceneKit display -payloads. No vessel-analysis capability is currently exposed. - -Start with [Architecture](docs/ARCHITECTURE.md), [Code Audit](docs/CODE_AUDIT.md), -[ADR-005](docs/ADR-005-independent-slice-viewer.md), and -[Scientific Validation](SCIENTIFIC_VALIDATION.md). - -## Environment - -- Apple Silicon (`arm64`) -- macOS 14 or later -- CPython `>=3.12,<3.13` -- Swift 6-compatible toolchain -- committed `uv.lock` +## Setup ```bash uv python install 3.12 -uv lock --check uv sync --frozen --group dev +uv run --frozen mouse-brain-planner atlas download allen_mouse_25um +``` + +Build and open the macOS app: + +```bash +native/Brain3D/Scripts/build-app.sh +open native/Brain3D/build/Brain3D.app ``` -The default dependency graph deliberately excludes PySide6, PyVista, PyVistaQt, and VTK. Do not -reintroduce a second GUI or renderer into the base environment. +The Swift app communicates with the Python service over typed NDJSON. Start with +[Architecture](docs/ARCHITECTURE.md), [Coordinate Systems](COORDINATE_SYSTEMS.md), and the +architecture decisions in [docs](docs/). -## Quality gate +## Checks ```bash uv lock --check -uv sync --frozen --group dev uv run --frozen ruff format --check . uv run --frozen ruff check . uv run --frozen mypy --no-incremental uv run --frozen pytest -q -.venv/bin/python scripts/verify_minimal_runtime.py -swift test --package-path native/Brain3D +swift test --package-path native/Brain3D --no-parallel native/Brain3D/Scripts/build-app.sh codesign --verify --deep --strict native/Brain3D/build/Brain3D.app ``` -The minimal verifier checks that Qt/VTK modules are absent and exchanges real `hello` and -`shutdown` frames through `mouse-brain-planner bridge`. A green gate establishes engineering -consistency, not stereotaxic, vascular-clearance, usability, or surgical accuracy. - -Useful focused runs: - -```bash -uv run --frozen pytest tests/unit/test_atlas_space.py -q -uv run --frozen pytest tests/unit/test_brainglobe_adapter.py -q -uv run --frozen pytest tests/unit/test_slice_renderer.py -q -uv run --frozen pytest tests/integration/test_bridge_stdio.py -q -swift test --package-path native/Brain3D -``` - -The normal Python suite uses synthetic volumes and local fakes. The real density-archive and -real-atlas checks remain explicit opt-in tests because their external data are not committed. - -## Run the product path - -```bash -native/Brain3D/Scripts/build-app.sh -open native/Brain3D/build/Brain3D.app -uv run --frozen mouse-brain-planner atlas list -uv run --frozen mouse-brain-planner atlas download allen_mouse_25um -uv run --frozen mouse-brain-planner validate-project /absolute/path/Plan.mouseplan -``` - -`mouse-brain-planner` without a subcommand prints help. `mouse-brain-planner bridge` is intended -for the native process boundary and integration diagnostics. - -## Source layout +## Repository layout ```text -native/Brain3D/ SwiftUI shell, typed client, native tests/build - Sources/Brain3DScene/ SceneKit brain/probe rendering; archived vessel primitives -src/mouse_brain_planner/ - atlas/ BrainGlobe 2.3.1 boundary - bridge/ versioned service and handlers - coordinates/ named frames, order conversion, transforms - domain/ Pydantic scientific/project models - persistence/ deterministic migrations and atomic packages - rendering/ raster slices and dorsal projection - probes/ source-traceable NP1 and synthetic test catalog - surgery/ product-reachable probe/measurement geometry - analysis/ region traversal plus archived vessel-analysis algorithms - vasculature/ VesSAP display geometry plus archived qualification workflows -tests/ headless unit/contract/integration tests -docs/ audit, architecture, validation, and ADRs +native/Brain3D/ SwiftUI and SceneKit application +src/mouse_brain_planner/ Python atlas and planning service +tests/ Python tests and fixtures +docs/ architecture and technical references +scripts/ verification and benchmark utilities ``` -The old Qt/PyVista/VTK application was removed after the audit in -[Legacy Removal](docs/LEGACY_REMOVAL.md). Its last complete snapshot is Git commit `51fe26d`. - -## Dependency direction - -- Swift never reimplements coordinate transforms or scientific classification. -- The bridge imports no macOS UI framework and must run headlessly. -- Only the BrainGlobe adapter owns upstream atlas objects. -- All API coordinates name their frame, order, units, origin, directions, atlas identity, and - transform/calibration identity. -- Project files store model state and immutable data references, never UI objects or complete - atlas/vessel volumes. -- The rejected LAMBADA derivative, archived population density, and archived 2D subject evidence - remain separate types and workflows. None is a subject-vessel layer. The runtime vessel - geometry endpoint serves only the digest-checked VesSAP display reference; analysis fails - closed with `VESSEL_ANALYSIS_UNAVAILABLE`, and LAMBADA is never loaded. - -## Coordinate-change checklist - -Any change involving axes, units, indexing, laterality, transforms, picking, or projection must: - -1. name the source and destination frames and units; -2. use centralized BrainGlobe `[AP,DV,ML]` ↔ domain `[AP,ML,DV]` conversion; -3. validate finite values and half-open atlas bounds; -4. distinguish continuous voxel coordinates, discrete indices, corner anchors, and centers; -5. add asymmetric laterality and forward/inverse golden tests; -6. require either the exact source-pinned v4 bregma/surface evidence or, for legacy v1–v3, - a passing subject calibration; never treat atlas origin as bregma; and -7. update coordinate/validation documentation with exact evidence and tolerances. - -## Bridge-change checklist - -- Preserve strict envelope and exact-parameter validation. -- Reject bool where a number is expected, plus NaN and infinity. -- Use stable error codes and include recoverability/suggested action in protocol v2 errors. -- Every mutation checks the expected project revision and returns the new revision. -- Long work runs off the UI thread and stale results cannot overwrite current state. -- Add Python contract tests, matching Swift Codable tests, and a production handler/capability - inventory assertion. -- Do not send full atlas volumes or platform-specific index objects across NDJSON. - -## Review discipline - -Before committing, inspect `git diff --check`, run the relevant focused tests, then run the full -gate in proportion to the change. Never weaken frame/provenance validation to make a test pass. -Update [Code Audit](docs/CODE_AUDIT.md) when a previously orphaned module becomes product-reachable. +Atlas volumes, vascular datasets, PDF inputs, and user projects stay outside source control. +Use `MOUSE_BRAIN_PLANNER_CONFIG_DIR`, `MOUSE_BRAIN_PLANNER_DATA_DIR`, and +`MOUSE_BRAIN_PLANNER_CACHE_DIR` for isolated development environments. diff --git a/KNOWN_LIMITATIONS.md b/KNOWN_LIMITATIONS.md index 8fd472f..236fd46 100644 --- a/KNOWN_LIMITATIONS.md +++ b/KNOWN_LIMITATIONS.md @@ -1,131 +1,35 @@ # Known Limitations -Brain3D is an engineering testing build for non-human mouse research. It is not a medical, -clinical, veterinary, or qualified surgical-navigation product, and it has no prospective animal, -phantom-targeting, histological-outcome, or formal usability validation. - ## Atlas and coordinates -- Only BrainGlobe `allen_mouse_25um` package `1.2` is accepted. The 25 µm value is sampling - resolution, not targeting accuracy. The 10 µm atlas is outside the testing phase. -- The Allen CCF is a population reference from fixed brains and does not provide one official, - subject-specific bregma/lambda transform. -- Direct v4 AP/ML uses a source-pinned Pinpoint/Urchin population-atlas profile (AP− posterior, - ML− animal-left), not the individual animal's measured bregma. Its annotation-derived surface - is not a measured pia or skull surface. -- Legacy v1–v3 AP/ML/DV projection remains dependent on its stored matched landmarks, frame - definitions, laterality, QC thresholds, and uncertainty. -- Region labels are annotation-voxel results, not subject histology. Boundary, hemisphere, and - registration errors remain possible. -- The complete ontology can contain a named structure with no voxels in the reviewed annotation. - In `allen_mouse_25um` v1.2, `RSPd4` (structure 545) is such an ontology-only entry. Brain3D - keeps the selection visible but reports zero 2D highlight pixels and no reviewed 3D geometry; - it does not synthesize or infer a mesh for anatomy absent from the annotation. - -## Native views - -- Dorsal is an AP/ML atlas reference projection. It shows selected-probe landmarks and shank paths - but intentionally omits the collapsed recording-site cloud. It shows the VesSAP display-only - major-vessel projection; missing lines must not be interpreted as absence of vessels. -- SceneKit 3D is a display and picking view of backend-validated geometry payloads. Camera - interaction and rendering do not add anatomical accuracy. -- Probe and vessel overlays are clipped/projected according to documented slice/slab rules. -- There is no capillary layer, artery/vein classification, craniotomy design, manipulator/hardware - collision model, or intraoperative tracking. - -## Probe model and placement - -- The two supported choices—Neuropixels 2.0 single-shank (1,280 sites) and standard four-shank - (5,120 sites)—are complete source transcriptions but remain - `source-transcribed-review-pending`. No independent human has reviewed every coordinate and - dimension against the pinned sources or physical probes. -- The NP2 four-shank target/entry belongs to the official leftmost `shank-0`; the other shanks - follow the probe-local lateral axis at 250 µm pitch. The software does not infer a centered-array - target or independently rotate the shanks. -- NP2 plans expose all 1,280 physical sites per shank for geometry and region inspection. They do - not import the actual IMRO/electrode selection, and must not imply that all sites are recorded - simultaneously. Both supported hardware choices provide 384 simultaneously configurable - recording channels. -- V4 depth is path length from a 25 µm annotation boundary. Its single signed sagittal angle and - two whole-array layouts do not model arbitrary ML tilt, probe bending, or independent shank - rotation. -- Manufacturer tolerances, insertion bending, tissue deformation, brain shift, skull mounting, - probe-base/headstage geometry, and manipulator backlash are not modeled. -- The generic 16-site model is synthetic software-test geometry, not a physical device profile. -- Region traversal and site assignment use the selected Allen annotation and the persisted v4 - surface/profile evidence (or a preserved legacy calibration); they do not predict histological - recording location or post-insertion displacement. - -## VesSAP display-reference limits - -- The visible layer is one fixed, cleared, ex-vivo adult C57BL/6J specimen (`BL6J-no1`), not the - current animal and not live vasculature. -- Only source skeleton points with nominal radius ≥15 µm (diameter ≥30 µm) are retained. - Capillaries and smaller vessels are intentionally absent. -- Source topology is coalesced on a 50 µm display grid. Nearby branches can merge and isolated - display voxels without a retained segment are omitted. -- Pial/choroidal coverage is not separately classified. Artery/vein identity is unavailable. -- The published Euler + B-spline atlas transform was checked for axis order, laterality, bounds, - and label agreement, but no target-registration-error, bregma/skull-registration-error, clearing - distortion, or inter-animal variation bound is published. -- The nonlinear transform is anisotropic. The displayed scalar tube radius is a visual reference, - not a qualified atlas-space vessel surface. -- The layer cannot establish clearance, absence of a vessel, trajectory suitability, or safety - for an individual animal. It remains CC BY-NC 4.0. - -See [the exact derivation and validation record](docs/VESSAP_MAJOR_VESSELS.md). +- Brain3D currently supports BrainGlobe `allen_mouse_25um` package `1.2`. +- Atlas sampling resolution is separate from stereotaxic targeting accuracy. +- The Allen CCF is a population reference; direct AP/ML planning uses the documented + Pinpoint/Urchin reference profile. +- Region labels and meshes follow the installed atlas package. -## Archived LAMBADA major-vessel evidence +## Probe planning -- The derivative is archived evidence only. It is not loaded, served, displayed, mirrored, or - analyzed by the current runtime. -- AP and DV orientation evidence passed, but the primary source describes hemisphere specimens - and the exact graph has no persisted biological hemisphere/laterality binding. Whole-brain - coverage and ML polarity are unqualified. -- The source workflow removed pial and choroidal vessels. The bundled derivative also removes - points below 15 µm radius (30 µm diameter), so missing vessels are expected. -- The sampled biological side is not bound in the graph; artery/vein identity is unavailable. -- The source corrected/reconnected endpoints and removed short terminal offshoots. Some paths are - reconstructed rather than directly observed. -- Radius conversion uses the source's mean atlas-resampling scale rather than a local Jacobian. - Fixed/cleared tissue distortion, atlas registration error, biological variation, and omitted - vessels are not bounded. -- Out-of-bounds or rejected points split runs and are omitted without clipping/interpolation. -- No source-backed mirror transform is qualified, so mirroring is prohibited. +- The primary workflow supports NP2003 and NP2013. +- Probe geometry is source-transcribed and should be checked against the hardware used in a study. +- Planning does not model probe bending, tissue deformation, brain shift, manipulator backlash, + or headstage collisions. -See [the full extraction record](docs/LAMBADA_MAJOR_VESSELS.md). +## Vascular overlay -## Vessel analysis unavailable +- VesSAP data is optional and installed outside the repository. +- The reference represents one cleared specimen and omits vessels below the configured diameter + threshold. +- The overlay is a display layer; the application does not expose vessel-clearance analysis. -The runtime advertises `auditedReferenceMajorVessels` for VesSAP metadata and display geometry, -but not `radiusAwareReferenceVesselAnalysis`. Analysis fails with -`VESSEL_ANALYSIS_UNAVAILABLE` before reading points or touching a project. Legacy V3 algorithm -code and synthetic tests do not authorize product use. This build emits no vessel intersection, -threshold violation, absence, clearance, suitability, or safety result. +## Export -The older LAMBADA path remains independently rejected. Its digest-bound report is -[`docs/evidence/lambada_p60_606_coordinate_qualification_rejected_v1.json`](docs/evidence/lambada_p60_606_coordinate_qualification_rejected_v1.json), -SHA-256 `0993d5a0ad6c0d62094dc395fe2bc4f284870e6e7c0b602be7df5a7da867c93a`. +- PDF export requires user-selected protocol and reference-atlas documents. +- Historical atlas-page matching is coordinate based and does not register the page to an + individual subject. -## Archived and distribution paths +## Platform -- Surgery-plan export depends on the user-prepared Headplate protocol PDF and the user-supplied - 132-page `MBSC_Figs_with_Layers.pdf`. Those files and their rights are not bundled or - relicensed. Their locations are configured once in **Brain3D → Settings** and reused until - replaced. A disconnected volume must be reconnected or selected again. Export reads both PDFs - directly and does not require Word, Illustrator, or macOS Automation permission. -- Plate matching chooses the nearest reviewed coronal AP or sagittal `|ML|` coordinate. Coronal - AP follows the historical atlas convention `Bregma = Interaural − 3.80 mm`. The appended - historical plate is a coordinate reference, not a subject registration. Its output copy gets - an identity/plan summary, but no claimed trajectory transform is drawn on the artwork. -- A FINAL surgery-plan label records exact saved-project and current v4 profile/surface/probe - provenance gates. It does not establish biological, vascular, procedural, or surgical - validity. Preserved legacy exports retain their original target/calibration requirements. -- Population vascular density and subject-image registration remain archived as compatibility - code and persisted data. Their methods/capabilities are not registered by the primary bridge; - they are absent from the UI and are not used as vessel paths or clearance geometry. -- `build-app.sh` creates an ad-hoc-signed development bundle that depends on the source checkout. - A deterministic bundled runtime, SBOM, Developer ID signing, notarization, and clean-Mac release - qualification are not complete. -- Automated tests establish software invariants only. They do not establish biological, - stereotaxic, vascular, hardware, or procedural accuracy. +- The native application targets Apple Silicon and macOS 14 or later. +- Development builds use the repository Python environment; standalone builds use the release + script. diff --git a/LICENSE b/LICENSE index 65ad8e5..e9e64b1 100644 --- a/LICENSE +++ b/LICENSE @@ -7,10 +7,5 @@ code. No permission is granted to copy, modify, publish, distribute, sublicense, original source code without separate written authorization from the applicable copyright holders. -Third-party software and scientific data retain their own licenses and terms. See THIRD_PARTY.md. - -For clarity, the files -`src/mouse_brain_planner/assets/vasculature/vessap_bl6j1_major_vessels_50um_v1.npz`, -its adjacent manifest, and `VESSAP_DATA_LICENSE.txt` are adapted VesSAP data governed by -Creative Commons Attribution-NonCommercial 4.0 International (CC BY-NC 4.0). They are excluded -from the all-rights-reserved terms above and are not relicensed by this project. +Third-party software and externally supplied scientific data retain their own licenses and terms. +See THIRD_PARTY.md. diff --git a/PACKAGING.md b/PACKAGING.md index 1e9ae4f..ce31ec1 100644 --- a/PACKAGING.md +++ b/PACKAGING.md @@ -1,69 +1,38 @@ # Packaging and Distribution -The repository currently creates an ad-hoc-signed **development** `.app`. It is not a standalone -release: the executable still discovers Python and the bridge from the source checkout. +## Development bundle ```bash native/Brain3D/Scripts/build-app.sh codesign --verify --deep --strict native/Brain3D/build/Brain3D.app -open native/Brain3D/build/Brain3D.app ``` -`Scripts/build-app.sh` builds the Swift executable, creates the bundle layout, installs the -reviewed `Info.plist`, and applies an ad-hoc signature. It does not bundle `.venv`, Python, atlas -data, or project files. The archived LAMBADA derivative currently resolves from the repository's -Python package beside the source checkout; it is not copied into the development `.app` by this -script and the runtime gate never loads or serves it. +The development bundle uses the repository Python environment. -## Current release blockers +## Standalone Apple Silicon bundle -- deterministic Python 3.12 runtime/backend bundling is not implemented; -- backend discovery still depends on a development checkout; -- Developer ID signing, hardened runtime, entitlements, notarization, and stapling are absent; -- no clean-account/clean-Mac qualification has been completed; -- the build has no SBOM or assembled third-party license bundle; and -- the implemented workflow remains unqualified for animal procedures. - -The removed Qt/PyVista/VTK stack is not a packaging fallback and is not part of the lockfile. - -## Intended distribution shape - -```text -Brain3D.app/ - Contents/ - MacOS/Brain3D native SwiftUI executable - Resources/ reviewed UI resources, notices, qualification evidence - Frameworks/ or Resources/ deterministic Python runtime and scientific service +```bash +native/Brain3D/Scripts/build-release-app.sh ``` -Atlas archives, the optional archived population-density source, subject images, original vessel -graphs, and `.mouseplan` projects remain outside the signed bundle. The P60_606 derivative is not -a reviewed application asset: its hemisphere/laterality and whole-brain qualification is -rejected. A build may retain its manifest, attribution, and canonical rejection report as -evidence, but must not package or advertise it as runtime vessel geometry. Any future vessel -source requires a new qualification and release decision. The app records immutable source -identities and manages user-owned caches. +The release builder freezes the locked Python service, assembles `Brain3D.app`, creates +`Brain3D-macOS-arm64.zip`, verifies Mach-O architecture and deployment targets, checks signatures, +and performs an isolated bridge/app smoke test. Set `OUTPUT_DIR` to choose another destination. -## Release process still to implement +When a Developer ID Application identity is available, the builder uses hardened-runtime signing. +Otherwise it produces an ad-hoc-signed build. Notarization can be performed after the signed build +is produced. -1. Select and document a deterministic Python embedding strategy that preserves Python 3.12 and - the locked wheels without adding a second GUI stack. -2. Build the backend from a clean checkout and verify the minimal runtime script inside the - candidate bundle. -3. Generate an SBOM and license/notice inventory from the exact bundled artifacts. -4. Sign every nested executable/library with the intended Developer ID and reviewed entitlements. -5. Sign the outer bundle, enable the hardened runtime, notarize, staple, and verify offline. -6. Test Finder launch, bridge lifecycle, atlas download/open, save/reopen, update/replacement, - crash recovery, and uninstall behavior on a clean supported Mac/account. -7. Repeat the complete scientific and usability qualification on the immutable release artifact. +## External inputs -Example verification commands after a future signed build exists: +The application bundle does not include: -```bash -codesign --verify --deep --strict --verbose=2 Brain3D.app -spctl --assess --type execute --verbose=4 Brain3D.app -xcrun stapler validate Brain3D.app -``` +- BrainGlobe atlas packages; +- vascular datasets; +- surgery protocol or reference-atlas PDFs; +- user `.mouseplan` projects; or +- subject data. -Passing signing/notarization checks establishes code-distribution integrity only. It does not -establish coordinate, anatomy, vessel, probe, or surgical accuracy. +Runtime data is loaded from application-owned directories or paths selected in Settings. +Dependency notices, build metadata, and the generated SBOM are stored under +`Contents/Resources/Release`. diff --git a/PLAN.md b/PLAN.md deleted file mode 100644 index 8428764..0000000 --- a/PLAN.md +++ /dev/null @@ -1,75 +0,0 @@ -# End-to-End Implementation Plan - -This plan separates implemented engineering behavior from scientific and distribution -qualification. “Implemented” means reachable through the SwiftUI + SceneKit + Python product -path, with typed protocol coverage; it does not mean validated for an animal procedure. - -| Phase | Deliverable | Current state | -| --- | --- | --- | -| 0 | Baseline, reachability audit, risk inventory, architecture ADRs | Complete | -| 1 | One supported GUI and minimal dependencies | Complete | -| 2 | Independent single-view atlas navigation | Implemented | -| 3 | Source-pinned AP/ML profile and exact local annotation-surface entry | Implemented; population-atlas assumption only | -| 4 | Source-traceable NP2003/NP2013 catalog and direct depth/angle/layout placement | Implemented; supported NP2 independent review pending | -| 5 | Exact region traversal, site mapping, inspection, export | Implemented | -| 6 | Major-vessel source, provenance, and 2D overlays | VesSAP display-only overlay implemented; P60_606 archived | -| 7 | Tapered-radius reference vessel analysis | Algorithm tests retained; production capability absent | -| 8 | Native 3D brain/probe/vessel view and picking | Implemented with SceneKit | -| 9 | Animal-study validation and production distribution | Not started | - -## Implemented engineering checkpoint - -- The mode bar is exactly `Dorsal / Coronal / Sagittal / Horizontal / 3D`, with one selected - view. Coronal, sagittal, and horizontal retain independent depths. -- Slider, previous/next, wheel, pan, zoom, and reset share one verified 25 µm slice state. A - click replaces one compact region label without changing depth. -- The complete 840-structure Allen hierarchy and search are shared by all five modes. Selecting a - cortical or non-cortical structure replaces one global identity and lazily loads only that 3D - mesh; it does not couple the three slice depths. -- V4 plans persist the named Pinpoint/Urchin reference, annotation identity, exact local surface, - surface-entry AP/ML, path depth, signed sagittal angle, layout, and model snapshot. AP− is - posterior and ML− is animal-left. A separate target registration/calibration is not a v4 - prerequisite. -- Legacy subject calibrations and target records remain preserved with their original identity, - transforms, residuals, QC, uncertainty, version, and hash. -- The production probe catalog contains only NP2 single shank (1,280 sites / 384 channels) and - standard four shank (5,120 sites / 384 channels). Quad Base, NP1, and the synthetic fixture are - archived compatibility/test definitions and are absent from the new-plan selector. - Manufacturer entries stay `source-transcribed-review-pending` until independent full-table - reviews are recorded. -- Probe plans expose entry, tip, shank envelope, and recording sites in slice and 3D - views. Exact voxel traversal drives region inspection and CSV/JSON export. -- The VesSAP `BL6J-no1` diameter-≥30 µm reference is served after exact source/asset/transform - checks and overlaid in all five views. True source-skeleton adjacency is retained through a - 50 µm display reduction. It is one cleared population reference, not subject-specific. -- Vessel clearance analysis fails closed with `VESSEL_ANALYSIS_UNAVAILABLE`; missing numeric - subject-registration/tissue-distortion bounds cannot be replaced by user acknowledgements. - The V3 algorithm remains isolated engineering/test evidence, not a production feature. -- The older LAMBADA P60_606 derivative remains archived and is never rendered, served, mirrored, - or analyzed because hemisphere/laterality and whole-brain coverage failed qualification. -- Population density and subject-image registration are archived backend paths, not primary UI - features. The 10 µm atlas is excluded from this testing phase. - -## Phase 9 — evidence and distribution gates - -The remaining program is qualification rather than another UI feature sweep: - -1. independently review every supported NP2 transcribed dimension and all encoded site coordinates; -2. define and execute phantom, histology, targeting-error, and repeated-observer studies; -3. quantify calibration and registration uncertainty against declared ground truth; -4. obtain subject-specific vascular ground truth with measured bregma/subject registration, - clearing/live-tissue, and inter-animal uncertainty before validating any vessel analysis; -5. run formal animal-workflow usability and failure-recovery studies; -6. establish performance and visual-regression acceptance thresholds on supported Macs; -7. bundle a deterministic Python runtime and produce an SBOM; -8. complete Developer ID signing, hardened-runtime review, notarization, and clean-Mac tests; and -9. complete a final animal-only/non-human/non-clinical requirement audit. - -The VesSAP layer is display-only and cannot establish vessel absence, clearance, suitability, or -safety; see [its exact evidence record](docs/VESSAP_MAJOR_VESSELS.md). The older P60_606 -derivative is not a layer. Its source is a hemisphere specimen, has no persisted -biological laterality binding, excludes pial and choroidal vessels upstream, and the derivative -filters smaller vessels. The runtime rejects it before producing any conflict or no-conflict -result. The canonical decision is recorded in -[`docs/evidence/lambada_p60_606_coordinate_qualification_rejected_v1.json`](docs/evidence/lambada_p60_606_coordinate_qualification_rejected_v1.json), -SHA-256 `0993d5a0ad6c0d62094dc395fe2bc4f284870e6e7c0b602be7df5a7da867c93a`. diff --git a/PROJECT_STATUS.md b/PROJECT_STATUS.md deleted file mode 100644 index 413e50f..0000000 --- a/PROJECT_STATUS.md +++ /dev/null @@ -1,114 +0,0 @@ -# Project Status - -Status reviewed: 2026-07-25 - -## Bottom line - -The current development tree implements the native research-planning path for independent 25 µm -atlas slices, direct atlas-surface NP2003/NP2013 planning, shared region highlighting, and a -synchronized SceneKit 3D view. A pinned VesSAP C57BL/6J major-vessel reference is visible in all -five views; clearance analysis remains unavailable. The application remains an engineering -testing build—not a qualified distribution or a validated animal-surgery navigation system. - -The source is public at [KaiCao2003/brain3d](https://github.com/KaiCao2003/brain3d). - -## Current product path - -| Area | Implemented behavior | Important boundary | -| --- | --- | --- | -| Native workspace | Exactly `Dorsal / Coronal / Sagittal / Horizontal / 3D`, one full-size view | No focus mode, crosshair, or 2×2 layout | -| Atlas regions | Complete paged 840-structure ontology, common hierarchy/search/selection, descendant-inclusive annotation highlight in all four 2D modes, and the same selected mesh in 3D | Selecting a region never moves or couples slice depths; zero-voxel structures are not fabricated | -| Atlas slices | Independent persisted depths, buttons/slider/wheel, editable one-based slice number, pan/zoom, click-to-replace region label, and explicit atlas-physical coordinate header | `allen_mouse_25um` v1.2 only; 10 µm excluded from testing; header coordinates are not bregma-relative | -| 3D | SceneKit whole-brain mesh, camera control/reset, non-cortical region highlight, probe envelopes, and major-vessel tubes | Rendering consumes schema-checked brain/probe/vessel geometry from the backend | -| Coordinates | Required animal-only context plus signed surface-insertion AP/ML millimetres from a named Pinpoint/Urchin profile, local annotation-surface depth, one sagittal angle, and layout | AP+ anterior/AP− posterior; ML+ right/ML− left; positive angle A→P/negative P→A; the profile is not Allen-official or animal-specific | -| Direct surface | Exact superior boundary of the first nonzero annotation voxel at the resolved AP/ML column; persisted source/digest and load-time rederivation | Population-atlas surface, not measured pia/skull/current-animal anatomy | -| Probes | Primary selector contains only `NP2003` and `NP2013`; normal NP2013 layout is sagittal and `90°` is clockwise from dorsal; numeric edits commit on Return/focus loss and atomic selector changes update immediately | Built-ins are source-transcribed and review-pending, not independently verified; v4 has no target/calibration prerequisite, geometry checkbox, or Apply button | -| Major vessels | VesSAP BL6J-no1, nominal diameter ≥30 µm, overlaid on Dorsal/Coronal/Sagittal/Horizontal/3D; visible-diameter filter is adjustable from 30–250 µm | One cleared ex-vivo population reference; geometry is reduced on a 50 µm spatial grid; no capillaries | -| Surgery-plan PDF | Direct-PDF two-page protocol prefill, one plan-centred view or all five, then one AP/`|ML|`-matched page from `MBSC_Figs_with_Layers.pdf`; v4 pages print AP/ML, local-surface depth, signed angle, and layout | User-owned PDF locations are saved once in Settings and are not bundled; disconnected volumes must be reconnected; planning titles/coordinates and final order are verified | -| Vessel analysis | Unavailable; geometry is display-only and analysis returns `VESSEL_ANALYSIS_UNAVAILABLE` | No conflict, no-conflict, clearance, absence, suitability, or safety claim is produced | -| Projects | Schema-9 revisioned, checksummed `.mouseplan` save/open, migrations, backup recovery, v4 surface rederivation, exact probe-model validation, and legacy v2/v3 reconstruction | Schema-8→9 preserves existing v1–v3 targets/calibrations without guessed conversion; stale sources, altered geometry, forged surface evidence, and invalid legacy fits fail closed | -| Draft lifecycle | One main planning window with an app-owned probe draft; same-context window/sidebar reconstruction preserves typed values; open/reconnect/quit and plan changes are dirty-state guarded | Closing the main window does not quit the app or silently discard the current draft | - -Population vascular density and subject dorsal-image registration remain archived compatibility -code and persisted data. Their methods/capabilities are not registered by the primary bridge, -they are not shown in the primary UI, and they are not interpreted as vessel paths. - -## Probe evidence state - -The primary selector exposes only the complete 1,280-site `NP2003` and 5,120-site standard -four-shank `NP2013`; both have 384 simultaneous channels. `NP2004`/`NP2014` remain cited where -the pinned sources group equivalent physical geometries, not as extra UI choices. Quad Base, -NP1, and the synthetic fixture remain archived definitions for old-project compatibility and -tests. Digests, retrieval dates, coordinate rules, product identities, shank dimensions and -offsets, tip geometry, references, and banks are retained. - -Its exact status is `source-transcribed-review-pending`. Independent full-table review has not -been completed. The v4 UI does not add a per-plan geometry checkbox and does not describe either -model as independently verified. The generic 16-site entry is a synthetic software-test model. - -Project validation compares every catalog-owned persisted model against its exact source-pinned -definition, including provenance, verification state, shank and tip geometry, and the complete -ordered recording-site table. A known catalog ID with an unknown version, or a changed model with -recomputed surrounding hashes, is rejected. Unknown custom identities are limited to historical -v1 audit/review and cannot enter current planning geometry or analysis. - -## Vessel evidence state - -The runtime reference comes from VesSAP `BL6J-no1`, one fixed cleared adult C57BL/6J brain, -[DOI `10.1038/s41592-020-0792-1`](https://doi.org/10.1038/s41592-020-0792-1), -CC BY-NC 4.0. Source skeleton voxels with radius ≥5 on the 3 µm grid are traced through true -26-neighbour adjacency, transformed through the authors' Euler + B-spline Allen registration, -and coalesced on a 50 µm display grid. The exact NPZ is 1,853,131 bytes, -SHA-256 `9300dacf25ca57a5d23377ca0dc885e34ff0d18e8d21ef7590c6dcd156cf5db7`, -with 196,377 points, 76,622 runs, and 119,755 segments. - -Deterministic sampling validated the required `[T_y,T_z,T_x] → [AP,DV,ML]` permutation, mandatory -ML reflection, atlas bounds, and label agreement. This establishes a trustworthy transform -interpretation for display—not biological surgical accuracy. No numeric subject-registration, -clearing-distortion, or inter-animal bound exists, and nonlinear deformation changes a circular -source radius anisotropically. - -The runtime therefore advertises `auditedReferenceMajorVessels` but not -`radiusAwareReferenceVesselAnalysis`. Metadata and geometry are served after exact identity and -buffer checks; analysis fails before project access with `VESSEL_ANALYSIS_UNAVAILABLE`. The -archived LAMBADA P60_606 derivative remains rejected and is never displayed or mirrored. -See [the VesSAP derivation and validation record](docs/VESSAP_MAJOR_VESSELS.md). - -## Engineering evidence and remaining qualification work - -Python tests cover source integrity, coordinate transforms, direct AP/ML sign handling, exact -annotation-surface resolution, positive/negative sagittal angle and layout direction, probe -placement, region overlays, voxel traversal, the vessel display/analysis boundary, persistence, -and real cached atlas paths. Persisted-plan tests rederive v4 surface geometry and reconstruct -legacy v2/v3 geometry from preserved planning inputs; they reject forged surface evidence, -translated/rehashed geometry, same-target alternate-angle geometry, projection-digest forgeries, -changed/rehashed catalog model snapshots, and legacy calibrations that violate AP ordering or -atlas midline/laterality semantics. Calibration tests independently rerun both production fits -and reject forged matrices, correspondences, residuals, leveling angles, and QC. Historical v1 -records remain loadable for review but cannot enter 2D/3D planning overlays, PDF planning pages, -or region analysis until updated; vessel-clearance analysis is unavailable for every version. -Legacy synthetic tests preserve isolated tapered-geometry contracts, but the production bridge -cannot expose them. Swift tests cover strict protocol decoding, independent view state, viewport -math, complete ontology paging/closure, slice overlays, SceneKit transforms, bounded mesh -caching, descendant-inclusive 2D region compositing, a singleton main-window policy with -app-owned same-context draft retention, and an off-screen composite containing whole brain, -Thalamus, NP2, and VesSAP vessels. -Surgery-export tests cover the exact 132-page atlas catalog, historical Bregma/Interaural -convention, AP/`|ML|` matching, direct protocol-PDF overlay units, per-page vessel -identity/count disclosure, bounded long subject/target identity, exact planning-page coordinate -retention, and deterministic ordered page assembly. -The app can be built and ad-hoc signed for development and is exercised as a real macOS process. - -Viewer-only slice and pick updates use a targeted viewer-state validator and copy only those -fields onto the already validated project graph. This removes unrelated trajectory -reconstruction from interactive navigation; project create/open/save, surgery-plan mutation, -analysis, and export retain full semantic validation. - -That evidence validates software behavior, not biological or procedural accuracy. Remaining work -before any qualified distribution includes independent supported-NP2 transcription review, reference- -and subject-ground-truth studies, measured workflow/error studies, formal usability work, a -bundled deterministic Python runtime, Developer ID signing, notarization, and clean-Mac -qualification. - -No prospective animal study, phantom targeting study, histological outcome study, or formal -clinical/veterinary-device validation is claimed. diff --git a/README.md b/README.md index 8b214c4..81be508 100644 --- a/README.md +++ b/README.md @@ -1,236 +1,57 @@ # Brain3D -Brain3D is a macOS research planner for mouse stereotaxy and Neuropixels placement. The current -development tree provides one native, single-view workspace with exactly five modes: -**Dorsal / Coronal / Sagittal / Horizontal / 3D**. +Brain3D is a native macOS application for mouse-brain atlas exploration, stereotaxic planning, +Neuropixels placement, and surgery-plan export. -> **Animal research only — non-human and non-clinical.** Brain3D is an engineering testing build, -> not a qualified surgical-navigation or veterinary device. Its atlas reference, coordinates, -> and probe outputs require independent review before an animal procedure. The current build displays a -> population-reference major-vessel layer, but exposes no vessel-clearance or safety result. +## Highlights -The repository is public at [KaiCao2003/brain3d](https://github.com/KaiCao2003/brain3d). -Published source is available for review, but the current tree is not an installable production -release or a validation claim. +- Dorsal, coronal, sagittal, horizontal, and SceneKit 3D views. +- BrainGlobe `allen_mouse_25um` atlas support with ontology search and region highlighting. +- Direct AP/ML surface planning for Neuropixels 2.0 single- and four-shank probes. +- Synchronized probe geometry across slice and 3D views. +- Optional external VesSAP major-vessel overlay. +- Checksummed `.mouseplan` projects and PDF surgery-plan export. -## Implemented workflow +## Requirements -| Capability | Current development behavior | -| --- | --- | -| Atlas | BrainGlobe `allen_mouse_25um` package `1.2` only; 10 µm is excluded from this testing phase | -| Slice navigation | One full-size coronal, sagittal, or horizontal view; each retains an independent depth with buttons, an editable one-based slice number, slider, wheel, pan, and zoom; the header labels its atlas-native physical coordinate | -| Region inspection | Complete 840-structure Allen ontology search/browse; one shared selection drives descendant-aware 2D/3D highlighting without changing slice depth; ontology-only entries with no reviewed voxels or mesh remain selected with an explicit no-geometry message | -| Dorsal | Atlas surface with the selected probe's AP/ML entry and shank projection plus display-only major vessels | -| 3D | Native SceneKit brain mesh with camera control, atlas-region picking/highlight, probes, and major-vessel tubes | -| Direct implant definition | AP and ML locate user-facing Shank 1 at an exact local annotation-surface crossing; depth runs from that crossing to its distal target; one signed sagittal angle and one layout orientation | -| Probes | The primary selector contains only `NP2003` (1 shank) and `NP2013` (4 shanks); exact source-pinned model snapshots; automatic committed edits and slice/3D overlays | -| Vessels | VesSAP BL6J-no1 diameter-≥30 µm display reference overlaid in all five views; no capillary layer | -| Surgery plan | Prefilled two-page protocol + selectable Dorsal/Coronal/Sagittal/Horizontal/3D planning pages + one coordinate-matched legacy atlas page | -| Reference analysis | Unavailable: clearance calls fail closed with `VESSEL_ANALYSIS_UNAVAILABLE`; geometry remains display-only | -| Persistence | Schema-9 checksummed `.mouseplan` packages with revisions, provenance, migrations, backup recovery, v4 surface rederivation, and probe/model reprojection checks; legacy v1–v3 records remain preserved | +- Apple Silicon Mac +- macOS 14 or later +- Python 3.12 +- Swift +- [uv](https://docs.astral.sh/uv/) -Population vascular density and subject-image registration remain archived compatibility code and -persisted data only. Their methods and capabilities are not registered by the primary bridge, they -are absent from the planning UI, and they are not substituted for vessel paths. There is no focus -mode, crosshair, 2×2 layout, or capillary layer. - -## Coordinate contract - -The primary implant controls are the surface insertion AP/ML, surface-relative depth, insertion -angle, and probe layout. The card labels the axes `AP (+A / −P)` and `ML (+R / −L)`. AP and ML -locate the insertion site on the atlas surface in millimetres from the explicitly named -Pinpoint/Urchin profile: - -| Axis | Positive | Negative | -| --- | --- | --- | -| AP | anterior / forward | posterior / back | -| ML | right | left | - -For example, `AP -1.25`, `ML -0.70` means 1.25 mm posterior and 0.70 mm left. Brain3D resolves -that AP/ML column in the loaded 25 µm annotation and uses the superior boundary of its first -non-background voxel as the surface crossing for user-facing **Shank 1** (catalog ID `shank-0`). -The AP/ML values are therefore neither an array-centre coordinate nor a point inside the brain. -**Depth (mm)** is positive path length from that Shank 1 surface crossing to its distal target—not -a DV coordinate and not a distance from the atlas box. Angle `0°` inserts inward; a positive angle -advances from anterior toward posterior (A→P), while a negative angle advances from posterior -toward anterior (P→A). - -For NP2013, `Sagittal` keeps the four-shank plane parallel to the sagittal plane and makes Shank 1 -the most anterior shank; the other three extend posterior at 250 µm pitch. `90° CW` rotates the -whole array clockwise as viewed dorsally and makes Shank 1 animal-left-most; the other three extend -toward animal right. In 3D, each NP2 shank is the complete catalogued 10 mm shaft from proximal end -to distal tip. At depth `d`, the remaining `10 − d` mm extends proximally from the surface, normally -outside the brain. Slice overlays, atlas-region traversal, and any path analysis retain the -surface-to-tip implanted segment rather than treating that external remainder as inserted tissue. - -The named reference is pinned to a specific Virtual Brain Lab Urchin/Pinpoint source revision -and stored with its source digest. It is a Pinpoint planning convention for the population -atlas, **not** an Allen-supplied official bregma and not a registration to the individual mouse. -BrainGlobe arrays still use `[AP, DV, ML]` in micrometres, and the coordinate shown above a slice -remains an atlas-native physical coordinate. See -[Coordinate conventions](docs/ADR-002-coordinate-conventions.md). - -On Dorsal, Coronal, and Horizontal atlas images, animal right is the screen-left edge and animal -left is the screen-right edge. Therefore a negative ML input appears on the screen-right, -`L`-labelled side; a negative AP input moves toward the `P`-labelled edge. - -New v4 probe creation does not require a separately registered target, a subject calibration, or -a per-plan geometry checkbox. The exact built-in source snapshots and their review-pending -provenance remain visible and validation remains fail-closed; removing the checkbox does not -promote either transcription to independently verified hardware geometry. Older project packages -may retain archived v1–v3 target/calibration and multi-angle records. Brain3D preserves those -records and their original semantics, but does not expose them as primary new-plan controls. - -There is no probe Apply/Create button. NP2003/NP2013 and the sagittal/90° layout are atomic -choices and update immediately. AP, ML, surface depth, and angle update on Return or when the -numeric field loses focus, so typing `90` cannot submit the intermediate `9`. Mutations are -serialized and coalesced to the newest committed edit; the views and PDF never claim an -unfinished keystroke as operative geometry. - -A partially typed or invalid numeric value remains an editor draft and does not replace the -last valid plan. Opening another project, reconnecting, quitting, saving, or exporting is guarded -while such text remains unresolved, so it cannot be mistaken for the displayed trajectory. - -There is one main planning window, and its probe draft is owned by the app-level planner model -rather than by the sidebar view. Closing that window does not quit the app or clear a draft; -reopening the same project/plan context restores the same typed values. A real project, plan, or -saved plan-input change synchronizes the draft only after the dirty-state guard has been -resolved. - -On project validation, Brain3D re-resolves a v4 entry against the loaded annotation and -reconstructs its AP/ML, surface, depth, signed angle, layout, and probe geometry. It also keeps -the legacy validator: Brain3D fully reconstructs planning-algorithm v2/v3 placements from the -preserved mode, entry (when applicable), angles, depth, roll, probe model, source target, and -calibration, then compares every physical geometry field. It therefore rejects translated or -same-target alternate-angle geometry even if its record hash is recomputed. Historical v1 -records lack sufficient preserved inputs for this reconstruction: they remain loadable for -review only and cannot enter 2D/3D planning overlays, PDF planning pages, or region analysis -until updated. Vessel-clearance analysis remains unavailable for every plan version. Persistence -schema 9 adds the optional calibration-free v4 representation; migration preserves v1–v3 records -without guessing or silently rewriting their scientific geometry. - -## Probe geometry boundary - -The primary selector intentionally contains only the two requested Neuropixels 2.0 choices: - -- single shank, 1,280 sites (`NP2003`); -- standard four shanks at 250 µm pitch, 5,120 sites and 384 simultaneous channels - (`NP2013`). - -`NP2004` and `NP2014` remain cited where the pinned source artifacts group equivalent physical -geometries, but they are not separate UI choices. Quad Base, Neuropixels 1.0, and the synthetic -fixture remain archived in code for old-project compatibility and test evidence. Exact -manufacturer, user-manual, electrode-mapping, ProbeTable, and SpikeGLX snapshots and SHA-256 -digests are recorded. - -When a project is validated, a catalog-owned model snapshot must equal its source-pinned -definition field for field. This includes identity/version, verification and provenance, -shank dimensions and offsets, tip geometry, and the complete ordered recording-site table; a -recomputed plan hash cannot legitimize a modified width, site, or source record. Unknown custom -identities are retained only on the historical v1 audit path and cannot enter current planning -geometry or analysis. - -Both selectable models remain deliberately **`source-transcribed-review-pending`**. No independent -human has reviewed every encoded coordinate against the cited sources or physical probes. The -direct v4 path does not add a geometry checkbox and never presents the models as independently -verified geometry. See [Probe Models](PROBE_MODELS.md). - -## Major-vessel reference boundary - -The visible layer is a reproducible derivative of the public VesSAP `BL6J-no1` whole-brain -3 µm skeleton and radius volumes from Todorov et al., -[Nature Methods 2020](https://doi.org/10.1038/s41592-020-0792-1), CC BY-NC 4.0. It retains -source radius ≥15 µm (nominal diameter ≥30 µm), preserves only true 26-neighbour source-skeleton -adjacency, applies the authors' Euler + B-spline Allen transform with an independently checked ML -reflection, and reduces paths on a 50 µm display grid. The pinned asset contains 196,377 points, -76,622 runs, and 119,755 segments. - -The backend advertises `auditedReferenceMajorVessels` and serves digest-checked geometry. -Swift overlays it on Dorsal, Coronal, Sagittal, Horizontal, and 3D. The backend deliberately does -not advertise `radiusAwareReferenceVesselAnalysis`: `vessel.major.reference.analyze` returns -`VESSEL_ANALYSIS_UNAVAILABLE` before reading or mutating project state. - -This is one cleared ex-vivo C57BL/6J reference, not the operative animal. Capillaries are omitted; -artery/vein identity is unavailable; the nonlinear transform is anisotropic; and no numeric -subject-registration, clearing-distortion, or inter-animal error bound is published. The layer -cannot establish clearance, vessel absence, trajectory suitability, or safety. - -The older LAMBADA P60_606 derivative remains archived and rejected: its hemisphere/laterality -binding and whole-brain coverage are not qualified, so it is never served or mirrored. See -[VesSAP Major Vessels](docs/VESSAP_MAJOR_VESSELS.md), -[LAMBADA Major Vessels](docs/LAMBADA_MAJOR_VESSELS.md), and -[Known Limitations](KNOWN_LIMITATIONS.md). - -## Surgery-plan PDF - -**Export PDF…** fills pages 1–2 of the supplied Headplate Protocol from the selected animal -plan, renders either one plan-centred view or all five views with the current probe and VesSAP -layer, and appends one coordinate-matched page from the user-owned 132-page -`MBSC_Figs_with_Layers.pdf`. The final atlas page is chosen by AP for coronal plates or `|ML|` -for sagittal plates; the signed left/right ML value remains explicit in the planning pages. The -source template's third-page sketch is a placeholder and is replaced, not emitted. - -Set the prepared Headplate protocol PDF and `MBSC_Figs_with_Layers.pdf` once in -**Brain3D → Settings**. Every export reuses those saved locations until they are replaced there; -a disconnected volume is reported instead of opening a chooser on every export. The -implementation reads both PDFs directly, uses SceneKit for the 3D snapshot, and uses -PDFKit/Core Graphics for overlays, assembly, and verification. It does not open Word, -Illustrator, or another converter. Neither supplied PDF is copied into this public repository. -Each v4 planning page puts the complete AP/ML/local-surface-depth text on its own fixed coordinate -line, separate from the bounded subject/plan identity, and prints the signed A↔P angle and layout. -Brain3D verifies that full coordinate text and the expected view title on the rendered page and -again in the ordered final packet. V4 export does not require a separate target projection or -subject calibration. -See -[Surgery-plan export](docs/SURGERY_PLAN_EXPORT.md). - -## Architecture - -```text -SwiftUI macOS application - ├─ one selected Dorsal / Coronal / Sagittal / Horizontal / 3D workspace - ├─ native controls, file panels, accessibility, and 2D overlays - └─ SceneKit brain/probe/reference-vessel rendering and camera interaction - ↕ strict typed NDJSON -Python 3.12 scientific service - ├─ pinned BrainGlobe atlas access and coordinate transforms - ├─ annotation-surface resolution, probe geometry, vessel geometry, and voxel traversal - └─ provenance, integrity checks, qualification gates, stale-result rejection, and persistence -``` - -Swift owns presentation. Python owns scientific coordinate conversion and analysis. Both paths -reject stale revisions and mismatched source identities. See [Architecture](docs/ARCHITECTURE.md) -and [ADR-004](docs/ADR-004-swiftui-hybrid-shell.md). - -Viewer-only slice and region-pick mutations validate their bounded viewer state and preserve the -already validated surgery-plan graph, so moving a slice does not reconstruct every trajectory. -Project create/open/save, probe mutation, analysis, and PDF export retain the full -semantic-validation boundary. Legacy calibration mutation remains isolated to preserved v1–v3 -records. - -Pinpoint is the workflow reference, not an embedded state engine. Its hosted WebGL build does not -expose a supported bidirectional contract for probe identity, coordinates, atlas/camera state, or -the VesSAP overlay. Brain3D therefore uses its existing full BrainGlobe/Allen service instead of -copying or embedding an unsynchronized Unity canvas. See -[Pinpoint interoperability](docs/PINPOINT_INTEGRATION.md). - -## Developer quick start - -Requirements: Apple Silicon Mac, macOS 14 or later, Python 3.12, Swift, and -[uv](https://docs.astral.sh/uv/). +## Quick start ```bash uv python install 3.12 uv sync --frozen --group dev +uv run --frozen mouse-brain-planner atlas download allen_mouse_25um native/Brain3D/Scripts/build-app.sh open native/Brain3D/build/Brain3D.app ``` -The development app discovers the repository `.venv` and Python bridge. If discovery fails, it -reports the failure instead of substituting demo anatomy. The Allen 25 µm reference and annotation -arrays require about 0.43 GiB before rendering overhead. +The development app uses the repository Python environment. To build the standalone Apple Silicon +application: + +```bash +native/Brain3D/Scripts/build-release-app.sh +``` + +## Data and documents + +Third-party datasets and PDFs are not stored in this repository. + +- The Allen atlas is downloaded on demand to the application data directory. +- Surgery protocol and reference-atlas PDFs are selected by the user in **Brain3D → Settings**. +- The optional VesSAP overlay uses an externally supplied data package. Place + `vessap_bl6j1_major_vessels_50um_v1.npz` and its + `.npz.manifest.json` file in `/vasculature/`. Set + `MOUSE_BRAIN_PLANNER_DATA_DIR` to override the platform data directory. + +External files retain their original licenses and terms. See [Third-party software and +data](THIRD_PARTY.md). -Useful commands: +## Commands ```bash uv run --frozen mouse-brain-planner atlas list @@ -238,7 +59,7 @@ uv run --frozen mouse-brain-planner atlas download allen_mouse_25um uv run --frozen mouse-brain-planner validate-project /absolute/path/Plan.mouseplan ``` -Development checks: +## Development checks ```bash uv lock --check @@ -248,78 +69,32 @@ uv run --frozen mypy --no-incremental uv run --frozen pytest -q swift test --package-path native/Brain3D --no-parallel native/Brain3D/Scripts/build-app.sh -codesign --verify --deep --strict native/Brain3D/build/Brain3D.app ``` -`build-app.sh` creates an ad-hoc-signed development bundle that depends on the source checkout. - -## Standalone Apple Silicon release - -Build the distributable app on an Apple Silicon Mac: - -```bash -native/Brain3D/Scripts/build-release-app.sh -``` - -The release builder uses the locked production dependency graph, exact uv-managed CPython -3.12.12, and pinned PyInstaller 6.21.0. It emits -`native/Brain3D/dist/Brain3D.app` plus `Brain3D-macOS-arm64.zip`; the zip contains exactly one -top-level `Brain3D.app`. Set `OUTPUT_DIR=/absolute/path` to select another output directory. - -The frozen bridge lives at -`Brain3D.app/Contents/Resources/Bridge/brain3d-bridge`. The app prefers that verified bundled -executable over repository discovery. The builder hard-fails if any Mach-O is not arm64-only, -requires newer than macOS 14.0, uses a host-only Homebrew/Xcode/local dependency or runpath, or if -a symlink leaves the app. It then unzips the archive, denies the source checkout to both processes, -performs a bridge `hello`, launches the real Swift app, observes that exact bundled bridge child, -and verifies both processes remain alive before controlled shutdown. The release environment -contains no test/lint/type-check dependencies, and the bundle includes its production CycloneDX -SBOM, exact dependency notices, and build provenance under `Contents/Resources/Release`. - -If a `Developer ID Application` identity is available, the builder uses it for PyInstaller and -the outer app with the hardened runtime; otherwise it uses ad-hoc signing. Ad-hoc output is useful -for local qualification but is not notarized and should not be presented as a normal Gatekeeper- -ready download. Notarization and testing on a separate clean macOS 14 Apple Silicon machine remain -release-operator gates. - -The app contains code and the display-only VesSAP derivative, but not the Allen atlas, user -protocol PDF, or Mouse Brain atlas PDF. Allen data is still downloaded to the user's application -cache when requested; the two surgery-packet PDFs remain external user-configured inputs. Public -distribution must preserve the bundled notices and the VesSAP CC BY-NC 4.0 noncommercial terms. - -## Repository map +## Architecture ```text -native/Brain3D/ SwiftUI + SceneKit application and native tests -src/mouse_brain_planner/ - atlas/ pinned BrainGlobe access boundary - bridge/ typed service used by the native app - coordinates/ named coordinate frames and transforms - probes/ source-traceable probe catalog - analysis/ region traversal plus archived vessel-analysis code - vasculature/ display-only VesSAP geometry plus archived evidence paths - persistence/ checksummed project packages and migrations -tests/ Python software tests and fixtures -docs/ architecture, decisions, audits, and source records +SwiftUI + SceneKit macOS app + ↕ typed NDJSON +Python atlas and planning service ``` +Swift owns presentation and native file workflows. Python owns atlas access, coordinate +conversion, planning geometry, persistence validation, and scientific data checks. + ## Documentation -- [Project Status](PROJECT_STATUS.md) -- [User Guide](USER_GUIDE.md) -- [Known Limitations](KNOWN_LIMITATIONS.md) -- [Scientific Validation](SCIENTIFIC_VALIDATION.md) -- [Probe Models](PROBE_MODELS.md) -- [VesSAP Major Vessels](docs/VESSAP_MAJOR_VESSELS.md) -- [Surgery-plan export](docs/SURGERY_PLAN_EXPORT.md) -- [Pinpoint interoperability](docs/PINPOINT_INTEGRATION.md) -- [LAMBADA Major Vessels](docs/LAMBADA_MAJOR_VESSELS.md) -- [Third-Party Software and Data](THIRD_PARTY.md) - [Installation](INSTALL.md) +- [User guide](USER_GUIDE.md) +- [Architecture](docs/ARCHITECTURE.md) +- [Atlas data](ATLAS_DATA.md) +- [Coordinate systems](COORDINATE_SYSTEMS.md) +- [Probe models](PROBE_MODELS.md) +- [Surgery-plan export](docs/SURGERY_PLAN_EXPORT.md) +- [VesSAP external data](docs/VESSAP_MAJOR_VESSELS.md) +- [Third-party software and data](THIRD_PARTY.md) ## License -This public repository is **not open source**. Original source code remains all rights reserved -under [LICENSE](LICENSE). Atlas, probe-source documents, and scientific data retain their own -terms. In particular, the bundled VesSAP derivative remains CC BY-NC 4.0 and is not covered by -the source-code terms; see [Third-Party Software and Data](THIRD_PARTY.md). +Original source code is covered by [LICENSE](LICENSE). Dependencies and externally supplied data +retain their own licenses. diff --git a/SCIENTIFIC_VALIDATION.md b/SCIENTIFIC_VALIDATION.md index 198d3f8..07b4d9f 100644 --- a/SCIENTIFIC_VALIDATION.md +++ b/SCIENTIFIC_VALIDATION.md @@ -1,234 +1,34 @@ -# Scientific Validation Record +# Verification Scope -Reviewed: 2026-07-25 +Brain3D's automated checks cover software behavior and data-contract consistency. -This file distinguishes implemented and tested software behavior from scientific or procedural -validation. Brain3D is restricted to non-human mouse research. No clinical, veterinary-device, -animal-outcome, phantom-targeting, histological-accuracy, or formal usability validation is -claimed. +## Covered by tests -## Current evidence matrix +- Allen atlas identity, orientation, shape, and coordinate conversions. +- AP/ML surface resolution, probe placement, and trajectory reconstruction. +- Probe catalog dimensions, shank/site layout, and persistence round trips. +- Slice, region, SceneKit, and bridge protocol behavior. +- Project checksums, migrations, backup recovery, and stale-state rejection. +- Optional VesSAP manifest, digest, array, and coordinate checks when external data is supplied. -| Area | Software evidence in the current tree | What remains unestablished | -| --- | --- | --- | -| Native workspace | Single selected five-mode view, independent depths, shared selected-region overlays in four 2D modes plus 3D, pan/zoom/wheel, stale-frame checks | Procedure usability and error rate | -| Allen atlas | Strict `allen_mouse_25um` v1.2 identity, metadata/array checks, real-cache smoke paths | Subject anatomy and 25 µm targeting accuracy | -| Direct atlas-surface input | Source-pinned Pinpoint/Urchin reference; AP/ML sign tests; exact first-annotated-voxel surface; depth/angle/layout reconstruction and persisted-digest checks | Accuracy of the profile for an individual animal, pia/skull correspondence, and achieved path | -| Legacy calibration | Typed CRUD, matched-landmark fits, residual/QC gates, transform round trips, hash/revision binding retained for v1–v3 | Accuracy of user measurements and biological registration | -| NP2 geometry | Selectable NP2003/NP2013 geometries are derived from pinned source constants; shank/site count, pitch, offset, transform, and plan-creation tests | Independent full-table review and physical-probe conformance | -| NP1 geometry | All 960 sites derived from pinned source snapshots; schema/count/pattern tests | Independent full-table review and physical-probe conformance | -| VesSAP display geometry | Pinned source/asset/transform digests; true-path extraction; axis/laterality/bounds/label checks; Python→Swift binary protocol; slice/3D render tests | Subject registration, live anatomy, clearing/inter-animal error, and qualified vessel surfaces | -| LAMBADA evidence | Reproducible extraction, source/asset hashes, schema/bounds/radius/run checks, and digest-bound rejected qualification | Biological laterality, whole-brain coverage, and subject-specific vasculature | -| Vessel V3 | Legacy synthetic AABB-candidate and tapered-surface tests; production capability is absent | Subject-specific geometry, empirical uncertainty, and biological decision thresholds | -| SceneKit | Coordinate-transform, brain/probe/vessel mesh, picking, and off-screen render tests | Anatomical truth beyond supplied geometry | -| Persistence | Project revisions, checksums, migrations, backup recovery, source/input digests | Long-term regulated record requirements | +## Study-specific verification -The macOS application is built and ad-hoc signed as a development bundle and exercised as a real -process. That does not constitute clean-Mac production packaging, notarization, or release -qualification. +A research workflow should independently verify: -## Atlas boundary +- subject registration and landmark acquisition; +- probe/hardware geometry used in the experiment; +- planned versus achieved trajectories; +- tissue deformation and manipulator effects; +- vascular coverage and uncertainty; and +- the exact application, atlas, and data versions used for analysis. -The supported identity is: +The test suite can be run with: -| Field | Value | -| --- | --- | -| BrainGlobe key/version | `allen_mouse_25um` / `1.2` | -| Voxel spacing | `(25,25,25)` µm | -| Array shape | `(528,320,456)` in `[AP,DV,ML]` order | -| CCF citation | Wang et al., Cell 2020, DOI `10.1016/j.cell.2020.04.007` | - -The 10 µm package is excluded from this testing phase. The accepted 25 µm sampling interval -does not bound stereotaxic, registration, or procedural error. Slice and SceneKit coordinate -round trips test the application convention, not biological correspondence to an individual -mouse. - -## Direct bregma, surface, angle, and layout convention - -The v4 direct path uses AP and ML millimetres from a named Pinpoint/Urchin profile: - -| Axis | Positive | Negative | -| --- | --- | --- | -| AP | anterior | posterior / back | -| ML | left | right | - -The pinned reference resolves to BrainGlobe physical `[AP,DV,ML] = [5200,332,5700] µm` for -`allen_mouse_25um` v1.2. The exact Urchin revision and source SHA-256 are persisted. This is a -reproducible Pinpoint planning convention, not an Allen-supplied official bregma and not a -subject-specific registration. - -At the requested AP/ML column, software tests require the entry to equal the superior boundary -of the first nonzero annotation voxel. Depth is the positive entry-to-tip path length. For the -single visible sagittal angle, positive advances A→P and negative advances P→A. NP2013 layout -`0°` places the shank plane sagittally; `90°` rotates it clockwise from dorsal. These tests -establish implementation signs and reconstruction, not accuracy in a live animal. - -V1–v3 target/calibration records remain preserved with their original requirements. Their -projection still requires declared skull-frame metadata, matched landmarks, laterality, a DV -reference, a rigid/similarity fit, and user-sourced QC limits. They are not silently converted -into v4 surface plans. - -## Neuropixels 2.0 evidence - -The primary selector exposes `NP2003` (single shank) and standard `NP2013` (four shanks); both -expose 384 simultaneous channels. `NP2004` and `NP2014` remain cited in the pinned source -artifacts where they share the transcribed physical geometries, but are not extra UI choices. -The transcription uses -pinned imec data-sheet, User Manual V1.0.6, electrode-channel mapping, ProbeTable 1.8, and -SpikeGLX snapshots. Quad Base remains archived compatibility evidence and is not selectable. - -Each 10 mm × 70 µm × 24 µm shank contains 1,280 point sites in 640 two-site rows. Coordinates -use the source-backed 206 µm tip-to-lowest-row-center distance, 15 µm axial pitch, and −8/+24 µm -lateral centers. Four-shank centers are 0/+250/+500/+750 µm from primary leftmost `shank-0`. -The manufacturer 175 µm chisel-tip length remains a distinct physical dimension. - -The two selectable choices and archived Quad Base definition are -**`source-transcribed-review-pending`**. Exact software reconstruction and end-to-end creation -tests do not establish manufacturing tolerance, independent transcription review, -physical-device conformance, implantation accuracy, or tissue response. - -## Neuropixels 1.0 evidence - -The NP1 NP1000 / `PRB_1_4_0480_1` entry transcribes all 960 sites from four pinned sources: - -- imec Neuropixels 1.0 specification; -- ProbeTable 1.8 `probe_features.json` at commit - `207f7bf424b0fa26f271700b970e27a58a9a1111`; -- SpikeGLX `IMROTbl.cpp` at commit `d67bee45fa2635873456eb5d3f5e5a051690e64f`; and -- SpikeGLX metadata coordinate definitions at that commit. - -Each snapshot has a recorded SHA-256. The model distinguishes the manufacturer 175 µm chisel -tip from SpikeGLX's 209 µm physical-tip-to-lowest-row-center measurement. It includes the 10 mm -shank, 70 µm width, 24 µm thickness, row/site pattern, references, and banks. - -The verification status is **`source-transcribed-review-pending`**. The transcription has not -received an independent full-table human review, so it cannot be described as independently -verified manufacturer geometry. See [Probe Models](PROBE_MODELS.md). - -## VesSAP display-reference evidence - -Source: VesSAP `BL6J-no1`, Todorov et al., *Machine learning analysis of whole mouse brain -vasculature*, [DOI `10.1038/s41592-020-0792-1`](https://doi.org/10.1038/s41592-020-0792-1), -public repository release 2021.10.01, CC BY-NC 4.0. - -The derivative begins with the official same-grid 3 µm skeleton and radius volumes. Radius ≥5 -source voxels retains 1,262,706 centerline voxels (nominal radius ≥15 µm / diameter ≥30 µm). -Only mapped edges induced by true 26-neighbour source adjacency are retained; graph endpoint -chords are never substituted. Paths are coalesced on a 50 µm display grid, yielding 196,377 -points, 76,622 runs, and 119,755 segments in a 1,853,131-byte NPZ with SHA-256 -`9300dacf25ca57a5d23377ca0dc885e34ff0d18e8d21ef7590c6dcd156cf5db7`. - -The authors' Euler + B-spline transform is interpreted as -`AP_um = 30*T_y`, `DV_um = 30*T_z`, `ML_um = 11390 - 30*T_x`. The ML reflection is mandatory: -VesSAP moving-x increases left-to-right while BrainGlobe ASR ML increases right-to-left. -Deterministic validation used 20,000 graph nodes. It found 99.705% in atlas bounds, 99.4618% -grouped-region agreement at 10 µm excluding background, 99.9787% agreement between the official -registered signed-label NIfTI and supplied processed groups, and 99.9794% laterality agreement at -the canonical midpoint. Four laterality discrepancies were all within 5.38 µm of midline. - -These checks qualify the coordinate interpretation for a display reference. They do not qualify -clearance. This is one ex-vivo cleared specimen; no target-registration-error, bregma/skull -registration, clearing-distortion, or inter-animal bound is published. Sampled nonlinear-transform -singular values span 0.672–1.560, so a scalar source radius is not a qualified circular -atlas-space surface. The runtime therefore exposes geometry but rejects clearance analysis. -See [the full evidence record](docs/VESSAP_MAJOR_VESSELS.md). - -## LAMBADA major-vessel evidence - -Source: Renier, de Launoit, and Skriabine, *Vascular graphs of the developing post-natal mouse -brain*, [Zenodo record 18876865](https://zenodo.org/records/18876865), DOI -`10.5281/zenodo.18876865`, CC BY 4.0; associated Cell paper DOI -`10.1016/j.cell.2026.03.013`. - -The pinned source archive is `P60_606_graph_2024-12-03.gt.7z`, 5,050,194,723 bytes, Zenodo MD5 -`218ed346c6d7dc501301204f811be37f`. The bundled 814,393-byte derivative has SHA-256 -`fb2344e845e604be3424bd63f4222d273eafba34db0df2eaff32f4400fa9afec` and contains: - -- 71,313 radius-bearing points; -- 59,495 segments; -- 11,818 maximal consecutive in-bounds runs; and -- 10,907 contributing source edges. - -The extraction threshold is point radius ≥15 µm, equivalent to diameter ≥30 µm. It does not -select whole edges by mean or maximum radius. Coordinates are converted from ClearMap to -BrainGlobe `[AP,DV,ML]` and then to physical micrometres without a half-voxel shift. The loader -checks the adjacent manifest, asset identity, arrays, dtypes, shapes, bounds, thresholds, ordering, -and run lengths in the archived evidence path. - -This evidence supports deterministic extraction from the declared reference. It does not support -display or analysis. The exact qualification found supporting AP and DV orientation evidence but -rejected the asset because the primary source describes hemisphere specimens and the exact graph -has no persisted biological hemisphere/laterality binding. Whole-brain coverage and ML polarity -are unqualified; numeric points on both sides of the array midpoint do not resolve either issue, -and no mirroring is permitted. - -No LAMBADA handler or capability is registered in the current runtime, and the production -reference endpoints never load this asset; those endpoints now belong exclusively to the -separately qualified VesSAP display reference. The canonical LAMBADA rejection report is -[`docs/evidence/lambada_p60_606_coordinate_qualification_rejected_v1.json`](docs/evidence/lambada_p60_606_coordinate_qualification_rejected_v1.json), -SHA-256 `0993d5a0ad6c0d62094dc395fe2bc4f284870e6e7c0b602be7df5a7da867c93a`. -The source also excludes pial and choroidal vessels; the derivative filters smaller vessels; and -artery/vein identity, biological variation, tissue distortion, and registration error are not -resolved. See [the derivation record](docs/LAMBADA_MAJOR_VESSELS.md). - -## Archived V3 algorithm evidence - -Algorithm `major-vessel-aabb-tapered-surface-v3` computes conservative AABB lower bounds and -feasible upper bounds before running exact finite-segment and tapered-surface minimization only on -the sound candidate set. In isolated synthetic tests it reports centerline/surface geometry, -interpolated radius, probe envelope, required margin, -registration uncertainty, adjusted clearance, closest points, insertion depth, conflict class, -source identity, algorithm version, and input hashes. - -This algorithm is not a production capability for VesSAP or P60_606. The display-only gate runs -before project, asset, or analysis access, so the application produces neither positive conflicts -nor absence results. User-supplied margins or uncertainty cannot manufacture missing source -uncertainty bounds. - -### Reproducible performance evidence - -The offline, non-product [major-vessel benchmark](scripts/benchmark_major_vessel_analysis.py) -loads the SHA-256-verified archived asset and uses a fixed synthetic probe/profile input. Run it -with: - -```console -uv run python scripts/benchmark_major_vessel_analysis.py --iterations 20 +```bash +uv run --frozen pytest -q +swift test --package-path native/Brain3D --no-parallel ``` -One run on 2026-07-22 used an Apple M4 (`Mac16,13`, arm64), macOS 26.5.2, and CPython -3.12.13. The asset contained 71,313 points, 11,818 runs, and 59,495 segments. V3 selected -243 candidates, performed exactly 243 narrow-phase measurements, and reported 153 loaded-geometry -algorithmic conflicts in that offline fixture. Timings were: - -| Measurement | Milliseconds | -| --- | ---: | -| Verified asset load and analysis-geometry construction | 44.205 | -| First analysis after load | 27.283 | -| Warm analysis median, 20 iterations | 28.656 | -| Warm analysis P95, 20 iterations | 45.697 | - -Python/module startup is excluded. The benchmark enforces `--iterations` in `[5,100]`, verifies -the bundled asset counts and digest, rejects changing outputs across identical runs, and emits the -machine, Python, geometry, result counts, and timings as JSON. These measurements are one-machine -engineering evidence for archived code, not runtime availability, a cross-hardware latency -guarantee, or scientific/surgical validation. - -## Archived evidence paths - -The Kim 2022 population vascular-length-density field and user subject-image registration remain -in the Python backend for reproducibility of older work. They are absent from the primary UI and -do not bypass the VesSAP display-only or LAMBADA rejection gates. A scalar density projection contains no -individual vessel path/radius; a registered image is not automatically a vessel segmentation. - -## Required evidence before qualification - -1. Independent review of every NP1 source-derived dimension and site coordinate. -2. Repeated-observer subject calibration studies with declared ground truth. -3. Phantom and histology studies for planned versus achieved probe paths and sites. -4. Subject-specific vascular ground truth with measured bregma/skull registration error, - clearing/live-tissue distortion, inter-animal variation, and false-negative coverage. -5. Reference-graph failure-mode studies and prospectively defined margin/uncertainty criteria. -6. Formal animal-workflow usability, accessibility, interruption, and recovery studies. -7. Deterministic runtime packaging, SBOM, signing/notarization, and clean-Mac qualification. - -Passing repository tests or rendering a complete scene does not satisfy these scientific gates. +Technical source records are maintained in [Probe Models](PROBE_MODELS.md), +[Coordinate Systems](COORDINATE_SYSTEMS.md), and +[VesSAP external data](docs/VESSAP_MAJOR_VESSELS.md). diff --git a/THIRD_PARTY.md b/THIRD_PARTY.md index 1fa750f..6c1e990 100644 --- a/THIRD_PARTY.md +++ b/THIRD_PARTY.md @@ -1,141 +1,43 @@ -# Third-party software, data, and prior art +# Third-party Software and Data -This inventory was reviewed on **2026-07-23**. Exact direct pins come from `pyproject.toml`; -`uv.lock` is the machine-resolved transitive inventory. Before each release, generate an SBOM -from the final lock/build environment, retain all license texts and notices required by the -bundled artifacts, and reconcile it with this human-readable list. +Third-party datasets, atlas volumes, probe PDFs, protocol PDFs, and atlas artwork are not stored in +this repository. They are downloaded by the user or selected from user-owned locations. -The classification column is intentional: +## Runtime dependencies -- **Code — dependency / declared dependency:** code installed by the default environment. The row - states when a package is reserved for later work rather than imported by the current workflow. -- **Data — downloaded:** separately governed content acquired at user request and not bundled. -- **Data — optional downloaded:** a reviewed source acquired only when the user enables the - corresponding feature; it is cached outside the application bundle. -- **Data — bundled derivative:** an adapted data artifact shipped with the application under - its own data license, attribution, immutable manifest, and scientific-use limitations. The - application source-code license does not relicense it. -- **Candidate data:** a source reviewed for possible later work, but not integrated. -- **Cited source:** factual geometry or terminology was independently encoded with a pinned - citation/digest; the upstream document, code, diagram, or binary is not copied or bundled. -- **Concept only:** documentation or workflow reviewed as prior art; no code or assets copied. +| Package | Version | License | Purpose | +| --- | ---: | --- | --- | +| BrainGlobe AtlasAPI | 2.3.1 | BSD-3-Clause | Atlas discovery, download, and access | +| nibabel | 5.4.2 | MIT | NIfTI access | +| NumPy | 2.5.1 | BSD-3-Clause | Arrays and numerical operations | +| Pillow | 12.3.0 | MIT-CMU | Image processing | +| platformdirs | 4.11.0 | MIT | Application data paths | +| Pydantic | 2.13.4 | MIT | Validated models | +| scikit-image | 0.26.0 | BSD-3-Clause | Image registration | +| SciPy | 1.18.0 | BSD-3-Clause | Numerical operations | +| tifffile | 2026.7.14 | BSD-3-Clause | TIFF loading | -The supported interactive shell uses Apple SwiftUI/AppKit/SceneKit system frameworks and a -versioned subprocess bridge to the Python scientific service. SceneKit displays backend-validated -brain, probe, and display-only VesSAP vessel geometry payloads; archived LAMBADA vessel primitives -receive no runtime payload. -Scientific coordinate conversion and analysis remain in Python. The former PySide6/PyVista/VTK -application was removed from the package and lockfile after the Phase 1 reachability audit. +The exact dependency graph is recorded in `uv.lock`. Release builds include a generated SBOM and +license inventory. -Surgery-plan export adds no third-party Swift document parser. Apple PDFKit/Core Graphics read -the user-prepared protocol PDF and 132-page `MBSC_Figs_with_Layers.pdf` directly, add fields, -preserve vector pages, assemble, and verify the result; SceneKit produces the offscreen 3D page. -Word, Illustrator, Apple Events automation, and document-conversion subprocesses are not part of -the export path. +## External data and documents -## Direct runtime dependencies +| Item | Terms | Use | +| --- | --- | --- | +| Allen Mouse CCF via BrainGlobe `allen_mouse_25um` | [Allen Institute Terms of Use](https://alleninstitute.org/legal/terms-of-use) | Downloaded on demand to the user's application data directory | +| VesSAP BL6J-no1 vasculature | [CC BY-NC 4.0](https://creativecommons.org/licenses/by-nc/4.0/) | Optional external major-vessel display data; not stored in the repository or application bundle | +| Kim et al. vascular-density data | [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/) | Optional external compatibility workflow | +| Headplate protocol PDF | Source-owner terms | User-selected input for surgery-plan export | +| Mouse Brain reference-atlas PDF | Source-owner terms | User-selected input for surgery-plan export | -| Package/version | License | Purpose | Classification / source | -|---|---|---|---| -| brainglobe-atlasapi 2.3.1 | BSD-3-Clause | Atlas discovery, download, metadata, arrays | Code — dependency; [PyPI](https://pypi.org/project/brainglobe-atlasapi/2.3.1/), [source](https://github.com/brainglobe/brainglobe-atlasapi/tree/v2.3.1) | -| nibabel 5.4.2 | MIT | Strict NIfTI header/data access for the pinned population-density source | Code — dependency; [PyPI](https://pypi.org/project/nibabel/5.4.2/) | -| numpy 2.5.1 | BSD-3-Clause | N-dimensional arrays | Code — dependency; [PyPI](https://pypi.org/project/numpy/2.5.1/) | -| platformdirs 4.11.0 | MIT | macOS application/cache paths | Code — dependency; [PyPI](https://pypi.org/project/platformdirs/4.11.0/) | -| pydantic 2.13.4 | MIT | Validated settings and project models | Code — dependency; [PyPI](https://pypi.org/project/pydantic/2.13.4/) | -| Pillow 12.3.0 | MIT-CMU | Image decoding, validation, and raster output | Code — dependency; [PyPI](https://pypi.org/project/pillow/12.3.0/) | -| scikit-image 0.26.0 | BSD-3-Clause | Similarity/affine registration of subject dorsal images | Code — dependency; [PyPI](https://pypi.org/project/scikit-image/0.26.0/) | -| scipy 1.18.0 | BSD-3-Clause | Population-density resampling and overlay operations | Code — dependency; [PyPI](https://pypi.org/project/scipy/1.18.0/) | -| tifffile 2026.7.14 | BSD-3-Clause | Validated atlas TIFF loading | Code — dependency; [PyPI](https://pypi.org/project/tifffile/2026.7.14/) | +VesSAP source and paper: [VesSAP data record](https://www.discotechnologies.org/VesSAP/) and +[Todorov et al., Nature Methods 2020](https://doi.org/10.1038/s41592-020-0792-1). -## Build and test dependencies +## Cited specifications and prior art -These tools are not application features, but some are present in build environments or affect -the generated bundle. +Neuropixels geometry is transcribed from cited imec, ProbeTable, and SpikeGLX sources; the source +documents are not copied into the repository. The planning coordinate profile cites the Virtual +Brain Lab Urchin/Pinpoint implementation. Exact source identities are listed in +[Probe Models](PROBE_MODELS.md) and [Coordinate Systems](COORDINATE_SYSTEMS.md). -| Package/version | License | Purpose | Source | -|---|---|---|---| -| hatchling 1.27.0 | MIT | Python build backend | [PyPI](https://pypi.org/project/hatchling/1.27.0/) | -| mypy 2.3.0 | MIT | Static type checking | [PyPI](https://pypi.org/project/mypy/2.3.0/) | -| pytest 9.1.1 | MIT | Test runner | [PyPI](https://pypi.org/project/pytest/9.1.1/) | -| pytest-cov 7.1.0 | MIT | Coverage integration | [PyPI](https://pypi.org/project/pytest-cov/7.1.0/) | -| ruff 0.15.22 | MIT | Linting and formatting checks | [PyPI](https://pypi.org/project/ruff/0.15.22/) | -| PyInstaller 6.21.0 | GPL-2.0-or-later with the upstream bootloader exception; selected files are Apache-2.0 | Apple Silicon standalone bridge freezer; exact COPYING text is embedded in release builds | [release documentation](https://pyinstaller.org/en/v6.21.0/) | -| pyinstaller-hooks-contrib 2026.6 | Apache-2.0 / GPL-2.0-or-later by file | Pinned analysis hooks used only while freezing production dependencies | [PyPI](https://pypi.org/project/pyinstaller-hooks-contrib/2026.6/) | - -## Material transitive dependencies - -This is a review aid, not a substitute for the full lockfile/SBOM. - -| Package/version | License | Why material | Source | -|---|---|---|---| -| brainglobe-space 1.0.3 | BSD-3-Clause | Atlas axis/orientation conversions | [PyPI](https://pypi.org/project/brainglobe-space/1.0.3/) | -| pandas 3.0.3 | BSD-3-Clause | BrainGlobe tabular dependency; not directly declared by this project | [PyPI](https://pypi.org/project/pandas/3.0.3/) | -| pyarrow 25.0.0 | Apache-2.0 | pandas columnar/serialization dependency in the resolved set | [PyPI](https://pypi.org/project/pyarrow/25.0.0/) | -| meshio 5.3.5 | MIT | Mesh-format support pulled by BrainGlobe AtlasAPI | [PyPI](https://pypi.org/project/meshio/5.3.5/) | -| pydantic-core 2.46.4 | MIT | Native validation engine bundled with Pydantic | [PyPI](https://pypi.org/project/pydantic-core/2.46.4/) | - -## Downloaded data and cited prior art - -| Item/version | Terms | Use in this project | Classification / source | -|---|---|---|---| -| Allen Mouse CCF via BrainGlobe `allen_mouse_25um`, atlas version 1.2 as observed 2026-07-21 | [Allen Institute Terms of Use](https://alleninstitute.org/legal/terms-of-use); scientific citation required | The only current reference/annotation/mesh atlas; downloaded on demand to user cache, never bundled in the `.app` or installer | Data — downloaded; [BrainGlobe version manifest](https://gin.g-node.org/brainglobe/atlases/raw/master/last_versions.conf), [generator/citation](https://github.com/brainglobe/brainglobe-atlasapi/blob/v2.3.1/atlas_scripts/allen_mouse.py) | -| Neuropixels 2.0 single- and four-shank geometry sources, snapshots recorded 2026-07-23 | Source-specific terms; the cited artifacts are not redistributed | Complete 1,280-site single-shank and 5,120-site four-shank factual transcriptions from imec specifications/manual/mapping, ProbeTable 1.8, and SpikeGLX. The production selector exposes only `NP2003` and standard four-shank `NP2013`, both with 384 simultaneous channels. The former combined NP2003/NP2004 and NP2013/NP2014 identities and Quad Base source evidence remain archived for old-package compatibility but are not selectable. Status is `source-transcribed-review-pending`; no independent full-table review is claimed. | Cited sources; [imec NP2 product/support page](https://www.neuropixels.org/probe-2-0-single-shank), [exact sources and digests](PROBE_MODELS.md) | -| Neuropixels 1.0 NP1000 / `PRB_1_4_0480_1` geometry sources, snapshots recorded 2026-07-22 | Source-specific terms; the cited artifacts are not redistributed | Complete 960-site factual transcription from the imec specification, ProbeTable 1.8, and SpikeGLX geometry/metadata snapshots. The catalog status is `source-transcribed-review-pending`; no independent full-table review is claimed. | Cited source; [imec specification](https://www.neuropixels.org/_files/ugd/328966_c5e4d31e8a974962b5eb8ec975408c9f.pdf), [exact sources and digests](PROBE_MODELS.md) | -| Kim, Yongsoo (2022), *Cerebrovascular, pericyte, and neuronal cell type mapping data 2022*, Mendeley Data V1, DOI `10.17632/stxvn5sv44.1` | [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/) | Optional pinned population vascular length-density field. Archive `NVU_mapping_Adult_mouse_brain (1).7z`, 311,493,514 bytes, SHA-256 `c715c92ad153bff7f676b883f47108f886147e5d6fcd4502bcc04a0f92ed98fe`; downloaded to user cache and never bundled. The archived backend can prepare a symmetrized dorsal DV maximum projection, but it is absent from the primary UI, is not subject-specific, has no vessel paths, and is not used for clearance. | Data — optional downloaded; [versioned dataset](https://data.mendeley.com/datasets/stxvn5sv44/1), [Wu et al. Cell Reports paper](https://doi.org/10.1016/j.celrep.2022.110978), [open-access paper](https://pmc.ncbi.nlm.nih.gov/articles/PMC9271215/) | -| Renier, Nicolas; de Launoit, Elisa; Skriabine, Sophie (2026), *Vascular graphs of the developing post-natal mouse brain*, record DOI `10.5281/zenodo.18876865` | [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/) | Packaged archived derivative evidence for specimen P60_606: maximal consecutive in-bounds graph runs with point radius at least 15 µm. NPZ is 814,393 bytes, SHA-256 `fb2344e845e604be3424bd63f4222d273eafba34db0df2eaff32f4400fa9afec`. The 5,050,194,723-byte source archive and 12,282,574,483-byte extracted graph are not bundled. Coordinate qualification is rejected because the source is a hemisphere and the graph lacks a persisted biological laterality binding. It is not displayed, mirrored, served, or analyzed. | Data — bundled derived evidence; [versioned Zenodo record](https://zenodo.org/records/18876865), [Cell paper](https://doi.org/10.1016/j.cell.2026.03.013), [derivation and qualification record](docs/LAMBADA_MAJOR_VESSELS.md) | -| Todorov et al. (2020), VesSAP BL6J-no1 whole-brain vasculature, public repository release 2021.10.01 | [CC BY-NC 4.0](https://creativecommons.org/licenses/by-nc/4.0/) | Bundled display-only major-vessel derivative from the official 3 µm skeleton and radius volumes. Source radius ≥5 voxels retains nominal diameter ≥30 µm; true 26-neighbour source adjacency is transformed with the published Euler plus B-spline registration and reduced on a 50 µm display grid. NPZ is 1,853,131 bytes, SHA-256 `9300dacf25ca57a5d23377ca0dc885e34ff0d18e8d21ef7590c6dcd156cf5db7`, with 196,377 points, 76,622 runs, and 119,755 segments. It is one fixed, cleared C57BL/6J specimen and cannot establish vessel absence, subject-specific clearance, trajectory suitability, or safety. | Data — bundled derivative; [VesSAP data record](https://www.discotechnologies.org/VesSAP/), [Nature Methods paper](https://doi.org/10.1038/s41592-020-0792-1), [exact derivation and validation](docs/VESSAP_MAJOR_VESSELS.md) | -| User-prepared Headplate Protocol PDF | Source-owner terms; not redistributed or relicensed | Read directly; pages 1–2 receive a flattened prefill overlay and the template's page-3 sketch placeholder is replaced. The source stays outside the repository and app bundle; its digest prefix is printed on planning pages. | External user-owned document | -| User-supplied Mouse Brain CD `MBSC_Figs_with_Layers.pdf`, Figures 1–132 | Source-owner terms; not redistributed or relicensed | Read directly. Brain3D validates all 132 landscape-Letter pages against the reviewed figure/coordinate catalog, verifies the source SHA, selects the nearest coronal AP or sagittal `|ML|` page, and adds an identity summary to the output copy. No atlas artwork is committed or bundled. | External user-owned data/artwork | -| brainrender 2.2.0 | BSD-3-Clause | Scene/API design review only; package not imported or bundled | Concept only; [release](https://github.com/brainglobe/brainrender/releases/tag/v2.2.0) | -| iblatlas 1.2.0 | MIT | Coordinate and trajectory design review only; package and PyQt5 GUI extra not imported or bundled | Concept only; [release](https://github.com/int-brain-lab/iblatlas/releases/tag/1.2.0) | -| Neuropixels Trajectory Explorer v2.0.0 | GPL-3.0 | Workflow review only; no code or assets copied | Concept only; [release](https://github.com/petersaj/neuropixels_trajectory_explorer/releases/tag/v2.0.0) | -| Virtual Brain Lab Urchin/Pinpoint Allen CCF coordinate reference, Urchin commit `57be3cdc7d6230543ebbd367be1cbcf1a47862a5` | GPL-3.0 | Workflow review plus factual transcription of the `CCF_BREGMA` default (Pinpoint CCF order AP 5.2 mm, ML 5.7 mm, DV 0.332 mm). No Unity code or assets are copied. The exact raw source SHA-256 `23880cef9abacbadd75b2b898a9b43cf1195ca75446f2c34f9ad0c278781f215` is persisted with new direct plans. | Cited source; [exact source snapshot](https://github.com/VirtualBrainLab/Urchin/blob/57be3cdc7d6230543ebbd367be1cbcf1a47862a5/UnityClient/Packages/vbl.urchin/Scripts/Utils/Utils.cs), [Pinpoint v2.0.0 release](https://github.com/VirtualBrainLab/Pinpoint/releases/tag/v2.0.0) | -| cortex-lab/allenCCF, including SHARP-Track, commit `e5a57fe7e1c9…` reviewed 2026-07-21 | No explicit repository license found | Workflow review only; no code or assets copied | Concept only; [repository](https://github.com/cortex-lab/allenCCF) | -| Simulation-ready vascular tracing data from adult mouse brains, version 1, DOI `10.17632/mjtyry6v85.1` | CC BY 4.0 | Rejected for planning integration. The four adult-mouse MATLAB graphs are separate from the archived scalar-density deposit, and their documentation exposes raw specimen-space XYZ without qualified axis orientation, laterality, Allen registration, or a bregma relationship. No graph file is integrated or bundled. | Rejected data candidate; [versioned dataset](https://data.mendeley.com/datasets/mjtyry6v85/1), [associated open-access paper](https://pmc.ncbi.nlm.nih.gov/articles/PMC9271215/) | -| VesselGraph | Software: MIT; data: CC BY-NC 4.0 | Vascular graph/data design review only; no code, models, or data copied; NC terms make the data unsuitable as an unrestricted redistributable default | Concept only; [repository and data terms](https://github.com/jocpae/VesselGraph) | -| VesSAP repository code | MIT | Segmentation and feature-extraction implementation reviewed as provenance; no VesSAP repository code or models are copied into Brain3D. The separately licensed data derivative is inventoried above. | Concept only; [repository](https://github.com/vessap/vessap) | - -Allen atlas content is not relicensed by BrainGlobe's BSD-3-Clause software license. The Allen -terms observed on the review date limit covered Content to noncommercial research unless -otherwise stated. Show the source/terms before download and obtain legal review before any -commercial redistribution or hosted data service. - -The Mendeley density is not relicensed by this application's source-code license. Preserve the -CC BY 4.0 attribution, dataset version/DOI, archive identity, and associated paper citation in any -scientific output that uses the layer. The dataset's four-animal population scope must remain -visible; attribution does not turn the scalar field into subject-specific vessels or a clearance -model. - -The archived LAMBADA P60_606 derivative is likewise not relicensed by this application's -source-code license. Preserve its CC BY 4.0 attribution, versioned record DOI, source and asset -digests, adjacent manifest, rejected coordinate/coverage status, and fixed/cleared hemisphere -limitations in every distributed evidence package or scientific output that studies the asset. -Do not describe its presence in the package as a runtime vessel layer. - -The bundled VesSAP BL6J-no1 derivative is not relicensed by this application's source-code -license. It remains CC BY-NC 4.0 adapted data. Keep the NPZ, adjacent immutable manifest, -`VESSAP_DATA_LICENSE.txt`, source/paper attribution, release identity, exact digests, transform -interpretation, and display-only limitations together. The noncommercial restriction applies to -the derivative independently of the application's original-source terms. Its presence must never be -described as subject-specific registration, vessel clearance, proof of vessel absence, or a -surgical safety determination. - -## Release obligations - -1. Build only from the reviewed lockfile and record Python, architecture, package, and atlas - versions in release metadata. -2. Bundle required copyright notices and license texts for every exact release artifact. -3. Keep downloaded Allen and Mendeley data outside the signed application and installer; - preserve each source, terms, citation, version, and recorded digest. -4. Keep the archived LAMBADA derivative, manifest, and canonical rejection report together and - preserve exact attribution, identity, extraction rule, limitations, and fail-closed status. -5. Keep the bundled VesSAP derivative, manifest, separate CC BY-NC 4.0 license text, provenance, - transform identity, and display-only/clearance-disabled limitations together. -6. Preserve the NP1 model's pinned source identities/digests and review-pending status until a - separately recorded independent review is complete; do not bundle upstream documents or code - without a new terms review. -7. Do not distribute the user-supplied protocol or Mouse Brain atlas through the repository, - application, test fixtures, or release package without a separate rights review. -8. Re-review this file when a dependency, atlas version, packaging mode, copied asset, or - distribution model changes. A concept-only item must be reclassified before any code or asset - is copied. +Third-party software and data remain governed by their respective licenses and terms. diff --git a/USER_GUIDE.md b/USER_GUIDE.md index 2df2ca8..b4c4644 100644 --- a/USER_GUIDE.md +++ b/USER_GUIDE.md @@ -135,9 +135,10 @@ resolved. ## Use the major-vessel display reference -Brain3D automatically loads the pinned VesSAP `BL6J-no1` major-vessel layer after the reviewed -25 µm atlas opens. It appears in Dorsal, Coronal, Sagittal, Horizontal, and 3D. The status panel -identifies the specimen, source, CC BY-NC 4.0 license, segment count, and diameter threshold. +When the external VesSAP `BL6J-no1` data package is installed, Brain3D loads the +major-vessel layer after the 25 µm atlas opens. It appears in Dorsal, Coronal, Sagittal, +Horizontal, and 3D. The status panel identifies the specimen, source, license, segment count, and +diameter threshold. The source layer intentionally includes only nominal diameter ≥30 µm centerlines and uses a 50 µm spatial reduction. The visible-diameter slider can be adjusted from 30–250 µm without @@ -152,8 +153,7 @@ clearing-distortion, or inter-animal uncertainty bounds, so the backend rejects Do not interpret a gap in the overlay as absence of a vessel or use it to approve a trajectory. For exact source files, hashes, transform validation, extraction, and limitations, see -[VesSAP Major Vessels](docs/VESSAP_MAJOR_VESSELS.md). The older LAMBADA P60_606 derivative -remains archived and rejected; it is not mixed with or mirrored into this layer. +[VesSAP Major Vessels](docs/VESSAP_MAJOR_VESSELS.md). ## Export a prefilled surgery plan diff --git a/docs/ADR-003-atlas-and-data-sources.md b/docs/ADR-003-atlas-and-data-sources.md index 770128d..7d89ab2 100644 --- a/docs/ADR-003-atlas-and-data-sources.md +++ b/docs/ADR-003-atlas-and-data-sources.md @@ -1,254 +1,48 @@ # ADR-003: Atlas and external data sources -- **Status:** Accepted -- **Decision date:** 2026-07-21 -- **Applies to:** atlas discovery, download, cache, provenance, and external reference material +- Status: Accepted +- Date: 2026-07-26 -## Implementation update — 2026-07-23 +## Context -The atlas decision is unchanged. A CC BY-NC 4.0 diameter-≥30 µm derivative of VesSAP -`BL6J-no1` is now the display-only major-vessel reference in all five views. Exact coordinate, -laterality, topology, source, transform, and asset evidence permits display but not subject -clearance. The older LAMBADA P60_606 derivative remains archived and rejected. Population density -and subject-image registration also remain archived backend paths and are absent from the primary -UI. See [the VesSAP record](VESSAP_MAJOR_VESSELS.md) and -[the LAMBADA record](LAMBADA_MAJOR_VESSELS.md). +Brain3D needs one reproducible atlas identity and clear boundaries for third-party data. Atlas +volumes, vascular datasets, and licensed documents should not be committed to the source +repository. ## Decision -Use [BrainGlobe AtlasAPI 2.3.1](https://pypi.org/project/brainglobe-atlasapi/2.3.1/) -as the authoritative atlas access layer. `allen_mouse_25um` package version `1.2` is the only -runtime-allowlisted identity in the current build, for both remote and local-only catalog views. The 10 µm -identity is deferred until the lower-memory path is release-qualified; existing cache data is -ignored and not deleted. Arbitrary BrainGlobe keys, non-mouse atlases, and future unreviewed -versions fail closed. Other identities may be added through the same adapter only after their -coordinates, version, schema, citation, data terms, and native acceptance evidence are reviewed. +### Atlas -Never silently substitute one resolution or atlas for another, and never combine annotation, -reference, mesh, or coordinate metadata from different atlas versions. Every saved project -records the atlas identifier, installed atlas version, resolution, shape, orientation, framework, -source annotation, source URL, cache path, SHA-256 of the installed `metadata.json`, and -application version. The current build does **not** claim to record the install time or a package-wide -content digest. +Use BrainGlobe AtlasAPI with the allowlisted `allen_mouse_25um` package version `1.2`. +BrainGlobe arrays use `[AP,DV,ML]` order and the package's `asr` orientation. The application +validates the package key, version, shape, resolution, orientation, metadata, and structures before +use. -## Atlas identity and coordinate metadata +Atlas files are downloaded to the application data directory and are never added to the +repository, application bundle, or project packages. -As checked on 2026-07-21, BrainGlobe's -[`last_versions.conf`](https://gin.g-node.org/brainglobe/atlases/raw/master/last_versions.conf) -lists version `1.2` for `allen_mouse_25um`. The generator uses Allen -CCF 2017 annotations (`annotation/ccf_2017`) and BrainGlobe orientation `asr`: array axes are -`[AP,DV,ML]`, the origin lies toward anterior/superior/right, and increasing indices move toward -posterior/inferior/left. Treat those values as discovered metadata, not permanent constants: -the installed metadata written to a project is authoritative for reopening it. +### Major-vessel display -The Allen atlas generator and citation are recorded in -[`allen_mouse.py`](https://github.com/brainglobe/brainglobe-atlasapi/blob/v2.3.1/atlas_scripts/allen_mouse.py). -Scientific outputs using this atlas must cite Wang et al., *The Allen Mouse Brain Common -Coordinate Framework: A 3D Reference Atlas*, Cell 2020, -[doi:10.1016/j.cell.2020.04.007](https://doi.org/10.1016/j.cell.2020.04.007), together with the -exact BrainGlobe atlas name and version. +Keep the VesSAP loader and provenance checks, but require the NPZ and manifest to be installed in +the application data directory: -All UI and saved trajectories use an application-defined coordinate model. Conversion to atlas -array indices, BrainGlobe axes, or any future iblatlas representation happens only at named -adapter boundaries and is covered by round-trip tests. +```text +/vasculature/vessap_bl6j1_major_vessels_50um_v1.npz +/vasculature/vessap_bl6j1_major_vessels_50um_v1.npz.manifest.json +``` -## Memory policy +The bridge advertises the display capability only when both external files are present. The data +remains governed by CC BY-NC 4.0 and is not part of Brain3D's source-code license. -AtlasAPI 2.3.1 loads complete TIFF arrays with `tifffile.imread`; it does not provide chunked -array access in this stable release. The implementation is visible in -[`core.py`](https://github.com/brainglobe/brainglobe-atlasapi/blob/v2.3.1/brainglobe_atlasapi/core.py#L122-L135), -and its [descriptors](https://github.com/brainglobe/brainglobe-atlasapi/blob/v2.3.1/brainglobe_atlasapi/descriptors.py#L51-L57) -declare uint16 reference and uint32 annotation arrays. +### Documents -| Atlas | Shape | Reference + annotation raw bytes | Approx. raw memory | -|---|---:|---:|---:| -| `allen_mouse_25um` | `528 × 320 × 456` | uint16 + uint32 | 0.462 GB / 0.43 GiB | - -These figures exclude Python objects, temporary copies, derived masks, and display meshes, so peak -resident memory is higher. Load structures and meshes on demand, release intermediates promptly, -and never create an unbounded resident full-volume copy merely for display. Adding another -resolution is a separate product/validation decision; no silent substitution is allowed. - -[BrainGlobe AtlasAPI 3.0.0rc1](https://pypi.org/project/brainglobe-atlasapi/3.0.0rc1/) -introduces a newer storage path, but it is a prerelease and is not the production dependency. The -adapter boundary exists so a future chunked implementation can be evaluated without changing -project coordinates or provenance. - -## Archived population vascular-density decision - -The optional population layer uses exactly Yongsoo Kim's *Cerebrovascular, pericyte, and -neuronal cell type mapping data 2022*, -[Mendeley Data v1, DOI 10.17632/stxvn5sv44.1](https://data.mendeley.com/datasets/stxvn5sv44/1), -licensed CC BY 4.0 and associated with -[Wu et al., Cell Reports 2022](https://doi.org/10.1016/j.celrep.2022.110978). - -Acquisition is fail-closed. The accepted archive is -`NVU_mapping_Adult_mouse_brain (1).7z`, exactly 311,493,514 bytes, SHA-256 -`c715c92ad153bff7f676b883f47108f886147e5d6fcd4502bcc04a0f92ed98fe`. Only two exact members are -streamed into application-owned staging: the vascular length-density NIfTI and its Allen template. -Their names, sizes, SHA-256 values, NIfTI header evidence, and the derived-cache manifest are -validated before an atomic promotion. - -The reviewed source contract is a 20 µm `(570,400,660)` `[ML,DV,AP]` field with values in -`m/mm^3`, four fixed adult mice, and a 100 µm local window. The density NIfTI itself has unit -zooms, unknown units, and no qform/sform, so the implementation validates that exact caveat and -uses the pinned README/template contract rather than treating the header as authoritative. AP is -reversed into BrainGlobe ASR, ML is deliberately symmetrized because source polarity is not -documented, and the prepared result is a 50 µm `[AP,DV,ML]` scalar field. Preparation requires an -exact target-atlas identity and template correlation of at least 0.99. - -The backend can produce a declared transparent AP-by-ML DV maximum projection with the source, -atlas binding, units, display window, and limitations intact. That path is retained for archived -work but is not requested by the primary SwiftUI workspace. This is a population scalar density, -not individual vessel paths, not subject-specific anatomy, and not used for vessel analysis. The -separate simulation-ready graph deposit `10.17632/mjtyry6v85.1` is rejected for planning -integration: its documentation exposes raw specimen-space XYZ without a qualified axis -orientation, laterality, Allen transform, or bregma relationship. - -## Archived LAMBADA major-vessel decision - -The archived evidence uses the atlas-registered P60_606 graph from Renier, de Launoit, and -Skriabine's *Vascular graphs of the developing post-natal mouse brain*, Zenodo record -`10.5281/zenodo.18876865`, CC BY 4.0. The repository bundles a deterministic compact derivative, -not the 5.05 GB source archive or 12.28 GB extracted graph. - -Extraction keeps maximal consecutive in-bounds source-edge runs only where each point has radius -≥15 µm. The manifest binds the source/archive identities, conversion from ClearMap to -BrainGlobe `[AP,DV,ML]`, physical 25 µm scaling, output arrays/counts, asset SHA-256, and mandatory -limitations. These checks establish deterministic derivation, not coordinate qualification. - -The exact qualification rerun found supporting AP and DV orientation evidence. It rejected the -asset because the primary record describes hemisphere specimens and the exact graph does not -persist a graph, vertex, or edge property that binds its numerical ML coordinates to biological -hemisphere/laterality. Coordinates occurring on both sides of an array midpoint do not establish -whole-brain coverage. No approved exact-specimen transform supports mirroring, so the application -does not infer a side or mirror the derivative. - -No LAMBADA handler or capability is registered in the current runtime. The production -`auditedReferenceMajorVessels` capability and reference metadata/geometry methods now belong -exclusively to the separately qualified VesSAP display reference; they never load P60_606. -`radiusAwareReferenceVesselAnalysis` remains absent. The canonical LAMBADA rejection report is -[`lambada_p60_606_coordinate_qualification_rejected_v1.json`](evidence/lambada_p60_606_coordinate_qualification_rejected_v1.json), -SHA-256 `0993d5a0ad6c0d62094dc395fe2bc4f284870e6e7c0b602be7df5a7da867c93a`. - -This remains a fixed cleared reference, not the animal being planned. The source omits pial and -choroidal vessels; the derivative omits smaller vessels; artery/vein identity is unavailable; -and biological variation, tissue distortion, registration error, and omitted vessels are not -bounded. It cannot support a visual overlay, vessel conflict, or surgical-clearance claim. - -## VesSAP display-only major-vessel decision - -Use the official VesSAP `BL6J-no1` 3 µm skeleton/radius volumes and published Euler + B-spline -Allen registration for a C57BL/6J population-reference display layer. Retain source skeleton -points with radius ≥5 voxels, preserve only true 26-neighbour source adjacency, transform -continuous coordinates with the mandatory ML reflection, and coalesce on a 50 µm display grid. -The bundled NPZ, adjacent strict manifest, CC BY-NC 4.0 license text, and exact source/transform -digests must remain together. - -The backend may advertise `auditedReferenceMajorVessels` and serve metadata/geometry only after -all integrity checks pass. It must not advertise `radiusAwareReferenceVesselAnalysis`. -Clearance analysis fails with `VESSEL_ANALYSIS_UNAVAILABLE` before reading or mutating project -state. The display must identify one fixed cleared ex-vivo specimen, omitted capillaries, and the -absence of subject-registration/tissue-distortion bounds. - -This decision does not supersede the LAMBADA rejection or authorize mixing/mirroring datasets. -The detailed transform, validation statistics, counts, and digests are in -[VESSAP_MAJOR_VESSELS.md](VESSAP_MAJOR_VESSELS.md). - -## Cache, download, and offline behavior - -Atlas data is user cache/application data and is **not bundled inside the application**. Resolve -paths with platformdirs under the user's macOS Application Support/cache directories. Set -`BRAINGLOBE_CONFIG_DIR` before the first `brainglobe_atlasapi` import, because version 2.3.1 -reads it during module initialization; pass the application-owned atlas and intermediate-download -directories explicitly where the API permits. The relevant behavior is documented in -[`config.py`](https://github.com/brainglobe/brainglobe-atlasapi/blob/v2.3.1/brainglobe_atlasapi/config.py#L16-L25). - -Downloads run outside the GUI thread. The supported SwiftUI shell currently shows indeterminate -download/preparation progress and does not expose a cancellation control; cancellation and -quantitative progress remain future UI work. Exact-version acquisition uses BrainGlobe inside -unique application-owned archive and atlas staging directories; -the adapter parses metadata and structures and inspects reference/annotation TIFF headers to -validate reviewed identity, species, package version, orientation, resolution, hierarchy, -uint16/uint32 types, volume shapes, and path containment without loading whole arrays. Only -packages passing that check receive a catalog `downloaded` state. The staged package is then -atomically renamed into the visible cache. Existing valid versions and valid concurrent targets -are preserved. An invalid same-name target is atomically moved beneath the app-owned -`atlases/quarantine/` directory before promotion, so evidence remains recoverable and the valid -replacement prevents a redownload loop. -Cached-only open uses BrainGlobe's low-level local reader, cannot call its downloader, and must -validate the exact requested version. Cache-removal and quarantine-management UI are deferred. - -Catalog work is invoked only by explicit native/CLI acquisition operations; ordinary atlas open -is cache-only. -Production does not call AtlasAPI 2.3.1's timeout-less catalog helper or abandon it in a daemon -thread. The adapter fetches the same official `last_versions.conf` endpoint directly with a short -socket timeout, a 15-second total deadline, cooperative checks between bounded reads, and a 1 MiB -response limit. A validated response replaces the app-owned cache atomically; network or deadline -failure falls back to a previously validated cached catalog. Cancellation returns without leaving -catalog work running in another thread. - -Stable AtlasAPI downloads do not publish or enforce a cryptographic expected hash for each -atlas archive. The application records the SHA-256 of the installed `metadata.json` for exact metadata -identity. The separately measured whole-file hashes in `SCIENTIFIC_VALIDATION.md` are validation -evidence, not persisted project fields and not upstream authentication. A future package-wide -integrity feature must define and version its manifest before claiming corruption or content -drift detection. Structural JSON/TIFF-header validation detects malformed or internally -inconsistent packages, not anatomically plausible tampering. Pooch may be used for other external -files only when an authoritative expected hash is available. - -## Code, data, and prior art are separate - -| Item | Classification | License/terms decision | -|---|---|---| -| BrainGlobe AtlasAPI | Executed dependency code | BSD-3-Clause; pin and attribute it | -| Allen Mouse CCF data obtained through BrainGlobe | Downloaded scientific data | Governed by the [Allen Institute Terms of Use](https://alleninstitute.org/legal/terms-of-use), not by AtlasAPI's BSD license | -| brainrender 2.2.0 | Prior-art concept/API reference only | BSD-3-Clause; not a runtime dependency | -| iblatlas 1.2.0 | Prior-art coordinate/trajectory reference only | MIT; not a runtime dependency; do not install its PyQt5 GUI extra | -| Neuropixels Trajectory Explorer v2.0.0 | Prior-art workflow reference only | GPL-3.0; no copied code or assets | -| Pinpoint v2.0.0 | Prior-art workflow reference only | GPL-3.0; no copied code or assets | -| cortex-lab/allenCCF and SHARP-Track | Prior-art workflow reference only | No repository license found; no copied code or assets | -| Kim 2022 population vascular length-density data, DOI `10.17632/stxvn5sv44.1` | Optional downloaded scientific data | Mendeley Data v1, CC BY 4.0; exact archive/member identities are pinned. Archived backend preparation only; never vessel paths, a subject layer, or clearance geometry. | -| LAMBADA P60_606 vascular graph, DOI `10.5281/zenodo.18876865` | Archived derived scientific evidence | CC BY 4.0; exact source and derivative identities are pinned. Coordinate qualification is rejected, so it is not displayed, served, mirrored, or analyzed. | -| VesSAP BL6J-no1 whole-brain vasculature | Bundled display-only derived scientific data | CC BY-NC 4.0; exact source/transform/asset identities are pinned. It is displayed in all five views but cannot support subject clearance, vessel absence, suitability, or safety. | -| Wu et al. simulation-ready vascular tracing data, DOI `10.17632/mjtyry6v85.1` | Rejected for planning integration | Version 1 is CC BY 4.0 and documents four traced adult-mouse graphs in MATLAB format, but the documented coordinates are raw specimen-space XYZ with no qualified axis orientation, laterality, Allen transform, or bregma relationship. No graph is integrated or bundled. | -| VesselGraph | Prior-art vascular graph/data reference only | Software is MIT; data is CC BY-NC 4.0. No code, models, or data copied. The noncommercial restriction prevents treating it as an unrestricted distributable default. | -| VesSAP repository code | Reviewed derivation/workflow reference only; not a runtime dependency | Repository code is MIT and was not copied. The separately licensed bundled BL6J-no1 data derivative is the CC BY-NC 4.0 artifact documented in the row above. | - -The Allen terms currently restrict covered Content to noncommercial research unless otherwise -stated. The application must show the source and terms before first download, retain attribution, -and avoid redistributing the atlas inside the `.app` or an installer. Commercial distribution, -hosted redistribution, or a change in Allen terms requires legal review before release. - -Pinpoint remains a workflow reference rather than a runtime component. The hosted Unity WebGL -application does not provide a supported bidirectional contract for exact probe identity, -coordinate/transform provenance, camera/region state, or the VesSAP overlay. Brain3D uses its -existing typed BrainGlobe endpoints for the complete ontology and lazy meshes instead. See -[the Pinpoint interoperability decision](PINPOINT_INTEGRATION.md). - -The archived `stxvn5sv44.1` density and rejected `mjtyry6v85.1` vessel graphs are distinct -deposits and must never be conflated. A new source still needs a stable URL, version, coordinate -registration, citation, redistribution terms, integrity strategy, scientific semantics, and -fail-closed UI labeling before it can be displayed. Visual ideas from prior art may inform -independently written code, but repository code, meshes, screenshots, icons, and other assets -must not be copied unless their license is explicitly compatible and the reuse is recorded in -`THIRD_PARTY.md`. +Protocol and reference-atlas PDFs are selected by the user in Settings. Brain3D reads them from +their original locations and does not copy them into source control or the application bundle. ## Consequences -- First use requires a network download; later use is offline from an application-owned cache. -- Only the explicitly reviewed Allen mouse 25 µm package-v1.2 identity is discoverable or - openable in the current build. -- 10 µm is deferred. Existing source or derived cache data is left untouched but cannot enter a - current project package. -- The Mendeley density and subject-image workflows remain archived and absent from the primary UI. -- VesSAP `BL6J-no1` is the only runtime major-vessel layer; it is display-only and separately - licensed CC BY-NC 4.0. -- The LAMBADA derivative remains archived evidence and cannot render or enter analysis. Any future - vessel source needs a new qualification that binds trustworthy whole-brain coverage and - biological laterality without an inferred or mirrored hemisphere. -- Project files carry enough provenance to enforce exact metadata identity and prevent silent - coordinate reinterpretation; the current build does not claim package-wide content-drift - detection. -- Atlas upgrades, new external datasets, or copied prior-art material require a new review of - scientific provenance, terms, and `THIRD_PARTY.md` before implementation. +- A clean checkout contains code, tests, fixtures, and documentation but no third-party datasets + or PDFs. +- Users control acquisition and acceptance of external terms. +- Missing optional data disables its capability without affecting atlas and probe planning. +- Source identities and digests remain explicit at data-loading boundaries. diff --git a/docs/ADR-004-swiftui-hybrid-shell.md b/docs/ADR-004-swiftui-hybrid-shell.md index 44988e7..f0afd67 100644 --- a/docs/ADR-004-swiftui-hybrid-shell.md +++ b/docs/ADR-004-swiftui-hybrid-shell.md @@ -80,20 +80,19 @@ process decision, is: complete 10 mm shaft while slice/traversal analysis uses only the implanted surface-to-tip path; - no separate target-registration, subject-calibration, or geometry-checkbox prerequisite for a v4 direct plan; legacy v1–v3 records retain their original archived semantics; -- a VesSAP BL6J-no1 diameter-≥30 µm reference overlaid in all five views, with the older - LAMBADA P60_606 derivative retained only as rejected evidence; and +- an optional external VesSAP BL6J-no1 diameter-≥30 µm reference overlaid in all five views; and - no reference-vessel analysis capability. Region export uses a two-phase boundary: Python generates and hashes content without mutation, the native client performs an atomic file write, and only a matching confirmation records the `exported` audit event. Cancelling the save panel or a failed write leaves the project unchanged. -The service advertises `auditedReferenceMajorVessels` and serves the exact VesSAP display -geometry after source/transform/asset checks. It omits +When the external data package is installed, the service advertises +`auditedReferenceMajorVessels` and serves VesSAP display geometry after +source/transform/asset checks. It omits `radiusAwareReferenceVesselAnalysis`; `vessel.major.reference.analyze` fails with `VESSEL_ANALYSIS_UNAVAILABLE`. The source is one cleared ex-vivo C57BL/6J reference with no -published subject-registration or tissue-distortion error bounds. The older LAMBADA rejection -remains bound to its canonical report and cannot be bypassed or mirrored into VesSAP. +published subject-registration or tissue-distortion error bounds. Population density and subject-image registration remain archived compatibility code and persisted data. The primary bridge does not register their methods or capabilities, and the diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md index ea15bbb..2f5dbdb 100644 --- a/docs/ARCHITECTURE.md +++ b/docs/ARCHITECTURE.md @@ -78,14 +78,14 @@ Pinpoint/Urchin AP/ML reference + loaded annotation selected Allen ontology identity → descendant annotation mask ─→ Dorsal/Coronal/Sagittal/Horizontal highlight └─→ reviewed mesh or explicit no-geometry result ─→ SceneKit 3D -VesSAP BL6J-no1 → pinned transform + true-path 50 µm reduction - ├─→ five-view display geometry - └─→ VESSEL_ANALYSIS_UNAVAILABLE (no clearance) -LAMBADA P60_606 → archived rejected evidence (no runtime payload) +external VesSAP BL6J-no1 → manifest + digest validation + ├─→ five-view display geometry + └─→ VESSEL_ANALYSIS_UNAVAILABLE (no clearance) ``` -The runtime advertises `auditedReferenceMajorVessels`, verifies the VesSAP manifest/NPZ, and -serves metadata plus binary display geometry. It does not advertise +When the external files are installed, the runtime advertises +`auditedReferenceMajorVessels`, verifies the VesSAP manifest/NPZ, and serves metadata plus +binary display geometry. It does not advertise `radiusAwareReferenceVesselAnalysis`; `vessel.major.reference.analyze` fails before project access. The UI cannot manufacture a clearance result from displayed geometry or pixels. @@ -96,12 +96,6 @@ laterality, topology, bounds, and asset identity have display evidence, but subj clearing/inter-animal variation, and a qualified transformed vessel surface remain unbounded. See [the VesSAP derivation record](VESSAP_MAJOR_VESSELS.md). -The P60_606 diameter-≥30 µm derivative remains archived evidence, not a primary vessel layer. Its AP -and DV orientation evidence passed, but the source is a hemisphere specimen and the exact graph -has no persisted biological hemisphere/laterality binding. Whole-brain coverage and ML polarity -are unqualified; no mirroring is permitted. The digest-bound canonical rejection report is -[`lambada_p60_606_coordinate_qualification_rejected_v1.json`](evidence/lambada_p60_606_coordinate_qualification_rejected_v1.json), -SHA-256 `0993d5a0ad6c0d62094dc395fe2bc4f284870e6e7c0b602be7df5a7da867c93a`. Population density and subject-image registration remain separate archived compatibility code and persisted data. Their methods/capabilities are not registered by the primary bridge, they remain @@ -117,13 +111,11 @@ only after the native atomic write is confirmed. Atlas identity mismatch, changed bregma/annotation source evidence, failed v4 surface rederivation, failed legacy calibration QC, unknown coordinate frames, source/asset digest -mismatch, a missing or changed manifest/qualification report, malformed geometry, and stale +mismatch, a missing or changed external manifest, malformed geometry, and stale project or plan hashes fail closed. The app does not substitute another atlas, present the -Pinpoint/Urchin convention as Allen ground truth, extrude a 2D density field into vessels, infer -hemisphere, mirror P60_606, or treat either fixed reference as a subject measurement. +Pinpoint/Urchin convention as Allen ground truth, extrude a 2D density field into vessels, infer anatomy from missing external data, or treat a fixed reference as a subject measurement. See [ADR-004](ADR-004-swiftui-hybrid-shell.md), [ADR-005](ADR-005-independent-slice-viewer.md), and -[the Pinpoint interoperability decision](PINPOINT_INTEGRATION.md), -[the VesSAP derivation record](VESSAP_MAJOR_VESSELS.md), and -[the LAMBADA derivation record](LAMBADA_MAJOR_VESSELS.md). +[the Pinpoint interoperability decision](PINPOINT_INTEGRATION.md), and +[the VesSAP data contract](VESSAP_MAJOR_VESSELS.md). diff --git a/docs/CODE_AUDIT.md b/docs/CODE_AUDIT.md deleted file mode 100644 index 011948e..0000000 --- a/docs/CODE_AUDIT.md +++ /dev/null @@ -1,237 +0,0 @@ -# Code Audit - -Audit baseline: `main` commit `51fe26d637b700d39944bbee86e3ea42ded4e7a2`, reviewed on -2026-07-22 before the end-to-end planner refactor. - -> **Historical snapshot:** every present-tense “current” statement below refers to that baseline, -> not the current development tree. A later refactor made calibration, target projection, probe -> planning/region traversal, SceneKit 3D, and temporarily the LAMBADA/V3 path product-reachable. -> The current digest-bound qualification gate rejects P60_606 geometry and analysis; a separate -> VesSAP BL6J-no1 derivative is now available as display-only geometry. See -> [Project Status](../PROJECT_STATUS.md) for the live inventory. - -This document records evidence about what is reachable from the supported product, what is only -covered by tests, and what belongs to the superseded Qt/VTK application. A symbol is not deleted -merely because a text search finds no caller: the audit checks static imports, repository-wide -references, dynamic bridge registration, and an actual runtime entry point. - -## Classification - -- `PRODUCT_REACHABLE`: reachable from the SwiftUI application through the Python bridge or an - explicit supported CLI command. -- `TEST_ONLY`: reached by tests but not a supported runtime entry point. -- `LEGACY_ONLY`: reached only from the former Qt/PyVista/VTK application. -- `ORPHANED_BUT_REUSABLE`: not product-reachable, but required by the target planner and covered - by useful scientific tests. -- `DELETE_CANDIDATE`: no supported caller, no retained roadmap consumer, and no independent - compatibility obligation found. -- `UNKNOWN_DYNAMIC_USAGE`: static evidence is insufficient; retain until runtime evidence exists. - -## Audit method - -1. Trace imports from `native/Brain3D` backend launch, `mouse_brain_planner.bridge.server`, each - CLI command, and every test collector. -2. Search all symbol references and bridge method strings with `rg`; inspect dynamic - `dispatcher.register(...)` calls separately. -3. Launch/test the native package and bridge transport, then compare methods requested by - `PlannerViewModel.swift` with registered Python handlers. -4. Classify the current path before changing it; record affected tests with each removal. - -## Baseline - -### Repository and environment - -```text -branch: main, clean, tracking origin/main -commit: 51fe26d637b700d39944bbee86e3ea42ded4e7a2 -remote: https://github.com/KaiCao2003/brain3d.git -host: macOS 26.5.2 (25F84), arm64 -system python: 3.14.3 -project python through uv: 3.12.13 -uv: 0.10.6 -Swift: 6.3.3, target arm64-apple-macosx26.0 -tracked files: 156 -Python test files: 45 -Swift test files: 7 -``` - -The system Python is outside the project contract. All Python evidence below uses `uv run ---frozen`, which selects Python 3.12.13 from the locked environment. - -### Commands and exact pre-refactor results - -| Command | Exit | Result | -| --- | ---: | --- | -| `uv lock --check` | 0 | 76 packages resolved; lock accepted | -| `uv run --frozen ruff format --check .` | 0 | 106 files already formatted | -| `uv run --frozen ruff check .` | 0 | all checks passed | -| `uv run --frozen mypy --no-incremental` | 0 | no issues in 61 source files | -| `QT_QPA_PLATFORM=offscreen PYVISTA_OFF_SCREEN=true uv run --frozen pytest -q` | 0 | 484 passed, 1 opt-in real-archive test skipped, 487 VTK deprecation warnings | -| `swift test --package-path native/Brain3D` | 0 | 42 tests in 7 suites passed | -| `native/Brain3D/Scripts/build-app.sh` | 0 | production Swift executable and development `.app` built | -| `codesign --verify --deep --strict native/Brain3D/build/Brain3D.app` | 0 | ad-hoc signature verified | - -These results are engineering baseline evidence, not evidence that the supported UI is usable. -The Swift test target contains `Brain3DCore` only; it does not exercise `Brain3DApp` view -interaction. Most of the 487 warnings come from tests of the legacy VTK path. - -## Runtime entry points - -| Entry point | Current behavior at baseline | Status | Decision | -| --- | --- | --- | --- | -| `native/Brain3D/.../Brain3DApp.swift` | Supported macOS UI; discovers `.venv` and launches `python -u -m mouse_brain_planner.bridge.server` | `PRODUCT_REACHABLE` | Retain and extend | -| `python -m mouse_brain_planner.bridge.server` | NDJSON service with atlas, project, density, image-registration, and unprojected-target handlers | `PRODUCT_REACHABLE` | Retain; split by protocol responsibility as v2 grows | -| `mouse-brain-planner atlas ...` | Explicit atlas cache commands | `PRODUCT_REACHABLE` | Retain under explicit subcommands | -| `mouse-brain-planner validate ...` | Explicit project validation | `PRODUCT_REACHABLE` | Rename to `validate-project`, with a documented compatibility window if needed | -| `mouse-brain-planner` with no subcommand | Imports `app.py` and silently launches the old Qt UI | `LEGACY_ONLY` | Remove; print help without importing Qt | -| `mouse-brain-planner --smoke-test` | Exercises only the old Qt UI | `LEGACY_ONLY` | Remove with Qt UI tests | - -## Path-level audit table - -| Path / symbol | Current caller | Product status | Decision | Reason | Tests affected | -| --- | --- | --- | --- | --- | --- | -| `native/Brain3D/Sources/Brain3DApp` | macOS executable | `PRODUCT_REACHABLE` | Extend | Sole supported GUI | Add app/ViewModel tests; current core tests are insufficient | -| `native/Brain3D/Sources/Brain3DCore` | Swift app | `PRODUCT_REACHABLE` | Extend | Typed subprocess transport and fail-closed validation | Existing 42 Swift core tests | -| `bridge/server.py` | Swift subprocess | `PRODUCT_REACHABLE` | Retain | Framing, dispatcher, atlas opening/slicing | Bridge unit/integration tests | -| `bridge/planning.py` | Dynamic handler registration | `PRODUCT_REACHABLE` | Split incrementally | Too many project/vascular/session duties in one file | Planning and persistence integration tests | -| `bridge/atlas_interaction.py` slice/point methods | Dispatcher; slice used by Swift, point not yet used | Mixed `PRODUCT_REACHABLE` / `ORPHANED_BUT_REUSABLE` | Reuse for tri-planar cursor/region lookup | Backend already supports arbitrary slice and point queries | Atlas interaction tests | -| `bridge/implant_targets.py` | Swift sidebar | `PRODUCT_REACHABLE` | Migrate, do not infer calibration | Preserves exact unprojected AP/ML/DV | Implant bridge tests and migrations | -| `atlas/brainglobe_adapter.py` | Bridge and CLI | `PRODUCT_REACHABLE` | Retain | Allowlisted atlas and metadata boundary | Atlas adapter tests | -| `rendering/slice_renderer.py` | `atlas.slice` | `PRODUCT_REACHABLE` | Retain | Arbitrary index rendering already exists | Slice renderer tests | -| `persistence/project_io.py` provenance-aware APIs | Bridge | `PRODUCT_REACHABLE` | Extend with deterministic migrations | Atomic/checksummed persistence is valuable | Project I/O and integration tests | -| `coordinates/atlas_space.py` | Atlas/bridge/rendering | `PRODUCT_REACHABLE` | Centralize all order conversions here or a named adjacent module | Prevent distributed AP/DV/ML permutations | Coordinate golden tests | -| `coordinates/transforms.py` | Calibration and probe bridge | `PRODUCT_REACHABLE` | Retain behind explicit frame contracts | Fits and applies the subject-to-atlas boundary | Transform, calibration, and probe tests | -| `domain/*probe*`, `*stereotaxy*`, `*transform*`, `*measurement*` | Calibration/probe bridge | `PRODUCT_REACHABLE` | Retain strict schemas and provenance | Four-mode probe planning depends on these constraints | Domain, bridge, and workflow tests | -| `surgery/trajectory.py` | Probe planning service | `PRODUCT_REACHABLE` | Retain complete-pose rigid/similarity projection | Preserves shank/site geometry and rejects affine shear | Probe trajectory and bridge tests | -| `surgery/stereotaxy.py` | Calibration and probe services | `PRODUCT_REACHABLE` | Retain | Defines the measured bregma/skull frame | Stereotaxy and calibration tests | -| `surgery/measurements.py` | Tests only | `ORPHANED_BUT_REUSABLE` | Retain exact segment-distance kernel; rename centerline-only APIs before exposure | Needed by future radius-aware clearance | Measurement tests | -| `surgery/craniotomy.py` | Tests only | `TEST_ONLY` | Move out of MVP product package or defer | Not a current MVP consumer | Craniotomy tests | -| `vasculature/reference_*`, `density_overlay.py` | Archive compatibility tests only | `ARCHIVED_NOT_REGISTERED` | Preserve code/data; do not register in the primary bridge | Scalar population density is not vessel geometry | Density/store/archive tests | -| `vasculature/subject_image.py`, `registration.py`, `subject_overlay.py` | Archive compatibility tests only | `ARCHIVED_NOT_REGISTERED` | Preserve code/data; do not register in the primary bridge | A 2D subject image cannot support deep clearance | Subject image/registration archive tests | -| `rendering/sagittal_cache.py` | Tests only | `TEST_ONLY` | Remove from MVP package | Optimizes a deferred 10 µm path with no product caller | Sagittal cache tests | -| `gui/**` | `app.py` and Qt tests | `LEGACY_ONLY` | Remove from default package; preserve history in Git | Second UI/state/rendering stack conflicts with supported path | All `tests/gui/**` and Qt worker tests | -| `app.py` | CLI no-subcommand path | `LEGACY_ONLY` | Remove after CLI behavior changes | It makes an unsupported GUI the default | Smoke and CLI default tests | -| `rendering/scene_controller.py` | Legacy Qt UI/tests | `LEGACY_ONLY` | Remove from default package | PyVista-only scene state duplicates the target Swift path | Scene-controller and Qt atlas tests | - -## Bridge runtime reachability - -Primary registration is the supported product boundary and must not be inferred from Python -imports alone. Archived subject-image and population-density implementations remain callable only -when compatibility tests bind them explicitly; the production dispatcher does not register or -advertise them. - -The supported primary Swift workspace requests project/atlas state, `atlas.open`, arbitrary slice -and dorsal rendering, mesh/ray-pick methods, revision-safe implant/calibration CRUD, atomic viewer -navigation, the versioned probe catalog and four-mode plan CRUD, exact region traversal, and the -two-phase region-export methods. - -Legacy population-density and subject-image calls remain implemented behind archived compatibility -code, but are neither registered nor exposed in the primary UI. Their presence is migration and -research-code retention, not evidence that they are supported surgical-planning inputs. - -Registered compatibility/read-only methods that are not central to the current Swift path include -`atlas.list`, `atlas.regions`, `atlas.search`, `atlas.point`, and `shutdown`. The independent viewer -uses `viewer.region.pick`; `atlas.mesh` and atlas ray picking are product-reachable through 3D. -The three `vessel.major.reference.*` methods remain registered only to return the explicit -`VESSEL_GEOMETRY_UNAVAILABLE` fail-closed response; they never serve the rejected graph. - -## Resolved contradictions and remaining archive boundaries - -- The project schema retains legacy linked-cursor fields for migration; the supported viewer uses - independent slice depths plus a replace-only region selection. -- Native SceneKit 3D brain/probe rendering and atlas-region picking are product-reachable; the old - Qt/PyVista renderer is preserved only in Git history. -- Qt/VTK and the legacy GUI are absent from the default package and dependency graph. -- The console command without a subcommand prints help; bridge operation is explicit. -- The archived density overlay is reproducible population context, not individual blood vessels. -- The rejected P60_606 derivative remains evidence only and is never served as geometry. - -## Scientific and data risks to keep fail-closed - -- Allen atlas physical coordinates are not bregma-relative coordinates. -- No audited reference-only bregma profile currently exists; no implicit transform may be added. -- Population density has no vessel centerlines, radius, or retained depth in its dorsal maximum - projection. -- Subject dorsal images have no automatic vessel segmentation and cannot represent deep vessels. -- A radius/probe-envelope/margin/uncertainty kernel exists, but no coordinate-qualified vessel - graph is available; the product therefore exposes no clearance result. -- Probe profiles may not be called manufacturer-verified without complete source and independent - review records. - -## Phase 1 removal gate - -Before deleting legacy files, the refactor must demonstrate all of the following: - -1. no supported Swift/bridge import reaches Qt/VTK; -2. CLI no-subcommand and explicit bridge behavior have replacement tests; -3. minimal dependency installation can import and run the bridge without Qt/VTK; -4. tests retained from scientific modules remain green independently of GUI tests; and -5. Git history preserves the removed implementation for forensic reference. - -The symbol-by-symbol appendix below records the complete reference scan. Ambiguous dynamic -consumers would remain `UNKNOWN_DYNAMIC_USAGE`; this scan found none in the enumerated set. - -## Enumerated symbol audit - -The repository-wide search found no `getattr`, `importlib`, package entry-point, or method-string -loader for the functions below. Dynamic usage is confined to explicit bridge method -registration, which was audited separately. Therefore none of these symbols remains -`UNKNOWN_DYNAMIC_USAGE`. - -| Symbol | Current caller at baseline | Classification | Phase decision | -| --- | --- | --- | --- | -| `paths.configure_qt_environment` | `gui/__init__.py` import side effect | `LEGACY_ONLY` | Deleted with the Qt package | -| `coordinates.transforms.fit_anatomical_transform` | transform tests | `ORPHANED_BUT_REUSABLE` | Retain and connect to calibration | -| `coordinates.transforms.transform_point` | transform/stereotaxy tests | `ORPHANED_BUT_REUSABLE` | Retain and connect to calibration/projection | -| `coordinates.transforms.transform_points` | transform tests | `ORPHANED_BUT_REUSABLE` | Retain for bounded batch transforms | -| `coordinates.transforms.transform_vector` | transform tests | `ORPHANED_BUT_REUSABLE` | Retain for probe directions | -| `coordinates.transforms.invert_transform` | transform tests | `ORPHANED_BUT_REUSABLE` | Retain for explicit inverse mapping | -| `coordinates.transforms.compose_transforms` | transform tests | `ORPHANED_BUT_REUSABLE` | Retain for named transform chains | -| `surgery.stereotaxy.calibrate_skull_landmarks` | stereotaxy/probe tests | `ORPHANED_BUT_REUSABLE` | Retain and expose through calibration service | -| `surgery.measurements.point_to_point_distance` | measurement tests | `ORPHANED_BUT_REUSABLE` | Retain as explainable geometry primitive | -| `surgery.measurements.point_to_line_distance` | measurement tests | `ORPHANED_BUT_REUSABLE` | Retain as explainable geometry primitive | -| `surgery.measurements.path_length` | measurement tests | `ORPHANED_BUT_REUSABLE` | Retain for trajectory/region summaries | -| `surgery.measurements.angle_between_directions` | measurement tests | `ORPHANED_BUT_REUSABLE` | Retain for placement inspection | -| `surgery.measurements.distance_between_probe_centerlines` | measurement tests | `ORPHANED_BUT_REUSABLE` | Retain, keep centerline-only naming | -| `surgery.measurements.nearest_vessel_distance` | measurement tests | `ORPHANED_BUT_REUSABLE` | Keep compatibility temporarily; rename to centerline distance before product exposure | -| `surgery.trajectory.placement_from_entry_target` | probe/measurement tests | `ORPHANED_BUT_REUSABLE` | Retain for `ENTRY_AND_TARGET` | -| `surgery.trajectory.placement_from_entry_angles_depth` | probe tests | `ORPHANED_BUT_REUSABLE` | Retain for `ENTRY_ANGLES_DEPTH` | -| `surgery.trajectory.placement_from_bregma_relative_mm` | probe tests | `ORPHANED_BUT_REUSABLE` | Retain; expose only after active calibration passes QC | -| `surgery.trajectory.attach_surface_entries` | probe tests | `ORPHANED_BUT_REUSABLE` | Retain behind a validated surface-intersection service | -| `surgery.trajectory.placed_recording_sites` | probe tests | `ORPHANED_BUT_REUSABLE` | Retain and expose in plan/region analysis | -| `surgery.trajectory.placement_permits_final_export` | probe tests | `ORPHANED_BUT_REUSABLE` | Retain verification/provenance gate | -| `surgery.craniotomy.craniotomy_metrics` | craniotomy tests | `DELETE_CANDIDATE` for MVP | Leave isolated for now; remove or move before Phase 4 if no surface consumer exists | -| `surgery.craniotomy.craniotomy_outline_points` | craniotomy tests | `DELETE_CANDIDATE` for MVP | Same decision as craniotomy metrics | -| `domain.implant_site_models.validate_bregma_decimal_inputs` | unprojected-target test | `DELETE_CANDIDATE` | Remove when bridge/Swift parsing is consolidated in protocol v2 | -| `domain.implant_site_models.unprojected_bregma_target_summary` | unprojected-target test | `DELETE_CANDIDATE` | Remove duplicated presentation helper during target migration | -| `persistence.project_io.load_project` | compatibility tests | `TEST_ONLY` | Deprecate; product uses `load_project_with_provenance` | -| `rendering.sagittal_cache.prepare_sagittal_cache` | sagittal-cache tests only | `DELETE_CANDIDATE` | Deleted with deferred 10 µm cache path | -| `vasculature.registration.transform_image_pixels_to_atlas` | registration tests | `ORPHANED_BUT_REUSABLE` | Retain for 2D surface-entry analysis | -| `gui.dialogs.dorsal_vascular_registration.DorsalVascularRegistrationDialog` | Qt dialog tests only; no `MainWindow` caller | `DELETE_CANDIDATE` | Deleted; Swift already has `VascularRegistrationSheet` | - -## Phase 1 execution evidence - -The single-product-path cleanup was then applied in the same audited working tree: - -- the no-subcommand CLI now prints help and returns 2; -- `mouse-brain-planner bridge` is an explicit, tested service entry point; -- `app.py`, `gui/**`, `scene_controller.py`, `sagittal_cache.py`, and their legacy-only tests were - deleted; the prior implementation remains at Git commit `51fe26d`; -- PySide6, PyVista, PyVistaQt, VTK, pytest-qt, PyInstaller, unused direct pandas, and unused pooch - declarations were removed; -- the lock resolved 48 packages instead of 76 and contains none of those removed packages; -- the retained default runtime imports exactly the BrainGlobe/array/model/image dependencies - documented in `pyproject.toml`; -- `.venv/bin/python scripts/verify_minimal_runtime.py` confirmed all five forbidden module roots - are unavailable and completed a real explicit-CLI `hello`/`shutdown` exchange; and -- the retained suite completed with `377 passed, 1` external real-archive test skipped and no VTK - warnings. - -Removing `tests/conftest.py` initially caused eight fixture-package import failures during pytest -collection. It was restored as a one-line, side-effect-free repository test root; all former -Qt/PyVista environment mutation was removed. This is evidence that the file had a non-GUI pytest -role and was not a valid deletion candidate. - -The source and runtime scans prove the dependency/path boundary. They do not prove live SwiftUI -control behavior: no automated native UI target existed at this phase, and no running-window -click journey was used to classify handlers. diff --git a/docs/LAMBADA_MAJOR_VESSELS.md b/docs/LAMBADA_MAJOR_VESSELS.md deleted file mode 100644 index 9257a62..0000000 --- a/docs/LAMBADA_MAJOR_VESSELS.md +++ /dev/null @@ -1,178 +0,0 @@ -# LAMBADA P60_606 archived derivative and rejected qualification - -The repository retains a compact evidence derivative of the atlas-registered P60_606 graph -from Renier, de Launoit, and Skriabine's *Vascular graphs of the developing post-natal mouse -brain*. The source is [Zenodo record 18876865](https://zenodo.org/records/18876865), DOI -`10.5281/zenodo.18876865`, licensed under -[CC BY 4.0](https://creativecommons.org/licenses/by/4.0/). The associated Cell paper is DOI -[`10.1016/j.cell.2026.03.013`](https://doi.org/10.1016/j.cell.2026.03.013). - -This derivative is not an application layer, a medical device, a subject image, or a live -vascular measurement. It cannot establish subject-specific clearance or trajectory suitability. - -## Runtime status - -Coordinate qualification is rejected. The current runtime does not load, serve, display, mirror, -or analyze the P60_606 derivative. The production `auditedReferenceMajorVessels` capability and -reference metadata/geometry endpoints now belong exclusively to the separately qualified VesSAP -BL6J-no1 display reference. `radiusAwareReferenceVesselAnalysis` remains absent and clearance -analysis fails with `VESSEL_ANALYSIS_UNAVAILABLE`. - -No geometry from this derivative may appear in a slice, Dorsal, or 3D view. It cannot support a -probe/vessel intersection, margin, distance, conflict, no-conflict, or surgical-clearance claim. - -## Coordinate and coverage qualification - -The reproducible qualification reran the exact extraction against the exact pinned source graph -and tested coordinate orientation against the exact BrainGlobe `allen_mouse_25um` v1.2 -annotation. AP and DV orientation evidence passed. Qualification still failed for two independent -reasons: - -1. The primary source describes the specimens as mouse brain hemispheres, so the specimen does - not establish whole-brain coverage. -2. The exact P60_606 graph contains no persisted graph, vertex, or edge property that binds the - numerical ML axis to biological hemisphere/laterality. - -Points occur numerically on both sides of the atlas-array midpoint, but that is not evidence that -the source represents both biological hemispheres. No trustworthy source documents a permitted -mirror transform for this exact specimen. The application therefore does not choose an ML sign, -infer the sampled side, or mirror the derivative. - -The canonical report is -[`docs/evidence/lambada_p60_606_coordinate_qualification_rejected_v1.json`](evidence/lambada_p60_606_coordinate_qualification_rejected_v1.json), -SHA-256 `0993d5a0ad6c0d62094dc395fe2bc4f284870e6e7c0b602be7df5a7da867c93a`. -Its blocking reason codes are `SOURCE_HEMISPHERE_PROPERTY_MISSING` and -`SOURCE_SPECIMEN_COVERAGE_IS_HEMISPHERE`; `qualifiedMapping` is null. - -## Sources rejected for subject-clearance integration - -No audited source met all five requirements for subject-clearance integration: downloadable -major-vessel centerlines/radii, an -explicit Allen 25 µm transform, biological laterality, a documented CCF-to-bregma relationship, -and terms that permit the required redistributed derivative. - -| Candidate | Why it is not integrated | -| --- | --- | -| [Mendeley `mjtyry6v85.1`](https://data.mendeley.com/datasets/mjtyry6v85/1) | The graph coordinates are raw specimen-space XYZ without qualified orientation, laterality, Allen registration, or bregma relationship. | -| [Mendeley `stxvn5sv44.1`](https://data.mendeley.com/datasets/stxvn5sv44/1) | This is an averaged four-mouse 20 µm population vessel-length-density NIfTI, not major-vessel centerlines/radii; population/capillary-density display is outside the planner's accepted scope. | -| [Hinz et al. 2021](https://www.biorxiv.org/content/10.1101/2021.10.21.465264v1) | Geometry is in a study-specific 78 µm MRI template with no Allen/bregma transform, and CC BY-NC-ND terms do not permit the required transformed redistribution. | -| [Xiong et al. 2017](https://www.frontiersin.org/journals/neuroanatomy/articles/10.3389/fnana.2017.00128/full) | Named vessels are documented, but no downloadable graph or CCF transform is provided. | - -These are rejected alternatives, not integrated data. VesSAP is separately accepted only for a -display reference; it does not meet the subject/bregma/error-bound requirements for clearance. A -future clearance source must be audited against the same requirements before any analysis -capability can be advertised. - -## Archived derivative - -The 12.28 GB extracted graph and its 5.05 GB archive are not committed. The bundled NPZ is -814,393 bytes with SHA-256 -`fb2344e845e604be3424bd63f4222d273eafba34db0df2eaff32f4400fa9afec`. -Its adjacent manifest records source identity, attribution, schema, extraction version, array -shapes, output statistics, and the mandatory limitations checked by the runtime loader. - -The derivative contains five arrays: - -| Array | Meaning | -| --- | --- | -| `points_asr_voxel_f32 [71313,3]` | Continuous Allen 25 µm coordinates in BrainGlobe `[AP,DV,ML]` order | -| `radii_um_f32 [71313]` | Approximate source radius in micrometres | -| `source_annotation_ids_i32 [71313]` | Coarse source ancestor IDs; not a replacement for current atlas lookup | -| `run_offsets_i64 [11819]` | End-exclusive boundaries for 11,818 independent polylines | -| `source_edge_indices_i32 [11818]` | Serialized source edge index for each polyline | - -The archived loader can verify the manifest and NPZ identities, inventory, dtypes, shapes, -bounds, radius threshold, run lengths, edge ordering, and total path length. Those checks establish -deterministic bytes and extraction behavior; they do not qualify biological laterality or -whole-brain coverage and do not authorize runtime geometry access. - -## Extraction rule - -Algorithm `lambada-p60-606-major-runs-v1` applies the following rule independently to every -source edge: - -1. Accept a point only when all three coordinates and its radius are finite. -2. Require ClearMap bounds `0 <= [c0,c1,c2] < [320,528,456]`. -3. Require physical radius at least 15 µm, equivalent to source `radii_atlas >= 0.6` at 25 µm. -4. Form maximal consecutive accepted blocks. Keep only blocks with at least two points. -5. Split and drop at rejected points. Never clip or interpolate coordinates. -6. Permute ClearMap `[c0,c1,c2]` to BrainGlobe ASR `[c1,c0,c2]`. -7. Convert continuous voxel coordinates using `physical_um = voxel * 25`, with no half-voxel - shift. - -The source edge maximum is used only as a conservative prefilter. Selection within an edge is -always pointwise. Keeping an entire edge whenever any point passes would retain 832,850.19 µm -of path, including 632,459.33 µm whose points fail the radius criterion. Selecting edges by -their mean radius would miss 5,597 edges that contain qualifying segments. - -The reproduced derivative output is: - -- 16,156 source edges pass the conservative edge-maximum prefilter. -- 78,048 source geometry points are finite, in bounds, and at least 15 µm in radius. -- 71,313 points form drawable runs; isolated qualifying points are omitted. -- 59,495 segments form 11,818 runs from 10,907 serialized source edges. -- Source-float64 retained path length is 200,390.862919 µm. - -The source contains 292,368 atlas-coordinate points outside the reviewed array bounds, all past -the second-axis upper bound. They are not clamped back into the brain. - -## Reproduction - -The Zenodo-distributed archive identity is: - -- filename: `P60_606_graph_2024-12-03.gt.7z` -- bytes: `5,050,194,723` -- SHA-256: `cc6d252ee57154f5bc0f06605a703253470a57c210d76e075831effa2098d66f` -- Zenodo MD5: `218ed346c6d7dc501301204f811be37f` - -After extraction, the accepted graph identity is: - -- filename: `606_graph_2024-12-03.gt` -- bytes: `12,282,574,483` -- SHA-256: `c2568cfbecd3f3eb720519be9d042f0cb41606741b8dd2f018bad1c54d36ef85` -- MD5: `b0bedc97ed2c6e00a41565649b3dd86b` - -From the repository root, with that extracted file available locally: - -```bash -uv run python scripts/extract_lambada_major_vessels.py \ - /absolute/path/to/606_graph_2024-12-03.gt -``` - -The script hashes the full source before reading any pinned binary offsets. It rejects any other -file, validates contiguous source edge ranges, asserts every reviewed extraction count, writes -NPZ members in a fixed order with fixed ZIP metadata, rebuilds the asset in a temporary -directory, and requires both builds to have identical bytes before writing the manifest. - -To reproduce the separate qualification report, provide the exact extracted graph and exact -BrainGlobe atlas package directory: - -```bash -uv run python scripts/qualify_lambada_coordinates.py \ - /absolute/path/to/606_graph_2024-12-03.gt \ - /absolute/path/to/allen_mouse_25um_v1.2 \ - --output /absolute/path/to/qualification.json -``` - -A rejected run exits nonzero by design. Compare its canonical bytes and SHA-256 with the checked-in -report; do not promote a locally edited or differently sourced report. - -## Mandatory interpretation limits - -- Evidence use only; this derivative is not displayed or analyzed and is not validated for - surgery. -- The source is an atlas-registered fixed and cleared P60 mouse-brain reference, not live or - subject-specific vasculature. -- Pial and choroidal vessels were removed by the source workflow. This derivative also - suppresses points below a 15 µm radius, so missing vessels are expected. -- Registration error, tissue distortion, biological variation, and omitted vessels are not - bounded by this graph. -- The primary record describes hemisphere specimens, while the exact graph has no persisted - hemisphere/laterality binding. Whole-brain coverage and biological ML polarity are therefore - unqualified, and mirroring is prohibited. -- No artery-versus-vein identity is available. -- The source workflow corrected endpoints, linearly reconnected nearby endpoints, and removed - short terminal offshoots. Some paths are reconstructed rather than directly observed. -- Radii use the source's mean atlas-resampling scale, not a local Jacobian correction, and fixed - cleared tissue may be distorted. -- Out-of-bounds points are omitted without clipping or interpolation. diff --git a/docs/LEGACY_REMOVAL.md b/docs/LEGACY_REMOVAL.md deleted file mode 100644 index 77db450..0000000 --- a/docs/LEGACY_REMOVAL.md +++ /dev/null @@ -1,49 +0,0 @@ -# Legacy Qt/VTK Removal - -## Scope - -The baseline contains two desktop products. The supported SwiftUI application launches the -Python bridge, while the default Python console command launches an older PySide6/PyVista/VTK -application. Phase 1 removes the latter from the default product and dependency graph. - -## Legacy-only manifest - -- `src/mouse_brain_planner/app.py` -- `src/mouse_brain_planner/gui/**` -- `src/mouse_brain_planner/rendering/scene_controller.py` -- Qt/VTK-specific GUI and scene-controller tests -- `--smoke-test` and `--no-download` CLI flags that configure only the Qt application -- PySide6, PyVista, PyVistaQt, and VTK default dependencies - -`rendering/sagittal_cache.py` is a separate test-only 10 µm optimization with no supported -consumer. Because the MVP intentionally allows only the 25 µm atlas, it is removed from the -default package rather than coupled to the linked viewer. - -## Reusable code that is not legacy - -The following tested Python modules are retained and wired into the new product instead of being -deleted with the old GUI: - -- named coordinate frames and anatomical transforms; -- stereotaxic calibration models/math; -- probe definitions, placements, and recording-site geometry; -- exact finite-segment measurement kernels; -- atlas slicing/point lookup and project persistence; -- population-density and subject-image provenance/registration. - -## Removal sequence - -1. Change the CLI so no subcommand prints help and an explicit `bridge` command starts the service. -2. Add tests proving CLI parsing/import does not import Qt/VTK. -3. Remove legacy modules and their UI-only tests from the package. -4. Remove Qt/VTK from base dependencies and regenerate the lockfile. -5. Install/test the minimal locked environment and build the native app. -6. Keep the deletion commit in Git history; do not copy the old stack into a second maintained - `experimental` package. - -## Rollback - -The pre-removal implementation remains retrievable from Git commit -`51fe26d637b700d39944bbee86e3ea42ded4e7a2`. A historical bug investigation may inspect that -commit without restoring Qt/VTK to the supported package. Any proposal to revive it must explain -how duplicated state, dependencies, accessibility, and two-GUI maintenance will be avoided. diff --git a/docs/SURGERY_PLAN_EXPORT.md b/docs/SURGERY_PLAN_EXPORT.md index 29c533e..a11825f 100644 --- a/docs/SURGERY_PLAN_EXPORT.md +++ b/docs/SURGERY_PLAN_EXPORT.md @@ -79,8 +79,7 @@ converting schema-8 target/calibration records. The packet is marked `FINAL` only when the project has a subject, records the animal-research acknowledgement, is saved and clean, and has a current matching v4 probe plan. A separate target projection, active subject calibration, and geometry checkbox are not v4 prerequisites. Otherwise -the packet is visibly marked `DRAFT`. `FINAL` describes software-state agreement only; it is not -a biological or procedural validation claim. +the packet is visibly marked `DRAFT`. `FINAL` describes software-state agreement for the exported packet. ## Selectable planning views diff --git a/docs/VESSAP_MAJOR_VESSELS.md b/docs/VESSAP_MAJOR_VESSELS.md index deb0b9a..0f6aa05 100644 --- a/docs/VESSAP_MAJOR_VESSELS.md +++ b/docs/VESSAP_MAJOR_VESSELS.md @@ -1,51 +1,47 @@ -# VesSAP BL6J-no1 major-vessel reference +# VesSAP External Major-vessel Data -## Decision +Brain3D can display an optional derivative of the VesSAP `BL6J-no1` whole-brain skeleton and +radius volumes described by Todorov et al., +[Nature Methods 2020](https://doi.org/10.1038/s41592-020-0792-1). -**Display-only GO:** Brain3D may display the pinned, bilateral BL6J-no1 derivative as an -ex-vivo C57BL/6J major-vessel reference. +## Repository boundary -**Clearance NO-GO:** the layer must not classify clearance, vessel absence, trajectory -suitability, navigation accuracy, or safety for an individual animal. It is one fixed and -cleared specimen, has no published landmark target-registration-error bound, and is not -registered to the operative animal's skull or bregma. +The NPZ, manifest, source volumes, transform archive, and license text are not stored in this +repository or bundled with the application. The external files retain the VesSAP +[CC BY-NC 4.0](https://creativecommons.org/licenses/by-nc/4.0/) terms. -The runtime therefore exposes geometry and provenance but fails closed for clearance analysis. +Install the prepared files at: -## Pinned sources +```text +/vasculature/vessap_bl6j1_major_vessels_50um_v1.npz +/vasculature/vessap_bl6j1_major_vessels_50um_v1.npz.manifest.json +``` -The source is the VesSAP public repository release dated 2021-10-01 associated with Todorov et -al., *Machine learning analysis of whole mouse brain vasculature*, -[DOI 10.1038/s41592-020-0792-1](https://doi.org/10.1038/s41592-020-0792-1). -The public data-access record is [DISCO Technologies VesSAP](https://www.discotechnologies.org/VesSAP/). +Set `MOUSE_BRAIN_PLANNER_DATA_DIR` to override ``. -The derivative manifest pins the aggregate source-bundle identity -`d0216b9f6fcec428845f8ecc24860b4838f27b6b69496a88d8b7ab0dcd1dbf8a`. +## Data contract -| File | Role | Bytes | SHA-256 | -|---|---|---:|---| -| `BL6J-no1_iso3um_stitched_skeleton.nii.gz` | True source centerlines | 114,305,865 | `1ea1a489dfacfa509f60d984ee955e43107a579d181bb5e6f3df51da6b68beac` | -| `BL6J-no1_iso3um_stitched_radius.nii.gz` | Euclidean-distance radius values on the centerlines | 136,114,699 | `6a8728d4518957c0e84687b077a76e64362edc9644c6e2adce60c2c244d4b887` | -| `elastix_atlas_registration_parameters.7z` | Published BL6J-no1 Euler plus B-spline transform | 38,472 | `ab2509e07dcab65f176b72337624fa2ccc38f9e32f17530964ed17af5e1e88ad` | -| `BL6J-no1_iso3um_stitched_atlas_registration_result.nii.gz` | Signed-label validation only; not bundled | 320,422,938 | `c3b9904ba95f8566b7161fd0f197cfe064ee0c5cfb24d49e553bc9c8793677c1` | -| `C57BL_6_no1.zip` | VesselGraph node/edge and grouped-label validation only; not bundled | 353,177,007 | `41d4d9d44e4b04190d7f1039615fb08e58aacd707d6d0baffb6c4090162d236f` | -| `C57BL_6_no1_raw.zip` | Raw VesselGraph endpoint-feature validation only; not bundled | 348,965,728 | `2bfd91617100085fd869b5e20970c46c6a1120b2474f78b34b634231e98ccae9` | +The loader expects: -The independent 10 µm label check used BrainGlobe `allen_mouse_10um` version 1.2, -shape `[1320, 800, 1140]`, orientation `asr`, annotation SHA-256 -`9f04278b0e539c88029b7caafa1b9fffed1cd2bf8a4e587903a823e182ce2ab4`. -The shipped display geometry is bounded against `allen_mouse_25um` version 1.2, -shape `[528, 320, 456]`. +| Property | Value | +| --- | --- | +| Asset SHA-256 | `9300dacf25ca57a5d23377ca0dc885e34ff0d18e8d21ef7590c6dcd156cf5db7` | +| Asset bytes | 1,853,131 | +| Atlas | `allen_mouse_25um` version `1.2` | +| Output points | 196,377 | +| Output runs | 76,622 | +| Output segments | 119,755 | +| Minimum source radius | 15 µm | +| Display grid | 50 µm | -## Exact transform interpretation +Before returning geometry, Brain3D checks the manifest schema, source identity, transform identity, +asset digest and size, array inventory, data types, shapes, bounds, run offsets, radius threshold, +and path length. -For a source skeleton voxel `g = [x, y, z]`, form the Elastix fixed physical point -`p = 0.1 * g`. Apply the published Euler transform followed by the B-spline transform and -call the Transformix physical result `T = [T_x, T_y, T_z]`. +## Coordinate mapping -The moving atlas order is `[ML left-to-right, AP, DV]`. BrainGlobe ASR order is -`[AP anterior-to-posterior, DV superior-to-inferior, ML right-to-left]`, so ML must be -reflected: +The prepared data uses BrainGlobe physical `[AP,DV,ML]` micrometres. The VesSAP transform output +is mapped as: ```text AP_um = 30 * T_y @@ -53,107 +49,10 @@ DV_um = 30 * T_z ML_um = 11390 - 30 * T_x ``` -Equivalent continuous atlas indices are: +The ML reflection converts the source left-to-right direction to BrainGlobe ASR right-to-left. -```text -10 um: [AP, DV, ML] = [3*T_y, 3*T_z, 1139 - 3*T_x] -25 um: [AP, DV, ML] = [1.2*T_y, 1.2*T_z, 455.6 - 1.2*T_x] -``` +## Runtime behavior -Use the atlas voxel-center conversion when discretizing. A no-flip ML interpretation swaps -biological left and right even though the symmetric Allen annotation cannot reveal that error -from unsigned region labels alone. - -## Path-preserving extraction - -The graph CSV files contain endpoints, path length, curveness, and radius summaries, but no -intermediate centerline coordinates. They cannot preserve vessel paths. Among the 371,759 graph -edges with average radius at least five source voxels, 30.72% have curveness above 1.10 and the -99th-percentile chord-versus-path length deficit is 38.5 µm; drawing endpoint chords is rejected. - -The bundled derivative instead follows this pinned procedure: - -1. Read the same-grid uint8 skeleton and radius volumes, each - `2804 x 3661 x 2012` at 3 µm sampling. -2. Retain skeleton voxels with radius at least five voxels: nominal radius at least 15 µm and - diameter at least 30 µm. Capillaries and smaller vessels are intentionally excluded. -3. Recover 26-neighbour adjacency in source space before any transform. -4. Transform retained source points with the published BL6J-no1 transform and the reviewed ML - reflection above. -5. Coalesce on a 50 µm display grid. Retain only edges induced by source adjacency; never join - points merely because their target bins are adjacent. -6. Store the mean continuous transformed coordinate per occupied display bin and the maximum - retained source EDT radius multiplied by 3 µm. - -The source skeleton and radius supports match exactly: 58,313,813 nonzero centerline voxels and -zero support mismatches across 20,654,073,328 source voxels. The radius threshold retains -1,262,706 source points. - -## Bundled derivative identity - -| Property | Pinned value | -|---|---| -| File | `vessap_bl6j1_major_vessels_50um_v1.npz` | -| Asset ID | `vessap-bl6j-no1-major-vessels-50um-v1` | -| Extraction algorithm | `vessap-bl6j1-major-skeleton-50um-v1` | -| Bytes | 1,853,131 | -| SHA-256 | `9300dacf25ca57a5d23377ca0dc885e34ff0d18e8d21ef7590c6dcd156cf5db7` | -| In-bounds threshold points | 1,258,140 | -| Occupied 50 µm display voxels | 198,262 | -| Omitted isolated display voxels | 63,320 | -| Output points | 196,377 | -| Output runs | 76,622 | -| Output segments | 119,755 | -| Output path length | 3,817,312.085 µm | - -The adjacent JSON manifest, the NPZ, and -`src/mouse_brain_planner/assets/vasculature/VESSAP_DATA_LICENSE.txt` are one evidence package. -The loader checks filename, byte count, SHA-256, member inventory, array types/shapes, transform -identity, extraction statistics, bounds, radius threshold, and mandatory limitations before -returning geometry. - -## Independent validation - -- Twenty thousand deterministic, evenly spaced nodes were sampled from 3,820,133 graph nodes. -- Correct `[T_y, T_z, T_x] -> [AP, DV, ML]` AP/DV orientation achieved 98.5025% non-background - grouped-region agreement before subvoxel adjustment; the strongest incorrect AP/DV - orientation achieved 20.8207%. -- With the required ML reflection, 10 µm agreement was 18,664/18,765 = 99.4618% for - non-background nodes and 17,505/17,585 = 99.5451% after also excluding root and fiber tracts. -- Corrected 25 µm non-background agreement was 97.8524%; 99.705% of sampled transformed points - were within the 25 µm atlas bounds. -- Direct comparison of the official signed registered-atlas NIfTI with the independently loaded - `atlas_processed` labels agreed at 18,761/18,765 = 99.9787% non-background sampled nodes. -- The official signed registered annotation supplied 19,429 nonzero labels: 8,985 left and - 10,444 right. The canonical midline classified 19,425/19,429 = 99.9794%; all four discrepancies - were within 5.38 µm of the midline. The best empirical separator classified - 19,428/19,429 = 99.9949%. This proves the required ML reversal. -- At 1,000 sampled nodes, finite-difference transform Jacobian determinants were - 1.269–2.451 with no sampled fold; singular values were 0.672–1.560. The large anisotropy is - another reason a scalar source radius cannot become a surgical clearance bound. -- On 2,000 graph nodes, 93.85% were within `sqrt(3)` source voxels of the official skeleton and - 98.2% within three voxels, supporting common spatial identity while confirming that graph - endpoints are not substitutes for the voxel paths. - -These checks qualify transform interpretation, laterality, source identity, and deterministic -derivation. They do **not** independently bound biological registration error. - -## Mandatory limitations - -- Animal research use only; not a medical device. -- One fixed, cleared adult C57BL/6J brain, not the current animal or live vasculature. -- No numeric registration-error, tissue-clearing-distortion, bregma, or skull-registration bound. -- No artery-versus-vein identity; pial and choroidal coverage is not separately classified. -- The 50 µm display reduction can merge close paths and omits isolated target voxels without a - retained segment. -- A displayed path does not establish its location in the current animal; a missing path does - not establish vessel absence. -- Clearance, conflict, safe-entry, and trajectory-suitability conclusions remain disabled. - -## Separate data license - -The source data and this adapted derivative are licensed -[CC BY-NC 4.0](https://creativecommons.org/licenses/by-nc/4.0/). They are not relicensed by -Brain3D's source-code license. Redistribution must preserve the VesSAP attribution, paper DOI, -release identity, exact source and derivative digests, this limitations record, and the bundled -license text. Commercial use requires separate authorization from the rights holders. +When both external files are available, the bridge exposes metadata and geometry for the five +views. If either file is missing or fails validation, the capability is not advertised. Vessel +clearance analysis remains unavailable. diff --git a/docs/evidence/lambada_p60_606_coordinate_qualification_rejected_v1.json b/docs/evidence/lambada_p60_606_coordinate_qualification_rejected_v1.json deleted file mode 100644 index dc46559..0000000 --- a/docs/evidence/lambada_p60_606_coordinate_qualification_rejected_v1.json +++ /dev/null @@ -1 +0,0 @@ -{"algorithmVersion":"lambada-p60-606-coordinate-signs-v1","annotationOrientation":{"method":"For each of all eight AP/DV/ML sign combinations, truncate retained continuous voxels exactly as ClearMap label_points does, sample the pinned Allen annotation, and count the source annotation when it is an ancestor of the sampled Allen structure. AP/DV are judged separately from bilateral ML symmetry.","scores":[{"ancestorAgreement":{"decimal":"0.866910610","denominator":52198,"numerator":45251},"evaluableFraction":{"decimal":"0.731956305","denominator":71313,"numerator":52198},"evaluablePointCount":52198,"exactLeafAgreement":{"decimal":"0.182439940","denominator":52198,"numerator":9523},"flipAP":false,"flipDV":false,"flipML":false,"label":"AP_KEEP_DV_KEEP_ML_KEEP","selectedPointCount":71313},{"ancestorAgreement":{"decimal":"0.846928089","denominator":51564,"numerator":43671},"evaluableFraction":{"decimal":"0.723065921","denominator":71313,"numerator":51564},"evaluablePointCount":51564,"exactLeafAgreement":{"decimal":"0.155088822","denominator":51564,"numerator":7997},"flipAP":false,"flipDV":false,"flipML":true,"label":"AP_KEEP_DV_KEEP_ML_FLIP","selectedPointCount":71313},{"ancestorAgreement":{"decimal":"0.247666836","denominator":41253,"numerator":10217},"evaluableFraction":{"decimal":"0.578477977","denominator":71313,"numerator":41253},"evaluablePointCount":41253,"exactLeafAgreement":{"decimal":"0.027028337","denominator":41253,"numerator":1115},"flipAP":false,"flipDV":true,"flipML":false,"label":"AP_KEEP_DV_FLIP_ML_KEEP","selectedPointCount":71313},{"ancestorAgreement":{"decimal":"0.245175056","denominator":41244,"numerator":10112},"evaluableFraction":{"decimal":"0.578351773","denominator":71313,"numerator":41244},"evaluablePointCount":41244,"exactLeafAgreement":{"decimal":"0.020851518","denominator":41244,"numerator":860},"flipAP":false,"flipDV":true,"flipML":true,"label":"AP_KEEP_DV_FLIP_ML_FLIP","selectedPointCount":71313},{"ancestorAgreement":{"decimal":"0.179921852","denominator":33270,"numerator":5986},"evaluableFraction":{"decimal":"0.466534853","denominator":71313,"numerator":33270},"evaluablePointCount":33270,"exactLeafAgreement":{"decimal":"0.006822964","denominator":33270,"numerator":227},"flipAP":true,"flipDV":false,"flipML":false,"label":"AP_FLIP_DV_KEEP_ML_KEEP","selectedPointCount":71313},{"ancestorAgreement":{"decimal":"0.177183301","denominator":33344,"numerator":5908},"evaluableFraction":{"decimal":"0.467572532","denominator":71313,"numerator":33344},"evaluablePointCount":33344,"exactLeafAgreement":{"decimal":"0.006208013","denominator":33344,"numerator":207},"flipAP":true,"flipDV":false,"flipML":true,"label":"AP_FLIP_DV_KEEP_ML_FLIP","selectedPointCount":71313},{"ancestorAgreement":{"decimal":"0.152792247","denominator":31677,"numerator":4840},"evaluableFraction":{"decimal":"0.444196710","denominator":71313,"numerator":31677},"evaluablePointCount":31677,"exactLeafAgreement":{"decimal":"0.003882943","denominator":31677,"numerator":123},"flipAP":true,"flipDV":true,"flipML":false,"label":"AP_FLIP_DV_FLIP_ML_KEEP","selectedPointCount":71313},{"ancestorAgreement":{"decimal":"0.153758289","denominator":31517,"numerator":4846},"evaluableFraction":{"decimal":"0.441953080","denominator":71313,"numerator":31517},"evaluablePointCount":31517,"exactLeafAgreement":{"decimal":"0.003521909","denominator":31517,"numerator":111},"flipAP":true,"flipDV":true,"flipML":true,"label":"AP_FLIP_DV_FLIP_ML_FLIP","selectedPointCount":71313}]},"atlas":{"annotation":{"filename":"annotation.tiff","sha256":"52775c6086ae6aa7d9df2ad2c9e5ab9ce36effa09f5355c7c45efeada6647e06","sizeBytes":308270826},"atlasKey":"allen_mouse_25um","atlasPackageVersion":"1.2","metadata":{"filename":"metadata.json","sha256":"119132150055826484fa75a4dc17c476d02251c0db8b4d0b7b28f6b8c4b8ae60","sizeBytes":423},"orientation":"asr","resolutionUm":[25,25,25],"shapeVoxels":[528,320,456],"structures":{"filename":"structures.json","sha256":"7157e6130b354f6bebc21aa79f882274cbbe253ac916d97123805af3b4ff8abd","sizeBytes":138106}},"checks":[{"id":"annotation-evaluable-point-count","observed":"52198","passed":true,"required":">=50000"},{"id":"annotation-evaluable-fraction","observed":"52198/71313","passed":true,"required":">=7/10"},{"id":"annotation-ancestor-agreement","observed":"45251/52198","passed":true,"required":">=4/5"},{"id":"expected-orientation-is-global-winner","observed":"AP_KEEP_DV_KEEP_ML_KEEP","passed":true,"required":"AP_KEEP_DV_KEEP_ML_KEEP"},{"id":"expected-orientation-global-lead","observed":"8963951/448589612","passed":true,"required":">=1/100"},{"id":"ap-dv-sign-margin","observed":"1333432537/2153324094","passed":true,"required":">=1/5"},{"id":"source-hemisphere-property-present","observed":"0 matching properties","passed":false,"required":"1 graph:edge_geometry_hemisphere typeIndex=14"},{"id":"source-coverage-whole-brain","observed":"official source declares brain-hemisphere specimens","passed":false,"required":"whole-brain source or explicitly reviewed bilateral reconstruction"}],"clearMapImplementationEvidence":{"annotationAndHemispherePropertyCreation":"https://github.com/ClearAnatomics/ClearMap/blob/71444a5c7456901f15e8d0ceb06fab72b74161df/ClearMap/processors/tube_map.py#L660-L682","commit":"71444a5c7456901f15e8d0ceb06fab72b74161df","evidenceScope":"Context only. Generic repository code does not bind a workflow commit, configuration, or transform to the exact P60_606 artifact.","exactArtifactSpecimenConfiguration":null,"exactArtifactWorkflowCommit":null,"graphLevelEdgeGeometryStorage":"https://github.com/ClearAnatomics/ClearMap/blob/71444a5c7456901f15e8d0ceb06fab72b74161df/ClearMap/Analysis/Graphs/GraphGt.py#L869-L910","hemisphereLabelSemantics":"https://github.com/ClearAnatomics/ClearMap/blob/71444a5c7456901f15e8d0ceb06fab72b74161df/ClearMap/Analysis/Statistics/data_frame_operations.py#L53-L59","repository":"https://github.com/ClearAnatomics/ClearMap"},"decision":{"apDvWinnerMargin":"1333432537/2153324094","bestAPDVAlternative":"AP_KEEP_DV_FLIP_ML_KEEP","blockingReasons":["SOURCE_HEMISPHERE_PROPERTY_MISSING","SOURCE_SPECIMEN_COVERAGE_IS_HEMISPHERE"],"expectedGlobalLead":"8963951/448589612","globalRunnerUp":"AP_KEEP_DV_KEEP_ML_FLIP","globalWinner":"AP_KEEP_DV_KEEP_ML_KEEP","hemisphereWinnerMargin":null,"qualifiedMapping":null},"graphTool":{"byteOrder":"little","directed":false,"edgeCount":3301619,"formatDocumentation":"https://graph-tool.skewed.de/static/docs/stable/gt_format.html","hemisphereProperty":null,"propertyCount":31,"propertyInventory":[{"itemCount":1,"kind":"graph","name":"shape","objectPayloadOffset":31054881,"objectPayloadSizeBytes":23,"recordOffset":31054858,"typeIndex":14,"valuesEndOffset":31054904,"valuesOffset":31054873},{"itemCount":1,"kind":"graph","name":"edge_geometry_type","objectPayloadOffset":31054940,"objectPayloadSizeBytes":20,"recordOffset":31054904,"typeIndex":14,"valuesEndOffset":31054960,"valuesOffset":31054932},{"itemCount":1,"kind":"graph","name":"edge_geometry_coordinates","objectPayloadOffset":31055003,"objectPayloadSizeBytes":1847541951,"recordOffset":31054960,"typeIndex":14,"valuesEndOffset":1878596954,"valuesOffset":31054995},{"itemCount":1,"kind":"graph","name":"edge_geometry_radii","objectPayloadOffset":1878596991,"objectPayloadSizeBytes":615847421,"recordOffset":1878596954,"typeIndex":14,"valuesEndOffset":2494444412,"valuesOffset":1878596983},{"itemCount":1,"kind":"graph","name":"edge_geometry_coordinates_atlas","objectPayloadOffset":2494444461,"objectPayloadSizeBytes":1847541951,"recordOffset":2494444412,"typeIndex":14,"valuesEndOffset":4341986412,"valuesOffset":2494444453},{"itemCount":1,"kind":"graph","name":"edge_geometry_radii_atlas","objectPayloadOffset":4341986455,"objectPayloadSizeBytes":615847421,"recordOffset":4341986412,"typeIndex":14,"valuesEndOffset":4957833876,"valuesOffset":4341986447},{"itemCount":1,"kind":"graph","name":"edge_geometry_annotation","objectPayloadOffset":4957833918,"objectPayloadSizeBytes":615847421,"recordOffset":4957833876,"typeIndex":14,"valuesEndOffset":5573681339,"valuesOffset":4957833910},{"itemCount":1,"kind":"graph","name":"edge_geometry_distance_to_surface","objectPayloadOffset":5573681390,"objectPayloadSizeBytes":615847421,"recordOffset":5573681339,"typeIndex":14,"valuesEndOffset":6189528811,"valuesOffset":5573681382},{"itemCount":1,"kind":"graph","name":"edge_geometry_coordinates_mri","objectPayloadOffset":6189528858,"objectPayloadSizeBytes":1847541951,"recordOffset":6189528811,"typeIndex":14,"valuesEndOffset":8037070809,"valuesOffset":6189528850},{"itemCount":1,"kind":"graph","name":"length_coordinates","objectPayloadOffset":8037070845,"objectPayloadSizeBytes":615847421,"recordOffset":8037070809,"typeIndex":14,"valuesEndOffset":8652918266,"valuesOffset":8037070837},{"itemCount":1,"kind":"graph","name":"length_coordinates_mri","objectPayloadOffset":8652918306,"objectPayloadSizeBytes":615847421,"recordOffset":8652918266,"typeIndex":14,"valuesEndOffset":9268765727,"valuesOffset":8652918298},{"itemCount":1,"kind":"graph","name":"length_coordinates_atlas","objectPayloadOffset":9268765769,"objectPayloadSizeBytes":615847421,"recordOffset":9268765727,"typeIndex":14,"valuesEndOffset":9884613190,"valuesOffset":9268765761},{"itemCount":1,"kind":"graph","name":"edge_geometry_length_coordinates","objectPayloadOffset":9884613240,"objectPayloadSizeBytes":615847421,"recordOffset":9884613190,"typeIndex":14,"valuesEndOffset":10500460661,"valuesOffset":9884613232},{"itemCount":1,"kind":"graph","name":"edge_geometry_length_coordinates_mri","objectPayloadOffset":10500460715,"objectPayloadSizeBytes":615847421,"recordOffset":10500460661,"typeIndex":14,"valuesEndOffset":11116308136,"valuesOffset":10500460707},{"itemCount":1,"kind":"graph","name":"edge_geometry_length_coordinates_atlas","objectPayloadOffset":11116308192,"objectPayloadSizeBytes":615847421,"recordOffset":11116308136,"typeIndex":14,"valuesEndOffset":11732155613,"valuesOffset":11116308184},{"itemCount":2231019,"kind":"vertex","name":"coordinates","objectPayloadOffset":null,"objectPayloadSizeBytes":null,"recordOffset":11732155613,"typeIndex":11,"valuesEndOffset":11803548242,"valuesOffset":11732155634},{"itemCount":2231019,"kind":"vertex","name":"radii","objectPayloadOffset":null,"objectPayloadSizeBytes":null,"recordOffset":11803548242,"typeIndex":4,"valuesEndOffset":11821396409,"valuesOffset":11803548257},{"itemCount":2231019,"kind":"vertex","name":"coordinates_atlas","objectPayloadOffset":null,"objectPayloadSizeBytes":null,"recordOffset":11821396409,"typeIndex":11,"valuesEndOffset":11892789044,"valuesOffset":11821396436},{"itemCount":2231019,"kind":"vertex","name":"radii_atlas","objectPayloadOffset":null,"objectPayloadSizeBytes":null,"recordOffset":11892789044,"typeIndex":4,"valuesEndOffset":11910637217,"valuesOffset":11892789065},{"itemCount":2231019,"kind":"vertex","name":"annotation","objectPayloadOffset":null,"objectPayloadSizeBytes":null,"recordOffset":11910637217,"typeIndex":3,"valuesEndOffset":11928485389,"valuesOffset":11910637237},{"itemCount":2231019,"kind":"vertex","name":"distance_to_surface","objectPayloadOffset":null,"objectPayloadSizeBytes":null,"recordOffset":11928485389,"typeIndex":4,"valuesEndOffset":11946333570,"valuesOffset":11928485418},{"itemCount":2231019,"kind":"vertex","name":"coordinates_mri","objectPayloadOffset":null,"objectPayloadSizeBytes":null,"recordOffset":11946333570,"typeIndex":11,"valuesEndOffset":12017726203,"valuesOffset":11946333595},{"itemCount":2231019,"kind":"vertex","name":"former_degree","objectPayloadOffset":null,"objectPayloadSizeBytes":null,"recordOffset":12017726203,"typeIndex":3,"valuesEndOffset":12035574378,"valuesOffset":12017726226},{"itemCount":2231019,"kind":"vertex","name":"tip_cell_prediction","objectPayloadOffset":null,"objectPayloadSizeBytes":null,"recordOffset":12035574378,"typeIndex":3,"valuesEndOffset":12053422559,"valuesOffset":12035574407},{"itemCount":2231019,"kind":"vertex","name":"tip_cell_score","objectPayloadOffset":null,"objectPayloadSizeBytes":null,"recordOffset":12053422559,"typeIndex":4,"valuesEndOffset":12071270735,"valuesOffset":12053422583},{"itemCount":3301619,"kind":"edge","name":"length","objectPayloadOffset":null,"objectPayloadSizeBytes":null,"recordOffset":12071270735,"typeIndex":4,"valuesEndOffset":12097683703,"valuesOffset":12071270751},{"itemCount":3301619,"kind":"edge","name":"radii","objectPayloadOffset":null,"objectPayloadSizeBytes":null,"recordOffset":12097683703,"typeIndex":4,"valuesEndOffset":12124096670,"valuesOffset":12097683718},{"itemCount":3301619,"kind":"edge","name":"edge_geometry_indices","objectPayloadOffset":null,"objectPayloadSizeBytes":null,"recordOffset":12124096670,"typeIndex":10,"valuesEndOffset":12203335557,"valuesOffset":12124096701},{"itemCount":3301619,"kind":"edge","name":"radii_atlas","objectPayloadOffset":null,"objectPayloadSizeBytes":null,"recordOffset":12203335557,"typeIndex":4,"valuesEndOffset":12229748530,"valuesOffset":12203335578},{"itemCount":3301619,"kind":"edge","name":"distance_to_surface","objectPayloadOffset":null,"objectPayloadSizeBytes":null,"recordOffset":12229748530,"typeIndex":4,"valuesEndOffset":12256161511,"valuesOffset":12229748559},{"itemCount":3301619,"kind":"edge","name":"annotation","objectPayloadOffset":null,"objectPayloadSizeBytes":null,"recordOffset":12256161511,"typeIndex":3,"valuesEndOffset":12282574483,"valuesOffset":12256161531}],"vertexCount":2231019},"hemisphereLaterality":{"genericPipelineEvidenceSubstituted":false,"observedMatchingPropertyCount":0,"reason":"The exact pinned graph has no graph-level edge_geometry_hemisphere property. Its bilateral annotation cannot distinguish the two ML signs, and generic pipeline defaults are not specimen evidence.","reasonCode":"SOURCE_HEMISPHERE_PROPERTY_MISSING","requiredProperty":{"cardinality":1,"kind":"graph","name":"edge_geometry_hemisphere","typeIndex":14},"status":"unavailable"},"predeclaredAcceptanceCriteria":{"criteriaAreRuntimeConfigurable":false,"expectedOrientation":"AP_KEEP_DV_KEEP_ML_KEEP","minimumAPDVWinnerMargin":"1/5","minimumAnnotationAncestorAgreement":"4/5","minimumAnnotationEvaluableFraction":"7/10","minimumAnnotationEvaluablePoints":50000,"minimumExpectedGlobalLead":"1/100","minimumHemisphereAgreement":"99/100","minimumHemisphereDiscriminatingFraction":"19/20","minimumHemisphereWinnerMargin":"49/50","requireWholeBrainCoverageOrExplicitQualifiedMirroring":true,"requiredSourceHemisphereProperty":{"cardinality":1,"kind":"graph","name":"edge_geometry_hemisphere","typeIndex":14}},"retainedSelection":{"assetSha256":"fb2344e845e604be3424bd63f4222d273eafba34db0df2eaff32f4400fa9afec","candidateEdges":16156,"exactBundledAssetMatch":true,"outputPoints":71313,"outputRuns":11818,"outputSegments":59495,"qualifyingInBoundsPoints":78048,"sourceEdgesWithRuns":10907},"schemaVersion":1,"scopeLimitations":["This report qualifies only axis permutation, AP/DV signs, and ML laterality for the exact pinned files when every check passes.","It does not bound atlas-registration error, tissue distortion, biological variation, or omitted vessels.","It does not make the fixed-tissue reference subject-specific or validate it for surgical navigation.","It does not establish artery/vein identity or clearance when no loaded-geometry conflict is found."],"source":{"filename":"606_graph_2024-12-03.gt","sha256":"c2568cfbecd3f3eb720519be9d042f0cb41606741b8dd2f018bad1c54d36ef85","sizeBytes":12282574483},"sourceCoverage":{"atlasMidlineIndex":228,"bilateralMirroringQualified":false,"bilateralMirroringReason":"Neither the exact graph metadata nor the dataset record defines a reviewed mirror operation that reconstructs a whole-brain major-vessel reference.","graphMetadataAssessment":{"conclusion":"The graph stores raw, atlas, and MRI coordinates plus a raw image shape, but no source-orientation label, alignment transform, resampling metadata, workflow commit, or specimen configuration that binds ML sign.","graphLevelPropertyNames":["shape","edge_geometry_type","edge_geometry_coordinates","edge_geometry_radii","edge_geometry_coordinates_atlas","edge_geometry_radii_atlas","edge_geometry_annotation","edge_geometry_distance_to_surface","edge_geometry_coordinates_mri","length_coordinates","length_coordinates_mri","length_coordinates_atlas","edge_geometry_length_coordinates","edge_geometry_length_coordinates_mri","edge_geometry_length_coordinates_atlas"],"propertiesBindingBiologicalLaterality":[],"propertiesBindingExactWorkflowCommit":[],"propertiesContainingSourceOrientationOrAlignmentTransform":[]},"numericalBothSidesDoNotProveWholeBrain":true,"primarySource":{"finding":"The official dataset description identifies the specimens as mouse brain hemispheres, not whole-brain vascular graphs.","url":"https://zenodo.org/records/18876865"},"reasonCode":"SOURCE_SPECIMEN_COVERAGE_IS_HEMISPHERE","retainedMajorVesselCoordinatesAtlas":{"axis2AtOrAboveMidlineCount":15969,"axis2BelowMidlineCount":55344,"axis2Maximum":"275.406550802","axis2Minimum":"14.709281147","pointCount":71313},"status":"unqualified-for-whole-brain-use","vertexCoordinatesAtlas":{"axis2AtOrAboveMidlineCount":358880,"axis2BelowMidlineCount":1872139,"maximumByClearMapAxis":["301.038440935","545.949368519","282.876755910"],"minimumByClearMapAxis":["6.633265227","72.521141143","13.791384656"],"pointCount":2231019}},"status":"rejected"} diff --git a/native/Brain3D/README.md b/native/Brain3D/README.md index 9199ad5..f3adb71 100644 --- a/native/Brain3D/README.md +++ b/native/Brain3D/README.md @@ -1,134 +1,38 @@ # Brain3D native macOS app -This Swift package is the native half of the Brain3D animal-research planner. SwiftUI owns the -single-view workspace and controls; SceneKit renders backend-validated 3D brain and probe geometry -plus display-only major-vessel payloads; a typed NDJSON subprocess boundary delegates scientific -state and analysis to Python. +The Swift package provides the Brain3D interface. SwiftUI owns the workspace and controls, +SceneKit renders 3D atlas and planning geometry, and the Python service provides atlas, +coordinate, persistence, and planning operations over typed NDJSON. -> **Animal research only — non-human and non-clinical.** This development app is not an -> installable production release or a qualified navigation device. +## Workspace -## Current workspace - -- Exactly `Dorsal / Coronal / Sagittal / Horizontal / 3D`, one selected mode at a time. -- Independent retained slice depths, with slider/buttons/wheel, pan, zoom, and compact - click-to-replace region labels. A region click never changes depth. -- Complete paged Allen ontology browsing/search with one selection shared across all five modes; - descendant-inclusive annotation overlays highlight reviewed voxels in all four 2D modes and a - reviewed selected-structure mesh highlights in 3D. An entry with neither voxels nor mesh stays - selected with zero 2D pixels and an explicit 3D no-geometry state. -- No focus mode, crosshair, 2×2 layout, or capillary display. -- Direct surface-insertion AP/ML planning from a source-pinned Pinpoint/Urchin profile: AP− is - posterior and ML− is animal-left; AP/ML anchors user-facing Shank 1 at its exact local - annotation-surface crossing, not at the array midpoint. -- Positive path depth from Shank 1's surface crossing to its distal target, one sagittal angle - (+ A→P / − P→A), and NP2013 layout. `Sagittal` makes Shank 1 most anterior; `90° CW` from - dorsal makes it animal-left-most and extends the other shanks toward animal right. 3D renders - each full 10 mm shaft, while slice/traversal analysis uses the implanted surface-to-tip path. -- VesSAP `BL6J-no1` nominal diameter-≥30 µm paths overlaid in all five views. The layer is one - cleared C57BL/6J reference, reduced at 50 µm for display, not subject-specific. -- No vessel-analysis capability. Geometry is display-only and analysis fails with - `VESSEL_ANALYSIS_UNAVAILABLE`. - -The primary selector contains exactly NP2 single shank `NP2003` and standard four shank -`NP2013`. `NP2004`/`NP2014` remain source-artifact aliases rather than extra UI choices. Both -source transcriptions remain independent-review pending; v4 has no per-plan geometry checkbox. -Quad Base, NP1, and the synthetic fixture remain archived definitions, not new-plan choices. - -## Surgery-plan PDF export - -The native app can export a prefilled animal-research planning packet for the current probe plan. -Choose `Dorsal`, `Coronal`, `Sagittal`, `Horizontal`, `3D`, or `All`; `All` emits the five -planning views in that order. AP/ML locates Shank 1's local atlas-surface crossing and path depth -ends at that shank's distal target; both are millimetres. The signed A↔P angle is degrees and -layout is sagittal or 90° clockwise from dorsal. Coronal -historical plates use `Bregma = Interaural − 3.80 mm`; sagittal matching uses `|ML|` while -preserving the implant's left/right sign. - -The fixed packet order is: - -1. two portrait-Letter pages rendered from the user-supplied Headplate protocol; -2. one selected planning page, or five pages for `All`; and -3. one matched landscape-Letter historical atlas page. - -The app reads the prepared three-page protocol PDF directly, prefills pages 1–2, and replaces its -page-3 sketch placeholder with the matched page from the consolidated 132-page -`MBSC_Figs_with_Layers.pdf`. Configure both PDF locations once in **Brain3D → Settings**; later -exports reuse those locations until they are replaced there. PDFKit/Core Graphics read the -sources directly, merge the packet, and flatten an audit stamp onto every page. The original -source files are never rewritten. Word, Illustrator, and Apple Events automation are not used. - -V4 export freezes and repeatedly revalidates the source-pinned bregma profile, AP/ML, annotation -surface, depth, signed angle, layout, matching probe, atlas, vessel asset, project revision, and -saved/dirty state. It does not require a separate target projection or subject calibration. A -state change during rendering or assembly aborts the export instead of mixing revisions. - -The vessel overlay is the one-specimen VesSAP `BL6J-no1` ex-vivo reference, nominal diameter -at least 30 µm. It is not the operative animal, capillaries are omitted, and neither a displayed -path nor zero visible intersections establishes subject-specific location, vessel absence, -clearance, trajectory suitability, or safety. The packet is for non-human animal research only, -not clinical use or qualified navigation. - -The protocol and Mouse Brain atlas sources are supplied by the user and are not redistributed -with Brain3D. See the full [surgery-plan export contract](../../docs/SURGERY_PLAN_EXPORT.md), -including prerequisites, atlas matching, immutable provenance, and source-rights boundaries. - -The older P60_606 vessel source remains archived because its hemisphere/laterality and whole-brain -coverage are unqualified; it is not mirrored or mixed into VesSAP. The visible VesSAP layer also -cannot establish subject-specific clearance, absence, suitability, or safety; see -[its evidence record](../../docs/VESSAP_MAJOR_VESSELS.md). -Population density and subject-image registration remain archived in Python and absent from the -primary UI. Only `allen_mouse_25um` v1.2 is accepted in this 25 µm testing phase. +- Dorsal, coronal, sagittal, horizontal, and 3D modes. +- Independent slice depths with pan, zoom, wheel, and region picking. +- Shared Allen ontology selection across 2D and 3D. +- Direct AP/ML surface planning for NP2003 and NP2013. +- Optional external VesSAP overlay when its data package is installed. +- Surgery-plan PDF export using documents selected in Settings. ## Development -```sh +```bash swift test --package-path native/Brain3D --no-parallel native/Brain3D/Scripts/build-app.sh open native/Brain3D/build/Brain3D.app ``` -Backend discovery is deterministic and fail-closed: - -1. `BRAIN3D_BRIDGE_EXECUTABLE` plus optional JSON-array `BRAIN3D_BRIDGE_ARGUMENTS` and - `BRAIN3D_BRIDGE_WORKING_DIRECTORY` override discovery. -2. A regular executable at `Contents/Resources/Bridge/brain3d-bridge` is used from a standalone - app, provided its resolved path remains inside that app's Resources directory. -3. Development builds walk upward from the current directory, executable, and source package for - both `.venv/bin/python` and `src/mouse_brain_planner/bridge/server.py`. -4. If none of those routes resolves, the UI reports **Backend not configured** and does not - substitute demo anatomy or geometry. +Backend discovery checks explicit `BRAIN3D_BRIDGE_*` settings, a bridge bundled inside a +standalone application, and the repository Python environment for development builds. -The development app launches Python with `-u -m mouse_brain_planner.bridge.server` and sets -`PYTHONPATH` to the repository `src` directory. UI requests are asynchronous and strict decoders -reject source, schema, revision, hash, coordinate-frame, and size mismatches. +## Standalone build -## Distribution - -`Scripts/build-app.sh` remains the fast development bundle and depends on the checkout. For an -Apple-Silicon-only standalone build, run from the repository root: - -```sh +```bash native/Brain3D/Scripts/build-release-app.sh ``` -This uses exact uv-managed CPython 3.12.12 and PyInstaller 6.21.0 from the lockfile. It writes -`dist/Brain3D.app` and `dist/Brain3D-macOS-arm64.zip` below this package unless `OUTPUT_DIR` is set. -The bundled NDJSON executable is `Contents/Resources/Bridge/brain3d-bridge`; no Python installation, -checkout, `.venv`, or inherited `PYTHONPATH` is used at runtime. - -The builder verifies all Swift, Python, extension-module, and dynamic-library Mach-O files are -arm64-only and have deployment targets no newer than macOS 14.0. It rejects host-only absolute -dependencies/runpaths, escaping symlinks, development Python modules, extra archive roots, and a -failed code signature. It then extracts the zip into an isolated directory, sandbox-denies the -source checkout, checks bridge `hello`, launches the real Swift executable, and confirms Swift -starts the exact bridge inside the extracted app without an immediate crash. - -A discovered `Developer ID Application` identity enables hardened-runtime signing; otherwise the -builder ad-hoc signs the app. Ad-hoc builds are not notarized. A public release still needs -notarization and clean-machine macOS 14 qualification by the release operator. Production SBOM, -dependency license copies, and build provenance are embedded in `Contents/Resources/Release`. +The release script builds the Swift executable and frozen Python bridge, assembles the application +and zip archive, verifies architecture and signatures, and runs an isolated launch smoke test. -Downloaded Allen atlas data and the user-selected surgery protocol/atlas PDFs are intentionally -outside the app. The bundled VesSAP derivative remains CC BY-NC 4.0, so public distribution must -remain noncommercial and preserve its attribution, license, manifest, and display-only limits. +Atlas packages, vascular data, PDF inputs, and user projects remain outside the application +bundle. See [Packaging](../../PACKAGING.md) and +[Surgery-plan export](../../docs/SURGERY_PLAN_EXPORT.md). diff --git a/scripts/benchmark_major_vessel_analysis.py b/scripts/benchmark_major_vessel_analysis.py deleted file mode 100644 index 8368417..0000000 --- a/scripts/benchmark_major_vessel_analysis.py +++ /dev/null @@ -1,270 +0,0 @@ -#!/usr/bin/env python3 -"""Benchmark V3 analysis against the verified bundled LAMBADA asset. - -The script performs no network access. It reports one asset-load measurement, -one cold analysis, and bounded repeated warm analyses as machine-readable JSON. -Python/module startup time is intentionally outside the reported measurements. -""" - -from __future__ import annotations - -import argparse -import json -import math -import platform -import subprocess -from statistics import median -from time import perf_counter - -import numpy as np - -from mouse_brain_planner.analysis.vessel_clearance import ( - ProbeShankASR, - RadiusBearingVesselRuns, - analyze_probe_vessel_clearance, -) -from mouse_brain_planner.domain.vessel_clearance_models import ( - MajorVesselSourceProvenance, - ProbeVesselAnalysis, - VesselRiskProfile, -) -from mouse_brain_planner.vasculature.lambada_major_vessels import ( - ASSET_SHA256, - EXPECTED_RUN_COUNT, - EXPECTED_RUN_POINT_COUNT, - EXPECTED_SEGMENT_COUNT, - EXTRACTION_ALGORITHM_VERSION, - MINIMUM_DIAMETER_UM, - SOURCE_ARCHIVE_SHA256, - SOURCE_LICENSE, - SOURCE_PAPER_DOI, - SOURCE_RECORD_DOI, - SOURCE_RECORD_URL, - LambadaMajorVesselGraph, - load_lambada_major_vessels, -) - -MINIMUM_ITERATIONS = 5 -MAXIMUM_ITERATIONS = 100 -DEFAULT_ITERATIONS = 20 -MAXIMUM_CONFLICTS = 10_000 - - -def _percentile(values: list[float], fraction: float) -> float: - """Return a linearly interpolated percentile for one nonempty sample.""" - - ordered = sorted(values) - position = (len(ordered) - 1) * fraction - lower = math.floor(position) - upper = math.ceil(position) - if lower == upper: - return ordered[lower] - weight = position - lower - return ordered[lower] * (1.0 - weight) + ordered[upper] * weight - - -def _darwin_sysctl(name: str) -> str | None: - if platform.system() != "Darwin": - return None - try: - completed = subprocess.run( - ["sysctl", "-n", name], - check=False, - capture_output=True, - text=True, - timeout=2, - ) - except (OSError, subprocess.SubprocessError): - return None - value = completed.stdout.strip() - return value if completed.returncode == 0 and value else None - - -def _machine_payload() -> dict[str, str]: - os_version = platform.mac_ver()[0] if platform.system() == "Darwin" else platform.release() - return { - "system": platform.system(), - "osVersion": os_version, - "architecture": platform.machine(), - "hardwareModel": _darwin_sysctl("hw.model") or "unknown", - "processor": ( - _darwin_sysctl("machdep.cpu.brand_string") or platform.processor() or "unknown" - ), - } - - -def _analysis_geometry(graph: LambadaMajorVesselGraph) -> RadiusBearingVesselRuns: - return RadiusBearingVesselRuns( - points_asr_um=np.asarray(graph.points_asr_um, dtype=np.float64), - radii_um=np.asarray(graph.radii_um, dtype=np.float64), - run_offsets=graph.run_offsets, - source_edge_indices=np.asarray(graph.source_edge_indices, dtype=np.int64), - ) - - -def _source_provenance(graph: LambadaMajorVesselGraph) -> MajorVesselSourceProvenance: - value = graph.provenance - return MajorVesselSourceProvenance( - source_id="lambada-p60-606-major-vessels-v1", - source_doi=SOURCE_RECORD_DOI, - source_record_url=SOURCE_RECORD_URL, - source_paper_doi=SOURCE_PAPER_DOI, - source_version="P60_606 / 606_graph_2024-12-03.gt", - source_license=SOURCE_LICENSE, - dataset_title=value.dataset_title, - authors=value.authors, - specimen_id=value.specimen_id, - source_archive_digest=f"sha256:{SOURCE_ARCHIVE_SHA256}", - derived_asset_sha256=value.asset_sha256, - extraction_algorithm_version=EXTRACTION_ALGORITHM_VERSION, - minimum_included_diameter_um=MINIMUM_DIAMETER_UM, - ) - - -def _benchmark_inputs() -> tuple[ProbeShankASR, VesselRiskProfile]: - shank = ProbeShankASR( - shank_id="benchmark-shank", - entry_asr_um=np.asarray((6_600.0, 0.0, 5_700.0), dtype=np.float64), - tip_asr_um=np.asarray((6_600.0, 6_000.0, 5_700.0), dtype=np.float64), - envelope_radius_um=35.0, - ) - profile = VesselRiskProfile( - profile_id="benchmark-profile-v1", - minimum_vessel_diameter_um=MINIMUM_DIAMETER_UM, - required_margin_um=100.0, - registration_uncertainty_um=100.0, - source_or_lab_policy="Deterministic offline performance benchmark only.", - confirmed_by_user=True, - reference_only_coverage_acknowledged=True, - ) - return shank, profile - - -def _run_analysis( - *, - shank: ProbeShankASR, - vessels: RadiusBearingVesselRuns, - profile: VesselRiskProfile, - provenance: MajorVesselSourceProvenance, -) -> ProbeVesselAnalysis: - return analyze_probe_vessel_clearance( - shanks=(shank,), - vessels=vessels, - risk_profile=profile, - provenance=provenance, - maximum_conflicts=MAXIMUM_CONFLICTS, - ) - - -def _result_signature(result: ProbeVesselAnalysis) -> tuple[object, ...]: - return ( - result.algorithm_version, - result.input_sha256, - result.result_status, - result.candidate_segment_count, - result.measured_segment_count, - len(result.conflicts), - result.conflicts_truncated, - ) - - -def main() -> int: - parser = argparse.ArgumentParser( - description="Benchmark bundled major-vessel analysis without network access." - ) - parser.add_argument("--iterations", type=int, default=DEFAULT_ITERATIONS) - args = parser.parse_args() - if not MINIMUM_ITERATIONS <= args.iterations <= MAXIMUM_ITERATIONS: - parser.error(f"--iterations must be in [{MINIMUM_ITERATIONS}, {MAXIMUM_ITERATIONS}]") - - loaded_at = perf_counter() - graph = load_lambada_major_vessels() - vessels = _analysis_geometry(graph) - load_ms = (perf_counter() - loaded_at) * 1_000.0 - segment_count = int(np.sum(np.diff(graph.run_offsets) - 1)) - if ( - graph.points_asr_um.shape[0] != EXPECTED_RUN_POINT_COUNT - or graph.run_offsets.size - 1 != EXPECTED_RUN_COUNT - or segment_count != EXPECTED_SEGMENT_COUNT - or graph.provenance.asset_sha256 != ASSET_SHA256 - ): - raise RuntimeError("bundled benchmark geometry does not match its reviewed constants") - - shank, profile = _benchmark_inputs() - provenance = _source_provenance(graph) - cold_at = perf_counter() - cold_result = _run_analysis( - shank=shank, - vessels=vessels, - profile=profile, - provenance=provenance, - ) - cold_ms = (perf_counter() - cold_at) * 1_000.0 - expected_signature = _result_signature(cold_result) - - warm_ms: list[float] = [] - for _iteration in range(args.iterations): - warm_at = perf_counter() - warm_result = _run_analysis( - shank=shank, - vessels=vessels, - profile=profile, - provenance=provenance, - ) - warm_ms.append((perf_counter() - warm_at) * 1_000.0) - if _result_signature(warm_result) != expected_signature: - raise RuntimeError("warm benchmark result changed across identical inputs") - - if cold_result.candidate_segment_count != cold_result.measured_segment_count: - raise RuntimeError("exact measured count differs from the candidate count") - if cold_result.conflicts_truncated: - raise RuntimeError("benchmark conflict output was truncated") - - print( - json.dumps( - { - "benchmark": "bundled-lambada-major-vessel-analysis-v1", - "machine": _machine_payload(), - "python": { - "implementation": platform.python_implementation(), - "version": platform.python_version(), - }, - "asset": { - "sha256": graph.provenance.asset_sha256, - "points": int(graph.points_asr_um.shape[0]), - "runs": int(graph.run_offsets.size - 1), - "segments": segment_count, - }, - "analysisInput": { - "entryASRMicrometres": shank.entry_asr_um.tolist(), - "tipASRMicrometres": shank.tip_asr_um.tolist(), - "probeEnvelopeRadiusMicrometres": shank.envelope_radius_um, - "requiredMarginMicrometres": profile.required_margin_um, - "registrationUncertaintyMicrometres": (profile.registration_uncertainty_um), - }, - "analysisResult": { - "algorithmVersion": cold_result.algorithm_version, - "status": cold_result.result_status.value, - "candidateSegments": cold_result.candidate_segment_count, - "measuredSegments": cold_result.measured_segment_count, - "conflicts": len(cold_result.conflicts), - }, - "timingMilliseconds": { - "load": round(load_ms, 3), - "cold": round(cold_ms, 3), - "warm": { - "iterations": args.iterations, - "median": round(median(warm_ms), 3), - "p95": round(_percentile(warm_ms, 0.95), 3), - }, - }, - }, - indent=2, - sort_keys=True, - ) - ) - return 0 - - -if __name__ == "__main__": - raise SystemExit(main()) diff --git a/scripts/extract_lambada_major_vessels.py b/scripts/extract_lambada_major_vessels.py deleted file mode 100644 index 6ddc2ec..0000000 --- a/scripts/extract_lambada_major_vessels.py +++ /dev/null @@ -1,94 +0,0 @@ -#!/usr/bin/env python3 -"""Build the pinned compact LAMBADA P60_606 major-vessel asset.""" - -from __future__ import annotations - -import argparse -import json -import tempfile -from dataclasses import asdict -from pathlib import Path - -from mouse_brain_planner.vasculature.lambada_major_vessels import ( - ASSET_FILENAME, - MANIFEST_FILENAME, - build_asset_manifest, - bundled_asset_paths, - extract_pinned_lambada_major_vessels, - write_asset_manifest, - write_deterministic_npz, -) - - -def _arguments() -> argparse.Namespace: - bundled_asset, bundled_manifest = bundled_asset_paths() - parser = argparse.ArgumentParser( - description=( - "Verify the exact 12 GB P60_606 graph and derive the reviewed >=15 um radius " - "in-bounds polyline runs." - ) - ) - parser.add_argument("source_graph", type=Path, help="extracted 606_graph_2024-12-03.gt") - parser.add_argument( - "--output", - type=Path, - default=bundled_asset, - help=f"asset destination (default: {bundled_asset})", - ) - parser.add_argument( - "--manifest", - type=Path, - default=bundled_manifest, - help=f"manifest destination (default: {bundled_manifest})", - ) - return parser.parse_args() - - -def main() -> int: - """Run the fail-closed extraction and byte-reproducibility check.""" - - arguments = _arguments() - source = arguments.source_graph.expanduser() - output = arguments.output.expanduser() - manifest_path = arguments.manifest.expanduser() - if output.name != ASSET_FILENAME: - raise SystemExit(f"--output filename must be {ASSET_FILENAME}") - if manifest_path.name != MANIFEST_FILENAME: - raise SystemExit(f"--manifest filename must be {MANIFEST_FILENAME}") - if output.parent != manifest_path.parent: - raise SystemExit("the asset and manifest must be adjacent") - - data, report = extract_pinned_lambada_major_vessels(source) - asset_sha256, asset_size_bytes = write_deterministic_npz(output, data) - with tempfile.TemporaryDirectory(prefix="lambada-major-vessels-") as temporary_directory: - reproducibility_path = Path(temporary_directory) / ASSET_FILENAME - repeated_sha256, repeated_size = write_deterministic_npz(reproducibility_path, data) - if (repeated_sha256, repeated_size) != (asset_sha256, asset_size_bytes): - raise SystemExit("deterministic asset reproduction failed") - - manifest = build_asset_manifest( - asset_sha256=asset_sha256, - asset_size_bytes=asset_size_bytes, - data=data, - report=report, - ) - write_asset_manifest(manifest_path, manifest) - print( - json.dumps( - { - "asset": str(output.resolve()), - "asset_sha256": asset_sha256, - "asset_size_bytes": asset_size_bytes, - "manifest": str(manifest_path.resolve()), - "report": asdict(report), - "reproducible": True, - }, - indent=2, - sort_keys=True, - ) - ) - return 0 - - -if __name__ == "__main__": - raise SystemExit(main()) diff --git a/scripts/qualify_lambada_coordinates.py b/scripts/qualify_lambada_coordinates.py deleted file mode 100644 index fad5e05..0000000 --- a/scripts/qualify_lambada_coordinates.py +++ /dev/null @@ -1,76 +0,0 @@ -#!/usr/bin/env python3 -"""Generate a fail-closed coordinate attestation for LAMBADA P60_606.""" - -from __future__ import annotations - -import argparse -import hashlib -import json -from pathlib import Path - -from mouse_brain_planner.vasculature.lambada_coordinate_qualification import ( - QUALIFICATION_ALGORITHM_VERSION, - QUALIFICATION_SCHEMA_VERSION, - LambadaCoordinateQualificationError, - canonical_report_bytes, - qualify_pinned_lambada_coordinates, - write_canonical_report, -) - - -def _arguments() -> argparse.Namespace: - parser = argparse.ArgumentParser( - description=( - "Qualify AP/DV signs and ML laterality for the exact pinned LAMBADA " - "P60_606 graph against the exact Allen 25 um v1.2 atlas." - ) - ) - parser.add_argument("source_graph", type=Path, help="extracted 606_graph_2024-12-03.gt") - parser.add_argument( - "atlas_directory", - type=Path, - help="exact BrainGlobe allen_mouse_25um_v1.2 package directory", - ) - parser.add_argument( - "--output", - type=Path, - required=True, - help="canonical JSON report destination", - ) - return parser.parse_args() - - -def main() -> int: - arguments = _arguments() - try: - report = qualify_pinned_lambada_coordinates( - arguments.source_graph.expanduser(), - arguments.atlas_directory.expanduser(), - ) - except LambadaCoordinateQualificationError as error: - report = { - "schemaVersion": QUALIFICATION_SCHEMA_VERSION, - "algorithmVersion": QUALIFICATION_ALGORITHM_VERSION, - "status": "rejected", - "qualificationError": str(error), - } - report_sha256 = write_canonical_report(arguments.output.expanduser(), report) - payload = canonical_report_bytes(report) - if hashlib.sha256(payload).hexdigest() != report_sha256: - raise AssertionError("canonical report digest changed after writing") - print( - json.dumps( - { - "output": str(arguments.output.expanduser().resolve()), - "reportSha256": report_sha256, - "status": report["status"], - }, - indent=2, - sort_keys=True, - ) - ) - return 0 if report["status"] == "qualified" else 2 - - -if __name__ == "__main__": - raise SystemExit(main()) diff --git a/src/mouse_brain_planner/analysis/vessel_clearance.py b/src/mouse_brain_planner/analysis/vessel_clearance.py index 568fcab..910c3d9 100644 --- a/src/mouse_brain_planner/analysis/vessel_clearance.py +++ b/src/mouse_brain_planner/analysis/vessel_clearance.py @@ -163,7 +163,7 @@ def analyze_probe_vessel_clearance( abs_tol=1e-9, ): raise VesselClearanceInputError( - "risk-profile diameter must equal the pointwise threshold used by the bundled asset" + "risk-profile diameter must equal the pointwise threshold used by the configured asset" ) segments = _segments(vessels) diff --git a/src/mouse_brain_planner/assets/vasculature/VESSAP_DATA_LICENSE.txt b/src/mouse_brain_planner/assets/vasculature/VESSAP_DATA_LICENSE.txt deleted file mode 100644 index c657cab..0000000 --- a/src/mouse_brain_planner/assets/vasculature/VESSAP_DATA_LICENSE.txt +++ /dev/null @@ -1,407 +0,0 @@ -Attribution-NonCommercial 4.0 International - -======================================================================= - -Creative Commons Corporation ("Creative Commons") is not a law firm and -does not provide legal services or legal advice. Distribution of -Creative Commons public licenses does not create a lawyer-client or -other relationship. Creative Commons makes its licenses and related -information available on an "as-is" basis. Creative Commons gives no -warranties regarding its licenses, any material licensed under their -terms and conditions, or any related information. Creative Commons -disclaims all liability for damages resulting from their use to the -fullest extent possible. - -Using Creative Commons Public Licenses - -Creative Commons public licenses provide a standard set of terms and -conditions that creators and other rights holders may use to share -original works of authorship and other material subject to copyright -and certain other rights specified in the public license below. The -following considerations are for informational purposes only, are not -exhaustive, and do not form part of our licenses. - - Considerations for licensors: Our public licenses are - intended for use by those authorized to give the public - permission to use material in ways otherwise restricted by - copyright and certain other rights. Our licenses are - irrevocable. Licensors should read and understand the terms - and conditions of the license they choose before applying it. - Licensors should also secure all rights necessary before - applying our licenses so that the public can reuse the - material as expected. Licensors should clearly mark any - material not subject to the license. This includes other CC- - licensed material, or material used under an exception or - limitation to copyright. More considerations for licensors: - wiki.creativecommons.org/Considerations_for_licensors - - Considerations for the public: By using one of our public - licenses, a licensor grants the public permission to use the - licensed material under specified terms and conditions. If - the licensor's permission is not necessary for any reason--for - example, because of any applicable exception or limitation to - copyright--then that use is not regulated by the license. Our - licenses grant only permissions under copyright and certain - other rights that a licensor has authority to grant. Use of - the licensed material may still be restricted for other - reasons, including because others have copyright or other - rights in the material. A licensor may make special requests, - such as asking that all changes be marked or described. - Although not required by our licenses, you are encouraged to - respect those requests where reasonable. More considerations - for the public: - wiki.creativecommons.org/Considerations_for_licensees - -======================================================================= - -Creative Commons Attribution-NonCommercial 4.0 International Public -License - -By exercising the Licensed Rights (defined below), You accept and agree -to be bound by the terms and conditions of this Creative Commons -Attribution-NonCommercial 4.0 International Public License ("Public -License"). To the extent this Public License may be interpreted as a -contract, You are granted the Licensed Rights in consideration of Your -acceptance of these terms and conditions, and the Licensor grants You -such rights in consideration of benefits the Licensor receives from -making the Licensed Material available under these terms and -conditions. - - -Section 1 -- Definitions. - - a. Adapted Material means material subject to Copyright and Similar - Rights that is derived from or based upon the Licensed Material - and in which the Licensed Material is translated, altered, - arranged, transformed, or otherwise modified in a manner requiring - permission under the Copyright and Similar Rights held by the - Licensor. For purposes of this Public License, where the Licensed - Material is a musical work, performance, or sound recording, - Adapted Material is always produced where the Licensed Material is - synched in timed relation with a moving image. - - b. Adapter's License means the license You apply to Your Copyright - and Similar Rights in Your contributions to Adapted Material in - accordance with the terms and conditions of this Public License. - - c. Copyright and Similar Rights means copyright and/or similar rights - closely related to copyright including, without limitation, - performance, broadcast, sound recording, and Sui Generis Database - Rights, without regard to how the rights are labeled or - categorized. For purposes of this Public License, the rights - specified in Section 2(b)(1)-(2) are not Copyright and Similar - Rights. - d. Effective Technological Measures means those measures that, in the - absence of proper authority, may not be circumvented under laws - fulfilling obligations under Article 11 of the WIPO Copyright - Treaty adopted on December 20, 1996, and/or similar international - agreements. - - e. Exceptions and Limitations means fair use, fair dealing, and/or - any other exception or limitation to Copyright and Similar Rights - that applies to Your use of the Licensed Material. - - f. Licensed Material means the artistic or literary work, database, - or other material to which the Licensor applied this Public - License. - - g. Licensed Rights means the rights granted to You subject to the - terms and conditions of this Public License, which are limited to - all Copyright and Similar Rights that apply to Your use of the - Licensed Material and that the Licensor has authority to license. - - h. Licensor means the individual(s) or entity(ies) granting rights - under this Public License. - - i. NonCommercial means not primarily intended for or directed towards - commercial advantage or monetary compensation. For purposes of - this Public License, the exchange of the Licensed Material for - other material subject to Copyright and Similar Rights by digital - file-sharing or similar means is NonCommercial provided there is - no payment of monetary compensation in connection with the - exchange. - - j. Share means to provide material to the public by any means or - process that requires permission under the Licensed Rights, such - as reproduction, public display, public performance, distribution, - dissemination, communication, or importation, and to make material - available to the public including in ways that members of the - public may access the material from a place and at a time - individually chosen by them. - - k. Sui Generis Database Rights means rights other than copyright - resulting from Directive 96/9/EC of the European Parliament and of - the Council of 11 March 1996 on the legal protection of databases, - as amended and/or succeeded, as well as other essentially - equivalent rights anywhere in the world. - - l. You means the individual or entity exercising the Licensed Rights - under this Public License. Your has a corresponding meaning. - - -Section 2 -- Scope. - - a. License grant. - - 1. Subject to the terms and conditions of this Public License, - the Licensor hereby grants You a worldwide, royalty-free, - non-sublicensable, non-exclusive, irrevocable license to - exercise the Licensed Rights in the Licensed Material to: - - a. reproduce and Share the Licensed Material, in whole or - in part, for NonCommercial purposes only; and - - b. produce, reproduce, and Share Adapted Material for - NonCommercial purposes only. - - 2. Exceptions and Limitations. For the avoidance of doubt, where - Exceptions and Limitations apply to Your use, this Public - License does not apply, and You do not need to comply with - its terms and conditions. - - 3. Term. The term of this Public License is specified in Section - 6(a). - - 4. Media and formats; technical modifications allowed. The - Licensor authorizes You to exercise the Licensed Rights in - all media and formats whether now known or hereafter created, - and to make technical modifications necessary to do so. The - Licensor waives and/or agrees not to assert any right or - authority to forbid You from making technical modifications - necessary to exercise the Licensed Rights, including - technical modifications necessary to circumvent Effective - Technological Measures. For purposes of this Public License, - simply making modifications authorized by this Section 2(a) - (4) never produces Adapted Material. - - 5. Downstream recipients. - - a. Offer from the Licensor -- Licensed Material. Every - recipient of the Licensed Material automatically - receives an offer from the Licensor to exercise the - Licensed Rights under the terms and conditions of this - Public License. - - b. No downstream restrictions. You may not offer or impose - any additional or different terms or conditions on, or - apply any Effective Technological Measures to, the - Licensed Material if doing so restricts exercise of the - Licensed Rights by any recipient of the Licensed - Material. - - 6. No endorsement. Nothing in this Public License constitutes or - may be construed as permission to assert or imply that You - are, or that Your use of the Licensed Material is, connected - with, or sponsored, endorsed, or granted official status by, - the Licensor or others designated to receive attribution as - provided in Section 3(a)(1)(A)(i). - - b. Other rights. - - 1. Moral rights, such as the right of integrity, are not - licensed under this Public License, nor are publicity, - privacy, and/or other similar personality rights; however, to - the extent possible, the Licensor waives and/or agrees not to - assert any such rights held by the Licensor to the limited - extent necessary to allow You to exercise the Licensed - Rights, but not otherwise. - - 2. Patent and trademark rights are not licensed under this - Public License. - - 3. To the extent possible, the Licensor waives any right to - collect royalties from You for the exercise of the Licensed - Rights, whether directly or through a collecting society - under any voluntary or waivable statutory or compulsory - licensing scheme. In all other cases the Licensor expressly - reserves any right to collect such royalties, including when - the Licensed Material is used other than for NonCommercial - purposes. - - -Section 3 -- License Conditions. - -Your exercise of the Licensed Rights is expressly made subject to the -following conditions. - - a. Attribution. - - 1. If You Share the Licensed Material (including in modified - form), You must: - - a. retain the following if it is supplied by the Licensor - with the Licensed Material: - - i. identification of the creator(s) of the Licensed - Material and any others designated to receive - attribution, in any reasonable manner requested by - the Licensor (including by pseudonym if - designated); - - ii. a copyright notice; - - iii. a notice that refers to this Public License; - - iv. a notice that refers to the disclaimer of - warranties; - - v. a URI or hyperlink to the Licensed Material to the - extent reasonably practicable; - - b. indicate if You modified the Licensed Material and - retain an indication of any previous modifications; and - - c. indicate the Licensed Material is licensed under this - Public License, and include the text of, or the URI or - hyperlink to, this Public License. - - 2. You may satisfy the conditions in Section 3(a)(1) in any - reasonable manner based on the medium, means, and context in - which You Share the Licensed Material. For example, it may be - reasonable to satisfy the conditions by providing a URI or - hyperlink to a resource that includes the required - information. - - 3. If requested by the Licensor, You must remove any of the - information required by Section 3(a)(1)(A) to the extent - reasonably practicable. - - 4. If You Share Adapted Material You produce, the Adapter's - License You apply must not prevent recipients of the Adapted - Material from complying with this Public License. - - -Section 4 -- Sui Generis Database Rights. - -Where the Licensed Rights include Sui Generis Database Rights that -apply to Your use of the Licensed Material: - - a. for the avoidance of doubt, Section 2(a)(1) grants You the right - to extract, reuse, reproduce, and Share all or a substantial - portion of the contents of the database for NonCommercial purposes - only; - - b. if You include all or a substantial portion of the database - contents in a database in which You have Sui Generis Database - Rights, then the database in which You have Sui Generis Database - Rights (but not its individual contents) is Adapted Material; and - - c. You must comply with the conditions in Section 3(a) if You Share - all or a substantial portion of the contents of the database. - -For the avoidance of doubt, this Section 4 supplements and does not -replace Your obligations under this Public License where the Licensed -Rights include other Copyright and Similar Rights. - - -Section 5 -- Disclaimer of Warranties and Limitation of Liability. - - a. UNLESS OTHERWISE SEPARATELY UNDERTAKEN BY THE LICENSOR, TO THE - EXTENT POSSIBLE, THE LICENSOR OFFERS THE LICENSED MATERIAL AS-IS - AND AS-AVAILABLE, AND MAKES NO REPRESENTATIONS OR WARRANTIES OF - ANY KIND CONCERNING THE LICENSED MATERIAL, WHETHER EXPRESS, - IMPLIED, STATUTORY, OR OTHER. THIS INCLUDES, WITHOUT LIMITATION, - WARRANTIES OF TITLE, MERCHANTABILITY, FITNESS FOR A PARTICULAR - PURPOSE, NON-INFRINGEMENT, ABSENCE OF LATENT OR OTHER DEFECTS, - ACCURACY, OR THE PRESENCE OR ABSENCE OF ERRORS, WHETHER OR NOT - KNOWN OR DISCOVERABLE. WHERE DISCLAIMERS OF WARRANTIES ARE NOT - ALLOWED IN FULL OR IN PART, THIS DISCLAIMER MAY NOT APPLY TO YOU. - - b. TO THE EXTENT POSSIBLE, IN NO EVENT WILL THE LICENSOR BE LIABLE - TO YOU ON ANY LEGAL THEORY (INCLUDING, WITHOUT LIMITATION, - NEGLIGENCE) OR OTHERWISE FOR ANY DIRECT, SPECIAL, INDIRECT, - INCIDENTAL, CONSEQUENTIAL, PUNITIVE, EXEMPLARY, OR OTHER LOSSES, - COSTS, EXPENSES, OR DAMAGES ARISING OUT OF THIS PUBLIC LICENSE OR - USE OF THE LICENSED MATERIAL, EVEN IF THE LICENSOR HAS BEEN - ADVISED OF THE POSSIBILITY OF SUCH LOSSES, COSTS, EXPENSES, OR - DAMAGES. WHERE A LIMITATION OF LIABILITY IS NOT ALLOWED IN FULL OR - IN PART, THIS LIMITATION MAY NOT APPLY TO YOU. - - c. The disclaimer of warranties and limitation of liability provided - above shall be interpreted in a manner that, to the extent - possible, most closely approximates an absolute disclaimer and - waiver of all liability. - - -Section 6 -- Term and Termination. - - a. This Public License applies for the term of the Copyright and - Similar Rights licensed here. However, if You fail to comply with - this Public License, then Your rights under this Public License - terminate automatically. - - b. Where Your right to use the Licensed Material has terminated under - Section 6(a), it reinstates: - - 1. automatically as of the date the violation is cured, provided - it is cured within 30 days of Your discovery of the - violation; or - - 2. upon express reinstatement by the Licensor. - - For the avoidance of doubt, this Section 6(b) does not affect any - right the Licensor may have to seek remedies for Your violations - of this Public License. - - c. For the avoidance of doubt, the Licensor may also offer the - Licensed Material under separate terms or conditions or stop - distributing the Licensed Material at any time; however, doing so - will not terminate this Public License. - - d. Sections 1, 5, 6, 7, and 8 survive termination of this Public - License. - - -Section 7 -- Other Terms and Conditions. - - a. The Licensor shall not be bound by any additional or different - terms or conditions communicated by You unless expressly agreed. - - b. Any arrangements, understandings, or agreements regarding the - Licensed Material not stated herein are separate from and - independent of the terms and conditions of this Public License. - - -Section 8 -- Interpretation. - - a. For the avoidance of doubt, this Public License does not, and - shall not be interpreted to, reduce, limit, restrict, or impose - conditions on any use of the Licensed Material that could lawfully - be made without permission under this Public License. - - b. To the extent possible, if any provision of this Public License is - deemed unenforceable, it shall be automatically reformed to the - minimum extent necessary to make it enforceable. If the provision - cannot be reformed, it shall be severed from this Public License - without affecting the enforceability of the remaining terms and - conditions. - - c. No term or condition of this Public License will be waived and no - failure to comply consented to unless expressly agreed to by the - Licensor. - - d. Nothing in this Public License constitutes or may be interpreted - as a limitation upon, or waiver of, any privileges and immunities - that apply to the Licensor or You, including from the legal - processes of any jurisdiction or authority. - -======================================================================= - -Creative Commons is not a party to its public -licenses. Notwithstanding, Creative Commons may elect to apply one of -its public licenses to material it publishes and in those instances -will be considered the “Licensor.” The text of the Creative Commons -public licenses is dedicated to the public domain under the CC0 Public -Domain Dedication. Except for the limited purpose of indicating that -material is shared under a Creative Commons public license or as -otherwise permitted by the Creative Commons policies published at -creativecommons.org/policies, Creative Commons does not authorize the -use of the trademark "Creative Commons" or any other trademark or logo -of Creative Commons without its prior written consent including, -without limitation, in connection with any unauthorized modifications -to any of its public licenses or any other arrangements, -understandings, or agreements concerning use of licensed material. For -the avoidance of doubt, this paragraph does not form part of the -public licenses. - -Creative Commons may be contacted at creativecommons.org. diff --git a/src/mouse_brain_planner/assets/vasculature/lambada_p60_606_coordinate_qualification_rejected_v1.json b/src/mouse_brain_planner/assets/vasculature/lambada_p60_606_coordinate_qualification_rejected_v1.json deleted file mode 100644 index dc46559..0000000 --- a/src/mouse_brain_planner/assets/vasculature/lambada_p60_606_coordinate_qualification_rejected_v1.json +++ /dev/null @@ -1 +0,0 @@ -{"algorithmVersion":"lambada-p60-606-coordinate-signs-v1","annotationOrientation":{"method":"For each of all eight AP/DV/ML sign combinations, truncate retained continuous voxels exactly as ClearMap label_points does, sample the pinned Allen annotation, and count the source annotation when it is an ancestor of the sampled Allen structure. AP/DV are judged separately from bilateral ML symmetry.","scores":[{"ancestorAgreement":{"decimal":"0.866910610","denominator":52198,"numerator":45251},"evaluableFraction":{"decimal":"0.731956305","denominator":71313,"numerator":52198},"evaluablePointCount":52198,"exactLeafAgreement":{"decimal":"0.182439940","denominator":52198,"numerator":9523},"flipAP":false,"flipDV":false,"flipML":false,"label":"AP_KEEP_DV_KEEP_ML_KEEP","selectedPointCount":71313},{"ancestorAgreement":{"decimal":"0.846928089","denominator":51564,"numerator":43671},"evaluableFraction":{"decimal":"0.723065921","denominator":71313,"numerator":51564},"evaluablePointCount":51564,"exactLeafAgreement":{"decimal":"0.155088822","denominator":51564,"numerator":7997},"flipAP":false,"flipDV":false,"flipML":true,"label":"AP_KEEP_DV_KEEP_ML_FLIP","selectedPointCount":71313},{"ancestorAgreement":{"decimal":"0.247666836","denominator":41253,"numerator":10217},"evaluableFraction":{"decimal":"0.578477977","denominator":71313,"numerator":41253},"evaluablePointCount":41253,"exactLeafAgreement":{"decimal":"0.027028337","denominator":41253,"numerator":1115},"flipAP":false,"flipDV":true,"flipML":false,"label":"AP_KEEP_DV_FLIP_ML_KEEP","selectedPointCount":71313},{"ancestorAgreement":{"decimal":"0.245175056","denominator":41244,"numerator":10112},"evaluableFraction":{"decimal":"0.578351773","denominator":71313,"numerator":41244},"evaluablePointCount":41244,"exactLeafAgreement":{"decimal":"0.020851518","denominator":41244,"numerator":860},"flipAP":false,"flipDV":true,"flipML":true,"label":"AP_KEEP_DV_FLIP_ML_FLIP","selectedPointCount":71313},{"ancestorAgreement":{"decimal":"0.179921852","denominator":33270,"numerator":5986},"evaluableFraction":{"decimal":"0.466534853","denominator":71313,"numerator":33270},"evaluablePointCount":33270,"exactLeafAgreement":{"decimal":"0.006822964","denominator":33270,"numerator":227},"flipAP":true,"flipDV":false,"flipML":false,"label":"AP_FLIP_DV_KEEP_ML_KEEP","selectedPointCount":71313},{"ancestorAgreement":{"decimal":"0.177183301","denominator":33344,"numerator":5908},"evaluableFraction":{"decimal":"0.467572532","denominator":71313,"numerator":33344},"evaluablePointCount":33344,"exactLeafAgreement":{"decimal":"0.006208013","denominator":33344,"numerator":207},"flipAP":true,"flipDV":false,"flipML":true,"label":"AP_FLIP_DV_KEEP_ML_FLIP","selectedPointCount":71313},{"ancestorAgreement":{"decimal":"0.152792247","denominator":31677,"numerator":4840},"evaluableFraction":{"decimal":"0.444196710","denominator":71313,"numerator":31677},"evaluablePointCount":31677,"exactLeafAgreement":{"decimal":"0.003882943","denominator":31677,"numerator":123},"flipAP":true,"flipDV":true,"flipML":false,"label":"AP_FLIP_DV_FLIP_ML_KEEP","selectedPointCount":71313},{"ancestorAgreement":{"decimal":"0.153758289","denominator":31517,"numerator":4846},"evaluableFraction":{"decimal":"0.441953080","denominator":71313,"numerator":31517},"evaluablePointCount":31517,"exactLeafAgreement":{"decimal":"0.003521909","denominator":31517,"numerator":111},"flipAP":true,"flipDV":true,"flipML":true,"label":"AP_FLIP_DV_FLIP_ML_FLIP","selectedPointCount":71313}]},"atlas":{"annotation":{"filename":"annotation.tiff","sha256":"52775c6086ae6aa7d9df2ad2c9e5ab9ce36effa09f5355c7c45efeada6647e06","sizeBytes":308270826},"atlasKey":"allen_mouse_25um","atlasPackageVersion":"1.2","metadata":{"filename":"metadata.json","sha256":"119132150055826484fa75a4dc17c476d02251c0db8b4d0b7b28f6b8c4b8ae60","sizeBytes":423},"orientation":"asr","resolutionUm":[25,25,25],"shapeVoxels":[528,320,456],"structures":{"filename":"structures.json","sha256":"7157e6130b354f6bebc21aa79f882274cbbe253ac916d97123805af3b4ff8abd","sizeBytes":138106}},"checks":[{"id":"annotation-evaluable-point-count","observed":"52198","passed":true,"required":">=50000"},{"id":"annotation-evaluable-fraction","observed":"52198/71313","passed":true,"required":">=7/10"},{"id":"annotation-ancestor-agreement","observed":"45251/52198","passed":true,"required":">=4/5"},{"id":"expected-orientation-is-global-winner","observed":"AP_KEEP_DV_KEEP_ML_KEEP","passed":true,"required":"AP_KEEP_DV_KEEP_ML_KEEP"},{"id":"expected-orientation-global-lead","observed":"8963951/448589612","passed":true,"required":">=1/100"},{"id":"ap-dv-sign-margin","observed":"1333432537/2153324094","passed":true,"required":">=1/5"},{"id":"source-hemisphere-property-present","observed":"0 matching properties","passed":false,"required":"1 graph:edge_geometry_hemisphere typeIndex=14"},{"id":"source-coverage-whole-brain","observed":"official source declares brain-hemisphere specimens","passed":false,"required":"whole-brain source or explicitly reviewed bilateral reconstruction"}],"clearMapImplementationEvidence":{"annotationAndHemispherePropertyCreation":"https://github.com/ClearAnatomics/ClearMap/blob/71444a5c7456901f15e8d0ceb06fab72b74161df/ClearMap/processors/tube_map.py#L660-L682","commit":"71444a5c7456901f15e8d0ceb06fab72b74161df","evidenceScope":"Context only. Generic repository code does not bind a workflow commit, configuration, or transform to the exact P60_606 artifact.","exactArtifactSpecimenConfiguration":null,"exactArtifactWorkflowCommit":null,"graphLevelEdgeGeometryStorage":"https://github.com/ClearAnatomics/ClearMap/blob/71444a5c7456901f15e8d0ceb06fab72b74161df/ClearMap/Analysis/Graphs/GraphGt.py#L869-L910","hemisphereLabelSemantics":"https://github.com/ClearAnatomics/ClearMap/blob/71444a5c7456901f15e8d0ceb06fab72b74161df/ClearMap/Analysis/Statistics/data_frame_operations.py#L53-L59","repository":"https://github.com/ClearAnatomics/ClearMap"},"decision":{"apDvWinnerMargin":"1333432537/2153324094","bestAPDVAlternative":"AP_KEEP_DV_FLIP_ML_KEEP","blockingReasons":["SOURCE_HEMISPHERE_PROPERTY_MISSING","SOURCE_SPECIMEN_COVERAGE_IS_HEMISPHERE"],"expectedGlobalLead":"8963951/448589612","globalRunnerUp":"AP_KEEP_DV_KEEP_ML_FLIP","globalWinner":"AP_KEEP_DV_KEEP_ML_KEEP","hemisphereWinnerMargin":null,"qualifiedMapping":null},"graphTool":{"byteOrder":"little","directed":false,"edgeCount":3301619,"formatDocumentation":"https://graph-tool.skewed.de/static/docs/stable/gt_format.html","hemisphereProperty":null,"propertyCount":31,"propertyInventory":[{"itemCount":1,"kind":"graph","name":"shape","objectPayloadOffset":31054881,"objectPayloadSizeBytes":23,"recordOffset":31054858,"typeIndex":14,"valuesEndOffset":31054904,"valuesOffset":31054873},{"itemCount":1,"kind":"graph","name":"edge_geometry_type","objectPayloadOffset":31054940,"objectPayloadSizeBytes":20,"recordOffset":31054904,"typeIndex":14,"valuesEndOffset":31054960,"valuesOffset":31054932},{"itemCount":1,"kind":"graph","name":"edge_geometry_coordinates","objectPayloadOffset":31055003,"objectPayloadSizeBytes":1847541951,"recordOffset":31054960,"typeIndex":14,"valuesEndOffset":1878596954,"valuesOffset":31054995},{"itemCount":1,"kind":"graph","name":"edge_geometry_radii","objectPayloadOffset":1878596991,"objectPayloadSizeBytes":615847421,"recordOffset":1878596954,"typeIndex":14,"valuesEndOffset":2494444412,"valuesOffset":1878596983},{"itemCount":1,"kind":"graph","name":"edge_geometry_coordinates_atlas","objectPayloadOffset":2494444461,"objectPayloadSizeBytes":1847541951,"recordOffset":2494444412,"typeIndex":14,"valuesEndOffset":4341986412,"valuesOffset":2494444453},{"itemCount":1,"kind":"graph","name":"edge_geometry_radii_atlas","objectPayloadOffset":4341986455,"objectPayloadSizeBytes":615847421,"recordOffset":4341986412,"typeIndex":14,"valuesEndOffset":4957833876,"valuesOffset":4341986447},{"itemCount":1,"kind":"graph","name":"edge_geometry_annotation","objectPayloadOffset":4957833918,"objectPayloadSizeBytes":615847421,"recordOffset":4957833876,"typeIndex":14,"valuesEndOffset":5573681339,"valuesOffset":4957833910},{"itemCount":1,"kind":"graph","name":"edge_geometry_distance_to_surface","objectPayloadOffset":5573681390,"objectPayloadSizeBytes":615847421,"recordOffset":5573681339,"typeIndex":14,"valuesEndOffset":6189528811,"valuesOffset":5573681382},{"itemCount":1,"kind":"graph","name":"edge_geometry_coordinates_mri","objectPayloadOffset":6189528858,"objectPayloadSizeBytes":1847541951,"recordOffset":6189528811,"typeIndex":14,"valuesEndOffset":8037070809,"valuesOffset":6189528850},{"itemCount":1,"kind":"graph","name":"length_coordinates","objectPayloadOffset":8037070845,"objectPayloadSizeBytes":615847421,"recordOffset":8037070809,"typeIndex":14,"valuesEndOffset":8652918266,"valuesOffset":8037070837},{"itemCount":1,"kind":"graph","name":"length_coordinates_mri","objectPayloadOffset":8652918306,"objectPayloadSizeBytes":615847421,"recordOffset":8652918266,"typeIndex":14,"valuesEndOffset":9268765727,"valuesOffset":8652918298},{"itemCount":1,"kind":"graph","name":"length_coordinates_atlas","objectPayloadOffset":9268765769,"objectPayloadSizeBytes":615847421,"recordOffset":9268765727,"typeIndex":14,"valuesEndOffset":9884613190,"valuesOffset":9268765761},{"itemCount":1,"kind":"graph","name":"edge_geometry_length_coordinates","objectPayloadOffset":9884613240,"objectPayloadSizeBytes":615847421,"recordOffset":9884613190,"typeIndex":14,"valuesEndOffset":10500460661,"valuesOffset":9884613232},{"itemCount":1,"kind":"graph","name":"edge_geometry_length_coordinates_mri","objectPayloadOffset":10500460715,"objectPayloadSizeBytes":615847421,"recordOffset":10500460661,"typeIndex":14,"valuesEndOffset":11116308136,"valuesOffset":10500460707},{"itemCount":1,"kind":"graph","name":"edge_geometry_length_coordinates_atlas","objectPayloadOffset":11116308192,"objectPayloadSizeBytes":615847421,"recordOffset":11116308136,"typeIndex":14,"valuesEndOffset":11732155613,"valuesOffset":11116308184},{"itemCount":2231019,"kind":"vertex","name":"coordinates","objectPayloadOffset":null,"objectPayloadSizeBytes":null,"recordOffset":11732155613,"typeIndex":11,"valuesEndOffset":11803548242,"valuesOffset":11732155634},{"itemCount":2231019,"kind":"vertex","name":"radii","objectPayloadOffset":null,"objectPayloadSizeBytes":null,"recordOffset":11803548242,"typeIndex":4,"valuesEndOffset":11821396409,"valuesOffset":11803548257},{"itemCount":2231019,"kind":"vertex","name":"coordinates_atlas","objectPayloadOffset":null,"objectPayloadSizeBytes":null,"recordOffset":11821396409,"typeIndex":11,"valuesEndOffset":11892789044,"valuesOffset":11821396436},{"itemCount":2231019,"kind":"vertex","name":"radii_atlas","objectPayloadOffset":null,"objectPayloadSizeBytes":null,"recordOffset":11892789044,"typeIndex":4,"valuesEndOffset":11910637217,"valuesOffset":11892789065},{"itemCount":2231019,"kind":"vertex","name":"annotation","objectPayloadOffset":null,"objectPayloadSizeBytes":null,"recordOffset":11910637217,"typeIndex":3,"valuesEndOffset":11928485389,"valuesOffset":11910637237},{"itemCount":2231019,"kind":"vertex","name":"distance_to_surface","objectPayloadOffset":null,"objectPayloadSizeBytes":null,"recordOffset":11928485389,"typeIndex":4,"valuesEndOffset":11946333570,"valuesOffset":11928485418},{"itemCount":2231019,"kind":"vertex","name":"coordinates_mri","objectPayloadOffset":null,"objectPayloadSizeBytes":null,"recordOffset":11946333570,"typeIndex":11,"valuesEndOffset":12017726203,"valuesOffset":11946333595},{"itemCount":2231019,"kind":"vertex","name":"former_degree","objectPayloadOffset":null,"objectPayloadSizeBytes":null,"recordOffset":12017726203,"typeIndex":3,"valuesEndOffset":12035574378,"valuesOffset":12017726226},{"itemCount":2231019,"kind":"vertex","name":"tip_cell_prediction","objectPayloadOffset":null,"objectPayloadSizeBytes":null,"recordOffset":12035574378,"typeIndex":3,"valuesEndOffset":12053422559,"valuesOffset":12035574407},{"itemCount":2231019,"kind":"vertex","name":"tip_cell_score","objectPayloadOffset":null,"objectPayloadSizeBytes":null,"recordOffset":12053422559,"typeIndex":4,"valuesEndOffset":12071270735,"valuesOffset":12053422583},{"itemCount":3301619,"kind":"edge","name":"length","objectPayloadOffset":null,"objectPayloadSizeBytes":null,"recordOffset":12071270735,"typeIndex":4,"valuesEndOffset":12097683703,"valuesOffset":12071270751},{"itemCount":3301619,"kind":"edge","name":"radii","objectPayloadOffset":null,"objectPayloadSizeBytes":null,"recordOffset":12097683703,"typeIndex":4,"valuesEndOffset":12124096670,"valuesOffset":12097683718},{"itemCount":3301619,"kind":"edge","name":"edge_geometry_indices","objectPayloadOffset":null,"objectPayloadSizeBytes":null,"recordOffset":12124096670,"typeIndex":10,"valuesEndOffset":12203335557,"valuesOffset":12124096701},{"itemCount":3301619,"kind":"edge","name":"radii_atlas","objectPayloadOffset":null,"objectPayloadSizeBytes":null,"recordOffset":12203335557,"typeIndex":4,"valuesEndOffset":12229748530,"valuesOffset":12203335578},{"itemCount":3301619,"kind":"edge","name":"distance_to_surface","objectPayloadOffset":null,"objectPayloadSizeBytes":null,"recordOffset":12229748530,"typeIndex":4,"valuesEndOffset":12256161511,"valuesOffset":12229748559},{"itemCount":3301619,"kind":"edge","name":"annotation","objectPayloadOffset":null,"objectPayloadSizeBytes":null,"recordOffset":12256161511,"typeIndex":3,"valuesEndOffset":12282574483,"valuesOffset":12256161531}],"vertexCount":2231019},"hemisphereLaterality":{"genericPipelineEvidenceSubstituted":false,"observedMatchingPropertyCount":0,"reason":"The exact pinned graph has no graph-level edge_geometry_hemisphere property. Its bilateral annotation cannot distinguish the two ML signs, and generic pipeline defaults are not specimen evidence.","reasonCode":"SOURCE_HEMISPHERE_PROPERTY_MISSING","requiredProperty":{"cardinality":1,"kind":"graph","name":"edge_geometry_hemisphere","typeIndex":14},"status":"unavailable"},"predeclaredAcceptanceCriteria":{"criteriaAreRuntimeConfigurable":false,"expectedOrientation":"AP_KEEP_DV_KEEP_ML_KEEP","minimumAPDVWinnerMargin":"1/5","minimumAnnotationAncestorAgreement":"4/5","minimumAnnotationEvaluableFraction":"7/10","minimumAnnotationEvaluablePoints":50000,"minimumExpectedGlobalLead":"1/100","minimumHemisphereAgreement":"99/100","minimumHemisphereDiscriminatingFraction":"19/20","minimumHemisphereWinnerMargin":"49/50","requireWholeBrainCoverageOrExplicitQualifiedMirroring":true,"requiredSourceHemisphereProperty":{"cardinality":1,"kind":"graph","name":"edge_geometry_hemisphere","typeIndex":14}},"retainedSelection":{"assetSha256":"fb2344e845e604be3424bd63f4222d273eafba34db0df2eaff32f4400fa9afec","candidateEdges":16156,"exactBundledAssetMatch":true,"outputPoints":71313,"outputRuns":11818,"outputSegments":59495,"qualifyingInBoundsPoints":78048,"sourceEdgesWithRuns":10907},"schemaVersion":1,"scopeLimitations":["This report qualifies only axis permutation, AP/DV signs, and ML laterality for the exact pinned files when every check passes.","It does not bound atlas-registration error, tissue distortion, biological variation, or omitted vessels.","It does not make the fixed-tissue reference subject-specific or validate it for surgical navigation.","It does not establish artery/vein identity or clearance when no loaded-geometry conflict is found."],"source":{"filename":"606_graph_2024-12-03.gt","sha256":"c2568cfbecd3f3eb720519be9d042f0cb41606741b8dd2f018bad1c54d36ef85","sizeBytes":12282574483},"sourceCoverage":{"atlasMidlineIndex":228,"bilateralMirroringQualified":false,"bilateralMirroringReason":"Neither the exact graph metadata nor the dataset record defines a reviewed mirror operation that reconstructs a whole-brain major-vessel reference.","graphMetadataAssessment":{"conclusion":"The graph stores raw, atlas, and MRI coordinates plus a raw image shape, but no source-orientation label, alignment transform, resampling metadata, workflow commit, or specimen configuration that binds ML sign.","graphLevelPropertyNames":["shape","edge_geometry_type","edge_geometry_coordinates","edge_geometry_radii","edge_geometry_coordinates_atlas","edge_geometry_radii_atlas","edge_geometry_annotation","edge_geometry_distance_to_surface","edge_geometry_coordinates_mri","length_coordinates","length_coordinates_mri","length_coordinates_atlas","edge_geometry_length_coordinates","edge_geometry_length_coordinates_mri","edge_geometry_length_coordinates_atlas"],"propertiesBindingBiologicalLaterality":[],"propertiesBindingExactWorkflowCommit":[],"propertiesContainingSourceOrientationOrAlignmentTransform":[]},"numericalBothSidesDoNotProveWholeBrain":true,"primarySource":{"finding":"The official dataset description identifies the specimens as mouse brain hemispheres, not whole-brain vascular graphs.","url":"https://zenodo.org/records/18876865"},"reasonCode":"SOURCE_SPECIMEN_COVERAGE_IS_HEMISPHERE","retainedMajorVesselCoordinatesAtlas":{"axis2AtOrAboveMidlineCount":15969,"axis2BelowMidlineCount":55344,"axis2Maximum":"275.406550802","axis2Minimum":"14.709281147","pointCount":71313},"status":"unqualified-for-whole-brain-use","vertexCoordinatesAtlas":{"axis2AtOrAboveMidlineCount":358880,"axis2BelowMidlineCount":1872139,"maximumByClearMapAxis":["301.038440935","545.949368519","282.876755910"],"minimumByClearMapAxis":["6.633265227","72.521141143","13.791384656"],"pointCount":2231019}},"status":"rejected"} diff --git a/src/mouse_brain_planner/assets/vasculature/lambada_p60_606_major_vessels_v1.npz b/src/mouse_brain_planner/assets/vasculature/lambada_p60_606_major_vessels_v1.npz deleted file mode 100644 index 9ad2966..0000000 Binary files a/src/mouse_brain_planner/assets/vasculature/lambada_p60_606_major_vessels_v1.npz and /dev/null differ diff --git a/src/mouse_brain_planner/assets/vasculature/lambada_p60_606_major_vessels_v1.npz.manifest.json b/src/mouse_brain_planner/assets/vasculature/lambada_p60_606_major_vessels_v1.npz.manifest.json deleted file mode 100644 index db7efdc..0000000 --- a/src/mouse_brain_planner/assets/vasculature/lambada_p60_606_major_vessels_v1.npz.manifest.json +++ /dev/null @@ -1,125 +0,0 @@ -{ - "asset": { - "arrays": { - "points_asr_voxel_f32": { - "dtype": "float32", - "shape": [ - 71313, - 3 - ] - }, - "radii_um_f32": { - "dtype": "float32", - "shape": [ - 71313 - ] - }, - "run_offsets_i64": { - "dtype": "int64", - "shape": [ - 11819 - ] - }, - "source_annotation_ids_i32": { - "dtype": "int32", - "shape": [ - 71313 - ] - }, - "source_edge_indices_i32": { - "dtype": "int32", - "shape": [ - 11818 - ] - } - }, - "filename": "lambada_p60_606_major_vessels_v1.npz", - "sha256": "fb2344e845e604be3424bd63f4222d273eafba34db0df2eaff32f4400fa9afec", - "size_bytes": 814393 - }, - "extraction": { - "algorithm_version": "lambada-p60-606-major-runs-v1", - "atlas_identifier": "allen_mouse_25um", - "atlas_shape_asr": [ - 528, - 320, - 456 - ], - "atlas_version": "1.2", - "atlas_voxel_size_um": [ - 25.0, - 25.0, - 25.0 - ], - "minimum_diameter_um": 30.0, - "minimum_radius_um": 15.0, - "out_of_bounds_rule": "split and drop; never clip or interpolate", - "output_axis_order": [ - "AP", - "DV", - "ML" - ], - "output_frame": "BRAINGLOBE_VOXEL_ASR", - "radius_semantics": "source radii_atlas * 25 um; source used a mean resampling scale, not a local Jacobian correction", - "run_rule": "maximal consecutive blocks of at least two source edge-geometry points for which every point is finite, in bounds, and radius >= 15 um", - "source_annotation_semantics": "coarse source ancestor IDs; use the installed atlas annotation at display or interaction coordinates for current region identity", - "source_coordinate_order": [ - "ClearMap axis 0", - "ClearMap axis 1", - "ClearMap axis 2" - ], - "statistics": { - "asset_path_length_um_f32": 200390.96364328437, - "candidate_edges_by_source_edge_max": 16156, - "output_points": 71313, - "output_runs": 11818, - "output_segments": 59495, - "qualifying_in_bounds_points": 78048, - "selected_path_length_um_source_f64": 200390.8629190015, - "source_edges_with_runs": 10907 - }, - "voxel_anchor": "index anchor: physical_um = continuous_voxel * 25; no half-voxel shift" - }, - "limitations": [ - "Animal research use only; this derivative is not a medical device and is not validated for surgery.", - "The source is an atlas-registered fixed and cleared P60 mouse-brain reference, not live or subject-specific vasculature.", - "Pial and choroidal vessels were removed by the source workflow, and this derivative also suppresses points below a 15 um radius; missing vessels are expected.", - "This graph cannot establish subject-specific clearance or trajectory suitability; registration error, tissue distortion, biological variation, and omitted vessels are not bounded here.", - "The exact sex and sampled side of specimen P60_606 are unpublished, and the graph does not identify arteries versus veins.", - "The source workflow corrected endpoints, linearly reconnected nearby endpoints, and removed short terminal offshoots, so some paths are reconstructed rather than observed.", - "Displayed radii use the source's mean atlas-resampling scale and are not locally Jacobian-corrected lumen measurements.", - "Out-of-bounds source points split runs and are dropped without clipping or interpolation." - ], - "schema_version": 1, - "source": { - "authors": [ - "Nicolas Renier", - "Elisa de Launoit", - "Sophie Skriabine" - ], - "concept_doi": "10.5281/zenodo.18876864", - "dataset_title": "Vascular graphs of the developing post-natal mouse brain", - "distributed_archive": { - "filename": "P60_606_graph_2024-12-03.gt.7z", - "md5": "218ed346c6d7dc501301204f811be37f", - "sha256": "cc6d252ee57154f5bc0f06605a703253470a57c210d76e075831effa2098d66f", - "size_bytes": 5050194723 - }, - "extracted_graph": { - "filename": "606_graph_2024-12-03.gt", - "md5": "b0bedc97ed2c6e00a41565649b3dd86b", - "sha256": "c2568cfbecd3f3eb720519be9d042f0cb41606741b8dd2f018bad1c54d36ef85", - "size_bytes": 12282574483 - }, - "license": "CC BY 4.0", - "license_url": "https://creativecommons.org/licenses/by/4.0/", - "paper_doi": "10.1016/j.cell.2026.03.013", - "paper_url": "https://www.cell.com/cell/fulltext/S0092-8674(26)00280-1", - "record_doi": "10.5281/zenodo.18876865", - "record_url": "https://zenodo.org/records/18876865", - "specimen_age": "P60", - "specimen_id": "P60_606", - "specimen_sex": "unpublished", - "specimen_side": "unpublished" - } -} diff --git a/src/mouse_brain_planner/assets/vasculature/vessap_bl6j1_major_vessels_50um_v1.npz b/src/mouse_brain_planner/assets/vasculature/vessap_bl6j1_major_vessels_50um_v1.npz deleted file mode 100644 index 197bad7..0000000 Binary files a/src/mouse_brain_planner/assets/vasculature/vessap_bl6j1_major_vessels_50um_v1.npz and /dev/null differ diff --git a/src/mouse_brain_planner/assets/vasculature/vessap_bl6j1_major_vessels_50um_v1.npz.manifest.json b/src/mouse_brain_planner/assets/vasculature/vessap_bl6j1_major_vessels_50um_v1.npz.manifest.json deleted file mode 100644 index 70d5a7a..0000000 --- a/src/mouse_brain_planner/assets/vasculature/vessap_bl6j1_major_vessels_50um_v1.npz.manifest.json +++ /dev/null @@ -1,171 +0,0 @@ -{ - "schema_version": 1, - "asset": { - "filename": "vessap_bl6j1_major_vessels_50um_v1.npz", - "size_bytes": 1853131, - "sha256": "9300dacf25ca57a5d23377ca0dc885e34ff0d18e8d21ef7590c6dcd156cf5db7", - "arrays": { - "points_asr_um_f32": { - "dtype": "float32", - "shape": [ - 196377, - 3 - ] - }, - "radii_um_f32": { - "dtype": "float32", - "shape": [ - 196377 - ] - }, - "run_offsets_i64": { - "dtype": "int64", - "shape": [ - 76623 - ] - }, - "source_run_indices_i32": { - "dtype": "int32", - "shape": [ - 76622 - ] - } - } - }, - "source": { - "dataset_title": "Machine learning analysis of whole mouse brain vasculature", - "record_url": "https://www.discotechnologies.org/VesSAP/", - "paper_doi": "10.1038/s41592-020-0792-1", - "version": "VesSAP public repository release 2021.10.01", - "license": "CC BY-NC 4.0", - "license_url": "https://creativecommons.org/licenses/by-nc/4.0/", - "specimen_id": "BL6J-no1", - "source_bundle_sha256": "d0216b9f6fcec428845f8ecc24860b4838f27b6b69496a88d8b7ab0dcd1dbf8a", - "files": { - "BL6J-no1_iso3um_stitched_skeleton.nii.gz": { - "size_bytes": 114305865, - "sha256": "1ea1a489dfacfa509f60d984ee955e43107a579d181bb5e6f3df51da6b68beac" - }, - "BL6J-no1_iso3um_stitched_radius.nii.gz": { - "size_bytes": 136114699, - "sha256": "6a8728d4518957c0e84687b077a76e64362edc9644c6e2adce60c2c244d4b887" - }, - "elastix_atlas_registration_parameters.7z": { - "size_bytes": 38472, - "sha256": "ab2509e07dcab65f176b72337624fa2ccc38f9e32f17530964ed17af5e1e88ad" - } - } - }, - "registration": { - "transform_id": "vessap-bl6j-no1-rigid-bspline-fullres-sha256-ab2509e07dcab65f176b72337624fa2ccc38f9e32f17530964ed17af5e1e88ad", - "source_grid_order": [ - "x", - "y", - "z" - ], - "source_voxel_size_um": [ - 3.0, - 3.0, - 3.0 - ], - "published_transform_order": [ - "EulerTransform", - "BSplineTransform" - ], - "moving_grid_order": [ - "ML-left-to-right", - "AP", - "DV" - ], - "output_frame": "BRAINGLOBE_PHYSICAL_ASR_UM", - "output_axis_order": [ - "AP", - "DV", - "ML-right-to-left" - ], - "mapping": "AP_um=30*T_y; DV_um=30*T_z; ML_um=11390-30*T_x, where T is transformix physical output", - "atlas_identifier": "allen_mouse_25um", - "atlas_version": "1.2", - "atlas_shape_asr": [ - 528, - 320, - 456 - ], - "atlas_voxel_size_um": [ - 25.0, - 25.0, - 25.0 - ] - }, - "extraction": { - "algorithm_version": "vessap-bl6j1-major-skeleton-50um-v1", - "minimum_source_radius_voxels": 5, - "minimum_radius_um": 15.0, - "minimum_diameter_um": 30.0, - "display_connectivity_grid_um": 50.0, - "connectivity": "retain only mapped edges induced by 26-neighbour adjacency in the thresholded source skeleton; never connect merely adjacent target bins", - "target_point": "mean continuous transformed coordinate of retained source points in each occupied 50 um display bin", - "target_radius": "maximum retained source EDT radius in each occupied display bin * 3 um", - "statistics": { - "source_skeleton_points": 58313813, - "threshold_points": 1262706, - "in_bounds_points": 1258140, - "occupied_display_voxels": 198262, - "omitted_isolated_display_voxels": 63320, - "output_points": 196377, - "output_runs": 76622, - "output_segments": 119755, - "output_path_length_um": 3817312.0853968207 - } - }, - "validation": { - "node_sample_count": 20000, - "graph_node_count": 3820133, - "within_atlas_bounds_fraction": 0.99705, - "correct_axis_permutation": "[T_y,T_z,T_x] -> [AP,DV,ML]", - "grouped_region_agreement_10um_excluding_background": { - "matched": 18664, - "evaluated": 18765, - "fraction": 0.994618 - }, - "grouped_region_agreement_10um_excluding_background_root_and_fiber_tracts": { - "matched": 17505, - "evaluated": 17585, - "fraction": 0.995451 - }, - "grouped_region_agreement_25um_excluding_background_fraction": 0.978524, - "official_registered_label_group_agreement": { - "matched": 18761, - "evaluated": 18765, - "fraction": 0.999787 - }, - "laterality_midpoint_agreement": { - "matched": 19425, - "evaluated": 19429, - "fraction": 0.999794, - "maximum_discrepancy_distance_from_midline_um": 5.38 - }, - "laterality_evidence": "official registered atlas uses negative IDs for anatomical left; sampled signed labels prove moving ML increases left-to-right and therefore requires reversal for BrainGlobe ASR", - "sampled_jacobian_determinant_range": [ - 1.269, - 2.451 - ], - "sampled_jacobian_singular_value_range": [ - 0.672, - 1.56 - ], - "sampled_jacobian_fold_count": 0, - "uncertainty_bounds_published": false, - "clearance_enabled": false - }, - "limitations": [ - "Animal research use only; this population reference is not a medical device and is not validated for stereotaxic navigation.", - "The source is one fixed, cleared adult C57BL/6J mouse brain, not the current animal and not live vasculature.", - "Only source centerline voxels with radius at least 15 um are retained; capillaries and smaller vessels are intentionally omitted.", - "The published specimen-to-Allen rigid plus B-spline transform is retained, but no numeric registration-error or clearing-distortion bound is published.", - "Source skeleton adjacency is coalesced on a 50 um display grid; close paths can merge and isolated display voxels without a retained segment are omitted.", - "Pial and choroidal coverage is not separately classified, and artery-versus-vein identity is unavailable.", - "This layer is display-only: it cannot establish clearance, absence of a vessel, trajectory suitability, or safety for an individual animal.", - "The derived data remain CC BY-NC 4.0 and may be used only under that license." - ] -} diff --git a/src/mouse_brain_planner/bridge/major_vessels.py b/src/mouse_brain_planner/bridge/major_vessels.py index e19637f..352957b 100644 --- a/src/mouse_brain_planner/bridge/major_vessels.py +++ b/src/mouse_brain_planner/bridge/major_vessels.py @@ -68,6 +68,7 @@ SOURCE_VERSION, VesSAPMajorVesselError, VesSAPMajorVesselGraph, + default_asset_is_available, load_vessap_major_vessels, ) @@ -75,7 +76,7 @@ MAXIMUM_PROFILE_DISTANCE_UM: Final = 10_000.0 REFERENCE_PROFILE_ID: Final = "vessap-bl6j-no1-major-30um-v1" REFERENCE_POLICY: Final = ( - "Display the bundled diameter >= 30 micrometre VesSAP reference only. " + "Display the configured external diameter >= 30 micrometre VesSAP reference only. " "Clearance classification is unavailable without published subject-registration " "and tissue-distortion uncertainty bounds." ) @@ -98,8 +99,9 @@ class MajorVesselReferenceBridge: _graph_cache: VesSAPMajorVesselGraph | None = field(default=None, init=False, repr=False) _analysis_cache: RadiusBearingVesselRuns | None = field(default=None, init=False, repr=False) - def register(self) -> None: - self.dispatcher.declare_capability("auditedReferenceMajorVessels") + def register(self, *, advertise_capability: bool = True) -> None: + if advertise_capability: + self.dispatcher.declare_capability("auditedReferenceMajorVessels") self.dispatcher.register("vessel.major.reference.get", self.reference_get) self.dispatcher.register("vessel.major.reference.geometry", self.reference_geometry) self.dispatcher.register("vessel.major.reference.analyze", self.reference_analyze) @@ -429,7 +431,11 @@ def register_major_vessel_handlers( replace_project=replace_project, graph_loader=graph_loader, ) - extension.register() + extension.register( + advertise_capability=( + graph_loader is not load_vessap_major_vessels or default_asset_is_available() + ) + ) return extension diff --git a/src/mouse_brain_planner/vasculature/lambada_coordinate_qualification.py b/src/mouse_brain_planner/vasculature/lambada_coordinate_qualification.py deleted file mode 100644 index a783b97..0000000 --- a/src/mouse_brain_planner/vasculature/lambada_coordinate_qualification.py +++ /dev/null @@ -1,1582 +0,0 @@ -"""Fail-closed coordinate qualification for the pinned LAMBADA P60_606 graph. - -This module does not make the reference graph subject-specific or suitable for -surgical navigation. It answers a narrower, reproducible question: whether -the exact pinned graph contains enough internal annotation and hemisphere -evidence to support the currently proposed ClearMap-to-BrainGlobe axis signs. - -The acceptance criteria are constants, not command-line knobs. A report is an -attestation only when every check passes; otherwise its status is ``rejected``. -""" - -from __future__ import annotations - -import hashlib -import io -import json -import os -import pickle -import struct -import tempfile -from collections.abc import Mapping, Sequence -from dataclasses import dataclass -from fractions import Fraction -from pathlib import Path -from typing import Final, Literal, cast - -import numpy as np -import tifffile -from numpy.typing import NDArray - -from mouse_brain_planner.vasculature.lambada_major_vessels import ( - ANNOTATIONS_RAW_OFFSET, - ASSET_SHA256, - ATLAS_SHAPE_ASR, - ATLAS_SHAPE_CLEARMAP, - ATLAS_VOXEL_SIZE_UM, - COORDINATES_ATLAS_RAW_OFFSET, - EDGE_GEOMETRY_INDICES_PAYLOAD_OFFSET, - EDGE_RADII_ATLAS_RAW_OFFSET, - EXPECTED_CANDIDATE_EDGE_COUNT, - EXPECTED_QUALIFYING_IN_BOUNDS_POINT_COUNT, - EXPECTED_RUN_COUNT, - EXPECTED_RUN_POINT_COUNT, - EXPECTED_SEGMENT_COUNT, - EXPECTED_SOURCE_EDGES_WITH_RUNS, - MINIMUM_RADIUS_ATLAS_VOXEL, - RADII_ATLAS_RAW_OFFSET, - SOURCE_EDGE_COUNT, - SOURCE_FILENAME, - SOURCE_GEOMETRY_POINT_COUNT, - SOURCE_SHA256, - SOURCE_SIZE_BYTES, - SOURCE_VERTEX_COUNT, - load_lambada_major_vessels, -) - -QUALIFICATION_SCHEMA_VERSION: Final = 1 -QUALIFICATION_ALGORITHM_VERSION: Final = "lambada-p60-606-coordinate-signs-v1" -GT_FORMAT_DOCUMENTATION_URL: Final = ( - "https://graph-tool.skewed.de/static/docs/stable/gt_format.html" -) -CLEARMAP_REPOSITORY_URL: Final = "https://github.com/ClearAnatomics/ClearMap" -CLEARMAP_EVIDENCE_COMMIT: Final = "71444a5c7456901f15e8d0ceb06fab72b74161df" -CLEARMAP_ANNOTATION_SOURCE_URL: Final = ( - "https://github.com/ClearAnatomics/ClearMap/blob/" - f"{CLEARMAP_EVIDENCE_COMMIT}/ClearMap/processors/tube_map.py#L660-L682" -) -CLEARMAP_HEMISPHERE_SEMANTICS_URL: Final = ( - "https://github.com/ClearAnatomics/ClearMap/blob/" - f"{CLEARMAP_EVIDENCE_COMMIT}/ClearMap/Analysis/Statistics/" - "data_frame_operations.py#L53-L59" -) -CLEARMAP_GRAPH_STORAGE_URL: Final = ( - "https://github.com/ClearAnatomics/ClearMap/blob/" - f"{CLEARMAP_EVIDENCE_COMMIT}/ClearMap/Analysis/Graphs/GraphGt.py#L869-L910" -) -ZENODO_DATASET_RECORD_URL: Final = "https://zenodo.org/records/18876865" - -ALLEN_ATLAS_DIRECTORY_NAME: Final = "allen_mouse_25um_v1.2" -ALLEN_ANNOTATION_FILENAME: Final = "annotation.tiff" -ALLEN_STRUCTURES_FILENAME: Final = "structures.json" -ALLEN_METADATA_FILENAME: Final = "metadata.json" -ALLEN_ANNOTATION_SIZE_BYTES: Final = 308_270_826 -ALLEN_STRUCTURES_SIZE_BYTES: Final = 138_106 -ALLEN_METADATA_SIZE_BYTES: Final = 423 -ALLEN_ANNOTATION_SHA256: Final = "52775c6086ae6aa7d9df2ad2c9e5ab9ce36effa09f5355c7c45efeada6647e06" -ALLEN_STRUCTURES_SHA256: Final = "7157e6130b354f6bebc21aa79f882274cbbe253ac916d97123805af3b4ff8abd" -ALLEN_METADATA_SHA256: Final = "119132150055826484fa75a4dc17c476d02251c0db8b4d0b7b28f6b8c4b8ae60" - -HEMISPHERE_PROPERTY_NAME: Final = "edge_geometry_hemisphere" -MISSING_HEMISPHERE_REASON_CODE: Final = "SOURCE_HEMISPHERE_PROPERTY_MISSING" -HEMISPHERE_COVERAGE_REASON_CODE: Final = "SOURCE_SPECIMEN_COVERAGE_IS_HEMISPHERE" -CLEARMAP_LEFT_HEMISPHERE_VALUE: Final = 0 -CLEARMAP_RIGHT_HEMISPHERE_VALUE: Final = 255 -ML_MIDLINE_VOXEL: Final = ATLAS_SHAPE_ASR[2] // 2 - -# Predeclared, non-configurable acceptance criteria. Fractions keep both the -# comparisons and the canonical JSON independent of binary floating point. -MIN_ANNOTATION_EVALUABLE_POINTS: Final = 50_000 -MIN_ANNOTATION_EVALUABLE_FRACTION: Final = Fraction(7, 10) -MIN_ANNOTATION_ANCESTOR_AGREEMENT: Final = Fraction(4, 5) -MIN_EXPECTED_GLOBAL_LEAD: Final = Fraction(1, 100) -MIN_AP_DV_WINNER_MARGIN: Final = Fraction(1, 5) -MIN_HEMISPHERE_DISCRIMINATING_FRACTION: Final = Fraction(19, 20) -MIN_HEMISPHERE_AGREEMENT: Final = Fraction(99, 100) -MIN_HEMISPHERE_WINNER_MARGIN: Final = Fraction(49, 50) -MAX_HEMISPHERE_PICKLE_BYTES: Final = 1_000_000_000 - -_GT_MAGIC: Final = b"\xe2\x9b\xbe gt" -_GT_VERSION: Final = 1 -_GT_PROPERTY_KINDS: Final = {0: "graph", 1: "vertex", 2: "edge"} -_GT_FIXED_VALUE_SIZES: Final = {0: 1, 1: 2, 2: 4, 3: 8, 4: 8, 5: 16} -_GT_VECTOR_VALUE_SIZES: Final = {7: 1, 8: 2, 9: 4, 10: 8, 11: 8, 12: 16} - - -class LambadaCoordinateQualificationError(ValueError): - """Raised when qualification input violates a pinned trust boundary.""" - - -@dataclass(frozen=True, slots=True) -class GTPropertyRecord: - """One graph-tool property record located without decoding its value.""" - - kind: Literal["graph", "vertex", "edge"] - name: str - type_index: int - item_count: int - record_offset: int - values_offset: int - values_end_offset: int - object_payload_offset: int | None = None - object_payload_size_bytes: int | None = None - - -@dataclass(frozen=True, slots=True) -class GTFileInventory: - """Validated graph-tool header, topology counts, and property inventory.""" - - byte_order: Literal["little", "big"] - directed: bool - vertex_count: int - edge_count: int - properties: tuple[GTPropertyRecord, ...] - - -@dataclass(frozen=True, slots=True) -class RetainedSourceSelection: - """Exact retained derivative plus its original graph-geometry indices.""" - - points_clearmap_voxel_f64: NDArray[np.float64] - source_geometry_indices_i64: NDArray[np.int64] - radii_um_f32: NDArray[np.float32] - source_annotation_ids_i32: NDArray[np.int32] - run_offsets_i64: NDArray[np.int64] - source_edge_indices_i32: NDArray[np.int32] - qualifying_in_bounds_points: int - source_edges_with_runs: int - - -@dataclass(frozen=True, slots=True) -class AtlasEvidence: - """Pinned atlas arrays and hierarchy used by the orientation scorer.""" - - annotation: NDArray[np.uint32] - structure_paths: Mapping[int, frozenset[int]] - identity_report: Mapping[str, object] - - -def _hash_file(path: Path) -> str: - digest = hashlib.sha256() - with path.open("rb") as stream: - while chunk := stream.read(16 * 1024 * 1024): - digest.update(chunk) - return digest.hexdigest() - - -def _read_exact(stream: io.BufferedReader, size: int, *, field: str) -> bytes: - if size < 0: - raise LambadaCoordinateQualificationError(f"negative byte count for {field}") - value = stream.read(size) - if len(value) != size: - raise LambadaCoordinateQualificationError(f"truncated graph-tool {field}") - return value - - -def _unpack_unsigned(raw: bytes, *, byte_order: str) -> int: - prefix = "<" if byte_order == "little" else ">" - format_code = {1: "B", 2: "H", 4: "I", 8: "Q"}.get(len(raw)) - if format_code is None: - raise AssertionError("unsupported integer width") - return int(struct.unpack(f"{prefix}{format_code}", raw)[0]) - - -def _read_u64(stream: io.BufferedReader, *, byte_order: str, field: str) -> int: - return _unpack_unsigned(_read_exact(stream, 8, field=field), byte_order=byte_order) - - -def _seek_forward_checked( - stream: io.BufferedReader, - byte_count: int, - *, - file_size: int, - field: str, -) -> None: - if byte_count < 0 or byte_count > file_size - stream.tell(): - raise LambadaCoordinateQualificationError(f"graph-tool {field} exceeds the file") - stream.seek(byte_count, os.SEEK_CUR) - - -def _read_gt_string( - stream: io.BufferedReader, - *, - byte_order: str, - file_size: int, - field: str, - maximum_size: int = 1024 * 1024, -) -> str: - size = _read_u64(stream, byte_order=byte_order, field=f"{field} length") - if size > maximum_size: - raise LambadaCoordinateQualificationError(f"graph-tool {field} is unreasonably large") - raw = _read_exact(stream, size, field=field) - try: - return raw.decode("utf-8") - except UnicodeDecodeError as error: - raise LambadaCoordinateQualificationError( - f"graph-tool {field} is not valid UTF-8" - ) from error - - -def _skip_length_prefixed_items( - stream: io.BufferedReader, - *, - count: int, - element_size: int, - byte_order: str, - file_size: int, - field: str, -) -> tuple[int | None, int | None]: - first_payload_offset: int | None = None - first_payload_size: int | None = None - for item_index in range(count): - length = _read_u64( - stream, - byte_order=byte_order, - field=f"{field}[{item_index}] length", - ) - if length > file_size // max(element_size, 1): - raise LambadaCoordinateQualificationError( - f"graph-tool {field}[{item_index}] length overflows" - ) - payload_size = length * element_size - if item_index == 0: - first_payload_offset = stream.tell() - first_payload_size = payload_size - _seek_forward_checked( - stream, - payload_size, - file_size=file_size, - field=f"{field}[{item_index}] payload", - ) - return first_payload_offset, first_payload_size - - -def scan_gt_file(path: str | Path) -> GTFileInventory: - """Scan graph-tool metadata using its documented binary encoding. - - No pickle is executed here. Variable-size property payloads are bounded - and skipped by their on-disk lengths. - """ - - source = Path(path) - if source.is_symlink() or not source.is_file(): - raise LambadaCoordinateQualificationError( - "graph-tool input must be a regular non-symlink file" - ) - file_size = source.stat().st_size - with source.open("rb") as stream: - if _read_exact(stream, 6, field="magic") != _GT_MAGIC: - raise LambadaCoordinateQualificationError("graph-tool magic does not match") - version = _read_exact(stream, 1, field="version")[0] - if version != _GT_VERSION: - raise LambadaCoordinateQualificationError( - f"unsupported graph-tool version {version}; expected {_GT_VERSION}" - ) - endian_flag = _read_exact(stream, 1, field="endianness")[0] - if endian_flag not in (0, 1): - raise LambadaCoordinateQualificationError("invalid graph-tool endianness flag") - byte_order: Literal["little", "big"] = "little" if endian_flag == 0 else "big" - _read_gt_string( - stream, - byte_order=byte_order, - file_size=file_size, - field="comment", - maximum_size=16 * 1024 * 1024, - ) - directed_flag = _read_exact(stream, 1, field="directed flag")[0] - if directed_flag not in (0, 1): - raise LambadaCoordinateQualificationError("invalid graph-tool directed flag") - vertex_count = _read_u64(stream, byte_order=byte_order, field="vertex count") - if vertex_count == 0 or vertex_count > (1 << 40): - raise LambadaCoordinateQualificationError("graph-tool vertex count is implausible") - if vertex_count <= 1 << 8: - vertex_index_size = 1 - elif vertex_count <= 1 << 16: - vertex_index_size = 2 - elif vertex_count <= 1 << 32: - vertex_index_size = 4 - else: - vertex_index_size = 8 - edge_count = 0 - for vertex_index in range(vertex_count): - degree = _read_u64( - stream, - byte_order=byte_order, - field=f"degree[{vertex_index}]", - ) - if degree > file_size // vertex_index_size: - raise LambadaCoordinateQualificationError( - f"graph-tool degree[{vertex_index}] overflows" - ) - edge_count += degree - if edge_count > (1 << 48): - raise LambadaCoordinateQualificationError("graph-tool edge count is implausible") - _seek_forward_checked( - stream, - degree * vertex_index_size, - file_size=file_size, - field=f"neighbors[{vertex_index}]", - ) - property_count = _read_u64(stream, byte_order=byte_order, field="property count") - if property_count > 100_000: - raise LambadaCoordinateQualificationError("graph-tool property count is implausible") - records: list[GTPropertyRecord] = [] - names: set[tuple[str, str]] = set() - for property_index in range(property_count): - record_offset = stream.tell() - kind_index = _read_exact(stream, 1, field=f"property[{property_index}] kind")[0] - try: - kind = _GT_PROPERTY_KINDS[kind_index] - except KeyError as error: - raise LambadaCoordinateQualificationError( - f"property[{property_index}] has an unknown kind" - ) from error - name = _read_gt_string( - stream, - byte_order=byte_order, - file_size=file_size, - field=f"property[{property_index}] name", - ) - if not name: - raise LambadaCoordinateQualificationError( - f"property[{property_index}] has an empty name" - ) - key = (kind, name) - if key in names: - raise LambadaCoordinateQualificationError( - f"duplicate graph-tool property {kind}:{name}" - ) - names.add(key) - type_index = _read_exact(stream, 1, field=f"property[{property_index}] type")[0] - item_count = {"graph": 1, "vertex": vertex_count, "edge": edge_count}[kind] - values_offset = stream.tell() - object_offset: int | None = None - object_size: int | None = None - if type_index in _GT_FIXED_VALUE_SIZES: - _seek_forward_checked( - stream, - item_count * _GT_FIXED_VALUE_SIZES[type_index], - file_size=file_size, - field=f"property[{property_index}] values", - ) - elif type_index in (6, 14): - object_offset, object_size = _skip_length_prefixed_items( - stream, - count=item_count, - element_size=1, - byte_order=byte_order, - file_size=file_size, - field=f"property[{property_index}] values", - ) - if type_index != 14 or kind != "graph": - object_offset = None - object_size = None - elif type_index in _GT_VECTOR_VALUE_SIZES: - _skip_length_prefixed_items( - stream, - count=item_count, - element_size=_GT_VECTOR_VALUE_SIZES[type_index], - byte_order=byte_order, - file_size=file_size, - field=f"property[{property_index}] vectors", - ) - elif type_index == 13: - for item_index in range(item_count): - vector_length = _read_u64( - stream, - byte_order=byte_order, - field=f"property[{property_index}] vector[{item_index}] length", - ) - if vector_length > file_size // 8: - raise LambadaCoordinateQualificationError( - "graph-tool vector length is implausible" - ) - _skip_length_prefixed_items( - stream, - count=vector_length, - element_size=1, - byte_order=byte_order, - file_size=file_size, - field=f"property[{property_index}] vector[{item_index}] strings", - ) - else: - raise LambadaCoordinateQualificationError( - f"property[{property_index}] has unsupported value type {type_index}" - ) - records.append( - GTPropertyRecord( - kind=cast(Literal["graph", "vertex", "edge"], kind), - name=name, - type_index=type_index, - item_count=item_count, - record_offset=record_offset, - values_offset=values_offset, - values_end_offset=stream.tell(), - object_payload_offset=object_offset, - object_payload_size_bytes=object_size, - ) - ) - if stream.tell() != file_size: - raise LambadaCoordinateQualificationError( - "graph-tool property inventory does not consume the exact file" - ) - return GTFileInventory( - byte_order=byte_order, - directed=bool(directed_flag), - vertex_count=vertex_count, - edge_count=edge_count, - properties=tuple(records), - ) - - -class _RestrictedNumpyUnpickler(pickle.Unpickler): - """Unpickler that permits only the globals required by a NumPy ndarray.""" - - _ALLOWED_GLOBALS: Final = { - ("numpy", "dtype"), - ("numpy", "ndarray"), - ("numpy.core.multiarray", "_reconstruct"), - ("numpy._core.multiarray", "_reconstruct"), - } - - def find_class(self, module: str, name: str) -> object: - if (module, name) not in self._ALLOWED_GLOBALS: - raise LambadaCoordinateQualificationError( - f"pickle global {module}.{name} is not permitted" - ) - return super().find_class(module, name) - - def persistent_load(self, pid: object) -> object: - del pid - raise LambadaCoordinateQualificationError("pickle persistent IDs are not permitted") - - -def load_graph_numpy_property( - path: str | Path, - record: GTPropertyRecord, - *, - expected_shape: tuple[int, ...], -) -> NDArray[np.generic]: - """Decode one bounded graph-level NumPy pickle with an allowlisted loader.""" - - if record.kind != "graph" or record.type_index != 14: - raise LambadaCoordinateQualificationError( - "requested property is not a graph-level python::object" - ) - offset = record.object_payload_offset - size = record.object_payload_size_bytes - if offset is None or size is None: - raise LambadaCoordinateQualificationError("graph object payload location is missing") - if size <= 0 or size > MAX_HEMISPHERE_PICKLE_BYTES: - raise LambadaCoordinateQualificationError( - "graph object pickle is empty or exceeds the qualification memory bound" - ) - source = Path(path) - with source.open("rb") as stream: - stream.seek(offset) - payload = stream.read(size) - if len(payload) != size: - raise LambadaCoordinateQualificationError("graph object pickle is truncated") - payload_stream = io.BytesIO(payload) - try: - value = _RestrictedNumpyUnpickler(payload_stream).load() - except LambadaCoordinateQualificationError: - raise - except (EOFError, pickle.UnpicklingError, ValueError, TypeError) as error: - raise LambadaCoordinateQualificationError( - "graph object is not an accepted NumPy pickle" - ) from error - if payload_stream.read(1): - raise LambadaCoordinateQualificationError("graph object pickle has trailing bytes") - if not isinstance(value, np.ndarray): - raise LambadaCoordinateQualificationError("graph object pickle is not a NumPy ndarray") - array = np.asarray(value) - if array.shape != expected_shape: - raise LambadaCoordinateQualificationError( - f"graph object shape {array.shape} does not match {expected_shape}" - ) - if array.dtype.kind not in "iu" or array.dtype.itemsize not in (1, 2, 4, 8): - raise LambadaCoordinateQualificationError( - "graph hemisphere ndarray must use a fixed-width integer dtype" - ) - if not array.flags.c_contiguous: - raise LambadaCoordinateQualificationError("graph hemisphere ndarray must be C-contiguous") - array.setflags(write=False) - return array - - -def verify_pinned_graph_identity(path: str | Path) -> Mapping[str, object]: - """Require the exact reviewed extracted graph before reading fixed offsets.""" - - source = Path(path) - if source.name != SOURCE_FILENAME: - raise LambadaCoordinateQualificationError( - f"source graph filename must be {SOURCE_FILENAME}" - ) - if source.is_symlink() or not source.is_file(): - raise LambadaCoordinateQualificationError("source graph must be a regular non-symlink file") - size = source.stat().st_size - if size != SOURCE_SIZE_BYTES: - raise LambadaCoordinateQualificationError( - "source graph byte count does not match the pinned file" - ) - digest = _hash_file(source) - if digest != SOURCE_SHA256: - raise LambadaCoordinateQualificationError( - "source graph SHA-256 does not match the pinned file" - ) - return {"filename": SOURCE_FILENAME, "sha256": digest, "sizeBytes": size} - - -def _verify_regular_file( - path: Path, - *, - size_bytes: int, - sha256: str, -) -> Mapping[str, object]: - if path.is_symlink() or not path.is_file(): - raise LambadaCoordinateQualificationError( - f"atlas input {path.name} must be a regular non-symlink file" - ) - if path.stat().st_size != size_bytes: - raise LambadaCoordinateQualificationError( - f"atlas input {path.name} byte count does not match the pinned package" - ) - digest = _hash_file(path) - if digest != sha256: - raise LambadaCoordinateQualificationError( - f"atlas input {path.name} SHA-256 does not match the pinned package" - ) - return {"filename": path.name, "sha256": digest, "sizeBytes": size_bytes} - - -def load_pinned_atlas_evidence(atlas_directory: str | Path) -> AtlasEvidence: - """Open the exact BrainGlobe Allen 25 um v1.2 annotation and hierarchy.""" - - directory = Path(atlas_directory) - if directory.name != ALLEN_ATLAS_DIRECTORY_NAME or not directory.is_dir(): - raise LambadaCoordinateQualificationError( - f"atlas directory must be {ALLEN_ATLAS_DIRECTORY_NAME}" - ) - annotation_path = directory / ALLEN_ANNOTATION_FILENAME - structures_path = directory / ALLEN_STRUCTURES_FILENAME - metadata_path = directory / ALLEN_METADATA_FILENAME - annotation_identity = _verify_regular_file( - annotation_path, - size_bytes=ALLEN_ANNOTATION_SIZE_BYTES, - sha256=ALLEN_ANNOTATION_SHA256, - ) - structures_identity = _verify_regular_file( - structures_path, - size_bytes=ALLEN_STRUCTURES_SIZE_BYTES, - sha256=ALLEN_STRUCTURES_SHA256, - ) - metadata_identity = _verify_regular_file( - metadata_path, - size_bytes=ALLEN_METADATA_SIZE_BYTES, - sha256=ALLEN_METADATA_SHA256, - ) - try: - metadata_value = json.loads(metadata_path.read_text(encoding="utf-8")) - except (UnicodeDecodeError, json.JSONDecodeError) as error: - raise LambadaCoordinateQualificationError("atlas metadata is not valid JSON") from error - if not isinstance(metadata_value, dict): - raise LambadaCoordinateQualificationError("atlas metadata must be a JSON object") - expected_metadata = { - "name": "allen_mouse", - "version": "1.2", - "resolution": [25.0, 25.0, 25.0], - "orientation": "asr", - "shape": list(ATLAS_SHAPE_ASR), - "symmetric": True, - } - for key, expected in expected_metadata.items(): - if metadata_value.get(key) != expected: - raise LambadaCoordinateQualificationError( - f"atlas metadata field {key!r} does not match {expected!r}" - ) - try: - raw_structures = json.loads(structures_path.read_text(encoding="utf-8")) - except (UnicodeDecodeError, json.JSONDecodeError) as error: - raise LambadaCoordinateQualificationError("atlas structures are not valid JSON") from error - if not isinstance(raw_structures, list) or not raw_structures: - raise LambadaCoordinateQualificationError("atlas structures must be a non-empty array") - paths: dict[int, frozenset[int]] = {} - for index, raw_structure in enumerate(raw_structures): - if not isinstance(raw_structure, dict): - raise LambadaCoordinateQualificationError(f"atlas structure[{index}] must be an object") - structure_id = raw_structure.get("id") - raw_path = raw_structure.get("structure_id_path") - if ( - isinstance(structure_id, bool) - or not isinstance(structure_id, int) - or structure_id <= 0 - or not isinstance(raw_path, list) - or not raw_path - or any(isinstance(item, bool) or not isinstance(item, int) for item in raw_path) - or raw_path[-1] != structure_id - ): - raise LambadaCoordinateQualificationError( - f"atlas structure[{index}] has an invalid hierarchy path" - ) - if structure_id in paths: - raise LambadaCoordinateQualificationError( - f"atlas contains duplicate structure ID {structure_id}" - ) - paths[structure_id] = frozenset(raw_path) - try: - annotation_value = tifffile.memmap(annotation_path, mode="r") - except (OSError, ValueError) as error: - raise LambadaCoordinateQualificationError( - "atlas annotation TIFF must be directly memory-mappable" - ) from error - annotation = np.asarray(annotation_value) - if annotation.shape != ATLAS_SHAPE_ASR or annotation.dtype != np.dtype(np.uint32): - raise LambadaCoordinateQualificationError( - "atlas annotation shape or dtype does not match Allen 25 um v1.2" - ) - annotation.setflags(write=False) - return AtlasEvidence( - annotation=cast(NDArray[np.uint32], annotation), - structure_paths=paths, - identity_report={ - "atlasKey": "allen_mouse_25um", - "atlasPackageVersion": "1.2", - "orientation": "asr", - "resolutionUm": [25, 25, 25], - "shapeVoxels": list(ATLAS_SHAPE_ASR), - "annotation": annotation_identity, - "structures": structures_identity, - "metadata": metadata_identity, - }, - ) - - -def _true_blocks(mask: NDArray[np.bool_]) -> list[tuple[int, int]]: - padded = np.empty(mask.size + 2, dtype=np.bool_) - padded[0] = False - padded[-1] = False - padded[1:-1] = mask - transitions = np.diff(padded.astype(np.int8, copy=False)) - starts = np.flatnonzero(transitions == 1) - ends = np.flatnonzero(transitions == -1) - return [(int(start), int(end)) for start, end in zip(starts, ends, strict=True)] - - -def rerun_retained_major_vessel_selection(source_path: str | Path) -> RetainedSourceSelection: - """Repeat the exact v1 pointwise selection and match the bundled bytes.""" - - path = Path(source_path) - coordinates = np.memmap( - path, - mode="r", - dtype=" edge_indices[:, 0])) - or not bool(np.all(edge_indices[1:, 0] == edge_indices[:-1, 1])) - ): - raise LambadaCoordinateQualificationError("source edge geometry ranges are not contiguous") - edge_radii = np.memmap( - path, - mode="r", - dtype="= MINIMUM_RADIUS_ATLAS_VOXEL) - if candidate_edges.size != EXPECTED_CANDIDATE_EDGE_COUNT: - raise LambadaCoordinateQualificationError("source candidate edge count changed") - - point_chunks: list[NDArray[np.float64]] = [] - source_index_chunks: list[NDArray[np.int64]] = [] - radius_chunks: list[NDArray[np.float32]] = [] - annotation_chunks: list[NDArray[np.int32]] = [] - source_edges: list[int] = [] - run_offsets = [0] - qualifying_in_bounds_points = 0 - edges_with_runs: set[int] = set() - shape = np.asarray(ATLAS_SHAPE_CLEARMAP, dtype=np.float64) - for edge_value in candidate_edges: - edge_index = int(edge_value) - edge_start = int(edge_indices[edge_index, 0]) - edge_end = int(edge_indices[edge_index, 1]) - edge_coordinates = np.asarray(coordinates[edge_start:edge_end]) - edge_radius_values = np.asarray(radii[edge_start:edge_end]) - qualifying = np.all(np.isfinite(edge_coordinates), axis=1) - qualifying &= np.isfinite(edge_radius_values) - qualifying &= np.all(edge_coordinates >= 0.0, axis=1) - qualifying &= np.all(edge_coordinates < shape, axis=1) - qualifying &= edge_radius_values >= MINIMUM_RADIUS_ATLAS_VOXEL - qualifying_in_bounds_points += int(np.count_nonzero(qualifying)) - for block_start, block_end in _true_blocks(qualifying): - if block_end - block_start < 2: - continue - source_start = edge_start + block_start - source_end = edge_start + block_end - source_slice = slice(source_start, source_end) - point_chunks.append( - np.ascontiguousarray(np.asarray(coordinates[source_slice], dtype=np.float64)) - ) - source_index_chunks.append(np.arange(source_start, source_end, dtype=np.int64)) - radius_chunks.append( - np.ascontiguousarray( - ( - np.asarray(radii[source_slice], dtype=np.float64) * ATLAS_VOXEL_SIZE_UM - ).astype(np.float32) - ) - ) - source_annotations = np.asarray(annotations[source_slice]) - if source_annotations.size and ( - int(np.min(source_annotations)) < np.iinfo(np.int32).min - or int(np.max(source_annotations)) > np.iinfo(np.int32).max - ): - raise LambadaCoordinateQualificationError("source annotation ID does not fit int32") - annotation_chunks.append( - np.ascontiguousarray(source_annotations.astype(np.int32, copy=False)) - ) - source_edges.append(edge_index) - edges_with_runs.add(edge_index) - run_offsets.append(run_offsets[-1] + block_end - block_start) - selection = RetainedSourceSelection( - points_clearmap_voxel_f64=np.ascontiguousarray(np.concatenate(point_chunks)), - source_geometry_indices_i64=np.ascontiguousarray(np.concatenate(source_index_chunks)), - radii_um_f32=np.ascontiguousarray(np.concatenate(radius_chunks)), - source_annotation_ids_i32=np.ascontiguousarray(np.concatenate(annotation_chunks)), - run_offsets_i64=np.asarray(run_offsets, dtype=np.int64), - source_edge_indices_i32=np.asarray(source_edges, dtype=np.int32), - qualifying_in_bounds_points=qualifying_in_bounds_points, - source_edges_with_runs=len(edges_with_runs), - ) - expected_counts = ( - EXPECTED_QUALIFYING_IN_BOUNDS_POINT_COUNT, - EXPECTED_RUN_POINT_COUNT, - EXPECTED_SEGMENT_COUNT, - EXPECTED_RUN_COUNT, - EXPECTED_SOURCE_EDGES_WITH_RUNS, - ) - observed_counts = ( - selection.qualifying_in_bounds_points, - int(selection.points_clearmap_voxel_f64.shape[0]), - int(selection.points_clearmap_voxel_f64.shape[0] - len(source_edges)), - len(source_edges), - selection.source_edges_with_runs, - ) - if observed_counts != expected_counts: - raise LambadaCoordinateQualificationError( - f"retained selection counts changed: {observed_counts}" - ) - - bundled_graph = load_lambada_major_vessels() - expected_points_asr_um = np.ascontiguousarray( - selection.points_clearmap_voxel_f64[:, (1, 0, 2)].astype(np.float32) - * np.float32(ATLAS_VOXEL_SIZE_UM) - ) - exact_matches = ( - np.array_equal(expected_points_asr_um, bundled_graph.points_asr_um) - and np.array_equal(selection.radii_um_f32, bundled_graph.radii_um) - and np.array_equal( - selection.source_annotation_ids_i32, - bundled_graph.source_annotation_ids, - ) - and np.array_equal(selection.run_offsets_i64, bundled_graph.run_offsets) - and np.array_equal( - selection.source_edge_indices_i32, - bundled_graph.source_edge_indices, - ) - ) - if not exact_matches: - raise LambadaCoordinateQualificationError( - "rerun selection does not exactly match the integrity-checked bundled asset" - ) - return selection - - -def _rate_report(numerator: int, denominator: int) -> Mapping[str, object]: - decimal = "0.000000000" if denominator == 0 else f"{numerator / denominator:.9f}" - return { - "numerator": numerator, - "denominator": denominator, - "decimal": decimal, - } - - -def _score_fraction(score: Mapping[str, object]) -> Fraction: - raw = cast(Mapping[str, object], score["ancestorAgreement"]) - numerator = cast(int, raw["numerator"]) - denominator = cast(int, raw["denominator"]) - return Fraction(numerator, denominator) if denominator else Fraction(0, 1) - - -def score_annotation_orientations( - points_clearmap_voxel: NDArray[np.generic], - source_annotation_ids: NDArray[np.generic], - atlas_annotation_asr: NDArray[np.generic], - structure_paths: Mapping[int, frozenset[int]], -) -> list[Mapping[str, object]]: - """Score all eight sign combinations using source-ancestor agreement.""" - - points = np.asarray(points_clearmap_voxel, dtype=np.float64) - source_ids = np.asarray(source_annotation_ids, dtype=np.int64) - atlas = np.asarray(atlas_annotation_asr) - if points.ndim != 2 or points.shape[1:] != (3,) or source_ids.shape != points.shape[:1]: - raise LambadaCoordinateQualificationError("orientation score inputs have invalid shapes") - if atlas.shape != ATLAS_SHAPE_ASR or atlas.dtype.kind not in "iu": - raise LambadaCoordinateQualificationError("orientation atlas has invalid shape or dtype") - known_ids = frozenset(structure_paths) - source_known = np.fromiter( - (int(value) in known_ids for value in source_ids), - dtype=np.bool_, - count=source_ids.size, - ) - base_asr = points[:, (1, 0, 2)] - shape = np.asarray(ATLAS_SHAPE_ASR, dtype=np.float64) - scores: list[Mapping[str, object]] = [] - for flip_ap in (False, True): - for flip_dv in (False, True): - for flip_ml in (False, True): - flips = (flip_ap, flip_dv, flip_ml) - transformed = base_asr.copy() - for axis, flip in enumerate(flips): - if flip: - transformed[:, axis] = shape[axis] - 1.0 - transformed[:, axis] - indices = transformed.astype(np.int64) - if bool(np.any(indices < 0)) or bool( - np.any(indices >= np.asarray(ATLAS_SHAPE_ASR, dtype=np.int64)) - ): - raise LambadaCoordinateQualificationError( - "retained point left atlas bounds during sign scoring" - ) - sampled_ids = np.asarray( - atlas[indices[:, 0], indices[:, 1], indices[:, 2]], - dtype=np.int64, - ) - sampled_known = np.fromiter( - (int(value) in known_ids for value in sampled_ids), - dtype=np.bool_, - count=sampled_ids.size, - ) - evaluable = source_known & sampled_known - evaluable_indices = np.flatnonzero(evaluable) - compatible_count = sum( - int(source_ids[index]) in structure_paths[int(sampled_ids[index])] - for index in evaluable_indices - ) - evaluable_count = int(evaluable_indices.size) - exact_count = int(np.count_nonzero(source_ids[evaluable] == sampled_ids[evaluable])) - label = "_".join( - ( - f"AP_{'FLIP' if flip_ap else 'KEEP'}", - f"DV_{'FLIP' if flip_dv else 'KEEP'}", - f"ML_{'FLIP' if flip_ml else 'KEEP'}", - ) - ) - scores.append( - { - "label": label, - "flipAP": flip_ap, - "flipDV": flip_dv, - "flipML": flip_ml, - "selectedPointCount": int(points.shape[0]), - "evaluablePointCount": evaluable_count, - "evaluableFraction": _rate_report(evaluable_count, int(points.shape[0])), - "ancestorAgreement": _rate_report( - compatible_count, - evaluable_count, - ), - "exactLeafAgreement": _rate_report(exact_count, evaluable_count), - } - ) - return scores - - -def score_hemisphere_laterality( - points_clearmap_voxel: NDArray[np.generic], - source_hemisphere_labels: NDArray[np.generic], -) -> Mapping[str, object]: - """Test whether ClearMap axis 2 already follows right-origin ASR ML.""" - - points = np.asarray(points_clearmap_voxel, dtype=np.float64) - labels = np.asarray(source_hemisphere_labels) - if points.ndim != 2 or points.shape[1:] != (3,) or labels.shape != points.shape[:1]: - raise LambadaCoordinateQualificationError("hemisphere score inputs have invalid shapes") - if labels.dtype.kind not in "iu": - raise LambadaCoordinateQualificationError("source hemisphere labels must be integral") - labels_i64 = np.asarray(labels, dtype=np.int64) - allowed = np.isin( - labels_i64, - [CLEARMAP_LEFT_HEMISPHERE_VALUE, CLEARMAP_RIGHT_HEMISPHERE_VALUE], - ) - if not bool(np.all(allowed)): - invalid = np.unique(labels_i64[~allowed]) - raise LambadaCoordinateQualificationError( - f"source hemisphere labels contain unsupported values {invalid[:10].tolist()}" - ) - ml_coordinates = points[:, 2] - ml_indices = ml_coordinates.astype(np.int64) - if bool(np.any(ml_indices < 0)) or bool(np.any(ml_indices >= ATLAS_SHAPE_ASR[2])): - raise LambadaCoordinateQualificationError("source ML point is outside atlas bounds") - expected_keep = np.where( - ml_indices < ML_MIDLINE_VOXEL, - CLEARMAP_RIGHT_HEMISPHERE_VALUE, - CLEARMAP_LEFT_HEMISPHERE_VALUE, - ) - flipped_ml_indices = (ATLAS_SHAPE_ASR[2] - 1.0 - ml_coordinates).astype(np.int64) - expected_flip = np.where( - flipped_ml_indices < ML_MIDLINE_VOXEL, - CLEARMAP_RIGHT_HEMISPHERE_VALUE, - CLEARMAP_LEFT_HEMISPHERE_VALUE, - ) - discriminating = expected_keep != expected_flip - discriminating_count = int(np.count_nonzero(discriminating)) - if discriminating_count == 0: - raise LambadaCoordinateQualificationError( - "retained hemisphere evidence has no points that distinguish ML signs" - ) - keep_count = int(np.count_nonzero(labels_i64[discriminating] == expected_keep[discriminating])) - flip_count = int(np.count_nonzero(labels_i64[discriminating] == expected_flip[discriminating])) - return { - "sourceLabelSemantics": { - "0": "LH", - "255": "RH", - "evidence": ( - "ClearMap Analysis/Statistics/data_frame_operations.py maps 0 to LH and 255 to RH" - ), - "sourceUrl": CLEARMAP_HEMISPHERE_SEMANTICS_URL, - }, - "brainGlobeMLSemantics": "ASR ML origin is right and increasing indices move left", - "midlineIndex": ML_MIDLINE_VOXEL, - "selectedPointCount": int(points.shape[0]), - "discriminatingPointCount": discriminating_count, - "discriminatingPointFraction": _rate_report(discriminating_count, int(points.shape[0])), - "nonDiscriminatingMidlinePointCount": int(points.shape[0]) - discriminating_count, - "keepAxis2Agreement": _rate_report(keep_count, discriminating_count), - "flipAxis2Agreement": _rate_report(flip_count, discriminating_count), - } - - -def _criteria_report() -> Mapping[str, object]: - return { - "minimumAnnotationEvaluablePoints": MIN_ANNOTATION_EVALUABLE_POINTS, - "minimumAnnotationEvaluableFraction": str(MIN_ANNOTATION_EVALUABLE_FRACTION), - "minimumAnnotationAncestorAgreement": str(MIN_ANNOTATION_ANCESTOR_AGREEMENT), - "minimumExpectedGlobalLead": str(MIN_EXPECTED_GLOBAL_LEAD), - "minimumAPDVWinnerMargin": str(MIN_AP_DV_WINNER_MARGIN), - "minimumHemisphereDiscriminatingFraction": str(MIN_HEMISPHERE_DISCRIMINATING_FRACTION), - "minimumHemisphereAgreement": str(MIN_HEMISPHERE_AGREEMENT), - "minimumHemisphereWinnerMargin": str(MIN_HEMISPHERE_WINNER_MARGIN), - "requiredSourceHemisphereProperty": { - "kind": "graph", - "name": HEMISPHERE_PROPERTY_NAME, - "typeIndex": 14, - "cardinality": 1, - }, - "requireWholeBrainCoverageOrExplicitQualifiedMirroring": True, - "expectedOrientation": "AP_KEEP_DV_KEEP_ML_KEEP", - "criteriaAreRuntimeConfigurable": False, - } - - -def _evaluate_annotation_checks( - orientation_scores: Sequence[Mapping[str, object]], -) -> tuple[list[Mapping[str, object]], Mapping[str, object]]: - expected = next( - score for score in orientation_scores if score["label"] == "AP_KEEP_DV_KEEP_ML_KEEP" - ) - ordered = sorted( - orientation_scores, - key=lambda score: (_score_fraction(score), str(score["label"])), - reverse=True, - ) - global_winner = ordered[0] - global_runner_up = ordered[1] - expected_fraction = _score_fraction(expected) - runner_fraction = _score_fraction(global_runner_up) - expected_evaluable = int(cast(int, expected["evaluablePointCount"])) - selected_count = int(cast(int, expected["selectedPointCount"])) - - ap_dv_best: dict[tuple[bool, bool], Mapping[str, object]] = {} - for score in orientation_scores: - key = (bool(score["flipAP"]), bool(score["flipDV"])) - previous = ap_dv_best.get(key) - if previous is None or _score_fraction(score) > _score_fraction(previous): - ap_dv_best[key] = score - expected_ap_dv = ap_dv_best[(False, False)] - ap_dv_runner = max( - (score for key, score in ap_dv_best.items() if key != (False, False)), - key=_score_fraction, - ) - ap_dv_margin = _score_fraction(expected_ap_dv) - _score_fraction(ap_dv_runner) - - check_values = [ - ( - "annotation-evaluable-point-count", - expected_evaluable >= MIN_ANNOTATION_EVALUABLE_POINTS, - str(expected_evaluable), - f">={MIN_ANNOTATION_EVALUABLE_POINTS}", - ), - ( - "annotation-evaluable-fraction", - Fraction(expected_evaluable, selected_count) >= MIN_ANNOTATION_EVALUABLE_FRACTION, - str(Fraction(expected_evaluable, selected_count)), - f">={MIN_ANNOTATION_EVALUABLE_FRACTION}", - ), - ( - "annotation-ancestor-agreement", - expected_fraction >= MIN_ANNOTATION_ANCESTOR_AGREEMENT, - str(expected_fraction), - f">={MIN_ANNOTATION_ANCESTOR_AGREEMENT}", - ), - ( - "expected-orientation-is-global-winner", - global_winner["label"] == expected["label"], - str(global_winner["label"]), - str(expected["label"]), - ), - ( - "expected-orientation-global-lead", - expected_fraction - runner_fraction >= MIN_EXPECTED_GLOBAL_LEAD, - str(expected_fraction - runner_fraction), - f">={MIN_EXPECTED_GLOBAL_LEAD}", - ), - ( - "ap-dv-sign-margin", - ap_dv_margin >= MIN_AP_DV_WINNER_MARGIN, - str(ap_dv_margin), - f">={MIN_AP_DV_WINNER_MARGIN}", - ), - ] - checks: list[Mapping[str, object]] = [ - {"id": identifier, "passed": passed, "observed": observed, "required": required} - for identifier, passed, observed, required in check_values - ] - summary: Mapping[str, object] = { - "globalWinner": global_winner["label"], - "globalRunnerUp": global_runner_up["label"], - "expectedGlobalLead": str(expected_fraction - runner_fraction), - "bestAPDVAlternative": ap_dv_runner["label"], - "apDvWinnerMargin": str(ap_dv_margin), - } - return checks, summary - - -def _evaluate_hemisphere_checks( - hemisphere_score: Mapping[str, object], -) -> tuple[list[Mapping[str, object]], Mapping[str, object]]: - keep_raw = cast(Mapping[str, object], hemisphere_score["keepAxis2Agreement"]) - flip_raw = cast(Mapping[str, object], hemisphere_score["flipAxis2Agreement"]) - keep_fraction = Fraction( - cast(int, keep_raw["numerator"]), - cast(int, keep_raw["denominator"]), - ) - flip_fraction = Fraction( - cast(int, flip_raw["numerator"]), - cast(int, flip_raw["denominator"]), - ) - selected_count = cast(int, hemisphere_score["selectedPointCount"]) - discriminating_count = cast(int, hemisphere_score["discriminatingPointCount"]) - check_values = [ - ( - "source-hemisphere-property-present", - True, - f"1 graph:{HEMISPHERE_PROPERTY_NAME} typeIndex=14", - f"1 graph:{HEMISPHERE_PROPERTY_NAME} typeIndex=14", - ), - ( - "ml-hemisphere-discriminating-fraction", - Fraction(discriminating_count, selected_count) - >= MIN_HEMISPHERE_DISCRIMINATING_FRACTION, - str(Fraction(discriminating_count, selected_count)), - f">={MIN_HEMISPHERE_DISCRIMINATING_FRACTION}", - ), - ( - "ml-hemisphere-agreement", - keep_fraction >= MIN_HEMISPHERE_AGREEMENT, - str(keep_fraction), - f">={MIN_HEMISPHERE_AGREEMENT}", - ), - ( - "ml-hemisphere-winner-margin", - keep_fraction - flip_fraction >= MIN_HEMISPHERE_WINNER_MARGIN, - str(keep_fraction - flip_fraction), - f">={MIN_HEMISPHERE_WINNER_MARGIN}", - ), - ] - checks: list[Mapping[str, object]] = [ - {"id": identifier, "passed": passed, "observed": observed, "required": required} - for identifier, passed, observed, required in check_values - ] - return checks, {"hemisphereWinnerMargin": str(keep_fraction - flip_fraction)} - - -def _evaluate_checks( - orientation_scores: Sequence[Mapping[str, object]], - hemisphere_score: Mapping[str, object], -) -> tuple[list[Mapping[str, object]], Mapping[str, object]]: - annotation_checks, annotation_summary = _evaluate_annotation_checks(orientation_scores) - hemisphere_checks, hemisphere_summary = _evaluate_hemisphere_checks(hemisphere_score) - return annotation_checks + hemisphere_checks, { - **annotation_summary, - **hemisphere_summary, - } - - -def _property_record_report(record: GTPropertyRecord) -> Mapping[str, object]: - return { - "kind": record.kind, - "name": record.name, - "typeIndex": record.type_index, - "itemCount": record.item_count, - "recordOffset": record.record_offset, - "valuesOffset": record.values_offset, - "valuesEndOffset": record.values_end_offset, - "objectPayloadOffset": record.object_payload_offset, - "objectPayloadSizeBytes": record.object_payload_size_bytes, - } - - -def _graph_tool_report( - inventory: GTFileInventory, - *, - hemisphere_property: Mapping[str, object] | None, -) -> Mapping[str, object]: - return { - "formatDocumentation": GT_FORMAT_DOCUMENTATION_URL, - "byteOrder": inventory.byte_order, - "directed": inventory.directed, - "vertexCount": inventory.vertex_count, - "edgeCount": inventory.edge_count, - "propertyCount": len(inventory.properties), - "propertyInventory": [_property_record_report(record) for record in inventory.properties], - "hemisphereProperty": hemisphere_property, - } - - -def _retained_selection_report(selection: RetainedSourceSelection) -> Mapping[str, object]: - return { - "assetSha256": ASSET_SHA256, - "exactBundledAssetMatch": True, - "candidateEdges": EXPECTED_CANDIDATE_EDGE_COUNT, - "qualifyingInBoundsPoints": selection.qualifying_in_bounds_points, - "outputPoints": int(selection.points_clearmap_voxel_f64.shape[0]), - "outputSegments": int( - selection.points_clearmap_voxel_f64.shape[0] - - selection.source_edge_indices_i32.shape[0] - ), - "outputRuns": int(selection.source_edge_indices_i32.shape[0]), - "sourceEdgesWithRuns": selection.source_edges_with_runs, - } - - -def assess_pinned_source_coverage( - source_path: str | Path, - inventory: GTFileInventory, - selection: RetainedSourceSelection, -) -> tuple[Mapping[str, object], Mapping[str, object]]: - """Measure atlas-space coverage without treating numerical sides as laterality labels.""" - - matches = [ - record - for record in inventory.properties - if record.kind == "vertex" and record.name == "coordinates_atlas" - ] - if len(matches) != 1 or matches[0].type_index != 11: - raise LambadaCoordinateQualificationError( - "pinned graph must contain one vector vertex coordinates_atlas property" - ) - record = matches[0] - record_dtype = np.dtype([("length", "[3] records" - ) - records = np.memmap( - Path(source_path), - mode="r", - dtype=record_dtype, - offset=record.values_offset, - shape=(record.item_count,), - ) - if not bool(np.all(records["length"] == 3)): - raise LambadaCoordinateQualificationError( - "vertex coordinates_atlas contains a non-three-dimensional vector" - ) - coordinates = records["values"] - if not bool(np.all(np.isfinite(coordinates))): - raise LambadaCoordinateQualificationError( - "vertex coordinates_atlas contains a non-finite value" - ) - minimum = np.min(coordinates, axis=0) - maximum = np.max(coordinates, axis=0) - ml_values = coordinates[:, 2] - low_side_count = int(np.count_nonzero(ml_values < ML_MIDLINE_VOXEL)) - high_side_count = int(ml_values.size - low_side_count) - retained_ml = selection.points_clearmap_voxel_f64[:, 2] - retained_low_count = int(np.count_nonzero(retained_ml < ML_MIDLINE_VOXEL)) - retained_high_count = int(retained_ml.size - retained_low_count) - graph_level_names = [ - property_record.name - for property_record in inventory.properties - if property_record.kind == "graph" - ] - report: Mapping[str, object] = { - "status": "unqualified-for-whole-brain-use", - "reasonCode": HEMISPHERE_COVERAGE_REASON_CODE, - "primarySource": { - "url": ZENODO_DATASET_RECORD_URL, - "finding": ( - "The official dataset description identifies the specimens as mouse brain " - "hemispheres, not whole-brain vascular graphs." - ), - }, - "atlasMidlineIndex": ML_MIDLINE_VOXEL, - "vertexCoordinatesAtlas": { - "pointCount": int(coordinates.shape[0]), - "minimumByClearMapAxis": [f"{float(value):.9f}" for value in minimum], - "maximumByClearMapAxis": [f"{float(value):.9f}" for value in maximum], - "axis2BelowMidlineCount": low_side_count, - "axis2AtOrAboveMidlineCount": high_side_count, - }, - "retainedMajorVesselCoordinatesAtlas": { - "pointCount": int(retained_ml.size), - "axis2Minimum": f"{float(np.min(retained_ml)):.9f}", - "axis2Maximum": f"{float(np.max(retained_ml)):.9f}", - "axis2BelowMidlineCount": retained_low_count, - "axis2AtOrAboveMidlineCount": retained_high_count, - }, - "graphMetadataAssessment": { - "graphLevelPropertyNames": graph_level_names, - "propertiesBindingBiologicalLaterality": [], - "propertiesBindingExactWorkflowCommit": [], - "propertiesContainingSourceOrientationOrAlignmentTransform": [], - "conclusion": ( - "The graph stores raw, atlas, and MRI coordinates plus a raw image shape, " - "but no source-orientation label, alignment transform, resampling metadata, " - "workflow commit, or specimen configuration that binds ML sign." - ), - }, - "numericalBothSidesDoNotProveWholeBrain": True, - "bilateralMirroringQualified": False, - "bilateralMirroringReason": ( - "Neither the exact graph metadata nor the dataset record defines a reviewed " - "mirror operation that reconstructs a whole-brain major-vessel reference." - ), - } - check: Mapping[str, object] = { - "id": "source-coverage-whole-brain", - "passed": False, - "observed": "official source declares brain-hemisphere specimens", - "required": "whole-brain source or explicitly reviewed bilateral reconstruction", - } - return report, check - - -def _scope_limitations() -> list[str]: - return [ - ( - "This report qualifies only axis permutation, AP/DV signs, and ML laterality " - "for the exact pinned files when every check passes." - ), - ( - "It does not bound atlas-registration error, tissue distortion, biological " - "variation, or omitted vessels." - ), - ( - "It does not make the fixed-tissue reference subject-specific or validate it " - "for surgical navigation." - ), - ( - "It does not establish artery/vein identity or clearance when no " - "loaded-geometry conflict is found." - ), - ] - - -def qualify_pinned_lambada_coordinates( - source_graph: str | Path, - atlas_directory: str | Path, -) -> Mapping[str, object]: - """Run the full qualification and return a canonicalizable report mapping.""" - - source_identity = verify_pinned_graph_identity(source_graph) - inventory = scan_gt_file(source_graph) - if ( - inventory.byte_order != "little" - or inventory.vertex_count != SOURCE_VERTEX_COUNT - or inventory.edge_count != SOURCE_EDGE_COUNT - ): - raise LambadaCoordinateQualificationError( - "graph-tool topology inventory does not match the pinned source" - ) - selection = rerun_retained_major_vessel_selection(source_graph) - atlas = load_pinned_atlas_evidence(atlas_directory) - orientation_scores = score_annotation_orientations( - selection.points_clearmap_voxel_f64, - selection.source_annotation_ids_i32, - atlas.annotation, - atlas.structure_paths, - ) - annotation_checks, annotation_summary = _evaluate_annotation_checks(orientation_scores) - source_coverage, source_coverage_check = assess_pinned_source_coverage( - source_graph, - inventory, - selection, - ) - common_report: dict[str, object] = { - "schemaVersion": QUALIFICATION_SCHEMA_VERSION, - "algorithmVersion": QUALIFICATION_ALGORITHM_VERSION, - "source": source_identity, - "clearMapImplementationEvidence": { - "repository": CLEARMAP_REPOSITORY_URL, - "commit": CLEARMAP_EVIDENCE_COMMIT, - "annotationAndHemispherePropertyCreation": CLEARMAP_ANNOTATION_SOURCE_URL, - "graphLevelEdgeGeometryStorage": CLEARMAP_GRAPH_STORAGE_URL, - "hemisphereLabelSemantics": CLEARMAP_HEMISPHERE_SEMANTICS_URL, - "evidenceScope": ( - "Context only. Generic repository code does not bind a workflow commit, " - "configuration, or transform to the exact P60_606 artifact." - ), - "exactArtifactWorkflowCommit": None, - "exactArtifactSpecimenConfiguration": None, - }, - "atlas": atlas.identity_report, - "retainedSelection": _retained_selection_report(selection), - "sourceCoverage": source_coverage, - "predeclaredAcceptanceCriteria": _criteria_report(), - "annotationOrientation": { - "method": ( - "For each of all eight AP/DV/ML sign combinations, truncate retained " - "continuous voxels exactly as ClearMap label_points does, sample the pinned " - "Allen annotation, and count the source annotation when it is an ancestor " - "of the sampled Allen structure. AP/DV are judged separately from bilateral " - "ML symmetry." - ), - "scores": orientation_scores, - }, - "scopeLimitations": _scope_limitations(), - } - hemisphere_records = [ - record - for record in inventory.properties - if record.kind == "graph" and record.name == HEMISPHERE_PROPERTY_NAME - ] - if not hemisphere_records: - property_check: Mapping[str, object] = { - "id": "source-hemisphere-property-present", - "passed": False, - "observed": "0 matching properties", - "required": f"1 graph:{HEMISPHERE_PROPERTY_NAME} typeIndex=14", - } - return { - **common_report, - "status": "rejected", - "graphTool": _graph_tool_report(inventory, hemisphere_property=None), - "hemisphereLaterality": { - "status": "unavailable", - "reasonCode": MISSING_HEMISPHERE_REASON_CODE, - "reason": ( - "The exact pinned graph has no graph-level edge_geometry_hemisphere " - "property. Its bilateral annotation cannot distinguish the two ML signs, " - "and generic pipeline defaults are not specimen evidence." - ), - "observedMatchingPropertyCount": 0, - "requiredProperty": cast( - Mapping[str, object], - _criteria_report()["requiredSourceHemisphereProperty"], - ), - "genericPipelineEvidenceSubstituted": False, - }, - "checks": [*annotation_checks, property_check, source_coverage_check], - "decision": { - **annotation_summary, - "hemisphereWinnerMargin": None, - "blockingReasons": [ - MISSING_HEMISPHERE_REASON_CODE, - HEMISPHERE_COVERAGE_REASON_CODE, - ], - "qualifiedMapping": None, - }, - } - hemisphere_record = hemisphere_records[0] - if hemisphere_record.type_index != 14: - invalid_type_reason = "SOURCE_HEMISPHERE_PROPERTY_INVALID_TYPE" - property_check = { - "id": "source-hemisphere-property-present", - "passed": False, - "observed": ( - f"1 graph:{HEMISPHERE_PROPERTY_NAME} typeIndex={hemisphere_record.type_index}" - ), - "required": f"1 graph:{HEMISPHERE_PROPERTY_NAME} typeIndex=14", - } - return { - **common_report, - "status": "rejected", - "graphTool": _graph_tool_report( - inventory, - hemisphere_property=_property_record_report(hemisphere_record), - ), - "hemisphereLaterality": { - "status": "unavailable", - "reasonCode": invalid_type_reason, - "reason": ( - "The exact pinned graph's edge_geometry_hemisphere property is not a " - "graph-level python::object and cannot be decoded as pinned evidence." - ), - "observedMatchingPropertyCount": 1, - "requiredProperty": cast( - Mapping[str, object], - _criteria_report()["requiredSourceHemisphereProperty"], - ), - "genericPipelineEvidenceSubstituted": False, - }, - "checks": [*annotation_checks, property_check, source_coverage_check], - "decision": { - **annotation_summary, - "hemisphereWinnerMargin": None, - "blockingReasons": [ - invalid_type_reason, - HEMISPHERE_COVERAGE_REASON_CODE, - ], - "qualifiedMapping": None, - }, - } - hemispheres = load_graph_numpy_property( - source_graph, - hemisphere_record, - expected_shape=(SOURCE_GEOMETRY_POINT_COUNT,), - ) - unique_hemispheres = {int(value) for value in np.unique(hemispheres)} - if unique_hemispheres != { - CLEARMAP_LEFT_HEMISPHERE_VALUE, - CLEARMAP_RIGHT_HEMISPHERE_VALUE, - }: - raise LambadaCoordinateQualificationError( - f"full source hemisphere inventory changed: {sorted(unique_hemispheres)}" - ) - selected_hemispheres = np.ascontiguousarray(hemispheres[selection.source_geometry_indices_i64]) - del hemispheres - hemisphere_score = score_hemisphere_laterality( - selection.points_clearmap_voxel_f64, - selected_hemispheres, - ) - checks, decision_summary = _evaluate_checks(orientation_scores, hemisphere_score) - checks.append(source_coverage_check) - passed = all(bool(check["passed"]) for check in checks) - return { - **common_report, - "status": "qualified" if passed else "rejected", - "graphTool": _graph_tool_report( - inventory, - hemisphere_property={ - **_property_record_report(hemisphere_record), - "name": hemisphere_record.name, - "kind": hemisphere_record.kind, - "typeIndex": hemisphere_record.type_index, - "recordOffset": hemisphere_record.record_offset, - "picklePayloadOffset": hemisphere_record.object_payload_offset, - "picklePayloadSizeBytes": hemisphere_record.object_payload_size_bytes, - "decodedShape": [SOURCE_GEOMETRY_POINT_COUNT], - "decodedLabels": sorted(unique_hemispheres), - "restrictedUnpickler": True, - }, - ), - "hemisphereLaterality": hemisphere_score, - "checks": checks, - "decision": { - **decision_summary, - "blockingReasons": ([] if passed else [HEMISPHERE_COVERAGE_REASON_CODE]), - "qualifiedMapping": ( - { - "sourceOrder": ["ClearMap axis 0", "ClearMap axis 1", "ClearMap axis 2"], - "brainGlobeASROrder": ["AP", "DV", "ML"], - "permutation": [1, 0, 2], - "axisFlips": [False, False, False], - "mlOrigin": "right", - "mlIncreasingDirection": "left", - } - if passed - else None - ), - }, - } - - -def canonical_report_bytes(report: Mapping[str, object]) -> bytes: - """Serialize a report with stable key ordering and no environment paths.""" - - return ( - json.dumps( - report, - ensure_ascii=True, - allow_nan=False, - sort_keys=True, - separators=(",", ":"), - ) - + "\n" - ).encode("utf-8") - - -def write_canonical_report(path: str | Path, report: Mapping[str, object]) -> str: - """Atomically write canonical JSON and return its SHA-256 digest.""" - - output = Path(path) - output.parent.mkdir(parents=True, exist_ok=True) - payload = canonical_report_bytes(report) - descriptor, temporary_name = tempfile.mkstemp( - prefix=f".{output.name}.", - suffix=".tmp", - dir=output.parent, - ) - temporary = Path(temporary_name) - try: - with os.fdopen(descriptor, "wb") as stream: - stream.write(payload) - stream.flush() - os.fsync(stream.fileno()) - temporary.replace(output) - except BaseException: - temporary.unlink(missing_ok=True) - raise - return hashlib.sha256(payload).hexdigest() diff --git a/src/mouse_brain_planner/vasculature/lambada_major_vessels.py b/src/mouse_brain_planner/vasculature/lambada_major_vessels.py deleted file mode 100644 index d73251b..0000000 --- a/src/mouse_brain_planner/vasculature/lambada_major_vessels.py +++ /dev/null @@ -1,985 +0,0 @@ -"""Pinned LAMBADA P60 vessel geometry and its reproducible compact extractor. - -The bundled asset is intentionally a small, display-only derivative of one -atlas-registered, fixed-tissue graph. It contains maximal consecutive runs -whose point radii are at least 15 micrometres and whose coordinates are inside -the reviewed Allen 25 micrometre array bounds. It is not a clearance map and -must not be interpreted as subject-specific surgical vasculature. -""" - -from __future__ import annotations - -import hashlib -import io -import json -import math -import os -import tempfile -import zipfile -from collections.abc import Mapping, Sequence -from dataclasses import dataclass -from itertools import pairwise -from pathlib import Path -from typing import Final, cast - -import numpy as np -from numpy.typing import NDArray - -ASSET_SCHEMA_VERSION: Final = 1 -EXTRACTION_ALGORITHM_VERSION: Final = "lambada-p60-606-major-runs-v1" -ASSET_FILENAME: Final = "lambada_p60_606_major_vessels_v1.npz" -MANIFEST_FILENAME: Final = f"{ASSET_FILENAME}.manifest.json" -ASSET_SIZE_BYTES: Final = 814_393 -ASSET_SHA256: Final = "fb2344e845e604be3424bd63f4222d273eafba34db0df2eaff32f4400fa9afec" - -SOURCE_FILENAME: Final = "606_graph_2024-12-03.gt" -SOURCE_SIZE_BYTES: Final = 12_282_574_483 -SOURCE_SHA256: Final = "c2568cfbecd3f3eb720519be9d042f0cb41606741b8dd2f018bad1c54d36ef85" -SOURCE_MD5: Final = "b0bedc97ed2c6e00a41565649b3dd86b" -SOURCE_ARCHIVE_FILENAME: Final = "P60_606_graph_2024-12-03.gt.7z" -SOURCE_ARCHIVE_SIZE_BYTES: Final = 5_050_194_723 -SOURCE_ARCHIVE_SHA256: Final = "cc6d252ee57154f5bc0f06605a703253470a57c210d76e075831effa2098d66f" -SOURCE_ARCHIVE_MD5: Final = "218ed346c6d7dc501301204f811be37f" -SOURCE_RECORD_DOI: Final = "10.5281/zenodo.18876865" -SOURCE_CONCEPT_DOI: Final = "10.5281/zenodo.18876864" -SOURCE_RECORD_URL: Final = "https://zenodo.org/records/18876865" -SOURCE_LICENSE: Final = "CC BY 4.0" -SOURCE_LICENSE_URL: Final = "https://creativecommons.org/licenses/by/4.0/" -SOURCE_PAPER_DOI: Final = "10.1016/j.cell.2026.03.013" -SOURCE_PAPER_URL: Final = "https://www.cell.com/cell/fulltext/S0092-8674(26)00280-1" - -ATLAS_IDENTIFIER: Final = "allen_mouse_25um" -ATLAS_VERSION: Final = "1.2" -ATLAS_SHAPE_CLEARMAP: Final = (320, 528, 456) -ATLAS_SHAPE_ASR: Final = (528, 320, 456) -ATLAS_VOXEL_SIZE_UM: Final = 25.0 -MINIMUM_RADIUS_UM: Final = 15.0 -MINIMUM_DIAMETER_UM: Final = 30.0 -MINIMUM_RADIUS_ATLAS_VOXEL: Final = MINIMUM_RADIUS_UM / ATLAS_VOXEL_SIZE_UM - -# Exact graph-tool property layout for the one SHA-256-pinned extracted source. -# These offsets are never accepted for a file whose byte count and digest differ. -SOURCE_VERTEX_COUNT: Final = 2_231_019 -SOURCE_EDGE_COUNT: Final = 3_301_619 -SOURCE_GEOMETRY_POINT_COUNT: Final = 76_980_908 -COORDINATES_ATLAS_RAW_OFFSET: Final = 2_494_444_615 -RADII_ATLAS_RAW_OFFSET: Final = 4_341_986_607 -ANNOTATIONS_RAW_OFFSET: Final = 4_957_834_070 -EDGE_GEOMETRY_INDICES_PAYLOAD_OFFSET: Final = 12_124_096_701 -EDGE_RADII_ATLAS_RAW_OFFSET: Final = 12_203_335_578 - -EXPECTED_CANDIDATE_EDGE_COUNT: Final = 16_156 -EXPECTED_QUALIFYING_IN_BOUNDS_POINT_COUNT: Final = 78_048 -EXPECTED_RUN_POINT_COUNT: Final = 71_313 -EXPECTED_SEGMENT_COUNT: Final = 59_495 -EXPECTED_RUN_COUNT: Final = 11_818 -EXPECTED_SOURCE_EDGES_WITH_RUNS: Final = 10_907 - -ARRAY_DTYPES: Final[Mapping[str, np.dtype[np.generic]]] = { - "points_asr_voxel_f32": np.dtype(" dict[str, NDArray[np.generic]]: - """Return the fixed on-disk array mapping.""" - - return { - "points_asr_voxel_f32": self.points_asr_voxel_f32, - "radii_um_f32": self.radii_um_f32, - "source_annotation_ids_i32": self.source_annotation_ids_i32, - "run_offsets_i64": self.run_offsets_i64, - "source_edge_indices_i32": self.source_edge_indices_i32, - } - - -@dataclass(frozen=True, slots=True) -class LambadaExtractionReport: - """Deterministic statistics recorded beside the derived asset.""" - - candidate_edges_by_source_edge_max: int - qualifying_in_bounds_points: int - output_points: int - output_segments: int - output_runs: int - source_edges_with_runs: int - selected_path_length_um_source_f64: float - asset_path_length_um_f32: float - - -@dataclass(frozen=True, slots=True) -class LambadaMajorVesselProvenance: - """Attribution and mandatory interpretation limits carried at runtime.""" - - dataset_title: str - authors: tuple[str, ...] - specimen_id: str - record_doi: str - record_url: str - license: str - license_url: str - source_sha256: str - asset_sha256: str - extraction_algorithm_version: str - minimum_radius_um: float - limitations: tuple[str, ...] - - -@dataclass(frozen=True, slots=True, eq=False) -class LambadaMajorVesselGraph: - """Immutable display graph in BrainGlobe physical ASR ``[AP,DV,ML]`` micrometres.""" - - points_asr_um: NDArray[np.float32] - radii_um: NDArray[np.float32] - source_annotation_ids: NDArray[np.int32] - run_offsets: NDArray[np.int64] - source_edge_indices: NDArray[np.int32] - provenance: LambadaMajorVesselProvenance - - @property - def run_count(self) -> int: - """Number of independent polylines.""" - - return int(self.source_edge_indices.shape[0]) - - def run_points_asr_um(self, run_index: int) -> NDArray[np.float32]: - """Return one immutable physical-ASR polyline view.""" - - if isinstance(run_index, bool) or not isinstance(run_index, int): - raise TypeError("run_index must be an integer") - if not 0 <= run_index < self.run_count: - raise IndexError("run_index is outside the vessel graph") - start = int(self.run_offsets[run_index]) - end = int(self.run_offsets[run_index + 1]) - return self.points_asr_um[start:end] - - -def _as_numeric_array( - value: NDArray[np.generic], - *, - name: str, - ndim: int, -) -> NDArray[np.generic]: - array = np.asarray(value) - if array.ndim != ndim: - raise LambadaMajorVesselError(f"{name} must be {ndim}-dimensional") - if array.dtype.kind not in "fiu": - raise LambadaMajorVesselError(f"{name} must be numeric") - return array - - -def extract_major_vessel_runs( - coordinates_clearmap_voxel: NDArray[np.generic], - radii_atlas_voxel: NDArray[np.generic], - source_annotation_ids: NDArray[np.generic], - edge_geometry_indices: NDArray[np.generic], - *, - atlas_shape_clearmap: tuple[int, int, int] = ATLAS_SHAPE_CLEARMAP, - atlas_voxel_size_um: float = ATLAS_VOXEL_SIZE_UM, - minimum_radius_um: float = MINIMUM_RADIUS_UM, -) -> LambadaMajorVesselAssetData: - """Extract maximal in-bounds pointwise-radius runs from graph geometry. - - Each edge range is end-exclusive. Runs shorter than two points have no - drawable segment and are omitted. ClearMap ``[c0,c1,c2]`` coordinates are - permuted to BrainGlobe ASR ``[c1,c0,c2]`` without a half-voxel shift. - """ - - coordinates = _as_numeric_array( - coordinates_clearmap_voxel, - name="coordinates_clearmap_voxel", - ndim=2, - ) - radii = _as_numeric_array(radii_atlas_voxel, name="radii_atlas_voxel", ndim=1) - annotations = _as_numeric_array( - source_annotation_ids, - name="source_annotation_ids", - ndim=1, - ) - edge_indices = _as_numeric_array( - edge_geometry_indices, - name="edge_geometry_indices", - ndim=2, - ) - if coordinates.shape[1:] != (3,): - raise LambadaMajorVesselError("coordinates_clearmap_voxel must have shape [N,3]") - point_count = coordinates.shape[0] - if radii.shape != (point_count,) or annotations.shape != (point_count,): - raise LambadaMajorVesselError("point radii and annotations must have shape [N]") - if edge_indices.shape[1:] != (2,): - raise LambadaMajorVesselError("edge_geometry_indices must have shape [E,2]") - if any( - isinstance(size, bool) or not isinstance(size, int) or size <= 0 - for size in atlas_shape_clearmap - ): - raise LambadaMajorVesselError("atlas_shape_clearmap must contain three positive integers") - if len(atlas_shape_clearmap) != 3: - raise LambadaMajorVesselError("atlas_shape_clearmap must contain exactly three values") - if not math.isfinite(atlas_voxel_size_um) or atlas_voxel_size_um <= 0.0: - raise LambadaMajorVesselError("atlas_voxel_size_um must be finite and positive") - if not math.isfinite(minimum_radius_um) or minimum_radius_um <= 0.0: - raise LambadaMajorVesselError("minimum_radius_um must be finite and positive") - if edge_indices.dtype.kind not in "iu": - raise LambadaMajorVesselError("edge_geometry_indices must contain integers") - if annotations.dtype.kind not in "iu": - raise LambadaMajorVesselError("source_annotation_ids must contain integers") - annotations_i64 = np.asarray(annotations, dtype=np.int64) - if annotations_i64.size and ( - int(np.min(annotations_i64)) < 0 or int(np.max(annotations_i64)) > np.iinfo(np.int32).max - ): - raise LambadaMajorVesselError("source annotation IDs do not fit int32") - - edge_indices_i64 = np.asarray(edge_indices, dtype=np.int64) - if edge_indices_i64.size: - starts = edge_indices_i64[:, 0] - ends = edge_indices_i64[:, 1] - if bool(np.any(starts < 0)) or bool(np.any(ends > point_count)): - raise LambadaMajorVesselError("edge geometry range is outside the point arrays") - if bool(np.any(ends <= starts)): - raise LambadaMajorVesselError("edge geometry ranges must be non-empty") - - point_chunks: list[NDArray[np.float32]] = [] - radius_chunks: list[NDArray[np.float32]] = [] - annotation_chunks: list[NDArray[np.int32]] = [] - source_edges: list[int] = [] - run_offsets = [0] - radius_threshold_voxel = minimum_radius_um / atlas_voxel_size_um - shape = np.asarray(atlas_shape_clearmap, dtype=np.float64) - - for edge_index, (edge_start, edge_end) in enumerate(edge_indices_i64): - start = int(edge_start) - end = int(edge_end) - edge_coordinates = np.asarray(coordinates[start:end], dtype=np.float64) - edge_radii = np.asarray(radii[start:end], dtype=np.float64) - qualifying = np.all(np.isfinite(edge_coordinates), axis=1) - qualifying &= np.isfinite(edge_radii) - qualifying &= np.all(edge_coordinates >= 0.0, axis=1) - qualifying &= np.all(edge_coordinates < shape, axis=1) - qualifying &= edge_radii >= radius_threshold_voxel - for block_start, block_end in _true_blocks(qualifying): - if block_end - block_start < 2: - continue - source_slice = slice(start + block_start, start + block_end) - clearmap_points = np.asarray(coordinates[source_slice], dtype=np.float32) - point_chunks.append(np.ascontiguousarray(clearmap_points[:, (1, 0, 2)])) - radius_chunks.append( - np.ascontiguousarray( - ( - np.asarray(radii[source_slice], dtype=np.float64) * atlas_voxel_size_um - ).astype(np.float32) - ) - ) - annotation_chunks.append( - np.ascontiguousarray(np.asarray(annotations_i64[source_slice], dtype=np.int32)) - ) - source_edges.append(edge_index) - run_offsets.append(run_offsets[-1] + block_end - block_start) - - if point_chunks: - output_points = np.concatenate(point_chunks).astype(np.float32, copy=False) - output_radii = np.concatenate(radius_chunks).astype(np.float32, copy=False) - output_annotations = np.concatenate(annotation_chunks).astype(np.int32, copy=False) - else: - output_points = np.empty((0, 3), dtype=np.float32) - output_radii = np.empty((0,), dtype=np.float32) - output_annotations = np.empty((0,), dtype=np.int32) - return LambadaMajorVesselAssetData( - points_asr_voxel_f32=np.ascontiguousarray(output_points), - radii_um_f32=np.ascontiguousarray(output_radii), - source_annotation_ids_i32=np.ascontiguousarray(output_annotations), - run_offsets_i64=np.asarray(run_offsets, dtype=np.int64), - source_edge_indices_i32=np.asarray(source_edges, dtype=np.int32), - ) - - -def _true_blocks(mask: NDArray[np.bool_]) -> list[tuple[int, int]]: - padded = np.empty(mask.size + 2, dtype=np.bool_) - padded[0] = False - padded[-1] = False - padded[1:-1] = mask - transitions = np.diff(padded.astype(np.int8, copy=False)) - starts = np.flatnonzero(transitions == 1) - ends = np.flatnonzero(transitions == -1) - return [(int(start), int(end)) for start, end in zip(starts, ends, strict=True)] - - -def _hash_file(path: Path) -> str: - digest = hashlib.sha256() - with path.open("rb") as stream: - while chunk := stream.read(16 * 1024 * 1024): - digest.update(chunk) - return digest.hexdigest() - - -def _verify_pinned_source(path: Path) -> None: - if path.is_symlink() or not path.is_file(): - raise LambadaMajorVesselError("source graph must be a regular non-symlink file") - if path.stat().st_size != SOURCE_SIZE_BYTES: - raise LambadaMajorVesselError("source graph byte count does not match the pinned file") - if _hash_file(path) != SOURCE_SHA256: - raise LambadaMajorVesselError("source graph SHA-256 does not match the pinned file") - with path.open("rb") as stream: - if stream.read(6) != b"\xe2\x9b\xbe gt": - raise LambadaMajorVesselError("pinned source does not have the graph-tool magic") - - -def extract_pinned_lambada_major_vessels( - source_path: str | Path, -) -> tuple[LambadaMajorVesselAssetData, LambadaExtractionReport]: - """Extract the audited derivative from the exact 12 GB P60_606 graph.""" - - path = Path(source_path).expanduser() - _verify_pinned_source(path) - - coordinates = np.memmap( - path, - mode="r", - dtype=" edge_indices[:, 0])) - or not bool(np.all(edge_indices[1:, 0] == edge_indices[:-1, 1])) - ): - raise LambadaMajorVesselError("pinned edge geometry ranges are not contiguous") - edge_radii = np.memmap( - path, - mode="r", - dtype="= MINIMUM_RADIUS_ATLAS_VOXEL) - if candidate_edges.size != EXPECTED_CANDIDATE_EDGE_COUNT: - raise LambadaMajorVesselError("pinned candidate edge count changed") - - point_chunks: list[NDArray[np.float32]] = [] - radius_chunks: list[NDArray[np.float32]] = [] - annotation_chunks: list[NDArray[np.int32]] = [] - source_edge_chunks: list[int] = [] - run_offsets = [0] - qualifying_in_bounds_points = 0 - selected_path_length_f64 = 0.0 - edges_with_runs: set[int] = set() - shape = np.asarray(ATLAS_SHAPE_CLEARMAP, dtype=np.float64) - - for edge_value in candidate_edges: - edge_index = int(edge_value) - start = int(edge_indices[edge_index, 0]) - end = int(edge_indices[edge_index, 1]) - edge_coordinates = np.asarray(coordinates[start:end]) - edge_radii_values = np.asarray(radii[start:end]) - qualifying = np.all(np.isfinite(edge_coordinates), axis=1) - qualifying &= np.isfinite(edge_radii_values) - qualifying &= np.all(edge_coordinates >= 0.0, axis=1) - qualifying &= np.all(edge_coordinates < shape, axis=1) - qualifying &= edge_radii_values >= MINIMUM_RADIUS_ATLAS_VOXEL - qualifying_in_bounds_points += int(np.count_nonzero(qualifying)) - for block_start, block_end in _true_blocks(qualifying): - if block_end - block_start < 2: - continue - source_slice = slice(start + block_start, start + block_end) - clearmap_points_f64 = np.asarray(coordinates[source_slice]) - deltas = np.diff(clearmap_points_f64, axis=0) - selected_path_length_f64 += float( - np.sum(np.linalg.norm(deltas, axis=1), dtype=np.float64) * ATLAS_VOXEL_SIZE_UM - ) - points_f32 = np.asarray(clearmap_points_f64, dtype=np.float32) - point_chunks.append(np.ascontiguousarray(points_f32[:, (1, 0, 2)])) - radius_chunks.append( - np.ascontiguousarray( - ( - np.asarray(radii[source_slice], dtype=np.float64) * ATLAS_VOXEL_SIZE_UM - ).astype(np.float32) - ) - ) - annotation_values = np.asarray(annotations[source_slice]) - if annotation_values.size and ( - int(np.min(annotation_values)) < np.iinfo(np.int32).min - or int(np.max(annotation_values)) > np.iinfo(np.int32).max - ): - raise LambadaMajorVesselError("source annotation ID does not fit int32") - annotation_chunks.append( - np.ascontiguousarray(annotation_values.astype(np.int32, copy=False)) - ) - source_edge_chunks.append(edge_index) - edges_with_runs.add(edge_index) - run_offsets.append(run_offsets[-1] + block_end - block_start) - - data = LambadaMajorVesselAssetData( - points_asr_voxel_f32=np.ascontiguousarray(np.concatenate(point_chunks)), - radii_um_f32=np.ascontiguousarray(np.concatenate(radius_chunks)), - source_annotation_ids_i32=np.ascontiguousarray(np.concatenate(annotation_chunks)), - run_offsets_i64=np.asarray(run_offsets, dtype=np.int64), - source_edge_indices_i32=np.asarray(source_edge_chunks, dtype=np.int32), - ) - asset_length_f32 = _polyline_length_um( - data.points_asr_voxel_f32 * np.float32(ATLAS_VOXEL_SIZE_UM), - data.run_offsets_i64, - ) - report = LambadaExtractionReport( - candidate_edges_by_source_edge_max=int(candidate_edges.size), - qualifying_in_bounds_points=qualifying_in_bounds_points, - output_points=int(data.points_asr_voxel_f32.shape[0]), - output_segments=int(data.points_asr_voxel_f32.shape[0] - len(source_edge_chunks)), - output_runs=len(source_edge_chunks), - source_edges_with_runs=len(edges_with_runs), - selected_path_length_um_source_f64=selected_path_length_f64, - asset_path_length_um_f32=asset_length_f32, - ) - _validate_expected_report(report) - return data, report - - -def _validate_expected_report(report: LambadaExtractionReport) -> None: - expected = ( - EXPECTED_CANDIDATE_EDGE_COUNT, - EXPECTED_QUALIFYING_IN_BOUNDS_POINT_COUNT, - EXPECTED_RUN_POINT_COUNT, - EXPECTED_SEGMENT_COUNT, - EXPECTED_RUN_COUNT, - EXPECTED_SOURCE_EDGES_WITH_RUNS, - ) - observed = ( - report.candidate_edges_by_source_edge_max, - report.qualifying_in_bounds_points, - report.output_points, - report.output_segments, - report.output_runs, - report.source_edges_with_runs, - ) - if observed != expected: - raise LambadaMajorVesselError( - f"extraction statistics changed: expected {expected}, observed {observed}" - ) - - -def _polyline_length_um( - points_asr_um: NDArray[np.generic], - run_offsets: NDArray[np.generic], -) -> float: - points = np.asarray(points_asr_um, dtype=np.float64) - offsets = np.asarray(run_offsets, dtype=np.int64) - total = 0.0 - for start_value, end_value in pairwise(offsets): - start = int(start_value) - end = int(end_value) - total += float(np.sum(np.linalg.norm(np.diff(points[start:end], axis=0), axis=1))) - return total - - -def write_deterministic_npz( - output_path: str | Path, - data: LambadaMajorVesselAssetData, -) -> tuple[str, int]: - """Write fixed-timestamp, fixed-order compressed NPY members atomically.""" - - output = Path(output_path) - output.parent.mkdir(parents=True, exist_ok=True) - arrays = data.arrays() - unknown = set(arrays) - set(ARRAY_DTYPES) - if unknown: - raise LambadaMajorVesselError(f"unexpected output arrays: {sorted(unknown)}") - descriptor, temporary_name = tempfile.mkstemp( - prefix=f".{output.name}.", - suffix=".tmp", - dir=output.parent, - ) - os.close(descriptor) - temporary = Path(temporary_name) - try: - with zipfile.ZipFile( - temporary, - mode="w", - compression=zipfile.ZIP_DEFLATED, - compresslevel=9, - allowZip64=True, - strict_timestamps=True, - ) as archive: - for name in sorted(ARRAY_DTYPES): - expected_dtype = ARRAY_DTYPES[name] - array = np.ascontiguousarray(arrays[name], dtype=expected_dtype) - buffer = io.BytesIO() - np.save(buffer, array, allow_pickle=False) - info = zipfile.ZipInfo(f"{name}.npy", date_time=(1980, 1, 1, 0, 0, 0)) - info.compress_type = zipfile.ZIP_DEFLATED - info.create_system = 3 - info.external_attr = 0o100644 << 16 - archive.writestr( - info, - buffer.getvalue(), - compress_type=zipfile.ZIP_DEFLATED, - compresslevel=9, - ) - temporary.chmod(0o644) - temporary.replace(output) - finally: - if temporary.exists(): - temporary.unlink() - return _hash_file(output), output.stat().st_size - - -def build_asset_manifest( - *, - asset_sha256: str, - asset_size_bytes: int, - data: LambadaMajorVesselAssetData, - report: LambadaExtractionReport, -) -> dict[str, object]: - """Build the canonical attribution, schema, and extraction manifest.""" - - _validate_expected_report(report) - if (asset_sha256, asset_size_bytes) != (ASSET_SHA256, ASSET_SIZE_BYTES): - raise LambadaMajorVesselError( - "derived asset bytes differ from the reviewed reproducible build" - ) - arrays = { - name: { - "dtype": str(ARRAY_DTYPES[name]), - "shape": list(data.arrays()[name].shape), - } - for name in sorted(ARRAY_DTYPES) - } - return { - "schema_version": ASSET_SCHEMA_VERSION, - "asset": { - "filename": ASSET_FILENAME, - "sha256": asset_sha256, - "size_bytes": asset_size_bytes, - "arrays": arrays, - }, - "source": _expected_source_manifest(), - "extraction": { - "algorithm_version": EXTRACTION_ALGORITHM_VERSION, - "atlas_identifier": ATLAS_IDENTIFIER, - "atlas_version": ATLAS_VERSION, - "source_coordinate_order": ["ClearMap axis 0", "ClearMap axis 1", "ClearMap axis 2"], - "output_frame": "BRAINGLOBE_VOXEL_ASR", - "output_axis_order": ["AP", "DV", "ML"], - "atlas_shape_asr": list(ATLAS_SHAPE_ASR), - "atlas_voxel_size_um": [ATLAS_VOXEL_SIZE_UM] * 3, - "voxel_anchor": ( - "index anchor: physical_um = continuous_voxel * 25; no half-voxel shift" - ), - "minimum_radius_um": MINIMUM_RADIUS_UM, - "minimum_diameter_um": MINIMUM_DIAMETER_UM, - "run_rule": ( - "maximal consecutive blocks of at least two source edge-geometry points for " - "which every point is finite, in bounds, and radius >= 15 um" - ), - "out_of_bounds_rule": "split and drop; never clip or interpolate", - "source_annotation_semantics": ( - "coarse source ancestor IDs; use the installed atlas annotation at display or " - "interaction coordinates for current region identity" - ), - "radius_semantics": ( - "source radii_atlas * 25 um; source used a mean resampling scale, not a local " - "Jacobian correction" - ), - "statistics": { - "candidate_edges_by_source_edge_max": report.candidate_edges_by_source_edge_max, - "qualifying_in_bounds_points": report.qualifying_in_bounds_points, - "output_points": report.output_points, - "output_segments": report.output_segments, - "output_runs": report.output_runs, - "source_edges_with_runs": report.source_edges_with_runs, - "selected_path_length_um_source_f64": (report.selected_path_length_um_source_f64), - "asset_path_length_um_f32": report.asset_path_length_um_f32, - }, - }, - "limitations": list(MANDATORY_LIMITATIONS), - } - - -def _expected_source_manifest() -> dict[str, object]: - return { - "dataset_title": "Vascular graphs of the developing post-natal mouse brain", - "authors": ["Nicolas Renier", "Elisa de Launoit", "Sophie Skriabine"], - "specimen_id": "P60_606", - "specimen_age": "P60", - "specimen_sex": "unpublished", - "specimen_side": "unpublished", - "record_doi": SOURCE_RECORD_DOI, - "concept_doi": SOURCE_CONCEPT_DOI, - "record_url": SOURCE_RECORD_URL, - "license": SOURCE_LICENSE, - "license_url": SOURCE_LICENSE_URL, - "paper_doi": SOURCE_PAPER_DOI, - "paper_url": SOURCE_PAPER_URL, - "distributed_archive": { - "filename": SOURCE_ARCHIVE_FILENAME, - "size_bytes": SOURCE_ARCHIVE_SIZE_BYTES, - "sha256": SOURCE_ARCHIVE_SHA256, - "md5": SOURCE_ARCHIVE_MD5, - }, - "extracted_graph": { - "filename": SOURCE_FILENAME, - "size_bytes": SOURCE_SIZE_BYTES, - "sha256": SOURCE_SHA256, - "md5": SOURCE_MD5, - }, - } - - -def write_asset_manifest(output_path: str | Path, manifest: Mapping[str, object]) -> None: - """Write canonical JSON atomically.""" - - output = Path(output_path) - output.parent.mkdir(parents=True, exist_ok=True) - payload = (json.dumps(manifest, indent=2, sort_keys=True, ensure_ascii=True) + "\n").encode() - descriptor, temporary_name = tempfile.mkstemp( - prefix=f".{output.name}.", suffix=".tmp", dir=output.parent - ) - temporary = Path(temporary_name) - try: - with os.fdopen(descriptor, "wb") as stream: - stream.write(payload) - stream.flush() - os.fsync(stream.fileno()) - temporary.chmod(0o644) - temporary.replace(output) - finally: - if temporary.exists(): - temporary.unlink() - - -def _reject_duplicate_json_keys(pairs: Sequence[tuple[str, object]]) -> dict[str, object]: - result: dict[str, object] = {} - for key, value in pairs: - if key in result: - raise LambadaMajorVesselError(f"duplicate manifest key: {key}") - result[key] = value - return result - - -def _read_manifest(path: Path) -> dict[str, object]: - if path.is_symlink() or not path.is_file(): - raise LambadaMajorVesselError("vessel manifest must be a regular non-symlink file") - if path.stat().st_size > 128 * 1024: - raise LambadaMajorVesselError("vessel manifest is unexpectedly large") - try: - parsed = json.loads( - path.read_text(encoding="utf-8"), object_pairs_hook=_reject_duplicate_json_keys - ) - except (OSError, UnicodeError, json.JSONDecodeError) as error: - raise LambadaMajorVesselError("vessel manifest is not valid UTF-8 JSON") from error - if not isinstance(parsed, dict): - raise LambadaMajorVesselError("vessel manifest root must be an object") - return cast(dict[str, object], parsed) - - -def _mapping(value: object, *, name: str) -> dict[str, object]: - if not isinstance(value, dict) or not all(isinstance(key, str) for key in value): - raise LambadaMajorVesselError(f"manifest {name} must be an object") - return cast(dict[str, object], value) - - -def _integer(value: object, *, name: str) -> int: - if isinstance(value, bool) or not isinstance(value, int): - raise LambadaMajorVesselError(f"manifest {name} must be an integer") - return value - - -def _number(value: object, *, name: str) -> float: - if isinstance(value, bool) or not isinstance(value, (int, float)): - raise LambadaMajorVesselError(f"manifest {name} must be numeric") - result = float(value) - if not math.isfinite(result): - raise LambadaMajorVesselError(f"manifest {name} must be finite") - return result - - -def _validate_manifest(manifest: dict[str, object], asset_path: Path) -> tuple[str, float]: - if set(manifest) != {"schema_version", "asset", "source", "extraction", "limitations"}: - raise LambadaMajorVesselError("vessel manifest root fields changed") - if _integer(manifest["schema_version"], name="schema_version") != ASSET_SCHEMA_VERSION: - raise LambadaMajorVesselError("vessel manifest schema version is unsupported") - if manifest["source"] != _expected_source_manifest(): - raise LambadaMajorVesselError("vessel source provenance does not match the reviewed source") - limitations = manifest["limitations"] - if limitations != list(MANDATORY_LIMITATIONS): - raise LambadaMajorVesselError("mandatory vessel limitations are missing or changed") - - asset = _mapping(manifest["asset"], name="asset") - if set(asset) != {"filename", "sha256", "size_bytes", "arrays"}: - raise LambadaMajorVesselError("vessel asset manifest fields changed") - if asset["filename"] != ASSET_FILENAME: - raise LambadaMajorVesselError("vessel asset filename is not the reviewed filename") - expected_size = _integer(asset["size_bytes"], name="asset.size_bytes") - expected_hash = asset["sha256"] - if not isinstance(expected_hash, str) or len(expected_hash) != 64: - raise LambadaMajorVesselError("vessel asset SHA-256 is invalid") - if (expected_hash, expected_size) != (ASSET_SHA256, ASSET_SIZE_BYTES): - raise LambadaMajorVesselError("vessel asset identity is not the reviewed build") - if asset_path.is_symlink() or not asset_path.is_file(): - raise LambadaMajorVesselError("vessel asset must be a regular non-symlink file") - if asset_path.stat().st_size != expected_size or expected_size > 32 * 1024 * 1024: - raise LambadaMajorVesselError("vessel asset byte count is invalid") - if _hash_file(asset_path) != expected_hash: - raise LambadaMajorVesselError("vessel asset SHA-256 does not match its manifest") - - extraction = _mapping(manifest["extraction"], name="extraction") - required_exact = { - "algorithm_version": EXTRACTION_ALGORITHM_VERSION, - "atlas_identifier": ATLAS_IDENTIFIER, - "atlas_version": ATLAS_VERSION, - "source_coordinate_order": ["ClearMap axis 0", "ClearMap axis 1", "ClearMap axis 2"], - "output_frame": "BRAINGLOBE_VOXEL_ASR", - "output_axis_order": ["AP", "DV", "ML"], - "atlas_shape_asr": list(ATLAS_SHAPE_ASR), - "atlas_voxel_size_um": [ATLAS_VOXEL_SIZE_UM] * 3, - "voxel_anchor": "index anchor: physical_um = continuous_voxel * 25; no half-voxel shift", - "minimum_radius_um": MINIMUM_RADIUS_UM, - "minimum_diameter_um": MINIMUM_DIAMETER_UM, - "run_rule": ( - "maximal consecutive blocks of at least two source edge-geometry points for which " - "every point is finite, in bounds, and radius >= 15 um" - ), - "out_of_bounds_rule": "split and drop; never clip or interpolate", - "source_annotation_semantics": ( - "coarse source ancestor IDs; use the installed atlas annotation at display or " - "interaction coordinates for current region identity" - ), - "radius_semantics": ( - "source radii_atlas * 25 um; source used a mean resampling scale, not a local " - "Jacobian correction" - ), - } - if set(extraction) != set(required_exact) | {"statistics"}: - raise LambadaMajorVesselError("vessel extraction manifest fields changed") - for key, expected in required_exact.items(): - if extraction[key] != expected: - raise LambadaMajorVesselError(f"vessel extraction field {key} changed") - statistics = _mapping(extraction["statistics"], name="extraction.statistics") - expected_counts = { - "candidate_edges_by_source_edge_max": EXPECTED_CANDIDATE_EDGE_COUNT, - "qualifying_in_bounds_points": EXPECTED_QUALIFYING_IN_BOUNDS_POINT_COUNT, - "output_points": EXPECTED_RUN_POINT_COUNT, - "output_segments": EXPECTED_SEGMENT_COUNT, - "output_runs": EXPECTED_RUN_COUNT, - "source_edges_with_runs": EXPECTED_SOURCE_EDGES_WITH_RUNS, - } - if set(statistics) != set(expected_counts) | { - "selected_path_length_um_source_f64", - "asset_path_length_um_f32", - }: - raise LambadaMajorVesselError("vessel extraction statistics fields changed") - for key, expected in expected_counts.items(): - if _integer(statistics[key], name=f"statistics.{key}") != expected: - raise LambadaMajorVesselError(f"vessel extraction statistic {key} changed") - asset_path_length = _number( - statistics["asset_path_length_um_f32"], name="statistics.asset_path_length_um_f32" - ) - _number( - statistics["selected_path_length_um_source_f64"], - name="statistics.selected_path_length_um_source_f64", - ) - - arrays = _mapping(asset["arrays"], name="asset.arrays") - expected_shapes = { - "points_asr_voxel_f32": [EXPECTED_RUN_POINT_COUNT, 3], - "radii_um_f32": [EXPECTED_RUN_POINT_COUNT], - "source_annotation_ids_i32": [EXPECTED_RUN_POINT_COUNT], - "run_offsets_i64": [EXPECTED_RUN_COUNT + 1], - "source_edge_indices_i32": [EXPECTED_RUN_COUNT], - } - if set(arrays) != set(ARRAY_DTYPES): - raise LambadaMajorVesselError("vessel asset array inventory changed") - for name, expected_shape in expected_shapes.items(): - descriptor = _mapping(arrays[name], name=f"asset.arrays.{name}") - if descriptor != {"dtype": str(ARRAY_DTYPES[name]), "shape": expected_shape}: - raise LambadaMajorVesselError(f"vessel array descriptor {name} changed") - return expected_hash, asset_path_length - - -def _validate_zip_members(asset_path: Path) -> None: - expected_names = {f"{name}.npy" for name in ARRAY_DTYPES} - try: - with zipfile.ZipFile(asset_path) as archive: - infos = archive.infolist() - if {info.filename for info in infos} != expected_names or len(infos) != len( - expected_names - ): - raise LambadaMajorVesselError("vessel NPZ member inventory changed") - if any(info.file_size > 4 * 1024 * 1024 for info in infos): - raise LambadaMajorVesselError("vessel NPZ member is unexpectedly large") - if sum(info.file_size for info in infos) > 8 * 1024 * 1024: - raise LambadaMajorVesselError("vessel NPZ expands beyond its reviewed bound") - except (OSError, zipfile.BadZipFile) as error: - raise LambadaMajorVesselError("vessel asset is not a valid NPZ archive") from error - - -def _validate_loaded_asset( - arrays: Mapping[str, NDArray[np.generic]], - *, - expected_path_length_um: float, -) -> LambadaMajorVesselAssetData: - if set(arrays) != set(ARRAY_DTYPES): - raise LambadaMajorVesselError("loaded vessel array inventory changed") - for name, expected_dtype in ARRAY_DTYPES.items(): - if arrays[name].dtype != expected_dtype: - raise LambadaMajorVesselError(f"loaded vessel array {name} has the wrong dtype") - if not arrays[name].flags.c_contiguous: - raise LambadaMajorVesselError(f"loaded vessel array {name} is not C-contiguous") - points = cast(NDArray[np.float32], arrays["points_asr_voxel_f32"]) - radii = cast(NDArray[np.float32], arrays["radii_um_f32"]) - annotations = cast(NDArray[np.int32], arrays["source_annotation_ids_i32"]) - offsets = cast(NDArray[np.int64], arrays["run_offsets_i64"]) - source_edges = cast(NDArray[np.int32], arrays["source_edge_indices_i32"]) - if points.shape != (EXPECTED_RUN_POINT_COUNT, 3): - raise LambadaMajorVesselError("loaded vessel points have the wrong shape") - if radii.shape != (EXPECTED_RUN_POINT_COUNT,) or annotations.shape != radii.shape: - raise LambadaMajorVesselError("loaded vessel point attributes have the wrong shape") - if offsets.shape != (EXPECTED_RUN_COUNT + 1,) or source_edges.shape != (EXPECTED_RUN_COUNT,): - raise LambadaMajorVesselError("loaded vessel run arrays have the wrong shape") - if not bool(np.all(np.isfinite(points))) or not bool(np.all(np.isfinite(radii))): - raise LambadaMajorVesselError("loaded vessel geometry contains non-finite values") - shape = np.asarray(ATLAS_SHAPE_ASR, dtype=np.float32) - if bool(np.any(points < 0.0)) or bool(np.any(points >= shape)): - raise LambadaMajorVesselError("loaded vessel point is outside the atlas") - if bool(np.any(radii < np.float32(MINIMUM_RADIUS_UM))): - raise LambadaMajorVesselError("loaded vessel point is below the radius threshold") - if bool(np.any(annotations < 0)): - raise LambadaMajorVesselError("loaded vessel source annotation ID is negative") - if int(offsets[0]) != 0 or int(offsets[-1]) != EXPECTED_RUN_POINT_COUNT: - raise LambadaMajorVesselError("loaded vessel run offsets do not span the point array") - if not bool(np.all(np.diff(offsets) >= 2)): - raise LambadaMajorVesselError("loaded vessel run is shorter than one segment") - if bool(np.any(source_edges < 0)) or bool(np.any(source_edges >= SOURCE_EDGE_COUNT)): - raise LambadaMajorVesselError("loaded vessel source edge index is invalid") - if bool(np.any(source_edges[1:] < source_edges[:-1])): - raise LambadaMajorVesselError("loaded vessel source edge order is not deterministic") - observed_length = _polyline_length_um(points * np.float32(ATLAS_VOXEL_SIZE_UM), offsets) - if not math.isclose(observed_length, expected_path_length_um, rel_tol=0.0, abs_tol=1e-4): - raise LambadaMajorVesselError("loaded vessel path length does not match its manifest") - return LambadaMajorVesselAssetData(points, radii, annotations, offsets, source_edges) - - -def bundled_asset_paths() -> tuple[Path, Path]: - """Return the installed compact asset and adjacent manifest paths.""" - - asset_root = Path(__file__).resolve().parent.parent / "assets" / "vasculature" - return asset_root / ASSET_FILENAME, asset_root / MANIFEST_FILENAME - - -def load_lambada_major_vessels( - asset_path: str | Path | None = None, - manifest_path: str | Path | None = None, -) -> LambadaMajorVesselGraph: - """Load, verify, physically scale, and freeze the compact display graph.""" - - bundled_asset, bundled_manifest = bundled_asset_paths() - asset = bundled_asset if asset_path is None else Path(asset_path) - manifest = bundled_manifest if manifest_path is None else Path(manifest_path) - parsed_manifest = _read_manifest(manifest) - asset_sha256, expected_path_length = _validate_manifest(parsed_manifest, asset) - _validate_zip_members(asset) - try: - with np.load(asset, allow_pickle=False) as archive: - loaded_arrays = {name: np.asarray(archive[name]) for name in archive.files} - except (OSError, ValueError, KeyError, zipfile.BadZipFile) as error: - raise LambadaMajorVesselError("vessel asset arrays could not be loaded") from error - disk_data = _validate_loaded_asset( - loaded_arrays, - expected_path_length_um=expected_path_length, - ) - points_asr_um = np.ascontiguousarray( - disk_data.points_asr_voxel_f32 * np.float32(ATLAS_VOXEL_SIZE_UM) - ) - arrays_to_freeze: tuple[NDArray[np.generic], ...] = ( - points_asr_um, - disk_data.radii_um_f32, - disk_data.source_annotation_ids_i32, - disk_data.run_offsets_i64, - disk_data.source_edge_indices_i32, - ) - for array in arrays_to_freeze: - array.setflags(write=False) - source = _expected_source_manifest() - authors = cast(list[str], source["authors"]) - provenance = LambadaMajorVesselProvenance( - dataset_title=cast(str, source["dataset_title"]), - authors=tuple(authors), - specimen_id=cast(str, source["specimen_id"]), - record_doi=SOURCE_RECORD_DOI, - record_url=SOURCE_RECORD_URL, - license=SOURCE_LICENSE, - license_url=SOURCE_LICENSE_URL, - source_sha256=SOURCE_SHA256, - asset_sha256=asset_sha256, - extraction_algorithm_version=EXTRACTION_ALGORITHM_VERSION, - minimum_radius_um=MINIMUM_RADIUS_UM, - limitations=MANDATORY_LIMITATIONS, - ) - return LambadaMajorVesselGraph( - points_asr_um=points_asr_um, - radii_um=disk_data.radii_um_f32, - source_annotation_ids=disk_data.source_annotation_ids_i32, - run_offsets=disk_data.run_offsets_i64, - source_edge_indices=disk_data.source_edge_indices_i32, - provenance=provenance, - ) diff --git a/src/mouse_brain_planner/vasculature/vessap_major_vessels.py b/src/mouse_brain_planner/vasculature/vessap_major_vessels.py index 285bbf9..d82b479 100644 --- a/src/mouse_brain_planner/vasculature/vessap_major_vessels.py +++ b/src/mouse_brain_planner/vasculature/vessap_major_vessels.py @@ -1,6 +1,6 @@ -"""Pinned display-only VesSAP major-vessel geometry. +"""External display-only VesSAP major-vessel geometry. -The bundled derivative starts from the public BL6J-no1 whole-brain centerline +The optional derivative starts from the public BL6J-no1 whole-brain centerline and radius volumes released with VesSAP. Centerline voxels with a source radius below five 3-micrometre voxels are removed, the retained skeleton is mapped through the authors' published rigid plus B-spline Allen transform, and @@ -26,6 +26,8 @@ import numpy as np from numpy.typing import NDArray +from mouse_brain_planner.paths import app_paths + ASSET_SCHEMA_VERSION: Final = 1 ASSET_FILENAME: Final = "vessap_bl6j1_major_vessels_50um_v1.npz" MANIFEST_FILENAME: Final = f"{ASSET_FILENAME}.manifest.json" @@ -278,11 +280,19 @@ def _expected_validation() -> dict[str, object]: } -def bundled_asset_paths() -> tuple[Path, Path]: - asset_root = Path(__file__).resolve().parent.parent / "assets" / "vasculature" +def default_asset_paths() -> tuple[Path, Path]: + """Return the expected paths for the user-installed VesSAP data package.""" + + asset_root = app_paths().data / "vasculature" return asset_root / ASSET_FILENAME, asset_root / MANIFEST_FILENAME +def default_asset_is_available() -> bool: + """Return whether both external data files are regular, non-symlink files.""" + + return all(path.is_file() and not path.is_symlink() for path in default_asset_paths()) + + def _read_and_validate_manifest(manifest_path: Path, asset_path: Path) -> None: if manifest_path.is_symlink() or not manifest_path.is_file(): raise VesSAPMajorVesselError("VesSAP vessel manifest must be a regular file") @@ -407,11 +417,11 @@ def load_vessap_major_vessels( asset_path: str | Path | None = None, manifest_path: str | Path | None = None, ) -> VesSAPMajorVesselGraph: - """Load and freeze the exact reviewed display derivative.""" + """Load and freeze the exact externally installed display derivative.""" - bundled_asset, bundled_manifest = bundled_asset_paths() - asset = bundled_asset if asset_path is None else Path(asset_path) - manifest = bundled_manifest if manifest_path is None else Path(manifest_path) + default_asset, default_manifest = default_asset_paths() + asset = default_asset if asset_path is None else Path(asset_path) + manifest = default_manifest if manifest_path is None else Path(manifest_path) _read_and_validate_manifest(manifest, asset) _validate_zip_members(asset) points, radii, offsets, source_runs = _load_arrays(asset) diff --git a/tests/integration/test_bridge_major_vessel_persistence.py b/tests/integration/test_bridge_major_vessel_persistence.py index c6efb8e..94c69c1 100644 --- a/tests/integration/test_bridge_major_vessel_persistence.py +++ b/tests/integration/test_bridge_major_vessel_persistence.py @@ -87,7 +87,7 @@ def test_analysis_save_open_round_trip_and_plan_mutations_invalidate( "registrationUncertaintyMicrometres": 20.0, "requiredMarginMicrometres": 10.0, "sourceOrLabPolicy": ( - "Display the bundled diameter >= 30 micrometre VesSAP reference only. " + "Display the configured external diameter >= 30 micrometre VesSAP reference only. " "Clearance classification is unavailable without published subject-registration " "and tissue-distortion uncertainty bounds." ), diff --git a/tests/integration/test_real_25um_planning_workflow.py b/tests/integration/test_real_25um_planning_workflow.py index 1c45671..82ff63f 100644 --- a/tests/integration/test_real_25um_planning_workflow.py +++ b/tests/integration/test_real_25um_planning_workflow.py @@ -137,7 +137,7 @@ def _calibration_params(project_id: str, revision: int) -> dict[str, object]: "limitsSource": "Software integration-test thresholds; not a surgical SOP", "atlasTransformMethod": "rigid", "affineDistortionAcknowledged": False, - "notes": "Engineering QA fixture only; no animal or surgical validation claim", + "notes": "Engineering QA fixture", } diff --git a/tests/unit/test_bridge_major_vessels.py b/tests/unit/test_bridge_major_vessels.py index 263a65e..9603422 100644 --- a/tests/unit/test_bridge_major_vessels.py +++ b/tests/unit/test_bridge_major_vessels.py @@ -21,8 +21,6 @@ from mouse_brain_planner.domain.project_models import PlannerProject from mouse_brain_planner.vasculature.vessap_major_vessels import ( ASSET_SHA256, - EXPECTED_POINT_COUNT, - EXPECTED_RUN_COUNT, EXTRACTION_ALGORITHM_VERSION, MANDATORY_LIMITATIONS, REGISTRATION_TRANSFORM_ID, @@ -31,7 +29,6 @@ VesSAPMajorVesselError, VesSAPMajorVesselGraph, VesSAPMajorVesselProvenance, - load_vessap_major_vessels, ) @@ -197,13 +194,17 @@ def loader() -> VesSAPMajorVesselGraph: assert load_count == 0 -def test_planning_session_declares_display_only_major_vessels() -> None: +def test_planning_session_omits_external_vessel_capability_when_data_is_missing( + monkeypatch: pytest.MonkeyPatch, + tmp_path: Path, +) -> None: from tests.integration.test_bridge_probe_planning import _probe_dispatcher + monkeypatch.setenv("MOUSE_BRAIN_PLANNER_DATA_DIR", str(tmp_path)) dispatcher, _session = _probe_dispatcher() hello = _call(dispatcher, "hello", client="planning-registration-test") capabilities = _mapping(hello["capabilities"]) - assert capabilities["auditedReferenceMajorVessels"] is True + assert "auditedReferenceMajorVessels" not in capabilities assert "radiusAwareReferenceVesselAnalysis" not in capabilities @@ -333,63 +334,6 @@ def loader() -> VesSAPMajorVesselGraph: assert load_count == 1 -def test_real_bundled_graph_round_trips_through_bridge_buffers( - qualified_test_reference: None, -) -> None: - dispatcher = _dispatcher_with_atlas(_exact_atlas()) - graph = load_vessap_major_vessels() - register_major_vessel_handlers( - dispatcher, - get_project=lambda: pytest.fail("project not expected"), - get_revision=lambda: 0, - replace_project=_unexpected_project_replace, - graph_loader=lambda: graph, - ) - - geometry = _call(dispatcher, "vessel.major.reference.geometry") - assert geometry["pointCount"] == EXPECTED_POINT_COUNT - assert geometry["runCount"] == EXPECTED_RUN_COUNT - assert geometry["segmentCount"] == EXPECTED_POINT_COUNT - EXPECTED_RUN_COUNT - np.testing.assert_array_equal( - _decode_buffer( - geometry["pointsASRMicrometres"], - dtype=np.dtype(" None: diff --git a/tests/unit/test_lambada_coordinate_qualification.py b/tests/unit/test_lambada_coordinate_qualification.py deleted file mode 100644 index 175ad35..0000000 --- a/tests/unit/test_lambada_coordinate_qualification.py +++ /dev/null @@ -1,282 +0,0 @@ -from __future__ import annotations - -import hashlib -import json -import pickle -import struct -from pathlib import Path - -import numpy as np -import pytest - -import mouse_brain_planner.vasculature.lambada_coordinate_qualification as qualification -from mouse_brain_planner.vasculature.lambada_coordinate_qualification import ( - ATLAS_SHAPE_ASR, - LambadaCoordinateQualificationError, - RetainedSourceSelection, - assess_pinned_source_coverage, - canonical_report_bytes, - load_graph_numpy_property, - scan_gt_file, - score_annotation_orientations, - score_hemisphere_laterality, - write_canonical_report, -) - - -def _u64(value: int) -> bytes: - return struct.pack(" bytes: - payload = value.encode("utf-8") - return _u64(len(payload)) + payload - - -def _write_small_gt(path: Path, *, object_payload: bytes, trailing: bytes = b"") -> None: - payload = bytearray(b"\xe2\x9b\xbe gt\x01\x00") - payload.extend(_gt_string("qualification fixture")) - payload.extend(b"\x00") # undirected - payload.extend(_u64(3)) - payload.extend(_u64(1) + b"\x01") - payload.extend(_u64(1) + b"\x02") - payload.extend(_u64(0)) - payload.extend(_u64(3)) - - payload.extend(b"\x00") - payload.extend(_gt_string("edge_geometry_hemisphere")) - payload.extend(b"\x0e") - payload.extend(_u64(len(object_payload))) - payload.extend(object_payload) - - payload.extend(b"\x01") - payload.extend(_gt_string("vertex_id")) - payload.extend(b"\x02") - payload.extend(struct.pack(" None: - payload = bytearray(b"\xe2\x9b\xbe gt\x01\x00") - payload.extend(_gt_string("coverage fixture")) - payload.extend(b"\x00") - payload.extend(_u64(3)) - payload.extend(_u64(1) + b"\x01") - payload.extend(_u64(1) + b"\x02") - payload.extend(_u64(0)) - payload.extend(_u64(1)) - payload.extend(b"\x01") - payload.extend(_gt_string("coordinates_atlas")) - payload.extend(b"\x0b") - for coordinates in ((1.0, 2.0, 100.0), (3.0, 4.0, 227.0), (5.0, 6.0, 240.0)): - payload.extend(_u64(3)) - payload.extend(struct.pack(" None: - hemisphere = np.array([255, 0, 255], dtype=np.int64) - source = tmp_path / "fixture.gt" - _write_small_gt(source, object_payload=pickle.dumps(hemisphere, protocol=4)) - - inventory = scan_gt_file(source) - - assert inventory.byte_order == "little" - assert not inventory.directed - assert inventory.vertex_count == 3 - assert inventory.edge_count == 2 - assert [(record.kind, record.name, record.type_index) for record in inventory.properties] == [ - ("graph", "edge_geometry_hemisphere", 14), - ("vertex", "vertex_id", 2), - ("edge", "edge_geometry_indices", 10), - ] - record = inventory.properties[0] - assert record.object_payload_size_bytes == len(pickle.dumps(hemisphere, protocol=4)) - decoded = load_graph_numpy_property(source, record, expected_shape=(3,)) - np.testing.assert_array_equal(decoded, hemisphere) - assert not decoded.flags.writeable - - -def test_restricted_numpy_property_rejects_non_numpy_pickle_global(tmp_path: Path) -> None: - source = tmp_path / "fixture.gt" - _write_small_gt(source, object_payload=pickle.dumps(eval, protocol=4)) - record = scan_gt_file(source).properties[0] - - with pytest.raises(LambadaCoordinateQualificationError, match="not permitted"): - load_graph_numpy_property(source, record, expected_shape=(3,)) - - -def test_gt_scanner_rejects_unaccounted_trailing_bytes(tmp_path: Path) -> None: - source = tmp_path / "fixture.gt" - _write_small_gt( - source, - object_payload=pickle.dumps(np.array([0, 255, 0]), protocol=4), - trailing=b"unexpected", - ) - - with pytest.raises(LambadaCoordinateQualificationError, match="exact file"): - scan_gt_file(source) - - -def test_scores_all_flips_with_source_annotation_as_ancestor(tmp_path: Path) -> None: - annotation_path = tmp_path / "annotation.raw" - annotation = np.memmap( - annotation_path, - mode="w+", - dtype=np.uint32, - shape=ATLAS_SHAPE_ASR, - ) - points = np.array( - [ - [20.25, 40.5, 60.75], - [21.25, 41.5, 61.75], - [22.25, 42.5, 62.75], - [23.25, 43.5, 63.75], - ], - dtype=np.float64, - ) - source_ids = np.array([10, 10, 10, 10], dtype=np.int32) - expected_asr = points[:, (1, 0, 2)].astype(np.int64) - annotation[expected_asr[:, 0], expected_asr[:, 1], expected_asr[:, 2]] = 20 - annotation.flush() - - scores = score_annotation_orientations( - points, - source_ids, - annotation, - {10: frozenset({10}), 20: frozenset({10, 20})}, - ) - - assert len(scores) == 8 - expected = next(score for score in scores if score["label"] == "AP_KEEP_DV_KEEP_ML_KEEP") - assert expected["evaluablePointCount"] == 4 - assert expected["ancestorAgreement"] == { - "numerator": 4, - "denominator": 4, - "decimal": "1.000000000", - } - alternatives = [score for score in scores if score is not expected] - assert all(score["evaluablePointCount"] == 0 for score in alternatives) - - -def test_hemisphere_score_uses_right_origin_left_increasing_ml() -> None: - points = np.array( - [ - [10.0, 20.0, 0.0], - [10.0, 20.0, 227.0], - [10.0, 20.0, 228.0], - [10.0, 20.0, 455.0], - ] - ) - source_labels = np.array([255, 255, 0, 0], dtype=np.int64) - - score = score_hemisphere_laterality(points, source_labels) - - assert score["keepAxis2Agreement"] == { - "numerator": 4, - "denominator": 4, - "decimal": "1.000000000", - } - assert score["flipAxis2Agreement"] == { - "numerator": 0, - "denominator": 4, - "decimal": "0.000000000", - } - - -def test_canonical_report_writer_is_stable_and_atomic(tmp_path: Path) -> None: - report = {"status": "rejected", "schemaVersion": 1, "nested": {"z": 2, "a": 1}} - expected = b'{"nested":{"a":1,"z":2},"schemaVersion":1,"status":"rejected"}\n' - output = tmp_path / "report.json" - - assert canonical_report_bytes(report) == expected - digest = write_canonical_report(output, report) - - assert output.read_bytes() == expected - assert digest == hashlib.sha256(expected).hexdigest() - assert json.loads(output.read_text(encoding="utf-8"))["status"] == "rejected" - - -def test_source_coverage_does_not_promote_numerical_both_sides_to_whole_brain( - tmp_path: Path, -) -> None: - source = tmp_path / "coverage.gt" - _write_coverage_gt(source) - selection = RetainedSourceSelection( - points_clearmap_voxel_f64=np.array( - [[1.0, 2.0, 120.0], [3.0, 4.0, 235.0]], dtype=np.float64 - ), - source_geometry_indices_i64=np.array([0, 1], dtype=np.int64), - radii_um_f32=np.array([15.0, 15.0], dtype=np.float32), - source_annotation_ids_i32=np.array([1, 1], dtype=np.int32), - run_offsets_i64=np.array([0, 2], dtype=np.int64), - source_edge_indices_i32=np.array([1], dtype=np.int32), - qualifying_in_bounds_points=2, - source_edges_with_runs=1, - ) - - report, check = assess_pinned_source_coverage(source, scan_gt_file(source), selection) - - assert report["status"] == "unqualified-for-whole-brain-use" - assert report["numericalBothSidesDoNotProveWholeBrain"] is True - assert report["bilateralMirroringQualified"] is False - vertex_report = report["vertexCoordinatesAtlas"] - assert isinstance(vertex_report, dict) - assert vertex_report["axis2BelowMidlineCount"] == 2 - assert vertex_report["axis2AtOrAboveMidlineCount"] == 1 - assert check["passed"] is False - - -def test_committed_rejection_report_preserves_inventory_scores_and_blockers() -> None: - report_path = ( - Path(qualification.__file__).parent.parent - / "assets" - / "vasculature" - / "lambada_p60_606_coordinate_qualification_rejected_v1.json" - ) - documentation_report_path = ( - Path(qualification.__file__).parents[3] - / "docs" - / "evidence" - / "lambada_p60_606_coordinate_qualification_rejected_v1.json" - ) - report_bytes = report_path.read_bytes() - report = json.loads(report_bytes) - - assert hashlib.sha256(report_bytes).hexdigest() == ( - "0993d5a0ad6c0d62094dc395fe2bc4f284870e6e7c0b602be7df5a7da867c93a" - ) - assert documentation_report_path.read_bytes() == report_bytes - assert report["status"] == "rejected" - assert report["source"] == { - "filename": "606_graph_2024-12-03.gt", - "sha256": "c2568cfbecd3f3eb720519be9d042f0cb41606741b8dd2f018bad1c54d36ef85", - "sizeBytes": 12_282_574_483, - } - assert report["graphTool"]["propertyCount"] == 31 - assert len(report["graphTool"]["propertyInventory"]) == 31 - assert report["graphTool"]["hemisphereProperty"] is None - assert len(report["annotationOrientation"]["scores"]) == 8 - assert report["retainedSelection"]["exactBundledAssetMatch"] is True - assert report["retainedSelection"]["outputPoints"] == 71_313 - assert report["retainedSelection"]["outputRuns"] == 11_818 - assert report["sourceCoverage"]["bilateralMirroringQualified"] is False - assert report["sourceCoverage"]["status"] == "unqualified-for-whole-brain-use" - checks = {check["id"]: check["passed"] for check in report["checks"]} - assert checks["ap-dv-sign-margin"] is True - assert checks["source-hemisphere-property-present"] is False - assert checks["source-coverage-whole-brain"] is False - assert report["decision"]["qualifiedMapping"] is None - assert report["decision"]["blockingReasons"] == [ - "SOURCE_HEMISPHERE_PROPERTY_MISSING", - "SOURCE_SPECIMEN_COVERAGE_IS_HEMISPHERE", - ] diff --git a/tests/unit/test_lambada_major_vessels.py b/tests/unit/test_lambada_major_vessels.py deleted file mode 100644 index 92b8244..0000000 --- a/tests/unit/test_lambada_major_vessels.py +++ /dev/null @@ -1,167 +0,0 @@ -from __future__ import annotations - -import hashlib -import json -import shutil -from pathlib import Path - -import numpy as np -import pytest - -from mouse_brain_planner.vasculature.lambada_major_vessels import ( - ASSET_FILENAME, - EXPECTED_RUN_COUNT, - EXPECTED_RUN_POINT_COUNT, - MANDATORY_LIMITATIONS, - MANIFEST_FILENAME, - MINIMUM_RADIUS_UM, - LambadaMajorVesselError, - bundled_asset_paths, - extract_major_vessel_runs, - load_lambada_major_vessels, - write_deterministic_npz, -) - - -def _small_source_arrays() -> tuple[ - np.ndarray[tuple[int, int], np.dtype[np.float64]], - np.ndarray[tuple[int], np.dtype[np.float64]], - np.ndarray[tuple[int], np.dtype[np.int64]], - np.ndarray[tuple[int, int], np.dtype[np.int64]], -]: - coordinates = np.array( - [ - [10.0, 20.0, 30.0], - [11.0, 21.0, 31.0], - [12.0, 22.0, 32.0], - [13.0, 23.0, 33.0], - [14.0, 528.0, 34.0], - [15.0, 25.0, 35.0], - [16.0, 26.0, 36.0], - [17.0, 27.0, 37.0], - [np.nan, 28.0, 38.0], - [19.0, 29.0, 39.0], - [20.0, 30.0, 40.0], - [21.0, 31.0, 41.0], - ], - dtype=np.float64, - ) - radii_atlas_voxel = np.array( - [0.6, 0.7, 0.59, 0.8, 0.9, 0.9, 1.0, 0.7, 0.9, 0.6, 0.7, 0.8], - dtype=np.float64, - ) - annotations = np.arange(100, 112, dtype=np.int64) - edge_indices = np.array([[0, 8], [8, 12]], dtype=np.int64) - return coordinates, radii_atlas_voxel, annotations, edge_indices - - -def test_extracts_maximal_pointwise_runs_without_clipping_or_half_voxel_shift() -> None: - coordinates, radii, annotations, edge_indices = _small_source_arrays() - - data = extract_major_vessel_runs(coordinates, radii, annotations, edge_indices) - - selected = np.array([0, 1, 5, 6, 7, 9, 10, 11]) - expected_points = coordinates[selected][:, [1, 0, 2]].astype(np.float32) - np.testing.assert_array_equal(data.points_asr_voxel_f32, expected_points) - np.testing.assert_array_equal( - data.radii_um_f32, - (radii[selected] * 25.0).astype(np.float32), - ) - np.testing.assert_array_equal( - data.source_annotation_ids_i32, - annotations[selected].astype(np.int32), - ) - np.testing.assert_array_equal(data.run_offsets_i64, [0, 2, 5, 8]) - np.testing.assert_array_equal(data.source_edge_indices_i32, [0, 0, 1]) - - -def test_pointwise_run_extractor_rejects_fractional_edge_ranges() -> None: - coordinates, radii, annotations, _ = _small_source_arrays() - - with pytest.raises(LambadaMajorVesselError, match="must contain integers"): - extract_major_vessel_runs( - coordinates, - radii, - annotations, - np.array([[0.0, 2.0]], dtype=np.float64), - ) - - -def test_deterministic_npz_writer_reproduces_identical_bytes(tmp_path: Path) -> None: - coordinates, radii, annotations, edge_indices = _small_source_arrays() - data = extract_major_vessel_runs(coordinates, radii, annotations, edge_indices) - first = tmp_path / "first.npz" - second = tmp_path / "second.npz" - - first_identity = write_deterministic_npz(first, data) - second_identity = write_deterministic_npz(second, data) - - assert first_identity == second_identity - assert first.read_bytes() == second.read_bytes() - - -def test_bundled_asset_loads_as_immutable_physical_asr_geometry() -> None: - graph = load_lambada_major_vessels() - - assert graph.points_asr_um.shape == (EXPECTED_RUN_POINT_COUNT, 3) - assert graph.radii_um.shape == (EXPECTED_RUN_POINT_COUNT,) - assert graph.run_count == EXPECTED_RUN_COUNT - assert graph.run_offsets.shape == (EXPECTED_RUN_COUNT + 1,) - assert graph.points_asr_um.dtype == np.dtype("= 0.0)) - assert bool(np.all(graph.points_asr_um < np.array([13_200.0, 8_000.0, 11_400.0]))) - assert float(np.min(graph.radii_um)) >= MINIMUM_RADIUS_UM - assert graph.provenance.limitations == MANDATORY_LIMITATIONS - assert not graph.run_points_asr_um(0).flags.writeable - with pytest.raises(TypeError, match="integer"): - graph.run_points_asr_um(True) - with pytest.raises(IndexError, match="outside"): - graph.run_points_asr_um(graph.run_count) - - -def test_loader_rejects_asset_bytes_that_do_not_match_manifest(tmp_path: Path) -> None: - bundled_asset, bundled_manifest = bundled_asset_paths() - copied_asset = tmp_path / ASSET_FILENAME - copied_manifest = tmp_path / MANIFEST_FILENAME - shutil.copyfile(bundled_asset, copied_asset) - shutil.copyfile(bundled_manifest, copied_manifest) - payload = bytearray(copied_asset.read_bytes()) - payload[len(payload) // 2] ^= 0x01 - copied_asset.write_bytes(payload) - - with pytest.raises(LambadaMajorVesselError, match="SHA-256"): - load_lambada_major_vessels(copied_asset, copied_manifest) - - -def test_loader_rejects_removed_mandatory_limitation(tmp_path: Path) -> None: - bundled_asset, bundled_manifest = bundled_asset_paths() - copied_asset = tmp_path / ASSET_FILENAME - copied_manifest = tmp_path / MANIFEST_FILENAME - shutil.copyfile(bundled_asset, copied_asset) - manifest = json.loads(bundled_manifest.read_text(encoding="utf-8")) - manifest["limitations"] = manifest["limitations"][:-1] - copied_manifest.write_text(json.dumps(manifest), encoding="utf-8") - - with pytest.raises(LambadaMajorVesselError, match="limitations"): - load_lambada_major_vessels(copied_asset, copied_manifest) - - -def test_loader_rejects_rehashed_replacement_asset(tmp_path: Path) -> None: - bundled_asset, bundled_manifest = bundled_asset_paths() - copied_asset = tmp_path / ASSET_FILENAME - copied_manifest = tmp_path / MANIFEST_FILENAME - shutil.copyfile(bundled_asset, copied_asset) - payload = bytearray(copied_asset.read_bytes()) - payload[len(payload) // 2] ^= 0x01 - copied_asset.write_bytes(payload) - manifest = json.loads(bundled_manifest.read_text(encoding="utf-8")) - manifest["asset"]["sha256"] = hashlib.sha256(payload).hexdigest() - copied_manifest.write_text(json.dumps(manifest), encoding="utf-8") - - with pytest.raises(LambadaMajorVesselError, match="reviewed build"): - load_lambada_major_vessels(copied_asset, copied_manifest) diff --git a/tests/unit/test_vessap_major_vessels.py b/tests/unit/test_vessap_major_vessels.py index 0e4d878..772985a 100644 --- a/tests/unit/test_vessap_major_vessels.py +++ b/tests/unit/test_vessap_major_vessels.py @@ -1,110 +1,37 @@ from __future__ import annotations -import json -import shutil from pathlib import Path -import numpy as np import pytest from mouse_brain_planner.vasculature.vessap_major_vessels import ( ASSET_FILENAME, - ASSET_SHA256, - EXPECTED_PATH_LENGTH_UM, - EXPECTED_POINT_COUNT, - EXPECTED_RUN_COUNT, - EXPECTED_SEGMENT_COUNT, - MANDATORY_LIMITATIONS, MANIFEST_FILENAME, - MINIMUM_RADIUS_UM, - REGISTRATION_TRANSFORM_ID, - SOURCE_LICENSE, - SOURCE_SPECIMEN_ID, VesSAPMajorVesselError, - bundled_asset_paths, + default_asset_is_available, + default_asset_paths, load_vessap_major_vessels, ) -def test_bundled_vessap_graph_is_exact_immutable_physical_asr_geometry() -> None: - graph = load_vessap_major_vessels() +def test_default_asset_paths_use_application_data_directory( + monkeypatch: pytest.MonkeyPatch, + tmp_path: Path, +) -> None: + monkeypatch.setenv("MOUSE_BRAIN_PLANNER_DATA_DIR", str(tmp_path)) - assert graph.points_asr_um.shape == (EXPECTED_POINT_COUNT, 3) - assert graph.radii_um.shape == (EXPECTED_POINT_COUNT,) - assert graph.run_count == EXPECTED_RUN_COUNT - assert graph.run_offsets.shape == (EXPECTED_RUN_COUNT + 1,) - assert EXPECTED_POINT_COUNT - graph.run_count == EXPECTED_SEGMENT_COUNT - assert graph.points_asr_um.dtype == np.dtype("= 0.0)) - assert bool(np.all(graph.points_asr_um < np.array([13_200.0, 8_000.0, 11_400.0]))) - assert float(np.min(graph.radii_um)) >= MINIMUM_RADIUS_UM - assert graph.provenance.specimen_id == SOURCE_SPECIMEN_ID - assert graph.provenance.license == SOURCE_LICENSE - assert graph.provenance.asset_sha256 == ASSET_SHA256 - assert graph.provenance.registration_transform_id == REGISTRATION_TRANSFORM_ID - assert graph.provenance.limitations == MANDATORY_LIMITATIONS - assert any("display-only" in item for item in graph.provenance.limitations) - assert any("capillaries" in item for item in graph.provenance.limitations) + asset, manifest = default_asset_paths() - deltas = graph.points_asr_um[1:].astype(np.float64) - graph.points_asr_um[:-1].astype( - np.float64 - ) - keep = np.ones(deltas.shape[0], dtype=np.bool_) - keep[graph.run_offsets[1:-1] - 1] = False - assert float(np.linalg.norm(deltas[keep], axis=1).sum()) == pytest.approx( - EXPECTED_PATH_LENGTH_UM, - abs=1e-3, - ) + assert asset == tmp_path / "vasculature" / ASSET_FILENAME + assert manifest == tmp_path / "vasculature" / MANIFEST_FILENAME + assert default_asset_is_available() is False - assert not graph.run_points_asr_um(0).flags.writeable - with pytest.raises(TypeError, match="integer"): - graph.run_points_asr_um(True) - with pytest.raises(IndexError, match="outside"): - graph.run_points_asr_um(graph.run_count) +def test_missing_external_asset_fails_closed( + monkeypatch: pytest.MonkeyPatch, + tmp_path: Path, +) -> None: + monkeypatch.setenv("MOUSE_BRAIN_PLANNER_DATA_DIR", str(tmp_path)) -def test_vessap_loader_rejects_asset_byte_tampering(tmp_path: Path) -> None: - bundled_asset, bundled_manifest = bundled_asset_paths() - copied_asset = tmp_path / ASSET_FILENAME - copied_manifest = tmp_path / MANIFEST_FILENAME - shutil.copyfile(bundled_asset, copied_asset) - shutil.copyfile(bundled_manifest, copied_manifest) - payload = bytearray(copied_asset.read_bytes()) - payload[len(payload) // 2] ^= 0x01 - copied_asset.write_bytes(payload) - - with pytest.raises(VesSAPMajorVesselError, match="SHA-256"): - load_vessap_major_vessels(copied_asset, copied_manifest) - - -def test_vessap_loader_rejects_relaxed_display_only_limitations(tmp_path: Path) -> None: - bundled_asset, bundled_manifest = bundled_asset_paths() - copied_asset = tmp_path / ASSET_FILENAME - copied_manifest = tmp_path / MANIFEST_FILENAME - shutil.copyfile(bundled_asset, copied_asset) - manifest = json.loads(bundled_manifest.read_text(encoding="utf-8")) - manifest["limitations"] = manifest["limitations"][:-1] - copied_manifest.write_text(json.dumps(manifest), encoding="utf-8") - - with pytest.raises(VesSAPMajorVesselError, match="limitations"): - load_vessap_major_vessels(copied_asset, copied_manifest) - - -def test_vessap_loader_rejects_transform_identity_changes(tmp_path: Path) -> None: - bundled_asset, bundled_manifest = bundled_asset_paths() - copied_asset = tmp_path / ASSET_FILENAME - copied_manifest = tmp_path / MANIFEST_FILENAME - shutil.copyfile(bundled_asset, copied_asset) - manifest = json.loads(bundled_manifest.read_text(encoding="utf-8")) - manifest["registration"]["mapping"] = "unreviewed replacement" - copied_manifest.write_text(json.dumps(manifest), encoding="utf-8") - - with pytest.raises(VesSAPMajorVesselError, match="registration evidence"): - load_vessap_major_vessels(copied_asset, copied_manifest) + with pytest.raises(VesSAPMajorVesselError, match="manifest must be a regular file"): + load_vessap_major_vessels()