Add V_zeta schema version with Brainstorm I architecture#63
Merged
Conversation
…ed observations) Adds schemas/V_zeta, the sandbox iteration implementing Brainstorm I from ndi-next-steps Summer 2026/1_Ingestion, superseding V_epsilon (Brainstorm E, retained on main as an archived reference). Full coverage set: the documents needed to express every existing V_delta document. Core Brainstorm-I model (vs V_epsilon): - One spine. subject_interaction (<- base) carries the whole identity/where/when payload: subject_id, time_reference_# (>=1), method (verb), variable (noun, required), target_structure (optional locus, Path T). observation / manipulation / annotation are empty direction markers; isa replaces a direction filter. subject_statement removed; subject_assertion re-rooted under base. - Off-class identity. The observation tier is named by DATA-TYPE (shape), not property: 12 dimensional scalar_<dim>_observation leaves + a generic escape hatch replace E's ~20 property classes, and categorical_observation collapses to one concrete class. WHICH property rides on the `variable` term (the answer to E's EPM enumeration failure). The shared shape library (scalar_mass ... scalar_score, generic_scalar) is inherited by both the observation that reads a quantity and the manipulation that imposes it. - Shaped time. time_reference gains a `sampling` structure (point / grid / enumerated); per-sample times are derived from the shaped reference, so sample_time is retired from the scalar genera and regular grids are O(1). - dataseries_observation branch (timeseries_/imageseries_observation) retained as the observation-tier handle for acquired data, with the record/array channel model + dataseries_channel_map sidecar. Manipulation families, time_reference frames, dataseries/expression bodies, all infrastructure, and the five deprecated classes carry over from V_epsilon (design-neutral for I); the spine-owned target_structure is stripped where a family redeclared it. hartley_calc's long-standing malformed file record is fixed so the whole set passes meta-validation. 202 document classes (156 stable, 40 draft, 6 deprecated) + 3 meta. index.json and topics.json regenerated from disk; V_zeta_SPEC.md + V_zeta_notes.md document the deltas and the resolved open questions (Path T locus, method/variable, shaped time; both spine-shaping forks confirmed with the maintainer). New tests/test_vzeta.py enforces meta-schema validity, index<->disk agreement, superclass/dependency resolution, tier/maturity agreement, and spine composition. 515 tests pass (104 prior + 411 new). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015zjbvyLbSLMYz1uCizYwQh
…he only production source The production document set is did_v1 (our first version); V_alpha..V_epsilon were sandbox iterations that never shipped, so conversions/from_did_v1/ is the sole conversion tree and there is nothing to migrate from the intermediate sandboxes. - index.json: legacy_schema_version_values -> ["did_v1"] (was ["did_v1", "V_delta"]); notes clarify did_v1 is the only production/legacy version. - Retarget the whole from_did_v1 tree from V_epsilon (Brainstorm E) to V_zeta (Brainstorm I): version labels, schema paths (families are stable/ in V_zeta), and the target model in prose. - treatment.md: fix the schema-level bug — thermal identity mapped to the retired scalar_manipulation.applied_property; in the I model identity is the spine `variable` and the verb is the spine `method`. Reworked the worked example accordingly (identity/verb/target_structure on the subject_interaction block; value as a typed array; scalar_manipulation carries only `notes`). - ontology_table_row.md (previous commit) already rewritten for the shape-typed observation tier; _index.md and the format/template docs (_universal_renames, _files, _TEMPLATE) retargeted to V_zeta, preserving the true V_alpha/beta/gamma format lineage. - V_zeta_notes.md: correct the conversion follow-up (no from_v_epsilon path; from_did_v1 only) and the legacy-value note. 515 tests pass; full set still validates against the meta-schema. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015zjbvyLbSLMYz1uCizYwQh
…add generic_manipulation In Brainstorm I a manipulation is a class only when it adds STRUCTURE (a typed value, a dependency, or an invariant) — identity rides on the spine `variable`. procedural_manipulation and environmental_manipulation carried only an ontology identity field (procedure/factor) + notes, i.e. no distinct structure, so they are pure-identity classes the framework forbids (the manipulation-side mirror of the EPM property-class problem on the observation tier). - Delete procedural_manipulation and environmental_manipulation. - Add generic_manipulation (concrete, <- manipulation): the escape-hatch leaf for payload-free acts (surgical/physical procedures, environmental/husbandry regimes, behavioral training). No typed value; the act is named by the spine `variable`, its ontology branch carrying the coarse kind. Manipulation-side analog of generic_scalar_observation. - Move the shared `notes` prose up to the abstract `manipulation` base so every manipulation inherits it (removed the duplicate from scalar_manipulation). - Re-parent biological_transfer onto `manipulation` (it earns its class via the donor_id dependency). Docs/tests updated: V_zeta_SPEC.md §5 rewritten around "classes earn their place by structure"; treatment.md / treatment_transfer.md / _index.md route procedures and regimes to generic_manipulation and drop the removed classes; V_zeta_notes.md change-list gains the manipulation-tier point; test_vzeta.py asserts the two classes are gone, generic_manipulation is a concrete manipulation leaf, biological_transfer re-parents with donor_id, and notes is inherited. index.json + topics.json regenerated (topics generation made idempotent; the tuning/vm classes V_epsilon left uncategorized are now filed). 201 document classes (155 stable, 40 draft, 6 deprecated) + 3 meta. 514 tests pass; full set validates against the meta-schema. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015zjbvyLbSLMYz1uCizYwQh
The transferred material was carried in biological_transfer.entity, which duplicated the spine `variable` (the noun/identity). Remove it: biological_transfer is now `donor_id` (dependency) + `kind` (coarse bucket) — the structure that earns its class — with the specific material named by the spine `variable`. Updates the treatment_transfer conversion mapping (entity term -> variable). class_version 3.0.0. 514 tests pass; full set validates. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015zjbvyLbSLMYz1uCizYwQh
…t session_id edge Two Brainstorm-I spine refinements: 1. element_id (individuated referent). subject_interaction gains an optional element_id -> element dependency, all the way up on the abstract spine so every observation/manipulation leaf inherits it. It names the specific element that is part of the subject (an ndi.neuron, a probe, a derived signal) when that entity has its own document identity and is neither a subject_group nor an anatomy term -- the case target_structure (a KIND of place) cannot address. Completes the spine referent set: subject_id (whole specimen) / target_structure (ontological kind of locus) / element_id (which specific one). Optional; identity stays off the class (the EPM lesson). 2. session_relative_reference declares no session_id edge. Session identity rides on base.session_id (every document carries it), so the ordinal anchor needs no session_id dependency. The edge was redundant with base and produced only discovery-mode reference-integrity orphans -- the migrators synthesize thousands of these anchors and the session document is absent from a corpus dump, so each dangled (~41k JH, ~8k Dab). The real ndi.migrate.local path is unaffected (it referenced the session via base the same way). SPEC (new S3.1), notes, and tests updated; V_zeta suite green (412). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015zjbvyLbSLMYz1uCizYwQh
…r_image_epochdata_ingested Retire the standalone image document onto NDI's imaging stack (the model NDI-main ships: ndi.element.image / ndi.probe.image, frames timed through the epoch clock; 'no_time' for a clockless stack). Three roles replace the one doc: - imageseries_observation -- the discoverable spine handle (subject_id + shaped time_reference + variable/kind = modality + element_id; caption is dataseries_observation.label, added here). - daqreader_image_epochdata_ingested -- the DIGITAL, in-database pixels: a frames.bin raw binary + a queryable YXCZT header (dimension_order/size, data_type, num_frames, frametimes, clocktype). This is exactly what ndi.daq.reader.image.ingest_epochfiles writes -- once ingested the image is digital data resident in the database, no external file dependency. The class lived in NDI's ndi_common but was ABSENT from did-schema; ported into V_zeta (stable), mirroring the daqreader_mfdaq_epochdata_ingested sibling. - ndi.element.image + element_epoch -- the element the frames belong to. Changes: - image_stack, image_stack_parameters -> deprecated/ (index + topics updated). - new stable/daqreader_image_epochdata_ingested.json. - element_epoch.storage + dataseries_data.storage gain data_type + data_limits. - dataseries_data gains the element_id link its siblings (ngrid / image_zarr / dataseries_pyramid) already carry. - dataseries_observation gains an optional label (the folded image_stack caption). - SPEC S4.1 + notes document the fold; test_vzeta asserts it. Suite: 415 pass. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015zjbvyLbSLMYz1uCizYwQh
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR introduces the V_zeta schema version, implementing the Brainstorm I architecture for the NDI (Neurodata Data Interface) specification. V_zeta supersedes V_epsilon and represents a major restructuring of the schema around a unified subject_interaction spine with shaped time references, observation/manipulation direction, and a new dataseries_observation branch.
Key Changes
variableontology termNotable Implementation Details
https://claude.ai/code/session_015zjbvyLbSLMYz1uCizYwQh