V_epsilon: implement Brainstorm E observation tier + shared shape library

schemas/V_epsilon/conversions/from_did_v1/_index.md

@@ -45,6 +45,12 @@ field-level changes on top.
 | `hartley_calc` | NDIcalc-vis-matlab `calc/hartley_calc` | drafted | [hartley_calc.md](hartley_calc.md) |
 | `probe_location` | legacy NDI/DID `probe_location` (V_alpha shape) | drafted | [probe_location.md](probe_location.md) |
 | `treatment` | legacy NDI/DID `treatment` (V_alpha shape) | drafted | [treatment.md](treatment.md) |
+| `treatment` → manipulation tiers (Brainstorm E **split**) | legacy `treatment` → `injection`/`bath`/`procedural_manipulation`/`temperature_manipulation`/`environmental_manipulation` (+ out-of-tier) | drafted | [treatment.md](treatment.md) |
+| `ontology_table_row` → observation tiers (Brainstorm E **split**, 1→N) | legacy `ontology_table_row` → scalar/categorical observation property classes (+ out-of-tier) | drafted | [ontology_table_row.md](ontology_table_row.md) |
+| `subject_group` → `subject` (`is_group`) (Brainstorm E **fold**) | legacy `subject_group` → `subject` flagged `is_group` (+ `group_assignment` in NDI layer) | drafted | [subject_group.md](subject_group.md) |
+| `treatment_drug` → `injection` (`kind: drug`) (Brainstorm E **fold**) | legacy `treatment_drug` → `injection` (mixture from `mixture_table`) | drafted | [treatment_drug.md](treatment_drug.md) |
+| `virus_injection` → `injection` (`kind: virus`) (Brainstorm E **fold**) | legacy `virus_injection` → `injection` (virus+dilution in mixture) | drafted | [virus_injection.md](virus_injection.md) |
+| `treatment_transfer` → `biological_transfer` (Brainstorm E **fold**) | legacy `treatment_transfer` → `biological_transfer` (recipient→subject, donor carried) | drafted | [treatment_transfer.md](treatment_transfer.md) |
 | `ontology_image` | legacy NDI/DID `ontologyImage` (V_alpha shape) | drafted | [ontology_image.md](ontology_image.md) |
 | `ontology_label` | legacy NDI/DID `ontologyLabel` (V_alpha shape) | drafted | [ontology_label.md](ontology_label.md) |
 

schemas/V_epsilon/conversions/from_did_v1/ontology_table_row.md

@@ -0,0 +1,120 @@
+# Conversion: did_v1 → V_epsilon — `ontology_table_row` → observation tiers (Brainstorm E split)
+
+> The observation-tier companion to [`treatment.md`](treatment.md). The legacy `ontology_table_row` open key/value bag is **retired** and **split** across the Brainstorm E observation property classes (and, for non-observation rows, out of the tier). This is the genuinely **one-to-many** migration: one `ontology_table_row` document → N destination documents.
+
+## Identity
+
+- **Target tier:** V_epsilon observation families (`schemas/V_epsilon/draft/`).
+- **did_v1 source:** legacy NDI/DID `ontology_table_row` (open table of `(ontology term, value[, unit])` rows attached to a subject at a time).
+- **Status:** `drafted` — needs the 1→N migrator capability (see [Engine note](#engine-note)) and the per-term dispatch table (finalized in discovery mode).
+- **Cardinality:** **1 → N.** Each row dispatches independently; a single source document fans out to several destinations across tiers.
+
+## Summary
+
+Each row names a property (an ontology term) and a value. Brainstorm E dispatches per row by **what the property is** and **what shape its value takes**:
+- a **scalar** measurement → the matching scalar property class (`body_weight_observation`, `core_temperature_observation`, …), value as the typed composite;
+- a **categorical** label → the matching categorical property class (`developmental_stage_observation`, …), value as a bound `ontology_term`;
+- a **subject-intrinsic, set-once** fact (species, strain, sex) → **not an observation** → the `subject` document / `openminds_subject`;
+- a **relational/assigned** fact (cohort, housing, derivation source) → the relevant **event/annotation** class (`group_assignment`, `placement`, `derivation`);
+- a fact with **no minted property class yet** → the escape hatch (`generic_scalar_observation` / `generic_categorical_observation`), promoted later.
+
+## Per-row dispatch table
+
+First match wins, on the row's property term (branch) and value shape.
+
+| Row property (branch) + value shape | Destination | Mapping |
+|---|---|---|
+| Mass-dimensioned, numeric (body weight, brain/tumor mass) | `body_weight_observation` (`value : scalar_mass`) | term → `measured_property`; structure → `target_structure`; value+unit → `scalar_mass` composite |
+| Length-dimensioned | `body_length_observation` (`scalar_length`) | as above |
+| Duration / age | `age_observation` (`scalar_duration`) | |
+| Temperature | `core_temperature_observation` (`scalar_temperature`) | |
+| Frequency (heart/respiration rate) | `heart_rate_observation` / `respiration_rate_observation` (`scalar_frequency`) | |
+| Pressure (BP, IOP, partial pressure) | `blood_pressure_observation` (`scalar_pressure`) | |
+| Count (litter size, cell count) | `litter_size_observation` / `cell_count_observation` (`scalar_count`) | |
+| Score (body condition, behavioral) | `body_condition_observation` / `behavioral_score_observation` (`scalar_score`) | |
+| Concentration (glucose, cortisol, titer) | `concentration_observation` (`scalar_concentration`) | |
+| Voltage (transcribed Vm) | `membrane_potential_observation` (`scalar_voltage`) | |
+| **Scalar with no minted property class** | `generic_scalar_observation` (`generic_scalar`) | value → `{source_unit, source_value, approximate}`; promote later |
+| Categorical: developmental/life stage | `developmental_stage_observation` (life/developmental stage under `UBERON:0000105`, by convention — not schema-enforced) | term → `value`; property → `measured_property` |
+| Categorical: health status / coat / estrous / behavioral label | `health_status_observation` / `pigmentation_observation` / `estrous_stage_observation` / `behavioral_phenotype_observation` | term → `value` |
+| **Categorical with no minted property class** | `generic_categorical_observation` (free `ontology_term`, no binding) | term → `value`; promote later |
+| **Subject-intrinsic, set-once** (species, strain, sex, DOB) | **out of tier** → `subject` / `openminds_subject` (merge) | not an observation |
+| **Relational / assigned** (cohort, housing, derivation source) | **out of tier** → `group_assignment` / `placement` / `derivation` | event/annotation tiers |
+| Empty / unresolvable term | **curator review queue** (routing off) | flagged |
+
+## Common field mapping (per generated observation)
+
+| did_v1 row element | V_epsilon field | Transformation |
+|---|---|---|
+| row property term | `measured_property` (inherited from `observation`) | `ontology_term` |
+| anatomical qualifier (if any) | `target_structure` | `ontology_term[]`; else `[]` |
+| row value (+ unit) | `value` on the destination block | scalar: typed composite in the shape-mixin block (`scalar_mass`, …). categorical: bound `ontology_term` in the **concrete class's own block** (`categorical_concept.value` has `placement: concrete_class`, so `categorical_concept` contributes no block) |
+| source `subject_id` | inherited `subject_id` | identity, shared by all generated docs |
+| source time anchor | inherited `time_reference_#` | one **`session_relative_reference`** (`relation: during`, → `session` from `base.session_id`) is emitted and **shared** by all observations from the table (the whole row burst is one session); each obs `time_reference_1` points at it |
+
+## Engine note
+
+This conversion is **1 → N+1** (N observations + one shared session anchor). The 1→N capability **is implemented**: `did2.convert.v1_to_v2` (under `TargetVersion='V_epsilon'`) routes the class to `+did2.+convert.+migrators_e.ontology_table_row`, which may return a cell of bodies; the dispatcher lands each. The migrator emits one observation per row (each with a fresh `did.ido.unique_id()`), all sharing the source `subject_id` and one `session_relative_reference` anchor, and routes non-observation rows out of tier.
+
+## Worked example — a 2-row intake burst → two observations
+
+### Before (did_v1, one document, two rows)
+```json
+{
+    "document_class": { "class_name": "ontology_table_row", "class_version": "1.0.0",
+        "superclasses": [ { "class_name": "base", "class_version": "1.0.0" } ] },
+    "depends_on": [ { "name": "subject_id", "document_id": "aabb1122ccdd3344_aabb1122ccdd3344" } ],
+    "base": { "id": "aabb1122ccdd3344_1122334455667788", "session_id": "aabb1122ccdd3344_9900aabbccddeeff",
+        "name": "intake_row", "datestamp": "2024-05-15T09:30:00.000Z" },
+    "ontology_table_row": {
+        "rows": [
+            { "ontologyName": "schema:weight", "name": "weight", "value": 22.5, "unit": "g" },
+            { "ontologyName": "uberon:0000105", "name": "life cycle stage", "value": "FBdv:00005336" }
+        ]
+    }
+}
+```
+
+### After (V_epsilon, two documents)
+```json
+[
+  {
+    "document_class": { "class_name": "body_weight_observation", "class_version": "1.0.0",
+        "superclasses": [ { "class_name": "scalar_observation" }, { "class_name": "scalar_mass" } ] },
+    "depends_on": [
+        { "name": "subject_id",       "value": "aabb1122ccdd3344_aabb1122ccdd3344" },
+        { "name": "time_reference_1", "value": "aabb1122ccdd3344_synthesized" } ],
+    "base": { "id": "aabb1122ccdd3344_aaaa000000000001", "session_id": "aabb1122ccdd3344_9900aabbccddeeff",
+        "name": "intake_weight", "datestamp": "2024-05-15T09:30:00.000Z" },
+    "observation": { "measured_property": { "node": "schema:weight", "name": "weight" }, "target_structure": [] },
+    "scalar_mass": { "value": { "kilograms": 0.0225, "source_unit": "g", "source_value": 22.5, "approximate": false } }
+  },
+  {
+    "document_class": { "class_name": "developmental_stage_observation", "class_version": "1.0.0",
+        "superclasses": [ { "class_name": "categorical_observation" }, { "class_name": "categorical_concept" } ] },
+    "depends_on": [
+        { "name": "subject_id",       "value": "aabb1122ccdd3344_aabb1122ccdd3344" },
+        { "name": "time_reference_1", "value": "aabb1122ccdd3344_synthesized" } ],
+    "base": { "id": "aabb1122ccdd3344_aaaa000000000002", "session_id": "aabb1122ccdd3344_9900aabbccddeeff",
+        "name": "intake_stage", "datestamp": "2024-05-15T09:30:00.000Z" },
+    "observation": { "measured_property": { "node": "uberon:0000105", "name": "life cycle stage" }, "target_structure": [] },
+    "developmental_stage_observation": { "value": { "node": "fbdv:00005336", "name": "larval stage" } }
+  }
+]
+```
+
+## Real v1 row layout (implemented)
+
+The dominant `ontology_table_row` form is **parallel char fields** — comma-separated `names`, `variable_names`, `ontology_nodes` plus a `data` struct keyed by `variable_names`. One document is one table **row**; each **column** is a measured property. The migrator splits it accordingly: column *i* → one observation (`measured_property` = `ontology_nodes[i]` + `names[i]`; value = `data.(variable_names[i])`), dispatched scalar-vs-categorical on the value type. Columns with no usable value (missing key, empty, `NaN`) are skipped. Anything whose layout still isn't recognised falls back to migrating unchanged as an `ontology_table_row` (the class exists in V_epsilon) rather than quarantining.
+
+## Open questions
+
+- **Per-term dispatch table** — the property-term → property-class map is finalized in discovery mode against real corpora (the same report-only loop as `treatment`). Terms with no minted class fall to the generic escape hatches.
+- **New `base.id` per generated doc** — N new documents need N stable ids; define the derivation (e.g. source id + row index) so re-runs are idempotent.
+- **1→N engine capability** — prerequisite; see [Engine note](#engine-note).
+
+## Cross-references
+
+- Manipulation-tier split: [`treatment.md`](treatment.md)
+- Universal renames: [`_universal_renames.md`](_universal_renames.md)
+- Design sources (ndi-next-steps): `Scalar_Observation_Proposal.md`, `Categorical_Observation_Proposal.md`, `20260615/Brainstorm_E_Class_Catalog.md` §3, `Pulakat_Datatype_Homes.md`.

schemas/V_epsilon/conversions/from_did_v1/subject_group.md

@@ -0,0 +1,42 @@
+# `subject_group` → `subject` (`is_group: true`)  [Brainstorm E]
+
+Status: **drafted**
+
+`subject_group` is **deprecated** in V_epsilon (see `V_epsilon_SPEC.md`,
+"Deprecated / folded" table). A group of subjects is modeled as a
+`subject` flagged `is_group: true`, and membership becomes
+`group_assignment` events.
+
+## Mapping (per document, 1 → 1)
+
+| did_v1 `subject_group` | V_epsilon `subject` | Transformation |
+|---|---|---|
+| (class) `subject_group` | (class) `subject`, `is_group: true` | class fold |
+| `subject_group.group_name` (optional; absent in v1 corpus) | `subject.local_identifier` | char copy |
+| `subject_group.description` (optional) | `subject.description` | char copy |
+| — | `subject.is_biological` | `false` (a group is not a biological individual) |
+| `base.*` | `base.*` | carried unchanged (same document id) |
+
+The legacy `subject_group` document is an (essentially empty) marker; it
+carries **no members** — membership is expressed by member subjects
+referencing the group. So the per-document migration produces only the
+group `subject`.
+
+## Membership → `group_assignment` (relational, NDI layer)
+
+In V_epsilon, member→group membership is event-sourced as `group_assignment`
+annotations (member `subject_id` + `group_id`). **However, did_v1 does not
+record group membership anywhere**: the `subject_group` body is empty and no
+v1 document depends on a `subject_group` (verified across the did_v1 document
+set). There is therefore **no v1 source to migrate into `group_assignment`**
+— synthesizing edges would invent data. The migration is complete with the
+group `subject` alone; `group_assignment` is a forward-looking class for
+newly authored data, not a migration target. If a specific corpus turns out
+to encode membership out-of-band (e.g., a lab-specific table), that becomes
+a targeted, corpus-specific NDI-layer pass at that time.
+
+## Engine
+
+Routed by `did2.convert.v1_to_v2` under `TargetVersion='V_epsilon'` to
+`+did2/+convert/+migrators_e/subject_group.m`. A 1 → 1 fold; the default
+`V_delta` target is unaffected (it keeps `subject_group` as-is).