What to build
Land ADR-0007 in docs/adr/ (and a companion reference at docs/references/aperture-bigquery-mapping.md) that:
-
Documents the field-mapping contract with the Aperture BigQuery ingest team (schema in iac-aperture/deploy/arch/bigquery.tf). For every BigQuery column, states one of:
zarrmony-supplied — column source in the audit (path into attrs.zarrmony) and which reader paths populate it
user-supplied — the ingest form provides it, zarrmony does not attempt extraction
derived — the ingest layer computes it from a zarrmony field (e.g. uri_ome_zarr from the output path)
-
Decides the audit shape for the new instrument-metadata block that follow-on slices populate. Open sub-decisions to resolve in the ADR body:
- Per-scene vs file-level placement for
acquisition_date, microscope (model), microscope_serial, imaging_method. Recommendation: per-scene consistently (values are simply duplicated across scenes when constant) so consumers only read from one place.
- Shape of
per_scene[i].channels — the identity block projecting existing extract_channels output for LIF and reader.ome_metadata.images[0].pixels.channels for CZI/ND2/OME-TIFF into one common surface (dye, fluor, excitation_nm, emission_low_nm, emission_high_nm).
- Whether
ome_ngff_version lands under config or a new top-level output block (folded in from the earlier standalone slice — trivial addition, but the ADR names its home).
-
Publishes a per-format extractability matrix so downstream ingest knows what to expect. Rows: BigQuery columns marked zarrmony-supplied. Columns: LIF, CZI, ND2, default (OME-TIFF + other bioio plugins). Each cell: ✓ / ✗ / partial with a one-line note. Flags fields present in some readers but absent in others (e.g. objective in LIF today, absent in CZI/ND2/default until the follow-on slices land).
-
Calls out channel_targets specifically — the ingest form is the primary source (RNAscope target genes, antibody names). Zarrmony populates it only when the source encodes probe/target names into channel names (occasional in Leica RNAscope acquisitions). The mapping doc flags this so ingest doesn't over-trust zarrmony's output.
Motivation
Another team is building the Aperture metadata parser/gate (iac-aperture) that will trigger data transfer, launch zarrmony conversion, and merge instrument metadata with user-supplied biological metadata in BigQuery. Today zarrmony's audit surfaces a fraction of what BigQuery wants; the rest sits in the input file and is discarded. Before writing per-format extractors we need a contract with the ingest team so both sides agree on shape, ownership, and per-format coverage.
Acceptance criteria
Blocked by
None — can start immediately.
What to build
Land ADR-0007 in
docs/adr/(and a companion reference atdocs/references/aperture-bigquery-mapping.md) that:Documents the field-mapping contract with the Aperture BigQuery ingest team (schema in
iac-aperture/deploy/arch/bigquery.tf). For every BigQuery column, states one of:zarrmony-supplied— column source in the audit (path intoattrs.zarrmony) and which reader paths populate ituser-supplied— the ingest form provides it, zarrmony does not attempt extractionderived— the ingest layer computes it from a zarrmony field (e.g.uri_ome_zarrfrom the output path)Decides the audit shape for the new instrument-metadata block that follow-on slices populate. Open sub-decisions to resolve in the ADR body:
acquisition_date,microscope(model),microscope_serial,imaging_method. Recommendation: per-scene consistently (values are simply duplicated across scenes when constant) so consumers only read from one place.per_scene[i].channels— the identity block projecting existingextract_channelsoutput for LIF andreader.ome_metadata.images[0].pixels.channelsfor CZI/ND2/OME-TIFF into one common surface (dye,fluor,excitation_nm,emission_low_nm,emission_high_nm).ome_ngff_versionlands underconfigor a new top-leveloutputblock (folded in from the earlier standalone slice — trivial addition, but the ADR names its home).Publishes a per-format extractability matrix so downstream ingest knows what to expect. Rows: BigQuery columns marked
zarrmony-supplied. Columns: LIF, CZI, ND2, default (OME-TIFF + other bioio plugins). Each cell:✓/✗/partialwith a one-line note. Flags fields present in some readers but absent in others (e.g. objective in LIF today, absent in CZI/ND2/default until the follow-on slices land).Calls out
channel_targetsspecifically — the ingest form is the primary source (RNAscope target genes, antibody names). Zarrmony populates it only when the source encodes probe/target names into channel names (occasional in Leica RNAscope acquisitions). The mapping doc flags this so ingest doesn't over-trust zarrmony's output.Motivation
Another team is building the Aperture metadata parser/gate (
iac-aperture) that will trigger data transfer, launch zarrmony conversion, and merge instrument metadata with user-supplied biological metadata in BigQuery. Today zarrmony's audit surfaces a fraction of what BigQuery wants; the rest sits in the input file and is discarded. Before writing per-format extractors we need a contract with the ingest team so both sides agree on shape, ownership, and per-format coverage.Acceptance criteria
docs/adr/following the existing ADR template style.docs/references/aperture-bigquery-mapping.md(or wherever fits the repo's docs convention) enumerating all BigQuery columns fromiac-aperture/deploy/arch/bigquery.tfwith status + source.per_scene[i].acquisitionandper_scene[i].channelsand file-vs-scene placement decisions locked.ome_ngff_versionhome decided (folded in from earlier slice).channel_targetsprovenance rules called out explicitly.Blocked by
None — can start immediately.