Each array data type is associated with a scalar type. Those scalar values, i.e. the individual elements we write to arrays, have two representations described by the Zarr specs:
- as elements of arrays, stored in chunks as encoded bytes.
- as values associated with the
fill_value key in metadata, in which case the value must be composed of JSON types.
There are many cases where users need to include scalars in array metadata. For example, if a codec takes an array of 64-bit floats as part of its configuration, or if a user wants to store an affine transformation in the attributes field, they would need a routine for storing these values in metadata.
Zarr itself has this problem with the fill_value field, which it solves in a very narrow fashion by defining a "fill value encoding" for each scalar type. But the specs could more generally define a JSON encoding for each scalar type, and then state that, e.g., "the fill_value field must be a scalar consistent with the array data type encoding uses the JSON scalar encoding defined with that data type".
Of course users today can just use language like "value X is saved in Zarr metadata using the fill value encoding for that data type", but I think it would be valuable to people if we re-arranged the language so that the fill value encoding was described as an application of a more generic JSON encoding for scalars.
As a bonus, if we formally define a JSON encoding for scalar values, it's also easy to define a JSON encoding for n-dimensional arrays of scalar values, by arranging JSON-encoded scalars into appropriately structured JSON arrays. But I don't think we need to include this language in the spec.
so: I'm proposing that we change the currentl language about fill_value encoding to instead describe JSON encoding for scalars, which is then applied to the fill_value field.
Each array data type is associated with a scalar type. Those scalar values, i.e. the individual elements we write to arrays, have two representations described by the Zarr specs:
fill_valuekey in metadata, in which case the value must be composed ofJSONtypes.There are many cases where users need to include scalars in array metadata. For example, if a codec takes an array of 64-bit floats as part of its configuration, or if a user wants to store an affine transformation in the
attributesfield, they would need a routine for storing these values in metadata.Zarr itself has this problem with the
fill_valuefield, which it solves in a very narrow fashion by defining a "fill value encoding" for each scalar type. But the specs could more generally define a JSON encoding for each scalar type, and then state that, e.g., "thefill_valuefield must be a scalar consistent with the array data type encoding uses the JSON scalar encoding defined with that data type".Of course users today can just use language like "value
Xis saved in Zarr metadata using the fill value encoding for that data type", but I think it would be valuable to people if we re-arranged the language so that the fill value encoding was described as an application of a more generic JSON encoding for scalars.As a bonus, if we formally define a
JSONencoding for scalar values, it's also easy to define aJSONencoding for n-dimensional arrays of scalar values, by arrangingJSON-encoded scalars into appropriately structuredJSONarrays. But I don't think we need to include this language in the spec.so: I'm proposing that we change the currentl language about
fill_valueencoding to instead describeJSONencoding for scalars, which is then applied to thefill_valuefield.