From ba2ad18f86424208e17d18c3ced13c1e1e9a5177 Mon Sep 17 00:00:00 2001 From: Davis Vann Bennett Date: Fri, 11 Jul 2025 11:35:38 +0200 Subject: [PATCH 1/3] update release note for dtype refactor --- changes/2874.feature.rst | 24 +++++++++++++++++------- 1 file changed, 17 insertions(+), 7 deletions(-) diff --git a/changes/2874.feature.rst b/changes/2874.feature.rst index 4c50532ae0..e5432ff53f 100644 --- a/changes/2874.feature.rst +++ b/changes/2874.feature.rst @@ -1,9 +1,19 @@ -Adds zarr-specific data type classes. This replaces the internal use of numpy data types for zarr -v2 and a fixed set of string enums for zarr v3. This change is largely internal, but it does -change the type of the ``dtype`` and ``data_type`` fields on the ``ArrayV2Metadata`` and -``ArrayV3Metadata`` classes. It also changes the JSON metadata representation of the -variable-length string data type, but the old metadata representation can still be -used when reading arrays. The logic for automatically choosing the chunk encoding for a given data -type has also changed, and this necessitated changes to the ``config`` API. +Adds zarr-specific data type classes. + +This change adds a ``ZDType`` base class for Zarr V2 and Zarr V3 data types. Child classes are +defined for each NumPy data type. Each child class defines routines for ``JSON`` serialization. +New data types can be created and registered dynamically. + +Prior to this change, Zarr Python had two streams for handling data types. For Zarr V2 arrays, +we used NumPy data type identifiers. For Zarr V3 arrays, we used a fixed set of string enums. Both +of these systems proved hard to extend. + +This change is largely internal, but it does change the type of the ``dtype`` and ``data_type`` +fields on the ``ArrayV2Metadata`` and ``ArrayV3Metadata`` classes. Previously, ``ArrayV2Metadata.dtype`` +was a NumPy ``dtype`` object, and ``ArrayV3Metadata.data_type`` was an internally-defined ``enum``. +After this change, both ``ArrayV2Metadata.dtype`` and ``ArrayV3Metadata.data_type`` are instances of +``ZDType``. A NumPy data type can be generated from a ``ZDType`` via the ``ZDType.to_native_dtype()`` +method. The internally-defined Zarr V3 ``enum`` class is gone entirely, but the ``ZDType._zarr_v3_name`` +attribute contains the string value previously associated with that ``enum``. For more on this new feature, see the `documentation `_ \ No newline at end of file From 80d6679f32ef973ed61baa15e749d3ab54212f42 Mon Sep 17 00:00:00 2001 From: Davis Vann Bennett Date: Fri, 11 Jul 2025 12:05:59 +0200 Subject: [PATCH 2/3] public API for getting the zarr v3 name --- changes/2874.feature.rst | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/changes/2874.feature.rst b/changes/2874.feature.rst index e5432ff53f..56aa8548f9 100644 --- a/changes/2874.feature.rst +++ b/changes/2874.feature.rst @@ -13,7 +13,8 @@ fields on the ``ArrayV2Metadata`` and ``ArrayV3Metadata`` classes. Previously, ` was a NumPy ``dtype`` object, and ``ArrayV3Metadata.data_type`` was an internally-defined ``enum``. After this change, both ``ArrayV2Metadata.dtype`` and ``ArrayV3Metadata.data_type`` are instances of ``ZDType``. A NumPy data type can be generated from a ``ZDType`` via the ``ZDType.to_native_dtype()`` -method. The internally-defined Zarr V3 ``enum`` class is gone entirely, but the ``ZDType._zarr_v3_name`` -attribute contains the string value previously associated with that ``enum``. +method. The internally-defined Zarr V3 ``enum`` class is gone entirely, but the ``ZDType.to_json(zarr_format=3)`` +method can be used to generate a dictionary that has a ``name`` field that contains the string value +previously associated with that ``enum``. For more on this new feature, see the `documentation `_ \ No newline at end of file From be3e51a0ac7a63046805c174a8f0fe4e99a6b670 Mon Sep 17 00:00:00 2001 From: Davis Vann Bennett Date: Fri, 11 Jul 2025 12:08:15 +0200 Subject: [PATCH 3/3] clarify that to_json(zarr_format=3) might return a string --- changes/2874.feature.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/changes/2874.feature.rst b/changes/2874.feature.rst index 56aa8548f9..093f566f74 100644 --- a/changes/2874.feature.rst +++ b/changes/2874.feature.rst @@ -14,7 +14,7 @@ was a NumPy ``dtype`` object, and ``ArrayV3Metadata.data_type`` was an internall After this change, both ``ArrayV2Metadata.dtype`` and ``ArrayV3Metadata.data_type`` are instances of ``ZDType``. A NumPy data type can be generated from a ``ZDType`` via the ``ZDType.to_native_dtype()`` method. The internally-defined Zarr V3 ``enum`` class is gone entirely, but the ``ZDType.to_json(zarr_format=3)`` -method can be used to generate a dictionary that has a ``name`` field that contains the string value -previously associated with that ``enum``. +method can be used to generate either a string, or dictionary that has a string ``name`` field, that +represents the string value previously associated with that ``enum``. For more on this new feature, see the `documentation `_ \ No newline at end of file