Test failures with xarray<=2025.4.0 and zarr v3 (Float64 dtype error)

[ashum9]

Problem

Running the neural-lam test suite on a fresh environment with recent
dependency versions (e.g. xarray 2025.4.0 and zarr 3.1.5) results in
multiple datastore-related test failures:

AttributeError: 'Float64' object has no attribute 'value'

The error occurs during xr.open_zarr() calls when loading the example
datastores.

Cause

The issue originates in xarray's zarr backend where xarray assumes that
zarr dtype objects expose a .value attribute. This assumption does not
hold for zarr v3 dtype objects such as Float64.

The neural-lam codebase does not use .value; the error occurs entirely
inside xarray.

Workaround

Upgrading xarray resolves the issue:

xarray >= 2026.2.0

With this version the full test suite passes:

110 passed, 0 failed

Notes

The test datasets appear to use zarr v3 format, so zarr v3 support is
required.

I opened a PR proposing a minimal dependency constraint:

xarray>=2026.2.0

[ashum9]

@observingClouds sir kindly review this issue . Thankyou :)

[observingClouds]

Could you please provide a full error traceback so we can see where this AttributeError occurs?

[ArnabTechiee]

Hi @observingClouds and @ashum9! I am currently setting up my local environment and testing the mdp datastores. I ran into this exact same Float64 AttributeError on a fresh install when it tries to hit zarr_array.metadata.data_type.value.

To help move PR #260 forward, here is the traceback from my local run showing exactly where it fails inside xarray:

if self._use_zarr_fill_value_as_mask:
            # Setting this attribute triggers CF decoding for missing values
            # by interpreting Zarr's fill_value to mean the same as netCDF's _FillValue
            if zarr_array.fill_value is not None:
                attributes["_FillValue"] = zarr_array.fill_value
        elif "_FillValue" in attributes:
            original_zarr_dtype = zarr_array.metadata.data_type
            attributes["_FillValue"] = FillValueCoder.decode(
>               attributes["_FillValue"], original_zarr_dtype.value
                                          ^^^^^^^^^^^^^^^^^^^^^^^^^
            )
E           AttributeError: 'Float64' object has no attribute 'value'

.....venv\Lib\site-packages\xarray\backends\zarr.py:882: AttributeError 

My local environment: Windows, Python 3.13.7, xarray==2025.4.0, zarr==3.1.5.
Let me know if you need me to test the xarray>=2026.2.0 bump on any specific datastore endpoints locally to verify the fix!

[observingClouds]

Thanks @ArnabTechiee for the traceback. Would you mind providing a full traceback so we can see where in the neural-lam code this error occurs? I'm surprised that you got this older version of xarray when creating a fresh environment, but if you could test it with bumping xarray, that would be great.

[ArnabTechiee]

Hi @observingClouds!

Following your suggestion, I upgraded the environment to xarray==2026.2.0 and zarr==3.1.5.

The upgrade successfully resolved the AttributeError! Almost all mdp datastore tests are now passing (53 passed).

However, I encountered a new failure in test_get_standardized_da[state-mdp]. It appears to be a RuntimeWarning: invalid value encountered in divide during the np.allclose check for standardization. This suggests there might be a zero or NaN in the mdp dataset's mean or standard deviation statistics.

Full Traceback of the new failure:

platform win32 -- Python 3.13.7, pytest-8.4.2, pluggy-1.6.0
plugins: anyio-4.11.0, zarr-3.1.5

_________________ test_get_standardized_da[state-mdp] __________________
datastore_name = 'mdp', category = 'state'

    def test_get_standardized_da(datastore_name, category):
        ...
>       assert np.allclose(standard_da, (non_standard_da - mean) / std, atol=1e-6)
E       AssertionError: assert False

================ warnings summary =================
tests/test_datastores.py::test_get_standardized_da[state-mdp]
  RuntimeWarning: invalid value encountered in divide
    return self.func(*new_argspec)

========== 1 failed, 53 passed, 4 warnings in 19.76s ==========

I'm happy to help investigate the state_mean or state_std tensors in the mdp datastore further tomorrow!

[ArnabTechiee]

I have confirmed the root cause of the RuntimeWarning locally.

While the upgrade to xarray==2026.2.0 resolved the AttributeError, the tests hit a division-by-zero because the example mdp datastore (danra_100m_winds) has zero standard deviation for all state features.

My diagnostic script showed that state__train__std is [0. 0. 0. 0.] in the test Zarr store. The code logic is stable; it is simply the dummy test data that lacks the variance required for standardization. I am now focusing on the Project 4 proposal as the infrastructure fix is verified.

[sadamov]

@ashum9 I assigned this issue to you. could you check if the solution proposed by @ArnabTechiee works for you, too. And if so update your PR #260

[ashum9]

Hi @sadamov,

I verified the proposed solution by testing with:

• xarray 2026.2.0
• zarr 3.1.5
• numpy 2.x
• astropy 7.2.0
• Python 3.13

Result : 110 passed, 0 failed

The Float64 AttributeError is fully resolved with xarray>=2026.2.0.

I also confirmed that the test suite works correctly with numpy 2.x, so the numpy<2 constraint is not necessary. I have updated PR #260 accordingly to only require:

xarray>=2026.2.0

Thanks to @ArnabTechiee for confirming the traceback and testing as well.

[observingClouds]

My diagnostic script showed that state__train__std is [0. 0. 0. 0.] in the test Zarr store. The code logic is stable; it is simply the dummy test data that lacks the variance required for standardization. I am now focusing on the Project 4 proposal as the infrastructure fix is verified.
I'm happy to help investigate the state_mean or state_std tensors in the mdp datastore further tomorrow!

Does this get fixed with #201 ?

[ArnabTechiee]

Hi @observingClouds!

I believe PR #201 could indeed resolve the RuntimeWarning if it includes updated sample datasets with non-zero variance.

As I noted in my diagnostic, the invalid value encountered in divide error in the current mdp tests stems from the state__train__std being [0. 0. 0. 0.] in the danra_100m_winds Zarr store. Since the code logic is attempting to standardize using these zero values, a division-by-zero is inevitable.

If #201 refreshes these datastores with real-world variance data, the tests should pass cleanly. I’m happy to pull the changes from #201 and verify this on my local Windows environment if needed!

[observingClouds]

Ah I actually meant #136. The epsilon we introduce there should avoid a decision by zero.

The data is actually as close as we get with a model simulation and 100m wind is actually also quite variable. Our test domain might just be too small, which comes here as a benefit as we can actually see this warning and can fix it 😅

[ArnabTechiee]

Hi @observingClouds!

That makes perfect sense. Implementing a small epsilon as in #136 is definitely the most robust way to handle these edge cases, especially when working with small test domains or static dummy data.

I've verified that the current RuntimeWarning is specifically tied to those zero-variance tensors in the mdp store, so the epsilon approach will stabilize the standardization logic across all environments.

Thanks for the clarification on the test domain! I'll keep an eye on how #136 and #260 progress.