There seems to be a bug in the output.coord_ranges slicing, coming from the lines:
|
if output_coord_ranges is not None: |
|
output_coord_ranges = { |
|
k: w for k, w in output_coord_ranges.items() if k in output_dims |
|
} |
|
da_target = select_by_kwargs(da_target, **output_coord_ranges) |
Note here how the output_coord_ranges variable is overwritten. This means in particular that keys are dropped from this dict, if the key is not in the output_dims of the current dataset being processed. The problem with this is that while this key might not be in the output dims of the current dataset, it might be in the output dims of datasets processed later. So this variable should not be overwritten.
For example, for a config like:
output:
variables:
static: [grid_index, static_feature]
state: [time, grid_index, state_feature]
forcing: [time, grid_index, forcing_feature]
coord_ranges:
time:
start: 2000-01-01T00:00
end: 2000-01-05T00:00
step: PT24H
The time slicing should not be applied to the static output (that does not have a time dimension), but should be applied to state and forcing. Now, everything works fine as long as you order things as state, forcing, static in your config. But in your order things as state, static, forcing, then the time slicing is removed from output_coord_ranges when static is processed, and therefore not applied to the forcing.
I think a fix should be as easy as just using a different name for the variable inside the if-statement, but needs testing.
There seems to be a bug in the output.coord_ranges slicing, coming from the lines:
mllam-data-prep/mllam_data_prep/create_dataset.py
Lines 238 to 242 in af5eb65
Note here how the
output_coord_rangesvariable is overwritten. This means in particular that keys are dropped from this dict, if the key is not in the output_dims of the current dataset being processed. The problem with this is that while this key might not be in the output dims of the current dataset, it might be in the output dims of datasets processed later. So this variable should not be overwritten.For example, for a config like:
The time slicing should not be applied to the static output (that does not have a time dimension), but should be applied to state and forcing. Now, everything works fine as long as you order things as state, forcing, static in your config. But in your order things as state, static, forcing, then the time slicing is removed from
output_coord_rangeswhen static is processed, and therefore not applied to the forcing.I think a fix should be as easy as just using a different name for the variable inside the if-statement, but needs testing.