Fix ContainsGroupError when writing a DataTree with region/append_dim#1036
Merged
TomNicholas merged 2 commits intoJul 16, 2026
Merged
Conversation
virtual_datatree_to_icechunk unconditionally created each node's group with Group.from_store, so forwarding region or append_dim (which require the group and its arrays to already exist) always failed with ContainsGroupError. Mirror the branch virtual_dataset_to_icechunk already had: open the existing group for region/append_dim, create it otherwise. Removes the now-passing xfail on test_write_datatree_region, whose stated reason (an xarray region limitation) was a misdiagnosis of this ContainsGroupError. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #1036 +/- ##
=======================================
Coverage 90.32% 90.33%
=======================================
Files 37 37
Lines 2233 2235 +2
=======================================
+ Hits 2017 2019 +2
Misses 216 216
🚀 New features to boost your workflow:
|
Contributor
Author
|
CI note: the 3 red build jobs ( |
TomNicholas
enabled auto-merge (squash)
July 15, 2026 23:57
Member
|
Thanks @aaronspring ! |
TomNicholas
added a commit
to aaronspring/VirtualiZarr
that referenced
this pull request
Jul 16, 2026
The merge of main (zarr-developers#1036) into this branch resolved the group-creation loop in favor of zarr-developers#1036's Group.open/from_store branch, discarding the open_group(mode=open_mode) call this PR introduced. open_mode was computed but never used, so mode="a" against an existing group raised ContainsGroupError. Restore the unified open_group call, which subsumes zarr-developers#1036's append_dim/region branch via _resolve_mode returning "r+".
TomNicholas
added a commit
that referenced
this pull request
Jul 16, 2026
* Open existing groups when writing a DataTree with region/append_dim
virtual_datatree_to_icechunk unconditionally created each node's group
with Group.from_store, so forwarding region or append_dim (which require
the group and its arrays to already exist) always failed with
ContainsGroupError. Mirror the branch virtual_dataset_to_icechunk
already had: open the existing group for region/append_dim, create it
otherwise.
Removes the now-passing xfail on test_write_datatree_region, whose
stated reason (an xarray region limitation) was a misdiagnosis of this
ContainsGroupError.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Surface mode in to_icechunk, keeping current defaults
Add a mode parameter ("w", "w-", "a") to vds.vz.to_icechunk and
dt.vz.to_icechunk. The default (None) keeps the existing behaviour of
raising ContainsGroupError when the target group already exists, while
mode="a" opens the existing group so datasets can e.g. be split across
commits by variable, and mode="w" overwrites existing contents.
A _resolve_mode helper validates mode and resolves it to the effective
zarr group-open mode ("r+" when append_dim/region require an existing
group), so both writers open groups through a single zarr.open_group
call.
Closes #1001.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Fix merge botch: honor mode in virtual_datatree_to_icechunk loop
The merge of main (#1036) into this branch resolved the group-creation
loop in favor of #1036's Group.open/from_store branch, discarding the
open_group(mode=open_mode) call this PR introduced. open_mode was
computed but never used, so mode="a" against an existing group raised
ContainsGroupError. Restore the unified open_group call, which subsumes
#1036's append_dim/region branch via _resolve_mode returning "r+".
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Tom Nicholas <tom@earthmover.io>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Writing a virtual
DataTreewithregionorappend_dim(forwarded to each node) always failed withzarr.errors.ContainsGroupError.virtual_datatree_to_icechunkcreated every node's group unconditionally withGroup.from_store, butregion/append_dimrequire the group and its arrays to already exist — so the write blew up before it ever got to writing the region/append.This is the same class of failure as #1001, but for the DataTree path.
Fix
Mirror the branch that
virtual_dataset_to_icechunkalready had onmain: open the existing group forregion/append_dim, create it otherwise.Test
test_write_datatree_regionwas markedxfailwith the reason "This doesn't work in xarray either, maybe not even intended?" — but that was a misdiagnosis. Onmain, running it with--runxfailfails withContainsGroupErrorat the group-creation step, not in any xarray region logic. With this fix the test runs to completion and its assertions pass, so thexfailmarker is removed.🤖 Generated with Claude Code