From 59644e4afd4c3c6e19861749c3447c6fc66f5b83 Mon Sep 17 00:00:00 2001 From: Anshul Singhvi Date: Wed, 20 May 2026 13:52:46 -0400 Subject: [PATCH 01/10] use bulk copy in zcompress! fallback to avoid per-byte growth MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `zcompress(data, NoCompressor())` returns a lazy `reinterpret(UInt8, data)` view. The old `empty!` + `append!` fallback walked that view element by element through `_growend!` / `push!`, materialising bytes one at a time — roughly 67% of CPU time on uncompressed full-chunk V2 writes in profiling. Replace with `resize!` + `copyto!` so the same path issues a single SIMD / memcpy bulk copy. Real compressors (Blosc, Zlib, Zstd) are unaffected: they already return a freshly-allocated `Vector{UInt8}` and `copyto!` handles that case identically. Bytes-on-disk are bit-identical; full test suite (2499 tests) passes. Measured on Apple M2 Ultra, Julia 1.12.6, V2 + NoCompressor + chunks=(Nx,Ny,Nz,1): | size | baseline | patched | speed-up | |---------------------|---------:|---------:|---------:| | 128×128×16×50 50M | 702 MB/s | 995 MB/s | 1.42× | | 256×256×32×50 400M | 799 MB/s |1530 MB/s | 1.92× | | 512×512×32×50 1.5G | 776 MB/s |1104 MB/s | 1.42× | Reads are unchanged (this fallback is not on the read path). Co-Authored-By: Claude Opus 4.7 (1M context) --- src/Compressors/Compressors.jl | 15 +++++++++++---- 1 file changed, 11 insertions(+), 4 deletions(-) diff --git a/src/Compressors/Compressors.jl b/src/Compressors/Compressors.jl index 60b52558..e7997745 100644 --- a/src/Compressors/Compressors.jl +++ b/src/Compressors/Compressors.jl @@ -29,10 +29,17 @@ getCompressor(::Nothing) = NoCompressor() zcompress!(compressed,data,c,::Nothing) = zcompress!(compressed,data,c) zuncompress!(data,compressed,c,::Nothing) = zuncompress!(data,compressed,c) -# Fallback definition of mutating form of compress and uncompress -function zcompress!(compressed, data, c) - empty!(compressed) - append!(compressed,zcompress(data, c)) +# Fallback definition of mutating form of compress and uncompress. +# `zcompress(data, c)` may return either a freshly-allocated `Vector{UInt8}` +# (real compressors) or a lazy `reinterpret` view (`NoCompressor`). For the +# view case, the old `empty!` + `append!` path walked the view element by +# element through `_growend!` / `push!`, materialising bytes one at a time; +# `resize!` + `copyto!` issues a single bulk copy (SIMD / memcpy under the +# hood) instead. +function zcompress!(compressed, data, c) + src = zcompress(data, c) + resize!(compressed, length(src)) + copyto!(compressed, src) end zuncompress!(data, compressed, c) = copyto!(data, zuncompress(compressed, c, eltype(data))) From 50fbc169c32af5ef7ca2cb1aa46922e69d0a913c Mon Sep 17 00:00:00 2001 From: Anshul Singhvi Date: Wed, 20 May 2026 14:50:32 -0400 Subject: [PATCH 02/10] use bulk copy in NoCompressor zuncompress! to skip lazy-view walk MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The generic `zuncompress!` fallback at the top of `Compressors.jl` ends up at `copyto!(::Array{T}, ::ReinterpretArray)` when `c isa NoCompressor`, since `zuncompress(bytes, ::NoCompressor, T)` returns a lazy `reinterpret(T, bytes)` view. That `copyto!` walks element by element at ~17 GB/s on Apple silicon vs. ~85 GB/s for `unsafe_copyto!` on the same memory — a 5× gap. End-to-end V2 reads absorb most of it via DiskArrays slicing, channel ferry, and storage I/O, leaving ~25-99% throughput improvement depending on chunk size. Add a `::NoCompressor`-dispatched `zuncompress!` method alongside the existing per-compressor versions for Zstd / Zlib / Blosc. Mirror image of the encode-side bulk copy that landed in the post-A1 `zcompress!`. Guards on `data::Array{T}` and `isbitstype(T)` keep the fast path off `SenMissArray`, `MaxLengthString`, ragged `Vector{T}` eltypes, and any non-contiguous input — those fall back to the existing generic `copyto!` path. Tests: 2499/2499 pass. Measured on Apple M2 Ultra, Julia 1.12.6, V2 + NoCompressor + chunks=(Nx,Ny,Nz,1), 1 warm-up + 7 timed repeats, A/B via git stash on the same wall-clock: | size | A1 only read | A1 + B1 read | speed-up | |---------------------|-------------:|-------------:|---------:| | 128×128×16×50 50M | 3080 MB/s | 3884 MB/s | +26% | | 256×256×32×50 400M | 3155 MB/s | 4130 MB/s | +31% | | 512×512×32×50 1.5G | 2243 MB/s | 4455 MB/s | +99% | Writes are unchanged (B1 doesn't touch the encode path). Reference: Zarrs.jl on the same V2-uncompressed workload reads at 5166 / 4672 / 5418 MB/s, so B1 closes most of the read gap; the 1.6 GiB case goes from ~58% of Zarrs.jl to ~82%. Co-Authored-By: Claude Opus 4.7 (1M context) --- src/Compressors/Compressors.jl | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) diff --git a/src/Compressors/Compressors.jl b/src/Compressors/Compressors.jl index e7997745..b6ad16d4 100644 --- a/src/Compressors/Compressors.jl +++ b/src/Compressors/Compressors.jl @@ -79,6 +79,22 @@ function zcompress(a, ::NoCompressor) _reinterpret(UInt8,a) end +# Fast path: NoCompressor decode is a bulk byte copy. The generic +# `zuncompress!` fallback at the top of this file ends up at +# `copyto!(::Array{T}, ::ReinterpretArray)`, which walks element by +# element at ~17 GB/s on Apple silicon vs. ~85 GB/s for `unsafe_copyto!`. +# Mirror image of the encode-side bulk copy inside `zcompress!`. +function zuncompress!(data::Array{T}, compressed::Vector{UInt8}, ::NoCompressor) where {T} + isbitstype(T) || return copyto!(data, _reinterpret(T, compressed)) + n = sizeof(data) + n == length(compressed) || throw(DimensionMismatch( + "Encoded byte length $(length(compressed)) does not match output byte size $n" + )) + GC.@preserve data compressed unsafe_copyto!(Ptr{UInt8}(pointer(data)), + pointer(compressed), n) + return data +end + JSON.lower(::NoCompressor) = nothing compressortypes[nothing] = NoCompressor From c04633804b9c003d8f739c8e06faa982a2da4d8e Mon Sep 17 00:00:00 2001 From: Anshul Singhvi Date: Wed, 20 May 2026 14:57:41 -0400 Subject: [PATCH 03/10] add getchunkarray_undef and skip dead zero-fill on full-overwrite paths MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Both `readblock!` and `writeblock!` allocated the chunk-shaped scratch buffer via `getchunkarray(z) = fill(_zero(eltype(z)), z.metadata.chunks)`, which zero-fills the chunk before any other work touches it. On the read side that fill is always clobbered: `pipeline_decode!` writes every element, or the fill-value branch in `uncompress_raw!` calls `fill!` itself. On the write side it's clobbered for full-chunk overwrites (the common case for `z[:,:,:,t] = buf`-style appends) and re-initialised by `resetbuffer!` for partial-chunk RMW. Add `getchunkarray_undef(z::ZArray{T})` which returns `Array{T}(undef, …)` for plain isbits eltypes and falls back to `getchunkarray` for `Missing <: T` (the `SenMissArray` path: Blosc and friends reject `Union{Missing,T}` directly, so the inner buffer has to be a real `Array{T}`). - `readblock!`: always use `getchunkarray_undef`. Decode-into and fill-value branches both initialise every element. - `writeblock!`: use `getchunkarray_undef` when `z.metadata.fill_value !== nothing` (resetbuffer! handles init). Keep the legacy zero-fill when `fill_value === nothing` so partial writes to a fresh chunk still default un-written cells to zero. Port of the V2-side hunks from JuliaIO/Zarr.jl#272 (verbatim, with the same `Missing <: T` carve-out the PR added in commit 75bea38 to fix a Blosc rejection regression). Tests: 2499/2499 pass — including the SenMissArray-exercising `Fillvalue as missing`, `getindex/setindex` (amiss), `MaxLengthString large-chunk read path`, and `ragged arrays` tests. Measured on Apple M2 Ultra, Julia 1.12.6, V2 + NoCompressor + chunks=(Nx,Ny,Nz,1), 1 warm-up + 7 timed repeats, A/B via git stash: | size | B1 only (R) | B1+A2 (R) | speed-up | |---------------------|------------:|-----------:|---------:| | 128×128×16×50 50M | 3466 MB/s | 3866 MB/s | +12% | | 256×256×32×50 400M | 3589 MB/s | 4169 MB/s | +16% | | 512×512×32×50 1.5G | ~3000 MB/s | ~3500 MB/s | noisy | Write side smaller (most write time is storage-bound, not memset): | size | B1 only (W) | B1+A2 (W) | speed-up | |---------------------|------------:|-----------:|---------:| | 128×128×16×50 50M | 1148 MB/s | 1203 MB/s | +5% | | 256×256×32×50 400M | 1327 MB/s | 1354 MB/s | +2% | | 512×512×32×50 1.5G | 1132 MB/s | 1329 MB/s | +17% | The 1.5 GiB row is the noisiest across runs because chunks no longer fit in L3 — exact numbers vary with cache state at run start. Co-Authored-By: Claude Opus 4.7 (1M context) --- src/ZArray.jl | 29 +++++++++++++++++++++++------ 1 file changed, 23 insertions(+), 6 deletions(-) diff --git a/src/ZArray.jl b/src/ZArray.jl index fdc23364..f864319e 100644 --- a/src/ZArray.jl +++ b/src/ZArray.jl @@ -160,6 +160,19 @@ _zero(::Type{<:Vector{T}}) where T = T[] _zero(::Type{Char}) = Char(0) getchunkarray(z::ZArray) = fill(_zero(eltype(z)), z.metadata.chunks) +# Same as `getchunkarray` but skips the zero/fill_value-fill. Use only when +# the caller guarantees the buffer will be fully overwritten before any read +# (e.g. decode-into or full-chunk-write paths). +# +# Falls back to the standard `getchunkarray` for `>:Missing` element types: +# the codec pipeline requires the underlying buffer to be the `isbits` inner +# of a `SenMissArray`, not an `Array{Union{Missing,T}}` directly (Blosc and +# friends reject non-isbits eltypes). +function getchunkarray_undef(z::ZArray{T}) where {T} + Missing <: T && return getchunkarray(z) + return Array{T}(undef, z.metadata.chunks) +end + maybeinner(a::Array) = a maybeinner(a::SenMissArray) = a.x resetbuffer!(fv,a::Array) = fv === nothing || fill!(a,fv) @@ -172,9 +185,11 @@ function readblock!(aout::AbstractArray{<:Any,N}, z::ZArray{<:Any, N}, r::Cartes output_base_offsets = map(i->first(i)-1,r.indices) # Determines which chunks are affected blockr = CartesianIndices(map(trans_ind, r.indices, z.metadata.chunks)) - # Allocate array of the size of a chunks where uncompressed data can be held - #bufferdict = IdDict((current_task()=>getchunkarray(z),)) - a = getchunkarray(z) + # Allocate the chunk-shaped scratch buffer. Reads always either fill it + # from a decode (which writes every element) or fall through to the + # fill-value path (which calls `fill!` itself), so we don't need to + # pre-zero it. + a = getchunkarray_undef(z) # Now loop through the chunks c = Channel{Pair{eltype(blockr),Union{Nothing,Vector{UInt8}}}}(channelsize(z.storage)) @@ -208,9 +223,11 @@ function writeblock!(ain::AbstractArray{<:Any,N}, z::ZArray{<:Any, N}, r::Cartes input_base_offsets = map(i->first(i)-1,r.indices) # Determines which chunks are affected blockr = CartesianIndices(map(trans_ind, r.indices, z.metadata.chunks)) - # Allocate array of the size of a chunks where uncompressed data can be held - #bufferdict = IdDict((current_task()=>getchunkarray(z),)) - a = getchunkarray(z) + # If `fill_value === nothing`, the legacy behaviour is that un-written + # cells in a partial-write to a new chunk default to zero (via the + # zero-fill of `getchunkarray`). Preserve that. Otherwise `resetbuffer!` + # handles initialisation explicitly and we save the dead memset. + a = z.metadata.fill_value === nothing ? getchunkarray(z) : getchunkarray_undef(z) # Now loop through the chunks readchannel = Channel{Pair{eltype(blockr),Union{Nothing,Vector{UInt8}}}}(channelsize(z.storage)) From e9d900470208b0113c8b8b8c647d2acc88bcd212 Mon Sep 17 00:00:00 2001 From: Anshul Singhvi Date: Wed, 20 May 2026 15:05:25 -0400 Subject: [PATCH 04/10] add CHANGELOG entries for #280 (V2 perf) One bullet per commit on the PR. Co-Authored-By: Claude Opus 4.7 (1M context) --- CHANGELOG.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index 7547eb61..bf29186d 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,6 +2,9 @@ ## Unreleased +- V2 NoCompressor writes: replace `append!` over the reinterpret view with bulk `resize!` + `copyto!` in the generic `zcompress!` fallback [#280](https://github.com/JuliaIO/Zarr.jl/pull/280) +- V2 NoCompressor reads: add bulk-copy `zuncompress!` method dispatched on `::NoCompressor` to bypass `copyto!(::Array, ::ReinterpretArray)`'s element-by-element walk [#280](https://github.com/JuliaIO/Zarr.jl/pull/280) +- V2 read+write chunk allocation: add `getchunkarray_undef` and skip the dead zero-fill of the chunk-shaped scratch buffer on full-overwrite paths [#280](https://github.com/JuliaIO/Zarr.jl/pull/280) - Fix CondaPkg branch in CI, use release version instead [#273](https://github.com/JuliaIO/Zarr.jl/pull/273) - Fix creation of on-disk arrays that do not fit in memory [#269](https://github.com/JuliaIO/Zarr.jl/pull/269) From 2e4d09b17b6a44fc7cee02b8367623b3668262ad Mon Sep 17 00:00:00 2001 From: Anshul Singhvi Date: Wed, 20 May 2026 15:17:43 -0400 Subject: [PATCH 05/10] shrink comments around zcompress! / zuncompress! fallbacks The multi-paragraph rationale comments are now stale post-landing; one line each is enough to flag why bulk copy beats append! / generic ReinterpretArray copy. Co-Authored-By: Claude Opus 4.7 (1M context) --- src/Compressors/Compressors.jl | 14 ++------------ 1 file changed, 2 insertions(+), 12 deletions(-) diff --git a/src/Compressors/Compressors.jl b/src/Compressors/Compressors.jl index b6ad16d4..d2f631e2 100644 --- a/src/Compressors/Compressors.jl +++ b/src/Compressors/Compressors.jl @@ -29,13 +29,7 @@ getCompressor(::Nothing) = NoCompressor() zcompress!(compressed,data,c,::Nothing) = zcompress!(compressed,data,c) zuncompress!(data,compressed,c,::Nothing) = zuncompress!(data,compressed,c) -# Fallback definition of mutating form of compress and uncompress. -# `zcompress(data, c)` may return either a freshly-allocated `Vector{UInt8}` -# (real compressors) or a lazy `reinterpret` view (`NoCompressor`). For the -# view case, the old `empty!` + `append!` path walked the view element by -# element through `_growend!` / `push!`, materialising bytes one at a time; -# `resize!` + `copyto!` issues a single bulk copy (SIMD / memcpy under the -# hood) instead. +# Bulk `resize!` + `copyto!` (not `append!`): avoids elementwise growth over `NoCompressor`'s view. function zcompress!(compressed, data, c) src = zcompress(data, c) resize!(compressed, length(src)) @@ -79,11 +73,7 @@ function zcompress(a, ::NoCompressor) _reinterpret(UInt8,a) end -# Fast path: NoCompressor decode is a bulk byte copy. The generic -# `zuncompress!` fallback at the top of this file ends up at -# `copyto!(::Array{T}, ::ReinterpretArray)`, which walks element by -# element at ~17 GB/s on Apple silicon vs. ~85 GB/s for `unsafe_copyto!`. -# Mirror image of the encode-side bulk copy inside `zcompress!`. +# Fast path: bulk `unsafe_copyto!` avoids the elementwise `ReinterpretArray` copy of the fallback. function zuncompress!(data::Array{T}, compressed::Vector{UInt8}, ::NoCompressor) where {T} isbitstype(T) || return copyto!(data, _reinterpret(T, compressed)) n = sizeof(data) From ebc3ea140142e8915d1bdb9627febd044af200d7 Mon Sep 17 00:00:00 2001 From: Anshul Singhvi Date: Wed, 20 May 2026 17:21:28 -0400 Subject: [PATCH 06/10] skip readtask/writetask ferry on single-chunk full-overwrite writes MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The slow writeblock! path spawns a readtask and a writetask connected by 0-buffered channels for every call. Profiling the 400 MiB headline workload showed 93% of write CPU pinned in __psynch_cvwait — the channel ferry synchronisation cost — for the dominant case of writing one full chunk at a time (`z[:,:,:,t] = buf`). Add a fast path at the top of writeblock! that, when the call touches exactly one chunk and `ain` is a plain Array matching the chunk shape and eltype, encodes `ain` directly via `compress_raw` and calls `store_writechunk`/`store_deletechunk` synchronously. Fill-value elision is preserved (delete-if-initialised when encode returns `nothing`). Restricted to non-Missing element types so the SenMissArray indirection required by the codec pipeline still goes through the slow path. Bench (M2 Ultra, ZS_REPEATS=3, V2 + NoCompressor, 50 timesteps): size write MiB/s read MiB/s before after before after 128³×16×50 1235 2228 3864 ~3500 (write +80%) 256³×32×50 1392 2160 3870 ~3300 (write +55%) 512²×32×50 1339 2240 4651 ~3500 (write +67%) Read variance in the combined bench is system noise; the isolated read-only bench shows reads unchanged. Tests: 2499/2499 pass. Co-Authored-By: Claude Opus 4.7 (1M context) --- CHANGELOG.md | 1 + src/ZArray.jl | 21 ++++++++++++++++++++- 2 files changed, 21 insertions(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index bf29186d..5a020f15 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,6 +2,7 @@ ## Unreleased +- V2 writes: add `writeblock!` fast path for single-chunk full-overwrites that bypasses the readtask/writetask channels and the chunk-shaped scratch buffer, encoding straight from the input array into the store [#280](https://github.com/JuliaIO/Zarr.jl/pull/280) - V2 NoCompressor writes: replace `append!` over the reinterpret view with bulk `resize!` + `copyto!` in the generic `zcompress!` fallback [#280](https://github.com/JuliaIO/Zarr.jl/pull/280) - V2 NoCompressor reads: add bulk-copy `zuncompress!` method dispatched on `::NoCompressor` to bypass `copyto!(::Array, ::ReinterpretArray)`'s element-by-element walk [#280](https://github.com/JuliaIO/Zarr.jl/pull/280) - V2 read+write chunk allocation: add `getchunkarray_undef` and skip the dead zero-fill of the chunk-shaped scratch buffer on full-overwrite paths [#280](https://github.com/JuliaIO/Zarr.jl/pull/280) diff --git a/src/ZArray.jl b/src/ZArray.jl index f864319e..603b65e0 100644 --- a/src/ZArray.jl +++ b/src/ZArray.jl @@ -218,11 +218,30 @@ function readblock!(aout::AbstractArray{<:Any,N}, z::ZArray{<:Any, N}, r::Cartes end function writeblock!(ain::AbstractArray{<:Any,N}, z::ZArray{<:Any, N}, r::CartesianIndices{N}) where {N} - + z.writeable || error("Can not write to read-only ZArray") input_base_offsets = map(i->first(i)-1,r.indices) # Determines which chunks are affected blockr = CartesianIndices(map(trans_ind, r.indices, z.metadata.chunks)) + # Fast path: single-chunk full-overwrite skips the readtask/writetask channels and the scratch buffer. + T = eltype(z) + if length(blockr) == 1 && ain isa Array && !(Missing <: T) && + eltype(ain) === T && size(ain) == z.metadata.chunks + bI = first(blockr) + current_chunk_offsets = map((s,i)->s*(i-1), z.metadata.chunks, Tuple(bI)) + indranges = map(boundint, r.indices, z.metadata.chunks, current_chunk_offsets) + if length.(indranges) == z.metadata.chunks + data_encoded = compress_raw(ain, z) + if isnothing(data_encoded) + if store_isinitialized(z.storage, z.path, bI, z.metadata.chunk_key_encoding) + store_deletechunk(z.storage, z.path, bI, z.metadata.chunk_key_encoding) + end + else + store_writechunk(z.storage, data_encoded, z.path, bI, z.metadata.chunk_key_encoding) + end + return ain + end + end # If `fill_value === nothing`, the legacy behaviour is that un-written # cells in a partial-write to a new chunk default to zero (via the # zero-fill of `getchunkarray`). Preserve that. Otherwise `resetbuffer!` From d84698665d5cfa87c6e265b39b6220494825af89 Mon Sep 17 00:00:00 2001 From: Anshul Singhvi Date: Wed, 20 May 2026 17:25:12 -0400 Subject: [PATCH 07/10] skip readtask ferry on single-chunk full-overwrite reads MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Mirrors the writeblock! fast path: for the common case of reading exactly one full chunk into an Array of matching shape and eltype, bypass the readtask + 0-buffered channel and the chunk-shaped scratch buffer. Read the compressed bytes synchronously via `store_readchunk` and decode directly into `aout`, eliminating both the channel-ferry sync wait (80% of read CPU per profiling) and the per-call scratch allocation + final copy (the 13% GC pressure source). Bench (M2 Ultra, ZS_REPEATS=3, V2 + NoCompressor, isolated read-only): size before after speedup 128³×16×50 1528 MiB/s ~5300 MiB/s ~3.5x 256³×32×50 3128 MiB/s ~4900 MiB/s ~1.6x 512²×32×50 3248 MiB/s ~4800 MiB/s ~1.5x Same guards as the write fast path: single chunk touched, `aout isa Array`, non-Missing eltype, eltype and shape match the chunk. Otherwise falls through to the existing channel-based path. Tests: 2499/2499 pass. Co-Authored-By: Claude Opus 4.7 (1M context) --- CHANGELOG.md | 1 + src/ZArray.jl | 15 ++++++++++++++- 2 files changed, 15 insertions(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 5a020f15..bc9d4ec8 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,6 +2,7 @@ ## Unreleased +- V2 reads: add `readblock!` fast path for single-chunk full-reads that bypasses the readtask channel and the chunk-shaped scratch buffer, decoding straight into the output array [#280](https://github.com/JuliaIO/Zarr.jl/pull/280) - V2 writes: add `writeblock!` fast path for single-chunk full-overwrites that bypasses the readtask/writetask channels and the chunk-shaped scratch buffer, encoding straight from the input array into the store [#280](https://github.com/JuliaIO/Zarr.jl/pull/280) - V2 NoCompressor writes: replace `append!` over the reinterpret view with bulk `resize!` + `copyto!` in the generic `zcompress!` fallback [#280](https://github.com/JuliaIO/Zarr.jl/pull/280) - V2 NoCompressor reads: add bulk-copy `zuncompress!` method dispatched on `::NoCompressor` to bypass `copyto!(::Array, ::ReinterpretArray)`'s element-by-element walk [#280](https://github.com/JuliaIO/Zarr.jl/pull/280) diff --git a/src/ZArray.jl b/src/ZArray.jl index 603b65e0..7eadf2e6 100644 --- a/src/ZArray.jl +++ b/src/ZArray.jl @@ -181,10 +181,23 @@ resetbuffer!(_,a::SenMissArray) = fill!(a,missing) # Function to read or write from a zarr array. Could be refactored # using type system to get rid of the `if readmode` statements. function readblock!(aout::AbstractArray{<:Any,N}, z::ZArray{<:Any, N}, r::CartesianIndices{N}) where {N} - + output_base_offsets = map(i->first(i)-1,r.indices) # Determines which chunks are affected blockr = CartesianIndices(map(trans_ind, r.indices, z.metadata.chunks)) + # Fast path: single-chunk full-read decodes directly into `aout`, skipping the readtask channel and scratch buffer. + T = eltype(z) + if length(blockr) == 1 && aout isa Array && !(Missing <: T) && + eltype(aout) === T && size(aout) == z.metadata.chunks + bI = first(blockr) + current_chunk_offsets = map((s,i)->s*(i-1), z.metadata.chunks, Tuple(bI)) + indranges = map(boundint, r.indices, z.metadata.chunks, current_chunk_offsets) + if length.(indranges) == z.metadata.chunks + chunk_compressed = store_readchunk(z.storage, z.path, bI, z.metadata.chunk_key_encoding) + uncompress_raw!(aout, z, chunk_compressed) + return aout + end + end # Allocate the chunk-shaped scratch buffer. Reads always either fill it # from a decode (which writes every element) or fall through to the # fill-value path (which calls `fill!` itself), so we don't need to From 3839b95888484a67ff8d8aada21001f59d61d0a4 Mon Sep 17 00:00:00 2001 From: Anshul Singhvi Date: Wed, 20 May 2026 17:34:23 -0400 Subject: [PATCH 08/10] factor singlechunk_fastpath guard out of readblock!/writeblock! MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The two A3 fast paths share the same eligibility check (single chunk, plain Array of matching shape and eltype, non-Missing T). Extract that into a helper that returns the chunk index or `nothing`, so each caller's fast-path body shrinks to the I/O calls. The previous version also re-derived `indranges` to verify full-chunk coverage; that check is redundant. If `size(arr) == chunks` and `length(blockr) == 1`, the slice extent equals the chunk extent within one chunk, which forces it to align with that chunk's range — otherwise the slice would straddle into a neighbouring chunk and bump `length(blockr)`. The helper's docstring states this so the elision is not mysterious. Tests: 2499/2499 pass. Co-Authored-By: Claude Opus 4.7 (1M context) --- src/ZArray.jl | 51 +++++++++++++++++++++++++-------------------------- 1 file changed, 25 insertions(+), 26 deletions(-) diff --git a/src/ZArray.jl b/src/ZArray.jl index 7eadf2e6..905d5e63 100644 --- a/src/ZArray.jl +++ b/src/ZArray.jl @@ -178,6 +178,17 @@ maybeinner(a::SenMissArray) = a.x resetbuffer!(fv,a::Array) = fv === nothing || fill!(a,fv) resetbuffer!(_,a::SenMissArray) = fill!(a,missing) +# Returns the chunk index when the call qualifies for the single-chunk fast +# path: a plain `Array{T,N}` matching the chunk shape, `Missing <: T` false, +# and only one chunk touched. `size(arr) == chunks` with `length(blockr) == 1` +# implies the slice aligns with that chunk's full range, so no extra +# coverage check is needed. +function singlechunk_fastpath(arr, z::ZArray{T,N}, blockr::CartesianIndices{N}) where {T,N} + arr isa Array{T,N} && !(Missing <: T) && + size(arr) == z.metadata.chunks && length(blockr) == 1 || return nothing + return first(blockr) +end + # Function to read or write from a zarr array. Could be refactored # using type system to get rid of the `if readmode` statements. function readblock!(aout::AbstractArray{<:Any,N}, z::ZArray{<:Any, N}, r::CartesianIndices{N}) where {N} @@ -186,17 +197,11 @@ function readblock!(aout::AbstractArray{<:Any,N}, z::ZArray{<:Any, N}, r::Cartes # Determines which chunks are affected blockr = CartesianIndices(map(trans_ind, r.indices, z.metadata.chunks)) # Fast path: single-chunk full-read decodes directly into `aout`, skipping the readtask channel and scratch buffer. - T = eltype(z) - if length(blockr) == 1 && aout isa Array && !(Missing <: T) && - eltype(aout) === T && size(aout) == z.metadata.chunks - bI = first(blockr) - current_chunk_offsets = map((s,i)->s*(i-1), z.metadata.chunks, Tuple(bI)) - indranges = map(boundint, r.indices, z.metadata.chunks, current_chunk_offsets) - if length.(indranges) == z.metadata.chunks - chunk_compressed = store_readchunk(z.storage, z.path, bI, z.metadata.chunk_key_encoding) - uncompress_raw!(aout, z, chunk_compressed) - return aout - end + bI = singlechunk_fastpath(aout, z, blockr) + if bI !== nothing + chunk_compressed = store_readchunk(z.storage, z.path, bI, z.metadata.chunk_key_encoding) + uncompress_raw!(aout, z, chunk_compressed) + return aout end # Allocate the chunk-shaped scratch buffer. Reads always either fill it # from a decode (which writes every element) or fall through to the @@ -237,23 +242,17 @@ function writeblock!(ain::AbstractArray{<:Any,N}, z::ZArray{<:Any, N}, r::Cartes # Determines which chunks are affected blockr = CartesianIndices(map(trans_ind, r.indices, z.metadata.chunks)) # Fast path: single-chunk full-overwrite skips the readtask/writetask channels and the scratch buffer. - T = eltype(z) - if length(blockr) == 1 && ain isa Array && !(Missing <: T) && - eltype(ain) === T && size(ain) == z.metadata.chunks - bI = first(blockr) - current_chunk_offsets = map((s,i)->s*(i-1), z.metadata.chunks, Tuple(bI)) - indranges = map(boundint, r.indices, z.metadata.chunks, current_chunk_offsets) - if length.(indranges) == z.metadata.chunks - data_encoded = compress_raw(ain, z) - if isnothing(data_encoded) - if store_isinitialized(z.storage, z.path, bI, z.metadata.chunk_key_encoding) - store_deletechunk(z.storage, z.path, bI, z.metadata.chunk_key_encoding) - end - else - store_writechunk(z.storage, data_encoded, z.path, bI, z.metadata.chunk_key_encoding) + bI = singlechunk_fastpath(ain, z, blockr) + if bI !== nothing + data_encoded = compress_raw(ain, z) + if isnothing(data_encoded) + if store_isinitialized(z.storage, z.path, bI, z.metadata.chunk_key_encoding) + store_deletechunk(z.storage, z.path, bI, z.metadata.chunk_key_encoding) end - return ain + else + store_writechunk(z.storage, data_encoded, z.path, bI, z.metadata.chunk_key_encoding) end + return ain end # If `fill_value === nothing`, the legacy behaviour is that un-written # cells in a partial-write to a new chunk default to zero (via the From 2eee84c2c1bf61cf8ca17793904e89d67f30d1be Mon Sep 17 00:00:00 2001 From: Anshul Singhvi Date: Thu, 21 May 2026 00:48:17 +0000 Subject: [PATCH 09/10] zero-copy chunk write for V2 NoCompressor + no filters MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The single-chunk fastpath in writeblock! went through compress_raw → pipeline_encode, which allocates a chunk-sized Vector{UInt8} and copyto!'s the bytes in. For the V2-uncompressed-no-filters case the encoded bytes are just the host-endian bit representation of the input, so reinterpret(UInt8, ain) is a valid byte source — pass that straight to the store and skip the allocation + memcpy. Factor the fastpath body into write_singlechunk_fastpath! so the specialization can dispatch on metadata type (V2 + NoCompressor + Nothing filters) without touching the slow path. The slow path continues to materialize an owned Vector{UInt8} since its scratch buffer is reused across iterations and aliasing would race the writetask. ZarrBenchmarks v2 uncompressed write throughput (256x256x32x50 chunks, NVMe on Btrfs): - sequential: 1045 → 2513 MB/s (+141%), 95-101% of raw write() ceiling - 8 threads: 1294 → 2444 MB/s (+89%) - 512^2 chunk-size regression eliminated (the prior allocation dominated at large chunks). Tests cover dispatch (which() inspection), round-trip across rank 0-4 and multiple bitstypes, all-fill-value chunk elision, and caller-array aliasing safety. --- src/ZArray.jl | 50 +++++++++++++++++++++++----- test/runtests.jl | 87 ++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 129 insertions(+), 8 deletions(-) diff --git a/src/ZArray.jl b/src/ZArray.jl index 905d5e63..812627f5 100644 --- a/src/ZArray.jl +++ b/src/ZArray.jl @@ -189,6 +189,47 @@ function singlechunk_fastpath(arr, z::ZArray{T,N}, blockr::CartesianIndices{N}) return first(blockr) end +# Single-chunk full-overwrite write. Encodes `ain` and pushes the chunk to +# the store. Split out of `writeblock!` so we can specialize on metadata +# type below. +function write_singlechunk_fastpath!(z::ZArray, ain::Array, bI::CartesianIndex) + data_encoded = compress_raw(ain, z) + if data_encoded === nothing + if store_isinitialized(z.storage, z.path, bI, z.metadata.chunk_key_encoding) + store_deletechunk(z.storage, z.path, bI, z.metadata.chunk_key_encoding) + end + else + store_writechunk(z.storage, data_encoded, z.path, bI, z.metadata.chunk_key_encoding) + end + return nothing +end + +# Zero-copy write for V2 with no compressor and no filters: the on-disk +# bytes are exactly `ain`'s in-memory representation (Zarr V2 default +# little-endian matches Julia on little-endian hosts — i.e. all currently +# supported platforms), so pass a `reinterpret(UInt8, ain)` view straight +# to the store and skip the chunk-sized `Vector{UInt8}` allocation + +# memcpy that the generic pipeline performs. +# +# Safe to hand out a view (not an owned copy) because the singlechunk +# fastpath guarantees `ain` is the caller's input array, not the shared +# scratch buffer the multi-chunk path reuses across iterations. +function write_singlechunk_fastpath!( + z::ZArray{T,N,<:AbstractStore,<:MetadataV2{T,N,NoCompressor,Nothing}}, + ain::Array{T,N}, bI::CartesianIndex, +) where {T,N} + fv = z.metadata.fill_value + if fv !== nothing && all(isequal(fv), ain) + if store_isinitialized(z.storage, z.path, bI, z.metadata.chunk_key_encoding) + store_deletechunk(z.storage, z.path, bI, z.metadata.chunk_key_encoding) + end + return nothing + end + store_writechunk(z.storage, _reinterpret(UInt8, ain), + z.path, bI, z.metadata.chunk_key_encoding) + return nothing +end + # Function to read or write from a zarr array. Could be refactored # using type system to get rid of the `if readmode` statements. function readblock!(aout::AbstractArray{<:Any,N}, z::ZArray{<:Any, N}, r::CartesianIndices{N}) where {N} @@ -244,14 +285,7 @@ function writeblock!(ain::AbstractArray{<:Any,N}, z::ZArray{<:Any, N}, r::Cartes # Fast path: single-chunk full-overwrite skips the readtask/writetask channels and the scratch buffer. bI = singlechunk_fastpath(ain, z, blockr) if bI !== nothing - data_encoded = compress_raw(ain, z) - if isnothing(data_encoded) - if store_isinitialized(z.storage, z.path, bI, z.metadata.chunk_key_encoding) - store_deletechunk(z.storage, z.path, bI, z.metadata.chunk_key_encoding) - end - else - store_writechunk(z.storage, data_encoded, z.path, bI, z.metadata.chunk_key_encoding) - end + write_singlechunk_fastpath!(z, ain, bI) return ain end # If `fill_value === nothing`, the legacy behaviour is that un-written diff --git a/test/runtests.jl b/test/runtests.jl index d1ee7c0c..6bb66083 100644 --- a/test/runtests.jl +++ b/test/runtests.jl @@ -83,6 +83,93 @@ using Dates GC.gc() end end + + @testset "NoCompressor single-chunk zero-copy fastpath" begin + # The V2+NoCompressor+no-filters specialization of + # `write_singlechunk_fastpath!` hands a `reinterpret(UInt8, ain)` view + # straight to the store instead of allocating a chunk-sized + # `Vector{UInt8}` and memcpy'ing into it. The on-disk bytes must + # still match the array's bit pattern exactly. + + # Dispatch: the specialized method is selected for V2 + NoCompressor + no filters. + mktempdir(@__DIR__) do dir + z3 = zcreate(Float32, 4, 4, 2; path=joinpath(dir, "disp"), + chunks=(4, 4, 2), compressor=Zarr.NoCompressor()) + ain = rand(Float32, 4, 4, 2) + m = which(Zarr.write_singlechunk_fastpath!, (typeof(z3), typeof(ain), CartesianIndex{3})) + @test occursin("MetadataV2", string(m.sig)) + @test occursin("NoCompressor", string(m.sig)) + GC.gc() + end + + # Round-trip across rank 0, 1, 3, 4 and a mix of bitstypes. + mktempdir(@__DIR__) do dir + for (T, shape) in ((Float32, (8, 8, 4)), + (Int64, (5, 3)), + (UInt16, (16,)), + (ComplexF32, (4, 4)), + (Float64, ())) + name = "rt_$(T)_$(length(shape))d" + # Match chunks to full shape so writes hit the single-chunk fastpath. + chunks = shape == () ? () : shape + z = zcreate(T, shape...; path=joinpath(dir, name), + chunks=chunks, compressor=Zarr.NoCompressor()) + # Drive `ain` through a controlled byte pattern so round-trip + # failure can't be masked by zero-init. + ain = T === ComplexF32 ? + reshape(ComplexF32[ComplexF32(i, -i) for i in 1:prod(shape == () ? (1,) : shape)], + shape == () ? () : shape) : + reshape(T[T(i) for i in 1:prod(shape == () ? (1,) : shape)], + shape == () ? () : shape) + if shape == () + z[] = ain[] + else + z[(Colon() for _ in shape)...] = ain + end + # On-disk chunk file equals reinterpret(UInt8, ain). + chunk_path = shape == () ? joinpath(dir, name, "0") : + joinpath(dir, name, join(fill("0", length(shape)), ".")) + @test isfile(chunk_path) + on_disk = read(chunk_path) + expected = Vector{UInt8}(undef, sizeof(ain)) + GC.@preserve ain expected unsafe_copyto!(pointer(expected), + Ptr{UInt8}(pointer(ain)), sizeof(ain)) + @test on_disk == expected + # Reads round-trip via the regular Zarr.jl path. + @test Array(zopen(joinpath(dir, name))) == ain + GC.gc() + end + end + + # All-fill-value chunks are elided (no on-disk file written) — same + # semantics as the generic V2 pipeline_encode. + mktempdir(@__DIR__) do dir + z = zcreate(Float32, 4, 4; path=joinpath(dir, "elide"), + chunks=(4, 4), compressor=Zarr.NoCompressor(), + fill_value=Float32(7)) + z[:, :] = fill(Float32(7), 4, 4) + @test !ispath(joinpath(dir, "elide", "0.0")) + # Overwriting an existing chunk with all-fill removes the file. + z[:, :] = rand(Float32, 4, 4) + @test ispath(joinpath(dir, "elide", "0.0")) + z[:, :] = fill(Float32(7), 4, 4) + @test !ispath(joinpath(dir, "elide", "0.0")) + GC.gc() + end + + # The store must fully consume the reinterpret view before returning: + # mutating `ain` after the write must not affect on-disk content. + mktempdir(@__DIR__) do dir + z = zcreate(Float32, 8, 8; path=joinpath(dir, "alias"), + chunks=(8, 8), compressor=Zarr.NoCompressor()) + ain = rand(Float32, 8, 8) + snapshot = copy(ain) + z[:, :] = ain + fill!(ain, Float32(NaN)) # clobber after the write returns + @test zopen(joinpath(dir, "alias"))[:, :] == snapshot + GC.gc() + end + end end @testset "Groups" begin From 031bf40ad8cda136427ab5f8a7781af1ba5b6a18 Mon Sep 17 00:00:00 2001 From: Anshul Singhvi Date: Thu, 21 May 2026 01:29:30 +0000 Subject: [PATCH 10/10] consolidate #280 CHANGELOG entries Per @lazarusA's PR review: collapse the five repeated [#280] entries into one bullet with the sub-items as a list, matching the v0.10.0 #241 formatting. Adds the zero-copy NoCompressor fastpath as a new sub-item. --- CHANGELOG.md | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index bc9d4ec8..e960553d 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,11 +2,13 @@ ## Unreleased -- V2 reads: add `readblock!` fast path for single-chunk full-reads that bypasses the readtask channel and the chunk-shaped scratch buffer, decoding straight into the output array [#280](https://github.com/JuliaIO/Zarr.jl/pull/280) -- V2 writes: add `writeblock!` fast path for single-chunk full-overwrites that bypasses the readtask/writetask channels and the chunk-shaped scratch buffer, encoding straight from the input array into the store [#280](https://github.com/JuliaIO/Zarr.jl/pull/280) -- V2 NoCompressor writes: replace `append!` over the reinterpret view with bulk `resize!` + `copyto!` in the generic `zcompress!` fallback [#280](https://github.com/JuliaIO/Zarr.jl/pull/280) -- V2 NoCompressor reads: add bulk-copy `zuncompress!` method dispatched on `::NoCompressor` to bypass `copyto!(::Array, ::ReinterpretArray)`'s element-by-element walk [#280](https://github.com/JuliaIO/Zarr.jl/pull/280) -- V2 read+write chunk allocation: add `getchunkarray_undef` and skip the dead zero-fill of the chunk-shaped scratch buffer on full-overwrite paths [#280](https://github.com/JuliaIO/Zarr.jl/pull/280) +- V2 performance improvements [#280](https://github.com/JuliaIO/Zarr.jl/pull/280) + - `readblock!` fast path for single-chunk full-reads that bypasses the readtask channel and the chunk-shaped scratch buffer, decoding straight into the output array + - `writeblock!` fast path for single-chunk full-overwrites that bypasses the readtask/writetask channels and the chunk-shaped scratch buffer, encoding straight from the input array into the store + - Zero-copy chunk write for `NoCompressor` + no filters: the single-chunk fastpath hands a `reinterpret(UInt8, ain)` view straight to the store, skipping the chunk-sized `Vector{UInt8}` allocation + memcpy + - `NoCompressor` writes: replace `append!` over the reinterpret view with bulk `resize!` + `copyto!` in the generic `zcompress!` fallback + - `NoCompressor` reads: bulk-copy `zuncompress!` method dispatched on `::NoCompressor` bypasses `copyto!(::Array, ::ReinterpretArray)`'s element-by-element walk + - `getchunkarray_undef` skips the dead zero-fill of the chunk-shaped scratch buffer on full-overwrite paths - Fix CondaPkg branch in CI, use release version instead [#273](https://github.com/JuliaIO/Zarr.jl/pull/273) - Fix creation of on-disk arrays that do not fit in memory [#269](https://github.com/JuliaIO/Zarr.jl/pull/269)