Upgrade to zarr-java 0.1.3 by melissalinkert · Pull Request #327 · glencoesoftware/bioformats2raw

melissalinkert · 2026-06-11T16:13:55Z

Splitting zarr-java upgrade out of #322, since it can be considered separately (as discussed briefly with @sbesson).

melissalinkert · 2026-06-11T16:54:18Z

Build failing with:

Execution failed for task ':compileJava'.
> Could not resolve all files for configuration ':compileClasspath'.

   > Could not find edu.ucar:cdm-core:5.9.1.
     Searched in the following locations:
Deprecated Gradle features were used in this build, making it incompatible with Gradle 10.
       - file:/home/runner/.m2/repository/edu/ucar/cdm-core/5.9.1/cdm-core-5.9.1.pom

       - https://repo.maven.apache.org/maven2/edu/ucar/cdm-core/5.9.1/cdm-core-5.9.1.pom
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
       - https://artifacts.glencoesoftware.com/artifactory/ome.releases/edu/ucar/cdm-core/5.9.1/cdm-core-5.9.1.pom

       - https://artifacts.glencoesoftware.com/artifactory/ome.external/edu/ucar/cdm-core/5.9.1/cdm-core-5.9.1.pom
For more on this, please refer to https://docs.gradle.org/9.3.1/userguide/command_line_interface.html#sec:command_line_warnings in the Gradle documentation.
1 actionable task: 1 executed
       - https://artifacts.glencoesoftware.com/artifactory/scijava-thirdparty/edu/ucar/cdm-core/5.9.1/cdm-core-5.9.1.pom
       - https://artifacts.glencoesoftware.com/artifactory/unidata-releases/edu/ucar/cdm-core/5.9.1/cdm-core-5.9.1.pom
     Required by:
         root project 'bioformats2raw' > ome:formats-gpl:8.5.0
   > Could not find edu.ucar:cdm-core:5.9.1.
     Searched in the following locations:
       - file:/home/runner/.m2/repository/edu/ucar/cdm-core/5.9.1/cdm-core-5.9.1.pom
       - https://repo.maven.apache.org/maven2/edu/ucar/cdm-core/5.9.1/cdm-core-5.9.1.pom
       - https://artifacts.glencoesoftware.com/artifactory/ome.releases/edu/ucar/cdm-core/5.9.1/cdm-core-5.9.1.pom
       - https://artifacts.glencoesoftware.com/artifactory/ome.external/edu/ucar/cdm-core/5.9.1/cdm-core-5.9.1.pom
       - https://artifacts.glencoesoftware.com/artifactory/scijava-thirdparty/edu/ucar/cdm-core/5.9.1/cdm-core-5.9.1.pom
       - https://artifacts.glencoesoftware.com/artifactory/unidata-releases/edu/ucar/cdm-core/5.9.1/cdm-core-5.9.1.pom
     Required by:
         root project 'bioformats2raw' > dev.zarr:zarr-java:0.1.3

edu.ucar:cdm-core:5.9.1 has been required for a while, independent of zarr-java 0.1.3. I think I would have expected it to exist in https://artifacts.glencoesoftware.com/ui/native/unidata-releases/edu/ucar/cdm-core/5.9.1/, but that is currently empty.

If I remove my local ~/.m2/repository/edu/ucar/cdm-core/, then I can reproduce the build failure on this branch, current master, and v0.12.0.

sbesson

Thanks for extracting these changes @melissalinkert. The primary motivation is to add support for converting dataset using the default blosc compression in Linux/arm64 environments. /cc @Tom-TBT

Tested using a minimal AWS EC2 instance based on the Ubuntu 26.04 arm64 AMI. Using bioformats2raw 0.12.0, a minimal conversion using blosc as the default compression fails while trying to load the native library:

ubuntu@ip-10-0-0-243:~$ ./bioformats2raw-0.12.0/bin/bioformats2raw test.fake test.zarr
2026-06-12 08:10:01,096 [pool-1-thread-1] ERROR c.g.bioformats2raw.Converter - Failure processing chunk; resolution=0 plane=0 xx=0 yy=0 zz=0 width=512 height=512 depth=1
java.lang.UnsatisfiedLinkError: /tmp/libbloscjni11160771468050511226.so: /tmp/libbloscjni11160771468050511226.so: cannot open shared object file: No such file or directory (Possible cause: can't load AMD 64 .so on a AARCH64 platform)
	at java.base/jdk.internal.loader.NativeLibraries.load(Native Method)
	at java.base/jdk.internal.loader.NativeLibraries$NativeLibraryImpl.open(NativeLibraries.java:331)
	at java.base/jdk.internal.loader.NativeLibraries.loadLibrary(NativeLibraries.java:197)
	at java.base/jdk.internal.loader.NativeLibraries.loadLibrary(NativeLibraries.java:139)
	at java.base/java.lang.ClassLoader.loadLibrary(ClassLoader.java:2418)
	at java.base/java.lang.Runtime.load0(Runtime.java:852)
	at java.base/java.lang.System.load(System.java:2026)
	at com.scalableminds.bloscjava.Blosc.<clinit>(Blosc.java:17)
	at dev.zarr.zarrjava.v2.codec.core.BloscCodec.encode(BloscCodec.java:65)
	at dev.zarr.zarrjava.core.codec.CodecPipeline.encode(CodecPipeline.java:150)
	at dev.zarr.zarrjava.core.Array.writeChunk(Array.java:153)
	at dev.zarr.zarrjava.core.Array.lambda$write$0(Array.java:121)
	at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
	at java.base/java.util.Spliterators$ArraySpliterator.forEachRemaining(Spliterators.java:1024)
	at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
	at java.base/java.util.stream.ForEachOps$ForEachTask.compute(ForEachOps.java:291)
	at java.base/java.util.concurrent.CountedCompleter.exec(CountedCompleter.java:754)
	at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:387)
	at java.base/java.util.concurrent.ForkJoinTask.invoke(ForkJoinTask.java:667)
	at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateParallel(ForEachOps.java:160)
	at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateParallel(ForEachOps.java:174)
	at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:233)
	at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:596)
	at java.base/java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:765)
	at dev.zarr.zarrjava.core.Array.write(Array.java:99)
	at dev.zarr.zarrjava.core.Array.write(Array.java:309)
	at com.glencoesoftware.bioformats2raw.Converter.writeBytes(Converter.java:2014)
	at com.glencoesoftware.bioformats2raw.Converter.processChunk(Converter.java:2391)
	at com.glencoesoftware.bioformats2raw.Converter.lambda$saveResolutions$6(Converter.java:2623)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
	at java.base/java.lang.Thread.run(Thread.java:1583)
2026-06-12 08:10:01,095 [main] ERROR c.g.bioformats2raw.Converter - Error while writing series 0
java.util.concurrent.CompletionException: java.lang.UnsatisfiedLinkError: /tmp/libbloscjni11160771468050511226.so: /tmp/libbloscjni11160771468050511226.so: cannot open shared object file: No such file or directory (Possible cause: can't load AMD 64 .so on a AARCH64 platform)
	at java.base/java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:332)
	at java.base/java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:347)
	at java.base/java.util.concurrent.CompletableFuture$BiRelay.tryFire(CompletableFuture.java:1498)
	at java.base/java.util.concurrent.CompletableFuture$CoCompletion.tryFire(CompletableFuture.java:1219)
	at java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:510)
	at java.base/java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:2194)
	at com.glencoesoftware.bioformats2raw.Converter.lambda$saveResolutions$6(Converter.java:2632)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
	at java.base/java.lang.Thread.run(Thread.java:1583)
Caused by: java.lang.UnsatisfiedLinkError: /tmp/libbloscjni11160771468050511226.so: /tmp/libbloscjni11160771468050511226.so: cannot open shared object file: No such file or directory (Possible cause: can't load AMD 64 .so on a AARCH64 platform)
	at java.base/jdk.internal.loader.NativeLibraries.load(Native Method)
	at java.base/jdk.internal.loader.NativeLibraries$NativeLibraryImpl.open(NativeLibraries.java:331)
	at java.base/jdk.internal.loader.NativeLibraries.loadLibrary(NativeLibraries.java:197)
	at java.base/jdk.internal.loader.NativeLibraries.loadLibrary(NativeLibraries.java:139)
	at java.base/java.lang.ClassLoader.loadLibrary(ClassLoader.java:2418)
	at java.base/java.lang.Runtime.load0(Runtime.java:852)
	at java.base/java.lang.System.load(System.java:2026)
	at com.scalableminds.bloscjava.Blosc.<clinit>(Blosc.java:17)
	at dev.zarr.zarrjava.v2.codec.core.BloscCodec.encode(BloscCodec.java:65)
	at dev.zarr.zarrjava.core.codec.CodecPipeline.encode(CodecPipeline.java:150)
	at dev.zarr.zarrjava.core.Array.writeChunk(Array.java:153)
	at dev.zarr.zarrjava.core.Array.lambda$write$0(Array.java:121)
	at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
	at java.base/java.util.Spliterators$ArraySpliterator.forEachRemaining(Spliterators.java:1024)
	at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
	at java.base/java.util.stream.ForEachOps$ForEachTask.compute(ForEachOps.java:291)
	at java.base/java.util.concurrent.CountedCompleter.exec(CountedCompleter.java:754)
	at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:387)
	at java.base/java.util.concurrent.ForkJoinTask.invoke(ForkJoinTask.java:667)
	at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateParallel(ForEachOps.java:160)
	at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateParallel(ForEachOps.java:174)
	at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:233)
	at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:596)
	at java.base/java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:765)
	at dev.zarr.zarrjava.core.Array.write(Array.java:99)
	at dev.zarr.zarrjava.core.Array.write(Array.java:309)
	at com.glencoesoftware.bioformats2raw.Converter.writeBytes(Converter.java:2014)
	at com.glencoesoftware.bioformats2raw.Converter.processChunk(Converter.java:2391)
	at com.glencoesoftware.bioformats2raw.Converter.lambda$saveResolutions$6(Converter.java:2623)
	... 3 common frames omitted
java.lang.RuntimeException: java.util.concurrent.CompletionException: java.lang.UnsatisfiedLinkError: /tmp/libbloscjni11160771468050511226.so: /tmp/libbloscjni11160771468050511226.so: cannot open shared object file: No such file or directory (Possible cause: can't load AMD 64 .so on a AARCH64 platform)
	at com.glencoesoftware.bioformats2raw.Converter.unwrapException(Converter.java:3186)
	at com.glencoesoftware.bioformats2raw.Converter.convert(Converter.java:1681)
	at com.glencoesoftware.bioformats2raw.Converter.call(Converter.java:1487)
	at com.glencoesoftware.bioformats2raw.Converter.call(Converter.java:113)
	at picocli.CommandLine.executeUserObject(CommandLine.java:2041)
	at picocli.CommandLine.access$1500(CommandLine.java:148)
	at picocli.CommandLine$RunLast.executeUserObjectOfLastSubcommandWithSameParent(CommandLine.java:2461)
	at picocli.CommandLine$RunLast.handle(CommandLine.java:2453)
	at picocli.CommandLine$RunLast.handle(CommandLine.java:2415)
	at picocli.CommandLine$AbstractParseResultHandler.execute(CommandLine.java:2273)
	at picocli.CommandLine$RunLast.execute(CommandLine.java:2417)
	at picocli.CommandLine.execute(CommandLine.java:2170)
	at com.glencoesoftware.bioformats2raw.Converter.main(Converter.java:3561)
Caused by: java.util.concurrent.CompletionException: java.lang.UnsatisfiedLinkError: /tmp/libbloscjni11160771468050511226.so: /tmp/libbloscjni11160771468050511226.so: cannot open shared object file: No such file or directory (Possible cause: can't load AMD 64 .so on a AARCH64 platform)
	at java.base/java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:332)
	at java.base/java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:347)
	at java.base/java.util.concurrent.CompletableFuture$BiRelay.tryFire(CompletableFuture.java:1498)
	at java.base/java.util.concurrent.CompletableFuture$CoCompletion.tryFire(CompletableFuture.java:1219)
	at java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:510)
	at java.base/java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:2194)
	at com.glencoesoftware.bioformats2raw.Converter.lambda$saveResolutions$6(Converter.java:2632)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
	at java.base/java.lang.Thread.run(Thread.java:1583)
Caused by: java.lang.UnsatisfiedLinkError: /tmp/libbloscjni11160771468050511226.so: /tmp/libbloscjni11160771468050511226.so: cannot open shared object file: No such file or directory (Possible cause: can't load AMD 64 .so on a AARCH64 platform)
	at java.base/jdk.internal.loader.NativeLibraries.load(Native Method)
	at java.base/jdk.internal.loader.NativeLibraries$NativeLibraryImpl.open(NativeLibraries.java:331)
	at java.base/jdk.internal.loader.NativeLibraries.loadLibrary(NativeLibraries.java:197)
	at java.base/jdk.internal.loader.NativeLibraries.loadLibrary(NativeLibraries.java:139)
	at java.base/java.lang.ClassLoader.loadLibrary(ClassLoader.java:2418)
	at java.base/java.lang.Runtime.load0(Runtime.java:852)
	at java.base/java.lang.System.load(System.java:2026)
	at com.scalableminds.bloscjava.Blosc.<clinit>(Blosc.java:17)
	at dev.zarr.zarrjava.v2.codec.core.BloscCodec.encode(BloscCodec.java:65)
	at dev.zarr.zarrjava.core.codec.CodecPipeline.encode(CodecPipeline.java:150)
	at dev.zarr.zarrjava.core.Array.writeChunk(Array.java:153)
	at dev.zarr.zarrjava.core.Array.lambda$write$0(Array.java:121)
	at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
	at java.base/java.util.Spliterators$ArraySpliterator.forEachRemaining(Spliterators.java:1024)
	at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
	at java.base/java.util.stream.ForEachOps$ForEachTask.compute(ForEachOps.java:291)
	at java.base/java.util.concurrent.CountedCompleter.exec(CountedCompleter.java:754)
	at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:387)
	at java.base/java.util.concurrent.ForkJoinTask.invoke(ForkJoinTask.java:667)
	at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateParallel(ForEachOps.java:160)
	at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateParallel(ForEachOps.java:174)
	at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:233)
	at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:596)
	at java.base/java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:765)
	at dev.zarr.zarrjava.core.Array.write(Array.java:99)
	at dev.zarr.zarrjava.core.Array.write(Array.java:309)
	at com.glencoesoftware.bioformats2raw.Converter.writeBytes(Converter.java:2014)
	at com.glencoesoftware.bioformats2raw.Converter.processChunk(Converter.java:2391)
	at com.glencoesoftware.bioformats2raw.Converter.lambda$saveResolutions$6(Converter.java:2623)
	... 3 more

With this PR included, the version of the blosc-java library includes the blosc library compiled under Linux arm64 and the samel conversion tests succeeds with OME-Zarr 0.4

ubuntu@ip-10-0-0-243:~$ ./bioformats2raw-0.13.0-SNAPSHOT/bin/bioformats2raw test.fake test.zarr

or OME-Zarr 0.5

ubuntu@ip-10-0-0-243:~$ ./bioformats2raw-0.13.0-SNAPSHOT/bin/bioformats2raw test.fake test.zarr --ngff-version 0.5

As noted in #322, the renaming of byteshuffle to shuffle is related to an upstream change of the enum in blosc-java, specifically scalableminds/blosc-java@228abf8. My understanding is that the former byteshuffle value was inherited from the jzarr implementation. The shuffle term is consistent with the Zarr v3 blosc codec specification and reference libraries like python-blosc.

In addition, dev.zarr:zarr-java:0.1.3 is the version of the library we have been using in the current work for adding Zarr v3 support in OMERO.

sbesson · 2026-06-30T14:10:34Z

The integration of this work has been primarily held off due to open questions regarding the state of dev.zarr:zarr-java:0.1.3. More specifically, there were unclarities in the mapping between the underlying com.scalableminds:blosc-java:0.3-1.21.6 dependency and the source code used to generate the artifact published to Maven Central. These concerns, originally raised here, have been addressed with the integration of the relevant commits in the main development line of blosc-java. As mentioned in the conversation linked above, there will be additional discussions to consolidate the maintenance process of zarr-java.

Following a discussion with @erindiel, @melissalinkert and @chris-allan and in the absence of objections, merging this in preparation of a patch release of bioformats2raw.

Upgrade to zarr-java 0.1.3

0472207

melissalinkert added this to the 0.13.0 milestone Jun 11, 2026

melissalinkert mentioned this pull request Jun 11, 2026

Automatically use zip storage if the .ozx output file extension is used #322

Draft

sbesson approved these changes Jun 12, 2026

View reviewed changes

melissalinkert marked this pull request as ready for review June 12, 2026 13:28

melissalinkert modified the milestones: 0.13.0, 0.12.1 Jun 24, 2026

sbesson merged commit b03b288 into glencoesoftware:master Jun 30, 2026
6 of 12 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Upgrade to zarr-java 0.1.3#327

Upgrade to zarr-java 0.1.3#327
sbesson merged 1 commit into
glencoesoftware:masterfrom
melissalinkert:zarr-java-0.1.3

melissalinkert commented Jun 11, 2026

Uh oh!

melissalinkert commented Jun 11, 2026

Uh oh!

sbesson left a comment

Uh oh!

sbesson commented Jun 30, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

melissalinkert commented Jun 11, 2026

Uh oh!

melissalinkert commented Jun 11, 2026

Uh oh!

sbesson left a comment

Choose a reason for hiding this comment

Uh oh!

sbesson commented Jun 30, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants