Flesh out the Download Data button

[ianhi]

Built on top of #26 so I could see the javascript and CSS changes locally.

Closes: #16
Closes: #25

This no longer copies the HTTPS/S3 link. But instead opens a modal with different options for download and makes it clear how to construct an S3 compatible URL

and here is the button click feedback:

Screen.Recording.2025-09-30.at.6.58.26.PM.mov

Going to github ping @will-moore because I see you've made changes and issues in this area but are not watching the repo. Just wanted to let you know that this exists

[will-moore]

Thanks for the PR...

I found I needed to specify a TMPDIR location to avoid Permission errors when installing pixi itself and with:

$ TMPDIR=/Users/wmoore && pixi install

Running pixi serve worked as expected and I can view the page.
But I don't see any Download button to launch the dialogs in your screenshots? I'll see if I can spot it in your changes...

[will-moore]

Ah - I see, it's the "Copy" button that launches the Download dialog. That's the one button I didn't try!

The S3 Endpoint URL is just https://uk1s3.embassy.ebi.ac.uk (same for all images). Is that right? Or is that supposed to be e.g. https://uk1s3.embassy.ebi.ac.uk/idr/zarr/v0.4/idr0048A/9846152.zarr/?

The download dialog is really nice, but maybe it should live under a download button and leave the "Copy" button to just do Copy. That's the one-click action that I use the most for these samples.

[ianhi]

The download dialog is really nice, but maybe it should live under a download button and leave the "Copy" button to just do Copy. That's the one-click action that I use the most for these samples.

Sure thing I can make it be a download button. That said this was originally an issue for me because the copy URL isn't really a valid S3 URL. See discussion here: zarr-developers/zarr-python#3496 (comment)

The S3 Endpoint URL is just https://uk1s3.embassy.ebi.ac.uk (same for all images). Is that right? Or is that supposed to be e.g. https://uk1s3.embassy.ebi.ac.uk/idr/zarr/v0.4/idr0048A/9846152.zarr/?

Yes that's end point, then each image gets an S3 url with the rest of the path. I took inspiration from the aws cli download link at the top of the page

I found I needed to specify a TMPDIR location to avoid Permission errors when installing pixi itself and with:

Huh weird, can you post the error message so I can update the readme properly. I've never experienced that

[will-moore]

$ echo $TMPDIR 
/var/folders/zz/zyxvpxvq6csfxvn_n0000000000000/T/

$ curl -fsSL https://pixi.sh/install.sh | sh
This script will automatically download and install Pixi (latest) for you.
Getting it from this url: https://github.com/prefix-dev/pixi/releases/latest/download/pixi-aarch64-apple-darwin.tar.gz
mktemp: mkstemp failed on /var/folders/zz/zyxvpxvq6csfxvn_n0000000000000/T//.pixi_install.OGyooWFE: Permission denied

I'm not sure how to replicate the $ pixi install error, now that everything is already installed, but it was similar.

[will-moore]

I'm not sure I understand zarr-developers/zarr-python#3496.
It may be correct that "https:" URLs are not "S3" URLs, so the naming needs fixed.

But this works OK:

import zarr
z = zarr.open("https://uk1s3.embassy.ebi.ac.uk/idr/zarr/v0.5/idr0062A/6001240_labels.zarr")
print(dict(z.attrs))

[ianhi]

I found that very confusing at first. My understanding is that

It may be correct that "https:" URLs are not "S3" URLs, so the naming needs fixed.

Unfortunately there seems to be no mechanism for specifying both an endpoint url and the bucket/path in a single string. So there isn't really a valid way to "fix" the naming.

It took me a while to figure out why the ome-zarr cli could handle this, and why your code example runs. I think the short of it is that there are enough similarities between an S3 store and HTTPS store that we can kind of get away with it. You code example runs because it is requesting the content of a file in a known spot. However if you try to list the keys available on the store it will (incorrectly) return nothing.

For example:
(run this with uv run <script-name>.py)

# /// script
# requires-python = ">=3.11"
# dependencies = [
#   "zarr",
#   "s3fs",
# ]
# ///


import zarr

z = zarr.open(
    "https://uk1s3.embassy.ebi.ac.uk/idr/zarr/v0.5/idr0062A/6001240_labels.zarr"
)


print("KEYS")
print(list(z.keys()))

# despite there not being keys you can just go ahead an list things

print(z["0"].shape)


# Opening as a true S3 store
z = zarr.open(
    "s3://idr/zarr/v0.5/idr0062A/6001240_labels.zarr",
    storage_options={
        "anon": True,
        "client_kwargs": {"endpoint_url": "https://uk1s3.embassy.ebi.ac.uk/"},
    },
)

print(z["0"].shape)

gives:

KEYS
[] # <- not right!
(2, 236, 275, 271)


KEYS
['1', '2', '0', 'labels']
(2, 236, 275, 271)

which can potentially have an impact on any downstream application that loads the zarr and lists the groups.

ome-zarr-py and some of the other ome-zarr visualizers have escaped this because they directly use the OME metadata to know what is avaiable to them, rather than the zarr api.

I'm certainly not arguing that the second way to access is better, e.g. see my rejected PR. Just that is it technically correct way to access that store.

[will-moore]

Ah, OK - I understand a bit better now. I've never actually used group.keys() because I'm often dealing with web apps where we don't expect to be able to list keys from http URLs. So OME-Zarr doesn't assume that you can list keys, and I've always assumed you wouldn't be able to.
Interesting to see that you can anyway.

In your example, I see that you get a warning when you do this at the end:

print(list(z.keys()))

/Users/wmoore/opt/anaconda3/envs/omeroweb_zarrv3/lib/python3.12/site-packages/zarr/core/group.py:3390: 
ZarrUserWarning: Object at 6001240_labels.zarr is not recognized as a component of a Zarr hierarchy.
  warnings.warn(
['0', '1', 'labels', '2']

[ianhi]

Interesting how we have different perspectives on zarr usage from using it in different contexts!

So is this URL meant to be used as an HTTP store? If so, why is it using S3 infrastructure to achieve that, can that be easily changed? Or maybe im just fully not understanding there. it is also an HTTP url hosted on S3 style infrastructure, and I should not have used .keys() on an http store and expected it to work

In your example, I see that you get a warning when you do this at the end:

I listed all the objects with : aws s3 ls s3://idr/zarr/v0.5/idr0062A/6001240_labels.zarr/ --endpoint-url https://uk1s3.embassy.ebi.ac.uk/ --no-sign-request --recursive

and that error is coming from this line: https://github.com/zarr-developers/zarr-python/blob/be519b006f3073c3c0d56801cd2daa597538fd59/src/zarr/core/group.py#L3535-L3549

due to this extra object in the hierarchy:

zarr/v0.5/idr0062A/6001240_labels.zarr/6001240_labels.zarr/ro-crate-metadata.json

[ianhi]

my one real concern with leaving hte copy button as is is that there will me more users who, like I did, try to put that into zarr and get hit with a confusing situation. Is there an easy way to protect people from that, while also maintaining the ease of the one click workflow for knowledgeable users? What if shift-click the button still copied, and normal click opened the modal. With some instructions at the top about this.

[will-moore]

OK, I guess it doesn't have to be just one click!
Maybe just add the https:// URL to the dialog, so it can be copied there like the others. I'm just not sure what to call it!

Also, it would be nice to enable dialog closing when you click on the background.
This seems to do the job for me:

+++ b/index.md
@@ -217,10 +217,15 @@ function showS3Options(httpsUrl, imageName) {
     document.getElementById('pythonCodeInOptions').dataset.copyText = python.code;
 
     modal.style.display = 'block';
+
+    modal.onclick = closeS3Options;
 }
 
-function closeS3Options() {
-    document.getElementById('s3OptionsModal').style.display = 'none';
+function closeS3Options(event) {
+    // event is undifined if click comes from x button
+    if (event == undefined || event?.target.id == "s3OptionsModal") {
+        document.getElementById('s3OptionsModal').style.display = 'none';
+    }
 }

I tried this again...

$ pixi run serve
✨ Pixi task (install): gem install bundler && bundle install                                                                                        Error:   × failed to activate environment
  ╰─▶ An activation error occurred: IoError(Custom { kind: PermissionDenied, error: PathError { path: "/var/folders/zz/
      zyxvpxvq6csfxvn_n0000000000000/T/.tmprKyZeg", err: Os { code: 13, kind: PermissionDenied, message: "Permission denied" } } })

[ianhi]

Also, it would be nice to enable dialog closing when you click on the background.

ahhh yes good call. I hate it when modals don't do that, can't believe i didn't think of that.

I tried this again...

gahh. that is so bizarre. I've never seen that before. This is after just following the instructions here? https://pixi.sh/dev/installation/ I wonder if it's that your pixi install is somewhat broken, or if somehow the ruby bundler is trying to access other areas

[ianhi]

Looking at your earlier error. that looks like some sort of permission error on your laptop with temp dirs. can you maybe try installing wiht brew? https://pixi.sh/latest/installation/#homebrew

[will-moore]

Seems that the problem is that I don't have permissions to write to my default $TMPDIR location.

echo $TMPDIR
/var/folders/zz/zyxvpxvq6csfxvn_n0000000000000/T/

$ ls $TMPDIR
ls: /var/folders/zz/zyxvpxvq6csfxvn_n0000000000000/T/: Permission denied

I also don't have sudo permissions on my Unversity-administered laptop. But I can at least run the commands with that work-around so that's fine for now.

[ianhi]

I added your closing modal change. I'll add a line to readme about this tmpdir stuff

[ianhi]

You can probably permanently apply the workaround:

echo 'export TMPDIR=~/tmp' >> ~/.zshrc

I also don't have sudo permissions on my Unversity-administered laptop.

:'(

[ianhi]

Also if you're in a mood to review, can I ask you to take a look at #27

[ianhi]

anything more to change here? @joshmoore heads up on this one and #27 in addition to #26

[will-moore]

@ianhi "Maybe just add the https:// URL to the dialog, so it can be copied there like the others. I'm just not sure what to call it!"

The URL that is currently copied to the clipboard should still be easily copyable. So you can see & copy it directly from the dialog. Preferably being the first (most obvious) option from the dialog, so that users who previously just clicked the "copy" button can easily see where to get the same thing from the dialog.

[ianhi]

Changes:

1. added HTTPS URL to the modal
2. Updated the docs at the top from just aws download to poitning users to the button:

3. grouped each section with an outline to make it easy to see that the two copy links for S3 URLs are the grouped. Added to everything so it didn't look weird.

[will-moore]

Testing locally, with main merged in. Looks great, thanks.

@joshmoore you OK to merge (I don't have permissions)?

[will-moore]

@ianhi could you merge origin/main into your branch so we can see the changes here a bit clearer? Thanks.

[joshmoore]

I merged in main. Looks good. Thanks, all!

[ianhi]

thanks both!