py-tensorstore: bump version to 0.1.84 with bazel-8.7.0 to fix build

[giordano]

This fixes a compilation error, similar to what was described in RenderKit/openvkl#12,
the current tensorstore version fixed building with Python 3.13 on Ubuntu 24.04.

[bernhardkaindl]

@giordano, it fails to build here in a very fresh spack develop install with python-3.13.5:

pip._internal.exceptions.InstallationError: Failed to build installable wheels for some pyproject.toml based projects (tensorstore)
Removed build tracker: '/tmp/pip-build-tracker-38l17zfv'
==> Error: ProcessError: Command exited with status 1:
    '/home/bkaindl/spack/opt/spack/linux-ubuntu22.04-skylake/none-none/python-venv-1.0-fczlfuctogwhtxqmor73jl6et336aznk/bin/python3' '-m' 'pip' '-vvv' '--no-input' '--no-cache-dir' '--disable-pip-version-check' 'install' '--no-deps' '--ignore-installed' '--no-build-isolation' '--no-warn-script-location' '--no-index' '--prefix=/home/bkaindl/spack/opt/spack/linux-ubuntu22.04-skylake/gcc-13.1.0/py-tensorstore-0.1.54-r5fiq22sz657mu7c2ex56proubyw5rsy' '.'

5 errors found in build log:
     61       Loading:
     62       Loading:
     63       Loading:
     64       Loading: 0 packages loaded
     65       Analyzing: target //python/tensorstore:_tensorstore__shared_objects (1 packages loaded, 0 targets configured)
     66       INFO: Repository local_config_python instantiated at:
  >> 67     /home/bkaindl/spack/.stage/spack-stage-py-tensorstore-0.1.54-r5fiq22sz657mu7c2ex56proubyw5rsy/spack-src/WORKSPACE:7:25: in <toplevel>
  >> 68         /home/bkaindl/spack/.stage/spack-stage-py-tensorstore-0.1.54-r5fiq22sz657mu7c2ex56proubyw5rsy/spack-src/external.bzl:19:21: in tensorstore_dependencies
     69       Repository rule python_configure defined at:
  >> 70         /home/bkaindl/spack/.stage/spack-stage-py-tensorstore-0.1.54-r5fiq22sz657mu7c2ex56proubyw5rsy/spack-src/third_party/python/python_configure.bzl:313:35: in <toplevel>
     71       ERROR: An error occurred during the fetch of repository 'local_config_python':
     72          Traceback (most recent call last):
     73             File "/home/bkaindl/spack/.stage/spack-stage-py-tensorstore-0.1.54-r5fiq22sz657mu7c2ex56proubyw5rsy/spack-src/third_party/python/python_configure.bzl", line 303,
            column 40, in _python_autoconf_impl
     74                     _create_local_python_repository(repository_ctx)
     75             File "/home/bkaindl/spack/.stage/spack-stage-py-tensorstore-0.1.54-r5fiq22sz657mu7c2ex56proubyw5rsy/spack-src/third_party/python/python_configure.bzl", line 260,
            column 41, in _create_local_python_repository
     76                     python_include = _get_python_include(repository_ctx, python_bin)

     ...

     86         File "<string>", line 1, in <module>
     87           import importlib; import importlib.util; print(importlib.import_module('distutils.sysconfig').get_python_inc() if importlib.util.find_spec('distutils.sysconfig') el
            se importlib.import_module('sysconfig').get_path('include'))
     88                                                                                                                             ~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^
     89         File "<frozen importlib.util>", line 91, in find_spec
     90       ModuleNotFoundError: No module named 'distutils'
     91       Is the Python binary path set up right? (See PYTHON_BIN_PATH.)
  >> 92     ERROR: /home/bkaindl/spack/.stage/spack-stage-py-tensorstore-0.1.54-r5fiq22sz657mu7c2ex56proubyw5rsy/spack-src/WORKSPACE:7:25: fetching python_configure rule //external:l
            ocal_config_python: Traceback (most recent call last):
     93             File "/home/bkaindl/spack/.stage/spack-stage-py-tensorstore-0.1.54-r5fiq22sz657mu7c2ex56proubyw5rsy/spack-src/third_party/python/python_configure.bzl", line 303,
            column 40, in _python_autoconf_impl
     94                     _create_local_python_repository(repository_ctx)
     95             File "/home/bkaindl/spack/.stage/spack-stage-py-tensorstore-0.1.54-r5fiq22sz657mu7c2ex56proubyw5rsy/spack-src/third_party/python/python_configure.bzl", line 260,
            column 41, in _create_local_python_repository
     96                     python_include = _get_python_include(repository_ctx, python_bin)
     97             File "/home/bkaindl/spack/.stage/spack-stage-py-tensorstore-0.1.54-r5fiq22sz657mu7c2ex56proubyw5rsy/spack-src/third_party/python/python_configure.bzl", line 193,
            column 22, in _get_python_include
     98                     result = _execute(

     ...

     117                                                                                                                            ~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^
     118        File "<frozen importlib.util>", line 91, in find_spec
     119      ModuleNotFoundError: No module named 'distutils'
     120      Is the Python binary path set up right? (See PYTHON_BIN_PATH.)
     121      INFO: Elapsed time: 4.304s
     122      INFO: 0 processes.
  >> 123    FAILED: Build did NOT complete successfully (30 packages loaded, 8 targets configured)
     124      error: command '/home/bkaindl/spack/opt/spack/linux-ubuntu22.04-skylake/none-none/python-venv-1.0-fczlfuctogwhtxqmor73jl6et336aznk/bin/python3' failed with exit code 1

I addume you need to replace distutils (dropped in recent python versions) or restrict the range of python versions to the last version that has distutils.

[bernhardkaindl]

@giordano, py-tensorstore^python@3.11 and py-tensorstore^python@3.10 failed with other errors:
3.11:

ERROR: /home/bkaindl/.cache/bazel/_bazel_bkaindl/a9135e5437c7d35802fbbb4453e79c65/external/com_google_absl/absl/base/BUILD.bazel:53:11: Compiling absl/base/log_severity.cc [for tool] failed: (Exit 2): gcc failed: error executing command (from target @@com_google_absl//absl/base:log_severity)
ERROR: /home/bkaindl/spack/.stage/spack-stage-py-tensorstore-0.1.54-lrtgjk2wpdc3ftdrf7ue6q547xunmt3a/spack-src/tensorstore/internal/poly/BUILD:28:23: Compiling tensorstore/internal/poly/storage.cc failed: (Exit 2): gcc failed: error executing command (from target //tensorstore/internal/poly:storage)

Those are apparently caused by:

/home/bkaindl/spack/opt/spack/linux-ubuntu22.04-skylake/none-none/compiler-wrapper-1.0-bwtmydmaoqk5yft7tcgg5vyqw6c5v6vb/libexec/spack/gcc/gcc: 1: eval: SPACK_CC_LINKER_ARG: ERROR: LINKER ARG WAS NOT SET, MAYBE THE PACKAGE DOES NOT DEPEND ON CC?

Adding depends_on("c", type="build") appears to help for Python 3.11, this resulted in a single successful build.

For 3.10, it changes those to many other errors that could be compiler-version related (I've gcc-13), so this combination does not work. You'd have to add conflicts() or proper versions to depends_on() to that is supported.

[bernhardkaindl]

@giordano, I don't want to block this, so I want to drop my changes requested block, so the only I way I know is to approve it, from my previous comment, it looks like an conflics for gcc@13: may have to be added when python@3.10, at least this is whay my comment back then seems to suggest:

For 3.10, it changes those to many other errors that could be compiler-version related (I've gcc-13), so this combination does not work. You'd have to add conflicts() or proper versions to depends_on() to that is supported.

I'm unassigning myself as I can't drive this forward right now.

[giordano]

I'll try to have a look next week.

[bernhardkaindl]

@giordano, this old version 0.1.54 needs the depends_on updated to python@3.9:3.11, but then it does not build with gcc-13 on Ubuntu 24.04, so I suggest to skip this old version.

The latest 0.1.84 needs bazel@8.5.1 which does not exist in spack yet and might need an updated special version of unix_cc_configure-3.0.patch.

@giordano, do you still want to catch up on this PR?

It looks like it needs quite a bit of work to be buildable in general by defining the versions of compilers etc correctly.

[bernhardkaindl]

@giordano, a first successfuly build on Ubuntu 24.04:

spack install py-tensorstore 
[+] 6visxo3 bazel@8.5.1 /home/bkaindl/gh/spack/opt/spack/linux-skylake/bazel-8.5.1-6visxo37x7c44ruuup33v6hgzjc32aga (8m48s)
[+] wrws3lt py-tensorstore@0.1.84 /home/bkaindl/gh/spack/opt/spack/linux-skylake/py-tensorstore-0.1.84-wrws3ltapaya3t4j5sfcuu4qxf7rdpcy (11m56s)

Its .bazelversion file says 8.5.1, but bazel 8.7.0 works too.
(bazel-9.1.1 fails to build on Ubuntu 24.04. #5149 would need fixing in this regard)

I'll copy the bazel version update to sh#5149 as well.

[bernhardkaindl]

The current tensorstore version fixed building with Python 3.13 on Ubuntu 24.04