Skip to content

Installation issue: Packages depending on swig and numpy build with gcc@14: #5269

Description

@FlyingSamson

Steps to reproduce the issue

faiss (@bhatiaharsh, @lpottier, @rblake-llnl) is one package that uses both py-numpy and swig that allows to reproduce the issue. I think this is mainly an issue to be solved in the swig (@sethrj) and potentially numpy (@adamjstewart, @rgommers) recipes, though.

$ spack spec -I faiss ^py-numpy@2.2.6 ^swig@4.4.1
 -   faiss@1.8.0~cuda~ipo+python~shared~tests build_system=cmake build_type=Release generator=make platform=linux os=ubuntu24.04 target=x86_64_v3 %c,cxx=gcc@14.2.0
[+]      ^cmake@3.24.4~doc+ncurses+ownlibs~qtgui build_system=generic build_type=Release patches:=dbc3892,fdea723 platform=linux os=ubuntu24.04 target=x86_64_v3 %c,cxx=gcc@14.2.0
[e]          ^curl@8.5.0~gssapi~ldap~libidn2~librtmp~libssh~libssh2+nghttp2 build_system=autotools libs:=shared,static tls:=openssl platform=linux os=ubuntu24.04 target=x86_64 
[e]          ^ncurses@6.3~symlinks+termlib abi=none build_system=autotools platform=linux os=ubuntu24.04 target=x86_64 
[^]          ^zlib-ng@2.3.3+compat+new_strategies+opt+pic+shared build_system=autotools platform=linux os=ubuntu24.04 target=x86_64_v3 %c,cxx=gcc@14.2.0
[^]      ^compiler-wrapper@1.0 build_system=generic platform=linux os=ubuntu24.04 target=x86_64_v3 
[e]      ^gcc@14.2.0+binutils+bootstrap~graphite+libsanitizer~mold~nvptx~piclibs~profiled~strip build_system=autotools build_type=RelWithDebInfo languages:='c,c++,fortran' platform=linux os=ubuntu24.04 target=x86_64 
[^]      ^gcc-runtime@14.2.0 build_system=generic platform=linux os=ubuntu24.04 target=x86_64_v3 
[e]      ^glibc@2.39 build_system=autotools platform=linux os=ubuntu24.04 target=x86_64 
[^]      ^gmake@4.4.1~guile build_system=generic platform=linux os=ubuntu24.04 target=x86_64_v3 %c=gcc@14.2.0
[+]      ^openblas@0.3.30~bignuma~consistent_fpcsr+dynamic_dispatch+fortran~ilp64~ipo+locking+pic+shared build_system=cmake build_type=Release generator=make symbol_suffix=none threads=none platform=linux os=ubuntu24.04 target=x86_64_v3 %c,cxx,fortran=gcc@14.2.0
[+]      ^py-numpy@2.2.6 build_system=python_pip patches:=1c9cb08,873745d platform=linux os=ubuntu24.04 target=x86_64_v3 %c,cxx=gcc@14.2.0
[+]          ^py-cython@3.2.4 build_system=python_pip platform=linux os=ubuntu24.04 target=x86_64_v3 %c,cxx=gcc@14.2.0
[+]          ^py-meson-python@0.19.0 build_system=python_pip platform=linux os=ubuntu24.04 target=x86_64_v3 %c=gcc@14.2.0
[+]              ^meson@1.10.1 build_system=python_pip patches:=0f0b1bd platform=linux os=ubuntu24.04 target=x86_64_v3 
[+]              ^ninja@1.13.2+re2c build_system=generic platform=linux os=ubuntu24.04 target=x86_64_v3 %c,cxx=gcc@14.2.0
[+]                  ^re2c@4.4 build_system=autotools platform=linux os=ubuntu24.04 target=x86_64_v3 %c,cxx=gcc@14.2.0
[+]              ^py-packaging@26.0 build_system=python_pip platform=linux os=ubuntu24.04 target=x86_64_v3 
[+]                  ^py-flit-core@3.12.0 build_system=python_pip platform=linux os=ubuntu24.04 target=x86_64_v3 
[+]              ^py-pyproject-metadata@0.11.0 build_system=python_pip platform=linux os=ubuntu24.04 target=x86_64_v3 
[+]      ^py-pip@25.1.1 build_system=generic platform=linux os=ubuntu24.04 target=x86_64_v3 
[+]      ^py-setuptools@63.4.3 build_system=generic platform=linux os=ubuntu24.04 target=x86_64_v3 
[+]      ^py-wheel@0.45.1 build_system=generic platform=linux os=ubuntu24.04 target=x86_64_v3 
[+]      ^python@3.11.15+bz2+crypt+ctypes+dbm~debug+libxml2+lzma~optimizations+pic+pyexpat+pythoncmd+readline+shared+sqlite3+ssl~static~tests~tkinter+uuid+zlib build_system=generic patches:=13fa8bf,b0615b2,ebdca64,f2fd060 platform=linux os=ubuntu24.04 target=x86_64_v3 %c,cxx=gcc@14.2.0
[e]          ^bzip2@1.0.8~debug~pic+shared build_system=generic platform=linux os=ubuntu24.04 target=x86_64 
[e]          ^expat@2.6.1+libbsd build_system=autotools platform=linux os=ubuntu24.04 target=x86_64 
[e]          ^gdbm@1.23.1 build_system=autotools platform=linux os=ubuntu24.04 target=x86_64 
[+]          ^gettext@1.0+bzip2+curses+git~libunistring+libxml2+pic+shared+tar+xz build_system=autotools platform=linux os=ubuntu24.04 target=x86_64_v3 %c,cxx=gcc@14.2.0
[^]              ^libiconv@1.18 build_system=autotools libs:=shared,static platform=linux os=ubuntu24.04 target=x86_64_v3 %c=gcc@14.2.0
[^]              ^libxml2@2.15.1+pic~python+shared build_system=autotools platform=linux os=ubuntu24.04 target=x86_64_v3 %c=gcc@14.2.0
[+]              ^tar@1.35 build_system=autotools zip=pigz platform=linux os=ubuntu24.04 target=x86_64_v3 %c=gcc@14.2.0
[+]                  ^pigz@2.8 build_system=makefile platform=linux os=ubuntu24.04 target=x86_64_v3 %c=gcc@14.2.0
[+]                  ^zstd@1.5.7+programs build_system=makefile compression:=none libs:=shared,static platform=linux os=ubuntu24.04 target=x86_64_v3 %c,cxx=gcc@14.2.0
[e]          ^libffi@3.4.2 build_system=autotools platform=linux os=ubuntu24.04 target=x86_64 
[+]          ^libxcrypt@4.5.2~obsolete_api build_system=autotools platform=linux os=ubuntu24.04 target=x86_64_v3 %c=gcc@14.2.0
[e]              ^perl@5.38.2+cpanm+opcode+open+shared+threads build_system=generic platform=linux os=ubuntu24.04 target=x86_64 
[e]          ^openssl@3.0.13~docs+shared build_system=generic certs=mozilla platform=linux os=ubuntu24.04 target=x86_64 
[e]          ^pkg-config@1.8.1+internal_glib build_system=autotools platform=linux os=ubuntu24.04 target=x86_64 
[+]          ^readline@8.3 build_system=autotools patches:=21f0a03 platform=linux os=ubuntu24.04 target=x86_64_v3 %c=gcc@14.2.0
[e]          ^sqlite@3.37.2+column_metadata+dynamic_extensions~fts~functions+rtree build_system=autotools platform=linux os=ubuntu24.04 target=x86_64 
[e]          ^util-linux-uuid@2.37.2 build_system=autotools platform=linux os=ubuntu24.04 target=x86_64 
[e]          ^xz@5.2.4~pic build_system=autotools libs:=shared,static platform=linux os=ubuntu24.04 target=x86_64 
[+]      ^python-venv@1.0 build_system=generic platform=linux os=ubuntu24.04 target=x86_64_v3 
[+]      ^swig@4.4.1 build_system=autotools platform=linux os=ubuntu24.04 target=x86_64_v3 %c,cxx=gcc@14.2.0
[+]          ^pcre2@10.44~jit+multibyte+pic build_system=autotools platform=linux os=ubuntu24.04 target=x86_64_v3 %c=gcc@14.2.0

Error message

Error message
==> Installing faiss-1.8.0-hxvgql7ss2h2vftggt4n5iwxig3zmgz5 [35/35]
==> Using cached archive: /opt/spack/var/spack/cache/_source-cache/archive/56/56ece0a419d62eaa11e39022fa27c8ed6d5a9b9eb7416cc5a0fdbeab07ec2f0c.tar.gz
==> No patches needed for faiss
==> faiss: Executing phase: 'cmake'
==> faiss: Executing phase: 'build'
==> Error: ProcessError: Command exited with status 2:
    '/opt/mpi_installation/gmake/bin/make' '-j8'

5 errors found in build log:
     262    make[2]: Leaving directory '/tmp/fs/spack-stage/spack-stage-faiss-1.8.0-hxvgql7ss2h2vftggt4n5iwxig3zmgz5/spack-build-hxvgql7'
     263    /opt/mpi_installation/gmake/bin/make  -f faiss/python/CMakeFiles/swigfaiss.dir/build.make faiss/python/CMakeFiles/swigfaiss.dir/build
     264    make[2]: Entering directory '/tmp/fs/spack-stage/spack-stage-faiss-1.8.0-hxvgql7ss2h2vftggt4n5iwxig3zmgz5/spack-build-hxvgql7'
     265    [100%] Building CXX object faiss/python/CMakeFiles/swigfaiss.dir/CMakeFiles/swigfaiss.dir/swigfaissPYTHON_wrap.cxx.o
     266    cd /tmp/fs/spack-stage/spack-stage-faiss-1.8.0-hxvgql7ss2h2vftggt4n5iwxig3zmgz5/spack-build-hxvgql7/faiss/python && /opt/mpi_installation/compiler-wrapper/libexec/spack/gcc/g++ -Dswigfaiss_EXPORTS -I/tmp/fs/spack-stage/spack-stage-faiss-1.8.0-hxvgql7ss2h2vftgg
            t4n5iwxig3zmgz5/spack-src/faiss/python/../.. -I/tmp/fs/spack-stage/spack-stage-faiss-1.8.0-hxvgql7ss2h2vftggt4n5iwxig3zmgz5/spack-src -isystem /opt/fs/spack/install/linux-x86_64_v3/python-3.11.15-nh4prrfgrcpu46qwofhmgc64ibselgti/include/python3.11 -isystem /op
            t/fs/spack/install/linux-x86_64_v3/py-numpy-2.2.6-yyh5lw62a6v4ad2fjxn5cyfg2iawl46z/lib/python3.11/site-packages/numpy/_core/include -O3 -DNDEBUG -fPIC -fopenmp -std=gnu++17 -MD -MT faiss/python/CMakeFiles/swigfaiss.dir/CMakeFiles/swigfaiss.dir/swigfaissPYTHON_
            wrap.cxx.o -MF CMakeFiles/swigfaiss.dir/CMakeFiles/swigfaiss.dir/swigfaissPYTHON_wrap.cxx.o.d -o CMakeFiles/swigfaiss.dir/CMakeFiles/swigfaiss.dir/swigfaissPYTHON_wrap.cxx.o -c /tmp/fs/spack-stage/spack-stage-faiss-1.8.0-hxvgql7ss2h2vftggt4n5iwxig3zmgz5/spack-
            build-hxvgql7/faiss/python/CMakeFiles/swigfaiss.dir/swigfaissPYTHON_wrap.cxx
     267    /tmp/fs/spack-stage/spack-stage-faiss-1.8.0-hxvgql7ss2h2vftggt4n5iwxig3zmgz5/spack-build-hxvgql7/faiss/python/CMakeFiles/swigfaiss.dir/swigfaissPYTHON_wrap.cxx: In function 'PyObject* swig_ptr(PyObject*)':
  >> 268    /tmp/fs/spack-stage/spack-stage-faiss-1.8.0-hxvgql7ss2h2vftggt4n5iwxig3zmgz5/spack-build-hxvgql7/faiss/python/CMakeFiles/swigfaiss.dir/swigfaissPYTHON_wrap.cxx:5386:41: error: 'SWIGTYPE_p_unsigned_long_long' was not declared in this scope; did you mean 'SWIGTY
            PE_p_unsigned_long'?
     269     5386 |         return SWIG_NewPointerObj(data, SWIGTYPE_p_unsigned_long_long, 0);
     270          |                                         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
     271    /tmp/fs/spack-stage/spack-stage-faiss-1.8.0-hxvgql7ss2h2vftggt4n5iwxig3zmgz5/spack-build-hxvgql7/faiss/python/CMakeFiles/swigfaiss.dir/swigfaissPYTHON_wrap.cxx:1354:94: note: in definition of macro 'SWIG_NewPointerObj'
     272     1354 | #define SWIG_NewPointerObj(ptr, type, flags)            SWIG_Python_NewPointerObj(NULL, ptr, type, flags)
     273          |                                                                                              ^~~~
  >> 274    /tmp/fs/spack-stage/spack-stage-faiss-1.8.0-hxvgql7ss2h2vftggt4n5iwxig3zmgz5/spack-build-hxvgql7/faiss/python/CMakeFiles/swigfaiss.dir/swigfaissPYTHON_wrap.cxx:5393:41: error: 'SWIGTYPE_p_long_long' was not declared in this scope; did you mean 'SWIGTYPE_p_long
            '?
     275     5393 |         return SWIG_NewPointerObj(data, SWIGTYPE_p_long_long, 0);
     276          |                                         ^~~~~~~~~~~~~~~~~~~~
     277    /tmp/fs/spack-stage/spack-stage-faiss-1.8.0-hxvgql7ss2h2vftggt4n5iwxig3zmgz5/spack-build-hxvgql7/faiss/python/CMakeFiles/swigfaiss.dir/swigfaissPYTHON_wrap.cxx:1354:94: note: in definition of macro 'SWIG_NewPointerObj'
     278     1354 | #define SWIG_NewPointerObj(ptr, type, flags)            SWIG_Python_NewPointerObj(NULL, ptr, type, flags)
     279          |                                                                                              ^~~~
     280    /tmp/fs/spack-stage/spack-stage-faiss-1.8.0-hxvgql7ss2h2vftggt4n5iwxig3zmgz5/spack-build-hxvgql7/faiss/python/CMakeFiles/swigfaiss.dir/swigfaissPYTHON_wrap.cxx: In function 'int SWIG_mod_exec(PyObject*)':
     281    /tmp/fs/spack-stage/spack-stage-faiss-1.8.0-hxvgql7ss2h2vftggt4n5iwxig3zmgz5/spack-build-hxvgql7/faiss/python/CMakeFiles/swigfaiss.dir/swigfaissPYTHON_wrap.cxx:232826:3: warning: converting to non-pointer type 'int' from NULL [-Wconversion-null]
     282    232826 |   import_array();
     283           |   ^~~~~~~~~~~~
  >> 284    make[2]: *** [faiss/python/CMakeFiles/swigfaiss.dir/build.make:79: faiss/python/CMakeFiles/swigfaiss.dir/CMakeFiles/swigfaiss.dir/swigfaissPYTHON_wrap.cxx.o] Error 1
     285    make[2]: Leaving directory '/tmp/fs/spack-stage/spack-stage-faiss-1.8.0-hxvgql7ss2h2vftggt4n5iwxig3zmgz5/spack-build-hxvgql7'
  >> 286    make[1]: *** [CMakeFiles/Makefile2:272: faiss/python/CMakeFiles/swigfaiss.dir/all] Error 2
     287    make[1]: Leaving directory '/tmp/fs/spack-stage/spack-stage-faiss-1.8.0-hxvgql7ss2h2vftggt4n5iwxig3zmgz5/spack-build-hxvgql7'
  >> 288    make: *** [Makefile:139: all] Error 2

Information on your system

Additional information

spack-build-out.txt

This issue has been reported in this numpy issue and fixed in numpy 2.4.0 in this PR.

Specifically for faiss the issue has already been fixed in this PR which was released in v1.14.1.

However, since this is an issue that will likely affect multiple packages (and, for instance, continues to effect old versions of faiss), I was wondering if there would be a more general solution.
I tried adding

conflicts("py-numpy@2.4.0:", when="@4.4.0:")

to the swig recipe, but this did not prevent spack to concretize faiss ^py-numpy@2.2.6 ^swig@4.4.1.

General information

  • I have run spack debug report and reported the version of Spack/Python/Platform
  • I have run spack maintainers <name-of-the-package> and @mentioned any maintainers
  • I have uploaded the build log and environment files
  • I have searched the issues of this repo and believe this is not a duplicate

Metadata

Metadata

Assignees

No one assigned

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions