Steps to reproduce the issue
faiss (@bhatiaharsh, @lpottier, @rblake-llnl) is one package that uses both py-numpy and swig that allows to reproduce the issue. I think this is mainly an issue to be solved in the swig (@sethrj) and potentially numpy (@adamjstewart, @rgommers) recipes, though.
$ spack spec -I faiss ^py-numpy@2.2.6 ^swig@4.4.1
- faiss@1.8.0~cuda~ipo+python~shared~tests build_system=cmake build_type=Release generator=make platform=linux os=ubuntu24.04 target=x86_64_v3 %c,cxx=gcc@14.2.0
[+] ^cmake@3.24.4~doc+ncurses+ownlibs~qtgui build_system=generic build_type=Release patches:=dbc3892,fdea723 platform=linux os=ubuntu24.04 target=x86_64_v3 %c,cxx=gcc@14.2.0
[e] ^curl@8.5.0~gssapi~ldap~libidn2~librtmp~libssh~libssh2+nghttp2 build_system=autotools libs:=shared,static tls:=openssl platform=linux os=ubuntu24.04 target=x86_64
[e] ^ncurses@6.3~symlinks+termlib abi=none build_system=autotools platform=linux os=ubuntu24.04 target=x86_64
[^] ^zlib-ng@2.3.3+compat+new_strategies+opt+pic+shared build_system=autotools platform=linux os=ubuntu24.04 target=x86_64_v3 %c,cxx=gcc@14.2.0
[^] ^compiler-wrapper@1.0 build_system=generic platform=linux os=ubuntu24.04 target=x86_64_v3
[e] ^gcc@14.2.0+binutils+bootstrap~graphite+libsanitizer~mold~nvptx~piclibs~profiled~strip build_system=autotools build_type=RelWithDebInfo languages:='c,c++,fortran' platform=linux os=ubuntu24.04 target=x86_64
[^] ^gcc-runtime@14.2.0 build_system=generic platform=linux os=ubuntu24.04 target=x86_64_v3
[e] ^glibc@2.39 build_system=autotools platform=linux os=ubuntu24.04 target=x86_64
[^] ^gmake@4.4.1~guile build_system=generic platform=linux os=ubuntu24.04 target=x86_64_v3 %c=gcc@14.2.0
[+] ^openblas@0.3.30~bignuma~consistent_fpcsr+dynamic_dispatch+fortran~ilp64~ipo+locking+pic+shared build_system=cmake build_type=Release generator=make symbol_suffix=none threads=none platform=linux os=ubuntu24.04 target=x86_64_v3 %c,cxx,fortran=gcc@14.2.0
[+] ^py-numpy@2.2.6 build_system=python_pip patches:=1c9cb08,873745d platform=linux os=ubuntu24.04 target=x86_64_v3 %c,cxx=gcc@14.2.0
[+] ^py-cython@3.2.4 build_system=python_pip platform=linux os=ubuntu24.04 target=x86_64_v3 %c,cxx=gcc@14.2.0
[+] ^py-meson-python@0.19.0 build_system=python_pip platform=linux os=ubuntu24.04 target=x86_64_v3 %c=gcc@14.2.0
[+] ^meson@1.10.1 build_system=python_pip patches:=0f0b1bd platform=linux os=ubuntu24.04 target=x86_64_v3
[+] ^ninja@1.13.2+re2c build_system=generic platform=linux os=ubuntu24.04 target=x86_64_v3 %c,cxx=gcc@14.2.0
[+] ^re2c@4.4 build_system=autotools platform=linux os=ubuntu24.04 target=x86_64_v3 %c,cxx=gcc@14.2.0
[+] ^py-packaging@26.0 build_system=python_pip platform=linux os=ubuntu24.04 target=x86_64_v3
[+] ^py-flit-core@3.12.0 build_system=python_pip platform=linux os=ubuntu24.04 target=x86_64_v3
[+] ^py-pyproject-metadata@0.11.0 build_system=python_pip platform=linux os=ubuntu24.04 target=x86_64_v3
[+] ^py-pip@25.1.1 build_system=generic platform=linux os=ubuntu24.04 target=x86_64_v3
[+] ^py-setuptools@63.4.3 build_system=generic platform=linux os=ubuntu24.04 target=x86_64_v3
[+] ^py-wheel@0.45.1 build_system=generic platform=linux os=ubuntu24.04 target=x86_64_v3
[+] ^python@3.11.15+bz2+crypt+ctypes+dbm~debug+libxml2+lzma~optimizations+pic+pyexpat+pythoncmd+readline+shared+sqlite3+ssl~static~tests~tkinter+uuid+zlib build_system=generic patches:=13fa8bf,b0615b2,ebdca64,f2fd060 platform=linux os=ubuntu24.04 target=x86_64_v3 %c,cxx=gcc@14.2.0
[e] ^bzip2@1.0.8~debug~pic+shared build_system=generic platform=linux os=ubuntu24.04 target=x86_64
[e] ^expat@2.6.1+libbsd build_system=autotools platform=linux os=ubuntu24.04 target=x86_64
[e] ^gdbm@1.23.1 build_system=autotools platform=linux os=ubuntu24.04 target=x86_64
[+] ^gettext@1.0+bzip2+curses+git~libunistring+libxml2+pic+shared+tar+xz build_system=autotools platform=linux os=ubuntu24.04 target=x86_64_v3 %c,cxx=gcc@14.2.0
[^] ^libiconv@1.18 build_system=autotools libs:=shared,static platform=linux os=ubuntu24.04 target=x86_64_v3 %c=gcc@14.2.0
[^] ^libxml2@2.15.1+pic~python+shared build_system=autotools platform=linux os=ubuntu24.04 target=x86_64_v3 %c=gcc@14.2.0
[+] ^tar@1.35 build_system=autotools zip=pigz platform=linux os=ubuntu24.04 target=x86_64_v3 %c=gcc@14.2.0
[+] ^pigz@2.8 build_system=makefile platform=linux os=ubuntu24.04 target=x86_64_v3 %c=gcc@14.2.0
[+] ^zstd@1.5.7+programs build_system=makefile compression:=none libs:=shared,static platform=linux os=ubuntu24.04 target=x86_64_v3 %c,cxx=gcc@14.2.0
[e] ^libffi@3.4.2 build_system=autotools platform=linux os=ubuntu24.04 target=x86_64
[+] ^libxcrypt@4.5.2~obsolete_api build_system=autotools platform=linux os=ubuntu24.04 target=x86_64_v3 %c=gcc@14.2.0
[e] ^perl@5.38.2+cpanm+opcode+open+shared+threads build_system=generic platform=linux os=ubuntu24.04 target=x86_64
[e] ^openssl@3.0.13~docs+shared build_system=generic certs=mozilla platform=linux os=ubuntu24.04 target=x86_64
[e] ^pkg-config@1.8.1+internal_glib build_system=autotools platform=linux os=ubuntu24.04 target=x86_64
[+] ^readline@8.3 build_system=autotools patches:=21f0a03 platform=linux os=ubuntu24.04 target=x86_64_v3 %c=gcc@14.2.0
[e] ^sqlite@3.37.2+column_metadata+dynamic_extensions~fts~functions+rtree build_system=autotools platform=linux os=ubuntu24.04 target=x86_64
[e] ^util-linux-uuid@2.37.2 build_system=autotools platform=linux os=ubuntu24.04 target=x86_64
[e] ^xz@5.2.4~pic build_system=autotools libs:=shared,static platform=linux os=ubuntu24.04 target=x86_64
[+] ^python-venv@1.0 build_system=generic platform=linux os=ubuntu24.04 target=x86_64_v3
[+] ^swig@4.4.1 build_system=autotools platform=linux os=ubuntu24.04 target=x86_64_v3 %c,cxx=gcc@14.2.0
[+] ^pcre2@10.44~jit+multibyte+pic build_system=autotools platform=linux os=ubuntu24.04 target=x86_64_v3 %c=gcc@14.2.0
Error message
Error message
==> Installing faiss-1.8.0-hxvgql7ss2h2vftggt4n5iwxig3zmgz5 [35/35]
==> Using cached archive: /opt/spack/var/spack/cache/_source-cache/archive/56/56ece0a419d62eaa11e39022fa27c8ed6d5a9b9eb7416cc5a0fdbeab07ec2f0c.tar.gz
==> No patches needed for faiss
==> faiss: Executing phase: 'cmake'
==> faiss: Executing phase: 'build'
==> Error: ProcessError: Command exited with status 2:
'/opt/mpi_installation/gmake/bin/make' '-j8'
5 errors found in build log:
262 make[2]: Leaving directory '/tmp/fs/spack-stage/spack-stage-faiss-1.8.0-hxvgql7ss2h2vftggt4n5iwxig3zmgz5/spack-build-hxvgql7'
263 /opt/mpi_installation/gmake/bin/make -f faiss/python/CMakeFiles/swigfaiss.dir/build.make faiss/python/CMakeFiles/swigfaiss.dir/build
264 make[2]: Entering directory '/tmp/fs/spack-stage/spack-stage-faiss-1.8.0-hxvgql7ss2h2vftggt4n5iwxig3zmgz5/spack-build-hxvgql7'
265 [100%] Building CXX object faiss/python/CMakeFiles/swigfaiss.dir/CMakeFiles/swigfaiss.dir/swigfaissPYTHON_wrap.cxx.o
266 cd /tmp/fs/spack-stage/spack-stage-faiss-1.8.0-hxvgql7ss2h2vftggt4n5iwxig3zmgz5/spack-build-hxvgql7/faiss/python && /opt/mpi_installation/compiler-wrapper/libexec/spack/gcc/g++ -Dswigfaiss_EXPORTS -I/tmp/fs/spack-stage/spack-stage-faiss-1.8.0-hxvgql7ss2h2vftgg
t4n5iwxig3zmgz5/spack-src/faiss/python/../.. -I/tmp/fs/spack-stage/spack-stage-faiss-1.8.0-hxvgql7ss2h2vftggt4n5iwxig3zmgz5/spack-src -isystem /opt/fs/spack/install/linux-x86_64_v3/python-3.11.15-nh4prrfgrcpu46qwofhmgc64ibselgti/include/python3.11 -isystem /op
t/fs/spack/install/linux-x86_64_v3/py-numpy-2.2.6-yyh5lw62a6v4ad2fjxn5cyfg2iawl46z/lib/python3.11/site-packages/numpy/_core/include -O3 -DNDEBUG -fPIC -fopenmp -std=gnu++17 -MD -MT faiss/python/CMakeFiles/swigfaiss.dir/CMakeFiles/swigfaiss.dir/swigfaissPYTHON_
wrap.cxx.o -MF CMakeFiles/swigfaiss.dir/CMakeFiles/swigfaiss.dir/swigfaissPYTHON_wrap.cxx.o.d -o CMakeFiles/swigfaiss.dir/CMakeFiles/swigfaiss.dir/swigfaissPYTHON_wrap.cxx.o -c /tmp/fs/spack-stage/spack-stage-faiss-1.8.0-hxvgql7ss2h2vftggt4n5iwxig3zmgz5/spack-
build-hxvgql7/faiss/python/CMakeFiles/swigfaiss.dir/swigfaissPYTHON_wrap.cxx
267 /tmp/fs/spack-stage/spack-stage-faiss-1.8.0-hxvgql7ss2h2vftggt4n5iwxig3zmgz5/spack-build-hxvgql7/faiss/python/CMakeFiles/swigfaiss.dir/swigfaissPYTHON_wrap.cxx: In function 'PyObject* swig_ptr(PyObject*)':
>> 268 /tmp/fs/spack-stage/spack-stage-faiss-1.8.0-hxvgql7ss2h2vftggt4n5iwxig3zmgz5/spack-build-hxvgql7/faiss/python/CMakeFiles/swigfaiss.dir/swigfaissPYTHON_wrap.cxx:5386:41: error: 'SWIGTYPE_p_unsigned_long_long' was not declared in this scope; did you mean 'SWIGTY
PE_p_unsigned_long'?
269 5386 | return SWIG_NewPointerObj(data, SWIGTYPE_p_unsigned_long_long, 0);
270 | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
271 /tmp/fs/spack-stage/spack-stage-faiss-1.8.0-hxvgql7ss2h2vftggt4n5iwxig3zmgz5/spack-build-hxvgql7/faiss/python/CMakeFiles/swigfaiss.dir/swigfaissPYTHON_wrap.cxx:1354:94: note: in definition of macro 'SWIG_NewPointerObj'
272 1354 | #define SWIG_NewPointerObj(ptr, type, flags) SWIG_Python_NewPointerObj(NULL, ptr, type, flags)
273 | ^~~~
>> 274 /tmp/fs/spack-stage/spack-stage-faiss-1.8.0-hxvgql7ss2h2vftggt4n5iwxig3zmgz5/spack-build-hxvgql7/faiss/python/CMakeFiles/swigfaiss.dir/swigfaissPYTHON_wrap.cxx:5393:41: error: 'SWIGTYPE_p_long_long' was not declared in this scope; did you mean 'SWIGTYPE_p_long
'?
275 5393 | return SWIG_NewPointerObj(data, SWIGTYPE_p_long_long, 0);
276 | ^~~~~~~~~~~~~~~~~~~~
277 /tmp/fs/spack-stage/spack-stage-faiss-1.8.0-hxvgql7ss2h2vftggt4n5iwxig3zmgz5/spack-build-hxvgql7/faiss/python/CMakeFiles/swigfaiss.dir/swigfaissPYTHON_wrap.cxx:1354:94: note: in definition of macro 'SWIG_NewPointerObj'
278 1354 | #define SWIG_NewPointerObj(ptr, type, flags) SWIG_Python_NewPointerObj(NULL, ptr, type, flags)
279 | ^~~~
280 /tmp/fs/spack-stage/spack-stage-faiss-1.8.0-hxvgql7ss2h2vftggt4n5iwxig3zmgz5/spack-build-hxvgql7/faiss/python/CMakeFiles/swigfaiss.dir/swigfaissPYTHON_wrap.cxx: In function 'int SWIG_mod_exec(PyObject*)':
281 /tmp/fs/spack-stage/spack-stage-faiss-1.8.0-hxvgql7ss2h2vftggt4n5iwxig3zmgz5/spack-build-hxvgql7/faiss/python/CMakeFiles/swigfaiss.dir/swigfaissPYTHON_wrap.cxx:232826:3: warning: converting to non-pointer type 'int' from NULL [-Wconversion-null]
282 232826 | import_array();
283 | ^~~~~~~~~~~~
>> 284 make[2]: *** [faiss/python/CMakeFiles/swigfaiss.dir/build.make:79: faiss/python/CMakeFiles/swigfaiss.dir/CMakeFiles/swigfaiss.dir/swigfaissPYTHON_wrap.cxx.o] Error 1
285 make[2]: Leaving directory '/tmp/fs/spack-stage/spack-stage-faiss-1.8.0-hxvgql7ss2h2vftggt4n5iwxig3zmgz5/spack-build-hxvgql7'
>> 286 make[1]: *** [CMakeFiles/Makefile2:272: faiss/python/CMakeFiles/swigfaiss.dir/all] Error 2
287 make[1]: Leaving directory '/tmp/fs/spack-stage/spack-stage-faiss-1.8.0-hxvgql7ss2h2vftggt4n5iwxig3zmgz5/spack-build-hxvgql7'
>> 288 make: *** [Makefile:139: all] Error 2
Information on your system
Additional information
spack-build-out.txt
This issue has been reported in this numpy issue and fixed in numpy 2.4.0 in this PR.
Specifically for faiss the issue has already been fixed in this PR which was released in v1.14.1.
However, since this is an issue that will likely affect multiple packages (and, for instance, continues to effect old versions of faiss), I was wondering if there would be a more general solution.
I tried adding
conflicts("py-numpy@2.4.0:", when="@4.4.0:")
to the swig recipe, but this did not prevent spack to concretize faiss ^py-numpy@2.2.6 ^swig@4.4.1.
General information
Steps to reproduce the issue
faiss(@bhatiaharsh, @lpottier, @rblake-llnl) is one package that uses both py-numpy and swig that allows to reproduce the issue. I think this is mainly an issue to be solved in the swig (@sethrj) and potentially numpy (@adamjstewart, @rgommers) recipes, though.Error message
Error message
==> Installing faiss-1.8.0-hxvgql7ss2h2vftggt4n5iwxig3zmgz5 [35/35] ==> Using cached archive: /opt/spack/var/spack/cache/_source-cache/archive/56/56ece0a419d62eaa11e39022fa27c8ed6d5a9b9eb7416cc5a0fdbeab07ec2f0c.tar.gz ==> No patches needed for faiss ==> faiss: Executing phase: 'cmake' ==> faiss: Executing phase: 'build' ==> Error: ProcessError: Command exited with status 2: '/opt/mpi_installation/gmake/bin/make' '-j8' 5 errors found in build log: 262 make[2]: Leaving directory '/tmp/fs/spack-stage/spack-stage-faiss-1.8.0-hxvgql7ss2h2vftggt4n5iwxig3zmgz5/spack-build-hxvgql7' 263 /opt/mpi_installation/gmake/bin/make -f faiss/python/CMakeFiles/swigfaiss.dir/build.make faiss/python/CMakeFiles/swigfaiss.dir/build 264 make[2]: Entering directory '/tmp/fs/spack-stage/spack-stage-faiss-1.8.0-hxvgql7ss2h2vftggt4n5iwxig3zmgz5/spack-build-hxvgql7' 265 [100%] Building CXX object faiss/python/CMakeFiles/swigfaiss.dir/CMakeFiles/swigfaiss.dir/swigfaissPYTHON_wrap.cxx.o 266 cd /tmp/fs/spack-stage/spack-stage-faiss-1.8.0-hxvgql7ss2h2vftggt4n5iwxig3zmgz5/spack-build-hxvgql7/faiss/python && /opt/mpi_installation/compiler-wrapper/libexec/spack/gcc/g++ -Dswigfaiss_EXPORTS -I/tmp/fs/spack-stage/spack-stage-faiss-1.8.0-hxvgql7ss2h2vftgg t4n5iwxig3zmgz5/spack-src/faiss/python/../.. -I/tmp/fs/spack-stage/spack-stage-faiss-1.8.0-hxvgql7ss2h2vftggt4n5iwxig3zmgz5/spack-src -isystem /opt/fs/spack/install/linux-x86_64_v3/python-3.11.15-nh4prrfgrcpu46qwofhmgc64ibselgti/include/python3.11 -isystem /op t/fs/spack/install/linux-x86_64_v3/py-numpy-2.2.6-yyh5lw62a6v4ad2fjxn5cyfg2iawl46z/lib/python3.11/site-packages/numpy/_core/include -O3 -DNDEBUG -fPIC -fopenmp -std=gnu++17 -MD -MT faiss/python/CMakeFiles/swigfaiss.dir/CMakeFiles/swigfaiss.dir/swigfaissPYTHON_ wrap.cxx.o -MF CMakeFiles/swigfaiss.dir/CMakeFiles/swigfaiss.dir/swigfaissPYTHON_wrap.cxx.o.d -o CMakeFiles/swigfaiss.dir/CMakeFiles/swigfaiss.dir/swigfaissPYTHON_wrap.cxx.o -c /tmp/fs/spack-stage/spack-stage-faiss-1.8.0-hxvgql7ss2h2vftggt4n5iwxig3zmgz5/spack- build-hxvgql7/faiss/python/CMakeFiles/swigfaiss.dir/swigfaissPYTHON_wrap.cxx 267 /tmp/fs/spack-stage/spack-stage-faiss-1.8.0-hxvgql7ss2h2vftggt4n5iwxig3zmgz5/spack-build-hxvgql7/faiss/python/CMakeFiles/swigfaiss.dir/swigfaissPYTHON_wrap.cxx: In function 'PyObject* swig_ptr(PyObject*)': >> 268 /tmp/fs/spack-stage/spack-stage-faiss-1.8.0-hxvgql7ss2h2vftggt4n5iwxig3zmgz5/spack-build-hxvgql7/faiss/python/CMakeFiles/swigfaiss.dir/swigfaissPYTHON_wrap.cxx:5386:41: error: 'SWIGTYPE_p_unsigned_long_long' was not declared in this scope; did you mean 'SWIGTY PE_p_unsigned_long'? 269 5386 | return SWIG_NewPointerObj(data, SWIGTYPE_p_unsigned_long_long, 0); 270 | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 271 /tmp/fs/spack-stage/spack-stage-faiss-1.8.0-hxvgql7ss2h2vftggt4n5iwxig3zmgz5/spack-build-hxvgql7/faiss/python/CMakeFiles/swigfaiss.dir/swigfaissPYTHON_wrap.cxx:1354:94: note: in definition of macro 'SWIG_NewPointerObj' 272 1354 | #define SWIG_NewPointerObj(ptr, type, flags) SWIG_Python_NewPointerObj(NULL, ptr, type, flags) 273 | ^~~~ >> 274 /tmp/fs/spack-stage/spack-stage-faiss-1.8.0-hxvgql7ss2h2vftggt4n5iwxig3zmgz5/spack-build-hxvgql7/faiss/python/CMakeFiles/swigfaiss.dir/swigfaissPYTHON_wrap.cxx:5393:41: error: 'SWIGTYPE_p_long_long' was not declared in this scope; did you mean 'SWIGTYPE_p_long '? 275 5393 | return SWIG_NewPointerObj(data, SWIGTYPE_p_long_long, 0); 276 | ^~~~~~~~~~~~~~~~~~~~ 277 /tmp/fs/spack-stage/spack-stage-faiss-1.8.0-hxvgql7ss2h2vftggt4n5iwxig3zmgz5/spack-build-hxvgql7/faiss/python/CMakeFiles/swigfaiss.dir/swigfaissPYTHON_wrap.cxx:1354:94: note: in definition of macro 'SWIG_NewPointerObj' 278 1354 | #define SWIG_NewPointerObj(ptr, type, flags) SWIG_Python_NewPointerObj(NULL, ptr, type, flags) 279 | ^~~~ 280 /tmp/fs/spack-stage/spack-stage-faiss-1.8.0-hxvgql7ss2h2vftggt4n5iwxig3zmgz5/spack-build-hxvgql7/faiss/python/CMakeFiles/swigfaiss.dir/swigfaissPYTHON_wrap.cxx: In function 'int SWIG_mod_exec(PyObject*)': 281 /tmp/fs/spack-stage/spack-stage-faiss-1.8.0-hxvgql7ss2h2vftggt4n5iwxig3zmgz5/spack-build-hxvgql7/faiss/python/CMakeFiles/swigfaiss.dir/swigfaissPYTHON_wrap.cxx:232826:3: warning: converting to non-pointer type 'int' from NULL [-Wconversion-null] 282 232826 | import_array(); 283 | ^~~~~~~~~~~~ >> 284 make[2]: *** [faiss/python/CMakeFiles/swigfaiss.dir/build.make:79: faiss/python/CMakeFiles/swigfaiss.dir/CMakeFiles/swigfaiss.dir/swigfaissPYTHON_wrap.cxx.o] Error 1 285 make[2]: Leaving directory '/tmp/fs/spack-stage/spack-stage-faiss-1.8.0-hxvgql7ss2h2vftggt4n5iwxig3zmgz5/spack-build-hxvgql7' >> 286 make[1]: *** [CMakeFiles/Makefile2:272: faiss/python/CMakeFiles/swigfaiss.dir/all] Error 2 287 make[1]: Leaving directory '/tmp/fs/spack-stage/spack-stage-faiss-1.8.0-hxvgql7ss2h2vftggt4n5iwxig3zmgz5/spack-build-hxvgql7' >> 288 make: *** [Makefile:139: all] Error 2Information on your system
Additional information
spack-build-out.txt
This issue has been reported in this numpy issue and fixed in numpy 2.4.0 in this PR.
Specifically for faiss the issue has already been fixed in this PR which was released in v1.14.1.
However, since this is an issue that will likely affect multiple packages (and, for instance, continues to effect old versions of
faiss), I was wondering if there would be a more general solution.I tried adding
to the swig recipe, but this did not prevent spack to concretize
faiss ^py-numpy@2.2.6 ^swig@4.4.1.General information
spack debug reportand reported the version of Spack/Python/Platformspack maintainers <name-of-the-package>and @mentioned any maintainers