cannot compile quippy with `make quippy

[bernstei]

Latest public branch a4523cb, make quippy fails with

error: Command "gcc -pthread -B /home/Software/python/miniconda3/compiler_compat -DNDEBUG -fwrapv -O2 -Wall -fPIC -O2 -isystem /home/Software/python/miniconda3/include -fPIC -O2 -isystem /home/Software/python/miniconda3/include -fPIC -DNPY_DISABLE_OPTIMIZATION=1 -I./src.linux-x86_64-3.11/./src.linux-x86_64-3.11 -I/home/Software/python/miniconda3/extra/lib/python3.11/site-packages/numpy/core/include -I/home/Software/python/miniconda3/include/python3.11 -c ./src.linux-x86_64-3.11/_quippymodule.c -o ./src.linux-x86_64-3.11/_quippymodule.o -MMD -MF ./src.linux-x86_64-3.11/_quippymodule.o.d" failed with exit status 1
make[1]: *** [Makefile:122: quippy/_quippy.cpython-311-x86_64-linux-gnu.so] Error 1
make[1]: Leaving directory '/home/cluster/bernstei/src/work/QUIP/source/QUIP_github_clean/build/linux_x86_64_gfortran'
make: *** [Makefile:230: quippy] Error 2

going back up past a lot of warnings, there are two underlying errors that I can find, wrapping System.f95 s2a and pad

./src.linux-x86_64-3.11/_quippymodule.c: In function ‘f2py_rout__quippy_f90wrap_system_module__s2a’:
./src.linux-x86_64-3.11/_quippymodule.c:46097:9: error: ‘ret_a_capi’ undeclared (first use in this function); did you mean ‘ret_a_Rank’?
46097 |     if (ret_a_capi != Py_None) slen(ret_a) = f2py_itemsize(ret_a);
      |         ^~~~~~~~~~
      |         ret_a_Rank
./src.linux-x86_64-3.11/_quippymodule.c:46097:9: note: each undeclared identifier is reported only once for each function it appears in
./src.linux-x86_64-3.11/_quippymodule.c: In function ‘f2py_rout__quippy_f90wrap_system_module__pad’:
./src.linux-x86_64-3.11/_quippymodule.c:46339:9: error: ‘ret_a_capi’ undeclared (first use in this function); did you mean ‘ret_a_Rank’?
46339 |     if (ret_a_capi != Py_None) slen(ret_a) = f2py_itemsize(ret_a);
      |         ^~~~~~~~~~
      |         ret_a_Rank

Indeed, the code in _quippymodule.c uses ret_a_capi without declaring it.

f90wrap --version reports 0.2.13.

[Yike-Ye]

Hi, have you figured out how to fixed this? I'm also facing this issue and my python version is 3.11.

[jameskermode]

I’ve got f90wrap working again in the mesonify branch see open PR #694

[Yike-Ye]

Dear Prof., thanks for your quick reply. I'm wondering if these changes also affect the building process of QUIP itself?

[jameskermode]

Yes, changed completely from Make to meson. This is not yet finalised but there is a new BUILD.md document in the quippy/ directory of that branch which also describes how to build QUIP.

[Yike-Ye]

Dear Prof., I didn't find an updated BUILD.md in the quippy/ directory, and I am recompiling QUIP since you mentioned most of the source code has been updated. Till now, I found three issues which I didn't encounter in the last version.

1. When using make config to set up the environment, if I type n to say no, it will be assigned to variables such as ASAP_LIBDIR (with HAVE_ASAP=1). As a result, -n will be shown as flags during compilation.
2. It seems that you want to remove .f95 files, but some remain in src/GAP directory, such as make_permutations_noncommercial_v2.f95
3. The third issue might be specific to my setup, as I’m modifying the arch file and using the AMD AOCC compiler. It seems that the definition of F77 (F77=mpif77) in Makefile.arch was not successfully applied, so the build process follows the rule in Makefile.rules (.F.o: ${F77} ${F77FLAGS} -c $< -o $@) and defaults to f77 (from gfortran) for compilation.

Just removed the second one, as I realized I gited the wrong version.

[Yike-Ye]

I have rebuilt QUIP and quippy with meson. While QUIP works perfectly fine, the test of quippy failed. It seems that some temporary files are required for self._rebuild() when importing quippy. However, they were deleted after pip install -e .

$ python
Python 3.12.2 | packaged by conda-forge | (main, Feb 16 2024, 20:50:58) [GCC 12.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import quippy
Traceback (most recent call last):
  File "/home/yike/anaconda3/lib/python3.12/site-packages/_quippy_ase_editable_loader.py", line 345, in _rebuild
    subprocess.run(self._build_cmd, cwd=self._build_path, env=env, stdout=subprocess.DEVNULL, check=True)
  File "/home/yike/anaconda3/lib/python3.12/subprocess.py", line 571, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/home/yike/anaconda3/bin/ninja']' returned non-zero exit status 127.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/yike/anaconda3/lib/python3.12/site-packages/_quippy_ase_editable_loader.py", line 311, in find_spec
    tree = self._rebuild()
           ^^^^^^^^^^^^^^^
  File "/home/yike/anaconda3/lib/python3.12/site-packages/_quippy_ase_editable_loader.py", line 347, in _rebuild
    raise ImportError(f're-building the {self._name} meson-python editable wheel package failed') from exc
ImportError: re-building the quippy-ase meson-python editable wheel package failed

For example, build.ninja was generated during pip install -e ., and it requires meson in the temporary directory /tmp/pip-build-env-bs4aj7fi/overlay/bin/meson.

$ grep -R "tmp/pip-build-env" -n /home/yike/Software/QUIP_pr694/quippy/build/cp312
匹配到二进制文件 /home/yike/Software/QUIP_pr694/quippy/build/cp312/meson-private/coredata.dat
匹配到二进制文件 /home/yike/Software/QUIP_pr694/quippy/build/cp312/meson-private/install.dat
匹配到二进制文件 /home/yike/Software/QUIP_pr694/quippy/build/cp312/meson-private/build.dat
匹配到二进制文件 /home/yike/Software/QUIP_pr694/quippy/build/cp312/meson-private/coredata.dat.prev
/home/yike/Software/QUIP_pr694/quippy/build/cp312/meson-logs/meson-log.txt:68:Program f90wrap found: YES (/tmp/pip-build-env-bs4aj7fi/overlay/bin/f90wrap)
/home/yike/Software/QUIP_pr694/quippy/build/cp312/meson-logs/meson-log.txt:70:Program f2py-f90wrap found: YES (/tmp/pip-build-env-bs4aj7fi/overlay/bin/f2py-f90wrap)
/home/yike/Software/QUIP_pr694/quippy/build/cp312/meson-logs/meson-log.txt:186:/tmp/pip-build-env-bs4aj7fi/overlay/bin/f2py-f90wrap --build-dir "/home/yike/Software/QUIP_pr694/quippy/build/cp312/f2py_build" -c -m _quippy "$@" \
/home/yike/Software/QUIP_pr694/quippy/build/cp312/meson-logs/meson-log.txt:215:/tmp/pip-build-env-bs4aj7fi/overlay/bin/f2py-f90wrap --build-dir "/home/yike/Software/QUIP_pr694/quippy/build/cp312/f2py_build" -c -m _quippy "$@" \
/home/yike/Software/QUIP_pr694/quippy/build/cp312/meson-info/intro-targets.json:414:          "/tmp/pip-build-env-bs4aj7fi/overlay/bin/f90wrap",
/home/yike/Software/QUIP_pr694/quippy/build/cp312/f2py_wrapper.sh:3:/tmp/pip-build-env-bs4aj7fi/overlay/bin/f2py-f90wrap --build-dir "/home/yike/Software/QUIP_pr694/quippy/build/cp312/f2py_build" -c -m _quippy "$@" \
/home/yike/Software/QUIP_pr694/quippy/build/cp312/build.ninja:21: command = /tmp/pip-build-env-bs4aj7fi/overlay/bin/meson --internal regenerate /home/yike/Software/QUIP_pr694/quippy .
/home/yike/Software/QUIP_pr694/quippy/build/cp312/build.ninja:32: COMMAND = /tmp/pip-build-env-bs4aj7fi/overlay/bin/meson --internal exe --capture Atoms_types.fpp -- /usr/bin/gfortran -E -cpp -DHAVE_GAP -DHAVE_TB -DGFORTRAN -DGETARG_F2003 -DGETENV_F2003 -DFORTRAN_UNDERSCORE '-DQUIP_ARCH="darwin_arm64_gfortran"' -DSIZEOF_FORTRAN_T=2 ../../../src/libAtoms/Atoms_types.F90
/home/yike/Software/QUIP_pr694/quippy/build/cp312/build.ninja:36: COMMAND = /tmp/pip-build-env-bs4aj7fi/overlay/bin/meson --internal exe --capture Atoms.fpp -- /usr/bin/gfortran -E -cpp -DHAVE_GAP -DHAVE_TB -DGFORTRAN -DGETARG_F2003 -DGETENV_F2003 -DFORTRAN_UNDERSCORE '-DQUIP_ARCH="darwin_arm64_gfortran"' -DSIZEOF_FORTRAN_T=2 ../../../src/libAtoms/Atoms.F90
/home/yike/Software/QUIP_pr694/quippy/build/cp312/build.ninja:40: COMMAND = /tmp/pip-build-env-bs4aj7fi/overlay/bin/meson --internal exe --capture System.fpp -- /usr/bin/gfortran -E -cpp -DHAVE_GAP -DHAVE_TB -DGFORTRAN -DGETARG_F2003 -DGETENV_F2003 -DFORTRAN_UNDERSCORE '-DQUIP_ARCH="darwin_arm64_gfortran"' -DSIZEOF_FORTRAN_T=2 ../../../src/libAtoms/System.F90
/home/yike/Software/QUIP_pr694/quippy/build/cp312/build.ninja:44: COMMAND = /tmp/pip-build-env-bs4aj7fi/overlay/bin/meson --internal exe --capture Dictionary.fpp -- /usr/bin/gfortran -E -cpp -DHAVE_GAP -DHAVE_TB -DGFORTRAN -DGETARG_F2003 -DGETENV_F2003 -DFORTRAN_UNDERSCORE '-DQUIP_ARCH="darwin_arm64_gfortran"' -DSIZEOF_FORTRAN_T=2 ../../../src/libAtoms/Dictionary.F90
/home/yike/Software/QUIP_pr694/quippy/build/cp312/build.ninja:48: COMMAND = /tmp/pip-build-env-bs4aj7fi/overlay/bin/meson --internal exe --capture DynamicalSystem.fpp -- /usr/bin/gfortran -E -cpp -DHAVE_GAP -DHAVE_TB -DGFORTRAN -DGETARG_F2003 -DGETENV_F2003 -DFORTRAN_UNDERSCORE '-DQUIP_ARCH="darwin_arm64_gfortran"' -DSIZEOF_FORTRAN_T=2 ../../../src/libAtoms/DynamicalSystem.F90
/home/yike/Software/QUIP_pr694/quippy/build/cp312/build.ninja:52: COMMAND = /tmp/pip-build-env-bs4aj7fi/overlay/bin/meson --internal exe --capture nye_tensor.fpp -- /usr/bin/gfortran -E -cpp -DHAVE_GAP -DHAVE_TB -DGFORTRAN -DGETARG_F2003 -DGETENV_F2003 -DFORTRAN_UNDERSCORE '-DQUIP_ARCH="darwin_arm64_gfortran"' -DSIZEOF_FORTRAN_T=2 ../../../src/libAtoms/nye_tensor.F90
/home/yike/Software/QUIP_pr694/quippy/build/cp312/build.ninja:56: COMMAND = /tmp/pip-build-env-bs4aj7fi/overlay/bin/meson --internal exe --capture Potential.fpp -- /usr/bin/gfortran -E -cpp -DHAVE_GAP -DHAVE_TB -DGFORTRAN -DGETARG_F2003 -DGETENV_F2003 -DFORTRAN_UNDERSCORE '-DQUIP_ARCH="darwin_arm64_gfortran"' -DSIZEOF_FORTRAN_T=2 ../../../src/Potentials/Potential.F90
/home/yike/Software/QUIP_pr694/quippy/build/cp312/build.ninja:60: COMMAND = /tmp/pip-build-env-bs4aj7fi/overlay/bin/meson --internal exe --capture Potential_simple.fpp -- /usr/bin/gfortran -E -cpp -DHAVE_GAP -DHAVE_TB -DGFORTRAN -DGETARG_F2003 -DGETENV_F2003 -DFORTRAN_UNDERSCORE '-DQUIP_ARCH="darwin_arm64_gfortran"' -DSIZEOF_FORTRAN_T=2 ../../../src/Potentials/Potential_simple.F90
/home/yike/Software/QUIP_pr694/quippy/build/cp312/build.ninja:64: COMMAND = /tmp/pip-build-env-bs4aj7fi/overlay/bin/meson --internal exe --capture TB.fpp -- /usr/bin/gfortran -E -cpp -DHAVE_GAP -DHAVE_TB -DGFORTRAN -DGETARG_F2003 -DGETENV_F2003 -DFORTRAN_UNDERSCORE '-DQUIP_ARCH="darwin_arm64_gfortran"' -DSIZEOF_FORTRAN_T=2 ../../../src/Potentials/TB.F90
/home/yike/Software/QUIP_pr694/quippy/build/cp312/build.ninja:68: COMMAND = /tmp/pip-build-env-bs4aj7fi/overlay/bin/meson --internal exe --capture descriptors.fpp -- /usr/bin/gfortran -E -cpp -DHAVE_GAP -DHAVE_TB -DGFORTRAN -DGETARG_F2003 -DGETENV_F2003 -DFORTRAN_UNDERSCORE '-DQUIP_ARCH="darwin_arm64_gfortran"' -DSIZEOF_FORTRAN_T=2 ../../../src/GAP/descriptors.F90
/home/yike/Software/QUIP_pr694/quippy/build/cp312/build.ninja:71:build f90wrap_Atoms_types.f90 f90wrap_Atoms.f90 f90wrap_System.f90 f90wrap_Dictionary.f90 f90wrap_DynamicalSystem.f90 f90wrap_nye_tensor.f90 f90wrap_Potential.f90 f90wrap_Potential_simple.f90 f90wrap_TB.f90 f90wrap_descriptors.f90: CUSTOM_COMMAND Atoms_types.fpp Atoms.fpp System.fpp Dictionary.fpp DynamicalSystem.fpp nye_tensor.fpp Potential.fpp Potential_simple.fpp TB.fpp descriptors.fpp | ../../KIND_MAP ../../STRING_LENGTHS ../../doc_plugin.py ../../init.py /tmp/pip-build-env-bs4aj7fi/overlay/bin/f90wrap
/home/yike/Software/QUIP_pr694/quippy/build/cp312/build.ninja:72: COMMAND = /tmp/pip-build-env-bs4aj7fi/overlay/bin/f90wrap -m quippy --f90-mod-name quippy._quippy Atoms_types.fpp Atoms.fpp System.fpp Dictionary.fpp DynamicalSystem.fpp nye_tensor.fpp Potential.fpp Potential_simple.fpp TB.fpp descriptors.fpp -P -a c_error_abort --init-file /home/yike/Software/QUIP_pr694/quippy/init.py --move-methods --shorten-routine-names -k /home/yike/Software/QUIP_pr694/quippy/KIND_MAP -s /home/yike/Software/QUIP_pr694/quippy/STRING_LENGTHS -S 10240 --skip atoms_shallowcopy atoms_initialise_ptr atoms_finalise_multi potential_initialise_inoutput cplx_2d cplx_2d_array1_finalise potential_local_e_mix_initialise potential_local_e_mix_finalise --skip-types spherical_harmonics_type potential_local_e_mix --force-public set_cutoff --documentation-plugin /home/yike/Software/QUIP_pr694/quippy/doc_plugin.py --py-max-line-length 120 --f90-max-line-length 120
/home/yike/Software/QUIP_pr694/quippy/build/cp312/build.ninja:94: COMMAND = /tmp/pip-build-env-bs4aj7fi/overlay/bin/meson test --no-rebuild --print-errorlogs
/home/yike/Software/QUIP_pr694/quippy/build/cp312/build.ninja:101: COMMAND = /tmp/pip-build-env-bs4aj7fi/overlay/bin/meson test --benchmark --logbase benchmarklog --num-processes=1 --no-rebuild
/home/yike/Software/QUIP_pr694/quippy/build/cp312/build.ninja:111: COMMAND = /tmp/pip-build-env-bs4aj7fi/overlay/bin/meson install --no-rebuild
/home/yike/Software/QUIP_pr694/quippy/build/cp312/build.ninja:118: COMMAND = /tmp/pip-build-env-bs4aj7fi/overlay/bin/meson dist
/home/yike/Software/QUIP_pr694/quippy/build/cp312/build.ninja:126: COMMAND = /tmp/pip-build-env-bs4aj7fi/overlay/bin/meson --internal scanbuild /home/yike/Software/QUIP_pr694/quippy /home/yike/Software/QUIP_pr694/quippy/build/cp312 subprojects /tmp/pip-build-env-bs4aj7fi/overlay/bin/meson setup
/home/yike/Software/QUIP_pr694/quippy/build/cp312/build.ninja:132: COMMAND = /tmp/pip-build-env-bs4aj7fi/overlay/bin/meson --internal tags ctags /home/yike/Software/QUIP_pr694/quippy
/home/yike/Software/QUIP_pr694/quippy/build/cp312/build.ninja:138: COMMAND = /tmp/pip-build-env-bs4aj7fi/overlay/bin/meson --internal uninstall
/home/yike/Software/QUIP_pr694/quippy/build/cp312/build.ninja:152: COMMAND = /tmp/pip-build-env-bs4aj7fi/overlay/bin/meson --internal cleantrees /home/yike/Software/QUIP_pr694/quippy/build/cp312/meson-private/cleantrees.dat
匹配到二进制文件 /home/yike/Software/QUIP_pr694/quippy/build/cp312/_quippy.cpython-312-x86_64-linux-gnu.so

[jameskermode]

Editable installs don’t work well with meson. Use plain pip install .. I will try to update the installation instructions soon, but this is work in progress. In the meantime the CI is building correctly on Ubuntu with Python 3.12. There are some test failures so I don’t recommend using this for production calculations at this time — this is also why the PR has not yet been merged.