Build Windows ARM wheels

[mkoeppe]

This can be done on GH Actions using the new windows-11-arm runners, see e.g.
https://github.com/passagemath/passagemath/blob/main/.github/workflows/dist-wheels-windows.yml#L182

[oscarbenjamin]

I think when I added manylinux ARM wheels it was not yet possible to use Windows ARM runners. It might be as easy as gh-259 now but I can imagine that there might be more problems in the Windows case. For a start I don't know how well msys handles Windows on ARM.

[skirpichev]

I tried to use this recipe in gmpy2/gmpy2#582, but without luck: the GMP build fails (with msystem=clangarm64 and mingw-w64-clang-aarch64-gcc).

Though, maybe ucrt64 build will work for the python-flint, as it doesn't use MVSC.

[mkoeppe]

@skirpichev Based on your experiments in gmpy2/gmpy2#582, with some modifications, I was able to build GMP/MPFR/MPC in passagemath/passagemath#1194

[oscarbenjamin]

It might be as easy as gh-259 now

I suppose I should just have a go at this but I'm interested to know where you two are at. Do you have mingw working smoothly to build GMP/MPFR/FLINT for Windows on ARM now?

Pending discussion in gh-349 and gh-338 we might be getting to the point of using stable ABI which would massively reduce the number of wheels and also increase the supported range of Python versions. At that point Windows on ARM could be a single wheel and it would definitely be worth adding if it works.

[skirpichev]

Do you have mingw working smoothly to build GMP/MPFR/FLINT for Windows on ARM now?

No. After gmpy2/gmpy2#582 we use MSVC to build the gmpy2 extension on Windows.

the point of using stable ABI which would massively reduce the number of wheels

There are issues in gmpy2 and python-gmp, proposing using the Limited API, but this is not easy. First, we have to wait till last supported version will be v3.13 or v3.14. Second, IIUIC, using the Limited API requires using heap types. Not sure if this is relevant for the python-flint, but my experiments for the python-gmp (diofant/python-gmp#195) shows noticeable speed regression vs static types.

[mkoeppe]

After gmpy2/gmpy2#582 we use MSVC to build the gmpy2 extension on Windows.

Well, the extension is built using MSVC, but the libraries sure are built using mingw32 (MSYS2).

[mkoeppe]

Do you have mingw working smoothly to build GMP/MPFR/FLINT for Windows on ARM now?

In passagemath, I have been building Windows ARM wheels using GMP/MPFR/MPC since https://github.com/passagemath/passagemath/releases/tag/passagemath-10.6.40; no FLINT yet because of an unfortunate dependency on NTL in the Sage codebase.

[mkoeppe]

we might be getting to the point of using stable ABI

In passagemath, I ran into some obstructions from basic Sage library code that is not compatible with the stable API. passagemath/passagemath#572 (comment)
I do use abi3 builds for wheels that I ship that do not make use these parts of the Sage library, since https://github.com/passagemath/passagemath/releases/tag/passagemath-10.6.29

[oscarbenjamin]

The libraries sure are built using mingw32 (MSYS2).

This is the main thing I am asking. I assume that if mingw64 can build GMP, MPFR and FLINT then we would be able to use it to build the extension modules in python-flint. The thing that I was previously unsure about was whether mingw64 worked at all for Windows on ARM but it sounds like it does.

I spent some time thinking about whether or not it was possible to use MSVC to build GMP et al and the issue is that we want --enable-fat to get the runtime selection of GMP's architecture specific assembly code but that assembly code and the mechanism to select it at runtime doesn't work with MSVC. Me and @BrianGladman looked into this and we also spoke to @zooba about it. The conclusion was that @BrianGladman is probably the only person who knows enough about this to make that somehow work but neither me nor him knows enough about MSVC to get it to work without some help from someone else.

If we accept that we are going to use mingw64 to build GMP then from my perspective we may as well just use mingw64 for everything because then it is a single toolchain and also it means we are using gcc/clang for every OS so we can have a fully unixy build system and toolchain while still supporting Windows. I assume that it is still possible to build GMP etc and python-flint with MSVC if you just don't have --enable-fat (although I don't actually know how to do that myself). I imagine though that no one does that and that us here in CI and conda are the only people who ever actually build python-flint for Windows and everyone else just uses the packages we build.

The only question then is whether it would be better to use mingw64's UCRT mode rather than msvcrt. I tried that a long time ago (gh-41) and it didn't work at the time but could be worth a revisit.

[BrianGladman]

The problem I face isn't MSVC itself but rather that of designing and implementing the data structures needed to produce a FAT build. This is because my own builds are individually optimised with separate DLLs for each processor architecture rather than one DLL with dynamic processor selection .

[mkoeppe]

I assume that if mingw64 can build GMP, MPFR and FLINT then we would be able to use it to build the extension modules in python-flint. The thing that I was previously unsure about was whether mingw64 worked at all for Windows on ARM but it sounds like it does.

It does. The MSYS2 folks build for this architecture, see in particular
https://packages.msys2.org/packages/mingw-w64-clang-aarch64-flint

[mkoeppe]

If we accept that we are going to use mingw64 to build GMP then from my perspective we may as well just use mingw64 for everything because then it is a single toolchain and also it means we are using gcc/clang for every OS so we can have a fully unixy build system and toolchain while still supporting Windows. [...]
The only question then is whether it would be better to use mingw64's UCRT mode rather than msvcrt. I tried that a long time ago (gh-41) and it didn't work at the time but could be worth a revisit.

If you get this to work, it would be quite interesting for me because I am actually running into some issues from a mingw64 / MSVC mismatch when C++ gets involved, for example passagemath/passagemath#1175 (comment)

[oscarbenjamin]

In gh-360 I got as far as getting GMP to configure and start compiling but it eventually failed with:

libtool: compile:  cc -DHAVE_CONFIG_H -I. -I.. -D__GMP_WITHIN_GMP -I.. -DOPERATION_mu_divappr_q -O2 -pedantic -march=armv8-a -c mu_divappr_q.c  -DDLL_EXPORT -DPIC -o .libs/mu_divappr_q.o
libtool: compile:  cc -DHAVE_CONFIG_H -I. -I.. -D__GMP_WITHIN_GMP -I.. -DOPERATION_mu_div_q -O2 -pedantic -march=armv8-a -c mu_div_q.c  -DDLL_EXPORT -DPIC -o .libs/mu_div_q.o
libtool: compile:  ../mpn/m4-ccas --m4=m4 cc -c -DHAVE_CONFIG_H -I. -I.. -D__GMP_WITHIN_GMP -I.. -DOPERATION_bdiv_q_1 -O2 -pedantic -march=armv8-a bdiv_q_1.asm  -DDLL_EXPORT -DPIC -o .libs/bdiv_q_1.o
/bin/sh ../libtool  --tag=CC   --mode=compile cc -DHAVE_CONFIG_H -I. -I..  -D__GMP_WITHIN_GMP -I.. -DOPERATION_`echo dcpi1_bdiv_q | sed 's/_$//'`   -O2 -pedantic -march=armv8-a -c -o dcpi1_bdiv_q.lo dcpi1_bdiv_q.c
/bin/sh ../libtool  --tag=CC   --mode=compile cc -DHAVE_CONFIG_H -I. -I..  -D__GMP_WITHIN_GMP -I.. -DOPERATION_`echo dcpi1_bdiv_qr | sed 's/_$//'`   -O2 -pedantic -march=armv8-a -c -o dcpi1_bdiv_qr.lo dcpi1_bdiv_qr.c
m4  -DHAVE_CONFIG_H -D__GMP_WITHIN_GMP -DOPERATION_bdiv_q_1 -DDLL_EXPORT -DPIC bdiv_q_1.asm >tmp-bdiv_q_1.s
 cc -c -DHAVE_CONFIG_H -I. -I.. -D__GMP_WITHIN_GMP -I.. -DOPERATION_bdiv_q_1 -O2 -pedantic -march=armv8-a tmp-bdiv_q_1.s -DDLL_EXPORT -DPIC -o .libs/bdiv_q_1.o
libtool: compile:  cc -DHAVE_CONFIG_H -I. -I.. -D__GMP_WITHIN_GMP -I.. -DOPERATION_sbpi1_bdiv_q -O2 -pedantic -march=armv8-a -c sbpi1_bdiv_q.c  -DDLL_EXPORT -DPIC -o .libs/sbpi1_bdiv_q.o
libtool: compile:  cc -DHAVE_CONFIG_H -I. -I.. -D__GMP_WITHIN_GMP -I.. -DOPERATION_sbpi1_bdiv_qr -O2 -pedantic -march=armv8-a -c sbpi1_bdiv_qr.c  -DDLL_EXPORT -DPIC -o .libs/sbpi1_bdiv_qr.o
cc: warning: argument unused during compilation: '-D HAVE_CONFIG_H' [-Wunused-command-line-argument]
cc: warning: argument unused during compilation: '-D __GMP_WITHIN_GMP' [-Wunused-command-line-argument]
cc: warning: argument unused during compilation: '-D OPERATION_bdiv_q_1' [-Wunused-command-line-argument]
cc: warning: argument unused during compilation: '-O2' [-Wunused-command-line-argument]
cc: warning: argument unused during compilation: '-pedantic' [-Wunused-command-line-argument]
cc: warning: argument unused during compilation: '-D DLL_EXPORT' [-Wunused-command-line-argument]
cc: warning: argument unused during compilation: '-D PIC' [-Wunused-command-line-argument]
tmp-bdiv_q_1.s:75:2: error: relocation variant :got: unsupported on COFF targets
        adrp    x7, :got:__gmp_binvert_limb_table
        ^
tmp-bdiv_q_1.s:77:2: error: relocation variant :got_lo12: unsupported on COFF targets
        ldr     x7, [x7, #:got_lo12:__gmp_binvert_limb_table]
        ^
make[2]: *** [Makefile:769: bdiv_q_1.lo] Error 1

I'm assuming that __gmp_binvert_limb_table is exactly the mechanism used for --enable-fat.

@skirpichev I see that in https://github.com/gmpy2/gmpy2/pull/582/files you have used --disable-assembly instead of --enable-fat. Is this basically the road block that you hit?