[nanvix] E: Bake ninja and Cython into toolchain image#683
Open
esaurez wants to merge 1 commit into
Open
Conversation
Adds `ninja-build`, `python3-pip`, and `Cython<3` to the
toolchain-python docker image so that meson- and Cython-based Python
extension cross-builds (numpy, scipy, ...) work out-of-the-box,
without an apt/pip preamble on every `docker run` invocation.
What changed in `.nanvix/docker/Dockerfile`:
- Added `python3-pip` and `ninja-build` to the apt install list.
- Added `pip3 install --break-system-packages 'Cython<3'` (pinned for
numpy 1.26.x compatibility; lift the pin when bumping numpy).
- Added `rm -rf /usr/include/python3.12` after the install. The
`python3-pip`/`ninja-build` apt packages transitively pull in
`libpython3.12-dev`, whose headers under `/usr/include/python3.12`
would otherwise be picked up by meson's regen step ahead of the
Nanvix cross sysroot headers and silently corrupt the cross-build.
- Comment block explaining the rationale for each addition and the
`/usr/include/python3.12` purge.
Why this matters:
The numpy `.so` cross-build (validated end-to-end on 2026-05-27 with
the STB_WEAK loader fix landed) requires two tools that were not
present in the image as shipped:
- `ninja` — meson's default backend; missing it makes every
meson-based Python extension build fail immediately.
- `Cython` — used by `numpy/_build_utils/tempita.py` to template
`.pyx.in` files; without it the `numpy.random` codegen step fails.
Before this change, the workaround was to inject:
```bash
apt-get update -qq
apt-get install -qq -y --no-install-recommends ninja-build python3-pip
pip3 install --quiet --break-system-packages 'Cython<3'
rm -rf /usr/include/python3.12
```
into every numpy build invocation, which (a) was fragile, (b) required
the docker container to have outbound network access on every build
(non-hermetic), and (c) re-paid the apt install cost in CI every run.
Validated locally:
- `docker build -f .nanvix/docker/Dockerfile -t toolchain-python:pr13
.nanvix/docker/` succeeds.
- `docker run --rm <image> bash -c 'ninja --version'` → `1.11.1`.
- `docker run --rm <image> bash -c 'python3 -c "import Cython;
print(Cython.__version__)"'` → `0.29.37`.
- `docker run --rm <image> bash -c 'ls /usr/include/python3.12'` →
exits non-zero / "No such file or directory".
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
Updates the Nanvix toolchain-python Docker image to include additional build-time dependencies needed for cross-compiling third-party Python extension modules (notably meson/ninja and Cython-based projects like numpy/scipy), aiming to remove per-build apt/pip bootstrapping.
Changes:
- Install
python3-pipandninja-buildin the toolchain image. - Install
Cython<3via pip for numpy 1.26.x compatibility. - Purge host Python headers under
/usr/include/python3.12to avoid meson accidentally consuming host headers instead of the Nanvix sysroot.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| python3-pip \ | ||
| ninja-build \ | ||
| && pip3 install --break-system-packages --no-cache-dir 'Cython<3' \ | ||
| && rm -rf /usr/include/python3.12 \ |
Comment on lines
+19
to
+23
| # We deliberately purge `/usr/include/python3.12` after the install. The | ||
| # `python3-pip` / `ninja-build` apt packages transitively pull in | ||
| # `libpython3.12-dev`, whose headers under `/usr/include/python3.12` would | ||
| # otherwise be picked up by meson's regen step ahead of the Nanvix cross | ||
| # sysroot headers and silently corrupt the cross-build. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
[nanvix] E: Bake ninja and Cython into toolchain image
Adds
ninja-build,python3-pip, andCython<3to the toolchain-python docker image so that meson- and Cython-based Python extension cross-builds (numpy, scipy, ...) work out-of-the-box, without an apt/pip preamble on everydocker runinvocation.What changed in
.nanvix/docker/Dockerfile:python3-pipandninja-buildto the apt install list.pip3 install --break-system-packages 'Cython<3'(pinned for numpy 1.26.x compatibility; lift the pin when bumping numpy).rm -rf /usr/include/python3.12after the install. Thepython3-pip/ninja-buildapt packages transitively pull inlibpython3.12-dev, whose headers under/usr/include/python3.12would otherwise be picked up by meson's regen step ahead of the Nanvix cross sysroot headers and silently corrupt the cross-build./usr/include/python3.12purge.Why this matters:
The numpy
.socross-build (validated end-to-end on 2026-05-27 with the STB_WEAK loader fix landed) requires two tools that were not present in the image as shipped:ninja— meson's default backend; missing it makes every meson-based Python extension build fail immediately.Cython— used bynumpy/_build_utils/tempita.pyto template.pyx.infiles; without it thenumpy.randomcodegen step fails.Before this change, the workaround was to inject:
apt-get update -qq apt-get install -qq -y --no-install-recommends ninja-build python3-pip pip3 install --quiet --break-system-packages 'Cython<3' rm -rf /usr/include/python3.12into every numpy build invocation, which (a) was fragile, (b) required the docker container to have outbound network access on every build (non-hermetic), and (c) re-paid the apt install cost in CI every run.
Validated locally:
docker build -f .nanvix/docker/Dockerfile -t toolchain-python:pr13 .nanvix/docker/succeeds.docker run --rm <image> bash -c 'ninja --version'→1.11.1.docker run --rm <image> bash -c 'python3 -c "import Cython; print(Cython.__version__)"'→0.29.37.docker run --rm <image> bash -c 'ls /usr/include/python3.12'→ exits non-zero / "No such file or directory".Co-authored-by: Copilot 223556219+Copilot@users.noreply.github.com