Skip to content

[nanvix] E: Bake ninja and Cython into toolchain image#683

Open
esaurez wants to merge 1 commit into
nanvix/v3.12.3from
feat/toolchain-python-bake-ninja-cython
Open

[nanvix] E: Bake ninja and Cython into toolchain image#683
esaurez wants to merge 1 commit into
nanvix/v3.12.3from
feat/toolchain-python-bake-ninja-cython

Conversation

@esaurez
Copy link
Copy Markdown

@esaurez esaurez commented May 30, 2026

[nanvix] E: Bake ninja and Cython into toolchain image

Adds ninja-build, python3-pip, and Cython<3 to the toolchain-python docker image so that meson- and Cython-based Python extension cross-builds (numpy, scipy, ...) work out-of-the-box, without an apt/pip preamble on every docker run invocation.

What changed in .nanvix/docker/Dockerfile:

  • Added python3-pip and ninja-build to the apt install list.
  • Added pip3 install --break-system-packages 'Cython<3' (pinned for numpy 1.26.x compatibility; lift the pin when bumping numpy).
  • Added rm -rf /usr/include/python3.12 after the install. The python3-pip / ninja-build apt packages transitively pull in libpython3.12-dev, whose headers under /usr/include/python3.12 would otherwise be picked up by meson's regen step ahead of the Nanvix cross sysroot headers and silently corrupt the cross-build.
  • Comment block explaining the rationale for each addition and the /usr/include/python3.12 purge.

Why this matters:

The numpy .so cross-build (validated end-to-end on 2026-05-27 with the STB_WEAK loader fix landed) requires two tools that were not present in the image as shipped:

  • ninja — meson's default backend; missing it makes every meson-based Python extension build fail immediately.
  • Cython — used by numpy/_build_utils/tempita.py to template .pyx.in files; without it the numpy.random codegen step fails.

Before this change, the workaround was to inject:

apt-get update -qq
apt-get install -qq -y --no-install-recommends ninja-build python3-pip
pip3 install --quiet --break-system-packages 'Cython<3'
rm -rf /usr/include/python3.12

into every numpy build invocation, which (a) was fragile, (b) required the docker container to have outbound network access on every build (non-hermetic), and (c) re-paid the apt install cost in CI every run.

Validated locally:

  • docker build -f .nanvix/docker/Dockerfile -t toolchain-python:pr13 .nanvix/docker/ succeeds.
  • docker run --rm <image> bash -c 'ninja --version'1.11.1.
  • docker run --rm <image> bash -c 'python3 -c "import Cython; print(Cython.__version__)"'0.29.37.
  • docker run --rm <image> bash -c 'ls /usr/include/python3.12' → exits non-zero / "No such file or directory".

Co-authored-by: Copilot 223556219+Copilot@users.noreply.github.com

Adds `ninja-build`, `python3-pip`, and `Cython<3` to the
toolchain-python docker image so that meson- and Cython-based Python
extension cross-builds (numpy, scipy, ...) work out-of-the-box,
without an apt/pip preamble on every `docker run` invocation.

What changed in `.nanvix/docker/Dockerfile`:

  - Added `python3-pip` and `ninja-build` to the apt install list.
  - Added `pip3 install --break-system-packages 'Cython<3'` (pinned for
    numpy 1.26.x compatibility; lift the pin when bumping numpy).
  - Added `rm -rf /usr/include/python3.12` after the install. The
    `python3-pip`/`ninja-build` apt packages transitively pull in
    `libpython3.12-dev`, whose headers under `/usr/include/python3.12`
    would otherwise be picked up by meson's regen step ahead of the
    Nanvix cross sysroot headers and silently corrupt the cross-build.
  - Comment block explaining the rationale for each addition and the
    `/usr/include/python3.12` purge.

Why this matters:

The numpy `.so` cross-build (validated end-to-end on 2026-05-27 with
the STB_WEAK loader fix landed) requires two tools that were not
present in the image as shipped:

  - `ninja` — meson's default backend; missing it makes every
    meson-based Python extension build fail immediately.
  - `Cython` — used by `numpy/_build_utils/tempita.py` to template
    `.pyx.in` files; without it the `numpy.random` codegen step fails.

Before this change, the workaround was to inject:

```bash
apt-get update -qq
apt-get install -qq -y --no-install-recommends ninja-build python3-pip
pip3 install --quiet --break-system-packages 'Cython<3'
rm -rf /usr/include/python3.12
```

into every numpy build invocation, which (a) was fragile, (b) required
the docker container to have outbound network access on every build
(non-hermetic), and (c) re-paid the apt install cost in CI every run.

Validated locally:

  - `docker build -f .nanvix/docker/Dockerfile -t toolchain-python:pr13
    .nanvix/docker/` succeeds.
  - `docker run --rm <image> bash -c 'ninja --version'` → `1.11.1`.
  - `docker run --rm <image> bash -c 'python3 -c "import Cython;
    print(Cython.__version__)"'` → `0.29.37`.
  - `docker run --rm <image> bash -c 'ls /usr/include/python3.12'` →
    exits non-zero / "No such file or directory".

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings May 30, 2026 03:12
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Updates the Nanvix toolchain-python Docker image to include additional build-time dependencies needed for cross-compiling third-party Python extension modules (notably meson/ninja and Cython-based projects like numpy/scipy), aiming to remove per-build apt/pip bootstrapping.

Changes:

  • Install python3-pip and ninja-build in the toolchain image.
  • Install Cython<3 via pip for numpy 1.26.x compatibility.
  • Purge host Python headers under /usr/include/python3.12 to avoid meson accidentally consuming host headers instead of the Nanvix sysroot.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread .nanvix/docker/Dockerfile
python3-pip \
ninja-build \
&& pip3 install --break-system-packages --no-cache-dir 'Cython<3' \
&& rm -rf /usr/include/python3.12 \
Comment thread .nanvix/docker/Dockerfile
Comment on lines +19 to +23
# We deliberately purge `/usr/include/python3.12` after the install. The
# `python3-pip` / `ninja-build` apt packages transitively pull in
# `libpython3.12-dev`, whose headers under `/usr/include/python3.12` would
# otherwise be picked up by meson's regen step ahead of the Nanvix cross
# sysroot headers and silently corrupt the cross-build.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants