Skip to content

Install pkg-config in builder images#9297

Open
ShahanaFarooqui wants to merge 3 commits into
ElementsProject:masterfrom
ShahanaFarooqui:repro-add-pkg-config
Open

Install pkg-config in builder images#9297
ShahanaFarooqui wants to merge 3 commits into
ElementsProject:masterfrom
ShahanaFarooqui:repro-add-pkg-config

Conversation

@ShahanaFarooqui

@ShahanaFarooqui ShahanaFarooqui commented Jul 9, 2026

Copy link
Copy Markdown
Collaborator

All three reproducible builds started failing after our last commit to pin coincurve builds from source. This is an environment gap in the repro images, not a problem with the coincurve pin itself.

coincurve==20.0.0 shipped prebuilt wheels, so nothing was ever compiled during uv sync. The git pin has no wheels, so the repro build now compiles coincurve from source, and its CMake build hard-requires pkg-config. Regular CI already installs pkg-config via .github/scripts/setup.sh, which is why PR CI passed; the repro Dockerfiles keep a separate apt list and were missing it.

Failed Action: https://github.com/ElementsProject/lightning/actions/runs/28993800082

Changelog-None.

@ShahanaFarooqui ShahanaFarooqui marked this pull request as ready for review July 9, 2026 05:05
@ShahanaFarooqui ShahanaFarooqui requested a review from cdecker as a code owner July 9, 2026 05:05
@madelinevibes madelinevibes added this to the v26.09 milestone Jul 9, 2026
daywalker90
daywalker90 previously approved these changes Jul 9, 2026
@daywalker90

Copy link
Copy Markdown
Collaborator

The installation instructions also need updating. I took the liberty to update arch linux instructions as a whole, since they were horribly outdated. pkg-config command is provided by different package names, depending on the distro.

@madelinevibes madelinevibes self-requested a review July 9, 2026 13:42
madelinevibes
madelinevibes previously approved these changes Jul 9, 2026

@madelinevibes madelinevibes left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! ebb8532 & 985d668

@daywalker90 daywalker90 dismissed stale reviews from madelinevibes and themself via 473265c July 9, 2026 14:19
@daywalker90 daywalker90 force-pushed the repro-add-pkg-config branch from 985d668 to 473265c Compare July 9, 2026 14:19
@daywalker90

Copy link
Copy Markdown
Collaborator

There were two problems with the docker build step. One was that it required the target arch's python headers and the other was that the vendored cmake is dynamically linked against libatomic1 which is usually included when installing build-essentials but we don't do that in that bookworm-slim image. So i kept it slim and only pulled in libatomic1.

cdecker
cdecker previously approved these changes Jul 9, 2026
ShahanaFarooqui and others added 3 commits July 9, 2026 13:02
…images

Pinning coincurve to a git revision means it builds from source rather than
installing a prebuilt wheel, which needs pkg-config for its CMake build.
And with no system cmake in the image, scikit-build-core fetches the cmake
wheel from PyPI; the arm stages install python3-dev:${target_arch_dpkg},
which swaps in a target-arch python (under qemu), so uv resolves target-arch
wheels - and the armv7l/aarch64 cmake wheels need target-arch libatomic.so.1,
while only the amd64 libatomic1 was installed.

Regular CI installs both via .github/scripts/setup.sh, but the repro
Dockerfiles maintain their own package list.
Also updated Arch linux instructions to use necessary dependencies, uv and rust

Removed python plugin instructions, they can not be generalized and this repo no
longer has any python plugins that need extra dependencies in production.
Python plugins from the plugins repo have their own documentation.

Changelog-None
… target

Pass the computed VERSION as a build-arg, mirroring the docker-release workflow. Without it the container's VERSION is set-but-empty, `VERSION ?=` in the Makefile won't overwrite it, the git-describe fallback never runs, and `make install-program` fails with "git is required for generating version information".

Also add --no-push, so a release captain can do a full test build without publishing elementsproject/lightningd:$VERSION and :latest (the target always runs `docker buildx build --push`, since multi-arch images cannot be --load'ed into the classic docker store). The build only populates the builder's cache; a subsequent run without the flag pushes from cache quickly.

Co-Authored-By: Claude <claude@users.noreply.github.com>
@daywalker90

Copy link
Copy Markdown
Collaborator
 => [linux/amd64 lightningd 3/4] COPY --from=builder       /tmp/lightning_install/               /usr/local/                                                                                                                                        0.4s
 => [linux/amd64 lightningd 4/4] COPY tools/docker-entrypoint.sh    /entrypoint.sh                                                                                                                                                                  0.1s
 => ERROR [linux/amd64 builder 13/13] RUN find /tmp/lightning_install -type f -executable -exec     file {} + |     awk -F: '/ELF/ {print $1}' |     xargs -r arm-linux-gnueabihf-strip --strip-unneeded                                            0.7s
------
 > [linux/amd64 builder 13/13] RUN find /tmp/lightning_install -type f -executable -exec     file {} + |     awk -F: '/ELF/ {print $1}' |     xargs -r arm-linux-gnueabihf-strip --strip-unneeded:
0.383 arm-linux-gnueabihf-strip: /tmp/lightning_install/libexec/c-lightning/plugins/cln-lsps-client: file format not recognized
0.410 arm-linux-gnueabihf-strip: /tmp/lightning_install/libexec/c-lightning/plugins/clnrest: file format not recognized
0.454 arm-linux-gnueabihf-strip: /tmp/lightning_install/libexec/c-lightning/plugins/cln-bip353: file format not recognized
0.467 arm-linux-gnueabihf-strip: /tmp/lightning_install/libexec/c-lightning/plugins/cln-lsps-service: file format not recognized
0.474 arm-linux-gnueabihf-strip: /tmp/lightning_install/libexec/c-lightning/plugins/cln-currencyrate: file format not recognized
0.567 arm-linux-gnueabihf-strip: /tmp/lightning_install/libexec/c-lightning/plugins/wss-proxy: file format not recognized
------
WARNING: No output specified with docker-container driver. Build result will only remain in the build cache. To push result image into registry use --push or to load image into docker use --load
WARNING: local cache import at /tmp/docker-cache skipped due to err: could not read /tmp/docker-cache/index.json: open /tmp/docker-cache/index.json: no such file or directory
ERROR: failed to build: failed to solve: process "/bin/bash -euo pipefail -c find /tmp/lightning_install -type f -executable -exec     file {} + |     awk -F: '/ELF/ {print $1}' |     xargs -r ${STRIP} --strip-unneeded" did not complete successfully: exit code: 123

The dependency issue is fixed but my run ends like this, not sure if that is ok.

What do we think about this approach?
#9303

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants