Note, this is me mostly thinking out loud
I'm not 100% sure about all of the build environments that we need to support...
I know I do some local development using python and dependencies in a virtual environment to trigger wheel builds.
I assume others require whatever is available from the distribution (python3-setuptools, python3-build, etc) to satisfy the dependencies for building tests, sdists, and wheels.
I'm guessing based on ye ol' #85 that we also have to support the scenarios where a source tree or sdist is fetched for Yocto to build the proper wheels.
Buildroot utilizes the sdist from pypi to perform builds https://github.com/buildroot/buildroot/blob/master/package/python-gpiod/python-gpiod.mk
Beyond the environments where builds need to work, there is the structure of the builds currently:
setup.py
- this is intended to build the production sdist and wheels
- For some environments, it's expected to link against system libraries (Buildroot, Yocto) and in others it builds the C library sources to link against
- It has a function decorator that wraps the sdist and bdist_ext commands to fetch the released C libraries
- the published wheels utilize
build to generate the sdist and wheels
build_tests.py
- This is intended to build the gpiod extension and the extensions used for the tests
- It has a duplicate definition of the gpiod extension from
setup.py so could drift out of sync
- it wraps and invokes
build_ext directly so has no shared infrastructure/Distribution info with setup.py
It makes sense to me to try to centralize all of the build information into a single spot so it's apparent what all scenarios need to be considered.
I realize this is basically asking to revert d588a6a but as I look into #162 the waters became murky with what all scenarios need to be handled
- do we need to fetch the tarball for
build_ext in setup.py if wheels are built from source distributions? All required files should presumably be in the sdist. Is there an expectation that build_ext gets called without an sdist and is instead called from a git checkout?
- We don't care about fetching sources for test builds
- We don't care about copying the license file for test builds
- Would we get any value out of having a custom build backend? (
WITH_TESTS=1 python3 -m build could build everything for testing?)
- Would editable installs suffice over the hand rolled in-place builds for tests?
For now, i will go with a more hacky approach for the LICENSE issue to solve the immediate problem and worry about reworking the build infrastructure when I understand the landscape a bit better.
Note, this is me mostly thinking out loud
I'm not 100% sure about all of the build environments that we need to support...
I know I do some local development using python and dependencies in a virtual environment to trigger wheel builds.
I assume others require whatever is available from the distribution (python3-setuptools, python3-build, etc) to satisfy the dependencies for building tests, sdists, and wheels.
I'm guessing based on ye ol' #85 that we also have to support the scenarios where a source tree or sdist is fetched for Yocto to build the proper wheels.
Buildroot utilizes the sdist from pypi to perform builds https://github.com/buildroot/buildroot/blob/master/package/python-gpiod/python-gpiod.mk
Beyond the environments where builds need to work, there is the structure of the builds currently:
setup.pybuildto generate the sdist and wheelsbuild_tests.pysetup.pyso could drift out of syncbuild_extdirectly so has no shared infrastructure/Distributioninfo withsetup.pyIt makes sense to me to try to centralize all of the build information into a single spot so it's apparent what all scenarios need to be considered.
I realize this is basically asking to revert d588a6a but as I look into #162 the waters became murky with what all scenarios need to be handled
build_extinsetup.pyif wheels are built from source distributions? All required files should presumably be in the sdist. Is there an expectation thatbuild_extgets called without an sdist and is instead called from a git checkout?WITH_TESTS=1 python3 -m buildcould build everything for testing?)For now, i will go with a more hacky approach for the LICENSE issue to solve the immediate problem and worry about reworking the build infrastructure when I understand the landscape a bit better.