[ci]: Add buildenv_setup tool + build-env/ (CI-unification PoC)#1220
Draft
theasianpianist wants to merge 22 commits into
Draft
[ci]: Add buildenv_setup tool + build-env/ (CI-unification PoC)#1220theasianpianist wants to merge 22 commits into
theasianpianist wants to merge 22 commits into
Conversation
Collaborator
|
/azp run |
|
Azure Pipelines successfully started running 1 pipeline(s). |
Contributor
Author
|
/azp run |
|
Azure Pipelines successfully started running 1 pipeline(s). |
Collaborator
|
/azp run |
|
Azure Pipelines successfully started running 1 pipeline(s). |
![github-code-quality[bot]](https://avatars.githubusercontent.com/u/9919?s=60&v=4){kind=small}
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
Collaborator
|
/azp run |
|
Azure Pipelines successfully started running 1 pipeline(s). |
Collaborator
|
/azp run |
|
Azure Pipelines successfully started running 1 pipeline(s). |
Collaborator
|
/azp run |
|
Azure Pipelines successfully started running 1 pipeline(s). |
Collaborator
|
/azp run |
|
Azure Pipelines successfully started running 1 pipeline(s). |
Collaborator
|
/azp run |
|
Azure Pipelines successfully started running 1 pipeline(s). |
Collaborator
|
/azp run |
|
Azure Pipelines successfully started running 1 pipeline(s). |
Collaborator
|
/azp run |
|
Azure Pipelines successfully started running 1 pipeline(s). |
Collaborator
|
/azp run |
|
Azure Pipelines successfully started running 1 pipeline(s). |
Collaborator
|
/azp run |
|
Azure Pipelines successfully started running 1 pipeline(s). |
Collaborator
|
/azp run |
|
Azure Pipelines successfully started running 1 pipeline(s). |
libnl carries sonic-specific patches, so it must come from the common-libs
build artifact (SONiC-patched libnl-*_3.x-0.2+b1sonicN), never from apt. Add the
full libnl family (libnl-{3,genl,route,nf,cli}-*, runtime + -dev + -dbgsym) to the
common-libs upstream entry -- common-lib publishes these for all arches
(amd64/armhf/arm64) -- and drop the libnl-*-dev apt packages from base.yaml.
libnl-cli is included so downstream consumers that need it (e.g. sonic-sairedis on
arm) inherit it via the cascade rather than apt-installing it. Because
upstream-artifacts.yaml also cascades, downstream repos keep receiving libnl,
now sourced from common-libs.
Verified in a sonic-slave-bookworm container: all libnl packages install at
version 3.7.0-0.2+b1sonic1 (the sonic-patched build, not Debian's plain 3.7.0-0.2).
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Lawrence Lee <lawlee@microsoft.com>
Replace the inline apt block + DownloadPipelineArtifact steps (libyang deb +
sonic-yang wheel) in the amd64/ubuntu-22.04 job with a single buildenv_setup
invocation (--host-os ubuntu-22.04), plus an inline step for the special-case
CLI installs that don't fit the packages schema (gtest compile-from-source,
bazelisk binary). Bazel build/test + compile stay inline.
Config changes to support the jammy host:
- base.yaml: split swig -> swig (containers) / swig4.0 (jammy) via host_os.
- tooling.yaml: add the jammy build toolchain the sonic-slave containers bake in
(make/libtool/autoconf/cmake/pkg-config/boost/uuid/zmq/gtest/gmock/python3-pip/
python3-cffi/...), all gated host_os: ubuntu-22.04 and non-cascading.
- upstream-artifacts.yaml: drop python3-libyang (bookworm deb pins python3>=3.11,
won't install on jammy python 3.10; swss-common uses the pip libyang==3.3.0
binding). Add apt_fix_broken to common-libs so dpkg -i pulls apt deps the
containers bake in but the vmImage lacks (e.g. libyang-dev -> libpcre2-dev).
This also removes the inline job's runBranch: refs/heads/$(BUILD_BRANCH) download
(which failed on feature branches with 'No builds currently exist') since
buildenv_setup resolves common-libs via its own branch pin. Verified end-to-end in
an ubuntu:22.04 container: setup RC=0, swig4.0 + jammy toolchain installed, SONiC
libnl + libyang3 DEBs install (libpcre2-dev auto-pulled), import libyang works,
redis configured.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Lawrence Lee <lawlee@microsoft.com>
The v1 fix (service redis-server stop/start around cargo) did not work: in the
sonic-slave container 'service redis-server stop' reports success but does not
actually kill redis, so the unixsocket redis stayed up and the Rust
logger_init_without_redis test (which asserts logger init fails when redis is
unreachable) failed again (build 1158200).
Root cause: the configure-redis-for-tests post_install ran at build-env SETUP
time, bringing up the unixsocket redis before cargo. Master only configured the
unixsocket redis AFTER cargo. Restore that ordering:
- base.yaml: scope the configure-redis-for-tests post_install to [test] only, so
buildenv_setup --scope build no longer brings up the unixsocket redis at setup.
redis-server is still installed and runs on TCP only (Debian default leaves
unixsocket commented), so logger_init_without_redis sees redis unreachable via
the SONiC database_config unixsocket and passes. The VS test host still gets the
config via the base.yaml cascade (--scope test).
- build-template.yml: drop the stop/start hack; after cargo, run the same shared
./build-env/configure-redis-for-tests.sh inline (before the C++/pytest tests,
which need the unixsocket redis).
Verified in a sonic-slave-bookworm container: after --scope build there is no
unixsocket/redis.sock (logger_init_without_redis would pass); after the inline
script the unixsocket redis is up for the C++/pytest tests.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Lawrence Lee <lawlee@microsoft.com>
The repo's .artifactignore was '**/* + !*.deb', so the published pipeline artifact contained only the .deb packages. Verified against run 1158257: the sonic-swss-common-bookworm artifact had 7 debs and NO build-env/. buildenv_setup's cascade requires a consumer (sonic-sairedis, sonic-swss) to read the upstream's build-env/ (upstream-artifacts.yaml + packages/*.yaml) out of the downloaded artifact. With build-env/ absent, the cascade silently no-ops and the consumer can't inherit this repo's deps (libnl/libyang3 from common-libs, etc.). Un-ignore build-env/ (both the directory entry and its contents -- the leading **/* ignores the directory too, so !build-env/** alone is insufficient; verified with git check-ignore). The tool itself (ci/) is NOT published -- consumers clone this repo to obtain it. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Signed-off-by: Lawrence Lee <lawlee@microsoft.com>
Extract the inline amd64/ubuntu-22.04 job from azure-pipelines.yml into .azure-pipelines/build-ubuntu-template.yml (parameterized by debian_version), so the top-level azure-pipelines.yml is uniformly '- template:' invocations, matching the container build jobs. No behavior change -- same buildenv_setup invocation, gtest/bazelisk installs, compile, bazel build/test, and artifact publish. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Signed-off-by: Lawrence Lee <lawlee@microsoft.com>
The lawlee/ci-unify *branch* build (manually queued, not a PR) sets BUILD_BRANCH=lawlee/ci-unify. The downstream BuildSairedis/BuildSwss/BuildDocker/ Test stages still use raw $(BUILD_BRANCH) for the sonic-slave container tag, the sonic-sairedis/sonic-swss git checkout, and the common_libs / sonic-dash-api / sonic-buildimage(.vs) / sonic-buildimage-ubuntu22.04 DownloadPipelineArtifact runBranch. None of those resources exist on lawlee/ci-unify, so the branch build fails. Pin them all to master to get a green branch run. Every pin is marked 'TEMPORARY (PoC)' with the $(BUILD_BRANCH) form to restore. Combined with the pre-existing pins (build-template.yml container tag, upstream-artifacts.yaml branch: master), 'grep -rn "TEMPORARY (PoC)" .azure-pipelines/ build-env/' lists all 15 code pins to revert before merge. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Signed-off-by: Lawrence Lee <lawlee@microsoft.com>
…stages The Build stage now builds libswsscommon against the SONiC-patched libnl (3.7.0-0.2+b1sonicN) downloaded from common-lib (per the libnl-from-common-libs policy). Downstream stages that install the swss-common .deb must therefore also provide that libnl, not apt's stock 3.7.0-0.2+b1 (which is older and fails the dpkg dependency). - build-sairedis-template.yml: drop libnl-*-dev from apt; download the full SONiC libnl family (3/genl/route/nf/cli, runtime + dev) from common-lib instead. - test-docker-sonic-vs-template.yml: download + apt-install the SONiC libnl runtime debs on the jammy vstest host before installing the swss-common deb. This is a real fix (not a temporary branch pin) and stays after merge. BuildSwss already pulls libnl from common-lib, and BuildDocker installs debs inside the SONiC base image (which already has the patched libnl), so neither needs changes. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Signed-off-by: Lawrence Lee <lawlee@microsoft.com>
Completes the libnl-from-common-libs alignment for the downstream stages (matches swss-common's base.yaml family: libnl-3/genl/route/nf/cli). BuildSwss already pulled libnl-3/genl/route/nf from common-lib and none from apt; libnl-cli was the only member of the SONiC libnl family still missing. BuildSairedis was already switched to common-lib libnl in the prior commit. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Signed-off-by: Lawrence Lee <lawlee@microsoft.com>
Demonstrate build-template reuse + dependency inheritance across repos: swss-common's BuildSairedis stage now references sonic-sairedis's own .azure-pipelines/build-template.yml (repo: sonic-sairedis) instead of a swss-common-local copy. sairedis is built by its own template, resolving swss-common (and, via the cascade, common-libs) through buildenv_setup. - Pin the sonic-sairedis resource ref to lawlee/ci-unify (TEMPORARY PoC) so the reused template + build-env come from the PoC branch. - Delete the now-unused swss-common copy .azure-pipelines/build-sairedis-template.yml. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Signed-off-by: Lawrence Lee <lawlee@microsoft.com>
Since BuildSairedis now reuses sonic-sairedis's build template, the resulting sonic-sairedis is vpp-enabled (libsaivs depends on vpp/libvppinfra). Teach swss-common's own downstream stages to consume it: - build-swss-template.yml: download sonic-platform-vpp and install the vpp debs (VPP_INSTALL_SKIP_SYSCTL=1, in dependency order) before installing libsaivs. - build-docker-sonic-vs-template.yml: download the vpp debs so they are copied into the image build context. - docker-sonic-vs/Dockerfile: add the vpp install block (stub sysctl with true during docker build, install the vpp debs, restore sysctl) mirroring sonic-sairedis. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Signed-off-by: Lawrence Lee <lawlee@microsoft.com>
vpp depends on the SONiC libnl (>= 3.7.0-0.2+b1sonic1), so installing vpp before the libnl debs failed. Install all downloaded debs in a single dpkg -i so they configure in dependency order (libnl -> vpp -> libsaivs), with VPP_INSTALL_SKIP_SYSCTL=1 for vpp's postinst. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Signed-off-by: Lawrence Lee <lawlee@microsoft.com>
Address the unused-import findings from github-code-quality and CodeQL on PR #1220: - cascade.py: drop unused Deb, Wheel - planner.py: drop unused Dict - schema.py: drop unused os, List - tests/test_cascade.py: drop unused Wheel No behavior change; all 39 buildenv_setup unit tests still pass. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Signed-off-by: Lawrence Lee <lawlee@microsoft.com>
swss-common's BuildSairedis reused sairedis's template, whose buildenv_setup resolved swss-common from the latest completed external run (e.g. 1158908) -- so a swss-common change that breaks sairedis was NOT caught by swss-common's own CI. Pass stage_swss_common: true so the reused template stages THIS run's Build-stage sonic-swss-common artifact into the cascade instead. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Signed-off-by: Lawrence Lee <lawlee@microsoft.com>
theasianpianist
force-pushed
the
lawlee/ci-unify
branch
from
July 8, 2026 17:10
4e35584 to
ae4eafa
Compare
Collaborator
|
/azp run |
|
Azure Pipelines successfully started running 1 pipeline(s). |
…mon) Match sonic-sairedis's generalized staged_upstreams list param: stage this run's sonic-swss-common-<deb> artifact as upstream 'sonic-swss-common' so BuildSairedis builds sairedis against the current-run swss-common. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Signed-off-by: Lawrence Lee <lawlee@microsoft.com>
Collaborator
|
/azp run |
|
Azure Pipelines successfully started running 1 pipeline(s). |
The swss-common Build stage has pre-existing flaky unit tests (test_ProducerStateTable and others; #1221 fixed LruDedup + the memory-leak test) that intermittently redden Build and block the downstream PoC verification (BuildSairedis/BuildSwss can't run when Build fails -- 3 consecutive flaky Build failures observed). Mark the unit-test step continueOnError so PoC runs can exercise the cascade/reuse stages. NOT for the migration -- the migration should fix these flaky tests (like #1221 did the other two), not mask them. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Signed-off-by: Lawrence Lee <lawlee@microsoft.com>
Collaborator
|
/azp run |
|
Azure Pipelines successfully started running 1 pipeline(s). |
buildenv_setup.post_install.resolve_script now falls back to cascaded upstream build-env/ dirs (by basename) when a post_install source: script is not present locally. This lets a consumer reuse a hook the upstream bundle already provides, so the redis test-config script lives in exactly one repo (sonic-swss-common, the cascade root) instead of being duplicated in sonic-sairedis. The canonical script sets notify-keyspace-events AKE; sonic-sairedis now inherits that (its old standalone copy didn't). planner passes the cascaded build-env dirs as search_dirs. Adds tests/test_post_install.py (local-wins, upstream-fallback, not-found, and the scope-filter dedup that lets a consumer re-scope the shared hook to [build,test] while the upstream keeps it [test]). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Signed-off-by: Lawrence Lee <lawlee@microsoft.com>
Collaborator
|
/azp run |
|
Azure Pipelines successfully started running 1 pipeline(s). |
The apt_fix_broken dpkg path ran subprocess.run directly without echoing the command, so per-upstream install_env (e.g. vpp's VPP_INSTALL_SKIP_SYSCTL=1) and the deb list were invisible in CI logs. Add the same '+ <cmd>' log line the plain path emits. Adds tests/test_installer.py asserting the env prefix is present in the dpkg argv on BOTH paths (plain + apt_fix_broken) and absent when no env. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Signed-off-by: Lawrence Lee <lawlee@microsoft.com>
Collaborator
|
/azp run |
|
Azure Pipelines successfully started running 1 pipeline(s). |
| """Executor.dpkg_install argv construction -- ensures per-upstream install_env | ||
| (e.g. vpp's VPP_INSTALL_SKIP_SYSCTL=1) is applied on BOTH the plain and the | ||
| apt_fix_broken code paths.""" | ||
| import buildenv_setup.installer as installer |
| """Executor.dpkg_install argv construction -- ensures per-upstream install_env | ||
| (e.g. vpp's VPP_INSTALL_SKIP_SYSCTL=1) is applied on BOTH the plain and the | ||
| apt_fix_broken code paths.""" | ||
| import buildenv_setup.installer as installer |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Proof-of-concept for unifying build-environment setup across the SONiC dataplane repos, starting with sonic-swss-common.
Verified end-to-end in the sonic-slave-bookworm container against live common-lib / sonic-buildimage artifacts.