Skip to content

feat(flatcar): add sysext download and provider-id to Ignition config#770

Draft
mgrzybek wants to merge 10 commits into
mainfrom
feat/flatcar-sysexts
Draft

feat(flatcar): add sysext download and provider-id to Ignition config#770
mgrzybek wants to merge 10 commits into
mainfrom
feat/flatcar-sysexts

Conversation

@mgrzybek

@mgrzybek mgrzybek commented Jul 6, 2026

Copy link
Copy Markdown
Contributor

Extend the existing flatcarKubeadmConfigSpec helper to wire up the full Kubernetes bootstrap on Flatcar Linux via systemd-sysext.

  • Add flatcar.sysext{Registry,KubernetesTag,KubernetesDigest, ContainerdTag,ContainerdDigest} values for addressing OCI artifacts in Harbor
  • Inject storage.files (sysext blob download) and storage.links (symlinks into /etc/extensions/) into the Ignition additionalConfig; Ignition fetches them synchronously before any service starts
  • Set cloud-provider=external and provider-id=openstack:///<UUID> kubelet args via nodeRegistration, substituted at runtime by envsubst from the coreos-metadata EnvironmentFile
  • Export COREOS_OPENSTACK_INSTANCE_UUID alongside the existing hostname export so envsubst can resolve both variables in /etc/kubeadm.yml

mgrzybek added 7 commits July 7, 2026 11:16
Extend the existing flatcarKubeadmConfigSpec helper to wire up the full
Kubernetes bootstrap on Flatcar Linux via systemd-sysext.

- Add flatcar.sysext{Registry,KubernetesTag,KubernetesDigest,
  ContainerdTag,ContainerdDigest} values for addressing OCI artifacts
  in Harbor
- Inject storage.files (sysext blob download) and storage.links
  (symlinks into /etc/extensions/) into the Ignition additionalConfig;
  Ignition fetches them synchronously before any service starts
- Set cloud-provider=external and provider-id=openstack:///<UUID>
  kubelet args via nodeRegistration, substituted at runtime by envsubst
  from the coreos-metadata EnvironmentFile
- Export COREOS_OPENSTACK_INSTANCE_UUID alongside the existing hostname
  export so envsubst can resolve both variables in /etc/kubeadm.yml

Signed-off-by: Mathieu Grzybek <mathieu@stackhpc.com>
- Add values_flatcar.yaml and snapshot_flatcar_test.yaml for helm-unittest
  coverage of the Ignition/sysext rendering path
- Add kubeconform validation of Flatcar values in lint.yaml, runs on
  every PR without cloud credentials
- Add flatcar integration test job in test.yaml (tests-full only),
  skipped unless FLATCAR_IMAGE_ID repository variable is set; documents
  the six vars required to activate the job
- Refresh snapshot_full_test.yaml.snap to reflect AMD GPU operator removal
  from the chart (snapshot was stale)

Signed-off-by: Mathieu Grzybek <mathieu@stackhpc.com>
- Change kubeletExtraArgs to map format (cloud-provider, provider-id) to
  avoid silent data loss in mergeConcat when controlPlane.nodeLabels is set
  alongside osDistro=flatcar — list vs map type mismatch caused mergeConcat
  to drop node labels via `default $left $right`

- Add osDistro != flatcar guard to trustedCAs and additionalPackages helpers
  (update-ca-certificates and apt do not exist on Flatcar's immutable rootfs);
  the containerd helper already had this guard at line 289

- Add required() guards on all five flatcar.* values in flatcarKubeadmConfigSpec
  so helm template fails fast with a clear message rather than generating
  broken Ignition source URLs like https:///v2/.../blobs/ at node boot time

- Remove the no-op `export COREOS_OPENSTACK_INSTANCE_UUID=...` preKubeadmCommand
  (variable is already in the service environment via EnvironmentFile=/run/metadata/flatcar)

- Quote the heredoc delimiter as <<'EOF' in the Flatcar CI job to prevent
  shell re-expansion of already-substituted GHA variable values

- Use machineImage instead of machineImageId in values_flatcar.yaml to match
  the name-based lookup pattern of values_base.yaml (machineImageId expects
  a Glance UUID, not a release name)

Signed-off-by: Mathieu Grzybek <mathieu@stackhpc.com>
Signed-off-by: Mathieu Grzybek <mathieu@stackhpc.com>
Signed-off-by: Mathieu Grzybek <mathieu@stackhpc.com>
commit bf8e029 added amd-gpu-operator.yaml to cluster-addons sources but
did not update the vendored cluster-addons-0.1.0.tgz in openstack-cluster.
The CI Docker image resolves file:// dependencies from the live source
tree and therefore rendered the template; the local helm-unittest plugin
used the stale tgz and did not, causing a snapshot mismatch. Re-ran
`helm dependency update charts/openstack-cluster` and regenerated all
snapshots (202 → 205, +amd-gpu-operator).

Signed-off-by: Mathieu Grzybek <mathieu@stackhpc.com>
Replace the five Harbor-centric values (sysextRegistry, *Tag, *Digest)
with four S3-oriented values (sysextKubernetesUrl/Checksum,
sysextContainerdUrl/Checksum), consistent with how azimuth-images
publishes sysexts to Cloudflare R2.

The Ignition additionalConfig is updated accordingly:
- storage.files downloads from the S3 URL with Ignition checksum
  verification (sha256- prefix)
- storage.links preserved per Flatcar docs: sysexts land in
  /opt/extensions/ and are symlinked from /etc/extensions/

Update tests/values_flatcar.yaml and regenerate snapshots.

Signed-off-by: Mathieu Grzybek <mathieu@stackhpc.com>
@irt-shpc

irt-shpc commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

Needs rebasing

mgrzybek and others added 3 commits July 13, 2026 18:55
…ysext config

The schema merged in from main (PR #643) predates the flatcar sysext values
block added in this branch, so ct lint/helm lint rejected it as an
unexpected additional property.
Signed-off-by: Mathieu Grzybek <mathieu@stackhpc.com>
@mgrzybek
mgrzybek marked this pull request as draft July 15, 2026 09:17
@mgrzybek

Copy link
Copy Markdown
Contributor Author

We should wait for azimuth-cloud/azimuth-images#482 to be merged and the Flatcar images to be availabe.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants