Skip to content

feat: validate extraConfig against a per-release option catalog#711

Merged
berendt merged 7 commits into
mainfrom
implement/issue-703-extraconfig-option-catalog
Jul 21, 2026
Merged

feat: validate extraConfig against a per-release option catalog#711
berendt merged 7 commits into
mainfrom
implement/issue-703-extraconfig-option-catalog

Conversation

@berendt

@berendt berendt commented Jul 21, 2026

Copy link
Copy Markdown
Contributor

Closes #703

Summary

A misspelled option name in spec.extraConfig used to reach the rendered
config file unchanged and surface as a crash-looping pod. This branch
validates extraConfig option and section names at admission against a
per-release option catalog extracted from the service image forge ships,
for both the Keystone and Glance operators.

The three decisions fixed with the author hold throughout: unknown
sections are rejected (with an exemption list for plugin and dynamic
sections plus the ownership registry), UPDATE re-validates only when a
validated input changed
(extraConfig, the plugin section set, or the
release field), and an unresolvable catalog fails open with a single
admission warning, backed by a CI gate that requires a parseable catalog
for every shipped release.

Change set (in commit order)

  1. feat(common): add per-release option catalog module — adds
    internal/common/config/catalog.go (+ test), the shared,
    webhook-agnostic entry point. ParseOptionCatalog reads a
    service/release catalog; FindUnknownOptions reports options no
    release accepts and deprecated-but-accepted options, exempting oslo
    plugin/dynamic sections whole and ownership-registry (section, key)
    pairs individually via KeyExemptionsFromRegistry. Pure functions, no
    Kubernetes or webhook dependencies, so both webhooks — and the future
    ControlPlane caller (Expose freeform service configuration on the ControlPlane #704) — can consume it directly.

  2. feat(images): generate per-release option catalogs from service images — adds hack/gen-option-catalog.sh, which runs the image's
    own oslo-config-generator --format json against the upstream
    generator config for the pinned release and post-processes it into the
    compact catalog format. Option/deprecated/replacement names are
    normalized from the generator's dash spelling to the underscore spelling
    oslo reads from a file. A plausibility floor (≥10 sections, ≥100
    options, a DEFAULT section) guards against a truncated run
    overwriting a good catalog. Commits the four generated catalogs
    (keystone + glance × 2025.2 + 2026.1) as byte-stable caches, plus the
    gen-option-catalogs / verify-option-catalogs Makefile targets.

  3. feat(keystone): validate extraConfig against the release option catalog — the keystone validating webhook embeds the per-release
    catalogs and rejects unknown extraConfig options/sections against the
    release derived from spec.image.tag. Sections from spec.plugins and
    ownership-registry keys are exempt; a digest pin or unshipped release
    fails open with one warning; deprecated options are accepted with a
    warning naming their replacement; UPDATE re-runs only on a relevant
    change. Because the gate now fires on the config-pruning marker key,
    that e2e test switches its marker to a real oslo.log option.

  4. feat(glance): validate extraConfig against the release option catalog — mirrors the keystone check in the glance webhook, keyed off
    spec.openStackRelease. Additionally exempts the reserved
    glance_store sections os_glance_staging_store /
    os_glance_tasks_store whole, while an arbitrary backend-named section
    is still rejected (backend options belong on the GlanceBackend CR,
    not spec.extraConfig).

  5. ci: gate service images on regenerated option catalogs — on PRs
    the build-service-images job re-extracts each catalog from the freshly
    built image via gen-option-catalog.sh --check (passing SOURCE_DIR so
    release patches are honored) and diffs against the committed file; the
    post-merge verify-service-images job repeats it against the pushed
    composite image. A new per-release gate in verify_release_config.sh
    requires a parseable catalog for every release under releases/, which
    keeps the webhook's fail-open path unreachable on main for known
    releases. Catalog paths and the generator script join the workflow path
    filters.

  6. test(e2e): add unknown-option and unknown-section invalid-cr fixtures — pins the admission-time rejection through chainsaw for both
    operators. Each invalid-cr suite gains two generated fixtures (a typo'd
    option in a known section, and a typo'd section name) asserting the
    no such option/section in the <service> 2025.2 option catalog webhook
    messages.

  7. docs: document admission-time extraConfig catalog validation
    documents the catalog check in the Keystone/Glance CRD references and
    the advanced-configuration guide (rejection of unknown sections/options
    with the section and key named; plugin/operator-owned/Glance-store
    exemptions; deprecated-and-unresolvable warn-not-block; webhook-only,
    no CEL backstop), and rewrites the stale "content is not validated"
    claim.

Notes for the reviewer

  • The catalog module is deliberately webhook-agnostic and carries no
    field.Error; each webhook renders its own paths and messages so the
    planned Expose freeform service configuration on the ControlPlane #704 ControlPlane caller can attribute a finding to the level
    that contributed the key. The ControlPlane wiring itself is not in
    this branch — it is the Expose freeform service configuration on the ControlPlane #704 follow-up.
  • Keystone keys the catalog off spec.image.tag, Glance off
    spec.openStackRelease, matching how each operator already names its
    release. Both fall open (warn, don't block) when the release cannot be
    resolved to an embedded catalog.
  • gen-option-catalogs / verify-option-catalogs need docker + network
    and are intentionally wired into no aggregate target; the CI gates run
    the check against built images instead.

Implemented by planwerk-agent c4682b7 with Claude:claude-opus-4-8

berendt added 7 commits July 21, 2026 19:32
Add the shared, webhook-agnostic entry point for validating extraConfig
option names against a per-release option catalog. The module parses a
service/release catalog and, via FindUnknownOptions, reports options no
release accepts and deprecated-but-accepted options, exempting oslo
plugin and dynamic sections whole and ownership-registry (section, key)
pairs individually. It carries no Kubernetes or webhook dependencies so
the keystone and glance webhooks can consume it directly, and the
ControlPlane-level check will run FindUnknownOptions against its merged
configuration result.

On-behalf-of: @SAP
Assisted-by: Claude:claude-fable-5
Signed-off-by: Christian Berendt <berendt@23technologies.cloud>
Add hack/gen-option-catalog.sh, which extracts a per-release option
catalog from a shipped service image. It runs the image's own
oslo-config-generator with --format json against the upstream generator
config for the pinned release, then post-processes the machine-readable
output into the compact catalog format ParseOptionCatalog consumes.

The generator emits raw option names with dashes; a config file spells
them with underscores, so option names, deprecated names, and their
replacements are normalized to the underscore form oslo.config reads
from a file. A deprecated old name becomes an entry in the section it
lived in, pointing at its current section and name; a live option never
doubles as a deprecated key.

A plausibility floor (at least ten sections, at least a hundred options,
and a DEFAULT section) runs inside the transform before anything is
written, so a truncated or degraded run can never overwrite a good
catalog.

Commit the four generated catalogs for keystone and glance across the
2025.2 and 2026.1 releases. They are byte-stable caches: re-running the
script in --check mode diffs a fresh extraction against the committed
file and fails on any drift. Two Makefile targets, gen-option-catalogs
and verify-option-catalogs, drive the generate and check loops; both
need docker and network access and are wired into no aggregate target.

On-behalf-of: @SAP
Assisted-by: Claude:claude-fable-5
Signed-off-by: Christian Berendt <berendt@23technologies.cloud>
The operator now embeds per-release option catalogs, and the
validating webhook rejects unknown extraConfig options and sections
against the release derived from spec.image.tag. Sections declared
via spec.plugins and keys owned through the ownership registry are
exempt from the check. A digest pin or an unshipped release cannot be
resolved to a catalog, so validation fails open with a single
admission warning instead of blocking. Deprecated options are
accepted with a warning that names their replacement. On UPDATE the
catalog check re-runs only when extraConfig, the plugin section set,
or the image tag changed. Because that gate now fires on the inert
marker key, the config-pruning e2e test had to switch its marker to a
real oslo.log option.

On-behalf-of: @SAP
Assisted-by: Claude:claude-fable-5
Signed-off-by: Christian Berendt <berendt@23technologies.cloud>
The validating webhook now rejects unknown spec.extraConfig options and
sections against the option catalog embedded for the release named by
spec.openStackRelease, mirroring the keystone check. The reserved
glance_store sections os_glance_staging_store and os_glance_tasks_store
are exempt whole, alongside sections declared via spec.plugins and the
keys owned through the ownership registry, so file-store overrides stay
settable. An arbitrary backend-named section is rejected because backend
options belong to the GlanceBackend CR, not spec.extraConfig. When
spec.openStackRelease cannot be resolved to an embedded catalog, the
check fails open with a single admission warning instead of blocking,
and deprecated options are accepted with a warning naming their
replacement.
On UPDATE the catalog check re-runs only when extraConfig, the plugin
config-section set, or spec.openStackRelease changed.

On-behalf-of: @SAP
Assisted-by: Claude:claude-fable-5
Signed-off-by: Christian Berendt <berendt@23technologies.cloud>
On pull requests, the build-service-images job re-extracts each
keystone and glance option catalog from the freshly built image via
gen-option-catalog.sh --check, passing SOURCE_DIR so release patches
are honored, and diffs it against the committed catalog. The
post-merge verify-service-images job repeats the same check against
the pushed composite image, this time curling the generator config
from the pinned ref. A new per-release static gate in
verify_release_config.sh requires a parseable catalog for every
release under releases/, which keeps the webhook's fail-open path
unreachable on main for known releases. The catalog paths and the
generator script are added to the workflow path filters so catalog
or script edits retrigger the gates.

On-behalf-of: @SAP
Assisted-by: Claude:claude-fable-5
Signed-off-by: Christian Berendt <berendt@23technologies.cloud>
Pin the admission-time option-catalog rejection through chainsaw for
both the keystone and glance operators. Each invalid-cr suite gains two
generated fixtures: a typo'd option in a known section and a typo'd
section name. They assert the "no such option/section in the <service>
2025.2 option catalog" webhook messages against the 2025.2 release the
scaffolds already pin.

On-behalf-of: @SAP
Assisted-by: Claude:claude-fable-5
Signed-off-by: Christian Berendt <berendt@23technologies.cloud>
Describe the per-release option catalog the Keystone and Glance
validating webhooks now check spec.extraConfig against: unknown
sections and options are rejected with the section and key named,
plugin sections, operator-owned keys, and Glance store sections are
exempt, deprecated options and unresolvable releases warn instead of
blocking, and the rule is webhook-only with no CEL backstop. Also
rewrite the stale "content is not validated" claim in the advanced
configuration guide.

On-behalf-of: @SAP
Assisted-by: Claude:claude-fable-5
Signed-off-by: Christian Berendt <berendt@23technologies.cloud>
@berendt
berendt force-pushed the implement/issue-703-extraconfig-option-catalog branch from c5c3921 to c25ad59 Compare July 21, 2026 18:43
@berendt
berendt marked this pull request as ready for review July 21, 2026 20:35
@berendt
berendt merged commit a19da63 into main Jul 21, 2026
89 checks passed
@berendt
berendt deleted the implement/issue-703-extraconfig-option-catalog branch July 21, 2026 20:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Validate freeform configuration against a per-release option catalog

1 participant