Problem
The Foreman gate runs Go-only checks (make fmt/vet/lint/test + bite check). It does not validate sample manifests against the CRD OpenAPI schemas. So a coder can produce a config/samples/** change with invalid or hallucinated CRD fields (wrong types, non-existent fields) and it GATE-PASSes, because nothing checks YAML against the schema.
Observed (dogfood #699): a coder rewrote config/samples/model_amd_vulkan_igpu.yaml using Model.spec.url (field is source), spec.parameters.{contextSize,embeddingSize,layerCount} (no such block), InferenceService.spec.modelRef: {name: …} (it's a string), runtime: {…} (it's a string), spec.service (it's endpoint). The manifest would fail kubectl apply, yet it GATE-PASSed because the change touched no Go.
Proposed fix
Add a make validate-samples target that:
- Extracts JSON Schemas from
config/crd/bases/*.yaml (kubeconform's CRD extractor / openapi2jsonschema).
- Runs
kubeconform on config/samples/**.yaml against those schemas plus the built-in Kubernetes schemas.
Wire it into CI (a workflow step on config/{samples,crd}/** changes). Optionally add it to the gate's check list so a coder's sample change is validated in the self-gate, not just at CI.
Acceptance
make validate-samples fails on a sample with an unknown/mistyped CRD field and passes on the current valid samples.
- CI runs it on PRs touching samples or CRDs.
Problem
The Foreman gate runs Go-only checks (
make fmt/vet/lint/test+ bite check). It does not validate sample manifests against the CRD OpenAPI schemas. So a coder can produce aconfig/samples/**change with invalid or hallucinated CRD fields (wrong types, non-existent fields) and it GATE-PASSes, because nothing checks YAML against the schema.Observed (dogfood #699): a coder rewrote
config/samples/model_amd_vulkan_igpu.yamlusingModel.spec.url(field issource),spec.parameters.{contextSize,embeddingSize,layerCount}(no such block),InferenceService.spec.modelRef: {name: …}(it's a string),runtime: {…}(it's a string),spec.service(it'sendpoint). The manifest would failkubectl apply, yet it GATE-PASSed because the change touched no Go.Proposed fix
Add a
make validate-samplestarget that:config/crd/bases/*.yaml(kubeconform's CRD extractor /openapi2jsonschema).kubeconformonconfig/samples/**.yamlagainst those schemas plus the built-in Kubernetes schemas.Wire it into CI (a workflow step on
config/{samples,crd}/**changes). Optionally add it to the gate's check list so a coder's sample change is validated in the self-gate, not just at CI.Acceptance
make validate-samplesfails on a sample with an unknown/mistyped CRD field and passes on the current valid samples.