fix(security): harden containers with read-only root filesystems (Kubescape C-0017)#2455
Conversation
…escape C-0017) Set readOnlyRootFilesystem: true (plus the minimal writable emptyDir mounts each workload genuinely needs) on every non-privileged workload that tolerates it — homepage, umami, actual-budget, backstage, crossview + its bundled Postgres, ascoachingogvaner, and the four Crossplane providers — so Kubescape C-0017 passes for real instead of being suppressed by a blanket exception. Narrow the exception layer: drop C-0017 from the cluster-wide pod-security-mutations exception (now set in-spec), add the kubescape namespace to infrastructure-privileged (its node-agent + storage APIserver genuinely need host/writable access), and scope the remaining C-0017 exception (readonly-rootfs-pending) to just opencost (its nginx UI rewrites baked /var/www assets on boot — opencost/opencost#2655) and wedding-app (securityContext lives in its own OCI artifact — app-repo follow-up). Adding a writable /tmp emptyDir to provider-upjet-unifi's DeploymentRuntimeConfig also fixes its currently-stalled reconciliation (cluster-wireguard SYNCED=False: "mkdir /tmp/<uuid>: read-only file system"). Crossplane appends its TLS/SA volumes after the DRC template, so the emptyDir coexists with them. Each workload's writable-path list was researched against its live image and adversarially verified not to crash the app. Both overlays validate (ksail workload validate). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Organization UI Review profile: ASSERTIVE Plan: Pro Plus Run ID: 📒 Files selected for processing (6)
🔗 Linked repositories identifiedCodeRabbit considers these linked repositories for cross-repo context during reviews:
📜 Recent review details🧰 Additional context used📓 Path-based instructions (4)**/*.{yaml,yml}📄 CodeRabbit inference engine (AGENTS.md)
Files:
k8s/**📄 CodeRabbit inference engine (AGENTS.md)
Files:
k8s/bases/infrastructure/**📄 CodeRabbit inference engine (AGENTS.md)
Files:
k8s/bases/infrastructure/**/*.{yaml,yml}📄 CodeRabbit inference engine (AGENTS.md)
Files:
🧠 Learnings (2)📚 Learning: 2026-07-01T21:13:36.950ZApplied to files:
📚 Learning: 2026-07-04T13:30:04.759ZApplied to files:
🪛 Trivy (0.69.3)k8s/bases/infrastructure/cluster-security-exceptions/kubescape-privileged.yaml[info] 17-53: limit range usage A LimitRange policy with a default requests and limits for each container should be configured Rule: KSV-0039 (IaC/Kubernetes) 🔀 Multi-repo context devantler-tech/wedding-app, devantler-tech/ascoachingogvanerLinked repositories findingsdevantler-tech/wedding-app
devantler-tech/ascoachingogvaner
🔇 Additional comments (6)
📝 WalkthroughWalkthroughThis PR enables Changes
Sequence Diagram(s)Not applicable — these are declarative Kubernetes manifest updates without a new runtime interaction flow. Possibly related PRs
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
Comment |
|
@coderabbitai review |
✅ Action performedReview finished.
|
|
@coderabbitai review |
✅ Action performedReview finished.
|
Heads-up: #2467 fast-tracks the provider-upjet-unifi |
|
@coderabbitai review
|
|
✅ Action performedReview finished.
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 4
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@k8s/bases/apps/ascoachingogvaner/flux-kustomization.yaml`:
- Around line 69-75: The three new emptyDir volumes in the volume list are
unbounded, so cap each one with a sizeLimit to match the /tmp pattern used
elsewhere in the PR. Update the tmp, nginx-cache, and nginx-run volume
definitions in the kustomization so each emptyDir includes an explicit limit
(for example, 1Gi) to prevent unchecked node disk usage. Keep the change
localized to the volume specs in this manifest.
In `@k8s/bases/apps/crossview/helm-release.yaml`:
- Around line 88-92: The postgres-data emptyDir volume is uncapped and can grow
with live PGDATA usage, unlike the tmp scratch mount. Update the volume
definition in the Helm release manifest for the Postgres pod so postgres-data
has an explicit size limit while leaving the tmp volume unchanged. Use the
existing volumes block in the crossview release to locate the postgres-data
entry and add the cap there.
In `@k8s/bases/apps/umami/helm-release.yaml`:
- Around line 161-170: The `extraVolumes` entries for `next-cache` and `tmp` are
using uncapped `emptyDir` volumes, which can grow without bound. Update the
`helm-release.yaml` volume definitions for these mounts to include a
`sizeLimit`, following the same pattern used for backstage’s `/tmp` volume in
this PR. Keep the fix scoped to the `extraVolumes`/`extraVolumeMounts` section
so the `next-cache` and `tmp` mounts remain unchanged except for the new limits.
In
`@k8s/bases/infrastructure/cluster-security-exceptions/infrastructure-privileged.yaml`:
- Around line 10-12: Scope the exception in the cluster security policy to only
the intended Kubescape workloads instead of the whole namespace. Update the
policy around the `kubescape` node-agent/storage exception to use
`match.resources` with kind/name globs for the specific objects, so the
`kubescape` operator and kubevuln are not exempted. Refer to the existing
`infrastructure-privileged` exception definition and replace the namespace-level
selector with resource-level matching for the node-agent and storage workloads
only.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: c42991a1-2973-4c8b-9efb-bf9247786121
📒 Files selected for processing (15)
k8s/bases/apps/actual-budget/helm-release.yamlk8s/bases/apps/ascoachingogvaner/flux-kustomization.yamlk8s/bases/apps/backstage/helm-release.yamlk8s/bases/apps/crossview/helm-release.yamlk8s/bases/apps/homepage/helm-release.yamlk8s/bases/apps/umami/helm-release.yamlk8s/bases/infrastructure/cluster-security-exceptions/infrastructure-privileged.yamlk8s/bases/infrastructure/cluster-security-exceptions/kustomization.yamlk8s/bases/infrastructure/cluster-security-exceptions/pod-security-mutations.yamlk8s/bases/infrastructure/cluster-security-exceptions/readonly-rootfs-pending.yamlk8s/bases/infrastructure/opencost/helm-release.yamlk8s/providers/hetzner/infrastructure/crossplane/deployment-runtime-config-aws-iam.yamlk8s/providers/hetzner/infrastructure/crossplane/deployment-runtime-config-family-aws.yamlk8s/providers/hetzner/infrastructure/crossplane/deployment-runtime-config-upjet-unifi.yamlk8s/providers/hetzner/infrastructure/crossplane/deployment-runtime-config.yaml
🔗 Linked repositories identified
CodeRabbit considers these linked repositories for cross-repo context during reviews:
devantler-tech/actions(auto-detected)devantler-tech/aws(auto-detected)devantler-tech/reusable-workflows(auto-detected)devantler-tech/ksail(auto-detected)devantler-tech/ascoachingogvaner(auto-detected)devantler-tech/wedding-app(auto-detected)devantler-tech/agent-skills(auto-detected)
📜 Review details
🧰 Additional context used
📓 Path-based instructions (4)
**/*.{yaml,yml}
📄 CodeRabbit inference engine (AGENTS.md)
**/*.{yaml,yml}: Use Kustomize overlays rather than editing base resources directly;k8s/bases/is immutable from overlays and changes should be made withpatches:in provider or cluster overlays.
Keep manifest changes small and use YAML/schema validation before submitting a manifest PR; for files with cluster context, preferksail workload validate/kubectl kustomize/kubectl apply --dry-run=clientas appropriate.
Files:
k8s/bases/apps/backstage/helm-release.yamlk8s/bases/infrastructure/cluster-security-exceptions/kustomization.yamlk8s/bases/infrastructure/cluster-security-exceptions/readonly-rootfs-pending.yamlk8s/bases/infrastructure/opencost/helm-release.yamlk8s/bases/apps/homepage/helm-release.yamlk8s/bases/apps/ascoachingogvaner/flux-kustomization.yamlk8s/bases/apps/actual-budget/helm-release.yamlk8s/providers/hetzner/infrastructure/crossplane/deployment-runtime-config.yamlk8s/providers/hetzner/infrastructure/crossplane/deployment-runtime-config-upjet-unifi.yamlk8s/bases/apps/crossview/helm-release.yamlk8s/providers/hetzner/infrastructure/crossplane/deployment-runtime-config-family-aws.yamlk8s/providers/hetzner/infrastructure/crossplane/deployment-runtime-config-aws-iam.yamlk8s/bases/apps/umami/helm-release.yamlk8s/bases/infrastructure/cluster-security-exceptions/infrastructure-privileged.yamlk8s/bases/infrastructure/cluster-security-exceptions/pod-security-mutations.yaml
k8s/**
📄 CodeRabbit inference engine (AGENTS.md)
k8s/**: Respect Flux dependency order:bootstrap→infrastructure-controllers→infrastructure→apps, with the prod-onlyinfrastructure-overprovisioninglayer hanging offinfrastructurewithout gatingapps.
Follow the hierarchical Kustomization flow: base configurations ink8s/bases/feed provider overlays ink8s/providers/, which feed cluster overlays ink8s/clusters/.
Files:
k8s/bases/apps/backstage/helm-release.yamlk8s/bases/infrastructure/cluster-security-exceptions/kustomization.yamlk8s/bases/infrastructure/cluster-security-exceptions/readonly-rootfs-pending.yamlk8s/bases/infrastructure/opencost/helm-release.yamlk8s/bases/apps/homepage/helm-release.yamlk8s/bases/apps/ascoachingogvaner/flux-kustomization.yamlk8s/bases/apps/actual-budget/helm-release.yamlk8s/providers/hetzner/infrastructure/crossplane/deployment-runtime-config.yamlk8s/providers/hetzner/infrastructure/crossplane/deployment-runtime-config-upjet-unifi.yamlk8s/bases/apps/crossview/helm-release.yamlk8s/providers/hetzner/infrastructure/crossplane/deployment-runtime-config-family-aws.yamlk8s/providers/hetzner/infrastructure/crossplane/deployment-runtime-config-aws-iam.yamlk8s/bases/apps/umami/helm-release.yamlk8s/bases/infrastructure/cluster-security-exceptions/infrastructure-privileged.yamlk8s/bases/infrastructure/cluster-security-exceptions/pod-security-mutations.yaml
k8s/bases/infrastructure/**
📄 CodeRabbit inference engine (AGENTS.md)
k8s/bases/infrastructure/**: Underk8s/bases/infrastructure/, organize resources component-folder-first: a component's HelmRelease/HelmRepository and its own CRs should live together in a folder named after the component unless a split is required.
Split a custom resource into its own plural-Kind folder only when it cannot live with its component, such as for CRD dependency ordering or because it is cluster-scoped/cross-cutting.
Files:
k8s/bases/infrastructure/cluster-security-exceptions/kustomization.yamlk8s/bases/infrastructure/cluster-security-exceptions/readonly-rootfs-pending.yamlk8s/bases/infrastructure/opencost/helm-release.yamlk8s/bases/infrastructure/cluster-security-exceptions/infrastructure-privileged.yamlk8s/bases/infrastructure/cluster-security-exceptions/pod-security-mutations.yaml
k8s/bases/infrastructure/**/*.{yaml,yml}
📄 CodeRabbit inference engine (AGENTS.md)
k8s/bases/infrastructure/**/*.{yaml,yml}: For component-folder files, name manifests after the resource Kind in kebab-case; if a folder contains multiple resources of the same Kind, qualify filenames with a purpose suffix.
For CR-folder files, omit the folder-implied Kind from the filename and use theverb-purpose.yamlform.
Name FluxKustomizationresourcesflux-kustomization*.yaml; keep the kustomize build file named exactlykustomization.yaml.
Files:
k8s/bases/infrastructure/cluster-security-exceptions/kustomization.yamlk8s/bases/infrastructure/cluster-security-exceptions/readonly-rootfs-pending.yamlk8s/bases/infrastructure/opencost/helm-release.yamlk8s/bases/infrastructure/cluster-security-exceptions/infrastructure-privileged.yamlk8s/bases/infrastructure/cluster-security-exceptions/pod-security-mutations.yaml
🧠 Learnings (2)
📚 Learning: 2026-07-01T21:13:36.950Z
Learnt from: devantler
Repo: devantler-tech/platform PR: 2359
File: k8s/bases/apps/actual-budget/helm-release.yaml:62-111
Timestamp: 2026-07-01T21:13:36.950Z
Learning: When reviewing Kustomize/Helm YAML in this repo, keep the base vs provider overlay split: `k8s/bases/apps/**` and `k8s/bases/infrastructure/**` should contain each app’s full, environment-agnostic configuration (including base-level postRenderer Kustomize patches such as deployment strategy, topology spread, probes, and env injection). `k8s/providers/{docker,hetzner}/**` should only add small provider-specific deltas (e.g., `interval`, `persistence.size`) via patch files (like `k8s/providers/<provider>/apps/<app>/patches/helm-release-patch.yaml`). If configuration is identical across providers (e.g., OIDC/OAuth env vars where `${domain}` is resolved per cluster via envsubst), it belongs in the base and must not be duplicated into provider overlays.
Applied to files:
k8s/bases/apps/backstage/helm-release.yamlk8s/bases/infrastructure/cluster-security-exceptions/kustomization.yamlk8s/bases/infrastructure/cluster-security-exceptions/readonly-rootfs-pending.yamlk8s/bases/infrastructure/opencost/helm-release.yamlk8s/bases/apps/homepage/helm-release.yamlk8s/bases/apps/ascoachingogvaner/flux-kustomization.yamlk8s/bases/apps/actual-budget/helm-release.yamlk8s/providers/hetzner/infrastructure/crossplane/deployment-runtime-config.yamlk8s/providers/hetzner/infrastructure/crossplane/deployment-runtime-config-upjet-unifi.yamlk8s/bases/apps/crossview/helm-release.yamlk8s/providers/hetzner/infrastructure/crossplane/deployment-runtime-config-family-aws.yamlk8s/providers/hetzner/infrastructure/crossplane/deployment-runtime-config-aws-iam.yamlk8s/bases/apps/umami/helm-release.yamlk8s/bases/infrastructure/cluster-security-exceptions/infrastructure-privileged.yamlk8s/bases/infrastructure/cluster-security-exceptions/pod-security-mutations.yaml
📚 Learning: 2026-07-04T13:30:04.759Z
Learnt from: devantler
Repo: devantler-tech/platform PR: 2446
File: k8s/bases/infrastructure/cluster-security-exceptions/delete-rbac.yaml:38-125
Timestamp: 2026-07-04T13:30:04.759Z
Learning: For Kubescape ClusterSecurityException (apiVersion kubescape.io/v1beta1) and the mirrored Headlamp exception config, do NOT pin `spec.match.resources[].name` (and Headlamp `attributes.name`) to a single literal value when the identifier includes a generated hash. These fields are compared using `regexCompare`, so match such resources with an anchored regular expression that covers the stable prefix and the hash pattern (e.g., `^crossplane:provider:provider-upjet-github-[0-9a-f]+:system$`) rather than the current hash, so the exception remains valid across provider re-installs/revisions.
Applied to files:
k8s/bases/infrastructure/cluster-security-exceptions/kustomization.yamlk8s/bases/infrastructure/cluster-security-exceptions/readonly-rootfs-pending.yamlk8s/bases/infrastructure/cluster-security-exceptions/infrastructure-privileged.yamlk8s/bases/infrastructure/cluster-security-exceptions/pod-security-mutations.yaml
🪛 Trivy (0.69.3)
k8s/bases/infrastructure/cluster-security-exceptions/readonly-rootfs-pending.yaml
[info] 23-40: limit range usage
A LimitRange policy with a default requests and limits for each container should be configured
Rule: KSV-0039
(IaC/Kubernetes)
k8s/providers/hetzner/infrastructure/crossplane/deployment-runtime-config-upjet-unifi.yaml
[info] 9-49: limit range usage
A LimitRange policy with a default requests and limits for each container should be configured
Rule: KSV-0039
(IaC/Kubernetes)
k8s/providers/hetzner/infrastructure/crossplane/deployment-runtime-config-family-aws.yaml
[info] 8-47: limit range usage
A LimitRange policy with a default requests and limits for each container should be configured
Rule: KSV-0039
(IaC/Kubernetes)
k8s/providers/hetzner/infrastructure/crossplane/deployment-runtime-config-aws-iam.yaml
[info] 6-45: limit range usage
A LimitRange policy with a default requests and limits for each container should be configured
Rule: KSV-0039
(IaC/Kubernetes)
k8s/bases/infrastructure/cluster-security-exceptions/infrastructure-privileged.yaml
[info] 17-58: limit range usage
A LimitRange policy with a default requests and limits for each container should be configured
Rule: KSV-0039
(IaC/Kubernetes)
🔀 Multi-repo context devantler-tech/wedding-app, devantler-tech/ascoachingogvaner, devantler-tech/ksail
Linked repositories findings
devantler-tech/wedding-app
deploy/deployment.yaml:35-60— the deployment already has pod-level hardening, but the containersecurityContextstill setsreadOnlyRootFilesystem: false. This matches the PR’s new temporary C-0017 exception forwedding-app; the downstream repo is not yet hardened. [::devantler-tech/wedding-app::]
devantler-tech/ascoachingogvaner
deploy/deployment.yaml:30-60— the deployment also has container hardening in place, butreadOnlyRootFilesystemremainsfalse. This lines up with the PR change that hardensascoachingogvanervia a Flux patch. [::devantler-tech/ascoachingogvaner::]
devantler-tech/ksail
pkg/svc/mirror/inject.go:144-148— ksail’s mirror injection already defaultsReadOnlyRootFilesystemtotrue, so the PR’s hardening direction is consistent with existing ksail behavior. [::devantler-tech/ksail::]
🔇 Additional comments (12)
k8s/bases/apps/actual-budget/helm-release.yaml (1)
189-199: LGTM!k8s/bases/apps/backstage/helm-release.yaml (1)
130-156: LGTM!k8s/bases/apps/crossview/helm-release.yaml (1)
38-56: LGTM!k8s/bases/apps/homepage/helm-release.yaml (1)
164-190: LGTM!k8s/providers/hetzner/infrastructure/crossplane/deployment-runtime-config-aws-iam.yaml (1)
14-45: LGTM!k8s/providers/hetzner/infrastructure/crossplane/deployment-runtime-config-family-aws.yaml (1)
16-47: LGTM!k8s/providers/hetzner/infrastructure/crossplane/deployment-runtime-config-upjet-unifi.yaml (1)
17-49: LGTM!k8s/providers/hetzner/infrastructure/crossplane/deployment-runtime-config.yaml (1)
17-37: 🩺 Stability & AvailabilityDouble-check the no-
/tmpassumption for provider-upjet-github. This runtime omits a writable/tmp; any reconcile/init path that still writes workspaces locally will fail under the read-only root.k8s/bases/infrastructure/cluster-security-exceptions/pod-security-mutations.yaml (1)
2-34: LGTM!k8s/bases/infrastructure/cluster-security-exceptions/readonly-rootfs-pending.yaml (1)
1-41: LGTM!k8s/bases/infrastructure/cluster-security-exceptions/kustomization.yaml (1)
13-13: LGTM!k8s/bases/infrastructure/opencost/helm-release.yaml (1)
190-199: LGTM!
github-project-automation
Bot
moved this from 🫴 Ready
to 🏃🏻♂️ In Progress
in 🌊 Project Board
…xt-readonly-rootfs
…xception to node-agent/storage Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
github-project-automation
Bot
moved this from 🏃🏻♂️ In Progress
to ✅ Done
in 🌊 Project Board
|
🎉 This PR is included in version 1.98.1 🎉 The release is available on GitHub release Your semantic-release bot 📦🚀 |
1 participant
Why: Kubescape control C-0017 (immutable container filesystem) was failing cluster-wide. The platform suppressed the finding with a blanket exception instead of actually hardening the workloads, so several apps genuinely ran with a writable root filesystem. Investigating also surfaced a live bug — the UniFi Crossplane provider is currently failing to reconcile (its
cluster-wireguardresource is unsynced) because its read-only root has no writable temp dir.What: Give a genuine read-only root filesystem (with only the minimal writable mounts each app truly needs) to every workload that tolerates it — homepage, umami, actual-budget, backstage, crossview (+ its bundled Postgres), ascoachingogvaner, and the four Crossplane providers — so C-0017 passes for real rather than being suppressed. The blanket exception is narrowed to only the genuinely-incompatible cases: opencost's nginx UI (rewrites its baked assets on boot — upstream bug) and wedding-app (securityContext lives in its own repo), plus the privileged infra namespaces. Adding a writable
/tmpto the UniFi provider also fixes its stalled reconciliation.Each workload's writable-path list was researched against its live image and adversarially verified not to crash the app.
Operational notes (post-promotion):
cluster-wireguardflips toSYNCED=Trueonce the new DeploymentRuntimeConfig rolls out./tmp), then drop it from the pending exception.