Skip to content

fix(kubescape): remount scanner config.json so offline posture scans persist#2439

Closed
devantler wants to merge 1 commit into
mainfrom
claude/kubescape-offline-config-mount
Closed

fix(kubescape): remount scanner config.json so offline posture scans persist#2439
devantler wants to merge 1 commit into
mainfrom
claude/kubescape-offline-config-mount

Conversation

@devantler

Copy link
Copy Markdown
Contributor

🤖 Generated by the Daily AI Assistant

Why

The in-cluster Kubescape compliance dashboard (Headlamp) is frozen — it serves stale data from 2026-06-27 and shows controls like C-0007 "Roles with delete capabilities" as 0 of 26 passed. Root cause: an offline-mode bug in the kubescape chart mounts the scanner's config file as a directory, so every scan aborts before it finishes — no fresh results are stored and none of the platform's by-design posture exceptions are ever applied (aggregate compliance reads 0.00 on every framework).

What

Reproduces the merged upstream fix (kubescape/helm-charts#862, issue #857) with a Flux post-render that remounts the config volume correctly, so scans finalize again, the dashboard refreshes, and the exception CRs already shipped in #2316 finally take effect. Offline/air-gapped mode is untouched — no cloud submit is re-enabled. Revert this once the chart ships a release containing the upstream fix.

Note: this unblocks the whole dashboard but does not on its own make C-0007 pass — that control's exception needs a separate match-type fix (follow-up PR). After promotion Flux reconciles the HelmRelease; a fresh scan should then persist and the score reappear. Related: #2264.

…persist

The kubescape-operator 1.40.2 scanner Deployment mounts the empty
`kubescape-volume` emptyDir onto the FILE path
/home/nonroot/.kubescape/config.json (subPath config.json). kubelet
materialises the non-existent target as a DIRECTORY, so the scanner's
config write fails ("open .../config.json: is a directory"), the scan
aborts at finalize, the aggregate ConfigurationScanSummary is never
written, and no posture (Cluster)SecurityException is ever applied — so
the compliance dashboard freezes (last persisted scan 2026-06-27) and
every by-design exception (e.g. C-0007) renders failed.

Reproduce upstream fix kubescape/helm-charts#862 (issue #857) via a Flux
postRenderer that remounts the volume at the parent dir, until a chart
release ships it. Offline mode / submit=false unchanged.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Jul 4, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 996cc7cf-9418-4380-a184-8971a6fb6bbf

📥 Commits

Reviewing files that changed from the base of the PR and between 32ce888 and 0470209.

📒 Files selected for processing (1)
  • k8s/bases/infrastructure/controllers/kubescape/helm-release.yaml
📜 Recent review details
⏰ Context from checks skipped due to timeout. (4)
  • GitHub Check: 🧪 Validate Manifests
  • GitHub Check: Analyze (actions)
  • GitHub Check: Analyze (python)
  • GitHub Check: Analyze (python)
🧰 Additional context used
📓 Path-based instructions (4)
**/*.{yaml,yml}

📄 CodeRabbit inference engine (AGENTS.md)

**/*.{yaml,yml}: Use Kustomize overlays rather than editing base resources directly; k8s/bases/ is immutable from overlays and changes should be made with patches: in provider or cluster overlays.
Keep manifest changes small and use YAML/schema validation before submitting a manifest PR; for files with cluster context, prefer ksail workload validate / kubectl kustomize / kubectl apply --dry-run=client as appropriate.

Files:

  • k8s/bases/infrastructure/controllers/kubescape/helm-release.yaml
k8s/**

📄 CodeRabbit inference engine (AGENTS.md)

k8s/**: Respect Flux dependency order: bootstrapinfrastructure-controllersinfrastructureapps, with the prod-only infrastructure-overprovisioning layer hanging off infrastructure without gating apps.
Follow the hierarchical Kustomization flow: base configurations in k8s/bases/ feed provider overlays in k8s/providers/, which feed cluster overlays in k8s/clusters/.

Files:

  • k8s/bases/infrastructure/controllers/kubescape/helm-release.yaml
k8s/bases/infrastructure/**

📄 CodeRabbit inference engine (AGENTS.md)

k8s/bases/infrastructure/**: Under k8s/bases/infrastructure/, organize resources component-folder-first: a component's HelmRelease/HelmRepository and its own CRs should live together in a folder named after the component unless a split is required.
Split a custom resource into its own plural-Kind folder only when it cannot live with its component, such as for CRD dependency ordering or because it is cluster-scoped/cross-cutting.

Files:

  • k8s/bases/infrastructure/controllers/kubescape/helm-release.yaml
k8s/bases/infrastructure/**/*.{yaml,yml}

📄 CodeRabbit inference engine (AGENTS.md)

k8s/bases/infrastructure/**/*.{yaml,yml}: For component-folder files, name manifests after the resource Kind in kebab-case; if a folder contains multiple resources of the same Kind, qualify filenames with a purpose suffix.
For CR-folder files, omit the folder-implied Kind from the filename and use the verb-purpose.yaml form.
Name Flux Kustomization resources flux-kustomization*.yaml; keep the kustomize build file named exactly kustomization.yaml.

Files:

  • k8s/bases/infrastructure/controllers/kubescape/helm-release.yaml
🧠 Learnings (1)
📚 Learning: 2026-07-01T21:13:36.950Z
Learnt from: devantler
Repo: devantler-tech/platform PR: 2359
File: k8s/bases/apps/actual-budget/helm-release.yaml:62-111
Timestamp: 2026-07-01T21:13:36.950Z
Learning: When reviewing Kustomize/Helm YAML in this repo, keep the base vs provider overlay split: `k8s/bases/apps/**` and `k8s/bases/infrastructure/**` should contain each app’s full, environment-agnostic configuration (including base-level postRenderer Kustomize patches such as deployment strategy, topology spread, probes, and env injection). `k8s/providers/{docker,hetzner}/**` should only add small provider-specific deltas (e.g., `interval`, `persistence.size`) via patch files (like `k8s/providers/<provider>/apps/<app>/patches/helm-release-patch.yaml`). If configuration is identical across providers (e.g., OIDC/OAuth env vars where `${domain}` is resolved per cluster via envsubst), it belongs in the base and must not be duplicated into provider overlays.

Applied to files:

  • k8s/bases/infrastructure/controllers/kubescape/helm-release.yaml
🔇 Additional comments (3)
k8s/bases/infrastructure/controllers/kubescape/helm-release.yaml (3)

145-163: Summary/line-range metadata states the replace op sets mountPath to /home/nonroot/.kubescape/config.json, but the code (Line 176) replaces it with the parent dir /home/nonroot/.kubescape. The code is authoritative here; flagging the discrepancy for awareness.


164-169: LGTM!


170-176: 🎯 Functional Correctness

Keep subPath: config.json here The parent-directory mountPath is the right fix; removing subPath would break the file mount.

			> Likely an incorrect or invalid review comment.

📝 Walkthrough

Walkthrough

Adds a Flux postRenderers Kustomize JSON patch to the kubescape HelmRelease that tests and then overrides the second volume mount's mountPath to /home/nonroot/.kubescape/config.json, working around a chart offline-mode bug in kubescape-operator 1.40.2.

Changes

Kubescape HelmRelease Patch

Layer / File(s) Summary
postRenderers config mount patch
k8s/bases/infrastructure/controllers/kubescape/helm-release.yaml
Adds a Kustomize postRenderers JSON patch that asserts volumeMounts[1].mountPath and replaces it with /home/nonroot/.kubescape/config.json to work around an offline-mode emptyDir mounting bug in kubescape-operator chart 1.40.2.

Estimated code review effort: 2 (Simple) | ~10 minutes

Related PRs: None found.

Suggested labels: kubescape, kubernetes, patch

Suggested reviewers: None determined.

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly summarizes the main change: remounting Kubescape's config.json to keep offline posture scans persisting.
Description check ✅ Passed The description is directly related to the change and explains the offline Kubescape remount fix and its intended effect.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch claude/kubescape-offline-config-mount

Comment @coderabbitai help to get the list of available commands.

@devantler

Copy link
Copy Markdown
Contributor Author

🤖 Generated by the Daily AI Assistant

Closing as a duplicate of #2443, and superseded by the same reasoning that closed #2444.

This PR fixes only the config.json emptyDir/subPath mount (via a Flux postRenderer). That alone is insufficient: with the mount fixed, the scheduled scan still posts Submit=true, so the v4.0.9 scanner attempts the ARMO cloud submit and aborts with a 402 before persisting any WorkloadConfigurationScan — the failure just moves from "config.json is a directory" to the cloud submit. #2443 applies the load-bearing fix — scanV1.keepLocal: true in the scheduler request body — which runs the scan local-only (Submit=false, no cloud call, config.json never touched), mirroring upstream helm-charts#862. Deferring to #2443.

(I re-derived this root cause independently before finding #2443/#2444 already covered it — apologies for the duplicate.)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: ✅ Done

Development

Successfully merging this pull request may close these issues.

1 participant