feat(k8score): idle reaper for namespace-scoped dynamic informers#772
Open
nadaverell wants to merge 1 commit into
Open
feat(k8score): idle reaper for namespace-scoped dynamic informers#772nadaverell wants to merge 1 commit into
nadaverell wants to merge 1 commit into
Conversation
b073af6 to
979ddff
Compare
e5f3876 to
fd252dc
Compare
979ddff to
789b641
Compare
fd252dc to
a06adb5
Compare
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit a06adb5. Configure here.
7c35d55 to
bc65871
Compare
a06adb5 to
c0fb7a0
Compare
bc65871 to
ee22b11
Compare
c0fb7a0 to
a085db6
Compare
ee22b11 to
12ad04e
Compare
Per-namespace dynamic informers (the B2.1 model) can accumulate over the life of a long-running, namespace-restricted deployment as different namespaces get watched. Add an idle reaper that stops a namespace-scoped informer after InformerIdleTTL (default 30m) without a read, re-creating it transparently on the next access. Liveness is the read path itself: List/Get/Count/EnsureWatching stamp the entry's lastAccess. The frontend re-lists open views every 60-120s (React Query refetchInterval), and SSE topology rebuilds also list — both flow through that stamp — so an open view's informer is never reaped out from under it, and an explicit SSE lease/refcount is unnecessary. Cluster-wide informers (one per GVR, serve every namespace) are exempt; a negative TTL disables reaping. To make eviction safe, informers are now constructed directly via NewFilteredDynamicInformer instead of through a shared factory: a factory caches one informer per GVR and would hand back the stopped instance after a reap (a SharedInformer cannot be re-Run). Standalone informers, each under its own cancelable context, can be stopped and re-created freely. This also removes the per-namespace factory map and simplifies Stop(). Builds on #770.
a085db6 to
721654d
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

Summary
Follow-up to #770 (stacked on it — base is the B2.1 branch; retarget to
mainonce #770 merges). B2.1 lets a GVR have many namespace-scoped informers; over the life of a long-running, namespace-restricted deployment those can accumulate as new namespaces get watched. This adds an idle reaper that stops a namespace-scoped informer afterInformerIdleTTL(default 30m) without a read, re-creating it transparently on next access.Liveness = the read path (no SSE lease needed)
Every read —
List/Get/Count/ theEnsureWatchingconfirm-step — stamps the entry'slastAccess(atomic, off the hot lock). The frontend re-lists open views every 60–120s (ResourcesView.tsx/GitOpsView.tsxReact QueryrefetchInterval), and SSE topology rebuilds alsoList— both flow through that stamp. So an open view keeps its informer warm and can't be reaped out from under a live stream, which is the concern that would otherwise motivate an SSE subscriber lease. The TTL (30m) is comfortably above the poll interval, so the lease/refcount coupling is unnecessary; I went with the simpler read-driven design deliberately.InformerIdleTTLdisables reaping; zero selects the default.Why informers are now built directly (not via a factory)
A
DynamicSharedInformerFactorycaches one informer per GVR, so after the reaper stops an informer the factory would hand back the stopped instance on re-request — and aSharedInformercan't be re-Run. Switching toNewFilteredDynamicInformergives standalone informers, each under its own cancelable context, that can be stopped and re-created freely. This also let me drop the per-namespace factory map and simplifyStop()(closingstopChcancels every informer's context; no factory shutdown).Tests
pkg/k8score: idle namespace-scoped informer is reaped while a freshly-read one is kept and re-created on next access; cluster-wide informers are exempt even when ancient; negative TTL disables reaping. Reaper is driven with an explicitnowfor deterministic, timing-free tests.go build ./...,go test ./internal/server ./internal/k8s ./pkg/k8scoreall green.Note
Default behavior change is limited to namespace-restricted deployments (reaper trims idle per-ns informers there). Cluster-wide deployments see no behavioral change.
Note
Medium Risk
Changes core dynamic cache lifecycle and informer construction; mis-tuned TTL or missed
touch()paths could stop watches for idle namespaces (transparent re-sync on next access). Cluster-wide RBAC deployments are explicitly exempt from reaping.Overview
Namespace-restricted dynamic CRD watches can accumulate one informer per namespace; this PR adds an idle reaper that stops namespace-scoped informers after
InformerIdleTTL(default 30m without a read) and re-creates them on the next List/Get. Cluster-wide informers are never reaped; a negative TTL turns reaping off.Liveness is read-driven:
lastAccessis updated atomically on list/get/count paths (includingEnsureWatchingconfirm andListWatchedfor background scanners), so active UI polling keeps informers warm without an SSE lease.Implementation shift: informers are built with
NewFilteredDynamicInformerinstead of a sharedDynamicSharedInformerFactory, so reaped/stopped informers are not reused from a factory cache.Stop()no longer shuts down factories—per-informer contexts cancel viastopCh. Sync completion only markssyncedif the entry is still the one that started syncing (avoids races after a reap/re-create).DynamicCacheConfig.InformerIdleTTLis the new knob (zero = default, negative = disabled). Tests cover idle vs fresh namespaces, cluster-wide exemption, disable flag, and transparent re-create after reap.Reviewed by Cursor Bugbot for commit 721654d. Bugbot is set up for automated code reviews on this repo. Configure here.