You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Aggregated apiservers that delegate to milo-apiserver (IPAM today; Activity and others on the same pattern) can't use default client behavior. client-go's WatchListClient (now on by default) initializes informers with a streaming initial list (watch=true&sendInitialEvents=true&resourceVersionMatch=NotOlderThan). milo rejects these — e.g. 504 Timeout: Too large resource version: 1280714733, current: 1280614472 — so the delegating apiserver's informers (FlowSchema, PriorityLevelConfiguration, auth configmaps) never sync, /readyz hangs, and that service's API stays offline.
Impact
Any service delegating auth/flow-control to milo can silently fail readiness as WatchListClient becomes the client default — taking its API down on an otherwise-routine upgrade.
IPAM hit this and is offline until a per-service workaround ships. Each delegating service would otherwise need the same opt-out, and all of them lose the efficiency streaming list is meant to provide.
Ask
milo-apiserver should serve streaming watch-list consistently — honor sendInitialEvents with correct resourceVersion/NotOlderThan semantics and emit the initial-events-end bookmark — so delegating apiservers work with default client settings and need no per-service opt-out.
Reproduce: kubectl --kubeconfig <milo> get --raw "/apis/flowcontrol.apiserver.k8s.io/v1/flowschemas?watch=true&sendInitialEvents=true&resourceVersionMatch=NotOlderThan&allowWatchBookmarks=true" returns a 504 ResourceVersionTooLarge instead of the initial list + bookmark.
Problem
Aggregated apiservers that delegate to milo-apiserver (IPAM today; Activity and others on the same pattern) can't use default client behavior. client-go's
WatchListClient(now on by default) initializes informers with a streaming initial list (watch=true&sendInitialEvents=true&resourceVersionMatch=NotOlderThan). milo rejects these — e.g.504 Timeout: Too large resource version: 1280714733, current: 1280614472— so the delegating apiserver's informers (FlowSchema, PriorityLevelConfiguration, auth configmaps) never sync,/readyzhangs, and that service's API stays offline.Impact
WatchListClientbecomes the client default — taking its API down on an otherwise-routine upgrade.Ask
milo-apiserver should serve streaming watch-list consistently — honor
sendInitialEventswith correctresourceVersion/NotOlderThansemantics and emit the initial-events-end bookmark — so delegating apiservers work with default client settings and need no per-service opt-out.Context
KUBE_FEATURE_WatchListClient=falseon the delegating apiserver (datum-cloud/infra#2871; fix: stop disabling APF so ipam-apiserver readyz passes ipam#38 enabled APF so the informers run).kubectl --kubeconfig <milo> get --raw "/apis/flowcontrol.apiserver.k8s.io/v1/flowschemas?watch=true&sendInitialEvents=true&resourceVersionMatch=NotOlderThan&allowWatchBookmarks=true"returns a 504ResourceVersionTooLargeinstead of the initial list + bookmark.