Skip to content

Commit a46c964

Browse files
committed
KEP-5966: Use ListStream on kubernetes.Interface instead of ListOptions.Stream
1 parent 33e99fd commit a46c964

1 file changed

Lines changed: 15 additions & 42 deletions

File tree

  • keps/sig-etcd/5966-etcd-range-stream

keps/sig-etcd/5966-etcd-range-stream/README.md

Lines changed: 15 additions & 42 deletions
Original file line numberDiff line numberDiff line change
@@ -200,51 +200,24 @@ The following components are modified:
200200
RangeStream is gated behind a `RangeStream` feature gate in kube-apiserver
201201
(Alpha in 1.37, default disabled).
202202

203-
The primary integration point is the watch cache initialization path. When
204-
the feature gate is enabled, the watch cache `sync()` uses
205-
`KV.GetStream()` on the lower-level `clientv3.KV` interface directly.
206-
This eliminates the client-side pagination loop that advances the range
207-
key, pins revisions, and checks for more results. Each chunk's key-value
203+
A new `ListStream` method is added to the etcd `kubernetes.Interface`
204+
as a thin wrapper around the etcd client's `KV.GetStream()`. This
205+
returns a channel of chunks so callers receive key-value pairs as they
206+
arrive from the server, keeping the `kubernetes.Interface` abstraction
207+
consistent rather than reaching into `client.KV` directly.
208+
209+
The primary integration point is the watch cache initialization path.
210+
When the feature gate is enabled, the watch cache `sync()` uses
211+
`ListStream` to receive chunks incrementally. Each chunk's key-value
208212
pairs are converted to synthetic "created" events and queued inline
209213
without assembling the full list response in memory.
210214

211-
The watch cache also falls back to direct etcd reads when the cache is
212-
stale or snapshots are unavailable after restart. These fallbacks are
213-
rare (<1% of requests) but costly. For the fallback path, a new
214-
`Stream` field is added to the existing `ListOptions` on the etcd
215-
`kubernetes.Interface`:
216-
217-
```go
218-
type ListOptions struct {
219-
Revision int64
220-
Limit int64
221-
Continue string
222-
Stream bool // Use RangeStream instead of unary Range
223-
}
224-
```
225-
226-
When `Stream` is set, `List` uses `GetStream()` internally and
227-
reassembles the streamed chunks into a `ListResponse`. The `Revision`,
228-
`Limit`, and `Continue` fields remain functional when streaming is
229-
enabled. `Revision` pins the snapshot, `Limit` caps the total keys
230-
returned, and `Continue` resumes from a key. In practice, callers using
231-
`Stream` will typically omit `Limit` and `Continue` since the server
232-
handles chunking internally.
233-
234-
If the server does not support `RangeStream` (returns `Unimplemented`),
235-
`List` returns the error to the caller rather than silently falling back
236-
to a unary Range. A transparent fallback is unsafe because streamed
237-
`List` calls are typically issued without a `Limit`. Falling back to a
238-
unary Range without a limit would attempt to return the entire key space
239-
in a single response, which can exceed the gRPC response size limit and
240-
fail the request entirely. Instead, callers are responsible for detecting
241-
the error and falling back to client-side pagination with an appropriate
242-
`Limit` and `Continue` token.
243-
244-
The etcd3 store's `GetList` calls `List` with `Stream: true` instead of
245-
managing pagination externally. When the server returns `Unimplemented`
246-
or the feature gate is disabled, the store falls back to paginated
247-
`List` with a conservative limit. The `storage.Interface` is unchanged.
215+
For direct `GetList` calls (e.g., from controllers or when WatchList is
216+
disabled), the store consumes `ListStream` and decodes each chunk's
217+
key-value pairs inline as they arrive, overlapping network I/O with
218+
decode. When the server returns `Unimplemented` or the feature gate is
219+
disabled, the store falls back to paginated `List` with a conservative
220+
limit. The `storage.Interface` is unchanged.
248221

249222
### Upgrade / Downgrade Strategy
250223

0 commit comments

Comments
 (0)