Skip to content

Commit 33e99fd

Browse files
committed
KEP-5966: Use ListOptions.Stream instead of separate ListStream method
1 parent da9f765 commit 33e99fd

1 file changed

Lines changed: 30 additions & 11 deletions

File tree

  • keps/sig-etcd/5966-etcd-range-stream

keps/sig-etcd/5966-etcd-range-stream/README.md

Lines changed: 30 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -211,21 +211,40 @@ without assembling the full list response in memory.
211211
The watch cache also falls back to direct etcd reads when the cache is
212212
stale or snapshots are unavailable after restart. These fallbacks are
213213
rare (<1% of requests) but costly. For the fallback path, a new
214-
`ListStream` method is added to the etcd `kubernetes.Interface`:
214+
`Stream` field is added to the existing `ListOptions` on the etcd
215+
`kubernetes.Interface`:
215216

216217
```go
217-
ListStream(ctx context.Context, prefix string, opts ListStreamOptions, cb func(ListStreamResponse) error) error
218+
type ListOptions struct {
219+
Revision int64
220+
Limit int64
221+
Continue string
222+
Stream bool // Use RangeStream instead of unary Range
223+
}
218224
```
219225

220-
The etcd3 store's `GetList` calls `ListStream` instead of paginated
221-
`List`, decoding and filtering each chunk inline. When the feature gate
222-
is disabled or the server returns `Unimplemented`, the store falls back
223-
to paginated `List`. `ListStream` is a separate method from `List`
224-
because the calling pattern is fundamentally different: with paginated
225-
`List`, clients must set a conservative limit (e.g., 10k) and manage
226-
pagination to avoid etcd memory spikes. With `ListStream`, clients
227-
request the full range and the server handles chunking internally.
228-
The `storage.Interface` is unchanged.
226+
When `Stream` is set, `List` uses `GetStream()` internally and
227+
reassembles the streamed chunks into a `ListResponse`. The `Revision`,
228+
`Limit`, and `Continue` fields remain functional when streaming is
229+
enabled. `Revision` pins the snapshot, `Limit` caps the total keys
230+
returned, and `Continue` resumes from a key. In practice, callers using
231+
`Stream` will typically omit `Limit` and `Continue` since the server
232+
handles chunking internally.
233+
234+
If the server does not support `RangeStream` (returns `Unimplemented`),
235+
`List` returns the error to the caller rather than silently falling back
236+
to a unary Range. A transparent fallback is unsafe because streamed
237+
`List` calls are typically issued without a `Limit`. Falling back to a
238+
unary Range without a limit would attempt to return the entire key space
239+
in a single response, which can exceed the gRPC response size limit and
240+
fail the request entirely. Instead, callers are responsible for detecting
241+
the error and falling back to client-side pagination with an appropriate
242+
`Limit` and `Continue` token.
243+
244+
The etcd3 store's `GetList` calls `List` with `Stream: true` instead of
245+
managing pagination externally. When the server returns `Unimplemented`
246+
or the feature gate is disabled, the store falls back to paginated
247+
`List` with a conservative limit. The `storage.Interface` is unchanged.
229248

230249
### Upgrade / Downgrade Strategy
231250

0 commit comments

Comments
 (0)