Skip to content

Document ttl and the drain window for Kafka splitting - #293

Open
dmzmk wants to merge 2 commits into
mainfrom
INT-498/kafka-idle-timeout
Open

Document ttl and the drain window for Kafka splitting#293
dmzmk wants to merge 2 commits into
mainfrom
INT-498/kafka-idle-timeout

Conversation

@dmzmk

@dmzmk dmzmk commented Jul 29, 2026

Copy link
Copy Markdown
Contributor

No description provided.

@dmzmk
dmzmk requested a review from vladrbg July 29, 2026 22:55
@linear-code

linear-code Bot commented Jul 29, 2026

Copy link
Copy Markdown

INT-498

Comment thread docs/sharing-the-cluster/queue-splitting/kafka.md Outdated
Comment thread docs/sharing-the-cluster/queue-splitting/kafka.md
@greptile-apps

greptile-apps Bot commented Jul 29, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR updates the Kafka queue-splitting docs to document a new two-window lifecycle — an idle window (spec.ttl) that keeps the split warm for instant reconnects, followed by a drain window (spec.drainTimeout) that lets the workload finish consuming the backlog before teardown. It also clarifies that there is no cluster-wide equivalent for MirrordSplitConfig and removes the operator.kafkaSplittingDrainTimeout Helm row accordingly.

  • spec.ttl is introduced as a new MirrordSplitConfig field; the hint correctly gates both spec.ttl and the new drain semantics behind operator version x.y.z.
  • The semantics of spec.drainTimeout: unset changed: on earlier operators it meant "unpatch immediately" (same as 0), but on x.y.z+ it means "wait indefinitely." This breaking change is not called out explicitly enough in the hint, which only says spec.drainTimeout "alone controls" the stay-patched window on earlier operators.

Important Files Changed

Filename Overview
docs/sharing-the-cluster/queue-splitting/kafka.md Documents new two-window lifecycle (idle TTL + drain window) for Kafka splitting sessions; the drainTimeout unset semantics changed silently compared to pre-x.y.z operators, which the hint does not make explicit.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[Last splitting session ends] --> B{spec.ttl set and > 0?}
    B -- Yes --> C[Idle window\noperator keeps forwarding\nup to spec.ttl seconds]
    B -- No --> D[Skip idle window]
    C -- New session reconnects --> E[Reuse split instantly\nno restart needed]
    C -- ttl elapses, no reconnect --> F[Stop forwarding new messages]
    D --> F
    F --> G{spec.drainTimeout value?}
    G -- "0" --> H[Unpatch immediately\nmessages in temp topic lost]
    G -- "N seconds" --> I[Drain window\nwait up to N seconds\nends early if topic empty]
    G -- "Unset x.y.z+: no cap" --> J[Wait indefinitely\nuntil backlog consumed]
    G -- "Unset pre-x.y.z: same as 0" --> H
    I -- New session reconnects --> K[Reuse split\nresume forwarding]
    J -- New session reconnects --> K
    I -- drained or timeout --> L[Delete temp queues\nUnpatch workload\nrestart onto original topic]
    J -- Fully drained --> L
    H --> L
Loading

Reviews (3): Last reviewed commit: "Rename idelTimeout to ttl" | Re-trigger Greptile

@dmzmk dmzmk changed the title Document idleTimeout and the drain window for Kafka splitting Document ttl and the drain window for Kafka splitting Jul 30, 2026
@dmzmk
dmzmk force-pushed the INT-498/kafka-idle-timeout branch from ebdafc0 to 77889e4 Compare July 31, 2026 15:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant