Use Case
When running Telegraf with the NATS Consumer input plugin against JetStream streams, restarting Telegraf causes the ephemeral push consumer to be destroyed. On restart, a new ephemeral consumer is created and — depending on the deliver policy — messages are either replayed from the beginning of the stream or missed entirely. This makes Telegraf unsuitable for production JetStream workloads where at-least-once delivery and restart resilience are required.
Durable consumers persist on the NATS server and track their delivery position. When Telegraf reconnects, the durable consumer resumes from where it left off — no replay, no message loss.
Expected behavior
The NATS Consumer plugin should support configuring a durable consumer name and related JetStream consumer options (deliver policy, ack wait, max deliver, filter subjects). When a durable name is set, the consumer should survive Telegraf restarts and resume processing from the last acknowledged position. On shutdown, the subscription should be drained (not unsubscribed) to preserve the consumer on the server.
Actual behavior
The plugin only supports ephemeral push consumers. There is no way to configure a durable consumer name, deliver policy, or other JetStream consumer options. On shutdown the subscription is unsubscribed, destroying the ephemeral consumer. Restarting Telegraf creates a fresh consumer with no knowledge of previously delivered messages.
Additional info
A PR implementing this feature is available: #18720
Use Case
When running Telegraf with the NATS Consumer input plugin against JetStream streams, restarting Telegraf causes the ephemeral push consumer to be destroyed. On restart, a new ephemeral consumer is created and — depending on the deliver policy — messages are either replayed from the beginning of the stream or missed entirely. This makes Telegraf unsuitable for production JetStream workloads where at-least-once delivery and restart resilience are required.
Durable consumers persist on the NATS server and track their delivery position. When Telegraf reconnects, the durable consumer resumes from where it left off — no replay, no message loss.
Expected behavior
The NATS Consumer plugin should support configuring a durable consumer name and related JetStream consumer options (deliver policy, ack wait, max deliver, filter subjects). When a durable name is set, the consumer should survive Telegraf restarts and resume processing from the last acknowledged position. On shutdown, the subscription should be drained (not unsubscribed) to preserve the consumer on the server.
Actual behavior
The plugin only supports ephemeral push consumers. There is no way to configure a durable consumer name, deliver policy, or other JetStream consumer options. On shutdown the subscription is unsubscribed, destroying the ephemeral consumer. Restarting Telegraf creates a fresh consumer with no knowledge of previously delivered messages.
Additional info
A PR implementing this feature is available: #18720