Skip to content

Add support for NATS JetStream as a transport#2299

Open
joeriddles wants to merge 19 commits into
celery:mainfrom
joeriddles:nats
Open

Add support for NATS JetStream as a transport#2299
joeriddles wants to merge 19 commits into
celery:mainfrom
joeriddles:nats

Conversation

@joeriddles

@joeriddles joeriddles commented May 16, 2025

Copy link
Copy Markdown

closes #2103

Implement support for NATS using its built-in persistence layer JetStream.

Example

You can test using either:

  • Installing the NATS server locally and running it with JetStream enabled: nats-server --js
  • Using the demo NATS server at demo.nats.io

The transport can be test using examples/nats_receive.py and examples/nats_send.py:

  1. Start the local NATS server (optional)
  2. Run python -m examples.nats_receive (append --demo if using the demo server)
  3. In another window, run python -m examples.nats_send (append --demo if using the demo server)

In the receive window, you should see something like:

Received message: 'hello world'
  properties:
{   'body_encoding': 'base64',
    'delivery_info': {'exchange': 'exchange', 'routing_key': 'messages'},
    'delivery_mode': 2,
    'delivery_tag': 'fa01f1a9-f666-46a0-af71-35e5164e1ad9',
    'priority': 0}
  delivery_info:
{'exchange': 'exchange', 'routing_key': 'messages'}

@joeriddles joeriddles mentioned this pull request May 16, 2025
@auvipy auvipy requested review from auvipy and Copilot May 17, 2025 08:16

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

Adds initial support for NATS JetStream as a Kombu transport, including dependency, transport registration, and usage examples.

  • Adds nats-py extra requirement
  • Registers the nats transport in Kombu’s transport registry
  • Provides example scripts for sending and receiving messages via NATS JetStream

Reviewed Changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 2 comments.

File Description
requirements/extras/nats.txt Pins the nats-py[nkeys] dependency
kombu/transport/init.py Registers 'nats' transport for JetStream
examples/nats_send.py Example for publishing messages to JetStream
examples/nats_receive.py Example for consuming messages from JetStream
Comments suppressed due to low confidence (1)

kombu/transport/init.py:48

  • [nitpick] The transport key 'nats' is generic and could conflict with other NATS-based transports; consider renaming it to 'nats_jetstream' to clearly distinguish this implementation.
'nats': 'kombu.transport.nats_jetstream:Transport',

Comment thread requirements/extras/nats.txt Outdated
Comment thread examples/nats_send.py

@Nusnus Nusnus left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please ping me when the PR is ready for review. Thank you for working on this!

@Nusnus Nusnus force-pushed the nats branch 2 times, most recently from a263b01 to f2df5f1 Compare May 31, 2025 15:50
@joeriddles joeriddles marked this pull request as ready for review June 11, 2025 02:49
@joeriddles

Copy link
Copy Markdown
Author

Please ping me when the PR is ready for review. Thank you for working on this!

@Nusnus ready for review!

Comment thread tox.ini Outdated
@auvipy auvipy added this to the 5.7.0 milestone Jun 17, 2025
@joeriddles

Copy link
Copy Markdown
Author

@Nusnus @auvipy I fixed the merge conflicts and rebased on main

Comment thread t/unit/transport/test_azureservicebus.py Outdated
@auvipy

auvipy commented Oct 23, 2025

Copy link
Copy Markdown
Member

tried to fix the merge conflicts, but feel free to improve it

@joeriddles

Copy link
Copy Markdown
Author

Thanks @auvipy, I fixed tox.ini but otherwise the merge looks good

@maxyloon

Copy link
Copy Markdown

Just wanted to spread the love. So looking forward for this integration!

@sysblade

sysblade commented Dec 9, 2025

Copy link
Copy Markdown

Same we are looking to use this, hopefully this will be merged soon ?
Let me know if some help is needed.
Thanks

@codecov

codecov Bot commented Dec 16, 2025

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 0% with 255 lines in your changes missing coverage. Please review.
✅ Project coverage is 79.12%. Comparing base (a12f3a9) to head (c61a8e3).
⚠️ Report is 97 commits behind head on main.

Files with missing lines Patch % Lines
kombu/transport/nats.py 0.00% 255 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #2299      +/-   ##
==========================================
- Coverage   81.19%   79.12%   -2.08%     
==========================================
  Files          77       78       +1     
  Lines        9738     9993     +255     
  Branches     1098     1120      +22     
==========================================
  Hits         7907     7907              
- Misses       1625     1880     +255     
  Partials      206      206              

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

@joeriddles

Copy link
Copy Markdown
Author

@Nusnus I'll see if I can get the codecov CI passing.. I did add tests so not sure off hand why it's failing.

emattiza pushed a commit to emattiza/kombu that referenced this pull request Feb 2, 2026
emattiza pushed a commit to emattiza/kombu that referenced this pull request Feb 2, 2026
@lgblkb

lgblkb commented Feb 25, 2026

Copy link
Copy Markdown

Guys you are amazing!

@auvipy auvipy left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please fix the merge conflicts

@maxyloon

maxyloon commented May 9, 2026

Copy link
Copy Markdown

Was missing unit tests. Added unit tests, some docs and configurable stream/consumer name prefixes in #2541

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

NATS support

7 participants