-
Notifications
You must be signed in to change notification settings - Fork 232
Add nats-client package #732
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
caspervonb
wants to merge
114
commits into
main
Choose a base branch
from
add-nats-client-package
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+10,598
−533
Open
Changes from 42 commits
Commits
Show all changes
114 commits
Select commit
Hold shift + click to select a range
2650f9e
Add nats-client package
caspervonb 850486d
Use named tuples for parsing
caspervonb dd075df
Add `__version__`
caspervonb 78912e1
Make operation parsing case-insensitive in message protocol
caspervonb 484a5e2
Raise ParseError for header lines missing colon
caspervonb d93f72e
Improve error reporting for unknown operation using repr
caspervonb 4de3c85
Refactor protocol message parsing into more readable dispatch
caspervonb 7e85715
Handle IPv6 addresses correctly in server URLs
caspervonb 56b38de
Add no_randomize option to connect and test sequential cluster reconnect
caspervonb 91183fd
Refactor Connection to use Protocol instead of ABC
caspervonb e1f5a53
Add explicit attribute declarations to client classes
caspervonb 22b3530
Remove redundant comments and add Reader protocol to parser module
caspervonb 39358ca
Switch build system from Hatchling to Setuptools
caspervonb 2693ab8
Add nats-client to workspace members in pyproject.toml
caspervonb fff225b
Add dev dependencies for nats-client and update uv.lock
caspervonb 82f9351
Add asdict method to Headers and use in Client
caspervonb 451219b
Improve cluster reconnect test to shut down connected server
caspervonb e1c4666
Parametrize cluster reconnect test by cluster size
caspervonb a6327a9
Prevent concurrent reconnection attempts with asyncio.Lock
caspervonb 138ca95
Require Python 3.11+ and remove dependency on typing extensions
caspervonb f17f594
Refactor message parsing functions and update tests
caspervonb 7c1c919
Refactor reconnection logic for clarity and remove redundant checks
caspervonb 49e99ef
Handle exceptions when closing connection in Client
caspervonb c6181ce
Close connection only if it is currently connected
caspervonb 044ee5b
Refactor reconnect attempts variable naming for clarity
caspervonb 7e07832
Refactor server pool management and remove _collect_servers
caspervonb 5e7aebd
Log and handle CancelledError and TimeoutError on connect
caspervonb 9686354
Fix exception handling to use asyncio.TimeoutError
caspervonb 59ebbc8
Avoid duplicate servers when updating server pool
caspervonb 23f854b
Set reconnect_time_wait to 0.0 in cluster reconnect test
caspervonb a6d91ee
Reduce client connection timeout in cluster reconnect test
caspervonb 5f06a09
Add reconnect_timeout parameter to connect function
caspervonb 13969e5
Rename reconnect_attempts to reconnect_max_attempts
caspervonb a7a1bdf
Fix cluster reconnect test to match server by port only
caspervonb 171ad83
Remove 2-node cluster test from cluster reconnect parametrize
caspervonb 345382f
Format with ruff
caspervonb 51553bb
Fix lints
caspervonb dce2956
Fix more lints
caspervonb 6c78343
Format again
caspervonb 8819beb
Remove deprecation note from _subscribe docstring
caspervonb e04016c
Add type annotations to StatusError attributes
caspervonb 327dc56
Remove callback parameter from Client.subscribe
caspervonb 25aeb39
Update nats-client/src/nats/client/__init__.py
caspervonb b6dc5a8
Update nats-client/src/nats/client/__init__.py
caspervonb 9558633
Add test for iterator stopping on subscription close
caspervonb 7aa664a
Inline control line parsing and add tests
caspervonb 986b1b4
Fix import ordering
caspervonb 9babd5c
Remove BadRequestError and simplify status handling
caspervonb bf62bfc
Use _pending_queue and remove queue property
caspervonb 53d582a
Remove public sid property from subscription
caspervonb b65a86c
Remove the close alias for Subscription.unsubscribe.
caspervonb a5ef0b3
Use shutdown (minimum 3.13) on subscription message queues
caspervonb 86166ce
Add `inbox_prefix` option and `Client.new_inbox()`
caspervonb 7d8a2e1
Add ping configuration to Client and connect
caspervonb e1bc256
Add Subscription.drain and refine unsubscribe
caspervonb 6b3edd9
Format
caspervonb 777a0e0
Add --client option to bench script
caspervonb 39632b3
Remove Status.is_error and message status helpers
caspervonb bfbc456
Add inline docstrings to protocol TypedDicts
caspervonb b4c0851
Add context to subscription and error logs
caspervonb b76f34d
Rename _headers to _data in Headers
caspervonb 5b53c0e
Move Status tests into test_message
caspervonb 873acd1
s/tho/though
caspervonb 6e4bcf4
Remove benchmark results from README
caspervonb fafb740
Add Private :: Do Not Upload classifier
caspervonb aa540eb
Merge branch 'main' into add-nats-client-package
caspervonb ca90b10
Merge branch 'main' into add-nats-client-package
caspervonb 3d1a708
Set tls_required to False in ConnectInfo
caspervonb f40a4c6
Add tls_required to ConnectInfo
caspervonb 70b7cfa
Add nats-client/src to ty environment root
caspervonb dec6cb3
Fix typing and suppress type checker errors in tools/bench.py
caspervonb 01ba7e7
Improve type checking and runtime guards
caspervonb 70a72a3
Remove TYPE_CHECKING conditional import
caspervonb e10594d
Remove host/port from TcpConnection
caspervonb d1ec510
Add set, delete and append methods to Headers
caspervonb 75ce2f9
Add TCP connection tests using echo server
caspervonb 71eb800
Add test for server-initiated PING handling
caspervonb a7a99eb
Make latency tracking optional in bench tool
caspervonb 4c259d8
Add token authentication and handshake verification
caspervonb 541e28a
Add user/password authentication support
caspervonb 553090b
Add nkey authentication support
caspervonb 2292614
Rename auth_token to token
caspervonb dc13b84
Add reconnection under load tests
caspervonb 5b02ffe
Add subscription concurrency tests
caspervonb 614060c
Add examples
caspervonb b4b1b3c
Add smoke tests for examples
caspervonb a5dc995
Restore nats directory
caspervonb bd8a5f0
Add benches for nats-client
caspervonb 581828f
Add client drain support and tests
caspervonb 7c0d43c
Add ClientStatistics and tracking counters
caspervonb 0bd1b38
Add bounded MessageQueue and slow consumer handling
caspervonb facfd98
Add tests for slow consumer and pending limits
caspervonb a182352
Move MessageQueue logic into Subscription
caspervonb b4e3799
Access subscription.pending as property
caspervonb b2b9bfc
Record dropped messages and bytes on subscriptions
caspervonb c8543d1
Implement TLS
caspervonb e129beb
Fix formatting
caspervonb 572a1db
Rename `Message.reply_to` to `reply`
caspervonb 8b4ddda
Rename subscription `queue_group` to `queue`
caspervonb c3e1153
Reduce sleeps in tests, use flush and events
caspervonb a9acece
Add no_echo option to `connect`
caspervonb bd27417
Add `Subscription.messages` for API compatability for nats.aio
caspervonb 3fcf82a
Remove redundant comments
caspervonb bca8360
Be consistent with msg/msgs -> message/messages
caspervonb 00d2ab2
Optimize encode_pub and encode_hpub
caspervonb e53424c
Limit control line split and drop op uppercasing
caspervonb 6dec236
Enable dataclass slots for Status and Message
caspervonb a48b8fa
Read payload and CRLF in one readexactly call
caspervonb 8ccd800
Bump minimum pytest-benchmark version
caspervonb f421dcc
Allow subjects to be passed as bytes or strings
caspervonb b8dbc1a
Allow subscribe to take str | byte as subject
caspervonb e3ca6e3
Harden flaky no_echo test
caspervonb fe9fdd3
Allow queue to be bytes | str for consistency
caspervonb d247990
Add guards around hot-path logging
caspervonb File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,82 @@ | ||
| # NATS Client | ||
|
|
||
| A Python client for the NATS messaging system. | ||
|
|
||
| ## Features | ||
|
|
||
| - Support for publish/subscribe | ||
| - Support for request/reply | ||
| - Support for queue groups | ||
| - Support for multi-value message headers | ||
|
|
||
| ## Installation | ||
|
|
||
| ```bash | ||
| pip install nats-client | ||
| ``` | ||
|
|
||
| ## Usage | ||
|
|
||
| ```python | ||
| import asyncio | ||
| from nats.client import connect | ||
|
|
||
| async def main(): | ||
| client = await connect("nats://localhost:4222") | ||
|
|
||
| # Subscribe | ||
| async with await client.subscribe("foo") as subscription: | ||
| # Publish | ||
| await client.publish("foo", "Hello World!") | ||
|
|
||
| # Receive message | ||
| message = await subscription.next() | ||
| print(f"Received: {message.data}") | ||
|
|
||
| await client.close() | ||
|
|
||
| if __name__ == "__main__": | ||
| asyncio.run(main()) | ||
| ``` | ||
|
|
||
| ## 🚀 Performance | ||
|
|
||
| This client implementation delivers significant performance improvements over the nats.aio client, particularly for high-frequency, small message workloads. | ||
|
|
||
| Do note tho, it is not as feature complete at this point in time. | ||
|
|
||
| | Message Size | nats.py (python3) | nats.py (pypy3) | experimental-nats.py (python3) | experimental-nats (pypy3) | Performance Gain | | ||
caspervonb marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| |--------------|-------------------|-----------------|--------------------------------|---------------------------|------------------| | ||
| | 1B | 127,411 | 153,009 | 1,522,673 | **5,376,113** | **35.1x** 🚀 | | ||
| | 2B | 136,485 | 148,981 | 1,544,513 | **5,396,347** | **36.2x** 🚀 | | ||
| | 4B | 131,630 | 149,297 | 1,548,191 | **5,356,600** | **35.9x** 🚀 | | ||
| | 8B | 138,229 | 141,117 | 1,530,825 | **5,307,400** | **37.6x** 🚀 | | ||
| | 16B | 140,874 | 149,826 | 1,539,244 | **5,211,168** | **34.8x** 🚀 | | ||
| | 32B | 141,427 | 146,670 | 1,515,068 | **5,115,238** | **34.9x** 🚀 | | ||
| | 64B | 145,257 | 153,542 | 1,505,724 | **5,339,967** | **34.8x** 🚀 | | ||
| | 128B | 163,181 | 164,723 | 1,479,100 | **4,923,321** | **29.9x** 🔥 | | ||
| | 256B | 145,824 | 161,017 | 1,452,996 | **4,130,165** | **25.7x** 🔥 | | ||
| | 512B | 243,641 | 277,321 | 1,297,250 | **3,430,092** | **12.4x** ⚡ | | ||
| | 1K | 738,895 | 802,283 | 1,253,102 | **2,374,747** | **3.0x** ⚡ | | ||
| | 2K | 696,945 | 736,925 | 1,060,123 | **1,381,177** | **1.9x** ✨ | | ||
| | 4K | 577,335 | 625,935 | 798,797 | **814,393** | **1.3x** ✨ | | ||
| | 8K | 414,077 | 463,383 | 532,429 | 450,211 | 0.97x | | ||
| | 16K | 266,104 | 309,680 | 345,651 | 228,815 | 0.74x | | ||
| | 32K | 102,460 | 128,852 | 166,028 | 125,662 | 0.98x | | ||
| | 64K | 55,208 | 63,563 | 74,359 | 56,804 | 0.89x | | ||
|
|
||
| ### Key Performance Insights | ||
|
|
||
| **🎯 Sweet Spot: Small to Medium Messages** | ||
| - **35-37x faster** for tiny messages (1B-64B) | ||
| - **25-30x faster** for small messages (128B-256B) | ||
| - **12x faster** for medium messages (512B) | ||
|
|
||
| ### Benchmark Environment | ||
|
|
||
| - **CPU**: Apple M3 Max | ||
| - **Memory**: 36 GB | ||
| - **Python**: 3.x | ||
| - **PyPy**: 3.x | ||
|
|
||
| > **Note**: Benchmarks may vary based on your specific hardware, network conditions, and NATS server configuration. We recommend running your own benchmarks for production workloads. | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,50 @@ | ||
| [build-system] | ||
| requires = ["setuptools>=61.0"] | ||
| build-backend = "setuptools.build_meta" | ||
|
|
||
| [project] | ||
| name = "nats-client" | ||
| version = "0.0.0" | ||
| description = "NATS client implementation in Python" | ||
| readme = "README.md" | ||
| requires-python = ">=3.11" | ||
| license = "MIT" | ||
| keywords = ["nats", "messaging", "client"] | ||
| authors = [ | ||
| { name = "Casper Beyer", email = "[email protected]" }, | ||
| ] | ||
| classifiers = [ | ||
| "Development Status :: 4 - Beta", | ||
| "Programming Language :: Python", | ||
| "Programming Language :: Python :: 3.11", | ||
| "Programming Language :: Python :: 3.12", | ||
| "Programming Language :: Python :: 3.13", | ||
| "Programming Language :: Python :: Implementation :: CPython", | ||
| "Programming Language :: Python :: Implementation :: PyPy", | ||
| ] | ||
| dependencies = [] | ||
|
|
||
| [project.urls] | ||
| Documentation = "https://github.com/nats-io/nats.py" | ||
| Issues = "https://github.com/nats-io/nats.py/issues" | ||
| Source = "https://github.com/nats-io/nats.py" | ||
|
|
||
| [tool.setuptools.packages.find] | ||
| where = ["src"] | ||
|
|
||
| [tool.uv] | ||
| dev-dependencies = [ | ||
| "nats-server", | ||
| "pytest>=7.0.0", | ||
| "pytest-asyncio>=0.21.0", | ||
| "pytest-cov>=7.0.0", | ||
| "pytest-xdist>=3.0.0", | ||
| "pytest-benchmark", | ||
| ] | ||
|
|
||
| [tool.uv.sources] | ||
| nats-server = { workspace = true } | ||
|
|
||
| [tool.pytest.ini_options] | ||
| asyncio_default_fixture_loop_scope = "function" | ||
| asyncio_mode = "auto" |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
tho->though