Implement ConcurrencyLimitRateLimiter strategy#363
Open
dimartiro wants to merge 2 commits into
Open
Conversation
Reintroduces the concurrency limit strategy on top of the current storage API. Unlike the window based strategies it bounds the number of in-flight requests: `hit` acquires slots and `release` hands them back, with a safety expiry so leaked slots are eventually reclaimed. - Add `ConcurrencyLimitSupport` capability mixin (sync + async) exposing `decr`; wire it into the in-memory storage which already implements it. - Implement `ConcurrencyLimitRateLimiter` for both `limits.strategies` and `limits.aio.strategies`, registered as the `concurrency-limit` strategy. The limiter requires a storage supporting `decr` and raises `NotImplementedError` otherwise (mirrors the moving/sliding window checks). - Acquire-then-rollback in `hit` keeps the counter accurate under contention and never counts rejected hits. - Add sync + async test suites exercising acquire/release, multi-cost, rejection without consumption, release flooring at zero, and clear. - Document the strategy in the README, strategies guide and API reference.
dimartiro
force-pushed
the
revive/concurrency-limiter
branch
from
004d0fd to
cbffd62
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This revives #233, which was approved by reviewers but closed for inactivity after a request to add tests. The original branch was ~2 years (307 commits) behind
master, so this reintroduces the strategy on top of the current storage API and addresses the test request.What this adds
ConcurrencyLimitRateLimiter(synclimits.strategies+ asynclimits.aio.strategies), registered as theconcurrency-limitstrategy. Unlike the window-based strategies it bounds in-flight concurrency:hit()acquires slots andrelease()hands them back. A safety expiry (derived from the limit period) reclaims slots leaked by callers that never release.ConcurrencyLimitSupportcapability mixin (sync + async) exposingdecr, mirroring the existingMovingWindowSupport/SlidingWindowCounterSupportpattern. The in-memory storage (which already implementsdecr) opts in; the limiter raisesNotImplementedErrorfor storages that don't support it.Notes
hit()uses acquire-then-rollback (incr, thendecron overflow) so the counter stays accurate under contention and rejected hits never consume a slot — an improvement over the original get-then-incr which had a race.decrcurrently only exists on the in-memory storage, so the strategy is wired for in-memory here; extendingdecrto the redis / memcached / mongodb backends to broaden support can follow.Supersedes #233.