Basic RateLimiter .NET: algorithms and queues

# Basic Rate Limiters in the Universal .NET RateLimiter: Architecture and Algorithms

The universal .NET rate limiting component includes four basic algorithmic limiters: FixedWindowRateLimiter, SlidingWindowRateLimiter, TokenBucketRateLimiter, and ConcurrencyLimiter. The first implements a fixed window, the second a sliding window, the third a token bucket, and the fourth limits concurrency.

Rate limiter classes inherit from ReplenishingRateLimiter, which defines the logic for replenishing permits via a timer. ConcurrencyLimiter inherits directly from RateLimiter. Constructors accept configuration parameters, which are stored in a copy of Options.

All implement AttemptAcquireCore (synchronous), AcquireAsyncCore (asynchronous), Dispose/DisposeAsync. Rate limiters also add ReplenishInternal.

Handling Permit Requests

Requests are divided into trial requests (permitCount = 0) and real requests (permitCount > 0). Negative values throw ArgumentOutOfRangeException.

Processing algorithm:

• Verify permitCount ≤ AutoReplenishment.MaxPermits.
• For trial requests: if availablePermits > 0 — immediate success.
• Acquire lock on the queue deque.
• If availablePermits ≥ permitCount — grant permits, decrement, and release lock.
• Otherwise: synchronous rejection or add to queue (asynchronous).

The asynchronous method returns ValueTask<RateLimitLease>. On queue overflow — rejection.

Queue Structure for Requests

The queue is a deque<RequestRegistration>, where RequestRegistration inherits from TaskCompletionSource<RateLimitLease>.

Queue settings in Options:

• QueueLimit: maximum sum of permitCount in the queue.
• QueueProcessingOrder: OldestFirst (FIFO, default) or NewestFirst (LIFO).

Current size: _queueTotalPermits. Locking on deque.

On permit replenishment: Dequeue, CompleteTask with successful Lease, decrement _queueTotalPermits.

Returning Leases and Statistics

Methods return derivatives of RateLimitLease:

• FixedWindowLease, SlidingWindowLease, TokenBucketLease (similar).
• ConcurrencyLease (different).

Counters: _successfulAcquires, _failedAcquires (Interlocked.Increment).

RateLimiterStatistics provides:

• AvailablePermits.
• CurrentQueueLength.
• Counters.

The IdleDuration property tracks time spent idle without requests.

Features of Each Limiter

FixedWindowRateLimiter

Fixed window: permits are replenished at the start of the window (permitPeriod).

_availablePermits is reset to MaxPermits.

SlidingWindowRateLimiter

Sliding window: tracks history across SlidingWindowSize segments (segmentSize = permitPeriod / SlidingWindowSize).

Restores based on previous segments.

TokenBucketRateLimiter

Token bucket: _tokens replenished every ReplenishmentPeriod, rate = TokensPerPeriod.

Limits spending based on refillAmount.

ConcurrencyLimiter

Limits _currentConcurrency ≤ PermitLimit.

Queue launches tasks upon completion of active ones (OnCompleted).

Key Points

• Versatility: basic limiters form the foundation of ASP.NET Core rate limiting and are applicable outside web contexts.
• Queue: supports FIFO/LIFO, limit by total permits sum, asynchronous execution.
• Trial requests: permitCount=0 for checking without consumption.
• Statistics: atomic counters + real-time metrics.
• Replenishment: timer-driven for all ReplenishingRateLimiter.

— Editorial Team