Controllers

class rate_control.RateController

Bases: ABC

Abstract base class for rate controllers.

async __aenter__() → Self

Enter the controller’s context.

async __aexit__(*_: Any) → bool | None

Exit the controller’s context.

abstract can_acquire(tokens: float = 1) → bool

Parameters:

tokens (float) – The amount of tokens to acquire for the request. Defaults to 1.

Returns:

Whether a request for the given amount of tokens can be processed instantly.

abstract request(tokens: float = 1, **kwargs: Any) → AsyncIterator[None]

Asynchronous context manager that requests the given amount of tokens before execution.

Parameters:

tokens (float) – The number of tokens required for the request. Defaults to 1.

class rate_control.RateLimiter

Bases: BucketBasedRateController

Rate controller that raises an error if a request cannot be fulfilled instantly.

__init__(*buckets: Bucket, should_enter_context: bool = True, max_concurrency: int | None = None, **kwargs: Any) → None

Parameters:

• buckets (Bucket) – The buckets that will be managed by the rate controller, optional.

• should_enter_context (bool) – Whether entering the context of the rate controller should also enter the context of the underlying buckets, if any. Defaults to True.

• max_concurrency (Optional[int]) – The maximum amount of concurrent requests allowed. Defaults to None (no limit).

request(tokens: float = 1, **_: Any) → AsyncIterator[None]

Context manager that acquires the given amount of tokens while holding concurrency.

Parameters:

tokens (float) – The number of tokens to acquire. Defaults to 1.

Raises:

RateLimit – The request cannot be fulfilled instantly.

class rate_control.Scheduler

Bases: BucketBasedRateController, ContextAware, RateController

Rate controller that schedules requests for later processing.

__init__(*buckets: Bucket, should_enter_context: bool = True, max_concurrency: int | None = None, max_pending: int | None = None, queue_factory: Callable[[], Queue[Request]] = PriorityQueue, **kwargs: Any) → None

Parameters:

• buckets (Bucket) – The buckets that will be managed by the rate controller, optional.

• should_enter_context (bool) – Whether entering the context of the rate controller should also enter the context of the underlying buckets, if any. Defaults to True.

• max_concurrency (Optional[int]) – The maximum amount of concurrent requests allowed. Defaults to None (no limit).

• max_pending (Optional[int]) – The maximum amount of requests waiting to be processed. Defaults to None (no limit).

• queue_factory (Callable[[], Queue[Request]]) – The factory for initializing the request queues. Defaults to PriorityQueue: requests are processed by ascending weight.

request(tokens: float = 1, priority: Priority = Priority.NORMAL, fill_or_kill: bool = False, **_: Any) → AsyncIterator[None]

Asynchronous context manager that schedules the execution of the contained statements.

Waits until all the conditions of token availability and allowed concurrency are met, before actually consuming tokens and holding a spot for the concurrency.

Parameters:

• tokens (float) – The number of tokens required for the request. Defaults to 1.

• priority (Priority) – The priority of the request. Requests with higher priority will be processed before the others. Defaults to Priority.NORMAL.

• fill_or_kill (bool) – Whether RateLimit should be raised if the request cannot be process instantly. Defaults to False.

Raises:

• RateLimit – The request cannot be processed instantly but the fill_or_kill flag was set to True.

• ReachedMaxPending – The limit of pending requests was reached.

class rate_control.NoopController

Bases: RateController

Rate controller that accepts all requests and does nothing.

static __new__(cls, *_: Any, **kwargs: Any) → NoopController

Positional arguments, accepted for consistency with other rate controllers, are ignored. Keyword arguments are passed to the super class constructor.

Note

Implementation detail: The __new__ method returns a singleton instance, for better memory management.

can_acquire(tokens: float = 1) → Literal[True]

Always returns True.

request(tokens: float = 1, **_: Any) → AsyncIterator[None]

Asynchronous context manager that does nothing else than yield.