Internals

This section documents private internal objects for reference. You will not need to manipulate them directly unless you are seeking to contribute to the library.

Controller mixins

class rate_control._controllers._bucket_based.BucketBasedRateController

Bases: RateController, ABC

Mixin for rate controllers that use buckets.

async __aenter__() → Self

Enter the controller’s context.

Also enters the context of the underlying buckets, if the should_enter_context flag was set to True.

async __aexit__(*exc_info: Any) → bool | None

Exit the controller’s context.

Also exits the context of the underlying buckets, if the should_enter_context flag was set to True.

__init__(*buckets: Bucket, should_enter_context: bool = True, max_concurrency: int | None = None, **kwargs: Any) → None

Parameters:

• buckets (Bucket) – The buckets that will be managed by the rate controller, optional.

• should_enter_context (bool) – Whether entering the context of the rate controller should also enter the context of the underlying buckets, if any. Defaults to True.

• max_concurrency (Optional[int]) – The maximum amount of concurrent requests allowed. Defaults to None (no limit).

can_acquire(tokens: float = 1) → bool

Parameters:

tokens (float) – The amount of tokens to acquire for the request. Defaults to 1.

Returns:

Whether a request for the given amount of tokens can be processed instantly.

abstract request(tokens: float = 1, **kwargs: Any) → AsyncIterator[None]

Asynchronous context manager that requests the given amount of tokens before execution.

Parameters:

tokens (float) – The number of tokens required for the request. Defaults to 1.

Bucket mixins

class rate_control._buckets._base.TokenBasedBucket

Bases: Bucket, ABC

Base class for buckets that monitor the requests using tokens.

__init__(capacity: float, **kwargs: Any) → None

acquire(tokens: float) → None

Acquire the given amount of tokens.

Parameters:

tokens (float) – The amount of tokens to acquire.

Raises:

RateLimit – Cannot acquire the given amount of tokens.

can_acquire(tokens: float) → bool

Whether the given amount of tokens can be acquired.

Parameters:

tokens (float) – The amount of tokens that we want to acquire.

Returns:

Whether the given amount of tokens is available to consume.

class rate_control._buckets._base.BaseRateBucket

Bases: TokenBasedBucket, ContextAware, Bucket, ABC

Base class for token buckets that refill at a certain rate.

__init__(capacity: float, delay: float, **kwargs: Any) → None

Parameters:

• capacity (float) – The number of tokens that can be acquired within delay.

• delay (float) – The refill delay in seconds.

acquire(tokens: float) → None

Acquire the given amount of tokens.

Parameters:

tokens (float) – The amount of tokens to acquire.

Raises:

RateLimit – Cannot acquire the given amount of tokens.

can_acquire(tokens: float) → bool

Whether the given amount of tokens can be acquired.

Parameters:

tokens (float) – The amount of tokens that we want to acquire.

Returns:

Whether the given amount of tokens is available to consume.

async wait_for_refill() → None

Wait until some tokens are replenished.

class rate_control._buckets._base.BaseWindowedTokenBucket

Bases: BaseRateBucket, ABC

Base class for token buckets that follow strategies based on time windows.

__init__(capacity: float, duration: float, **kwargs: Any) → None

Parameters:

• capacity (float) – The number of tokens that can be acquired within duration.

• duration (float) – The window duration in seconds.

acquire(tokens: float) → None

Acquire the given amount of tokens.

Parameters:

tokens (float) – The amount of tokens to acquire.

Raises:

RateLimit – Cannot acquire the given amount of tokens.

can_acquire(tokens: float) → bool

Whether the given amount of tokens can be acquired.

Parameters:

tokens (float) – The amount of tokens that we want to acquire.

Returns:

Whether the given amount of tokens is available to consume.

async wait_for_refill() → None

Wait until some tokens are replenished.

class rate_control._buckets._base.CapacityUpdatingBucket

Bases: TokenBasedBucket

Mixin for buckets which token capacity can be updated.

update_capacity(new_capacity: float) → None

Update the bucket’s token capacity.

Changes take effect instantly, and the amount of remaining tokens is updated accordingly.

Parameters:

new_capacity (float) – The new token capacity of the bucket.

Miscellaneous

class rate_control._helpers.Request

Bases: Comparable

Represents a user’s request for tokens

__init__(cost: float, **kwargs: Any) → None

Parameters:

cost (float) – The number of tokens requested.

__lt__(other: Self) → bool

Comparison operator so that requests can be placed in a priority queue and processed by ascending cost.

ack() → None

Acknowledge the request.

fire() → None

Fire the request.

async wait_for_ack() → None

Wait until the request has been acknowledged.

async wait_for_validation() → None

Wait until the request has been fired.

class rate_control._helpers._protocols.Comparable

Bases: Protocol

__init__(*args, **kwargs)

abstract __lt__(other: Self, /) → bool

Comparison operator.

Parameters:

other (Self) – Another object of the same type.

Returns:

Whether self < other.

class rate_control._helpers.ContextAware

Bases: AbstractAsyncContextManager, ABC

Mixin for raising an exception if the async context manager is entered more than once.

async __aenter__() → Any

Sets the state of the context manager to State.ENTERED.

Raises:

RuntimeError – The context manager has already been entered.

async __aexit__(*exc_info: Any) → bool | None

Sets the state of the context manager to State.CLOSED.

__init__(**kwargs: Any) → None

Enumerations

enum rate_control._enums.State(value)

Bases: Enum

The current state of the context manager.

Valid values are as follows:

UNENTERED = <State.UNENTERED: 1>

ENTERED = <State.ENTERED: 2>

CLOSED = <State.CLOSED: 3>

Exceptions

exception rate_control._errors.Empty

Bases: Exception

Collection is empty.