guidellm.scheduler.constraints.constraint
Core constraint system protocols and base classes.
Defines the fundamental protocols and base classes that form the foundation of the constraint system. Constraints control scheduler behavior by evaluating scheduler state and individual requests to determine whether processing should continue or stop based on predefined limits. The constraint system enables sophisticated benchmark stopping criteria through composable constraint types with support for serialization, validation, and dynamic instantiation.
The module provides: - Protocols defining the constraint interface contract (Constraint, ConstraintInitializer) - Base classes for Pydantic-based constraint initializers with serialization support - Placeholder classes for handling unserializable constraint states
Example: :: from guidellm.scheduler.constraints import ( Constraint, PydanticConstraintInitializer, )
class MyConstraint(PydanticConstraintInitializer):
type_: str = "my_constraint"
def create_constraint(self) -> Constraint:
def evaluate(state, request):
return SchedulerUpdateAction(request_queuing="continue")
return evaluate
Constraint
Bases: Protocol
Protocol for constraint evaluation functions that control scheduler behavior.
Defines the interface that all constraint implementations must follow. Constraints are callable objects that evaluate scheduler state and request information to determine whether processing should continue or stop. The protocol enables type checking and runtime validation of constraint implementations while allowing flexible implementation approaches (functions, classes, closures).
Example: :: def my_constraint( state: SchedulerState, request: RequestInfo ) -> SchedulerUpdateAction: if state.processing_requests > 100: return SchedulerUpdateAction(request_queuing="stop") return SchedulerUpdateAction(request_queuing="continue")
Source code in src/guidellm/scheduler/constraints/constraint.py
__call__(state, request)
Evaluate constraint against scheduler state and request information.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
state | SchedulerState | Current scheduler state with metrics and timing information | required |
request | RequestInfo | Individual request information and metadata | required |
Returns:
| Type | Description |
|---|---|
SchedulerUpdateAction | Action indicating whether to continue or stop scheduler operations |
Source code in src/guidellm/scheduler/constraints/constraint.py
ConstraintInitializer
Bases: Protocol
Protocol for constraint initializer factory functions that create constraints.
Defines the interface for factory objects that create constraint instances from configuration parameters. Constraint initializers enable dynamic constraint creation and configuration, supporting both simple boolean flags and complex parameter dictionaries. The protocol allows type checking while maintaining flexibility for different initialization patterns.
Example: :: class MaxRequestsInitializer: def init(self, max_requests: int): self.max_requests = max_requests
def create_constraint(self) -> Constraint:
def evaluate(state, request):
if state.total_requests >= self.max_requests:
return SchedulerUpdateAction(request_queuing="stop")
return SchedulerUpdateAction(request_queuing="continue")
return evaluate
Source code in src/guidellm/scheduler/constraints/constraint.py
create_constraint(**kwargs)
Create a constraint instance from configuration parameters.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
kwargs | Configuration parameters for constraint creation | {} |
Returns:
| Type | Description |
|---|---|
Constraint | Configured constraint evaluation function |
Source code in src/guidellm/scheduler/constraints/constraint.py
PydanticConstraintInitializer
Bases: StandardBaseModel, ABC, InfoMixin
Abstract base for Pydantic-based constraint initializers.
Provides standardized serialization, validation, and metadata handling for constraint initializers using Pydantic models. Subclasses implement specific constraint creation logic while inheriting validation and persistence support. Integrates with the constraint factory system for dynamic instantiation and configuration management.
Example: :: @ConstraintsInitializerFactory.register("max_duration") class MaxDurationConstraintInitializer(PydanticConstraintInitializer): type_: str = "max_duration" max_seconds: float = Field(description="Maximum duration in seconds")
def create_constraint(self) -> Constraint:
def evaluate(state, request):
if time.time() - state.start_time > self.max_seconds:
return SchedulerUpdateAction(request_queuing="stop")
return SchedulerUpdateAction(request_queuing="continue")
return evaluate
Attributes:
| Name | Type | Description |
|---|---|---|
type_ | str | Type identifier for the constraint initializer |
Source code in src/guidellm/scheduler/constraints/constraint.py
info property
Extract serializable information from this constraint initializer.
Returns:
| Type | Description |
|---|---|
dict[str, Any] | Dictionary containing constraint configuration and metadata |
create_constraint(**kwargs) abstractmethod
Create a constraint instance.
Must be implemented by subclasses to return their specific constraint type with appropriate configuration and validation. The returned constraint should be ready for evaluation against scheduler state and requests.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
kwargs | Additional keyword arguments (usually unused) | {} |
Returns:
| Type | Description |
|---|---|
Constraint | Configured constraint instance |
Raises:
| Type | Description |
|---|---|
NotImplementedError | Must be implemented by subclasses |
Source code in src/guidellm/scheduler/constraints/constraint.py
SerializableConstraintInitializer
Bases: Protocol
Protocol for serializable constraint initializers supporting persistence.
Extends ConstraintInitializer with serialization capabilities, enabling constraint configurations to be saved, loaded, and transmitted. Serializable initializers support validation, model-based configuration, and dictionary-based serialization for integration with configuration systems and persistence layers.
Example: :: class SerializableInitializer: @classmethod def model_validate(cls, data: dict) -> ConstraintInitializer: return cls(**data)
def model_dump(self) -> dict[str, Any]:
return {"type_": "max_requests", "max_requests": self.max_requests}
def create_constraint(self) -> Constraint:
# ... create constraint
Source code in src/guidellm/scheduler/constraints/constraint.py
create_constraint(**kwargs)
Create constraint instance from this initializer.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
kwargs | Additional configuration parameters | {} |
Returns:
| Type | Description |
|---|---|
Constraint | Configured constraint evaluation function |
Source code in src/guidellm/scheduler/constraints/constraint.py
model_dump()
Serialize constraint initializer to dictionary format.
Returns:
| Type | Description |
|---|---|
dict[str, Any] | Dictionary representation of constraint initializer |
model_validate(**kwargs) classmethod
Create validated constraint initializer from configuration.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
kwargs | Configuration dictionary for initializer creation | {} |
Returns:
| Type | Description |
|---|---|
ConstraintInitializer | Validated constraint initializer instance |
Source code in src/guidellm/scheduler/constraints/constraint.py
UnserializableConstraintInitializer
Bases: PydanticConstraintInitializer
Placeholder for constraints that cannot be serialized or executed.
Represents constraint initializers that failed serialization or contain non-serializable components. Cannot be executed and raises errors when invoked to prevent runtime failures from invalid constraint state. Used by the factory system to preserve constraint information even when full serialization is not possible.
Example: :: # Created automatically by factory when serialization fails unserializable = UnserializableConstraintInitializer( orig_info={"type_": "custom", "data": non_serializable_object} )
# Attempting to use it raises RuntimeError
constraint = unserializable.create_constraint() # Raises RuntimeError
Attributes:
| Name | Type | Description |
|---|---|---|
type_ | Literal['unserializable'] | Always "unserializable" to identify placeholder constraints |
orig_info | dict[str, Any] | Original constraint information before serialization failure |
Source code in src/guidellm/scheduler/constraints/constraint.py
__call__(state, request)
Raise error since unserializable constraints cannot be invoked.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
state | SchedulerState | Current scheduler state (unused) | required |
request | RequestInfo | Individual request information (unused) | required |
Raises:
| Type | Description |
|---|---|
RuntimeError | Always raised for unserializable constraints |
Source code in src/guidellm/scheduler/constraints/constraint.py
create_constraint(**_kwargs)
Raise error for unserializable constraint creation attempt.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
kwargs | Additional keyword arguments (unused) | required |
Raises:
| Type | Description |
|---|---|
RuntimeError | Always raised since unserializable constraints cannot be executed |