Agent Scheduling

The foundational constraints for scheduling. Resource requests specify the minimum guaranteed CPU and memory an agent needs to run, which the scheduler uses to find a node with sufficient capacity. Resource limits define the maximum amount an agent can consume, preventing a single agent from starving others on the same node. These directly influence the agent's Quality of Service (QoS) class (Guaranteed, Burstable, BestEffort), which affects its priority during resource contention and eviction.

• Example: An agent with a request of 500m CPU and 1Gi memory will only be placed on a node with at least that much allocatable resource.

Rules that guide the scheduler toward or away from specific nodes or groups of nodes.

• Node Selectors: Simple key-value pairs that schedule an agent only on nodes with matching labels (e.g., accelerator: gpu-a100).
• Node Affinity/Anti-Affinity: More expressive rules using operators like In, NotIn, Exists. Affinity attracts agents to nodes with certain properties, while Anti-Affinity repels them, crucial for distributing agents for high availability.
• Inter-Pod Affinity/Anti-Affinity: Controls co-location. Use affinity to place latency-sensitive agents together; use anti-affinity to prevent a single node failure from taking down all replicas of a critical agent.

A push model for node-level constraints. A taint is applied to a node to repel all pods unless they have a matching toleration. This is used for dedicating nodes to specific workloads (e.g., dedicated=ai-agent:NoSchedule) or marking nodes as problematic.

Node conditions like MemoryPressure, DiskPressure, or NodeNotReady are system-generated taints that prevent new agents from being scheduled onto unhealthy nodes. The scheduler also respects node capacity and existing allocated resources when making placement decisions.

Advanced rules for controlling the distribution of agent pods across failure domains to maximize availability and performance. These constraints spread agents evenly across:

• Zones/Regions (Cloud failure domains)
• Hosts/Nodes
• Custom topology keys (e.g., rack labels in a data center)

You define a maxSkew, which is the maximum difference in the number of pods between any two topology domains. This ensures agents are not overly concentrated in a single zone or rack, protecting against domain-level failures.

The scheduler's internal scoring and filtering logic. The scheduler first filters out nodes that don't meet hard requirements (resources, taints). Then, it scores remaining nodes based on policies:

• LeastAllocated: Favors nodes with the most free resources.
• BalancedResourceAllocation: Favors nodes with balanced CPU and memory usage.
• NodeAffinity/InterPodAffinity: Scores higher for nodes matching affinity rules.
• ImageLocality: Prefers nodes that already have the required container image cached.

The node with the highest aggregate score is selected. These policies can be customized via scheduler profiles.

Mechanisms for managing cluster contention.

• Pod Priority: Indicates the relative importance of an agent pod. Higher-priority pods can preempt (evict) lower-priority pods from a node if resources are needed.
• Resource Quotas: Enforced at the namespace level, limiting the total amount of CPU, memory, or number of pods a team's agents can collectively consume. This prevents a single project from monopolizing cluster resources and is a critical constraint the scheduler must respect.
• Pod Disruption Budgets (PDBs): While not a direct scheduler input, PDBs limit voluntary disruptions (e.g., node drains) by ensuring a minimum number of pods for an application remain available, indirectly influencing rescheduling decisions during maintenance.

The process of creating and launching a new agent instance, which is the prerequisite event for scheduling. This involves loading its code, configuration, and initial state into an execution environment. The scheduler's job begins once instantiation is requested, as it must select a suitable compute node to host the new instance based on resource availability and constraints.

Declarative constraints that directly influence scheduling decisions by specifying placement preferences for agent pods.

• Affinity Rules request that certain agents be co-located on the same node, often to reduce latency for frequent communication.
• Anti-Affinity Rules mandate that agents be distributed across different nodes, zones, or hardware to improve fault tolerance and resource distribution.

These rules are evaluated by the scheduler alongside resource requests.

A cluster-level policy constraint that limits the aggregate compute resources (CPU, memory) or object counts (pods, services) that a collection of agents within a namespace can consume. The scheduler must respect these quotas; it cannot place an agent on a node if doing so would cause its namespace to exceed its allocated quota, even if the node itself has available capacity.

A classification (Guaranteed, Burstable, BestEffort) assigned by an orchestrator based on the resource requests and limits defined in an agent's pod specification. This classification influences both scheduling and eviction behavior.

• Guaranteed pods (equal requests & limits) have the highest scheduling priority and are last to be evicted.
• BestEffort pods (no requests/limits) have the lowest priority and are first to be terminated under system memory pressure.

A Kubernetes controller that automatically scales the number of replicas in an agent deployment or statefulset based on observed metrics like CPU utilization or custom application metrics. The HPA triggers scaling events, which in turn create new scheduling workloads. The scheduler must then find appropriate nodes for the newly requested agent pods, making HPA and scheduling tightly coupled processes for dynamic workloads.

An orchestration capability where the system automatically detects agent failures (via failed health checks) and takes corrective action. A core part of self-healing is rescheduling: if an agent pod crashes or its node fails, the scheduler is invoked to place a replacement instance on a healthy, available node. This ensures the desired state of the system is maintained despite failures.