Agent Quality of Service (QoS)

The lowest priority class, assigned to Pods where no container has any CPU or memory requests or limits specified.

• Scheduling: Has no resource guarantees; scheduled onto nodes with allocatable space.
• Eviction: These Pods are the first to be terminated when the node experiences memory pressure.
• Use Case: Non-critical, batch-oriented, or test agents where interruption is acceptable.

The scheduler uses the resource request (not the limit) to determine if a node has enough capacity to run a Pod.

• A Guaranteed Pod with a 1 CPU request requires a node with at least 1 allocatable CPU.
• A BestEffort Pod, with no request, can be scheduled anywhere but competes for resources without protection.
• Nodes use a Quality of Service (QoS) cgroup hierarchy to enforce these classes, prioritizing CPU time for higher-class Pods.

When a node runs out of memory, the kubelet triggers pod eviction to reclaim resources. The order is deterministic:

1. BestEffort Pods are killed first.
2. Burstable Pods are killed next, starting with those consuming the most resources relative to their request.
3. Guaranteed Pods are killed last, only if they are using more than their request or if all other Pods are terminated.

This ensures critical agent workloads have the highest survivability.

Guaranteed QoS Pod Spec:

yamlCopy
resources:
  requests:
    memory: "256Mi"
    cpu: "500m"
  limits:
    memory: "256Mi"
    cpu: "500m"

Burstable QoS Pod Spec:

yamlCopy
resources:
  requests:
    memory: "128Mi"
    cpu: "250m"
  limits:
    memory: "512Mi"  # Limit > Request
    cpu: "1"

BestEffort QoS Pod Spec:

yamlCopy
# No resources section defined

A Pod Disruption Budget (PDB) is a Kubernetes policy that limits the number of agent pods in a voluntary disruption that can be down simultaneously, ensuring high availability during maintenance. It defines the minimum number or percentage of pods that must remain available.

• Voluntary Disruptions: Includes actions initiated by cluster administrators, such as node draining for upgrades.
• QoS Interaction: The orchestrator respects PDBs when evicting pods under resource pressure. A BestEffort pod with a PDB may be evicted after a Burstable pod without a PDB, as PDBs influence but do not override the QoS-based eviction order.
• Key Parameters: minAvailable (e.g., "90%") or maxUnavailable (e.g., "1").

Agent Scheduling is the process by which an orchestration system decides which compute node should run a specific agent instance. The scheduler uses the agent's QoS class, derived from its resource requests and limits, as a primary input for this decision.

• Bin Packing: The scheduler attempts to place pods on nodes with sufficient allocatable resources to meet the pod's requests.
• Priority for Guaranteed: Pods with Guaranteed QoS are often easier to schedule predictably because their resource usage is strictly bounded.
• Node Pressure: If a node experiences resource pressure (e.g., memory exhaustion), the kubelet will evict pods, starting with BestEffort, then Burstable pods exceeding their requests, before touching Guaranteed pods.

Agent Self-Healing is an orchestration capability where the system automatically detects agent failures and takes corrective action, such as restarting the agent or rescheduling it. QoS influences the system's response to resource-related failures.

• Health Probes: Uses liveness and readiness probes to determine agent health.
• Eviction vs. Termination: A pod evicted due to memory pressure (a QoS-driven action) is different from a termination due to a failed liveness probe. The former is a resource management event, while the latter is a health failure.
• Restart Policy: Defines whether a failed or evicted pod should be restarted on the same or a different node, interacting with the agent's persistent state requirements.

Agent Declarative Configuration is a practice where the desired state of an agent system is declared in version-controlled files (like YAML), and an orchestration tool ensures the actual state matches this specification. QoS is a property derived from this declared configuration.

• Source of Truth: Resource requests and limits are defined in the pod spec, from which the QoS class is automatically calculated.
• GitOps: This declarative spec is often managed via GitOps workflows, where changes to resource definitions in a Git repository trigger automated rollouts.
• Immutable Intent: The orchestrator continuously reconciles the cluster state to match these declarations, including enforcing the resource guarantees and limits that define QoS.