Agent HorizontalPodAutoscaler (HPA)

The Agent HPA scales the agent deployment based on observed resource consumption or custom application metrics. It continuously monitors the specified metrics and compares them against target values.

• Core Metrics: Traditionally scales based on average CPU or memory utilization across all pods.
• Custom Metrics: Can scale based on application-specific metrics exposed via the Kubernetes Metrics API, such as queue length, requests per second, or business logic metrics.
• External Metrics: Can even scale based on metrics from systems outside the cluster, like a cloud provider's message queue depth.

The scaling decision is made by the formula: desiredReplicas = ceil[currentReplicas * (currentMetricValue / desiredMetricValue)].

Scaling behavior is defined declaratively using a Kubernetes resource manifest (YAML). The desired state—what to scale, which metrics to use, and the target values—is specified, and the HPA controller works to reconcile the actual state to match.

Key configuration fields include:

• scaleTargetRef: References the Deployment, StatefulSet, or other scalable resource (e.g., the agent deployment) to control.
• metrics: A list of metrics (resource, pod, object, external) and their target values.
• minReplicas / maxReplicas: The hard bounds for the number of pods, preventing runaway scaling.
• behavior: Configures scaling speed and stabilization windows to prevent flapping.

To prevent rapid, unnecessary scaling oscillations (flapping) due to metric noise, the HPA incorporates stabilization logic.

• Scale-Down Stabilization Window: A configurable period (default 5 minutes) where the HPA remembers the highest recommended replica count, delaying scale-down actions. This ensures scaling down only after a sustained period of lower demand.
• Scaling Policies: The behavior field allows fine-tuning of the pace of scaling. You can set policies like podsPerSecond to limit how quickly replicas are added or removed.
• Tolerance: A small default tolerance (10%) is applied to metric targets, meaning scaling won't occur for very minor deviations, further reducing noise.

For scaling on business logic (not just CPU/Memory), the Agent HPA relies on the Kubernetes Custom Metrics API. This requires an additional metrics adapter installation, such as Prometheus Adapter or Datadog Cluster Agent.

How it works:

1. An agent exposes an application metric (e.g., jobs_in_queue) via its endpoint or to a monitoring system like Prometheus.
2. The metrics adapter (e.g., prometheus-adapter) scrapes this data and makes it available to the Kubernetes Metrics API.
3. The HPA is configured with a metric of type Pods or Object, targeting this custom metric name.
4. The HPA controller queries the Metrics API for the current value and scales the agent replicas proportionally to bring the metric to its target value.

While HPA scales the number of pods (horizontal), the Vertical Pod Autoscaler (VPA) adjusts the resource requests/limits (CPU, memory) of individual pods. For stateful or memory-intensive agents, they can be used together cautiously.

Considerations for Combined Use:

• Primary Use Case: Use HPA for stateless, scalable agent workloads. Use VPA for agents with unpredictable memory growth or to right-size resource requests.
• Caution: Running HPA and VPA on the same workload for the same resource (e.g., CPU) is not recommended as they will conflict. A common pattern is to use HPA for custom metrics and VPA for memory, with VPA in Off or Initial mode for CPU.
• Agent StatefulSets: For stateful agents, HPA can still scale replicas, but careful design of persistent volume claims and application logic is required.

The automatic adjustment of the number of active agent instances in a pool based on real-time metrics. While the Agent HPA is a specific Kubernetes controller for this purpose, auto-scaling is the broader architectural goal. It can be driven by:

• Standard metrics like CPU and memory utilization.
• Custom metrics from an application, such as queue length or request latency.
• External metrics from systems outside the Kubernetes cluster. The goal is to maintain performance while optimizing resource costs.

A Kubernetes workload API object used to manage stateful agent applications. Unlike a standard Deployment (which an HPA typically scales), a StatefulSet provides guarantees about:

• Stable, unique network identifiers (hostnames).
• Persistent storage that follows the pod.
• Ordered, graceful deployment and scaling. This is critical for agents that require stable identity or durable local state, such as a database agent or a leader in a distributed consensus group. Scaling a StatefulSet with an HPA requires careful consideration of state management.

A Kubernetes policy that limits the number of pods in a voluntary disruption that can be down simultaneously. When an Agent HPA scales down pods or during cluster maintenance (node drains), the PDB ensures high availability by preventing too many replicas of a critical agent from being terminated at once.

• Key Parameters: minAvailable (e.g., "90%") or maxUnavailable (e.g., "1").
• Voluntary Disruptions: Actions initiated by users or controllers (e.g., eviction by HPA, node maintenance). It acts as a safeguard to prevent scaling actions from violating application availability SLOs.

A cluster-level policy constraint that limits the aggregate resource consumption for agents within a namespace. While the Agent HPA dictates how many pods to run, Resource Quotas define the maximum resources those pods can collectively use, preventing a single auto-scaled agent deployment from consuming all cluster capacity.

• Compute Resources: Total CPU and memory requests/limits.
• Object Counts: Maximum number of pods, services, or configmaps.
• Storage Resources: Total amount of persistent storage claims. Quotas are essential for multi-tenant environments to ensure fair resource allocation.

A classification (Guaranteed, Burstable, BestEffort) assigned by Kubernetes based on an agent pod's resource requests and limits. This classification influences pod scheduling and eviction under resource pressure, which directly interacts with HPA behavior.

• Guaranteed: Sets equal requests and limits for CPU/memory. Highest priority, last to be evicted.
• Burstable: Sets a request lower than its limit. Common for HPA-targeted workloads.
• BestEffort: No requests or limits set. First to be terminated. Proper QoS configuration ensures critical, auto-scaled agents are not prematurely killed during node memory pressure.

The extension point in Kubernetes that allows the HorizontalPodAutoscaler (HPA) to scale based on application-specific metrics, not just CPU/memory. This enables scaling agents based on business logic, such as:

• Message queue depth from RabbitMQ or Kafka.
• HTTP request rate or latency percentiles.
• Custom business metrics (e.g., "orders processed per second"). Implementing this typically requires a metrics adapter (like the Prometheus Adapter) that translates metrics from a monitoring system (Prometheus, Datadog) into the Kubernetes Custom Metrics API format for the HPA to consume.