Horizontal Pod Autoscaler (HPA)

The HPA's core logic relies on a continuous feedback loop that queries aggregated resource metrics to make scaling decisions. It fetches per-pod resource metrics from the metrics-server or custom metrics from an external provider via the custom.metrics.k8s.io API.

• Resource Metrics: Default support for target average CPU utilization and average memory utilization.
• Custom Metrics: Scale on any metric exposed by your pods, such as requests-per-second or queue depth.
• External Metrics: Scale based on metrics from outside the cluster, like cloud provider message queue length.

The controller calculates the desired replica count using the formula: desiredReplicas = ceil[currentReplicas * (currentMetricValue / desiredMetricValue)].

To prevent thrashing—rapid, oscillating scale-up and scale-down events—the HPA implements configurable stabilization windows. These windows define a look-back period during which the controller considers past recommendations before acting.

• Scale-Down Stabilization: Defaults to 5 minutes. The HPA uses the highest recommendation from this window, preventing premature termination of pods during transient load drops.
• Scale-Up Policy: Can be configured with a specific policy (e.g., Pods or Percent) and a limit to control the rate of rapid expansion.
• Customizable Behavior: The behavior field in the HPA v2 API allows fine-tuning of both scale-up and scale-down policies independently.

Introduced in the autoscaling/v2 API, the HPA can target resource utilization of individual containers within a pod, rather than the aggregate pod total. This is critical for multi-container pods where a sidecar proxy or logging agent should not influence the scaling decision of the main application container.

• Target Type: Set ContainerResource as the metric type.
• Precision: Specify the exact container name to isolate its CPU or memory usage.
• Use Case: Essential for service mesh architectures where an Istio sidecar or Cloud SQL Auth proxy runs alongside the primary workload.

The Horizontal Pod Autoscaler can operate alongside the Vertical Pod Autoscaler (VPA) to achieve both horizontal elasticity and right-sized resource requests. However, they must not fight over the same resource dimensions.

• Avoid Conflict: If VPA manages CPU or memory, the HPA must not use those same metrics for scaling. Instead, use custom or external metrics for the HPA.
• Complementary Strategy: Use VPA in Off or Auto mode to set optimal base resource requests, while the HPA scales replica count based on throughput or latency.
• Golden Signal: A common pattern is to use VPA for memory sizing and HPA for CPU or custom metrics like request latency.

The HPA has evolved through distinct API versions, each adding more granular control. Understanding the differences is crucial for writing portable and modern manifests.

• autoscaling/v1: Supports only CPU utilization scaling. The simplest form, now considered legacy.
• autoscaling/v2beta2: Introduced support for memory and custom/external metrics. This was the bridge to the modern API.
• autoscaling/v2: The stable, current version. Supports all metric types (Resource, Pods, Object, External, ContainerResource) and the behavior field for fine-tuning scaling policies.

The HPA has specific logic for handling pods that are not in a Ready state or are missing metrics, ensuring that a single faulty pod does not skew the scaling algorithm.

• Missing Metrics: Pods that fail to report metrics are ignored during the calculation of the current utilization ratio. They are treated as if they have 0% utilization for scale-down decisions and 100% for scale-up, erring on the side of caution.
• Unready Pods: Pods in a non-Ready state with a deployment.kubernetes.io/revision label are ignored entirely during scaling calculations.
• Startup Tolerance: The HPA includes a configurable tolerance flag (default 0.1) to ignore minor fluctuations around the target metric, preventing unnecessary scaling noise.