Horizontal Pod Autoscaler (HPA)

HPA scales workloads based on observed metrics, not schedules or predictions. It continuously queries the Kubernetes Metrics API to collect data points (e.g., average CPU utilization across all pods) and compares them against target values you define.

• Default Metric: CPU utilization (millicores).
• Custom & External Metrics: Can scale based on memory usage, application-specific metrics (e.g., requests per second from a Prometheus adapter), or even cloud service metrics.
• Target Value: You set a target, such as 70% CPU utilization. HPA's algorithm calculates the desired replica count to meet that target.

HPA uses a deterministic algorithm to calculate the desired number of replicas.

desiredReplicas = ceil[currentReplicas * ( currentMetricValue / desiredMetricValue )]

• The calculation is performed per metric. When multiple metrics are specified, HPA calculates a replica count for each and chooses the highest value.
• It incorporates stabilization windows to prevent rapid, flapping scale events. The --horizontal-pod-autoscaler-downscale-stabilization flag (default 5 minutes) dictates how long HPA must observe low demand before scaling down.
• Tolerance: A configurable tolerance band (default 0.1) prevents trivial rescaling. A change of less than 10% in the desired replica count may be ignored.

HPA supports distinct metric types that require different API setups.

• Resource Metrics: Core Kubernetes resource usage (CPU, memory). These use the metrics.k8s.io API, typically provided by the Metrics Server.
• Custom Metrics: Application-level metrics (e.g., HTTP requests queue depth, messages processed per second). These use the custom.metrics.k8s.io API, provided by an adapter like the Prometheus Adapter.
• External Metrics: Metrics from outside the Kubernetes cluster (e.g., a queue length from AWS SQS or Google Pub/Sub). These use the external.metrics.k8s.io API.

Custom and external metrics enable scaling based on business logic, not just infrastructure saturation.

The behavior field in the HPA spec allows fine-grained control over scaling speed and stabilization.

• Scale-Up/Down Policies: Control the rate of scaling. You can specify the number of pods that can be added/removed per minute (podsPerSecond is also an option) and a periodSeconds window.
• Stabilization Windows: Define separate windows for scale-up and scale-down. A longer downscale stabilization window protects against removing pods too quickly after a brief traffic spike.
• Select Policies: Specify if the highest or lowest calculated replica change from a policy should be used.

Example: A policy allowing a max increase of 100% of current replicas every 60 seconds for rapid scale-up, but a max decrease of 10% every 5 minutes for cautious scale-down.

HPA and the Cluster Autoscaler (CA) work together for full-stack elasticity.

1. HPA adjusts the number of pods (application layer) based on demand.
2. If scaling up creates pods that cannot be scheduled due to insufficient node resources (CPU/memory), the Cluster Autoscaler detects this.
3. CA automatically provisions new nodes in the cloud provider (infrastructure layer) to accommodate the pending pods.
4. Conversely, when HPA scales down pods and nodes become underutilized, CA can safely drain and remove those nodes to reduce infrastructure costs.

This creates a fully automated, cost-efficient scaling pipeline from application to infrastructure.

Effective HPA usage requires awareness of its operational boundaries.

• Pod Readiness: HPA only scales pods that are ready. Pods stuck in ContainerCreating or failing readiness probes are not counted, which can cause excessive scaling.
• Resource Requests Must Be Set: For CPU-based scaling, your pod spec must define resources.requests.cpu. HPA cannot calculate utilization percentage without a request value as the denominator.
• Cool-Down Delays: After a scaling action, HPA waits a cooldown period (--horizontal-pod-autoscaler-upscale-delay, default 3 min down, 0 sec up in K8s >=1.24) before evaluating metrics again to allow metrics to stabilize.
• Minimum and Maximum Replicas: Always set sensible minReplicas and maxReplicas to prevent runaway scaling (to zero or to exhaust cluster resources).