Auto Scaling

The fundamental capability of auto scaling is to provision or terminate compute instances in real-time without manual intervention. This is governed by scaling policies that define the conditions for change.

• Scale-out (Horizontal Scaling): Adds instances to the pool to handle increased load.
• Scale-in: Removes instances during periods of low demand to reduce costs.
• Trigger Metrics: Policies are activated by metrics like CPU utilization, network I/O, request count per target, or custom application metrics.

Auto scaling systems employ two primary methodologies to anticipate or respond to load changes.

• Reactive Scaling: The most common approach. It reacts to real-time metric thresholds (e.g., scale out when average CPU > 70% for 5 minutes).
• Predictive Scaling: Uses machine learning or historical load patterns to forecast traffic (e.g., daily peaks) and provisions capacity in advance. This minimizes latency spikes during rapid ramp-up.

Auto scaling integrates with health check systems to maintain pool integrity. If an instance fails its health check (becomes unhealthy), the auto scaling group automatically terminates it and launches a replacement.

• This ensures the desired number of healthy instances is always maintained.
• It provides a self-healing infrastructure, crucial for high-availability architectures.

Auto scaling groups are inherently linked to load balancers (Application Load Balancers, Network Load Balancers). New instances are automatically registered with the load balancer upon launch and deregistered during termination.

• Connection Draining: Ensures graceful termination by allowing the load balancer to complete existing requests to an instance before it is shut down.
• This creates a fully automated, elastic front-end to backend pipeline.

For predictable, recurring workload patterns, auto scaling supports scheduled actions. These are time-based policies that adjust the desired capacity at a specified date and time.

• Example: Scale to 10 instances at 9 AM Monday for the weekly batch job, then scale back to 2 instances at 6 PM.
• This allows for cost optimization without requiring reactive metric thresholds to be breached.

A cooldown period is a configurable timer that prevents the auto scaling group from initiating further scaling activities immediately after a scaling action completes.

• Purpose: Allows metrics to stabilize after new instances launch (which initially have low CPU) before evaluating the need for another scale-out event. This prevents thrashing—rapid, unnecessary oscillation in instance count.
• Typically, cooldown periods are different for scale-in and scale-out events.

Horizontal scaling, or scaling out, is the foundational practice of adding more identical instances or nodes to a system to increase capacity. This is the primary mechanism used by auto scaling systems.

• Contrasts with vertical scaling (scaling up), which involves upgrading the resources (CPU, RAM) of a single instance.
• Enables stateless application design, where any instance can handle any request, simplifying the scaling process.
• Essential for achieving high availability and fault tolerance in cloud-native architectures.

A load balancer is a critical companion to an auto scaling group. It distributes incoming application traffic across the pool of healthy instances, ensuring no single instance is overwhelmed.

• Performs regular health checks to route traffic only to instances passing the check.
• Works in tandem with auto scaling: as new instances are launched, they are automatically registered with the load balancer; as instances are terminated, they are deregistered.
• Types include Application Load Balancers (ALB) (Layer 7) and Network Load Balancers (NLB) (Layer 4).

A scaling policy is the rule or set of rules that defines when and how an auto scaling group should add or remove instances. It is the core logic of the auto scaling system.

• Target Tracking Scaling: Adjusts capacity to keep a specific metric (e.g., average CPU utilization) at a target value.
• Step Scaling: Adds or removes a specified number of instances based on the magnitude of a CloudWatch alarm breach.
• Simple Scaling: Uses a single alarm to trigger a scaling action, then waits for a cooldown period.

A health check is a periodic test performed to determine the operational status of an instance. Auto scaling uses health checks to decide if an instance should be replaced.

• ELB Health Check: Determines if an instance can receive traffic from the load balancer (e.g., responds to HTTP/HTTPS requests).
• EC2 Status Check: Monitors the underlying physical host (system status) and the instance's operating system (instance status).
• If an instance fails a health check consecutively, the auto scaling group can terminate it and launch a replacement.

Desired capacity is the central control variable of an auto scaling group. It represents the ideal number of instances that should be running at any given time.

• The auto scaling system's primary job is to match the actual number of instances to this desired count.
• Manually adjustable, but is more commonly managed automatically by scaling policies.
• Bounded by minimum capacity (the floor) and maximum capacity (the ceiling) settings, which define the operational range for the group.

A cooldown period is a configurable wait time after a scaling activity completes before another scaling activity can start. It prevents the auto scaling group from launching or terminating instances too rapidly.

• Allows time for newly launched instances to fully initialize, boot the application, and begin passing health checks before additional scaling is evaluated.
• Helps stabilize the system and avoid costly thrashing—a cycle of rapid scaling in and out due to metric fluctuations.
• Can be overridden by more advanced features like instance warm-up timers in target tracking policies.