Auto Scaling VMs on AWS, GCP, and Azure

A single VM sized for your busiest hour sits mostly idle the rest of the day. A single VM sized for an average hour falls over at the busiest one. Auto scaling avoids that choice: you describe the VM once, set a floor and a ceiling, and let the platform add machines when load rises and remove them when it falls.

All three big clouds do this in much the same way, with a managed group of identical VMs, a scaling policy and a health check. The names differ, and so do the defaults that decide whether your fleet reacts in time or swings back and forth. This article covers how each one works, how to set it up from the CLI, and the settings that matter most in practice.

It closes the Virtual Machines series. Comparing VM Offerings Across AWS, GCP, and Azure introduced the managed groups, Instance Types and Families Explained covers picking the size each VM in the group should be, and Pricing Models: On-Demand, Reserved, Savings Plans, and Spot explains how to pay for the baseline and the peaks.

The building blocks

Every auto scaling setup has the same four parts:

• A template that describes one VM: image, machine type, disks, network, startup script and identity.
• A group that launches VMs from the template across one or more zones and keeps a target number of them running.
• A policy that changes that target number in response to a metric, a schedule or a forecast.
• Health checks that replace VMs that stop working, usually wired to a load balancer.

flowchart LR
    T["Template"] --> G["Managed group"]
    M["Metrics, schedules, forecasts"] --> P["Scaling policy"]
    P -->|"Sets target size"| G
    G --> Z1["VMs in zone A"]
    G --> Z2["VMs in zone B"]
    Z1 --> LB["Load balancer"]
    Z2 --> LB
    LB -->|"Health checks"| G

The key idea is that you stop managing individual VMs. You never patch or resize a VM in the group by hand; you change the template and let the group roll the change out. Any VM can be deleted at any time, by the autoscaler or by a failed health check, so the VMs must be stateless or keep their state somewhere else, such as a managed database, object storage or a shared file system.

How scaling decisions are made

There are three ways to change the target size, and most production fleets use more than one.

Reactive, metric-based scaling watches a metric and adjusts capacity to keep it near a target. This is the default choice and handles most unpredictable load. The most useful form is target tracking: you say “keep average CPU at 60 percent” and the platform works out how many VMs that takes. The older form is a set of threshold rules (“add two VMs when CPU is above 70 percent for ten minutes”), which gives you more control and more ways to get it wrong.

Scheduled scaling changes the minimum or the target size at set times. It suits load you can see on a calendar: business hours, a nightly batch window, a known sales event.

Predictive scaling looks at past load, forecasts the next hours and adds capacity before the load arrives. It helps when your VMs take minutes to become ready and your traffic follows a daily or weekly cycle. It doesn’t help with sudden, one-off spikes, which only reactive scaling can catch.

flowchart TB
    L["Your load pattern"] --> Q1{"Repeats daily or weekly?"}
    Q1 -->|"Yes"| Q2{"VMs slow to start?"}
    Q1 -->|"No"| R["Reactive target tracking"]
    Q2 -->|"Yes"| PR["Predictive plus reactive"]
    Q2 -->|"No"| R
    L --> Q3{"Known events on a calendar?"}
    Q3 -->|"Yes"| S["Add scheduled scaling"]

The metric you scale on matters as much as the policy. CPU is the easy default and works for compute-bound services. For web services behind a load balancer, requests per VM is often better, because it tracks the work you’re actually doing. For queue workers, the number of messages waiting per VM is usually the right signal, since CPU can stay low while a backlog grows.

Auto Scaling groups on AWS

An Auto Scaling group launches EC2 instances from a launch template into the subnets you list, spreading them across Availability Zones. It has a minimum, a maximum and a desired capacity. Policies change the desired capacity; the group then launches or terminates instances to match it.

Creating a group across two zones, attached to a load balancer target group:

aws autoscaling create-auto-scaling-group \
  --auto-scaling-group-name web \
  --launch-template LaunchTemplateName=web,Version='$Latest' \
  --min-size 2 \
  --max-size 10 \
  --desired-capacity 2 \
  --vpc-zone-identifier "subnet-0aaa1111,subnet-0bbb2222" \
  --target-group-arns arn:aws:elasticloadbalancing:us-east-1:123456789012:targetgroup/web/0123456789abcdef \
  --health-check-type ELB \
  --health-check-grace-period 300 \
  --default-instance-warmup 120

--health-check-type ELB makes the group replace instances that fail the load balancer’s health check, not just those whose EC2 status checks fail. Without it, an instance whose application has crashed but whose OS is fine stays in service.

Scaling policies

EC2 Auto Scaling has four kinds of policy:

• Target tracking keeps a metric at a target value. AWS provides predefined metrics for average CPU, average network in and out, and requests per target from an Application Load Balancer, and you can use any CloudWatch metric. AWS creates and manages the CloudWatch alarms for you.
• Step scaling runs your own CloudWatch alarms and adds or removes capacity in steps depending on how far the metric is past the threshold.
• Simple scaling is the original version of step scaling, with a single adjustment and a cooldown after each action. There’s little reason to choose it for new groups.
• Predictive scaling forecasts load from the group’s history and schedules capacity ahead of it. It can run in forecast-only mode first, so you can compare the forecast to real traffic before letting it act.

A target tracking policy for 50 percent average CPU:

aws autoscaling put-scaling-policy \
  --auto-scaling-group-name web \
  --policy-name cpu50 \
  --policy-type TargetTrackingScaling \
  --target-tracking-configuration '{
    "PredefinedMetricSpecification": {
      "PredefinedMetricType": "ASGAverageCPUUtilization"
    },
    "TargetValue": 50.0
  }'

And a scheduled action that raises the floor on weekday mornings, in local time:

aws autoscaling put-scheduled-update-group-action \
  --auto-scaling-group-name web \
  --scheduled-action-name weekday-morning \
  --recurrence "0 7 * * 1-5" \
  --time-zone "America/Edmonton" \
  --min-size 4

Raising the minimum rather than setting a fixed desired capacity is deliberate. Target tracking can still add more VMs above the new floor if traffic is heavier than usual.

Warmup and other settings worth knowing

Default instance warmup tells the group how long a new instance takes before its metrics are meaningful. Until then, the instance doesn’t count toward the group’s aggregated metrics, which stops a booting instance’s high or low CPU from triggering another scaling action. It isn’t set unless you set it, and AWS recommends you do; if you don’t know the right number, its documentation suggests starting at 300 seconds and measuring.

A few others come up often:

• Mixed instances policies let one group use several instance types and a mix of on-demand and Spot capacity, which is how the Spot fleets in the pricing article are built.
• Instance refresh replaces instances in batches after you change the launch template, keeping a minimum percentage healthy throughout.
• Lifecycle hooks pause an instance on launch or termination so you can run setup or drain connections first.
• Termination policies decide which instance goes first on scale-in, and scale-in protection keeps chosen instances from being removed.

Managed instance groups on Google Cloud

A managed instance group creates identical VMs from an instance template. A zonal MIG keeps all its VMs in one zone; a regional MIG spreads them across zones in a region and is the right default for anything that needs to stay up. The autoscaler is a separate resource attached to the MIG.

gcloud compute instance-groups managed create web-mig \
  --region=us-central1 \
  --template=web-template \
  --size=2 \
  --health-check=web-hc \
  --initial-delay=120

gcloud compute instance-groups managed set-autoscaling web-mig \
  --region=us-central1 \
  --min-num-replicas=2 \
  --max-num-replicas=10 \
  --target-cpu-utilization=0.6 \
  --cool-down-period=90

Two details catch people. The CPU target is a fraction, so 0.6 means 60 percent. And the health check on the MIG is for autohealing: it recreates VMs that fail it, and --initial-delay gives new VMs time to boot before the first check counts.

Signals and timing

The autoscaler can scale on CPU utilization, on load balancer serving capacity (a fraction of the backend’s configured capacity, such as a maximum rate of requests per second per VM), and on Cloud Monitoring metrics, including your own. With several signals configured, it calculates the size each signal needs and uses the largest.

Three timing settings shape its behaviour:

• Initialization period, set with --cool-down-period, defaults to 60 seconds. For scale-out decisions the autoscaler ignores data from VMs still in this period. Set it to how long your application really takes to start serving.
• Stabilization period defaults to 10 minutes and applies only to scale-in. The autoscaler bases its scale-in decisions on the peak load over that window, so a brief dip doesn’t remove VMs it will need again a minute later.
• Scale-in controls cap how many VMs can be removed within a time window, for services that take a long time to warm caches.

gcloud compute instance-groups managed update-autoscaling web-mig \
  --region=us-central1 \
  --scale-in-control=max-scaled-in-replicas=2,time-window=600

Schedules and predictions

Scaling schedules set a minimum number of VMs for a period, defined with cron syntax and a time zone. Like a scheduled raise of the minimum on AWS, they set a floor rather than a fixed size.

gcloud compute instance-groups managed update-autoscaling web-mig \
  --region=us-central1 \
  --set-schedule=business-hours \
  --schedule-cron="0 7 * * MON-FRI" \
  --schedule-duration-sec=36000 \
  --schedule-min-required-replicas=6 \
  --schedule-time-zone="America/Edmonton"

Predictive autoscaling is turned on with --cpu-utilization-predictive-method=optimize-availability and works only when CPU utilization is the scaling signal. It uses the initialization period to decide how far ahead to add VMs: with a 300-second period, VMs are created about five minutes before the forecast load. That makes an accurate initialization period doubly important.

The autoscaler also has modes. only-scale-out stops it from removing VMs, which is useful while you’re still learning how a new service behaves, and off keeps the configuration but stops all scaling.

Virtual Machine Scale Sets on Azure

A Virtual Machine Scale Set runs a group of VMs that you can spread across availability zones. Scale sets have two orchestration modes. Flexible orchestration is the one Microsoft recommends for new workloads and lets you mix VM sizes and manage the VMs with the standard VM APIs. Uniform orchestration is the older model, built for large fleets of identical VMs. Both work with autoscale.

az vmss create \
  --resource-group web-rg \
  --name web-vmss \
  --image Ubuntu2404 \
  --vm-sku Standard_D2s_v5 \
  --orchestration-mode Flexible \
  --instance-count 2 \
  --zones 1 2 3 \
  --admin-username azureuser \
  --generate-ssh-keys

Autoscale rules and profiles

Autoscale on Azure belongs to Azure Monitor rather than to the scale set itself. An autoscale setting attaches to the scale set and holds one or more profiles. Each profile has a minimum, a maximum and a default instance count, and a set of rules. You can add profiles that apply on a recurring schedule or on fixed dates, which is how Azure does scheduled scaling.

Rules are threshold-based. Each says which metric to watch, how to aggregate it over what window, and what to do when it crosses the threshold:

az monitor autoscale create \
  --resource-group web-rg \
  --resource web-vmss \
  --resource-type Microsoft.Compute/virtualMachineScaleSets \
  --name web-autoscale \
  --min-count 2 \
  --max-count 10 \
  --count 2

az monitor autoscale rule create \
  --resource-group web-rg \
  --autoscale-name web-autoscale \
  --condition "Percentage CPU > 70 avg 10m" \
  --scale out 2

az monitor autoscale rule create \
  --resource-group web-rg \
  --autoscale-name web-autoscale \
  --condition "Percentage CPU < 30 avg 10m" \
  --scale in 1

Host metrics such as Percentage CPU, network and disk throughput are available without installing anything on the VMs. Metrics from inside the guest, like memory use, need the Azure Monitor agent, and you can also scale on other resources’ metrics, such as the length of a Service Bus queue. Each rule has a cooldown, five minutes by default in the CLI, during which autoscale won’t act on that rule again.

Because the rules are thresholds rather than a target, you have to keep the scale-out and scale-in thresholds far enough apart. If scaling in would push the metric straight back over the scale-out threshold, the fleet would swing up and down. Azure calls this flapping and checks for it: before scaling in, it estimates what the metric would be afterwards and skips the action if it would trigger a scale-out. A wide gap, like 70 and 30 above, keeps that check from blocking scale-in all the time.

Predictive autoscale and scale-in behaviour

Predictive autoscale for scale sets forecasts CPU load from history and adds VMs ahead of it. It supports only the Percentage CPU metric with average aggregation, only scales out, so your standard rules still handle scale-in, and needs some history: Microsoft says forecasts reach full accuracy after about 15 days. Like AWS, it has a forecast-only mode for checking the predictions first.

The scale-in policy chooses which VMs are removed first: the default, which balances across zones and then removes the highest instance IDs, the newest VMs, or the oldest. Instance protection keeps specific VMs from being removed by scale-in, and automatic instance repairs replace VMs that fail an application health probe or the Application Health extension.

Comparing the three

The practical difference is the reactive model. AWS and Google Cloud let you name a target and do the arithmetic for you, which is hard to misconfigure. Azure’s threshold rules give you more control over step sizes, but you have to choose the thresholds, windows and cooldowns yourself and keep them consistent.

Tuning a group that scales well

The defaults are a starting point. These are the settings that most often decide whether scaling is useful.

1. Measure startup time. Time from launch to the VM passing its health check and serving traffic. Use it for the warmup, initialization period and health check delay. Too short, and the autoscaler reacts to VMs that aren’t ready; too long, and it scales late.
2. Make startup fast. Bake dependencies into the image instead of installing them in a startup script. A VM that is ready in 60 seconds needs much less spare capacity than one that takes ten minutes.
3. Pick a target with headroom. A 50 to 70 percent CPU target leaves room for traffic to rise while new VMs boot. A 90 percent target looks efficient until the first spike.
4. Set a sensible floor. The minimum should cover your quiet-hours load across at least two zones, so a zone failure doesn’t take you to zero.
5. Set the ceiling on purpose. The maximum protects your budget and your downstream systems. A database sized for 20 web VMs won’t cope with 200, and check that your regional vCPU quota allows the maximum.
6. Scale out fast, scale in slowly. Adding capacity early costs a little money; removing it too soon costs availability. Every platform’s defaults lean this way, and so should your changes.
7. Drain before terminating. Use the load balancer’s connection draining or deregistration delay, plus lifecycle hooks or termination notices, so in-flight requests finish.

sequenceDiagram
    participant M as Metrics
    participant A as Autoscaler
    participant G as Group
    participant V as New VM
    participant LB as Load balancer
    M->>A: CPU above target
    A->>G: Raise target size
    G->>V: Launch from template
    V->>V: Boot and start app
    LB->>V: Health check passes
    LB->>V: Send traffic
    A->>A: Ignore VM metrics until warmup ends

Then test it. Run a load test that ramps traffic up and down, and watch how long scale-out takes, whether latency or error rates rise while it happens, and whether the group settles or oscillates. The scaling activity history in each console (Auto Scaling group activity, the MIG’s autoscaler logs and the Azure autoscale run history) shows why each decision was made.

Common mistakes

• Keeping state on the VMs. Sessions, uploads or caches that only exist on one VM are lost when it’s removed. Move them to a shared service before turning on scaling.
• Scaling on the wrong metric. CPU-based scaling for an I/O-bound or queue-driven service will never react. Scale on the signal that tracks the work.
• One zone only. A zonal group can’t survive a zone failure, however well it scales.
• Health checks that are too strict or too loose. A check that depends on the database will mark every VM unhealthy when the database blips, and the group will replace the whole fleet. Check the VM’s own readiness, not every dependency.
• Editing VMs by hand. The next scale-out or repair uses the template, so manual changes disappear. Change the template and roll it out.
• Forgetting quotas. A maximum of 50 VMs means nothing if the account’s regional vCPU quota stops at 32.

Further reading

• Amazon EC2 Auto Scaling user guide
• Target tracking scaling policies for Amazon EC2 Auto Scaling
• Set the default instance warmup for an Auto Scaling group
• Compute Engine autoscaling groups of instances
• Compute Engine scaling based on predictions
• Azure Virtual Machine Scale Sets autoscale overview
• Understand autoscale settings in Azure Monitor
• Predictive autoscale for Virtual Machine Scale Sets

To try the CLI commands above, the first-VM guides for EC2, Compute Engine and Azure cover the account and network setup they assume.

Key takeaways

• Auto scaling on every cloud uses the same parts: a template, a managed group across zones, a scaling policy and health checks. AWS calls the group an Auto Scaling group, Google Cloud a managed instance group and Azure a Virtual Machine Scale Set.
• VMs in a scaled group must be disposable. Keep state elsewhere and make every change through the template.
• Use reactive scaling for unpredictable load, schedules for calendar-driven load and predictive scaling for daily cycles with slow-starting VMs. Most fleets combine them.
• AWS and Google Cloud centre on target tracking. Azure uses threshold rules, so keep its scale-out and scale-in thresholds well apart to avoid flapping.
• Set warmup and initialization periods from measured startup time. Google Cloud defaults to 60 seconds and AWS recommends starting at 300 seconds if you don’t know.
• Scale out quickly and in slowly: stabilization periods, scale-in controls and cooldowns exist to stop a fleet from removing capacity it needs again minutes later.
• Choose the floor for zone resilience and the ceiling for budget, downstream limits and quota, then load test before trusting either.