Autoscaler performance and tuning

Autoscaling stays fast on a small cluster no matter how it is set up. As the number of scaled applications grows into the thousands, the layout of the control loop starts to decide how quickly a workload reacts. This page is a tuning guide built from benchmark results: it shows how scale-decision latency behaves as the population grows, and the two levers that keep it fast, the autoscaler’s client-side Kubernetes API request rate (qps), and sharding the control loop across many tenant Kedify Pod Autoscalers (KPA). The numbers come from a single test environment on AKS with a provisioned (paid tier) control plane, and are meant to illustrate the trend, not to serve as guaranteed figures. Measure your own workload for capacity planning.

Key takeaways

• One control loop hits a wall at scale; sharding does not. A single KEDA or KPA controller stays fast to about 1,000 applications, slows past 2,500, and stops completing near 5,000 without tuning. Multi-tenant KPA stayed at the sync floor on warm decisions and near-instant on cold start from 100 to 5,000 applications, with no tuning.
• Lever one, request rate. Raise a single controller’s kube-api-qps / burst as its population grows (roughly 100 / 200 by 1,000 objects, 200 / 400 by 5,000). It helps, but it does not remove the ceiling and it runs into the shared etcd write limit past about qps 250.
• Lever two, sharding, is the general answer. Splitting the population across tenant KPAs keeps every shard small, so none needs tuning and none stresses etcd. Multi-tenant KPA was the only layout that was fast on both warm decisions and cold start across the whole range.

What was compared

Three layouts drove the same generated population of ScaledObjects, each with a metric trigger, across a range of application counts:

• Native HPA (HPA): one cluster-wide Kedify KEDA installation emitting Kubernetes HPAs, reconciled by the cluster’s single HPA controller.
• Single KPA (sKPA): one cluster-wide KEDA installation with a single Kedify Pod Autoscaler reconciling every workload.
• Multi-tenant KPA (mtKPA): ten tenant KEDA operators, each paired with its own KPA, so the population is sharded across ten independent control loops.

“Native HPA” here means this KEDA-emitted HPA driven by an external metric, not a hand-authored HPA. Driving an HPA from an external metric is what KEDA is for, so the HPA layout is the realistic baseline, and it exercises the same external-metrics path the doc measures.

Two things are measured. Warm decision latency is, with autoscalers already established, how long from a synchronized metric change to the autoscaler acting on it. Cold start covers creating all autoscalers at once and reaching a first scale action. Because the metric change is large enough to move every object, the autoscaler’s lastScaleTime advances on each one, and that timestamp is what we measure against, so the numbers reflect a real scale action rather than an internal decision that may or may not have happened. Both isolate the autoscaling control loop from pod scheduling. Percentiles are across every measured object, so p99 reflects the slowest one percent.

Lever one: set the client-side request rate for the population

Every autoscaler reconcile makes several Kubernetes API calls, and those calls pass through a client-side rate limiter (qps and burst). At the out-of-box default of 20 qps, a single controller that owns thousands of ScaledObjects starves on that limiter, and warm decisions stretch out. Raising the rate removes most of the bottleneck: a single KPA at 1,000 applications drops from 47s to 9s at the median, and at 2,500 from 118s to 23s.

Set the rate to match how many objects one controller reconciles. Because a multi-tenant layout shards the population, each shard owns only its slice and never approaches the limiter, so it needs no tuning at all.

Layout	What to tune	Recommended kube-api-qps / burst
Native HPA	KEDA operator flags	Scale with the population: 100 / 200 by ~1,000 objects, 200 / 400 by ~5,000. Keep ≤ ~250 to stay under the etcd write ceiling. The in-tree HPA controller is cluster-managed and not tunable; the operator rate is the lever, because KEDA serves the metric.
Single KPA	KEDA operator and the KPA, same flags	Same schedule as above; the one KPA owns every workload.
Multi-tenant KPA	nothing per shard	Leave the default 20 / 30. Shard so each KPA owns at most ~1,000 objects, then add shards as the population grows rather than raising any one rate.

Raising the rate helps but does not by itself remove the ceiling: at 2,500 applications a single KPA is still 23s, against 8s for the sharded layout, and the rate has to keep climbing with the population (5,000 on one controller needs qps=200 just to complete). The band is also bounded from above, because past roughly qps=250 the writes outrun etcd. The rest of this page shows where that ceiling appears and why sharding sidesteps it. All results below use a tuned rate (qps=100, or qps=200 where noted at 5,000) so the comparison reflects the control loop rather than the default rate limiter.

Warm decision latency

The chart and table below show warm latency as deviation above the sync floor, not absolute time. A warm decision cannot beat the autoscaler’s evaluation interval: with a 15-second sync period and a metric change that lands at a random point in each object’s cycle, the best achievable median is about 7.5s and the best p99 about 15s (see How this was measured). Showing the excess over that floor makes the impact of scale obvious, <1s means a layout is essentially at the floor, and any larger number is delay the control loop is adding on top.

Applications	HPA			sKPA			mtKPA
	p50	p95	p99	p50	p95	p99	p50	p95	p99
100	<1s	<1s	<1s	<1s	<1s	<1s	<1s	<1s	<1s
500	<1s	<1s	<1s	+2s	+1s	<1s	<1s	<1s	<1s
1,000	<1s	<1s	<1s	+2s	+2s	+1s	<1s	+1s	<1s
2,500	+10s	+20s	+19s	+16s	+30s	+31s	<1s	<1s	<1s
5,000	+34s†	+63s†	+66s†	+16s†	+31s†	+32s†	<1s	+1s	<1s

at the floor (<3s over)small (3–10s)large (11–30s)severe (>30s)

^† At 5,000, native HPA and single KPA do not complete at the qps=100 used for every other cell; those two values are measured at qps=200. Multi-tenant KPA is at qps=100 throughout. The same convention applies to the cold-start table below.

Through 1,000 applications every layout sits at the floor, all in the green. The separation appears at 2,500: both single-controller layouts add 10 to 30 seconds on top, while the sharded layout is still at the floor, the same as at 100 applications. Multi-tenant KPA holds there at 5,000 with no change in settings, whereas the single-controller layouts do not complete at 5,000 at qps=100 at all and reach the numbers above (marked ^†) only with the request rate raised to 200. This is the single-controller ceiling: on warm decisions it shows up once one controller owns a few thousand workloads, and by 5,000 it needs twice the request rate just to keep up.

Cold start and fan-out: where sharding wins

Cold start is the clearest separation. This table shows absolute time to a first scale action (ideal is near zero), colored on the same good-to-bad scale:

Applications	HPA			sKPA			mtKPA
	p50	p95	p99	p50	p95	p99	p50	p95	p99
100	17s	20s	20s	2s	5s	5s	<1s	1s	1s
500	34s	53s	55s	19s	38s	40s	<1s	2s	2s
1,000	56s	103s	107s	41s	88s	92s	<1s	7s	7s
2,500	141s	406s	452s	127s	400s	447s	1s	26s	27s
5,000	129s†	394s†	443s†	112s†	378s†	427s†	1s	5s	25s

near-instant (≤2s)small (3–10s)large (11–30s)severe (>30s, up to minutes)

^† At 5,000, native HPA and single KPA do not complete at qps=100; those values are measured at qps=200 (mtKPA stays at qps=100). Every other cell is qps=100.

When thousands of autoscalers are created at once, single-controller layouts queue every workload behind one reconcile loop, and the wait climbs with the population: 56s to 141s at the median from 1,000 to 2,500 applications, with tails past 400s. The sharded layout reached a first scale action in about a second at the median across every count measured, essentially flat, because autoscaler creation and evaluation spread across ten independent control loops instead of one.

At 5,000, at the qps=100 used everywhere else, neither single-controller layout completes: their autoscaler-creation rate collapses as the backlog grows and both plateau near 2,950 of 5,000, while the multi-tenant layout sets up all 5,000 and reaches first actions in about a second, unchanged from smaller counts. The 5,000 figures marked ^† are with the request rate raised to 200, which is enough to get a single controller there, but only within a band before the control plane itself becomes the limit. The next section explains why, and why sharding avoids the problem entirely.

Why one control loop stops keeping up

The reason a single controller does not just slow down but stalls is worth stating directly, because it is the core of the result. One KEDA operator reconciles the entire ScaledObject population through a single work queue. As the population grows, two costs compound. First, the initial creation backlog is N objects deep. Second, once created, every object is periodically re-reconciled (resync), so the steady-state maintenance load also grows with N. Past a few thousand objects the workers spend nearly all their time re-reconciling what already exists, leaving almost no throughput for creating the rest, and the creation rate decays toward zero. At the tuned qps=100 both single-controller layouts plateaued near 2,950 of 5,000 and never finished.

What is the limiting resource? Not worker count. Raising maxConcurrentReconciles from 5 to 20 made a single KPA start faster (about 536 versus 190 new autoscalers per minute) but it collapsed to the same ~2,950 plateau: more workers cannot exceed the request budget they share. The limiting resource is the client-side request rate. Raising the operator to qps=200 let both single-controller layouts create all 5,000 and measure cleanly, at the numbers in the 5,000 rows of the tables above: a single KPA at warm 24s / cold 112s, and native HPA at warm 41s / cold 129s (p50). That is roughly each layout’s 2,500-application latency, reached with more headroom. The rate has to scale with the population, though: qps=150 was not enough (a single KPA plateaued near 4,300 of 5,000), and qps=100 plateaus near 2,950. So a single controller needs twice the default request rate to reach 5,000, and more as the population grows, while the sharded layout reached the same 5,000 at the default rate with no tuning.

That headroom is not unlimited, because the bottleneck then moves to the control plane. At qps=300 the writes outran etcd, which began dropping raft proposals (rpc error: raft proposal dropped) and failed the run. A single controller therefore has a usable band between too-slow (starved on qps) and too-fast (overrunning etcd), and the band narrows as the population grows.

These two limits sit at different levels, which is the crux of the write story. kube-api-qps is a per-writer client-side limit: every controller process has its own. Etcd’s write capacity is cluster-wide and shared: it commits writes from every client at once, so what matters for etcd is the combined rate across all controllers, the API server, and any bulk apply happening at the same time. Raising one controller’s qps concentrates more of that shared budget onto a single writer, which is why a lone controller pushed to qps=300 tipped etcd over. Sharding moves the same total write volume through many writers that each stay at a modest qps, so no single writer has to burst hard enough to threaten the shared etcd ceiling.

Multi-tenant KPA removes the problem by horizontal scaling of the control loop itself. Ten tenant operators each own about a tenth of the population, each with its own worker pool, its own client-side rate limiter, and its own modest share of etcd writes, so every shard stays comfortably inside that band. That is why mtKPA is flat at the floor on warm decisions and about 1s cold at 5,000 with default settings, while a single controller needs careful tuning to get there at all. It is not a faster loop, it is more loops, and adding loops is the general answer where tuning one loop is not.

Less load on the shared metrics path

There is also an architectural reason KPA scales well. Native HPA reads external metrics through the KEDA metrics API server (the external.metrics.k8s.io aggregated API), so that component does more work as the HPA population grows. KPA reads metrics directly from its paired KEDA operator over gRPC and skips that hop, taking the shared aggregated API server out of the autoscaling hot path.

Putting it together

Two levers keep autoscaling fast as a cluster grows. Raise the client-side request rate so a controller is not starved by the default limiter. Shard the control loop across tenant KPAs so no single controller owns the whole population. Raising the rate alone helps but does not remove the ceiling; a single controller still climbs on warm decisions past a couple thousand applications and stalls on cold fan-out. A multi-tenant KPA layout applies both levers at once and was the only layout that stayed at the floor on warm decisions and near-instant on cold start across the whole range, from 100 to 5,000 applications, with no tuning.

How this was measured

• Platform: a dedicated AKS cluster, Kubernetes 1.34, on a provisioned (Standard, paid tier) control plane. Control-plane sizing matters; on an under-provisioned control plane every layout is slower.
• Nodes: the scaled workloads ran on 48 worker nodes (Standard_D2ds_v5 and Standard_D2s_v5, 2 vCPU / 8 GiB, 250 pods per node), spread across two VM families. They hold the target Deployments, up to about 10,000 pods at 5,000 applications scaling to 2 replicas each; the pods are lightweight, so the sizing is bounded by pod count, not CPU. The KEDA operator, KPA controllers, and the metric backend ran on a separate small controller pool (3 to 6 Standard_D2ds_v5 nodes; the multi-tenant layout runs its ten tenant operators there).
• Population: generated ScaledObjects, each with one metric trigger, most driven by a shared metric source whose value could be changed in one step to create a synchronized scale event. A representative metrics-api object:

apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  name: target-00001
  namespace: tenant-3
  annotations:
    autoscaling.kedify.io/class: kpa   # "hpa" selects the native-HPA layout
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: target-00001
  pollingInterval: 30
  minReplicaCount: 1
  maxReplicaCount: 2
  advanced:
    horizontalPodAutoscalerConfig:
      behavior:                        # no stabilization, so warm latency
        scaleUp: { stabilizationWindowSeconds: 0 }    # reflects the ~15s
        scaleDown: { stabilizationWindowSeconds: 0 }  # sync interval, nothing else
  triggers:
    - type: metrics-api
      metadata:
        url: http://metric-backend.<control-namespace>.svc:8080/metric
        format: json
        valueLocation: output
        targetValue: "1"

• The sync floor: the autoscaler evaluated on a 15-second sync interval (the KPA syncPeriod, and the Kubernetes HPA controller’s default). Because the metric change is not correlated with each object’s sync phase, a warm decision lands at a uniformly random point in that 15s window, so the best achievable warm latency is about 7.5s at the median and about 15s at p99 whenever a controller is not otherwise bottlenecked. The warm table reports latency as the excess over this floor.
• Decision latency: measured from the metric change to the autoscaler’s lastScaleTime, which records when the scaling action was applied. The synchronized metric change is large enough to move every object, so a scale action does occur for each. Cold metrics are measured from each ScaledObject’s own creation timestamp. All latencies derive from second-granular Kubernetes timestamps, so a value shown as <1s means the action landed within the same second.
• Percentiles are computed across every measured object, so p99 reflects the slowest one percent of applications, not an average.
• Two client-side request rates were tested: the default 20 qps / 30 burst, and a tuned 100 qps / 200 burst, with 200 / 400 added at 5,000 for the single-controller layouts.

These are rough measurements from one environment. Treat them as directional evidence that a fast autoscaling installation at scale needs an adequate client-side request rate and a sharded control loop, both of which a multi-tenant KPA layout provides. Measure your own workload before making capacity decisions.