Scale vCluster workloads from central KEDA

Configure Kedify Agent once to discover vClusters hosted in the central cluster automatically. Run KEDA alongside it, and use a separate DistributedScaledObject (DSO) for each workload in each member vCluster. Each DSO explicitly selects one member cluster.

This walkthrough uses two shared-node vClusters hosted in the central cluster. The application Deployments belong to the vClusters. Their pods run on shared host nodes through vCluster’s normal synchronization.

There are no application Deployment targets in the central Kubernetes API. KEDA’s generated ScaledObjects and HPAs stay central and target the DSOs. The central cluster also holds a ConfigMap with one metric value per workload for this exercise.

Prerequisites

• A central cluster with Kedify KEDA and Kedify Agent installed in keda. The Agent image and chart must support automatic vCluster discovery; Agent v0.7.0 does not include this feature.
• kubectl, Helm, jq, and the vCluster CLI 0.36.0.
• A default StorageClass that can provision persistent volumes for the vCluster control planes.

The manifests below pin vCluster OSS 0.36.0 and virtual Kubernetes 1.34.3. Review vCluster’s shared-node prerequisites for your host cluster.

1. Enable automatic discovery

Merge these values into your central Agent’s existing Helm configuration and apply your usual Helm or GitOps upgrade:

global:
  features:
    multicluster:
      enabled: true
      vClusterDiscoveryEnabled: true

This is a one-time setup. The chart grants the Agent get, list, and watch access to host Secrets and Services. Discovery reads credentials directly from the vCluster namespaces and keeps connections in memory. The Agent discovers both existing vClusters and new ones as they appear, using vCluster’s native Secrets and labels.

2. Create the member vClusters

Save as vcluster.yaml:

controlPlane:
  distro:
    k8s:
      enabled: true
      image:
        tag: v1.34.3
  statefulSet:
    image:
      repository: loft-sh/vcluster-oss
    resources:
      requests:
        cpu: 100m
        memory: 256Mi
      limits:
        memory: 768Mi
    persistence:
      volumeClaim:
        enabled: true
telemetry:
  enabled: false

The control planes use persistent volumes from the host cluster’s default StorageClass. If your cluster has no default, set controlPlane.statefulSet.persistence.volumeClaim.storageClass to an available StorageClass name.

Choose either installation method. Both install the same vCluster Helm chart and produce the native Service and kubeconfig Secret used by automatic discovery.

Helm chart

for member in a b; do
  helm upgrade --install "vc-$member" vcluster \
    --repo https://charts.loft.sh --version 0.36.0 \
    --namespace "vcluster-$member" --create-namespace \
    --values vcluster.yaml \
    --wait --timeout 5m
done

vCluster CLI

for member in a b; do
  vcluster create "vc-$member" \
    --driver helm \
    --namespace "vcluster-$member" \
    --chart-version 0.36.0 \
    --values vcluster.yaml \
    --connect=false
done

The Agent reads each vCluster’s default kubeconfig export and connects through its internal Service with TLS verification enabled. It leaves the native export unchanged, so vcluster connect continues to work.

3. Verify automatic discovery

Check member status in the central cluster:

kubectl -n keda get kedifyconfiguration -o json |
  jq -r '["MEMBER", "PROVIDER", "STATE"],
    (.items[].status.multiClusterStatus.clusters // {} | to_entries[] |
      [.key, .value.provider, .value.state]) | @tsv'

Repeat the status command until both vc-a.vcluster-a and vc-b.vcluster-b report Connected through the vcluster provider before continuing. Each alias combines the vCluster name and its host namespace, keeping identically named vClusters in different namespaces distinct.

The Agent reuses vCluster’s default tenant-admin credentials. It manages registration, credential updates when vCluster rotates its export, and removal when a vCluster is deleted.

For dedicated credentials with narrower permissions instead of the exported tenant-admin identity, see the GitOps setup guide.

4. Deploy one workload per vCluster

Use the vCluster CLI’s command mode for tenant operations. It handles the temporary connection and kubeconfig for each command without changing your current central context:

for member in a b; do
  vcluster connect "vc-$member" -n "vcluster-$member" -- kubectl create namespace scaling-demo
  vcluster connect "vc-$member" -n "vcluster-$member" -- kubectl apply -f - <<EOF_WORKER
apiVersion: apps/v1
kind: Deployment
metadata:
  name: worker-$member
  namespace: scaling-demo
spec:
  replicas: 0
  selector:
    matchLabels:
      app: worker-$member
  template:
    metadata:
      labels:
        app: worker-$member
    spec:
      containers:
        - name: worker
          image: registry.k8s.io/pause:3.10
          resources:
            requests:
              cpu: 1m
              memory: 4Mi
            limits:
              memory: 16Mi
EOF_WORKER
done

These pods only demonstrate replica changes. Replace them with your workers for a real metric source.

5. Create DistributedScaledObject

Most KEDA scalers can provide metrics for a DSO, including queues, Prometheus, and cloud event sources. Configure the metric endpoint and authentication for access from central KEDA. CPU and memory triggers require pod metrics from the cluster running the workload, so they cannot measure these tenant pods through the central DSO. The HTTP add-on also needs separate traffic-routing integration and is outside this walkthrough.

For a self-contained example, use Kedify’s kubernetes-resource scaler to read one ConfigMap value per workload.

Save as metrics.yaml and apply it to the central cluster:

apiVersion: v1
kind: Namespace
metadata:
  name: scaling-demo
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: demo-metrics
  namespace: scaling-demo
data:
  a: "0"
  b: "0"

kubectl apply -f metrics.yaml

for member in a b; do
  kubectl apply -f - <<EOF_DSO
apiVersion: keda.kedify.io/v1alpha1
kind: DistributedScaledObject
metadata:
  name: worker-$member
  namespace: scaling-demo
spec:
  memberClusters:
    - name: vc-$member.vcluster-$member
  scaledObjectSpec:
    scaleTargetRef:
      kind: Deployment
      name: worker-$member
    minReplicaCount: 0
    maxReplicaCount: 5
    pollingInterval: 5
    cooldownPeriod: 15
    advanced:
      horizontalPodAutoscalerConfig:
        behavior:
          scaleDown:
            stabilizationWindowSeconds: 0
    triggers:
      - type: kubernetes-resource
        metadata:
          resourceKind: ConfigMap
          resourceName: demo-metrics
          key: $member
          targetValue: "1"
EOF_DSO
done

Selecting memberClusters restricts scaling to a subset of registered tenant clusters. Omitting the list selects all registered members and distributes the workload according to the configured strategy. Repeat this single-member pattern for additional workloads. By default, each target Deployment must exist in the same namespace as its DSO.

6. Verify independent scaling

Set different metric values:

kubectl -n scaling-demo patch configmap demo-metrics \
  --type merge -p '{"data":{"a":"3","b":"1"}}'

Wait for A to reach three available replicas and B to reach one:

vcluster connect vc-a -n vcluster-a -- kubectl -n scaling-demo wait deployment/worker-a \
  --for=jsonpath='{.status.availableReplicas}'=3 --timeout=180s
vcluster connect vc-b -n vcluster-b -- kubectl -n scaling-demo wait deployment/worker-b \
  --for=jsonpath='{.status.availableReplicas}'=1 --timeout=180s

Then bring A to zero while increasing B:

kubectl -n scaling-demo patch configmap demo-metrics \
  --type merge -p '{"data":{"a":"0","b":"2"}}'

vcluster connect vc-a -n vcluster-a -- kubectl -n scaling-demo wait deployment/worker-a \
  --for=jsonpath='{.spec.replicas}'=0 --timeout=180s
vcluster connect vc-b -n vcluster-b -- kubectl -n scaling-demo wait deployment/worker-b \
  --for=jsonpath='{.status.availableReplicas}'=2 --timeout=180s
vcluster connect vc-a -n vcluster-a -- kubectl -n scaling-demo get pods
kubectl -n scaling-demo get dso,scaledobject,hpa

Allow for polling, HPA reconciliation, pod startup, and termination. A’s pods should disappear while B remains scaled. The short cooldown and stabilization settings are for this exercise; choose suitable values for real workloads.

Finally, return both workloads to zero:

kubectl -n scaling-demo patch configmap demo-metrics \
  --type merge -p '{"data":{"a":"0","b":"0"}}'
for member in a b; do
  vcluster connect "vc-$member" -n "vcluster-$member" -- kubectl -n scaling-demo wait "deployment/worker-$member" \
    --for=jsonpath='{.spec.replicas}'=0 --timeout=180s
  vcluster connect "vc-$member" -n "vcluster-$member" -- kubectl -n scaling-demo wait pod \
    -l "app=worker-$member" --for=delete --timeout=120s
done

Verify that each member has only its own application Deployment and no KEDA installation:

vcluster connect vc-a -n vcluster-a -- kubectl get deployments -A
vcluster connect vc-b -n vcluster-b -- kubectl get deployments -A
vcluster connect vc-a -n vcluster-a -- kubectl get crds
vcluster connect vc-b -n vcluster-b -- kubectl get crds

Troubleshooting and scope

• Member does not appear: check that the Agent image and chart support discovery, both feature values from step 1 are enabled, and the chart’s RBAC is installed. This integration expects the standard shared-node Helm export vc-<name> and Service <name> in the vCluster’s host namespace, with their native vCluster labels. Custom export names and externally hosted or private-node vClusters are outside this walkthrough.
• Member does not connect: check the API endpoint from the Agent pod’s network, TLS SANs, the CA, and the permissions of the exported credentials. A successful vcluster connect command does not establish Agent connectivity.
• DSO reports a missing Deployment: inspect status.memberClusterStatuses and check the target name and namespace in the member API. Do not target the host’s synchronized pod names.
• Metrics fail: check that the central scaling-demo/demo-metrics ConfigMap exists and its a and b values are numeric. The kubernetes-resource scaler requires Kedify KEDA. For other metric sources, ensure the endpoint and authentication work from central KEDA.
• Member API is unavailable: its workloads cannot receive replica updates. A DSO pinned to that member has no other scaling destination. Check per-member status before relying on the aggregate replica count.
• Jobs: similar configuration is available for DistributedScaledJob. The global multi-cluster switch enables Kedify KEDA’s required raw metrics service; see the DSJ guide.

Shared-node vClusters share host capacity and infrastructure failures. This walkthrough demonstrates scaling integration. For hard tenant isolation and production control-plane availability, see vCluster tenancy models.

Cleanup

Use these commands only for the disposable resources created by this walkthrough:

kubectl delete -f metrics.yaml
for member in a b; do
  helm uninstall "vc-$member" -n "vcluster-$member"
  kubectl delete namespace "vcluster-$member"
done

Deleting the disposable central scaling-demo namespace removes its DSOs, generated scaling resources, and metric ConfigMap. Deleting the host vCluster namespaces also deletes their persistent volume claims. The Agent removes the discovered members automatically.