# Scale vCluster workloads centrally

Configure Kedify Agent once to discover vClusters hosted in the central cluster automatically. Run KEDA alongside it, and use a separate `DistributedScaledObject` (DSO) for each workload in each member vCluster. Each DSO explicitly selects one member cluster.

This walkthrough selects **distributed** mode. To place ordinary ScaledObjects or ScaledJobs in the vClusters and scale each independently through central KEDA/KPA, use [dedicated mode](https://docs.kedify.io/how-to/vcluster-dedicated-scaling/).

This walkthrough uses two shared-node vClusters hosted in the central cluster. The application Deployments belong to the vClusters. Their pods run on shared host nodes through vCluster’s normal synchronization. Automatic discovery also works with private-node vClusters hosted in the central cluster because both modes expose the same Service and kubeconfig Secret.

[![Central KEDA reads a metric source and uses separate DistributedScaledObjects to scale worker-a in vc-a and worker-b in vc-b through their virtual APIs.](https://docs.kedify.io/assets/images/docs/vcluster-central-scaling.svg)](https://docs.kedify.io/assets/images/docs/vcluster-central-scaling.svg)

Scroll to exploreDiagram description

One host cluster contains the central KEDA and Kedify Agent installation and two member vClusters. The worker-a DSO selects vc-a and the worker-b DSO selects vc-b. Each DSO updates a Deployment through the selected virtual Kubernetes API; the example shows three worker-a replicas and one worker-b replica.

There are no application Deployment targets in the central Kubernetes API. KEDA’s generated ScaledObjects and HPAs stay central and target the DSOs. The central cluster also holds a ConfigMap with one metric value per workload for this exercise.

## Demo

Watch two vClusters join a central Kedify installation automatically, scale their workloads independently, and return to zero.

Use the chapter buttons to jump between steps.

[Open the player](https://docs.kedify.io/assets/demos/vcluster-scaling/watch.html), [download the recording](https://docs.kedify.io/assets/demos/vcluster-scaling/distributed-scaling.cast), or [read the transcript](https://docs.kedify.io/assets/demos/vcluster-scaling/transcript.txt).

## Prerequisites

- A central cluster with a compatible Kedify `0.8.1` release bundle or later installed in `keda`. The registration and CRD bootstrap behavior described here requires that bundle’s Agent and chart.

- The four KEDA CRDs and `kedifypodautoscalers.autoscaling.kedify.io` installed centrally for automatic bootstrap. Enabling the chart’s `kpa.enabled` dependency installs the KPA CRD. If you do not run the optional KPA controller in distributed mode, install the matching KPA CRD from your release bundle separately.

- A working host `metrics.k8s.io` API for the recommended metrics-server integration. Check `kubectl top pods -A` in the host context before creating members.

- A default StorageClass that can provision persistent volumes for the vCluster control planes, or the name of another StorageClass to configure below.

The commands below use vCluster’s version defaults. Keep the CLI current to use the current stable release. For a reproducible deployment, add `--version <version>` to Helm or `--chart-version <version>` to the CLI. Review [vCluster’s shared-node prerequisites](https://www.vcluster.com/docs/vcluster/quick-start/shared-nodes) for your host cluster and selected release.

## 1. Enable automatic discovery

Merge these values into your central Agent’s existing [Helm configuration](https://docs.kedify.io/installation/helm/) and apply your usual Helm or GitOps upgrade:

```yaml
global:
  features:
    multicluster:
      enabled: true
      type: distributed
      vClusterDiscoveryEnabled: true
```

The chart grants the Agent cluster-wide `get`, `list`, and `watch` access to host Secrets and Services, read access to the five source CRDs, and permission to manage registration resources in its installation namespace. The Agent discovers existing and new vClusters through their native Secrets and labels.

With vCluster discovery enabled, Kedify also copies the live central KEDA/KPA CRDs into each vCluster: `scaledobjects.keda.sh`, `scaledjobs.keda.sh`, `triggerauthentications.keda.sh`, `clustertriggerauthentications.keda.sh`, and `kedifypodautoscalers.autoscaling.kedify.io`. After all five are established, it creates a managed kubeconfig Secret and `ClusterRegistration` in `keda`. It periodically reconciles the CRDs, refreshes credentials when the native export changes, and removes the managed registration when its source disappears. The original vCluster Secret remains unchanged.

The distributed provider also maintains direct connections to the native exports. Its `Connected` status does not by itself prove the registration/bootstrap path succeeded. A missing central KPA CRD can leave distributed scaling connected while automatic registration reports `VClusterBootstrapFailed`.

## 2. Create the member vClusters

We recommend **`integrations.metricsServer.enabled=true` by default for shared-node vClusters used with Kedify**. It forwards host CPU/memory metrics into the member and supports dedicated KPA scaling and tenant telemetry. The central ConfigMap/DSO exercise does not consume these metrics, but both commands below enable the integration so members are ready for those uses. This is the guide’s recommended setting; vCluster’s chart default is `false`.

See the [vCluster configuration guide](https://www.vcluster.com/docs/vcluster/configure/what-is-vcluster-yaml) for available options. Both installation methods below produce the native Service and kubeconfig Secret used by automatic discovery.

If the host cluster has no default StorageClass, save this as `vcluster.yaml` and replace `my-storage-class` with an available class:

```yaml
controlPlane:
  statefulSet:
    persistence:
      volumeClaim:
        storageClass: my-storage-class
```

Add `--values vcluster.yaml` to either command below only when you create this optional values file.

Enabling [vCluster’s built-in network policy](https://www.vcluster.com/docs/vcluster/configure/vcluster-yaml/policies/network-policy) does not require an additional ingress rule. The generated control-plane policy permits external access to the virtual API on TCP port `8443`. If you use custom host network policies, allow the Kedify Agent to reach the vCluster control-plane pods on that port.

### Helm chart

```bash
for member in a b; do
  helm upgrade --install "vc-$member" vcluster \
    --repo https://charts.loft.sh \
    --namespace "vcluster-$member" --create-namespace \
    --set integrations.metricsServer.enabled=true \
    --wait --timeout 5m
done
```

### vCluster CLI

```bash
for member in a b; do
  vcluster create "vc-$member" \
    --driver helm \
    --namespace "vcluster-$member" \
    --set integrations.metricsServer.enabled=true \
    --connect=false
done
```

`--connect=false` keeps the current `kubectl` context unchanged; the commands later in this guide connect explicitly. The Agent reads each vCluster’s default kubeconfig export and connects through its internal Service with TLS verification enabled. It leaves the native export unchanged, so `vcluster connect` continues to work.

### Verify the recommended CPU and memory metrics integration

The commands above enable [vCluster’s host metrics-server integration](https://www.vcluster.com/docs/vcluster/configure/vcluster-yaml#integrations). The host must already expose a working `metrics.k8s.io` API; this setting does not install a host metrics-server. For existing members, apply it through their maintained Helm values or vCluster configuration.

Pass the Helm value through the [vCluster CLI’s `--set` flag](https://www.vcluster.com/docs/vcluster/cli/vcluster_create):

```bash
vcluster create my-vcluster \
  --driver helm \
  --namespace my-vcluster \
  --connect=false \
  --set integrations.metricsServer.enabled=true
```

The same `--set integrations.metricsServer.enabled=true` option works with `helm upgrade --install`. For a values file, add:

```yaml
integrations:
  metricsServer:
    enabled: true
```

After connecting and deploying pods, check `kubectl top pods -A` in each member. A missing metrics API or a `NotFound` response prevents CPU and memory evaluation. Utilization targets also require the corresponding container resource requests; the percentage is calculated against those requests. Private-node vClusters need a working metrics API for their own nodes; host shared-node metrics forwarding is not a substitute.

These tenant metrics do not enable CPU or memory triggers on the central DSO. A DSO’s generated ScaledObject targets the DSO in the central API, rather than the member Deployment.

## 3. Verify automatic discovery

Check member status in the central cluster:

```bash
kubectl -n keda get kedifyconfiguration -o json |
  jq -r '["MEMBER", "PROVIDER", "STATE"],
    (.items[].status.multiClusterStatus.clusters // {} | to_entries[] |
      [.key, .value.provider, .value.state]) | @tsv'
```

Repeat the status command until both `vc-a.vcluster-a` and `vc-b.vcluster-b` report `Connected` through the `vcluster` provider before continuing. Each alias combines the vCluster name and its host namespace, keeping identically named vClusters in different namespaces distinct.

The Agent reuses vCluster’s default **tenant-admin credentials**. It manages registration, credential updates when vCluster rotates its export, and removal when a vCluster is deleted.

Also verify automatic registration and CRD bootstrap:

```bash
kubectl -n keda get clusterregistrations
kubectl -n keda wait --for=condition=Ready clusterregistration --all --timeout=120s
for member in a b; do
  vcluster connect "vc-$member" -n "vcluster-$member" -- kubectl get crd \
    scaledobjects.keda.sh scaledjobs.keda.sh \
    triggerauthentications.keda.sh clustertriggerauthentications.keda.sh \
    kedifypodautoscalers.autoscaling.kedify.io
done
```

Confirm a registration exists for each expected member before using the readiness command. For dedicated scaling, central KEDA and KPA must also connect to those ready registrations; see [dedicated verification](https://docs.kedify.io/how-to/vcluster-dedicated-scaling/#3-verify-automatic-registration).

For distributed scaling with a separate identity and narrower permissions, see the [GitOps setup guide](https://docs.kedify.io/how-to/multicluster-gitops/).

## 4. Deploy one workload per vCluster

Use the vCluster CLI’s command mode for tenant operations. It handles the temporary connection and kubeconfig for each command without changing your current central context:

```bash
for member in a b; do
  vcluster connect "vc-$member" -n "vcluster-$member" -- kubectl create namespace scaling-demo
  vcluster connect "vc-$member" -n "vcluster-$member" -- kubectl apply -f - <<EOF_WORKER
apiVersion: apps/v1
kind: Deployment
metadata:
  name: worker-$member
  namespace: scaling-demo
spec:
  replicas: 0
  selector:
    matchLabels:
      app: worker-$member
  template:
    metadata:
      labels:
        app: worker-$member
    spec:
      containers:
        - name: worker
          image: registry.k8s.io/pause:3.10
          resources:
            requests:
              cpu: 1m
              memory: 4Mi
            limits:
              memory: 16Mi
EOF_WORKER
done
```

These pods only demonstrate replica changes. Replace them with your workers for a real metric source.

## 5. Create DistributedScaledObject

Most [KEDA scalers](https://keda.sh/docs/2.20/scalers/) can provide metrics for a DSO, including queues, Prometheus, and cloud event sources. Configure the metric endpoint and authentication for access from central KEDA. CPU and memory triggers require pod metrics from the cluster running the workload, so they cannot measure these tenant pods through the central DSO. The HTTP add-on also needs separate traffic-routing integration and is outside this walkthrough.

For a self-contained example, use Kedify’s `kubernetes-resource` scaler to read one ConfigMap value per workload.

Save as `metrics.yaml` and apply it to the central cluster:

```yaml
apiVersion: v1
kind: Namespace
metadata:
  name: scaling-demo
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: demo-metrics
  namespace: scaling-demo
data:
  a: "0"
  b: "0"
```

```bash
kubectl apply -f metrics.yaml

for member in a b; do
  kubectl apply -f - <<EOF_DSO
apiVersion: keda.kedify.io/v1alpha1
kind: DistributedScaledObject
metadata:
  name: worker-$member
  namespace: scaling-demo
spec:
  memberClusters:
    - name: vc-$member.vcluster-$member
  scaledObjectSpec:
    scaleTargetRef:
      kind: Deployment
      name: worker-$member
    minReplicaCount: 0
    maxReplicaCount: 5
    pollingInterval: 5
    cooldownPeriod: 15
    advanced:
      horizontalPodAutoscalerConfig:
        behavior:
          scaleDown:
            stabilizationWindowSeconds: 0
    triggers:
      - type: kubernetes-resource
        metadata:
          resourceKind: ConfigMap
          resourceName: demo-metrics
          key: $member
          targetValue: "1"
EOF_DSO
done
```

Selecting `memberClusters` restricts scaling to a subset of registered tenant clusters. Omitting the list selects all registered members and distributes the workload according to the configured strategy. Repeat this single-member pattern for additional workloads. By default, each target Deployment must exist in the same namespace as its DSO.

## 6. Verify independent scaling

Set different metric values:

```bash
kubectl -n scaling-demo patch configmap demo-metrics \
  --type merge -p '{"data":{"a":"3","b":"1"}}'
```

Wait for A to reach three available replicas and B to reach one:

```bash
vcluster connect vc-a -n vcluster-a -- kubectl -n scaling-demo wait deployment/worker-a \
  --for=jsonpath='{.status.availableReplicas}'=3 --timeout=180s
vcluster connect vc-b -n vcluster-b -- kubectl -n scaling-demo wait deployment/worker-b \
  --for=jsonpath='{.status.availableReplicas}'=1 --timeout=180s
```

Then bring A to zero while increasing B:

```bash
kubectl -n scaling-demo patch configmap demo-metrics \
  --type merge -p '{"data":{"a":"0","b":"2"}}'

vcluster connect vc-a -n vcluster-a -- kubectl -n scaling-demo wait deployment/worker-a \
  --for=jsonpath='{.spec.replicas}'=0 --timeout=180s
vcluster connect vc-b -n vcluster-b -- kubectl -n scaling-demo wait deployment/worker-b \
  --for=jsonpath='{.status.availableReplicas}'=2 --timeout=180s
vcluster connect vc-a -n vcluster-a -- kubectl -n scaling-demo get pods
kubectl -n scaling-demo get dso,scaledobject,hpa
```

Allow for polling, HPA reconciliation, pod startup, and termination. A’s pods should disappear while B remains scaled. The short cooldown and stabilization settings are for this exercise; choose suitable values for real workloads.

Finally, return both workloads to zero:

```bash
kubectl -n scaling-demo patch configmap demo-metrics \
  --type merge -p '{"data":{"a":"0","b":"0"}}'
for member in a b; do
  vcluster connect "vc-$member" -n "vcluster-$member" -- kubectl -n scaling-demo wait "deployment/worker-$member" \
    --for=jsonpath='{.spec.replicas}'=0 --timeout=180s
  vcluster connect "vc-$member" -n "vcluster-$member" -- kubectl -n scaling-demo wait pod \
    -l "app=worker-$member" --for=delete --timeout=120s
done
```

Verify that each member has only its own application Deployment and no KEDA installation:

```bash
vcluster connect vc-a -n vcluster-a -- kubectl get deployments -A
vcluster connect vc-b -n vcluster-b -- kubectl get deployments -A
vcluster connect vc-a -n vcluster-a -- kubectl get crds
vcluster connect vc-b -n vcluster-b -- kubectl get crds
```

## Troubleshooting and scope

- **Member does not appear:** check that the Agent image and chart support discovery, both feature values from step 1 are enabled, and the chart’s RBAC is installed. Automatic discovery supports shared-node and private-node vClusters hosted in the central cluster. It expects the standard Helm export `vc-<name>` and Service `<name>` in the vCluster’s host namespace, with their native vCluster labels. Custom export names and externally hosted vClusters are outside this walkthrough.

- **Member does not connect:** check the API endpoint from the Agent pod’s network, TLS SANs, the CA, and the permissions of the exported credentials. vCluster’s built-in network policy allows the virtual API port, but custom host policies must also permit the Agent to reach TCP port `8443` on the vCluster control-plane pods. A successful `vcluster connect` command does not establish Agent connectivity.

- **Connected, but no ClusterRegistration:** inspect Agent logs and `VClusterBootstrapFailed` events. All five central source CRDs must exist, and the exported credentials must permit CRD installation in the member. Distributed connectivity and registration/bootstrap are separate checks.

- **DSO reports a missing Deployment:** inspect `status.memberClusterStatuses` and check the target name and namespace in the member API. Do not target the host’s synchronized pod names.

- **Metrics fail:** check that the central `scaling-demo/demo-metrics` ConfigMap exists and its `a` and `b` values are numeric. The `kubernetes-resource` scaler requires Kedify KEDA. For other metric sources, ensure the endpoint and authentication work from central KEDA.

- **Member API is unavailable:** its workloads cannot receive replica updates. A DSO pinned to that member has no other scaling destination. Check per-member status before relying on the aggregate replica count.

- **Jobs:** similar configuration is available for `DistributedScaledJob`. The global multi-cluster switch enables Kedify KEDA’s required raw metrics service; see the [DSJ guide](https://docs.kedify.io/how-to/multicluster-dsj/).

Shared-node vClusters share host capacity and infrastructure failures. This walkthrough demonstrates scaling integration. For hard tenant isolation and production control-plane availability, see [vCluster tenancy models](https://www.vcluster.com/guides/tenancy-models-with-vcluster).

## Cleanup

Use these commands only for the disposable resources created by this walkthrough:

```bash
kubectl delete -f metrics.yaml
for member in a b; do
  helm uninstall "vc-$member" -n "vcluster-$member"
  kubectl delete namespace "vcluster-$member"
done
```

Deleting the disposable central `scaling-demo` namespace removes its DSOs, generated scaling resources, and metric ConfigMap. Deleting the host vCluster namespaces also deletes their persistent volume claims. The Agent removes the discovered members automatically.

## Continue with this topic

**Reference:** [Capability availability and enablement](https://docs.kedify.io/reference/capability-availability/) · [DistributedScaledObject API reference](https://docs.kedify.io/reference/kubernetes-api/distributedscaledobject/).

**Diagnose:** [Diagnose multi-cluster and tenant controllers](https://docs.kedify.io/troubleshooting/platform-scaling/).

---
Canonical: https://docs.kedify.io/how-to/vcluster-scaling/
Source: src/content/docs/how-to/vcluster-scaling.mdx
Documentation index: https://docs.kedify.io/llms.txt