Register member clusters with GitOps

This guide shows two GitOps-friendly ways to add member clusters for distributed scaling. Both rely on the same member-cluster preparation; only the central-cluster wiring differs.

• Per-cluster Secret (recommended): one labeled Secret per member, picked up by Kedify Agent’s kubeconfig provider. The Secret can be created directly in the KEDA cluster or synchronized from the member cluster with External Secrets Operator’s PushSecret. Members can be added or removed without touching a bundled Secret or restarting the agent.
• Bundled Secret: a single kedify-agent-multicluster-kubeconfigs Secret carrying all member kubeconfigs under separate data keys, read from a mounted volume by Kedify Agent’s file provider.

Both paths can be used at the same time on the same agent.

About the two providers

Each path is backed by a different provider from the upstream sigs.k8s.io/multicluster-runtime library that Kedify Agent embeds. The provider names reflect where the agent discovers cluster credentials from, not what the credentials look like (both are kubeconfigs).

• Per-cluster Secret is handled by the kubeconfig provider, which watches labeled Secrets in the agent’s install namespace and treats each Secret’s data.kubeconfig payload as a member.
• Bundled Secret is handled by the file provider, which watches a directory of *.kubeconfig files on disk. The bundled kedify-agent-multicluster-kubeconfigs Secret is mounted as that directory, so each data key shows up as a separate file the provider picks up.

The provider name surfaces as the provider field on each KedifyConfiguration.status.multiClusterStatus.clusters[] entry and on each DSO/DSJ MemberClusterStatus. It also appears as the prefix in canonical cluster IDs (kubeconfig#…, file#…).

Generate the installation and member templates

In the Dashboard installation dialog, select Argo CD or Flux, then choose Multi-cluster mode. Use Distributed for shared scaling across member clusters or Dedicated for dedicated scaling control. Configure vCluster discovery and local-cluster registration if needed.

For explicitly registered members, add each member and download its access Secret template. Supply the member kubeconfig through your secret-management workflow, then reconcile the Secret in the Agent’s namespace in the cluster running Kedify. The template does not create member-cluster RBAC or obtain credentials; complete the member preparation below. Agent API credentials and member kubeconfigs are separate Secrets. Use unique member Secret names that do not overlap Agent or KEDA certificate Secrets.

See Argo CD installation or Flux installation for the chart resources. The registration procedure below covers distributed members.

Prerequisites

• Access to the member cluster.
• Access to the KEDA cluster.
• kubectl, base64, jq.
• For the distributed steps below, Kedify Agent installed with global.features.multicluster.enabled: true and global.features.multicluster.type: distributed. agent.features.multicluster.enabled is also supported when KEDA’s raw metrics gRPC service is configured separately.

1. Apply static resources on the member cluster

Apply these manifests in the member cluster (for example from an Argo app). This RBAC covers distributed scaling. The same set is used for both distributed registration providers.

apiVersion: v1
kind: Namespace
metadata:
  name: keda
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: kedify-agent
  namespace: keda
---
apiVersion: v1
kind: Secret
metadata:
  name: kedify-agent-token
  namespace: keda
  annotations:
    kubernetes.io/service-account.name: kedify-agent
type: kubernetes.io/service-account-token
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: kedify-agent
rules:
  - apiGroups: ["*"]
    resources: ["*/scale"]
    verbs: ["get", "list", "watch", "update", "patch"]
  - apiGroups: ["apps"]
    resources: ["deployments"]
    verbs: ["get", "list", "watch", "update", "patch"]
  - apiGroups: [""]
    resources: ["pods"]
    verbs: ["get", "list", "watch"]
  - apiGroups: ["batch"]
    resources: ["jobs"]
    verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: kedify-agent
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: kedify-agent
subjects:
  - kind: ServiceAccount
    name: kedify-agent
    namespace: keda

Once the kedify-agent-token Secret is applied, the serviceaccount controller populates it with a generated auth token and CA bundle.

2. Retrieve dynamic auth values from the member cluster

Describe the remote cluster:

MEMBER_NAME="member-a"
MEMBER_CONTEXT="member-a-context"
NAMESPACE="keda"

Read token, CA and server address:

TOKEN=$(kubectl --context "$MEMBER_CONTEXT" -n "$NAMESPACE" get secret kedify-agent-token -o jsonpath='{.data.token}' | base64 -d)
CA_DATA=$(kubectl --context "$MEMBER_CONTEXT" -n "$NAMESPACE" get secret kedify-agent-token -o jsonpath='{.data.ca\.crt}')
SERVER=$(kubectl config view --raw --minify --context="$MEMBER_CONTEXT" -o jsonpath='{.clusters[0].cluster.server}')

Generate and encode the kubeconfig for the Secret payload (base64 is encoding, not encryption):

KCFG_B64=$(echo -n "apiVersion: v1
kind: Config
clusters:
- cluster:
    certificate-authority-data: ${CA_DATA}
    server: ${SERVER}
  name: ${MEMBER_NAME}-cluster
contexts:
- context:
    cluster: ${MEMBER_NAME}-cluster
    user: kedify-agent
  name: kedify-agent@${MEMBER_NAME}
current-context: kedify-agent@${MEMBER_NAME}
users:
- name: kedify-agent
  user:
    token: ${TOKEN}
" | base64 | tr -d '\n')

Keep the kubeconfig in your secret store or encrypt it before committing it to Git.

3. Register the member in the KEDA cluster

Per-cluster Secret

Create one labeled Secret per member in the KEDA cluster in the same namespace where Kedify Agent is deployed. Each Secret carries one member’s kubeconfig under the kubeconfig data key, and the Secret’s metadata.name becomes the cluster’s alias used in spec.memberClusters[].name.

apiVersion: v1
kind: Secret
metadata:
  name: member-a                                          # this becomes the cluster alias
  namespace: keda
  labels:
    sigs.k8s.io/multicluster-runtime-kubeconfig: "true"   # selector the agent watches
type: Opaque
data:
  kubeconfig: <BASE64_OF_MEMBER_KUBECONFIG>

Apply one Secret per member you want to register. The agent’s kubeconfig provider picks the Secret up immediately and engages the cluster; no agent restart is needed and no other Secret needs editing when a member is added or removed.

Bundled Secret

Members registered via this path share a single Secret in the KEDA cluster: kedify-agent-multicluster-kubeconfigs. The agent’s file provider reads them from a mounted volume. Members registered via the per-cluster Secret path coexist independently and are not affected by edits to this Secret.

apiVersion: v1
kind: Secret
metadata:
  name: kedify-agent-multicluster-kubeconfigs
  namespace: keda
type: Opaque
data:
  member-a-cluster.kubeconfig: <BASE64_OF_MEMBER_A_KUBECONFIG>
  member-b-cluster.kubeconfig: <BASE64_OF_MEMBER_B_KUBECONFIG>

Replace each <BASE64_OF_...> placeholder with the output from KCFG_B64 for that member cluster. Adding or removing a member requires editing this single Secret; the agent picks up the change via the mounted volume’s filesystem notification.

4. Verify

kubectl kedify mc list-members

Or read the status directly:

kubectl -n keda get kedifyconfiguration kedify \
  -o json | jq '.status.multiClusterStatus.clusters'

Each entry carries a provider field showing which provider registered the cluster (file or kubeconfig).

Distributed reference forms in spec.memberClusters[].name

DistributedScaledObject and DistributedScaledJob accept three equivalent forms when referencing a cluster:

Form	Example
Alias	member-a
Canonical	kubeconfig#member-a or file#/etc/mc/kubeconfigs/member-a-cluster.kubeconfig+kedify-agent@member-a
Provider-local	identical to the alias for the kubeconfig provider; for the file provider: /etc/mc/kubeconfigs/member-a-cluster.kubeconfig+kedify-agent@member-a

The alias form is the everyday one. Canonical form is the disambiguating form when you have a cross-provider name collision (see below).

Cross-provider name collisions

If both the bundled file Secret and a per-cluster kubeconfig Secret register the same name, the agent picks the kubeconfig entry. The file cluster stays in the inventory and remains reachable through its canonical ID. A Warning event with reason MultiClusterNameCollision is recorded on KedifyConfiguration, and the file entry’s ClusterStatus.Info notes the masked alias.

To target the file cluster explicitly during a collision, use its canonical form in spec.memberClusters[].name (for example file#/etc/mc/kubeconfigs/member-a-cluster.kubeconfig+kedify-agent@member-a).

The simplest remediation is to pick different names: rename the per-cluster Secret, or remove the conflicting entry from the bundled Secret. Once one side is gone, the next status tick clears the collision and the surviving cluster reclaims the plain alias.

Continue with this topic

Configure and verify: Distribute workload replicas across clusters · Distribute jobs across clusters · Scale vCluster workloads centrally · Scale vClusters in dedicated mode.

Reference: kubectl kedify command reference.

Diagnose: Diagnose multi-cluster and tenant controllers.

Related capabilities: Distributed multi-cluster scaling.