Diagnose multi-cluster and tenant controllers

Use remote member connectivity when a DSO/DSJ cannot distribute work to another cluster. Use dedicated member scaling for ordinary ScaledObjects/ScaledJobs reconciled remotely by central KEDA/KPA. Use tenant controller ownership when a workload in a shared cluster has missing or competing autoscalers. Dashboard cluster connections alone do not register scaling members.

Remote member connectivity

Set the central context, Agent namespace and distributed resource. Use distributedscaledjobs for a DSJ:

CENTRAL_CONTEXT=central
AGENT_NS=keda
WORKLOAD_NS=default
RESOURCE=distributedscaledobjects
NAME=example
kubectl --context "$CENTRAL_CONTEXT" get "$RESOURCE" "$NAME" -n "$WORKLOAD_NS" -o json | \
  jq '{members: .spec.memberClusters, status: .status}'
kubectl --context "$CENTRAL_CONTEXT" get kedifyconfiguration -n "$AGENT_NS" -o json | \
  jq '.items[] | {name: .metadata.name, clusters: .status.multiClusterStatus.clusters}'
kubectl --context "$CENTRAL_CONTEXT" get events -n "$AGENT_NS" --sort-by=.lastTimestamp
kubectl --context "$CENTRAL_CONTEXT" get events -n "$WORKLOAD_NS" --sort-by=.lastTimestamp

Evidence	Check and remedy
Member absent from configuration status	Confirm its registration Secret and provider. A Dashboard connection does not create a member entry.
MultiClusterNameCollision event	Two providers registered the same alias. Use the intended canonical member ID or remove the duplicate registration through its owner.
Certificate or authentication failure	Replace expired/incorrect credentials through the registration source. Use the CA expiry metric and member status where supported by the installed Agent.
Forbidden response	Compare remote RBAC with the GitOps registration procedure; verify target namespace and resource permissions.
Timeout / connection refused	Check the API server address, DNS and network path from the central controller. Success from a laptop does not prove controller reachability.
Connected member but no replicas/jobs	Inspect the DSO/DSJ status for scheduling, capacity and failover decisions; for DSJ, also check the KEDA raw-metrics prerequisite.

For per-member Secret registration, inspect names and key presence without printing credentials:

kubectl --context "$CENTRAL_CONTEXT" get secrets -n "$AGENT_NS" \
  -l sigs.k8s.io/multicluster-runtime-kubeconfig=true -o json | \
  jq '.items[] | {name: .metadata.name, keys: (.data | keys)}'

Each entry should contain kubeconfig. For file-provider registration, inspect the mounted Secret named in the registration guide. Correct the owning configuration, then repeat the status checks. Verify the expected Deployment or Jobs in the intended member context and namespace; do not blindly resubmit distributed jobs, which can duplicate work.

Dedicated member scaling

Confirm global.features.multicluster.type: dedicated, both KEDA/KPA dependencies enabled, and Agent DSO/DSJ switches disabled. Scaling objects and generated KPAs belong to the member context, not the central DSO/DSJ inventory.

kubectl --context central -n keda get clusterregistrations -o json | \
  jq '.items[] | {name: .metadata.name, generation: .metadata.generation, status: .status}'
kubectl --context central -n keda logs deploy/keda-operator --tail=100
kubectl --context central -n keda logs deploy/kpa --tail=100
kubectl --context member-a -n default get scaledobject,scaledjob,kpa
kubectl --context member-a -n default get events --sort-by=.lastTimestamp

Symptom	Check and remedy
Secret exists but dedicated controllers do not connect	Create a ClusterRegistration referencing that Secret in the Agent namespace. Distributed provider labels or bundled Secret entries alone are insufficient.
Registration is not ready	Inspect Ready, observedGeneration and canonicalName; check Secret key/context, identity uniqueness, credentials and member API access from Agent.
vCluster is connected but registration is absent	Inspect VClusterBootstrapFailed events and Agent logs. All five source KEDA/KPA CRDs must exist centrally; native credentials must permit member CRD installation.
Ready registration, but no KPA or scaling	Check KEDA and KPA connection logs, member CRDs/RBAC, member ScaledObject conditions, and that scaling is not paused on the registration. Ready validates connectivity, not all controller permissions.
CPU/memory unavailable	Run kubectl top pods in the member, verify container requests, and for shared-node vClusters enable integrations.metricsServer.enabled=true with a working host metrics API.
External metrics unavailable	Check central KPA’s mTLS connection to its paired KEDA operator on 9666, member-scoped metric errors, and central KEDA’s access to the metric source.

Follow dedicated setup and verification and mode-specific networking.

Tenant controller ownership

Inspect the affected workload and the Agent’s tenant inventory:

kubectl config current-context
NS=tenant-workloads
SO=example
kubectl get scaledobject "$SO" -n "$NS" -o json | \
  jq '{annotations: .metadata.annotations, target: .spec.scaleTargetRef, status: .status}'
kubectl get hpa -n "$NS" -o json | \
  jq '.items[] | {name: .metadata.name, target: .spec.scaleTargetRef, owners: .metadata.ownerReferences}'
kubectl get kedifyconfiguration -n keda -o json | \
  jq '.items[] | {name: .metadata.name, tenants: .status.discoveredTenants}'
kubectl get events -n keda --field-selector reason=KedifyTenantConnectionNotReady
kubectl get events -n "$NS" --sort-by=.lastTimestamp

When KPA is installed, also run:

kubectl get kpa -n "$NS" -o json | \
  jq '.items[] | {name: .metadata.name, target: .spec.scaleTargetRef, owners: .metadata.ownerReferences, status: .status}'

Tenant status / symptom	Action
operatorReady: false	Inspect the registered operatorDeploymentName with kubectl describe deployment and its rollout status in the tenant’s control-plane namespace.
tlsCertReady: false or configSynced: false	Inspect tenant connection events and the paired KEDA certificate/service configuration. Follow installation verification.
KPA intended, but kpa.paired: false	Compare KEDA/KPA namespace watch sets and kpa.kedaMetricsAddress. Correct the Helm values for the mismatched pair.
kpa.operatorObserved: false	The Agent has not inspected the operator configuration. Check Agent logs/access before treating this as intentionally disabled KPA.
Both an HPA and KPA target the workload	Check autoscaling.kedify.io/class, ScalingGroup membership and overlapping tenant scopes. Restore one owner through the ScaledObject and installation values; use the KPA rollback procedure when reverting.
CPU/memory metrics unavailable	Run kubectl top pods -n "$NS" --containers; repair the Kubernetes Metrics API path before changing scaler targets.

An intentionally HPA-only tenant can have kpa.enabled, kpa.controllerReady and kpa.paired all false. Do not treat those fields alone as a failure. Namespace scoping is not a complete hostile-tenant security boundary; see access and data boundaries.

Verify or escalate

Confirm the affected tenant connection becomes ready and exactly one intended autoscaler controls the target. Repeat a bounded demand test, comparing the source metric, desired replicas and ready workload capacity. For unresolved failures, collect diagnostics with central/member context names, affected tenant, resource names and timestamps, excluding credential values.