# Metrics, status and health signals

Metrics are emitted by specific installed components. Check units, labels and sample freshness before comparing control-plane health with application success. Use [Workload diagnostics](https://docs.kedify.io/troubleshooting/workload-scaling/) for the measurement workflow.

For endpoints and collection with Datadog, Prometheus and other stacks, see [Monitoring](https://docs.kedify.io/observability/monitoring/).

## Source and enablement

Verify the component’s `/metrics` output for the installed release. Missing telemetry does not by itself mean the scaling control loop failed.

| Emitter | Evidence | Enablement / task |
| --- | --- | --- |
| Check runners | End-to-end canary timing and step outcomes | [Install and verify checks](https://docs.kedify.io/features/autoscaling-checks/) |
| HTTP proxy/interceptor | Request rate, concurrency, errors and route ownership | [HTTP log/metric diagnosis](https://docs.kedify.io/how-to/kedify-proxy-access-logs-and-metrics/) |
| Agent PRA controller | Resource decisions, blocked changes, deltas and evaluation timing | [Enable PRA](https://docs.kedify.io/features/pod-resource-autoscaler/#enablement-and-pod-opt-in) |
| HPA/KPA and custom resources | Desired/current state and conditions | [Kubernetes API catalog](https://docs.kedify.io/concepts/kedify-custom-resource-definitions/); exact status fields stay with each kind |

## Member-cluster CA expiry

The Agent exposes `kedify_agent_kedifyconfiguration_multicluster_ca_expiration_timestamp_seconds` for parseable CA certificates in member kubeconfigs. See [multi-cluster monitoring](https://docs.kedify.io/features/multi-cluster-scaling/#monitoring) for labels, days-remaining queries and expiry alerts.

## Cardinality and interpretation

Keep namespace/workload/container dimensions bounded to the workloads you operate; do not add request IDs, error messages or timestamps as metric labels. Check-runner result labels are a bounded vocabulary. A fresh healthy canary does not establish that a business request, model or queue worker meets its objective. [HTTP traces](https://docs.kedify.io/how-to/otel-tracing-proxy-and-interceptor/) provide a separate request-timing view.

Every runner exposes `/healthz` and `/metrics`.

For Kedify Dashboard ingestion, the Kedify agent auto-discovers labeled runner Services and relays these metrics with `kedify_agent_id`, which the dashboard API uses to scope results to the selected cluster.

## Primary Metric

```text
kedify_autoscaling_check_duration_seconds{
  check="http",
  result="success",
  signal="success",
  source_metric="skipped",
  keda_metric="success",
  hpa_metric="success",
  hpa_active="success",
  scale_up="success",
  scale_down="success"
} 42.5
```

This gauge is the primary alerting metric. It records the duration of the latest complete check iteration, in seconds. Alert on this metric when autoscaling control-plane latency exceeds your threshold, for example:

```promql
kedify_autoscaling_check_duration_seconds > 300
```

Example Prometheus alert rule:

```yaml
groups:
  - name: kedify-autoscaling-checks
    rules:
      - alert: KedifyAutoscalingCheckSlow
        expr: kedify_autoscaling_check_duration_seconds > 300
        for: 2m
        labels:
          severity: warning
        annotations:
          summary: Kedify autoscaling check is slow
          description: Check {{ $labels.check }} took {{ $value }}s in the latest iteration.
```

The `result` and step labels explain what happened during that iteration. Label values are bounded to:

```text
success | failure | skipped | unknown
```

Dynamic object names, timestamps, reasons, and error messages are not emitted as labels. Result labels are bounded; `kedify_autoscaling_check_info` is the static metadata exception and exposes bounded values such as the runner namespace, primary metric name, and interval.

## Supporting Metrics

```text
kedify_autoscaling_check_step_duration_seconds{check="http",step="keda_metric"}
kedify_autoscaling_check_last_run_timestamp_seconds{check="http"}
kedify_autoscaling_check_next_run_timestamp_seconds{check="http"}
kedify_autoscaling_check_info{check="http",runner_namespace="keda",primary_metric="kedify_autoscaling_check_duration_seconds",interval_seconds="300",autoscaler_class="hpa"}
kedify_autoscaling_check_metric_threshold{check="prometheus",metric="source"}
kedify_autoscaling_check_observed_metric_value{check="prometheus",metric="source"}
kedify_autoscaling_check_observed_metric_value{check="otel",metric="keda"}
kedify_autoscaling_check_observed_replicas{check="cpu",phase="scale_up"}
kedify_autoscaling_check_observed_replicas{check="memory",phase="scale_up"}
```

Metric types:

- `kedify_autoscaling_check_duration_seconds`: gauge, latest complete iteration duration.

- `kedify_autoscaling_check_step_duration_seconds`: gauge, latest step duration.

- `kedify_autoscaling_check_last_run_timestamp_seconds`: gauge, Unix timestamp of the latest iteration.

- `kedify_autoscaling_check_next_run_timestamp_seconds`: gauge, Unix timestamp of the next scheduled iteration.

- `kedify_autoscaling_check_info`: gauge, static runner metadata for the check.

- `kedify_autoscaling_check_metric_threshold`: gauge, minimum value used to decide whether the metric check passes.

- `kedify_autoscaling_check_observed_metric_value`: gauge, observed source, KEDA, or selected pod-autoscaler metric value.

- `kedify_autoscaling_check_observed_replicas`: gauge, observed target replica count.

Metric threshold and observed value labels use static names such as:

- `source`

- `keda`

- `hpa`

The `hpa` source and the `hpa_metric`/`hpa_active` result labels are retained for dashboard and alert compatibility. When `autoscaler.class=kpa`, they represent the HPA-compatible KPA status read directly from `KedifyPodAutoscaler`.

`kedify_autoscaling_check_info` includes `autoscaler_class="hpa|kpa"` so consumers can distinguish which control-plane path each runner validates without changing the compatibility step labels.

Replica phases use:

- `scale_up`

- `scale_down`

## Check Semantics

HTTP:

- `signal`: in-cluster HTTP traffic was sent to the sample HTTP server.

- `source_metric`: skipped.

- `keda_metric`: `external.metrics.k8s.io` returned a value at least equal to the exported pass threshold for a metric from `ScaledObject.status.externalMetricNames`.

- `hpa_metric`: the selected pod autoscaler’s `status.currentMetrics` included the KEDA external metric at a value at least equal to the exported pass threshold.

- `hpa_active`: the selected pod autoscaler reported `ScalingActive=True`.

- `scale_up` and `scale_down`: target Deployment replicas crossed configured thresholds.

Prometheus:

- `signal`: `/work` traffic was sent to work-simulator.

- `source_metric`: the Prometheus query returned a value at least equal to the configured and exported pass threshold.

- `keda_metric`: `external.metrics.k8s.io` returned the generated KEDA metric at a value at least equal to the exported pass threshold.

- `hpa_metric`: the selected pod autoscaler’s `status.currentMetrics` included the KEDA external metric at a value at least equal to the exported pass threshold.

- `hpa_active`, `scale_up`, and `scale_down`: same as HTTP.

OTEL:

- `signal`: a synthetic OTLP gauge was exported to the Kedify OTEL add-on.

- `source_metric`: the add-on REST query API returned a value at least equal to the configured and exported pass threshold from its metric store.

- `keda_metric`: `external.metrics.k8s.io` returned the generated KEDA metric at a value at least equal to the exported pass threshold.

- `hpa_metric`: the selected pod autoscaler’s `status.currentMetrics` included the KEDA external metric at a value at least equal to the exported pass threshold.

- `hpa_active`, `scale_up`, and `scale_down`: same as HTTP.

CPU:

- `signal`: sample load-generator CPU profile was activated.

- `source_metric`: `metrics.k8s.io` reported target Pod CPU at least equal to the configured and exported pass threshold.

- `keda_metric`: skipped.

- `hpa_metric`: the selected pod autoscaler’s `status.currentMetrics` included CPU resource metrics at a value at least equal to the exported pass threshold.

- `hpa_active`, `scale_up`, and `scale_down`: same as HTTP.

Memory:

- `signal`: sample load-generator memory profile was activated.

- `source_metric`: `metrics.k8s.io` reported target Pod memory at least equal to the configured and exported pass threshold.

- `keda_metric`: skipped.

- `hpa_metric`: the selected pod autoscaler’s `status.currentMetrics` included memory resource metrics at a value at least equal to the exported pass threshold.

- `hpa_active`, `scale_up`, and `scale_down`: same as HTTP.

## PRA metrics

PRA publishes internal Prometheus metrics (names prefixed with `kedify_agent_pra_`), including:

- `kedify_agent_pra_reconcile_count`

- `kedify_agent_pra_resource_updates`

- `kedify_agent_pra_total`

- `kedify_agent_pra_error_total`

- `kedify_agent_pra_scaling_active`

Additional decision and sizing metrics are also exported:

- `kedify_agent_pra_scale_decisions_total`

- `kedify_agent_pra_scale_blocked_total`

- `kedify_agent_pra_scale_delta_absolute`

- `kedify_agent_pra_scale_delta_percent`

- `kedify_agent_pra_evaluation_duration_seconds`

- `kedify_agent_pra_time_to_apply_seconds`

Notes:

- `kedify_agent_pra_scaling_active` is derived from `lastScaleTime + cooldown` window.

- Delta histogram units: CPU absolute delta is measured in millicores; memory absolute delta is measured in MiB.

## HTTP metric keys

Metrics are keyed by `namespace/httpscaledobject-name`. This means:

- All traffic matching an HTTPScaledObject is aggregated together

- Different paths, hosts, or query parameters within the same HSO share metrics

## HTTP scaling metrics

### RPS (Requests Per Second)

- **Source**: `cluster.upstream_rq_total` from Envoy

- **Use case**: `scalingMetric: requestRate`

### Concurrency

- **Source**: `cluster.upstream_rq_active` from Envoy

- **Use case**: `scalingMetric: concurrency`

Last updated: Sep 21, 2026

---
Canonical: https://docs.kedify.io/reference/metrics/
Source: src/content/docs/reference/metrics.md
Documentation index: https://docs.kedify.io/llms.txt