# How HTTP scaling works

This document explains how HTTP traffic metrics are collected, aggregated, and used for scaling decisions in Kedify.

## Overview

Kedify uses a distributed metrics collection system where:

1. **kedify-proxy** (Envoy) handles traffic and collects metrics

2. **Interceptor** aggregates metrics and pushes them to the scaler via gRPC bridge

3. **Scaler** receives metrics and reports to KEDA

4. **KEDA** makes scaling decisions based on metrics

## Architecture Diagram

[![HTTP scaler data and metric flow](https://docs.kedify.io/assets/images/docs/http_scaler_metrics_architecture.svg)](https://docs.kedify.io/assets/images/docs/http_scaler_metrics_architecture.svg)

Scroll to exploreDiagram description

Kedify Proxy sends Envoy stats over gRPC to the interceptor's envoysink on port 9901. The interceptor aggregates them and streams batches through the gRPC bridge to the external scaler. KEDA reads the external scaler and supplies metrics to HPA or KPA. HPA or KPA changes active replicas; KEDA handles activation and return to zero. HTTP requests travel separately to the application Service and pods.

## Components

### 1. kedify-proxy (Envoy)

The kedify-proxy is an Envoy-based proxy that:

- Routes HTTP traffic to backend services

- Collects traffic metrics (RPS, Concurrency)

- Pushes metrics to the interceptor via gRPC

**Envoy Configuration:**

```yaml
stats_flush_interval: 1s
stats_sinks:
  - name: kedify_metrics_sink
    typed_config:
      "@type": type.googleapis.com/envoy.config.metrics.v3.MetricsServiceConfig
      transport_api_version: V3
      report_counters_as_deltas: true
      grpc_service:
        envoy_grpc:
          cluster_name: kedify_metrics_service
```

### 2. Interceptor - envoysink (Port 9901)

The interceptor runs a gRPC server on port 9901 that receives metrics from kedify-proxy. This server is exposed trough a kubernetes service pointing to port 9901 on the interceptor.

**Key Components:**

- `MetricsServiceServer`: Implements Envoy’s MetricsService gRPC interface

- `externalQueues`: In-memory storage for metrics received from Envoy

- Cluster-to-HSO mapping: Maps Envoy cluster names to HTTPScaledObjects

**Service:** `keda-add-ons-http-interceptor-kedify-proxy-metric-sink`

```yaml
ports:
  - name: proxy
    port: 9901
    targetPort: 9901
  - name: control-plane
    port: 5678
    targetPort: 5678
```

### 3. gRPC Bridge (Interceptor → Scaler)

The interceptor pushes metrics to the scaler via a **gRPC bridge** (enabled by default). This is a streaming connection where the interceptor periodically sends metric batches.

**How It Works:**

- Interceptor acts as gRPC **client**, connects to scaler

- Scaler acts as gRPC **server**, receives metric streams

- Metrics are pushed periodically

**Note:** The `/queue` REST endpoint still exists for debugging. See [Debugging](https://docs.kedify.io/concepts/http-scaler-metrics-architecture/#debugging) section.

### 4. Scaler

The scaler:

- Receives metrics from interceptors via gRPC bridge (see above)

- Implements KEDA’s External Scaler gRPC interface

- Aggregates metrics across all interceptors when KEDA queries

### 5. KEDA and HPA

KEDA uses the scaler’s metrics to:

1. Determine if workload should be active (scale from zero)

2. Calculate desired replica count

3. Update the HorizontalPodAutoscaler (HPA)

## Metrics

See [Metrics, status and health signals](https://docs.kedify.io/reference/metrics/) for fields and validation.

## Metric Keys

See [Metrics, status and health signals](https://docs.kedify.io/reference/metrics/) for fields and validation.

## Debugging

For the request-path diagnostic procedure and error ownership, use [HTTP logs and error metrics](https://docs.kedify.io/how-to/kedify-proxy-access-logs-and-metrics/). The endpoints below inspect the current metric path.

### Query Current Metrics

```bash
kubectl get --raw /api/v1/namespaces/keda/services/keda-add-ons-http-interceptor-admin:9090/proxy/queue
```

### Query Envoy-to-HSO Mapping

```bash
kubectl get --raw /api/v1/namespaces/keda/services/keda-add-ons-http-interceptor-admin:9090/proxy/envoy-metrics-map
```

## Inference Traffic Specifics

For inference workloads using `inferencePool`:

- Traffic flows through port 9002 on kedify-proxy

- External Processing (ext_proc) filter integrates with Endpoint Picker

- Metrics are collected the same way as regular traffic

- Key format remains `namespace/httpscaledobject-name`

See [HTTP Scaler for Inference](https://docs.kedify.io/scalers/http-scaler-for-inference/) for more details.

## Continue with this topic

**Reference:** [HTTP scaler and inference configuration](https://docs.kedify.io/reference/http-scaler/).

**Related capabilities:** [Workload diagnostics](https://docs.kedify.io/troubleshooting/workload-scaling/).

---
Canonical: https://docs.kedify.io/concepts/http-scaler-metrics-architecture/
Source: src/content/docs/concepts/http-scaler-metrics-architecture.md
Documentation index: https://docs.kedify.io/llms.txt