How HTTP scaling works

This document explains how HTTP traffic metrics are collected, aggregated, and used for scaling decisions in Kedify.

Overview

Kedify uses a distributed metrics collection system where:

1. kedify-proxy (Envoy) handles traffic and collects metrics
2. Interceptor aggregates metrics and pushes them to the scaler via gRPC bridge
3. Scaler receives metrics and reports to KEDA
4. KEDA makes scaling decisions based on metrics

Architecture Diagram

Scroll to explore

Diagram description

Kedify Proxy sends Envoy stats over gRPC to the interceptor's envoysink on port 9901. The interceptor aggregates them and streams batches through the gRPC bridge to the external scaler. KEDA reads the external scaler and supplies metrics to HPA or KPA. HPA or KPA changes active replicas; KEDA handles activation and return to zero. HTTP requests travel separately to the application Service and pods.

Components

1. kedify-proxy (Envoy)

The kedify-proxy is an Envoy-based proxy that:

• Routes HTTP traffic to backend services
• Collects traffic metrics (RPS, Concurrency)
• Pushes metrics to the interceptor via gRPC

Envoy Configuration:

stats_flush_interval: 1s
stats_sinks:
  - name: kedify_metrics_sink
    typed_config:
      "@type": type.googleapis.com/envoy.config.metrics.v3.MetricsServiceConfig
      transport_api_version: V3
      report_counters_as_deltas: true
      grpc_service:
        envoy_grpc:
          cluster_name: kedify_metrics_service

2. Interceptor - envoysink (Port 9901)

The interceptor runs a gRPC server on port 9901 that receives metrics from kedify-proxy. This server is exposed trough a kubernetes service pointing to port 9901 on the interceptor.

Key Components:

• MetricsServiceServer: Implements Envoy’s MetricsService gRPC interface
• externalQueues: In-memory storage for metrics received from Envoy
• Cluster-to-HSO mapping: Maps Envoy cluster names to HTTPScaledObjects

Service: keda-add-ons-http-interceptor-kedify-proxy-metric-sink

ports:
  - name: proxy
    port: 9901
    targetPort: 9901
  - name: control-plane
    port: 5678
    targetPort: 5678

3. gRPC Bridge (Interceptor → Scaler)

The interceptor pushes metrics to the scaler via a gRPC bridge (enabled by default). This is a streaming connection where the interceptor periodically sends metric batches.

How It Works:

• Interceptor acts as gRPC client, connects to scaler
• Scaler acts as gRPC server, receives metric streams
• Metrics are pushed periodically

Note: The /queue REST endpoint still exists for debugging. See Debugging section.

4. Scaler

The scaler:

• Receives metrics from interceptors via gRPC bridge (see above)
• Implements KEDA’s External Scaler gRPC interface
• Aggregates metrics across all interceptors when KEDA queries

5. KEDA and HPA

KEDA uses the scaler’s metrics to:

1. Determine if workload should be active (scale from zero)
2. Calculate desired replica count
3. Update the HorizontalPodAutoscaler (HPA)

Metrics

See Metrics, status and health signals for fields and validation.

Metric Keys

See Metrics, status and health signals for fields and validation.

Debugging

For the request-path diagnostic procedure and error ownership, use HTTP logs and error metrics. The endpoints below inspect the current metric path.

Query Current Metrics

kubectl get --raw /api/v1/namespaces/keda/services/keda-add-ons-http-interceptor-admin:9090/proxy/queue

Query Envoy-to-HSO Mapping

kubectl get --raw /api/v1/namespaces/keda/services/keda-add-ons-http-interceptor-admin:9090/proxy/envoy-metrics-map

Inference Traffic Specifics

For inference workloads using inferencePool:

• Traffic flows through port 9002 on kedify-proxy
• External Processing (ext_proc) filter integrates with Endpoint Picker
• Metrics are collected the same way as regular traffic
• Key format remains namespace/httpscaledobject-name

See HTTP Scaler for Inference for more details.

Continue with this topic

Reference: HTTP scaler and inference configuration.

Related capabilities: Workload diagnostics.