# Kedify Proxy Envoy configuration

At the core of `kedify-http` scaler is `kedify-proxy`, which forms a fleet of Envoy proxies. The fleet is configured over xDS control plane, implemented as part of the http-add-on `interceptor` component.

There are two parts of envoy configuration that support override of the default configuration, both are set as values in the `kedify-agent` helm chart:

1. `cluster` - [chart](https://github.com/kedify/charts/blob/960b33a0a479bf858ec23e35847be3f2c4e9a0ca/kedify-agent/values.yaml#L64-L65), [envoy options](https://www.envoyproxy.io/docs/envoy/latest/api-v3/config/cluster/v3/cluster.proto)

2. `route` - [chart](https://github.com/kedify/charts/blob/960b33a0a479bf858ec23e35847be3f2c4e9a0ca/kedify-agent/values.yaml#L60-L61), [envoy options](https://www.envoyproxy.io/docs/envoy/latest/api-v3/config/route/v3/route_components.proto#config-route-v3-routeaction)

### Retry Configuration on Error

By default, `kedify-proxy` will not retry requests that fail with any error code and will return the error code to the client. With `route` configuration, you can [enable automatic retries](https://www.envoyproxy.io/docs/envoy/latest/api-v3/config/route/v3/route_components.proto#envoy-v3-api-msg-config-route-v3-retrypolicy) for specific error situations. For example, to retry on 5xx errors, you can set the following in your `kedify-agent` values:

```yaml
agent:
  kedifyProxy:
    globalEnvoyConfigs:
      route:
        retry_policy:
          retry_on: 5xx # any internal or external 5xx error
          num_retries: 5 # retry up to 5 times
          retry_back_off:
            base_interval: 1s # first retry will be after 1 second
            max_interval: 10s # maximum interval between retries is 10 seconds with exponential backoff
```

This envoy config snippet means `kedify-proxy` will retry requests that fail with 5xx errors up to 5 times, with an exponential backoff starting at 1 second and capping at 10 seconds.

### Slow Start Configuration

The `kedify-proxy` envoy uses `ROUND_ROBIN` load balancing strategy by default. This means that all endpoints in the cluster are treated equally, no matter how long they have been up. This can lead to issues if some endpoints are slow to start, as they may receive high load of traffic before they are ready. To mitigate this, you can enable [slow start](https://www.envoyproxy.io/docs/envoy/latest/api-v3/config/cluster/v3/cluster.proto#envoy-v3-api-field-config-cluster-v3-cluster-slowstartconfig-slow-start-window) for the cluster by setting the `slow_start_config` configuration in the `cluster` section of your `kedify-agent` values:

```yaml
agent:
  kedifyProxy:
    globalEnvoyConfigs:
      cluster:
        lb_policy: ROUND_ROBIN
        round_robin_lb_config:
          slow_start_config:
            slow_start_window: 60s # slow start window will take effect for 60 seconds, after that it's ROUND_ROBIN
            min_weight_percent:
              value: 1.0 # as little as 1% of the traffic can be sent to the new endpoint to warm it up
            aggression:
              default_value: 1.0 # pace of traffic increase during the slow start window, lower number means slower in the beginning
              runtime_key: slow_start_aggression
```

This envoy config snippet will instruct `kedify-proxy` to use a [slow start window](https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/upstream/load_balancing/slow_start) of 60 seconds, during which as little as 1% of the traffic will be sent to the new endpoint and it will gradually increase. After the slow start window, the endpoint will be treated equally with other endpoints in the service and receive its fair share of the traffic.

### Preconnecting

Envoy supports [preconnecting](https://www.envoyproxy.io/docs/envoy/latest/api-v3/config/cluster/v3/cluster.proto#config-cluster-v3-cluster-preconnectpolicy) `endpoints` in the `cluster`, which can help reduce latency for requests by anticipating a request and establishing a TCP session before it’s needed.

```yaml
agent:
  kedifyProxy:
    globalEnvoyConfigs:
      cluster:
        preconnect_policy:
          per_upstream_preconnect_ratio: 1.05 # preconnect 5 upstream connections for every 100 request
          predictive_preconnect_ratio: 1.05 # preconnect 5 spare connection on the anticipated next endpoint for 100 requests
```

Having `per_upstream_preconnect_ratio` set to 1.05 means that for each 100 requests, `kedify-proxy` will preconnect 5 upstream connections in the cluster instead of waiting for the new request to arrive and then establishing the connection. Setting `predictive_preconnect_ratio` to 1.05 means that for each 100 requests, kedify-proxy will try to predict the next 5 connections anticipating what the next endpoint should be based on the envoy cluster’s internal loadbalancing configuration.

### Active Health Checks

Envoy supports [active health checks](https://www.envoyproxy.io/docs/envoy/latest/api-v3/config/core/v3/health_check.proto#config-core-v3-healthcheck) for endpoints in the cluster. This can help ensure that only healthy endpoints receive traffic. Because this configuration is set globally for each cluster, we recommend only very rudimentary TCP health checks.

```yaml
agent:
  kedifyProxy:
    globalEnvoyConfigs:
      cluster:
        common_lb_config:
          ignore_new_hosts_until_first_hc: true # apply health checks to new hosts
          healthy_panic_threshold:
            value: 0.2 # panic if more than 80% of the hosts are unhealthy
        health_checks:
          - timeout: 1s # timeout for the health check request
            interval: 2s # interval between health checks during endpoint's healthy state
            unhealthy_interval: 10s # interval between health checks during endpoint's unhealthy state
            unhealthy_threshold: 3 # endpoint is considered unhealthy after 3 consecutive failed health checks
            healthy_threshold: 2 # endpoint is considered healthy after 2 consecutive successful health checks
            tcp_health_check: {} # check if envoy can establish a TCP connection to the endpoint
```

This envoy config snippet will instruct `kedify-proxy` to perform active health checks on the endpoints in the cluster. The health check will timeout after 1 second, and will be performed every 2 seconds while the endpoint is healthy, and every 10 seconds while the endpoint is unhealthy. The endpoint will be considered unhealthy after 3 consecutive failed health checks, and healthy again after 2 consecutive successful health checks. The health check will simply try to establish a TCP connection to the endpoint and then send FIN packet.

The [panic threshold](https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/upstream/load_balancing/panic_threshold) is set to 20%, which means that if more than 80% of the endpoints in the cluster are unhealthy, `kedify-proxy` will panic and sending traffic to all endpoints regardless of their health status. This is to prevent the situation where all endpoints are unhealthy and no traffic is being sent. Change this parameter with care, because it directly relates to the `ScaledObject` scale-up behavior. If the threshold is too high, even regular rapid scale-up events can trigger the panic mode.

### Passive Health Checks

Envoy also supports passive health checks through [outlier detection](https://www.envoyproxy.io/docs/envoy/latest/api-v3/config/cluster/v3/outlier_detection.proto#config-cluster-v3-outlierdetection). These checks are performed on the actual traffic and can help detect unhealthy endpoints based on their response codes and latency.

```yaml
agent:
  kedifyProxy:
    globalEnvoyConfigs:
      cluster:
        outlier_detection:
          interval: 5s # interval between outlier detection checks
          max_ejection_percent: 50 # maximum percentage of endpoints that can be ejected from the cluster
          split_external_local_origin_errors: true # treat external and local origin errors separately
          consecutive_gateway_failure: 3 # number of consecutive requests that must fail with gateway error or timeout to eject the endpoint
          enforcing_consecutive_gateway_failure: 100 # enforce the ejection each time the threshold is reached
```

This envoy config snippet will instruct `kedify-proxy` to perform passive health checks on the endpoints in the cluster. The outlier detection will be performed every 5 seconds, and unless 50% or more are already ejected, it will eject endpoints that fail 3 consecutive requests with envoy internal error or timeout. The `split_external_local_origin_errors` parameter is set to true, which means that external and local origin errors will be treated separately. This is useful if you want to treat errors from external services differently from errors from your own services.

### Envoy Bootstrap Configuration

The `cluster` and `route` overrides above are delivered to the proxies over xDS and customize the dynamic configuration. A few Envoy settings live in the proxy’s static [bootstrap](https://www.envoyproxy.io/docs/envoy/latest/api-v3/config/bootstrap/v3/bootstrap.proto) instead and cannot be changed over xDS. For those, the `kedify-proxy` chart exposes `config.bootstrap`, a map that is merged verbatim into the bootstrap as additional top-level sections. This is a `kedify-proxy` chart value, so it is set through `agent.kedifyProxy.globalValues` (or `namespacedValues` for a single namespace).

A common use case is raising Envoy’s RE2 regex program size limit. Each `pathRegex` route compiles into an RE2 program whose size grows with the length and complexity of the expression. Envoy rejects the whole listener when that size exceeds `re2.max_program_size.error_level`, which defaults to `100`, and the proxy logs an error such as:

```plaintext
regex '/very-long-path-.../.*' RE2 program size of 181 > max program size of 100 set for the error level threshold
```

Raise the limit through the `layered_runtime` bootstrap section:

```yaml
agent:
  kedifyProxy:
    globalValues:
      config:
        bootstrap:
          layered_runtime:
            layers:
              - name: static_layer
                static_layer:
                  re2.max_program_size.error_level: 1000
                  re2.max_program_size.warn_level: 1000
```

`config.bootstrap` is additive. Do not redefine sections that Kedify already manages (`node`, `admin`, `dynamic_resources`, `static_resources`, `stats_sinks`, and `overload_manager`), because duplicate top-level keys produce an invalid bootstrap. Any change to the bootstrap restarts all `kedify-proxy` instances.

## Continue with this topic

**Configure and verify:** [Tune Kedify Proxy performance](https://docs.kedify.io/how-to/http-proxy-performance-tuning/).

**Reference:** [HTTP scaler and inference configuration](https://docs.kedify.io/reference/http-scaler/).

**Related capabilities:** [HTTP scaling](https://docs.kedify.io/scalers/http-scaler/) · [Diagnose HTTP requests with logs and error metrics](https://docs.kedify.io/how-to/kedify-proxy-access-logs-and-metrics/).

---
Canonical: https://docs.kedify.io/how-to/kedify-proxy-envoy-configuration/
Source: src/content/docs/how-to/kedify-proxy-envoy-configuration.md
Documentation index: https://docs.kedify.io/llms.txt