# Tune Kedify Proxy performance

Tune proxy capacity against measured request concurrency, payload size, response latency and memory use. Record the installed Agent, HTTP Add-on and Proxy versions before changing values; repeat the same load test after each change.

## Step 4: Optimize Proxy Configuration

### Resource requests and limits

Adjust Kedify Proxy resource requests and limits based on your application’s needs. Depending on your installation method, you may need to adjust the resource requests and limits in either the Agent or the Kedify Proxy helm chart. Values used by all instances of Kedify Proxy across all namespaces are defined in the `globalValues` section of the Agent helm chart. The following example shows how to set resource requests and limits for Kedify Proxy in the Agent helm chart:

```yaml
agent:
  kedifyProxy:
    globalValues:
      resources:
        requests:
          cpu: 100m
          memory: 128Mi
        limits:
          cpu: 200m
          memory: 256Mi
```

To adjust the resource requests and limits for a Kedify Proxy in a specific namespace, you can use the `namespacedValues` section of the Agent helm chart:

```yaml
agent:
  kedifyProxy:
    namespacedValues:
      namespace1:
        resources:
          requests:
            cpu: 100m
            memory: 128Mi
          limits:
            cpu: 200m
            memory: 256Mi
```

If you are installing Kedify Proxy as a standalone service, you can adjust the resource requests and limits in the Kedify Proxy helm chart:

```yaml
resources:
  requests:
    cpu: 100m
    memory: 128Mi
  limits:
    cpu: 200m
    memory: 256Mi
```

### Timeouts

The HTTP Add-on chart **0.15.0** exposes separate interceptor deadlines. These are the chart’s documented code defaults when values are unset, not a universal timeout for every Envoy or client connection:

| Helm value | Default | Scope |
| --- | --- | --- |
| `interceptor.tcpConnectTimeout` | `500ms` | One TCP connection attempt to the backend |
| `interceptor.responseHeaderTimeout` | `300s` | Waiting for backend response headers |
| `interceptor.readinessTimeout` | Disabled | Waiting for ready backend capacity, including activation from zero |
| `interceptor.requestTimeout` | Disabled | Total request lifecycle |

Set an explicit readiness budget based on measured node provisioning, image pull and model/application startup. Client, ingress and Envoy route deadlines can expire first. Example override, not a default:

```yaml
interceptor:
  readinessTimeout: "120s"
```

`interceptor.replicas.waitTimeout` is deprecated in this chart; use `readinessTimeout`. Check the values for your installed release before migrating an older configuration.

### Graceful Termination

When Kedify Proxy scales in or one of its pods is getting evicted to a new node, it uses `preStop` hook to gracefully terminate and let all active HTTP connections to complete. During this time, Kedify Proxy will not accept new connections, and the hook waits for `http.kedify-proxy.downstream_cx_active` metric to report 0. The default timeout for the hook is 30 seconds with 5 seconds as an additional buffer after all connections are already drained. You can adjust this through `kedify-agent` helm chart:

```yaml
agent:
  kedifyProxy:
    globalValues:
      pod:
        preStopHookWaitSeconds: 2 # <- additional short timeout after all connections are drained
        terminationGracePeriodSeconds: 60 # <- total timeout for the preStop hook
    namespacedValues:
      default:
        pod:
          preStopHookWaitSeconds: 3 # <- additional short timeout after all connections are drained in default namespace
          terminationGracePeriodSeconds: 90 # <- total timeout for the preStop hook in default namespace
```

The `terminationGracePeriodSeconds` should be equal or larger than the response time of your application. If the pod reaches this deadline, Kubernetes can kill it with requests still in flight. Clients may observe a reset, timeout or gateway error; do not rely on a specific HTTP status.

### Circuit Breaker

Envoy has a built-in circuit breaker that can help prevent cascading failures in your application. But when set too low, it can create a bottleneck in your environment. You can configure the circuit breaker in the Kedify HTTP Addon helm chart:

```yaml
interceptor:
  envoy:
    upstreamRateLimiting:
      maxConnections: 8192
      maxRequests: 8192
      maxPendingRequests: 8192
      maxRetries: 3
```

### Overload Manager

Envoy has a built-in overload manager that can help prevent resource exhaustion during high traffic periods. But when set too low, it can create a bottleneck in your environment. You can configure the overload manager in the Kedify Proxy helm chart:

```yaml
config:
  overloadManager:
    enabled: true
    refreshInterval: 0.25s
    maxActiveDownstreamConnections: 10000
```

## Step 5: Autoscale Kedify Proxy

Kedify Proxy can also be autoscaled and help your infrastructure during high load periods. The following example shows how to configure autoscaling for Kedify Proxy in the Kedify Proxy helm chart:

```yaml
autoscaling:
  enabled: true
  minReplicaCount: 1
  maxReplicaCount: 10
```

## Step 6: Monitor Performance

Envoy provides a variety of metrics that can help you monitor the performance of your autoscaling setup. These metrics can be accessed through the Envoy admin interface. Typically on port 9901/admin of the Kedify Proxy pod. You can also use a `ServiceMonitor` to scrape these metrics and send them to a monitoring system like Prometheus. The following example shows how to create a ServiceMonitor for an example Kedify Proxy in namespace `http-server`:

```yaml
apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
  name: kedify-http-proxy
spec:
  endpoints:
    - port: admin
      scheme: http
      path: /stats/prometheus
  namespaceSelector:
    matchNames:
      - http-server
  selector:
    matchLabels:
      app: kedify-proxy
```

## Step 7: How we perform load testing at Kedify

We use load testing tool Locust to perform load testing on Kedify Proxy. Locust is a powerful and flexible load testing tool that allows us to simulate a large number of users and http requests to the proxy and the http application that is being scaled by Kedify.

We combine statistics provided by Locust with metrics collected from Envoy and our http application. We assume that the application runs intensive calculations or waits for other services to respond. We simulate this by using a delay in the application’s response that is randomly chosen between 1 and 30 seconds. With this assumption, we expect to see the 50th percentile of the response time to be around 15 seconds and the 95th percentile to be around 29 seconds. With thousands of clients that try to connect to the application, we set the maximum number of potential application replicas to 10 and the requests per second scaling metric to a relatively low value, we use a range between 10 and 100, so that Kedify can add new application pods aggressively, as new requests are coming in. Kedify Proxy autoscaling is enabled and the maximum number of replicas is set to 10. This way we want to take the advantage of horizontal scaling to spread the load. With this high number of clients, but very limited throughput, Kedify Proxy pods cache a lot of requests and send them to the application when available.

With many concurrent clients and slow responses, queued requests can increase proxy memory even while CPU use remains low. Monitor `upstream_rq_pending_active` for current queue depth and the rate of change of `upstream_rq_pending_total` for newly queued requests. `upstream_rq_pending_total` is a cumulative counter: a nonzero value alone does not show current overload. Correlate queue growth with overflow/error counters, ready replicas and application latency before raising limits. See the [Envoy cluster statistics definitions](https://www.envoyproxy.io/docs/envoy/latest/configuration/upstream/cluster_manager/cluster_stats).

![Kedify Proxy Load Test With Locust](https://docs.kedify.io/assets/images/how-to/http-proxy-performance-tuning/proxy-load-test.png)

## Next steps

Repeat the same load profile after changing one setting. Compare successful throughput, tail latency, pending requests, errors and proxy memory. Restore the previous values if the change worsens these results.

## Continue with this topic

**Reference:** [Kedify Proxy Envoy configuration](https://docs.kedify.io/how-to/kedify-proxy-envoy-configuration/) · [HTTP scaler and inference configuration](https://docs.kedify.io/reference/http-scaler/).

**Related capabilities:** [Diagnose HTTP requests with logs and error metrics](https://docs.kedify.io/how-to/kedify-proxy-access-logs-and-metrics/) · [Prepare workloads for reliable autoscaling](https://docs.kedify.io/best-practices/kubernetes-autoscaling-best-practices/).

Technical content reviewed Sep 21, 2026.

---
Canonical: https://docs.kedify.io/how-to/http-proxy-performance-tuning/
Source: src/content/docs/how-to/http-proxy-performance-tuning.md
Documentation index: https://docs.kedify.io/llms.txt