HTTP Scaling with Istio VirtualServices

This guide demonstrates how to scale applications within an Istio service mesh using VirtualService resources to manage HTTP traffic. You’ll deploy a sample application, configure the necessary Istio resources (Gateway, VirtualService), create a KEDA ScaledObject with a kedify-http trigger, and observe how Kedify automatically manages traffic routing for efficient load-based scaling including scale-to-zero when there’s no demand.

Architecture Overview

For applications managed by Istio and exposed via VirtualService, Kedify integrates with the service mesh using its autowiring feature. When using the kedify-http scaler, the traffic flow is adjusted to include Kedify’s proxy:

Istio Gateway -> VirtualService -> kedify-proxy -> Service -> Deployment

The kedify-proxy intercepts traffic routed by the VirtualService, collects metrics based on hosts defined in the HTTPScaledObject, and enables scaling decisions. When traffic increases, Kedify scales your application up; when traffic decreases, it scales down—even to zero if configured. Kedify automatically modifies the VirtualService destination host to point to the kedify-proxy service.

Prerequisites

A running Kubernetes cluster with Istio installed.

If you want to just replicate the tutorial, start a local cluster using k3d:

# Create k3d cluster
k3d cluster create --port "9080:80@loadbalancer" --k3s-arg "--disable=traefik@server:*"

# Install Istio
kubectl create ns istio-system

helm repo add istio https://istio-release.storage.googleapis.com/charts
helm repo update

helm install istio-base istio/base -n istio-system --wait
helm install istiod istio/istiod -n istio-system --wait
helm install istio-ingressgateway istio/gateway -n istio-system --wait

The kubectl command line utility installed and accessible.
Install hey to send load to a web application.

Step 1: Prepare Istio Sidecar Injection

Enable Istio sidecar injection for the namespaces where your application will run:

kubectl create ns keda
kubectl patch namespace keda -p '{"metadata": {"labels": {"istio-injection": "enabled"}}}'
kubectl patch namespace default -p '{"metadata": {"labels": {"istio-injection": "enabled"}}}'

Step 2: Install KEDA

Install KEDA and its HTTP add-on in the keda namespace:

helm repo add kedacore https://kedacore.github.io/charts
helm install keda kedacore/keda --namespace keda
helm install http-add-on kedacore/keda-add-ons-http --namespace keda

Verify that KEDA and its components are running:

kubectl get pods -n keda

You should see output similar to:

NAME                                                    READY   STATUS    RESTARTS   AGE
keda-add-ons-http-controller-manager-79db447c7d-7r9xb   3/3     Running   0          39s
keda-add-ons-http-external-scaler-dfb9f7bcc-dxqft       2/2     Running   0          39s
keda-add-ons-http-external-scaler-dfb9f7bcc-fh76m       2/2     Running   0          39s
keda-add-ons-http-external-scaler-dfb9f7bcc-phcw9       2/2     Running   0          39s
keda-add-ons-http-interceptor-6dd68bbc87-5jz6v          2/2     Running   0          38s
keda-add-ons-http-interceptor-6dd68bbc87-5ng2w          2/2     Running   0          39s
keda-add-ons-http-interceptor-6dd68bbc87-pnp47          2/2     Running   0          38s
keda-admission-webhooks-554fc8d77f-n7mml                2/2     Running   0          39s
keda-operator-dd878ddf6-7ww29                           2/2     Running   0          39s
keda-operator-metrics-apiserver-86cc9c6fff-t4prd        2/2     Running   0          39s

Step 3: Deploy Sample Application

For this example, we’ll use podinfo as our sample application:

helm upgrade --install --wait podinfo --namespace default oci://ghcr.io/stefanprodan/charts/podinfo

Create the Istio Gateway and initial VirtualService:

apiVersion: networking.istio.io/v1beta1
kind: Gateway
metadata:
  name: app
  namespace: default
spec:
  selector:
    istio: ingressgateway
  servers:
    - hosts:
        - "www.podinfo.com"
      port:
        name: http
        number: 80
        protocol: HTTP
---
apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  name: podinfo
  namespace: default
spec:
  hosts:
    - "www.podinfo.com"
  gateways:
    - app
  http:
    - match:
        - uri:
            prefix: /
      route:
        - destination:
            host: podinfo
            port:
              number: 9898

Gateway: Defines the entry point for traffic into the Istio service mesh.
VirtualService: Configures Istio’s routing. It attaches to the specified gateway, matches requests for the host www.podinfo.com, and routes traffic to the podinfo service. Kedify will automatically update the destination.host field to point to the kedify-proxy when autowiring is active.

Apply the configuration:

kubectl apply -f istio.yaml

Step 4: Configure Autoscaling

Create the source-of-truth ScaledObject. Kedify generates its HTTP routing resource; do not create a competing HTTPScaledObject for the same workload.

apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  name: podinfo
  namespace: default
spec:
  scaleTargetRef:
    name: podinfo
  minReplicaCount: 0
  maxReplicaCount: 10
  cooldownPeriod: 30
  triggers:
    - type: kedify-http
      metadata:
        hosts: www.podinfo.com
        service: podinfo
        port: "9898"
        scalingMetric: requestRate
        targetValue: "1"
        trafficAutowire: virtualservice

The hostname must match the VirtualService and the Service must select the target pods. The target value is requests per second per replica. Review HTTP trigger fields for the installed release.

kubectl apply -f scaledobject.yaml
kubectl get scaledobject,httpscaledobject,hpa -n default
kubectl describe scaledobject podinfo -n default

Step 5: Configure Traffic Routing

Keep the application VirtualService from Step 3 as the owned configuration. Check its live destination after the Agent reconciles: autowiring should point it through Kedify. Do not overwrite that destination with the upstream add-on’s interceptor address while Kedify owns routing.

kubectl get virtualservice podinfo -n default -o yaml
kubectl get endpoints podinfo -n default

If the route remains unchanged, check trafficAutowire, host matching, Agent permissions and proxy health using HTTP diagnostics. Verify healthy original routing before debugging activation from zero.

Step 6: Test Autoscaling

First, let’s verify that the application is accessible through the Gateway:

GATEWAY_IP=$(kubectl get svc -nistio-system istio-ingressgateway -o json | jq --raw-output '.status.loadBalancer.ingress[0].ip')
curl -H "host: www.podinfo.com" "http://$GATEWAY_IP"

If everything is correctly configured, you should receive a successful HTTP response:

{
  "hostname": "podinfo-5965fc9856-4tfpg",
  "version": "6.6.3",
  "revision": "b0c487c6b217bed8e6a53fca25f6ee1a7dd573e3",
  "color": "#34577c",
  "logo": "https://raw.githubusercontent.com/stefanprodan/podinfo/gh-pages/cuddle_clap.gif",
  "message": "greetings from podinfo v6.6.3",
  "goos": "linux",
  "goarch": "amd64",
  "runtime": "go1.22.3",
  "num_goroutine": "8",
  "num_cpu": "16"
}

Now, let’s simulate higher load using hey:

hey -n 10000 -c 150 -host "www.podinfo.com" "http://$GATEWAY_IP"

After sending the load, you’ll see a response time histogram in the terminal:

Response time histogram:
  0.301 [1]     |
  0.310 [2746]  |■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■
  0.319 [3499]  |■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■
  0.327 [2694]  |■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■
  0.336 [683]   |■■■■■■■■
  0.345 [99]    |■
  0.354 [20]    |
  0.362 [1]     |
  0.371 [23]    |
  0.380 [37]    |
  0.389 [80]    |■

In the Kedify Dashboard, you can also observe the traffic load and resulting scaling.

Network Policies

If you’re using NetworkPolicies in your cluster, you’ll need to configure them to allow traffic between Kedify’s proxy and your application. Here are example policies:

For Kedify interceptor egress:

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: allow-interceptor
  namespace: keda
spec:
  podSelector:
    matchLabels:
      app.kubernetes.io/component: interceptor
      app.kubernetes.io/instance: http-add-on
      app.kubernetes.io/part-of: keda-add-ons-http
  egress:
    - to:
        - namespaceSelector:
            matchLabels:
              istio-injection: enabled
          podSelector:
            matchLabels:
              app.kubernetes.io/name: podinfo
      ports:
        - port: 9898
  policyTypes:
    - Egress

For application ingress:

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: allow-from-keda
  namespace: default
spec:
  podSelector:
    matchLabels:
      app.kubernetes.io/name: podinfo
  ingress:
    - from:
        - namespaceSelector:
            matchLabels:
              istio-injection: enabled
      ports:
        - protocol: TCP
          port: 9898
  policyTypes:
    - Ingress

Next Steps

Explore the complete HTTP Scaler documentation for more advanced configurations and details about its architecture and features, including autowiring for various ingress types.

Continue with this topic

Reference: HTTP scaler and inference configuration.

Diagnose: Workload does not scale, or scales too slowly.

Related capabilities: Diagnose HTTP requests with logs and error metrics.