In Kubernetes, an ingress controller acts as an entry point for the cluster: it sits at the edge, accepts incoming HTTP and HTTPS requests & intelligently routes them to the correct pods based on predefined routing rules. Beyond basic routing, an Ingress controller also handles critical tasks like SSL/TLS termination, load balancing & application security.

‘Ingress NGINX’ was a very popular ingress controller, until it retired in March 2026. See Ingress NGINX Retirement: What You Need to Know. This article describes how we migrated EKS from Ingress NGINX to F5 NGINX ingress controller. We chose F5 because it provided the simplest migration path. Other ingress controllers like Kong, Traefix & others, while very capable, needed significant migration effort, especially around NGINX config & annotations.

Ingress NGINX supports per-ingress config using annotations like nginx.ingress.kubernetes.io/client-body-buffer-size. This makes it possible to configure NGINX on a per-app basis instead of configuring NGINX globally to meet the needs of all EKS apps. This is especially important to us since our EKS is a shared cluster with hundreds of apps with very different behaviors.

F5 NGINX provided an (almost) drop-in replacement for the annotations & global NGINX config, making it the simplest migration path. F5 also provides a lot of guidance to ease this migration. See The Ingress NGINX Alternative: Open Source NGINX Ingress Controller for the Long Term & Migrate from Ingress-NGINX Controller to NGINX Ingress Controller.

Install F5 NGINX Ingress Controller

Before proceeding, ensure no existing ingress classes in EKS are set as default, since F5 will become the new default. To do so, list ingress classes & ensure none of them have the ingressclass.kubernetes.io/is-default-class annotation.

F5 NGINX is available as a Helm chart. Using Flux for GitOps, install the Helm repo & Helm release into EKS:

kind: HelmRepository
apiVersion: source.toolkit.fluxcd.io/v1
metadata:
  name: f5-nginx
  namespace: f5-nginx
spec:
  type: oci
  interval: 1h
  url: oci://ghcr.io/nginx/charts

kind: HelmRelease
apiVersion: helm.toolkit.fluxcd.io/v2
metadata:
  name: f5-nginx
  namespace: f5-nginx
spec:
  interval: 1h
  chart:
    spec:
      chart: nginx-ingress
      version: 2.4.4
      sourceRef:
        name: f5-nginx
        kind: HelmRepository

  values:
    controller:
      kind: daemonset
      healthStatus: true
      ingressClass:
        name: f5-nginx
        setAsDefaultIngress: true
      service:
        type: NodePort
        httpPort:
          nodePort: 30080
        httpsPort:
          enable: false

We terminate TLS at the EKS ALB & run NGINX as a DaemonSet on every EKS node. Hence the above Helm values config. But NGINX can also run as a standard deployment if that’s your current ingress controller setup.

Migrate Existing Ingresses to F5 NGINX

To switch existing ingresses to F5, set their ingress class name to F5. Also, if the ingress has any NGINX annotations, add equivalent F5 annotations to it. Since this is a one-time activity, it can be done either by a Bash script with kubectl or using a Kyverno policy, like this:

kind: ClusterPolicy
apiVersion: kyverno.io/v1
metadata:
  name: migrate-existing-ingresses-to-f5-nginx
spec:
  background: true
  mutateExistingOnPolicyUpdate: true

  rules:
  - name: migrate-existing-ingresses-to-f5-nginx
    match:
      any:
      - resources:
          kinds: [ Ingress ]
    mutate:
      targets:
      - kind: Ingress
        apiVersion: networking.k8s.io/v1
      patchStrategicMerge:
        spec:
          ingressClassName: f5-nginx

  - name: migrate-proxy-buffer-size-annotation
    match:
      any:
      - resources:
          kinds: [ Ingress ]
          annotations:
            nginx.ingress.kubernetes.io/proxy-buffer-size: "*"
    mutate:
      targets:
      - kind: Ingress
        apiVersion: networking.k8s.io/v1
      patchStrategicMerge:
        metadata:
          annotations:
            nginx.org/proxy-buffer-size: "{{ request.object.metadata.annotations.\"nginx.ingress.kubernetes.io/proxy-buffer-size\" }}"

The first rule in this Kyverno policy sets the ingress class name & the second rule adds an F5 annotation. The second rule can be duplicated for every NGINX annotation used across your ingresses. To find all NGINX annotations in use, run:

kubectl get ingresses -Ao yaml | \
grep nginx.ingress.kubernetes.io | \
cut -d':' -f1 | sort | uniq

As soon as this Kyverno policy is created, ingresses will start switching to F5. It’s the kyverno-background-controller deployment that executes this policy. In case you have hundreds or thousands of ingresses, scale up the background controller (add CPU & memory) to handle the policy execution. Only 1 background controller is active at a time. Other replicas are idle & only provide HA in case the primary pod dies. So adding more replicas does not help here. Here’s a snippet of the background controller showing scale up & increased parallelization for faster processing:

spec:
  replicas: 1
  template:
    spec:
      containers:
      - name: controller
        args:
        - ...
        - --genWorkers=50
        - --clientRateLimitQPS=50
        - --clientRateLimitBurst=100
        resources:
          requests:
            cpu: '1'
            memory: 1Gi
          limits:
            cpu: '8'
            memory: 4Gi

Kyverno creates UpdateRequests for every ingress it’s working on. Watch the UpdateRequests to track progress. After all ingresses have switched to F5, delete the Kyverno policy. To confirm that all ingresses have switched to the new ingress class, run:

kubectl get ingresses -Ao \
jsonpath='{.items[?(@.spec.ingressClassName!="f5-nginx")].metadata.name}'

Migrate New Ingresses to F5 NGINX

Deploy the following Kyverno policy. It ensures that any new ingresses that are created or updated henceforth, auto-switch to F5:

kind: ClusterPolicy
apiVersion: kyverno.io/v1
metadata:
  name: migrate-new-ingresses-to-f5-nginx
spec:
  background: false

  rules:
  - name: migrate-new-ingresses-to-f5-nginx
    match:
      any:
      - resources:
          kinds: [ Ingress ]
          operations: [ CREATE, UPDATE ]
    mutate:
      patchStrategicMerge:
        spec:
          ingressClassName: f5-nginx

  - name: migrate-proxy-buffer-size-annotation
    match:
      any:
      - resources:
          kinds: [ Ingress ]
          annotations:
            nginx.ingress.kubernetes.io/proxy-buffer-size: "*"
    mutate:
      patchStrategicMerge:
        metadata:
          annotations:
            nginx.org/proxy-buffer-size: "{{ request.object.metadata.annotations.\"nginx.ingress.kubernetes.io/proxy-buffer-size\" }}"

X-Forwarded-* Headers

This is a special case we encountered. Depending on how your EKS load balancer + ingress controller is setup, you may not need to do this. X-Forwarded-Port & X-Forwarded-Proto headers must be 443 & https to match Ingress NGINX behavior. F5 defaults these to 80 & http since we send traffic from EKS ALB target groups to F5 over HTTP.

F5 has no ConfigMap key to override these defaults globally. The well-documented proxy-set-headers ConfigMap key simply does not work! F5 does not “see” it at all. Bug? 🤔 When added to location-snippets in F5’s ConfigMap, these headers are added to F5’s config alongside its own defaults of these headers, instead of replacing them.

The resulting values are: X-Forwarded-Port: 443, 80 & X-Forwarded-Proto: https, http

The desired values are: X-Forwarded-Port: 443 & X-Forwarded-Proto: https

The ways to prevent F5 from appending values of these headers, also don’t work when both values are in the same F5 config block. So we added an init container to F5 pods to modify NGINX defaults before it starts. To do this, add this to the NGINX Helm release’s Helm values:

volumes:
- emptyDir: {}
  name: nginx-ingress-tmpl
volumeMounts:
- name: nginx-ingress-tmpl
  subPath: nginx.ingress.tmpl
  mountPath: /nginx.ingress.tmpl

initContainers:
- name: set-forwarded-headers
  image: nginx/nginx-ingress:5.3.4
  volumeMounts:
  - name: nginx-ingress-tmpl
    mountPath: /nginx-ingress-tmpl

  securityContext:
    capabilities:
      drop: [ ALL ]
    runAsUser: 101
    runAsNonRoot: true
    allowPrivilegeEscalation: false

  command:
  - /bin/sh
  - -c
  - >
    cp /nginx.ingress.tmpl /nginx-ingress-tmpl/nginx.ingress.tmpl

    sed -i 's/proxy_set_header X-Forwarded-Port
    \$server_port;/proxy_set_header X-Forwarded-Port 443;/g'
    /nginx-ingress-tmpl/nginx.ingress.tmpl

    sed -i 's/proxy_set_header X-Forwarded-Proto {{if
    \$server.RedirectToHTTPS}}https{{else}}\$scheme{{end}};/proxy_set_header
    X-Forwarded-Proto https;/g' /nginx-ingress-tmpl/nginx.ingress.tmpl

All ingresses at this point have now switched over to F5 NGINX & Ingress NGINX can now to uninstalled from the cluster. The Kyverno policy we left in the cluster will auto-switch all new ingresses to F5 NGINX automatically.

EKS NGINX