Envoy Gateway

API Gateway and ingress using Envoy Gateway with GKE Gateway API

The Infra Management Cluster runs multiple Envoy Gateway instances to handle ingress traffic for different domains. Each instance operates in namespace mode, watching specific namespaces for Gateway API resources.

ArgoCD Resources

Gateway System (Helm Charts)

These Applications deploy the Envoy Gateway controller and CRDs:

ApplicationNamespaceController Name
envoy-infra-titanbay-comenvoy-infra-titanbay-comgateway.envoyproxy.io/infra-titanbay-com-controller
envoy-infra-titanbayapis-comenvoy-infra-titanbayapis-comgateway.envoyproxy.io/infra-titanbayapis-com-controller
envoy-infra-titanbay-internalenvoy-infra-titanbay-internalgateway.envoyproxy.io/infra-titanbay-internal-controller

File Path: apps/envoy-gateway-system.yaml

Gateway Routes (Kustomize)

These Applications deploy Gateway, HTTPRoute, and related resources:

ApplicationNamespaceSource Path
envoy-gateway-infra-titanbay-comenvoy-infra-titanbayapis-comk8s/infra-services/envoy-gateway/overlays/infra-titanbay-com
envoy-gateway-infra-titanbayapis-comenvoy-infra-titanbayapis-comk8s/infra-services/envoy-gateway/overlays/infra-titanbayapis-com
envoy-gateway-infra-titanbay-internalenvoy-infra-titanbay-internalk8s/infra-services/envoy-gateway/overlays/infra-titanbay-internal

File Path: apps/envoy-gateway.yaml

Architecture

graph TD
    subgraph "External Traffic"
        A[HTTP Client]
    end
    
    subgraph "GKE Gateway API L7 LB"
        B[External Load Balancer]
    end
    
    subgraph "Envoy Gateway"
        C[envoy-infra-titanbay-com]
        D[envoy-infra-titanbayapis-com]
        E[envoy-infra-titanbay-internal]
    end
    
    subgraph "Workloads"
        F[ArgoCD]
        G[Atlantis]
        H[Grafana]
        I[Other Services]
    end
    
    A --> B
    B --> C
    B --> D
    B --> E
    C --> F
    C --> G
    D --> H
    E --> I

Source Structure

k8s/infra-services/envoy-gateway/
├── base/                              # Shared resources
└── overlays/
    ├── infra-titanbay-com/            # *.infra.titanbay.com routes
    │   ├── gateway.yaml
    │   ├── httproutes/
    │   └── kustomization.yaml
    ├── infra-titanbayapis-com/        # *.infra.titanbayapis.com routes
    │   ├── gateway.yaml
    │   ├── httproutes/
    │   └── kustomization.yaml
    └── infra-titanbay-internal/       # Internal routes
        ├── gateway.yaml
        ├── httproutes/
        └── kustomization.yaml

Gateway System Configuration

Each Envoy Gateway instance is deployed via Helm with namespace-scoped watching:

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: envoy-infra-titanbay-com
spec:
  syncPolicy:
    syncOptions:
      - ServerSideApply=true
  destination:
    namespace: envoy-infra-titanbay-com
    server: https://kubernetes.default.svc
  project: infra-services
  source:
    chart: gateway-helm
    repoURL: 'registry-1.docker.io/envoyproxy'
    targetRevision: 1.4.2
    helm:
      valuesObject:
        deployment:
          replicas: 2
        config:
          envoyGateway:
            gateway:
              controllerName: gateway.envoyproxy.io/infra-titanbay-com-controller
            provider:
              type: Kubernetes
              kubernetes:
                watch:
                  type: Namespaces
                  namespaces:
                    - argocd
                    - atlantis
                    - envoy-infra-titanbay-com
                    - infra-docs-portal
                    - monitoring
                    - netbox
                    - tb-ops
                    - grafana-alloy
                    - pam-system
                    - onepassword

Watched Namespaces

Each controller watches specific namespaces:

ControllerWatched Namespaces
infra-titanbay-comargocd, atlantis, infra-docs-portal, monitoring, netbox, tb-ops, grafana-alloy, pam-system, onepassword
infra-titanbayapis-comargo, argo-events, argocd, atlantis, dex, monitoring, netbox, grafana-alloy, pam-system, grafana-loki
infra-titanbay-internalmonitoring, grafana-alloy

Gateway Routes Configuration

The route Applications deploy Gateway API resources:

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: envoy-gateway-infra-titanbay-com
spec:
  syncPolicy:
    syncOptions:
      - ServerSideApply=true
  destination:
    namespace: envoy-infra-titanbayapis-com
    server: https://kubernetes.default.svc
  project: infra-services
  source:
    repoURL: https://github.com/Titanbay/infra-services
    path: k8s/infra-services/envoy-gateway/overlays/infra-titanbay-com
    targetRevision: 'main'

How to Update

Adding a New Route

  1. Create an HTTPRoute in the appropriate overlay:
    k8s/infra-services/envoy-gateway/overlays/<domain>/httproutes/<service>.yaml
    
  2. Add the file to the overlay’s kustomization.yaml
  3. Ensure the service’s namespace is in the controller’s watch list
  4. Commit and push to main

Adding a Namespace to Watch List

  1. Edit apps/envoy-gateway-system.yaml
  2. Find the appropriate controller’s namespaces list
  3. Add the new namespace
  4. Commit and push to main

Upgrading Envoy Gateway

  1. Update targetRevision in apps/envoy-gateway-system.yaml
  2. Review the Envoy Gateway changelog
  3. Commit and push to main
ResourcePurpose
apps/envoy-gateway-oci-secret.yamlOCI registry credentials for Helm chart
Gateway API CRDsInstalled by the Helm chart
GKE Gateway ControllerNative GKE integration for L7 LB

Ignored Differences

The Applications ignore certain CRD differences to prevent sync loops:

ignoreDifferences:
  - kind: CustomResourceDefinition
    group: apiextensions.k8s.io
    name: gatewayclasses.gateway.networking.k8s.io
    jsonPointers:
      - /metadata/annotations
      - /spec/versions

This is necessary because GKE’s Gateway controller may modify CRD annotations.