Config Connector

Google Cloud Config Connector (KCC) is the operator we use to manage GCP resources as Kubernetes Custom Resources. Every GCP resource that has a corresponding KCC CRD is created, updated, and (occasionally) deleted by KCC controllers reconciling YAML in this repository against the live GCP API.

Plain GitOps applies: the desired state lives in main, ArgoCD syncs it into the cluster, KCC controllers actuate it onto GCP. There are no out-of-band gcloud or terraform apply steps for the projects KCC owns.

Overview

Why namespaced mode

The cluster-level ConfigConnector CR runs the operator in namespaced mode:

# k8s/infra-services/gcp-config-connector/configconnector.yaml
apiVersion: core.cnrm.cloud.google.com/v1beta1
kind: ConfigConnector
metadata:
  name: configconnector.core.cnrm.cloud.google.com
spec:
  mode: namespaced
  stateIntoSpec: Absent

In namespaced mode, each Kubernetes namespace that wants to manage GCP resources must declare its own ConfigConnectorContext resource pointing at the Google Service Account it should impersonate. This gives us:

• Per-target-project isolation: A namespace can only impersonate the SA it has been bound to.
• Workload Identity end-to-end: No service account JSON keys are stored in the cluster.
• Multiple managed GCP projects from a single cluster: One namespace per managed GCP project.

Architecture

KCC runs as a hub-and-spoke topology. The Infra Management Cluster is the GitOps hub - it hosts the namespaces that reconcile GCP resources for the central platform infra projects and for every tb-platform environment. The tb-platform spoke clusters run the operator too, but only for the small slice of resources that must live inside their own cluster.

graph TD
    subgraph hub["Infra Management Cluster (hub)"]
        op["KCC Operator<br/>v1.148.0<br/>namespaced mode"]
        ns_mgmt["ns: tb-infra-mgmt-project<br/>→ tb-infra-mgmt"]
        ns_vpc["ns: tb-infra-mgmt-vpc-project<br/>→ tb-infra-mgmt-vpc"]
        ns_sec["ns: tb-infra-security-project<br/>→ tb-infra-security"]
        ns_plat["ns: tb-platform-infra<br/>→ shared platform infra"]
        ns_env["ns: tb-platform-{dev,qa,prod}<br/>→ tb-platform GCP projects"]
        ns_vpcenv["ns: tb-platform-vpc-{dev,qa,prod}<br/>→ tb-platform VPC projects"]
        op --> ns_mgmt & ns_vpc & ns_sec & ns_plat & ns_env & ns_vpcenv
    end

    subgraph spokes["TB Platform Clusters (dev / qa / prod)"]
        op_spoke["KCC Operator<br/>(ApplicationSet:<br/>tb-platform-config-connector-operator)"]
    end

    ns_mgmt -.reconciles.-> gcp_mgmt[("GCP: tb-infra-mgmt")]
    ns_vpc -.reconciles.-> gcp_vpc[("GCP: tb-infra-mgmt-vpc")]
    ns_sec -.reconciles.-> gcp_sec[("GCP: tb-infra-security")]
    ns_plat -.reconciles.-> gcp_plat[("GCP: platform org/folders")]
    ns_env -.reconciles.-> gcp_env[("GCP: tb-platform-{env}")]
    ns_vpcenv -.reconciles.-> gcp_vpcenv[("GCP: tb-platform-vpc-{env}")]

The single workload-identity service account that backs every hub namespace is:

gke-platform-infra@tb-infra-mgmt-gke-prod-uk-40fd.iam.gserviceaccount.com

This GSA holds the IAM grants on each target GCP project / folder that allow it to act as a fleet-wide infrastructure administrator. Per-namespace blast-radius is enforced by:

1. The ArgoCD AppProject (which namespaces and clusters each Application may target).
2. The namespace’s ConfigConnectorContext (which GSA, and therefore which GCP scope, KCC may use in that namespace).

Repository layout

KCC content lives in two top-level trees:

k8s/
├── infra-services/
│   ├── gcp-config-connector/         # Operator install (CRDs + ConfigConnector CR)
│   ├── tb-infra-mgmt-project/        # KCC resources for the tb-infra-mgmt GCP project
│   ├── tb-infra-mgmt-vpc-project/    # KCC resources for the tb-infra-mgmt-vpc GCP project
│   └── tb-infra-security-project/    # KCC resources for the tb-infra-security GCP project
└── tb-platform-infra/                # KCC resources for the tb-platform GCP projects (per-env)
    ├── core/                         # Shared ConfigConnectorContext + cluster mgmt SA
    ├── env/                          # Per-env workload projects (dev/qa/prod) + base
    ├── vpc/                          # Per-env VPC host projects (dev/qa/prod) + base
    └── iam/                          # Shared IAM resources

Inside each tree, child directories are named after the Config Connector API group they correspond to (compute/, iam/, dns/, kms/, sql/, storage/, resourcemanager/, pubsub/, firestore/, bigquery/, serviceusage/, privateca/, networkconnectivity/, monitoring/, run/, redis/, artifactregistry/). This convention is enforced by k8s/tb-platform-infra/README.md.

secretmanager/ / secretsmanager/ directories are an exception: they are named after the GCP Secret Manager API but contain External Secrets Operator resources (SecretStore, ExternalSecret), not KCC resources. See Managed resources for details.

A core/config-connector-context.yaml at the root of each tree (or each env overlay) wires the namespace up to KCC.

Operator install

The operator is installed by ArgoCD from k8s/infra-services/gcp-config-connector/:

# k8s/infra-services/gcp-config-connector/kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
  #- gcp-config-connector-operator.yaml          # legacy non-Autopilot manifest (v1.127.0)
  - autopilot-configconnector-operator.yaml      # active, v1.148.0
  - configconnector.yaml

What the active manifest installs:

• Namespace/configconnector-operator-system (annotated cnrm.cloud.google.com/operator-version: 1.148.0).
• All KCC operator CRDs in the core.cnrm.cloud.google.com and customize.core.cnrm.cloud.google.com API groups (ConfigConnector, ConfigConnectorContext, ControllerReconciler, ControllerResource, MutatingWebhookConfigurationCustomization, NamespacedControllerReconciler, NamespacedControllerResource).
• The cluster-scoped ConfigConnector resource that selects namespaced mode.

The same Kustomization is reused for every cluster that runs KCC:

The non-Autopilot manifest (gcp-config-connector-operator.yaml, pinned at v1.127.0) is kept in-tree for reference only - it is commented out of kustomization.yaml and is not applied to any cluster.

Authentication model (Workload Identity)

The cluster-level ConfigConnector CR intentionally has no googleServiceAccount and no credentialSecretName. In namespaced mode each namespace supplies its own GSA via a ConfigConnectorContext:

# k8s/tb-platform-infra/env/dev/core/config-connector-context.yaml
apiVersion: core.cnrm.cloud.google.com/v1beta1
kind: ConfigConnectorContext
metadata:
  name: configconnectorcontext.core.cnrm.cloud.google.com
  namespace: tb-platform-dev
spec:
  googleServiceAccount: "gke-platform-infra@tb-infra-mgmt-gke-prod-uk-40fd.iam.gserviceaccount.com"
  stateIntoSpec: Absent

The GKE node identity binds to this GSA via Workload Identity - the cnrm-system per-namespace controller manager is granted roles/iam.workloadIdentityUser on gke-platform-infra and runs as it when calling Google Cloud APIs. No JSON keys are stored anywhere in the cluster, so credential rotation is handled entirely by GCP IAM.

Hub namespace → GCP project mapping

Every KCC namespace that the hub cluster reconciles is wired to the same workload-identity GSA, but each namespace targets a different GCP project via the resources it contains:

GitOps delivery

ArgoCD delivers four classes of KCC content to the hub cluster:

graph LR
    git[(infra-services<br/>main)]

    git --> opApp[Application:<br/>config-connector-operator]
    git --> mgmtApp[Application:<br/>infra-mgmt-project]
    git --> vpcApp[Application:<br/>infra-mgmt-vpc-project]
    git --> secApp[Application:<br/>infra-security-project]
    git --> platApp[Application:<br/>tb-platform-infra]
    git --> envSet[ApplicationSet:<br/>tb-platform-environments]
    git --> vpcSet[ApplicationSet:<br/>tb-platform-vpc-config]
    git --> opSet[ApplicationSet:<br/>tb-platform-config-connector-operator]

    opApp -->|infra-services| operator((Operator CRDs<br/>+ ConfigConnector CR))
    mgmtApp -->|infra-mgmt| nsMgmt((ns: tb-infra-mgmt-project))
    vpcApp -->|infra-mgmt| nsVpc((ns: tb-infra-mgmt-vpc-project))
    secApp -->|infra-mgmt| nsSec((ns: tb-infra-security-project))
    platApp -->|infra-services| nsPlat((ns: tb-platform-infra))
    envSet -->|tb-platform-infra| nsEnv((ns: tb-platform-{env}))
    vpcSet -->|tb-platform-infra| nsVpcEnv((ns: tb-platform-vpc-{env}))
    opSet --> opSpoke((Operator on<br/>tb-platform clusters))

AppProjects

See Managed Services → Config Connector for the full per-Application sync policy and Slack notification wiring.

Sync options

Every KCC Application uses ServerSideApply=true. KCC resources are large, the controller updates them out-of-band as it reconciles state, and a client-side three-way merge will fight with the controller. With SSA, Kubernetes tracks field ownership and Argo only owns the fields it actually writes.

prune is set per-Application based on blast radius:

What KCC manages

The footprint covers the full set of GCP services we use - IAM, networking, compute, GKE, KMS, DNS, SQL, storage, pub/sub, secrets, BigQuery, Firestore, Artifact Registry, Private CA, Network Connectivity, Cloud Run, Redis, Monitoring, and Service Usage. The full inventory by GCP project, including kind counts and the relevant API groups, lives in Managed resources. Day-to-day workflows (adding resources, debugging stuck reconciles, recovering from drift) are in Operations.

What’s next

• Managed resources - Per-project inventory of GCP resources reconciled by KCC.
• Operations - Adding new resources, sync workflow, troubleshooting and recovery.
• Managed Services → Config Connector - ArgoCD Application / ApplicationSet reference.
• GCP Config Connector docs - Upstream Google reference.