TB Ops

Applications and GCP resources for the Ops and Finance teams

TB Ops encompasses applications and GCP infrastructure resources for the Ops and Finance teams. This includes both Kubernetes applications and Config Connector-managed GCP resources.

Overview

TB Ops is managed through two ArgoCD resources:

ResourceKindPurpose
feeder-fund-simulatorApplicationKubernetes application deployment
tb-ops-project-environmentsApplicationSetGCP resources via Config Connector

Feeder Fund Simulator (Application)

ArgoCD Resource

PropertyValue
KindApplication
Namefeeder-fund-simulator
Namespaceargocd
Destination Namespacetb-ops
Projectinfra-services
Source TypeKustomize
File Pathk8s/infra-services/argocd/overlays/infra-platform-cluster/apps/tb-ops-apps.yaml

Application Definition

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: feeder-fund-simulator
  labels:
    cluster: 'infra-platform-mgmt'
    environment: 'prod'
  finalizers:
    - resources-finalizer.argocd.argoproj.io
  annotations:
    notifications.argoproj.io/subscribe.on-app-synced.slack: platform-infra-notifications
    notifications.argoproj.io/subscribe.on-app-outofsync.slack: platform-infra-notifications
    notifications.argoproj.io/subscribe.on-app-sync-failed.slack: platform-infra-notifications
    notifications.argoproj.io/subscribe.on-app-degraded.slack: platform-infra-notifications
spec:
  syncPolicy:
    automated: {}
  destination:
    namespace: tb-ops
    server: https://kubernetes.default.svc
  project: infra-services
  source:
    path: k8s/infra-services/tb-ops-apps/feeder-fund-simulator
    repoURL: https://github.com/Titanbay/infra-services
    targetRevision: 'chore/deploy-tb-ops-simulator-app'

Manifest Organisation

k8s/infra-services/tb-ops-apps/
├── kustomization.yaml                  # Top-level kustomization
└── feeder-fund-simulator/
    ├── kustomization.yaml              # App kustomization (namespace: tb-ops)
    ├── deployment.yaml                 # Main deployment
    └── svc.yaml                        # ClusterIP service

TB Ops Project Environments (ApplicationSet)

ArgoCD Resource

PropertyValue
KindApplicationSet
Nametb-ops-project-environments
Namespaceargocd
Projectinfra-services
GeneratorGit directory
File Pathk8s/infra-services/argocd/overlays/infra-platform-cluster/application-sets/tb-ops-project.yaml

ApplicationSet Definition

apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
  name: tb-ops-project-environments
  namespace: argocd
spec:
  goTemplate: true
  goTemplateOptions: ['missingkey=error']
  generators:
    - git:
        repoURL: https://github.com/Titanbay/infra-services.git
        revision: 'main'
        directories:
          - path: k8s/infra-services/tb-ops-project/env/*
          - path: k8s/infra-services/tb-ops-project/env/base
            exclude: true
  template:
    metadata:
      name: 'tb-ops-project-{{.path.basename}}'
      labels:
        cluster: 'infra-platform-mgmt'
        environment: '{{.path.basename}}'
    spec:
      project: 'infra-services'
      source:
        repoURL: https://github.com/Titanbay/infra-services.git
        targetRevision: 'main'
        path: '{{.path.path}}'
      destination:
        server: https://kubernetes.default.svc
        namespace: 'tb-ops-project-{{.path.basename}}'
      syncPolicy:
        syncOptions:
          - ServerSideApply=true

Generated Applications

The ApplicationSet generates one Application per environment directory:

ApplicationEnvironmentNamespace
tb-ops-project-devdevtb-ops-project-dev
tb-ops-project-qaqatb-ops-project-qa
tb-ops-project-prodprodtb-ops-project-prod

Manifest Organisation

k8s/infra-services/tb-ops-project/
├── kustomization.yaml
├── core/
│   └── config-connector-context.yaml   # Config Connector context
└── env/
    ├── base/                           # Shared resources
    │   ├── kustomization.yaml
    │   ├── sql/
    │   │   ├── cloudsql-postgres.yaml  # Cloud SQL instance
    │   │   └── ip-allowlist.yaml       # IP allowlist patch
    │   └── secretmanager/
    │       ├── secret-store.yaml       # SecretStore
    │       └── postgres-secrets.yaml   # Database secrets
    ├── dev/
    │   ├── kustomization.yaml          # Dev overlay
    │   └── core/
    │       └── config-connector-context.yaml
    ├── qa/
    │   ├── kustomization.yaml          # QA overlay
    │   └── core/
    │       └── config-connector-context.yaml
    └── prod/
        ├── kustomization.yaml          # Prod overlay
        └── core/
            └── config-connector-context.yaml

GCP Resources Managed

Resource TypePurpose
SQLInstanceCloud SQL PostgreSQL instance
SecretManagerSecretDatabase credentials
ConfigConnectorContextPer-environment GCP project binding

How to Update

Adding a New TB Ops Application

  1. Create a new directory under k8s/infra-services/tb-ops-apps/
  2. Add Kustomize manifests (deployment, service, etc.)
  3. Create a new Application in apps/tb-ops-apps.yaml or a separate file
  4. Add to apps/kustomization.yaml
  5. Commit and push to main

Modifying GCP Resources

  1. Edit the appropriate file in k8s/infra-services/tb-ops-project/env/
  2. Use base/ for shared resources, environment directories for overrides
  3. Commit and push to main
  4. Config Connector reconciles the GCP resources

Adding a New Environment

  1. Create a new directory under k8s/infra-services/tb-ops-project/env/
  2. Add kustomization.yaml referencing the base
  3. Add environment-specific config-connector-context.yaml
  4. The ApplicationSet automatically generates a new Application

Notifications

Both the Application and generated ApplicationSet Applications send Slack notifications to #platform-infra-notifications for:

  • Sync success
  • Out of sync detection
  • Sync failures
  • Degraded health