Netbox

Infrastructure resource modelling and documentation tool

Netbox is an infrastructure resource modelling (IRM) application for documenting and managing networks, IP addresses, equipment, and connections. It serves as a source of truth for network infrastructure documentation.

ArgoCD Resource

PropertyValue
KindApplication
Namenetbox
Namespaceargocd
Destination Namespacenetbox
Projectinfra-services
Source TypeKustomize
File Pathk8s/infra-services/argocd/overlays/infra-platform-cluster/apps/netbox.yaml

Application Definition

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: netbox
  labels:
    cluster: 'infra-platform-mgmt'
    environment: 'prod'
  finalizers:
    - resources-finalizer.argocd.argoproj.io
  annotations:
    notifications.argoproj.io/subscribe.on-app-synced.slack: platform-infra-notifications
    notifications.argoproj.io/subscribe.on-app-outofsync.slack: platform-infra-notifications
    notifications.argoproj.io/subscribe.on-app-sync-failed.slack: platform-infra-notifications
    notifications.argoproj.io/subscribe.on-app-degraded.slack: platform-infra-notifications
spec:
  syncPolicy:
    automated:
      prune: true
  destination:
    namespace: argocd
    server: https://kubernetes.default.svc
  project: infra-services
  source:
    path: k8s/infra-services/netbox
    repoURL: https://github.com/Titanbay/infra-services
    targetRevision: 'main'

Manifest Organisation

The Netbox manifests use a simple Kustomize structure with a base directory:

netbox/
├── kustomization.yaml                    # Top-level kustomization (sets namespace: netbox)
└── base/
    ├── kustomization.yaml                # Base resources
    ├── netbox-deployment.yaml            # Main Netbox deployment and service
    ├── netbox-worker.yaml                # Background job worker (rqworker)
    ├── netbox-redis.yaml                 # Redis StatefulSet, config, PVC and Services
    ├── netbox-redis-networkpolicy.yaml   # Restricts Redis ingress to Netbox pods
    ├── configmap.yaml                    # Netbox configuration
    ├── netbox-secret.yaml                # ExternalSecret for credentials
    ├── netbox-service-account.yaml       # Service account
    └── secret-store.yaml                 # GCP Secret Manager SecretStore

Key Components

ResourcePurpose
Deployment (netbox)Netbox web application. Applies database migrations on startup
Deployment (netbox-worker)Executes background jobs via rqworker
StatefulSet (netbox-redis)Caching and the background task queue, on a 10Gi PersistentVolume
NetworkPolicyRestricts Redis ingress to Netbox pods only
ConfigMapDatabase and Redis connection settings
ExternalSecretCredentials from GCP Secret Manager
SecretStoreGCP Secret Manager integration

Background jobs

From Netbox v4.4 bulk import, modify and delete operations run as background jobs, and housekeeping is a system job. Without a running netbox-worker these operations are accepted and then never complete, so the worker is required rather than optional.

The worker deliberately overrides the image entrypoint. That entrypoint applies database migrations on startup, which must happen in exactly one place, so the web Deployment owns migrations and the worker executes rqworker directly.

Upgrade ordering

Because the worker runs application code directly, it must not process jobs while the web Deployment is still migrating the schema. ArgoCD sync waves enforce that ordering:

Resourceargocd.argoproj.io/sync-wave
netbox (web)0
netbox-worker1

ArgoCD waits for every resource in a wave to report Healthy before applying the next. The web Deployment’s readinessProbe hits a real page, so Healthy implies migrations have completed and the application is serving — only then is the worker updated.

Both Deployments also use strategy: Recreate, so no old pod survives alongside a new one.

Redis configuration

Redis runs as a StatefulSet with a 10Gi PersistentVolume, so the task queue survives a pod restart. Settings live in the netbox-redis-config ConfigMap:

SettingValueWhy
maxmemory128mbRedis must hit its own limit before the kernel OOM-kills the container. Held at half the 256Mi container limit because AOF rewrite and RDB snapshot both fork, and copy-on-write can approach a second copy of the dataset
maxmemory-policynoevictionProtects the queue — evicting a queued job loses work silently. The cost is that caching returns write errors rather than degrading once full
appendonlyyesAOF is the durability guarantee; RDB snapshots give fast restarts

Three details are load-bearing and are commented in the manifest:

  • args, not command. This preserves the image entrypoint, which chowns /data to the redis user and drops privileges via gosu. Overriding command skips both, leaving the server running as root on a root-owned volume.
  • exec probes, not tcpSocket. Kubelet probes originate from the node, which the NetworkPolicy does not admit. An exec probe runs inside the container and bypasses the network path.
  • The pod label app: netbox-redis must not change. The NetworkPolicy selects on it. If it drifted, the policy would match nothing and Redis would silently become reachable from anywhere in the cluster.

A single instance serves both the tasks (db 0) and caching (db 1) databases, which want opposite eviction policies. noeviction is chosen to protect jobs. Splitting into separate instances is the only complete fix, and is not warranted at current usage (~1 MB).

How to Update

Modifying Configuration

  1. Edit the appropriate file in k8s/infra-services/netbox/base/
  2. Commit and push to main
  3. ArgoCD auto-syncs the changes

Updating Netbox Version

  1. Update the container image in both base/netbox-deployment.yaml and base/netbox-worker.yaml
  2. Review the Netbox changelog
  3. Take an on-demand Cloud SQL backup of the infra-management instance
  4. Commit and push to main

Secrets Management

Netbox credentials are managed via External Secrets:

SecretSource
Database credentialsGCP Secret Manager
Secret keyGCP Secret Manager

The SecretStore in base/secret-store.yaml configures access to GCP Secret Manager.

Notifications

The Application sends Slack notifications to #platform-infra-notifications for:

  • Sync success
  • Out of sync detection
  • Sync failures
  • Degraded health