Talos OS: Running Postgres with CloudNativePG on Ceph

Introduction

The previous post got me a ceph-rbd StorageClass backed by three-way replicated storage across the cluster. The thing I actually wanted it for is Postgres, so this post is a CloudNativePG (CNPG) cluster on top of it, with automated backups to S3 through the Barman Cloud plugin.

Prerequisites

A working ceph-rbd default StorageClass from the previous post.
Helm 3.
An S3-compatible bucket and credentials for backups.

What Is CloudNativePG

CloudNativePG (CNPG) is a Kubernetes operator for running Postgres - you describe the cluster you want as a Cluster custom resource and it handles the rest: provisioning the instances, promoting a replica if the primary dies, wiring up roles and databases declaratively, and driving backups through a plugin like Barman Cloud. The part I actually wanted it for is not having to hand-roll failover myself - before this I’d have needed a StatefulSet, my own replication setup, and something watching the primary to promote a replica if it went down. CNPG does all of that from one CR.

Installing the Operator

CNPG needs cert-manager for its webhooks, plus the Barman Cloud plugin, which has to run in the same namespace as the CNPG operator itself. I wrap all of that - cert-manager, the operator, the backup plugin, and the cluster chart from the next section - as dependencies in my own postgres chart, so the whole thing lands as one Helm release. This part of the values file is the operator side:

yaml
cert-manager:
  crds:
    enabled: true

cloudnative-pg:
  config:
    clusterWide: true

plugin-barman-cloud: {}

clusterWide: true on the operator means it watches Cluster resources in every namespace, not just its own - useful once more than one app needs its own database.

The Cluster

I run this through the CNPG project’s own cluster chart, using its TimescaleDB packaging (Postgres 17 + the TimescaleDB extension bundled in, even though I’m not using hypertables yet - it was just the variant I standardized on):

yaml
shared:
  enabled: true
  type: timescaledb
  version:
    postgresql: "17"
    timescaledb: "2.17"
  mode: standalone   # fresh cluster, not a recovery

  cluster:
    instances: 1
    storage:
      size: 100Gi
      storageClass: ""   # empty = cluster default StorageClass = ceph-rbd
    resources:
      requests:
        cpu: "1"
        memory: 2Gi
      limits:
        cpu: "2"
        memory: 4Gi
    plugins:
      - name: barman-cloud.cloudnative-pg.io
        enabled: true
        isWALArchiver: true

With both pieces of the values file in place, one helm install brings up cert-manager, the CNPG operator, the Barman Cloud plugin, and the Postgres cluster itself together:

shell

❯ helm dependency build
❯ helm install postgres . -n postgres --create-namespace -f values.yaml

Leaving storageClass empty is deliberate - it falls back to whatever the cluster’s default StorageClass is, so I don’t have to hardcode ceph-rbd in two places. mode: standalone is what makes this a fresh cluster instead of a restore; flipping that field plus a bootstrap.recovery block pointed at the same backup location is how I’d stand up a replacement cluster from backups later.

A Test Role and Database

CNPG can create roles and databases declaratively instead of running SQL by hand:

yaml
  cluster:
    roles:
      - name: demo
        ensure: present
        login: true
        superuser: false
        createdb: false
        connectionLimit: -1
        inherit: true
        passwordSecret:
          name: demo-postgres-credentials

yaml
apiVersion: postgresql.cnpg.io/v1
kind: Database
metadata:
  name: demo
spec:
  name: demo
  owner: demo
  cluster:
    name: postgres-cluster

Connecting to It

To poke at it from inside the cluster, I run a throwaway client pod rather than exec into the database pod directly:

yaml
apiVersion: v1
kind: Pod
metadata:
  name: psql-client
spec:
  containers:
    - name: psql
      image: postgres:17
      command: ["sleep", "infinity"]

shell
❯ kubectl exec -it psql-client -- bash
root@psql-client:/# psql "host=postgres-cluster-rw.postgres.svc port=5432 dbname=demo user=demo"
Password for user demo:
psql (17.2)
Type "help" for help.

demo=> \dt
Did not find any relations.
demo=> \q

Scheduled Backups

Backups go through the Barman Cloud plugin to S3, on a daily schedule with a 30-day retention window:

yaml
  backups:
    enabled: true
    method: plugin
    pluginConfiguration:
      name: barman-cloud.cloudnative-pg.io
    provider: s3
    s3:
      region: eu-central-1
      bucket: my-pg-backups
      path: "/shared"
      accessKey: "<redacted>"
      secretKey: "<redacted>"
    scheduledBackups:
      - name: daily-backup
        schedule: "0 0 0 * * *"
        backupOwnerReference: self
        method: plugin
        pluginConfiguration:
          name: barman-cloud.cloudnative-pg.io
    retentionPolicy: "30d"

Those keys sit plaintext in the same values file as everything else in this homelab config - fine for a private repo, not something I’d do anywhere more exposed. I can also trigger a one-off backup outside the schedule with a Backup object:

shell
❯ kubectl apply -f - <<EOF
apiVersion: postgresql.cnpg.io/v1
kind: Backup
metadata:
  name: postgres-manual-backup
  namespace: postgres
spec:
  cluster:
    name: postgres-cluster
  method: plugin
  pluginConfiguration:
    name: barman-cloud.cloudnative-pg.io
EOF

❯ kubectl get backup postgres-manual-backup -n postgres
NAME                     AGE   CLUSTER            METHOD   PHASE       ERROR
postgres-manual-backup   14s   postgres-cluster    plugin   completed

I haven’t had to pull the recovery lever on this cluster for real yet. Restoring is a separate Cluster with mode: recovery (instead of standalone) pointed at the same S3 path - CNPG replays the base backup plus WAL and comes back up as a new cluster, rather than overwriting the one that’s still running.

Conclusion

One Helm release gets me an operator-managed Postgres cluster, replicated storage underneath it courtesy of Ceph, and backups leaving the cluster every night without me doing anything. It’s a fresh cluster and I haven’t tested a disaster-recovery restore against it yet - that’s the next thing on my list, not something I’d claim works until I’ve actually broken it on purpose. Next post: this cluster is only reachable from inside the LAN, so I’ll expose an app in front of it to the internet through Cloudflare.