Persistent Volumes in Kubernetes — stateful workloads

“Kubernetes is great for stateless applications.” We’ve heard this many times. But what about databases, message brokers? PersistentVolumes and StorageClasses push Kubernetes into the world of stateful workloads.

PV, PVC, StorageClass — the trio you need to know¶

PersistentVolume (PV) is a piece of storage in the cluster. PersistentVolumeClaim (PVC) is a storage request from an application. StorageClass defines how PVs are dynamically created.

Dynamic provisioning is key. StorageClass automatically provisions a new volume when a PVC is created.

Our setup: Ceph on-premise, EBS in the cloud¶

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: ceph-rbd
provisioner: kubernetes.io/rbd
parameters:
  monitors: 10.0.0.1:6789,10.0.0.2:6789,10.0.0.3:6789
  pool: kube
reclaimPolicy: Retain

StatefulSets + PVC = stateful applications¶

StatefulSet guarantees stable hostname and persistent storage for each pod. Pod postgres-0 always gets PVC data-postgres-0. Even after restart, the pod connects to the same volume.

We run PostgreSQL, Redis and Elasticsearch this way. It’s not without problems — node failure with local storage means manual intervention. But for most cases it works reliably.

Backup strategy¶

• Volume snapshots (Ceph supports natively)
• Application backups (pg_dump for PostgreSQL)
• reclaimPolicy: Retain — never Delete for production data
• Test restore — backup without verified restore is an illusion

Stateful workloads in Kubernetes are reality¶

PersistentVolumes and StatefulSets push Kubernetes beyond the boundaries of stateless microservices. With good storage backend and thoughtful strategy, it works even for databases.