Postgresql storage on K3s - postgresql

Question

I am trying to setup a Postgresql Stateful-Set on a k3s single node (Raspberry Pi 4 8Gb) cluster which will be common between any services that end up needing postgresql. Currently, since this is just a single node, I am using the Rancher local-path storage provisioner (this will probably change as I add nodes) pointing to an external hdd set to mount, though this will probably change when I get around to adding nodes.
My pod spins up (after figuring out a small headache with mounting the data directory which was as per this issue), and I can access the postgresql instance with kubectl port-forward -n common pod/postgresql-stateful-set-0 5432:5432, and making whatever changes I need.
At this point, my problem manifests, whereby I notice that no data is persisted in my data directory. I have found this issue which is the exact issue I've encountered, but following all the examples of "fixes", from playing with the directory locations (which gave me chmod issues like encountered with the mounting data directory issues), to changing to a pv I defined.
Find below my yml file which is use kubectl apply -f postgres.yml and is based on the bitnami helm template:
---
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: postgresql-storage
provisioner: rancher.io/local-path
reclaimPolicy: Retain
volumeBindingMode: WaitForFirstConsumer
---
kind: PersistentVolume
apiVersion: v1
metadata:
name: postgresql-pv
namespace: common
spec:
storageClassName: postgresql-storage
persistentVolumeReclaimPolicy: Retain
capacity:
storage: 20Gi
accessModes:
- ReadWriteMany
hostPath:
path: "/mnt/storage/k3s/common/postgresql"
---
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
namespace: common
name: pvc-postgresql
spec:
storageClassName: postgresql-storage
accessModes:
- ReadWriteMany
resources:
requests:
storage: 20Gi
---
apiVersion: v1
kind: Secret
metadata:
name: postgresql-password
namespace: common
data:
POSTGRES_PASSWORD: <PWD>
---
apiVersion: v1
kind: ConfigMap
metadata:
name: postgresql-configmap
namespace: common
data:
POSTGRESQL_PORT_NUMBER: "5432"
PGDATA: /var/lib/postgresql/data/pgdata
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: postgresql-stateful-set
namespace: common
labels:
name: postgres-stateful-set
spec:
replicas: 1
serviceName: postgresql-stateful-set
updateStrategy:
rollingUpdate: {}
type: RollingUpdate
selector:
matchLabels:
app: postgresql
template:
metadata:
labels:
app: postgresql
spec:
containers:
- name: postgresql
image: docker.io/postgres:14.2-alpine
envFrom:
- configMapRef:
name: postgresql-configmap
- secretRef:
name: postgresql-password
ports:
- name: tcp-postgresql
containerPort: 5432
livenessProbe:
failureThreshold: 6
initialDelaySeconds: 30
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 5
exec:
command:
- /bin/sh
- -c
- exec pg_isready -U "rootAdmin" -h 127.0.0.1 -p 5432 -d rootDefault
readinessProbe:
failureThreshold: 6
initialDelaySeconds: 5
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 5
exec:
command:
- /bin/sh
- -c
- -e
- exec pg_isready -U "rootAdmin" -h 127.0.0.1 -p 5432 -d rootDefault
resources:
limits:
memory: "300Mi"
cpu: "300m"
requests:
cpu: 250m
memory: 256Mi
volumeMounts:
- name: dshm
mountPath: /dev/shm
- name: postgresql-data
mountPath: /var/lib/postgresql
volumes:
- name: dshm
emptyDir:
medium: Memory
volumeClaimTemplates:
- metadata:
name: postgresql-data
spec:
storageClassName: postgresql-storage
accessModes:
- "ReadWriteOnce"
resources:
requests:
storage: "20Gi"
---
kind: Service
apiVersion: v1
metadata:
name: postgresql-svc
namespace: common
spec:
selector:
app: postgresql
type: ClusterIP
clusterIP: None
ports:
- name: tcp-postgresql
port: 5432
protocol: TCP
Couple of things I should probably mention:
I added my own Storage class for the purposes of retaining the record rather then delete on the record being removed.
using the alpine version just for the sake of smaller image size.
The base OS that k3s is hosted on is Ubuntu 20.04 LTS
K3s was setup using this ansible playbook here
I have exec-ed into the container to see what is in the directory defined by PGDATA and confirmed that data was written.