Kubernetes "shared" persistent volume on DigitalOcean - kubernetes

I have a persistent volume defined as
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: ghost-cms-content
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 5Gi
storageClassName: do-block-storage
and a deployment defined as
---
kind: Deployment
apiVersion: apps/v1
metadata:
name: ghost-cms
spec:
replicas: 4
strategy:
type: RollingUpdate
rollingUpdate:
maxSurge: 1
maxUnavailable: 1
selector:
matchLabels:
app: ghost-cms
tier: frontend
template:
metadata:
labels:
app: ghost-cms
tier: frontend
spec:
topologySpreadConstraints:
- maxSkew: 1
topologyKey: topology.kubernetes.io/region
whenUnsatisfiable: ScheduleAnyway
labelSelector:
matchLabels:
app: ghost-cms
tier: frontend
containers:
- name: ghost-cms
image: ghost:4.6-alpine
imagePullPolicy: Always
ports:
- containerPort: 2368
volumeMounts:
- mountPath: /var/lib/ghost/content
name: content
env:
- name: url
value: https://ghost.site
resources:
requests:
cpu: 100m
memory: 128Mi
limits:
cpu: 250m
memory: 256Mi
volumes:
- name: content
persistentVolumeClaim:
claimName: ghost-cms-content
but each replica appears to have a unique volume that is not shared with the rest of the replicas. For instance, when I create a text file inside /var/lib/ghost/content in one of the pods, I don't see it in the volume of the other pods. What am I doing wrong?

PVC with permission
accessModes:
- ReadWriteOnce
Each pod will get the one volume or PVC, as it's readwrite once.
If you want to keep shared volume across replicas you can use the NFS with accessMode ReadWriteMany
accessModes:
- ReadWriteMany
Read more at : https://kubernetes.io/docs/concepts/storage/persistent-volumes/#access-modes
Example : https://medium.com/asl19-developers/create-readwritemany-persistentvolumeclaims-on-your-kubernetes-cluster-3a8db51f98e3
You can also use Minio, GlusterFS to creeat the NFS or any managed service like GCP filestore providing NFS and attach that to POD.
GKE example : https://medium.com/#Sushil_Kumar/readwritemany-persistent-volumes-in-google-kubernetes-engine-a0b93e203180

Related

Accessing PVC created by pod from Statefulset in another pod created by daemonset using azure disk

I have app-1 pods created by StatefulSet and in that, I am creating PVC as well
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: app-1
spec:
replicas: 3
selector:
matchLabels:
app: app-1
serviceName: "app-1"
template:
metadata:
labels:
app: app-1
spec:
containers:
- name: app-1
image: registry.k8s.io/nginx-slim:0.8
resources:
requests:
cpu: 100m
memory: 128Mi
limits:
cpu: 250m
memory: 256Mi
ports:
- containerPort: 4567
volumeMounts:
- name: app-1-state-volume-claim
mountPath: /app1Data
- name: app-2-data-volume-claim
mountPath: /app2Data
volumeClaimTemplates:
- metadata:
name: app-1-state-volume-claim
spec:
accessModes: [ "ReadWriteOnce" ]
storageClassName: "managed-csi-premium"
resources:
requests:
storage: 1Gi
- metadata:
name: app-2-data-volume-claim
spec:
accessModes: [ "ReadWriteOnce" ]
storageClassName: "managed-csi-premium"
resources:
requests:
storage: 1Gi
state of the app1 is maintained in PVC - app-1-state-volume-claim
app1 is also creating data for app2 in PVC - app-2-data-volume-claim
I want to access app-2-data-volume-claim in another pod deployment described below
apiVersion: apps/v1
kind: DaemonSet
metadata:
name: app-2
spec:
selector:
matchLabels:
name: app-2
template:
metadata:
labels:
name: app-2
spec:
containers:
- name: app-2
image: quay.io/fluentd_elasticsearch/fluentd:v2.5.2
resources:
limits:
memory: 200Mi
requests:
cpu: 100m
memory: 200Mi
volumeMounts:
- name: app2Data
mountPath: /app2Data
volumes:
- name: app2Data
persistentVolumeClaim:
claimName: app-2-data-volume-claim
This is failing with below output
persistentvolumeclaim "app-2-data-volume-claim" not found
How can I do that?I cannot use Azure file share due to app-1 limitation.

SHOULD CSI max volume per node work with ephemeral volumes?

I created a CSI, which will allocate host path space for pods. I set the maxVolumePerNode as '1' in NodeGetInfoResponse. I also added ControllerServiceCapability_RPC_PUBLISH_UNPUBLISH_VOLUME capability to ControllerGetCapabilities in CSI controller.
However, though in the CSINode, I can see maxvolumepernode was set to 1 for my CSI driver, still I can place any number of pods onto one node and many pvcs were created.
My question is, is 'maxvolumepernode' suppose to work with ephemeral volumes? If yes,what else I need to enable this threshold?
Reference:
Kubernetes version: 1.20
CSINode in my k8s cluster
CSINode:
apiVersion: storage.k8s.io/v1
kind: CSINode
metadata:
spec:
drivers:
- allocatable:
count: 1
name: mycsidriver
nodeID: nodeid
topologyKeys: null
3) Multiple PVC were created:
<!-- begin snippet: js hide: false console: true babel: false -->
Deployment spec
apiVersion: apps/v1
kind: Deployment
metadata:
name: test
spec:
replicas: 5
selector:
matchLabels:
app: test
template:
metadata:
labels:
app: test
spec:
containers:
- name: testcontainer
image: ubuntu
command:
- sleep
- "3600"
resources:
limits:
memory: "4Gi"
requests:
memory: "1Mi"
volumeMounts:
- name: csi-volume
mountPath: /data/test
securityContext:
privileged: true
volumes:
- name: csi-volume
ephemeral:
volumeClaimTemplate:
metadata:
labels:
type: mycsidriver
spec:
accessModes: [ "ReadWriteOnce" ]
storageClassName: "mycsidriver"
resources:
requests:
storage: 1Gi

WaitForFirstConsumer PersistentVolumeClaim waiting for first consumer to be created before binding. Auto provisioning does not work

Hi I know this might be a possible duplicate, but I cannot get the answer from this question.
I have a prometheus deployment and would like to give it a persistent volume.
apiVersion: apps/v1
kind: Deployment
metadata:
name: prometheus-deployment
namespace: monitoring
labels:
app: prometheus-server
spec:
replicas: 1
selector:
matchLabels:
app: prometheus-server
template:
metadata:
labels:
app: prometheus-server
spec:
containers:
- name: prometheus
image: prom/prometheus
args:
- "--storage.tsdb.retention.time=60d"
- "--config.file=/etc/prometheus/prometheus.yml"
- "--storage.tsdb.path=/prometheus/"
ports:
- containerPort: 9090
resources:
requests:
cpu: 500m
memory: 500M
limits:
cpu: 1
memory: 1Gi
volumeMounts:
- name: prometheus-config-volume
mountPath: /etc/prometheus/
- name: prometheus-storage-volume
mountPath: /prometheus/
volumes:
- name: prometheus-config-volume
configMap:
defaultMode: 420
name: prometheus-server-conf
- name: prometheus-storage-volume
persistentVolumeClaim:
claimName: prometheus-pv-claim
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: prometheus-pv-claim
spec:
storageClassName: default
volumeMode: Filesystem
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 4Gi
Now both the pvc and the deployment cannot be scheduled because the pvc waits for the deployment and the other way around. As far as I am concerned we have a cluster with automatic provisioning, thus I cannot just create a pv. How can I solve this problem, because other deployments do use pvc in the same style and it works.
Its because of namespace. PVC is a namespaced object you can look here. Your PVC is on the default namespace. Moving it to monitoring namespace should work.
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: prometheus-pv-claim
namespace: monitoring
spec:
storageClassName: default
volumeMode: Filesystem
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 4Gi
Does your PVC deploy the correct namespace with deployment?
And make sure the StorageClass has volumeBindingMode: WaitForFirstConsumer
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: default
...
volumeBindingMode: WaitForFirstConsumer

How to deploy a single instance mongodb with persistant volume using NFS

I have a microservice that is working on my laptop. However, I am using docker compose. I am working to deploy to a kubernetes cluster which I have already set up. I am stuck on making data persistent. E.g here is my mongodb in docker-compose
systemdb:
container_name: system-db
image: mongo:4.4.1
restart: always
ports:
- '9000:27017'
volumes:
- ./system_db:/data/db
networks:
- backend
Since it is an on premise solution, I went with an NFS server. I have created a Persistent Volume and Persistent Volume Claim (pvc-nfs-pv1) which seem to work well when testing with nginx. However, I don't know how to deploy a mongodb statefulset to use the pvc. I am not implementing a replicaset.
Here is my yaml:
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: mongod
spec:
serviceName: mongodb-service
replicas: 1
selector:
matchLabels:
role: mongo
template:
metadata:
labels:
role: mongo
environment: test
spec:
terminationGracePeriodSeconds: 10
containers:
- name: mongod-container
image: mongo
resources:
requests:
cpu: "0.2"
memory: 200Mi
ports:
- containerPort: 27017
volumeMounts:
- name: pvc-nfs-pv1
mountPath: /data/db
volumeClaimTemplates:
- metadata:
name: pvc-nfs-pv1
annotations:
volume.beta.kubernetes.io/storage-class: "standard"
spec:
accessModes: [ "ReadWriteOnce" ]
resources:
requests:
storage: 500Mi
How do i do it?
volumeClaimTemplates are used for dynamic volume provisioning. So you're defining one volume claim template which will be used to create a PersistentVolumeClaim for each pod.
The volumeClaimTemplates will provide stable storage using
PersistentVolumes
provisioned by a PersistentVolume Provisioner
So for your use case you would need to create storageclass with nfs provisioner. NFS Subdir external provisioner is an automatic provisioner that use your existing and already configured NFS server to support dynamic provisioning of Kubernetes Persistent Volumes via Persistent Volume Claims. Persistent volumes are provisioned as ${namespace}-${pvcName}-${pvName}.
Here`s an example how to define storage class:
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: managed-nfs-storage
provisioner: fuseim.pri/ifs # or choose another name, must match deployment's env PROVISIONER_NAME'
parameters:
pathPattern: "${.PVC.namespace}/${.PVC.annotations.nfs.io/storage-path}" # waits for nfs.io/storage-path annotation, if not specified will accept as empty string.
onDelete: delete
Ok, I have a solution. It works simply by selecting the volume by using the matchLabels selector.
apiVersion: v1
kind: PersistentVolume
metadata:
name: mongodb-data-volume
labels:
app: moderetic
type: mongodb
role: data
spec:
storageClassName: hcloud-volumes
capacity:
storage: 10Gi
accessModes:
- ReadWriteOnce
csi:
volumeHandle: "11099996"
driver: csi.hetzner.cloud
fsType: ext4
---
---
apiVersion: mongodbcommunity.mongodb.com/v1
kind: MongoDBCommunity
metadata:
name: system-mongodb
labels:
app: moderetic
type: mongodb
spec:
members: 1
type: ReplicaSet
version: "4.2.6"
logLevel: INFO
security:
authentication:
modes: ["SCRAM"]
users:
- name: moderetic
db: moderetic
passwordSecretRef:
name: mongodb-secret
roles:
- name: clusterAdmin
db: moderetic
- name: userAdminAnyDatabase
db: moderetic
scramCredentialsSecretName: moderetic-scram-secret
additionalMongodConfig:
storage.wiredTiger.engineConfig.journalCompressor: zlib
persistent: true
statefulSet:
spec:
template:
spec:
containers:
- name: mongod
resources:
requests:
cpu: 1
memory: 1Gi
limits:
memory: 8Gi
- name: mongodb-agent
resources:
requests:
memory: 50Mi
limits:
cpu: 500m
memory: 256Mi
volumeClaimTemplates:
- metadata:
name: data-volume
spec:
accessModes: ["ReadWriteOnce"]
storageClassName: hcloud-volumes
resources:
requests:
storage: 10Gi
selector:
matchLabels:
app: moderetic
type: mongodb
role: data
- metadata:
name: logs-volume
spec:
accessModes: ["ReadWriteOnce"]
storageClassName: hcloud-volumes
resources:
requests:
storage: 10Gi
selector:
matchLabels:
app: moderetic
type: mongodb
role: logs
Your question is how the mongo StatefulSet is going to use the pvc u have created ? By default It wont . It will create numbers of new pvc (depending of number of replicaset) automatically via the volumeClaimTemplates and it will be named like so : pvc-nfs-pv1-mongod-0 , pvc-nfs-pv1-mongod-1 etc ..
So if you want to use the pvc you created change the name to match pvc-nfs-pv1-mongod-0
something like this
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
labels:
role: mongo
name: pvc-nfs-pv1-mongod-0
namespace: default
spec:
...
volumeName: nfs-pv1
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 500Mi
...
However I dont recommend using this method (issue: when you have many other replicasets .. do you have to create all the pvcs manually and the correspondent pv ).. Here is similar questions is asked in here and also in here , I recommend using Dynamic NFS Provisioning.
Hope I helped
I do not use NFS but volumes at hetzner.com where my dev server is running. But I have exactly the same problem: As it is my dev system I destroy and rebuild it regularly. And by doing so I want the data on my volumes survive the deletion of the whole cluster. And when I rebuild it, all the volumes shall be mounted to the right pod.
For my postgres this works just fine. But using the mongodb kubernetes operator I am not able to get this running. The one mongodb pod stays forever in the state "Pending" because the PVC I created and bound manually to the volume is already bound to a volume. Or so it seems to me.
I am thankful for any help,
Tobias
The exact message I can see is:
0/1 nodes are available: 1 pod has unbound immediate PersistentVolumeClaims
PVC and PV:
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: data-volume-system-mongodb-0
labels:
app: moderetic
type: mongodb
spec:
storageClassName: hcloud-volumes
volumeName: mongodb-data-volume
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 10Gi
---
apiVersion: v1
kind: PersistentVolume
metadata:
name: mongodb-data-volume
labels:
app: moderetic
type: mongodb
spec:
storageClassName: hcloud-volumes
claimRef:
name: data-volume-system-mongodb-0
capacity:
storage: 10Gi
accessModes:
- ReadWriteOnce
csi:
volumeHandle: "11099996"
driver: csi.hetzner.cloud
fsType: ext4
And the mongodb StatefulSet:
apiVersion: mongodbcommunity.mongodb.com/v1
kind: MongoDBCommunity
metadata:
name: system-mongodb
labels:
app: moderetic
type: mongodb
spec:
members: 1
type: ReplicaSet
version: "4.2.6"
security:
authentication:
modes: ["SCRAM"]
users:
- name: moderetic
db: moderetic
passwordSecretRef:
name: mongodb-secret
roles:
- name: clusterAdmin
db: moderetic
- name: userAdminAnyDatabase
db: moderetic
scramCredentialsSecretName: moderetic-scram-secret
additionalMongodConfig:
storage.wiredTiger.engineConfig.journalCompressor: zlib
persistent: true
statefulSet:
spec:
template:
spec:
containers:
- name: mongod
resources:
requests:
cpu: 1
memory: 1Gi
limits:
memory: 8Gi
- name: mongodb-agent
resources:
requests:
memory: 50Mi
limits:
cpu: 500m
memory: 256Mi
volumeClaimTemplates:
- metadata:
name: data-volume
spec:
accessModes: ["ReadWriteOnce"]
storageClassName: hcloud-volumes
resources:
requests:
storage: 10Gi
- metadata:
name: logs-volume
spec:
accessModes: ["ReadWriteOnce"]
storageClassName: hcloud-volumes
resources:
requests:
storage: 10Gi

Persistent volume Kubernetes on Google Cloud

I have a Redis pod on my Kubernetes cluster on Google Cloud. I have built PV and the claim.
kind: PersistentVolume
apiVersion: v1
metadata:
name: redis-pv
labels:
type: local
spec:
storageClassName: manual
capacity:
storage: my-size
accessModes:
- ReadWriteOnce
hostPath:
path: "/data"
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
labels:
app: postgres
name: redis-pv-claim
spec:
storageClassName: manual
accessModes:
- ReadWriteOnce
resources:
requests:
storage: my size
I also mounted it in my deployment.yaml
volumeMounts:
- mountPath: /data
name: redis-pv-claim
volumes:
- name: redis-pv-claim
persistentVolumeClaim:
claimName: redis-pv-claim
I can't see any error while running describe pod
Volumes:
redis-pv-claim:
Type: PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
ClaimName: redis-pv-claim
ReadOnly: false
But it just can't save any key. After every deployment, the "/data" folder is just empty.
My NFS is active now but i still cant keep data .
Describe pvc
Namespace: my namespace
StorageClass: nfs-client
Status: Bound
Volume: pvc-5d278b27-a51e-4262-8c1b-68b290b21fc3
Labels: <none>
Annotations: pv.kubernetes.io/bind-completed: yes
pv.kubernetes.io/bound-by-controller: yes
volume.beta.kubernetes.io/storage-class: nfs-client
volume.beta.kubernetes.io/storage-provisioner: cluster.local/ext1-nfs-client-provisioner
Finalizers: [kubernetes.io/pvc-protection]
Capacity: 1Gi
Access Modes: RWX
VolumeMode: Filesystem
Mounted By: my grafana pod
Events: <none>
Describe pod gives me an error though.
Warning FailedMount 18m kubelet, gke-devcluster-pool-1-36e6a393-rg7d MountVolume.SetUp failed for volume "pvc-5d278b27-a51e-4262-8c1b-68b290b21fc3" : mount failed: exit status 1
Mounting command: systemd-run
Mounting arguments: --description=Kubernetes transient mount for /var/lib/kubelet/pods/8f7b6630-ed9b-427a-9ada-b75e1805ed60/volumes/kubernetes.io~nfs/pvc-5d278b27-a51e-4262-8c1b-68b290b21fc3 --scope -- /
home/kubernetes/containerized_mounter/mounter mount -t nfs 192.168.1.21:/mnt/nfs/development-test-claim-pvc-5d278b27-a51e-4262-8c1b-68b290b21fc3 /var/lib/kubelet/pods/8f7b6630-ed9b-427a-9ada-b75e1805ed60
/volumes/kubernetes.io~nfs/pvc-5d278b27-a51e-4262-8c1b-68b290b21fc3
Output: Running scope as unit: run-ra5925a8488ef436897bd44d526c57841.scope
Mount failed: mount failed: exit status 32
Mounting command: chroot
Working redis with PV and PVC on GKE
apiVersion: v1
kind: Service
metadata:
name: redis
spec:
type: LoadBalancer
ports:
- port: 6379
name: redis
selector:
app: redis
---
apiVersion: apps/v1beta2
kind: StatefulSet
metadata:
name: redis
spec:
selector:
matchLabels:
app: redis
serviceName: redis
replicas: 1
template:
metadata:
labels:
app: redis
spec:
containers:
- name: redis
image: redislabs/rejson
args: ["--requirepass", "password", "--appendonly", "no", "--loadmodule", "/usr/lib/redis/modules/rejson.so"]
ports:
- containerPort: 6379
name: redis
resources:
limits:
cpu: .50
memory: 1500Mi
requests:
cpu: .25
memory: 1000Mi
volumeMounts:
- name: redis-volume
mountPath: /data
volumeClaimTemplates:
- metadata:
name: redis-volume
spec:
accessModes: [ "ReadWriteOnce" ]
resources:
requests:
storage: 5Gi
you can update image in this stateful sets as per need.