PV file not saved on host - kubernetes

hi all quick question on host paths for persistent volumes
I created a PV and PVC here
apiVersion: v1
kind: PersistentVolume
metadata:
name: task-pv-volume
labels:
type: local
spec:
storageClassName: manual
capacity:
storage: 10Gi
accessModes:
- ReadWriteOnce
hostPath:
path: "/mnt/data"
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: task-pv-claim
spec:
storageClassName: manual
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 3Gi
and I ran a sample pod
apiVersion: v1
kind: Pod
metadata:
name: task-pv-pod
spec:
volumes:
- name: task-pv-storage
persistentVolumeClaim:
claimName: task-pv-claim
containers:
- name: task-pv-container
image: nginx
ports:
- containerPort: 80
name: "http-server"
volumeMounts:
- mountPath: "/usr/share/nginx/html"
name: task-pv-storage
i exec the pod and created a file
root#task-pv-pod:/# cd /usr/share/nginx/html
root#task-pv-pod:/usr/share/nginx/html# ls
tst.txt
However, when I go back to my host and try to ls the file , its not appearing. Any idea why? My PV and PVC are correct as I can see that it has been bounded.
ubuntu#ip-172-31-24-21:/home$ cd /mnt/data
ubuntu#ip-172-31-24-21:/mnt/data$ ls -lrt
total 0

A persistent volume (PV) is a kubernetes resource which has its own lifecycle independent of the pod pv documentation. Using a PVC to consume from a PV makes it visible in some other tool. For example azure files, ELB, a server with NFS, etc. My point here is that there is no reason why the PV should exist in the node.
If you want your persistence to be saved in the node use the hostPath option for PVs. Check this link. Though this is not a good production practice.

First of all, you don't need to create a PV if you are creating a PVC. PVCs create PV, if you have the right storageClass.
Second, hostPath is one delicate PV in Kubernetes world. That's the only PV that doen't need to be created to be mounted in a Pod. So you could have not created neither PV nor PVC and a hostPath volume would work just fine.
To make a test, delete your PV and PVC, and create your Pod like this:
apiVersion: v1
kind: Pod
metadata:
name: nginx-volume
labels:
app: nginx
spec:
containers:
- image: nginx
name: nginx
securityContext:
privileged: true
ports:
- containerPort: 80
name: nginx-http
volumeMounts:
- name: nginx
mountPath: /root/nginx-volume # path in the pod
volumes:
- name: nginx
hostPath:
path: /var/test # path in the host machine
I know this is a confusing concept, but that's how it is.

Related

Why local persistent volumes not visible in EKS?

In order to test if I can get self written software deployed in amazon using docker images,
I have a test eks cluster.
I have written a small test script that reads and writes a file to see if I understand how to deploy. I have successfully deployed it in minikube, using three replica's. The replica's all use a shared directory on my local file system, and in minikube that is mounted into the pods with a volume
The next step was to deploy that in the eks cluster. However, I cannot get it working in eks. The problem is that the pods don't see the contents of the mounted directory.
This does not completely surprise me, since in minikube I had to create a mount first to a local directory on the server. I have not done something similar on the eks server.
My question is what I should do to make this working (if possible at all).
I use this yaml file to create a pod in eks:
apiVersion: v1
kind: PersistentVolume
metadata:
name: "pv-volume"
spec:
storageClassName: local-storage
capacity:
storage: "1Gi"
accessModes:
- "ReadWriteOnce"
hostPath:
path: /data/k8s
type: DirectoryOrCreate
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: "pv-claim"
spec:
storageClassName: local-storage
accessModes:
- "ReadWriteOnce"
resources:
requests:
storage: "500M"
---
apiVersion: v1
kind: Pod
metadata:
name: ruudtest
spec:
containers:
- name: ruud
image: MYIMAGE
volumeMounts:
- name: cmount
mountPath: "/config"
volumes:
- name: cmount
persistentVolumeClaim:
claimName: pv-claim
So what I expect is that I have a local directory, /data/k8s, that is visible in the pods as path /config.
When I apply this yaml, I get a pod that gives an error message that makes clear the data in the /data/k8s directory is not visible to the pod.
Kubectl gives me this info after creation of the volume and claim
[rdgon#NL013-PPDAPP015 probeer]$ kubectl get pv,pvc
NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE
persistentvolume/pv-volume 1Gi RWO Retain Available 15s
persistentvolume/pvc-156edfef-d272-4df6-ae16-09b12e1c2f03 1Gi RWO Delete Bound default/pv-claim gp2 9s
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
persistentvolumeclaim/pv-claim Bound pvc-156edfef-d272-4df6-ae16-09b12e1c2f03 1Gi RWO gp2 15s
Which seems to indicate everything is OK. But it seems that the filesystem of the master node, on which I run the yaml file to create the volume, is not the location where the pods look when they access the /config dir.
On EKS, there's no storage class named 'local-storage' by default.
There is only a 'gp2' storage class, which is also used when you don't specify a storageClassName.
The 'gp2' storage class creates a dedicated EBS volume and attaches it your Kubernetes Node when required, so it doesn't use a local folder. You also don't need to create the pv manually, just the pvc:
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: "pv-claim"
spec:
storageClassName: gp2
accessModes:
- "ReadWriteOnce"
resources:
requests:
storage: "500M"
---
apiVersion: v1
kind: Pod
metadata:
name: ruudtest
spec:
containers:
- name: ruud
image: MYIMAGE
volumeMounts:
- name: cmount
mountPath: "/config"
volumes:
- name: cmount
persistentVolumeClaim:
claimName: pv-claim
If you want a folder on the Node itself, you can use a 'hostPath' volume, and you don't need a pv or pvc for that:
apiVersion: v1
kind: Pod
metadata:
name: ruudtest
spec:
containers:
- name: ruud
image: MYIMAGE
volumeMounts:
- name: cmount
mountPath: "/config"
volumes:
- name: cmount
hostPath:
path: /data/k8s
This is a bad idea, since the data will be lost if another node starts up, and your pod is moved to the new node.
If it's for configuration only, you can also use a configMap, and put the files directly in your kubernetes manifest files.
apiVersion: v1
kind: ConfigMap
metadata:
name: ruud-config
data:
ruud.properties: |
my ruud.properties file content...
---
apiVersion: v1
kind: Pod
metadata:
name: ruudtest
spec:
containers:
- name: ruud
image: MYIMAGE
volumeMounts:
- name: cmount
mountPath: "/config"
volumes:
- name: cmount
configMap:
name: ruud-config
Please check whether the pv got created and its "bound" to PVC by running below commands
kubectl get pv
kubectl get pvc
Which will give information whether the objects are created properly
The local path you refer to is not valid. Try:
apiVersion: v1
kind: Pod
metadata:
name: ruudtest
spec:
containers:
- name: ruud
image: MYIMAGE
volumeMounts:
- name: cmount
mountPath: /config
volumes:
- name: cmount
hostPath:
path: /data/k8s
type: DirectoryOrCreate # <-- You need this since the directory may not exist on the node.

How to have multiple pods access an existing NFS folder in Kubernetes?

I have a folder of TFRecords on a network that I want to expose to multiple pods. The folder has been exported via NFS.
I have tried creating a Persistent Volume, followed by a Persistent Volume Claim. However, that just creates a folder inside the NFS mount, which I don't want. Instead, I want to Pod to access the folder with the TFRecords.
I have listed the manifests for the PV and PVC.
apiVersion: v1
kind: PersistentVolume
metadata:
name: nfs-tfrecord-pv
spec:
capacity:
storage: 30Gi
accessModes:
- ReadWriteMany
persistentVolumeReclaimPolicy: Retain
nfs:
path: /media/veracrypt1/
server: 1.2.3.4
readOnly: false
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: nfs-tfrecord-pvc
namespace: default
spec:
accessModes:
- ReadWriteMany
storageClassName: nfs-tfrecord
resources:
requests:
storage: 1Gi
I figured it out. The issue was I was looking at the problem the wrong way. I didn't need any provisioning. Instead, what was need was to simply mount the NFS volume within the container:
kind: Pod
apiVersion: v1
metadata:
name: pod-using-nfs
spec:
containers:
- name: app
image: alpine
volumeMounts:
- name: data
mountPath: /mnt/data
command: ["/bin/sh"]
args: ["-c", "sleep 500000"]
volumes:
- name: data
nfs:
server: 1.2.3.4
path: /media/foo/DATA

How to exempt a directory when using readOnlyRootFilesystem in kubernetes?

I need to block my k8s pods from writing to root folders, with an exemption to /tmp dir. There are 2 reasons I need to write to this dir:
Flask needs to write to somewhere. It's trying to write to /tmp and /etc/... and /opt/... , but all of them are blocked because it's under root folder
I'm going to need to write to a file for liveness probe, but if the entire file system is blocked, then I can't do it
I'm running kubernetes 1.13.6-gke.13 in GKE
The relevant part from the yaml file:
securityContext:
runAsUser: 1000
readOnlyRootFilesystem: true
runAsNonRoot: true
I expect the pod to be able to write to a predefined folder, maybe a mounted one.
Create a volume mount for /tmp directory.
volumeMounts:
- mountPath: /tmp
name: tmp
And in Volumes -
volumes:
- emptyDir: {}
name: tmp
As I understand, you would like to create a POD with access to a local directory. You need to create PV, PVC and POD.
PV definition:
kind: PersistentVolume
apiVersion: v1
metadata:
name: pv-flaskapp
labels:
type: local
spec:
storageClassName: <your-storageclass-name>
capacity:
storage: 3Gi
accessModes:
- ReadWriteOnce
hostPath:
path: "/opt/test_flask/app"
PVC definition:
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
name: pvc-flaskapp
spec:
storageClassName: <your-storageclass-name>
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 1Gi
POD definition:
apiVersion: v1
kind: Pod
metadata:
name: flaskapp
spec:
containers:
- image: flask:latest
name: flaskapp
ports:
- containerPort: 8080
name: flaskapp
volumeMounts:
- mountPath: /usr/local/flask/webapps
name: test-volume
volumes:
- name: test-volume
persistentVolumeClaim:
claimName: pvc-flask
Now you can check if everything works fine:
$ kubectl exec -it flaskapp bash
root#flaskapp:/usr/local/flask# mkdir /usr/local/flask/webapps/sample
root#flaskapp:/usr/local/flask# touch /usr/local/flask/webapps/sample/testfile
root#flaskapp:/usr/local/flask# ls /usr/local/flask/webapps/sample/
testfile
Now when you look at host, you will see the newly created file:
[root#master user]# ls /opt/test_flask/app/sample/
testfile
I hope it will helps you.

Creating a NFS sidecar for Kubernetes

I am trying to create a NFS sidecar for Kubernetes. The goal is to be able to mount an NFS volume to an existing pod without affecting performance. At the same time, I want to be able to mount the same NFS volume onto another pod or server (read-only perhaps) in order to view the content there. Has anyone tried this? Do anyone have the procedure?
Rather than use a sidecar I would suggest using a PersistentVolume which uses the NFS driver and PersistentVolumeClaim. If you use the RWX/ReadWriteMany access mode, you'll be able to mount the share into multiple pods.
For examplen the pv:
kind: PersistentVolume
apiVersion: v1
metadata:
name: mypv
spec:
capacity:
storage: 2Gi
nfs:
server: my.nfs.server
path: /myshare
accessModes:
- ReadWriteMany
persistentVolumeReclaimPolicy: Recycle
the pvc:
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
name: myclaim
spec:
accessModes:
- ReadWriteMany
resources:
requests:
storage: 2Gi
and mounted in a pod:
apiVersion: v1
kind: Pod
metadata:
name: mypod
spec:
containers:
- name: myfrontend
image: nginx
volumeMounts:
- mountPath: "/var/www/html"
name: mypd
volumes:
- name: mypd
persistentVolumeClaim:
claimName: myclaim
Kubernetes Docs on Persistent Volumes

kubernetes persistence volume and persistence volume claim exceeded storage

By following kubernetes guide i have created a pv, pvc and pod. i have claimed only 10Mi of out of 20Mi pv. I have copied 23Mi that is more than my pv. But my pod is still running. Can any one explain ?
pv-volume.yaml
kind: PersistentVolume
apiVersion: v1
metadata:
name: task-pv-volume
labels:
type: local
spec:
storageClassName: manual
capacity:
storage: 20Mi
accessModes:
- ReadWriteOnce
hostPath:
path: "/mnt/data"
pv-claim.yaml
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
name: task-pv-claim
spec:
storageClassName: manual
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 10Mi
pv-pod.yaml
kind: Pod
apiVersion: v1
metadata:
name: task-pv-pod
spec:
volumes:
- name: task-pv-storage
persistentVolumeClaim:
claimName: task-pv-claim
containers:
- name: task-pv-container
image: nginx
ports:
- containerPort: 80
name: "http-server"
volumeMounts:
- mountPath: "/usr/share/nginx/html"
name: task-pv-storage
Probably you can copy as much data into shared storage /mnt/data (on your active node) using any of applied POD's storages ,/usr/share/nginx/html, shared between node and pods till your node will stop responding.
In case you need to test this scenario in more real conditions could you please consider create NFS persistent storage using GlusterFS, nfs-utils, or mount a raw partition file made with dd.
In Minikube nodes are using ephemeral-storages. Detailed information about node/pod resources you can find here:
https://kubernetes.io/docs/tasks/administer-cluster/reserve-compute-resources/#node-allocatable
Hope this help.