How to specify a storage class to user pvc in kubeflow - kubernetes

I am trying to attach a storage class to all the PVC request created by individual user pods for jupyter notebooks in kubeflow.
I tried editing some values and specify storage_class. but none of it is working, whenever a new pvc comes up it doesnot come with a storage class name.
The desired result is whenever a pvc of user pods comes, it should have the name of storage class attached with it. Kindly help with this. I am stuck from last day

you need to have a default storage class in your cluster, so if a pvc does not specify any storage class then default one would be selected.
List the StorageClasses in your cluster:
kubectl get storageclass
Mark a StorageClass as default:
set the annotation storageclass.kubernetes.io/is-default-class=true.
kubectl patch storageclass <your-class-name> -p '{"metadata": {"annotations":{"storageclass.kubernetes.io/is-default-class":"true"}}}'
Here are the detail steps change-default-storage-class

Basing on documentation
While PersistentVolumeClaims allow a user to consume abstract storage resources, it is common that users need PersistentVolumes with varying properties, such as performance, for different problems.
Cluster administrators need to be able to offer a variety of PersistentVolumes that differ in more ways than just size and access modes, without exposing users to the details of how those volumes are implemented. For these needs there is the StorageClass resource.
A PersistentVolume (PV) is a piece of storage in the cluster that has been provisioned by an administrator or dynamically provisioned using Storage Classes.
apiVersion: v1
kind: PersistentVolume
metadata:
name: task-pv-volume
labels:
type: local
spec:
storageClassName: <name_of_your_StorageClass>
capacity:
storage: 10Gi
accessModes:
- ReadWriteOnce
hostPath:
path: "/mnt/data"
A PersistentVolumeClaim (PVC) is a request for storage by a user.
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: task-pv-claim
spec:
storageClassName: <name_of_your_StorageClass>
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 3Gi
Then you can create a Pod that uses your PVC as a volume(which uses PV with storageClass)
apiVersion: v1
kind: Pod
metadata:
name: task-pv-pod
spec:
volumes:
- name: task-pv-storage
persistentVolumeClaim:
claimName: task-pv-claim
containers:
- name: task-pv-container
image: nginx
ports:
- containerPort: 80
name: "http-server"
volumeMounts:
- mountPath: "/usr/share/nginx/html"
name: task-pv-storage
Before you are going to create a PV and PVC StorageClass must already exist, if it's not a default one will be used instead.
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: <name_of_your_StorageClass>
provisioner: kubernetes.io/aws-ebs
parameters:
type: gp2
reclaimPolicy: Retain
allowVolumeExpansion: true
mountOptions:
- debug
volumeBindingMode: Immediate
You can check your StorageClasses with this command:
kubectl get sc

Related

What Persistent Volume storage solutions I can use when I run my Kubernetes cluster on GCP?

I have deployed my Kubernetes cluster on GCP Compute Engines and having 3 Master Nodes and 3 Worker Nodes (It's not a GKE Cluster). Can anybody suggest me what storage options I can use for my cluster? If I create a virtual disk on GCP, can I use that disk as a persistent storage?
You can use GCE Persistent Disk Storage Class.
Here is how you create the storage class:
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: ssd
provisioner: kubernetes.io/gce-pd
parameters:
type: pd-ssd
Then you do the following to create the PV & PVC and to attach to your pod.
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: gce-claim
spec:
accessModes:
- ReadWriteOnce
storageClassName: ssd
resources:
requests:
storage: 10Gi
---
apiVersion: v1
kind: Pod
metadata:
name: webserver-pd
spec:
containers:
- image: httpd
name: webserver
volumeMounts:
- mountPath: /data
name: dynamic-volume
volumes:
- name: dynamic-volume
persistentVolumeClaim:
claimName: gce-claim
Example taken from this blog post
There are two types of provisioning Persistent Volumes: Static Provisioning and Dynamic Provisioning.
I will briefly describe each of these types.
Static Provisioning
Using this approach you need to create Disk, PersistentVolume and PersistentVolumeClaim manually.
I've create simple example for you to illustrate how it works.
First I created disk, on GCP we can use gcloud command:
$ gcloud compute disks create --size 10GB --region europe-west3-c test-disk
NAME ZONE SIZE_GB TYPE STATUS
test-disk europe-west3-c 10 pd-standard READY
Next I created PV and PVC using this manifest files:
---
apiVersion: v1
kind: PersistentVolume
metadata:
name: pv-test
spec:
accessModes:
- ReadWriteOnce
capacity:
storage: 10Gi
gcePersistentDisk:
pdName: test-disk # This GCE PD must already exist.
fsType: ext4
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: claim-test
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 10Gi
After applaying this manifest files, we can check status of PV and PVC:
root#km:~# kubectl get pv,pvc
NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE
persistentvolume/pv-test 10Gi RWO Retain Bound default/claim-test 12m
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
persistentvolumeclaim/claim-test Bound pv-test 10Gi RWO 12m
Finally I used above claim as volume:
apiVersion: v1
kind: Pod
metadata:
name: web
spec:
containers:
- name: web
image: nginx
volumeMounts:
- mountPath: "/usr/share/nginx"
name: vol-test
volumes:
- name: vol-test
persistentVolumeClaim:
claimName: claim-test
We can inspect created Pod to check if it works as expected:
root#km:~# kubectl exec -it web -- bash
root#web:/# df -h
Filesystem Size Used Avail Use% Mounted on
...
/dev/sdb 9.8G 37M 9.8G 1% /usr/share/nginx
...
Dynamic Provisioning
In this case volume is provisioned automatically when application requires it.
First you need to create StorageClass object to define a provisioner such e.g. kubernetes.io/gce-pd.
We don't need to create PersistenVolume anymore, it's created automatically by StorageClass for us.
I've also created simple example for you to illustrate how it works.
First I created StorageClass as default storage class:
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
annotations:
storageclass.kubernetes.io/is-default-class: "true"
name: standard
provisioner: kubernetes.io/gce-pd
parameters:
type: pd-standard
fstype: ext4
And then PVC (the same as in the previous example) - but in this case PV was created automatically:
root#km:~# kubectl get pv,pvc
NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE
persistentvolume/pvc-8dcd69f1-7081-45a7-8424-cc02e61a4976 10Gi RWO Delete Bound default/claim-test standard 3m10s
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
persistentvolumeclaim/claim-test Bound pvc-8dcd69f1-7081-45a7-8424-cc02e61a4976 10Gi RWO standard 3m12s
In more advanced cases it may be useful to create multiple StorageClasses with differnet persistent disks types.

how to find my persistent volume location

I tried creating persistent volume using the host path. I can bind it to a specific node using node affinity but I didn't provide that. My persistent volume YAML looks like this
apiVersion: v1
kind: PersistentVolume
metadata:
name: jenkins
labels:
type: fast
spec:
capacity:
storage: 1Gi
accessModes:
- ReadWriteOnce
persistentVolumeReclaimPolicy: Recycle
hostPath:
path: /mnt/data
After this I created PVC
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: myclaim
spec:
accessModes:
- ReadWriteOnce
volumeMode: Filesystem
resources:
requests:
storage: 1Gi
And finally attached it onto the pod.
apiVersion: v1
kind: Pod
metadata:
name: mypod
spec:
containers:
- name: myfrontend
image: thinkingmonster/nettools
volumeMounts:
- mountPath: "/var/www/html"
name: mypd
volumes:
- name: mypd
persistentVolumeClaim:
claimName: myclaim
Now in describe command for pv or pvc it does not tell that on which node it has actually kept the volume /mnt/data
and I had to ssh to all nodes to locate the same.
And pod is smart enough to be created on that node only where Kubernetes had mapped host directory to PV
How can I know that on which node Kubernetes has created Persistent volume? Without the requirement to ssh the nodes or check that where is pod running.
It's only when a volume is bound to a claim that it's associated with a particular node. HostPath volumes are a bit different than the regular sort, making it a little less clear. When you get the volume claim, the annotations on it should give you a bunch of information, including what you're looking for. In particular, look for the:
volume.kubernetes.io/selected-node: ${NODE_NAME}
annotation on the PVC. You can see the annotations, along with the other computed configuration, by asking the Kubernetes api server for that info:
kubectl get pvc -o yaml -n ${NAMESPACE} ${PVC_NAME}

pod has unbound immediate persistentvolumeclaims after deleting namespace

I have configured the Postgres pod with static provisioning of persistence volume in my local environment . It works fine at the first time but when i delete the namespace and
rerun the pod then its status is pending and give me error
pod has unbound immediate persistentvolumeclaims
I tried to remove the storageClassName from Persistance Volume claim but not works
I also tried to change the storeageclass from manual to block storage but same problem
my yaml file
apiVersion: v1
kind: PersistentVolume
metadata:
name: task-pv-volume
namespace: manhattan
labels:
type: local
spec:
storageClassName: manual
capacity:
storage: 10Gi
accessModes:
- ReadWriteOnce
hostPath:
path: "/opt/manhattan/current/pgdata"
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: task-pv-claim
namespace: manhattan
spec:
storageClassName: manual
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 1Gi
---
apiVersion: v1
kind: Pod
metadata:
name: postgres
namespace: manhattan
spec:
volumes:
- name: task-pv-storage
persistentVolumeClaim:
claimName: task-pv-claim
containers:
- name: dbr-postgres
image: postgres-custome
tty: true
volumeMounts:
- mountPath: "/var/lib/pgsql/9.3/data"
name: task-pv-storage
nodeSelector:
kubernetes.io/hostname: k8s-master
I want my pod to be running even when i delete the namespace and rerun the pod.yaml file
Data will be kept in the kubernetes node because hostpath uses the node filesystem to store the data. The problem is that if you have multiple nodes, then your pod can start on any other node. To solve this, you can either specify the node where you want your pod to start or implement a nfs or glusterfs in your kubernetes nodes. This might be the cause of your problem.
There is one more thing I can think of that might be your issue. When you remove a namespace all the kubernetes resources inside it are removed as well. There is no easy way to recover those. This means that you have to create the pv, pvc and pod in the new namespace.
I solved this issue by using persistentVolumeReclaimPolicy to recycle. Now I can rebound the persistence volume even after deleting the namespace and recreating it
apiVersion: v1
kind: PersistentVolume
metadata:
name: task-pv-volume
labels:
type: local
spec:
storageClassName: manual
capacity:
storage: 10Gi
accessModes:
- ReadWriteOnce
persistentVolumeReclaimPolicy: Recycle
hostPath:
path: "/opt/manhattan/current/pgdata"

Persistent volume isn't matched with a claim

I created a simple local storage volume. Something like this:
apiVersion: v1
kind: PersistentVolume
metadata:
name: vol1
spec:
capacity:
storage: 1Gi
accessModes:
- ReadWriteOnce
persistentVolumeReclaimPolicy: Retain
storageClassName: local-storage
local:
path: /srv/volumes/vol1
nodeAffinity:
required:
nodeSelectorTerms:
- matchExpressions:
- key: kubernetes.io/hostname
operator: In
values:
- my-node
The I create a claim:
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
name: myclaim
spec:
accessModes:
- ReadWriteOnce
volumeMode: Filesystem
resources:
requests:
storage:1Gi
For unknow reason they don't get matches. What am I doing wrong?
About local storage it is worth to note that:
Using local storage ties your application to that specific node,
making your application harder to schedule. If that node or local
volume encounters a failure and becomes inaccessible, then that pod
also becomes inaccessible. In addition, many cloud providers do not
provide extensive data durability guarantees for local storage, so you
could lose all your data in certain scenarios.
This is for Kubernetes 1.10. In Kubernetes 1.14 local persistent volumes became GA.
You posted an answer that user is required. Just to clarify the user you meant is a consumer like a pod, deployment, statefullset etc.
So using just a simple pod definition would make your PV to become bound:
kind: Pod
apiVersion: v1
metadata:
name: mypod
spec:
containers:
- name: myfrontend
image: nginx
volumeMounts:
- mountPath: "/var/www/html"
name: mypd
volumes:
- name: mypd
persistentVolumeClaim:
claimName: myclaim
Now the problem happens when you would delete the pod and try to run another one. In this case if you or someone else wold look for a solution it has been described in this GitHub issue.
Hope this clears things out.
You should specify volumeName in your PVC to bind that specifically to the PV that you just created as so:
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
name: myclaim
spec:
accessModes:
- ReadWriteOnce
volumeName: "vol1"
resources:
requests:
storage:1Gi
Additionally, if you specify storageClassName in your PVC, your PVC will also get bound to a PV matching that specification (though it doesn't guarantee that it will be bound to your "vol1" PV if there are more than 1 PVs for that storage class).
Hope this helps!
I figured it out. I just needed a user. As long as I had a user, everything worked perfectly.

Kubernetes: PersistentVolume And PersistentVolumeClaim - Sharing Claims

This question is about the behavior of PersistentVolume and PersistentVolumeClaim configurations within Kubernetes. We have read through the documentation and are left with a few lingering questions.
We are using Azure Kubernetes Service to host our cluster and we want to provide a shared persistent storage backend for many of our Pods. We are planning on using PersistentVolumes to accomplish this.
In this scenario, we want to issue a PersistentVolume backed by an AzureFile storage resource. We will deploy Jenkins to our cluster and store the jenkins_home directory in the PersistentVolume so that our instance can survive pod and node failures. We will be running multiple Master Jenkins nodes, all configured with a similar deployment yaml.
We have created all the needed storage accounts and applicable shares ahead of time, as well as the needed secrets.
First, we issued the following PersistentVolume configuration;
apiVersion: v1
kind: PersistentVolume
metadata:
name: jenkins-azure-file-share
labels:
usage: jenkins-azure-file-share
spec:
capacity:
storage: 100Gi
accessModes:
- ReadWriteMany
persistentVolumeReclaimPolicy: Retain
azureFile:
secretName: azure-file-secret
shareName: jenkins
readOnly: false
mountOptions:
- dir_mode=0777
- file_mode=0777
- uid=1000
- gid=1000
Following that, we then issued the following PersistentVolumeClaim configuration;
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: jenkins-file-claim
annotations:
volume.beta.kubernetes.io/storage-class: ""
spec:
accessModes:
- ReadWriteMany
resources:
requests:
storage: 10Gi
volumeName: "jenkins-azure-file-share"
Next, we use this claim within our deployments in the following manner;
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: jenkins-instance-name
spec:
replicas: 1
template:
metadata:
labels:
role: jenkins
app: jenkins-instance-name
spec:
containers:
- name: jenkins-instance-name
image: ContainerRegistry.azurecr.io/linux/jenkins_master:latest
ports:
- name: jenkins-port
containerPort: 8080
volumeMounts:
- name: jenkins-home
mountPath: /var/jenkins_home
subPath: "jenkins-instance-name"
volumes:
- name: jenkins-home
persistentVolumeClaim:
claimName: "jenkins-file-claim"
imagePullSecrets:
- name: ImagePullSecret
This is all working as expected. We have deployed multiple Jenkins Masters to our Kubernetes cluster and each one is correctly allocating a new folder on the share specific to each master instance.
Now for my questions
The PersistentVolume is configured with 100Gig of Storage. Does this mean that Kubernetes will only allow a maximum of 100Gig of total storage in this volume?
When the PersistentVolumeClaim is bound to the PersistentVolume, the PersistentVolumeClaim seems to show that it has 100Gig of total storage available, even though the PersistentVolumeClaim was configured for 10Gig of storage;
C:\ashley\scm\kubernetes>kubectl get pv
NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE
jenkins-azure-file-share 100Gi RWX Retain Bound default/jenkins-file-claim 2d
C:\ashley\scm\kubernetes>kubectl get pvc
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
jenkins-homes-file-claim Bound jenkins-azure-file-share 100Gi RWX 2d
Is this just bad output from the get pvc command or am I misinterpreting the output of the get pvc command?
When sharing a PersistentVolumeClaim in this way;
Does each deployment ONLY have access to the configured maximum of 10Gig of storage from the PersistentVolume's 100Gig capacity?
Or, does each deployment have access to its own 10Gig slice of the total 100Gig of storage configured for the PersistentVolume?
With this configuration, what happens when a single PersistentVolumeClaim capacity gets fully utilized? Do all the Deployments using this single PersistentVolumeClaim stop working?
So for the pvc it is definitely the case that it has only 10Gig available with this config. For the pv I assume it is the same but in this case I don't know for sure but should be, because of consistency. And it stops working if any of this limits are reached so if you have 11 Jenkins running it will even fail although you not reached the limit on a single pvc.