I am using EKS with Kubernetes version 1.15 and when I create a Storageclass, Persistent-Volume, Persistent-Volume-Claim, and Deployment the pod fails with:
Warning FailedAttachVolume 71s (x2 over 3m11s) attachdetach-controller AttachVolume.Attach failed for volume "efs-pv" : attachment timeout for volume fs-<volume>
Warning FailedMount 53s (x2 over 3m8s) kubelet, ip-<ip-address>.ec2.internal Unable to mount volumes for pod "influxdb-deployment-555f4c8b94-mldfs_default(2525d10b-e30b-4c4c-893e-10971e0c683e)": timeout expired waiting for volumes to attach or mount for pod "default"/"influxdb-deployment-555f4c8b94-mldfs". list of unmounted volumes=[persistent-storage]. list of unattached volumes=[persistent-storage]
However when I try the same without building the Persistent-Volume it is successful, and creates its own that seemingly skips CSI. This is what I am working with:
deployment.yaml:
apiVersion: apps/v1
kind: Deployment
metadata:
name: influxdb-deployment
spec:
selector:
matchLabels:
app: influxdb
replicas: 1
template:
metadata:
labels:
app: influxdb
spec:
containers:
- name: influxdb
image: influxdb:1.7.10-alpine
ports:
- containerPort: 8086
volumeMounts:
- name: persistent-storage
mountPath: /var/lib/influx
volumes:
- name: persistent-storage
persistentVolumeClaim:
claimName: efs-claim
storageclass.yaml:
kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
name: efs-sc
provisioner: efs.csi.aws.com
reclaimPolicy: Retain
persistent-volume.yaml:
apiVersion: v1
kind: PersistentVolume
metadata:
name: efs-pv
spec:
capacity:
storage: 5Gi
volumeMode: Filesystem
accessModes:
- ReadWriteMany
persistentVolumeReclaimPolicy: Retain
storageClassName: efs-sc
csi:
driver: efs.csi.aws.com
volumeHandle: fs-<volume-id>
persistent-volume-claim.yaml:
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: efs-claim
spec:
accessModes:
- ReadWriteMany
storageClassName: efs-sc
resources:
requests:
storage: 5Gi
Any idea on what is happening?
This is something unclarified in aws EKS documents, but a must-do when you are going to use storageclass as first time setting.
EKS has no default storage class to be set up
I didn't work on EFS with EKS before, but does set with gp2 (EBS) with below yaml file successfully.
Paste here for your reference
kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
name: gp2
annotations:
storageclass.kubernetes.io/is-default-class: "true"
provisioner: kubernetes.io/aws-ebs
parameters:
type: gp2
fsType: ext4
Reference link
https://docs.aws.amazon.com/eks/latest/userguide/storage-classes.html
Related
I try to mount a linux directory as a shared directory for multiple containers in minikube.
Here is my config:
minikube start --insecure-registry="myregistry.com:5000" --mount --mount-string="/tmp/myapp/k8s/:/data/myapp/share/"
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: manual
provisioner: kubernetes.io/no-provisioner
volumeBindingMode: WaitForFirstConsumer
---
apiVersion: v1
kind: PersistentVolume
metadata:
name: myapp-share-storage
spec:
storageClassName: manual
capacity:
storage: 10Gi
accessModes:
- ReadWriteMany
local:
path: "/data/myapp/share/"
nodeAffinity:
required:
nodeSelectorTerms:
- matchExpressions:
- key: kubernetes.io/hostname
operator: In
values:
- minikube
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: myapp-share-claim
spec:
storageClassName: manual
accessModes:
- ReadWriteMany
resources:
requests:
storage: 10Gi
---
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
io.kompose.service: myapp-server
name: myapp-server
spec:
selector:
matchLabels:
io.kompose.service: myapp-server
template:
metadata:
labels:
io.kompose.service: myapp-server
spec:
containers:
- name: myapp-server
image: myregistry.com:5000/server-myapp:alpine
ports:
- containerPort: 80
resources: {}
volumeMounts:
- mountPath: /data/myapp/share
name: myapp-share
env:
- name: storage__root_directory
value: /data/myapp/share
volumes:
- name: myapp-share
persistentVolumeClaim:
claimName: myapp-share-claim
status: {}
It works with pitfalls: Statefulset are not supported, they bring deadlock errors :
pending PVC: waiting for first consumer to be created before binding
pending POD: 0/1 nodes are available: 1 node(s) didn't find available persistent volumes to bind
Another option is to use minikube persistentvolumeclaim without persistentvolume (it will be created automatically). However:
The volume is created in /tmp (ex: /tmp/hostpath-provisioner/default/myapp-share-claim)
Minikube doesn't honor mount request
How can I make it just work?
Using your yaml file I've managed to create the volumes and deploy it without issue, but i had to use the command minikube mount /mydir/:/data/myapp/share/ after starting the minikube since --mount --mount-strings="/mydir/:/data/myapp/share/" wasn't working.
EDIT: SEE BELOW
I am new trying to build a local cluster with 2 physical machines with kubeadm. I am following this https://github.com/mongodb/mongodb-enterprise-kubernetes steps and everything is ok. At first i am installing kubernetes operator, but when i tried to install ops manager i am geting:
0/2 nodes are available: 2 pod has unbound immediate PersistentVolumeClaims ops manager.
the yaml i used to install ops manager is:
---
apiVersion: mongodb.com/v1
kind: MongoDBOpsManager
metadata:
name: opsmanager1
spec:
replicas: 2
version: 4.2.0
adminCredentials: mongo-db-admin1 # Should match metadata.name
# in the Kubernetes secret
# for the admin user
externalConnectivity:
type: NodePort
applicationDatabase:
members: 3
version: 4.4.0
persistent: true
podSpec:
persistence:
single:
storage: 1Gi
i can't figure out what the problem is. I am at a testing phase, and my goal is to make a scaling mongo database. Thanks in advance
edit: i made a few changes.I created storage class like this:
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: localstorage
provisioner: kubernetes.io/no-provisioner
volumeBindingMode: Immediate
reclaimPolicy: Delete
allowVolumeExpansion: True
---
kind: PersistentVolume
apiVersion: v1
metadata:
name: mongo-01
labels:
type: local
spec:
storageClassName: localstorage
capacity:
storage: 2Gi
accessModes:
- ReadWriteOnce
hostPath:
path: "/home/master/mongo01"
---
kind: PersistentVolume
apiVersion: v1
metadata:
name: mongo-02
labels:
type: local
spec:
storageClassName: localstorage
capacity:
storage: 2Gi
accessModes:
- ReadWriteOnce
hostPath:
path: "/home/master/mongo02"
And now my yaml for ops manger is:
apiVersion: mongodb.com/v1
kind: MongoDBOpsManager
metadata:
name: ops-manager-localmode
spec:
replicas: 2
version: 4.2.12
adminCredentials: mongo-db-admin1
externalConnectivity:
type: NodePort
statefulSet:
spec:
# the Persistent Volume Claim will be created for each Ops Manager Pod
volumeClaimTemplates:
- metadata:
name: mongodb-versions
spec:
storageClassName: localstorage
accessModes: [ "ReadWriteOnce" ]
resources:
requests:
storage: 2Gi
template:
spec:
containers:
- name: mongodb-ops-manager
volumeMounts:
- name: mongodb-versions
# this is the directory in each Pod where all MongoDB
# archives must be put
mountPath: /mongodb-ops-manager/mongodb-releases
backup:
enabled: false
applicationDatabase:
members: 3
version: 4.4.0
persistent: true
But i get a new error : Warning ProvisioningFailed 44s (x26 over 6m53s) persistentvolume-controller no volume plugin matched name: kubernetes.io/no-provisioner
At a quick glance, it looks like you don't have any volume that can create a PVC on your cluster. see https://v1-15.docs.kubernetes.io/docs/concepts/storage/volumes/
Your app needs to create a persistant volume, but your cluster doesn't know how to do that.
I have two VM Instance on GCE with kubernetes self install (using the following https://medium.com/edureka/install-kubernetes-on-ubuntu-5cd1f770c9e4).
I'm trying to create volume and use it in my pods.
I have been created the following disk:
gcloud compute disks create --type=pd-ssd --size=10GB manual-disk-1
And create the following yaml files
pv_manual.yaml:
apiVersion: v1
kind: PersistentVolume
metadata:
name: manually-created-pv
spec:
accessModes:
- ReadWriteMany
capacity:
storage: 10Gi
persistentVolumeReclaimPolicy: Retain
gcePersistentDisk:
pdName: manual-disk-1
pvc_manual.yaml:
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: mypvc
spec:
accessModes:
- ReadWriteMany
resources:
requests:
storage: 10Gi
pod.yaml:
apiVersion: v1
kind: Pod
metadata:
name: sleppypod
spec:
volumes:
- name: data
persistentVolumeClaim:
claimName: mypvc
containers:
- name: sleppycontainer
image: gcr.op/google_containers/busybox
command:
- sleep
- "5000"
volumeMounts:
- name: data
mountPath: /data
readOnly: false
And when I'm trying to create the pod the pode get status ContainerCreating and on kubectl get events I see:
7s Warning FailedAttachVolume AttachVolume.NewAttacher failed for volume : Failed to get GCE GCECloudProvider with error
I run my two instances using ServiceAccount with compute instance admin role (according Kubernetes: Failed to get GCE GCECloudProvider with error <nil>) and my kubelet running with --cloud-provider=gce
How can I solve it?
You need to create a storageclass
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
annotations:
storageclass.kubernetes.io/is-default-class: "true"
name: standard
provisioner: kubernetes.io/gce-pd
parameters:
type: pd-standard
fstype: ext4
replication-type: none
For GCE details here
You can also follow GCE documentation here
Is it required to create the directory manually in nodes or will it be auto created by pv?
Here is my pv & pvc file, and I'm seeing this error
no persistent volumes available for this claim and no storage class is set
how to resolve this?
kind: PersistentVolume
apiVersion: v1
metadata:
name: zk1-pv
labels:
type: local
spec:
storageClassName: manual
capacity:
storage: 10Mi
accessModes:
- ReadWriteOnce
hostPath:
path: "/mr/zk"
cat zk1-pvc.yaml
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: zk1-pvc
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 10Mi
kubectl describe pvc zk1-pvc
Name: zk1-pvc
Namespace: instavote
StorageClass:
Status: Pending
Volume:
Labels: <none>
Annotations: kubectl.kubernetes.io/last-applied-configuration:
{"apiVersion":"v1","kind":"PersistentVolumeClaim","metadata":{"annotations":{},"name":"zk1-pvc","namespace":"instavote"},"spec":{"accessMo...
Finalizers: [kubernetes.io/pvc-protection]
Capacity:
Access Modes:
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal FailedBinding 12s (x14 over 3m7s) *persistentvolume-controller no persistent volumes available for this claim and no storage class is set*
Mounted By: zk1-745b7cbf46-lg7v9
Back to your main question
Is it required to create the directory manually in nodes or will it be
auto created by pv?
First of all, error in your output is not related with your question. As an answer for your question - Yes. It is crated by PV automatically.
In order to do achieve this, first you have to create StorageClass with no-provisioner as an example below
kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
name: manual
provisioner: kubernetes.io/no-provisioner
volumeBindingMode: WaitForFirstConsumer
Then you have to create PersistentVolume by defining this storageClassName and hostPath parameter like below:
apiVersion: v1
kind: PersistentVolume
metadata:
name: zk1-pv
spec:
storageClassName: manual
capacity:
storage: 1Gi
accessModes:
- ReadWriteOnce
hostPath:
path: /mr/zk
Then you have to create PVC and Pod/Deployment as an example below:
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: myclaim
spec:
storageClassName: manual
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 1Gi
---
apiVersion: v1
kind: Pod
metadata:
name: mypod
spec:
containers:
- name: containerName
image: gcr.io/google-containers/nginx:1.7.9
volumeMounts:
- mountPath: "/var/www/html"
name: mypd
volumes:
- name: mypd
persistentVolumeClaim:
claimName: myclaim
NOTE:
Don't forget put storageClassName: manual parameter on both PVC and PV manifests. Otherwise they will not be able to bound to each other.
Hope it clears
You forgot to specify storageClassName: manual in PersistentVolumeClaim.
We have the setup in GKE with two different clusters. One cluster is running a nfs-server and on that cluster we have a persistent-volume which points to the server. This PV is then mounted on a pod running on this cluster. The second cluster also has a PV and a pod that should mount the same nfs volume. Here is where the problem occurs. Where we point out the server it does not work with using the nfs-server clusterIp address. This is understandable but I wonder how to best achieve this.
The setup is basically this:
Persistent Volume and Persistent Volume Claim used by NFS
apiVersion: v1
kind: PersistentVolume
metadata:
name: nfs-pv
spec:
capacity:
storage: 20Gi
storageClassName: manual
accessModes:
- ReadWriteMany
gcePersistentDisk:
pdName: files
fsType: ext4
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: nfs-pvc
spec:
storageClassName: manual
accessModes:
- ReadWriteMany
resources:
requests:
storage: 20Gi
NFS server deployment
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: nfs-server
spec:
replicas: 1
selector:
matchLabels:
role: nfs-server
template:
metadata:
labels:
role: nfs-server
spec:
containers:
- name: nfs-server
image: gcr.io/google_containers/volume-nfs:0.8
ports:
- name: nfs
containerPort: 2049
- name: mountd
containerPort: 20048
- name: rpcbind
containerPort: 111
securityContext:
privileged: true
volumeMounts:
- mountPath: /exports
name: mypvc
volumes:
- name: mypvc
persistentVolumeClaim:
claimName: nfs-pvc
NFS-server service
apiVersion: v1
kind: Service
metadata:
name: nfs-server
spec:
ports:
- name: nfs
port: 2049
- name: mountd
port: 20048
- name: rpcbind
port: 111
selector:
role: nfs-server
Persistent volume and Persistent Volume Claim used by the pods:
apiVersion: v1
kind: PersistentVolume
metadata:
name: nfs
spec:
capacity:
storage: 20Gi
storageClassName: manual
accessModes:
- ReadWriteMany
nfs:
server: 10.4.0.20
path: "/"
---
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
name: nfs
spec:
storageClassName: manual
accessModes:
- ReadWriteMany
resources:
requests:
storage: 20Gi
Part of deployment file for pod mounting the nfs
volumes:
- name: files
persistentVolumeClaim:
claimName: nfs
Output of kubectl get pv and kubectl get pvc
user#HP-EliteBook:~/Downloads$ kubectl get pv
NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE
nfs 100Gi RWX Retain Bound default/nfs manual 286d
nfs-pv 100Gi RWO Retain Bound default/nfs-pvc manual 286d
user#HP-EliteBook:~/Downloads$ kubectl get pvc
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
nfs Bound nfs 100Gi RWX manual 286d
nfs-pvc Bound nfs-pv 100Gi RWO manual 286d
The ip in the PV used by the pods is the problem. The pod on the same cluster can connect to it but not the pod on the other cluster. I can use the actual podIP from the other cluster but the podIP changes with every deploy so that is not a working solution. What is the best way to get around this problem, I only want this second cluster to have access to the nfs server and not opening it to the world for example.