I am using kubeadm localy at two physical machines. I don't have any cloud resources, and i want to build a mongodb auto scaling (localy for start, maybe later at cloud). So i have to use the local storage of my two physical machines. I suppose i have to create a local storage class and volumes. I am very new to kubernetes so dont judge me hard. As i read here https://kubernetes.io/blog/2019/04/04/kubernetes-1.14-local-persistent-volumes-ga/ local persisent volumes are only for one node? Is there any way to take advance of my both physical machines storages and build a simple mongo db scaling, using kubernetes mongo operator and ops manager? I made a few tests here, but i could achieve my goal. pod has unbound immediate PersistentVolumeClaims ops manager
What i was thinking in first place, was to "break" my two hard drives into many piecies, and use sharding for mongo dv scaling
thanks in advace.
Well, you can use a NFS Server with the same volume mounted in both nodes sharing the same mount point.
Please be aware this approach is not recommended for production.
There are tons of howtos of how configure nfs server, example:
https://www.tecmint.com/install-nfs-server-on-ubuntu/
https://www.tecmint.com/how-to-setup-nfs-server-in-linux/
With NFS working you can use the hostPath to mount the nfs in you pods:
Create the PV and the PVC:
apiVersion: v1
kind: PersistentVolume
metadata:
name: pv-volume
labels:
type: local
spec:
storageClassName: manual
capacity:
storage: 10Gi
accessModes:
- ReadWriteOnce
hostPath:
path: "/nfs/data"
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: pv-claim
spec:
storageClassName: manual
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 3Gi
And use the volume in your deployment file:
apiVersion: apps/v1
kind: Deployment
metadata:
name: test-pv
spec:
replicas: 1
selector:
matchLabels:
app: test-pv
template:
metadata:
labels:
app: test-pv
spec:
containers:
- image: nginx
name: nginx
volumeMounts:
- mountPath: /data
name: pv-storage
volumes:
- name: pv-storage
persistentVolumeClaim:
claimName: pv-claim
Related
We are trying to create a solution where we want to replicate changes made to a folder inside any of the pods of the Statefulset. Any file changes inside that folder on any POD should also reflect in other pods. Is there a sidecar solution for this requirement? Because we know that Statefulset will create separate PVs for each POD and there won't be any common mount across the pods of the Statefulset.
You can try using the NFS or file system like EFS using that you will be able to implement the ReadWritemany.
For ref Azure File.
apiVersion: apps/v1beta1
kind: StatefulSet
metadata:
name: statefulset-azurefile
labels:
k8s-app: nginx
version: v1
spec:
serviceName: statefulset-azurefile
replicas: 1
template:
metadata:
labels:
k8s-app: nginx
version: v1
spec:
containers:
- name: statefulset-azurefile
image: nginx
volumeMounts:
- name: persistent-storage
mountPath: /mnt/azurefile
volumeClaimTemplates:
- metadata:
name: persistent-storage
annotations:
volume.beta.kubernetes.io/storage-class: azurefile
spec:
accessModes: [ "ReadWriteMany" ]
resources:
requests:
storage: 5Gi
Demo : https://github.com/andyzhangx/demo/tree/master/linux/statefulset
If volumeClaimTemplates not work as expected use the persistentVolumeClaim
Article to read about ReadWriteMany access Mode :https://learn.microsoft.com/en-us/azure/aks/azure-files-volume
If you are on other cloud providers like GCP, AWS, Oracle(OCI) provides different file services.
GCP- Filestore
AWS- EFS
OCI- Filestorage
OCI article if you want to explore : https://enabling-cloud.github.io/oci-learning/manual/StaticPersistentVolumeOnOCI.html
I create a deployment yaml for a microservice.
I am using hostpath volume type for persistentVolume and I have to copy data to a path in host. But I want to mount a directory from container into the host because data is in the container and I need this data in host.
My deployment yaml:
#create persistent volume
---
apiVersion: v1
kind: PersistentVolume
metadata:
name: pv-vol
spec:
storageClassName: manual
capacity:
storage: 10Gi
accessModes:
- ReadWriteOnce
hostPath:
path: /opt/storage/app
#create persistent volume clame
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: app-pv-claim
spec:
storageClassName: manual
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 10Gi
#create Deployment
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: app-deployment
spec:
replicas: 1
selector:
matchLabels:
deploy: app
template:
metadata:
labels:
deploy: app
spec:
hostname: app
hostNetwork: false
containers:
- name: app
image: 192.168.10.10:2021/project/app:latest
volumeMounts:
- mountPath: /opt/app
name: project-volume
volumes:
- name: project-volume
persistentVolumeClaim:
claimName: app-pv-claim
Due to information gaps, I am writing a general answer.
First of all you should know:
HostPath volumes present many security risks, and it is a best practice to avoid the use of HostPaths when possible. When a HostPath volume must be used, it should be scoped to only the required file or directory, and mounted as ReadOnly.
But the use of hostPath also offers a powerful escape hatch for some applications.
If you still want to use it, firstly you should check if both pods (the one that created the data and the second one that want to access the data) are on the same node. The following command will show you that.
kubectl get pods -o wide
All data created by any of pods should stay in hostPath directory and be available for every pod as long as they are running on the same node.
See also this documentation about hostPath.
I have Java API which exports the data to an excel and generates a file on the POD where the request is served.
Now the next request (to download the file) might go to a different POD and the download fails.
How do I get around this?
How do I generate files on all the POD? Or how do I make sure the subsequent request goes to the same POD where file was generated?
I cant give the direct POD URL as it will not be accessible to clients.
Thanks.
You need to use a persistent volumes to share the same files between your containers. You could use the node storage mounted on containers (easiest way) or other distributed file system like NFS, EFS (AWS), GlusterFS etc...
If you you need a simplest to share the file and your pods are in the same node, you could use hostpath to store the file and share the volume with other containers.
Assuming you have a kubernetes cluster that has only one Node, and you want to share the path /mtn/data of your node with your pods:
Create a PersistentVolume:
A hostPath PersistentVolume uses a file or directory on the Node to emulate network-attached storage.
apiVersion: v1
kind: PersistentVolume
metadata:
name: task-pv-volume
labels:
type: local
spec:
storageClassName: manual
capacity:
storage: 10Gi
accessModes:
- ReadWriteOnce
hostPath:
path: "/mnt/data"
Create a PersistentVolumeClaim:
Pods use PersistentVolumeClaims to request physical storage
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: task-pv-claim
spec:
storageClassName: manual
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 3Gi
Look at the PersistentVolumeClaim:
kubectl get pvc task-pv-claim
The output shows that the PersistentVolumeClaim is bound to your PersistentVolume, task-pv-volume.
NAME STATUS VOLUME CAPACITY ACCESSMODES STORAGECLASS AGE
task-pv-claim Bound task-pv-volume 10Gi RWO manual 30s
Create a deployment with 2 replicas for example:
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx
spec:
replicas: 2
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
volumes:
- name: task-pv-storage
persistentVolumeClaim:
claimName: task-pv-claim
containers:
- name: task-pv-container
image: nginx
ports:
- containerPort: 80
name: "http-server"
volumeMounts:
- mountPath: "/mnt/data"
name: task-pv-storage
Now you can check inside both container the path /mnt/data has the same files.
If you have cluster with more than 1 node I recommend you to think about the other types of persistent volumes.
References:
Configure persistent volumes
Persistent volumes
Volume Types
I am working on an application on Kubernetes in GCP and I need a really huge SSD storage for it.
So I created a StorageClass recourse, a PersistentVolumeClaim that requests 500Gi of space and then a Deployment recourse.
StorageClass.yaml:
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: faster
provisioner: kubernetes.io/gce-pd
parameters:
type: pd-ssd
PVC.yaml:
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: mongo-volume
spec:
storageClassName: faster
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 500Gi
Deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: mongo-deployment
spec:
replicas: 2
selector:
matchLabels:
app: mongo
template:
metadata:
creationTimestamp: null
labels:
app: mongo
spec:
containers:
- image: mongo
name: mongo
ports:
- containerPort: 27017
volumeMounts:
- mountPath: /data/db
name: mongo-volume
volumes:
- name: mongo-volume
persistentVolumeClaim:
claimName: mongo-volume
When I applied the PVC, it stuck in Pending... state for hours. I found out experimentally that it binds correctly with maximum 200Gi of requested storage space.
However, I can create several 200Gi PVCs. Is there a way to bind them to one path to work as one big PVC in Deployment.yaml? Or maybe the 200Gi limit can be expanded?
I have just tested it on my own env and it works perfectly. So the problem is in Quotas.
For this check:
IAM & admin -> Quotas -> Compute Engine API Local SSD (GB) "your region"
Amount which you used.
I've created the situation when I`m run out of Quota and it stack in pending status the same as your.
It happens because you create PVC for each pod for 500GB each.
I have a 3-node Kubernetes cluster running on vagrant using the oracle Kubernetes vagrant boxes from http://github.com/oracle/vagrant-boxes.git.
I want to add a pod including an Oracle database and persist the data so that in case all nodes go down, I don't lose my data.
According to how I read the Kubernetes documentation persistent volumes cannot be created on a local filesystem only on a cloud-backed device. I want to configure the persistent volume and persistent volume claim on my vagrant boxes as a proof of concept and training exercise for my Kubernetes learning.
Are there any examples of how I might go about creating the PV and PVC in this configuration?
As a complete Kubernetes newbie, any code samples would be greatly appreciated.
Use host path:
create PV:
kind: PersistentVolume
apiVersion: v1
metadata:
name: task-pv-volume
labels:
type: local
spec:
storageClassName: manual
capacity:
storage: 10Gi
accessModes:
- ReadWriteOnce
hostPath:
path: "/mnt/data
create PVC:
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
name: task-pv-claim
spec:
storageClassName: manual
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 3Gi
Use it in a pod:
kind: Pod
apiVersion: v1
metadata:
name: task-pv-pod
spec:
volumes:
- name: task-pv-storage
persistentVolumeClaim:
claimName: task-pv-claim
containers:
- name: task-pv-container
image: nginx
ports:
- containerPort: 80
name: "http-server"
volumeMounts:
- mountPath: "/usr/share/nginx/html"
name: task-pv-storage
documentation
This is just an example, for testing only.
For production use case, you will need dynamic provisioning using the StorageClass for PVC, so that the volume/data is available when the pod moves across the cluster.