I want to add or copy files into persistent volume and then use it in container using volume mount ?any help
Once PVC/PV are created (https://kubernetes.io/docs/concepts/storage/persistent-volumes/), there are number of possible solutions.
For specific question, options 1 and 2 will suffice. Listing more for reference, however this list does not try to be complete.
Simplest and native, kubectl cp: https://kubernetes.io/docs/reference/generated/kubectl/kubectl-commands#cp
rsync - still quite simple, but also robust. Recommended for a task (both of below options were tested)
TO pod: https://itecnotes.com/server/rsync-files-to-a-kubernetes-pod/
FROM pod: https://cybercyber.org/using-rsync-to-copy-files-to-and-from-a-kubernetes-pod.html
tar, but incremental: https://www.freshleafmedia.co.uk/blog/incrementally-copying-rsyncing-files-from-a-kubernetes-pod
Tools for synchronisation, backup, etc
For example, https://github.com/backube/volsync
Related
What is the best approach to passing multiple configuration files into a POD?
Assume that we have a legacy application that we have to dockerize and run in a Kubernetes environment. This application requires more than 100 configuration files to be passed. What is the best solution to do that? Create hostPath volume and mount it to some directory containing config files on the host machine? Or maybe config maps allow passing everything as a single compressed file, and then extracting it in the pod volume?
Maybe helm allows somehow to iterate over some directory, and create automatically one big configMap that will act as a directory?
Any suggestions are welcomed
Create hostPath volume and mount it to some directory containing config files on the host machine
This should be avoided.
Accessing hostPaths may not always be allowed. Kubernetes may use PodSecurityPolicies (soon to be replaced by OPA/Gatekeeper/whatever admission controller you want ...), OpenShift has a similar SecurityContextConstraint objects, allowing to define policies for which user can do what. As a general rule: accessing hostPaths would be forbidden.
Besides, hostPaths devices are local to one of your node. You won't be able to schedule your Pod some place else, if there's any outage. Either you've set a nodeSelector restricting its deployment to a single node, and your application would be done as long as your node is. Or there's no placement rule, and your application may restart without its configuration.
Now you could say: "if I mount my volume from an NFS share of some sort, ...". Which is true. But then, you would probably be better using a PersistentVolumeClaim.
Create automatically one big configMap that will act as a directory
This could be an option. Although as noted by #larsks in comments to your post: beware that ConfigMaps are limited in terms of size. While manipulating large objects (frequent edit/updates) could grow your etcd database size.
If you really have ~100 files, ConfigMaps may not be the best choice here.
What next?
There's no one good answer, not knowing exactly what we're talking about.
If you want to allow editing those configurations without restarting containers, it would make sense to use some PersistentVolumeClaim.
If that's not needed, ConfigMaps could be helpful, if you can somewhat limit their volume, and stick with non-critical data. While Secrets could be used storing passwords or any sensitive configuration snippet.
Some emptyDir could also be used, assuming you can figure out a way to automate provisioning of those configurations during container startup (eg: git clone in some initContainer, and/or some shell script contextualizing your configuration based on some environment variables)
If there are files that are not expected to change over time, or whose lifecycle is closely related to that of the application version shipping in your container image: I would consider adding them to my Dockerfile. Maybe even add some startup script -- something you could easily call from an initContainer, generating whichever configuration you couldn't ship in the image.
Depending on what you're dealing with, you could combine PVC, emptyDirs, ConfigMaps, Secrets, git stored configurations, scripts, ...
I am using locust helm chart. I need to perform load testing by supplying a text files whose size is greater than 3MB. The configuration maps do not work such big files. I tried splitting my files into smaller chunks of less than 1MB. But I am still getting the error "requested entity too large: limit is 3145728" as I got before when the file was a single huge one. Is there any option available?
Put the file somewhere accessible via the network and curl it down to a shared emptyDir volume using an initContainer. You can also use a ROX volume that you've populated manually if your hosting environment/provider offers those.
You got this error due to a limitation for ConfigMap that is equal to 1Mb. According to the official ConfigMap documentation:
A ConfigMap is not designed to hold large chunks of data. The data
stored in a ConfigMap cannot exceed 1 MiB. If you need to store
settings that are larger than this limit, you may want to consider
mounting a volume or use a separate database or file service.
At this point I can suggest you check #coderanger answer - that's the only available options for you.
To read and check
1)Configure a Pod to Use a Volume for Storage
2)Great SO answer Make large static data files available to kubernetes pods
You may also place the file on a persistent volume and mount the volume to the pod. Also, you may also use a hostPath volume but your pod will then be scheduled to only that host.
You can use the PVC or use the Node path for storing the file.
PVC : https://cloud.google.com/kubernetes-engine/docs/concepts/persistent-volumes
If you are using the Host path method you have to make sure your pod each time get schedule on same Node or host.
For that you can also use : https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/
Or you can use the taints & toleration.
I have an audit pod, which has logic to generate a report file. Currently, this file is present in the pod itself. I have only one pod having only one replica.
I know, I can run kubectl cp to copy those files from my pod. This command has to be executed on the Kubernetes node itself, but the task is to copy the file from the pod itself due to many restrictions.
I cannot use a Persistent Volume due to restrictions. I checked the Kubernetes API, but couldn't find anything by which I can do a copy.
Is there another way to copy that file out of the pod?
This is a community wiki answer posted to sum up the whole scenario and for better visibility. Feel free to edit and expand on it.
Taking under consideration all the mentioned restrictions:
not supposed to use the Kubernetes volumes
no cloud storage
pod names not accessible to your user
no sidecar containers
the only workaround for your use case is the one you currently use:
the dynamic PV with the annotations."helm.sh/resource-policy": keep
use PVCs and explicitly mention the user to not to delete the
namespace
If any one has a better idea. Feel free to contribute.
I am migrating my previous deployment made with docker-compose to Kubernetes.
In my previous deployment, some containers do have some data made at build time in some paths and these paths are mounted in persistent volumes.
Therefore, as the Docker volume documentation states,the persistent volume (not a bind mount) will be pre-populated with the container directory content.
I'd like to achieve this behavior with Kubernetes and its persistent volumes, How can I do ? Do I need to add some kind of logic using scripts in order to copy my container's files to the mounted path when data is not present the first time the container starts ?
Possibly related question: Kubernetes mount volume on existing directory with files inside the container
I think your options are
ConfigMap (are "some data" configuration files?)
Init containers (as mentioned)
CSI Volume Cloning (clone combining an init or your first app container)
there used to be a gitRepo; deprecated in favour of init containers where you can clone your config and data from
HostPath volume mount is an option too
An NFS volume is probably a very reasonable option and similar from an approach point of view to your Docker Volumes
Storage type: NFS, iscsi, awsElasticBlockStore, gcePersistentDisk and others can be pre-populated. There are constraints. NFS probably the most flexible for sharing bits & bytes.
FYI
The subPath might be of interest too depending on your use case and
PodPreset might help in streamlining the op across the fleet of your pods
HTH
I have node on google kubernetes engine using persistent volume. Is possible edit files on this volume from gcloud, or google cloud shell? For example edit config and recreate node? Or it is only posiible from running pod using kubectl exec?
i think you can have a look to gsutil command it allows you to interact with your buckets .
Guide to Gsutil
The volume would be a block device, so I’d expect it’d not be possible to edit it outside of the pod it’s attached to. So yes, expecting into the pod would do it, but you could also just use kubectl cp to copy files (and directories!) directly from your local machine onto the volume, mounted to the pod.
Here’s the relevant doc:
https://kubernetes.io/docs/reference/generated/kubectl/kubectl-commands#cp