I am using Kubernetes yaml to mount a volume.
I know I can set the mount folder to be for a specific group using this configuration:
securityContext:
fsGroup: 999
but no where I can find a way to also set user ownership and not just the group.
When I access the container folder to check ownership, it is root.
Anyway to do so via kubernetes Yaml?
I would expect fsUser: 999 for example, but there is no such thing. :/
There is no way to set the UID using the definition of Pod but you can use an initContainer with the same volumeMount as the main container to set the required permissions.
It is handy in cases like yours where user ownership needs to be set to a non root value.
Here is a sample configuration (change it as per your need):
apiVersion: v1
kind: Pod
metadata:
name: security-context-demo
spec:
volumes:
- name: sec-ctx-vol
emptyDir: {}
containers:
- name: sec-ctx-demo
image: busybox
command: [ "sh", "-c", "sleep 1h" ]
volumeMounts:
- name: sec-ctx-vol
mountPath: /data/demo
securityContext:
allowPrivilegeEscalation: false
initContainers:
- name: volume-mount-hack
image: busybox
command: ["sh", "-c", "chown -R 999:999 /data/demo"]
volumeMounts:
- name: sec-ctx-vol
mountPath: /data/demo
No, there is no such option. To check every option, available in securityContext, you may use
kubectl explain deployment.spec.template.spec.securityContext
As per doc
fsGroup <integer>
A special supplemental group that applies to all containers in a pod. Some
volume types allow the Kubelet to change the ownership of that volume to be
owned by the pod: 1. The owning GID will be the FSGroup 2. The setgid bit
is set (new files created in the volume will be owned by FSGroup) 3. The
permission bits are OR'd with rw-rw---- If unset, the Kubelet will not
modify the ownership and permissions of any volume.
It's usually a good idea to handle access to files via group ownership, because in restricted kubernetes configurations you can't actually control user id or group id, for example in RedHat Openshift.
What you can do is to use runAsUser, if your kubernetes provider allows it
runAsUser <integer>
The UID to run the entrypoint of the container process. Defaults to user
specified in image metadata if unspecified. May also be set in
SecurityContext. If set in both SecurityContext and PodSecurityContext, the
value specified in SecurityContext takes precedence for that container.
You application will work with uid you want, and naturally will create and access files as you want. As noted earlier, it's usually not the best idea to do it that way, because it makes distribution of your applications harder on different platforms.
Related
I've been searching and every answer seems to be the same example (https://kubernetes.io/docs/tasks/access-application-cluster/communicate-containers-same-pod-shared-volume/). In a pod you can create an empty volume, then mount that into two containers and any content written in that mount will be seen on each container. While this is fine my use case is slightly different.
Container A
/opt/content
Container B
/data
Container A has an install of about 4G of data. What I would like to do is mount /opt/content into Container B at /content. This way the 4G of data is accessible to Container B at runtime and I don't have to copy content or specially build Container B.
My question, is this possible. If it is, what would be the proper pod syntax.
apiVersion: v1
kind: Pod
metadata:
name: two-containers
spec:
restartPolicy: Never
volumes:
- name: shared-data
emptyDir: {}
containers:
- name: nginx-container
image: nginx
volumeMounts:
- name: shared-data
mountPath: /opt/content
- name: debian-container
image: debian
volumeMounts:
- name: shared-data
mountPath: /content
From my research and testing the best I can tell is within a POD two containers can not see each others file system. The volume mount will allow each container to have a mount created in the pod to the specified path (as the example shows) and then any items written to it after that point, will be seen on both. This works great for logs and stuff.
In my context, this proves to not be possible and creating this mount, and then having Container A copy the 4G directory to the newly created mount is to time consuming to make this an option.
Best I can tell is the only way to do this is create a Persistent Volume or other similar and mount that in the Container B. This way Container A contents are stored in the Persistent Volume and it can be easily mounted when needed. The only issue with this is the Persistent Volume will have to be setup in every Kube cluster defined which is the pain point.
If any of this is wrong and I just didn't find the right document please correct me. I would love to be able to do this.
Your code example in your question should work. Both are using the same volume and you mount them under different locations in the container.
nginx-container will have the shared-data content in /opt/content and debian-container will have it in /content.
With mountPath you specify where the volume should be mounted in the container
When a container is started, first the container image (or more precisely the layers of an image) are mounted. Afterwards, your custom volumes are mounted, hiding any data from the image at and below the mount path. So sharing data from an image among several containers without copying them is not possible.
The typical solution is and stays to use an init container which downloads or copies the actual data into an ephemeral volume, which then is shared by one or more other containers (https://kubernetes.io/docs/tasks/configure-pod-container/configure-pod-initialization/#create-a-pod-that-has-an-init-container).
initContainers:
- name: init
image: <image-containing-the-data-based-on-some-basic-image>
command: ["sh", "-c", "cp -ar /opt/content/* /mnt/target/"]
volumeMounts:
- name: shared-data
mountPath: /mnt/target
What you actually would need, is a kind of container storage interface (CSI) driver which supports creating volumes from container images. I found two projects which would exactly do that, but none of them states to be ready for production.
https://github.com/kubernetes-csi/csi-driver-image-populator
https://github.com/warm-metal/csi-driver-image
I want to share my non-empty local directory with kind cluster.
Based on answer here: How to reference a local volume in Kind (kubernetes in docker)
I tried few variations of the following:
Kind Cluster yaml:
kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
nodes:
- role: control-plane
extraMounts:
- hostPath: /Users/xyz/documents/k8_automation/data/manual/
containerPath: /host_manual
extraPortMappings:
- containerPort: 30000
hostPort: 10000
Pod yaml:
apiVersion: v1
kind: Pod
metadata:
name: manual
spec:
serviceAccountName: manual-sa
containers:
- name: tools
image: tools:latest
imagePullPolicy: Never
command:
- bash
tty: true
volumeMounts:
- mountPath: /home/jenkins/agent/data
name: data
volumes:
- name: data
hostPath:
path: /host_manual
type: Directory
---
I see that the directory /home/jenkins/agent/data does exist when the pod gets created. However, the folder is empty.
kinds documentation here: https://kind.sigs.k8s.io/docs/user/configuration/#extra-mounts
It should be the case that whatever is in the local machine at hostpath (/Users/xyz/documents/k8_automation/data/manual/) in extraMounts in the cluster yaml be available to the node at containerPath (/host_manual), which then gets mounted at container volume mounthPath (/home/jenkins/agent/data).
I should add that even if I change the hostPath in the cluster yaml file to a non-existent folder, the empty "data" folder still gets mounted in the container, so I think it's the connection from my local to kind cluster that's the issue.
Why am I not getting the contents of /Users/xyz/documents/k8_automation/data/manual/ with it's many files also available at /home/jenkins/agent/data in the container?
How can I fix this?
Any alternatives if there is no fix?
Turns out these yaml configuration was just fine.
The reason the directory was not showing up in the container was related with docker settings. And because "kind is a tool for running local Kubernetes clusters using Docker container “nodes”", it matters.
It seems docker restricts resource sharing and allows only specific directories to be bind mounted into Docker containers by default. Once I added the specific directory I wanted to show up in the container to the list of directories under Preferences -> Resources -> File sharing, it worked!
I have a StatefulSet that starts a MYSQL cluster. The only downside at it for the moment is that for every replica I need to create a Persistent Volume and a Persistent Volume Claim with a select that matches label and podindex.
This means I cannot dynamically add replicas whithout manual interaction.
For this reason I'm searching for a soluction that gives me the option to have only 1 Volume and 1 Claim. And during the pod creation it knows his own pod name for the subPath during mount. (initContainer would be used to check and create the directories on the volume before the application container starts).
So I search a correct way for a code like:
volumeMounts:
- name: mysql-datadir
mountPath: /var/lib/mysql
subPath: "${PODNAME}/datadir"
You can get POD_NAME from the metadata ( the downward API ) by setting ENV var:
env:
- name: MY_POD_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
But, you you cannot use ENV vars in volumes declarations (as far as i know...). So, everything else could be reached via workarounds. One of the workarounds is described here
Is it possible to run commands on host from within a pod in kubernetes.
So for example, I have a pod running a python image which calculates the size of the os. But the command it uses runs inside the pod, not in the host. Is it possible to run the command on the host from pod.
Actually a command run inside a pod is run on the host. It's a container (Docker), not a virtual machine.
That means when you execute something in the pod like retrieving the size of your RAM it will usually return the RAM of the whole machine.
If you want to get the "size of the os" and you mean the hard drive with it, you need to mount the hard drive to count it.
If your actual problem is that you want to do something, which a normal container isn't allowed to, you can run a pod in privileged mode or configure whatever you exactly need.
You need to add a security context to your pod like this (taken from the docs):
apiVersion: v1
kind: Pod
metadata:
name: security-context-demo
spec:
securityContext:
runAsUser: 1000
fsGroup: 2000
volumes:
- name: sec-ctx-vol
emptyDir: {}
containers:
- name: sec-ctx-demo
image: gcr.io/google-samples/node-hello:1.0
volumeMounts:
- name: sec-ctx-vol
mountPath: /data/demo
securityContext:
privileged: true
Sources:
https://kubernetes.io/docs/tasks/configure-pod-container/security-context/
https://kubernetes.io/docs/concepts/policy/pod-security-policy/#privileged
Is there any way to set per-volume permissions/ownership in Kubernetes declaratively?
Usecase:
a pod is composed of two containers, running as two distinct users/groups, both of them non-root, and are unable to sudo
the containers mount a volume each, and need to create files in these volumes (e.g. both of them want to write logs)
We know that we can use fsGroup, however that is a pod-level declaration. So even if we pick fsGroup equal to user in first container, then we are going to have permission issues in the other one. (ref: Kubernetes: how to set VolumeMount user group and file permissions)
One solution is to use init-container to change permissions of mounted directories.
The init-container would need to mount both volumes (from both containers), and do the needed chown/chmod operations.
Drawbacks:
extra container that needs to be aware of other containers' specific (ie. uid/gid)
init container needs to run as root to perform chown
It can be done with adding one init container with root access.
initContainers:
- name: changeowner
image: busybox
command: ["sh", "-c", "chown -R 200:200 /<volume>"]
volumeMounts:
- name: <your volume>
mountPath: /<volume>