I run Jenkins in k8s, and I have mount /var/jenkins_home folder with PVC already, now I want to mount /var/jenkins_home/config.xml as a configmap, other the folder still mount with pvc.
below is my yaml file:
volumeMounts:
- mountPath: /var/jenkins_home
name: jenkins-data
subPath: jenkins
- mountPath: /var/jenkins_home/config.xml
name: configxml
subPath: config.xml
volumes:
- name: jenkins-data
persistentVolumeClaim:
claimName: shdr-jenkins-k-test
- name: configxml
configMap:
name: jenkins-k-config
ites:
- key: jenkins-configfile
path: config.xml
when I open jenkins, it says:
Also: java.nio.file.FileSystemException: /var/jenkins_home/atomic14997153162086721303tmp -> /var/jenkins_home/config.xml: Device or resource busy
at java.base/sun.nio.fs.UnixException.translateToIOException(Unknown Source)
at java.base/sun.nio.fs.UnixException.rethrowAsIOException(Unknown Source)
at java.base/sun.nio.fs.UnixCopyFile.move(Unknown Source)
at java.base/sun.nio.fs.UnixFileSystemProvider.move(Unknown Source)
at java.base/java.nio.file.Files.move(Unknown Source)
at hudson.util.AtomicFileWriter.commit(AtomicFileWriter.java:194)
java.nio.file.FileSystemException: /var/jenkins_home/config.xml: Device or resource busy
Short Answer
You need to set the mountPath of jenkins-data volume to be /var/jenkins_home/jenkins. This will correctly configure the subPath functionality.
Detailed Explanation
If I understand correctly, what you are trying to achieve is the following:
You have a ConfigMap named jenkins-k-config. This contains a single parameter config.xml with contents of your Jenkins configuration as a value.
You want to mount this ConfigMap at the path /var/jenkins_home/ in your Jenkins pod, so the pod can use the /var/jenkins_home/config.xml file.
Here's how you can do this:
You will update your pod specification to add the ConfigMap as a volume.
You will then add a volumeMount to mount that the contents of that ConfigMap into the specified mount point in your pod's container.
Since your ConfigMap only contains a single key named config.xml, you don't have to specify the items anyway. Simply mounting the ConfigMap will work. See the manifest below:
volumeMounts:
- mountPath: /var/jenkins_home/
name: configxml
subPath: config.xml
readOnly: true. #<==== Recommended, so config remains immutable.
volumes:
- name:configxml
configMap:
name: jenkins-k-config
I also noticed that when all these mistakes have been fixed, we will end up with 2 volumes (jenkins-data, configxml) attempting to mount to the same mountPoint inside the pod. This is the reason you're seeing device or resource busy error, since the mountPoint is already busy, being mounted with the jenkins-data volume.
You can change the mount point to /var/jenkins_home/jenkins. This will also put your subPath variable in affect and you will be able to mount both volumes.
jenkins-data ====> /var/jenkins_home/jenkins and
config.xml ====> /var/jenkins_home/config.xml.
Related
I am mounting db secrets as a file in my Kubernetes container. Db secrets will get updated after the password expiry time. I am using polling mechanism to check if Db secrets has been reset to updated value. Is it possible to change mounted secret inside file.
is secret mounted as file is editable from application code in kubernetes
The file which gets loaded into the container will be loaded in readonly format, so loaded file can't be edited from inside the container. But secret can be edited from either updating the secret or copying the file into different location within the container.
I'm not sure how you did it. Putting the yaml format of pod configuration would help more.
for example if you use hostPath to mount a file inside the container, every time you change the source file, you see the changes inside the container.
for example
apiVersion: v1
kind: Pod
metadata:
name: test-pod
spec:
containers:
- image: busybox
name: test-container
command: ["/bin/sh", "-c", "sleep 36000"]
volumeMounts:
- mountPath: /etc/db_pass
name: password-volume
volumes:
- name: password-volume
hostPath:
path: /var/lib/original_password
type: File
I'm deploying a basic "solr:8.9.0" image to local Kubernetes env.
If I'm trying to mount pod's "/var/solr" directory, it works well.
I can see the files inside /var/solr in the mounted directory.
spec:
containers:
- image: solr:8.6.0
imagePullPolicy: IfNotPresent
name: solr
ports:
- name: solrport
containerPort: 8983
volumeMounts:
- mountPath: /var/solr/
name: solr-volume
volumes:
- name: solr-volume
persistentVolumeClaim:
claimName: solr-pvc
But somehow I can't mount "/etc/default/" directory. That doesn't work.
I knew there are files inside that directory but they are disappearing.
Any idea why?
Thanks!
this is because of how volumeMounts work.
A standard volumeMount mounts the volume in the suplied directory overwriting everything that is inside that directory.
You want to specify a subpath for the data you actually want to mount. By doing this the original contents of the directory won't get overridden.
see here for more information regarding the usage of subpaths.
I am trying to add files in volumeMounts to .dockerignore and trying to understand the difference between subPath and mountPath. Reading official documentation isn't clear to me.
I should add from what I read mountPath is the directory in the pod where volumes will be mounted.
from official documentation: "subPath The volumeMounts.subPath property specifies a sub-path inside the referenced volume instead of its root." https://kubernetes.io/docs/concepts/storage/volumes/#using-subpath (this part isn't clear)
- mountPath: /root/test.pem
name: test-private-key
subPath: test.testing.com.key
In this example should I include both test.pem and test.testing.com.key to dockerignore?
mountPath shows where the referenced volume should be mounted in the container. For instance, if you mount a volume to mountPath: /a/b/c, the volume will be available to the container under the directory /a/b/c.
Mounting a volume will make all of the volume available under mountPath. If you need to mount only part of the volume, such as a single file in a volume, you use subPath to specify the part that must be mounted. For instance, mountPath: /a/b/c, subPath: d will make whatever d is in the mounted volume under directory /a/b/c
The difference between mountPath & subPath is that subPath is an addition to mountPath and it exists to solve a problem.
Look at my comments inside the example Pod manifest, I explain the problem and how subPath solves it.
To further understand the difference look at the "under the hood" section to see how kubernetes treats these two properties.
apiVersion: v1
kind: Pod
metadata:
name: mypod
spec:
volumes:
- name: vol1
emptyDir: {}
containers:
- image: nginx
name: mypod-container
volumeMounts:
# In our example let's say we want to mount "vol1" on path "/a/b/c"
# So we declare this:
- name: vol1
mountPath: /a/b/c
# But what if we also want to use a child folder "d" ?
# If we try to use "/a/b/c/d" then we wont have access to /a/b/c
# because the mountPath /a/b/c is overwritten by mountPath /a/b/c/d
# So if we try the following mountPath we lose our above declaration:
# - name: vol1
# mountPath: /a/b/c/d # This overwrites the above mount to /a/b/c
# The solution:
# Using subPath we enjoy both worlds.
# (1) We dont overwrite the info in our volume at path /a/b/c .
# (2) We have a separate path /a/b/c/d that when we can write to
# without affecting the content in path /a/b/c.
# Here is how we write the correct declaration:
- name: vol1
mountPath: /a/b/c/d
subPath: d
Under the hood of mountPath & subPath
Lets look under the hood of kubernetes to see how it manages the mountPath & the subPath properties differently:
1. How kubernetes manages mountPath:
When a mountPath is declared then kubernetes creates a file with the name of the volume in the following path:
/var/lib/kubelet/pods/<pod-id>/volumes/kubernetes.io~empty-dir/<volume name>
So in our above manifest example this is what was created ("vol1" is the volume name):
/var/lib/kubelet/pods/301d...a71c/volumes/kubernetes.io~empty-dir/vol1
Now you can see that if we defined the "/a/b/c/d" mountPath we would have triggered a creation of another file "vol1" in same directory thus overwritting the original.
2. How kubernetes manages subPath:
When a subPath is declared then kubernetes creates a file with the SAME VOLUME name Volume but in a different path:
/var/lib/kubelet/pods/<pod-id>/volume-subpaths/<volume name>
So in our above manifest example this is what was created ("vol1" is the volume name):
/var/lib/kubelet/pods/3eaa...6d1/volume-subpaths/vol1
Conclusion:
Now you see that the subPath enables us to define an additional volumePath without colliding with the root voulmePath.
It does this by creating files with the same volume name but in different directories in kubernetes kubelet.
subPath is used to select a specific Path within the volume from which the container's volume should be mounted. This Defaults to "" (volume's root).
Check this mentioned here. So what that means is that you can still mount a volume on the path mentioned in your mountPath but instead of mounting it from the volume root, you can specify a separate subpath within the volume to be mounted under the volumeMount directory in your container.
To clarify on what this means, i created a simple volume on my minikube node.
docker#minikube:/mnt/data$ ls -lrth
total 8.0K
drwxr-xr-x 2 root root 4.0K Dec 30 16:23 path1
drwxr-xr-x 2 root root 4.0K Dec 30 16:23 path2
docker#minikube:/mnt/data$
docker#minikube:/mnt/data$ pwd
/mnt/data
As you can see that in this case, i have a directory and i have created two sub directories inside this volume. Under each of these path1 or path2 folder i have placed a different index file.
docker#minikube:/mnt/data$ pwd
/mnt/data
docker#minikube:/mnt/data$ cat path1/index.html
This is index file from path1
docker#minikube:/mnt/data$
docker#minikube:/mnt/data$ cat path2/index.html
This is index file from path2
docker#minikube:/mnt/data$
Now I created a sample PV using this volume on my minikube node using the sample manifest as below
apiVersion: v1
kind: PersistentVolume
metadata:
name: task-pv-volume
labels:
type: local
spec:
storageClassName: manual
capacity:
storage: 1Gi
accessModes:
- ReadWriteOnce
hostPath:
path: "/mnt/data"
After this, i created the sample PVC using the below manifest
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: task-pv-claim
spec:
storageClassName: manual
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 800Mi
Now if i created a nginx pod and used this PVC under my volume, depending on the subPath config that i use in my pod spec, i will have the volume mounted from that specific subfolder.
i.e. If i used the below manifest for my nginx pod
apiVersion: v1
kind: Pod
metadata:
name: test
spec:
containers:
- name: test
image: nginx
volumeMounts:
# a mount for site-data
- name: config
mountPath: /usr/share/nginx/html
subPath: path1
volumes:
- name: config
persistentVolumeClaim:
claimName: task-pv-claim
and i do a curl on the POD IP, i get the index.html served from path1.
Gauravs-MBP:K8s alieninvader$ kubectl exec -it mycurlpod -- /bin/sh
/ $ curl 172.17.0.3
This is index file from path1
And if i changed my pod manifest and used the subpath as path2, so that the new manifest becomes this
apiVersion: v1
kind: Pod
metadata:
name: test
spec:
containers:
- name: test
image: nginx
volumeMounts:
# a mount for site-data
- name: config
mountPath: /usr/share/nginx/html
subPath: path2
volumes:
- name: config
persistentVolumeClaim:
claimName: task-pv-claim
Then as expected the curl to nginx pod would produce the below output serving the file from path2.
Gauravs-MBP:K8s alieninvader$ kubectl exec -it mycurlpod -- /bin/sh
/ $ curl 172.17.0.3
This is index file from path2
/ $
I need to share a directory between two containers: myapp and monitoring and to achieve this I created an emptyDir: {} and then volumeMount on both the containers.
spec:
volumes:
- name: shared-data
emptyDir: {}
containers:
- name: myapp
volumeMounts:
- name: shared-data
mountPath: /etc/myapp/
- name: monitoring
volumeMounts:
- name: shared-data
mountPath: /var/read
This works fine as the data I write to the shared-data directory is visible in both containers. However, the config file that is created when creating the container under /etc/myapp/myapp.config is hidden as the shared-data volume is mounted over /etc/myapp path (overlap).
How can I force the container to first mount the volume to /etc/myapp path and then cause the docker image to place the myapp.config file under the default path /etc/myapp except that it is the mounted volume thus allowing the config file to be accessible by the monitoring container under /var/read?
Summary: let the monitoring container read the /etc/myapp/myapp.config file sitting on myapp container.
Can anyone advice please?
Can you mount shared-data at /var/read in an init container and copy config file from /etc/myapp/myapp.config to /var/read?
Consider using ConfigMaps with SubPaths.
A ConfigMap is an API object used to store non-confidential data in
key-value pairs. Pods can consume ConfigMaps as environment variables,
command-line arguments, or as configuration files in a volume.
Sometimes, it is useful to share one volume for multiple uses in a
single pod. The volumeMounts.subPath property specifies a sub-path
inside the referenced volume instead of its root.
ConfigMaps can be used as volumes. The volumeMounts inside the template.spec are the same as any other volume. However, the volumes section is different. Instead of specifying a persistentVolumeClaim or other volume type you reference the configMap by name. Than you can add the subPath property which would look something like this:
volumeMounts:
- name: shared-data
mountPath: /etc/myapp/
subPath: myapp.config
Here are the resources that would show you how to set it up:
Configure a Pod to Use a ConfigMap: official docs
Using ConfigMap SubPaths to Mount Files: step by step guide
Mount a file in your Pod using a ConfigMap: supplement
I have a volume with a secret called config-volume. I want to have that file in the /home/code/config folder, which is where the rest of the configuration files are. For that, I mount it as this:
volumeMounts:
- name: config-volumes
- mountPath: /home/code/config
The issue is that, after deploying, in the /home/code/config I only have the secret file and the rest of them are gone
So the /home/code/config is an existing folder (not empty), I suspect that the volumeMount overwrites the folder.
Is there a way that this can be done without overwriting everything?
You can do the following, taken from this GitHub issue
containers:
- volumeMounts:
- name: config-volumes
mountPath: /home/code/config
subPath: config
volumes:
- name: config-volumes
configMap:
name: my-config
Suggested that your ConfigMapis called my-config and that you have a key config in it.
Kubernetes Secrets are mounted as a directory, with each key as a file in that directory. So in your case, the config-volumes secret is mounted to /home/code/config, shadowing whatever that directory was before.
You could specify your volume mount as:
volumeMounts:
- name: config-volumes
- mountPath: /home/code/config/config-volumes
which would provide a config-volumes directory inside the config directory with files for your secret's keys inside.