As a beginner in container administration, I can't find a clear description of OpenShift's deployment stages and related YAML statements, specifically when persistent volume mounting and shell commands execution are involved. For example, in the RedHat documentation there is a lot of examples. A simple one is 16.4. Pod Object Definition:
apiVersion: v1
kind: Pod
metadata:
name: busybox-nfs-pod
labels:
name: busybox-nfs-pod
spec:
containers:
- name: busybox-nfs-pod
image: busybox
command: ["sleep", "60000"]
volumeMounts:
- name: nfsvol-2
mountPath: /usr/share/busybox
readOnly: false
securityContext:
supplementalGroups: [100003]
privileged: false
volumes:
- name: nfsvol-2
persistentVolumeClaim:
claimName: nfs-pvc
Now the question is: does the command sleep (or any other) execute before or after the mount of nfsvol-2 is finished? In other words, is it possible to use the volume's resources in such commands? And if it's not possible in this config, then which event handlers to use instead? I don't see any mention about an event like volume mounted.
does the command sleep (or any other) execute before or after the
mount of nfsvol-2 is finished?
To understand this, lets dig into the underlying concepts for Openshift.
OpenShift is a container application platform that brings docker and Kubernetes to the enterprise. So Openshift is nothing but an abstraction layer on top of docker and kubernetes along with additional features.
Regarding the volumes and commands lets consider the following example:
Let's run the docker container by mounting a volume, which is the home directory of host machine to the root path of the container(-v is option to attach volume).
$ docker run -it -v /home:/root ubuntu /bin/bash
Unable to find image 'ubuntu:latest' locally
latest: Pulling from library/ubuntu
50aff78429b1: Pull complete
f6d82e297bce: Pull complete
275abb2c8a6f: Pull complete
9f15a39356d6: Pull complete
fc0342a94c89: Pull complete
Digest: sha256:f871d0805ee3ce1c52b0608108dbdf1b447a34d22d5c7278a3a9dd78fc12c663
Status: Downloaded newer image for ubuntu:latest
root#1f07f083ba79:/# cd /root/
root#1f07f083ba79:~# ls
lost+found raghavendralokineni raghu user1
root#1f07f083ba79:~/raghavendralokineni# pwd
/root/raghavendralokineni
Now execute the sleep command in the container and exit.
root#1f07f083ba79:~/raghavendralokineni# sleep 10
root#1f07f083ba79:~/raghavendralokineni#
root#1f07f083ba79:~/raghavendralokineni# exit
Check the files available in the /home path which we have mounted to the container. This content is same as that of /root path in the container.
raghavendralokineni#iconic-glider-186709:/home$ ls
lost+found raghavendralokineni raghu user1
So when a volume is mounted to the container, any changes in the volume will be effected in the host machine as well.
Hence the volume will be mounted along with the container and commands will be executed after container is started.
Coming back to the your YAML file,
volumeMounts:
- name: nfsvol-2
mountPath: /usr/share/busybox
It says ,mount the volume nfsvol-2 to the container and the information regarding the volume is mentioned in volumes:
volumes:
- name: nfsvol-2
persistentVolumeClaim:
claimName: nfs-pvc
So mount the volume to the container and execute the command which is specifed:
containers:
- name: busybox-nfs-pod
image: busybox
command: ["sleep", "60000"]
Hope this helps.
Related
I need to copy a file inside my pod during the time of creation. I don't want to use ConfigMap and Secrets. I am trying to create a volumeMounts and copy the source file using the kubectl cp command—my manifest looks like this.
apiVersion: v1
kind: Pod
metadata:
name: copy
labels:
app: hello
spec:
containers:
- name: init-myservice
image: bitnami/kubectl
command: ['kubectl','cp','./test.json','init-myservice:./data']
volumeMounts:
- name: my-storage
mountPath: data
- name: init-myservices
image: nginx
volumeMounts:
- name: my-storage
mountPath: data
volumes:
- name: my-storage
emptyDir: {}
But I am getting a CrashLoopBackOff error. Any help or suggestion is highly appreciated.
it's not possible.
let me explain : you need to think of it like two different machine. here your local machine is the one where the file exist and you want to copy it in another machine with cp. but it's not possible. and this is what you are trying to do here. you are trying to copy file from your machine to pod's machine.
here you can do one thing just create your own docker image for init-container. and copy the file you want to store before building the docker image. then you can copy that file in shared volume where you want to store the file.
I do agree with an answer provided by H.R. Emon, it explains why you can't just run kubectl cp inside of the container. I do also think there are some resources that could be added to show you how you can tackle this particular setup.
For this particular use case it is recommended to use an initContainer.
initContainers - specialized containers that run before app containers in a Pod. Init containers can contain utilities or setup scripts not present in an app image.
Kubernetes.io: Docs: Concepts: Workloads: Pods: Init-containers
You could use the example from the official Kubernetes documentation (assuming that downloading your test.json is feasible):
apiVersion: v1
kind: Pod
metadata:
name: init-demo
spec:
containers:
- name: nginx
image: nginx
ports:
- containerPort: 80
volumeMounts:
- name: workdir
mountPath: /usr/share/nginx/html
# These containers are run during pod initialization
initContainers:
- name: install
image: busybox
command:
- wget
- "-O"
- "/work-dir/index.html"
- http://info.cern.ch
volumeMounts:
- name: workdir
mountPath: "/work-dir"
dnsPolicy: Default
volumes:
- name: workdir
emptyDir: {}
-- Kubernetes.io: Docs: Tasks: Configure Pod Initalization: Create a pod that has an initContainer
You can also modify above example to your specific needs.
Also, referring to your particular example, there are some things that you will need to be aware of:
To use kubectl inside of a Pod you will need to have required permissions to access the Kubernetes API. You can do it by using serviceAccount with some permissions. More can be found in this links:
Kubernetes.io: Docs: Reference: Access authn authz: Authentication: Service account tokens
Kubernetes.io: Docs: Reference: Access authn authz: RBAC
Your bitnami/kubectl container will run into CrashLoopBackOff errors because of the fact that you're passing a single command that will run to completion. After that Pod would report status Completed and it would be restarted due to this fact resulting in before mentioned CrashLoopBackOff. To avoid that you would need to use initContainer.
You can read more about what is happening in your setup by following this answer (connected with previous point):
Stackoverflow.com: Questions: What happens one of the container process crashes in multiple container POD?
Additional resources:
Kubernetes.io: Pod lifecycle
A side note!
I also do consider including the reason why Secrets and ConfigMaps cannot be used to be important in this particular setup.
i'm stack at k8s log storage.we have logs that can't output to stdout,but have to save to dir.we want to save to glusterfs shared dir like /data/logs/./xxx.log our apps are written in java ,how can we do that
This is mostly up to your CRI plugin, usually Docker command line options. They already do write to local disk by default, you just need to mount your volume at the right place (probably /var/log/containers or similar, look at your Docker config).
I had the same problem with one 3rd party application. It was writing logs in a log file and I wanted Fluentd to be able to get them so I wanted somehow to print them on the stdout.
I found a workaround with one additional container running alongside with the app container in the same pod.
Lets say the 3rd party app is writing logs in the following file:
/some/folders/logs/app_log_file.log
The following pod will run two containers, one with the app and the other with busybox image which we will use to fetch the logs from the app container.
apiVersion: v1
kind: Pod
metadata:
name: application-pod
spec:
containers:
- name: app-container
image: <path-to-app-image>
imagePullPolicy: IfNotPresent
volumeMounts:
- name: log-volume
mountPath: /some/folders/logs
- name: log-fetcher-container
image: busybox
args: [/bin/sh, -c, 'sleep 60 && tail -n+1 -f /var/log/app_log_file.log']
volumeMounts:
- name: log-volume
mountPath: /var/log
volumes:
- name: log-volume
emptyDir: {}
As you can see this manifest is creating a empty volume and mounting volume to the /some/folders/logs folder in the app container and to the /var/log folder in the log fetcher container. Now every file that the app container writes to /some/folders/logs will be visible in /var/log also.
That's why the busybox image is running a shell command:
sleep 60 && tail -n+1 -f /var/log/app_log_file.log
First we wait 60 seconds because the app container must have time to start up and create the log file and then the tail command is going to print every new line in the log file to the stdout of the log fetcher container.
And now the fluentd will be able to get the logs from the log file of the app container getting the stdout logs of the log fetcher container.
I have a container running inside a pod and I want to be able to monitor its content every week. I want to write a Kube cronjob for it. Is there a best way to do this?
At the moment I am doing this by running a script in my local machine that does kubectl exec my-container and monitors the content of the directory in that container.
kubectl exec my-container sounds perfectly fine to me. You might want to look at this if you want to run kubectl in a pod (Kubernetes CronJob).
There are other ways but depending on what you are trying to do in the long term it might be an overkill. For example:
You can set up a Fluentd or tail/grep sidecar (or ls, if you are using a binary file?) to send the content or part of the content of that file to an Elasticsearch cluster.
You can set up Prometheus in Kubernetes to scrape metrics on the pod mounted filesystems. You will probably have to use a custom exporter in the pod or something else that exports files in mount points in the pod. This is a similar example.
You can run your script in another sidecar of your pod.
Define a empty directory volume
Mount this volume as your content directory
Also mount this directory to sidecar, so that it can access and able to monitor.
Example:
apiVersion: v1
kind: Pod
metadata:
name: monitor-by-sidecar
spec:
restartPolicy: Never
volumes: # empty directory volume
- name: shared-data
emptyDir: {}
containers:
- name: container-which-produce-content # This container is main application which generate contect. Suppose in /usr/share/nginx/html directory
image: debian
volumeMounts:
- name: shared-data
mountPath: /usr/share/nginx/html
command: ["/bin/bash", "-c"]
args:
- while true;
do
echo "hello world";
echo "----------------" > /usr/share/nginx/html/index.html;
cat /usr/share/nginx/html/index.html;
done
- name: container-which-run-script-to-monitor # this container will run your monitor scripts. this container mount main application's volume in /pod-data directory and run required scripts.
image: debian
volumeMounts:
- name: shared-data
mountPath: /pod-data
command: ["/bin/sh", "-c"]
args:
- while true;
do
echo "hello";
sleep 10;
ls -la /pod-data/;
cat /pod-data/index.html;
done
Example Description
First container(named container-which-produce-content) is main application, which mount a emptyDir volume in /usr/share/nginx/html. In this directory main application will generate data.
Second container(named container-which-run-script-to-monitor) will mount same emptyDir volume (named shared-data which also mounted by main application in /usr/share/nginx/html dir) in /pod-data directory. This /pod-data contains whole data which main application generated in /usr/share/nginx/html directory. You can then run your scripts on this directory.
My pod has two containers - a primary container, and a sidecar container that monitors the /var/run/utmp file in the primary container and takes action when it changes. I'm trying to figure out how to make this file visible in the sidecar container.
This page describes how to use an emptyDir volume to share directories between containers in a pod. However, this only seems to work for directories, not single files. I also can't use this strategy to share the entire /var/run/ directory in the primary container, since mounting a volume there erases the contents of the directory, which the container needs to run.
I tried to work around this by creating a symlink to utmp in another directory and mounting that directory, but it doesn't look like symlinks in volumes are resolved in the way they would need to be for this to work.
Is there any way I can make one file in a container visible to other containers in the same pod? The manifest I'm experimenting with looks like this:
apiVersion: v1
kind: Pod
metadata:
name: utmp-demo
spec:
restartPolicy: Never
containers:
- name: main
image: debian
command: ["/bin/bash"]
args: ["-c", "sleep infinity"]
volumeMounts:
- name: main-run
mountPath: /var/run # or /var/run/utmp, which crashes
- name: helper
image: debian
command: ["/bin/bash"]
args: ["-c", "sleep infinity"]
volumeMounts:
- name: main-run
mountPath: /tmp/main-run
volumes:
- name: main-run
emptyDir: {}
If you can move the file to be shared in an empty subfolder this could be a simple solution.
For example, move your file to /var/run/utmp/utmp and share /var/run/utmp folder with an emptydir.
Very simple question, where is the emptyDir located in my minikube VM? Since the emptyDir volume is pod dependent, it should exist on the VM otherwise it will die with a container exiting. When I do minikube ssh I cannot locate the volume. I need to inspect it and see if my containers are behaving how I want them to, copying some files to the volume mounted on them. Trying find / -type d -name cached results in many permission denieds and the volume is not in the rest of the dirs. My YAML has the following part:
...
volumes:
- name: cached
emptyDir: {}
and also commands in a container where the container copies some files to the volume:
containers:
- name: plum
image: plumsempy/plum
command: ["/bin/sh", "-c"]
args: ["mkdir /plum/cached"]
volumeMounts:
- mountPath: /plum/cached
name: cahced
command: ["bin/sh/", "-c"]
args: ["cp /plum/prune/cert.crt /plume/cached/"]
The container naturally exists after doing its job.
A better way to see if your containers are behaving is by logging in into the container using the kubectl command.
That said: The location should of emptyDir should be in /var/lib/kubelet/pods/{podid}/volumes/kubernetes.io~empty-dir/ on the given node where your pod is running.