I want to run this docker image in kubernetes:
https://hub.docker.com/_/rabbitmq
This is not a problem, and it is running. The problem is that I need to send through switches do the "docker run" command. For this image, when starting the container in docker you would run this:
docker run -d --hostname my-rabbit --name some-rabbit rabbitmq:3
The yaml file will then look something like this:
apiVersion: apps/v1
kind: Deployment
metadata:
name: rb
namespace: rabbittest
spec:
selector:
matchLabels:
app: rb
replicas: 1
template:
metadata:
labels:
app: rb
spec:
containers:
- name: rb-container
env:
- name: HOSTNAME
value: "rbnode001"
image: rabbitmq:3-management
resources:
requests:
cpu: 100m
memory: 128Mi
limits:
cpu: 250m
memory: 256Mi
volumeMounts:
- name: azure
mountPath: /mnt/azure
ports:
- containerPort: 5672
volumes:
- name: azure
azureFile:
secretName: azure-secret
shareName: rabbittest
readOnly: false
My question is, how do I get kubernetes to apply the --name and --hostname when kubernetes executes the "docker run" commands?
First of all, you need to create StatefulSets to RabbitMQ.
In your StatefulSet, add this ENV.
env:
- name: POD_NAME
valueFrom:
fieldRef:
apiVersion: v1
fieldPath: metadata.name
- name: POD_NAMESPACE
valueFrom:
fieldRef:
apiVersion: v1
fieldPath: metadata.namespace
- name: K8S_SERVICE_NAME
value: rabbitmq-headless
- name: RABBITMQ_NODENAME
value: $(POD_NAME).$(K8S_SERVICE_NAME).$(POD_NAMESPACE).svc.cluster.local
Here, K8S_SERVICE_NAME is headless service required by statefulset.
Finally, RABBITMQ_NODENAME will hold the HOSTNAME.
Kubernetes's options are just different from Docker's options. Many options have equivalents in the pod spec object; some options don't have direct equivalents, or are split into multiple layers (volume mounts, published ports).
For the two options you mention:
docker run --name sets the name of the container. In Kubernetes you don't usually directly interact with containers, so you can't set their names. The metadata: {name: } sets the name of the Deployment, and generated Pods have names derived from that; the containers within a Pod also have names but it's very rare to use these.
docker run --hostname sets the name a container believes its hostname is. You almost never need this option; RabbitMQ is the notable exception. The pod spec has a hostname: option that can set this.
spec:
template:
spec:
hostname: my-rabbit
containers: [...]
As #Shahriar's answer suggests, a StatefulSet is a better match for deployments that need some backing persistent state. In this case the StatefulSet automatically sets the host name based on the pod identity, so you don't need to do anything.
(The important detail for RabbitMQ is that the hostname must be consistent across recreating the container, and if the hostname is always rb-0 from a StatefulSet that satisfies this rule. An autogenerated Docker container ID, or a variable Kubernetes Deployment pod name, will be different on every restart, and Rabbit will lose track of its state.)
Related
I want to be able to know the Kubernetes uid of the Deployment that created the pod, from within the pod.
The reason for this is so that the Pod can spawn another Deployment and set the OwnerReference of that Deployment to the original Deployment (so it gets Garbage Collected when the original Deployment is deleted).
Taking inspiration from here, I've tried*:
Using field refs as env vars:
containers:
- name: test-operator
env:
- name: DEPLOYMENT_UID
valueFrom:
fieldRef: {fieldPath: metadata.uid}
Using downwardAPI and exposing through files on a volume:
containers:
volumeMounts:
- mountPath: /etc/deployment-info
name: deployment-info
volumes:
- name: deployment-info
downwardAPI:
items:
- path: "uid"
fieldRef: {fieldPath: metadata.uid}
*Both of these are under spec.template.spec of a resource of kind: Deployment.
However for both of these the uid is that of the Pod, not the Deployment. Is what I'm trying to do possible?
The behavior is correct, the Downward API is for pod rather than deployment/replicaset.
So I guess the solution is set the name of deployment manually in spec.template.metadata.labels, then adopt Downward API to inject the labels as env variables.
I think it's impossible to get the UID of Deployment itself, you can set any range of runAsUser while creating the deployment.
Try this command to get the UIDs of the existing pods:
kubectl get pod -o jsonpath='{range .items[*]}{#.metadata.name}{" runAsUser: "}{#.spec.containers[*].securityContext.runAsUser}{" fsGroup: "}{#.spec.securityContext.fsGroup}{" seLinuxOptions: "}{#.spec.securityContext.seLinuxOptions.level}{"\n"}{end}'
It's not the exact what you wanted to get, but it can be a hint for you.
To set the UID while creating the Deployment, see the example below:
apiVersion: apps/v1
kind: Deployment
metadata:
name: toolbox2
labels:
app: toolbox2
spec:
replicas: 3
selector:
matchLabels:
app: toolbox2
template:
metadata:
labels:
app: toolbox2
spec:
securityContext:
supplementalGroups: [1000620001]
seLinuxOptions:
level: s0:c25,c10
containers:
- name: net-toolbox
image: quay.io/wcaban/net-toolbox
ports:
- containerPort: 2000
securityContext:
runAsUser: 1000620001
I have my application running in K8S pods. my application writes logs to a particular path for which I already have volume mounted on the pod. my requirement is to schedule cronjob which will trigger weekly once and read the logs from that pod volume and generate a report base on my script (which is basically filtering the logs based on some keywords). and send the report via mail.
unfortunately I am not sure how I will proceed on this as I couldn't get any doc or blog which talks about integrating conrjob to existing pod or volume.
apiVersion: v1
kind: Pod
metadata:
name: webserver
spec:
volumes:
- name: shared-logs
emptyDir: {}
containers:
- name: nginx
image: nginx
volumeMounts:
- name: shared-logs
mountPath: /var/log/nginx
- name: sidecar-container
image: busybox
command: ["sh","-c","while true; do cat /var/log/nginx/access.log /var/log/nginx/error.log; sleep 30; done"]
volumeMounts:
- name: shared-logs
mountPath: /var/log/nginx
apiVersion: batch/v1beta1
kind: CronJob
metadata:
name: "discovery-cronjob"
labels:
app.kubernetes.io/name: discovery
spec:
schedule: "*/5 * * * *"
jobTemplate:
spec:
template:
spec:
containers:
- name: log-report
image: busybox
command: ['/bin/sh']
args: ['-c', 'cat /var/log/nginx/access.log > nginx.log']
volumeMounts:
- mountPath: /log
name: shared-logs
restartPolicy: Never
I see two things here that you need to know:
Unfortunately, it is not possible to schedule a cronjob on an existing pod. Pods are ephemeral and job needs to finish. It would be impossible to tell if the job completed or not. This is by design.
Also in order to be able to see the files from one pod to another you must use a PVC. The logs created by your app have to be persisted if your job wants to access it. Here you can find some examples of how to Create ReadWriteMany PersistentVolumeClaims on your Kubernetes Cluster:
Kubernetes allows us to provision our PersistentVolumes dynamically
using PersistentVolumeClaims. Pods treat these claims as volumes. The
access mode of the PVC determines how many nodes can establish a
connection to it. We can refer to the resource provider’s docs for
their supported access modes.
I'm currently using Kubernetes to schedule a DaemonSet on both master and worker nodes.
The DaemonSet definition is the same for both node types (same image, same volumes, etc), the only difference is that when the entrypoint is executed, I need to write a different configuration file (which is generated in Python with some dynamic values) if the node is a master or a worker.
Currently, to overcome this I'm using two different DaemonSet definitions with an env value which tells if the node is a master or not. Here's the yaml file (only relevant parts):
apiVersion: extensions/v1beta1
kind: DaemonSet
metadata:
name: worker-ds
namespace: kube-system
labels:
k8s-app: worker
spec:
...
spec:
hostNetwork: true
containers:
- name: my-image
...
env:
- name: NODE_IP
valueFrom:
fieldRef:
fieldPath: status.hostIP
- name: IS_MASTER
value: "false"
...
---
apiVersion: extensions/v1beta1
kind: DaemonSet
metadata:
name: master-ds
namespace: kube-system
labels:
k8s-app: master
spec:
...
spec:
hostNetwork: true
nodeSelector:
node-role.kubernetes.io/master: ""
tolerations:
- key: node-role.kubernetes.io/master
operator: Exists
effect: NoSchedule
containers:
- name: my-image
...
env:
- name: NODE_IP
valueFrom:
fieldRef:
fieldPath: status.hostIP
- name: IS_MASTER
value: "true"
...
However, since the only difference is the IS_MASTER value, I want to collapse both the definitions in a single one that programmatically understands if the current node where the pod is being scheduled is a master or a worker.
Is there any way to know this information about the node programmatically (even reading a configuration file [for example something that only the master has or viceversa] in the node or something like that)?
Thanks in advance.
Unfortunately, there is not a convenient way to access the node information in pod.
If you only want a single DaemonSet definition, you can add a sidecar container to your pod, the sidecar container can access the k8s api, then your main container can get something useful from the sidecar.
By the way, I think your current solution is properly :)
You can tell a node is the master if it has the label node-role.kubernetes.io/master: "".
What you need to do is access that label from your containers which can be done with the Downward Api (Edit: Wrong, only Pod information can be accessed from the Downward Api). You can mount the labels inside your containers using:
volumes:
- name: podinfo
downwardAPI:
items:
- path: "labels"
fieldRef:
fieldPath: metadata.labels
You can then search the content of that file from within the container.
I am trying to "pass" a value from the init container to a container. Since values in a configmap are shared across the namespace, I figured I can use it for this purpose. Here is my job.yaml (with faked-out info):
apiVersion: batch/v1
kind: Job
metadata:
name: installer-test
spec:
template:
spec:
containers:
- name: installer-test
image: installer-test:latest
env:
- name: clusterId
value: "some_cluster_id"
- name: in_artifactoryUrl
valueFrom:
configMapKeyRef:
name: test-config
key: artifactorySnapshotUrl
initContainers:
- name: artifactory-snapshot
image: busybox
command: ['kubectl', 'create configmap test-config --from-literal=artifactorySnapshotUrl=http://artifactory.com/some/url']
restartPolicy: Never
backoffLimit: 0
This does not seem to work (EDIT: although the statements following this edit note may still be correct, this is not working because kubectl is not a recognizable command in the busybox image), and I am assuming that the pod can only read values from a configmap created BEFORE the pod is created. Has anyone else come across the difficulty of passing values between containers, and what did you do to solve this?
Should I deploy the configmap in another pod and wait to deploy this one until the configmap exists?
(I know I can write files to a volume, but I'd rather not go that route unless it's absolutely necessary, since it essentially means our docker images must be coupled to an environment where some specific files exist)
You can create an EmptyDir volume, and mount this volume onto both containers. Unlike persistent volume, EmptyDir has no portability issue.
apiVersion: batch/v1
kind: Job
metadata:
name: installer-test
spec:
template:
spec:
containers:
- name: installer-test
image: installer-test:latest
env:
- name: clusterId
value: "some_cluster_id"
volumeMounts:
- name: tmp
mountPath: /tmp/artifact
initContainers:
- name: artifactory-snapshot
image: busybox
command: ['/bin/sh', '-c', 'cp x /tmp/artifact/x']
volumeMounts:
- name: tmp
mountPath: /tmp/artifact
restartPolicy: Never
volumes:
- name: tmp
emptyDir: {}
backoffLimit: 0
If for various reasons, you don't want to use share volume. And you want to create a configmap or a secret, here is a solution.
First you need to use a docker image which contains kubectl : gcr.io/cloud-builders/kubectl:latest for example. (docker image which contains kubectl manage by Google).
Then this (init)container needs enough rights to create resource on Kubernetes cluster. Ok by default, kubernetes inject a token of default service account named : "default" in container, but I prefer to make more explicit, then add this line :
...
initContainers:
- # Already true by default but if use it, prefer to make it explicit
automountServiceAccountToken: true
name: artifactory-snapshot
And add "edit" role to "default" service account:
kubectl create rolebinding default-edit-rb --clusterrole=edit --serviceaccount=default:myapp --namespace=default
Then complete example :
apiVersion: batch/v1
kind: Job
metadata:
name: installer-test
spec:
template:
spec:
initContainers:
- # Already true by default but if use it, prefer to make it explicit.
automountServiceAccountToken: true
name: artifactory-snapshot
# You need to use docker image which contains kubectl
image: gcr.io/cloud-builders/kubectl:latest
command:
- sh
- -c
# the "--dry-run -o yaml | kubectl apply -f -" is to make command idempotent
- kubectl create configmap test-config --from-literal=artifactorySnapshotUrl=http://artifactory.com/some/url --dry-run -o yaml | kubectl apply -f -
containers:
- name: installer-test
image: installer-test:latest
env:
- name: clusterId
value: "some_cluster_id"
- name: in_artifactoryUrl
valueFrom:
configMapKeyRef:
name: test-config
key: artifactorySnapshotUrl
First of all, kubectl is a binary. It was downloaded in your machine before you could use the command. But, In your POD, the kubectl binary doesn't exist. So, you can't use kubectl command from a busybox image.
Furthermore, kubectl uses some credential that is saved in your machine (probably in ~/.kube path). So, If you try to use kubectl from inside an image, this will fail because of missing credentials.
For your scenario, I will suggest the same as #ccshih, use volume sharing.
Here is the official doc about volume sharing between init-container and container.
The yaml that is used here is ,
apiVersion: v1
kind: Pod
metadata:
name: init-demo
spec:
containers:
- name: nginx
image: nginx
ports:
- containerPort: 80
volumeMounts:
- name: workdir
mountPath: /usr/share/nginx/html
# These containers are run during pod initialization
initContainers:
- name: install
image: busybox
command:
- wget
- "-O"
- "/work-dir/index.html"
- http://kubernetes.io
volumeMounts:
- name: workdir
mountPath: "/work-dir"
dnsPolicy: Default
volumes:
- name: workdir
emptyDir: {}
Here init-containers saves a file in the volume and later the file was available in inside the container. Try the tutorial by yourself for better understanding.
I want the all processes within the pod see the same network and process table, as well as share any IPCs with the host processes.
I know it possible when we use docker by leveraging the following command.
docker run -it --privileged --ipc=host --net=host --pid=host \
-v /:/host -v /run:/run -v /etc/localtime:/etc/localtime \
--name privcontainer centos7 /bin/bash
On the other hand, is there any way to run super-privileged containers using Kubernetes?
If possible, I would like to know the way to write pod yaml file.
There is a privileged flag on the SecurityContext of the container spec.
Check out documentation for more details.
I could only find an example from the v1.4 docs:
apiVersion: v1
kind: Pod
metadata:
name: hello-world
spec:
containers:
- name: hello-world-container
# The container definition
# ...
securityContext:
privileged: true ###Here is what you are looking for
seLinuxOptions:
level: "s0:c123,c456"
Even more infos here
I'm sure you're aware, but as a general word of caution, the privileged will remove all container security settings and open up the cluster to potential security vulnerabilities.
To disable the namespacing of a container PIDs, and thus allowing this container to view all processes on a host, you need to specify hostPID: true in the pod specs.
You might find this manifest useful if you want to inspect a Kubernetes host from within a pod:
apiVersion: apps/v1
kind: DaemonSet
metadata:
name: debug
spec:
selector:
matchLabels:
app: debug
template:
metadata:
labels:
app: debug
name: debug
spec:
tolerations:
- key: node-role.kubernetes.io/master
effect: NoSchedule
hostNetwork: true
hostPID: true
containers:
- name: linux
image: alpine
args:
- sleep
- "3600"
securityContext:
privileged: true
runAsGroup: 0
runAsUser: 0
volumeMounts:
- mountPath: /mnt/host
name: host
volumes:
- hostPath:
path: /
type: ""
name: host
This will instantiate a "debug" pod on each node on your cluster (including control-plane node if they are visible to you). This pod will have access to all PIDs from the host, will see all its networks, and the node filesystem will be browseable at /mnt/host.