I'm trying to figure out a way that a container or pod can know some specific information about the node that it's being scheduled to. For example, my container might have to know if a GPU is present or not on that node in order to decide whether or not to enable GPU acceleration. Another example would be knowing the $DISPLAY variable of the node to know what X server to output graphics to.
What's the best approach to this?
Thanks
Update: If I could get the node-name from within the container, I could do a lookup against an external service to get the information I need. Is there a way to do this?
OP Here. I've found a somewhat decent way of accomplishing this.
On setting the node up with my cluster i can install a script to source environment variables to a file then volume-mount that file into the container.
Alterntively I could also store config for each ndoe in a separate service and inject the nodeName to lookup properties of a specific node as follows:
https://kubernetes.io/docs/tasks/inject-data-application/environment-variable-expose-pod-information/#use-pod-fields-as-values-for-environment-variables
Then based on the name, my container can look-up via service or config map a mapping of nodeName to whatever information I need form the node. All I have to do is keep this service/config map up-to-date with the node's information.
Taints and Tolerations were designed for that.
Related
Problem:
I want every pod created in my cluster to hold\point the same data
e.g. let's say I want all of them to have an env vars like "OWNER=MYNAME".
there are multiple users in my cluster and I don't want them to start changing their YAMLs and manually assign OWNER:MYNAME to env.
Is there a way to have all current/future pods to be assigned automatically with a predefined value or mount a configmap so that the same information will be available in every single pod?
can this be done on the cluster level? namespace level?
I want it to be transparent to the user, meaning a user would apply whatever pod to the cluster, and the info could be available to him without even asking.
Thanks, everyone!
Pod Preset might help you here to partially achieve what you need. Pod Preset resource allows injecting additional runtime requirements into a Pod at creation time. You use label selectors to specify the Pods to which a given PodPreset applies.
Check this to know how pod preset works.
First you need to enable pod preset in your cluster.
You can use Pod Preset to inject env variables or volumes in your pod.
You can also inject configmap in your pod.
Make use of some common label for all the pods which you want to have common config, use this common label in your pod preset resource.
Unfortunately there are plans to remove pod presets altogether in coming releases, but I guess you can still use it with current releases. Although there are other implementations similar to pod presets, which you can try.
Is there any way to get a node labels from within a container for use as an environment variable?
It's similar to this https://kubernetes.io/docs/tasks/inject-data-application/environment-variable-expose-pod-information/, but I need to use a label from the node for injecting into the container instead.
Thanks in advance!
You will not be able to get node labels without sending some requests to the k8s api server. You could do that - but that would mean every pod will need read access and that's not great security wise.
How about an alternate solution - if you need to make sure the pod is running on nodes with specific labels, you can use taints and tolerations to achieve that more easily.
i am new to kubernetes and i have some functionally that i need to implement.
i need to set an env variable for only one docker container in a service.
for example- if i have 3 users containers then 1 of them need to have env variable named master
i did it with nomad. nomad set an env variable named NOMAD_ALLOC_INDEX, that give me the index of the container, this way i checked that if the container index was 0 then it is master.
i try find if kubernetes have a similar variable but didn't find anywhere.
i also try find in google an alternative solution but ended up with nothing.
any ideas of how i can achieve it ?
If you want sequential indexes, StatefulSet is your solution. Otherwise lookup kubernetes leader election, there are ways to solve it with ie. sidecar container performing leader election and exposing status via http call so you can curl localhost:port and see if the pod is master or not.
I wonder how would one implement a colocated auxiliary container in a Pod within a Deployment which does not provide a service but rather a job/batch workload?
Background of my questions is, that I want to deploy a scalable service at which each instance needs configuration after its start. This configuration is done via a HTTP POST to its local colocated service instance. I've implemented a auxiliary container for this in order to benefit from the feature of colocation. So the auxiliary container always knows which instance needs to be configured.
Problem is, that the restartPolicy needs to be defined at the Pod level. I am looking for something like restart policy always for the service and a different restart policy onFailurefor the configuration job.
I know that k8s provides the Job resource for such workloads. But is there an option to colocate those jobs to Pods?
Furthermore I've stumbled across the so called init containers which might be defined via annotations. But these suffer the drawback, that k8s ensures that the actual Pod is only started after the init container did run. So for my very scenario it seems unsuitable.
As I understand you need your service running to configure it.
Your solution is workable and you can set restartPolicy: always you just need a way to tell your one off configuration container that it already ran. You could create and attach an emptyDir volume to your configuration container, create a file on it to mark your configuration successful and check for this file from your process. After your initialization you enter sleep in a loop. The downside is that some resources will be taken up by that container too.
Or you can just add an extra process in the same container and do the configuration (maybe with the file mentioned above as a guard to avoid configuring twice). So write a simple shell script like this and run it instead of your main process:
#!/bin/sh
(
[ -f /mnt/guard-vol/stamp ] && exit 0
/opt/my-config-process parameters && touch /mnt/guard-vol/stamp
) &
exec /opt/my-main-process "$#"
Alternatively you could implement a separate pod that queries the kubernetes API for pods of your service with label configured=false. Configure it and remove the label with the API. You should also modify your Service to select configured=true pods.
I am following up guide [1] to create multi-node K8S cluster which has 1 master and 2 nodes. Also, a label needs to set to each node respectively.
Node 1 - label name=orders
Node 2 - label name=payment
I know that above could be achieved running kubectl command
kubectl get nodes
kubectl label nodes <node-name> <label-key>=<label-value>
But I would like to know how to set label when creating a node. Node creation guidance is in [2].
Appreciate your input.
[1] https://coreos.com/kubernetes/docs/latest/getting-started.html
[2] https://coreos.com/kubernetes/docs/latest/deploy-workers.html
In fact there is a trivial way to achieve that since 1.3 or something like that.
What is responsible for registering your node is the kubelet process launched on it, all you need to do is pass it a flag like this --node-labels 'role=kubemaster'. This is how I differentiate nodes between different autoscaling groups in my AWS k8s cluster.
This answer is now incorrect (and has been for several versions of Kubernetes). Please see the correct answer by Radek 'Goblin' Pieczonka
There are a few options available to you. The easiest IMHO would be to use a systemd unit to install and configure kubectl, then run the kubectl label command. Alternatively, you could just use curl to update the labels in the node's metadata directly.
That being said, while I don't know your exact use case, the way you are using the labels on the nodes seems to be an effort to bypass some of Kubernetes key features, like dynamic scheduling of components across nodes. I would suggest rather than work on labeling the nodes automatically that you try to address why you need to identify the nodes in the first place.
I know this isn't creation time but, the following is pretty easy (labels follow pattern of key=value):
k label node minikube gpu.nvidia.com/model=Quadro_RTX_4000 node.coreweave.cloud/cpu=intel-xeon-v2
node/minikube labeled