I am containerizing spring-boot applications on kubernetes and I want to have a different application property file for each replica of POD.
As I want to have different config for different pod replicas.
Any help on above would be appreciated.
They're not really replicas if you want a unique configuration for each pod. I think you may be looking for a StatefulSet. Quoting from the docs:
Like a Deployment, a StatefulSet manages Pods that are based on an identical container spec. Unlike a Deployment, a StatefulSet maintains a sticky identity for each of their Pods. These pods are created from the same spec, but are not interchangeable: each has a persistent identifier that it maintains across any rescheduling.
For example, given a StatefulSet like this:
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: example
spec:
selector:
matchLabels:
app: example
serviceName: "example"
replicas: 3
template:
metadata:
labels:
app: example
spec:
containers:
- name: nginx
image: docker.io/nginxinc/nginx-unprivileged:mainline
ports:
- containerPort: 80
name: http
I end up with:
$ kubectl get pod
NAME READY STATUS RESTARTS AGE
example-0 1/1 Running 0 34s
example-1 1/1 Running 0 31s
example-2 1/1 Running 0 28s
In each pod, I can look at the value of $HOSTNAME to find my unique name, and I could use that to extract appropriate configuration from a directory path/structured file/etc.
Related
Say I have a pod YAML such as:
apiVersion: v1
kind: Pod
metadata:
name: nginx
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx:1.19.1
And a Deployment:
apiVersion: apps/v1
kind: Deployment
metadata:
name: my-deployment
spec:
replicas: 3
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx:1.17.1
ports:
- containerPort: 80
Now I first create the Pod:
$ kubectl apply -f pod.yaml
And only then the Deployment:
$ kubectl apply -f deployment.yaml
I thought that, since the pod.yaml metadata includes a app: nginx selector, the Deployment controller will only create 2 nginx:1.17.1 pods, but I see that all 3 are created. Why is that?
In addition to creating the app: nginx label, Deployment controller also added the pod-template-hash label for each pod that was created.
If we check labels for running pods, we can see pod-template-hash=5d5dd5dd49 label for my-deployment pods:
$ kubectl get pods --show-labels
NAME READY STATUS RESTARTS AGE LABELS
my-deployment-5d5dd5dd49-9tbcx 1/1 Running 0 55s app=nginx,pod-template-hash=5d5dd5dd49
my-deployment-5d5dd5dd49-b88f4 1/1 Running 0 55s app=nginx,pod-template-hash=5d5dd5dd49
my-deployment-5d5dd5dd49-x7n8q 1/1 Running 0 55s app=nginx,pod-template-hash=5d5dd5dd49
nginx 1/1 Running 0 62s app=nginx
According to the official documentation:
The pod-template-hash label ensures that child ReplicaSets of a Deployment do not
overlap. It is generated by hashing the PodTemplate of the
ReplicaSet and using the resulting hash as the label value that is
added to the ReplicaSet selector, Pod template labels, and in any
existing Pods that the ReplicaSet might have.
This is why the changes from the Deployment didn't apply to a single pod with only the app: nginx label.
I want to maintain different configuration for each pod, so planning to fetch properties from spring cloud config based on pod name.
Ex:
Properties in cloud
PodName1.property1 = "xxx"
PodName2.property1 ="yyy";
Property value will be different for each pod. Planning to fetch properties from cloud ,based on container name Environment.get("current pod name"+ " propertyName").
So I want to set fixed hostname/pod name
If the above is not possible, is there any alternative ?
You can use statefulsets if you want fixed pod names for your application.
e.g.
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: web # this will be used as prefix in pod name
spec:
serviceName: "nginx"
replicas: 2 # specify number of pods that should be running
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: k8s.gcr.io/nginx-slim:0.8
ports:
- containerPort: 80
name: web
This template will create 2 pods of nginx in default namespace with names as following:
kubectl get pods -l app=nginx
NAME READY STATUS RESTARTS AGE
web-0 1/1 Running 0 1m
web-1 1/1 Running 0 1m
A basic example can be found here.
I have an existing kubernetes deployment which is running fine. Now I want to edit it with some new environment variables which I will use in the pod.
Editing a deployment will delete and create new pod or it will update the existing pod.
My requirement is I want to create a new pod whenever I edit/update the deployment.
Kubernetes is always going to recreate your pods in case you change/create env vars.
Lets check this together creating a deployment without any env var on it:
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-deployment
labels:
app: nginx
spec:
replicas: 3
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx
ports:
- containerPort: 80
Let's check and note these pod names so we can compare later:
$ kubectl get pod
NAME READY STATUS RESTARTS AGE
nginx-deployment-56db997f77-9mpjx 1/1 Running 0 8s
nginx-deployment-56db997f77-mgdv9 1/1 Running 0 8s
nginx-deployment-56db997f77-zg96f 1/1 Running 0 8s
Now let's edit this deployment and include one env var making the manifest look like this:
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-deployment
labels:
app: nginx
spec:
replicas: 3
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx
env:
- name: STACK_GREETING
value: "Hello from the MARS"
ports:
- containerPort: 80
After we finish the edition, let's check our pod names and see if it changed:
$ kubectl get pod
nginx-deployment-5b4b68cb55-9ll7p 1/1 Running 0 25s
nginx-deployment-5b4b68cb55-ds9kb 1/1 Running 0 23s
nginx-deployment-5b4b68cb55-wlqgz 1/1 Running 0 21s
As we can see, all pod names changed. Let's check if our env var got applied:
$ kubectl exec -ti nginx-deployment-5b4b68cb55-9ll7p -- sh -c 'echo $STACK_GREETING'
Hello from the MARS
The same behavior will occur if you change the var or even remove it. All pods need to be removed and created again for the changes to take place.
If you would like to create a new pod, then you need to create a new deployment for that. By design deployments are managing the replicas of pods that belong to them.
With node_selector we can schedule a particular deployment replicas to a certain node pool. But how to make sure that at least one pod is running in every node (say size of node pool is more than 1)
I need this to ensure my pods are spread across the node pool, so that if a particular node face an issue (say disconnected from cluster) my application would still run.
With nodeSelector you can directly tie a Pod to a node, but it doesn't provide any means for spreading the Pods of a Deployment across the nodes.
To spread Pods across the nodes, you can use Pod anti-affinity.
For example:
apiVersion: apps/v1
kind: Deployment
metadata:
name: my-app
spec:
selector:
matchLabels:
app: my-app
replicas: 3
template:
metadata:
labels:
app: my-app
spec:
affinity:
podAntiAffinity:
preferredDuringSchedulingIgnoredDuringExecution:
- labelSelector:
matchExpressions:
- key: app
operator: In
values:
- my-app
topologyKey: "kubernetes.io/hostname"
containers:
- name: my-app
image: my-app:1.0.0
This schedules the Pods so that no two Pods of the Deployment are located on the same node, if possible.
For example, if you have 5 nodes and 3 replicas in the Deployment, then each Pod should be scheduled to a different node. If you have 5 nodes and 6 replicas, then the first 5 Pods should be scheduled to a different node each and the 6th Pod is scheduled to a node which already already has Pod (because there's no other possibility).
See more examples in the Kubernetes documentation.
Kubernetes having dedicated resource type called daemonset.This will ensure your pod is running on each node
kind: DaemonSet
metadata:
name: ssd-monitor
spec:
selector:
matchLabels:
app: ssd-monitor
template:
metadata:
labels:
app: ssd-monitor
spec:
containers:
- name: main
image: luksa/ssd-monitor
You can see 2 pods running on 2 nodes
[root#master ~]# kubectl get po -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
ssd-monitor-24qd7 1/1 Running 0 2m17s 10.36.0.7 node2.k8s <none> <none>
ssd-monitor-w7nxr 1/1 Running 0 2m17s 10.44.0.12 node1.k8s <none> <none>
Assume I have a cluster with 2 nodes and a POD with 2 replicas. Can I have the guarantee that my 2 replicas are deployed in 2 differents nodes. So that when a node is down, the application keeps running. By default does the scheduler work on best effort mode to assign the 2 replicas in distinct nodes?
Pod AntiAffinity
Pod anti-affinity can also to repel the pod from each other. so no two pods can be scheduled on same node.
Use following configurations.
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx
spec:
replicas: 2
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
affinity:
podAntiAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
- labelSelector:
matchExpressions:
- key: app
operator: In
values:
- nginx
topologyKey: "kubernetes.io/hostname"
containers:
- name: nginx
image: nginx
This will use the anti-affinity feature so if you are having more than 2 nodes the there will be guarantee that no two pod will be scheduled on same node.
You can use kind: DeamonSet . Here is a link to Kubernetes DeamonSet documentation.
A DaemonSet ensures that all (or some) Nodes run a copy of a Pod. As nodes are added to the cluster, Pods are added to them. As nodes are removed from the cluster, those Pods are garbage collected. Deleting a DaemonSet will clean up the Pods it created.
Here is a link to documentation about DeamonSets in OpenShift
Example might look like the following:
This is available on Openshift >= 3.2 version of openshift This use case is to run a specific docker container (veermuchandi/welcome) on all nodes (or a set nodes with specific label
Enable HostPorts expose on Openshift
$ oc edit scc restricted #as system:admin user
change allowHostPorts: true and save
apiVersion: extensions/v1beta1
kind: DaemonSet
metadata:
name: welcome
spec:
template:
metadata:
name: welcome
labels:
daemon: welcome
spec:
containers:
- name: c
image: veermuchandi/welcome
ports:
- containerPort: 8080
hostPort: 8080
name: serverport
$ oc create -f myDaemonset.yaml #with system:admin user
Source available here
Daemonset is not a good option. It will schedule one pod on every node. In future if you scale your cluster and then pods get scaled as many as nodes. Instead Use pod affinity to schedule no more than one pod on any node