I have a Kubernetes deployment that has 3 replicas. It starts 3 pods which are distributed across a given cluster. I would like to know how to reliably get one pod to contact another pod within the same ReplicaSet.
The deployment above is already wrapped up in a Kubernetes Service. But Services do not cover my use case. I need each instance of my container (each Pod) to start-up a local in memory cache and have these cache communicate/sync with other cache instances running on other Pods. This is how I see a simple distributed cache working on for my service. Pod to pod communication within the same cluster is allowed as per the Kubernetes Network Model but I cannot see a reliable way to address each a pod from another pod.
I believe I can use a StatefulSet, however, I don't want to lose the ClusterIP assigned to the service which is required by Ingress for load balancing.
Ofcourse you can use statefulset, and ingress doesn't need ClusterIP that assigned to the service, since it uses the endpoints, so 'headless service' is ok.
Related
As per the Headless service definition:
Kubernetes allows clients to discover pod IPs through DNS lookups. Usually, when you perform a DNS lookup for a service, the DNS server returns a single IP which is the service’s cluster IP. But if you don’t need the cluster IP for your service, you can set ClusterIP to None , then the DNS server will return the individual pod IPs instead of the service IP.Then client can connect to any of them
Looks like its similar to creating a clusterIP with one backend pod. If so why should we use clusterIP with one backend pod?
You don't usually control the Pod's name, and you can't guarantee that there will always be exactly one matching Pod.
There are a couple of standard reasons to not directly create Pods, but instead to rely on higher-level constructs. A Deployment can do zero-downtime upgrades; a StatefulSet manages associated PersistentVolumeClaims; a Job can retry on failure. A Pod also can't be edited once created, so you'd need to delete and recreate the Pod on any update. If you're using a Deployment you'll have Pod names like deployment-name-12345678-abcde, but you don't control the ending parts at all. A Service will give you a consistent name.
During an upgrade there won't necessarily be exactly the requested number of replicas:. A Deployment by default will create an additional Pod, wait for it to pass its readiness checks, then tear down the old Pod; an associated Service will route traffic correctly in this case. The alternative is to tear down the old Pod first. In both cases, you could have zero or two matching Pods and not just one.
Having the Service as the standard pattern also helps if you do ever decide to increase the replica count; for example you might choose to run multiple replicas just for additional resiliency in case ons of the Nodes fails. Communicating via a Service works exactly the same way whether you have just one Pod or several.
There is a difference. A DNS lookup for Headless service returns as many IP addresses as there are pods running, while a DNS lookup for ClusterIP service always returns exactly one IP address.
So yes, if you ever have only one Pod then Headless service may be sufficient.
But if you have more than one pod, for Headless service the client must implement some kind of load balancing while for ClusterIP kubernetes will do the load balancing.
One use case for Headless service with multiple pods is when you want to send requests to every pod, e.g. local cache draining.
How do I be able to go to a specific Pod in a DaemonSet without hostNetwork? The reason is my Pods in the DaemonSet are stateful, and I prefer to have at most one worker on each Node (that's why I used DaemonSet).
My original implementation was to use hostNetwork so the worker Pods can be found by Node IP by outside clients. But in many production environment hostNetwork is disabled, so we have to create one NodePort service for each Pod of the DaemonSet. This is not flexible and obviously cannot work in the long run.
Some more background on how my application is stateful
The application works in an HDFS-taste, where Workers(datanodes) register with Masters(namenodes) with their hostname. The masters and outside clients need to go to a specific worker for what it's hosting.
hostNetwork is an optional setting and is not necessary. You can connect to your pods without specifying it.
To communicate with pods in DaemonSet you can specify hostPort in the DaemonSet’s pod spec to expose it on the node. You can then communicate with it directly by using the IP of the node it is running on.
Another approach to connect to stateful application is StatefulSet. It allows you to specify network identifiers. However it requires headless service for network identity of the Pods and you are responsible for creating such services.
I want to deploy a single Pod on a Node to host my service (like GitLab for the example). The problem is : a Pod will not be re-created after the Node failure (like a reboot). The solution(s) : Use a StatefulSet, ReplicaSet or DaemonSet to ensure the Pod creation after a Node failure. But what is the best for this case ?
This Pod is stateful (I am using volume hostPath to keep the data) and is deployed using nodeSelector to keep it always on the same Node.
Here is a simple YAML file for the example : https://pastebin.com/WNDYTqSG
It creates 3 Pods (one for each Set) with a volume to keep the data statefully. In practice, all of these solutions can feet my needs, but I don't know if there are best practices for this case.
Can you help me to choose between these solutions to deploy a single stateful Pod please ?
Deployment is the most common option to manage a Pod or set of Pods. These are normally used instead of ReplicaSets as they are more flexible and creating a Deployment results in a ReplicaSet - see https://www.mirantis.com/blog/kubernetes-replication-controller-replica-set-and-deployments-understanding-replication-options/
You would only need a StatefulSet if you had multiple Pods and needed dedicated persistence per Pod or you had multiple Pods and the Pods need individual names because they relate to each other (e.g. one is a leader) - https://stackoverflow.com/a/48006210/9705485
A DaemonSet would be used when you want one Pod/replica per Node
In Kelsey Hightower's Kubernetes Up and Running, he gives two commands :
kubectl get daemonSets --namespace=kube-system kube-proxy
and
kubectl get deployments --namespace=kube-system kube-dns
Why does one use daemonSets and the other deployments?
And what's the difference?
Kubernetes deployments manage stateless services running on your cluster (as opposed to for example StatefulSets which manage stateful services). Their purpose is to keep a set of identical pods running and upgrade them in a controlled way. For example, you define how many replicas(pods) of your app you want to run in the deployment definition and kubernetes will make that many replicas of your application spread over nodes. If you say 5 replica's over 3 nodes, then some nodes will have more than one replica of your app running.
DaemonSets manage groups of replicated Pods. However, DaemonSets attempt to adhere to a one-Pod-per-node model, either across the entire cluster or a subset of nodes. A Daemonset will not run more than one replica per node. Another advantage of using a Daemonset is that, if you add a node to the cluster, then the Daemonset will automatically spawn a pod on that node, which a deployment will not do.
DaemonSets are useful for deploying ongoing background tasks that you need to run on all or certain nodes, and which do not require user intervention. Examples of such tasks include storage daemons like ceph, log collection daemons like fluentd, and node monitoring daemons like collectd
Lets take the example you mentioned in your question: why iskube-dns a deployment andkube-proxy a daemonset?
The reason behind that is that kube-proxy is needed on every node in the cluster to run IP tables, so that every node can access every pod no matter on which node it resides. Hence, when we make kube-proxy a daemonset and another node is added to the cluster at a later time, kube-proxy is automatically spawned on that node.
Kube-dns responsibility is to discover a service IP using its name and only one replica of kube-dns is enough to resolve the service name to its IP. Hence we make kube-dns a deployment, because we don't need kube-dns on every node.
I am trying to understand k8s and helm.
When I create a helm chart, there are 2 files: service.yaml and deployment.yaml. Both of them have a name field.
If I understand correctly, the deployment will be responsible for managing the pods, replicasets, etc and thus the service.
Basically, why am I allowed use a separate name for the service and for the deployment? Under what scenario would we want these 2 names to differ? Can a deployment have more than 1 service?
The "service" creates a persistent IP address in your cluster which is how everything else connects it. The Deployment creates a ReplicaSet, which creates a Pod, and this Pod is the backend for that service. There can be more than 1 pod, in which case the service load balances, and these pods can change over time, change IP's, but your service remains constant.
Think of the service as a load balancer which points to your pods. It's analogous to interfaces and implementations. The service is like an interface, which is backed by the pods, the impementations.
The mapping is m:n. You can have multiple services backed by a single pod, or multiple pods backing a single service.