I have test environment where HA is not important but rather resources efficiency, so would you recommend in that regard to create one Pod with multiple containers where it make sense of course, where containers are tight coupled or to have one Pod for every service? Does this have any impact on resources at all?
I will give an example if for instance I have php application, and then nginx proxy and then filebeat service that is listening logs, what would be better to have 3 pods for this 3 things or one pod with 3 containers. And when I say better I mean to use less memory, cpu, etc.
The difference between both solutions should not be significant (negligible ?).
However, depending on your approach, the management effort might be quite significantly different.
With one component in a dedicated pod you need to somehow synchronise live cycle of all pods (php + nginx + filebeat) whenever you spin just one, new application up.
With all of them in one pod you just need to create/delete one pod.
Related
What's the benefit of having multiple containers in a pod versus having standalone containers?
If you have multiple containers in the same pod, they can speak to each other as localhost and can share mounted volumes.
If you have multiple pods of one container each, you can restart one without restarting the other. Assuming they're controlled by deployments, you can add additional replicas of one without necessarily scaling the other. If the version or some other characteristic of one of them changes, you're not forced to restart the other. You'd need to set up a service to talk from one to the other, and they can't communicate via a filesystem.
The general approach I've always seen is to always have one container per pod within a deployment, unless you have a specific reason to need an additional container. Usually this is some kind of special-purpose "sidecar" that talks to a credentials service, or manages logging, or runs a network proxy, or something else that's secondary to the main thing the pod does (and isn't a separate service in its own right).
Apart from the points pointed out , the CPU and Memory(under technical preview) are associated with a POD so if we have a single container in a POD it is easy to understand and implement the application resourcerequirement inside the POD with more than one container inside the POD we could face issues/challenges when we want to do a horizontal scale
Secondly the deployments (Blue/Green,Canary,A/B) are also more aligned with the approach of single container/POD
From the Kubernetes documentation
A Pod might encapsulate an application composed of multiple co-located containers that are tightly coupled and need to share resources. These co-located containers might form a single cohesive unit of service–one container serving files from a shared volume to the public, while a separate “sidecar” container refreshes or updates those files. The Pod wraps these containers and storage resources together as a single manageable entity.
I am trying to deploy multiple pods in k8s like say MySQL, Mango, Redis etc
Can i create a single deployment resource for this and have multiple containers defined in template section? Is this allowed? If so, how will replication behave in this case?
Thanks
Pavan
I am trying to deploy multiple pods in k8s like say MySQL, Mango,
Redis etc
From microservices architecture perspective it is actually quite a bad idea to place all those containers in a single Pod. Keep in mind that a Pod is a smallest deployable unit that can be created and managed by Kubernetes. There are quite many good reasons you don't want to have all above mentioned services in a single Pod. Difficulties in scaling such solution is just one of them.
Can i create a single deployment resource for this and have multiple
containers defined in template section? Is this allowed? If so, how
will replication behave in this case?
No, it is not allowed in Kubernetes. As to Deployments and StatefulSets, (which you need for statefull applications such as databases) both manage Pods that are based on identical container spec so it is not possible to have a Deployment or StatefulSet consisting of different types of Pods, based on different specs.
To sum up:
Many Deployments and StatefulSets objects, serving for different purposes are the right solution.
A deployment can have multiple containers inside of it.
Generaly it's used to have one master container for the app and some sidecar container that are needed for the app. I don't have an example right now.
Still it's a best practice to split deployments for scalling purpose, your front may need to scale more than the back depending on cache and you may not want to have pods too big. For cahing purpose like redis it's better to have a cluster on the side as each time a pod start or stop, you will loose data.
It's common having multiple containers per Pod in order to share namespaces and volumes between them: take as example the Ambassador pattern that is used to present the application to outside adding a layer for the authentication, making it totally transparent to the main app.
Other examples using the sidecar pattern consist of log parsers or configurators that hot reload credentials without the main app to worry about it.
That's the theory, according to your needs you have to use one deployment per component, so a Deployment for your app, a StatefulSet for the DB and so on. Keep in mind to use a container per process and a Kubernetes resource per backing service.
I'm trying to build a web app where each user gets their own instance of the app, running in its own container. I'm new to kubernetes so I'm probably not understanding something correctly.
I will have a few physical servers to use, which in kubernetes as I understand are called nodes. For each node, there is a limitation of 100 pods. So if I am building the app so that each user gets their own pod, will I be limited to 100 users per physical server? (If I have 10 servers, I can only have 500 users?) I suppose I could run multiple VMs that act as nodes on each physical server but doesn't that defeat the purpose of containerization?
The main issue in having too many pods in a node is because it will degrade the node performance and makes is slower(and sometimes unreliable) to manage the containers, each pod is managed individually, increasing the amount will take more time and more resources.
When you create a POD, the runtime need to keep a constant track, doing probes (readiness and Liveness), monitoring, Routing rules many other small bits that adds up to the load in the node.
Containers also requires processor time to run properly, even though you can allocate fractions of a CPU, adding too many containers\pod will increase the context switch and degrade the performance when the PODs are consuming their quota.
Each platform provider also set their own limits to provide a good quality of service and SLAs, overloading the nodes is also a risk, because a node is a single point of failure, and any fault in high density nodes might have a huge impact in the cluster and applications.
You should either consider:
Smaller nodes and add more nodes to the cluster or
Use Actors instead, where each client will be one Actor. And many actor will be running in a single container. To make it more balanced around the cluster, you partition the actors into multiple containers instances.
Regarding the limits, this thread has a good discussion about the concerns
Because of the hard limit if you have 10 servers you're limited to 1000 pods.
You might want to count also control plane pods in your 1000 available pods. Usually located in the namespace kube-system it can include (but is not limited to) :
node log exporters (1 per node)
metrics exporters
kube proxy (usually 1 per node)
kubernetes dashboard
DNS (scaling according to the number of nodes)
controllers like certmanager
A pretty good rule of thumb could be 80-90 application pods per node, so 10 nodes will be able to handle 800-900 clients considering you don't have any other big deployment on those nodes.
If you're using containers in order to gain perfs, creating node VMs will be against your goal. But if you're using containers as a way to deploy coherent environments and scale stateless applications then using VMs as node can make sense.
There are no magic rules and your context will dictate what to do.
As managing a virtualization cluster and a kubernetes cluster may skyrocket your infrastructure complexity, maybe kubernetes is not the most efficient tool to manage your workload.
You may also want to take a look at Nomad wich does not seem to have those kind of limitations and may provide features that are closer to your needs.
As I have been using kubernetes more I keep on seeing the reference that a pod can contain 1 container or more and I have even looked at examples.
My question is whether there is a case where this would be best practice and more efficient to create multi container pods since you can scale and replicate your pods coupling it with a service.
Thanks in advance
A Pod can contain multiple containers, but for the most portion of the situations, it makes perfect sense for the Pod to be simply an abstraction over a single running container.
In what situations does it make sense to have a multi-container deployed Pod?
What comes to my mind are the scenarios where you have a primary Pod running, but you need to tightly couple helper processes, such as a log watcher. In those situations, it makes perfect sense to actually have multiple containers running inside a single pod.
Another big example that comes to my mind is from the Istio project, which is a platform made to connect, manage and secure microservices and is generally referred as a Service Mesh.
A huge part of what it does and is able to accomplish to provide a greater control and customization over the deployed microservices network, is due to the fact that it deploys a sidecar proxy, denominated Envoy, throughout the environment intercepting all network communication between microservices.
Here, you can check an example of load balancing in a Istio service mesh. As you can see the Proxy is deployed inside the Pod, intercepting all communication that goes through it.
If I have 10 different services, each of which are independent from each other and run from their own container, can I get kubernetes to run all of those services on, say, 1 host?
This is unclear in the kubernetes documentation. It states that you can force it to schedule containers from the same pod onto one host, using a "multi-container pod", but it doesn't seem to approach the subject of whether you can have multiple pods running on one host.
In fact kubernetes will do exactly what you want by default. It is capable of running dozens if not hundreds of containers on a single host (depending on its specs).
If you want very advanced control over scheduling pods, there is an alpha feature for that, which introduces concept of node/pod (anti)affinities. But I would say it is a rather advanced k8s topic at the moment, so you are probably good with what is in stable/beta for most use cases.
Honorable mention: there is a nasty trick that allows you to control when pods can not be collocated on the same node. An that is when they both declare same hostPort in their ports section. It can be usefull for some cases, but be aware it affects ie. how rolling deployments happen in some situations.
You can use node selectors and assign the same node for each of the pod to the same node / host
http://kubernetes.io/docs/user-guide/node-selection/
Having said that, the whole point to Kubernetes is to manage a cluster where you can deploy apps / pods across them.