running 2 pods vs 1 larger pod on Kubernetes? - kubernetes

What is the difference between running 2 pods (2 replicas) in Kubernetes vs a one larger pod ?
I have set a pod with 20m memory request limit. Is it better to have 2 replicas with 20m limits or a single pod with 40m memory request limit?

Personally, I think the performance had better to run multiple pods on the same host. I don't know what web server you use, but the requests are processed by limited cpu time, though it has multiple processes or threads for work. Additionally it's more efficient to utilize cpu time during network I/O waiting in using multiple processes. In order to improve the throughput, you should increase the processes or instances to work horizontally, because the response time is getting slower as time past.

Depends mainly on the requirements of the web/mobile application being hosted, which you can ascertain by benchmarking the app performance under 20m & 40m configurations. Overall, you can expect better performance for the application running at 40m and scaling elastically when required by user traffic. Running two pods in different data centers will give better fail-over performance in case of system crash or other issues. You may have higher billing rates running two pods when supporting the same rate of web traffic.

I think there is no golden rule on how to plan your infrastructure capacity to met specific level of your application/service`s objectives. You should start collecting some key performance metrics of your application, and based on these monitoring stats start doing proper dimensioning of your PODs, for which you can use Kubernetes features like Horizontal/Vertical Pod Autoscaling.

Related

Kubernetes - Multiple pods per node vs one pod per node

What is usually preferred in Kubernetes - having a one pod per node configuration, or multiple pods per node?
From a performance standpoint, what are the benefits of having multiple pods per node, if there is an overhead in having multiple pods living on the same node?
From a performance standpoint, wouldn't it be better to have a single pod per node?
The answer to your question is heavily dependent on your workload.
There are very specific scenarios (machine learning, big data, GPU intensive tasks) where you might have a one pod per node configuration due to an IO or hardware requirement for a singular pod. However, this is normally not a efficient use of resources and sort of eliminates a lot of the benefits of containerization.
The benefit of multiple pods per node is a more efficient use of all available resources. Generally speaking, managed kubernetes clusters will automatically schedule and manage the amount of pods that run on a node for you automatically, and many providers offer simple autoscaling solutions to ensure that you are always able to run all your workloads.
Running only a single pod per node has its cons as well. For example each node will need its own "support" pods such as metrics, logs, network agents and other system pods which most likely will not have its all resources
fully utilized. Which in terms of performance would mean that selecting the correct node size to pods amount ratio might result with less costs for the same performance as single pod per node.
On the contrary running too many pods in a massive node can cause lack of those resources and cause metrics or logs gaps or lost packets OOM errors etc.
Finally, when we also consider auto scaling, scaling up couple more pods on an existing nodes will be lot more responsive than scaling up a new node for each pod.

How to use CPU effectively with a large number of inactive pods in Kubernetes?

I have many services. In a day, a few services are busy for about ten hours, while most other services are idle or use a small amount of cpu.
In the past, I put all services in a virtual machine with two cpus, and scale by cpu usage, there are two virtual machine at the busiest time, but most of the time there is only one.
services
instances
busy time in a day
cpu when busy(core/service)
cpu when idle(core/service)
busy services
2
8~12 hours
0.5~1
0.1~0.5
busy services
2
8~12 hours
0.3~0.8
0.1~0.3
inactive services
30
0~1 hours
0.1~0.3
< 0.1
Now, I want to put them in kubernetes, each node has two CPUs, and use node autoscaling and HPA, in order to make the node autoscaling, I must set requests CPU for all services, which is exactly the difficulty I encountered.
This is my setting.
services
instances
busy time
requests cpu (cpu/service)
total requests cpu
busy services
2
8~12 hours
300m
600m
busy services
2
8~12 hours
300m
600m
inactive services
30
0~1 hours
100m
3000m
Note: The inactive service requests CPU is set to 100m because it will not work well if it is less than 100m when it is busy.
With this setting, the number of nodes will always be greater than three, which is too costly. I think the problem is that although these services require 100m of CPU to work properly, they are mostly idle.
I really hope that all services can autoscaling, I think this is the benefit of kubernetes, which can help me assign pods more flexibly. Is my idea wrong? Shouldn't I set a request CPU for an inactive service?
Even if I ignore inactive services. I find that kubernetes more often has more than two nodes. If I have more active services, even in off-peak hours, the requests CPU will exceed 2000m. Is there any solution?
I put all services in a virtual machine with two cpus, and scale by cpu usage, there are two virtual machine at the busiest time, but most of the time there is only one.
First, if you have any availability requirements, I would recommend to always have at least two nodes. If you have only one node and that one crash (e.g. hardware failure or kernel panic) it will take some minutes before this is detected and it will take some minutes before a new node is up.
The inactive service requests cpu is set to 100m because it will not work well if it is less than 100m when it is busy.
I think the problem is that although these services require 100m of cpu to work properly, they are mostly idle.
The CPU request is a guaranteed reserved resource amount. Here you reserve too much resources for your almost idling services. Set the CPU request lower, maybe as low as 20m or even 5m? But since these services will need more resources during busy periods, set a higher limit so that the container can "burst" and also use Horizontal Pod Autoscaler for these. When using the Horizontal Pod Autoscaler more replicas will be created and the traffic will be load balanced across all replicas. Also see Managing Resources for Containers.
This is also true for your "busy services", reserve less CPU resources and use Horizontal Pod Autoscaling more actively so that the traffic is spread to more nodes during high load, but can scale down and save cost when the traffic is low.
I really hope that all services can autoscaling, I think this is the benefit of kubernetes, which can help me assign pods more flexibly. Is my idea wrong?
Yes, I agree with you.
Shouldn't I set a request cpu for an inactive service?
It is a good practice to always set some value for request and limit, at least for a production environment. The scheduling and autoscaling will not work well without resource requests.
If I have more active services, even in off-peak hours, the requests cpu will exceed 2000m. Is there any solution?
In general, try to use lower resource requests and use Horizontal Pod Autoscaling more actively. This is true for both your "busy services" and your "inactive services".
I find that kubernetes more often has more than two nodes.
Yes, there are two aspects of this.
If you only use two nodes, your environment probably is small and the Kubernetes control plane probably consists of more nodes and is the majority of the cost. For very small environments, Kubernetes may be expensive and it would be more attractive to use e.g. a serverless alternative like Google Cloud Run
Second, for availability. It is good to have at least two nodes in case of an abrupt crash e.g. hardware failure or a kernel panic, so that your "service" is still available meanwhile the node autoscaler scales up a new node. This is also true for the number of replicas for a Deployment, if availability is important, use at least two replicas. When you e.g. drain a node for maintenance or node upgrade, the pods will be evicted - but not created on a different node first. The control plane will detect that the Deployment (technically ReplicaSet) has less than the desired number of replicas and create a new pod. But when a new Pod is created on a new node, the container image will first be pulled before the Pod is running. To avoid downtime during these events, use at least two replicas for your Deployment and Pod Topology Spread Constraints to make sure that those two replicas run on different nodes.
Note: You might run into the same problem as How to use K8S HPA and autoscaler when Pods normally need low CPU but periodically scale and that should be mitigated by an upcoming Kubernetes feature: KEP - Trimaran: Real Load Aware Scheduling

what are recommendation for pod size(CPU, memory) in kubernetes

I want to know the recommendation set for pod size. I.e. when to put application within pod or at what size it will be better to use machine itself in place of pod.
Ex. when to think of coming out of k8s and used as external service for some application, when pod required 8GB or 16GB or 32GB? Same for CPU intensive.
Because if pod required 16GB or 16 CPU and we have a machine/node of the same size then I think there is no sense of running pod on that machine. If we run in that scenario then it will be like we will having 10 pods and which required 8 Nodes.
Hopes you understand my concern.
So if some one have some recommendation for that then please share your thoughts on that. Some references will be more better.
Recommendation for ideal range:
size of pods in terms of RAM and CPU
Pods is to nodes ratio, i.e. number of pods per nodes
Whether good for stateless or stateful or both type of application or not
etc.
Running 16cpu/16gb pod on 16cpu/16gb machine is normal. Why not? You think of pods to be tiny but there is no such requirement. Pods can be gigantic, there is no issue with that. Remember container is just a process on a node, why you refuse to run a fat process on a fat node? Kubernetes adds very nice orchestration level to containers, why not make use of it?
There is no such thing as a universal or recommended pod size. Asking for recommended pod size is the same as asking for a recommended size for VM or bare metal server. It is totally up to your application. If your application requires 16 or 64 GB of RAM - this is the recommended size for you, you see?
Regarding pods to nodes ratio - current upper limit of Kubernetes is 110 pods per node. Everything below that watermark is fine. The only thing is that recommended master node size increases with total number of pods. If you have like 1000 pods - you go with small to medium size master nodes. If you have over 10 000 pods - you should increase your master nodes size.
Regarding statefulness - stateless application generally survive better. But often state also should be stored somewhere and stored reliably. So if you plan your application as a set of microservices - create as much stateless apps you can and as few stateful as you can. Ideally, only the relational databases should be truly stateful.

Kubernetes Cluster with different CPU configuration

I have created a K8S cluster of 10 machines. which is having cpus of different memory and cores (4 core 32 GB, 4 core 8 GB). Now when I am deploying any application on the cluster it is creating pods in a random manner. It is not creating the POD on the basis of memory or load.
How is Kubernetes master distributing the Pods in the cluster? I am not getting any significant answers. How can i configure the cluster for best use of resources?
Kubernetes uses a scheduler for deciding which pod is started on which node. One improvement is to tell the scheduler what your pods need as minimum and maximum resources.
Resources are Memory (measured in bytes), CPU (measured in cpu units) and ephemeral storage for things like emtpy dir(with 1.11). When you provide these information for your deployments Kubernetes can make better decisions where to run.
Without these information a nginx pod will be scheduled the same way as any heavy Java application.
The limits and requests config is described here. Setting both limits is a good idea to make scheduling easier and to avoid pods running amok and using all node resources.
If this is not enough there is also the possibility to add a custom scheduler which is explained in this documentation

Kubernetes - NodeUnderMemoryPressure Issue

I'm very new to Kubernetes. We are using Kubernetes cluster on Google Cloud Platform.
I have created Cluster, Services, Pod, Replica controllers.
I have created Horizontal Pod Autoscaler and it is based on CPU Params.
Cluster details
Default running node count is set to 3
3GB allocatable memory per node
Default running node count is 3 in the cluster.
After running for 1 hour Service and Nodes showing NodeUnderMemoryPressure Issues.
How to resolve this ??
If you any more details, please ask
Thanks
I don't know how much traffic is hitting your cluster, but I would highly recommend running Prometheus in your cluster.
Prometheus is an open-source monitoring and alerting tool, and integrates very well with Kubernetes.
This tool should give you a much better view of memory consumption, CPU usage, amongst many other monitoring capabilities, that will allow you to effectively troubleshoot these types of issues.
There are several ways to address this issue that depends on the type of your workloads.
The easiest is simply scale your nodes, but it can be useless if there is a memory leakage. Even if now you are not affected by it you should always consider the possibility of a memory leakage happening, therefore the best practise is to introduce always memory limits for PODs and Namespaces.
Scale the cluster
if you have many pods running and there are not some of them way bigger that the others it would be useful to scale horizontally your cluster, in this way the number of running pods per nodes will reduce and the NodeUnderMemoryPressure warning should disappear.
if you are running few PODs or some of them are capable to make the cluster suffering alone, then the only option is to scale the nodes vertically adding a new node pool with Compute Engine instances having more memory and possibly delete the old one.
if your workload is correct and you memory suffer because in certain moment of the day you receive 100 times more the usual traffic and you create more pods to support this traffic, you should consider to make use of the Autoscaler.
Check Memory leakages
On the other hand if it is not an "healthy" situation and you have pods consuming way more RAM than expected then you should follow the advice of grizzthedj and understand why your PODs are consuming so much and maybe verify if some of your container is affected by memory leakage and in this case scale the amount of RAM is useless since at some point you will run out of it anyway.
Therefore start to understand which are the PODs consuming too much and then troubleshoot why they have this behaviour, if you do not want to make use of Prometeus simply SSH into the container and check with the classical Linux commands.
Limit the RAM consumed by PODs
To prevent this to happen in the future I advise you when writing YAML file to always limit the amount of RAM they can make use of, in this way you will control them and you will be sure that there is not the risk that they cause the Kubernetes "node agent" to fail because out of memory.
Consider also to limit the CPU and introduce minimum requirements of both RAM and CPU for PODs to help the scheduler to properly schedule the PODs to avoid to hit NodeUnderMemoryPressure under high workload.