I'm trying to figure out the pieces and how to fit them together for having a pod be able to control aspects of a deployment, like scaling. I'm thinking I need to set up a service account for it, but I'm not finding the information on how to link it all together, and then how to get the pod to use the service account. I'll be writing this in python, which might add to the complexity of how to use the service account
Try to set up Horizontal Pod Autpscaler.
The Horizontal Pod Autoscaler automatically scales the number of pods in a replication controller, deployment, replica set or stateful set based on observed CPU utilization (or, with custom metrics support, on some other application-provided metrics). Horizontal Pod Autoscaling does not apply to objects that can’t be scaled, for example, DaemonSets.
The Horizontal Pod Autoscaler is implemented as a Kubernetes API resource and a controller. The resource determines the behavior of the controller. The controller periodically adjusts the number of replicas in a replication controller or deployment to match the observed average CPU utilization to the target specified by user.
Documentations: hpa-setup, autoscaling.
Related
(I'm new to Kubernetes and not sure this is best practice)
I have a pipeline of jobs in a Firestore database that need to be completed as quickly as possible.
I want to set up a Kubernetes cluster (on GKE) that will scale up when there is a large backlog of tasks to complete. Each pod/node needs a single GPU to complete the task.
Is it possible to use a cloud function to manually scale the number of nodes depending on the number of jobs in the pipeline?
I was planning on using the clusters.nodePools.setSize() function from the GKE client library but I'm not sure if this is just intended for initial cluster setup rather than manual scaling.
Thanks
https://cloud.google.com/kubernetes-engine/docs/reference/rest/v1beta1/projects.locations.clusters.nodePools/setSize
You can configure and use Horizontal pod scaling on your cluster to scale the number of Pods .
As suggested by #somethingsomething refer these links on Horizontal Pod Autoscaler and Cluster autoscaler :
The Horizontal Pod Autoscaler changes the shape of your Kubernetes workload by automatically increasing or decreasing the number of Pods in response to the workload's CPU or memory consumption, or in response to custom metrics reported from within Kubernetes or external metrics from sources outside of your cluster.
Horizontal Pod autoscaling helps to ensure that your workload functions consistently in different situations, and allows you to control costs by only paying for extra capacity when you need it.
It's not always easy to predict the indicators that show whether your workload is under-resourced or under-utilized. The Horizontal Pod Autoscaler can automatically scale the number of Pods in your workload based on one or more metrics.
I want to know if there is a way to use autoscalers in Kubernetes with pods directly created from a "pod creation yaml files" not the pods created as part of a higher-level controller like deployments or replicasets?
The short answer to your question is no.
Horizontal Pod Autoscaler changes the number of replicas of a Deployment reacting to changes in load utilization. So you need a Deployment for it to work.
Regarding Vertical Pod Autoscaler, I think it should work with spare pods as well, but only at Pod creation time. In fact, I read the following statement in the Known limitations section of the README:
VPA does not evict pods which are not run under a controller. For such
pods Auto mode is currently equivalent to Initial.
That sentence make me conclude that VPA should work on Pods not backed by a Controller, but in a limited way. In fact, the documentation about Initial mode states:
VPA only assigns resource requests on pod creation and never changes
them later.
Making it basically useless.
I think it is not possible to use Pod object as the target resource for an HPA.
The document describes HPA as:
The Horizontal Pod Autoscaler automatically scales the number of Pods
in a replication controller, deployment, replica set or stateful set
based on observed CPU utilization (or, with custom metrics support, on
some other application-provided metrics). Note that Horizontal Pod
Autoscaling does not apply to objects that can't be scaled, for
example, DaemonSets.
The document also described how the algorithm is implemented at the backend as:
desiredReplicas = ceil[currentReplicas * ( currentMetricValue / desiredMetricValue )]
and since the Pod resource does not have the replicas field as part of its spec therefore we can say that the same is not supported for auto scaling using the HPA.
Although it seems the VPA does support working with Pod object but there is a limitation when using VPA just with Pods.
VPA does not evict pods which are not run under a controller. For such
pods Auto mode is currently equivalent to Initial.
You can read about the different updatePolicy.updateModes in the docs.
Some context: I have multiple cron jobs running daily, weekly, hourly and some of which require significant processing power.
I would like to add requests and limitations to these container cron pods to try and enable vertical scaling and ensure that the assigned node will have enough capacity when being initialized. This will prevent me from having to have multiple large node available at all times and also letting me modify how many crons I can run in parallel easily.
I would like to try and avoid timed scaling since the cron jobs processing time can increase as the application grows.
Edit - Additional Information :
Currently I am using Digital Ocean and utilizing it's UI for cluster autoscaling. I have it working with HPA's on deployments but not crons. Adding limits to crons does not trigger cluster autoscaling to my knowledge.
I have tried to enable HPA scaling with the cron but with no success. Basically it just sits on a pending status signalling that there is insufficient CPU available and does not generate a new node.
Does HPA scaling work with cron job pods and is there a way to achieve the same type of scaling?
HPA is used to scale more pods when pod loads are high, but this won't increase the resources on your cluster.
I think you're looking for cluster autoscaler (works on AWS, GKE and Azure) and will increase cluster capacity when pods can't be scheduled.
This is a Community Wiki answer so feel free to edit it and add any additional details you consider important.
As Dom already mentioned "this won't increase the resources on your cluster." Saying more specifically, it won't create an additional node as Horizontal Pod Autoscaler doesn't have such capability and in fact it has nothing to do with cluster scaling. It's name is pretty self-exlpanatory. HPA is able only to scale Pods and it scales them horizontaly, in other words it can automatically increase or decrease number of replicas of your "replication controller, deployment, replica set or stateful set based on observed CPU utilization (or, with custom metrics support, on some other application-provided metrics)" as per the docs.
As to cluster autoscaling, as already said by Dom, such solutions are implemented in so called managed kubernetes solutions such as GKE on GCP, EKS on AWS or AKS on Azure and many more. You typically don't need to do anything to enable them as they are available out of the box.
You may wonder how HPA and CA fit together. It's really well explained in FAQ section of the Cluster Autoscaler project:
How does Horizontal Pod Autoscaler work with Cluster Autoscaler?
Horizontal Pod Autoscaler changes the deployment's or replicaset's
number of replicas based on the current CPU load. If the load
increases, HPA will create new replicas, for which there may or may
not be enough space in the cluster. If there are not enough resources,
CA will try to bring up some nodes, so that the HPA-created pods have
a place to run. If the load decreases, HPA will stop some of the
replicas. As a result, some nodes may become underutilized or
completely empty, and then CA will terminate such unneeded nodes.
Background
I have app running in kubernetes cluster using sharded mongodb and elasticsearch statefulsets. I setup horizontal pod autoscalers for deployment components in my app and everything works well.
Problems
Problems arise when the traffic goes up. My server deployment scales out just fine, but mongodb shards and elasticsearch nodes cannot handle this much traffic and throttle overall response time.
Simple solution is to configure those statefulset with more shards, more replicas. What bugs me is that traffic spike happens like 3-4 hours a day, thus it's kinda wasteful to let all those boys sitting idly for the rest of the day.
I did some research and looks like database in general is not supposed to scale out/in dynamically as it will consume a lot of network and disk io just to do replication between them. Also there is potential of data loss and inconsistency during scaling up, scaling down.
Questions
If possible, what is proper way to handle dynamic scaling in mongodb, elasticsearch... and database in general?
If not, what can I do to save some cents off my cloud bill as we only need the maximum power from database pods for a short period per day.
You should read about Kubernetes autoscaling - HPA.
The Horizontal Pod Autoscaler automatically scales the number of pods in a replication controller, deployment, replica set or stateful set based on observed CPU utilization (or, with custom metrics support, on some other application-provided metrics). Note that Horizontal Pod Autoscaling does not apply to objects that can't be scaled, for example, DaemonSets.
The Horizontal Pod Autoscaler is implemented as a Kubernetes API resource and a controller. The resource determines the behavior of the controller. The controller periodically adjusts the number of replicas in a replication controller or deployment to match the observed average CPU utilization to the target specified by user.
With HPA you should have to also take care about the volume mounting and data latency.
As #Serge mentioned in comments, I would suggest to check the native scaling cluster option provided by the MongoDB and Elasticsearch itself.
Take a look at
MongoDB operator documentation
Elasticsearch operator documentation
Elasticsearch future release autoscaling
I am not very familiar with MongoDB and Elasticsearch with Kubernetes, but maybe those tutorials help you:
https://medium.com/faun/scaling-mongodb-on-kubernetes-32e446c16b82
https://www.youtube.com/watch?v=J7h0F34iBx0
https://kubernetes.io/blog/2017/01/running-mongodb-on-kubernetes-with-statefulsets/
https://sematext.com/blog/elasticsearch-operator-on-kubernetes/#toc-what-is-the-elasticsearch-operator-1
If you use helm take a look at banzaicloud Horizontal Pod Autoscaler operator
You may not want nor can edit a Helm chart just to add an autoscaling feature. Nearly all charts supports custom annotations so we believe that it would be a good idea to be able to setup autoscaling just by adding some simple annotations to your deployment.
We have open sourced a Horizontal Pod Autoscaler operator. This operator watches for your Deployment or StatefulSet and automatically creates an HorizontalPodAutoscaler resource, should you provide the correct autoscale annotations.
Hope you find this useful.
I think I have a pretty simple scenario: I need to auto-scale on Google Kubernetes Engine with a pod that runs one per node and uses all available remaining resources on the node.
"Remaining" resources means that there are certain basic pod services running on each node such logging and metrics, which need their requested resources. But everything left should go to this particular pod, which is in fact the main web service for my cluster.
Also, these remaining resources should be available when the pod's container starts up, rather than through vertical autoscaling with pod restarts. The reason is that the container has certain constraints that make restarts sort of expensive: heavy disk caching, and issues with licensing of some 3rd party software I use. So although certainly the container/pod is restartable, I'd like to avoid except for rolling updates.
The cluster should scale nodes when CPU utilization gets too high (say, 70%). And I don't mean requested CPU utilization of a node's pods, but rather the actual utilization, which is mainly determined by the web service's load.
How should I configure the cluster for this scenario? I've seen there's cluster auto scaling, vertical pod autoscaling, and horizontal pod autoscaling. There's also Deployment vs DaemonSet, although it does not seem that DaemonSet is designed for pods that need to scale. So I think Deployment may be necessary, but in a way that limits one web service pod per node (pod anti affinity??).
How do I put all this together?
You could set up a Deployment with a resource request that equals a single node's allocatable resources (i.e., total resources minus auxiliary services as you mentioned). Then configure Horizontal Pod Autoscaling to scale up your deployment when CPU request utilization goes above 70%; this should do the trick as in this case request utilization rate is essentially the same as total node resource utilization rate, right? However if you do want to base scaling on actual node CPU utilization, there's always scaling by external metrics.
Technically the Deployment's resource request doesn't have to exactly equal remaining resources; rather it's enough for the request to be large enough to prevent two pods being ran on the same node. As long as that's the case and there's no resource limits, the pod ends up consuming all the available node resources.
Finally configure cluster autoscaling on your GKE node pool and we should be good to go. Vertical Pod Autoscaling doesn't really come into play here as pod resource request stays constant, and DaemonSets aren't applicable as they're not scalable via HPA as mentioned.