I'm using Istio at the moment combined with the cert-manager. Because I need to have multiple certificates I'm using SDS instead of the volume mount approach.
But the hardware requirements for this stuff are really high. For GKE it is recommended to use a node-pool of 4x n1-standard-2 machines. This sums up to 200$ per month just for Istio. The recommendation for EKS is 2x m5.large machines. So it is a little bit cheaper but still around 150$. What confuses me is, that Minikube "just" needs 4vCPUs and 16GB memory in total which is round about the half of the requirements for GKE and EKS.
You'll see the resource hungry components by looking at the istio-system namespace, I mean especially the limits. For me it is:
istio-telemetry > 1100m / 6800m (requested / limits)
istio-policys (I have 5 of them) > 110m / 2000m
My question is:
Did you manage to reduce the limits without facing issues in production?
What node-pool size / machine type are your running your Istio plane?
Did someone tried auto-scaling for this node-pool? Did it reduce the costs?
Kind regards from Berlin.
Managed Istio for GKE is offered by Google as a pre-configured bundle. 4x n1-standard-2 is recommended to provide enough resources for all Istio components being installed.
Downsizing a cluster below the recommended size does not make sense.
Installation of managed Istio onto a standard GKE cluster (3x n1-standard-1)
will fail due to lack of resources. Besides that you wouldn't have
free computing capacity for your workloads. Recommended cluster size
seems reasonable.
Apart from recommended hardware configuration (4x n1-standard-2),
managed Istio can be installed and running on a cluster with configuration
8x n1-standard-1.
Taking into account mentioned in the point ##1, autoscaling could be beneficial
mostly for volatile workloads, but won't help that much for saving resources
allocated for Istio.
If the managed Istio for GKE seemed too resource consuming, you could install original version of Istio and select an installation profile with the components you actually need, as described here:
Customizable Install with Helm
Related
I'm trying to use the free tier (autopilot mode) to learn k8s on gcp. However I cam across the following Is it possible to have Google Cloud Kubernetes cluster in the free tier?. However when I checked the link given in the question I could not find the specified limitation f1-micro machines are not supported due to insufficient memory. Is this still valid ? can I use k8s on gcp in the free tier without incurring any cost?
There is no way to get a free GKE cluster on GCP, but you can get a very cheap one by following the instructions at
https://github.com/Neutrollized/free-tier-gke.
Using a combination of GKE's free management tier and a low cost machine type, the cost estimate is less than $5 per month: .
More details on what is available as part of the free tier can be found here: https://cloud.google.com/free.
Also, for your question regarding limitation of f1-micro to be used in GKE,if you follow the documentation limitation
It is written that- Minimum CPU platform cannot be used with shared core machine types. Now since f1-micro machines are shared core machine types. So it is valid and cannot be used.
As described in the documentation, there is no management cost for 1 Autopilot or 1 GKE standard zonal mode.
You don't pay for the control plane. But then you have to pay for your workload measured in seconds for autopilot (pod level) and seconds for GKE standard (node level (compute engine))
For study purpose, I believe its better to have classic (Standard) GKE cluster rather than autopilot, where you have less managing options.
When its come to pricing, using Preemptible nodes in GKE cluster is a better option where pricing is really low.
you can enable this selecting as below
Preemptible nodes
Hi If you want to learn the K8s on GCP you can try using qwiklabs where you get some initial credits by which you can access some of the labs where you can learn and practice your activities. https://go.qwiklabs.com/qwiklabs-free
I want to deploy many softwares in a kubernetes cluster. I have information like the configuration of each software like the number of pods, request and limits of cpu and RAM for each software.
My requirement is all the softwares should be provisioned successfully or none of them should be provisioned even if one software fails. Failure can be because there are no enough resources in the kubernetes cluster
How do I check if my cluster has sufficient resources to provision all the softwares even before actual deployment of the softwares
AFAIK kubernetes does not support deploying either all or none application.
I think you have to do the maths by yourself.
You said, every information you need is there (Requirements for all the Services).
This should help you planning your clusters dimensions.
Know you should calculate this on node basis. Lets say, you need 16GB Memory. Your nodes bring 8gb per Machine. Your Cluster should provide at least 24GB (3 Nodes) Memory for your application (beside monitoring tools etc.).
Always calculate something on top, because OS and Monitoring-Tools will take a little bit of your nodes resource.
Is there a maximum number of namespaces supported by a Kubernetes cluster? My team is designing a system to run user workloads via K8s and we are considering using one namespace per user to offer logical segmentation in the cluster, but we don't want to hit a ceiling with the number of users who can use our service.
We are using Amazon's EKS managed Kubernetes service and Kubernetes v1.11.
This is quite difficult to answer which has dependency on a lot of factors, Here are some facts which were created on the k8s 1.7 cluster kubernetes-theresholds the Number of namespaces (ns) are 10000 with few assumtions
The are no limits from the code point of view because is just a Go type that gets instantiated as a variable.
In addition to link that #SureshVishnoi posted, the limits will depend on your setup but some of the factors that can contribute to how your namespaces (and resources in a cluster) scale can be:
Physical or VM hardware size where your masters are running
Unfortunately, EKS doesn't provide that yet (it's a managed service after all)
The number of nodes your cluster is handling.
The number of pods in each namespace
The number of overall K8s resources (deployments, secrets, service accounts, etc)
The hardware size of your etcd database.
Storage: how many resources can you persist.
Raw performance: how much memory and CPU you have.
The network connectivity between your master components and etcd store if they are on different nodes.
If they are on the same nodes then you are bound by the server's memory, CPU and storage.
There is no limit on number of namespaces. You can create as many as you want. It doesn't actually consume cluster resources like cpu, memory etc.
I tried to launch Istio on Google Kubernetes Engine using the Google Cloud Deployment Manager as described in the Istio Quick Start Guide.
My goal is to have a cluster as small as possible for a few very lightweight microservices.
Unfortunately, Istio pods in the cluster failed to boot up correctly when using a
1 node GKE
g1-small or
n1-standard-1
cluster.
For example, istio-pilot fails and the status is "0 of 1 updated replicas available - Unschedulable".
I did not find any hints that the resources of my cluster are exceeded so I am wondering:
What is the minimum GKE cluster size to successfully run Istio (and a few lightweight microservices)?
What I found is the issue Istio#216 but it did not contain the answer. Also, of course, the cluster size depends on the microservices but I am basically interested in the minimum cluster to start with.
As per this page
If you use GKE, please ensure your cluster has at least 4 standard GKE nodes. If you use Minikube, please ensure you have at least 4GB RAM.
We have some Kubernetes clusters that have been deployed using kops in AWS.
We really like using the upstream/official images.
We have been wondering whether or not there was a good way to monitor the systems without installing software directly on the hosts? Are there docker containers that can extract the information from the host? I think that we are likely concerned with:
Disk space (this seems to be passed through to docker via df
Host CPU utilization
Host memory utilization
Is this host/node level information already available through heapster?
Not really a question about kops, but a question about operating Kubernetes. kops stops at the point of having a functional k8s cluster. You have networking, DNS, and nodes have joined the cluster. From there your world is your oyster.
There are many different options for monitoring with k8s. If you are a small team I usually recommend offloading monitoring and logging to a provider.
If you are a larger team or have more specific needs then you can look at such options as Prometheus and others. Poke around in the https://github.com/kubernetes/charts repository, as I know there is a Prometheus chart there.
As with any deployment of any form of infrastructure you are going to need Logging, Monitoring, and Metrics. Also, do not forget to monitor the monitoring ;)
I am using https://prometheus.io/, it goes naturally with kubernetes.
Kubernetes api already exposes a bunch of metrics in prometheus format,
https://github.com/kubernetes/ingress-nginx also exposes prometheus metrics (enable-vts-status: "true"), and you can also install https://github.com/prometheus/node_exporter as a daemonset to monitor CPU, disk, etc...
I install one prometheus inside the cluster to monitor internal metrics and one outside the cluster to monitor LBs and URLs.
Both send alerts to the same https://github.com/prometheus/alertmanager that MUST be outside the cluster.
It took me about a week to configure everything properly.
It was worth it.