I am thinking to use the stack Geoserver, PostGIS, Openlayers, ReactJS for my GIS project. I also plan to deploy this solution on Kubernetes with AWS.
Questions:
Assume the traffic will be : 100 requests/s -> to 1000rq/s
What is the minimum resources (vCPU, RAM) for:
- Each node K8s
- Each Geoserver (pod)
- PostGIS
Is there any formula so I can apply to have that result?
Thank you in advance
Lp Ccmu
Not really. It all depends on the footprint of all the different components of your specific application. I suggest you start small, gather a lot of metrics, and adjust.
Either grow or shrink depending on what you see on your metrics and make use of Kubernetes autoscaling tools like HPAs and the cluster autoscaler.
You can gather metrics using the AWS tools or something like Prometheus. There are many available resources on how to use Prometheus to gather Kubernetes metrics on the web.
Not really. For GeoServer it depends on the type of data and the size and complexity of the data sets as well as they styling you are applying.
You can integrate APM, Elastic and kibana to have react app to send the metrics related API endpoints requests and page hits to monitor traffic. Based on data you can adjust your deployment's resources.
You can see this post for GIS stack on Kubernetes.
https://link.medium.com/r645NGwpejb
Related
I have an EKS cluster in AWS where a specific service will work as a live feed to clients, I expect it to use very low resources but it will be using server-sent event which require long lived http connections. Given I will be serving thousands of users I believe the best metric to auto scaling would be open connections.
My understanding is that k8s only have cpu and memory usage as out of the box metrics for scaling, I may be wrong here. I looked into custom metrics but k8s documentation on that is extremely shallow.
Any sugestions or guidance on this is very welcome.
I deployed a web application on kubernetes cluster. This application consists of multiple nodes, and each node has multiple pods. How can I do the performance measurement of whole application? I can see some metrics results on prometheus/ grafana but these results for each node/pod, not for whole application. I am just trying to understand the bigger picture. For example, I see that application data is stored in redis(pod), but is it enough to look into only that pod to measure latency?
Every kubelet has cAdvisor (link) integrated into the binary. cAdvisor provides container users an understanding of the resource usage and performance characteristics of their running containers. Combine cAdvisor with an additional exporter like JMX exporter (for java, link), BlackBox exporter for probe-ing urls (response time, monitor http codes, etc. link). There are also frameworks, that provide metrics such as Java Springboot on path /actuator/prometheus and you can scrape these metrics. There are many different exporters (link), with each doing something else. When you gather all these metrics, you can have a bigger overview about the state of your application. Couple this with handmade Grafana dashboards and Alerting (AlertManager e.g.) and you can monitor almost everything about your application.
As per the prometheus/grafana stack, I guess what you are talking about is the kube-prom-stack with default dashboards already implemented into them.
I have a cluster with 7 nodes and a lot of services, nodes, etc in the Google Cloud Platform. I'm trying to get some metrics with StackDriver Legacy, so in the Google Cloud Console -> StackDriver -> Metrics Explorer I have all the set of anthos metrics listed but when I try to create a chart based on that metrics it doesn't show the data, actually the only response that I get in the panel is no data is available for the selected time frame even changing the time frame and stuffs.
Is right to think that with anthos metrics I can retrieve information about my cronjobs, pods, services like failed initializations, jobs failures ? And if so, I can do it with StackDriver Legacy or I need to Update to StackDriver kubernetes Engine Monitoring ?
Anthos solution, includes what’s called GKE-on prem. I’d take a look at the instructions to use logging and monitoring on GKE-on prem. Stackdriver monitors GKE On-Prem clusters in a similar way as cloud-based GKE clusters.
However, there’s a note where they say that currently, Stackdriver only collects cluster logs and system component metrics. The full Kubernetes Monitoring experience will be available in a future release.
You can also check that you’ve met all the configuration requirements.
I have a setup where I manage to monitor every container of my Rancher 1.6 environnement with a stack Prometheus(2.4.3)/Grafana (with cAdvisor v0.27.4 and NodeExporter v0.16.0).
Here is my issue. I manage to monitor every container consumption but I can't relate the consumption of a container based on the host.
For example, if I want to show information about CPU usage I use container_cpu_user_seconds_total from cAdvisor which provides cpu usage of the container in percentage related to its host but I can't find which host is concerned (I Have 4 hosts on this environnement) so the cpu consumption cumulative tends to go over 100%.
I would like to either show charts by host (I saw I could create dynamic charts in Grafana but it seems a bit hard so manually creating them doesn't bother me).
Should I try to create my own metrics in prom-conf file ? Seems a bit overkill for such stuff
I find this very strange that this information only interests me. That's why I ask it here.
I'm new to all of these tools.
Thank you in advance.
We have some Kubernetes clusters that have been deployed using kops in AWS.
We really like using the upstream/official images.
We have been wondering whether or not there was a good way to monitor the systems without installing software directly on the hosts? Are there docker containers that can extract the information from the host? I think that we are likely concerned with:
Disk space (this seems to be passed through to docker via df
Host CPU utilization
Host memory utilization
Is this host/node level information already available through heapster?
Not really a question about kops, but a question about operating Kubernetes. kops stops at the point of having a functional k8s cluster. You have networking, DNS, and nodes have joined the cluster. From there your world is your oyster.
There are many different options for monitoring with k8s. If you are a small team I usually recommend offloading monitoring and logging to a provider.
If you are a larger team or have more specific needs then you can look at such options as Prometheus and others. Poke around in the https://github.com/kubernetes/charts repository, as I know there is a Prometheus chart there.
As with any deployment of any form of infrastructure you are going to need Logging, Monitoring, and Metrics. Also, do not forget to monitor the monitoring ;)
I am using https://prometheus.io/, it goes naturally with kubernetes.
Kubernetes api already exposes a bunch of metrics in prometheus format,
https://github.com/kubernetes/ingress-nginx also exposes prometheus metrics (enable-vts-status: "true"), and you can also install https://github.com/prometheus/node_exporter as a daemonset to monitor CPU, disk, etc...
I install one prometheus inside the cluster to monitor internal metrics and one outside the cluster to monitor LBs and URLs.
Both send alerts to the same https://github.com/prometheus/alertmanager that MUST be outside the cluster.
It took me about a week to configure everything properly.
It was worth it.