In the official Kubernetes documentation:
https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale-walkthrough/
We can see the following:
This example requires a running Kubernetes cluster and kubectl, version 1.2 or later. Metrics server monitoring needs to be deployed in the cluster to provide metrics through the Metrics API. Horizontal Pod Autoscaler uses this API to collect metrics. To learn how to deploy the metrics-server, see the metrics-server documentation.
To specify multiple resource metrics for a Horizontal Pod Autoscaler, you must have a Kubernetes cluster and kubectl at version 1.6 or later. To make use of custom metrics, your cluster must be able to communicate with the API server providing the custom Metrics API. Finally, to use metrics not related to any Kubernetes object you must have a Kubernetes cluster at version 1.10 or later, and you must be able to communicate with the API server that provides the external Metrics API. See the Horizontal Pod Autoscaler user guide for more details.
In order to verify I can "make use of custom metrics", I ran:
kubectl get metrics-server
And got the result: error: the server doesn't have a resource type "metrics-server"
May I ask what can I do to verify "Metrics server monitoring needs to be deployed in the cluster" please?
Thank you
The actual behavior behind the kubectl is to send an API request to a particular endpoint in the Kubernetes API server. There are a couple of predefined objects coming along with kubectl. But if you have some endpoints that are not defined with kubectl, you can use the flag --raw to send the request to API server.
In your case, you can checkout the built-in metrics with this command.
> kubectl get --raw /apis/metrics.k8s.io
{"kind":"APIGroup","apiVersion":"v1","name":"metrics.k8s.io","versions":[{"groupVersion":"metrics.k8s.io/v1beta1","version":"v1beta1"}],"preferredVersion":{"groupVersion":"metrics.k8s.io/v1beta1","version":"v1beta1"}}
You will get the JSON response from kubectl. Then, you can follow the path under the response to query your target resources. In my case, in order to get the actual metrics, I will need to use this command.
> kubectl get --raw /apis/metrics.k8s.io/v1beta1/pods
For this metrics endpoint, it refers to the built-in metrics. They are CPU and memory. If you want to use the custom metrics, you will need to install the prometheus, prometheus adaptor and corresponding exporter depending on your application. For the custom metrics verification, you can go to the following endpoint.
> kubectl get --raw /apis/custom.metrics.k8s.io
Related
I'm trying to get additional metrics like CPU and Memory displayed on the Kubernetes Dashboard. Based on the different forums, it looks like you have to install Heapter under the Kube System namespace.
I installed heapster, however I'm not seeing any metrics on the dashboard and when I go visit the URL, it shows 404.
How do I show additional heapster metrics on Kubernetes Dashboard?
Heapter is deprecated in favor of metrics-server, it provides the same functionality you are looking for, i.e CPU and Memory usage in the dashboard
if you are using Kubernetes 1.8+ you can install it using
kubectl apply -f https://github.com/kubernetes-sigs/metrics-server/releases/download/v0.3.7/components.yaml
for more information check https://github.com/kubernetes-sigs/metrics-server
I have my kubernetes cluster on ibm cloud account which i have access to.Using api key im able to generate IAM token(Bearer token) for authorization to use API's in this swagger https://containers.cloud.ibm.com/global/swagger-global-api/ and get details till namespaces list.
But i need to still dig through namespaces and get pods of that and each pod image there.only using REST API/client libraries(no kubectl or commands).How would i achieve this from my external node application?
I installed prometheus-adapter with helm.
Now I don't know how to configure prometheus-adapter so that my kubernetes cluster can communicate with a extern server where prometheus is installed.
Where and how can i connect the prometheus-adapter to prometheus.
I want to use data from prometheus for my external metrics in kubernetes.
First, you'll need to deploy the Prometheus Operator.
This walkthrough assumes that Prometheus is deployed in the prom namespace. Most of the sample commands and files are namespace-agnostic, but there are a few commands or pieces of configuration that rely on that namespace. If you're using a different namespace, simply substitute that in for prom when it appears.
Note that if you are deploying on a non-x86_64 (amd64) platform, you'll need to change the image field in the Deployment to be the appropriate image for your platform.
Make sure that you have default adapter which configuration should work with standard Prometheus Operator configuration, but if you've got custom relabelling rules, or your labels above weren't exactly namespace and pod, you may need to edit the configuration in the ConfigMap. The configuration walkthrough provides an overview of how configuration works.
Make sure that you have registered the API with the API aggregator (part of the main Kubernetes API server).
Try fetching the discovery information for it:
$ kubectl get --raw /apis/custom.metrics.k8s.io/v1beta1
Since you've set up Prometheus to collect your app's metrics, you should see a pods/http_request resource show up. This represents the http_requests_total metric, converted into a rate, aggregated to have one datapoint per pod. Notice that this translates to the same API that our HorizontalPodAutoscaler was trying to use above.
The API is registered as custom.metrics.k8s.io/v1beta1, and you can find more information about aggregation at Concepts: Aggregation.
More information you can find in this instruction.
Let me know if it helps.
if you just want to communicate between prometheus-adapter and prometheus, you need to mount prometheus service url prometheus-adapter, so that prometheus-adapter will know where to grab the metric.
the default prometheus service url is http://prometheus.svc:9090 . you need to figure out what is your prometheus service url.
I am exploring client-go library for collecting resource metric for a kubernetes cluster. I am more keen on collecting the container metrics from all the pods.
But according to the wiki, https://kubernetes.io/docs/tasks/debug-application-cluster/resource-usage-monitoring/, i see that we can only get pod or node level metrics and not container level metrics.
Is there a way i can collect container level metric (like the way docker api gives the metrics for a container)?
You should deploy the External Metric server like Prometheus Adapter (example of full metric solution mentioned in official doc you linked).
The quickest way to achieve it is via kube-prometheus repository, which is part of prometheus-operator project.
It already includes kube-state-metrics, which generates custom metrics of your interest (Pod Metrics), example of container related one: kube_pod_container_resource_requests_cpu_cores.
I'd like to know if it's possible to get information from a pod when it's just created.
I'm spending time in developing a kubernetes controller process that reacts itself when a pod is created in cluster.
When a pod is just created, the service has to be able to get some basic information from pod. For example, ip, annotations...
I'd like to use a java service.
Any ideas?
You can use kubernetes
api-server
to get information regarding
endpoints (service)
. Kubernetes expose its API via REST so, you can use anything to communicate. Also, verify the results using 'kubectl' tool while development. For example, if you want to monitor pods related to service say, myservice.
kubectl get endpoints <myservice_pod> --watch
This will notify you with any activity with pods related to myservice. IMO, in java you have to use polling mechanism to mimic --watch functionality.
well, if you use kubernetes API client you can just watch on changes for all pods and then get their details (assuming you have granted RBAC auth)