I tried with the below command but it provides only the created date. How do I get when was it last updated?
kubectl describe hpa -n xyz hpa-hello
Example: Today I create a hpa with max replica 3 and tomorrow I apply the same yaml but with max replica 6. If I do that, I can see only the created date and not the updated date. The description correctly shows the updated max replicas as 6
Update: There is no direct way to obtain this information !
I Assume here that the last updated date or time nothing but when was last auto scaled?
To get details about the Horizontal Pod Autoscaler, you can use kubectl get hpa with the -o yaml flag. The status field contains information about the current number of replicas and any recent autoscaling events.
kubectl get hpa <hpa name> -o yaml
In this output we can see the last transition times along with messages.
Refer to this doc which might help you and also to view the details about the HPA.
Edit 1:
Moreover, I have gone through many docs and I don't think we can get the info about it from a get command. For this, an auditing resource in k8's can be created to log all activities generated by users for a specific type of resource . This might help you to find the logs.
Related
is there a way to pull out the username who was the last one that updated a pod in a namespace?
I have already tried the below command but non of them get me the user name
helm get values *myservice*
kubectl get pod *mypod*
If you are the cluster-admin, then you can check the kubernetes audit logs and determine the activities done in any particular namespace.
You can find more about auditing here.
We can see updates to deployment using command:
kubectl rollout history deploy/<name>
We can also see updated config using:
kubectl rollout history --revision=<revision-#> deploy/<name>
I'm not sure how to find out given revision's update time. Is it possible to find it?
If you are storing events from the namespace or api server logs, you might be able to find out. One crude way will be to look at creation time for replica sets of the deployment - kubectl get replicaset
I have Prometheus setup via Helm from Terraform and it's is configured to connect to my Kubernetes cluster. I open my Prometheus but I am not sure which metric to choose from the list to be able to view the CPU/MEM of running pods/jobs.
Here are all the pods running with the command (test1 is the kube namespace):
kubectl -n test1 get pods
podsrunning
When, I am on Prometheus, I see many metrics related to CPU, but not sure which one to choose:
prom1
I tried to choose one, but the namespace = prometheus and it uses prometheus-node-exporter and I don't see my cluster or my namespace test1 anywhere here.
prom2
Could you please help me? Thank you very much in advance.
UPDATE SCREENSHOT
UPDATE SCREENSHOT
I need to concentrate on this specific namespace, normally with the command:
kubectl get pods --all-namespaces | grep hermatwin
I see the first line with namespace = jobs I think this is namespace.
No result when set calendar to last Friday:
UPDATE SCREENSHOT April 20
I tried to select 2 days with starting date on last Saturday 17 April but I don't see any result:
ANd, if I remove (namespace="jobs") condition, I don't see any result either:
I tried to rerun the job (simulation jobs) again just now and tried to execute the prometheus query while the job was still running mode but I don't get any result :-( Here you can see my jobs where running.
I don't get any result:
When using simple filter, just container_cpu_usage_seconds_total, I can see the namespace="jobs"
node_cpu_seconds_total is a metric from node-exporter, the exporter that brings machine statistics and its metrics are prefixed with node_. You need metrics from cAdvisor, this one produces metrics related to containers and they are prefixed with container_:
container_cpu_usage_seconds_total
container_cpu_load_average_10s
container_memory_usage_bytes
container_memory_rss
Here are some basic queries for you to get started. Be ready that they may require tweaking (you may have different label names):
CPU Utilisation Per Pod
sum(irate(container_cpu_usage_seconds_total{container!="POD", container=~".+"}[2m])) by (pod)
RAM Usage Per Pod
sum(container_memory_usage_bytes{container!="POD", container=~".+"}) by (pod)
In/Out Traffic Rate Per Pod
Beware that pods with host network mode (not isolated) show traffic rate for the whole node. * 8 is to convert bytes to bits for convenience (MBit/s, GBit/s, etc).
# incoming
sum(irate(container_network_receive_bytes_total[2m])) by (pod) * 8
# outgoing
sum(irate(container_network_transmit_bytes_total[2m])) by (pod) * 8
How to list the current deployments running in Kubernetes with custom columns displayed as mentioned below:
DEPLOYMENT CONTAINER_IMAGE READY_REPLICAS NAMESPACE
The data should be sorted by the increasing order of the deployment name.
Look a the -o custom-columns feature. https://kubernetes.io/docs/reference/kubectl/overview/#custom-columns shows the basics. The hard one would be container_image, since a pod can contain more than one, but assuming you just want the first, something like .spec.template.containers[0].image? Give a shot and see how it goes.
Command to get the custom columns as per the question:
Kubectl get deployments -o custom-columns=DEPLOYMENT:.metadata.name,CONTAINER_IMAGE:..image,READY_REPLICAS:..replicas,NAMESPACE:..namespace
I am scraping the kubernetes metrics from prometheus and would need to extract the number of running pods.
I can see container_last_seen metrics but how should i get no of pods running. Can someone help on this?
If you need to get number of running pods, you can use a metric from the list of pods metrics https://github.com/kubernetes/kube-state-metrics/blob/master/docs/pod-metrics.md for that (To get the info purely on pods, it'd make sens to use pod-specific metrics).
For example if you need to get the number of pods per namespace, it'll be:
count(kube_pod_info{namespace="$namespace_name"}) by (namespace)
To get the number of all pods running on the cluster, then just do:
count(kube_pod_info)
Assuming you want to display that in Grafana according to your question tags, from this Kubernetes App Metrics dashboard for example:
count(count(container_memory_usage_bytes{container_name="$container", namespace="$namespace"}) by (pod_name))
You can just import the dashboard and play with the queries.
Depending on your configuration/deployment, you can adjust the variables container_name and namespace, grouping by (pod_name) and count'ing it does the trick. Some other label than pod_name can be used as long as it's shared between the pods you want to count.
If you want to see only the number of "deployed" pods in some namespace, you can use the solutions in previous answers.
My use case was to see the current running pods in some namespace and below is my solution:
'min_over_time(sum(group(kube_pod_container_status_ready{namespace="BC_NAME"}) by (pod,uid)) [5m:1m]) OR on() vector(0)'
Please replace BC_NAME with your namespace name.
The timespan provides you fine the data.
If no data found - no pod currently running it returns '0'