I was trying to deploy prometheus-kubernetes. After I ran ./deploy,
Deploying Prometheus Operator
serviceaccount "prometheus-operator" configured
clusterrole "prometheus-operator" configured
clusterrolebinding "prometheus-operator" configured
service "prometheus-operator" configured
deployment "prometheus-operator" configured
Waiting for Operator to register custom resource definitions..............^C
Then I looked into the deploy and try to run kctl get customresourcedefinitions servicemonitors.monitoring.coreos.com.
ubuntu#test-box:~/prometheus-kubernetes$ kctl get customresourcedefinitions servicemonitors.monitoring.coreos.com
kubectl get customresourcedefinitions servicemonitors.monitoring.coreos.com
Error from server (NotFound): customresourcedefinitions.apiextensions.k8s.io "servicemonitors.monitoring.coreos.com" not found
My kubectl version,deployed on AWS instance.
Client Version: version.Info{Major:"1", Minor:"8", GitVersion:"v1.8.2", GitCommit:"bdaeafa71f6c7c04636251031f93464384d54963", GitTreeState:"clean", BuildDate:"2017-10-24T19:48:57Z", GoVersion:"go1.8.3", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"8", GitVersion:"v1.8.2", GitCommit:"bdaeafa71f6c7c04636251031f93464384d54963", GitTreeState:"clean", BuildDate:"2017-10-24T19:38:10Z", GoVersion:"go1.8.3", Compiler:"gc", Platform:"linux/amd64"}
Related
while using grafana for monitoring with Prometheus, we saw that sometimes grafana showed no pods for a service but when I checked in the cluster, all pods are running without any issue.
This issue is not continuous. Now I have to find out why grafana is alerting? But I don't know where to start.
Pls, ask if any info needed and pls show me the path, where I can start investigating.
Other info
This cluster is AWS EKS. Using prometheus:v2.22.1. Deployment of Prometheus & EKS cluster is done by Terraform.
$ kubectl version
Client Version: version.Info{Major:"1", Minor:"22", GitVersion:"v1.22.4", GitCommit:"b695d79d4f967c403a96986f1750a35eb75e75f1", GitTreeState:"clean", BuildDate:"2021-11-17T15:48:33Z", GoVersion:"go1.16.10", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"18+", GitVersion:"v1.18.20-eks-8c49e2", GitCommit:"8c49e2efc3cfbb7788a58025e679787daed22018", GitTreeState:"clean", BuildDate:"2021-10-17T05:13:46Z", GoVersion:"go1.13.15", Compiler:"gc", Platform:"linux/amd64"}
WARNING: version difference between client (1.22) and server (1.18) exceeds the supported minor version skew of +/-1
I'm getting this error message using istio v1.9 on a Kubernetes cluster. I tried searching it online but haven't found anything related to istio, a config that I could modify or an additional log to analyze. Do u have an idea of what could be happening?
I have a few apps running but nothing really big, one pod of a few apps.. not sure which limit I'm reaching here .. could this be an istio limit?
istio proxy sidecar log:
....
....
istio-proxy 2021-02-22T13:44:03.255958Z warning envoy config StreamSecrets gRPC config stream closed: 8, grpc: received message larger than max (XXXXXXX vs. 4194304)
....
$ istioctl version
fclient version: 1.9.0
control plane version: 1.9.0
data plane version: 1.9.0 (5 proxies)
$ kubectl version
Client Version: version.Info{Major:"1", Minor:"19", GitVersion:"v1.19.3", GitCommit:"1e11e4a2108024935ecfcb2912226cedeafd99df", GitTreeState:"clean", BuildDate:"2020-10-14T12:50:19Z", GoVersion:"go1.15.2", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"19", GitVersion:"v1.19.2", GitCommit:"f5743093fd1c663cb0cbc89748f730662345d44d", GitTreeState:"clean", BuildDate:"2020-09-16T13:32:58Z", GoVersion:"go1.15", Compiler:"gc", Platform:"linux/amd64"}
I'm trying to get kube-apiserver prometheus metrics with kubectl command.
I can get it from kubectl proxy [which proxy the api in localhost:8001(by default)]. then go to http://localhost:8001/metrics
I want to get this metrics directly through kubectl command, without proxying.
My kubernetes version
Client Version: version.Info{Major:"1", Minor:"11", GitVersion:"v1.11.0", GitCommit:"91e7b4fd31fcd3d5f436da26c980becec37ceefe", GitTreeState:"clean", BuildDate:"2018-06-27T20:17:28Z", GoVersion:"go1.10.2", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"13+", GitVersion:"v1.13.7-gke.24", GitCommit:"2ce02ef1754a457ba464ab87dba9090d90cf0468", GitTreeState:"clean", BuildDate:"2019-08-12T22:05:28Z", GoVersion:"go1.11.5b4", Compiler:"gc", Platform:"linux/amd64"}
Got the answer
$ kubectl get --raw /metrics
This command directly communicate with kube apiserver and get the prometheus metrics of kube apiserver.
You can install helm chart present here -
https://github.com/helm/charts/tree/master/stable/prometheus-operator
It will provide you all metrics, you can run the prometheus service locally by port-forwarding and can get all metrics at its dashboard
I am trying to execute describe on ingress but does not work. Get command works fine but not describe. Is anything that I am doing wrong? I am running this against AKS.
usr#test:/mnt/c/Repos/user/charts/canary$ kubectl get ingress
NAME HOSTS ADDRESS PORTS AGE
ingress-route xyz.westus.cloudapp.azure.com 80 6h
usr#test:/mnt/c/Repos/user/charts/canary$ kubectl describe ingress ingress-route
Error from server (NotFound): the server could not find the requested resource
Version seems fine:
Client Version: version.Info{Major:"1", Minor:"15", GitVersion:"v1.15.3", ..}
Server Version: version.Info{Major:"1", Minor:"13", GitVersion:"v1.13.10"...}
This could be caused by the incompatibility between the Kubernetes cluster version and the kubectl version.
Run kubectl version to print the client and server versions for the current context, example:
$ kubectl version
Client Version: version.Info{Major:"1", Minor:"16", GitVersion:"v1.16.1", GitCommit:"d647ddbd755faf07169599a625faf302ffc34458", GitTreeState:"clean", BuildDate:"2019-10-02T17:01:15Z", GoVersion:"go1.12.10", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"13+", GitVersion:"v1.13.10-gke.0", GitCommit:"569511c9540f78a94cc6a41d895c382d0946c11a", GitTreeState:"clean", BuildDate:"2019-08-21T23:28:44Z", GoVersion:"go1.11.13b4", Compiler:"gc", Platform:"linux/amd64"}
You might want to upgrade your cluster version or downgrade your kubectl version. See more details in https://github.com/kubernetes/kubectl/issues/675
I am new to kubernetes and have successfully setup minikube, kubectl and docker on Windows 10 Pro with Hyper-V
I am now trying to create a Pod using the following kubectl apply -f first-pod.yaml.
Here is a copy of my .yaml file
apiVersion: v1
kind: Pod
metadata:
name: webapp
spec:
containers:
- name: webapp
image: richardchesterwood/k8s-fleetman-webapp-angular:release0
A number of Stack Overflow post recommend checking kubectl version. I have done that and believe it is correct. I am running the latest version of kubectl and kubernetes.
Client Version: version.Info{Major:"1", Minor:"10", GitVersion:"v1.10.11", GitCommit:"637c7e288581ee40ab4ca210618a89a555b6e7e9", GitTreeState:"clean", BuildDate:"2018-11-26T14:38:32Z", GoVersion:"go1.9.3", Compiler:"gc", Platform:"windows/amd64"}
Server Version: version.Info{Major:"1", Minor:"15", GitVersion:"v1.15.0", GitCommit:"e8462b5b5dc2584fdcd18e6bcfe9f1e4d970a529", GitTreeState:"clean", BuildDate:"2019-06-19T16:32:14Z", GoVersion:"go1.12.5", Compiler:"gc", Platform:"linux/amd64"}
SchemaError(io.k8s.api.admissionregistration.v1beta1.RuleWithOperations): invalid object doesn't have additional properties
I have also stopped and restarted both minikube and docker. Any other ideas?
My version of kubectl is conflicting with the one provided by the docker-desktop. Solved it by:
Running Get-Command kubectl in Powershell returned
C:\ProgramFiles\Docker\Docker\Resources\bin
Going to Environment
Variables moving C:\kube above
C:\ProgramFiles\Docker\Docker\Resources\bin
Restarting Powershell