I have a new install of kubernetes on Ubuntu-18 using version 1.24.3 with Calico. The calico-controller will not start:
$ sudo kubectl get pods --all-namespaces
NAMESPACE NAME READY STATUS RESTARTS AGE
kube-system calico-kube-controllers-555bc4b957-z4q2p 0/1 Pending 0 5m14s
kube-system calico-node-jz2j7 1/1 Running 0 5m15s
kube-system coredns-6d4b75cb6d-hwfx9 1/1 Running 0 5m14s
kube-system coredns-6d4b75cb6d-wdh55 1/1 Running 0 5m14s
kube-system etcd-ubuntu-18-extssd 1/1 Running 1 5m27s
kube-system kube-apiserver-ubuntu-18-extssd 1/1 Running 1 5m28s
kube-system kube-controller-manager-ubuntu-18-extssd 1/1 Running 1 5m26s
kube-system kube-proxy-t5z2r 1/1 Running 0 5m15s
kube-system kube-scheduler-ubuntu-18-extssd 1/1 Running 1 5m27s
Someone suggested setting a couple of Calico timeouts to 60 seconds, but that didn't work either.
What could be causing the calico-controller to fail to start, especially since the calico-node is running?
Also, is there a more trouble-free CNI implementation to use? Calico seems very error-prone.
I solved this by installing Weave:
kubectl apply -f "https://cloud.weave.works/k8s/net?k8s-version=$(kubectl version | base64 | tr -d '\n')"
with this cidr:
sudo kubeadm init --pod-network-cidr=192.168.0.0/16
Related
I print all of my Pods with:
$ kubectl get pods --all-namespaces
and the output is:
NAMESPACE NAME READY STATUS RESTARTS AGE
calico-system calico-kube-controllers-7487d7f956-hx4fp 1/1 Running 0 88m
calico-system calico-node-vn52p 1/1 Running 0 88m
calico-system calico-typha-7588984c44-m6tsz 1/1 Running 0 88m
gitlab-managed-apps install-ingress 0/1 Error 0 14m********
gitlab-managed-apps install-prometheus 0/1 Error 0 12m
kube-system coredns-f9fd979d6-2n2pg 1/1 Running 0 91m
kube-system coredns-f9fd979d6-sq9bl 1/1 Running 0 91m
kube-system etcd-tuoputuo-iamnotstone-server 1/1 Running 0 91m
kube-system kube-apiserver-tuoputuo-iamnotstone-server 1/1 Running 0 91m
kube-system kube-controller-manager-tuoputuo-iamnotstone-server 1/1 Running 0 91m
kube-system kube-proxy-87jkr 1/1 Running 0 91m
kube-system kube-scheduler-tuoputuo-iamnotstone-server 1/1 Running 0 91m
tigera-operator tigera-operator-58f56c4958-4x9tp 1/1 Running 0 89m
But when I execute the logs command:
$ kubectl logs -f install-ingress
I see this error
Error from server (NotFound): pods "install-ingress" not found
The install-ingress pod is in gitlab-managed-apps namespace. If you do not specify namespace in the kubectl command then it will search for the pod in default namespace where the install-ingress pod is not present.
Could you try below command (specifying the namespace of the pod).
kubectl logs -f install-ingress -n gitlab-managed-apps
How do you list components running on the master Kubernetes node?
I assume there should be a kubeadm or kubectl command but can't find anything.
E.g. I'm looking to see if the Scheduler is running and I've used kubeadm config view which lists:
scheduler: {}
but not sure if that means the Scheduler is not running or there's simply no config for it.
Since you have installed with kubeadm, the control plane components must be running as pods in kube-system namespace. So you can run the following command to see if scheduler is running.
# kubectl get pod -n kube-system
NAME READY STATUS RESTARTS AGE
calico-node-4x9fp 2/2 Running 0 4d6h
coredns-86c58d9df4-bw2q9 1/1 Running 0 4d6h
coredns-86c58d9df4-gvcl9 1/1 Running 0 4d6h
etcd-k1 1/1 Running 0 4d6h
kube-apiserver-k1 1/1 Running 0 4d6h
kube-controller-manager-k1 1/1 Running 83 4d6h
kube-dash-kubernetes-dashboard-5b7cf769bc-pd2n2 1/1 Running 0 4d6h
kube-proxy-jmrrz 1/1 Running 0 4d6h
kube-scheduler-k1 1/1 Running 82 4d6h
metrics-server-8544b5c78b-k2lwt 1/1 Running 16 4d6h
tiller-deploy-5f4fc5bcc6-gvhlz 1/1 Running 0 4d6h
If you want to know all pods running on a master node(or any particular node), you can use field-selector to select the node.
kubectl get pod --all-namespaces --field-selector spec.nodeName=<nodeName>
To filter pods only in kube-system namespace running on particular node -
kubectl get pod -n kube-system --field-selector spec.nodeName=<nodeName>
Assuming that you want to check what is running in master node and you are unable not do that via Kubernetes API server.
For kubelet since its running as systemd service you can check systemctl status kubelet.service.
Other components such as scheduler is run as container by kubelet so you can check them with standard docker command such as docker ps.
I have a 2 node kubernetes cluster with calico networking. All the pods are up and running.
NAMESPACE NAME READY STATUS RESTARTS AGE
kube-system calico-etcd-94466 1/1 Running 0 21h
kube-system calico-kube-controllers-5fdcfdbdf7-xsjxb 1/1 Running 0 14d
kube-system calico-node-hmnf5 2/2 Running 0 14d
kube-system calico-node-vmmmk 2/2 Running 0 14d
kube-system coredns-78fcdf6894-dlqg6 1/1 Running 0 14d
kube-system coredns-78fcdf6894-zwrd6 1/1 Running 0 14d
kube-system etcd-kube-master-01 1/1 Running 0 14d
kube-system kube-apiserver-kube-master-01 1/1 Running 0 14d
kube-system kube-controller-manager-kube-master-01 1/1 Running 0 14d
kube-system kube-proxy-nxfht 1/1 Running 0 14d
kube-system kube-proxy-qnn45 1/1 Running 0 14d
kube-system kube-scheduler-kube-master-01 1/1 Running 0 14d
I wanted to query calico-etcd using etcdctl, but I get the following error.
# etcdctl --debug --endpoints "http://10.142.137.11:6666" get calico
start to sync cluster using endpoints(http://10.142.137.11:6666)
cURL Command: curl -X GET http://10.142.137.11:6666/v2/members
got endpoints(http://10.142.137.11:6666) after sync
Cluster-Endpoints: http://10.142.137.11:6666
cURL Command: curl -X GET http://10.142.137.11:6666/v2/keys/calico?quorum=false&recursive=false&sorted=false
Error: 100: Key not found (/calico) [4]
Any pointers on why I get this error?
As #JakubBujny mentioned, ETCDCTL_API=3 should be set to get the appropriate result.
I have just started a new Kubernetes 1.8.0 environment using minikube (0.27) on Windows 10.
I followed this steps but it didn't work:
https://kubernetes.io/docs/tasks/access-application-cluster/web-ui-dashboard/
When I list pods this is the result:
C:\WINDOWS\system32>kubectl get pods --all-namespaces
NAMESPACE NAME READY STATUS RESTARTS AGE
kube-system etcd-minikube 1/1 Running 0 23m
kube-system heapster-69b5d4974d-s9vrf 1/1 Running 0 5m
kube-system kube-addon-manager-minikube 1/1 Running 0 23m
kube-system kube-apiserver-minikube 1/1 Running 0 23m
kube-system kube-controller-manager-minikube 1/1 Running 0 23m
kube-system kube-dns-545bc4bfd4-xkt7l 3/3 Running 3 1h
kube-system kube-proxy-7jnk6 1/1 Running 0 23m
kube-system kube-scheduler-minikube 1/1 Running 0 23m
kube-system kubernetes-dashboard-5569448c6d-8zqnc 1/1 Running 2 52m
kube-system kubernetes-dashboard-869db7f6b4-ddlmq 0/1 CrashLoopBackOff 19 51m
kube-system monitoring-influxdb-78d4c6f5b6-b66m9 1/1 Running 0 4m
kube-system storage-provisioner 1/1 Running 2 1h
As you can see, I have 2 kubernets-dashboard pods now, one of then is running and the other one is CrashLookBackOff.
When I try to run minikube dashboard this is the result:
"Waiting, endpoint for service is not ready yet..."
I have tried to remove kubernetes-dashboard-869db7f6b4-ddlmq pod:
kubectl delete pod kubernetes-dashboard-869db7f6b4-ddlmq
This is the result:
"Error from server (NotFound): pods "kubernetes-dashboard-869db7f6b4-ddlmq" not found"
"Error from server (NotFound): pods "kubernetes-dashboard-869db7f6b4-ddlmq" not found"
You failed to delete the pod due to the lack of namespace (add -n kube-system). And it should be 1 dashboard pod if no modification's applied. If it still fails to run minikube dashboard after you delete the abnormal pod, more logs should be provided.
I followed the guide on "Using kubeadm to Create a Cluster" but I am not able to view logs using kubectl:
root#o1:~# kubectl logs -n kube-system etcd-o1
Error from server: Get https://149.156.11.4:10250/containerLogs/kube-system/etcd-o1/etcd: tls: first record does not look like a TLS handshake
The above IP address is the cloud frontend address not the address of the VM which probably causes the problem. Some other kubectl cmds seem to work:
root#o1:~# kubectl cluster-info
Kubernetes master is running at https://10.6.16.88:6443
KubeDNS is running at https://10.6.16.88:6443/api/v1/namespaces/kube-system/services/kube-dns/proxy
To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.
root#o1:~# kubectl get pods --all-namespaces
NAMESPACE NAME READY STATUS RESTARTS AGE
kube-system etcd-o1 1/1 Running 0 3h
kube-system kube-apiserver-o1 1/1 Running 0 3h
kube-system kube-controller-manager-o1 1/1 Running 0 3h
kube-system kube-dns-545bc4bfd4-mhbfb 3/3 Running 0 3h
kube-system kube-flannel-ds-lw87h 2/2 Running 0 1h
kube-system kube-flannel-ds-rkqxg 2/2 Running 2 1h
kube-system kube-proxy-hnhfs 1/1 Running 0 3h
kube-system kube-proxy-qql4r 1/1 Running 0 1h
kube-system kube-scheduler-o1 1/1 Running 0 3h
Please help.
Maybe change the address in the $HOME/admin.conf.