I have just started a new Kubernetes 1.8.0 environment using minikube (0.27) on Windows 10.
I followed this steps but it didn't work:
https://kubernetes.io/docs/tasks/access-application-cluster/web-ui-dashboard/
When I list pods this is the result:
C:\WINDOWS\system32>kubectl get pods --all-namespaces
NAMESPACE NAME READY STATUS RESTARTS AGE
kube-system etcd-minikube 1/1 Running 0 23m
kube-system heapster-69b5d4974d-s9vrf 1/1 Running 0 5m
kube-system kube-addon-manager-minikube 1/1 Running 0 23m
kube-system kube-apiserver-minikube 1/1 Running 0 23m
kube-system kube-controller-manager-minikube 1/1 Running 0 23m
kube-system kube-dns-545bc4bfd4-xkt7l 3/3 Running 3 1h
kube-system kube-proxy-7jnk6 1/1 Running 0 23m
kube-system kube-scheduler-minikube 1/1 Running 0 23m
kube-system kubernetes-dashboard-5569448c6d-8zqnc 1/1 Running 2 52m
kube-system kubernetes-dashboard-869db7f6b4-ddlmq 0/1 CrashLoopBackOff 19 51m
kube-system monitoring-influxdb-78d4c6f5b6-b66m9 1/1 Running 0 4m
kube-system storage-provisioner 1/1 Running 2 1h
As you can see, I have 2 kubernets-dashboard pods now, one of then is running and the other one is CrashLookBackOff.
When I try to run minikube dashboard this is the result:
"Waiting, endpoint for service is not ready yet..."
I have tried to remove kubernetes-dashboard-869db7f6b4-ddlmq pod:
kubectl delete pod kubernetes-dashboard-869db7f6b4-ddlmq
This is the result:
"Error from server (NotFound): pods "kubernetes-dashboard-869db7f6b4-ddlmq" not found"
"Error from server (NotFound): pods "kubernetes-dashboard-869db7f6b4-ddlmq" not found"
You failed to delete the pod due to the lack of namespace (add -n kube-system). And it should be 1 dashboard pod if no modification's applied. If it still fails to run minikube dashboard after you delete the abnormal pod, more logs should be provided.
Related
I have a new install of kubernetes on Ubuntu-18 using version 1.24.3 with Calico. The calico-controller will not start:
$ sudo kubectl get pods --all-namespaces
NAMESPACE NAME READY STATUS RESTARTS AGE
kube-system calico-kube-controllers-555bc4b957-z4q2p 0/1 Pending 0 5m14s
kube-system calico-node-jz2j7 1/1 Running 0 5m15s
kube-system coredns-6d4b75cb6d-hwfx9 1/1 Running 0 5m14s
kube-system coredns-6d4b75cb6d-wdh55 1/1 Running 0 5m14s
kube-system etcd-ubuntu-18-extssd 1/1 Running 1 5m27s
kube-system kube-apiserver-ubuntu-18-extssd 1/1 Running 1 5m28s
kube-system kube-controller-manager-ubuntu-18-extssd 1/1 Running 1 5m26s
kube-system kube-proxy-t5z2r 1/1 Running 0 5m15s
kube-system kube-scheduler-ubuntu-18-extssd 1/1 Running 1 5m27s
Someone suggested setting a couple of Calico timeouts to 60 seconds, but that didn't work either.
What could be causing the calico-controller to fail to start, especially since the calico-node is running?
Also, is there a more trouble-free CNI implementation to use? Calico seems very error-prone.
I solved this by installing Weave:
kubectl apply -f "https://cloud.weave.works/k8s/net?k8s-version=$(kubectl version | base64 | tr -d '\n')"
with this cidr:
sudo kubeadm init --pod-network-cidr=192.168.0.0/16
I print all of my Pods with:
$ kubectl get pods --all-namespaces
and the output is:
NAMESPACE NAME READY STATUS RESTARTS AGE
calico-system calico-kube-controllers-7487d7f956-hx4fp 1/1 Running 0 88m
calico-system calico-node-vn52p 1/1 Running 0 88m
calico-system calico-typha-7588984c44-m6tsz 1/1 Running 0 88m
gitlab-managed-apps install-ingress 0/1 Error 0 14m********
gitlab-managed-apps install-prometheus 0/1 Error 0 12m
kube-system coredns-f9fd979d6-2n2pg 1/1 Running 0 91m
kube-system coredns-f9fd979d6-sq9bl 1/1 Running 0 91m
kube-system etcd-tuoputuo-iamnotstone-server 1/1 Running 0 91m
kube-system kube-apiserver-tuoputuo-iamnotstone-server 1/1 Running 0 91m
kube-system kube-controller-manager-tuoputuo-iamnotstone-server 1/1 Running 0 91m
kube-system kube-proxy-87jkr 1/1 Running 0 91m
kube-system kube-scheduler-tuoputuo-iamnotstone-server 1/1 Running 0 91m
tigera-operator tigera-operator-58f56c4958-4x9tp 1/1 Running 0 89m
But when I execute the logs command:
$ kubectl logs -f install-ingress
I see this error
Error from server (NotFound): pods "install-ingress" not found
The install-ingress pod is in gitlab-managed-apps namespace. If you do not specify namespace in the kubectl command then it will search for the pod in default namespace where the install-ingress pod is not present.
Could you try below command (specifying the namespace of the pod).
kubectl logs -f install-ingress -n gitlab-managed-apps
I'm running a simple example with Helm. Take a look below at values.yaml file:
cat << EOF | helm install helm/vitess -n vitess -f -
topology:
cells:
- name: 'zone1'
keyspaces:
- name: 'vitess'
shards:
- name: '0'
tablets:
- type: 'replica'
vttablet:
replicas: 1
mysqlProtocol:
enabled: true
authType: secret
username: vitess
passwordSecret: vitess-db-password
etcd:
replicas: 3
vtctld:
replicas: 1
vtgate:
replicas: 3
vttablet:
dataVolumeClaimSpec:
storageClassName: nfs-slow
EOF
Take a look at the output of current pods running below:
$ kubectl get pods --all-namespaces
NAMESPACE NAME READY STATUS RESTARTS AGE
kube-system coredns-fb8b8dccf-8f5kt 1/1 Running 0 32m
kube-system coredns-fb8b8dccf-qbd6c 1/1 Running 0 32m
kube-system etcd-master1 1/1 Running 0 32m
kube-system kube-apiserver-master1 1/1 Running 0 31m
kube-system kube-controller-manager-master1 1/1 Running 0 32m
kube-system kube-flannel-ds-amd64-bkg9z 1/1 Running 0 32m
kube-system kube-flannel-ds-amd64-q8vh4 1/1 Running 0 32m
kube-system kube-flannel-ds-amd64-vqmnz 1/1 Running 0 32m
kube-system kube-proxy-bd8mf 1/1 Running 0 32m
kube-system kube-proxy-nlc2b 1/1 Running 0 32m
kube-system kube-proxy-x7cd5 1/1 Running 0 32m
kube-system kube-scheduler-master1 1/1 Running 0 32m
kube-system tiller-deploy-8458f6c667-cx2mv 1/1 Running 0 27m
vitess etcd-global-6pwvnv29th 0/1 Init:0/1 0 16m
vitess etcd-operator-84db9bc774-j4wml 1/1 Running 0 30m
vitess etcd-zone1-zwgvd7spzc 0/1 Init:0/1 0 16m
vitess vtctld-86cd78b6f5-zgfqg 0/1 CrashLoopBackOff 7 16m
vitess vtgate-zone1-58744956c4-x8ms2 0/1 CrashLoopBackOff 7 16m
vitess zone1-vitess-0-init-shard-master-mbbph 1/1 Running 0 16m
vitess zone1-vitess-0-replica-0 0/6 Init:CrashLoopBackOff 7 16m
Running logs I see this error:
$ kubectl logs -n vitess vtctld-86cd78b6f5-zgfqg
++ cat
+ eval exec /vt/bin/vtctld '-cell="zone1"' '-web_dir="/vt/web/vtctld"' '-web_dir2="/vt/web/vtctld2/app"' -workflow_manager_init -workflow_manager_use_election -logtostderr=true -stderrthreshold=0 -port=15000 -grpc_port=15999 '-service_map="grpc-vtctl"' '-topo_implementation="etcd2"' '-topo_global_server_address="etcd-global-client.vitess:2379"' -topo_global_root=/vitess/global
++ exec /vt/bin/vtctld -cell=zone1 -web_dir=/vt/web/vtctld -web_dir2=/vt/web/vtctld2/app -workflow_manager_init -workflow_manager_use_election -logtostderr=true -stderrthreshold=0 -port=15000 -grpc_port=15999 -service_map=grpc-vtctl -topo_implementation=etcd2 -topo_global_server_address=etcd-global-client.vitess:2379 -topo_global_root=/vitess/global
ERROR: logging before flag.Parse: E0422 02:35:34.020928 1 syslogger.go:122] can't connect to syslog
F0422 02:35:39.025400 1 server.go:221] Failed to open topo server (etcd2,etcd-global-client.vitess:2379,/vitess/global): grpc: timed out when dialing
I'm running behind vagrant with 1 master and 2 nodes. I suspect that is a issue with eth1.
The storage are configured to use NFS.
$ kubectl logs etcd-operator-84db9bc774-j4wml
time="2019-04-22T17:26:51Z" level=info msg="skip reconciliation: running ([]), pending ([etcd-zone1-zwgvd7spzc])" cluster-name=etcd-zone1 cluster-namespace=vitess pkg=cluster
time="2019-04-22T17:26:51Z" level=info msg="skip reconciliation: running ([]), pending ([etcd-zone1-zwgvd7spzc])" cluster-name=etcd-global cluster-namespace=vitess pkg=cluster
It appears that etcd is not fully initializing. Note that neither the pod for the global lockserver (etcd-global-6pwvnv29th) nor the local one for cell zone1 (pod etcd-zone1-zwgvd7spzc) are ready.
I have a 2 node kubernetes cluster with calico networking. All the pods are up and running.
NAMESPACE NAME READY STATUS RESTARTS AGE
kube-system calico-etcd-94466 1/1 Running 0 21h
kube-system calico-kube-controllers-5fdcfdbdf7-xsjxb 1/1 Running 0 14d
kube-system calico-node-hmnf5 2/2 Running 0 14d
kube-system calico-node-vmmmk 2/2 Running 0 14d
kube-system coredns-78fcdf6894-dlqg6 1/1 Running 0 14d
kube-system coredns-78fcdf6894-zwrd6 1/1 Running 0 14d
kube-system etcd-kube-master-01 1/1 Running 0 14d
kube-system kube-apiserver-kube-master-01 1/1 Running 0 14d
kube-system kube-controller-manager-kube-master-01 1/1 Running 0 14d
kube-system kube-proxy-nxfht 1/1 Running 0 14d
kube-system kube-proxy-qnn45 1/1 Running 0 14d
kube-system kube-scheduler-kube-master-01 1/1 Running 0 14d
I wanted to query calico-etcd using etcdctl, but I get the following error.
# etcdctl --debug --endpoints "http://10.142.137.11:6666" get calico
start to sync cluster using endpoints(http://10.142.137.11:6666)
cURL Command: curl -X GET http://10.142.137.11:6666/v2/members
got endpoints(http://10.142.137.11:6666) after sync
Cluster-Endpoints: http://10.142.137.11:6666
cURL Command: curl -X GET http://10.142.137.11:6666/v2/keys/calico?quorum=false&recursive=false&sorted=false
Error: 100: Key not found (/calico) [4]
Any pointers on why I get this error?
As #JakubBujny mentioned, ETCDCTL_API=3 should be set to get the appropriate result.
I followed the guide on "Using kubeadm to Create a Cluster" but I am not able to view logs using kubectl:
root#o1:~# kubectl logs -n kube-system etcd-o1
Error from server: Get https://149.156.11.4:10250/containerLogs/kube-system/etcd-o1/etcd: tls: first record does not look like a TLS handshake
The above IP address is the cloud frontend address not the address of the VM which probably causes the problem. Some other kubectl cmds seem to work:
root#o1:~# kubectl cluster-info
Kubernetes master is running at https://10.6.16.88:6443
KubeDNS is running at https://10.6.16.88:6443/api/v1/namespaces/kube-system/services/kube-dns/proxy
To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.
root#o1:~# kubectl get pods --all-namespaces
NAMESPACE NAME READY STATUS RESTARTS AGE
kube-system etcd-o1 1/1 Running 0 3h
kube-system kube-apiserver-o1 1/1 Running 0 3h
kube-system kube-controller-manager-o1 1/1 Running 0 3h
kube-system kube-dns-545bc4bfd4-mhbfb 3/3 Running 0 3h
kube-system kube-flannel-ds-lw87h 2/2 Running 0 1h
kube-system kube-flannel-ds-rkqxg 2/2 Running 2 1h
kube-system kube-proxy-hnhfs 1/1 Running 0 3h
kube-system kube-proxy-qql4r 1/1 Running 0 1h
kube-system kube-scheduler-o1 1/1 Running 0 3h
Please help.
Maybe change the address in the $HOME/admin.conf.