Can not connect to service in 192.168.99.101 - kubernetes

I am new on kubernetes and need your help.
I followed installation instruction on this page and everything worked fine. Here are some Info after installation:
~ > kubectl version
Client Version: version.Info{Major:"1", Minor:"13", GitVersion:"v1.13.1", GitCommit:"eec55b9ba98609a46fee712359c7b5b365bdd920", GitTreeState:"clean", BuildDate:"2018-12-13T19:44:10Z", GoVersion:"go1.11.2", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"12", GitVersion:"v1.12.4", GitCommit:"f49fa022dbe63faafd0da106ef7e05a29721d3f1", GitTreeState:"clean", BuildDate:"2018-12-14T06:59:37Z", GoVersion:"go1.10.4", Compiler:"gc", Platform:"linux/amd64"}
~ > kubectl cluster-info
Kubernetes master is running at https://192.168.99.101:8443
~ > minikube ip
192.168.99.101
Then I try to run a simple hello world service:
~ > minikube start
~ > kubectl get nodes
NAME STATUS ROLES AGE VERSION
minikube Ready <none> 18h v1.12.4
~ > kubectl run hw --image=karthequian/helloworld --port=80
kubectl run --generator=deployment/apps.v1 is DEPRECATED and will be removed in a future version. Use kubectl run --generator=run-pod/v1 or kubectl create instead.
deployment.apps/hw created
~ > kubectl get deployments
NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE
hw 1 1 1 1 21s
~ > kubectl get rs
NAME DESIRED CURRENT READY AGE
hw-854c64787 1 1 1 71s
~ > kubectl get pods
NAME READY STATUS RESTARTS AGE
hw-854c64787-8xhw6 1/1 Running 0 59s
~ > kubectl expose deployment hw --type=NodePort
service/hw exposed
~> kubectl get services
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
hw NodePort 10.106.4.194 <none> 80:31269/TCP 6m23s
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 20h
~ > minikube service hw
Opening kubernetes service default/hw in default browser...
After the last line I got this output:
Did I do something wrong? Is there anything I have not configured right? Thank you.

Here is how I solved this problem. First I go to Docker -> Preferences -> Kubernetes and Tick "Enable Kubernetes". It will take around 5 mins to install.
Then I delete my current minikube with minikube delete and repeat the whole process(minikube start, kubectl run hw --image=karthequian/helloworld --port=80, kubectl expose deployment hw --type=NodePort, minikube service hw
) and it works :).

Related

K8s Error from server (NotFound): deployments.apps "nginx" not found

Problem
Coursera Google Cloud Fundamentals: Getting Started with Kubernetes Engine
has the instructions to run and expose the pod. The demo video shows it is working.
However, it causes the error in my execution. How to fix?
kubectl run nginx --image=nginx:1.10.0
kubectl expose deployment nginx --type LoadBalancer --port 80
---
Error from server (NotFound): deployments.apps "nginx" not found
Environment
GCP k8s.
$ kubectl version
Client Version: version.Info{Major:"1", Minor:"22", GitVersion:"v1.22.1", GitCommit:"632ed300f2c34f6d6d15ca4cef3d3c7073412212", GitTreeState:"clean", BuildDate:"2021-08-19T15:45:37Z", GoVersion:"go1.16.7", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"20+", GitVersion:"v1.20.8-gke.2100", GitCommit:"4cd085fda961821985d176d25b67445c1efb6ba1", GitTreeState:"clean", BuildDate:"2021-07-16T09:22:57Z", GoVersion:"go1.15.13b5", Compiler:"gc", Platform:"linux/amd64"}
WARNING: version difference between client (1.22) and server (1.20) exceeds the supported minor version skew of +/-1
Cause
kubectl run does not create deployment.
'Error from server (NotFound): deployments.extensions "nginx" not found' when exposing NodePort #31
Did you create a pod with "kubectl run"? If yes, that doesn't create a deployment with (at least) kubectl v1.18.2, so instead use "kubectl create deployment nginx --image=nginx:1.10.0"
Fix
Create a deployment as in Creating and exploring an nginx deployment from a YAML or run kubectl create deployment ....
$ kubectl create deployment nginx --image=nginx:1.12.0
deployment.apps/nginx created
$ kubectl expose deployment nginx --type LoadBalancer --port 80
service/nginx exposed
$ kubectl get services
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes ClusterIP 10.83.240.1 <none> 443/TCP 44m
nginx LoadBalancer 10.83.246.84 35.225.127.227 80:30825/TCP 60s

unable to launch kubectl dashboard on Mac

I have deployed minikube on MacOS using the instructions here https://kubernetes.io/docs/tasks/tools/install-minikube/
The brew install was ok and the minikube status shows
$ minikube status
host: Running
kubelet: Running
apiserver: Running
kubectl: Correctly Configured: pointing to minikube-vm at 192.168.99.102
I am able to interact with the cluster using kubectl
$kubectl run hello-minikube --image=k8s.gcr.io/echoserver:1.10 --port=8080
Viewing the pods is also ok
$kubectl get pod
NAME READY STATUS RESTARTS AGE
hello-minikube-856979d68c-glhsx 1/1 Running 0 18m
But when i try to launch the kubectl dashboard, i get 503 error
$minikube dashboard
Temporary Error: unexpected response code: 503
Temporary Error: unexpected response code: 503
The Dashboard service seems to present
$kubectl -n kube-system get svc -o wide
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR
kube-dns ClusterIP 10.96.0.10 <none> 53/UDP,53/TCP,9153/TCP 3h19m k8s-app=kube-dns
kubernetes-dashboard ClusterIP 10.109.210.119 <none> 80/TCP 119m app=kubernetes-dashboard
below is the kubectl version info
$kubectl version
Client Version: version.Info{Major:"1", Minor:"15", GitVersion:"v1.15.2", GitCommit:"f6278300bebbb750328ac16ee6dd3aa7d3549568", GitTreeState:"clean", BuildDate:"2019-08-05T16:57:42Z", GoVersion:"go1.12.7", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"15", GitVersion:"v1.15.2", GitCommit:"f6278300bebbb750328ac16ee6dd3aa7d3549568", GitTreeState:"clean", BuildDate:"2019-08-05T09:15:22Z", GoVersion:"go1.12.5", Compiler:"gc", Platform:"linux/amd64"}
Any pointers on what is missing ? How to get the dashboard working
Thanks
Praveen
Check kubectl cluster-info, you can find more here
kubectl -n kube-system port-forward svc/kubernetes-dashboard 8080:80
Your dashboard should be accessible on http://localhost:8080, keep in mind that the dashboard is deprecated so you can check octant.
Try $ minikube dashboard command. It will open a new tab in your default browser showing the minikube dashbord.
Finally able to solve this issue with instructions from GitHub issues
https://github.com/kubernetes/minikube/issues/4352
Basically, with these commands
minikube stop
minikube start --extra-config=apiserver.authorization-mode=RBAC
kubectl create clusterrolebinding add-on-cluster-admin --clusterrole=cluster-admin --serviceaccount=kube-system:default
minikube dashboard

Running Kubernetes Sample Controller

I am trying to run the Kubernetes sample controller example by following the link https://github.com/kubernetes/sample-controller. I have the repo set up on an Ubuntu 18.04 system and was able to build the sample-controller package.
However, when I try to run the go package, I am getting some errors and am unable to debug the issue. Can someone please help me with this?
Here are the steps that I followed :
user#ubuntu-user:~$ go get k8s.io/sample-controller
user#ubuntu-user:~$ cd $GOPATH/src/k8s.io/sample-controller
Here's the error that I get on running the controller:
user#ubuntu-user:~/go/src/k8s.io/sample-controller$ ./sample-controller -kubeconfig=$HOME/.kube/config
E0426 15:05:57.721696 31517 reflector.go:125] k8s.io/sample-controller/pkg/generated/informers/externalversions/factory.go:117: Failed to list *v1alpha1.Foo: the server could not find the requested resource (get foos.samplecontroller.k8s.io)
Kubectl Version :
user#ubuntu-user:~$ kubectl version
Client Version: version.Info{Major:"1", Minor:"14", GitVersion:"v1.14.1", GitCommit:"b7394102d6ef778017f2ca4046abbaa23b88c290", GitTreeState:"clean", BuildDate:"2019-04-08T17:11:31Z", GoVersion:"go1.12.1", Compiler:"gc", Platform:"linux/amd64"} Server Version: version.Info{Major:"1", Minor:"14", GitVersion:"v1.14.1", GitCommit:"b7394102d6ef778017f2ca4046abbaa23b88c290", GitTreeState:"clean", BuildDate:"2019-04-08T17:02:58Z", GoVersion:"go1.12.1", Compiler:"gc", Platform:"linux/amd64"```
I have reproduced your issue. The order of commands in this tutorial is wrong.
In this case you received this error due to lack of resource (samplecontroller)
$ ./sample-controller -kubeconfig=$HOME/.kube/config
E0430 12:55:05.089624 147744 reflector.go:125] k8s.io/sample-controller/pkg/generated/informers/externalversions/factory.go:117: Failed to list *v1alpha1.Foo: the server could not find the requested resource (get foos.samplecontroller.k8s.io)
^CF0430 12:55:05.643778 147744 main.go:74] Error running controller: failed to wait for caches to sync
goroutine 1 [running]:
k8s.io/klog.stacks(0xc0002feb00, 0xc000282200, 0x66, 0x1f5)
/usr/local/google/home/user/go/src/k8s.io/klog/klog.go:840 +0xb1
k8s.io/klog.(*loggingT).output(0x2134040, 0xc000000003, 0xc0002e12d0, 0x20afafb, 0x7, 0x4a, 0x0)
/usr/local/google/home/user/go/src/k8s.io/klog/klog.go:791 +0x303
k8s.io/klog.(*loggingT).printf(0x2134040, 0x3, 0x14720f2, 0x1c, 0xc0003c1f48, 0x1, 0x1)
/usr/local/google/home/user/go/src/k8s.io/klog/klog.go:690 +0x14e
k8s.io/klog.Fatalf(...)
/usr/local/google/home/user/go/src/k8s.io/klog/klog.go:1241
main.main()
/usr/local/google/home/user/go/src/k8s.io/sample-controller/main.go:74 +0x3f5
You can verify that this api was not created
$ kubectl api-versions | grep sample
$ <emptyResult>
In the tutorial you have command to create Custom Resource Definition
$ kubectl create -f artifacts/examples/crd.yaml
customresourcedefinition.apiextensions.k8s.io/foos.samplecontroller.k8s.io created
Now you can search this CRD, it will be on the list now.
$ kubectl api-versions | grep sample
samplecontroller.k8s.io/v1alpha1
Next step is to create Foo resource
$ kubectl create -f artifacts/examples/example-foo.yaml
foo.samplecontroller.k8s.io/example-foo created
Those commands will not create any objects yet.
user#user:~/go/src/k8s.io/sample-controller$ kubectl get all
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/kubernetes ClusterIP XX.XXX.XXX.XX <none> 443/TCP 14d
All resources will be created after you will run ./sample-controller -kubeconfig=$HOME/.kube/config
user#user:~/go/src/k8s.io/sample-controller$ ./sample-controller -kubeconfig=$HOME/.kube/config
user#user:~/go/src/k8s.io/sample-controller$ kubectl get all
NAME READY STATUS RESTARTS AGE
pod/example-foo-6cbc69bf5d-8k59h 1/1 Running 0 43s
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/kubernetes ClusterIP 10.39.240.1 <none> 443/TCP 14d
NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE
deployment.apps/example-foo 1 1 1 1 43s
NAME DESIRED CURRENT READY AGE
replicaset.apps/example-foo-6cbc69bf5d 1 1 1 43s
Correct order:
$ go get k8s.io/sample-controller
$ cd $GOPATH/src/k8s.io/sample-controller
$ go build -o sample-controller .
$ kubectl create -f artifacts/examples/crd.yaml
$ kubectl create -f artifacts/examples/example-foo.yaml
$ ./sample-controller -kubeconfig=$HOME/.kube/config
$ kubectl get deployments
Did you run those two commands from readme?
# create a CustomResourceDefinition
kubectl create -f artifacts/examples/crd.yaml
# create a custom resource of type Foo
kubectl create -f artifacts/examples/example-foo.yaml

Get ServiceUnavailable from `kubectl top` with heapster

I have a managed kubernetes setup, called Cluster Container Engine (CCE), in the Open Telekom Cloud. Their documentation can be found online.
My CCE has one master and three nodes which run k8s version 1.9.2 (more details below). I can access the CCE through kubectl and deploy new pods onto it.
The CCE has a deployment of heapster preinstalled. However, attempting to inspect node resource usage fails (I can observe the same effect for pod usage):
$ kubectl top pods
Error from server (ServiceUnavailable): the server is currently unable to handle the request (get services http:heapster:)
I've attempted all debugging steps I could think of (see below) and I'm still lost when it comes to fixing this. Any advice?
The deployment, pod and service items for heapster are present (outputs filtered to include only heapster):
$ kubectl get po -n kube-system
NAME READY STATUS RESTARTS AGE
heapster-apiserver-84b844ffcf-lzh4b 1/1 Running 0 47m
$ kubectl get svc -n kube-system
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
heapster ClusterIP 10.247.150.244 <none> 80/TCP 19d
$ kubectl get deploy -n kube-system
NAME READY UP-TO-DATE AVAILABLE AGE
heapster-apiserver 1/1 1 1 19d
To check that heapster does indeed collect metrics properly, I've ssh'd into one of the nodes and executed:
$ curl -k http://10.247.150.244:80/api/v1/model/metrics/
[
"cpu/usage_rate",
"memory/usage",
"cpu/request",
"cpu/limit",
"memory/request",
"memory/limit"
]
Pod Log Output
Finally, I checked the log output from the heapster-apiserver-84b844ffcf-lzh4b pod:
$ kubectl logs -n kube-system heapster-apiserver-84b844ffcf-lzh4b
I0311 13:38:18.334525 1 heapster.go:78] /heapster --source=kubernetes.summary_api:''?kubeletHttps=true&inClusterConfig=false&insecure=true&auth=/srv/config --api-server --secure-port=6443
I0311 13:38:18.334718 1 heapster.go:79] Heapster version v1.5.3
I0311 13:38:18.340912 1 configs.go:61] Using Kubernetes client with master "https://192.168.1.228:5443" and version <nil>
I0311 13:38:18.340996 1 configs.go:62] Using kubelet port 10255
I0311 13:38:18.358918 1 heapster.go:202] Starting with Metric Sink
I0311 13:38:18.510751 1 serving.go:327] Generated self-signed cert (/var/run/kubernetes/apiserver.crt, /var/run/kubernetes/apiserver.key)
E0311 13:38:18.540860 1 heapster.go:128] Could not create the API server: missing clientCA file
I0311 13:38:18.558944 1 heapster.go:112] Starting heapster on port 8082
Cluster Info
$ kubectl version
Client Version: version.Info{Major:"1", Minor:"13", GitVersion:"v1.13.3", GitCommit:"721bfa751924da8d1680787490c54b9179b1fed0", GitTreeState:"clean", BuildDate:"2019-02-01T20:08:12Z", GoVersion:"go1.11.5", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"9+", GitVersion:"v1.9.2-CCE2.0.7-B003", GitCommit:"302f471a1e2caa114c9bb708c077fbb363aa2f13", GitTreeState:"clean", BuildDate:"2018-06-20T03:27:16Z", GoVersion:"go1.9.2", Compiler:"gc", Platform:"linux/amd64"}
$ kubectl get nodes
192.168.1.163 Ready worker 19d v1.9.2-CCE2.0.7-B003
192.168.1.211 Ready nfs-server 19d v1.9.2-CCE2.0.7-B003
192.168.1.227 Ready worker 19d v1.9.2-CCE2.0.7-B003
All nodes use EulerOS_2.0_SP2 with kernel version 3.10.0-327.59.59.46.h38.x86_64.
I0311 13:38:18.510751 1 serving.go:327] Generated self-signed cert (/var/run/kubernetes/apiserver.crt, /var/run/kubernetes/apiserver.key)
It seems like your API server is running on HTTP but heapster has a https url configured. You need to set the --source parameter to override the Kubernetes master like described here:
--source=kubernetes:http://master-ip?inClusterConfig=false&useServiceAccount=true&auth=
BTW: heapster has been deprecated and it is advised to switch to metrics server.

How to resolve Kubernetes API from minikube?

I can't seem to be able to find the Kubernetes API IP address from within the minikube machine, using minikube ssh.
nslookup kubernetes.default.svc.cluster.local 10.0.0.1
Server: 10.0.0.1
Address 1: 10.0.0.1
nslookup: can't resolve 'kubernetes.default.svc.cluster.local'
Same thing happens with just kubernetes. This happens even though it seems that the kube-dns pods are up-and-running just fine:
# kubectl -n kube-system get pods
NAME READY STATUS RESTARTS AGE
kube-addon-manager-minikube 1/1 Running 0 9m
kube-dns-910330662-16bvw 3/3 Running 0 8m
kubernetes-dashboard-vbn9m 1/1 Running 0 8m
There is no KubeDNS info in cluster-info (but nothing major in the dump):
# kubectl cluster-info
Kubernetes master is running at https://192.168.64.26:8443
To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.
Version for kubectl:
# kubectl version
Client Version: version.Info{Major:"1", Minor:"7", GitVersion:"v1.7.5", GitCommit:"17d7182a7ccbb167074be7a87f0a68bd00d58d97", GitTreeState:"clean", BuildDate:"2017-08-31T09:14:02Z", GoVersion:"go1.8.3", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"7", GitVersion:"v1.7.0", GitCommit:"d3ada0119e776222f11ec7945e6d860061339aad", GitTreeState:"clean", BuildDate:"2017-07-26T00:12:31Z", GoVersion:"go1.8.3", Compiler:"gc", Platform:"linux/amd64"}
and minikube:
# minikube version
minikube version: v0.21.0
This happens on OSX and both with xhyve and virtualbox as drivers.
There are quite a few minikube/dns issues in the minikube issue tracker, but none of them quite like this. What should the kubectl cluster-info output look like for minikube that has kube-dns working? What about the nslookup, shouldn't that work?
EDIT. It seems like kubectl cluster-info for minikube is returning what it should, but I can't even resolve kubernetes from inside the kube-dns pod either:
# kubectl -n kube-system exec -it kube-dns-910330662-1n64f -c kubedns -- nslookup kubernetes.default.svc.cluster.local localhost
Server: 127.0.0.1
Address 1: 127.0.0.1 localhost
nslookup: can't resolve 'kubernetes.default.svc.cluster.local': Name does not resolve
which really should work?