Unable to access minikube IP address - kubernetes

I am an absolute beginner to Kubernetes, and I was following this tutorial to get started. I have managed writing the yaml files. However once I deploy it, I am not able to access the web app.
This is my webapp yaml file
apiVersion: apps/v1
kind: Deployment
metadata:
name: webapp-deployment
labels:
app: webapp
spec:
replicas: 1
selector:
matchLabels:
app: webapp
template:
metadata:
labels:
app: webapp
spec:
containers:
- name: webapp
image: nanajanashia/k8s-demo-app:v1.0
ports:
- containerPort: 3000
env:
- name: USER_NAME
valueFrom:
secretKeyRef:
name: mongo-secret
key: mongo-user
- name: USER_PWD
valueFrom:
secretKeyRef:
name: mongo-secret
key: mongo-password
- name: DB_URL
valueFrom:
configMapKeyRef:
name: mongo-config
key: mongo-url
apiVersion: v1
kind: Service
metadata:
name: webapp-servicel
spec:
type: NodePort
selector:
app: webapp
ports:
- protocol: TCP
port: 3000
targetPort: 3000
nodePort: 30200
When I run the command : kubectl get node
When I run the command: kubectl get pods, i can see the pods running
kubectl get svc
I then checked the logs for webapp, I dont see any errors
I then checked the details logs by running the command: kubectl describe pod podname
I dont see any obvious errors in the result above, but again I am not experienced enough to check if there is any config thats not set properly.
Other things I have done as troubleshooting
Ran the following command for the minikube to open up the app : minikube service webapp-servicel, it opens up the web page, but again does not connect to the IP.
Uninstalled minikube, kubectl and all relevant folders, and run everything again.
pinged the ip address directly from command line, and cannot reach.
I would appreciate if someone can help me fix this.

Try these 3 options
can you do the kubectl get node -o wide and get the ip address of node and then open in web browser NODE_IP_ADDRESS:30200
Alternative you can run this command minikube service <SERVICE_NAME> --url which will give you direct url to access application and access the url in web browser.
kubectl port-forward svc/<SERVICE_NAME> 3000:3000
and access application on localhost:3000

Ran the following command for the minikube to open up the app : minikube service webapp-servicel, it opens up the web page, but again does not connect to the IP.
Uninstalled minikube, kubectl and .kube and run everything again.
pinged the ip address directly from command line, and cannot reach.
I suggest you to try port forwarding
https://kubernetes.io/docs/tasks/access-application-cluster/port-forward-access-application-cluster/
kubectl port-forward svc/x-service NodePort:Port

I got stuck here as well. After looking through some of the gitlab issues, I found a helpful tip about the minikube driver. The instructions for starting minikub are incorrect in the video if you used
minikube start -driver docker
Here's how to fix your problem.
stop minikube
minikube stop
delete minikube (this deletes your cluster)
minikube delete
start up minikube again, but this time specify the hyperkit driver
minikube start --vm-driver=hyperkit
check status
minikube status
reapply your components in this order by.
kubectl apply -f mongo-config.yaml
kubectl apply -f mongo-secret.yaml
kubectl apply -f mongo.yaml
kubectl aplly -f webapp.yaml
get your ip
minikube ip
open a browser, go to ip address:30200 (or whatever the port you defined was, mine was 30100). You should see an image of a dog and a form.
Some information in this SO post is useful too.

On Windows 11 with Ubuntu 20.04 WSL, it worked for me by using:
minikube start --driver=hyperv

On Windows 10 with Docker-Desktop one can even do not need to use minikube. Just enable Kubernetes in Docker-Desktop settings and use kubectl. Check the link for further information.
Using Kubernetes of Docker-Desktop I could simply reach webapp with localhost:30100. In my case, for some reason I had to pull mongo docker image manually with docker pull mongo:5.0.

Related

kubernetes dones't reach internal registry

I've deployed an docker registry inside my kubernetes:
$ kubectl get service
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
registry-docker-registry ClusterIP 10.43.39.81 <none> 443/TCP 162m
I'm able to pull images from my machine (service is exposed via an ingress rule):
$ docker pull registry-docker-registry.registry/skaffold-covid-backend:c5dfd81-dirty#sha256:76312ebc62c4b3dd61b4451fe01b1ecd2e6b03a2b3146c7f25df3d3cfb4512cd
...
Status: Downloaded newer image for registry-do...
When I'm trying to test it in order to deploy my image into the same kubernetes:
apiVersion: apps/v1
kind: Deployment
metadata:
name: covid-backend
namespace: skaffold
spec:
replicas: 3
selector:
matchLabels:
app: covid-backend
template:
metadata:
labels:
app: covid-backend
spec:
containers:
- image: registry-docker-registry.registry/skaffold-covid-backend:c5dfd81-dirty#sha256:76312ebc62c4b3dd61b4451fe01b1ecd2e6b03a2b3146c7f25df3d3cfb4512cd
name: covid-backend
ports:
- containerPort: 8080
Then, I've tried to deploy it:
$ cat pod.yaml | kubectl apply -f -
However, kubernetes isn't able to reach registry:
Extract of kubectl get events:
6s Normal Pulling pod/covid-backend-774bd78db5-89vt9 Pulling image "registry-docker-registry.registry/skaffold-covid-backend:c5dfd81-dirty#sha256:76312ebc62c4b3dd61b4451fe01b1ecd2e6b03a2b3146c7f25df3d3cfb4512cd"
1s Warning Failed pod/covid-backend-774bd78db5-89vt9 Failed to pull image "registry-docker-registry.registry/skaffold-covid-backend:c5dfd81-dirty#sha256:76312ebc62c4b3dd61b4451fe01b1ecd2e6b03a2b3146c7f25df3d3cfb4512cd": rpc error: code = Unknown desc = failed to pull and unpack image "registry-docker-registry.registry/skaffold-covid-backend#sha256:76312ebc62c4b3dd61b4451fe01b1ecd2e6b03a2b3146c7f25df3d3cfb4512cd": failed to resolve reference "registry-docker-registry.registry/skaffold-covid-backend#sha256:76312ebc62c4b3dd61b4451fe01b1ecd2e6b03a2b3146c7f25df3d3cfb4512cd": failed to do request: Head https://registry-docker-registry.registry/v2/skaffold-covid-backend/manifests/sha256:76312ebc62c4b3dd61b4451fe01b1ecd2e6b03a2b3146c7f25df3d3cfb4512cd: dial tcp: lookup registry-docker-registry.registry: Try again
1s Warning Failed pod/covid-backend-774bd78db5-89vt9 Error: ErrImagePull
As you can see, kubernetes is not able to get access to the internal deployed registry...
Any ideas?
I would recommend to follow docs from k3d, they are here.
More precisely this one
Using your own local registry
If you don't want k3d to manage your registry, you can start it with some docker commands, like:
docker volume create local_registry
docker container run -d --name registry.local -v local_registry:/var/lib/registry --restart always -p 5000:5000 registry:2
These commands will start you registry in registry.local:5000. In order to push to this registry, you will need to add the line at /etc/hosts as we described in the previous section . Once your registry is up and running, we will need to add it to your registries.yaml configuration file. Finally, you must connect the registry network to the k3d cluster network: docker network connect k3d-k3s-default registry.local. And then you can check you local registry.
Pushing to your local registry address
The registry will be located, by default, at registry.local:5000 (customizable with the --registry-name and --registry-port parameters). All the nodes in your k3d cluster can resolve this hostname (thanks to the DNS server provided by the Docker daemon) but, in order to be able to push to this registry, this hostname but also be resolved from your host.
The easiest solution for this is to add an entry in your /etc/hosts file like this:
127.0.0.1 registry.local
Once again, this will only work with k3s >= v0.10.0 (see the section below when using k3s <= v0.9.1)
Local registry volume
The local k3d registry uses a volume for storying the images. This volume will be destroyed when the k3d registry is released. In order to persist this volume and make these images survive the removal of the registry, you can specify a volume with the --registry-volume and use the --keep-registry-volume flag when deleting the cluster. This will create a volume with the given name the first time the registry is used, while successive invocations will just mount this existing volume in the k3d registry container.
Docker Hub cache
The local k3d registry can also be used for caching images from the Docker Hub. You can start the registry as a pull-through cache when the cluster is created with --enable-registry-cache. Used in conjuction with --registry-volume/--keep-registry-volume can speed up all the downloads from the Hub by keeping a persistent cache of images in your local machine.
Testing your registry
You should test that you can
push to your registry from your local development machine.
use images from that registry in Deployments in your k3d cluster.
We will verify these two things for a local registry (located at registry.local:5000) running in your development machine. Things would be basically the same for checking an external registry, but some additional configuration could be necessary in your local machine when using an authenticated or secure registry (please refer to Docker's documentation for this).
Firstly, we can download some image (like nginx) and push it to our local registry with:
docker pull nginx:latest
docker tag nginx:latest registry.local:5000/nginx:latest
docker push registry.local:5000/nginx:latest
Then we can deploy a pod referencing this image to your cluster:
cat <<EOF | kubectl apply -f -
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-test-registry
labels:
app: nginx-test-registry
spec:
replicas: 1
selector:
matchLabels:
app: nginx-test-registry
template:
metadata:
labels:
app: nginx-test-registry
spec:
containers:
- name: nginx-test-registry
image: registry.local:5000/nginx:latest
ports:
- containerPort: 80
EOF
Then you should check that the pod is running with kubectl get pods -l "app=nginx-test-registry".
Additionaly there are 2 github links worth visting
K3d not able resolve dns
You could try to use an answer provided by #rjshrjndrn, might solve your issue with dns.
docker images are not pulled from docker repository behind corporate proxy
Open github issue on k3d with same problem as yours.

Docker Desktop + k8s plus https proxy multiple external ports to pods on http in deployment?

I'm trying to do a straight up thing that I would think is simple. I need to have https://localhost:44301, https://localhost:5002, https://localhost:5003 to be listened to in my k8s environment in docker desktop, and be proxied using a pfx file/password that I specify and have it forward by the port to pods listening on specific addresses (could be port 80, doesn't matter)
The documentation is mind numbingly complex for what looks like it should be straight forward. I can get the pods running, I can use kubectl port-forward and they work fine, but I can't figure out how to get ingress working with ha-proxy or nginx or anything else in a way that makes any sense.
Can someone do an ELI5 telling me how to turn this on? I'm on Windows 10 2004 with WSL2 and Docker experimental so I should have access to the ingress stuff they reference in the docs and make clear as mud.
Thanks!
As discussed in the comments this is a community wiki answer:
I have managed to create Ingress resource in Kubernetes on Docker in Windows.
Steps to reproduce:
Enable Hyper-V
Install Docker for Windows and enable Kubernetes
Connect kubectl
Enable Ingress
Create deployment
Create service
Create ingress resource
Add host into local hosts file
Test
Enable Hyper-V
From Powershell with administrator access run below command:
Enable-WindowsOptionalFeature -Online -FeatureName Microsoft-Hyper-V -All
System could ask you to reboot your machine.
Install Docker for Windows and enable Kubernetes
Install Docker application with all the default options and enable Kubernetes
Connect kubectl
Install kubectl .
Enable Ingress
Run this commands:
kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/static/mandatory.yaml
kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/static/provider/cloud-generic.yaml
Edit: Make sure no other service is using port 80
Restart your machine. From a cmd prompt running as admin, do:
net stop http
Stop the listed services using services.msc
Use: netstat -a -n -o -b and check for other processes listening on port 80.
Create deployment
Below is simple deployment with pods that will reply to requests:
apiVersion: apps/v1
kind: Deployment
metadata:
name: hello
spec:
selector:
matchLabels:
app: hello
version: 2.0.0
replicas: 3
template:
metadata:
labels:
app: hello
version: 2.0.0
spec:
containers:
- name: hello
image: "gcr.io/google-samples/hello-app:2.0"
env:
- name: "PORT"
value: "50001"
Apply it by running command:
$ kubectl apply -f file_name.yaml
Create service
For pods to be able for you to communicate with them you need to create a service.
Example below:
apiVersion: v1
kind: Service
metadata:
name: hello-service
spec:
type: NodePort
selector:
app: hello
version: 2.0.0
ports:
- name: http
protocol: TCP
port: 80
targetPort: 50001
Apply this service definition by running command:
$ kubectl apply -f file_name.yaml
Create Ingress resource
Below is simple Ingress resource using service created above:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: hello-ingress
spec:
rules:
- host: kubernetes.docker.internal
http:
paths:
- path: /
backend:
serviceName: hello-service
servicePort: http
Take a look at:
spec:
rules:
- host: hello-test.internal
hello-test.internal will be used as the hostname to connect to your pods.
Apply your Ingress resource by invoking command:
$ kubectl apply -f file_name.yaml
Add host into local hosts file
I found this Github link that will allow you to connect to your Ingress resource by hostname.
To achieve that add a line 127.0.0.1 hello-test.internal to your C:\Windows\System32\drivers\etc\hosts file and save it.
You will need Administrator privileges to do that.
Edit: The newest version of Docker Desktop for Windows already adds a hosts file entry:
127.0.0.1 kubernetes.docker.internal
Test
Display the information about Ingress resources by invoking command:
kubectl get ingress
It should show:
NAME HOSTS ADDRESS PORTS AGE
hello-ingress hello-test.internal localhost 80 6m2s
Now you can access your Ingress resource by opening your web browser and typing
http://kubernetes.docker.internal/
The browser should output:
Hello, world!
Version: 2.0.0
Hostname: hello-84d554cbdf-2lr76
Hostname: hello-84d554cbdf-2lr76 is the name of the pod that replied.
If this solution is not working please check connections with the command:
netstat -a -n -o
(with Administrator privileges) if something is not using port 80.

How to edit code in kubernetes pod containers using VS Code?

Typically, if I have a remote server, I could access it using ssh, and VS Code gives a beautiful extension for editing and debugging codes for the remote server. But when I create pods in Kuberneters, I can't really ssh into the container and so I cannot edit the code inside the pod or machine. And the kuberneters plugin in VSCode does not really help because the plugin is used to deploy the code. So, I was wondering whether there is a way edit codes inside a pod using VSCode.
P.S. Alternatively if there is a way to ssh into a pod in a kuberneters, that will do too.
If your requirement is "kubectl edit xxx" to use VSCode.
The solution is:
For Linux,macos: export EDITOR='code --wait'
For Windows: set EDITOR=code --wait
Kubernetes + Remote Development extensions now allow:
attaching to k8s pods
open remote folders
execute remotely
debug on remote
integrated terminal into remote
must have:
kubectl
docker (minimum = docker cli - Is it possible to install only the docker cli and not the daemon)
required vscode extentions:
Kubernetes. https://marketplace.visualstudio.com/items?itemName=ms-kubernetes-tools.vscode-kubernetes-tools
Remote Development - https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.vscode-remote-extensionpack
Well a pod is just a unit of deployment in kubernetes which means you can tune the containers inside it to receive an ssh connection.
Let's start by getting a docker image that allows ssh connections. rastasheep/ubuntu-sshd:18.04 image is quite nice for this. Create a deployment with it.
---
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: debugger
name: debugger
namespace: default
spec:
replicas: 1
selector:
matchLabels:
app: debugger
template:
metadata:
creationTimestamp: null
labels:
app: debugger
spec:
containers:
- name: debugger
image: rastasheep/ubuntu-sshd:18.04
imagePullPolicy: "Always"
hostname: debugger
restartPolicy: Always
Now let's create a service of type LoadBalancer such that we can access the pod remotely.
---
apiVersion: v1
kind: Service
metadata:
namespace: default
labels:
app: debugger
name: debugger
spec:
type: LoadBalancer
ports:
- name: "22"
port: 22
targetPort: 22
selector:
app: debugger
status:
loadBalancer: {}
Finally, get the external ip address by running kubectl get svc | grep debugger and use it to test the ssh connection ssh root#external_ip_address
Note the user / pass of this docker image is root / root respectively.
UPDATE
Nodeport example. I tested this and it worked running ssh -p30036#ipBUT I had to enable a firewall rule to make it work. So, the nmap command that I gave you has the answer. Obviously the machines that run kubernetes don't allow inbound traffic on weird ports. Talk to them such that they can give you an external ip address or at least a port in a node.
---
apiVersion: v1
kind: Service
metadata:
name: debugger
namespace: default
labels:
app: debugger
spec:
type: NodePort
ports:
- name: "ssh"
port: 22
nodePort: 30036
selector:
app: debugger
status:
loadBalancer: {}
As mentioned in some of the other answers, you can do this although it is fraught with danger as the cluster can/will replace pods regularly and when it does, it starts a new pod idempotently from the configuration which will not have your changes.
The command below will get you a shell session in your pod , which can sometimes be helpful for debugging if you don't have adequate monitoring/local testing facilities to recreate an issue.
kubectl --namespace=exmaple exec -it my-cool-pod-here -- /bin/bash
Note You can replace the command with any tool that is installed in your container (python3, sh, bash, etc). Also know that that some base images like alpine wont have bash/shell installed be default.
This will open a bash session in the running container on the cluster, assuming you have the correct k8s RBAC permissions.
There is an Cloud Code extension available for VS Code that will serve your purpose.
You can install it in your Visual Studio Code to interact with your Kubernetes cluster.
It allows you to create minikube cluster, Google GKE, Amazon EKS or Azure AKS and manage it from VS Code (you can access cluster information, stream/view logs from pods and open interactive terminal to the container).
You can also enable continuous deployment so it will continuously watch for changes in your files, rebuild the container and redeploy application to the cluster.
It is well explained in Documentation
Hope it will be useful for your use case.

minikube - how to access pod via pod ip using curl

I use minikube to create local kubernetes cluster.
I create ReplicationController via webapp-rc.yaml file.
apiVersion: v1
kind: ReplicationController
metadata:
name: webapp
spec:
replicas: 2
template:
metadata:
name: webapp
labels:
app: webapp
spec:
containers:
- name: webapp
image: tomcat
ports:
- containerPort: 8080
and, I print the pods' ip to stdout:
kubectl get pods -l app=webapp -o yaml | grep podIP
podIP: 172.17.0.18
podIP: 172.17.0.1
and, I want to access pod using curl
curl 172.17.0.18:8080
But, the stdout give me: curl: (52) Empty reply from server
I know I can access my application in docker container in pod via service.
I find this code in a book. But the book does not give the context for executing this code.
Using minikube, how to access pod via pod ip using curl in host machine?
update 1
I find a way using kubectl proxy:
➜ ~ kubectl proxy
Starting to serve on 127.0.0.1:8001
and then I can access pod via curl like this:
curl http://localhost:8001/api/v1/namespaces/default/pods/webapp-jkdwz/proxy/
webapp-jkdwz can be found by command kubectl get pods -l app=webapp
update 2
minikube ssh - log into minikube VM
and then, I can use curl <podIP>:<podPort>, for my case is curl 172.17.0.18:8080
First of all, tomcat image expose port 8080 not 80, so the correct YAML would be:
apiVersion: v1
kind: ReplicationController
metadata:
name: webapp
spec:
replicas: 2
template:
metadata:
name: webapp
labels:
app: webapp
spec:
containers:
- name: webapp
image: tomcat
ports:
- containerPort: 8080
minikube is executed inside a virtual machine, so the curl 172.17.0.18:8080 would only work from inside that virtual machine.
You can always create a service to expose your apps:
kubectl expose rc webapp --type=NodePort
And use the following command to get the URL:
minikube service webapp --url
If you need to query a specific pod, use port forwarding:
kubectl port-forward <POD NAME> 8080
Or just ssh into minikube's virtual machine and query from there.
That command is correct, but it only works from a machine that has access to the overlay network. (In case of minikube the host machine does not have that by default).
You can set up a proxy to your pod with:
kubectl port-forward [name of your pod] [pod port]
Thereafter you can (from another shell):
curl 127.0.0.1:port/path
See also: https://kubernetes.io/docs/tasks/access-application-cluster/port-forward-access-application-cluster/#forward-a-local-port-to-a-port-on-the-pod

How to connect MySQL running on Kubernetes

I have deployed my application on Google gcloud container engine. My application required MySQL. Application is running fine and connecting to MySQL correctly.
But I want to connect MySQL database from my local machine using MySQL Client (Workbench, or command line), Can some one help me how to expose this to local machine? and how can I open MySQL command line on gcloud shell ?
I have run below command but external ip is not there :
$ kubectl get deployment
NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE
app-mysql 1 1 1 1 2m
$ kubectl get pods
NAME READY STATUS RESTARTS AGE
app-mysql-3323704556-nce3w 1/1 Running 0 2m
$ kubectl get service
NAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE
app-mysql 11.2.145.79 <none> 3306/TCP 23h
EDIT
I am using below yml file:
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: app-mysql
spec:
replicas: 1
template:
metadata:
labels:
app: app-mysql
spec:
volumes:
- name: data
emptyDir: {}
containers:
- name: mysql
image: mysql:5.6.22
env:
- name: MYSQL_USER
value: root
- name: MYSQL_DATABASE
value: appdb
ports:
- containerPort: 3306
volumeMounts:
- name: data
mountPath: /var/lib/mysql/
---
apiVersion: v1
kind: Service
metadata:
name: app-mysql
spec:
selector:
app: app-mysql
ports:
- port: 3306
Try the kubectl port-forward command.
In your case; kubectl port-forward app-mysql-3323704556-nce3w 3306:3306
See The documentation for all available options.
There are 2 steps involved:
1 ) You first perform port forwarding from localhost to your pod:
kubectl port-forward <your-mysql-pod-name> 3306:3306 -n <your-namespace>
2 ) Connect to database:
mysql -u root -h 127.0.0.1 -p <your-password>
Notice that you might need to change 127.0.0.1 to localhost - depends on your setup.
If host is set to:
localhost - then a socket or pipe is used.
127.0.0.1 - then the client is forced to use TCP/IP.
You can check if your database is listening for TCP connections with netstat -nlp.
Read more in:
Cant connect to local mysql server through socket tmp mysql sock
Can not connect to server
To add to the above answer, when you add --address 0.0.0.0 kubectl should open 3306 port to the INTERNET too (not only localhost)!
kubectl port-forward POD_NAME 3306:3306 --address 0.0.0.0
Use it with caution for short debugging sessions only, on development systems at most. I used it in the following situation:
colleague who uses Windows
didn't have ssh key ready
environment was a playground I was not afraid to expose to the world
You need to add a service to your deployment. The service will add a load balancer with a public ip in front of your pod, so it can be accessed over the public internet.
See the documentation on how to add a service to a Kubernetes deployment. Use the following code to add a service to your app-mysql deployment:
kubectl expose deployment/app-mysql
You may also need to configure your MySQL service so it allows remote connections. See this link on how to enable remote access on MySQL server: