Summary
trying to get minikube-test-ifs.com to map to my deployment using minikube.
What I Did
minikube start
minikube addons enable ingress
kubectl apply -f <path-to-yaml-below>
kubectl get ingress
Added ingress ip mapping to /etc/hosts file in form <ip> minikube-test-ifs.com
I go to chrome and enter minikube-test-ifs.com and it doesn't load.
I get "site can't be reached, took too long to respond"
yaml file
note - it's all in the default namespace, I don't know if that's a problem.
There may be a problem in this yaml, but I checked and double checled and see no potential error... unless I'm missing something
apiVersion: apps/v1
kind: Deployment
metadata:
name: test-deployment
labels:
app: test
spec:
replicas: 1
selector:
matchLabels:
app: test
template:
metadata:
labels:
app: test
spec:
containers:
- name: test
image: nginx
ports:
- name: client
containerPort: 3000
---
apiVersion: v1
kind: Service
metadata:
name: test-service
spec:
selector:
app: test
ports:
- name: client
protocol: TCP
port: 3000
targetPort: 3000
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: test-ingress
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /
spec:
rules:
- host: minikube-test-ifs.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: test-service
port:
number: 3000
OS
Windows 10
Other Stuff
I checked Minikube with ingress example not working but I already added to my /etc/hosts and I also tried removing the spec.host but that still doesn't work...
also checked Minikube Ingress (Nginx Controller) not working but that person has his page already loading so not really relevent to me from what I can tell
Any Ideas?
I watched so many Youtube tutorials on this and I follow everything perfectly. I'm still new to this but I don't see a reason for it not working?
Edit
When I run kubectl describe ingress <ingress> I get:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Sync 8s (x5 over 19m) nginx-ingress-controller Scheduled for sync
How do I get it to sync? Is there a problem since it's been "Scheduled for sync" for a long time
Overview
Ingress addon for Minikube using docker driver only works on linux
Docker for Windows uses Hyper-V, therefore, if the Docker daemon is running, you will not be able to use VM platforms such as VirtualBox or VMware
If you have Windows Pro, Enterprise or Education, you may be able to get it working if you use Hyper-V as your minikube cluster (see Solution 1)
If you don't want to upgrade Windows, you can open a minikube cluster on a Linux Virtual Machine and run all your tests there. This will require you to configure some Windows VM settings in order to get your VM's to run (see Solution 2). Note that you can only run either Docker or a VM platform (other than Hyper-V) but not both (See The Second Problem for why this is the case).
The Problem
For those of you who are in the same situation as I was, the problem lies in the fact that the minikube ingress addon only works for Linux OS when using the docker driver (Thanks to #rriovall for showing me this documentation).
The Second Problem
So the solution should be simple, right? Just use a different driver and it should work. The problem here is, when Docker is intalled on Windows, it uses the built in Hyper-V virtualization technology which by default seems to disable all other virtualizatioin tech.
I have tested this hypothesis and this seems to be the case. When the Docker daeomon is running, I am unable to boot any virtual machine that I have. For instance, I get an error when I tried to run my VM's on VirtualBox and on VMWare.
Furthermore, when I attemt to start a minikube cluster using the virtualbox driver, it gets stuck "booting the kernel" and then I get a This computer doesn't have VT-X/AMD-v enabled error. This error is false as I do have VT-X enabled (I checked my BIOS). This is most likely due to the fact that when Hyper-V is enabled, all other types of virtualization tech seems to be disabled.
For my personal machine, when I do a search for "turn windows features on or off" the Docker daemon enabled "Virtual Machine Platform" and then asked me to restart my computer. This happened when I installed Docker. As a test, I turned off both "Virtual Machine Platform" and "Windows Hypervsor Platform" features and restarted my computer.
What happened when I did that? The Docker daemon stopped running and I could no longer work with docker, however, I was able to open my VM's and I was able to start my minikube cluster with virtualbox as the driver. The problem? Well, Docker doesn't work so when my cluster tries to pull the docker image I am using, it won't be able to.
So here lies the problem. Either you have VM tech enables and Docker disabled, or you have VM tech (other than Hyper-V, I'll touch on that soon) disabled and Docker enabled. But you can't have both.
Solution 1 (Untested)
The simplest solution would probably be upgrading to Windows Pro, Enterpriseor or Education. The Hyper-V platform is not accessable on normal Windows. Once you have upgraded, you should be able to use Hyper-V as your driver concurrently with the Docker daemon. This, in theory, should make the ingress work.
Solution 2 (Tested)
If you're like me and don't want to do a system upgrade for something so miniscule, there's another solution.
First, search your computer for the "turn windows features on or off" section and disable "Virtual Machine Platform" and "Windows Hypervisor Platform" and restart your computer. (See you in a bit :D)
After that, install a virtual machine platform on your computer. I prefer VirtualBox but you can also use others such as VMware.
Once you have a VM platform installed, add a new Linux VM. I would recommend either Debian or Ubuntu. If you are unfamiliar with how to set up a VM, this video will show you how to do so. This will be the general set up for most iso images.
After you have your VM up and running, download minikube and Docker on it. Be sure to install the correct version for your VM (for Debian, install Debian versions, for Ubuntu, install Ubuntu versions. Some downlaods may just be general Linux wich should work on most Linux versions).
Once you have everything installed, create a minikube cluster with docker as the driver, apply your Kubernetes configurations (deployment, service and ingress). Configure your /etc/hosts file and go to your browser and it should work. If you don't know how to set up an ingress, you can watch this video for an explanation on what an ingress is, how it works, and an example of how to set it up.
Related
I am trying to run a local cluster on Mac with M1 chip using Minikube (Docker driver). I enabled ingress addon in Minikube, I have a separate terminal in which I'm running minikube tunnel and I enabled Minikube dashboard, which I want to expose using Ingress.
This is my configuration file:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: dashboard-ingress
namespace: kubernetes-dashboard
spec:
rules:
- host: dashboard.com
http:
paths:
- backend:
service:
name: kubernetes-dashboard
port:
number: 80
pathType: Prefix
path: /
I also put "dashboard.com" in my /etc/hosts file and it's actually resolving to the right IP, but it's not responding when I put "http://dashboard.com" in a browser or when I try to ping it and I always receive a timeout.
NOTE: when I run minikube tunnel I get
❗ The service/ingress dashboard-ingress requires privileged ports to be exposed: [80 443]
🔑 sudo permission will be asked for it.
I insert my sudo password and then nothing gets printed afterwards. Not sure if this is is an issue or the expected behavior.
What am I doing wrong?
I had the same behavior, and apparently what's needed for minikube tunnel to work is to map "127.0.0.1" in /etc/hosts, instead of the output from minikube ip or the ingress description.
that fixed it for me
had a similar issue on mac m1, initialy tried addon ingress-dns but then realised while it can be enabled its not currently working or supported using the docker driver https://github.com/kubernetes/minikube/issues/7332#issuecomment-608133325
some other mac intel users have got it working using hyperkit driver but thats not available for mac m1 yet
my answer for now is to use minikube tunnel https://minikube.sigs.k8s.io/docs/handbook/accessing/ and add entry to /etc/hosts for ingress, also have to pass in the cluster name using the -p parameter eg: minikube tunnel --cleanup -p <CLUSTER_NAME>
How do I expose an ingress when running kubernetes with minikube in windows 10?
I have enabled the minikube ingress add on.
My ingress is running here...
NAME CLASS HOSTS ADDRESS PORTS AGE
helmtest-ingress nginx helmtest.info 192.168.49.2 80 37m
I have added my hosts entry...
192.168.49.2 helmtest.info
I just get nothing when attempting to browse or ping either 192.168.49.2 or helmtest.info
My ingress looks like the following
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: helmtest-ingress
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /$1
spec:
rules:
- host: helmtest.info
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: helmtest-service
port:
number: 80
My service looks like the following...
apiVersion: v1
kind: Service
metadata:
name: helmtest-service
labels:
app: helmtest-service
spec:
type: ClusterIP
selector:
app: helmtest
ports:
- port: 80
targetPort: 80
protocol: TCP
I can access my service successfully in the browser after running minikube service helmtest-service --url
If I run minikube tunnel it just hangs here....
minikube tunnel
❗ Access to ports below 1024 may fail on Windows with OpenSSH clients older than v8.1. For more information, see: https://minikube.sigs.k8s.io/docs/handbook/accessing/#access-to-ports-1024-on-windows-requires-root-permission
🏃 Starting tunnel for service helmtest-ingress.
Where am I going wrong here?
OP didn't provide further information so I will provide answer based on the current information.
You can run Ingress on Minikube using the $ minikube addons enable ingress command. However, ingress has more addons, like Ingress DNS using minikube addons enabled ingress-dns. In Minikube documentation you can find more details about this addon and when you should use it.
Minikube has quite a well described section about tunnel. Quite important fact about the tunnel is that it must be run in a separate terminal window to keep the LoadBalancer running.
Services of type LoadBalancer can be exposed via the minikube tunnel command. It must be run in a separate terminal window to keep the LoadBalancer running. Ctrl-C in the terminal can be used to terminate the process at which time the network routes will be cleaned up.
This part is described in Accessing apps documentation.
As OP mention
I can access my service successfully in the browser after running minikube service helmtest-service --url
If I run minikube tunnel it just hangs here....
Possible Solution
You might use the old version of SSH, update it.
You are using ports <1024. This situation it's described in this known issue part. Try to use higher port like 5000 like in this example
It might look like it just hangs, but you need a separate terminal window. Maybe it works correctly but you have to use another terminal
Useful links
How do I expose ingress to my local machine? (minikube on windows)
Cannot export a IP in minikube and haproxy loadBalancer - using minikube tunnel
It might be the host file missing minikube ip address with your host name. If ingress cannot resolve the hostname you set in yaml file it just stays in the schedule to sync phase
Similar answer
I am trying to deploy an image from my private registry that's hosted on my local network and pointed using my local machines /etc/hosts file.
I am getting the resolution error as below:
Failed to pull image "gitlab.example.com:5050/group/project:latest": rpc error: code = Unknown desc = failed to resolve image
My /etc/hosts file contains:
192.168.1.100 gitlab.example.com
Using docker the pull/push works perfectly fine as the resolution happens using /etc/hosts
I've tried editing corefile of coredns to make the resolution happen, but it isn't working.
Can someone point me in right direction over here.
You can try to use hostAliases 💡 and add a host entry to your pod. Kubernetes will not honor the /etc/hosts file from the hosts/nodes. Either the one in the pod or resolves through CoreDNS. For example:
apiVersion: v1
kind: Pod
metadata:
name: hostaliases-pod
spec:
restartPolicy: Never
hostAliases:
- ip: "192.168.1.100" 👈
hostnames:
- "gitlab.example.com" 👈
containers:
- name: gitlab-hosts
image: myimage
command:
- mygitlabjob
args:
- "arg1"
Update:
I think see the problem here. microk8s on Ubuntu runs in a snap. That means it's confined/sandboxed in a container of its own. This also means that it probably doesn't care about your machine's /etc/hosts file. Unfortunately, snap's file systems are mounted as read-only for security reasons and to prevent tampering.
○ → pwd
/snap/microk8s/current
○ → sudo touch hosts
touch: cannot touch 'hosts': Read-only file system
If you'd like to use a private registry this way, some recommendations:
Ask your system admin to add that entry into your local DNS server, or add it if you are the system admin.
Use an alternative small K8s distro that uses Docker.
KinD
minikube
Build your own microk8s snap with a modified /etc/hosts file (hard)
Typically, if I have a remote server, I could access it using ssh, and VS Code gives a beautiful extension for editing and debugging codes for the remote server. But when I create pods in Kuberneters, I can't really ssh into the container and so I cannot edit the code inside the pod or machine. And the kuberneters plugin in VSCode does not really help because the plugin is used to deploy the code. So, I was wondering whether there is a way edit codes inside a pod using VSCode.
P.S. Alternatively if there is a way to ssh into a pod in a kuberneters, that will do too.
If your requirement is "kubectl edit xxx" to use VSCode.
The solution is:
For Linux,macos: export EDITOR='code --wait'
For Windows: set EDITOR=code --wait
Kubernetes + Remote Development extensions now allow:
attaching to k8s pods
open remote folders
execute remotely
debug on remote
integrated terminal into remote
must have:
kubectl
docker (minimum = docker cli - Is it possible to install only the docker cli and not the daemon)
required vscode extentions:
Kubernetes. https://marketplace.visualstudio.com/items?itemName=ms-kubernetes-tools.vscode-kubernetes-tools
Remote Development - https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.vscode-remote-extensionpack
Well a pod is just a unit of deployment in kubernetes which means you can tune the containers inside it to receive an ssh connection.
Let's start by getting a docker image that allows ssh connections. rastasheep/ubuntu-sshd:18.04 image is quite nice for this. Create a deployment with it.
---
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: debugger
name: debugger
namespace: default
spec:
replicas: 1
selector:
matchLabels:
app: debugger
template:
metadata:
creationTimestamp: null
labels:
app: debugger
spec:
containers:
- name: debugger
image: rastasheep/ubuntu-sshd:18.04
imagePullPolicy: "Always"
hostname: debugger
restartPolicy: Always
Now let's create a service of type LoadBalancer such that we can access the pod remotely.
---
apiVersion: v1
kind: Service
metadata:
namespace: default
labels:
app: debugger
name: debugger
spec:
type: LoadBalancer
ports:
- name: "22"
port: 22
targetPort: 22
selector:
app: debugger
status:
loadBalancer: {}
Finally, get the external ip address by running kubectl get svc | grep debugger and use it to test the ssh connection ssh root#external_ip_address
Note the user / pass of this docker image is root / root respectively.
UPDATE
Nodeport example. I tested this and it worked running ssh -p30036#ipBUT I had to enable a firewall rule to make it work. So, the nmap command that I gave you has the answer. Obviously the machines that run kubernetes don't allow inbound traffic on weird ports. Talk to them such that they can give you an external ip address or at least a port in a node.
---
apiVersion: v1
kind: Service
metadata:
name: debugger
namespace: default
labels:
app: debugger
spec:
type: NodePort
ports:
- name: "ssh"
port: 22
nodePort: 30036
selector:
app: debugger
status:
loadBalancer: {}
As mentioned in some of the other answers, you can do this although it is fraught with danger as the cluster can/will replace pods regularly and when it does, it starts a new pod idempotently from the configuration which will not have your changes.
The command below will get you a shell session in your pod , which can sometimes be helpful for debugging if you don't have adequate monitoring/local testing facilities to recreate an issue.
kubectl --namespace=exmaple exec -it my-cool-pod-here -- /bin/bash
Note You can replace the command with any tool that is installed in your container (python3, sh, bash, etc). Also know that that some base images like alpine wont have bash/shell installed be default.
This will open a bash session in the running container on the cluster, assuming you have the correct k8s RBAC permissions.
There is an Cloud Code extension available for VS Code that will serve your purpose.
You can install it in your Visual Studio Code to interact with your Kubernetes cluster.
It allows you to create minikube cluster, Google GKE, Amazon EKS or Azure AKS and manage it from VS Code (you can access cluster information, stream/view logs from pods and open interactive terminal to the container).
You can also enable continuous deployment so it will continuously watch for changes in your files, rebuild the container and redeploy application to the cluster.
It is well explained in Documentation
Hope it will be useful for your use case.
I'm kind of new to the Kubernetes world. In my project we are planning to use windows containers(.net full framework) in short term and linux containers(.net core) for the long run.
We have a K8 cluster provided by infrastructure team and the cluster has mix of Linux and Windows nodes. I just wanted to know how my windows containers will only be deployed to windows nodes in the K8 cluster. Is it handled by K8 or Do I need anything else ?
Below are the details from the Kubernetes Windows Documentation.
Because your cluster has both Linux and Windows nodes, you must explicitly set the nodeSelector constraint to be able to schedule pods to Windows nodes. You must set nodeSelector with the label beta.kubernetes.io/os to the value windows; see the following example:
apiVersion: v1
kind: Pod
metadata:
name: iis
labels:
name: iis
spec:
containers:
- name: iis
image: microsoft/iis:windowsservercore-1709
ports:
- containerPort: 80
nodeSelector:
"kubernetes.io/os": windows
You would need to add following lines to your YAML file. Details are available here https://kubernetes.io/docs/getting-started-guides/windows/
nodeSelector:
"beta.kubernetes.io/os": windows