I am very new to the idea of kubernetes.
I found some good tutorials online to get my kubernetes cluster up & running.
Right Now I want to add a kubernetes dashboard to my cluster so that it would be easy and good to have a page where I can watch how my pods and node's react (even do I'm more of a CLI guy, some GUI is not bad).
I've downloaded the dashboard pod and it's up and running. Because the kubernetes cluster is running on a Raspberry Pi cluster I've set up a NodePort to access it from outside my cluster. But I've come to some problems where I can't find any problems too online.
In my Linux host machine I can access the kubernetes dashboard but somehow in my Linux machine, my browsers won't add the cert exception.
Some people online are convinced that NodePort is not safe enough. So I've done some research on other possibilities. I am very interested in the ingress controller to connect my dashboard. But I didn't find any good and full documentation how to set up an ingress controller (and more importantly what is happening, cause there are a lot of yaml files online and they say just run this but i have no clue what he is doing).
Can someone direct me to the right documentation/tutorial / or give me any help for my Kube dashboard?
Someone in an other forum send me this tutorial which was very helpfull. I'll share it here also for all the people who've came to this post with the same question.
https://akomljen.com/kubernetes-nginx-ingress-controller/
The first thing you will need to use in the deployment is Ingress so lets start with it.
Firstly you should create an Ingress controller, you can find the Installation Guide here
The most relevant is the first part - Generic Deployment which includes following:
Namespace for the Ingress controller installation:
curl https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/namespace.yaml \ | kubectl apply -f -
Default backend for Ingress controller:
curl https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/default-backend.yaml \ | kubectl apply -f -
And config maps:
curl https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/configmap.yaml \ | kubectl apply -f -
curl https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/tcp-services-configmap.yaml \ | kubectl apply -f -
curl https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/udp-services-configmap.yaml \ | kubectl apply -f -
Because you deployed your cluster on Raspberry Pi, all of this needs to be created manually.
After the Ingress controller is installed you can deploy specific configuration for your Ingress with rules to route traffic to your service.
Here is an example of Ingress yaml file:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: endpoint-to-the-world
annotations:
ingress.kubernetes.io/rewrite-target: /
spec:
rules:
- host: your-external-address-for-the-cluster
http:
paths:
- path: /console
backend:
serviceName: kubernetes-dashboard
servicePort: 443
- path: /some-other-path
backend:
serviceName: different-service
servicePort: 22
This will act like an external proxy for your cluster and you can route all traffic to any service. More details can be read here.
This should be enough to have Kubernetes Dashboard exposed.
Related
To start playing with Kubernetes I created a small project where I create a three nodes cluster on my local machine using Vagrant and Vbox:
https://github.com/sasadangelo/k8s-cluster
Just typing vagrant up my cluster is up and running and I can start to play with it. I got familiar with the main concepts and commands and created a simple "Hello World" application composed by a docker image with an NGINX web server where you can connect via browser and a "Hello World" message appears with the hostname. The hostname helps me to understand which Pod responded to my request.
Here the project:
https://github.com/sasadangelo/k8s-tutorials/tree/master/hello-k8s
Currently, I can create a deployment with kubectl create exposing it as a service with kubectl expose and then increase replica with kubect scale. At the moment, I am able to use use NodePort service, so with kubectl describe service I can see on which IP the 5 Pods listen, I connect to them and everything works fine.
My problem is that now I want to load balance the traffic. I want to connect to a single IP and as soon as I click on the browser Reload button I want to see that different Pod names appear.
I know that Kubernetes by default offers only NodePort service and if I want a load balancer I need something like an Ingress. I also know that a simple reverse proxy like Ingress Nginx is a good implementation for my needs.
However, I read lot of tutorials but I have hard time understanding how I should configure it to achieve what I need.
Here code I have so far:
https://github.com/sasadangelo/k8s-cluster/tree/master/ingress
Can anyone help me how to fix Ingress code to achieve what I need?
Kube Proxy which is a kubernetes component provides load balancing at L4 layer via iptables load balancing.So with ClusterIP type service itself you get load balancing between pods. But with clusterIP type service you can not access it from outside the cluster i.e from a browser. Nodeport type service provides access to the pods from outside the cluster with load balancing at L4 layer by kube proxy.
LoadBalancer and Ingress provides load balancing and rule based routing at L7 layer if that's what you are looking for.
Looking at your code you have yamls for nginx ingress controller and exposed it as Nodeport. What you need next is to create an ingress resource
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: example-ingress
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /$1
spec:
rules:
- host: hello-world.info
http:
paths:
- path: /
backend:
serviceName: web
servicePort: 8080
Once you create above ingress and you have a ClusterIP type service for your pod with a name web which exposed port 8080, You should be able to access it.
Here is the doc on using nginx ingress for an hello world application on Minikube which you can follow to make ingress work in your setup as well.
Here is what I have done in detail to solve the issue. First of all, I installed Nginx Ingress resources:
kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/nginx-0.30.0/deploy/static/mandatory.yaml
kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/nginx-0.30.0/deploy/static/provider/baremetal/service-nodeport.yaml
then I create an Nginx Ingress resource in a ingress-nginx.yaml file:
---
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: hello-k8s-ingress
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /$1
spec:
rules:
- host: hello-k8s.info
http:
paths:
- path: /
backend:
serviceName: hello-k8s
servicePort: 80
and installed it with the command:
kubectl apply -f ingress-nginx.yaml
You can see code here.
Inside the cluster (doing vagrant ssh on whatever node) I tested:
curl -H "Host: hello-k8s.info" IP
where IP is got from the command:
kubectl get ingress
Suppose it is 10.97.139.101. Outside the cluster, on my Mac (Running on 3 VirtualBox on my Mac) to access the application I needed to add a route:
sudo route add -net 10.97.0.0/16 192.168.205.10
where 192.168.205.10 is the IP of the master node. In /etc/hosts I added the line:
10.97.139.101 hello-k8s.info
Now typing hello-k8s.info in the browser I see the "Hello World" web page appear.
Thanks to Arghya Sadhu for help.
I am developing an application that allows users to play around in their own sandboxes with a finite life time. Conceptually, it can be thought as if users were playing games of Pong. Users can interact with a web interface hosted at main/ to start a game of Pong. Each game of Pong will exist in its own pod. Since each game has a finite lifetime, the pods are dynamically created on-demand (through the Kubernetes API) as Kubernetes jobs with a single pod. There is therefore a one-to-one relationship between games of Pong and pods. Up to this point I have it all figured out.
My problem is, how do I set up an ingress to map dynamically created URLs, for example main/game1, to the corresponding pods? That is, if a user starts a game through the main interface, I would like him to be redirected to the URL of the corresponding pod where his game is hosted.
I could pre-allocate a set of urls, check if they have active jobs, and redirect if they do not, but the does not scale well. I am thinking dynamically assigning URLs is a common pattern in Kubernetes, so there must be a standard way to do this. I have looked at using nginx-ingress, but that is not a requirement.
Furthermore the comment, I created for you a little demo on minikube providing a working Ingress Class controller (enabled via minikube addons enable ingress).
Replicating the multiple Deployment that simulates the games.
kubectl create deployment deployment-1 --image=nginxdemos/hello
kubectl create deployment deployment-2 --image=nginxdemos/hello
kubectl create deployment deployment-3 --image=nginxdemos/hello
kubectl create deployment deployment-4 --image=nginxdemos/hello
kubectl create deployment deployment-5 --image=nginxdemos/hello
Same for Services resources:
kubectl create service clusterip deployment-1 --tcp=80:80
kubectl create service clusterip deployment-2 --tcp=80:80
kubectl create service clusterip deployment-3 --tcp=80:80
kubectl create service clusterip deployment-4 --tcp=80:80
kubectl create service clusterip deployment-5 --tcp=80:80
Finally, it's time for the Ingress one but we have to be quite hacky since we don't have the subcommand create available.
for number in `seq 5`; do echo "
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: deployment-$number
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /$1
spec:
rules:
- host: hello-world.info
http:
paths:
- path: /game$number
backend:
serviceName: deployment-$number
servicePort: 80
" | kubectl create -f -; done
Now you have Pod, Service and Ingress: obviously, you have to replicate the same result using Kubernetes API but, as I suggested in the comment, you should create a single Ingress resource and update accordingly Path subkey in a dynamic way.
However, if you try to simulate the cURL call faking the Host header, you can see the working result:
# curl `minikube ip`/game2 -sH 'Host: hello-world.info'|grep -i server
<p><span>Server address:</span> <span>172.17.0.5:80</span></p>
<p><span>Server name:</span> <span>deployment-2-5b98b954f6-8g5fl</span></p>
# curl `minikube ip`/game4 -sH 'Host: hello-world.info'|grep -i server
<p><span>Server address:</span> <span>172.17.0.7:80</span></p>
<p><span>Server name:</span> <span>deployment-4-767ff76774-d2fgj</span></p>
You can see the Pod IP and name as well.
I agree with Efrat Levitan. It's not the task for ingress/kubernetes itself.
You need another application (different layer of abstraction) to distinguish where the traffic should be routed for example istio and Routing rule for HTTP traffic based on cookies.
I tested ingress in minikube successfully, no issue at all.
Then I deployed my app into ubuntu, if I am using service NodePort, it is also worked very well. After that, I was thinking to use Ingress as load balancer to router traffic, so that external url is no longer the ugly long port.
But unfortunately, I did not succeed, always failed.
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: dv
annotations:
ingress.kubernetes.io/rewrite-target: /
spec:
rules:
- http:
paths:
- path: /test
backend:
serviceName: ngsc
servicePort: 3000
kubectl get ing
NAME HOSTS ADDRESS PORTS AGE
dv * 80 12s
root#kmaster:/home/ubuntu/datavisor# kubectl describe ing dv
Name: dv
Namespace: default
Address:
Default backend: default-http-backend:80 (<none>)
Rules:
Host Path Backends
---- ---- --------
*
/ ngsc:3000 (192.168.1.14:3000,192.168.1.17:3000,192.168.1.18:3000)
Annotations:
ingress.kubernetes.io/rewrite-target: /
Events: <none>
Then when I tried to access, I got following error:
curl http://cluster-ip
curl: (7) Failed to connect to <cluster-ip> port 80: Connection refused
What I really hope is to let the url exposed outside is http://ipaddress, instead of http://ipaddress:30080
I know that I can easily use nginx out of kubernete to meet this requirement, but that is not ideal, I want kubernete to handle it so that even service port changed, everything is still up.
Can you check above output and tell me what is the error? I checked a lot doc, every place seemed focus only in minikube, nothing related to real cluster deployment. Do I need to install anything to make ingress working? when I use kubectl get all --all-namespaces I did not see the ingress-controller installed at all. How can I install it if needed?
Thanks for your advice
Well, actually Kubernetes does not provide any Ingress controller out of box. You have to install Nginx Ingress or Traefik Ingress or anything else. Ingress controller must run somewhere in your cluster, it's a must. Actually ingress controller is the actual proxy that proxies traffic to your applications.
And I think you should know that minikube under the hood also uses nginx-ingress-controller (see https://github.com/kubernetes/minikube/tree/master/deploy/addons/ingress).
In a cloud environments ingress controllers run behind the cloud load balancer that performs load balancing between cluster nodes.
If you run on-prem cluster - then usually your ingress-controller is run as NodePort service and you may create DNS record pointing to your node IP addresses. It is also possible to run ingress controller on dedicated nodes and use hostNetwork: true. That will allow to use standard 80/443 ports. So there are many options here.
I'm confused about nginx ingress with Kubernetes. I've been able to use it with "basic nginx auth" (unable to do so with oauth2 yet).
I've installed via helm:
helm install stable/nginx-ingress --name app-name --set rbac.create=true
This creates two services, an nginx-ingress-controller and an nginx-ingress-backend.
When I create an ingress, this ingress is targeted towards one and only one nginx-ingress-controller, but I have no idea how:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: tomcat
annotations:
nginx.ingress.kubernetes.io/auth-type: basic
nginx.ingress.kubernetes.io/auth-secret: basic-auth
nginx.ingress.kubernetes.io/auth-realm: "Authentication Required - foo"
nginx.ingress.kubernetes.io/rewrite-target: /
namespace: kube-system
spec:
rules:
- host:
http:
paths:
- path: /
backend:
serviceName: tomcat-deployment-service
servicePort: 8080
When I get this Ingress from the output of kubectl get ingress -n kube-system, it has a public, external IP.
What's concerning is that basic-auth DOESN'T APPLY to that external IP; it's wide open! Nginx authentication only kicks in when I try to visit the nginx-ingress-controller's IP.
I have a lot of questions.
How do I made an ingress created from kubectl apply -f
ingress.yaml target a specific nginx-ingress-controller?
How do I keep this new ingress from having an external IP?
Why isn't nginx authentication kicking in?
What IP am I suppose to use (the nginx-ingress-controller or the
generated one?)
If I'm suppose to use the generated IP, what about the one from the controller?
I have been searching for descent, working examples (and pouring over sparse, changing documentation, and github issues) for literally days.
EDIT:
In this "official" documentation, it's unclear as to weather or not http://10.2.29.4/ is the IP from the ingress or the controller. I assume the controller because when I run this, the other doesn't even authenticate (it let's me in without asking for a password). Both IP's I'm using are external IPs (publicly available) on GCP.
I think you might have some concept definition misunderstanding.
Ingress is not a job ( Nor a service, nor a pod ). It is just a configuration. It cannot have "IP". think of ingress as a routing rule or a routing table in your cluster.
Nginx-ingress-controller is the service with type Loadbalancer with actual running pods behind it that facilitates those ingress rules that you created for your cluster.
Nginx-ingress-backend is likely to be a default-backend that your nginx-ingress-controller will route to if no matching routes are found. see this
In general, your nginx-ingress-controller should be the only entry of your cluster. Other services in your cluster should have type ClusterIP such that they are not exposed to outside the cluster and only accessible through your nginx-ingress-controller. In you case, since your service could be access from outside directly, it should not be of type ClusterIP. Just change the service type to get it protected.
Based on above understanding, I will glad to provide further help for the question you have.
Some readings:
What is ingress: https://kubernetes.io/docs/concepts/services-networking/ingress/
K8s Services and external accessibility: https://kubernetes.io/docs/concepts/services-networking/service/#publishing-services-service-types
I am trying to deploy and access Sock-shop on Google Cloud Platform.
https://github.com/microservices-demo/microservices-demo
I was able to deploy it using the deployment script
https://github.com/microservices-demo/microservices-demo/blob/master/deploy/kubernetes/complete-demo.yaml
Based on the tutorial here
https://www.weave.works/docs/cloud/latest/tasks/deploy/sockshop-deploy/
It says
Display the Sock Shop in the browser using:
<master-node-IP>:<NodePort>
But on GCP master node is hidden from the user.
So I changed the type from NodePort to LoadBalancer.
And I was able to get an external IP.
But it says the page cannot be found. enter code here
Do I need to set up more stuff for LoadBalancer?
I dont know If you solve the issue but I did it so I would like to share with you my solution that works for me.
You can do it through two ways:
1st) By creating a Load Balancer, where you expose the front-end service.
I assume that you have already created a namespace called sock-shop so any further command should specify and referred to that namespace.
If you type and execute the command:
kubectl get services --namespace=sock-shop
you should be able to see a list with all the services included a service called "front-end". So now you want to expose that service not as NodePort but as LoadBalancer. So, execute the command:
kubectl expose service front-end --name=front-end-lb --port=80 --target-port=8079 --type=LoadBalancer --namespace=sock-shop
After this give some time and you will able to access the Front end of the Sock Shop via public IP address (ephimeral)
2nd) More advanced way is by configuring an Ingress Load Balancer.
You need to configure another yaml file and put this code inside and run it as you did with the previous .yaml file.
nano basic-ingress.yaml
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
namespace : sock-shop
name: basic-ingress
spec:
backend:
serviceName: front-end
servicePort: 80
kubectl apply -f basic-ingress.yaml --namespace=sock-shop
Locate the Public IP address through this command and after maximun 15mins you should be able to access the Sock Shop.
kubectl get ingress --namespace=sock-shop
I would recommend to return back for NodePort in the corresponded Service and create Ingress resource in your GCP cluster.
If you desire to access the related application from outside the cluster, Kubernetes provides Ingress mechanism to expose HTTP and HTTPS routes to your internal services.
Basically, HTTP(S) Load Balancer is created by default in GKE once Ingress resource has been implemented successfully, therefore it will take care for routing all the external HTTP/S traffic to the nested Kubernetes services.
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: basic-ingress
spec:
backend:
serviceName: web
servicePort: 8080
You can check the External IP address for Load Balancer by the following command:
kubectl get ingress basic-ingress
I found this Article would be very useful in your common research.