Nginx Ingress Kube - kubernetes

I'm confused about nginx ingress with Kubernetes. I've been able to use it with "basic nginx auth" (unable to do so with oauth2 yet).
I've installed via helm:
helm install stable/nginx-ingress --name app-name --set rbac.create=true
This creates two services, an nginx-ingress-controller and an nginx-ingress-backend.
When I create an ingress, this ingress is targeted towards one and only one nginx-ingress-controller, but I have no idea how:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: tomcat
annotations:
nginx.ingress.kubernetes.io/auth-type: basic
nginx.ingress.kubernetes.io/auth-secret: basic-auth
nginx.ingress.kubernetes.io/auth-realm: "Authentication Required - foo"
nginx.ingress.kubernetes.io/rewrite-target: /
namespace: kube-system
spec:
rules:
- host:
http:
paths:
- path: /
backend:
serviceName: tomcat-deployment-service
servicePort: 8080
When I get this Ingress from the output of kubectl get ingress -n kube-system, it has a public, external IP.
What's concerning is that basic-auth DOESN'T APPLY to that external IP; it's wide open! Nginx authentication only kicks in when I try to visit the nginx-ingress-controller's IP.
I have a lot of questions.
How do I made an ingress created from kubectl apply -f
ingress.yaml target a specific nginx-ingress-controller?
How do I keep this new ingress from having an external IP?
Why isn't nginx authentication kicking in?
What IP am I suppose to use (the nginx-ingress-controller or the
generated one?)
If I'm suppose to use the generated IP, what about the one from the controller?
I have been searching for descent, working examples (and pouring over sparse, changing documentation, and github issues) for literally days.
EDIT:
In this "official" documentation, it's unclear as to weather or not http://10.2.29.4/ is the IP from the ingress or the controller. I assume the controller because when I run this, the other doesn't even authenticate (it let's me in without asking for a password). Both IP's I'm using are external IPs (publicly available) on GCP.

I think you might have some concept definition misunderstanding.
Ingress is not a job ( Nor a service, nor a pod ). It is just a configuration. It cannot have "IP". think of ingress as a routing rule or a routing table in your cluster.
Nginx-ingress-controller is the service with type Loadbalancer with actual running pods behind it that facilitates those ingress rules that you created for your cluster.
Nginx-ingress-backend is likely to be a default-backend that your nginx-ingress-controller will route to if no matching routes are found. see this
In general, your nginx-ingress-controller should be the only entry of your cluster. Other services in your cluster should have type ClusterIP such that they are not exposed to outside the cluster and only accessible through your nginx-ingress-controller. In you case, since your service could be access from outside directly, it should not be of type ClusterIP. Just change the service type to get it protected.
Based on above understanding, I will glad to provide further help for the question you have.
Some readings:
What is ingress: https://kubernetes.io/docs/concepts/services-networking/ingress/
K8s Services and external accessibility: https://kubernetes.io/docs/concepts/services-networking/service/#publishing-services-service-types

Related

Google Kubernetes Engine ingress annotations

I configure Ingress on google Kubernetes engine. I am new on ingress but as i understood ingress can serve different Loadbalancers and different LBs should be differently configured.
I have started with a simple ingress config on GKE :
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: basic-ingress
spec:
rules:
- http:
paths:
- path: /*
backend:
serviceName: web-np
servicePort: 8080
- path: /v2/keys
backend:
serviceName: etcd-np
servicePort: 2379
And it works fine so I have 2 different NodePort services web-np and etcd-np . But now I need to extend this logic with some rewrite rules so that request that points to /service1 - will be redirected to the other service1-np service but before /service1/hello.html must be replaced to /hello.html. That's why I have the following questions:
How can I configure rewrite in ingress and if it is possible with default load balancer.
What is default load balancer on GKE.
Where can I find a list of all annotations to it. I have thought that the full list is on https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/annotations/ but there is a completly different list and there is no kubernetes.io/ingress.global-static-ip-name annotation that is widely used in google examples.
Ingress - API object that manages external access to the services in a cluster, typically HTTP.
Ingress may provide load balancing, SSL termination and name-based virtual hosting.
Kubernetes.io: Ingress
Kubernetes can have multiple Ingress controllers. This controllers are different from each other. The Ingress controllers mentioned by you in this particular question are:
Ingress-GCE - a default Ingress resource for GKE cluster:
Github.com: Kubernetes: Ingress GCE
Ingress-nginx - an alternative Ingress controller which can be deployed to your GKE cluster:
Github.com: Kubernetes: Ingress-nginx
Ingress configuration you pasted will use the Ingress-GCE controller. If you want to switch to Ingress-nginx one, you will need to deploy it and set an annotation like:
kubernetes.io/ingress.class: "nginx"
How can I configure rewrite in ingress and if it is possible with default load balancer.
There is an ongoing feature request to support rewrites with Ingress-GCE here: Github.com: Ingress-GCE: Rewrite.
You can use Ingress-nginx to have support for rewrites. There is an official documentation about deploying it: Kubernetes.github.io: Ingress-nginx: Deploy
For more resources about rewrites you can use:
Kubernetes.github.io: Ingress nginx: Examples: Rewrite
Stackoverflow.com: Ingress nginx how to serve assests to application - this is an answer which shows an example on how to configure a playground for experimenting with rewrites
What is default load balancer on GKE.
If you create an Ingress resource with a default Ingress-GCE option you will create a L7 HTTP&HTTPS LoadBalancer.
If you create a service of type LoadBalancer in GKE you will create an L4 Network Load Balancer
If you deploy an Ingress-nginx controller in GKE cluster you will create a L4 Network Loadbalancer pointing to the Ingress-nginx controller which after that will route the traffic accordingly to your Ingress definition. If you are willing to use Ingress-nginx you will need to specify:
kubernetes.io/ingress.class: "nginx"
in your Ingress definition.
Please take a look on this article: Medium.com: Google Cloud: Kubernetes Nodeport vs Loadbalancer vs Ingress
Where can I find a list of all annotations to it. I have thought that the full list is on https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/annotations/ but there is a completly different list and there is no kubernetes.io/ingress.global-static-ip-name annotation that is widely used in google examples.
The link that you provided with annotations is specifically for Ingress-nginx. This annotations will not work with Ingress-GCE.
The annotations used in GCP examples are specific to Ingress-GCE.
You can create a Feature Request for a list of available annotations for Ingress-GCE on Issuetracker.google.com.
Answering an old question, but hopefully it can help someone.
I found the list of annotations for GCP Ingress in the source code for ingress-gce.

Load Balancing "Hello World" Kubernetes application

To start playing with Kubernetes I created a small project where I create a three nodes cluster on my local machine using Vagrant and Vbox:
https://github.com/sasadangelo/k8s-cluster
Just typing vagrant up my cluster is up and running and I can start to play with it. I got familiar with the main concepts and commands and created a simple "Hello World" application composed by a docker image with an NGINX web server where you can connect via browser and a "Hello World" message appears with the hostname. The hostname helps me to understand which Pod responded to my request.
Here the project:
https://github.com/sasadangelo/k8s-tutorials/tree/master/hello-k8s
Currently, I can create a deployment with kubectl create exposing it as a service with kubectl expose and then increase replica with kubect scale. At the moment, I am able to use use NodePort service, so with kubectl describe service I can see on which IP the 5 Pods listen, I connect to them and everything works fine.
My problem is that now I want to load balance the traffic. I want to connect to a single IP and as soon as I click on the browser Reload button I want to see that different Pod names appear.
I know that Kubernetes by default offers only NodePort service and if I want a load balancer I need something like an Ingress. I also know that a simple reverse proxy like Ingress Nginx is a good implementation for my needs.
However, I read lot of tutorials but I have hard time understanding how I should configure it to achieve what I need.
Here code I have so far:
https://github.com/sasadangelo/k8s-cluster/tree/master/ingress
Can anyone help me how to fix Ingress code to achieve what I need?
Kube Proxy which is a kubernetes component provides load balancing at L4 layer via iptables load balancing.So with ClusterIP type service itself you get load balancing between pods. But with clusterIP type service you can not access it from outside the cluster i.e from a browser. Nodeport type service provides access to the pods from outside the cluster with load balancing at L4 layer by kube proxy.
LoadBalancer and Ingress provides load balancing and rule based routing at L7 layer if that's what you are looking for.
Looking at your code you have yamls for nginx ingress controller and exposed it as Nodeport. What you need next is to create an ingress resource
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: example-ingress
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /$1
spec:
rules:
- host: hello-world.info
http:
paths:
- path: /
backend:
serviceName: web
servicePort: 8080
Once you create above ingress and you have a ClusterIP type service for your pod with a name web which exposed port 8080, You should be able to access it.
Here is the doc on using nginx ingress for an hello world application on Minikube which you can follow to make ingress work in your setup as well.
Here is what I have done in detail to solve the issue. First of all, I installed Nginx Ingress resources:
kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/nginx-0.30.0/deploy/static/mandatory.yaml
kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/nginx-0.30.0/deploy/static/provider/baremetal/service-nodeport.yaml
then I create an Nginx Ingress resource in a ingress-nginx.yaml file:
---
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: hello-k8s-ingress
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /$1
spec:
rules:
- host: hello-k8s.info
http:
paths:
- path: /
backend:
serviceName: hello-k8s
servicePort: 80
and installed it with the command:
kubectl apply -f ingress-nginx.yaml
You can see code here.
Inside the cluster (doing vagrant ssh on whatever node) I tested:
curl -H "Host: hello-k8s.info" IP
where IP is got from the command:
kubectl get ingress
Suppose it is 10.97.139.101. Outside the cluster, on my Mac (Running on 3 VirtualBox on my Mac) to access the application I needed to add a route:
sudo route add -net 10.97.0.0/16 192.168.205.10
where 192.168.205.10 is the IP of the master node. In /etc/hosts I added the line:
10.97.139.101 hello-k8s.info
Now typing hello-k8s.info in the browser I see the "Hello World" web page appear.
Thanks to Arghya Sadhu for help.

Is it possible to serve up applications through a Kubernetes controller node?

I have built a K3s (https://k3s.io) cluster on a set of Raspberry Pi4 computers.
The controller (ctrl-1) node is a gateway in that it has 2 network interfaces. One is connected to my LAN and the other is connected to a network that it creates, e.g. K3S-LAN. The two nodes (node-1 and node-2) are deployed to the K3S-LAN.
I want to be able to access the applications running on the nodes through ctrl-1, e.g. from the LAN. This is because this cluster is meant to be portable so only the ctrl-1 node needs to be connected to the guest LAN. (Yes there are issues with DNS names etc to be sorted out, but I want to get the basics running first).
This means that I need to be able to "proxy" the ingress through ctrl-1. I thought I had the right idea for this in that I deployed "nginx-ingress" to the master, using Helm. However I forgot about the service for this - this has been scheduled on the nodes, whereas it needs to be on the controller so that the ports are opened up (I think). However I cannot find how to make the service run on the controller.
At the moment I have the service running with a type of NodePort. I could install MetalLB so that I have LoadBalancer capabilities. However with what I have seen I am not sure if this would help or not.
ctrl-1 does not have any taints setup on it, just the role of master.
Am I barking up the wrong tree here? I guess this might not be the intended use case of Kubernetes, but I am playing around with an idea. Thanks for any ideas that people have.
Update*
I have just thought that the way around this might be to run HAProxy on ctrl-1 (as another service on the host) and setup rules to proxy to the necessary services within the cluster. That would act as the bridge between the networks.
You just need to expose your pod via a Nodeport type service and it can be accessed via http://master-node-ip:nodeport. Make sure that kube-proxy is running on all master and worker nodes.
The ingress approach also should work as long as you have kube-proxy running on your master. You deploy nginx ingress on your cluster and it will get deployed into a worker node. Then you can expose nginx ingress controller itself using a NodePort service. After this you can create ingress resource for configuring the nginx ingress controller to route traffic to your backend pods and services running on worker nodes. The services for backend pods should be of type ClusterIP.
Deploy nginx ingress controller and expose it via NodePort service using kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/nginx-0.27.1/deploy/static/provider/baremetal/service-nodeport.yaml
Deploy nginx pod(nginx is an example..this should be your pod) kubectl run nginx --generator=run-pod/v1 --image=nginx
Expose nginx pod via ClusterIP service
apiVersion: v1
kind: Service
metadata:
labels:
run: nginx
name: nginx-service
spec:
ports:
- port: 80
protocol: TCP
targetPort: 80
selector:
run: nginx
Create ingress resource
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: mycha-ingress
annotations:
kubernetes.io/ingress.class: "nginx"
spec:
rules:
- http:
paths:
- path: /
backend:
serviceName: nginx-service
servicePort: 80
With above setup I can now access nginx and get "Welcome to nginx! " via http://master-node-ip:NodePort of nginx ingress controller

Is Ingress useable only for Minikube? It is not working in Ubuntu installation

I tested ingress in minikube successfully, no issue at all.
Then I deployed my app into ubuntu, if I am using service NodePort, it is also worked very well. After that, I was thinking to use Ingress as load balancer to router traffic, so that external url is no longer the ugly long port.
But unfortunately, I did not succeed, always failed.
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: dv
annotations:
ingress.kubernetes.io/rewrite-target: /
spec:
rules:
- http:
paths:
- path: /test
backend:
serviceName: ngsc
servicePort: 3000
kubectl get ing
NAME HOSTS ADDRESS PORTS AGE
dv * 80 12s
root#kmaster:/home/ubuntu/datavisor# kubectl describe ing dv
Name: dv
Namespace: default
Address:
Default backend: default-http-backend:80 (<none>)
Rules:
Host Path Backends
---- ---- --------
*
/ ngsc:3000 (192.168.1.14:3000,192.168.1.17:3000,192.168.1.18:3000)
Annotations:
ingress.kubernetes.io/rewrite-target: /
Events: <none>
Then when I tried to access, I got following error:
curl http://cluster-ip
curl: (7) Failed to connect to <cluster-ip> port 80: Connection refused
What I really hope is to let the url exposed outside is http://ipaddress, instead of http://ipaddress:30080
I know that I can easily use nginx out of kubernete to meet this requirement, but that is not ideal, I want kubernete to handle it so that even service port changed, everything is still up.
Can you check above output and tell me what is the error? I checked a lot doc, every place seemed focus only in minikube, nothing related to real cluster deployment. Do I need to install anything to make ingress working? when I use kubectl get all --all-namespaces I did not see the ingress-controller installed at all. How can I install it if needed?
Thanks for your advice
Well, actually Kubernetes does not provide any Ingress controller out of box. You have to install Nginx Ingress or Traefik Ingress or anything else. Ingress controller must run somewhere in your cluster, it's a must. Actually ingress controller is the actual proxy that proxies traffic to your applications.
And I think you should know that minikube under the hood also uses nginx-ingress-controller (see https://github.com/kubernetes/minikube/tree/master/deploy/addons/ingress).
In a cloud environments ingress controllers run behind the cloud load balancer that performs load balancing between cluster nodes.
If you run on-prem cluster - then usually your ingress-controller is run as NodePort service and you may create DNS record pointing to your node IP addresses. It is also possible to run ingress controller on dedicated nodes and use hostNetwork: true. That will allow to use standard 80/443 ports. So there are many options here.

Sock-shop on GCP with Loadbalancer

I am trying to deploy and access Sock-shop on Google Cloud Platform.
https://github.com/microservices-demo/microservices-demo
I was able to deploy it using the deployment script
https://github.com/microservices-demo/microservices-demo/blob/master/deploy/kubernetes/complete-demo.yaml
Based on the tutorial here
https://www.weave.works/docs/cloud/latest/tasks/deploy/sockshop-deploy/
It says
Display the Sock Shop in the browser using:
<master-node-IP>:<NodePort>
But on GCP master node is hidden from the user.
So I changed the type from NodePort to LoadBalancer.
And I was able to get an external IP.
But it says the page cannot be found. enter code here
Do I need to set up more stuff for LoadBalancer?
I dont know If you solve the issue but I did it so I would like to share with you my solution that works for me.
You can do it through two ways:
1st) By creating a Load Balancer, where you expose the front-end service.
I assume that you have already created a namespace called sock-shop so any further command should specify and referred to that namespace.
If you type and execute the command:
kubectl get services --namespace=sock-shop
you should be able to see a list with all the services included a service called "front-end". So now you want to expose that service not as NodePort but as LoadBalancer. So, execute the command:
kubectl expose service front-end --name=front-end-lb --port=80 --target-port=8079 --type=LoadBalancer --namespace=sock-shop
After this give some time and you will able to access the Front end of the Sock Shop via public IP address (ephimeral)
2nd) More advanced way is by configuring an Ingress Load Balancer.
You need to configure another yaml file and put this code inside and run it as you did with the previous .yaml file.
nano basic-ingress.yaml
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
namespace : sock-shop
name: basic-ingress
spec:
backend:
serviceName: front-end
servicePort: 80
kubectl apply -f basic-ingress.yaml --namespace=sock-shop
Locate the Public IP address through this command and after maximun 15mins you should be able to access the Sock Shop.
kubectl get ingress --namespace=sock-shop
I would recommend to return back for NodePort in the corresponded Service and create Ingress resource in your GCP cluster.
If you desire to access the related application from outside the cluster, Kubernetes provides Ingress mechanism to expose HTTP and HTTPS routes to your internal services.
Basically, HTTP(S) Load Balancer is created by default in GKE once Ingress resource has been implemented successfully, therefore it will take care for routing all the external HTTP/S traffic to the nested Kubernetes services.
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: basic-ingress
spec:
backend:
serviceName: web
servicePort: 8080
You can check the External IP address for Load Balancer by the following command:
kubectl get ingress basic-ingress
I found this Article would be very useful in your common research.