Apologies if this has been answered before, but I am a little confused on Ingress Nginx is working together with services.
I am trying to implement an nginx ingress in my Kubernetes environment.
So far I have an ingress-nginx-controller-deployment setup, as well as a deployment and service for the default backend. I still need to create my actual Ingress resources, the ingress-nginx-controller-service and also my backend.
curl <NodeIP>
returns "default backend 404" on port 80 for the Node which the ingress-nginx-controller-deployment is deployed on.
However, my understanding is that exposing anything out of the cluster requires a service (Nodeport/Loadbalancer), which is the duty of the ingress-nginx-controller-service.
My question is how is this possible, that I can access port 80 for my Node on my browser, which is outside the cluster?
Could I then deploy my backend app on port 80 the same way the above is done?
I feel like I am misunderstanding a key concept here.
default backend image: gcr.io/google_containers/defaultbackend:1.0
nginx-controller image: gcr.io/google_containers/nginx-ingress-controller:0.9.0-beta.3
I think you missed a really good article about how nginx-ingress expose to the world!
I short:
If you're using hostNetwork: true then you bypass the kubernetes network (kube-proxy). in a simple word, you bypass the container and orchestration network and just using the host network then the node with nginx-ingress container will expose port 80 to the world.
There are other ways that you can use to expose nginx port to outside of the cluster (node-port, network load balancer like MetalLB).
Related
I am a bit new to Kubernetes and I am working with EKS.
I have two main apps for which there is a number of pods and I have set up a ELB for external access.
I also have a small app with say 1-2 pods. I don't want to set up a ELB just for this small app. I checked the node port, but in that case, I can't use the default HTTPS port 443.
So I feel the best thing to do in this case would be to bring the small app outside the cluster, then maybe set it up in a EC2 instance. Or is there some other way to expose the small app while keeping it inside the cluster itself?
You can try to use the Host network (Node) like hostport (Not recommended in k8s to use in prod)
apiVersion: v1
kind: Pod
metadata:
name: nginx
spec:
containers:
- name: nginx
image: nginx
ports:
- containerPort: 80
hostPort: 443
The hostPort feature allows to expose a single container port on the
host IP. Using the hostPort to expose an application to the outside of
the Kubernetes cluster has the same drawbacks as the hostNetwork
approach discussed in the previous section. The host IP can change
when the container is restarted, two containers using the same
hostPort cannot be scheduled on the same node and the usage of the
hostPort is considered a privileged operation on OpenShift.
Extra
I don't want to set up a elb just for this small app.
Ideally, you have to use the deployments with the ingress and ingress controller. So there will be single ELB for the whole EKS cluster and all services will be using that single point.
All PODs or deployment will be running into a single cluster if you want. Single point ingress will work as handling the traffic into EKS cluster.
https://kubernetes.io/docs/concepts/services-networking/ingress/
You can read this article how to setup the ingress in EKS aws so you will get an idea.
You can use a different domains for exposing services.
Example :
https://aws.amazon.com/premiumsupport/knowledge-center/terminate-https-traffic-eks-acm/
I'm pretty sure that this is a basic use case when running apps on kubernetes, but till now I wasn't able to find a tutorial, nor understand from the documentation, how to make it work.
I have an application, which is listening on a port 9000. So when run on my localhost, I can access it through a web browser on a localhost:9000. When run in a docker container, which is running on my VPS, it's also accessible on myVPSAddress:9000. Now the question is, how to deploy it on kubernetes running on the very same Virtual Private Server and expose the application to be visible as well, as when deployed on docker. I can access the application from within the VPS on the address of the cluster, but not on the IP address of the server itself. Can somebody show me some basic dockerfile with a description what is it doing or show me some idiot-proof way, how to make it work? Thanks
While one would think that this is a very basic use-case, that is not the case for people running their own kubernetes clusters on bare metal servers. (The way you are on your VPS).
The recommended way of exposing an application to "the world" is to use kubernetes services, see this piece of documentation about exposing services. You define a kubernetes service, either of the type NodePort or of type Loadbalancer *.
Here is what a dead simple service looks like (hint: it's of the default type NodePort):
kind: Service
apiVersion: v1
metadata:
name: my-service
spec:
selector:
app: MyApp
ports:
- protocol: TCP
port: 9000
targetPort: 9376
This will expose your service with label name: my-service (interally running on port 9000) on all nodes in your VPS cluster at port 9376.
Assuming your nodes have a public IP (which from your question I assume they do), you can safely do curl localhost:9376.
Because this is usually not ideal UX/UI to expose to users, people use services of type Loadbalancer. This service type provides a unique IP to each of your services instead of a port.
These services are first class citizens on cloud managed clusters, such as Google's GKE, but if you run your own Kubernetes cluster (setup using say kubeadm), then you need to deploy your Loadbalancer service provider. I've used the excellent MetalLB and it works flawlessly once it's been setup, but you need to set it up yourself. If you want dns names for you services as well, you should also look at ExternalDNS.
* Caveat here is that you can also use a service of type ExternalIP if you can somehow make that IP routable, but unless the network is in your control, this is usually not a feasible approach, and I'd recommend looking at an LB provider instead.
I'm trying to use Kubernetes to make configurations and deployments explicitly defined and I also like Kubernetes' pod scheduling mechanisms. There are (for now) just 2 apps running on 2 replicas on 3 nodes. But Google's Kubernetes Engine's load balancer is extremely expensive for a small app like ours (at least for the moment) at the same time I'm not willing to change to a single instance hosting solution on a container or deploying the app on Docker swarm etc.
Using node's IP seemed like a hack and I thought that it might expose some security issues inside the cluster. Therefore I configured a Træfik ingress and an ingress controller to overcome Google's expensive flat rate for load balancing but turns out an outward facing ingress spins up a standart load balancer or I'm missing something.
I hope I'm missing something since at this rates ($16 a month) I cannot rationalize using kubernetes from start up for this app.
Is there a way to use GKE without using Google's load balancer?
An Ingress is just a set of rules that tell the cluster how to route to your services, and a Service is another set of rules to reach and load-balance across a set of pods, based on the selector. A service can use 3 different routing types:
ClusterIP - this gives the service an IP that's only available inside the cluster which routes to the pods.
NodePort - this creates a ClusterIP, and then creates an externally reachable port on every single node in the cluster. Traffic to those ports routes to the internal service IP and then to the pods.
LoadBalancer - this creates a ClusterIP, then a NodePort, and then provisions a load balancer from a provider (if available like on GKE). Traffic hits the load balancer, then a port on one of the nodes, then the internal IP, then finally a pod.
These different types of services are not mutually exclusive but actually build on each other, and it explains why anything public must be using a NodePort. Think about it - how else would traffic reach your cluster? A cloud load balancer just directs requests to your nodes and points to one of the NodePort ports. If you don't want a GKE load balancer then you can already skip it and access those ports directly.
The downside is that the ports are limited between 30000-32767. If you need standard HTTP port 80/443 then you can't accomplish this with a Service and instead must specify the port directly in your Deployment. Use the hostPort setting to bind the containers directly to port 80 on the node:
containers:
- name: yourapp
image: yourimage
ports:
- name: http
containerPort: 80
hostPort: 80 ### this will bind to port 80 on the actual node
This might work for you and routes traffic directly to the container without any load-balancing, but if a node has problems or the app stops running on a node then it will be unavailable.
If you still want load-balancing then you can run a DaemonSet (so that it's available on every node) with Nginx (or any other proxy) exposed via hostPort and then that will route to your internal services. An easy way to run this is with the standard nginx-ingress package, but skip creating the LoadBalancer service for it and use the hostPort setting. The Helm chart can be configured for this:
https://github.com/helm/charts/tree/master/stable/nginx-ingress
One option is to completely disable this feature on your GKE cluster. When creating the cluster (on console.cloud.google.com) under Add-ons disable HTTP load balancing. If you are using gcloud you can use gcloud beta container clusters create ... --disable-addons=HttpLoadBalancing.
Alternatively, you can also inhibit the GCP Load Balancer by adding an annotation to your Ingress resources, kubernetes.io/ingress.class=somerandomstring.
For newly created ingresses, you can put this in the yaml document:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
annotations:
kubernetes.io/ingress.class: somerandomstring
...
If you want to do that for all of your Ingresses you can use this example snippet (be careful!):
kubectl get ingress --all-namespaces \
-o jsonpath='{range .items[*]}{"kubectl annotate ingress -n "}{.metadata.namespace}{" "}{.metadata.name}{" kubernetes.io/ingress.class=somerandomstring\n"}{end}' \
| sh -x
Now using Ingresses is pretty useful with Kubernetes, so I suggest you check out the nginx ingress controller and after deployment, annotate your Ingresses accordingly.
If you specify the Ingress class as an annotation on the Ingress object
kubernetes.io/ingress.class: traefik
Traefik will pick it up while the Google Load Balancer will ignore it. There is also a bit of Traefik documentation on this part.
You could deploy the nginx ingress controller using NodePort mode (e.g. if using the helm chart set controller.service.type to NodePort) and then load-balance amongst your instances using DNS. Just make sure you have static IPs for the nodes or you could even create a DaemonSet that somehow updates your DNS with each node's IP.
Traefik seems to support a similar configuration (e.g. through serviceType in its helm chart).
I have deployment that hosts a website on port 9001 and a service attached to it. I want to allow anyone (from outside cluster) to be able to connect to that site.
Any help would be appreciated.
I want to allow anyone (from outside cluster) to be able to connect to that site
There are many ways to do this using kubernetes services to expose port 9001 of the website to the outside world:
Service type LoadBalancer if you have an external, cloud-provider's load-balancer.
ExternalIPs. The website can be hit at ExternalIP:Port.
Service type NodePort if the cluster's nodes are reachable from the users. The website can be hit at NodeIP:NodePort.
Ingress controller and ingress resource.
As you wrote that this is not a cloud deployment, you need to consider how to correctly expose this to the world in a decent fashion. First and formost, create a NodePort type service for your deployment. With this, your nodes will expose that service on a high port.
Depending on your network, at this point you either need to configure a loadbalancer in your network to forward traffic for some IP:80 to your node(s) high NodePort, or for example deploy HAProxy in a DeamonSet with hostNetwork: true that will proxy 80 to your NodePort.
A bit more complexity can be added by deployment of Nginx IngressController (exposed as above) and use of Ingress to make the Ingress Controller expose all your services without the need to fiddle with NodePort/LB/HAProxy for each of them individualy any more.
I have a docker image that serves a simple static web page.
I have a working Kubernetes cluster of 4 nodes (physical servers not in the cloud anywhere).
I want to run that docker image on 2 of the 4 Kubernetes nodes and have it be accessible to the world outside the cluster and load balanced and have it move it to another node if one dies.
Do I need to make a pod then a replication controller then a kube proxy something?
Or do I need to just make a replication controller and expose it somehow?
Do I need to make service?
I don't need help with how to make any of those things, that seems well documented, but what I can't tell what I need to make.
What you need is to expose your service (that consists of pods which are run/scaled/restarted by your replication controller). Using deployment instead of replication controller has additional benefits (mainly for updating the app).
If you are on bare metal then you probably wish to expose your service via type: NodePort - so every node in your cluster will open a static port that routes traffic to pods.
You can then either point your load balancer to that nodes on that port, or make a DNS entry with all Kubernetes nodes.
Docs: http://kubernetes.io/docs/user-guide/quick-start/
You'll need:
1) A load balancer on one of your nodes in your cluster, that is a reverse proxy Pod like nginx to proxy the traffic to an upstream.
This Pod will need to be exposed to the outside using hostPort like
ports:
- containerPort: 80
hostPort: 80
name: http
- containerPort: 443
hostPort: 443
name: https
2) A Service that will use the web server selector as target.
3) Set the Service name (which will resolve to the Service IP) as the upstream in nginx config
4) Deploy your web server Pods, which will have the selector to be targeted by the Service.
You might also want to look at External IP for the Service
http://kubernetes.io/docs/user-guide/services/#external-ips
but I personally never managed to get that working on my bare metal cluster.