Kubernetes, Loadbalancing, and Nginx Ingress - AKS - kubernetes

Stack:
Azure Kubernetes Service
NGINX Ingress Controller - https://github.com/kubernetes/ingress-nginx
AKS Loadbalancer
Docker containers
My goal is to create a K8s cluster that will allow me to use multiple pods, under a single IP, to create a microservice architecture. After working with tons of tutorials and documentation, I'm not having any luck with my endgoal. I got to the point of being able to access a single deployment using the Loadbalancer, but introducing the ingress has not been successful so far. The services are separated into their respective files for readability and ease of control.
Additionally, the Ingress Controller was added to my cluster as described in the installation instructions using: kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v0.35.0/deploy/static/provider/cloud/deploy.yaml
LoadBalancer.yml:
apiVersion: v1
kind: Service
metadata:
name: backend
spec:
loadBalancerIP: x.x.x.x
selector:
app: ingress-service
tier: backend
ports:
- name: "default"
port: 80
targetPort: 80
type: LoadBalancer
IngressService.yml:
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: ingress-service
annotations:
kubernetes.io/ingress.class: nginx
spec:
rules:
- http:
paths:
- path: /api
backend:
serviceName: api-service
servicePort: 80
api-deployment.yml
apiVersion: v1
kind: Service
metadata:
name: api-service
spec:
selector:
app: api
ports:
- port: 80
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: api-deployment
spec:
selector:
matchLabels:
app: api
tier: backend
track: stable
replicas: 1
template:
metadata:
labels:
app: api
tier: backend
track: stable
spec:
containers:
- name: api
image: image:tag
ports:
- containerPort: 80
imagePullPolicy: Always
imagePullSecrets:
- name: SECRET
The API in the image is exposed on port 80 correctly.
After applying each of the above yml services and deployments, I attempt a web request to one of the api resources via the LoadBalancer's IP and receive only a timeout on my requests.

Found my answer after hunting around enough. Basically, the problem was that the Ingress Controller has a Load Balancer built into the yaml, as mentioned in comments above. However, the selector for that LoadBalancer requires marking your Ingress service as part of the class. Then that Ingress service points to each of the services attached to your pods. I also had to make a small modification to allow using a static IP in the provided load balancer.

Related

Connect two pods to the same external access point

I have two pods (deployments) running on minikube. Each pod has the same port exposed (say 8081), but use different images. Now I want to configure so that I can access either of the pods using the same external URL, in a load balanced way. So what I tried to do is put same matching label in both pods and map them to same service and then expose through NodePort. Example:
#pod1.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: dep1
labels:
apps: dep1
tier: cloud
spec:
template:
metadata:
name: dep1-pod
labels:
app: deployment1
spec:
containers:
- name: cont1
image: cont1
ports:
- containerPort: 8081
selector:
matchLabels:
app:deployment1
Now second pod
#pod2.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: dep2
labels:
apps: dep2
tier: cloud
spec:
template:
metadata:
name: dep2-pod
labels:
app: deployment1
spec:
containers:
- name: cont2
image: cont2
ports:
- containerPort: 8081
selector:
matchLabels:
app:deployment1
Now the service:
#service.yaml
apiVersion: v1
kind: Service
metadata:
name: service1
spec:
type: NodePort
ports:
- port: 8081
targetPort: 8081
nodePort: 30169
selector:
app: deployment1
Now this does not work as intended as it refuses to connect to my IP:30169. However, I can connect if only one of the pods are deployed.
Now I know I can achieve this functionality using replicas and just one image, but in this case, I want to do this using 2 images. Any help is much appreciated.
You can use Ingress to achieve it.
Ingress exposes HTTP and HTTPS routes from outside the cluster to services within the cluster. Traffic routing is controlled by rules defined on the Ingress resource.
An Ingress may be configured to give Services externally-reachable URLs, load balance traffic, terminate SSL / TLS, and offer name-based virtual hosting.
An Ingress controller is responsible for fulfilling the Ingress, usually with a load balancer, though it may also configure your edge router or additional frontends to help handle the traffic.
An Ingress does not expose arbitrary ports or protocols. Exposing services other than HTTP and HTTPS to the internet typically uses a service of type Service.Type=NodePort or Service.Type=LoadBalancer.
In your situation Ingress will forward traffic to your services using the same URL. It depends which path you type: URL for first Pod and URL/v2 for second Pod. Of course you can change /v2 to something else.
On the beginning you need enable Ingress on minikube. You can do it using a command below. You can red more about it here
minikube addons enable ingress
Next step you need create a Ingress using a yaml file. Here is an example how to do it step by step.
Yaml file of Ingress looks as below.
As you can see in this configuration, you can access to one Pod using URL and it will forward traffic to first service attached to the first Pod. For the second Pod using URL/v2 it will forward traffic to second service on attached to the second Pod.
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: example-ingress
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /$1
spec:
rules:
- host: hello-world.info
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: web
port:
number: 8080
- path: /v2
pathType: Prefix
backend:
service:
name: web2
port:
number: 8080

GKE Ingress Bad Gateway or Backend not found

I'm creating this issue though there are various answers available but couldn't answer my problem. I'm using GKE Ingress in my example.
I've been using GKE and setup GKE Ingress mere to manage the path for all which we have kept in GCR.
I've created a .net core based API which is has various path , if I expose the deployment with service as type Loadbalancer it works perfectly. Though when I utilize GKE Ingress and setup service and its backend it throws an error like Bad Gateway or sometime backend not found.
.Net core api application exposes following path when i exposed on service as loadbalancer, kindly follow below screenshot for reference:
http://35.202.38.40/api/books
http://35.202.38.40/api/categories
The same way we do have other post api's.
Now I'm stuck when I utilize GKE ingress and setup my ingress.yaml as follows , it doesn't work and throws an error.
Ingress.Yaml
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: gke-ingress
annotations:
# If the class annotation is not specified it defaults to "gce".
kubernetes.io/ingress.class: "gce"
spec:
rules:
- http:
paths:
- path: /*
backend:
serviceName: hello-world
servicePort: 80
- path: /kube
backend:
serviceName: hello-kubernetes
servicePort: 80
NOTE: my hello-word is exposing a .Net Core API application, kindly don't get confuse with name as "hello-world" for testing.
Service Yaml
apiVersion: v1
kind: Service
metadata:
name: hello-world
spec:
type: LoadBalancer
selector:
greeting: hello
department: world
ports:
- protocol: TCP
port: 80
targetPort: 80
Deployment Yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: hello-world-deployment
spec:
selector:
matchLabels:
greeting: hello
department: world
replicas: 2
template:
metadata:
labels:
greeting: hello
department: world
spec:
containers:
- name: hello
image: gcr.io/gcpone-yuy-123114/oneplus2:631
ports:
- containerPort: 80
Kindly advise how to setup path based routing in GKE Ingress for .Net Core Api app
NOTE
Inside the bash shell of pod i tried to reach my api on localhost and it responded as shown below: What am i missing ???

Interplay between Ingress and Service

Ingress can be used to route traffic from an endpoint (or name) to a service which is then forwarded to a pod with matching labels. However, an ingress does not expose arbitrary ports or protocols, which is why services need to be either of type LoadBalancer or NodePort.
So far, so good. I followed a guide to install Traefik as an ingress controller and everything is up and running, but I don't understand why it even works. This question is not related to Traefik; I saw this pattern many times.
The first manifest consists of the ingress deployment and a service called traefik-ingress-service of type LoadBalancer (works with NodePort as well).
---
kind: Deployment
apiVersion: extensions/v1beta1
metadata:
name: traefik-ingress-controller
labels:
k8s-app: traefik-ingress-lb-dep
spec:
replicas: 1
selector:
matchLabels:
k8s-app: traefik-ingress-lb
template:
metadata:
labels:
k8s-app: traefik-ingress-lb
name: traefik-ingress-lb
spec:
terminationGracePeriodSeconds: 60
containers:
- image: traefik:v1.7
name: traefik-ingress-lb
ports:
- name: http
containerPort: 80
- name: admin
containerPort: 8080
args:
- --api
- --kubernetes
- --logLevel=INFO
---
kind: Service
apiVersion: v1
metadata:
name: traefik-ingress-service
spec:
selector:
k8s-app: traefik-ingress-lb
ports:
- protocol: TCP
port: 80
name: web
- protocol: TCP
port: 8080
name: admin
# type: NodePort (works as well, but don't forget to add port to uri)
type: LoadBalancer
In order to expose Traefik's UI there is another manifest including a second service (type: default) traefik-web-ui and an Ingress which connects traffic from / to traefik-web-ui.
apiVersion: v1
kind: Service
metadata:
name: traefik-web-ui
spec:
#type: NodePort
selector:
k8s-app: traefik-ingress-lb
ports:
- name: web
port: 80
targetPort: 8080
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: traefik-web-ui
annotations:
kubernetes.io/ingress.class: traefik
spec:
rules:
- host: dashboard.localhost #optional, otherwise cluster's ip
http:
#- http:
paths:
- path: /
backend:
serviceName: traefik-web-ui
servicePort: web
I don't understand why this is working, because service traefik-web-ui should not be reachable from external. The other service traefik-ingress-service is, but they are not connected. So, how does this work?
Moreover, I'm curious why I would not just create one service traefik-web-ui of type NodePort or LoadBalancer?
That's how an ingress controller is supposed to work.
You have one load balancer managed by the ingress controller. Each time you create an ingress resource, the traefik configuration (running in your traefik pod) is updated accordingly to route the traffic to your services internally (traefik-web-ui service in your case).
This image, taken from this very good article, is quite illustrative.
The ingress controller (the service + deployment in your first manifest) is the big Ingress box, routing the traffic to the internal services.

Bad Gateway with traefik Ingress

I'm using minikube with traefik ingress to create a sticky sessions.
So i have done the deploy of traefik that user guide kubernetes provides me. https://docs.traefik.io/user-guide/kubernetes/
I deploy traefik using DaemonSet. Cause it's a small project and is my first time using kubernetes and docker.
This is my ingress yaml file
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: cp-pluggin
annotations:
kubernetes.io/ingress.class: traefik
spec:
rules:
- host: cppluggins.minikube
http:
paths:
- path: /
backend:
serviceName: cp-pluggin
servicePort: 80
My service yaml file
apiVersion: v1
kind: Service
metadata:
name: cp-pluggin
annotations:
traefik.ingress.kubernetes.io/affinity: "true"
traefik.ingress.kubernetes.io/session-cookie-name: "sticky"
spec:
type: NodePort
ports:
- port: 80
targetPort: 8080
protocol: TCP
name: http
selector:
app: cp-pluggin-app
Finally, my deployment yaml file
apiVersion: apps/v1
kind: Deployment
metadata:
name: cp-pluggin-app
labels:
app: cp-pluggin-app
spec:
replicas: 3
selector:
matchLabels:
app: cp-pluggin-app
template:
metadata:
labels:
app: cp-pluggin-app
spec:
containers:
- name: cp-pluggin-app
image: essoca/ubuntu-tornado
ports:
- containerPort: 8080
I expected
Hello world from: [ipserver]
But i get a
bad gateway
I assume you are using Traefik 2.0, the latest version as of now. There are quite some changes in this version, i.e. the annotations are not used anymore. Besides that, I think the code that you posted is missing a big part of the required changes.
Also, it's not very useful to use a DaemonSet because you are using minikube and that's always one node. Using a Deployment will at least allow you to play with the scale up/down functionality of Kubernetes.
I wrote this article that might be useful for you Traefik 2 as Ingress Controller

How to set up https on kubernetes bare metal using traefik ingress controller

I'm running a kubernetes cluster which consists of three nodes and brilliantly works, but it's time to make my web application secure, so I deployed an ingress controller(traefik). But I was unable to find instructions for setting up https on it. I know most of things I will have to do, like setting up a "secret"(container with certs) etc. but I was wondering how to configure my ingress controller and all files related to it so I would be able to use secure connection
I have already configured ingress controller and created some frontends and backends. Also I configured nginx server(It's actually a web application I'm running) to work on 443 port
My web application deployment
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx
spec:
strategy:
type: Recreate
selector:
matchLabels:
app: nginx
replicas: 3 # tells deployment to run 3 pods matching the template
template: # create pods using pod definition in this template
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: ilchub/my-nginx
ports:
- containerPort: 443
tolerations:
- key: "primary"
operator: Equal
value: "true"
effect: "NoSchedule"
Traefik ingress controller deployment code
kind: Deployment
apiVersion: extensions/v1beta1
metadata:
name: traefik-ingress
namespace: kube-system
labels:
k8s-app: traefik-ingress-lb
spec:
replicas: 1
selector:
matchLabels:
k8s-app: traefik-ingress-lb
template:
metadata:
labels:
k8s-app: traefik-ingress-lb
name: traefik-ingress-lb
spec:
serviceAccountName: traefik-ingress
terminationGracePeriodSeconds: 60
containers:
- image: traefik
name: traefik-ingress-lb
ports:
- name: http
containerPort: 80
- name: https
containerPort: secure
- name: admin
containerPort: 8080
args:
- --api
- --kubernetes
- --logLevel=INFO
Ingress for traefik dashboard
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: traefik-web-ui
namespace: kube-system
spec:
rules:
- host: cluster.aws.ctrlok.dev
http:
paths:
- path: /
backend:
serviceName: traefik-web-ui
servicePort: web
External expose related config
kind: Service
apiVersion: v1
metadata:
name: traefik-ingress-service
namespace: kube-system
spec:
selector:
k8s-app: traefik-ingress-lb
ports:
- protocol: TCP
port: 80
nodePort: 30036
name: web
- protocol: TCP
port: 443
nodePort: 30035
name: secure
- protocol: TCP
port: 8080
nodePort: 30034
name: admin
type: NodePort
What I want to do is securing my application which is already running. Final result has to be a webpage running over https
Actually you have 3 ways to configure Traefik to use https to communicate with backend pods:
If the service port defined in the ingress spec is 443 (note that you can still use targetPort to use a different port on your pod).
If the service port defined in the ingress spec has a name that starts with https (such as https-api, https-web or just https).
If the ingress spec includes the annotation ingress.kubernetes.io/protocol: https.
If either of those configuration options exist, then the backend communication protocol is assumed to be TLS, and will connect via TLS automatically.
Also additional authentication annotations should be added to the Ingress object, like:
ingress.kubernetes.io/auth-tls-secret: secret
And of course, add a TLS Certificate to the Ingress