Recently I am moving a project to Kubernetes and have used Traefik as the ingress controller. For Traefik I have used the Traefik Kubernetes Ingress provider for routing. When I tried to add the Traefik dashboard, I found that seems it can only be added using IngressRoute (ie. using Kubernetes CRD as provider).
I have a few questions:
Is it possible to use Traefik Kubernetes Ingress provider to bring up the dashboard?
Can I use both kubernetesingress and kubernetescrd as provider? Can both Ingress and IngressRoute co-exist?
So I have solved the Traefik Dashboard problem using Traefik Kubernetes Ingress only, the answer to the first question is 'Yes':
The following is my configuration:
traefik-deployment.yaml
kind: Deployment
apiVersion: apps/v1
metadata:
name: traefik
namespace: ingress-traefik
labels:
app: traefik
spec:
replicas: 1
selector:
matchLabels:
app: traefik
template:
metadata:
labels:
app: traefik
spec:
serviceAccountName: traefik-ingress-controller
containers:
- name: traefik
image: traefik:v2.2
ports:
- name: web
containerPort: 80
- name: websecure
containerPort: 443
- name: admin
containerPort: 8080
args:
- --api
- --api.insecure=true
- --api.dashboard=true
- --providers.kubernetesingress
- --providers.kubernetescrd
- --entrypoints.web.Address=:80
- --entrypoints.websecure.Address=:443
traefik-dashboard-ingress.yaml
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: traefik-dashboard-ingress
namespace: ingress-traefik
annotations:
kubernetes.io/ingress.class: traefik
traefik.ingress.kubernetes.io/router.entrypoints: web, websecure
traefik.ingress.kubernetes.io/router.tls: "true"
traefik.ingress.kubernetes.io/router.middlewares: ingress-traefik-traefikbasicauth#kubernetescrd
cert-manager.io/cluster-issuer: letsencrypt-prod
spec:
tls:
- secretName: cert-stage-wildcard
rules:
- host: traefik.your-domain.io
http:
paths:
- path: /
backend:
serviceName: traefik-service
servicePort: 8080
The key to bringing up this is to set api.insecure=true, with this I can port-forward and test the Traefik Dashboard on my localhost, and then route the service through the traefik kubernetes ingress.
Another question (Can I use both kubernetesingress and kubernetescrd as provider) is also confirmed to be 'Yes', as I am now using them together, with kubernetesingress for routing and kubernetescrd on the basicAuth MiddleWare.
But I guess the two routing schemes ingress and ingressRoute may not be able to co-exist as they are both for routing and only one of them will be used by the system when both of them exist. Please correct me if I am wrong.
Related
I've written a small Blazor application which looks to work all well when containerized and when accessed through k3s port forwarding, but struggling to find a guideline on how that application needs to be correctly exposed via a ingress controller. To show this:
If I run the Blazor application and access via port-forwarding (blazor routing works perfectly well etc.):
kubectl port-forward deployment/ 8000:80
and page routing working as expected
However, when I add a clusterIP service to the deployment and connect to it through Traefik ingress controller, I get:
and changing the route will give a 404 page not found error
My Ingress serviceIp and ingress controller setup:
ClusterIP:
apiVersion: v1
kind: Service
metadata:
name: driverpassthrough
spec:
selector:
app: driverpassthrough
ports:
- name: ui
protocol: TCP
port: 8010
targetPort: 80
ingress
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: ingress-test
annotations:
kubernetes.io/ingress.class: "traefik"
traefik.ingress.kubernetes.io/rewrite-target: /
spec:
rules:
- http:
paths:
- path: /passthrough
backend:
serviceName: driverpassthrough
servicePort: 8010
so in my case, I use k3s and traefik. I also have 3 replicas of my blazor server app. To make it work, I had to enable sticky session (in annotations) on the cluster ip like so:
Service
apiVersion: v1
kind: Service
metadata:
name: qscale-healthcheck-service
annotations:
traefik.ingress.kubernetes.io/affinity: "true"
labels:
name: qscale-healthcheck-service
spec:
type: ClusterIP
selector:
app: healthcheck
ports:
- name: http
port: 80
targetPort: 80
Ingress
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: traefik-ingress
annotations:
kubernetes.io/ingress.class: "traefik"
spec:
rules:
- http:
paths:
- path: /
backend:
serviceName: qscale-healthcheck-service
servicePort: 80
This is the link where I found the annotation: Traefik Doc
Stack:
Azure Kubernetes Service
NGINX Ingress Controller - https://github.com/kubernetes/ingress-nginx
AKS Loadbalancer
Docker containers
My goal is to create a K8s cluster that will allow me to use multiple pods, under a single IP, to create a microservice architecture. After working with tons of tutorials and documentation, I'm not having any luck with my endgoal. I got to the point of being able to access a single deployment using the Loadbalancer, but introducing the ingress has not been successful so far. The services are separated into their respective files for readability and ease of control.
Additionally, the Ingress Controller was added to my cluster as described in the installation instructions using: kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v0.35.0/deploy/static/provider/cloud/deploy.yaml
LoadBalancer.yml:
apiVersion: v1
kind: Service
metadata:
name: backend
spec:
loadBalancerIP: x.x.x.x
selector:
app: ingress-service
tier: backend
ports:
- name: "default"
port: 80
targetPort: 80
type: LoadBalancer
IngressService.yml:
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: ingress-service
annotations:
kubernetes.io/ingress.class: nginx
spec:
rules:
- http:
paths:
- path: /api
backend:
serviceName: api-service
servicePort: 80
api-deployment.yml
apiVersion: v1
kind: Service
metadata:
name: api-service
spec:
selector:
app: api
ports:
- port: 80
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: api-deployment
spec:
selector:
matchLabels:
app: api
tier: backend
track: stable
replicas: 1
template:
metadata:
labels:
app: api
tier: backend
track: stable
spec:
containers:
- name: api
image: image:tag
ports:
- containerPort: 80
imagePullPolicy: Always
imagePullSecrets:
- name: SECRET
The API in the image is exposed on port 80 correctly.
After applying each of the above yml services and deployments, I attempt a web request to one of the api resources via the LoadBalancer's IP and receive only a timeout on my requests.
Found my answer after hunting around enough. Basically, the problem was that the Ingress Controller has a Load Balancer built into the yaml, as mentioned in comments above. However, the selector for that LoadBalancer requires marking your Ingress service as part of the class. Then that Ingress service points to each of the services attached to your pods. I also had to make a small modification to allow using a static IP in the provided load balancer.
Getting error msg when I trying to access with public IP:
"{"message":"failure to get a peer from the ring-balancer"}"
Looks like Kong is unable to the upstream services.
I am using voting app
ingress.yaml
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: telehealth-ingress
namespace: kong
annotations:
kubernetes.io/ingress.class: "kong"
spec:
rules:
- http:
paths:
- backend:
serviceName: voting-service
servicePort: 80
service.yaml
apiVersion: v1
kind: Service
metadata:
name: voting-service
labels:
name: voting-service
app: voting-app
spec:
ports:
- targetPort: 80
port: 80
selector:
name: voting-app-pod
app: voting-app
deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: voting-app-pod
labels:
name: voting-app-pod
app: voting-app
spec:
template:
metadata:
labels:
name: voting-app-pod
app: voting-app
spec:
containers:
- name: voting-app
image: dockersamples/examplevotingapp_vote
ports:
- containerPort: 80
replicas: 2
selector:
matchLabels:
app: voting-app
There could be one of many things wrong here. But essentially your ingress cannot get to your backend.
If your backend up and running?
Check backend pods are "Running"
kubectl get pods
Check backend deployment has all replicas up
kubectl get deploy
Connect to the app pod and run a localhost:80 request
kubectl exec -it <pod-name> sh
# curl http://localhost
Connect to the ingress pod and see if you can reach the service from there
kubectl exec -it <ingress-pod-name> sh
# dig voting-service (can you DNS resolve it)
# telnet voting-sevice 80
# curl http://voting-service
This issue might shed some insights as to why you can't reach the backend service. What http error code are you seeing?
The problem is resolved after deploying services and deployments in kong namespace instead of default namespace. Now I can access the application with Kong ingress public IP.
Looks like kong ingress is not able to resolve DNS with headless DNS. We need mention FQDN in ingress yaml
ingress.yaml
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: telehealth-ingress
namespace: kong
annotations:
kubernetes.io/ingress.class: "kong"
spec:
rules:
- http:
paths:
- backend:
name: voting-service
Port:
number: 80
Try this i thing it will work
I am currently playing around with a rpi based k3s cluster and I am observing a weird phenomenon.
I deployed two applications.
The first one is nginx which I can reach on the url http://external-ip/foo based on the following ingress rule:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: foo
namespace: foo
annotations:
kubernetes.io/ingress.class: traefik
traefik.ingress.kubernetes.io/rule-type: "PathPrefixStrip"
traefik.ingress.kubernetes.io/rewrite-target: "/"
spec:
rules:
- http:
paths:
- path: /foo
backend:
serviceName: foo-service
servicePort: 8081
And the other one is grafana which I cannot reach on the url http://external-ip/grafana based on the below ingress rule:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: grafana
namespace: grafana
annotations:
kubernetes.io/ingress.class: traefik
traefik.ingress.kubernetes.io/rule-type: "PathPrefixStrip"
traefik.ingress.kubernetes.io/rewrite-target: "/"
spec:
rules:
- http:
paths:
- path: /grafana
backend:
serviceName: grafana-service
servicePort: 3000
When I do a port-forward directly on the pod I can reach the grafana app, when I use the port-forward on the grafana service it also works.
However as soon as I try to reach it through the subpath I will get a gateway timeout.
Does anyone have a guess what I am missing?
Here the deployment and service for the grafana deployment:
apiVersion: apps/v1
kind: Deployment
metadata:
name: grafana
namespace: grafana
labels:
app: grafana
tier: frontend
service: monitoring
spec:
selector:
matchLabels:
app: grafana
tier: frontend
template:
metadata:
labels:
app: grafana
tier: frontend
service: monitoring
spec:
containers:
- image: grafana
imagePullPolicy: IfNotPresent
name: grafana
envFrom:
- configMapRef:
name: grafana-config
ports:
- name: frontend
containerPort: 3000
protocol: TCP
---
apiVersion: v1
kind: Service
metadata:
name: grafana-service
namespace: grafana
labels:
app: grafana
tier: frontend
service: monitoring
spec:
selector:
app: grafana
tier: frontend
type: NodePort
ports:
- name: frontend
port: 3000
protocol: TCP
targetPort: 3000
Solution
I had to add the following two parameters to my configmap to make it work:
GF_SERVER_ROOT_URL=http://localhost:3000/grafana/
GF_SERVER_FROM_SUB_PATH=true
As I mentioned in comments grafana is not listening on / like default nginx.
There is related github issue about this, and if you want to make it work you should specify root_url
grafana.ini:
server:
root_url: https://subdomain.example.com/grafana
Specifically take a look at this and this comment.
#tehemaroo add his own solution which include changing root url and sub_path in configmap
I had to add the following two parameters to my configmap to make it work:
GF_SERVER_ROOT_URL=http://localhost:3000/grafana/
GF_SERVER_FROM_SUB_PATH=true
And related documentation about that
To serve Grafana behind a sub path:
Include the sub path at the end of the root_url.
Set serve_from_sub_path to true.
[server]
domain = example.com
root_url = %(protocol)s://%(domain)s:%(http_port)s/grafana/
serve_from_sub_path = true
I'm using minikube with traefik ingress to create a sticky sessions.
So i have done the deploy of traefik that user guide kubernetes provides me. https://docs.traefik.io/user-guide/kubernetes/
I deploy traefik using DaemonSet. Cause it's a small project and is my first time using kubernetes and docker.
This is my ingress yaml file
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: cp-pluggin
annotations:
kubernetes.io/ingress.class: traefik
spec:
rules:
- host: cppluggins.minikube
http:
paths:
- path: /
backend:
serviceName: cp-pluggin
servicePort: 80
My service yaml file
apiVersion: v1
kind: Service
metadata:
name: cp-pluggin
annotations:
traefik.ingress.kubernetes.io/affinity: "true"
traefik.ingress.kubernetes.io/session-cookie-name: "sticky"
spec:
type: NodePort
ports:
- port: 80
targetPort: 8080
protocol: TCP
name: http
selector:
app: cp-pluggin-app
Finally, my deployment yaml file
apiVersion: apps/v1
kind: Deployment
metadata:
name: cp-pluggin-app
labels:
app: cp-pluggin-app
spec:
replicas: 3
selector:
matchLabels:
app: cp-pluggin-app
template:
metadata:
labels:
app: cp-pluggin-app
spec:
containers:
- name: cp-pluggin-app
image: essoca/ubuntu-tornado
ports:
- containerPort: 8080
I expected
Hello world from: [ipserver]
But i get a
bad gateway
I assume you are using Traefik 2.0, the latest version as of now. There are quite some changes in this version, i.e. the annotations are not used anymore. Besides that, I think the code that you posted is missing a big part of the required changes.
Also, it's not very useful to use a DaemonSet because you are using minikube and that's always one node. Using a Deployment will at least allow you to play with the scale up/down functionality of Kubernetes.
I wrote this article that might be useful for you Traefik 2 as Ingress Controller