Traefik - Middleware redirectRegex doesn't work - kubernetes

I have a simple website running in my Kubernetes cluster and exposed to the Internet using Traefik. My Ingress object looks like this (the only things I've changed here are the name and domain names):
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
annotations:
cert-manager.io/cluster-issuer: letsencrypt-staging
kubernetes.io/ingress.class: traefik
name: my-cool-website
namespace: default
spec:
rules:
- host: my-cool-website.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: my-cool-website
port:
number: 80
- host: www.my-cool-website.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: my-cool-website
port:
number: 80
tls:
- hosts:
- my-cool-website.com
- www.my-cool-website.com
secretName: my-cool-website-tls
This works. It allows me to access the site either from my-cool-website.com or from www.my-cool-website.com. But what I'd like to have happen is that if someone visits the former, that Traefik automatically redirects them to the latter. I found a couple of guides online that recommended creating a Traefik middleware, so I did just that, but unfortunately it doesn't work as intended. Here is my middleware definition:
apiVersion: traefik.containo.us/v1alpha1
kind: Middleware
metadata:
name: my-cool-website-force-www
spec:
redirectRegex:
regex: ^https?://my-cool-website.com/(.*)
replacement: https://www.my-cool-website.com/${1}
And then I add the following annotation back to the ingress object:
traefik.ingress.kubernetes.io/router.middlewares: my-cool-website-force-www
But as soon as I do that, it breaks my web app. By that I mean, when that annotation is applied, instead of serving my website, I start seeing a generic nginx page that looks like this when I try to access the domain (and also it does not do the redirect):
I have to assume this Hello World page is being served by Traefik as some sort of generic default page, as it definitely does not originate from my pod. So that tells me something about this middleware (or perhaps how I'm calling it with the annotation) isn't working. What am I missing?

I figured it out, by port-forwarding to the Traefik Dashboard and looking at the service there. It was showing an error for the middleware not found. I then clicked over to the middlewares and realized that they end up with a longer canonical name. So in my annotation I had to change the reference from my-cool-website-force-www to default-my-cool-website-force-www#kubernetescrd and then everything worked.

Related

How to expose a single path of a service via ingress? (not the whole service)

I want my Jenkins to be only accessible in the private network via VPN to protect it from the dangers of the public internet. Nonetheless, I also want to use GitHub webhooks. So the Jenkins needs to expose a single path to the public internet.
To expose the whole service I could simply create an ingress rule like:
rules:
- host: example.com
http:
paths:
- path: /jenkins
pathType: Prefix
backend:
service:
name: jenkins-service
port:
number: 80
This would expose the WHOLE service at https://example.com/jenkins. But this is bad from a security perspective. Therefore I would like to expose only the webhook path of the service. I.e. /generic-webhook-trigger/invoke. So that it is the only URL you can reach from the public internet. Like this:
curl https://example.com/generic-webhook-trigger/invoke
Is that possible? Thanks for any suggestion!
So this doesn't seem to be possible with the native ingress functionalities but requires an ingress controller, like the Nginx Ingress Controller. These come with additional functionalities, like URI rewrite rules (see here).
In case of the Nginx Ingress Controller you need the nginx.ingress.kubernetes.io/rewrite-target and rewrite your ingress path to the desired path.
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: jenkins-ingress
namespace: jenkins
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /jenkins/generic-webhook-trigger/invoke
spec:
ingressClassName: nginx
rules:
- host: example.com
http:
paths:
- path: /jenkins
pathType: Prefix
backend:
service:
name: jenkins-service
port:
number: 80
If you now visit https://example.com/jenkins then it will get internally rewritten to https://example.com/jenkins/generic-webhook-trigger/invoke. No redirects involved.

why annotation: rewrite-target are required in ingress service

I created an ingress service from the kubernetes documentation, but I found that not putting below annotation made service unavailable. Why was this so?
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /
nginx.ingress.kubernetes.io/ssl-redirect: "false"
Main reason to ask this question was that in K8s documentation, I found below code for ingress, but this did not work until and unless I put the above annotation. So, why does the below code not work?
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: ingress-wear-watch
namespace: app-space
spec:
rules:
- http:
paths:
- path: /wear
pathType: Prefix
backend:
service:
name: wear-service
port:
number: 8080
- path: /watch
pathType: Prefix
backend:
service:
name: video-service
port:
number: 8080
Based on k8s docs, all you need to know in this specific case is that
you can use Kubernetes annotations to attach arbitrary non-identifying metadata to objects. Clients such as tools and libraries can retrieve this metadata.
[annotations are] Directives from the end-user to the implementations to modify behavior or engage non-standard features.
So basically this specific annotations are used to modify behaviour of ingress and are specififc to nginx ingress controller (this means that if you use different controller, this annotations wont work).
All supported annotations for nginx ingress controler are explained in Nginx Ingress Controller documention.

Kubernetes Google OAuth2 Sign In

I'm trying to figure this out for a while and can't seem to find an answer anywhere. Maybe someone here will have an idea.
I have deployed an application on Kubernetes Cluster (in GCP). The Application is pretty much a micro-service type of app under a single domain. What I'm trying to achieve at the moment is to have a domain-wide-authentication via Google and OAuth2 protocol.
So I have a main app under the main domain https://example.com and pretty much every single path under that domain will be a separate K8s Service + Deployment. So it's a different set of containers for https://example.com/foo and different for https://example.com/bar and obviously different set of containers for the main https://example.com. I don't want to "connect" each individual App to Google due to two reasons:
Would require for me that each app has the OAuth2 Protocol implemented, and
A token granted by one app would be invalid for the other one, so it would require the user to log in each time he/she changes the URL, kind bad UX if you ask me
So, I'm trying to set up in the Kubernetes Cluster an Nginx Ingress Controller that would facilitate authentication and validate all incoming requests before they would reach any of the backend apps. This is where the problem lies...
I've managed to set up OAuth2 Proxy in my Nginx Ingress Controller and the user can log in. The whole OAuth2 Flow works. HOWEVER, the user is not forced to log in. The user can navigate to the https://example.com but he/she won't be required to have an Authentication cookie granted after successful login via Google.
Here is my YAML config for Ingress:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
annotations:
nginx.ingress.kubernetes.io/auth-url: "https://$host/oauth2/auth"
nginx.ingress.kubernetes.io/auth-signin: "https://$host/oauth2/start?rd=$escaped_request_uri"
kubernetes.io/ingress.class: "nginx"
ingress.kubernetes.io/force-ssl-redirect: "true"
name: example-com-ingress
namespace: default
spec:
tls:
- hosts:
- example.com
secretName: example-com-tls
rules:
- host: example.com
http:
paths:
- backend:
serviceName: oauth2-proxy
servicePort: http-proxy
path: /oauth2
- backend:
serviceName: example-com-nginx-service
servicePort: 80
path: /
Any ideas anyone?
Make sure that while configuring oauth2 proxy you set your cookie domain (set --cookie-domain=example.com to allow the cookie to be read).
You have to also add the whitelist-domain feature while configuring oauth2 proxy:
--whitelist-domain=example.com
Then have to create ingress' obects:
1. Expose endpoint /oauth2, look at the example below:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: oauth2-proxy
annotations:
kubernetes.io/ingress.class: "nginx"
nginx.ingress.kubernetes.io/backend-protocol: "HTTPS"
spec:
rules:
- host: example.com
http:
paths:
- backend:
serviceName: oauth2-proxy
servicePort: 4180
path: /oauth2
tls:
- hosts:
- example.com
secretName: example-com-tls
2. Then create ingress for your application:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: example-com-ingress
annotations:
kubernetes.io/ingress.class: "nginx"
nginx.ingress.kubernetes.io/auth-url: "https://$host/oauth2/auth"
nginx.ingress.kubernetes.io/auth-signin: "https://$host/oauth2/start?rd=$escaped_request_uri"
nginx.ingress.kubernetes.io/force-ssl-redirect: "true
nginx.ingress.kubernetes.io/backend-protocol: "HTTPS"
spec:
tls:
- hosts:
- example.com
secretName: example-com-tls
rules:
- host: example.com
http:
paths:
- path: /
backend:
serviceName: example-com-nginx-service
servicePort: 80
It is clearly to create two separate ingresses, but you can also create just one with two defined backends as you wanted.
Take a look here: nginx-ingress-controller-oauth2, oauth2-dynamic-callbacks, oauth2-proxy.
Once again make sure that you have follow this instruction: external-OAUTH-authentication.

Kubernetes Ingress not loading static assets

I have a ReactJS front end, Spring boot backend app deployed on a baremetal Kubernetes cluster which is running Kubernetes Ingress and requests are proxied to it by HAProxy. When visiting the URL of the app, I can see it loads the index.html of the app but all other requests to static assets are not done properly.
The ingress resource of my app:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
namespace: app
name: app-ingress
annotations:
kubernetes.io/ingress.class: "nginx"
nginx.ingress.kubernetes.io/ssl-redirect: "false"
nginx.ingress.kubernetes.io/rewrite-target: /$3
spec:
rules:
- host: devops
http:
paths:
- path: /dev/app1(/|$)(.*)
backend:
serviceName: app1
servicePort: 80
When inspecting the page which loads using Chrome Developer tools, I see that there are 6 outgoing calls to the static assets. The call that retrieves the index.html of the app completes succesfully but the calls that retrieve the static assets (ex: http://devops/dev/app1/static/js/4.2761693d.chunk.js) does not work properly as it retrieves the index.html page as well. (only the index.html page is served by all calls basically)
I had a feeling it is because of the nginx.ingress.kubernetes.io/rewrite-target annotation but removing it causes a 404 even on the index.html page.
I am using nginx ingress controller 0.25.1
EDIT:
This is the output when I exec into the container and run curl localhost:8080/dev/app1/static/js/4.2761693d.chunk.js (error fallback page)
This is the output when I run curl localhost:8080/tatic/js/4.2761693d.chunk.js (correctly loads the css)
Somehow, when I change the rewrite annotation to this, it works:
nginx.ingress.kubernetes.io/rewrite-target: /$2
I didnt change anything else.
Now the application is accessible at devops/dev/app1/ (but it does not work without the / at the end)
I am not sure how this works. I had no logic behind it, I was just changing values in the ingress file to see if anything works.
Can someone explain why it works?
You are most likely using rewrite target for wrong reasons. In this case all elements should start with /dev/app1 including inner calls. What you are doing is working for index page because you wrote it to have "/dev/app1" in before and ingress redirected it to "/" but inner call just called "/static/js/4.2761693d.chunk.js" . This caused the problem basically, because ingress didn't know the route therefore your service never get called to get the js file.
Im using kubernetes 1.25.2 along with Nginx-Ingress v2.4.1
https://docs.nginx.com/nginx-ingress-controller/installation/installation-with-manifests/
Just us the below 2...the Path should have / at the end.
annotations:
nginx.org/rewrites: "serviceName=app1-svc rewrite=/;serviceName=app2-svc rewrite=/"
spec:
path: /app1/
below is my Ingress resource file manifest...
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: relouat-ingress
namespace: relouat
annotations:
#nginx.ingress.kubernetes.io/rewrite-target: /$2
nginx.org/rewrites: "serviceName=app1-svc rewrite=/;serviceName=app2-svc rewrite=/"
spec:
ingressClassName: nginx
rules:
- host: uat.relo.com
http:
paths:
- backend:
service:
name: app1-svc
port:
number: 2041
path: /app1/
pathType: Prefix
- backend:
service:
name: app2-svc
port:
number: 2042
path: /app2/
pathType: Prefix

Kubernetes: path based only works with root

I have an EKS cluster and currently use path based routing. I previously posted this thread, so all my configuration is on there: Kubernetes: 502 (Bad getaway)
My ingress controller is from there: https://raw.githubusercontent.com/kubernetes/ingress-nginx/nginx-0.24.1/deploy/mandatory.yaml
Here is my ingress:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: simple-fanout-example
namespace : default
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/rewrite-target: /$2
spec:
rules:
- host: domain.com
http:
paths:
- path: /bleble(/|$)(.*)
backend:
serviceName: bleble-svc
servicePort: 8080
- path: /hello-world
backend:
serviceName: hello-world-svc
servicePort: 8080
This works absolutly fine, until you try to go anywhere that is not the domain.com/bleble or domain.com/hello-world. The services need to refer to each other, and the uri they request is just domain.com, which obviously doesn't work because the service is at domain.com/nameoftheservice.
The 2 problems are :
The service name is bleble, so we decided the path should be /bleble. I saw on this documentation (kubernetes.github.io/ingress-nginx/examples/rewrite) that adding (/|$)(.*) would allow rewrite. We want to use url as domain.com/bleble/swagger or domain.com/bleble/clients, ... But when we type those, it brings back to what was on /bleble
I need bleble to get an info from hello-world. Right now, instead of going from domain.com/bleble to domain.com/hello-world, it goes from domain.com/bleble to domain.com. it seems to only be able to call the host name, and not the path.
I've tried doing a single ingress resource and have nginx.ingress.kubernetes.io/app-root : /bleble but this doesn't seem to be working. I've tried the annotation as well nginx.ingress.kubernetes.io/rewrite-target: /coretest
Thanks for the help!