Hi I am very new to kubernetes. I have a k3s.io cluster setup (Server version 1.20) and I want to run multiple web apps under the same domain. I use the k3s default ingress controller (traefik).
Depending on the path given, the request should be routed to the configured web app.
dev.xxxxxxx.de/app -> should go to my self developed .net blazor webassembly app
dev.xxxxxxx.de/graf -> should go to the grafana service/pod
Both apps/services are running in the browser and can be reached sucessfully by their service name via NodePort (without ingress). So I suspect the problem in ingress routing.
My ingress.yaml:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: ingress
labels:
environment: dev
annotations:
kubernetes.io/ingress.class: "traefik"
spec:
tls:
- hosts:
- dev.xxxxxxx.de
rules:
- host: dev.xxxxxxx.de
http:
paths:
- path: /app
pathType: ImplementationSpecific
backend:
service:
name: iot-app
port:
number: 80
- path: /graf
pathType: ImplementationSpecific
backend:
service:
name: grafana
port:
number: 3000
The problem now is that when browsing to dev.xxxxxxx.de/app or dev.xxxxxxx.de/graf, in both cases the initial request is returned with a 200, but the subsequent requests like assets (css, js) return a 404.
I suspect that the URL
dev.xxxxx.de/app/bootstrap.min.css
is getting transformed to
dev.xxxxx.de/bootstrap.min.css
which leads to a 404.
All hints why I cannot fully browse both of the apps appreciated! Thanks.
As I solved the very same issue right now on my K3s-cluster:
Have you tried to start your master with the --cluster-domain option (see the docs)?
I added that option in the /etc/systemd/system/k3s.service of my master and that did the trick for me.
Related
My modern application hosted within kubernetes works based on the host and the path app.dev.sandbox.com/ (root / path). I need access external legacy application hosted on EC2/VM with host name app-legacy.dev.sandbox.com via the kubernetes ingress. I want to be able to access the legacy sites pages with the same host name of application deployed on kubernetes like this app.dev.sandbox.com/nbs/login or app.dev.sandbox.com/nbs/homepage.do should actually request app-legacy.dev.sandbox.com/abc/login or app-legacy.dev.sandbox.com/nbs/homepage.do but preserve the modern hostname of app.dev.sandbox.com when displaying the page.
This is needed because, we are using strangler fig pattern to navigate between legacy and modern pages seamlessly. The request needs to be captured by the kubernetes ingress when the user clicks on certain links on the legacy pages. I am currently using the following external service and ingress resource.
apiVersion: v1
kind: Service
metadata:
name: nbs-legacy
spec:
type: ExternalName
externalName: app-legacy.dev.sandbox.com
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: ingress-resource
annotations:
kubernetes.io/ingress.class: "nginx"
cert-manager.io/cluster-issuer: "letsencrypt-production"
nginx.ingress.kubernetes.io/rewrite-target: /
nginx.ingress.kubernetes.io/preserve-host: "true"
spec:
ingressClassName: nginx
tls:
- secretName: app.dev.sandbox.com
hosts:
- app.dev.sandbox.com
rules:
- host: app.dev.sandbox.com
http:
paths:
- path: /nbs #I would like the path to be appened to the nbs-legacy request. For example /nbs/login should be appened while requesting app-legacy.dev.sandbox.com/nbs/login
pathType: Exact
backend:
service:
name: nbs-legacy
port:
number: 80
- path: /
pathType: Prefix
backend:
service:
name: modern-service
port:
number: 80
I am using nginx ingress. I tried with the above manifests but did not work.
I am expecting the following:
Navigate to external legacy application seamlessly while preserving the host name of the kubernetes.
Capture path from the request and pass that to legacy host name request. For example, /nbs/login or /nbs/dashboard should be passed when requesting legacy application pages.
Avoid routing of anything that starts with /nbs(eg: /nbs/abc) to the default root / path.
I have a simple website running in my Kubernetes cluster and exposed to the Internet using Traefik. My Ingress object looks like this (the only things I've changed here are the name and domain names):
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
annotations:
cert-manager.io/cluster-issuer: letsencrypt-staging
kubernetes.io/ingress.class: traefik
name: my-cool-website
namespace: default
spec:
rules:
- host: my-cool-website.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: my-cool-website
port:
number: 80
- host: www.my-cool-website.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: my-cool-website
port:
number: 80
tls:
- hosts:
- my-cool-website.com
- www.my-cool-website.com
secretName: my-cool-website-tls
This works. It allows me to access the site either from my-cool-website.com or from www.my-cool-website.com. But what I'd like to have happen is that if someone visits the former, that Traefik automatically redirects them to the latter. I found a couple of guides online that recommended creating a Traefik middleware, so I did just that, but unfortunately it doesn't work as intended. Here is my middleware definition:
apiVersion: traefik.containo.us/v1alpha1
kind: Middleware
metadata:
name: my-cool-website-force-www
spec:
redirectRegex:
regex: ^https?://my-cool-website.com/(.*)
replacement: https://www.my-cool-website.com/${1}
And then I add the following annotation back to the ingress object:
traefik.ingress.kubernetes.io/router.middlewares: my-cool-website-force-www
But as soon as I do that, it breaks my web app. By that I mean, when that annotation is applied, instead of serving my website, I start seeing a generic nginx page that looks like this when I try to access the domain (and also it does not do the redirect):
I have to assume this Hello World page is being served by Traefik as some sort of generic default page, as it definitely does not originate from my pod. So that tells me something about this middleware (or perhaps how I'm calling it with the annotation) isn't working. What am I missing?
I figured it out, by port-forwarding to the Traefik Dashboard and looking at the service there. It was showing an error for the middleware not found. I then clicked over to the middlewares and realized that they end up with a longer canonical name. So in my annotation I had to change the reference from my-cool-website-force-www to default-my-cool-website-force-www#kubernetescrd and then everything worked.
I want my Jenkins to be only accessible in the private network via VPN to protect it from the dangers of the public internet. Nonetheless, I also want to use GitHub webhooks. So the Jenkins needs to expose a single path to the public internet.
To expose the whole service I could simply create an ingress rule like:
rules:
- host: example.com
http:
paths:
- path: /jenkins
pathType: Prefix
backend:
service:
name: jenkins-service
port:
number: 80
This would expose the WHOLE service at https://example.com/jenkins. But this is bad from a security perspective. Therefore I would like to expose only the webhook path of the service. I.e. /generic-webhook-trigger/invoke. So that it is the only URL you can reach from the public internet. Like this:
curl https://example.com/generic-webhook-trigger/invoke
Is that possible? Thanks for any suggestion!
So this doesn't seem to be possible with the native ingress functionalities but requires an ingress controller, like the Nginx Ingress Controller. These come with additional functionalities, like URI rewrite rules (see here).
In case of the Nginx Ingress Controller you need the nginx.ingress.kubernetes.io/rewrite-target and rewrite your ingress path to the desired path.
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: jenkins-ingress
namespace: jenkins
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /jenkins/generic-webhook-trigger/invoke
spec:
ingressClassName: nginx
rules:
- host: example.com
http:
paths:
- path: /jenkins
pathType: Prefix
backend:
service:
name: jenkins-service
port:
number: 80
If you now visit https://example.com/jenkins then it will get internally rewritten to https://example.com/jenkins/generic-webhook-trigger/invoke. No redirects involved.
I am using FastAPI to create multiple services.
It works in a somewhat strange way, but basically I have a generic FastAPI service and a CI/CD that downloads multiple installable packages to create different services from it. It's all packaged neatly, I create multiple Docker images, then they are served by Kubernetes using deployments, services (separate pods, one container per pod) and ingress.
Now, I've configured ingress to serve those services from the same host. The only thing that is different for those services is the app root path. So for example I have:
foo.example.com/foo - 1st service
foo.example.com/bar - 2nd service
If I go to foo.example.com/foo/docs, I am greeted with Swagger documentation and I can select the app root path at the top of the page. Unfortunately, there's only one option: /foo. Same for the second service, but this time it's /bar.
Now I know that those services run in separate containers in separate pods in my k8s cluster, but I was wondering if there's a way to for example go to foo.example.com or to a particular service like foo.example.com/foo and have all the services listed in Swagger's drop-down menu, so I can select any of them and be redirected to it. It would be huge improvement from a user experience point of view.
#Edit, Ingress manifest file:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: my-ingress
annotations:
kubernetes.io/ingress.class: "nginx"
nginx.ingress.kubernetes.io/rewrite-target: /$1
external-dns/manage-entries: "true"
spec:
rules:
- http:
paths:
- path: /foo/(.*)
pathType: Prefix
backend:
service:
name: foo-service
port:
number: 80
host: test.example.com
- http:
paths:
- path: /bar/(.*)
pathType: Prefix
backend:
service:
name: bar-service
port:
number: 80
host: test.example.com
- http:
paths:
- path: /baz/(.*)
pathType: Prefix
backend:
service:
name: baz-service
port:
number: 80
host: test.example.com
I have a ReactJS front end, Spring boot backend app deployed on a baremetal Kubernetes cluster which is running Kubernetes Ingress and requests are proxied to it by HAProxy. When visiting the URL of the app, I can see it loads the index.html of the app but all other requests to static assets are not done properly.
The ingress resource of my app:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
namespace: app
name: app-ingress
annotations:
kubernetes.io/ingress.class: "nginx"
nginx.ingress.kubernetes.io/ssl-redirect: "false"
nginx.ingress.kubernetes.io/rewrite-target: /$3
spec:
rules:
- host: devops
http:
paths:
- path: /dev/app1(/|$)(.*)
backend:
serviceName: app1
servicePort: 80
When inspecting the page which loads using Chrome Developer tools, I see that there are 6 outgoing calls to the static assets. The call that retrieves the index.html of the app completes succesfully but the calls that retrieve the static assets (ex: http://devops/dev/app1/static/js/4.2761693d.chunk.js) does not work properly as it retrieves the index.html page as well. (only the index.html page is served by all calls basically)
I had a feeling it is because of the nginx.ingress.kubernetes.io/rewrite-target annotation but removing it causes a 404 even on the index.html page.
I am using nginx ingress controller 0.25.1
EDIT:
This is the output when I exec into the container and run curl localhost:8080/dev/app1/static/js/4.2761693d.chunk.js (error fallback page)
This is the output when I run curl localhost:8080/tatic/js/4.2761693d.chunk.js (correctly loads the css)
Somehow, when I change the rewrite annotation to this, it works:
nginx.ingress.kubernetes.io/rewrite-target: /$2
I didnt change anything else.
Now the application is accessible at devops/dev/app1/ (but it does not work without the / at the end)
I am not sure how this works. I had no logic behind it, I was just changing values in the ingress file to see if anything works.
Can someone explain why it works?
You are most likely using rewrite target for wrong reasons. In this case all elements should start with /dev/app1 including inner calls. What you are doing is working for index page because you wrote it to have "/dev/app1" in before and ingress redirected it to "/" but inner call just called "/static/js/4.2761693d.chunk.js" . This caused the problem basically, because ingress didn't know the route therefore your service never get called to get the js file.
Im using kubernetes 1.25.2 along with Nginx-Ingress v2.4.1
https://docs.nginx.com/nginx-ingress-controller/installation/installation-with-manifests/
Just us the below 2...the Path should have / at the end.
annotations:
nginx.org/rewrites: "serviceName=app1-svc rewrite=/;serviceName=app2-svc rewrite=/"
spec:
path: /app1/
below is my Ingress resource file manifest...
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: relouat-ingress
namespace: relouat
annotations:
#nginx.ingress.kubernetes.io/rewrite-target: /$2
nginx.org/rewrites: "serviceName=app1-svc rewrite=/;serviceName=app2-svc rewrite=/"
spec:
ingressClassName: nginx
rules:
- host: uat.relo.com
http:
paths:
- backend:
service:
name: app1-svc
port:
number: 2041
path: /app1/
pathType: Prefix
- backend:
service:
name: app2-svc
port:
number: 2042
path: /app2/
pathType: Prefix