I would like to use parameter passed in URL to route traffic to pods with specific labels. I can't tell in advance which values of app will be used, so it must somehow dynamically create Ingres rules/Services:
Example:
https://foo.bar.com/?app=application1 --> route to service/pods with label: app=application1
https://foo.bar.com/?app=application2 --> route to service/pods with label: app=application2
https://foo.bar.com/?app=non-existing-app --> route to service/pods without any label app.
I am wondering if it's possible using variables in path and corresponding services, here is the idea and pseudo-code:
kind: Ingress
spec:
rules:
- host: foo.bar.com
http:
paths:
- path: /app=${APPLICATION}
backend:
serviceName: app-${APPLICATION}
servicePort: 8080
Related
In a Kubernetes ingress, I'd like to dynamically change the service name to a match on a certain part of the path in an ingress. Something like this:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: example
spec:
rules:
- host: example.com
http:
paths:
- path: /api/(^/+)/(.*)
backend:
serviceName: api-(&1)
servicePort: 80
Where &1 in the example above would resolve to the first regular expression match in the string /api/(^/+)/(.*)
So basically if a path is something like this /api/asset/fetch/anything-at-all, then it would direct the call to a service named api-asset
I know that this can potentially lead to some security issues, but if I force the prefix 'api-' to the service name then only services with the prefix 'api-' would technically be exposed to the internet. And all other services would still be protected inside the Kubernetes cluster. I can even change the prefix to be 'public-' for simplicity's sake.
Is this something that is possible? And what is the correct syntax to set it up?
I am trying to set up an Ingress controller on Microk8s to host a react (NextJS) app.
The pod and service are up and running, as well as reachable internally via machine.domain.eu:31111
My goal is to make this service available via machine.domain.eu/dev on the default port 80 using an nginx ingress controller:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: web-ingress
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /$2
spec:
rules:
- host: machine.domain.eu
http:
paths:
- path: /dev(/|$)(.*)
pathType: Prefix
backend:
service:
name: web-svc
port:
number: 3000
If I issue a curl command via curl machine.domain.eu/dev from the machine, where the node is running, I am getting back the desired html, although I cannot see, if assets are being loaded.
When switching back to my local machine opening my browser of choice and hit that http endpoint I do get a blank screen and all static assets 404. The index.html comes through fine, yet all the other resources are not found.
I've tried any number of combinations on the annotation to rewrite to /$1 /$2, while changing the path to - path: /dev/?(.*) but to no effect.
In other words: The document itself is loaded via
http://machine.domain.eu/dev
Yet the assets, which should be loaded via
http://machine.domain.eu/dev/_next/{....}
are actually being requested via:
http://machine.domain.eu/_next/{....}
What excactly am I doing wrong here?
First a little background:
We currently have several websites and services hosted on a Plesk server and I am setting up a bare-metal development server to provide an area where we can test updates, etc. before going to production. I am using a 3 node kubernetes cluster running microk8s on Ubunutu 20.04.01. The services we host are pretty diverse: we have a couple Moodle sites, a few Wordpress sites, a site running limesurvery, an instance of Mantis bugtracker, and a few more. I have successfully gotten most of the sites containerized and running on k8s. I can also access each individual site either through a NodePort or a MetalLB load balancer.
However, I'd really like to use the NGINX Ingress Controller on top of the load balancer so that I can have a consistent way to access the sites without using a bunch of IP addresses (or in the NodePort case, ports that change). No matter what I've done, I cannot seem to get the Ingress to do what I want. I simply want to do the following:
http://<LB IP Address>/bugtracker to access the Mantis Bug Tracker site
http://<LB IP Address>/moodle1 to access one of the Moodle sites
http://<LB IP Address>/limesurvey to access the limesurvey,
etc. I seem to be able to get to the main page of the site (e.g. index.html, index.php, etc.), but any references from there on do not work - i.e. they give path not found errors or 404 errors.
Here's a sample of my Ingress file:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: my-ingress
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /$2
spec:
rules:
- http:
paths:
- path: /limesurvey(/|$)(.*)
pathType: Prefix
backend:
service:
name: limesurvey-svc
port:
number: 80
- path: /moodle(/|$)(.*)
pathType: Prefix
backend:
service:
name: moodle-svc
port:
number: 8080
This ingress does not work (I get the 404s). However, if I only have one path in the file and just use '/' it works (but I can only use it for one service):
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: my-ingress
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /
spec:
rules:
- http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: limesurvey-svc
port:
number: 80
I think what I need is for the path (limesurvey) to continue to be pre-pended onto each URL request, but I cannot seem to make this happen. The reason I think this is because when I navigate to http://<LB IP Address>/limesurvey and click on a survey, I get a 404 error at http://<LB IP Address>/index.php/<survey number>. However, if I manually change the URL in my browser to http://<LB IP Address>/limesurvey/index.php/<survey number> it will access the survey (but still have 404s with supporting assets).
Am I trying to do something outside of what the Ingress controller was designed for? I feel like I should be able to use the rewrite-target to accomplish this but I'm missing something critical.
I appreciate any help.
Some applications relay on static content served from different URL webserver locations and moreover do internal path routing (e.g. you hit "/" path but get served "/admin" section immediately).
In such cases creation of right Ingress rules gets more tricky, and requires you to better understand behavior and constructs of your web application, to predict all possible URL path locations that user may visit (these forced by app internal redirects too), and these sourced by html code as well.
Seems like your case with limesurvey app falls into that category:
Why do I think that?
Just try to open limesurvey public demo (https://demo.limesurvey.org/) and inspect site content.
You will learn that main page is using a lot of static files (e.g. css, javascrpt files), referenced from absolute path starting with: /tmp/assets/...
<link rel="stylesheet" type="text/css" href="/tmp/assets/2d523ae6/survey.css" />
of course variants of different path locations can be matched with single smart reg-ex pattern, to avoid creation of dozen of individual ingress rules (what you tried).
What's the issue?
rules:
- http:
paths:
- path: /limesurvey(/|$)(.*) <---- it won't match "/tmp/assets/..." location
pathType: Prefix
backend:
service:
name: limesurvey-svc
port:
number: 80
Please try to create additional Ingress rule to support static file location (watch out, I'm using old syntax of Ingress resource, adjust it to your needs):
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/rewrite-target: /$1
name: ingress-limesurvey-static
spec:
rules:
- http:
paths:
- backend:
serviceName: limesurvey-svc
servicePort: 80
path: /?(.*)
Best Solution (in my opinion)
You should define custom public URL within your application directly. Detailed information can be found in Advanced path settings, see publicurl option.
This way you wouldn't need to define internal reference for static files, however it should be done during installation.
Is there an order to Ingress rule specification i.e. will the first qualifying rule be honored?
The intention of following spec is to route all requests that do not have headers Host: foo.com and Host: bar.com and route them to service3. I am not sure if the spec is syntactically correct and more so, if it will serve the desired purpose?
spec:
rules:
- host: foo.com
http:
paths:
- backend:
serviceName: service1
servicePort: http
path: /
- host: bar.com
http:
paths:
- backend:
serviceName: service2
servicePort: http
path: /a/b/c
- http:
paths:
- path: /a/b/c
backend:
serviceName: service3
servicePort: http
Don't think it matters, but I am using the Contour Ingress controller.
I am not familar with Contour, I just quickly browse the doc.
How Contour works with Ingress rule is not clear from its document. I think Contour perfer using its CRD IngressRoute to specify how request rule works.
So I infer your Ingress behavior from offical kubernetes ingress rule:
Request with host: foo.com will route to service1 or service3
Request with host: bar.com will route to service2 or service3
Other request will route to service3
What you have created in syntactically correct and should route http://*/a/b/c and http://*/a/b/c/* to service3 in most ingress controllers.
An Ingress definition is just data that is supplied to an ingress controller though. The implementation of converting that data into config is ingress controller specific.
Code
Contours route config looks to be rooted by a "virtualhost" name. In the route.go code I can't see any handling of the "no virtualhost" case.
From the route.go tests it looks like a virtualhost of * is how the default host is handled.
This sorting of virtualhosts would hopefully always put * in the right place for contour to default as you describe but then I think there is also an interface for this config to be applied to envoy, the actual proxy process.
So it would appear (without testing) that no matter what order you put the Ingress definition in, contour will sort out the default host routes for you as '*'
This kind of makes sense when you consider contour also supports a custom resource definition of IngressRoute which only allows one virtualhost per definition. These CRDs as a group have no specific ordering so the sort is required.
I am using GKE Identity-aware proxy > L7 load balancer > Custom host and path rules. It works fine for the root-path. But it does not work for custom paths.
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: basic-ingress
annotations:
kubernetes.io/ingress.global-static-ip-name: "my-static-ip"
spec:
rules:
- host: my.custom.org
http:
paths:
- path: /v1/*
backend:
serviceName: webv1
servicePort: 8080
- path: /v3/*
backend:
serviceName: webv3
servicePort: 8080
- path: /nginx/*
backend:
serviceName: nginx
servicePort: 80
- path: /*
backend:
serviceName: nginx
servicePort: 80
Path / and /nginx/ works fine. /v2/ and /v3/ throws There was a problem with your request. Error code 11.
If I disable the IAP, everything works fine.
Environment:
Kubernetes version (use kubectl version): 1.12.7-gke.17
Cloud provider or hardware configuration: GKE
Setting up HTTP Load Balancing with Ingress # https://cloud.google.com/kubernetes-engine/docs/tutorials/http-balancer
Enabling Cloud IAP for GKE # https://cloud.google.com/iap/docs/enabling-kubernetes-howto
This error code 11 is related to a misconfiguration of the OAuth client ID1.
I see that you the failing one are using a different serviceport. If you are serving the following 2 URLs from two different backends both may have IAP enabled but configured with different clientIds.
As of GCP does not support different clientIds for different paths, and the effort for IAP to use a single clientId across b/e services is underway. For now, if you can verify the clientId, and change it to a single one if they are different, that would work.
The recommended approach would be setting up an NGINX proxy. The ingress + IAP only have one service to deal with instead of two, and it will work like a charm.