In a Kubernetes ingress, I'd like to dynamically change the service name to a match on a certain part of the path in an ingress. Something like this:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: example
spec:
rules:
- host: example.com
http:
paths:
- path: /api/(^/+)/(.*)
backend:
serviceName: api-(&1)
servicePort: 80
Where &1 in the example above would resolve to the first regular expression match in the string /api/(^/+)/(.*)
So basically if a path is something like this /api/asset/fetch/anything-at-all, then it would direct the call to a service named api-asset
I know that this can potentially lead to some security issues, but if I force the prefix 'api-' to the service name then only services with the prefix 'api-' would technically be exposed to the internet. And all other services would still be protected inside the Kubernetes cluster. I can even change the prefix to be 'public-' for simplicity's sake.
Is this something that is possible? And what is the correct syntax to set it up?
Related
On AWS EKS, I have three pods in a cluster each of which have been exposed by services. The problem is the services can not communicate with each other as discussed here Error while doing inter pod communication on EKS. It has not been answered yet but further search said that it can be done through Ingress. I am having confusion as to how to do it? Can anybody help ?
Code:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
namespace: test
name: ingress-test
annotations:
alb.ingress.kubernetes.io/scheme: internet-facing
alb.ingress.kubernetes.io/target-type: ip
spec:
ingressClassName: alb
rules:
- http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: server-service
port:
number: 8000
My server-service has APIs like /api/v1/getAll, /api/v1/updateAll, etc.
So, what should I write in path and for a database service what should I do??
And say in future I make another microservice and open another service which has APIs like /api/v1/showImage, /api/v1/deleteImage will I have to write all paths in ingress or is their another way for it to work?
A Ingress is a really good solution to expose both a frontend and a backend at the same domain with different paths (but reading your other question, it will be of no help in exposing the database)
With this said, you don't have to write all the paths in the Ingress (unless you want to) as you can instead use pathType: Prefix as it is already in your example.
Let me link you to the documentation Examples which explain how it works really well. Basically, you can add a rule with:
path: /api
pathType: Prefix
In order to expose your backend under /api and all the child paths.
The case where you put a second backend, which wants to be exposed under /api as the first one, is way more complex instead. If two Pods wants to be exposed at the same paths, you will probably need to list all the subpaths in a way that differentiate them.
For example:
Backed A
/api/v1/foo/listAll
/api/v1/foo/save
/api/v1/foo/delete
Backend B
/api/v1/bar/listAll
/api/v1/bar/save
/api/v1/bar/delete
Then you could expose one under subPath /api/v1/foo (Prefix) and the other under /api/v1/bar (Prefix).
As another alternative, you may want to expose the backends at different paths from what they actually expect using a rewrite target rule.
I'm trying to setup a ingress network for my Google GKE, I have tested locally on Minikube and its working as I expect.
When I hit the domain with the prefix /test-1 or /test-2 its sending me to the root of the my service /.
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: api-test-domain-ingress
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /$2
spec:
tls:
- hosts:
- api.test-domain.com
secretName: tls-secret
rules:
- host: api.test-domain.com
http:
paths:
- path: /test-1(/|$)(.*)
pathType: Prefix
backend:
service:
name: test-1-port-forwarding
port:
number: 8080
- path: /test-2(/|$)(.*)
pathType: Prefix
backend:
service:
name: test-2-port-forwarding
port:
number: 8081
The issue is when I put it into my Kubernetes cluster on Google (GKE) then I get this error
Translation failed: invalid ingress spec: failed to validate prefix
path /test-1(/|$)(.) due to invalid wildcard; failed to validate
prefix path /test-2(/|$)(.) due to invalid wildcard
I have trying in hours to trying to get it to working and what's going on, whiteout any kind of result, so really hope some one here can explain about what I did wrong and what I shut change to resolved my problem.
GKE Built-in Ingress supports wildcard but with some conditions. From the doc:
The only supported wildcard character for the path field of an Ingress is the * character. The * character must follow a forward slash (/) and must be the last character in the pattern. For example, /*, /foo/*, and /foo/bar/* are valid patterns, but *, /foo/bar*, and /foo/*/bar are not.
If you want to use NGINX you will have to deploy it, GKE doesn't ship with NGINX out of the box. Keep in mind that this is something you will have to maintain and take care of yourself. It's a valid choice to make if the GKE default ingress doesn't support what you need to do (like headers re-write for example) but just be aware of the fact that it's an extra piece of software.
I am fairly new to Kubernetes and have just deployed my first cluster to IBM Cloud. When I created the cluster, I get a dedicated ingress subdomain, which I will be referring to as <long-k8subdomain>.cloud for the scope of this post. Now, this subdomain works for my app. For example: <long-k8subdomain>.cloud/ping works from my browser/curl just fine- I get the expected JSON response back. But, if I add this subdomain to a CNAME record on my domain provider's DNS settings (I have used Bluehost and IBM Cloud's Internet Services), I get a 404 response back from all routes. However this response is the default nginx 404 response (it says "nginx" under "404 Not Found"). I believe this means that this means the ingress load balancer is being reached, but the request does not get routed right. I am using Kubernetes version 1.20.12_1561 on VPC gen 2 and this is my ingress-config.yaml file:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: my-ingress-resource
annotations:
kubernetes.io/ingress.class: "public-iks-k8s-nginx"
nginx.ingress.kubernetes.io/configuration-snippet: |
more_set_headers "Host: <long-k8subdomain>.cloud";
spec:
rules:
- host: <long-k8subdomain>.cloud
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: my-service-name
port:
number: 80
I am pretty sure this problem is due to the annotations. Maybe I am using the wrong ones or I do not have enough. Ideally, I would like something like this: api..com/ to route correctly. I have also read a little bit about default backends, but I have not dove too much into that just yet. Any help would be greatly appreciated, as I have spent multiple hours trying to fix this.
Some sources I have used:
https://cloud.ibm.com/docs/containers?topic=containers-cs_network_planning
https://cloud.ibm.com/docs/containers?topic=containers-ingress-types
https://cloud.ibm.com/docs/containers?topic=containers-comm-ingress-annotations#annotations
Note: The reason why I have the second annotation is because for some reason, requests without that header were not being routed directly. So that was part of my debugging process and I just ended up leaving it as I am not sure if that annotation solves that, so I left it for now.
For the NGINX ingress controller to route requests for your own domain's CNAME record to the service instead of the IBM Cloud one, you need a rule in the ingress where the host identifies your domain.
For instance, if your domain's DNS entry is api.example.com, then change the resource YAML to:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: my-ingress-resource
annotations:
kubernetes.io/ingress.class: "public-iks-k8s-nginx"
spec:
rules:
- host: api.example.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: my-service-name
port:
number: 80
You should not need the second annotation for this to work.
If you want both of the hosts to work, then you could add a second rule instead of replacing host in the existing one.
First a little background:
We currently have several websites and services hosted on a Plesk server and I am setting up a bare-metal development server to provide an area where we can test updates, etc. before going to production. I am using a 3 node kubernetes cluster running microk8s on Ubunutu 20.04.01. The services we host are pretty diverse: we have a couple Moodle sites, a few Wordpress sites, a site running limesurvery, an instance of Mantis bugtracker, and a few more. I have successfully gotten most of the sites containerized and running on k8s. I can also access each individual site either through a NodePort or a MetalLB load balancer.
However, I'd really like to use the NGINX Ingress Controller on top of the load balancer so that I can have a consistent way to access the sites without using a bunch of IP addresses (or in the NodePort case, ports that change). No matter what I've done, I cannot seem to get the Ingress to do what I want. I simply want to do the following:
http://<LB IP Address>/bugtracker to access the Mantis Bug Tracker site
http://<LB IP Address>/moodle1 to access one of the Moodle sites
http://<LB IP Address>/limesurvey to access the limesurvey,
etc. I seem to be able to get to the main page of the site (e.g. index.html, index.php, etc.), but any references from there on do not work - i.e. they give path not found errors or 404 errors.
Here's a sample of my Ingress file:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: my-ingress
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /$2
spec:
rules:
- http:
paths:
- path: /limesurvey(/|$)(.*)
pathType: Prefix
backend:
service:
name: limesurvey-svc
port:
number: 80
- path: /moodle(/|$)(.*)
pathType: Prefix
backend:
service:
name: moodle-svc
port:
number: 8080
This ingress does not work (I get the 404s). However, if I only have one path in the file and just use '/' it works (but I can only use it for one service):
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: my-ingress
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /
spec:
rules:
- http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: limesurvey-svc
port:
number: 80
I think what I need is for the path (limesurvey) to continue to be pre-pended onto each URL request, but I cannot seem to make this happen. The reason I think this is because when I navigate to http://<LB IP Address>/limesurvey and click on a survey, I get a 404 error at http://<LB IP Address>/index.php/<survey number>. However, if I manually change the URL in my browser to http://<LB IP Address>/limesurvey/index.php/<survey number> it will access the survey (but still have 404s with supporting assets).
Am I trying to do something outside of what the Ingress controller was designed for? I feel like I should be able to use the rewrite-target to accomplish this but I'm missing something critical.
I appreciate any help.
Some applications relay on static content served from different URL webserver locations and moreover do internal path routing (e.g. you hit "/" path but get served "/admin" section immediately).
In such cases creation of right Ingress rules gets more tricky, and requires you to better understand behavior and constructs of your web application, to predict all possible URL path locations that user may visit (these forced by app internal redirects too), and these sourced by html code as well.
Seems like your case with limesurvey app falls into that category:
Why do I think that?
Just try to open limesurvey public demo (https://demo.limesurvey.org/) and inspect site content.
You will learn that main page is using a lot of static files (e.g. css, javascrpt files), referenced from absolute path starting with: /tmp/assets/...
<link rel="stylesheet" type="text/css" href="/tmp/assets/2d523ae6/survey.css" />
of course variants of different path locations can be matched with single smart reg-ex pattern, to avoid creation of dozen of individual ingress rules (what you tried).
What's the issue?
rules:
- http:
paths:
- path: /limesurvey(/|$)(.*) <---- it won't match "/tmp/assets/..." location
pathType: Prefix
backend:
service:
name: limesurvey-svc
port:
number: 80
Please try to create additional Ingress rule to support static file location (watch out, I'm using old syntax of Ingress resource, adjust it to your needs):
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/rewrite-target: /$1
name: ingress-limesurvey-static
spec:
rules:
- http:
paths:
- backend:
serviceName: limesurvey-svc
servicePort: 80
path: /?(.*)
Best Solution (in my opinion)
You should define custom public URL within your application directly. Detailed information can be found in Advanced path settings, see publicurl option.
This way you wouldn't need to define internal reference for static files, however it should be done during installation.
I have a Kubernetes ingress that I want to be the default for all paths on a set of hosts, provided there is not a more specific match:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: default-ing
spec:
rules:
- host: host1.sub.example.com
http:
paths:
- backend:
serviceName: my-default-service
servicePort: http
# Note: here we specify the root path intended as a default
path: /
- backend:
serviceName: my-default-service
servicePort: http
path: /route/path/to/default
A second ingress defines a custom service for a specific path:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: special-ing
spec:
rules:
- host: host1.sub.example.com
http:
paths:
- backend:
serviceName: special-service
servicePort: http
path: /special
I would expect that the order of adding/deleting the ingresses would not matter, or at least I could have some way of indicating that the path: / in default-ing is always to be ordered last.
When I try the above, the routing is fine as long as I add special-ing before default-ing (or alternatively, add default-ing, then the special-ing, then delete default-ing and re-add it again). When I add them as default-ing, then special-ing, requests to /special are routed to my-default-service instead of special-service.
I want the order of adding/deleting to be independent of the routing that is generated by nginx-ingress-controller, so that my kubectl manipulations are more robust, and if one of the ingresses is recreated nothing will break.
I'm using nginx-ingress-controller:0.19.0
Thanks for any help you can offer!
The short answer is no. I believe your configs should be disallowed by the nginx ingress controller or documented somewhere. Basically what's happening when you have 2 hosts rules with the same value: host1.sub.example.com one is overwriting the other in the server {} block in the nginx.conf that your nginx ingress controller is managing.
So if your add default-ing before special-ing then special-ing will be the actual config. When you add special-ing before default-ing then default-ing will be your only config, special-ing is not supposed to work at all.
Add special-ing, and the configs look something like this:
server {
server_name host1.sub.example.com;
...
location /special {
...
}
location / { # default backend
...
}
...
}
Now add default-ing, and the configs will change to like this:
server {
server_name host1.sub.example.com;
...
location /route/path/to/default {
...
}
location / { # default backend
...
}
...
}
If you add them the other way around the config will look like 1. in the end.
You can find more by shelling into your nginx ingress controller pod and looking at the nginx.conf file.
$ kubectl -n <namespace> exec -it nginx-ingress-controller-pod sh
# cat /etc/nginx/nginx.conf
Update 03/31/2022:
It seems like on newer nginx ingress controller versions. All the rules with the same host get merged into a server block in the nginx.conf