Issue with Kubernetes ingress routing to Nextjs applications - kubernetes

So I have an interesting use case. I am running multiple micro-services on my Kubernetes cluster. My applications use NextJS which make internal calls to _next routes.
My issue came from the fact that I needed a way to differentiate between services and their requests to the _next files. So I implemented NextJS's assetPrefix feature which works perfectly in development, appending my prefix in front of _next so the requests look like .../${PREFIX}/_next/.... That way I could set up an ingress and route files base on the prefix to the appropriate service on my cluster. I set up a Kubernetes Ingress controller following this guide: https://akomljen.com/kubernetes-nginx-ingress-controller/
My ingress config is:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: dev-ingress
spec:
rules:
- host: baseurl.com
http:
paths:
- path: /auth
backend:
serviceName: auth-svc
servicePort: 80
- path: /static/auth
backend:
serviceName: auth-svc
servicePort: 80
- path: /login
backend:
serviceName: auth-svc
servicePort: 80
- path: /settings
backend:
serviceName: auth-svc
servicePort: 80
- path: /artwork
backend:
serviceName: artwork-svc
servicePort: 80
- path: /static/artwork
backend:
serviceName: artwork-svc
servicePort: 80
So here is the problem. Now that everything is set up, properly deployed, and the ingress is running following the above guide and using the above rules, my services are trying to make requests to .../_next/... instead of .../${PREFIX}/_next/... so they can't find the proper files and nothing is working. I cannot seem to figure out what is going on. Anyone have any ideas? Thanks in advance!

You are using built-in NGINX Ingress Controller that, unfortunately, has no such functionality.
My advice is to use NGINX Plus Ingress Controller annotation functionality if you can afford it.
You can find official example here.
Example:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: cafe-ingress
annotations:
nginx.org/rewrites: "serviceName=tea-svc rewrite=/;serviceName=coffee-svc rewrite=/beans/"
spec:
rules:
- host: cafe.example.com
http:
paths:
- path: /tea/
backend:
serviceName: tea-svc
servicePort: 80
- path: /coffee/
backend:
serviceName: coffee-svc
servicePort: 80
Below are the examples of how the URI of requests to the tea-svc are rewritten (Note that the /tea requests are redirected to /tea/).
/tea/ -> /
/tea/abc -> /abc
Below are the examples of how the URI of requests to the coffee-svc are rewritten (Note that the /coffee requests are redirected to /coffee/).
/coffee/ -> /beans/
/coffee/abc -> /beans/abc

Related

In Kubernetes ingress, when use-regex is true, what's the correct way to define a route for /

I need to define the following routes on my Kubernetes ingress :
my-website.com/* => web client
my-website.com/api/* => graphql api
I've tried to set the following but only the /api route works:
kind: Ingress
metadata:
name: "my-ingress"
annotations:
"kubernetes.io/ingress.class": nginx
"nginx.ingress.kubernetes.io/affinity": cookie
"nginx.ingress.kubernetes.io/rewrite-target": "/$2"
"nginx.ingress.kubernetes.io/use-regex": "true"
spec:
rules:
- host: my-website.com
http:
paths:
- path: "/api(/|$)(.*)"
backend:
serviceName: web-client
servicePort: 80
- path: "/"
backend:
serviceName: graphql-server
servicePort: 80
What's the problem with the / route?
I've tried to use / and /(.*) paths but none redirects to the correct service.
Could you after replace "nginx.ingress.kubernetes.io/rewrite-target": "/$2" to "nginx.ingress.kubernetes.io/rewrite-target: /"
Also change path from /api(/|$)(.*) to /api/(.*)
I am assuming web-client and graphql-server services are running fine.
in case you still face the issue, please share your ingress controller logs for more details

same path with different service names in ingress resources

I'm having some issues when using a path to point to different service names, my ingress resource looks as below
nginx-static service is a nginx container which has static content... I have to load this static content while calling service-1, since both nginx-static and service-1 but I cannot keep the sme same host path.... Please suggest how to correct the below ingress resources...
kindly note static content has lot of files(csv,js,html,directories, files etc)
kind: Ingress
metadata:
name: myingress
annotations:
kubernetes.io/ingress.class: "nginx"
nginx.ingress.kubernetes.io/ssl-redirect: "false"
spec:
rules:
- http:
paths:
- path: "/"
backend:
serviceName: nginx-static
servicePort: 80
- path: "/"
backend:
serviceName: service1
servicePort: 8989
- path: "/test1"
backend:
serviceName: service2
servicePort: 9001
Any expert help is appreciated!!!
You cannot have the same path pointing to different backend resources. You have to put either your static files or the service into a different path, and rewrite the URL, for instance:
rewrite annotation:
metadata:
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /$2
name: rewrite
namespace: default
paths:
- backend:
serviceName: nginx-statix
servicePort: 80
path: /static(/|$)(.*)
- backend:
path: /
serviceName: service1
With this, your static content will be exposed under /static/, and all /static/name will be rewritten as /name by the ingress.
More info here: https://kubernetes.github.io/ingress-nginx/examples/rewrite/
Unfortunately, requirements from the initial questions aren't' clear and there were no additional clarifications given. However, I'd like to elaborate on Burak Serdar's answer and add , that Kubernetes Ingress allows you listing the same path ad port for a multiple services under condition that you are listing these for different hosts.
foo.bar.com --| |-> foo.bar.com nginx-static:80
| 178.91.123.132 |
bar.foo.com --| |-> bar.foo.com service1:8989
you can achieve that scenario with the following config:
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: name-virtual-host-ingress
spec:
rules:
- host: foo.bar.com
http:
paths:
- backend:
serviceName: nginx-static
servicePort: 80
- host: bar.foo.com
http:
paths:
- backend:
serviceName: service2
servicePort: 8989
However that would work only if you can split your web site between two hosts.
Hope that helps.
You can check more on Ingress on K8s documentation.

Kubenetes Ingress API Routing

I have a react web application listening on the default path and I'm looking to include my API backend on the same URL.
A snippet of my ingress is below:
http:
paths:
- backend:
serviceName: atsweb
servicePort: 80
path: /(.*)
- backend:
serviceName: atsapi
servicePort: 80
path: /api(/|$)(.*)
My API has a bunch of functions which are routed after /api/ and I have a test page at mydomain.io/api/values which I cannot get to. My frontend service works fine.
Is it just the pathing which is incorrect?
I've deployed a standalone API just to check the container port/service ports are correct.
Looks like you copied the example from . What are your ingress annotations? Check the rewrite as it looks like is making a redirect. Nonetheless, the ingress that would work for looks like this:
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: your-ingress
namespace: default
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/ssl-redirect: "false"
spec:
rules:
- http:
paths:
- path: /
backend:
serviceName: atsweb
servicePort: 80
- path: /api/
backend:
serviceName: atsapi
servicePort: 80
Check there is no rewrite annotation. This makes your uri be appended fully to the next proxy. Thus, making mydomain.io/api/values go to atsapi:80/api/values

Specific requests being sent to general path on Nginx Ingress

I have a Flask Python APP that has many routes. This app (container) is served in many different deployments, one for each plan type. Example: we have one deployment called esg-enterprise to process the enterprise plan, another esg-professional for professional and so on. Finally, we have another pod just to serve our frontend application with authentication and the models it needs. All this is the same container.
As you can see in the Ingress file bellow, we have the backend rules to route the traffic to specific services. The problem is that, most of the specific requests like /task/connections/update/advanced/ or /task/connections/update/advanced/ are being sent to the root path / that should only serve the front-end (which works since they use the same container). The problem is that those specific requests are very massive causing the front end API to become unavailable. The specific services run on stronger nodes so they can handle this load, while the front-end api runs on weaker nodes. When I run kubectl get hpa I can see that most of the load (replicas) stays on the API. I even had to increase the max replicas to try to mitigate the issue. I've seen in the logs that some requests are being sent to the specific routes as they should, but the majority is not being sent.
All I wanted to do is to prevent / route to receive those specific requests.
Our ingress looks like this:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/affinity: cookie
nginx.ingress.kubernetes.io/proxy-body-size: 150m
nginx.ingress.kubernetes.io/proxy-connect-timeout: "1200"
nginx.ingress.kubernetes.io/proxy-read-timeout: "1200"
nginx.ingress.kubernetes.io/proxy-send-timeout: "1200"
nginx.ingress.kubernetes.io/upstream-fail-timeout: "1200"
name: ingress-nginx
namespace: default
spec:
tls:
- hosts:
- app.ourdomain.com
- api.ourdomain.com
secretName: ourdomain-certificate
rules:
- host: app.ourdomain.com
http:
paths:
- backend:
serviceName: frontend
servicePort: 80
path: /
- host: api.ourdomain.com
http:
paths:
- backend:
serviceName: api
servicePort: 8000
path: /
- backend:
serviceName: esg
servicePort: 8000
path: /cron/
- backend:
serviceName: esg-bigquery-migration
servicePort: 8000
path: /cron/big-query/
- backend:
serviceName: esg
servicePort: 8000
path: /task/
- backend:
serviceName: esg-trial
servicePort: 8000
path: /task/connections/update/trial/
- backend:
serviceName: esg-advanced
servicePort: 8000
path: /task/connections/update/advanced/
- backend:
serviceName: esg-professional
servicePort: 8000
path: /task/connections/update/professional/
- backend:
serviceName: esg-enterprise
servicePort: 8000
path: /task/connections/update/enterprise/
From documentation:
Starting in Version 0.22.0, ingress definitions using the annotation
nginx.ingress.kubernetes.io/rewrite-target are not backwards
compatible with previous versions. In Version 0.22.0 and beyond, any
substrings within the request URI that need to be passed to the
rewritten path must explicitly be defined in a capture group.
In some scenarios the exposed URL in the backend service differs from the specified path in the Ingress rule. Without a rewrite any request will return 404. To circumvent this you can set the annotation ingress.kubernetes.io/rewrite-target to the path expected by the service. Please, refer to documentation to learn more about this. Also, you should add following line to your annotations:
nginx.ingress.kubernetes.io/rewrite-target: /$1

Any solution for multi-tenant setup with different DNS?

I have set up my frontend cluster in my Kubernetes and exposed as frontend.loaner.com and I want to point the DNS record of these both johndoe.loaner.com, janedoe.loaner.com to see the frontend.loaner.com.
Is that possible to just point two DNS to a 1 running server and works fine still having the hostname?
I read about the CNAME but it will redirect me to frontend.loaner.com
You can do it with a Kubernetes Ingress. Basically, something like this:
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: test-ingress
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /
spec:
rules:
- host: frontend.loaner.com
http:
paths:
- path: /
backend:
serviceName: backend1
servicePort: 80
- host: johndoe.loaner.com
http:
paths:
- path: /
backend:
serviceName: backend2
servicePort: 80
- host: janedoe.loaner.com
http:
paths:
- path: /
backend:
serviceName: backend3
servicePort: 80
The above Ingress resource assumes you are using an Nginx Ingress Controller in your cluster.