I have this nginx ingress controller:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: my-ingress
annotations:
nginx.ingress.kubernetes.io/use-regex: "true"
kubernetes.io/ingress.allow-http: "true"
nginx.ingress.kubernetes.io/enable-cors: "true"
nginx.ingress.kubernetes.io/ssl-redirect: "false"
nginx.ingress.kubernetes.io/force-ssl-redirect: "false"
# Limit uploads to 8TB
nginx.ingress.kubernetes.io/proxy-body-size: 800000m
spec:
rules:
- host: myhost.info
http:
paths:
# NOTE: this one should come after all other routes. To avoid hijacking requests.
- path: /api/walrus(/|$)(.*)
backend:
serviceName: service-a
servicePort: 8080
- path: /api(/|$)(.*)
backend:
serviceName: service-b
servicePort: 8080
- path: /(.*)
backend:
serviceName: frontend
servicePort: 8080
- http:
paths:
# NOTE: this one should come after all other routes. To avoid hijacking requests.
- path: /api/walrus(/|$)(.*)
backend:
serviceName: service-a
servicePort: 8080
- path: /api(/|$)(.*)
backend:
serviceName: service-b
servicePort: 8080
- path: /(.*)
backend:
serviceName: frontend
servicePort: 8080
I duplicated the paths just make it clear. My question is, what is the difference in the end result when I add the host key vs when I don't?
Until now I've used it because if I don't have it I'm getting my POST request redirected into get request as in this question: Kubernetes NGINX Ingress changes HTTP request from a POST to a GET
But I also noticed that on EKS, if I DO add a host, the ingress just returns 404 for everything until I remove it and leavit only with http. So I'm a bit confused on this and wanted someone to clarify the correct way to do things here.
Also, for a production enviroment, how do I set the host correctly to a public domain and how do I set the tls certificates?
Regarding HTTPS: https://aws.amazon.com/blogs/opensource/network-load-balancer-nginx-ingress-controller-eks/#bGA9CAkdlMh has a section "Defining the Ingress resource (with SSL termination) to route traffic to the services created above" that shows how to terminate TLS at nginx-ingress. Even if you're not using an AWS Network Load Balancer (NLB), that may be helpful. In the case of AWS with an NLB, you have another option, terminating at the NLB: https://aws.amazon.com/blogs/aws/new-tls-termination-for-network-load-balancers/
There are two nginx ingress controllers, and it's unclear which one you're using. The nginxinc controller requires a 'host'. The other, https://github.com/kubernetes/ingress-nginx, I'm not sure about. When you use TLS, nginx uses SNI for HTTPS, which seems like it would require a 'host': http://nginx.org/en/docs/http/configuring_https_servers.html#sni
Related
I'm trying to configure NGINX Ingress controller as the correct entry point to my Kubernetes cluster. Inside the cluster, I've created two REST Web services as well as frontend application. I'm trying to achieve the following scenario.
When the ingress IP is hit without any parameters it should be routed
to the frontend app. Example: 192.168.1.20 should lead to frontend
service on port 80.
When parameters are given, the request should be
routed to correct REST service. Example:
192.168.1.20/first-rest/api/flower?id=1 should route the request to
the first-rest service so that it could return the flower with id =
1.
I can correctly access the frontend application but when trying to access any REST service I'm getting 404 error or no response at all. First-rest, Second-rest and frontend are running correctly and are accessible when configured as LoadBalancer services. With Nginx, they are configured as ClusterIp services.
My ingress configuration
------------------------
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: main-routes
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/ssl-redirect: "false"
nginx.ingress.kubernetes.io/use-regex: "true"
nginx.ingress.kubernetes.io/rewrite-target: /$1
spec:
rules:
- http:
paths:
- path: /first-rest(/|$)(.*)
backend:
serviceName: first-rest
servicePort: 8090
- path: /second-rest(/|$)(.*)
backend:
serviceName: second-rest
servicePort: 9000
- path: /(.*)
backend:
serviceName: frontend
servicePort: 80
It seems like NGINX is cutting short my URL parameters that are required for my REST API. Is there any way to pass the right URL path so that `192.168.1.20/first-rest/api/flower?id=1` would be routed to `[first-rest add and port]/api/flower?id=1` ?
You need to specify the ingress path type otherwise , depending on the ingress class specific implementation it will default to either exact or prefix (I assume in your case it is defaulting to exact)
So, you need to do something like
spec:
rules:
- http:
paths:
- path: /first-rest
pathType: Prefix
backend:
serviceName: first-rest
servicePort: 8090
see docs on ingress path here
I need to define the following routes on my Kubernetes ingress :
my-website.com/* => web client
my-website.com/api/* => graphql api
I've tried to set the following but only the /api route works:
kind: Ingress
metadata:
name: "my-ingress"
annotations:
"kubernetes.io/ingress.class": nginx
"nginx.ingress.kubernetes.io/affinity": cookie
"nginx.ingress.kubernetes.io/rewrite-target": "/$2"
"nginx.ingress.kubernetes.io/use-regex": "true"
spec:
rules:
- host: my-website.com
http:
paths:
- path: "/api(/|$)(.*)"
backend:
serviceName: web-client
servicePort: 80
- path: "/"
backend:
serviceName: graphql-server
servicePort: 80
What's the problem with the / route?
I've tried to use / and /(.*) paths but none redirects to the correct service.
Could you after replace "nginx.ingress.kubernetes.io/rewrite-target": "/$2" to "nginx.ingress.kubernetes.io/rewrite-target: /"
Also change path from /api(/|$)(.*) to /api/(.*)
I am assuming web-client and graphql-server services are running fine.
in case you still face the issue, please share your ingress controller logs for more details
I have a Flask Python APP that has many routes. This app (container) is served in many different deployments, one for each plan type. Example: we have one deployment called esg-enterprise to process the enterprise plan, another esg-professional for professional and so on. Finally, we have another pod just to serve our frontend application with authentication and the models it needs. All this is the same container.
As you can see in the Ingress file bellow, we have the backend rules to route the traffic to specific services. The problem is that, most of the specific requests like /task/connections/update/advanced/ or /task/connections/update/advanced/ are being sent to the root path / that should only serve the front-end (which works since they use the same container). The problem is that those specific requests are very massive causing the front end API to become unavailable. The specific services run on stronger nodes so they can handle this load, while the front-end api runs on weaker nodes. When I run kubectl get hpa I can see that most of the load (replicas) stays on the API. I even had to increase the max replicas to try to mitigate the issue. I've seen in the logs that some requests are being sent to the specific routes as they should, but the majority is not being sent.
All I wanted to do is to prevent / route to receive those specific requests.
Our ingress looks like this:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/affinity: cookie
nginx.ingress.kubernetes.io/proxy-body-size: 150m
nginx.ingress.kubernetes.io/proxy-connect-timeout: "1200"
nginx.ingress.kubernetes.io/proxy-read-timeout: "1200"
nginx.ingress.kubernetes.io/proxy-send-timeout: "1200"
nginx.ingress.kubernetes.io/upstream-fail-timeout: "1200"
name: ingress-nginx
namespace: default
spec:
tls:
- hosts:
- app.ourdomain.com
- api.ourdomain.com
secretName: ourdomain-certificate
rules:
- host: app.ourdomain.com
http:
paths:
- backend:
serviceName: frontend
servicePort: 80
path: /
- host: api.ourdomain.com
http:
paths:
- backend:
serviceName: api
servicePort: 8000
path: /
- backend:
serviceName: esg
servicePort: 8000
path: /cron/
- backend:
serviceName: esg-bigquery-migration
servicePort: 8000
path: /cron/big-query/
- backend:
serviceName: esg
servicePort: 8000
path: /task/
- backend:
serviceName: esg-trial
servicePort: 8000
path: /task/connections/update/trial/
- backend:
serviceName: esg-advanced
servicePort: 8000
path: /task/connections/update/advanced/
- backend:
serviceName: esg-professional
servicePort: 8000
path: /task/connections/update/professional/
- backend:
serviceName: esg-enterprise
servicePort: 8000
path: /task/connections/update/enterprise/
From documentation:
Starting in Version 0.22.0, ingress definitions using the annotation
nginx.ingress.kubernetes.io/rewrite-target are not backwards
compatible with previous versions. In Version 0.22.0 and beyond, any
substrings within the request URI that need to be passed to the
rewritten path must explicitly be defined in a capture group.
In some scenarios the exposed URL in the backend service differs from the specified path in the Ingress rule. Without a rewrite any request will return 404. To circumvent this you can set the annotation ingress.kubernetes.io/rewrite-target to the path expected by the service. Please, refer to documentation to learn more about this. Also, you should add following line to your annotations:
nginx.ingress.kubernetes.io/rewrite-target: /$1
I have a backend using https.
I want to separate load on that back-end based on URL/path.
I decided to use ingress to do this url/path based logic in order to move traffic to different back-ends ( same back-ends , just duplicated to different NodePorts )
my question is how I can configure the ingress to receive https requests and to forward those https requests to the https back-end?
thanks
edit:
I added the yaml file:
spec:
rules:
- http:
paths:
- backend:
serviceName: service
servicePort: 9443
path: /carbon
- backend:
serviceName: service2
servicePort: 9443
path: /oauth
for some reason I can;t change the rule form http to https
Attention: This answer applies to the ingress-nginx solution provided by the kubernetes organisation on github (https://github.com/kubernetes/ingress-nginx)
If you want to use load balancing mechanisms in k8s you should use services instead and start multiple instances behind that service that way k8s will do the load balancing. If you want to use different versions of your backend (e.g. prod and test) your way of separating them is fine
if your service is only reachable via https you need to add the following annotation to your ingress yaml: (documentation)
nginx.ingress.kubernetes.io/backend-protocol: "HTTPS"
To secure ingress itself take a look at this: https://kubernetes.io/docs/concepts/services-networking/ingress/#tls
But if you want that the backend services decrypt the TLS communication use the following annotation instead: (documentation)
nginx.ingress.kubernetes.io/ssl-passthrough: "true"
Edit:
The Ingress YAML should look like this if you want to reach the backend via TLS:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: ingress-name
namespace: namespace-name
annotations:
nginx.ingress.kubernetes.io/backend-protocol: "HTTPS"
spec:
rules:
- http:
paths:
- backend:
serviceName: service
servicePort: 9443
path: /carbon
- backend:
serviceName: service2
servicePort: 9443
path: /oauth
The Ingress YAML should look like this if you want to reach the backend via TLS with TLS decryption in the ingress controller:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: ingress-name
namespace: namespace-name
annotations:
nginx.ingress.kubernetes.io/backend-protocol: "HTTPS"
spec:
tls:
- hosts:
- app.myorg.com
secretName: tls-secret
rules:
- http:
paths:
- backend:
serviceName: service
servicePort: 9443
path: /carbon
- backend:
serviceName: service2
servicePort: 9443
path: /oauth
It's important to note that tls-secret is the name of a SecretConfig with a valid Certificate issued for the host (app.myorg.com)
The Ingress YAML should look like this if you want to reach the backend via TLS with TLS decryption in the backend:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: ingress-name
namespace: namespace-name
annotations:
nginx.ingress.kubernetes.io/ssl-passthrough: "true"
spec:
rules:
- http:
paths:
- backend:
serviceName: service
servicePort: 9443
path: /carbon
- backend:
serviceName: service2
servicePort: 9443
path: /oauth
I never tested the last version myself so i don't know if that actually works but I'd strongly advise reading this passage for that variant.
If you are using the NGINX Ingress controller (https://docs.nginx.com/nginx-ingress-controller/), the nginx.ingress.kubernetes.io/backend-protocol: "HTTPS" does not work. However, the nginx.org/ssl-services will let you pick the services that require TLS on the backend. The name is confusing, so it took me a while to realize the real purpose of it.
This does not work with the standard Kubernetes Ingress controller that uses NGINX under the hood; it only works with the NGINX-sourced controller.
Advanced annotation docs: https://docs.nginx.com/nginx-ingress-controller/configuration/ingress-resources/advanced-configuration-with-annotations/
In this example, NGINX will connect to the ssl-svc using TLS; it ignores any self-signed certificates. Example (https://github.com/nginxinc/kubernetes-ingress/tree/v1.12.0/examples/ssl-services):
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: cafe-ingress
annotations:
nginx.org/ssl-services: "ssl-svc"
spec:
rules:
- host: cafe.example.com
http:
paths:
- path: /tea
backend:
serviceName: tea-svc
servicePort: 80
- path: /coffee
backend:
serviceName: coffee-svc
servicePort: 80
- path: /ssl
backend:
serviceName: ssl-svc
servicePort: 443
I have setup Custom Nginx ingress controller with Ingress resource in Kubernetes and instead of "default-http-backend service", I used custom application as the default backend service to be served for default requests. I have also used custom SSL which is set as kubernetes secret, for my service. The issue is that when I request the hostnames which are mentioned in the rules, the https redirection works. But when the requests other than the hosts mentioned in the rules are made, it serves the default app, but the https redirection does not work.
How can I redirect requests from http to https for all the requests including default requests. In other words, how to setup https redirection for wildcard domains in ingress resource.
Please find my yaml files for ingress resource.
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: ingress-resource
namespace: default
annotations:
kubernetes.io/ingress.class: "nginx"
kubernetes.io/ingress.allow-http: "false"
ingress.kubernetes.io/rewrite-target: /
ingress.kubernetes.io/ssl-redirect: "true"
ingress.kubernetes.io/proxy-connect-timeout: "14400"
ingress.kubernetes.io/proxy-send-timeout: "14400"
ingress.kubernetes.io/proxy-read-timeout: "14400"
spec:
tls:
- secretName: tls-secret
rules:
- host: service1.example.com
http:
paths:
- path: /
backend:
serviceName: service1
servicePort: 80
- host: service2.example.com
http:
paths:
- path: /
backend:
serviceName: service2
servicePort: 80
---
I needed to configure custom service (not default-http-backend service) for default requests which does not have rules set and this custom service should use custom SSL. At present nginx-ingress-controller doesn't do anything if the domain names are omitted from the Ingress rules (with the intention of the "wildcard" TLS cert being used).
Therefore I have added the following code in the ingress yaml I used and this works perfectly. I have added the wildcard tls name in ingress rules at the bottom for the custom default service. Please find the code below:
rules:
- host: service1.example.com
http:
paths:
- path: /
backend:
serviceName: service1
servicePort: 80
- host: service2.example.com
http:
paths:
- path: /
backend:
serviceName: service2
servicePort: 80
- host: '*.example.com'
http:
paths:
- path: /
backend:
serviceName: custom-backend-service
servicePort: 80