Can later ingress subpaths override earlier ingress parent paths? - kubernetes

I have a Kubernetes ingress that I want to be the default for all paths on a set of hosts, provided there is not a more specific match:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: default-ing
spec:
rules:
- host: host1.sub.example.com
http:
paths:
- backend:
serviceName: my-default-service
servicePort: http
# Note: here we specify the root path intended as a default
path: /
- backend:
serviceName: my-default-service
servicePort: http
path: /route/path/to/default
A second ingress defines a custom service for a specific path:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: special-ing
spec:
rules:
- host: host1.sub.example.com
http:
paths:
- backend:
serviceName: special-service
servicePort: http
path: /special
I would expect that the order of adding/deleting the ingresses would not matter, or at least I could have some way of indicating that the path: / in default-ing is always to be ordered last.
When I try the above, the routing is fine as long as I add special-ing before default-ing (or alternatively, add default-ing, then the special-ing, then delete default-ing and re-add it again). When I add them as default-ing, then special-ing, requests to /special are routed to my-default-service instead of special-service.
I want the order of adding/deleting to be independent of the routing that is generated by nginx-ingress-controller, so that my kubectl manipulations are more robust, and if one of the ingresses is recreated nothing will break.
I'm using nginx-ingress-controller:0.19.0
Thanks for any help you can offer!

The short answer is no. I believe your configs should be disallowed by the nginx ingress controller or documented somewhere. Basically what's happening when you have 2 hosts rules with the same value: host1.sub.example.com one is overwriting the other in the server {} block in the nginx.conf that your nginx ingress controller is managing.
So if your add default-ing before special-ing then special-ing will be the actual config. When you add special-ing before default-ing then default-ing will be your only config, special-ing is not supposed to work at all.
Add special-ing, and the configs look something like this:
server {
server_name host1.sub.example.com;
...
location /special {
...
}
location / { # default backend
...
}
...
}
Now add default-ing, and the configs will change to like this:
server {
server_name host1.sub.example.com;
...
location /route/path/to/default {
...
}
location / { # default backend
...
}
...
}
If you add them the other way around the config will look like 1. in the end.
You can find more by shelling into your nginx ingress controller pod and looking at the nginx.conf file.
$ kubectl -n <namespace> exec -it nginx-ingress-controller-pod sh
# cat /etc/nginx/nginx.conf
Update 03/31/2022:
It seems like on newer nginx ingress controller versions. All the rules with the same host get merged into a server block in the nginx.conf

Related

SSL Certificate added but shows "Kubernetes Ingress controller fake certificate"

I am having the following issue.
I am new to GCP/Cloud, I have created a cluster in GKE and deployed our application there, installed nginx as a POD in the cluster, our company has a authorized SSL certificate which i have uploaded in Certificates in GCP.
In the DNS Service, i have created an A record which matched the IP of Ingress.
When i call the URL in the browser, it still shows that the website is still unsecure with message "Kubernetes Ingress controller fake certificate".
I used the following guide https://cloud.google.com/load-balancing/docs/ssl-certificates/self-managed-certs#console_1
however i am not able to execute step 3 "Associate an SSL certificate with a target proxy", because it asks "URL Maps" and i am not able to find it in the GCP Console.
Has anybody gone through the same issue like me or if anybody helps me out, it would be great.
Thanks and regards,
I was able to fix this problem by adding an extra argument to the ingress-nginx-controller deployment.
For context: my TLS secret was at the default namespace and was named letsencrypt-secret-prod, so I wanted to add this as the default SSL certificate for the Nginx controller.
My first solution was to edit the deployment.yaml of the Nginx controller and add at the end of the containers[0].args list the following line:
- '--default-ssl-certificate=default/letsencrypt-secret-prod'
Which made that section of the yaml look like this:
containers:
- name: controller
image: >-
k8s.gcr.io/ingress-nginx/controller:v1.2.0-beta.0#sha256:92115f5062568ebbcd450cd2cf9bffdef8df9fc61e7d5868ba8a7c9d773e0961
args:
- /nginx-ingress-controller
- '--publish-service=$(POD_NAMESPACE)/ingress-nginx-controller'
- '--election-id=ingress-controller-leader'
- '--controller-class=k8s.io/ingress-nginx'
- '--ingress-class=nginx'
- '--configmap=$(POD_NAMESPACE)/ingress-nginx-controller'
- '--validating-webhook=:8443'
- '--validating-webhook-certificate=/usr/local/certificates/cert'
- '--validating-webhook-key=/usr/local/certificates/key'
- '--default-ssl-certificate=default/letsencrypt-secret-prod'
But I was using the helm chart: ingress-nginx/ingress-nginx, so I wanted this config to be in the values.yaml file of that chart so that I could upgrade it later if necessary.
So reading the values file I replaced the attribute: controller.extraArgs, which looked like this:
extraArgs: {}
For this:
extraArgs:
default-ssl-certificate: default/letsencrypt-secret-prod
This restarted the deployment with the argument in the correct place.
Now I can use ingresses without specifying the tls.secretName for each of them, which is awesome.
Here's an example ingress that is working for me with HTTPS:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: some-ingress-name
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/backend-protocol: "HTTP"
spec:
rules:
- http:
paths:
- path: /some-prefix
pathType: Prefix
backend:
service:
name: some-service-name
port:
number: 80
You can save your SSL/TLS certificate into the K8s secret and attach it to the ingress.
you need to config the TLS block in ingress, dont forget to add ingress.class details in ingress
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: tls-example-ingress
spec:
rules:
- host: mydomain.com
http:
paths:
-
backend:
serviceName: my-service
servicePort: 80
path: /
tls:
- hosts:
- mydomain.com
secretName: my-tls-secret
You can read more at : https://medium.com/avmconsulting-blog/how-to-secure-applications-on-kubernetes-ssl-tls-certificates-8f7f5751d788
You might be seeing something like this in browser :
that's from the ingress controller and wrong certificate attached to ingress or ingress controller default fake cert.
in my fault, I upgraded new tsl on cattle-system name space, but not in my name-space, therefore some how, ingress recognize with K8s fake cert.
Solution: upgrade all old cert to new cert (only ingress user cert, WARNING - system cert maybe damaged your system - cannot explain)
Also don't forget to check that Subject Alternative Name actually contains the same value the CN contains. If it does not, the certificate is not valid because the industry moves away from CN. Just learned this now

Kubernetes Ingress not forwarding routes

I am fairly new to Kubernetes and have just deployed my first cluster to IBM Cloud. When I created the cluster, I get a dedicated ingress subdomain, which I will be referring to as <long-k8subdomain>.cloud for the scope of this post. Now, this subdomain works for my app. For example: <long-k8subdomain>.cloud/ping works from my browser/curl just fine- I get the expected JSON response back. But, if I add this subdomain to a CNAME record on my domain provider's DNS settings (I have used Bluehost and IBM Cloud's Internet Services), I get a 404 response back from all routes. However this response is the default nginx 404 response (it says "nginx" under "404 Not Found"). I believe this means that this means the ingress load balancer is being reached, but the request does not get routed right. I am using Kubernetes version 1.20.12_1561 on VPC gen 2 and this is my ingress-config.yaml file:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: my-ingress-resource
annotations:
kubernetes.io/ingress.class: "public-iks-k8s-nginx"
nginx.ingress.kubernetes.io/configuration-snippet: |
more_set_headers "Host: <long-k8subdomain>.cloud";
spec:
rules:
- host: <long-k8subdomain>.cloud
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: my-service-name
port:
number: 80
I am pretty sure this problem is due to the annotations. Maybe I am using the wrong ones or I do not have enough. Ideally, I would like something like this: api..com/ to route correctly. I have also read a little bit about default backends, but I have not dove too much into that just yet. Any help would be greatly appreciated, as I have spent multiple hours trying to fix this.
Some sources I have used:
https://cloud.ibm.com/docs/containers?topic=containers-cs_network_planning
https://cloud.ibm.com/docs/containers?topic=containers-ingress-types
https://cloud.ibm.com/docs/containers?topic=containers-comm-ingress-annotations#annotations
Note: The reason why I have the second annotation is because for some reason, requests without that header were not being routed directly. So that was part of my debugging process and I just ended up leaving it as I am not sure if that annotation solves that, so I left it for now.
For the NGINX ingress controller to route requests for your own domain's CNAME record to the service instead of the IBM Cloud one, you need a rule in the ingress where the host identifies your domain.
For instance, if your domain's DNS entry is api.example.com, then change the resource YAML to:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: my-ingress-resource
annotations:
kubernetes.io/ingress.class: "public-iks-k8s-nginx"
spec:
rules:
- host: api.example.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: my-service-name
port:
number: 80
You should not need the second annotation for this to work.
If you want both of the hosts to work, then you could add a second rule instead of replacing host in the existing one.

Setting service name dynamically based on path in Kubernetes Ingress

In a Kubernetes ingress, I'd like to dynamically change the service name to a match on a certain part of the path in an ingress. Something like this:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: example
spec:
rules:
- host: example.com
http:
paths:
- path: /api/(^/+)/(.*)
backend:
serviceName: api-(&1)
servicePort: 80
Where &1 in the example above would resolve to the first regular expression match in the string /api/(^/+)/(.*)
So basically if a path is something like this /api/asset/fetch/anything-at-all, then it would direct the call to a service named api-asset
I know that this can potentially lead to some security issues, but if I force the prefix 'api-' to the service name then only services with the prefix 'api-' would technically be exposed to the internet. And all other services would still be protected inside the Kubernetes cluster. I can even change the prefix to be 'public-' for simplicity's sake.
Is this something that is possible? And what is the correct syntax to set it up?

Ingress controller cannot find assets

I am trying to set up an Ingress controller on Microk8s to host a react (NextJS) app.
The pod and service are up and running, as well as reachable internally via machine.domain.eu:31111
My goal is to make this service available via machine.domain.eu/dev on the default port 80 using an nginx ingress controller:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: web-ingress
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /$2
spec:
rules:
- host: machine.domain.eu
http:
paths:
- path: /dev(/|$)(.*)
pathType: Prefix
backend:
service:
name: web-svc
port:
number: 3000
If I issue a curl command via curl machine.domain.eu/dev from the machine, where the node is running, I am getting back the desired html, although I cannot see, if assets are being loaded.
When switching back to my local machine opening my browser of choice and hit that http endpoint I do get a blank screen and all static assets 404. The index.html comes through fine, yet all the other resources are not found.
I've tried any number of combinations on the annotation to rewrite to /$1 /$2, while changing the path to - path: /dev/?(.*) but to no effect.
In other words: The document itself is loaded via
http://machine.domain.eu/dev
Yet the assets, which should be loaded via
http://machine.domain.eu/dev/_next/{....}
are actually being requested via:
http://machine.domain.eu/_next/{....}
What excactly am I doing wrong here?

Kubernetes and failover service

I have next situation and I am not sure if it is possible to achieve it using kubernetes only.
There are two pods, one with live service is available on e.g. givemedata.example.com, temporary one on givemedata2.example.com. Temporary one is without any dependency and there just to allow some long term jobs to continue run while real one is down for maintenance.
Cluster is on GKE.
I need to setup failover so that if live one is down for maintenance, failover should redirect calls to temporary one.
Main reason is to use only one endpoint, e.g. givemedata.example.com.
My question is if this is possible and how to do it?
You can install NGINX ingress controller. Add
nginx.ingress.kubernetes.io/server-snippet: annotation to ingress configuration file and add block of code which will check if domain is available, if not it will redirect traffic to different one.
Thanks to this annotation it is possible to add custom configuration in the server configuration block.
Read more: server snippet.
Remember to add this server to nginx.conf file in /etc/nginx directory.
Here is example cofiguration:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: my-ingress
namespace: ingress-nginx
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/add-base-url : "true"
nginx.ingress.kubernetes.io/configuration-snippet: |
set $agentflag 0;
if ($http_user_agent ~* "(Mobile)" ){
set $agentflag 1;
}
if ( $agentflag = 1 ) {
return 301 https://givemedata2.example.com;
}
spec:
rules:
- host: givemedata.example.com
http:
paths:
- path: /
backend:
serviceName: service-name
servicePort: service-port