kubernetes nginx ingress server-snippet annotation not taking effect - kubernetes

I have following ingress.yaml
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: nginx-configuration-snippet
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /$2
nginx.ingress.kubernetes.io/server-snippet: |
location /base/path/v1/api/update {
deny all;
return 404;
}
spec:
rules:
- http:
paths:
- path: /base/path(/|$)(.*)
backend:
serviceName: myApi
servicePort: 8080
But when I send PUT request to /base/path/v1/api/update instead of getting 404 I am getting 500 which means that the path is reachable from ingress controller. Can anybody help me identify why ?
I changed to configuration-snippet and the error I got is :
Error: exit status 1
2020/08/06 18:35:07 [emerg] 1734#1734: location "/base/path/v1/api/update" is outside location "^/base/path(/|$)(.*)" in /tmp/nginx-cfg008325631:2445
nginx: [emerg] location "/base/path/v1/api/update" is outside location "^/base/path(/|$)(.*)" in /tmp/nginx-cfg008325631:2445
nginx: configuration file /tmp/nginx-cfg008325631 test failed

Adding my own answer :
The final config that worked for me was :
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: nginx-configuration-snippet
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /$2
nginx.ingress.kubernetes.io/server-snippet: |
location ~* "^/base/path/v1/api/update" {
deny all;
return 403;
}
spec:
rules:
- http:
paths:
- path: /base/path(/|$)(.*)
backend:
serviceName: myApi
servicePort: 8080

Just for other coming along here looking for a solution, worth knowing that snippets are ignored unless the nginx controller is started with the -enable-snippet option on the command line. It should be noted that this applies to nginxinc/kubernetes-ingress project.
The docs say:
Before creating a Deployment or Daemonset resource, make sure to
update the command-line arguments of the Ingress Controller container
in the corresponding manifest file according to your requirements.
For instance if you are running the Daemonset, you will have to edit the sudo nano kubernetes-ingress/deployments/daemon-set/nginx-ingress.yaml manifest and add your argument as shown below:
args:
- -nginx-configmaps=$(POD_NAMESPACE)/nginx-config
- -default-server-tls-secret=$(POD_NAMESPACE)/default-server-secret
- -enable-snippets
#- -v=3 # Enables extensive logging. Useful for troubleshooting.
#- -report-ingress-status
#- -external-service=nginx-ingress
#- -enable-prometheus-metrics
#- -global-configuration=$(POD_NAMESPACE)/nginx-configuration
Also check other Command-line Arguments in the docs here
If you are looking for helm follow this guide in the docs

Related

error: resource mapping not found || make sure CRDs are installed first

error: resource mapping not found for name: "ingress-srv" namespace: "" from "ingress-srv.yaml": no matches for kind "Ingress" in version "networking.k8s.io/v1beta1"
ensure CRDs are installed first
I am new to Kubernetes, I was setting up ingress nginx on minikube and it installed successfully but when I try to run using kubectl apply -f filename it gives above error
here is the code
filename: ingress-srv.yaml
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: ingress-srv
annotations:
kubernetes.io/ingress.class: nginx
spec:
rules:
- host: posts.com
http:
paths:
- path: /posts
pathType: Prefix
backend:
serviceName: posts-clusterip-srv
servicePort: 4000
The resource type specified in your manifest, networking.k8s.io/v1beta1 Ingress, was removed in Kubernetes v1.22 and replaced by networking.k8s.io/v1 Ingress (see the deprecation guide for details). If your cluster's Kubernetes server version is 1.22 or higher (which I suspect it is) trying to create an Ingress resource from your manifest will result in exactly the error you're getting.
You can check your cluster's Kubernetes server version (as Kamol Hasan points out) using the command kubectl version --short.
If the version is indeed 1.22 or higher, you'll need to modify your YAML file so that its format is valid in the new version of the API. This pull request summarises the format differences. In your case, ingress-srv.yaml needs to be changed to:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: ingress-srv
annotations:
kubernetes.io/ingress.class: nginx
spec:
rules:
- host: posts.com
http:
paths:
- path: /posts
pathType: Prefix
backend:
service:
name: posts-clusterip-srv
port:
number: 4000

what is the difference between / and /* in ingress spec/rules/http/paths/path

I have the following ingress resource
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: kubernetes-demo-ing
annotations:
nginx.ingress.kubernetes.io/ssl-redirect: \"false\"
nginx.ingress.kubernetes.io/rewrite-target: /
spec:
rules:
- host: test.my-docker-kubernetes-demo.com
http:
paths:
- path: /*
backend:
serviceName: my-demo-service
servicePort: 3000
My app was not getting accessed here test.my-docker-kubernetes-demo.com i was getting too many redirects error
but when i replaced under path from path: /* to path: /, it worked.
But i am not able to find how it fixed the problem, any help in understanding or explaining this would be great.
The meaning of / and /* depends on your ingress implementation, for example there are different ways of selecting a range of paths using the NGINX vs GCE ingress implementations:
NGINX: https://kubernetes.github.io/ingress-nginx/user-guide/ingress-path-matching/
path: /foo/.*
GCE: https://cloud.google.com/kubernetes-engine/docs/tutorials/http-balancer#step_6_optional_serving_multiple_applications_on_a_load_balancer
path: /*
You can choose the implementation to use by setting the kubernetes.io/ingress.class annotation.
In your case, assuming you're using NGINX, /* isn't interpreted as a glob pattern so only allows connecting literally to /*. Anything else would be sent to the default backend.
You should check NGINX Ingress Controller - Rewrite.
Starting in Version 0.22.0, ingress definitions using the annotation nginx.ingress.kubernetes.io/rewrite-targetare not backwards compatible with previous versions. In Version 0.22.0 and beyond, any substrings within the request URI that need to be passed to the rewritten path must explicitly be defined in a capture group
Captured groups are saved in numbered placeholders, chronologically, in the form $1, $2 ... $n. These placeholders can be used as parameters in the rewrite-target annotation.
You can check version in a following way:
kubectl exec -it <nginx-ingress-controller-pod-name> -n ingress-nginx -- /nginx-ingress-controller --version
I think your Ingress should look like the following:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: kubernetes-demo-ing
annotations:
nginx.ingress.kubernetes.io/ssl-redirect: "false"
nginx.ingress.kubernetes.io/rewrite-target: /$1
spec:
rules:
- host: test.my-docker-kubernetes-demo.com
http:
paths:
- path: /(.*)
backend:
serviceName: my-demo-service
servicePort: 3000
You can find a short explanation for spec.rules[].http.paths[].path in k8s.io/api/extensions/v1beta1.

Kubernetes: path based only works with root

I have an EKS cluster and currently use path based routing. I previously posted this thread, so all my configuration is on there: Kubernetes: 502 (Bad getaway)
My ingress controller is from there: https://raw.githubusercontent.com/kubernetes/ingress-nginx/nginx-0.24.1/deploy/mandatory.yaml
Here is my ingress:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: simple-fanout-example
namespace : default
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/rewrite-target: /$2
spec:
rules:
- host: domain.com
http:
paths:
- path: /bleble(/|$)(.*)
backend:
serviceName: bleble-svc
servicePort: 8080
- path: /hello-world
backend:
serviceName: hello-world-svc
servicePort: 8080
This works absolutly fine, until you try to go anywhere that is not the domain.com/bleble or domain.com/hello-world. The services need to refer to each other, and the uri they request is just domain.com, which obviously doesn't work because the service is at domain.com/nameoftheservice.
The 2 problems are :
The service name is bleble, so we decided the path should be /bleble. I saw on this documentation (kubernetes.github.io/ingress-nginx/examples/rewrite) that adding (/|$)(.*) would allow rewrite. We want to use url as domain.com/bleble/swagger or domain.com/bleble/clients, ... But when we type those, it brings back to what was on /bleble
I need bleble to get an info from hello-world. Right now, instead of going from domain.com/bleble to domain.com/hello-world, it goes from domain.com/bleble to domain.com. it seems to only be able to call the host name, and not the path.
I've tried doing a single ingress resource and have nginx.ingress.kubernetes.io/app-root : /bleble but this doesn't seem to be working. I've tried the annotation as well nginx.ingress.kubernetes.io/rewrite-target: /coretest
Thanks for the help!

Kubernetes Ingress needs Reverse Proxy setting

In my Kubernetes Cluster i have some challenges with the Ingress. As example i installed NodeRed und the Nginx-ingress via Helm. NodeRed is available via
FQDN: http://my.server.name:31827
IP: http://10.x.x.x:31827
Now i created an Ingress:
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: nr-ingress
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /
spec:
tls:
- secretName: tls-secret1
hosts:
- my.server.name
rules:
- host: my.server.name
http:
paths:
- path: /nr
backend:
serviceName: my-nodered-node-red
servicePort: 1880
When i do a Get http://my.server.name/nr i see only parts working, see the screenshot:
It looks to me, that i missed the reverse proxy settings. Normally i would place those things in a reverse proxy setting in the nginx like this. But this is not possible because i am using the Nginx-ingress.
location / {
proxy_pass http://localhost:1880/;
}
But i do not know how to do that in Kubernetes ? What do i miss ? kubernetes version is 1.14.1.
I haven't used it so I'm not sure if it helps, but you might want to try adding an annotation for Proxy redirect.
With the annotations nginx.ingress.kubernetes.io/proxy-redirect-from and nginx.ingress.kubernetes.io/proxy-redirect-to it is possible to set the text that should be changed in the Location and Refresh header fields of a proxied server response
Setting "off" or "default" in the annotation nginx.ingress.kubernetes.io/proxy-redirect-from disables nginx.ingress.kubernetes.io/proxy-redirect-to, otherwise, both annotations must be used in unison. Note that each annotation must be a string without spaces.
By default the value of each annotation is "off".
Also you can use ConfigMap for customizing your Nginx configuration.
kind: ConfigMap
apiVersion: v1
metadata:
name: nginx-configuration
namespace: ingress-nginx
labels:
app: ingress-nginx
data:
http-snippet: |
location = / {
proxy_pass http://localhost:1880/;
}
Or server-snippets annotation for your Ingress:
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: nr-ingress
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /
nginx.ingress.kubernetes.io/server-snippet: |
location = / {
proxy_pass http://localhost:1880/;
}
spec:
tls:
- secretName: tls-secret1
hosts:
- my.server.name
rules:
- host: my.server.name
http:
paths:
- path: /nr
backend:
serviceName: my-nodered-node-red
servicePort: 1880
I hope this helps.
Maybe too late for the answer, but I had the same problem and solve it:
1-changed httpRoot: '/nr' in node red settings.xml configuration file (in kubernetes, probably defined in a PV) (#vasili-angapov mentions)
2- set ingress:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: nodered-ingress
annotations:
kubernetes.io/ingress.class: "nginx"
spec:
rules:
- http:
paths:
- path: /nr
pathType: Prefix
backend:
service:
name: nodered
port:
number: 1880
It looks like your static content is still using root path prefix, you can verify that using browser developer console. Common applications should have a configuration to understand that they are running on non-root path prefix. You should find that configuration option in your application and configure it properly. Nginx ingress has nothing to do with this error.

traefik ingress custom error in kubernetes

I need to set a custom error in traefik ingress on kubernetes so that when there is no endpoint or when the status is "404", or "[500-600]" it redirects to another error service or another custom error message I used the annotation as it's in the documentation in the ingress file as this (Note: this a helm template output of passing the annotation as a yaml in the values.yaml file)
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: frontend
namespace: "default"
annotations:
external-dns.alpha.kubernetes.io/target: "domain.com"
kubernetes.io/ingress.class: "traefik"
traefik.ingress.kubernetes.io/error-pages: "map[/:map[backend:hello-world status:[502 503]]]"
spec:
rules:
- host: frontend.domain.com
http:
paths:
- backend:
serviceName: frontend
servicePort: 3000
path: /
The answer by ldez is correct, but there are a few caveats:
First off, these annotations only work for traefik >= 1.6.x (earlier versions may support error pages, but not for the kubernetes backend)
Second, the traefik backend must be configured through kubernetes. You cannot create a backend in a config file and use it with kubernetes, at least not in traefik 1.6.x
Here's how the complete thing looks like. foo is just a name, as explained in the other answer, and can be anything:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: frontend
namespace: "default"
annotations:
external-dns.alpha.kubernetes.io/target: "domain.com"
kubernetes.io/ingress.class: "traefik"
traefik.ingress.kubernetes.io/error-pages: |-
foo:
status:
- "404"
- "500"
# See below on where "error-pages" comes from
backend: error-pages
query: "/{{status}}.html"
spec:
rules:
# This creates an ingress on an non-existing host name,
# which binds to a service. As part of this a traefik
# backend "error-pages" will be created, which is the one
# we use above
- host: error-pages
http:
paths:
- backend:
serviceName: error-pages-service
servicePort: https
- host: frontend.domain.com
http:
# The configuration for your "real" Ingress goes here
# This is the service to back the ingress defined above
# Note that you can use anything for this, including an internal app
# Also: If you use https, the cert on the other side has to be valid
---
kind: Service
apiVersion: v1
metadata:
name: error-pages-service
namespace: default
spec:
ports:
- name: https
port: 443
type: ExternalName
externalName: my-awesome-errors.mydomain.test
If you use this configuration, and your app sends a 404, then https://my-awesome-errors.mydomain.test/404.html would be shown as the error page.
The correct syntax is:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: frontend
namespace: "default"
annotations:
external-dns.alpha.kubernetes.io/target: "domain.com"
kubernetes.io/ingress.class: "traefik"
traefik.ingress.kubernetes.io/error-pages: |-
foo:
status:
- "404"
backend: bar
query: /bar
fii:
status:
- "500-600"
backend: bar
query: /bir
spec:
rules:
- host: frontend.domain.com
http:
paths:
- backend:
serviceName: frontend
servicePort: 3000
path: /
https://docs.traefik.io/v1.6/configuration/backends/kubernetes/#general-annotations
Note that, currently, the Helm Charts doesn't support this feature.
Ingress does not support that annotations that you guys are using there!
That annotations are supported with Service only, Ingress is using host section.