Setting up kubernets Ingress proxy-body-size based on request method - kubernetes

I've been trying to set up a max body size in the Ingress controller based on the HTTP method of a given path.
Basically the POST method should allow 3m as max size and all the other methods should allow 1m.
Right now my main idea was to do something like:
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: ingress-custom-service
namespace: development
annotations:
nginx.ingress.kubernetes.io/ssl-redirect: "false"
nginx.ingress.kubernetes.io/force-ssl-redirect: "false"
kubernetes.io/ingress.class: "nginx-dev"
nginx.ingress.kubernetes.io/configuration-snippet: |
internal;
rewrite ^ $original_uri break;
nginx.ingress.kubernetes.io/server-snippet: |
location /api/v1/my-endpoint {
if ( $request_method = POST) {
set $target_destination '/_post';
client_max_body_size 3M;
}
if ( $request_method != POST) {
set $target_destination '/_not_post';
client_max_body_size 1M;
}
set $original_uri $uri;
rewrite ^ $target_destination last;
}
spec:
tls:
rules:
- host: my-host.com
http:
paths:
- path: /_post
backend:
serviceName: my-service
servicePort: 8080
- path: /_not_post
backend:
serviceName: my-service
servicePort: 8080
But then I'm getting the following error in the pod:
Is there any way I can correctly set-up the max body size via the ingress controller?

Try changing your annotations with the configuration-snippet
nginx.ingress.kubernetes.io/configuration-snippet: |
location /upload-path {
client_max_body_size 8M;
}
Read more at : https://github.com/kubernetes/ingress-nginx/blob/main/docs/user-guide/nginx-configuration/annotations.md#configuration-snippet

Related

Example Nginx plus ingress for sticky sessions during canary deployment

I’m deploying 2 services to kubernetes pods which simply echo a version number; echo-v1 & echo-v2
Where echo-v2 is considered the canary deployment, I can demonstrate sticky sessions as canary weight is reconfigured from 0 to 100 using canary & canary-weight annotations.
2 ingresses are used:
The first routes to echo-v1 with a session cookie annotation.
The second routes to echo-v2 with canary true,canary weight and session cookie annotations.
The second ingress I can apply without impacting those sessions started on the first ingress and new sessions follow the canary weighting as expected.
However I’ve since learned that those annotations are for nginx community and won’t work with nginx plus.
How can I achieve the same using ingress(es) with nginx plus?
This is the ingress configuration that works for me using Nginx community vs Nginx plus.
Nginx community:
(coffee-v1 service)
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
annotations:
kubernetes.io/ingress.class: nginx
ingress.kubernetes.io/rewrite-target: /
nginx.ingress.kubernetes.io/affinity: "cookie"
name: ingress-coffee
spec:
rules:
- http:
paths:
- path: /coffee
pathType: Exact
backend:
service:
name: coffee-v1
port:
number: 80
(coffee-v2 'canary' service)
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
annotations:
kubernetes.io/ingress.class: nginx
ingress.kubernetes.io/rewrite-target: /
nginx.ingress.kubernetes.io/affinity: "cookie"
nginx.ingress.kubernetes.io/canary: "true"
nginx.ingress.kubernetes.io/canary-weight: "100"
name: ingress-coffee-canary
spec:
rules:
- http:
paths:
- path: /coffee
pathType: Exact
backend:
service:
name: coffee-v2
port:
number: 80
Nginx plus:
(coffee-v1 & coffee-v2 as type 'virtualserver' not 'ingress')
apiVersion: k8s.nginx.org/v1
kind: VirtualServer
metadata:
name: cafe
spec:
host: cloudbees-training.group.net
tls:
secret: cloudbees-trn.aks.group.net-tls
upstreams:
- name: coffee-v1
service: coffee-v1-svc
port: 80
sessionCookie:
enable: true
name: srv_id_v1
path: /coffee
expires: 2h
- name: coffee-v2
service: coffee-v2-svc
port: 80
sessionCookie:
enable: true
name: srv_id_v2
path: /coffee
expires: 2h
routes:
- path: /coffee
matches:
- conditions:
- cookie: srv_id_v1
value: ~*
action:
pass: coffee-v1
- conditions:
- cookie: srv_id_v2
value: ~*
action:
pass: coffee-v2
# 3 options to handle new session below:
#
# 1) All new sessions to v1:
# action:
# pass: coffee-v1
#
# 2) All new sessions to v2:
# action:
# pass: coffee-v2
#
# 3) Split new sessions by weight
# Note: 0,100 / 100,0 weightings causes sessions
# to drop for the 0 weighted service:
# splits:
# - weight: 50
# action:
# pass: coffee-v1
# - weight: 50
# action:
# pass: coffee-v2

Need to help on routing https URL with redirectpath

I am running Nginx ingress inside one of our EKS cluster and earlier had issue to http-->https redirect as we are terminating SSL at AWS NLB. I was able to get it fixed using the method listed in this ticket https://github.com/kubernetes/ingress-nginx/issues/2724 ( Thanks to #Ariseaz )
However, along with https redirect we want to append path and redirect which is not working. Here are some methods I have tried so far
The container webpage serving paths /coffee and /tea
http://cafe.com ------> https://cafe.com ## This works because of the http --> https redirection
http://cafe.com/tea -----> https://cafe.com/tea ## This works
http://cafe.com/coffee -----> https://cafe.com/coffee ## This works
Now when I want to redirect https://cafe.com to https://cafe/coffee it does not work.
Can anyone please tell me now to append path to https:hostname and redirect.. I was able-to get is working with AWS ALB Ingress with this annotation and I am trying to get the same method with nginx ingress.
alb.ingress.kubernetes.io/actions.svc-cafe: >
{"Type":"redirect","RedirectConfig":{"Path":"/coffee","Protocol":"HTTPS", "Port": "443","StatusCode":"HTTP_301"}} ## This is to append /coffee to hostname and redirect ( https://cafe.com ---> htttps://cafe.com/coffee)
Here is my ingress file using Nginx ingress controller
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: cafe-example
annotations:
nginx.ingress.kubernetes.io/server-snippet: |
if ($host = "https://cafe.com") {
return 301 https://$host$request_uri/coffee;
}
#nginx.ingress.kubernetes.io/rewrite-target: /
#nginx.ingress.kubernetes.io/configuration-snippet: |
# rewrite ^(/coffee)$ $1/ permanent;
spec:
ingressClassName: internal-nginx
rules:
- host: cafe.com
http:
paths:
- path: /tea
pathType: Prefix
backend:
service:
name: tea-svc
port:
number: 80
- path: /coffee
pathType: Prefix
backend:
service:
name: coffee-svc
port:
number: 80
Please try :
nginx.ingress.kubernetes.io/server-snippet: |
if ($host ~ "https://cafe.com")
{
rewrite ^ https://$host$request_uri/coffee permanent;
}
or else try
nginx.ingress.kubernetes.io/configuration-snippet: |
if ($host = 'https://cafe.com' ) {
return 301 https://$host$request_uri/coffee;
}
When I change the host to cafe.com instead of https://cafe.com I can see it the url getting redirected but its going in a loop
nginx.ingress.kubernetes.io/configuration-snippet: |
if ($host = 'cafe.com' ) {
return 301 https://$host$request_uri/coffee;
}
https://cafe.com//coffee/coffee/coffee/coffee/coffee/coffee/coffee/coffee/coffee/coffee/coffee/coffee/coffee/coffee/coffee/coffee/coffee/coffee/coffee/coffee/coffee/coffee/coffee/coffee/coffee/coffee/coffee/coffee/coffee/coffee/coffee/coffee/coffee/coffee/coffee/coffee/coffee/coffee/coffee/coffee/coffee/coffee/coffee/coffee/coffee/coffee/coffee/coffee/coffee/coffee/coffee/coffee/coffee/coffee/coffee/coffee/coffee/coffee/coffee/coffee/coffee/coffee/coffee/coffee/coffee/coffee/coffee/coffee/coffee/coffee/coffee/coffee/coffee/coffee/coffee/coffee/coffee/coffee/coffee/coffee/coffee/coffee/coffee/coffee/coffee/coffee/coffee/coffee/coffee/coffee/coffee/coffee/coffee/coffee/coffee/coffee/coffee/coffee/coffee/coffee
You can try this
nginx.ingress.kubernetes.io/configuration-snippet: |
if ($host = 'cafe.com' ) {
return 301 https://cafe.com/coffee;
}
OR
nginx.ingress.kubernetes.io/configuration-snippet: |
if ($host = 'cafe.com' ) {
rewrite ^([^.]*[^/])$ https://cafe.com/coffee permanent;
}

Kubernetes ingres.yml unknown field "service.port.number" in io.k8s.api.networking.v1

I'm running into issues with my ingress.yml. Getting this error:
error: error validating "ingress.yml": error validating data: [ValidationError(Ingress.spec.rules[0].http.paths[0].backend): unknown field "service.name" in io.k8s.api.networking.v1.IngressBackend, ValidationError(Ingress.spec.rules[0].http.paths[0].backend): unknown field "service.port.number" in io.k8s.api.networking.v1.IngressBackend, ValidationError(Ingress.spec.rules[0].http.paths[1].backend): unknown field "service.name" in io.k8s.api.networking.v1.IngressBackend, ValidationError(Ingress.spec.rules[0].http.paths[1].backend): unknown field "service.port.number" in io.k8s.api.networking.v1.IngressBackend];
Based on this documentation serviceName changed to service.name and servicePort changed to service.port and so on. Here is the ingress file:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: bitwarden
namespace: bitwarden
labels:
app: bitwarden
annotations:
# the name of the nginx-ingress-controller class
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/configuration-snippet: |
more_set_headers "Request-Id: $req_id";
nginx.ingress.kubernetes.io/connection-proxy-header: "keep-alive"
nginx.ingress.kubernetes.io/enable-cors: "true"
nginx.ingress.kubernetes.io/force-ssl-redirect: "true"
nginx.ingress.kubernetes.io/limit-connections: "25"
nginx.ingress.kubernetes.io/limit-rps: "15"
nginx.ingress.kubernetes.io/proxy-body-size: 1024m
nginx.ingress.kubernetes.io/proxy-connect-timeout: "10"
nginx.ingress.kubernetes.io/proxy-send-timeout: "1800"
nginx.ingress.kubernetes.io/proxy-read-timeout: "1800"
nginx.ingress.kubernetes.io/ssl-redirect: "true"
cert-manager.io/cluster-issuer: "letsencrypt-prod"
spec:
tls:
- hosts:
- kryskanbit.ddns.net
secretName: bitwarden-tls
rules:
- host: kryskanbit.ddns.net
http:
paths:
- path: /
pathType: Prefix
backend:
service.name: bitwarden
service.port.number: 80
- path: /notifications/hub
pathType: Exact
backend:
service.name: bitwarden
service.port.number: 3012
Kubernetes has a special treatment for YAML format, so basically, you just need to extend the properties individually the service.name and service.port.number:
...
paths:
- path: /
pathType: Prefix
backend:
service:
name: bitwarden
port:
number: 80
- path: /notifications/hub
pathType: Exact
backend:
service:
name: bitwarden
port:
number: 3012
It should be as mentioned below:
backend:
serviceName: bitwarden
servicePort: 80

can ingress rewrite 405 to the origin url and change the http-errors 405 to 200?

Can ingress rewrite 405 to the origin url and change the http-errors 405 to 200?
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: frontend-ingress
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/rewrite-target: /$1
spec:
rules:
- http:
paths:
- path: /page/user/(.*)
pathType: Prefix
backend:
serviceName: front-user
servicePort: 80
- path: /page/manager/(.*)
pathType: Prefix
backend:
serviceName: front-admin
servicePort: 80
Ngnix can realize that visit a html page by a post method but I want to know how to realize by ingress.
server {
listen 80;
# ...
error_page 405 =200 #405;
location #405 {
root /srv/http;
proxy_method GET;
proxy_pass http://static_backend;
}
}
This is an e.g. that ngnix realize that visit a html page by a post method to change 405 to 200 and change the method to get
You can use server snippet annotation to achieve it.
Also I rewrote your ingress from extensions/v1beta1 apiVersion to networking.k8s.io/v1, because starting kubernetes v1.22 previous apiVersion is be removed:
$ kubectl apply -f ingress-snippit.yaml
Warning: extensions/v1beta1 Ingress is deprecated in v1.14+, unavailable in v1.22+; use networking.k8s.io/v1 Ingress
Ingress-snippet-v1.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: frontend-ingress
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/rewrite-target: /$1
nginx.ingress.kubernetes.io/server-snippet: | # adds this block to server
error_page 405 =200 #405;
location #405 {
root /srv/http;
proxy_method GET;
proxy_pass http://static_backend; # tested with IP since I don't have this upstream
}
spec:
rules:
- http:
paths:
- path: /page/user/(.*)
pathType: Prefix
backend:
service:
name: front-user
port:
number: 80
- path: /page/manager/(.*)
pathType: Prefix
backend:
service:
name: front-admin
port:
number: 80
Applying manifest above and verifying /etc/nginx/nginx.conf in ingress-nginx-controller pod:
$ kubectl exec -it ingress-nginx-controller-xxxxxxxxx-yyyy -n ingress-nginx -- cat /etc/nginx/nginx.conf | less
...
## start server _
server {
server_name _ ;
listen 80 default_server reuseport backlog=4096 ;
listen 443 default_server reuseport backlog=4096 ssl http2 ;
set $proxy_upstream_name "-";
ssl_certificate_by_lua_block {
certificate.call()
}
# Custom code snippet configured for host _
error_page 405 =200 #405;
location #405 {
root /srv/http;
proxy_method GET;
proxy_pass http://127.0.0.1; # IP for testing purposes
}
location ~* "^/page/manager/(.*)" {
set $namespace "default";
set $ingress_name "frontend-ingress";
set $service_name "front-admin";
set $service_port "80";
set $location_path "/page/manager/(.*)";
set $global_rate_limit_exceeding n;
...

Problem Sub Path Ingress Controller for Backend Service

I have problem setting path ingress controller for backend service. For example i want setup :
frontend app with angular (Path :/)
backend service with NodeJs (Path :/webservice).
NodeJS : Index.js
const express = require('express')
const app = express()
const port = 4000
app.get('/', (req, res) => res.send('Welcome to myApp!'))
app.use('/data/office', require('./roffice'));
app.listen(port, () => console.log(`Example app listening on port ${port}!`))
Another Route:roffice.js
var express = require('express')
var router = express.Router()
router.get('/getOffice', async function (req, res) {
res.send('Get Data Office')
});
module.exports = router
Deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: ws-stack
spec:
selector:
matchLabels:
run: ws-stack
replicas: 2
template:
metadata:
labels:
run: ws-stack
spec:
containers:
- name: ws-stack
image: wsstack/node/img
imagePullPolicy: IfNotPresent
ports:
- containerPort: 4000
Service.yaml
apiVersion: v1
kind: Service
metadata:
name: service-wsstack
labels:
run: service-wsstack
spec:
type: NodePort
ports:
- port: 80
protocol: TCP
nodePort: 30009
targetPort: 4000
selector:
run: ws-stack
ingress.yaml
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: stack-ingress
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /
nginx.ingress.kubernetes.io/use-regex: "true"
spec:
rules:
- host: hello-world.info
- http:
paths:
- path: /
backend:
serviceName: service-ngstack --> frondend
servicePort: 80
- path: /webservice
backend:
serviceName: service-wsstack --> backend
servicePort: 80
i setup deployment, service and ingress successfully. but when i called with curl
curl http://<minikubeip>/webservice --> Welcome to myApp! => Correct
curl http://<minikubeip>/webservice/data/office/getOffice --> Welcome to myApp! => Not correct
if i called another route, the result is the same 'Welcome to myApp'. But if i used Nodeport
curl http://<minikubeip>:30009/data/office/getOffice => 'Get Data Office', working properly.
What is the problem? any solution? Thank you
TL;DR
nginx.ingress.kubernetes.io/rewrite-target: /$2
path: /webservice($|/)(.*)
Explanation
The problem is from that line in your ingress:
nginx.ingress.kubernetes.io/rewrite-target: /
You're telling nginx to rewrite your url to / whatever it matched.
/webservice => /
/webservice/data/office/getOffice => /
To do what you're trying to do use regex, here is a simple example:
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: stack-ingress
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /$2
nginx.ingress.kubernetes.io/use-regex: "true"
spec:
rules:
- host: hello-world.info
- http:
paths:
- path: /
backend:
serviceName: service-ngstack --> frondend
servicePort: 80
- path: /webservice($|/)(.*)
backend:
serviceName: service-wsstack --> backend
servicePort: 80
This way you're asking nginx to rewrite your url with the second matching group.
Finally it gives you:
/webservice => /
/webservice/data/office/getOffice => /data/office/getOffice