Setting "Cache-Control" header with Kubernetes ingress - kubernetes

I have a kubernetes cluster running in AWS and am trying to modify the Cache Controller headers via the kubernetes ingress as such:
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: example-ingress-lab-static
namespace: lab
annotations:
ingress.kubernetes.io/rewrite-target: /$1
ingress.kubernetes.io/enable-cors: "true"
ingress.kubernetes.io/cors-allow-methods: GET, POST,PUT, OPTIONS, DELETE, HEAD, PATCH
ingress.kubernetes.io/cors-allow-headers: >-
Accept-Charset, Accept-Encoding, Access-Control-Request-Headers, Access-Control-Request-Method, Authorization,
Cache-Control, Connection, Content-Encoding, Content-Type, Content-Length, DNT, Date, Host, If-Modified-Since,
Keep-Alive, Origin, Referer, Server, TokenIssueTime, Transfer-Encoding, User-Agent, Vary, X-CustomHeader, X-Requested-With,
password, username, x-request-id, x-ratelimit-app, x-auth-id, x-auth-key, x-guest-token, X-HTTP-Method-Override,
x-oesp-username, x-oesp-token, x-cus, x-dev, X-Client-Id, X-Device-Code, X-Language-Code, UserRole, x-session-id, x-entitlements-token
ingress.kubernetes.io/configuration-snippet: |
more_set_headers 'Access-Control-Allow-Origin:$origin';
ingress.kubernetes.io/proxy-buffering: "on"
ingress.kubernetes.io/proxy-buffer-size: "2048k"
ingress.kubernetes.io/server-snippet: |
chunked_transfer_encoding off;
location ((https|http):\/\/.*\/test-service\/images\/.*\/imageName.*) {
more_set_headers 'Cache-Control: public, max-age=14400';
}
spec:
rules:
- host: static-url-lab.lab.cdn.example.com
http:
paths:
- path: /test-service/(.*)
backend:
serviceName: test-service
servicePort: 80
However this does not seem to be working. When I curl a resource matching that pattern I get the default values back:
# Example curl - not exact
curl -v "https://static-url-lab.lab.cdn.example.com/test-service/intent/test/image_name" -o /dev/null 2>&1 grep -E "(Cache-Control: max|X-Cache)"
< Cache-Control: max-age=172800, public
As far as I can tell the regex should be matching, but no change is taking place, what am I missing?

Try something like this :
working for me
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/configuration-snippet: |
if ($request_uri ~* \.(js|css|gif|jpe?g|png)) {
expires 1M;
add_header Cache-Control "public";
}
nginx.ingress.kubernetes.io/proxy-body-size: 50m
nginx.ingress.kubernetes.io/proxy-read-timeout: "1800"
nginx.ingress.kubernetes.io/proxy-send-timeout: "1800"

Related

access forbidden by rule, on ingress nginx log

which ingress rule is blocking the request coming in.
app : based on drupal.
any suggestions would help.
ingress log :
2022/08/11 10:00:59 [error] 20516#20516: *159406145 access forbidden by rule, client: 2a02:9b0:3d:54a2:40bf:a951:e203:79a5, server: example.com, request: "GET / HTTP/1.1", host: "example.com"
{"time": "2022-08-11T10:00:59+00:00", "remote_addr": "-", "x_forward_for": "2a02:9b0:3d:54a2:40bf:a951:e203:79a5, 172.70.156.137, 2a02:9b0:3d:54a2:40bf:a951:e203:79a5", "request_id": "d8d07bd09c84b802af91a60adbf46a73", "remote_user": "-", "bytes_sent": 583, "request_time": 0.000, "status": 403, "vhost": "example.com", "request_proto": "HTTP/1.1", "path": "/", "request_query": "-", "request_length": 647, "duration": 0.000,"method": "GET", "http_referrer": "-", "http_user_agent": "Mozilla/5.0 (iPhone; CPU iPhone OS 15_6 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/15.6 Mobile/15E148 Safari/604.1", "auth_apikey": "-", "auth_authorization": "-", "auth_X-Api-caps": "-", "auth_uid": "-"}
ingress.yaml
apiVersion: v1
items:
- apiVersion: extensions/v1beta1
kind: Ingress
metadata:
annotations:
meta.helm.sh/release-name: example-production
meta.helm.sh/release-namespace: example
nginx.ingress.kubernetes.io/configuration-snippet: |
more_clear_headers "Server";
more_set_headers "X-Content-Type-Options: nosniff";
more_set_headers "X-XSS-Protection: 1";
more_set_headers "X-Request-Id: $request_id";
more_set_headers "X-Router-Id: example-portal-anonymous";
more_set_headers "Strict-Transport-Security: max-age=15768000";
more_set_headers "Content-Security-Policy: frame-ancestors 'self'";
nginx.ingress.kubernetes.io/cors-allow-headers: X-Forwarded-For, Authorization
nginx.ingress.kubernetes.io/cors-allow-methods: GET, POST, PUT, PATCH, DELETE,
OPTIONS
nginx.ingress.kubernetes.io/cors-allow-origin: example.com
nginx.ingress.kubernetes.io/enable-cors: "true"
nginx.ingress.kubernetes.io/enable-modsecurity: "true"
nginx.ingress.kubernetes.io/enable-owasp-core-rules: "false"
nginx.ingress.kubernetes.io/limit-rpm: "0"
nginx.ingress.kubernetes.io/modsecurity-transaction-id: $request_id
nginx.ingress.kubernetes.io/proxy-buffer-size: 20k
nginx.ingress.kubernetes.io/use-regex: "true"
nginx.ingress.kubernetes.io/whitelist-source-range: 0.0.0.0/0
creationTimestamp: "2022-08-10T11:19:21Z"
generation: 1
labels:
app.kubernetes.io/managed-by: example
name: portal-anonymous
namespace: example
resourceVersion: "523173759"
selfLink: /apis/extensions/v1beta1/namespaces/example/ingresses/portal-anonymous
uid: 9cdb29b2-a463-4851-b4c4-b5a58be28580
spec:
rules:
- host: example.com
http:
paths:
- backend:
serviceName: portal
servicePort: 80
path: /
pathType: Prefix
status:
loadBalancer: {}
kind: List
metadata:
resourceVersion: ""
selfLink: ""
I suspect the problem is actually revealed in your config:
nginx.ingress.kubernetes.io/whitelist-source-range: 0.0.0.0/0
You are allowing all IPv4 IPs with this range, but in your log, you have this:
"x_forward_for": "2a02:9b0:3d:54a2:40bf:a951:e203:79a5, 172.70.156.137, 2a02:9b0:3d:54a2:40bf:a951:e203:79a5"
So this means you're using IPv6 as well, so you might want to add ::/0 to the whitelist-source-range to allow all IPv6 as well as all IPv4 ranges. OR just omit the annotation entirely if you want to allow anything in.

Setting up kubernets Ingress proxy-body-size based on request method

I've been trying to set up a max body size in the Ingress controller based on the HTTP method of a given path.
Basically the POST method should allow 3m as max size and all the other methods should allow 1m.
Right now my main idea was to do something like:
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: ingress-custom-service
namespace: development
annotations:
nginx.ingress.kubernetes.io/ssl-redirect: "false"
nginx.ingress.kubernetes.io/force-ssl-redirect: "false"
kubernetes.io/ingress.class: "nginx-dev"
nginx.ingress.kubernetes.io/configuration-snippet: |
internal;
rewrite ^ $original_uri break;
nginx.ingress.kubernetes.io/server-snippet: |
location /api/v1/my-endpoint {
if ( $request_method = POST) {
set $target_destination '/_post';
client_max_body_size 3M;
}
if ( $request_method != POST) {
set $target_destination '/_not_post';
client_max_body_size 1M;
}
set $original_uri $uri;
rewrite ^ $target_destination last;
}
spec:
tls:
rules:
- host: my-host.com
http:
paths:
- path: /_post
backend:
serviceName: my-service
servicePort: 8080
- path: /_not_post
backend:
serviceName: my-service
servicePort: 8080
But then I'm getting the following error in the pod:
Is there any way I can correctly set-up the max body size via the ingress controller?
Try changing your annotations with the configuration-snippet
nginx.ingress.kubernetes.io/configuration-snippet: |
location /upload-path {
client_max_body_size 8M;
}
Read more at : https://github.com/kubernetes/ingress-nginx/blob/main/docs/user-guide/nginx-configuration/annotations.md#configuration-snippet

can ingress rewrite 405 to the origin url and change the http-errors 405 to 200?

Can ingress rewrite 405 to the origin url and change the http-errors 405 to 200?
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: frontend-ingress
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/rewrite-target: /$1
spec:
rules:
- http:
paths:
- path: /page/user/(.*)
pathType: Prefix
backend:
serviceName: front-user
servicePort: 80
- path: /page/manager/(.*)
pathType: Prefix
backend:
serviceName: front-admin
servicePort: 80
Ngnix can realize that visit a html page by a post method but I want to know how to realize by ingress.
server {
listen 80;
# ...
error_page 405 =200 #405;
location #405 {
root /srv/http;
proxy_method GET;
proxy_pass http://static_backend;
}
}
This is an e.g. that ngnix realize that visit a html page by a post method to change 405 to 200 and change the method to get
You can use server snippet annotation to achieve it.
Also I rewrote your ingress from extensions/v1beta1 apiVersion to networking.k8s.io/v1, because starting kubernetes v1.22 previous apiVersion is be removed:
$ kubectl apply -f ingress-snippit.yaml
Warning: extensions/v1beta1 Ingress is deprecated in v1.14+, unavailable in v1.22+; use networking.k8s.io/v1 Ingress
Ingress-snippet-v1.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: frontend-ingress
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/rewrite-target: /$1
nginx.ingress.kubernetes.io/server-snippet: | # adds this block to server
error_page 405 =200 #405;
location #405 {
root /srv/http;
proxy_method GET;
proxy_pass http://static_backend; # tested with IP since I don't have this upstream
}
spec:
rules:
- http:
paths:
- path: /page/user/(.*)
pathType: Prefix
backend:
service:
name: front-user
port:
number: 80
- path: /page/manager/(.*)
pathType: Prefix
backend:
service:
name: front-admin
port:
number: 80
Applying manifest above and verifying /etc/nginx/nginx.conf in ingress-nginx-controller pod:
$ kubectl exec -it ingress-nginx-controller-xxxxxxxxx-yyyy -n ingress-nginx -- cat /etc/nginx/nginx.conf | less
...
## start server _
server {
server_name _ ;
listen 80 default_server reuseport backlog=4096 ;
listen 443 default_server reuseport backlog=4096 ssl http2 ;
set $proxy_upstream_name "-";
ssl_certificate_by_lua_block {
certificate.call()
}
# Custom code snippet configured for host _
error_page 405 =200 #405;
location #405 {
root /srv/http;
proxy_method GET;
proxy_pass http://127.0.0.1; # IP for testing purposes
}
location ~* "^/page/manager/(.*)" {
set $namespace "default";
set $ingress_name "frontend-ingress";
set $service_name "front-admin";
set $service_port "80";
set $location_path "/page/manager/(.*)";
set $global_rate_limit_exceeding n;
...

Kubernetes rewrite-target paths for appending path to match

I'm using the OSS ingress-nginx Ingress controller and trying to create a rewrite-target rule such that I can append a path string before my string match.
If I wanted to create a rewrite rule with regex that matches /matched/path and rewrites that to /prefix/matched/path, how might I be able to do that?
I've tried something like the following but it's no good, and I'm just confused about the syntax of this ingress definition:
metadata:
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/rewrite-target: /$2
spec:
rules:
- path: /(/prefix/)(/|$)(/matched/path)(.*)
backend:
serviceName: webapp1
If I wanted to create a rewrite rule with regex that matches
/matched/path and rewrites that to /prefix/matched/path, how might
I be able to do that?
In order to achieve this you have add /prefix into your rewrite-target.
Here's a working example with ingress syntax from k8s v1.18:
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: example-ingress-v118
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /prefix/$1
spec:
rules:
http:
paths:
- path: /(matched/path/?.*)
backend:
serviceName: test
servicePort: 80
Since the syntax for the new ingress changed in 1.19 (see release notes and some small info at the end) I`m placing also an example with it:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: example-ingress-v119
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /prefix/$1
spec:
rules:
- http:
paths:
- path: /(matched/path/?.*)
pathType: Prefix
backend:
service:
name: test
port:
number: 80
Here is a test with http echo server:
➜ ~ curl 172.17.0.4/matched/path
{
"path": "/prefix/matched/path",
"headers": {
"host": "172.17.0.4",
"x-request-id": "011585443ebc6adcf913db1c506abbe6",
"x-real-ip": "172.17.0.1",
"x-forwarded-for": "172.17.0.1",
"x-forwarded-host": "172.17.0.4",
"x-forwarded-port": "80",
"x-forwarded-proto": "http",
"x-scheme": "http",
"user-agent": "curl/7.52.1",
"accept": "*/*"
},
This rule will also ignore the / at the end of the request:
➜ ~ curl 172.17.0.4/matched/path/
{
"path": "/prefix/matched/path/",
"headers": {
"host": "172.17.0.4",
"x-request-id": "0575e9022d814ba07457395f78dbe0fb",
"x-real-ip": "172.17.0.1",
"x-forwarded-for": "172.17.0.1",
"x-forwarded-host": "172.17.0.4",
"x-forwarded-port": "80",
"x-forwarded-proto": "http",
"x-scheme": "http",
"user-agent": "curl/7.52.1",
"accept": "*/*"
},
Worth to mention some notable differences/changes in the new ingress syntax:
spec.backend -> spec.defaultBackend
serviceName -> service.name
servicePort -> service.port.name (for string values)
servicePort -> service.port.number (for numeric values) pathType no longer has a default value in v1; "Exact", "Prefix", or
"ImplementationSpecific" must be specified Other Ingress API updates
backends can now be resource or service backends
path is no longer required to be a valid regular expression (#89778,
#cmluciano) [SIG API Machinery, Apps, CLI, Network and Testing]

SignalR behind k8s nginx ingress disconnects immediately upon connection

Here's what I have for the nginx ingress:
nginx.ingress.kubernetes.io/proxy-set-headers: xyz/proxy-headers
nginx.ingress.kubernetes.io/read-timeout: 3600
nginx.ingress.kubernetes.io/session-cookie-expires: 14400
nginx.ingress.kubernetes.io/ssl-protocols: TLSv1.3 TLSv1.2
cert-manager.io/cluster-issuer: letsencrypt
nginx.ingress.kubernetes.io/affinity: cookie
nginx.ingress.kubernetes.io/configuration-snippet: if ($host = 'example.com' ) {
rewrite ^ https://www.example
.com$request_uri permanent;
}
nginx.ingress.kubernetes.io/proxy-buffer-size: 16k
nginx.ingress.kubernetes.io/force-ssl-redirect: true
nginx.ingress.kubernetes.io/proxy-body-size: 8m
nginx.ingress.kubernetes.io/proxy-send-timeout: 3600
nginx.ingress.kubernetes.io/server-alias: example.com
nginx.ingress.kubernetes.io/session-cookie-max-age: 14400
nginx.ingress.kubernetes.io/session-cookie-name: affinity
...
}
Here's the proxy headers:
apiVersion: v1
kind: ConfigMap
metadata:
name: proxy-headers
namespace: xyz
data:
X-Content-Type-Options: "nosniff"
X-XSS-Protection: "1; mode=block"
Referrer-Policy: "no-referrer-when-downgrade"
Feature-Policy: "notifications 'self'; usemedia *;gyroscope: 'none'"
When connected directly, the websocket is fine. When it's behind nginx I get the following:
main.5f9ab0903b48ce06734b.js:6 [2020-04-20T18:54:39.190Z] Information: Connection disconnected.
main.5f9ab0903b48ce06734b.js:6 Could not connect Error: Cannot send data if the connection is not in the 'Connected' State.
main.5f9ab0903b48ce06734b.js:6 ERROR Error: Uncaught (in promise): [object Undefined]
at _ (polyfills.d1de8a43b27b04443379.js:1)
at t.handshakeRejecter (polyfills.d1de8a43b27b04443379.js:1)
at t.connectionClosed (main.5f9ab0903b48ce06734b.js:6)
at t.connection.onclose (main.5f9ab0903b48ce06734b.js:6)
at t.stopConnection (main.5f9ab0903b48ce06734b.js:6)
at t.Y.transport.onclose (main.5f9ab0903b48ce06734b.js:6)
at t.close (main.5f9ab0903b48ce06734b.js:6)
at t.stop (main.5f9ab0903b48ce06734b.js:6)
at t.<anonymous> (main.5f9ab0903b48ce06734b.js:6)
at main.5f9ab0903b48ce06734b.js:6
I can't find anything that would cause it to do what it's doing and I can't get any deeper insight into what's causing it. Ideas?
Here's what I ultimately came up with:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/x-forwarded-proto: https
nginx.ingress.kubernetes.io/use-forwarded-headers: "true"
nginx.ingress.kubernetes.io/rewrite-target: /
nginx.ingress.kubernetes.io/proxy-buffer-size: "16k"
nginx.ingress.kubernetes.io/read-timeout: "3600"
nginx.ingress.kubernetes.io/send-timeout: "3600"
nginx.ingress.kubernetes.io/ssl-protocols: "TLSv1.3 TLSv1.2"
nginx.ingress.kubernetes.io/proxy-send-timeout: "3600"
nginx.ingress.kubernetes.io/session-cookie-name: "affinity"
nginx.ingress.kubernetes.io/session-cookie-expires: "14400"
nginx.ingress.kubernetes.io/session-cookie-max-age: "14400"
nginx.ingress.kubernetes.io/affinity: cookie
nginx.ingress.kubernetes.io/force-ssl-redirect: "true"
nginx.ingress.kubernetes.io/proxy-body-size: 16m
nginx.ingress.kubernetes.io/add-headers: "yournamespace/add-headers"
And then the following in the add-headers config map:
apiVersion: v1
kind: ConfigMap
metadata:
name: add-headers
namespace: yournamespace
data:
X-Content-Type-Options: "nosniff"
X-XSS-Protection: "1; mode=block"
Referrer-Policy: "no-referrer-when-downgrade"
Feature-Policy: "notifications 'self'; usemedia *;gyroscope: 'none'"
kind: ConfigMap
This enables SignalR to work properly behind the proxy, handles affinity (which apparently was the original issue) and gives you full HSTS encryption and an A+ rating on crypto tests.