I have recently been using the nginxdemo/nginx-ingress controller.
As I understand it this controller cannot do SSL Passthrough (by that I mean pass the client certificate all the way through to the backend service for authentication), so instead I have been passing the clients subject DN through a header.
Ultimately I would prefer SSL-Passthrough and have been looking at the kubernetes/ingress-nginx project which apparently supports SSL passthrough.
Does anyone have an experience with this controller and SSL Passthrough.
The few Ingress examples showing passthrough that I have found leave the path setting blank.
Is this because passthrough has to take place at the TCP level (4) rather then at HTTP (7)?
Right now, I have a single host rule that services mutiple paths.
completing on lch answer I would like to add that I had the same problem recently and I sorted it out modifiying the ingress-service deployment (I know, it should be a DaemonSet but that's a different story)
The change was adding the parameter to spec.containers.args:
--enable-ssl-passthrough
Then I've added the following annotations to my ingress:
kubernetes.io/ingress.allow-http: "false"
nginx.ingress.kubernetes.io/secure-backends: "true"
nginx.ingress.kubernetes.io/ssl-passthrough: "true"
nginx.ingress.kubernetes.io/ssl-redirect: "true"
The important one are secure-backends and ssl-passthrough but I think the rest are a good idea, provided you're not expecting http traffic there
SSH-Passthrough is working fine for me. Here is the Official Documentation
And here is an example usage:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: my-service-ingress
namespace: my-service
annotations:
kubernetes.io/ingress.allow-http: "false"
nginx.ingress.kubernetes.io/ssl-passthrough: "true"
nginx.ingress.kubernetes.io/secure-backends: "true"
spec:
rules:
- host: my.example.com
http:
paths:
- backend:
serviceName: my-service
Related
So my goal is to be able to access my CockroachDB from domain like db.test.com with cert.
I want to use cert-manager letsencrypt to issue keys. And it should work with CF (in non proxy mode as I think they do not support tcp for this)
At first to test everything I used normal kubectl port-forward which worked, but now I needed to expose it always.
I have tried using Ingress (using ingress-nginx)
I know that Ingress is mostly HTTP/HTTPS but I saw it can be used for the thing I need and IN CF I cannot point to port that I needed.
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: tcp-example-ingress
namespace: default
annotations:
kubernetes.io/ingress.class: "nginx"
nginx.ingress.kubernetes.io/tcp-services: "cluster-cockroachdb-public"
nginx.ingress.kubernetes.io/tcp-service-port: "26257"
nginx.ingress.kubernetes.io/backend-protocol: "TCP"
nginx.ingress.kubernetes.io/rewrite-target: /
spec:
tls:
- hosts:
- db.test.com
secretName: db-access-ssl-cert-production
rules:
- host: db.test.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: cluster-cockroachdb-public
port:
number: 26257
Attempting to connect does not work, and in logs I can see 400 status code with strange characters like \x20...
No matter what I tried I could not get it to work..
I did manage to get web-ui portion working that was easy enough.
Other resource that might be helpful is the values.yaml that I used
conf:
cache: "2Gi"
max-sql-memory: "2Gi"
# My WEB-UI that works
ingress:
enabled: true
labels: {}
annotations:
kubernetes.io/ingress.class: nginx
cert-manager.io/cluster-issuer: letsencrypt-production
paths: [/]
hosts:
- db-ui.test.com
tls:
- hosts: [db-ui.test.com]
secretName: ssl-cert-production
Everything else is default
I solved my issue by following the tutorial below:
https://mailazy.com/blog/exposing-tcp-udp-services-ingress/
also mentioned here
https://kubernetes.github.io/ingress-nginx/user-guide/exposing-tcp-udp-services/
Ingress does not support TCP or UDP services so we use ingress-nginx config for it we patch ingress-nginx values of chart and add custom one (Copy default values.yaml from github helm chart for ingress-nginx)
I just edited this portion:
# -- TCP service key-value pairs
## Ref: https://github.com/kubernetes/ingress-nginx/blob/main/docs/user-guide/exposing-tcp-udp-services.md
##
tcp:
"26257": "default/cluster-cockroachdb-public:26257"
After that we run helm upgrade command to replace values of ingress-nginx and after that it should work for anyone else as well.
If you are using cloudflare make sure to disable proxy!
I have a project on k8s with 3 services, that I want to cover with basic auth, and 1 service that I'd like to be public.
In ingress 4 services devided by url rules, that have different prefixes.
I had found tutorial about basic auth setup in ingress for all rules, but not about excluded, included urls.
Unfortunately GCP Ingress does not provide basic auth authentication as this feature is specific for Nginx Ingress.
As workaround for basic auth in GCP Ingress you can use IAP. Detailed How To information can be found in Enabling IAP for GKE article.
If you would still like to use Nginx Ingress basic auth you can do it on GKE but you need specify nginx annotation.
metadata:
name: foo
annotations:
kubernetes.io/ingress.class: "nginx"
Regarding using basic auth on only one service out of four, you can createa 2 Ingress. Very similar issue was discussed in another stackoverflow thread, which contains good solution - Nginx-ingress Kubernetes routing with basic auth.
Basic Auth Ingress
First Ingress should be without annotations:
nginx.ingress.kubernetes.io/auth-type
nginx.ingress.kubernetes.io/auth-secret
nginx.ingress.kubernetes.io/auth-realm
Second Ingress should contain proper annotations and should look similar to below YAML.
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: auth-ingress
annotations:
kubernetes.io/ingress.class: "nginx"
nginx.ingress.kubernetes.io/auth-type: basic
nginx.ingress.kubernetes.io/auth-secret: basic-auth
nginx.ingress.kubernetes.io/auth-realm: 'Authentication Required'
#cert-manager.io/cluster-issuer: if you would use cert manager like letsencrypt
spec:
tls:
- hosts:
- example.com
secretName: example-tls
rules:
- host: example.com
http:
paths:
- path: /auth
backend:
serviceName: auth-service
servicePort: <auth-service-port>
Aditional information
There is an option to deny all traffic to specific path. It can be achieved by configuration-snippet annotation.
annotations:
nginx.ingress.kubernetes.io/configuration-snippet: |
location /specificpath {
deny all;
}
There is k8s cluster with nginx-ingress. There is domain without certificate (example.io). I want redirect all requests to another my domain with certificate (example.com) without exception about bad certificate on domain example.io in browser. How can I do it? Thanks.
nginx.ingress.kubernetes.io/permanent-redirect, ingress.kubernetes.io/configuration-snippet didn;t help me.
Example:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: redirect-ingress
annotations:
??????
spec:
rules:
- host: example.io
http:
paths:
- path: /
backend:
serviceName: some-cluster-service
servicePort: 2020
You can use nginx.ingress.kubernetes.io/force-ssl-redirect: "true" annotation:
When using SSL offloading outside of cluster (e.g. AWS ELB) it may be
useful to enforce a redirect to HTTPS even when there is no TLS
certificate available. This can be achieved by using the
nginx.ingress.kubernetes.io/force-ssl-redirect: "true" annotation in
the particular resource.
I have a pod that has following chart:
ingress:
enabled: true
annotations:
kubernetes.io/ingress.class: nginx
// ...
paths:
- /api/myservice
My pod exposes api and additionally a /prometheus endpoint that is accessible through /api/myservice/prometheus.
I would like to have prometheus visible inside my cluster but not from /api/myservice/prometheus. How I can achieve that?
You can add an Ingress rule that redirects the endpoint to the default-backend:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: block
annotations:
ingress.kubernetes.io/ssl-redirect: "true"
ingress.kubernetes.io/rewrite-target: /
spec:
tls:
- hosts:
- myapp.to
rules:
- host: myapp.to
http:
paths:
- path: /metrics
backend:
serviceName: ingress-default-backend
servicePort: 8080
If you just need internal access, I don't think you should put it into ingress. You could make use of the DNS Service together with Kubernetes cluster already.
If your prometheus service is running the same namespace as your working pod, you could use following address to access it.
http://prometheus-svc-name
If your prometheus service is running in a different namespace, you could use:
http://prometheus-svc-name.prometheus-namespace.svc.cluster.local
to access the service.
Update:
More clear about the question after comments.
You could write another rule to hide your /prometheus endpoint like this:
ingress:
enabled: true
annotations:
kubernetes.io/ingress.class: nginx
// ...
paths:
- /api/myservice => myservice
- /api/myservice/prometheus => 404 default backend.
nginx-ingress will match the longest route first. It will route /api/myservice/prometheus to 404. See this documentation
Best way to achieve this would be to remove the prometheus endpoint from ingress and if you are aware prometheus supports auto discovery using annotations on the service, configure prometheus for k8s auto discovery and then add annotations in you service yaml file.
https://prometheus.io/docs/prometheus/latest/configuration/configuration/#kubernetes_sd_config
change the endpoint to point to your API.
/api/myservice/my-api or use app-root annotation, to point to your api inside /api/myservice:
nginx.ingress.kubernetes.io/app-root: /path/to/my/app/root/dir
more details here: https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/annotations/#rewrite
Hi i tried the new annotation for ingress explained here
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: ssl-iagree-ingress
annotations:
kubernetes.io/ingress.allowHTTP: "false"
spec:
tls:
- secretName: secret-cert-myown
backend:
serviceName: modcluster
servicePort: 80
but i can still access it trough http, this is my setup on gcloud ingress--apache:80
Well i was able to resolve the issue, thanks to Mr Danny, from this pull request here, there was a typo in
kubernetes.io/ingress.allowHTTP: "false"
change it to
kubernetes.io/ingress.allow-http: "false"
and it works fine now.
ps: only for master version 1.3.5