K8S traffic to one service via two separate ingress (http + https) - kubernetes

So I have a bunch of services running in a cluster, all exposed via HTTP only ingress object, example:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /$2
name: some-ingress
spec:
ingressClassName: nginx
rules:
- http:
paths:
- backend:
service:
name: some-svc
port:
number: 80
path: /some-svc(/|$)(.*)
pathType: Prefix
They are accessed by http://<CLUSTER_EXTERNAL_IP>/some-svc, and it works ofc.
Now I want to create an additional ingress object for every service which will force SSL connections and allow the use of a domain instead of an IP address.
The problem is that the newer SSL ingresses always return 404 while testing the connection.
The manifests are as follows:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: "some-ingress-ssl"
annotations:
ingress.kubernetes.io/ssl-redirect: "true"
ingress.kubernetes.io/app-root: "/some-svc"
spec:
tls:
- hosts:
- foobar.com
secretName: foobar-tls
rules:
- host: foobar.com
http:
paths:
- path: /some-svc(/|$)(.*)
pathType: Prefix
backend:
service:
name: some-svc
port:
number: 80
tests (foobar.com point to CLUSTER_EXTERNAL_IP):
> curl -I http://<CLUSTER_EXTERNAL_IP>/some-svc
HTTP/1.1 200 OK
> curl -I https://foobar.com/some-svc
HTTP/2 404
Is it possible to have both ingresses simultaneously? (one enforcing SSL, the other not)
If so what am I doing wrong here?

Figured out I was missing this annotation:
nginx.ingress.kubernetes.io/rewrite-target: /$2
in SSL ingress...
works like a charm now, maybe someone will find this usefull

Related

Unable to access ingress k8s resources after using rewrite-target annotation

I have a K8s cluster, with two services - one service serving the front end static files and the other for backend. To differentiate between the two, I am using an ingress and a rest call to xyz.com/ is directed to the frontend service and rest calls to xyz.com/rest/abc are directed to the backend service (which is an api gateway and collection of services behind it). When creating an ingress, if i use the rewrite target annotation to rewrite xyz.com/rest/abc to xyz/com/abc, then i start getting errors when trying to access the front end service..
The errors that I have faced are
Did not parse stylesheet .... because non css mime types are not allowed in strict mode
Syntax error: Unexpected token < (while parsing bundle.js)
but if i am not using the rewrite annotation, then all files are transferred without any issue and the page is rendered as it should
annotation being used:
nginx.ingress.kubernetes.io/rewrite-target: /$2
using nginx ingress controller
Not sure why this is happening, could anyone help, any pointers, or any error that I am making.
Instead of using the path: "/" for the frontend service, i tried giving a more exact path "/ui" but it still gives the same issue.
New to K8s and ingress, so stuck here, and not sure what to do.
Ingress ->
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: sample-ingress
labels:
name: sample-ingress
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /$2
nginx.ingress.kubernetes.io/use-regex: "true"
spec:
rules:
- host: xyz.com
http:
paths:
- pathType: Prefix
path: /rest(/|$)(.*)
backend:
service:
name: backend-service
port:
number: 8080
- pathType: Prefix
path: /()
backend:
service:
name: frontend-service
port:
number: 3000
For me it's still not clear what the desired behavior is. If you want to have
/rest/restpath -> backend-service/restpath
and
/uipath -> frontend-service/uipath
then your ingress could look like this:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: sample-ingress
labels:
name: sample-ingress
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /$1
nginx.ingress.kubernetes.io/use-regex: "true"
spec:
rules:
- host: xyz.com
http:
paths:
- pathType: Prefix
path: /rest/(.*)
backend:
service:
name: backend-service
port:
number: 8080
- pathType: Prefix
path: /(.*)
backend:
service:
name: frontend-service
port:
number: 3000

Creating ingress resource

How do I create an ingress(ping) to expose a single service(hello) given a path (/hello )and a port (6789) in a given namespace (dev)?
the following is right? Also how to verify the same?
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: ping
namespace: dev
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /
spec:
rules:
- http:
paths:
- path: /hello
pathType: Prefix
backend:
service:
name: hello
port:
number: 6789
You might need to add the host into the ingress YAML if you are looking forward to use the domain for resolution like
hello-world.info forward the traffic to hello service.
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: example-ingress
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /$1
spec:
rules:
- host: hello-world.info
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: web
port:
number: 8080
to verify the changes you can use the Curl to check and test the endpoint also.
Once your YAML file is applied and ingress is created on cluster you can hit the endpoint and verify.
i would recommend checking out the part test your ingress :
https://kubernetes.io/docs/tasks/access-application-cluster/ingress-minikube/#test-your-ingress

K8S Ingress controller redirect to the wrong path

I have an Nginx ingress controller running on one k8s namespace,
and on another k8s namespace, I defined a pod, a service, and an ingress resource.
this is the ingress resource definition:
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: my-app-ingress
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /$2
spec:
ingressClassName: general-internal
rules:
- host: example.com
http:
paths:
- path: "/my-app(/|$)(.*)"
backend:
serviceName: my-app-svc
servicePort: 443
now, when I access this link:
http://example.com/my-app/some-path/
then everything is ok because the "my-app-svc" knows the path "/some-path/" and returns a 200 response (the forwarding is to http://my-app-svc/some-path and that's great because my-app-svc doesn't and shouldn't know or care for the /my-app prefix that exists only for the nginx ingress controller so it will know to forward that request to "my-app-svc" internally).
but when I access this link (notice no "/" in the end):
http://example.com/my-app/some-path
I get a redirection response from the "my-app-svc" service, and the "Location" header contains "/some-path/", so the redirection is to:
http://example.com/some-path/
which does not lead to the "my-app-svc" service because it doesn't have the "/my-app" prefix.
If the Location header was "/my-app/some-path/" instead of "/some-path/" then everything was ok because the redirection would be to:
http://example.com/my-app/some-path/
which would give me the 200 response.
the question is, how can I do that the ingress controller will add a "my-app" prefix to the Location header when it returns the redirection response to the client?
Thanks
Thanks to my co-worker, we found a solution for the problem:
The solution is to add these annotations to the ingess resource:
nginx.ingress.kubernetes.io/proxy-redirect-from: /
nginx.ingress.kubernetes.io/proxy-redirect-to: /my-app/
meaning:
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: my-app-ingress
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /$2
nginx.ingress.kubernetes.io/proxy-redirect-from: /
nginx.ingress.kubernetes.io/proxy-redirect-to: /my-app/
spec:
ingressClassName: general-internal
rules:
- host: example.com
http:
paths:
- path: "/my-app(/|$)(.*)"
backend:
serviceName: my-app-svc
servicePort: 443
It seems that the annotations above check the "Location" header in the redirection response to the client, and replace the first / with /my-app/ and only after this change - the redirection response is sent to the client.
For anyone landing here and finding the above solution not working (like myself), you might be using nginx-based ingress https://www.nginx.com/blog/guide-to-choosing-ingress-controller-part-4-nginx-ingress-controller-options/ which needs different annotation to your ingress resource:
nginx.org/rewrites: "serviceName=service1 rewrite=rewrite1[;serviceName=service2 rewrite=rewrite2;...]"
which worked for me.
A complete declaration from https://github.com/nginxinc/kubernetes-ingress/tree/v2.3.0/examples/rewrites copied here:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: cafe-ingress
annotations:
nginx.org/rewrites: "serviceName=tea-svc rewrite=/;serviceName=coffee-svc rewrite=/beans/"
spec:
rules:
- host: cafe.example.com
http:
paths:
- path: /tea/
pathType: Prefix
backend:
service:
name: tea-svc
port:
number: 80
- path: /coffee/
pathType: Prefix
backend:
service:
name: coffee-svc
port:
number: 80

Can I have both http and https in Ingress configuration?

I have ingress yaml like below which makes ingress to proxy HTTPS to HTTP connection. I'm confused how can I make this same ingress to process also HTTP to HTTP connection. Meaning I want it to use the same rule for both incoming HTTP or HTTPS. Removing tls portion makes it work with HTTP but adding it stops HTTP and makes it HTTPS only. Is it limitation of Kubernetes which prevents both HTTP and HTTPS routing in the same ingress controller?
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: ingress
annotations:
kubernetes.io/ingress.class: azure/application-gateway
spec:
tls:
- hosts:
- "*.mydomain.com"
secretName: aks-ingress-tls
rules:
- host: "*.mydomain.net"
http:
paths:
- pathType: Prefix
path: "/"
backend:
service:
name: frontend-service
port:
number: 80
You should redirect the HTTP request to your HTTPS listener. The requests hitting your Azure LB 80/443 listeners will be handled in the same way.
Due to Azure App gateway limitation, you cannot use a wildcard host in your ingress rules and you have to use workarounds.
See: https://azure.github.io/application-gateway-kubernetes-ingress/annotations/#ssl-redirect
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: ingress
annotations:
kubernetes.io/ingress.class: azure/application-gateway
appgw.ingress.kubernetes.io/ssl-redirect: "true"
spec:
tls:
- hosts:
- "*.mydomain.com"
secretName: aks-ingress-tls
rules:
- host: "www.mydomain.com"
http:
paths:
- pathType: Prefix
path: "/"
backend:
service:
name: frontend-service
port:
number: 80

Is it possible to use same hostname with multiple Ingress resources running in different namespaces?

I want to use the same hostname let's say example.com with multiple Ingress resources running in different namespaces i.e monitoring and myapp. I'm using Kubernetes nginx-ingress controller.
haproxy-ingress.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: haproxy-ingress
namespace: myapp
annotations:
kubernetes.io/ingress.class: "nginx"
spec:
tls:
- hosts:
# fill in host here
- example.com
rules:
- host: example.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: haproxy
port:
number: 80
grafana-ingress.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: grafana-ingress
namespace: monitoring
annotations:
nginx.ingress.kubernetes.io/use-regex: "true"
spec:
tls:
- hosts:
- example.com
rules:
- host: example.com
http:
paths:
# only match /grafana and paths under /grafana/
- path: /grafana(/|$)(.*)
pathType: Prefix
backend:
service:
name: grafana
port:
number: 3000
When I'm doing curl example.com then it is redirecting me to the deployment running in namespace one(as expected) but when I'm doing curl example.com/grafana then still it is redirecting me to namespace one deployment.
Please help.
Yes it is possible.
There can be two issues in your case.
One is you don't need the regex path for grafana ingress. Simple /grafana path will be fine with path type Prefix as with path type Prefix any /grafana/... will be redirected associated service. So the manifest file will be:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: grafana-ingress
namespace: monitoring
spec:
tls:
- hosts:
- example.com
rules:
- host: example.com
http:
paths:
- path: /grafana
pathType: Prefix
backend:
service:
name: grafana
port:
number: 3000
And the second issue can be the related service or deployment might not be the under same namespace monitoring. Please make sure the deployment/service/secret or other resources needed for grafana remains under the same namespace monitoring.