Multiple deployments in single service - kubernetes

My backend has 5 different deployments, each serving the request in a specific port.
Frontend service contacts the backend service with the necessary details. The backend service should be able to decide the required pod (from one of the 5 deployments) to serve the request.
Instead of creating 5 different services, is it possible to link a single service to multiple deployments?
Kubernetes version: 1.12
Cloud being used: Amazon EKS
PS: My requirement is different from https://github.com/kubernetes/kubernetes/issues/24875
Similar question is unanswered: "Wire" multiple Deployments to one Service in Kubernetes

The exact answer to your quest is: it is not possible today.
As you have correctly seen in the issue and in the question (both facing the same situation) this could be a future implementation.
A possible solution/workaround is to delegate the problem to an upper layer but basically it depends on the situation and different services are always required.
Assuming that your deployments are 5 different application that do different things (otherwise why 5 different deployment and not 1 with n replicas?) and assuming they are http applications, you can use the ingress resource to ruote the traffic to the right deployment/service (assuming one service per deployment).
If your 5 deployment are created/updated/managed together (eg: are all in the same helm deployment) you can create a fanout ingress:
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: simple-fanout-example
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /
spec:
rules:
- host: foo.bar.com
http:
paths:
- path: /foo
backend:
serviceName: service1
servicePort: 4200
- path: /bar
backend:
serviceName: service2
servicePort: 8080
- path: /aaaa
backend:
serviceName: service3
servicePort: 4200
- path: /bbbbb
backend:
serviceName: service4
servicePort: 8080
- path: /ccc
backend:
serviceName: service5
servicePort: 8080
Or, if your 5 deployment are separated you can create 5 Different ingress serources with the same idea:
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: ingress-for-app-1
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /
spec:
rules:
- host: foo.bar.com
http:
paths:
- path: /foo
backend:
serviceName: service1
servicePort: 4200
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: ingress-for-app-1
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /
spec:
rules:
- host: foo.bar.com
http:
paths:
- path: /bar
backend:
serviceName: service2
servicePort: 8080
and so on....
Creating 5 ingress or 1 fanout should produce the same result.
This approach works well with a nginx ingress controller but pay attention only to two things
path match: with the nginx controller version > 0.22 nginx.ingress.kubernetes.io/rewrite-target: / is an exact match. For example, if you wold like to redirect from /foo to / preserving all the uri after /foo (/foo/something?parameter=parameter_value to /something?parameter=parameter_value) your ingress rewrite rule should be like this:
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: ingress-for-app-1
annotations:
nginx.ingress.kubernetes.io/rewrite-target: "/$1"
spec:
rules:
- host: foo.bar.com
http:
paths:
- path: /foo/(.*)
backend:
serviceName: service1
servicePort: 4200
conflict route: avoid conflicting rout, eg path: /foo/(.*) and path: /foo/bar/(.*) where a request for /foo/bar/something would match both the paths. The behavior could be difficult to predict and, if it worked as expected, it would not be stable

Related

GKE ingress; apply rewrite to specific service

Given this yaml:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: test-ingress-2
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /$1
spec:
rules:
- host: test.com
http:
paths:
- backend:
serviceName: admin
servicePort: 8080
path: /admin/*
pathType: ImplementationSpecific
- backend:
serviceName: keycloak
servicePort: 8080
path: /auth/*
pathType: ImplementationSpecific
I would like the rewrite-target to only apply to the service admin. Requests to keycloak should not be affected. How can I achieve that?
You can separate out ingress file or config, just make sure you keep the different name
So you can you create TWO ingress
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: test-ingress-1
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /$1
spec:
rules:
- host: test.com
http:
paths:
- backend:
serviceName: admin
servicePort: 8080
path: /admin/*
pathType: ImplementationSpecific
Ingress: 2 with different config, you can edit anything as per need now remove annotation.
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: test-ingress-2
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /$1
spec:
rules:
- host: test.com
http:
paths:
- backend:
serviceName: keycloak
servicePort: 8080
path: /auth/*
pathType: ImplementationSpecific
If you want to store everything in a single YAML file you can also do it by merging them with ---.
User Harsh Manvar's answer is good and shows the correct solution. But I am going to expand it a bit and tell you why. Always create deployments that are as little interdependent as possible. For example, if you need to change something in the keycloak, it shouldn't have any effect on admin. Additionally, if the ingress breaks down, you will have 2 broken services instead of one. You should practically always create one ingress to one service.
Additionally, you may want to create completely different rewrite rules. Then, creating separate ingress will be a very good idea. Look at this question.
See also documentation about rewrite rules.

Do I have to define an ingress per service with Linkerd?

Looking at the linkerd ingress documentation here it says that I need to create an ingress with an annotation of
ingress.kubernetes.io/custom-request-headers: l5d-dst-override:web-svc.emojivoto.svc.cluster.local:80
this annotation is specific to a single service, which makes it sound like there must be a new ingress with it's own annotation for every service. I couldn't have something like the following for example:
spec:
rules:
- host: example.com
http:
paths:
- path: /path-one
backend:
serviceName: service-1
servicePort: 80
- path: /path-two
backend:
serviceName: service-2
servicePort: 80
where I could define paths to different services in a single ingress class.
Is my reading of these docs accurate? or am I missing something? I am hoping to avoid creating an ingress for every service I run in linkerd.
Yes, unfortunately you understood correctly about creating separate ingress for each service if you want use ingress.kubernetes.io/custom-request-headers.
Yes, if you would have 1000 services - you should create 1000 ingresses to make it work properly.
Ingress1:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: web-ingress
namespace: marcus
annotations:
kubernetes.io/ingress.class: "traefik"
ingress.kubernetes.io/custom-request-headers: l5d-dst-override:service1.marcus.svc.cluster.local:80
spec:
rules:
- host: example.com
http:
paths:
- backend:
serviceName: service1
servicePort: 80
path: /
Ingress2:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: web-ingress
namespace: marcus
annotations:
kubernetes.io/ingress.class: "traefik"
ingress.kubernetes.io/custom-request-headers: l5d-dst-override:service2.marcus.svc.cluster.local:80
spec:
rules:
- host: example.com
http:
paths:
- backend:
serviceName: service2
servicePort: 80
path: /
Traefik is a great solution, and in this case it would be great if it had the option to dynamically set the service in a header.
There is an open issue on this in the traefik project that has been open for a while. The last update is to use an Ingress per service in these scenarios.
Here's similar question.

same path with different service names in ingress resources

I'm having some issues when using a path to point to different service names, my ingress resource looks as below
nginx-static service is a nginx container which has static content... I have to load this static content while calling service-1, since both nginx-static and service-1 but I cannot keep the sme same host path.... Please suggest how to correct the below ingress resources...
kindly note static content has lot of files(csv,js,html,directories, files etc)
kind: Ingress
metadata:
name: myingress
annotations:
kubernetes.io/ingress.class: "nginx"
nginx.ingress.kubernetes.io/ssl-redirect: "false"
spec:
rules:
- http:
paths:
- path: "/"
backend:
serviceName: nginx-static
servicePort: 80
- path: "/"
backend:
serviceName: service1
servicePort: 8989
- path: "/test1"
backend:
serviceName: service2
servicePort: 9001
Any expert help is appreciated!!!
You cannot have the same path pointing to different backend resources. You have to put either your static files or the service into a different path, and rewrite the URL, for instance:
rewrite annotation:
metadata:
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /$2
name: rewrite
namespace: default
paths:
- backend:
serviceName: nginx-statix
servicePort: 80
path: /static(/|$)(.*)
- backend:
path: /
serviceName: service1
With this, your static content will be exposed under /static/, and all /static/name will be rewritten as /name by the ingress.
More info here: https://kubernetes.github.io/ingress-nginx/examples/rewrite/
Unfortunately, requirements from the initial questions aren't' clear and there were no additional clarifications given. However, I'd like to elaborate on Burak Serdar's answer and add , that Kubernetes Ingress allows you listing the same path ad port for a multiple services under condition that you are listing these for different hosts.
foo.bar.com --| |-> foo.bar.com nginx-static:80
| 178.91.123.132 |
bar.foo.com --| |-> bar.foo.com service1:8989
you can achieve that scenario with the following config:
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: name-virtual-host-ingress
spec:
rules:
- host: foo.bar.com
http:
paths:
- backend:
serviceName: nginx-static
servicePort: 80
- host: bar.foo.com
http:
paths:
- backend:
serviceName: service2
servicePort: 8989
However that would work only if you can split your web site between two hosts.
Hope that helps.
You can check more on Ingress on K8s documentation.

how to configure ingress to direct traffic to an https backend using https

I have a backend using https.
I want to separate load on that back-end based on URL/path.
I decided to use ingress to do this url/path based logic in order to move traffic to different back-ends ( same back-ends , just duplicated to different NodePorts )
my question is how I can configure the ingress to receive https requests and to forward those https requests to the https back-end?
thanks
edit:
I added the yaml file:
spec:
rules:
- http:
paths:
- backend:
serviceName: service
servicePort: 9443
path: /carbon
- backend:
serviceName: service2
servicePort: 9443
path: /oauth
for some reason I can;t change the rule form http to https
Attention: This answer applies to the ingress-nginx solution provided by the kubernetes organisation on github (https://github.com/kubernetes/ingress-nginx)
If you want to use load balancing mechanisms in k8s you should use services instead and start multiple instances behind that service that way k8s will do the load balancing. If you want to use different versions of your backend (e.g. prod and test) your way of separating them is fine
if your service is only reachable via https you need to add the following annotation to your ingress yaml: (documentation)
nginx.ingress.kubernetes.io/backend-protocol: "HTTPS"
To secure ingress itself take a look at this: https://kubernetes.io/docs/concepts/services-networking/ingress/#tls
But if you want that the backend services decrypt the TLS communication use the following annotation instead: (documentation)
nginx.ingress.kubernetes.io/ssl-passthrough: "true"
Edit:
The Ingress YAML should look like this if you want to reach the backend via TLS:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: ingress-name
namespace: namespace-name
annotations:
nginx.ingress.kubernetes.io/backend-protocol: "HTTPS"
spec:
rules:
- http:
paths:
- backend:
serviceName: service
servicePort: 9443
path: /carbon
- backend:
serviceName: service2
servicePort: 9443
path: /oauth
The Ingress YAML should look like this if you want to reach the backend via TLS with TLS decryption in the ingress controller:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: ingress-name
namespace: namespace-name
annotations:
nginx.ingress.kubernetes.io/backend-protocol: "HTTPS"
spec:
tls:
- hosts:
- app.myorg.com
secretName: tls-secret
rules:
- http:
paths:
- backend:
serviceName: service
servicePort: 9443
path: /carbon
- backend:
serviceName: service2
servicePort: 9443
path: /oauth
It's important to note that tls-secret is the name of a SecretConfig with a valid Certificate issued for the host (app.myorg.com)
The Ingress YAML should look like this if you want to reach the backend via TLS with TLS decryption in the backend:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: ingress-name
namespace: namespace-name
annotations:
nginx.ingress.kubernetes.io/ssl-passthrough: "true"
spec:
rules:
- http:
paths:
- backend:
serviceName: service
servicePort: 9443
path: /carbon
- backend:
serviceName: service2
servicePort: 9443
path: /oauth
I never tested the last version myself so i don't know if that actually works but I'd strongly advise reading this passage for that variant.
If you are using the NGINX Ingress controller (https://docs.nginx.com/nginx-ingress-controller/), the nginx.ingress.kubernetes.io/backend-protocol: "HTTPS" does not work. However, the nginx.org/ssl-services will let you pick the services that require TLS on the backend. The name is confusing, so it took me a while to realize the real purpose of it.
This does not work with the standard Kubernetes Ingress controller that uses NGINX under the hood; it only works with the NGINX-sourced controller.
Advanced annotation docs: https://docs.nginx.com/nginx-ingress-controller/configuration/ingress-resources/advanced-configuration-with-annotations/
In this example, NGINX will connect to the ssl-svc using TLS; it ignores any self-signed certificates. Example (https://github.com/nginxinc/kubernetes-ingress/tree/v1.12.0/examples/ssl-services):
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: cafe-ingress
annotations:
nginx.org/ssl-services: "ssl-svc"
spec:
rules:
- host: cafe.example.com
http:
paths:
- path: /tea
backend:
serviceName: tea-svc
servicePort: 80
- path: /coffee
backend:
serviceName: coffee-svc
servicePort: 80
- path: /ssl
backend:
serviceName: ssl-svc
servicePort: 443

How to specify a prefix to a service exposed with an ingress

I want exposing various services with a single ingress.
rules:
- http:
paths:
# The path is the URL prefix for the service, e.g. /api/* or just /*
# Note that the service will receive the entire URL with the prefix
- path: /service1/*
backend:
serviceName: service1
servicePort: 5000
- path: /service2/*
backend:
serviceName: service2
servicePort: 5000
The problem is the whole URL including the prefix is passed to the underlying services so all requests return 404 errors: service1 and api don't respond on /service1/some/path but directly on /some/path
How can I specify a prefix to the underlying services?
UPDATE
I tried using rewrite-target as follows. Requests are sent to the rasa-nlu service, but they all trigger 404 because rasa-nlu still gets the /nlu
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: ingress
annotations:
ingress.kubernetes.io/rewrite-target: /
spec:
rules:
- http:
paths:
- path: /nlu
backend:
serviceName: rasa-nlu
servicePort: 5000
This might be what you are looking for;
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
annotations:
ingress.kubernetes.io/rewrite-target: /
name: rewrite
namespace: default
spec:
rules:
- host: rewrite.bar.com
http:
paths:
- backend:
serviceName: echoheaders
servicePort: 80
path: /something
Note the annotation to rewrite-target.
Found this here
This thread might be solved till now, but just for the sake of solution.
below will solve the issue, the default path will be /nlu when it is added to the re-write annotation.
It is from the nginx re-write rule which gets applies to the definition of location directive.
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: ingress
annotations:
ingress.kubernetes.io/rewrite-target: /nlu
spec:
rules:
- http:
paths:
- path: /nlu
backend:
serviceName: rasa-nlu
servicePort: 5000