In the process of testing out Istio I'm in need of rewriting all incomming requests on the Istio ingress controller in the same manner as with Kubernetes's own ingress controller, where I use the rewrite-target annotation.
# Existing Kubernetes ingress configuration
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: api
annotations:
ingress.kubernetes.io/rewrite-target: /
spec:
rules:
- host: some.host.com
http:
paths:
- path: /svc
backend:
serviceName: svc
servicePort: 80
This makes all requests, e.g. http://some.host.com/svc/endpoint hit the service with /endpoint instead of /svc/endpoint
How can I effectively do the same with Istio's ingress controller?
The Route rules object can handle rewrites, but is only available as a per destination manner, i.e. I have to create a route rule for every service.
You are right. You need a route rule per service to do setup the rewrite targets. See https://istio.io/docs/reference/config/traffic-rules/routing-rules.html#httprewrite for an example.
Related
i am looking at creating an alb using https://github.com/kubernetes-sigs/aws-load-balancer-controller
Lets say i have two namespaces kubernetes-dashboard and otherns.
In the first namespace i have a service called kubernetes-dashboard and in the second namespace i have a service called otherservice
Would the below ingress work?
ingress.yml
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: general-ingress
namespace: default
annotations:
kubernetes.io/ingress.class: alb
alb.ingress.kubernetes.io/scheme: internet-facing
alb.ingress.kubernetes.io/listen-ports: '[{"HTTP":80,"HTTPS": 443}]'
alb.ingress.kubernetes.io/certificate-arn: <redacted>
alb.ingress.kubernetes.io/tags: Environment=staging,Team=dev
alb.ingress.kubernetes.io/healthcheck-path: /health
alb.ingress.kubernetes.io/healthcheck-interval-seconds: '300'
spec:
rules:
- host: k8s.acme.com
http:
paths:
- path: /*
backend:
serviceName: kubernetes-dashboard.kubernetes-dashboard
servicePort: 8080
- host: otherservice.acme.com
http:
paths:
- path: /*
backend:
serviceName: otherservice.otherns
servicePort: 80
I found there is two solutions to this problem.
The second solution did not work for me because i am using a library which does not support that. https://github.com/kubernetes-sigs/aws-load-balancer-controller
Create multiple ingress files delcaring the same group.name.
https://kubernetes-sigs.github.io/aws-load-balancer-controller/latest/guide/ingress/annotations/#ingressgroup
IngressGroup feature enables you to group multiple Ingress resources
together. The controller will automatically merge Ingress rules for
all Ingresses within IngressGroup and support them with a single ALB.
In addition, most annotations defined on a Ingress only applies to the
paths defined by that Ingress.
Use an externalName service
https://kubernetes.io/docs/concepts/services-networking/service/#externalname
You can create an ExternalName service in the same namespace as your ingress, your ingress will then point to this special ExternalName service
Namespaces are security boundaries. An Ingress resource in a namespace cannot direct traffic to a service in a different namespace. If that would be possible, users could hijack traffic to services.
In the first namespace i have a service called kubernetes-dashboard and in the second namespace i have a service called otherservice
What you need to do in this case is to use two different Ingress-resources. You can still use the same Application Load Balancer.
I have a deployment which is WEB API. I apply it to Kubernetes. Then add service.yml file to expose it. It is working. I have 12 microservices. All of them have service and deployment .yaml files.
So what is Ingress controller. Why should I use it?
(Except sidecar proxy like istio. Its perfect for resilince and metrics.)
In order for the Ingress resource to work, the cluster must have an ingress controller running. The controller can be for example Nginx Ingress Controller and it can be adjusted in various ways. After you have deployed the controller, all you need is an Ingress resource object deployed in K8s.
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: ingress-sample
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /
spec:
rules:
- http:
paths:
- path: /api
pathType: Prefix
backend:
service:
name: web-api
port:
number: 80
service.name: web-api is the place where you specify link between the ingress and the service.
I have two ingress controllers (for public/internal traffic), what I would like is for all endpoints to use the public ingress except for /metrics, which should be internal, all using the same host.
E.g.
example.com/ -> public ingress
example.com/metrics -> internal ingress
This is what I have tried:
internal ingress
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: example-metrics-ingress
annotations:
kubernetes.io/ingress.class: ingress-internal
spec:
rules:
- host: example.com
http:
paths:
- path: /metrics
backend:
serviceName: example-servicename
servicePort: 80
and public ingress
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: example-ingress
annotations:
kubernetes.io/ingress.class: nginx
spec:
rules:
- host: example.com
http:
paths:
- path:
backend:
serviceName: example-servicename
servicePort: 80
The internal ingress is currently being ignored when I visit example.com/metrics (it uses the public one instead).
If I change the internal ingress to use the same ingress controller as the public one and change the service port to 81 (as an example), this provides an error (which is expected), this shows that the two different ingresses are being used. However, as soon as I use two different ingress controllers, then the one ingress' rules are not being picked up.
How can I configure my ingresses to achieve my desired result?
When running multiple ingress-nginx controllers, it will only process an unset class annotation if one of the controllers uses the default --ingress-class value (see IsValid method in internal/ingress/annotations/class/main.go), otherwise the class annotation become required.
If --ingress-class is set to the default value of nginx, the controller will monitor Ingresses with no class annotation and Ingresses with annotation class set to nginx. Use a non-default value for --ingress-class, to ensure that the controller only satisfied the specific class of Ingresses.
In your case use the combination of the annotation kubernetes.io/ingress.class: "EXTERNAL|INTERNAL" and the flag --ingress-class=EXTERNAL|INTERNAL allows you to filter which Ingress rules should be picked by the nginx ingress controller.
Take a look: multiple-ingress, ingress-nginx-traffic.
I have had an issue like that on AKS (K8s version 1.22.4). I have two Nginx Ingress Controllers, Internal and External.
Only one worked at a time, Internal or external.
After specifying a unique election-id for each one the problem was fixed.
The following post may help: https://stackoverflow.com/a/72591382/4049017
I created a service and each service is creating a new load balancer, I don't want to create a new load balancer for each service. For that, I found solution ingress controller but it's not happening.
I will try to describe the objects you need in just words.
You don't need to create a load balancer for each service. When you're using an ingress controller (like nginx), the ingress controller itself will be the type load balancer. All your other services need to be something like ClusterIP type.
Afterwards you can decide how to link your ClusterIP services with the Nginx LoadBalancer: create an ingress for each service or one ingress that exposes each service based on some rule (like paths as #harsh-manvar shows in the post above).
When you say "it's not happening", it would be good if you could provide details on your setup.
In order for Nginx ingress controller to work, it needs to be defined either as a NodePort or LoadBalancer service type. The examples provided in the nginx documentation are using LoadBalancer. However, LoadBalancer only works when your cluster supports this object (that means running in most cloud providers like AWS/GCP/Azure/DigitalOcean or newer versions of minikube). On the other hand, NodePort will expose the ingress controller on the Kubernetes node where it runs (when using minikube, that usually means a VM of sorts which then needs to be port forwarded to be accessible).
To use ingress in a local environment, you can look into minikube. All you need is to run minikube addons enable ingress and it will deploy an nginx controller for you. Afterwards, all you need to do is define an ingress and depending on your setup you may need to use kubectl port-forward to port forward port 80 on an nginx controller pod to a local port on your machine.
There are different types of services: ClusterIP, NodePort, LoadBalancer and ExternalName. You can specify it in spec.type. Actually the default one, when not specified is not LoadBalancer, but ClusterIP, so in your case, simply leave away the type: LoadBalancer definition and use your serviceName as backend in your ingress resource. Example:
spec:
rules:
- host: your.fully.qualified.host.name
http:
paths:
- backend:
serviceName: your-internal-service-name
servicePort: 80
path: /
Keep in mind that for some cloud providers there's also the possibility to use an internal LoadBalancer without a public IP. This is done by adding an annotation to the service configuration. For Azure AKS it looks like this:
metadata:
annotations:
service.beta.kubernetes.io/azure-load-balancer-internal: "true"
For Google's GKE the annotation is cloud.google.com/load-balancer-type: "Internal"
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: ingress
annotations:
kubernetes.io/ingress.class: "nginx"
certmanager.k8s.io/cluster-issuer: wordpress-prod
nginx.ingress.kubernetes.io/rewrite-target: /
spec:
tls:
- hosts:
- test.test.com
secretName: prod
rules:
- host: test.test.com
http:
paths:
- path: /service-1
backend:
serviceName: service-1
servicePort: 80
- path: /service-2
backend:
serviceName: service-2
servicePort: 5000
Sharing here documentation for ingress to target multiple services you can redirect to multi-service.
Using this you can access services like
https://test.test.com/service-1
https://test.test.com/service-2
Following documentation you should do the following.
More information: kubernetes.github.com
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /$2
name: rewrite
namespace: default
spec:
rules:
- host: rewrite.bar.com
http:
paths:
- backend:
serviceName: http-svc
servicePort: 80
path: /something(/|$)(.*)
For example, the ingress definition above will result in the following rewrites:
rewrite.bar.com/something rewrites to rewrite.bar.com/
rewrite.bar.com/something/ rewrites to rewrite.bar.com/
rewrite.bar.com/something/new rewrites to rewrite.bar.com/new
I saw some example where the Kubernetes cluster is installed with ingress controller and then the ingress class is added with annotations and host as below.
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
annotations:
kubernetes.io/ingress.class: nginx
spec:
rules:
- host: testsvc.k8s.privatecloud.com
http:
I am not sure which service is installed and which IP is configured with the DNS "k8s.privatecloud.com" so as to route requests?
How the DNS routing "k8s.privatecloud.com" routes requests to Kubernetes cluster? How the ingress to kubernetes bridging works?
Also, There could be many services configured with the hosts rule like,
testsvc.k8s.privatecloud.com
testsvc1.k8s.privatecloud.com
testsvc2.k8s.privatecloud.com
How the subdomain routing works here when we hit the service testsvc.k8s.privatecloud.com or testsvc1.k8s.privatecloud.com ...
Thanks
The DNS for all the hostnames in your given example (e.g. testsvc.k8s.privatecloud.com) would point to the machine or load-balancer through which traffic will reach the Ingress controller's nginx, as is described in the kuberetes Ingress documentation
Subdomain routing is traditionally done via "virtual-hosting", sometimes called "v-host-ing", and the nginx ingress uses the HTTP Host: header to know which backend service should receive that traffic. Some Ingress controllers are able to use SNI for that same trick over https.
In addition to #Matthew L Daniel answer.
The kubernetes Ingress works as a proxy between external network and your cluster. The behavior of the ingress is explained in the object ingress. For example:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: test
annotations:
ingress.kubernetes.io/rewrite-target: /
spec:
rules:
- host: foo.bar.com
http:
paths:
- path: /foo
backend:
serviceName: s1
servicePort: 80
- path: /bar
backend:
serviceName: s2
servicePort: 80
Above it`s explained how to route traffic between 2 backends s1 and s2. Ingress does not hold any information about services except its name and port, every time it needs more details it would need to be requested from the api-server.