I created an ingress service from the kubernetes documentation, but I found that not putting below annotation made service unavailable. Why was this so?
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /
nginx.ingress.kubernetes.io/ssl-redirect: "false"
Main reason to ask this question was that in K8s documentation, I found below code for ingress, but this did not work until and unless I put the above annotation. So, why does the below code not work?
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: ingress-wear-watch
namespace: app-space
spec:
rules:
- http:
paths:
- path: /wear
pathType: Prefix
backend:
service:
name: wear-service
port:
number: 8080
- path: /watch
pathType: Prefix
backend:
service:
name: video-service
port:
number: 8080
Based on k8s docs, all you need to know in this specific case is that
you can use Kubernetes annotations to attach arbitrary non-identifying metadata to objects. Clients such as tools and libraries can retrieve this metadata.
[annotations are] Directives from the end-user to the implementations to modify behavior or engage non-standard features.
So basically this specific annotations are used to modify behaviour of ingress and are specififc to nginx ingress controller (this means that if you use different controller, this annotations wont work).
All supported annotations for nginx ingress controler are explained in Nginx Ingress Controller documention.
Related
I have a simple website running in my Kubernetes cluster and exposed to the Internet using Traefik. My Ingress object looks like this (the only things I've changed here are the name and domain names):
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
annotations:
cert-manager.io/cluster-issuer: letsencrypt-staging
kubernetes.io/ingress.class: traefik
name: my-cool-website
namespace: default
spec:
rules:
- host: my-cool-website.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: my-cool-website
port:
number: 80
- host: www.my-cool-website.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: my-cool-website
port:
number: 80
tls:
- hosts:
- my-cool-website.com
- www.my-cool-website.com
secretName: my-cool-website-tls
This works. It allows me to access the site either from my-cool-website.com or from www.my-cool-website.com. But what I'd like to have happen is that if someone visits the former, that Traefik automatically redirects them to the latter. I found a couple of guides online that recommended creating a Traefik middleware, so I did just that, but unfortunately it doesn't work as intended. Here is my middleware definition:
apiVersion: traefik.containo.us/v1alpha1
kind: Middleware
metadata:
name: my-cool-website-force-www
spec:
redirectRegex:
regex: ^https?://my-cool-website.com/(.*)
replacement: https://www.my-cool-website.com/${1}
And then I add the following annotation back to the ingress object:
traefik.ingress.kubernetes.io/router.middlewares: my-cool-website-force-www
But as soon as I do that, it breaks my web app. By that I mean, when that annotation is applied, instead of serving my website, I start seeing a generic nginx page that looks like this when I try to access the domain (and also it does not do the redirect):
I have to assume this Hello World page is being served by Traefik as some sort of generic default page, as it definitely does not originate from my pod. So that tells me something about this middleware (or perhaps how I'm calling it with the annotation) isn't working. What am I missing?
I figured it out, by port-forwarding to the Traefik Dashboard and looking at the service there. It was showing an error for the middleware not found. I then clicked over to the middlewares and realized that they end up with a longer canonical name. So in my annotation I had to change the reference from my-cool-website-force-www to default-my-cool-website-force-www#kubernetescrd and then everything worked.
I'm trying to setup a simple ingress with path rewriting to pass requests to my backend services.
Ref.: https://haproxy-ingress.github.io/v0.10/docs/configuration/keys/#rewrite-target
The ingress controller uses this image: quay.io/jcmoraisjr/haproxy-ingress:v0.10-beta.1.
Here is sample YAML:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: "myapp-apis-ingress"
namespace: "my-namespace"
labels:
app: myapp
tier: ingress
annotations:
kubernetes.io/ingress.class: "haproxy"
haproxy.org/rewrite-target: "/"
spec:
rules:
- host: "myapp.mydomain"
http:
paths:
- path: /api/v1/hello
pathType: Prefix
backend:
service:
name: "myapp-hello-svc"
port:
number: 8080
Expected behaviour:
It should route requests from https://myapp.mydomain/api/v1/hello/* to the GKE service at myapp-hello-svc:8080/*
Actual behaviour:
It routes everything to myapp-hello-svc:8080/api/v1/hello/* (myapp-hello-svc pod receives GET /api/v1/hello/*
I've tried some other combinations and rules, but none seemed working neither.
Any ideas what I may have missed here?
Thanks!
UPDATE: WORKAROUND
As currently I still can't find solution to this, I decided to use workaround by adding NGINX ingress controller to K8S cluster and routing the traffic for new APIs through it instead.
I created a service and each service is creating a new load balancer, I don't want to create a new load balancer for each service. For that, I found solution ingress controller but it's not happening.
I will try to describe the objects you need in just words.
You don't need to create a load balancer for each service. When you're using an ingress controller (like nginx), the ingress controller itself will be the type load balancer. All your other services need to be something like ClusterIP type.
Afterwards you can decide how to link your ClusterIP services with the Nginx LoadBalancer: create an ingress for each service or one ingress that exposes each service based on some rule (like paths as #harsh-manvar shows in the post above).
When you say "it's not happening", it would be good if you could provide details on your setup.
In order for Nginx ingress controller to work, it needs to be defined either as a NodePort or LoadBalancer service type. The examples provided in the nginx documentation are using LoadBalancer. However, LoadBalancer only works when your cluster supports this object (that means running in most cloud providers like AWS/GCP/Azure/DigitalOcean or newer versions of minikube). On the other hand, NodePort will expose the ingress controller on the Kubernetes node where it runs (when using minikube, that usually means a VM of sorts which then needs to be port forwarded to be accessible).
To use ingress in a local environment, you can look into minikube. All you need is to run minikube addons enable ingress and it will deploy an nginx controller for you. Afterwards, all you need to do is define an ingress and depending on your setup you may need to use kubectl port-forward to port forward port 80 on an nginx controller pod to a local port on your machine.
There are different types of services: ClusterIP, NodePort, LoadBalancer and ExternalName. You can specify it in spec.type. Actually the default one, when not specified is not LoadBalancer, but ClusterIP, so in your case, simply leave away the type: LoadBalancer definition and use your serviceName as backend in your ingress resource. Example:
spec:
rules:
- host: your.fully.qualified.host.name
http:
paths:
- backend:
serviceName: your-internal-service-name
servicePort: 80
path: /
Keep in mind that for some cloud providers there's also the possibility to use an internal LoadBalancer without a public IP. This is done by adding an annotation to the service configuration. For Azure AKS it looks like this:
metadata:
annotations:
service.beta.kubernetes.io/azure-load-balancer-internal: "true"
For Google's GKE the annotation is cloud.google.com/load-balancer-type: "Internal"
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: ingress
annotations:
kubernetes.io/ingress.class: "nginx"
certmanager.k8s.io/cluster-issuer: wordpress-prod
nginx.ingress.kubernetes.io/rewrite-target: /
spec:
tls:
- hosts:
- test.test.com
secretName: prod
rules:
- host: test.test.com
http:
paths:
- path: /service-1
backend:
serviceName: service-1
servicePort: 80
- path: /service-2
backend:
serviceName: service-2
servicePort: 5000
Sharing here documentation for ingress to target multiple services you can redirect to multi-service.
Using this you can access services like
https://test.test.com/service-1
https://test.test.com/service-2
Following documentation you should do the following.
More information: kubernetes.github.com
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /$2
name: rewrite
namespace: default
spec:
rules:
- host: rewrite.bar.com
http:
paths:
- backend:
serviceName: http-svc
servicePort: 80
path: /something(/|$)(.*)
For example, the ingress definition above will result in the following rewrites:
rewrite.bar.com/something rewrites to rewrite.bar.com/
rewrite.bar.com/something/ rewrites to rewrite.bar.com/
rewrite.bar.com/something/new rewrites to rewrite.bar.com/new
I have a pod that has following chart:
ingress:
enabled: true
annotations:
kubernetes.io/ingress.class: nginx
// ...
paths:
- /api/myservice
My pod exposes api and additionally a /prometheus endpoint that is accessible through /api/myservice/prometheus.
I would like to have prometheus visible inside my cluster but not from /api/myservice/prometheus. How I can achieve that?
You can add an Ingress rule that redirects the endpoint to the default-backend:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: block
annotations:
ingress.kubernetes.io/ssl-redirect: "true"
ingress.kubernetes.io/rewrite-target: /
spec:
tls:
- hosts:
- myapp.to
rules:
- host: myapp.to
http:
paths:
- path: /metrics
backend:
serviceName: ingress-default-backend
servicePort: 8080
If you just need internal access, I don't think you should put it into ingress. You could make use of the DNS Service together with Kubernetes cluster already.
If your prometheus service is running the same namespace as your working pod, you could use following address to access it.
http://prometheus-svc-name
If your prometheus service is running in a different namespace, you could use:
http://prometheus-svc-name.prometheus-namespace.svc.cluster.local
to access the service.
Update:
More clear about the question after comments.
You could write another rule to hide your /prometheus endpoint like this:
ingress:
enabled: true
annotations:
kubernetes.io/ingress.class: nginx
// ...
paths:
- /api/myservice => myservice
- /api/myservice/prometheus => 404 default backend.
nginx-ingress will match the longest route first. It will route /api/myservice/prometheus to 404. See this documentation
Best way to achieve this would be to remove the prometheus endpoint from ingress and if you are aware prometheus supports auto discovery using annotations on the service, configure prometheus for k8s auto discovery and then add annotations in you service yaml file.
https://prometheus.io/docs/prometheus/latest/configuration/configuration/#kubernetes_sd_config
change the endpoint to point to your API.
/api/myservice/my-api or use app-root annotation, to point to your api inside /api/myservice:
nginx.ingress.kubernetes.io/app-root: /path/to/my/app/root/dir
more details here: https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/annotations/#rewrite
I'm moving to kubernetes using traefik as my Ingress Controller.
I have a single backend that should respond to 3000+ websites. Depending on the host, I need to add a custom header to the request before proxy passing it to the backend.
I can use the ingress.kubernetes.io/custom-request-headers annotation to add a custom header to the request but it's an annotation for the whole Ingress, so I would need to create 3000+ Ingresses, one for each website.
Is there another way to do this? Creating 3000+ Ingresses is the same thing as creating one Ingress with 3000+ rules?
Yes, you need to create one Ingress object per one host, if you want different headers her host.
You can do it by Traefik:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: traeffic-custom-request-header
annotations:
ingress.kubernetes.io/custom-request-headers: "mycustomheader: myheadervalue"
spec:
rules:
- host: custom.configuration.com
http:
paths:
- backend:
serviceName: http-svc
servicePort: 80
path: /
Also, the same thing you can do by Nginx Ingress Controller.
It has the support for configuration snipper. Here is an example of using it to set a custom header per Ingress object:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: nginx-configuration-snippet
annotations:
nginx.ingress.kubernetes.io/configuration-snippet: |
more_set_headers "Request-Id: $request_id";
spec:
rules:
- host: custom.configuration.com
http:
paths:
- backend:
serviceName: http-svc
servicePort: 80
path: /
BTW, you can use several different ingress controllers on your cluster, so it does not need to migrate everything to only one type of Ingress.