My Kubernetes ingress has the following:
ingress:
enabled: true
annotations:
kubernetes.io/ingress.class: alb
...
...
# -- Defines which ingress controller will implement the resource
ingressClassName: ""
I'm a bit confused about the difference between "kubernetes.io/ingress.class" and "ingressClassName.".
I believe "kubernetes.io/ingress.class" needs to match the ingressClass as defined in the AWS ALB Ingress Controller.
However, then I'm confused about the role of "ingressClassName"? As that seems to be the same thing.
Any clarity would be appreciated.
Actually, both refer to the same thing, but kubernetes.io/ingress.class is deprecated from Kubernetes v1.22+. and ingressClassName is introduced in 1.18, so if you are using higher version you can ingressClassName
Before the IngressClass resource and ingressClassName field were added in Kubernetes 1.18, Ingress classes were specified with a kubernetes.io/ingress.class annotation on the Ingress. This annotation was never formally defined, but was widely supported by Ingress controllers.
The newer ingressClassName field on Ingresses is a replacement for that annotation, but is not a direct equivalent. While the annotation was generally used to reference the name of the Ingress controller that should implement the Ingress, the field is a reference to an IngressClass resource that contains additional Ingress configuration,
ingress-deprecated-annotation
Your example is
ingress:
enabled: true
annotations:
kubernetes.io/ingress.class: alb
its equivalent value is className: "nginx"
ingress:
enabled: true
className: "alb" -> ingressClassName
if you check the ingress template, it will be like this
ingressClassName: {{ .Values.ingress.className }}
Related
I'm migrating an architecture to kubernetes and I'd like to use the Haproxy ingress controller that I'm installling with helm, according the documentation (version 1.3).
Thing is, when I'm defining path rules through an ingress file, I can't define Regex or Beging path types as seen on documentation here : https://haproxy-ingress.github.io/docs/configuration/keys/#path-type.
My configuration file :
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: bo-ingress
annotations:
haproxy.org/path-rewrite: "/"
kubernetes.io/ingress.class: haproxy
spec:
rules:
- host: foo.com
http:
paths:
- path: /
pathType: Beging
backend:
service:
name: foo-service
port:
number: 80
When I install my ingress helm chart with this configuration, I have that error message :
Error: UPGRADE FAILED: cannot patch "bo-ingress" with kind Ingress: Ingress.extensions "bo-ingress" is invalid: spec.rules[2].http.paths[12].pathType: Unsupported value: "Begin": supported values: "Exact", "ImplementationSpecific", "Prefix"
Am I missing something ? Is this feature only available for Enterprise plan ?
Thanks, Greg
HAProxy Ingress follows “Ingress v1 spec”, so any Ingress spec configuration should work as stated by the Kubernetes documentation.
As per the kubernetes documentation, the supported path types are ImplementationSpecific, Exact and Prefix. Paths that do not include an explicit pathType will fail validation. Here the path type Begin is not supported as per kubernetes documentation. So use either of those 3 types.
For more information on kubernetes supported path types refer to the documentation.
I am setting up an ingress service following some k8s documentation, but I am not able to understand the following annotations:
kubernetes.ip/ingress.class:
nginx.ingress.kubernetes.io/rewrite-target:
Do you know what these annotations do?
Thanks in advance.
kubernetes.io/ingress.class annotation is officially deprecated:
Before the IngressClass resource was added in Kubernetes 1.18, a
similar concept of Ingress class was often specified with a
kubernetes.io/ingress.class annotation on the Ingress. Although this
annotation was never formally defined, it was widely supported by
Ingress controllers, and should now be considered formally deprecated.
Instead you should use the ingressClassName:
The newer ingressClassName field on Ingresses is a replacement for
that annotation, but is not a direct equivalent. While the annotation
was generally used to reference the name of the Ingress controller
that should implement the Ingress, the field is a reference to an
IngressClass resource that contains additional Ingress
configuration, including the name of the Ingress controller.
The rewrite annotation does as follows:
In some scenarios the exposed URL in the backend service differs from
the specified path in the Ingress rule. Without a rewrite any request
will return 404. Set the annotation
nginx.ingress.kubernetes.io/rewrite-target to the path expected by
the service.
If the Application Root is exposed in a different path and needs to be
redirected, set the annotation nginx.ingress.kubernetes.io/app-root
to redirect requests for /.
For a more detailed example I strongly suggest you can check out this source. It shows exactly how rewriting works.
Since Kubernetes 1.18+, kubernetes.io/ingress.class annotation is deprecated.
You have to create an IngressClass like:
apiVersion: networking.k8s.io/v1
kind: IngressClass
metadata:
name: alb-ingress-class
spec:
controller: ingress.k8s.aws/alb
And then reference it in your Ingress declaration:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
namespace: ...
name: my-fabulous-ingress
annotations:
...
labels:
...
spec:
ingressClassName: "alb-ingress-class"
rules:
...
Important: Be sure to create the IngressClass before the Ingress (because it is referenced by the Ingress)
Note: If in the same manifest, putting the IngressClass block above the Ingress one is enough.
More info:
https://kubernetes.io/blog/2020/04/02/improvements-to-the-ingress-api-in-kubernetes-1.18/
https://kubernetes-sigs.github.io/aws-load-balancer-controller/v2.2/guide/ingress/ingress_class/
I was doing some research about ingress and it seems I have to create a new ingress resource for each namespace. Is that correct?
I just created 2 separate ingress resources in different namespaces in my GKE cluster but it seems to use the same LB in(which is great for cost) but I would think it is possible to have clashes then. (when using same path). I just tried it and the first one I've created is still working on the path, the other newer one on the same path is just not working.
Can someone explain me the correct setup for ingress?
As Kubernetes works, ingress controller won't pass a packet to a service that is in a different namespace from the ingress resource. So, if you create an ingress resource in the default namespace, all your services must be in the default namespace as well.
This is something that won't change. EVER. There has been a feature request years ago, and kubernetes team announced that it's not going to happen. It introduces a security hole when ingress controller is being able to transpass a namespace.
Now, what we do in these situations is actually pretty neat. You will have to do the following:
Say you have 2 services in the namespaces you need. e.g. service1.foo and service2.bar.
create 2 headless services without selectors and 2 Endpoint objects pointing to the IP addresses of the services service1.foo and service2.bar, in the same namespace as the ingress resource. The headless service without selectors will force kube-dns (or coreDNS) to search for either ExternalName type service or an Endpoint object. Now, the only requirement here is that your headless service and the Endpoint object must have the same name.
Create your ingress resource pointing to the headless services.
It should look like this (for 1 service):
Say the IP address of service1.foo is 10.10.10.10. Your headless service and the Endpoint object would be:
apiVersion: v1
kind: Service
metadata:
name: bait-svc
spec:
clusterIP: None
ports:
- name: http
port: 80
targetPort: 80
---
apiVersion: v1
kind: Endpoints
metadata:
name: bait-svc
subsets:
- addresses:
- ip: 10.10.10.10
ports:
- port: 80
protocol: TCP
and Ingress resource:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: ingress
annotations:
kubernetes.io/ingress.class: "nginx"
spec:
tls:
- secretName: ssl-certs
rules:
- host: site1.training.com
http:
paths:
- path: /
backend:
serviceName: bait-svc
servicePort: 80
So, the Ingress points to the bait-svc, and bait-svc points to service1.foo. And you will do this for each service.
UPDATE
I am thinking now, it might not work with GKE Ingress Controller, as on GKE you need a NodePort type service for the HTTP load balancer to reach the service. As you can see, in my example I've got nginx Ingress Controller.
Independently if it works or not, I would recommend using some other Ingress Controller. It's not that GKE IC is not good. It is quite robust, but almost always you end up hitting some limitation. Other ICs are more flexible.
The behavior of conflicting Ingress routes is undefined and implementation dependent. In most cases it’s just last writer wins.
I configure Ingress on google Kubernetes engine. I am new on ingress but as i understood ingress can serve different Loadbalancers and different LBs should be differently configured.
I have started with a simple ingress config on GKE :
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: basic-ingress
spec:
rules:
- http:
paths:
- path: /*
backend:
serviceName: web-np
servicePort: 8080
- path: /v2/keys
backend:
serviceName: etcd-np
servicePort: 2379
And it works fine so I have 2 different NodePort services web-np and etcd-np . But now I need to extend this logic with some rewrite rules so that request that points to /service1 - will be redirected to the other service1-np service but before /service1/hello.html must be replaced to /hello.html. That's why I have the following questions:
How can I configure rewrite in ingress and if it is possible with default load balancer.
What is default load balancer on GKE.
Where can I find a list of all annotations to it. I have thought that the full list is on https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/annotations/ but there is a completly different list and there is no kubernetes.io/ingress.global-static-ip-name annotation that is widely used in google examples.
Ingress - API object that manages external access to the services in a cluster, typically HTTP.
Ingress may provide load balancing, SSL termination and name-based virtual hosting.
Kubernetes.io: Ingress
Kubernetes can have multiple Ingress controllers. This controllers are different from each other. The Ingress controllers mentioned by you in this particular question are:
Ingress-GCE - a default Ingress resource for GKE cluster:
Github.com: Kubernetes: Ingress GCE
Ingress-nginx - an alternative Ingress controller which can be deployed to your GKE cluster:
Github.com: Kubernetes: Ingress-nginx
Ingress configuration you pasted will use the Ingress-GCE controller. If you want to switch to Ingress-nginx one, you will need to deploy it and set an annotation like:
kubernetes.io/ingress.class: "nginx"
How can I configure rewrite in ingress and if it is possible with default load balancer.
There is an ongoing feature request to support rewrites with Ingress-GCE here: Github.com: Ingress-GCE: Rewrite.
You can use Ingress-nginx to have support for rewrites. There is an official documentation about deploying it: Kubernetes.github.io: Ingress-nginx: Deploy
For more resources about rewrites you can use:
Kubernetes.github.io: Ingress nginx: Examples: Rewrite
Stackoverflow.com: Ingress nginx how to serve assests to application - this is an answer which shows an example on how to configure a playground for experimenting with rewrites
What is default load balancer on GKE.
If you create an Ingress resource with a default Ingress-GCE option you will create a L7 HTTP&HTTPS LoadBalancer.
If you create a service of type LoadBalancer in GKE you will create an L4 Network Load Balancer
If you deploy an Ingress-nginx controller in GKE cluster you will create a L4 Network Loadbalancer pointing to the Ingress-nginx controller which after that will route the traffic accordingly to your Ingress definition. If you are willing to use Ingress-nginx you will need to specify:
kubernetes.io/ingress.class: "nginx"
in your Ingress definition.
Please take a look on this article: Medium.com: Google Cloud: Kubernetes Nodeport vs Loadbalancer vs Ingress
Where can I find a list of all annotations to it. I have thought that the full list is on https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/annotations/ but there is a completly different list and there is no kubernetes.io/ingress.global-static-ip-name annotation that is widely used in google examples.
The link that you provided with annotations is specifically for Ingress-nginx. This annotations will not work with Ingress-GCE.
The annotations used in GCP examples are specific to Ingress-GCE.
You can create a Feature Request for a list of available annotations for Ingress-GCE on Issuetracker.google.com.
Answering an old question, but hopefully it can help someone.
I found the list of annotations for GCP Ingress in the source code for ingress-gce.
I am trying to deploy keycloak on google kubernetes engine and got it working using the ingress.class type nginx as follows
kubernetes.io/ingress.class: nginx
Full manifest can be found here
https://github.com/vsomasvr/keycloak-gke/blob/master/keycloak-gke-ingress/ingress.yaml
But, my intent is to use ingress.class type "gce". For that I had changed the ingress annotations from the following
annotations:
ingress.kubernetes.io/ssl-redirect: "true"
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/force-ssl-redirect: "true"
to the following
kubernetes.io/ingress.allow-http: "false"
annotations: kubernetes.io/ingress.class: gce
After the above change, I consistently get a message indicating that the ingress has unhealthy backend (0/3).
I wonder what other changes would "gce" require when "nginx" could run without any issues.
I ensured that its not a firewall issue as the ports that the app is using are allowed for all, I also have livenessProbe and readinessProbe setting in place.
Is there anything else that this configuration is missing?
I placed all the manifest files here
https://github.com/vsomasvr/keycloak-gke/tree/master/keycloak-gke-ingress
Any help is appreciated
EDIT
I had added the annotation
kubernetes.io/ingress.allow-http: "false"
to the nginx ingress, tested & ensured that it would not cause any conflict. The app worked without issues.
On the other end, the gce ingress has the same behavior even when I remove the above mentioned annotation
Your service seems to be configured to use HTTP traffic, which means the health checks for the Load Balancer will also use HTTP traffic (port 80) yet you are using an annotation that disables HTTP
kubernetes.io/ingress.allow-http: "false"