I am using traefik as ingress controller in a K8s cluster. I'd like to enforce all traffic via HTTPS.
It is a bit confusing the documentation because apparently there are different ways to do the same thing? Namely, I'd like to know what is the difference between these 2:
https://docs.traefik.io/user-guide/examples/#http-redirect-on-http
https://docs.traefik.io/configuration/backends/kubernetes/#general-annotations
If I use K8s is enough to use general annotations or I still have to edit the TOML file?
I have tried to use traefik.ingress.kubernetes.io/redirect-entry-point: https but doesn't find the service so I guess something is missing in my config.
If I remove the above line then everything works but HTTP -> HTTPS is, of course, not working. When I place the line back it returns a 404.
FWIW, the ingress definition is as follows (with traefik redirecting to 404):
- apiVersion: extensions/v1beta1
kind: Ingress
metadata:
generation: 1
name: rekover-ingress
namespace: prod
annotations:
kubernetes.io/ingress.class: traefik
traefik.ingress.kubernetes.io/frontend-entry-points: http, https
traefik.ingress.kubernetes.io/redirect-entry-point: https
spec:
rules:
- host: www.xxx.com
http:
paths:
- backend:
serviceName: xxx-frontend
servicePort: 80
I have tried the very same configuration with nginx and changing to respective metadata and worked!. Below metadata used for nginx and Ingress:
nginx.ingress.kubernetes.io/force-ssl-redirect: "true"
kubernetes.io/ingress.class: nginx
For completeness, I copy-paste the service definition for both nginx and traefik. First one works as expected:
kind: Service
apiVersion: v1
metadata:
name: ingress-nginx
namespace: ingress-nginx
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
annotations:
service.beta.kubernetes.io/aws-load-balancer-backend-protocol: http
service.beta.kubernetes.io/aws-load-balancer-ssl-cert: arn:aws:acm:xxxx
service.beta.kubernetes.io/aws-load-balancer-ssl-ports: https
spec:
selector:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
ports:
- name: http
port: 80
targetPort: http
- name: https
port: 443
targetPort: http
type: LoadBalancer
</code>
<code>
kind: Service
apiVersion: v1
metadata:
name: traefik-ingress-service
namespace: ingress-prod
annotations:
service.beta.kubernetes.io/aws-load-balancer-backend-protocol: http
service.beta.kubernetes.io/aws-load-balancer-ssl-cert: arn:aws:acm:xxxx
service.beta.kubernetes.io/aws-load-balancer-ssl-ports: https
spec:
selector:
k8s-app: traefik-ingress-lb
ports:
- name: http
port: 80
targetPort: http
protocol: TCP
- name: https
port: 443
targetPort: http
protocol: TCP
type: LoadBalancer
The first one is the toml Traefik configuration file, that can be modified in Kubernetes through a ConfigMap or in the Traefik container itself, or in the Traefik container command line (depending on how you deployed the Traefik Ingress Controller).
The second is what is mostly used to manage the configuration in the Traefik Ingress Controller through the Ingress Kubernetes resource.
You didn't post your Ingress definition but most likely it doesn't have a way to handle HTTPS and that's why when you add the annotation it sends traffic to port 443 and Traefik returns a 404.
To handle HTTPS/TLS you would need to enable it first (Create a K8s cert secret, configure TLS in the Ingress, etc). Or this is another example on how to enable it using a ConfigMap.
I give you the solution as valid because you provided a correct answer to the original question though the original question was badly formulated, my bad.
I found a related question which answers my question and worked like a charm:
Unable to redirect from http to https behind AWS load balancer
What frustates me is that, for instance, with nginx is just enough to use metadata annotations to force redirect to ssl whereas with traefik you have to edit the TOML file (or at least I didnt find the equivalent annotation for the problem).
Related
Why is my LoadBalancer service in Kubernetes not reachable?
I have deployed an nginx-ingress-controller helm chart and it has a service of LoadBalancer type in EKS. This service receives a url (EXTERNAL-IP) and this url has an IP but when I'm trying the reach this url it's not reachable.
I did kubectl port-forward -n ingress-nginx services/ingress-nginx-controller8080:80 and then I can reach nginx in localhost:8080 so I know the problem is to reach the service itself from the internet. I've checked and VPC and subnets security-groups and inbound/outbound rules and it seems ok. .
can anyone provide some guidance on how to troubleshoot this issue?
This is the definition of the
apiVersion: v1
kind: Service
metadata:
annotations:
service.beta.kubernetes.io/aws-load-balancer-backend-protocol: http
service.beta.kubernetes.io/aws-load-balancer-ssl-cert: arn:aws:acm:eu-central-1:xxx:certificate/xxx
service.beta.kubernetes.io/aws-load-balancer-ssl-ports: https
service.beta.kubernetes.io/aws-load-balancer-type: nlb
labels:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.5.1
helm.sh/chart: ingress-nginx-4.4.2
name: ingress-nginx-controller
namespace: ingress-nginx
spec:
allocateLoadBalancerNodePorts: true
clusterIP: xxx
clusterIPs:
- xxx
externalTrafficPolicy: Cluster
internalTrafficPolicy: Cluster
ipFamilies:
- IPv4
ipFamilyPolicy: SingleStack
ports:
- name: http
nodePort: xxx
port: 80
protocol: TCP
targetPort: http
- name: https
nodePort: xxx
port: 443
protocol: TCP
targetPort: http
selector:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
sessionAffinity: None
type: LoadBalancer
This is the command I'm using to deploy the nginx helm chart:
helm upgrade --install ingress-nginx ingress-nginx --repo https://kubernetes.github.io/ingress-nginx --namespace ingress-nginx --create-namespace --version 4.4.2 -f values.yaml
And this is my values.yaml:
controller:
config:
allow-snippet-annotations: "true"
http-snippet: |
server {
listen 2443;
return 308 https://$host$request_uri;
}
use-forwarded-headers: "false"
service:
enabled: true
annotations:
service.beta.kubernetes.io/aws-load-balancer-backend-protocol: http
service.beta.kubernetes.io/aws-load-balancer-internal: "true"
service.beta.kubernetes.io/aws-load-balancer-ssl-cert: arn:aws:acm:eu-central-1:xxx:certificate/xxx
service.beta.kubernetes.io/aws-load-balancer-ssl-ports: https
service.beta.kubernetes.io/aws-load-balancer-type: nlb
ports:
http: 80
https: 443
targetPorts:
http: http
https: http
type: LoadBalancer
to reach the nginx service I either go to it on browser or just do:
curl xxx-xxx.elb.eu-central-1.amazonaws.com
but I always get This site can’t be reached
First and foremost: give up. Nginx controller won't just work with ACM properly, I've wasted enourmous hours to accept this and move on.
Now that you did, here's approach I employed just yesterday and it worked brilliant.
Install nginx controller as helm release, without making any changes to controller service, this will create CLB (instead of NLB) and this is fine. NLB is mentioned in the guides on the internet as a crutch to get ACM certificate working, but all it does is produces redirect loops.
MOST IMPORTANT - Please, go through this to install cert-manager to manage LE certificates for you - ACM won't just let you export certificate. If you have external one already - good, just put it as secret for TLS definition below, otherwise after installation of cert-manager (if it takes too long to install helm for it, you didn't specify true for CRD installation - THIS IS CRUCIAL and if it gets stuck you will need to uninstall release and retry again properly).
Here's example of nginx ingress you can adapt to your needs:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: yeaboi-lb
namespace: yeaboi
annotations:
nginx.ingress.kubernetes.io/server-snippet: |
rewrite ^/(/?)$ /yeaboi$1 break;
nginx.ingress.kubernetes.io/ssl-redirect: "false"
kubernetes.io/ingress.class: "nginx"
cert-manager.io/cluster-issuer: "letsencrypt"
acme.cert-manager.io/http01-edit-in-place: "true"
spec:
tls:
- secretName: yeaboi-tls
hosts:
- yeaboi.io
rules:
- host: yeaboi.io
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: yeaboi-web
port:
number: 80
ingressClassName: nginx
Here's example of yeaboi-web service you can use in conjunction with abovementioned ingress (obviously make sure to specify targetPort exposed for your deployment):
apiVersion: v1
kind: Service
metadata:
name: yeaboi-web
namespace: yeaboi
labels:
app: yeaboi-web
spec:
ports:
- port: 80
targetPort: 9000
protocol: TCP
selector:
app: yeaboi-web
Point your domain to CLB created by nginx (just in case make sure it has your EC2 in inService status etc).
Enjoy having working nginx ingress you can customize a lot (unlike ALB ingress which is severely limited in comparison).
I don't mean being able to route to a specific port, I mean to actually change the port the ingress listens on.
Is this possible? How? Where is this documented?
No. From the kubernetes documentation:
An Ingress does not expose arbitrary ports or protocols. Exposing services other than HTTP and HTTPS to the internet typically uses a service of type Service.Type=NodePort or Service.Type=LoadBalancer.
It may be possible to customize a LoadBalancer on a cloud provider like AWS to listen on other ports.
I assume you are using NGINX Ingress Controller. In this case, during installation, instead of doing a kubectl apply in the official yaml like this is one, you can try downloading the yaml and changing the port. The file above, which is used for an L4 AWS ELB, would become like this:
kind: Service
apiVersion: v1
metadata:
name: ingress-nginx
namespace: ingress-nginx
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
annotations:
service.beta.kubernetes.io/aws-load-balancer-proxy-protocol: "*"
service.beta.kubernetes.io/aws-load-balancer-connection-idle-timeout: "60"
spec:
type: LoadBalancer
selector:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
ports:
- port: {custom port 1}
targetPort: http
- port: {custom port 2}
targetPort: https
An alternative is to use a more powerful ingress controller.
Here is a list of different controllers.
My personal choice is Ambassador. If you follow the getting-started page, you just need to change the service definition for the port of your choice:
---
apiVersion: v1
kind: Service
metadata:
name: ambassador
spec:
type: LoadBalancer
externalTrafficPolicy: Local
ports:
- port: {custom port}
targetPort: 8080
selector:
service: ambassador
An Ingress definition is backed by an ingress controller. The ingress controller is deployed with normal Kubernetes objects so will have a Service associated with it that exposes ports for the ingress controller.
The kubernetes/ingress-nginx static deploys have a deploy.yaml with a Service type LoadBalancer:
kind: Service
apiVersion: v1
metadata:
name: ingress-nginx
namespace: ingress-nginx
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
spec:
externalTrafficPolicy: Local
type: LoadBalancer
selector:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
ports:
- name: http
port: 80
targetPort: http
- name: https
port: 443
targetPort: https
Modify the ports the load balancer is configured with, in spec.ports[*].port in the external service, however that is deployed.
If you're using Helm to deploy the Kubernetes ingress-nginx controller, you can do this to change the ports from their defaults of 80 and 443 in a Helm values override file:
ingress-nginx:
enabled: true
...
controller:
service:
ports:
http: 8123
https: 9456
See here: https://artifacthub.io/packages/helm/ingress-nginx/ingress-nginx#values
In this example, any services exposed through the ingress are now available on ports 8123 (http) and 9456 (https).
Hi I have a react docker that uses nginx
with this service
apiVersion: v1
kind: Service
metadata:
labels:
appcluster: ethernial
app: clientweb
visibility: external
name: clientweb-service-ext
spec:
ports:
- port: 80
name: http
selector:
app: clientweb
type: ClusterIp
I want to expose it, I have only 1 Node that is the Master, but the port 80 is already in use by apache running on master node (cannot shutdown it yet)
I want to expose my react app so I can reach it by http://:30000 for example
(I also need to expose other REST apis externally and internally, one hosted on a pod and each one uses port 80)
so how I setup my ingress?
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: clientweb-ingress
spec:
defaultBackend:
service:
name: clientweb
port:
number: 8080
thanks!
You need to expose the ingress controller using a NodePort service on port 30000. Once you do that you can access backend pods exposed via ingress resource using 30000 port. If you are using nginx ingress controller then follow this doc and the NodePort service(taken from the nginx installation docs) would look like below with your desired port 30000 and 30001.
apiVersion: v1
kind: Service
metadata:
labels:
helm.sh/chart: ingress-nginx-2.13.0
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/version: 0.35.0
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/component: controller
name: ingress-nginx-controller
namespace: ingress-nginx
spec:
type: NodePort
ports:
- name: http
port: 80
protocol: TCP
targetPort: http
nodePort: 30000 # Specified nodeport
- name: https
port: 443
protocol: TCP
targetPort: https
nodePort: 30001 # Specified nodeport
selector:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/component: controller
In this case you can still continue to have apache on port 80 on the host system.
curl http://NODEIP:30000/<path-in-ingress>
curl https://NODEIP:30001/<path-in-ingress>
First, you need to understand the relationship between Ingress and Ingress Controller. Ingress is just a kind of resources, and it will do nothing except declare the ingress rules. All Ingress rules will need a certain Ingress Controller to implement its rules.
Then you need to deploy an Ingress Controller, typically a deployment(for certain pod) and a service(for external access). You can have a look at Nginx Ingress Controller at https://kubernetes.github.io/ingress-nginx/ and use kubectl or helm to make the deployment. Do not forget to annotate the ingress-class as it will be used later.
After this, you can apply any Ingress to this certain Ingress Controller by adding kubernetes.io/ingress.class: "nginx" annotation to your Ingress. And ingress controller(nginx server) will add your rules to its config, that means your Ingress rules has been applied.
Finally, as your ingress-controller-service has expose it self(LoadBalanceIp or NodePort from port 30000), all traffic to this endpoint will go through your Ingress rules and redirect to the desire service.
I don't mean being able to route to a specific port, I mean to actually change the port the ingress listens on.
Is this possible? How? Where is this documented?
No. From the kubernetes documentation:
An Ingress does not expose arbitrary ports or protocols. Exposing services other than HTTP and HTTPS to the internet typically uses a service of type Service.Type=NodePort or Service.Type=LoadBalancer.
It may be possible to customize a LoadBalancer on a cloud provider like AWS to listen on other ports.
I assume you are using NGINX Ingress Controller. In this case, during installation, instead of doing a kubectl apply in the official yaml like this is one, you can try downloading the yaml and changing the port. The file above, which is used for an L4 AWS ELB, would become like this:
kind: Service
apiVersion: v1
metadata:
name: ingress-nginx
namespace: ingress-nginx
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
annotations:
service.beta.kubernetes.io/aws-load-balancer-proxy-protocol: "*"
service.beta.kubernetes.io/aws-load-balancer-connection-idle-timeout: "60"
spec:
type: LoadBalancer
selector:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
ports:
- port: {custom port 1}
targetPort: http
- port: {custom port 2}
targetPort: https
An alternative is to use a more powerful ingress controller.
Here is a list of different controllers.
My personal choice is Ambassador. If you follow the getting-started page, you just need to change the service definition for the port of your choice:
---
apiVersion: v1
kind: Service
metadata:
name: ambassador
spec:
type: LoadBalancer
externalTrafficPolicy: Local
ports:
- port: {custom port}
targetPort: 8080
selector:
service: ambassador
An Ingress definition is backed by an ingress controller. The ingress controller is deployed with normal Kubernetes objects so will have a Service associated with it that exposes ports for the ingress controller.
The kubernetes/ingress-nginx static deploys have a deploy.yaml with a Service type LoadBalancer:
kind: Service
apiVersion: v1
metadata:
name: ingress-nginx
namespace: ingress-nginx
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
spec:
externalTrafficPolicy: Local
type: LoadBalancer
selector:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
ports:
- name: http
port: 80
targetPort: http
- name: https
port: 443
targetPort: https
Modify the ports the load balancer is configured with, in spec.ports[*].port in the external service, however that is deployed.
If you're using Helm to deploy the Kubernetes ingress-nginx controller, you can do this to change the ports from their defaults of 80 and 443 in a Helm values override file:
ingress-nginx:
enabled: true
...
controller:
service:
ports:
http: 8123
https: 9456
See here: https://artifacthub.io/packages/helm/ingress-nginx/ingress-nginx#values
In this example, any services exposed through the ingress are now available on ports 8123 (http) and 9456 (https).
I'm having difficulties getting my Ingress controller running on Google Container Engine. I want to use an NGINX Ingress Controller with Basic Auth and use a reserved global static ip name (this can be made in the External IP addresses section in the Google Cloud Admin interface). When I use the gce class everything works fine except for the Basic Auth (which I think is not supported on the gce class), anenter code hered when I try to use the nginx class the Ingress Controller launches but the IP address that I reserved in the Google Cloud Admin interface will not be attached to the Ingress Controller. Does anyone know how to get this working? Here is my config file:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: webserver
annotations:
kubernetes.io/ingress.global-static-ip-name: "myreservedipname"
kubernetes.io/ingress.class: "nginx"
ingress.kubernetes.io/auth-type: basic
ingress.kubernetes.io/auth-realm: "Auth required"
ingress.kubernetes.io/auth-secret: htpasswd
spec:
tls:
- secretName: tls
backend:
serviceName: webserver
servicePort: 80
I found a solution with helm.
helm install --name nginx-ingress stable/nginx-ingress \
--set controller.service.loadBalancerIP=<YOUR_EXTERNAL_IP>
You should use the external-ip and not the name you gave with gcloud.
Also, in my case I also added --set rbac.create=true for permissions.
External IP address can be attached to the Load Balancer which you can point to your Ingress controller.
One major remark - the External IP address should be reserved in the same region as the Kubernetes cluster.
To do it, you just need to deploy your Nginx-ingress service with type: LoadBalancer and set ExternalIP value, like this:
kind: Service
apiVersion: v1
metadata:
name: ingress-nginx
namespace: ingress-nginx
labels:
app: ingress-nginx
spec:
loadBalancerIP: <YOUR_EXTERNAL_IP>
type: LoadBalancer
selector:
app: ingress-nginx
ports:
- name: http
port: 80
targetPort: http
- name: https
port: 443
targetPort: https
After deployment, Kubernetes will create a new Load Balancer with desired static IP which will be an entry-point for your Ingress.
#silgon, as I see, you already tried to do it, but without a positive result. But, it should work. If not - check the region of IP address and configuration once again.
Here's an example that I know works, could be an issue around your syntax:
kind: Ingress
metadata:
name: nginx
spec:
rules:
- host: nginx.192.168.99.100.nip.io
http:
paths:
- backend:
serviceName: nginx
servicePort: 80