How to configure tls with traefik in kubernetes using yaml? - kubernetes

Question

I am having trouble exposing a service over http and https using traefik 2.9 in Kubernetes.
The http endpoint kinda works, I introduced CORS errors somehow once I tried to add https but that is not my main concern. The https ingress is broken and I cant find any indication of why its not working. The traefik pod doesn't log any errors and the dotnet service isn't receiving the requests. Also both routes show up in the dashboard and websecure is displayed as having TLS enabled.
Excluding ClusterRole, ServiceAccount, and ClusterRoleBinding because I believe that's configured correctly as the http route wouldn't work if it wasnt.
Traefik config:
kind: Deployment
apiVersion: apps/v1
metadata:
name: traefik-deployment
labels:
app: traefik
spec:
replicas: 1
selector:
matchLabels:
app: traefik
template:
metadata:
labels:
app: traefik
spec:
serviceAccountName: traefik-account
containers:
- name: traefik
image: traefik:v2.9
args:
- --api.insecure
- --providers.kubernetesingress
- --entrypoints.web.address=:80
- --entrypoints.websecure.address=:443
- --entrypoints.websecure.http.tls
ports:
- name: web
containerPort: 80
- name: dashboard
containerPort: 8080
- name: websecure
containerPort: 443
Traefik services:
apiVersion: v1
kind: Service
metadata:
name: traefik-dashboard-service
spec:
type: LoadBalancer
ports:
- port: 8080
targetPort: dashboard
selector:
app: traefik
---
apiVersion: v1
kind: Service
metadata:
name: traefik-web-service
spec:
type: LoadBalancer
loadBalancerIP: 10.10.1.38
ports:
- targetPort: web
port: 80
name: http
- targetPort: websecure
port: 443
name: https
selector:
app: traefik
Secret for tls:
apiVersion: v1
data:
comptech.pem: <contents of pem file base64 encoded>
comptech.crt: <contents of crt file base64 encoded>
comptech.key: <contents of key file base64 encoded>
kind: Secret
metadata:
name: comptech-cert
namespace: default
type: Opaque
Service for dotnet application:
apiVersion: v1
kind: Service
metadata:
name: control-api-service
spec:
ports:
- name: http
port: 80
targetPort: 5000
protocol: TCP
- name: https
port: 443
targetPort: 5000
protocol: TCP
selector:
app: control-api
Ingresses:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: control-api-ingress
annotations:
traefik.ingress.kubernetes.io/router.entrypoints: web
spec:
rules:
- host: sub.domain.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: control-api-service
port:
name: http
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: control-api-secure-ingress
annotations:
traefik.ingress.kubernetes.io/router.entrypoints: websecure
traefik.ingress.kubernetes.io/router.tls: "true"
spec:
rules:
- host: sub.domain.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: control-api-service
port:
name: https
tls:
- secretName: comptech-cert
My hope here is that someone with much more experience with traefik/tls will be able to quickly realize what I'm doing incorrectly. Any input is greatly appreciated!
UPDATE:
The firewall was only allowing http traffic, we reconfigured it to support https and it is responding with Traefiks default certs. So i can hit the container but tls is still not configured using my supplied cert.

Answer 1

The pem file is not needed and the crt file was generated incorrectly using openssl the command that worked for me was: openssl crl2pkcs7 -nocrl -certfile comptech.pem | openssl pkcs7 -print_certs -out cert.crt
Pointing to the https port of the control-api-service was not working and needed to be changed to http
A config map needed to be created for the traefik deployment to work correctly:
apiVersion: v1 kind: ConfigMap metadata: name: traefik-config labels:
name: traefik-config namespace: default data: dyn.yaml: |
# https://doc.traefik.io/traefik/https/tls/
tls:
stores:
default:
defaultCertificate:
certFile: '/certs/tls.crt'
keyFile: '/certs/tls.key'
Finally the configmap and secret must be used in the traefik deployment like below:
kind: Deployment
apiVersion: apps/v1
metadata:
name: traefik-deployment
labels:
app: traefik
spec:
replicas: 1
selector:
matchLabels:
app: traefik
template:
metadata:
labels:
app: traefik
spec:
serviceAccountName: traefik-account
containers:
- name: traefik
image: traefik:v2.9
args:
- --api.insecure
- --providers.kubernetesingress
- --entrypoints.web.address=:80
- --entrypoints.websecure.address=:443
- --entrypoints.websecure.http.tls
- --providers.file.filename=/config/dyn.yaml
ports:
- name: web
containerPort: 80
- name: dashboard
containerPort: 8080
- name: websecure
containerPort: 443
volumeMounts:
- name: comptech-cert-volume
mountPath: /certs
- name: traefik-config-volume
mountPath: /config
volumes:
- name: comptech-cert-volume
secret:
secretName: comptech-cert
- name: traefik-config-volume
configMap:
name: traefik-config

Answer 2

In my setup, I use the IngressRoute CRD implementation from Traefik.
The CRDs were automatically installed when I setup the Traefik controller using Helm.
Is it a possibility for you to use this in your setup? You can check if the CRDs already exist using below command on your k8s cluster.
kubectl get crd
Below is a snippet from one of my projects where I also use a custom wildcard certificate from a secret using the IngressRoute manifest.
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
name: blue-api-ingressroute
spec:
entryPoints:
- websecure
routes:
- match: "Host(`blue.domain.com`)" && PathPrefix(`/swagger`)"
kind: Rule
services:
- name: blue-api-svc
port: 80
tls:
secretName: bluecert
You can also include other custom resources that are available from Traefik. The complete set of configuration that is available can be seen here. For example, below is the same snippet with middleware and tlsoptions resources included for improving the security of the endpoint.
apiVersion: traefik.containo.us/v1alpha1
kind: TLSOption
metadata:
name: tlsoptions
spec:
minVersion: VersionTLS12
cipherSuites:
- TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256
- TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256
- TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
- TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305
- TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384
- TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305
- TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305
- TLS_AES_256_GCM_SHA384
- TLS_AES_128_GCM_SHA256
- TLS_CHACHA20_POLY1305_SHA256
- TLS_FALLBACK_SCSV
curvePreferences:
- CurveP521
- CurveP384
sniStrict: true
---
apiVersion: traefik.containo.us/v1alpha1
kind: Middleware
metadata:
name: security
spec:
headers:
frameDeny: true
sslRedirect: true
browserXssFilter: true
contentTypeNosniff: true
stsIncludeSubdomains: true
stsPreload: true
stsSeconds: 31536000
---
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
name: blue-api-ingressroute
spec:
entryPoints:
- websecure
routes:
- match: "Host(`blue.domain.com`)" && PathPrefix(`/swagger`)"
kind: Rule
services:
- name: blue-api-svc
port: 80
middlewares:
- name: security
tls:
secretName: bluecert
options:
name: tlsoptions