automate letsencrypt on a kubernetes app with lots of domains - kubernetes

Question

I have a node app that loads its data based on domain name. domains are configured with a CNAME like app.service.com (which is the node app).
The Node app sees the request domain and sends a request to API to get app data.
for example: domain.com CNAME app.service.com
-> then node app asks api for domain.com data
the problem is setting up HTTPS (with letsencrypt) for all the domains. I think cert-manager can help but have no idea how to automate this without the need to manually change config file for each new domain.
or is there a better way to achieve this in Kubernetes?

Answer 1

The standard method to support more than one domain name and / or subdomain names is to use one SSL Certificate and implement SAN (Subject Alternative Names). The extra domain names are stored together in the SAN. All SSL certificates support SAN, but not all certificate authorities will issue multi-domain certificates. Let's Encrypt does support SAN so their certificates will meet your goal.
First, you have to create a job in our cluster that uses an image to run a shell script. The script will spin up an HTTP service, create the certs, and save them into a predefined secret. Your domain and email are environment variables, so be sure to fill those in:
apiVersion: batch/v1
kind: Job
metadata:
name: letsencrypt-job
labels:
app: letsencrypt
spec:
template:
metadata:
name: letsencrypt
labels:
app: letsencrypt
spec:
containers:
# Bash script that starts an http server and launches certbot
# Fork of github.com/sjenning/kube-nginx-letsencrypt
- image: quay.io/hiphipjorge/kube-nginx-letsencrypt:latest
name: letsencrypt
imagePullPolicy: Always
ports:
- name: letsencrypt
containerPort: 80
env:
- name: DOMAINS
value: kubernetes-letsencrypt.jorge.fail # Domain you want to use. CHANGE ME!
- name: EMAIL
value: jorge#runnable.com # Your email. CHANGE ME!
- name: SECRET
value: letsencrypt-certs
restartPolicy: Never
You have a job running, so you can create a service to direct traffic to this job:
apiVersion: v1
kind: Service
metadata:
name: letsencrypt
spec:
selector:
app: letsencrypt
ports:
- protocol: "TCP"
port: 80
This job will now be able to run, but you still have three things we need to do before our job actually succeeds and we’re able to access our service over HTTPs.
First, you need to create a secret for the job to actually update and store our certs. Since you don’t have any certs when we create the secret, the secret will just start empty.
apiVersion: v1
kind: Secret
metadata:
name: letsencrypt-certs
type: Opaque
# Create an empty secret (with no data) in order for the update to work
Second, you’ll have to add the secret to the Ingress controller in order for it to fetch the certs. Remember that it is the Ingress controller that knows about our host, which is why our certs need to be specified here. The addition of our secret to the Ingress controller will look something like this:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: "kubernetes-demo-app-ingress-service"
spec:
tls:
- hosts:
- kubernetes-letsencrypt.jorge.fail # Your host. CHANGE ME
secretName: letsencrypt-certs # Name of the secret
rules:
Finally you have to redirect traffic through the host, down to the job, through our Nginx deployment. In order to do that you’ll add a new route and an upstream to our Nginx configuration: This could be done through the Ingress controller by adding a /.well-known/* entry and redirecting it to the letsencrypt service. That’s more complex because you would also have to add a health route to the job, so instead you’ll just redirect traffic through the Nginx deployment:
apiVersion: v1
kind: ConfigMap
metadata:
name: nginx-config
data:
default.conf: |
...
# Add upstream for letsencrypt job
upstream letsencrypt {
server letsencrypt:80 max_fails=0 fail_timeout=1s;
}
server {
listen 80;
...
# Redirect all traffic in /.well-known/ to letsencrypt
location ^~ /.well-known/acme-challenge/ {
proxy_pass http://letsencrypt;
}
}
After you apply all these changes, destroy your Nginx Pod(s) in order to make sure that the ConfigMap gets updated correctly in the new Pods:
$ kubectl get pods | grep ngi | awk '{print $1}' | xargs kubectl delete pods
Make sure it works.
In order to verify that this works, you should make sure the job actually succeeded. You can do this by getting the job through kubectl, you can also check the Kubernetes dashboard.
$ kubectl get job letsencrypt-job
NAME DESIRED SUCCESSFUL AGE
letsencrypt-job 1 1 1d
You can also check the secret to make sure the certs have been properly populated. You can do this through kubectl or through the dashboard:
$ kubectl describe secret letsencrypt-certs
Name: letsencrypt-certs
Namespace: default
Labels: <none>
Annotations:
Type: Opaque
Data
====
tls.crt: 3493 bytes
tls.key: 1704 bytes
Now that as you can see that the certs have been successfully created, you can do the very last step in this whole process. For the Ingress controller to pick up the change in the secret (from having no data to having the certs), you need to update it so it gets reloaded. In order to do that, we’ll just add a timestamp as a label to the Ingress controller:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: "kubernetes-demo-app-ingress-service"
labels:
# Timestamp used in order to force reload of the secret
last_updated: "1494099933"
...
Please take a look at: kubernetes-letsencrypt.