Gitlab CI Kubernetes Agent Self Signed Certificate - kubernetes

Question

I have following .gitlab-ci.yml config:
deploy:
stage: deploy
image:
name: bitnami/kubectl:latest
entrypoint: ['']
script:
- echo "Hello, Rules!"
- kubectl config get-contexts
- kubectl config use-context OurGroup/our-repo:agent-0
- kubectl get pods
rules:
- if: '$CI_COMMIT_REF_NAME == "master"'
when: manual
allow_failure: true
- if: '$CI_COMMIT_REF_NAME == "develop"'
when: manual
allow_failure: true
tags:
- docker
This fails on following error:
Unable to connect to the server: x509: certificate signed by unknown authority
We are running a self-hosted gitlab instance with a self-signed certificate. The issue is bitnami/kubectl:latest being a non root docker container, and it is described here in the official gitlab docu to be used:
https://docs.gitlab.com/ee/user/clusters/agent/ci_cd_workflow.html#update-your-gitlab-ciyml-file-to-run-kubectl-commands
I have tried the "echo "$CA_CERTIFICATE" > /usr/local/share/ca-certificates/my-ca.crt && update-ca-certificates" for injecting a certificate, but that fails due to not having privileges and SUDO not existing in this container.
kubectl get certificates fails on not being able to connect to localhost:8080
Any pointers on how to get a self-signed certificate to work with connection with kubectl and agent authentication, or what is perhaps considered a secure way of making this work?

Answer 1

Thankfully Gitlab provides a way to provided a path to a ca-crt-file via flag during start up of the agent.
Broadly the process for loading a self-signed certificate is:
create a file containing the public key of your self signed certificate, ensuring to concatenate any other public keys involved in the signing chain
if you can do the above it's possible to apply this file as a K8S ConfigMap, if not (you include the private key) then create it as a K8S Secret as the proceeding documentation indicates.
modify the K8S Deployment to mount the ConfigMap/Secret to the agent container then add a new line to the container args to point to the file - --ca-cert-file=/certs/ca.crt
GitLab Kubernetes Agent: provide custom certificates to the Agent Issue #280518 (thank you Philipp Hahn)
In the kind: Deployment deployment section add the following things:
In spec:template:spec:containers:args append - --ca-cert-file=/certs/${YOUR_CA}.crt - expand ${YOUR_CA} here manually
In spec:template:spec:contaiers:volumeMounts add a new block:
- name: custom-certs
readOnly: true
mountPath: /certs
In spec:template:spec:volumes add a new block:
- name: custom-certs
secret:
secretName: ca
Official Docs:
Providing a custom certificate for accessing GitLab
You can provide a Kubernetes Secret to the GitLab Runner Helm Chart, which will be used to populate the container’s /home/gitlab-runner/.gitlab-runner/certs directory.
Each key name in the Secret will be used as a filename in the directory, with the file content being the value associated with the key:
The key/file name used should be in the format <gitlab.hostname>.crt, for example gitlab.your-domain.com.crt.
Any intermediate certificates need to be concatenated to your server certificate in the same file.
The hostname used should be the one the certificate is registered for.
Gitlab Agent Helm Chart Deployment
- --ca-cert-file=/etc/agentk/config/ca.crt