I am getting confused with all the certificates and keys in Kubernetes.
Inside the master I have the following certificates and keys:
/etc/kubernetes/pki/apiserver.crt
/etc/kubernetes/pki/apiserver.key
/etc/kubernetes/pki/apiserver-kubelet-client.crt
/etc/kubernetes/pki/apiserver-kubelet-client.key
/var/lib/kubelet/pki/kubelet.crt
/var/lib/kubelet/pki/kubelet.key
The apiserver-kubelet-client (crt\key) is used for secure communication between the apiserver and the kubelet inside the Master.
Why I have also the kubelet (crt\key)? Isn't the apiserver-kubelet-client (crt\key) is enough ?
Why I need apiserver (crt\key) if I have apiserver-kubelet-client (crt\key) ?
Hope someone can make some order with this.
I read this article about Kubernetes PKI but I am still don't understand what is difference between the above certificates and keys.
While preparing certs, you will end up with at least 1 file.
CA_CERT
put in on node where apiserver runs, for example in /srv/kubernetes/ca.crt.
MASTER_CERT
signed by CA_CERT
put in on node where apiserver runs, for example in /srv/kubernetes/server.crt
MASTER_KEY
put in on node where apiserver runs, for example in /srv/kubernetes/server.key
You can read about it in Kubernetes docs here.
It’s hard to say by the names of your certs what are they used for.
You should refer to the standard configuration.
Besides, you can have a look at kelseyhightower/kubernetes-the-hard-way as he’s showing how and when to use certificates.
You can check this comment posted on Github regarding what kind of certifications are needed between which services.
I was also asking my self similar question.
The first thing u need to know is that both the kubelet and the api-server communicate with each other [0].
The second thing is that Kubernetes uses client certificates to authenticate a client identity to the server and a server certificate to authenticate the server’s identity to the client and establish an encrypted connection .
In case of kubelet -> api-server kubelet needs a client certificate and api-server a server certificate and for api-server -> kubelet then the otherway around.
you can check if a certificate is a server or client one using:
openssl x509 -noout -text -in apiserver.crt
Now back to the question :
/etc/kubernetes/pki/apiserver.crt and /var/lib/kubelet/pki/kubelet.crt are server certificates
/etc/kubernetes/pki/apiserver-kubelet-client.crt and /var/lib/kubelet/pki/kubelet-client-current.pem are client certificates
https://kubernetes.io/docs/concepts/architecture/control-plane-node-communication/
Related
Deployed Chaos with helm on AKS. all pods up&running and all resources created.
I try then to create a crd NetworkChaos as in the official documentation but I get a x509 signed by unknown authority when trying to contact the Controller Manager svc. the certificate used is stored in the webhook-certs automatically generated by the helm.
Can anyone help me? Should I put a trusted certificate instead? should I trust the generated certificate by the cluster CA?
I also would be ok on skipping tls but I didn't find how to. Thanks
Quick question, in Rancher is it possible to use lets-encrypt to sign the k8s TLS certs (etcd, kub-api, etc). I have a compliance requirement to sign my k8s environment with a valid trusted CA chain?
Yes, it is actually one of the recommended options for the source of the certificate used for TLS termination at the Rancher server:
Let’s Encrypt: The Let’s Encrypt option also uses cert-manager.
However, in this case, cert-manager is combined with a special Issuer
for Let’s Encrypt that performs all actions (including request and
validation) necessary for getting a Let’s Encrypt issued cert.
In the links below you will find a walkthrough showing how to:
Install cert-manager
Install Rancher with Helm and Your Chosen Certificate Option
This option uses cert-manager to automatically request and renew Let’s
Encrypt certificates. This is a free service that provides you with a
valid certificate as Let’s Encrypt is a trusted CA.
Please let me know if that helped.
I have been running K8s cluster(v1.13.5) for a year and the control plane certs and Kubelet certs are about to expire. I found a way to rotate all the control plane certs and I wanted to know how to rotate the Kubelet certs. Can someone help me to understand how to rotate the K certs for worker node and master (if needed)? This K8s cluster is deployed using Kubespray.
From Kubernetes version 1.8.0 a beta feature is available Certificate Rotation.
The kubelet uses certificates for authenticating to the Kubernetes API. By default, these certificates are issued with one year expiration so that they do not need to be renewed too frequently.
Kubernetes 1.8 contains kubelet certificate rotation, a beta feature that will automatically generate a new key and request a new certificate from the Kubernetes API as the current certificate approaches expiration. Once the new certificate is available, it will be used for authenticating connections to the Kubernetes API.
This needs to be enabled with Feature Gates because this is a beta feature. So you need to add
--feature-gates=RotateKubeletClientCertificate=true
When a kubelet starts up, if it is configured to bootstrap (using the --bootstrap-kubeconfig flag), it will use its initial certificate to connect to the Kubernetes API and issue a certificate signing request. You can view the status of certificate signing requests using:
kubectl get csr
Initially a certificate signing request from the kubelet on a node will have a status of Pending. If the certificate signing requests meets specific criteria, it will be auto approved by the controller manager, then it will have a status of Approved. Next, the controller manager will sign a certificate, issued for the duration specified by the --experimental-cluster-signing-duration parameter, and the signed certificate will be attached to the certificate signing requests.
The kubelet will retrieve the signed certificate from the Kubernetes API and write that to disk, in the location specified by --cert-dir. Then the kubelet will use the new certificate to connect to the Kubernetes API.
As the expiration of the signed certificate approaches, the kubelet will automatically issue a new certificate signing request, using the Kubernetes API. Again, the controller manager will automatically approve the certificate request and attach a signed certificate to the certificate signing request. The kubelet will retrieve the new signed certificate from the Kubernetes API and write that to disk. Then it will update the connections it has to the Kubernetes API to reconnect using the new certificate.
I have deployed a CockroachDB single instance cluster in my Kubernetes cluster on DO following this link https://www.cockroachlabs.com/docs/stable/orchestrate-a-local-cluster-with-kubernetes.html.
I followed this link to generate a server.crt file from https://kubernetes.io/docs/tasks/tls/managing-tls-in-a-cluster using the command:
kubectl get csr my-svc.my-namespace -o jsonpath='{.status.certificate}' \
| base64 --decode > server.crt
Not sure that gets me a client cert.
The following is my output for the command:
kubectl get csr
NAME AGE REQUESTOR CONDITION
default.client.root 44m system:serviceaccount:default:my-release-cockroachdb Approved,Issued
I need to connect to CockroachDB using my Java client. How do I generate a client cert and key so I can access CockroachDB from Java?
Thanks
There are multiple concerns here:
the certificate you are requesting from the k8s PKI will not have any of the fields required
the key format will not work for java clients
Let's address them one at a time:
Requesting a client certificate from the kubernetes PKI
A client certificate for user with CockroachDB must the subject's Common Name set to the username. eg: CN=root. This must also be properly configured to allow Client Authentication in the key usage.
In the kubernetes docs, we include an example to bring up a client within the same kubernetes cluster. The config for secure clients includes an init container that requests a client certificate and makes it available to the main job.
If your client is running in Kubernetes, I recommend adapting that config for your own client.
Key format for java clients
Java clients expect keys in PKCS#8 format, whereas the certificates output by both your command and the request-cert tool both output PEM encoded keys.
You can convert the key using openssl:
openssl pkcs8 -topk8 -inform PEM -outform DER -in client.myuser.key -out client.myuser.pk8
You can find more details on the CockroachDB Build a Java app page.
Is there a specific method or process to replace all of the certificates required in a Kubernetes 1.7 cluster? Is this even possible?
Client is worried about using certificate auth and not being able to revoke/blacklist certs properly if someone leaves.