Is there a specific method or process to replace all of the certificates required in a Kubernetes 1.7 cluster? Is this even possible?
Client is worried about using certificate auth and not being able to revoke/blacklist certs properly if someone leaves.
Related
I have four kubernetes clusters, and I want to check the expiration time of them with kubernetes-python-client.
I am following this page: https://github.com/kubernetes-client/python
Is there anyone know how to get it?
The apiserver certificate is generally handled out of band, either by your Kubernetes installer tool (kubeadm, rancher, talos, etc) or off-cluster in a load balancer layer. As such the K8s API won't help you with this.
That said, you can get the certificate of any HTTPS server in Python using ssl.get_server_certificate() (https://docs.python.org/3/library/ssl.html#ssl.get_server_certificate) along with other functions in the ssl module to parse the cert data and then look at the Not After timestamp.
I have been running K8s cluster(v1.13.5) for a year and the control plane certs and Kubelet certs are about to expire. I found a way to rotate all the control plane certs and I wanted to know how to rotate the Kubelet certs. Can someone help me to understand how to rotate the K certs for worker node and master (if needed)? This K8s cluster is deployed using Kubespray.
From Kubernetes version 1.8.0 a beta feature is available Certificate Rotation.
The kubelet uses certificates for authenticating to the Kubernetes API. By default, these certificates are issued with one year expiration so that they do not need to be renewed too frequently.
Kubernetes 1.8 contains kubelet certificate rotation, a beta feature that will automatically generate a new key and request a new certificate from the Kubernetes API as the current certificate approaches expiration. Once the new certificate is available, it will be used for authenticating connections to the Kubernetes API.
This needs to be enabled with Feature Gates because this is a beta feature. So you need to add
--feature-gates=RotateKubeletClientCertificate=true
When a kubelet starts up, if it is configured to bootstrap (using the --bootstrap-kubeconfig flag), it will use its initial certificate to connect to the Kubernetes API and issue a certificate signing request. You can view the status of certificate signing requests using:
kubectl get csr
Initially a certificate signing request from the kubelet on a node will have a status of Pending. If the certificate signing requests meets specific criteria, it will be auto approved by the controller manager, then it will have a status of Approved. Next, the controller manager will sign a certificate, issued for the duration specified by the --experimental-cluster-signing-duration parameter, and the signed certificate will be attached to the certificate signing requests.
The kubelet will retrieve the signed certificate from the Kubernetes API and write that to disk, in the location specified by --cert-dir. Then the kubelet will use the new certificate to connect to the Kubernetes API.
As the expiration of the signed certificate approaches, the kubelet will automatically issue a new certificate signing request, using the Kubernetes API. Again, the controller manager will automatically approve the certificate request and attach a signed certificate to the certificate signing request. The kubelet will retrieve the new signed certificate from the Kubernetes API and write that to disk. Then it will update the connections it has to the Kubernetes API to reconnect using the new certificate.
I have copied rancher config file to local kube config, and once I tried to connect, get an error
Unable to connect to the server: x509: certificate signed by unknown authority
I'm not admin of this cluster, and can't really change settings. So I googled that I can add
insecure-skip-tls-verify: true
And removed certificates, leaving only username and token, and it starts to work.
Can you explain me, is it safe to use it like so, and why do we need certs at all if it could work without it as well?
You may treat it as additional layer of security. If you allow someone ( in this case to yourself ) to connect to cluster and manage it without a need to have a proper certificate, just keep in mind you allow it for everyone else.
insecure-skip-tls-verify: true
is pretty self-explanatory - yes, it's insecure as it skips tls verification and it is not recommended on production. As you can read in documentation:
--insecure-skip-tls-verify If true, the server's certificate will not be checked for validity. This will make your HTTPS connections
insecure
Username and token provide some level of security as they are still required to be able to connect but it has nothing to do with establishing secure trusted connection. By default it can only be done by clients who have also proper certificate.
If you don't want to skip tls verification, you may want to try this solution. Only for kubernetes >= 1.15 use command kubeadm alpha certs renew all.
More about managing TLS Certificates in a Kubernetes Cluster you can read here.
We're using Fabric secure cluster and need client certificate for CI/CD tools.
I've created both Cluster primary certificate and client certificate with this script https://gist.github.com/kagarlickij/d63a4061a1066d3a85abcc658f0856f5
so both have been uploaded to the same Kay vault and both have been installed to local keystore on my machine.
I've added client certificate to my Fabric security settings (Authentication type = Admin client, Authorization method = Certificate thumbprint).
The problem is that I can connect (I'm using Connect-ServiceFabricCluster in PowerShell) to Fabric cluster with Cluster primary certificate but can't with Client certificate.
I'm getting this error: Connect-ServiceFabricCluster : FABRIC_E_SERVER_AUTHENTICATION_FAILED: 0x800b0109
Please advice what can be done?
Based on this link the corresponding error code for 0x800b0109 is:
A certificate chain processed, but terminated in a root certificate
which is not trusted by the trust provider.
You're using a self-signed certificate as client cert. I'm not sure it's supported as explained in the Service Fabric Security documentation, moreover you'll have to make sure the SSL certificate has been added inside your local Store.
Client X.509 certificates
Client certificates typically are not issued by a third-party CA.
Instead, the Personal store of the current user location typically
contains client certificates placed there by a root authority, with an
Intended Purposes value of Client Authentication. The client can use
this certificate when mutual authentication is required. Note
All management operations on a Service Fabric cluster require server certificates. Client certificates cannot be used for management.
I had the same issue managing my cluster through powershell, I only had 1 cert on the cluster (the one azure generates when creating the cluster) and I believe it is a client cert since I have to select it in my browser when managing the cluster.
Ultimately I had to add the self signed cert to my Root certificate store (in addition to my personal store where I already had it) to get the powershell module to stop complaining about it.
I have a Kubernetes cluster running in High Availability mode with 3 master nodes. When I try to run the DNS cluster add-on as-is, the kube2sky application errors with an x509 signed by unknown certificate authority message for the API Server service address (which in my case is 10.100.0.1). Reading through some of the GitHub issues, it looked like Tim Hockin had fixed this type of issue via using the default service account tokens available.
All 3 of my master nodes generate their own certificates for the secured API port, so is there something special I need to do configuration-wise on the API servers to get the CA certificate included in the default service account token?
It would be ideal to have the service IP of the API in the SAN field of all your server certificates.
If this is not possible in your setup, set the clusters{}.cluster.insecure-skip-tls-verify field to true in your kubeconfig file, or the pass the --insecure-skip-tls-verify flag to kubectl.
If you are trying to reach the API from within a pod you could use the secrets mounted via the Service Account. By default, if you use the default secret, the CA certificate and a signed token are mounted to /var/run/secrets/kubernetes.io/serviceaccount/ in every pod, and any client can use them from within the pod to communicate with the API. This would help you solving the unknown certificate authority error and provide you with an easy way to authenticate against your API servers at the same time.