I'm trying to generate another kubeconfig for a microk8s cluster. For this I chose the certificates approach and I'm using the following script to generate the certificates, create the certificate signing request and populate the kubeconfig file..
rm -rf ./certs_dir || true
mkdir ./certs_dir
sleep 5
openssl genrsa -out ./certs_dir/$USER_NAME.key 2048
openssl req -new -key ./certs_dir/$USER_NAME.key -out ./certs_dir/$USER_NAME.csr -subj "/CN=$USER_NAME"
CERT_S_REQ="
apiVersion: certificates.k8s.io/v1
kind: CertificateSigningRequest
metadata:
name: user-$USER_NAME-csr
spec:
groups:
- system:authenticated
request: $(cat $USER_NAME.csr | base64)
signerName: kubernetes.io/kube-apiserver-client
expirationSeconds: 864000000
usages:
- digital signature
- key encipherment
- client auth
"
export KUBECONFIG=../output/$NAME-kubeconfig.yaml
echo -e "$CERT_S_REQ" > ./certs_dir/user_csr.yaml
kubectl apply -f ./certs_dir/user_csr.yaml
kubectl get csr
kubectl certificate approve user-$USER_NAME-csr
sleep 10
kubectl get csr user-$USER_NAME-csr -o jsonpath='{.status.certificate}' | base64 -D > ./certs_dir/$USER_NAME.crt
kubectl create rolebinding user-$USER_NAME --clusterrole=cluster-admin --user=$USER_NAME
APISERVER=$(kubectl config view --raw -o 'jsonpath={..cluster.server}')
unset KUBECONFIG
kubectl config set-credentials "$USER_NAME" \
--client-certificate="./certs_dir/$USER_NAME.crt" \
--client-key="./certs_dir/$USER_NAME.key" \
--kubeconfig=../output/$USER_NAME.yaml \
--embed-certs=true
kubectl config set-cluster $CLUSTER_NAME --server=$APISERVER --kubeconfig=../output/$USER_NAME.yaml
kubectl config set-context default --user=$USER_NAME --cluster=$CLUSTER_NAME --kubeconfig=../output/$USER_NAME.yaml
kubectl config use-context default --kubeconfig=../output/$USER_NAME.yaml
Everything seems to work, but when trying to use the new kubeconfig file with the embedded certs it does not work, failing with following error whenever trying to execute a kubectl command
error: tls: private key does not match public key
Did I miss something?
I'm on MAC OS, running the microk8s cluster via multipass.
The microk8s cluster has the following enabled: ingress, storage, dns, rbac and also dashboard install: https://raw.githubusercontent.com/kubernetes/dashboard/v2.3.1/aio/deploy/recommended.yaml
Related
The end goal I'm trying to achieve is to create a kubernetes secret (potentially with a key) and a pod which uses that. But the catch is, the pod created should not be able to decode/decrypt the secret value without a particular key.
I have tried the secrets with data encryption at rest but that's not sufficient for my requirement.
Edit: I am trying to making this as step by step solution. (as asked by #Dawid in comments)
Encrypt your data using your-key (your encryption-logic, probably, in a script).
./encrypt.sh --key your-key --data your-data
Create a secret of this encrypted data
kubectl create secret generic your-secret-name --from-literal=secretdata=your-encrypted-data
You could add decryption logic like this in your pod ( either as a sidecar or initContainer)
# decrypt.sh will decode base64 then your decryption logic using your-key
./decrypt.sh --key your-key --data /var/my-secrets
Also you need to mount this secret as volume to your container .
spec:
containers:
- image: "image"
name: app
...
volumeMounts:
- mountPath: "/var/my-secrets"
name: my-secret
volumes:
- name: my-secret
secret:
secretName: your-secret-name
As answered by #Kiran here are the steps I followed to obtain the solution.
Encrypt using the openssl
echo -n "preetham" | openssl enc -e -aes-256-cbc -a -salt -pass pass:<PASSWORD>
Created the secret from the YAML file. preetham-secrets-test.yaml
apiVersion: v1
kind: Secret
metadata:
name: preetham-secrets
type: Opaque
stringData: # Using stringData instead data
username: U2FsdGVkX18VsbQaVpeqrCCJCDEd3LCbefT6nupChvw= # output from the step 1
Create the secret
kubectl apply -f preetham-secrets-test.yaml -n <NAMESPACE>
Mount the secret to volume and exec into the pod. Kubernetes reference
Inside the pod assuming the secret is mounted to /opt/mnt/secrets/.
bash-4.2# cat /opt/mnt/secrets/username
U2FsdGVkX18VsbQaVpeqrCCJCDEd3LCbefT6nupChvw=bash-4.2#
Decrypt the same using the openssl.( you may have to install the openssl based on the image using
bash-4.2# echo "U2FsdGVkX18VsbQaVpeqrCCJCDEd3LCbefT6nupChvw=" | openssl enc -d -aes-256-cbc -a -salt -pass pass:<PASSWORD>
preethambash-4.2#
I want to access an external k8s cluster which is running in private cloud. Do you have any idea how can I get these parameters? What should I do in order to generate them?
${CLIENT_CERTIFICATE_DATA}
fake-cert-file
fake-key-file
apiVersion: v1
kind: Config
clusters:
- cluster:
certificate-authority-data: ${CLUSTER_CA}
server: ${CLUSTER_ENDPOINT}
name: ${CLUSTER_NAME}
users:
- name: ${USER}
user:
client-certificate-data: $**{CLIENT_CERTIFICATE_DATA}**
contexts:
- context:
cluster: ${CLUSTER_NAME}
user:
client-certificate: **fake-cert-file**
client-key: **fake-key-file**
name: ${USER}-${CLUSTER_NAME}
current-context: ${USER}-${CLUSTER_NAME}
The steps to allow an access for a "bob" user are the followings:
Create a new CSR via openssl
openssl req -new -newkey rsa:4096 -nodes -keyout bob-k8s.key -out bob-k8s.csr -subj "/CN=bob/O=devops"
Create Kubernetes CertificateSigningRequest object
use
kubectl create –edit -f k8s-csr.yaml
and you should input the following
apiVersion: certificates.k8s.io/v1beta1
kind: CertificateSigningRequest
metadata:
name: bob-k8s-access
spec:
groups:
- system:authenticated
request: # replace with output from shell command: cat bob-k8s.csr | base64 | tr -d '\n'
usages:
- client auth
Verify your CSR object
kubectl get csr
Approve your certificate
kubectl certificate approve bob-k8s-access
Verify your Bob's certificate
kubectl get csr bob-k8s-access -o jsonpath='{.status.certificate}' | base64 --decode > bob-k8s-access.crt
Retrieve the cluster CA certificate
kubectl config view -o jsonpath='{.clusters[0].cluster.certificate-authority-data}' --raw | base64 --decode - > k8s-ca.crt
Setup Bob's kubeconfig file
$ kubectl config set-cluster $(kubectl config view -o jsonpath='{.clusters[0].name}') --server=$(kubectl config view -o jsonpath='{.clusters[0].cluster.server}') --certificate-authority=k8s-ca.crt --kubeconfig=bob-k8s-config --embed-certs
after this command a bob-k8s-config file should be created with Bob's .kube configuration
Setup Bob's credential accesses
kubectl config set-credentials bob --client-certificate=bob-k8s-access.crt --client-key=bob-k8s.key --embed-certs --kubeconfig=bob-k8s-config
Create a context in your config
kubectl config set-context bob --cluster=$(kubectl config view -o jsonpath='{.clusters[0].name}') --namespace=<ns-for-bob> --user=bob --kubeconfig=bob-k8s-config
Assign roles within the namespace
kubectl create rolebinding bob-admin --namespace=<ns-for-bob> --clusterrole=admin --user=bob
For more information about permissions, please, look at the Kubernetes configuration page
I've written this instructions starting from this guide that is more exhaustive!
command below gives an error: error: flag key is required
kubectl create secret tls k8-secret2 \
-n ingress-tls-test1 \
--cert ingress-tls-test1.pfx
I am able to create the secret using .crt and .key file:
kubectl create secret tls aks-ingress-tls \
--namespace ingress-basic \
--key aks-ingress-tls.key \
--cert aks-ingress-tls.crt
While creating k8s( up to v1.19) secret of type: kubernetes.io/tls, you must provide two keys; tls.key and tls.crt. If you use kubectl to create a secret, you can use --cert and --key flags to provide the values of those keys.
The public key certificate for --cert must be .PEM encoded (Base64-encoded DER format), and match the given private key for --key.
Since the .pfx certificate uses different encoding and stores all into a single encryptable file, you don't have separate certs and keys files to fulfil the requirements.
But you can create a secret of the type Opaque instead of TLS.
$ kubectl create secret generic k8-secret2 --from-file=crt.pfx=./ingress-tls-test1.pfx
I needed to create a kube tls secret from .pfx file today
Credits to: https://adolfi.dev/blog/tls-kubernetes/
## you will enter the pfx PW on on the CMD/terminal
openssl pkcs12 -in pfx-filename.pfx -nocerts -out key-filename.key
openssl rsa -in key-filename.key -out key-filename-decrypted.key
openssl pkcs12 -in pfx-filename.pfx -clcerts -nokeys -out crt-filename.crt ##remove clcerts to get the full chain in your cert
kubectl create secret tls your-secret-name --cert crt-filename.crt --key key-filename-decrypted.key
kubernetes v1.20 - you can create TLS secret imperatively:
Syntax:
kubectl create secret (command) (secret-name) (namespace) (cert) (key)
Example:
kubectl create secret tls webhook-server-tls --namespace webhook-demo --cert /root/keys/webhook-server-tls.crt --key /root/keys/webhook-server-tls.key
I'm trying to create a user in a Kubernetes cluster.
I spinned up 2 droplets on DigitalOcean using a Terraform script of mine.
Then I logged in the master node droplet using ssh:
doctl compute ssh droplet1
Following this, I created a new cluster and a namespace in it:
kubectl create namespace thalasoft
I created a user role in the role-deployment-manager.yml file:
kind: Role
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
namespace: thalasoft
name: deployment-manager
rules:
- apiGroups: ["", "extensions", "apps"]
resources: ["deployments", "replicasets", "pods"]
verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
and executed the command:
kubectl create -f role-deployment-manager.yml
I created a role grant in the rolebinding-deployment-manager.yml file:
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
name: deployment-manager-binding
namespace: thalasoft
subjects:
- kind: User
name: stephane
apiGroup: ""
roleRef:
kind: Role
name: deployment-manager
apiGroup: ""
and executed the command:
kubectl create -f rolebinding-deployment-manager.yml
Here is my terminal output:
Last login: Wed Dec 19 10:48:48 2018 from 90.191.151.182
root#droplet1:~# kubectl create namespace thalasoft
namespace/thalasoft created
root#droplet1:~# vi role-deployment-manager.yml
root#droplet1:~# kubectl create -f role-deployment-manager.yml
role.rbac.authorization.k8s.io/deployment-manager created
root#droplet1:~# vi rolebinding-deployment-manager.yml
root#droplet1:~# kubectl create -f rolebinding-deployment-manager.yml
rolebinding.rbac.authorization.k8s.io/deployment-manager-binding created
root#droplet1:~#
Now I'd like to first create a user in the cluster, and then configure the client kubectl with this user so as to operate from my laptop and avoid logging via ssh̀ to the droplet.
I know I can configure a user in the kubectl client:
# Create a context, that is, a user against a namespace of a cluster, in the client configuration
kubectl config set-context digital-ocean-context --cluster=digital-ocean-cluster --namespace=digital-ocean-namespace --user=stephane
# Configure the client with a user credentials
cd;
kubectl config set-credentials stephane --client-certificate=.ssh/id_rsa.pub --client-key=.ssh/id_rsa
But this is only some client side configuration as I understand.
UPDATE: I could add a user credentials with a certificate signed by the Kubernetes CA, running the following commands on the droplet hosting the Kubernetes master node:
# Create a private key
openssl genrsa -out .ssh/thalasoft.key 4096
# Create a certificate signing request
openssl req -new -key .ssh/thalasoft.key -out .ssh/thalasoft.csr -subj "/CN=stephane/O=thalasoft"
# Sign the certificate
export CA_LOCATION=/etc/kubernetes/pki/
openssl x509 -req -in .ssh/thalasoft.csr -CA $CA_LOCATION/ca.crt -CAkey $CA_LOCATION/ca.key -CAcreateserial -out .ssh/thalasoft.crt -days 1024
# Configure a cluster in the client
kubectl config set-cluster digital-ocean-cluster --server=https://${MASTER_IP}:6443 --insecure-skip-tls-verify=true
# Configure a user in the client
# Copy the key and the certificate to the client
scp -o "StrictHostKeyChecking no" root#165.227.171.72:.ssh/thalasoft.* .
# Configure the client with a user credentials
kubectl config set-credentials stephane --client-certificate=.ssh/thalasoft.crt --client-key=.ssh/thalasoft.key
# Create a context, that is, a user against a namespace of a cluster, in the client configuration
kubectl config set-context digital-ocean-context --cluster=digital-ocean-cluster --namespace=digital-ocean-namespace --user=stephane
But this is only some client side configuration as I understand.
What command I should use to create the user ?
Kubernetes doesn't provide user management. This is handled through x509 certificates that can be signed by your cluster CA.
First, you'll need to create a Key:
openssl genrsa -out my-user.key 4096
Second, you'll need to create a signing request:
openssl req -new -key my-user.key -out my-user.csr -subj "/CN=my-user/O=my-organisation"
Third, sign the certificate request:
openssl x509 -req -in my-user.csr -CA CA_LOCATION/ca.crt -CAkey CA_LOCATION/ca.key -CAcreateserial -out my-user.crt -days 500
ca.crt and ca.key is the same cert/key provided by kubeadm or within your master configuration.
You can then give this signed certificate to your user, along with their key, and then can configure access with:
kubectl config set-credentials my-user --client-certificate=my-user.crt --client-key=my-user.key
kubectl config set-context my-k8s-cluster --cluster=cluster-name --namespace=whatever --user=my-user
Bitnami provide a great resource that explains all of this:
https://docs.bitnami.com/kubernetes/how-to/configure-rbac-in-your-kubernetes-cluster/#use-case-1-create-user-with-limited-namespace-access
I'm trying to integrate Kubernetes cluster with Gitlab for using the Gitlab Review Apps feature.
Kubernetes cluster is created via Rancher 1.6
Running the kubectl get all from the kubernetes shell gives
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
svc/my-service LoadBalancer x.x.144.67 x.x.13.89 80:32701/TCP 30d
svc/kubernetes ClusterIP 10.43.0.1 <none> 443/TCP 30d
On the Gitlab CI / CD > Kubernetes page, we need to enter mainly 3 fields:
API URL
CA Certificate
Token
API URL
If I'm not wrong, we can get the Kubernetes API URL from Rancher Dashboard > Kubernetes > CLI > Generate Config and copy the server url under cluster
apiVersion: v1
kind: Config
clusters:
- cluster:
api-version: v1
insecure-skip-tls-verify: true
server: "https://x.x.122.197:8080/r/projects/1a7/kubernetes:6443"
CA Certificate & Token?
Now, the question is, where to get the CA Certificate (pem format) and the Token?
I tried all the ca.crt and token values from all the namespaces from the Kubernetes dashboard, but I'm getting this error on the Gitlab when trying to install Helm Tiller application:
Something went wrong while installing Helm Tiller
Can't start installation process
Here is how my secrets page look like
I'm also dying out with kubernetes and GitLab. I've created a couple single-node "clusters" for testing, one with minikube and another via kubeadm.
I answered this question on the GitLab forum but I'm posting my solution below:
API URL
According to the official documentation, the API URL is only https://hostname:port without trailing slash
List secrets
First, I listed the secrets as usual:
$ kubectl get secrets
NAME TYPE DATA AGE
default-token-tpvsd kubernetes.io/service-account-token 3 2d
k8s-dashboard-sa-token-XXXXX kubernetes.io/service-account-token 3 1d
Get the service token
$ kubectl -o json get secret k8s-dashboard-sa-token-XXXXX | jq -r '.data.token' | base64 -d
eyJhbGci ... sjcuNA8w
Get the CA certificate
Then I got the CA certificate directly from the JSON output via jq with a custom selector:
$ kubectl -o json get secret k8s-dashboard-sa-token-XXXXX | jq -r '.data."ca.crt"' | base64 -d - | tee ca.crt
-----BEGIN CERTIFICATE-----
MIICyDCCAbCgAwIBAgIBADANBgkqhkiG9w0BAQsFADAVMRMwEQYDVQQDEwprdWJl
... ... ... ... ... ...
FT55iMtPtFqAOnoYBCiLH6oT6Z1ACxduxPZA/EeQmTUoRJG8joczI0V1cnY=
-----END CERTIFICATE-----
Verity the CA certificate
With the CA certificate on hand you can verify as usual:
$ openssl x509 -in ca.crt -noout -subject -issuer
subject= /CN=kubernetes
issuer= /CN=kubernetes
$ openssl s_client -showcerts -connect 192.168.100.20:6443 < /dev/null &> apiserver.crt
$ openssl verify -verbose -CAfile ca.crt apiserver.crt
apiserver.crt: OK