How to create a user in a Kubernetes cluster? - kubernetes

I'm trying to create a user in a Kubernetes cluster.
I spinned up 2 droplets on DigitalOcean using a Terraform script of mine.
Then I logged in the master node droplet using ssh:
doctl compute ssh droplet1
Following this, I created a new cluster and a namespace in it:
kubectl create namespace thalasoft
I created a user role in the role-deployment-manager.yml file:
kind: Role
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
namespace: thalasoft
name: deployment-manager
rules:
- apiGroups: ["", "extensions", "apps"]
resources: ["deployments", "replicasets", "pods"]
verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
and executed the command:
kubectl create -f role-deployment-manager.yml
I created a role grant in the rolebinding-deployment-manager.yml file:
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
name: deployment-manager-binding
namespace: thalasoft
subjects:
- kind: User
name: stephane
apiGroup: ""
roleRef:
kind: Role
name: deployment-manager
apiGroup: ""
and executed the command:
kubectl create -f rolebinding-deployment-manager.yml
Here is my terminal output:
Last login: Wed Dec 19 10:48:48 2018 from 90.191.151.182
root#droplet1:~# kubectl create namespace thalasoft
namespace/thalasoft created
root#droplet1:~# vi role-deployment-manager.yml
root#droplet1:~# kubectl create -f role-deployment-manager.yml
role.rbac.authorization.k8s.io/deployment-manager created
root#droplet1:~# vi rolebinding-deployment-manager.yml
root#droplet1:~# kubectl create -f rolebinding-deployment-manager.yml
rolebinding.rbac.authorization.k8s.io/deployment-manager-binding created
root#droplet1:~#
Now I'd like to first create a user in the cluster, and then configure the client kubectl with this user so as to operate from my laptop and avoid logging via ssh̀ to the droplet.
I know I can configure a user in the kubectl client:
# Create a context, that is, a user against a namespace of a cluster, in the client configuration
kubectl config set-context digital-ocean-context --cluster=digital-ocean-cluster --namespace=digital-ocean-namespace --user=stephane
# Configure the client with a user credentials
cd;
kubectl config set-credentials stephane --client-certificate=.ssh/id_rsa.pub --client-key=.ssh/id_rsa
But this is only some client side configuration as I understand.
UPDATE: I could add a user credentials with a certificate signed by the Kubernetes CA, running the following commands on the droplet hosting the Kubernetes master node:
# Create a private key
openssl genrsa -out .ssh/thalasoft.key 4096
# Create a certificate signing request
openssl req -new -key .ssh/thalasoft.key -out .ssh/thalasoft.csr -subj "/CN=stephane/O=thalasoft"
# Sign the certificate
export CA_LOCATION=/etc/kubernetes/pki/
openssl x509 -req -in .ssh/thalasoft.csr -CA $CA_LOCATION/ca.crt -CAkey $CA_LOCATION/ca.key -CAcreateserial -out .ssh/thalasoft.crt -days 1024
# Configure a cluster in the client
kubectl config set-cluster digital-ocean-cluster --server=https://${MASTER_IP}:6443 --insecure-skip-tls-verify=true
# Configure a user in the client
# Copy the key and the certificate to the client
scp -o "StrictHostKeyChecking no" root#165.227.171.72:.ssh/thalasoft.* .
# Configure the client with a user credentials
kubectl config set-credentials stephane --client-certificate=.ssh/thalasoft.crt --client-key=.ssh/thalasoft.key
# Create a context, that is, a user against a namespace of a cluster, in the client configuration
kubectl config set-context digital-ocean-context --cluster=digital-ocean-cluster --namespace=digital-ocean-namespace --user=stephane

But this is only some client side configuration as I understand.
What command I should use to create the user ?
Kubernetes doesn't provide user management. This is handled through x509 certificates that can be signed by your cluster CA.
First, you'll need to create a Key:
openssl genrsa -out my-user.key 4096
Second, you'll need to create a signing request:
openssl req -new -key my-user.key -out my-user.csr -subj "/CN=my-user/O=my-organisation"
Third, sign the certificate request:
openssl x509 -req -in my-user.csr -CA CA_LOCATION/ca.crt -CAkey CA_LOCATION/ca.key -CAcreateserial -out my-user.crt -days 500
ca.crt and ca.key is the same cert/key provided by kubeadm or within your master configuration.
You can then give this signed certificate to your user, along with their key, and then can configure access with:
kubectl config set-credentials my-user --client-certificate=my-user.crt --client-key=my-user.key
kubectl config set-context my-k8s-cluster --cluster=cluster-name --namespace=whatever --user=my-user
Bitnami provide a great resource that explains all of this:
https://docs.bitnami.com/kubernetes/how-to/configure-rbac-in-your-kubernetes-cluster/#use-case-1-create-user-with-limited-namespace-access

Related

Microk8s : Generating Auth Certificates

I'm trying to generate another kubeconfig for a microk8s cluster. For this I chose the certificates approach and I'm using the following script to generate the certificates, create the certificate signing request and populate the kubeconfig file..
rm -rf ./certs_dir || true
mkdir ./certs_dir
sleep 5
openssl genrsa -out ./certs_dir/$USER_NAME.key 2048
openssl req -new -key ./certs_dir/$USER_NAME.key -out ./certs_dir/$USER_NAME.csr -subj "/CN=$USER_NAME"
CERT_S_REQ="
apiVersion: certificates.k8s.io/v1
kind: CertificateSigningRequest
metadata:
name: user-$USER_NAME-csr
spec:
groups:
- system:authenticated
request: $(cat $USER_NAME.csr | base64)
signerName: kubernetes.io/kube-apiserver-client
expirationSeconds: 864000000
usages:
- digital signature
- key encipherment
- client auth
"
export KUBECONFIG=../output/$NAME-kubeconfig.yaml
echo -e "$CERT_S_REQ" > ./certs_dir/user_csr.yaml
kubectl apply -f ./certs_dir/user_csr.yaml
kubectl get csr
kubectl certificate approve user-$USER_NAME-csr
sleep 10
kubectl get csr user-$USER_NAME-csr -o jsonpath='{.status.certificate}' | base64 -D > ./certs_dir/$USER_NAME.crt
kubectl create rolebinding user-$USER_NAME --clusterrole=cluster-admin --user=$USER_NAME
APISERVER=$(kubectl config view --raw -o 'jsonpath={..cluster.server}')
unset KUBECONFIG
kubectl config set-credentials "$USER_NAME" \
--client-certificate="./certs_dir/$USER_NAME.crt" \
--client-key="./certs_dir/$USER_NAME.key" \
--kubeconfig=../output/$USER_NAME.yaml \
--embed-certs=true
kubectl config set-cluster $CLUSTER_NAME --server=$APISERVER --kubeconfig=../output/$USER_NAME.yaml
kubectl config set-context default --user=$USER_NAME --cluster=$CLUSTER_NAME --kubeconfig=../output/$USER_NAME.yaml
kubectl config use-context default --kubeconfig=../output/$USER_NAME.yaml
Everything seems to work, but when trying to use the new kubeconfig file with the embedded certs it does not work, failing with following error whenever trying to execute a kubectl command
error: tls: private key does not match public key
Did I miss something?
I'm on MAC OS, running the microk8s cluster via multipass.
The microk8s cluster has the following enabled: ingress, storage, dns, rbac and also dashboard install: https://raw.githubusercontent.com/kubernetes/dashboard/v2.3.1/aio/deploy/recommended.yaml

Parameters to access multiple clusters

I want to access an external k8s cluster which is running in private cloud. Do you have any idea how can I get these parameters? What should I do in order to generate them?
${CLIENT_CERTIFICATE_DATA}
fake-cert-file
fake-key-file
apiVersion: v1
kind: Config
clusters:
- cluster:
certificate-authority-data: ${CLUSTER_CA}
server: ${CLUSTER_ENDPOINT}
name: ${CLUSTER_NAME}
users:
- name: ${USER}
user:
client-certificate-data: $**{CLIENT_CERTIFICATE_DATA}**
contexts:
- context:
cluster: ${CLUSTER_NAME}
user:
client-certificate: **fake-cert-file**
client-key: **fake-key-file**
name: ${USER}-${CLUSTER_NAME}
current-context: ${USER}-${CLUSTER_NAME}
The steps to allow an access for a "bob" user are the followings:
Create a new CSR via openssl
openssl req -new -newkey rsa:4096 -nodes -keyout bob-k8s.key -out bob-k8s.csr -subj "/CN=bob/O=devops"
Create Kubernetes CertificateSigningRequest object
use
kubectl create –edit -f k8s-csr.yaml
and you should input the following
apiVersion: certificates.k8s.io/v1beta1
kind: CertificateSigningRequest
metadata:
name: bob-k8s-access
spec:
groups:
- system:authenticated
request: # replace with output from shell command: cat bob-k8s.csr | base64 | tr -d '\n'
usages:
- client auth
Verify your CSR object
kubectl get csr
Approve your certificate
kubectl certificate approve bob-k8s-access
Verify your Bob's certificate
kubectl get csr bob-k8s-access -o jsonpath='{.status.certificate}' | base64 --decode > bob-k8s-access.crt
Retrieve the cluster CA certificate
kubectl config view -o jsonpath='{.clusters[0].cluster.certificate-authority-data}' --raw | base64 --decode - > k8s-ca.crt
Setup Bob's kubeconfig file
$ kubectl config set-cluster $(kubectl config view -o jsonpath='{.clusters[0].name}') --server=$(kubectl config view -o jsonpath='{.clusters[0].cluster.server}') --certificate-authority=k8s-ca.crt --kubeconfig=bob-k8s-config --embed-certs
after this command a bob-k8s-config file should be created with Bob's .kube configuration
Setup Bob's credential accesses
kubectl config set-credentials bob --client-certificate=bob-k8s-access.crt --client-key=bob-k8s.key --embed-certs --kubeconfig=bob-k8s-config
Create a context in your config
kubectl config set-context bob --cluster=$(kubectl config view -o jsonpath='{.clusters[0].name}') --namespace=<ns-for-bob> --user=bob --kubeconfig=bob-k8s-config
Assign roles within the namespace
kubectl create rolebinding bob-admin --namespace=<ns-for-bob> --clusterrole=admin --user=bob
For more information about permissions, please, look at the Kubernetes configuration page
I've written this instructions starting from this guide that is more exhaustive!

How do I get the K8s `ca.crt` and `ca.key` running K8s on a service provider(GKE)

Note: I am not running locally on Minikube or something, but GKE - but could be any provider.
I want to be able to create users/contexts in K8s with openssl:
openssl x509 -req -in juan.csr -CA CA_LOCATION/ca.crt -CAKey CA_LOCATION/ca.key -CAcreateserial -out juan.crt -days 500
How do I get the K8s ca.crt and ca.key? - I found this for ca.crt, but is this the way and still missing the ca.key?
kubectl get secret -o jsonpath="{.items[?(#.type==\"kubernetes.io/service-account-token\")].data['ca\.crt']}" | base64 --decode
And, other way than logging into master node /etc/kubernetes/pki/.
I would suggest viewing the following documentation on how to generate a ca.key and ca.crt for your kubernetes cluster. Please keep in mind this is not an official google document, however this may help you achieve what you are looking for.
Here are the commands found in the document.
Generate ca.key: openssl genrsa -out ca.key 2048
Generate ca.cert: openssl req -x509 -new -nodes -key ca.key -subj "/CN=${MASTER_IP}" -days 10000 -out ca.crt
EDIT
I found 2 unsupported documents [1] [2] on generating a certificate and key with openssl, it should be applicable with kubernetes.
You don't. Google do not expose the keys as per the documentation.
Specifically and I quote:
An internal Google service manages root keys for this CA, which are non-exportable. This service accepts certificate signing requests, including those from the kubelets in each GKE cluster. Even if the API server in a cluster were compromised, the CA would not be compromised, so no other clusters would be affected.
Using Kubernetes v1.19 and higher you can sign your CSR using the Kubernetes API itself as referenced here.
Encode your CSR $ ENCODED=$(cat mia.csr | base64 | tr -d "\n").
Then post it to k8s:
$ cat <<EOF | kubectl apply -f -
apiVersion: certificates.k8s.io/v1
kind: CertificateSigningRequest
metadata:
name: mia
spec:
request: $ENCODED
signerName: kubernetes.io/kube-apiserver-client
#expirationSeconds: 86400 # Only supported on >=1.22
usages:
- client auth
EOF
And then approve the CSR:
$ kubectl certificate approve mia
certificatesigningrequest.certificates.k8s.io/mia approved
Then download the signed certificate:
kubectl get csr mia -o jsonpath='{.status.certificate}'| base64 -d > mia.crt
I wrote an example of the end to end flow here.

How to have a single user with access to multiple namespaces with separate certs for each namespace

I'm trying to setup some basic Authorization and Authentication for various users to access a shared K8s cluster.
Requirement: Multiple users can have access to multiple namespaces with a separate set of cert and keys for each of them.
Proposal:
openssl genrsa -out $PRIV_KEY 2048
# Generate CSR
openssl req -new -key $PRIV_KEY -out $CSR -subj "/CN=$USER"
# Create k8s CSR
K8S_CSR=user-request-$USER-$NAMESPACE_NAME-admin
cat <<EOF >./$K8S_CSR.yaml
apiVersion: certificates.k8s.io/v1beta1
kind: CertificateSigningRequest
metadata:
name: $K8S_CSR
spec:
groups:
- system:authenticated
request: $(cat $CSR | base64 | tr -d '\n')
usages:
- digital signature
- key encipherment
- client auth
EOF
kubectl create -n $NAMESPACE_NAME -f $K8S_CSR.yaml
# Approve K8s CSR
kubectl certificate approve $K8S_CSR
# Fetch User Certificate
kubectl get csr $K8S_CSR -o jsonpath='{.status.certificate}' | base64 -d > $USER-$NAMESPACE_NAME-admin.crt
# Create Admin Role Binding
kubectl create rolebinding $NAMESPACE_NAME-$USER-admin-binding --clusterrole=admin --user=$USER --namespace=$NAMESPACE_NAME
Problem:
The user cert and/or pkey are not specific to this namespace.
If I just create another rolebinding for the same user in a different namespace he will able to authenticate. How do I prevent that from happening?
The CA in Kubernetes is a cluster-wide CA (not namespaced) so you won't be able to create certs tied to a specific namespace.
system:authenticated and system:unauthenticated are built-in groups in Kubernetes to identify whether a Role or ClusterRole is authenticated. And you can't directly manager groups or users in Kubernetes. You will have to configure an alternative cluster authentication methods to take advantage of users and groups. For example, a static token file or OpenID
Then you can, restrict users or groups defined in your identity provider to a Role that doesn't allow them to create either another Role or RoleBinding, that way they are not able to give themselves access to other namespaces and only the cluster-admin is the one who decides which RoleBindings (or namespaces) that specific user is part of.
For example, in your Role:
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
namespace: mynamespace
name: myrole
rules:
- apiGroups: ["extensions", "apps"]
resources: ["deployments"] <== never include role, clusterrole, rolebinding, and clusterrolebinding.
verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
Another alternative is to restrict base on a Service Account Token by RoleBinding to the service account.

Can't pull image with Error x509: certificate signed by unknown authority in minikube

I practice about kubernetes using minikube
I make docker-registry as a pods and I create docker-registry service
edit /etc/docker/daemon.json
{
"insecure-registries":["192.168.99.100:30050"]
}
edit openssl.conf
[v3_req]
subjectAltName = IP:192.168.99.100
And I make certificate
openssl genrsa -out my.crt
openssl req -x509 -new -nodes -key my.key -subj "CN=192.168.99.100:30050" -days 5000 -out my.crt
create registry-tls-secret
kubectl create secret generic registry-tls-secret --from-file=my.crt=my.crt --from-file=my.key=my.key
and make directory and copy my.crt file into the directory
/etc/docker/certs.d/192.168.99.100:30050
So I can push & pull 192.168.99.100:30050/[image]:[tag] on the host
And After I try to make hello-world pods
hello-world image is into the docker-registry(192.168.99.100:30050) already
I create secrets docker-registry regcred
kubectl create secret docker-registry regcred --docker-server=192.168.99.100:30050 --docker-usernmae=<user-name> --docker-password=<user-password> --docker-email=<user-email>
and write helloworld-deployment.yaml
...
image: 192.168.99.100:30050/hello-world:v1
...
imagePullSecrets:
-name: regcred
...
Finally I apply helloworld-deployment.yaml
But I got an error message that
Failed to pull image "192.168.99.100:30050/hello-world:v1": rpc error: code = Unknown desc = Error response from daemon: Get https://192.168.99.100:30050/v2/: x509: certificate signed by unknown authority
I don't really what I'm missing... please help me...
There is no standard for storing a port number in the certificate so Common Name of the certificate should not contain a port. Create Certificate with CN=192.168.99.100 and repeat same steps. For more information on common name refer here.
Make sure you Copy the certificate data to /etc/docker/certs.d/192.168.99.100:30050/ca.crt.