I deployed HA K8s Cluster with 3 masters & 2 worker Nodes. I access my K8s Dashboard through kubectl client(local), kubectl proxy. My K8s Dashboard is accessed through tokens by some RBAC users, where they have limited access on namespaces & Cluster admin users. I want to give anonymous access to all my users for viewing the deployment logs i.e., to Kibana Dashboard(Add-on). Can anyone help me regarding this?
Below, I specified the required artifacts that are running on my cluster with their versions:
K8s version: 1.8.0
kibana: 5.6.4
elasticsearch-logging : 5.6.4
You can try creating a ClusterRoleBinding for some specific users. In my case, I am using LDAP authentication for accessing the Kubernetes API. I have assigned admin privileges to some users and readonly access to some specific users. Refer to the ClusterRoleBinding yaml below:
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: oidc-readonly-binding
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: system:aggregate-to-view
subjects:
- kind: User
name: https://dex.domain.com/dex#user1#domain.com
I am using dex tool for the LDAP authentication. You can try giving the RBAC username directly.
Related
I would like to read state of K8s using µK8s, but I don't want to have rights to modify anything. How to achieve this?
The following will give me full access:
microk8s.kubectl Insufficient permissions to access MicroK8s. You can either try again with sudo or add the user digital to the 'microk8s' group:
sudo usermod -a -G microk8s digital sudo chown -f -R digital ~/.kube
The new group will be available on the user's next login.
on Unix/Linux we can just set appropriate file/directory access
permission - just rx, decrease shell limits (like max memory/open
file descriptors), decrease process priority (nice -19). We are
looking for similar solution for K8S
This kind of solutions in Kubernetes are handled via RBAC (Role-based access control). RBAC prevents unauthorized users from viewing or modifying the cluster state. Because the API server exposes a REST interface, users perform actions by sending HTTP requests to the server. Users authenticate themselves by including credentials in the request (an authentication token, username and password, or a client certificate).
As for REST clients you get GET, POST, PUT,DELETE etc. These are send to specific URL paths that represents specific REST API resources (Pods, Services, Deployments and so).
RBAC auth is configured with two groups:
Roles and ClusterRoles - this specify which actions/verbs can be performed
RoleBinding and ClusterRoleBindings - this bind the above roles to a user, group or service account.
As you might already find out the ClusterRole is the one your might be looking for. This will allow to restrict specific user or group against the cluster.
In the example below we are creating ClusterRole that can only list pods. The namespace is omitted since ClusterRoles are not namepsaced.
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: pod-viewer
rules:
- apiGroups: [""]
resources: ["pods"]
verbs: ["list"]
This permission has to be bound then via ClusterRoleBinding :
apiVersion: rbac.authorization.k8s.io/v1
# This cluster role binding allows anyone in the "manager" group to list pods in any namespace.
kind: ClusterRoleBinding
metadata:
name: list-pods-global
subjects:
- kind: Group
name: manager # Name is case sensitive
apiGroup: rbac.authorization.k8s.io
roleRef:
kind: ClusterRole
name: pod-viewer
apiGroup: rbac.authorization.k8s.io
Because you don't have the enough permissions on your own you have to reach out to appropriate person who manage those to create user for you that has the ClusterRole: View. View role should be predefined already in cluster ( kubectl get clusterrole view)
If you wish to read more Kubernetes docs explains well its whole concept of authorization.
I have the problem that I can log on to my dashboard via OIDC, but then the oidc group information is not mapped correctly and I cannot access the corresponding resources.
Basic setup
K8s version: 1.19.0
K8s setup: 1 master + 2 worker nodes
Based on Debian 10 VMs
CNI: Calico
Louketo Proxy as OIDC proxy
OIDC: Keycloak Server (Keycloak X [Quarkus])
Configurations
I have configured the K8s apiserver with these parameters.
kube-apiserver.yaml
- --oidc-issuer-url=https://test.test.com/auth/realms/Test
- --oidc-client-id=test
- --oidc-username-claim=preferred_username
- --oidc-username-prefix="oidc:"
- --oidc-groups-claim=groups
- --oidc-groups-prefix="oidc:"
ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: "test-cluster-admin"
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: cluster-admin
subjects:
- apiGroup: rbac.authorization.k8s.io
kind: Group
name: "Test"
I used the following louketo parameters
Louketo Proxy
/usr/bin/louketo-proxy --discovery-url=$OIDC_DISCOVERY_URL --client-id=$OIDC_CLIENT_ID --client-secret=$OIDC_CLIENT_SECRET -listen=$OIDC_LISTEN_URL --encryption-key=$OIDC_ENCRYPTION_KEY --redirection-url=$OIDC_REDIRECTION_KEY --enable-refresh-tokens=true --upstream-url=$OIDC_UPSTREAM_URL --enable-metrics
I get the following error message inside the dashboard.
K8s error
replicasets.apps is forbidden: User "\"oidc:\"<user_name>" cannot list resource "replicasets" in API group "apps" in the namespace "default"
I hope you can help me with this problem, I already tried most of the manuals from the internet, but haven't found a solution yet.
PS: I have done the corresponding group mapping in the Keycloak server and also validated that the group entry is transferred.
If you are facing the same challenge as I did and you want to integrate Keycloak into your K8s cluster, share the dashboard and connect it to Keycloak, you can find my configuration below. Within my cluster I use the Louketo Proxy as interface between Kubernetes and Keycloak. The corresponding configuration of the deployment is not included in this post.
Keycloak
I want to start with the configuration of Keycloak. In the first step I created a corresponding client with the following settings.
After that I created the two group membership and audience (needed by the louketo proxy) mappers.
The exact settings of the mappers can be taken from the two images.
Group membership mapping
Audience mapping
Kubernetes
In the second step I had to update the api server manifest and create the RoleBinding and ClusterRoleBinding within the Kubernetes cluster.
Api server manifest (default path: /etc/kubernetes/manifests/kube-apiserver.yaml)
- --oidc-issuer-url=https://test.test.com/auth/realms/Test
- --oidc-client-id=test
- --oidc-username-claim=preferred_username
- --oidc-username-prefix="oidc:"
- --oidc-groups-claim=groups
- --oidc-groups-prefix="oidc:"
RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: "test"
namespace: "kubernetes-dashboard"
subjects:
- kind: User
name: "\"oidc:\"Test"
namespace: "kube-system"
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: cluster-admin
ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: "test"
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: cluster-admin
subjects:
- apiGroup: rbac.authorization.k8s.io
kind: Group
name: "\"oidc:\"Test"
#Community I hope I can help you with this configuration. If you have any questions, feel free to ask me.
This is a community wiki answer aimed to approach the issue from the Kubernetes side. Any one familiar with the possible Keycloak group/role mapping solution feel free to edit it.
The error you see means that the service account for OIDC doesn't have the proper privileges to list replicasets in the default namespace. The easiest way out of it would be to simply setup the ServiceAccount, ClusterRole and ClusterRoleBinding from scratch and make sure it has the proper privileges. For example, you can create a clusterrolebinding with permissions “admin” by executing:
kubectl create clusterrolebinding OIDCrolebinding - -clusterrole=admin - - group=system:serviceaccounts:OIDC
The same can be done for the ClusterRole:
kubectl create clusterrole OIDC --verb=get,list,watch --resource=replicasets --namespace=default
More examples of how to use the kubectl create in this scenario can be found here.
Here you can find a whole official guide regarding the RBAC Authorization.
EDIT:
Also, please also check if your ClusterRoleBinding for the "\"oidc:\"<user_name>" is in the "default" namespace.
How can we login to a AKS cluster created , by using service account?
We are asked to execute kubectl create clusterrolebinding add-on-cluster-admin ......... but we are not aware how to use this and login to the created cluster in Azure
you can use this quick start tutorial: https://learn.microsoft.com/en-us/azure/aks/kubernetes-walkthrough#connect-to-the-cluster
basically you need to install kubectl:
az aks install-cli
and pull credentials for AKS:
az aks get-credentials --resource-group myResourceGroup --name myAKSCluster
As per ducemtaion:
User accounts vs service accounts
Kubernetes distinguishes between the concept of a user account and a service account for a number of reasons:
User accounts are for humans. Service accounts are for processes, which run in pods.
User accounts are intended to be global. Names must be unique across all namespaces of a cluster, future user resource will not be namespaced. Service accounts are namespaced.
Typically, a cluster’s User accounts might be synced from a corporate database, where new user account creation requires special privileges and is tied to complex business processes. Service account creation is intended to be more lightweight, allowing cluster users to create service accounts for specific tasks (i.e. principle of least privilege).
Auditing considerations for humans and service accounts may differ.
A config bundle for a complex system may include definition of various service accounts for components of that system. Because service accounts can be created ad-hoc and have namespaced names, such config is portable.
As an example:
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: secret-reader
rules:
- apiGroups: [""]
resources: ["secrets"]
verbs: ["get", "watch", "list"]
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: read-secrets-global
subjects:
- kind: User
name: manager
apiGroup: rbac.authorization.k8s.io
roleRef:
kind: ClusterRole
name: secret-reader
apiGroup: rbac.authorization.k8s.io
you can find other helpful information here, in official kubernetes documentation, and Azure Kubernetes Service AKS
According to Spring Cloud Kubernetes docs, in order to discover services/pods in RBAC enabled Kubernetes distros:
you need to make sure a pod that runs with spring-cloud-kubernetes has access to the Kubernetes API. For any service accounts you assign to a deployment/pod, you need to make sure it has the correct roles. For example, you can add cluster-reader permissions to your default service account depending on the project you’re in.
What are cluster-reader permissions in order to discover services/pods?
Error I receiving is:
io.fabric8.kubernetes.client.KubernetesClientException: Failure executing: GET at: https://x.x.x.x/api/v1/namespaces/jx-staging/services.
Message: Forbidden!Configured service account doesn't have access.
Service account may have been revoked. services is forbidden:
User "system:serviceaccount:jx-staging:default" cannot list services in the namespace "jx-staging"
Read endpoints and services seems to be a bare minimum for Spring Cloud Kubernetes to discover pods and services.
Example adds permissions to default service account in default namespace.
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: cluster-read-role
rules:
- apiGroups:
- ""
resources:
- endpoints
- pods
- services
- configmaps
verbs:
- get
- list
- watch
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: cluster-read-rolebinding
subjects:
- kind: ServiceAccount
name: default
namespace: default
roleRef:
kind: ClusterRole
name: cluster-read-role
apiGroup: rbac.authorization.k8s.io
Kubernetes generally categorizes roles into two types:
Role: This are specific to the namespace to which they are granted
ClusterRole: Applies to the whole cluster, meaning that it applies to all namespaces
So what the Spring Cloud Kubernetes docs mean there is that in order to be able to read properly discover services/pods across all namespaces, the ServiceAccount which will be associated with the application should have a ClusterRole that allows it to read Pods, Services etc.
This part of the Kubernetes docs (which also contains great examples) is a must-read for a general understanding of Kubernetes RBAC.
Desired Outcome:
I want to set up a CSV file with userids and passwords and access Kubernetes Dashboard as a full admin, preferably from anywhere with a browser. I am just learning kubernetes and want to experiment with cluster management, deployments, etc. This is just for learning and is not a Production Setup. I am using Kubernetes version 1.9.2 and created a 3-machine cluster (master and 2 workers)
Background/What I've done so far:
I read the Dashboard README and I created an admin-user and admin-role-binding with the files shown below. I can then use the kubectl describe secret command to get the admin user's token. I run kubectl proxy on the cluster master and authenticate to the Dashboard with that token using a browser running on the cluster master. All of this works.
admin-user.yaml:
apiVersion: v1
kind: ServiceAccount
metadata:
name: admin-user
namespace: kube-system
admin-role-binding.yaml:
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
name: admin-user
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: cluster-admin
subjects:
- kind: ServiceAccount
name: admin-user
namespace: kube-system
I can login to the dashboard as admin IF:
I run kubectl proxy
I access the dashboard with a browser where I ran command (1)
I use the "token" option to login and paste the admin user's token which I get using the kubectl describe secret command.
What I'd like to do:
Set up a CSV file with userids/passwords
Login as admin with userid/password
Be able to login from anywhere
To that end, I created a CSV file, e.g. /home/chris/myusers.txt:
mypasswd,admin,42
I did not know what value to use for id so I just punted with 42.
I then edited the file:
/etc/kubernetes/manifests/kube-apiserver.yaml and adding this line:
--basic-auth-file=/home/chris/myusers.txt
and then restarting kubelet:
sudo systemctl restart kubelet
However when I did that, my cluster stopped working and I couldn't access the Dashboard so I reverted back where I still use the admin user's token.
My questions are:
Is it possible to do what I'm trying to do here?
What id values do I use in the user CSV file? What groups would I specify?
What other changes do I need to make to get all of this to work? If I modify the apiserver manifest to use a file with userids/passwords, does that mess up the rest of the configuration for my cluster?
You can try this one it is working for me. Taking reference from here.
volumeMounts:
- mountPath: /etc/kubernetes/auth.csv
name: kubernetes-dashboard
readOnly: true
volumes:
- hostPath:
path: /etc/kubernetes/auth.csv
name: kubernetes-dashboard
How to config simple login/pass authentication for kubernetes desktop UI