I set up a k8s cluster in GKE with rbac enabled, and I install Istio into the cluster.
I follow this step (link) to create key/certs for the Istio ingress controller, and key/certs are stored as secret whose name is istio-ingress-certs.
Now I want to use RBAC to limit access to istio-ingress-certs, so that every component in istio-system is allowed to read the secret, but none could modify or delete it.
I create a secrets-rbac.yaml file, and run kubectl apply -f secrets-rbac.yaml, which creates a role to read the secret, and binds this role to all serviceaccounts in istio-system namespace.
To verify that a serviceaccount is not allowed to modify istio-ingress-certs. I use this command to test.
kubectl auth can-i edit secrets/istio-ingress-certs -n istio-system --as system:serviceaccount:istio-system:istio-pilot-service-account
I expect that the command would return false, but it returns true. I think I didn't set up rbac correctly in the yaml file, but I am not clear which part is not correct.
kind: Role
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
namespace: istio-system
name: istio-ingress-certs-reader
rules:
- apiGroups: ["*"]
resources: ["secrets"]
resourceNames: ["istio-ingress-certs"]
verbs: ["get"]
---
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
namespace: istio-system
name: read-istio-ingress-certs
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: istio-ingress-certs-reader
subjects:
- kind: Group
name: system:serviceaccounts:istio-system
apiGroup: rbac.authorization.k8s.io
- kind: Group
name: system:authenticated
apiGroup: rbac.authorization.k8s.io
- kind: Group
name: system:unauthenticated
apiGroup: rbac.authorization.k8s.io
Related
We have a shared tenant cluster, and we want our developers to be able to run kubectl top pods --namespace dev-namespace
But it seems to me that for top to be usable, you need to be able to run kubectl get nodes. But nodes are not namespaced.
Is there a solution?
We the namespace admin setup like this:
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: admin
subjects:
- apiGroup: rbac.authorization.k8s.io
kind: User
name: username#domain
And as a cluster admin I can run the top command, so metrics-server seems to be working fine.
Kubernetes has API group metrics.k8s.io, that you can use to give read permission for kubectl top pods -n <namespace>. If you grant get and list permissions for pods, you can run the command.
I tested the configuration below in a GKE cluster running Kubernetes 1.21 with kubectl top pod --as=system:serviceaccount:monitoring:test-account -n monitoring. With these permissions, I can only run kubectl top pod in the monitoring namespace, other commands will fail.
apiVersion: v1
kind: ServiceAccount
metadata:
name: test-account
namespace: monitoring
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: pod-reader
namespace: monitoring
rules:
- apiGroups: ["metrics.k8s.io"]
resources: ["pods"]
verbs: ["get", "list"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: read-pods
namespace: monitoring
subjects:
- kind: ServiceAccount
name: test-account
roleRef:
kind: Role
name: pod-reader
apiGroup: rbac.authorization.k8s.io
create a Cluster role and ClusterRoleBinding for see pods logs but i cant get access
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: read-permissions
rules:
- apiGroups:
- ""
resources:
- nodes
- services
- configmaps
- pods
- deployments
- endpoints
verbs:
- list
- get
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: read-permissions-binding
namespace: scourge
subjects:
- kind: Group
name: system:read-logs
apiGroup: rbac.authorization.k8s.io
roleRef:
kind: ClusterRole
name: read-permissions
apiGroup: rbac.authorization.k8s.io
kubectl auth can-i get pod --as="user" --as-group=read-pod
Warning: the server doesn't have a resource type 'pod'
Error from server (Forbidden): selfsubjectaccessreviews.authorization.k8s.io is forbidden:
User "idow" cannot create resource "selfsubjectaccessreviews"
in API group "authorization.k8s.io" at the cluster scope
if i ran this command with another group it succeed
I have an app which interacts with an existing service account ("the agent") on a designated namespace. I want the agent to be able to create additional service accounts and roles on other namespaces. Is there a way to do so?
I've already answered to this question in a comment section, but I've also decided to provide more comprehensive information with examples.
Background
Kubernetes includes RBAC (role-based access control) mechanism that enables you to specify which actions are permitted for specific user or group of users. From Kubernetes v1.6 RBAC is enabled by default.
There are four Kubernetes objects: Role, ClusterRole, RoleBinding and ClusterRoleBinding, that we can use to configure needed RBAC rules. Role and RoleBinding are namespaced and ClusterRole and ClusterRoleBinding are cluster scoped resources.
We use Role and RoleBinding to authorize user to namespaced resources and we use ClusterRole and ClusterRoleBinding for cluster wide resources.
However, we can also mix this resurces.
Below I will briefly describe common combinations.
NOTE: It is impossible to link ClusterRoleBindings with Role.
For every test case I created new test namespace and test-agent service account.
Role and RoleBinding
I created simple Role and RoleBinding in specific namespace:
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: test-role
namespace: test
rules:
- apiGroups:
- ""
resources:
- '*'
verbs:
- '*'
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: test-rolebinding
namespace: test
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: test-role
subjects:
- kind: ServiceAccount
name: test-agent
We can see that test-agent has access only to resources in test namespace:
$ kubectl auth can-i get pod -n test --as=system:serviceaccount:test:test-agent
yes
$ kubectl auth can-i get pod -n default --as=system:serviceaccount:test:test-agent
no
ClusterRole and RoleBinding
I created ClusterRole and RoleBinding:
NOTE: I didn't specify any namespace for ClusterRole.
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: test-clusterrole
rules:
- apiGroups:
- ""
resources:
- '*'
verbs:
- '*'
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: test-rolebinding
namespace: test
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: test-clusterrole
subjects:
- kind: ServiceAccount
name: test-agent
Now we can see, that if a ClusterRole is linked to a ServiceAccount using a RoleBinding, the ClusterRole permissions apply ONLY to the namespace in which this RoleBinding has been created:
$ kubectl auth can-i get pod -n test --as=system:serviceaccount:test:test-agent
yes
$ kubectl auth can-i get pod -n default --as=system:serviceaccount:test:test-agent
no
ClusterRole and ClusterRoleBinding
Finally I created ClusterRole and ClusterRoleBinding:
NOTE: I didn't specify any namespace for ClusterRole and ClusterRoleBinding.
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: test-clusterrole
rules:
- apiGroups:
- ""
resources:
- '*'
verbs:
- '*'
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: test-clusterrolebinding
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: test-clusterrole
subjects:
- kind: ServiceAccount
name: test-agent
namespace: test
Now we can see, that if a ClusterRole is linked to a ServiceAccount using a ClusterRoleBinding, the ClusterRole permissions apply to all namespaces:
$ kubectl auth can-i get pod -n test --as=system:serviceaccount:test:test-agent
yes
$ kubectl auth can-i get pod -n default --as=system:serviceaccount:test:test-agent
yes
$ kubectl auth can-i get pod -n kube-system --as=system:serviceaccount:test:test-agent
yes
Useful note: You can display all possible verbs for specific resource using
kubectl api-resources -o wide, e.g. to display all possible verbs for Deployment we can use:
$ kubectl api-resources -o wide | grep deployment
deployments deploy apps/v1 true Deployment [create delete deletecollection get list patch update watch]
I have a custom role related to a specific namespace. I want to create a service account that will have access to the Dashboard and only being able to see this namespace assigned to that role.
I have tried the following:
apiVersion: v1
kind: Namespace
metadata:
name: namespace-green
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: green
namespace: kubernetes-dashboard
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: role-green
namespace: namespace-green
rules:
- apiGroups: [""]
resources: ["*"]
verbs: ["*"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: testDashboard
namespace: kubernetes-dashboard
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: role-green
subjects:
- kind: ServiceAccount
name: green
namespace: kubernetes-dashboard
I retrieved the token with the following command:
kubectl -n kubernetes-dashboard describe secret $(kubectl -n kubernetes-dashboard get secret | grep green | awk '{print $1}')
When I login to the Dashboard I see only the default namespace although I have assigned the new namespace to that role.
I am not able to to figure out how to view the resources of the new namespace only and based on the permissions of the role the service account should have limited access.
You dont need to create a new role.
You can just create a RoleBinding to the 'edit' clusterrole with the new service account you have created and it will work as you expect it to. Also the access will be limited to just one namespace - kubernetes-dashboard
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: testDashboard
namespace: kubernetes-dashboard
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: edit
subjects:
- kind: ServiceAccount
name: green
namespace: kubernetes-dashboard
After that the you can use the same old token to test.
I am trying to delete tekton kubernetes resources in the context of a service account with an on-cluster kubernetes config, and am experiencing errors specific to accessing deletecollection with all tekton resources. An example error is as follows:
pipelines.tekton.dev is forbidden: User "system:serviceaccount:my-account:default" cannot deletecollection resource "pipelines" in API group "tekton.dev" in the namespace "my-namespace"
I have tried to apply RBAC to help here, but continue to experience the same errors. My RBAC attempt is as follows:
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: my-role
namespace: my-namespace
rules:
- apiGroups: ["tekton.dev"]
resources: ["pipelines", "pipelineruns", "tasks", "taskruns"]
verbs: ["get", "watch", "list", "delete", "deletecollection"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: my-role-binding
namespace: my-namespace
subjects:
- kind: User
name: system:serviceaccount:my-account:default
apiGroup: rbac.authorization.k8s.io
roleRef:
kind: Role
name: my-role
apiGroup: rbac.authorization.k8s.io
These RBAC configurations continue to result in the same error. Is this, or similar necessary? Are there any examples of RBAC when interfacing with, specifically deleting, tekton resources?
Given two namespaces my-namespace and my-account the default service account in the my-account namespace is correctly granted permissions to the deletecollection verb on pipelines in my-namespace.
You can verify this using kubectl auth can-i like this after applying:
$ kubectl -n my-namespace --as="system:serviceaccount:my-account:default" auth can-i deletecollection pipelines.tekton.de
yes
Verify that you have actually applied your RBAC manifests.
Change the RBAC as below
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: my-role
namespace: my-namespace
rules:
- apiGroups: ["tekton.dev"]
resources: ["pipelines", "pipelineruns", "tasks", "taskruns"]
verbs: ["get", "watch", "list", "delete", "deletecollection"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: my-rolebinding
namespace: my-namespace
subjects:
- kind: ServiceAccount
name: default
namespace: my-account
roleRef:
kind: Role
name: my-role
apiGroup: rbac.authorization.k8s.io
Few things to note:
Fixed subjects to use ServiceAccount from User. This is actually the cause of the failure because the service account was not granted the RBAC.
I assumed that you want to delete the Tekton resources in my-namespace by the default service account of my-account namespace . If it's different then changes in Role and RoleBinding need to be done accordingly.