First off, I'm aware of the Kubernetes RBAC method. My question is: is there a way to create Kubernetes resources that can only be read and/or written by a specific Role (or a ClusterRole)?
For example, let's say I have a Kubernetes Secret. I want this Secret to be bound to a specific ClusterRole, then only a ServiceAccount bound to this specific ClusterRole could read it. Is there a way to set up something like that?
Edit: it looks like what I want here is not possible. Kubernetes RBAC was designed to GRANT access to certain resources. I wanted to DENY access based on a specific group (or set of rules).
You can use the RBAC for managing the Role-based access in K8s
For example, let's say I have a Kubernetes Secret. I want this Secret
to be bound to a specific ClusterRole, so only a ServiceAccount bound
to this specific ClusterRole could read it. Is there a way to set up
something like that?
No, you can not use the ClusterRole for granular level access, however, you can create some Role to restrict secret.
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
namespace: default
name: secret-read-role
rules:
- apiGroups: ["*"]
resources: ["secret"]
verbs: ["get", "watch", "list"]
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: secret-read-sa
---
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: secret-read-rolebinding
namespace: default
subjects:
- kind: ServiceAccount
name: secret-read-sa
apiGroup: ""
roleRef:
kind: Role
name: secret-read-role
apiGroup: ""
Checkout about the resourceNames you can also give a name or pattern in name so this way it might be helpful to attach a specific secret to Role.
- apiGroups: [""]
resources: ["namespaces"]
verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
resourceNames: ["userA-*"]
If you planning to Go with RBAC you can use the RBAC manager for better management : https://github.com/FairwindsOps/rbac-manager
Extra :
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
namespace: data-engineering
name: umbrella:data-engineering-app
rules:
– apiGroups: [“”]
resources: [“configmaps”]
resourceNames: [“data-engineering-app-configmap”] <<<<<<<<<
verbs: [“get”]
—
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: RoleBinding
metadata:
name: umbrella:data-engineering-app
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: umbrella:data-engineering-app
subjects:
– kind: ServiceAccount
name: data-engineering-app
namespace: data-engineering
You can also refer to resources by name for certain requests through the resourceNames list. When specified, requests can be restricted to individual instances of a resource. Here is an example that restricts its subject to only get or update a ConfigMap named my-configmap
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
namespace: default
name: configmap-updater
rules:
- apiGroups: [""]
#
# at the HTTP level, the name of the resource for accessing ConfigMap
# objects is "configmaps"
resources: ["configmaps"]
resourceNames: ["my-configmap"]
verbs: ["update", "get"]
https://kubernetes.io/docs/reference/access-authn-authz/rbac/#referring-to-resources
Good Example : https://thenewstack.io/three-realistic-approaches-to-kubernetes-rbac/
It is not possible to restrict access on a resource per resource basis.
The RBAC framework works by allowing specified Roles to perform certain actions (get, update, delete etc.) over certain resources (pods, secrets etc) in a certain namespace.
Clusterroles are used to grant access across all namespaces or to non namespaced resources like nodes.
To achieve what you are looking for you need to isolate your Kubernetes secret in a namespace where you only allow your specific role to read secrets.
Related
I was trying to create a ClusterRole based on the default view ClusterRole, and apply extra rules to the new ClusterRole. The problem is that the default system clusterRole view does not have a unique label selector.
I would like to achive something like:
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: deployment-clusterrole
aggregationRule:
clusterRoleSelectors:
- selectors:
metadata.name: "view"
rules:
- apiGroups: ["*"] # "" indicates the core API group
resources: ["deployment"]
verbs: ["create", "delete"]
From the docs ClusterRole aggregation, we know that we can aggregate several ClusterRoles into one combined ClusterRole.
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: monitoring
aggregationRule:
clusterRoleSelectors:
- matchLabels:
rbac.example.com/aggregate-to-monitoring: "true"
rules: [] # The control plane automatically fills in the rules
refer to select-a-pod-by-name
I know that you can assign multiple roles to one service account when you want your service account to access multiple namespaces, but what I wonder is how it will behave when you assign to it more than one clusterrole which is cluster scoped. From my perspective, I think that it will choose one of them but I'm not sure.
Permissions are purely additive (there are no "deny" rules).
reference
This is the golden 🥇 rule here that we must memorize for kubernetes RBAC roles.
"purely additive" means always ALLOW no revoke.
Hence, "purely additive" means there are neither conflicts nor order of precedence.
It's not like AWS IAM policies where we have DENY and ALLOW .. That's time, we have to know which one has the highest order of precedence.
It's not like also subnets ACL , where we have DENY and ALLOW .. That's time, we need to assign number for each rule. This number will decide the order of precedence.
Example:
---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
# "namespace" omitted since ClusterRoles are not namespaced
name: pod-reader
rules:
- apiGroups: [""]
resources: ["pods"]
verbs: ["get", "watch", "list"]
---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
# "namespace" omitted since ClusterRoles are not namespaced
name: node-reader
rules:
- apiGroups: [""]
resources: ["nodes"]
verbs: ["get", "watch", "list"]
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
name: pod-reader
subjects:
- kind: User
name: abdennour
apiGroup: rbac.authorization.k8s.io
roleRef:
kind: ClusterRole
name: pod-reader
apiGroup: rbac.authorization.k8s.io
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
name: node-reader
subjects:
- kind: User
name: abdennour
apiGroup: rbac.authorization.k8s.io
roleRef:
kind: ClusterRole
name: node-reader
apiGroup: rbac.authorization.k8s.io
as you can see in this example, the user Abdennour should have at the end the wide read access for both: nodes & pods.
If you assign a service account multiple clusterroles using multiple role or clusterrole bindings the service account will have permission which is aggregate of all of those cluster roles meaning all the verbs on all the resources defined in those clusterroles.
i'm doing some resource level RBAC for k8s custom objects and finding it difficult to get filter resources using native k8s calls
cluster is my custom CRD and user john has access to only one crd instance not all instances of CRD using k8s native RBAC
➜ k get clusters
NAME AGE
aws-gluohfhcwo 3d2h
azure-cikivygyxd 3d1h
➜ k get clusters --as=john
Error from server (Forbidden): clusters.operator.biqmind.com is forbidden: User "ranbir" cannot list resource "clusters" in API group "operator.biqmind.com" in the namespace "biqmind"
➜ k get clusters --as=john aws-gluohfhcwo
NAME AGE
aws-gluohfhcwo 3d2h
i have explicitly specify object name to get the list of objects to which user is authenticated. any suggestions on how this can be solved?
full rbac is posted here
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
namespace: biqmind
name: cluster-admin-aws-gluohfhcwo
rules:
- apiGroups: ["operator.biqmind.com"]
resources: ["clusters"]
resourceNames: ["aws-gluohfhcwo"]
verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: cluster-admin-aws-gluohfhcwo-binding
namespace: biqmind
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: cluster-admin-aws-gluohfhcwo
subjects:
- apiGroup: rbac.authorization.k8s.io
kind: User
name: ranbir
User "ranbir" cannot list resource "clusters" in API group "operator.biqmind.com" in the namespace "biqmind"
You must add RBAC permissions with the verb list for the specified user in the specified namespace, to let that user list "clusters".
When doing
kubectl get clusters --as=john aws-gluohfhcwo
you use the RBAC verb get, but to list without specifying a specific name, the user also need permission to list.
Example of giving list permission, without resourceName::
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
namespace: biqmind
name: cluster-admin-aws-gluohfhcwo
rules:
- apiGroups: ["operator.biqmind.com"]
resources: ["clusters"]
resourceNames: ["aws-gluohfhcwo"]
verbs: ["get", "watch", "create", "update", "patch", "delete"]
- apiGroups: ["operator.biqmind.com"]
resources: ["clusters"]
verbs: ["get", "list"]
Enforcing RBAC on the user side is easy in concept. You can create RoleBindings for individual users, but this is not the recommended path as there’s a high risk of operator insanity.
The better approach for sane RBAC is to create that your users map to; how this mapping is done is dependent on your cluster’s authenticator (e.g. the aws-iam-authenticator for EKS uses mapRoles to map a role ARN to a set of groups).
Groups and the APIs they have access to are ultimately determined based on an organization’s needs, but a generic reader (for new engineers just getting the hang of things), writer (for your engineers), and admin (for you) role is a good start. (Hey, it’s better than admin for everyone.)
Here is example of configuration file:
# An example reader ClusterRole – ClusterRole so you’re not worried about namespaces at this time. Remember, we’re talking generic reader/writer/admin roles.
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: reader
rules:
– apiGroups: [“*”]
resources:
– deployments
– configmaps
– pods
– secrets
– services
verbs:
– get
– list
– watch
---
# An example reader ClusterRoleBinding that gives read permissions to
# the engineering and operations groups
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
name: reader-binding
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: reader
subjects:
- kind: Group
name: umbrella:engineering
- kind: Group
name: umbrella:operations
---
# An example writer ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: writer
rules:
- apiGroups: [“*”]
resources:
- deployments
- configmaps
- pods
- secrets
- services
verbs:
- create
- delete
- patch
- update
---
# An example writer ClusterRoleBinding that gives write permissions to
# the operations group
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
name: reader-binding
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: reader
subjects:
- kind: Group
name: umbrella:operations
Here is exact explanation: rbac-article.
Take notice that default Roles and Role Bindings
API servers create a set of default ClusterRole and ClusterRoleBinding objects. Many of these are system: prefixed, which indicates that the resource is “owned” by the infrastructure. Modifications to these resources can result in non-functional clusters. One example is the system:node ClusterRole. This role defines permissions for kubelets. If the role is modified, it can prevent kubelets from working.
All of the default cluster roles and rolebindings are labeled with kubernetes.io/bootstrapping=rbac-defaults.
Remember about auto-reconciliation
At each start-up, the API server updates default cluster roles with any missing permissions, and updates default cluster role bindings with any missing subjects. This allows the cluster to repair accidental modifications, and to keep roles and rolebindings up-to-date as permissions and subjects change in new releases.
To opt out of this reconciliation, set the rbac.authorization.kubernetes.io/autoupdate annotation on a default cluster role or rolebinding to false. Be aware that missing default permissions and subjects can result in non-functional clusters.
Auto-reconciliation is enabled in Kubernetes version 1.6+ when the RBAC authorizer is active.
Useful article: understanding-rbac.
I know there are a lot of similar questions but none of them has a solution as far as I have browsed.
Coming to the issue, I have created a service account (using command), role (using .yaml file), role binding (using .yaml files). The role grants access only to the pods. But when I login into the dashboard (Token method) using the SA that the role is attached to, I'm able to view all the resources without any restrictions. Here are the file and commands used by me.
Role.yaml:
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
namespace: assembly-prod
name: testreadrole
rules:
- apiGroups: [""]
resources: ["pods"]
verbs: ["get", "watch", "list"]
RoleBinding.yaml
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: testrolebinding
namespace: assembly-prod
subjects:
- kind: ServiceAccount
name: testsa
apiGroup: ""
roleRef:
kind: Role
name: testreadrole
apiGroup: rbac.authorization.k8s.io
Command used to create service account:
kubectl create serviceaccount <saname> --namespace <namespacename>
UPDATE: I create a service account and did not attach any kind of role to it. When I tried to login with this SA, It let me through and I was able to perform all kinds activities including deleting "secrets". So by default all SA are assuming admin access and that is the reason why my above roles are not working. Is this behavior expected, If yes then how can I change it?
Try the below steps
# create service account
kubectl create serviceaccount pod-viewer
# Create cluster role/role
---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: pod-viewer
rules:
- apiGroups: [""] # core API group
resources: ["pods", "namespaces"]
verbs: ["get", "watch", "list"]
---
# create cluster role binding
kubectl create clusterrolebinding pod-viewer \
--clusterrole=pod-viewer \
--serviceaccount=default:pod-viewer
# get service account secret
kubectl get secret | grep pod-viewer
pod-viewer-token-6fdcn kubernetes.io/service-account-token 3 2m58s
# get token
kubectl describe secret pod-viewer-token-6fdcn
Name: pod-viewer-token-6fdcn
Namespace: default
Labels: <none>
Annotations: kubernetes.io/service-account.name: pod-viewer
kubernetes.io/service-account.uid: bbfb3c4e-2254-11ea-a26c-0242ac110009
Type: kubernetes.io/service-account-token
Data
====
token: eyJhbGciOiJSUzI1NiIsImtpZCI6IiJ9.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJkZWZhdWx0Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZWNyZXQubmFtZSI6InBvZC12aWV3ZXItdG9rZW4tNmZkY24iLCJrdWJlcm5ldGVzLmlvL3NlcnZpY2VhY2NvdW50L3NlcnZpY2UtYWNjb3VudC5uYW1lIjoicG9kLXZpZXdlciIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VydmljZS1hY2NvdW50LnVpZCI6ImJiZmIzYzRlLTIyNTQtMTFlYS1hMjZjLTAyNDJhYzExMDAwOSIsInN1YiI6InN5c3RlbTpzZXJ2aWNlYWNjb3VudDpkZWZhdWx0OnBvZC12aWV3ZXIifQ.Pgco_4UwTCiOfYYS4QLwqgWnG8nry6JxoGiJCDuO4ZVDWUOkGJ3w6-8K1gGRSzWFOSB8E0l2YSQR4PB9jlc_9GYCFQ0-XNgkuiZBPvsTmKXdDvCNFz7bmg_Cua7HnACkKDbISKKyK4HMH-ShgVXDoMG5KmQQ_TCWs2E_a88COGMA543QL_BxckFowQZk19Iq8yEgSEfI9m8qfz4n6G7dQu9IpUSmVNUVB5GaEsaCIg6h_AXxDds5Ot6ngWUawvhYrPRv79zVKfAxYKwetjC291-qiIM92XZ63-YJJ3xbxPAsnCEwL_hG3P95-CNzoxJHKEfs_qa7a4hfe0k6HtHTWA
ca.crt: 1025 bytes
namespace: 7 bytes
```
Login to dashboard using the above token. you should see only pods and namespaces
[![Refer the below link][1]][1]
[1]: https://i.stack.imgur.com/D9bDi.png
Okay I've found the solution for this. The major issue was I'm running my cluster on Azure AKS, which I should have mentioned in the question but did not. It was my mistake. In Azure AKS, if rbac is not enabled during cluster creation, then there is no use of roles and role-bindings at all. All request to the api-server will be treated as requests from Admin. This was confirmed by Azure support too. So that was the reason my cluster-role-binding and roles didn't apply.
I see that the .yamls you provided need some adjustments.
Role has wrong formatting after the rules part.
RoleBinding is missing namespace: after subjects:, and also is formatted wrongly.
Try something like this:
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
namespace: assembly-prod
name: testreadrole
rules:
- apiGroups: [""]
resources: ["pods"]
verbs: ["get", "watch", "list"]
---
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: testrolebinding
namespace: assembly-prod
subjects:
- kind: ServiceAccount
name: testsa
namespace: assembly-prod
roleRef:
kind: Role
name: testreadrole
apiGroup: rbac.authorization.k8s.io
There is a very useful guide about Non-Privileged RBAC User Administration in Kubernetes where you can find more detailed info regarding this particular topic.
I have two PodSecurityPolicy:
000-privileged (only kube-system service accounts and admin users)
100-restricted (everything else)
I have a problem with their assignment to pods.
First policy binding:
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: psp:privileged
rules:
- apiGroups:
- extensions
resources:
- podsecuritypolicies
resourceNames:
- 000-privileged
verbs:
- use
---
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: psp:privileged-kube-system
namespace: kube-system
subjects:
- kind: Group
name: system:serviceaccounts
apiGroup: rbac.authorization.k8s.io
roleRef:
kind: ClusterRole
name: psp:privileged
apiGroup: rbac.authorization.k8s.io
Second policy binding:
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: psp:restricted
rules:
- apiGroups:
- extensions
resources:
- podsecuritypolicies
resourceNames:
- 100-restricted
verbs:
- use
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: psp:restricted
subjects:
- kind: Group
name: system:authenticated
apiGroup: rbac.authorization.k8s.io
- kind: Group
name: system:serviceaccounts
apiGroup: rbac.authorization.k8s.io
roleRef:
kind: ClusterRole
name: psp:restricted
apiGroup: rbac.authorization.k8s.io
Everything works fine in kube-system.
However, in other namespaces, it does not work as expected:
If I create a Deployment (kubectl apply -f deployment.yml), its pod gets tagged with psp 100-restricted.
If I create a Pod (kubectl apply -f pod.yml), it gets tagged with psp 000-privileged. I really don't get why its not 100-restricted.
My kubectl is configured with external authentication token from OpenID Connect (OIDC).
I verified the access and everything seems ok:
kubectl auth can-i use psp/100-restricted
yes
kubectl auth can-i use psp/000-privileged
no
Any clue?
The problem was that my user had access to all verbs (*) of all resources (*) of the 'extensions' apiGroup in its Role.
The documentation is a bit unclear (https://github.com/kubernetes/examples/tree/master/staging/podsecuritypolicy/rbac):
The use verb is a special verb that grants access to use a policy while not permitting any other access. Note that a user with superuser permissions within a namespace (access to * verbs on * resources) would be allowed to use any PodSecurityPolicy within that namespace.
I got confused by the mention of 'within that namespace'. Since PodSecurityGroup are not "namespaced", I assumed they could not be used without a ClusterRole/RoleBinding in the namespace giving explicit access. Seems I was wrong...
I modified my role to specify the following:
rules:
- apiGroups: ["", "apps", "autoscaling", "batch"]
resources: ["*"]
verbs: ["*"]
- apiGroups: ["extensions"]
resources: ["*"]
# Avoid using * here to prevent use of 'use' verb for podsecuritypolicies resource
verbs: ["create", "get", "watch", "list", "patch", "delete", "deletecollection", "update"]
And now it picks up the proper PSP. One interesting thing, it also prevent users from modifying (create/delete/update/etc) podsecuritypolicies.
It seems 'use' verb is quite special after all.....