Is it possible to create a Kubernetes cluster admin without the ability to read namespace secrets?
I know you can create a ClusterRole and list every single resource and omit secret but seems unintuitive.
Can you use Aggregated ClusterRoles to remove a permission? so using ClusterRole cluster-admin and have a role that uses:
rules:
- apiGroups: [""]
resources: ["secrets"]
verbs: [""]
Not really Aggregated Cluster Roles is a set union of several ClusterRoles. To get the behavior you want you would need a set subtraction of cluster-admin role minus the rules that you have defined. It's not supported in K8s as of this writing.
Related
We're creating dynamic test environments for our developers. Each environment goes into one namespace called test-<something>, where <something> is entered by the developer when creating the environment (we use Gitlab-CI for the automation).
We want to grant them limited access to the K8s API to see deployments, exec into pods for instance. So the plan is to apply a (cluster)role (yet to decide) like this.
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
namespace: "{{ ns }}"
name: "test-{{ ns }}"
rules:
- apiGroups: ["apps"]
resources: ["deploys"]
verbs: ["get", "list"]
- apiGroups: [""]
resources: ["pods", "pods/log"]
verbs: ["get", "list"]
- apiGroups: [""]
resources: ["pods/exec"]
verbs: ["create"]
However we preferably don't want to apply it for all namespaces but only the test-* ones.
We could add the creation of the namespaced role and rolebinding during the app deploy, but that would mean granting our Gitlab-CI runner the permission to create and delete roles and rolebindings. We're concerned by the security implications of this and the possible privilege escalations.
Is it possible to create a clusterrolebinding limited to a regexp-ed set of namespaces?
Alternatively, if we want to grant the permissions via the automation, is it possible to limit the namespaces where the rolebindings can be created by the runner?
We looked at the docs but couldn't find such things
Worst case scenario is probably to go for the clusterrolebinding route and not give too many privileges to the automation. So asking if we can find a better way
Thanks in advance
I also stumbled into this problem and Hierarchical Namespaces seem like a decent solution, as you can give the permissions to a single "static" namespace. Every namespace afterwards will inherit the permissions. Hope it helps.
I want to create a user that has permission to create temporary namespaces (i.e. create and then delete the namespaces) and have full access within these namespaces only, and without being able to delete any other namespaces within the cluster. My problem is that the namespaces in this case are created dynamically by the user with a random name each time...is there any way to meet this requirement for a user to:
Create namespace (with a random name that is only known after execution)
Have full access to this name space only
Able to delete this namespace only
You can first restrict user to create/delete resources only in the namespace using a Role bind that role to the user.
Then you can use ClusterRole with only access to the namespaces resource and allow it to create, delete, etc.
ClusterRole and Role define which resources you can operate on. If your role needs to manage resources in multiple namespaces, you need to use ClusterRole
and RoleBinding defines which namespace your account will be granted
Example:
Create a Role in the namespace that can be used to grant access to resources.
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
namespace: namespace1
name: user-namespace-role
rules:
- apiGroups: [""]
resources: ["services", "endpoints", "pods"] # etc...
verbs: ["get", "list", "create",”delete”] # etc
Then execute the below command which creates rolebinding that defines which namespace your account will be granted.
$ kubectl create rolebinding user-namespace-binding --role=user-namespace-role --user=<user-name> --namespace=namespace1
Now Create a ClusterRole that can be used to grant access in any particular namespace, or across all namespaces
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: cluster-role-all-namespaces
rules:
- apiGroups: [""]
resources: ["namespaces"]
verbs: ["get", "list", "watch", "create", "update", "patch", "delete"] # etc
Then Execute the below command to create clusterrolebinding:
$ kubectl create clusterrolebinding all-namespaces-binding --clusterrole=cluster-role-all-namespaces --user=<username>
Refer to Using RBAC authorization for more information.
On my understanding there is no way to accomplish this kind of permissions - I know it's surprising because it would seem like a common use case. Using Roles, the rules only apply to the namespace the role belongs to, using an appropriate RoleBinding. Therefore we can limit actions on specific namespaces, but we cannot define a permission on a namespace with unknown name. Also using regex is not supported to match namespaces names. To solve it with Roles, we would have to apply the role every time a new namespace is created, I think makes no sense for the use case.
As you may know Kubernetes also defines cluster roles and cluster role bindings. Those are similar to normal roles but without a namespace limitation. Of course, using them in this case can break through the isolation we want, that's what they are meant to do.
What I suggest is to predefine a list of namespaces for which those accounts will have permissions. That means you need to create/use the same namespaces and to not generate them dinamically with random names. It also means that it will not be able to create new namespaces.
Lastly, I wanted to mention Hierarchical Namespaces. Under HNS, administrators can organize tenants according to a group hierarchy and allocate capabilities accordingly.
I have 2 teams:
devs: they create a new Kubernetes namespace each time they deploy a branch/tag of their app
ops: they manage access control to the cluster with (cluster)roles and (cluster)rolebindings
The problem is that 'devs' cannot kubectl their namespaces until 'ops' have created RBAC resources. And 'devs' cannot create RBAC resources themselves as they don't have the list of subjects to put in the rolebinding resource (sharing the list is not an option).
I have read the official documentation about Admission webhooks but what I understood is that they only act on the resource that triggered the webhook.
Is there a native and/or simple way in Kubernetes to apply resources whenever a new namespace is created?
I've come up with a solution by writing a custom controller.
With the following custom resource deployed, the controller injects the role and rolebinding in namespaces matching dev-.* and fix-.*:
kind: NamespaceResourcesInjector
apiVersion: blakelead.com/v1alpha1
metadata:
name: nri-test
spec:
namespaces:
- dev-.*
- fix-.*
resources:
- |
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: dev-role
rules:
- apiGroups: [""]
resources: ["pods","pods/portforward", "services", "deployments", "ingresses"]
verbs: ["list", "get"]
- apiGroups: [""]
resources: ["pods/portforward"]
verbs: ["create"]
- apiGroups: [""]
resources: ["namespaces"]
verbs: ["list", "get"]
- |
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: dev-rolebinding
subjects:
- kind: User
name: dev
roleRef:
kind: Role
name: dev-role
apiGroup: rbac.authorization.k8s.io
The controller is still in early stages of development but I'm using it successfully in more and more clusters.
Here it is for those interested: https://github.com/blakelead/nsinjector
Yes, there is a native way but not an out of the box feature.
You can do what you have described by using/creating an operator. Essentially extending Kubernetes APIs for your need.
As operator is just an open pattern which can implement things in many ways, in the scenario you gave one way the control flow could look like could be:
Operator with privileges to create RBAC is deployed and subscribed to changes to a k8s namespace object kind
Devs create namespace containing an agreed label
Operator is notified about changes to the cluster
Operator checks namespace validation (this can also be done by a separate admission webhook)
Operator creates RBAC in the newly created namespace
If RBACs are cluster wide, same operator can do the RBAC cleanup once namespace is deleted
It's kind of related to how the user is authenticated to the cluster and how they get a kubeconfig file.You can put a group in the client certificate or the bearer token that kubectl uses from the kubeconfig. Ahead of time you can define a clusterrole having a clusterrolebinding to that group which gives them permission to certain verbs on certain resources(for example ability to create namespace)
Additionally you can use an admission webhook to validate if the user is supposed to be part of that group or not.
Upon deploying a service with Spring Cloud Kubernetes Discovery Client, I get the KubernetesClientException stating that the user "default" is forbidden to access pods.
I have already added a Role and a Rolebinding as specified here
The guide states that a ClusterRole is necessary. But that is not an option for me, as we share the cluster with other departments. I only want the role to affect our project / namespace.
Is ClusterRole required or should Role be sufficient?
To allow a service account access to these one needs to create a role with the necessary permissions and assign it to the account.This is done with a cluster role, or a role, if one only wants it in
one namespace, and a role binding, which is specific to a namespace.
It says that you can use either Role or ClusterRole.
Just bear in mind when creating a Role a namespace should be defined. i.e.
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
namespace: mynamespace
name: service-discovery-client
rules:
- apiGroups: [""] # "" indicates the core API group
resources: ["services", "pods", "configmaps", "endpoints"]
verbs: ["get", "watch", "list"]
I have created a role, below is definition the role:
---
kind: Role
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
name: ${SERVICE_ACCOUNT_NAME}-full-access-role
namespace: ${NAMESPACE}
rules:
- apiGroups: ["", "extensions", "apps"]
resources: ["*"]
verbs: ["*"]
- apiGroups: ["batch"]
resources:
- jobs
- cronjobs
verbs: ["*"]
Using this role a user can create, list and delete any resource in a namespace but problem is it can also list all namespaces in cluster.
I want to enable users to create namespaces and can perform any action in it as well but also they can't list any resource even other namespaces that are't created by the user.
I think you cannot deny user access right to list all namespaces and give them the access you need at the same. At least not in the bare metal kubernetes. Read the following GH issue, it's about the similar issue that you encounter.
It would help if you can tell us which cloud platform you are on. To my knowledge, different cloud platforms handle Role based access differently.
With that being said, I believe there should be predefined roles for the specific level of access you want. Usually role based authorization trickles down. If you grant someone access to a resource at the folder level, the user will be able to access all resources in that folder. Since you want to grant access to any resource in an namespace, I would grant the user access at that level.