I have couple of namespaces - assume NS1 and NS2. I have serviceaccounts created in those - sa1 in NS1 and sa2 in NS2. I have created roles and rolebindings for sa1 to do stuff within NS1 and sa2 within NS2.
What I want is give sa1 certain access within NS2 (say only Pod Reader role).
I am wondering if that's possible or not?
You can simply reference a ServiceAccount from another namespace in the RoleBinding:
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: Role
metadata:
name: pod-reader
namespace: ns2
rules:
- apiGroups: [""]
resources: ["pods"]
verbs: ["get", "list", "watch"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: pod-reader-from-ns1
namespace: ns2
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: pod-reader
subjects:
- kind: ServiceAccount
name: ns1-service-account
namespace: ns1
Related
Im running a pipeline that creates a kubernetes namespace but when I run it I get:
Error from server (Forbidden): namespaces is forbidden: User "system:serviceaccount:gitlab-runner:default" cannot create resource "namespaces" in API group "" at the cluster scope
I created a ClusterRole and a ClusterRoleBinding to allow the service user default in the gitlab-runner namespace to create namespaces with:
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: modify-namespace
rules:
- apiGroups: [""]
resources:
- namespace
verbs:
- create
and:
ind: ClusterRoleBinding
metadata:
name: modify-namespace-binding
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: modify-namespace
subjects:
- kind: ServiceAccount
name: default
namespace: gitlab-runner
But that gives me the same error.
What am I doing wrong?
[""] in clusterrole manifest it should be just "".
because [""] will be array where apiGroups expects a string.
under resources it should be namespaces not namespace because :
kubectl api-resources | grep 'namespace\|NAME'
NAME SHORTNAMES APIVERSION NAMESPACED KIND
namespaces ns v1 false Namespace
so clusterrole manifest should be as following :
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: modify-namespace
rules:
- apiGroups: ""
resources:
- namespaces
verbs:
- create
I had this issue below:
Namespaces is forbidden: User "system:serviceaccount:openshift-operators:minio-operator" cannot create resource "namespaces" in API group "" at the cluster scope
Got solved with below yaml:
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: cluster-role-cesar-3
rules:
- apiGroups: [""]
resources:
- namespaces
verbs:
- create
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: role-binding-cesar-3
namespace: openshift-operators
subjects:
- kind: ServiceAccount
name: minio-operator
namespace: openshift-operators
roleRef:
kind: ClusterRole
name: cluster-role-cesar-3
apiGroup: rbac.authorization.k8s.io
in order to make a high availability test in kubernetes cluster, i use a tool such as chaoskube or kube-monkey , which kills random pods in namespaces to create a "chaos" and to see how the system and applications will react.
by default these tools need a cluster role, in order to let its service account to list/kill pods for all namespaces in cluster.
in my situation i want to install this tool and make the test just in one namespace (namespace x)
is there any way to restrict the permissions of the service account just to give it the permissions to list/kill pods from (namespace x) and the whole cluster ?
i already tried to create a role & rolebinding in (namespace x) but still have the same RBAC error, as the service account expects to have the cluster permissions :
"pods is forbidden: User \"system:serviceaccount:x:chaoskube-sa\" cannot list resource \"pods\" in API group \"\ at the cluster scope"
update: role & rolebinding
this is the default permissions for its service account:
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: chaoskube-role
rules:
- apiGroups: [""]
resources: ["pods"]
verbs: ["list", "delete"]
- apiGroups: [""]
resources: ["events"]
verbs: ["create"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: chaoskube-rolebinding
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: chaoskube-role
subjects:
- kind: ServiceAccount
name: chaoskube-sa
namespace: x
with these configration it works fine.
now with restricted permissions for a specific namespace :
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: chaoskube-role
namespace: x
rules:
- apiGroups: [""]
resources: ["pods"]
verbs: ["list", "delete"]
- apiGroups: [""]
resources: ["events"]
verbs: ["create"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: chaoskube-rolebinding
namespace: x
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: chaoskube-role
subjects:
- kind: ServiceAccount
name: chaoskube-sa
namespace: x
it can not list the pods , and i receive the RBAC error.
In my scenario user has access to four namespaces only, he will switch between namespaces using contexts below. How can I give him access to CRD's along with his exiting access to four namespaces.
CURRENT NAME CLUSTER AUTHINFO NAMESPACE
* dev-crd-ns-user dev dev-crd-ns-user dev-crd-ns
dev-mon-fe-ns-user dev dev-mon-fe-ns-user dev-mon-fe-ns
dev-strimzi-operator-ns dev dev-strimzi-operator-ns-user dev-strimzi-operator-ns
dev-titan-ns-1 dev dev-titan-ns-1-user dev-titan-ns-1
hifi#101common:/root$ kubectl get secret
NAME TYPE DATA AGE
default-token-mh7xq kubernetes.io/service-account-token 3 8d
dev-crd-ns-user-token-zd6xt kubernetes.io/service-account-token 3 8d
exfo#cmme101common:/root$ kubectl get crd
Error from server (Forbidden): customresourcedefinitions.apiextensions.k8s.io is forbidden: User "system:serviceaccount:dev-crd-ns:dev-crd-ns-user" cannot list resource "customresourcedefinitions" in API group "apiextensions.k8s.io" at the cluster scope
Tried below two options. Option 2 is the recommendation but didn't work with either one.
Error from server (Forbidden): customresourcedefinitions.apiextensions.k8s.io is forbidden: User "system:serviceaccount:dev-crd-ns:dev-crd-ns-user" cannot list resource "customresourcedefinitions" in API group "apiextensions.k8s.io" at the **cluster scope**
Option 1: Adding CRD to existing role
role
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
annotations:
name: dev-ns-user-full-access
namespace: dev-crd-ns
rules:
- apiGroups:
- ""
- extensions
- apps
- networking.k8s.io
- apiextensions.k8s.io
resources:
- '*'
- customresourcedefinitions
verbs:
- '*'
- apiGroups:
- batch
resources:
- jobs
- cronjobs
verbs:
- '*'
role binding
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
annotations:
name: dev-crd-ns-user-view
namespace: dev-crd-ns
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: dev-crd-ns-user-full-access
subjects:
- kind: ServiceAccount
name: dev-crd-ns-user
namespace: dev-crd-ns
Option 2 : Adding CRD as a new role to "dev-crd-ns" namespace
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
namespace: dev-crd-ns
name: crd-admin
rules:
- apiGroups: ["apiextensions.k8s.io"]
resources: ["customresourcedefinitions"]
verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: crd-admin
namespace: dev-crd-ns
subjects:
- kind: ServiceAccount
name: dev-crd-ns-user
namespace: dev-crd-ns
roleRef:
kind: Role
name: crd-admin
apiGroup: rbac.authorization.k8s.io
You need to create Role and RoleBinding for each service account like dev-crd-ns-user.
For dev-crd-ns-user:
Update the existing Role or create a new one:
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
namespace: dev-crd-ns
name: crd-admin
rules:
- apiGroups: ["apiextensions.k8s.io"]
resources: ["customresourcedefinitions"]
verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
$ kubectl apply -f crd-admin-role.yaml
Update the existing RoleBinding with this new Role or create a new one:
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: crd-admin
namespace: dev-crd-ns
subjects:
- kind: ServiceAccount
name: dev-crd-ns-user
namespace: dev-crd-ns
roleRef:
kind: Role
name: crd-admin
apiGroup: rbac.authorization.k8s.io
$ kubectl apply -f crd-admin-role-binding.yaml
Now, the SA dev-crd-ns-user will have all the access to customresourcedefinitions.
Follow similar steps for the rest of the service accounts.
I have a service account, in the default namespace, with a clusterRoleBinding to a clusterRole that can observe jobs.
I wish to use this service account in any namespace rather than have to define a new service account in each new namespace. The service account is used by an init container to check a job has completed before allowing deployment to continue.
Not sure what extra info I need to provide but will do so on request.
You can simply reference a ServiceAccount from another namespace in the RoleBinding:
For example, below is sample use to refer the service account in one namespace to another for just reading the pods.
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: Role
metadata:
name: pod-reader
namespace: ns2
rules:
- apiGroups: [""]
resources: ["pods"]
verbs: ["get", "list", "watch"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: pod-reader-from-ns1
namespace: ns2
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: pod-reader
subjects:
- kind: ServiceAccount
name: ns1-service-account
namespace: ns1
So I have namespaces
ns1, ns2, ns3, and ns4.
I have a service account sa1 in ns1. I am deploying pods to ns2, ns4 that use sa1. when I look at the logs it tells me that the sa1 in ns2 can't be found.
error:
Error creating: pods "web-test-2-795f5fd489-" is forbidden: error looking up service account ns2/sa: serviceaccount "sa" not found
Is there a way to make service accounts cluster wide? Or, can I create multiple service accounts with the same secret? in different namespaces?
you can use that
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: kubernetes-enforce
rules:
- apiGroups: ["apps"]
resources: ["deployments","pods","daemonsets"]
verbs: ["get", "list", "watch", "patch"]
- apiGroups: ["*"]
resources: ["namespaces"]
verbs: ["get", "list", "watch"]
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: kubernetes-enforce
namespace: kube-system
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: kubernetes-enforce-logging
namespace: cattle-logging
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: kubernetes-enforce
subjects:
- kind: ServiceAccount
name: kubernetes-enforce
namespace: kube-system
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: kubernetes-enforce-prome
namespace: cattle-prometheus
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: kubernetes-enforce
subjects:
- kind: ServiceAccount
name: kubernetes-enforce
namespace: kube-system
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: kubernetes-enforce-system
namespace: cattle-system
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: kubernetes-enforce
subjects:
- kind: ServiceAccount
name: kubernetes-enforce
namespace: kube-system
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: kubernetes-enforce-default
namespace: default
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: kubernetes-enforce
subjects:
- kind: ServiceAccount
name: kubernetes-enforce
namespace: kube-system
No there is no way to create a cluster wide service account as service account is a namespace scoped resources. This follows the principle of least privilege.
You can create a service account with same name(for example default) into all the necessary namespaces where you are deploying pod pretty easily by applying the service account yaml targeting those namespaces.
Then you can deploy the pod using yaml. This way you don't need to change anything in the pod because the service account name is same although it will have different secret and that should not matter as long as you have defined RBAC via role and rolebinding to all the service accounts across those namespaces.
While service accounts can not be cluster scoped you can have clusterrole and clusterrolebinding which are cluster scoped.
If your namespaces for example are in values.yaml (that is they are somehow dynamic), you could do:
apiVersion: v1
kind: List
items:
{{- range $namespace := .Values.namespaces }}
- kind: ServiceAccount
apiVersion: v1
metadata:
name: <YourAccountName>
namespace: {{ $namespace }}
{{- end }}
where in values.yaml you would have:
namespaces:
- namespace-a
- namespace-b
- default
# define a clusterrole.
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: supercr
rules:
- apiGroups: ["*"]
resources: ["*"]
verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
---
# define a serviceaccount
apiVersion: v1
kind: ServiceAccount
metadata:
name: supersa
namespace: namespace-1
---
# bind serviceaccount to clusterrole
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: supercrb
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: supercr
subjects:
- kind: ServiceAccount
name: supersa
namespace: namespace-1
Please note serviceaccount is namespaced. You can't create a cluster-wide serviceaccount. However you can bind a serviceaccount to a clusterrole with permissions to all api resources.