I'm new to k8s.
I have a deployment with a single pod inside k8test (custom) namespace. for learning purposes, I want to give that pod an admin access.
I failed to achieve this by creating a namespace-role:
kind: 'Role',
apiVersion: 'rbac.authorization.k8s.io/v1',
metadata: {
name: 'super-duper-admin',
namespace: 'k8test',
},
rules: [
{
apiGroups: [''],
resources: ['ResourceAll'],
verbs: ['VerbAll'],
},
],
from the pod`s log:
services is forbidden: User "system:serviceaccount:k8test:default" cannot list resource "services" in API group "" in the namespace "k8test"
I couldn't find a simple explanantion to what is apiGroups. What is it?
You can find all api groups here.
As documented here
API groups make it easier to extend the Kubernetes API. The API group
is specified in a REST path and in the apiVersion field of a
serialized object
Pod comes under core API Group and version v1.
In your RBAC '' indicates the core API group.
Create the role as below which gives permission to all apigroups, all resources and all verbs.
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
namespace: k8test
name: super-duper-admin
rules:
- apiGroups:
- '*'
resources:
- '*'
verbs:
- '*'
Bind the role to the service account as below
apiVersion: rbac.authorization.k8s.io/v1
# This role binding allows "jane" to read pods in the "default" namespace.
# You need to already have a Role named "pod-reader" in that namespace.
kind: RoleBinding
metadata:
name: admin-rolebinding
namespace: k8test
subjects:
# You can specify more than one "subject"
- kind: ServiceAccount
name: default # "name" is case sensitive
namespace: k8test
roleRef:
# "roleRef" specifies the binding to a Role / ClusterRole
kind: Role #this must be Role or ClusterRole
name: super-duper-admin # this must match the name of the Role or ClusterRole you wish to bind to
apiGroup: rbac.authorization.k8s.io
Execute below command to verify the RABC is properly applied
kubectl auth can-i list services --as=system:serviceaccount:k8test:default -n k8test
More information and examples about RBAC here
Related
I'm trying to get access to my kubernetes cluster in my self hosted gitlab instance as it is described in the docs.
deploy:
stage: deployment
script:
- kubectl create secret docker-registry gitlab-registry --docker-server="$CI_REGISTRY" --docker-username="$CI_DEPLOY_USER" --docker-password="$CI_DEPLOY_PASSWORD" --docker-email="$GITLAB_USER_EMAIL" -o yaml --dry-run=client | kubectl apply -f -
But I do get the error
Error from server (Forbidden): error when retrieving current configuration of:
Resource: "/v1, Resource=secrets", GroupVersionKind: "/v1, Kind=Secret"
Name: "gitlab-registry", Namespace: "gitlab"
from server for: "STDIN": secrets "gitlab-registry" is forbidden: User "system:serviceaccount:gitlab:default" cannot get resource "secrets" in API group "" in the namespace "gitlab"
I do not understand the error. Why do I get a forbidden error?
Update
The kubernetes cluster is integrated in gitlab at instance level.
But running kubectl config view in the CI pipeline gives me
apiVersion: v1
clusters: null
contexts: null
current-context: ""
kind: Config
preferences: {}
users: null
Update2
Thanks to AndD, the secret can be created with this role / service account:
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
namespace: gitlab
name: gitlab-deploy
rules:
- apiGroups: [""] # "" indicates the core API group
resources: ["secrets"]
verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: use-secrets
namespace: gitlab
subjects:
- kind: ServiceAccount
name: default
namespace: gitlab
roleRef:
kind: ClusterRole
name: gitlab-deploy
apiGroup: rbac.authorization.k8s.io
But running a simple apply for this namespace.yaml file
apiVersion: v1
kind: Namespace
metadata:
name: myns
gives me a similar error:
Error from server (Forbidden): error when retrieving current configuration of:
Resource: "/v1, Resource=namespaces", GroupVersionKind: "/v1, Kind=Namespace"
Name: "myns", Namespace: ""
from server for: "namespace.yaml": namespaces "myns" is forbidden: User "system:serviceaccount:gitlab:default" cannot get resource "namespaces" in API group "" in the namespace "myns"
I used ClusterBinding to get this working even for a different namespace. What am I doing wrong?
Kubernetes makes use of a Role-based access control (RBAC) to prevent Pods and Users from being able to interact with resources in the cluster, unless they are not authorized.
From the error, you can see that Gitlab is trying to use the secrets resource and also that it is using as ServiceAccount the default service account in its namespace.
This means that Gitlab is not configured to use a particular ServiceAccount, which means it makes use of the default one (there's a default service account in each namespace of the cluster)
You can attach role auth and permissions to service accounts by using Role / ClusterRole and RoleBinding / ClusterRoleBinding.
Roles or ClusterRoles describe permissions. For example, a Role could be:
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
namespace: gitlab
name: secret-user
rules:
- apiGroups: [""] # "" indicates the core API group
resources: ["secrets"]
verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
and this states that "whoever has this role, can do whatever (all the verbs) with secrets but only in the namespace gitlab"
If you want to give generic permissions in all namespaces, you can use a ClusterRole instead, which is very similar.
Once the Role is created, you then can attach it to a User, a Group or a ServiceAccount, for example:
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: use-secrets
namespace: gitlab
subjects:
subjects:
- kind: ServiceAccount
name: default
namespace: gitlab
roleRef:
# "roleRef" specifies the binding to a Role / ClusterRole
kind: Role # this must be Role or ClusterRole
name: secret-user # this must match the name of the Role or ClusterRole you wish to bind to
apiGroup: rbac.authorization.k8s.io
and this bind the role previously created to the ServiceAccount called default in the namespace gitlab.
Then, all Pods running in the namespace gitlab and using the default service account, will be able to use secrets (use the verbs listed in the Role) but only in the namespace specified by the Role.
As you can see, this aspect of Kubernetes is pretty complex and powerful, so have a look at the docs because they explain things really well and are also full of examples:
Service Accounts - https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-account/
RBAC - https://kubernetes.io/docs/reference/access-authn-authz/rbac/
A list of RBAC resources - How to refer to all subresources in a Role definition?
UPDATE
You are doing nothing wrong. It's just that you are trying to use the resource namespace but Gitlab has no Bind that gives access to that type of resource. With your ClusterRole you just gave it access to secrets, but nothing more.
Consider giving the ClusterRole more permissions, changing it to list all resources that you need to access:
rules:
- apiGroups: [""] # "" indicates the core API group
resources: ["secrets", "namespaces", "pods"]
verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
For example this will give access to secrets, namespaces and Pods.
As an alternative, you can bind Gitlab's service account to cluster-admin to directly give it access to everything.
kubectl create clusterrolebinding gitlab-is-now-cluster-admin \
--clusterrole=cluster-admin \
--serviceaccount=gitlab:default
Before doing so tho, consider the following:
Fine-grained role bindings provide greater security, but require more
effort to administrate. Broader grants can give unnecessary (and
potentially escalating) API access to ServiceAccounts, but are easier
to administrate.
So, it is way more secure to first decide which resources can be used by Gitlab and then create a Role / ClusterRole giving access to only those resources (and with the verbs that you need)
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: p-viewer-role
namespace: pepsi
rules:
- apiGroups:
- ""
resourceNames:
- p83
resources:
- pods
verbs:
- list
- get
- watch
When we use resourceNames in the Roles, the following command works
kubectl get pods -n pepsi p83
returns a proper value. However,
kubectl get pods -n pepsi
returns forbidden. Why doesn't it list p83
RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: p-viewer-rolebinding
namespace: pepsi
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: p-viewer-role
subjects:
- apiGroup: rbac.authorization.k8s.io
kind: Group
name: pepsi-project-viewer
namespace: project
This is expected behavior. You have defined a role which is scoped to the namespace pepsi to pod resources with specific resourceName p83.
For kubectl get pods -n peps command to work you need to remove the resourceNames p83 from the Role
This kind of advanced validation is best handled by OPA where you can define fine grained policies.
Short answer:
list(and watch)actually can be restricted by their resource name, and permits list(and watch) requests using a fieldSelector of metadata.name=... to match a single object (for example, /api/v1/namespaces/$ns/configmaps?fieldSelector=metadata.name=foo)
For more details and some tests you can check this link: https://github.com/kubernetes/website/pull/29468
#sftim and #liggitt do offer a lot of help!
i'm doing some resource level RBAC for k8s custom objects and finding it difficult to get filter resources using native k8s calls
cluster is my custom CRD and user john has access to only one crd instance not all instances of CRD using k8s native RBAC
➜ k get clusters
NAME AGE
aws-gluohfhcwo 3d2h
azure-cikivygyxd 3d1h
➜ k get clusters --as=john
Error from server (Forbidden): clusters.operator.biqmind.com is forbidden: User "ranbir" cannot list resource "clusters" in API group "operator.biqmind.com" in the namespace "biqmind"
➜ k get clusters --as=john aws-gluohfhcwo
NAME AGE
aws-gluohfhcwo 3d2h
i have explicitly specify object name to get the list of objects to which user is authenticated. any suggestions on how this can be solved?
full rbac is posted here
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
namespace: biqmind
name: cluster-admin-aws-gluohfhcwo
rules:
- apiGroups: ["operator.biqmind.com"]
resources: ["clusters"]
resourceNames: ["aws-gluohfhcwo"]
verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: cluster-admin-aws-gluohfhcwo-binding
namespace: biqmind
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: cluster-admin-aws-gluohfhcwo
subjects:
- apiGroup: rbac.authorization.k8s.io
kind: User
name: ranbir
User "ranbir" cannot list resource "clusters" in API group "operator.biqmind.com" in the namespace "biqmind"
You must add RBAC permissions with the verb list for the specified user in the specified namespace, to let that user list "clusters".
When doing
kubectl get clusters --as=john aws-gluohfhcwo
you use the RBAC verb get, but to list without specifying a specific name, the user also need permission to list.
Example of giving list permission, without resourceName::
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
namespace: biqmind
name: cluster-admin-aws-gluohfhcwo
rules:
- apiGroups: ["operator.biqmind.com"]
resources: ["clusters"]
resourceNames: ["aws-gluohfhcwo"]
verbs: ["get", "watch", "create", "update", "patch", "delete"]
- apiGroups: ["operator.biqmind.com"]
resources: ["clusters"]
verbs: ["get", "list"]
Enforcing RBAC on the user side is easy in concept. You can create RoleBindings for individual users, but this is not the recommended path as there’s a high risk of operator insanity.
The better approach for sane RBAC is to create that your users map to; how this mapping is done is dependent on your cluster’s authenticator (e.g. the aws-iam-authenticator for EKS uses mapRoles to map a role ARN to a set of groups).
Groups and the APIs they have access to are ultimately determined based on an organization’s needs, but a generic reader (for new engineers just getting the hang of things), writer (for your engineers), and admin (for you) role is a good start. (Hey, it’s better than admin for everyone.)
Here is example of configuration file:
# An example reader ClusterRole – ClusterRole so you’re not worried about namespaces at this time. Remember, we’re talking generic reader/writer/admin roles.
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: reader
rules:
– apiGroups: [“*”]
resources:
– deployments
– configmaps
– pods
– secrets
– services
verbs:
– get
– list
– watch
---
# An example reader ClusterRoleBinding that gives read permissions to
# the engineering and operations groups
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
name: reader-binding
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: reader
subjects:
- kind: Group
name: umbrella:engineering
- kind: Group
name: umbrella:operations
---
# An example writer ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: writer
rules:
- apiGroups: [“*”]
resources:
- deployments
- configmaps
- pods
- secrets
- services
verbs:
- create
- delete
- patch
- update
---
# An example writer ClusterRoleBinding that gives write permissions to
# the operations group
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
name: reader-binding
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: reader
subjects:
- kind: Group
name: umbrella:operations
Here is exact explanation: rbac-article.
Take notice that default Roles and Role Bindings
API servers create a set of default ClusterRole and ClusterRoleBinding objects. Many of these are system: prefixed, which indicates that the resource is “owned” by the infrastructure. Modifications to these resources can result in non-functional clusters. One example is the system:node ClusterRole. This role defines permissions for kubelets. If the role is modified, it can prevent kubelets from working.
All of the default cluster roles and rolebindings are labeled with kubernetes.io/bootstrapping=rbac-defaults.
Remember about auto-reconciliation
At each start-up, the API server updates default cluster roles with any missing permissions, and updates default cluster role bindings with any missing subjects. This allows the cluster to repair accidental modifications, and to keep roles and rolebindings up-to-date as permissions and subjects change in new releases.
To opt out of this reconciliation, set the rbac.authorization.kubernetes.io/autoupdate annotation on a default cluster role or rolebinding to false. Be aware that missing default permissions and subjects can result in non-functional clusters.
Auto-reconciliation is enabled in Kubernetes version 1.6+ when the RBAC authorizer is active.
Useful article: understanding-rbac.
I am trying to add new user to EKS cluster and giving then access. So far I was able to add the user just by editing configmap/aws-auth (kubectl edit -n kube-system configmap/aws-auth) and adding new user to
mapUsers: |
- userarn: arn:aws:iam::123456789:user/user01
username: user01
groups:
- system:masters
How can add user to EKS cluster and give full access to to specific namespace, but nothing outside of it ?
I tried to create Roles & RoleBinding as
---
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
namespace: namespace1
name: namespace1-user
rules:
- apiGroups: ["*"]
resources: ["*"]
verbs: ["*"]
# This role binding allows "user01" to read pods in the "namespace1" namespace.
---
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: namespace1-user-role-binding
namespace: namespace1
subjects:
- kind: User
name: user01
apiGroup: rbac.authorization.k8s.io
roleRef:
kind: Role
name: namespace1-user
user01 can see all the pods from other users with kubectl get pods --all-namespaces, is there any way to restrict this ?
Essentially what you want is to define a cluster role and use a role binding to apply it to a specific namespace. Using a cluster role (rather than a role) allows you to re-use it across namespaces. Using a role binding allows you to target a specific namespace rather than giving cluster-wide permissions.
I am trying to create a script that records project resources every 15 minutes. How do I authenticate with Openshift API? Is there a token I can use that has read access on all namespaces? How do I create a service account that has access over all namespaces?
You'll need to create a ClusterRole that has read access to the resources and use ClusterRoleBinding to associate the ServiceAccount to that ClusterRole. Rough example, not tested but it should work:
# creates the service account "ns-reader"
apiVersion: v1
kind: ServiceAccount
metadata:
name: ns-reader
namespace: default
---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
# "namespace" omitted since ClusterRoles are not namespaced
name: global-reader
rules:
- apiGroups: [""]
# add other rescources you wish to read
resources: ["pods", "secrets"]
verbs: ["get", "watch", "list"]
---
# This cluster role binding allows service account "ns-reader" to read pods in all available namespace
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: read-ns
subjects:
- kind: ServiceAccount
name: ns-reader
namespace: default
roleRef:
kind: ClusterRole
name: global-reader
apiGroup: rbac.authorization.k8s.io
When the ServiceAccount is setup, a number of secrets are created automatically associated with it. A couple of these secrets hold a token which can then be used when using the REST API directly or using oc. Use oc describe on the ServiceAccount to see the names of the Secret for the tokens. Then use oc describe on one of the Secrets to see the token.