I have configured Pod Security Policies on my cluster, which prevents Pods run as root.
However, if any deployment uses a tiller service account, they can start pods as root, basically, they are full admin, no Pod Security Policies restrictions.
Is there a way to restrict which service accounts pods can use?
Yeah, it is possible.
You should Bind a Role/ClusterRole.
Example:
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: read-pods
namespace: default
subjects:
- kind: User
name: jane
apiGroup: rbac.authorization.k8s.io
roleRef:
kind: Role
name: pod-reader
apiGroup: rbac.authorization.k8s.io
Nice to read:
https://kubernetes.io/docs/reference/access-authn-authz/rbac/
https://docs.giantswarm.io/guides/securing-with-rbac-and-psp/
Related
We have a shared tenant cluster, and we want our developers to be able to run kubectl top pods --namespace dev-namespace
But it seems to me that for top to be usable, you need to be able to run kubectl get nodes. But nodes are not namespaced.
Is there a solution?
We the namespace admin setup like this:
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: admin
subjects:
- apiGroup: rbac.authorization.k8s.io
kind: User
name: username#domain
And as a cluster admin I can run the top command, so metrics-server seems to be working fine.
Kubernetes has API group metrics.k8s.io, that you can use to give read permission for kubectl top pods -n <namespace>. If you grant get and list permissions for pods, you can run the command.
I tested the configuration below in a GKE cluster running Kubernetes 1.21 with kubectl top pod --as=system:serviceaccount:monitoring:test-account -n monitoring. With these permissions, I can only run kubectl top pod in the monitoring namespace, other commands will fail.
apiVersion: v1
kind: ServiceAccount
metadata:
name: test-account
namespace: monitoring
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: pod-reader
namespace: monitoring
rules:
- apiGroups: ["metrics.k8s.io"]
resources: ["pods"]
verbs: ["get", "list"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: read-pods
namespace: monitoring
subjects:
- kind: ServiceAccount
name: test-account
roleRef:
kind: Role
name: pod-reader
apiGroup: rbac.authorization.k8s.io
I'd like to grant a service account the ability to access the metrics exposed by the metrics-server service (https://metrics-server.kube-system/metrics). If I create a serviceaccount...
apiVersion: v1
kind: ServiceAccount
metadata:
name: metrics-reader
namespace: prometheus
...and then grant it cluster-admin privileges...
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: metrics-reader-crb
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: cluster-admin
subjects:
- kind: ServiceAccount
name: metrics-reader
namespace: prometheus
...it works! I can use the account token to access the metrics server:
curl -k --header "Authorization: Bearer $token" https://metrics-server.kube-system/metrics
But I don't want to require cluster-admin access just to read
metrics. I tried to use the view cluster role instead of
cluster-admin, but that fails.
Is there an existing role that would grant the appropriate access?
If not, what are the specific permissions necessary to grant read-only
access to the metrics-server /metrics endpoint?
Interesting question. I've found some info for you, however i'm not sure that 100% helpful. It needs more research and reproduce.
check RBAC Deny when requesting metrics. Smth like below?
apiVersion: v1
kind: ServiceAccount
metadata:
name: metrics-reader
namespace: prometheus
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: view-metrics
rules:
- apiGroups:
- metrics.k8s.io
resources:
- pods
- nodes
verbs:
- get
- list
- watch
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: view-metrics
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: view-metrics
subjects:
- kind: ServiceAccount
name: metrics-reader
namespace: prometheus
It seems, there is a aggregated-metrics-reader clusterrole (or there was)
Aggregated ClusterRoles are documented in:
https://kubernetes.io/docs/reference/access-authn-authz/rbac/#aggregated-clusterroles.
The purpose of the system:aggregated-metrics-reader ClusterRole, is to
aggregate the rules, that grant permission to get the pod and node
metrics, to the view, edit and admin roles.
however I wasnt able to find any reference to aggregated-metrics-reader clusterrole in current version of that doc.
You can find huge example of using this clusterrole in Metrics server unable to scrape
IN addition check This adds the aggregated-metrics-reader ClusterRole which was missing github PR:
What this PR does / why we need it: This adds the
aggregated-metrics-reader ClusterRole which was missing, and seems to
be required for k8s 1.8+ per the metrics-server documentation and
default deploy manfiests
Unfortunately link in that PR direct to nowhere. I start thinking this obsolete info for 1.8 clusters.. Will update answer in case find anything more relevant
I have a group of service accounts in namespace prometheus and I have a cluster role for reading all pods in my cluster. How can I build ClusterRoleBinding to do it?
If you go to K8s docs about Using RBAC Authorization
And scroll down to examples, you can see this one:
For all service accounts in the "qa" namespace:
subjects:
- kind: Group
name: system:serviceaccounts:qa
apiGroup: rbac.authorization.k8s.io
You can now take it and apply to your usecase:
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
creationTimestamp: null
name: <some-fancy-name>
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: <clusterrolename>
subjects:
- kind: Group
name: system:serviceaccounts:prometheus
apiGroup: rbac.authorization.k8s.io
There is also one different example worth to notice:
Grant a role to all service accounts in a namespace
If you want all applications in a namespace to have a role, no matter
what service account they use, you can grant a role to the service
account group for that namespace.
For example, grant read-only permission within "my-namespace" to all
service accounts in that namespace:
kubectl create rolebinding serviceaccounts-view \
--clusterrole=view \
--group=system:serviceaccounts:my-namespace \
--namespace=my-namespace
I'm going over RBAC in Kubernetes. It appears to me that
a ServiceAccount can be bound to a Role within a namespace
(or)
a ServiceAccount can be bound to a ClusterRole and have cluster-wide access (all namespaces?)
Is it possible for a single Service Account (or User) to not have cluster-wide access but only have read-only access in only a subset of namespaces? If so, can someone elaborate on how this can be achieved. Thanks!
You need to create a RoleBinding for every namespace in each namespace the ServiceAccount should have access to.
There is an example to give the default ServiceAccount permissions to read pods in the development namespace.
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: read-secrets
namespace: development # This only grants permissions within the "development" namespace.
subjects:
- kind: ServiceAccount
name: default
namespace: kube-system
roleRef:
kind: Role
name: pod-reader
apiGroup: rbac.authorization.k8s.io
i'm getting this error i also created rbac.yaml. But it require admin permission. is it possible to apply rbac.yaml without admin role ??
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: hazelcast-rbac
subjects:
name: default-cluster
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: view
subjects:
- kind: ServiceAccount
name: default
namespace: default
By default, only cluster admin can create ClusterRoleBinding. If you are project admin, please create RoleBinding instead.
To check if you can create rolebinding:
oc adm policy who-can create rolebinding