We have a shared tenant cluster, and we want our developers to be able to run kubectl top pods --namespace dev-namespace
But it seems to me that for top to be usable, you need to be able to run kubectl get nodes. But nodes are not namespaced.
Is there a solution?
We the namespace admin setup like this:
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: admin
subjects:
- apiGroup: rbac.authorization.k8s.io
kind: User
name: username#domain
And as a cluster admin I can run the top command, so metrics-server seems to be working fine.
Kubernetes has API group metrics.k8s.io, that you can use to give read permission for kubectl top pods -n <namespace>. If you grant get and list permissions for pods, you can run the command.
I tested the configuration below in a GKE cluster running Kubernetes 1.21 with kubectl top pod --as=system:serviceaccount:monitoring:test-account -n monitoring. With these permissions, I can only run kubectl top pod in the monitoring namespace, other commands will fail.
apiVersion: v1
kind: ServiceAccount
metadata:
name: test-account
namespace: monitoring
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: pod-reader
namespace: monitoring
rules:
- apiGroups: ["metrics.k8s.io"]
resources: ["pods"]
verbs: ["get", "list"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: read-pods
namespace: monitoring
subjects:
- kind: ServiceAccount
name: test-account
roleRef:
kind: Role
name: pod-reader
apiGroup: rbac.authorization.k8s.io
Related
I have a namespace namespace:development in my K8s cluster. I wanted to deploy Fluentd following:
fluentd-daemonset-elasticsearch-rbac.yaml
I ONLY changed:
Type of role from ClusterRole to Role (the rules parts is the same)
Name of the ServiceAccount
Instead of namespace: kube-system I changed it to namespace: development in ServiceAccount, Role and RoleBinding
ServiceAccount in RoleBinding to my own service account
When I deployed I got the following error:
start_pod_watch: Exception encountered setting up pod watch from Kubernetes API v1 endpoint https://<ip>:443/api: pods is forbidden: User "system:serviceaccount:development:my-svc-account" cannot list resource "pods" in API group "" at the cluster scope ({"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"pods is forbidden: User \\"system:serviceaccount:development:my-svc-account\\" cannot list resource \\"pods\\" in API group \\"\\" at the cluster scope","reason":"Forbidden","details":{"kind":"pods"},"code":403} (Fluent::ConfigError)
My question: Is this mandatory to have a clusterRole to deploy Fluentd in a cluster?
If you have change the Clusterrole to role you also have to update the bindings.
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: fluentd
namespace: development
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: fluentd
rules:
- apiGroups:
- ""
resources:
- pods
verbs:
- get
- list
- watch
---
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: fluentd
roleRef:
kind: Role
name: fluentd
apiGroup: rbac.authorization.k8s.io
subjects:
- kind: ServiceAccount
name: fluentd
namespace: development
---
I have a custom role related to a specific namespace. I want to create a service account that will have access to the Dashboard and only being able to see this namespace assigned to that role.
I have tried the following:
apiVersion: v1
kind: Namespace
metadata:
name: namespace-green
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: green
namespace: kubernetes-dashboard
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: role-green
namespace: namespace-green
rules:
- apiGroups: [""]
resources: ["*"]
verbs: ["*"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: testDashboard
namespace: kubernetes-dashboard
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: role-green
subjects:
- kind: ServiceAccount
name: green
namespace: kubernetes-dashboard
I retrieved the token with the following command:
kubectl -n kubernetes-dashboard describe secret $(kubectl -n kubernetes-dashboard get secret | grep green | awk '{print $1}')
When I login to the Dashboard I see only the default namespace although I have assigned the new namespace to that role.
I am not able to to figure out how to view the resources of the new namespace only and based on the permissions of the role the service account should have limited access.
You dont need to create a new role.
You can just create a RoleBinding to the 'edit' clusterrole with the new service account you have created and it will work as you expect it to. Also the access will be limited to just one namespace - kubernetes-dashboard
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: testDashboard
namespace: kubernetes-dashboard
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: edit
subjects:
- kind: ServiceAccount
name: green
namespace: kubernetes-dashboard
After that the you can use the same old token to test.
I am trying to access all the namespaces and pods from my another pod. So, I have created clusterrole, clusterrolebinding and service account. I am able access the only customer namespace resources. But I need to access all the namespace resources. Is it possible?
apiVersion: v1
kind: ServiceAccount
metadata:
name: spinupcontainers
namespace: customer
---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: spinupcontainers
namespace: customer
rules:
- apiGroups: [""]
resources: ["pods", "pods/exec"]
verbs: ["get", "list", "delete", "patch", "create"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: spinupcontainers
namespace: customer
subjects:
- kind: ServiceAccount
name: spinupcontainers
roleRef:
kind: ClusterRole
name: spinupcontainers
apiGroup: rbac.authorization.k8s.io
Could anyone help to resolve this problem?
Thanks in advance
It seems in your YAML example you are using a RoleBinding as opposed to a ClusterRoleBinding. A RoleBinding only grants those permissions inside of a namespace. See also the Kubernetes Documentation on this topic:
A RoleBinding grants permissions within a specific namespace whereas a
ClusterRoleBinding grants that access cluster-wide.
Most important thing is that you have to connect your service account to your cluster role with proper cluster role binding. Because binding types decide that scope of service account abilities. Under these circumstances, you have to describe cluster role binding as shown below;
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: spinupcontainers
subjects:
- kind: ServiceAccount
name: spinupcontainers
namespace: customer
roleRef:
kind: ClusterRole
name: spinupcontainers
apiGroup: "rbac.authorization.k8s.io"
If you want to test this within the pod you would describe respective service account for pod like below:
apiVersion: v1
kind: Pod
metadata:
creationTimestamp: null
labels:
run: busybox
name: busybox
spec:
containers:
- args:
- sleep
- "4800"
image: busybox:1.28
name: busybox
resources: {}
dnsPolicy: ClusterFirst
restartPolicy: Never
serviceAccountName: default
status: {}
And then finally you need to ssh to pod and can execute proper curl command with using service account token. Do not forget that you can find the token file in pod by defined service account to pod yaml before (in /var/run/secrets/kubernetes.io/serviceaccount). After that you have to execute API call to use kubernetes API server service (ıf you used kubeadm to create the cluster. It has been already defined in default namespace as named kubernetes). In the below, you can find proper apı call to get default namespace secrets
curl -k -H "Authorization: Bearer $TOKEN" https://<kubernetes-apı-fqdn>/api/v1/namespaces/default/secrets
I'm trying to lock down a namespace in kubernetes using RBAC so I followed this tutorial.
I'm working on a baremetal cluster (no minikube, no cloud provider) and installed kubernetes using Ansible.
I created the folowing namespace :
apiVersion: v1
kind: Namespace
metadata:
name: lockdown
Service account :
apiVersion: v1
kind: ServiceAccount
metadata:
name: sa-lockdown
namespace: lockdown
Role :
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: lockdown
rules:
- apiGroups: [""] # "" indicates the core API group
resources: [""]
verbs: [""]
RoleBinding :
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: rb-lockdown
subjects:
- kind: ServiceAccount
name: sa-lockdown
roleRef:
kind: Role
name: lockdown
apiGroup: rbac.authorization.k8s.io
And finally I tested the authorization using the next command
kubectl auth can-i get pods --namespace lockdown --as system:serviceaccount:lockdown:sa-lockdown
This SHOULD be returning "No" but I got "Yes" :-(
What am I doing wrong ?
Thx
A couple possibilities:
are you running the "can-i" check against the secured port or unsecured port (add --v=6 to see). Requests made against the unsecured (non-https) port are always authorized.
RBAC is additive, so if there is an existing clusterrolebinding or rolebinding granting "get pods" permissions to that service account (or one of the groups system:serviceaccounts:lockdown, system:serviceaccounts, or system:authenticated), then that service account will have that permission. You cannot "ungrant" permissions by binding more restrictive roles
I finally found what was the problem.
The role and rolebinding must be created inside the targeted namespace.
I changed the following role and rolebinding types by specifying the namespace inside the yaml directly.
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: lockdown
namespace: lockdown
rules:
- apiGroups:
- ""
resources:
- pods
verbs:
- get
- watch
- list
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: rb-lockdown
namespace: lockdown
subjects:
- kind: ServiceAccount
name: sa-lockdown
roleRef:
kind: Role
name: lockdown
apiGroup: rbac.authorization.k8s.io
In this example I gave permission to the user sa-lockdown to get, watch and list the pods in the namespace lockdown.
Now if I ask to get the pods : kubectl auth can-i get pods --namespace lockdown --as system:serviceaccount:lockdown:sa-lockdown it will return yes.
On the contrary if ask to get the deployments : kubectl auth can-i get deployments --namespace lockdown --as system:serviceaccount:lockdown:sa-lockdown it will return no.
You can also leave the files like they were in the question and simply create them using kubectl create -f <file> -n lockdown.
I set up a k8s cluster in GKE with rbac enabled, and I install Istio into the cluster.
I follow this step (link) to create key/certs for the Istio ingress controller, and key/certs are stored as secret whose name is istio-ingress-certs.
Now I want to use RBAC to limit access to istio-ingress-certs, so that every component in istio-system is allowed to read the secret, but none could modify or delete it.
I create a secrets-rbac.yaml file, and run kubectl apply -f secrets-rbac.yaml, which creates a role to read the secret, and binds this role to all serviceaccounts in istio-system namespace.
To verify that a serviceaccount is not allowed to modify istio-ingress-certs. I use this command to test.
kubectl auth can-i edit secrets/istio-ingress-certs -n istio-system --as system:serviceaccount:istio-system:istio-pilot-service-account
I expect that the command would return false, but it returns true. I think I didn't set up rbac correctly in the yaml file, but I am not clear which part is not correct.
kind: Role
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
namespace: istio-system
name: istio-ingress-certs-reader
rules:
- apiGroups: ["*"]
resources: ["secrets"]
resourceNames: ["istio-ingress-certs"]
verbs: ["get"]
---
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
namespace: istio-system
name: read-istio-ingress-certs
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: istio-ingress-certs-reader
subjects:
- kind: Group
name: system:serviceaccounts:istio-system
apiGroup: rbac.authorization.k8s.io
- kind: Group
name: system:authenticated
apiGroup: rbac.authorization.k8s.io
- kind: Group
name: system:unauthenticated
apiGroup: rbac.authorization.k8s.io