Kubernetes service account default permissions - kubernetes

I am experimenting with service accounts. I believe the following should produce an access error (but it doesn't):
apiVersion: v1
kind: ServiceAccount
metadata:
name: test-sa
---
apiVersion: v1
kind: Pod
metadata:
name: test-pod
spec:
serviceAccountName: test-sa
containers:
- image: alpine
name: test-container
command: [sh]
args:
- -ec
- |
apk add curl;
KUBE_NAMESPACE="$(cat /var/run/secrets/kubernetes.io/serviceaccount/namespace)";
curl \
--cacert "/var/run/secrets/kubernetes.io/serviceaccount/ca.crt" \
-H "Authorization: Bearer $(cat /var/run/secrets/kubernetes.io/serviceaccount/token)" \
"https://kubernetes.default.svc/api/v1/namespaces/$KUBE_NAMESPACE/services";
while true; do sleep 1; done;
kubectl apply -f test.yml
kubectl logs test-pod
What I see is a successful listing of services, but I would expect a permissions error because I never created any RoleBindings or ClusterRoleBindings for test-sa.
I'm struggling to find ways to list the permissions available to a particular SA, but according to Kubernetes check serviceaccount permissions, it should be possible with:
kubectl auth can-i list services --as=system:serviceaccount:default:test-sa
> yes
Though I'm skeptical whether that command is actually working, because I can replace test-sa with any gibberish and it still says "yes".
According to the documentation, service accounts by default have "discovery permissions given to all authenticated users". It doesn't say what that actually means, but from more reading I found this resource which is probably what it's referring to:
kubectl get clusterroles system:discovery -o yaml
> [...]
> rules:
> - nonResourceURLs:
> - /api
> - /api/*
> [...]
> verbs:
> - get
Which would imply that all service accounts have get permissions on all API endpoints, though the "nonResourceURLs" bit implies this wouldn't apply to APIs for resources like services, even though those APIs live under that path… (???)
If I remove the Authorization header entirely, I see an access error as expected. But I don't understand why it's able to get data using this empty service account. What's my misunderstanding and how can I restrict permissions correctly?

It turns out this is a bug in Docker Desktop for Mac's Kubernetes support.
It automatically adds a ClusterRoleBinding giving cluster-admin to all service accounts (!). It only intends to give this to service accounts inside the kube-system namespace.
It was originally raised in docker/for-mac#3694 but fixed incorrectly. I have raised a new issue docker/for-mac#4774 (the original issue is locked due to age).
A quick fix while waiting for the bug to be resolved is to run:
kubectl apply -f - <<EOF
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: docker-for-desktop-binding
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: cluster-admin
subjects:
- apiGroup: rbac.authorization.k8s.io
kind: Group
name: system:serviceaccounts:kube-system
EOF
I don't know if that might cause issues with future Docker Desktop upgrades but it does the job for now.
With that fixed, the code above correctly gives a 403 error, and would require the following to explicitly grant access to the services resource:
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: service-reader
rules:
- apiGroups: [""]
resources: [services]
verbs: [get, list]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: test-sa-service-reader-binding
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: service-reader
subjects:
- kind: ServiceAccount
name: test-sa
A useful command for investigating is kubectl auth can-i --list --as system:serviceaccount, which shows the rogue permissions were applying to all service accounts:
Resources Non-Resource URLs Resource Names Verbs
*.* [] [] [*]
[*] [] [*]
[...]

The same bug exists in Docker-Desktop for Windows.
It automatically adds a ClusterRoleBinding giving cluster-admin to all service accounts (!). It only intends to give this to service accounts inside the kube-system namespace.

This is because in Docker Desktop by default a clusterrolebinding docker-for-desktop-binding gives cluster-admin role to all the service accounts created.
For more details check the issue here

Related

Kubernetes kube-apiserver to kubelet permissions

What controls the permissions when you call
kubectl logs pod-name ?
I've played around and tried calling the kubelet api from on of the controller nodes.
sudo curl -k --key /var/lib/kubernetes/cert-k8s-apiserver-key.pem --cert /var/lib/kubernetes/cert-k8s-apiserver.pem https://worker01:10250/pods
This fails with Forbidden (user=apiserver, verb=get, resource=nodes, subresource=proxy).
I've tried the same call using the admin key and cert and it succeeds and return a healthy blob of JOSN.
I'm guessing this is why kubectl logs pod-name doesn't work.
A little more reading suggests that the CN of the certificate determines the user that is authenticated and authorized.
What controls whether a user is authorized to access the kubelet API?
Background
I'm setting up a K8s cluster using the following instructions Kubernetes the not so hard way with Ansible
Short Answer
The short answer is that you need to grant the user apiserver access to the resource node by creating a ClusterRole and ClusterRoleBinding.
Longer Explanation
Kubernetes has a bunch of resources. The relevant ones here are:
Role
Node
ClusterRole
ClusterRoleBinding
Roles and ClusterRoles are similar, except ClusterRoles are not namespaced.
A ClsuterRole can be associated (bound) to a user with a ClusterRoleBinding object.
Kubelet provides some resources (maybe more)
nodes/proxy
nodes/stats
nodes/log
nodes/spec
nodes/metrics
To make this work, you need to create a ClusterRole that allow access to the resource and sub-resource on the Node.
cat <<EOF | kubectl apply -f -
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
annotations:
rbac.authorization.kubernetes.io/autoupdate: "true"
labels:
kubernetes.io/bootstrapping: rbac-defaults
name: system:kube-apiserver-to-kubelet
rules:
- apiGroups:
- ""
resources:
- nodes/proxy
- nodes/stats
- nodes/log
- nodes/spec
- nodes/metrics
verbs:
- "*"
EOF
Then you associate this ClusteRole with a user. In my case, the kube-apiserver is using a certificate with CN=apiserver.
cat <<EOF | kubectl apply --kubeconfig admin.kubeconfig -f -
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: system:kube-apiserver
namespace: ""
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: system:kube-apiserver-to-kubelet
subjects:
- apiGroup: rbac.authorization.k8s.io
kind: User
name: apiserver
EOF

Which rights are missing? Unable to continue with install: could not get information about the resource: podsecuritypolicies.policy

I am trying to install loki with helm
$ helm upgrade --install loki grafana/loki-stack
I got the following error msg:
Release "loki" does not exist. Installing it now.
Error: rendered manifests contain a resource that already exists. Unable to continue with install: could not get information about the resource: podsecuritypolicies.policy "loki" is forbidden: User "secret user :)" cannot get resource "podsecuritypolicies" in API group "policy" at the cluster scope
$ helm list -all
NAME NAMESPACE REVISION UPDATED STATUS CHART APP VERSION
I'm a simple user, but I can make deployment/pods via yaml files manual.
I need to use helm charts.
It seems that your User has insufficient privileges to create policies. You need to ask your cluster administrator for more privileges, unless you can assign them yourself to this user. I'm providing example yaml below to achieve that. First, create ClusterRole with proper privileges:
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: <role name>
rules:
- apiGroups: ['policy']
resources: ['podsecuritypolicies']
verbs: ['get']
Then, you need to bind this ClusterRole to user(s):
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: <binding name>
roleRef:
kind: ClusterRole
name: <role name>
apiGroup: rbac.authorization.k8s.io
subjects:
# Authorize all service accounts in a namespace (recommended):
- kind: Group
apiGroup: rbac.authorization.k8s.io
name: system:serviceaccounts:<authorized namespace>
# Authorize specific service accounts (not recommended):
- kind: ServiceAccount
name: <authorized service account name>
namespace: <authorized pod namespace>
# Authorize specific users (not recommended):
- kind: User
apiGroup: rbac.authorization.k8s.io
name: <authorized user name>
Go here for more detailed explanation.

k3s: permissions necessary to access metrics server?

I'd like to grant a service account the ability to access the metrics exposed by the metrics-server service (https://metrics-server.kube-system/metrics). If I create a serviceaccount...
apiVersion: v1
kind: ServiceAccount
metadata:
name: metrics-reader
namespace: prometheus
...and then grant it cluster-admin privileges...
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: metrics-reader-crb
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: cluster-admin
subjects:
- kind: ServiceAccount
name: metrics-reader
namespace: prometheus
...it works! I can use the account token to access the metrics server:
curl -k --header "Authorization: Bearer $token" https://metrics-server.kube-system/metrics
But I don't want to require cluster-admin access just to read
metrics. I tried to use the view cluster role instead of
cluster-admin, but that fails.
Is there an existing role that would grant the appropriate access?
If not, what are the specific permissions necessary to grant read-only
access to the metrics-server /metrics endpoint?
Interesting question. I've found some info for you, however i'm not sure that 100% helpful. It needs more research and reproduce.
check RBAC Deny when requesting metrics. Smth like below?
apiVersion: v1
kind: ServiceAccount
metadata:
name: metrics-reader
namespace: prometheus
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: view-metrics
rules:
- apiGroups:
- metrics.k8s.io
resources:
- pods
- nodes
verbs:
- get
- list
- watch
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: view-metrics
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: view-metrics
subjects:
- kind: ServiceAccount
name: metrics-reader
namespace: prometheus
It seems, there is a aggregated-metrics-reader clusterrole (or there was)
Aggregated ClusterRoles are documented in:
https://kubernetes.io/docs/reference/access-authn-authz/rbac/#aggregated-clusterroles.
The purpose of the system:aggregated-metrics-reader ClusterRole, is to
aggregate the rules, that grant permission to get the pod and node
metrics, to the view, edit and admin roles.
however I wasnt able to find any reference to aggregated-metrics-reader clusterrole in current version of that doc.
You can find huge example of using this clusterrole in Metrics server unable to scrape
IN addition check This adds the aggregated-metrics-reader ClusterRole which was missing github PR:
What this PR does / why we need it: This adds the
aggregated-metrics-reader ClusterRole which was missing, and seems to
be required for k8s 1.8+ per the metrics-server documentation and
default deploy manfiests
Unfortunately link in that PR direct to nowhere. I start thinking this obsolete info for 1.8 clusters.. Will update answer in case find anything more relevant

Attach IAM Role to Serviceaccount from a Pod in EKS

I am trying to attach an IAM role to a pod's service account from within the POD in EKS.
kubectl annotate serviceaccount -n $namespace $serviceaccount eks.amazonaws.com/role-arn=$ARN
The current role attached to the $serviceaccountis outlined below:
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: common-role
rules:
- apiGroups: [""]
resources:
- event
- secrets
- configmaps
- serviceaccounts
verbs:
- get
- create
However, when I execute the kubectl command I get the following:
error from server (forbidden): serviceaccounts $serviceaccount is forbidden: user "system:servi...." cannot get resource "serviceaccounts" in API group "" ...
Is my role correct? Why can't I modify the service account?
Kubernetes by default will run the pods with service account: default which don`t have the right permissions. Since I cannot determine which one you are using for your pod I can only assume that you are using either default or some other created by you. In both cases the error suggest that the service account your are using to run your pod does not have proper rights.
If you run this pod with service account type default you will have add the appropriate rights to it. Alternative way is to run your pod with another service account created for this purpose. Here`s an example:
apiVersion: v1
kind: ServiceAccount
metadata:
name: run-kubectl-from-pod
Then you will have to create appropriate role (you can find full list of verbs here):
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: modify-service-accounts
rules:
- apiGroups: [""]
resources:
- serviceaccounts
verbs:
- get
- create
- patch
- list
I'm using here more verbs as a test. Get and Patch would be enough for this use case. I`m mentioning this since its best practice to provide as minimum rights as possible.
Then create your role accordingly:
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: modify-service-account-bind
subjects:
- kind: ServiceAccount
name: run-kubectl-from-pod
roleRef:
kind: Role
name: modify-service-accounts
apiGroup: rbac.authorization.k8s.io
And now you just have reference that service account when your run your pod:
apiVersion: v1
kind: Pod
metadata:
name: run-kubectl-in-pod
spec:
serviceAccountName: run-kubectl-from-pod
containers:
- name: kubectl-in-pod
image: bitnami/kubectl
command:
- sleep
- "3600"
Once that is done, you just exec into the pod:
➜ kubectl-pod kubectl exec -ti run-kubectl-in-pod sh
And then annotate the service account:
$ kubectl get sa
NAME SECRETS AGE
default 1 19m
eks-sa 1 36s
run-kubectl-from-pod 1 17m
$ kubectl annotate serviceaccount eks-sa eks.amazonaws.com/role-arn=$ARN
serviceaccount/eks-sa annotated
$ kubectl describe sa eks-sa
Name: eks-sa
Namespace: default
Labels: <none>
Annotations: eks.amazonaws.com/role-arn:
Image pull secrets: <none>
Mountable secrets: eks-sa-token-sldnn
Tokens: <none>
Events: <none>
If you encounter any issues with request being refused please start with reviewing your request attributes and determine the appropriate request verb.
You can also check your access with kubectl auth can-i command:
kubectl-pod kubectl auth can-i patch serviceaccount
API server will respond with simple yes or no.
Please Note that If you want to patch a service account to use an IAM role you will have delete and re-create any existing pods that are assocaited with the service account to apply credentials environment variables. You can read more about it here.
While your role appears to be correct, please keep in mind that when executing kubectl, the RBAC permissions of your account in kubeconfig are relevant for whether you are allowed to perform an action.
From your question, I understand that your role is attached to the service account you are trying to annotate, which is irrelevant to the kubectl permission check.

RBAC not working as expected when trying to lock namespace

I'm trying to lock down a namespace in kubernetes using RBAC so I followed this tutorial.
I'm working on a baremetal cluster (no minikube, no cloud provider) and installed kubernetes using Ansible.
I created the folowing namespace :
apiVersion: v1
kind: Namespace
metadata:
name: lockdown
Service account :
apiVersion: v1
kind: ServiceAccount
metadata:
name: sa-lockdown
namespace: lockdown
Role :
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: lockdown
rules:
- apiGroups: [""] # "" indicates the core API group
resources: [""]
verbs: [""]
RoleBinding :
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: rb-lockdown
subjects:
- kind: ServiceAccount
name: sa-lockdown
roleRef:
kind: Role
name: lockdown
apiGroup: rbac.authorization.k8s.io
And finally I tested the authorization using the next command
kubectl auth can-i get pods --namespace lockdown --as system:serviceaccount:lockdown:sa-lockdown
This SHOULD be returning "No" but I got "Yes" :-(
What am I doing wrong ?
Thx
A couple possibilities:
are you running the "can-i" check against the secured port or unsecured port (add --v=6 to see). Requests made against the unsecured (non-https) port are always authorized.
RBAC is additive, so if there is an existing clusterrolebinding or rolebinding granting "get pods" permissions to that service account (or one of the groups system:serviceaccounts:lockdown, system:serviceaccounts, or system:authenticated), then that service account will have that permission. You cannot "ungrant" permissions by binding more restrictive roles
I finally found what was the problem.
The role and rolebinding must be created inside the targeted namespace.
I changed the following role and rolebinding types by specifying the namespace inside the yaml directly.
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: lockdown
namespace: lockdown
rules:
- apiGroups:
- ""
resources:
- pods
verbs:
- get
- watch
- list
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: rb-lockdown
namespace: lockdown
subjects:
- kind: ServiceAccount
name: sa-lockdown
roleRef:
kind: Role
name: lockdown
apiGroup: rbac.authorization.k8s.io
In this example I gave permission to the user sa-lockdown to get, watch and list the pods in the namespace lockdown.
Now if I ask to get the pods : kubectl auth can-i get pods --namespace lockdown --as system:serviceaccount:lockdown:sa-lockdown it will return yes.
On the contrary if ask to get the deployments : kubectl auth can-i get deployments --namespace lockdown --as system:serviceaccount:lockdown:sa-lockdown it will return no.
You can also leave the files like they were in the question and simply create them using kubectl create -f <file> -n lockdown.