I'm trying to understand why on one cluster an operation is permitted but on the other i'm getting the following
Exception encountered setting up namespace watch from Kubernetes API v1 endpoint https://10.100.0.1:443/api: namespaces is forbidden: User \"system:serviceaccount:kube-system:default\" cannot list resource \"namespaces\" in API group \"\" at the cluster scope ({\"kind\":\"Status\",\"apiVersion\":\"v1\",\"metadata\":{},\"status\":\"Failure\",\"message\":\"namespaces is forbidden: User \\\"system:serviceaccount:kube-system:default\\\" cannot list resource \\\"namespaces\\\" in API group \\\"\\\" at the cluster scope\",\"reason\":\"Forbidden\",\"details\":{\"kind\":\"namespaces\"},\"code\":403}\n)"
I'm managing two Kubernetes clusters -
clusterA booted with Kops version v1.14.8
clusterB booted on AWS EKS version v1.14.9-eks-f459c0
So i've tried using the kubectl auth command to try figuring out and I do see that on one i'm allowed however on the second i'm not as you can see:
kubectl config use-context clusterA
Switched to context "clusterA".
kubectl auth can-i list pods --as=system:serviceaccount:kube-system:default -n kube-system
yes
kubectl config use-context clusterB
Switched to context "clusterB".
kubectl auth can-i list pods --as=system:serviceaccount:kube-system:default -n kube-system
no
Is there a way to understand what are these two decisions based on yes/no?
Thanks for helping out!
The decision yes/no is based on whether there is a clusterrole and a clusterrolebinding or rolebinding which permits the default serviceaccount in kube-system namespace to perform verb list on resource namespace.
The trick in case of namespace resource is that there needs to be a clusterrole instead of role because namespace is a cluster scoped resource.
You check what are the clusterrole,role, clusterrolebinding,rolebinding exists in a kubernetes cluster using below command
kubectl get clusterrole,clusterrolebinding
kubectl get role,rolebinding -n namespacename
For more details refer Kubernetes RBAC here
Related
I am following a tutorial regarding RBAC, I think I understand the main idea but I don't get why this is failing:
kc auth can-i "*" pod/compute --as deploy#test.com
no
kc create clusterrole deploy --verb="*" --resource=pods --resource-name=compute
clusterrole.rbac.authorization.k8s.io/deploy created
kc create clusterrolebinding deploy --user=deploy#test.com --clusterrole=deploy
clusterrolebinding.rbac.authorization.k8s.io/deploy created
# this tells me that deploy#test.com should be able to create a pod named compute
kc auth can-i "*" pod/compute --as deploy#test.com
yes
# but it fails when trying to do so
kc run compute --image=nginx --as deploy#test.com
Error from server (Forbidden): pods is forbidden: User "deploy#test.com" cannot create resource "pods" in API group "" in the namespace "default"
the namespace name should be irrelevant afaik, since this is a clusterrole.
Restricting the create permission to a specific resource name is not supported.
This is from the Kubernetes documentation:
Note: You cannot restrict create or deletecollection requests by resourceName. For create, this limitation is because the object name is not known at authorization time.
This means the ClusterRole you created doesn't allow you to create any Pod.
You need to have another ClusterRole assigned where you don't specify the resource name.
How can I debug all Rules with their Verbs and ApiGroups for a kubernetes ServiceAccount?
kubectl supports testing the RBAC rights for a particular Service Account with the auth can-i sub-command. The syntax looks as follows:
kubectl auth can-i -n --as system:serviceaccount:<namespace>:<serviceaccount> <verb> <resource>
For example:
kubectl auth can-i --as system:serviceaccount:default:my-serviceaccount create deployments
More information can be found here.
k9s offer some nice views for this task:
1. Find Role Bindings
First you have to find the ClusterRoleBindings or RoleBindings for a ServiceAccount:
Type : and clusterrolebindings or rolebindings
Search for your ServiceAccount by / and name of ServiceAccount (e.g. monitor-kube-prometheus-st-operator)
Now k9s lists all (Cluster)RoleBindings
2. Display Role Bindings
Open (Cluster)RoleBindings
All given and forbidden rules are displayed:
I'm reading myself currently into RBAC and am using Docker For Desktop with a local Kubernetes cluster enabled.
If I run kubectl auth can-i get pods which user or group or serviceaccount is used by default?
Is it the same call like:
kubectl auth can-i get pods --as docker-for-desktop --as-group system:serviceaccounts ?
kubectl config view shows:
contexts:
- context:
cluster: docker-for-desktop-cluster
namespace: default
user: docker-for-desktop
name: docker-for-desktop
...
users:
- name: docker-for-desktop
user:
client-certificate-data: REDACTED
client-key-data: REDACTED
But simply calling kubectl auth can-i get pods --as docker-for-desktop returns NO.
Thanks,
Kim
To answer your question
If I run kubectl auth can-i get pods which user or group or serviceaccount is used by default?
As you can read on Configure Service Accounts for Pods:
When you (a human) access the cluster (for example, using kubectl), you are authenticated by the apiserver as a particular User Account (currently this is usually admin, unless your cluster administrator has customized your cluster). Processes in containers inside pods can also contact the apiserver. When they do, they are authenticated as a particular Service Account (for example, default).
You can use kubectl get serviceaccount to see what serviceaccounts are setup in the cluster.
Try checking which contexts you have available and switching into a which ever you need:
kubectl config get-contexts
kubectl config use-context docker-for-desktop
If you are experiencing an issue with missing Role please check Referring to Resources to set they correctly for docker-for-desktop
I am trying to access my Kubernetes cluster on google cloud with the service account, but I am not able to make this works. I have a running system with some pods and ingress. I want to be able to update images of deployments.
I would like to use something like this (remotely):
kubectl config set-cluster cluster --server="<IP>" --insecure-skip-tls-verify=true
kubectl config set-credentials foo --token="<TOKEN>"
kubectl config set-context my-context --cluster=cluster --user=foo --namespace=default
kubectl config use-context cluster
kubectl set image deployment/my-deployment boo=eu.gcr.io/project-123456/image:v1
So I created the service account and then get the secret token:
kubectl create serviceaccount foo
kubectl get secret foo-token-gqvgn -o yaml
But, when I try to update the image in any deployment, I receive:
error: You must be logged in to the server (Unauthorized)
IP address for API I use the address, which is shown in GKE administration as cluster endpoint IP.
Any suggestions? Thanks.
I have tried to recreate your problem.
Steps I have followed
kubectl create serviceaccount foo
kubectl get secret foo-token-* -o yaml
Then, I have tried to do what you have done
What I have used as token is base64 decoded Token.
Then I tried this:
$ kubectl get pods
Error from server (Forbidden): pods is forbidden: User "system:serviceaccount:default:foo" cannot list pods in the namespace "default": Unknown user "system:serviceaccount:default:foo"
This gave me error as expected. Because, I need to grant permission to this ServiceAccount.
How can I grant permission to this ServiceAccount? I need to create ClusterRole & ClusterRoleBinding with necessary permission.
Read more to learn more role-based-access-control
I can do another thing
$ kubectl config set-credentials foo --username="admin" --password="$PASSWORD"
This will grant you admin authorization.
You need to provide cluster credential.
Username: admin
Password: -----
You will get this info in GKE -> Kubernetes Engine -> {cluster} -> Show credential
I'm trying to deploy an application in a GKE 1.6.2 cluster running ContainerOS but the instructions on the website / k8s are not accurate anymore.
The error that I'm getting is:
Error from server (Forbidden): User "circleci#gophers-slack-bot.iam.gserviceaccount.com"
cannot get deployments.extensions in the namespace "gopher-slack-bot".:
"No policy matched.\nRequired \"container.deployments.get\" permission."
(get deployments.extensions gopher-slack-bot)
The repository for the application is available here available here.
Thank you.
I had a few breaking changes in the past with using the gcloud tool to authenticate kubectl to a cluster, so I ended up figuring out how to auth kubectl to a specific namespace independent of GKE. Here's what works for me:
On CircleCI:
setup_kubectl() {
echo "$KUBE_CA_PEM" | base64 --decode > kube_ca.pem
kubectl config set-cluster default-cluster --server=$KUBE_URL --certificate-authority="$(pwd)/kube_ca.pem"
kubectl config set-credentials default-admin --token=$KUBE_TOKEN
kubectl config set-context default-system --cluster=default-cluster --user=default-admin --namespace default
kubectl config use-context default-system
}
And here's how I get each of those env vars from kubectl.
kubectl get serviceaccounts $namespace -o json
The service account will contain the name of it's secret. In my case, with the default namespace, it's
"secrets": [
{
"name": "default-token-655ls"
}
]
Using the name, I get the contents of the secret
kubectl get secrets $secret_name -o json
The secret will contain ca.crt and token fields, which match the $KUBE_CA_PEM and $KUBE_TOKEN in the shell script above.
Finally, use kubectl cluster-info to get the $KUBE_URL value.
Once you run setup_kubectl on CI, your kubectl utility will be authenticated to the namespace you're deploying to.
In Kubernetes 1.6 and GKE, we introduce role based cess control. The authors of your took need to give the service account the ability to get deployments (along with probably quite a few others) to its account creation.
https://kubernetes.io/docs/admin/authorization/rbac/