I still wondering how supposed to do to change the service account user. Let say I have 2 service account (A and B), which each user has different role in different project. After done being use user B, when I want to change to service account A and access the resource, gcloud command says
Error from server (Forbidden): pods is forbidden: User "user-B#project.iam.gserviceaccount.com" cannot list resource "pods" in API group "" in the namespace "default": requires one of ["container.pods.list"] permission(s).
I'm done change my service account user with gcloud config set account [service-account], but the gcloud still read another service account. Did I missed something?
Here's a contrived example of what I think you're doing:
# gcloud is using my regular User credentials
gcloud config get account
me#gmail.com
# Access GKE as me#gmail.com
kubectl get pods --namespace=default
pod/foo-c7b7995df-vxrmh
# Authenticate as a GCP Service Account with **no** permissions
EMAIL="{ACCOUNT}#{PROJECT}.iam.gserviceaccount.com"
gcloud auth activate-service-account ${EMAIL} \
--key-file=${KEY_FILE}
# gcloud is now using the Service Account credentials
gcloud config get account
${EMAIL}
# Using new GKE auth plugin
gke-gcloud-auth-plugin \
| jq -r .status.expirationTimestamp
2022-00-00T17:10:00Z
# Need to either delete the token
# Or wait until 17:10 for it to expire
# Then...
kubectl get pods --namespace=default
Error from server (Forbidden): pods is forbidden: ...
ERROR Error from server (Forbidden): pods is forbidden: User "{ACCOUNT}#{PROJECT}.iam.gserviceaccount.com" cannot list resource "pods" in API group "" in the namespace "default": requires one of ["container.pods.list"] permission(s).
One solution is to grant the GCP (!) Service Account one of the Kubernetes Engine roles that has permission to list Pods, i.e. container.pods.* which is part of roles/container.developer:
# Grant the Service Account Kubernetes Engine role
ROLE="roles/container.developer"
gcloud projects add-iam-policy-binding ${PROJECT} \
--member=serviceAccount:${EMAIL} \
--role=${ROLE}
# Try again
kubectl get pods --namespace=default --output=name
pod/foo-c7b7995df-vxrmh
Related
Env: AKS,
k8s version: 1.21.2
Enabled AKS managed AAD in cluster, and I have admin permission
When I run
kubectl auth can-i get pods -n dev
it returns yes, which means I have permission
But when I try to impersonate myself or a teammate
kubectl auth can-i get pods -n dev --as myname#mydomain.com
It returns
no - Azure does not have opinion for this non AAD user.
If you are an AAD user, please set Extra:oid parameter for impersonated user in the kubeconfig
I can't see a reason why user impersonation not work
I have an aks cluster running kubernetes 1.21.2 with those options :
Kubernetes RBAC --> enable
AKS-managed AAD --> enable
Local accounts --> disabled
When I run the az aks get-credentials --resource-group <resource-group> --name <cluster-name> --admin it fails as expected. So far it's looking good.
I still want someone to be admin, so I give a user following privileges :
Azure Kubernetes Service Cluster User Role
Azure Kubernetes Service RBAC Cluster Admin
I followed the following steps to try out my user is a cluster admin :
az login and input my user's credentials to login
az aks get-credentials --resource-group <resource-group> --name <cluster-name> to download the clusterUser_kubeconfig
kubectl get pods -A to list pods in all namespaces
it prompts me something like To sign in, use a web browser to open the page https://microsoft.com/devicelogin and enter the code DKXXXXX8T to authenticate.
I login using my user's credential again and allow the k8s application to connect to my user.
kubectl get pods -A fails :
Error from server (Forbidden): pods is forbidden: User "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx" cannot list resource "pods" in API group "" at the cluster scope
According to what I have read so far, it should be working as my user is Azure Kubernetes Service RBAC Cluster Admin. Could someone enlighten me what I've missed or misunderstood ?
My Objective
I want to use GCP impersonation to fetch my GKE cluster credentials. And then I want to run kubectl commands.
Initial Context
I have a GCP project called rakib-example-project
I have 2 ServiceAccounts in the project called:
owner#rakib-example-project.iam.gserviceaccount.com
it has the project-wide roles/owner role - so it can do anything and everything inside the GCP project
executor#rakib-example-project.iam.gserviceaccount.com
it only has the project-wide roles/iam.serviceAccountTokenCreator role - so it can impersonate the owner ServiceAccount in the GCP project
I have 1 GKE cluster in the project called my-gke-cluster
The Problem
✅ I have authenticated as the executor ServiceAccount:
$ gcloud auth activate-service-account --key-file=my_executor_sa_key.json
Activated service account credentials for: [executor#rakib-example-project.iam.gserviceaccount.com]
✅ I have fetched GKE cluster credentials by impersonating the owner:
$ gcloud container clusters get-credentials my-gke-cluster \
--zone asia-southeast1-a \
--project rakib-example-project \
--impersonate-service-account=owner#rakib-example-project.iam.gserviceaccount.com
WARNING: This command is using service account impersonation. All API calls will be executed as [owner#rakib-example-project.iam.gserviceaccount.com].
WARNING: This command is using service account impersonation. All API calls will be executed as [owner#rakib-example-project.iam.gserviceaccount.com].
Fetching cluster endpoint and auth data.
kubeconfig entry generated for my-gke-cluster.
❌ I am failing to list cluster nodes due to missing container.nodes.list permission:
$ kubectl get nodes
Error from server (Forbidden): nodes is forbidden: User "executor#rakib-example-project.iam.gserviceaccount.com" cannot list resource "nodes" in API group "" at the cluster scope: requires one of ["container.nodes.list"] permission(s).
But I have already impersonated the Owner ServiceAccount. Why would it still have missing permissions? 😧😧😧
My Limitations
It works well if i grant my executor ServiceAccount the roles/container.admin role. However, I am not allowed to grant such roles to my executor ServiceAccount due to compliance requirements. I can only impersonate the owner ServiceAccount and THEN do whatever I want through it - not directly.
If you have a look to your kubeconfig file at this location ~/.kube/config, you can see the list of authorization and the secrets, such as
- name: gke_gdglyon-cloudrun_us-central1-c_knative
user:
auth-provider:
config:
access-token: ya29.<secret>-9XQmaTQodj4kS39w
cmd-args: config config-helper --format=json
cmd-path: /usr/lib/google-cloud-sdk/bin/gcloud
expiry: "2020-08-25T17:48:39Z"
expiry-key: '{.credential.token_expiry}'
token-key: '{.credential.access_token}'
name: gcp
You see external references (expiry-key and token-key) and a cmd-path. The command path is interesting because when a new token need to be generated, it will be called.
However, you see any mention of the impersonation. You have to add it in the command path, to be used by default. For this, add it in your config like this:
gcloud config set auth/impersonate_service_account owner#rakib-example-project.iam.gserviceaccount.com
Now, every use of the gcloud CLI will use the impersonate service account, and it's what you want to generate a valid access_token to reach your GKE cluster
I changed something and deployed on a new cluster then I got this error even though I didn't change anything in the code. Has anybody seen it before?
from server for:
"/builds/dropcunt/nettmoster.com/deployment/webapp.yml": ingresses.extensions "nettmoster.comn-273414" is forbidden: User "system:serviceaccount:gitlab-managed-apps:default" cannot get resource "ingresses" in API group "extensions" in the namespace "nettmoster-com-9777808"
59 Error from server (Forbidden): error when retrieving current configuration of:
As suggested I runkubectl auth can-i --list --as=system:serviceaccount:gitlab-managed-apps:default
It returns:
This is a RBAC problem. The service account system:serviceaccount:gitlab-managed-apps:default does not have permission to get ingress resource in the new cluster.
You can compare what permission a service account have by running below command in both the cluster
kubectl auth can-i --list --as=system:serviceaccount:gitlab-managed-apps:default
Run below commands to get permission via RBAC
kubectl create role ingress-reader --verb=get,list,watch,update --resource=ingress
kubectl create rolebinding ingress-reader-role --role=ingress-reader --serviceaccount=gitlab-managed-apps:default
I am getting an error message after running some kubectl commands (GCP command line - gcloud). I have a K8S cluster created in GKE.
Example:
kubectl describe node
gke_k8s_cluster_name
Error from server (Forbidden): leases.coordination.k8s.io "gke_k8s_cluster_name" is forbidden: User "MY_SERVICE_ACCOUNT" cannot get resource "leases" in API group "coordination.k8s.io" in the namespace "kube-node-lease": Required "container.leases.get" permission.
The point is that "container.leases.get" permission is not listed in IAM (as custom permissions or regular role).
How could I grant that permission to the service account in GCP ?
thanks,
Jose
You may need to grant additional permissions to yourself on GCP IAM and GKE sides, for example:
PROJECT_ID=$(gcloud config get-value core/project)
USER_ID=$(gcloud config get-value core/account)
gcloud projects add-iam-policy-binding ${PROJECT_ID} --member=user:${USER_ID} --role=roles/container.admin
kubectl create clusterrolebinding cluster-admin-binding --clusterrole cluster-admin --user ${USER_ID}
See also GCP IAM & GKE RBAC integration.